Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Enhancements to malloc.html documentation to address concerns arising out of heartbleed. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
2bc13084bc1c98f1d736c707158757c7 |
User & Date: | drh 2014-05-30 17:01:05.004 |
Context
2014-05-30
| ||
18:01 | Add a section describing the Win32 native memory allocator. (check-in: ef8f1ed845 user: mistachkin tags: trunk) | |
17:01 | Enhancements to malloc.html documentation to address concerns arising out of heartbleed. (check-in: 2bc13084bc user: drh tags: trunk) | |
2014-05-29
| ||
15:46 | Change the default maximum POST size in althttpd to 20MB. (check-in: 35fcd6cd5b user: drh tags: trunk) | |
Changes
Changes to pages/malloc.in.
1 2 3 4 5 6 7 8 9 10 | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | - + | <title>Dynamic Memory Allocation In SQLite</title> <h1>Dynamic Memory Allocation In SQLite</h1> <tcl>hd_keywords {memory allocation}</tcl> <p>SQLite uses dynamic memory allocation to obtain memory for storing various objects (ex: [database connections] and [prepared statements]) and to build a memory cache of the database file and to hold the results of queries. Much effort has gone into making the dynamic memory allocation subsystem |
︙ | |||
55 56 57 58 59 60 61 | 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | - - + + | The [sqlite3_soft_heap_limit64()] mechanism allows the application to set a memory usage limit that SQLite strives to stay below. SQLite will attempt to reuse memory from its caches rather than allocating new memory as it approaches the soft limit. </p></li> <li><p> |
︙ | |||
84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | + + + + + + + + - - + | <li><p> <b>Memory usage statistics.</b> Applications can see how much memory they are using and detect when memory usage is approaching or exceeding design boundaries. </p></li> <a name="pwwo"></a> <li><p> <b>Plays well with memory debuggers.</b> Memory allocation in SQLite is structured so that standard third-party memory debuggers (such as [http://dmalloc.com | dmalloc] or [http://valgrind.org | valgrind]) can be used to verify correct memory allocation behavior.</p> <li><p> <b>Minimal calls to the allocator.</b> The system malloc() and free() implementations are inefficient on many systems. SQLite strives to reduce overall processing time by minimizing its use of malloc() and free(). </p></li> <li><p> <b>Open access.</b> Pluggable SQLite extensions or even the application itself can access to the same underlying memory allocation routines used by SQLite through the [sqlite3_malloc()], [sqlite3_realloc()], and [sqlite3_free()] interfaces. </p></li> </ul> <a name="testing"></a> <h2>2.0 Testing</h2> |
︙ | |||
188 189 190 191 192 193 194 195 196 197 198 199 200 201 | 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 | + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + + | memory leak detector both work over the entire SQLite test suite and the [TCL test suite] provides over 99% statement test coverage and that the [TH3] test harness provides [test coverage | 100% branch test coverage] with no leak leaks. This is strong evidence that dynamic memory allocation is used correctly everywhere within SQLite.</p> <a name="allocarray"></a> <h3>2.1 Use of reallocarray()</h3> <p>The reallocarray() interface is a recent innovation (circa 2014) from the OpenBSD community that grow out of efforts to prevent the next [http://heartbleed.com | "heartbleed" bug] by avoiding 32-bit integer arithmetic overflow on memory allocation size computations. The reallocarray() function has both unit-size and count parameters. To allocate memory sufficient to hold an array of N elements each X-bytes in size, one calls "reallocarray(0,X,N)". This is preferred over the traditional technique of invoking "malloc(X*N)" as reallocarray() eliminates the risk that the X*N multiplication will overflow and cause malloc() to return a buffer that is a different size from what the application expected.</p> <p>SQLite does not use reallocarray(). The reason is that reallocarray() is not useful to SQLite. It turns out that SQLite never does memory allocations that are the simple product of two integers. Instead, SQLite does allocations of the form "X+C" or "N*X+C" or "M*N*X+C" or "N*X+M*Y+C", and so forth. The reallocarray() interface is not helpful in avoiding integer overflow in those cases.</p> <p>Nevertheless, integer overflow in the computation of memory allocation sizes is a concern that SQLite would like to deal with. To prevent problems, all SQLite internal memory allocations occur using thin wrapper functions that take a signed 64-bit integer size parameter. The SQLite source code is audited to ensure that all size computations are carried out using 64-bit signed integers as well. SQLite will refuse to allocate more than about 2GB of memory at one go. (In common use, SQLite seldom ever allocates more than about 8KB of memory at a time so a 2GB allocation limit is not a burden.) So the 64-bit size parameter provides lots of headroom for detecting overflows. The same audit that verifies that all size computations are done as 64-bit signed integers also verifies that it is impossible to overflow a 64-bit integer during the computation.</p> <p>The code audits used to ensure that memory allocation size computations do not overflow in SQLite are repeated prior to every SQLite release.</p> <a name="config"></a> <h2>3.0 Configuration</h2> <p>The default memory allocation settings in SQLite are appropriate for most applications. However, applications with unusual or particularly strict requirements may want to adjust the configuration to more closely align SQLite to their needs. |
︙ | |||
936 937 938 939 940 941 942 | 982 983 984 985 986 987 988 989 990 991 | - - - - - - - - - - - - - - - - - + + + - - - - - | The [memory statistics] interfaces of SQLite provide the application with all the mechanism necessary to complete the monitoring portion of this task.</p> <a name="stability"></a> <h2>5.0 Stability Of Memory Interfaces</h2> |