Documentation Source Text

Check-in [ef8f1ed845]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add a section describing the Win32 native memory allocator.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: ef8f1ed84593558fdca2fa432f12b6f4c12ae7df
User & Date: mistachkin 2014-05-30 18:01:46.901
Context
2014-05-30
18:14
Document the SQLITE_WIN32_HEAP_CREATE and SQLITE_WIN32_MALLOC_VALIDATE compile-time options. (check-in: 9816bbf88c user: mistachkin tags: trunk)
18:01
Add a section describing the Win32 native memory allocator. (check-in: ef8f1ed845 user: mistachkin tags: trunk)
17:01
Enhancements to malloc.html documentation to address concerns arising out of heartbleed. (check-in: 2bc13084bc user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/malloc.in.
297
298
299
300
301
302
303
304












305
306
307
308
309
310
311
312
313
of newly allocated memory and that memory allocations are not used after
they have been freed.</p>

<p>The heavy wrapper employed by [SQLITE_MEMDEBUG] is intended for use
only during testing, analysis, and debugging of SQLite.  The heavy wrapper
has a significant performance and memory overhead and probably should not
be used in production.</p>













<tcl>hd_fragment memsys5 *memsys5 {zero-malloc memory allocator}</tcl>
<h4>3.1.3 Zero-malloc memory allocator</h4>

<p>When SQLite is compiled with the [SQLITE_ENABLE_MEMSYS5] option, an
alternative memory allocator that does not use malloc() is included in the
build.  The SQLite developers refer to this alternative memory allocator
as "memsys5".  Even when it is included in the build, memsys5 is 
disabled by default.
To enable memsys5, the application must invoke the following SQLite 








>
>
>
>
>
>
>
>
>
>
>
>

|







297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
of newly allocated memory and that memory allocations are not used after
they have been freed.</p>

<p>The heavy wrapper employed by [SQLITE_MEMDEBUG] is intended for use
only during testing, analysis, and debugging of SQLite.  The heavy wrapper
has a significant performance and memory overhead and probably should not
be used in production.</p>

<tcl>hd_fragment win32heap {Win32 native memory allocator}</tcl>
<h4>3.1.3 The Win32 native memory allocator</h4>

<p>If SQLite is compiled for Windows with the [SQLITE_WIN32_MALLOC]
compile-time option, then a different, thin wrapper is used around
HeapAlloc(), HeapReAlloc(), and HeapFree().  The thin wrapper uses the
configured SQLite heap, which will be different from the default process
heap if the [SQLITE_WIN32_HEAP_CREATE] compile-time option is used.  In
addition, when an allocation is made or freed, HeapValidate() will be
called if SQLite is compiled with assert() enabled and the
[SQLITE_WIN32_MALLOC_VALIDATE] compile-time option.</p>

<tcl>hd_fragment memsys5 *memsys5 {zero-malloc memory allocator}</tcl>
<h4>3.1.4 Zero-malloc memory allocator</h4>

<p>When SQLite is compiled with the [SQLITE_ENABLE_MEMSYS5] option, an
alternative memory allocator that does not use malloc() is included in the
build.  The SQLite developers refer to this alternative memory allocator
as "memsys5".  Even when it is included in the build, memsys5 is 
disabled by default.
To enable memsys5, the application must invoke the following SQLite 
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
requests are rounded up to a power of two and the request is satisfied
by the first free slot in pBuf that is large enough.  Adjacent freed
allocations are coalesced using a buddy system. When used appropriately,
this algorithm provides mathematical guarantees against fragmentation and
breakdown, as described further <a href="#nofrag">below</a>.</p>

<tcl>hd_fragment memsysx {experimental memory allocators}</tcl>
<h4>3.1.4 Experimental memory allocators</h4>

<p>The name "memsys5" used for the zero-malloc memory allocator implies
that there are several additional memory allocators available, and indeed
there are.  The default memory allocator is "memsys1".  The debugging
memory allocator is "memsys2".  Those have already been covered.</p>

<p>If SQLite is compiled with [SQLITE_ENABLE_MEMSYS3] than another







|







351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
requests are rounded up to a power of two and the request is satisfied
by the first free slot in pBuf that is large enough.  Adjacent freed
allocations are coalesced using a buddy system. When used appropriately,
this algorithm provides mathematical guarantees against fragmentation and
breakdown, as described further <a href="#nofrag">below</a>.</p>

<tcl>hd_fragment memsysx {experimental memory allocators}</tcl>
<h4>3.1.5 Experimental memory allocators</h4>

<p>The name "memsys5" used for the zero-malloc memory allocator implies
that there are several additional memory allocators available, and indeed
there are.  The default memory allocator is "memsys1".  The debugging
memory allocator is "memsys2".  Those have already been covered.</p>

<p>If SQLite is compiled with [SQLITE_ENABLE_MEMSYS3] than another
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
[version 3.6.5].</p>

<p>Other experimental memory allocators might be added in future releases
of SQLite.  One may anticipate that these will be called memsys7, memsys8,
and so forth.</p>

<a name="appalloc"></a>
<h4>3.1.5 Application-defined memory allocators</h4>

<p>New memory allocators do not have to be part of the SQLite source tree
nor included in the sqlite3.c [amalgamation].  Individual applications can
supply their own memory allocators to SQLite at start-time.</p>

<p>To cause SQLite to use a new memory allocator, the application
simply calls:</p>







|







408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
[version 3.6.5].</p>

<p>Other experimental memory allocators might be added in future releases
of SQLite.  One may anticipate that these will be called memsys7, memsys8,
and so forth.</p>

<a name="appalloc"></a>
<h4>3.1.6 Application-defined memory allocators</h4>

<p>New memory allocators do not have to be part of the SQLite source tree
nor included in the sqlite3.c [amalgamation].  Individual applications can
supply their own memory allocators to SQLite at start-time.</p>

<p>To cause SQLite to use a new memory allocator, the application
simply calls:</p>
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435

<p>In a multi-threaded application, access to the [sqlite3_mem_methods]
is serialized if and only if [SQLITE_CONFIG_MEMSTATUS] is enabled.
If [SQLITE_CONFIG_MEMSTATUS] is disabled then the methods in
[sqlite3_mem_methods] must take care of their own serialization needs.</p>

<a name="overlayalloc"></a>
<h4>3.1.6 Memory allocator overlays</h4>

<p>An application can insert layers or "overlays" in between the
SQLite core and the underlying memory allocator.
For example, the <a href="#oomtesting">out-of-memory test logic</a>
for SQLite uses an overlay that can simulate memory allocation
failures.</p>








|







433
434
435
436
437
438
439
440
441
442
443
444
445
446
447

<p>In a multi-threaded application, access to the [sqlite3_mem_methods]
is serialized if and only if [SQLITE_CONFIG_MEMSTATUS] is enabled.
If [SQLITE_CONFIG_MEMSTATUS] is disabled then the methods in
[sqlite3_mem_methods] must take care of their own serialization needs.</p>

<a name="overlayalloc"></a>
<h4>3.1.7 Memory allocator overlays</h4>

<p>An application can insert layers or "overlays" in between the
SQLite core and the underlying memory allocator.
For example, the <a href="#oomtesting">out-of-memory test logic</a>
for SQLite uses an overlay that can simulate memory allocation
failures.</p>

443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
The existing allocator is saved by the overlay and is used as
a fallback to do real memory allocation.  Then the overlay is
inserted in place of the existing memory allocator using
the [sqlite3_config]([SQLITE_CONFIG_MALLOC],...) as described
<a href="#appalloc">above</a>.

<a name="stuballoc"></a>
<h4>3.1.7 No-op memory allocator stub</h4>

<p>If SQLite is compiled with the [SQLITE_ZERO_MALLOC] option, then
the [default memory allocator] is omitted and replaced by a stub
memory allocator that never allocates any memory.  Any calls to the
stub memory allocator will report back that no memory is available.</p>

<p>The no-op memory allocator is not useful by itself.  It exists only







|







455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
The existing allocator is saved by the overlay and is used as
a fallback to do real memory allocation.  Then the overlay is
inserted in place of the existing memory allocator using
the [sqlite3_config]([SQLITE_CONFIG_MALLOC],...) as described
<a href="#appalloc">above</a>.

<a name="stuballoc"></a>
<h4>3.1.8 No-op memory allocator stub</h4>

<p>If SQLite is compiled with the [SQLITE_ZERO_MALLOC] option, then
the [default memory allocator] is omitted and replaced by a stub
memory allocator that never allocates any memory.  Any calls to the
stub memory allocator will report back that no memory is available.</p>

<p>The no-op memory allocator is not useful by itself.  It exists only