Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Work on documentation of the changes in 3.6.0. Still a lot more work to do. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
b96e5d72ea35cfb43ce0a2e8c733cb10 |
User & Date: | drh 2008-06-11 20:53:46.000 |
Context
2008-06-18
| ||
12:18 | Fix for CVSTrac ticket #3177. (check-in: 47fbff8bc7 user: drh tags: trunk) | |
2008-06-11
| ||
20:53 | Work on documentation of the changes in 3.6.0. Still a lot more work to do. (check-in: b96e5d72ea user: drh tags: trunk) | |
13:22 | Fix an unresolved hyperlink in the pragma.html documentation. (check-in: 571ec9cab5 user: drh tags: trunk) | |
Changes
Added pages/35to36.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | <title>SQLite Changes From Version 3.5.9 To 3.6.0</title> <tcl> proc CODE {text} { hd_puts "<blockquote><pre>" hd_puts $text hd_puts "</pre></blockquote>" } proc PARAGRAPH {text} { hd_resolve <p>$text</p>\n } set level(0) 0 set level(1) 0 proc HEADING {n name {tag {}}} { if {$tag!=""} { hd_fragment $tag } global level incr level($n) for {set i [expr {$n+1}]} {$i<10} {incr i} { set level($i) 0 } if {$n==0} { set num {} } elseif {$n==1} { set num $level(1).0 } else { set num $level(1) for {set i 2} {$i<=$n} {incr i} { append num .$level($i) } } incr n 1 hd_puts "<h$n>$num $name</h$n>" } hd_keywords 35to36 HEADING 0 {Moving From SQLite 3.5.9 to 3.6.0} PARAGRAPH { SQLite version 3.5.0 introduced a new OS interface layer that provided an abstraction of the underlying operating system. This was an important innovation and has proven to be helpful in porting and maintaining SQLite. However, the developers have discovered some minor flaws in the original "virtual file system" design introduced in version 3.5.0 and so SQLite 3.6.0 includes some small yet incompatible changes to address these flaws. The purpose of these document is to detail the incompatible changes so that system developers will know what alterations they need to make in order to move from SQLite version 3.5 to version 3.6. } PARAGRAPH { <b>Important Note:</b> The interface changes in SQLite version 3.6.0 only effect the rare applications that make use of the [sqlite3_vfs | virtual file system] interface in SQLite. The changes introduced by SQLite version 3.6.0 will no impact on the vast majority of SQLite applications that use the built-in interfaces to unix, windows, and os/2. } HEADING 1 {Overview Of Changes} PARAGRAPH { A quick enumeration of the changes in SQLite version 3.6.0 is provide here. Subsequent sections will describe these changes in more detail. } PARAGRAPH { <ol> <li><p>The signature of the xAccess method on the [sqlite3_vfs] object has been modified to return an [error code] and to store its output into an integer pointed to by a parameter, rather than returning the output directly. This change allows the xAccess() method to report failures.</p></li> <li><p>A new error code, [SQLITE_IOERR_ACCESS], was added for cases where the xAccess method of [sqlite3_vfs] fails due to an I/O error. </p></li> <li><p>The xGetTempname method has been removed from [sqlite3_vfs]. In its place, the xOpen method is enhanced to open a temporary file of its own invention when the filename parameter is NULL.</p></li> <li><p>The signature of the xCheckReservedLock method on [sqlite3_io_methods] has been modifed so that it returns an [error code] and stores its boolean result into an integer pointed to by a parameter.</p></li> <li><p><i>TBD: local time method</i></p></li> <li><p>Ports of SQLite to new operation systems (operating systems other than the unix, windows, and os/2 ports provided as part of the core) must provide implementations of two new functions [sqlite3_initialize()] and [sqlite3_shutdown()].</p></li> </ol> } </tcl> |
Changes to pages/amalgamation.in.
1 2 3 4 | <title>The SQLite Amalgamation</title> <h2>The SQLite Amalgamation</h2> | | | | | | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | <title>The SQLite Amalgamation</title> <h2>The SQLite Amalgamation</h2> <p>The core SQLite library consists of about 76 files of C code (as of version 3.6.0) in the core with 11 additional files in the FTS3 and RTREE extensions. Most of these are "source" files in the sense that they are stored in the configuration management system and are maintained directly. But 6 of the core C files are generated automatically during the compilation process. Of the 87 code files, 66 are C code and 21 are C header files.</p> <p>The standard makefiles for SQLite have a target for building an object we call the "amalgamation". The amalgamation is a single C code file, named "sqlite3.c", that contains all C code for the core SQLite library and the FTS3 and RTREE extensions. This file contains about 91K lines of code (55K if you omit blank lines and comments) and is over 3 megabytes in size.</p> <p>The amalgamation contains everything you need to integrate SQLite into a larger project. Just copy the amalgamation into your source directory and compile it along with the other C code files in your project. You may also want to make use of the "sqlite3.h" header file that defines the programming API for SQLite. |
︙ | ︙ |
Added pages/custombuild.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 | <title>Custom Builds Of SQLite 3.6</title> <h1 align="center"> Custom Builds Of SQLite Version 3.6<br> or<br> Porting SQLite 3.6 To New Operating Systems Or Unusual Platforms </h1> <h2>1.0 Introduction</h2> <p>For most applications, the recommended method for building SQLite is to use <a href="amalgamation.html">the amalgamation</a> code file, <b>sqlite3.c</b>, and its corresponding header file <b>sqlite3.h</b>. The sqlite3.c code file should compile and run on any unix, windows, OS/2, or Mac OSX system without any changes or special compiler options. Most applications can simply include the sqlite3.c file together with the other C code files that make up the application, compile them all together, and have working and well configured version of SQLite.</p> <blockquote><i>Most applications work great with SQLite in its default configuration and with no special compile-time configuration. Most developers should be able to completely ignore this document and simply build SQLite from <a href="amalgamation.html">the amalgamation</a> without any special knowledge and without taking any special actions.</i></blockquote> <p>However, highly tuned and specialized applications may want or need to replace some of SQLite's built-in system interface with alternative implementations more suitable for the needs of the application. SQLite is designed to be easily reconfigured at compile-time to meet the specific needs of individual projects. Among the compile-time configuration options for SQLite are these:</p> <ul> <li><p> Replace the built-in mutex subsystem with an alternative implementation. </p></li> <li><p> Completely disable all mutexing for use in single-threaded applications. </p></li> <li><p> Reconfigure the memory allocation subsystem to use a memory allocator other the malloc() implementation from the standard library. </p></li> <li><p> Realign the memory allocation subsystem so that it never calls malloc() at all but instead satisfies all memory requests using a fixed-size memory buffer assigned to SQLite at startup. </p></li> <li><p> Replace the interface to the file system with an alternative design. In other words, override all of the system calls that SQLite makes in order to talk to the disk with a completely different set of system calls. </p></li> <li><p> Override other operating system interfaces such as calls to obtain Zulu or local time. </p></li> </ul> <p>Generally speaking, there are three separate subsystems within SQLite that can be modified or overridden at compile-time. The mutex subsystem is used to serialize access to SQLite resources that are shared among threads. The memory allocation subsystem is used to allocate memory required by SQLite objects and for the database cache. Finally, the [sqlite3_vfs | Virtual File System] subsystem is used to provide a portable interface between SQLite and the underlying operating system and especially the file system. We call these three subsystems the "interface" subsystems of SQLite.</p> <p>We emphasis that most applications are well-served by the built-in default implementations of the SQLite interface subsystems. Developers are encouraged to use the default built-in implementations whenever possible and to build SQLite without any special compile-time options or parameters. However, some highly specialized applications may benefit from substituting or modifying one or more of these built-in SQLite interface subsystems. Or, if SQLite is used on an operating system other than unix, windows, or OS/2 (recognizing that Mac OSX is unix) then none of the interface subsystems that come built into SQLite will work and the application will need to provide alternative implementations suitable for the target platform.</p> <h2>2.0 Configuring Or Replacing The Mutex Subsystem</h2> <p>In a multithreaded environment, SQLite uses mutexes to serialize access to shared resources. The mutex subsystem is only required for applications that access SQLite from multiple threads. For single-threaded applications, or applications which only call SQLite from a single thread, the mutex subsystem can be completely disabled by recompiling with the following option:</p> <blockquote><pre> -DSQLITE_THREADSAFE=0 </pre></blockquote> <p>Mutexes are cheap but they are not free, so performance will be better when mutexes are completely disabled. The resulting library footprint will also be a little smaller. Disability the mutexes as compile-time is a recommended optimization for applications were it makes sense.</p> <p>When using SQLite as a shared library, an application can test to see whether or not mutexes have been disabled using the [sqlite3_threadsafe()] API. Applications that link against SQLite at run-time and use SQLite from multiple threads should probably check this API to make sure they did not accidently get linked against a version of the SQLite library that has its mutexes disabled. Single-threaded applications will, of course, work correctly regardless of whether or not SQLite is configured to be threadsafe, though they will be a little bit faster when using versions of SQLite with mutexes disabled.</p> <p>SQLite mutexes can also be disabled at run-time using the the [sqlite3_config()] interface. To completely disable all mutexing, the application can invoke:</p> <blockquote><pre> sqlite3_config(SQLITE_CONFIG_SINGLETHREAD); </pre></blockquote> <p>Disabling mutexes at run-time is not as effective as disabling them at compile-time since SQLite still must do a test of a boolean variable to see if mutexes are enabled or disabled at each point where a mutex might be required. But there is still a performance advantage for disabling mutexes at run-time.</p> <p>For multi-threaded applications that are careful about how they manage threads, SQLite supports an alternative run-time configuration that is half way between not using any mutexes and the default situation of mutexing everything in sight. This in-the-middle mutex alignment can be established as follows:</p> <blockquote><pre> sqlite3_config(SQLITE_CONFIG_MULTITHREAD); sqlite3_config(SQLITE_CONFIG_MEMSTATS, 0); </pre></blockquote> <p>There are two separate configuration changes here which can be used either togethr or separately. The [SQLITE_CONFIG_MULTITHREAD] setting disables the mutexes that serialize access to [database connection] objects and [prepared statement] objects. With this setting, the application is free to use SQLite from multiple threads, but it must make sure than no two threads try to access the same [database connection] or any [prepared statements] associated with the same [database connection] at the same time. Two threads can use SQLite at the same time, but they must use separate [database connections]. The second [SQLITE_CONFIG_MEMSTATS] setting disables the mechanism in SQLite that tracks the total size of all outstanding memory allocation requests. This omits the need to mutex each call to [sqlite3_malloc()] and [sqlite3_free()], which saves a huge number of mutex operations. But a consequence of disabling the memory statistics mechanism is that the [sqlite3_memory_used()], [sqlite3_memory_highwater()], and [sqlite3_soft_heap_limit()] interfaces cease to work. </p> <p>SQLite uses pthreads for its mutex implementation on unix and SQLite requires a recursive mutex. Most modern pthread implementations support recursive mutexes, but not all do. For systems that do not support recursive mutexes, it is recommended that applications operate in single-threaded mode only. If this is not possible, SQLite provides an alternative recursive mutex implementation built on top of the standard "fast" mutexes of pthreads. This alternative implementation should work correctly as long as pthread_equal() is atomic and the processor has a coherient data cache. The alternative recursive mutex implementation is enabled by the following compiler command-line switch:</p> <blockquote><pre> -DSQLITE_HOMEGROWN_RECURSIVE_MUTEX=1 </pre></blockquote> <p>When porting SQLite to a new operating system, it is usually necessary to completely replace the built-in mutex subsystem with an alternative built around the mutex primitives of the new operating system. This is accomplished by compiling SQLite with the following option:</p> <blockquote><pre> -DSQLITE_MUTEX_APPDEF=1 </pre></blockquote> <p>When SQLite is compiled with the SQLITE_MUTEX_APPDEF=1 option, it completely omits the implementation of its [sqlite3_mutex_alloc | mutex primitive functions]. But the SQLite library still attempts to call these functions where necessary, so the application must itself implement the [sqlite3_mutex_alloc | mutex primitive functions] and link them together with SQLite.</p> <h2>3.0 Configuring Or Replacing The Memory Allocation Subsystem</h2> <p>By default, SQLite obtains the memory it needs for objects and cache from the malloc()/free() implementation of the standard library. There is also on-going work with experimental memory allocators that satisfy all memory requests from a single fixed memory buffer handed to SQLite at application start. Additional information on these experimental memory allocators will be provided in a future revision of this document.</p> <p>SQLite supports the ability of an application to specify an alternative memory allocator at run-time using the [sqlite3_config()] interface. For example:</p> <blockquote><pre> sqlite3_config(SQLITE_CONFIG_MALLOC, dlmalloc, dlfree, dlrealloc, dlmalloc_usable_size); </pre></blockquote> <p>The [SQLITE_CONFIG_MALLOC] setting to [sqlite3_config()] passes four function pointers into SQLite. The first three functions work exactly like malloc(), free(), and realloc(), respectively, from the standard C library. The fourth function pointer must be for a routine that returns the size of a memory allocation given a pointer to that allocation. In the default memory allocator implementation for SQLite, these fourth "memsize" function is implemented by prepending an 8-byte size integer to the beginning of every allocation. The memsize function is not a standard part of must memory alloction libraries and so must be implemented in this way. However, Doug Lea's dlmalloc implementation, as shown in the example above, does provide an implementation of memsize which, if used, reduces the size overhead and execution time of every memory allocation and deallocation.</p> <p><i>TBD: Talk about alternative zero-malloc implementations and how to select them at compile-time.</i></p> <p><i>TBD: Talk about how to disable any built-in memory allocator so that an application is required to register the memory allocator at startup.</i></p> <h2>4.0 Adding New Virtual File Systems</h2> <p>Since [version 3.5.0], SQLite has supported an interface called the [sqlite3_vfs | virtual file system] or "VFS". This object is somewhat misnamed since it is really an interface to whole underlying operating system, just the filesystem.</p> <p> One of the interesting features of the VFS interface is that SQLite can support multiple VFSes at the same time. Each [database connection] has to choose a single VFS for its use when the connection is first opened using [sqlite3_open_v2()]. But if a process contains multiple [database connections] each can choose a different VFS. VFSes can be added at run-time using the [sqlite3_vfs_register()] interface.</p> <p>The default builds for SQLite on unix, windows, and OS/2 include a VFS appropriate for the target platform. SQLite builds for other operating systems do not contain a VFS by default, but the application can register one or more at run-time.</p> <h2>5.0 Porting SQLite To A New Operating System</h2> <p>In order to port SQLite to a new operating system - an operating system not supported by default - the application must provide...</p> <ul> <li> a working mutex subsystem (but only if it is multithreaded), </li> <li> a working memory allocation subsystem (assuming it lacks malloc() in its standard library), and</li> <li> a working VFS implementation.</li> </ul> <p>All of these things can be provided in a single auxiliary C code file and then linked with the stock "sqlite3.c" code file to generate a working SQLite build for the target operating system. In addition to the alternative mutex and memory allocation subsystems and the new VFS, the auxiliary C code file should contain implementations for the following two routines:</p> <ul> <li> [sqlite3_initialize()] </li> <li> [sqlite3_shutdown()] </li> </ul> <p>The "sqlite3.c" code file contains default implementations of a VFS and of the [sqlite3_initialize()] and [sqlite3_shutdown()] functions that are appropriate for unix, windows, and os/2. To prevent one of these default components from being loaded when sqlite3.c is compiled, it is necessary to add the following compile-time option:</p> <blockquote><pre> -DSQLITE_OS_OTHER=1 </pre></blockquote> <p> <p>The SQLite core will call [sqlite3_initialize()] early. The auxiliary C code file can contain an implementation of sqlite3_initialize() that registers an appropriate VFS and also perhaps initializes an alternative mutex system (if mutexes are required) or does any memory allocation subsystem initialization that is required. The SQLite core never calls [sqlite3_shutdown()] but it is part of the official SQLite API and is not otherwise provided when compiled with -DSQLITE_OS_OTHER=1, so the auxiliary C code file should probably provide it for completeness.</p> <p> |