Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Fix typos in new documents. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
224d684b8bd2d44031b4a8fe96c68a00 |
User & Date: | drh 2013-04-16 13:57:56.615 |
Context
2013-04-17
| ||
12:58 | Add a documentation page describing how to write run-time loadable extensions. (check-in: 6da9f893bc user: drh tags: trunk) | |
2013-04-16
| ||
13:57 | Fix typos in new documents. (check-in: 224d684b8b user: drh tags: trunk) | |
13:09 | Added the mmap.html and errlog.html documents. Updates to the "How to Corrupt" document. Update the change log for 3.7.17. (check-in: ccb921fd36 user: drh tags: trunk) | |
Changes
Changes to pages/changes.in.
︙ | ︙ | |||
47 48 49 50 51 52 53 | <li>Report rollback recovery in the [error log] as SQLITE_NOTICE_RECOVER_ROLLBACK. Change the error log code for WAL recover from SQLITE_OK to SQLITE_NOTICE_RECOVER_WAL. <li>Report the risky uses of [unlinked database files] and [database filename aliasing] as SQLITE_WARNING messages in the [error log]. } | | | 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | <li>Report rollback recovery in the [error log] as SQLITE_NOTICE_RECOVER_ROLLBACK. Change the error log code for WAL recover from SQLITE_OK to SQLITE_NOTICE_RECOVER_WAL. <li>Report the risky uses of [unlinked database files] and [database filename aliasing] as SQLITE_WARNING messages in the [error log]. } chng {2013-04-12 (3.7.16.2)} { <li>Fix a bug (present since version 3.7.13) that could result in database corruption on windows if two or more processes try to access the same database file at the same time and immediately after third process crashed in the middle of committing to that same file. See ticket [http://www.sqlite.org/src/info/7ff3120e4f | 7ff3120e4f] for further information. |
︙ | ︙ |
Changes to pages/errlog.in.
1 2 3 4 5 6 7 | <title>The Error And Warning Log</title> <tcl>hd_keywords {errlog} {error log}</tcl> <h1 align="center">The Error And Warning Log</h1> <p>SQLite can be configured to invoke a callback function containing an error code and a terse error message whenever anomalies occur. | | | | | | | | | | | | > | | > | > | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | <title>The Error And Warning Log</title> <tcl>hd_keywords {errlog} {error log}</tcl> <h1 align="center">The Error And Warning Log</h1> <p>SQLite can be configured to invoke a callback function containing an error code and a terse error message whenever anomalies occur. This mechanism is very helpful in tracking obscure problems that occur rarely and in the field. Application developers are encouraged to take advantage of the error logging facility of SQLite in their products, as it is very low CPU and memory cost but can be a huge aid for debugging.</p> <h2>Setting Up The Error Logging Callback</h2> <p>There can only be a single error logging callback per process. The error logging callback is registered at start-time using C-code similar to the following: <blockquote><pre> [sqlite3_config]([SQLITE_CONFIG_LOG], errLogCallback, pData); </pre></blockquote> <p>The error logger callback function might look something like this:</p> <blockquote><pre> void errorLogCallback(void *pArg, int iErrCode, const char *zMsg){ fprintf(stderr, "(%d) %s\n", iErrCode, zMsg); } </pre></blockquote> <p>The example above illustrates the signature of the error logger callback. However, in an embedded application, one usually does not print messages on stderr. Instead, one might store the messages in a preallocated circular buffer where they can be accessed when diagnostic information is needed during debugging. Or perhaps the messages can be sent to [http://en.wikipedia.org/wiki/Syslog | Syslog]. Somehow, the messages need to be stored where they are accessible to developers, not displayed to end users.</p> <p>Do not misunderstand: There is nothing technically wrong with displaying the error logger messages to end users. The messages do not contain sensitive or private information that must be protected from unauthorized viewing. Rather the messages are technical in nature and are not useful or meaningful to the typical end user. The messages coming from the error logger are intended for database geeks. Display them accordingly.</p> <h2>Interface Details</h2> <p>The third argument to the [sqlite3_config]([SQLITE_CONFIG_LOG],...) interface (the "pData" argument in the example above) is a pointer to arbitrary data. SQLite passes this pointer through to the first argument of the error logger callback. The pointer can be used to pass application-specific setup or state information, if desired. Or it can simply be a NULL pointer which is ignored by the callback.</p> <p>The second argument to the error logger callback is an integer [extended error code]. The third argument to the error logger is the text of the error message. The error message text is stored in a fixed-length stack buffer in the calling function and so will only be valid for the duration of the error logger callback function. The error logger should make a copy of this message into persistent storage if retention of the message is needed.</p> <p>The error logger callback should be treated like a signal handler. The application should save off or otherwise process the error, then return as soon as possible. No other SQLite APIs should be invoked, directly or indirectly, from the error logger. SQLite is <u>not</u> reentrant through the error logger callback. In particular, the error logger callback is invoked when a memory allocation fails, so it is generally a bad idea to try to allocate memory inside the error logger. Do not even think about trying to store the error message in another SQLite database.</p> <p>Applications can use the [sqlite3_log(E,F,..)] API to send new messages to the log, if desired, but this is discouraged. The [sqlite3_log()] interface is intended for use by extensions only, not by applications.</p> <h2>Variety of Error Messages</h2> <p>The error messages that might be sent to the error logger and their exact format is subject to changes from one release to the next. So applications should not depend on any particular error message text formats or error codes. Things do not change capriciously, but they do sometimes changes.</p> <p>The following is a partial list of the kinds of messages that might appear in the error logger callback.</p> <ul> |
︙ | ︙ | |||
110 111 112 113 114 115 116 | SQLITE_WARNING messages are logged when database files are renamed or aliased in ways that can lead to database corruption. (See [unlink corruption | 1] and [database filename aliasing | 2] for additional information.) </p> <li><p> | | | > > > | | < | | 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 | SQLITE_WARNING messages are logged when database files are renamed or aliased in ways that can lead to database corruption. (See [unlink corruption | 1] and [database filename aliasing | 2] for additional information.) </p> <li><p> Out of memory (OOM) error conditions generate error logging events with the SQLITE_NOMEM error code and a message that says how many bytes of memory were requested by the failed allocation. </p> <li><p>I/O errors in the OS-interface generate error logging events. The message to these events gives the line number in the source code where the error originated and the filename associated with the event when there is a corresponding file. </p> <li><p>When database corruption is detected, an SQLITE_CORRUPT error logger callback is invoked. As with I/O errors, the error message text contains the line number in the original source code where the error was first detected.</p> <li><p> An error logger callback is invoked on SQLITE_MISUSE errors. This is useful in detecting application design issues when return codes are not consistently checked in the application code. </ul> <p>SQLite strives to keep error logger traffic low and only send messages to the error logger when there really is something wrong. Applications might further cull the error message traffic by deliberately ignore certain classes of error messages that they do not care about. For example, an application that makes frequent database schema changes might want to ignore all SQLITE_SCHEMA errors.</p> <h2>Summary</h2> <p>The use of the error logger callback is highly recommended. The debugging information that the error logger provides has proven very useful in tracking down obscure problems that occurs with applications after they get into the field. The error logger callback has also proven useful in catching errors occasional errors that the application misses because of inconsistent checking of API return codes. Developers are encouraged to implement an error logger callback early in the development cycle in order to spot unexpected behavior quickly, and to leave the error logger callback turned on through deployment. If the error logger never finds a problem, then no harm is done. But failure to set up an appropriate error logger might compromise diagnostic capabilities later on.</p> |
Changes to pages/mmap.in.
1 2 3 4 5 6 7 8 9 10 11 | <title>Memory-Mapped I/O</title> <tcl>hd_keywords {mmap} {memory-mapped I/O}</tcl> <h1 align="center">Memory-Mapped I/O</h1> <p>The default mechanism by which SQLite accesses and updates database disk files is the xRead() and xWrite() methods of the [sqlite3_io_methods] VFS object. These methods are typically implemented as "read()" and "write()" system calls which cause the operating system to copy disk content between the kernel buffer cache and user space.</p> | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | <title>Memory-Mapped I/O</title> <tcl>hd_keywords {mmap} {memory-mapped I/O}</tcl> <h1 align="center">Memory-Mapped I/O</h1> <p>The default mechanism by which SQLite accesses and updates database disk files is the xRead() and xWrite() methods of the [sqlite3_io_methods] VFS object. These methods are typically implemented as "read()" and "write()" system calls which cause the operating system to copy disk content between the kernel buffer cache and user space.</p> <p>Beginning with [version 3.7.17], SQLite has the option of accessing disk content directly using memory-mapped I/O and the new xFetch() and xUnfetch() methods on [sqlite3_io_methods].</p> <p>There are advantages and disadvantages to using memory-mapped I/O. Advantages include:</p> <ol> <li><p>Many operations, especially I/O intensive operations, can be much faster since content does need to be copied between kernel space and user space. In some cases, performance can nearly double.</p> <li><p>The SQLite library may need less RAM since it shares pages with the operating-system page cache and does not always need its own copy of working pages.</p> </ol> <p>But there are also disadvantages:</p> <ol> |
︙ | ︙ | |||
71 72 73 74 75 76 77 | into the newly allocated heap memory. This involves (at a minimum) a copy of the entire page.</p> <p>But if SQLite wants to access a page of the databse file and memory mapped I/O is enabled, it first calls the xFetch() method. The xFetch() method asks the operating system to return a pointer to the requested page, if possible. If the requested page has been or | | | | | | > > > > > > > > > > > > > | | | | | 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 | into the newly allocated heap memory. This involves (at a minimum) a copy of the entire page.</p> <p>But if SQLite wants to access a page of the databse file and memory mapped I/O is enabled, it first calls the xFetch() method. The xFetch() method asks the operating system to return a pointer to the requested page, if possible. If the requested page has been or can be mapped into the application address space, then xFetch returns a pointer to that page for SQLite to use without having to copy anything. Skipping the copy step is what makes memory mapped I/O faster.</p> <p>SQLite does not assume that the xFetch() method will work. If a call to xFetch() returns a NULL pointer (indicating that the requested page is not currently mapped into the applications address space) then SQLite silently falls back to using xRead(). An error is only reported if xRead() also fails.</p> <p>When updating the database file, SQLite always makes a copy of the page content into heap memory before modifying the page. This is necessary since the changes are not suppose to be visible to other processes until after the transaction commits and so the changes must occur in private memory. After all needed changes are completed, xWrite() is used to move the content back into the database file. The current xWrite() implementions for both unix and windows check to see if section of the file being written is mapped into the applications address space, and if it is the write operation is implemented using memcpy() rather than invoking a "write()" system call, but that is just an implementation detail. A memory copy occurs either way. So the use of memory mapped I/O does not significantly change the performance of database changes. Memory mapped I/O is mostly a benefit for queries.</p> <h2>Configuring Memory-Mapped I/O</h2> <p>The "mmap_size" is the maximum number of bytes of the database file that SQLite will try to map into the process address space at one time. The mmap_size applies separately to each database file, so the total amount of process address space that could potentially be used is the mmap_size times the number of open database files.</p> <p>To activate memory-mapped I/O, an application an set the mmap_size to some large value. For example:</p> <blockquote><pre> PRAGMA mmap_size=268435456; </pre></blockquote> <p>To disable memory-mapped I/O, simply set the mmap_size to zero:</p> <blockquote><pre> PRAGMA mmap_size=0; </pre></blockquote> <p>If mmap_size is set to N then all current implementations map the first N bytes of the database file and use legacy xRead() calls for any content beyond N bytes. If the database file is smaller then N bytes, then the entire file is mapped. In the future, new OS interfaces could, in theory, map regions of the file other than the first N bytes, but no such implementation currently exists.</p> <p>The mmap_size is set separately for each database file using the "[PRAGMA mmap_size]" statement. The usual default mmap_size is zero, meaning that memory mapped I/O is disabled by default. However, the default mmap_size can be increased either at compile-time using the [SQLITE_DEFAULT_MMAP_SIZE] macro or at start-time using the [sqlite3_config]([SQLITE_CONFIG_MMAP_SIZE],...) interface.</p> <p>SQLite also maintains a hard upper bound on the mmap_size. Attempts to increase the mmap_size above this hard upper bound (using [PRAGMA mmap_size]) will automatically cap the mmap_size at the hard upper bound. If the hard upper bound is zero, then memory mapped I/O is impossible. The hard upper bound can be set at compile-time using the [SQLITE_MAX_MMAP_SIZE] macro. If [SQLITE_MAX_MMAP_SIZE] is set to zero, then the code used to implement memory mapped I/O is omitted from the build. The hard upper bound is automatically set to zero on certain platforms (ex: OpenBSD) where memory mapped I/O does not work due to the lack of a unified buffer cache.</p> |
︙ | ︙ |