Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Fix typos in new documents. |
|---|---|
| Timelines: | family | ancestors | descendants | both | trunk |
| Files: | files | file ages | folders |
| SHA1: |
224d684b8bd2d44031b4a8fe96c68a00 |
| User & Date: | drh 2013-04-16 13:57:56 |
Context
|
2013-04-17
| ||
| 12:58 | Add a documentation page describing how to write run-time loadable extensions. check-in: 6da9f893bc user: drh tags: trunk | |
|
2013-04-16
| ||
| 13:57 | Fix typos in new documents. check-in: 224d684b8b user: drh tags: trunk | |
| 13:09 | Added the mmap.html and errlog.html documents. Updates to the "How to Corrupt" document. Update the change log for 3.7.17. check-in: ccb921fd36 user: drh tags: trunk | |
Changes
Changes to pages/changes.in.
47 47 <li>Report rollback recovery in the [error log] as SQLITE_NOTICE_RECOVER_ROLLBACK. 48 48 Change the error log code for WAL recover from 49 49 SQLITE_OK to SQLITE_NOTICE_RECOVER_WAL. 50 50 <li>Report the risky uses of [unlinked database files] and 51 51 [database filename aliasing] as SQLITE_WARNING messages in the [error log]. 52 52 } 53 53 54 -chng {2012-04-12 (3.7.16.2)} { 54 +chng {2013-04-12 (3.7.16.2)} { 55 55 <li>Fix a bug (present since version 3.7.13) that could result in database corruption 56 56 on windows if two or more processes try to access the same database file at the 57 57 same time and immediately after third process crashed in the middle of committing 58 58 to that same file. See ticket 59 59 [http://www.sqlite.org/src/info/7ff3120e4f | 7ff3120e4f] for further 60 60 information. 61 61
Changes to pages/errlog.in.
1 1 <title>The Error And Warning Log</title> 2 2 <tcl>hd_keywords {errlog} {error log}</tcl> 3 3 4 4 <h1 align="center">The Error And Warning Log</h1> 5 5 6 6 <p>SQLite can be configured to invoke a callback function containing 7 7 an error code and a terse error message whenever anomalies occur. 8 -The mechanism is very helpful in tracking obscure problems that 8 +This mechanism is very helpful in tracking obscure problems that 9 9 occur rarely and in the field. Application developers are encouraged 10 10 to take advantage of the error logging facility of SQLite in their 11 -products, as it presents hardly any overhead at all, but can be a 11 +products, as it is very low CPU and memory cost but can be a 12 12 huge aid for debugging.</p> 13 13 14 14 <h2>Setting Up The Error Logging Callback</h2> 15 15 16 16 <p>There can only be a single error logging callback per process. 17 17 The error logging callback is registered at start-time using C-code 18 18 similar to the following: ................................................................................ 25 25 26 26 <blockquote><pre> 27 27 void errorLogCallback(void *pArg, int iErrCode, const char *zMsg){ 28 28 fprintf(stderr, "(%d) %s\n", iErrCode, zMsg); 29 29 } 30 30 </pre></blockquote> 31 31 32 -<p>The example above illustrates the signature of the error logger callback, 33 -at least. In an embedded application, one usually does not print 32 +<p>The example above illustrates the signature of the error logger callback. 33 +However, in an embedded application, one usually does not print 34 34 messages on stderr. Instead, one might store the messages in a 35 35 preallocated circular buffer where they can be accessed when diagnostic 36 36 information is needed during debugging. Or perhaps the messages can be 37 37 sent to [http://en.wikipedia.org/wiki/Syslog | Syslog]. Somehow, the 38 -messages need to be stored where they are accessible to developers debugging 39 -issues, not displayed to end users.</p> 38 +messages need to be stored where they are accessible to developers, 39 +not displayed to end users.</p> 40 40 41 41 <p>Do not misunderstand: There is nothing technically wrong with displaying 42 42 the error logger messages to end users. The messages do not contain 43 43 sensitive or private information that must be protected from unauthorized 44 44 viewing. Rather the messages are technical in nature and are not useful 45 -or meaningful to the typical end user. These messages are intended for 46 -database geeks. Display them accordingly.</p> 45 +or meaningful to the typical end user. The messages coming from the 46 +error logger are intended for database geeks. Display them accordingly.</p> 47 47 48 48 <h2>Interface Details</h2> 49 49 50 -<p>The third argument to the [sqlite3_config]([SQLITE_CONFIG_LOG],...) call 51 -(the "pData" argument in the example above) is a pointer to arbitrary 50 +<p>The third argument to the [sqlite3_config]([SQLITE_CONFIG_LOG],...) 51 +interface (the "pData" argument in the example above) is a pointer to arbitrary 52 52 data. SQLite passes this pointer through to the first argument of the 53 53 error logger callback. The pointer can be used to pass application-specific 54 54 setup or state information, if desired. Or it can simply be a NULL 55 55 pointer which is ignored by the callback.</p> 56 56 57 57 <p>The second argument to the error logger callback is an integer 58 58 [extended error code]. The third argument to the error logger is the ................................................................................ 59 59 text of the error message. The error message text is stored in a fixed-length 60 60 stack buffer in the calling function and so will only be valid for the 61 61 duration of the error logger callback function. The error logger should 62 62 make a copy of this message into persistent storage if retention of the 63 63 message is needed.</p> 64 64 65 65 <p>The error logger callback should be treated like a signal handler. 66 -The application should do something with the error, quickly, then return 66 +The application should save off or otherwise process the error, then return 67 67 as soon as possible. No other SQLite APIs should be invoked, directly or 68 -indirectly, from the error logger. In particular, the error logger callback 68 +indirectly, from the error logger. SQLite is <u>not</u> reentrant through 69 +the error logger callback. In particular, the error logger callback 69 70 is invoked when a memory allocation fails, so it is generally a bad idea 70 -to try to allocate memory inside the error logger.</p> 71 +to try to allocate memory inside the error logger. Do not even think 72 +about trying to store the error message in another SQLite database.</p> 71 73 72 74 <p>Applications can use the [sqlite3_log(E,F,..)] API to send new messages 73 -to the log, if desired.</p> 75 +to the log, if desired, but this is discouraged. The [sqlite3_log()] 76 +interface is intended for use by extensions only, not by applications.</p> 74 77 75 78 <h2>Variety of Error Messages</h2> 76 79 77 80 <p>The error messages that might be sent to the error logger and their 78 81 exact format is subject to changes from one release to the next. So 79 -applications should not depend on any particular error message text or 82 +applications should not depend on any particular error message text formats or 80 83 error codes. Things do not change capriciously, but they do sometimes 81 84 changes.</p> 82 85 83 86 <p>The following is a partial list of the kinds of messages that might 84 87 appear in the error logger callback.</p> 85 88 86 89 <ul> ................................................................................ 110 113 SQLITE_WARNING messages are logged when database files are renamed or 111 114 aliased in ways that can lead to database corruption. 112 115 (See [unlink corruption | 1] and [database filename aliasing | 2] for 113 116 additional information.) 114 117 </p> 115 118 116 119 <li><p> 117 -Out of memory (OOM) errors conditions generate error logging events 120 +Out of memory (OOM) error conditions generate error logging events 118 121 with the SQLITE_NOMEM error code and a message that says how many bytes 119 122 of memory were requested by the failed allocation. 120 123 </p> 121 124 122 125 <li><p>I/O errors in the OS-interface generate error logging events. 123 126 The message to these events gives the line number in the source code where 124 127 the error originated and the filename associated with the event when ................................................................................ 126 129 127 130 <li><p>When database corruption is detected, an SQLITE_CORRUPT error 128 131 logger callback is invoked. As with I/O errors, the error message text 129 132 contains the line number in the original source code where the error 130 133 was first detected.</p> 131 134 132 135 <li><p> 133 -An error logger callbacked is invoked on SQLITE_MISUSE errors. 136 +An error logger callback is invoked on SQLITE_MISUSE errors. 137 +This is useful in detecting application design issues when return codes 138 +are not consistently checked in the application code. 134 139 </ul> 135 140 136 141 <p>SQLite strives to keep error logger traffic low and only send messages 137 142 to the error logger when there really is something wrong. Applications 138 -might take this further and deliberately ignore certain classes of error 143 +might further cull the error message traffic 144 +by deliberately ignore certain classes of error 139 145 messages that they do not care about. For example, an application that 140 146 makes frequent database schema changes might want to ignore all 141 147 SQLITE_SCHEMA errors.</p> 142 148 143 149 <h2>Summary</h2> 144 150 145 151 <p>The use of the error logger callback is highly recommended. ................................................................................ 148 154 after they get into the field. The error logger callback has also 149 155 proven useful in catching errors occasional errors that the application 150 156 misses because of inconsistent checking of API return codes. 151 157 Developers are encouraged to implement an error logger callback early 152 158 in the development cycle in order to spot unexpected behavior quickly, 153 159 and to leave the error logger callback turned on through deployment. 154 160 If the error logger never finds a problem, then no harm is done. 155 -But failure to set up an appropriate error logger might leave the 156 -developres wishing they had after mysterious and unreproducible 157 -bug reports start coming back from the field.</p> 161 +But failure to set up an appropriate error logger might compromise 162 +diagnostic capabilities later on.</p>
Changes to pages/mmap.in.
5 5 6 6 <p>The default mechanism by which SQLite accesses and updates database disk 7 7 files is the xRead() and xWrite() methods of the 8 8 [sqlite3_io_methods] VFS object. These methods are typically implemented as 9 9 "read()" and "write()" system calls which cause the operating system 10 10 to copy disk content between the kernel buffer cache and user space.</p> 11 11 12 -<p>Beginning with [version 3.7.17], SQLite now also has the option of 12 +<p>Beginning with [version 3.7.17], SQLite has the option of 13 13 accessing disk content directly using memory-mapped I/O and the new 14 14 xFetch() and xUnfetch() methods on [sqlite3_io_methods].</p> 15 15 16 -<p>There are advantages and disadvantages to using memory-mapped I/O: 16 +<p>There are advantages and disadvantages to using memory-mapped I/O. 17 17 Advantages include:</p> 18 18 19 19 <ol> 20 20 <li><p>Many operations, especially I/O intensive operations, can be much 21 21 faster since content does need to be copied between kernel space 22 22 and user space. In some cases, performance can nearly double.</p> 23 23 24 -<li><p>The SQLite library may need less RAM since it is shares pages with 24 +<li><p>The SQLite library may need less RAM since it shares pages with 25 25 the operating-system page cache and does not always need its own copy of 26 26 working pages.</p> 27 27 </ol> 28 28 29 29 <p>But there are also disadvantages:</p> 30 30 31 31 <ol> ................................................................................ 71 71 into the newly allocated heap memory. This involves (at a minimum) 72 72 a copy of the entire page.</p> 73 73 74 74 <p>But if SQLite wants to access a page of the databse file and 75 75 memory mapped I/O is enabled, it first calls the xFetch() method. 76 76 The xFetch() method asks the operating system to return a pointer to 77 77 the requested page, if possible. If the requested page has been or 78 -can be mapped into the applications address space, then xFetch returns 78 +can be mapped into the application address space, then xFetch returns 79 79 a pointer to that page for SQLite to use without having to copy anything. 80 80 Skipping the copy step is what makes memory mapped I/O faster.</p> 81 81 82 82 <p>SQLite does not assume that the xFetch() method will work. If 83 83 a call to xFetch() returns a NULL pointer (indicating that the requested 84 -pages is not currently mapped into the applications address space) then 84 +page is not currently mapped into the applications address space) then 85 85 SQLite silently falls back to using xRead(). An error is only reported 86 86 if xRead() also fails.</p> 87 87 88 88 <p>When updating the database file, SQLite always makes a copy of the 89 89 page content into heap memory before modifying the page. This is necessary 90 90 since the changes are not suppose to be visible to other processes until 91 -after the transaction commits and so the changes must occur in private space. 91 +after the transaction commits and so the changes must occur in private memory. 92 92 After all needed changes are completed, xWrite() is used to move the content 93 93 back into the database file. The current xWrite() implementions for both 94 94 unix and windows check to see if section of the file being written is 95 95 mapped into the applications address space, and if it is the write operation 96 96 is implemented using memcpy() rather than invoking a "write()" system call, 97 97 but that is just an implementation detail. A memory copy occurs either way. 98 98 So the use of memory mapped I/O does not significantly change the performance 99 99 of database changes. Memory mapped I/O is mostly a benefit for queries.</p> 100 100 101 101 <h2>Configuring Memory-Mapped I/O</h2> 102 102 103 103 <p>The "mmap_size" is the maximum number of bytes of the database file that 104 104 SQLite will try to map into the process address space at one time. The 105 -"mmap_size" applies separately to each database file, so the total amount 106 -of process address space that could potentially be used is the "mmap_size" 105 +mmap_size applies separately to each database file, so the total amount 106 +of process address space that could potentially be used is the mmap_size 107 107 times the number of open database files.</p> 108 108 109 -<p>If mmap_size is set to N then all current implementation map the first 109 +<p>To activate memory-mapped I/O, an application an set the mmap_size to some 110 +large value. For example:</p> 111 + 112 +<blockquote><pre> 113 +PRAGMA mmap_size=268435456; 114 +</pre></blockquote> 115 + 116 +<p>To disable memory-mapped I/O, simply set the mmap_size to zero:</p> 117 + 118 +<blockquote><pre> 119 +PRAGMA mmap_size=0; 120 +</pre></blockquote> 121 + 122 +<p>If mmap_size is set to N then all current implementations map the first 110 123 N bytes of the database file and use legacy xRead() calls for any content 111 -beyond N bytes. If the database file is smaller then N, then the entire 124 +beyond N bytes. If the database file is smaller then N bytes, then the entire 112 125 file is mapped. In the future, new OS interfaces could, in theory, map 113 -regions of the file other than the first N bytes, but no such OS interface 126 +regions of the file other than the first N bytes, but no such 114 127 implementation currently exists.</p> 115 128 116 129 <p>The mmap_size is set separately for each database file using the 117 130 "[PRAGMA mmap_size]" statement. The usual default mmap_size is zero, 118 131 meaning that memory mapped I/O is disabled by default. However, the 119 132 default mmap_size can be increased either at compile-time using 120 133 the [SQLITE_DEFAULT_MMAP_SIZE] macro or at start-time using the 121 134 [sqlite3_config]([SQLITE_CONFIG_MMAP_SIZE],...) interface.</p> 122 135 123 136 <p>SQLite also maintains a hard upper bound on the mmap_size. Attempts 124 137 to increase the mmap_size above this hard upper bound (using 125 -[PRAGMA mmap_size] will automatically reduce the mmap_size to the hard 138 +[PRAGMA mmap_size]) will automatically cap the mmap_size at the hard 126 139 upper bound. If the hard upper bound is zero, then memory mapped I/O 127 140 is impossible. The hard upper bound can be set at compile-time using 128 141 the [SQLITE_MAX_MMAP_SIZE] macro. If [SQLITE_MAX_MMAP_SIZE] is set to 129 142 zero, then the code used to implement memory mapped I/O is omitted from 130 143 the build. The hard upper bound is automatically set to zero on certain 131 144 platforms (ex: OpenBSD) where memory mapped I/O does not work due to the 132 145 lack of a unified buffer cache.</p>