Documentation Source Text

Check-in [224d684b8b]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Fix typos in new documents.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 224d684b8bd2d44031b4a8fe96c68a007bb5c170
User & Date: drh 2013-04-16 13:57:56
Context
2013-04-17
12:58
Add a documentation page describing how to write run-time loadable extensions. check-in: 6da9f893bc user: drh tags: trunk
2013-04-16
13:57
Fix typos in new documents. check-in: 224d684b8b user: drh tags: trunk
13:09
Added the mmap.html and errlog.html documents. Updates to the "How to Corrupt" document. Update the change log for 3.7.17. check-in: ccb921fd36 user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to pages/changes.in.

    47     47   <li>Report rollback recovery in the [error log] as SQLITE_NOTICE_RECOVER_ROLLBACK.
    48     48       Change the error log code for WAL recover from 
    49     49       SQLITE_OK to SQLITE_NOTICE_RECOVER_WAL.
    50     50   <li>Report the risky uses of [unlinked database files] and 
    51     51      [database filename aliasing] as SQLITE_WARNING messages in the [error log].
    52     52   }
    53     53   
    54         -chng {2012-04-12 (3.7.16.2)} {
           54  +chng {2013-04-12 (3.7.16.2)} {
    55     55   <li>Fix a bug (present since version 3.7.13) that could result in database corruption
    56     56       on windows if two or more processes try to access the same database file at the
    57     57       same time and immediately after third process crashed in the middle of committing
    58     58       to that same file.  See ticket 
    59     59       [http://www.sqlite.org/src/info/7ff3120e4f | 7ff3120e4f] for further
    60     60       information.
    61     61   

Changes to pages/errlog.in.

     1      1   <title>The Error And Warning Log</title>
     2      2   <tcl>hd_keywords {errlog} {error log}</tcl>
     3      3   
     4      4   <h1 align="center">The Error And Warning Log</h1>
     5      5   
     6      6   <p>SQLite can be configured to invoke a callback function containing
     7      7   an error code and a terse error message whenever anomalies occur.
     8         -The mechanism is very helpful in tracking obscure problems that
            8  +This mechanism is very helpful in tracking obscure problems that
     9      9   occur rarely and in the field.  Application developers are encouraged
    10     10   to take advantage of the error logging facility of SQLite in their
    11         -products, as it presents hardly any overhead at all, but can be a
           11  +products, as it is very low CPU and memory cost but can be a
    12     12   huge aid for debugging.</p>
    13     13   
    14     14   <h2>Setting Up The Error Logging Callback</h2>
    15     15   
    16     16   <p>There can only be a single error logging callback per process.
    17     17   The error logging callback is registered at start-time using C-code
    18     18   similar to the following:
................................................................................
    25     25   
    26     26   <blockquote><pre>
    27     27   void errorLogCallback(void *pArg, int iErrCode, const char *zMsg){
    28     28     fprintf(stderr, "(%d) %s\n", iErrCode, zMsg);
    29     29   }
    30     30   </pre></blockquote>
    31     31   
    32         -<p>The example above illustrates the signature of the error logger callback,
    33         -at least.  In an embedded application, one usually does not print
           32  +<p>The example above illustrates the signature of the error logger callback.
           33  +However, in an embedded application, one usually does not print
    34     34   messages on stderr.  Instead, one might store the messages in a
    35     35   preallocated circular buffer where they can be accessed when diagnostic
    36     36   information is needed during debugging.  Or perhaps the messages can be
    37     37   sent to [http://en.wikipedia.org/wiki/Syslog | Syslog].  Somehow, the
    38         -messages need to be stored where they are accessible to developers debugging
    39         -issues, not displayed to end users.</p>
           38  +messages need to be stored where they are accessible to developers,
           39  +not displayed to end users.</p>
    40     40   
    41     41   <p>Do not misunderstand: There is nothing technically wrong with displaying 
    42     42   the error logger messages to end users.  The messages do not contain
    43     43   sensitive or private information that must be protected from unauthorized
    44     44   viewing.  Rather the messages are technical in nature and are not useful
    45         -or meaningful to the typical end user.  These messages are intended for
    46         -database geeks.  Display them accordingly.</p>
           45  +or meaningful to the typical end user.  The messages coming from the
           46  +error logger are intended for database geeks.  Display them accordingly.</p>
    47     47   
    48     48   <h2>Interface Details</h2>
    49     49   
    50         -<p>The third argument to the [sqlite3_config]([SQLITE_CONFIG_LOG],...) call
    51         -(the "pData" argument in the example above) is a pointer to arbitrary
           50  +<p>The third argument to the [sqlite3_config]([SQLITE_CONFIG_LOG],...) 
           51  +interface (the "pData" argument in the example above) is a pointer to arbitrary
    52     52   data.  SQLite passes this pointer through to the first argument of the
    53     53   error logger callback.  The pointer can be used to pass application-specific 
    54     54   setup or state information, if desired.  Or it can simply be a NULL 
    55     55   pointer which is ignored by the callback.</p>
    56     56   
    57     57   <p>The second argument to the error logger callback is an integer
    58     58   [extended error code].  The third argument to the error logger is the
................................................................................
    59     59   text of the error message.  The error message text is stored in a fixed-length
    60     60   stack buffer in the calling function and so will only be valid for the
    61     61   duration of the error logger callback function.  The error logger should
    62     62   make a copy of this message into persistent storage if retention of the
    63     63   message is needed.</p>
    64     64   
    65     65   <p>The error logger callback should be treated like a signal handler.
    66         -The application should do something with the error, quickly, then return
           66  +The application should save off or otherwise process the error, then return
    67     67   as soon as possible.  No other SQLite APIs should be invoked, directly or
    68         -indirectly, from the error logger.  In particular, the error logger callback
           68  +indirectly, from the error logger.  SQLite is <u>not</u> reentrant through
           69  +the error logger callback.  In particular, the error logger callback
    69     70   is invoked when a memory allocation fails, so it is generally a bad idea
    70         -to try to allocate memory inside the error logger.</p>
           71  +to try to allocate memory inside the error logger.  Do not even think
           72  +about trying to store the error message in another SQLite database.</p>
    71     73   
    72     74   <p>Applications can use the [sqlite3_log(E,F,..)] API to send new messages
    73         -to the log, if desired.</p>
           75  +to the log, if desired, but this is discouraged.  The [sqlite3_log()]
           76  +interface is intended for use by extensions only, not by applications.</p>
    74     77   
    75     78   <h2>Variety of Error Messages</h2>
    76     79   
    77     80   <p>The error messages that might be sent to the error logger and their
    78     81   exact format is subject to changes from one release to the next.  So
    79         -applications should not depend on any particular error message text or
           82  +applications should not depend on any particular error message text formats or
    80     83   error codes.  Things do not change capriciously, but they do sometimes
    81     84   changes.</p>
    82     85   
    83     86   <p>The following is a partial list of the kinds of messages that might
    84     87   appear in the error logger callback.</p>
    85     88   
    86     89   <ul>
................................................................................
   110    113   SQLITE_WARNING messages are logged when database files are renamed or
   111    114   aliased in ways that can lead to database corruption.
   112    115   (See [unlink corruption | 1] and [database filename aliasing | 2] for
   113    116   additional information.)
   114    117   </p>
   115    118   
   116    119   <li><p>
   117         -Out of memory (OOM) errors conditions generate error logging events
          120  +Out of memory (OOM) error conditions generate error logging events
   118    121   with the SQLITE_NOMEM error code and a message that says how many bytes
   119    122   of memory were requested by the failed allocation.
   120    123   </p>
   121    124   
   122    125   <li><p>I/O errors in the OS-interface generate error logging events.
   123    126   The message to these events gives the line number in the source code where
   124    127   the error originated and the filename associated with the event when
................................................................................
   126    129   
   127    130   <li><p>When database corruption is detected, an SQLITE_CORRUPT error
   128    131   logger callback is invoked.  As with I/O errors, the error message text
   129    132   contains the line number in the original source code where the error
   130    133   was first detected.</p>
   131    134   
   132    135   <li><p>
   133         -An error logger callbacked is invoked on SQLITE_MISUSE errors.
          136  +An error logger callback is invoked on SQLITE_MISUSE errors.
          137  +This is useful in detecting application design issues when return codes
          138  +are not consistently checked in the application code.
   134    139   </ul>
   135    140   
   136    141   <p>SQLite strives to keep error logger traffic low and only send messages
   137    142   to the error logger when there really is something wrong.  Applications
   138         -might take this further and deliberately ignore certain classes of error
          143  +might further cull the error message traffic 
          144  +by deliberately ignore certain classes of error
   139    145   messages that they do not care about.  For example, an application that
   140    146   makes frequent database schema changes might want to ignore all
   141    147   SQLITE_SCHEMA errors.</p>
   142    148   
   143    149   <h2>Summary</h2>
   144    150   
   145    151   <p>The use of the error logger callback is highly recommended.
................................................................................
   148    154   after they get into the field.  The error logger callback has also 
   149    155   proven useful in catching errors occasional errors that the application
   150    156   misses because of inconsistent checking of API return codes.
   151    157   Developers are encouraged to implement an error logger callback early
   152    158   in the development cycle in order to spot unexpected behavior quickly,
   153    159   and to leave the error logger callback turned on through deployment.
   154    160   If the error logger never finds a problem, then no harm is done.  
   155         -But failure to set up an appropriate error logger might leave the
   156         -developres wishing they had after mysterious and unreproducible
   157         -bug reports start coming back from the field.</p>
          161  +But failure to set up an appropriate error logger might compromise
          162  +diagnostic capabilities later on.</p>

Changes to pages/mmap.in.

     5      5   
     6      6   <p>The default mechanism by which SQLite accesses and updates database disk
     7      7   files is the xRead() and xWrite() methods of the
     8      8   [sqlite3_io_methods] VFS object.  These methods are typically implemented as
     9      9   "read()" and "write()" system calls which cause the operating system
    10     10   to copy disk content between the kernel buffer cache and user space.</p>
    11     11   
    12         -<p>Beginning with [version 3.7.17], SQLite now also has the option of 
           12  +<p>Beginning with [version 3.7.17], SQLite has the option of 
    13     13   accessing disk content directly using memory-mapped I/O and the new
    14     14   xFetch() and xUnfetch() methods on [sqlite3_io_methods].</p>
    15     15   
    16         -<p>There are advantages and disadvantages to using memory-mapped I/O:
           16  +<p>There are advantages and disadvantages to using memory-mapped I/O.
    17     17   Advantages include:</p>
    18     18   
    19     19   <ol>
    20     20   <li><p>Many operations, especially I/O intensive operations, can be much
    21     21       faster since content does need to be copied between kernel space
    22     22       and user space.  In some cases, performance can nearly double.</p>
    23     23   
    24         -<li><p>The SQLite library may need less RAM since it is shares pages with
           24  +<li><p>The SQLite library may need less RAM since it shares pages with
    25     25       the operating-system page cache and does not always need its own copy of
    26     26       working pages.</p>
    27     27   </ol>
    28     28   
    29     29   <p>But there are also disadvantages:</p>
    30     30   
    31     31   <ol>
................................................................................
    71     71   into the newly allocated heap memory.  This involves (at a minimum)
    72     72   a copy of the entire page.</p>
    73     73   
    74     74   <p>But if SQLite wants to access a page of the databse file and
    75     75   memory mapped I/O is enabled, it first calls the xFetch() method.
    76     76   The xFetch() method asks the operating system to return a pointer to
    77     77   the requested page, if possible.  If the requested page has been or
    78         -can be mapped into the applications address space, then xFetch returns
           78  +can be mapped into the application address space, then xFetch returns
    79     79   a pointer to that page for SQLite to use without having to copy anything.
    80     80   Skipping the copy step is what makes memory mapped I/O faster.</p>
    81     81   
    82     82   <p>SQLite does not assume that the xFetch() method will work.  If
    83     83   a call to xFetch() returns a NULL pointer (indicating that the requested
    84         -pages is not currently mapped into the applications address space) then
           84  +page is not currently mapped into the applications address space) then
    85     85   SQLite silently falls back to using xRead().  An error is only reported
    86     86   if xRead() also fails.</p>
    87     87   
    88     88   <p>When updating the database file, SQLite always makes a copy of the
    89     89   page content into heap memory before modifying the page.  This is necessary
    90     90   since the changes are not suppose to be visible to other processes until
    91         -after the transaction commits and so the changes must occur in private space.
           91  +after the transaction commits and so the changes must occur in private memory.
    92     92   After all needed changes are completed, xWrite() is used to move the content
    93     93   back into the database file.  The current xWrite() implementions for both
    94     94   unix and windows check to see if section of the file being written is 
    95     95   mapped into the applications address space, and if it is the write operation
    96     96   is implemented using memcpy() rather than invoking a "write()" system call,
    97     97   but that is just an implementation detail.  A memory copy occurs either way.
    98     98   So the use of memory mapped I/O does not significantly change the performance
    99     99   of database changes.  Memory mapped I/O is mostly a benefit for queries.</p>
   100    100   
   101    101   <h2>Configuring Memory-Mapped I/O</h2>
   102    102   
   103    103   <p>The "mmap_size" is the maximum number of bytes of the database file that
   104    104   SQLite will try to map into the process address space at one time.  The
   105         -"mmap_size" applies separately to each database file, so the total amount
   106         -of process address space that could potentially be used is the "mmap_size"
          105  +mmap_size applies separately to each database file, so the total amount
          106  +of process address space that could potentially be used is the mmap_size
   107    107   times the number of open database files.</p>
   108    108   
   109         -<p>If mmap_size is set to N then all current implementation map the first
          109  +<p>To activate memory-mapped I/O, an application an set the mmap_size to some
          110  +large value.  For example:</p>
          111  +
          112  +<blockquote><pre>
          113  +PRAGMA mmap_size=268435456;
          114  +</pre></blockquote>
          115  +
          116  +<p>To disable memory-mapped I/O, simply set the mmap_size to zero:</p>
          117  +
          118  +<blockquote><pre>
          119  +PRAGMA mmap_size=0;
          120  +</pre></blockquote>
          121  +
          122  +<p>If mmap_size is set to N then all current implementations map the first
   110    123   N bytes of the database file and use legacy xRead() calls for any content
   111         -beyond N bytes.  If the database file is smaller then N, then the entire
          124  +beyond N bytes.  If the database file is smaller then N bytes, then the entire
   112    125   file is mapped.  In the future, new OS interfaces could, in theory, map
   113         -regions of the file other than the first N bytes, but no such OS interface
          126  +regions of the file other than the first N bytes, but no such 
   114    127   implementation currently exists.</p>
   115    128   
   116    129   <p>The mmap_size is set separately for each database file using the
   117    130   "[PRAGMA mmap_size]" statement.  The usual default mmap_size is zero,
   118    131   meaning that memory mapped I/O is disabled by default.  However, the
   119    132   default mmap_size can be increased either at compile-time using
   120    133   the [SQLITE_DEFAULT_MMAP_SIZE] macro or at start-time using the
   121    134   [sqlite3_config]([SQLITE_CONFIG_MMAP_SIZE],...) interface.</p>
   122    135   
   123    136   <p>SQLite also maintains a hard upper bound on the mmap_size.  Attempts
   124    137   to increase the mmap_size above this hard upper bound (using
   125         -[PRAGMA mmap_size] will automatically reduce the mmap_size to the hard
          138  +[PRAGMA mmap_size]) will automatically cap the mmap_size at the hard
   126    139   upper bound.  If the hard upper bound is zero, then memory mapped I/O
   127    140   is impossible.  The hard upper bound can be set at compile-time using
   128    141   the [SQLITE_MAX_MMAP_SIZE] macro.  If [SQLITE_MAX_MMAP_SIZE] is set to
   129    142   zero, then the code used to implement memory mapped I/O is omitted from
   130    143   the build.  The hard upper bound is automatically set to zero on certain
   131    144   platforms (ex: OpenBSD) where memory mapped I/O does not work due to the
   132    145   lack of a unified buffer cache.</p>