Documentation Source Text

Check-in [7c6ef04be8]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Additional notes on how to corrupt an SQLite database file.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:7c6ef04be878ce39b8305ee5fb4920e900800dd1
User & Date: drh 2013-10-02 13:34:01
Context
2013-10-04
17:27
Add new bullets to the change log for 3.8.1. check-in: dc132e270c user: drh tags: trunk
2013-10-02
13:34
Additional notes on how to corrupt an SQLite database file. check-in: 7c6ef04be8 user: drh tags: trunk
2013-09-30
12:10
Fix to the releaselog/current.html generator. check-in: e7d9c2101a user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to pages/howtocorrupt.in.

    30     30   Later, some other thread continued to write log information into the
    31     31   old file descriptor, not realizing that the log file had been closed
    32     32   already.  But because the file descriptor had been reopened by SQLite,
    33     33   the information that was intended to go into the log file ended up
    34     34   overwriting parts of the SQLite database, leading to corruption of the
    35     35   database.</p>
    36     36   
           37  +<p>One example of this occurred circa 2013-08-30 on the canonical repository
           38  +for the <a href="http://www.fossil-scm.org/">Fossil DVCS</a>.  In that event,
           39  +file descriptor 2 (standard error) was being erroneously closed prior to 
           40  +[sqlite3_open_v2()] so that the file descriptor used for the
           41  +repository database file was 2.  Later, another application 
           42  +bug caused an assert() statement to emit
           43  +an error message by invoking write(2,...).  But since file descriptor 2 was 
           44  +now connected to a database file, the error message
           45  +overwrote part of the database.  To guard against this kind of problem,
           46  +SQLite [version 3.8.1] and later refuse to use low-numbered file descriptors
           47  +for database files. 
           48  +(See [SQLITE_MINIMUM_FILE_DESCRIPTOR] for additional information.)</p>
           49  +
    37     50   <h3>1.2 Backup or restore while a transaction is active</h3>
    38     51   
    39     52   <p>Systems that run automatic backups in the background might try to
    40     53   make a backup copy of an SQLite database file while it is in the middle
    41     54   of a transaction.  The backup copy then might contain some old and some
    42     55   new content, and thus be corrupt.</p>
    43     56   
................................................................................
    62     75   <p>SQLite must see the journal files in order to recover from a crash
    63     76   or power failure.  If the journal files are moved, deleted, or renamed
    64     77   after a crash or power failure, then automatic recovery will not work
    65     78   and the database may go corrupt.</p>
    66     79   
    67     80   <p>Another manifestation of this problem is
    68     81   [database corruption caused by inconsistent use of 8+3 filenames].</p>
           82  +
    69     83   
    70     84   <h2>2.0 File locking problems</h2>
    71     85   
    72     86   <p>SQLite uses file locks on the database file, and on the 
    73     87   [write-ahead log] or [WAL] file, to coordinate access between concurrent
    74     88   processes.  Without coordination, two threads or processes might try
    75     89   to make incompatible changes to a database file at the same time,
................................................................................
   192    206   if one process crashes, the other process will be unable to recover the
   193    207   transaction in progress because it will be looking in the wrong place
   194    208   for the appropriate journal.</p>
   195    209   
   196    210   <p>Beginning with SQLite [version 3.7.17], the unix OS interface will
   197    211   send SQLITE_WARNING messages to the [error log] if a database file has 
   198    212   multiple hard links.  As of this writing, SQLite still does not yet detect 
   199         -and warn about the use of database files through soft links.</p>
          213  +or warn about the use of database files through soft links.</p>
   200    214   
   201    215   
   202    216   <h2>3.0 Failure to sync</h2>
   203    217   
   204    218   <p>In order to guarantee that database files are always consistent, SQLite
   205    219   will occasionally ask the operating system to flush all pending writes to
   206    220   persistent storage then wait for that flush to complete.  This is 
................................................................................
   349    363   the time you read this.</p>
   350    364   
   351    365   <p>When running on QNX, it is recommended that [memory-mapped I/O] never
   352    366   be used.  Furthermore, to use [WAL mode], it is recommended that applications
   353    367   employ the [locking_mode | exclusive locking mode] in order to 
   354    368   use [WAL without shared memory].
   355    369   
          370  +
          371  +<tcl>hd_fragment fscorruption {filesystem corruption}</tcl>
          372  +<h3>6.3 Filesystem Corruption</h3>
          373  +
          374  +<p>Since SQLite databases are ordinary disk files, any malfunction in the
          375  +filesystem can corrupt the database.  Filesystems in modern operating systems
          376  +are very reliable, but errors do still occur.  For example, on 2013-10-01
          377  +the SQLite database that holds the
          378  +<a href="http://wiki.tcl.tk/">Wiki for Tcl/Tk</a> went corrupt a few days
          379  +after an erroneous attempt to increase the size of an EXT3 partition caused
          380  +the filesystem to begin malfunctioning.  In that event, the filesystem
          381  +eventually became so badly corrupted that the machine was unusable and
          382  +had to be rebuilt.  But the earliest symptom of trouble was the corrupted
          383  +SQLite database.</p>
          384  +
   356    385   <h2>7.0 Bugs in SQLite</h2>
   357    386   
   358    387   <p>SQLite is [testing | very carefully tested] to help ensure that it is
   359    388   as bug-free as possible.  Among the many tests that are carried out for
   360    389   every SQLite version are tests that simulate power failures, I/O errors,
   361    390   and out-of-memory (OOM) errors and verify that no database corrupt occurs
   362    391   during any of these events.  SQLite is also field-proven with approximately