Documentation Source Text
Check-in [7c6ef04be8]
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
SHA1 Hash:7c6ef04be878ce39b8305ee5fb4920e900800dd1
Date: 2013-10-02 13:34:01
User: drh
Comment:Additional notes on how to corrupt an SQLite database file.
Tags And Properties
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/howtocorrupt.in

30
31
32
33
34
35
36













37
38
39
40
41
42
43
..
62
63
64
65
66
67
68

69
70
71
72
73
74
75
...
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
...
349
350
351
352
353
354
355















356
357
358
359
360
361
362
Later, some other thread continued to write log information into the
old file descriptor, not realizing that the log file had been closed
already.  But because the file descriptor had been reopened by SQLite,
the information that was intended to go into the log file ended up
overwriting parts of the SQLite database, leading to corruption of the
database.</p>














<h3>1.2 Backup or restore while a transaction is active</h3>

<p>Systems that run automatic backups in the background might try to
make a backup copy of an SQLite database file while it is in the middle
of a transaction.  The backup copy then might contain some old and some
new content, and thus be corrupt.</p>

................................................................................
<p>SQLite must see the journal files in order to recover from a crash
or power failure.  If the journal files are moved, deleted, or renamed
after a crash or power failure, then automatic recovery will not work
and the database may go corrupt.</p>

<p>Another manifestation of this problem is
[database corruption caused by inconsistent use of 8+3 filenames].</p>


<h2>2.0 File locking problems</h2>

<p>SQLite uses file locks on the database file, and on the 
[write-ahead log] or [WAL] file, to coordinate access between concurrent
processes.  Without coordination, two threads or processes might try
to make incompatible changes to a database file at the same time,
................................................................................
if one process crashes, the other process will be unable to recover the
transaction in progress because it will be looking in the wrong place
for the appropriate journal.</p>

<p>Beginning with SQLite [version 3.7.17], the unix OS interface will
send SQLITE_WARNING messages to the [error log] if a database file has 
multiple hard links.  As of this writing, SQLite still does not yet detect 
and warn about the use of database files through soft links.</p>


<h2>3.0 Failure to sync</h2>

<p>In order to guarantee that database files are always consistent, SQLite
will occasionally ask the operating system to flush all pending writes to
persistent storage then wait for that flush to complete.  This is 
................................................................................
the time you read this.</p>

<p>When running on QNX, it is recommended that [memory-mapped I/O] never
be used.  Furthermore, to use [WAL mode], it is recommended that applications
employ the [locking_mode | exclusive locking mode] in order to 
use [WAL without shared memory].
















<h2>7.0 Bugs in SQLite</h2>

<p>SQLite is [testing | very carefully tested] to help ensure that it is
as bug-free as possible.  Among the many tests that are carried out for
every SQLite version are tests that simulate power failures, I/O errors,
and out-of-memory (OOM) errors and verify that no database corrupt occurs
during any of these events.  SQLite is also field-proven with approximately







>
>
>
>
>
>
>
>
>
>
>
>
>







 







>







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
..
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
...
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
...
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
Later, some other thread continued to write log information into the
old file descriptor, not realizing that the log file had been closed
already.  But because the file descriptor had been reopened by SQLite,
the information that was intended to go into the log file ended up
overwriting parts of the SQLite database, leading to corruption of the
database.</p>

<p>One example of this occurred circa 2013-08-30 on the canonical repository
for the <a href="http://www.fossil-scm.org/">Fossil DVCS</a>.  In that event,
file descriptor 2 (standard error) was being erroneously closed prior to 
[sqlite3_open_v2()] so that the file descriptor used for the
repository database file was 2.  Later, another application 
bug caused an assert() statement to emit
an error message by invoking write(2,...).  But since file descriptor 2 was 
now connected to a database file, the error message
overwrote part of the database.  To guard against this kind of problem,
SQLite [version 3.8.1] and later refuse to use low-numbered file descriptors
for database files. 
(See [SQLITE_MINIMUM_FILE_DESCRIPTOR] for additional information.)</p>

<h3>1.2 Backup or restore while a transaction is active</h3>

<p>Systems that run automatic backups in the background might try to
make a backup copy of an SQLite database file while it is in the middle
of a transaction.  The backup copy then might contain some old and some
new content, and thus be corrupt.</p>

................................................................................
<p>SQLite must see the journal files in order to recover from a crash
or power failure.  If the journal files are moved, deleted, or renamed
after a crash or power failure, then automatic recovery will not work
and the database may go corrupt.</p>

<p>Another manifestation of this problem is
[database corruption caused by inconsistent use of 8+3 filenames].</p>


<h2>2.0 File locking problems</h2>

<p>SQLite uses file locks on the database file, and on the 
[write-ahead log] or [WAL] file, to coordinate access between concurrent
processes.  Without coordination, two threads or processes might try
to make incompatible changes to a database file at the same time,
................................................................................
if one process crashes, the other process will be unable to recover the
transaction in progress because it will be looking in the wrong place
for the appropriate journal.</p>

<p>Beginning with SQLite [version 3.7.17], the unix OS interface will
send SQLITE_WARNING messages to the [error log] if a database file has 
multiple hard links.  As of this writing, SQLite still does not yet detect 
or warn about the use of database files through soft links.</p>


<h2>3.0 Failure to sync</h2>

<p>In order to guarantee that database files are always consistent, SQLite
will occasionally ask the operating system to flush all pending writes to
persistent storage then wait for that flush to complete.  This is 
................................................................................
the time you read this.</p>

<p>When running on QNX, it is recommended that [memory-mapped I/O] never
be used.  Furthermore, to use [WAL mode], it is recommended that applications
employ the [locking_mode | exclusive locking mode] in order to 
use [WAL without shared memory].


<tcl>hd_fragment fscorruption {filesystem corruption}</tcl>
<h3>6.3 Filesystem Corruption</h3>

<p>Since SQLite databases are ordinary disk files, any malfunction in the
filesystem can corrupt the database.  Filesystems in modern operating systems
are very reliable, but errors do still occur.  For example, on 2013-10-01
the SQLite database that holds the
<a href="http://wiki.tcl.tk/">Wiki for Tcl/Tk</a> went corrupt a few days
after an erroneous attempt to increase the size of an EXT3 partition caused
the filesystem to begin malfunctioning.  In that event, the filesystem
eventually became so badly corrupted that the machine was unusable and
had to be rebuilt.  But the earliest symptom of trouble was the corrupted
SQLite database.</p>

<h2>7.0 Bugs in SQLite</h2>

<p>SQLite is [testing | very carefully tested] to help ensure that it is
as bug-free as possible.  Among the many tests that are carried out for
every SQLite version are tests that simulate power failures, I/O errors,
and out-of-memory (OOM) errors and verify that no database corrupt occurs
during any of these events.  SQLite is also field-proven with approximately