Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Additional notes on how to corrupt an SQLite database file. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
7c6ef04be878ce39b8305ee5fb4920e9 |
User & Date: | drh 2013-10-02 13:34:01.790 |
Context
2013-10-04
| ||
17:27 | Add new bullets to the change log for 3.8.1. (check-in: dc132e270c user: drh tags: trunk) | |
2013-10-02
| ||
13:34 | Additional notes on how to corrupt an SQLite database file. (check-in: 7c6ef04be8 user: drh tags: trunk) | |
2013-09-30
| ||
12:10 | Fix to the releaselog/current.html generator. (check-in: e7d9c2101a user: drh tags: trunk) | |
Changes
Changes to pages/howtocorrupt.in.
︙ | ︙ | |||
30 31 32 33 34 35 36 37 38 39 40 41 42 43 | Later, some other thread continued to write log information into the old file descriptor, not realizing that the log file had been closed already. But because the file descriptor had been reopened by SQLite, the information that was intended to go into the log file ended up overwriting parts of the SQLite database, leading to corruption of the database.</p> <h3>1.2 Backup or restore while a transaction is active</h3> <p>Systems that run automatic backups in the background might try to make a backup copy of an SQLite database file while it is in the middle of a transaction. The backup copy then might contain some old and some new content, and thus be corrupt.</p> | > > > > > > > > > > > > > | 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | Later, some other thread continued to write log information into the old file descriptor, not realizing that the log file had been closed already. But because the file descriptor had been reopened by SQLite, the information that was intended to go into the log file ended up overwriting parts of the SQLite database, leading to corruption of the database.</p> <p>One example of this occurred circa 2013-08-30 on the canonical repository for the <a href="http://www.fossil-scm.org/">Fossil DVCS</a>. In that event, file descriptor 2 (standard error) was being erroneously closed prior to [sqlite3_open_v2()] so that the file descriptor used for the repository database file was 2. Later, another application bug caused an assert() statement to emit an error message by invoking write(2,...). But since file descriptor 2 was now connected to a database file, the error message overwrote part of the database. To guard against this kind of problem, SQLite [version 3.8.1] and later refuse to use low-numbered file descriptors for database files. (See [SQLITE_MINIMUM_FILE_DESCRIPTOR] for additional information.)</p> <h3>1.2 Backup or restore while a transaction is active</h3> <p>Systems that run automatic backups in the background might try to make a backup copy of an SQLite database file while it is in the middle of a transaction. The backup copy then might contain some old and some new content, and thus be corrupt.</p> |
︙ | ︙ | |||
62 63 64 65 66 67 68 69 70 71 72 73 74 75 | <p>SQLite must see the journal files in order to recover from a crash or power failure. If the journal files are moved, deleted, or renamed after a crash or power failure, then automatic recovery will not work and the database may go corrupt.</p> <p>Another manifestation of this problem is [database corruption caused by inconsistent use of 8+3 filenames].</p> <h2>2.0 File locking problems</h2> <p>SQLite uses file locks on the database file, and on the [write-ahead log] or [WAL] file, to coordinate access between concurrent processes. Without coordination, two threads or processes might try to make incompatible changes to a database file at the same time, | > | 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | <p>SQLite must see the journal files in order to recover from a crash or power failure. If the journal files are moved, deleted, or renamed after a crash or power failure, then automatic recovery will not work and the database may go corrupt.</p> <p>Another manifestation of this problem is [database corruption caused by inconsistent use of 8+3 filenames].</p> <h2>2.0 File locking problems</h2> <p>SQLite uses file locks on the database file, and on the [write-ahead log] or [WAL] file, to coordinate access between concurrent processes. Without coordination, two threads or processes might try to make incompatible changes to a database file at the same time, |
︙ | ︙ | |||
192 193 194 195 196 197 198 | if one process crashes, the other process will be unable to recover the transaction in progress because it will be looking in the wrong place for the appropriate journal.</p> <p>Beginning with SQLite [version 3.7.17], the unix OS interface will send SQLITE_WARNING messages to the [error log] if a database file has multiple hard links. As of this writing, SQLite still does not yet detect | | | 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 | if one process crashes, the other process will be unable to recover the transaction in progress because it will be looking in the wrong place for the appropriate journal.</p> <p>Beginning with SQLite [version 3.7.17], the unix OS interface will send SQLITE_WARNING messages to the [error log] if a database file has multiple hard links. As of this writing, SQLite still does not yet detect or warn about the use of database files through soft links.</p> <h2>3.0 Failure to sync</h2> <p>In order to guarantee that database files are always consistent, SQLite will occasionally ask the operating system to flush all pending writes to persistent storage then wait for that flush to complete. This is |
︙ | ︙ | |||
349 350 351 352 353 354 355 356 357 358 359 360 361 362 | the time you read this.</p> <p>When running on QNX, it is recommended that [memory-mapped I/O] never be used. Furthermore, to use [WAL mode], it is recommended that applications employ the [locking_mode | exclusive locking mode] in order to use [WAL without shared memory]. <h2>7.0 Bugs in SQLite</h2> <p>SQLite is [testing | very carefully tested] to help ensure that it is as bug-free as possible. Among the many tests that are carried out for every SQLite version are tests that simulate power failures, I/O errors, and out-of-memory (OOM) errors and verify that no database corrupt occurs during any of these events. SQLite is also field-proven with approximately | > > > > > > > > > > > > > > > | 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 | the time you read this.</p> <p>When running on QNX, it is recommended that [memory-mapped I/O] never be used. Furthermore, to use [WAL mode], it is recommended that applications employ the [locking_mode | exclusive locking mode] in order to use [WAL without shared memory]. <tcl>hd_fragment fscorruption {filesystem corruption}</tcl> <h3>6.3 Filesystem Corruption</h3> <p>Since SQLite databases are ordinary disk files, any malfunction in the filesystem can corrupt the database. Filesystems in modern operating systems are very reliable, but errors do still occur. For example, on 2013-10-01 the SQLite database that holds the <a href="http://wiki.tcl.tk/">Wiki for Tcl/Tk</a> went corrupt a few days after an erroneous attempt to increase the size of an EXT3 partition caused the filesystem to begin malfunctioning. In that event, the filesystem eventually became so badly corrupted that the machine was unusable and had to be rebuilt. But the earliest symptom of trouble was the corrupted SQLite database.</p> <h2>7.0 Bugs in SQLite</h2> <p>SQLite is [testing | very carefully tested] to help ensure that it is as bug-free as possible. Among the many tests that are carried out for every SQLite version are tests that simulate power failures, I/O errors, and out-of-memory (OOM) errors and verify that no database corrupt occurs during any of these events. SQLite is also field-proven with approximately |
︙ | ︙ |