Documentation Source Text

Check-in [a2d97df5b9]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add a link to the Facebook corruption report caused by using a closed file descriptor.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: a2d97df5b995313272c462fe63938186371c937d
User & Date: drh 2014-08-19 10:18:57
Context
2014-08-19
10:23
Add a link to the "How To Corrupt" document from the topical index. check-in: b50b87b898 user: drh tags: trunk
10:18
Add a link to the Facebook corruption report caused by using a closed file descriptor. check-in: a2d97df5b9 user: drh tags: trunk
2014-08-16
18:44
Fix some text in the limits.html page. check-in: 26162d3d2b user: drh tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/howtocorrupt.in.

21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39

40
41
42
43
44
45
46
47
48
49




50
51
52
53
54
55
56
<p>SQLite database files are ordinary disk files.
That means that any process can open the file and 
overwrite it with garbage.  There is nothing that the SQLite
library can do to defend against this.</p>

<h3>1.1 Continuing to use a file descriptor after it has been closed</h3>

<p>We have seen cases where a file descriptor was open on a log file,
then that file descriptor was closed and reopened on an SQLite database.
Later, some other thread continued to write log information into the
old file descriptor, not realizing that the log file had been closed
already.  But because the file descriptor had been reopened by SQLite,
the information that was intended to go into the log file ended up
overwriting parts of the SQLite database, leading to corruption of the
database.</p>

<p>One example of this occurred circa 2013-08-30 on the canonical repository
for the <a href="http://www.fossil-scm.org/">Fossil DVCS</a>.  In that event,
file descriptor 2 (standard error) was being erroneously closed prior to 

[sqlite3_open_v2()] so that the file descriptor used for the
repository database file was 2.  Later, another application 
bug caused an assert() statement to emit
an error message by invoking write(2,...).  But since file descriptor 2 was 
now connected to a database file, the error message
overwrote part of the database.  To guard against this kind of problem,
SQLite [version 3.8.1] and later refuse to use low-numbered file descriptors
for database files. 
(See [SQLITE_MINIMUM_FILE_DESCRIPTOR] for additional information.)</p>





<h3>1.2 Backup or restore while a transaction is active</h3>

<p>Systems that run automatic backups in the background might try to
make a backup copy of an SQLite database file while it is in the middle
of a transaction.  The backup copy then might contain some old and some
new content, and thus be corrupt.</p>








|

|
|

|





|
>

|








>
>
>
>







21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
<p>SQLite database files are ordinary disk files.
That means that any process can open the file and 
overwrite it with garbage.  There is nothing that the SQLite
library can do to defend against this.</p>

<h3>1.1 Continuing to use a file descriptor after it has been closed</h3>

<p>We have seen multiple cases where a file descriptor was open on a file,
then that file descriptor was closed and reopened on an SQLite database.
Later, some other thread continued to write into the
old file descriptor, not realizing that the original file had been closed
already.  But because the file descriptor had been reopened by SQLite,
the information that was intended to go into the original file ended up
overwriting parts of the SQLite database, leading to corruption of the
database.</p>

<p>One example of this occurred circa 2013-08-30 on the canonical repository
for the <a href="http://www.fossil-scm.org/">Fossil DVCS</a>.  In that event,
file descriptor 2 (standard error) was being erroneously closed (by
[http://www.stunnel.org/|stunnel], we suspect) prior to 
[sqlite3_open_v2()] so that the file descriptor used for the
repository database file was 2.  Later, an application 
bug caused an assert() statement to emit
an error message by invoking write(2,...).  But since file descriptor 2 was 
now connected to a database file, the error message
overwrote part of the database.  To guard against this kind of problem,
SQLite [version 3.8.1] and later refuse to use low-numbered file descriptors
for database files. 
(See [SQLITE_MINIMUM_FILE_DESCRIPTOR] for additional information.)</p>

<p>Another example of corruption caused by using a closed file
descriptor was 
[https://code.facebook.com/posts/313033472212144/debugging-file-corruption-on-ios/|reported by facebook engineers] in a blog post on 2014-08-12.</p>

<h3>1.2 Backup or restore while a transaction is active</h3>

<p>Systems that run automatic backups in the background might try to
make a backup copy of an SQLite database file while it is in the middle
of a transaction.  The backup copy then might contain some old and some
new content, and thus be corrupt.</p>