Documentation Source Text

Check-in [92d2de3c1e]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:More work on fileformat.html.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 92d2de3c1e1da4ded1b810769db8ed2011f6c848
User & Date: dan 2009-04-09 13:25:04
Context
2009-04-09
14:55
Begin adding the details of how databases are safely updated to fileformat.html. check-in: aece6e4e5f user: dan tags: trunk
13:25
More work on fileformat.html. check-in: 92d2de3c1e user: dan tags: trunk
2009-04-08
18:38
Changes to fileformat.html. check-in: c3c9601bb3 user: dan tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to images/fileformat/filesystem1.gif.

cannot compute difference between binary files

Changes to images/fileformat/indexpage.odg.

cannot compute difference between binary files

Changes to images/fileformat/rtdocs.css.

55
56
57
58
59
60
61







62
63
64
65
66
67
  */
  .todo        { color: #AA3333 ; font-style : italic }
  .todo:before { content: 'TODO:' }
  p.todo       { border: solid #AA3333 1px; padding: 1ex }

  cite         { font-style: normal; font-weight: normal }
  cite a       { color: inherit; text-decoration: none }







  :link:hover,:visited:hover { background: wheat }

  img {
    display:block;
  }








>
>
>
>
>
>
>






55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
  */
  .todo        { color: #AA3333 ; font-style : italic }
  .todo:before { content: 'TODO:' }
  p.todo       { border: solid #AA3333 1px; padding: 1ex }

  cite         { font-style: normal; font-weight: normal }
  cite a       { color: inherit; text-decoration: none }
  .defnlink    { 
    color: inherit; 
    text-decoration: none;
    font-style: italic;
  }
  .defnlink:visited    { color: inherit }
  h1 .defnlink, h2 .defnlink, h3 .defnlink, h4 .defnlink, .defn .defnlink{ font-style: inherit }
  :link:hover,:visited:hover { background: wheat }

  img {
    display:block;
  }

Changes to pages/fileformat.in.

32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
..
81
82
83
84
85
86
87





88
89
90
91
92









93
94
95
96
97
98
99
...
350
351
352
353
354
355
356








357
358
359
360
361
362
363
....
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091













2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
....
2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
....
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
....
2216
2217
2218
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
2235
2236
2237
2238
2239
2240
2241
2242
2243
....
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316
2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358

2359






2360























2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389
2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
set ::SectionNumbers(1) 0
set ::SectionNumbers(2) 0
set ::SectionNumbers(3) 0
set ::SectionNumbers(fig) 0
catch { set TOC "" }
catch { array unset ::References }


proc H {iLevel zTitle {zName ""}} {

  set zNumber ""
  for {set i 1} {$i <= 4} {incr i} {
    if {$i < $iLevel} {
      append zNumber "$::SectionNumbers($i)."
    }
................................................................................
      <img src="images/fileformat/$zImage">
      <p><i>Figure $::SectionNumbers(fig) - $zCaption</i>
      </center>
  }
}

proc FixReferences {body} {





  foreach {key value} [array get ::References] {
    foreach {zNumber zTitle} $value {}
    lappend l <cite>$key</cite> "<cite><a href=\"#$key\" title=\"$zTitle\">$zNumber</a></cite>"
  }
  string map $l $body









}

proc Table {} {
  set ::Stripe 1
  return "<table class=striped>"
}
proc Tr {} {
................................................................................
        A format used for storing 64-bit signed integer values in SQLite 
        database files. Consumes between 1 and 9 bytes of space, depending
        on the precise value being stored.

      <tr><td>Well formed database file <td>
        An SQLite database file that meets all the criteria laid out in
        section <cite>database_file_format</cite> of this document.








    </table>

[h1 "SQLite Database Files" sqlite_database_files]
 
  <p>
    The bulk of this document, section <cite>database_file_format</cite>,
    contains the definition of a <i>well-formed SQLite database file</i>.
................................................................................
    <p class=req id=H31420>
          [fileformat_import_requirement H31420]

[h1 "Database File-System Representation" file_system_usage]

    <p>
      The previous section, section <cite>database_file_format</cite> 
      describes the format of an SQLite <i>database image</i>. A <i>database
      image</i> is the serialized form of a logical SQLite database. Normally,
      a <i>database image</i> is stored within the file-system in a single
      file, a <i>database file</i>. In this case no other data is stored
      within the <i>database file</i>, the first byte of the <i>database
      file</i> is the first byte of the <i>database image</i>, and the last 
      byte of the <i>database file</i> is the last byte of the <i>database 
      image</i>. For this reason, SQLite is often described as a "single-file 
      database system", implying that the contents of a database are always 
      stored in a single file within the file-system. However, this is not
      always the case. It is also possible for an SQLite <i>database image</i>
      to be distributed between the <i>database file</i> and a 
      <i>journal file</i>. A third file, a <i>master-journal file</i> may
      also be part of the file-system representation. Although a
      <i>master-journal file</i> never contains any part of the <i>database
      image</i>, it can contain meta-data that helps determine which parts of 
      the <i>database image</i> are stored within the <i>database file</i>, and
      which parts are stored within the <i>journal file</i>.

    <p>
      In other words, the file-system representation of an SQLite database
      consists of the following:

    <ul>
      <li> <p>A main <b>database file</b>. The <i>database file</i> is
           always present. It may be zero bytes in size, but it is always
           present.
           
      <li> <p>Optionally, a <b>journal file</b>. If present, the <i>journal
           file</i> is stored in the same file-system directory as the
           <i>database file</i>. The name of the <i>journal file</i> is the
           same as that of the database file with the string "-journal"
           appended to it.

      <li> <p>Optionally, a <b>master-journal file</b> may also be part of the
           file-system representation of a database image. A master-journal
           file may only be part of the representation if the journal file 
           is present. A <i>master-journal file</i> may be located anywhere
           within the file-system and may take any name. If present, the
           <i>master-journal</i> is identified by the <i>master-journal
           pointer</i> stored in the <i>journal file</i> (see section
           <cite>master_journal_ptr</cite> for details).
    </ul>

    <p>
      Usually, a <i>database image</i> is stored entirely within the <i>database
      file</i>. Other configurations, where the <i>database image</i> data
      is distributed between the <i>database file</i> and its <i>journal
      file</i>, are used as interim states when modifying the contents of
      the <i>database image</i> to commit a database transaction.















    <p>
      Sub-section <cite>journal_file_formats</cite> describes the formats 
      used by <i>journal</i> and <i>master-journal</i> files.

    <p>
      Sub-section <cite>reading_from_files</cite> contains a precise 
      description of the various ways a <i>database image</i> may be
      distributed between the <i>database file</i> and <i>journal file</i>, 
      and the rules that must be followed to extract it. In other words, a 
      description of how SQLite or compatible software reads the <i>database 
      image</i> from the file-system.

    <p>
      Sub-section <cite>writing_to_files</cite> describes the guidelines that
      must be followed by SQLite or compatible software to safely modify the
      contents of an SQLite database within the file-system. These techniques
      allow a complex set of modifications to be made to the database image
      atomically, eliminating the risk of database corruption due to 
................................................................................
      database.

[h2 "Journal File Formats" journal_file_formats]

[h3 "Journal File Details" journal_file_format]

    <p>
      This section describes the format used by an SQLite <i>journal file</i>.

    <p>
      A journal file consists of one or more <i>journal headers</i>, zero
      or more <i>journal records</i> and optionally a <i>master journal
      pointer</i>. Each journal file always begins with a
      <i>journal header</i>, followed by zero or more <i>journal records</i>.
      Following this may be a second <i>journal header</i> followed by a
................................................................................
      [Tr]<td>0<td>8<td>The <b>journal magic</b> field always contains a
                        well-known 8-byte string value used to identify SQLite
                        journal files. The well-known sequence of byte values
                        is:
                        <pre>0xd9 0xd5 0x05 0xf9 0x20 0xa1 0x63 0xd7</pre>
      [Tr]<td>8<td>4<td>This field, the <b>record count</b>, is set to the
                        number of <i>journal records</i> that follow this
                        <i>journal header</i> in the <i>journal file</i>.
      [Tr]<td>12<td>4<td>The <b>checksum initializer</b> field is set to a 
                         pseudo-random value. It is used as part of the
                         algorithm to calculate the checksum for all <i>journal
                         records</i> that follow this <i>journal header</i>.
      [Tr]<td>16<td>4<td>This field, the <b>database page count</b>, is set
                         to the number of pages that the <i>database file</i>
                         contained before any modifications associated with
                         <i>write transaction</i> are applied.
      [Tr]<td>20<td>4<td>This field, the <b>sector size</b>, is set to the
                         <i>sector size</i> of the device on which the 
                         <i>journal file</i> was created, in bytes. This value
                         is required when reading the journal file to determine
                         the size of each <i>journal header</i>.
      [Tr]<td>24<td>4<td>The <b>page size</b> field contains the database page
                         size used by the corresponding <i>database file</i>
                         when the <i>journal file</i> was created, in bytes.
    </table>

    <p>
      All <i>journal headers</i> are positioned in the file so that they 
      start at a <i>sector size</i> aligned offset. To achieve this, unused
      space may be left between the start of the second and subsequent
      <i>journal headers</i> and the end of the <i>journal records</i>
................................................................................
                        (the previous field) and the values stored in the
                        <i>checksum initializer</i> field of the preceding
                        <i>journal header</i>.
    </table>

    <p>
      The set of <i>journal records</i> that follow a <i>journal header</i>
      in a <i>journal file</i> are packed tightly together. There are no
      alignment requirements for <i>journal records</i> as there are for
      <i>journal headers</i>.

  [h4 "Master Journal Pointer" master_journal_ptr]

    <p>
      To support <i>atomic</i> transactions that modify more than one 
      database file, SQLite sometimes includes a <i>master journal pointer</i>
      record in a <i>journal file</i>. A <i>master journal pointer</i>
      contains the name of a <i>master journal-file</i> along with a 
      check-sum and some well-known values that allow the 
      <i>master journal pointer</i> to be recognized as such when
      the <i>journal file</i> is read during a rollback operation.

    <p>
      As is the case for a <i>journal header</i>, the start of a <i>master
      journal pointer</i> is always positioned at a <i>sector size</i> 
      aligned offset. If the <i>journal record</i> or <i>journal header</i>
      that appears immediately before the <i>master journal pointer</i> does
      not end at an aligned offset, then unused space is left between the
................................................................................
  <p>
    As described in section <cite>pages_and_page_types</cite> of this document,
    an SQLite database image is a set of contiguously numbered fixed size 
    pages. The numbering starts at 1, not 0. Page 1 contains the 
    <i>database file header</i> and the root page of the <i>schema table</i>, 
    and all other pages within the database image are somehow referred to 
    by number from page 1, either directly or indirectly. In order to be able
    to read the <i>database image</i> from within the file-system, a database
    reader needs to be able to ascertain:

  <ol>
    <li> The <i>page-size</i> used by the database image,
    <li> The number of pages in the <i>database image</i>, and
    <li> The content of each database page.
  </ol>

  <p>
    Usually, the <i>database file</i> contains the complete, unadulterated 
    <i>database image</i>. In this case, reading the <i>database image</i> is 
    straightforward. The <i>page-size</i> used by the database image can be
    read from the 2-byte big-endian integer field stored at byte offset 16 of
    the database file (see section <cite>file_header). The number of pages in
    the <i>database image</i> can be determined by querying the size of
    the database file in bytes and then dividing by the <i>page-size</i>.
    Reading the contents of a <i>database page</i> is a simple matter of 
    reading a block of <i>page-size</i> bytes from an offset calculated from
    the page-number of the required page.
    
  <p>
    However, if there is a valid <i>journal file</i> corresponding to the 
    <i>database file</i> present within the file-system then the situation 
    is more complicated. The file-system is considered to contain a valid 
    <i>journal file</i> if each of the following conditions are met:

  <ul>
    <li> A <i>journal file</i> is present in the file system.
    <li> The <i>journal file</i> either does not end with a well-formed 
         <i>master-journal pointer</i> (see section 
         <cite>master_journal_ptr</cite>) or the <i>master-journal file</i> 
         referred to by the <i>master-journal pointer</i> is present in
         the file-system.
    <li> The first 28 bytes of the <i>journal file</i> contain a 
         well-formed <i>journal header</i> (see section
         <cite>journal_header_format</cite>).  
  </ul>

  <p>
    If the file system contains a valid <i>journal file</i>, then the
    <i>page-size</i> used by and the number of pages in the <i>database
    image</i> are stored in the first <i>journal header</i> of the 
    <i>journal file</i>. Specifically, the page-size is stored as a 4-byte
    big-endian unsigned integer at byte offset 24 of the journal file, and the
    number of pages in the database image is stored as a 4-byte big-endian
    unsigned integer at byte offset of 16 of the same file.
    
  <p>
    The current data for each page of the 
    <i>database image</i>
    may be stored within the <i>database file</i> at a file offset
    based on its page number as it normally is, or the current version of

    the data may be stored somewhere within the <i>journal file</i>.






























    [Figure filesystem1.gif figure_filesystem1 "Two ways to store the same database image"]

  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid <i>journal file</i> within the 
    file-system.

    [fileformat_import_requirement2 H35000]
    [fileformat_import_requirement2 H35010]
    [fileformat_import_requirement2 H35020]

  <p>
    If there is a valid <i>journal file</i> within the file-system, the 
    following requirements govern how a reader should determine the set
    of valid <i>journal records</i> that it contains.

  <p class=todo>Requirements for determining the validity of records.

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the <i>database image</i>
    should be determined by the reader.

    [fileformat_import_requirement2 H35030]
    [fileformat_import_requirement2 H35040]
    [fileformat_import_requirement2 H35050]
    [fileformat_import_requirement2 H35060]

  <p>
    The following requirements dictate the way in which the data for each
    page of the <i>database image</i> can be located within the file-system
    by a database reader.

    [fileformat_import_requirement2 H35070]
    [fileformat_import_requirement2 H35080]

[h2 "Writing to an SQLite Database" writing_to_files]

  <p>
    When an SQLite user commits a transaction that modifies the contents
    of the database, the database representation on disk must be modified
    to reflect the new contents of the database. SQLite is required to do
    make all modifications associated with the transaction such that the 
    logical contents of the database file is modified atomically. If an 
    application, operating system (OS) or power failure occurs while SQLite 
    is updating the database, upon recovery the contents of the database 
    must reflect either that all modifications associated with the 
    database transaction were successfully applied, or that none of the
    modifications were applied and the contents of the database are as
    they were before the failed attempt to modify the database.

  <p>
    Some operations on a file-system may be considered atomic. For example
    deleting a file, or on some systems writing to a single disk sector.
    However, in general there exists no atomic file-system operation
    that may be used to update an SQLite database file with the effects
    of an arbitrary database transaction, which may remove, modify or







<







 







>
>
>
>
>




|
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>







 







|
|
|
|
|
|
|



|
|
|



|
|






|





|









|




|
|
|
|
|
<
>
>
>
>
>
>
>
>
>
>
>
>
>







|
|

|
|







 







|







 







|





|




|



|
|







 







|








|



|







 







|




|




|
|



|






|
|

|


|
|



|
|





|


|





|
|
<
|
>
|
>
>
>
>
>
>

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>




|







|







|









|










|
|
|
<
|
|
|
|
|







32
33
34
35
36
37
38

39
40
41
42
43
44
45
..
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
...
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
....
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111

2112
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
....
2145
2146
2147
2148
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
....
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
2216
2217
2218
....
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263
2264
2265
2266
2267
2268
2269
2270
2271
2272
2273
2274
2275
2276
....
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383
2384
2385
2386
2387
2388
2389

2390
2391
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
2422
2423
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466

2467
2468
2469
2470
2471
2472
2473
2474
2475
2476
2477
2478
set ::SectionNumbers(1) 0
set ::SectionNumbers(2) 0
set ::SectionNumbers(3) 0
set ::SectionNumbers(fig) 0
catch { set TOC "" }
catch { array unset ::References }


proc H {iLevel zTitle {zName ""}} {

  set zNumber ""
  for {set i 1} {$i <= 4} {incr i} {
    if {$i < $iLevel} {
      append zNumber "$::SectionNumbers($i)."
    }
................................................................................
      <img src="images/fileformat/$zImage">
      <p><i>Figure $::SectionNumbers(fig) - $zCaption</i>
      </center>
  }
}

proc FixReferences {body} {
  foreach {term anchor} $::Glossary {
    set re [string map {" " [[:space:]]+} $term]
    set body [regsub -all -nocase $re $body "<a class=defnlink href=\"#$anchor\">\\0</a>"]
  }

  foreach {key value} [array get ::References] {
    foreach {zNumber zTitle} $value {}
    lappend l <cite>$key</cite> "<cite><a href=\"#$key\" title=\"$zTitle\">$zNumber</a></cite>"
  }
  set body [string map $l $body]
}

set ::Glossary {}
proc Glossary {term definition} {
  set anchor [string map {" " _ ' _} $term]
  set anchor "glossary_$anchor"
  lappend ::Glossary $term $anchor

  return "<tr><td class=defn><a name=\"$anchor\"></a>$term <td>$definition"
}

proc Table {} {
  set ::Stripe 1
  return "<table class=striped>"
}
proc Tr {} {
................................................................................
        A format used for storing 64-bit signed integer values in SQLite 
        database files. Consumes between 1 and 9 bytes of space, depending
        on the precise value being stored.

      <tr><td>Well formed database file <td>
        An SQLite database file that meets all the criteria laid out in
        section <cite>database_file_format</cite> of this document.

      [Glossary "Database image" {
        A serialized blob of data representing an SQLite database. The
        contents of a database file are usually a valid database image.
      }]
      [Glossary "Database file" {<span class=todo>This.</span>}]
      [Glossary "Journal file" {<span class=todo>This.</span>}]

    </table>

[h1 "SQLite Database Files" sqlite_database_files]
 
  <p>
    The bulk of this document, section <cite>database_file_format</cite>,
    contains the definition of a <i>well-formed SQLite database file</i>.
................................................................................
    <p class=req id=H31420>
          [fileformat_import_requirement H31420]

[h1 "Database File-System Representation" file_system_usage]

    <p>
      The previous section, section <cite>database_file_format</cite> 
      describes the format of an SQLite database image. A database
      image is the serialized form of a logical SQLite database. Normally,
      a database image is stored within the file-system in a single
      file, a database file. In this case no other data is stored
      within the database file, the first byte of the <i>database
      file</i> is the first byte of the database image, and the last 
      byte of the database file is the last byte of the <i>database 
      image</i>. For this reason, SQLite is often described as a "single-file 
      database system", implying that the contents of a database are always 
      stored in a single file within the file-system. However, this is not
      always the case. It is also possible for an SQLite database image
      to be distributed between the database file and a 
      journal file. A third file, a <i>master-journal file</i> may
      also be part of the file-system representation. Although a
      <i>master-journal file</i> never contains any part of the <i>database
      image</i>, it can contain meta-data that helps determine which parts of 
      the database image are stored within the database file, and
      which parts are stored within the journal file.

    <p>
      In other words, the file-system representation of an SQLite database
      consists of the following:

    <ul>
      <li> <p>A main <b>database file</b>. The database file is
           always present. It may be zero bytes in size, but it is always
           present.
           
      <li> <p>Optionally, a <b>journal file</b>. If present, the <i>journal
           file</i> is stored in the same file-system directory as the
           database file. The name of the journal file is the
           same as that of the database file with the string "-journal"
           appended to it.

      <li> <p>Optionally, a <b>master-journal file</b> may also be part of the
           file-system representation of a database image. A master-journal
           file may only be part of the representation if the journal file 
           is present. A <i>master-journal file</i> may be located anywhere
           within the file-system and may take any name. If present, the
           <i>master-journal</i> is identified by the <i>master-journal
           pointer</i> stored in the journal file (see section
           <cite>master_journal_ptr</cite> for details).
    </ul>

    <p>
      Usually, a database image is stored entirely within the database
      file. Other configurations, where the database image data
      is distributed between the database file and its journal
      file, are used as interim states when modifying the contents of
      the database image to commit a database transaction. In practice,

      a database reader only encounters such a configuration if a previous
      attempt to modify the database image on disk was interrupted by an
      application, OS or power failure. The most practical approach (and
      that taken by SQLite) is to extract the subset of the database image
      currently stored within the journal file and write it into the database
      file, thus restoring the system to a state where the database file
      contains the verbatim database image. Other SQLite documentation, and 
      the comments in the SQLite source code, identify this process as <i>hot 
      journal rollback</i>. Instead of focusing on the <i>hot journal
      rollback</i> process, this document describes only how the data within
      a journal file should be interpret to extract the current database 
      image, not how the database file should be transformed as a result of 
      it.

    <p>
      Sub-section <cite>journal_file_formats</cite> describes the formats 
      used by <i>journal</i> and <i>master-journal</i> files.

    <p>
      Sub-section <cite>reading_from_files</cite> contains a precise 
      description of the various ways a database image may be
      distributed between the database file and journal file, 
      and the rules that must be followed to extract it. In other words, a 
      description of how SQLite or compatible software reads the database 
      image from the file-system.

    <p>
      Sub-section <cite>writing_to_files</cite> describes the guidelines that
      must be followed by SQLite or compatible software to safely modify the
      contents of an SQLite database within the file-system. These techniques
      allow a complex set of modifications to be made to the database image
      atomically, eliminating the risk of database corruption due to 
................................................................................
      database.

[h2 "Journal File Formats" journal_file_formats]

[h3 "Journal File Details" journal_file_format]

    <p>
      This section describes the format used by an SQLite journal file.

    <p>
      A journal file consists of one or more <i>journal headers</i>, zero
      or more <i>journal records</i> and optionally a <i>master journal
      pointer</i>. Each journal file always begins with a
      <i>journal header</i>, followed by zero or more <i>journal records</i>.
      Following this may be a second <i>journal header</i> followed by a
................................................................................
      [Tr]<td>0<td>8<td>The <b>journal magic</b> field always contains a
                        well-known 8-byte string value used to identify SQLite
                        journal files. The well-known sequence of byte values
                        is:
                        <pre>0xd9 0xd5 0x05 0xf9 0x20 0xa1 0x63 0xd7</pre>
      [Tr]<td>8<td>4<td>This field, the <b>record count</b>, is set to the
                        number of <i>journal records</i> that follow this
                        <i>journal header</i> in the journal file.
      [Tr]<td>12<td>4<td>The <b>checksum initializer</b> field is set to a 
                         pseudo-random value. It is used as part of the
                         algorithm to calculate the checksum for all <i>journal
                         records</i> that follow this <i>journal header</i>.
      [Tr]<td>16<td>4<td>This field, the <b>database page count</b>, is set
                         to the number of pages that the database file
                         contained before any modifications associated with
                         <i>write transaction</i> are applied.
      [Tr]<td>20<td>4<td>This field, the <b>sector size</b>, is set to the
                         <i>sector size</i> of the device on which the 
                         journal file was created, in bytes. This value
                         is required when reading the journal file to determine
                         the size of each <i>journal header</i>.
      [Tr]<td>24<td>4<td>The <b>page size</b> field contains the database page
                         size used by the corresponding database file
                         when the journal file was created, in bytes.
    </table>

    <p>
      All <i>journal headers</i> are positioned in the file so that they 
      start at a <i>sector size</i> aligned offset. To achieve this, unused
      space may be left between the start of the second and subsequent
      <i>journal headers</i> and the end of the <i>journal records</i>
................................................................................
                        (the previous field) and the values stored in the
                        <i>checksum initializer</i> field of the preceding
                        <i>journal header</i>.
    </table>

    <p>
      The set of <i>journal records</i> that follow a <i>journal header</i>
      in a journal file are packed tightly together. There are no
      alignment requirements for <i>journal records</i> as there are for
      <i>journal headers</i>.

  [h4 "Master Journal Pointer" master_journal_ptr]

    <p>
      To support <i>atomic</i> transactions that modify more than one 
      database file, SQLite sometimes includes a <i>master journal pointer</i>
      record in a journal file. A <i>master journal pointer</i>
      contains the name of a <i>master journal-file</i> along with a 
      check-sum and some well-known values that allow the 
      <i>master journal pointer</i> to be recognized as such when
      the journal file is read during a rollback operation.

    <p>
      As is the case for a <i>journal header</i>, the start of a <i>master
      journal pointer</i> is always positioned at a <i>sector size</i> 
      aligned offset. If the <i>journal record</i> or <i>journal header</i>
      that appears immediately before the <i>master journal pointer</i> does
      not end at an aligned offset, then unused space is left between the
................................................................................
  <p>
    As described in section <cite>pages_and_page_types</cite> of this document,
    an SQLite database image is a set of contiguously numbered fixed size 
    pages. The numbering starts at 1, not 0. Page 1 contains the 
    <i>database file header</i> and the root page of the <i>schema table</i>, 
    and all other pages within the database image are somehow referred to 
    by number from page 1, either directly or indirectly. In order to be able
    to read the database image from within the file-system, a database
    reader needs to be able to ascertain:

  <ol>
    <li> The <i>page-size</i> used by the database image,
    <li> The number of pages in the database image, and
    <li> The content of each database page.
  </ol>

  <p>
    Usually, the database file contains the complete, unadulterated 
    database image. In this case, reading the database image is 
    straightforward. The <i>page-size</i> used by the database image can be
    read from the 2-byte big-endian integer field stored at byte offset 16 of
    the database file (see section <cite>file_header). The number of pages in
    the database image can be determined by querying the size of
    the database file in bytes and then dividing by the <i>page-size</i>.
    Reading the contents of a <i>database page</i> is a simple matter of 
    reading a block of <i>page-size</i> bytes from an offset calculated from
    the page-number of the required page.
    
  <p>
    However, if there is a valid journal file corresponding to the 
    database file present within the file-system then the situation 
    is more complicated. The file-system is considered to contain a valid 
    journal file if each of the following conditions are met:

  <ul>
    <li><p> A journal file is present in the file system, and
    <li><p> the journal file either does not end with a well-formed 
         <i>master-journal pointer</i> (see section 
         <cite>master_journal_ptr</cite>) or the <i>master-journal file</i> 
         referred to by the <i>master-journal pointer</i> is present in
         the file-system, and
    <li><p> the first 28 bytes of the journal file contain a 
         well-formed <i>journal header</i> (see section
         <cite>journal_header_format</cite>).  
  </ul>

  <p>
    If the file system contains a valid journal file, then the
    <i>page-size</i> used by and the number of pages in the <i>database
    image</i> are stored in the first <i>journal header</i> of the 
    journal file. Specifically, the page-size is stored as a 4-byte
    big-endian unsigned integer at byte offset 24 of the journal file, and the
    number of pages in the database image is stored as a 4-byte big-endian
    unsigned integer at byte offset of 16 of the same file.
    
  <p>
    The current data for each page of the database image may be stored 
    within the database file at a file offset based on its page number as 

    it normally is, or the current version of the data may be stored 
    somewhere within the journal file. For each page within the database 
    image, if the journal file contains a valid journal record for the
    corresponding page-number, then the current content of the database 
    image page is the blob of data stored in the page data field of the
    journal record. If the journal file does not contain a valid journal
    record for a page, then the current content of the database image page
    is the blob of data currently stored in the corresponding region of
    the database file.

  <p class=todo> About "valid" journal records

  <p>
    Figure <cite>figure_filesystem1</cite> illustrates two distinct ways
    to store a database image within the file system. In this example, the
    database image consists of 4 pages of <i>page-size</i> bytes each. The
    content of each of the 4 pages is designated A, B, C and D, respectively.
    Representation 1 uses only the database file. In this case the entire
    database image is stored in the database file.

  <p>
    In representation 2 of figure <cite>figure_filesystem1</cite>, the current
    database images is stored using both the journal file and the database 
    file. The size and page-size of the database image are both stored in
    the first (in this case only) journal header in the journal file. 
    Following the journal header are two valid journal records. These contain
    the data for pages 3 and 4 of the database image. Because there are no
    valid journal records for pages 1 and 2 of the database image, the content
    for each of these is stored in the database file. Even though the contents
    of the file-system is quite different in representation 2 as in
    representation 1, the stored database image is the same in each case: 4
    pages of page-size bytes each, content A, B, C and D respectively.

    [Figure filesystem1.gif figure_filesystem1 "Two ways to store the same database image"]

  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid journal file within the 
    file-system.

    [fileformat_import_requirement2 H35000]
    [fileformat_import_requirement2 H35010]
    [fileformat_import_requirement2 H35020]

  <p>
    If there is a valid journal file within the file-system, the 
    following requirements govern how a reader should determine the set
    of valid <i>journal records</i> that it contains.

  <p class=todo>Requirements for determining the validity of records.

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the database image
    should be determined by the reader.

    [fileformat_import_requirement2 H35030]
    [fileformat_import_requirement2 H35040]
    [fileformat_import_requirement2 H35050]
    [fileformat_import_requirement2 H35060]

  <p>
    The following requirements dictate the way in which the data for each
    page of the database image can be located within the file-system
    by a database reader.

    [fileformat_import_requirement2 H35070]
    [fileformat_import_requirement2 H35080]

[h2 "Writing to an SQLite Database" writing_to_files]

  <p>
    When an SQLite user commits a transaction that modifies the contents
    of the database, the database representation on disk must be modified
    to reflect the new contents of the database image. SQLite is required 
    to do make all modifications associated with the transaction such that 
    the database image is modified atomically. If an application, OS or 

    power failure occurs while SQLite is updating the database, upon recovery 
    the contents of the database must reflect either that all modifications 
    associated with the database transaction were successfully applied, or 
    that none of the modifications were applied and the contents of the 
    database are as they were before the failed attempt to modify the database.

  <p>
    Some operations on a file-system may be considered atomic. For example
    deleting a file, or on some systems writing to a single disk sector.
    However, in general there exists no atomic file-system operation
    that may be used to update an SQLite database file with the effects
    of an arbitrary database transaction, which may remove, modify or