Documentation Source Text

Check-in [c3c9601bb3]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Changes to fileformat.html.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: c3c9601bb362525a58c8b14f1f0d4604e113e1bb
User & Date: dan 2009-04-08 18:38:35
Context
2009-04-09
13:25
More work on fileformat.html. check-in: 92d2de3c1e user: dan tags: trunk
2009-04-08
18:38
Changes to fileformat.html. check-in: c3c9601bb3 user: dan tags: trunk
13:43
Further incremental changes to fileformat.html. check-in: b9c6d4b78b user: dan tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Added images/fileformat/filesystem1.gif.

cannot compute difference between binary files

Changes to images/fileformat/indexpage.odg.

cannot compute difference between binary files

Changes to pages/fileformat.in.

2084
2085
2086
2087
2088
2089
2090

2091
2092
2093
2094
2095
2096
2097
....
2219
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
....
2282
2283
2284
2285
2286
2287
2288







2289
2290
2291
2292
2293
2294
2295
2296
2297


2298






2299
2300
2301
2302






2303
2304



2305
2306

























2307

2308
2309
2310


2311
2312
2313
2314
2315
2316
2317
2318
2319
2320





2321
2322
2323
2324
2325
2326
2327

    <p>
      Usually, a <i>database image</i> is stored entirely within the <i>database
      file</i>. Other configurations, where the <i>database image</i> data
      is distributed between the <i>database file</i> and its <i>journal
      file</i>, are used as interim states when modifying the contents of
      the <i>database image</i> to commit a database transaction.


    <p>
      Sub-section <cite>journal_file_formats</cite> describes the formats 
      used by <i>journal</i> and <i>master-journal</i> files.

    <p>
      Sub-section <cite>reading_from_files</cite> contains a precise 
................................................................................

    <p>
      The set of <i>journal records</i> that follow a <i>journal header</i>
      in a <i>journal file</i> are packed tightly together. There are no
      alignment requirements for <i>journal records</i> as there are for
      <i>journal headers</i>.

  [h4 "Master Journal Pointer"]

    <p>
      To support <i>atomic</i> transactions that modify more than one 
      database file, SQLite sometimes includes a <i>master journal pointer</i>
      record in a <i>journal file</i>. A <i>master journal pointer</i>
      contains the name of a <i>master journal-file</i> along with a 
      check-sum and some well-known values that allow the 
................................................................................
               first 8 bytes of a <i>journal header</i>. The well-known
               sequence of bytes is:
                 <pre>0xd9 0xd5 0x05 0xf9 0x20 0xa1 0x63 0xd7</pre>
    </table>

[h3 "Master-Journal File Details" masterjournal_file_format]








[h2 "Reading an SQLite Database" reading_from_files]

  <p>
    As described in section <cite>pages_and_page_types</cite> of this document,
    an SQLite database image is a set of contiguously numbered fixed size 
    pages. The numbering starts at 1, not 0. Page 1 contains the 
    <i>database file header</i> and the root page of the <i>schema table</i>, 
    and all other pages within the database image are somehow referred to 
    by number from page 1, either directly or indirectly.


    






  <p>
    Usually, the <i>database file</i> contains the complete, unadulterated 
    <i>database image</i>. When this is the case, the file offset of each 
    database page can be calculated based on its page number. Reading the






    contents of a <i>database page</i> is a simple matter of reading a
    block of <i>page-size</i> bytes from the calculated offset. However, if 



    there is a <i>journal file</i> corresponding to the <i>database file</i>
    present within the file-system then the situation can be more 

























    complicated. The current data for each page of the <i>database image</i>

    may currently be stored within the <i>database file</i> at a file offset
    based on its page number as it normally is, or the current version of
    the data may be stored somewhere within the <i>journal file</i>.



  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid <i>journal file</i> within the 
    file-system.

    [fileformat_import_requirement2 H35000]
    [fileformat_import_requirement2 H35010]
    [fileformat_import_requirement2 H35020]






  <p class=todo>Requirements for determining the validity of records.

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the <i>database image</i>
    should be determined by the reader.








>







 







|







 







>
>
>
>
>
>
>








|
>
>
|
>
>
>
>
>
>


<
<
>
>
>
>
>
>
|
|
>
>
>
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
|


>
>










>
>
>
>
>







2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
....
2220
2221
2222
2223
2224
2225
2226
2227
2228
2229
2230
2231
2232
2233
2234
....
2283
2284
2285
2286
2287
2288
2289
2290
2291
2292
2293
2294
2295
2296
2297
2298
2299
2300
2301
2302
2303
2304
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315
2316


2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327
2328
2329
2330
2331
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
2377
2378
2379
2380
2381
2382
2383

    <p>
      Usually, a <i>database image</i> is stored entirely within the <i>database
      file</i>. Other configurations, where the <i>database image</i> data
      is distributed between the <i>database file</i> and its <i>journal
      file</i>, are used as interim states when modifying the contents of
      the <i>database image</i> to commit a database transaction.


    <p>
      Sub-section <cite>journal_file_formats</cite> describes the formats 
      used by <i>journal</i> and <i>master-journal</i> files.

    <p>
      Sub-section <cite>reading_from_files</cite> contains a precise 
................................................................................

    <p>
      The set of <i>journal records</i> that follow a <i>journal header</i>
      in a <i>journal file</i> are packed tightly together. There are no
      alignment requirements for <i>journal records</i> as there are for
      <i>journal headers</i>.

  [h4 "Master Journal Pointer" master_journal_ptr]

    <p>
      To support <i>atomic</i> transactions that modify more than one 
      database file, SQLite sometimes includes a <i>master journal pointer</i>
      record in a <i>journal file</i>. A <i>master journal pointer</i>
      contains the name of a <i>master journal-file</i> along with a 
      check-sum and some well-known values that allow the 
................................................................................
               first 8 bytes of a <i>journal header</i>. The well-known
               sequence of bytes is:
                 <pre>0xd9 0xd5 0x05 0xf9 0x20 0xa1 0x63 0xd7</pre>
    </table>

[h3 "Master-Journal File Details" masterjournal_file_format]

  <p>
    A <i>master-journal file</i> contains the full paths to two or more
    <i>journal files</i>, each encoded using UTF-8 encoding and terminated
    by a single nul character (byte value 0x00). There is no padding 
    between the journal paths, each UTF-8 encoded path begins immediately
    after the nul character that terminates the previous one.

[h2 "Reading an SQLite Database" reading_from_files]

  <p>
    As described in section <cite>pages_and_page_types</cite> of this document,
    an SQLite database image is a set of contiguously numbered fixed size 
    pages. The numbering starts at 1, not 0. Page 1 contains the 
    <i>database file header</i> and the root page of the <i>schema table</i>, 
    and all other pages within the database image are somehow referred to 
    by number from page 1, either directly or indirectly. In order to be able
    to read the <i>database image</i> from within the file-system, a database
    reader needs to be able to ascertain:

  <ol>
    <li> The <i>page-size</i> used by the database image,
    <li> The number of pages in the <i>database image</i>, and
    <li> The content of each database page.
  </ol>

  <p>
    Usually, the <i>database file</i> contains the complete, unadulterated 


    <i>database image</i>. In this case, reading the <i>database image</i> is 
    straightforward. The <i>page-size</i> used by the database image can be
    read from the 2-byte big-endian integer field stored at byte offset 16 of
    the database file (see section <cite>file_header). The number of pages in
    the <i>database image</i> can be determined by querying the size of
    the database file in bytes and then dividing by the <i>page-size</i>.
    Reading the contents of a <i>database page</i> is a simple matter of 
    reading a block of <i>page-size</i> bytes from an offset calculated from
    the page-number of the required page.
    
  <p>
    However, if there is a valid <i>journal file</i> corresponding to the 
    <i>database file</i> present within the file-system then the situation 
    is more complicated. The file-system is considered to contain a valid 
    <i>journal file</i> if each of the following conditions are met:

  <ul>
    <li> A <i>journal file</i> is present in the file system.
    <li> The <i>journal file</i> either does not end with a well-formed 
         <i>master-journal pointer</i> (see section 
         <cite>master_journal_ptr</cite>) or the <i>master-journal file</i> 
         referred to by the <i>master-journal pointer</i> is present in
         the file-system.
    <li> The first 28 bytes of the <i>journal file</i> contain a 
         well-formed <i>journal header</i> (see section
         <cite>journal_header_format</cite>).  
  </ul>

  <p>
    If the file system contains a valid <i>journal file</i>, then the
    <i>page-size</i> used by and the number of pages in the <i>database
    image</i> are stored in the first <i>journal header</i> of the 
    <i>journal file</i>. Specifically, the page-size is stored as a 4-byte
    big-endian unsigned integer at byte offset 24 of the journal file, and the
    number of pages in the database image is stored as a 4-byte big-endian
    unsigned integer at byte offset of 16 of the same file.
    
  <p>
    The current data for each page of the 
    <i>database image</i>
    may be stored within the <i>database file</i> at a file offset
    based on its page number as it normally is, or the current version of
    the data may be stored somewhere within the <i>journal file</i>.

    [Figure filesystem1.gif figure_filesystem1 "Two ways to store the same database image"]

  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid <i>journal file</i> within the 
    file-system.

    [fileformat_import_requirement2 H35000]
    [fileformat_import_requirement2 H35010]
    [fileformat_import_requirement2 H35020]

  <p>
    If there is a valid <i>journal file</i> within the file-system, the 
    following requirements govern how a reader should determine the set
    of valid <i>journal records</i> that it contains.

  <p class=todo>Requirements for determining the validity of records.

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the <i>database image</i>
    should be determined by the reader.