Documentation Source Text

Check-in [b9c6d4b78b]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Further incremental changes to fileformat.html.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:b9c6d4b78b83ff262de42b622839677be4258597
User & Date: dan 2009-04-08 13:43:48
Context
2009-04-08
18:38
Changes to fileformat.html. check-in: c3c9601bb3 user: dan tags: trunk
13:43
Further incremental changes to fileformat.html. check-in: b9c6d4b78b user: dan tags: trunk
2009-04-07
18:34
Merge two leaves. check-in: 17aaeb85f9 user: dan tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/fileformat.in.

19
20
21
22
23
24
25



26
27
28
29
30
31
32
....
2030
2031
2032
2033
2034
2035
2036








2037
2038
2039
2040
2041
2042








2043




2044
2045
2046
2047











2048
2049





2050
2051




























2052
2053
2054
2055
2056
2057
2058
2059
....
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259
2260
2261
2262
2263



2264

2265




2266
2267

2268
2269
2270
2271
2272
2273
2274




2275
2276
2277


2278
2279
2280
2281
2282
2283
2284
#            <t*l>fileformat_import_requirement H00000</t*l>
#
unset -nocomplain ffreq
hd_read_requirement_file $::DOC/req/hlr30000.txt ffreq
proc fileformat_import_requirement {reqid} {
  return [lindex $::ffreq($reqid) 1]
}



###############################################################################

catch { array unset ::SectionNumbers }
set ::SectionNumbers(1) 0
set ::SectionNumbers(2) 0
set ::SectionNumbers(3) 0
set ::SectionNumbers(fig) 0
................................................................................
          [fileformat_import_requirement H31410]
    <p class=req id=H31420>
          [fileformat_import_requirement H31420]

[h1 "Database File-System Representation" file_system_usage]

    <p>








      SQLite is often described as a "single-file database system", implying 
      that the contents of a database are always stored in a single file
      within the file-system. However, although it is the case that an
      SQLite database image can be and usually is stored within a single 
      file, it is also true that an SQLite database image may be stored
      using two or even three files, as follows:













    <ul>
      <li> A database file.
      <li> Optionally, a journal file.
      <li> Optionally, master-journal file may also be part of the file-system











           representation of a database image. A master-journal file may only
           be part of the representation if the journal file is present.





    </ul>





























[h2 "Journal File Formats"]

[h3 "Journal File Details" journal_file_format]

    <p>
      This section describes the format used by an SQLite <i>journal file</i>.

    <p>
................................................................................
    there is a <i>journal file</i> corresponding to the <i>database file</i>
    present within the file-system then the situation can be more 
    complicated. The current data for each page of the <i>database image</i>
    may currently be stored within the <i>database file</i> at a file offset
    based on its page number as it normally is, or the current version of
    the data may be stored somewhere within the <i>journal file</i>.

  <ol>
    <li> <p>If the <i>journal file</i> contains a well-formed 
      <i>master-journal pointer</i>, and the <i>master-journal</i> is either
      not present in the file-system or it does not contain the name of
      the <i>journal file</i>, then the contents of the <i>journal file</i>
      are invalid. The <i>database file</i> contains the database image
      verbatim.

    <li> <p>If the first 28 bytes of the <i>journal file</i> do not contain
      a well-formed <i>journal header</i>, or they contain a well-formed
      <i>journal header</i> with the <i>record count</i> field set to zero,
      then the contents of the <i>journal file</i> are invalid. As in case
      1 above, the <i>database file</i> contains the database image verbatim.













      <li> <p>If the first 28 bytes of the <i>journal file</i> contain a
      well-formed <i>journal header</i> with a non-zero value for the <i>record
      count</i> field, then the <i>page-size</i> used by the <i>database
      image</i> is as stored at byte offset 24 of the <i>journal header</i>.
      The size of the database image in pages is as stored at byte offset 16 of
      the <i>journal header</i>.






  </ol>




[h2 "Writing to an SQLite Database" writing_to_files]

  <p>
    When an SQLite user commits a transaction that modifies the contents
    of the database, the database representation on disk must be modified
    to reflect the new contents of the database. SQLite is required to do







>
>
>







 







>
>
>
>
>
>
>
>
|
|
|
<
<
<
>
>
>
>
>
>
>
>

>
>
>
>

<
<
<
>
>
>
>
>
>
>
>
>
>
>
|
|
>
>
>
>
>


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|







 







|
|
|
|
<
<
<

<
<
<
<
<
>
>
>

>

>
>
>
>

<
>
|
|
|
<
<
<

>
>
>
>

<
<
>
>







19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
....
2033
2034
2035
2036
2037
2038
2039
2040
2041
2042
2043
2044
2045
2046
2047
2048
2049
2050



2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064



2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
2119
2120
....
2305
2306
2307
2308
2309
2310
2311
2312
2313
2314
2315



2316





2317
2318
2319
2320
2321
2322
2323
2324
2325
2326
2327

2328
2329
2330
2331



2332
2333
2334
2335
2336
2337


2338
2339
2340
2341
2342
2343
2344
2345
2346
#            <t*l>fileformat_import_requirement H00000</t*l>
#
unset -nocomplain ffreq
hd_read_requirement_file $::DOC/req/hlr30000.txt ffreq
proc fileformat_import_requirement {reqid} {
  return [lindex $::ffreq($reqid) 1]
}
proc fileformat_import_requirement2 {reqid} {
  return "<p class=req id=$reqid>[fileformat_import_requirement $reqid]</p>"
}
###############################################################################

catch { array unset ::SectionNumbers }
set ::SectionNumbers(1) 0
set ::SectionNumbers(2) 0
set ::SectionNumbers(3) 0
set ::SectionNumbers(fig) 0
................................................................................
          [fileformat_import_requirement H31410]
    <p class=req id=H31420>
          [fileformat_import_requirement H31420]

[h1 "Database File-System Representation" file_system_usage]

    <p>
      The previous section, section <cite>database_file_format</cite> 
      describes the format of an SQLite <i>database image</i>. A <i>database
      image</i> is the serialized form of a logical SQLite database. Normally,
      a <i>database image</i> is stored within the file-system in a single
      file, a <i>database file</i>. In this case no other data is stored
      within the <i>database file</i>, the first byte of the <i>database
      file</i> is the first byte of the <i>database image</i>, and the last 
      byte of the <i>database file</i> is the last byte of the <i>database 
      image</i>. For this reason, SQLite is often described as a "single-file 
      database system", implying that the contents of a database are always 
      stored in a single file within the file-system. However, this is not



      always the case. It is also possible for an SQLite <i>database image</i>
      to be distributed between the <i>database file</i> and a 
      <i>journal file</i>. A third file, a <i>master-journal file</i> may
      also be part of the file-system representation. Although a
      <i>master-journal file</i> never contains any part of the <i>database
      image</i>, it can contain meta-data that helps determine which parts of 
      the <i>database image</i> are stored within the <i>database file</i>, and
      which parts are stored within the <i>journal file</i>.

    <p>
      In other words, the file-system representation of an SQLite database
      consists of the following:

    <ul>



      <li> <p>A main <b>database file</b>. The <i>database file</i> is
           always present. It may be zero bytes in size, but it is always
           present.
           
      <li> <p>Optionally, a <b>journal file</b>. If present, the <i>journal
           file</i> is stored in the same file-system directory as the
           <i>database file</i>. The name of the <i>journal file</i> is the
           same as that of the database file with the string "-journal"
           appended to it.

      <li> <p>Optionally, a <b>master-journal file</b> may also be part of the
           file-system representation of a database image. A master-journal
           file may only be part of the representation if the journal file 
           is present. A <i>master-journal file</i> may be located anywhere
           within the file-system and may take any name. If present, the
           <i>master-journal</i> is identified by the <i>master-journal
           pointer</i> stored in the <i>journal file</i> (see section
           <cite>master_journal_ptr</cite> for details).
    </ul>

    <p>
      Usually, a <i>database image</i> is stored entirely within the <i>database
      file</i>. Other configurations, where the <i>database image</i> data
      is distributed between the <i>database file</i> and its <i>journal
      file</i>, are used as interim states when modifying the contents of
      the <i>database image</i> to commit a database transaction.

    <p>
      Sub-section <cite>journal_file_formats</cite> describes the formats 
      used by <i>journal</i> and <i>master-journal</i> files.

    <p>
      Sub-section <cite>reading_from_files</cite> contains a precise 
      description of the various ways a <i>database image</i> may be
      distributed between the <i>database file</i> and <i>journal file</i>, 
      and the rules that must be followed to extract it. In other words, a 
      description of how SQLite or compatible software reads the <i>database 
      image</i> from the file-system.

    <p>
      Sub-section <cite>writing_to_files</cite> describes the guidelines that
      must be followed by SQLite or compatible software to safely modify the
      contents of an SQLite database within the file-system. These techniques
      allow a complex set of modifications to be made to the database image
      atomically, eliminating the risk of database corruption due to 
      application, Operating System (OS) or power failure while updating the
      database.

[h2 "Journal File Formats" journal_file_formats]

[h3 "Journal File Details" journal_file_format]

    <p>
      This section describes the format used by an SQLite <i>journal file</i>.

    <p>
................................................................................
    there is a <i>journal file</i> corresponding to the <i>database file</i>
    present within the file-system then the situation can be more 
    complicated. The current data for each page of the <i>database image</i>
    may currently be stored within the <i>database file</i> at a file offset
    based on its page number as it normally is, or the current version of
    the data may be stored somewhere within the <i>journal file</i>.

  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid <i>journal file</i> within the 
    file-system.









    [fileformat_import_requirement2 H35000]
    [fileformat_import_requirement2 H35010]
    [fileformat_import_requirement2 H35020]

  <p class=todo>Requirements for determining the validity of records.

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the <i>database image</i>
    should be determined by the reader.


    [fileformat_import_requirement2 H35030]
    [fileformat_import_requirement2 H35040]
    [fileformat_import_requirement2 H35050]
    [fileformat_import_requirement2 H35060]




  <p>
    The following requirements dictate the way in which the data for each
    page of the <i>database image</i> can be located within the file-system
    by a database reader.



    [fileformat_import_requirement2 H35070]
    [fileformat_import_requirement2 H35080]

[h2 "Writing to an SQLite Database" writing_to_files]

  <p>
    When an SQLite user commits a transaction that modifies the contents
    of the database, the database representation on disk must be modified
    to reflect the new contents of the database. SQLite is required to do

Changes to req/hlr30000.txt.

846
847
848
849
850
851
852
853

























































HLR H31420
For each page that is not a root page but is a part of a B-Tree tree
structure (not part of an overflow chain), the page type of the
corresponding pointer-map entry is set to the value 0x05 and the parent
page number field is set to the page number of the parent node in the
B-Tree structure.

































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909

HLR H31420
For each page that is not a root page but is a part of a B-Tree tree
structure (not part of an overflow chain), the page type of the
corresponding pointer-map entry is set to the value 0x05 and the parent
page number field is set to the page number of the parent node in the
B-Tree structure.



HLR H35000
If a <i>journal file</i> contains a well-formed <i>master-journal 
pointer</i>, and the named <i>master-journal file</i> either does
not exist or does not contain the name of the <i>journal file</i>,
then the <i>journal file</i> shall be considered invalid.

HLR H35010
If the first 28 bytes of a <i>journal file</i> do not contain a well-formed
<i>journal header</i>, then the <i>journal file</i> shall be considered
invalid.

HLR H35020
If the journal file exists within the file-system and neither H35000 nor 
H35010 apply, then the journal file shall be considered valid.



HLR H35030
If there exists a valid <i>journal file</i> in the file-system, then the
database <i>page-size</i> in bytes used to interpret the <i>database image</i>
shall be the value stored as a 4-byte big-endian unsigned integer at byte
offset 24 of the <i>journal file</i>.

HLR H35040
If there exists a valid <i>journal file</i> in the file-system, then the
number of pages in the <i>database image</i> shall be the value stored as 
a 4-byte big-endian unsigned integer at byte offset 24 of the 
<i>journal file</i>.



HLR H35050
If there is no valid <i>journal file</i> in the file-system, then the
database <i>page-size</i> in bytes used to interpret the <i>database image</i>
shall be the value stored as a 2-byte big-endian unsigned integer at byte
offset 16 of the <i>database file</i>.

HLR H35060
If there is no valid <i>journal file</i> in the file-system, then the
number of pages in the <i>database image</i> shall be calculated by dividing
the size of the <i>database file</i> in bytes by the database <i>page-size</i>.


HLR H35070
If there exists a valid <i>journal file</i> in the file-system, then the
contents of each page of the <i>database image</i> for which there is a valid
<i>journal record</i> in the <i>journal file</i> shall be read from the
corresponding journal record.

HLR H35080
The contents of all <i>database image</i> pages for which there is no valid
<i>journal record</i> shall be read from the database file.