Documentation Source Text

Check-in [0f0af25556]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Modifications to the introduction of the file format document.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:0f0af25556cf2cd20093dc16f24736e7af4123d4
User & Date: dan 2009-05-20 11:36:19
Context
2009-05-21
11:36
Minor edits of fileformat.html. check-in: e216c81d99 user: dan tags: trunk
2009-05-20
11:36
Modifications to the introduction of the file format document. check-in: 0f0af25556 user: dan tags: trunk
08:58
Reduce the number of requirements in fileformat.in governing updating the database file. It is not possible to have too much detail without also defining the expectations SQLite has of a file-system, which is not in scope. check-in: fcd47d8a18 user: dan tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/fileformat.in.

130
131
132
133
134
135
136

137
138







139
140

141
142

143
144


145







146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
...
186
187
188
189
190
191
192
193
194
195
196
197
198

199
200


201

202

203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
....
1095
1096
1097
1098
1099
1100
1101
1102
1103

1104
1105
1106
1107
1108
1109
1110
1111
....
2525
2526
2527
2528
2529
2530
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
set body [subst -novariables {

[h1 "Document Overview"]

  [h2 "Scope and Purpose"]

  <p>

    This document provides an engineering guide to the file formats used by 
    SQLite to store databases on disk. There are two purposes for this:








  <ol>

    <li><p> to make it easier to continue to maintain, test and improve the SQLite 
        software library, and

    <li><p> to facilitate the development of external (non-SQLite) software that may 
        operate directly on SQLite databases stored within a file-system.


  </ol>








  <p>
    None of the information contained in this document is required by programmers
    wishing to use the SQLite library in applications. The intended audience is
    engineers working on SQLite itself or those interested in creating alternative
    methods of accessing SQLite databases (without using SQLite).

  <p>
    It is intended that this document shall provide all the information required to
    create an system that reads and writes SQLite databases in a way that is
    completely compatible with SQLite itself. Or, put another way, this document
    defines the protocols that all SQLite database users (including SQLite) are
    required to follow. The availability of this information makes an SQLite
    database an even safer choice for long-term data storage. If at some point
    in the future the SQLite software library cannot be used to access an
    SQLite database that contains useful data, a procedure or software module
    may be developed based on the content of this document to extract the
    required data.

  <p>
    As well as file format descriptions, this document also describes the way in
    which SQLite compatible clients are required to lock database files when
    reading and/or writing them. It also contains other requirements that must
    be adhered to by all writers in order to allow the implementation of in-memory 
    schema and data caches by database clients.

  [h2 "Document and Requirements Organization"]

    <p>
      The content of this document is divided into three sections.

    <p>
................................................................................
      convoluted format, distributed between two files, the database file
      and a journal file. If a failure occurs while modifying a database image
      in this fashion, then the database image must be extracted from the
      database and journal files found in the file-system following recovery
      (other documentation refers to this as "hot journal rollback"). <b>Section 
      <cite>file_system_usage</cite></b> describes the format used by the
      journal file and the rules for correctly reading a database image from
      the combination of a database file and journal file. The same section
      also includes guidelines and requirements describing the intended method
      for atomically updating the database image within the file-system without
      risking database corruption.

    <p>

      <b>Section <cite>interoperability_requirements</cite></b> contains a description of
      and software requirements related to:




    <ul>

      <li>The locking protocol used by SQLite to manage read and write access
          to the database and journal files within the file-system, and
      <li>the change-counter and schema-cookie protocols that must be followed
          by all database writers to facilitate the implementation of
          efficient in-memory caches of the database schema and content by
          readers and writers.
    </ul>

    <p>
      The information in this document may be used to create non-SQLite
      software that reads or writes SQLite databases. Each section contains
      information that may be used to make external software increasingly 
      compatible with SQLite. All data required by an software module that 
      is required to interpret or create SQLite database images may be found
      in section <cite>database_file_format</cite>. Requirements and guidelines
      for software required to safely read and write database images to disk
      using an SQLite compatible system failure recovery mechanism to prevent
      database corruption or data loss are found in section 
      <cite>file_system_usage</cite>. Finally, rules and requirements for software 
      systems required to read and write live SQLite databases within a system that
      includes real SQLite clients that may also read and write databases
      are presented in section <cite>interoperability_requirements</cite>.

  [h2 "Glossary"]
    <table id=glossary>
      <tr><td>Auto-vacuum last root-page<td>
        A page number stored as 32-bit integer at byte offset 52 of the
        database file header (see section <cite>file_header</cite>). In
        an auto-vacuum database, this is the numerically largest 
        <i>root-page</i> number in the database. Additionally, all pages that
................................................................................
        a data area. The record header consists of <i>N+1</i> variable
        length integers (see section <cite>varint_format</cite>), where
        <i>N</i> is the number of values stored in the record.
      <p>
        The first variable length integer in a record header contains the
        size of the record header in bytes. The following <i>N</i> variable
        length integer values each describe the type and size of the 
        records corresponding SQL value (the second integer in the record
        header describes the first value in the record, etc.). Integer

        values are interpreted according to the following table:
      [Table]
        [Tr]<th>Header Value <th>Data type and size
        [Tr]<td>0 
            <td>An SQL NULL value (type SQLITE_NULL). This value
                consumes zero bytes of space in the record's data area.
        [Tr]<td>1
            <td>An SQL integer value (type SQLITE_INTEGER), stored as a
................................................................................
    [fileformat_import_requirement2 H32300]

  <p>
    The following two sections, <cite>rollback_journal_method</cite>
    and <cite>atomic_write_method</cite>, are somewhat advisory in nature.
    They contain descriptions of two different methods used by SQLite to
    modify a database image within a database file-system representation in
    accordance with the above requirements. These are not the only methods
    that could be used. So long as the above requirements (and
    those in sections <cite>locking_protocol</cite> and 
    <cite>database_header_cookies_protocol</cite>) are honoured, any method may
    be used by an SQLite database writer to update the database file-system
    representation. Sections <cite>rollback_journal_method</cite> and 
    <cite>atomic_write_method</cite> do not contain formal requirements. Formal
    requirements governing the way in which SQLite safely updates database
    file-system representations may be found in <span class=todo>ref</span>.







>

|
>
>
>
>
>
>
>

<
>
|
|
>
|
|
>
>
|
>
>
>
>
>
>
>







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







|
<
<
<
|
<
>
|
|
>
>

>

>
|







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







 







|
|
>
|







 







|
|







130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147

148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170


















171
172
173
174
175
176
177
...
186
187
188
189
190
191
192
193



194

195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211















212
213
214
215
216
217
218
....
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
....
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
set body [subst -novariables {

[h1 "Document Overview"]

  [h2 "Scope and Purpose"]

  <p>
    
    This document provides an engineering guide to the file formats used by 
    SQLite to store databases on disk. It also contains a description of the
    file locking protocol used by SQLite to control read and write access to 
    the files and other protocols for safely modifying the database in a live
    system (one that may contain other database clients). It is intended that 
    this document shall provide all the information required to create an 
    system that reads and writes SQLite databases in a way that is completely 
    compatible with SQLite itself. There are two broad purposes for providing 
    this information:


  <ul>
    <li><p> To make it easier to maintain, test and improve the SQLite 
            software library.

    <li><p> To facilitate the development of external (non-SQLite) software that may 
            operate directly on SQLite databases stored within a file-system. For
	    example a database space analysis utility or a competing database
	    client implementation.
  </ul>

  <p>
    The availability of this information makes an SQLite database an even safer
    choice for long-term data storage. If at some point in the future the
    SQLite software library cannot be used to access an SQLite database that
    contains useful data, a procedure or software module may be developed based
    on the content of this document to extract the required data.

  <p>
    None of the information contained in this document is required by programmers
    wishing to use the SQLite library in applications. The intended audience is
    engineers working on SQLite itself or those interested in creating alternative
    methods of accessing SQLite databases (without using SQLite).




















  [h2 "Document and Requirements Organization"]

    <p>
      The content of this document is divided into three sections.

    <p>
................................................................................
      convoluted format, distributed between two files, the database file
      and a journal file. If a failure occurs while modifying a database image
      in this fashion, then the database image must be extracted from the
      database and journal files found in the file-system following recovery
      (other documentation refers to this as "hot journal rollback"). <b>Section 
      <cite>file_system_usage</cite></b> describes the format used by the
      journal file and the rules for correctly reading a database image from
      the combination of a database file and journal file. 



      


    <p><b>Section <cite>interoperability_requirements</cite></b> contains 
      descriptions of and software requirements related to other protocols that
      must be observed by software that reads and writes SQLite databases
      within a live system, including:


    <ul>
      <li>requirements governing the integrity of database file-system representations,
      <li>the locking protocol used by SQLite to manage read and write access
          to the database and journal files within the file-system, and
      <li>the change-counter and schema-cookie protocols that must be followed
          by all database writers to facilitate the implementation of
          efficient in-memory caches of the database schema and content by
          readers and writers.
    </ul>
















  [h2 "Glossary"]
    <table id=glossary>
      <tr><td>Auto-vacuum last root-page<td>
        A page number stored as 32-bit integer at byte offset 52 of the
        database file header (see section <cite>file_header</cite>). In
        an auto-vacuum database, this is the numerically largest 
        <i>root-page</i> number in the database. Additionally, all pages that
................................................................................
        a data area. The record header consists of <i>N+1</i> variable
        length integers (see section <cite>varint_format</cite>), where
        <i>N</i> is the number of values stored in the record.
      <p>
        The first variable length integer in a record header contains the
        size of the record header in bytes. The following <i>N</i> variable
        length integer values each describe the type and size of the 
	corresponding SQL value within the record (the second integer in the 
	record header describes the first value in the record, etc.). The
	second and subsequent integer values in a record header are interpreted
	according to the following table:
      [Table]
        [Tr]<th>Header Value <th>Data type and size
        [Tr]<td>0 
            <td>An SQL NULL value (type SQLITE_NULL). This value
                consumes zero bytes of space in the record's data area.
        [Tr]<td>1
            <td>An SQL integer value (type SQLITE_INTEGER), stored as a
................................................................................
    [fileformat_import_requirement2 H32300]

  <p>
    The following two sections, <cite>rollback_journal_method</cite>
    and <cite>atomic_write_method</cite>, are somewhat advisory in nature.
    They contain descriptions of two different methods used by SQLite to
    modify a database image within a database file-system representation in
    accordance with the above requirements. They are not the only methods
    that can be used. So long as the above requirements (and
    those in sections <cite>locking_protocol</cite> and 
    <cite>database_header_cookies_protocol</cite>) are honoured, any method may
    be used by an SQLite database writer to update the database file-system
    representation. Sections <cite>rollback_journal_method</cite> and 
    <cite>atomic_write_method</cite> do not contain formal requirements. Formal
    requirements governing the way in which SQLite safely updates database
    file-system representations may be found in <span class=todo>ref</span>.