Documentation Source Text

Check-in [9a9955c381]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Modify some requirements in fileformat.html to make them more testable.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 9a9955c381353c1f05b8ce23ffe87da3f8f68c69
User & Date: dan 2009-05-26 04:32:49
Context
2009-05-26
15:13
Add a "todo" note to fix the schema file-format range. Also fix another todo in fileformat.html. check-in: 14e677d1dc user: dan tags: trunk
04:32
Modify some requirements in fileformat.html to make them more testable. check-in: 9a9955c381 user: dan tags: trunk
2009-05-25
12:47
Enhancements to the news.html text for the 3.6.14.2 release. check-in: f286448352 user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to pages/fileformat.in.

   209    209             readers and writers.
   210    210       </ul>
   211    211   
   212    212     [h2 "Glossary"]
   213    213       <table id=glossary>
   214    214         <tr><td>Auto-vacuum last root-page<td>
   215    215           A page number stored as 32-bit integer at byte offset 52 of the
   216         -        database file header (see section <cite>database_header</cite>). In
          216  +        database header (see section <cite>database_header</cite>). In
   217    217           an auto-vacuum database, this is the numerically largest 
   218    218           <i>root-page</i> number in the database. Additionally, all pages that
   219    219           occur before this page in the database are either B-Tree <i>root
   220    220           pages</i>, <i>pointer-map pages</i> or the <i>locking page</i>.
   221    221   
   222    222         <tr><td>Auto-vacuum database      <td>
   223    223           Each database is either an auto-vacuum database or a non auto-vacuum
   224    224           database. Auto-vacuum databases feature pointer-map pages (section
   225    225           <cite>pointer_map_pages</cite>) and have a non-zero value stored
   226         -        as a 4-byte big-endian integer at offset 52 of the file header (section
          226  +        as a 4-byte big-endian integer at offset 52 of the database header (section
   227    227           <cite>database_header</cite>).
   228    228         <tr><td>B-Tree                    <td>
   229    229           A B-Tree is a tree structure optimized for offline storage. The table
   230    230           and index data in an SQLite database file is stored in B-Tree
   231    231           structures.
   232    232   
   233    233         <tr><td>B-Tree cell               <td>
................................................................................
   247    247         <tr><td>Cell content area         <td>
   248    248           The area within a B-Tree page in which the B-Tree cells are stored.
   249    249   
   250    250         <tr><td>(Database) text encoding  <td>
   251    251           The text encoding used for all text values in the database file. One
   252    252           of UTF-8, big-endian UTF-16 and little-endian UTF-16. The database
   253    253           text encoding is defined by a 4 byte field stored at byte offset
   254         -        56 of the database file header (see section <cite>database_header</cite>).
          254  +        56 of the database header (see section <cite>database_header</cite>).
   255    255   
   256         -      <tr><td>(Database) file header    <td>
   257         -        The first 100 bytes of an SQLite database file constitute the
   258         -        database file header.
          256  +      [Glossary "Database header" {
          257  +        The first 100 bytes of an SQLite database image constitute the
          258  +	database header. See section <cite>database_header</cite> for details.
          259  +      }]
   259    260   
   260    261         <tr><td>(Database) page size      <td>
   261    262           An SQLite database file is divided into one or more pages of
   262    263           page-size bytes each.
   263    264   
   264    265         <tr><td>Database record           <td>
   265    266           A database record is a blob of data containing the serialized
................................................................................
   280    281           A 32-bit integer field stored at byte offset 48 of the database file
   281    282           header (see section <cite>database_header</cite>).
   282    283   
   283    284         <tr><td style="white-space:nowrap">(Database) usable page size <td>
   284    285           The number of bytes of each database page that is usable. This
   285    286           is the page-size less the number of bytes left unused at the end
   286    287           of each page. The number of bytes left unused is governed by the
   287         -        value stored at offset 20 of the file header (see section
          288  +        value stored at offset 20 of the database header (see section
   288    289           <cite>database_header</cite>).
   289    290   
   290    291         <tr><td>File format read version  <td>
   291         -        Single byte field stored at byte offset 20 of the database file header
          292  +        Single byte field stored at byte offset 20 of the database header
   292    293           (see section <cite>database_header</cite>).
   293    294   
   294    295         <tr><td>File format write version  <td>
   295         -        Single byte field stored at byte offset 19 of the database file header
          296  +        Single byte field stored at byte offset 19 of the database header
   296    297           (see section <cite>database_header</cite>).
   297    298   
   298    299         <tr><td>File change counter       <td>
   299    300           A 32-bit integer field stored at byte offset 24 of the database file
   300    301           header (see section <cite>database_header</cite>). Normally, SQLite
   301    302           increments this value each time it commits a transaction.
   302    303   
................................................................................
   342    343           database properties that may be set by the user (auto-vacuum,
   343    344           page-size, user-cookie value etc.),
   344    345   
   345    346         <tr><td>Non-auto-vacuum database  <td>
   346    347           Any database that is not an auto-vacuum database. A non-auto-vacuum
   347    348           database contains no pointer-map pages and has a zero value stored
   348    349           in the 4-byte big-endian integer field at offset 52 of the database
   349         -        file header (section <cite>database_header</cite>).
          350  +        database header (section <cite>database_header</cite>).
   350    351   
   351    352         <tr><td>Overflow chain             <td>
   352    353           A linked list of overflow pages across which a single (large)
   353    354           database record is stored (see section 
   354    355           <cite>overflow_page_chains</cite>).
   355    356   
   356    357         <tr><td>Overflow page             <td>
................................................................................
   369    370   
   370    371         <tr><td>Root page                 <td>
   371    372           A root page is a database page used to store the root node of a
   372    373           B-Tree data structure.
   373    374   
   374    375         <tr><td>Schema layer file format  <td>
   375    376           An integer between 1 and 4 stored as a 4 byte big-endian integer at
   376         -        offset 44 of the file header (section <cite>database_header</cite>).
          377  +        offset 44 of the database header (section <cite>database_header</cite>).
   377    378           Certain file format constructions may only be present in databases
   378    379           with a certain minimum schema layer file format value.
   379    380   
   380    381         <tr><td>Schema table              <td>
   381    382           The table B-Tree with root-page 1 used to store database records
   382    383           describing the database schema. Accessible as the "sqlite_master" 
   383    384           table from within SQLite.
................................................................................
   450    451   	a journal file. It contains a full file-system path identifying 
   451    452   	a master-journal file.
   452    453         }]
   453    454         [Glossary "Database File-System Representation" {
   454    455           A file or files within the file-system used to store an SQLite 
   455    456           database image.
   456    457         }]
          458  +
          459  +      [Glossary "Database user-cookie" {
          460  +        An SQLite database contains a single 32-bit signed integer field known
          461  +	as the database user-cookie. Applications may read and write this field
          462  +	for any purpose.
          463  +      }]
   457    464   
   458    465       </table>
   459    466   
   460    467   <!--
   461    468   h1 "SQLite Database Files" sqlite_database_files
   462    469    
   463    470     <p>
................................................................................
   513    520       <li>The database schema: The set of database tables, virtual tables, 
   514    521           indexes, triggers and views stored in the database.
   515    522   
   516    523       <li>The database contents: The set of tuples (rows) stored in
   517    524   	each database table.
   518    525   
   519    526       <li>Other database properties, as follows:
   520         -      <ul>
          527  +      <ol>
   521    528   	<li>The page-size of the database.
   522    529   	<li>The text-encoding of the database.
   523    530   	<li>A flag indicating whether or not the database is an auto-vacuum 
   524    531   	    database.
   525    532   	<li>The value of the database user-cookie.
   526    533   	<li>If the database is an auto-vacuum database, a flag indicating 
   527    534   	    whether or not the database is in incremental vacuum mode or not.
   528    535   	<li>The default page cache size in pages to use with the database (an 
   529    536   	    integer field).
   530         -      </ul>
          537  +      </ol>
   531    538     </ul>
          539  +
          540  +  <p class=todo>
          541  +    Say something about items 5 and 6 being advisory only. SQLite places an
          542  +    interpretation on each field, but other applications may safely ignore
          543  +    them.
   532    544   
   533    545     <p class=todo>
   534    546       The concept of a logical database and its contents should be defined
   535    547       properly in some requirements document so that it can be referenced from
   536    548       here and other places. The definition will be something like the list of
   537    549       bullet points above.
   538    550   
          551  +  <p>
          552  +    Many of the numbered requirements in the following sub-sections describe 
          553  +    the relationship between the contents of the logical database, as itemized
          554  +    above, and the contents of the serialized database image. Others describe
          555  +    the relationships between various database image substructures, invariants
          556  +    that are true for all well-formed database images.
          557  +
   539    558     <p>
   540    559       A well-formed SQLite database image is defined as an image for which
   541    560       all of the statements itemized as requirements within this section
   542    561       are true. <span class=todo>mention the requirements numbering scheme
   543    562       here.</span> A software system that wishes to interoperate with other
   544    563       systems using the SQLite database image format should only ever
   545    564       output well-formed SQLite databases. In the case of SQLite itself,
................................................................................
   553    572         for a single key or range of keys. As implemented in SQLite, each
   554    573         entry may be associated with a blob of data that is not part of the
   555    574         key. For the canonical introduction to the B-Tree and its variants, 
   556    575         refer to reference <cite>ref_comer_btree</cite>. The B-Tree 
   557    576         implementation in SQLite also adopts some of the enhancements 
   558    577         suggested in <cite>ref_knuth_btree</cite>.
   559    578       <p>
   560         -      An SQLite database file contains one or more B-Tree structures. Each
          579  +      An SQLite database image contains one or more B-Tree structures. Each
   561    580         B-Tree structure stores the data for a single database table or 
   562    581         index. Hence each database file contains a single B-Tree to store
   563    582         the contents of the <i>sqlite_master</i> table, and one B-Tree
   564    583         for each database table or index created by the user. If the database
   565    584         uses auto-increment integer primary keys, then the database file
   566    585         also contains a B-Tree to store the contents of the automatically 
   567    586         created <i>sqlite_sequence</i> table.
................................................................................
   620    639         The following sections and sub-sections describe precisely the format
   621    640         used to serialize the B-Tree structures within an SQLite database image.
   622    641   
   623    642     [h2 "Global Structure"]
   624    643   
   625    644       [h3 "Database Header" "database_header"]
   626    645         <p>
   627         -        An SQLite database image begins with a 100-byte database header. The header
   628         -        file consists of a well known 16-byte sequence followed by a series of
   629         -        1, 2 and 4 byte unsigned integers. All integers in the file header (as
          646  +        An SQLite database image begins with a 100-byte database header. The database 
          647  +        header consists of a well known 16-byte sequence followed by a series of
          648  +        1, 2 and 4 byte unsigned integers. All integers in the database header (as
   630    649           well as the rest of the database file) are stored in big-endian format.
   631    650           
   632    651         <p>
   633    652           The well known 16-byte sequence that begins every SQLite database file
   634    653           is:
   635    654         <pre>
   636    655             0x53 0x51 0x4c 0x69 0x74 0x65 0x20 0x66 0x6f 0x72 0x6d 0x61 0x74 0x20 0x33 0x00</pre>
................................................................................
   639    658           Interpreted as UTF-8 encoded text, this byte sequence corresponds 
   640    659           to the string "SQLite format 3" followed by a nul-terminator byte.
   641    660   
   642    661             [fileformat_import_requirement2 H30030]
   643    662   
   644    663         <p>
   645    664           The 1, 2 and 4 byte unsigned integers that make up the rest of the
   646         -        database file header are described in the following table.
          665  +        database header are described in the following table.
   647    666   
   648    667         [Table]
   649         -        [Tr]<th>Byte Range <th>Byte Size <th width=100%>Description
          668  +        [Tr]<th>Byte Range <th>Byte Size <th width=100%>Description <th>Reqs
   650    669   	[Tr]<td>16..17 <td>2<td>
   651    670               Database page size in bytes. See section 
   652    671               <cite>pages_and_page_types</cite> for details.
          672  +            <td>H30190
   653    673   
   654    674           [Tr]<td>18     <td>1<td>
   655    675               <p style="margin-top:0">
   656    676               File-format "write version". Currently, this field
   657    677               is always set to 1. If a value greater than 1 is read by SQLite,
   658    678               then the library will only open the file for read-only access.
   659    679   
................................................................................
   661    681               This field and the next one are intended to be used for 
   662    682               forwards compatibility, should the need ever arise. If in the
   663    683               future a version of SQLite is created that uses a file format
   664    684               that may be safely read but not written by older versions of
   665    685               SQLite, then this field will be set to a value greater than 1
   666    686               to prevent older SQLite versions from writing to a file that
   667    687               uses the new format. 
          688  +            <td>H30040
   668    689   
   669    690           [Tr]<td>19     <td>1<td>
   670    691               <p style="margin-top:0">
   671    692                File-format "read version". Currently, this 
   672    693               field is always set to 1. If a value greater than 1 is read 
   673    694               by SQLite, then the library will refuse to open the database 
   674    695   
................................................................................
   675    696               <p style="margin-bottom:0">
   676    697               Like the "write version" described above, this field exists
   677    698               to facilitate some degree of forwards compatibility, in case
   678    699               it is ever required. If a version of SQLite created in the 
   679    700               future uses a file format that may not be safely read by older
   680    701               SQLite versions, then this field will be set to a value greater
   681    702               than 1.
          703  +            <td>H30040
   682    704   
   683    705           [Tr]<td>20     <td>1<td>
   684    706               Number of bytes of unused space at the end of each database
   685    707               page. Usually this field is set to 0. If it is non-zero, then 
   686    708               it contains the number of bytes that are left unused at the
   687    709               end of every database page (see section
   688    710               <cite>pages_and_page_types</cite> for a description of a
   689    711               database page).
          712  +            <td>H30040
   690    713   
   691    714           [Tr]<td>21     <td>1<td>
   692    715                Maximum fraction of an index tree page to use for 
   693    716               embedded content. This value is used to determine the maximum
   694    717               size of a B-Tree cell to store as embedded content on a
   695    718               page that is part of an index B-Tree. Refer to section 
   696    719               <cite>index_btree_cell_format</cite> for details.
          720  +            <td>H30040
   697    721   
   698    722           [Tr]<td>22     <td>1<td>
   699    723               Minimum fraction of an index B-Tree page to use for
   700    724               embedded content when an entry uses one or more overflow pages.
   701    725               This value is used to determine the portion of a B-Tree cell 
   702    726               that requires one or more overflow pages to store as embedded
   703    727               content on a page that is part of an index B-Tree. Refer to
   704    728               section <cite>index_btree_cell_format</cite> for details.
          729  +            <td>H30040
   705    730   
   706    731           [Tr]<td>23     <td>1<td>
   707    732               Minimum fraction of an table B-Tree leaf page to use for
   708    733               embedded content when an entry uses one or more overflow pages.
   709    734               This value is used to determine the portion of a B-Tree cell 
   710    735               that requires one or more overflow pages to store as embedded
   711    736               content on a page that is a leaf of a table B-Tree. Refer to
   712    737               section <cite>table_btree_cell_format</cite> for details.
          738  +            <td>H30040
   713    739   
   714    740           [Tr]<td>24..27 <td>4<td>
   715    741               <p style="margin-top:0">
   716    742               The file change counter. Each time a database transaction is
   717    743               committed, the value of the 32-bit unsigned integer stored in
   718    744               this field is incremented.
   719    745               <p style="margin-bottom:0">
................................................................................
   722    748               a portion of the file cached in memory. However, since the file
   723    749               is unlocked, another process may use SQLite to modify the 
   724    750               contents of the file, invalidating the internal cache of the
   725    751               first process. When the file is relocked, the first process can
   726    752               check if the value of the file change counter has been modified
   727    753               since the file was unlocked. If it has not, then the internal
   728    754               cache may be assumed to be valid and may be reused.
          755  +            <td>H33040
   729    756   
   730    757           [Tr]<td>32..35 <td>4<td>
   731    758               Page number of first freelist trunk page. 
   732    759               For more details, refer to section <cite>free_page_list</cite>.
          760  +            <td>H31320
   733    761   
   734    762           [Tr]<td>36..39 <td>4<td>
   735    763               Number of free pages in the database file.
   736    764               For more details, refer to section <cite>free_page_list</cite>.
          765  +            <td>H31310
   737    766   
   738    767           [Tr]<td>40..43 <td>4<td>
   739    768               The schema version. Each time the database schema is modified (by
   740    769               creating or deleting a database table, index, trigger or view)
   741    770               the value of the 32-bit unsigned integer stored in this field
   742    771               is incremented.
          772  +            <td>H33050
   743    773   
   744    774           [Tr]<td>44..47 <td>4<td>
   745    775               <p style="margin-top:0">
   746    776               Schema layer file-format. This value is similar to the
   747    777               "read-version" and "write-version" fields at offsets 18 and 19
   748         -            of the database file header. If SQLite encounters a database
          778  +            of the database header. If SQLite encounters a database
   749    779               with a schema layer file-format value greater than the file-format
   750    780               that it understands (currently 4), then SQLite will refuse to
   751    781               access the database.
   752    782               <p>
   753    783               Usually, this value is set to 1. However, if any of the following
   754    784               file-format features are used, then the schema layer file-format
   755    785               must be set to the corresponding value or greater:
................................................................................
   759    789                 <li> Implicit default (non-NULL) values at the end of table
   760    790                      records (see section <cite>table_btree_content</cite>).
   761    791                 <li> Descending indexes (see section
   762    792                      <cite>index_btree_compare_func</cite>) and Boolean values
   763    793                      in database records (see section <cite>record_format</cite>,
   764    794                      serial types 8 and 9).
   765    795               </ol>
   766         -            
          796  +            <td>H30120
   767    797   
   768    798           [Tr]<td>48..51 <td>4<td>
   769    799               Default pager cache size. This field is used by SQLite to store
   770    800               the recommended pager cache size to use for the database.
          801  +            <td>H30130
   771    802   
   772    803           [Tr]<td>52..55 <td>4<td>
   773    804               For auto-vacuum capable databases, the numerically largest 
   774    805               root-page number in the database. Since page 1 is always the
   775    806               root-page of the schema table (section <cite>schema_table</cite>),
   776    807               this value is always non-zero for auto-vacuum databases. For
   777    808               non-auto-vacuum databases, this value is always zero.
          809  +            <td>H30140, H30141
   778    810   
   779    811           [Tr]<td>56..59 <td>4<td>
   780    812               (constant) Database text encoding. A value of 1 means all 
   781    813               text values are stored using UTF-8 encoding. 2 indicates
   782    814               little-endian UTF-16 text. A value of 3 means that the database
   783    815               contains big-endian UTF-16 text.  
          816  +            <td>H30150
   784    817   
   785    818           [Tr]<td>60..63 <td>4<td>
   786    819               The user-cookie value. A 32-bit integer value available to the
   787    820               user for read/write access.
          821  +            <td>H30160
   788    822   
   789    823           [Tr]<td>64..67 <td>4<td>
   790    824               The incremental-vacuum flag. In non-auto-vacuum databases this
   791    825               value is always zero. In auto-vacuum databases, this field is
   792    826               set to 1 if "incremental vacuum" mode is enabled. If incremental
   793    827               vacuum mode is not enabled, then the database file is reorganized
   794    828               so that it contains no free pages (section
   795    829               <cite>free_page_list</cite>) at the end of each database
   796    830               transaction. If incremental vacuum mode is enabled, then the
   797    831               reorganization is not performed until explicitly requested
   798    832               by the user.
          833  +            <td>H30171
   799    834   
   800    835         </table>
   801    836   
   802    837         <p>
   803    838           The four byte block beginning at offset 28 is unused. As is the
   804    839           32 byte block beginning at offset 68.
   805    840         </p>
................................................................................
   832    867         </ul>
   833    868   
   834    869         <p>
   835    870           The following requirement encompasses all of the above.
   836    871   
   837    872             [fileformat_import_requirement2 H30040]
   838    873   
          874  +      <p>
          875  +        Section <cite>database_file_format</cite> identifies six persistent
          876  +        user-visible properties of an SQLite database. The following 
          877  +        requirements describe the way in which these properties are stored.
   839    878   
   840         -      <p>
   841         -        Following the 16 byte magic string in the file header is the
   842         -        <i>page size</i>, a 2-byte field. See section
   843         -        <cite>pages_and_page_types</cite> for details.
   844         -
   845         -          [fileformat_import_requirement2 H30100]
   846         -
   847         -      <p>
   848         -        Following the <i>file change counter</i> in the database header are
   849         -        two 4-byte fields related to the database file <i>free page list</i>.
   850         -        See section <cite>free_page_list</cite> for details.
   851         -
   852         -          [fileformat_import_requirement2 H30110]
   853         -          [fileformat_import_requirement2 H30120]
   854         -          [fileformat_import_requirement2 H30130]
          879  +          [fileformat_import_requirement2 H30190]
          880  +          [fileformat_import_requirement2 H30191]
          881  +          [fileformat_import_requirement2 H30150]
   855    882             [fileformat_import_requirement2 H30140]
   856         -          [fileformat_import_requirement2 H30150]
          883  +          [fileformat_import_requirement2 H30141]
   857    884             [fileformat_import_requirement2 H30160]
   858    885             [fileformat_import_requirement2 H30170]
   859         -          [fileformat_import_requirement2 H30180]
          886  +          [fileformat_import_requirement2 H30171]
          887  +          [fileformat_import_requirement2 H30130]
          888  +
          889  +      <p>
          890  +        The following requirement describes the valid range of values for the
          891  +        schema layer file format field.
          892  +
          893  +          [fileformat_import_requirement2 H30120]
   860    894   
   861    895       [h3 "Pages and Page Types" "pages_and_page_types"]
   862    896         <p>
   863    897           The entire database file is divided into pages, each page consisting
   864    898           of <i>page-size</i> bytes, where <i>page-size</i> is the 2-byte 
   865         -        integer value stored at offset 16 of the file header (see above).
          899  +        integer value stored at offset 16 of the database header (see above).
   866    900           The <i>page-size</i> is always a power of two between 512 
   867    901           (2<sup>9</sup>) and 32768 (2<sup>15</sup>). SQLite database files
   868    902           always consist of an exact number of pages.
   869    903         <p>
   870    904           Pages are numbered beginning from 1, not 0. Page 1 consists of
   871         -        the first <i>page-size</i> bytes of the database file. The file header
          905  +        the first <i>page-size</i> bytes of the database file. The database header
   872    906           described in the previous section consumes the first 100 bytes of page
   873    907           1.
   874    908         <p>
   875    909           Each page of the database file is one of the following:
   876    910         <ul>
   877    911           <li><b>A B-Tree page</b>. B-Tree pages are part of the tree 
   878    912               structures used to store database tables and indexes.
   879    913           <li><b>An overflow page</b>. Overflow pages are used by particularly
   880    914               large database records that do not fit on a single B-Tree page.
   881    915           <li><b>A free page</b>. Free pages are pages within the database file
   882    916               that are not being used to store meaningful data.
   883    917           <li><b>A "pointer-map" page</b>. In auto-vacuum capable databases
   884    918               (databases for which the 4 byte big-endian integer stored at
   885         -            byte offset 52 of the file header is non-zero) some pages are
          919  +            byte offset 52 of the database header is non-zero) some pages are
   886    920               permanently designated "pointer-map" pages. See section 
   887    921               <cite>pointer_map_pages</cite> for details.
   888    922           <li><b>The locking page</b>. The database page that starts at
   889    923               byte offset 2<sup>30</sup>, if it is large enough to contain
   890    924               such a page, is always left unused.
   891    925         </ul>
   892    926   
   893         -          [fileformat_import_requirement2 H30190]
   894    927             [fileformat_import_requirement2 H30200]
   895    928             [fileformat_import_requirement2 H30210]
   896    929             [fileformat_import_requirement2 H30220]
   897    930           
   898    931   
   899    932       [h3 "The Schema Table" schema_table]
   900    933         <p>
................................................................................
   973   1006           be used as pointer-map pages (see section
   974   1007           <cite>pointer_map_pages</cite>).
   975   1008         <p>
   976   1009           As noted in section <cite>database_header</cite>, in an auto-vacuum
   977   1010           database the page number of the page immediately following the
   978   1011           final root page in the contiguous set of root pages is stored
   979   1012           as a 4 byte big-endian integer at byte offset 52 of the database
   980         -        file header. Unless that page is itself a pointer-map page, in which
         1013  +        header. Unless that page is itself a pointer-map page, in which
   981   1014           case the page number of the page following it is stored instead.
   982   1015   
   983   1016         <p>
   984   1017           For example, if the schema of a logical database is created using
   985   1018           the following SQL statements:
   986   1019         <pre>
   987   1020             CREATE TABLE abc(a, b, c);
................................................................................
  1188   1221               <td>An SQL real value (type SQLITE_FLOAT), stored as an
  1189   1222                   8-byte IEEE floating point value.
  1190   1223           [Tr]<td>8
  1191   1224               <td>The literal SQL integer 0 (type SQLITE_INTEGER). The value 
  1192   1225                   consumes zero bytes of space in the record's data area.
  1193   1226                   Values of this type are only present in databases with
  1194   1227                   a schema file format (the 32-bit integer at byte offset 44
  1195         -                of the database file header) value of 4 or greater.
         1228  +                of the database header) value of 4 or greater.
  1196   1229   
  1197   1230           [Tr]<td>9
  1198   1231               <td>The literal SQL integer 1 (type SQLITE_INTEGER). The value
  1199   1232                   consumes zero bytes of space in the record's data area.
  1200   1233                   Values of this type are only present in databases with
  1201   1234                   a schema file format (the 32-bit integer at byte offset 44
  1202         -                of the database file header) value of 4 or greater.
         1235  +                of the database header) value of 4 or greater.
  1203   1236   
  1204   1237           [Tr]<td style="white-space:nowrap"><i>bytes</i> * 2 + 12
  1205   1238               <td>Even values greater than 12 are used to signify a blob of
  1206   1239                   data (type SQLITE_BLOB) (<i>n</i>-12)/2 bytes in length, where
  1207   1240                   <i>n</i> is the integer value stored in the record header.
  1208   1241                   
  1209   1242           [Tr]<td style="white-space:nowrap"><i>bytes</i> * 2 + 13
................................................................................
  1223   1256           in the records data area. If the corresponding integer type value
  1224   1257           in the record header is 0 (NULL), 8 (integer value 0) or 9 (integer
  1225   1258           value 1), then the blob of data is zero bytes in length. Otherwise,
  1226   1259           the length of the data field is as described in the table above.
  1227   1260         <p>
  1228   1261           The data field associated with a string value contains the string
  1229   1262           encoded using the database encoding, as defined in the database
  1230         -        file header (see section <cite>database_header</cite>). No 
         1263  +        header (see section <cite>database_header</cite>). No 
  1231   1264           nul-terminator character is stored in the database.
  1232   1265   
  1233   1266             [fileformat_import_requirement2 H30560]
  1234   1267             [fileformat_import_requirement2 H30570]
  1235   1268             [fileformat_import_requirement2 H30580]
  1236   1269             [fileformat_import_requirement2 H30590]
  1237   1270             [fileformat_import_requirement2 H30600]
................................................................................
  1519   1552             the cell if it consists of less than:
  1520   1553           <pre>
  1521   1554               <i>max-local</i> := (<i>usable-size</i> - 12) * <i>max-embedded-fraction</i> / 255 - 23
  1522   1555   </pre>
  1523   1556           <p>
  1524   1557             bytes. In the formula above, <i>usable-size</i> is the page-size
  1525   1558             in bytes less the number of unused bytes left at the end of every
  1526         -          page (as read from byte offset 20 of the file header), and
         1559  +          page (as read from byte offset 20 of the database header), and
  1527   1560             <i>max-embedded-fraction</i> is the value read from byte offset 
  1528         -          21 of the file header.
         1561  +          21 of the database header.
  1529   1562           [Figure indexshortrecord.gif figure_indexshortrecord "Small Record Index B-Tree Cell"]
  1530   1563           <p>
  1531   1564             If the cell record is larger than the maximum size identified by
  1532   1565             the formula above, then only the first part of the record is stored
  1533   1566             within the cell. The remainder is stored in an overflow-chain (see
  1534   1567             section <cite>overflow_page_chains</cite> for details). Following 
  1535   1568             the part of the record stored within the cell is the page number 
................................................................................
  1544   1577               <i>local-size</i> := <i>min-local</i> + (<i>record-size</i> - <i>min-local</i>) % (<i>usable-size</i> - 4)
  1545   1578               if( <i>local-size</i> &gt; <i>max-local</i> )
  1546   1579                   <i>local-size</i> := <i>min-local</i>
  1547   1580   </pre>
  1548   1581           <p>
  1549   1582             In the formula above, <i>usable-size</i> is the page-size
  1550   1583             in bytes less the number of unused bytes left at the end of every
  1551         -          page (as read from byte offset 20 of the file header), and
         1584  +          page (as read from byte offset 20 of the database header), and
  1552   1585             <i>max-embedded-fraction</i> and <i>min-embedded-fraction</i> are
  1553         -          the values read from byte offsets 21 and 22 of the file header,
         1586  +          the values read from byte offsets 21 and 22 of the database header,
  1554   1587             respectively.
  1555   1588           [Figure indexlongrecord.gif figure_indexlongrecord "Large Record Index B-Tree Cell"]
  1556   1589   
  1557   1590             [fileformat_import_requirement2 H30970]
  1558   1591             [fileformat_import_requirement2 H30980]
  1559   1592             [fileformat_import_requirement2 H30990]
  1560   1593             [fileformat_import_requirement2 H31000]
................................................................................
  1562   1595   
  1563   1596         <p>
  1564   1597           Requirements H31010 and H30990 are similar to the algorithms 
  1565   1598           presented in the text above. However instead of 
  1566   1599           <i>min-embedded-fraction</i> and <i>max-embedded-fraction</i> the
  1567   1600           requirements use the constant values 32 and 64, as well-formed 
  1568   1601           database files are required by H30080 and H30070 to store these 
  1569         -        values in the relevant database file header fields.
         1602  +        values in the relevant database database header fields.
  1570   1603   
  1571   1604       [h3 "Table B-Trees" table_btrees]
  1572   1605         <p>
  1573   1606           As noted in section <cite>fileformat_overview</cite>, table B-Trees
  1574   1607           store a set of unique 64-bit signed integer keys. Associated with
  1575   1608           each key is a database record. As with index B-Trees, the database
  1576   1609           file pages that make up a table B-Tree are organized into a tree
................................................................................
  1671   1704             <li>The first byte of the page-header, the "flags" field, is set to 
  1672   1705                 0x05 for internal tree node pages, and 0x0D for leaf pages.
  1673   1706             <li>The content and format of the B-Tree cells is different. See
  1674   1707                 section <cite>table_btree_cell_format</cite> for details.
  1675   1708             <li>The format of page 1 is the same as any other table B-Tree,
  1676   1709                 except that 100 bytes less than usual is available for content.
  1677   1710                 The first 100 bytes of page 1 is consumed by the database
  1678         -              file header.
         1711  +              header.
  1679   1712           </ul>
  1680   1713   
  1681   1714             [fileformat_import_requirement2 H31130]
  1682   1715             [fileformat_import_requirement2 H31140]
  1683   1716           
  1684   1717         <p>
  1685   1718           Most of the requirements specified in section 
................................................................................
  1710   1743             sufficiently small is defined as less than or equal to:
  1711   1744           <pre>
  1712   1745             max-local := <i>usable-size</i> - 35
  1713   1746   </pre>
  1714   1747           <p>
  1715   1748             bytes. Where <i>usable-size</i> is defined as the page-size
  1716   1749             in bytes less the number of unused bytes left at the end of every
  1717         -          page (as read from byte offset 20 of the file header). 
         1750  +          page (as read from byte offset 20 of the database header). 
  1718   1751             This scenario, where the entire record is
  1719   1752             stored within the B-Tree cell, is depicted in figure
  1720   1753             <cite>figure_tableshortrecord</cite>.
  1721   1754           [Figure tableshortrecord.gif figure_tableshortrecord "Table B-Tree Small Record Leaf Node Cell"]
  1722   1755   
  1723   1756           <p>
  1724   1757             If the record is too large to be stored entirely within the B-Tree
................................................................................
  1735   1768               <i>max-local</i> := <i>usable-size</i> - 35
  1736   1769               <i>local-size</i> := <i>min-local</i> + (<i>record-size</i> - <i>min-local</i>) % (<i>usable-size</i> - 4)
  1737   1770               if( <i>local-size</i> &gt; <i>max-local</i> )
  1738   1771                   <i>local-size</i> := <i>min-local</i>
  1739   1772   </pre>
  1740   1773           <p>
  1741   1774             In this case, <i>min-embedded-fraction</i> is the value read from
  1742         -          byte offset 22 of the file header. The layout of the cell in this
         1775  +          byte offset 22 of the database header. The layout of the cell in this
  1743   1776             case, when an overflow-chain is required, is shown in figure
  1744   1777             <cite>figure_tablelongrecord</cite>.
  1745   1778   
  1746   1779           [Figure tablelongrecord.gif figure_tablelongrecord "Table B-Tree Large Record Leaf Node Cell"]
  1747   1780   
  1748   1781           <p>
  1749   1782             If the leaf page is page 1, then the value of <i>usable-size</i> is
................................................................................
  1796   1829           page is:
  1797   1830         <pre>
  1798   1831           <i>available-space</i> := <i>usable-size</i> - 4
  1799   1832   </pre>
  1800   1833         <p>
  1801   1834           Where <i>usable-size</i> is defined as the page-size in bytes less the
  1802   1835           number of unused bytes left at the end of every page (as read from 
  1803         -        byte offset 20 of the file header).
         1836  +        byte offset 20 of the database header).
  1804   1837         <p>
  1805   1838           Each overflow page except for the last one in the linked list 
  1806   1839           contains <i>available-space</i> bytes of record data. The last
  1807   1840           page in the list contains the remaining data, starting at byte
  1808   1841           offset 4. The value of the "next page" field on the last page
  1809   1842           in an overflow chain is undefined.
  1810   1843   
................................................................................
  1840   1873         Each free-list trunk page contains up to:
  1841   1874       <pre>
  1842   1875           <i>max-leaf-pointers</i> := (<i>usable-size</i> - 8) / 4
  1843   1876   </pre>
  1844   1877       <p>
  1845   1878         pointers, where <i>usable-size</i> is defined as the page-size in bytes
  1846   1879         less the number of unused bytes left at the end of every page (as read
  1847         -      from byte offset 20 of the file header).
         1880  +      from byte offset 20 of the database header).
  1848   1881   
  1849   1882         [Figure freelistpage.gif figure_freelistpage "Free List Trunk Page Format"]
  1850   1883       <p>
  1851   1884         All trunk pages in the free-list except for the first contain the 
  1852   1885         maximum possible number of references to leaf pages. <span class=todo>Is this actually true in an auto-vacuum capable database?</span> The page number
  1853   1886         of the first page in the linked list of free-list trunk pages is 
  1854   1887         stored as a 4-byte big-endian unsigned integer at offset 32 of the
  1855         -      file header (section <cite>database_header</cite>).
         1888  +      database header (section <cite>database_header</cite>).
  1856   1889   
  1857   1890             [fileformat_import_requirement2 H31240]
  1858   1891             [fileformat_import_requirement2 H31250]
  1859   1892             [fileformat_import_requirement2 H31260]
  1860   1893             [fileformat_import_requirement2 H31270]
  1861   1894             [fileformat_import_requirement2 H31280]
  1862   1895             [fileformat_import_requirement2 H31290]
  1863   1896             [fileformat_import_requirement2 H31300]
  1864   1897   
  1865   1898       <p>The following statements govern the two 4-byte big-endian integers
  1866   1899          associated with the <i>free page list</i> structure in the database
  1867         -       file header.
         1900  +       header.
  1868   1901   
  1869   1902             [fileformat_import_requirement2 H31310]
  1870   1903             [fileformat_import_requirement2 H31320]
  1871   1904     
  1872   1905   
  1873   1906     [h2 "Pointer Map Pages" pointer_map_pages]
  1874   1907       <p>
................................................................................
  1918   1951         pointer-map page may therefore contain:
  1919   1952       <pre>
  1920   1953           <i>num-entries</i> := <i>usable-size</i> / 5
  1921   1954   </pre>
  1922   1955       <p>
  1923   1956         entries, where <i>usable-size</i> is defined as the page-size in bytes
  1924   1957         less the number of unused bytes left at the end of every page (as read
  1925         -      from byte offset 20 of the file header).
         1958  +      from byte offset 20 of the database header).
  1926   1959       <p>
  1927   1960         Assuming the database is auto-vacuum capable, page 2 is always a 
  1928   1961         pointer map page. It contains the pointer map lookup table entries for
  1929   1962         pages 3 through (2 + <i>num-entries</i>), inclusive. The first 5 bytes
  1930   1963         of page 2 contain the pointer map lookup table entry for page 3. Bytes
  1931   1964         5 through 9, inclusive, contain the pointer map lookup table entry
  1932   1965         for page 4, and so on.
................................................................................
  2322   2355   
  2323   2356   [h2 "Reading an SQLite Database" reading_from_files]
  2324   2357   
  2325   2358     <p>
  2326   2359       As described in section <cite>pages_and_page_types</cite> of this document,
  2327   2360       an SQLite database image is a set of contiguously numbered fixed size 
  2328   2361       pages. The numbering starts at 1, not 0. Page 1 contains the 
  2329         -    <i>database file header</i> and the root page of the <i>schema table</i>, 
         2362  +    database header and the root page of the <i>schema table</i>, 
  2330   2363       and all other pages within the database image are somehow referenced
  2331   2364       by number, either directly or indirectly, from page 1, either directly 
  2332   2365       or indirectly. In order to be able to read the database image from within
  2333   2366       the file-system, a database reader needs to be able to ascertain:
  2334   2367   
  2335   2368     <ol>
  2336   2369       <li> The <i>page-size</i> used by the database image,

Changes to req/hlr30000.txt.

    17     17   single nul-terminator byte.
    18     18   
    19     19   HLR H30040
    20     20   The 6 bytes beginning at byte offset 18 of a well-formed database 
    21     21   image shall contain the values 0x01, 0x01, 0x00, 0x40, 0x20 and 
    22     22   0x20, respectively.
    23     23   
    24         -
    25         -HLR H30100
    26         -The 4 byte block starting at byte offset 24 of a well-formed
    27         -database file contains the <i>file change counter</i> formatted
    28         -as a 4-byte big-endian integer.
    29         -
    30         -HLR H30110
    31         -The 4 byte block starting at byte offset 40 of a well-formed
    32         -database file contains the <i>schema version</i> formatted
    33         -as a 4-byte big-endian integer.
    34         -
    35     24   HLR H30120
    36         -The 4 byte block starting at byte offset 44 of a well-formed
    37         -database file, the <i>schema layer file format</i>, contains a
    38         -big-endian integer value between 1 and 4, inclusive.
           25  +The 4-byte big-endian signed integer field at byte offset 44 of a 
           26  +well-formed database image, the schema layer file format field, 
           27  +shall be set to an integer value between 1 and 4, inclusive. 
    39     28   
    40     29   HLR H30130
    41         -The 4 byte block starting at byte offset 48 of a well-formed
    42         -database file contains the <i>default pager cache size</i> formatted
    43         -as a 4-byte big-endian integer.
           30  +The 4-byte big-endian unsigned integer field at byte offset 48 of a 
           31  +well-formed database image shall be set to the value of the 
           32  +database default page-cache size.
    44     33   
    45     34   HLR H30140
    46         -The 4 byte block starting at byte offset 52 of a well-formed
    47         -database file contains the <i>auto-vacuum last root-page</i>
    48         -formatted as a 4-byte big-endian integer. If this value is non-zero,
    49         -the database is said to be an <i>auto-vacuum database</i>.
           35  +If the database is not an auto-vacuum capable database, then the 4 byte 
           36  +big-endian unsigned integer field at byte offset 52 of a well-formed 
           37  +database image shall contain the value 0.
           38  +
           39  +HLR H30141
           40  +If the database is an auto-vacuum capable database, then the 4 byte 
           41  +big-endian unsigned integer field at byte offset 52 of a well-formed 
           42  +database image shall contain the numerically largest root-page number 
           43  +of any table or index B-Tree within the database image.
    50     44   
    51     45   HLR H30150
    52         -The 4 byte block starting at byte offset 56 of a well-formed
    53         -database file, the <i>text encoding</i> contains a big-endian integer
    54         -value between 1 and 3, inclusive.
           46  +The 4 byte big-endian unsigned integer field at byte offset 56 of a 
           47  +well-formed database image shall be set to 1 if the database text-encoding
           48  +is UTF-8, 2 if the database text-encoding is little-endian UTF-16, and 3
           49  +if the database text-encoding is big-endian UTF-16.
    55     50   
    56     51   HLR H30160
    57         -The 4 byte block starting at byte offset 60 of a well-formed
    58         -database file contains the <i>user cookie</i> formatted
    59         -as a 4-byte big-endian integer.
           52  +The 4-byte big-endian unsigned integer field at byte offset 60 of a 
           53  +well-formed database image shall be set to the value of the 
           54  +database user-cookie.
           55  +
           56  +HLR H30190
           57  +The 2-byte big-endian unsigned integer field at byte offset 16 of a
           58  +well-formed database image shall be set to the value of the database
           59  +page-size.
           60  +
           61  +HLR H30191
           62  +The page-size of an SQLite database in bytes shall be an integer power 
           63  +of 2 between 512 and 32768, inclusive.
    60     64   
    61     65   HLR H30170
    62         -The 4 byte block starting at byte offset 64 of a well-formed
    63         -database file, the <i>incremental vaccum flag</i> contains a big-endian
    64         -integer value between 0 and 1, inclusive.
           66  +The 4-byte big-endian unsigned integer field at byte offset 64 of a
           67  +well-formed database image shall be set to the value of the database
           68  +incremental-vacuum flag.
           69  +
           70  +HLR H30171
           71  +The value of the incremental-vacuum flag of an SQLite database shall be
           72  +either 0 or 1.
    65     73   
    66     74   HLR H30180
    67     75   In a well-formed non-autovacuum database (one with a zero stored
    68     76   in the 4-byte big-endian integer value beginning at byte offset
    69     77   52 of the database file header, the incremental vacuum flag is
    70     78   set to 0.
    71     79   
    72         -HLR H30190
    73         -The <i>database page size</i> of a well-formed database, stored as a
    74         -2-byte big-endian unsigned integer at byte offset 16 of the file,
    75         -shall be an integer power of 2 between 512 and 32768, inclusive.
    76     80   
    77     81   HLR H30200
    78         -The size of a <i>well formed database file</i> shall be an integer
    79         -multiple of the <i>database page size</i>.
           82  +The size of a well formed database image shall be an integer
           83  +multiple of the database page size.
    80     84   
    81     85   HLR H30210
    82         -Each page of a <i>well formed database file</i> is exactly one of a
           86  +Each page of a well formed database image shall be exactly one of a
    83     87   <i>B-Tree page</i>, an <i>overflow page</i>, a <i>free page</i>, a
    84     88   <i>pointer-map page</i> or the <i>locking page</i>.
    85     89   
    86     90   HLR H30220
    87     91   The database page that starts at byte offset 2<sup>30</sup>, the
    88         -<i>locking page</i>, is never used for any purpose.
           92  +<i>locking page</i>, shall never be used for any purpose.
    89     93   
    90     94   HLR H30230
    91     95   In a <i>well-formed database file</i>, the portion of the first
    92     96   database page not consumed by the database file-header (all but the
    93     97   first 100 bytes) contains the root node of a table B-Tree,
    94     98   the <i>schema table</i>.
    95     99   
................................................................................
  1089   1093   If another database client holds either a RESERVED or PENDING lock on the
  1090   1094   database file-system representation, then any journal file that exists within
  1091   1095   the file system shall be considered invalid.
  1092   1096   
  1093   1097   
  1094   1098   HLR H33040
  1095   1099   A database writer shall increment the value of the database header change
  1096         -counter field (H30100) as part of the first database image modification 
         1100  +counter field, a 4-byte big-endian unsigned integer field stored at byte offset 24
         1101  +of the database header, as part of the first database image modification 
  1097   1102   that it performs after obtaining an EXCLUSIVE lock.
  1098   1103   
  1099   1104   HLR H33050
  1100   1105   A database writer shall increment the value of the database schema version 
  1101         -field (H30110) as part of the first database image modification that includes
  1102         -a schema change that it performs after obtaining an EXCLUSIVE lock.
         1106  +field, a 4-byte big-endian unsigned integer field stored at byte offset 40
         1107  +of the database header, as part of the first database image modification that
         1108  +includes a schema change that it performs after obtaining an EXCLUSIVE lock.
  1103   1109   
  1104   1110   HLR H33070
  1105   1111   If a database writer is required by either H33050 or H33040 to increment a
  1106   1112   database header field, and that header field already contains the maximum
  1107   1113   value possible (0xFFFFFFFF, or 4294967295 for 32-bit unsigned integer 
  1108   1114   fields), "incrementing" the field shall be interpreted to mean setting it to 
  1109   1115   zero.
  1110   1116   
  1111   1117