Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Modify fileio.in to add requirements to the database as it is parsed. |
|---|---|
| Downloads: | Tarball | ZIP archive |
| Timelines: | family | ancestors | descendants | both | trunk |
| Files: | files | file ages | folders |
| SHA1: |
a9b591daa0b48eebe40473f0758799a3 |
| User & Date: | dan 2008-07-22 18:55:23.000 |
Context
|
2008-07-23
| ||
| 15:38 | Updates to system requirements. (check-in: a1807b9496 user: drh tags: trunk) | |
|
2008-07-22
| ||
| 18:55 | Modify fileio.in to add requirements to the database as it is parsed. (check-in: a9b591daa0 user: dan tags: trunk) | |
| 18:46 | Added the requirements derivation matrix report. (check-in: fd2d3b3be4 user: drh tags: trunk) | |
Changes
Changes to pages/fileio.in.
1 2 3 4 5 6 7 | <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd"> <html> <head> <link type="text/css" rel="stylesheet" href="images/fileformat/rtdocs.css"> <script type="text/javascript" src=images/fileformat/rtdocs.js></script> </head> | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 |
<tcl>
proc process {text} {
set zOut ""
set isReq 0
foreach zLine [split $text "\n"] {
switch -regexp $zLine {
{^ *REQ *[^ ][^ ]* *$} {
regexp { *REQ *([^ ]+) *} $zLine -> zRecid
append zOut "<p class=req id=$zRecid>"
set isReq 1
set zRecText ""
}
{^ *$} {
if {$isReq} {
hd_requirement $zRecid $zRecText
set isReq 0
append zOut </p>
}
}
default {
if {$isReq} {
if {[regexp {^ *\. *$} $zLine]} {set zLine ""}
append zRecText "$zLine\n"
}
append zOut "$zLine\n"
}
}
}
set zOut
}
hd_resolve [process {
<!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01//EN" "http://www.w3.org/TR/html4/strict.dtd">
<html>
<head>
<link type="text/css" rel="stylesheet" href="images/fileformat/rtdocs.css">
<script type="text/javascript" src=images/fileformat/rtdocs.js></script>
</head>
|
| ︙ | ︙ | |||
243 244 245 246 247 248 249 |
is to determine the <i>page-size</i> used by the database file.
Because it is not possible to be certain as to the <i>page-size</i>
without holding at least a <i>shared lock</i> on the database file
(because some other <i>database connection</i> might have changed it
since the <i>database file header</i> was read), the value read from the
<i>database file header</i> is known as the <i>expected page size</i>.
| | | | | | 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 |
is to determine the <i>page-size</i> used by the database file.
Because it is not possible to be certain as to the <i>page-size</i>
without holding at least a <i>shared lock</i> on the database file
(because some other <i>database connection</i> might have changed it
since the <i>database file header</i> was read), the value read from the
<i>database file header</i> is known as the <i>expected page size</i>.
REQ H21006
When a new <i>database connection</i> is required, SQLite shall attempt
to open a file-handle on the database file. If the attempt fails, then
no new <i>database connection</i> is created and an error returned.
REQ H21007
When a new <i>database connection</i> is required, after opening the
new file-handle, SQLite shall attempt to read the first 100 bytes
of the database file. If the attempt fails for any other reason than
that the opened file is less than 100 bytes in size, then
the file-handle is closed, no new <i>database connection</i> is created
and an error returned instead.
REQ H21008
If the <i>database file header</i> is successfully read from a newly
opened database file, the connections <i>expected page-size</i> shall
be set to the value stored in the <i>page-size field</i> of the
database header.
REQ H21009
If the <i>database file header</i> cannot be read from a newly opened
database file (because the file is less than 100 bytes in size), the
connections <i>expected page-size</i> shall be set to the compile time
value of the SQLITE_DEFAULT_PAGESIZE option.
<h2>Closing a Connection</h2>
|
| ︙ | ︙ | |||
318 319 320 321 322 323 324 |
pages are read one at a time. SQLite never reads partial pages and
always uses a single call to xRead() for each required page. After
reading the data for a database page, SQLite adds it to the connections
<i>page cache</i> so that it does not have to be read if required
again. Refer to section <cite>page_cache_algorithms</cite> for a
description of how this affects the IO performed by SQLite.
| | | | 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 |
pages are read one at a time. SQLite never reads partial pages and
always uses a single call to xRead() for each required page. After
reading the data for a database page, SQLite adds it to the connections
<i>page cache</i> so that it does not have to be read if required
again. Refer to section <cite>page_cache_algorithms</cite> for a
description of how this affects the IO performed by SQLite.
REQ H21001
Except for the read operation required by H21007 and those reads made
as part of opening a read-only transaction, SQLite shall only read
data from a <i>database connection</i> while the
<i>database connection</i> has an open read-only or read/write transaction.
<p>
In the above requirement, reading data from a database connection
includes retrieving data from the connections <i>page cache</i>.
REQ H21002
Aside from those read operations described by H21007 and H21XXX, SQLite
shall read data from the database in aligned blocks of <i>page-size</i>
bytes, where <i>page-size</i> is the database page size used by the
database file.
<h2 id=open_read_only_trans>Opening a Read-Only Transaction</h2>
<p>
|
| ︙ | ︙ | |||
381 382 383 384 385 386 387 |
enumerated above. If this happens, then the <i>shared-lock</i> is
released (if it was obtained) and an error returned to the user.
Step 2 of the procedure above is described in more detail in section
<cite>hot_journal_detection</cite>. Section <cite>cache_validation</cite>
describes the process identified by step 3 above. Further detail
on step 4 may be found in section <cite>read_page_one</cite>.
| | | | | | | | 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 |
enumerated above. If this happens, then the <i>shared-lock</i> is
released (if it was obtained) and an error returned to the user.
Step 2 of the procedure above is described in more detail in section
<cite>hot_journal_detection</cite>. Section <cite>cache_validation</cite>
describes the process identified by step 3 above. Further detail
on step 4 may be found in section <cite>read_page_one</cite>.
REQ H21010
When required to open a <i>read-only transaction</i> using a
<i>database connection</i>, SQLite shall first attempt to obtain
a <i>shared-lock</i> on the file-handle open on the database file.
REQ H21011
If, while opening a <i>read-only transaction</i>, SQLite fails to obtain
the <i>shared-lock</i> on the database file, then the process is
abandoned, no transaction is opened and an error returned to the user.
<p>
The most common reason an attempt to obtain a <i>shared-lock</i> may
fail is that some other connection is holding an <i>exclusive</i> or
<i>pending lock</i>. However it may also fail because some other
error (e.g. IO, comms related) occurs within the call to the xLock()
method.
REQ H21003
While opening a <i>read-only transaction</i>, after successfully
obtaining a <i>shared lock</i> on the database file, SQLite shall
attempt to detect and roll back a <i>hot journal file</i> associated
with the same database file.
REQ H21012
If, while opening a <i>read-only transaction</i>, SQLite encounters
an error while attempting to detect or roll back a <i>hot journal
file</i>, then the <i>shared-lock</i> on the database file is released,
no transaction is opened and an error returned to the user.
<p>
Section <cite>hot_journal_detection</cite> contains a description of
and requirements governing the detection of a hot-journal file refered
to in the above requirements.
REQ H21004
Assuming no errors have occured, then after attempting to detect and
roll back a <i>hot journal file</i>, if the connections
<i>page cache</i> is not empty, then SQLite shall validate the contents
of the <i>page cache</i> by testing the <i>file change counter</i>.
This procedure is known as <i>cache validiation</i>.
REQ H21005
If the contents of the <i>page cache</i> are found to be invalid by
the check prescribed by F20040, SQLite shall discard the cache contents
before continuing.
<h3 id=hot_journal_detection>Hot Journal Detection</h3>
<p>
|
| ︙ | ︙ | |||
495 496 497 498 499 500 501 |
<cite>rollback</cite>).
</ol>
<p>
The following requirements describe step 1 of the above procedure in
more detail.
| | | | | | 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 |
<cite>rollback</cite>).
</ol>
<p>
The following requirements describe step 1 of the above procedure in
more detail.
REQ H21014
When required to attempt to detect a <i>hot-journal file</i>, SQLite
shall first use the xAccess() method of the VFS layer to check if a
journal file exists in the file-system.
REQ H21015
When required to attempt to detect a <i>hot-journal file</i>, if the
call to xAccess() required by H21014 indicates that a journal file does
not exist, then the attempt to detect a <i>hot-journal file</i> is
finished. A <i>hot-journal file</i> was not detected.
<p>
The following requirements describe step 2 of the above procedure in
more detail.
REQ H21016
When required to attempt to detect a <i>hot-journal file</i>, if the
call to xAccess() required by H21014 indicates that a journal file
is present, then the xCheckReservedLock() method of the database file
file-handle is invoked to determine whether or not some other
process is holding a <i>reserved</i> or greater lock on the database
file.
REQ H21017
If the call to xCheckReservedLock() required by H21016 indicates that
some other <i>database connection</i> is holding a <i>reserved</i>
or greater lock on the database file,
<p class=todo>
Finish this section.
|
| ︙ | ︙ | |||
551 552 553 554 555 556 557 |
opening a new <i>read-only transaction</i>, SQLite checks the value
of the <i>file change counter</i> stored in the database file. If the
value has not changed since the database file was unlocked, then the
contents of the <i>page cache</i> can be trusted. If the value has
changed, then the <i>page cache</i> cannot be trusted and all data
is discarded.
| | | | | 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 |
opening a new <i>read-only transaction</i>, SQLite checks the value
of the <i>file change counter</i> stored in the database file. If the
value has not changed since the database file was unlocked, then the
contents of the <i>page cache</i> can be trusted. If the value has
changed, then the <i>page cache</i> cannot be trusted and all data
is discarded.
REQ H21018
When a file-handle open on a database file is unlocked, if the
<i>page cache</i> belonging to the associated <i>database connection</i>
is not empty, SQLite shall store the value of the <i>file change
counter</i> internally.
REQ H21019
When required to perform <i>cache validation</i> as part of opening
a <i>read transaction</i>, SQLite shall read a 16 byte block
starting at byte offset 24 of the <i>database file</i> using the xRead()
method of the <i>database connections</i> file handle.
<p class=todo>
Why a 16 byte block? Why not 4? (something to do with encrypted
databases).
REQ H21020
While performing <i>cache validation</i>, after loading the 16 byte
block as required by H21019, SQLite shall compare the 32-bit big-endian
integer stored in the first 4 bytes of the block to the most
recently stored value of the <i>file change counter</i> (see H21018).
If the values are not the same, then SQLite shall conclude that
the contents of the cache are invalid.
|
| ︙ | ︙ | |||
599 600 601 602 603 604 605 |
the connections <i>expected page size</i>. The <i>expected page size</i>
is the value of the <i>page-size</i> field read from the
<i>database file header</i> while opening the database connection
(see section <cite>open_new_connection</cite>), or the <i>page-size</i>
stored of the database file when the most <i>read transaction</i> was
concluded.
| | | | | | | 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 |
the connections <i>expected page size</i>. The <i>expected page size</i>
is the value of the <i>page-size</i> field read from the
<i>database file header</i> while opening the database connection
(see section <cite>open_new_connection</cite>), or the <i>page-size</i>
stored of the database file when the most <i>read transaction</i> was
concluded.
REQ H21021
During the conclusing of a <i>read transaction</i>, before unlocking
the database file, SQLite shall set the connections
<i>expected page size</i> to the current database <i>page-size</i>.
REQ H21022
As part of opening a new <i>read transaction</i>, immediately after
performing <i>cache validation</i>, if there is no data for database
page 1 in the <i>page cache</i>, SQLite shall read <i>N</i> bytes from
the start of the database file using the xRead() method of the
connections file handle, where <i>N</i> is the connections current
<i>expected page size</i> value.
REQ H21023
If page 1 data is read as required by H21023, then the value of the
<i>page-size</i> field that appears in the database file header that
consumes the first 100 bytes of the read block is not the same as the
connections current <i>expected page size</i>, then the
<i>expected page size</i> is set to this value, the database file is
unlocked and the entire procedure to open a <i>read transaction</i>
is repeated.
REQ H21024
If page 1 data is read as required by H21023, then the value of the
<i>page-size</i> field that appears in the database file header that
consumes the first 100 bytes of the read block is the same as the
connections current <i>expected page size</i>, then the block of data
read is added to the connections <i>page cache</i> as page 1.
<h2>Ending a Read-only Transaction</h2>
<p>
To end a <i>read-only transaction</i>, SQLite simply relinquishes the
<i>shared lock</i> on the file-handle open on the database file. No
other action is required.
REQ H21013
When required to end a <i>read-only transaction</i>, SQLite shall
relinquish the <i>shared lock</i> held on the database file by
calling the xUnlock() method of the file-handle.
<p>
See also requirements H21018 and H21021 above.
|
| ︙ | ︙ | |||
714 715 716 717 718 719 720 |
started may be required. This may occur if the user explicitly requests
transaction rollback (i.e. by issuing a "ROLLBACK" command), or
automatically, as a result of encountering an SQL constraint (see
<cite>sql_sqlitert_requirements</cite>). For this reason, the original
page content is stored in the <i>journal file</i> before the page is
even modified within the <i>page cache</i>.
| | | | 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 |
started may be required. This may occur if the user explicitly requests
transaction rollback (i.e. by issuing a "ROLLBACK" command), or
automatically, as a result of encountering an SQL constraint (see
<cite>sql_sqlitert_requirements</cite>). For this reason, the original
page content is stored in the <i>journal file</i> before the page is
even modified within the <i>page cache</i>.
REQ H21025
Before modifying or adding any in-memory <i>page cache</i> pages in
preparation for writing to the <i>database file</i>, the
<i>database connection</i> shall open a <i>write transaction</i> on
the database file.
REQ H21026
Before modifying the <i>page cache</i> image of a database page that
existed and was not a <i>free-list leaf</i> page when the current <i>write
transaction</i> began, SQLite shall ensure that the original page content
has been written to the journal file (<i>journalled</i>).
<p class=todo>
If the sector size is larger than the page-size, coresident pages must
|
| ︙ | ︙ | |||
835 836 837 838 839 840 841 |
the VFS xOpen method), and a <i>journal file header</i> written
to the start of it using a single call to the file handles xWrite
method.
</ol>
<p>
Requirements describing step 1 of the above procedure in detail:
| > | | | | | | 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 |
the VFS xOpen method), and a <i>journal file header</i> written
to the start of it using a single call to the file handles xWrite
method.
</ol>
<p>
Requirements describing step 1 of the above procedure in detail:
REQ H21035
When required to open a <i>write transaction</i> on the database,
SQLite shall first open a <i>read transaction</i>, if the <i>database
connection</i> in question has not already opened one.
REQ H21036
When required to open a <i>write transaction</i> on the database, after
ensuring a <i>read transaction</i> has already been opened, SQLite
shall obtain a <i>reserved lock</i> on the database file by calling
the xLock method of the file-handle open on the database file.
<p>
Requirements describing step 2 of the above procedure in detail:
REQ H21037
When required to open a <i>write transaction</i> on the database, after
obtaining a <i>reserved lock</i> on the database file, SQLite shall
open a read/write file-handle on the corresponding <i>journal file</i>.
REQ H21038
When required to open a <i>write transaction</i> on the database, after
opening a file-handle on the <i>journal file</i>, SQLite shall write
a <i>journal header</i> into the first <i>sector-size</i> bytes of the
journal file, using single call to the xWrite method of the recently
opened file-handle.
<p>
Requirements describing the <i>journal header</i> written to
the <i>journal file</i>:
REQ H21039
The first 8 bytes of the <i>journal header</i> required to be written
by H21038 shall be:
<p class=todo>
Reqirements describing the details of opening a <i>write transaction</i>.
|
| ︙ | ︙ | |||
890 891 892 893 894 895 896 |
back.
<p>
A page is journalled by adding a <i>journal record</i> to the <i>
journal file</i>. The format of a <i>journal record</i> is described
in section <cite>journal_record_format</cite>.
| | | | | | 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 |
back.
<p>
A page is journalled by adding a <i>journal record</i> to the <i>
journal file</i>. The format of a <i>journal record</i> is described
in section <cite>journal_record_format</cite>.
REQ H21027
When required to <i>journal a database page</i>, SQLite shall first
append the <i>page number</i> of the page being journalled to the
<i>journal file</i>, formatted as a 4-byte big-endian unsigned integer,
using a single call to the xWrite method of the file-handle opened
on the journal file.
REQ H21028
When required to <i>journal a database page</i>, if the attempt to
append the <i>page number</i> to the journal file is successful,
then the current page data (<i>page-size</i> bytes) shall be appended
to the journal file, using a single call to the xWrite method of the
file-handle opened on the journal file.
REQ H21029
When required to <i>journal a database page</i>, if the attempt to
append the current page data to the journal file is successful,
then SQLite shall append a 4-byte big-endian integer checksum value
to the to the journal file, using a single call to the xWrite method
of the file-handle opened on the journal file.
<p>
The checksum value written to the <i>journal file</i> immediately after
the page data (requirement H21029), is a function of both the page
data and the <i>checksum initializer</i> field stored in the
<i>journal header</i> (see section <cite>journal_header_format</cite>).
Specifically, it is the sum of the <i>checksum initializer</i> and
the value of every 200th byte of page data interpreted as an 8-bit
unsigned integer, starting with the (<i>page-size</i> % 200)'th
byte of page data. For example, if the <i>page-size</i> is 1024 bytes,
then a checksum is calculated by adding the values of the bytes at
offsets 23, 223, 423, 623, 823 and 1023 (the last byte of the page)
together with the value of the <i>checksum initializer</i>.
REQ H21030
The checksum value written to the <i>journal file</i> by the write
required by H21029 shall be equal to the sum of the <i>checksum
initializer</i> field stored in the <i>journal header</i> (H21XXX) and
every 200th byte of the page data, beginning with the
(<i>page-size</i> % 200)th byte.
<p>
|
| ︙ | ︙ | |||
1024 1025 1026 1027 1028 1029 1030 |
prevent any other <i>database connection</i> from reading the database
after a subset of the modifications that have been or will be made by
a <i>write transaction</i> have been written into the database file.
<p class=todo>
Journal header operations?
| | | | | | 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 |
prevent any other <i>database connection</i> from reading the database
after a subset of the modifications that have been or will be made by
a <i>write transaction</i> have been written into the database file.
<p class=todo>
Journal header operations?
REQ H21031
Unless a <i>pending</i> or <i>exclusive</i> lock has already been
obtained, when SQLite is required to <i>write out a page cache</i>, it
shall first upgrade the lock on the database file to a
<i>pending lock</i> using a call to the xLock method of the file-handle
open on the database file.
REQ H21032
Unless one has already been obtained, when SQLite is required to
<i>write out a page cache</i>, after successfully obtaining a
<i>pending lock</i> it shall upgrade the lock on the database file
to an <i>exclusive lock</i> using a call to the xLock method of the
file-handle open on the database file.
<p class=todo>
If obtaining the lock fails?
REQ H21033
When SQLite is required to <i>write out a page cache</i>, if the
required <i>exclusive lock</i> is already held or successfully
obtained, SQLite shall copy the contents of all pages that have been
modified within the <i>page cache</i> to the database file, using a
single write of <i>page-size</i> bytes for each.
REQ H21034
When the modified contents of a <i>page cache</i> is copied into the
database file, as required by H21033, the write operations shall
occur in <i>page number</i> order, from lowest to highest.
<p>
The above requirement to write data to the database file in the order
in which it occurs in the file is added to improve performance. On
|
| ︙ | ︙ | |||
1080 1081 1082 1083 1084 1085 1086 1087 |
<tr><td style="width:5ex" id="capi_sqlitert_requirements">[1]<td>
C API Requirements Document.
<tr><td style="width:5ex" id="sql_sqlitert_requirements">[2]<td>
SQL Requirements Document.
<tr><td style="width:5ex" id="ff_sqlitert_requirements">[3]<td>
File Format Requirements Document.
</table>
| > | 1116 1117 1118 1119 1120 1121 1122 1123 1124 |
<tr><td style="width:5ex" id="capi_sqlitert_requirements">[1]<td>
C API Requirements Document.
<tr><td style="width:5ex" id="sql_sqlitert_requirements">[2]<td>
SQL Requirements Document.
<tr><td style="width:5ex" id="ff_sqlitert_requirements">[3]<td>
File Format Requirements Document.
</table>
}]</tcl>
|