Documentation Source Text

    [Figure journal_format.gif figure_journal_format "Journal File Format"]

    <p>
      The following requirements define a well-formed journal section. This concept
      is used in section <cite>reading_from_files</cite>. 

        [fileformat_import_requirement2 H35210]
        [fileformat_import_requirement2 H35220]
        [fileformat_import_requirement2 H35230]
        [fileformat_import_requirement2 H35240]

    <p>
      Note that a journal section that is not strictly speaking a well-formed
      journal section often contains important data. For example, many journal 
      files created by SQLite that consist of a single journal section and no
      master journal pointer contain a journal section that is not well-formed
      according to requirement H35240. See section <cite>reading_from_files</cite> 
      for details on when well-formedness is an important property of journal
      sections and when it is not.

    [h4 "Journal Header Format" journal_header_format]

    <p>
      A journal header is sector-size bytes in size, where sector-size is the 
................................................................................
      is defined as a blob of 28 bytes for which the journal magic field is set
      correctly and for which both the page size and sector size fields are set
      to power of two values greater than 512. Because there are no
      restrictions on the values that may be stored in the record count,
      checksum initializer or database page count fields, they do not enter
      into the definition of a well-formed journal header.

      [fileformat_import_requirement2 H35090]
      [fileformat_import_requirement2 H35180]
      [fileformat_import_requirement2 H35190]
      [fileformat_import_requirement2 H35200]

  [h4 "Journal Record Format" journal_record_format]

    <p>
    Each <i>journal record</i> contains the data for a single database page,
    a page number identifying the page, and a checksum value used to help
    detect journal file corruption.
................................................................................


    <p>
      The set of <i>journal records</i> that follow a <i>journal header</i>
      in a journal file are packed tightly together. There are no alignment 
      requirements for <i>journal records</i>.

      [fileformat_import_requirement2 H35100]
      [fileformat_import_requirement2 H35110]
      [fileformat_import_requirement2 H35120]


  [h4 "Master Journal Pointer" master_journal_ptr]

    <p>
      If present, a master journal pointer occurs at the end of a journal file.
      There may or may not be unused space between the end of the final journal 
................................................................................
               Finally, the <b>journal magic</b> field always contains a
               well-known 8-byte string value; the same value stored in the
               first 8 bytes of a <i>journal header</i>. The well-known
               sequence of bytes is:
                 <pre>0xd9 0xd5 0x05 0xf9 0x20 0xa1 0x63 0xd7</pre>
    </table>

      [fileformat_import_requirement2 H35140]
      [fileformat_import_requirement2 H35150]
      [fileformat_import_requirement2 H35160]
      [fileformat_import_requirement2 H35170]

[h3 "Master-Journal File Details" masterjournal_file_format]

  <p>
    A <i>master-journal file</i> contains the full paths to two or more
    <i>journal files</i>, each encoded using UTF-8 encoding and terminated
    by a single nul character (byte value 0x00). There is no padding 
................................................................................
    [Figure filesystem1.gif figure_filesystem1 "Two ways to store the same database image"]

  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid journal file within the 
    file-system.

    [fileformat_import_requirement2 H35000]
    [fileformat_import_requirement2 H35010]
    [fileformat_import_requirement2 H35020]

  <p>
    If there is a valid journal file within the file-system, the 
    following requirements govern how a reader should determine the set
    of valid <i>journal records</i> that it contains.

    [fileformat_import_requirement2 H35250]
    [fileformat_import_requirement2 H35260]
    [fileformat_import_requirement2 H35270]
    [fileformat_import_requirement2 H35280]

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the database image
    should be determined by the reader.

    [fileformat_import_requirement2 H35030]
    [fileformat_import_requirement2 H35040]
    [fileformat_import_requirement2 H35050]
    [fileformat_import_requirement2 H35060]

  <p>
    The following requirements dictate the way in which the data for each
    page of the database image can be located within the file-system
    by a database reader.

    [fileformat_import_requirement2 H35070]
    [fileformat_import_requirement2 H35080]

[h2 "Writing to an SQLite Database" writing_to_files]

  <p>
    When an SQLite user commits a transaction that modifies the contents
    of the database, the database representation on disk must be modified
    to reflect the new contents of the database image. SQLite is required 

    [Figure journal_format.gif figure_journal_format "Journal File Format"]

    <p>
      The following requirements define a well-formed journal section. This concept
      is used in section <cite>reading_from_files</cite>. 

        [fileformat_import_requirement2 H32210]
        [fileformat_import_requirement2 H32220]
        [fileformat_import_requirement2 H32230]
        [fileformat_import_requirement2 H32240]

    <p>
      Note that a journal section that is not strictly speaking a well-formed
      journal section often contains important data. For example, many journal 
      files created by SQLite that consist of a single journal section and no
      master journal pointer contain a journal section that is not well-formed
      according to requirement H32240. See section <cite>reading_from_files</cite> 
      for details on when well-formedness is an important property of journal
      sections and when it is not.

    [h4 "Journal Header Format" journal_header_format]

    <p>
      A journal header is sector-size bytes in size, where sector-size is the 
................................................................................
      is defined as a blob of 28 bytes for which the journal magic field is set
      correctly and for which both the page size and sector size fields are set
      to power of two values greater than 512. Because there are no
      restrictions on the values that may be stored in the record count,
      checksum initializer or database page count fields, they do not enter
      into the definition of a well-formed journal header.

      [fileformat_import_requirement2 H32090]
      [fileformat_import_requirement2 H32180]
      [fileformat_import_requirement2 H32190]
      [fileformat_import_requirement2 H32200]

  [h4 "Journal Record Format" journal_record_format]

    <p>
    Each <i>journal record</i> contains the data for a single database page,
    a page number identifying the page, and a checksum value used to help
    detect journal file corruption.
................................................................................


    <p>
      The set of <i>journal records</i> that follow a <i>journal header</i>
      in a journal file are packed tightly together. There are no alignment 
      requirements for <i>journal records</i>.

      [fileformat_import_requirement2 H32100]
      [fileformat_import_requirement2 H32110]
      [fileformat_import_requirement2 H32120]


  [h4 "Master Journal Pointer" master_journal_ptr]

    <p>
      If present, a master journal pointer occurs at the end of a journal file.
      There may or may not be unused space between the end of the final journal 
................................................................................
               Finally, the <b>journal magic</b> field always contains a
               well-known 8-byte string value; the same value stored in the
               first 8 bytes of a <i>journal header</i>. The well-known
               sequence of bytes is:
                 <pre>0xd9 0xd5 0x05 0xf9 0x20 0xa1 0x63 0xd7</pre>
    </table>

      [fileformat_import_requirement2 H32140]
      [fileformat_import_requirement2 H32150]
      [fileformat_import_requirement2 H32160]
      [fileformat_import_requirement2 H32170]

[h3 "Master-Journal File Details" masterjournal_file_format]

  <p>
    A <i>master-journal file</i> contains the full paths to two or more
    <i>journal files</i>, each encoded using UTF-8 encoding and terminated
    by a single nul character (byte value 0x00). There is no padding 
................................................................................
    [Figure filesystem1.gif figure_filesystem1 "Two ways to store the same database image"]

  <p>
    These requirements describe the way a database reader must determine
    whether or not there is a valid journal file within the 
    file-system.

    [fileformat_import_requirement2 H32000]
    [fileformat_import_requirement2 H32010]
    [fileformat_import_requirement2 H32020]

  <p>
    If there is a valid journal file within the file-system, the 
    following requirements govern how a reader should determine the set
    of valid <i>journal records</i> that it contains.

    [fileformat_import_requirement2 H32250]
    [fileformat_import_requirement2 H32260]
    [fileformat_import_requirement2 H32270]
    [fileformat_import_requirement2 H32280]

  <p>
    The following requirements dictate the way in which database
    <i>page-size</i> and the number of pages in the database image
    should be determined by the reader.

    [fileformat_import_requirement2 H32030]
    [fileformat_import_requirement2 H32040]
    [fileformat_import_requirement2 H32050]
    [fileformat_import_requirement2 H32060]

  <p>
    The following requirements dictate the way in which the data for each
    page of the database image can be located within the file-system
    by a database reader.

    [fileformat_import_requirement2 H32070]
    [fileformat_import_requirement2 H32080]

[h2 "Writing to an SQLite Database" writing_to_files]

  <p>
    When an SQLite user commits a transaction that modifies the contents
    of the database, the database representation on disk must be modified
    to reflect the new contents of the database image. SQLite is required 

<h1 align="center">The Virtual Table Mechanism Of SQLite</h1>
<tcl>hd_keywords {virtual table} {virtual tables}</tcl>

<h2>1.0 Introduction</h2>

<p>A virtual table is an object that is registered with an open SQLite
database connection. From the perspective of an SQL statement,
the virtual table object looks like any other table or view. 
But behind the scenes, queries from and updates to a virtual table
invoke callback methods on the virtual table object instead of
reading and writing to the database file.

<p>The virtual table mechanism allows an application to publish
interfaces that are accessible from SQL statements as if they were
................................................................................

<p>
<ul>
<li> One cannot create a trigger on a virtual table.
<li> One cannot create additional indices on a virtual table. 
     (Virtual tables can have indices but that must be built into
     the virtual table implementation.  Indices cannot be added
     separately using CREATE INDEX statements.)

<li> One cannot run ALTER TABLE commands against a virtual table.
<li> Virtual tables cannot be used in a database that makes use of
     the [shared cache] feature. 
</ul>

<p>Particular virtual table implementations might impose additional
constraints. For example, some virtual implementations might provide
read-only tables. Or some virtual table implementations might allow
INSERT or DELETE but not UPDATE.  Or some virtual table implementations
might limit the kinds of UPDATEs that can be made.

<p>A virtual table might represent an in-memory data structures. 
Or it might represent a view of data on disk that is not in the
SQLite format. Or the application might compute the content of the 
virtual table on demand.

<p>Here are some postulated uses for virtual tables:

<ul>
<li> A full-text search interface
<li> Spatial indices using R-Trees


<li> Access to the filesystem of the host computer
<li> Enabling SQL manipulation of data in statistics packages like R
</ul>

<h3>1.1 Usage</h3>

<p>A virtual table is created using using a CREATE VIRTUAL TABLE statement.
This statement creates a table with a particular name and associates the
table with a "module".

<blockquote><pre>
   CREATE VIRTUAL TABLE tablename USING modulename;
</pre></blockquote>

................................................................................
<blockquote><pre>
   CREATE VIRTUAL TABLE tablename USING modulename(arg1, arg2, ...);
</pre></blockquote>

<p>The format of the arguments to the module is very general. Each argument 
can consist of keywords, string literals, identifiers, numbers, and 
punctuation. The arguments are passed as written (as text) into the
constructor metod of the virtual table implementation when the virtual 

table is created and that constructor is responsible for parsing and 
interpreting the arguments. The argument syntax is sufficiently general 
that a virtual table implementation can, if it wants to, interpret its
arguments as column definitions in an ordinary CREATE TABLE statement. 
The implementation could also impose some other interpretation on the 
arguments.

<p>Once a virtual table has been created, it can be used like any other 
table with the exceptions noted above and imposed by specific virtual
table implementations. A virtual table is destroyed using the ordinary
DROP TABLE syntax.

<h2>Implementation</h2>

<p>Several new C-level objects are used by the virtual table implementation:

<blockquote><pre>
  typedef struct sqlite3_vtab sqlite3_vtab;
  typedef struct sqlite3_index_info sqlite3_index_info;
  typedef struct sqlite3_vtab_cursor sqlite3_vtab_cursor;
  typedef struct sqlite3_module sqlite3_module;
</pre></blockquote>

<p>The sqlite3_module structure defines a module object used to implement
a virtual table. Think of a module as a class from which you can 
construct multiple virtual tables having similar properties. For example,
one might have a module that provides read-only access to 
comma-separated-value (CSV) files on disk. That one module can then be
used to create several virtual tables where each virtual table refers
to a different CSV file.

<p>The module structure contains methods that are invoked by SQLite to
perform various actions on the virtual table such as creating new
instances of a virtual table or destroying old ones, reading and
writing data, searching for and deleting, updating, or inserting rows. 
The module structure is explained in more detail below.

<p>Each virtual table instance is represented by an sqlite3_vtab structure. 
The sqlite3_vtab structure looks like this:

<blockquote><pre>
  struct sqlite3_vtab {
    const sqlite3_module *pModule;
    int nRef;
    char *zErrMsg;
................................................................................
  };
</pre></blockquote>

<p>Virtual table implementations will normally subclass this structure 
to add additional private and implementation-specific fields. 
The nRef field is used internally by the SQLite core and should not 
be altered by the virtual table implementation. The virtual table 
implementation can pass error message text to the core by putting 
an error message string obtained from [sqlite3_mprintf()] in zErrMsg.



Prior to assigning a new value to zErrMsg, the virtual table 
implementation should free any preexisting content of zErrMsg using 
[sqlite3_free()]. Failure to do this might result in a memory leak. 
The SQLite core will free and zero the content of zErrMsg when it 
delivers the error message text to the client application or when 
it destroys the virtual table. The virtual table implementation only 
needs to worry about freeing the zErrMsg content when it overwrites 
the content with a new, different error message.

<p>The sqlite3_vtab_cursor structure represents a pointer to a specific
row of a virtual table. This is what an sqlite3_vtab_cursor looks like:

<blockquote><pre>
  struct sqlite3_vtab_cursor {
    sqlite3_vtab *pVtab;
  };
</pre></blockquote>

<p>Once again, practical implementations will likely subclass this 
structure to add additional private fields.

<p>The sqlite3_index_info structure is used to pass information into
and out of the xBestIndex method of the module that implements a 
virtual table.

<p>Before a CREATE VIRTUAL TABLE statement can be run, the module 
specified in that statement must be registered with the database 
connection. This is accomplished using either of the [sqlite3_create_module()]
or [sqlite3_create_module_v2()] interfaces:

<blockquote><pre>
  int sqlite3_create_module(
    sqlite3 *db,               /* SQLite connection to register module with */
................................................................................
    const char *zName,         /* Name of the module */
    const sqlite3_module *,    /* Methods for the module */
    void *,                    /* Client data for xCreate/xConnect */
    void(*xDestroy)(void*)     /* Client data destructor function */
  );
</pre></blockquote>


<p>The sqlite3_create_module() routines associates a module name with 
an sqlite3_module structure and a separate client data that is specific 
to each module.  The only difference between the two create_module methods
is that the _v2 method includes an extra parameter that specifies a
destructor for client data pointer.  The module structure is what defines
the behavior of a virtual table.  The module structure looks like this:

<blockquote><pre>  
  struct sqlite3_module {
................................................................................
                       void **ppArg);
    int (*Rename)(sqlite3_vtab *pVtab, const char *zNew);
  };
</pre></blockquote>

<p>The module structure defines all of the methods for each virtual 
table object. The module structure also contains the iVersion field which
 defines the particular edition of the module table structure. Currently, 
iVersion is always 1, but in future releases of SQLite the module structure 
definition might be extended with additional methods and in that case 
the iVersion value will be increased.

<p>The rest of the module structure consists of methods used to implement
various features of the virtual table. Details on what each of these 
methods do are provided in the sequel.

<h3>1.2 Virtual Tables And Shared Cache</h3>

<p>The virtual table mechanism assumes that each database connection keeps 
its own copy of the database schema. Hence, the virtual table mechanism
cannot be used in a database that has [shared cache] enabled. 
The [sqlite3_create_module()] interface will return an error if 
the [shared cache] feature is enabled.

<h3>1.3 Creating New Virtual Table Implementations</h3>

<p>Follow these steps to create your own virtual table:

<p>
<ol>
<li> Write all necessary methods.
<li> Create an instance of the [sqlite3_module] structure containing pointers
     to all the methods from step 1.
<li> Register your [sqlite3_module] structure using the
     [sqlite3_create_module()] API.
<li> Run a [CREATE VIRTUAL TABLE] command that specifies your module in 
     the USING clause. 
</ol>

<p>The only really hard part is step 1. You might want to start with an 
existing virtual table implementation and modify it to suit your needs.
There are several virtual table implementations in the SQLite source tree 
(for testing purposes). You might use one of those as a guide. Locate 
................................................................................
for "sqlite3_create_module".

<p>You might also want to implement your new virtual table as a 
[sqlite3_load_extension | loadable extension].

<h2>2.0 Virtual Table Methods</h2>



<h3>2.1 The xCreate Method</h3>

<blockquote><pre>
  int (*xCreate)(sqlite3 *db, void *pAux,
               int argc, char **argv,
               sqlite3_vtab **ppVTab,
               char **pzErr);
................................................................................

<p>This method is called to create a new instance of a virtual table 
in response to a [CREATE VIRTUAL TABLE] statement. 
The db parameter is a pointer to the SQLite [database connection] that 
is executing the [CREATE VIRTUAL TABLE] statement. 
The pAux argument is the copy of the client data pointer that was the 
fourth argument to the [sqlite3_create_module()] or
[sqlite3_create_module_v2()] call that registered the virtual table module. 
The argv parameter is an array of argc pointers to null terminated strings. 
The first string, argv[0], is the name of the module being invoked. 



The second, argv[1], is the name of the database in which the new virtual table is being created. The third element of the array, argv[2], 


is the name of the new virtual table. 

If present, the fourth and subsquent strings in the argv[] array report 
the arguments to the module name in the [CREATE VIRTUAL TABLE] statement.

<p>The job of this method is to construct the new virtual table object
(an [sqlite3_vtab] object) and return a pointer to it in *ppVTab.

<p>As part of the task of creating a new [sqlite3_vtab] structure, this 
method must invoke [sqlite3_declare_vtab()] to tell the SQLite core about 
the columns and datatypes in the virtual table. 
The [sqlite3_declare_vtab()] API has the following prototype:

<blockquote><pre>
    int sqlite3_declare_vtab(sqlite3 *db, const char *zCreateTable)
</pre></blockquote>

<p>The first argument to [sqlite3_declare_vtab()] is the pointer to the 
sqlite [database connection].  The first argument must be the same
database connection pointer that was passed into the xCreate method.
The second argument to [sqlite3_declare_vtab()] is a zero-terminated 
UTF-8 string that contains a well-formed [CREATE TABLE] statement that 
defines the columns in the virtual table and their data types. 
The name of the table in this [CREATE TABLE] statement is ignored, 
as are all constraints. Only the column names and datatypes matter.





<p>If a column datatype contains the special keyword "HIDDEN"
(in any combination of upper and lower case letters) then it is removed 

from the column datatype and the column marked as a hidden column internally. 
A hidden column differs from a normal column in three respects:

<p>
<ul>
<li> Hidden columns are not listed in the dataset returned by 
     "[PRAGMA table_info]",
<li> Hidden columns are not included in the expansion of a "*"
................................................................................

<p>For example, if the following SQL is passed to [sqlite3_declare_vtab()]:

<blockquote><pre>
   CREATE TABLE x(a HIDDEN VARCHAR(12), b INTEGER, c INTEGER Hidden);
</pre></blockquote>

<p>Then the virtual table would be created with two hidden columns, declaration types "VARCHAR(12)" and "INTEGER".


<p>The xCreate method should return [SQLITE_OK] if it is successful in 
creating the new virtual table, or [SQLITE_ERROR] if it is not successful.
If not successful, no [sqlite3_vtab] structure should be allocated. 
An error message may optionally be returned in *pzErr if unsuccessful.
The text of the error message should be obtained from [sqlite3_mprintf()].













<h3>2.2 The xConnect Method</h3>

<blockquote><pre>
  int (*xConnect)(sqlite3*, void *pAux,
               int argc, char **argv,
               sqlite3_vtab **ppVTab,
               char **pzErr);
</pre></blockquote>

<p>The xConnect method is very similar to xCreate. 
It has the same parameters and constructs a new [sqlite3_vtab] structure 
just like xCreate. 
And it must also call [sqlite3_declare_vtab()] like xCreate.

<p>The difference is that xConnect is called to establish a new 
connection to an existing virtual table whereas xCreate is called 
to create a new virtual table from scratch.
................................................................................

<p>Another example is a virtual table that implements a full-text index. 
The xCreate method must create and initialize data structures to hold 
the dictionary and posting lists for that index. The xConnect method,
on the other hand, only has to locate and use an existing dictionary 
and posting lists that were created by a prior xCreate call.

<p>The xConnect method should return [SQLITE_OK] if it is successful 
in creating the new virtual table, or [SQLITE_ERROR] if it is not 
successful. If not successful, no [sqlite3_vtab] structure should be 
allocated. An error message may optionally be returned in *pzErr if 
unsuccessful. 
The text of the error message should be obtained from [sqlite3_mprintf()].












<h3>2.3 The xBestIndex Method</h3>



<p>The xBestIndex method has a prototype like this:

<blockquote><pre>
  int (*xBestIndex)(sqlite3_vtab *pVTab, sqlite3_index_info*);
</pre></blockquote>

<p>The SQLite core communicates with the xBestIndex method by filling 
in certain fields of the [sqlite3_index_info] structure and passing a 
................................................................................
  #define SQLITE_INDEX_CONSTRAINT_LE    8
  #define SQLITE_INDEX_CONSTRAINT_LT    16
  #define SQLITE_INDEX_CONSTRAINT_GE    32
  #define SQLITE_INDEX_CONSTRAINT_MATCH 64
</pre></blockquote>

<p>The SQLite core calls the xBestIndex method when it is compiling a query
 that involves a virtual table. In other words, SQLite calls this method 
when it is running [sqlite3_prepare()]. By calling this method, the 

SQLite core is saying to the virtual table that it needs to access 
some subset of the rows in the virtual table and it wants to know the
most efficient way to do that access. The xBestIndex method replies 
with information that the SQLite core can then use to conduct an 
efficient search of the virtual table.

<p>While compiling a single SQL query, the SQLite core might call 
xBestIndex multiple times with different settings in [sqlite3_index_info].
The SQLite core will then select the combination that appears to 
give the best performance.

<h4>2.3.1 Inputs</h4>

<p>Before calling this method, the SQLite core initializes an instance 
of the [sqlite3_index_info] structure with information about the
query that it is currently trying to process. This information 
derives mainly from the WHERE clause and ORDER BY or GROUP BY clauses 
of the query, but also from any ON or USING clauses if the query is a 
join. The information that the SQLite core provides to the xBestIndex 
method is held in the part of the structure that is marked as "Inputs". 
The "Outputs" section is initialized to zero.

















<p>The main thing that the SQLite core is trying to communicate to 
the virtual table is the constraints that are available to limit 
the number of rows that need to be searched. The aConstraint[] array 
contains one entry for each constraint. There will be exactly 
nConstraint entries in that array.

................................................................................
if the WHERE clause contained a term like this:

<blockquote><pre>
     a = 5
</pre></blockquote>

<p>Then one of the constraints would be on the "a" column with 
operator "=" and an expression of "5". Constraints are not a 
literal representation of the WHERE clause. The query optimizer 

translates the WHERE clause in order to extract as many constraints 
as it can. So, for example, if the WHERE clause contained something 
like this:

<blockquote><pre>
     x BETWEEN 10 AND 100 AND 999>y
</pre></blockquote>

<p>The query optimizer would translate this into three separate constraints:

<blockquote><pre>
     x >= 10
     x <= 100
     y < 999
</pre></blockquote>

................................................................................
<p>For each constraint, the aConstraint[].iColumn field indicates which 
column appears on the left-hand side of the constraint.
The first column of the virtual table is column 0. 
The rowid of the virtual table is column -1. 
The aConstraint[].op field indicates which operator is used. 
The SQLITE_INDEX_CONSTRAINT_* constants map integer constants 
into operator values.




<p>The aConstraint[] array contains information about all constraints 
that apply to the virtual table. But some of the constraints might
 not be usable because of the way tables are ordered in a join. 
The xBestIndex method should therefore only consider constraints 
that have a aConstraint[].usable flag which is true.

<p>In addition to WHERE clause constraints, the SQLite core also 
tells the xBestIndex method about the ORDER BY clause. 
(In an aggregate query, the SQLite core might put in GROUP BY clause 
information in place of the ORDER BY clause information, but this fact should not make any difference to the xBestIndex method.) 

If all terms of the ORDER BY clause are columns in the virtual table, 
then nOrderBy will be the number of terms in the ORDER BY clause 
and the aOrderBy[] array will identify the column for each term 
in the order by clause and whether or not that column is ASC or DESC.

<h4>2.3.2 Outputs</h4>

<p>Given all of the information above, the job of the xBestIndex 
method it to figure out the best way to search the virtual table.

<p>The xBestIndex method fills the idxNum and idxStr fields with 
information that communicates an indexing strategy to the xFilter 
method. The information in idxNum and idxStr is arbitrary as far 
as the SQLite core is concerned. The SQLite core just copies the 
information through to the xFilter method. Any desired meaning can 
be assigned to idxNum and idxStr as long as xBestIndex and xFilter 
agree on what that meaning is.

<p>The idxStr value can be a string obtained from [sqlite3_mprintf()]. 

If this is the case, then the needToFreeIdxStr flag must be set to 
true so that the SQLite core will know to call [sqlite3_free()] on 
that string when it has finished with it, and thus avoid a memory leak.

<p>If the virtual table will output rows in the order specified by 
the ORDER BY clause, then the orderByConsumed flag should be set to 
true. If the output is not automatically in the correct order 
then orderByConsumed should be left in its default false setting. 
This will indicate to the SQLite core that it will need to do a 
separate sorting pass over the data after it comes out of the virtual table.

<p>The estimatedCost field should be set to the estimated number
of disk access operations required to execute this query against 
the virtual table. The SQLite core will often call xBestIndex 
multiple times with different constraints, obtain multiple cost
................................................................................
the EXPR value of the aConstraint[3] constraint.

<p>By default, the SQLite core double checks all constraints on 
each row of the virtual table that it receives. If such a check 
is redundant, the xBestFilter method can suppress that check by 
setting aConstraintUsage[].omit.



<h3>2.4 The xDisconnect Method</h3>

<blockquote><pre>
  int (*xDisconnect)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method releases a connection to a virtual table. 

The virtual table is not destroyed and any backing store 
associated with the virtual table persists. 
This method is the opposite of xConnect.












<h3>2.5 The xDestroy Method</h3>

<blockquote><pre>
  int (*xDestroy)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method releases a connection to a virtual table, just like 
the xDisconnect method, and it also destroys the underlying 
table implementation. This method is the opposite of xCreate.

<p>The xDisconnect method is called whenever a database connection
that uses a virtual table is closed. The xDestroy method is only 
called when a [DROP TABLE] statement is executed against the virtual table.







<h3>2.6 The xOpen Method</h3>

<blockquote><pre>
  int (*xOpen)(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor);
</pre></blockquote>

<p>The xOpen method creates a new cursor used for reading information
out of a virtual table.  A successful invocation of this method 
will allocate the memory for the [sqlite3_vtab_cursor] (or a subclass),
initialize the new object, and make *ppCursor point to the new object.
The successful call then returns [SQLITE_OK].

<p>The cursor is not immediately usable. It must first be positioned 
using xFilter. Then elements of a row of the virtual table can be 
accessed using xColumn and xRowid.

<p>The xClose method is used to close the cursor.










<h3>2.7 The xClose Method</h3>

<blockquote><pre>
  int (*xClose)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xClose method closes a cursor previously opened by xOpen. 

The SQLite core will always call xClose once for each cursor opened 
using xOpen.

<p>This method must release all resources allocated by the
corresponding xOpen call. The routine will not be called again if it
returns an error.






<h3>2.8 The xEof Method</h3>

<blockquote><pre>
  int (*xEof)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xEof method must return false (zero) if the specified cursor 
currently points to a valid row of data, or true (non-zero) otherwise. 
This method is called by the SQL engine immediately after each 
xFilter and xNext invocation.





<h3>2.9 The xFilter Method</h3>

<blockquote><pre>
  int (*xFilter)(sqlite3_vtab_cursor*, int idxNum, const char *idxStr,
                int argc, sqlite3_value **argv);
</pre></blockquote>

<p>This method to begin a search of a virtual table. 
The first argument is a cursor opened by xOpen. 
The next two argument define a particular search index previously 
choosen by xBestIndex The specific meanings of idxNum and idxStr 
are unimportant as long as xFilter and xBestIndex agree on what 
that meaning is.

<p>The xBestIndex function may have requested the values of 
certain expressions using the aConstraintUsage[].argvIndex values 
of the sqlite3_index_info structure. 
Those values are passed to xFilter using the argc and argv parameters.

<p>If the filtering constraints configured by idxNum and idxStr do 
not match any rows of the virtual table (i.e. no rows of data will 
be returned), then a subsequent call to the xEof method of the same 
table should return non-zero. Otherwise, xEof should return zero 
and the cursor should be left pointing at a row of the virtual table. 
The xColumn and xRowid methods can be used to access that row. 

The xNext method can be used to advance to the next row.

<p>This method should return SQLITE_OK if successful, or an sqlite 
error code if an error occurs.





<h3>2.10 The xNext Method</h3>

<blockquote><pre>
  int (*xNext)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xNext method advances a virtual table cursor to the next row 
of a result set initiated by xFilter. 
If the cursor is already pointing at the last row when this 
routine is called, then the cursor no longer points to valid 
data and a subsequent call to the xEof method should return non-zero. 
Otherwise, the xEof method returns zero.


<p>This method should return SQLITE_OK if successful, or an sqlite 
error code if an error occurs.





<h3>2.11 The xColumn Method</h3>

<blockquote><pre>
  int (*xColumn)(sqlite3_vtab_cursor*, sqlite3_context*, int N);
</pre></blockquote>

<p>The SQLite core invokes this method in order to find the value for 
the N-th column of the current row. N is zero-based so the first column 
is numbered 0. The xColumn method uses one of the 
[sqlite3_result_blob | sqlite3_result_*() APIs] 
to return the result. 
This method can invoke [sqlite3_result_error()] to raise an exception, 
if desired.






















<h3>2.12 The xRowid Method</h3>

<blockquote><pre>
  int (*xRowid)(sqlite3_vtab_cursor*, sqlite_int64 *pRowid);
</pre></blockquote>

<p>This method writes the 64-bit rowid of the current row into *pRowid.









<h3>2.13 The xUpdate Method</h3>

<blockquote><pre>
  int (*xUpdate)(
    sqlite3_vtab *pVTab,
    int argc,
    sqlite3_value **argv,
................................................................................
<p>All changes to a virtual table are made using the xUpdate method.
This one method can be used to insert, delete, or update.

<p>The argc parameter specifies the number of entries in the argv array. 
Every argv entry will have a non-NULL value in C (but may contain the 
SQL value NULL).

<p>The argv[0] parameter is the rowid of a row in the virtual table 
to be deleted. If argv[0] is NULL, then no deletion occurs.

<p>The argv[1] parameter is the rowid of a new row to be inserted 
into the virtual table. If argv[1] is NULL, then the implementation 
must choose a rowid for the newly inserted row. Subsequent argv[] 
entries contain values of the columns of the virtual table, in the 
order that the columns were declared. The number of columns will
match the table declaration that the xConnect or xCreate method made 
using the [sqlite3_declare_vtab()] call.

<p>When doing an insert without a rowid (argc>1, argv[1]==NULL), the 
implementation must set *pRowid to the id of the newly inserted row; 
this will become the value returned by the [sqlite3_last_insert_rowid()]
function. Setting this value in all the other cases has no effect,
and cannot be used to change the rowid of an existing row or if you 
don't like what was in argv[1].

<p>Each call to xUpdate will fall into one of the following cases:

<blockquote>
<dl>
<dt><b>argc == 1</b>
<dd><p>The single row argv[0] is deleted; no insert occurs
................................................................................
when an SQL statement updates a rowid, as in the statement:
<blockquote>
   [UPDATE] table SET rowid=rowid+1 WHERE ...; 
</blockquote>
</dl>
</blockquote>

<p>It is currently undocumented what effect changes should have on 















existing cursors.









<h3>2.14 The xFindFunction Method</h3>

<blockquote><pre>
  int (*xFindFunction)(
    sqlite3_vtab *pVtab,
    int nArg,
    const char *zName,
    void (**pxFunc)(sqlite3_context*,int,sqlite3_value**),
    void **ppArg
  );
</pre></blockquote>

<p>This method is called during sqlite3_prepare to give the virtual
table implementation an opportunity to overload functions. 
This method may be set to NULL in which case no overloading occurs.

<p>When a function uses a column from a virtual table as its first 
argument, this method is called to see if the virtual table would 
like to overload the function. The first three parameters are inputs: 
the virtual table, the number of arguments to the function, and the 
name of the function. If no overloading is desired, this method
returns 0. To overload the function, this method writes the new 
function implementation into *pxFunc and writes user data into *ppArg 
and returns 1.

<p>Note that infix functions (LIKE, GLOB, REGEXP, and MATCH) reverse 
the order of their arguments. So "like(A,B)" is equivalent to "B like A". 
For the form "B like A" the B term is considered the first argument 
to the function. But for "like(A,B)" the A term is considered the 
first argument.

<p>It is currently undocumented how often this function is called or 
when you can free any resources allocated in order to respond - #2095



<h3>2.15 The xBegin Method</h3>

<blockquote><pre>
  int (*xBegin)(sqlite3_vtab *pVTab);
</pre></blockquote>

<i>TBD...</i>













<h3>2.16 The xSync Method</h3>

<blockquote><pre>
  int (*xSync)(sqlite3_vtab *pVTab);
</pre></blockquote>

<i>TBD...</i>














<h3>2.17 The xCommit Method</h3>

<blockquote><pre>
  int (*xCommit)(sqlite3_vtab *pVTab);
</pre></blockquote>

<i>TBD...</i>









<h3>2.18 The xRollback Method</h3>

<blockquote><pre>
  int (*xRollback)(sqlite3_vtab *pVTab);
</pre></blockquote>

<i>TBD...</i>








<h3>2.19 The xRename Method</h3>

<blockquote><pre>
  int (*xRename)(sqlite3_vtab *pVtab, const char *zNew);
</pre></blockquote>

<p>This method provides sotification that the virtual table implementation
that the virtual table will be given a new name. 
If this method returns SQLITE_OK then SQLite renames the table.
If this method returns an error then the renaming is prevented.

<h1 align="center">The Virtual Table Mechanism Of SQLite</h1>
<tcl>hd_keywords {virtual table} {virtual tables}</tcl>

<h2>1.0 Introduction</h2>

<p>A virtual table is an object that is registered with an open SQLite
[database connection]. From the perspective of an SQL statement,
the virtual table object looks like any other table or view. 
But behind the scenes, queries from and updates to a virtual table
invoke callback methods on the virtual table object instead of
reading and writing to the database file.

<p>The virtual table mechanism allows an application to publish
interfaces that are accessible from SQL statements as if they were
................................................................................

<p>
<ul>
<li> One cannot create a trigger on a virtual table.
<li> One cannot create additional indices on a virtual table. 
     (Virtual tables can have indices but that must be built into
     the virtual table implementation.  Indices cannot be added
     separately using [CREATE INDEX] statements.)
<li> One cannot run [ALTER TABLE | ALTER TABLE ... ADD COLUMN]
     commands against a virtual table.
<li> Virtual tables cannot be used in a database that makes use of
     the [shared cache] feature. 
</ul>

<p>Particular virtual table implementations might impose additional
constraints. For example, some virtual implementations might provide
read-only tables. Or some virtual table implementations might allow
[INSERT] or [DELETE] but not [UPDATE].  Or some virtual table implementations
might limit the kinds of UPDATEs that can be made.

<p>A virtual table might represent an in-memory data structures. 
Or it might represent a view of data on disk that is not in the
SQLite format. Or the application might compute the content of the 
virtual table on demand.

<p>Here are some postulated uses for virtual tables:

<ul>
<li> A full-text search interface
<li> Spatial indices using R-Trees
<li> Read and/or write the content of a comma-separated value (CSV)
     file
<li> Access to the filesystem of the host computer
<li> Enabling SQL manipulation of data in statistics packages like R
</ul>

<h3>1.1 Usage</h3>

<p>A virtual table is created using using a [CREATE VIRTUAL TABLE] statement.
This statement creates a table with a particular name and associates the
table with a "module".

<blockquote><pre>
   CREATE VIRTUAL TABLE tablename USING modulename;
</pre></blockquote>

................................................................................
<blockquote><pre>
   CREATE VIRTUAL TABLE tablename USING modulename(arg1, arg2, ...);
</pre></blockquote>

<p>The format of the arguments to the module is very general. Each argument 
can consist of keywords, string literals, identifiers, numbers, and 
punctuation. The arguments are passed as written (as text) into the
[xCreate | constructor method] of the virtual table implementation 
when the virtual 
table is created and that constructor is responsible for parsing and 
interpreting the arguments. The argument syntax is sufficiently general 
that a virtual table implementation can, if it wants to, interpret its
arguments as column definitions in an ordinary [CREATE TABLE] statement. 
The implementation could also impose some other interpretation on the 
arguments.

<p>Once a virtual table has been created, it can be used like any other 
table with the exceptions noted above and imposed by specific virtual
table implementations. A virtual table is destroyed using the ordinary
[DROP TABLE] syntax.

<h2>2.2 Implementation</h2>

<p>Several new C-level objects are used by the virtual table implementation:

<blockquote><pre>
  typedef struct sqlite3_vtab sqlite3_vtab;
  typedef struct sqlite3_index_info sqlite3_index_info;
  typedef struct sqlite3_vtab_cursor sqlite3_vtab_cursor;
  typedef struct sqlite3_module sqlite3_module;
</pre></blockquote>

<p>The [sqlite3_module] structure defines a module object used to implement
a virtual table. Think of a module as a class from which one can 
construct multiple virtual tables having similar properties. For example,
one might have a module that provides read-only access to 
comma-separated-value (CSV) files on disk. That one module can then be
used to create several virtual tables where each virtual table refers
to a different CSV file.

<p>The module structure contains methods that are invoked by SQLite to
perform various actions on the virtual table such as creating new
instances of a virtual table or destroying old ones, reading and
writing data, searching for and deleting, updating, or inserting rows. 
The module structure is explained in more detail below.

<p>Each virtual table instance is represented by an [sqlite3_vtab] structure. 
The sqlite3_vtab structure looks like this:

<blockquote><pre>
  struct sqlite3_vtab {
    const sqlite3_module *pModule;
    int nRef;
    char *zErrMsg;
................................................................................
  };
</pre></blockquote>

<p>Virtual table implementations will normally subclass this structure 
to add additional private and implementation-specific fields. 
The nRef field is used internally by the SQLite core and should not 
be altered by the virtual table implementation. The virtual table 
implementation may pass error message text to the core by putting 
an error message string in zErrMsg.
Space to hold this error message string must be obtained from an
SQLite memory allocation function such as [sqlite3_mprintf()] or
[sqlite3_malloc()].
Prior to assigning a new value to zErrMsg, the virtual table 
implementation must free any preexisting content of zErrMsg using 
[sqlite3_free()]. Failure to do this will result in a memory leak. 
The SQLite core will free and zero the content of zErrMsg when it 
delivers the error message text to the client application or when 
it destroys the virtual table. The virtual table implementation only 
needs to worry about freeing the zErrMsg content when it overwrites 
the content with a new, different error message.

<p>The [sqlite3_vtab_cursor] structure represents a pointer to a specific
row of a virtual table. This is what an sqlite3_vtab_cursor looks like:

<blockquote><pre>
  struct sqlite3_vtab_cursor {
    sqlite3_vtab *pVtab;
  };
</pre></blockquote>

<p>Once again, practical implementations will likely subclass this 
structure to add additional private fields.

<p>The [sqlite3_index_info] structure is used to pass information into
and out of the xBestIndex method of the module that implements a 
virtual table.

<p>Before a [CREATE VIRTUAL TABLE] statement can be run, the module 
specified in that statement must be registered with the database 
connection. This is accomplished using either of the [sqlite3_create_module()]
or [sqlite3_create_module_v2()] interfaces:

<blockquote><pre>
  int sqlite3_create_module(
    sqlite3 *db,               /* SQLite connection to register module with */
................................................................................
    const char *zName,         /* Name of the module */
    const sqlite3_module *,    /* Methods for the module */
    void *,                    /* Client data for xCreate/xConnect */
    void(*xDestroy)(void*)     /* Client data destructor function */
  );
</pre></blockquote>

<p>The [sqlite3_create_module()] and [sqlite3_create_module_v2()]
routines associates a module name with 
an [sqlite3_module] structure and a separate client data that is specific 
to each module.  The only difference between the two create_module methods
is that the _v2 method includes an extra parameter that specifies a
destructor for client data pointer.  The module structure is what defines
the behavior of a virtual table.  The module structure looks like this:

<blockquote><pre>  
  struct sqlite3_module {
................................................................................
                       void **ppArg);
    int (*Rename)(sqlite3_vtab *pVtab, const char *zNew);
  };
</pre></blockquote>

<p>The module structure defines all of the methods for each virtual 
table object. The module structure also contains the iVersion field which
defines the particular edition of the module table structure. Currently, 
iVersion is always 1, but in future releases of SQLite the module structure 
definition might be extended with additional methods and in that case 
the iVersion value will be increased.

<p>The rest of the module structure consists of methods used to implement
various features of the virtual table. Details on what each of these 
methods do are provided in the sequel.

<h3>1.3 Virtual Tables And Shared Cache</h3>

<p>The virtual table mechanism assumes that each [database connection] keeps 
its own copy of the database schema. Hence, the virtual table mechanism
cannot be used in a database that has [shared cache] enabled. 
The [sqlite3_create_module()] interface will return an error if 
the [shared cache] feature is enabled.

<h3>1.4 Creating New Virtual Table Implementations</h3>

<p>Follow these steps to create your own virtual table:

<p>
<ol>
<li> Write all necessary methods.
<li> Create an instance of the [sqlite3_module] structure containing pointers
     to all the methods from step 1.
<li> Register your [sqlite3_module] structure using one of the
     [sqlite3_create_module()] or [sqlite3_create_module_v2()] interfaces.
<li> Run a [CREATE VIRTUAL TABLE] command that specifies the new module in 
     the USING clause. 
</ol>

<p>The only really hard part is step 1. You might want to start with an 
existing virtual table implementation and modify it to suit your needs.
There are several virtual table implementations in the SQLite source tree 
(for testing purposes). You might use one of those as a guide. Locate 
................................................................................
for "sqlite3_create_module".

<p>You might also want to implement your new virtual table as a 
[sqlite3_load_extension | loadable extension].

<h2>2.0 Virtual Table Methods</h2>

<tcl>############################################################### xCreate
hd_fragment xcreate {sqlite3_module.xCreate} {xCreate}</tcl>
<h3>2.1 The xCreate Method</h3>

<blockquote><pre>
  int (*xCreate)(sqlite3 *db, void *pAux,
               int argc, char **argv,
               sqlite3_vtab **ppVTab,
               char **pzErr);
................................................................................

<p>This method is called to create a new instance of a virtual table 
in response to a [CREATE VIRTUAL TABLE] statement. 
The db parameter is a pointer to the SQLite [database connection] that 
is executing the [CREATE VIRTUAL TABLE] statement. 
The pAux argument is the copy of the client data pointer that was the 
fourth argument to the [sqlite3_create_module()] or
[sqlite3_create_module_v2()] call that registered the [virtual table module]. 
The argv parameter is an array of argc pointers to null terminated strings. 
The first string, argv[0], is the name of the module being invoked.   The
module name is the name provided as the second argument to 
[sqlite3_create_module()] and as the argument to the USING clause of the
[CREATE VIRTUAL TABLE] statement that is running.
The second, argv[1], is the name of the database in which the new virtual table is being created. The database name is "main" for the primary database, or
"temp" for TEMP database, or the name given at the end of the [ATTACH]
statement for attached databases.  The third element of the array, argv[2], 
is the name of the new virtual table, as specified following the TABLE
keyword in the [CREATE VIRTUAL TABLE] statement.
If present, the fourth and subsquent strings in the argv[] array report 
the arguments to the module name in the [CREATE VIRTUAL TABLE] statement.

<p>The job of this method is to construct the new virtual table object
(an [sqlite3_vtab] object) and return a pointer to it in *ppVTab.

<p>As part of the task of creating a new [sqlite3_vtab] structure, this 
method <u>must</u> invoke [sqlite3_declare_vtab()] to tell the SQLite 
core about the columns and datatypes in the virtual table. 
The [sqlite3_declare_vtab()] API has the following prototype:

<blockquote><pre>
    int sqlite3_declare_vtab(sqlite3 *db, const char *zCreateTable)
</pre></blockquote>

<p>The first argument to [sqlite3_declare_vtab()] must be the same 
[database connection] pointer as the first parameter to this method.

The second argument to [sqlite3_declare_vtab()] must a zero-terminated 
UTF-8 string that contains a well-formed [CREATE TABLE] statement that 
defines the columns in the virtual table and their data types. 
The name of the table in this CREATE TABLE statement is ignored, 
as are all constraints. Only the column names and datatypes matter.
The CREATE TABLE statement string need not to be 
held in persistent memory.  The string can be
deallocated and/or reused as soon as the [sqlite3_declare_vtab()]
routine returns.

<p>If a column datatype contains the special keyword "HIDDEN"
(in any combination of upper and lower case letters) then that keyword
it is omitted from the column datatype name and the column is marked 
as a hidden column internally. 
A hidden column differs from a normal column in three respects:

<p>
<ul>
<li> Hidden columns are not listed in the dataset returned by 
     "[PRAGMA table_info]",
<li> Hidden columns are not included in the expansion of a "*"
................................................................................

<p>For example, if the following SQL is passed to [sqlite3_declare_vtab()]:

<blockquote><pre>
   CREATE TABLE x(a HIDDEN VARCHAR(12), b INTEGER, c INTEGER Hidden);
</pre></blockquote>

<p>Then the virtual table would be created with two hidden columns,
and with datatypes of "VARCHAR(12)" and "INTEGER".

<p>The xCreate must should return [SQLITE_OK] if it is successful in 
creating the new virtual table, or [SQLITE_ERROR] if it is not successful.
If not successful, the [sqlite3_vtab] structure must not be allocated. 
An error message may optionally be returned in *pzErr if unsuccessful.

Space to hold the error message string must be allocated using
an SQLite memory allocation function like 
[sqlite3_malloc()] or [sqlite3_mprintf()] as the SQLite core will
attempt to free the space using [sqlite3_free()] after the error has
been reported up to the application.

<p>The xCreate method is required for every virtual table implementation, 
though the xCreate and [xConnect] pointers of the [sqlite3_module] object
may point to the same function the virtual table does not need to initialize
backing store.

<tcl>############################################################# xConnect
hd_fragment xconnect {sqlite3_module.xConnect} {xConnect}</tcl>
<h3>2.2 The xConnect Method</h3>

<blockquote><pre>
  int (*xConnect)(sqlite3*, void *pAux,
               int argc, char **argv,
               sqlite3_vtab **ppVTab,
               char **pzErr);
</pre></blockquote>

<p>The xConnect method is very similar to [xCreate]. 
It has the same parameters and constructs a new [sqlite3_vtab] structure 
just like xCreate. 
And it must also call [sqlite3_declare_vtab()] like xCreate.

<p>The difference is that xConnect is called to establish a new 
connection to an existing virtual table whereas xCreate is called 
to create a new virtual table from scratch.
................................................................................

<p>Another example is a virtual table that implements a full-text index. 
The xCreate method must create and initialize data structures to hold 
the dictionary and posting lists for that index. The xConnect method,
on the other hand, only has to locate and use an existing dictionary 
and posting lists that were created by a prior xCreate call.

<p>The xConnect method must return [SQLITE_OK] if it is successful 
in creating the new virtual table, or [SQLITE_ERROR] if it is not 
successful. If not successful, the [sqlite3_vtab] structure must not be 
allocated. An error message may optionally be returned in *pzErr if 
unsuccessful. 
Space to hold the error message string must be allocated using
an SQLite memory allocation function like 
[sqlite3_malloc()] or [sqlite3_mprintf()] as the SQLite core will
attempt to free the space using [sqlite3_free()] after the error has
been reported up to the application.

<p>The xConnect method is required for every virtual table implementation, 
though the [xCreate] and xConnect pointers of the [sqlite3_module] object
may point to the same function the virtual table does not need to initialize
backing store.

<tcl>############################################################ xBestIndex
hd_fragment xbestindex {sqlite3_module.xBestIndex} {xBestIndex}</tcl>
<h3>2.3 The xBestIndex Method</h3>

<p>SQLite uses the xBestIndex method of a virtual table module to determine
the best way to access the virtual table. 
The xBestIndex method has a prototype like this:

<blockquote><pre>
  int (*xBestIndex)(sqlite3_vtab *pVTab, sqlite3_index_info*);
</pre></blockquote>

<p>The SQLite core communicates with the xBestIndex method by filling 
in certain fields of the [sqlite3_index_info] structure and passing a 
................................................................................
  #define SQLITE_INDEX_CONSTRAINT_LE    8
  #define SQLITE_INDEX_CONSTRAINT_LT    16
  #define SQLITE_INDEX_CONSTRAINT_GE    32
  #define SQLITE_INDEX_CONSTRAINT_MATCH 64
</pre></blockquote>

<p>The SQLite core calls the xBestIndex method when it is compiling a query
that involves a virtual table. In other words, SQLite calls this method 
when it is running [sqlite3_prepare()] or the equivalent. 
By calling this method, the 
SQLite core is saying to the virtual table that it needs to access 
some subset of the rows in the virtual table and it wants to know the
most efficient way to do that access. The xBestIndex method replies 
with information that the SQLite core can then use to conduct an 
efficient search of the virtual table.

<p>While compiling a single SQL query, the SQLite core might call 
xBestIndex multiple times with different settings in [sqlite3_index_info].
The SQLite core will then select the combination that appears to 
give the best performance.



<p>Before calling this method, the SQLite core initializes an instance 
of the [sqlite3_index_info] structure with information about the
query that it is currently trying to process. This information 
derives mainly from the WHERE clause and ORDER BY or GROUP BY clauses 
of the query, but also from any ON or USING clauses if the query is a 
join. The information that the SQLite core provides to the xBestIndex 
method is held in the part of the structure that is marked as "Inputs". 
The "Outputs" section is initialized to zero.

<p>The information in the [sqlite3_index_info] structure is ephemeral
and may be overwritten or deallocated as soon as the xBestIndex method
returns.  If the xBestIndex method needs to remember any part of the
[sqlite3_index_info] structure, it should make a copy.  Care must be
take to store the copy in a place where it will be deallocated, such
as in the idxStr field with needToFreeIdxStr set to 1.

<p>Note that xBestIndex will always be called before [xFilter], since
the idxNum and idxStr outputs from xBestIndex are required inputs to
xFilter.  However, there is no guarantee that xFilter will be called
following a successful xBestIndex.  

<p>The xBestIndex method is required for every virtual table implementation.

<h4>2.3.1 Inputs</h4>

<p>The main thing that the SQLite core is trying to communicate to 
the virtual table is the constraints that are available to limit 
the number of rows that need to be searched. The aConstraint[] array 
contains one entry for each constraint. There will be exactly 
nConstraint entries in that array.

................................................................................
if the WHERE clause contained a term like this:

<blockquote><pre>
     a = 5
</pre></blockquote>

<p>Then one of the constraints would be on the "a" column with 
operator "=" and an expression of "5". Constraints need not have a
literal representation of the WHERE clause. The query optimizer might
make transformations to the 
WHERE clause in order to extract as many constraints 
as it can. So, for example, if the WHERE clause contained something 
like this:

<blockquote><pre>
     x BETWEEN 10 AND 100 AND 999>y
</pre></blockquote>

<p>The query optimizer might translate this into three separate constraints:

<blockquote><pre>
     x >= 10
     x <= 100
     y < 999
</pre></blockquote>

................................................................................
<p>For each constraint, the aConstraint[].iColumn field indicates which 
column appears on the left-hand side of the constraint.
The first column of the virtual table is column 0. 
The rowid of the virtual table is column -1. 
The aConstraint[].op field indicates which operator is used. 
The SQLITE_INDEX_CONSTRAINT_* constants map integer constants 
into operator values.
Columns occur in the order they were defined by the call to
[sqlite3_declare_vtab()] in the [xCreate] or [xConnect] method.
Hidden columns are counted when determining the column index.

<p>The aConstraint[] array contains information about all constraints 
that apply to the virtual table. But some of the constraints might
not be usable because of the way tables are ordered in a join. 
The xBestIndex method must therefore only consider constraints 
that have a aConstraint[].usable flag which is true.

<p>In addition to WHERE clause constraints, the SQLite core also 
tells the xBestIndex method about the ORDER BY clause. 
(In an aggregate query, the SQLite core might put in GROUP BY clause 
information in place of the ORDER BY clause information, but this fact
should not make any difference to the xBestIndex method.) 
If all terms of the ORDER BY clause are columns in the virtual table, 
then nOrderBy will be the number of terms in the ORDER BY clause 
and the aOrderBy[] array will identify the column for each term 
in the order by clause and whether or not that column is ASC or DESC.

<h4>2.3.2 Outputs</h4>

<p>Given all of the information above, the job of the xBestIndex 
method it to figure out the best way to search the virtual table.

<p>The xBestIndex method fills the idxNum and idxStr fields with 
information that communicates an indexing strategy to the [xFilter] 
method. The information in idxNum and idxStr is arbitrary as far 
as the SQLite core is concerned. The SQLite core just copies the 
information through to the xFilter method. Any desired meaning can 
be assigned to idxNum and idxStr as long as xBestIndex and xFilter 
agree on what that meaning is.

<p>The idxStr value may be a string obtained from an SQLite
memory allocation function such as [sqlite3_mprintf()]. 
If this is the case, then the needToFreeIdxStr flag must be set to 
true so that the SQLite core will know to call [sqlite3_free()] on 
that string when it has finished with it, and thus avoid a memory leak.

<p>If the virtual table will output rows in the order specified by 
the ORDER BY clause, then the orderByConsumed flag may be set to 
true. If the output is not automatically in the correct order 
then orderByConsumed must be left in its default false setting. 
This will indicate to the SQLite core that it will need to do a 
separate sorting pass over the data after it comes out of the virtual table.

<p>The estimatedCost field should be set to the estimated number
of disk access operations required to execute this query against 
the virtual table. The SQLite core will often call xBestIndex 
multiple times with different constraints, obtain multiple cost
................................................................................
the EXPR value of the aConstraint[3] constraint.

<p>By default, the SQLite core double checks all constraints on 
each row of the virtual table that it receives. If such a check 
is redundant, the xBestFilter method can suppress that check by 
setting aConstraintUsage[].omit.

<tcl>########################################################## xDisconnect
hd_fragment xdisconnect {sqlite3_module.xDisconnect} {xDisconnect}</tcl>
<h3>2.4 The xDisconnect Method</h3>

<blockquote><pre>
  int (*xDisconnect)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method releases a connection to a virtual table. 
Only the [sqlite3_vtab] object is destroyed.
The virtual table is not destroyed and any backing store 
associated with the virtual table persists. 

This method undoes the work of [xConnect].

<p>This method is a destructor for a connection to the virtual table.
Constrast this method with [xDestroy].  The xDestroy is a destructor
for the entire virtual table.

<p>The xDestroy method is required for every virtual table implementation,
though it is acceptable for the [xDisconnect] and xDestroy methods to be
the same function if that makes sense for the particular virtual table.

<tcl>########################################################## xDestroy
hd_fragment {sqlite3_module.xDestroy} {xDestroy}</tcl>
<h3>2.5 The xDestroy Method</h3>

<blockquote><pre>
  int (*xDestroy)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method releases a connection to a virtual table, just like 
the [xDisconnect] method, and it also destroys the underlying 
table implementation. This method undoes the work of [xCreate].

<p>The [xDisconnect] method is called whenever a database connection
that uses a virtual table is closed. The xDestroy method is only 
called when a [DROP TABLE] statement is executed against the virtual table.

<p>The xDisconnect method is required for every virtual table implementation,
though it is acceptable for the xDisconnect and [xDestroy] methods to be
the same function if that makes sense for the particular virtual table.

<tcl>########################################################## xOpen
hd_fragment xopen {sqlite3_module.xOpen}</tcl>
<h3>2.6 The xOpen Method</h3>

<blockquote><pre>
  int (*xOpen)(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor);
</pre></blockquote>

<p>The xOpen method creates a new cursor used for accessing (read and/or
writing) a virtual table.  A successful invocation of this method 
will allocate the memory for the [sqlite3_vtab_cursor] (or a subclass),
initialize the new object, and make *ppCursor point to the new object.
The successful call then returns [SQLITE_OK].

<p>For every successful call to this method, the SQLite core will
later invoke the [sqlite3_module.xClose | xClose] method to destroy 
the allocated cursor.

<p>A virtual table implementation must be able to support an arbitrary
number of simultaneously open cursors.

<p>When initially opened, the cursor is in an undefined state.
The SQLite core will invoke the [xFilter] method
on the cursor prior to any attempt to position or read from the cursor.

<p>The xOpen method is required for every virtual table implementation.

<tcl>############################################################### xClose
hd_fragment xclose {sqlite3_module.xClose}</tcl>
<h3>2.7 The xClose Method</h3>

<blockquote><pre>
  int (*xClose)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xClose method closes a cursor previously opened by 
[sqlite3_module.xOpen | xOpen]. 
The SQLite core will always call xClose once for each cursor opened 
using xOpen.

<p>This method must release all resources allocated by the
corresponding xOpen call. The routine will not be called again even if it
returns an error.  The SQLite core will not use the
[sqlite3_vtab_cursor] again after it has been closed.

<p>The xClose method is required for every virtual table implementation.

<tcl>############################################################## xEof
hd_fragment xeof {sqlite3_module.xEof} {xEof}</tcl>
<h3>2.8 The xEof Method</h3>

<blockquote><pre>
  int (*xEof)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xEof method must return false (zero) if the specified cursor 
currently points to a valid row of data, or true (non-zero) otherwise. 
This method is called by the SQL engine immediately after each 
[xFilter] and [xNext] invocation.

<p>The xEof method is required for every virtual table implementation.

<tcl>############################################################## xFilter
hd_fragment xfilter {sqlite3_module.xFilter} {xFilter}</tcl>
<h3>2.9 The xFilter Method</h3>

<blockquote><pre>
  int (*xFilter)(sqlite3_vtab_cursor*, int idxNum, const char *idxStr,
                int argc, sqlite3_value **argv);
</pre></blockquote>

<p>This method begins a search of a virtual table. 
The first argument is a cursor opened by [sqlite3_module.xOpen | xOpen]. 
The next two argument define a particular search index previously 
choosen by [xBestIndex]. The specific meanings of idxNum and idxStr 
are unimportant as long as xFilter and xBestIndex agree on what 
that meaning is.

<p>The xBestIndex function may have requested the values of 
certain expressions using the aConstraintUsage[].argvIndex values 
of the [sqlite3_index_info] structure. 
Those values are passed to xFilter using the argc and argv parameters.

<p>If the virtual table contains one or more rows that match the
search criteria, then the cursor must be left point at the first row.
Subsequent calls to [xEof] must return false (zero).
If there are no rows match, then the cursor must be left in a state 
that will cause the [xEof] to return true (non-zero).
The SQLite engine will use
the [xColumn] and [xRowid] methods to access that row content.
The [xNext] method will be used to advance to the next row.

<p>This method must return [SQLITE_OK] if successful, or an sqlite 
[error code] if an error occurs.

<p>The xFilter method is required for every virtual table implementation.

<tcl>############################################################### xNext
hd_fragment xnext {sqlite3_module.xNext} {xNext}</tcl>
<h3>2.10 The xNext Method</h3>

<blockquote><pre>
  int (*xNext)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xNext method advances a [virtual table cursor] to the next row 
of a result set initiated by [xFilter]. 
If the cursor is already pointing at the last row when this 
routine is called, then the cursor no longer points to valid 
data and a subsequent call to the [xEof] method must return true (non-zero). 
If the cursor is successfully advanced to another row of content, then
subsequent calls to [xEof] must return false (zero).

<p>This method must return [SQLITE_OK] if successful, or an sqlite 
[error code] if an error occurs.

<p>The xNext method is required for every virtual table implementation.

<tcl>############################################################## xColumn
hd_fragment xcolumn {sqlite3_module.xColumn} {xColumn}</tcl>
<h3>2.11 The xColumn Method</h3>

<blockquote><pre>
  int (*xColumn)(sqlite3_vtab_cursor*, sqlite3_context*, int N);
</pre></blockquote>

<p>The SQLite core invokes this method in order to find the value for 
the N-th column of the current row. N is zero-based so the first column 
is numbered 0. The xColumn method must uses one of the 
[sqlite3_result_blob | sqlite3_result_*() APIs] 
to return the result. 
The xColumn method returns its result back to SQLite using one of the
following interface:

<p>
<li> [sqlite3_result_blob()]
<li> [sqlite3_result_double()]
<li> [sqlite3_result_int()]
<li> [sqlite3_result_int64()]
<li> [sqlite3_result_null()]
<li> [sqlite3_result_text()]
<li> [sqlite3_result_text16()]
<li> [sqlite3_result_text16le()]
<li> [sqlite3_result_text16be()]
<li> [sqlite3_result_zeroblob()]
</p>

<p>To raise an error, the xColumn method should use one of the result_text() 
methods to set the error message text, then return an appropriate
[error code].  The xColumn method must return [SQLITE_OK] on success.

<p>The xColumn method is required for every virtual table implementation.

<tcl>############################################################# xRowid
hd_fragment xrowid {sqlite3_module.xRowid} {xRowid}</tcl>
<h3>2.12 The xRowid Method</h3>

<blockquote><pre>
  int (*xRowid)(sqlite3_vtab_cursor *pCur, sqlite_int64 *pRowid);
</pre></blockquote>

<p>A successful invocation of this method will cause *pRowid to be
filled with the [rowid] of row that the
[virtual table cursor] pCur is currently pointing at.
This method returns [SQLITE_OK] on success.
It returns an appropriate [error code] on failure.</p>

<p>The xRowid method is required for every virtual table implementation.

<tcl>############################################################# xUpdate
hd_fragment xupdate {sqlite3_module.xUpdate} {xUpdate}</tcl>
<h3>2.13 The xUpdate Method</h3>

<blockquote><pre>
  int (*xUpdate)(
    sqlite3_vtab *pVTab,
    int argc,
    sqlite3_value **argv,
................................................................................
<p>All changes to a virtual table are made using the xUpdate method.
This one method can be used to insert, delete, or update.

<p>The argc parameter specifies the number of entries in the argv array. 
Every argv entry will have a non-NULL value in C (but may contain the 
SQL value NULL).

<p>The argv[0] parameter is the [rowid] of a row in the virtual table 
to be deleted. If argv[0] is NULL, then no deletion occurs.

<p>The argv[1] parameter is the rowid of a new row to be inserted 
into the virtual table. If argv[1] is NULL, then the implementation 
must choose a rowid for the newly inserted row. Subsequent argv[] 
entries contain values of the columns of the virtual table, in the 
order that the columns were declared. The number of columns will
match the table declaration that the [xConnect] or [xCreate] method made 
using the [sqlite3_declare_vtab()] call.  All hidden columns are included.

<p>When doing an insert without a rowid (argc>1, argv[1]==NULL), the 
implementation must set *pRowid to the rowid of the newly inserted row; 
this will become the value returned by the [sqlite3_last_insert_rowid()]
function. Setting this value in all the other cases is a harmless no-op;
the SQLite engine ignores the *pRowid return value if argc==1 or 
argv[1]!=NULL.

<p>Each call to xUpdate will fall into one of the following cases:

<blockquote>
<dl>
<dt><b>argc == 1</b>
<dd><p>The single row argv[0] is deleted; no insert occurs
................................................................................
when an SQL statement updates a rowid, as in the statement:
<blockquote>
   [UPDATE] table SET rowid=rowid+1 WHERE ...; 
</blockquote>
</dl>
</blockquote>

<p>The xUpdate method must return [SQLITE_OK] if and only if it is
successful.  If a failure occurs, the xUpdate must return an appropriate
[error code].  On a failure, the pVTab->zErrMsg element may optionally
be replaced with error message text stored in memory allocated from SQLite 
using functions such as [sqlite3_mprintf()] or [sqlite3_malloc()].

<p>If the xUpdate method violates some constraint of the virtual table
(including, but not limited to, attempting to store a value of the wrong 
datatype, attempting to store a value that is too
large or too small, or attempting to change a read-only value) then the
xUpdate must fail with an appropriate [error code].

<p>There might be one or more [sqlite3_vtab_cursor] objects open and in use 
on the virtual table instance and perhaps even on the row of the virtual
table when the xUpdate method is invoked.  The implementation of
xUpdate must be prepared for attempts to delete or modify rows of the table
out from other existing cursors.  If the virtual table cannot accommodate
such changes, the xUpdate method must return an [error code].

<p>The xUpdate method is optional.
If the xUpdate pointer in the [sqlite3_module] for a virtual table
is a NULL pointer, then the virtual table is read-only.


<tcl>########################################################## xFindFunction
hd_fragment xfindfunction {sqlite3_module.xFindFunction} {xFindFunction}</tcl>
<h3>2.14 The xFindFunction Method</h3>

<blockquote><pre>
  int (*xFindFunction)(
    sqlite3_vtab *pVtab,
    int nArg,
    const char *zName,
    void (**pxFunc)(sqlite3_context*,int,sqlite3_value**),
    void **ppArg
  );
</pre></blockquote>

<p>This method is called during [sqlite3_prepare()] to give the virtual
table implementation an opportunity to overload functions. 
This method may be set to NULL in which case no overloading occurs.

<p>When a function uses a column from a virtual table as its first 
argument, this method is called to see if the virtual table would 
like to overload the function. The first three parameters are inputs: 
the virtual table, the number of arguments to the function, and the 
name of the function. If no overloading is desired, this method
returns 0. To overload the function, this method writes the new 
function implementation into *pxFunc and writes user data into *ppArg 
and returns 1.

<p>Note that infix functions ([LIKE], [GLOB], [REGEXP], and [MATCH]) reverse 
the order of their arguments. So "like(A,B)" is equivalent to "B like A". 
For the form "B like A" the B term is considered the first argument 
to the function. But for "like(A,B)" the A term is considered the 
first argument.

<p>The function pointer returned by this routine must be valid for
the lifetime of the [sqlite3_vtab] object given in the first parameter.

<tcl>############################################################ xBegin
hd_fragment xBegin {sqlite3_module.xBegin} {xBegin}</tcl>
<h3>2.15 The xBegin Method</h3>

<blockquote><pre>
  int (*xBegin)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method begins a transaction on a virtual table.
This is method is optional.  The xBegin pointer of [sqlite3_module]
may be NULL.

<p>This method is always followed by one call to either the
[xCommit] or [xRollback] method.  Virtual table transactions do
not nest, so the xBegin method will not be invoked more than once
on a single virtual table
without an intervening call to either [xCommit] or [xRollback].
Multiple calls to other methods can and likely will occur in between
the xBegin and the corresponding [xCommit] or [xRollback].

<tcl>############################################################ xSync
hd_fragment xsync {sqlite3_module.xSync}</tcl>
<h3>2.16 The xSync Method</h3>

<blockquote><pre>
  int (*xSync)(sqlite3_vtab *pVTab);
</pre></blockquote>



<p>This method signals the start of a two-phase commit on a virtual
table.
This is method is optional.  The xSync pointer of [sqlite3_module]
may be NULL.

<p>This method is only invoked after call to the [xBegin] method and
prior to an [xCommit] or [xRollback].  In order to implement two-phase
commit, the xSync method on all virtual tables is invoked prior to
invoking the [xCommit] method on any virtual table.  If any of the 
xSync methods fail, the entire transaction is rolled back.

<tcl>########################################################### xCommit
hd_fragment xcommit {sqlite3_module.xCommit} {xCommit}</tcl>
<h3>2.17 The xCommit Method</h3>

<blockquote><pre>
  int (*xCommit)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method causes a virtual table transaction to commit.
This is method is optional.  The xCommit pointer of [sqlite3_module]
may be NULL.

<p>A call to this method always follows a prior call to [xBegin] and
[sqlite3_module.xSync|xSync].


<tcl>############################################################## xRollback
hd_fragment xrollback {sqlite3_module.xRollback} {xRollback}</tcl>
<h3>2.18 The xRollback Method</h3>

<blockquote><pre>
  int (*xRollback)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method causes a virtual table transaction to rollback.
This is method is optional.  The xRollback pointer of [sqlite3_module]
may be NULL.

<p>A call to this method always follows a prior call to [xBegin].


<tcl>############################################################# xRename
hd_fragment xrename {sqlite3_module.xRename} {xRename}</tcl>
<h3>2.19 The xRename Method</h3>

<blockquote><pre>
  int (*xRename)(sqlite3_vtab *pVtab, const char *zNew);
</pre></blockquote>

<p>This method provides notification that the virtual table implementation
that the virtual table will be given a new name. 
If this method returns [SQLITE_OK] then SQLite renames the table.
If this method returns an [error code] then the renaming is prevented.

<p>The xRename method is required for every virtual table implementation.

structure (not part of an overflow chain), the page type of the
corresponding pointer-map entry is set to the value 0x05 and the parent
page number field is set to the page number of the parent node in the
B-Tree structure.



HLR H35000
If a <i>journal file</i> contains a well-formed <i>master-journal 
pointer</i>, and the named <i>master-journal file</i> either does
not exist or does not contain the name of the <i>journal file</i>,
then the <i>journal file</i> shall be considered invalid.

HLR H35010
If the first 28 bytes of a <i>journal file</i> do not contain a well-formed
<i>journal header</i>, then the <i>journal file</i> shall be considered
invalid.

HLR H35020
If the journal file exists within the file-system and neither H35000 nor 
H35010 apply, then the journal file shall be considered valid.



HLR H35030
If there exists a valid <i>journal file</i> in the file-system, then the
database <i>page-size</i> in bytes used to interpret the <i>database image</i>
shall be the value stored as a 4-byte big-endian unsigned integer at byte
offset 24 of the <i>journal file</i>.

HLR H35040
If there exists a valid <i>journal file</i> in the file-system, then the
number of pages in the <i>database image</i> shall be the value stored as 
a 4-byte big-endian unsigned integer at byte offset 24 of the 
<i>journal file</i>.



HLR H35050
If there is no valid <i>journal file</i> in the file-system, then the
database <i>page-size</i> in bytes used to interpret the <i>database image</i>
shall be the value stored as a 2-byte big-endian unsigned integer at byte
offset 16 of the <i>database file</i>.

HLR H35060
If there is no valid <i>journal file</i> in the file-system, then the
number of pages in the <i>database image</i> shall be calculated by dividing
the size of the <i>database file</i> in bytes by the database <i>page-size</i>.


HLR H35070
If there exists a valid <i>journal file</i> in the file-system, then the
contents of each page of the <i>database image</i> for which there is a valid
<i>journal record</i> in the <i>journal file</i> shall be read from the
corresponding journal record.

HLR H35080
The contents of all <i>database image</i> pages for which there is no valid
<i>journal record</i> shall be read from the database file.


HLR H35090
A buffer of 28 bytes shall be considered a well-formed journal 
header if it is not excluded by requirements H35180, H35190 or H35200.

HLR H35180
A buffer of 28 bytes shall only be considered a well-formed journal
header if the first eight bytes of the buffer contain the values 0xd9, 
0xd5, 0x05, 0xf9, 0x20, 0xa1, 0x63, and 0xd7, respectively.

HLR H35190
A buffer of 28 bytes shall only be considered a well-formed journal
header if the value stored in the sector size field (the 4-byte big-endian 
unsigned integer at offset 20 of the buffer) contains a value that
is an integer power of two greater than 512.

HLR H35200
A buffer of 28 bytes shall only be considered a well-formed journal
header if the value stored in the page size field (the 4-byte big-endian 
unsigned integer at offset 24 of the buffer) contains a value that
is an integer power of two greater than 512.




HLR H35100
A buffer of (8 + page size) bytes shall be considered a well-formed journal 
record if it is not excluded by requirements H35110 or H35120.

HLR H35110
A journal record shall only be considered to be well-formed if the page number
field contains a value other than zero and the locking-page number, calculated
using the page size found in the first journal header of the journal file that
contains the journal record.

HLR H35120
A journal record shall only be considered to be well-formed if the checksum 
field contains a value equal to the sum of the value stored in the 
checksum-initializer field of the journal header that precedes the record
and the value stored in every 200th byte of the page data field, interpreted
as an 8-bit unsigned integer), starting with byte offset (page-size % 200) and
ending with the byte at byte offset (page-size - 200).

HLR H35130
A buffer shall be considered to contain a well-formed master journal pointer 
record if it is not excluded from this category by requirements H35140,
H35150, H35160 or H35170.

HLR H35140
A buffer shall only be considered to be a well-formed master journal pointer
if the final eight bytes of the buffer contain the values 0xd9, 0xd5, 0x05, 
0xf9, 0x20, 0xa1, 0x63, and 0xd7, respectively.

HLR H35150
A buffer shall only be considered to be a well-formed master journal pointer
if the size of the buffer in bytes is equal to the value stored as a 4-byte 
big-endian unsigned integer starting 16 bytes before the end of the buffer.

HLR H35160
A buffer shall only be considered to be a well-formed master journal pointer
if the first four bytes of the buffer, interpreted as a big-endian unsigned
integer, contain the page number of the locking page (the value
(1 + 2<sup>30</sup> / page-size), where page-size is the value stored in
the page-size field of the first journal header of the journal file).

HLR H35170
A buffer shall only be considered to be a well-formed master journal pointer
if the value stored as a 4-byte big-endian integer starting 12 bytes before
the end of the buffer is equal to the sum of all bytes, each interpreted
as an 8-bit unsigned integer, starting at offset 4 of the buffer and continuing
until offset (buffer-size - 16) (the 17th last byte of the buffer).



HLR H35210
A buffer shall be considered to contain a well-formed journal section 
record if it is not excluded from this category by requirements H35220,
H35230 or H35240.

HLR H35220
A buffer shall only be considered to contain a well-formed journal section 
if the first 28 bytes of it contain a well-formed journal header.

HLR H35230
A buffer shall only be considered to contain a well-formed journal section 
if, beginning at byte offset sector-size, it contains a sequence of 
record-count well-formed journal records. In this case sector-size and
record-count are the integer values stored in the sector size and record
count fields of the journal section's journal header.

HLR H35240
A buffer shall only be considered to contain a well-formed journal section 
if it is an integer multiple of sector-size bytes in size, where sector-size 
is the value stored in the sector size field of the journal section's journal
header.

HLR H35250
A journal record found within a valid journal file shall be considered a valid
journal record if it is not excluded from this category by requirement H35260,
H35270 or H35280.

HLR H35260
A journal record shall only be considered a valid journal record if it and any
other journal records that occur before it within the same journal section are
well-formed.

HLR H35270
A journal record shall only be considered a valid journal record if the journal
section to which it belongs begins with a well-formed journal header.

HLR H35280
A journal record shall only be considered a valid journal record if all journal
sections that occur before the journal section containing the journal record
are well-formed journal sections.

structure (not part of an overflow chain), the page type of the
corresponding pointer-map entry is set to the value 0x05 and the parent
page number field is set to the page number of the parent node in the
B-Tree structure.



HLR H32000
If a <i>journal file</i> contains a well-formed <i>master-journal 
pointer</i>, and the named <i>master-journal file</i> either does
not exist or does not contain the name of the <i>journal file</i>,
then the <i>journal file</i> shall be considered invalid.

HLR H32010
If the first 28 bytes of a <i>journal file</i> do not contain a well-formed
<i>journal header</i>, then the <i>journal file</i> shall be considered
invalid.

HLR H32020
If the journal file exists within the file-system and neither H32000 nor 
H32010 apply, then the journal file shall be considered valid.



HLR H32030
If there exists a valid <i>journal file</i> in the file-system, then the
database <i>page-size</i> in bytes used to interpret the <i>database image</i>
shall be the value stored as a 4-byte big-endian unsigned integer at byte
offset 24 of the <i>journal file</i>.

HLR H32040
If there exists a valid <i>journal file</i> in the file-system, then the
number of pages in the <i>database image</i> shall be the value stored as 
a 4-byte big-endian unsigned integer at byte offset 24 of the 
<i>journal file</i>.



HLR H32050
If there is no valid <i>journal file</i> in the file-system, then the
database <i>page-size</i> in bytes used to interpret the <i>database image</i>
shall be the value stored as a 2-byte big-endian unsigned integer at byte
offset 16 of the <i>database file</i>.

HLR H32060
If there is no valid <i>journal file</i> in the file-system, then the
number of pages in the <i>database image</i> shall be calculated by dividing
the size of the <i>database file</i> in bytes by the database <i>page-size</i>.


HLR H32070
If there exists a valid <i>journal file</i> in the file-system, then the
contents of each page of the <i>database image</i> for which there is a valid
<i>journal record</i> in the <i>journal file</i> shall be read from the
corresponding journal record.

HLR H32080
The contents of all <i>database image</i> pages for which there is no valid
<i>journal record</i> shall be read from the database file.


HLR H32090
A buffer of 28 bytes shall be considered a well-formed journal 
header if it is not excluded by requirements H32180, H32190 or H32200.

HLR H32180
A buffer of 28 bytes shall only be considered a well-formed journal
header if the first eight bytes of the buffer contain the values 0xd9, 
0xd5, 0x05, 0xf9, 0x20, 0xa1, 0x63, and 0xd7, respectively.

HLR H32190
A buffer of 28 bytes shall only be considered a well-formed journal
header if the value stored in the sector size field (the 4-byte big-endian 
unsigned integer at offset 20 of the buffer) contains a value that
is an integer power of two greater than 512.

HLR H32200
A buffer of 28 bytes shall only be considered a well-formed journal
header if the value stored in the page size field (the 4-byte big-endian 
unsigned integer at offset 24 of the buffer) contains a value that
is an integer power of two greater than 512.




HLR H32100
A buffer of (8 + page size) bytes shall be considered a well-formed journal 
record if it is not excluded by requirements H32110 or H32120.

HLR H32110
A journal record shall only be considered to be well-formed if the page number
field contains a value other than zero and the locking-page number, calculated
using the page size found in the first journal header of the journal file that
contains the journal record.

HLR H32120
A journal record shall only be considered to be well-formed if the checksum 
field contains a value equal to the sum of the value stored in the 
checksum-initializer field of the journal header that precedes the record
and the value stored in every 200th byte of the page data field, interpreted
as an 8-bit unsigned integer), starting with byte offset (page-size % 200) and
ending with the byte at byte offset (page-size - 200).

HLR H32130
A buffer shall be considered to contain a well-formed master journal pointer 
record if it is not excluded from this category by requirements H32140,
H32150, H32160 or H32170.

HLR H32140
A buffer shall only be considered to be a well-formed master journal pointer
if the final eight bytes of the buffer contain the values 0xd9, 0xd5, 0x05, 
0xf9, 0x20, 0xa1, 0x63, and 0xd7, respectively.

HLR H32150
A buffer shall only be considered to be a well-formed master journal pointer
if the size of the buffer in bytes is equal to the value stored as a 4-byte 
big-endian unsigned integer starting 16 bytes before the end of the buffer.

HLR H32160
A buffer shall only be considered to be a well-formed master journal pointer
if the first four bytes of the buffer, interpreted as a big-endian unsigned
integer, contain the page number of the locking page (the value
(1 + 2<sup>30</sup> / page-size), where page-size is the value stored in
the page-size field of the first journal header of the journal file).

HLR H32170
A buffer shall only be considered to be a well-formed master journal pointer
if the value stored as a 4-byte big-endian integer starting 12 bytes before
the end of the buffer is equal to the sum of all bytes, each interpreted
as an 8-bit unsigned integer, starting at offset 4 of the buffer and continuing
until offset (buffer-size - 16) (the 17th last byte of the buffer).



HLR H32210
A buffer shall be considered to contain a well-formed journal section 
record if it is not excluded from this category by requirements H32220,
H32230 or H32240.

HLR H32220
A buffer shall only be considered to contain a well-formed journal section 
if the first 28 bytes of it contain a well-formed journal header.

HLR H32230
A buffer shall only be considered to contain a well-formed journal section 
if, beginning at byte offset sector-size, it contains a sequence of 
record-count well-formed journal records. In this case sector-size and
record-count are the integer values stored in the sector size and record
count fields of the journal section's journal header.

HLR H32240
A buffer shall only be considered to contain a well-formed journal section 
if it is an integer multiple of sector-size bytes in size, where sector-size 
is the value stored in the sector size field of the journal section's journal
header.

HLR H32250
A journal record found within a valid journal file shall be considered a valid
journal record if it is not excluded from this category by requirement H32260,
H32270 or H32280.

HLR H32260
A journal record shall only be considered a valid journal record if it and any
other journal records that occur before it within the same journal section are
well-formed.

HLR H32270
A journal record shall only be considered a valid journal record if the journal
section to which it belongs begins with a well-formed journal header.

HLR H32280
A journal record shall only be considered a valid journal record if all journal
sections that occur before the journal section containing the journal record
are well-formed journal sections.