Documentation Source Text

    hd_close_aux
    hd_enable_main 1
  }
}

chng {2013-08-15 (3.8.0)} {
<li>Cut-over to the [next generation query planner] for faster and better query plans.
<li>The [EXPLAIN QUERY PLAN] output no longer shows an estimate of the number of 
    rows generated by each loop in a join.
<li>Added the [FTS4 notindexed option], allowing non-indexed columns in an FTS4 table.
<li>Added the [SQLITE_STMTSTATUS_VM_STEP] option to [sqlite3_stmt_status()].
<li>Added the "percentile()" function as a [loadable extension] in the ext/misc
    subdirectory of the source tree.
<li>A running SELECT statement that lacks a FROM clause (or any other statement that
    never reads or writes from any database file) will not prevent a read
    transaction from closing.
<li>Add the [SQLITE_DEFAULT_AUTOMATIC_INDEX] compile-time option.  Setting this option
    to 0 disables automatic indices by default.
<li>Issue an [SQLITE_WARNING_AUTOINDEX] warning on the [SQLITE_CONFIG_LOG] whenever
    the query planner uses an automatic index.
<li>Added the [SQLITE_FTS3_MAX_EXPR_DEPTH] compile-time option.
<li>Added an optional 5th parameter defining the collating sequence to the 
    next_char() extension SQL function.
<li>The [SQLITE_BUSY_SNAPSHOT] extended error code is returned in WAL mode when
    a read transaction cannot be upgraded to a write transaction because the read is
    on an older snapshot.
<li>Allow read transactions to be freely opened and closed by SQL statements run 
    from within the implementation of [application-defined SQL functions] if the
    function is called by a SELECT statement that does not access any database table.
<li>Disable the use of posix_fallocate() on all (unix) systems unless the
    HAVE_POSIX_FALLOCATE compile-time option is used.
<li>Update the ".import" command in the [command-line shell] to support multi-line
    fields and correct RFC-4180 quoting and to issue warning and/or error messages
    if the input text is not strictly RFC-4180 compliant.
<li>Bug fix: In the [unicode61] tokenizer of [FTS4], treat all private code points
    as identifier symbols.
<li>Bug fixes: Multiple problems in the legacy query optimizer were fixed by the 
    move to [NGQP].
}

chng {2013-05-20 (3.7.17)} {
<li>Add support for [memory-mapped I/O].
<li>Add the [sqlite3_strglob()] convenience interface.
<li>Assigned the integer at offset 68 in the [database header] as the
    [Application ID] for when SQLite is used as an [application file-format].

}

COMPILE_OPTION {SQLITE_DEFAULT_WAL_AUTOCHECKPOINT=<i>&lt;pages&gt;</i>} {
  This macro sets the default page count for the [WAL]
  [checkpointing | automatic checkpointing] feature.  If unspecified,
  the default page count is 1000.
}

COMPILE_OPTION {SQLITE_FTS3_MAX_EXPR_DEPTH=<i>N</i>} {
  This macro sets the maximum depth of the search tree that corresponds to
  the right-hand side of the MATCH operator in an [FTS3] or [FTS4] full-text
  index.  The full-text search uses a recursive algorithm, so the depth of
  the tree is limited to prevent using too much stack space.  The default
  limit is 12.  This limit is sufficient for up to 4095 search terms on the
  right-hand side of the MATCH operator and it holds stack space usage to 
  less than 2000 bytes.
  <p>
  For ordinary FTS3/FTS4 queries, the search tree depth is approximately
  the base-2 logarithm of the number of terms in the right-hand side of the
  MATCH operator.  However, for [phrase queries] and [NEAR queries] the
  search tree depth is linear in the number of right-hand side terms.
  So the default depth limit of 12 is sufficient for up to 4095 ordinary
  terms on a MATCH, it is only sufficient for 11 or 12 phrase or NEAR
  terms.  Even so, the default is more than enough for most application.
}

COMPILE_OPTION {SQLITE_MAX_MMAP_SIZE=<i>N</i>} {
  This macro sets a hard upper bound on the amount of address space that
  can be used by any single database for memory-mapped I/O.
  Setting this value to 0 completely disables memory-mapped I/O and
  causes logic associated with memory-mapped I/O to be omitted from the
  build.  This option does change the default memory-mapped I/O address

  <i>-- Query the database for documents for which the term "linux" appears in</i>
  <i>-- the document title, and the term "driver" appears in the body of the document</i>
  <i>-- ("driver" may also appear in the title, but this alone will not satisfy the</i>.
  <i>-- query criteria).</i>
  SELECT * FROM docs WHERE body MATCH 'title:linux driver';
</codeblock>

<tcl>hd_fragment phrase {phrase queries}</tcl>
<ul>
  <li><p><b>Phrase queries</b>.
    A phrase query is a query that retrieves all documents that contain a
    nominated set of terms or term prefixes in a specified order with no
    intervening tokens. Phrase queries are specified by enclosing a space
    separated sequence of terms or term prefixes in double quotes (").
    For example:
</ul>

<codeblock>
  <i>-- Query for all documents that contain the phrase "linux applications".</i>
  SELECT * FROM docs WHERE docs MATCH '"linux applications"';

  <i>-- Query for all documents that contain a phrase that matches "lin* app*". As well as</i>
  <i>-- "linux applications", this will match common phrases such as "linoleum appliances"</i>
  <i>-- or "link apprentice".</i>
  SELECT * FROM docs WHERE docs MATCH '"lin* app*"';
</codeblock>

<tcl>hd_fragment near {NEAR queries}</tcl>
<ul>
  <li><p><b>NEAR queries</b>. 
    A NEAR query is a query that returns documents that contain a two or
    more nominated terms or phrases within a specified proximity of each 
    other (by default with 10 or less intervening terms). A NEAR query is 
    specified by putting the keyword "NEAR" between two phrase, term or 
    [prefix queries]. To specify a proximity other than the default,

<title>Isolation In SQLite</title>
<tcl>hd_keywords isolation</tcl>

<h1 align="center">
Isolation In SQLite
</h1>

<p>
The "isolation" property of a database determines when changes made to 
the database by one operation become visible to other concurrent operations.
</p>

<h2>Isolation Between Database Connections</h2>

<p>
If the same database is being read and written using two different
[database connections] (two different [sqlite3] objects returned by
separate calls to [sqlite3_open()]) and the two database connections
do not have a [shared cache], then the reader is only able to
see complete committed transactions from the writer.  Partial changes
by the writer that have not been committed are invisible to the reader.
This is true regardless of whether the two database connections are in
the same thread, the different threads of the same process, or in
different processes.  This
is the usual and expected behavior for SQL database systems.
</p>

<p>
The previous paragraph is also true (separate database connections are
isolated from one another) in [shared cache mode] as long as the
[read_uncommitted pragma] remains turned off.  The [read_uncommitted pragma]
is off by default and so if the application does nothing to turn it on, 
it will remain off.  Hence, unless the [read_uncommitted pragma] is used
to change the default behavior, changes made by one database connection
are invisible to readers on a different database connection sharing the
same cache until the writer commits its transaction.
</p>

<p>
If two database connections shared the same cache and the reader has 
enabled the [read_uncommitted pragma], then the reader will be able to
see changes made by the writer before the writer transaction commits.
The combined use of [shared cache mode] and the [read_uncommitted pragma] 
is the only way that one database connection can see uncommitted changes
on a different database connection.  In all other circumstances, separate
database connections are completely isolated from one another.
</p>

<p>Except in the case of [shared cache] database connections with
[PRAGMA read_uncommitted] turned on, all transactions in SQLite show
"serializable" isolation.  SQLite implements serializable tranactions
by actually serializing the writes.  There can only be a single writer
at a time to an SQLite database.  There can be multiple database connections
open at the same time, and all of those database connections can write
to the database file, but they have to take turns.  SQLite uses locks
to serialization of the writes automatically; this is not something that
the applications using SQLite need to worry with.</p>


<h2>Isolation And Concurrency</h2>

<p>
SQLite implements isolation and concurrency control (and atomicity) using
transient
journal files that appear in the same directory in as the database file.
There are two major "journal modes".
The older "rollback mode" corresponds to using the "DELETE", "PERSIST",
or "TRUNCATE" options to the [journal_mode pragma].  In rollback mode,
changes are written directly into the database file, while simultaneously
a separate rollback journal file is constructed that is able to restore
the database to its original state if the transaction rolls back.
Rollback mode (specifically DELETE mode, meaning that the rollback journal
is deleted from disk at the conclusion of each transaction) is the current
default behavior.
</p>

<p>Since version 3.7.0, SQLite also supports "[WAL mode]".  In WAL mode,
changes are not written to the original database file.  Instead, changes
go into a separate "write-ahead log" or "WAL" file.  
Later, after the transaction
commits, those changes will be moved from the WAL file back into the
original database in an operation called "checkpoint".  WAL mode is
enabled by running "[PRAGMA journal_mode|PRAGMA journal_mode=WAL]".
</p>

<p>
In rollback mode, SQLite implements isolation by locking the database
file and preventing any reads by other database connections
while each write transaction is underway.
Readers can be be active at the beginning of a write, before any content
is flushed to disk and while all changes are still held in the writer's
private memory space.  But before any changes are made to the database file
on disk, all readers must be (temporally) expelled in order to give the writer
exclusive access to the database file.  
Hence, readers are prohibited from seeing incomplete
transactions by virtue of being locked out of the database while the
transaction is being written to disk.  Only after the transaction is
complete written and synced to disk and commits are the readers allowed
back into the database.  Hence readers never get a chance to see partially
written changes.
</p>

<p>
WAL mode permits simultenous readers and writers.  It can do this because
changes do not overwrite the original database file, but rather go
into the separate write-ahead log file.  That means that readers can continue
to read the old, original, unaltered content from the original database file
at the same time that the writer is appending to the write-ahead log.
In [WAL mode], SQLite exhibits "snapshot isolation".  When a read transaction
starts, that reader continues to see an unchanging "snapshot" of the database
file as it existed at the moment in time when the read transaction started.
Any write transactions that commit while the read transaction is
active are still invisible to the read transaction, because the reader is
seeing a snapshot of database file from a prior moment in time.
</p>

<p>
An example:  Suppose there are two database connections X and Y.  X starts
a read transaction using [BEGIN] followed by one or more [SELECT] statements.
Then Y comes along and runs an [UPDATE] statement to modify the database.
X can subsequently do a [SELECT] against the records that Y modified but
X will see the older unmodified entries because Y's changes are all
invisible to X while X is holding a read transaction.  If X wants to see
the changes that Y made, then X must ends its read transaction and
start a new one (by running [COMMIT] followed by another [BEGIN].)
</p>

<p>
Another example: X starts a read tranaction using [BEGIN] and [SELECT], then
Y makes a changes to the database using [UPDATE].  Then X tries to make a
change to the database using [UPDATE].  The attempt by X to escalate its
transaction from a read transaction to a write transation fails with an
[SQLITE_BUSY_SNAPSHOT] error because the snapshot of the database being
viewed by X is no longer the latest version of the database.  If X were
allowed to write, it would fork the history of the database file, which is
something SQLite does not support.  In order for X to write to the database,
it must first release its snapshot (using [ROLLBACK] for example) then
start a new transaction with a subsequent [BEGIN].
</p>

<p>
If X starts a transaction that will initially only read but X knows it
will eventually want to write and does not want to be troubled with
possible SQLITE_BUSY_SNAPSHOT errors that arise because another connection
jumped ahead of it in line, then X can issue [BEGIN IMMEDIATE] to start
its transaction instead of just an ordinary BEGIN.  The [BEGIN IMMEDIATE]
command goes ahead and starts a write transaction, and thus blocks all
other writers.  If the [BEGIN IMMEDIATE] operation succeeds, then no
subsequent operations in that transaction will ever fail with an
[SQLITE_BUSY] error.
</p>

<h2>No Isolation Between Operations On The Same Database Connection</h2>

<p>SQLite provides isolation between operations in separate database
connections.  However, there is no isolation between operations that
occur within the same database connection.</p>

<p>In other words, if X begins a write transaction using [BEGIN IMMEDIATE]
then issues one or more [UPDATE], [DELETE], and/or [INSERT]
statements, then those changes are visible to subsequent [SELECT] statements
that are evaluated in database connection X.  [SELECT] statements on
a different  database connection Y will show no changes until the X
transaction commits.  But [SELECT] statements in X will show the changes
prior to the commit.</p>

<p>^Within a single database connection X, a SELECT statement always sees all
changes to the database that are completed prior to the start of the SELECT
statement, whether committed or uncommitted.  And the SELECT statement
obviously does not see any changes that occur after the SELECT statement
completes.  But what about changes that occur while the SELECT statement
is running?  What if a SELECT statement is started and the [sqlite3_step()]
interface steps through roughly half of its output, then some [UPDATE]
statements are run by the application that modify the table that the
SELECT statement is reading, then more calls to [sqlite3_step()] are made
to finish out the SELECT statement?  Will the later steps of the SELECT
statement see the changes made by the UPDATE or not?  The answer is that
this behavior is undefined.  In particular, whether or not the SELECT statement
sees the concurrent changes depends on which release of SQLite is
running, the schema of the database file, whether or not [ANALYZE] has
been run, and the details of the query.  In some cases, it might depend
on the content of the database file, too.  There is no good way to know whether
or not a SELECT statement will see changes that were made to the database
by the same database connection after the SELECT statement was started.
And hence, developers should diligently avoid writing applications
that make assumptions about what will occur in that circumstance.</p>

<p>
If an application issues a SELECT statement on a single table like
"<i>SELECT rowid, * FROM table WHERE ...</i>" and starts stepping through 
the output of that statement using [sqlite3_step()] and examining each
row, then it is safe for the application to delete the current row or
any prior row using "DELETE FROM table WHERE rowid=?".  It is also safe
(in the sense that it will not harm the database) for the application to
delete a row that expected to appear later in the query but has not
appeared yet.  If a future row is deleted, however, it might happen that
the row turns up after a subsequent sqlite3_step(), even after it has
allegedly been deleted.  Or it might not.  That behavior is undefined.
The application can 
also INSERT new rows into the table while the SELECT statement is 
running, but whether or not the new rows appear
in subsequent sqlite3_step()s of the query is undefined.  And the application
can UPDATE the current row or any prior row, though doing so might cause 
that row to reappear in a subsequent sqlite3_step().  As long as the 
application is prepared to deal with these ambiguities, the operations 
themselves are safe and will not harm the database file.</p>

<p>
For the purposes of the previous two paragraphs, two database connections
that have the same [shared cache] and which have enabled
[PRAGMA read_uncommitted] are considered to be the same database connection.
</p>

<h2>Summary</h2>

<ol>
<li><p>
Transactions in SQLite are SERIALIZABLE.
</p>

<li><p>
Changes made in one database connection are invisible to all other database
connections prior to commit.
</p>

<li><p>
A query sees all changes that are completed on the same database connection
prior to the start of the query, regardless of whether or not those changes
have been committed.
</p>

<li><p>
If changes occur on the same database connection after a query 
starts running but before the query completes, then it is undefined whether 
or not the query will see those changes.
</p>

<li><p>
If changes occur on the same database connection after a query 
starts running but before the query completes, then the query might return
a changed row more than once, or it might return a row that was previously
deleted, or not; the behavior is undefined.
</p>

<li><p>
For the purposes of the previous four items, two database connections that 
use the same [shared cache] and which enable [PRAGMA read_uncommitted] are
considered to be the same database connection, not separate database
connections.
</p>
</ol>

transactions.  ^An attempt to invoke the BEGIN command within
a transaction will fail with an error, regardless of whether
the transaction was started by [SAVEPOINT] or a prior BEGIN.
^The COMMIT command and the ROLLBACK command without the TO clause
work the same on [SAVEPOINT] transactions as they do with transactions
started by BEGIN.</p>

<tcl>hd_fragment immmediate {BEGIN IMMEDIATE} {BEGIN EXCLUSIVE}</tcl>
<p>
^Transactions can be deferred, immediate, or exclusive.  
^The default transaction behavior is deferred.
^Deferred means that no locks are acquired
on the database until the database is first accessed.  ^Thus with a
deferred transaction, the BEGIN statement itself does nothing to the
filesystem.  ^Locks

<title>Run-Time Loadable Extensions</title>
<tcl>hd_keywords {loadext} {loadable extensions} {extension loading} \
                 {SQLite extension} {SQLite extensions} \
                 {loadable extension}</tcl>

<h1 align="center">Run-Time Loadable Extensions</h1>

<p>SQLite has the ability to load extensions (including new
[application-defined SQL functions],
[collating sequences], [virtual tables], and [VFSes]) at run-time.
This feature allows the code for extensions to be developed and

  **     sqlite3_create_module_v2(), and/or
  **     sqlite3_vfs_register()
  ** to register the new features that your extension adds.
  */
  return rc;
}
</pre></blockquote>

<p>Many examples of complete and working loadable extensions can be seen in the
SQLite source tree in the
<a href="http://www.sqlite.org/src/dir?name=ext/misc&ci=trunk">ext/misc</a>
subdirectory.</p>

<h2>Statically Linking A Run-Time Loadable Extension</h2>

<p>The exact same source code can be used for both a run-time loadable
shared library or DLL and as a module that is statically linked with your
application.  This provides flexibility and allows you to reuse the same
code in different ways.</p>

  hd_puts "<h3>$date - $title</h3>"
  regsub -all "\n( *\n)+" $text "</p>\n\n<p>" txt
  regsub -all {[Tt]icket #(\d+)} $txt \
      {<a href="http://www.sqlite.org/cvstrac/tktview?tn=\1">\0</a>} txt
  hd_resolve "<blockquote>$txt</blockquote>"
  hd_puts "<hr width=\"50%\">"
}

newsitem {2013-08-15} {Release 3.8.0} {
  SQLite [version 3.8.0] features the cutover of the
  [next generation query planner] or [NGQP].
  The NGQP will normally cause applications to run faster.  However,
  there are [hazards of upgrading to the NGQP | hazards] associated
  with this upgrade.  A [query planner checklist] is available to
  application developers to aid in avoiding problems.
}

newsitem {2013-05-20} {Release 3.7.17} {
  SQLite [version 3.7.17] is a regularly schedule maintenance release.
  Visit the [version 3.7.17 | change log] for a full explanation of the
  changes in this release.

  There are many bug fixes in version 3.7.17.  But this does not indicate

<tcl>hd_fragment mailinglists {mailing lists}</tcl>
<h3>Mailing Lists</h3>
<p>Three separate mailing lists have been established to help support
SQLite:</p>

<ul>
<li><a href="http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users">
sqlite-users</a> - general discussion for programmers using SQLite in their applications.  
Most postings belong here.</li>
<li><a href="http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-dev">
sqlite-dev</a> - conversations about SQLite internals, for those who work
directly on enhancing SQLite itself.</li>
<li><a href="http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-announce">
sqlite-announce</a> - announcements of new

releases or significant developments; a very-low traffic list.</li>
</ul>

<p>
Most users of SQLite will want to join the
<a href="http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-announce">
sqlite-announce</a> list and many will want to join the
<a href="http://sqlite.org:8080/cgi-bin/mailman/listinfo/sqlite-users">

<blockquote>
[http://www.mail-archive.com/sqlite-users%40sqlite.org/]<br>
<a href="http://marc.info/?l=sqlite-users&r=1&w=2">http://marc.info/?l=sqlite-users&r=1&w=2</a><br>
[http://news.gmane.org/gmane.comp.db.sqlite.general]
</blockquote>

</p>