Documentation Source Text

completed successfully or else were completely rolled back.  The
[integrity_check] [PRAGMA] is used to make sure no database corruption
occurs.</p>

<p>The TH3 test harness needs to run on embedded systems that do not
necessarily have the ability to spawn child processes, so it uses
an in-memory VFS to simulate crashes.  The in-memory VFS can be rigged
to make a snapshot of the entire filesystem after a set number of I/O
operations.  Crash tests run in a loop.  On each iteration of the loop,
the point at which a snapshot is made is advanced until the SQLite
operations being tested run to completion without ever hitting a
snapshot.  Within the loop, after the SQLite operation under test has
completed, the filesystem is reverted to the snapshot and random file
damage is introduced that is characteristic of the kinds of damage
one expects to see following a power loss.  Then the database is opened

programming dilemma:  Does one remove defensive code in order to obtain
100% branch coverage?</p>

<p>In SQLite, the answer to the previous question is "no".
For testing purposes, the SQLite source code defines
macros called ALWAYS() and NEVER().   The ALWAYS() macro
surrounds conditions
which are expected to always evaluate as true and NEVER() surrounds
conditions that are always evaluated to false.  These macros serve as
comments to indicate that the conditions are defensive code.
For standard builds, these macros are pass-throughs:</p>

<blockquote><pre>
#define ALWAYS(X)  (X)
#define NEVER(X)   (X)
</pre></blockquote>

<p>The developers of SQLite have found that full coverage testing is an
extremely productive method for preventing the introduction of new bugs
as the system evolves.  Because every single branch
instruction in SQLite core code is covered by test cases, the developers
can be confident that changes they make in one part of the code
do not have unintended consequences in other parts of the code.
It would be extremely difficult maintain the quality of SQLite without such
assurances.</p>

<h2>8.0 Dynamic Analysis</h2>

<p>Dynamic analysis refers to internal and external checks on the
SQLite code which are performed while the code is live and running.
Dynamic analysis has proven to be a great help in maintaining the

<p>Because it is a simulator, running a binary in valgrind is slower than 
running it on native hardware.  So it is impractical to run the full
SQLite test suite through valgrind.  However, the veryquick tests and
a subset of the TH3 tests are run through valgrind prior to every release.</p>

<h3>8.3 Memsys2</h3>

<p>SQLite contains a pluggable
[memory allocation | memory allocation subsystem].
The default implementation uses system malloc() and free(). 
However, if SQLite is compiled with [SQLITE_MEMDEBUG], an alternative
memory allocation wrapper ([memsys2])
is inserted that looks for memory allocation
errors at run-time.  The memsys2 wrapper checks for memory leaks, of
course, but also looks for buffer overruns, uses of uninitialized memory,
and attempts to use memory after it has been freed.  These same checks
are also done by valgrind (and, indeed, valgrind does them better)
but memsys2 has the advantage of being much faster than valgrind, which

are atomic across system crashes and power failures is to write
all changes into the rollback journal file prior to changing the
database.  The TCL test harness contains an alternative
[sqlite3_vfs | Virtual File System ] implementation that helps to
verify this is occurring correctly.  The "journal-test VFS" monitors
all disk I/O traffic between the database file and rollback journal,
checking to make sure that nothing is written into the database
file which has not first been written and synced to the rollback journal.
If any discrepancies are found, an assertion fault is raised.</p>

<p>The journal tests are an additional double-check over and above
the crash tests to make sure that SQLite transactions will be atomic
across system crashes and power failures.</p>

<h2>9.0 Static Analysis</h2>

linked against the SQLite library under test.  The generated test
program is compiled and run on the target platform in order to verify
correct operation of SQLite on that platform.</p>

<p>The inputs to TH3 are test modules written in C or SQL and
small configuration
files that determine how to initialize SQLite.  The
TH3 package includes hundreds of test
modules and dozens of configuration files.  New modules and configurations
can be added to customize TH3 for specialized applications.
Each time TH3 is run, it reads
a subset of the available test modules and configuration files to generate
a custom C program the performs all of the specified tests under all
configurations.  A complete test of SQLite normally involves running
TH3 multiple times to generate multiple test programs covering different
aspects of SQLite's operation, then linking all test programs against
a common SQLite library and running them separately on the target platform.
SQLite will be found to work if all test programs pass.</p>

<p>There are no arbitrary limits in TH3.  One could generate a
single test program that contained all test modules and configuration files.
However, such a test program might be too large to deploy on embedded
platforms.  Hence, TH3 provides the ability to break the library of test
modules up into smaller, more easily digested pieces.</p>

<p>Each individual test module might contain dozens, hundreds, or thousands
of separate tests.  The test modules can be written in C or as scripts of
SQL.
Test modules done in SQL are very easy to write and
the test modules written in C are not as nearly as cumbersome 
to write as one might suppose.  The TH3
system provides high-level interfaces that simplify test writing.
A typical C-language test case in TH3 might contains slightly more syntax that 
the corresponding TCL-script test, but the difference is not that great.
The test modules written as SQL contain special comments that define
the boundaries and operation of each test case and the expected results.</p>

a license to access and use TH3.</p>

<p>Licensees of TH3 are given read access to the software configuration
management system used to manage TH3 and so can download the latest version
of TH3 (or any historical version) whenever they like.</p>

<p>Even though open-source users do not have direct access to TH3, all
users of SQLite benefit from TH3 indirectly since each version of SQLite is
validated by TH3 prior to release.  So anyone using an official release
of SQLite can deploy their application with the confidence of knowing that
it has been tested using TH3.  They simply cannot rerun those tests
themselves without purchasing a TH3 license.</p>