Documentation Source Text

<li><p> TH3 runs on embedded platforms that lack the support
     infrastructure of workstations.</p></li>

<li><p> TH3 tests SQLite in an as-deployed configuration using only
     published and documented interfaces.
     In other words, TH3 tests the compiled object code, not
     the source code, thus verifying that no problems were introduced
     by compiler bugs.</p></li>

<li><p> TH3 checks SQLite's response to out-of-memory errors, disk I/O
     errors, and power loss during transaction commit. </p></li>

<li><p> TH3 exercises SQLite in a variety of run-time configurations
     (UTF8 vs UTF16, different pages sizes, varying journal modes, etc.)
     </p></li>
................................................................................
linked against the SQLite library under test.  The generated test
program is compiled and run on the target platform in order to verify
correct operation of SQLite on that platform.</p>

<p>The inputs to TH3 are test modules written in C or SQL and
small configuration
files that determine how to initialize SQLite.  The
TH3 package includes approximately one thousand test
modules and several dozens configuration files.
New modules and configurations
can be added to customize TH3 for specialized applications.
Each time TH3 is run, it reads
a subset of the available test modules and configuration files to generate
a custom C program that performs all of the specified tests under all
configurations.  A complete test of SQLite normally involves running
TH3 multiple times to generate multiple test programs covering different
................................................................................
single test program that contained all test modules and configuration files.
However, such a test program might be too large to deploy on embedded
platforms.  Hence, TH3 provides the ability to break the library of test
modules up into smaller, more easily digested pieces.</p>

<p>Each individual test module might contain dozens, hundreds, or thousands
of separate tests.  The test modules can be written in C or as scripts of
SQL.
Test modules done in SQL are very easy to write and
the test modules written in C are not as nearly as cumbersome 
to write as one might suppose.  The TH3
system provides high-level interfaces that simplify test writing.
A typical C-language test case in TH3 might contains slightly more syntax that 
the corresponding TCL-script test, but the difference is not that great.
The test modules written as SQL contain special comments that define
the boundaries and operation of each test case and the expected results.</p>


<p>Each test module file contains a header which describes the circumstances
under which the test is valid.  For a particular configuration, only those
modules that are compatible with the configuration are run.  </p>

<h2>3.0 Generating A Test Program</h2>

................................................................................
cc -o testprog1 testprog1.c sqlite3.c
</pre></blockquote>

<p>The compilation step shown immediately above is merely representative.
In a working installation, one would normally want
to specify optimization parameters and compile-time switches on the
compiler command line.</p>






<p>Once the test program is generated, it is run with no arguments to
perform the tests.  Progress information as well as error diagnostics


appear on standard output.  The program returns zero if there are no
errors and non-zero if any problems were detected.</p>

<p>TH3 comes with additional TCL scripts (for example:
"th3make" and "fulltest.tcl")
which help to automate the process of running mkth3.tcl, compiling th3.c and
sqlite3.c, running the resulting test programs, and analyzing the results.
These other scripts are for convenience only and are not required in order
................................................................................
<h2>4.0 Test Coverage</h2>

<p>Using one particular subset of the available TH3 test modules (the "cov1"
tests) SQLite obtained 
[test coverage | 100% branch test coverage] and 100% [MC/DC] as measured
by [http://gcc.gnu.org/onlinedocs/gcc/Gcov.html | gcov]
on Linux x86 and x86_64 hardware.  All releases of SQLite since
[version 3.6.17] have been tested to this standard. The SQLite developers 

are committed to maintaining 100% branch coverage and MC/DC for all 
future releases of SQLite.</p>

<p>The cov1 test set used to obtain 100% branch test coverage are only a
subset of the tests currently implemented using TH3.  New test modules are
added on a regular basis.</p>

................................................................................

<p>Licensees of TH3 are given read access to the software configuration
management system used to manage TH3 and so can download the latest version
of TH3 (or any historical version) whenever they like.</p>

<p>Even though open-source users do not have direct access to TH3, all
users of SQLite benefit from TH3 indirectly since each version of SQLite is

validated by TH3 prior to release.  So anyone using an official release
of SQLite can deploy their application with the confidence of knowing that
it has been tested using TH3.  They simply cannot rerun those tests
themselves without purchasing a TH3 license.</p>

<li><p> TH3 runs on embedded platforms that lack the support
     infrastructure of workstations.</p></li>

<li><p> TH3 tests SQLite in an as-deployed configuration using only
     published and documented interfaces.
     In other words, TH3 tests the compiled object code, not
     the source code, thus verifying that no problems were introduced
     by compiler bugs.  "Fly what you test and test what you fly."</p></li>

<li><p> TH3 checks SQLite's response to out-of-memory errors, disk I/O
     errors, and power loss during transaction commit. </p></li>

<li><p> TH3 exercises SQLite in a variety of run-time configurations
     (UTF8 vs UTF16, different pages sizes, varying journal modes, etc.)
     </p></li>
................................................................................
linked against the SQLite library under test.  The generated test
program is compiled and run on the target platform in order to verify
correct operation of SQLite on that platform.</p>

<p>The inputs to TH3 are test modules written in C or SQL and
small configuration
files that determine how to initialize SQLite.  The
TH3 package includes over one thousand test
modules and several dozen configuration files.
New modules and configurations
can be added to customize TH3 for specialized applications.
Each time TH3 is run, it reads
a subset of the available test modules and configuration files to generate
a custom C program that performs all of the specified tests under all
configurations.  A complete test of SQLite normally involves running
TH3 multiple times to generate multiple test programs covering different
................................................................................
single test program that contained all test modules and configuration files.
However, such a test program might be too large to deploy on embedded
platforms.  Hence, TH3 provides the ability to break the library of test
modules up into smaller, more easily digested pieces.</p>

<p>Each individual test module might contain dozens, hundreds, or thousands
of separate tests.  The test modules can be written in C or as scripts of
SQL or a mixture of the two.  About two-thirds of the existing test modules are
written SQL with the remainder either in pure C or a combination of C and SQL.







</p>

<p>Each test module file contains a header which describes the circumstances
under which the test is valid.  For a particular configuration, only those
modules that are compatible with the configuration are run.  </p>

<h2>3.0 Generating A Test Program</h2>

................................................................................
cc -o testprog1 testprog1.c sqlite3.c
</pre></blockquote>

<p>The compilation step shown immediately above is merely representative.
In a working installation, one would normally want
to specify optimization parameters and compile-time switches on the
compiler command line.</p>

<p>For testing on embedded systems, the mkth3.tcl script and the compiler
steps shown above a typically performed on an ordinary workstation using
a cross-compiler for the second step, then the resulting test program is
transfer onto the device to be run.</p>

<p>Once the test program is generated, it is run with no arguments to
perform the tests.  Progress information as well as error diagnostics
appear on standard output.  (Alternative output arrangements can be made
using a compile-time option for embedded devices that lack a standard
output channel.) The program returns zero if there are no
errors and non-zero if any problems were detected.</p>

<p>TH3 comes with additional TCL scripts (for example:
"th3make" and "fulltest.tcl")
which help to automate the process of running mkth3.tcl, compiling th3.c and
sqlite3.c, running the resulting test programs, and analyzing the results.
These other scripts are for convenience only and are not required in order
................................................................................
<h2>4.0 Test Coverage</h2>

<p>Using one particular subset of the available TH3 test modules (the "cov1"
tests) SQLite obtained 
[test coverage | 100% branch test coverage] and 100% [MC/DC] as measured
by [http://gcc.gnu.org/onlinedocs/gcc/Gcov.html | gcov]
on Linux x86 and x86_64 hardware.  All releases of SQLite since
[version 3.6.17] (circa 2009-08-10) have been tested to this standard. 
The SQLite developers 
are committed to maintaining 100% branch coverage and MC/DC for all 
future releases of SQLite.</p>

<p>The cov1 test set used to obtain 100% branch test coverage are only a
subset of the tests currently implemented using TH3.  New test modules are
added on a regular basis.</p>

................................................................................

<p>Licensees of TH3 are given read access to the software configuration
management system used to manage TH3 and so can download the latest version
of TH3 (or any historical version) whenever they like.</p>

<p>Even though open-source users do not have direct access to TH3, all
users of SQLite benefit from TH3 indirectly since each version of SQLite is
validated running TH3 on multiple platforms (Linux, Windows, WinRT, Mac,
OpenBSD, Solaris Sparc) prior to release.  So anyone using an official release
of SQLite can deploy their application with the confidence of knowing that
it has been tested using TH3.  They simply cannot rerun those tests
themselves without purchasing a TH3 license.</p>