Documentation Source Text

Check-in [f7aa8bfebf]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update documentation to talk about 100% test coverage in SQLite.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:f7aa8bfebfa1289dddcd3601aeaf74ab1a457ce1
User & Date: drh 2009-07-30 10:51:45
Context
2009-07-31
20:36
Expanded documentation on test coverage. Added the "th3.html" page for describing the TH3 program. check-in: 6bac792113 user: drh tags: trunk
2009-07-30
10:51
Update documentation to talk about 100% test coverage in SQLite. check-in: f7aa8bfebf user: drh tags: trunk
2009-07-29
23:07
Merge lang.html and vtab.html corrections that got marooned on a branch into the main line. check-in: ab7c4f5247 user: drh tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/btreemodule.in.

18
19
20
21
22
23
24

25
26
27
28
29
30
31
      set re_tail {[^\{]*\};\n}
    } elseif { [string match BTREE* $api] } {
      set re_tail {[^A-Za-z][^\n]*\n}
    } else {
      set re_tail {[^;A-Za-z][^;]*;}
    }


    regexp $re_head$api$re_tail $::headers($header) api_defn
    lappend ret [string trim $api_defn]
  }
  return "<pre class=api>[join $ret "\n"]</pre>"
}

proc btree_api_defn {args} {







>







18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
      set re_tail {[^\{]*\};\n}
    } elseif { [string match BTREE* $api] } {
      set re_tail {[^A-Za-z][^\n]*\n}
    } else {
      set re_tail {[^;A-Za-z][^;]*;}
    }

    set api_defn {}
    regexp $re_head$api$re_tail $::headers($header) api_defn
    lappend ret [string trim $api_defn]
  }
  return "<pre class=api>[join $ret "\n"]</pre>"
}

proc btree_api_defn {args} {

Changes to pages/features.in.

23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
<li><a href="speed.html">Faster</a> than popular client/server database
    engines for most common operations.</li>
<li>Simple, easy to use <a href="c3ref/intro.html">API</a>.</li>
<li>Written in ANSI-C.  <a href="tclsqlite.html">TCL bindings</a> included.
    Bindings for dozens of other languages 
    <a href="http://www.sqlite.org/cvstrac/wiki?p=SqliteWrappers">
    available separately.</a></li>
<li>Well-commented source code with over 99% statement test coverage.</li>
<li>Available as a 
    <a href="amalgamation.html">single ANSI-C source-code file</a> 
    that you can easily drop into another project.
<li><a href="selfcontained.html">Self-contained</a>:
    no external dependencies.</li>
<li>Cross-platform: Unix (Linux and Mac OS X), OS/2, and Windows
    (Win32 and WinCE)







|







23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
<li><a href="speed.html">Faster</a> than popular client/server database
    engines for most common operations.</li>
<li>Simple, easy to use <a href="c3ref/intro.html">API</a>.</li>
<li>Written in ANSI-C.  <a href="tclsqlite.html">TCL bindings</a> included.
    Bindings for dozens of other languages 
    <a href="http://www.sqlite.org/cvstrac/wiki?p=SqliteWrappers">
    available separately.</a></li>
<li>Well-commented source code with 100% branch test coverage.</li>
<li>Available as a 
    <a href="amalgamation.html">single ANSI-C source-code file</a> 
    that you can easily drop into another project.
<li><a href="selfcontained.html">Self-contained</a>:
    no external dependencies.</li>
<li>Cross-platform: Unix (Linux and Mac OS X), OS/2, and Windows
    (Win32 and WinCE)

Changes to pages/testing.in.

136
137
138
139
140
141
142
143

144
145
146
147
148
149
150
151
152
153
...
366
367
368
369
370
371
372
373
374
375

376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
so that on a full test run, about <tcl>MB {$stat(tclNEval)}</tcl> million 
separate tests are performed.
</p>
</li>

<li><p>
The <b>TH3</b> test harness is a set of proprietary tests, written in
C.  The impetus for TH3 was the need to have a set of tests that ran

on embedded and specialized platforms that would not easily support
TCL or other workstation services.  TH3 tests use only the published 
SQLite interfaces.  These tests are free to [SQLite Consortium] members 
and are available by license to others.  TH3 consists of about
<tcl>MB {$stat(th3NByte)}</tcl> MB or <tcl>KB {$stat(th3SLOC)}</tcl> KSLOC
of C code implementing <tcl>N {$stat(th3NTest)}</tcl> distinct test cases.
TH3 tests are heavily parameterized, though, so a full test runs
about <tcl>MB {$stat(th3NEval)}</tcl> million different test
instances.</p></li>

................................................................................
an exception such as an OOM error or disk I/O error.  The test
harnesses are zealous to enforce this.</p>

<h2>7.0 Test Coverage</h2>

<p>The <a href="http://gcc.gnu.org/onlinedocs/gcc/Gcov.html">gcov</a>
utility is used to measure the "test coverage" of the SQLite test suite.
SQLite strives for but does not yet obtain 100% test coverage.  A major
goal of the SQLite project is to obtain 100% branch coverage
during 2009.</P.


<p>Test coverage can be measured in several ways.  "Statement coverage"
measures (as a percentage of the whole) how many lines of code are
exercised by the test cases. Statement coverage for the SQLite core
is <tcl>hd_puts $stat(allStmtCov)</tcl>% for the full TCL test suite,
<tcl>hd_puts $stat(vqStmtCov)</tcl>% for the "veryquick.test" abbreviated
TCL test suite and <tcl>hd_puts $stat(th3StmtCov)</tcl>% for the TH3
test suite. (The SQLite core, in this case, excludes the operating-system
dependent [sqlite3_vfs | VFS] backends.)
"Branch" coverage measures (again, as a percentage of the
whole) how many machine-code branch instructions are taken at least
once in both directions.  Branch coverage
is <tcl>hd_puts $stat(allBrCov)</tcl>% for the full TCL test suite,
<tcl>hd_puts $stat(vqBrCov)</tcl>% for the abbreviated TCL test suite, and
<tcl>hd_puts $stat(th3BrCov)</tcl>% for the TH3 test harness.</p>

<p>To illustrate the difference between statement coverage and
branch coverage, consider the following hypothetical
line of C code:</p>

<blockquote><pre>
if( a>b && c!=25 ){ d++; }







|
>


|







 







|
<
<
>






|







|







136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
...
367
368
369
370
371
372
373
374


375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
so that on a full test run, about <tcl>MB {$stat(tclNEval)}</tcl> million 
separate tests are performed.
</p>
</li>

<li><p>
The <b>TH3</b> test harness is a set of proprietary tests, written in
C that provide 100% branch test coverage (and 100% MC/DC test coverage) to
the core SQLite library.  The TH3 tests are designed to run
on embedded and specialized platforms that would not easily support
TCL or other workstation services.  TH3 tests use only the published 
SQLite interfaces.  TH3 is free to [SQLite Consortium] members 
and are available by license to others.  TH3 consists of about
<tcl>MB {$stat(th3NByte)}</tcl> MB or <tcl>KB {$stat(th3SLOC)}</tcl> KSLOC
of C code implementing <tcl>N {$stat(th3NTest)}</tcl> distinct test cases.
TH3 tests are heavily parameterized, though, so a full test runs
about <tcl>MB {$stat(th3NEval)}</tcl> million different test
instances.</p></li>

................................................................................
an exception such as an OOM error or disk I/O error.  The test
harnesses are zealous to enforce this.</p>

<h2>7.0 Test Coverage</h2>

<p>The <a href="http://gcc.gnu.org/onlinedocs/gcc/Gcov.html">gcov</a>
utility is used to measure the "test coverage" of the SQLite test suite.
As of July 2009, the SQLite core has 100% test coverage.</p>




<p>Test coverage can be measured in several ways.  "Statement coverage"
measures (as a percentage of the whole) how many lines of code are
exercised by the test cases. Statement coverage for the SQLite core
is <tcl>hd_puts $stat(allStmtCov)</tcl>% for the full TCL test suite,
<tcl>hd_puts $stat(vqStmtCov)</tcl>% for the "veryquick.test" abbreviated
TCL test suite and 100% for the TH3
test suite. (The SQLite core, in this case, excludes the operating-system
dependent [sqlite3_vfs | VFS] backends.)
"Branch" coverage measures (again, as a percentage of the
whole) how many machine-code branch instructions are taken at least
once in both directions.  Branch coverage
is <tcl>hd_puts $stat(allBrCov)</tcl>% for the full TCL test suite,
<tcl>hd_puts $stat(vqBrCov)</tcl>% for the abbreviated TCL test suite, and
100% for the TH3 test harness.</p>

<p>To illustrate the difference between statement coverage and
branch coverage, consider the following hypothetical
line of C code:</p>

<blockquote><pre>
if( a>b && c!=25 ){ d++; }

Changes to pages/vtab.in.

363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
...
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
...
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
...
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
...
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
...
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
...
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
...
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
....
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
....
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
<p>The xCreate method is required for every virtual table implementation, 
though the xCreate and [xConnect] pointers of the [sqlite3_module] object
may point to the same function the virtual table does not need to initialize
backing store.

<tcl>############################################################# xConnect
hd_fragment xconnect {sqlite3_module.xConnect} {xConnect}</tcl>
<tcl>hd_fragment xconnect {sqlite3_module.xConnect} {xConnect}</tcl>
<h3>2.2 The xConnect Method</h3>

<blockquote><pre>
  int (*xConnect)(sqlite3*, void *pAux,
               int argc, char **argv,
               sqlite3_vtab **ppVTab,
               char **pzErr);
................................................................................
<p>The xConnect method is required for every virtual table implementation, 
though the [xCreate] and xConnect pointers of the [sqlite3_module] object
may point to the same function the virtual table does not need to initialize
backing store.

<tcl>############################################################ xBestIndex
hd_fragment xbestindex {sqlite3_module.xBestIndex} {xBestIndex}</tcl>
<tcl>hd_fragment xbestindex {sqlite3_module.xBestIndex} {xBestIndex}</tcl>
<h3>2.3 The xBestIndex Method</h3>

<p>SQLite uses the xBestIndex method of a virtual table module to determine
the best way to access the virtual table. 
The xBestIndex method has a prototype like this:

<blockquote><pre>
................................................................................
The SQLite core will invoke the [xFilter] method
on the cursor prior to any attempt to position or read from the cursor.

<p>The xOpen method is required for every virtual table implementation.

<tcl>############################################################### xClose
hd_fragment xclose {sqlite3_module.xClose}</tcl>
<tcl>hd_fragment xclose {sqlite3_module.xClose}</tcl>
<h3>2.7 The xClose Method</h3>

<blockquote><pre>
  int (*xClose)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xClose method closes a cursor previously opened by 
................................................................................
returns an error.  The SQLite core will not use the
[sqlite3_vtab_cursor] again after it has been closed.

<p>The xClose method is required for every virtual table implementation.

<tcl>############################################################## xEof
hd_fragment xeof {sqlite3_module.xEof} {xEof}</tcl>
<tcl>hd_fragment xeof {sqlite3_module.xEof} {xEof}</tcl>
<h3>2.8 The xEof Method</h3>

<blockquote><pre>
  int (*xEof)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xEof method must return false (zero) if the specified cursor 
................................................................................
<p>This method must return [SQLITE_OK] if successful, or an sqlite 
[error code] if an error occurs.

<p>The xFilter method is required for every virtual table implementation.

<tcl>############################################################### xNext
hd_fragment xnext {sqlite3_module.xNext} {xNext}</tcl>
<tcl>hd_fragment xnext {sqlite3_module.xNext} {xNext}</tcl>
<h3>2.10 The xNext Method</h3>

<blockquote><pre>
  int (*xNext)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xNext method advances a [sqlite3_vtab_cursor | virtual table cursor]
................................................................................
methods to set the error message text, then return an appropriate
[error code].  The xColumn method must return [SQLITE_OK] on success.

<p>The xColumn method is required for every virtual table implementation.

<tcl>############################################################# xRowid
hd_fragment xrowid {sqlite3_module.xRowid} {xRowid}</tcl>
<tcl>hd_fragment xrowid {sqlite3_module.xRowid} {xRowid}</tcl>
<h3>2.12 The xRowid Method</h3>

<blockquote><pre>
  int (*xRowid)(sqlite3_vtab_cursor *pCur, sqlite_int64 *pRowid);
</pre></blockquote>

<p>A successful invocation of this method will cause *pRowid to be
................................................................................
<p>The xUpdate method is optional.
If the xUpdate pointer in the [sqlite3_module] for a virtual table
is a NULL pointer, then the virtual table is read-only.


<tcl>########################################################## xFindFunction
hd_fragment xfindfunction {sqlite3_module.xFindFunction} {xFindFunction}</tcl>
<tcl>hd_fragment xfindfunction {sqlite3_module.xFindFunction} {xFindFunction}</tcl>
<h3>2.14 The xFindFunction Method</h3>

<blockquote><pre>
  int (*xFindFunction)(
    sqlite3_vtab *pVtab,
    int nArg,
    const char *zName,
................................................................................
first argument.

<p>The function pointer returned by this routine must be valid for
the lifetime of the [sqlite3_vtab] object given in the first parameter.

<tcl>############################################################ xBegin
hd_fragment xBegin {sqlite3_module.xBegin} {xBegin}</tcl>
<tcl>hd_fragment xBegin {sqlite3_module.xBegin} {xBegin}</tcl>
<h3>2.15 The xBegin Method</h3>

<blockquote><pre>
  int (*xBegin)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method begins a transaction on a virtual table.
................................................................................
on a single virtual table
without an intervening call to either [xCommit] or [xRollback].
Multiple calls to other methods can and likely will occur in between
the xBegin and the corresponding [xCommit] or [xRollback].

<tcl>############################################################ xSync
hd_fragment xsync {sqlite3_module.xSync}</tcl>
<tcl>hd_fragment xsync {sqlite3_module.xSync}</tcl>
<h3>2.16 The xSync Method</h3>

<blockquote><pre>
  int (*xSync)(sqlite3_vtab *pVTab);
</pre></blockquote>


................................................................................
prior to an [xCommit] or [xRollback].  In order to implement two-phase
commit, the xSync method on all virtual tables is invoked prior to
invoking the [xCommit] method on any virtual table.  If any of the 
xSync methods fail, the entire transaction is rolled back.

<tcl>########################################################### xCommit
hd_fragment xcommit {sqlite3_module.xCommit} {xCommit}</tcl>
<tcl>hd_fragment xcommit {sqlite3_module.xCommit} {xCommit}</tcl>
<h3>2.17 The xCommit Method</h3>

<blockquote><pre>
  int (*xCommit)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method causes a virtual table transaction to commit.







<







 







<







 







<







 







<







 







<







 







<







 







<







 







<







 







<







 







<







363
364
365
366
367
368
369

370
371
372
373
374
375
376
...
418
419
420
421
422
423
424

425
426
427
428
429
430
431
...
711
712
713
714
715
716
717

718
719
720
721
722
723
724
...
731
732
733
734
735
736
737

738
739
740
741
742
743
744
...
781
782
783
784
785
786
787

788
789
790
791
792
793
794
...
840
841
842
843
844
845
846

847
848
849
850
851
852
853
...
954
955
956
957
958
959
960

961
962
963
964
965
966
967
...
990
991
992
993
994
995
996

997
998
999
1000
1001
1002
1003
....
1010
1011
1012
1013
1014
1015
1016

1017
1018
1019
1020
1021
1022
1023
....
1030
1031
1032
1033
1034
1035
1036

1037
1038
1039
1040
1041
1042
1043
<p>The xCreate method is required for every virtual table implementation, 
though the xCreate and [xConnect] pointers of the [sqlite3_module] object
may point to the same function the virtual table does not need to initialize
backing store.

<tcl>############################################################# xConnect
hd_fragment xconnect {sqlite3_module.xConnect} {xConnect}</tcl>

<h3>2.2 The xConnect Method</h3>

<blockquote><pre>
  int (*xConnect)(sqlite3*, void *pAux,
               int argc, char **argv,
               sqlite3_vtab **ppVTab,
               char **pzErr);
................................................................................
<p>The xConnect method is required for every virtual table implementation, 
though the [xCreate] and xConnect pointers of the [sqlite3_module] object
may point to the same function the virtual table does not need to initialize
backing store.

<tcl>############################################################ xBestIndex
hd_fragment xbestindex {sqlite3_module.xBestIndex} {xBestIndex}</tcl>

<h3>2.3 The xBestIndex Method</h3>

<p>SQLite uses the xBestIndex method of a virtual table module to determine
the best way to access the virtual table. 
The xBestIndex method has a prototype like this:

<blockquote><pre>
................................................................................
The SQLite core will invoke the [xFilter] method
on the cursor prior to any attempt to position or read from the cursor.

<p>The xOpen method is required for every virtual table implementation.

<tcl>############################################################### xClose
hd_fragment xclose {sqlite3_module.xClose}</tcl>

<h3>2.7 The xClose Method</h3>

<blockquote><pre>
  int (*xClose)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xClose method closes a cursor previously opened by 
................................................................................
returns an error.  The SQLite core will not use the
[sqlite3_vtab_cursor] again after it has been closed.

<p>The xClose method is required for every virtual table implementation.

<tcl>############################################################## xEof
hd_fragment xeof {sqlite3_module.xEof} {xEof}</tcl>

<h3>2.8 The xEof Method</h3>

<blockquote><pre>
  int (*xEof)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xEof method must return false (zero) if the specified cursor 
................................................................................
<p>This method must return [SQLITE_OK] if successful, or an sqlite 
[error code] if an error occurs.

<p>The xFilter method is required for every virtual table implementation.

<tcl>############################################################### xNext
hd_fragment xnext {sqlite3_module.xNext} {xNext}</tcl>

<h3>2.10 The xNext Method</h3>

<blockquote><pre>
  int (*xNext)(sqlite3_vtab_cursor*);
</pre></blockquote>

<p>The xNext method advances a [sqlite3_vtab_cursor | virtual table cursor]
................................................................................
methods to set the error message text, then return an appropriate
[error code].  The xColumn method must return [SQLITE_OK] on success.

<p>The xColumn method is required for every virtual table implementation.

<tcl>############################################################# xRowid
hd_fragment xrowid {sqlite3_module.xRowid} {xRowid}</tcl>

<h3>2.12 The xRowid Method</h3>

<blockquote><pre>
  int (*xRowid)(sqlite3_vtab_cursor *pCur, sqlite_int64 *pRowid);
</pre></blockquote>

<p>A successful invocation of this method will cause *pRowid to be
................................................................................
<p>The xUpdate method is optional.
If the xUpdate pointer in the [sqlite3_module] for a virtual table
is a NULL pointer, then the virtual table is read-only.


<tcl>########################################################## xFindFunction
hd_fragment xfindfunction {sqlite3_module.xFindFunction} {xFindFunction}</tcl>

<h3>2.14 The xFindFunction Method</h3>

<blockquote><pre>
  int (*xFindFunction)(
    sqlite3_vtab *pVtab,
    int nArg,
    const char *zName,
................................................................................
first argument.

<p>The function pointer returned by this routine must be valid for
the lifetime of the [sqlite3_vtab] object given in the first parameter.

<tcl>############################################################ xBegin
hd_fragment xBegin {sqlite3_module.xBegin} {xBegin}</tcl>

<h3>2.15 The xBegin Method</h3>

<blockquote><pre>
  int (*xBegin)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method begins a transaction on a virtual table.
................................................................................
on a single virtual table
without an intervening call to either [xCommit] or [xRollback].
Multiple calls to other methods can and likely will occur in between
the xBegin and the corresponding [xCommit] or [xRollback].

<tcl>############################################################ xSync
hd_fragment xsync {sqlite3_module.xSync}</tcl>

<h3>2.16 The xSync Method</h3>

<blockquote><pre>
  int (*xSync)(sqlite3_vtab *pVTab);
</pre></blockquote>


................................................................................
prior to an [xCommit] or [xRollback].  In order to implement two-phase
commit, the xSync method on all virtual tables is invoked prior to
invoking the [xCommit] method on any virtual table.  If any of the 
xSync methods fail, the entire transaction is rolled back.

<tcl>########################################################### xCommit
hd_fragment xcommit {sqlite3_module.xCommit} {xCommit}</tcl>

<h3>2.17 The xCommit Method</h3>

<blockquote><pre>
  int (*xCommit)(sqlite3_vtab *pVTab);
</pre></blockquote>

<p>This method causes a virtual table transaction to commit.