Documentation Source Text

Check-in [643e08dfb8]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk.
Timelines: family | ancestors | descendants | both | branch-3.8.10
Files: files | file ages | folders
SHA1: 643e08dfb8f8b021b125bbcfdb56e8f8eb25009a
User & Date: drh 2015-05-15 14:53:51
Context
2015-05-15
15:56
Improvements to the testing.html document. check-in: 0b4aaced39 user: drh tags: branch-3.8.10
14:53
Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk. check-in: 643e08dfb8 user: drh tags: branch-3.8.10
14:51
Link from the ATTACH documentation page to the DETACH page. check-in: b3cbd71993 user: drh tags: trunk
14:48
Increased information about TH3. check-in: 7d4f8c9ada user: drh tags: trunk
2015-05-11
17:09
Updates to the dbstat documentation to explain how it can be used to get information about attached databases other than "main". check-in: d33510f222 user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to pages/lang.in.

   400    400   Section {ATTACH DATABASE} attach {attached *ATTACH}
   401    401   
   402    402   RecursiveBubbleDiagram attach-stmt
   403    403   </tcl>
   404    404   
   405    405   <p> ^The ATTACH DATABASE statement adds another database 
   406    406   file to the current [database connection]. 
   407         -^The filename for the database to be attached is the value of
          407  +Database files that were previously attached can be removed using
          408  +the [DETACH DATABASE] command.
          409  +
          410  +<p>^The filename for the database to be attached is the value of
   408    411   the expression that occurs before the AS keyword.
   409    412   ^The filename of the database follows the same semantics as the
   410    413   filename argument to [sqlite3_open()] and [sqlite3_open_v2()]; the
   411    414   special name "[:memory:]" results in an [in-memory database] and an
   412    415   empty string results in a new temporary database.
   413    416   ^The filename argument can be a [URI filename] if URI filename processing
   414    417   is enable on the database connection.  The default behavior is for

Changes to pages/th3.in.

    31     31       (Test coverage of the operating-system specific [VFSes] and extensions
    32     32       such as FTS and RTREE is less than 100%). </p></li>
    33     33   </ul>
    34     34   
    35     35   <p>TH3 was originally written for validation testing only, but has
    36     36   subsequently been used for development testing and debugging
    37     37   as well, and has proven very helpful in those roles.  A full-coverage
    38         -test takes less than three minutes on a workstation and hence
           38  +test takes less than five minutes on a workstation and hence
    39     39   serves as a fast regression test during day-to-day maintenance
    40     40   of the SQLite code base.</p>
    41     41   
    42     42   <h2>2.0 Operation</h2>
    43     43   
    44     44   <p>TH3 is a test program generator.  The output of TH3 is a program
    45         -written in ANSI-C and intended to be
           45  +implemented in C-code and intended to be
    46     46   linked against the SQLite library under test.  The generated test
    47     47   program is compiled and run on the target platform in order to verify
    48     48   correct operation of SQLite on that platform.</p>
    49     49   
    50     50   <p>The inputs to TH3 are test modules written in C or SQL and
    51     51   small configuration
    52     52   files that determine how to initialize SQLite.  The
    53         -TH3 package includes over one thousand test
    54         -modules and several dozen configuration files.
           53  +TH3 package includes over eleven hundred test
           54  +modules and more than three dozen configuration files.
    55     55   New modules and configurations
    56     56   can be added to customize TH3 for specialized applications.
    57     57   Each time TH3 is run, it reads
    58     58   a subset of the available test modules and configuration files to generate
    59     59   a custom C program that performs all of the specified tests under all
    60         -configurations.  A complete test of SQLite normally involves running
           60  +specified configurations.  A complete test of SQLite normally involves running
    61     61   TH3 multiple times to generate multiple test programs covering different
    62     62   aspects of SQLite's operation, then linking all test programs against
    63     63   a common SQLite library and running them separately on the target platform.
    64     64   SQLite will be found to work if all test programs pass.</p>
    65     65   
    66     66   <p>There are no arbitrary limits in TH3.  One could generate a
    67         -single test program that contained all test modules and configuration files.
           67  +single test program that contained all test modules and all configuration files.
    68     68   However, such a test program might be too large to deploy on embedded
    69     69   platforms.  Hence, TH3 provides the ability to break the library of test
    70     70   modules up into smaller, more easily digested pieces.</p>
    71     71   
    72     72   <p>Each individual test module might contain dozens, hundreds, or thousands
    73     73   of separate tests.  The test modules can be written in C or as scripts of
    74     74   SQL or a mixture of the two.  About two-thirds of the existing test modules are
................................................................................
   117    117   <p>Once the test program is generated, it is run with no arguments to
   118    118   perform the tests.  Progress information as well as error diagnostics
   119    119   appear on standard output.  (Alternative output arrangements can be made
   120    120   using a compile-time option for embedded devices that lack a standard
   121    121   output channel.) The program returns zero if there are no
   122    122   errors and non-zero if any problems were detected.</p>
   123    123   
   124         -<p>TH3 comes with additional TCL scripts (for example:
   125         -"th3make" and "fulltest.tcl")
   126         -which help to automate the process of running mkth3.tcl, compiling th3.c and
   127         -sqlite3.c, running the resulting test programs, and analyzing the results.
   128         -These other scripts are for convenience only and are not required in order
   129         -to make use of TH3.
          124  +<p>Typical output from a single TH3 test program run looks like this:
          125  +
          126  +<blockquote><pre>
          127  +With SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e
          128  +-DSQLITE_COVERAGE_TEST
          129  +-DSQLITE_NO_SYNC
          130  +-DSQLITE_SYSTEM_MALLOC
          131  +-DSQLITE_THREADSAFE=1
          132  +Config-begin c1.
          133  +Begin c1.pager08
          134  +End c1.pager08
          135  +Begin c1.build33
          136  +End c1.build33
          137  +Begin c1.orderby01
          138  +End c1.orderby01
          139  +<i>... 15014 lines of output omitted ....</i>
          140  +Begin 64k.syscall01
          141  +End 64k.syscall01
          142  +Begin 64k.build01
          143  +End 64k.build01
          144  +Begin 64k.auth01
          145  +End 64k.auth01
          146  +Config-end 64k. TH3 memory used: 6373738
          147  +Config-begin wal1.
          148  +Begin wal1.wal37
          149  +End wal1.wal37
          150  +Config-end wal1. TH3 memory used: 100961
          151  +All 226 VDBE coverage points reached
          152  +th3: 0 errors out of 1442264 tests in 213.741 seconds. 64-bit little-endian
          153  +th3: SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e
          154  +</pre></blockquote>
          155  +
          156  +<p>The output begins with a report of the [SQLITE_SOURCE_ID]
          157  +(cross-checked again [sqlite3_sourceid()]) for the
          158  +SQLite under test and the compile-time options used as reported
          159  +by [sqlite3_compileoption_get()].  The output concludes with a summary
          160  +of the test results and a repeat of the [SQLITE_SOURCE_ID].  If any
          161  +errors are detected, additional lines detail the problem.  The error
          162  +reporting lines always begin with a single space character so that they
          163  +can be quickly extracted from large output files using:
          164  +
          165  +<blockquote><pre>
          166  +grep "&#94; "
          167  +</pre></blockquote>
          168  +
          169  +<p>The default output shows the beginning and end of each configuration
          170  +and test module combination.  In the example above "c1" and "64k" are
          171  +configurations and "pager08", "build33", "orderby01", etc. are test modules.
          172  +Compile-time and run-time options are available to increase or decrease
          173  +the amount of output.
          174  +The output can be increased by showing each test case within each
          175  +test module.  The output can be decreased
          176  +by degrees: omitting test modules starts and stops,
          177  +omitting configuration starts and stops, and finally by omitting all output.
          178  +<h3>3.1 Test Automation Scripts</h3>
          179  +
          180  +<p>TH3 comes with additional TCL scripts that help automate the testing
          181  +process on workstations.  The "th3make" script automatically runs "mkth3.tcl"
          182  +and "gcc" and then runs the resulting test program and checks the results.
          183  +Arguments to th3make include all of the "*.test" test modules and 
          184  +"*.cfg" configurations that are to be included in the test.  Additional
          185  +options to th3make can cause the test program to be compiled using different
          186  +compilers (GCC, Clang, MSVC), to use different output verbosity levels, to
          187  +run the test program under valgrind, to check the output for coverage using
          188  +gcov, and so forth.  The th3make script also accepts "*.rc" filenames as
          189  +arguments.  These *.rc files are just collections of other arguments that
          190  +are commonly used together for a single purpose.  For example, the "quick.rc"
          191  +file contains a set of eight arguments to th3make that run a fast (3-minute)
          192  +full-coverage test.  This allows the operator to type "./th3make quick.rc" as
          193  +a short-cut to typing out all of the required command-line options.  The
          194  +following are a few of the more than 40 available *.rc files:
          195  +
          196  +<ul>
          197  +<li><b>alignment</b><i>N</i><b>.rc</b> - 
          198  +    These files contain -D options to the compiler that are used by
          199  +    various notable downstreams.
          200  +<li><b>cov.rc</b> - Options for measuring test coverage
          201  +<li><b>extensions.rc</b> - Options to enable [FTS4], [R-Trees],
          202  +    and [SQLITE_STAT4|STAT4].
          203  +<li><b>fast.rc</b> - Run mosts tests, including those not needed for
          204  +    coverage, skipping only soak tests, using delivery compiler options
          205  +    (ex: -O3)
          206  +<li><b>memdebug.rc</b> - like test.rc but also enable
          207  +    [SQLITE_MEMDEBUG|-DSQLITE_MEMDEBUG].
          208  +<li><b>min.rc</b> - Run the minimum set of tests needed for 100% coverage.
          209  +<li><b>quick.rc</b> - Run all tests required for 100% coverage tests
          210  +    using -Os and [SQLITE_DEBUG|-DSQLITE_DEBUG].
          211  +<li><b>test.rc</b> - Run the same tests as in fast.rc but without
          212  +    compiler optimization and enabling options like
          213  +    [SQLITE_DEBUG|-DSQLITE_DEBUG] and
          214  +    -DSQLITE_ENABLE_EXPENSIVE_ASSERT.
          215  +<li><b>test-ex.rc</b> - long-running soak tests.
          216  +</ul>
          217  +
          218  +<p>The TH3 repository also includes the "multitest.tcl" script, another
          219  +TCL script used to automate TH3 testing on workstations.  Multitest.tcl
          220  +automatically compiles SQLite, then
          221  +runs ./th3make repeatedly with a variety of alignments, and captures
          222  +the output in a succinct summary screen.  A typical multitest.tcl run
          223  +generates output that looks like this:
          224  +
          225  +<blockquote><pre>
          226  +file mkdir sqlite3bld
          227  +cd sqlite3bld
          228  +exec sh /home/drh/sqlite/sqlite/configure
          229  +file copy -force config.h ../config.h
          230  +exec make clean sqlite3.c
          231  +file rename sqlite3.c ../sqlite3.c
          232  +aa4f0f90c9c77424943e026a2ecee4a6c7f9e0d3  ../sqlite3.c
          233  +file rename sqlite3.h ../sqlite3.h
          234  +exec make clean sqlite3.c OPTS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1
          235  +file rename sqlite3.c ../sqlite3udl.c
          236  +0d3bbc92c433f940253bb2c7c19de7783133929d  ../sqlite3udl.c
          237  +exec make clean sqlite3.c OPTS=-DSQLITE_SMALL_STACK=1
          238  +file rename sqlite3.c ../sqlite3ss.c
          239  +fcf6963e94096324461076d3b9e9dc1888e066e1  ../sqlite3ss.c
          240  +cd ..
          241  +*******************************************************************************
          242  +t01: quick.rc.................................................. Ok   (00:04:00)
          243  +t02: cov.rc.................................................... Ok   (00:04:40)
          244  +t03: quick.rc extensions.rc -D_HAVE_SQLITE_CONFIG_H............ Ok   (00:05:22)
          245  +t04: cov.rc -DSQLITE_ENABLE_STAT4 -D_HAVE_SQLITE_CONFIG_H...... Ok   (00:05:20)
          246  +t05: test.rc ../th3private/*.test.............................. Ok   (00:00:17)
          247  +t06: test.rc ../th3private/*.test -DSQLITE_ENABLE_STAT4........ Ok   (00:00:43)
          248  +t07: quick.rc -DSQLITE_TEST_REALLOC_STRESS -funsigned-char..... Ok   (00:04:56)
          249  +t08: quick.rc -DSQLITE_THREADSAFE=0 -fsigned-char.............. Ok   (00:03:12)
          250  +t09: quick.rc sqlite3ss.c -DSQLITE_MAX_ATTACHED=125............ Ok   (00:04:04)
          251  +t10: quick.rc -DSQLITE_RUNTIME_BYTEORDER....................... Ok   (00:03:58)
          252  +t11: quick.rc -DSQLITE_DIRECT_OVERFLOW_READ.................... Ok   (00:04:01)
          253  +t12: fast.rc................................................... Ok   (00:14:19)
          254  +t13: fast.rc alignment1.rc -m32................................ Ok   (00:20:51)
          255  +t14: fast.rc alignment2.rc sqlite3udl.c........................ Ok   (00:16:06)
          256  +t15: fast.rc alignment4.rc..................................... Ok   (00:12:55)
          257  +t16: fast.rc alignment5.rc..................................... Ok   (00:14:58)
          258  +t17: fast.rc alignment6.rc..................................... Ok   (00:14:31)
          259  +t18: fast.rc alignment7.rc..................................... Ok   (00:16:06)
          260  +t19: fast.rc alignment8.rc sqlite3udl.c........................ Ok   (00:24:09)
          261  +t20: test.rc alignment1.rc..................................... Ok   (00:49:27)
          262  +t21: test.rc alignment2.rc sqlite3udl.c........................ Ok   (00:38:43)
          263  +t22: test.rc alignment4.rc -m32 CC=clang....................... Ok   (00:39:49)
          264  +t23: test.rc alignment5.rc..................................... Ok   (00:36:33)
          265  +t24: test.rc alignment6.rc..................................... Ok   (00:33:53)
          266  +t25: test.rc alignment7.rc..................................... Ok   (00:42:16)
          267  +t26: test.rc alignment8.rc sqlite3udl.c........................ Ok   (01:05:22)
          268  +t27: memdebug.rc extensions.rc................................. Ok   (01:35:56)
          269  +t28: fast.rc -fsanitize=undefined.............................. Ok   (00:15:09)
          270  +t29: min.rc -O3 -valgrind...................................... Ok   (01:26:10)
          271  +t30: min.rc -O3 -valgrind extensions.rc........................ Ok   (01:47:12)
          272  +t31: test-ex.rc................................................ Ok   (03:20:18)
          273  +*******************************************************************************
          274  +0 failures on 31 th3make runs and 166721387 tests in (16:25:29)
          275  +SQLite 3.8.10 2015-05-05 18:52:54 04afa3febee32854fbb09ef8d4ffffd432119716
          276  +</pre></blockquote>
          277  +
          278  +<p>As can be seen above, a single run
          279  +of multitest.tcl invokes th3make dozens times and takes between 12 and 24
          280  +hours.  The middle section of the output shows the arguments to each individual
          281  +th3make run and the result and elapse time for that th3make.
          282  +All build products and output for the separate th3make runs are
          283  +captures in subdirectories for post-test analysis.
          284  +The two-line summary at the bottom shows the total number of errors and tests
          285  +over all th3make runs and the total elapse time, together with the 
          286  +[SQLITE_SOURCE_ID] information for the version of SQLite that was
          287  +tested.  This summary information is recorded in the
          288  +<a href="https://www.sqlite.org/checklists/3081000/index#c6">release
          289  +checklist</a> during testing.
   130    290   
   131    291   <h2>4.0 Test Coverage</h2>
   132    292   
   133    293   <p>Using one particular subset of the available TH3 test modules (the "cov1"
   134    294   tests) SQLite obtained 
   135    295   [test coverage | 100% branch test coverage] and 100% [MC/DC] as measured
   136    296   by [http://gcc.gnu.org/onlinedocs/gcc/Gcov.html | gcov]