Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk. |
|---|---|
| Timelines: | family | ancestors | descendants | both | branch-3.8.10 |
| Files: | files | file ages | folders |
| SHA1: |
643e08dfb8f8b021b125bbcfdb56e8f8 |
| User & Date: | drh 2015-05-15 14:53:51 |
Context
|
2015-05-15
| ||
| 15:56 | Improvements to the testing.html document. check-in: 0b4aaced39 user: drh tags: branch-3.8.10 | |
| 14:53 | Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk. check-in: 643e08dfb8 user: drh tags: branch-3.8.10 | |
| 14:51 | Link from the ATTACH documentation page to the DETACH page. check-in: b3cbd71993 user: drh tags: trunk | |
| 14:48 | Increased information about TH3. check-in: 7d4f8c9ada user: drh tags: trunk | |
|
2015-05-11
| ||
| 17:09 | Updates to the dbstat documentation to explain how it can be used to get information about attached databases other than "main". check-in: d33510f222 user: drh tags: trunk | |
Changes
Changes to pages/lang.in.
400 400 Section {ATTACH DATABASE} attach {attached *ATTACH} 401 401 402 402 RecursiveBubbleDiagram attach-stmt 403 403 </tcl> 404 404 405 405 <p> ^The ATTACH DATABASE statement adds another database 406 406 file to the current [database connection]. 407 -^The filename for the database to be attached is the value of 407 +Database files that were previously attached can be removed using 408 +the [DETACH DATABASE] command. 409 + 410 +<p>^The filename for the database to be attached is the value of 408 411 the expression that occurs before the AS keyword. 409 412 ^The filename of the database follows the same semantics as the 410 413 filename argument to [sqlite3_open()] and [sqlite3_open_v2()]; the 411 414 special name "[:memory:]" results in an [in-memory database] and an 412 415 empty string results in a new temporary database. 413 416 ^The filename argument can be a [URI filename] if URI filename processing 414 417 is enable on the database connection. The default behavior is for
Changes to pages/th3.in.
31 31 (Test coverage of the operating-system specific [VFSes] and extensions 32 32 such as FTS and RTREE is less than 100%). </p></li> 33 33 </ul> 34 34 35 35 <p>TH3 was originally written for validation testing only, but has 36 36 subsequently been used for development testing and debugging 37 37 as well, and has proven very helpful in those roles. A full-coverage 38 -test takes less than three minutes on a workstation and hence 38 +test takes less than five minutes on a workstation and hence 39 39 serves as a fast regression test during day-to-day maintenance 40 40 of the SQLite code base.</p> 41 41 42 42 <h2>2.0 Operation</h2> 43 43 44 44 <p>TH3 is a test program generator. The output of TH3 is a program 45 -written in ANSI-C and intended to be 45 +implemented in C-code and intended to be 46 46 linked against the SQLite library under test. The generated test 47 47 program is compiled and run on the target platform in order to verify 48 48 correct operation of SQLite on that platform.</p> 49 49 50 50 <p>The inputs to TH3 are test modules written in C or SQL and 51 51 small configuration 52 52 files that determine how to initialize SQLite. The 53 -TH3 package includes over one thousand test 54 -modules and several dozen configuration files. 53 +TH3 package includes over eleven hundred test 54 +modules and more than three dozen configuration files. 55 55 New modules and configurations 56 56 can be added to customize TH3 for specialized applications. 57 57 Each time TH3 is run, it reads 58 58 a subset of the available test modules and configuration files to generate 59 59 a custom C program that performs all of the specified tests under all 60 -configurations. A complete test of SQLite normally involves running 60 +specified configurations. A complete test of SQLite normally involves running 61 61 TH3 multiple times to generate multiple test programs covering different 62 62 aspects of SQLite's operation, then linking all test programs against 63 63 a common SQLite library and running them separately on the target platform. 64 64 SQLite will be found to work if all test programs pass.</p> 65 65 66 66 <p>There are no arbitrary limits in TH3. One could generate a 67 -single test program that contained all test modules and configuration files. 67 +single test program that contained all test modules and all configuration files. 68 68 However, such a test program might be too large to deploy on embedded 69 69 platforms. Hence, TH3 provides the ability to break the library of test 70 70 modules up into smaller, more easily digested pieces.</p> 71 71 72 72 <p>Each individual test module might contain dozens, hundreds, or thousands 73 73 of separate tests. The test modules can be written in C or as scripts of 74 74 SQL or a mixture of the two. About two-thirds of the existing test modules are ................................................................................ 117 117 <p>Once the test program is generated, it is run with no arguments to 118 118 perform the tests. Progress information as well as error diagnostics 119 119 appear on standard output. (Alternative output arrangements can be made 120 120 using a compile-time option for embedded devices that lack a standard 121 121 output channel.) The program returns zero if there are no 122 122 errors and non-zero if any problems were detected.</p> 123 123 124 -<p>TH3 comes with additional TCL scripts (for example: 125 -"th3make" and "fulltest.tcl") 126 -which help to automate the process of running mkth3.tcl, compiling th3.c and 127 -sqlite3.c, running the resulting test programs, and analyzing the results. 128 -These other scripts are for convenience only and are not required in order 129 -to make use of TH3. 124 +<p>Typical output from a single TH3 test program run looks like this: 125 + 126 +<blockquote><pre> 127 +With SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e 128 +-DSQLITE_COVERAGE_TEST 129 +-DSQLITE_NO_SYNC 130 +-DSQLITE_SYSTEM_MALLOC 131 +-DSQLITE_THREADSAFE=1 132 +Config-begin c1. 133 +Begin c1.pager08 134 +End c1.pager08 135 +Begin c1.build33 136 +End c1.build33 137 +Begin c1.orderby01 138 +End c1.orderby01 139 +<i>... 15014 lines of output omitted ....</i> 140 +Begin 64k.syscall01 141 +End 64k.syscall01 142 +Begin 64k.build01 143 +End 64k.build01 144 +Begin 64k.auth01 145 +End 64k.auth01 146 +Config-end 64k. TH3 memory used: 6373738 147 +Config-begin wal1. 148 +Begin wal1.wal37 149 +End wal1.wal37 150 +Config-end wal1. TH3 memory used: 100961 151 +All 226 VDBE coverage points reached 152 +th3: 0 errors out of 1442264 tests in 213.741 seconds. 64-bit little-endian 153 +th3: SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e 154 +</pre></blockquote> 155 + 156 +<p>The output begins with a report of the [SQLITE_SOURCE_ID] 157 +(cross-checked again [sqlite3_sourceid()]) for the 158 +SQLite under test and the compile-time options used as reported 159 +by [sqlite3_compileoption_get()]. The output concludes with a summary 160 +of the test results and a repeat of the [SQLITE_SOURCE_ID]. If any 161 +errors are detected, additional lines detail the problem. The error 162 +reporting lines always begin with a single space character so that they 163 +can be quickly extracted from large output files using: 164 + 165 +<blockquote><pre> 166 +grep "^ " 167 +</pre></blockquote> 168 + 169 +<p>The default output shows the beginning and end of each configuration 170 +and test module combination. In the example above "c1" and "64k" are 171 +configurations and "pager08", "build33", "orderby01", etc. are test modules. 172 +Compile-time and run-time options are available to increase or decrease 173 +the amount of output. 174 +The output can be increased by showing each test case within each 175 +test module. The output can be decreased 176 +by degrees: omitting test modules starts and stops, 177 +omitting configuration starts and stops, and finally by omitting all output. 178 +<h3>3.1 Test Automation Scripts</h3> 179 + 180 +<p>TH3 comes with additional TCL scripts that help automate the testing 181 +process on workstations. The "th3make" script automatically runs "mkth3.tcl" 182 +and "gcc" and then runs the resulting test program and checks the results. 183 +Arguments to th3make include all of the "*.test" test modules and 184 +"*.cfg" configurations that are to be included in the test. Additional 185 +options to th3make can cause the test program to be compiled using different 186 +compilers (GCC, Clang, MSVC), to use different output verbosity levels, to 187 +run the test program under valgrind, to check the output for coverage using 188 +gcov, and so forth. The th3make script also accepts "*.rc" filenames as 189 +arguments. These *.rc files are just collections of other arguments that 190 +are commonly used together for a single purpose. For example, the "quick.rc" 191 +file contains a set of eight arguments to th3make that run a fast (3-minute) 192 +full-coverage test. This allows the operator to type "./th3make quick.rc" as 193 +a short-cut to typing out all of the required command-line options. The 194 +following are a few of the more than 40 available *.rc files: 195 + 196 +<ul> 197 +<li><b>alignment</b><i>N</i><b>.rc</b> - 198 + These files contain -D options to the compiler that are used by 199 + various notable downstreams. 200 +<li><b>cov.rc</b> - Options for measuring test coverage 201 +<li><b>extensions.rc</b> - Options to enable [FTS4], [R-Trees], 202 + and [SQLITE_STAT4|STAT4]. 203 +<li><b>fast.rc</b> - Run mosts tests, including those not needed for 204 + coverage, skipping only soak tests, using delivery compiler options 205 + (ex: -O3) 206 +<li><b>memdebug.rc</b> - like test.rc but also enable 207 + [SQLITE_MEMDEBUG|-DSQLITE_MEMDEBUG]. 208 +<li><b>min.rc</b> - Run the minimum set of tests needed for 100% coverage. 209 +<li><b>quick.rc</b> - Run all tests required for 100% coverage tests 210 + using -Os and [SQLITE_DEBUG|-DSQLITE_DEBUG]. 211 +<li><b>test.rc</b> - Run the same tests as in fast.rc but without 212 + compiler optimization and enabling options like 213 + [SQLITE_DEBUG|-DSQLITE_DEBUG] and 214 + -DSQLITE_ENABLE_EXPENSIVE_ASSERT. 215 +<li><b>test-ex.rc</b> - long-running soak tests. 216 +</ul> 217 + 218 +<p>The TH3 repository also includes the "multitest.tcl" script, another 219 +TCL script used to automate TH3 testing on workstations. Multitest.tcl 220 +automatically compiles SQLite, then 221 +runs ./th3make repeatedly with a variety of alignments, and captures 222 +the output in a succinct summary screen. A typical multitest.tcl run 223 +generates output that looks like this: 224 + 225 +<blockquote><pre> 226 +file mkdir sqlite3bld 227 +cd sqlite3bld 228 +exec sh /home/drh/sqlite/sqlite/configure 229 +file copy -force config.h ../config.h 230 +exec make clean sqlite3.c 231 +file rename sqlite3.c ../sqlite3.c 232 +aa4f0f90c9c77424943e026a2ecee4a6c7f9e0d3 ../sqlite3.c 233 +file rename sqlite3.h ../sqlite3.h 234 +exec make clean sqlite3.c OPTS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 235 +file rename sqlite3.c ../sqlite3udl.c 236 +0d3bbc92c433f940253bb2c7c19de7783133929d ../sqlite3udl.c 237 +exec make clean sqlite3.c OPTS=-DSQLITE_SMALL_STACK=1 238 +file rename sqlite3.c ../sqlite3ss.c 239 +fcf6963e94096324461076d3b9e9dc1888e066e1 ../sqlite3ss.c 240 +cd .. 241 +******************************************************************************* 242 +t01: quick.rc.................................................. Ok (00:04:00) 243 +t02: cov.rc.................................................... Ok (00:04:40) 244 +t03: quick.rc extensions.rc -D_HAVE_SQLITE_CONFIG_H............ Ok (00:05:22) 245 +t04: cov.rc -DSQLITE_ENABLE_STAT4 -D_HAVE_SQLITE_CONFIG_H...... Ok (00:05:20) 246 +t05: test.rc ../th3private/*.test.............................. Ok (00:00:17) 247 +t06: test.rc ../th3private/*.test -DSQLITE_ENABLE_STAT4........ Ok (00:00:43) 248 +t07: quick.rc -DSQLITE_TEST_REALLOC_STRESS -funsigned-char..... Ok (00:04:56) 249 +t08: quick.rc -DSQLITE_THREADSAFE=0 -fsigned-char.............. Ok (00:03:12) 250 +t09: quick.rc sqlite3ss.c -DSQLITE_MAX_ATTACHED=125............ Ok (00:04:04) 251 +t10: quick.rc -DSQLITE_RUNTIME_BYTEORDER....................... Ok (00:03:58) 252 +t11: quick.rc -DSQLITE_DIRECT_OVERFLOW_READ.................... Ok (00:04:01) 253 +t12: fast.rc................................................... Ok (00:14:19) 254 +t13: fast.rc alignment1.rc -m32................................ Ok (00:20:51) 255 +t14: fast.rc alignment2.rc sqlite3udl.c........................ Ok (00:16:06) 256 +t15: fast.rc alignment4.rc..................................... Ok (00:12:55) 257 +t16: fast.rc alignment5.rc..................................... Ok (00:14:58) 258 +t17: fast.rc alignment6.rc..................................... Ok (00:14:31) 259 +t18: fast.rc alignment7.rc..................................... Ok (00:16:06) 260 +t19: fast.rc alignment8.rc sqlite3udl.c........................ Ok (00:24:09) 261 +t20: test.rc alignment1.rc..................................... Ok (00:49:27) 262 +t21: test.rc alignment2.rc sqlite3udl.c........................ Ok (00:38:43) 263 +t22: test.rc alignment4.rc -m32 CC=clang....................... Ok (00:39:49) 264 +t23: test.rc alignment5.rc..................................... Ok (00:36:33) 265 +t24: test.rc alignment6.rc..................................... Ok (00:33:53) 266 +t25: test.rc alignment7.rc..................................... Ok (00:42:16) 267 +t26: test.rc alignment8.rc sqlite3udl.c........................ Ok (01:05:22) 268 +t27: memdebug.rc extensions.rc................................. Ok (01:35:56) 269 +t28: fast.rc -fsanitize=undefined.............................. Ok (00:15:09) 270 +t29: min.rc -O3 -valgrind...................................... Ok (01:26:10) 271 +t30: min.rc -O3 -valgrind extensions.rc........................ Ok (01:47:12) 272 +t31: test-ex.rc................................................ Ok (03:20:18) 273 +******************************************************************************* 274 +0 failures on 31 th3make runs and 166721387 tests in (16:25:29) 275 +SQLite 3.8.10 2015-05-05 18:52:54 04afa3febee32854fbb09ef8d4ffffd432119716 276 +</pre></blockquote> 277 + 278 +<p>As can be seen above, a single run 279 +of multitest.tcl invokes th3make dozens times and takes between 12 and 24 280 +hours. The middle section of the output shows the arguments to each individual 281 +th3make run and the result and elapse time for that th3make. 282 +All build products and output for the separate th3make runs are 283 +captures in subdirectories for post-test analysis. 284 +The two-line summary at the bottom shows the total number of errors and tests 285 +over all th3make runs and the total elapse time, together with the 286 +[SQLITE_SOURCE_ID] information for the version of SQLite that was 287 +tested. This summary information is recorded in the 288 +<a href="https://www.sqlite.org/checklists/3081000/index#c6">release 289 +checklist</a> during testing. 130 290 131 291 <h2>4.0 Test Coverage</h2> 132 292 133 293 <p>Using one particular subset of the available TH3 test modules (the "cov1" 134 294 tests) SQLite obtained 135 295 [test coverage | 100% branch test coverage] and 100% [MC/DC] as measured 136 296 by [http://gcc.gnu.org/onlinedocs/gcc/Gcov.html | gcov]