Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | branch-3.8.10 |
Files: | files | file ages | folders |
SHA1: |
643e08dfb8f8b021b125bbcfdb56e8f8 |
User & Date: | drh 2015-05-15 14:53:51.361 |
Context
2015-05-15
| ||
15:56 | Improvements to the testing.html document. (check-in: 0b4aaced39 user: drh tags: branch-3.8.10) | |
14:53 | Link from the ATTACH documentation page to the DETACH page. Increased information about TH3. These are cherry-picks from trunk. (check-in: 643e08dfb8 user: drh tags: branch-3.8.10) | |
14:51 | Link from the ATTACH documentation page to the DETACH page. (check-in: b3cbd71993 user: drh tags: trunk) | |
14:48 | Increased information about TH3. (check-in: 7d4f8c9ada user: drh tags: trunk) | |
2015-05-11
| ||
17:09 | Updates to the dbstat documentation to explain how it can be used to get information about attached databases other than "main". (check-in: d33510f222 user: drh tags: trunk) | |
Changes
Changes to pages/lang.in.
︙ | ︙ | |||
400 401 402 403 404 405 406 | Section {ATTACH DATABASE} attach {attached *ATTACH} RecursiveBubbleDiagram attach-stmt </tcl> <p> ^The ATTACH DATABASE statement adds another database file to the current [database connection]. | > > > | | 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 | Section {ATTACH DATABASE} attach {attached *ATTACH} RecursiveBubbleDiagram attach-stmt </tcl> <p> ^The ATTACH DATABASE statement adds another database file to the current [database connection]. Database files that were previously attached can be removed using the [DETACH DATABASE] command. <p>^The filename for the database to be attached is the value of the expression that occurs before the AS keyword. ^The filename of the database follows the same semantics as the filename argument to [sqlite3_open()] and [sqlite3_open_v2()]; the special name "[:memory:]" results in an [in-memory database] and an empty string results in a new temporary database. ^The filename argument can be a [URI filename] if URI filename processing is enable on the database connection. The default behavior is for |
︙ | ︙ |
Changes to pages/th3.in.
︙ | ︙ | |||
31 32 33 34 35 36 37 | (Test coverage of the operating-system specific [VFSes] and extensions such as FTS and RTREE is less than 100%). </p></li> </ul> <p>TH3 was originally written for validation testing only, but has subsequently been used for development testing and debugging as well, and has proven very helpful in those roles. A full-coverage | | | | | | | | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | (Test coverage of the operating-system specific [VFSes] and extensions such as FTS and RTREE is less than 100%). </p></li> </ul> <p>TH3 was originally written for validation testing only, but has subsequently been used for development testing and debugging as well, and has proven very helpful in those roles. A full-coverage test takes less than five minutes on a workstation and hence serves as a fast regression test during day-to-day maintenance of the SQLite code base.</p> <h2>2.0 Operation</h2> <p>TH3 is a test program generator. The output of TH3 is a program implemented in C-code and intended to be linked against the SQLite library under test. The generated test program is compiled and run on the target platform in order to verify correct operation of SQLite on that platform.</p> <p>The inputs to TH3 are test modules written in C or SQL and small configuration files that determine how to initialize SQLite. The TH3 package includes over eleven hundred test modules and more than three dozen configuration files. New modules and configurations can be added to customize TH3 for specialized applications. Each time TH3 is run, it reads a subset of the available test modules and configuration files to generate a custom C program that performs all of the specified tests under all specified configurations. A complete test of SQLite normally involves running TH3 multiple times to generate multiple test programs covering different aspects of SQLite's operation, then linking all test programs against a common SQLite library and running them separately on the target platform. SQLite will be found to work if all test programs pass.</p> <p>There are no arbitrary limits in TH3. One could generate a single test program that contained all test modules and all configuration files. However, such a test program might be too large to deploy on embedded platforms. Hence, TH3 provides the ability to break the library of test modules up into smaller, more easily digested pieces.</p> <p>Each individual test module might contain dozens, hundreds, or thousands of separate tests. The test modules can be written in C or as scripts of SQL or a mixture of the two. About two-thirds of the existing test modules are |
︙ | ︙ | |||
117 118 119 120 121 122 123 | <p>Once the test program is generated, it is run with no arguments to perform the tests. Progress information as well as error diagnostics appear on standard output. (Alternative output arrangements can be made using a compile-time option for embedded devices that lack a standard output channel.) The program returns zero if there are no errors and non-zero if any problems were detected.</p> | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > | > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > | 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 | <p>Once the test program is generated, it is run with no arguments to perform the tests. Progress information as well as error diagnostics appear on standard output. (Alternative output arrangements can be made using a compile-time option for embedded devices that lack a standard output channel.) The program returns zero if there are no errors and non-zero if any problems were detected.</p> <p>Typical output from a single TH3 test program run looks like this: <blockquote><pre> With SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e -DSQLITE_COVERAGE_TEST -DSQLITE_NO_SYNC -DSQLITE_SYSTEM_MALLOC -DSQLITE_THREADSAFE=1 Config-begin c1. Begin c1.pager08 End c1.pager08 Begin c1.build33 End c1.build33 Begin c1.orderby01 End c1.orderby01 <i>... 15014 lines of output omitted ....</i> Begin 64k.syscall01 End 64k.syscall01 Begin 64k.build01 End 64k.build01 Begin 64k.auth01 End 64k.auth01 Config-end 64k. TH3 memory used: 6373738 Config-begin wal1. Begin wal1.wal37 End wal1.wal37 Config-end wal1. TH3 memory used: 100961 All 226 VDBE coverage points reached th3: 0 errors out of 1442264 tests in 213.741 seconds. 64-bit little-endian th3: SQLite 3.8.11 2015-05-15 04:13:15 56ef98a04765c34c1c2f3ed7a6f03a732f3b886e </pre></blockquote> <p>The output begins with a report of the [SQLITE_SOURCE_ID] (cross-checked again [sqlite3_sourceid()]) for the SQLite under test and the compile-time options used as reported by [sqlite3_compileoption_get()]. The output concludes with a summary of the test results and a repeat of the [SQLITE_SOURCE_ID]. If any errors are detected, additional lines detail the problem. The error reporting lines always begin with a single space character so that they can be quickly extracted from large output files using: <blockquote><pre> grep "^ " </pre></blockquote> <p>The default output shows the beginning and end of each configuration and test module combination. In the example above "c1" and "64k" are configurations and "pager08", "build33", "orderby01", etc. are test modules. Compile-time and run-time options are available to increase or decrease the amount of output. The output can be increased by showing each test case within each test module. The output can be decreased by degrees: omitting test modules starts and stops, omitting configuration starts and stops, and finally by omitting all output. <h3>3.1 Test Automation Scripts</h3> <p>TH3 comes with additional TCL scripts that help automate the testing process on workstations. The "th3make" script automatically runs "mkth3.tcl" and "gcc" and then runs the resulting test program and checks the results. Arguments to th3make include all of the "*.test" test modules and "*.cfg" configurations that are to be included in the test. Additional options to th3make can cause the test program to be compiled using different compilers (GCC, Clang, MSVC), to use different output verbosity levels, to run the test program under valgrind, to check the output for coverage using gcov, and so forth. The th3make script also accepts "*.rc" filenames as arguments. These *.rc files are just collections of other arguments that are commonly used together for a single purpose. For example, the "quick.rc" file contains a set of eight arguments to th3make that run a fast (3-minute) full-coverage test. This allows the operator to type "./th3make quick.rc" as a short-cut to typing out all of the required command-line options. The following are a few of the more than 40 available *.rc files: <ul> <li><b>alignment</b><i>N</i><b>.rc</b> - These files contain -D options to the compiler that are used by various notable downstreams. <li><b>cov.rc</b> - Options for measuring test coverage <li><b>extensions.rc</b> - Options to enable [FTS4], [R-Trees], and [SQLITE_STAT4|STAT4]. <li><b>fast.rc</b> - Run mosts tests, including those not needed for coverage, skipping only soak tests, using delivery compiler options (ex: -O3) <li><b>memdebug.rc</b> - like test.rc but also enable [SQLITE_MEMDEBUG|-DSQLITE_MEMDEBUG]. <li><b>min.rc</b> - Run the minimum set of tests needed for 100% coverage. <li><b>quick.rc</b> - Run all tests required for 100% coverage tests using -Os and [SQLITE_DEBUG|-DSQLITE_DEBUG]. <li><b>test.rc</b> - Run the same tests as in fast.rc but without compiler optimization and enabling options like [SQLITE_DEBUG|-DSQLITE_DEBUG] and -DSQLITE_ENABLE_EXPENSIVE_ASSERT. <li><b>test-ex.rc</b> - long-running soak tests. </ul> <p>The TH3 repository also includes the "multitest.tcl" script, another TCL script used to automate TH3 testing on workstations. Multitest.tcl automatically compiles SQLite, then runs ./th3make repeatedly with a variety of alignments, and captures the output in a succinct summary screen. A typical multitest.tcl run generates output that looks like this: <blockquote><pre> file mkdir sqlite3bld cd sqlite3bld exec sh /home/drh/sqlite/sqlite/configure file copy -force config.h ../config.h exec make clean sqlite3.c file rename sqlite3.c ../sqlite3.c aa4f0f90c9c77424943e026a2ecee4a6c7f9e0d3 ../sqlite3.c file rename sqlite3.h ../sqlite3.h exec make clean sqlite3.c OPTS=-DSQLITE_ENABLE_UPDATE_DELETE_LIMIT=1 file rename sqlite3.c ../sqlite3udl.c 0d3bbc92c433f940253bb2c7c19de7783133929d ../sqlite3udl.c exec make clean sqlite3.c OPTS=-DSQLITE_SMALL_STACK=1 file rename sqlite3.c ../sqlite3ss.c fcf6963e94096324461076d3b9e9dc1888e066e1 ../sqlite3ss.c cd .. ******************************************************************************* t01: quick.rc.................................................. Ok (00:04:00) t02: cov.rc.................................................... Ok (00:04:40) t03: quick.rc extensions.rc -D_HAVE_SQLITE_CONFIG_H............ Ok (00:05:22) t04: cov.rc -DSQLITE_ENABLE_STAT4 -D_HAVE_SQLITE_CONFIG_H...... Ok (00:05:20) t05: test.rc ../th3private/*.test.............................. Ok (00:00:17) t06: test.rc ../th3private/*.test -DSQLITE_ENABLE_STAT4........ Ok (00:00:43) t07: quick.rc -DSQLITE_TEST_REALLOC_STRESS -funsigned-char..... Ok (00:04:56) t08: quick.rc -DSQLITE_THREADSAFE=0 -fsigned-char.............. Ok (00:03:12) t09: quick.rc sqlite3ss.c -DSQLITE_MAX_ATTACHED=125............ Ok (00:04:04) t10: quick.rc -DSQLITE_RUNTIME_BYTEORDER....................... Ok (00:03:58) t11: quick.rc -DSQLITE_DIRECT_OVERFLOW_READ.................... Ok (00:04:01) t12: fast.rc................................................... Ok (00:14:19) t13: fast.rc alignment1.rc -m32................................ Ok (00:20:51) t14: fast.rc alignment2.rc sqlite3udl.c........................ Ok (00:16:06) t15: fast.rc alignment4.rc..................................... Ok (00:12:55) t16: fast.rc alignment5.rc..................................... Ok (00:14:58) t17: fast.rc alignment6.rc..................................... Ok (00:14:31) t18: fast.rc alignment7.rc..................................... Ok (00:16:06) t19: fast.rc alignment8.rc sqlite3udl.c........................ Ok (00:24:09) t20: test.rc alignment1.rc..................................... Ok (00:49:27) t21: test.rc alignment2.rc sqlite3udl.c........................ Ok (00:38:43) t22: test.rc alignment4.rc -m32 CC=clang....................... Ok (00:39:49) t23: test.rc alignment5.rc..................................... Ok (00:36:33) t24: test.rc alignment6.rc..................................... Ok (00:33:53) t25: test.rc alignment7.rc..................................... Ok (00:42:16) t26: test.rc alignment8.rc sqlite3udl.c........................ Ok (01:05:22) t27: memdebug.rc extensions.rc................................. Ok (01:35:56) t28: fast.rc -fsanitize=undefined.............................. Ok (00:15:09) t29: min.rc -O3 -valgrind...................................... Ok (01:26:10) t30: min.rc -O3 -valgrind extensions.rc........................ Ok (01:47:12) t31: test-ex.rc................................................ Ok (03:20:18) ******************************************************************************* 0 failures on 31 th3make runs and 166721387 tests in (16:25:29) SQLite 3.8.10 2015-05-05 18:52:54 04afa3febee32854fbb09ef8d4ffffd432119716 </pre></blockquote> <p>As can be seen above, a single run of multitest.tcl invokes th3make dozens times and takes between 12 and 24 hours. The middle section of the output shows the arguments to each individual th3make run and the result and elapse time for that th3make. All build products and output for the separate th3make runs are captures in subdirectories for post-test analysis. The two-line summary at the bottom shows the total number of errors and tests over all th3make runs and the total elapse time, together with the [SQLITE_SOURCE_ID] information for the version of SQLite that was tested. This summary information is recorded in the <a href="https://www.sqlite.org/checklists/3081000/index#c6">release checklist</a> during testing. <h2>4.0 Test Coverage</h2> <p>Using one particular subset of the available TH3 test modules (the "cov1" tests) SQLite obtained [test coverage | 100% branch test coverage] and 100% [MC/DC] as measured by [http://gcc.gnu.org/onlinedocs/gcc/Gcov.html | gcov] |
︙ | ︙ |