Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Fix typos in the testing.html document. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
41a06717b5ff5f0345c9b57151c02c7f |
User & Date: | drh 2009-08-24 23:05:28.000 |
Context
2009-08-24
| ||
23:31 | Fix typos in malloc.html. (check-in: 292c7157b4 user: drh tags: trunk) | |
23:05 | Fix typos in the testing.html document. (check-in: 41a06717b5 user: drh tags: trunk) | |
14:50 | Begin updating the change history in preparation for 3.6.18. Modify the download page to reference the new fossil repositories. Link in the www2.sqlite.org backup website. (check-in: 9f9705ea83 user: drh tags: trunk) | |
Changes
Changes to pages/testing.in.
︙ | ︙ | |||
279 280 281 282 283 284 285 | completed successfully or else were completely rolled back. The [integrity_check] [PRAGMA] is used to make sure no database corruption occurs.</p> <p>The TH3 test harness needs to run on embedded systems that do not necessarily have the ability to spawn child processes, so it uses an in-memory VFS to simulate crashes. The in-memory VFS can be rigged | | | 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 | completed successfully or else were completely rolled back. The [integrity_check] [PRAGMA] is used to make sure no database corruption occurs.</p> <p>The TH3 test harness needs to run on embedded systems that do not necessarily have the ability to spawn child processes, so it uses an in-memory VFS to simulate crashes. The in-memory VFS can be rigged to make a snapshot of the entire filesystem after a set number of I/O operations. Crash tests run in a loop. On each iteration of the loop, the point at which a snapshot is made is advanced until the SQLite operations being tested run to completion without ever hitting a snapshot. Within the loop, after the SQLite operation under test has completed, the filesystem is reverted to the snapshot and random file damage is introduced that is characteristic of the kinds of damage one expects to see following a power loss. Then the database is opened |
︙ | ︙ | |||
447 448 449 450 451 452 453 | programming dilemma: Does one remove defensive code in order to obtain 100% branch coverage?</p> <p>In SQLite, the answer to the previous question is "no". For testing purposes, the SQLite source code defines macros called ALWAYS() and NEVER(). The ALWAYS() macro surrounds conditions | | | | 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 | programming dilemma: Does one remove defensive code in order to obtain 100% branch coverage?</p> <p>In SQLite, the answer to the previous question is "no". For testing purposes, the SQLite source code defines macros called ALWAYS() and NEVER(). The ALWAYS() macro surrounds conditions which are expected to always evaluate as true and NEVER() surrounds conditions that are always evaluated to false. These macros serve as comments to indicate that the conditions are defensive code. For standard builds, these macros are pass-throughs:</p> <blockquote><pre> #define ALWAYS(X) (X) #define NEVER(X) (X) </pre></blockquote> |
︙ | ︙ | |||
577 578 579 580 581 582 583 | <p>The developers of SQLite have found that full coverage testing is an extremely productive method for preventing the introduction of new bugs as the system evolves. Because every single branch instruction in SQLite core code is covered by test cases, the developers can be confident that changes they make in one part of the code do not have unintended consequences in other parts of the code. | | | 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 | <p>The developers of SQLite have found that full coverage testing is an extremely productive method for preventing the introduction of new bugs as the system evolves. Because every single branch instruction in SQLite core code is covered by test cases, the developers can be confident that changes they make in one part of the code do not have unintended consequences in other parts of the code. It would be extremely difficult maintain the quality of SQLite without such assurances.</p> <h2>8.0 Dynamic Analysis</h2> <p>Dynamic analysis refers to internal and external checks on the SQLite code which are performed while the code is live and running. Dynamic analysis has proven to be a great help in maintaining the |
︙ | ︙ | |||
627 628 629 630 631 632 633 | <p>Because it is a simulator, running a binary in valgrind is slower than running it on native hardware. So it is impractical to run the full SQLite test suite through valgrind. However, the veryquick tests and a subset of the TH3 tests are run through valgrind prior to every release.</p> <h3>8.3 Memsys2</h3> | | > | | 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 | <p>Because it is a simulator, running a binary in valgrind is slower than running it on native hardware. So it is impractical to run the full SQLite test suite through valgrind. However, the veryquick tests and a subset of the TH3 tests are run through valgrind prior to every release.</p> <h3>8.3 Memsys2</h3> <p>SQLite contains a pluggable [memory allocation | memory allocation subsystem]. The default implementation uses system malloc() and free(). However, if SQLite is compiled with [SQLITE_MEMDEBUG], an alternative memory allocation wrapper ([memsys2]) is inserted that looks for memory allocation errors at run-time. The memsys2 wrapper checks for memory leaks, of course, but also looks for buffer overruns, uses of uninitialized memory, and attempts to use memory after it has been freed. These same checks are also done by valgrind (and, indeed, valgrind does them better) but memsys2 has the advantage of being much faster than valgrind, which |
︙ | ︙ | |||
660 661 662 663 664 665 666 | are atomic across system crashes and power failures is to write all changes into the rollback journal file prior to changing the database. The TCL test harness contains an alternative [sqlite3_vfs | Virtual File System ] implementation that helps to verify this is occurring correctly. The "journal-test VFS" monitors all disk I/O traffic between the database file and rollback journal, checking to make sure that nothing is written into the database | | | 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 | are atomic across system crashes and power failures is to write all changes into the rollback journal file prior to changing the database. The TCL test harness contains an alternative [sqlite3_vfs | Virtual File System ] implementation that helps to verify this is occurring correctly. The "journal-test VFS" monitors all disk I/O traffic between the database file and rollback journal, checking to make sure that nothing is written into the database file which has not first been written and synced to the rollback journal. If any discrepancies are found, an assertion fault is raised.</p> <p>The journal tests are an additional double-check over and above the crash tests to make sure that SQLite transactions will be atomic across system crashes and power failures.</p> <h2>9.0 Static Analysis</h2> |
︙ | ︙ |
Changes to pages/th3.in.
︙ | ︙ | |||
40 41 42 43 44 45 46 | linked against the SQLite library under test. The generated test program is compiled and run on the target platform in order to verify correct operation of SQLite on that platform.</p> <p>The inputs to TH3 are test modules written in C or SQL and small configuration files that determine how to initialize SQLite. The | | | | 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 | linked against the SQLite library under test. The generated test program is compiled and run on the target platform in order to verify correct operation of SQLite on that platform.</p> <p>The inputs to TH3 are test modules written in C or SQL and small configuration files that determine how to initialize SQLite. The TH3 package includes hundreds of test modules and dozens of configuration files. New modules and configurations can be added to customize TH3 for specialized applications. Each time TH3 is run, it reads a subset of the available test modules and configuration files to generate a custom C program the performs all of the specified tests under all configurations. A complete test of SQLite normally involves running TH3 multiple times to generate multiple test programs covering different aspects of SQLite's operation, then linking all test programs against a common SQLite library and running them separately on the target platform. SQLite will be found to work if all test programs pass.</p> <p>There are no arbitrary limits in TH3. One could generate a single test program that contained all test modules and configuration files. However, such a test program might be too large to deploy on embedded platforms. Hence, TH3 provides the ability to break the library of test modules up into smaller, more easily digested pieces.</p> <p>Each individual test module might contain dozens, hundreds, or thousands of separate tests. The test modules can be written in C or as scripts of SQL. Test modules done in SQL are very easy to write and the test modules written in C are not as nearly as cumbersome to write as one might suppose. The TH3 system provides high-level interfaces that simplify test writing. A typical C-language test case in TH3 might contains slightly more syntax that the corresponding TCL-script test, but the difference is not that great. The test modules written as SQL contain special comments that define the boundaries and operation of each test case and the expected results.</p> |
︙ | ︙ | |||
137 138 139 140 141 142 143 | a license to access and use TH3.</p> <p>Licensees of TH3 are given read access to the software configuration management system used to manage TH3 and so can download the latest version of TH3 (or any historical version) whenever they like.</p> <p>Even though open-source users do not have direct access to TH3, all | | | 137 138 139 140 141 142 143 144 145 146 147 148 | a license to access and use TH3.</p> <p>Licensees of TH3 are given read access to the software configuration management system used to manage TH3 and so can download the latest version of TH3 (or any historical version) whenever they like.</p> <p>Even though open-source users do not have direct access to TH3, all users of SQLite benefit from TH3 indirectly since each version of SQLite is validated by TH3 prior to release. So anyone using an official release of SQLite can deploy their application with the confidence of knowing that it has been tested using TH3. They simply cannot rerun those tests themselves without purchasing a TH3 license.</p> |