Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Updates to the testing page: talk of signed-integer overflow and verifying that it is not used. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
3793e03ba1929edae9d5bab57fdccf49 |
User & Date: | drh 2011-03-06 03:32:08.112 |
Context
2011-03-07
| ||
13:40 | Fix typos in Appendix A of fts3.html. (check-in: b3c69199b5 user: dan tags: trunk) | |
2011-03-06
| ||
03:32 | Updates to the testing page: talk of signed-integer overflow and verifying that it is not used. (check-in: 3793e03ba1 user: drh tags: trunk) | |
2011-02-23
| ||
21:27 | Make the opcode formatter resistent to comment text that looks like hyperlinks but is really an array index. (check-in: 479052b445 user: drh tags: trunk) | |
Changes
Changes to pages/testing.in.
︙ | ︙ | |||
128 129 130 131 132 133 134 135 136 137 138 139 140 141 | <li> Fuzz tests <li> Boundary value tests <li> Disabled optimization tests <li> Regression tests <li> Malformed database tests <li> Extensive use of assert() and run-time checks <li> Valgrind analysis </ul> <tcl>hd_fragment {harnesses} {test harness} {three test harnesses}</tcl> <h2>2.0 Test Harnesses</h2> <p>There are three independent test harnesses used for testing the core SQLite library. | > | 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 | <li> Fuzz tests <li> Boundary value tests <li> Disabled optimization tests <li> Regression tests <li> Malformed database tests <li> Extensive use of assert() and run-time checks <li> Valgrind analysis <li> Signed-integer overflow checks </ul> <tcl>hd_fragment {harnesses} {test harness} {three test harnesses}</tcl> <h2>2.0 Test Harnesses</h2> <p>There are three independent test harnesses used for testing the core SQLite library. |
︙ | ︙ | |||
717 718 719 720 721 722 723 724 725 726 727 728 729 730 | checking to make sure that nothing is written into the database file which has not first been written and synced to the rollback journal. If any discrepancies are found, an assertion fault is raised.</p> <p>The journal tests are an additional double-check over and above the crash tests to make sure that SQLite transactions will be atomic across system crashes and power failures.</p> <tcl>hd_fragment disopttest</tcl> <h2>9.0 Disabled Optimization Tests</h2> <p>The [sqlite3_test_control]([SQLITE_TESTCTRL_OPTIMIZATIONS], ...) interface allows selected SQL statement optimizations to be disabled at run-time. SQLite should always generate exactly the same answer with optimizations | > > > > > > > > > > > > > > > > > > > > > > > | 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 | checking to make sure that nothing is written into the database file which has not first been written and synced to the rollback journal. If any discrepancies are found, an assertion fault is raised.</p> <p>The journal tests are an additional double-check over and above the crash tests to make sure that SQLite transactions will be atomic across system crashes and power failures.</p> <tcl>hd_fragment intoverflow</tcl> <h3>8.6 Signed-Integer Overflow Checks</h3> <p>The various C language standards say that the signed-integer overflow behavior is undefined. In other words, when you add a value to a signed integer such that the result is too large to fit in that integer, the value does not necessarily wrap around to a negative number, as most programmers expect. It might do that. But it might do something completely different. See, for example, <a href="http://thiemonagel.de/2010/01/signed-integer-overflow/">here</a> and <a href="http://blog.regehr.org/archives/482">here</a>. Even the same compiler might do something different with signed integer overflow in different places in the code or at different optimizations settings.</p> <p>SQLite never overflows a signed integer. To verify this, the test suites are run at least once when compiled with the -ftrapv option to GCC. The -ftrapv option causes GCC to generate code that will panic() on a signed integer overflow. In addition, there are many test cases the strive to provoke integer overflows using boundary value calculations such as "<b>SELECT -1*(-9223372036854775808);</b>". <tcl>hd_fragment disopttest</tcl> <h2>9.0 Disabled Optimization Tests</h2> <p>The [sqlite3_test_control]([SQLITE_TESTCTRL_OPTIMIZATIONS], ...) interface allows selected SQL statement optimizations to be disabled at run-time. SQLite should always generate exactly the same answer with optimizations |
︙ | ︙ | |||
742 743 744 745 746 747 748 749 750 751 752 753 | computation by counting the number of disk accesses, sort operations, full-scan steps, or other processing steps that occur during queries. Those test cases will appear to fail when optimizations are disabled. But the majority of test cases simply check that the correct answer was obtained, and all of those cases can be run successfully with and without the optimizations, in order to show that the optimizations do not cause malfunctions.</p> <tcl>hd_fragment staticanalysis</tcl> <h2>10.0 Static Analysis</h2> <p>Static analysis means analyzing code at or before compile-time to | > | | > > | > > > > > | | < | | < < < < < | 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 | computation by counting the number of disk accesses, sort operations, full-scan steps, or other processing steps that occur during queries. Those test cases will appear to fail when optimizations are disabled. But the majority of test cases simply check that the correct answer was obtained, and all of those cases can be run successfully with and without the optimizations, in order to show that the optimizations do not cause malfunctions.</p> <tcl>hd_fragment staticanalysis</tcl> <h2>10.0 Static Analysis</h2> <p>Static analysis means analyzing code at or before compile-time to check for correctness. Static analysis includes looking at compiler warning messages and running the code through more in-depth analysis engines such as the [http://clang-analyzer.llvm.org/ | Clang Static Analyzer]. SQLite is developed primarily using GCC and it does compile without warnings on GCC using the -Wall and -Wextra flags. The SQLite source code is also run through Clang from time to time as well, though most of the warnings there are false-positives. VC++ often will generate a number of warnings from SQLite source code, but the experience of SQLite developers is that VC++ warnings are of lower quality and can be safely ignored. Users are encouraged not to stress over VC++ warnings.</p> <p>Static analysis has not proven to be helpful in finding bugs in SQLite. We cannot call to mind a single problem in SQLite that was detected by static analysis that was not first seen by one of the other testing methods described above. On the other hand, we have on occasion introduced new bugs in our efforts to get SQLite to compile without warnings.</p> <p>Our experience, then, is that static analysis is not helpful to maintaining high code quality and it is therefore deemphasized in SQLite development and maintenance.</p> <tcl>hd_fragment summary</tcl> <h2>11.0 Summary</h2> <p>SQLite is open source. This gives many people the idea that it is not well tested as commercial software and is perhaps unreliable. But that impression is false. |
︙ | ︙ |