Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Add a note about the use of checklists to the "testing.html" page. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
835c236ee84d2d33dc0a851eb77e43ce |
User & Date: | drh 2014-03-17 18:30:59.458 |
Context
2014-03-26
| ||
18:12 | Add notes for the 3.8.4.2 patch release. (check-in: 35bef59b0d user: drh tags: trunk) | |
2014-03-17
| ||
18:30 | Add a note about the use of checklists to the "testing.html" page. (check-in: 835c236ee8 user: drh tags: trunk) | |
18:05 | Fix a hyperlink to the "application file format" page. (check-in: 06eecd10f7 user: drh tags: trunk) | |
Changes
Changes to pages/testing.in.
︙ | ︙ | |||
766 767 768 769 770 771 772 773 774 | full-scan steps, or other processing steps that occur during queries. Those test cases will appear to fail when optimizations are disabled. But the majority of test cases simply check that the correct answer was obtained, and all of those cases can be run successfully with and without the optimizations, in order to show that the optimizations do not cause malfunctions.</p> <tcl>hd_fragment staticanalysis</tcl> | > > > > > > > > > > > > > > > > > > > > | | 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 | full-scan steps, or other processing steps that occur during queries. Those test cases will appear to fail when optimizations are disabled. But the majority of test cases simply check that the correct answer was obtained, and all of those cases can be run successfully with and without the optimizations, in order to show that the optimizations do not cause malfunctions.</p> <tcl>hd_fragment cklist</tcl> <h2>10.0 Checklists</h2> <p>The SQLite developers use an on-line checklist to coordinate testing activity and to verify that all tests pass prior each SQLite release. <a href="http://www.sqlite.org/checklists/index.html">Past checklists</a> are retained for historical reference.</p> <p>The latest checklists contain approximately 200 items that are individually checked for each release. Some checklist items only take a few seconds to verify and mark off. Others involve test suites that run for many hours.</p> <p>The release checklist is not automated: developers run each item on the checklist manually. We find that it is important to keep a human in the loop. Sometimes problems are found while running a checklist item even though the test itself passed. It is important to have a human reviewing the test output at the highest level, and constantly asking "Is this really right?"</p> <tcl>hd_fragment staticanalysis</tcl> <h2>11.0 Static Analysis</h2> <p>Static analysis means analyzing code at or before compile-time to check for correctness. Static analysis includes looking at compiler warning messages and running the code through more in-depth analysis engines such as the [http://clang-analyzer.llvm.org/ | Clang Static Analyzer]. SQLite compiles without warnings on GCC and Clang using |
︙ | ︙ | |||
791 792 793 794 795 796 797 | <p>Static analysis has not proven to be especially helpful in finding bugs in SQLite. Static analysis has found a few bugs in SQLite, but those are the exceptions. More bugs have been introduced into SQLite while trying to get it to compile without warnings than have been found by static analysis.</p> <tcl>hd_fragment summary</tcl> | | | 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 | <p>Static analysis has not proven to be especially helpful in finding bugs in SQLite. Static analysis has found a few bugs in SQLite, but those are the exceptions. More bugs have been introduced into SQLite while trying to get it to compile without warnings than have been found by static analysis.</p> <tcl>hd_fragment summary</tcl> <h2>12.0 Summary</h2> <p>SQLite is open source. This gives many people the idea that it is not well tested as commercial software and is perhaps unreliable. But that impression is false. SQLite has exhibited very high reliability in the field and a very low defect rate, especially considering how rapidly it is evolving. The quality of SQLite is achieved in part by careful code design and implementation. But extensive testing also plays a vital role in maintaining and improving the quality of SQLite. This document has summarized the testing procedures that every release of SQLite undergoes with the hopes of inspiring the reader to understand that SQLite is suitable for use in mission-critical applications.</p> |