Documentation Source Text
Check-in [835c236ee8]
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
SHA1 Hash:835c236ee84d2d33dc0a851eb77e43ce3c7afb42
Date: 2014-03-17 18:30:59
User: drh
Comment:Add a note about the use of checklists to the "testing.html" page.
Tags And Properties
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/testing.in

766
767
768
769
770
771
772
773




















774
775
776
777
778
779
780
781
782
...
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
full-scan steps, or other processing steps that occur during queries.
Those test cases will appear to fail when optimizations are disabled.
But the majority of test cases simply check that the correct answer
was obtained, and all of those cases can be run successfully with and
without the optimizations, in order to show that the optimizations do not
cause malfunctions.</p>






















<tcl>hd_fragment staticanalysis</tcl>
<h2>10.0 Static Analysis</h2>

<p>Static analysis means analyzing code at or before compile-time to
check for correctness.  Static analysis includes looking at compiler
warning messages and running the code through more in-depth
analysis engines such as the
[http://clang-analyzer.llvm.org/ | Clang Static Analyzer].
SQLite compiles without warnings on GCC and Clang using 
................................................................................
<p>Static analysis has not proven to be especially helpful in finding
bugs in SQLite.  Static analysis has found a few bugs in SQLite, but
those are the exceptions.  More bugs have been
introduced into SQLite while trying to get it to compile without 
warnings than have been found by static analysis.</p>

<tcl>hd_fragment summary</tcl>
<h2>11.0 Summary</h2>

<p>SQLite is open source.  This gives many people the idea that
it is not well tested as commercial software and is perhaps unreliable.
But that impression is false.  
SQLite has exhibited very high reliability in the field and
a very low defect rate, especially considering how rapidly it is evolving.
The quality of SQLite is achieved in part by careful code design and
implementation.  But extensive testing also plays a vital role in
maintaining and improving the quality of SQLite.  This document has
summarized the testing procedures that every release of SQLite undergoes
with the hopes of inspiring the reader to understand that SQLite is
suitable for use in mission-critical applications.</p>








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

|







 







|












766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
...
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
full-scan steps, or other processing steps that occur during queries.
Those test cases will appear to fail when optimizations are disabled.
But the majority of test cases simply check that the correct answer
was obtained, and all of those cases can be run successfully with and
without the optimizations, in order to show that the optimizations do not
cause malfunctions.</p>


<tcl>hd_fragment cklist</tcl>
<h2>10.0 Checklists</h2>

<p>The SQLite developers use an on-line checklist to coordinate testing
activity and to verify that all tests pass prior each SQLite release.
<a href="http://www.sqlite.org/checklists/index.html">Past checklists</a>
are retained for historical reference.</p>

<p>The latest checklists contain approximately 200 items that are
individually checked for each release.  Some checklist items only take
a few seconds to verify and mark off.  Others involve test suites
that run for many hours.</p>

<p>The release checklist is not automated: developers run each item on
the checklist manually.  We find that it is important to keep a human in
the loop.  Sometimes problems are found while running a checklist item
even though the test itself passed.  It is important to have a human
reviewing the test output at the highest level, and constantly asking
"Is this really right?"</p>

<tcl>hd_fragment staticanalysis</tcl>
<h2>11.0 Static Analysis</h2>

<p>Static analysis means analyzing code at or before compile-time to
check for correctness.  Static analysis includes looking at compiler
warning messages and running the code through more in-depth
analysis engines such as the
[http://clang-analyzer.llvm.org/ | Clang Static Analyzer].
SQLite compiles without warnings on GCC and Clang using 
................................................................................
<p>Static analysis has not proven to be especially helpful in finding
bugs in SQLite.  Static analysis has found a few bugs in SQLite, but
those are the exceptions.  More bugs have been
introduced into SQLite while trying to get it to compile without 
warnings than have been found by static analysis.</p>

<tcl>hd_fragment summary</tcl>
<h2>12.0 Summary</h2>

<p>SQLite is open source.  This gives many people the idea that
it is not well tested as commercial software and is perhaps unreliable.
But that impression is false.  
SQLite has exhibited very high reliability in the field and
a very low defect rate, especially considering how rapidly it is evolving.
The quality of SQLite is achieved in part by careful code design and
implementation.  But extensive testing also plays a vital role in
maintaining and improving the quality of SQLite.  This document has
summarized the testing procedures that every release of SQLite undergoes
with the hopes of inspiring the reader to understand that SQLite is
suitable for use in mission-critical applications.</p>