Documentation Source Text

Check-in [8a2b7d834e]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Additional remarks on the use of checklists in the testing.html document.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 8a2b7d834ec47f478615cc2c53ccaa69d8f5fd99
User & Date: drh 2014-05-26 13:00:46.811
Context
2014-05-26
13:10
Fix typos and clarify the text in the TH3 document. (check-in: 8e68e11794 user: drh tags: trunk)
13:00
Additional remarks on the use of checklists in the testing.html document. (check-in: 8a2b7d834e user: drh tags: trunk)
12:43
Better support for Windows in the howtocompile.html document. (check-in: 55ab2074b0 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/testing.in.
128
129
130
131
132
133
134

135
136
137
138
139
140
141
<li> Boundary value tests
<li> Disabled optimization tests
<li> Regression tests
<li> Malformed database tests
<li> Extensive use of assert() and run-time checks
<li> Valgrind analysis
<li> Signed-integer overflow checks

</ul>

<tcl>hd_fragment {harnesses} {test harness} {three test harnesses}</tcl>
<h2>2.0 Test Harnesses</h2>

<p>There are three independent test harnesses used for testing the 
core SQLite library.







>







128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
<li> Boundary value tests
<li> Disabled optimization tests
<li> Regression tests
<li> Malformed database tests
<li> Extensive use of assert() and run-time checks
<li> Valgrind analysis
<li> Signed-integer overflow checks
<li> Checklists
</ul>

<tcl>hd_fragment {harnesses} {test harness} {three test harnesses}</tcl>
<h2>2.0 Test Harnesses</h2>

<p>There are three independent test harnesses used for testing the 
core SQLite library.
773
774
775
776
777
778
779
780




781
782
783
784
785
786
787
788
789
790
791
792







793
794
795
796
797
798
799

<tcl>hd_fragment cklist</tcl>
<h2>10.0 Checklists</h2>

<p>The SQLite developers use an on-line checklist to coordinate testing
activity and to verify that all tests pass prior each SQLite release.
<a href="http://www.sqlite.org/checklists/index.html">Past checklists</a>
are retained for historical reference.</p>





<p>The latest checklists contain approximately 200 items that are
individually checked for each release.  Some checklist items only take
a few seconds to verify and mark off.  Others involve test suites
that run for many hours.</p>

<p>The release checklist is not automated: developers run each item on
the checklist manually.  We find that it is important to keep a human in
the loop.  Sometimes problems are found while running a checklist item
even though the test itself passed.  It is important to have a human
reviewing the test output at the highest level, and constantly asking
"Is this really right?"</p>








<tcl>hd_fragment staticanalysis</tcl>
<h2>11.0 Static Analysis</h2>

<p>Static analysis means analyzing code at or before compile-time to
check for correctness.  Static analysis includes looking at compiler
warning messages and running the code through more in-depth







|
>
>
>
>












>
>
>
>
>
>
>







774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811

<tcl>hd_fragment cklist</tcl>
<h2>10.0 Checklists</h2>

<p>The SQLite developers use an on-line checklist to coordinate testing
activity and to verify that all tests pass prior each SQLite release.
<a href="http://www.sqlite.org/checklists/index.html">Past checklists</a>
are retained for historical reference.
The use of checklists for SQLite testing and other development activites
is inspired by <i>
[http://atulgawande.com/book/the-checklist-manifesto/ | The Checklist Manifesto]
</i>.</p>

<p>The latest checklists contain approximately 200 items that are
individually checked for each release.  Some checklist items only take
a few seconds to verify and mark off.  Others involve test suites
that run for many hours.</p>

<p>The release checklist is not automated: developers run each item on
the checklist manually.  We find that it is important to keep a human in
the loop.  Sometimes problems are found while running a checklist item
even though the test itself passed.  It is important to have a human
reviewing the test output at the highest level, and constantly asking
"Is this really right?"</p>

<p>The release checklist is continuously evolving.  As new problems or
potential problems are discovered, new checklist items are added to
make sure those problems do not appear in subsequent releases.  The
release checklist has proven to be an invaluable tool in helping to
ensure that nothing is overlooked during the testing process.</p>


<tcl>hd_fragment staticanalysis</tcl>
<h2>11.0 Static Analysis</h2>

<p>Static analysis means analyzing code at or before compile-time to
check for correctness.  Static analysis includes looking at compiler
warning messages and running the code through more in-depth