SQLite 3.14.0 Status Board

History    Checklist    Baseline

 
1.   Test procedure review. drh ok  *
 
a.   Review the "test/releasetest.tcl" script for completeness and accuracy. drh ok  *
b.   Review "multitest.tcl" and "alignment*.rc" scripts in TH3 for completeness. drh ok  *
c.   Review this checklist for completeness and accuracy. drh ok  *
 
2.   Source code change review. fossil diff --from release --to trunk drh ok  *
 
a.   No stray changes. drh ok+  *
b.   All significant changes are mentioned in the "changes" log of the documentation. drh ok  *
c.   New code complies with style guidelines. drh ok  *
d.   New code complies with design rules. drh ok  *
e.   Comments have been updated to reflect code changes. drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use. drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h). drh ok  *
k.   Significant compile-time options are recognized by the sqlite3_compileoption_used() interface. drh ok  *
 
3.   Documentation review: drh ok  *
 
a.   Latest release on the "index.html" page. drh ok  *
b.   Release announcement in "changes.in". drh ok  *
c.   Release accouncement in "news.html". drh ok  *
d.   No unresolved hyperlinks in the documentation build. drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code. drh ok  *
f.   The compile-time options are all up-to-date (compile.html). drh ok  *
g.   Database footprint is up-to-date (features.html). drh ok  *
h.   Test metrics are up-to-date (testing.html). drh ok  *
i.   Check new/changed documentation for spelling and gramatical mistakes. drh ok  *
 
4.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok++  *
b.   Mac OS X GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
c.   OpenBSD GCC using options -Wall drh ok  *
d.   Windows (32-bit) MSVC using option /W4 drh ok  *
e.   Windows (64-bit) MSVC using option /W4 drh ok  *
 
5.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a drh ok  *
 
6.   Tests for platform Linux x86_64: drh ok  *
 
a.   tclsh multitest.tcl (take care to update ../th3private)
0 failures on 37 th3makes and 171342851 tests in (08:15:09) 2 cores on bella • SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
b.   Check the "multitest.tcl" log files for undetected errors. drh ok  *
c.   tclsh releasetest.tcl
0 failures out of 9329983 tests in (02:29:27) 2 cores on bella • SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
d.   Check the "releasetest.tcl" log files for undetected errors. drh ok  *
e.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
g.   sh tool/symbols.sh - no incorrect exports or dependencies drh ok  *
 
7.   Tests for platform Linux ARM (BeagleBoard) drh ok  *
 
a.   tclsh th3make min.rc
th3: 0 errors in 1591357 tests. 4658.394 seconds on beaglebone 32-bit little-endian th3: SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
dan ok  *
b.   Recompile and test Fossil.
Rebuild works. Cloning works. Opening a repository works.
dan ok  *
 
8.   Tests for platform Mac OS X (x86): drh ok  *
 
a.   tclsh releasetest.tcl
0 failures out of 4641370 tests in (06:22:20) on Chanphen-Kruatawongs-iMac.local SQLite 3.14.0 2016-08-01 16:57:30 fd184e5a8f4577aa9a817eb55e771d8d1021b946
dan ok  *
b.   tclsh multitest.tcl --omit test-ex (after updating th3private)
0 failures on 28 th3makes and 124249480 tests in (21:05:18) on Chanphen-Kruatawongs-iMac.local SQLite 3.14.0 2016-08-01 16:57:30 fd184e5a8f4577aa9a817eb55e771d8d1021b946
dan ok  *
c.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
d.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
Sanitizer does not detect the induced errors on this mac.
drh ok  *
 
9.   Tests for platform Mac OS X 10.2 (PPC): drh ok  *
 
a.   tclsh th3make quick.rc
th3: 0 errors in 1587500 tests. 4877.491 seconds on crabapple.local. 32-bit big-endian • th3: SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
b.   Recompile and test Fossil. drh ok  *
c.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
10.   Tests for platform Windows 7.0/8.x (x86): drh ok  *
 
a.   tclsh multitest.tcl (after updating th3private)
0 failures on 31 th3makes and 128413326 tests in (12:08:20) 3 cores on poulenc • SQLite 3.14.0 2016-08-01 21:17:53 d8ef9f58643f13dd3d16dcde0d829ae08324f04b • 0 failures on 10 th3makes and 12033268 tests in (00:55:18) 3 cores on poulenc • SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok+  *
b.   tclsh multitest.tcl --scenario failures - verify failures detected drh ok  *
c.   tclsh test/releasetest.tcl
0 failures out of 1578796 tests in (04:24:26) on win7dev • SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
d.   tclsh test/releasetest.tcl --msvc
0 failures out of 1889353 tests in (03:13:07) on poulenc • SQLite 3.14.0 2016-08-01 17:06:44 90d2c490fc2ed4e073711b84f989ca4d496dcfb5 • 0 failures out of 1889432 tests in (03:16:44) on poulenc • SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok+  *
e.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok+  *
f.   nmake /f makefile.msc MEMDEBUG=1 DEBUG=3 fulltestonly
SQLite 2016-08-05 15:34:42 0f57effa3642e66a863f32cc7fba86d167084af3 • 0 errors out of 1203259 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
g.   nmake /f makefile.msc WIN32HEAP=1 DEBUG=4 fulltestonly
SQLite 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f • 0 errors out of 1205079 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
h.   sh tool/symbols-mingw.sh - no incorrect exports or dependencies drh ok  *
i.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
 
11.   Tests for platform OpenBSD (x86): drh ok  *
 
a.   make test
Failures in trace3-5.1 trace3-5.2. Deemed pass.
drh ok  *
b.   th3make quick.rc
th3: 0 errors in 1588696 tests. 2644.074 seconds on wolfman.devio.us 32-bit little-endian • th3: SQLite 3.14.0 2016-08-01 21:17:53 d8ef9f58643f13dd3d16dcde0d829ae08324f04b
drh ok  *
 
12.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation. drh ok  *
b.   No SQLite compiler warnings. drh ok  *
c.   tclsh run-all.tcl - on unix
0 errors out of 11879758 tests and 1244 invocations in 791 seconds
drh ok  *
d.   tclsh run-all.tcl - on windows
0 errors out of 11879758 tests and 1244 invocations in 3727 seconds
drh ok  *
 
13.   Build using the configure/make in main source tree. dan ok  *
 
a.   Make sure that autoconf has been run to update the configure script. dan ok  *
b.   Verify the correct version numbers have been installed. dan ok  *
 
14.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No SQLite compiler warnings. drh ok  *
b.   Used on active websites with no issues. drh ok  *
c.   fossil all dbstat --db-check drh ok  *
 
15.   Rebuild and use Firefox with the latest SQLite drh ok  *
 
16.   CEROD tests in the "test/" subdirectory. drh ok  *
 
17.   SEE tests using TH3: drh ok  *
 
a.   see.c
th3: 0 errors in 1660795 tests. 1079.901 seconds on win7dev 32-bit little-endian • th3: SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
b.   see-aes128-ofb.c
th3: 0 errors in 8092458 tests. 2841.571 seconds on bella 64-bit little-endian • th3: SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
c.   see-aes256-ofb.c
th3: 0 errors in 8091538 tests. 4833.135 seconds on bella 64-bit little-endian • th3: SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
d.   see-rc4.c
th3: 0 errors in 1633766 tests. 255.924 seconds on bella 64-bit little-endian • th3: SQLite 3.14.0 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f
drh ok  *
e.   see-aes128-ccm.c
th3: 0 errors in 1533436 tests. 638.470 seconds on macpro01 64-bit little-endian • th3: SQLite 3.14.0 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f
drh ok  *
f.   see-cccrypt.c
th3: 0 errors in 1614823 tests. 491.228 seconds on macpro 64-bit little-endian • th3: SQLite 3.14.0 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2
drh ok  *
g.   At least one of the above using "test.rc". drh ok  *
h.   At least one of the above using "memdebug.rc". drh ok  *
i.   At least one of the above on Linux. drh ok  *
j.   At least one of the above on Mac OS X. drh ok  *
k.   At least one of the above on Windows. drh ok  *
 
18.   SEE using configure; make test drh ok  *
 
a.   Linux
SQLite 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2 • 0 errors out of 134756 tests on bella Linux 64-bit little-endian
drh ok  *
b.   Mac
SQLite 2016-08-04 13:23:28 9adda385267d1a0ecff259b42a284913668441a2 • 0 errors out of 134103 tests on macpro Darwin 64-bit little-endian
drh ok  *
 
19.   ZIPVFS tests: drh ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.) dan ok  *
b.   th3make zipvfs.rc test.rc
th3: 0 errors in 9885589 tests. 3067.906 seconds on darkstar 64-bit little-endian th3: SQLite 3.14.0 2016-08-05 15:34:42 0f57effa3642e66a863f32cc7fba86d167084af3
dan ok  *
c.   th3make zipvfs.rc memdebug.rc
th3: 0 errors in 9900884 tests. 8181.998 seconds on bella 64-bit little-endian • th3: SQLite 3.14.0 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f
drh ok  *
d.   th3make zipvfs.rc quick.rc -valgrind
th3: 0 errors in 2712984 tests. 58812.910 seconds on darkstar 64-bit little-endian th3: SQLite 3.14.0 2016-08-05 16:16:26 ae72513af37cd806a6d94aaa7c47a740e119d3b1 ==21204== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
dan ok  *
e.   th3make zipvfs.rc quick.rc
Running on OSX: th3: 0 errors in 2692845 tests. 1561.434 seconds on Chanphen-Kruatawongs-iMac.local 64-bit little-endian th3: SQLite 3.14.0 2016-08-05 16:16:26 ae72513af37cd806a6d94aaa7c47a740e119d3b1
dan ok  *
f.   At least one of the above on Linux. dan ok  *
g.   At least one of the above on Mac OS X. dan ok  *
h.   At least one of the above on Windows.
th3: 0 errors in 2742202 tests. 2293.155 seconds on poulenc 64-bit little-endian • th3: SQLite 3.14.0 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f • Using: msvc.rc zipvfs.rc quick.rc
drh ok  *
i.   th3make zipvfs-cov.rc - verify full coverage
th3: 0 errors in 1230445 tests. 312.467 seconds on bella 64-bit little-endian • th3: SQLite 3.14.0 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f • Statement coverage: 100.00% hit 1567 missed 0 total 1567 • Branch coverage: 100.00% hit 924 missed 0 total 924
drh ok  *
 
20.   NX-Devkit testing drh ok  *
 
a.   Verify that devkit sources are up-to-date with SQLite and ZIPVFS by running update-src.sh and mkdevkit.tcl in the sbin subdirectory of the devkit source tree. Record nds_sqlite3.[ch] hashes in the comment.
e5ab6fe7f84de4b571c74692096dcd80b7f76a18 nds_sqlite3.c • ef3464731510a36bcdc9e4170f138ecdaca0eae2 nds_sqlite3.h
drh ok  *
b.   Build using: sh mkdevkit.sh; gcc -o nxc.o -c nx_compress.c drh ok  *
c.   Test using: tclsh th3make test.rc nxc.o -DNDS_ENABLE_COMPRESS
th3: 0 errors in 7006197 tests. 1203.423 seconds on bella 64-bit little-endian • th3: SQLite 3.14.0 2016-08-05 20:54:45 95578898835b933901603bd4d5e063f1219a016f
drh ok  *
d.   Verify failure if run without -DNDS_ENABLE_COMPRESS drh ok  *
 
21.   Amalgamation autoconf tarball. drh ok  *
 
a.   Builds.
[../sqlite/configure && make amalgamation-tarball] works
dan ok  *
b.   Works on Linux. dan ok  *
c.   Works on Mac OS X. drh ok  *
d.   Works on Windows MinGW
TEA works too
drh ok  *
e.   Works on Windows MSVC drh ok  *
 
22.   TEA archive builds correctly and runs: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X
Requires --with-tcl=/Applications/Xcode.app/Contents/Developer/Platforms/MacOSX.platform/Developer/SDKs/MacOSX10.11.sdk/System/Library/Frameworks/Tcl.framework/Versions/8.5
drh ok  *
 
23.   Run performance tests comparing the new SQLite release against the previous release and/or a release from one year ago. Document and justify any performance decrease. drh ok  *
 
a.   The Tcl "speed*.test" scripts.
32-bit windows. 24772334 (3.13) vs 23599983 (3.14)
drh ok  *
b.   The TH3 speed tests.
Mac 10.11.6. 3.171s (3.13) vs 3.131s (3.14)
drh ok  *
c.   The "speedtest1" program.
3.8.11.1: 1183G • 3.13.0: 1016G • 3.14.0: 994G • 16% and 2% faster
drh ok  *
d.   ./speedtest1 --nosync --trace 2 >x.txt; time ./sqlite3 x.db <x.txt >/dev/null
4.487s (3.13) vs 4.246s (3.14)
drh ok  *
e.   One or more of the above on Linux. drh ok  *
f.   One or more of the above on Windows. drh ok  *
g.   One or more of the above on Mac OS X. drh ok  *
 
24.   Run make wordcount; bash run-wordcount.bash $OPTIONS with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c drh ok  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync drh ok  *
 
25.   Compute the size of a standard build (gcc -Os -c sqlite3.c) and and record the size change in the comments. drh ok  *
 
a.   Versus the previous release.
465622 to 467052: 1430 bytes larger. 0.3% growth
drh ok  *
b.   Versus the trunk from 12 months ago.
464057 (3.8.11.1) to 467052: 2995 bytes larger, 0.6% growth
drh ok  *
 
26.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   Linux
94f01233e2b2ab22a4d531c8c3496f6288fd42b0 sqlite3.c • f8cb04f18c220906ebed56c8806baebe24b28c12 sqlite3.h
drh ok+  *
b.   Mac OS X
94f01233e2b2ab22a4d531c8c3496f6288fd42b0 sqlite3.c • f8cb04f18c220906ebed56c8806baebe24b28c12 sqlite3.h
drh ok+  *
c.   OpenBSD
94f01233e2b2ab22a4d531c8c3496f6288fd42b0 sqlite3.c • f8cb04f18c220906ebed56c8806baebe24b28c12 sqlite3.h
drh ok+  *
d.   Windows using native MinGW.
9fb970dddfffa2b8e5d03385c682ef99cfb6b836 sqlite3.c • f8cb04f18c220906ebed56c8806baebe24b28c12 sqlite3.h
drh ok  *
e.   Windows using NMAKE.
94f01233e2b2ab22a4d531c8c3496f6288fd42b0 sqlite3.c • f8cb04f18c220906ebed56c8806baebe24b28c12 sqlite3.h
drh ok  *
 
27.   Verify that databases (including FTS3 and RTREE databases), rollback journals, and WAL files created by the release candidate are readable and writeable by historical versions of SQLite (as long as no unsupported features are used) and vice versa.
Tested against 3.7.3 and 3.8.2 using backcompat.test: 0 errors out of 265 tests on darkstar Linux 64-bit little-endian
dan ok  *
 
28.   Cross-platform tests. Verify the ability of databases, WAL files, and rollback journals to be copied and used between 32-bit and 64-bit systems and between big-endian and little-endian systems. drh ok  *
 
a.   32-big to 64-little, database. drh ok  *
b.   32-big to 64-little, WAL. drh ok  *
c.   32-big to 64-little, rollback. drh ok  *
d.   64-little to 32-big, SEE database. drh ok  *
e.   64-little to 32-big, ZIPVFS database. drh ok  *
f.   64-little to 32-big, CEROD database. drh ok  *
 
29.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
c.   Windows (MinGW) drh ok  *
d.   Windows (MSVC) drh ok  *
e.   A database containing WITHOUT ROWID tables. drh ok  *
 
30.   Build and verify sqldiff drh ok  *
 
a.   Linux drh ok  *
b.   Mac drh ok  *
c.   Windows drh ok  *
 
31.   Verify that the "sqlite3-all.c" build target works and that the resulting "sqlite3-all.c" file and its include files ("sqlite3-?.c") compile and run the same as the single-file amalgamation. drh ok  *