SQLite 3.18.0 Status Board

History    Checklist    Baseline

 
1.   Test procedure review. drh ok  *
 
a.   Review the "test/releasetest.tcl" script for completeness and accuracy. drh ok  *
b.   Review "multitest.tcl" and "alignment*.rc" scripts in TH3 for completeness. drh ok  *
c.   Review this checklist for completeness and accuracy. drh ok  *
 
2.   Source code change review. fossil diff --from release --to trunk drh ok  *
 
a.   No stray changes. drh ok  *
b.   All significant changes are mentioned in the "changes" log of the documentation. drh ok  *
c.   New code complies with style guidelines. drh ok  *
d.   New code complies with design rules. drh ok  *
e.   Comments have been updated to reflect code changes. drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use. drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h). drh ok  *
k.   Significant compile-time options are recognized by the sqlite3_compileoption_used() interface. drh ok  *
 
3.   Documentation review: drh ok  *
 
a.   Latest release on the "index.html" page. drh ok  *
b.   Release announcement in "changes.in". drh ok  *
c.   Release accouncement in "news.html". drh ok  *
d.   No unresolved hyperlinks in the documentation build. drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code. drh ok  *
f.   The compile-time options are all up-to-date (compile.html). drh ok  *
g.   Database footprint is up-to-date (features.html). drh ok  *
h.   Test metrics are up-to-date (testing.html). drh ok  *
i.   Check new/changed documentation for spelling and gramatical mistakes. drh ok  *
 
4.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
b.   Mac OS X GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
c.   OpenBSD GCC using options -Wall
[gcc -DSQLITE_ENABLE_FTS5 -DSQLITE_ENABLE_RTREE -c -Wall sqlite3.c] runs clean. OpenBSD 5.0. gcc version 4.2.1 20070719
dan ok  *
d.   Windows (32-bit) MSVC using option /W4 (nmake /f makefile.msc USE_FULLWARN=1) drh ok  *
e.   Windows (64-bit) MSVC using option /W4 (nmake /f makefile.msc USE_FULLWARN=1) drh ok  *
 
5.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a drh ok  *
 
6.   Tests for platform Linux x86_64: drh ok  *
 
a.   tclsh multitest.tcl (take care to update ../th3private)
0 failures on 42 th3makes and 184565712 tests in (05:30:40) 3 cores on bella • SQLite 3.18.0 2017-03-24 19:45:05 c2c3dd84534bb5ea81c974847b74a166c9cba1545fc749ce625929f303bf22e4
drh ok  *
b.   Check the "multitest.tcl" log files for undetected errors. drh ok  *
c.   tclsh releasetest.tcl
0 failures out of 9754130 tests in (01:42:50) 3 cores on darkstar • SQLite 3.18.0 2017-03-25 18:31:42 55df410b2cd27dc4c7670bcc1196fa3a0e9e601fc24c42e53d6b0a26ee3e4b45 • 0 failures out of 9755979 tests in (03:44:57) on bella • SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok+  *
d.   Check the "releasetest.tcl" log files for undetected errors. dan ok  *
e.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
All configurations report "FAIL". • 26 failures out of 786 tests in (00:08:22) on darkstar • SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
dan ok  *
g.   sh tool/symbols.sh - no incorrect exports or dependencies
All exports are sqlite3_*, sqlite3changeset_* or sqlite3session_*. Core deps are: malloc, free, realloc, localtime, memset, memmove, memcpy, memcmp, strcmp, strlen, strncmp.
dan ok++  *
 
7.   Tests for platform Linux ARM (BeagleBoard) dan ok  *
 
a.   tclsh th3make min.rc
th3: 0 errors in 1646876 tests. 4685.086 seconds on beaglebone 32-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
dan ok  *
b.   Recompile and test Fossil.
[fossil all rebuild] works. [fossil sync] works.
dan ok  *
 
8.   Tests for platform Mac OS X: dan ok+  *
 
a.   tclsh releasetest.tcl
0 failures out of 4745197 tests in (04:52:38) 2 cores on Chanphen-Kruatawongs-iMac.local • SQLite 3.18.0 2017-03-25 19:16:41 • 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
dan ok  *
b.   tclsh multitest.tcl --omit test-ex (after updating th3private)
0 failures on 37 th3makes and 148890242 tests in (09:04:55) 2 cores on macpro01 • SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok  *
c.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
d.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
 
9.   Tests for platform Mac OS X 10.2 (PPC): drh ok  *
 
a.   tclsh th3make quick.rc
th3: 0 errors in 2014952 tests. 12330.943 seconds on crabapple 32-bit big-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok  *
b.   Recompile and test Fossil. drh ok  *
c.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
10.   Tests for platform Windows: drh ok  *
 
a.   tclsh multitest.tcl (after updating th3private)
0 failures on 36 th3makes and 139722833 tests in (09:58:16) 2 cores on DESKTOP-8U9G1N8 • SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok  *
b.   tclsh multitest.tcl --quick on x86
0 failures on 15 th3makes and 21194460 tests in (04:13:21) on win7dev • SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok  *
c.   tclsh multitest.tcl --scenario failures - verify failures detected drh ok  *
d.   tclsh test/releasetest.tcl --msvc
0 failures out of 1960169 tests in (02:04:01) on DESKTOP-8U9G1N8 • SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok  *
e.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
f.   nmake /f makefile.msc MEMDEBUG=1 DEBUG=3 test
SQLite 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • 0 errors out of 112987 tests on DESKTOP-8U9G1N8 Windows NT 64-bit little-endian
drh ok  *
g.   nmake /f makefile.msc WIN32HEAP=1 DEBUG=4 test
SQLite 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • 0 errors out of 114036 tests on DESKTOP-8U9G1N8 Windows NT 64-bit little-endian
drh ok  *
h.   sh tool/symbols-mingw.sh - no incorrect exports or dependencies drh ok  *
i.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
 
11.   Tests for platform OpenBSD (x86): dan ok  *
 
a.   make test
SQLite 2017-03-25 12:08:11 32be9c3faaafd736da3f49d475dc9279af4e4ba38070b60033d898345080acf0 • 0 errors out of 140446 tests on obsd.my.domain OpenBSD 32-bit little-endian
dan ok  *
b.   th3make quick.rc
th3: 0 errors in 1768723 tests. 586.047 seconds on obsd.my.domain 32-bit little-endian • th3: SQLite 3.18.0 2017-03-25 12:08:11 32be9c3faaafd736da3f49d475dc9279af4e4ba38070b60033d898345080acf0
dan ok  *
 
12.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation. drh ok  *
b.   No SQLite compiler warnings. drh ok  *
c.   tclsh run-all.tcl - on unix
0 errors out of 11879758 tests and 1244 invocations in 1540 seconds
drh ok  *
d.   tclsh run-all.tcl - on windows
0 errors out of 11879758 tests and 1244 invocations in 2322 seconds
drh ok  *
 
13.   Build using the configure/make in main source tree. dan ok  *
 
a.   Make sure that autoconf has been run to update the configure script. dan ok  *
b.   Verify the correct version numbers have been installed. dan ok  *
 
14.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No SQLite compiler warnings. drh ok  *
b.   Used on active websites with no issues. drh ok  *
c.   fossil all dbstat --db-check drh ok  *
 
15.   Rebuild and use Firefox with the latest SQLite drh ok  *
 
a.   Routine use with no problems drh ok  *
b.   ./mach mochitest dom/indexedDB
1 INFO Passed: 26767 • 2 INFO Failed: 0
drh ok  *
c.   ./mach xpcshell-test dom/indexedDB
Ran 207 tests • Expected results: 205 • Unexpected results: 0 • Skipped: 2
drh ok  *
d.   ./mach web-platform-tests IndexedDB
Ran 1260 tests (279 parents, 981 subtests) • Expected results: 1260 • Unexpected results: 0
drh ok  *
 
16.   CEROD tests in the "test/" subdirectory. drh ok  *
 
17.   SEE tests using TH3: drh ok  *
 
a.   see.c
th3: 0 errors in 1721372 tests. 648.124 seconds on DESKTOP-8U9G1N8 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • Win10, quick.rc
drh ok  *
b.   see-aes128-ofb.c
th3: 0 errors in 8240519 tests. 4474.413 seconds on bella 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • Linux, memdebug.rc
drh ok  *
c.   see-aes256-ofb.c
th3: 0 errors in 1690391 tests. 315.932 seconds on macpro01 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • Mac, quick.rc
drh ok  *
d.   see-rc4.c
th3: 0 errors in 1690441 tests. 318.181 seconds on macpro01 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • Mac, quick.rc
drh ok  *
e.   see-aes128-ccm.c
th3: 0 errors in 1690620 tests. 363.802 seconds on macpro01 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • Mac, quick.rc
drh ok  *
f.   see-cccrypt.c
th3: 0 errors in 6456924 tests. 2263.575 seconds on macpro01 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • test.rc
drh ok+  *
g.   At least one of the above using "test.rc". drh ok  *
h.   At least one of the above using "memdebug.rc". drh ok  *
i.   At least one of the above on Linux. drh ok  *
j.   At least one of the above on Mac OS X. drh ok  *
k.   At least one of the above on Windows. drh ok  *
 
18.   SEE using configure; make test drh ok  *
 
a.   Linux
SQLite 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • 0 errors out of 140602 tests on bella Linux 64-bit little-endian
drh ok  *
b.   Mac
SQLite 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • 0 errors out of 139988 tests on macpro01 Darwin 64-bit little-endian
drh ok  *
 
19.   ZIPVFS tests: drh ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.)
SQLite 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • 0 errors out of 85663 tests on darkstar Linux 64-bit little-endian
dan ok  *
b.   th3make zipvfs.rc test.rc
th3: 0 errors in 10071701 tests. 2777.407 seconds on darkstar 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
dan ok  *
c.   th3make zipvfs.rc memdebug.rc
On OSX. • th3: 0 errors in 8316643 tests. 15297.417 seconds on Chanphen-Kruatawongs-iMac.local 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
dan ok  *
d.   th3make zipvfs.rc quick.rc -valgrind
th3: 0 errors in 2829955 tests. 25287.291 seconds on darkstar 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • ==31567== ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
dan ok  *
e.   th3make zipvfs.rc quick.rc
th3: 0 errors in 2895021 tests. 1584.360 seconds on DESKTOP-8U9G1N8 32-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
drh ok  *
f.   At least one of the above on Linux. dan ok  *
g.   At least one of the above on Mac OS X.
20c
dan ok  *
h.   At least one of the above on Windows. drh ok  *
i.   th3make zipvfs-cov.rc - verify full coverage
th3: 0 errors in 1254091 tests. 374.463 seconds on darkstar 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • Branch coverage: 100.00% hit 941 missed 0 total 941
dan ok  *
 
20.   NDS-Devkit testing drh ok  *
 
a.   Verify that devkit sources are up-to-date with SQLite and ZIPVFS by running update-src.sh and mkdevkit.tcl in the sbin subdirectory of the devkit source tree. Record nds_sqlite3.[ch] hashes in the comment.
Devkit updated. But the commit is not marked as release and there are no hashes in the comment.
drh ok  *
b.   Test using: tclsh th3make test.rc nds_devkit.rc
th3: 0 errors in 7085713 tests. 1487.272 seconds on darkstar 64-bit little-endian • th3: SQLite 3.18.0 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb
dan ok  *
c.   Verify the "NDS DevKit" version number shown at the end of the test
th3: NDS DevKit 3.18.0 2017-03-27 16:39:34 6fe0aa7fdd0fe14b
dan ok  *
 
21.   Amalgamation autoconf tarball. drh ok  *
 
a.   Builds. dan ok  *
b.   Works on Linux. dan ok  *
c.   Works on Mac OS X. dan ok  *
d.   Works on Windows MinGW drh ok  *
e.   Works on Windows MSVC drh ok  *
 
22.   TEA archive builds correctly and runs: dan ok  *
 
a.   Linux dan ok  *
b.   Mac OS X dan ok  *
 
23.   Performance tests using speedtest1 compared against the previous release and a release from about one year ago. drh ok  *
 
a.   Android
133.790s down to 128.610s
drh ok  *
b.   Linux
3.12.0: 4.773s, 3.17.0: 4.385s, 3.18.0: 4.322s. 9.4% and 1.4% faster. Wall-clock time on linux using "./configure; make speedtest1; time ./speedtest1 st1.db"
drh ok  *
c.   Mac
3.18.0 is faster than 3.16.0. Real-time preformance difference from 3.17.0 is difficult to measure.
drh ok  *
d.   Windows
Windows10 MSVC 64-bit. 3.12.0: 6.077s. 3.17.0: 5.484s. 3.18.0: 5.423s.
drh ok  *
 
24.   Performance tests using speedtest1 --testset rtree drh ok  *
 
a.   Linux
7.513 vs 7.504
drh ok  *
b.   Windows
9.986 vs 9.922 • MSVC on Win10 64-bit
drh ok  *
 
25.   Run make wordcount; bash run-wordcount.bash $OPTIONS with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c
14.060s on 3.17.0 down to 13.192 for 3.18.0. About 6% faster. (Mac) • 11.347s on 3.17.0 down to 11.020 for 3.18.0 (2.9%) on Linux.
drh ok+  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync
From 27.286s down to 26.700s (Mac, 2.1%)
drh ok  *
 
26.   Compare performance to the previous release using kvtest. drh ok  *
 
a.   Android
2858 microseconds down to 2819.
drh ok  *
b.   Linux
0.497 to 0.472
drh ok  *
c.   Mac
Same speed
drh ok  *
d.   Windows
Same speed
drh ok  *
 
27.   Use the ".selftest" command of the shell to verify that legacy database files are readable and cross-platform drh ok  *
 
a.   iBook PPC drh ok  *
b.   Linux drh ok  *
c.   Mac drh ok  *
d.   Windows drh ok  *
 
28.   Verify that the performance and size measurement spreadsheet in the documentation (misc/speed-size-graph.ods) is up-to-date. drh ok  *
 
29.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   Linux
254f84222b3d53f99980a63cc768da07b76d19211166ce20eac80dcbe867dbb6 sqlite3.c • 6c6ed9bc6793e48060a3815dd553ed13bf9568facf74ff5de2883d0737543091 sqlite3.h
drh ok  *
b.   Mac OS X
254f84222b3d53f99980a63cc768da07b76d19211166ce20eac80dcbe867dbb6 sqlite3.c • 6c6ed9bc6793e48060a3815dd553ed13bf9568facf74ff5de2883d0737543091 sqlite3.h
drh ok  *
c.   OpenBSD
254f84222b3d53f99980a63cc768da07b76d19211166ce20eac80dcbe867dbb6 sqlite3.c • 6c6ed9bc6793e48060a3815dd553ed13bf9568facf74ff5de2883d0737543091 sqlite3.h
dan ok  *
d.   Windows using native MinGW.
254f84222b3d53f99980a63cc768da07b76d19211166ce20eac80dcbe867dbb6 sqlite3.c • 6c6ed9bc6793e48060a3815dd553ed13bf9568facf74ff5de2883d0737543091 sqlite3.h
drh ok  *
e.   Windows using NMAKE.
254f84222b3d53f99980a63cc768da07b76d19211166ce20eac80dcbe867dbb6 sqlite3.c • 6c6ed9bc6793e48060a3815dd553ed13bf9568facf74ff5de2883d0737543091 sqlite3.h
drh ok  *
 
30.   Verify that databases (including FTS3 and RTREE databases), rollback journals, and WAL files created by the release candidate are readable and writeable by historical versions of SQLite (as long as no unsupported features are used) and vice versa.
Tested with backcompat.test against 3.8.2 and 3.7.3. • SQLite 2017-03-25 19:16:41 8469fc0d48d6af0accef9b8a84e08ad2ca32351907510d177b4ca4815c1ea7cb • 0 errors out of 265 tests on darkstar Linux 64-bit little-endian
dan ok  *
 
31.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
c.   Windows (MinGW) drh ok  *
d.   Windows (MSVC) drh ok  *
e.   A database containing WITHOUT ROWID tables. drh ok  *
 
32.   Build and verify sqldiff drh ok  *
 
a.   Linux drh ok  *
b.   Mac drh ok  *
c.   Windows drh ok  *
 
33.   Run the rbu/run_rbu_tests.tcl script in the test-dbs project on Linux. dan ok  *
 
34.   Verify that the "sqlite3-all.c" build target works and that the resulting "sqlite3-all.c" file and its include files ("sqlite3-?.c") compile and run the same as the single-file amalgamation.
[make sqlite3-all.c && gcc -I. sqlite3-all.c shell.c -ldl -lpthread -o shell] works on Linux with either main.mk or the non-amalgamation configure script.
dan ok  *