SQLite 3.15.0 Status Board

History    Checklist    Baseline

 
1.   Test procedure review. drh ok  *
 
a.   Review the "test/releasetest.tcl" script for completeness and accuracy. drh ok  *
b.   Review "multitest.tcl" and "alignment*.rc" scripts in TH3 for completeness. drh ok  *
c.   Review this checklist for completeness and accuracy. drh ok  *
 
2.   Source code change review. fossil diff --from release --to trunk drh ok  *
 
a.   No stray changes. drh ok  *
b.   All significant changes are mentioned in the "changes" log of the documentation. drh ok  *
c.   New code complies with style guidelines. drh ok  *
d.   New code complies with design rules. drh ok  *
e.   Comments have been updated to reflect code changes. drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use. drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h). drh ok  *
k.   Significant compile-time options are recognized by the sqlite3_compileoption_used() interface. drh ok  *
 
3.   Documentation review: drh ok  *
 
a.   Latest release on the "index.html" page. drh ok  *
b.   Release announcement in "changes.in". drh ok  *
c.   Release accouncement in "news.html". drh ok  *
d.   No unresolved hyperlinks in the documentation build. drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code. drh ok  *
f.   The compile-time options are all up-to-date (compile.html). drh ok  *
g.   Database footprint is up-to-date (features.html). drh ok  *
h.   Test metrics are up-to-date (testing.html). drh ok  *
i.   Check new/changed documentation for spelling and gramatical mistakes. drh ok  *
 
4.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok+  *
b.   Mac OS X GCC -Wall -Wextra (the "tool/warnings.sh" script)
No warnings. "gcc version 4.2.1".
dan ok  *
c.   OpenBSD GCC using options -Wall
[gcc -c -Wall sqlite3.c] builds without warnings. There are some warnings for [gcc -O3 -Wall sqlite3.c]. None concerning. • No warnings with -O2. Call this a compiler bug.
drh ok  *
d.   Windows (32-bit) MSVC using option /W4 (nmake /f makefile.msc USE_FULLWARN=1) drh ok  *
e.   Windows (64-bit) MSVC using option /W4 (nmake /f makefile.msc USE_FULLWARN=1) drh ok  *
 
5.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a
Building with -DSQLITE_ENABLE_LOCKING_STYLE fails on Linux. Is this expected? • Yes - Linux lacks the system calls necessary for SQLITE_ENABLE_LOCKING_STYLE.
drh ok  *
 
6.   Tests for platform Linux x86_64: drh ok  *
 
a.   tclsh multitest.tcl (take care to update ../th3private)
0 failures on 38 th3makes and 174363577 tests in (05:11:00) 3 cores on bella • SQLite 3.15.0 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3
drh ok  *
b.   Check the "multitest.tcl" log files for undetected errors. drh ok  *
c.   tclsh releasetest.tcl
0 failures out of 9464438 tests in (02:49:36) 2 cores on darkstar • SQLite 3.15.0 2016-10-08 16:10:29 010ec22541a4c37f2d3b01b6a531b4b4264102b7
dan ok  *
d.   Check the "releasetest.tcl" log files for undetected errors. dan ok  *
e.   tclsh multitest.tcl --scenario failures - all failures detected
5 failures on 5 th3makes and 118 tests in (00:01:15) on marcus SQLite 3.15.0 2016-10-08 09:21:34 dbe2997f57f8d8f95e4b767bf73b8df3a5e4f783
dan ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
26 failures out of 786 tests in (00:12:49) on marcus • SQLite 3.15.0 2016-10-08 09:21:34 dbe2997f57f8d8f95e4b767bf73b8df3a5e4f783
dan ok  *
g.   sh tool/symbols.sh - no incorrect exports or dependencies dan ok  *
 
7.   Tests for platform Linux ARM (BeagleBoard) dan ok  *
 
a.   tclsh th3make min.rc
th3: 0 errors in 1627303 tests. 4175.007 seconds on beaglebone 32-bit little-endian SQLite 3.15.0 2016-10-08 11:55:12 cbed3d75cf9a24037263d126727db7dd048a4f97
dan ok  *
b.   Recompile and test Fossil. dan ok  *
 
8.   Tests for platform Mac OS X: drh ok  *
 
a.   tclsh releasetest.tcl
0 failures out of 4656338 tests in (05:39:02) 2 cores SQLite 3.15.0 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3
dan ok  *
b.   tclsh multitest.tcl --omit test-ex (after updating th3private)
Restarted using [443913d5].
dan ok  *
c.   tclsh multitest.tcl --scenario failures - all failures detected
2 failures on 2 th3makes and 42 tests in (00:00:25) • SQLite 3.15.0 2016-10-10 10:06:59 788f86bebe413b5a0ab41ba3d75ba8728827a964
dan ok  *
d.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
20 failures out of 394 tests in (00:19:00) • SQLite 3.15.0 2016-10-10 10:06:59 788f86bebe413b5a0ab41ba3d75ba8728827a964
dan ok  *
 
9.   Tests for platform Mac OS X 10.2 (PPC): drh ok  *
 
a.   tclsh th3make quick.rc
th3: 0 errors in 1624228 tests. 4974.004 seconds on crabapple 32-bit big-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
b.   Recompile and test Fossil. drh ok  *
c.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
10.   Tests for platform Windows: drh ok  *
 
a.   tclsh multitest.tcl (after updating th3private)
0 failures on 32 th3makes and 130998435 tests in (13:16:20) 2 cores on poulenc • SQLite 3.15.0 2016-10-08 16:10:29 010ec22541a4c37f2d3b01b6a531b4b4264102b7
drh ok  *
b.   tclsh multitest.tcl --quick on x86
0 failures on 11 th3makes and 14174385 tests in (01:47:14) on poulenc • SQLite 3.15.0 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3
drh ok  *
c.   tclsh multitest.tcl --scenario failures - verify failures detected drh ok  *
d.   tclsh test/releasetest.tcl --msvc
0 failures out of 1909195 tests in (02:52:43) on poulenc • SQLite 3.15.0 2016-10-08 16:10:29 010ec22541a4c37f2d3b01b6a531b4b4264102b7 • 0 failures out of 1909290 tests in (02:58:37) on poulenc • SQLite 3.15.0 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3
drh ok+  *
e.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
f.   nmake /f makefile.msc MEMDEBUG=1 DEBUG=3 fulltestonly
SQLite 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3 • 0 errors out of 1200650 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
g.   nmake /f makefile.msc WIN32HEAP=1 DEBUG=4 fulltestonly
SQLite 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3 • 0 errors out of 1202494 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
h.   sh tool/symbols-mingw.sh - no incorrect exports or dependencies drh ok  *
i.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
 
11.   Tests for WinRT mistachkin ok  *
 
a.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (x86)
0 errors in 1623599 tests. 783.686 seconds on 32-bit little-endian, SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
mistachkin ok  *
b.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (ARM)
0 errors in 1623598 tests. 5093.667 seconds on 32-bit little-endian, SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
mistachkin ok  *
c.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.1
0 errors in 1623627 tests. 5937.718 seconds on 32-bit little-endian, SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
mistachkin ok  *
d.   th3make msvc.rc wp80.rc min.rc - Windows Phone 8.0
0 errors in 1622325 tests. 1112.045 seconds on 32-bit little-endian, SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
mistachkin ok  *
e.   th3make msvc.rc wp81.rc min.rc - Windows Phone 8.1
0 errors in 1622305 tests. 3385.653 seconds on 32-bit little-endian, SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
mistachkin ok  *
f.   th3make msvc.rc wince.rc min.rc - Windows Embedded Compact 2013 - When tests are going to be run on the [Hyper-V based] emulator, no-mmap.rc must be used as well.
0 errors in 1630024 tests. 3843.392 seconds on 32-bit little-endian, SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
mistachkin ok  *
 
12.   Tests for platform OpenBSD (x86): dan ok  *
 
a.   make test
0 errors out of 139008 tests on obsd.my.domain OpenBSD 32-bit little-endian
dan ok  *
b.   th3make quick.rc
th3: 0 errors in 1728028 tests. 534.674 seconds on obsd.my.domain 32-bit little-endian • th3: SQLite 3.15.0 2016-10-08 16:10:29 010ec22541a4c37f2d3b01b6a531b4b4264102b7
dan ok  *
 
13.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation. drh ok  *
b.   No SQLite compiler warnings. drh ok  *
c.   tclsh run-all.tcl - on unix
0 errors out of 11879758 tests and 1244 invocations in 1020 seconds macpro
drh ok  *
d.   tclsh run-all.tcl - on windows
0 errors out of 11879758 tests and 1244 invocations in 3563 seconds
drh ok  *
 
14.   Build using the configure/make in main source tree. dan ok  *
 
a.   Make sure that autoconf has been run to update the configure script.
configure script now generates 3.15.
dan ok  *
b.   Verify the correct version numbers have been installed.
And installs 3.15 too.
dan ok  *
 
15.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No SQLite compiler warnings. drh ok  *
b.   Used on active websites with no issues. drh ok  *
c.   fossil all dbstat --db-check drh ok  *
 
16.   Rebuild and use Firefox with the latest SQLite drh ok  *
 
17.   CEROD tests in the "test/" subdirectory. drh ok  *
 
18.   SEE tests using TH3: drh ok  *
 
a.   see.c
th3: 0 errors in 8155404 tests. 4673.438 seconds on bella 64-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
b.   see-aes128-ofb.c
th3: 0 errors in 1683413 tests. 896.698 seconds on poulenc 64-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
c.   see-aes256-ofb.c
th3: 0 errors in 1654117 tests. 310.413 seconds on macpro01 64-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
d.   see-rc4.c
th3: 0 errors in 1705161 tests. 1346.913 seconds on win7dev 32-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
e.   see-aes128-ccm.c
th3: 0 errors in 1670014 tests. 266.143 seconds on bella 64-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
f.   see-cccrypt.c
th3: 0 errors in 6405644 tests. 2149.935 seconds on macpro01 64-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
g.   At least one of the above using "test.rc". drh ok  *
h.   At least one of the above using "memdebug.rc". drh ok  *
i.   At least one of the above on Linux. drh ok  *
j.   At least one of the above on Mac OS X. drh ok  *
k.   At least one of the above on Windows. drh ok  *
 
19.   SEE using configure; make test drh ok  *
 
a.   Linux
SQLite 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4 • 0 errors out of 139213 tests on bella Linux 64-bit little-endian
drh ok  *
b.   Mac
SQLite 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3 • 0 errors out of 138597 tests on macpro01 Darwin 64-bit little-endian
drh ok  *
 
20.   ZIPVFS tests: drh ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.)
SQLite 2016-10-08 09:21:34 dbe2997f57f8d8f95e4b767bf73b8df3a5e4f783 • 0 errors out of 75690 tests on marcus Linux 64-bit little-endian
dan ok  *
b.   th3make zipvfs.rc test.rc
0 errors in 9966815 tests. 6569.899 seconds on marcus 64-bit little-endian • SQLite 3.15.0 2016-10-08 16:10:29 010ec22541a4c37f2d3b01b6a531b4b4264102b7
dan ok  *
c.   th3make zipvfs.rc memdebug.rc
On OSX.
dan ok  *
d.   th3make zipvfs.rc quick.rc -valgrind
0 errors in 2768765 tests. 25612.377 seconds on darkstar 64-bit little-endian • SQLite 3.15.0 2016-10-10 10:06:59 788f86bebe413b5a0ab41ba3d75ba8728827a964 • ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
dan ok  *
e.   th3make zipvfs.rc quick.rc
th3: 0 errors in 2827665 tests. 3644.380 seconds on win7dev 32-bit little-endian • th3: SQLite 3.15.0 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3
drh ok  *
f.   At least one of the above on Linux. dan ok  *
g.   At least one of the above on Mac OS X. dan ok  *
h.   At least one of the above on Windows. drh ok  *
i.   th3make zipvfs-cov.rc - verify full coverage
th3: 0 errors in 1242901 tests. 294.626 seconds on bella 64-bit little-endian • th3: SQLite 3.15.0 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3 • Statement coverage: 100.00% hit 1567 missed 0 total 1567 • Branch coverage: 100.00% hit 924 missed 0 total 924
drh ok  *
 
21.   NX-Devkit testing drh ok  *
 
a.   Verify that devkit sources are up-to-date with SQLite and ZIPVFS by running update-src.sh and mkdevkit.tcl in the sbin subdirectory of the devkit source tree. Record nds_sqlite3.[ch] hashes in the comment. drh ok  *
b.   Build using: sh mkdevkit.sh; gcc -o nxc.o -c nx_compress.c drh ok  *
c.   Test using: tclsh th3make test.rc nxc.o -DNDS_ENABLE_COMPRESS
th3: 0 errors in 7065944 tests. 1207.722 seconds on bella 64-bit little-endian • th3: SQLite 3.15.0 2016-10-13 12:56:18 4d66ac98deaa85218be7ff0eb254f78b96d8e8d4
drh ok  *
d.   Verify failure if run without -DNDS_ENABLE_COMPRESS drh ok  *
 
22.   Amalgamation autoconf tarball. drh ok  *
 
a.   Builds. dan ok  *
b.   Works on Linux. dan ok  *
c.   Works on Mac OS X. dan ok  *
d.   Works on Windows MinGW
Requires --diable-shared on the configure
drh ok  *
e.   Works on Windows MSVC drh ok  *
 
23.   TEA archive builds correctly and runs: dan ok  *
 
a.   Linux dan ok  *
b.   Mac OS X dan ok  *
 
24.   Run performance tests comparing the new SQLite release against the previous release and/or a release from one year ago. Document and justify any performance decrease. drh ok  *
 
a.   The Tcl "speed*.test" scripts.
3695427 us down from 3721044 us with 3.14.2
drh ok  *
b.   The TH3 speed tests.
3.571s from 3.607s in 3.14.0
drh ok  *
c.   The "speedtest1" program.
923957789 cycles down from 994310580 with 3.14.0.
drh ok  *
d.   ./speedtest1 --nosync --trace 2 >x.txt; time ./sqlite3 x.db <x.txt >/dev/null
7.783s (poulenc, msvc x64) down from 8.113s for 3.14.2.
drh ok  *
e.   One or more of the above on Linux. drh ok  *
f.   One or more of the above on Windows. drh ok  *
g.   One or more of the above on Mac OS X. drh ok  *
 
25.   Run make wordcount; bash run-wordcount.bash $OPTIONS with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c drh ok  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync drh ok  *
 
26.   Compute the size of a standard build (gcc -Os -c sqlite3.c) and and record the size change in the comments. drh ok  *
 
a.   Versus the previous release.
Grew from 465299 to 472060 due to the addition of row values
drh ok  *
b.   Versus the trunk from 12 months ago.
From 463493 (version 3.9.0) to 472060
drh ok  *
 
27.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   Linux
1e03ca67ea0997e8dc7eac21d286af470f345fae sqlite3.c • bb99350c5fec95e7de0af85569b5cc4cc5c9d4c6 sqlite3.h
drh ok  *
b.   Mac OS X
1e03ca67ea0997e8dc7eac21d286af470f345fae sqlite3.c • bb99350c5fec95e7de0af85569b5cc4cc5c9d4c6 sqlite3.h
drh ok  *
c.   OpenBSD
1e03ca67ea0997e8dc7eac21d286af470f345fae sqlite3.c • bb99350c5fec95e7de0af85569b5cc4cc5c9d4c6 sqlite3.h
drh ok  *
d.   Windows using native MinGW.
1e03ca67ea0997e8dc7eac21d286af470f345fae sqlite3.c • bb99350c5fec95e7de0af85569b5cc4cc5c9d4c6 sqlite3.h
drh ok  *
e.   Windows using NMAKE.
1e03ca67ea0997e8dc7eac21d286af470f345fae sqlite3.c • bb99350c5fec95e7de0af85569b5cc4cc5c9d4c6 sqlite3.h
drh ok  *
 
28.   Verify that databases (including FTS3 and RTREE databases), rollback journals, and WAL files created by the release candidate are readable and writeable by historical versions of SQLite (as long as no unsupported features are used) and vice versa.
Tested against 3.7.3 and 3.8.2 using backcompat.test: SQLite 2016-10-12 18:55:53 443913d582bcd953d85159047541592e2f68ade3 0 errors out of 265 tests on darkstar Linux 64-bit little-endian
dan ok  *
 
29.   Cross-platform tests. Verify the ability of databases, WAL files, and rollback journals to be copied and used between 32-bit and 64-bit systems and between big-endian and little-endian systems. drh ok  *
 
a.   32-big to 64-little, database. drh ok  *
b.   32-big to 64-little, WAL. drh ok  *
c.   32-big to 64-little, rollback. drh ok  *
d.   64-little to 32-big, SEE database. drh ok  *
e.   64-little to 32-big, ZIPVFS database. drh ok  *
f.   64-little to 32-big, CEROD database. drh ok  *
 
30.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
c.   Windows (MinGW) drh ok  *
d.   Windows (MSVC) drh ok  *
e.   A database containing WITHOUT ROWID tables. drh ok  *
 
31.   Build and verify sqldiff drh ok  *
 
a.   Linux drh ok  *
b.   Mac drh ok  *
c.   Windows drh ok  *
 
32.   Verify that the "sqlite3-all.c" build target works and that the resulting "sqlite3-all.c" file and its include files ("sqlite3-?.c") compile and run the same as the single-file amalgamation. drh ok  *
 
33.   System.Data.SQLite builds and checks out using the SQLite core. mistachkin ok  *
 
a.   Using Microsoft .NET Framework on Windows
4d66ac98deaa85218be7ff0eb254f78b96d8e8d4, Debug NetFx46 x86: 0 errors/558 tests, 939 seconds, Release NetFx46 x86: 0 errors/558 tests, 888 seconds, Debug NetFx451 x64: 0 errors/558 tests, 1083 seconds, Release NetFx451 x64: 0 errors/558 tests, 925 seconds, 0 errors/24 tests (Debug NetCF20-ARM/NetCF35-ARM), 0 errors/24 tests (Release NetCF20-ARM/NetCF35-ARM)
mistachkin ok  *
b.   Using Mono on unix
4d66ac98deaa85218be7ff0eb254f78b96d8e8d4, Debug Mono 4.6.1 on Ubuntu 15.10 x86_64: 0 errors/558 tests, 579 seconds, Release Mono 4.6.1 on Ubuntu 15.10 x86_64: 0 errors/558 tests, 531 seconds
mistachkin ok  *