SQLite 3.13.0 Status Board

History    Checklist    Baseline

 
1.   Test procedure review. drh ok  *
 
a.   Review the "test/releasetest.tcl" script for completeness and accuracy. drh ok  *
b.   Review "multitest.tcl" and "alignment*.rc" scripts in TH3 for completeness. drh ok  *
c.   Review this checklist for completeness and accuracy. drh ok  *
 
2.   Source code change review. fossil diff --from release --to trunk drh ok  *
 
a.   No stray changes. drh ok  *
b.   All significant changes are mentioned in the "changes" log of the documentation. drh ok  *
c.   New code complies with style guidelines. drh ok  *
d.   New code complies with design rules. drh ok  *
e.   Comments have been updated to reflect code changes. drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use. drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h). drh ok  *
k.   Significant compile-time options are recognized by the sqlite3_compileoption_used() interface. drh ok  *
 
3.   Documentation review: drh ok  *
 
a.   Latest release on the "index.html" page. drh ok  *
b.   Release announcement in "changes.in". drh ok  *
c.   Release accouncement in "news.html". drh ok  *
d.   No unresolved hyperlinks in the documentation build. drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code. drh ok  *
f.   The compile-time options are all up-to-date (compile.html). drh ok  *
g.   Database footprint is up-to-date (features.html). drh ok  *
h.   Test metrics are up-to-date (testing.html). drh ok  *
i.   Check new/changed documentation for spelling and gramatical mistakes. drh ok  *
 
4.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
b.   Mac OS X GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
c.   OpenBSD GCC using options -Wall drh ok  *
d.   Windows (32-bit) MSVC using option /W4 drh ok  *
e.   Windows (64-bit) MSVC using option /W4 drh ok  *
 
5.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a drh ok  *
 
6.   Tests for platform Linux x86_64: dan ok  *
 
a.   tclsh multitest.tcl (take care to update ../th3private)
0 failures on 35 th3makes and 170649867 tests in (04:20:58) 4 cores on bella • SQLite 3.13.0 2016-05-11 20:03:23 3bfd2621d13b4f842f3af6d35519653f4eb8cad7 • 0 failures on 35 th3makes and 170694991 tests in (04:22:51) 4 cores on bella • SQLite 3.13.0 2016-05-13 17:22:33 b369980f0c4550a9034833caa2c7c85d6030f5ff
drh ok+  *
b.   Check the "multitest.tcl" log files for undetected errors. drh ok  *
c.   tclsh releasetest.tcl
SQLite 3.13.0 2016-05-09 19:03:42 14e53d0e2f62d82ae1d64a72fd9711548e3bf5ea
dan ok  *
d.   Check the "releasetest.tcl" log files for undetected errors. drh ok  *
e.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
g.   sh tool/symbols.sh - no incorrect exports or dependencies drh ok  *
 
7.   Tests for platform Linux ARM (BeagleBoard) drh ok  *
 
a.   tclsh th3make min.rc dan ok  *
b.   Recompile and test Fossil. dan ok  *
 
8.   Tests for platform Mac OS X (x86): drh ok  *
 
a.   tclsh releasetest.tcl dan ok  *
b.   tclsh multitest.tcl --omit test-ex (after updating th3private) dan ok  *
c.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
d.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
16 failures out of 202 tests in (00:24:47) on macpro • SQLite 3.13.0 2016-05-16 14:35:15 995c084bde44e678facc5f5d95a2335ce61e57b0
drh ok  *
 
9.   Tests for platform Mac OS X 10.2 (PPC): drh ok  *
 
a.   tclsh th3make quick.rc
th3: 0 errors in 1553062 tests. 4628.289 seconds on crabapple.local. 32-bit big-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
b.   Recompile and test Fossil. drh ok  *
c.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
10.   Tests for platform Windows 7.0/8.x (x86): drh ok  *
 
a.   tclsh multitest.tcl (after updating th3private)
0 failures on 29 th3makes and 127631115 tests in (11:40:19) 3 cores on poulenc • SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
b.   tclsh multitest.tcl --scenario failures - verify failures detected drh ok  *
c.   tclsh test/releasetest.tcl
0 failures out of 1255637 tests in (02:55:23) on poulenc • SQLite 3.13.0 2016-05-16 11:55:09 386bcbba8ed733ee84958e6bd6e9d33c150e6c18
drh ok  *
d.   tclsh test/releasetest.tcl --msvc
0 failures out of 1623840 tests in (02:49:48) on poulenc • SQLite 3.13.0 2016-05-09 19:03:42 14e53d0e2f62d82ae1d64a72fd9711548e3bf5ea
drh ok  *
e.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
f.   nmake /f makefile.msc MEMDEBUG=1 DEBUG=3 fulltestonly
0 errors out of 1202299 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
g.   nmake /f makefile.msc WIN32HEAP=1 DEBUG=4 fulltestonly
SQLite 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2 • 1 errors out of 1201589 tests on win7dev Windows NT 32-bit little-endian • !Failures on these tests: selectG-100 • ! selectG-100 expected: [100000 5000050000 50000.5 1] • ! selectG-100 got: [100000 5000050000 50000.5 0] • That one error is a timing test that is too slow. But win7dev is a slow machine that WIN32HEAP and DEBUG=4 make even slower. Perviously, this test has run on poulenc, which is much faster. Deemed pass.
drh ok  *
h.   sh tool/symbols-mingw.sh - no incorrect exports or dependencies drh ok  *
i.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
 
11.   Tests for platform Windows (other): mistachkin ok  *
 
a.   th3make msvc.rc test.rc - Windows 7.0/8.x (x64)
th3: 0 errors in 6189135 tests. 4638.187 seconds on win7dev 64-bit little-endian • th3: SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
drh ok+  *
b.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (x86)
0 errors in 1552480 tests. 784.138 seconds on 32-bit little-endian, SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
mistachkin ok  *
c.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (ARM)
0 errors in 1552468 tests. 4879.146 seconds on 32-bit little-endian, SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
mistachkin ok  *
d.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.1
0 errors in 1553027 tests. 4673.381 seconds on 32-bit little-endian, SQLite 3.13.0 2016-05-11 18:20:10 4533ed352db2e05c7600dbfe5e02cf7f0d6f225a
mistachkin ok  *
e.   th3make msvc.rc wp80.rc min.rc - Windows Phone 8.0
0 errors in 1551276 tests. 981.083 seconds on 32-bit little-endian, SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
mistachkin ok  *
f.   th3make msvc.rc wp81.rc min.rc - Windows Phone 8.1
0 errors in 1551843 tests. 3376.742 seconds on 32-bit little-endian, SQLite 3.13.0 2016-05-11 18:20:10 4533ed352db2e05c7600dbfe5e02cf7f0d6f225a
mistachkin ok  *
g.   th3make msvc.rc wince.rc min.rc - Windows Embedded Compact 2013 - When tests are going to be run on the [Hyper-V based] emulator, no-mmap.rc must be used as well.
0 errors in 1558652 tests. 3771.469 seconds on 32-bit little-endian, SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
mistachkin ok  *
 
12.   Tests for platform OpenBSD (x86): drh ok  *
 
a.   make test drh ok  *
b.   th3make quick.rc
th3: 0 errors in 1645159 tests. 3321.551 seconds on wolfman.devio.us 32-bit little-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
 
13.   Tests for platform Solaris (SPARC): drh ok  *
 
a.   configure; OPTS=-lrt make test
0 errors out of 134655 tests on sparc.mikeowens.ws SunOS 32-bit big-endian
drh ok  *
b.   tclsh multitest.tcl --quick -q (after updating th3private)
0 failures on 9 th3makes and 12025507 tests in (04:31:38) on sparc.mikeowens.ws • SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
c.   Recompile and test Fossil. drh ok  *
d.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
14.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation. drh ok  *
b.   No SQLite compiler warnings. drh ok  *
c.   tclsh run-all.tcl - on unix
0 errors out of 11879758 tests and 1244 invocations in 813 seconds
drh ok  *
d.   tclsh run-all.tcl - on windows
0 errors out of 11879758 tests and 1244 invocations in 3592 seconds
drh ok  *
 
15.   Build using the configure/make in main source tree. drh ok  *
 
a.   Make sure that autoconf has been run to update the configure script. drh ok  *
b.   Verify the correct version numbers have been installed. drh ok  *
 
16.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No SQLite compiler warnings. drh ok  *
b.   Used on active websites with no issues. drh ok  *
c.   fossil all dbstat --db-check drh ok  *
 
17.   Rebuild and use Firefox with the latest SQLite drh ok  *
 
18.   CEROD tests in the "test/" subdirectory. drh ok  *
 
19.   SEE tests using TH3: drh ok  *
 
a.   see.c
th3: 0 errors in 8047955 tests. 4920.994 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-10 20:03:08 20cf8811caf705b482be100baecb3ef72aee2d5a • memdebug.rc • th3: 0 errors in 8063768 tests. 2190.624 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-13 17:22:33 b369980f0c4550a9034833caa2c7c85d6030f5ff • test.rc
drh ok+  *
b.   see-aes128-ofb.c
th3: 0 errors in 1621543 tests. 1199.720 seconds on win7dev 32-bit little-endian • th3: SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
drh ok  *
c.   see-aes256-ofb.c
th3: 0 errors in 1609521 tests. 238.858 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-10 20:03:08 20cf8811caf705b482be100baecb3ef72aee2d5a
drh ok  *
d.   see-rc4.c
th3: 0 errors in 1609521 tests. 238.858 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-10 20:03:08 20cf8811caf705b482be100baecb3ef72aee2d5a
drh ok  *
e.   see-aes128-ccm.c
th3: 0 errors in 7826574 tests. 1122.015 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-13 17:22:33 b369980f0c4550a9034833caa2c7c85d6030f5ff • th3: 0 errors in 8064199 tests. 2610.834 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-13 17:22:33 b369980f0c4550a9034833caa2c7c85d6030f5ff
drh ok+  *
f.   see-cccrypt.c
th3: 0 errors in 6322008 tests. 2245.503 seconds on macpro 64-bit little-endian • th3: SQLite 3.13.0 2016-05-10 20:16:43 223640243efc52c14bb2bb540833a2a624eaa41a
drh ok  *
g.   At least one of the above using "test.rc". drh ok  *
h.   At least one of the above using "memdebug.rc". drh ok  *
i.   At least one of the above on Linux. drh ok  *
j.   At least one of the above on Mac OS X. drh ok  *
k.   At least one of the above on Windows. drh ok  *
 
20.   SEE using configure; make test drh ok  *
 
a.   Linux
0 errors out of 134401 tests on bella Linux 64-bit little-endian
drh ok  *
b.   Mac
0 errors out of 133794 tests on macpro Darwin 64-bit little-endian
drh ok  *
 
21.   ZIPVFS tests: drh ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.)
SQLite 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2 • 0 errors out of 63942 tests on bella Linux 64-bit little-endian
drh ok  *
b.   th3make zipvfs.rc test.rc
th3: 0 errors in 8105737 tests. 3583.322 seconds on macpro 64-bit little-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
c.   th3make zipvfs.rc memdebug.rc
th3: 0 errors in 9837632 tests. 8061.141 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
d.   th3make zipvfs.rc quick.rc -valgrind
th3: 0 errors in 2683727 tests. 21052.180 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
e.   th3make zipvfs.rc quick.rc
th3: 0 errors in 2710015 tests. 3976.970 seconds on win7dev 32-bit little-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2
drh ok  *
f.   At least one of the above on Linux. drh ok  *
g.   At least one of the above on Mac OS X. drh ok  *
h.   At least one of the above on Windows. drh ok  *
i.   th3make zipvfs-cov.rc - verify full coverage
th3: 0 errors in 1217972 tests. 332.600 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2 • Statement coverage: 100.00% hit 1561 missed 0 total 1561 • Branch coverage: 100.00% hit 918 missed 0 total 918
drh ok  *
 
22.   NX-Devkit testing drh ok  *
 
a.   Verify that devkit sources are up-to-date with SQLite and ZIPVFS by running update-src.sh and mkdevkit.tcl in the sbin subdirectory of the devkit source tree. Record nds_sqlite3.[ch] hashes in the comment. drh ok  *
b.   Build using: sh mkdevkit.sh; gcc -o nxc.o -c nx_compress.c drh ok  *
c.   Test using: tclsh th3make test.rc nxc.o -DNDS_ENABLE_COMPRESS
th3: 0 errors in 6974221 tests. 1206.201 seconds on bella 64-bit little-endian • th3: SQLite 3.13.0 2016-05-16 11:55:09 386bcbba8ed733ee84958e6bd6e9d33c150e6c18
drh ok  *
d.   Verify failure if run without -DNDS_ENABLE_COMPRESS drh ok  *
 
23.   Amalgamation autoconf tarball. drh ok  *
 
a.   Builds. drh ok  *
b.   Works on Linux. drh ok  *
c.   Works on Mac OS X. drh ok  *
d.   Works on Windows MinGW drh ok  *
e.   Works on Windows MSVC drh ok  *
 
24.   TEA archive builds correctly and runs: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
 
25.   Run performance tests comparing the new SQLite release against the previous release and/or a release from one year ago. Document and justify any performance decrease. drh ok  *
 
a.   The Tcl "speed*.test" scripts.
On Win7 • 3.12.0: 23.564720 sec • 3.13.0: 22.984078 sec
drh ok  *
b.   The TH3 speed tests.
On MacPro • 3.12.0: 3.223 sec • 3.13.0: 3.177 sec
drh ok  *
c.   The "speedtest1" program.
Ubuntu • 3.12.0: 1032.993M cycles • 3.13.0: 1016.890M cycles
drh ok  *
d.   ./speedtest1 --nosync --size 25 --trace 2>x.txt
time ./sqlite3 x.db <x.txt >/dev/null
Ubuntu • 3.12.0: 0.877s • 3.13.0: 0.837s
drh ok  *
e.   One or more of the above on Linux. drh ok  *
f.   One or more of the above on Windows. drh ok  *
g.   One or more of the above on Mac OS X. drh ok  *
 
26.   Run make wordcount; bash run-wordcount.bash $OPTIONS with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c drh ok  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync drh ok  *
 
27.   Compute the size of a standard build (gcc -Os -c sqlite3.c) and and record the size change in the comments. drh ok  *
 
a.   Versus the previous release.
3.12.0: 485831 • 3.13.0: 486049
drh ok  *
b.   Versus the trunk from 12 months ago.
3.8.10: 481961 • 3.13.0: 486049
drh ok  *
 
28.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   Linux
34f2d1424433e7440b255a24dc95d46e50d6f033 sqlite3.c • 6df60269849748d16f0ba9e754c9d791548414e9 sqlite3.h
drh ok+  *
b.   Mac OS X
34f2d1424433e7440b255a24dc95d46e50d6f033 sqlite3.c • 6df60269849748d16f0ba9e754c9d791548414e9 sqlite3.h
drh ok  *
c.   OpenBSD
34f2d1424433e7440b255a24dc95d46e50d6f033 sqlite3.c • 6df60269849748d16f0ba9e754c9d791548414e9 sqlite3.h
drh ok  *
d.   Windows using native MinGW.
34f2d1424433e7440b255a24dc95d46e50d6f033 sqlite3.c • 6df60269849748d16f0ba9e754c9d791548414e9 sqlite3.h
drh ok  *
e.   Windows using NMAKE.
34f2d1424433e7440b255a24dc95d46e50d6f033 sqlite3.c • 6df60269849748d16f0ba9e754c9d791548414e9 sqlite3.h
drh ok  *
f.   Solaris (SPARC) using autoconf.
34f2d1424433e7440b255a24dc95d46e50d6f033 sqlite3.c • 6df60269849748d16f0ba9e754c9d791548414e9 sqlite3.h
drh ok  *
 
29.   Verify that databases (including FTS3 and RTREE databases), rollback journals, and WAL files created by the release candidate are readable and writeable by historical versions of SQLite (as long as no unsupported features are used) and vice versa.
Tested using backcompat.test against 3.6.4 and 3.7.3: • SQLite 2016-05-09 23:11:47 79ecd0ef20c12198e950a730871444a980a0f1b2 • 0 errors out of 261 tests on darkstar Linux 64-bit little-endian
dan ok  *
 
30.   Cross-platform tests. Verify the ability of databases, WAL files, and rollback journals to be copied and used between 32-bit and 64-bit systems and between big-endian and little-endian systems. drh ok  *
 
a.   32-big to 64-little, database. drh ok  *
b.   32-big to 64-little, WAL. drh ok  *
c.   32-big to 64-little, rollback. drh ok  *
d.   64-little to 32-big, SEE database. drh ok  *
e.   64-little to 32-big, ZIPVFS database. drh ok  *
f.   64-little to 32-big, CEROD database. drh ok  *
 
31.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
c.   Solaris (SPARC) drh ok  *
d.   Windows (MinGW) drh ok  *
e.   Windows (MSVC) drh ok  *
f.   A database containing WITHOUT ROWID tables. drh ok  *
 
32.   Build and verify sqldiff drh ok  *
 
a.   Linux drh ok  *
b.   Mac drh ok  *
c.   Windows drh ok  *
 
33.   Verify that the "sqlite3-all.c" build target works and that the resulting "sqlite3-all.c" file and its include files ("sqlite3-?.c") compile and run the same as the single-file amalgamation. drh ok  *
 
34.   System.Data.SQLite builds and checks out using the SQLite core. mistachkin ok  *
 
a.   Using Microsoft .NET Framework on Windows
223640243efc52c14bb2bb540833a2a624eaa41a, Debug NetFx46 x86: 0 errors/503 tests, 1285 seconds, Release NetFx46 x86: 0 errors/503 tests, 1007 seconds, Debug NetFx451 x64: 0 errors/503 tests, 1195 seconds, Release NetFx451 x64: 0 errors/503 tests, 1103 seconds, 0 errors/24 tests (Debug NetCF20-ARM/NetCF35-ARM), 0 errors/24 tests (Release NetCF20-ARM/NetCF35-ARM)
mistachkin ok  *
b.   Using Mono on unix
223640243efc52c14bb2bb540833a2a624eaa41a, Debug Mono 4.2.3 on Ubuntu 15.10 x86_64: 0 errors/503 tests, 548 seconds, Release Mono 4.2.3 on Ubuntu 15.10 x86_64: 0 errors/503 tests, 502 seconds
mistachkin ok  *