SQLite 3.11.0 Status Board

History    Checklist    Baseline

 
1.   Test procedure review. drh ok  *
 
a.   Review the "test/releasetest.tcl" script for completeness and accuracy. drh ok  *
b.   Review "multitest.tcl" and "alignment*.rc" scripts in TH3 for completeness. drh ok  *
c.   Review this checklist for completeness and accuracy. drh ok  *
 
2.   Source code change review. fossil diff --from release --to trunk drh ok  *
 
a.   No stray changes. drh ok  *
b.   All significant changes are mentioned in the "changes" log of the documentation. drh ok  *
c.   New code complies with style guidelines. drh ok  *
d.   New code complies with design rules. drh ok  *
e.   Comments have been updated to reflect code changes. drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use. drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h). drh ok  *
k.   Significant compile-time options are recognized by the sqlite3_compileoption_used() interface. drh ok  *
 
3.   Documentation review: drh ok  *
 
a.   Latest release on the "index.html" page. drh ok  *
b.   Release announcement in "changes.in". drh ok  *
c.   Release accouncement in "news.html". drh ok  *
d.   No unresolved hyperlinks in the documentation build. drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code. drh ok  *
f.   The compile-time options are all up-to-date (compile.html). drh ok  *
g.   Database footprint is up-to-date (features.html). drh ok  *
h.   Test metrics are up-to-date (testing.html). drh ok  *
i.   Check new/changed documentation for spelling and gramatical mistakes. drh ok  *
 
4.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
b.   Mac OS X GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
c.   OpenBSD GCC using options -Wall drh ok  *
d.   Windows (32-bit) MSVC using option /W4 drh ok  *
e.   Windows (64-bit) MSVC using option /W4 drh ok  *
 
5.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a
OK. 7m41.659s on bella.
drh ok  *
 
6.   Tests for platform Linux x86_64: drh ok  *
 
a.   tclsh multitest.tcl
t04: cov.rc -DSQLITE_ENABLE_STAT4 -D_HAVE_SQLITE_CONFIG_H...... FAIL (00:04:41) • Coverage misses 1 out of 22897 branches • 1 failures on 35 th3makes and 168219741 tests in (04:54:23) 3 cores on bella • SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76 • But then rerunning the since failed test works. • 0 failures on 1 th3makes and 1612514 tests in (00:04:13) on bella • SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76
drh ok  *
b.   Check the "multitest.tcl" log files for undetected errors. drh ok  *
c.   tclsh releasetest.tcl
0 failures out of 9047555 tests in (01:10:16) 4 cores on bella • SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
drh ok++  *
d.   Check the "releasetest.tcl" log files for undetected errors. drh ok  *
e.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
g.   sh tool/symbols.sh - no incorrect exports or dependencies drh ok  *
 
7.   Tests for platform Linux ARM (BeagleBoard) dan ok  *
 
a.   tclsh th3make min.rc
th3: 3 errors in 1540220 tests. 4274.420 seconds on beaglebone 32-bit little-endian th3: SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
dan ok  *
b.   Recompile and test Fossil. dan ok  *
 
8.   Tests for platform Mac OS X (x86): drh ok  *
 
a.   tclsh releasetest.tcl dan ok  *
b.   tclsh multitest.tcl --omit test-ex dan ok  *
c.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
d.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
ASAN tests do not hit any errors.
drh ok  *
 
9.   Tests for platform Mac OS X 10.2 (PPC): drh ok  *
 
a.   tclsh th3make quick.rc drh ok  *
b.   Recompile and test Fossil. drh ok  *
c.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
10.   Tests for platform Windows 7.0/8.x (x86): drh ok  *
 
a.   tclsh multitest.tcl --omit test-ex
0 failures on 29 th3makes and 125648307 tests in (07:58:51) 4 cores on poulenc • SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76 • Includes test-ex
drh ok  *
b.   tclsh multitest.tcl --scenario failures - verify failures detected drh ok  *
c.   tclsh test/releasetest.tcl
0 failures on 29 th3makes and 125648307 tests in (07:58:51) 4 cores on poulenc • SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac • Includes test-ex
drh ok++  *
d.   tclsh test/releasetest.tcl --msvc
0 failures out of 1610395 tests in (03:13:26) 2 cores on win7dev • SQLite 3.11.0 2016-02-12 17:30:39 d9c985878cf7c74a79ccdb9cf94a2d48fdde3e6f
drh ok  *
e.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
f.   nmake /f makefile.msc MEMDEBUG=1 DEBUG=3 fulltestonly
SQLite 2016-02-12 17:56:27 130580207ab5cee762b2893808acef7c8afad027 • 0 errors out of 1192511 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
g.   nmake /f makefile.msc WIN32HEAP=1 DEBUG=4 fulltestonly
SQLite 2016-02-12 17:56:27 130580207ab5cee762b2893808acef7c8afad027 • 0 errors out of 1194295 tests on poulenc Windows NT 64-bit little-endian
drh ok  *
h.   sh tool/symbols-mingw.sh - no incorrect exports or dependencies drh ok  *
i.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
 
11.   Tests for platform Windows (other): mistachkin ok  *
 
a.   th3make msvc.rc test.rc - Windows 7.0/8.x (x64)
0 errors in 6066846 tests. 3217.380 seconds on magnesium 64-bit little-endian, SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76
mistachkin ok  *
b.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (x86)
0 errors in 1629632 tests. 6249.787 seconds on magnesium 32-bit little-endian, SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76
mistachkin ok  *
c.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (ARM)
0 errors in 1541205 tests. 4612.919 seconds on 32-bit little-endian, SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
mistachkin ok  *
d.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.1
0 errors in 1541207 tests. 5088.091 seconds on 32-bit little-endian, SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
mistachkin ok  *
e.   th3make msvc.rc wp80.rc min.rc - Windows Phone 8.0
4 errors in 1553882 tests. 850.738 seconds on 32-bit little-endian, SQLite 3.11.0 2016-02-13 23:43:46 6a9c4a3ebfb7cc0738ef6634440ccab44a21ff28 (all errors are harmless log message differences)
mistachkin ok  *
f.   th3make msvc.rc wp81.rc min.rc - Windows Phone 8.1
0 errors in 1540206 tests. 2644.831 seconds on 32-bit little-endian, SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
mistachkin ok  *
g.   th3make msvc.rc wince.rc min.rc - Windows Embedded Compact 2013 - When tests are going to be run on the [Hyper-V based] emulator, no-mmap.rc must be used as well.
0 errors in 1547669 tests. 3673.667 seconds on 32-bit little-endian, SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
mistachkin ok  *
 
12.   Tests for platform OpenBSD (x86): drh ok  *
 
a.   make test drh ok  *
b.   th3make quick.rc
th3: 0 errors in 1631547 tests. 3627.130 seconds on wolfman.devio.us 32-bit little-endian • th3: SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76
drh ok  *
 
13.   Tests for platform Solaris (SPARC): drh ok  *
 
a.   make test
SQLite 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76 • 0 errors out of 134296 tests on sparc.mikeowens.ws SunOS 32-bit big-endian
drh ok  *
b.   tclsh multitest.tcl --quick -q
0 failures on 9 th3makes and 11915130 tests in (04:23:50) on sparc.mikeowens.ws • SQLite 3.11.0 2016-02-12 18:48:09 a049fbbde5da2e43d41aa8c2b41f9eb21507ac76
drh ok  *
c.   Recompile and test Fossil. drh ok  *
d.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
14.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation. drh ok  *
b.   No SQLite compiler warnings. drh ok  *
c.   tclsh run-all.tcl - on unix
0 errors out of 11879758 tests and 1244 invocations in 820 seconds
drh ok  *
d.   tclsh run-all.tcl - on windows
0 errors out of 11879758 tests and 1244 invocations in 3682 seconds
drh ok  *
 
15.   Build using the configure/make in main source tree. drh ok  *
 
a.   Make sure that autoconf has been run to update the configure script. drh ok  *
b.   Verify the correct version numbers have been installed. drh ok  *
 
16.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No SQLite compiler warnings. drh ok  *
b.   Used on active websites with no issues. drh ok  *
 
17.   CEROD tests in the "test/" subdirectory. drh ok  *
 
18.   SEE tests using TH3: drh ok  *
 
a.   see.c
th3: 0 errors in 1586887 tests. 610.994 seconds on macpro 64-bit little-endian • th3: SQLite 3.11.0 2016-02-15 15:06:47 f694e60a79024967a3f4574e3928f0b28589d381 • quick.rc
drh ok  *
b.   see-aes128-ofb.c
th3: 0 errors in 7922710 tests. 2288.757 seconds on bella 64-bit little-endian • th3: SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67 • test.rc
drh ok  *
c.   see-aes256-ofb.c
th3: 0 errors in 1613315 tests. 598.455 seconds on poulenc 64-bit little-endian • th3: SQLite 3.11.0 2016-02-15 15:06:47 f694e60a79024967a3f4574e3928f0b28589d381
drh ok  *
d.   see-rc4.c
th3: 0 errors in 7922355 tests. 4317.438 seconds on bella 64-bit little-endian • th3: SQLite 3.11.0 2016-02-15 15:06:47 f694e60a79024967a3f4574e3928f0b28589d381
drh ok  *
e.   see-aes128-ccm.c
th3: 0 errors in 1607764 tests. 1219.825 seconds on win7dev 32-bit little-endian • th3: SQLite 3.11.0 2016-02-15 15:06:47 f694e60a79024967a3f4574e3928f0b28589d381
drh ok  *
f.   At least one of the above using "test.rc". drh ok  *
g.   At least one of the above using "memdebug.rc". drh ok  *
h.   At least one of the above on Linux. drh ok  *
i.   At least one of the above on Mac OS X. drh ok  *
j.   At least one of the above on Windows. drh ok  *
 
19.   ZIPVFS tests: dan ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.)
SQLite 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67 0 errors out of 72753 tests on darkstar Linux 64-bit little-endian
dan ok  *
b.   th3make zipvfs.rc test.rc
th3: 0 errors in 9559139 tests. 3015.381 seconds on darkstar 64-bit little-endian th3: SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
dan ok  *
c.   th3make zipvfs.rc memdebug.rc
th3: 0 errors in 7858968 tests. 23883.446 seconds on Chanphen-Kruatawongs-iMac.local 64-bit little-endian • th3: SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
dan ok  *
d.   th3make zipvfs.rc quick.rc -valgrind dan ok  *
e.   th3make zipvfs.rc quick.rc
th3: 0 errors in 2501849 tests. 1555.311 seconds on macpro 64-bit little-endian • th3: SQLite 3.11.0 2016-02-15 15:06:47 f694e60a79024967a3f4574e3928f0b28589d381
drh ok  *
f.   At least one of the above on Linux. dan ok  *
g.   At least one of the above on Mac OS X. drh ok  *
h.   At least one of the above on Windows. drh ok  *
 
20.   NX-Devkit compiles and runs tclsh th3make test.rc nx_compress.o without error.
th3: 0 errors in 6963318 tests. 1194.789 seconds on bella 64-bit little-endian • th3: SQLite 3.11.0 2016-02-13 18:54:10 37ec3015ec95035d31e3672f520908a0d36c9d67
drh ok  *
 
21.   Amalgamation autoconf tarball. drh ok  *
 
a.   Builds. drh ok  *
b.   Works on Linux. drh ok  *
c.   Works on Mac OS X. drh ok  *
d.   Works on Windows MinGW drh ok  *
e.   Works on Windows MSVC drh ok  *
 
22.   TEA archive builds correctly and runs: drh ok  *
 
a.   Linux drh ok+  *
b.   Mac OS X drh ok  *
 
23.   Run performance tests comparing the new SQLite release against the previous release and/or a release from one year ago. Document and justify any performance decrease. drh ok  *
 
a.   The Tcl "speed*.test" scripts.
5.192 on 3.10.0 vs. 5.029 on 3.11.0beta. Bella.
drh ok  *
b.   The TH3 speed tests.
3.10.0 in 4.169s vs. 4.122s on 3.11.0. MacPro.
drh ok  *
c.   The "speedtest1" program.
3.10.0 is 1.143G cycles on cachegrind, vs 1.088G cycles for 3.11.0 beta, a 4.8% improvement.
drh ok  *
d.   ./speedtest1 --nosync --size 25 --trace 2>x.txt
time ./sqlite3 x.db <x.txt >/dev/null
No measurable difference on Win7. Timings for the same executable vary by 10% or more between successive runs.
drh ok  *
e.   One or more of the above on Linux. drh ok  *
f.   One or more of the above on Windows. drh ok  *
g.   One or more of the above on Mac OS X. drh ok  *
 
24.   Run make wordcount; bash run-wordcount.bash $OPTIONS with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c drh ok  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync drh ok  *
 
25.   Compute the size of a standard build (gcc -Os -c sqlite3.c) and and record the size change in the comments. drh ok  *
 
a.   Versus the previous release.
(v3.10.2) 464089 version (v3.11.0) 464134
drh ok  *
b.   Versus the trunk from 12 months ago.
(trunk:2015-02-12) 455421 versus (v3.11.0) 454134
drh ok  *
 
26.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   Linux
8d56d871dcff40a7cb146b83e2213ef0c90ae407 sqlite3.c • 57512fad7e9867894a4d870493051dbd023c7eca sqlite3.h
drh ok+  *
b.   Mac OS X
8d56d871dcff40a7cb146b83e2213ef0c90ae407 sqlite3.c • 57512fad7e9867894a4d870493051dbd023c7eca sqlite3.h
drh ok+  *
c.   OpenBSD
8d56d871dcff40a7cb146b83e2213ef0c90ae407 sqlite3.c • 57512fad7e9867894a4d870493051dbd023c7eca sqlite3.h
drh ok  *
d.   Windows using native MinGW.
8d56d871dcff40a7cb146b83e2213ef0c90ae407 sqlite3.c • 57512fad7e9867894a4d870493051dbd023c7eca sqlite3.h
drh ok  *
e.   Windows using NMAKE.
8d56d871dcff40a7cb146b83e2213ef0c90ae407 sqlite3.c • 57512fad7e9867894a4d870493051dbd023c7eca sqlite3.h
drh ok  *
f.   Solaris (SPARC) using autoconf.
8d56d871dcff40a7cb146b83e2213ef0c90ae407 sqlite3.c • 57512fad7e9867894a4d870493051dbd023c7eca sqlite3.h
drh ok  *
 
27.   Verify that databases (including FTS3 and RTREE databases), rollback journals, and WAL files created by the release candidate are readable and writeable by historical versions of SQLite (as long as no unsupported features are used) and vice versa.
Tested using backcompat.test against 3.6.4, 3.7.1 and 3.8.3. 0 errors out of 395 tests on darkstar Linux 64-bit little-endian
dan ok  *
 
28.   Cross-platform tests. Verify the ability of databases, WAL files, and rollback journals to be copied and used between 32-bit and 64-bit systems and between big-endian and little-endian systems. drh ok  *
 
a.   32-big to 64-little, database. drh ok  *
b.   32-big to 64-little, WAL. drh ok  *
c.   32-big to 64-little, rollback. drh ok  *
d.   64-little to 32-big, SEE database. drh ok  *
e.   64-little to 32-big, ZIPVFS database. drh ok  *
f.   64-little to 32-big, CEROD database. drh ok  *
 
29.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
c.   Solaris (SPARC) drh ok  *
d.   Windows (MinGW) drh ok  *
e.   Windows (MSVC) drh ok  *
f.   A database containing WITHOUT ROWID tables. drh ok  *
 
30.   Tests for the sessions branch with macros SQLITE_ENABLE_SESSION and SQLITE_ENABLE_PREUPDATE_HOOK. drh ok  *
 
a.   tclsh th3make session.rc test.rc (on Windows)
th3: 0 errors in 6063501 tests. 2292.668 seconds on poulenc 64-bit little-endian • th3: SQLite 3.11.0 2016-02-13 15:08:25 f6d1cf79437f892220e93635c377513160ee7fd7
drh ok  *
b.   tclsh th3make session.rc test.rc (on Unix)
th3: 0 errors in 7959312 tests. 1834.583 seconds on bella 64-bit little-endian • th3: SQLite 3.11.0 2016-02-13 15:08:25 f6d1cf79437f892220e93635c377513160ee7fd7
drh ok+  *
c.   tclsh th3make session.rc quick.rc -valgrind
th3: 0 errors in 1622775 tests. 7147.631 seconds on bella 64-bit little-endian • th3: SQLite 3.11.0 2016-02-13 15:08:25 f6d1cf79437f892220e93635c377513160ee7fd7 • ERROR SUMMARY: 0 errors from 0 contexts (suppressed: 0 from 0)
drh ok  *
d.   make test (on Windows)
SQLite 2016-02-13 15:08:25 f6d1cf79437f892220e93635c377513160ee7fd7 • 0 errors out of 106688 tests on win7dev Windows NT 32-bit little-endian
drh ok  *
e.   make test (on Unix)
SQLite 2016-02-13 15:08:25 f6d1cf79437f892220e93635c377513160ee7fd7 • 0 errors out of 148779 tests on bella Linux 64-bit little-endian
drh ok+  *
f.   testfixture test/session.test
SQLite 2016-02-13 15:08:25 f6d1cf79437f892220e93635c377513160ee7fd7 • 0 errors out of 49573 tests on bella Linux 64-bit little-endian
drh ok  *
 
31.   Verify that the "sqlite3-all.c" build target works and that the resulting "sqlite3-all.c" file and its include files ("sqlite3-?.c") compile and run the same as the single-file amalgamation. drh ok  *
 
32.   System.Data.SQLite builds and checks out using the SQLite core.
37ec3015ec95035d31e3672f520908a0d36c9d67, Debug NetFx46 x86: 0 errors/499 tests, 1151 seconds, Release NetFx46 x86: 0 errors/499 tests, 1114 seconds, Debug NetFx451 x64: 0 errors/499 tests, 1088 seconds, Release NetFx451 x64: 0 errors/499 tests, 1070 seconds, 0 errors/24 tests (Debug NetCF20-ARM/NetCF35-ARM), 0 errors/24 tests (Release NetCF20-ARM/NetCF35-ARM)
mistachkin ok  *