SQLite 3.9.0 Status Board

History    Checklist    Baseline

 
1.   Test procedure review. drh ok  *
 
a.   Review the "test/releasetest.tcl" script for completeness and accuracy. drh ok  *
b.   Review "multitest.tcl" and "alignment*.rc" scripts in TH3 for completeness. drh ok  *
c.   Review this checklist for completeness and accuracy. drh ok  *
 
2.   Source code change review. fossil diff --from release --to trunk drh ok  *
 
a.   No stray changes. drh ok  *
b.   All significant changes are mentioned in the "changes" log of the documentation. drh ok  *
c.   New code complies with style guidelines. drh ok  *
d.   New code complies with design rules. drh ok  *
e.   Comments have been updated to reflect code changes. drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use. drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h). drh ok  *
k.   Significant compile-time options are recognized by the sqlite3_compileoption_used() interface. drh ok  *
 
3.   Documentation review: drh ok  *
 
a.   Latest release on the "index.html" page. drh ok  *
b.   Release announcement in "changes.in". drh ok  *
c.   Release accouncement in "news.html". drh ok  *
d.   No unresolved hyperlinks in the documentation build. drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code. drh ok  *
f.   The compile-time options are all up-to-date (compile.html). drh ok  *
g.   Database footprint is up-to-date (features.html). drh ok  *
h.   Test metrics are up-to-date (testing.html). drh ok  *
i.   Check new/changed documentation for spelling and gramatical mistakes. drh ok  *
 
4.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the "tool/warnings.sh" script) drh ok  *
b.   Mac OS X GCC -Wall -Wextra (the "tool/warnings.sh" script)
Errors in "Android Configuration" since that is not compatible with Mac. Ok otherwise.
drh ok  *
c.   OpenBSD GCC using options -Wall drh ok  *
d.   Windows (32-bit) MSVC using option /W4 drh ok  *
e.   Windows (64-bit) MSVC using option /W4 drh ok  *
f.   sh tool/warnings-clang.sh
false positives only.
drh ok  *
 
5.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a drh ok  *
 
6.   Tests for platform Linux x86_64: drh ok  *
 
a.   tclsh multitest.tcl
0 failures on 30 th3make runs and 169067676 tests in (13:06:03) SQLite 3.9.0 2015-10-11 20:39:46 9ab9c8c6d747647f8ade58c2c4812fc69a813368
drh ok+  *
b.   Check the "multitest.tcl" log files for undetected errors. drh ok  *
c.   tclsh releasetest.tcl
0 failures out of 9511636 tests in (06:14:56) SQLite 3.9.0 2015-10-12 04:56:12 6f2858f6817ca70c132f0437ac2f0f74deb273d2
drh ok+++  *
d.   Check the "releasetest.tcl" log files for undetected errors. drh ok  *
e.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
g.   sh tool/symbols.sh - no incorrect exports or dependencies drh ok  *
 
7.   Tests for platform Linux ARM (BeagleBoard) dan ok  *
 
a.   tclsh th3make min.rc
th3: 0 errors out of 1476013 tests in 4489.973 seconds. 32-bit little-endian th3: SQLite 3.9.0 2015-10-12 19:12:29 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
dan ok  *
b.   Recompile and test Fossil. dan ok  *
 
8.   Tests for platform Mac OS X (x86): dan ok  *
 
a.   tclsh releasetest.tcl
0 failures out of 4419868 tests in (06:22:49) SQLite 3.9.0 2015-10-12 19:12:29 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
dan ok  *
b.   tclsh multitest.tcl --omit test-ex dan ok  *
c.   tclsh multitest.tcl --scenario failures - all failures detected drh ok  *
d.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected
Sanitize failures do not occur on the older mac. All actual errors are detected.
drh ok  *
 
9.   Tests for platform Mac OS X 10.2 (PPC): drh ok  *
 
a.   tclsh th3make quick.rc drh ok  *
b.   Recompile and test Fossil. drh ok  *
c.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
10.   Tests for platform Windows 7.0/8.x (x86): drh ok  *
 
a.   th3make msvc.rc test.rc /RTC1
th3: 0 errors out of 5928186 tests in 6193.591 seconds. 32-bit little-endian th3: SQLite 3.9.0 2015-10-12 22:31:02 9ecafc0c94102dca6da192f400278399464086fb
drh ok++  *
b.   tclsh multitest.tcl --omit test-ex
0 failures on 24 th3make runs and 110873922 tests in (11:18:14) SQLite 3.9.0 2015-10-12 19:12:29 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
drh ok  *
c.   tclsh multitest.tcl --scenario failures - verify failures detected drh ok  *
d.   tclsh test/releasetest.tcl
0 failures out of 1792426 tests in (01:56:39) SQLite 3.9.0 2015-10-12 22:31:02 9ecafc0c94102dca6da192f400278399464086fb
drh ok  *
e.   tclsh test/releasetest.tcl --msvc
0 failures out of 1789425 tests in (06:23:12) SQLite 3.9.0 2015-10-09 20:40:44 4a47f0177851721cdf6c981df962b6e8862372e7, win7
drh ok  *
f.   tclsh releasetest.tcl --platform Failure-Detection - all failures detected drh ok  *
g.   nmake /f makefile.msc MEMDEBUG=1 DEBUG=3 fulltestonly
0 errors out of 1189216 tests, win8
drh ok  *
h.   nmake /f makefile.msc WIN32HEAP=1 DEBUG=4 fulltestonly
0 errors out of 1188504 tests, 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8, 32-bit Windows 7 (x86)
mistachkin ok  *
i.   sh tool/symbols-mingw.sh - no incorrect exports or dependencies drh ok  *
j.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
 
11.   Tests for platform Windows (other): mistachkin ok  *
 
a.   th3make msvc.rc test.rc - Windows 7.0/8.x (x64)
0 errors out of 195386 tests, win7
drh ok  *
b.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (x86)
1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8, 0 errors out of 1481048 tests in 711.564 seconds. 32-bit little-endian
mistachkin ok  *
c.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.0 (ARM)
1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8, 0 errors out of 1481033 tests in 4346.758 seconds. 32-bit little-endian
mistachkin ok  *
d.   th3make msvc.rc winrt.rc min.rc - Windows Runtime 8.1
9ecafc0c94102dca6da192f400278399464086fb, 0 errors out of 1481048 tests in 2540.879 seconds. 32-bit little-endian
mistachkin ok  *
e.   th3make msvc.rc wp80.rc min.rc - Windows Phone 8.0
1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8, 0 errors out of 1480088 tests in 945.888 seconds. 32-bit little-endian
mistachkin ok  *
f.   th3make msvc.rc wp81.rc min.rc - Windows Phone 8.1
9ecafc0c94102dca6da192f400278399464086fb, 0 errors out of 1480072 tests in 2858.359 seconds. 32-bit little-endian
mistachkin ok  *
g.   th3make msvc.rc wince.rc min.rc - Windows Embedded Compact 2013 - When tests are going to be run on the [Hyper-V based] emulator, no-mmap.rc must be used as well.
1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8, 0 errors out of 1487436 tests in 3200.000 seconds. 32-bit little-endian
mistachkin ok  *
 
12.   Tests for platform OpenBSD (x86): drh ok  *
 
a.   make test
shell1-5.0 fails on 0xE0. Not going to worry about that corner case. 1 errors out of 139331 tests
drh ok  *
b.   th3make quick.rc
th3: 24 errors out of 1570149 tests in 3028.318 seconds. 32-bit little-endian th3: SQLite 3.9.0 2015-10-10 16:41:28 fab6f09044d033dd09ed8a22e06bc6a7851bbabf. All errors are ctime03.1.39.3 and ctime03.1.39.4 due to building sources separately and are not real errors.
drh ok  *
 
13.   Tests for platform Solaris (SPARC): drh ok  *
 
a.   make test
0 errors out of 139529 tests
drh ok  *
b.   tclsh multitest.tcl --only quick
0 failures on 7 th3make runs and 11436213 tests in (04:04:43) SQLite 3.9.0 2015-10-10 14:41:28 9ddef84d432813f3ece8012047d08441caa3315d
drh ok  *
c.   Recompile and test Fossil. drh ok  *
d.   Verify that Fossil repositories are cross-platform with x86/x64. drh ok  *
 
14.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation. drh ok  *
b.   No SQLite compiler warnings. drh ok  *
c.   tclsh run-all.tcl - on unix
0 errors out of 11879758 tests and 1244 invocations in 903 seconds
drh ok  *
d.   tclsh run-all.tcl - on windows
0 errors out of 11879758 tests and 1244 invocations in 3631 seconds
drh ok  *
 
15.   Build using the configure/make in main source tree. drh ok  *
 
a.   Make sure that autoconf has been run to update the configure script. drh ok  *
b.   Verify the correct version numbers have been installed. drh ok  *
 
16.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No SQLite compiler warnings. drh ok  *
b.   Used on active websites with no issues. drh ok  *
 
17.   CEROD tests in the "test/" subdirectory. drh ok  *
 
18.   SEE tests using TH3: drh ok  *
 
a.   see.c
ubuntu, memdebug.rc
drh ok  *
b.   see-aes128-ofb.c
th3: 0 errors out of 1556787 tests in 550.541 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-10 16:41:28 fab6f09044d033dd09ed8a22e06bc6a7851bbab, win8, msvc, quick.rc
drh ok  *
c.   see-aes256-ofb.c
th3: 0 errors out of 1532863 tests in 772.611 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-10 16:41:28 fab6f09044d033dd09ed8a22e06bc6a7851bbabf, mac, quick.rc
drh ok  *
d.   see-rc4.c
th3: 0 errors out of 7788011 tests in 2472.966 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-12 22:31:02 9ecafc0c94102dca6da192f400278399464086fb, ubuntu, test.rc
drh ok  *
e.   see-aes128-ccm.c
th3: 0 errors out of 1542389 tests in 265.326 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-12 22:31:02 9ecafc0c94102dca6da192f400278399464086fb, ubuntu, quick.rc
drh ok  *
f.   At least one of the above using "test.rc". drh ok  *
g.   At least one of the above using "memdebug.rc".
0 errors out of 11879758 tests and 1244 invocations in 903 seconds, ubuntu, see.c
drh ok  *
h.   At least one of the above on Linux. drh ok  *
i.   At least one of the above on Mac OS X. drh ok  *
j.   At least one of the above on Windows. drh ok  *
 
19.   ZIPVFS tests: dan ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.) dan ok  *
b.   th3make zipvfs.rc test.rc
th3: 0 errors out of 8240151 tests in 2447.273 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-12 19:12:29 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
dan ok  *
c.   th3make zipvfs.rc memdebug.rc
th3: 0 errors out of 8238942 tests in 5617.895 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-12 19:12:29 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
dan ok  *
d.   th3make zipvfs.rc quick.rc -valgrind
th3: 0 errors out of 2090713 tests in 10328.406 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-12 19:12:29 1c20c1c28b56411f106cf2f6961b3ad4b4d6f6c8
drh ok  *
e.   th3make zipvfs.rc quick.rc
th3: 0 errors out of 2115884 tests in 725.854 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-10 16:41:28 fab6f09044d033dd09ed8a22e06bc6a7851bbabf, win8, msvc
drh ok  *
f.   At least one of the above on Linux. drh ok  *
g.   At least one of the above on Mac OS X.
th3: 0 errors out of 2079242 tests in 1095.000 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-10 16:41:28 fab6f09044d033dd09ed8a22e06bc6a7851bbabf, quick.rc
drh ok  *
h.   At least one of the above on Windows. drh ok  *
 
20.   NX-Devkit compiles and runs tclsh th3make test.rc nx_compress.o without error.
th3: 0 errors out of 6863550 tests in 1628.202 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-12 22:31:02 9ecafc0c94102dca6da192f400278399464086fb
dan ok  *
 
21.   Amalgamation autoconf tarball. dan ok  *
 
a.   Builds. dan ok  *
b.   Works on Linux. dan ok  *
c.   Works on Mac OS X. dan ok  *
 
22.   TEA archive builds correctly and runs: dan ok  *
 
a.   Linux dan ok  *
b.   Mac OS X dan ok  *
 
23.   Run performance tests comparing the new SQLite release against the previous release and/or a release from one year ago. Document and justify any performance decrease. drh ok  *
 
a.   The Tcl "speed*.test" scripts.
On win7 32-bit. 3.8.7 2411ms, 3.8.11.1 2319ms, 3.9.0 2323ms. Much jitter in measurements.
drh ok  *
b.   The TH3 speed tests.
On iMac: 3.8.7 14.837s, 3.8.11.1 13.750s, trunk 13.598s.
drh ok  *
c.   The "speedtest1" program.
3.8.7: 2021M. 3.8.11.1: 1567M. trunk: 1551M. CPU usage decrease of 23% and 1%, respectively. Ubuntu.
drh ok  *
d.   ./speedtest1 --nosync --size 25 --trace 2>x.txt
time ./sqlite3 x.db <x.txt >/dev/null
Ubuntu. 3.8.7: 6,778,067,414 cycles. 3.8.11.1: 6,099,340,765, trunk: 6,048,346,945
drh ok  *
e.   One or more of the above on Linux. drh ok  *
f.   One or more of the above on Windows. drh ok  *
g.   One or more of the above on Mac OS X. drh ok  *
 
24.   Run make wordcount; bash run-wordcount.bash $OPTIONS with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c drh ok  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync drh ok  *
 
25.   Compute the size of a standard build (gcc -Os -c sqlite3.c) and and record the size change in the comments. drh ok  *
 
a.   Versus the previous release.
3.8.11.1: 482265. trunk: 483883. Size increase of 0.36%
drh ok  *
b.   Versus the trunk from 12 months ago.
3.8.7: 469805. trunk: 483883. Size increase of 3.0%
drh ok  *
 
26.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   Linux
584c219ad0946fdf7acb45417fd1e71bbd80fd99 sqlite3.c 31817313007b7e6ddfe48a4abfaab3e23f7781d3 sqlite3.h
drh ok+  *
b.   Mac OS X
584c219ad0946fdf7acb45417fd1e71bbd80fd99 sqlite3.c 31817313007b7e6ddfe48a4abfaab3e23f7781d3 sqlite3.h
drh ok  *
c.   OpenBSD
584c219ad0946fdf7acb45417fd1e71bbd80fd99 sqlite3.c 31817313007b7e6ddfe48a4abfaab3e23f7781d3 sqlite3.h
drh ok+  *
d.   Windows using native MinGW.
584c219ad0946fdf7acb45417fd1e71bbd80fd99 sqlite3.c 31817313007b7e6ddfe48a4abfaab3e23f7781d3 sqlite3.h
drh ok  *
e.   Windows using NMAKE.
584c219ad0946fdf7acb45417fd1e71bbd80fd99 sqlite3.c 31817313007b7e6ddfe48a4abfaab3e23f7781d3 sqlite3.h
drh ok  *
f.   Solaris (SPARC) using autoconf.
584c219ad0946fdf7acb45417fd1e71bbd80fd99 sqlite3.c 31817313007b7e6ddfe48a4abfaab3e23f7781d3 sqlite3.h
drh ok+  *
 
27.   Verify that databases (including FTS3 and RTREE databases), rollback journals, and WAL files created by the release candidate are readable and writeable by historical versions of SQLite (as long as no unsupported features are used) and vice versa.
Tested with backcompat.test against 3.6.4, 3.7.3 and 3.8.3. 0 errors out of 411 tests.
dan ok  *
 
28.   Cross-platform tests. Verify the ability of databases, WAL files, and rollback journals to be copied and used between 32-bit and 64-bit systems and between big-endian and little-endian systems. drh ok  *
 
a.   32-big to 64-little, database. drh ok  *
b.   32-big to 64-little, WAL. drh ok  *
c.   32-big to 64-little, rollback. drh ok  *
d.   64-little to 32-big, SEE database. drh ok  *
e.   64-little to 32-big, ZIPVFS database. drh ok  *
f.   64-little to 32-big, CEROD database. drh ok  *
 
29.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac OS X drh ok  *
c.   Solaris (SPARC) drh ok  *
d.   Windows (MinGW) drh ok  *
e.   Windows (MSVC) drh ok  *
f.   A database containing WITHOUT ROWID tables. drh ok  *
 
30.   Tests for the sessions branch with macros SQLITE_ENABLE_SESSION and SQLITE_ENABLE_PREUPDATE_HOOK. drh ok  *
 
a.   tclsh th3make session.rc test.rc (on Windows)
th3: 0 errors out of 5960258 tests in 5717.576 seconds. 32-bit little-endian th3: SQLite 3.9.0 2015-10-10 20:35:10 ed174ccf0ae615647ba026fed649d26dc9a98640
drh ok  *
b.   tclsh th3make session.rc test.rc (on Unix)
th3: 0 errors out of 7849090 tests in 1590.085 seconds. 64-bit little-endian th3: SQLite 3.9.0 2015-10-10 20:35:10 ed174ccf0ae615647ba026fed649d26dc9a98640
drh ok  *
c.   tclsh th3make session.rc quick.rc -valgrind
0 errors out of 11879758 tests and 1244 invocations in 903 seconds
drh ok  *
d.   make test (on Windows) drh ok  *
e.   make test (on Unix) drh ok  *
f.   testfixture test/session.test drh ok  *
 
31.   Verify that the "sqlite3-all.c" build target works and that the resulting "sqlite3-all.c" file and its include files ("sqlite3-?.c") compile and run the same as the single-file amalgamation. drh ok  *
 
32.   System.Data.SQLite builds and checks out using the SQLite core.
6f2858f6817ca70c132f0437ac2f0f74deb273d2, Debug NetFx46 x86: 0 errors/492 tests, 701 seconds, Release NetFx46 x86: 0 errors/492 tests, 662 seconds, Debug NetFx46 x64: 0 errors/492 tests, 766 seconds, Release NetFx46 x64: 0 errors/492 tests, 763 seconds, 0 errors/24 tests (Debug NetCF20-ARM/NetCF35-ARM), 0 errors/24 tests (Release NetCF20-ARM/NetCF35-ARM)
mistachkin ok  *