SQLite 3.8.7.2 Status Board

History    Checklist    Baseline

This is an abbreviated set of tests. The changes from 3.8.7.1 do not involve any code that is likely to behave different when moving between platforms. Therefore most of the testing is done on platforms where it is easy to run tests. A few quick sanity checks for other platforms are run, but not the whole analysis.

 
1.   Compiler warning checks. drh ok  *
 
a.   Linux GCC -Wall -Wextra (the tool/warnings.sh script)
The usual false-positives with -O3. No warnings with -O0
drh ok  *
b.   Mac GCC -Wall -Wextra (the tool/warnings.sh script) drh ok  *
c.   OpenBSD GCC using options -Wall
Complaints about strcpy(), strcat(), and sprintf() (used in test code and in the amatch extension). Will fix on the next release.
drh ok  *
d.   Windows MSVC Win32 /W3 drh ok  *
e.   Windows MSVC Win64 /W3 drh ok  *
f.   sh tool/warnings-clang.sh drh ok  *
 
2.   Documentation checks: drh ok  *
 
a.   Latest release on the index.html page drh ok  *
b.   Release announcement in changes.in drh ok  *
c.   Release accouncement in news.html drh ok  *
d.   No unresolved hyperlinks in the documentation build drh ok  *
e.   Requirements marks are up-to-date in both documentation and source code drh ok  *
f.   The compile-time options are all up-to-date (compile.html) drh ok  *
g.   Check new/changed documentation for spelling and gramatical mistakes drh ok  *
 
3.   Source code change review. "fossil diff --from version-3.8.7.1 --to branch-3.8.7" drh ok  *
 
a.   No stray changes drh ok  *
b.   All significant changes are mentioned in the "changes" log of the documentation drh ok  *
c.   New code complies with style guidelines drh ok  *
d.   New code complies with design rules drh ok  *
e.   Comments have been updated to reflect code changes drh ok  *
f.   Variables and functions have been renamed to reflect changes in their use drh ok  *
g.   Check all calls to sqlite3OsRead() and sqlite3OsWrite() to ensure that offset computations are 64-bit clean. drh ok  *
h.   Check all calls to memory allocation routines to ensure that size computations are 64-bit clean. Suggested regular expression for search: /(Re|M)alloc[FORZa-z]*\(/ drh ok  *
i.   Verify that source code contains no tabs and that surplus whitespace has been removed. drh ok  *
j.   Verify that the list of APIs exposed via the extension header is up-to-date (sqlite3ext.h) drh ok  *
 
4.   OMIT, ENABLE, and DISABLE options: drh ok  *
 
a.   tclsh ../tool/omittest.tcl -skip_run -target libsqlite3.a drh ok  *
 
5.   Tests for platform Linux x86_64: drh ok  *
 
a.   tclsh th3make cov.rc (verify 100% branch test coverage)
0 errors out of 1400669 tests in 196.605 seconds, 100% branch coverage.
drh ok  *
b.   tclsh th3make cov.rc -DHAVE_MALLOC_H -DHAVE_MALLOC_USABLE_SIZE (verify 100% branch test coverage)
0 errors out of 1398026 tests in 193.274 seconds, 100% branch coverage.
drh ok  *
c.   tclsh th3make cov.rc -DSQLITE_ENABLE_STAT4 (verify 100% branch test coverage)
0 errors out of 1477648 tests in 239.155 seconds, 100% branch coverage.
drh ok  *
d.   tclsh releasetest.tcl drh ok  *
e.   tclsh fulltest.tcl fast.rc test.rc
0 errors out of 78632106 tests in 13025.857 seconds
drh ok  *
f.   tclsh th3make memdebug.rc
0 errors out of 7625573 tests in 3454.573 seconds
drh ok  *
g.   tclsh th3make test-ex.rc
16 errors out of 18054260 tests in 20353.558 seconds. All errors are in hlr10000.1.1 and have to do with the 4-term version number.
drh ok+  *
h.   tclsh th3make test.rc CC=clang
0 errors out of 7653335 tests in 1609.319 seconds
drh ok  *
i.   tclsh th3make fast.rc CC=clang
0 errors out of 7446377 tests in 730.726 seconds
drh ok  *
j.   ./mptester x.db mptest/crash01.test
0 errors in 94 tests
drh ok  *
k.   ./mptester x.db mptest/multiwrite01.test
0 errors in 76 tests
drh ok  *
 
6.   Tests for 32-bit x86 Linux dan ok  *
 
a.   tclsh th3make test.rc -m32
0 errors out of 7653290 tests in 1558.920 seconds
drh ok  *
b.   tclsh th3make test.rc CC=clang -m32
0 errors out of 7653671 tests in 1772.588 seconds
drh ok  *
c.   tclsh th3make quick.rc -m32
./th3: 0 errors out of 1483198 tests in 700.147 seconds
dan ok  *
 
7.   Tests for platform Linux ARM (BeagleBoard) drh ok  *
 
a.   tclsh th3make min.rc
0 errors out of 1404247 tests in 3238.482 seconds
drh ok  *
 
8.   Tests for platform Mac OS-X x86: drh ok  *
 
a.   tclsh releasetest.tcl
One error when autovacuum_crash gave a different error from what was expected. Deemed pass.
drh ok  *
b.   tclsh fulltest.tcl min.rc
FINISHED (22:46:34) with 0 errors out of 7344362 tests in 2749.720 seconds
dan ok  *
c.   tclsh th3make fast.rc CC=clang
./th3: 0 errors out of 5842266 tests in 1776.524 seconds
dan ok  *
 
9.   Tests for platform Win32/64: drh ok  *
 
a.   make fulltestonly
0 errors out of 1167140 tests. Win8, msvc64.
drh ok  *
b.   th3make test.rc
8 errors out of 5816328 tests in 2131.564 seconds. All errors are req1_malloc02.311 and 312 and result from the fact that the memory allocator used gives exact-size allocations. This is a testing problem, not a problem in the code.
drh ok  *
c.   th3make fast.rc
0 errors out of 5865122 tests in 3115.144 seconds. Win7, mingw
drh ok  *
d.   Verify that EXEs and DLLs generated using MSVC contain the correct version number, product name, description, and copyright as resources. drh ok  *
e.   th3make msvc.rc test.rc /RTC1
8 errors out of 5816308 tests. All errors on req1_malloc02.31? as described above.
drh ok  *
f.   nmake /f makefile.msc DEBUG=2 test
0 errors out of 164096 tests. Win8, msvc64
drh ok  *
 
10.   Tests for platform Android: dan ok  *
 
a.   th3make quick.rc
/th3: 0 errors out of 1483122 tests in 2077.019 seconds
dan ok  *
 
11.   Tests for platform OpenBSD x86: drh ok  *
 
a.   make test
1 errors out of 137146 tests Failures on these tests: percentile-2.1.50. This is due to roundoff error in floating point computations and is acceptable.
drh ok  *
 
12.   Tests for platform Solaris Sparc: drh ok  *
 
a.   th3make fast.rc
0 errors out of 7451741 tests in 6846.098 seconds
drh ok  *
 
13.   Other test performed on any available platform: drh ok  *
 
a.   th3make alignment2.rc test.rc
0 errors out of 5490182 tests in 1261.280 seconds
drh ok  *
b.   th3make quick.rc -DHAVE_LOCALTIME_R
0 errors out of 1478491 tests in 164.930 seconds
drh ok  *
c.   th3make test.rc -DSQLITE_THREADSAFE=0
0 errors out of 7596661 tests in 1261.221 seconds
drh ok  *
d.   th3make test.rc -ftrapv
0 errors out of 7653382 tests in 2148.193 seconds
drh ok  *
e.   OPTS=-DSQLITE_SMALL_STACK make sqlite3.c; th3make quick.rc -DSQLITE_SMALL_STACK
0 errors out of 1483995 tests in 212.321 seconds
drh ok  *
f.   th3make test.rc CC=clang -fsanitize=undefined
0 errors out of 7653752 tests in 5666.729 seconds
drh ok  *
g.   th3make quick.rc -DSQLITE_MAX_ATTACHED=125
0 errors out of 1490258 tests in 298.481 seconds, win8
drh ok  *
h.   th3make quick.rc -valgrind
0 errors out of 1478386 tests in 6558.690 seconds, no valgrind errors or warnings.
drh ok  *
i.   th3make quick.rc -DSQLITE_ENABLE_STAT4 -valgrind
0 errors out of 1561276 tests in 8627.276 seconds
drh ok  *
j.   valgrind testfixture permutations.test valgrind
0 errors out of 162530 tests
drh ok  *
k.   th3make quick.rc -DSQLITE_TRACE_SIZE_LIMIT=15
0 errors out of 1490282 tests in 723.360 seconds, win7
drh ok  *
l.   th3make quick.rc -funsigned-char
o errors out of 1490263 tests in 724.494 seconds, win7
drh ok  *
m.   th3make quick.rc -fsigned-char
0 errors out of 1490263 tests in 724.783 seconds, win7
drh ok  *
n.   th3make quick.rc -DSQLITE_ENABLE_MEMSYS5 -DTH3_LOWMEM
0 errors out of 1407953 tests in 696.093 seconds. Win7
drh ok  *
o.   th3make quick.rc -DSQLITE_ENABLE_RTREE -DSQLITE_ENABLE_FTS4
0 errors out of 1519991 tests in 352.335 seconds. win8
drh ok  *
p.   th3make quick.rc -DSQLITE_TEST_REALLOC_STRESS
0 errors out of 1825349 tests in 934.988 seconds. Win7.
drh ok  *
q.   th3make test.rc ../th3private/*.test
0 errors out of 668 tests in 8.184 seconds
drh ok  *
r.   ./configure; make fulltest
0 errors out of 2538349 tests
drh ok  *
 
14.   Sqllogictest: drh ok  *
 
a.   Checked in latest SQLite amalgamation drh ok  *
b.   No compiler warnings drh ok  *
c.   run-all.sh with no errors drh ok  *
 
15.   Build using the configure/make in main source tree. drh ok  *
 
a.   Make sure that autoconf has been run to update the configure script. drh ok  *
b.   Run "make test" to verify that the build works. drh ok  *
c.   Verify the correct version numbers have been installed. drh ok  *
d.   Verify that the --disable-amalgamation option to configure works. drh ok  *
e.   Works on Linux drh ok  *
 
16.   Exported symbol checks: tool/symbols.sh drh ok  *
 
a.   No private symbols exported. (This is also checked by releasetest.tcl) drh ok  *
b.   No undesirable library dependencies drh ok  *
c.   No private symbols exported in MinGW (check using tool/symbols-mingw.sh) drh ok  *
d.   No undesirable library dependencies on MinGW drh ok  *
 
17.   Fossil updated to use the latest SQLite. drh ok  *
 
a.   No compiler warnings drh ok  *
b.   Used on active websites with no issues drh ok  *
 
18.   Firefox recompiled using lastest SQLite.
Tested by Ben Turner on Mozilla's test servers.
drh ok  *
 
a.   No SQLite compiler warnings
Tested by Ben Turner on Mozilla's test servers.
drh ok  *
b.   Active use with no issues noted
Tested by Ben Turner on Mozilla's test servers.
drh ok  *
 
19.   CEROD tests in the test/ subdirectory. drh ok  *
 
20.   SEE tests using TH3: drh ok  *
 
a.   see.c
0 errors out of 1460760 tests in 202.588 seconds
drh ok  *
 
21.   ZIPVFS tests: drh ok  *
 
a.   testfixture ../zipvfs/test/zipvfs.test (See test/README.txt in the ZIPVFS source tree for details.)
0 errors out of 46811 tests
drh ok  *
b.   th3make zipvfs.rc test.rc
0 errors out of 8032407 tests in 1794.877 seconds
drh ok  *
 
22.   NX-Devkit compiles and runs "tclsh th3make test.rc nx_compress.o" without error.
0 errors out of 6686754 tests in 1202.551 seconds
drh ok  *
 
23.   Amalgamation autoconf tarball. dan ok  *
 
a.   Builds dan ok  *
b.   Works on Linux dan ok  *
c.   Works on Mac dan ok  *
 
24.   Run performance tests comparing the new SQLite release against the previous release. drh ok  *
 
a.   The speedtest1 program
26.580 billion cycles on 3.8.7.2 versus 26.587 billion cycles on 3.8.7.1.
drh ok  *
 
25.   Run "make wordcount; bash run-wordcount.bash $OPTIONS" with options: drh ok  *
 
a.   bash run-wordcount.sh sqlite3.c drh ok  *
b.   bash run-wordcount.sh sqlite3.c --cachesize 5 --nosync drh ok  *
 
26.   Verify that the amalgamation builds are byte-for-byte identical on all workstation platforms. drh ok  *
 
a.   linux
687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c 10594b50adadce48993530aad6ab5410e98b576f sqlite3.h
drh ok  *
b.   mac
687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c 10594b50adadce48993530aad6ab5410e98b576f sqlite3.h
drh ok  *
c.   openbsd
687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c 10594b50adadce48993530aad6ab5410e98b576f sqlite3.h
drh ok  *
d.   windows native mingw
687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c 687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c
drh ok  *
e.   windows nmake
687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c 10594b50adadce48993530aad6ab5410e98b576f sqlite3.h
drh ok  *
f.   sparc solaris using autoconf
687a18970c4a420f98ee138b93e99bdaacadf525 sqlite3.c 10594b50adadce48993530aad6ab5410e98b576f sqlite3.h
drh ok  *
 
27.   Build and verify correct operation of sqlite3_analyzer on: drh ok  *
 
a.   Linux drh ok  *
b.   Mac drh ok  *
c.   Win32 drh ok  *
 
28.   Verify that the sqlite3-all.c build target works and that the resulting sqlite3-all.c file and its include files (sqlite3-?.c) compile and run the same as the single-file amalgamation. s drh ok  *