/ Check-in [ae88f8e1]
Login
SQLite training in Houston TX on 2019-11-05 (details)
Part of the 2019 Tcl Conference

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge latest trunk changes with this branch.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | reuse-schema
Files: files | file ages | folders
SHA3-256: ae88f8e1ffc33748e45308c21ba83499228ae05344df81e41dd61e14a1270588
User & Date: dan 2018-11-09 18:44:45
Wiki:reuse-schema
Context
2019-02-02
21:02
Try new approach ensuring that each Schema object is only used by one connection/database at any one time. check-in: 9e8e5f52 user: dan tags: reuse-schema
2018-11-15
21:20
Fix some problems with the feature on this branch. Many problems remain. check-in: 31b6aee7 user: dan tags: reuse-schema1
2018-11-09
18:44
Merge latest trunk changes with this branch. check-in: ae88f8e1 user: dan tags: reuse-schema
14:17
Fix for ticket [787fa716be3a7f650cac] check-in: 531eca61 user: drh tags: trunk
2018-10-08
23:29
Add the ability to open a database connection in -reuseschema mode in the CLI. Adjust memory allocation accounting so that a shared schema is allocated across each database sharing it. check-in: 3869b210 user: drh tags: reuse-schema
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to Makefile.in.

    18     18   TOP = @abs_srcdir@
    19     19   
    20     20   # C Compiler and options for use in building executables that
    21     21   # will run on the platform that is doing the build.
    22     22   #
    23     23   BCC = @BUILD_CC@ @BUILD_CFLAGS@
    24     24   
    25         -# TCC is the C Compile and options for use in building executables that 
           25  +# TCC is the C Compile and options for use in building executables that
    26     26   # will run on the target platform.  (BCC and TCC are usually the
    27     27   # same unless your are cross-compiling.)  Separate CC and CFLAGS macros
    28     28   # are provide so that these aspects of the build process can be changed
    29     29   # on the "make" command-line.  Ex:  "make CC=clang CFLAGS=-fsanitize=undefined"
    30     30   #
    31     31   CC = @CC@
    32     32   CFLAGS = @CPPFLAGS@ @CFLAGS@
    33     33   TCC = ${CC} ${CFLAGS} -I. -I${TOP}/src -I${TOP}/ext/rtree -I${TOP}/ext/icu
    34     34   TCC += -I${TOP}/ext/fts3 -I${TOP}/ext/async -I${TOP}/ext/session
    35     35   
    36     36   # Define this for the autoconf-based build, so that the code knows it can
    37     37   # include the generated config.h
    38         -# 
           38  +#
    39     39   TCC += -D_HAVE_SQLITE_CONFIG_H -DBUILD_sqlite
    40     40   
    41     41   # Define -DNDEBUG to compile without debugging (i.e., for production usage)
    42     42   # Omitting the define will cause extra debugging code to be inserted and
    43     43   # includes extra comments when "EXPLAIN stmt" is used.
    44     44   #
    45     45   TCC += @TARGET_DEBUG@
................................................................................
    62     62   LIBREADLINE = @TARGET_READLINE_LIBS@
    63     63   
    64     64   # Should the database engine be compiled threadsafe
    65     65   #
    66     66   TCC += -DSQLITE_THREADSAFE=@SQLITE_THREADSAFE@
    67     67   
    68     68   # Any target libraries which libsqlite must be linked against
    69         -# 
           69  +#
    70     70   TLIBS = @LIBS@ $(LIBS)
    71     71   
    72     72   # Flags controlling use of the in memory btree implementation
    73     73   #
    74     74   # SQLITE_TEMP_STORE is 0 to force temporary tables to be in a file, 1 to
    75     75   # default to file, 2 to default to memory, and 3 to force temporary
    76     76   # tables to always be in memory.
    77     77   #
    78     78   TEMP_STORE = -DSQLITE_TEMP_STORE=@TEMP_STORE@
    79     79   
    80     80   # Enable/disable loadable extensions, and other optional features
    81         -# based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*).  
    82         -# The same set of OMIT and ENABLE flags should be passed to the 
           81  +# based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*).
           82  +# The same set of OMIT and ENABLE flags should be passed to the
    83     83   # LEMON parser generator and the mkkeywordhash tool as well.
    84     84   OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@
    85     85   
    86     86   TCC += $(OPT_FEATURE_FLAGS)
    87     87   
    88     88   # Add in any optional parameters specified on the make commane line
    89     89   # ie.  make "OPTS=-DSQLITE_ENABLE_FOO=1 -DSQLITE_OMIT_FOO=1".
................................................................................
   122    122   # The suffix used on shared libraries.  Ex:  ".dll", ".so", ".dylib"
   123    123   #
   124    124   SHLIB_SUFFIX = @TCL_SHLIB_SUFFIX@
   125    125   
   126    126   # If gcov support was enabled by the configure script, add the appropriate
   127    127   # flags here.  It's not always as easy as just having the user add the right
   128    128   # CFLAGS / LDFLAGS, because libtool wants to use CFLAGS when linking, which
   129         -# causes build errors with -fprofile-arcs -ftest-coverage with some GCCs.  
   130         -# Supposedly GCC does the right thing if you use --coverage, but in 
          129  +# causes build errors with -fprofile-arcs -ftest-coverage with some GCCs.
          130  +# Supposedly GCC does the right thing if you use --coverage, but in
   131    131   # practice it still fails.  See:
   132    132   #
   133    133   # http://www.mail-archive.com/debian-gcc@lists.debian.org/msg26197.html
   134    134   #
   135    135   # for more info.
   136    136   #
   137    137   GCOV_CFLAGS1 = -DSQLITE_COVERAGE_TEST=1 -fprofile-arcs -ftest-coverage
................................................................................
   421    421     $(TOP)/src/test_vfs.c \
   422    422     $(TOP)/src/test_windirent.c \
   423    423     $(TOP)/src/test_window.c \
   424    424     $(TOP)/src/test_wsd.c       \
   425    425     $(TOP)/ext/fts3/fts3_term.c \
   426    426     $(TOP)/ext/fts3/fts3_test.c  \
   427    427     $(TOP)/ext/session/test_session.c \
   428         -  $(TOP)/ext/rbu/test_rbu.c 
          428  +  $(TOP)/ext/rbu/test_rbu.c
   429    429   
   430    430   # Statically linked extensions
   431    431   #
   432    432   TESTSRC += \
   433    433     $(TOP)/ext/expert/sqlite3expert.c \
   434    434     $(TOP)/ext/expert/test_expert.c \
   435    435     $(TOP)/ext/misc/amatch.c \
................................................................................
   503    503     $(TOP)/ext/fts3/fts3_aux.c \
   504    504     $(TOP)/ext/fts3/fts3_expr.c \
   505    505     $(TOP)/ext/fts3/fts3_term.c \
   506    506     $(TOP)/ext/fts3/fts3_tokenizer.c \
   507    507     $(TOP)/ext/fts3/fts3_write.c \
   508    508     $(TOP)/ext/async/sqlite3async.c \
   509    509     $(TOP)/ext/session/sqlite3session.c \
   510         -  $(TOP)/ext/misc/stmt.c 
          510  +  $(TOP)/ext/misc/stmt.c
   511    511   
   512    512   # Header files used by all library source files.
   513    513   #
   514    514   HDR = \
   515    515      $(TOP)/src/btree.h \
   516    516      $(TOP)/src/btreeInt.h \
   517    517      $(TOP)/src/hash.h \
................................................................................
   575    575   #
   576    576   FUZZDATA = \
   577    577     $(TOP)/test/fuzzdata1.db \
   578    578     $(TOP)/test/fuzzdata2.db \
   579    579     $(TOP)/test/fuzzdata3.db \
   580    580     $(TOP)/test/fuzzdata4.db \
   581    581     $(TOP)/test/fuzzdata5.db \
   582         -  $(TOP)/test/fuzzdata6.db
          582  +  $(TOP)/test/fuzzdata6.db \
          583  +  $(TOP)/test/fuzzdata7.db
   583    584   
   584    585   # Standard options to testfixture
   585    586   #
   586    587   TESTOPTS = --verbose=file --output=test-out.txt
   587    588   
   588    589   # Extra compiler options for various shell tools
   589    590   #
................................................................................
   592    593   SHELL_OPT += -DSQLITE_ENABLE_RTREE
   593    594   SHELL_OPT += -DSQLITE_ENABLE_EXPLAIN_COMMENTS
   594    595   SHELL_OPT += -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION
   595    596   SHELL_OPT += -DSQLITE_ENABLE_STMTVTAB
   596    597   SHELL_OPT += -DSQLITE_ENABLE_DBPAGE_VTAB
   597    598   SHELL_OPT += -DSQLITE_ENABLE_DBSTAT_VTAB
   598    599   SHELL_OPT += -DSQLITE_ENABLE_OFFSET_SQL_FUNC
          600  +SHELL_OPT += -DSQLITE_ENABLE_DESERIALIZE
   599    601   SHELL_OPT += -DSQLITE_INTROSPECTION_PRAGMAS
   600    602   FUZZERSHELL_OPT = -DSQLITE_ENABLE_JSON1
   601    603   FUZZCHECK_OPT = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_MEMSYS5 -DSQLITE_OSS_FUZZ
   602    604   FUZZCHECK_OPT += -DSQLITE_MAX_MEMORY=50000000
   603    605   FUZZCHECK_OPT += -DSQLITE_PRINTF_PRECISION_LIMIT=1000
   604    606   FUZZCHECK_SRC = $(TOP)/test/fuzzcheck.c $(TOP)/test/ossfuzz.c
   605         -DBFUZZ_OPT = 
          607  +DBFUZZ_OPT =
   606    608   
   607    609   # This is the default Makefile target.  The objects listed here
   608    610   # are what get build when you type just "make" with no arguments.
   609    611   #
   610    612   all:	sqlite3.h libsqlite3.la sqlite3$(TEXE) $(HAVE_TCL:1=libtclsqlite3.la)
   611    613   
   612    614   Makefile: $(TOP)/Makefile.in
................................................................................
   660    662   
   661    663   sessionfuzz$(TEXE):	$(TOP)/test/sessionfuzz.c sqlite3.c sqlite3.h
   662    664   	$(CC) $(CFLAGS) -I. -o $@ $(TOP)/test/sessionfuzz.c $(TLIBS)
   663    665   
   664    666   dbfuzz$(TEXE):	$(TOP)/test/dbfuzz.c sqlite3.c sqlite3.h
   665    667   	$(LTLINK) -o $@ $(DBFUZZ_OPT) $(TOP)/test/dbfuzz.c sqlite3.c $(TLIBS)
   666    668   
          669  +DBFUZZ2_OPTS = \
          670  +  -DSQLITE_THREADSAFE=0 \
          671  +  -DSQLITE_OMIT_LOAD_EXTENSION \
          672  +  -DSQLITE_ENABLE_DESERIALIZE \
          673  +  -DSQLITE_DEBUG \
          674  +  -DSQLITE_ENABLE_DBSTAT_VTAB \
          675  +  -DSQLITE_ENABLE_RTREE \
          676  +  -DSQLITE_ENABLE_FTS4 \
          677  +  -DSQLITE_EANBLE_FTS5
          678  +
          679  +dbfuzz2:	$(TOP)/test/dbfuzz2.c sqlite3.c sqlite3.h
          680  +	clang-6.0 -I. -g -O0 -fsanitize=fuzzer,undefined -o dbfuzz2 \
          681  +		$(DBFUZZ2_OPTS) $(TOP)/test/dbfuzz2.c sqlite3.c
          682  +	mkdir -p dbfuzz2-dir
          683  +	cp $(TOP)/test/dbfuzz2-seed* dbfuzz2-dir
          684  +
   667    685   mptester$(TEXE):	sqlite3.lo $(TOP)/mptest/mptest.c
   668    686   	$(LTLINK) -o $@ -I. $(TOP)/mptest/mptest.c sqlite3.lo \
   669    687   		$(TLIBS) -rpath "$(libdir)"
   670    688   
   671    689   MPTEST1=./mptester$(TEXE) mptest.db $(TOP)/mptest/crash01.test --repeat 20
   672    690   MPTEST2=./mptester$(TEXE) mptest.db $(TOP)/mptest/multiwrite01.test --repeat 20
   673    691   mptest:	mptester$(TEXE)
................................................................................
  1123   1141      fts5parse.c fts5parse.h \
  1124   1142      $(TOP)/ext/fts5/fts5_storage.c \
  1125   1143      $(TOP)/ext/fts5/fts5_tokenize.c \
  1126   1144      $(TOP)/ext/fts5/fts5_unicode2.c \
  1127   1145      $(TOP)/ext/fts5/fts5_varint.c \
  1128   1146      $(TOP)/ext/fts5/fts5_vocab.c  \
  1129   1147   
  1130         -fts5parse.c:	$(TOP)/ext/fts5/fts5parse.y lemon 
         1148  +fts5parse.c:	$(TOP)/ext/fts5/fts5parse.y lemon
  1131   1149   	cp $(TOP)/ext/fts5/fts5parse.y .
  1132   1150   	rm -f fts5parse.h
  1133   1151   	./lemon$(BEXE) $(OPTS) fts5parse.y
  1134   1152   
  1135   1153   fts5parse.h: fts5parse.c
  1136   1154   
  1137   1155   fts5.c: $(FTS5_SRC)
................................................................................
  1150   1168   # If using the amalgamation, use sqlite3.c directly to build the test
  1151   1169   # fixture.  Otherwise link against libsqlite3.la.  (This distinction is
  1152   1170   # necessary because the test fixture requires non-API symbols which are
  1153   1171   # hidden when the library is built via the amalgamation).
  1154   1172   #
  1155   1173   TESTFIXTURE_FLAGS  = -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1
  1156   1174   TESTFIXTURE_FLAGS += -DTCLSH_INIT_PROC=sqlite3TestInit
  1157         -TESTFIXTURE_FLAGS += -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" -DSQLITE_CORE 
         1175  +TESTFIXTURE_FLAGS += -DSQLITE_SERVER=1 -DSQLITE_PRIVATE="" -DSQLITE_CORE
  1158   1176   TESTFIXTURE_FLAGS += -DBUILD_sqlite
  1159   1177   TESTFIXTURE_FLAGS += -DSQLITE_SERIES_CONSTRAINT_VERIFY=1
  1160   1178   TESTFIXTURE_FLAGS += -DSQLITE_DEFAULT_PAGE_SIZE=1024
  1161   1179   TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_STMTVTAB
  1162   1180   TESTFIXTURE_FLAGS += -DSQLITE_ENABLE_DBPAGE_VTAB
  1163   1181   
  1164   1182   TESTFIXTURE_SRC0 = $(TESTSRC2) libsqlite3.la
................................................................................
  1166   1184   TESTFIXTURE_SRC = $(TESTSRC) $(TOP)/src/tclsqlite.c
  1167   1185   TESTFIXTURE_SRC += $(TESTFIXTURE_SRC$(USE_AMALGAMATION))
  1168   1186   
  1169   1187   testfixture$(TEXE):	$(TESTFIXTURE_SRC)
  1170   1188   	$(LTLINK) -DSQLITE_NO_SYNC=1 $(TEMP_STORE) $(TESTFIXTURE_FLAGS) \
  1171   1189   		-o $@ $(TESTFIXTURE_SRC) $(LIBTCL) $(TLIBS)
  1172   1190   
         1191  +coretestprogs:	$(TESTPROGS)
         1192  +
         1193  +testprogs:	coretestprogs srcck1$(BEXE) fuzzcheck$(TEXE) sessionfuzz$(TEXE)
         1194  +
  1173   1195   # A very detailed test running most or all test cases
  1174   1196   fulltest:	$(TESTPROGS) fuzztest
  1175   1197   	./testfixture$(TEXE) $(TOP)/test/all.test $(TESTOPTS)
  1176   1198   
  1177   1199   # Really really long testing
  1178   1200   soaktest:	$(TESTPROGS)
  1179   1201   	./testfixture$(TEXE) $(TOP)/test/all.test -soak=1 $(TESTOPTS)
................................................................................
  1225   1247   
  1226   1248   sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in
  1227   1249   	$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c
  1228   1250   
  1229   1251   sqlite3_analyzer$(TEXE): sqlite3_analyzer.c
  1230   1252   	$(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS)
  1231   1253   
  1232         -sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in	
         1254  +sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in
  1233   1255   	$(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in >sqltclsh.c
  1234   1256   
  1235   1257   sqltclsh$(TEXE): sqltclsh.c
  1236   1258   	$(LTLINK) sqltclsh.c -o $@ $(LIBTCL) $(TLIBS)
  1237   1259   
  1238   1260   sqlite3_expert$(TEXE): $(TOP)/ext/expert/sqlite3expert.h $(TOP)/ext/expert/sqlite3expert.c $(TOP)/ext/expert/expert.c sqlite3.c
  1239   1261   	$(LTLINK)	$(TOP)/ext/expert/sqlite3expert.h $(TOP)/ext/expert/sqlite3expert.c $(TOP)/ext/expert/expert.c sqlite3.c -o sqlite3_expert $(TLIBS)
................................................................................
  1292   1314   	$(LTLINK) $(ST_OPT) -o $@ $(TOP)/test/speedtest1.c sqlite3.c $(TLIBS)
  1293   1315   
  1294   1316   KV_OPT += -DSQLITE_DIRECT_OVERFLOW_READ
  1295   1317   
  1296   1318   kvtest$(TEXE):	$(TOP)/test/kvtest.c sqlite3.c
  1297   1319   	$(LTLINK) $(KV_OPT) -o $@ $(TOP)/test/kvtest.c sqlite3.c $(TLIBS)
  1298   1320   
  1299         -rbu$(EXE): $(TOP)/ext/rbu/rbu.c $(TOP)/ext/rbu/sqlite3rbu.c sqlite3.lo 
         1321  +rbu$(EXE): $(TOP)/ext/rbu/rbu.c $(TOP)/ext/rbu/sqlite3rbu.c sqlite3.lo
  1300   1322   	$(LTLINK) -I. -o $@ $(TOP)/ext/rbu/rbu.c sqlite3.lo $(TLIBS)
  1301   1323   
  1302   1324   loadfts$(EXE): $(TOP)/tool/loadfts.c libsqlite3.la
  1303   1325   	$(LTLINK) $(TOP)/tool/loadfts.c libsqlite3.la -o $@ $(TLIBS)
  1304   1326   
  1305   1327   # This target will fail if the SQLite amalgamation contains any exported
  1306   1328   # symbols that do not begin with "sqlite3_". It is run as part of the
................................................................................
  1320   1342   
  1321   1343   snapshot-tarball: sqlite3.c
  1322   1344   	TOP=$(TOP) sh $(TOP)/tool/mkautoconfamal.sh --snapshot
  1323   1345   
  1324   1346   # The next two rules are used to support the "threadtest" target. Building
  1325   1347   # threadtest runs a few thread-safety tests that are implemented in C. This
  1326   1348   # target is invoked by the releasetest.tcl script.
  1327         -# 
         1349  +#
  1328   1350   THREADTEST3_SRC = $(TOP)/test/threadtest3.c    \
  1329   1351                     $(TOP)/test/tt3_checkpoint.c \
  1330   1352                     $(TOP)/test/tt3_index.c      \
  1331   1353                     $(TOP)/test/tt3_vacuum.c      \
  1332   1354                     $(TOP)/test/tt3_stress.c      \
  1333   1355                     $(TOP)/test/tt3_lookaside1.c
  1334   1356   
  1335   1357   threadtest3$(TEXE): sqlite3.lo $(THREADTEST3_SRC)
  1336   1358   	$(LTLINK) $(TOP)/test/threadtest3.c $(TOP)/src/test_multiplex.c sqlite3.lo -o $@ $(TLIBS)
  1337   1359   
  1338   1360   threadtest: threadtest3$(TEXE)
  1339   1361   	./threadtest3$(TEXE)
  1340   1362   
  1341         -releasetest:	
         1363  +releasetest:
  1342   1364   	$(TCLSH_CMD) $(TOP)/test/releasetest.tcl
  1343   1365   
  1344   1366   # Standard install and cleanup targets
  1345   1367   #
  1346   1368   lib_install:	libsqlite3.la
  1347   1369   	$(INSTALL) -d $(DESTDIR)$(libdir)
  1348   1370   	$(LTINSTALL) libsqlite3.la $(DESTDIR)$(libdir)
  1349         -	
         1371  +
  1350   1372   install:	sqlite3$(TEXE) lib_install sqlite3.h sqlite3.pc ${HAVE_TCL:1=tcl_install}
  1351   1373   	$(INSTALL) -d $(DESTDIR)$(bindir)
  1352   1374   	$(LTINSTALL) sqlite3$(TEXE) $(DESTDIR)$(bindir)
  1353   1375   	$(INSTALL) -d $(DESTDIR)$(includedir)
  1354   1376   	$(INSTALL) -m 0644 sqlite3.h $(DESTDIR)$(includedir)
  1355   1377   	$(INSTALL) -m 0644 $(TOP)/src/sqlite3ext.h $(DESTDIR)$(includedir)
  1356   1378   	$(INSTALL) -d $(DESTDIR)$(pkgconfigdir)
................................................................................
  1360   1382   	echo 'package ifneeded sqlite3 $(RELEASE) [list load $(TCLLIBDIR)/libtclsqlite3$(SHLIB_SUFFIX) sqlite3]' > $@
  1361   1383   tcl_install:	lib_install libtclsqlite3.la pkgIndex.tcl
  1362   1384   	$(INSTALL) -d $(DESTDIR)$(TCLLIBDIR)
  1363   1385   	$(LTINSTALL) libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR)
  1364   1386   	rm -f $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.la $(DESTDIR)$(TCLLIBDIR)/libtclsqlite3.a
  1365   1387   	$(INSTALL) -m 0644 pkgIndex.tcl $(DESTDIR)$(TCLLIBDIR)
  1366   1388   
  1367         -clean:	
         1389  +clean:
  1368   1390   	rm -f *.lo *.la *.o sqlite3$(TEXE) libsqlite3.la
  1369   1391   	rm -f sqlite3.h opcodes.*
  1370   1392   	rm -rf .libs .deps
  1371   1393   	rm -f lemon$(BEXE) lempar.c parse.* sqlite*.tar.gz
  1372   1394   	rm -f mkkeywordhash$(BEXE) keywordhash.h
  1373   1395   	rm -f *.da *.bb *.bbg gmon.out
  1374   1396   	rm -rf tsrc .target_source

Changes to Makefile.msc.

  1619   1619   #
  1620   1620   FUZZDATA = \
  1621   1621     $(TOP)\test\fuzzdata1.db \
  1622   1622     $(TOP)\test\fuzzdata2.db \
  1623   1623     $(TOP)\test\fuzzdata3.db \
  1624   1624     $(TOP)\test\fuzzdata4.db \
  1625   1625     $(TOP)\test\fuzzdata5.db \
  1626         -  $(TOP)\test\fuzzdata6.db
         1626  +  $(TOP)\test\fuzzdata6.db \
         1627  +  $(TOP)\test\fuzzdata7.db
  1627   1628   # <</mark>>
  1628   1629   
  1629   1630   # Additional compiler options for the shell.  These are only effective
  1630   1631   # when the shell is not being dynamically linked.
  1631   1632   #
  1632   1633   !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0
  1633   1634   SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1
  1634   1635   SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1
  1635   1636   SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1
         1637  +SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_DESERIALIZE=1
  1636   1638   !ENDIF
  1637   1639   
  1638   1640   # <<mark>>
  1639   1641   # Extra compiler options for various test tools.
  1640   1642   #
  1641   1643   MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1 -DSQLITE_ENABLE_FTS5
  1642   1644   FUZZERSHELL_COMPILE_OPTS = -DSQLITE_ENABLE_JSON1
................................................................................
  2331   2333   		$(TESTFIXTURE_SRC) \
  2332   2334   		/link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS)
  2333   2335   
  2334   2336   extensiontest:	testfixture.exe testloadext.dll
  2335   2337   	@set PATH=$(LIBTCLPATH);$(PATH)
  2336   2338   	.\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS)
  2337   2339   
         2340  +coretestprogs:	$(TESTPROGS)
         2341  +
         2342  +testprogs:	coretestprogs srcck1.exe fuzzcheck.exe sessionfuzz.exe
         2343  +
  2338   2344   fulltest:	$(TESTPROGS) fuzztest
  2339   2345   	@set PATH=$(LIBTCLPATH);$(PATH)
  2340   2346   	.\testfixture.exe $(TOP)\test\all.test $(TESTOPTS)
  2341   2347   
  2342   2348   soaktest:	$(TESTPROGS)
  2343   2349   	@set PATH=$(LIBTCLPATH);$(PATH)
  2344   2350   	.\testfixture.exe $(TOP)\test\all.test -soak=1 $(TESTOPTS)

Changes to ext/fts3/fts3.c.

  1817   1817     sqlite3_int64 *piFirst,         /* OUT: Selected child node */
  1818   1818     sqlite3_int64 *piLast           /* OUT: Selected child node */
  1819   1819   ){
  1820   1820     int rc = SQLITE_OK;             /* Return code */
  1821   1821     const char *zCsr = zNode;       /* Cursor to iterate through node */
  1822   1822     const char *zEnd = &zCsr[nNode];/* End of interior node buffer */
  1823   1823     char *zBuffer = 0;              /* Buffer to load terms into */
  1824         -  int nAlloc = 0;                 /* Size of allocated buffer */
         1824  +  i64 nAlloc = 0;                 /* Size of allocated buffer */
  1825   1825     int isFirstTerm = 1;            /* True when processing first term on page */
  1826   1826     sqlite3_int64 iChild;           /* Block id of child node to descend to */
  1827   1827   
  1828   1828     /* Skip over the 'height' varint that occurs at the start of every 
  1829   1829     ** interior node. Then load the blockid of the left-child of the b-tree
  1830   1830     ** node into variable iChild.  
  1831   1831     **
................................................................................
  1855   1855       if( !isFirstTerm ){
  1856   1856         zCsr += fts3GetVarint32(zCsr, &nPrefix);
  1857   1857       }
  1858   1858       isFirstTerm = 0;
  1859   1859       zCsr += fts3GetVarint32(zCsr, &nSuffix);
  1860   1860       
  1861   1861       assert( nPrefix>=0 && nSuffix>=0 );
  1862         -    if( &zCsr[nSuffix]>zEnd ){
         1862  +    if( nPrefix>zCsr-zNode || nSuffix>zEnd-zCsr ){
  1863   1863         rc = FTS_CORRUPT_VTAB;
  1864   1864         goto finish_scan;
  1865   1865       }
  1866         -    if( nPrefix+nSuffix>nAlloc ){
         1866  +    if( (i64)nPrefix+nSuffix>nAlloc ){
  1867   1867         char *zNew;
  1868         -      nAlloc = (nPrefix+nSuffix) * 2;
  1869         -      zNew = (char *)sqlite3_realloc(zBuffer, nAlloc);
         1868  +      nAlloc = ((i64)nPrefix+nSuffix) * 2;
         1869  +      zNew = (char *)sqlite3_realloc64(zBuffer, nAlloc);
  1870   1870         if( !zNew ){
  1871   1871           rc = SQLITE_NOMEM;
  1872   1872           goto finish_scan;
  1873   1873         }
  1874   1874         zBuffer = zNew;
  1875   1875       }
  1876   1876       assert( zBuffer );

Changes to ext/fts3/fts3_write.c.

  1370   1370     rc = fts3SegReaderRequire(pReader, pNext, FTS3_VARINT_MAX*2);
  1371   1371     if( rc!=SQLITE_OK ) return rc;
  1372   1372     
  1373   1373     /* Because of the FTS3_NODE_PADDING bytes of padding, the following is 
  1374   1374     ** safe (no risk of overread) even if the node data is corrupted. */
  1375   1375     pNext += fts3GetVarint32(pNext, &nPrefix);
  1376   1376     pNext += fts3GetVarint32(pNext, &nSuffix);
  1377         -  if( nPrefix<0 || nSuffix<=0 
  1378         -   || &pNext[nSuffix]>&pReader->aNode[pReader->nNode] 
         1377  +  if( nSuffix<=0 
         1378  +   || (&pReader->aNode[pReader->nNode] - pNext)<nSuffix
         1379  +   || nPrefix>pReader->nTermAlloc
  1379   1380     ){
  1380   1381       return FTS_CORRUPT_VTAB;
  1381   1382     }
  1382   1383   
  1383         -  if( nPrefix+nSuffix>pReader->nTermAlloc ){
  1384         -    int nNew = (nPrefix+nSuffix)*2;
  1385         -    char *zNew = sqlite3_realloc(pReader->zTerm, nNew);
         1384  +  /* Both nPrefix and nSuffix were read by fts3GetVarint32() and so are
         1385  +  ** between 0 and 0x7FFFFFFF. But the sum of the two may cause integer
         1386  +  ** overflow - hence the (i64) casts.  */
         1387  +  if( (i64)nPrefix+nSuffix>(i64)pReader->nTermAlloc ){
         1388  +    i64 nNew = ((i64)nPrefix+nSuffix)*2;
         1389  +    char *zNew = sqlite3_realloc64(pReader->zTerm, nNew);
  1386   1390       if( !zNew ){
  1387   1391         return SQLITE_NOMEM;
  1388   1392       }
  1389   1393       pReader->zTerm = zNew;
  1390   1394       pReader->nTermAlloc = nNew;
  1391   1395     }
  1392   1396   
................................................................................
  1400   1404     pReader->aDoclist = pNext;
  1401   1405     pReader->pOffsetList = 0;
  1402   1406   
  1403   1407     /* Check that the doclist does not appear to extend past the end of the
  1404   1408     ** b-tree node. And that the final byte of the doclist is 0x00. If either 
  1405   1409     ** of these statements is untrue, then the data structure is corrupt.
  1406   1410     */
  1407         -  if( &pReader->aDoclist[pReader->nDoclist]>&pReader->aNode[pReader->nNode] 
         1411  +  if( (&pReader->aNode[pReader->nNode] - pReader->aDoclist)<pReader->nDoclist
  1408   1412      || (pReader->nPopulate==0 && pReader->aDoclist[pReader->nDoclist-1])
  1409   1413     ){
  1410   1414       return FTS_CORRUPT_VTAB;
  1411   1415     }
  1412   1416     return SQLITE_OK;
  1413   1417   }
  1414   1418   
................................................................................
  3726   3730       p->aNode = 0;
  3727   3731     }else{
  3728   3732       if( bFirst==0 ){
  3729   3733         p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nPrefix);
  3730   3734       }
  3731   3735       p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &nSuffix);
  3732   3736   
         3737  +    if( nPrefix>p->iOff || nSuffix>p->nNode-p->iOff ){
         3738  +      return SQLITE_CORRUPT_VTAB;
         3739  +    }
  3733   3740       blobGrowBuffer(&p->term, nPrefix+nSuffix, &rc);
  3734   3741       if( rc==SQLITE_OK ){
  3735   3742         memcpy(&p->term.a[nPrefix], &p->aNode[p->iOff], nSuffix);
  3736   3743         p->term.n = nPrefix+nSuffix;
  3737   3744         p->iOff += nSuffix;
  3738   3745         if( p->iChild==0 ){
  3739   3746           p->iOff += fts3GetVarint32(&p->aNode[p->iOff], &p->nDoclist);
         3747  +        if( (p->nNode-p->iOff)<p->nDoclist ){
         3748  +          return SQLITE_CORRUPT_VTAB;
         3749  +        }
  3740   3750           p->aDoclist = &p->aNode[p->iOff];
  3741   3751           p->iOff += p->nDoclist;
  3742   3752         }
  3743   3753       }
  3744   3754     }
  3745   3755   
  3746   3756     assert( p->iOff<=p->nNode );
  3747         -
  3748   3757     return rc;
  3749   3758   }
  3750   3759   
  3751   3760   /*
  3752   3761   ** Release all dynamic resources held by node-reader object *p.
  3753   3762   */
  3754   3763   static void nodeReaderRelease(NodeReader *p){

Changes to ext/session/session1.test.

   650    650     set CONFLICT_HANDLERS [list REPLACE REPLACE ABORT]
   651    651     execsql ROLLBACK
   652    652     execsql BEGIN
   653    653     catch { sqlite3changeset_apply_v2 -nosavepoint db $C xConflict } msg
   654    654     execsql { SELECT * FROM x1 }
   655    655   } {1 one 2 two 3 iii}
   656    656   execsql ROLLBACK
          657  +
          658  +do_test $tn.14.1 { sqlite3session_config strm_size -1 } 64
          659  +do_test $tn.14.2 { sqlite3session_config strm_size 65536 } 65536
          660  +do_test $tn.14.3 { sqlite3session_config strm_size 64 } 64
          661  +do_test $tn.14.4 { 
          662  +  list [catch {sqlite3session_config invalid 123} msg] $msg
          663  +} {1 SQLITE_MISUSE}
   657    664   
   658    665   }]
   659    666   }
   660    667   
   661    668   
   662    669   finish_test

Changes to ext/session/session_common.tcl.

    90     90     catch { S delete }
    91     91   
    92     92     if {$rc} {
    93     93       error $changeset
    94     94     }
    95     95     return $changeset
    96     96   }
           97  +
           98  +proc patchset_from_sql {sql {dbname main}} {
           99  +  set rc [catch {
          100  +    sqlite3session S db $dbname
          101  +    db eval "SELECT name FROM $dbname.sqlite_master WHERE type = 'table'" {
          102  +      S attach $name
          103  +    }
          104  +    db eval $sql
          105  +    S patchset
          106  +  } patchset]
          107  +  catch { S delete }
          108  +
          109  +  if {$rc} {
          110  +    error $patchset
          111  +  }
          112  +  return $patchset
          113  +}
    97    114   
    98    115   proc do_then_apply_sql {sql {dbname main}} {
    99    116     proc xConflict args { return "OMIT" }
   100    117     set rc [catch {
   101    118       sqlite3session S db $dbname
   102    119       db eval "SELECT name FROM $dbname.sqlite_master WHERE type = 'table'" {
   103    120         S attach $name

Added ext/session/sessioninvert.test.

            1  +# 2018 October 18
            2  +#
            3  +# The author disclaims copyright to this source code.  In place of
            4  +# a legal notice, here is a blessing:
            5  +#
            6  +#    May you do good and not evil.
            7  +#    May you find forgiveness for yourself and forgive others.
            8  +#    May you share freely, never taking more than you give.
            9  +#
           10  +#***********************************************************************
           11  +# This file implements regression tests for SQLite library.
           12  +#
           13  +
           14  +if {![info exists testdir]} {
           15  +  set testdir [file join [file dirname [info script]] .. .. test]
           16  +} 
           17  +source [file join [file dirname [info script]] session_common.tcl]
           18  +source $testdir/tester.tcl
           19  +ifcapable !session {finish_test; return}
           20  +
           21  +set testprefix sessioninvert
           22  +
           23  +proc iter_invert {C} {
           24  +  set x [list]
           25  +  sqlite3session_foreach -invert c $C { lappend x $c }
           26  +  set x
           27  +}
           28  +
           29  +proc do_invert_test {tn sql {iter {}}} {
           30  +
           31  +  forcecopy test.db test.db2
           32  +  sqlite3 db2 test.db2
           33  +
           34  +  set C [changeset_from_sql $sql]
           35  +
           36  +  forcecopy test.db test.db3
           37  +  sqlite3 db3 test.db3
           38  +  uplevel [list do_test $tn.1 [list compare_db db db3] {}]
           39  +
           40  +  set I [sqlite3changeset_invert $C]
           41  +  sqlite3changeset_apply db $I {}
           42  +  uplevel [list do_test $tn.2 [list compare_db db db2] {}]
           43  +  
           44  +  sqlite3changeset_apply_v2 -invert db3 $C {}
           45  +  uplevel [list do_test $tn.3 [list compare_db db db3] {}]
           46  +
           47  +  if {$iter!=""} {
           48  +    uplevel [list do_test $tn.4 [list iter_invert $C] [list {*}$iter]]
           49  +  }
           50  +
           51  +  catch { db2 close }
           52  +  catch { db3 close }
           53  +}
           54  +
           55  +do_execsql_test 1.0 {
           56  +  CREATE TABLE t1(a PRIMARY KEY, b, c);
           57  +  CREATE TABLE t2(d, e, f, PRIMARY KEY(e, f));
           58  +
           59  +  INSERT INTO t1 VALUES(1, 'one', 'i');
           60  +  INSERT INTO t1 VALUES(2, 'two', 'ii');
           61  +  INSERT INTO t1 VALUES(3, 'three', 'iii');
           62  +  INSERT INTO t1 VALUES(4, 'four', 'iv');
           63  +  INSERT INTO t1 VALUES(5, 'five', 'v');
           64  +  INSERT INTO t1 VALUES(6, 'six', 'vi');
           65  +
           66  +  INSERT INTO t2 SELECT * FROM t1;
           67  +}
           68  +
           69  +do_invert_test 1.1 {
           70  +  INSERT INTO t1 VALUES(7, 'seven', 'vii');
           71  +} {
           72  +  {DELETE t1 0 X.. {i 7 t seven t vii} {}}
           73  +}
           74  +
           75  +do_invert_test 1.2 {
           76  +  DELETE FROM t1 WHERE a<4;
           77  +} {
           78  +  {INSERT t1 0 X.. {} {i 1 t one t i}}
           79  +  {INSERT t1 0 X.. {} {i 2 t two t ii}}
           80  +  {INSERT t1 0 X.. {} {i 3 t three t iii}}
           81  +}
           82  +
           83  +do_invert_test 1.3 {
           84  +  UPDATE t1 SET c=5;
           85  +} {
           86  +  {UPDATE t1 0 X.. {i 1 {} {} i 5} {{} {} {} {} t i}}
           87  +  {UPDATE t1 0 X.. {i 2 {} {} i 5} {{} {} {} {} t ii}}
           88  +  {UPDATE t1 0 X.. {i 3 {} {} i 5} {{} {} {} {} t iii}}
           89  +  {UPDATE t1 0 X.. {i 4 {} {} i 5} {{} {} {} {} t iv}}
           90  +  {UPDATE t1 0 X.. {i 5 {} {} i 5} {{} {} {} {} t v}}
           91  +  {UPDATE t1 0 X.. {i 6 {} {} i 5} {{} {} {} {} t vi}}
           92  +}
           93  +
           94  +do_invert_test 1.4 {
           95  +  UPDATE t1 SET b = a+1 WHERE a%2;
           96  +  DELETE FROM t2;
           97  +  INSERT INTO t1 VALUES(10, 'ten', NULL);
           98  +}
           99  +
          100  +do_invert_test 1.5 {
          101  +  UPDATE t2 SET d = d-1;
          102  +} {
          103  +  {UPDATE t2 0 .XX {i 2 t three t iii} {i 3 {} {} {} {}}}
          104  +  {UPDATE t2 0 .XX {i 1 t two t ii} {i 2 {} {} {} {}}}
          105  +  {UPDATE t2 0 .XX {i 5 t six t vi} {i 6 {} {} {} {}}}
          106  +  {UPDATE t2 0 .XX {i 3 t four t iv} {i 4 {} {} {} {}}}
          107  +  {UPDATE t2 0 .XX {i 0 t one t i} {i 1 {} {} {} {}}}
          108  +  {UPDATE t2 0 .XX {i 4 t five t v} {i 5 {} {} {} {}}}
          109  +}
          110  +
          111  +do_execsql_test 2.0 { 
          112  +  ANALYZE;
          113  +  PRAGMA writable_schema = 1;
          114  +  DROP TABLE IF EXISTS sqlite_stat4;
          115  +  SELECT * FROM sqlite_stat1;
          116  +} {
          117  +  t2 sqlite_autoindex_t2_1 {6 1 1} 
          118  +  t1 sqlite_autoindex_t1_1 {6 1}
          119  +}
          120  +
          121  +do_invert_test 2.1 {
          122  +  INSERT INTO sqlite_stat1 VALUES('t3', 'idx2', '1 2 3');
          123  +} {
          124  +  {DELETE sqlite_stat1 0 XX. {t t3 t idx2 t {1 2 3}} {}}
          125  +}
          126  +
          127  +do_invert_test 2.2 {
          128  +  DELETE FROM sqlite_stat1;
          129  +} {
          130  +  {INSERT sqlite_stat1 0 XX. {} {t t1 t sqlite_autoindex_t1_1 t {6 1}}}
          131  +  {INSERT sqlite_stat1 0 XX. {} {t t2 t sqlite_autoindex_t2_1 t {6 1 1}}}
          132  +}
          133  +
          134  +do_invert_test 2.3 {
          135  +  UPDATE sqlite_stat1 SET stat = 'hello world';
          136  +}
          137  +
          138  +do_test 3.0 {
          139  +  forcecopy test.db test.db2
          140  +  sqlite3 db2 test.db2
          141  +  set P [patchset_from_sql {
          142  +    INSERT INTO t2 VALUES(1, 2, 3);
          143  +    DELETE FROM t2 WHERE d = 3;
          144  +  }]
          145  +
          146  +  list [catch { sqlite3changeset_apply_v2 -invert db2 $P {} } msg] $msg
          147  +} {1 SQLITE_CORRUPT}
          148  +
          149  +do_test 3.1 {
          150  +  list [catch { sqlite3session_foreach -invert db2 $P {} } msg] $msg
          151  +} {1 SQLITE_CORRUPT}
          152  +
          153  +do_test 3.2 {
          154  +  sqlite3changeset_apply_v2 db2 $P {} 
          155  +  compare_db db db2
          156  +} {}
          157  +
          158  +
          159  +finish_test

Changes to ext/session/sqlite3session.c.

    21     21   # ifdef SQLITE_TEST
    22     22   #   define SESSIONS_STRM_CHUNK_SIZE 64
    23     23   # else
    24     24   #   define SESSIONS_STRM_CHUNK_SIZE 1024
    25     25   # endif
    26     26   #endif
    27     27   
           28  +static int sessions_strm_chunk_size = SESSIONS_STRM_CHUNK_SIZE;
           29  +
    28     30   typedef struct SessionHook SessionHook;
    29     31   struct SessionHook {
    30     32     void *pCtx;
    31     33     int (*xOld)(void*,int,sqlite3_value**);
    32     34     int (*xNew)(void*,int,sqlite3_value**);
    33     35     int (*xCount)(void*);
    34     36     int (*xDepth)(void*);
................................................................................
    83     85   /*
    84     86   ** Structure for changeset iterators.
    85     87   */
    86     88   struct sqlite3_changeset_iter {
    87     89     SessionInput in;                /* Input buffer or stream */
    88     90     SessionBuffer tblhdr;           /* Buffer to hold apValue/zTab/abPK/ */
    89     91     int bPatchset;                  /* True if this is a patchset */
           92  +  int bInvert;                    /* True to invert changeset */
    90     93     int rc;                         /* Iterator error code */
    91     94     sqlite3_stmt *pConflict;        /* Points to conflicting row, if any */
    92     95     char *zTab;                     /* Current table */
    93     96     int nCol;                       /* Number of columns in zTab */
    94     97     int op;                         /* Current operation */
    95     98     int bIndirect;                  /* True if current change was indirect */
    96     99     u8 *abPK;                       /* Primary key array */
................................................................................
   239    242   ** columns and columns that are modified by the UPDATE are set to "undefined".
   240    243   ** PRIMARY KEY fields contain the values identifying the table row to update,
   241    244   ** and fields associated with modified columns contain the new column values.
   242    245   **
   243    246   ** The records associated with INSERT changes are in the same format as for
   244    247   ** changesets. It is not possible for a record associated with an INSERT
   245    248   ** change to contain a field set to "undefined".
          249  +**
          250  +** REBASE BLOB FORMAT:
          251  +**
          252  +** A rebase blob may be output by sqlite3changeset_apply_v2() and its 
          253  +** streaming equivalent for use with the sqlite3_rebaser APIs to rebase
          254  +** existing changesets. A rebase blob contains one entry for each conflict
          255  +** resolved using either the OMIT or REPLACE strategies within the apply_v2()
          256  +** call.
          257  +**
          258  +** The format used for a rebase blob is very similar to that used for
          259  +** changesets. All entries related to a single table are grouped together.
          260  +**
          261  +** Each group of entries begins with a table header in changeset format:
          262  +**
          263  +**   1 byte: Constant 0x54 (capital 'T')
          264  +**   Varint: Number of columns in the table.
          265  +**   nCol bytes: 0x01 for PK columns, 0x00 otherwise.
          266  +**   N bytes: Unqualified table name (encoded using UTF-8). Nul-terminated.
          267  +**
          268  +** Followed by one or more entries associated with the table.
          269  +**
          270  +**   1 byte: Either SQLITE_INSERT (0x12), DELETE (0x09).
          271  +**   1 byte: Flag. 0x01 for REPLACE, 0x00 for OMIT.
          272  +**   record: (in the record format defined above).
          273  +**
          274  +** In a rebase blob, the first field is set to SQLITE_INSERT if the change
          275  +** that caused the conflict was an INSERT or UPDATE, or to SQLITE_DELETE if
          276  +** it was a DELETE. The second field is set to 0x01 if the conflict 
          277  +** resolution strategy was REPLACE, or 0x00 if it was OMIT.
          278  +**
          279  +** If the change that caused the conflict was a DELETE, then the single
          280  +** record is a copy of the old.* record from the original changeset. If it
          281  +** was an INSERT, then the single record is a copy of the new.* record. If
          282  +** the conflicting change was an UPDATE, then the single record is a copy
          283  +** of the new.* record with the PK fields filled in based on the original
          284  +** old.* record.
   246    285   */
   247    286   
   248    287   /*
   249    288   ** For each row modified during a session, there exists a single instance of
   250    289   ** this structure stored in a SessionTable.aChange[] hash table.
   251    290   */
   252    291   struct SessionChange {
................................................................................
  1789   1828   **
  1790   1829   ** If successful, return zero. Otherwise, if an OOM condition is encountered,
  1791   1830   ** set *pRc to SQLITE_NOMEM and return non-zero.
  1792   1831   */
  1793   1832   static int sessionBufferGrow(SessionBuffer *p, int nByte, int *pRc){
  1794   1833     if( *pRc==SQLITE_OK && p->nAlloc-p->nBuf<nByte ){
  1795   1834       u8 *aNew;
  1796         -    int nNew = p->nAlloc ? p->nAlloc : 128;
         1835  +    i64 nNew = p->nAlloc ? p->nAlloc : 128;
  1797   1836       do {
  1798   1837         nNew = nNew*2;
  1799         -    }while( nNew<(p->nBuf+nByte) );
         1838  +    }while( (nNew-p->nBuf)<nByte );
  1800   1839   
  1801         -    aNew = (u8 *)sqlite3_realloc(p->aBuf, nNew);
         1840  +    aNew = (u8 *)sqlite3_realloc64(p->aBuf, nNew);
  1802   1841       if( 0==aNew ){
  1803   1842         *pRc = SQLITE_NOMEM;
  1804   1843       }else{
  1805   1844         p->aBuf = aNew;
  1806   1845         p->nAlloc = nNew;
  1807   1846       }
  1808   1847     }
................................................................................
  2392   2431             }else if( p->op!=SQLITE_INSERT ){
  2393   2432               rc = sessionAppendDelete(&buf, bPatchset, p, nCol, abPK);
  2394   2433             }
  2395   2434             if( rc==SQLITE_OK ){
  2396   2435               rc = sqlite3_reset(pSel);
  2397   2436             }
  2398   2437   
  2399         -          /* If the buffer is now larger than SESSIONS_STRM_CHUNK_SIZE, pass
         2438  +          /* If the buffer is now larger than sessions_strm_chunk_size, pass
  2400   2439             ** its contents to the xOutput() callback. */
  2401   2440             if( xOutput 
  2402   2441              && rc==SQLITE_OK 
  2403   2442              && buf.nBuf>nNoop 
  2404         -           && buf.nBuf>SESSIONS_STRM_CHUNK_SIZE 
         2443  +           && buf.nBuf>sessions_strm_chunk_size 
  2405   2444             ){
  2406   2445               rc = xOutput(pOut, (void*)buf.aBuf, buf.nBuf);
  2407   2446               nNoop = -1;
  2408   2447               buf.nBuf = 0;
  2409   2448             }
  2410   2449   
  2411   2450           }
................................................................................
  2536   2575   ** Do the work for either sqlite3changeset_start() or start_strm().
  2537   2576   */
  2538   2577   static int sessionChangesetStart(
  2539   2578     sqlite3_changeset_iter **pp,    /* OUT: Changeset iterator handle */
  2540   2579     int (*xInput)(void *pIn, void *pData, int *pnData),
  2541   2580     void *pIn,
  2542   2581     int nChangeset,                 /* Size of buffer pChangeset in bytes */
  2543         -  void *pChangeset                /* Pointer to buffer containing changeset */
         2582  +  void *pChangeset,               /* Pointer to buffer containing changeset */
         2583  +  int bInvert                     /* True to invert changeset */
  2544   2584   ){
  2545   2585     sqlite3_changeset_iter *pRet;   /* Iterator to return */
  2546   2586     int nByte;                      /* Number of bytes to allocate for iterator */
  2547   2587   
  2548   2588     assert( xInput==0 || (pChangeset==0 && nChangeset==0) );
  2549   2589   
  2550   2590     /* Zero the output variable in case an error occurs. */
................................................................................
  2556   2596     if( !pRet ) return SQLITE_NOMEM;
  2557   2597     memset(pRet, 0, sizeof(sqlite3_changeset_iter));
  2558   2598     pRet->in.aData = (u8 *)pChangeset;
  2559   2599     pRet->in.nData = nChangeset;
  2560   2600     pRet->in.xInput = xInput;
  2561   2601     pRet->in.pIn = pIn;
  2562   2602     pRet->in.bEof = (xInput ? 0 : 1);
         2603  +  pRet->bInvert = bInvert;
  2563   2604   
  2564   2605     /* Populate the output variable and return success. */
  2565   2606     *pp = pRet;
  2566   2607     return SQLITE_OK;
  2567   2608   }
  2568   2609   
  2569   2610   /*
................................................................................
  2570   2611   ** Create an iterator used to iterate through the contents of a changeset.
  2571   2612   */
  2572   2613   int sqlite3changeset_start(
  2573   2614     sqlite3_changeset_iter **pp,    /* OUT: Changeset iterator handle */
  2574   2615     int nChangeset,                 /* Size of buffer pChangeset in bytes */
  2575   2616     void *pChangeset                /* Pointer to buffer containing changeset */
  2576   2617   ){
  2577         -  return sessionChangesetStart(pp, 0, 0, nChangeset, pChangeset);
         2618  +  return sessionChangesetStart(pp, 0, 0, nChangeset, pChangeset, 0);
         2619  +}
         2620  +int sqlite3changeset_start_v2(
         2621  +  sqlite3_changeset_iter **pp,    /* OUT: Changeset iterator handle */
         2622  +  int nChangeset,                 /* Size of buffer pChangeset in bytes */
         2623  +  void *pChangeset,               /* Pointer to buffer containing changeset */
         2624  +  int flags
         2625  +){
         2626  +  int bInvert = !!(flags & SQLITE_CHANGESETSTART_INVERT);
         2627  +  return sessionChangesetStart(pp, 0, 0, nChangeset, pChangeset, bInvert);
  2578   2628   }
  2579   2629   
  2580   2630   /*
  2581   2631   ** Streaming version of sqlite3changeset_start().
  2582   2632   */
  2583   2633   int sqlite3changeset_start_strm(
  2584   2634     sqlite3_changeset_iter **pp,    /* OUT: Changeset iterator handle */
  2585   2635     int (*xInput)(void *pIn, void *pData, int *pnData),
  2586   2636     void *pIn
  2587   2637   ){
  2588         -  return sessionChangesetStart(pp, xInput, pIn, 0, 0);
         2638  +  return sessionChangesetStart(pp, xInput, pIn, 0, 0, 0);
         2639  +}
         2640  +int sqlite3changeset_start_v2_strm(
         2641  +  sqlite3_changeset_iter **pp,    /* OUT: Changeset iterator handle */
         2642  +  int (*xInput)(void *pIn, void *pData, int *pnData),
         2643  +  void *pIn,
         2644  +  int flags
         2645  +){
         2646  +  int bInvert = !!(flags & SQLITE_CHANGESETSTART_INVERT);
         2647  +  return sessionChangesetStart(pp, xInput, pIn, 0, 0, bInvert);
  2589   2648   }
  2590   2649   
  2591   2650   /*
  2592   2651   ** If the SessionInput object passed as the only argument is a streaming
  2593   2652   ** object and the buffer is full, discard some data to free up space.
  2594   2653   */
  2595   2654   static void sessionDiscardData(SessionInput *pIn){
  2596         -  if( pIn->xInput && pIn->iNext>=SESSIONS_STRM_CHUNK_SIZE ){
         2655  +  if( pIn->xInput && pIn->iNext>=sessions_strm_chunk_size ){
  2597   2656       int nMove = pIn->buf.nBuf - pIn->iNext;
  2598   2657       assert( nMove>=0 );
  2599   2658       if( nMove>0 ){
  2600   2659         memmove(pIn->buf.aBuf, &pIn->buf.aBuf[pIn->iNext], nMove);
  2601   2660       }
  2602   2661       pIn->buf.nBuf -= pIn->iNext;
  2603   2662       pIn->iNext = 0;
................................................................................
  2612   2671   **
  2613   2672   ** Return an SQLite error code if an error occurs, or SQLITE_OK otherwise.
  2614   2673   */
  2615   2674   static int sessionInputBuffer(SessionInput *pIn, int nByte){
  2616   2675     int rc = SQLITE_OK;
  2617   2676     if( pIn->xInput ){
  2618   2677       while( !pIn->bEof && (pIn->iNext+nByte)>=pIn->nData && rc==SQLITE_OK ){
  2619         -      int nNew = SESSIONS_STRM_CHUNK_SIZE;
         2678  +      int nNew = sessions_strm_chunk_size;
  2620   2679   
  2621   2680         if( pIn->bNoDiscard==0 ) sessionDiscardData(pIn);
  2622   2681         if( SQLITE_OK==sessionBufferGrow(&pIn->buf, nNew, &rc) ){
  2623   2682           rc = pIn->xInput(pIn->pIn, &pIn->buf.aBuf[pIn->buf.nBuf], &nNew);
  2624   2683           if( nNew==0 ){
  2625   2684             pIn->bEof = 1;
  2626   2685           }else{
................................................................................
  2960   3019       if( sessionChangesetReadTblhdr(p) ) return p->rc;
  2961   3020       if( (p->rc = sessionInputBuffer(&p->in, 2)) ) return p->rc;
  2962   3021       p->in.iCurrent = p->in.iNext;
  2963   3022       if( p->in.iNext>=p->in.nData ) return SQLITE_DONE;
  2964   3023       op = p->in.aData[p->in.iNext++];
  2965   3024     }
  2966   3025   
  2967         -  if( p->zTab==0 ){
         3026  +  if( p->zTab==0 || (p->bPatchset && p->bInvert) ){
  2968   3027       /* The first record in the changeset is not a table header. Must be a
  2969   3028       ** corrupt changeset. */
  2970         -    assert( p->in.iNext==1 );
         3029  +    assert( p->in.iNext==1 || p->zTab );
  2971   3030       return (p->rc = SQLITE_CORRUPT_BKPT);
  2972   3031     }
  2973   3032   
  2974   3033     p->op = op;
  2975   3034     p->bIndirect = p->in.aData[p->in.iNext++];
  2976   3035     if( p->op!=SQLITE_UPDATE && p->op!=SQLITE_DELETE && p->op!=SQLITE_INSERT ){
  2977   3036       return (p->rc = SQLITE_CORRUPT_BKPT);
................................................................................
  2988   3047         nVal = p->nCol;
  2989   3048       }
  2990   3049       p->rc = sessionChangesetBufferRecord(&p->in, nVal, pnRec);
  2991   3050       if( p->rc!=SQLITE_OK ) return p->rc;
  2992   3051       *paRec = &p->in.aData[p->in.iNext];
  2993   3052       p->in.iNext += *pnRec;
  2994   3053     }else{
         3054  +    sqlite3_value **apOld = (p->bInvert ? &p->apValue[p->nCol] : p->apValue);
         3055  +    sqlite3_value **apNew = (p->bInvert ? p->apValue : &p->apValue[p->nCol]);
  2995   3056   
  2996   3057       /* If this is an UPDATE or DELETE, read the old.* record. */
  2997   3058       if( p->op!=SQLITE_INSERT && (p->bPatchset==0 || p->op==SQLITE_DELETE) ){
  2998   3059         u8 *abPK = p->bPatchset ? p->abPK : 0;
  2999         -      p->rc = sessionReadRecord(&p->in, p->nCol, abPK, p->apValue);
         3060  +      p->rc = sessionReadRecord(&p->in, p->nCol, abPK, apOld);
  3000   3061         if( p->rc!=SQLITE_OK ) return p->rc;
  3001   3062       }
  3002   3063   
  3003   3064       /* If this is an INSERT or UPDATE, read the new.* record. */
  3004   3065       if( p->op!=SQLITE_DELETE ){
  3005         -      p->rc = sessionReadRecord(&p->in, p->nCol, 0, &p->apValue[p->nCol]);
         3066  +      p->rc = sessionReadRecord(&p->in, p->nCol, 0, apNew);
  3006   3067         if( p->rc!=SQLITE_OK ) return p->rc;
  3007   3068       }
  3008   3069   
  3009         -    if( p->bPatchset && p->op==SQLITE_UPDATE ){
         3070  +    if( (p->bPatchset || p->bInvert) && p->op==SQLITE_UPDATE ){
  3010   3071         /* If this is an UPDATE that is part of a patchset, then all PK and
  3011   3072         ** modified fields are present in the new.* record. The old.* record
  3012   3073         ** is currently completely empty. This block shifts the PK fields from
  3013   3074         ** new.* to old.*, to accommodate the code that reads these arrays.  */
  3014   3075         for(i=0; i<p->nCol; i++){
  3015         -        assert( p->apValue[i]==0 );
         3076  +        assert( p->bPatchset==0 || p->apValue[i]==0 );
  3016   3077           if( p->abPK[i] ){
         3078  +          assert( p->apValue[i]==0 );
  3017   3079             p->apValue[i] = p->apValue[i+p->nCol];
  3018   3080             if( p->apValue[i]==0 ) return (p->rc = SQLITE_CORRUPT_BKPT);
  3019   3081             p->apValue[i+p->nCol] = 0;
  3020   3082           }
  3021   3083         }
         3084  +    }else if( p->bInvert ){
         3085  +      if( p->op==SQLITE_INSERT ) p->op = SQLITE_DELETE;
         3086  +      else if( p->op==SQLITE_DELETE ) p->op = SQLITE_INSERT;
  3022   3087       }
  3023   3088     }
  3024   3089   
  3025   3090     return SQLITE_ROW;
  3026   3091   }
  3027   3092   
  3028   3093   /*
................................................................................
  3331   3396   
  3332   3397         default:
  3333   3398           rc = SQLITE_CORRUPT_BKPT;
  3334   3399           goto finished_invert;
  3335   3400       }
  3336   3401   
  3337   3402       assert( rc==SQLITE_OK );
  3338         -    if( xOutput && sOut.nBuf>=SESSIONS_STRM_CHUNK_SIZE ){
         3403  +    if( xOutput && sOut.nBuf>=sessions_strm_chunk_size ){
  3339   3404         rc = xOutput(pOut, sOut.aBuf, sOut.nBuf);
  3340   3405         sOut.nBuf = 0;
  3341   3406         if( rc!=SQLITE_OK ) goto finished_invert;
  3342   3407       }
  3343   3408     }
  3344   3409   
  3345   3410     assert( rc==SQLITE_OK );
................................................................................
  3410   3475     int nCol;                       /* Size of azCol[] and abPK[] arrays */
  3411   3476     const char **azCol;             /* Array of column names */
  3412   3477     u8 *abPK;                       /* Boolean array - true if column is in PK */
  3413   3478     int bStat1;                     /* True if table is sqlite_stat1 */
  3414   3479     int bDeferConstraints;          /* True to defer constraints */
  3415   3480     SessionBuffer constraints;      /* Deferred constraints are stored here */
  3416   3481     SessionBuffer rebase;           /* Rebase information (if any) here */
  3417         -  int bRebaseStarted;             /* If table header is already in rebase */
         3482  +  u8 bRebaseStarted;              /* If table header is already in rebase */
         3483  +  u8 bRebase;                     /* True to collect rebase information */
  3418   3484   };
  3419   3485   
  3420   3486   /*
  3421   3487   ** Formulate a statement to DELETE a row from database db. Assuming a table
  3422   3488   ** structure like this:
  3423   3489   **
  3424   3490   **     CREATE TABLE x(a, b, c, d, PRIMARY KEY(a, c));
................................................................................
  3807   3873   */
  3808   3874   static int sessionRebaseAdd(
  3809   3875     SessionApplyCtx *p,             /* Apply context */
  3810   3876     int eType,                      /* Conflict resolution (OMIT or REPLACE) */
  3811   3877     sqlite3_changeset_iter *pIter   /* Iterator pointing at current change */
  3812   3878   ){
  3813   3879     int rc = SQLITE_OK;
  3814         -  int i;
  3815         -  int eOp = pIter->op;
  3816         -  if( p->bRebaseStarted==0 ){
  3817         -    /* Append a table-header to the rebase buffer */
  3818         -    const char *zTab = pIter->zTab;
  3819         -    sessionAppendByte(&p->rebase, 'T', &rc);
  3820         -    sessionAppendVarint(&p->rebase, p->nCol, &rc);
  3821         -    sessionAppendBlob(&p->rebase, p->abPK, p->nCol, &rc);
  3822         -    sessionAppendBlob(&p->rebase, (u8*)zTab, (int)strlen(zTab)+1, &rc);
  3823         -    p->bRebaseStarted = 1;
  3824         -  }
  3825         -
  3826         -  assert( eType==SQLITE_CHANGESET_REPLACE||eType==SQLITE_CHANGESET_OMIT );
  3827         -  assert( eOp==SQLITE_DELETE || eOp==SQLITE_INSERT || eOp==SQLITE_UPDATE );
  3828         -
  3829         -  sessionAppendByte(&p->rebase, 
  3830         -      (eOp==SQLITE_DELETE ? SQLITE_DELETE : SQLITE_INSERT), &rc
  3831         -  );
  3832         -  sessionAppendByte(&p->rebase, (eType==SQLITE_CHANGESET_REPLACE), &rc);
  3833         -  for(i=0; i<p->nCol; i++){
  3834         -    sqlite3_value *pVal = 0;
  3835         -    if( eOp==SQLITE_DELETE || (eOp==SQLITE_UPDATE && p->abPK[i]) ){
  3836         -      sqlite3changeset_old(pIter, i, &pVal);
  3837         -    }else{
  3838         -      sqlite3changeset_new(pIter, i, &pVal);
  3839         -    }
  3840         -    sessionAppendValue(&p->rebase, pVal, &rc);
  3841         -  }
  3842         -
         3880  +  if( p->bRebase ){
         3881  +    int i;
         3882  +    int eOp = pIter->op;
         3883  +    if( p->bRebaseStarted==0 ){
         3884  +      /* Append a table-header to the rebase buffer */
         3885  +      const char *zTab = pIter->zTab;
         3886  +      sessionAppendByte(&p->rebase, 'T', &rc);
         3887  +      sessionAppendVarint(&p->rebase, p->nCol, &rc);
         3888  +      sessionAppendBlob(&p->rebase, p->abPK, p->nCol, &rc);
         3889  +      sessionAppendBlob(&p->rebase, (u8*)zTab, (int)strlen(zTab)+1, &rc);
         3890  +      p->bRebaseStarted = 1;
         3891  +    }
         3892  +
         3893  +    assert( eType==SQLITE_CHANGESET_REPLACE||eType==SQLITE_CHANGESET_OMIT );
         3894  +    assert( eOp==SQLITE_DELETE || eOp==SQLITE_INSERT || eOp==SQLITE_UPDATE );
         3895  +
         3896  +    sessionAppendByte(&p->rebase, 
         3897  +        (eOp==SQLITE_DELETE ? SQLITE_DELETE : SQLITE_INSERT), &rc
         3898  +        );
         3899  +    sessionAppendByte(&p->rebase, (eType==SQLITE_CHANGESET_REPLACE), &rc);
         3900  +    for(i=0; i<p->nCol; i++){
         3901  +      sqlite3_value *pVal = 0;
         3902  +      if( eOp==SQLITE_DELETE || (eOp==SQLITE_UPDATE && p->abPK[i]) ){
         3903  +        sqlite3changeset_old(pIter, i, &pVal);
         3904  +      }else{
         3905  +        sqlite3changeset_new(pIter, i, &pVal);
         3906  +      }
         3907  +      sessionAppendValue(&p->rebase, pVal, &rc);
         3908  +    }
         3909  +  }
  3843   3910     return rc;
  3844   3911   }
  3845   3912   
  3846   3913   /*
  3847   3914   ** Invoke the conflict handler for the change that the changeset iterator
  3848   3915   ** currently points to.
  3849   3916   **
................................................................................
  4178   4245     int rc = SQLITE_OK;
  4179   4246   
  4180   4247     while( pApply->constraints.nBuf ){
  4181   4248       sqlite3_changeset_iter *pIter2 = 0;
  4182   4249       SessionBuffer cons = pApply->constraints;
  4183   4250       memset(&pApply->constraints, 0, sizeof(SessionBuffer));
  4184   4251   
  4185         -    rc = sessionChangesetStart(&pIter2, 0, 0, cons.nBuf, cons.aBuf);
         4252  +    rc = sessionChangesetStart(&pIter2, 0, 0, cons.nBuf, cons.aBuf, 0);
  4186   4253       if( rc==SQLITE_OK ){
  4187   4254         int nByte = 2*pApply->nCol*sizeof(sqlite3_value*);
  4188   4255         int rc2;
  4189   4256         pIter2->bPatchset = bPatchset;
  4190   4257         pIter2->zTab = (char*)zTab;
  4191   4258         pIter2->nCol = pApply->nCol;
  4192   4259         pIter2->abPK = pApply->abPK;
................................................................................
  4244   4311     SessionApplyCtx sApply;         /* changeset_apply() context object */
  4245   4312     int bPatchset;
  4246   4313   
  4247   4314     assert( xConflict!=0 );
  4248   4315   
  4249   4316     pIter->in.bNoDiscard = 1;
  4250   4317     memset(&sApply, 0, sizeof(sApply));
         4318  +  sApply.bRebase = (ppRebase && pnRebase);
  4251   4319     sqlite3_mutex_enter(sqlite3_db_mutex(db));
  4252   4320     if( (flags & SQLITE_CHANGESETAPPLY_NOSAVEPOINT)==0 ){
  4253   4321       rc = sqlite3_exec(db, "SAVEPOINT changeset_apply", 0, 0, 0);
  4254   4322     }
  4255   4323     if( rc==SQLITE_OK ){
  4256   4324       rc = sqlite3_exec(db, "PRAGMA defer_foreign_keys = 1", 0, 0, 0);
  4257   4325     }
................................................................................
  4394   4462         rc = sqlite3_exec(db, "RELEASE changeset_apply", 0, 0, 0);
  4395   4463       }else{
  4396   4464         sqlite3_exec(db, "ROLLBACK TO changeset_apply", 0, 0, 0);
  4397   4465         sqlite3_exec(db, "RELEASE changeset_apply", 0, 0, 0);
  4398   4466       }
  4399   4467     }
  4400   4468   
  4401         -  if( rc==SQLITE_OK && bPatchset==0 && ppRebase && pnRebase ){
         4469  +  assert( sApply.bRebase || sApply.rebase.nBuf==0 );
         4470  +  if( rc==SQLITE_OK && bPatchset==0 && sApply.bRebase ){
  4402   4471       *ppRebase = (void*)sApply.rebase.aBuf;
  4403   4472       *pnRebase = sApply.rebase.nBuf;
  4404   4473       sApply.rebase.aBuf = 0;
  4405   4474     }
  4406   4475     sqlite3_finalize(sApply.pInsert);
  4407   4476     sqlite3_finalize(sApply.pDelete);
  4408   4477     sqlite3_finalize(sApply.pUpdate);
................................................................................
  4432   4501       sqlite3_changeset_iter *p     /* Handle describing change and conflict */
  4433   4502     ),
  4434   4503     void *pCtx,                     /* First argument passed to xConflict */
  4435   4504     void **ppRebase, int *pnRebase,
  4436   4505     int flags
  4437   4506   ){
  4438   4507     sqlite3_changeset_iter *pIter;  /* Iterator to skip through changeset */  
  4439         -  int rc = sqlite3changeset_start(&pIter, nChangeset, pChangeset);
         4508  +  int bInverse = !!(flags & SQLITE_CHANGESETAPPLY_INVERT);
         4509  +  int rc = sessionChangesetStart(&pIter, 0, 0, nChangeset, pChangeset,bInverse);
  4440   4510     if( rc==SQLITE_OK ){
  4441   4511       rc = sessionChangesetApply(
  4442   4512           db, pIter, xFilter, xConflict, pCtx, ppRebase, pnRebase, flags
  4443   4513       );
  4444   4514     }
  4445   4515     return rc;
  4446   4516   }
................................................................................
  4489   4559       sqlite3_changeset_iter *p     /* Handle describing change and conflict */
  4490   4560     ),
  4491   4561     void *pCtx,                     /* First argument passed to xConflict */
  4492   4562     void **ppRebase, int *pnRebase,
  4493   4563     int flags
  4494   4564   ){
  4495   4565     sqlite3_changeset_iter *pIter;  /* Iterator to skip through changeset */  
  4496         -  int rc = sqlite3changeset_start_strm(&pIter, xInput, pIn);
         4566  +  int bInverse = !!(flags & SQLITE_CHANGESETAPPLY_INVERT);
         4567  +  int rc = sessionChangesetStart(&pIter, xInput, pIn, 0, 0, bInverse);
  4497   4568     if( rc==SQLITE_OK ){
  4498   4569       rc = sessionChangesetApply(
  4499   4570           db, pIter, xFilter, xConflict, pCtx, ppRebase, pnRebase, flags
  4500   4571       );
  4501   4572     }
  4502   4573     return rc;
  4503   4574   }
................................................................................
  4862   4933       sessionAppendTableHdr(&buf, pGrp->bPatch, pTab, &rc);
  4863   4934       for(i=0; i<pTab->nChange; i++){
  4864   4935         SessionChange *p;
  4865   4936         for(p=pTab->apChange[i]; p; p=p->pNext){
  4866   4937           sessionAppendByte(&buf, p->op, &rc);
  4867   4938           sessionAppendByte(&buf, p->bIndirect, &rc);
  4868   4939           sessionAppendBlob(&buf, p->aRecord, p->nRecord, &rc);
  4869         -        if( rc==SQLITE_OK && xOutput && buf.nBuf>=SESSIONS_STRM_CHUNK_SIZE ){
         4940  +        if( rc==SQLITE_OK && xOutput && buf.nBuf>=sessions_strm_chunk_size ){
  4870   4941             rc = xOutput(pOut, buf.aBuf, buf.nBuf);
  4871   4942             buf.nBuf = 0;
  4872   4943           }
  4873   4944         }
  4874   4945       }
  4875   4946     }
  4876   4947   
................................................................................
  5258   5329       }
  5259   5330   
  5260   5331       if( bDone==0 ){
  5261   5332         sessionAppendByte(&sOut, pIter->op, &rc);
  5262   5333         sessionAppendByte(&sOut, pIter->bIndirect, &rc);
  5263   5334         sessionAppendBlob(&sOut, aRec, nRec, &rc);
  5264   5335       }
  5265         -    if( rc==SQLITE_OK && xOutput && sOut.nBuf>SESSIONS_STRM_CHUNK_SIZE ){
         5336  +    if( rc==SQLITE_OK && xOutput && sOut.nBuf>sessions_strm_chunk_size ){
  5266   5337         rc = xOutput(pOut, sOut.aBuf, sOut.nBuf);
  5267   5338         sOut.nBuf = 0;
  5268   5339       }
  5269   5340       if( rc ) break;
  5270   5341     }
  5271   5342   
  5272   5343     if( rc!=SQLITE_OK ){
................................................................................
  5368   5439   */
  5369   5440   void sqlite3rebaser_delete(sqlite3_rebaser *p){
  5370   5441     if( p ){
  5371   5442       sessionDeleteTable(p->grp.pList);
  5372   5443       sqlite3_free(p);
  5373   5444     }
  5374   5445   }
         5446  +
         5447  +/* 
         5448  +** Global configuration
         5449  +*/
         5450  +int sqlite3session_config(int op, void *pArg){
         5451  +  int rc = SQLITE_OK;
         5452  +  switch( op ){
         5453  +    case SQLITE_SESSION_CONFIG_STRMSIZE: {
         5454  +      int *pInt = (int*)pArg;
         5455  +      if( *pInt>0 ){
         5456  +        sessions_strm_chunk_size = *pInt;
         5457  +      }
         5458  +      *pInt = sessions_strm_chunk_size;
         5459  +      break;
         5460  +    }
         5461  +    default:
         5462  +      rc = SQLITE_MISUSE;
         5463  +      break;
         5464  +  }
         5465  +  return rc;
         5466  +}
  5375   5467   
  5376   5468   #endif /* SQLITE_ENABLE_SESSION && SQLITE_ENABLE_PREUPDATE_HOOK */

Changes to ext/session/sqlite3session.h.

   469    469   ** [sqlite3changeset_invert()] functions, all changes within the changeset 
   470    470   ** that apply to a single table are grouped together. This means that when 
   471    471   ** an application iterates through a changeset using an iterator created by 
   472    472   ** this function, all changes that relate to a single table are visited 
   473    473   ** consecutively. There is no chance that the iterator will visit a change 
   474    474   ** the applies to table X, then one for table Y, and then later on visit 
   475    475   ** another change for table X.
          476  +**
          477  +** The behavior of sqlite3changeset_start_v2() and its streaming equivalent
          478  +** may be modified by passing a combination of
          479  +** [SQLITE_CHANGESETSTART_INVERT | supported flags] as the 4th parameter.
          480  +**
          481  +** Note that the sqlite3changeset_start_v2() API is still <b>experimental</b>
          482  +** and therefore subject to change.
   476    483   */
   477    484   int sqlite3changeset_start(
   478    485     sqlite3_changeset_iter **pp,    /* OUT: New changeset iterator handle */
   479    486     int nChangeset,                 /* Size of changeset blob in bytes */
   480    487     void *pChangeset                /* Pointer to blob containing changeset */
   481    488   );
          489  +int sqlite3changeset_start_v2(
          490  +  sqlite3_changeset_iter **pp,    /* OUT: New changeset iterator handle */
          491  +  int nChangeset,                 /* Size of changeset blob in bytes */
          492  +  void *pChangeset,               /* Pointer to blob containing changeset */
          493  +  int flags                       /* SESSION_CHANGESETSTART_* flags */
          494  +);
          495  +
          496  +/*
          497  +** CAPI3REF: Flags for sqlite3changeset_start_v2
          498  +**
          499  +** The following flags may passed via the 4th parameter to
          500  +** [sqlite3changeset_start_v2] and [sqlite3changeset_start_v2_strm]:
          501  +**
          502  +** <dt>SQLITE_CHANGESETAPPLY_INVERT <dd>
          503  +**   Invert the changeset while iterating through it. This is equivalent to
          504  +**   inverting a changeset using sqlite3changeset_invert() before applying it.
          505  +**   It is an error to specify this flag with a patchset.
          506  +*/
          507  +#define SQLITE_CHANGESETSTART_INVERT        0x0002
   482    508   
   483    509   
   484    510   /*
   485    511   ** CAPI3REF: Advance A Changeset Iterator
   486    512   ** METHOD: sqlite3_changeset_iter
   487    513   **
   488    514   ** This function may only be used with iterators created by function
................................................................................
  1129   1155     int(*xConflict)(
  1130   1156       void *pCtx,                   /* Copy of sixth arg to _apply() */
  1131   1157       int eConflict,                /* DATA, MISSING, CONFLICT, CONSTRAINT */
  1132   1158       sqlite3_changeset_iter *p     /* Handle describing change and conflict */
  1133   1159     ),
  1134   1160     void *pCtx,                     /* First argument passed to xConflict */
  1135   1161     void **ppRebase, int *pnRebase, /* OUT: Rebase data */
  1136         -  int flags                       /* Combination of SESSION_APPLY_* flags */
         1162  +  int flags                       /* SESSION_CHANGESETAPPLY_* flags */
  1137   1163   );
  1138   1164   
  1139   1165   /*
  1140   1166   ** CAPI3REF: Flags for sqlite3changeset_apply_v2
  1141   1167   **
  1142   1168   ** The following flags may passed via the 9th parameter to
  1143   1169   ** [sqlite3changeset_apply_v2] and [sqlite3changeset_apply_v2_strm]:
................................................................................
  1147   1173   **   Usually, the sessions module encloses all operations performed by
  1148   1174   **   a single call to apply_v2() or apply_v2_strm() in a [SAVEPOINT]. The
  1149   1175   **   SAVEPOINT is committed if the changeset or patchset is successfully
  1150   1176   **   applied, or rolled back if an error occurs. Specifying this flag
  1151   1177   **   causes the sessions module to omit this savepoint. In this case, if the
  1152   1178   **   caller has an open transaction or savepoint when apply_v2() is called, 
  1153   1179   **   it may revert the partially applied changeset by rolling it back.
         1180  +**
         1181  +** <dt>SQLITE_CHANGESETAPPLY_INVERT <dd>
         1182  +**   Invert the changeset before applying it. This is equivalent to inverting
         1183  +**   a changeset using sqlite3changeset_invert() before applying it. It is
         1184  +**   an error to specify this flag with a patchset.
  1154   1185   */
  1155   1186   #define SQLITE_CHANGESETAPPLY_NOSAVEPOINT   0x0001
         1187  +#define SQLITE_CHANGESETAPPLY_INVERT        0x0002
  1156   1188   
  1157   1189   /* 
  1158   1190   ** CAPI3REF: Constants Passed To The Conflict Handler
  1159   1191   **
  1160   1192   ** Values that may be passed as the second argument to a conflict-handler.
  1161   1193   **
  1162   1194   ** <dl>
................................................................................
  1541   1573     int (*xOutput)(void *pOut, const void *pData, int nData),
  1542   1574     void *pOut
  1543   1575   );
  1544   1576   int sqlite3changeset_start_strm(
  1545   1577     sqlite3_changeset_iter **pp,
  1546   1578     int (*xInput)(void *pIn, void *pData, int *pnData),
  1547   1579     void *pIn
         1580  +);
         1581  +int sqlite3changeset_start_v2_strm(
         1582  +  sqlite3_changeset_iter **pp,
         1583  +  int (*xInput)(void *pIn, void *pData, int *pnData),
         1584  +  void *pIn,
         1585  +  int flags
  1548   1586   );
  1549   1587   int sqlite3session_changeset_strm(
  1550   1588     sqlite3_session *pSession,
  1551   1589     int (*xOutput)(void *pOut, const void *pData, int nData),
  1552   1590     void *pOut
  1553   1591   );
  1554   1592   int sqlite3session_patchset_strm(
................................................................................
  1568   1606     sqlite3_rebaser *pRebaser,
  1569   1607     int (*xInput)(void *pIn, void *pData, int *pnData),
  1570   1608     void *pIn,
  1571   1609     int (*xOutput)(void *pOut, const void *pData, int nData),
  1572   1610     void *pOut
  1573   1611   );
  1574   1612   
         1613  +/*
         1614  +** CAPI3REF: Configure global parameters
         1615  +**
         1616  +** The sqlite3session_config() interface is used to make global configuration
         1617  +** changes to the sessions module in order to tune it to the specific needs 
         1618  +** of the application.
         1619  +**
         1620  +** The sqlite3session_config() interface is not threadsafe. If it is invoked
         1621  +** while any other thread is inside any other sessions method then the
         1622  +** results are undefined. Furthermore, if it is invoked after any sessions
         1623  +** related objects have been created, the results are also undefined. 
         1624  +**
         1625  +** The first argument to the sqlite3session_config() function must be one
         1626  +** of the SQLITE_SESSION_CONFIG_XXX constants defined below. The 
         1627  +** interpretation of the (void*) value passed as the second parameter and
         1628  +** the effect of calling this function depends on the value of the first
         1629  +** parameter.
         1630  +**
         1631  +** <dl>
         1632  +** <dt>SQLITE_SESSION_CONFIG_STRMSIZE<dd>
         1633  +**    By default, the sessions module streaming interfaces attempt to input
         1634  +**    and output data in approximately 1 KiB chunks. This operand may be used
         1635  +**    to set and query the value of this configuration setting. The pointer
         1636  +**    passed as the second argument must point to a value of type (int).
         1637  +**    If this value is greater than 0, it is used as the new streaming data
         1638  +**    chunk size for both input and output. Before returning, the (int) value
         1639  +**    pointed to by pArg is set to the final value of the streaming interface
         1640  +**    chunk size.
         1641  +** </dl>
         1642  +**
         1643  +** This function returns SQLITE_OK if successful, or an SQLite error code
         1644  +** otherwise.
         1645  +*/
         1646  +int sqlite3session_config(int op, void *pArg);
         1647  +
         1648  +/*
         1649  +** CAPI3REF: Values for sqlite3session_config().
         1650  +*/
         1651  +#define SQLITE_SESSION_CONFIG_STRMSIZE 1
  1575   1652   
  1576   1653   /*
  1577   1654   ** Make sure we can call this stuff from C++.
  1578   1655   */
  1579   1656   #ifdef __cplusplus
  1580   1657   }
  1581   1658   #endif
  1582   1659   
  1583   1660   #endif  /* !defined(__SQLITESESSION_H_) && defined(SQLITE_ENABLE_SESSION) */

Changes to ext/session/test_session.c.

   733    733     int nRebase = 0;
   734    734     int flags = 0;                  /* Flags for apply_v2() */
   735    735   
   736    736     memset(&sStr, 0, sizeof(sStr));
   737    737     sStr.nStream = test_tcl_integer(interp, SESSION_STREAM_TCL_VAR);
   738    738   
   739    739     /* Check for the -nosavepoint flag */
   740         -  if( bV2 && objc>1 ){
   741         -    const char *z1 = Tcl_GetString(objv[1]);
   742         -    int n = strlen(z1);
   743         -    if( n>1 && n<=12 && 0==sqlite3_strnicmp("-nosavepoint", z1, n) ){
   744         -      flags = SQLITE_CHANGESETAPPLY_NOSAVEPOINT;
   745         -      objc--;
   746         -      objv++;
          740  +  if( bV2 ){
          741  +    if( objc>1 ){
          742  +      const char *z1 = Tcl_GetString(objv[1]);
          743  +      int n = strlen(z1);
          744  +      if( n>1 && n<=12 && 0==sqlite3_strnicmp("-nosavepoint", z1, n) ){
          745  +        flags |= SQLITE_CHANGESETAPPLY_NOSAVEPOINT;
          746  +        objc--;
          747  +        objv++;
          748  +      }
          749  +    }
          750  +    if( objc>1 ){
          751  +      const char *z1 = Tcl_GetString(objv[1]);
          752  +      int n = strlen(z1);
          753  +      if( n>1 && n<=7 && 0==sqlite3_strnicmp("-invert", z1, n) ){
          754  +        flags |= SQLITE_CHANGESETAPPLY_INVERT;
          755  +        objc--;
          756  +        objv++;
          757  +      }
   747    758       }
   748    759     }
   749    760   
   750    761     if( objc!=4 && objc!=5 ){
   751    762       const char *zMsg;
   752    763       if( bV2 ){
   753         -      zMsg = "?-nosavepoint? DB CHANGESET CONFLICT-SCRIPT ?FILTER-SCRIPT?";
          764  +      zMsg = "?-nosavepoint? ?-inverse? "
          765  +        "DB CHANGESET CONFLICT-SCRIPT ?FILTER-SCRIPT?";
   754    766       }else{
   755    767         zMsg = "DB CHANGESET CONFLICT-SCRIPT ?FILTER-SCRIPT?";
   756    768       }
   757    769       Tcl_WrongNumArgs(interp, 1, objv, zMsg);
   758    770       return TCL_ERROR;
   759    771     }
   760    772     if( 0==Tcl_GetCommandInfo(interp, Tcl_GetString(objv[1]), &info) ){
................................................................................
   965    977     int nChangeset;
   966    978     sqlite3_changeset_iter *pIter;
   967    979     int rc;
   968    980     Tcl_Obj *pVarname;
   969    981     Tcl_Obj *pCS;
   970    982     Tcl_Obj *pScript;
   971    983     int isCheckNext = 0;
          984  +  int isInvert = 0;
   972    985   
   973    986     TestStreamInput sStr;
   974    987     memset(&sStr, 0, sizeof(sStr));
   975    988   
   976         -  if( objc>1 ){
          989  +  while( objc>1 ){
   977    990       char *zOpt = Tcl_GetString(objv[1]);
   978         -    isCheckNext = (strcmp(zOpt, "-next")==0);
          991  +    int nOpt = strlen(zOpt);
          992  +    if( zOpt[0]!='-' ) break;
          993  +    if( nOpt<=7 && 0==sqlite3_strnicmp(zOpt, "-invert", nOpt) ){
          994  +      isInvert = 1;
          995  +    }else
          996  +    if( nOpt<=5 && 0==sqlite3_strnicmp(zOpt, "-next", nOpt) ){
          997  +      isCheckNext = 1;
          998  +    }else{
          999  +      break;
         1000  +    }
         1001  +    objv++;
         1002  +    objc--;
   979   1003     }
   980         -  if( objc!=4+isCheckNext ){
   981         -    Tcl_WrongNumArgs(interp, 1, objv, "?-next? VARNAME CHANGESET SCRIPT");
         1004  +  if( objc!=4 ){
         1005  +    Tcl_WrongNumArgs(
         1006  +        interp, 1, objv, "?-next? ?-invert? VARNAME CHANGESET SCRIPT");
   982   1007       return TCL_ERROR;
   983   1008     }
   984   1009   
   985         -  pVarname = objv[1+isCheckNext];
   986         -  pCS = objv[2+isCheckNext];
   987         -  pScript = objv[3+isCheckNext];
         1010  +  pVarname = objv[1];
         1011  +  pCS = objv[2];
         1012  +  pScript = objv[3];
   988   1013   
   989   1014     pChangeset = (void *)Tcl_GetByteArrayFromObj(pCS, &nChangeset);
   990   1015     sStr.nStream = test_tcl_integer(interp, SESSION_STREAM_TCL_VAR);
   991         -  if( sStr.nStream==0 ){
   992         -    rc = sqlite3changeset_start(&pIter, nChangeset, pChangeset);
         1016  +  if( isInvert ){
         1017  +    int f = SQLITE_CHANGESETSTART_INVERT;
         1018  +    if( sStr.nStream==0 ){
         1019  +      rc = sqlite3changeset_start_v2(&pIter, nChangeset, pChangeset, f);
         1020  +    }else{
         1021  +      void *pCtx = (void*)&sStr;
         1022  +      sStr.aData = (unsigned char*)pChangeset;
         1023  +      sStr.nData = nChangeset;
         1024  +      rc = sqlite3changeset_start_v2_strm(&pIter, testStreamInput, pCtx, f);
         1025  +    }
   993   1026     }else{
   994         -    sStr.aData = (unsigned char*)pChangeset;
   995         -    sStr.nData = nChangeset;
   996         -    rc = sqlite3changeset_start_strm(&pIter, testStreamInput, (void*)&sStr);
         1027  +    if( sStr.nStream==0 ){
         1028  +      rc = sqlite3changeset_start(&pIter, nChangeset, pChangeset);
         1029  +    }else{
         1030  +      sStr.aData = (unsigned char*)pChangeset;
         1031  +      sStr.nData = nChangeset;
         1032  +      rc = sqlite3changeset_start_strm(&pIter, testStreamInput, (void*)&sStr);
         1033  +    }
   997   1034     }
   998   1035     if( rc!=SQLITE_OK ){
   999   1036       return test_session_error(interp, rc, 0);
  1000   1037     }
  1001   1038   
  1002   1039     while( SQLITE_ROW==sqlite3changeset_next(pIter) ){
  1003   1040       int nCol;                     /* Number of columns in table */
................................................................................
  1197   1234   
  1198   1235     Tcl_CreateObjCommand(interp, Tcl_GetString(objv[1]), test_rebaser_cmd,
  1199   1236         (ClientData)pNew, test_rebaser_del
  1200   1237     );
  1201   1238     Tcl_SetObjResult(interp, objv[1]);
  1202   1239     return TCL_OK;
  1203   1240   }
         1241  +
         1242  +/*
         1243  +** tclcmd: sqlite3rebaser_configure OP VALUE
         1244  +*/
         1245  +static int SQLITE_TCLAPI test_sqlite3session_config(
         1246  +  void * clientData,
         1247  +  Tcl_Interp *interp,
         1248  +  int objc,
         1249  +  Tcl_Obj *CONST objv[]
         1250  +){
         1251  +  struct ConfigOpt {
         1252  +    const char *zSub;
         1253  +    int op;
         1254  +  } aSub[] = {
         1255  +    { "strm_size",    SQLITE_SESSION_CONFIG_STRMSIZE },
         1256  +    { "invalid",      0 },
         1257  +    { 0 }
         1258  +  };
         1259  +  int rc;
         1260  +  int iSub;
         1261  +  int iVal;
         1262  +
         1263  +  if( objc!=3 ){
         1264  +    Tcl_WrongNumArgs(interp, 1, objv, "OP VALUE");
         1265  +    return SQLITE_ERROR;
         1266  +  }
         1267  +  rc = Tcl_GetIndexFromObjStruct(interp, 
         1268  +      objv[1], aSub, sizeof(aSub[0]), "sub-command", 0, &iSub
         1269  +  );
         1270  +  if( rc!=TCL_OK ) return rc;
         1271  +  if( Tcl_GetIntFromObj(interp, objv[2], &iVal) ) return TCL_ERROR;
         1272  +
         1273  +  rc = sqlite3session_config(aSub[iSub].op, (void*)&iVal);
         1274  +  if( rc!=SQLITE_OK ){
         1275  +    return test_session_error(interp, rc, 0);
         1276  +  }
         1277  +  Tcl_SetObjResult(interp, Tcl_NewIntObj(iVal));
         1278  +  return TCL_OK;
         1279  +}
  1204   1280   
  1205   1281   int TestSession_Init(Tcl_Interp *interp){
  1206   1282     struct Cmd {
  1207   1283       const char *zCmd;
  1208   1284       Tcl_ObjCmdProc *xProc;
  1209   1285     } aCmd[] = {
  1210   1286       { "sqlite3session", test_sqlite3session },
................................................................................
  1213   1289       { "sqlite3changeset_concat", test_sqlite3changeset_concat },
  1214   1290       { "sqlite3changeset_apply", test_sqlite3changeset_apply },
  1215   1291       { "sqlite3changeset_apply_v2", test_sqlite3changeset_apply_v2 },
  1216   1292       { "sqlite3changeset_apply_replace_all", 
  1217   1293         test_sqlite3changeset_apply_replace_all },
  1218   1294       { "sql_exec_changeset", test_sql_exec_changeset },
  1219   1295       { "sqlite3rebaser_create", test_sqlite3rebaser_create },
         1296  +    { "sqlite3session_config", test_sqlite3session_config },
  1220   1297     };
  1221   1298     int i;
  1222   1299   
  1223   1300     for(i=0; i<sizeof(aCmd)/sizeof(struct Cmd); i++){
  1224   1301       struct Cmd *p = &aCmd[i];
  1225   1302       Tcl_CreateObjCommand(interp, p->zCmd, p->xProc, 0, 0);
  1226   1303     }
  1227   1304   
  1228   1305     return TCL_OK;
  1229   1306   }
  1230   1307   
  1231   1308   #endif /* SQLITE_TEST && SQLITE_SESSION && SQLITE_PREUPDATE_HOOK */

Changes to main.mk.

    15     15   # LIBS             Extra libraries options
    16     16   #
    17     17   # OPTS             Extra compiler command-line options.
    18     18   #
    19     19   # EXE              The suffix to add to executable files.  ".exe" for windows
    20     20   #                  and "" for Unix.
    21     21   #
    22         -# TCC              C Compiler and options for use in building executables that 
           22  +# TCC              C Compiler and options for use in building executables that
    23     23   #                  will run on the target platform.  This is usually the same
    24     24   #                  as BCC, unless you are cross-compiling.
    25     25   #
    26     26   # AR               Tools used to build a static library.
    27     27   # RANLIB
    28     28   #
    29     29   # TCL_FLAGS        Extra compiler options needed for programs that use the
................................................................................
    39     39   #
    40     40   # Once the macros above are defined, the rest of this make script will
    41     41   # build the SQLite library and testing tools.
    42     42   ################################################################################
    43     43   
    44     44   # This is how we compile
    45     45   #
    46         -TCCX =  $(TCC) $(OPTS) -I. -I$(TOP)/src -I$(TOP) 
           46  +TCCX =  $(TCC) $(OPTS) -I. -I$(TOP)/src -I$(TOP)
    47     47   TCCX += -I$(TOP)/ext/rtree -I$(TOP)/ext/icu -I$(TOP)/ext/fts3
    48     48   TCCX += -I$(TOP)/ext/async -I$(TOP)/ext/userauth
    49     49   TCCX += -I$(TOP)/ext/session
    50     50   TCCX += -I$(TOP)/ext/fts5
    51     51   THREADLIB += $(LIBS)
    52     52   
    53     53   # Object files for the SQLite library.
................................................................................
   232    232     $(TOP)/ext/rtree/rtree.c \
   233    233     $(TOP)/ext/rtree/geopoly.c
   234    234   SRC += \
   235    235     $(TOP)/ext/session/sqlite3session.c \
   236    236     $(TOP)/ext/session/sqlite3session.h
   237    237   SRC += \
   238    238     $(TOP)/ext/userauth/userauth.c \
   239         -  $(TOP)/ext/userauth/sqlite3userauth.h 
          239  +  $(TOP)/ext/userauth/sqlite3userauth.h
   240    240   SRC += \
   241    241     $(TOP)/ext/rbu/sqlite3rbu.c \
   242    242     $(TOP)/ext/rbu/sqlite3rbu.h
   243    243   SRC += \
   244    244     $(TOP)/ext/misc/json1.c \
   245    245     $(TOP)/ext/misc/stmt.c
   246    246   
................................................................................
   247    247   
   248    248   # FTS5 things
   249    249   #
   250    250   FTS5_HDR = \
   251    251      $(TOP)/ext/fts5/fts5.h \
   252    252      $(TOP)/ext/fts5/fts5Int.h \
   253    253      fts5parse.h
   254         -	   
          254  +
   255    255   FTS5_SRC = \
   256    256      $(TOP)/ext/fts5/fts5_aux.c \
   257    257      $(TOP)/ext/fts5/fts5_buffer.c \
   258    258      $(TOP)/ext/fts5/fts5_main.c \
   259    259      $(TOP)/ext/fts5/fts5_config.c \
   260    260      $(TOP)/ext/fts5/fts5_expr.c \
   261    261      $(TOP)/ext/fts5/fts5_hash.c \
................................................................................
   427    427     $(TOP)/ext/fts3/fts3_aux.c \
   428    428     $(TOP)/ext/fts3/fts3_expr.c \
   429    429     $(TOP)/ext/fts3/fts3_tokenizer.c \
   430    430     $(TOP)/ext/fts3/fts3_write.c \
   431    431     $(TOP)/ext/async/sqlite3async.c \
   432    432     $(TOP)/ext/misc/stmt.c \
   433    433     $(TOP)/ext/session/sqlite3session.c \
   434         -  $(TOP)/ext/session/test_session.c 
          434  +  $(TOP)/ext/session/test_session.c
   435    435   
   436    436   # Header files used by all library source files.
   437    437   #
   438    438   HDR = \
   439    439      $(TOP)/src/btree.h \
   440    440      $(TOP)/src/btreeInt.h \
   441    441      $(TOP)/src/hash.h \
................................................................................
   480    480     $(TOP)/ext/rtree/rtree.h \
   481    481     $(TOP)/ext/rtree/geopoly.c
   482    482   EXTHDR += \
   483    483     $(TOP)/ext/icu/sqliteicu.h
   484    484   EXTHDR += \
   485    485     $(TOP)/ext/fts5/fts5Int.h  \
   486    486     fts5parse.h                \
   487         -  $(TOP)/ext/fts5/fts5.h 
          487  +  $(TOP)/ext/fts5/fts5.h
   488    488   EXTHDR += \
   489    489     $(TOP)/ext/userauth/sqlite3userauth.h
   490    490   
   491    491   # executables needed for testing
   492    492   #
   493    493   TESTPROGS = \
   494    494     testfixture$(EXE) \
................................................................................
   503    503   #
   504    504   FUZZDATA = \
   505    505     $(TOP)/test/fuzzdata1.db \
   506    506     $(TOP)/test/fuzzdata2.db \
   507    507     $(TOP)/test/fuzzdata3.db \
   508    508     $(TOP)/test/fuzzdata4.db \
   509    509     $(TOP)/test/fuzzdata5.db \
   510         -  $(TOP)/test/fuzzdata6.db
          510  +  $(TOP)/test/fuzzdata6.db \
          511  +  $(TOP)/test/fuzzdata7.db
   511    512   
   512    513   # Standard options to testfixture
   513    514   #
   514    515   TESTOPTS = --verbose=file --output=test-out.txt
   515    516   
   516    517   # Extra compiler options for various shell tools
   517    518   #
................................................................................
   796    797   	$(TCCX) -DSQLITE_CORE -c $(TOP)/ext/misc/stmt.c
   797    798   
   798    799   rtree.o:	$(TOP)/ext/rtree/rtree.c $(HDR) $(EXTHDR)
   799    800   	$(TCCX) -DSQLITE_CORE -c $(TOP)/ext/rtree/rtree.c
   800    801   
   801    802   
   802    803   
   803         -fts5parse.c:	$(TOP)/ext/fts5/fts5parse.y lemon 
          804  +fts5parse.c:	$(TOP)/ext/fts5/fts5parse.y lemon
   804    805   	cp $(TOP)/ext/fts5/fts5parse.y .
   805    806   	rm -f fts5parse.h
   806    807   	./lemon $(OPTS) fts5parse.y
   807    808   
   808    809   fts5parse.h: fts5parse.c
   809    810   
   810    811   fts5.c: $(FTS5_SRC) $(FTS5_HDR)
................................................................................
   830    831   	$(TCCX) $(TCL_FLAGS) -DTCLSH -o tclsqlite3 \
   831    832   		$(TOP)/src/tclsqlite.c libsqlite3.a $(LIBTCL) $(THREADLIB)
   832    833   
   833    834   sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/sqlite3_analyzer.c.in $(TOP)/tool/mkccode.tcl
   834    835   	tclsh $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c
   835    836   
   836    837   sqlite3_analyzer$(EXE): sqlite3_analyzer.c
   837         -	$(TCCX) $(TCL_FLAGS) sqlite3_analyzer.c -o $@ $(LIBTCL) $(THREADLIB) 
          838  +	$(TCCX) $(TCL_FLAGS) sqlite3_analyzer.c -o $@ $(LIBTCL) $(THREADLIB)
   838    839   
   839    840   sqltclsh.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/sqltclsh.tcl $(TOP)/ext/misc/appendvfs.c $(TOP)/tool/mkccode.tcl
   840    841   	tclsh $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqltclsh.c.in >sqltclsh.c
   841    842   
   842    843   sqltclsh$(EXE): sqltclsh.c
   843         -	$(TCCX) $(TCL_FLAGS) sqltclsh.c -o $@ $(LIBTCL) $(THREADLIB) 
          844  +	$(TCCX) $(TCL_FLAGS) sqltclsh.c -o $@ $(LIBTCL) $(THREADLIB)
   844    845   
   845    846   sqlite3_expert$(EXE): $(TOP)/ext/expert/sqlite3expert.h $(TOP)/ext/expert/sqlite3expert.c $(TOP)/ext/expert/expert.c sqlite3.c
   846    847   	$(TCCX) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION $(TOP)/ext/expert/sqlite3expert.c $(TOP)/ext/expert/expert.c sqlite3.c -o sqlite3_expert$(EXE) $(THREADLIB)
   847    848   
   848    849   CHECKER_DEPS =\
   849    850     $(TOP)/tool/mkccode.tcl \
   850    851     sqlite3.c \
................................................................................
   889    890   
   890    891   fts3-testfixture$(EXE): sqlite3.c fts3amal.c $(TESTSRC) $(TOP)/src/tclsqlite.c
   891    892   	$(TCCX) $(TCL_FLAGS) $(TESTFIXTURE_FLAGS)                            \
   892    893   	-DSQLITE_ENABLE_FTS3=1                                               \
   893    894   		$(TESTSRC) $(TOP)/src/tclsqlite.c sqlite3.c fts3amal.c       \
   894    895   		-o testfixture$(EXE) $(LIBTCL) $(THREADLIB)
   895    896   
          897  +coretestprogs:	$(TESTPROGS)
          898  +
          899  +testprogs:	coretestprogs srcck1$(EXE) fuzzcheck$(EXE) sessionfuzz$(EXE)
          900  +
   896    901   fulltest:	$(TESTPROGS) fuzztest
   897    902   	./testfixture$(EXE) $(TOP)/test/all.test $(TESTOPTS)
   898    903   
   899    904   soaktest:	$(TESTPROGS)
   900    905   	./testfixture$(EXE) $(TOP)/test/all.test -soak=1 $(TESTOPTS)
   901    906   
   902    907   fulltestonly:	$(TESTPROGS) fuzztest
................................................................................
   945    950   #
   946    951   smoketest:	$(TESTPROGS) fuzzcheck$(EXE)
   947    952   	./testfixture$(EXE) $(TOP)/test/main.test $(TESTOPTS)
   948    953   
   949    954   # The next two rules are used to support the "threadtest" target. Building
   950    955   # threadtest runs a few thread-safety tests that are implemented in C. This
   951    956   # target is invoked by the releasetest.tcl script.
   952         -# 
          957  +#
   953    958   THREADTEST3_SRC = $(TOP)/test/threadtest3.c    \
   954    959                     $(TOP)/test/tt3_checkpoint.c \
   955    960                     $(TOP)/test/tt3_index.c      \
   956    961                     $(TOP)/test/tt3_vacuum.c      \
   957    962                     $(TOP)/test/tt3_stress.c      \
   958    963                     $(TOP)/test/tt3_lookaside1.c
   959    964   
................................................................................
  1009   1014   	$(TCC) -o LogEst$(EXE) $(TOP)/tool/logest.c
  1010   1015   
  1011   1016   wordcount$(EXE):	$(TOP)/test/wordcount.c sqlite3.c
  1012   1017   	$(TCC) -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION -o wordcount$(EXE) \
  1013   1018   		$(TOP)/test/wordcount.c sqlite3.c
  1014   1019   
  1015   1020   speedtest1$(EXE):	$(TOP)/test/speedtest1.c sqlite3.c
  1016         -	$(TCCX) -I. $(ST_OPT) -o speedtest1$(EXE) $(TOP)/test/speedtest1.c sqlite3.c $(THREADLIB) 
         1021  +	$(TCCX) -I. $(ST_OPT) -o speedtest1$(EXE) $(TOP)/test/speedtest1.c sqlite3.c $(THREADLIB)
  1017   1022   
  1018   1023   kvtest$(EXE):	$(TOP)/test/kvtest.c sqlite3.c
  1019         -	$(TCCX) -I. $(KV_OPT) -o kvtest$(EXE) $(TOP)/test/kvtest.c sqlite3.c $(THREADLIB) 
         1024  +	$(TCCX) -I. $(KV_OPT) -o kvtest$(EXE) $(TOP)/test/kvtest.c sqlite3.c $(THREADLIB)
  1020   1025   
  1021         -rbu$(EXE): $(TOP)/ext/rbu/rbu.c $(TOP)/ext/rbu/sqlite3rbu.c sqlite3.o 
         1026  +rbu$(EXE): $(TOP)/ext/rbu/rbu.c $(TOP)/ext/rbu/sqlite3rbu.c sqlite3.o
  1022   1027   	$(TCC) -I. -o rbu$(EXE) $(TOP)/ext/rbu/rbu.c sqlite3.o \
  1023   1028   	  $(THREADLIB)
  1024   1029   
  1025   1030   loadfts: $(TOP)/tool/loadfts.c libsqlite3.a
  1026   1031   	$(TCC) $(TOP)/tool/loadfts.c libsqlite3.a -o loadfts $(THREADLIB)
  1027   1032   
  1028   1033   # This target will fail if the SQLite amalgamation contains any exported
................................................................................
  1046   1051   # Standard install and cleanup targets
  1047   1052   #
  1048   1053   install:	sqlite3 libsqlite3.a sqlite3.h
  1049   1054   	mv sqlite3 /usr/bin
  1050   1055   	mv libsqlite3.a /usr/lib
  1051   1056   	mv sqlite3.h /usr/include
  1052   1057   
  1053         -clean:	
         1058  +clean:
  1054   1059   	rm -f *.o sqlite3 sqlite3.exe libsqlite3.a sqlite3.h opcodes.*
  1055   1060   	rm -f lemon lemon.exe lempar.c parse.* sqlite*.tar.gz
  1056   1061   	rm -f mkkeywordhash mkkeywordhash.exe keywordhash.h
  1057   1062   	rm -f $(PUBLISH)
  1058   1063   	rm -f *.da *.bb *.bbg gmon.out
  1059   1064   	rm -rf tsrc target_source
  1060   1065   	rm -f testloadext.dll libtestloadext.so
................................................................................
  1074   1079   	rm -f wordcount wordcount.exe
  1075   1080   	rm -f rbu rbu.exe
  1076   1081   	rm -f srcck1 srcck1.exe
  1077   1082   	rm -f sqlite3.c sqlite3-*.c fts?amal.c tclsqlite3.c
  1078   1083   	rm -f sqlite3rc.h
  1079   1084   	rm -f shell.c sqlite3ext.h
  1080   1085   	rm -f sqlite3_analyzer sqlite3_analyzer.exe sqlite3_analyzer.c
  1081         -	rm -f sqlite3_expert sqlite3_expert.exe 
         1086  +	rm -f sqlite3_expert sqlite3_expert.exe
  1082   1087   	rm -f sqlite-*-output.vsix
  1083   1088   	rm -f mptester mptester.exe
  1084   1089   	rm -f fuzzershell fuzzershell.exe
  1085   1090   	rm -f fuzzcheck fuzzcheck.exe
  1086   1091   	rm -f sessionfuzz
  1087   1092   	rm -f sqldiff sqldiff.exe
  1088   1093   	rm -f fts5.* fts5parse.*
  1089   1094   	rm -f lsm.h lsm1.c

Changes to src/btree.c.

  3108   3108         pBt->usableSize = usableSize;
  3109   3109         pBt->pageSize = pageSize;
  3110   3110         freeTempSpace(pBt);
  3111   3111         rc = sqlite3PagerSetPagesize(pBt->pPager, &pBt->pageSize,
  3112   3112                                      pageSize-usableSize);
  3113   3113         return rc;
  3114   3114       }
  3115         -    if( (pBt->db->flags & SQLITE_WriteSchema)==0 && nPage>nPageFile ){
         3115  +    if( sqlite3WritableSchema(pBt->db)==0 && nPage>nPageFile ){
  3116   3116         rc = SQLITE_CORRUPT_BKPT;
  3117   3117         goto page1_init_failed;
  3118   3118       }
  3119   3119       /* EVIDENCE-OF: R-28312-64704 However, the usable size is not allowed to
  3120   3120       ** be less than 480. In other words, if the page size is 512, then the
  3121   3121       ** reserved space size cannot exceed 32. */
  3122   3122       if( usableSize<480 ){
................................................................................
  3582   3582     Pager *pPager = pBt->pPager;
  3583   3583     int rc;
  3584   3584   
  3585   3585     assert( eType==PTRMAP_OVERFLOW2 || eType==PTRMAP_OVERFLOW1 || 
  3586   3586         eType==PTRMAP_BTREE || eType==PTRMAP_ROOTPAGE );
  3587   3587     assert( sqlite3_mutex_held(pBt->mutex) );
  3588   3588     assert( pDbPage->pBt==pBt );
         3589  +  if( iDbPage<3 ) return SQLITE_CORRUPT_BKPT;
  3589   3590   
  3590   3591     /* Move page iDbPage from its current location to page number iFreePage */
  3591   3592     TRACE(("AUTOVACUUM: Moving %d to free page %d (ptr page %d type %d)\n", 
  3592   3593         iDbPage, iFreePage, iPtrPage, eType));
  3593   3594     rc = sqlite3PagerMovepage(pPager, pDbPage->pDbPage, iFreePage, isCommit);
  3594   3595     if( rc!=SQLITE_OK ){
  3595   3596       return rc;

Changes to src/build.c.

   352    352       return 0;
   353    353     }
   354    354   
   355    355     p = sqlite3FindTable(db, zName, zDbase);
   356    356     if( p==0 ){
   357    357       const char *zMsg = flags & LOCATE_VIEW ? "no such view" : "no such table";
   358    358   #ifndef SQLITE_OMIT_VIRTUALTABLE
   359         -    if( sqlite3FindDbName(db, zDbase)<1 ){
   360         -      /* If zName is the not the name of a table in the schema created using
   361         -      ** CREATE, then check to see if it is the name of an virtual table that
   362         -      ** can be an eponymous virtual table. */
   363         -      Module *pMod = (Module*)sqlite3HashFind(&db->aModule, zName);
   364         -      if( pMod==0 && sqlite3_strnicmp(zName, "pragma_", 7)==0 ){
   365         -        pMod = sqlite3PragmaVtabRegister(db, zName);
   366         -      }
   367         -      if( pMod && sqlite3VtabEponymousTableInit(pParse, pMod) ){
   368         -        return pMod->pEpoTab;
   369         -      }
          359  +    /* If zName is the not the name of a table in the schema created using
          360  +    ** CREATE, then check to see if it is the name of an virtual table that
          361  +    ** can be an eponymous virtual table. */
          362  +    Module *pMod = (Module*)sqlite3HashFind(&db->aModule, zName);
          363  +    if( pMod==0 && sqlite3_strnicmp(zName, "pragma_", 7)==0 ){
          364  +      pMod = sqlite3PragmaVtabRegister(db, zName);
          365  +    }
          366  +    if( pMod && sqlite3VtabEponymousTableInit(pParse, pMod) ){
          367  +      return pMod->pEpoTab;
   370    368       }
   371    369   #endif
   372    370       if( (flags & LOCATE_NOERR)==0 ){
   373    371         if( zDbase ){
   374    372           sqlite3ErrorMsg(pParse, "%s: %s.%s", zMsg, zDbase, zName);
   375    373         }else{
   376    374           sqlite3ErrorMsg(pParse, "%s: %s", zMsg, zName);
................................................................................
   630    628     }
   631    629   
   632    630     /* Delete any foreign keys attached to this table. */
   633    631     sqlite3FkDelete(db, pTable);
   634    632   
   635    633     /* Delete the Table structure itself.
   636    634     */
          635  +#ifdef SQLITE_ENABLE_NORMALIZE
          636  +  if( pTable->pColHash ){
          637  +    sqlite3HashClear(pTable->pColHash);
          638  +    sqlite3_free(pTable->pColHash);
          639  +  }
          640  +#endif
   637    641     sqlite3DeleteColumnNames(db, pTable);
   638    642     sqlite3DbFree(db, pTable->zName);
   639    643     sqlite3DbFree(db, pTable->zColAff);
   640    644     sqlite3SelectDelete(db, pTable->pSelect);
   641    645     sqlite3ExprListDelete(db, pTable->pCheck);
   642    646   #ifndef SQLITE_OMIT_VIRTUALTABLE
   643    647     sqlite3VtabClear(db, pTable);
................................................................................
   787    791       assert( db->init.iDb==0 || db->init.busy || IN_RENAME_OBJECT
   788    792                || (db->mDbFlags & DBFLAG_Vacuum)!=0);
   789    793       iDb = db->init.iDb;
   790    794       *pUnqual = pName1;
   791    795     }
   792    796     return iDb;
   793    797   }
          798  +
          799  +/*
          800  +** True if PRAGMA writable_schema is ON
          801  +*/
          802  +int sqlite3WritableSchema(sqlite3 *db){
          803  +  testcase( (db->flags&(SQLITE_WriteSchema|SQLITE_Defensive))==0 );
          804  +  testcase( (db->flags&(SQLITE_WriteSchema|SQLITE_Defensive))==
          805  +               SQLITE_WriteSchema );
          806  +  testcase( (db->flags&(SQLITE_WriteSchema|SQLITE_Defensive))==
          807  +               SQLITE_Defensive );
          808  +  testcase( (db->flags&(SQLITE_WriteSchema|SQLITE_Defensive))==
          809  +               (SQLITE_WriteSchema|SQLITE_Defensive) );
          810  +  return (db->flags&(SQLITE_WriteSchema|SQLITE_Defensive))==SQLITE_WriteSchema;
          811  +}
   794    812   
   795    813   /*
   796    814   ** This routine is used to check if the UTF-8 string zName is a legal
   797    815   ** unqualified name for a new schema object (table, index, view or
   798    816   ** trigger). All names are legal except those that begin with the string
   799    817   ** "sqlite_" (in upper, lower or mixed case). This portion of the namespace
   800    818   ** is reserved for internal use.
   801    819   */
   802    820   int sqlite3CheckObjectName(Parse *pParse, const char *zName){
   803    821     if( !pParse->db->init.busy && pParse->nested==0 
   804         -          && (pParse->db->flags & SQLITE_WriteSchema)==0
          822  +          && sqlite3WritableSchema(pParse->db)==0
   805    823             && 0==sqlite3StrNICmp(zName, "sqlite_", 7) ){
   806    824       sqlite3ErrorMsg(pParse, "object name reserved for internal use: %s", zName);
   807    825       return SQLITE_ERROR;
   808    826     }
   809    827     return SQLITE_OK;
   810    828   }
   811    829   
................................................................................
  2419   2437   ** Also write code to modify the sqlite_master table and internal schema
  2420   2438   ** if a root-page of another table is moved by the btree-layer whilst
  2421   2439   ** erasing iTable (this can happen with an auto-vacuum database).
  2422   2440   */ 
  2423   2441   static void destroyRootPage(Parse *pParse, int iTable, int iDb){
  2424   2442     Vdbe *v = sqlite3GetVdbe(pParse);
  2425   2443     int r1 = sqlite3GetTempReg(pParse);
  2426         -  assert( iTable>1 );
         2444  +  if( iTable<2 ) sqlite3ErrorMsg(pParse, "corrupt schema");
  2427   2445     sqlite3VdbeAddOp3(v, OP_Destroy, iTable, r1, iDb);
  2428   2446     sqlite3MayAbort(pParse);
  2429   2447   #ifndef SQLITE_OMIT_AUTOVACUUM
  2430   2448     /* OP_Destroy stores an in integer r1. If this integer
  2431   2449     ** is non-zero, then it is the root page number of a table moved to
  2432   2450     ** location iTable. The following code modifies the sqlite_master table to
  2433   2451     ** reflect this.

Changes to src/callback.c.

   291    291     for(p=sqlite3BuiltinFunctions.a[h]; p; p=p->u.pHash){
   292    292       if( sqlite3StrICmp(p->zName, zFunc)==0 ){
   293    293         return p;
   294    294       }
   295    295     }
   296    296     return 0;
   297    297   }
          298  +#ifdef SQLITE_ENABLE_NORMALIZE
          299  +FuncDef *sqlite3FunctionSearchN(
          300  +  int h,               /* Hash of the name */
          301  +  const char *zFunc,   /* Name of function */
          302  +  int nFunc            /* Length of the name */
          303  +){
          304  +  FuncDef *p;
          305  +  for(p=sqlite3BuiltinFunctions.a[h]; p; p=p->u.pHash){
          306  +    if( sqlite3StrNICmp(p->zName, zFunc, nFunc)==0 ){
          307  +      return p;
          308  +    }
          309  +  }
          310  +  return 0;
          311  +}
          312  +#endif /* SQLITE_ENABLE_NORMALIZE */
   298    313   
   299    314   /*
   300    315   ** Insert a new FuncDef into a FuncDefHash hash table.
   301    316   */
   302    317   void sqlite3InsertBuiltinFuncs(
   303    318     FuncDef *aDef,      /* List of global functions to be inserted */
   304    319     int nDef            /* Length of the apDef[] list */
   305    320   ){
   306    321     int i;
   307    322     for(i=0; i<nDef; i++){
   308    323       FuncDef *pOther;
   309    324       const char *zName = aDef[i].zName;
   310    325       int nName = sqlite3Strlen30(zName);
   311         -    int h = (zName[0] + nName) % SQLITE_FUNC_HASH_SZ;
          326  +    int h = SQLITE_FUNC_HASH(zName[0], nName);
   312    327       assert( zName[0]>='a' && zName[0]<='z' );
   313    328       pOther = functionSearch(h, zName);
   314    329       if( pOther ){
   315    330         assert( pOther!=&aDef[i] && pOther->pNext!=&aDef[i] );
   316    331         aDef[i].pNext = pOther->pNext;
   317    332         pOther->pNext = &aDef[i];
   318    333       }else{
................................................................................
   383    398     ** install a new function.  Whatever FuncDef structure is returned it will
   384    399     ** have fields overwritten with new information appropriate for the
   385    400     ** new function.  But the FuncDefs for built-in functions are read-only.
   386    401     ** So we must not search for built-ins when creating a new function.
   387    402     */ 
   388    403     if( !createFlag && (pBest==0 || (db->mDbFlags & DBFLAG_PreferBuiltin)!=0) ){
   389    404       bestScore = 0;
   390         -    h = (sqlite3UpperToLower[(u8)zName[0]] + nName) % SQLITE_FUNC_HASH_SZ;
          405  +    h = SQLITE_FUNC_HASH(sqlite3UpperToLower[(u8)zName[0]], nName);
   391    406       p = functionSearch(h, zName);
   392    407       while( p ){
   393    408         int score = matchQuality(p, nArg, enc);
   394    409         if( score>bestScore ){
   395    410           pBest = p;
   396    411           bestScore = score;
   397    412         }

Changes to src/ctime.c.

   230    230     "ENABLE_FTS3_TOKENIZER",
   231    231   #endif
   232    232   #if SQLITE_ENABLE_FTS4
   233    233     "ENABLE_FTS4",
   234    234   #endif
   235    235   #if SQLITE_ENABLE_FTS5
   236    236     "ENABLE_FTS5",
          237  +#endif
          238  +#if SQLITE_ENABLE_GEOPOLY
          239  +  "ENABLE_GEOPOLY",
   237    240   #endif
   238    241   #if SQLITE_ENABLE_HIDDEN_COLUMNS
   239    242     "ENABLE_HIDDEN_COLUMNS",
   240    243   #endif
   241    244   #if SQLITE_ENABLE_ICU
   242    245     "ENABLE_ICU",
   243    246   #endif
................................................................................
   260    263     "ENABLE_MEMSYS3",
   261    264   #endif
   262    265   #if SQLITE_ENABLE_MEMSYS5
   263    266     "ENABLE_MEMSYS5",
   264    267   #endif
   265    268   #if SQLITE_ENABLE_MULTIPLEX
   266    269     "ENABLE_MULTIPLEX",
          270  +#endif
          271  +#if SQLITE_ENABLE_NORMALIZE
          272  +  "ENABLE_NORMALIZE",
   267    273   #endif
   268    274   #if SQLITE_ENABLE_NULL_TRIM
   269    275     "ENABLE_NULL_TRIM",
   270    276   #endif
   271    277   #if SQLITE_ENABLE_OVERSIZE_CELL_CHECK
   272    278     "ENABLE_OVERSIZE_CELL_CHECK",
   273    279   #endif

Changes to src/dbpage.c.

   309    309     char *zErr = 0;
   310    310     const char *zSchema;
   311    311     int iDb;
   312    312     Btree *pBt;
   313    313     Pager *pPager;
   314    314     int szPage;
   315    315   
          316  +  if( pTab->db->flags & SQLITE_Defensive ){
          317  +    zErr = "read-only";
          318  +    goto update_fail;
          319  +  }
   316    320     if( argc==1 ){
   317    321       zErr = "cannot delete";
   318    322       goto update_fail;
   319    323     }
   320    324     pgno = sqlite3_value_int(argv[0]);
   321    325     if( (Pgno)sqlite3_value_int(argv[1])!=pgno ){
   322    326       zErr = "cannot insert";

Changes to src/dbstat.c.

   250    250       pCsr->iDb = pTab->iDb;
   251    251     }
   252    252   
   253    253     *ppCursor = (sqlite3_vtab_cursor *)pCsr;
   254    254     return SQLITE_OK;
   255    255   }
   256    256   
   257         -static void statClearPage(StatPage *p){
          257  +static void statClearCells(StatPage *p){
   258    258     int i;
   259    259     if( p->aCell ){
   260    260       for(i=0; i<p->nCell; i++){
   261    261         sqlite3_free(p->aCell[i].aOvfl);
   262    262       }
   263    263       sqlite3_free(p->aCell);
   264    264     }
          265  +  p->nCell = 0;
          266  +  p->aCell = 0;
          267  +}
          268  +
          269  +static void statClearPage(StatPage *p){
          270  +  statClearCells(p);
   265    271     sqlite3PagerUnref(p->pPg);
   266    272     sqlite3_free(p->zPath);
   267    273     memset(p, 0, sizeof(StatPage));
   268    274   }
   269    275   
   270    276   static void statResetCsr(StatCursor *pCsr){
   271    277     int i;
................................................................................
   320    326     int isLeaf;
   321    327     int szPage;
   322    328   
   323    329     u8 *aData = sqlite3PagerGetData(p->pPg);
   324    330     u8 *aHdr = &aData[p->iPgno==1 ? 100 : 0];
   325    331   
   326    332     p->flags = aHdr[0];
          333  +  if( p->flags==0x0A || p->flags==0x0D ){
          334  +    isLeaf = 1;
          335  +    nHdr = 8;
          336  +  }else if( p->flags==0x05 || p->flags==0x02 ){
          337  +    isLeaf = 0;
          338  +    nHdr = 12;
          339  +  }else{
          340  +    goto statPageIsCorrupt;
          341  +  }
          342  +  if( p->iPgno==1 ) nHdr += 100;
   327    343     p->nCell = get2byte(&aHdr[3]);
   328    344     p->nMxPayload = 0;
   329         -
   330         -  isLeaf = (p->flags==0x0A || p->flags==0x0D);
   331         -  nHdr = 12 - isLeaf*4 + (p->iPgno==1)*100;
          345  +  szPage = sqlite3BtreeGetPageSize(pBt);
   332    346   
   333    347     nUnused = get2byte(&aHdr[5]) - nHdr - 2*p->nCell;
   334    348     nUnused += (int)aHdr[7];
   335    349     iOff = get2byte(&aHdr[1]);
   336    350     while( iOff ){
          351  +    int iNext;
          352  +    if( iOff>=szPage ) goto statPageIsCorrupt;
   337    353       nUnused += get2byte(&aData[iOff+2]);
   338         -    iOff = get2byte(&aData[iOff]);
          354  +    iNext = get2byte(&aData[iOff]);
          355  +    if( iNext<iOff+4 && iNext>0 ) goto statPageIsCorrupt;
          356  +    iOff = iNext;
   339    357     }
   340    358     p->nUnused = nUnused;
   341    359     p->iRightChildPg = isLeaf ? 0 : sqlite3Get4byte(&aHdr[8]);
   342         -  szPage = sqlite3BtreeGetPageSize(pBt);
   343    360   
   344    361     if( p->nCell ){
   345    362       int i;                        /* Used to iterate through cells */
   346    363       int nUsable;                  /* Usable bytes per page */
   347    364   
   348    365       sqlite3BtreeEnter(pBt);
   349    366       nUsable = szPage - sqlite3BtreeGetReserveNoMutex(pBt);
................................................................................
   352    369       if( p->aCell==0 ) return SQLITE_NOMEM_BKPT;
   353    370       memset(p->aCell, 0, (p->nCell+1) * sizeof(StatCell));
   354    371   
   355    372       for(i=0; i<p->nCell; i++){
   356    373         StatCell *pCell = &p->aCell[i];
   357    374   
   358    375         iOff = get2byte(&aData[nHdr+i*2]);
          376  +      if( iOff<nHdr || iOff>=szPage ) goto statPageIsCorrupt;
   359    377         if( !isLeaf ){
   360    378           pCell->iChildPg = sqlite3Get4byte(&aData[iOff]);
   361    379           iOff += 4;
   362    380         }
   363    381         if( p->flags==0x05 ){
   364    382           /* A table interior node. nPayload==0. */
   365    383         }else{
................................................................................
   368    386           iOff += getVarint32(&aData[iOff], nPayload);
   369    387           if( p->flags==0x0D ){
   370    388             u64 dummy;
   371    389             iOff += sqlite3GetVarint(&aData[iOff], &dummy);
   372    390           }
   373    391           if( nPayload>(u32)p->nMxPayload ) p->nMxPayload = nPayload;
   374    392           getLocalPayload(nUsable, p->flags, nPayload, &nLocal);
          393  +        if( nLocal<0 ) goto statPageIsCorrupt;
   375    394           pCell->nLocal = nLocal;
   376         -        assert( nLocal>=0 );
   377    395           assert( nPayload>=(u32)nLocal );
   378    396           assert( nLocal<=(nUsable-35) );
   379    397           if( nPayload>(u32)nLocal ){
   380    398             int j;
   381    399             int nOvfl = ((nPayload - nLocal) + nUsable-4 - 1) / (nUsable - 4);
   382    400             pCell->nLastOvfl = (nPayload-nLocal) - (nOvfl-1) * (nUsable-4);
   383    401             pCell->nOvfl = nOvfl;
................................................................................
   398    416             }
   399    417           }
   400    418         }
   401    419       }
   402    420     }
   403    421   
   404    422     return SQLITE_OK;
          423  +
          424  +statPageIsCorrupt:
          425  +  p->flags = 0;
          426  +  statClearCells(p);
          427  +  return SQLITE_OK;
   405    428   }
   406    429   
   407    430   /*
   408    431   ** Populate the pCsr->iOffset and pCsr->szPage member variables. Based on
   409    432   ** the current value of pCsr->iPageno.
   410    433   */
   411    434   static void statSizeAndOffset(StatCursor *pCsr){

Changes to src/delete.c.

    59     59     **      been specified.
    60     60     **
    61     61     ** In either case leave an error message in pParse and return non-zero.
    62     62     */
    63     63     if( ( IsVirtual(pTab) 
    64     64        && sqlite3GetVTable(pParse->db, pTab)->pMod->pModule->xUpdate==0 )
    65     65      || ( (pTab->tabFlags & TF_Readonly)!=0
    66         -     && (pParse->db->flags & SQLITE_WriteSchema)==0
    67         -     && pParse->nested==0 )
           66  +     && sqlite3WritableSchema(pParse->db)==0
           67  +     && pParse->nested==0)
    68     68     ){
    69     69       sqlite3ErrorMsg(pParse, "table %s may not be modified", pTab->zName);
    70     70       return 1;
    71     71     }
    72     72   
    73     73   #ifndef SQLITE_OMIT_VIEW
    74     74     if( !viewOk && pTab->pSelect ){

Changes to src/expr.c.

  1163   1163   ** This function returns the space in bytes required to store the copy 
  1164   1164   ** of the Expr structure and a copy of the Expr.u.zToken string (if that
  1165   1165   ** string is defined.)
  1166   1166   */
  1167   1167   static int dupedExprNodeSize(Expr *p, int flags){
  1168   1168     int nByte = dupedExprStructSize(p, flags) & 0xfff;
  1169   1169     if( !ExprHasProperty(p, EP_IntValue) && p->u.zToken ){
  1170         -    nByte += sqlite3Strlen30(p->u.zToken)+1;
         1170  +    nByte += sqlite3Strlen30NN(p->u.zToken)+1;
  1171   1171     }
  1172   1172     return ROUND8(nByte);
  1173   1173   }
  1174   1174   
  1175   1175   /*
  1176   1176   ** Return the number of bytes required to create a duplicate of the 
  1177   1177   ** expression passed as the first argument. The second argument is a
................................................................................
  2145   2145   */
  2146   2146   int sqlite3IsRowid(const char *z){
  2147   2147     if( sqlite3StrICmp(z, "_ROWID_")==0 ) return 1;
  2148   2148     if( sqlite3StrICmp(z, "ROWID")==0 ) return 1;
  2149   2149     if( sqlite3StrICmp(z, "OID")==0 ) return 1;
  2150   2150     return 0;
  2151   2151   }
         2152  +#ifdef SQLITE_ENABLE_NORMALIZE
         2153  +int sqlite3IsRowidN(const char *z, int n){
         2154  +  if( sqlite3StrNICmp(z, "_ROWID_", n)==0 ) return 1;
         2155  +  if( sqlite3StrNICmp(z, "ROWID", n)==0 ) return 1;
         2156  +  if( sqlite3StrNICmp(z, "OID", n)==0 ) return 1;
         2157  +  return 0;
         2158  +}
         2159  +#endif
  2152   2160   
  2153   2161   /*
  2154   2162   ** pX is the RHS of an IN operator.  If pX is a SELECT statement 
  2155   2163   ** that can be simplified to a direct table access, then return
  2156   2164   ** a pointer to the SELECT statement.  If pX is not a SELECT statement,
  2157   2165   ** or if the SELECT statement needs to be manifested into a transient
  2158   2166   ** table, then return NULL.

Changes to src/hash.c.

    60     60       ** 0x9e3779b1 is 2654435761 which is the closest prime number to
    61     61       ** (2**32)*golden_ratio, where golden_ratio = (sqrt(5) - 1)/2. */
    62     62       h += sqlite3UpperToLower[c];
    63     63       h *= 0x9e3779b1;
    64     64     }
    65     65     return h;
    66     66   }
           67  +#ifdef SQLITE_ENABLE_NORMALIZE
           68  +static unsigned int strHashN(const char *z, int n){
           69  +  unsigned int h = 0;
           70  +  int i;
           71  +  for(i=0; i<n; i++){
           72  +    /* Knuth multiplicative hashing.  (Sorting & Searching, p. 510).
           73  +    ** 0x9e3779b1 is 2654435761 which is the closest prime number to
           74  +    ** (2**32)*golden_ratio, where golden_ratio = (sqrt(5) - 1)/2. */
           75  +    h += sqlite3UpperToLower[z[i]];
           76  +    h *= 0x9e3779b1;
           77  +  }
           78  +  return h;
           79  +}
           80  +#endif /* SQLITE_ENABLE_NORMALIZE */
    67     81   
    68     82   
    69     83   /* Link pNew element into the hash table pH.  If pEntry!=0 then also
    70     84   ** insert pNew into the pEntry hash bucket.
    71     85   */
    72     86   static void insertElement(
    73     87     Hash *pH,              /* The complete hash table */
................................................................................
   171    185       if( sqlite3StrICmp(elem->pKey,pKey)==0 ){ 
   172    186         return elem;
   173    187       }
   174    188       elem = elem->next;
   175    189     }
   176    190     return &nullElement;
   177    191   }
          192  +#ifdef SQLITE_ENABLE_NORMALIZE
          193  +static HashElem *findElementWithHashN(
          194  +  const Hash *pH,     /* The pH to be searched */
          195  +  const char *pKey,   /* The key we are searching for */
          196  +  int nKey,           /* Number of key bytes to use */
          197  +  unsigned int *pHash /* Write the hash value here */
          198  +){
          199  +  HashElem *elem;                /* Used to loop thru the element list */
          200  +  int count;                     /* Number of elements left to test */
          201  +  unsigned int h;                /* The computed hash */
          202  +  static HashElem nullElement = { 0, 0, 0, 0 };
          203  +
          204  +  if( pH->ht ){   /*OPTIMIZATION-IF-TRUE*/
          205  +    struct _ht *pEntry;
          206  +    h = strHashN(pKey, nKey) % pH->htsize;
          207  +    pEntry = &pH->ht[h];
          208  +    elem = pEntry->chain;
          209  +    count = pEntry->count;
          210  +  }else{
          211  +    h = 0;
          212  +    elem = pH->first;
          213  +    count = pH->count;
          214  +  }
          215  +  if( pHash ) *pHash = h;
          216  +  while( count-- ){
          217  +    assert( elem!=0 );
          218  +    if( sqlite3StrNICmp(elem->pKey,pKey,nKey)==0 ){ 
          219  +      return elem;
          220  +    }
          221  +    elem = elem->next;
          222  +  }
          223  +  return &nullElement;
          224  +}
          225  +#endif /* SQLITE_ENABLE_NORMALIZE */
   178    226   
   179    227   /* Remove a single entry from the hash table given a pointer to that
   180    228   ** element and a hash on the element's key.
   181    229   */
   182    230   static void removeElementGivenHash(
   183    231     Hash *pH,         /* The pH containing "elem" */
   184    232     HashElem* elem,   /* The element to be removed from the pH */
................................................................................
   215    263   ** found, or NULL if there is no match.
   216    264   */
   217    265   void *sqlite3HashFind(const Hash *pH, const char *pKey){
   218    266     assert( pH!=0 );
   219    267     assert( pKey!=0 );
   220    268     return findElementWithHash(pH, pKey, 0)->data;
   221    269   }
          270  +#ifdef SQLITE_ENABLE_NORMALIZE
          271  +void *sqlite3HashFindN(const Hash *pH, const char *pKey, int nKey){
          272  +  assert( pH!=0 );
          273  +  assert( pKey!=0 );
          274  +  assert( nKey>=0 );
          275  +  return findElementWithHashN(pH, pKey, nKey, 0)->data;
          276  +}
          277  +#endif /* SQLITE_ENABLE_NORMALIZE */
   222    278   
   223    279   /* Insert an element into the hash table pH.  The key is pKey
   224    280   ** and the data is "data".
   225    281   **
   226    282   ** If no element exists with a matching key, then a new
   227    283   ** element is created and NULL is returned.
   228    284   **

Changes to src/insert.c.

   142    142         zColAff[i] = pTab->aCol[i].affinity;
   143    143       }
   144    144       do{
   145    145         zColAff[i--] = 0;
   146    146       }while( i>=0 && zColAff[i]==SQLITE_AFF_BLOB );
   147    147       pTab->zColAff = zColAff;
   148    148     }
   149         -  i = sqlite3Strlen30(zColAff);
          149  +  assert( zColAff!=0 );
          150  +  i = sqlite3Strlen30NN(zColAff);
   150    151     if( i ){
   151    152       if( iReg ){
   152    153         sqlite3VdbeAddOp4(v, OP_Affinity, iReg, i, 0, zColAff, i);
   153    154       }else{
   154    155         sqlite3VdbeChangeP4(v, -1, zColAff, i);
   155    156       }
   156    157     }

Changes to src/loadext.c.

   447    447     sqlite3_str_appendall,
   448    448     sqlite3_str_appendchar,
   449    449     sqlite3_str_reset,
   450    450     sqlite3_str_errcode,
   451    451     sqlite3_str_length,
   452    452     sqlite3_str_value,
   453    453     /* Version 3.25.0 and later */
   454         -  sqlite3_create_window_function
          454  +  sqlite3_create_window_function,
          455  +  /* Version 3.26.0 and later */
          456  +#ifdef SQLITE_ENABLE_NORMALIZE
          457  +  sqlite3_normalized_sql
          458  +#else
          459  +  0
          460  +#endif
   455    461   };
   456    462   
   457    463   /*
   458    464   ** Attempt to load an SQLite extension library contained in the file
   459    465   ** zFile.  The entry point is zProc.  zProc may be 0 in which case a
   460    466   ** default entry point name (sqlite3_extension_init) is used.  Use
   461    467   ** of the default name is recommended.

Changes to src/main.c.

   831    831           { SQLITE_DBCONFIG_ENABLE_TRIGGER,        SQLITE_EnableTrigger  },
   832    832           { SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER, SQLITE_Fts3Tokenizer  },
   833    833           { SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION, SQLITE_LoadExtension  },
   834    834           { SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE,      SQLITE_NoCkptOnClose  },
   835    835           { SQLITE_DBCONFIG_ENABLE_QPSG,           SQLITE_EnableQPSG     },
   836    836           { SQLITE_DBCONFIG_TRIGGER_EQP,           SQLITE_TriggerEQP     },
   837    837           { SQLITE_DBCONFIG_RESET_DATABASE,        SQLITE_ResetDatabase  },
          838  +        { SQLITE_DBCONFIG_DEFENSIVE,             SQLITE_Defensive      },
   838    839         };
   839    840         unsigned int i;
   840    841         rc = SQLITE_ERROR; /* IMP: R-42790-23372 */
   841    842         for(i=0; i<ArraySize(aFlagOp); i++){
   842    843           if( aFlagOp[i].op==op ){
   843    844             int onoff = va_arg(ap, int);
   844    845             int *pRes = va_arg(ap, int*);

Changes to src/os_unix.c.

  4431   4431       rc = SQLITE_IOERR_LOCK;
  4432   4432     }else if( lock.l_type==F_UNLCK ){
  4433   4433       if( pShmNode->isReadonly ){
  4434   4434         pShmNode->isUnlocked = 1;
  4435   4435         rc = SQLITE_READONLY_CANTINIT;
  4436   4436       }else{
  4437   4437         rc = unixShmSystemLock(pDbFd, F_WRLCK, UNIX_SHM_DMS, 1);
  4438         -      if( rc==SQLITE_OK && robust_ftruncate(pShmNode->hShm, 0) ){
         4438  +      /* The first connection to attach must truncate the -shm file.  We
         4439  +      ** truncate to 3 bytes (an arbitrary small number, less than the
         4440  +      ** -shm header size) rather than 0 as a system debugging aid, to
         4441  +      ** help detect if a -shm file truncation is legitimate or is the work
         4442  +      ** or a rogue process. */
         4443  +      if( rc==SQLITE_OK && robust_ftruncate(pShmNode->hShm, 3) ){
  4439   4444           rc = unixLogError(SQLITE_IOERR_SHMOPEN,"ftruncate",pShmNode->zFilename);
  4440   4445         }
  4441   4446       }
  4442   4447     }else if( lock.l_type==F_WRLCK ){
  4443   4448       rc = SQLITE_BUSY;
  4444   4449     }
  4445   4450   
................................................................................
  4730   4735               MAP_SHARED, pShmNode->hShm, szRegion*(i64)pShmNode->nRegion
  4731   4736           );
  4732   4737           if( pMem==MAP_FAILED ){
  4733   4738             rc = unixLogError(SQLITE_IOERR_SHMMAP, "mmap", pShmNode->zFilename);
  4734   4739             goto shmpage_out;
  4735   4740           }
  4736   4741         }else{
  4737         -        pMem = sqlite3_malloc64(szRegion);
         4742  +        pMem = sqlite3_malloc64(nMap);
  4738   4743           if( pMem==0 ){
  4739   4744             rc = SQLITE_NOMEM_BKPT;
  4740   4745             goto shmpage_out;
  4741   4746           }
  4742         -        memset(pMem, 0, szRegion);
         4747  +        memset(pMem, 0, nMap);
  4743   4748         }
  4744   4749   
  4745   4750         for(i=0; i<nShmPerMap; i++){
  4746   4751           pShmNode->apRegion[pShmNode->nRegion+i] = &((char*)pMem)[szRegion*i];
  4747   4752         }
  4748   4753         pShmNode->nRegion += nShmPerMap;
  4749   4754       }

Changes to src/pager.c.

  7017   7017   void sqlite3PagerSetCodec(
  7018   7018     Pager *pPager,
  7019   7019     void *(*xCodec)(void*,void*,Pgno,int),
  7020   7020     void (*xCodecSizeChng)(void*,int,int),
  7021   7021     void (*xCodecFree)(void*),
  7022   7022     void *pCodec
  7023   7023   ){
         7024  +  pager_reset(pPager);
  7024   7025     if( pPager->xCodecFree ) pPager->xCodecFree(pPager->pCodec);
  7025   7026     pPager->xCodec = pPager->memDb ? 0 : xCodec;
  7026   7027     pPager->xCodecSizeChng = xCodecSizeChng;
  7027   7028     pPager->xCodecFree = xCodecFree;
  7028   7029     pPager->pCodec = pCodec;
  7029   7030     setGetterMethod(pPager);
  7030   7031     pagerReportSize(pPager);

Changes to src/pragma.c.

  1033   1033   
  1034   1034   #ifndef SQLITE_OMIT_FLAG_PRAGMAS
  1035   1035     case PragTyp_FLAG: {
  1036   1036       if( zRight==0 ){
  1037   1037         setPragmaResultColumnNames(v, pPragma);
  1038   1038         returnSingleInt(v, (db->flags & pPragma->iArg)!=0 );
  1039   1039       }else{
  1040         -      int mask = pPragma->iArg;    /* Mask of bits to set or clear. */
         1040  +      u64 mask = pPragma->iArg;    /* Mask of bits to set or clear. */
  1041   1041         if( db->autoCommit==0 ){
  1042   1042           /* Foreign key support may not be enabled or disabled while not
  1043   1043           ** in auto-commit mode.  */
  1044   1044           mask &= ~(SQLITE_ForeignKeys);
  1045   1045         }
  1046   1046   #if SQLITE_USER_AUTHENTICATION
  1047   1047         if( db->auth.authLevel==UAUTH_User ){
................................................................................
  2114   2114         sqlite3VdbeMultiLoad(v, 1, "ss", db->aDb[i].zDbSName, zState);
  2115   2115       }
  2116   2116       break;
  2117   2117     }
  2118   2118   #endif
  2119   2119   
  2120   2120   #ifdef SQLITE_HAS_CODEC
         2121  +  /* Pragma        iArg
         2122  +  ** ----------   ------
         2123  +  **  key           0
         2124  +  **  rekey         1
         2125  +  **  hexkey        2
         2126  +  **  hexrekey      3
         2127  +  **  textkey       4
         2128  +  **  textrekey     5
         2129  +  */
  2121   2130     case PragTyp_KEY: {
  2122         -    if( zRight ) sqlite3_key_v2(db, zDb, zRight, sqlite3Strlen30(zRight));
  2123         -    break;
  2124         -  }
  2125         -  case PragTyp_REKEY: {
  2126         -    if( zRight ) sqlite3_rekey_v2(db, zDb, zRight, sqlite3Strlen30(zRight));
         2131  +    if( zRight ){
         2132  +      int n = pPragma->iArg<4 ? sqlite3Strlen30(zRight) : -1;
         2133  +      if( (pPragma->iArg & 1)==0 ){
         2134  +        sqlite3_key_v2(db, zDb, zRight, n);
         2135  +      }else{
         2136  +        sqlite3_rekey_v2(db, zDb, zRight, n);
         2137  +      }
         2138  +    }
  2127   2139       break;
  2128   2140     }
  2129   2141     case PragTyp_HEXKEY: {
  2130   2142       if( zRight ){
  2131   2143         u8 iByte;
  2132   2144         int i;
  2133   2145         char zKey[40];
  2134   2146         for(i=0, iByte=0; i<sizeof(zKey)*2 && sqlite3Isxdigit(zRight[i]); i++){
  2135   2147           iByte = (iByte<<4) + sqlite3HexToInt(zRight[i]);
  2136   2148           if( (i&1)!=0 ) zKey[i/2] = iByte;
  2137   2149         }
  2138         -      if( (zLeft[3] & 0xf)==0xb ){
         2150  +      if( (pPragma->iArg & 1)==0 ){
  2139   2151           sqlite3_key_v2(db, zDb, zKey, i/2);
  2140   2152         }else{
  2141   2153           sqlite3_rekey_v2(db, zDb, zKey, i/2);
  2142   2154         }
  2143   2155       }
  2144   2156       break;
  2145   2157     }

Changes to src/pragma.h.

    44     44   #define PragTyp_TEMP_STORE_DIRECTORY          36
    45     45   #define PragTyp_THREADS                       37
    46     46   #define PragTyp_WAL_AUTOCHECKPOINT            38
    47     47   #define PragTyp_WAL_CHECKPOINT                39
    48     48   #define PragTyp_ACTIVATE_EXTENSIONS           40
    49     49   #define PragTyp_HEXKEY                        41
    50     50   #define PragTyp_KEY                           42
    51         -#define PragTyp_REKEY                         43
    52         -#define PragTyp_LOCK_STATUS                   44
    53         -#define PragTyp_PARSER_TRACE                  45
    54         -#define PragTyp_STATS                         46
           51  +#define PragTyp_LOCK_STATUS                   43
           52  +#define PragTyp_PARSER_TRACE                  44
           53  +#define PragTyp_STATS                         45
    55     54   
    56     55   /* Property flags associated with various pragma. */
    57     56   #define PragFlg_NeedSchema 0x01 /* Force schema load before running */
    58     57   #define PragFlg_NoColumns  0x02 /* OP_ResultRow called with zero columns */
    59     58   #define PragFlg_NoColumns1 0x04 /* zero columns if RHS argument is present */
    60     59   #define PragFlg_ReadOnly   0x08 /* Read-only HEADER_VALUE */
    61     60   #define PragFlg_Result0    0x10 /* Acts as query when no argument */
................................................................................
   124    123   /* Definitions of all built-in pragmas */
   125    124   typedef struct PragmaName {
   126    125     const char *const zName; /* Name of pragma */
   127    126     u8 ePragTyp;             /* PragTyp_XXX value */
   128    127     u8 mPragFlg;             /* Zero or more PragFlg_XXX values */
   129    128     u8 iPragCName;           /* Start of column names in pragCName[] */
   130    129     u8 nPragCName;           /* Num of col names. 0 means use pragma name */
   131         -  u32 iArg;                /* Extra argument */
          130  +  u64 iArg;                /* Extra argument */
   132    131   } PragmaName;
   133    132   static const PragmaName aPragmaName[] = {
   134    133   #if defined(SQLITE_HAS_CODEC) || defined(SQLITE_ENABLE_CEROD)
   135    134    {/* zName:     */ "activate_extensions",
   136    135     /* ePragTyp:  */ PragTyp_ACTIVATE_EXTENSIONS,
   137    136     /* ePragFlg:  */ 0,
   138    137     /* ColNames:  */ 0, 0,
................................................................................
   321    320   #endif
   322    321   #endif
   323    322   #if defined(SQLITE_HAS_CODEC)
   324    323    {/* zName:     */ "hexkey",
   325    324     /* ePragTyp:  */ PragTyp_HEXKEY,
   326    325     /* ePragFlg:  */ 0,
   327    326     /* ColNames:  */ 0, 0,
   328         -  /* iArg:      */ 0 },
          327  +  /* iArg:      */ 2 },
   329    328    {/* zName:     */ "hexrekey",
   330    329     /* ePragTyp:  */ PragTyp_HEXKEY,
   331    330     /* ePragFlg:  */ 0,
   332    331     /* ColNames:  */ 0, 0,
   333         -  /* iArg:      */ 0 },
          332  +  /* iArg:      */ 3 },
   334    333   #endif
   335    334   #if !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   336    335   #if !defined(SQLITE_OMIT_CHECK)
   337    336    {/* zName:     */ "ignore_check_constraints",
   338    337     /* ePragTyp:  */ PragTyp_FLAG,
   339    338     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,
   340    339     /* ColNames:  */ 0, 0,
................................................................................
   500    499     /* ePragTyp:  */ PragTyp_FLAG,
   501    500     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,
   502    501     /* ColNames:  */ 0, 0,
   503    502     /* iArg:      */ SQLITE_RecTriggers },
   504    503   #endif
   505    504   #if defined(SQLITE_HAS_CODEC)
   506    505    {/* zName:     */ "rekey",
   507         -  /* ePragTyp:  */ PragTyp_REKEY,
          506  +  /* ePragTyp:  */ PragTyp_KEY,
   508    507     /* ePragFlg:  */ 0,
   509    508     /* ColNames:  */ 0, 0,
   510         -  /* iArg:      */ 0 },
          509  +  /* iArg:      */ 1 },
   511    510   #endif
   512    511   #if !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   513    512    {/* zName:     */ "reverse_unordered_selects",
   514    513     /* ePragTyp:  */ PragTyp_FLAG,
   515    514     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,
   516    515     /* ColNames:  */ 0, 0,
   517    516     /* iArg:      */ SQLITE_ReverseOrder },
................................................................................
   589    588     /* ColNames:  */ 0, 0,
   590    589     /* iArg:      */ 0 },
   591    590    {/* zName:     */ "temp_store_directory",
   592    591     /* ePragTyp:  */ PragTyp_TEMP_STORE_DIRECTORY,
   593    592     /* ePragFlg:  */ PragFlg_NoColumns1,
   594    593     /* ColNames:  */ 0, 0,
   595    594     /* iArg:      */ 0 },
          595  +#endif
          596  +#if defined(SQLITE_HAS_CODEC)
          597  + {/* zName:     */ "textkey",
          598  +  /* ePragTyp:  */ PragTyp_KEY,
          599  +  /* ePragFlg:  */ 0,
          600  +  /* ColNames:  */ 0, 0,
          601  +  /* iArg:      */ 4 },
          602  + {/* zName:     */ "textrekey",
          603  +  /* ePragTyp:  */ PragTyp_KEY,
          604  +  /* ePragFlg:  */ 0,
          605  +  /* ColNames:  */ 0, 0,
          606  +  /* iArg:      */ 5 },
   596    607   #endif
   597    608    {/* zName:     */ "threads",
   598    609     /* ePragTyp:  */ PragTyp_THREADS,
   599    610     /* ePragFlg:  */ PragFlg_Result0,
   600    611     /* ColNames:  */ 0, 0,
   601    612     /* iArg:      */ 0 },
   602    613   #if !defined(SQLITE_OMIT_SCHEMA_VERSION_PRAGMAS)
................................................................................
   648    659     /* iArg:      */ 0 },
   649    660   #endif
   650    661   #if !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   651    662    {/* zName:     */ "writable_schema",
   652    663     /* ePragTyp:  */ PragTyp_FLAG,
   653    664     /* ePragFlg:  */ PragFlg_Result0|PragFlg_NoColumns1,
   654    665     /* ColNames:  */ 0, 0,
   655         -  /* iArg:      */ SQLITE_WriteSchema },
          666  +  /* iArg:      */ SQLITE_WriteSchema|SQLITE_NoSchemaError },
   656    667   #endif
   657    668   };
   658         -/* Number of pragmas: 62 on by default, 79 total. */
          669  +/* Number of pragmas: 62 on by default, 81 total. */

Changes to src/prepare.c.

   353    353       }
   354    354   #endif
   355    355     }
   356    356     if( db->mallocFailed ){
   357    357       rc = SQLITE_NOMEM_BKPT;
   358    358       sqlite3ResetAllSchemasOfConnection(db);
   359    359     }
   360         -  if( rc==SQLITE_OK || (db->flags&SQLITE_WriteSchema)){
   361         -    /* Black magic: If the SQLITE_WriteSchema flag is set, then consider
          360  +  if( rc==SQLITE_OK || (db->flags&SQLITE_NoSchemaError)){
          361  +    /* Black magic: If the SQLITE_NoSchemaError flag is set, then consider
   362    362       ** the schema loaded, even if errors occurred. In this situation the 
   363    363       ** current sqlite3_prepare() operation will fail, but the following one
   364    364       ** will attempt to compile the supplied statement against whatever subset
   365    365       ** of the schema was loaded before the error occurred. The primary
   366    366       ** purpose of this is to allow access to the sqlite_master table
   367    367       ** even when its contents have been corrupted.
   368    368       */
................................................................................
   739    739     sqlite3BtreeLeaveAll(db);
   740    740     rc = sqlite3ApiExit(db, rc);
   741    741     assert( (rc&db->errMask)==rc );
   742    742     sqlite3_mutex_leave(db->mutex);
   743    743     return rc;
   744    744   }
   745    745   
          746  +#ifdef SQLITE_ENABLE_NORMALIZE
          747  +/*
          748  +** Checks if the specified token is a table, column, or function name,
          749  +** based on the databases associated with the statement being prepared.
          750  +** If the function fails, zero is returned and pRc is filled with the
          751  +** error code.
          752  +*/
          753  +static int shouldTreatAsIdentifier(
          754  +  sqlite3 *db,        /* Database handle. */
          755  +  const char *zToken, /* Pointer to start of token to be checked */
          756  +  int nToken,         /* Length of token to be checked */
          757  +  int *pRc            /* Pointer to error code upon failure */
          758  +){
          759  +  int bFound = 0;     /* Non-zero if token is an identifier name. */
          760  +  int i, j;           /* Database and column loop indexes. */
          761  +  Schema *pSchema;    /* Schema for current database. */
          762  +  Hash *pHash;        /* Hash table of tables for current database. */
          763  +  HashElem *e;        /* Hash element for hash table iteration. */
          764  +  Table *pTab;        /* Database table for columns being checked. */
          765  +
          766  +  if( sqlite3IsRowidN(zToken, nToken) ){
          767  +    return 1;
          768  +  }
          769  +  if( nToken>0 ){
          770  +    int hash = SQLITE_FUNC_HASH(sqlite3UpperToLower[(u8)zToken[0]], nToken);
          771  +    if( sqlite3FunctionSearchN(hash, zToken, nToken) ) return 1;
          772  +  }
          773  +  assert( db!=0 );
          774  +  sqlite3_mutex_enter(db->mutex);
          775  +  sqlite3BtreeEnterAll(db);
          776  +  for(i=0; i<db->nDb; i++){
          777  +    pHash = &db->aFunc;
          778  +    if( sqlite3HashFindN(pHash, zToken, nToken) ){
          779  +      bFound = 1;
          780  +      break;
          781  +    }
          782  +    pSchema = db->aDb[i].pSchema;
          783  +    if( pSchema==0 ) continue;
          784  +    pHash = &pSchema->tblHash;
          785  +    if( sqlite3HashFindN(pHash, zToken, nToken) ){
          786  +      bFound = 1;
          787  +      break;
          788  +    }
          789  +    for(e=sqliteHashFirst(pHash); e; e=sqliteHashNext(e)){
          790  +      pTab = sqliteHashData(e);
          791  +      if( pTab==0 ) continue;
          792  +      pHash = pTab->pColHash;
          793  +      if( pHash==0 ){
          794  +        pTab->pColHash = pHash = sqlite3_malloc(sizeof(Hash));
          795  +        if( pHash ){
          796  +          sqlite3HashInit(pHash);
          797  +          for(j=0; j<pTab->nCol; j++){
          798  +            Column *pCol = &pTab->aCol[j];
          799  +            sqlite3HashInsert(pHash, pCol->zName, pCol);
          800  +          }
          801  +        }else{
          802  +          *pRc = SQLITE_NOMEM_BKPT;
          803  +          bFound = 0;
          804  +          goto done;
          805  +        }
          806  +      }
          807  +      if( pHash && sqlite3HashFindN(pHash, zToken, nToken) ){
          808  +        bFound = 1;
          809  +        goto done;
          810  +      }
          811  +    }
          812  +  }
          813  +done:
          814  +  sqlite3BtreeLeaveAll(db);
          815  +  sqlite3_mutex_leave(db->mutex);
          816  +  return bFound;
          817  +}
          818  +
          819  +/*
          820  +** Attempt to estimate the final output buffer size needed for the fully
          821  +** normalized version of the specified SQL string.  This should take into
          822  +** account any potential expansion that could occur (e.g. via IN clauses
          823  +** being expanded, etc).  This size returned is the total number of bytes
          824  +** including the NUL terminator.
          825  +*/
          826  +static int estimateNormalizedSize(
          827  +  const char *zSql, /* The original SQL string */
          828  +  int nSql,         /* Length of original SQL string */
          829  +  u8 prepFlags      /* The flags passed to sqlite3_prepare_v3() */
          830  +){
          831  +  int nOut = nSql + 4;
          832  +  const char *z = zSql;
          833  +  while( nOut<nSql*5 ){
          834  +    while( z[0]!=0 && z[0]!='I' && z[0]!='i' ){ z++; }
          835  +    if( z[0]==0 ) break;
          836  +    z++;
          837  +    if( z[0]!='N' && z[0]!='n' ) break;
          838  +    z++;
          839  +    while( sqlite3Isspace(z[0]) ){ z++; }
          840  +    if( z[0]!='(' ) break;
          841  +    z++;
          842  +    nOut += 5; /* ?,?,? */
          843  +  }
          844  +  return nOut;
          845  +}
          846  +
          847  +/*
          848  +** Copy the current token into the output buffer while dealing with quoted
          849  +** identifiers.  By default, all letters will be converted into lowercase.
          850  +** If the bUpper flag is set, uppercase will be used.  The piOut argument
          851  +** will be used to update the target index into the output string.
          852  +*/
          853  +static void copyNormalizedToken(
          854  +  const char *zSql, /* The original SQL string */
          855  +  int iIn,          /* Current index into the original SQL string */
          856  +  int nToken,       /* Number of bytes in the current token */
          857  +  int tokenFlags,   /* Flags returned by the tokenizer */
          858  +  char *zOut,       /* The output string */
          859  +  int *piOut        /* Pointer to target index into the output string */
          860  +){
          861  +  int bQuoted = tokenFlags & SQLITE_TOKEN_QUOTED;
          862  +  int bKeyword = tokenFlags & SQLITE_TOKEN_KEYWORD;
          863  +  int j = *piOut, k = 0;
          864  +  for(; k<nToken; k++){
          865  +    if( bQuoted ){
          866  +      if( k==0 && iIn>0 ){
          867  +        zOut[j++] = '"';
          868  +        continue;
          869  +      }else if( k==nToken-1 ){
          870  +        zOut[j++] = '"';
          871  +        continue;
          872  +      }
          873  +    }
          874  +    if( bKeyword ){
          875  +      zOut[j++] = sqlite3Toupper(zSql[iIn+k]);
          876  +    }else{
          877  +      zOut[j++] = sqlite3Tolower(zSql[iIn+k]);
          878  +    }
          879  +  }
          880  +  *piOut = j;
          881  +}
          882  +
          883  +/*
          884  +** Perform normalization of the SQL contained in the prepared statement and
          885  +** store the result in the zNormSql field.  The schema for the associated
          886  +** databases are consulted while performing the normalization in order to
          887  +** determine if a token appears to be an identifier.  All identifiers are
          888  +** left intact in the normalized SQL and all literals are replaced with a
          889  +** single '?'.
          890  +*/
          891  +void sqlite3Normalize(
          892  +  Vdbe *pVdbe,      /* VM being reprepared */
          893  +  const char *zSql, /* The original SQL string */
          894  +  int nSql,         /* Size of the input string in bytes */
          895  +  u8 prepFlags      /* The flags passed to sqlite3_prepare_v3() */
          896  +){
          897  +  sqlite3 *db;           /* Database handle. */
          898  +  char *z;               /* The output string */
          899  +  int nZ;                /* Size of the output string in bytes */
          900  +  int i;                 /* Next character to read from zSql[] */
          901  +  int j;                 /* Next character to fill in on z[] */
          902  +  int tokenType = 0;     /* Type of the next token */
          903  +  int prevTokenType = 0; /* Type of the previous token, except spaces */
          904  +  int n;                 /* Size of the next token */
          905  +  int nParen = 0;        /* Nesting level of parenthesis */
          906  +  Hash inHash;           /* Table of parenthesis levels to output index. */
          907  +
          908  +  db = sqlite3VdbeDb(pVdbe);
          909  +  assert( db!=0 );
          910  +  assert( pVdbe->zNormSql==0 );
          911  +  if( zSql==0 ) return;
          912  +  nZ = estimateNormalizedSize(zSql, nSql, prepFlags);
          913  +  z = sqlite3DbMallocRawNN(db, nZ);
          914  +  if( z==0 ) return;
          915  +  sqlite3HashInit(&inHash);
          916  +  for(i=j=0; i<nSql && zSql[i]; i+=n){
          917  +    int flags = 0;
          918  +    if( tokenType!=TK_SPACE ) prevTokenType = tokenType;
          919  +    n = sqlite3GetTokenNormalized((unsigned char*)zSql+i, &tokenType, &flags);
          920  +    switch( tokenType ){
          921  +      case TK_SPACE: {
          922  +        break;
          923  +      }
          924  +      case TK_ILLEGAL: {
          925  +        sqlite3DbFree(db, z);
          926  +        sqlite3HashClear(&inHash);
          927  +        return;
          928  +      }
          929  +      case TK_STRING:
          930  +      case TK_INTEGER:
          931  +      case TK_FLOAT:
          932  +      case TK_VARIABLE:
          933  +      case TK_BLOB: {
          934  +        z[j++] = '?';
          935  +        break;
          936  +      }
          937  +      case TK_LP:
          938  +      case TK_RP: {
          939  +        if( tokenType==TK_LP ){
          940  +          nParen++;
          941  +          if( prevTokenType==TK_IN ){
          942  +            assert( nParen<nSql );
          943  +            sqlite3HashInsert(&inHash, zSql+nParen, SQLITE_INT_TO_PTR(j));
          944  +          }
          945  +        }else{
          946  +          int jj;
          947  +          assert( nParen<nSql );
          948  +          jj = SQLITE_PTR_TO_INT(sqlite3HashFind(&inHash, zSql+nParen));
          949  +          if( jj>0 ){
          950  +            sqlite3HashInsert(&inHash, zSql+nParen, 0);
          951  +            assert( jj+6<nZ );
          952  +            memcpy(z+jj+1, "?,?,?", 5);
          953  +            j = jj+6;
          954  +            assert( nZ-1-j>=0 );
          955  +            assert( nZ-1-j<nZ );
          956  +            memset(z+j, 0, nZ-1-j);
          957  +          }
          958  +          nParen--;
          959  +        }
          960  +        assert( nParen>=0 );
          961  +        /* Fall through */
          962  +      }
          963  +      case TK_MINUS:
          964  +      case TK_SEMI:
          965  +      case TK_PLUS:
          966  +      case TK_STAR:
          967  +      case TK_SLASH:
          968  +      case TK_REM:
          969  +      case TK_EQ:
          970  +      case TK_LE:
          971  +      case TK_NE:
          972  +      case TK_LSHIFT:
          973  +      case TK_LT:
          974  +      case TK_RSHIFT:
          975  +      case TK_GT:
          976  +      case TK_GE:
          977  +      case TK_BITOR:
          978  +      case TK_CONCAT:
          979  +      case TK_COMMA:
          980  +      case TK_BITAND:
          981  +      case TK_BITNOT:
          982  +      case TK_DOT:
          983  +      case TK_IN:
          984  +      case TK_IS:
          985  +      case TK_NOT:
          986  +      case TK_NULL:
          987  +      case TK_ID: {
          988  +        if( tokenType==TK_NULL ){
          989  +          if( prevTokenType==TK_IS || prevTokenType==TK_NOT ){
          990  +            /* NULL is a keyword in this case, not a literal value */
          991  +          }else{
          992  +            /* Here the NULL is a literal value */
          993  +            z[j++] = '?';
          994  +            break;
          995  +          }
          996  +        }
          997  +        if( j>0 && sqlite3IsIdChar(z[j-1]) && sqlite3IsIdChar(zSql[i]) ){
          998  +          z[j++] = ' ';
          999  +        }
         1000  +        if( tokenType==TK_ID ){
         1001  +          int i2 = i, n2 = n, rc = SQLITE_OK;
         1002  +          if( nParen>0 ){
         1003  +            assert( nParen<nSql );
         1004  +            sqlite3HashInsert(&inHash, zSql+nParen, 0);
         1005  +          }
         1006  +          if( flags&SQLITE_TOKEN_QUOTED ){ i2++; n2-=2; }
         1007  +          if( shouldTreatAsIdentifier(db, zSql+i2, n2, &rc)==0 ){
         1008  +            if( rc!=SQLITE_OK ){
         1009  +              sqlite3DbFree(db, z);
         1010  +              sqlite3HashClear(&inHash);
         1011  +              return;
         1012  +            }
         1013  +            if( sqlite3_keyword_check(zSql+i2, n2)==0 ){
         1014  +              z[j++] = '?';
         1015  +              break;
         1016  +            }
         1017  +          }
         1018  +        }
         1019  +        copyNormalizedToken(zSql, i, n, flags, z, &j);
         1020  +        break;
         1021  +      }
         1022  +    }
         1023  +  }
         1024  +  assert( j<nZ && "one" );
         1025  +  while( j>0 && z[j-1]==' ' ){ j--; }
         1026  +  if( j>0 && z[j-1]!=';' ){ z[j++] = ';'; }
         1027  +  z[j] = 0;
         1028  +  assert( j<nZ && "two" );
         1029  +  pVdbe->zNormSql = z;
         1030  +  sqlite3HashClear(&inHash);
         1031  +}
         1032  +#endif /* SQLITE_ENABLE_NORMALIZE */
         1033  +
   746   1034   /*
   747   1035   ** Rerun the compilation of a statement after a schema change.
   748   1036   **
   749   1037   ** If the statement is successfully recompiled, return SQLITE_OK. Otherwise,
   750   1038   ** if the statement cannot be recompiled because another connection has
   751   1039   ** locked the sqlite3_master table, return SQLITE_LOCKED. If any other error
   752   1040   ** occurs, return SQLITE_SCHEMA.

Changes to src/select.c.

  4070   4070   #endif
  4071   4071   
  4072   4072     return 1;
  4073   4073   }
  4074   4074   #endif /* !defined(SQLITE_OMIT_SUBQUERY) || !defined(SQLITE_OMIT_VIEW) */
  4075   4075   
  4076   4076   /*
  4077         -** A structure to keep track of all of the column values that fixed to
         4077  +** A structure to keep track of all of the column values that are fixed to
  4078   4078   ** a known value due to WHERE clause constraints of the form COLUMN=VALUE.
  4079   4079   */
  4080   4080   typedef struct WhereConst WhereConst;
  4081   4081   struct WhereConst {
  4082   4082     Parse *pParse;   /* Parsing context */
  4083   4083     int nConst;      /* Number for COLUMN=CONSTANT terms */
  4084   4084     int nChng;       /* Number of times a constant is propagated */
  4085   4085     Expr **apExpr;   /* [i*2] is COLUMN and [i*2+1] is VALUE */
  4086   4086   };
  4087   4087   
  4088   4088   /*
  4089         -** Add a new entry to the pConst object
         4089  +** Add a new entry to the pConst object.  Except, do not add duplicate
         4090  +** pColumn entires.
  4090   4091   */
  4091   4092   static void constInsert(
  4092         -  WhereConst *pConst,
  4093         -  Expr *pColumn,
  4094         -  Expr *pValue
         4093  +  WhereConst *pConst,      /* The WhereConst into which we are inserting */
         4094  +  Expr *pColumn,           /* The COLUMN part of the constraint */
         4095  +  Expr *pValue             /* The VALUE part of the constraint */
  4095   4096   ){
         4097  +  int i;
         4098  +  assert( pColumn->op==TK_COLUMN );
         4099  +
         4100  +  /* 2018-10-25 ticket [cf5ed20f]
         4101  +  ** Make sure the same pColumn is not inserted more than once */
         4102  +  for(i=0; i<pConst->nConst; i++){
         4103  +    const Expr *pExpr = pConst->apExpr[i*2];
         4104  +    assert( pExpr->op==TK_COLUMN );
         4105  +    if( pExpr->iTable==pColumn->iTable
         4106  +     && pExpr->iColumn==pColumn->iColumn
         4107  +    ){
         4108  +      return;  /* Already present.  Return without doing anything. */
         4109  +    }
         4110  +  }
  4096   4111   
  4097   4112     pConst->nConst++;
  4098   4113     pConst->apExpr = sqlite3DbReallocOrFree(pConst->pParse->db, pConst->apExpr,
  4099   4114                            pConst->nConst*2*sizeof(Expr*));
  4100   4115     if( pConst->apExpr==0 ){
  4101   4116       pConst->nConst = 0;
  4102   4117     }else{

Changes to src/shell.c.in.

  1055   1055   #define AUTOEQP_off      0           /* Automatic EXPLAIN QUERY PLAN is off */
  1056   1056   #define AUTOEQP_on       1           /* Automatic EQP is on */
  1057   1057   #define AUTOEQP_trigger  2           /* On and also show plans for triggers */
  1058   1058   #define AUTOEQP_full     3           /* Show full EXPLAIN */
  1059   1059   
  1060   1060   /* Allowed values for ShellState.openMode
  1061   1061   */
  1062         -#define SHELL_OPEN_UNSPEC     0      /* No open-mode specified */
  1063         -#define SHELL_OPEN_NORMAL     1      /* Normal database file */
  1064         -#define SHELL_OPEN_APPENDVFS  2      /* Use appendvfs */
  1065         -#define SHELL_OPEN_ZIPFILE    3      /* Use the zipfile virtual table */
  1066         -#define SHELL_OPEN_READONLY   4      /* Open a normal database read-only */
         1062  +#define SHELL_OPEN_UNSPEC      0      /* No open-mode specified */
         1063  +#define SHELL_OPEN_NORMAL      1      /* Normal database file */
         1064  +#define SHELL_OPEN_APPENDVFS   2      /* Use appendvfs */
         1065  +#define SHELL_OPEN_ZIPFILE     3      /* Use the zipfile virtual table */
         1066  +#define SHELL_OPEN_READONLY    4      /* Open a normal database read-only */
         1067  +#define SHELL_OPEN_DESERIALIZE 5      /* Open using sqlite3_deserialize() */
  1067   1068   #define SHELL_OPEN_REUSESCHEMA 5    /* Open for schema reuse */
  1068   1069   
  1069   1070   /*
  1070   1071   ** These are the allowed shellFlgs values
  1071   1072   */
  1072   1073   #define SHFLG_Pagecache      0x00000001 /* The --pagecache option is used */
  1073   1074   #define SHFLG_Lookaside      0x00000002 /* Lookaside memory is used */
................................................................................
  3429   3430     ".once (-e|-x|FILE)       Output for the next SQL command only to FILE",
  3430   3431     "     If FILE begins with '|' then open as a pipe",
  3431   3432     "     Other options:",
  3432   3433     "       -e    Invoke system text editor",
  3433   3434     "       -x    Open in a spreadsheet",
  3434   3435     ".open ?OPTIONS? ?FILE?   Close existing database and reopen FILE",
  3435   3436     "     Options:",
  3436         -  "        --append      Use appendvfs to append database to the end of FILE",
  3437         -  "        --new         Initialize FILE to an empty database",
  3438         -  "        --readonly    Open FILE readonly",
  3439         -  "        --zip         FILE is a ZIP archive",
         3437  +  "        --append        Use appendvfs to append database to the end of FILE",
         3438  +#ifdef SQLITE_ENABLE_DESERIALIZE
         3439  +  "        --deserialize   Load into memory useing sqlite3_deserialize()",
         3440  +#endif
         3441  +  "        --new           Initialize FILE to an empty database",
         3442  +  "        --readonly      Open FILE readonly",
         3443  +  "        --zip           FILE is a ZIP archive",
  3440   3444     ".output ?FILE?           Send output to FILE or stdout if FILE is omitted",
  3441   3445     "     If FILE begins with '|' then open it as a pipe.",
  3442   3446     ".print STRING...         Print literal STRING",
  3443   3447     ".prompt MAIN CONTINUE    Replace the standard prompts",
  3444   3448     ".quit                    Exit this program",
  3445   3449     ".read FILE               Read input from FILE",
  3446   3450     ".restore ?DB? FILE       Restore content of DB (default \"main\") from FILE",
................................................................................
  3503   3507   ** zPattern describes the set of commands for which help text is provided.
  3504   3508   ** If zPattern is NULL, then show all commands, but only give a one-line
  3505   3509   ** description of each.
  3506   3510   **
  3507   3511   ** Return the number of matches.
  3508   3512   */
  3509   3513   static int showHelp(FILE *out, const char *zPattern){
  3510         -  int i, j;
         3514  +  int i = 0;
         3515  +  int j = 0;
  3511   3516     int n = 0;
  3512   3517     char *zPat;
  3513   3518     if( zPattern==0
  3514   3519      || zPattern[0]=='0'
  3515   3520      || strcmp(zPattern,"-a")==0
  3516   3521      || strcmp(zPattern,"-all")==0
  3517   3522     ){
................................................................................
  3589   3594     size_t nRead;
  3590   3595     char *pBuf;
  3591   3596     if( in==0 ) return 0;
  3592   3597     fseek(in, 0, SEEK_END);
  3593   3598     nIn = ftell(in);
  3594   3599     rewind(in);
  3595   3600     pBuf = sqlite3_malloc64( nIn+1 );
  3596         -  if( pBuf==0 ) return 0;
         3601  +  if( pBuf==0 ){ fclose(in); return 0; }
  3597   3602     nRead = fread(pBuf, nIn, 1, in);
  3598   3603     fclose(in);
  3599   3604     if( nRead!=1 ){
  3600   3605       sqlite3_free(pBuf);
  3601   3606       return 0;
  3602   3607     }
  3603   3608     pBuf[nIn] = 0;
................................................................................
  3669   3674     if( f==0 ){
  3670   3675       if( dfltZip && sqlite3_strlike("%.zip",zName,0)==0 ){
  3671   3676          return SHELL_OPEN_ZIPFILE;
  3672   3677       }else{
  3673   3678          return SHELL_OPEN_NORMAL;
  3674   3679       }
  3675   3680     }
         3681  +  n = fread(zBuf, 16, 1, f);
         3682  +  if( n==1 && memcmp(zBuf, "SQLite format 3", 16)==0 ){
         3683  +    fclose(f);
         3684  +    return SHELL_OPEN_NORMAL;
         3685  +  }
  3676   3686     fseek(f, -25, SEEK_END);
  3677   3687     n = fread(zBuf, 25, 1, f);
  3678   3688     if( n==1 && memcmp(zBuf, "Start-Of-SQLite3-", 17)==0 ){
  3679   3689       rc = SHELL_OPEN_APPENDVFS;
  3680   3690     }else{
  3681   3691       fseek(f, -22, SEEK_END);
  3682   3692       n = fread(zBuf, 22, 1, f);
................................................................................
  3719   3729         }
  3720   3730       }
  3721   3731       switch( p->openMode ){
  3722   3732         case SHELL_OPEN_APPENDVFS: {
  3723   3733           sqlite3_open_v2(p->zDbFilename, &p->db, 
  3724   3734              SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE, "apndvfs");
  3725   3735           break;
         3736  +      }
         3737  +      case SHELL_OPEN_DESERIALIZE: {
         3738  +        sqlite3_open(0, &p->db);
         3739  +        break;
  3726   3740         }
  3727   3741         case SHELL_OPEN_ZIPFILE: {
  3728   3742           sqlite3_open(":memory:", &p->db);
  3729   3743           break;
  3730   3744         }
  3731   3745         case SHELL_OPEN_READONLY: {
  3732   3746           sqlite3_open_v2(p->zDbFilename, &p->db, SQLITE_OPEN_READONLY, 0);
................................................................................
  3774   3788   #endif
  3775   3789       if( p->openMode==SHELL_OPEN_ZIPFILE ){
  3776   3790         char *zSql = sqlite3_mprintf(
  3777   3791            "CREATE VIRTUAL TABLE zip USING zipfile(%Q);", p->zDbFilename);
  3778   3792         sqlite3_exec(p->db, zSql, 0, 0, 0);
  3779   3793         sqlite3_free(zSql);
  3780   3794       }
         3795  +#ifdef SQLITE_ENABLE_DESERIALIZE
         3796  +    else if( p->openMode==SHELL_OPEN_DESERIALIZE ){
         3797  +      int nData = 0;
         3798  +      unsigned char *aData = (unsigned char*)readFile(p->zDbFilename, &nData);
         3799  +      int rc = sqlite3_deserialize(p->db, "main", aData, nData, nData,
         3800  +                   SQLITE_DESERIALIZE_RESIZEABLE |
         3801  +                   SQLITE_DESERIALIZE_FREEONCLOSE);
         3802  +      if( rc ){
         3803  +        utf8_printf(stderr, "Error: sqlite3_deserialize() returns %d\n", rc);
         3804  +      }
         3805  +    }
         3806  +#endif
  3781   3807     }
  3782   3808   }
  3783   3809   
  3784   3810   /*
  3785   3811   ** Attempt to close the databaes connection.  Report errors.
  3786   3812   */
  3787   3813   void close_db(sqlite3 *db){
................................................................................
  4978   5004   ){
  4979   5005     *ppStmt = 0;
  4980   5006     if( *pRc==SQLITE_OK ){
  4981   5007       va_list ap;
  4982   5008       char *z;
  4983   5009       va_start(ap, zFmt);
  4984   5010       z = sqlite3_vmprintf(zFmt, ap);
         5011  +    va_end(ap);
  4985   5012       if( z==0 ){
  4986   5013         *pRc = SQLITE_NOMEM;
  4987   5014       }else{
  4988   5015         shellPrepare(db, pRc, z, ppStmt);
  4989   5016         sqlite3_free(z);
  4990   5017       }
  4991   5018     }
................................................................................
  5931   5958           { "enable_trigger",   SQLITE_DBCONFIG_ENABLE_TRIGGER         },
  5932   5959           { "fts3_tokenizer",   SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER  },
  5933   5960           { "load_extension",   SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION  },
  5934   5961           { "no_ckpt_on_close", SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE       },
  5935   5962           { "enable_qpsg",      SQLITE_DBCONFIG_ENABLE_QPSG            },
  5936   5963           { "trigger_eqp",      SQLITE_DBCONFIG_TRIGGER_EQP            },
  5937   5964           { "reset_database",   SQLITE_DBCONFIG_RESET_DATABASE         },
         5965  +        { "defensive",        SQLITE_DBCONFIG_DEFENSIVE              },
  5938   5966       };
  5939   5967       int ii, v;
  5940   5968       open_db(p, 0);
  5941   5969       for(ii=0; ii<ArraySize(aDbConfig); ii++){
  5942   5970         if( nArg>1 && strcmp(azArg[1], aDbConfig[ii].zName)!=0 ) continue;
  5943   5971         if( nArg>=3 ){
  5944   5972           sqlite3_db_config(p->db, aDbConfig[ii].op, booleanValue(azArg[2]), 0);
................................................................................
  6170   6198         raw_printf(stderr, "Usage: .headers on|off\n");
  6171   6199         rc = 1;
  6172   6200       }
  6173   6201     }else
  6174   6202   
  6175   6203     if( c=='h' && strncmp(azArg[0], "help", n)==0 ){
  6176   6204       if( nArg>=2 ){
  6177         -      int n = showHelp(p->out, azArg[1]);
         6205  +      n = showHelp(p->out, azArg[1]);
  6178   6206         if( n==0 ){
  6179   6207           utf8_printf(p->out, "Nothing matches '%s'\n", azArg[1]);
  6180   6208         }
  6181   6209       }else{
  6182   6210         showHelp(p->out, 0);
  6183   6211       }
  6184   6212     }else
................................................................................
  6655   6683   #endif
  6656   6684         }else if( optionMatch(z, "append") ){
  6657   6685           p->openMode = SHELL_OPEN_APPENDVFS;
  6658   6686         }else if( optionMatch(z, "readonly") ){
  6659   6687           p->openMode = SHELL_OPEN_READONLY;
  6660   6688         }else if( optionMatch(z, "reuseschema") ){
  6661   6689           p->openMode = SHELL_OPEN_REUSESCHEMA;
         6690  +      }else if( z[0]=='-' ){
         6691  +#ifdef SQLITE_ENABLE_DESERIALIZE
         6692  +      }else if( optionMatch(z, "deserialize") ){
         6693  +        p->openMode = SHELL_OPEN_DESERIALIZE;
         6694  +#endif
  6662   6695         }else if( z[0]=='-' ){
  6663   6696           utf8_printf(stderr, "unknown option: %s\n", z);
  6664   6697           rc = 1;
  6665   6698           goto meta_command_exit;
  6666   6699         }
  6667   6700       }
  6668   6701       /* If a filename is specified, try to open it first */
................................................................................
  8623   8656         zVfs = cmdline_option_value(argc, argv, ++i);
  8624   8657   #ifdef SQLITE_HAVE_ZLIB
  8625   8658       }else if( strcmp(z,"-zip")==0 ){
  8626   8659         data.openMode = SHELL_OPEN_ZIPFILE;
  8627   8660   #endif
  8628   8661       }else if( strcmp(z,"-append")==0 ){
  8629   8662         data.openMode = SHELL_OPEN_APPENDVFS;
         8663  +#ifdef SQLITE_ENABLE_DESERIALIZE
         8664  +    }else if( strcmp(z,"-deserialize")==0 ){
         8665  +      data.openMode = SHELL_OPEN_DESERIALIZE;
         8666  +#endif
  8630   8667       }else if( strcmp(z,"-readonly")==0 ){
  8631   8668         data.openMode = SHELL_OPEN_READONLY;
  8632   8669       }else if( strcmp(z,"-reuseschema")==0 ){
  8633   8670         data.openMode = SHELL_OPEN_REUSESCHEMA;
  8634   8671   #if !defined(SQLITE_OMIT_VIRTUALTABLE) && defined(SQLITE_HAVE_ZLIB)
  8635   8672       }else if( strncmp(z, "-A",2)==0 ){
  8636   8673         /* All remaining command-line arguments are passed to the ".archive"
................................................................................
  8720   8757         memcpy(data.colSeparator,",",2);
  8721   8758   #ifdef SQLITE_HAVE_ZLIB
  8722   8759       }else if( strcmp(z,"-zip")==0 ){
  8723   8760         data.openMode = SHELL_OPEN_ZIPFILE;
  8724   8761   #endif
  8725   8762       }else if( strcmp(z,"-append")==0 ){
  8726   8763         data.openMode = SHELL_OPEN_APPENDVFS;
         8764  +#ifdef SQLITE_ENABLE_DESERIALIZE
         8765  +    }else if( strcmp(z,"-deserialize")==0 ){
         8766  +      data.openMode = SHELL_OPEN_DESERIALIZE;
         8767  +#endif
  8727   8768       }else if( strcmp(z,"-readonly")==0 ){
  8728   8769         data.openMode = SHELL_OPEN_READONLY;
  8729   8770       }else if( strcmp(z,"-reuseschema")==0 ){
  8730   8771         data.openMode = SHELL_OPEN_REUSESCHEMA;
  8731   8772       }else if( strcmp(z,"-ascii")==0 ){
  8732   8773         data.mode = MODE_Ascii;
  8733   8774         sqlite3_snprintf(sizeof(data.colSeparator), data.colSeparator,
................................................................................
  8866   8907       }
  8867   8908       free(azCmd);
  8868   8909     }else{
  8869   8910       /* Run commands received from standard input
  8870   8911       */
  8871   8912       if( stdin_is_interactive ){
  8872   8913         char *zHome;
  8873         -      char *zHistory = 0;
         8914  +      char *zHistory;
  8874   8915         int nHistory;
  8875   8916         printf(
  8876   8917           "SQLite version %s %.19s\n" /*extra-version-info*/
  8877   8918           "Enter \".help\" for usage hints.\n",
  8878   8919           sqlite3_libversion(), sqlite3_sourceid()
  8879   8920         );
  8880   8921         if( warnInmemoryDb ){
  8881   8922           printf("Connected to a ");
  8882   8923           printBold("transient in-memory database");
  8883   8924           printf(".\nUse \".open FILENAME\" to reopen on a "
  8884   8925                  "persistent database.\n");
  8885   8926         }
  8886         -      zHome = find_home_dir(0);
  8887         -      if( zHome ){
         8927  +      zHistory = getenv("SQLITE_HISTORY");
         8928  +      if( zHistory ){
         8929  +        zHistory = strdup(zHistory);
         8930  +      }else if( (zHome = find_home_dir(0))!=0 ){
  8888   8931           nHistory = strlen30(zHome) + 20;
  8889   8932           if( (zHistory = malloc(nHistory))!=0 ){
  8890   8933             sqlite3_snprintf(nHistory, zHistory,"%s/.sqlite_history", zHome);
  8891   8934           }
  8892   8935         }
  8893   8936         if( zHistory ){ shell_read_history(zHistory); }
  8894   8937   #if HAVE_READLINE || HAVE_EDITLINE

Changes to src/sqlite.h.in.

  2156   2156   ** <li> sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 1, 0);
  2157   2157   ** <li> [sqlite3_exec](db, "[VACUUM]", 0, 0, 0);
  2158   2158   ** <li> sqlite3_db_config(db, SQLITE_DBCONFIG_RESET_DATABASE, 0, 0);
  2159   2159   ** </ol>
  2160   2160   ** Because resetting a database is destructive and irreversible, the
  2161   2161   ** process requires the use of this obscure API and multiple steps to help
  2162   2162   ** ensure that it does not happen by accident.
         2163  +**
         2164  +** <dt>SQLITE_DBCONFIG_DEFENSIVE</dt>
         2165  +** <dd>The SQLITE_DBCONFIG_DEFENSIVE option actives or deactivates the
         2166  +** "defensive" flag for a database connection.  When the defensive
         2167  +** flag is enabled, some obscure features of SQLite are disabled in order
         2168  +** to reduce the attack surface. Applications that run untrusted SQL
         2169  +** can activate this flag to reduce the risk of zero-day exploits.
         2170  +** <p>
         2171  +** Features disabled by the defensive flag include:
         2172  +** <ul>
         2173  +** <li>The [PRAGMA writable_schema=ON] statement.
         2174  +** <li>Writes to the [sqlite_dbpage] virtual table.
         2175  +** </ul>
         2176  +** New restrictions may be added in future releases.
         2177  +** <p>
         2178  +** To be clear: It should never be possible for hostile SQL to cause
         2179  +** arbitrary memory reads, memory leaks, buffer overflows, assertion
         2180  +** faults, arbitrary code execution, crashes, or other mischief, regardless
         2181  +** of the value of the defensive flag.  Any occurrance of these problems
         2182  +** is considered a serious bug and will be fixed promptly.  It is not
         2183  +** necessary to enable the defensive flag in order to make SQLite secure
         2184  +** against attack. The defensive flag merely provides an additional layer
         2185  +** of defense against unknown vulnerabilities.
  2163   2186   ** </dd>
  2164   2187   ** </dl>
  2165   2188   */
  2166   2189   #define SQLITE_DBCONFIG_MAINDBNAME            1000 /* const char* */
  2167   2190   #define SQLITE_DBCONFIG_LOOKASIDE             1001 /* void* int int */
  2168   2191   #define SQLITE_DBCONFIG_ENABLE_FKEY           1002 /* int int* */
  2169   2192   #define SQLITE_DBCONFIG_ENABLE_TRIGGER        1003 /* int int* */
  2170   2193   #define SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1004 /* int int* */
  2171   2194   #define SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION 1005 /* int int* */
  2172   2195   #define SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE      1006 /* int int* */
  2173   2196   #define SQLITE_DBCONFIG_ENABLE_QPSG           1007 /* int int* */
  2174   2197   #define SQLITE_DBCONFIG_TRIGGER_EQP           1008 /* int int* */
  2175   2198   #define SQLITE_DBCONFIG_RESET_DATABASE        1009 /* int int* */
  2176         -#define SQLITE_DBCONFIG_MAX                   1009 /* Largest DBCONFIG */
         2199  +#define SQLITE_DBCONFIG_DEFENSIVE             1010 /* int int* */
         2200  +#define SQLITE_DBCONFIG_MAX                   1010 /* Largest DBCONFIG */
  2177   2201   
  2178   2202   /*
  2179   2203   ** CAPI3REF: Enable Or Disable Extended Result Codes
  2180   2204   ** METHOD: sqlite3
  2181   2205   **
  2182   2206   ** ^The sqlite3_extended_result_codes() routine enables or disables the
  2183   2207   ** [extended result codes] feature of SQLite. ^The extended result
................................................................................
  3607   3631   ** probably reused many times.)^ ^Without this flag, [sqlite3_prepare_v3()]
  3608   3632   ** and [sqlite3_prepare16_v3()] assume that the prepared statement will 
  3609   3633   ** be used just once or at most a few times and then destroyed using
  3610   3634   ** [sqlite3_finalize()] relatively soon. The current implementation acts
  3611   3635   ** on this hint by avoiding the use of [lookaside memory] so as not to
  3612   3636   ** deplete the limited store of lookaside memory. Future versions of
  3613   3637   ** SQLite may act on this hint differently.
         3638  +**
         3639  +** [[SQLITE_PREPARE_NORMALIZE]] ^(<dt>SQLITE_PREPARE_NORMALIZE</dt>
         3640  +** <dd>The SQLITE_PREPARE_NORMALIZE flag indicates that a normalized
         3641  +** representation of the SQL statement should be calculated and then
         3642  +** associated with the prepared statement, which can be obtained via
         3643  +** the [sqlite3_normalized_sql()] interface.  The semantics used to
         3644  +** normalize a SQL statement are unspecified and subject to change.
         3645  +** At a minimum, literal values will be replaced with suitable
         3646  +** placeholders.
  3614   3647   ** </dl>
  3615   3648   */
  3616   3649   #define SQLITE_PREPARE_PERSISTENT              0x01
         3650  +#define SQLITE_PREPARE_NORMALIZE               0x02
  3617   3651   
  3618   3652   /*
  3619   3653   ** CAPI3REF: Compiling An SQL Statement
  3620   3654   ** KEYWORDS: {SQL statement compiler}
  3621   3655   ** METHOD: sqlite3
  3622   3656   ** CONSTRUCTOR: sqlite3_stmt
  3623   3657   **
................................................................................
  3767   3801   ** ^The sqlite3_sql(P) interface returns a pointer to a copy of the UTF-8
  3768   3802   ** SQL text used to create [prepared statement] P if P was
  3769   3803   ** created by [sqlite3_prepare_v2()], [sqlite3_prepare_v3()],
  3770   3804   ** [sqlite3_prepare16_v2()], or [sqlite3_prepare16_v3()].
  3771   3805   ** ^The sqlite3_expanded_sql(P) interface returns a pointer to a UTF-8
  3772   3806   ** string containing the SQL text of prepared statement P with
  3773   3807   ** [bound parameters] expanded.
         3808  +** ^The sqlite3_normalized_sql(P) interface returns a pointer to a UTF-8
         3809  +** string containing the normalized SQL text of prepared statement P.  The
         3810  +** semantics used to normalize a SQL statement are unspecified and subject
         3811  +** to change.  At a minimum, literal values will be replaced with suitable
         3812  +** placeholders.
  3774   3813   **
  3775   3814   ** ^(For example, if a prepared statement is created using the SQL
  3776   3815   ** text "SELECT $abc,:xyz" and if parameter $abc is bound to integer 2345
  3777   3816   ** and parameter :xyz is unbound, then sqlite3_sql() will return
  3778   3817   ** the original string, "SELECT $abc,:xyz" but sqlite3_expanded_sql()
  3779   3818   ** will return "SELECT 2345,NULL".)^
  3780   3819   **
................................................................................
  3782   3821   ** is available to hold the result, or if the result would exceed the
  3783   3822   ** the maximum string length determined by the [SQLITE_LIMIT_LENGTH].
  3784   3823   **
  3785   3824   ** ^The [SQLITE_TRACE_SIZE_LIMIT] compile-time option limits the size of
  3786   3825   ** bound parameter expansions.  ^The [SQLITE_OMIT_TRACE] compile-time
  3787   3826   ** option causes sqlite3_expanded_sql() to always return NULL.
  3788   3827   **
  3789         -** ^The string returned by sqlite3_sql(P) is managed by SQLite and is
  3790         -** automatically freed when the prepared statement is finalized.
         3828  +** ^The strings returned by sqlite3_sql(P) and sqlite3_normalized_sql(P)
         3829  +** are managed by SQLite and are automatically freed when the prepared
         3830  +** statement is finalized.
  3791   3831   ** ^The string returned by sqlite3_expanded_sql(P), on the other hand,
  3792   3832   ** is obtained from [sqlite3_malloc()] and must be free by the application
  3793   3833   ** by passing it to [sqlite3_free()].
  3794   3834   */
  3795   3835   const char *sqlite3_sql(sqlite3_stmt *pStmt);
  3796   3836   char *sqlite3_expanded_sql(sqlite3_stmt *pStmt);
         3837  +const char *sqlite3_normalized_sql(sqlite3_stmt *pStmt);
  3797   3838   
  3798   3839   /*
  3799   3840   ** CAPI3REF: Determine If An SQL Statement Writes The Database
  3800   3841   ** METHOD: sqlite3_stmt
  3801   3842   **
  3802   3843   ** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if
  3803   3844   ** and only if the [prepared statement] X makes no direct changes to

Changes to src/sqlite3ext.h.

   306    306     void (*str_append)(sqlite3_str*, const char *zIn, int N);
   307    307     void (*str_appendall)(sqlite3_str*, const char *zIn);
   308    308     void (*str_appendchar)(sqlite3_str*, int N, char C);
   309    309     void (*str_reset)(sqlite3_str*);
   310    310     int (*str_errcode)(sqlite3_str*);
   311    311     int (*str_length)(sqlite3_str*);
   312    312     char *(*str_value)(sqlite3_str*);
          313  +  /* Version 3.25.0 and later */
   313    314     int (*create_window_function)(sqlite3*,const char*,int,int,void*,
   314    315                               void (*xStep)(sqlite3_context*,int,sqlite3_value**),
   315    316                               void (*xFinal)(sqlite3_context*),
   316    317                               void (*xValue)(sqlite3_context*),
   317    318                               void (*xInv)(sqlite3_context*,int,sqlite3_value**),
   318    319                               void(*xDestroy)(void*));
          320  +  /* Version 3.26.0 and later */
          321  +  const char *(*normalized_sql)(sqlite3_stmt*);
   319    322   };
   320    323   
   321    324   /*
   322    325   ** This is the function signature used for all extension entry points.  It
   323    326   ** is also defined in the file "loadext.c".
   324    327   */
   325    328   typedef int (*sqlite3_loadext_entry)(
................................................................................
   599    602   #define sqlite3_str_appendchar         sqlite3_api->str_appendchar
   600    603   #define sqlite3_str_reset              sqlite3_api->str_reset
   601    604   #define sqlite3_str_errcode            sqlite3_api->str_errcode
   602    605   #define sqlite3_str_length             sqlite3_api->str_length
   603    606   #define sqlite3_str_value              sqlite3_api->str_value
   604    607   /* Version 3.25.0 and later */
   605    608   #define sqlite3_create_window_function sqlite3_api->create_window_function
          609  +/* Version 3.26.0 and later */
          610  +#define sqlite3_normalized_sql         sqlite3_api->normalized_sql
   606    611   #endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */
   607    612   
   608    613   #if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
   609    614     /* This case when the file really is being compiled as a loadable 
   610    615     ** extension */
   611    616   # define SQLITE_EXTENSION_INIT1     const sqlite3_api_routines *sqlite3_api=0;
   612    617   # define SQLITE_EXTENSION_INIT2(v)  sqlite3_api=v;

Changes to src/sqliteInt.h.

  1305   1305   };
  1306   1306   
  1307   1307   /*
  1308   1308   ** A hash table for built-in function definitions.  (Application-defined
  1309   1309   ** functions use a regular table table from hash.h.)
  1310   1310   **
  1311   1311   ** Hash each FuncDef structure into one of the FuncDefHash.a[] slots.
  1312         -** Collisions are on the FuncDef.u.pHash chain.
         1312  +** Collisions are on the FuncDef.u.pHash chain.  Use the SQLITE_FUNC_HASH()
         1313  +** macro to compute a hash on the function name.
  1313   1314   */
  1314   1315   #define SQLITE_FUNC_HASH_SZ 23
  1315   1316   struct FuncDefHash {
  1316   1317     FuncDef *a[SQLITE_FUNC_HASH_SZ];       /* Hash table for functions */
  1317   1318   };
         1319  +#define SQLITE_FUNC_HASH(C,L) (((C)+(L))%SQLITE_FUNC_HASH_SZ)
  1318   1320   
  1319   1321   #ifdef SQLITE_USER_AUTHENTICATION
  1320   1322   /*
  1321   1323   ** Information held in the "sqlite3" database connection object and used
  1322   1324   ** to manage user authentication.
  1323   1325   */
  1324   1326   typedef struct sqlite3_userauth sqlite3_userauth;
................................................................................
  1371   1373     sqlite3_vfs *pVfs;            /* OS Interface */
  1372   1374     struct Vdbe *pVdbe;           /* List of active virtual machines */
  1373   1375     CollSeq *pDfltColl;           /* The default collating sequence (BINARY) */
  1374   1376     sqlite3_mutex *mutex;         /* Connection mutex */
  1375   1377     Db *aDb;                      /* All backends */
  1376   1378     int nDb;                      /* Number of backends currently in use */
  1377   1379     u32 mDbFlags;                 /* flags recording internal state */
  1378         -  u32 flags;                    /* flags settable by pragmas. See below */
         1380  +  u64 flags;                    /* flags settable by pragmas. See below */
  1379   1381     i64 lastRowid;                /* ROWID of most recent insert (see above) */
  1380   1382     i64 szMmap;                   /* Default mmap_size setting */
  1381   1383     u32 nSchemaLock;              /* Do not reset the schema when non-zero */
  1382   1384     unsigned int openFlags;       /* Flags passed to sqlite3_vfs.xOpen() */
  1383   1385     int errCode;                  /* Most recent error code (SQLITE_*) */
  1384   1386     int errMask;                  /* & result codes with this before returning */
  1385   1387     int iSysErrno;                /* Errno value from last system error */
................................................................................
  1537   1539   #define SQLITE_QueryOnly      0x00100000  /* Disable database changes */
  1538   1540   #define SQLITE_CellSizeCk     0x00200000  /* Check btree cell sizes on load */
  1539   1541   #define SQLITE_Fts3Tokenizer  0x00400000  /* Enable fts3_tokenizer(2) */
  1540   1542   #define SQLITE_EnableQPSG     0x00800000  /* Query Planner Stability Guarantee*/
  1541   1543   #define SQLITE_TriggerEQP     0x01000000  /* Show trigger EXPLAIN QUERY PLAN */
  1542   1544   #define SQLITE_ResetDatabase  0x02000000  /* Reset the database */
  1543   1545   #define SQLITE_LegacyAlter    0x04000000  /* Legacy ALTER TABLE behaviour */
         1546  +#define SQLITE_NoSchemaError  0x08000000  /* Do not report schema parse errors*/
         1547  +#define SQLITE_Defensive      0x10000000  /* Input SQL is likely hostile */
  1544   1548   
  1545   1549   /* Flags used only if debugging */
         1550  +#define HI(X)  ((u64)(X)<<32)
  1546   1551   #ifdef SQLITE_DEBUG
  1547         -#define SQLITE_SqlTrace       0x08000000  /* Debug print SQL as it executes */
  1548         -#define SQLITE_VdbeListing    0x10000000  /* Debug listings of VDBE programs */
  1549         -#define SQLITE_VdbeTrace      0x20000000  /* True to trace VDBE execution */
  1550         -#define SQLITE_VdbeAddopTrace 0x40000000  /* Trace sqlite3VdbeAddOp() calls */
  1551         -#define SQLITE_VdbeEQP        0x80000000  /* Debug EXPLAIN QUERY PLAN */
         1552  +#define SQLITE_SqlTrace       HI(0x0001)  /* Debug print SQL as it executes */
         1553  +#define SQLITE_VdbeListing    HI(0x0002)  /* Debug listings of VDBE progs */
         1554  +#define SQLITE_VdbeTrace      HI(0x0004)  /* True to trace VDBE execution */
         1555  +#define SQLITE_VdbeAddopTrace HI(0x0008)  /* Trace sqlite3VdbeAddOp() calls */
         1556  +#define SQLITE_VdbeEQP        HI(0x0010)  /* Debug EXPLAIN QUERY PLAN */
  1552   1557   #endif
  1553   1558   
  1554   1559   /*
  1555   1560   ** Allowed values for sqlite3.mDbFlags
  1556   1561   */
  1557   1562   #define DBFLAG_SchemaChange   0x0001  /* Uncommitted Hash table changes */
  1558   1563   #define DBFLAG_PreferBuiltin  0x0002  /* Preference to built-in funcs */
................................................................................
  1943   1948   /*
  1944   1949   ** The schema for each SQL table and view is represented in memory
  1945   1950   ** by an instance of the following structure.
  1946   1951   */
  1947   1952   struct Table {
  1948   1953     char *zName;         /* Name of the table or view */
  1949   1954     Column *aCol;        /* Information about each column */
         1955  +#ifdef SQLITE_ENABLE_NORMALIZE
         1956  +  Hash *pColHash;      /* All columns indexed by name */
         1957  +#endif
  1950   1958     Index *pIndex;       /* List of SQL indexes on this table. */
  1951   1959     Select *pSelect;     /* NULL for tables.  Points to definition if a view. */
  1952   1960     FKey *pFKey;         /* Linked list of all foreign keys in this table */
  1953   1961     char *zColAff;       /* String defining the affinity of each column */
  1954   1962     ExprList *pCheck;    /* All CHECK constraints */
  1955   1963                          /*   ... also used as column name list in a VIEW */
  1956   1964     int tnum;            /* Root BTree page for this table */
................................................................................
  2279   2287     void *p;          /* Pointer to sampled record */
  2280   2288     int n;            /* Size of record in bytes */
  2281   2289     tRowcnt *anEq;    /* Est. number of rows where the key equals this sample */
  2282   2290     tRowcnt *anLt;    /* Est. number of rows where key is less than this sample */
  2283   2291     tRowcnt *anDLt;   /* Est. number of distinct keys less than this sample */
  2284   2292   };
  2285   2293   
         2294  +/*
         2295  +** Possible values to use within the flags argument to sqlite3GetToken().
         2296  +*/
         2297  +#define SQLITE_TOKEN_QUOTED    0x1 /* Token is a quoted identifier. */
         2298  +#define SQLITE_TOKEN_KEYWORD   0x2 /* Token is a keyword. */
         2299  +
  2286   2300   /*
  2287   2301   ** Each token coming out of the lexer is an instance of
  2288   2302   ** this structure.  Tokens are also used as part of an expression.
  2289   2303   **
  2290   2304   ** The memory that "z" points to is owned by other objects.  Take care
  2291   2305   ** that the owner of the "z" string does not deallocate the string before
  2292   2306   ** the Token goes out of scope!  Very often, the "z" points to some place
................................................................................
  3652   3666   int sqlite3IsIdChar(u8);
  3653   3667   
  3654   3668   /*
  3655   3669   ** Internal function prototypes
  3656   3670   */
  3657   3671   int sqlite3StrICmp(const char*,const char*);
  3658   3672   int sqlite3Strlen30(const char*);
         3673  +#define sqlite3Strlen30NN(C) (strlen(C)&0x3fffffff)
  3659   3674   char *sqlite3ColumnType(Column*,char*);
  3660   3675   #define sqlite3StrNICmp sqlite3_strnicmp
  3661   3676   
  3662   3677   int sqlite3MallocInit(void);
  3663   3678   void sqlite3MallocEnd(void);
  3664   3679   void *sqlite3Malloc(u64);
  3665   3680   void *sqlite3MallocZero(u64);
................................................................................
  3768   3783     void *sqlite3TestTextToPtr(const char*);
  3769   3784   #endif
  3770   3785   
  3771   3786   #if defined(SQLITE_DEBUG)
  3772   3787     void sqlite3TreeViewExpr(TreeView*, const Expr*, u8);
  3773   3788     void sqlite3TreeViewBareExprList(TreeView*, const ExprList*, const char*);
  3774   3789     void sqlite3TreeViewExprList(TreeView*, const ExprList*, u8, const char*);
         3790  +  void sqlite3TreeViewSrcList(TreeView*, const SrcList*);
  3775   3791     void sqlite3TreeViewSelect(TreeView*, const Select*, u8);
  3776   3792     void sqlite3TreeViewWith(TreeView*, const With*, u8);
  3777   3793   #ifndef SQLITE_OMIT_WINDOWFUNC
  3778   3794     void sqlite3TreeViewWindow(TreeView*, const Window*, u8);
  3779   3795     void sqlite3TreeViewWinFunc(TreeView*, const Window*, u8);
  3780   3796   #endif
  3781   3797   #endif
................................................................................
  4000   4016   #ifdef SQLITE_ENABLE_CURSOR_HINTS
  4001   4017   int sqlite3ExprContainsSubquery(Expr*);
  4002   4018   #endif
  4003   4019   int sqlite3ExprIsInteger(Expr*, int*);
  4004   4020   int sqlite3ExprCanBeNull(const Expr*);
  4005   4021   int sqlite3ExprNeedsNoAffinityChange(const Expr*, char);
  4006   4022   int sqlite3IsRowid(const char*);
         4023  +#ifdef SQLITE_ENABLE_NORMALIZE
         4024  +int sqlite3IsRowidN(const char*, int);
         4025  +#endif
  4007   4026   void sqlite3GenerateRowDelete(
  4008   4027       Parse*,Table*,Trigger*,int,int,int,i16,u8,u8,u8,int);
  4009   4028   void sqlite3GenerateRowIndexDelete(Parse*, Table*, int, int, int*, int);
  4010   4029   int sqlite3GenerateIndexKey(Parse*, Index*, int, int, int, int*,Index*,int);
  4011   4030   void sqlite3ResolvePartIdxLabel(Parse*,int);
  4012   4031   int sqlite3ExprReferencesUpdatedColumn(Expr*,int*,int);
  4013   4032   void sqlite3GenerateConstraintChecks(Parse*,Table*,int*,int,int,int,int,
................................................................................
  4026   4045   void sqlite3UniqueConstraint(Parse*, int, Index*);
  4027   4046   void sqlite3RowidConstraint(Parse*, int, Table*);
  4028   4047   Expr *sqlite3ExprDup(sqlite3*,Expr*,int);
  4029   4048   ExprList *sqlite3ExprListDup(sqlite3*,ExprList*,int);
  4030   4049   SrcList *sqlite3SrcListDup(sqlite3*,SrcList*,int);
  4031   4050   IdList *sqlite3IdListDup(sqlite3*,IdList*);
  4032   4051   Select *sqlite3SelectDup(sqlite3*,Select*,int);
         4052  +#ifdef SQLITE_ENABLE_NORMALIZE
         4053  +FuncDef *sqlite3FunctionSearchN(int,const char*,int);
         4054  +#endif
  4033   4055   void sqlite3InsertBuiltinFuncs(FuncDef*,int);
  4034   4056   FuncDef *sqlite3FindFunction(sqlite3*,const char*,int,u8,u8);
  4035   4057   void sqlite3RegisterBuiltinFunctions(void);
  4036   4058   void sqlite3RegisterDateTimeFunctions(void);
  4037   4059   void sqlite3RegisterPerConnectionBuiltinFunctions(sqlite3*);
  4038   4060   int sqlite3SafetyCheckOk(sqlite3*);
  4039   4061   int sqlite3SafetyCheckSickOrOk(sqlite3*);
................................................................................
  4183   4205   CollSeq *sqlite3ExprCollSeq(Parse *pParse, Expr *pExpr);
  4184   4206   CollSeq *sqlite3ExprNNCollSeq(Parse *pParse, Expr *pExpr);
  4185   4207   int sqlite3ExprCollSeqMatch(Parse*,Expr*,Expr*);
  4186   4208   Expr *sqlite3ExprAddCollateToken(Parse *pParse, Expr*, const Token*, int);
  4187   4209   Expr *sqlite3ExprAddCollateString(Parse*,Expr*,const char*);
  4188   4210   Expr *sqlite3ExprSkipCollate(Expr*);
  4189   4211   int sqlite3CheckCollSeq(Parse *, CollSeq *);
         4212  +int sqlite3WritableSchema(sqlite3*);
  4190   4213   int sqlite3CheckObjectName(Parse *, const char *);
  4191   4214   void sqlite3VdbeSetChanges(sqlite3 *, int);
  4192   4215   int sqlite3AddInt64(i64*,i64);
  4193   4216   int sqlite3SubInt64(i64*,i64);
  4194   4217   int sqlite3MulInt64(i64*,i64);
  4195   4218   int sqlite3AbsInt32(int);
  4196   4219   #ifdef SQLITE_ENABLE_8_3_NAMES
................................................................................
  4229   4252   #endif
  4230   4253   void sqlite3RootPageMoved(sqlite3*, int, int, int);
  4231   4254   void sqlite3Reindex(Parse*, Token*, Token*);
  4232   4255   void sqlite3AlterFunctions(void);
  4233   4256   void sqlite3AlterRenameTable(Parse*, SrcList*, Token*);
  4234   4257   void sqlite3AlterRenameColumn(Parse*, SrcList*, Token*, Token*);
  4235   4258   int sqlite3GetToken(const unsigned char *, int *);
         4259  +#ifdef SQLITE_ENABLE_NORMALIZE
         4260  +int sqlite3GetTokenNormalized(const unsigned char *, int *, int *);
         4261  +#endif
  4236   4262   void sqlite3NestedParse(Parse*, const char*, ...);
  4237   4263   void sqlite3ExpirePreparedStatements(sqlite3*, int);
  4238   4264   int sqlite3CodeSubselect(Parse*, Expr *, int, int);
  4239   4265   void sqlite3SelectPrep(Parse*, Select*, NameContext*);
  4240   4266   void sqlite3SelectWrongNumTermsError(Parse *pParse, Select *p);
  4241   4267   int sqlite3MatchSpanName(const char*, const char*, const char*, const char*);
  4242   4268   int sqlite3ResolveExprNames(NameContext*, Expr*);
................................................................................
  4389   4415   int sqlite3VtabCallDestroy(sqlite3*, int, const char *);
  4390   4416   int sqlite3VtabBegin(sqlite3 *, VTable *);
  4391   4417   FuncDef *sqlite3VtabOverloadFunction(sqlite3 *,FuncDef*, int nArg, Expr*);
  4392   4418   sqlite3_int64 sqlite3StmtCurrentTime(sqlite3_context*);
  4393   4419   int sqlite3VdbeParameterIndex(Vdbe*, const char*, int);
  4394   4420   int sqlite3TransferBindings(sqlite3_stmt *, sqlite3_stmt *);
  4395   4421   void sqlite3ParserReset(Parse*);
         4422  +#ifdef SQLITE_ENABLE_NORMALIZE
         4423  +void sqlite3Normalize(Vdbe*, const char*, int, u8);
         4424  +#endif
  4396   4425   int sqlite3Reprepare(Vdbe*);
  4397   4426   void sqlite3ExprListCheckLength(Parse*, ExprList*, const char*);
  4398   4427   CollSeq *sqlite3BinaryCompareCollSeq(Parse *, Expr *, Expr *);
  4399   4428   int sqlite3TempInMemory(const sqlite3*);
  4400   4429   const char *sqlite3JournalModename(int);
  4401   4430   #ifndef SQLITE_OMIT_WAL
  4402   4431     int sqlite3Checkpoint(sqlite3*, int, int, int*, int*);

Changes to src/test1.c.

  4214   4214     Tcl_Obj *CONST objv[]
  4215   4215   ){
  4216   4216     sqlite3 *db;
  4217   4217     const char *zSql;
  4218   4218     char *zCopy = 0;                /* malloc() copy of zSql */
  4219   4219     int bytes;
  4220   4220     const char *zTail = 0;
         4221  +  const char **pzTail;
  4221   4222     sqlite3_stmt *pStmt = 0;
  4222   4223     char zBuf[50];
  4223   4224     int rc;
  4224   4225   
  4225   4226     if( objc!=5 && objc!=4 ){
  4226   4227       Tcl_AppendResult(interp, "wrong # args: should be \"", 
  4227   4228          Tcl_GetString(objv[0]), " DB sql bytes tailvar", 0);
................................................................................
  4238   4239       zCopy = malloc(bytes);
  4239   4240       memcpy(zCopy, zSql, bytes);
  4240   4241     }else{
  4241   4242       int n = (int)strlen(zSql) + 1;
  4242   4243       zCopy = malloc(n);
  4243   4244       memcpy(zCopy, zSql, n);
  4244   4245     }
  4245         -  rc = sqlite3_prepare_v2(db, zCopy, bytes, &pStmt, objc>=5 ? &zTail : 0);
         4246  +  pzTail = objc>=5 ? &zTail : 0;
         4247  +  rc = sqlite3_prepare_v2(db, zCopy, bytes, &pStmt, pzTail);
  4246   4248     free(zCopy);
  4247   4249     zTail = &zSql[(zTail - zCopy)];
  4248   4250   
  4249   4251     assert(rc==SQLITE_OK || pStmt==0);
  4250   4252     Tcl_ResetResult(interp);
  4251   4253     if( sqlite3TestErrCode(interp, db, rc) ) return TCL_ERROR;
  4252   4254     if( rc==SQLITE_OK && zTail && objc>=5 ){
  4253   4255       if( bytes>=0 ){
  4254   4256         bytes = bytes - (int)(zTail-zSql);
  4255   4257       }
  4256   4258       Tcl_ObjSetVar2(interp, objv[4], 0, Tcl_NewStringObj(zTail, bytes), 0);
  4257   4259     }
         4260  +  if( rc!=SQLITE_OK ){
         4261  +    assert( pStmt==0 );
         4262  +    sqlite3_snprintf(sizeof(zBuf), zBuf, "(%d) ", rc);
         4263  +    Tcl_AppendResult(interp, zBuf, sqlite3_errmsg(db), 0);
         4264  +    return TCL_ERROR;
         4265  +  }
         4266  +
         4267  +  if( pStmt ){
         4268  +    if( sqlite3TestMakePointerStr(interp, zBuf, pStmt) ) return TCL_ERROR;
         4269  +    Tcl_AppendResult(interp, zBuf, 0);
         4270  +  }
         4271  +  return TCL_OK;
         4272  +}
         4273  +
         4274  +/*
         4275  +** Usage: sqlite3_prepare_v3 DB sql bytes flags ?tailvar?
         4276  +**
         4277  +** Compile up to <bytes> bytes of the supplied SQL string <sql> using
         4278  +** database handle <DB> and flags <flags>. The parameter <tailval> is
         4279  +** the name of a global variable that is set to the unused portion of
         4280  +** <sql> (if any). A STMT handle is returned.
         4281  +*/
         4282  +static int SQLITE_TCLAPI test_prepare_v3(
         4283  +  void * clientData,
         4284  +  Tcl_Interp *interp,
         4285  +  int objc,
         4286  +  Tcl_Obj *CONST objv[]
         4287  +){
         4288  +  sqlite3 *db;
         4289  +  const char *zSql;
         4290  +  char *zCopy = 0;                /* malloc() copy of zSql */
         4291  +  int bytes, flags;
         4292  +  const char *zTail = 0;
         4293  +  const char **pzTail;
         4294  +  sqlite3_stmt *pStmt = 0;
         4295  +  char zBuf[50];
         4296  +  int rc;
         4297  +
         4298  +  if( objc!=6 && objc!=5 ){
         4299  +    Tcl_AppendResult(interp, "wrong # args: should be \"", 
         4300  +       Tcl_GetString(objv[0]), " DB sql bytes flags tailvar", 0);
         4301  +    return TCL_ERROR;
         4302  +  }
         4303  +  if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
         4304  +  zSql = Tcl_GetString(objv[2]);
         4305  +  if( Tcl_GetIntFromObj(interp, objv[3], &bytes) ) return TCL_ERROR;
         4306  +  if( Tcl_GetIntFromObj(interp, objv[4], &flags) ) return TCL_ERROR;
         4307  +
         4308  +  /* Instead of using zSql directly, make a copy into a buffer obtained
         4309  +  ** directly from malloc(). The idea is to make it easier for valgrind
         4310  +  ** to spot buffer overreads.  */
         4311  +  if( bytes>=0 ){
         4312  +    zCopy = malloc(bytes);
         4313  +    memcpy(zCopy, zSql, bytes);
         4314  +  }else{
         4315  +    int n = (int)strlen(zSql) + 1;
         4316  +    zCopy = malloc(n);
         4317  +    memcpy(zCopy, zSql, n);
         4318  +  }
         4319  +  pzTail = objc>=6 ? &zTail : 0;
         4320  +  rc = sqlite3_prepare_v3(db, zCopy, bytes, (unsigned int)flags,&pStmt,pzTail);
         4321  +  free(zCopy);
         4322  +  zTail = &zSql[(zTail - zCopy)];
         4323  +
         4324  +  assert(rc==SQLITE_OK || pStmt==0);
         4325  +  Tcl_ResetResult(interp);
         4326  +  if( sqlite3TestErrCode(interp, db, rc) ) return TCL_ERROR;
         4327  +  if( rc==SQLITE_OK && zTail && objc>=6 ){
         4328  +    if( bytes>=0 ){
         4329  +      bytes = bytes - (int)(zTail-zSql);
         4330  +    }
         4331  +    Tcl_ObjSetVar2(interp, objv[5], 0, Tcl_NewStringObj(zTail, bytes), 0);
         4332  +  }
  4258   4333     if( rc!=SQLITE_OK ){
  4259   4334       assert( pStmt==0 );
  4260   4335       sqlite3_snprintf(sizeof(zBuf), zBuf, "(%d) ", rc);
  4261   4336       Tcl_AppendResult(interp, zBuf, sqlite3_errmsg(db), 0);
  4262   4337       return TCL_ERROR;
  4263   4338     }
  4264   4339   
................................................................................
  4672   4747   
  4673   4748     if( getStmtPointer(interp, Tcl_GetString(objv[1]), &pStmt) ) return TCL_ERROR;
  4674   4749     z = sqlite3_expanded_sql(pStmt);
  4675   4750     Tcl_SetResult(interp, z, TCL_VOLATILE);
  4676   4751     sqlite3_free(z);
  4677   4752     return TCL_OK;
  4678   4753   }
         4754  +#ifdef SQLITE_ENABLE_NORMALIZE
         4755  +static int SQLITE_TCLAPI test_norm_sql(
         4756  +  void * clientData,
         4757  +  Tcl_Interp *interp,
         4758  +  int objc,
         4759  +  Tcl_Obj *CONST objv[]
         4760  +){
         4761  +  sqlite3_stmt *pStmt;
         4762  +
         4763  +  if( objc!=2 ){
         4764  +    Tcl_WrongNumArgs(interp, 1, objv, "STMT");
         4765  +    return TCL_ERROR;
         4766  +  }
         4767  +
         4768  +  if( getStmtPointer(interp, Tcl_GetString(objv[1]), &pStmt) ) return TCL_ERROR;
         4769  +  Tcl_SetResult(interp, (char *)sqlite3_normalized_sql(pStmt), TCL_VOLATILE);
         4770  +  return TCL_OK;
         4771  +}
         4772  +#endif /* SQLITE_ENABLE_NORMALIZE */
  4679   4773   
  4680   4774   /*
  4681   4775   ** Usage: sqlite3_column_count STMT 
  4682   4776   **
  4683   4777   ** Return the number of columns returned by the sql statement STMT.
  4684   4778   */
  4685   4779   static int SQLITE_TCLAPI test_column_count(
................................................................................
  7458   7552       { "TRIGGER",         SQLITE_DBCONFIG_ENABLE_TRIGGER },
  7459   7553       { "FTS3_TOKENIZER",  SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER },
  7460   7554       { "LOAD_EXTENSION",  SQLITE_DBCONFIG_ENABLE_LOAD_EXTENSION },
  7461   7555       { "NO_CKPT_ON_CLOSE",SQLITE_DBCONFIG_NO_CKPT_ON_CLOSE },
  7462   7556       { "QPSG",            SQLITE_DBCONFIG_ENABLE_QPSG },
  7463   7557       { "TRIGGER_EQP",     SQLITE_DBCONFIG_TRIGGER_EQP },
  7464   7558       { "RESET_DB",        SQLITE_DBCONFIG_RESET_DATABASE },
         7559  +    { "DEFENSIVE",       SQLITE_DBCONFIG_DEFENSIVE },
  7465   7560     };
  7466   7561     int i;
  7467   7562     int v;
  7468   7563     const char *zSetting;
  7469   7564     sqlite3 *db;
  7470   7565   
  7471   7566     if( objc!=4 ){
................................................................................
  7642   7737        { "sqlite3_open_v2",               test_open_v2       ,0 },
  7643   7738        { "sqlite3_complete16",            test_complete16    ,0 },
  7644   7739        { "sqlite3_normalize",             test_normalize     ,0 },
  7645   7740   
  7646   7741        { "sqlite3_prepare",               test_prepare       ,0 },
  7647   7742        { "sqlite3_prepare16",             test_prepare16     ,0 },
  7648   7743        { "sqlite3_prepare_v2",            test_prepare_v2    ,0 },
         7744  +     { "sqlite3_prepare_v3",            test_prepare_v3    ,0 },
  7649   7745        { "sqlite3_prepare_tkt3134",       test_prepare_tkt3134, 0},
  7650   7746        { "sqlite3_prepare16_v2",          test_prepare16_v2  ,0 },
  7651   7747        { "sqlite3_finalize",              test_finalize      ,0 },
  7652   7748        { "sqlite3_stmt_status",           test_stmt_status   ,0 },
  7653   7749        { "sqlite3_reset",                 test_reset         ,0 },
  7654   7750        { "sqlite3_expired",               test_expired       ,0 },
  7655   7751        { "sqlite3_transfer_bindings",     test_transfer_bind ,0 },
  7656   7752        { "sqlite3_changes",               test_changes       ,0 },
  7657   7753        { "sqlite3_step",                  test_step          ,0 },
  7658   7754        { "sqlite3_sql",                   test_sql           ,0 },
  7659   7755        { "sqlite3_expanded_sql",          test_ex_sql        ,0 },
         7756  +#ifdef SQLITE_ENABLE_NORMALIZE
         7757  +     { "sqlite3_normalized_sql",        test_norm_sql      ,0 },
         7758  +#endif
  7660   7759        { "sqlite3_next_stmt",             test_next_stmt     ,0 },
  7661   7760        { "sqlite3_stmt_readonly",         test_stmt_readonly ,0 },
  7662   7761        { "sqlite3_stmt_busy",             test_stmt_busy     ,0 },
  7663   7762        { "uses_stmt_journal",             uses_stmt_journal ,0 },
  7664   7763   
  7665   7764        { "sqlite3_release_memory",        test_release_memory,     0},
  7666   7765        { "sqlite3_db_release_memory",     test_db_release_memory,  0},

Changes to src/test_config.c.

   757    757   #endif
   758    758   
   759    759   #ifdef SQLITE_ENABLE_URI_00_ERROR
   760    760     Tcl_SetVar2(interp, "sqlite_options", "uri_00_error", "1", TCL_GLOBAL_ONLY);
   761    761   #else
   762    762     Tcl_SetVar2(interp, "sqlite_options", "uri_00_error", "0", TCL_GLOBAL_ONLY);
   763    763   #endif
          764  +
          765  +#if defined(SQLITE_ENABLE_NORMALIZE)
          766  +  Tcl_SetVar2(interp, "sqlite_options", "normalize", "1", TCL_GLOBAL_ONLY);
          767  +#else
          768  +  Tcl_SetVar2(interp, "sqlite_options", "normalize", "0", TCL_GLOBAL_ONLY);
          769  +#endif
   764    770   
   765    771   #ifdef SQLITE_OMIT_WINDOWFUNC
   766    772     Tcl_SetVar2(interp, "sqlite_options", "windowfunc", "0", TCL_GLOBAL_ONLY);
   767    773   #else
   768    774     Tcl_SetVar2(interp, "sqlite_options", "windowfunc", "1", TCL_GLOBAL_ONLY);
   769    775   #endif
   770    776   

Changes to src/test_intarray.c.

     9      9   **    May you share freely, never taking more than you give.
    10     10   **
    11     11   *************************************************************************
    12     12   **
    13     13   ** This file implements a read-only VIRTUAL TABLE that contains the
    14     14   ** content of a C-language array of integer values.  See the corresponding
    15     15   ** header file for full details.
           16  +**
           17  +** This virtual table is used for internal testing of SQLite only.  It is
           18  +** not recommended for use in production.  For a similar virtual table that
           19  +** is production-ready, see the "carray" virtual table over in ext/misc.
    16     20   */
    17     21   #include "test_intarray.h"
    18     22   #include <string.h>
    19     23   #include <assert.h>
    20     24   
    21     25   
    22     26   /*
................................................................................
   311    315     }
   312    316     if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
   313    317     zName = Tcl_GetString(objv[2]);
   314    318   #ifndef SQLITE_OMIT_VIRTUALTABLE
   315    319     rc = sqlite3_intarray_create(db, zName, &pArray);
   316    320   #endif
   317    321     if( rc!=SQLITE_OK ){
   318         -    assert( pArray==0 );
   319    322       Tcl_AppendResult(interp, sqlite3ErrName(rc), (char*)0);
   320    323       return TCL_ERROR;
   321    324     }
   322    325     sqlite3TestMakePointerStr(interp, zPtr, pArray);
   323    326     Tcl_AppendResult(interp, zPtr, (char*)0);
   324    327     return TCL_OK;
   325    328   }

Changes to src/test_intarray.h.

     9      9   **    May you share freely, never taking more than you give.
    10     10   **
    11     11   *************************************************************************
    12     12   **
    13     13   ** This is the C-language interface definition for the "intarray" or
    14     14   ** integer array virtual table for SQLite.
    15     15   **
           16  +** This virtual table is used for internal testing of SQLite only.  It is
           17  +** not recommended for use in production.  For a similar virtual table that
           18  +** is production-ready, see the "carray" virtual table over in ext/misc.
           19  +**
    16     20   ** The intarray virtual table is designed to facilitate using an
    17     21   ** array of integers as the right-hand side of an IN operator.  So
    18     22   ** instead of doing a prepared statement like this:
    19     23   **
    20     24   **     SELECT * FROM table WHERE x IN (?,?,?,...,?);
    21     25   **
    22     26   ** And then binding indivdual integers to each of ? slots, a C-language
................................................................................
    68     72   ** longer needed.  The application must not change the intarray values
    69     73   ** while an intarray is in the middle of a query.
    70     74   **
    71     75   ** The intarray object is automatically destroyed when its corresponding
    72     76   ** virtual table is dropped.  Since the virtual tables are created in the
    73     77   ** TEMP database, they are automatically dropped when the database connection
    74     78   ** closes so the application does not normally need to take any special
    75         -** action to free the intarray objects.
           79  +** action to free the intarray objects.  Because of the way virtual tables
           80  +** work and the (somewhat goofy) way that the intarray virtual table is
           81  +** implemented, it is not allowed to invoke sqlite3_intarray_create(D,N,P)
           82  +** more than once with the same D and N values.
    76     83   */
    77     84   #include "sqlite3.h"
    78     85   #ifndef SQLITE_INTARRAY_H
    79     86   #define SQLITE_INTARRAY_H
    80     87   
    81     88   /*
    82     89   ** Make sure we can call this stuff from C++.

Changes to src/test_onefile.c.

   556    556     return SQLITE_OK;
   557    557   }
   558    558   
   559    559   /*
   560    560   ** File control method. For custom operations on an fs-file.
   561    561   */
   562    562   static int fsFileControl(sqlite3_file *pFile, int op, void *pArg){
          563  +  if( op==SQLITE_FCNTL_PRAGMA ) return SQLITE_NOTFOUND;
   563    564     return SQLITE_OK;
   564    565   }
   565    566   
   566    567   /*
   567    568   ** Return the sector-size in bytes for an fs-file.
   568    569   */
   569    570   static int fsSectorSize(sqlite3_file *pFile){

Changes to src/tokenize.c.

   541    541       }
   542    542     }
   543    543     while( IdChar(z[i]) ){ i++; }
   544    544     *tokenType = TK_ID;
   545    545     return i;
   546    546   }
   547    547   
          548  +#ifdef SQLITE_ENABLE_NORMALIZE
          549  +/*
          550  +** Return the length (in bytes) of the token that begins at z[0].
          551  +** Store the token type in *tokenType before returning.  If flags has
          552  +** SQLITE_TOKEN_NORMALIZE flag enabled, use the identifier token type
          553  +** for keywords.  Add SQLITE_TOKEN_QUOTED to flags if the token was
          554  +** actually a quoted identifier.  Add SQLITE_TOKEN_KEYWORD to flags
          555  +** if the token was recognized as a keyword; this is useful when the
          556  +** SQLITE_TOKEN_NORMALIZE flag is used, because it enables the caller
          557  +** to differentiate between a keyword being treated as an identifier
          558  +** (for normalization purposes) and an actual identifier.
          559  +*/
          560  +int sqlite3GetTokenNormalized(
          561  +  const unsigned char *z,
          562  +  int *tokenType,
          563  +  int *flags
          564  +){
          565  +  int n;
          566  +  unsigned char iClass = aiClass[*z];
          567  +  if( iClass==CC_KYWD ){
          568  +    int i;
          569  +    for(i=1; aiClass[z[i]]<=CC_KYWD; i++){}
          570  +    if( IdChar(z[i]) ){
          571  +      /* This token started out using characters that can appear in keywords,
          572  +      ** but z[i] is a character not allowed within keywords, so this must
          573  +      ** be an identifier instead */
          574  +      i++;
          575  +      while( IdChar(z[i]) ){ i++; }
          576  +      *tokenType = TK_ID;
          577  +      return i;
          578  +    }
          579  +    *tokenType = TK_ID;
          580  +    n = keywordCode((char*)z, i, tokenType);
          581  +    /* If the token is no longer considered to be an identifier, then it is a
          582  +    ** keyword of some kind.  Make the token back into an identifier and then
          583  +    ** set the SQLITE_TOKEN_KEYWORD flag.  Several non-identifier tokens are
          584  +    ** used verbatim, including IN, IS, NOT, and NULL. */
          585  +    switch( *tokenType ){
          586  +      case TK_ID: {
          587  +        /* do nothing, handled by caller */
          588  +        break;
          589  +      }
          590  +      case TK_IN:
          591  +      case TK_IS:
          592  +      case TK_NOT:
          593  +      case TK_NULL: {
          594  +        *flags |= SQLITE_TOKEN_KEYWORD;
          595  +        break;
          596  +      }
          597  +      default: {
          598  +        *tokenType = TK_ID;
          599  +        *flags |= SQLITE_TOKEN_KEYWORD;
          600  +        break;
          601  +      }
          602  +    }
          603  +  }else{
          604  +    n = sqlite3GetToken(z, tokenType);
          605  +    /* If the token is considered to be an identifier and the character class
          606  +    ** of the first character is a quote, set the SQLITE_TOKEN_QUOTED flag. */
          607  +    if( *tokenType==TK_ID && (iClass==CC_QUOTE || iClass==CC_QUOTE2) ){
          608  +      *flags |= SQLITE_TOKEN_QUOTED;
          609  +    }
          610  +  }
          611  +  return n;
          612  +}
          613  +#endif /* SQLITE_ENABLE_NORMALIZE */
          614  +
   548    615   /*
   549    616   ** Run the parser on the given SQL string.  The parser structure is
   550    617   ** passed in.  An SQLITE_ status code is returned.  If an error occurs
   551    618   ** then an and attempt is made to write an error message into 
   552    619   ** memory obtained from sqlite3_malloc() and to make *pzErrMsg point to that
   553    620   ** error message.
   554    621   */

Changes to src/treeview.c.

   117    117         sqlite3TreeViewSelect(pView, pCte->pSelect, 0);
   118    118         sqlite3TreeViewPop(pView);
   119    119       }
   120    120       sqlite3TreeViewPop(pView);
   121    121     }
   122    122   }
   123    123   
          124  +/*
          125  +** Generate a human-readable description of a SrcList object.
          126  +*/
          127  +void sqlite3TreeViewSrcList(TreeView *pView, const SrcList *pSrc){
          128  +  int i;
          129  +  for(i=0; i<pSrc->nSrc; i++){
          130  +    const struct SrcList_item *pItem = &pSrc->a[i];
          131  +    StrAccum x;
          132  +    char zLine[100];
          133  +    sqlite3StrAccumInit(&x, 0, zLine, sizeof(zLine), 0);
          134  +    sqlite3_str_appendf(&x, "{%d,*}", pItem->iCursor);
          135  +    if( pItem->zDatabase ){
          136  +      sqlite3_str_appendf(&x, " %s.%s", pItem->zDatabase, pItem->zName);
          137  +    }else if( pItem->zName ){
          138  +      sqlite3_str_appendf(&x, " %s", pItem->zName);
          139  +    }
          140  +    if( pItem->pTab ){
          141  +      sqlite3_str_appendf(&x, " tabname=%Q", pItem->pTab->zName);
          142  +    }
          143  +    if( pItem->zAlias ){
          144  +      sqlite3_str_appendf(&x, " (AS %s)", pItem->zAlias);
          145  +    }
          146  +    if( pItem->fg.jointype & JT_LEFT ){
          147  +      sqlite3_str_appendf(&x, " LEFT-JOIN");
          148  +    }
          149  +    sqlite3StrAccumFinish(&x);
          150  +    sqlite3TreeViewItem(pView, zLine, i<pSrc->nSrc-1); 
          151  +    if( pItem->pSelect ){
          152  +      sqlite3TreeViewSelect(pView, pItem->pSelect, 0);
          153  +    }
          154  +    if( pItem->fg.isTabFunc ){
          155  +      sqlite3TreeViewExprList(pView, pItem->u1.pFuncArg, 0, "func-args:");
          156  +    }
          157  +    sqlite3TreeViewPop(pView);
          158  +  }
          159  +}
   124    160   
   125    161   /*
   126    162   ** Generate a human-readable description of a Select object.
   127    163   */
   128    164   void sqlite3TreeViewSelect(TreeView *pView, const Select *p, u8 moreToFollow){
   129    165     int n = 0;
   130    166     int cnt = 0;
................................................................................
   171    207         for(pX=p->pWin; pX; pX=pX->pNextWin){
   172    208           sqlite3TreeViewWinFunc(pView, pX, pX->pNextWin!=0);
   173    209         }
   174    210         sqlite3TreeViewPop(pView);
   175    211       }
   176    212   #endif
   177    213       if( p->pSrc && p->pSrc->nSrc ){
   178         -      int i;
   179    214         pView = sqlite3TreeViewPush(pView, (n--)>0);
   180    215         sqlite3TreeViewLine(pView, "FROM");
   181         -      for(i=0; i<p->pSrc->nSrc; i++){
   182         -        struct SrcList_item *pItem = &p->pSrc->a[i];
   183         -        StrAccum x;
   184         -        char zLine[100];
   185         -        sqlite3StrAccumInit(&x, 0, zLine, sizeof(zLine), 0);
   186         -        sqlite3_str_appendf(&x, "{%d,*}", pItem->iCursor);
   187         -        if( pItem->zDatabase ){
   188         -          sqlite3_str_appendf(&x, " %s.%s", pItem->zDatabase, pItem->zName);
   189         -        }else if( pItem->zName ){
   190         -          sqlite3_str_appendf(&x, " %s", pItem->zName);
   191         -        }
   192         -        if( pItem->pTab ){
   193         -          sqlite3_str_appendf(&x, " tabname=%Q", pItem->pTab->zName);
   194         -        }
   195         -        if( pItem->zAlias ){
   196         -          sqlite3_str_appendf(&x, " (AS %s)", pItem->zAlias);
   197         -        }
   198         -        if( pItem->fg.jointype & JT_LEFT ){
   199         -          sqlite3_str_appendf(&x, " LEFT-JOIN");
   200         -        }
   201         -        sqlite3StrAccumFinish(&x);
   202         -        sqlite3TreeViewItem(pView, zLine, i<p->pSrc->nSrc-1); 
   203         -        if( pItem->pSelect ){
   204         -          sqlite3TreeViewSelect(pView, pItem->pSelect, 0);
   205         -        }
   206         -        if( pItem->fg.isTabFunc ){
   207         -          sqlite3TreeViewExprList(pView, pItem->u1.pFuncArg, 0, "func-args:");
   208         -        }
   209         -        sqlite3TreeViewPop(pView);
   210         -      }
          216  +      sqlite3TreeViewSrcList(pView, p->pSrc);
   211    217         sqlite3TreeViewPop(pView);
   212    218       }
   213    219       if( p->pWhere ){
   214    220         sqlite3TreeViewItem(pView, "WHERE", (n--)>0);
   215    221         sqlite3TreeViewExpr(pView, p->pWhere, 0);
   216    222         sqlite3TreeViewPop(pView);
   217    223       }

Changes to src/vdbeInt.h.

   402    402     bft usesStmtJournal:1;  /* True if uses a statement journal */
   403    403     bft readOnly:1;         /* True for statements that do not write */
   404    404     bft bIsReader:1;        /* True for statements that read */
   405    405     yDbMask btreeMask;      /* Bitmask of db->aDb[] entries referenced */
   406    406     yDbMask lockMask;       /* Subset of btreeMask that requires a lock */
   407    407     u32 aCounter[7];        /* Counters used by sqlite3_stmt_status() */
   408    408     char *zSql;             /* Text of the SQL statement that generated this */
          409  +#ifdef SQLITE_ENABLE_NORMALIZE
          410  +  char *zNormSql;         /* Normalization of the associated SQL statement */
          411  +#endif
   409    412     void *pFree;            /* Free this when deleting the vdbe */
   410    413     VdbeFrame *pFrame;      /* Parent frame */
   411    414     VdbeFrame *pDelFrame;   /* List of frame objects to free on VM reset */
   412    415     int nFrame;             /* Number of frames in pFrame list */
   413    416     u32 expmask;            /* Binding to these vars invalidates VM */
   414    417     SubProgram *pProgram;   /* Linked list of all sub-programs used by VM */
   415    418     AuxData *pAuxData;      /* Linked list of auxdata allocations */

Changes to src/vdbeapi.c.

  1698   1698       z = sqlite3VdbeExpandSql(p, zSql);
  1699   1699       sqlite3_mutex_leave(p->db->mutex);
  1700   1700     }
  1701   1701     return z;
  1702   1702   #endif
  1703   1703   }
  1704   1704   
         1705  +#ifdef SQLITE_ENABLE_NORMALIZE
         1706  +/*
         1707  +** Return the normalized SQL associated with a prepared statement.
         1708  +*/
         1709  +const char *sqlite3_normalized_sql(sqlite3_stmt *pStmt){
         1710  +  Vdbe *p = (Vdbe *)pStmt;
         1711  +  return p ? p->zNormSql : 0;
         1712  +}
         1713  +#endif /* SQLITE_ENABLE_NORMALIZE */
         1714  +
  1705   1715   #ifdef SQLITE_ENABLE_PREUPDATE_HOOK
  1706   1716   /*
  1707   1717   ** Allocate and populate an UnpackedRecord structure based on the serialized
  1708   1718   ** record in nKey/pKey. Return a pointer to the new UnpackedRecord structure
  1709   1719   ** if successful, or a NULL pointer if an OOM error is encountered.
  1710   1720   */
  1711   1721   static UnpackedRecord *vdbeUnpackRecord(

Changes to src/vdbeaux.c.

    60     60     if( p==0 ) return;
    61     61     p->prepFlags = prepFlags;
    62     62     if( (prepFlags & SQLITE_PREPARE_SAVESQL)==0 ){
    63     63       p->expmask = 0;
    64     64     }
    65     65     assert( p->zSql==0 );
    66     66     p->zSql = sqlite3DbStrNDup(p->db, z, n);
           67  +#ifdef SQLITE_ENABLE_NORMALIZE
           68  +  assert( p->zNormSql==0 );
           69  +  if( p->zSql && (prepFlags & SQLITE_PREPARE_NORMALIZE)!=0 ){
           70  +    sqlite3Normalize(p, p->zSql, n, prepFlags);
           71  +    assert( p->zNormSql!=0 || p->db->mallocFailed );
           72  +  }
           73  +#endif
    67     74   }
    68     75   
    69     76   /*
    70     77   ** Swap all content between two VDBE structures.
    71     78   */
    72     79   void sqlite3VdbeSwap(Vdbe *pA, Vdbe *pB){
    73     80     Vdbe tmp, *pTmp;
................................................................................
    81     88     pB->pNext = pTmp;
    82     89     pTmp = pA->pPrev;
    83     90     pA->pPrev = pB->pPrev;
    84     91     pB->pPrev = pTmp;
    85     92     zTmp = pA->zSql;
    86     93     pA->zSql = pB->zSql;
    87     94     pB->zSql = zTmp;
           95  +#ifdef SQLITE_ENABLE_NORMALIZE
           96  +  zTmp = pA->zNormSql;
           97  +  pA->zNormSql = pB->zNormSql;
           98  +  pB->zNormSql = zTmp;
           99  +#endif
    88    100     pB->expmask = pA->expmask;
    89    101     pB->prepFlags = pA->prepFlags;
    90    102     memcpy(pB->aCounter, pA->aCounter, sizeof(pB->aCounter));
    91    103     pB->aCounter[SQLITE_STMTSTATUS_REPREPARE]++;
    92    104   }
    93    105   
    94    106   /*
................................................................................
  3154   3166       releaseMemArray(p->aVar, p->nVar);
  3155   3167       sqlite3DbFree(db, p->pVList);
  3156   3168       sqlite3DbFree(db, p->pFree);
  3157   3169     }
  3158   3170     vdbeFreeOpArray(db, p->aOp, p->nOp);
  3159   3171     sqlite3DbFree(db, p->aColName);
  3160   3172     sqlite3DbFree(db, p->zSql);
         3173  +#ifdef SQLITE_ENABLE_NORMALIZE
         3174  +  sqlite3DbFree(db, p->zNormSql);
         3175  +#endif
  3161   3176   #ifdef SQLITE_ENABLE_STMT_SCANSTATUS
  3162   3177     {
  3163   3178       int i;
  3164   3179       for(i=0; i<p->nScan; i++){
  3165   3180         sqlite3DbFree(db, p->aScan[i].zName);
  3166   3181       }
  3167   3182       sqlite3DbFree(db, p->aScan);

Changes to src/vdbemem.c.

   373    373     */
   374    374     if( fg & MEM_Int ){
   375    375       sqlite3_snprintf(nByte, pMem->z, "%lld", pMem->u.i);
   376    376     }else{
   377    377       assert( fg & MEM_Real );
   378    378       sqlite3_snprintf(nByte, pMem->z, "%!.15g", pMem->u.r);
   379    379     }
   380         -  pMem->n = sqlite3Strlen30(pMem->z);
          380  +  assert( pMem->z!=0 );
          381  +  pMem->n = sqlite3Strlen30NN(pMem->z);
   381    382     pMem->enc = SQLITE_UTF8;
   382    383     pMem->flags |= MEM_Str|MEM_Term;
   383    384     if( bForce ) pMem->flags &= ~(MEM_Int|MEM_Real);
   384    385     sqlite3VdbeChangeEncoding(pMem, enc);
   385    386     return SQLITE_OK;
   386    387   }
   387    388   

Changes to src/where.c.

   850    850     if( pPartial ) sqlite3VdbeResolveLabel(v, iContinue);
   851    851     if( pTabItem->fg.viaCoroutine ){
   852    852       sqlite3VdbeChangeP2(v, addrCounter, regBase+n);
   853    853       testcase( pParse->db->mallocFailed );
   854    854       translateColumnToCopy(pParse, addrTop, pLevel->iTabCur,
   855    855                             pTabItem->regResult, 1);
   856    856       sqlite3VdbeGoto(v, addrTop);
   857         -    pTabItem->fg.viaCoroutine = 0;
   858    857     }else{
   859    858       sqlite3VdbeAddOp2(v, OP_Next, pLevel->iTabCur, addrTop+1); VdbeCoverage(v);
   860    859     }
   861    860     sqlite3VdbeChangeP5(v, SQLITE_STMTSTATUS_AUTOINDEX);
   862    861     sqlite3VdbeJumpHere(v, addrTop);
   863    862     sqlite3ReleaseTempReg(pParse, regRecord);
   864    863     

Changes to src/wherecode.c.

   421    421       ExprList *pLhs = 0;         /* New LHS after mods */
   422    422       int i;                      /* Loop counter */
   423    423       Select *pSelect;            /* Pointer to the SELECT on the RHS */
   424    424   
   425    425       for(i=iEq; i<pLoop->nLTerm; i++){
   426    426         if( pLoop->aLTerm[i]->pExpr==pX ){
   427    427           int iField = pLoop->aLTerm[i]->iField - 1;
   428         -        assert( pOrigRhs->a[iField].pExpr!=0 );
          428  +        if( pOrigRhs->a[iField].pExpr==0 ) continue; /* Duplicate PK column */
   429    429           pRhs = sqlite3ExprListAppend(pParse, pRhs, pOrigRhs->a[iField].pExpr);
   430    430           pOrigRhs->a[iField].pExpr = 0;
   431    431           assert( pOrigLhs->a[iField].pExpr!=0 );
   432    432           pLhs = sqlite3ExprListAppend(pParse, pLhs, pOrigLhs->a[iField].pExpr);
   433    433           pOrigLhs->a[iField].pExpr = 0;
   434    434         }
   435    435       }

Changes to src/whereexpr.c.

  1378   1378     ** virtual term of that form.
  1379   1379     **
  1380   1380     ** Note that the virtual term must be tagged with TERM_VNULL.
  1381   1381     */
  1382   1382     if( pExpr->op==TK_NOTNULL
  1383   1383      && pExpr->pLeft->op==TK_COLUMN
  1384   1384      && pExpr->pLeft->iColumn>=0
         1385  +   && !ExprHasProperty(pExpr, EP_FromJoin)
  1385   1386      && OptimizationEnabled(db, SQLITE_Stat34)
  1386   1387     ){
  1387   1388       Expr *pNewExpr;
  1388   1389       Expr *pLeft = pExpr->pLeft;
  1389   1390       int idxNew;
  1390   1391       WhereTerm *pNewTerm;
  1391   1392   
................................................................................
  1569   1570     Expr *pTerm;
  1570   1571     if( pItem->fg.isTabFunc==0 ) return;
  1571   1572     pTab = pItem->pTab;
  1572   1573     assert( pTab!=0 );
  1573   1574     pArgs = pItem->u1.pFuncArg;
  1574   1575     if( pArgs==0 ) return;
  1575   1576     for(j=k=0; j<pArgs->nExpr; j++){
         1577  +    Expr *pRhs;
  1576   1578       while( k<pTab->nCol && (pTab->aCol[k].colFlags & COLFLAG_HIDDEN)==0 ){k++;}
  1577   1579       if( k>=pTab->nCol ){
  1578   1580         sqlite3ErrorMsg(pParse, "too many arguments on %s() - max %d",
  1579   1581                         pTab->zName, j);
  1580   1582         return;
  1581   1583       }
  1582   1584       pColRef = sqlite3ExprAlloc(pParse->db, TK_COLUMN, 0, 0);
  1583   1585       if( pColRef==0 ) return;
  1584   1586       pColRef->iTable = pItem->iCursor;
  1585   1587       pColRef->iColumn = k++;
  1586   1588       pColRef->y.pTab = pTab;
  1587         -    pTerm = sqlite3PExpr(pParse, TK_EQ, pColRef,
  1588         -                         sqlite3ExprDup(pParse->db, pArgs->a[j].pExpr, 0));
         1589  +    pRhs = sqlite3PExpr(pParse, TK_UPLUS, 
         1590  +        sqlite3ExprDup(pParse->db, pArgs->a[j].pExpr, 0), 0);
         1591  +    pTerm = sqlite3PExpr(pParse, TK_EQ, pColRef, pRhs);
  1589   1592       whereClauseInsert(pWC, pTerm, TERM_DYNAMIC);
  1590   1593     }
  1591   1594   }

Changes to src/window.c.

   742    742   ** any SQL window functions, this function is a no-op. Otherwise, it 
   743    743   ** rewrites the SELECT statement so that window function xStep functions
   744    744   ** are invoked in the correct order as described under "SELECT REWRITING"
   745    745   ** at the top of this file.
   746    746   */
   747    747   int sqlite3WindowRewrite(Parse *pParse, Select *p){
   748    748     int rc = SQLITE_OK;
   749         -  if( p->pWin ){
          749  +  if( p->pWin && p->pPrior==0 ){
   750    750       Vdbe *v = sqlite3GetVdbe(pParse);
   751    751       sqlite3 *db = pParse->db;
   752    752       Select *pSub = 0;             /* The subquery */
   753    753       SrcList *pSrc = p->pSrc;
   754    754       Expr *pWhere = p->pWhere;
   755    755       ExprList *pGroupBy = p->pGroupBy;
   756    756       Expr *pHaving = p->pHaving;

Changes to test/attach.test.

   722    722       catchsql {
   723    723         CREATE TRIGGER r5 AFTER INSERT ON t5 BEGIN
   724    724           DELETE FROM t1 WHERE x<(SELECT min(x) FROM temp.t6);
   725    725         END;
   726    726       } db2
   727    727     } {1 {trigger r5 cannot reference objects in database temp}}
   728    728   } ;# endif subquery
          729  +ifcapable json1 {
          730  +  do_test attach-5.10 {
          731  +    db close
          732  +    forcedelete test.db
          733  +    sqlite3 db test.db
          734  +    db eval {
          735  +      CREATE TABLE t1(x);
          736  +      CREATE TABLE t2(a,b);
          737  +      CREATE TRIGGER x1 AFTER INSERT ON t1 BEGIN
          738  +        INSERT INTO t2(a,b) SELECT key, value FROM json_each(NEW.x);
          739  +      END;
          740  +      INSERT INTO t1(x) VALUES('{"a":1}');
          741  +      SELECT * FROM t2;
          742  +    }
          743  +  } {a 1}
          744  +  do_test attach-5.11 {
          745  +    sqlite3 db2 :memory:
          746  +    db2 eval {
          747  +      CREATE TABLE t3(y);
          748  +      ATTACH 'test.db' AS aux;
          749  +      INSERT INTO aux.t1(x) VALUES('{"b":2}');
          750  +      SELECT * FROM aux.t2;
          751  +    }
          752  +  } {a 1 b 2}
          753  +} ;# endif json1
   729    754   } ;# endif trigger
   730    755   
   731    756   # Check to make sure we get a sensible error if unable to open
   732    757   # the file that we are trying to attach.
   733    758   #
   734    759   do_test attach-6.1 {
   735    760     catchsql {

Changes to test/autoindex5.test.

   119    119       UNION ALL
   120    120       SELECT 0, 0 WHERE 0;
   121    121   
   122    122     SELECT (
   123    123         SELECT sum(z) FROM vvv WHERE x='aaa'
   124    124     ) FROM one;
   125    125   } {8.0}
          126  +
          127  +# Ticket https://www.sqlite.org/src/info/787fa716be3a7f65
          128  +# Segfault due to multiple uses of the same subquery where the
          129  +# subquery is implemented via coroutine.
          130  +#
          131  +sqlite3 db :memory:
          132  +do_execsql_test 3.0 {
          133  +  -- This is the original test case reported on the mailing list
          134  +  CREATE TABLE artists (
          135  +    id integer NOT NULL PRIMARY KEY AUTOINCREMENT,
          136  +    name varchar(255)
          137  +  );
          138  +  CREATE TABLE albums (
          139  +    id integer NOT NULL PRIMARY KEY AUTOINCREMENT,
          140  +    name varchar(255),
          141  +    artist_id integer REFERENCES artists
          142  +  );
          143  +  INSERT INTO artists (name) VALUES ('Ar');
          144  +  INSERT INTO albums (name, artist_id) VALUES ('Al', 1);
          145  +  SELECT artists.*
          146  +  FROM artists
          147  +  INNER JOIN artists AS 'b' ON (b.id = artists.id)
          148  +  WHERE (artists.id IN (
          149  +    SELECT albums.artist_id
          150  +    FROM albums
          151  +    WHERE ((name = 'Al')
          152  +      AND (albums.artist_id IS NOT NULL)
          153  +      AND (albums.id IN (
          154  +        SELECT id
          155  +        FROM (
          156  +          SELECT albums.id,
          157  +                 row_number() OVER (
          158  +                   PARTITION BY albums.artist_id
          159  +                   ORDER BY name
          160  +                 ) AS 'x'
          161  +          FROM albums
          162  +          WHERE (name = 'Al')
          163  +        ) AS 't1'
          164  +        WHERE (x = 1)
          165  +      ))
          166  +      AND (albums.id IN (1, 2)))
          167  +  ));
          168  +} {1 Ar}
          169  +# The remaining test cases were discovered (by Dan) during trouble-shooting
          170  +sqlite3 db :memory:
          171  +do_execsql_test 3.1 {
          172  +  CREATE TABLE t1 (a); INSERT INTO t1 (a) VALUES (104);
          173  +  CREATE TABLE t2 (b); INSERT INTO t2 (b) VALUES (104);
          174  +  CREATE TABLE t3 (c); INSERT INTO t3 (c) VALUES (104);
          175  +  CREATE TABLE t4 (d); INSERT INTO t4 (d) VALUES (104);
          176  +  SELECT *
          177  +  FROM t1 CROSS JOIN t2 ON (t1.a = t2.b) WHERE t2.b IN (
          178  +    SELECT t3.c
          179  +    FROM t3
          180  +    WHERE t3.c IN (
          181  +      SELECT d FROM (SELECT DISTINCT d FROM t4) AS x WHERE x.d=104
          182  +    )
          183  +  );
          184  +} {104 104}
          185  +sqlite3 db :memory:
          186  +do_execsql_test 3.2 {
          187  +  CREATE TABLE t5(a, b, c, d);
          188  +  CREATE INDEX t5a ON t5(a);
          189  +  CREATE INDEX t5b ON t5(b);
          190  +  CREATE TABLE t6(e);
          191  +  INSERT INTO t6 VALUES(1);
          192  +  INSERT INTO t5 VALUES(1,1,1,1), (2,2,2,2);
          193  +  SELECT * FROM t5 WHERE (a=1 OR b=2) AND c IN (
          194  +    SELECT e FROM (SELECT DISTINCT e FROM t6) WHERE e=1
          195  +  );
          196  +} {1 1 1 1}
          197  +sqlite3 db :memory:
          198  +do_execsql_test 3.3 {
          199  +  CREATE TABLE t1(a1, a2, a3);
          200  +  CREATE INDEX t1a2 ON t1(a2, a1);
          201  +  CREATE INDEX t1a3 ON t1(a3, a1);
          202  +  CREATE TABLE t2(d);
          203  +  INSERT INTO t1 VALUES(3, 1, 1), (3, 2, 2);
          204  +  INSERT INTO t2 VALUES(3);
          205  +  SELECT *, 'x' FROM t1 WHERE (a2=1 OR a3=2) AND a1 = (
          206  +    SELECT d FROM (SELECT DISTINCT d FROM t2) WHERE d=3
          207  +  );
          208  +} {3 1 1 x 3 2 2 x}
          209  +
          210  +
   126    211       
   127    212   
   128    213   finish_test

Changes to test/bestindex4.test.

   112    112               set {} {}
   113    113           } {}
   114    114         }
   115    115       }
   116    116   
   117    117     }
   118    118   }
          119  +
          120  +#-------------------------------------------------------------------------
          121  +# Test that a parameter passed to a table-valued function cannot be
          122  +# used to drive an index. i.e. that in the following:
          123  +#
          124  +#   SELECT * FROM tbl, vtab(tbl.x);
          125  +#
          126  +# The implicit constraint "tbl.x = vtab.hidden" is not optimized using
          127  +# an index on tbl.x.
          128  +#
          129  +reset_db
          130  +register_tcl_module db
          131  +proc vtab_command {method args} {
          132  +  switch -- $method {
          133  +    xConnect {
          134  +      return "CREATE TABLE t1(a, b, c, d HIDDEN)"
          135  +    }
          136  +
          137  +    xBestIndex {
          138  +      set clist [lindex $args 0]
          139  +      if {[llength $clist]!=1} { error "unexpected constraint list" }
          140  +      catch { array unset C }
          141  +      array set C [lindex $clist 0]
          142  +      if {$C(usable)} {
          143  +        return [list omit 0 idxnum 555 rows 10 cost 100]
          144  +      }
          145  +      return [list cost 100000000]
          146  +    }
          147  +
          148  +  }
          149  +
          150  +  return {}
          151  +}
          152  +
          153  +do_execsql_test 2.0 {
          154  +  CREATE VIRTUAL TABLE x1 USING tcl(vtab_command);
          155  +  CREATE TABLE t1 (x INT PRIMARY KEY);
          156  +} {}
          157  +
          158  +do_execsql_test 2.1 {
          159  +  EXPLAIN QUERY PLAN SELECT * FROM t1, x1 WHERE x1.d=t1.x;
          160  +} {
          161  +  3 0 0 {SCAN TABLE x1 VIRTUAL TABLE INDEX 0:}
          162  +  7 0 0 {SEARCH TABLE t1 USING COVERING INDEX sqlite_autoindex_t1_1 (x=?)}
          163  +}
          164  +
          165  +do_execsql_test 2.2 {
          166  +  EXPLAIN QUERY PLAN SELECT * FROM t1, x1(t1.x)
          167  +} {
          168  +  3 0 0 {SCAN TABLE t1} 
          169  +  5 0 0 {SCAN TABLE x1 VIRTUAL TABLE INDEX 555:}
          170  +}
          171  +
   119    172   
   120    173   finish_test

Added test/dbfuzz2-seed1.db.

cannot compute difference between binary files

Added test/dbfuzz2.c.

            1  +/*
            2  +** 2018-10-26
            3  +**
            4  +** The author disclaims copyright to this source code.  In place of
            5  +** a legal notice, here is a blessing:
            6  +**
            7  +**    May you do good and not evil.
            8  +**    May you find forgiveness for yourself and forgive others.
            9  +**    May you share freely, never taking more than you give.
           10  +**
           11  +*************************************************************************
           12  +**
           13  +** This program is designed for fuzz-testing SQLite database files using
           14  +** the -fsanitize=fuzzer option of clang.
           15  +**
           16  +** The -fsanitize=fuzzer option causes a main() to be inserted automatically.
           17  +** That main() invokes LLVMFuzzerTestOneInput(D,S) to be invoked repeatedly.
           18  +** Each D is a fuzzed database file.  The code in this file runs various
           19  +** SQL statements against that database, trying to provoke a failure.
           20  +**
           21  +** For best results the seed database files should have these tables:
           22  +**
           23  +**   Table "t1" with columns "a" and "b"
           24  +**   Tables "t2" and "t3 with the same number of compatible columns
           25  +**       "t3" should have a column names "x"
           26  +**   Table "t4" with a column "x" that is compatible with t3.x.
           27  +**
           28  +** Any of these tables can be virtual tables, for example FTS or RTree tables.
           29  +**
           30  +** To run this test:
           31  +**
           32  +**     mkdir dir
           33  +**     cp dbfuzz2-seed*.db dir
           34  +**     clang-6.0 -I. -g -O1 -fsanitize=fuzzer \
           35  +**       -DTHREADSAFE=0 -DSQLITE_ENABLE_DESERIALIZE \
           36  +**       -DSQLITE_ENABLE_DBSTAT_VTAB dbfuzz2.c sqlite3.c -ldl
           37  +**     ./a.out dir
           38  +*/
           39  +#include <assert.h>
           40  +#include <stdio.h>
           41  +#include <stdlib.h>
           42  +#include <string.h>
           43  +#include <stdarg.h>
           44  +#include <ctype.h>
           45  +#include <stdint.h>
           46  +#include "sqlite3.h"
           47  +
           48  +/*
           49  +** This is the is the SQL that is run against the database.
           50  +*/
           51  +static const char *azSql[] = {
           52  +  "PRAGMA integrity_check;",
           53  +  "SELECT * FROM sqlite_master;",
           54  +  "SELECT sum(length(name)) FROM dbstat;",
           55  +  "UPDATE t1 SET b=a, a=b WHERE a<b;",
           56  +  "ALTER TABLE t1 RENAME TO alkjalkjdfiiiwuer987lkjwer82mx97sf98788s9789s;",
           57  +  "INSERT INTO t3 SELECT * FROM t2;",
           58  +  "DELETE FROM t3 WHERE x IN (SELECT x FROM t4);",
           59  +  "REINDEX;",
           60  +  "DROP TABLE t3;",
           61  +  "VACUUM;",
           62  +};
           63  +
           64  +/* Output verbosity level.  0 means complete silence */
           65  +int eVerbosity = 0;
           66  +
           67  +/* libFuzzer invokes this routine with fuzzed database files (in aData).
           68  +** This routine run SQLite against the malformed database to see if it
           69  +** can provoke a failure or malfunction.
           70  +*/
           71  +int LLVMFuzzerTestOneInput(const uint8_t *aData, size_t nByte){
           72  +  unsigned char *a;
           73  +  sqlite3 *db;
           74  +  int rc;
           75  +  int i;
           76  +
           77  +  if( eVerbosity>=1 ){
           78  +    printf("************** nByte=%d ***************\n", (int)nByte);
           79  +    fflush(stdout);
           80  +  }
           81  +  rc = sqlite3_open(0, &db);
           82  +  if( rc ) return 1;
           83  +  a = sqlite3_malloc64(nByte+1);
           84  +  if( a==0 ) return 1;
           85  +  memcpy(a, aData, nByte);
           86  +  sqlite3_deserialize(db, "main", a, nByte, nByte,
           87  +        SQLITE_DESERIALIZE_RESIZEABLE |
           88  +        SQLITE_DESERIALIZE_FREEONCLOSE);
           89  +  for(i=0; i<sizeof(azSql)/sizeof(azSql[0]); i++){
           90  +    if( eVerbosity>=1 ){
           91  +      printf("%s\n", azSql[i]);
           92  +      fflush(stdout);
           93  +    }
           94  +    sqlite3_exec(db, azSql[i], 0, 0, 0);
           95  +  }
           96  +  rc = sqlite3_close(db);
           97  +  if( rc!=SQLITE_OK ){
           98  +    fprintf(stdout, "sqlite3_close() returns %d\n", rc);
           99  +  }
          100  +  if( sqlite3_memory_used()!=0 ){
          101  +    int nAlloc = 0;
          102  +    int nNotUsed = 0;
          103  +    sqlite3_status(SQLITE_STATUS_MALLOC_COUNT, &nAlloc, &nNotUsed, 0);
          104  +    fprintf(stderr,"Memory leak: %lld bytes in %d allocations\n",
          105  +            sqlite3_memory_used(), nAlloc);
          106  +    exit(1);
          107  +  }
          108  +  return 0;
          109  +}
          110  +
          111  +/* libFuzzer invokes this routine once when the executable starts, to
          112  +** process the command-line arguments.
          113  +*/
          114  +int LLVMFuzzerInitialize(int *pArgc, char ***pArgv){
          115  +  int i, j;
          116  +  int argc = *pArgc;
          117  +  char **newArgv;
          118  +  char **argv = *pArgv;
          119  +  newArgv = malloc( sizeof(char*)*(argc+1) );
          120  +  if( newArgv==0 ) return 0;
          121  +  newArgv[0] = argv[0];
          122  +  for(i=j=1; i<argc; i++){
          123  +    char *z = argv[i];
          124  +    if( z[0]=='-' ){
          125  +      z++;
          126  +      if( z[0]=='-' ) z++;
          127  +      if( strcmp(z,"v")==0 ){
          128  +        eVerbosity++;
          129  +        continue;
          130  +      }
          131  +    }
          132  +    newArgv[j++] = argv[i];
          133  +  }
          134  +  newArgv[j] = 0;
          135  +  *pArgv = newArgv;
          136  +  *pArgc = j;
          137  +  return 0;
          138  +}

Changes to test/dbstatus.test.

   375    375     }
   376    376   }
   377    377   
   378    378   #-------------------------------------------------------------------------
   379    379   # The following tests focus on DBSTATUS_CACHE_USED_SHARED
   380    380   #
   381    381   ifcapable shared_cache {
   382         -  if {[permutation]=="memsys3"
          382  +  if {([permutation]=="memsys3"
   383    383         || [permutation]=="memsys5"
   384         -      || $::tcl_platform(os)=="Linux"} {
          384  +      || $::tcl_platform(os)=="Linux") && ![sqlite3 -has-codec]} {
   385    385       proc do_cacheused_test {tn db res} {
   386    386         set cu [sqlite3_db_status $db SQLITE_DBSTATUS_CACHE_USED 0]
   387    387         set pcu [sqlite3_db_status $db SQLITE_DBSTATUS_CACHE_USED_SHARED 0]
   388    388         set cu [lindex $cu 1]
   389    389         set pcu [lindex $pcu 1]
   390    390         uplevel [list do_test $tn [list list $cu $pcu] "#/$res/"]
   391    391       }

Added test/fts3corrupt4.test.

            1  +# 2006 September 9
            2  +#
            3  +# The author disclaims copyright to this source code.  In place of
            4  +# a legal notice, here is a blessing:
            5  +#
            6  +#    May you do good and not evil.
            7  +#    May you find forgiveness for yourself and forgive others.
            8  +#    May you share freely, never taking more than you give.
            9  +#
           10  +#*************************************************************************
           11  +# This file implements regression tests for SQLite library.  The
           12  +# focus of this script is testing the FTS3 module.
           13  +#
           14  +# $Id: fts3aa.test,v 1.1 2007/08/20 17:38:42 shess Exp $
           15  +#
           16  +
           17  +set testdir [file dirname $argv0]
           18  +source $testdir/tester.tcl
           19  +set testprefix fts3corrupt4
           20  +
           21  +# If SQLITE_ENABLE_FTS3 is defined, omit this file.
           22  +ifcapable !fts3 {
           23  +  finish_test
           24  +  return
           25  +}
           26  +
           27  +do_execsql_test 1.0 {
           28  +  BEGIN;
           29  +    CREATE VIRTUAL TABLE ft USING fts3;
           30  +    INSERT INTO ft VALUES('aback');
           31  +    INSERT INTO ft VALUES('abaft');
           32  +    INSERT INTO ft VALUES('abandon');
           33  +  COMMIT;
           34  +}
           35  +
           36  +proc blob {a} { binary decode hex $a }
           37  +db func blob blob
           38  +
           39  +do_execsql_test 1.1 {
           40  +  SELECT quote(root) FROM ft_segdir;
           41  +} {X'0005616261636B03010200030266740302020003046E646F6E03030200'}
           42  +
           43  +do_execsql_test 1.2 {
           44  +  UPDATE ft_segdir SET root = blob(
           45  +    '0005616261636B03010200 FFFFFFFF0702 66740302020003046E646F6E03030200'
           46  +  );
           47  +}
           48  +
           49  +do_catchsql_test 1.3 {
           50  +  SELECT * FROM ft WHERE ft MATCH 'abandon';
           51  +} {1 {database disk image is malformed}}
           52  +
           53  +#-------------------------------------------------------------------------
           54  +reset_db
           55  +do_execsql_test 2.0.0 {
           56  +  CREATE VIRTUAL TABLE ft USING fts3;
           57  +  INSERT INTO ft(ft) VALUES('nodesize=32');
           58  +}
           59  +do_test 2.0.1 {
           60  +  for {set i 0} {$i < 12} {incr i} {
           61  +    execsql {
           62  +      BEGIN;
           63  +        INSERT INTO ft VALUES('abc' || $i);
           64  +        INSERT INTO ft VALUES('abc' || $i || 'x' );
           65  +        INSERT INTO ft VALUES('abc' || $i || 'xx' );
           66  +      COMMIT
           67  +    }
           68  +  }
           69  +  execsql {
           70  +    SELECT count(*) FROM ft_segdir;
           71  +    SELECT count(*) FROM ft_segments;
           72  +  }
           73  +} {12 0}
           74  +
           75  +do_execsql_test 2.1 {
           76  +  INSERT INTO ft(ft) VALUES('merge=1,4');
           77  +  SELECT count(*) FROM ft_segdir;
           78  +  SELECT count(*) FROM ft_segments;
           79  +} {12 3}
           80  +
           81  +do_execsql_test 2.2 {
           82  +  SELECT quote(block) FROM ft_segments WHERE blockid=2
           83  +} {X'00056162633130031F0200'}
           84  +
           85  +db func blob blob
           86  +do_execsql_test 2.3.1 {
           87  +  UPDATE ft_segments SET block = 
           88  +    blob('00056162633130031F0200 FFFFFFFF07FF55 66740302020003046E646F6E03030200')
           89  +    WHERE blockid=2;
           90  +} {}
           91  +do_catchsql_test 2.3.2 {
           92  +  INSERT INTO ft(ft) VALUES('merge=1,4');
           93  +} {1 {database disk image is malformed}}
           94  +
           95  +do_execsql_test 2.4.1 {
           96  +  UPDATE ft_segments SET block = 
           97  +    blob('00056162633130031F0200 02FFFFFFFF07 66740302020003046E646F6E03030200')
           98  +    WHERE blockid=2;
           99  +} {}
          100  +do_catchsql_test 2.4.2 {
          101  +  INSERT INTO ft(ft) VALUES('merge=1,4');
          102  +} {1 {database disk image is malformed}}
          103  +
          104  +do_execsql_test 2.5.1 {
          105  +  UPDATE ft_segments SET block = 
          106  +    blob('00056162633130031F0200 0202 6674 FFFFFF070302020003046E646F6E030200')
          107  +    WHERE blockid=2;
          108  +} {}
          109  +do_catchsql_test 2.5.2 {
          110  +  INSERT INTO ft(ft) VALUES('merge=1,4');
          111  +} {1 {database disk image is malformed}}
          112  +
          113  +#-------------------------------------------------------------------------
          114  +reset_db
          115  +do_execsql_test 3.0.0 {
          116  +  CREATE VIRTUAL TABLE ft USING fts3;
          117  +  INSERT INTO ft(ft) VALUES('nodesize=32');
          118  +}
          119  +do_test 3.0.1 {
          120  +  execsql BEGIN
          121  +  for {set i 0} {$i < 20} {incr i} {
          122  +    execsql { INSERT INTO ft VALUES('abc' || $i) }
          123  +  }
          124  +  execsql {
          125  +    COMMIT;
          126  +    SELECT count(*) FROM ft_segdir;
          127  +    SELECT count(*) FROM ft_segments;
          128  +  }
          129  +} {1 5}
          130  +
          131  +do_execsql_test 3.1 {
          132  +  SELECT quote(root) FROM ft_segdir
          133  +} {X'0101056162633132040136030132030136'}
          134  +
          135  +db func blob blob
          136  +do_execsql_test 3.2 {
          137  +  UPDATE ft_segdir 
          138  +  SET root = blob('0101056162633132FFFFFFFF070236030132030136');
          139  +}
          140  +
          141  +do_catchsql_test 3.1 {
          142  +  SELECT * FROM ft WHERE ft MATCH 'abc20'
          143  +} {1 {database disk image is malformed}}
          144  +
          145  +finish_test
          146  +
          147  +

Added test/fuzzdata7.db.

cannot compute difference between binary files

Changes to test/index.test.

   621    621   # little outside the focus of this test scripts, but this has got to be
   622    622   # tested somewhere.
   623    623   do_test index-18.1 {
   624    624     catchsql {
   625    625       CREATE TABLE sqlite_t1(a, b, c);
   626    626     }
   627    627   } {1 {object name reserved for internal use: sqlite_t1}}
          628  +do_test index-18.1.2 {
          629  +  sqlite3_db_config db DEFENSIVE 1
          630  +  catchsql {
          631  +    CREATE TABLE sqlite_t1(a, b, c);
          632  +  }
          633  +} {1 {object name reserved for internal use: sqlite_t1}}
          634  +sqlite3_db_config db DEFENSIVE 0
   628    635   do_test index-18.2 {
   629    636     catchsql {
   630    637       CREATE INDEX sqlite_i1 ON t7(c);
   631    638     }
   632    639   } {1 {object name reserved for internal use: sqlite_i1}}
   633    640   ifcapable view {
   634    641   do_test index-18.3 {

Changes to test/intarray.test.

    42     42     set ia3 [sqlite3_intarray_create db ia3]
    43     43     set ia4 [sqlite3_intarray_create db ia4]
    44     44     db eval {
    45     45       SELECT type, name FROM temp.sqlite_master
    46     46        ORDER BY name
    47     47     }
    48     48   } {table ia1 table ia2 table ia3 table ia4}
           49  +
           50  +# Verify the inability to DROP and recreate an intarray virtual table.
           51  +do_test intarray-1.1b {
           52  +  db eval {DROP TABLE ia1}
           53  +  set rc [catch {sqlite3_intarray_create db ia1} msg]
           54  +  lappend rc $msg
           55  +} {1 SQLITE_MISUSE}
    49     56   
    50     57   do_test intarray-1.2 {
    51     58     db eval {
    52     59       SELECT b FROM t1 WHERE a IN ia3 ORDER BY a
    53     60     }
    54     61   } {}
    55     62   

Changes to test/normalize.test.

    67     67     {select*from t1 where x in(?,?,?);}
    68     68     180
    69     69     {    }
    70     70     {}
    71     71   } {
    72     72     do_test $tnum [list sqlite3_normalize $sql] $norm
    73     73   }
           74  +
           75  +ifcapable normalize {
           76  +do_test 200 {
           77  +  execsql {
           78  +    CREATE TABLE t1(a,b);
           79  +  }
           80  +} {}
           81  +do_test 201 {
           82  +  set STMT [sqlite3_prepare_v3 $DB \
           83  +      "SELECT a, b FROM t1 WHERE b = ? ORDER BY a;" -1 0 TAIL]
           84  +
           85  +  sqlite3_bind_null $STMT 1
           86  +} {}
           87  +do_test 202 {
           88  +  sqlite3_normalized_sql $STMT
           89  +} {}
           90  +do_test 203 {
           91  +  sqlite3_finalize $STMT
           92  +} {SQLITE_OK}
           93  +
           94  +do_test 210 {
           95  +  set STMT [sqlite3_prepare_v3 $DB \
           96  +      "SELECT a, b FROM t1 WHERE b = ? ORDER BY a;" -1 2 TAIL]
           97  +
           98  +  sqlite3_bind_null $STMT 1
           99  +} {}
          100  +do_test 211 {
          101  +  sqlite3_normalized_sql $STMT
          102  +} {SELECT a,b FROM t1 WHERE b=?ORDER BY a;}
          103  +do_test 212 {
          104  +  sqlite3_finalize $STMT
          105  +} {SQLITE_OK}
          106  +
          107  +do_test 220 {
          108  +  set STMT [sqlite3_prepare_v3 $DB \
          109  +      "SELECT a, b FROM t1 WHERE b = 'a' ORDER BY a;" -1 2 TAIL]
          110  +} {/^[0-9A-Fa-f]+$/}
          111  +do_test 221 {
          112  +  sqlite3_normalized_sql $STMT
          113  +} {SELECT a,b FROM t1 WHERE b=?ORDER BY a;}
          114  +do_test 222 {
          115  +  sqlite3_finalize $STMT
          116  +} {SQLITE_OK}
          117  +
          118  +do_test 297 {
          119  +  execsql {
          120  +    DROP TABLE t1;
          121  +  }
          122  +} {}
          123  +do_test 298 {
          124  +  execsql {
          125  +    CREATE TABLE t1(a,b,c,d,e,"col f",w,x,y,z);
          126  +    CREATE TABLE t2(x,"col y");
          127  +  }
          128  +} {}
          129  +do_test 299 {
          130  +  sqlite3_create_function db
          131  +} {SQLITE_OK}
          132  +
          133  +foreach {tnum sql flags norm} {
          134  +  300
          135  +  {SELECT * FROM t1 WHERE a IN (1) AND b=51.42}
          136  +  0x2
          137  +  {0 {SELECT*FROM t1 WHERE a IN(?,?,?)AND b=?;}}
          138  +
          139  +  310
          140  +  {SELECT a, b+15, c FROM t1 WHERE d NOT IN (SELECT x FROM t2);}
          141  +  0x2
          142  +  {0 {SELECT a,b+?,c FROM t1 WHERE d NOT IN(SELECT x FROM t2);}}
          143  +
          144  +  320
          145  +  { SELECT NULL, b FROM t1 -- comment text
          146  +     WHERE d IN (WITH t(a) AS (VALUES(5)) /* CTE */
          147  +                 SELECT a FROM t)
          148  +        OR e='hello';
          149  +  }
          150  +  0x2
          151  +  {0 {SELECT?,b FROM t1 WHERE d IN(WITH t(a)AS(VALUES(?))SELECT a FROM t)OR e=?;}}
          152  +
          153  +  321
          154  +  {/*Initial comment*/
          155  +   -- another comment line
          156  +   SELECT NULL  /* comment */ , b FROM t1 -- comment text
          157  +     WHERE d IN (WITH t(a) AS (VALUES(5)) /* CTE */
          158  +                 SELECT a FROM t)
          159  +        OR e='hello';
          160  +  }
          161  +  0x2
          162  +  {0 {SELECT?,b FROM t1 WHERE d IN(WITH t(a)AS(VALUES(?))SELECT a FROM t)OR e=?;}}
          163  +
          164  +  330
          165  +  {/* Query containing parameters */
          166  +   SELECT x,$::abc(15),y,@abc,z,?99,w FROM t1 /* Trailing comment */}
          167  +  0x2
          168  +  {0 {SELECT x,?,y,?,z,?,w FROM t1;}}
          169  +
          170  +  340
          171  +  {/* Long list on the RHS of IN */
          172  +   SELECT 15 IN (1,2,3,(SELECT * FROM t1),'xyz',x'abcd',22*(x+5),null);}
          173  +  0x2
          174  +  {1 {(1) no such column: x}}
          175  +
          176  +  350
          177  +  {SELECT x'abc'; -- illegal token}
          178  +  0x2
          179  +  {1 {(1) unrecognized token: "x'abc'"}}
          180  +
          181  +  360
          182  +  {SELECT a,NULL,b FROM t1 WHERE c IS NOT NULL or D is null or e=5}
          183  +  0x2
          184  +  {0 {SELECT a,?,b FROM t1 WHERE c IS NOT NULL OR d IS NULL OR e=?;}}
          185  +
          186  +  370
          187  +  {/* IN list exactly 5 bytes long */
          188  +   SELECT * FROM t1 WHERE x IN (1,2,3);}
          189  +  0x2
          190  +  {0 {SELECT*FROM t1 WHERE x IN(?,?,?);}}
          191  +
          192  +  400
          193  +  {SELECT a FROM t1 WHERE x IN (1,2,3) AND sqlite_version();}
          194  +  0x2
          195  +  {0 {SELECT a FROM t1 WHERE x IN(?,?,?)AND sqlite_version();}}
          196  +
          197  +  410
          198  +  {SELECT a FROM t1 WHERE x IN (1,2,3) AND hex8();}
          199  +  0x2
          200  +  {1 {(1) wrong number of arguments to function hex8()}}
          201  +
          202  +  420
          203  +  {SELECT a FROM t1 WHERE x IN (1,2,3) AND hex8('abc');}
          204  +  0x2
          205  +  {0 {SELECT a FROM t1 WHERE x IN(?,?,?)AND hex8(?);}}
          206  +
          207  +  430
          208  +  {SELECT "a" FROM t1 WHERE "x" IN ("1","2",'3');}
          209  +  0x2
          210  +  {0 {SELECT"a"FROM t1 WHERE"x"IN(?,?,?);}}
          211  +
          212  +  440
          213  +  {SELECT 'a' FROM t1 WHERE 'x';}
          214  +  0x2
          215  +  {0 {SELECT?FROM t1 WHERE?;}}
          216  +
          217  +  450
          218  +  {SELECT [a] FROM t1 WHERE [x];}
          219  +  0x2
          220  +  {0 {SELECT"a"FROM t1 WHERE"x";}}
          221  +
          222  +  460
          223  +  {SELECT * FROM t1 WHERE x IN (x);}
          224  +  0x2
          225  +  {0 {SELECT*FROM t1 WHERE x IN(x);}}
          226  +
          227  +  470
          228  +  {SELECT * FROM t1 WHERE x IN (x,a);}
          229  +  0x2
          230  +  {0 {SELECT*FROM t1 WHERE x IN(x,a);}}
          231  +
          232  +  480
          233  +  {SELECT * FROM t1 WHERE x IN ([x],"a");}
          234  +  0x2
          235  +  {0 {SELECT*FROM t1 WHERE x IN("x","a");}}
          236  +
          237  +  500
          238  +  {SELECT * FROM t1 WHERE x IN ([x],"a",'b',sqlite_version());}
          239  +  0x2
          240  +  {0 {SELECT*FROM t1 WHERE x IN("x","a",?,sqlite_version());}}
          241  +
          242  +  520
          243  +  {SELECT * FROM t1 WHERE x IN (SELECT x FROM t1);}
          244  +  0x2
          245  +  {0 {SELECT*FROM t1 WHERE x IN(SELECT x FROM t1);}}
          246  +
          247  +  540
          248  +  {SELECT * FROM t1 WHERE x IN ((SELECT x FROM t1));}
          249  +  0x2
          250  +  {0 {SELECT*FROM t1 WHERE x IN(?,?,?);}}
          251  +
          252  +  550
          253  +  {SELECT a, a+1, a||'b', a+"b" FROM t1;}
          254  +  0x2
          255  +  {0 {SELECT a,a+?,a||?,a+"b"FROM t1;}}
          256  +
          257  +  570
          258  +  {SELECT * FROM t1 WHERE x IN (1);}
          259  +  0x2
          260  +  {0 {SELECT*FROM t1 WHERE x IN(?,?,?);}}
          261  +
          262  +  580
          263  +  {SELECT * FROM t1 WHERE x IN (1,2);}
          264  +  0x2
          265  +  {0 {SELECT*FROM t1 WHERE x IN(?,?,?);}}
          266  +
          267  +  590
          268  +  {SELECT * FROM t1 WHERE x IN (1,2,3);}
          269  +  0x2
          270  +  {0 {SELECT*FROM t1 WHERE x IN(?,?,?);}}
          271  +
          272  +  600
          273  +  {SELECT * FROM t1 WHERE x IN (1,2,3,4);}
          274  +  0x2
          275  +  {0 {SELECT*FROM t1 WHERE x IN(?,?,?);}}
          276  +
          277  +  610
          278  +  {SELECT * FROM t1 WHERE x IN (SELECT x FROM t1);}
          279  +  0x2
          280  +  {0 {SELECT*FROM t1 WHERE x IN(SELECT x FROM t1);}}
          281  +
          282  +  620
          283  +  {SELECT * FROM t1 WHERE x IN (SELECT x FROM t1 WHERE x IN (1,2,3));}
          284  +  0x2
          285  +  {0 {SELECT*FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(?,?,?));}}
          286  +
          287  +  630
          288  +  {SELECT * FROM t1 WHERE x IN (SELECT x FROM t1 WHERE x IN (x));}
          289  +  0x2
          290  +  {0 {SELECT*FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(x));}}
          291  +
          292  +  640
          293  +  {SELECT x FROM t1 WHERE x IN (SELECT x FROM t1 WHERE x IN (
          294  +   SELECT x FROM t1 WHERE x IN (SELECT x FROM t1 WHERE x IN (
          295  +   SELECT x FROM t1 WHERE x IN (x)))));}
          296  +  0x2
          297  +  {0 {SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(x)))));}}
          298  +
          299  +  650
          300  +  {SELECT x FROM t1 WHERE x IN (SELECT x FROM t1 WHERE x IN (
          301  +   SELECT x FROM t1 WHERE x IN (SELECT x FROM t1 WHERE x IN (
          302  +   SELECT x FROM t1 WHERE x IN (1)))));}
          303  +  0x2
          304  +  {0 {SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(SELECT x FROM t1 WHERE x IN(?,?,?)))));}}
          305  +
          306  +  660
          307  +  {SELECT x FROM t1 WHERE x IN (1) UNION ALL SELECT x FROM t1 WHERE x IN (1);}
          308  +  0x2
          309  +  {0 {SELECT x FROM t1 WHERE x IN(?,?,?)UNION ALL SELECT x FROM t1 WHERE x IN(?,?,?);}}
          310  +
          311  +  670
          312  +  {SELECT "col f", [col f] FROM t1;}
          313  +  0x2
          314  +  {0 {SELECT"col f","col f"FROM t1;}}
          315  +
          316  +  680
          317  +  {SELECT a, "col f" FROM t1 LEFT OUTER JOIN t2 ON [t1].[col f] == [t2].[col y];}
          318  +  0x2
          319  +  {0 {SELECT a,"col f"FROM t1 LEFT OUTER JOIN t2 ON"t1"."col f"=="t2"."col y";}}
          320  +
          321  +  690
          322  +  {SELECT * FROM ( WITH x AS ( SELECT * FROM t1 WHERE x IN ( 1)) SELECT 10);}
          323  +  0x2
          324  +  {0 {SELECT*FROM(WITH x AS(SELECT*FROM t1 WHERE x IN(?,?,?))SELECT?);}}
          325  +
          326  +  700
          327  +  {SELECT rowid, oid, _rowid_ FROM t1;}
          328  +  0x2
          329  +  {0 {SELECT rowid,oid,_rowid_ FROM t1;}}
          330  +
          331  +  710
          332  +  {SELECT x FROM t1 WHERE x IS NULL;}
          333  +  0x2
          334  +  {0 {SELECT x FROM t1 WHERE x IS NULL;}}
          335  +
          336  +  740
          337  +  {SELECT x FROM t1 WHERE x IS NOT NULL;}
          338  +  0x2
          339  +  {0 {SELECT x FROM t1 WHERE x IS NOT NULL;}}
          340  +
          341  +  750
          342  +  {SELECT x FROM t1 WHERE x = NULL;}
          343  +  0x2
          344  +  {0 {SELECT x FROM t1 WHERE x=?;}}
          345  +
          346  +  760
          347  +  {SELECT x FROM t1 WHERE x IN ([x] IS NOT NULL, NULL, 1, 'a', "b", x'00');}
          348  +  0x2
          349  +  {0 {SELECT x FROM t1 WHERE x IN("x"IS NOT NULL,?,?,?,"b",?);}}
          350  +} {
          351  +  do_test $tnum {
          352  +    set code [catch {
          353  +      set STMT [sqlite3_prepare_v3 $DB $sql -1 $flags TAIL]
          354  +      sqlite3_normalized_sql $STMT
          355  +    } res]
          356  +    if {[info exists STMT]} {
          357  +      sqlite3_finalize $STMT; unset STMT
          358  +    }
          359  +    list $code $res
          360  +  } $norm
          361  +}
          362  +}
    74    363   
    75    364   finish_test

Changes to test/permutations.test.

    44     44     foreach {k v} $args {
    45     45       set o [array names options ${k}*]
    46     46       if {[llength $o]>1}  { error "ambiguous option: $k" }
    47     47       if {[llength $o]==0} { error "unknown option: $k" }
    48     48       set options([lindex $o 0]) $v
    49     49     }
    50     50   
    51         -  # Transform "-presql" into a "dbconfig" script
    52         -  #
    53         -  if {[info exists options(-presql)]} {
    54         -    append options(-dbconfig) "\n\$::dbhandle eval { $options(-presql) }"
    55         -    unset options(-presql)
    56         -  }
    57         -
    58     51     set     ::testspec($name) [array get options]
    59     52     lappend ::testsuitelist $name
    60     53   }
    61     54   
    62     55   #-------------------------------------------------------------------------
    63     56   # test_set ARGS...
    64     57   #
................................................................................
   258    251   } -files {
   259    252     fts3aa.test fts3ab.test fts3ac.test fts3ad.test
   260    253     fts3ae.test fts3af.test fts3ag.test fts3ah.test
   261    254     fts3ai.test fts3aj.test fts3ak.test fts3al.test
   262    255     fts3am.test fts3an.test fts3ao.test fts3atoken.test
   263    256     fts3auto.test fts3aux1.test fts3aux2.test fts3b.test
   264    257     fts3comp1.test fts3conf.test fts3corrupt2.test fts3corrupt.test
          258  +  fts3corrupt4.test
   265    259     fts3cov.test fts3c.test fts3defer2.test fts3defer3.test
   266    260     fts3defer.test fts3drop.test fts3d.test fts3e.test
   267    261     fts3expr2.test fts3expr3.test fts3expr4.test fts3expr5.test
   268    262     fts3expr.test fts3fault2.test fts3fault.test fts3first.test
   269    263     fts3join.test fts3malloc.test fts3matchinfo.test fts3near.test
   270    264     fts3offsets.test fts3prefix2.test fts3prefix.test fts3query.test
   271    265     fts3shared.test fts3snippet.test fts3sort.test fts3tok1.test
................................................................................
  1085   1079   #       -dbconfig    SCRIPT
  1086   1080   #
  1087   1081   proc run_tests {name args} {
  1088   1082     set options(-initialize) ""
  1089   1083     set options(-shutdown) ""
  1090   1084     set options(-prefix) ""
  1091   1085     set options(-dbconfig) ""
         1086  +  set options(-presql) ""
  1092   1087   
  1093   1088     array set options $args
  1094   1089   
  1095   1090     set ::G(perm:name)         $name
  1096   1091     set ::G(perm:prefix)       $options(-prefix)
  1097   1092     set ::G(isquick)           1
  1098   1093     set ::G(perm:dbconfig)     $options(-dbconfig)
         1094  +  set ::G(perm:presql)       $options(-presql)
  1099   1095   
  1100   1096     foreach file [lsort $options(-files)] {
  1101   1097       uplevel $options(-initialize)
  1102   1098       if {[file tail $file] == $file} { set file [file join $::testdir $file] }
  1103   1099       slave_test_file $file
  1104   1100       uplevel $options(-shutdown)
  1105   1101   
  1106   1102       unset -nocomplain ::G(perm:sqlite3_args)
  1107   1103     }
  1108   1104   
  1109   1105     unset ::G(perm:name)
  1110   1106     unset ::G(perm:prefix)
  1111   1107     unset ::G(perm:dbconfig)
         1108  +  unset ::G(perm:presql)
  1112   1109   }
  1113   1110   
  1114   1111   proc run_test_suite {name} {
  1115   1112     if {[info exists ::testspec($name)]==0} {
  1116   1113       error "No such test suite: $name"
  1117   1114     }
  1118   1115     uplevel run_tests $name $::testspec($name)

Changes to test/pragma3.test.

    11     11   # This file implements regression tests for SQLite library.
    12     12   #
    13     13   # This file implements tests for PRAGMA data_version command.
    14     14   #
    15     15   
    16     16   set testdir [file dirname $argv0]
    17     17   source $testdir/tester.tcl
    18         -do_not_use_codec
           18  +
           19  +if {[sqlite3 -has-codec]} {
           20  +  finish_test
           21  +  return
           22  +}
    19     23   
    20     24   do_execsql_test pragma3-100 {
    21     25     PRAGMA data_version;
    22     26   } {1}
    23     27   do_execsql_test pragma3-101 {
    24     28     PRAGMA temp.data_version;
    25     29   } {1}

Changes to test/rowvalue.test.

   541    541   } {2 22 1 11}
   542    542   
   543    543   # 2018-02-18: Memory leak nexted row-value.  Detected by OSSFuzz.
   544    544   #
   545    545   do_catchsql_test 20.1 {
   546    546     SELECT 1 WHERE (2,(2,0)) IS (2,(2,0));
   547    547   } {0 1}
          548  +
          549  +# 2018-11-03: Ticket https://www.sqlite.org/src/info/1a84668dcfdebaf1
          550  +# Assertion fault when doing row-value operations on a primary key
          551  +# containing duplicate columns.
          552  +#
          553  +do_execsql_test 21.0 {
          554  +  DROP TABLE IF EXISTS t1;
          555  +  CREATE TABLE t1(a,b,PRIMARY KEY(b,b));
          556  +  INSERT INTO t1 VALUES(1,2),(3,4),(5,6);
          557  +  SELECT * FROM t1 WHERE (a,b) IN (VALUES(1,2));  
          558  +} {1 2}
   548    559   
   549    560   finish_test

Changes to test/shared.test.

  1141   1141   } {1 {no such table: t1}}
  1142   1142   do_test shared-$av-16.8 {
  1143   1143     file exists test1.db
  1144   1144   } {0}  ;# Verify that the database is in-memory
  1145   1145   
  1146   1146   # Shared cache on named memory databases attached to readonly connections.
  1147   1147   #
  1148         -do_test shared-$av-16.8.1 {
         1148  +if {![sqlite3 -has-codec]} {
         1149  +  do_test shared-$av-16.8.1 {
         1150  +    db1 close
         1151  +    db2 close
         1152  +  
         1153  +    sqlite3 db test1.db
         1154  +    db eval { 
         1155  +      CREATE TABLE yy(a, b);
         1156  +      INSERT INTO yy VALUES(77, 88);
         1157  +    }
         1158  +    db close
         1159  +  
         1160  +    sqlite3 db1 test1.db -uri 1 -readonly 1
         1161  +    sqlite3 db2 test2.db -uri 1 
         1162  +  
         1163  +    db1 eval { 
         1164  +      ATTACH 'file:mem?mode=memory&cache=shared' AS shared; 
         1165  +      CREATE TABLE shared.xx(a, b);
         1166  +      INSERT INTO xx VALUES(55, 66);
         1167  +    }
         1168  +    db2 eval { 
         1169  +      ATTACH 'file:mem?mode=memory&cache=shared' AS shared;
         1170  +      SELECT * FROM xx;
         1171  +    }
         1172  +  } {55 66}
         1173  +  
         1174  +  do_test shared-$av-16.8.2 { db1 eval { SELECT * FROM yy } } {77 88}
         1175  +  do_test shared-$av-16.8.3 { 
         1176  +    list [catch {db1 eval { INSERT INTO yy VALUES(1, 2) }} msg] $msg
         1177  +  } {1 {attempt to write a readonly database}}
         1178  +  
  1149   1179     db1 close
  1150   1180     db2 close
  1151         -
  1152         -  sqlite3 db test1.db
  1153         -  db eval { 
  1154         -    CREATE TABLE yy(a, b);
  1155         -    INSERT INTO yy VALUES(77, 88);
  1156         -  }
  1157         -  db close
  1158         -
  1159         -  sqlite3 db1 test1.db -uri 1 -readonly 1
  1160         -  sqlite3 db2 test2.db -uri 1 
  1161         -
  1162         -  db1 eval { 
  1163         -    ATTACH 'file:mem?mode=memory&cache=shared' AS shared; 
  1164         -    CREATE TABLE shared.xx(a, b);
  1165         -    INSERT INTO xx VALUES(55, 66);
  1166         -  }
  1167         -  db2 eval { 
  1168         -    ATTACH 'file:mem?mode=memory&cache=shared' AS shared;
  1169         -    SELECT * FROM xx;
  1170         -  }
  1171         -} {55 66}
  1172         -
  1173         -do_test shared-$av-16.8.2 { db1 eval { SELECT * FROM yy } } {77 88}
  1174         -do_test shared-$av-16.8.3 { 
  1175         -  list [catch {db1 eval { INSERT INTO yy VALUES(1, 2) }} msg] $msg
  1176         -} {1 {attempt to write a readonly database}}
  1177         -
  1178         -db1 close
  1179         -db2 close
         1181  +}
  1180   1182   
  1181   1183   }  ;# end of autovacuum on/off loop
  1182   1184   
  1183   1185   sqlite3_enable_shared_cache $::enable_shared_cache
  1184   1186   finish_test

Changes to test/tabfunc01.test.

   107    107     SELECT * FROM generate_series() LIMIT 5;
   108    108   } {0 1 2 3 4}
   109    109   
   110    110   do_execsql_test tabfunc01-3.1 {
   111    111     SELECT DISTINCT value FROM generate_series(1,x), t1 ORDER BY 1;
   112    112   } {1 2 3}
   113    113   
   114         -# Eponymous virtual table exists in the "main" schema only
          114  +# Eponymous virtual table exists in all schemas.
   115    115   #
   116    116   do_execsql_test tabfunc01-4.1 {
   117    117     SELECT * FROM main.generate_series(1,4)
   118    118   } {1 2 3 4}
   119         -do_catchsql_test tabfunc01-4.2 {
          119  +do_execsql_test tabfunc01-4.2 {
   120    120     SELECT * FROM temp.generate_series(1,4)
   121         -} {1 {no such table: temp.generate_series}}
   122         -do_catchsql_test tabfunc01-4.3 {
          121  +} {1 2 3 4}
          122  +do_execsql_test tabfunc01-4.3 {
   123    123     ATTACH ':memory:' AS aux1;
   124    124     CREATE TABLE aux1.t1(a,b,c);
   125    125     SELECT * FROM aux1.generate_series(1,4)
   126         -} {1 {no such table: aux1.generate_series}}
          126  +} {1 2 3 4}
   127    127   
   128    128   # The next series of tests is verifying that virtual table are able
   129    129   # to optimize the IN operator, even on terms that are not marked "omit".
   130    130   # When the generate_series virtual table is compiled for the testfixture,
   131    131   # the special -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 option is used, which
   132    132   # causes the xBestIndex method of generate_series to leave the
   133    133   # sqlite3_index_constraint_usage.omit flag set to 0, which should cause

Changes to test/wal64k.test.

    42     42     }
    43     43     execsql COMMIT
    44     44     file size test.db-shm
    45     45   } {131072}
    46     46   
    47     47   integrity_check 1.3
    48     48   
           49  +db close
           50  +forcedelete test.db
           51  +sqlite3 db test.db -vfs unix-excl
           52  +do_execsql_test 2.1 {
           53  +  PRAGMA page_size=512;
           54  +  PRAGMA journal_mode=WAL;
           55  +  CREATE TABLE t1(a,b);
           56  +  WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<8200)
           57  +  INSERT INTO t1(a,b) SELECT x, zeroblob(300) FROM c;
           58  +  PRAGMA integrity_check;
           59  +} {wal ok}
           60  +
    49     61   db close
    50     62   test_syscall pagesize -1
    51     63   finish_test

Changes to test/where.test.

  1408   1408     SELECT * FROM t12 WHERE 
  1409   1409     a = (SELECT * FROM (SELECT count(*) FROM t13 LIMIT 5) ORDER BY 1 LIMIT 10) 
  1410   1410     AND (b=1 OR c=1);
  1411   1411   } {
  1412   1412     4 1 0
  1413   1413     4 0 1
  1414   1414   }
         1415  +
         1416  +# 2018-11-05: ticket [https://www.sqlite.org/src/tktview/65eb38f6e46de8c75e188a]
         1417  +# Incorrect result in LEFT JOIN when STAT4 is enabled.
         1418  +#
         1419  +sqlite3 db :memory:
         1420  +do_execsql_test where-22.1 {
         1421  +  CREATE TABLE t1(a INT);
         1422  +  CREATE INDEX t1a ON t1(a);
         1423  +  INSERT INTO t1(a) VALUES(NULL),(NULL),(42),(NULL),(NULL);
         1424  +  CREATE TABLE t2(dummy INT);
         1425  +  SELECT count(*) FROM t1 LEFT JOIN t2 ON a IS NOT NULL;
         1426  +} {5}
         1427  +
  1415   1428   
  1416   1429   finish_test

Changes to test/whereL.test.

   106    106       FROM A,
   107    107            (SELECT id,yy,zz FROM C) subq,
   108    108            B
   109    109      WHERE A.id='1'
   110    110        AND A.id=subq.yy
   111    111        AND B.id=subq.zz;
   112    112   } {1}  
          113  +
          114  +# 2018-10-25: Ticket [cf5ed20f]
          115  +# Incorrect join result with duplicate WHERE clause constraint.
          116  +#
          117  +do_execsql_test 400 {
          118  +  CREATE TABLE x(a, b, c);
          119  +  CREATE TABLE y(a, b);
          120  +  INSERT INTO x VALUES (1, 0, 1);
          121  +  INSERT INTO y VALUES (1, 2);
          122  +  SELECT x.a FROM x JOIN y ON x.c = y.a WHERE x.b = 1 AND x.b = 1;
          123  +} {}
   113    124   
   114    125   finish_test

Changes to test/window1.test.

   544    544   do_execsql_test 12.110 {
   545    545     INSERT INTO t1 VALUES(6, 'F', 'three');
   546    546     INSERT INTO t1 VALUES(7, 'G', 'one');
   547    547     SELECT id, b, lead(c,1) OVER(ORDER BY c) AS x
   548    548       FROM t1 WHERE id>1
   549    549      ORDER BY b LIMIT 2;
   550    550   } {2 B two 3 C three}
          551  +
          552  +#-------------------------------------------------------------------------
          553  +
          554  +do_execsql_test 13.1 {
          555  +  DROP TABLE IF EXISTS t1;
          556  +  CREATE TABLE t1(a int, b int);
          557  +  INSERT INTO t1 VALUES(1,11);
          558  +  INSERT INTO t1 VALUES(2,12);
          559  +}
          560  +
          561  +do_execsql_test 13.2.1 {
          562  +  SELECT a, rank() OVER(ORDER BY b) FROM t1;
          563  +  SELECT a, rank() OVER(ORDER BY b DESC) FROM t1;
          564  +} {
          565  +  1 1   2 2   2 1   1 2
          566  +}
          567  +do_execsql_test 13.2.2 {
          568  +  SELECT a, rank() OVER(ORDER BY b) FROM t1
          569  +    UNION ALL
          570  +  SELECT a, rank() OVER(ORDER BY b DESC) FROM t1;
          571  +} {
          572  +  1 1   2 2   2 1   1 2
          573  +}
          574  +do_execsql_test 13.3 {
          575  +  SELECT a, rank() OVER(ORDER BY b) FROM t1
          576  +    UNION 
          577  +  SELECT a, rank() OVER(ORDER BY b DESC) FROM t1;
          578  +} {
          579  +  1 1   1 2   2 1   2 2  
          580  +}
          581  +
          582  +do_execsql_test 13.4 {
          583  +  SELECT a, rank() OVER(ORDER BY b) FROM t1
          584  +    EXCEPT 
          585  +  SELECT a, rank() OVER(ORDER BY b DESC) FROM t1;
          586  +} {
          587  +  1 1   2 2 
          588  +}
          589  +
          590  +do_execsql_test 13.5 {
          591  +  SELECT a, rank() OVER(ORDER BY b) FROM t1
          592  +    INTERSECT 
          593  +  SELECT a, rank() OVER(ORDER BY b DESC) FROM t1;
          594  +} {
          595  +}
   551    596   
   552    597   finish_test

Changes to tool/mkpragmatab.tcl.

   116    116     TYPE: FLAG
   117    117     ARG:  SQLITE_IgnoreChecks
   118    118     IF:   !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   119    119     IF:   !defined(SQLITE_OMIT_CHECK)
   120    120   
   121    121     NAME: writable_schema
   122    122     TYPE: FLAG
   123         -  ARG:  SQLITE_WriteSchema
          123  +  ARG:  SQLITE_WriteSchema|SQLITE_NoSchemaError
   124    124     IF:   !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   125    125   
   126    126     NAME: read_uncommitted
   127    127     TYPE: FLAG
   128    128     ARG:  SQLITE_ReadUncommit
   129    129     IF:   !defined(SQLITE_OMIT_FLAG_PRAGMAS)
   130    130   
................................................................................
   363    363   
   364    364     NAME: lock_status
   365    365     FLAG: Result0
   366    366     COLS: database status
   367    367     IF:   defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
   368    368   
   369    369     NAME: key
          370  +  TYPE: KEY
          371  +  ARG:  0
   370    372     IF:   defined(SQLITE_HAS_CODEC)
   371    373   
   372    374     NAME: rekey
          375  +  TYPE: KEY
          376  +  ARG:  1
   373    377     IF:   defined(SQLITE_HAS_CODEC)
   374    378   
   375    379     NAME: hexkey
          380  +  TYPE: HEXKEY
          381  +  ARG:  2
   376    382     IF:   defined(SQLITE_HAS_CODEC)
   377    383   
   378    384     NAME: hexrekey
   379    385     TYPE: HEXKEY
          386  +  ARG:  3
          387  +  IF:   defined(SQLITE_HAS_CODEC)
          388  +
          389  +  NAME: textkey
          390  +  TYPE: KEY
          391  +  ARG:  4
          392  +  IF:   defined(SQLITE_HAS_CODEC)
          393  +
          394  +  NAME: textrekey
          395  +  TYPE: KEY
          396  +  ARG:  5
   380    397     IF:   defined(SQLITE_HAS_CODEC)
   381    398   
   382    399     NAME: activate_extensions
   383    400     IF:   defined(SQLITE_HAS_CODEC) || defined(SQLITE_ENABLE_CEROD)
   384    401   
   385    402     NAME: soft_heap_limit
   386    403     FLAG: Result0
................................................................................
   558    575   puts $fd "typedef struct PragmaName \173"
   559    576   puts $fd "  const char *const zName; /* Name of pragma */"
   560    577   puts $fd "  u8 ePragTyp;             /* PragTyp_XXX value */"
   561    578   puts $fd "  u8 mPragFlg;             /* Zero or more PragFlg_XXX values */"
   562    579   puts $fd {  u8 iPragCName;           /* Start of column names in pragCName[] */}
   563    580   puts $fd "  u8 nPragCName;          \
   564    581   /* Num of col names. 0 means use pragma name */"
   565         -puts $fd "  u32 iArg;                /* Extra argument */"
          582  +puts $fd "  u64 iArg;                /* Extra argument */"
   566    583   puts $fd "\175 PragmaName;"
   567    584   puts $fd "static const PragmaName aPragmaName\[\] = \173"
   568    585   
   569    586   set current_if {}
   570    587   set spacer [format {    %26s } {}]
   571    588   foreach name $allnames {
   572    589     foreach {type arg if flag cx} $allbyname($name) break