Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch bedrock Excluding Merge-Ins
This is equivalent to a diff from 5a22010c35 to a6f6fbe617
2025-06-28
| ||
14:30 | Merge the latest trunk fixes and enhancements into the bedrock branch. (Leaf check-in: a6f6fbe617 user: drh tags: bedrock) | |
14:24 | Merge the latest trunk enhancements into the wal2 branch. (Leaf check-in: e7867c3992 user: drh tags: wal2) | |
2025-06-03
| ||
14:19 | Merge latest changes from the wal2 branch into this branch. (check-in: 93740658c8 user: dan tags: bedrock) | |
2023-02-02
| ||
20:27 | Instead of free()ing them, retain allocated page buffers on an internal per-connection reuse-list when a page-cache is reset. (check-in: 754f331614 user: dan tags: bedrock) | |
14:27 | Merge latest enhancments into this branch from branch wal2. (Leaf check-in: 5a22010c35 user: dan tags: begin-concurrent-pnu-wal2) | |
14:21 | Merge trunk enhancements into this branch. (check-in: c9b0180203 user: dan tags: wal2) | |
2023-01-12
| ||
15:44 | Fix a startup race condition that could occur if a wal file grows from 0 bytes while a BEGIN CONCURRENT transaction is running. Cherrypick of [c1feca4d]. (check-in: 2c14f3832e user: dan tags: begin-concurrent-pnu-wal2) | |
Added .fossil-settings/binary-glob.
> | 1 | *.db |
Changes to LICENSE.md.
1 2 3 4 5 6 | The author disclaims copyright to this source code. In place of a legal notice, here is a blessing: * May you do good and not evil. * May you find forgiveness for yourself and forgive others. * May you share freely, never taking more than you give. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | License Information =================== SQLite Is Public Domain ----------------------- The SQLite source code, including all of the files in the directories listed in the bullets below are [Public Domain](https://sqlite.org/copyright.html). The authors have submitted written affidavits releasing their work to the public for any use. Every byte of the public-domain code can be traced back to the original authors. The files of this repository that are public domain include the following: * All of the primary SQLite source code files found in the [src/ directory](https://sqlite.org/src/tree/src?type=tree&expand) * All of the test cases and testing code in the [test/ directory](https://sqlite.org/src/tree/test?type=tree&expand) * All of the SQLite extension source code and test cases in the [ext/ directory](https://sqlite.org/src/tree/ext?type=tree&expand) * All code that ends up in the "sqlite3.c" and "sqlite3.h" build products that actually implement the SQLite RDBMS. * All of the code used to compile the [command-line interface](https://sqlite.org/cli.html) * All of the code used to build various utility programs such as "sqldiff", "sqlite3_rsync", and "sqlite3_analyzer". The public domain source files usually contain a header comment similar to the following to make it clear that the software is public domain. > ~~~ The author disclaims copyright to this source code. In place of a legal notice, here is a blessing: * May you do good and not evil. * May you find forgiveness for yourself and forgive others. * May you share freely, never taking more than you give. ~~~ Almost every file you find in this source repository will be public domain. But there are a small number of exceptions: Non-Public-Domain Code Included With This Source Repository AS A Convenience ---------------------------------------------------------------------------- This repository contains a (relatively) small amount of non-public-domain code used to help implement the configuration and build logic. In other words, there are some non-public-domain files used to implement: > ~~~ ./configure && make ~~~ In all cases, the non-public-domain files included with this repository have generous BSD-style licenses. So anyone is free to use any of the code in this source repository for any purpose, though attribution may be required to reuse or republish the configure and build scripts. None of the non-public-domain code ever actually reaches the build products, such as "sqlite3.c", however, so no attribution is required to use SQLite itself. The non-public-domain code consists of scripts used to help compile SQLite. The non-public-domain code is technically not part of SQLite. The non-public-domain code is included in this repository as a convenience to developers, so that those who want to build SQLite do not need to go download a bunch of third-party build scripts in order to compile SQLite. Non-public-domain code included in this respository includes: * The ["autosetup"](http://msteveb.github.io/autosetup/) configuration system that is contained (mostly) the autosetup/ directory, but also includes the "./configure" script at the top-level of this archive. Autosetup has a separate BSD-style license. See the [autosetup/LICENSE](http://msteveb.github.io/autosetup/license/) for details. * There are BSD-style licenses on some of the configuration software found in the legacy autoconf/ directory and its subdirectories. The following unix shell command is can be run from the top-level of this source repository in order to remove all non-public-domain code: > ~~~ rm -rf configure autosetup autoconf ~~~ If you unpack this source repository and then run the command above, what is left will be 100% public domain. |
Changes to Makefile.in.
|
| | > | | > > | > | > < < < > > > | > > > > > > > > > > > > > > > > > > > > | | > > | > > > > > > > > > > > > > | > > | < | > | > > > > > | < < < < < > > > > > > > | > > > | < < | | > > | > > > > > | > > > > < > | < < < > > > | | < > < | | < | | > | < > | | | | < | | > | > > > > > | | | | < > | > > > > > | | > > | > > | > > > | | < < < < < > | > > | | < < | | | > > > > | | | | | < | | | | > > > > > | | | | > | | > > > > > > > > > > > > > > > > > > > > > > > | > > > > < > > > > | | | | < < | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < < < < < < < < < < < | < < < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | < < < | < < | < < | < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | < < | < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < < < < | < < | < < | < < < < < < < < | < < | < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 | #!/usr/bin/make # ^^^^ help out editors which guess this file's type. # # Makefile for SQLITE # # This makefile is intended to be configured automatically using the # configure script. # # The docs for many of its variables are in the primary static # makefile, main.mk (which this one includes at runtime). # all: ######################################################################## # # Known TODOs/FIXMEs/TOIMPROVEs for the autosetup port, in no # particular order... # # - TEA pieces. # # - Replace the autotools-specific distribution deliverable(s). # # - Confirm whether cross-compilation works and patch it # appropriately. # # Maintenance reminders: # # - This makefile should remain as POSIX-make-compatible as possible: # https://pubs.opengroup.org/onlinepubs/9799919799/utilities/make.html # # - The naming convention of some vars, using periods instead of # underscores, though unconventional, was selected for a couple of # reasons: 1) Personal taste (for which there is no accounting). 2) # It is thought to help defend against inadvertent injection of # those vars via environment variables (because X.Y is not a legal # environment variable name). "Feature or bug?" is debatable and # this naming convention may be reverted if it causes any grief. # # # The top-most directory of the source tree. This is the directory # that contains this "Makefile.in" and the "configure" script. # TOP = @abs_top_srcdir@ # # Autotools-conventional vars which are used by package installation # rules in main.mk. To get sane handling when a user overrides only # a subset of these, we perform some acrobatics with these vars # in the configure script: see [proj-remap-autoconf-dir-vars] for # full details. # # For completeness's sake, the aforementioned conventional vars which # are relevant to our installation rules are: # # datadir = $(prefix)/share # mandir = $(datadir)/man # includedir = $(prefix)/include # exec_prefix = $(prefix) # bindir = $(exec_prefix)/bin # libdir = $(exec_prefix)/lib # # Our builds do not require any of their relatives: # # sbindir = $(exec_prefix)/sbin # sysconfdir = /etc # sharedstatedir = $(prefix)/com # localstatedir = /var # runstatedir = /run # infodir = $(datadir)/info # libexecdir = $(exec_prefix)/libexec # prefix = @prefix@ datadir = @datadir@ mandir = @mandir@ includedir = @includedir@ exec_prefix = @exec_prefix@ bindir = @bindir@ libdir = @libdir@ INSTALL = @BIN_INSTALL@ AR = @AR@ AR.flags = cr CC = @CC@ B.cc = @CC_FOR_BUILD@ @BUILD_CFLAGS@ T.cc = $(CC) # # $(CFLAGS) is problematic because it is frequently overridden when # invoking make, which loses things like -fPIC. So... we avoid using # it directly and instead add a level of indirection. We combine # $(CFLAGS) and $(CPPFLAGS) here because that's the way the legacy # build did it and many builds rely on that. See main.mk for more # details. # # Historical note: the pre-3.48 build only honored CPPFLAGS at # configure-time, and expanded them into the generated Makefile. There # are, in that build, no uses of CPPFLAGS in the configure-expanded # Makefile. Ergo: if a client configures with CPPFLAGS=... and then # explicitly passes CFLAGS=... to make, the CPPFLAGS will be # lost. That behavior is retained in 3.48+. # CFLAGS = @CFLAGS@ @CPPFLAGS@ # # $(LDFLAGS.configure) represents any LDFLAGS=... the client passes to # configure. See main.mk. # LDFLAGS.configure = @LDFLAGS@ # # CFLAGS.core is documented in main.mk. # CFLAGS.core = @SH_CFLAGS@ LDFLAGS.shlib = @SH_LDFLAGS@ LDFLAGS.zlib = @LDFLAGS_ZLIB@ LDFLAGS.math = @LDFLAGS_MATH@ LDFLAGS.rpath = @LDFLAGS_RPATH@ LDFLAGS.pthread = @LDFLAGS_PTHREAD@ LDFLAGS.dlopen = @LDFLAGS_DLOPEN@ LDFLAGS.readline = @LDFLAGS_READLINE@ CFLAGS.readline = @CFLAGS_READLINE@ LDFLAGS.icu = @LDFLAGS_ICU@ LDFLAGS.rt = @LDFLAGS_RT@ CFLAGS.icu = @CFLAGS_ICU@ LDFLAGS.libsqlite3.soname = @LDFLAGS_LIBSQLITE3_SONAME@ # soname: see https://sqlite.org/src/forumpost/5a3b44f510df8ded LDFLAGS.libsqlite3.os-specific = \ @LDFLAGS_MAC_CVERSION@ @LDFLAGS_MAC_INSTALL_NAME@ @LDFLAGS_OUT_IMPLIB@ # os-specific: see # - https://sqlite.org/forum/forumpost/9dfd5b8fd525a5d7 # - https://sqlite.org/forum/forumpost/0c7fc097b2 # - https://sqlite.org/forum/forumpost/5651662b8875ec0a libsqlite3.DLL.basename = @SQLITE_DLL_BASENAME@ # DLL.basename: see https://sqlite.org/forum/forumpost/828fdfe904 libsqlite3.out.implib = @SQLITE_OUT_IMPLIB@ # libsqlite3.out.implib => the output filename part of LDFLAGS_OUT_IMPLIB. ENABLE_LIB_SHARED = @ENABLE_LIB_SHARED@ ENABLE_LIB_STATIC = @ENABLE_LIB_STATIC@ HAVE_WASI_SDK = @HAVE_WASI_SDK@ libsqlite3.DLL.install-rules = @SQLITE_DLL_INSTALL_RULES@ # -fsanitize flags for the fuzzcheck-asap app CFLAGS.fuzzcheck-asan.fsanitize = @CFLAGS_ASAN_FSANITIZE@ # # Intended to either be empty or be set to -g -DSQLITE_DEBUG=1. # T.cc.TARGET_DEBUG = @TARGET_DEBUG@ # # $(JIMSH) and $(CFLAGS.jimsh) are documented in main.mk. $(JIMSH) # must start with a path component so that it can be invoked as a # shell command. # CFLAGS.jimsh = @CFLAGS_JIMSH@ JIMSH = ./jimsh$(T.exe) # # $(B.tclsh) is documented in main.mk. # B.tclsh = @BTCLSH@ $(B.tclsh): # # $(OPT_FEATURE_FLAGS) is documented in main.mk. # # The appending of $(OPTIONS) to $(OPT_FEATURE_FLAGS) is historical # and somewhat confusing because there's another var, $(OPTS), which # has a similar (but not identical) role. # OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ $(OPTIONS) # # Version (X.Y.Z) number for the SQLite being compiled. # PACKAGE_VERSION = @PACKAGE_VERSION@ # # Filename extensions for binaries and libraries # B.exe = @BUILD_EXEEXT@ T.exe = @TARGET_EXEEXT@ B.dll = @BUILD_DLLEXT@ T.dll = @TARGET_DLLEXT@ B.lib = @BUILD_LIBEXT@ T.lib = @TARGET_LIBEXT@ # # $(HAVE_TCL) is 1 if the configure script was able to locate the # tclConfig.sh file, else it is 0. When this variable is 1, the TCL # extension library (libtclsqlite3.so) and related testing apps are # built. # HAVE_TCL = @HAVE_TCL@ # # $(TCLSH_CMD) is the command to use for tclsh - normally just # "tclsh", but we may know the specific version we want to use. This # must point to a canonical TCL interpreter, not JimTCL. # TCLSH_CMD = @TCLSH_CMD@ TCL_CONFIG_SH = @TCL_CONFIG_SH@ # # TCL config info from tclConfig.sh # # We have to inject this differently in main.mk to accommodate static # makefiles, so we don't currently bother to export it here. This # block is retained in case we decide that we do indeed need to export # it at configure-time instead of calculate it at make-time. # #TCL_INCLUDE_SPEC = @TCL_INCLUDE_SPEC@ #TCL_LIB_SPEC = @TCL_LIB_SPEC@ #TCL_STUB_LIB_SPEC = @TCL_STUB_LIB_SPEC@ #TCL_EXEC_PREFIX = @TCL_EXEC_PREFIX@ #TCL_VERSION = @TCL_VERSION@ TCL_MAJOR_VERSION = @TCL_MAJOR_VERSION@ # ^^^ main.mk optionally uses this for determining the Tcl extension's # DLL name. TCL_EXT_DLL_BASENAME = @TCL_EXT_DLL_BASENAME@ # ^^^ base name of the Tcl extension DLL. It varies by platform and # Tcl version. # # $(TCLLIBDIR) = where to install the tcl plugin. If this is empty, it # is calculated at make-time by the targets which need it but we # export it here so that it can be set at configure-time, so that # clients are not required to pass it at make-time, or may set it in # their environment to override it. # TCLLIBDIR = @TCLLIBDIR@ # # Additional options when running tests using testrunner.tcl # This is usually either blank or --status. # TSTRNNR_OPTS = @TSTRNNR_OPTS@ # # If gcov support was enabled by the configure script, add the appropriate # flags here. It's not always as easy as just having the user add the right # CFLAGS / LDFLAGS, because libtool wants to use CFLAGS when linking, which # causes build errors with -fprofile-arcs -ftest-coverage with some GCCs. # Supposedly GCC does the right thing if you use --coverage, but in # practice it still fails. See: # # http://www.mail-archive.com/debian-gcc@lists.debian.org/msg26197.html # # for more info. # CFLAGS.gcov1 = -DSQLITE_COVERAGE_TEST=1 -fprofile-arcs -ftest-coverage LDFLAGS.gcov1 = -lgcov USE_GCOV = @USE_GCOV@ T.compile.gcov = $(CFLAGS.gcov$(USE_GCOV)) T.link.gcov = $(LDFLAGS.gcov$(USE_GCOV)) # # Vars with the AS_ prefix are specifically related to AutoSetup. # # AS_AUTO_DEF is the main configure script. # AS_AUTO_DEF = $(TOP)/auto.def # # Shell commands to re-run $(TOP)/configure with the same args it was # invoked with to produce this makefile. # AS_AUTORECONFIG = @SQLITE_AUTORECONFIG@ USE_AMALGAMATION ?= @USE_AMALGAMATION@ LINK_TOOLS_DYNAMICALLY ?= @LINK_TOOLS_DYNAMICALLY@ AMALGAMATION_GEN_FLAGS ?= --linemacros=@AMALGAMATION_LINE_MACROS@ EXTRA_SRC ?= @AMALGAMATION_EXTRA_SRC@ STATIC_TCLSQLITE3 = @STATIC_TCLSQLITE3@ STATIC_CLI_SHELL = @STATIC_CLI_SHELL@ # # CFLAGS for sqlite3$(T.exe) # SHELL_OPT ?= @OPT_SHELL@ Makefile: $(TOP)/Makefile.in $(AS_AUTO_DEF) $(AS_AUTORECONFIG) @touch $@ sqlite3.pc: $(TOP)/sqlite3.pc.in $(AS_AUTO_DEF) $(AS_AUTORECONFIG) @touch $@ install: install-pc # defined in main.mk sqlite_cfg.h: $(AS_AUTO_DEF) $(AS_AUTORECONFIG) @touch $@ # # Fiddle app # # EMCC_WRAPPER must refer to the genuine emcc binary, or a # call-compatible wrapper, e.g. $(TOP)/tool/emcc.sh. If it's empty, # build components requiring Emscripten will not build. # # Achtung: though _this_ makefile is POSIX-make compatible, the fiddle # build requires GNU make. # EMCC_WRAPPER = @EMCC_WRAPPER@ fiddle: sqlite3.c shell.c @if [ x = "x$(EMCC_WRAPPER)" ]; then \ echo "Emscripten SDK not found by configure. Cannot build fiddle." 1&>2; \ exit 1; \ fi $(MAKE) -C ext/wasm fiddle emcc_opt=-Os # # Spell-checking for source comments # The sources checked are either C sources or C source templates. # Their comments are extracted and processed through aspell using # a custom dictionary that contains scads of odd identifiers that # find their way into the comments. # # Currently, this target is setup to be "made" in-tree only. # The output is ephemeral. Redirect it to guide spelling fixups, # either to correct spelling or add words to tool/custom.txt. # ./custom.rws: ./tool/custom.txt @echo 'Updating custom dictionary from tool/custom.txt' aspell --lang=en create master ./custom.rws < ./tool/custom.txt # Note that jimsh does not work here: # https://github.com/msteveb/jimtcl/issues/319 misspell: ./custom.rws has_tclsh84 $(TCLSH_CMD) ./tool/spellsift.tcl ./src/*.c ./src/*.h ./src/*.in # # clean/distclean are mostly defined in main.mk. In this makefile we # perform cleanup known to be relevant to (only) the autosetup-driven # build. # distclean-autosetup: rm -f sqlite_cfg.h config.log config.status config.defines.* Makefile sqlite3.pc rm -f $(TOP)/tool/emcc.sh rm -f libsqlite3*$(T.dll) rm -f jimsh0* distclean: distclean-autosetup # # tool/version-info: a utility for emitting sqlite3 version info # in various forms. # version-info$(T.exe): $(TOP)/tool/version-info.c Makefile sqlite3.h $(T.link) $(ST_OPT) -o $@ $(TOP)/tool/version-info.c include $(TOP)/main.mk |
Deleted Makefile.linux-gcc.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Added Makefile.linux-generic.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | #!/usr/make all: # # Makefile for SQLITE # # This is a template makefile for SQLite. Most people prefer to # use the autoconf generated "configure" script to generate the # makefile automatically. But that does not work for everybody # and in every situation. If you are having problems with the # "configure" script, you might want to try this makefile as an # alternative. Create a copy of this file, edit the parameters # below and type "make". # # Maintenance note: because this is the template for Linux systems, it # is assumed that the platform has GNU make and this file takes # advantage of that. # #### # # $(TOP) = The toplevel directory of the source tree. This is the # directory that contains "Makefile.in" and "auto.def". # TOP ?= $(realpath $(dir $(lastword $(MAKEFILE_LIST)))) # # $(CFLAGS) will be used when compiling the library and most # utilities. It must normally contain -fPIC on Linux systems, # but overriding CFLAGS is an easy way for users to inadvertently # remove -fPIC from their builds, so we generally expect to see # -fPIC in $(CFLAGS.core), which main.mk will integrate with # the CFLAGS where needed. # CFLAGS = CFLAGS.core = -fPIC # # $(SHELL_OPT) contains CFLAGS for building the sqlite3 CLI shell. # See main.mk for other potentially-relevant vars which may need # tweaking, like $(LDFLAGS_READLINE). # SHELL_OPT += -DHAVE_READLINE=1 SHELL_OPT += -DSQLITE_HAVE_ZLIB=1 LDFLAGS.readline = -lreadline # may need -lcurses etc, depending on the system CFLAGS.readline = # needs -I... if readline.h is in an unusual place. LDFLAGS.zlib = -lz # # Library's version number. # PACKAGE_VERSION ?= $(shell cat $(TOP)/VERSION 2>/dev/null) # sqlite_cfg.h is typically created by the configure script. It's # commonly not needed but main.mk does not know that so we have to # create a dummy if we don't already have one. sqlite_cfg.h: touch $@ distclean-.: rm -f sqlite_cfg.h # # With the above in place, we can now import the rules make use of # it... # include $(TOP)/main.mk |
Changes to Makefile.msc.
︙ | ︙ | |||
13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # <<mark>> # Set this non-0 to create and use the SQLite amalgamation file. # !IFNDEF USE_AMALGAMATION USE_AMALGAMATION = 1 !ENDIF # <</mark>> # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN USE_FULLWARN = 1 !ENDIF | > > > > > > > > > | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | # <<mark>> # Set this non-0 to create and use the SQLite amalgamation file. # !IFNDEF USE_AMALGAMATION USE_AMALGAMATION = 1 !ENDIF # <</mark>> # Optionally set EXTRA_SRC to a list of C files to append to # the generated sqlite3.c. Any sqlite3 extensions added this # way may require manual editing, as described in # https://sqlite.org/forum/forumpost/903f721f3e7c0d25 # !IFNDEF EXTRA_SRC EXTRA_SRC = !ENDIF # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN USE_FULLWARN = 1 !ENDIF |
︙ | ︙ | |||
47 48 49 50 51 52 53 54 55 56 57 58 59 60 | # Set this non-0 to use "stdcall" calling convention for the core library # and shell executable. # !IFNDEF USE_STDCALL USE_STDCALL = 0 !ENDIF # Set this non-0 to have the shell executable link against the core dynamic # link library. # !IFNDEF DYNAMIC_SHELL DYNAMIC_SHELL = 0 !ENDIF | > > > > > > > > > > > > > > > | 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | # Set this non-0 to use "stdcall" calling convention for the core library # and shell executable. # !IFNDEF USE_STDCALL USE_STDCALL = 0 !ENDIF # Use the USE_SEH=0 option on the nmake command line to omit structured # exception handling (SEH) support. SEH is on by default. # !IFNDEF USE_SEH USE_SEH = 1 !ENDIF # Use STATICALLY_LINK_TCL=1 to statically link against TCL # !IFNDEF STATICALLY_LINK_TCL STATICALLY_LINK_TCL = 0 !ELSEIF $(STATICALLY_LINK_TCL)!=0 CCOPTS = $(CCOPTS) -DSTATIC_BUILD !ENDIF # Set this non-0 to have the shell executable link against the core dynamic # link library. # !IFNDEF DYNAMIC_SHELL DYNAMIC_SHELL = 0 !ENDIF |
︙ | ︙ | |||
213 214 215 216 217 218 219 220 221 222 223 224 225 226 | # Set this to non-0 to enable OSTRACE() macros, which can be useful when # debugging. # !IFNDEF OSTRACE OSTRACE = 0 !ENDIF # Set this to one of the following values to enable various debugging # features. Each level includes the debugging options from the previous # levels. Currently, the recognized values for DEBUG are: # # 0 == NDEBUG: Disables assert() and other runtime diagnostics. # 1 == SQLITE_ENABLE_API_ARMOR: extra attempts to detect misuse of the API. | > > > > > > | 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 | # Set this to non-0 to enable OSTRACE() macros, which can be useful when # debugging. # !IFNDEF OSTRACE OSTRACE = 0 !ENDIF # enable address sanitizer using ASAN=1 on the command-line. # !IFNDEF ASAN ASAN = 0 !ENDIF # Set this to one of the following values to enable various debugging # features. Each level includes the debugging options from the previous # levels. Currently, the recognized values for DEBUG are: # # 0 == NDEBUG: Disables assert() and other runtime diagnostics. # 1 == SQLITE_ENABLE_API_ARMOR: extra attempts to detect misuse of the API. |
︙ | ︙ | |||
259 260 261 262 263 264 265 266 267 268 269 270 271 272 | !ENDIF # Set this to non-0 to enable support for the rbu extension. # !IFNDEF RBU RBU = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c | > > > > > > | 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 | !ENDIF # Set this to non-0 to enable support for the rbu extension. # !IFNDEF RBU RBU = 0 !ENDIF # Set this to non-0 to enable support for blocking locks. # !IFNDEF SETLK_TIMEOUT SETLK_TIMEOUT = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c |
︙ | ︙ | |||
355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 | !ENDIF # <</mark>> # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 !ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF # Should the session extension be enabled? If so, add compilation options # to enable it. # !IF $(SESSION)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 !ENDIF # Always enable math functions on Windows OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS # Should the rbu extension be enabled? If so, add compilation options # to enable it. # !IF $(RBU)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 !ENDIF # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # !IFNDEF EXT_FEATURE_FLAGS !IF $(FOR_WIN10)!=0 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS4=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_SYSTEM_MALLOC=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_OMIT_LOCALTIME=1 !ELSE EXT_FEATURE_FLAGS = !ENDIF !ENDIF ############################################################################### ############################### END OF OPTIONS ################################ ############################################################################### # When compiling for the Windows 10 platform, the PLATFORM macro must be set # to an appropriate value (e.g. x86, x64, arm, arm64, etc). | > > > > > > > > > > > > > > > > > > > > > > | 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 | !ENDIF # <</mark>> # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS OPT_FEATURE_FLAGS = $(OPT_XTRA) !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 !ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF # Additional feature-options above and beyond what are normally used can be # be added using OPTIONS=.... on the command-line. These values are # appended to the OPT_FEATURE_FLAGS variable. # !IFDEF OPTIONS OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) $(OPTIONS) !ENDIF # Should the session extension be enabled? If so, add compilation options # to enable it. # !IF $(SESSION)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 !ENDIF # Always enable math functions on Windows OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS # Should the rbu extension be enabled? If so, add compilation options # to enable it. # !IF $(RBU)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 !ENDIF # Should structured exception handling (SEH) be enabled for WAL mode in # the core library? It is on by default. Only omit it if the # USE_SEH=0 option is provided on the nmake command-line. # !IF $(USE_SEH)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1 !ENDIF # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # !IFNDEF EXT_FEATURE_FLAGS !IF $(FOR_WIN10)!=0 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS4=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_SYSTEM_MALLOC=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_OMIT_LOCALTIME=1 !ELSE EXT_FEATURE_FLAGS = !ENDIF !ENDIF !IF $(SETLK_TIMEOUT)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SETLK_TIMEOUT !ENDIF ############################################################################### ############################### END OF OPTIONS ################################ ############################################################################### # When compiling for the Windows 10 platform, the PLATFORM macro must be set # to an appropriate value (e.g. x86, x64, arm, arm64, etc). |
︙ | ︙ | |||
822 823 824 825 826 827 828 | !IF $(DEBUG)>0 || $(API_ARMOR)!=0 || $(FOR_WIN10)!=0 TCC = $(TCC) -DSQLITE_ENABLE_API_ARMOR=1 RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !ENDIF !IF $(DEBUG)>2 | | | 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 | !IF $(DEBUG)>0 || $(API_ARMOR)!=0 || $(FOR_WIN10)!=0 TCC = $(TCC) -DSQLITE_ENABLE_API_ARMOR=1 RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !ENDIF !IF $(DEBUG)>2 TCC = $(TCC) -DSQLITE_DEBUG=1 -DSQLITE_USE_W32_FOR_CONSOLE_IO RCC = $(RCC) -DSQLITE_DEBUG=1 !IF $(DYNAMIC_SHELL)==0 TCC = $(TCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE RCC = $(RCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE !ENDIF !ENDIF |
︙ | ︙ | |||
873 874 875 876 877 878 879 880 881 882 883 884 885 886 | # !IF $(DEBUG)>3 TCC = $(TCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 !ENDIF !ENDIF # <<mark>> # The locations of the Tcl header and library files. Also, the library that # non-stubs enabled programs using Tcl must link against. These variables # (TCLINCDIR, TCLLIBDIR, and LIBTCL) may be overridden via the environment # prior to running nmake in order to match the actual installed location and # version on this machine. # | > > > > > > > | | < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 | # !IF $(DEBUG)>3 TCC = $(TCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 !ENDIF !ENDIF # Address sanitizer if ASAN=1 # !IF $(ASAN)>0 TCC = $(TCC) /fsanitize=address !ENDIF # <<mark>> # The locations of the Tcl header and library files. Also, the library that # non-stubs enabled programs using Tcl must link against. These variables # (TCLINCDIR, TCLLIBDIR, and LIBTCL) may be overridden via the environment # prior to running nmake in order to match the actual installed location and # version on this machine. # !IF $(STATICALLY_LINK_TCL)!=0 TCLSUFFIX = s !ENDIF !IFNDEF TCLSUFFIX TCLSUFFIX = !ENDIF !IFNDEF TCLDIR TCLDIR = C:\Tcl !ENDIF !IFNDEF TCLVERSION !IF EXISTS("$(TCLDIR)\lib\tcl90$(TCLSUFFIX).lib") TCLVERSION = 90 !ELSEIF EXISTS("$(TCLDIR)\lib\tcl86$(TCLSUFFIX).lib") TCLVERSION = 86 !ELSEIF EXISTS("$(TCLDIR)\lib\tcl86t.lib") TCLSUFFIX = t TCLVERSION = 86 !ELSE TCLVERSION = 90 !ENDIF !ENDIF !IFNDEF TCLINCDIR TCLINCDIR = $(TCLDIR)\include !ENDIF !IFNDEF TCLLIBDIR TCLLIBDIR = $(TCLDIR)\lib !ENDIF !IFNDEF LIBTCL LIBTCL = tcl$(TCLVERSION)$(TCLSUFFIX).lib !ENDIF !IFNDEF TCLLIBS !IF $(STATICALLY_LINK_TCL)!=0 TCLLIBS = /NODEFAULTLIB:libucrt.lib netapi32.lib user32.lib ucrt.lib !ELSE TCLLIBS = !ENDIF !ENDIF !IFNDEF LIBTCLSTUB !IF EXISTS("$(TCLLIBDIR)\tclstub$(TCLVERSION)$(TCLSUFFIX).lib") LIBTCLSTUB = tclstub$(TCLVERSION)$(TCLSUFFIX).lib !ELSEIF EXISTS("$(TCLLIBDIR)\tclstub$(TCLSUFFIX).lib") LIBTCLSTUB = tclstub$(TCLSUFFIX).lib !ELSEIF EXISTS("$(TCLLIBDIR)\tclstub$(TCLVERSION).lib") LIBTCLSTUB = tclstub$(TCLVERSION).lib !ELSE LIBTCLSTUB = tclstub.lib !ENDIF !ENDIF !IFNDEF LIBTCLPATH LIBTCLPATH = $(TCLDIR)\bin !ENDIF # The locations of the zlib header and library files. These variables |
︙ | ︙ | |||
964 965 966 967 968 969 970 | # This is the command to use for tclsh - normally just "tclsh", but we may # know the specific version we want to use. This variable (TCLSH_CMD) may be # overridden via the environment prior to running nmake in order to select a # specific Tcl shell to use. # !IFNDEF TCLSH_CMD | > > > > > > > > | | | > > > > > > > > > > > > | 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 | # This is the command to use for tclsh - normally just "tclsh", but we may # know the specific version we want to use. This variable (TCLSH_CMD) may be # overridden via the environment prior to running nmake in order to select a # specific Tcl shell to use. # !IFNDEF TCLSH_CMD !IF EXISTS("$(TCLDIR)\bin\tclsh$(TCLVERSION).exe") TCLSH_CMD = $(TCLDIR)\bin\tclsh$(TCLVERSION).exe !ELSEIF EXISTS("$(TCLDIR)\bin\tclsh90.exe") TCLSH_CMD = $(TCLDIR)\bin\tclsh90.exe !ELSEIF EXISTS("$(TCLDIR)\bin\tclsh86.exe") TCLSH_CMD = $(TCLDIR)\bin\tclsh86.exe !ELSEIF EXISTS("$(TCLDIR)\bin\tclsh86t.exe") TCLSH_CMD = $(TCLDIR)\bin\tclsh86t.exe !ELSEIF EXISTS("$(TCLDIR)\bin\tclsh.exe") TCLSH_CMD = $(TCLDIR)\bin\tclsh.exe !ELSE TCLSH_CMD = tclsh !ENDIF !ENDIF # A light-weight TCLSH replacement that can be used for code generation # but which is not adequate for testing. This is "jimsh0" by default, # with source code in the repository. To force the whole build to use # the full, official tclsh, add WITHOUT_JIMSH=1 to the nmake command line. # !IFDEF WITHOUT_JIMSH JIM_TCLSH = $(TCLSH_CMD) !ENDIF !IFNDEF JIM_TCLSH JIM_TCLSH = jimsh0.exe !ENDIF # <</mark>> # Compiler options needed for programs that use the readline() library. # !IFNDEF READLINE_FLAGS READLINE_FLAGS = -DHAVE_READLINE=0 |
︙ | ︙ | |||
1000 1001 1002 1003 1004 1005 1006 | # Any target libraries which libsqlite must be linked against # !IFNDEF TLIBS TLIBS = !ENDIF | < < < < < < < < < | 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 | # Any target libraries which libsqlite must be linked against # !IFNDEF TLIBS TLIBS = !ENDIF # Enable/disable loadable extensions, and other optional features # based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*). # The same set of OMIT and ENABLE flags should be passed to the # LEMON parser generator and the mkkeywordhash tool as well. # These are the required SQLite compilation options used when compiling for # the Windows platform. |
︙ | ︙ | |||
1298 1299 1300 1301 1302 1303 1304 | $(TOP)\src\backup.c \ $(TOP)\src\bitvec.c \ $(TOP)\src\btmutex.c \ $(TOP)\src\btree.c \ $(TOP)\src\build.c \ $(TOP)\src\callback.c \ $(TOP)\src\complete.c \ | | | 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 | $(TOP)\src\backup.c \ $(TOP)\src\bitvec.c \ $(TOP)\src\btmutex.c \ $(TOP)\src\btree.c \ $(TOP)\src\build.c \ $(TOP)\src\callback.c \ $(TOP)\src\complete.c \ ctime.c \ $(TOP)\src\date.c \ $(TOP)\src\dbpage.c \ $(TOP)\src\dbstat.c \ $(TOP)\src\delete.c \ $(TOP)\src\expr.c \ $(TOP)\src\fault.c \ $(TOP)\src\fkey.c \ |
︙ | ︙ | |||
1397 1398 1399 1400 1401 1402 1403 | $(TOP)\src\os_win.h # Core header files, part 2. # SRC05 = \ $(TOP)\src\pager.h \ $(TOP)\src\pcache.h \ | | | 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 | $(TOP)\src\os_win.h # Core header files, part 2. # SRC05 = \ $(TOP)\src\pager.h \ $(TOP)\src\pcache.h \ pragma.h \ $(TOP)\src\sqlite.h.in \ $(TOP)\src\sqlite3ext.h \ $(TOP)\src\sqliteInt.h \ $(TOP)\src\sqliteLimit.h \ $(TOP)\src\vdbe.h \ $(TOP)\src\vdbeInt.h \ $(TOP)\src\vxworks.h \ |
︙ | ︙ | |||
1481 1482 1483 1484 1485 1486 1487 | TESTSRC = \ $(TOP)\src\test1.c \ $(TOP)\src\test2.c \ $(TOP)\src\test3.c \ $(TOP)\src\test4.c \ $(TOP)\src\test5.c \ $(TOP)\src\test6.c \ | < < | 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 | TESTSRC = \ $(TOP)\src\test1.c \ $(TOP)\src\test2.c \ $(TOP)\src\test3.c \ $(TOP)\src\test4.c \ $(TOP)\src\test5.c \ $(TOP)\src\test6.c \ $(TOP)\src\test8.c \ $(TOP)\src\test9.c \ $(TOP)\src\test_autoext.c \ $(TOP)\src\test_backup.c \ $(TOP)\src\test_bestindex.c \ $(TOP)\src\test_blob.c \ $(TOP)\src\test_btree.c \ $(TOP)\src\test_config.c \ $(TOP)\src\test_delete.c \ $(TOP)\src\test_demovfs.c \ |
︙ | ︙ | |||
1510 1511 1512 1513 1514 1515 1516 | $(TOP)\src\test_mutex.c \ $(TOP)\src\test_onefile.c \ $(TOP)\src\test_osinst.c \ $(TOP)\src\test_pcache.c \ $(TOP)\src\test_quota.c \ $(TOP)\src\test_rtree.c \ $(TOP)\src\test_schema.c \ | < < | | 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 | $(TOP)\src\test_mutex.c \ $(TOP)\src\test_onefile.c \ $(TOP)\src\test_osinst.c \ $(TOP)\src\test_pcache.c \ $(TOP)\src\test_quota.c \ $(TOP)\src\test_rtree.c \ $(TOP)\src\test_schema.c \ $(TOP)\src\test_superlock.c \ $(TOP)\src\test_syscall.c \ $(TOP)\src\test_tclsh.c \ $(TOP)\src\test_tclvar.c \ $(TOP)\src\test_thread.c \ $(TOP)\src\test_vdbecov.c \ $(TOP)\src\test_vfs.c \ $(TOP)\src\test_window.c \ $(TOP)\src\test_wsd.c \ $(TOP)\ext\fts3\fts3_term.c \ $(TOP)\ext\fts3\fts3_test.c \ $(TOP)\ext\rbu\test_rbu.c \ $(TOP)\ext\session\test_session.c # Statically linked extensions. # TESTEXT = \ $(TOP)\ext\expert\sqlite3expert.c \ $(TOP)\ext\expert\test_expert.c \ $(TOP)\ext\misc\amatch.c \ |
︙ | ︙ | |||
1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 | $(TOP)\ext\misc\ieee754.c \ $(TOP)\ext\misc\mmapwarm.c \ $(TOP)\ext\misc\nextchar.c \ $(TOP)\ext\misc\normalize.c \ $(TOP)\ext\misc\percentile.c \ $(TOP)\ext\misc\prefixes.c \ $(TOP)\ext\misc\qpvtab.c \ $(TOP)\ext\misc\regexp.c \ $(TOP)\ext\misc\remember.c \ $(TOP)\ext\misc\series.c \ $(TOP)\ext\misc\spellfix.c \ $(TOP)\ext\misc\totype.c \ $(TOP)\ext\misc\unionvtab.c \ $(TOP)\ext\misc\wholenumber.c \ $(TOP)\ext\rtree\test_rtreedoc.c \ $(TOP)\ext\recover\sqlite3recover.c \ $(TOP)\ext\recover\test_recover.c \ | > > > > | < | | | 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 | $(TOP)\ext\misc\ieee754.c \ $(TOP)\ext\misc\mmapwarm.c \ $(TOP)\ext\misc\nextchar.c \ $(TOP)\ext\misc\normalize.c \ $(TOP)\ext\misc\percentile.c \ $(TOP)\ext\misc\prefixes.c \ $(TOP)\ext\misc\qpvtab.c \ $(TOP)\ext\misc\randomjson.c \ $(TOP)\ext\misc\regexp.c \ $(TOP)\ext\misc\remember.c \ $(TOP)\ext\misc\series.c \ $(TOP)\ext\misc\spellfix.c \ $(TOP)\ext\misc\stmtrand.c \ $(TOP)\ext\misc\totype.c \ $(TOP)\ext\misc\unionvtab.c \ $(TOP)\ext\misc\wholenumber.c \ $(TOP)\ext\rtree\test_rtreedoc.c \ $(TOP)\ext\recover\sqlite3recover.c \ $(TOP)\ext\recover\test_recover.c \ $(TOP)\ext\intck\test_intck.c \ $(TOP)\ext\intck\sqlite3intck.c \ $(TOP)\ext\recover\dbdata.c # If use of zlib is enabled, add the "zipfile.c" source file. # !IF $(USE_ZLIB)!=0 TESTEXT = $(TESTEXT) $(TOP)\ext\misc\zipfile.c !ENDIF # Source code to the library files needed by the test fixture # (non-amalgamation) # TESTSRC2 = \ $(SRC00) \ $(SRC01) \ $(SRC07) \ $(SRC10) \ fts5.c # Header files used by all library source files. # HDR = \ $(TOP)\src\btree.h \ $(TOP)\src\btreeInt.h \ $(TOP)\src\hash.h \ $(TOP)\src\hwtime.h \ keywordhash.h \ $(TOP)\src\msvc.h \ $(TOP)\src\mutex.h \ opcodes.h \ $(TOP)\src\os.h \ $(TOP)\src\os_common.h \ $(TOP)\src\os_setup.h \ $(TOP)\src\os_win.h \ $(TOP)\src\pager.h \ $(TOP)\src\pcache.h \ parse.h \ pragma.h \ $(SQLITE3H) \ sqlite3ext.h \ $(TOP)\src\sqliteInt.h \ $(TOP)\src\sqliteLimit.h \ $(TOP)\src\vdbe.h \ $(TOP)\src\vdbeInt.h \ $(TOP)\src\vxworks.h \ |
︙ | ︙ | |||
1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 | $(TOP)\test\fuzzdata8.db # <</mark>> # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 !ENDIF # <<mark>> # Extra compiler options for various test tools. # MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_FTS5 FUZZERSHELL_COMPILE_OPTS = FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -I$(TOP)\test -I$(TOP)\ext\recover FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OSS_FUZZ FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzcheck.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\ossfuzz.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzinvariants.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\vt02.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\dbdata.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\sqlite3recover.c OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION KV_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ ST_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 # Standard options to testfixture. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 | $(TOP)\test\fuzzdata8.db # <</mark>> # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_STRICT_SUBTYPE=1 !ENDIF # <<mark>> # Extra compiler options for various test tools. # MPTESTER_COMPILE_OPTS = -DSQLITE_ENABLE_FTS5 FUZZERSHELL_COMPILE_OPTS = FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -I$(TOP)\test -I$(TOP)\ext\recover FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OSS_FUZZ FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBPAGE_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DESERIALIZE FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS3_PARENTHESIS FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MATH_FUNCTIONS FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_MEMSYS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_NORMALIZE FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_PREUPDATE_HOOK FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_SESSION FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STMTVTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STAT4 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MMAP_SIZE=0 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRIVATE="" FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_STRICT_SUBTYPE=1 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_STATIC_RANDOMJSON FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_MAX_MEMORY=50000000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_PRINTF_PRECISION_LIMIT=1000 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_OMIT_LOAD_EXTENSION FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS4 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_FTS5 FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_RTREE FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_GEOPOLY FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_DBSTAT_VTAB FUZZCHECK_OPTS = $(FUZZCHECK_OPTS) -DSQLITE_ENABLE_BYTECODE_VTAB FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzcheck.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\ossfuzz.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\fuzzinvariants.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\test\vt02.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\dbdata.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\recover\sqlite3recover.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\percentile.c FUZZCHECK_SRC = $(FUZZCHECK_SRC) $(TOP)\ext\misc\randomjson.c OSSSHELL_SRC = $(TOP)\test\ossshell.c $(TOP)\test\ossfuzz.c DBFUZZ_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_OMIT_LOAD_EXTENSION KV_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 -DSQLITE_DIRECT_OVERFLOW_READ ST_COMPILE_OPTS = -DSQLITE_THREADSAFE=0 # Standard options to testfixture. |
︙ | ︙ | |||
1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 | # dll: $(SQLITE3DLL) # Shell executable. # shell: $(SQLITE3EXE) # <<mark>> libsqlite3.lib: $(LIBOBJ) $(LTLIB) $(LTLIBOPTS) /OUT:$@ $(LIBOBJ) $(TLIBS) libtclsqlite3.lib: tclsqlite.lo libsqlite3.lib $(LTLIB) $(LTLIBOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite.lo libsqlite3.lib $(LIBTCLSTUB) $(TLIBS) tclsqlite3.def: tclsqlite.lo echo EXPORTS > tclsqlite3.def dumpbin /all tclsqlite.lo \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl include "^\s+/EXPORT:_?((?:Sqlite3|Tclsqlite3)_[^@]*)(?:@\d+)?$$" \1 \ | sort >> tclsqlite3.def pkgIndex.tcl: $(TOP)\VERSION for /F %%V in ('type "$(TOP)\VERSION"') do ( \ | > > > > > | > > > > > > > > > > > > > > > > | | | | > > > > > > > > > > > > > > > > > > > | > > > | 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 | # dll: $(SQLITE3DLL) # Shell executable. # shell: $(SQLITE3EXE) # jimsh0 - replacement for tclsh # jimsh0.exe: $(TOP)\autosetup\jimsh0.c cl -DHAVE__FULLPATH=1 $(TOP)\autosetup\jimsh0.c # <<mark>> libsqlite3.lib: $(LIBOBJ) $(LTLIB) $(LTLIBOPTS) /OUT:$@ $(LIBOBJ) $(TLIBS) libtclsqlite3.lib: tclsqlite.lo libsqlite3.lib $(LTLIB) $(LTLIBOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite.lo libsqlite3.lib $(LIBTCLSTUB) $(TLIBS) tclsqlite3.def: tclsqlite.lo echo EXPORTS > tclsqlite3.def dumpbin /all tclsqlite.lo \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl include "^\s+/EXPORT:_?((?:Sqlite3|Tclsqlite3)_[^@]*)(?:@\d+)?$$" \1 \ | sort >> tclsqlite3.def pkgIndex.tcl: $(TOP)\VERSION for /F %%V in ('type "$(TOP)\VERSION"') do ( \ echo package ifneeded sqlite3 @version@ [list load [file join $$dir $(SQLITE3TCLDLL)] Sqlite3] \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact @version@ %%V > pkgIndex.tcl \ ) $(SQLITE3TCLDLL): libtclsqlite3.lib $(LIBRESOBJS) tclsqlite3.def pkgIndex.tcl $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL /DEF:tclsqlite3.def /OUT:$@ libtclsqlite3.lib $(LIBRESOBJS) $(LTLIBS) $(TLIBS) tclextension: $(SQLITE3TCLDLL) tclextension-install: $(SQLITE3TCLDLL) $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --install-only tclextension-uninstall: $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --uninstall tclextension-list: @ $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --info tclextension-verify: sqlite3.h @ $(TCLSH_CMD) $(TOP)\tool\buildtclext.tcl --version-check # <</mark>> $(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) # <<block2>> sqlite3.def: libsqlite3.lib $(JIM_TCLSH) echo EXPORTS > sqlite3.def dumpbin /all libsqlite3.lib \ | $(JIM_TCLSH) $(TOP)\tool\replace.tcl include "^\s+1 _?(sqlite3(?:session|changeset|changegroup|rebaser|rbu)?_[^@]*)(?:@\d+)?$$" \1 \ | sort >> sqlite3.def # <</block2>> $(SQLITE3EXE): shell.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(SHELL_CORE_SRC) $(SQLITE3H) $(LTLINK) $(SHELL_COMPILE_OPTS) $(READLINE_FLAGS) shell.c $(SHELL_CORE_SRC) \ /link $(SQLITE3EXEPDB) $(LDFLAGS) $(LTLINKOPTS) $(SHELL_LINK_OPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LIBREADLINE) $(LTLIBS) $(TLIBS) # <<mark>> sqldiff.exe: $(TOP)\tool\sqldiff.c $(TOP)\ext\misc\sqlite3_stdio.h $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -I$(TOP)\ext\misc $(TOP)\tool\sqldiff.c $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS) dbhash.exe: $(TOP)\tool\dbhash.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(TOP)\tool\dbhash.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) RSYNC_SRC = \ $(TOP)\tool\sqlite3_rsync.c \ $(SQLITE3C) RSYNC_OPT = \ -DSQLITE_ENABLE_DBPAGE_VTAB \ -DSQLITE_THREADSAFE=0 \ -DSQLITE_OMIT_LOAD_EXTENSION \ -DSQLITE_OMIT_DEPRECATED sqlite3_rsync.exe: $(RSYNC_SRC) $(LIBRESOBJS) $(LTLINK) $(RSYNC_OPT) $(NO_WARN) $(RSYNC_SRC) /link $(LDFLAGS) $(LTLINKOPTS) $(LIBRESOBJS) scrub.exe: $(TOP)\ext\misc\scrub.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -DSCRUB_STANDALONE=1 $(TOP)\ext\misc\scrub.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) srcck1.exe: $(TOP)\tool\srcck1.c $(BCC) $(NO_WARN) -Fe$@ $(TOP)\tool\srcck1.c sourcetest: srcck1.exe $(SQLITE3C) srcck1.exe $(SQLITE3C) src-verify.exe: $(TOP)\tool\src-verify.c $(LTLINK) $(NO_WARN) $(TOP)\tool\src-verify.c verify-source: src-verify.exe src-verify.exe $(TOP) fuzzershell.exe: $(TOP)\tool\fuzzershell.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZERSHELL_COMPILE_OPTS) $(TOP)\tool\fuzzershell.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) dbfuzz.exe: $(TOP)\test\dbfuzz.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(DBFUZZ_COMPILE_OPTS) $(TOP)\test\dbfuzz.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) fuzzcheck.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H) $(LTLINK) /F 8388608 $(NO_WARN) $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) fuzzcheck-asan.exe: $(FUZZCHECK_SRC) $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) /fsanitize=address $(FUZZCHECK_OPTS) $(FUZZCHECK_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) ossshell.exe: $(OSSSHELL_SRC) $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(FUZZCHECK_OPTS) $(OSSSHELL_SRC) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) sessionfuzz.exe: zlib $(TOP)\test\sessionfuzz.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -I$(ZLIBINCDIR) $(TOP)\test\sessionfuzz.c /link $(LDFLAGS) $(LTLINKOPTS) /LIBPATH:$(ZLIBLIBDIR) $(ZLIBLIB) |
︙ | ︙ | |||
1816 1817 1818 1819 1820 1821 1822 | # This target creates a directory named "tsrc" and fills it with # copies of all of the C source code and header files needed to # build on the target system. Some of the C source code and header # files are automatically generated. This target takes care of # all that automatic generation. # | | < < < < < < < < < < | < | | | | | | 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 | # This target creates a directory named "tsrc" and fills it with # copies of all of the C source code and header files needed to # build on the target system. Some of the C source code and header # files are automatically generated. This target takes care of # all that automatic generation. # .target_source: $(SRC) $(TOP)\tool\vdbe-compress.tcl fts5.c $(SQLITE_TCL_DEP) $(JIM_TCLSH) -rmdir /Q/S tsrc 2>NUL -mkdir tsrc $(JIM_TCLSH) $(TOP)\tool\cp.tcl $(SRC00) $(SRC01) $(SRC03) $(SRC04) $(SRC05) $(SRC07) $(SRC09) $(SRC10) $(SRC11) $(SRC12) fts5.c fts5.h tsrc copy /B tsrc\fts5.c +,, copy /B tsrc\fts5.h +,, del /Q tsrc\sqlite.h.in tsrc\parse.y 2>NUL $(JIM_TCLSH) $(TOP)\tool\vdbe-compress.tcl $(OPTS) < tsrc\vdbe.c > vdbe.new move vdbe.new tsrc\vdbe.c echo > .target_source sqlite3.c: .target_source sqlite3ext.h sqlite3session.h $(MKSQLITE3C_TOOL) src-verify.exe $(JIM_TCLSH) $(JIM_TCLSH) $(MKSQLITE3C_TOOL) $(MKSQLITE3C_ARGS) $(EXTRA_SRC) sqlite3-all.c: sqlite3.c $(TOP)\tool\split-sqlite3c.tcl $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\tool\split-sqlite3c.tcl # <</mark>> # Rule to build the amalgamation # sqlite3.lo: $(SQLITE3C) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(SQLITE3C) |
︙ | ︙ | |||
1884 1885 1886 1887 1888 1889 1890 | $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c opcodes.c # <</mark>> # Rule to build the Win32 resources object file. # !IF $(USE_RC)!=0 # <<block1>> | | | | 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 | $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c opcodes.c # <</mark>> # Rule to build the Win32 resources object file. # !IF $(USE_RC)!=0 # <<block1>> $(LIBRESOBJS): $(TOP)\src\sqlite3.rc $(SQLITE3H) $(TOP)\VERSION $(JIM_TCLSH) echo #ifndef SQLITE_RESOURCE_VERSION > sqlite3rc.h for /F %%V in ('type "$(TOP)\VERSION"') do ( \ echo #define SQLITE_RESOURCE_VERSION %%V \ | $(JIM_TCLSH) $(TOP)\tool\replace.tcl exact . ^, >> sqlite3rc.h \ ) echo #endif >> sqlite3rc.h $(LTRCOMPILE) -fo $(LIBRESOBJS) $(TOP)\src\sqlite3.rc # <</block1>> !ENDIF # <<mark>> |
︙ | ︙ | |||
1931 1932 1933 1934 1935 1936 1937 | callback.lo: $(TOP)\src\callback.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\callback.c complete.lo: $(TOP)\src\complete.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\complete.c | > > > | | | 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 | callback.lo: $(TOP)\src\callback.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\callback.c complete.lo: $(TOP)\src\complete.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\complete.c ctime.c: $(TOP)\tool\mkctimec.tcl $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\tool\mkctimec.tcl ctime.lo: ctime.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c ctime.c date.lo: $(TOP)\src\date.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\date.c dbpage.lo: $(TOP)\src\dbpage.c $(HDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) -c $(TOP)\src\dbpage.c |
︙ | ︙ | |||
2149 2150 2151 2152 2153 2154 2155 | $(LTCOMPILE) $(NO_WARN) -DTCLSH -DBUILD_sqlite -I$(TCLINCDIR) -c $(TOP)\src\tclsqlite.c tclsqlite3.exe: tclsqlite-shell.lo $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS) $(LTLINK) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite-shell.lo $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) # Rules to build opcodes.c and opcodes.h # | | | | | > > > | | | 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 | $(LTCOMPILE) $(NO_WARN) -DTCLSH -DBUILD_sqlite -I$(TCLINCDIR) -c $(TOP)\src\tclsqlite.c tclsqlite3.exe: tclsqlite-shell.lo $(SQLITE3C) $(SQLITE3H) $(LIBRESOBJS) $(LTLINK) $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) /OUT:$@ tclsqlite-shell.lo $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) # Rules to build opcodes.c and opcodes.h # opcodes.c: opcodes.h $(TOP)\tool\mkopcodec.tcl $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\tool\mkopcodec.tcl opcodes.h > opcodes.c opcodes.h: parse.h $(TOP)\src\vdbe.c $(TOP)\tool\mkopcodeh.tcl $(JIM_TCLSH) type parse.h $(TOP)\src\vdbe.c | $(JIM_TCLSH) $(TOP)\tool\mkopcodeh.tcl > opcodes.h # Rules to build parse.c and parse.h - the outputs of lemon. # parse.h: parse.c parse.c: $(TOP)\src\parse.y lemon.exe del /Q parse.y parse.h parse.h.temp 2>NUL copy /Y $(TOP)\src\parse.y . copy /B parse.y +,, .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S parse.y pragma.h: $(TOP)\tool\mkpragmatab.tcl $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\tool\mkpragmatab.tcl $(SQLITE3H): $(TOP)\src\sqlite.h.in $(TOP)\manifest mksourceid.exe $(TOP)\VERSION $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\tool\mksqlite3h.tcl "$(TOP:\=/)" -o $(SQLITE3H) $(MKSQLITE3H_ARGS) sqlite3ext.h: .target_source !IF $(USE_STDCALL)!=0 || $(FOR_WIN10)!=0 type tsrc\sqlite3ext.h | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*\)" "(SQLITE_CALLBACK *)" \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl regsub "\(\*" "(SQLITE_APICALL *" > sqlite3ext.h copy /Y sqlite3ext.h tsrc\sqlite3ext.h !ELSE |
︙ | ︙ | |||
2189 2190 2191 2192 2193 2194 2195 | mkkeywordhash.exe: $(TOP)\tool\mkkeywordhash.c $(BCC) $(NO_WARN) -Fe$@ $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) \ $(TOP)\tool\mkkeywordhash.c /link $(LDFLAGS) $(NLTLINKOPTS) $(NLTLIBPATHS) keywordhash.h: $(TOP)\tool\mkkeywordhash.c mkkeywordhash.exe .\mkkeywordhash.exe > keywordhash.h | | | | > > > > | | | | | | | > > > | | > | > > > | | | | | | | < | | | | | 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 | mkkeywordhash.exe: $(TOP)\tool\mkkeywordhash.c $(BCC) $(NO_WARN) -Fe$@ $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) \ $(TOP)\tool\mkkeywordhash.c /link $(LDFLAGS) $(NLTLINKOPTS) $(NLTLIBPATHS) keywordhash.h: $(TOP)\tool\mkkeywordhash.c mkkeywordhash.exe .\mkkeywordhash.exe > keywordhash.h # Source and header files that shell.c depends on SHELL_DEP = \ $(TOP)\src\shell.c.in \ $(TOP)\ext\expert\sqlite3expert.c \ $(TOP)\ext\expert\sqlite3expert.h \ $(TOP)\ext\intck\sqlite3intck.c \ $(TOP)\ext\intck\sqlite3intck.h \ $(TOP)\ext\misc\appendvfs.c \ $(TOP)\ext\misc\base64.c \ $(TOP)\ext\misc\base85.c \ $(TOP)\ext\misc\completion.c \ $(TOP)\ext\misc\decimal.c \ $(TOP)\ext\misc\fileio.c \ $(TOP)\ext\misc\ieee754.c \ $(TOP)\ext\misc\memtrace.c \ $(TOP)\ext\misc\pcachetrace.c \ $(TOP)\ext\misc\percentile.c \ $(TOP)\ext\misc\regexp.c \ $(TOP)\ext\misc\series.c \ $(TOP)\ext\misc\sha1.c \ $(TOP)\ext\misc\shathree.c \ $(TOP)\ext\misc\sqlar.c \ $(TOP)\ext\misc\sqlite3_stdio.c \ $(TOP)\ext\misc\sqlite3_stdio.h \ $(TOP)\ext\misc\uint.c \ $(TOP)\ext\misc\vfstrace.c \ $(TOP)\ext\misc\windirent.h \ $(TOP)\ext\misc\zipfile.c \ $(TOP)\ext\recover\dbdata.c \ $(TOP)\ext\recover\sqlite3recover.c \ $(TOP)\ext\recover\sqlite3recover.h # If use of zlib is enabled, add the "zipfile.c" source file. # !IF $(USE_ZLIB)!=0 SHELL_DEP = $(SHELL_DEP) $(TOP)\ext\misc\sqlar.c SHELL_DEP = $(SHELL_DEP) $(TOP)\ext\misc\zipfile.c !ENDIF shell.c: $(SHELL_DEP) $(TOP)\tool\mkshellc.tcl $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\tool\mkshellc.tcl shell.c zlib: pushd $(ZLIBDIR) && $(MAKE) /f win32\Makefile.msc clean $(ZLIBLIB) && popd # Rules to build the extension objects. # icu.lo: $(TOP)\ext\icu\icu.c $(HDR) $(EXTHDR) |
︙ | ︙ | |||
2322 2323 2324 2325 2326 2327 2328 | copy /Y $(TOP)\ext\fts5\fts5parse.y . copy /B fts5parse.y +,, del /Q fts5parse.h 2>NUL .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S fts5parse.y fts5parse.h: fts5parse.c | | | | | | 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 | copy /Y $(TOP)\ext\fts5\fts5parse.y . copy /B fts5parse.y +,, del /Q fts5parse.h 2>NUL .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(EXT_FEATURE_FLAGS) $(OPTS) -S fts5parse.y fts5parse.h: fts5parse.c fts5.c: $(FTS5_SRC) $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\ext\fts5\tool\mkfts5c.tcl copy /Y $(TOP)\ext\fts5\fts5.h . copy /B fts5.h +,, lsm1.c: $(LSM1_SRC) $(JIM_TCLSH) $(JIM_TCLSH) $(TOP)\ext\lsm1\tool\mklsm1c.tcl copy /Y $(TOP)\ext\lsm1\lsm.h . copy /B lsm.h +,, fts5.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c fts5.c fts5_ext.lo: fts5.c $(HDR) $(EXTHDR) |
︙ | ︙ | |||
2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 | TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_DEFAULT_PAGE_SIZE=1024 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CKSUMVFS_STATIC=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) $(TEST_CCONV_OPTS) TESTFIXTURE_SRC0 = $(TESTEXT) $(TESTSRC2) TESTFIXTURE_SRC1 = $(TESTEXT) $(SQLITE3C) !IF $(USE_AMALGAMATION)==0 TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC0) !ELSE TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC1) | > > | 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 | TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_SERIES_CONSTRAINT_VERIFY=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_DEFAULT_PAGE_SIZE=1024 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_CKSUMVFS_STATIC=1 TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) $(TEST_CCONV_OPTS) TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_STATIC_RANDOMJSON TESTFIXTURE_FLAGS = $(TESTFIXTURE_FLAGS) -DSQLITE_STRICT_SUBTYPE=1 TESTFIXTURE_SRC0 = $(TESTEXT) $(TESTSRC2) TESTFIXTURE_SRC1 = $(TESTEXT) $(SQLITE3C) !IF $(USE_AMALGAMATION)==0 TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC0) !ELSE TESTFIXTURE_SRC = $(TESTSRC) $(TOP)\src\tclsqlite.c $(TESTFIXTURE_SRC1) |
︙ | ︙ | |||
2392 2393 2394 2395 2396 2397 2398 | | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "void (*freeProc)" "void (SQLITE_TCLAPI *freeProc)" \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "Tcl_HashEntry *(*findProc)" "Tcl_HashEntry *(SQLITE_TCLAPI *findProc)" \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "Tcl_HashEntry *(*createProc)" "Tcl_HashEntry *(SQLITE_TCLAPI *createProc)" >> $(SQLITETCLH) !ENDIF testfixture.exe: $(TESTFIXTURE_SRC) $(TESTFIXTURE_DEP) $(SQLITE3H) $(LIBRESOBJS) $(HDR) $(SQLITE_TCL_DEP) $(LTLINK) -DSQLITE_NO_SYNC=1 $(TESTFIXTURE_FLAGS) \ | | > > > > > > > > | | | 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 | | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "void (*freeProc)" "void (SQLITE_TCLAPI *freeProc)" \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "Tcl_HashEntry *(*findProc)" "Tcl_HashEntry *(SQLITE_TCLAPI *findProc)" \ | $(TCLSH_CMD) $(TOP)\tool\replace.tcl exact "Tcl_HashEntry *(*createProc)" "Tcl_HashEntry *(SQLITE_TCLAPI *createProc)" >> $(SQLITETCLH) !ENDIF testfixture.exe: $(TESTFIXTURE_SRC) $(TESTFIXTURE_DEP) $(SQLITE3H) $(LIBRESOBJS) $(HDR) $(SQLITE_TCL_DEP) $(LTLINK) -DSQLITE_NO_SYNC=1 $(TESTFIXTURE_FLAGS) \ -DBUILD_sqlite -I$(TCLINCDIR) -I$(TOP)\ext\misc \ $(TESTFIXTURE_SRC) \ /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) # A small helper for manually running individual tests tf.bat: testfixture.exe Makefile.msc echo @set PATH=$(LIBTCLPATH);%PATH% > $@ echo .\testfixture.exe %* >> $@ extensiontest: testfixture.exe testloadext.dll @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\loadext.test $(TESTOPTS) tool-zip: testfixture.exe sqlite3.exe sqldiff.exe sqlite3_analyzer.exe sqlite3_rsync.exe $(TOP)\tool\mktoolzip.tcl .\testfixture.exe $(TOP)\tool\mktoolzip.tcl coretestprogs: testfixture.exe sqlite3.exe testprogs: $(TESTPROGS) srcck1.exe fuzzcheck.exe sessionfuzz.exe fulltest: alltest fuzztest alltest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\all.test $(TESTOPTS) |
︙ | ︙ | |||
2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 | queryplantest: testfixture.exe shell @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\permutations.test queryplanner $(TESTOPTS) fuzztest: fuzzcheck.exe .\fuzzcheck.exe $(FUZZDATA) # Minimal testing that runs in less than 3 minutes (on a fast machine) # quicktest: testfixture.exe sourcetest @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\extraquick.test $(TESTOPTS) # This is the common case. Run many tests that do not take too long, # including fuzzcheck, sqlite3_analyzer, and sqldiff tests. # | > > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | | | | | | 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 | queryplantest: testfixture.exe shell @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\permutations.test queryplanner $(TESTOPTS) fuzztest: fuzzcheck.exe .\fuzzcheck.exe $(FUZZDATA) # Legacy testing target for third-party integrators. The SQLite # developers seldom use this target themselves. Instead # they use "nmake /f Makefile.msc devtest" which runs tests on # a standard set of options # test: $(TESTPROGS) sourcetest fuzztest tcltest # Minimal testing that runs in less than 3 minutes (on a fast machine) # quicktest: testfixture.exe sourcetest @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\extraquick.test $(TESTOPTS) # This is the common case. Run many tests that do not take too long, # including fuzzcheck, sqlite3_analyzer, and sqldiff tests. # # The veryquick.test TCL tests. # tcltest: testfixture.exe @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\veryquick.test $(TESTOPTS) # Runs all the same tests cases as the "tcltest" target but uses # the testrunner.tcl script to run them in multiple cores # concurrently. testrunner: testfixture.exe .\testfixture.exe $(TOP)\test\testrunner.tcl # This is the testing target preferred by the core SQLite developers. # It runs tests under a standard configuration. The devs run # "nmake /f Makefile.msc devtest" prior to each check-in, at a minimum. # Probably other tests too, but at least this one. # devtest: srctree-check sourcetest $(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest mdevtest: $(TCLSH_CMD) $(TOP)\test\testrunner.tcl mdevtest # Validate that various generated files in the source tree # are up-to-date. # srctree-check: $(TOP)\tool\srctree-check.tcl $(TCLSH_CMD) $(TOP)\tool\srctree-check.tcl # Testing for a release # releasetest: verify-source $(TCLSH_CMD) $(TOP)\test\testrunner.tcl release # xdevtest is like releasetest, except that it skips the # dependency on verify-source so that xdevtest can be run from # a modified source tree. # xdevtest: $(TCLSH_CMD) $(TOP)\test\testrunner.tcl release smoketest: $(TESTPROGS) @set PATH=$(LIBTCLPATH);$(PATH) .\testfixture.exe $(TOP)\test\main.test $(TESTOPTS) shelltest: $(TESTPROGS) .\testfixture.exe $(TOP)\test\permutations.test shell sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(TOP)\ext\misc\sqlite3_stdio.h $(TOP)\ext\misc\sqlite3_stdio.c $(SQLITE_TCL_DEP) $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl -DINCLUDE_SQLITE3_C $(TOP)\tool\sqlite3_analyzer.c.in > $@ sqlite3_analyzer.exe: sqlite3_analyzer.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_analyzer.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) sqltclsh.c: sqlite3.c $(TOP)\src\tclsqlite.c $(TOP)\tool\sqltclsh.tcl $(TOP)\ext\misc\appendvfs.c $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqltclsh.c.in $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqltclsh.c.in >sqltclsh.c sqltclsh.exe: sqltclsh.c $(SHELL_CORE_DEP) $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqltclsh.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) sqlite3_expert.exe: $(SQLITE3C) $(TOP)\ext\expert\sqlite3expert.h $(TOP)\ext\expert\sqlite3expert.c $(TOP)\ext\expert\expert.c $(LTLINK) $(NO_WARN) $(TOP)\ext\expert\sqlite3expert.c $(TOP)\ext\expert\expert.c $(SQLITE3C) $(TLIBS) CHECKER_DEPS =\ $(TOP)\tool\mkccode.tcl \ sqlite3.c \ $(TOP)\src\tclsqlite.c \ $(TOP)\ext\repair\sqlite3_checker.tcl \ $(TOP)\ext\repair\checkindex.c \ $(TOP)\ext\repair\checkfreelist.c \ $(TOP)\ext\misc\btreeinfo.c \ $(TOP)\ext\repair\sqlite3_checker.c.in sqlite3_checker.c: $(CHECKER_DEPS) $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\ext\repair\sqlite3_checker.c.in > $@ sqlite3_checker.exe: sqlite3_checker.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_checker.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(TCLLIBPATHS) $(LTLIBPATHS) $(LIBRESOBJS) $(TCLLIBS) $(LTLIBS) $(TLIBS) |
︙ | ︙ | |||
2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 | $(TOP)\test\tt3_vacuum.c \ $(TOP)\test\tt3_stress.c \ $(TOP)\test\tt3_lookaside1.c threadtest3.exe: $(THREADTEST3_SRC) $(TOP)\src\test_multiplex.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(TOP)\test\threadtest3.c $(TOP)\src\test_multiplex.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) LSMDIR=$(TOP)\ext\lsm1 !INCLUDE $(LSMDIR)\Makefile.msc moreclean: clean del /Q $(SQLITE3C) $(SQLITE3H) 2>NUL # <</mark>> clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL | > > > > > > > > > > > > | > | > > | > > | 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 | $(TOP)\test\tt3_vacuum.c \ $(TOP)\test\tt3_stress.c \ $(TOP)\test\tt3_lookaside1.c threadtest3.exe: $(THREADTEST3_SRC) $(TOP)\src\test_multiplex.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) $(TOP)\test\threadtest3.c $(TOP)\src\test_multiplex.c $(SQLITE3C) /link $(LDFLAGS) $(LTLINKOPTS) # Display key variables that control which version of TCL is to be used. # tcl-env: @echo TCLDIR = $(TCLDIR) @echo TCLVERSION = $(TCLVERSION) @echo TCLSUFFIX = $(TCLSUFFIX) @echo LIBTCL = $(LIBTCL) @echo LIBTCLSTUB = $(LIBTCLSTUB) @echo TCLSH_CMD = $(TCLSH_CMD) @echo JIM_TCLSH = $(JIM_TCLSH) @echo VISUALSTUDIOVERSION = $(VISUALSTUDIOVERSION) LSMDIR=$(TOP)\ext\lsm1 !INCLUDE $(LSMDIR)\Makefile.msc moreclean: clean del /Q $(SQLITE3C) $(SQLITE3H) 2>NUL # <</mark>> clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL del /Q *.bsc *.cod *.da *.bb *.bbg *.vc gmon.out 2>NUL del /Q sqlite3.def tclsqlite3.def ctime.c pragma.h 2>NUL del /Q $(SQLITE3EXE) $(SQLITE3DLL) Replace.exe 2>NUL # <<mark>> del /Q $(SQLITE3TCLDLL) pkgIndex.tcl 2>NUL del /Q opcodes.c opcodes.h 2>NUL del /Q lemon.* lempar.c parse.* 2>NUL del /Q mksourceid.* mkkeywordhash.* keywordhash.h 2>NUL del /Q notasharedlib.* 2>NUL -rmdir /Q/S .deps 2>NUL -rmdir /Q/S .libs 2>NUL -rmdir /Q/S tsrc 2>NUL del /Q .target_source 2>NUL del /Q tclsqlite3.exe $(SQLITETCLH) $(SQLITETCLDECLSH) 2>NUL del /Q lsm.dll lsmtest.exe 2>NUL del /Q atrc.exe changesetfuzz.exe dbtotxt.exe index_usage.exe 2>NUL del /Q testloadext.dll 2>NUL del /Q testfixture.exe test.db tf.bat 2>NUL del /Q /S testdir 2>/NUL -rmdir /Q /S testdir 2>NUL del /Q LogEst.exe fts3view.exe rollback-test.exe showdb.exe dbdump.exe 2>NUL del /Q changeset.exe 2>NUL del /Q showjournal.exe showstat4.exe showwal.exe speedtest1.exe 2>NUL del /Q mptester.exe wordcount.exe rbu.exe srcck1.exe 2>NUL del /Q sqlite3.c sqlite3-*.c sqlite3.h 2>NUL del /Q sqlite3rc.h 2>NUL del /Q shell.c sqlite3ext.h sqlite3session.h 2>NUL del /Q sqlite3_analyzer.exe sqlite3_analyzer.c sqlite3_rsync.exe 2>NUL del /Q sqlite-*-output.vsix 2>NUL del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe dbhash.exe 2>NUL del /Q sqltclsh.* 2>NUL del /Q dbfuzz.exe sessionfuzz.exe threadtest3.exe 2>NUL del /Q kvtest.exe ossshell.exe scrub.exe 2>NUL del /Q showshm.exe sqlite3_checker.* sqlite3_expert.exe 2>NUL del /Q fts5.* fts5parse.* 2>NUL del /Q lsm.h lsm1.c 2>NUL del /q src-verify.exe 2>NUL del /q jimsh.exe jimsh0.exe 2>NUL # <</mark>> |
Changes to README.md.
1 2 3 | <h1 align="center">SQLite Source Repository</h1> This repository contains the complete source code for the | | | > > > > > | | > > > > > > > > > > > > > > > > > > > > > | | | | | | | | | | | | | < | > | | | | > > > | | | | < > | > > > | > > | > > > > > > > > > > > > > > > > > > > > > | | > > > > | > | | > | > > > > > > > | < | | | > > | | > | | > | < | | < < < > | | | < | < < < | < | < | < | > > > > > | | > > | | | > > > > > > | > > | < > > > > > | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | <h1 align="center">SQLite Source Repository</h1> This repository contains the complete source code for the [SQLite database engine](https://sqlite.org/), including many test scripts. However, other test scripts and most of the documentation are managed separately. See the [on-line documentation](https://sqlite.org/) for more information about what SQLite is and how it works from a user's perspective. This README file is about the source code that goes into building SQLite, not about how SQLite is used. ## Version Control SQLite sources are managed using [Fossil](https://fossil-scm.org/), a distributed version control system that was specifically designed and written to support SQLite development. The [Fossil repository](https://sqlite.org/src/timeline) contains the urtext. If you are reading this on GitHub or some other Git repository or service, then you are looking at a mirror. The names of check-ins and other artifacts in a Git mirror are different from the official names for those objects. The official names for check-ins are found in a footer on the check-in comment for authorized mirrors. The official check-in name can also be seen in the `manifest.uuid` file in the root of the tree. Always use the official name, not the Git-name, when communicating about an SQLite check-in. If you pulled your SQLite source code from a secondary source and want to verify its integrity, there are hints on how to do that in the [Verifying Code Authenticity](#vauth) section below. ## Contacting The SQLite Developers The preferred way to ask questions or make comments about SQLite or to report bugs against SQLite is to visit the [SQLite Forum](https://sqlite.org/forum) at <https://sqlite.org/forum/>. Anonymous postings are permitted. If you think you have found a bug that has security implications and you do not want to report it on the public forum, you can send a private email to drh at sqlite dot org. ## Public Domain The SQLite source code is in the public domain. See <https://sqlite.org/copyright.html> for details. Because SQLite is in the public domain, we do not normally accept pull requests, because if we did take a pull request, the changes in that pull request might carry a copyright and the SQLite source code would then no longer be fully in the public domain. ## Obtaining The SQLite Source Code If you do not want to use Fossil, you can download tarballs or ZIP archives or [SQLite archives](https://sqlite.org/cli.html#sqlar) as follows: * Latest trunk check-in as [Tarball](https://sqlite.org/src/tarball/sqlite.tar.gz), [ZIP-archive](https://sqlite.org/src/zip/sqlite.zip), or [SQLite-archive](https://sqlite.org/src/sqlar/sqlite.sqlar). * Latest release as [Tarball](https://sqlite.org/src/tarball/sqlite.tar.gz?r=release), [ZIP-archive](https://sqlite.org/src/zip/sqlite.zip?r=release), or [SQLite-archive](https://sqlite.org/src/sqlar/sqlite.sqlar?r=release). * For other check-ins, substitute an appropriate branch name or tag or hash prefix in place of "release" in the URLs of the previous bullet. Or browse the [timeline](https://sqlite.org/src/timeline) to locate the check-in desired, click on its information page link, then click on the "Tarball" or "ZIP Archive" links on the information page. To access sources directly using [Fossil](https://fossil-scm.org/home), first install Fossil version 2.0 or later. Source tarballs and precompiled binaries available at <https://fossil-scm.org/home/uv/download.html>. Fossil is a stand-alone program. To install, simply download or build the single executable file and put that file someplace on your $PATH. Then run commands like this: mkdir -p ~/sqlite cd ~/sqlite fossil open https://sqlite.org/src The "fossil open" command will take two or three minutes. Afterwards, you can do fast, bandwidth-efficient updates to the whatever versions of SQLite you like. Some examples: fossil update trunk ;# latest trunk check-in fossil update release ;# latest official release fossil update trunk:2024-01-01 ;# First trunk check-in after 2024-01-01 fossil update version-3.39.0 ;# Version 3.39.0 Or type "fossil ui" to get a web-based user interface. ## Compiling for Unix-like systems First create a directory in which to place the build products. It is recommended, but not required, that the build directory be separate from the source directory. Cd into the build directory and then from the build directory run the configure script found at the root of the source tree. Then run "make". For example: apt install gcc make tcl-dev ;# Make sure you have all the necessary build tools tar xzf sqlite.tar.gz ;# Unpack the source tree into "sqlite" mkdir bld ;# Build will occur in a sibling directory cd bld ;# Change to the build directory ../sqlite/configure ;# Run the configure script make sqlite3 ;# Builds the "sqlite3" command-line tool make sqlite3.c ;# Build the "amalgamation" source file make sqldiff ;# Builds the "sqldiff" command-line tool # Makefile targets below this point require tcl-dev make tclextension-install ;# Build and install the SQLite TCL extension make devtest ;# Run development tests make releasetest ;# Run full release tests make sqlite3_analyzer ;# Builds the "sqlite3_analyzer" tool See the makefile for additional targets. For debugging builds, the core developers typically run "configure" with options like this: ../sqlite/configure --enable-all --enable-debug CFLAGS='-O0 -g' For release builds, the core developers usually do: ../sqlite/configure --enable-all Almost all makefile targets require a "tclsh" TCL interpreter version 8.6 or later. The "tclextension-install" target and the test targets that follow all require TCL development libraries too. ("apt install tcl-dev"). It is helpful, but is not required, to install the SQLite TCL extension (the "tclextension-install" target) prior to running tests. The "releasetest" target has additional requirements, such as "valgrind". On "make" command-lines, one can add "OPTIONS=..." to specify additional compile-time options over and above those set by ./configure. For example, to compile with the SQLITE_OMIT_DEPRECATED compile-time option, one could say: ./configure --enable-all make OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3 The configure script uses autoconf 2.61 and libtool. If the configure script does not work out for you, there is a generic makefile named "Makefile.linux-gcc" in the top directory of the source tree that you can copy and edit to suit your needs. Comments on the generic makefile show what changes are needed. ## Compiling for Windows Using MSVC On Windows, everything can be compiled with MSVC. You will also need a working installation of TCL. See the [compile-for-windows.md](doc/compile-for-windows.md) document for additional information about how to install MSVC and TCL and configure your build environment. If you want to run tests, you need to let SQLite know the location of your TCL library, using a command like this: set TCLDIR=c:\Tcl SQLite uses "tclsh.exe" as part of the build process, and so that program will need to be somewhere on your %PATH%. SQLite itself does not contain any TCL code, but it does use TCL to help with the build process and to run tests. You may need to install TCL development libraries in order to successfully complete some makefile targets. It is helpful, but is not required, to install the SQLite TCL extension (the "tclextension-install" target) prior to running tests. Build using Makefile.msc. Example: nmake /f Makefile.msc sqlite3.exe nmake /f Makefile.msc sqlite3.c nmake /f Makefile.msc sqldiff.exe # Makefile targets below this point require TCL development libraries nmake /f Makefile.msc tclextension-install nmake /f Makefile.msc devtest nmake /f Makefile.msc releasetest nmake /f Makefile.msc sqlite3_analyzer.exe There are many other makefile targets. See comments in Makefile.msc for details. As with the unix Makefile, the OPTIONS=... argument can be passed on the nmake command-line to enable new compile-time options. For example: nmake /f Makefile.msc OPTIONS=-DSQLITE_OMIT_DEPRECATED sqlite3.exe ## Source Tree Map * **src/** - This directory contains the primary source code for the SQLite core. For historical reasons, C-code used for testing is also found here. Source files intended for testing begin with "`test`". The `tclsqlite3.c` and `tclsqlite3.h` files are the TCL interface for SQLite and are also not part of the core. * **test/** - This directory and its subdirectories contains code used for testing. Files that end in "`.test`" are TCL scripts that run tests using an augmented TCL interpreter named "testfixture". Use a command like "`make testfixture`" (unix) or "`nmake /f Makefile.msc testfixture.exe`" (windows) to build that augmented TCL interpreter, then run individual tests using commands like "`testfixture test/main.test`". This test/ subdirectory also contains additional C code modules and scripts for other kinds of testing. * **tool/** - This directory contains programs and scripts used to build some of the machine-generated code that goes into the SQLite core, as well as to build and run tests and perform diagnostics. The source code to [the Lemon parser generator](./doc/lemon.html) is found here. There are also TCL scripts used to build and/or transform source code files. For example, the tool/mksqlite3h.tcl script reads the src/sqlite.h.in file and uses it as a template to construct the deliverable "sqlite3.h" file that defines the SQLite interface. * **ext/** - Various extensions to SQLite are found under this directory. For example, the FTS5 subsystem is in "ext/fts5/". Some of these extensions (ex: FTS3/4, FTS5, RTREE) might get built into the SQLite amalgamation, but not all of them. The "ext/misc/" subdirectory contains an assortment of one-file extensions, many of which are omitted from the SQLite core, but which are included in the [SQLite CLI](https://sqlite.org/cli.html). * **doc/** - Some documentation files about SQLite internals are found here. Note, however, that the primary documentation designed for application developers and users of SQLite is in a completely separate repository. Note also that the primary API documentation is derived from specially constructed comments in the src/sqlite.h.in file. ### Generated Source Code Files Several of the C-language source files used by SQLite are generated from other sources rather than being typed in manually by a programmer. This section will summarize those automatically-generated files. To create all of the automatically-generated files, simply run "make target_source". The "target_source" make target will create a subdirectory "tsrc/" and fill it with all the source files needed to build SQLite, both manually-edited files and automatically-generated files. The SQLite interface is defined by the **sqlite3.h** header file, which is generated from src/sqlite.h.in, ./manifest.uuid, and ./VERSION. The [Tcl script](https://www.tcl.tk) at tool/mksqlite3h.tcl does the conversion. The manifest.uuid file contains the SHA3 hash of the particular check-in and is used to generate the SQLITE\_SOURCE\_ID macro. The VERSION file contains the current SQLite version number. The sqlite3.h header is really just a copy of src/sqlite.h.in with the source-id and version number inserted at just the right spots. Note that comment text in the sqlite3.h file is used to generate much of the SQLite API documentation. The Tcl scripts used to generate that documentation are in a separate source repository. |
︙ | ︙ | |||
224 225 226 227 228 229 230 | all of the source code is contained within a single translation unit so that the compiler can do extra cross-procedure optimization, but no individual source file exceeds 32K lines in length. ## How It All Fits Together SQLite is modular in design. | | | | | | | | | | > > | > > | > > | 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | all of the source code is contained within a single translation unit so that the compiler can do extra cross-procedure optimization, but no individual source file exceeds 32K lines in length. ## How It All Fits Together SQLite is modular in design. See the [architectural description](https://sqlite.org/arch.html) for details. Other documents that are useful in helping to understand how SQLite works include the [file format](https://sqlite.org/fileformat2.html) description, the [virtual machine](https://sqlite.org/opcode.html) that runs prepared statements, the description of [how transactions work](https://sqlite.org/atomiccommit.html), and the [overview of the query planner](https://sqlite.org/optoverview.html). Decades of effort have gone into optimizing SQLite, both for small size and high performance. And optimizations tend to result in complex code. So there is a lot of complexity in the current SQLite implementation. It will not be the easiest library in the world to hack. ### Key source code files * **sqlite.h.in** - This file defines the public interface to the SQLite library. Readers will need to be familiar with this interface before trying to understand how the library works internally. This file is really a template that is transformed into the "sqlite3.h" deliverable using a script invoked by the makefile. * **sqliteInt.h** - this header file defines many of the data objects used internally by SQLite. In addition to "sqliteInt.h", some subsystems inside of sQLite have their own header files. These internal interfaces are not for use by applications. They can and do change from one release of SQLite to the next. * **parse.y** - This file describes the LALR(1) grammar that SQLite uses to parse SQL statements, and the actions that are taken at each step in the parsing process. The file is processed by the [Lemon Parser Generator](./doc/lemon.html) to produce the actual C code used for parsing. * **vdbe.c** - This file implements the virtual machine that runs prepared statements. There are various helper files whose names begin with "vdbe". The VDBE has access to the vdbeInt.h header file which defines internal data objects. The rest of SQLite interacts with the VDBE through an interface defined by vdbe.h. |
︙ | ︙ | |||
286 287 288 289 290 291 292 | "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed into "shell.c" as part of the build process. * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It is not part of the core SQLite library. But as most of the tests in this repository are written in Tcl, the Tcl language bindings are important. | | | | > > | | > > > > | > | > | > > > > > | | | > > > > | 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 | "sqlite3.exe" command-line shell. The "shell.c.in" file is transformed into "shell.c" as part of the build process. * **tclsqlite.c** - This file implements the Tcl bindings for SQLite. It is not part of the core SQLite library. But as most of the tests in this repository are written in Tcl, the Tcl language bindings are important. * **test\*.c** - Files in the src/ folder that begin with "test" go into building the "testfixture.exe" program. The testfixture.exe program is an enhanced Tcl shell. The testfixture.exe program runs scripts in the test/ folder to validate the core SQLite code. The testfixture program (and some other test programs too) is built and run when you type "make test". * **VERSION**, **manifest**, and **manifest.uuid** - These files define the current SQLite version number. The "VERSION" file is human generated, but the "manifest" and "manifest.uuid" files are automatically generated by the [Fossil version control system](https://fossil-scm.org/). There are many other source files. Each has a succinct header comment that describes its purpose and role within the larger system. <a name="vauth"></a> ## Verifying Code Authenticity The `manifest` file at the root directory of the source tree contains either a SHA3-256 hash or a SHA1 hash for every source file in the repository. The name of the version of the entire source tree is just the SHA3-256 hash of the `manifest` file itself, possibly with the last line of that file omitted if the last line begins with "`# Remove this line`". The `manifest.uuid` file should contain the SHA3-256 hash of the `manifest` file. If all of the above hash comparisons are correct, then you can be confident that your source tree is authentic and unadulterated. Details on the format for the `manifest` files are available [on the Fossil website](https://fossil-scm.org/home/doc/trunk/www/fileformat.wiki#manifest). The process of checking source code authenticity is automated by the makefile: > make verify-source Or on windows: > nmake /f Makefile.msc verify-source Using the makefile to verify source integrity is good for detecting accidental changes to the source tree, but malicious changes could be hidden by also modifying the makefiles. ## Contacts The main SQLite website is [https://sqlite.org/](https://sqlite.org/) with geographically distributed backups at [https://www2.sqlite.org/](https://www2.sqlite.org) and [https://www3.sqlite.org/](https://www3.sqlite.org). Contact the SQLite developers through the [SQLite Forum](https://sqlite.org/forum/). In an emergency, you can send private email to the lead developer at drh at sqlite dot org. |
Changes to VERSION.
|
| | | 1 | 3.51.0 |
Deleted aclocal.m4.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Added art/icon-243x273.gif.
cannot compute difference between binary files
Added art/icon-80x90.gif.
cannot compute difference between binary files
Added art/sqlite370.svg.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 | <?xml version="1.0" encoding="UTF-8" standalone="no"?> <!-- Created with Inkscape (http://www.inkscape.org/) --> <svg width="103.73447mm" height="46.723713mm" viewBox="0 0 103.73447 46.723713" version="1.1" id="svg470" xmlns="http://www.w3.org/2000/svg" xmlns:svg="http://www.w3.org/2000/svg"> <defs id="defs467"> <clipPath clipPathUnits="userSpaceOnUse" id="clipPath41"> <path d="m 994.055,997.609 c 6.725,-11.629 35.115,-61.371 40.815,-77.398 6.42,-18.121 7.77,-23.313 7.77,-23.313 0,0 -15.57,80.114 -41.12,126.862 5.6,18.9 12.29,39.79 19.86,62.12 8.85,-15.53 28.96,-51.2 36.23,-68.1 0.27,3.19 0.54,6.38 0.82,9.54 -6.44,24.75 -16.22,57.15 -28.74,82.54 32.14,167.25 135.59,386.25 247.04,504.48 H 509.602 c -37.883,0 -68.711,-30.82 -68.711,-68.71 V 731.789 c 173.738,66.68 383.23,127.633 563.529,125.02 -6.693,25.812 -14.315,49.152 -22.318,62.679 -4.122,6.981 0.449,35.653 11.953,78.121 z" id="path39" /> </clipPath> <linearGradient x1="0" y1="0" x2="1" y2="0" gradientUnits="userSpaceOnUse" gradientTransform="matrix(-4.02e-5,-918.907,-918.907,4.02e-5,858.809,1614.34)" spreadMethod="pad" id="linearGradient51"> <stop style="stop-opacity:1;stop-color:#97d9f6" offset="0" id="stop43" /> <stop style="stop-opacity:1;stop-color:#0f80cc" offset="0.66199914" id="stop45" /> <stop style="stop-opacity:1;stop-color:#0f80cc" offset="0.920245" id="stop47" /> <stop style="stop-opacity:1;stop-color:#0f80cc" offset="1" id="stop49" /> </linearGradient> </defs> <g id="g337"> <rect style="fill:#ffffff;stroke:#ffffff;stroke-width:0.239562;stroke-linecap:round;stroke-linejoin:round" id="rect366" width="103.4949" height="46.48415" x="0.119781" y="0.119781" ry="0" /> <g id="g465" transform="translate(-56.57816,-61.353828)"> <path d="m 79.198545,96.729705 c -0.02011,-0.25467 -0.0321,-0.420158 -0.0321,-0.420158 0,0 -0.77223,-5.206224 -1.691499,-6.759822 -0.145415,-0.246275 0.01584,-1.257759 0.421676,-2.755936 0.237243,0.410245 1.238779,2.165033 1.439862,2.73043 0.226484,0.639268 0.274109,0.822431 0.274109,0.822431 0,0 -0.549275,-2.826244 -1.450623,-4.47541 0.197556,-0.66675 0.433564,-1.403703 0.700617,-2.191456 0.341489,0.599017 1.158875,2.049286 1.338792,2.552348 0.03634,0.103364 0.06703,0.191205 0.09454,0.271639 0.0088,-0.04833 0.01764,-0.09666 0.02646,-0.144992 -0.206375,-0.875595 -0.611716,-2.399242 -1.166636,-3.524956 1.231195,-6.408209 5.430309,-14.973654 9.735256,-18.795296 H 60.806192 c -1.88214,0 -3.422086,1.540228 -3.422086,3.422298 v 30.970081 c 0,1.881964 1.539946,3.421944 3.422086,3.421944 h 18.49607 c -0.13335,-1.61438 -0.1778,-3.40046 -0.103717,-5.123145" style="fill:#0f80cc;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path21" /> <path d="m 99.934471,84.584659 h 0.474839 v 3.954957 h -0.503061 c -0.115005,-1.093188 -0.510469,-1.946593 -1.186392,-2.560426 -0.676627,-0.612775 -1.526822,-0.920045 -2.5527,-0.920045 -0.911578,0 -1.64465,0.230364 -2.200628,0.690387 -0.555977,0.459669 -0.833966,1.073467 -0.833966,1.840759 0,0.709401 0.230011,1.356536 0.690386,1.940948 0.459669,0.585258 1.457325,1.337134 2.991556,2.258907 1.782939,1.063978 2.981325,1.992912 3.594805,2.789414 0.61348,0.795549 0.92004,1.706421 0.92004,2.732229 0,1.322916 -0.46813,2.454275 -1.405815,3.393691 -0.939094,0.93983 -2.069042,1.40945 -3.389489,1.40945 -0.766233,0 -1.742369,-0.21096 -2.929467,-0.63334 -0.487891,-0.17237 -0.799394,-0.25781 -0.933097,-0.25781 -0.277989,0 -0.492831,0.21523 -0.646289,0.64699 h -0.503414 v -4.558981 h 0.503414 c 0.0092,1.256206 0.418747,2.262752 1.227314,3.020521 0.808567,0.75667 1.753306,1.13524 2.834217,1.13524 1.024819,0 1.844322,-0.27231 2.462389,-0.81979 0.617008,-0.54723 0.925689,-1.276066 0.925689,-2.188068 0,-0.681143 -0.254,-1.333394 -0.760942,-1.956823 -0.506589,-0.623958 -1.567392,-1.427621 -3.180292,-2.412259 -1.31445,-0.812765 -2.192161,-1.452422 -2.63278,-1.923027 -0.44062,-0.470182 -0.766234,-0.985591 -0.976842,-1.547284 -0.210608,-0.560564 -0.316089,-1.177537 -0.316089,-1.848485 0,-1.294659 0.424744,-2.345302 1.273175,-3.151399 0.848431,-0.805392 1.953331,-1.208264 3.314347,-1.208264 0.747889,0 1.563159,0.148519 2.445103,0.445558 0.411692,0.134409 0.699559,0.201436 0.862189,0.201436 0.135114,0 0.225778,-0.02858 0.27305,-0.08608 0.04868,-0.05786 0.1016,-0.186973 0.15875,-0.388409" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path23" /> <path d="m 119.14851,93.38576 c 0,-2.579547 -0.59019,-4.611476 -1.76953,-6.097835 -1.17969,-1.48583 -2.70369,-2.22878 -4.57412,-2.22878 -1.98437,0 -3.56799,0.746125 -4.75191,2.235977 -1.18463,1.491438 -1.77553,3.482834 -1.77553,5.975562 0,2.42577 0.58702,4.398857 1.76106,5.918271 1.17511,1.519415 2.70616,2.278595 4.59458,2.278595 1.96568,0 3.5426,-0.73562 4.7311,-2.207334 1.18851,-1.472177 1.78435,-3.430094 1.78435,-5.874456 z m 2.93265,12.9716 0.18662,0.46041 c -0.82409,0.37402 -1.7586,0.56085 -2.80353,0.56085 -0.86289,0 -1.66405,-0.11314 -2.40206,-0.33789 -0.73731,-0.22571 -1.43122,-0.57051 -2.07751,-1.03572 -0.647,-0.46496 -1.43616,-1.23941 -2.36608,-2.32198 -0.82515,-0.95928 -1.45768,-1.55445 -1.8983,-1.78404 -1.92723,-0.39232 -3.58246,-1.40973 -4.96852,-3.048067 -1.38536,-1.639323 -2.07822,-3.509716 -2.07822,-5.609167 0,-1.5627 0.39794,-3.033889 1.19345,-4.414556 0.79587,-1.380772 1.87466,-2.461718 3.23709,-3.242416 1.36031,-0.782108 2.87055,-1.172633 4.52896,-1.172633 2.56011,0 4.71699,0.851253 6.47206,2.55263 1.75366,1.7012 2.63067,3.79416 2.63067,6.276975 0,2.09102 -0.67522,3.951923 -2.02706,5.586695 -1.3522,1.635479 -3.08293,2.658779 -5.19148,3.070539 0.20179,0.16151 0.56233,0.5697 1.0795,1.22234 1.10031,1.36204 2.02071,2.28604 2.75837,2.76998 0.73871,0.48274 1.47214,0.72542 2.20133,0.72542 0.53728,0 1.04634,-0.0868 1.52471,-0.25937" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path25" /> <path d="m 134.47177,99.116318 -0.28116,2.694482 h -11.00385 l 0.0233,-0.53619 c 0.59654,-0.061 1.01458,-0.21413 1.25835,-0.45861 0.24342,-0.24624 0.36654,-0.81111 0.37571,-1.699682 l 0.0127,-11.904593 c -0.004,-0.621877 -0.0346,-1.050502 -0.0861,-1.284393 -0.13828,-0.653698 -0.75565,-0.980017 -1.85984,-0.980017 l -0.0236,-0.535164 h 5.74393 l 0.0219,0.535164 c -0.61242,0.04163 -1.03152,0.206728 -1.26612,0.496711 -0.23212,0.291042 -0.34713,0.880533 -0.34678,1.767699 l -0.0441,12.029053 c 0.002,0.204646 0.0191,0.391901 0.0572,0.565538 0.1845,0.877674 0.88053,1.315474 2.08774,1.315474 h 1.03681 c 1.34161,0 2.27259,-0.15088 2.79788,-0.45226 0.52388,-0.29919 0.8569,-0.818023 1.00048,-1.553212 h 0.49565" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path27" /> <path d="m 138.12055,87.618936 c 0.25965,0 0.49883,0.09095 0.71967,0.273685 0.21837,0.182034 0.36054,0.407071 0.41522,0.674829 0.0533,0.25975 0.009,0.48507 -0.133,0.676769 -0.14217,0.191523 -0.34114,0.28702 -0.60078,0.28702 -0.26846,0 -0.51858,-0.0955 -0.75001,-0.28702 -0.23283,-0.191699 -0.37535,-0.417019 -0.42827,-0.676769 -0.0547,-0.267758 -0.0106,-0.492795 0.14605,-0.674829 0.15311,-0.182738 0.36266,-0.273685 0.63112,-0.273685 z m 1.75754,4.127783 -0.0176,7.535404 c 0.003,0.526662 0.03,0.900957 0.0769,1.122117 0.0607,0.30688 0.19685,0.5147 0.40111,0.6248 0.20778,0.11081 0.67133,0.1662 1.39065,0.1662 l 0.0166,0.50299 h -5.17702 l 0.0123,-0.50299 c 0.70979,0 1.14582,-0.0656 1.31198,-0.1947 0.16475,-0.12887 0.24941,-0.56307 0.25859,-1.301433 l 0.0325,-5.335094 c -0.006,-0.291042 -0.0243,-0.518548 -0.0586,-0.683613 -0.1016,-0.493219 -0.42686,-0.740587 -0.97367,-0.740587 l -1.12324,0.143616 -0.005,-0.589527 3.26319,-0.747183 h 0.59161" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path29" /> <path d="m 148.56207,92.004952 0.0247,0.805074 h -2.70545 l -0.0194,6.728955 c 0,0.183409 0.0109,0.341065 0.0423,0.476109 0.11465,0.55605 0.41945,0.83467 0.90946,0.83467 0.79728,0 1.70709,-0.52239 2.73967,-1.567637 l 0.43427,0.366148 c -1.43016,1.587219 -2.79682,2.380119 -4.10033,2.380119 -0.901,0 -1.43828,-0.41229 -1.6129,-1.23677 -0.072,-0.34438 -0.10795,-1.025556 -0.11077,-2.041838 l 0.037,-5.939756 h -1.76954 l -0.0205,-0.502708 c 1.51836,-0.153105 2.50472,-1.106981 2.96087,-2.861769 h 0.51752 l -0.005,2.559403 h 2.67758" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path31" /> <path d="m 152.6931,96.836244 c 2.82998,-0.546417 4.09434,-1.527845 3.79906,-2.9464 -0.0603,-0.288819 -0.24483,-0.55245 -0.54716,-0.791527 -0.30656,-0.239925 -0.61418,-0.359975 -0.92921,-0.359975 -0.71932,0 -1.3081,0.424462 -1.76178,1.272505 -0.4572,0.848854 -0.6417,1.791017 -0.56091,2.825397 z m -0.0162,0.533436 c 0.024,0.248708 0.0575,0.48394 0.10442,0.704179 0.36089,1.725437 1.3328,2.588371 2.91395,2.588371 1.15005,0 2.67264,-0.694412 3.42829,-2.08471 l 0.38312,0.395217 c -0.47378,1.071703 -0.9899,1.848523 -1.54658,2.331513 -0.55527,0.48327 -1.87678,0.72414 -2.82434,0.72414 -1.01776,0 -1.86161,-0.28497 -2.53259,-0.85548 -0.67204,-0.57076 -1.11407,-1.378306 -1.33456,-2.423128 -0.37465,-1.783751 -0.0843,-3.405012 0.86854,-4.868334 0.95779,-1.460853 2.19145,-2.19329 3.70593,-2.19329 0.59584,0 1.66511,0.152118 2.07574,0.453637 0.40535,0.30166 0.66393,0.707072 0.77118,1.214614 0.18027,0.85411 -0.78352,1.660948 -1.74484,2.423689 -0.96203,0.762318 -2.38725,1.291767 -4.26826,1.589582" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path33" /> <g id="g37" clip-path="url(#clipPath41)" transform="matrix(0.03527778,0,0,-0.03527778,42.828565,121.98722)"> <path d="m 994.055,997.609 c 6.725,-11.629 35.115,-61.371 40.815,-77.398 6.42,-18.121 7.77,-23.313 7.77,-23.313 0,0 -15.57,80.114 -41.12,126.862 5.6,18.9 12.29,39.79 19.86,62.12 8.85,-15.53 28.96,-51.2 36.23,-68.1 0.27,3.19 0.54,6.38 0.82,9.54 -6.44,24.75 -16.22,57.15 -28.74,82.54 32.14,167.25 135.59,386.25 247.04,504.48 H 509.602 c -37.883,0 -68.711,-30.82 -68.711,-68.71 V 731.789 c 173.738,66.68 383.23,127.633 563.529,125.02 -6.693,25.812 -14.315,49.152 -22.318,62.679 -4.122,6.981 0.449,35.653 11.953,78.121" style="fill:url(#linearGradient51);fill-opacity:1;fill-rule:nonzero;stroke:none" id="path53" /> </g> <path d="m 95.439377,63.024997 c -1.923345,-1.715206 -4.252031,-1.026231 -6.550379,1.01353 -0.341136,0.303037 -0.681566,0.639234 -1.020233,0.998362 -3.931709,4.170892 -7.581195,11.896725 -8.715023,17.796934 0.441678,0.895703 0.786695,2.038703 1.013884,2.911828 0.05821,0.224014 0.110772,0.43427 0.152752,0.613128 0.09984,0.423298 0.153459,0.697794 0.153459,0.697794 0,0 -0.03528,-0.133385 -0.179917,-0.552802 -0.02752,-0.08043 -0.05821,-0.168275 -0.09454,-0.271639 -0.01552,-0.04269 -0.03704,-0.09454 -0.06068,-0.149931 -0.25647,-0.596194 -0.965906,-1.854553 -1.278114,-2.402417 -0.267053,0.787753 -0.503061,1.524706 -0.700617,2.191456 0.901348,1.649166 1.450623,4.47541 1.450623,4.47541 0,0 -0.04762,-0.183163 -0.274109,-0.822431 -0.201083,-0.565397 -1.202619,-2.320185 -1.439862,-2.73043 -0.405836,1.498177 -0.567091,2.509661 -0.421676,2.755936 0.282328,0.477202 0.551216,1.300586 0.78733,2.211176 0.5334,2.051367 0.904169,4.548646 0.904169,4.548646 0,0 0.01199,0.165488 0.0321,0.420158 -0.07408,1.722685 -0.02963,3.508765 0.103717,5.123145 0.176741,2.13706 0.509411,3.97288 0.93345,4.9554 l 0.287866,-0.15695 c -0.622652,-1.93573 -0.875594,-4.47255 -0.764822,-7.39814 0.16757,-4.471846 1.196623,-9.86469 3.098095,-15.485534 3.212395,-8.485012 7.669389,-15.292918 11.748559,-18.544118 -3.717925,3.357739 -8.749948,14.22647 -10.256309,18.251312 -1.686631,4.50709 -2.881842,8.736578 -3.602214,12.788831 1.242836,-3.798959 5.261328,-5.431932 5.261328,-5.431932 0,0 1.97097,-2.430745 4.274256,-5.903454 -1.379714,0.314678 -3.645253,0.853369 -4.404078,1.172281 -1.119364,0.469547 -1.420989,0.629708 -1.420989,0.629708 0,0 3.62585,-2.208036 6.736645,-3.207809 4.278136,-6.738056 8.939042,-16.310328 4.245328,-20.497448" style="fill:#003b57;fill-opacity:1;fill-rule:nonzero;stroke:none;stroke-width:0.0352778" id="path55" /> </g> </g> </svg> |
Added auto.def.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 | #!/do/not/tclsh # ^^^ help out editors which guess this file's content type. # # This is the main autosetup-compatible configure script for the # SQLite project. # # This script and all of its dependencies must be kept compatible with # JimTCL, a copy of which is included in this source tree as # ./autosetup/jimsh0.c. The number of incompatibilities between # canonical TCL and JimTCL is very low and alternative formulations of # incompatible constructs have, so far, been easy to find. # # JimTCL: https://jim.tcl.tk # use sqlite-config sqlite-configure canonical { proj-if-opt-truthy dev { # --enable-dev needs to come early so that the downstream tests # which check for the following flags use their updated state. proj-opt-set all 1 proj-opt-set debug 1 proj-opt-set amalgamation 0 define CFLAGS [get-env CFLAGS {-O0 -g}] # -------------^^^^^^^ intentionally using [get-env] instead of # [proj-get-env] here because [sqlite-setup-default-cflags] uses # [proj-get-env] and we want this to supercede that. sqlite-munge-cflags; # straighten out -DSQLITE_ENABLE/OMIT flags } sqlite-handle-debug ;# must come after --dev flag check sqlite-check-common-bins ;# must come before [sqlite-handle-wasi-sdk] sqlite-handle-wasi-sdk ;# must run relatively early, as it changes the environment sqlite-check-common-system-deps proj-define-for-opt amalgamation USE_AMALGAMATION "Use amalgamation for builds?" proj-define-for-opt gcov USE_GCOV "Use gcov?" proj-define-for-opt test-status TSTRNNR_OPTS \ "test-runner flags:" {--status} {} proj-define-for-opt linemacros AMALGAMATION_LINE_MACROS \ "Use #line macros in the amalgamation:" define AMALGAMATION_EXTRA_SRC \ [join [opt-val amalgamation-extra-src ""] " "] define LINK_TOOLS_DYNAMICALLY [proj-opt-was-provided dynlink-tools] if {[set fsan [join [opt-val asan-fsanitize] ","]] in {auto ""}} { set fsan address,bounds-strict } define CFLAGS_ASAN_FSANITIZE [proj-check-fsanitize [split $fsan ", "]] sqlite-handle-tcl sqlite-handle-emsdk proj-if-opt-truthy static-shells { proj-opt-set static-tclsqlite3 1 proj-opt-set static-cli-shell 1 } proj-define-for-opt static-tclsqlite3 STATIC_TCLSQLITE3 "Statically link tclsqlite3?" proj-define-for-opt static-cli-shell STATIC_CLI_SHELL "Statically link CLI shell?" }; # sqlite-configure |
Deleted autoconf/INSTALL.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted autoconf/Makefile.am.
|
| < < < < < < < < < < < < < < < < < < < < |
Added autoconf/Makefile.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 | ######################################################################## # This is a main makefile for the "autoconf" bundle of SQLite. This is # a trimmed-down version of the canonical makefile, devoid of most # documentation. For the full docs, see /main.mk in the canonical # source tree. # # Maintenance reminders: # # - To keep this working with an out-of-tree build, be sure to prefix # input file names with $(TOP)/ where appropriate (which is most # places). # # - The original/canonical recipes can be found in /main.mk in the # canonical source tree. all: TOP = @abs_top_srcdir@ PACKAGE_VERSION = @PACKAGE_VERSION@ # # Filename extensions for binaries and libraries # B.exe = @BUILD_EXEEXT@ T.exe = @TARGET_EXEEXT@ B.dll = @BUILD_DLLEXT@ T.dll = @TARGET_DLLEXT@ B.lib = @BUILD_LIBEXT@ T.lib = @TARGET_LIBEXT@ # # Autotools-compatibility dirs # prefix = @prefix@ datadir = @datadir@ mandir = @mandir@ includedir = @includedir@ exec_prefix = @exec_prefix@ bindir = @bindir@ libdir = @libdir@ # # Required binaries # INSTALL = @BIN_INSTALL@ AR = @AR@ AR.flags = cr CC = @CC@ ENABLE_LIB_SHARED = @ENABLE_LIB_SHARED@ ENABLE_LIB_STATIC = @ENABLE_LIB_STATIC@ HAVE_WASI_SDK = @HAVE_WASI_SDK@ CFLAGS = @CFLAGS@ @CPPFLAGS@ # # $(LDFLAGS.configure) represents any LDFLAGS=... the client passes to # configure. See main.mk. # LDFLAGS.configure = @LDFLAGS@ CFLAGS.core = @SH_CFLAGS@ LDFLAGS.shlib = @SH_LDFLAGS@ LDFLAGS.zlib = @LDFLAGS_ZLIB@ LDFLAGS.math = @LDFLAGS_MATH@ LDFLAGS.rpath = @LDFLAGS_RPATH@ LDFLAGS.pthread = @LDFLAGS_PTHREAD@ LDFLAGS.dlopen = @LDFLAGS_DLOPEN@ LDFLAGS.readline = @LDFLAGS_READLINE@ CFLAGS.readline = @CFLAGS_READLINE@ LDFLAGS.rt = @LDFLAGS_RT@ LDFLAGS.icu = @LDFLAGS_ICU@ CFLAGS.icu = @CFLAGS_ICU@ # INSTALL reminder: we specifically do not strip binaries, # as discussed in https://sqlite.org/forum/forumpost/9a67df63eda9925c. INSTALL.noexec = $(INSTALL) -m 0644 install-dir.bin = $(DESTDIR)$(bindir) install-dir.lib = $(DESTDIR)$(libdir) install-dir.include = $(DESTDIR)$(includedir) install-dir.pkgconfig = $(DESTDIR)$(libdir)/pkgconfig install-dir.man1 = $(DESTDIR)$(mandir)/man1 install-dir.all = $(install-dir.bin) $(install-dir.include) \ $(install-dir.lib) $(install-dir.man1) \ $(install-dir.pkgconfig) $(install-dir.all): @if [ ! -d "$@" ]; then set -x; $(INSTALL) -d "$@"; fi # ^^^^ on some platforms, install -d fails if the target already exists. # # Vars with the AS_ prefix are specifically related to AutoSetup. # # AS_AUTO_DEF is the main configure script. # AS_AUTO_DEF = $(TOP)/auto.def # # Shell commands to re-run $(TOP)/configure with the same args it was # invoked with to produce this makefile. # AS_AUTORECONFIG = @SQLITE_AUTORECONFIG@ Makefile: $(TOP)/Makefile.in $(AS_AUTO_DEF) $(AS_AUTORECONFIG) @touch $@ sqlite3.pc: $(TOP)/sqlite3.pc.in $(AS_AUTO_DEF) $(AS_AUTORECONFIG) @touch $@ sqlite_cfg.h: $(AS_AUTO_DEF) $(AS_AUTORECONFIG) @touch $@ # # CFLAGS for sqlite3$(T.exe) # SHELL_OPT ?= @OPT_SHELL@ # # Library-level feature flags # OPT_FEATURE_FLAGS = @OPT_FEATURE_FLAGS@ LDFLAGS.libsqlite3.soname = @LDFLAGS_LIBSQLITE3_SONAME@ # soname: see https://sqlite.org/src/forumpost/5a3b44f510df8ded LDFLAGS.libsqlite3.os-specific = \ @LDFLAGS_MAC_CVERSION@ @LDFLAGS_MAC_INSTALL_NAME@ @LDFLAGS_OUT_IMPLIB@ LDFLAGS.libsqlite3 = \ $(LDFLAGS.rpath) $(LDFLAGS.pthread) \ $(LDFLAGS.math) $(LDFLAGS.dlopen) \ $(LDFLAGS.zlib) $(LDFLAGS.icu) \ $(LDFLAGS.rt) $(LDFLAGS.configure) CFLAGS.libsqlite3 = -I. $(CFLAGS.core) $(CFLAGS.icu) $(OPT_FEATURE_FLAGS) sqlite3.o: $(TOP)/sqlite3.h $(TOP)/sqlite3.c $(CC) -c $(TOP)/sqlite3.c -o $@ $(CFLAGS) $(CFLAGS.libsqlite3) libsqlite3.LIB = libsqlite3$(T.lib) libsqlite3.DLL.basename = @SQLITE_DLL_BASENAME@ libsqlite3.out.implib = @SQLITE_OUT_IMPLIB@ libsqlite3.DLL = $(libsqlite3.DLL.basename)$(T.dll) libsqlite3.DLL.install-rules = @SQLITE_DLL_INSTALL_RULES@ $(libsqlite3.DLL): sqlite3.o $(CC) -o $@ sqlite3.o $(LDFLAGS.shlib) \ $(LDFLAGS) $(LDFLAGS.libsqlite3) \ $(LDFLAGS.libsqlite3.os-specific) $(LDFLAGS.libsqlite3.soname) $(libsqlite3.DLL)-1: $(libsqlite3.DLL) $(libsqlite3.DLL)-0: all: $(libsqlite3.DLL)-$(ENABLE_LIB_SHARED) $(libsqlite3.LIB): sqlite3.o $(AR) $(AR.flags) $@ sqlite3.o $(libsqlite3.LIB)-1: $(libsqlite3.LIB) $(libsqlite3.LIB)-0: all: $(libsqlite3.LIB)-$(ENABLE_LIB_STATIC) # # Maintenance reminder: the install-dll-... rules must be kept in sync # with the main copies rom /main.mk. # install-dll-out-implib: $(install-dir.lib) $(libsqlite3.DLL) if [ x != "x$(libsqlite3.out.implib)" ] && [ -f "$(libsqlite3.out.implib)" ]; then \ $(INSTALL) $(libsqlite3.out.implib) "$(install-dir.lib)"; \ fi install-dll-unix-generic: install-dll-out-implib $(INSTALL) $(libsqlite3.DLL) "$(install-dir.lib)" @echo "Setting up $(libsqlite3.DLL) version symlinks..."; \ cd "$(install-dir.lib)" || exit $$?; \ rm -f $(libsqlite3.DLL).0 $(libsqlite3.DLL).$(PACKAGE_VERSION) || exit $$?; \ mv $(libsqlite3.DLL) $(libsqlite3.DLL).$(PACKAGE_VERSION) || exit $$?; \ ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL) || exit $$?; \ ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL).0 || exit $$?; \ ls -la $(libsqlite3.DLL) $(libsqlite3.DLL).[a03]*; \ if [ -e $(libsqlite3.DLL).0.8.6 ]; then \ echo "ACHTUNG: legacy libtool-compatible install found. Re-linking it..."; \ rm -f libsqlite3.la $(libsqlite3.DLL).0.8.6 || exit $$?; \ ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL).0.8.6 || exit $$?; \ ls -la $(libsqlite3.DLL).0.8.6; \ elif [ x1 = "x$(INSTALL_SO_086_LINK)" ]; then \ echo "ACHTUNG: installing legacy libtool-style links because INSTALL_SO_086_LINK=1"; \ rm -f libsqlite3.la $(libsqlite3.DLL).0.8.6 || exit $$?; \ ln -s $(libsqlite3.DLL).$(PACKAGE_VERSION) $(libsqlite3.DLL).0.8.6 || exit $$?; \ ls -la $(libsqlite3.DLL).0.8.6; \ fi install-dll-msys: install-dll-out-implib $(install-dir.bin) $(INSTALL) $(libsqlite3.DLL) "$(install-dir.bin)" # ----------------------------------------------^^^ yes, bin # Each of {msys,mingw,cygwin} uses a different name for the DLL, but # that is already accounted for via $(libsqlite3.DLL). install-dll-mingw: install-dll-msys install-dll-cygwin: install-dll-msys install-dll-darwin: $(install-dir.lib) $(libsqlite3.DLL) $(INSTALL) $(libsqlite3.DLL) "$(install-dir.lib)" @echo "Setting up $(libsqlite3.DLL) version symlinks..."; \ cd "$(install-dir.lib)" || exit $$?; \ rm -f libsqlite3.0$(T.dll) libsqlite3.$(PACKAGE_VERSION)$(T.dll) || exit $$?; \ dllname=libsqlite3.$(PACKAGE_VERSION)$(T.dll); \ mv $(libsqlite3.DLL) $$dllname || exit $$?; \ ln -s $$dllname $(libsqlite3.DLL) || exit $$?; \ ln -s $$dllname libsqlite3.0$(T.dll) || exit $$?; \ ls -la $$dllname $(libsqlite3.DLL) libsqlite3.0$(T.dll) install-dll-1: install-dll-$(libsqlite3.DLL.install-rules) install-dll-0 install-dll-: install-dll: install-dll-$(ENABLE_LIB_SHARED) install: install-dll install-lib-1: $(install-dir.lib) $(libsqlite3.LIB) $(INSTALL.noexec) $(libsqlite3.LIB) "$(install-dir.lib)" install-lib-0 install-lib-: install-lib: install-lib-$(ENABLE_LIB_STATIC) install: install-lib # # Flags to link the shell app either directly against sqlite3.c # (ENABLE_STATIC_SHELL==1) or libsqlite3.so (ENABLE_STATIC_SHELL==0). # ENABLE_STATIC_SHELL = @ENABLE_STATIC_SHELL@ sqlite3-shell-link-flags.1 = $(TOP)/sqlite3.c $(LDFLAGS.libsqlite3) sqlite3-shell-link-flags.0 = -L. -lsqlite3 $(LDFLAGS.zlib) $(LDFLAGS.math) sqlite3-shell-deps.1 = $(TOP)/sqlite3.c sqlite3-shell-deps.0 = $(libsqlite3.DLL) # # STATIC_CLI_SHELL = 1 to statically link sqlite3$(T.exe), else # 0. Requires static versions of all requisite libraries. Primarily # intended for use with static-friendly environments like Alpine # Linux. # STATIC_CLI_SHELL = @STATIC_CLI_SHELL@ # # sqlite3-shell-static.flags.N = N is $(STATIC_CLI_SHELL) # sqlite3-shell-static.flags.1 = -static sqlite3-shell-static.flags.0 = sqlite3$(T.exe): $(TOP)/shell.c $(sqlite3-shell-deps.$(ENABLE_STATIC_SHELL)) $(CC) -o $@ \ $(TOP)/shell.c $(sqlite3-shell-link-flags.$(ENABLE_STATIC_SHELL)) \ $(sqlite3-shell-static.flags.$(STATIC_CLI_SHELL)) \ -I. $(OPT_FEATURE_FLAGS) $(SHELL_OPT) \ $(CFLAGS) $(CFLAGS.readline) $(CFLAGS.icu) \ $(LDFLAGS) $(LDFLAGS.readline) sqlite3$(T.exe)-1: sqlite3$(T.exe)-0: sqlite3$(T.exe) all: sqlite3$(T.exe)-$(HAVE_WASI_SDK) install-shell-0: sqlite3$(T.exe) $(install-dir.bin) $(INSTALL) sqlite3$(T.exe) "$(install-dir.bin)" install-shell-1: install: install-shell-$(HAVE_WASI_SDK) install-headers: $(TOP)/sqlite3.h $(install-dir.include) $(INSTALL.noexec) $(TOP)/sqlite3.h $(TOP)/sqlite3ext.h "$(install-dir.include)" install: install-headers install-pc: sqlite3.pc $(install-dir.pkgconfig) $(INSTALL.noexec) sqlite3.pc "$(install-dir.pkgconfig)" install: install-pc install-man1: $(TOP)/sqlite3.1 $(install-dir.man1) $(INSTALL.noexec) $(TOP)/sqlite3.1 "$(install-dir.man1)" install: install-man1 clean: rm -f *.o sqlite3$(T.exe) rm -f $(libsqlite3.LIB) $(libsqlite3.DLL) libsqlite3$(T.dll).a distclean: clean rm -f jimsh0$(T.exe) config.* sqlite3.pc sqlite_cfg.h Makefile DIST_FILES := \ README.txt VERSION \ auto.def autosetup configure tea \ sqlite3.h sqlite3.c shell.c sqlite3ext.h \ Makefile.in Makefile.msc Makefile.fallback \ sqlite3.rc sqlite3rc.h Replace.cs \ sqlite3.pc.in sqlite3.1 # # Maintenance note: dist_name must be sqlite-$(PACKAGE_VERSION) so # that tool/mkautoconfamal.sh knows how to find it. # dist_name = sqlite-$(PACKAGE_VERSION) dist_tarball = $(dist_name).tar.gz dist: rm -fr $(dist_name) mkdir -p $(dist_name) cp -rp $(DIST_FILES) $(dist_name)/. tar czf $(dist_tarball) $(dist_name) rm -fr $(dist_name) ls -l $(dist_tarball) |
Changes to autoconf/Makefile.msc.
︙ | ︙ | |||
13 14 15 16 17 18 19 20 21 22 23 24 25 26 | ############################################################################### # The toplevel directory of the source tree. This is the directory # that contains this "Makefile.msc". # TOP = . # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN USE_FULLWARN = 1 !ENDIF | > > > > > > > > > | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | ############################################################################### # The toplevel directory of the source tree. This is the directory # that contains this "Makefile.msc". # TOP = . # Optionally set EXTRA_SRC to a list of C files to append to # the generated sqlite3.c. Any sqlite3 extensions added this # way may require manual editing, as described in # https://sqlite.org/forum/forumpost/903f721f3e7c0d25 # !IFNDEF EXTRA_SRC EXTRA_SRC = !ENDIF # Set this non-0 to enable full warnings (-W4, etc) when compiling. # !IFNDEF USE_FULLWARN USE_FULLWARN = 1 !ENDIF |
︙ | ︙ | |||
47 48 49 50 51 52 53 54 55 56 57 58 59 60 | # Set this non-0 to use "stdcall" calling convention for the core library # and shell executable. # !IFNDEF USE_STDCALL USE_STDCALL = 0 !ENDIF # Set this non-0 to have the shell executable link against the core dynamic # link library. # !IFNDEF DYNAMIC_SHELL DYNAMIC_SHELL = 0 !ENDIF | > > > > > > > > > > > > > > > | 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 | # Set this non-0 to use "stdcall" calling convention for the core library # and shell executable. # !IFNDEF USE_STDCALL USE_STDCALL = 0 !ENDIF # Use the USE_SEH=0 option on the nmake command line to omit structured # exception handling (SEH) support. SEH is on by default. # !IFNDEF USE_SEH USE_SEH = 1 !ENDIF # Use STATICALLY_LINK_TCL=1 to statically link against TCL # !IFNDEF STATICALLY_LINK_TCL STATICALLY_LINK_TCL = 0 !ELSEIF $(STATICALLY_LINK_TCL)!=0 CCOPTS = $(CCOPTS) -DSTATIC_BUILD !ENDIF # Set this non-0 to have the shell executable link against the core dynamic # link library. # !IFNDEF DYNAMIC_SHELL DYNAMIC_SHELL = 0 !ENDIF |
︙ | ︙ | |||
175 176 177 178 179 180 181 182 183 184 185 186 187 188 | # Set this to non-0 to enable OSTRACE() macros, which can be useful when # debugging. # !IFNDEF OSTRACE OSTRACE = 0 !ENDIF # Set this to one of the following values to enable various debugging # features. Each level includes the debugging options from the previous # levels. Currently, the recognized values for DEBUG are: # # 0 == NDEBUG: Disables assert() and other runtime diagnostics. # 1 == SQLITE_ENABLE_API_ARMOR: extra attempts to detect misuse of the API. | > > > > > > | 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 | # Set this to non-0 to enable OSTRACE() macros, which can be useful when # debugging. # !IFNDEF OSTRACE OSTRACE = 0 !ENDIF # enable address sanitizer using ASAN=1 on the command-line. # !IFNDEF ASAN ASAN = 0 !ENDIF # Set this to one of the following values to enable various debugging # features. Each level includes the debugging options from the previous # levels. Currently, the recognized values for DEBUG are: # # 0 == NDEBUG: Disables assert() and other runtime diagnostics. # 1 == SQLITE_ENABLE_API_ARMOR: extra attempts to detect misuse of the API. |
︙ | ︙ | |||
212 213 214 215 216 217 218 219 220 221 222 223 224 225 | !ENDIF # Set this to non-0 to enable support for the rbu extension. # !IFNDEF RBU RBU = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c | > > > > > > | 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 | !ENDIF # Set this to non-0 to enable support for the rbu extension. # !IFNDEF RBU RBU = 0 !ENDIF # Set this to non-0 to enable support for blocking locks. # !IFNDEF SETLK_TIMEOUT SETLK_TIMEOUT = 0 !ENDIF # Set the source code file to be used by executables and libraries when # they need the amalgamation. # !IFNDEF SQLITE3C !IF $(SPLIT_AMALGAMATION)!=0 SQLITE3C = sqlite3-all.c |
︙ | ︙ | |||
277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 | !ENDIF # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 !ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF # Should the session extension be enabled? If so, add compilation options # to enable it. # !IF $(SESSION)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 !ENDIF # Always enable math functions on Windows OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS # Should the rbu extension be enabled? If so, add compilation options # to enable it. # !IF $(RBU)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 !ENDIF # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # !IFNDEF EXT_FEATURE_FLAGS !IF $(FOR_WIN10)!=0 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS4=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_SYSTEM_MALLOC=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_OMIT_LOCALTIME=1 !ELSE EXT_FEATURE_FLAGS = !ENDIF !ENDIF ############################################################################### ############################### END OF OPTIONS ################################ ############################################################################### # When compiling for the Windows 10 platform, the PLATFORM macro must be set # to an appropriate value (e.g. x86, x64, arm, arm64, etc). | > > > > > > > > > > > > > > > > > > > > > > | 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 | !ENDIF # These are the "standard" SQLite compilation options used when compiling for # the Windows platform. # !IFNDEF OPT_FEATURE_FLAGS OPT_FEATURE_FLAGS = $(OPT_XTRA) !IF $(MINIMAL_AMALGAMATION)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS3=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS5=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RTREE=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_GEOPOLY=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_STMTVTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBPAGE_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_DBSTAT_VTAB=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_BYTECODE_VTAB=1 !ENDIF OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_COLUMN_METADATA=1 !ENDIF # Additional feature-options above and beyond what are normally used can be # be added using OPTIONS=.... on the command-line. These values are # appended to the OPT_FEATURE_FLAGS variable. # !IFDEF OPTIONS OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) $(OPTIONS) !ENDIF # Should the session extension be enabled? If so, add compilation options # to enable it. # !IF $(SESSION)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SESSION=1 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_PREUPDATE_HOOK=1 !ENDIF # Always enable math functions on Windows OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_MATH_FUNCTIONS # Should the rbu extension be enabled? If so, add compilation options # to enable it. # !IF $(RBU)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_RBU=1 !ENDIF # Should structured exception handling (SEH) be enabled for WAL mode in # the core library? It is on by default. Only omit it if the # USE_SEH=0 option is provided on the nmake command-line. # !IF $(USE_SEH)==0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_OMIT_SEH=1 !ENDIF # These are the "extended" SQLite compilation options used when compiling for # the Windows 10 platform. # !IFNDEF EXT_FEATURE_FLAGS !IF $(FOR_WIN10)!=0 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_ENABLE_FTS4=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_SYSTEM_MALLOC=1 EXT_FEATURE_FLAGS = $(EXT_FEATURE_FLAGS) -DSQLITE_OMIT_LOCALTIME=1 !ELSE EXT_FEATURE_FLAGS = !ENDIF !ENDIF !IF $(SETLK_TIMEOUT)!=0 OPT_FEATURE_FLAGS = $(OPT_FEATURE_FLAGS) -DSQLITE_ENABLE_SETLK_TIMEOUT !ENDIF ############################################################################### ############################### END OF OPTIONS ################################ ############################################################################### # When compiling for the Windows 10 platform, the PLATFORM macro must be set # to an appropriate value (e.g. x86, x64, arm, arm64, etc). |
︙ | ︙ | |||
662 663 664 665 666 667 668 | !IF $(DEBUG)>0 || $(API_ARMOR)!=0 || $(FOR_WIN10)!=0 TCC = $(TCC) -DSQLITE_ENABLE_API_ARMOR=1 RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !ENDIF !IF $(DEBUG)>2 | | | 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 | !IF $(DEBUG)>0 || $(API_ARMOR)!=0 || $(FOR_WIN10)!=0 TCC = $(TCC) -DSQLITE_ENABLE_API_ARMOR=1 RCC = $(RCC) -DSQLITE_ENABLE_API_ARMOR=1 !ENDIF !IF $(DEBUG)>2 TCC = $(TCC) -DSQLITE_DEBUG=1 -DSQLITE_USE_W32_FOR_CONSOLE_IO RCC = $(RCC) -DSQLITE_DEBUG=1 !IF $(DYNAMIC_SHELL)==0 TCC = $(TCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE RCC = $(RCC) -DSQLITE_ENABLE_WHERETRACE -DSQLITE_ENABLE_SELECTTRACE !ENDIF !ENDIF |
︙ | ︙ | |||
713 714 715 716 717 718 719 720 721 722 723 724 725 726 | # !IF $(DEBUG)>3 TCC = $(TCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 !ENDIF !ENDIF # Compiler options needed for programs that use the readline() library. # !IFNDEF READLINE_FLAGS READLINE_FLAGS = -DHAVE_READLINE=0 !ENDIF | > > > > > > > | 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 | # !IF $(DEBUG)>3 TCC = $(TCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 RCC = $(RCC) -DSQLITE_WIN32_MALLOC_VALIDATE=1 !ENDIF !ENDIF # Address sanitizer if ASAN=1 # !IF $(ASAN)>0 TCC = $(TCC) /fsanitize=address !ENDIF # Compiler options needed for programs that use the readline() library. # !IFNDEF READLINE_FLAGS READLINE_FLAGS = -DHAVE_READLINE=0 !ENDIF |
︙ | ︙ | |||
742 743 744 745 746 747 748 | # Any target libraries which libsqlite must be linked against # !IFNDEF TLIBS TLIBS = !ENDIF | < < < < < < < < < | 807 808 809 810 811 812 813 814 815 816 817 818 819 820 | # Any target libraries which libsqlite must be linked against # !IFNDEF TLIBS TLIBS = !ENDIF # Enable/disable loadable extensions, and other optional features # based on configuration. (-DSQLITE_OMIT*, -DSQLITE_ENABLE*). # The same set of OMIT and ENABLE flags should be passed to the # LEMON parser generator and the mkkeywordhash tool as well. # These are the required SQLite compilation options used when compiling for # the Windows platform. |
︙ | ︙ | |||
951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 | !ENDIF # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 !ENDIF # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. # core: dll shell | > > > > | 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 | !ENDIF # Additional compiler options for the shell. These are only effective # when the shell is not being dynamically linked. # !IF $(DYNAMIC_SHELL)==0 && $(FOR_WIN10)==0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_DQS=0 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_FTS4=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_EXPLAIN_COMMENTS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_OFFSET_SQL_FUNC=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_UNKNOWN_SQL_FUNCTION=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_ENABLE_STMT_SCANSTATUS=1 SHELL_COMPILE_OPTS = $(SHELL_COMPILE_OPTS) -DSQLITE_STRICT_SUBTYPE=1 !ENDIF # This is the default Makefile target. The objects listed here # are what get build when you type just "make" with no arguments. # core: dll shell |
︙ | ︙ | |||
978 979 980 981 982 983 984 985 986 987 988 989 990 991 | # dll: $(SQLITE3DLL) # Shell executable. # shell: $(SQLITE3EXE) $(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) Replace.exe: $(CSC) /target:exe $(TOP)\Replace.cs | > > > > > | 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 | # dll: $(SQLITE3DLL) # Shell executable. # shell: $(SQLITE3EXE) # jimsh0 - replacement for tclsh # jimsh0.exe: $(TOP)\autosetup\jimsh0.c cl -DHAVE__FULLPATH=1 $(TOP)\autosetup\jimsh0.c $(SQLITE3DLL): $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) Replace.exe: $(CSC) /target:exe $(TOP)\Replace.cs |
︙ | ︙ | |||
1028 1029 1030 1031 1032 1033 1034 | echo #endif >> sqlite3rc.h $(LTRCOMPILE) -fo $(LIBRESOBJS) -DRC_VERONLY $(TOP)\sqlite3.rc !ENDIF clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL | | > | 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 | echo #endif >> sqlite3rc.h $(LTRCOMPILE) -fo $(LIBRESOBJS) -DRC_VERONLY $(TOP)\sqlite3.rc !ENDIF clean: del /Q *.exp *.lo *.ilk *.lib *.obj *.ncb *.pdb *.sdf *.suo 2>NUL del /Q *.bsc *.cod *.da *.bb *.bbg *.vc gmon.out 2>NUL del /Q sqlite3.def tclsqlite3.def ctime.c pragma.h 2>NUL del /Q $(SQLITE3EXE) $(SQLITE3DLL) Replace.exe 2>NUL |
Changes to autoconf/README.first.
|
| | | | > | | | | 1 2 3 4 5 6 7 8 9 10 11 12 | This directory contains components used to build an autoconf-like package of the SQLite amalgamation: sqlite-autoconf-30XXXXXX.tar.gz To build the autoconf amalgamation, run from the top of the canonical source tree: ./configure make amalgamation-tarball The amalgamation-tarball target (available in "main.mk") runs the script tool/mkautoconfamal.sh which does the work. Refer to that script for details. |
Changes to autoconf/README.txt.
1 2 3 4 5 6 | This package contains: * the SQLite library amalgamation source code file: sqlite3.c * the sqlite3.h and sqlite3ext.h header files that define the C-language interface to the sqlite3.c library file * the shell.c file used to build the sqlite3 command-line shell program | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | < | | > | < < | > | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | This package contains: * the SQLite library amalgamation source code file: sqlite3.c * the sqlite3.h and sqlite3ext.h header files that define the C-language interface to the sqlite3.c library file * the shell.c file used to build the sqlite3 command-line shell program * autoconf-like installation infrastucture for building on POSIX compliant systems * a Makefile.msc, sqlite3.rc, and Replace.cs for building with Microsoft Visual C++ on Windows WHY USE THIS PACKAGE? ===================== The canonical make system for SQLite requires TCL as part of the build process. Various TCL scripts are used to generate parts of the code and TCL is used to run tests. But some people would prefer to build SQLite using only generic tools and without having to install TCL. The purpose of this package is to provide that capability. This package contains a pre-build SQLite amalgamation file "sqlite3.c" (and its associated header file "sqlite3.h"). Because the amalgamation has been pre-built, no TCL is required for the code generate (the configure script itself is written in TCL but it can use the embedded copy of JimTCL). REASONS TO USE THE CANONICAL BUILD SYSTEM RATHER THAN THIS PACKAGE ================================================================== * the canonical build system allows you to run tests to verify that the build worked * the canonical build system supports more compile-time options * the canonical build system works for any arbitrary check-in to the SQLite source tree Step-by-step instructions on how to build using the canonical make system for SQLite can be found at: https://sqlite.org/src/doc/trunk/doc/compile-for-unix.md https://sqlite.org/src/doc/trunk/doc/compile-for-windows.md SUMMARY OF HOW TO BUILD USING THIS PACKAGE ========================================== Unix: ./configure; make Windows: nmake /f Makefile.msc BUILDING ON POSIX ================= The configure script follows common conventions, making it easy to use for anyone who has configured a software tree before. It supports a number of build-time flags, the full list of which can be seen by running: ./configure --help The default value for the CFLAGS variable (options passed to the C compiler) includes debugging symbols in the build, resulting in larger binaries than are necessary. Override it on the configure command line like this: $ CFLAGS="-Os" ./configure to produce a smaller installation footprint. Many SQLite compilation parameters can be defined by passing flags to the configure script. Others may be passed on in the CFLAGS. For example: $ CFLAGS="-Os -DSQLITE_OMIT_DEPRECATED" ./configure BUILDING WITH MICROSOFT VISUAL C++ ================================== To compile for Windows using Microsoft Visual C++: $ nmake /f Makefile.msc Using Microsoft Visual C++ 2005 (or later) is recommended. Several Windows platform variants may be built by adding additional macros to the NMAKE command line. Other preprocessor defines -------------------------- Additionally, preprocessor defines may be specified by using the OPTS macro on the NMAKE command line. However, not all possible preprocessor defines may be specified in this manner as some require the amalgamation to be built with them enabled (see http://sqlite.org/compile.html). For example, the following will work: "OPTS=-DSQLITE_ENABLE_STAT4=1 -DSQLITE_OMIT_JSON=1" However, the following will not compile unless the amalgamation was built with it enabled: |
︙ | ︙ |
Added autoconf/auto.def.
> > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | #!/do/not/tclsh # ^^^ help out editors which guess this file's content type. # # This is the main autosetup-compatible configure script for the # "autoconf" bundle of the SQLite project. use sqlite-config sqlite-configure autoconf { sqlite-handle-debug sqlite-check-common-bins ;# must come before [sqlite-handle-wasi-sdk] sqlite-handle-wasi-sdk ;# must run relatively early, as it changes the environment sqlite-check-common-system-deps proj-define-for-opt static-shell ENABLE_STATIC_SHELL \ "Link library statically into the CLI shell?" proj-define-for-opt static-cli-shell STATIC_CLI_SHELL "Statically link CLI shell?" if {![opt-bool static-shell] && [opt-bool static-cli-shell]} { proj-fatal "--disable-static-shell and --static-cli-shell are mutualy exclusive" } if {![opt-bool shared] && ![opt-bool static-shell]} { proj-opt-set shared 1 proj-indented-notice { NOTICE: ignoring --disable-shared because --disable-static-shell was specified. } } } |
Deleted autoconf/configure.ac.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to autoconf/tea/Makefile.in.
|
| | | | > > > > | < < > > | < < | > > > | | > > > | < < | < > | < < < | < > | | | | < | < < | < < > > > | | | | > > | | | | | | | | | < | > | | > > < < < < < < < < | < > > | | < | > | < | < < < < < < < | < < < < < | < > > > > | < < | | < | < | | | | | | | | | > | | | > > > > | | < | > | | | > > | | > | > | | < < > | < < < < < < > > > > > > < < < > | < > | | > | | | | | < < | > | | | | | | | | | | < | < < | | | < | > > > | | < > | > > | | < < | < < | | > | > | | < < | | | < < | | < < | > > | > > < < | < | < > > | < | > | | < < < | > > > | | | | < < > > | > > > > > > > > > < < < < | < < < < | < < | < < < < | < < < > | < < < | > > | < | | | > < | | > | > | | > | < < > > | | < | | > > < | < < < | < < < < | > > | > | > > | > > > > > > > > > > | | > > | < > | < | < < < < > > > > > < < | < < < < < < | < | < < | < | | < < < < < | < < > > > < < < > | > > | < | > | < | > > | | < | | | | | | | | | | | < < | | | < > | | > | | < < < < < | < | < | < < < | | | < > | < > > > < < < < < < | < < < < | | | < < | < < < < < < < | > > | < > | | | > > | | > > | < | < | | > > > | | > > > > | > > | | > > > > | | < | < | < < | < > > > > > > > > > > > > > > > > | | > > > | > > > > | < > > > | > | > > > > > > > > > > > > > > > > > > > > > > > > > | > | > > > > > > > > | | > > | < < > | > > > > > | | | | | | | | | | > > | > > > > > > > > | > > | > > > > | > > > > > | > | > > > > > > | > > > > | < > | < < < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 | all: # # Unless this file is named Makefile.in, you are probably looking # at an automatically generated/filtered copy and should probably not # edit it. # # This makefile is part of the teaish framework, a tool for building # Tcl extensions, conceptually related to TEA/tclconfig but using the # Autosetup configuration system instead of the GNU Autotools. # # A copy of this makefile gets processed for each extension separately # and populated with info about how to build, test, and install the # extension. # # Maintenance reminder: this file needs to stay portable with POSIX # Make, not just GNU Make. Yes, that's unfortunate because it makes # some things impossible (like skipping over swathes of rules when # 'make distclean' is invoked). # CC = @CC@ INSTALL = @BIN_INSTALL@ INSTALL.noexec = $(INSTALL) -m 0644 # # Var name prefixes: # # teaish. => teaish core # tx. => teaish extension # # Vars with a "tx." or "teaish." prefix are all "public" for purposes # of the extension makefile, but the extension must not any "teaish." # vars and must only modify "tx." vars where that allowance is # specifically noted. # # Vars with a "teaish__" prefix are "private" and must not be used by # the extension makefile. They may change semantics or be removed in # any given teaish build. # tx.name = @TEAISH_NAME@ tx.version = @TEAISH_VERSION@ tx.name.pkg = @TEAISH_PKGNAME@ tx.libdir = @TEAISH_LIBDIR_NAME@ tx.loadPrefix = @TEAISH_LOAD_PREFIX@ tx.tcl = @TEAISH_TCL@ tx.makefile = @TEAISH_MAKEFILE@ tx.makefile.in = @TEAISH_MAKEFILE_IN@ tx.dll8.basename = @TEAISH_DLL8_BASENAME@ tx.dll9.basename = @TEAISH_DLL9_BASENAME@ tx.dll8 = @TEAISH_DLL8@ tx.dll9 = @TEAISH_DLL9@ tx.dll = $(tx.dll$(TCL_MAJOR_VERSION)) tx.dir = @TEAISH_EXT_DIR@ @if TEAISH_TM_TCL # Input filename for tcl::tm-style module tx.tm = @TEAISH_TM_TCL@ # Target filename for tcl::tm-style installation tx.tm.tgt = $(tx.name.pkg)-$(tx.version).tm @endif @if TEAISH_DIST_NAME tx.name.dist = @TEAISH_DIST_NAME@ @else tx.name.dist = $(teaish.name) @endif teaish.dir = @abs_top_srcdir@ #teaish.dir.autosetup = @TEAISH_AUTOSETUP_DIR@ teaish.makefile = Makefile teaish.makefile.in = $(teaish.dir)/Makefile.in teaish__auto.def = $(teaish.dir)/auto.def # # Autotools-conventional vars. We don't actually use these in this # makefile but some may be referenced by vars imported via # tclConfig.sh. They are part of the public API and may be reliably # depended on from teaish.make.in. # bindir = @bindir@ datadir = @datadir@ exec_prefix = @exec_prefix@ includedir = @includedir@ infodir = @infodir@ libdir = @libdir@ libexecdir = @libexecdir@ localstatedir = @localstatedir@ mandir = @mandir@ prefix = @prefix@ runstatedir = @runstatedir@ sbindir = @sbindir@ sharedstatedir = @sharedstatedir@ sysconfdir = @sysconfdir@ # # Vars derived (mostly) from tclConfig.sh. These may be reliably # used from the extension makefile. # TCLSH = @TCLSH_CMD@ TCL_CONFIG_SH = @TCL_CONFIG_SH@ TCL_EXEC_PREFIX = @TCL_EXEC_PREFIX@ TCL_INCLUDE_SPEC = @TCL_INCLUDE_SPEC@ TCL_LIBS = @TCL_LIBS@ TCL_LIB_SPEC = @TCL_LIB_SPEC@ TCL_MAJOR_VERSION = @TCL_MAJOR_VERSION@ TCL_MINOR_VERSION = @TCL_MINOR_VERSION@ TCL_PATCH_LEVEL = @TCL_PATCH_LEVEL@ TCL_PREFIX = @TCL_PREFIX@ TCL_SHLIB_SUFFIX = @TCL_SHLIB_SUFFIX@ TCL_STUB_LIB_SPEC = @TCL_STUB_LIB_SPEC@ TCL_VERSION = @TCL_VERSION@ TCLLIBDIR = @TCLLIBDIR@ # # CFLAGS.configure = CFLAGS as known at configure-time. # # This ordering is deliberate: flags populated via tcl's # [teaish-cflags-add] should preceed CFLAGS and CPPFLAGS (which # typically come from the ./configure command-line invocation). # CFLAGS.configure = @SH_CFLAGS@ @TEAISH_CFLAGS@ @CFLAGS@ @CPPFLAGS@ $(TCL_INCLUDE_SPEC) #CFLAGS.configure += -DUSE_TCL_STUBS=1 # # LDFLAGS.configure = LDFLAGS as known at configure-time. # # This ordering is deliberate: flags populated via tcl's # [teaish-ldflags-add] should precede LDFLAGS (which typically # comes from the ./configure command-line invocation). # LDFLAGS.configure = @TEAISH_LDFLAGS@ @LDFLAGS@ # # Linker flags for linkhing a shared library. # LDFLAGS.shlib = @SH_LDFLAGS@ # # The following tx.XYZ vars may be populated/modified by teaish.tcl # and/or teaish.make. # # # tx.src is the list of source or object files to include in the # (single) compiler/linker invocation. This will initially contain any # sources passed to [teaish-src-add], but may also be appended to by # teaish.make. # tx.src = @TEAISH_EXT_SRC@ # # tx.CFLAGS is typically set by teaish.make, whereas TEAISH_CFLAGS # gets set up via the configure script. # tx.CFLAGS = # # tx.LDFLAGS is typically set by teaish.make, whereas TEAISH_LDFLAGS # gets set up via the configure script. # tx.LDFLAGS = # # The list of 'dist' files may be appended to from teaish.make.in. # It can also be set up from teaish.tcl using [teaish-dist-add] # and/or [teaish-src-add -dist ...]. # tx.dist.files = @TEAISH_DIST_FILES@ # # The base name for a distribution tar/zip file. # tx.dist.basename = $(tx.name.dist)-$(tx.version) # List of deps which may trigger an auto-reconfigure. # teaish__autogen.deps = \ $(tx.makefile.in) $(teaish.makefile.in) \ $(tx.tcl) \ @TEAISH_PKGINDEX_TCL_IN@ @TEAISH_TM_TCL_IN@ \ @AUTODEPS@ @if TEAISH_MAKEFILE_IN $(tx.makefile): $(tx.makefile.in) @endif teaish.autoreconfig = \ @TEAISH_AUTORECONFIG@ # # Problem: when more than one target can invoke TEAISH_AUTORECONFIG, # we can get parallel reconfigures running. Thus, targets which # may require reconfigure should depend on... # config.log: $(teaish__autogen.deps) $(teaish.autoreconfig) # ^^^ We would love to skip this when running [dist]clean, but there's # no POSIX Make-portable way to do that. GNU Make can. .PHONY: reconfigure reconfigure: $(teaish.autoreconfig) $(teaish.makefile): $(teaish__auto.def) $(teaish.makefile.in) \ @AUTODEPS@ @if TEAISH_TESTER_TCL_IN @TEAISH_TESTER_TCL_IN@: $(teaish__autogen.deps) config.log: @TEAISH_TESTER_TCL_IN@ @TEAISH_TESTER_TCL@: @TEAISH_TESTER_TCL_IN@ @endif @if TEAISH_TEST_TCL_IN @TEAISH_TEST_TCL_IN@: $(teaish__autogen.deps) config.log: @TEAISH_TEST_TCL_IN@ @TEAISH_TEST_TCL@: @TEAISH_TEST_TCL_IN@ @endif # # CC variant for compiling Tcl-using sources. # CC.tcl = \ $(CC) -o $@ $(CFLAGS.configure) $(CFLAGS) $(tx.CFLAGS) # # CC variant for linking $(tx.src) into an extension DLL. Note that # $(tx.src) must come before $(LDFLAGS...) for linking to third-party # libs to work. # CC.dll = \ $(CC.tcl) $(tx.src) $(LDFLAGS.shlib) \ $(tx.LDFLAGS) $(LDFLAGS.configure) $(LDFLAGS) $(TCL_STUB_LIB_SPEC) @if TEAISH_ENABLE_DLL # # The rest of this makefile exists solely to support this brief # target: the extension shared lib. # $(tx.dll): $(tx.src) config.log @if [ "x" = "x$(tx.src)" ]; then \ echo "Makefile var tx.src (source/object files) is empty" 1>&2; \ exit 1; \ fi $(CC.dll) all: $(tx.dll) @endif # TEAISH_ENABLE_DLL tclsh: $(teaish.makefile) config.log @{ echo "#!/bin/sh"; echo 'exec $(TCLSH) "$$@"'; } > $@ @chmod +x $@ @echo "Created $@" # # Run the generated test script. # .PHONY: test-pre test-prepre test-core test test-post test-extension test-extension: # this name is reserved for use by teaish.make[.in] @if TEAISH_ENABLE_DLL test-prepre: $(tx.dll) @endif @if TEAISH_TESTER_TCL teaish.tester.tcl = @TEAISH_TESTER_TCL@ test-core.args = $(teaish.tester.tcl) @if TEAISH_ENABLE_DLL test-core.args += '$(tx.dll)' '$(tx.loadPrefix)' @else test-core.args += '' '' @endif test-core.args += @TEAISH_TESTUTIL_TCL@ # Clients may pass additional args via test.args=... # and ::argv will be rewritten before the test script loads, to # remove $(test-core.args) test.args ?= test-core: test-pre $(TCLSH) $(test-core.args) $(test.args) test-gdb: $(teaish.tester.tcl) gdb --args $(TCLSH) $(test-core.args) $(test.args) test-vg.flags ?= --leak-check=full -v --show-reachable=yes --track-origins=yes test-vg: $(teaish.tester.tcl) valgrind $(test-vg.flags) $(TCLSH) $(test-core.args) $(test.args) @else # !TEAISH_TESTER_TCL test-prepre: @endif # TEAISH_TESTER_TCL test-pre: test-prepre test-core: test-pre test-post: test-core test: test-post # # Cleanup rules... # #.PHONY: clean-pre clean-core clean-post clean-extension # clean-pre: clean-core: clean-pre rm -f $(tx.dll8) $(tx.dll9) tclsh clean-post: clean-core clean: clean-post .PHONY: distclean-pre distclean-core distclean-post clean-extension distclean-pre: clean distclean-core: distclean-pre rm -f Makefile rm -f config.log config.defines.txt @if TEAISH_MAKEFILE_IN @if TEAISH_MAKEFILE rm -f @TEAISH_MAKEFILE@ @endif @endif @if TEAISH_TESTER_TCL_IN rm -f $(teaish.tester.tcl) @endif @if TEAISH_PKGINDEX_TCL_IN rm -f @TEAISH_PKGINDEX_TCL@ @endif @if TEAISH_PKGINIT_TCL_IN rm -f @TEAISH_PKGINIT_TCL@ @endif @if TEAISH_TEST_TCL_IN rm -f @TEAISH_TEST_TCL@ @endif distclean-post: distclean-core distclean: distclean-post # # The (dist)clean-extension targets are reserved for use by # client-side teaish.make. # # Client code which wants to clean up extra stuff should do so by # adding their cleanup target (e.g. clean-extension) as a dependency # to the 'clean' target, like so: # # clean: distclean-extension # distclean: distclean-extension # distclean-extension: clean-extension: # # Installation rules... # @if TEAISH_ENABLE_INSTALL .PHONY: install-pre install-core install-post install-test install-prepre install-extension install-extension: # this name is reserved for use by teaish.make @if TEAISH_ENABLE_DLL install-prepre: $(tx.dll) @else install-prepre: @endif @if TEAISH_TM_TCL install-core.tmdir = $(DESTDIR)@TEAISH_TCL_TM_DIR@ @endif install-pre: install-prepre install-core: install-pre @if [ ! -d "$(DESTDIR)$(TCLLIBDIR)" ]; then \ set -x; $(INSTALL) -d "$(DESTDIR)$(TCLLIBDIR)"; \ fi # ^^^^ on some platforms, install -d fails if the target already exists. @if TEAISH_ENABLE_DLL $(INSTALL) $(tx.dll) "$(DESTDIR)$(TCLLIBDIR)" @endif @if TEAISH_PKGINDEX_TCL $(INSTALL.noexec) "@TEAISH_PKGINDEX_TCL@" "$(DESTDIR)$(TCLLIBDIR)" @endif @if TEAISH_PKGINIT_TCL $(INSTALL.noexec) "@TEAISH_PKGINIT_TCL@" "$(DESTDIR)$(TCLLIBDIR)" @endif @if TEAISH_TM_TCL @if [ ! -d "$(install-core.tmdir)" ]; then \ set -x; $(INSTALL) -d "$(install-core.tmdir)"; \ fi $(INSTALL.noexec) "@TEAISH_TM_TCL@" "$(install-core.tmdir)/$(tx.tm.tgt)" @endif install-test: install-core @echo "Post-install test of [package require $(tx.name.pkg) $(tx.version)]..."; \ set xtra=""; \ if [ x != "x$(DESTDIR)" ]; then \ xtra='set ::auto_path [linsert $$::auto_path 0 [file normalize $(DESTDIR)$(TCLLIBDIR)/..]];'; \ fi; \ if echo \ 'set c 0; ' $$xtra \ '@TEAISH_POSTINST_PREREQUIRE@' \ 'if {[catch {package require $(tx.name.pkg) $(tx.version)} xc]} {incr c};' \ 'if {$$c && "" ne $$xc} {puts $$xc; puts "auto_path=$$::auto_path"};' \ 'exit $$c' \ | $(TCLSH) ; then \ echo "passed"; \ else \ echo "FAILED"; \ exit 1; \ fi install-post: install-test install: install-post # # Uninstall rules... # .PHONY: uninstall uninstall-pre uninstall-core uninstall-post uninstall-extension uninstall-extension: # this name is reserved for use by teaish.make uninstall-pre: uninstall-core: uninstall-pre @if TEAISH_ENABLE_DLL rm -fr "$(DESTDIR)$(TCLLIBDIR)" @endif @if TEAISH_TM_TCL rm -f "$(DESTDIR)$(install-core.tmdir)/$(tx.tm.tgt)" @endif uninstall-post: uninstall-core @echo "Uninstalled Tcl extension $(tx.name) $(tx.version)" uninstall: uninstall-post @endif # TEAISH_ENABLE_INSTALL @if TEAISH_MAKEFILE_IN Makefile: $(tx.makefile.in) config.log: $(teaish.makefile.in) @endif # # Package archive generation ("dist") rules... # @if TEAISH_ENABLE_DIST @if BIN_TAR @if BIN_ZIP # When installing teaish as part of "make dist", we need to run # configure with similar flags to what we last configured with but we # must not pass on any extension-specific flags, as those won't be # recognized when running in --teaish-install mode, causing # the sub-configure to fail. dist.flags = --with-tclsh=$(TCLSH) dist.reconfig = $(teaish.dir)/configure $(tx.dist.reconfig-flags) $(dist.flags) # Temp dir for dist.zip. Must be different than dist.tgz or else # parallel builds may hose the dist. teaish__dist.tmp.zip = teaish__dist_zip # # Make a distribution zip file... # dist.zip = $(tx.dist.basename).zip .PHONY: dist.zip dist.zip-core dist.zip-post #dist.zip-pre: # We apparently can't add a pre-hook here, else "make dist" rebuilds # the archive each time it's run. $(dist.zip): $(tx.dist.files) @rm -fr $(teaish__dist.tmp.zip) @mkdir -p $(teaish__dist.tmp.zip)/$(tx.dist.basename) @tar cf $(teaish__dist.tmp.zip)/tmp.tar $(tx.dist.files) @tar xf $(teaish__dist.tmp.zip)/tmp.tar -C $(teaish__dist.tmp.zip)/$(tx.dist.basename) @if TEAISH_DIST_FULL @$(dist.reconfig) \ --teaish-install=$(teaish__dist.tmp.zip)/$(tx.dist.basename) \ --t-e-d=$(teaish__dist.tmp.zip)/$(tx.dist.basename) >/dev/null @endif @rm -f $(tx.dist.basename)/tmp.tar $(dist.zip) @cd $(teaish__dist.tmp.zip) && zip -q -r ../$(dist.zip) $(tx.dist.basename) @rm -fr $(teaish__dist.tmp.zip) @ls -la $(dist.zip) dist.zip-core: $(dist.zip) dist.zip-post: dist.zip-core dist.zip: dist.zip-post dist: dist.zip undist-zip: rm -f $(dist.zip) undist: undist-zip @endif #BIN_ZIP # # Make a distribution tarball... # teaish__dist.tmp.tgz = teaish__dist_tgz dist.tgz = $(tx.dist.basename).tar.gz .PHONY: dist.tgz dist.tgz-core dist.tgz-post # dist.tgz-pre: # see notes in dist.zip $(dist.tgz): $(tx.dist.files) @rm -fr $(teaish__dist.tmp.tgz) @mkdir -p $(teaish__dist.tmp.tgz)/$(tx.dist.basename) @tar cf $(teaish__dist.tmp.tgz)/tmp.tar $(tx.dist.files) @tar xf $(teaish__dist.tmp.tgz)/tmp.tar -C $(teaish__dist.tmp.tgz)/$(tx.dist.basename) @if TEAISH_DIST_FULL @rm -f $(teaish__dist.tmp.tgz)/$(tx.dist.basename)/pkgIndex.tcl.in; # kludge @$(dist.reconfig) \ --teaish-install=$(teaish__dist.tmp.tgz)/$(tx.dist.basename) \ --t-e-d=$(teaish__dist.tmp.zip)/$(tx.dist.basename) >/dev/null @endif @rm -f $(tx.dist.basename)/tmp.tar $(dist.tgz) @cd $(teaish__dist.tmp.tgz) && tar czf ../$(dist.tgz) $(tx.dist.basename) @rm -fr $(teaish__dist.tmp.tgz) @ls -la $(dist.tgz) dist.tgz-core: $(dist.tgz) dist.tgz-post: dist.tgz-core dist.tgz: dist.tgz-post dist: dist.tgz undist-tgz: rm -f $(dist.tgz) undist: undist-tgz @else #!BIN_TAR dist: @echo "The dist rules require tar, which configure did not find." 1>&2; exit 1 @endif #BIN_TAR @else #!TEAISH_ENABLE_DIST undist: dist: @if TEAISH_OUT_OF_EXT_TREE @echo "'dist' can only be used from an extension's home dir" 1>&2; \ echo "In this case: @TEAISH_EXT_DIR@" 1>&2; exit 1 @endif @endif #TEAISH_ENABLE_DIST Makefile: @TEAISH_TCL@ @if TEAISH_MAKEFILE_CODE # # TEAISH_MAKEFILE_CODE may contain literal makefile code, which # gets pasted verbatim here. Either [define TEAISH_MAKEFILE_CODE # ...] or use [teaish-make-add] to incrementally build up this # content. # # <TEAISH_MAKEFILE_CODE> @TEAISH_MAKEFILE_CODE@ # </TEAISH_MAKEFILE_CODE> @endif @if TEAISH_MAKEFILE # # TEAISH_MAKEFILE[_IN] defines any extension-specific state this file # needs. # # It must set the following vars if they're not already accounted for # via teaish.tcl. # # - tx.src = list of the extension's source files, being sure to # prefix each with $(tx.dir) (if it's in the same dir as the # extension) so that out-of-tree builds can find them. Optionally, # [define] TEAISH_EXT_SRC or pass them to [teaish-src-add]. # # It may optionally set the following vars: # # - tx.CFLAGS = CFLAGS/CPPFLAGS. Optionally, [define] TEAISH_CFLAGS # or pass them to [teaish-cflags-add]. # # - tx.LDFLAGS = LDFLAGS. Optionally, [define] TEAISH_LDFLAGS or # pass them to [teaish-ldflags-add]. # # It may optionally hook into various targets as documented in # /doc/extensions.md in the canonical teaish source tree. # # Interestingly, we don't have to pre-filter teaish.makefile.in - we # can just @include it here. That skips its teaish-specific validation # though. Hmm. # # <TEAISH_MAKEFILE> Makefile: @TEAISH_MAKEFILE@ @include @TEAISH_MAKEFILE@ # </TEAISH_MAKEFILE> @endif |
Deleted autoconf/tea/README.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Added autoconf/tea/README.txt.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | This is the SQLite extension for Tcl using something akin to the Tcl Extension Architecture (TEA). To build it: ./configure ...flags... e.g.: ./configure --with-tcl=/path/to/tcl/install/root or: ./configure --with-tclsh=/path/to/tcl/install/root Run ./configure --help for the full list of flags. The configuration process will fail if tclConfig.sh cannot be found. The makefile will only honor CFLAGS and CPPFLAGS passed to the configure script, not those directly passed to the makefile. Then: make test install ----------------------- THE PREFERRED WAY --------------------------- The preferred way to build the TCL extension for SQLite is to use the canonical source code tarball. For Unix: ./configure --with-tclsh=$(TCLSH) make tclextension-install For Windows: nmake /f Makefile.msc tclextension-install TCLSH_CMD=$(TCLSH) In both of the above, replace $(TCLSH) with the full pathname of of the tclsh that you want the SQLite extension to work with. See step-by-step instructions at the links below for more information: https://sqlite.org/src/doc/trunk/doc/compile-for-unix.md https://sqlite.org/src/doc/trunk/doc/compile-for-windows.md And info about the extension's Tcl interface can be found at: https://sqlite.org/tclsqlite.html The whole point of the amalgamation-autoconf tarball (in which this README.txt file is embedded) is to provide a means of compiling SQLite that does not require first installing TCL and/or "tclsh". The canonical Makefile in the SQLite source tree provides more capabilities (such as the the ability to run test cases to ensure that the build worked) and is better maintained. The only downside of the canonical Makefile is that it requires a TCL installation. But if you are wanting to build the TCL extension for SQLite, then presumably you already have a TCL installation. So why not just use the more-capable and better-maintained canoncal Makefile? As of version 3.50.0, this build process uses "teaish": https://fossil.wanderinghorse.net/r/teaish which is conceptually derived from the pre-3.50 toolchain, TEA: http://core.tcl-lang.org/tclconfig http://core.tcl-lang.org/sampleextension It to works for us. It might also work for you. But we cannot promise that. If you want to use this TEA builder and it works for you, that's fine. But if you have trouble, the first thing you should do is go back to using the canonical Makefile in the SQLite source tree. ------------------------------------------------------------------ UNIX BUILD ========== Building under most UNIX systems is easy, just run the configure script and then run make. For example: $ cd sqlite-*-tea $ ./configure --with-tcl=/path/to/tcl/install/root $ make test $ make install WINDOWS BUILD ============= On Windows this build is known to work on Cygwin and some Msys2 environments. We do not currently support Microsoft makefiles for native Windows builds. |
Added autoconf/tea/_teaish.tester.tcl.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 | # -*- tcl -*- # # Unless this file is named _teaish.tester.tcl.in, you are probably # looking at an automatically generated/filtered copy and should # probably not edit it. # # This is the wrapper script invoked by teaish's "make test" recipe. # It gets passed 3 args: # # $1 = the DLL name, or "" if the extension has no DLL # # $2 = the "load prefix" for Tcl's [load] or empty if $1 is empty # # $3 = the /path/to/teaish/tester.tcl (test utility code) # @if TEAISH_VSATISFIES_CODE @TEAISH_VSATISFIES_CODE@ @endif if {[llength [lindex $::argv 0]] > 0} { load [file normalize [lindex $::argv 0]] [lindex $::argv 1]; # ----^^^^^^^ needed on Haiku when argv 0 is just a filename, else # load cannot find the file. } set ::argv [lassign $argv - -] source -encoding utf-8 [lindex $::argv 0]; # teaish/tester.tcl @if TEAISH_PKGINIT_TCL apply {{file} { set dir [file dirname $::argv0] source -encoding utf-8 $file }} [join {@TEAISH_PKGINIT_TCL@}] @endif @if TEAISH_TM_TCL apply {{file} { set dir [file dirname $::argv0] source -encoding utf-8 $file }} [join {@TEAISH_TM_TCL@}] @endif @if TEAISH_TEST_TCL apply {{file} { # Populate state for [tester.tcl::teaish-build-flag*] array set ::teaish__BuildFlags @TEAISH__DEFINES_MAP@ set dir [file normalize [file dirname $file]] #test-fail "Just testing" source -encoding utf-8 $file }} [join {@TEAISH_TEST_TCL@}] @else # TEAISH_TEST_TCL # No $TEAISH_TEST_TCL provided, so here's a default test which simply # loads the extension. puts {Extension @TEAISH_NAME@ @TEAISH_VERSION@ successfully loaded from @TEAISH_TESTER_TCL@} @endif |
Deleted autoconf/tea/aclocal.m4.
|
| < < < < < < < < < |
Added autoconf/tea/auto.def.
> > > > > > > > | 1 2 3 4 5 6 7 8 | #/do/not/tclsh # ^^^ help out editors which guess this file's content type. # # Main configure script entry point for the TEA(ish) framework. All # extension-specific customization goes in teaish.tcl.in or # teaish.tcl. use teaish/core teaish-configure-core |
Added autoconf/tea/configure.
> > > > > > > | 1 2 3 4 5 6 7 | #!/bin/sh dir0="`dirname "$0"`" dirA="$dir0/../autosetup" # This is the case ^^^^^^^^^^^^ in the SQLite "autoconf" bundle. WRAPPER="$0"; export WRAPPER; exec "`"$dirA/autosetup-find-tclsh"`" \ "$dirA/autosetup" --teaish-extension-dir="$dir0" \ "$@" |
Deleted autoconf/tea/configure.ac.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted autoconf/tea/doc/sqlite3.n.
|
| < < < < < < < < < < < < < < < |
Changes to autoconf/tea/pkgIndex.tcl.in.
1 | # -*- tcl -*- | | > > > > > > > > | > | > > > > > > > > | > > | > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 | # -*- tcl -*- # Tcl package index file # # Unless this file is named pkgIndex.tcl.in, you are probably looking # at an automatically generated/filtered copy and should probably not # edit it. # # Adapted from https://core.tcl-lang.org/tcltls @if TEAISH_VSATISFIES_CODE @TEAISH_VSATISFIES_CODE@ @endif if {[package vsatisfies [package provide Tcl] 9.0-]} { package ifneeded {@TEAISH_PKGNAME@} {@TEAISH_VERSION@} [list apply {{dir} { @if TEAISH_ENABLE_DLL load [file join $dir {@TEAISH_DLL9@}] @TEAISH_LOAD_PREFIX@ @endif @if TEAISH_PKGINIT_TCL_TAIL set initScript [file join $dir {@TEAISH_PKGINIT_TCL_TAIL@}] if {[file exists $initScript]} { source -encoding utf-8 $initScript } @endif }} $dir] } else { package ifneeded {@TEAISH_PKGNAME@} {@TEAISH_VERSION@} [list apply {{dir} { @if TEAISH_ENABLE_DLL if {[string tolower [file extension {@TEAISH_DLL8@}]] in [list .dll .dylib .so]} { load [file join $dir {@TEAISH_DLL8@}] @TEAISH_LOAD_PREFIX@ } else { load {} @TEAISH_LOAD_PREFIX@ } @endif @if TEAISH_PKGINIT_TCL_TAIL set initScript [file join $dir {@TEAISH_PKGINIT_TCL_TAIL@}] if {[file exists $initScript]} { source -encoding utf-8 $initScript } @endif }} $dir] } |
Deleted autoconf/tea/tclconfig/install-sh.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted autoconf/tea/tclconfig/tcl.m4.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Added autoconf/tea/teaish.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 | # Teaish configure script for the SQLite Tcl extension # # State for disparate config-time pieces. # array set sqlite__Config [proj-strip-hash-comments { # # The list of feature --flags which the --all flag implies. This # requires special handling in a few places. # all-flag-enables {fts3 fts4 fts5 rtree geopoly} # >0 if building in the canonical tree. -1=undetermined is-canonical -1 }] # # Set up the package info for teaish... # apply {{dir} { # Figure out the version number... set version "" if {[file exists $dir/../VERSION]} { # The canonical SQLite TEA(ish) build set version [proj-file-content -trim $dir/../VERSION] set ::sqlite__Config(is-canonical) 1 set distname sqlite-tcl } elseif {[file exists $dir/generic/tclsqlite3.c]} { # The copy from the teaish tree, used as a dev/test bed before # updating SQLite's tree. set ::sqlite__Config(is-canonical) 0 set fd [open $dir/generic/tclsqlite3.c rb] while {[gets $fd line] >=0} { if {[regexp {^#define[ ]+SQLITE_VERSION[ ]+"(3.+)"} \ $line - version]} { set distname sqlite-teaish break } } close $fd } if {"" eq $version} { proj-fatal "Cannot determine the SQLite version number" } proj-assert {$::sqlite__Config(is-canonical) > -1} proj-assert {[string match 3.*.* $version]} \ "Unexpected SQLite version: $version" set pragmas {} if {$::sqlite__Config(is-canonical)} { # Disable "make dist" in the canonical tree. That tree is # generated from several pieces and creating/testing working # "dist" rules for that sub-build currently feels unnecessary. The # copy in the teaish tree, though, should be able to "make dist". lappend pragmas no-dist } else { lappend pragmas full-dist } teaish-pkginfo-set -vars { -name sqlite -name.pkg sqlite3 -version $version -name.dist $distname -libDir sqlite$version -pragmas $pragmas -src generic/tclsqlite3.c } # We should also have: # -vsatisfies 8.6- # But at least one platform is failing this vsatisfies check # for no apparent reason: # https://sqlite.org/forum/forumpost/fde857fb8101a4be }} [teaish-get -dir] # # Must return either an empty string or a list in the form accepted by # autosetup's [options] function. # proc teaish-options {} { # These flags and defaults mostly derive from the historical TEA # build. Some, like ICU, are taken from the canonical SQLite tree. return [subst -nocommands -nobackslashes { with-system-sqlite=0 => {Use the system-level SQLite instead of the copy in this tree. Also requires use of --override-sqlite-version so that the build knows what version number to associate with the system-level SQLite.} override-sqlite-version:VERSION => {For use with --with-system-sqlite to set the version number.} threadsafe=1 => {Disable mutexing} with-tempstore:=no => {Use an in-RAM database for temporary tables: never,no,yes,always} load-extension=0 => {Enable loading of external extensions} math=1 => {Disable math functions} json=1 => {Disable JSON functions} fts3 => {Enable the FTS3 extension} fts4 => {Enable the FTS4 extension} fts5 => {Enable the FTS5 extension} update-limit => {Enable the UPDATE/DELETE LIMIT clause} geopoly => {Enable the GEOPOLY extension} rtree => {Enable the RTREE extension} session => {Enable the SESSION extension} all=1 => {Disable $::sqlite__Config(all-flag-enables)} with-icu-ldflags:LDFLAGS => {Enable SQLITE_ENABLE_ICU and add the given linker flags for the ICU libraries. e.g. on Ubuntu systems, try '-licui18n -licuuc -licudata'.} with-icu-cflags:CFLAGS => {Apply extra CFLAGS/CPPFLAGS necessary for building with ICU. e.g. -I/usr/local/include} with-icu-config:=auto => {Enable SQLITE_ENABLE_ICU. Value must be one of: auto, pkg-config, /path/to/icu-config} icu-collations=0 => {Enable SQLITE_ENABLE_ICU_COLLATIONS. Requires --with-icu-ldflags=... or --with-icu-config} }] } # # Gets called by tea-configure-core. Must perform any configuration # work needed for this extension. # proc teaish-configure {} { use teaish/feature if {[proj-opt-was-provided override-sqlite-version]} { teaish-pkginfo-set -version [opt-val override-sqlite-version] proj-warn "overriding sqlite version number:" [teaish-pkginfo-get -version] } elseif {[proj-opt-was-provided with-system-sqlite] && [opt-val with-system-sqlite] ne "0"} { proj-fatal "when using --with-system-sqlite also use" \ "--override-sqlite-version to specify a library version number." } define CFLAGS [proj-get-env CFLAGS {-O2}] sqlite-munge-cflags # # Add feature flags from legacy configure.ac which are not covered by # --flags. # sqlite-add-feature-flag { -DSQLITE_3_SUFFIX_ONLY=1 -DSQLITE_ENABLE_DESERIALIZE=1 -DSQLITE_ENABLE_DBPAGE_VTAB=1 -DSQLITE_ENABLE_BYTECODE_VTAB=1 -DSQLITE_ENABLE_DBSTAT_VTAB=1 } if {[opt-bool with-system-sqlite]} { msg-result "Using system-level sqlite3." teaish-cflags-add -DUSE_SYSTEM_SQLITE teaish-ldflags-add -lsqlite3 } elseif {$::sqlite__Config(is-canonical)} { teaish-cflags-add -I[teaish-get -dir]/.. } teaish-check-librt teaish-check-libz sqlite-handle-threadsafe sqlite-handle-tempstore sqlite-handle-load-extension sqlite-handle-math sqlite-handle-icu sqlite-handle-common-feature-flags; # must be late in the process }; # teaish-configure define OPT_FEATURE_FLAGS {} ; # -DSQLITE_OMIT/ENABLE flags. # # Adds $args, if not empty, to OPT_FEATURE_FLAGS. This is intended only for holding # -DSQLITE_ENABLE/OMIT/... flags, but that is not enforced here. proc sqlite-add-feature-flag {args} { if {"" ne $args} { define-append OPT_FEATURE_FLAGS {*}$args } } # # Check for log(3) in libm and die with an error if it is not # found. $featureName should be the feature name which requires that # function (it's used only in error messages). defines LDFLAGS_MATH to # the required linker flags (which may be empty even if the math APIs # are found, depending on the OS). proc sqlite-affirm-have-math {featureName} { if {"" eq [get-define LDFLAGS_MATH ""]} { if {![msg-quiet proj-check-function-in-lib log m]} { user-error "Missing math APIs for $featureName" } set lfl [get-define lib_log ""] undefine lib_log if {"" ne $lfl} { user-notice "Forcing requirement of $lfl for $featureName" } define LDFLAGS_MATH $lfl teaish-ldflags-prepend $lfl } } # # Handle various SQLITE_ENABLE/OMIT_... feature flags. proc sqlite-handle-common-feature-flags {} { msg-result "Feature flags..." if {![opt-bool all]} { # Special handling for --disable-all foreach flag $::sqlite__Config(all-flag-enables) { if {![proj-opt-was-provided $flag]} { proj-opt-set $flag 0 } } } foreach {boolFlag featureFlag ifSetEvalThis} [proj-strip-hash-comments { all {} { # The 'all' option must be first in this list. This impl makes # an effort to only apply flags which the user did not already # apply, so that combinations like (--all --disable-geopoly) # will indeed disable geopoly. There are corner cases where # flags which depend on each other will behave in non-intuitive # ways: # # --all --disable-rtree # # Will NOT disable geopoly, though geopoly depends on rtree. # The --geopoly flag, though, will automatically re-enable # --rtree, so --disable-rtree won't actually disable anything in # that case. foreach k $::sqlite__Config(all-flag-enables) { if {![proj-opt-was-provided $k]} { proj-opt-set $k 1 } } } fts3 -DSQLITE_ENABLE_FTS3 {sqlite-affirm-have-math fts3} fts4 -DSQLITE_ENABLE_FTS4 {sqlite-affirm-have-math fts4} fts5 -DSQLITE_ENABLE_FTS5 {sqlite-affirm-have-math fts5} geopoly -DSQLITE_ENABLE_GEOPOLY {proj-opt-set rtree} rtree -DSQLITE_ENABLE_RTREE {} session {-DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_PREUPDATE_HOOK} {} update-limit -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT {} scanstatus -DSQLITE_ENABLE_STMT_SCANSTATUS {} }] { if {$boolFlag ni $::autosetup(options)} { # Skip flags which are in the canonical build but not # the autoconf bundle. continue } proj-if-opt-truthy $boolFlag { sqlite-add-feature-flag $featureFlag if {0 != [eval $ifSetEvalThis] && "all" ne $boolFlag} { msg-result " + $boolFlag" } } { if {"all" ne $boolFlag} { msg-result " - $boolFlag" } } } # # Invert the above loop's logic for some SQLITE_OMIT_... cases. If # config option $boolFlag is false, [sqlite-add-feature-flag # $featureFlag], where $featureFlag is intended to be # -DSQLITE_OMIT_... foreach {boolFlag featureFlag} { json -DSQLITE_OMIT_JSON } { if {[proj-opt-truthy $boolFlag]} { msg-result " + $boolFlag" } else { sqlite-add-feature-flag $featureFlag msg-result " - $boolFlag" } } ## # Remove duplicates from the final feature flag sets and show them # to the user. set oFF [get-define OPT_FEATURE_FLAGS] if {"" ne $oFF} { define OPT_FEATURE_FLAGS [lsort -unique $oFF] msg-result "Library feature flags: [get-define OPT_FEATURE_FLAGS]" } if {[lsearch [get-define TARGET_DEBUG ""] -DSQLITE_DEBUG=1] > -1} { msg-result "Note: this is a debug build, so performance will suffer." } teaish-cflags-add -define OPT_FEATURE_FLAGS }; # sqlite-handle-common-feature-flags # # If --enable-threadsafe is set, this adds -DSQLITE_THREADSAFE=1 to # OPT_FEATURE_FLAGS and sets LDFLAGS_PTHREAD to the linker flags # needed for linking pthread (possibly an empty string). If # --enable-threadsafe is not set, adds -DSQLITE_THREADSAFE=0 to # OPT_FEATURE_FLAGS and sets LDFLAGS_PTHREAD to an empty string. # # It prepends the flags to the global LDFLAGS. proc sqlite-handle-threadsafe {} { msg-checking "Support threadsafe operation? " define LDFLAGS_PTHREAD "" set enable 0 if {[proj-opt-was-provided threadsafe]} { proj-if-opt-truthy threadsafe { if {[proj-check-function-in-lib pthread_create pthread] && [proj-check-function-in-lib pthread_mutexattr_init pthread]} { incr enable set ldf [get-define lib_pthread_create] define LDFLAGS_PTHREAD $ldf teaish-ldflags-prepend $ldf undefine lib_pthread_create undefine lib_pthread_mutexattr_init } else { user-error "Missing required pthread libraries. Use --disable-threadsafe to disable this check." } # Recall that LDFLAGS_PTHREAD might be empty even if pthreads if # found because it's in -lc on some platforms. } { msg-result "Disabled using --disable-threadsafe" } } else { # # If user does not specify --[disable-]threadsafe then select a # default based on whether it looks like Tcl has threading # support. # catch { scan [exec echo {puts [tcl::pkgconfig get threaded]} | [get-define TCLSH_CMD]] \ %d enable } if {$enable} { set flagName "--threadsafe" set lblAbled "enabled" msg-result yes } else { set flagName "--disable-threadsafe" set lblAbled "disabled" msg-result no } msg-result "Defaulting to ${flagName} because Tcl has threading ${lblAbled}." # ^^^ We (probably) don't need to link against -lpthread in the # is-enabled case. We might in the case of static linking. Unsure. } sqlite-add-feature-flag -DSQLITE_THREADSAFE=${enable} return $enable } # # Handles the --enable-load-extension flag. Returns 1 if the support # is enabled, else 0. If support for that feature is not found, a # fatal error is triggered if --enable-load-extension is explicitly # provided, else a loud warning is instead emitted. If # --disable-load-extension is used, no check is performed. # # Makes the following environment changes: # # - defines LDFLAGS_DLOPEN to any linker flags needed for this # feature. It may legally be empty on some systems where dlopen() # is in libc. # # - If the feature is not available, adds # -DSQLITE_OMIT_LOAD_EXTENSION=1 to the feature flags list. proc sqlite-handle-load-extension {} { define LDFLAGS_DLOPEN "" set found 0 proj-if-opt-truthy load-extension { set found [proj-check-function-in-lib dlopen dl] if {$found} { set ldf [get-define lib_dlopen] define LDFLAGS_DLOPEN $ldf teaish-ldflags-prepend $ldf undefine lib_dlopen } else { if {[proj-opt-was-provided load-extension]} { # Explicit --enable-load-extension: fail if not found proj-indented-notice -error { --enable-load-extension was provided but dlopen() not found. Use --disable-load-extension to bypass this check. } } else { # It was implicitly enabled: warn if not found proj-indented-notice { WARNING: dlopen() not found, so loadable module support will be disabled. Use --disable-load-extension to bypass this check. } } } } if {$found} { msg-result "Loadable extension support enabled." } else { msg-result "Disabling loadable extension support. Use --enable-load-extension to enable them." sqlite-add-feature-flag -DSQLITE_OMIT_LOAD_EXTENSION=1 } return $found } # # ICU - International Components for Unicode # # Handles these flags: # # --with-icu-ldflags=LDFLAGS # --with-icu-cflags=CFLAGS # --with-icu-config[=auto | pkg-config | /path/to/icu-config] # --enable-icu-collations # # --with-icu-config values: # # - auto: use the first one of (pkg-config, icu-config) found on the # system. # - pkg-config: use only pkg-config to determine flags # - /path/to/icu-config: use that to determine flags # # If --with-icu-config is used as neither pkg-config nor icu-config # are found, fail fatally. # # If both --with-icu-ldflags and --with-icu-config are provided, they # are cumulative. If neither are provided, icu-collations is not # honored and a warning is emitted if it is provided. # # Design note: though we could automatically enable ICU if the # icu-config binary or (pkg-config icu-io) are found, we specifically # do not. ICU is always an opt-in feature. proc sqlite-handle-icu {} { define LDFLAGS_LIBICU [join [opt-val with-icu-ldflags ""]] define CFLAGS_LIBICU [join [opt-val with-icu-cflags ""]] if {[proj-opt-was-provided with-icu-config]} { msg-result "Checking for ICU support..." set icuConfigBin [opt-val with-icu-config] set tryIcuConfigBin 1; # set to 0 if we end up using pkg-config if {$icuConfigBin in {auto pkg-config}} { uplevel 3 { use pkg-config } if {[pkg-config-init 0] && [pkg-config icu-io]} { # Maintenance reminder: historical docs say to use both of # (icu-io, icu-uc). icu-uc lacks a required lib and icu-io has # all of them on tested OSes. set tryIcuConfigBin 0 define LDFLAGS_LIBICU [get-define PKG_ICU_IO_LDFLAGS] define-append LDFLAGS_LIBICU [get-define PKG_ICU_IO_LIBS] define CFLAGS_LIBICU [get-define PKG_ICU_IO_CFLAGS] } elseif {"pkg-config" eq $icuConfigBin} { proj-fatal "pkg-config cannot find package icu-io" } else { proj-assert {"auto" eq $icuConfigBin} } } if {$tryIcuConfigBin} { if {"auto" eq $icuConfigBin} { set icuConfigBin [proj-first-bin-of \ /usr/local/bin/icu-config \ /usr/bin/icu-config] if {"" eq $icuConfigBin} { proj-indented-notice -error { --with-icu-config=auto cannot find (pkg-config icu-io) or icu-config binary. On Ubuntu-like systems try: --with-icu-ldflags='-licui18n -licuuc -licudata' } } } if {[file-isexec $icuConfigBin]} { set x [exec $icuConfigBin --ldflags] if {"" eq $x} { proj-indented-notice -error \ [subst { $icuConfigBin --ldflags returned no data. On Ubuntu-like systems try: --with-icu-ldflags='-licui18n -licuuc -licudata' }] } define-append LDFLAGS_LIBICU $x set x [exec $icuConfigBin --cppflags] define-append CFLAGS_LIBICU $x } else { proj-fatal "--with-icu-config=$icuConfigBin does not refer to an executable" } } } set ldflags [define LDFLAGS_LIBICU [string trim [get-define LDFLAGS_LIBICU]]] set cflags [define CFLAGS_LIBICU [string trim [get-define CFLAGS_LIBICU]]] if {"" ne $ldflags} { sqlite-add-feature-flag -DSQLITE_ENABLE_ICU msg-result "Enabling ICU support with flags: $ldflags $cflags" if {[opt-bool icu-collations]} { msg-result "Enabling ICU collations." sqlite-add-feature-flag -DSQLITE_ENABLE_ICU_COLLATIONS } teaish-ldflags-prepend $ldflags teaish-cflags-add $cflags } elseif {[opt-bool icu-collations]} { proj-warn "ignoring --enable-icu-collations because neither --with-icu-ldflags nor --with-icu-config provided any linker flags" } else { msg-result "ICU support is disabled." } }; # sqlite-handle-icu # # Handles the --with-tempstore flag. # # The test fixture likes to set SQLITE_TEMP_STORE on its own, so do # not set that feature flag unless it was explicitly provided to the # configure script. proc sqlite-handle-tempstore {} { if {[proj-opt-was-provided with-tempstore]} { set ts [opt-val with-tempstore no] set tsn 1 msg-checking "Use an in-RAM database for temporary tables? " switch -exact -- $ts { never { set tsn 0 } no { set tsn 1 } yes { set tsn 2 } always { set tsn 3 } default { user-error "Invalid --with-tempstore value '$ts'. Use one of: never, no, yes, always" } } msg-result $ts sqlite-add-feature-flag -DSQLITE_TEMP_STORE=$tsn } } # # Handles the --enable-math flag. proc sqlite-handle-math {} { proj-if-opt-truthy math { if {![proj-check-function-in-lib ceil m]} { user-error "Cannot find libm functions. Use --disable-math to bypass this." } set lfl [get-define lib_ceil] undefine lib_ceil define LDFLAGS_MATH $lfl teaish-ldflags-prepend $lfl sqlite-add-feature-flag -DSQLITE_ENABLE_MATH_FUNCTIONS msg-result "Enabling math SQL functions" } { define LDFLAGS_MATH "" msg-result "Disabling math SQL functions" } } # # Move -DSQLITE_OMIT... and -DSQLITE_ENABLE... flags from CFLAGS and # CPPFLAGS to OPT_FEATURE_FLAGS and remove them from BUILD_CFLAGS. proc sqlite-munge-cflags {} { # Move CFLAGS and CPPFLAGS entries matching -DSQLITE_OMIT* and # -DSQLITE_ENABLE* to OPT_FEATURE_FLAGS. This behavior is derived # from the pre-3.48 build. # # If any configure flags for features are in conflict with # CFLAGS/CPPFLAGS-specified feature flags, all bets are off. There # are no guarantees about which one will take precedence. foreach flagDef {CFLAGS CPPFLAGS} { set tmp "" foreach cf [get-define $flagDef ""] { switch -glob -- $cf { -DSQLITE_OMIT* - -DSQLITE_ENABLE* { sqlite-add-feature-flag $cf } default { lappend tmp $cf } } } define $flagDef $tmp } } |
Added autoconf/tea/teaish.test.tcl.
> > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 | test-expect 1.0-open { sqlite3 db :memory: } {} test-assert 1.1-version-3.x { [string match 3.* [db eval {select sqlite_version()}]] } test-expect 1.2-select { db eval {select 'hi, world',1,2,3} } {{hi, world} 1 2 3} test-expect 99.0-db-close {db close} {} |
Deleted autoconf/tea/win/makefile.vc.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted autoconf/tea/win/nmakehlp.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted autoconf/tea/win/rules.vc.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Added autosetup/LICENSE.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | Unless explicitly stated, all files which form part of autosetup are released under the following license: --------------------------------------------------------------------- autosetup - A build environment "autoconfigurator" Copyright (c) 2010-2011, WorkWare Systems <http://workware.net.au/> Redistribution and use in source and binary forms, with or without modification, are permitted provided that the following conditions are met: 1. Redistributions of source code must retain the above copyright notice, this list of conditions and the following disclaimer. 2. Redistributions in binary form must reproduce the above copyright notice, this list of conditions and the following disclaimer in the documentation and/or other materials provided with the distribution. THIS SOFTWARE IS PROVIDED BY THE WORKWARE SYSTEMS ``AS IS'' AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE ARE DISCLAIMED. IN NO EVENT SHALL WORKWARE SYSTEMS OR CONTRIBUTORS BE LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR BUSINESS INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER IN CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR OTHERWISE) ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED OF THE POSSIBILITY OF SUCH DAMAGE. The views and conclusions contained in the software and documentation are those of the authors and should not be interpreted as representing official policies, either expressed or implied, of WorkWare Systems. |
Added autosetup/README.autosetup.
> > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 | README.autosetup created by autosetup v0.7.2 This is the autosetup directory for a local install of autosetup. It contains autosetup, support files and loadable modules. *.tcl files in this directory are optional modules which can be loaded with the 'use' directive. *.auto files in this directory are auto-loaded. For more information, see https://msteveb.github.io/autosetup/ |
Added autosetup/README.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 | Maintaining Autosetup in the SQLite Tree ======================================================================== This document provides some tips and reminders for the SQLite developers regarding using and maintaining the [Autosetup][]-based build infrastructure. It is not an [Autosetup][] reference. **Table of Contents**: - [Autosetup API Reference](#apiref) - [API Tips](#apitips) - [Ensuring TCL Compatibility](#tclcompat) - [Design Conventions](#conventions) - Symbolic Names of Feature Flags - Do Not Update Global Shared State - [Updating Autosetup](#updating) - ***[Patching Autosetup for Project-local changes](#patching)*** - [Branch-specific Customization](#branch-customization) ------------------------------------------------------------------------ <a name="apiref"></a> Autosetup API Reference ======================================================================== The Autosetup API is quite extensive and can be read either in the [files in the `autosetup` dir](/dir/autosetup) or using: > ``` $ ./configure --reference | less ``` That will include any docs from any TCL files in the `./autosetup` dir which contain certain (simple) markup defined by autosetup. This project's own configuration-related TCL code is spread across the following files: - [proj.tcl][]: project-agnostic utility code for autosetup-driven projects. This file is designed to be shared between this project, other projects managed under the SQLite/Hwaci umbrella (e.g. Fossil), and personal projects of SQLite's developers. It is essentially an amalgamation of a decade's worth of autosetup-related utility code. - [sqlite-config.tcl][]: utility code which is too project-specific for `proj.tcl`. We split this out of `auto.def` so that it can be used by both `auto.def` and... - [auto.def][]: the primary driver for the `./configure` process. When we talk about "the configure script," we're technically referring to this file, though it actually contains very little of the TCL code. - [autoconf/auto.def][]: the main driver script for the "autoconf" bundle's configure script. It is essentially a slightly trimmed-down version of the main `auto.def` file. The `autoconf` dir was ported from the Autotools to Autosetup in the 3.49.0 dev cycle but retains the "autoconf" name to minimize downstream disruption. <a name="apitips"></a> Autosetup API Tips ======================================================================== This section briefly covers only APIs which are frequently useful in day-to-day maintenance and might not be immediately recognized as such from a casual perusal of the relevant TCL files. The complete docs of those with `proj-` prefix can be found in [proj.tcl][] and those with an `sqlite-` prefix are in [sqlite-config.tcl][]. The others are part of Autosetup's core packages and are scattered around [the TCL files in ./autosetup](/dir/autosetup). In (mostly) alphabetical order: - **`file-isexec filename`**\ Should be used in place of `[file executable]`, as it will also check for `${filename}.exe` on Windows platforms. However, on such platforms it also assumes that _any_ existing file is executable. - **`get-env VAR ?default?`**\ Will fetch an "environment variable" from the first of either: (1) a KEY=VALUE passed to the configure script or (2) the system's environment variables. Not to be confused with `getenv`, which only does the latter and is rarely, if ever, useful in this tree. - **`proj-get-env VAR ?default?`**\ Works like `get-env` but will, if that function finds no match, look for a file named `./.env-$VAR` and, if found, return its trimmed contents. This can be used, e.g., to set a developer's local preferences for the default `CFLAGS`.\ Tip: adding `-O0` to `.env-CFLAGS` reduces rebuild times considerably at the cost of performance in `make devtest` and the like. - **`proj-fatal msg`**\ Emits `$msg` to stderr and exits with non-zero. Its differences from autosetup's `user-error` are purely cosmetic. - **`proj-if-opt-truthy flag thenScript ?elseScript?`**\ Evals `thenScript` if the given `--flag` is truthy, else it evals the optional `elseScript`. - **`proj-indented-notice ?-error? ?-notice? msg`**\ Breaks its `msg` argument into lines, trims them, and emits them with consistent indentation. Exactly how it emits depends on the flags passed to it (or not), as covered in its docs. This will stick out starkly from normal output and is intended to be used only for important notices. - **`proj-opt-truthy flag`**\ Returns 1 if `--flag`'s value is "truthy," i.e. one of (1, on, enabled, yes, true). - **`proj-opt-was-provided FLAG`**\ Returns 1 if `--FLAG` was explicitly provided to configure, else 0. This distinction can be used to determine, e.g., whether `--with-readline` was provided or whether we're searching for readline by default. In the former case, failure to find it should be treated as fatal, where in the latter case it's not.\ Unlike most functions which deal with `--flags`, this one does not validate that `$FLAG` is a registered flag so will not fail fatally if `$FLAG` is not registered as an Autosetup option. - **`proj-val-truthy value`**\ Returns 1 if `$value` is "truthy," See `proj-opt-truthy` for the definition of "truthy." - **`proj-warn msg`**\ Emits `$msg` to stderr. Closely-related is autosetup's `user-notice` (described below). - **`sqlite-add-feature-flag ?-shell? FLAG...`**\ Adds the given feature flag to the CFLAGS which are specific to building libsqlite3. It's intended to be passed one or more `-DSQLITE_ENABLE_...`, or similar, flags. If the `-shell` flag is used then it also passes its arguments to `sqlite-add-shell-opt`. This is a no-op if `FLAG` is not provided or is empty. - **`sqlite-add-shell-opt FLAG...`**\ The shell-specific counterpart of `sqlite-add-feature-flag` which only adds the given flag(s) to the CLI-shell-specific CFLAGS. - **`sqlite-configure BUILD-NAME {script}`**\ This is where all configure `--flags` are defined for all known build modes ("canonical" or "autoconf"). After processing all flags, this function runs `$script`, which contains the build-mode-specific configuration bits, and then runs any finalization bits which are common to all build modes. The `auto.def` files are intended to contain exactly two commands: `use sqlite-config; sqlite-configure BUILD-NAME {script}` - **`user-notice msg`**\ Queues `$msg` to be sent to stderr, but does not emit it until either `show-notices` is called or the next time autosetup would output something (it internally calls `show-notices`). This can be used to generate warnings between a "checking for..." message and its resulting "yes/no/whatever" message in such a way as to not spoil the layout of such messages. <a name="tclcompat"></a> Ensuring TCL Compatibility ======================================================================== One of the significant benefits of using Autosetup is that (A) this project uses many TCL scripts in the build process and (B) Autosetup comes with a TCL interpreter named [JimTCL][]. It is important that any TCL files used by the configure process and makefiles remain compatible with both [JimTCL][] and the canonical TCL. Though JimTCL has outstanding compatibility with canonical TCL, it does have a few corners with incompatibilities, e.g. regular expressions. If a script runs in JimTCL without using any JimTCL-specific features, then it's a certainty that it will run in canonical TCL as well. The opposite, however, is not _always_ the case. When [`./configure`](/file/configure) is run, it goes through a bootstrapping process to find a suitable TCL with which to run the autosetup framework. The first step involves [finding or building a TCL shell](/file/autosetup/autosetup-find-tclsh). That will first search for an available `tclsh` (under several common names, e.g. `tclsh8.6`) before falling back to compiling the copy of `jimsh0.c` included in the source tree. i.e. it will prefer to use a system-installed TCL for running the configure script. Once it finds (or builds) a TCL shell, it then runs [a sanity test to ensure that the shell is suitable](/file/autosetup/autosetup-test-tclsh) before using it to run the main autosetup app. There are two simple ways to ensure that running of the configure process uses JimTCL instead of the canonical `tclsh`, and either approach provides equally high assurances about configure script compatibility across TCL implementations: 1. Build on a system with no `tclsh` installed in the `$PATH`. In that case, the configure process will fall back to building the in-tree copy of JimTCL. 2. Manually build `./jimsh0` in the top of the checkout with:\ `cc -o jimsh0 autosetup/jimsh0.c`\ With that in place, the configure script will prefer to use that before looking for a system-level `tclsh`. Be aware, though, that `make distclean` will remove that file. **Note that `./jimsh0` is distinctly different from the `./jimsh`** which gets built for code-generation purposes. The latter requires non-default build flags to enable features which are platform-dependent, most notably to make its `[file normalize]` work. This means, for example, that the configure script and its utility APIs must not use `[file normalize]`, but autosetup provides a TCL-only implementation of `[file-normalize]` (note the dash) for portable use in the configure script. Contrariwise, code-generation scripts invoked via `make` may use `[file normalize]`, as they'll use `./jimsh` or `tclsh` instead of `./jimsh0`. Known TCL Incompatibilities ------------------------------------------------------------------------ A summary of known incompatibilities in JimTCL - **CRNL line endings**: prior to 2025-02-05 `fconfigure -translation ...` was a no-op in JimTCL, and it emits CRNL line endings by default on Windows. Since then, it supports `-translation binary`, which is close enough to `-translation lf` for our purposes. When working with files using the `open` command, it is important to use mode `"rb"` or `"wb"`, as appropriate, so that the output does not get CRNL-mangled on Windows. - **`file copy`** does not support multiple source files. See [](/info/61f18c96183867fe) for a workaround. - **Regular expressions**: - Patterns treat `\nnn` octal values as back-references (which it does not support). Those can be reformulated as demonstrated in [](/info/aeac23359bb681c0). - `regsub` does not support the `\y` flag. A workaround is demonstrated in [](/info/c2e5dd791cce3ec4). <a name="conventions"></a> Design Conventions ======================================================================== This section describes the motivations for the most glaring of the build's design decisions, in particular how they deviate from historical, or even widely-conventional, practices. Symbolic Names of Feature Flags ------------------------------------------------------------------------ Historically, the project's makefile has exclusively used `UPPER_UNDERSCORE` form for makefile variables. This build, however, primarily uses `X.y` format, where `X` is often a category label, e.g. `CFLAGS`, and `y` is the specific instance of that category, e.g. `CFLAGS.readline`. When the configure script exports flags for consumption by filtered files, e.g. [Makefile.in][] and the generated `sqlite_cfg.h`, it does so in the more conventional `X_Y` form because those flags get exported as as C `#define`s to `sqlite_cfg.h`, where dots are not permitted. The `X.y` convention is used in the makefiles primarily because the person who did the initial port finds that considerably easier on the eyes and fingers. In practice, the `X_Y` form of such exports is used exactly once in [Makefile.in][], where it's translated from `@X_Y@` into into `X.y` form for consumption by [Makefile.in][] and [main.mk][]. For example: > ``` LDFLAGS.shobj = @SHOBJ_LDFLAGS@ LDFLAGS.zlib = @LDFLAGS_ZLIB@ LDFLAGS.math = @LDFLAGS_MATH@ ``` (That first one is defined by autosetup, and thus applies "LDFLAGS" as the suffix rather than the prefix. Which is more legible is a matter of taste, for which there is no accounting.) Do Not Update Global Shared State ------------------------------------------------------------------------ In both the legacy Autotools-driven build and common Autosetup usage, feature tests performed by the configure script may amend global flags such as `LIBS`, `LDFLAGS`, and `CFLAGS`[^as-cflags]. That's appropriate for a makefile which builds a single deliverable, but less so for makefiles which produce multiple deliverables. Drawbacks of that approach include: - It's unlikely that every single deliverable will require the same core set of those flags. - It can be difficult to determine the origin of any given change to that global state because those changes are hidden behind voodoo performed outside the immediate visibility of the configure script's maintainer. - It can force the maintainers of the configure script to place tests in a specific order so that the resulting flags get applied at the correct time and/or in the correct order.\ (A real-life example: before the approach described below was taken to collecting build-time flags, the test for `-rpath` had to come _after_ the test for zlib because the results of the `-rpath` test implicitly modified global state which broke the zlib feature test. Because the feature tests no longer (intentionally) modify shared global state, that is not an issue.) In this build, cases where feature tests modify global state in such a way that it may impact later feature tests are either (A) very intentionally defined to do so (e.g. the `--with-wasi-sdk` flag has invasive side-effects) or (B) are oversights (i.e. bugs). This tree's [configure script][auto.def], [utility APIs][proj.tcl], [Makefile.in][], and [main.mk][] therefore strive to separate the results of any given feature test into its own well-defined variables. For example: - The linker flags for zlib are exported from the configure script as `LDFLAGS_ZLIB`, which [Makefile.in][] and [main.mk][] then expose as `LDFLAGS.zlib`. - `CFLAGS_READLINE` (a.k.a. `CFLAGS.readline`) contains the `CFLAGS` needed for including `libreadline`, `libedit`, or `linenoise`, and `LDFLAGS_READLINE` (a.k.a. `LDFLAGS.readline`) is its link-time counterpart. It is then up to the Makefile to apply and order the flags however is appropriate. At the end of the configure script, the global `CFLAGS` _ideally_ holds only flags which are either relevant to all targets or, failing that, will have no unintended side-effects on any targets. That said: clients frequently pass custom `CFLAGS` to `./configure` or `make` to set library-level feature toggles, e.g. `-DSQLITE_OMIT_FOO`, in which case there is no practical way to avoid "polluting" the builds of arbitrary makefile targets with those. _C'est la vie._ [^as-cflags]: But see this article for a detailed discussion of how autosetup currently deals specifically with CFLAGS: <https://msteveb.github.io/autosetup/articles/handling-cflags/> <a name="updating"></a> Updating Autosetup ======================================================================== Updating autosetup is, more often than not, painless. It requires having a checked-out copy of [the autosetup git repository][autosetup-git]: > ``` $ git clone https://github.com/msteveb/autosetup $ cd autosetup # Or, if it's already checked out: $ git pull ``` Then, from the top-most directory of an SQLite checkout: > ``` $ /path/to/autosetup-checkout/autosetup --install . $ fossil status # show the modified files ``` Unless the upgrade made any incompatible changes (which is exceedingly rare), that's all there is to it. After that's done, **apply a patch for the change described in the following section**, test the configure process, and check it in. <a name="patching"></a> Patching Autosetup for Project-local Changes ------------------------------------------------------------------------ Autosetup reserves the flag name **`--debug`** for its own purposes, and its own special handling of `--enable-...` flags makes `--debug` an alias for `--enable-debug`. As this project has a long history of using `--enable-debug`, we patch autosetup to use the name `--autosetup-debug` in place of `--debug`. That requires (as of this writing) four small edits in [](/file/autosetup/autosetup), as demonstrated in [check-in 3296c8d3](/info/3296c8d3). If autosetup is upgraded and this patch is _not_ applied the invoking `./configure` will fail loudly because of the declaration of the `debug` flag in `auto.def` - duplicated flags are not permitted. <a name="branch-customization"></a> Branch-specific Customization ======================================================================== Certain vendor-specific branches require slight configure script customization. Rather than editing `sqlite-config.tcl` for this, which frequently leads to merge conflicts, the following approach is recommended: In the vendor-specific branch, create a file named `autosetup/sqlite-custom.tcl`. That file should contain the following content... If flag customization is required, add: > ``` proc sqlite-custom-flags {} { # If any existing --flags require different default values # then call: options-defaults { flag-name new-default-value ... } # ^^^ That will replace the default value but will not update # the --help text, which may lead to some confusion: # https://github.com/msteveb/autosetup/issues/77 return { {*} { new-flag-name => {Help text} ... } }; #see below } ``` That function must return either an empty string or a list in the form used internally by `sqlite-config.tcl:sqlite-configure`. Next, define: > ``` proc sqlite-custom-handle-flags {} { ... do any custom flag handling here ... } ``` That function, if defined, will be called relatively late in the configure process, before any filtered files are generated but after all other significant processing. [Autosetup]: https://msteveb.github.io/autosetup/ [auto.def]: /file/auto.def [autoconf/auto.def]: /file/autoconf/auto.def [autosetup-git]: https://github.com/msteveb/autosetup [proj.tcl]: /file/autosetup/proj.tcl [sqlite-config.tcl]: /file/autosetup/sqlite-config.tcl [Makefile.in]: /file/Makefile.in [main.mk]: /file/main.mk [JimTCL]: https://jim.tcl.tk |
Added autosetup/autosetup.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 | #!/bin/sh # Copyright (c) 2006-2011 WorkWare Systems http://www.workware.net.au/ # All rights reserved # vim:se syntax=tcl: # \ dir=`dirname "$0"`; exec "`$dir/autosetup-find-tclsh`" "$0" "$@" # Note that the version has a trailing + on unreleased versions set autosetup(version) 0.7.2 # Can be set to 1 to debug early-init problems set autosetup(debug) [expr {"--autosetup-debug" in $argv}] ################################################################## # # Main flow of control, option handling # proc main {argv} { global autosetup define # There are 3 potential directories involved: # 1. The directory containing autosetup (this script) # 2. The directory containing auto.def # 3. The current directory # From this we need to determine: # a. The path to this script (and related support files) # b. The path to auto.def # c. The build directory, where output files are created # This is also complicated by the fact that autosetup may # have been run via the configure wrapper ([getenv WRAPPER] is set) # Here are the rules. # a. This script is $::argv0 # => dir, prog, exe, libdir # b. auto.def is in the directory containing the configure wrapper, # otherwise it is in the current directory. # => srcdir, autodef # c. The build directory is the current directory # => builddir, [pwd] # 'misc' is needed before we can do anything, so set a temporary libdir # in case this is the development version set autosetup(libdir) [file dirname $::argv0]/lib use misc # (a) set autosetup(dir) [realdir [file dirname [realpath $::argv0]]] set autosetup(prog) [file join $autosetup(dir) [file tail $::argv0]] set autosetup(exe) [getenv WRAPPER $autosetup(prog)] if {$autosetup(installed)} { set autosetup(libdir) $autosetup(dir) } else { set autosetup(libdir) [file join $autosetup(dir) lib] } autosetup_add_dep $autosetup(prog) # (b) if {[getenv WRAPPER ""] eq ""} { # Invoked directly set autosetup(srcdir) [pwd] } else { # Invoked via the configure wrapper set autosetup(srcdir) [file-normalize [file dirname $autosetup(exe)]] } set autosetup(autodef) [relative-path $autosetup(srcdir)/auto.def] # (c) set autosetup(builddir) [pwd] set autosetup(argv) $argv set autosetup(cmdline) {} # options is a list of known options set autosetup(options) {} # optset is a dictionary of option values set by the user based on getopt set autosetup(optset) {} # optdefault is a dictionary of default values set autosetup(optdefault) {} # options-defaults is a dictionary of overrides for default values for options set autosetup(options-defaults) {} set autosetup(optionhelp) {} set autosetup(showhelp) 0 use util # Parse options use getopt # At the is point we don't know what is a valid option # We simply parse anything that looks like an option set autosetup(getopt) [getopt argv] #"=Core Options:" options-add { help:=all => "display help and options. Optional: module name, such as --help=system" licence license => "display the autosetup license" version => "display the version of autosetup" ref:=text manual:=text reference:=text => "display the autosetup command reference. 'text', 'wiki', 'asciidoc' or 'markdown'" autosetup-debug => "display debugging output as autosetup runs" install:=. => "install autosetup to the current or given directory" } if {$autosetup(installed)} { # hidden options so we can produce a nice error options-add { sysinstall:path } } else { options-add { sysinstall:path => "install standalone autosetup to the given directory (e.g.: /usr/local)" } } options-add { force init:=help => "create initial auto.def, etc. Use --init=help for known types" # Undocumented options option-checking=1 nopager quiet timing conf: } if {[opt-bool version]} { puts $autosetup(version) exit 0 } # autosetup --conf=alternate-auto.def if {[opt-str conf o]} { set autosetup(autodef) $o } # Debugging output (set this early) incr autosetup(debug) [opt-bool autosetup-debug] incr autosetup(force) [opt-bool force] incr autosetup(msg-quiet) [opt-bool quiet] incr autosetup(msg-timing) [opt-bool timing] # If the local module exists, source it now to allow for # project-local customisations if {[file exists $autosetup(libdir)/local.tcl]} { use local } # Now any auto-load modules autosetup_load_auto_modules if {[opt-str help o]} { incr autosetup(showhelp) use help autosetup_help $o } if {[opt-bool licence license]} { use help autosetup_show_license exit 0 } if {[opt-str {manual ref reference} o]} { use help autosetup_reference $o } # Allow combining --install and --init set earlyexit 0 if {[opt-str install o]} { use install autosetup_install $o incr earlyexit } if {[opt-str init o]} { use init autosetup_init $o incr earlyexit } if {$earlyexit} { exit 0 } if {[opt-str sysinstall o]} { use install autosetup_install $o 1 exit 0 } if {![file exists $autosetup(autodef)]} { # Check for invalid option first options {} user-error "No auto.def found in \"$autosetup(srcdir)\" (use [file tail $::autosetup(exe)] --init to create one)" } # Parse extra arguments into autosetup(cmdline) foreach arg $argv { if {[regexp {([^=]*)=(.*)} $arg -> n v]} { dict set autosetup(cmdline) $n $v define $n $v } else { user-error "Unexpected parameter: $arg" } } autosetup_add_dep $autosetup(autodef) # Add $argv to CONFIGURE_OPTS define-append-argv CONFIGURE_OPTS {*}$autosetup(argv) # Set up AUTOREMAKE to reconfigure with the same args define-append-argv AUTOREMAKE {*}$autosetup(exe) {*}$autosetup(argv) # Log how we were invoked configlog "Invoked as: [getenv WRAPPER $::argv0] [quote-argv $autosetup(argv)]" configlog "Tclsh: [info nameofexecutable]" # Load auto.def as module "auto.def" autosetup_load_module auto.def source $autosetup(autodef) # Could warn here if options {} was not specified show-notices if {$autosetup(debug)} { msg-result "Writing all defines to config.log" configlog "================ defines ======================" foreach n [lsort [array names define]] { configlog "define $n $define($n)" } } exit 0 } # @section Option Handling # @opt-bool ?-nodefault? option ... # # Check each of the named, boolean options and if any have been explicitly enabled # or disabled by the user, return 1 or 0 accordingly. # # If the option was specified more than once, the last value wins. # e.g. With '--enable-foo --disable-foo', '[opt-bool foo]' will return 0 # # If no value was specified by the user, returns the default value for the # first option. If '-nodefault' is given, this behaviour changes and # -1 is returned instead. # proc opt-bool {args} { set nodefault 0 if {[lindex $args 0] eq "-nodefault"} { set nodefault 1 set args [lrange $args 1 end] } option-check-names {*}$args foreach opt $args { if {[dict exists $::autosetup(optset) $opt]} { return [dict get $::autosetup(optset) $opt] } } if {$nodefault} { return -1 } # Default value is the default for the first option return [dict get $::autosetup(optdefault) [lindex $args 0]] } # @opt-val optionlist ?default=""? # # Returns a list containing all the values given for the non-boolean options in '$optionlist'. # There will be one entry in the list for each option given by the user, including if the # same option was used multiple times. # # If no options were set, '$default' is returned (exactly, not as a list). # # Note: For most use cases, 'opt-str' should be preferred. # proc opt-val {names {default ""}} { option-check-names {*}$names foreach opt $names { if {[dict exists $::autosetup(optset) $opt]} { lappend result {*}[dict get $::autosetup(optset) $opt] } } if {[info exists result]} { return $result } return $default } # @opt-str optionlist varname ?default? # # Sets '$varname' in the callers scope to the value for one of the given options. # # For the list of options given in '$optionlist', if any value is set for any option, # the option value is taken to be the *last* value of the last option (in the order given). # # If no option was given, and a default was specified with 'options-defaults', # that value is used. # # If no 'options-defaults' value was given and '$default' was given, it is used. # # If none of the above provided a value, no value is set. # # The return value depends on whether '$default' was specified. # If it was, the option value is returned. # If it was not, 1 is returns if a value was set, or 0 if not. # # Typical usage is as follows: # ## if {[opt-str {myopt altname} o]} { ## do something with $o ## } # # Or: ## define myname [opt-str {myopt altname} o "/usr/local"] # proc opt-str {names varname args} { global autosetup option-check-names {*}$names upvar $varname value if {[llength $args]} { # A default was given, so always return the string value of the option set default [lindex $args 0] set retopt 1 } else { # No default, so return 0 or 1 to indicate if a value was found set retopt 0 } foreach opt $names { if {[dict exists $::autosetup(optset) $opt]} { set result [lindex [dict get $::autosetup(optset) $opt] end] } } if {![info exists result]} { # No user-specified value. Has options-defaults been set? foreach opt $names { if {[dict exists $::autosetup(optdefault) $opt]} { set result [dict get $autosetup(optdefault) $opt] } } } if {[info exists result]} { set value $result if {$retopt} { return $value } return 1 } if {$retopt} { set value $default return $value } return 0 } proc option-check-names {args} { foreach o $args { if {$o ni $::autosetup(options)} { autosetup-error "Request for undeclared option --$o" } } } # Parse the option definition in $opts and update # ::autosetup(setoptions) and ::autosetup(optionhelp) appropriately # proc options-add {opts} { global autosetup # First weed out comment lines set realopts {} foreach line [split $opts \n] { if {![string match "#*" [string trimleft $line]]} { append realopts $line \n } } set opts $realopts for {set i 0} {$i < [llength $opts]} {incr i} { set opt [lindex $opts $i] if {[string match =* $opt]} { # This is a special heading lappend autosetup(optionhelp) [list $opt $autosetup(module)] continue } unset -nocomplain defaultvalue equal value #puts "i=$i, opt=$opt" regexp {^([^:=]*)(:)?(=)?(.*)$} $opt -> name colon equal value if {$name in $autosetup(options)} { autosetup-error "Option $name already specified" } #puts "$opt => $name $colon $equal $value" # Find the corresponding value in the user options # and set the default if necessary if {[string match "-*" $opt]} { # This is a documentation-only option, like "-C <dir>" set opthelp $opt } elseif {$colon eq ""} { # Boolean option lappend autosetup(options) $name # Check for override if {[dict exists $autosetup(options-defaults) $name]} { # A default was specified with options-defaults, so use it set value [dict get $autosetup(options-defaults) $name] } if {$value eq "1"} { set opthelp "--disable-$name" } else { set opthelp "--$name" } # Set the default if {$value eq ""} { set value 0 } set defaultvalue $value dict set autosetup(optdefault) $name $defaultvalue if {[dict exists $autosetup(getopt) $name]} { # The option was specified by the user. Look at the last value. lassign [lindex [dict get $autosetup(getopt) $name] end] type setvalue if {$type eq "str"} { # Can we convert the value to a boolean? if {$setvalue in {1 enabled yes}} { set setvalue 1 } elseif {$setvalue in {0 disabled no}} { set setvalue 0 } else { user-error "Boolean option $name given as --$name=$setvalue" } } dict set autosetup(optset) $name $setvalue #puts "Found boolean option --$name=$setvalue" } } else { # String option. lappend autosetup(options) $name if {$equal ne "="} { # Was the option given as "name:value=default"? # If so, set $value to the display name and $defaultvalue to the default # (This is the preferred way to set a default value for a string option) if {[regexp {^([^=]+)=(.*)$} $value -> value defaultvalue]} { dict set autosetup(optdefault) $name $defaultvalue } } # Maybe override the default value if {[dict exists $autosetup(options-defaults) $name]} { # A default was specified with options-defaults, so use it set defaultvalue [dict get $autosetup(options-defaults) $name] dict set autosetup(optdefault) $name $defaultvalue } elseif {![info exists defaultvalue]} { # No default value was given by value=default or options-defaults # so use the value as the default when the plain option with no # value is given (.e.g. just --opt instead of --opt=value) set defaultvalue $value } if {$equal eq "="} { # String option with optional value set opthelp "--$name?=$value?" } else { # String option with required value set opthelp "--$name=$value" } # Get the values specified by the user if {[dict exists $autosetup(getopt) $name]} { set listvalue {} foreach pair [dict get $autosetup(getopt) $name] { lassign $pair type setvalue if {$type eq "bool" && $setvalue} { if {$equal ne "="} { user-error "Option --$name requires a value" } # If given as a boolean, use the default value set setvalue $defaultvalue } lappend listvalue $setvalue } #puts "Found string option --$name=$listvalue" dict set autosetup(optset) $name $listvalue } } # Now create the help for this option if appropriate if {[lindex $opts $i+1] eq "=>"} { set desc [lindex $opts $i+2] if {[info exists defaultvalue]} { set desc [string map [list @default@ $defaultvalue] $desc] } # A multi-line description lappend autosetup(optionhelp) [list $opthelp $autosetup(module) $desc] incr i 2 } } } # @module-options optionlist # # Deprecated. Simply use 'options' from within a module. proc module-options {opts} { options $opts } proc max {a b} { expr {$a > $b ? $a : $b} } proc options-wrap-desc {text length firstprefix nextprefix initial} { set len $initial set space $firstprefix foreach word [split $text] { set word [string trim $word] if {$word == ""} { continue } if {$len && [string length $space$word] + $len >= $length} { puts "" set len 0 set space $nextprefix } incr len [string length $space$word] puts -nonewline $space$word set space " " } if {$len} { puts "" } } # Display options (from $autosetup(optionhelp)) for modules that match # glob pattern $what proc options-show {what} { set local 0 # Determine the max option width set max 0 foreach help $::autosetup(optionhelp) { lassign $help opt module desc if {![string match $what $module]} { continue } if {[string match =* $opt] || [string match \n* $desc]} { continue } set max [max $max [string length $opt]] } set indent [string repeat " " [expr {$max+4}]] set cols [getenv COLUMNS 80] catch { lassign [exec stty size] _ sttycols if {[string is integer -strict $sttycols]} { set cols $sttycols } } incr cols -1 # Now output foreach help $::autosetup(optionhelp) { lassign $help opt module desc if {![string match $what $module]} { continue } if {$local == 0 && $module eq "auto.def"} { puts "Local Options:" incr local } if {[string match =* $opt]} { # Output a special heading line" puts [string range $opt 1 end] continue } puts -nonewline " [format %-${max}s $opt]" if {[string match \n* $desc]} { # Output a pre-formatted help description as-is puts $desc } else { options-wrap-desc [string trim $desc] $cols " " $indent [expr {$max+2}] } } } # @options optionspec # # Specifies configuration-time options which may be selected by the user # and checked with 'opt-str' and 'opt-bool'. '$optionspec' contains a series # of options specifications separated by newlines, as follows: # # A boolean option is of the form: # ## name[=0|1] => "Description of this boolean option" # # The default is 'name=0', meaning that the option is disabled by default. # If 'name=1' is used to make the option enabled by default, the description should reflect # that with text like "Disable support for ...". # # An argument option (one which takes a parameter) is of one of the following forms: # ## name:value => "Description of this option" ## name:value=default => "Description of this option with a default value" ## name:=value => "Description of this option with an optional value" # # If the 'name:value' form is used, the value must be provided with the option (as '--name=myvalue'). # If the 'name:value=default' form is used, the option has the given default value even if not # specified by the user. # If the 'name:=value' form is used, the value is optional and the given value is used # if it is not provided. # # The description may contain '@default@', in which case it will be replaced with the default # value for the option (taking into account defaults specified with 'options-defaults'. # # Undocumented options are also supported by omitting the '=> description'. # These options are not displayed with '--help' and can be useful for internal options or as aliases. # # For example, '--disable-lfs' is an alias for '--disable=largefile': # ## lfs=1 largefile=1 => "Disable large file support" # proc options {optlist} { global autosetup options-add $optlist if {$autosetup(showhelp)} { # If --help, stop now to show help return -code break } if {$autosetup(module) eq "auto.def"} { # Check for invalid options if {[opt-bool option-checking]} { foreach o [dict keys $::autosetup(getopt)] { if {$o ni $::autosetup(options)} { user-error "Unknown option --$o" } } } } } # @options-defaults dictionary # # Specifies a dictionary of options and a new default value for each of those options. # Use before any 'use' statements in 'auto.def' to change the defaults for # subsequently included modules. proc options-defaults {dict} { foreach {n v} $dict { dict set ::autosetup(options-defaults) $n $v } } proc config_guess {} { if {[file-isexec $::autosetup(dir)/autosetup-config.guess]} { if {[catch {exec-with-stderr sh $::autosetup(dir)/autosetup-config.guess} alias]} { user-error $alias } return $alias } else { configlog "No autosetup-config.guess, so using uname" string tolower [exec uname -p]-unknown-[exec uname -s][exec uname -r] } } proc config_sub {alias} { if {[file-isexec $::autosetup(dir)/autosetup-config.sub]} { if {[catch {exec-with-stderr sh $::autosetup(dir)/autosetup-config.sub $alias} alias]} { user-error $alias } } return $alias } # @section Variable Definitions (defines) # @define name ?value=1? # # Defines the named variable to the given value. # These (name, value) pairs represent the results of the configuration check # and are available to be subsequently checked, modified and substituted. # proc define {name {value 1}} { set ::define($name) $value #dputs "$name <= $value" } # @define-push {name ...} script # # Save the values of the given defines, evaluation the script, then restore. # For example, to avoid updating AS_FLAGS and AS_CXXFLAGS: ## define-push {AS_CFLAGS AS_CXXFLAGS} { ## cc-check-flags -Wno-error ## } proc define-push {names script} { array set unset {} foreach name $names { if {[is-defined $name]} { set save($name) [get-define $name] } else { set unset($name) 1 } } uplevel 1 $script array set ::define [array get save] foreach name [array names unset] { unset -nocomplain ::define($name) } } # @undefine name # # Undefine the named variable. # proc undefine {name} { unset -nocomplain ::define($name) #dputs "$name <= <undef>" } # @define-append name value ... # # Appends the given value(s) to the given "defined" variable. # If the variable is not defined or empty, it is set to '$value'. # Otherwise the value is appended, separated by a space. # Any extra values are similarly appended. # # Note that define-append is not designed to add values containing spaces. # If values may contain spaces, consider define-append-argv instead. # proc define-append {name args} { if {[get-define $name ""] ne ""} { foreach arg $args { if {$arg eq ""} { continue } append ::define($name) " " $arg } } else { set ::define($name) [join $args] } #dputs "$name += [join $args] => $::define($name)" } # @define-append-argv name value ... # # Similar to define-append except designed to construct shell command # lines, including correct handling of parameters with spaces. # # Each non-empty value is quoted if necessary and then appended to the given variable # if it does not already exist. # proc define-append-argv {name args} { set seen {} set new {} foreach val [list {*}[get-define $name ""] {*}$args] { if {$val ne {} && ![dict exists $seen $val]} { lappend new [quote-if-needed $val] dict set seen $val 1 } } set ::define($name) [join $new " "] #dputs "$name += [join $args] => $::define($name)" } # @get-define name ?default=0? # # Returns the current value of the "defined" variable, or '$default' # if not set. # proc get-define {name {default 0}} { if {[info exists ::define($name)]} { #dputs "$name => $::define($name)" return $::define($name) } #dputs "$name => $default" return $default } # @is-defined name # # Returns 1 if the given variable is defined. # proc is-defined {name} { info exists ::define($name) } # @is-define-set name # # Returns 1 if the given variable is defined and is set # to a value other than "" or 0 # proc is-define-set {name} { if {[get-define $name] in {0 ""}} { return 0 } return 1 } # @all-defines # # Returns a dictionary (name, value list) of all defined variables. # # This is suitable for use with 'dict', 'array set' or 'foreach' # and allows for arbitrary processing of the defined variables. # proc all-defines {} { array get ::define } # @section Environment/Helpers # @get-env name default # # If '$name' was specified on the command line, return it. # Otherwise if '$name' was set in the environment, return it. # Otherwise return '$default'. # proc get-env {name default} { if {[dict exists $::autosetup(cmdline) $name]} { return [dict get $::autosetup(cmdline) $name] } getenv $name $default } # @env-is-set name # # Returns 1 if '$name' was specified on the command line or in the environment. # Note that an empty environment variable is not considered to be set. # proc env-is-set {name} { if {[dict exists $::autosetup(cmdline) $name]} { return 1 } if {[getenv $name ""] ne ""} { return 1 } return 0 } # @readfile filename ?default=""? # # Return the contents of the file, without the trailing newline. # If the file doesn't exist or can't be read, returns '$default'. # proc readfile {filename {default_value ""}} { set result $default_value catch { set f [open $filename] set result [read -nonewline $f] close $f } return $result } # @writefile filename value # # Creates the given file containing '$value'. # Does not add an extra newline. # proc writefile {filename value} { set f [open $filename w] puts -nonewline $f $value close $f } proc quote-if-needed {str} { if {[string match {*[\" ]*} $str]} { return \"[string map [list \" \\" \\ \\\\] $str]\" } return $str } proc quote-argv {argv} { set args {} foreach arg $argv { lappend args [quote-if-needed $arg] } join $args } # @list-non-empty list # # Returns a copy of the given list with empty elements removed proc list-non-empty {list} { set result {} foreach p $list { if {$p ne ""} { lappend result $p } } return $result } # @section Paths, Searching # @find-executable-path name # # Searches the path for an executable with the given name. # Note that the name may include some parameters, e.g. 'cc -mbig-endian', # in which case the parameters are ignored. # Returns the full path to the executable if found, or "" if not found. # proc find-executable-path {name} { # Ignore any parameters set name [lindex $name 0] # The empty string is never a valid executable if {$name ne ""} { foreach p [split-path] { dputs "Looking for $name in $p" set exec [file join $p $name] if {[file-isexec $exec]} { dputs "Found $name -> $exec" return $exec } } } return {} } # @find-executable name # # Searches the path for an executable with the given name. # Note that the name may include some parameters, e.g. 'cc -mbig-endian', # in which case the parameters are ignored. # Returns 1 if found, or 0 if not. # proc find-executable {name} { if {[find-executable-path $name] eq {}} { return 0 } return 1 } # @find-an-executable ?-required? name ... # # Given a list of possible executable names, # searches for one of these on the path. # # Returns the name found, or "" if none found. # If the first parameter is '-required', an error is generated # if no executable is found. # proc find-an-executable {args} { set required 0 if {[lindex $args 0] eq "-required"} { set args [lrange $args 1 end] incr required } foreach name $args { if {[find-executable $name]} { return $name } } if {$required} { if {[llength $args] == 1} { user-error "failed to find: [join $args]" } else { user-error "failed to find one of: [join $args]" } } return "" } # @section Logging, Messages and Errors # @configlog msg # # Writes the given message to the configuration log, 'config.log'. # proc configlog {msg} { if {![info exists ::autosetup(logfh)]} { set ::autosetup(logfh) [open config.log w] } puts $::autosetup(logfh) $msg } # @msg-checking msg # # Writes the message with no newline to stdout. # proc msg-checking {msg} { if {$::autosetup(msg-quiet) == 0} { maybe-show-timestamp puts -nonewline $msg set ::autosetup(msg-checking) 1 } } # @msg-result msg # # Writes the message to stdout. # proc msg-result {msg} { if {$::autosetup(msg-quiet) == 0} { maybe-show-timestamp puts $msg set ::autosetup(msg-checking) 0 show-notices } } # @msg-quiet command ... # # 'msg-quiet' evaluates it's arguments as a command with output # from 'msg-checking' and 'msg-result' suppressed. # # This is useful if a check needs to run a subcheck which isn't # of interest to the user. proc msg-quiet {args} { incr ::autosetup(msg-quiet) set rc [uplevel 1 $args] incr ::autosetup(msg-quiet) -1 return $rc } # Will be overridden by 'use misc' proc error-stacktrace {msg} { return $msg } proc error-location {msg} { return $msg } ################################################################## # # Debugging output # proc dputs {msg} { if {$::autosetup(debug)} { puts $msg } } ################################################################## # # User and system warnings and errors # # Usage errors such as wrong command line options # @user-error msg # # Indicate incorrect usage to the user, including if required components # or features are not found. # 'autosetup' exits with a non-zero return code. # proc user-error {msg} { show-notices puts stderr "Error: $msg" puts stderr "Try: '[file tail $::autosetup(exe)] --help' for options" exit 1 } # @user-notice msg # # Output the given message to stderr. # proc user-notice {msg} { lappend ::autosetup(notices) $msg } # Incorrect usage in the auto.def file. Identify the location. proc autosetup-error {msg} { autosetup-full-error [error-location $msg] } # Like autosetup-error, except $msg is the full error message. proc autosetup-full-error {msg} { show-notices puts stderr $msg exit 1 } proc show-notices {} { if {$::autosetup(msg-checking)} { puts "" set ::autosetup(msg-checking) 0 } flush stdout if {[info exists ::autosetup(notices)]} { puts stderr [join $::autosetup(notices) \n] unset ::autosetup(notices) } } proc maybe-show-timestamp {} { if {$::autosetup(msg-timing) && $::autosetup(msg-checking) == 0} { puts -nonewline [format {[%6.2f] } [expr {([clock millis] - $::autosetup(start)) % 10000 / 1000.0}]] } } # @autosetup-require-version required # # Checks the current version of 'autosetup' against '$required'. # A fatal error is generated if the current version is less than that required. # proc autosetup-require-version {required} { if {[compare-versions $::autosetup(version) $required] < 0} { user-error "autosetup version $required is required, but this is $::autosetup(version)" } } proc autosetup_version {} { return "autosetup v$::autosetup(version)" } ################################################################## # # Directory/path handling # proc realdir {dir} { set oldpwd [pwd] cd $dir set pwd [pwd] cd $oldpwd return $pwd } # Follow symlinks until we get to something which is not a symlink proc realpath {path} { while {1} { if {[catch { set path [file readlink $path] }]} { # Not a link break } } return $path } # Convert absolute path, $path into a path relative # to the given directory (or the current dir, if not given). # proc relative-path {path {pwd {}}} { set diff 0 set same 0 set newf {} set prefix {} set path [file-normalize $path] if {$pwd eq ""} { set pwd [pwd] } else { set pwd [file-normalize $pwd] } if {$path eq $pwd} { return . } # Try to make the filename relative to the current dir foreach p [split $pwd /] f [split $path /] { if {$p ne $f} { incr diff } elseif {!$diff} { incr same } if {$diff} { if {$p ne ""} { # Add .. for sibling or parent dir lappend prefix .. } if {$f ne ""} { lappend newf $f } } } if {$same == 1 || [llength $prefix] > 3} { return $path } file join [join $prefix /] [join $newf /] } # Add filename as a dependency to rerun autosetup # The name will be normalised (converted to a full path) # proc autosetup_add_dep {filename} { lappend ::autosetup(deps) [file-normalize $filename] } # @section Modules Support ################################################################## # # Library module support # # @use module ... # # Load the given library modules. # e.g. 'use cc cc-shared' # # Note that module 'X' is implemented in either 'autosetup/X.tcl' # or 'autosetup/X/init.tcl' # # The latter form is useful for a complex module which requires additional # support file. In this form, '$::usedir' is set to the module directory # when it is loaded. # proc use {args} { global autosetup libmodule modsource set dirs [list $autosetup(libdir)] if {[info exists autosetup(srcdir)]} { lappend dirs $autosetup(srcdir)/autosetup } foreach m $args { if {[info exists libmodule($m)]} { continue } set libmodule($m) 1 if {[info exists modsource(${m}.tcl)]} { autosetup_load_module $m eval $modsource(${m}.tcl) } else { set locs [list ${m}.tcl ${m}/init.tcl] set found 0 foreach dir $dirs { foreach loc $locs { set source $dir/$loc if {[file exists $source]} { incr found break } } if {$found} { break } } if {$found} { # For the convenience of the "use" source, point to the directory # it is being loaded from set ::usedir [file dirname $source] autosetup_load_module $m source $source autosetup_add_dep $source } else { autosetup-error "use: No such module: $m" } } } } proc autosetup_load_auto_modules {} { global autosetup modsource # First load any embedded auto modules foreach mod [array names modsource *.auto] { autosetup_load_module $mod eval $modsource($mod) } # Now any external auto modules foreach file [glob -nocomplain $autosetup(libdir)/*.auto $autosetup(libdir)/*/*.auto] { autosetup_load_module [file tail $file] source $file } } # Load module source in the global scope by executing the given command proc autosetup_load_module {module args} { global autosetup set prev $autosetup(module) set autosetup(module) $module if {[catch [list uplevel #0 $args] msg opts] ni {0 2 3}} { autosetup-full-error [error-dump $msg $opts $::autosetup(debug)] } set autosetup(module) $prev } # Initial settings set autosetup(exe) $::argv0 set autosetup(istcl) 1 set autosetup(start) [clock millis] set autosetup(installed) 0 set autosetup(sysinstall) 0 set autosetup(msg-checking) 0 set autosetup(msg-quiet) 0 set autosetup(inittypes) {} set autosetup(module) autosetup # Embedded modules are inserted below here set autosetup(installed) 1 set autosetup(sysinstall) 0 # ----- @module asciidoc-formatting.tcl ----- set modsource(asciidoc-formatting.tcl) { # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which provides text formatting # asciidoc format use formatting proc para {text} { regsub -all "\[ \t\n\]+" [string trim $text] " " } proc title {text} { underline [para $text] = nl } proc p {text} { puts [para $text] nl } proc code {text} { foreach line [parse_code_block $text] { puts " $line" } nl } proc codelines {lines} { foreach line $lines { puts " $line" } nl } proc nl {} { puts "" } proc underline {text char} { regexp "^(\[ \t\]*)(.*)" $text -> indent words puts $text puts $indent[string repeat $char [string length $words]] } proc section {text} { underline "[para $text]" - nl } proc subsection {text} { underline "$text" ~ nl } proc bullet {text} { puts "* [para $text]" } proc indent {text} { puts " :: " puts [para $text] } proc defn {first args} { set sep "" if {$first ne ""} { puts "${first}::" } else { puts " :: " } set defn [string trim [join $args \n]] regsub -all "\n\n" $defn "\n ::\n" defn puts $defn } } # ----- @module formatting.tcl ----- set modsource(formatting.tcl) { # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which provides common text formatting # This is designed for documentation which looks like: # code {...} # or # code { # ... # ... # } # In the second case, we need to work out the indenting # and strip it from all lines but preserve the remaining indenting. # Note that all lines need to be indented with the same initial # spaces/tabs. # # Returns a list of lines with the indenting removed. # proc parse_code_block {text} { # If the text begins with newline, take the following text, # otherwise just return the original if {![regexp "^\n(.*)" $text -> text]} { return [list [string trim $text]] } # And trip spaces off the end set text [string trimright $text] set min 100 # Examine each line to determine the minimum indent foreach line [split $text \n] { if {$line eq ""} { # Ignore empty lines for the indent calculation continue } regexp "^(\[ \t\]*)" $line -> indent set len [string length $indent] if {$len < $min} { set min $len } } # Now make a list of lines with this indent removed set lines {} foreach line [split $text \n] { lappend lines [string range $line $min end] } # Return the result return $lines } } # ----- @module getopt.tcl ----- set modsource(getopt.tcl) { # Copyright (c) 2006 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Simple getopt module # Parse everything out of the argv list which looks like an option # Everything which doesn't look like an option, or is after --, is left unchanged # Understands --enable-xxx as a synonym for --xxx to enable the boolean option xxx. # Understands --disable-xxx to disable the boolean option xxx. # # The returned value is a dictionary keyed by option name # Each value is a list of {type value} ... where type is "bool" or "str". # The value for a boolean option is 0 or 1. The value of a string option is the value given. proc getopt {argvname} { upvar $argvname argv set nargv {} set opts {} for {set i 0} {$i < [llength $argv]} {incr i} { set arg [lindex $argv $i] #dputs arg=$arg if {$arg eq "--"} { # End of options incr i lappend nargv {*}[lrange $argv $i end] break } if {[regexp {^--([^=][^=]+)=(.*)$} $arg -> name value]} { # --name=value dict lappend opts $name [list str $value] } elseif {[regexp {^--(enable-|disable-)?([^=]*)$} $arg -> prefix name]} { if {$prefix in {enable- ""}} { set value 1 } else { set value 0 } dict lappend opts $name [list bool $value] } else { lappend nargv $arg } } #puts "getopt: argv=[join $argv] => [join $nargv]" #array set getopt $opts #parray getopt set argv $nargv return $opts } } # ----- @module help.tcl ----- set modsource(help.tcl) { # Copyright (c) 2010 WorkWare Systems http://workware.net.au/ # All rights reserved # Module which provides usage, help and the command reference proc autosetup_help {what} { use_pager puts "Usage: [file tail $::autosetup(exe)] \[options\] \[settings\]\n" puts "This is [autosetup_version], a build environment \"autoconfigurator\"" puts "See the documentation online at https://msteveb.github.io/autosetup/\n" if {$what in {all local}} { # Need to load auto.def now if {[file exists $::autosetup(autodef)]} { # Load auto.def as module "auto.def" autosetup_load_module auto.def source $::autosetup(autodef) } if {$what eq "all"} { set what * } else { set what auto.def } } else { use $what puts "Options for module $what:" } options-show $what exit 0 } proc autosetup_show_license {} { global modsource autosetup use_pager if {[info exists modsource(LICENSE)]} { puts $modsource(LICENSE) return } foreach dir [list $autosetup(libdir) $autosetup(srcdir)] { set path [file join $dir LICENSE] if {[file exists $path]} { puts [readfile $path] return } } puts "LICENSE not found" } # If not already paged and stdout is a tty, pipe the output through the pager # This is done by reinvoking autosetup with --nopager added proc use_pager {} { if {![opt-bool nopager] && [getenv PAGER ""] ne "" && [isatty? stdin] && [isatty? stdout]} { if {[catch { exec [info nameofexecutable] $::argv0 --nopager {*}$::argv |& {*}[getenv PAGER] >@stdout <@stdin 2>@stderr } msg opts] == 1} { if {[dict get $opts -errorcode] eq "NONE"} { # an internal/exec error puts stderr $msg exit 1 } } exit 0 } } # Outputs the autosetup references in one of several formats proc autosetup_reference {{type text}} { use_pager switch -glob -- $type { wiki {use wiki-formatting} ascii* {use asciidoc-formatting} md - markdown {use markdown-formatting} default {use text-formatting} } title "[autosetup_version] -- Command Reference" section {Introduction} p { See https://msteveb.github.io/autosetup/ for the online documentation for 'autosetup'. This documentation can also be accessed locally with `autosetup --ref`. } p { 'autosetup' provides a number of built-in commands which are documented below. These may be used from 'auto.def' to test for features, define variables, create files from templates and other similar actions. } automf_command_reference exit 0 } proc autosetup_output_block {type lines} { if {[llength $lines]} { switch $type { section { section $lines } subsection { subsection $lines } code { codelines $lines } p { p [join $lines] } list { foreach line $lines { bullet $line } nl } } } } # Generate a command reference from inline documentation proc automf_command_reference {} { lappend files $::autosetup(prog) lappend files {*}[lsort [glob -nocomplain $::autosetup(libdir)/*.tcl]] # We want to process all non-module files before module files # and then modules in alphabetical order. # So examine all files and extract docs into doc($modulename) and doc(_core_) # # Each entry is a list of {type data} where $type is one of: section, subsection, code, list, p # and $data is a string for section, subsection or a list of text lines for other types. # XXX: Should commands be in alphabetical order too? Currently they are in file order. set doc(_core_) {} lappend doc(_core_) [list section "Core Commands"] foreach file $files { set modulename [file rootname [file tail $file]] set current _core_ set f [open $file] while {![eof $f]} { set line [gets $f] if {[regexp {^#.*@section (.*)$} $line -> section]} { lappend doc($current) [list section $section] continue } # Find embedded module names if {[regexp {^#.*@module ([^ ]*)} $line -> modulename]} { continue } # Find lines starting with "# @*" and continuing through the remaining comment lines if {![regexp {^# @(.*)} $line -> cmd]} { continue } # Synopsis or command? if {$cmd eq "synopsis:"} { set current $modulename lappend doc($current) [list section "Module: $modulename"] } else { lappend doc($current) [list subsection $cmd] } set lines {} set type p # Now the description while {![eof $f]} { set line [gets $f] if {![regexp {^#(#)? ?(.*)} $line -> hash cmd]} { break } if {$hash eq "#"} { set t code } elseif {[regexp {^- (.*)} $cmd -> cmd]} { set t list } else { set t p } #puts "hash=$hash, oldhash=$oldhash, lines=[llength $lines], cmd=$cmd" if {$t ne $type || $cmd eq ""} { # Finish the current block lappend doc($current) [list $type $lines] set lines {} set type $t } if {$cmd ne ""} { lappend lines $cmd } } lappend doc($current) [list $type $lines] } close $f } # Now format and output the results # _core_ will sort first foreach module [lsort [array names doc]] { foreach item $doc($module) { autosetup_output_block {*}$item } } } } # ----- @module init.tcl ----- set modsource(init.tcl) { # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module to help create auto.def and configure proc autosetup_init {type} { set help 0 if {$type in {? help}} { incr help } elseif {![dict exists $::autosetup(inittypes) $type]} { puts "Unknown type, --init=$type" incr help } if {$help} { puts "Use one of the following types (e.g. --init=make)\n" foreach type [lsort [dict keys $::autosetup(inittypes)]] { lassign [dict get $::autosetup(inittypes) $type] desc # XXX: Use the options-show code to wrap the description puts [format "%-10s %s" $type $desc] } return } lassign [dict get $::autosetup(inittypes) $type] desc script puts "Initialising $type: $desc\n" # All initialisations happens in the top level srcdir cd $::autosetup(srcdir) uplevel #0 $script } proc autosetup_add_init_type {type desc script} { dict set ::autosetup(inittypes) $type [list $desc $script] } # This is for in creating build-system init scripts # # If the file doesn't exist, create it containing $contents # If the file does exist, only overwrite if --force is specified. # proc autosetup_check_create {filename contents} { if {[file exists $filename]} { if {!$::autosetup(force)} { puts "I see $filename already exists." return } else { puts "I will overwrite the existing $filename because you used --force." } } else { puts "I don't see $filename, so I will create it." } writefile $filename $contents } } # ----- @module install.tcl ----- set modsource(install.tcl) { # Copyright (c) 2006-2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which can install autosetup # autosetup(installed)=1 means that autosetup is not running from source # autosetup(sysinstall)=1 means that autosetup is running from a sysinstall version # shared=1 means that we are trying to do a sysinstall. This is only possible from the development source. proc autosetup_install {dir {shared 0}} { global autosetup if {$shared} { if {$autosetup(installed) || $autosetup(sysinstall)} { user-error "Can only --sysinstall from development sources" } } elseif {$autosetup(installed) && !$autosetup(sysinstall)} { user-error "Can't --install from project install" } if {$autosetup(sysinstall)} { # This is the sysinstall version, so install just uses references cd $dir puts "[autosetup_version] creating configure to use system-installed autosetup" autosetup_create_configure 1 puts "Creating autosetup/README.autosetup" file mkdir autosetup autosetup_install_readme autosetup/README.autosetup 1 return } if {[catch { if {$shared} { set target $dir/bin/autosetup set installedas $target } else { if {$dir eq "."} { set installedas autosetup } else { set installedas $dir/autosetup } cd $dir file mkdir autosetup set target autosetup/autosetup } set targetdir [file dirname $target] file mkdir $targetdir set f [open $target w] set publicmodules {} # First the main script, but only up until "CUT HERE" set in [open $autosetup(dir)/autosetup] while {[gets $in buf] >= 0} { if {$buf ne "##-- CUT HERE --##"} { puts $f $buf continue } # Insert the static modules here # i.e. those which don't contain @synopsis: # All modules are inserted if $shared is set puts $f "set autosetup(installed) 1" puts $f "set autosetup(sysinstall) $shared" foreach file [lsort [glob $autosetup(libdir)/*.{tcl,auto}]] { set modname [file tail $file] set ext [file ext $modname] set buf [readfile $file] if {!$shared} { if {$ext eq ".auto" || [string match "*\n# @synopsis:*" $buf]} { lappend publicmodules $file continue } } dputs "install: importing lib/[file tail $file]" puts $f "# ----- @module $modname -----" puts $f "\nset modsource($modname) \{" puts $f $buf puts $f "\}\n" } if {$shared} { foreach {srcname destname} [list $autosetup(libdir)/README.autosetup-lib README.autosetup \ $autosetup(srcdir)/LICENSE LICENSE] { dputs "install: importing $srcname as $destname" puts $f "\nset modsource($destname) \\\n[list [readfile $srcname]\n]\n" } } } close $in close $f catch {exec chmod 755 $target} set installfiles {autosetup-config.guess autosetup-config.sub autosetup-test-tclsh} set removefiles {} if {!$shared} { autosetup_install_readme $targetdir/README.autosetup 0 # Install public modules foreach file $publicmodules { set tail [file tail $file] autosetup_install_file $file $targetdir/$tail } lappend installfiles jimsh0.c autosetup-find-tclsh LICENSE lappend removefiles config.guess config.sub test-tclsh find-tclsh } else { lappend installfiles {sys-find-tclsh autosetup-find-tclsh} } # Install support files foreach fileinfo $installfiles { if {[llength $fileinfo] == 2} { lassign $fileinfo source dest } else { lassign $fileinfo source set dest $source } autosetup_install_file $autosetup(dir)/$source $targetdir/$dest } # Remove obsolete files foreach file $removefiles { if {[file exists $targetdir/$file]} { file delete $targetdir/$file } } } error]} { user-error "Failed to install autosetup: $error" } if {$shared} { set type "system" } else { set type "local" } puts "Installed $type [autosetup_version] to $installedas" if {!$shared} { # Now create 'configure' if necessary autosetup_create_configure 0 } } proc autosetup_create_configure {shared} { if {[file exists configure]} { if {!$::autosetup(force)} { # Could this be an autosetup configure? if {![string match "*\nWRAPPER=*" [readfile configure]]} { puts "I see configure, but not created by autosetup, so I won't overwrite it." puts "Remove it or use --force to overwrite." return } } else { puts "I will overwrite the existing configure because you used --force." } } else { puts "I don't see configure, so I will create it." } if {$shared} { writefile configure \ {#!/bin/sh WRAPPER="$0"; export WRAPPER; "autosetup" "$@" } } else { writefile configure \ {#!/bin/sh dir="`dirname "$0"`/autosetup" #@@INITCHECK@@# WRAPPER="$0"; export WRAPPER; exec "`"$dir/autosetup-find-tclsh"`" "$dir/autosetup" "$@" } } catch {exec chmod 755 configure} } # Append the contents of $file to filehandle $f proc autosetup_install_append {f file} { dputs "install: include $file" set in [open $file] puts $f [read $in] close $in } proc autosetup_install_file {source target} { dputs "install: $source => $target" if {![file exists $source]} { error "Missing installation file '$source'" } writefile $target [readfile $source]\n # If possible, copy the file mode file stat $source stat set mode [format %o [expr {$stat(mode) & 0x1ff}]] catch {exec chmod $mode $target} } proc autosetup_install_readme {target sysinstall} { set readme "README.autosetup created by [autosetup_version]\n\n" if {$sysinstall} { append readme \ {This is the autosetup directory for a system install of autosetup. Loadable modules can be added here. } } else { append readme \ {This is the autosetup directory for a local install of autosetup. It contains autosetup, support files and loadable modules. } } append readme { *.tcl files in this directory are optional modules which can be loaded with the 'use' directive. *.auto files in this directory are auto-loaded. For more information, see https://msteveb.github.io/autosetup/ } dputs "install: autosetup/README.autosetup" writefile $target $readme } } # ----- @module markdown-formatting.tcl ----- set modsource(markdown-formatting.tcl) { # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which provides text formatting # markdown format (kramdown syntax) use formatting proc para {text} { regsub -all "\[ \t\n\]+" [string trim $text] " " text regsub -all {([^a-zA-Z])'([^']*)'} $text {\1**`\2`**} text regsub -all {^'([^']*)'} $text {**`\1`**} text regsub -all {(http[^ \t\n]*)} $text {[\1](\1)} text return $text } proc title {text} { underline [para $text] = nl } proc p {text} { puts [para $text] nl } proc codelines {lines} { puts "~~~~~~~~~~~~" foreach line $lines { puts $line } puts "~~~~~~~~~~~~" nl } proc code {text} { puts "~~~~~~~~~~~~" foreach line [parse_code_block $text] { puts $line } puts "~~~~~~~~~~~~" nl } proc nl {} { puts "" } proc underline {text char} { regexp "^(\[ \t\]*)(.*)" $text -> indent words puts $text puts $indent[string repeat $char [string length $words]] } proc section {text} { underline "[para $text]" - nl } proc subsection {text} { puts "### `$text`" nl } proc bullet {text} { puts "* [para $text]" } proc defn {first args} { puts "^" set defn [string trim [join $args \n]] if {$first ne ""} { puts "**${first}**" puts -nonewline ": " regsub -all "\n\n" $defn "\n: " defn } puts "$defn" } } # ----- @module misc.tcl ----- set modsource(misc.tcl) { # Copyright (c) 2007-2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module containing misc procs useful to modules # Largely for platform compatibility set autosetup(istcl) [info exists ::tcl_library] set autosetup(iswin) [string equal windows $tcl_platform(platform)] if {$autosetup(iswin)} { # mingw/windows separates $PATH with semicolons # and doesn't have an executable bit proc split-path {} { split [getenv PATH .] {;} } proc file-isexec {exec} { # Basic test for windows. We ignore .bat if {[file isfile $exec] || [file isfile $exec.exe]} { return 1 } return 0 } } else { # unix separates $PATH with colons and has and executable bit proc split-path {} { split [getenv PATH .] : } # Check for an executable file proc file-isexec {exec} { if {[file executable $exec] && [file isfile $exec]} { return 1 } return 0 } } # Assume that exec can return stdout and stderr proc exec-with-stderr {args} { exec {*}$args 2>@1 } if {$autosetup(istcl)} { # Tcl doesn't have the env command proc getenv {name args} { if {[info exists ::env($name)]} { return $::env($name) } if {[llength $args]} { return [lindex $args 0] } return -code error "environment variable \"$name\" does not exist" } proc isatty? {channel} { dict exists [fconfigure $channel] -xchar } # Jim-compatible stacktrace using info frame proc stacktrace {} { set stacktrace {} # 2 to skip the current frame for {set i 2} {$i < [info frame]} {incr i} { set frame [info frame -$i] if {[dict exists $frame file]} { # We don't need proc, so use "" lappend stacktrace "" [dict get $frame file] [dict get $frame line] } } return $stacktrace } } else { if {$autosetup(iswin)} { # On Windows, backslash convert all environment variables # (Assume that Tcl does this for us) proc getenv {name args} { string map {\\ /} [env $name {*}$args] } } else { # Jim on unix is simple alias getenv env } proc isatty? {channel} { set tty 0 catch { # isatty is a recent addition to Jim Tcl set tty [$channel isatty] } return $tty } } # In case 'file normalize' doesn't exist # proc file-normalize {path} { if {[catch {file normalize $path} result]} { if {$path eq ""} { return "" } set oldpwd [pwd] if {[file isdir $path]} { cd $path set result [pwd] } else { cd [file dirname $path] set result [file join [pwd] [file tail $path]] } cd $oldpwd } return $result } # If everything is working properly, the only errors which occur # should be generated in user code (e.g. auto.def). # By default, we only want to show the error location in user code. # We use [info frame] to achieve this, but it works differently on Tcl and Jim. # # This is designed to be called for incorrect usage in auto.def, via autosetup-error # proc error-location {msg} { if {$::autosetup(debug)} { return -code error $msg } # Search back through the stack trace for the first error in a .def file foreach {p f l} [stacktrace] { if {[string match *.def $f]} { return "[relative-path $f]:$l: Error: $msg" } #puts "Skipping $f:$l" } return $msg } # If everything is working properly, the only errors which occur # should be generated in user code (e.g. auto.def). # By default, we only want to show the error location in user code. # We use [info frame] to achieve this, but it works differently on Tcl and Jim. # # This is designed to be called for incorrect usage in auto.def, via autosetup-error # proc error-stacktrace {msg} { if {$::autosetup(debug)} { return -code error $msg } # Search back through the stack trace for the first error in a .def file for {set i 1} {$i < [info level]} {incr i} { if {$::autosetup(istcl)} { array set info [info frame -$i] } else { lassign [info frame -$i] info(caller) info(file) info(line) } if {[string match *.def $info(file)]} { return "[relative-path $info(file)]:$info(line): Error: $msg" } #puts "Skipping $info(file):$info(line)" } return $msg } # Given the return from [catch {...} msg opts], returns an appropriate # error message. A nice one for Jim and a less-nice one for Tcl. # If 'fulltrace' is set, a full stack trace is provided. # Otherwise a simple message is provided. # # This is designed for developer errors, e.g. in module code or auto.def code # # proc error-dump {msg opts fulltrace} { if {$::autosetup(istcl)} { if {$fulltrace} { return "Error: [dict get $opts -errorinfo]" } else { return "Error: $msg" } } else { lassign $opts(-errorinfo) p f l if {$f ne ""} { set result "$f:$l: Error: " } append result "$msg\n" if {$fulltrace} { append result [stackdump $opts(-errorinfo)] } # Remove the trailing newline string trim $result } } } # ----- @module text-formatting.tcl ----- set modsource(text-formatting.tcl) { # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which provides text formatting use formatting proc wordwrap {text length {firstprefix ""} {nextprefix ""}} { set len 0 set space $firstprefix foreach word [split $text] { set word [string trim $word] if {$word eq ""} { continue } if {[info exists partial]} { append partial " " $word if {[string first $quote $word] < 0} { # Haven't found end of quoted word continue } # Finished quoted word set word $partial unset partial unset quote } else { set quote [string index $word 0] if {$quote in {' *}} { if {[string first $quote $word 1] < 0} { # Haven't found end of quoted word # Not a whole word. set first [string index $word 0] # Start of quoted word set partial $word continue } } } if {$len && [string length $space$word] + $len >= $length} { puts "" set len 0 set space $nextprefix } incr len [string length $space$word] # Use man-page conventions for highlighting 'quoted' and *quoted* # single words. # Use x^Hx for *bold* and _^Hx for 'underline'. # # less and more will both understand this. # Pipe through 'col -b' to remove them. if {[regexp {^'(.*)'(.*)} $word -> quoted after]} { set quoted [string map {~ " "} $quoted] regsub -all . $quoted "&\b&" quoted set word $quoted$after } elseif {[regexp {^[*](.*)[*](.*)} $word -> quoted after]} { set quoted [string map {~ " "} $quoted] regsub -all . $quoted "_\b&" quoted set word $quoted$after } puts -nonewline $space$word set space " " } if {[info exists partial]} { # Missing end of quote puts -nonewline $space$partial } if {$len} { puts "" } } proc title {text} { underline [string trim $text] = nl } proc p {text} { wordwrap $text 80 nl } proc codelines {lines} { foreach line $lines { puts " $line" } nl } proc nl {} { puts "" } proc underline {text char} { regexp "^(\[ \t\]*)(.*)" $text -> indent words puts $text puts $indent[string repeat $char [string length $words]] } proc section {text} { underline "[string trim $text]" - nl } proc subsection {text} { underline "$text" ~ nl } proc bullet {text} { wordwrap $text 76 " * " " " } proc indent {text} { wordwrap $text 76 " " " " } proc defn {first args} { if {$first ne ""} { underline " $first" ~ } foreach p $args { if {$p ne ""} { indent $p } } } } # ----- @module util.tcl ----- set modsource(util.tcl) { # Copyright (c) 2012 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which contains miscellaneous utility functions # @section Utilities # @compare-versions version1 version2 # # Versions are of the form 'a.b.c' (may be any number of numeric components) # # Compares the two versions and returns: ## -1 if v1 < v2 ## 0 if v1 == v2 ## 1 if v1 > v2 # # If one version has fewer components than the other, 0 is substituted to the right. e.g. ## 0.2 < 0.3 ## 0.2.5 > 0.2 ## 1.1 == 1.1.0 # proc compare-versions {v1 v2} { foreach c1 [split $v1 .] c2 [split $v2 .] { if {$c1 eq ""} { set c1 0 } if {$c2 eq ""} { set c2 0 } if {$c1 < $c2} { return -1 } if {$c1 > $c2} { return 1 } } return 0 } # @suffix suf list # # Takes a list and returns a new list with '$suf' appended # to each element # ## suffix .c {a b c} => {a.c b.c c.c} # proc suffix {suf list} { set result {} foreach p $list { lappend result $p$suf } return $result } # @prefix pre list # # Takes a list and returns a new list with '$pre' prepended # to each element # ## prefix jim- {a.c b.c} => {jim-a.c jim-b.c} # proc prefix {pre list} { set result {} foreach p $list { lappend result $pre$p } return $result } # @lpop list # # Removes the last entry from the given list and returns it. proc lpop {listname} { upvar $listname list set val [lindex $list end] set list [lrange $list 0 end-1] return $val } } # ----- @module wiki-formatting.tcl ----- set modsource(wiki-formatting.tcl) { # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # Module which provides text formatting # wiki.tcl.tk format output use formatting proc joinlines {text} { set lines {} foreach l [split [string trim $text] \n] { lappend lines [string trim $l] } join $lines } proc p {text} { puts [joinlines $text] puts "" } proc title {text} { puts "*** [joinlines $text] ***" puts "" } proc codelines {lines} { puts "======" foreach line $lines { puts " $line" } puts "======" } proc code {text} { puts "======" foreach line [parse_code_block $text] { puts " $line" } puts "======" } proc nl {} { } proc section {text} { puts "'''$text'''" puts "" } proc subsection {text} { puts "''$text''" puts "" } proc bullet {text} { puts " * [joinlines $text]" } proc indent {text} { puts " : [joinlines $text]" } proc defn {first args} { if {$first ne ""} { indent '''$first''' } foreach p $args { p $p } } } ################################################################## # # Entry/Exit # if {$autosetup(debug)} { main $argv } if {[catch {main $argv} msg opts] == 1} { show-notices autosetup-full-error [error-dump $msg $opts $autosetup(debug)] if {!$autosetup(debug)} { puts stderr "Try: '[file tail $autosetup(exe)] --autosetup-debug' for a full stack trace" } exit 1 } |
Added autosetup/autosetup-config.guess.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 | #! /bin/sh # Attempt to guess a canonical system name. # Copyright 1992-2024 Free Software Foundation, Inc. # shellcheck disable=SC2006,SC2268 # see below for rationale timestamp='2024-07-27' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, see <https://www.gnu.org/licenses/>. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # # Originally written by Per Bothner; maintained since 2000 by Ben Elliston. # # You can get the latest version of this script from: # https://git.savannah.gnu.org/cgit/config.git/plain/config.guess # # Please send patches to <config-patches@gnu.org>. # The "shellcheck disable" line above the timestamp inhibits complaints # about features and limitations of the classic Bourne shell that were # superseded or lifted in POSIX. However, this script identifies a wide # variety of pre-POSIX systems that do not have POSIX shells at all, and # even some reasonably current systems (Solaris 10 as case-in-point) still # have a pre-POSIX /bin/sh. me=`echo "$0" | sed -e 's,.*/,,'` usage="\ Usage: $0 [OPTION] Output the configuration name of the system '$me' is run on. Options: -h, --help print this help, then exit -t, --time-stamp print date of last modification, then exit -v, --version print version number, then exit Report bugs and patches to <config-patches@gnu.org>." version="\ GNU config.guess ($timestamp) Originally written by Per Bothner. Copyright 1992-2024 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." help=" Try '$me --help' for more information." # Parse command line while test $# -gt 0 ; do case $1 in --time-stamp | --time* | -t ) echo "$timestamp" ; exit ;; --version | -v ) echo "$version" ; exit ;; --help | --h* | -h ) echo "$usage"; exit ;; -- ) # Stop option processing shift; break ;; - ) # Use stdin as input. break ;; -* ) echo "$me: invalid option $1$help" >&2 exit 1 ;; * ) break ;; esac done if test $# != 0; then echo "$me: too many arguments$help" >&2 exit 1 fi # Just in case it came from the environment. GUESS= # CC_FOR_BUILD -- compiler used by this script. Note that the use of a # compiler to aid in system detection is discouraged as it requires # temporary files to be created and, as you can see below, it is a # headache to deal with in a portable fashion. # Historically, 'CC_FOR_BUILD' used to be named 'HOST_CC'. We still # use 'HOST_CC' if defined, but it is deprecated. # Portable tmp directory creation inspired by the Autoconf team. tmp= # shellcheck disable=SC2172 trap 'test -z "$tmp" || rm -fr "$tmp"' 0 1 2 13 15 set_cc_for_build() { # prevent multiple calls if $tmp is already set test "$tmp" && return 0 : "${TMPDIR=/tmp}" # shellcheck disable=SC2039,SC3028 { tmp=`(umask 077 && mktemp -d "$TMPDIR/cgXXXXXX") 2>/dev/null` && test -n "$tmp" && test -d "$tmp" ; } || { test -n "$RANDOM" && tmp=$TMPDIR/cg$$-$RANDOM && (umask 077 && mkdir "$tmp" 2>/dev/null) ; } || { tmp=$TMPDIR/cg-$$ && (umask 077 && mkdir "$tmp" 2>/dev/null) && echo "Warning: creating insecure temp directory" >&2 ; } || { echo "$me: cannot create a temporary directory in $TMPDIR" >&2 ; exit 1 ; } dummy=$tmp/dummy case ${CC_FOR_BUILD-},${HOST_CC-},${CC-} in ,,) echo "int x;" > "$dummy.c" for driver in cc gcc c17 c99 c89 ; do if ($driver -c -o "$dummy.o" "$dummy.c") >/dev/null 2>&1 ; then CC_FOR_BUILD=$driver break fi done if test x"$CC_FOR_BUILD" = x ; then CC_FOR_BUILD=no_compiler_found fi ;; ,,*) CC_FOR_BUILD=$CC ;; ,*,*) CC_FOR_BUILD=$HOST_CC ;; esac } # This is needed to find uname on a Pyramid OSx when run in the BSD universe. # (ghazi@noc.rutgers.edu 1994-08-24) if test -f /.attbin/uname ; then PATH=$PATH:/.attbin ; export PATH fi UNAME_MACHINE=`(uname -m) 2>/dev/null` || UNAME_MACHINE=unknown UNAME_RELEASE=`(uname -r) 2>/dev/null` || UNAME_RELEASE=unknown UNAME_SYSTEM=`(uname -s) 2>/dev/null` || UNAME_SYSTEM=unknown UNAME_VERSION=`(uname -v) 2>/dev/null` || UNAME_VERSION=unknown case $UNAME_SYSTEM in Linux|GNU|GNU/*) LIBC=unknown set_cc_for_build cat <<-EOF > "$dummy.c" #if defined(__ANDROID__) LIBC=android #else #include <features.h> #if defined(__UCLIBC__) LIBC=uclibc #elif defined(__dietlibc__) LIBC=dietlibc #elif defined(__GLIBC__) LIBC=gnu #elif defined(__LLVM_LIBC__) LIBC=llvm #else #include <stdarg.h> /* First heuristic to detect musl libc. */ #ifdef __DEFINED_va_list LIBC=musl #endif #endif #endif EOF cc_set_libc=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^LIBC' | sed 's, ,,g'` eval "$cc_set_libc" # Second heuristic to detect musl libc. if [ "$LIBC" = unknown ] && command -v ldd >/dev/null && ldd --version 2>&1 | grep -q ^musl; then LIBC=musl fi # If the system lacks a compiler, then just pick glibc. # We could probably try harder. if [ "$LIBC" = unknown ]; then LIBC=gnu fi ;; esac # Note: order is significant - the case branches are not exclusive. case $UNAME_MACHINE:$UNAME_SYSTEM:$UNAME_RELEASE:$UNAME_VERSION in *:NetBSD:*:*) # NetBSD (nbsd) targets should (where applicable) match one or # more of the tuples: *-*-netbsdelf*, *-*-netbsdaout*, # *-*-netbsdecoff* and *-*-netbsd*. For targets that recently # switched to ELF, *-*-netbsd* would select the old # object file format. This provides both forward # compatibility and a consistent mechanism for selecting the # object file format. # # Note: NetBSD doesn't particularly care about the vendor # portion of the name. We always set it to "unknown". UNAME_MACHINE_ARCH=`(uname -p 2>/dev/null || \ /sbin/sysctl -n hw.machine_arch 2>/dev/null || \ /usr/sbin/sysctl -n hw.machine_arch 2>/dev/null || \ echo unknown)` case $UNAME_MACHINE_ARCH in aarch64eb) machine=aarch64_be-unknown ;; armeb) machine=armeb-unknown ;; arm*) machine=arm-unknown ;; sh3el) machine=shl-unknown ;; sh3eb) machine=sh-unknown ;; sh5el) machine=sh5le-unknown ;; earmv*) arch=`echo "$UNAME_MACHINE_ARCH" | sed -e 's,^e\(armv[0-9]\).*$,\1,'` endian=`echo "$UNAME_MACHINE_ARCH" | sed -ne 's,^.*\(eb\)$,\1,p'` machine=${arch}${endian}-unknown ;; *) machine=$UNAME_MACHINE_ARCH-unknown ;; esac # The Operating System including object format, if it has switched # to ELF recently (or will in the future) and ABI. case $UNAME_MACHINE_ARCH in earm*) os=netbsdelf ;; arm*|i386|m68k|ns32k|sh3*|sparc|vax) set_cc_for_build if echo __ELF__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ELF__ then # Once all utilities can be ECOFF (netbsdecoff) or a.out (netbsdaout). # Return netbsd for either. FIX? os=netbsd else os=netbsdelf fi ;; *) os=netbsd ;; esac # Determine ABI tags. case $UNAME_MACHINE_ARCH in earm*) expr='s/^earmv[0-9]/-eabi/;s/eb$//' abi=`echo "$UNAME_MACHINE_ARCH" | sed -e "$expr"` ;; esac # The OS release # Debian GNU/NetBSD machines have a different userland, and # thus, need a distinct triplet. However, they do not need # kernel version information, so it can be replaced with a # suitable tag, in the style of linux-gnu. case $UNAME_VERSION in Debian*) release='-gnu' ;; *) release=`echo "$UNAME_RELEASE" | sed -e 's/[-_].*//' | cut -d. -f1,2` ;; esac # Since CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM: # contains redundant information, the shorter form: # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM is used. GUESS=$machine-${os}${release}${abi-} ;; *:Bitrig:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/Bitrig.//'` GUESS=$UNAME_MACHINE_ARCH-unknown-bitrig$UNAME_RELEASE ;; *:OpenBSD:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/OpenBSD.//'` GUESS=$UNAME_MACHINE_ARCH-unknown-openbsd$UNAME_RELEASE ;; *:SecBSD:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/SecBSD.//'` GUESS=$UNAME_MACHINE_ARCH-unknown-secbsd$UNAME_RELEASE ;; *:LibertyBSD:*:*) UNAME_MACHINE_ARCH=`arch | sed 's/^.*BSD\.//'` GUESS=$UNAME_MACHINE_ARCH-unknown-libertybsd$UNAME_RELEASE ;; *:MidnightBSD:*:*) GUESS=$UNAME_MACHINE-unknown-midnightbsd$UNAME_RELEASE ;; *:ekkoBSD:*:*) GUESS=$UNAME_MACHINE-unknown-ekkobsd$UNAME_RELEASE ;; *:SolidBSD:*:*) GUESS=$UNAME_MACHINE-unknown-solidbsd$UNAME_RELEASE ;; *:OS108:*:*) GUESS=$UNAME_MACHINE-unknown-os108_$UNAME_RELEASE ;; macppc:MirBSD:*:*) GUESS=powerpc-unknown-mirbsd$UNAME_RELEASE ;; *:MirBSD:*:*) GUESS=$UNAME_MACHINE-unknown-mirbsd$UNAME_RELEASE ;; *:Sortix:*:*) GUESS=$UNAME_MACHINE-unknown-sortix ;; *:Twizzler:*:*) GUESS=$UNAME_MACHINE-unknown-twizzler ;; *:Redox:*:*) GUESS=$UNAME_MACHINE-unknown-redox ;; mips:OSF1:*.*) GUESS=mips-dec-osf1 ;; alpha:OSF1:*:*) # Reset EXIT trap before exiting to avoid spurious non-zero exit code. trap '' 0 case $UNAME_RELEASE in *4.0) UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $3}'` ;; *5.*) UNAME_RELEASE=`/usr/sbin/sizer -v | awk '{print $4}'` ;; esac # According to Compaq, /usr/sbin/psrinfo has been available on # OSF/1 and Tru64 systems produced since 1995. I hope that # covers most systems running today. This code pipes the CPU # types through head -n 1, so we only detect the type of CPU 0. ALPHA_CPU_TYPE=`/usr/sbin/psrinfo -v | sed -n -e 's/^ The alpha \(.*\) processor.*$/\1/p' | head -n 1` case $ALPHA_CPU_TYPE in "EV4 (21064)") UNAME_MACHINE=alpha ;; "EV4.5 (21064)") UNAME_MACHINE=alpha ;; "LCA4 (21066/21068)") UNAME_MACHINE=alpha ;; "EV5 (21164)") UNAME_MACHINE=alphaev5 ;; "EV5.6 (21164A)") UNAME_MACHINE=alphaev56 ;; "EV5.6 (21164PC)") UNAME_MACHINE=alphapca56 ;; "EV5.7 (21164PC)") UNAME_MACHINE=alphapca57 ;; "EV6 (21264)") UNAME_MACHINE=alphaev6 ;; "EV6.7 (21264A)") UNAME_MACHINE=alphaev67 ;; "EV6.8CB (21264C)") UNAME_MACHINE=alphaev68 ;; "EV6.8AL (21264B)") UNAME_MACHINE=alphaev68 ;; "EV6.8CX (21264D)") UNAME_MACHINE=alphaev68 ;; "EV6.9A (21264/EV69A)") UNAME_MACHINE=alphaev69 ;; "EV7 (21364)") UNAME_MACHINE=alphaev7 ;; "EV7.9 (21364A)") UNAME_MACHINE=alphaev79 ;; esac # A Pn.n version is a patched version. # A Vn.n version is a released version. # A Tn.n version is a released field test version. # A Xn.n version is an unreleased experimental baselevel. # 1.2 uses "1.2" for uname -r. OSF_REL=`echo "$UNAME_RELEASE" | sed -e 's/^[PVTX]//' | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` GUESS=$UNAME_MACHINE-dec-osf$OSF_REL ;; Amiga*:UNIX_System_V:4.0:*) GUESS=m68k-unknown-sysv4 ;; *:[Aa]miga[Oo][Ss]:*:*) GUESS=$UNAME_MACHINE-unknown-amigaos ;; *:[Mm]orph[Oo][Ss]:*:*) GUESS=$UNAME_MACHINE-unknown-morphos ;; *:OS/390:*:*) GUESS=i370-ibm-openedition ;; *:z/VM:*:*) GUESS=s390-ibm-zvmoe ;; *:OS400:*:*) GUESS=powerpc-ibm-os400 ;; arm:RISC*:1.[012]*:*|arm:riscix:1.[012]*:*) GUESS=arm-acorn-riscix$UNAME_RELEASE ;; arm*:riscos:*:*|arm*:RISCOS:*:*) GUESS=arm-unknown-riscos ;; SR2?01:HI-UX/MPP:*:* | SR8000:HI-UX/MPP:*:*) GUESS=hppa1.1-hitachi-hiuxmpp ;; Pyramid*:OSx*:*:* | MIS*:OSx*:*:* | MIS*:SMP_DC-OSx*:*:*) # akee@wpdis03.wpafb.af.mil (Earle F. Ake) contributed MIS and NILE. case `(/bin/universe) 2>/dev/null` in att) GUESS=pyramid-pyramid-sysv3 ;; *) GUESS=pyramid-pyramid-bsd ;; esac ;; NILE*:*:*:dcosx) GUESS=pyramid-pyramid-svr4 ;; DRS?6000:unix:4.0:6*) GUESS=sparc-icl-nx6 ;; DRS?6000:UNIX_SV:4.2*:7* | DRS?6000:isis:4.2*:7*) case `/usr/bin/uname -p` in sparc) GUESS=sparc-icl-nx7 ;; esac ;; s390x:SunOS:*:*) SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` GUESS=$UNAME_MACHINE-ibm-solaris2$SUN_REL ;; sun4H:SunOS:5.*:*) SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` GUESS=sparc-hal-solaris2$SUN_REL ;; sun4*:SunOS:5.*:* | tadpole*:SunOS:5.*:*) SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` GUESS=sparc-sun-solaris2$SUN_REL ;; i86pc:AuroraUX:5.*:* | i86xen:AuroraUX:5.*:*) GUESS=i386-pc-auroraux$UNAME_RELEASE ;; i86pc:SunOS:5.*:* | i86xen:SunOS:5.*:*) set_cc_for_build SUN_ARCH=i386 # If there is a compiler, see if it is configured for 64-bit objects. # Note that the Sun cc does not turn __LP64__ into 1 like gcc does. # This test works for both compilers. if test "$CC_FOR_BUILD" != no_compiler_found; then if (echo '#ifdef __amd64'; echo IS_64BIT_ARCH; echo '#endif') | \ (CCOPTS="" $CC_FOR_BUILD -m64 -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then SUN_ARCH=x86_64 fi fi SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` GUESS=$SUN_ARCH-pc-solaris2$SUN_REL ;; sun4*:SunOS:6*:*) # According to config.sub, this is the proper way to canonicalize # SunOS6. Hard to guess exactly what SunOS6 will be like, but # it's likely to be more like Solaris than SunOS4. SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` GUESS=sparc-sun-solaris3$SUN_REL ;; sun4*:SunOS:*:*) case `/usr/bin/arch -k` in Series*|S4*) UNAME_RELEASE=`uname -v` ;; esac # Japanese Language versions have a version number like '4.1.3-JL'. SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/-/_/'` GUESS=sparc-sun-sunos$SUN_REL ;; sun3*:SunOS:*:*) GUESS=m68k-sun-sunos$UNAME_RELEASE ;; sun*:*:4.2BSD:*) UNAME_RELEASE=`(sed 1q /etc/motd | awk '{print substr($5,1,3)}') 2>/dev/null` test "x$UNAME_RELEASE" = x && UNAME_RELEASE=3 case `/bin/arch` in sun3) GUESS=m68k-sun-sunos$UNAME_RELEASE ;; sun4) GUESS=sparc-sun-sunos$UNAME_RELEASE ;; esac ;; aushp:SunOS:*:*) GUESS=sparc-auspex-sunos$UNAME_RELEASE ;; # The situation for MiNT is a little confusing. The machine name # can be virtually everything (everything which is not # "atarist" or "atariste" at least should have a processor # > m68000). The system name ranges from "MiNT" over "FreeMiNT" # to the lowercase version "mint" (or "freemint"). Finally # the system name "TOS" denotes a system which is actually not # MiNT. But MiNT is downward compatible to TOS, so this should # be no problem. atarist[e]:*MiNT:*:* | atarist[e]:*mint:*:* | atarist[e]:*TOS:*:*) GUESS=m68k-atari-mint$UNAME_RELEASE ;; atari*:*MiNT:*:* | atari*:*mint:*:* | atarist[e]:*TOS:*:*) GUESS=m68k-atari-mint$UNAME_RELEASE ;; *falcon*:*MiNT:*:* | *falcon*:*mint:*:* | *falcon*:*TOS:*:*) GUESS=m68k-atari-mint$UNAME_RELEASE ;; milan*:*MiNT:*:* | milan*:*mint:*:* | *milan*:*TOS:*:*) GUESS=m68k-milan-mint$UNAME_RELEASE ;; hades*:*MiNT:*:* | hades*:*mint:*:* | *hades*:*TOS:*:*) GUESS=m68k-hades-mint$UNAME_RELEASE ;; *:*MiNT:*:* | *:*mint:*:* | *:*TOS:*:*) GUESS=m68k-unknown-mint$UNAME_RELEASE ;; m68k:machten:*:*) GUESS=m68k-apple-machten$UNAME_RELEASE ;; powerpc:machten:*:*) GUESS=powerpc-apple-machten$UNAME_RELEASE ;; RISC*:Mach:*:*) GUESS=mips-dec-mach_bsd4.3 ;; RISC*:ULTRIX:*:*) GUESS=mips-dec-ultrix$UNAME_RELEASE ;; VAX*:ULTRIX*:*:*) GUESS=vax-dec-ultrix$UNAME_RELEASE ;; 2020:CLIX:*:* | 2430:CLIX:*:*) GUESS=clipper-intergraph-clix$UNAME_RELEASE ;; mips:*:*:UMIPS | mips:*:*:RISCos) set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #ifdef __cplusplus #include <stdio.h> /* for printf() prototype */ int main (int argc, char *argv[]) { #else int main (argc, argv) int argc; char *argv[]; { #endif #if defined (host_mips) && defined (MIPSEB) #if defined (SYSTYPE_SYSV) printf ("mips-mips-riscos%ssysv\\n", argv[1]); exit (0); #endif #if defined (SYSTYPE_SVR4) printf ("mips-mips-riscos%ssvr4\\n", argv[1]); exit (0); #endif #if defined (SYSTYPE_BSD43) || defined(SYSTYPE_BSD) printf ("mips-mips-riscos%sbsd\\n", argv[1]); exit (0); #endif #endif exit (-1); } EOF $CC_FOR_BUILD -o "$dummy" "$dummy.c" && dummyarg=`echo "$UNAME_RELEASE" | sed -n 's/\([0-9]*\).*/\1/p'` && SYSTEM_NAME=`"$dummy" "$dummyarg"` && { echo "$SYSTEM_NAME"; exit; } GUESS=mips-mips-riscos$UNAME_RELEASE ;; Motorola:PowerMAX_OS:*:*) GUESS=powerpc-motorola-powermax ;; Motorola:*:4.3:PL8-*) GUESS=powerpc-harris-powermax ;; Night_Hawk:*:*:PowerMAX_OS | Synergy:PowerMAX_OS:*:*) GUESS=powerpc-harris-powermax ;; Night_Hawk:Power_UNIX:*:*) GUESS=powerpc-harris-powerunix ;; m88k:CX/UX:7*:*) GUESS=m88k-harris-cxux7 ;; m88k:*:4*:R4*) GUESS=m88k-motorola-sysv4 ;; m88k:*:3*:R3*) GUESS=m88k-motorola-sysv3 ;; AViiON:dgux:*:*) # DG/UX returns AViiON for all architectures UNAME_PROCESSOR=`/usr/bin/uname -p` if test "$UNAME_PROCESSOR" = mc88100 || test "$UNAME_PROCESSOR" = mc88110 then if test "$TARGET_BINARY_INTERFACE"x = m88kdguxelfx || \ test "$TARGET_BINARY_INTERFACE"x = x then GUESS=m88k-dg-dgux$UNAME_RELEASE else GUESS=m88k-dg-dguxbcs$UNAME_RELEASE fi else GUESS=i586-dg-dgux$UNAME_RELEASE fi ;; M88*:DolphinOS:*:*) # DolphinOS (SVR3) GUESS=m88k-dolphin-sysv3 ;; M88*:*:R3*:*) # Delta 88k system running SVR3 GUESS=m88k-motorola-sysv3 ;; XD88*:*:*:*) # Tektronix XD88 system running UTekV (SVR3) GUESS=m88k-tektronix-sysv3 ;; Tek43[0-9][0-9]:UTek:*:*) # Tektronix 4300 system running UTek (BSD) GUESS=m68k-tektronix-bsd ;; *:IRIX*:*:*) IRIX_REL=`echo "$UNAME_RELEASE" | sed -e 's/-/_/g'` GUESS=mips-sgi-irix$IRIX_REL ;; ????????:AIX?:[12].1:2) # AIX 2.2.1 or AIX 2.1.1 is RT/PC AIX. GUESS=romp-ibm-aix # uname -m gives an 8 hex-code CPU id ;; # Note that: echo "'`uname -s`'" gives 'AIX ' i*86:AIX:*:*) GUESS=i386-ibm-aix ;; ia64:AIX:*:*) if test -x /usr/bin/oslevel ; then IBM_REV=`/usr/bin/oslevel` else IBM_REV=$UNAME_VERSION.$UNAME_RELEASE fi GUESS=$UNAME_MACHINE-ibm-aix$IBM_REV ;; *:AIX:2:3) if grep bos325 /usr/include/stdio.h >/dev/null 2>&1; then set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #include <sys/systemcfg.h> int main () { if (!__power_pc()) exit(1); puts("powerpc-ibm-aix3.2.5"); exit(0); } EOF if $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` then GUESS=$SYSTEM_NAME else GUESS=rs6000-ibm-aix3.2.5 fi elif grep bos324 /usr/include/stdio.h >/dev/null 2>&1; then GUESS=rs6000-ibm-aix3.2.4 else GUESS=rs6000-ibm-aix3.2 fi ;; *:AIX:*:[4567]) IBM_CPU_ID=`/usr/sbin/lsdev -C -c processor -S available | sed 1q | awk '{ print $1 }'` if /usr/sbin/lsattr -El "$IBM_CPU_ID" | grep ' POWER' >/dev/null 2>&1; then IBM_ARCH=rs6000 else IBM_ARCH=powerpc fi if test -x /usr/bin/lslpp ; then IBM_REV=`/usr/bin/lslpp -Lqc bos.rte.libc | \ awk -F: '{ print $3 }' | sed s/[0-9]*$/0/` else IBM_REV=$UNAME_VERSION.$UNAME_RELEASE fi GUESS=$IBM_ARCH-ibm-aix$IBM_REV ;; *:AIX:*:*) GUESS=rs6000-ibm-aix ;; ibmrt:4.4BSD:*|romp-ibm:4.4BSD:*) GUESS=romp-ibm-bsd4.4 ;; ibmrt:*BSD:*|romp-ibm:BSD:*) # covers RT/PC BSD and GUESS=romp-ibm-bsd$UNAME_RELEASE # 4.3 with uname added to ;; # report: romp-ibm BSD 4.3 *:BOSX:*:*) GUESS=rs6000-bull-bosx ;; DPX/2?00:B.O.S.:*:*) GUESS=m68k-bull-sysv3 ;; 9000/[34]??:4.3bsd:1.*:*) GUESS=m68k-hp-bsd ;; hp300:4.4BSD:*:* | 9000/[34]??:4.3bsd:2.*:*) GUESS=m68k-hp-bsd4.4 ;; 9000/[34678]??:HP-UX:*:*) HPUX_REV=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*.[0B]*//'` case $UNAME_MACHINE in 9000/31?) HP_ARCH=m68000 ;; 9000/[34]??) HP_ARCH=m68k ;; 9000/[678][0-9][0-9]) if test -x /usr/bin/getconf; then sc_cpu_version=`/usr/bin/getconf SC_CPU_VERSION 2>/dev/null` sc_kernel_bits=`/usr/bin/getconf SC_KERNEL_BITS 2>/dev/null` case $sc_cpu_version in 523) HP_ARCH=hppa1.0 ;; # CPU_PA_RISC1_0 528) HP_ARCH=hppa1.1 ;; # CPU_PA_RISC1_1 532) # CPU_PA_RISC2_0 case $sc_kernel_bits in 32) HP_ARCH=hppa2.0n ;; 64) HP_ARCH=hppa2.0w ;; '') HP_ARCH=hppa2.0 ;; # HP-UX 10.20 esac ;; esac fi if test "$HP_ARCH" = ""; then set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #define _HPUX_SOURCE #include <stdlib.h> #include <unistd.h> int main () { #if defined(_SC_KERNEL_BITS) long bits = sysconf(_SC_KERNEL_BITS); #endif long cpu = sysconf (_SC_CPU_VERSION); switch (cpu) { case CPU_PA_RISC1_0: puts ("hppa1.0"); break; case CPU_PA_RISC1_1: puts ("hppa1.1"); break; case CPU_PA_RISC2_0: #if defined(_SC_KERNEL_BITS) switch (bits) { case 64: puts ("hppa2.0w"); break; case 32: puts ("hppa2.0n"); break; default: puts ("hppa2.0"); break; } break; #else /* !defined(_SC_KERNEL_BITS) */ puts ("hppa2.0"); break; #endif default: puts ("hppa1.0"); break; } exit (0); } EOF (CCOPTS="" $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null) && HP_ARCH=`"$dummy"` test -z "$HP_ARCH" && HP_ARCH=hppa fi ;; esac if test "$HP_ARCH" = hppa2.0w then set_cc_for_build # hppa2.0w-hp-hpux* has a 64-bit kernel and a compiler generating # 32-bit code. hppa64-hp-hpux* has the same kernel and a compiler # generating 64-bit code. GNU and HP use different nomenclature: # # $ CC_FOR_BUILD=cc ./config.guess # => hppa2.0w-hp-hpux11.23 # $ CC_FOR_BUILD="cc +DA2.0w" ./config.guess # => hppa64-hp-hpux11.23 if echo __LP64__ | (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | grep -q __LP64__ then HP_ARCH=hppa2.0w else HP_ARCH=hppa64 fi fi GUESS=$HP_ARCH-hp-hpux$HPUX_REV ;; ia64:HP-UX:*:*) HPUX_REV=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*.[0B]*//'` GUESS=ia64-hp-hpux$HPUX_REV ;; 3050*:HI-UX:*:*) set_cc_for_build sed 's/^ //' << EOF > "$dummy.c" #include <unistd.h> int main () { long cpu = sysconf (_SC_CPU_VERSION); /* The order matters, because CPU_IS_HP_MC68K erroneously returns true for CPU_PA_RISC1_0. CPU_IS_PA_RISC returns correct results, however. */ if (CPU_IS_PA_RISC (cpu)) { switch (cpu) { case CPU_PA_RISC1_0: puts ("hppa1.0-hitachi-hiuxwe2"); break; case CPU_PA_RISC1_1: puts ("hppa1.1-hitachi-hiuxwe2"); break; case CPU_PA_RISC2_0: puts ("hppa2.0-hitachi-hiuxwe2"); break; default: puts ("hppa-hitachi-hiuxwe2"); break; } } else if (CPU_IS_HP_MC68K (cpu)) puts ("m68k-hitachi-hiuxwe2"); else puts ("unknown-hitachi-hiuxwe2"); exit (0); } EOF $CC_FOR_BUILD -o "$dummy" "$dummy.c" && SYSTEM_NAME=`"$dummy"` && { echo "$SYSTEM_NAME"; exit; } GUESS=unknown-hitachi-hiuxwe2 ;; 9000/7??:4.3bsd:*:* | 9000/8?[79]:4.3bsd:*:*) GUESS=hppa1.1-hp-bsd ;; 9000/8??:4.3bsd:*:*) GUESS=hppa1.0-hp-bsd ;; *9??*:MPE/iX:*:* | *3000*:MPE/iX:*:*) GUESS=hppa1.0-hp-mpeix ;; hp7??:OSF1:*:* | hp8?[79]:OSF1:*:*) GUESS=hppa1.1-hp-osf ;; hp8??:OSF1:*:*) GUESS=hppa1.0-hp-osf ;; i*86:OSF1:*:*) if test -x /usr/sbin/sysversion ; then GUESS=$UNAME_MACHINE-unknown-osf1mk else GUESS=$UNAME_MACHINE-unknown-osf1 fi ;; parisc*:Lites*:*:*) GUESS=hppa1.1-hp-lites ;; C1*:ConvexOS:*:* | convex:ConvexOS:C1*:*) GUESS=c1-convex-bsd ;; C2*:ConvexOS:*:* | convex:ConvexOS:C2*:*) if getsysinfo -f scalar_acc then echo c32-convex-bsd else echo c2-convex-bsd fi exit ;; C34*:ConvexOS:*:* | convex:ConvexOS:C34*:*) GUESS=c34-convex-bsd ;; C38*:ConvexOS:*:* | convex:ConvexOS:C38*:*) GUESS=c38-convex-bsd ;; C4*:ConvexOS:*:* | convex:ConvexOS:C4*:*) GUESS=c4-convex-bsd ;; CRAY*Y-MP:*:*:*) CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` GUESS=ymp-cray-unicos$CRAY_REL ;; CRAY*[A-Z]90:*:*:*) echo "$UNAME_MACHINE"-cray-unicos"$UNAME_RELEASE" \ | sed -e 's/CRAY.*\([A-Z]90\)/\1/' \ -e y/ABCDEFGHIJKLMNOPQRSTUVWXYZ/abcdefghijklmnopqrstuvwxyz/ \ -e 's/\.[^.]*$/.X/' exit ;; CRAY*TS:*:*:*) CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` GUESS=t90-cray-unicos$CRAY_REL ;; CRAY*T3E:*:*:*) CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` GUESS=alphaev5-cray-unicosmk$CRAY_REL ;; CRAY*SV1:*:*:*) CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` GUESS=sv1-cray-unicos$CRAY_REL ;; *:UNICOS/mp:*:*) CRAY_REL=`echo "$UNAME_RELEASE" | sed -e 's/\.[^.]*$/.X/'` GUESS=craynv-cray-unicosmp$CRAY_REL ;; F30[01]:UNIX_System_V:*:* | F700:UNIX_System_V:*:*) FUJITSU_PROC=`uname -m | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz` FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` FUJITSU_REL=`echo "$UNAME_RELEASE" | sed -e 's/ /_/'` GUESS=${FUJITSU_PROC}-fujitsu-${FUJITSU_SYS}${FUJITSU_REL} ;; 5000:UNIX_System_V:4.*:*) FUJITSU_SYS=`uname -p | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/\///'` FUJITSU_REL=`echo "$UNAME_RELEASE" | tr ABCDEFGHIJKLMNOPQRSTUVWXYZ abcdefghijklmnopqrstuvwxyz | sed -e 's/ /_/'` GUESS=sparc-fujitsu-${FUJITSU_SYS}${FUJITSU_REL} ;; i*86:BSD/386:*:* | i*86:BSD/OS:*:* | *:Ascend\ Embedded/OS:*:*) GUESS=$UNAME_MACHINE-pc-bsdi$UNAME_RELEASE ;; sparc*:BSD/OS:*:*) GUESS=sparc-unknown-bsdi$UNAME_RELEASE ;; *:BSD/OS:*:*) GUESS=$UNAME_MACHINE-unknown-bsdi$UNAME_RELEASE ;; arm:FreeBSD:*:*) UNAME_PROCESSOR=`uname -p` set_cc_for_build if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_PCS_VFP then FREEBSD_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` GUESS=$UNAME_PROCESSOR-unknown-freebsd$FREEBSD_REL-gnueabi else FREEBSD_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` GUESS=$UNAME_PROCESSOR-unknown-freebsd$FREEBSD_REL-gnueabihf fi ;; *:FreeBSD:*:*) UNAME_PROCESSOR=`uname -p` case $UNAME_PROCESSOR in amd64) UNAME_PROCESSOR=x86_64 ;; i386) UNAME_PROCESSOR=i586 ;; esac FREEBSD_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` GUESS=$UNAME_PROCESSOR-unknown-freebsd$FREEBSD_REL ;; i*:CYGWIN*:*) GUESS=$UNAME_MACHINE-pc-cygwin ;; *:MINGW64*:*) GUESS=$UNAME_MACHINE-pc-mingw64 ;; *:MINGW*:*) GUESS=$UNAME_MACHINE-pc-mingw32 ;; *:MSYS*:*) GUESS=$UNAME_MACHINE-pc-msys ;; i*:PW*:*) GUESS=$UNAME_MACHINE-pc-pw32 ;; *:SerenityOS:*:*) GUESS=$UNAME_MACHINE-pc-serenity ;; *:Interix*:*) case $UNAME_MACHINE in x86) GUESS=i586-pc-interix$UNAME_RELEASE ;; authenticamd | genuineintel | EM64T) GUESS=x86_64-unknown-interix$UNAME_RELEASE ;; IA64) GUESS=ia64-unknown-interix$UNAME_RELEASE ;; esac ;; i*:UWIN*:*) GUESS=$UNAME_MACHINE-pc-uwin ;; amd64:CYGWIN*:*:* | x86_64:CYGWIN*:*:*) GUESS=x86_64-pc-cygwin ;; prep*:SunOS:5.*:*) SUN_REL=`echo "$UNAME_RELEASE" | sed -e 's/[^.]*//'` GUESS=powerpcle-unknown-solaris2$SUN_REL ;; *:GNU:*:*) # the GNU system GNU_ARCH=`echo "$UNAME_MACHINE" | sed -e 's,[-/].*$,,'` GNU_REL=`echo "$UNAME_RELEASE" | sed -e 's,/.*$,,'` GUESS=$GNU_ARCH-unknown-$LIBC$GNU_REL ;; *:GNU/*:*:*) # other systems with GNU libc and userland GNU_SYS=`echo "$UNAME_SYSTEM" | sed 's,^[^/]*/,,' | tr "[:upper:]" "[:lower:]"` GNU_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` GUESS=$UNAME_MACHINE-unknown-$GNU_SYS$GNU_REL-$LIBC ;; x86_64:[Mm]anagarm:*:*|i?86:[Mm]anagarm:*:*) GUESS="$UNAME_MACHINE-pc-managarm-mlibc" ;; *:[Mm]anagarm:*:*) GUESS="$UNAME_MACHINE-unknown-managarm-mlibc" ;; *:Minix:*:*) GUESS=$UNAME_MACHINE-unknown-minix ;; aarch64:Linux:*:*) set_cc_for_build CPU=$UNAME_MACHINE LIBCABI=$LIBC if test "$CC_FOR_BUILD" != no_compiler_found; then ABI=64 sed 's/^ //' << EOF > "$dummy.c" #ifdef __ARM_EABI__ #ifdef __ARM_PCS_VFP ABI=eabihf #else ABI=eabi #endif #endif EOF cc_set_abi=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^ABI' | sed 's, ,,g'` eval "$cc_set_abi" case $ABI in eabi | eabihf) CPU=armv8l; LIBCABI=$LIBC$ABI ;; esac fi GUESS=$CPU-unknown-linux-$LIBCABI ;; aarch64_be:Linux:*:*) UNAME_MACHINE=aarch64_be GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; alpha:Linux:*:*) case `sed -n '/^cpu model/s/^.*: \(.*\)/\1/p' /proc/cpuinfo 2>/dev/null` in EV5) UNAME_MACHINE=alphaev5 ;; EV56) UNAME_MACHINE=alphaev56 ;; PCA56) UNAME_MACHINE=alphapca56 ;; PCA57) UNAME_MACHINE=alphapca56 ;; EV6) UNAME_MACHINE=alphaev6 ;; EV67) UNAME_MACHINE=alphaev67 ;; EV68*) UNAME_MACHINE=alphaev68 ;; esac objdump --private-headers /bin/sh | grep -q ld.so.1 if test "$?" = 0 ; then LIBC=gnulibc1 ; fi GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; arc:Linux:*:* | arceb:Linux:*:* | arc32:Linux:*:* | arc64:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; arm*:Linux:*:*) set_cc_for_build if echo __ARM_EABI__ | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_EABI__ then GUESS=$UNAME_MACHINE-unknown-linux-$LIBC else if echo __ARM_PCS_VFP | $CC_FOR_BUILD -E - 2>/dev/null \ | grep -q __ARM_PCS_VFP then GUESS=$UNAME_MACHINE-unknown-linux-${LIBC}eabi else GUESS=$UNAME_MACHINE-unknown-linux-${LIBC}eabihf fi fi ;; avr32*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; cris:Linux:*:*) GUESS=$UNAME_MACHINE-axis-linux-$LIBC ;; crisv32:Linux:*:*) GUESS=$UNAME_MACHINE-axis-linux-$LIBC ;; e2k:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; frv:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; hexagon:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; i*86:Linux:*:*) GUESS=$UNAME_MACHINE-pc-linux-$LIBC ;; ia64:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; k1om:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; kvx:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; kvx:cos:*:*) GUESS=$UNAME_MACHINE-unknown-cos ;; kvx:mbr:*:*) GUESS=$UNAME_MACHINE-unknown-mbr ;; loongarch32:Linux:*:* | loongarch64:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; m32r*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; m68*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; mips:Linux:*:* | mips64:Linux:*:*) set_cc_for_build IS_GLIBC=0 test x"${LIBC}" = xgnu && IS_GLIBC=1 sed 's/^ //' << EOF > "$dummy.c" #undef CPU #undef mips #undef mipsel #undef mips64 #undef mips64el #if ${IS_GLIBC} && defined(_ABI64) LIBCABI=gnuabi64 #else #if ${IS_GLIBC} && defined(_ABIN32) LIBCABI=gnuabin32 #else LIBCABI=${LIBC} #endif #endif #if ${IS_GLIBC} && defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6 CPU=mipsisa64r6 #else #if ${IS_GLIBC} && !defined(__mips64) && defined(__mips_isa_rev) && __mips_isa_rev>=6 CPU=mipsisa32r6 #else #if defined(__mips64) CPU=mips64 #else CPU=mips #endif #endif #endif #if defined(__MIPSEL__) || defined(__MIPSEL) || defined(_MIPSEL) || defined(MIPSEL) MIPS_ENDIAN=el #else #if defined(__MIPSEB__) || defined(__MIPSEB) || defined(_MIPSEB) || defined(MIPSEB) MIPS_ENDIAN= #else MIPS_ENDIAN= #endif #endif EOF cc_set_vars=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^CPU\|^MIPS_ENDIAN\|^LIBCABI'` eval "$cc_set_vars" test "x$CPU" != x && { echo "$CPU${MIPS_ENDIAN}-unknown-linux-$LIBCABI"; exit; } ;; mips64el:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; openrisc*:Linux:*:*) GUESS=or1k-unknown-linux-$LIBC ;; or32:Linux:*:* | or1k*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; padre:Linux:*:*) GUESS=sparc-unknown-linux-$LIBC ;; parisc64:Linux:*:* | hppa64:Linux:*:*) GUESS=hppa64-unknown-linux-$LIBC ;; parisc:Linux:*:* | hppa:Linux:*:*) # Look for CPU level case `grep '^cpu[^a-z]*:' /proc/cpuinfo 2>/dev/null | cut -d' ' -f2` in PA7*) GUESS=hppa1.1-unknown-linux-$LIBC ;; PA8*) GUESS=hppa2.0-unknown-linux-$LIBC ;; *) GUESS=hppa-unknown-linux-$LIBC ;; esac ;; ppc64:Linux:*:*) GUESS=powerpc64-unknown-linux-$LIBC ;; ppc:Linux:*:*) GUESS=powerpc-unknown-linux-$LIBC ;; ppc64le:Linux:*:*) GUESS=powerpc64le-unknown-linux-$LIBC ;; ppcle:Linux:*:*) GUESS=powerpcle-unknown-linux-$LIBC ;; riscv32:Linux:*:* | riscv32be:Linux:*:* | riscv64:Linux:*:* | riscv64be:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; s390:Linux:*:* | s390x:Linux:*:*) GUESS=$UNAME_MACHINE-ibm-linux-$LIBC ;; sh64*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; sh*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; sparc:Linux:*:* | sparc64:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; tile*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; vax:Linux:*:*) GUESS=$UNAME_MACHINE-dec-linux-$LIBC ;; x86_64:Linux:*:*) set_cc_for_build CPU=$UNAME_MACHINE LIBCABI=$LIBC if test "$CC_FOR_BUILD" != no_compiler_found; then ABI=64 sed 's/^ //' << EOF > "$dummy.c" #ifdef __i386__ ABI=x86 #else #ifdef __ILP32__ ABI=x32 #endif #endif EOF cc_set_abi=`$CC_FOR_BUILD -E "$dummy.c" 2>/dev/null | grep '^ABI' | sed 's, ,,g'` eval "$cc_set_abi" case $ABI in x86) CPU=i686 ;; x32) LIBCABI=${LIBC}x32 ;; esac fi GUESS=$CPU-pc-linux-$LIBCABI ;; xtensa*:Linux:*:*) GUESS=$UNAME_MACHINE-unknown-linux-$LIBC ;; i*86:DYNIX/ptx:4*:*) # ptx 4.0 does uname -s correctly, with DYNIX/ptx in there. # earlier versions are messed up and put the nodename in both # sysname and nodename. GUESS=i386-sequent-sysv4 ;; i*86:UNIX_SV:4.2MP:2.*) # Unixware is an offshoot of SVR4, but it has its own version # number series starting with 2... # I am not positive that other SVR4 systems won't match this, # I just have to hope. -- rms. # Use sysv4.2uw... so that sysv4* matches it. GUESS=$UNAME_MACHINE-pc-sysv4.2uw$UNAME_VERSION ;; i*86:OS/2:*:*) # If we were able to find 'uname', then EMX Unix compatibility # is probably installed. GUESS=$UNAME_MACHINE-pc-os2-emx ;; i*86:XTS-300:*:STOP) GUESS=$UNAME_MACHINE-unknown-stop ;; i*86:atheos:*:*) GUESS=$UNAME_MACHINE-unknown-atheos ;; i*86:syllable:*:*) GUESS=$UNAME_MACHINE-pc-syllable ;; i*86:LynxOS:2.*:* | i*86:LynxOS:3.[01]*:* | i*86:LynxOS:4.[02]*:*) GUESS=i386-unknown-lynxos$UNAME_RELEASE ;; i*86:*DOS:*:*) GUESS=$UNAME_MACHINE-pc-msdosdjgpp ;; i*86:*:4.*:*) UNAME_REL=`echo "$UNAME_RELEASE" | sed 's/\/MP$//'` if grep Novell /usr/include/link.h >/dev/null 2>/dev/null; then GUESS=$UNAME_MACHINE-univel-sysv$UNAME_REL else GUESS=$UNAME_MACHINE-pc-sysv$UNAME_REL fi ;; i*86:*:5:[678]*) # UnixWare 7.x, OpenUNIX and OpenServer 6. case `/bin/uname -X | grep "^Machine"` in *486*) UNAME_MACHINE=i486 ;; *Pentium) UNAME_MACHINE=i586 ;; *Pent*|*Celeron) UNAME_MACHINE=i686 ;; esac GUESS=$UNAME_MACHINE-unknown-sysv${UNAME_RELEASE}${UNAME_SYSTEM}${UNAME_VERSION} ;; i*86:*:3.2:*) if test -f /usr/options/cb.name; then UNAME_REL=`sed -n 's/.*Version //p' </usr/options/cb.name` GUESS=$UNAME_MACHINE-pc-isc$UNAME_REL elif /bin/uname -X 2>/dev/null >/dev/null ; then UNAME_REL=`(/bin/uname -X|grep Release|sed -e 's/.*= //')` (/bin/uname -X|grep i80486 >/dev/null) && UNAME_MACHINE=i486 (/bin/uname -X|grep '^Machine.*Pentium' >/dev/null) \ && UNAME_MACHINE=i586 (/bin/uname -X|grep '^Machine.*Pent *II' >/dev/null) \ && UNAME_MACHINE=i686 (/bin/uname -X|grep '^Machine.*Pentium Pro' >/dev/null) \ && UNAME_MACHINE=i686 GUESS=$UNAME_MACHINE-pc-sco$UNAME_REL else GUESS=$UNAME_MACHINE-pc-sysv32 fi ;; pc:*:*:*) # Left here for compatibility: # uname -m prints for DJGPP always 'pc', but it prints nothing about # the processor, so we play safe by assuming i586. # Note: whatever this is, it MUST be the same as what config.sub # prints for the "djgpp" host, or else GDB configure will decide that # this is a cross-build. GUESS=i586-pc-msdosdjgpp ;; Intel:Mach:3*:*) GUESS=i386-pc-mach3 ;; paragon:*:*:*) GUESS=i860-intel-osf1 ;; i860:*:4.*:*) # i860-SVR4 if grep Stardent /usr/include/sys/uadmin.h >/dev/null 2>&1 ; then GUESS=i860-stardent-sysv$UNAME_RELEASE # Stardent Vistra i860-SVR4 else # Add other i860-SVR4 vendors below as they are discovered. GUESS=i860-unknown-sysv$UNAME_RELEASE # Unknown i860-SVR4 fi ;; mini*:CTIX:SYS*5:*) # "miniframe" GUESS=m68010-convergent-sysv ;; mc68k:UNIX:SYSTEM5:3.51m) GUESS=m68k-convergent-sysv ;; M680?0:D-NIX:5.3:*) GUESS=m68k-diab-dnix ;; M68*:*:R3V[5678]*:*) test -r /sysV68 && { echo 'm68k-motorola-sysv'; exit; } ;; 3[345]??:*:4.0:3.0 | 3[34]??A:*:4.0:3.0 | 3[34]??,*:*:4.0:3.0 | 3[34]??/*:*:4.0:3.0 | 4400:*:4.0:3.0 | 4850:*:4.0:3.0 | SKA40:*:4.0:3.0 | SDS2:*:4.0:3.0 | SHG2:*:4.0:3.0 | S7501*:*:4.0:3.0) OS_REL='' test -r /etc/.relid \ && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4.3"$OS_REL"; exit; } /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } ;; 3[34]??:*:4.0:* | 3[34]??,*:*:4.0:*) /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4; exit; } ;; NCR*:*:4.2:* | MPRAS*:*:4.2:*) OS_REL='.3' test -r /etc/.relid \ && OS_REL=.`sed -n 's/[^ ]* [^ ]* \([0-9][0-9]\).*/\1/p' < /etc/.relid` /bin/uname -p 2>/dev/null | grep 86 >/dev/null \ && { echo i486-ncr-sysv4.3"$OS_REL"; exit; } /bin/uname -p 2>/dev/null | /bin/grep entium >/dev/null \ && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } /bin/uname -p 2>/dev/null | /bin/grep pteron >/dev/null \ && { echo i586-ncr-sysv4.3"$OS_REL"; exit; } ;; m68*:LynxOS:2.*:* | m68*:LynxOS:3.0*:*) GUESS=m68k-unknown-lynxos$UNAME_RELEASE ;; mc68030:UNIX_System_V:4.*:*) GUESS=m68k-atari-sysv4 ;; TSUNAMI:LynxOS:2.*:*) GUESS=sparc-unknown-lynxos$UNAME_RELEASE ;; rs6000:LynxOS:2.*:*) GUESS=rs6000-unknown-lynxos$UNAME_RELEASE ;; PowerPC:LynxOS:2.*:* | PowerPC:LynxOS:3.[01]*:* | PowerPC:LynxOS:4.[02]*:*) GUESS=powerpc-unknown-lynxos$UNAME_RELEASE ;; SM[BE]S:UNIX_SV:*:*) GUESS=mips-dde-sysv$UNAME_RELEASE ;; RM*:ReliantUNIX-*:*:*) GUESS=mips-sni-sysv4 ;; RM*:SINIX-*:*:*) GUESS=mips-sni-sysv4 ;; *:SINIX-*:*:*) if uname -p 2>/dev/null >/dev/null ; then UNAME_MACHINE=`(uname -p) 2>/dev/null` GUESS=$UNAME_MACHINE-sni-sysv4 else GUESS=ns32k-sni-sysv fi ;; PENTIUM:*:4.0*:*) # Unisys 'ClearPath HMP IX 4000' SVR4/MP effort # says <Richard.M.Bartel@ccMail.Census.GOV> GUESS=i586-unisys-sysv4 ;; *:UNIX_System_V:4*:FTX*) # From Gerald Hewes <hewes@openmarket.com>. # How about differentiating between stratus architectures? -djm GUESS=hppa1.1-stratus-sysv4 ;; *:*:*:FTX*) # From seanf@swdc.stratus.com. GUESS=i860-stratus-sysv4 ;; i*86:VOS:*:*) # From Paul.Green@stratus.com. GUESS=$UNAME_MACHINE-stratus-vos ;; *:VOS:*:*) # From Paul.Green@stratus.com. GUESS=hppa1.1-stratus-vos ;; mc68*:A/UX:*:*) GUESS=m68k-apple-aux$UNAME_RELEASE ;; news*:NEWS-OS:6*:*) GUESS=mips-sony-newsos6 ;; R[34]000:*System_V*:*:* | R4000:UNIX_SYSV:*:* | R*000:UNIX_SV:*:*) if test -d /usr/nec; then GUESS=mips-nec-sysv$UNAME_RELEASE else GUESS=mips-unknown-sysv$UNAME_RELEASE fi ;; BeBox:BeOS:*:*) # BeOS running on hardware made by Be, PPC only. GUESS=powerpc-be-beos ;; BeMac:BeOS:*:*) # BeOS running on Mac or Mac clone, PPC only. GUESS=powerpc-apple-beos ;; BePC:BeOS:*:*) # BeOS running on Intel PC compatible. GUESS=i586-pc-beos ;; BePC:Haiku:*:*) # Haiku running on Intel PC compatible. GUESS=i586-pc-haiku ;; ppc:Haiku:*:*) # Haiku running on Apple PowerPC GUESS=powerpc-apple-haiku ;; *:Haiku:*:*) # Haiku modern gcc (not bound by BeOS compat) GUESS=$UNAME_MACHINE-unknown-haiku ;; SX-4:SUPER-UX:*:*) GUESS=sx4-nec-superux$UNAME_RELEASE ;; SX-5:SUPER-UX:*:*) GUESS=sx5-nec-superux$UNAME_RELEASE ;; SX-6:SUPER-UX:*:*) GUESS=sx6-nec-superux$UNAME_RELEASE ;; SX-7:SUPER-UX:*:*) GUESS=sx7-nec-superux$UNAME_RELEASE ;; SX-8:SUPER-UX:*:*) GUESS=sx8-nec-superux$UNAME_RELEASE ;; SX-8R:SUPER-UX:*:*) GUESS=sx8r-nec-superux$UNAME_RELEASE ;; SX-ACE:SUPER-UX:*:*) GUESS=sxace-nec-superux$UNAME_RELEASE ;; Power*:Rhapsody:*:*) GUESS=powerpc-apple-rhapsody$UNAME_RELEASE ;; *:Rhapsody:*:*) GUESS=$UNAME_MACHINE-apple-rhapsody$UNAME_RELEASE ;; arm64:Darwin:*:*) GUESS=aarch64-apple-darwin$UNAME_RELEASE ;; *:Darwin:*:*) UNAME_PROCESSOR=`uname -p` case $UNAME_PROCESSOR in unknown) UNAME_PROCESSOR=powerpc ;; esac if command -v xcode-select > /dev/null 2> /dev/null && \ ! xcode-select --print-path > /dev/null 2> /dev/null ; then # Avoid executing cc if there is no toolchain installed as # cc will be a stub that puts up a graphical alert # prompting the user to install developer tools. CC_FOR_BUILD=no_compiler_found else set_cc_for_build fi if test "$CC_FOR_BUILD" != no_compiler_found; then if (echo '#ifdef __LP64__'; echo IS_64BIT_ARCH; echo '#endif') | \ (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_64BIT_ARCH >/dev/null then case $UNAME_PROCESSOR in i386) UNAME_PROCESSOR=x86_64 ;; powerpc) UNAME_PROCESSOR=powerpc64 ;; esac fi # On 10.4-10.6 one might compile for PowerPC via gcc -arch ppc if (echo '#ifdef __POWERPC__'; echo IS_PPC; echo '#endif') | \ (CCOPTS="" $CC_FOR_BUILD -E - 2>/dev/null) | \ grep IS_PPC >/dev/null then UNAME_PROCESSOR=powerpc fi elif test "$UNAME_PROCESSOR" = i386 ; then # uname -m returns i386 or x86_64 UNAME_PROCESSOR=$UNAME_MACHINE fi GUESS=$UNAME_PROCESSOR-apple-darwin$UNAME_RELEASE ;; *:procnto*:*:* | *:QNX:[0123456789]*:*) UNAME_PROCESSOR=`uname -p` if test "$UNAME_PROCESSOR" = x86; then UNAME_PROCESSOR=i386 UNAME_MACHINE=pc fi GUESS=$UNAME_PROCESSOR-$UNAME_MACHINE-nto-qnx$UNAME_RELEASE ;; *:QNX:*:4*) GUESS=i386-pc-qnx ;; NEO-*:NONSTOP_KERNEL:*:*) GUESS=neo-tandem-nsk$UNAME_RELEASE ;; NSE-*:NONSTOP_KERNEL:*:*) GUESS=nse-tandem-nsk$UNAME_RELEASE ;; NSR-*:NONSTOP_KERNEL:*:*) GUESS=nsr-tandem-nsk$UNAME_RELEASE ;; NSV-*:NONSTOP_KERNEL:*:*) GUESS=nsv-tandem-nsk$UNAME_RELEASE ;; NSX-*:NONSTOP_KERNEL:*:*) GUESS=nsx-tandem-nsk$UNAME_RELEASE ;; *:NonStop-UX:*:*) GUESS=mips-compaq-nonstopux ;; BS2000:POSIX*:*:*) GUESS=bs2000-siemens-sysv ;; DS/*:UNIX_System_V:*:*) GUESS=$UNAME_MACHINE-$UNAME_SYSTEM-$UNAME_RELEASE ;; *:Plan9:*:*) # "uname -m" is not consistent, so use $cputype instead. 386 # is converted to i386 for consistency with other x86 # operating systems. if test "${cputype-}" = 386; then UNAME_MACHINE=i386 elif test "x${cputype-}" != x; then UNAME_MACHINE=$cputype fi GUESS=$UNAME_MACHINE-unknown-plan9 ;; *:TOPS-10:*:*) GUESS=pdp10-unknown-tops10 ;; *:TENEX:*:*) GUESS=pdp10-unknown-tenex ;; KS10:TOPS-20:*:* | KL10:TOPS-20:*:* | TYPE4:TOPS-20:*:*) GUESS=pdp10-dec-tops20 ;; XKL-1:TOPS-20:*:* | TYPE5:TOPS-20:*:*) GUESS=pdp10-xkl-tops20 ;; *:TOPS-20:*:*) GUESS=pdp10-unknown-tops20 ;; *:ITS:*:*) GUESS=pdp10-unknown-its ;; SEI:*:*:SEIUX) GUESS=mips-sei-seiux$UNAME_RELEASE ;; *:DragonFly:*:*) DRAGONFLY_REL=`echo "$UNAME_RELEASE" | sed -e 's/[-(].*//'` GUESS=$UNAME_MACHINE-unknown-dragonfly$DRAGONFLY_REL ;; *:*VMS:*:*) UNAME_MACHINE=`(uname -p) 2>/dev/null` case $UNAME_MACHINE in A*) GUESS=alpha-dec-vms ;; I*) GUESS=ia64-dec-vms ;; V*) GUESS=vax-dec-vms ;; esac ;; *:XENIX:*:SysV) GUESS=i386-pc-xenix ;; i*86:skyos:*:*) SKYOS_REL=`echo "$UNAME_RELEASE" | sed -e 's/ .*$//'` GUESS=$UNAME_MACHINE-pc-skyos$SKYOS_REL ;; i*86:rdos:*:*) GUESS=$UNAME_MACHINE-pc-rdos ;; i*86:Fiwix:*:*) GUESS=$UNAME_MACHINE-pc-fiwix ;; *:AROS:*:*) GUESS=$UNAME_MACHINE-unknown-aros ;; x86_64:VMkernel:*:*) GUESS=$UNAME_MACHINE-unknown-esx ;; amd64:Isilon\ OneFS:*:*) GUESS=x86_64-unknown-onefs ;; *:Unleashed:*:*) GUESS=$UNAME_MACHINE-unknown-unleashed$UNAME_RELEASE ;; *:Ironclad:*:*) GUESS=$UNAME_MACHINE-unknown-ironclad ;; esac # Do we have a guess based on uname results? if test "x$GUESS" != x; then echo "$GUESS" exit fi # No uname command or uname output not recognized. set_cc_for_build cat > "$dummy.c" <<EOF #ifdef _SEQUENT_ #include <sys/types.h> #include <sys/utsname.h> #endif #if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__) #if defined (vax) || defined (__vax) || defined (__vax__) || defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__) #include <signal.h> #if defined(_SIZE_T_) || defined(SIGLOST) #include <sys/utsname.h> #endif #endif #endif int main () { #if defined (sony) #if defined (MIPSEB) /* BFD wants "bsd" instead of "newsos". Perhaps BFD should be changed, I don't know.... */ printf ("mips-sony-bsd\n"); exit (0); #else #include <sys/param.h> printf ("m68k-sony-newsos%s\n", #ifdef NEWSOS4 "4" #else "" #endif ); exit (0); #endif #endif #if defined (NeXT) #if !defined (__ARCHITECTURE__) #define __ARCHITECTURE__ "m68k" #endif int version; version=`(hostinfo | sed -n 's/.*NeXT Mach \([0-9]*\).*/\1/p') 2>/dev/null`; if (version < 4) printf ("%s-next-nextstep%d\n", __ARCHITECTURE__, version); else printf ("%s-next-openstep%d\n", __ARCHITECTURE__, version); exit (0); #endif #if defined (MULTIMAX) || defined (n16) #if defined (UMAXV) printf ("ns32k-encore-sysv\n"); exit (0); #else #if defined (CMU) printf ("ns32k-encore-mach\n"); exit (0); #else printf ("ns32k-encore-bsd\n"); exit (0); #endif #endif #endif #if defined (__386BSD__) printf ("i386-pc-bsd\n"); exit (0); #endif #if defined (sequent) #if defined (i386) printf ("i386-sequent-dynix\n"); exit (0); #endif #if defined (ns32000) printf ("ns32k-sequent-dynix\n"); exit (0); #endif #endif #if defined (_SEQUENT_) struct utsname un; uname(&un); if (strncmp(un.version, "V2", 2) == 0) { printf ("i386-sequent-ptx2\n"); exit (0); } if (strncmp(un.version, "V1", 2) == 0) { /* XXX is V1 correct? */ printf ("i386-sequent-ptx1\n"); exit (0); } printf ("i386-sequent-ptx\n"); exit (0); #endif #if defined (vax) #if !defined (ultrix) #include <sys/param.h> #if defined (BSD) #if BSD == 43 printf ("vax-dec-bsd4.3\n"); exit (0); #else #if BSD == 199006 printf ("vax-dec-bsd4.3reno\n"); exit (0); #else printf ("vax-dec-bsd\n"); exit (0); #endif #endif #else printf ("vax-dec-bsd\n"); exit (0); #endif #else #if defined(_SIZE_T_) || defined(SIGLOST) struct utsname un; uname (&un); printf ("vax-dec-ultrix%s\n", un.release); exit (0); #else printf ("vax-dec-ultrix\n"); exit (0); #endif #endif #endif #if defined(ultrix) || defined(_ultrix) || defined(__ultrix) || defined(__ultrix__) #if defined(mips) || defined(__mips) || defined(__mips__) || defined(MIPS) || defined(__MIPS__) #if defined(_SIZE_T_) || defined(SIGLOST) struct utsname *un; uname (&un); printf ("mips-dec-ultrix%s\n", un.release); exit (0); #else printf ("mips-dec-ultrix\n"); exit (0); #endif #endif #endif #if defined (alliant) && defined (i860) printf ("i860-alliant-bsd\n"); exit (0); #endif exit (1); } EOF $CC_FOR_BUILD -o "$dummy" "$dummy.c" 2>/dev/null && SYSTEM_NAME=`"$dummy"` && { echo "$SYSTEM_NAME"; exit; } # Apollos put the system type in the environment. test -d /usr/apollo && { echo "$ISP-apollo-$SYSTYPE"; exit; } echo "$0: unable to guess system type" >&2 case $UNAME_MACHINE:$UNAME_SYSTEM in mips:Linux | mips64:Linux) # If we got here on MIPS GNU/Linux, output extra information. cat >&2 <<EOF NOTE: MIPS GNU/Linux systems require a C compiler to fully recognize the system type. Please install a C compiler and try again. EOF ;; esac cat >&2 <<EOF This script (version $timestamp), has failed to recognize the operating system you are using. If your script is old, overwrite *all* copies of config.guess and config.sub with the latest versions from: https://git.savannah.gnu.org/cgit/config.git/plain/config.guess and https://git.savannah.gnu.org/cgit/config.git/plain/config.sub EOF our_year=`echo $timestamp | sed 's,-.*,,'` thisyear=`date +%Y` # shellcheck disable=SC2003 script_age=`expr "$thisyear" - "$our_year"` if test "$script_age" -lt 3 ; then cat >&2 <<EOF If $0 has already been updated, send the following data and any information you think might be pertinent to config-patches@gnu.org to provide the necessary information to handle your system. config.guess timestamp = $timestamp uname -m = `(uname -m) 2>/dev/null || echo unknown` uname -r = `(uname -r) 2>/dev/null || echo unknown` uname -s = `(uname -s) 2>/dev/null || echo unknown` uname -v = `(uname -v) 2>/dev/null || echo unknown` /usr/bin/uname -p = `(/usr/bin/uname -p) 2>/dev/null` /bin/uname -X = `(/bin/uname -X) 2>/dev/null` hostinfo = `(hostinfo) 2>/dev/null` /bin/universe = `(/bin/universe) 2>/dev/null` /usr/bin/arch -k = `(/usr/bin/arch -k) 2>/dev/null` /bin/arch = `(/bin/arch) 2>/dev/null` /usr/bin/oslevel = `(/usr/bin/oslevel) 2>/dev/null` /usr/convex/getsysinfo = `(/usr/convex/getsysinfo) 2>/dev/null` UNAME_MACHINE = "$UNAME_MACHINE" UNAME_RELEASE = "$UNAME_RELEASE" UNAME_SYSTEM = "$UNAME_SYSTEM" UNAME_VERSION = "$UNAME_VERSION" EOF fi exit 1 # Local variables: # eval: (add-hook 'before-save-hook 'time-stamp) # time-stamp-start: "timestamp='" # time-stamp-format: "%:y-%02m-%02d" # time-stamp-end: "'" # End: |
Added autosetup/autosetup-config.sub.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 | #! /bin/sh # Configuration validation subroutine script. # Copyright 1992-2024 Free Software Foundation, Inc. # shellcheck disable=SC2006,SC2268,SC2162 # see below for rationale timestamp='2024-05-27' # This file is free software; you can redistribute it and/or modify it # under the terms of the GNU General Public License as published by # the Free Software Foundation, either version 3 of the License, or # (at your option) any later version. # # This program is distributed in the hope that it will be useful, but # WITHOUT ANY WARRANTY; without even the implied warranty of # MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU # General Public License for more details. # # You should have received a copy of the GNU General Public License # along with this program; if not, see <https://www.gnu.org/licenses/>. # # As a special exception to the GNU General Public License, if you # distribute this file as part of a program that contains a # configuration script generated by Autoconf, you may include it under # the same distribution terms that you use for the rest of that # program. This Exception is an additional permission under section 7 # of the GNU General Public License, version 3 ("GPLv3"). # Please send patches to <config-patches@gnu.org>. # # Configuration subroutine to validate and canonicalize a configuration type. # Supply the specified configuration type as an argument. # If it is invalid, we print an error message on stderr and exit with code 1. # Otherwise, we print the canonical config type on stdout and succeed. # You can get the latest version of this script from: # https://git.savannah.gnu.org/cgit/config.git/plain/config.sub # This file is supposed to be the same for all GNU packages # and recognize all the CPU types, system types and aliases # that are meaningful with *any* GNU software. # Each package is responsible for reporting which valid configurations # it does not support. The user should be able to distinguish # a failure to support a valid configuration from a meaningless # configuration. # The goal of this file is to map all the various variations of a given # machine specification into a single specification in the form: # CPU_TYPE-MANUFACTURER-OPERATING_SYSTEM # or in some cases, the newer four-part form: # CPU_TYPE-MANUFACTURER-KERNEL-OPERATING_SYSTEM # It is wrong to echo any other type of specification. # The "shellcheck disable" line above the timestamp inhibits complaints # about features and limitations of the classic Bourne shell that were # superseded or lifted in POSIX. However, this script identifies a wide # variety of pre-POSIX systems that do not have POSIX shells at all, and # even some reasonably current systems (Solaris 10 as case-in-point) still # have a pre-POSIX /bin/sh. me=`echo "$0" | sed -e 's,.*/,,'` usage="\ Usage: $0 [OPTION] CPU-MFR-OPSYS or ALIAS Canonicalize a configuration name. Options: -h, --help print this help, then exit -t, --time-stamp print date of last modification, then exit -v, --version print version number, then exit Report bugs and patches to <config-patches@gnu.org>." version="\ GNU config.sub ($timestamp) Copyright 1992-2024 Free Software Foundation, Inc. This is free software; see the source for copying conditions. There is NO warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE." help=" Try '$me --help' for more information." # Parse command line while test $# -gt 0 ; do case $1 in --time-stamp | --time* | -t ) echo "$timestamp" ; exit ;; --version | -v ) echo "$version" ; exit ;; --help | --h* | -h ) echo "$usage"; exit ;; -- ) # Stop option processing shift; break ;; - ) # Use stdin as input. break ;; -* ) echo "$me: invalid option $1$help" >&2 exit 1 ;; *local*) # First pass through any local machine types. echo "$1" exit ;; * ) break ;; esac done case $# in 0) echo "$me: missing argument$help" >&2 exit 1;; 1) ;; *) echo "$me: too many arguments$help" >&2 exit 1;; esac # Split fields of configuration type saved_IFS=$IFS IFS="-" read field1 field2 field3 field4 <<EOF $1 EOF IFS=$saved_IFS # Separate into logical components for further validation case $1 in *-*-*-*-*) echo "Invalid configuration '$1': more than four components" >&2 exit 1 ;; *-*-*-*) basic_machine=$field1-$field2 basic_os=$field3-$field4 ;; *-*-*) # Ambiguous whether COMPANY is present, or skipped and KERNEL-OS is two # parts maybe_os=$field2-$field3 case $maybe_os in cloudabi*-eabi* \ | kfreebsd*-gnu* \ | knetbsd*-gnu* \ | kopensolaris*-gnu* \ | linux-* \ | managarm-* \ | netbsd*-eabi* \ | netbsd*-gnu* \ | nto-qnx* \ | os2-emx* \ | rtmk-nova* \ | storm-chaos* \ | uclinux-gnu* \ | uclinux-uclibc* \ | windows-* ) basic_machine=$field1 basic_os=$maybe_os ;; android-linux) basic_machine=$field1-unknown basic_os=linux-android ;; *) basic_machine=$field1-$field2 basic_os=$field3 ;; esac ;; *-*) case $field1-$field2 in # Shorthands that happen to contain a single dash convex-c[12] | convex-c3[248]) basic_machine=$field2-convex basic_os= ;; decstation-3100) basic_machine=mips-dec basic_os= ;; *-*) # Second component is usually, but not always the OS case $field2 in # Do not treat sunos as a manufacturer sun*os*) basic_machine=$field1 basic_os=$field2 ;; # Manufacturers 3100* \ | 32* \ | 3300* \ | 3600* \ | 7300* \ | acorn \ | altos* \ | apollo \ | apple \ | atari \ | att* \ | axis \ | be \ | bull \ | cbm \ | ccur \ | cisco \ | commodore \ | convergent* \ | convex* \ | cray \ | crds \ | dec* \ | delta* \ | dg \ | digital \ | dolphin \ | encore* \ | gould \ | harris \ | highlevel \ | hitachi* \ | hp \ | ibm* \ | intergraph \ | isi* \ | knuth \ | masscomp \ | microblaze* \ | mips* \ | motorola* \ | ncr* \ | news \ | next \ | ns \ | oki \ | omron* \ | pc533* \ | rebel \ | rom68k \ | rombug \ | semi \ | sequent* \ | siemens \ | sgi* \ | siemens \ | sim \ | sni \ | sony* \ | stratus \ | sun \ | sun[234]* \ | tektronix \ | tti* \ | ultra \ | unicom* \ | wec \ | winbond \ | wrs) basic_machine=$field1-$field2 basic_os= ;; zephyr*) basic_machine=$field1-unknown basic_os=$field2 ;; *) basic_machine=$field1 basic_os=$field2 ;; esac ;; esac ;; *) # Convert single-component short-hands not valid as part of # multi-component configurations. case $field1 in 386bsd) basic_machine=i386-pc basic_os=bsd ;; a29khif) basic_machine=a29k-amd basic_os=udi ;; adobe68k) basic_machine=m68010-adobe basic_os=scout ;; alliant) basic_machine=fx80-alliant basic_os= ;; altos | altos3068) basic_machine=m68k-altos basic_os= ;; am29k) basic_machine=a29k-none basic_os=bsd ;; amdahl) basic_machine=580-amdahl basic_os=sysv ;; amiga) basic_machine=m68k-unknown basic_os= ;; amigaos | amigados) basic_machine=m68k-unknown basic_os=amigaos ;; amigaunix | amix) basic_machine=m68k-unknown basic_os=sysv4 ;; apollo68) basic_machine=m68k-apollo basic_os=sysv ;; apollo68bsd) basic_machine=m68k-apollo basic_os=bsd ;; aros) basic_machine=i386-pc basic_os=aros ;; aux) basic_machine=m68k-apple basic_os=aux ;; balance) basic_machine=ns32k-sequent basic_os=dynix ;; blackfin) basic_machine=bfin-unknown basic_os=linux ;; cegcc) basic_machine=arm-unknown basic_os=cegcc ;; cray) basic_machine=j90-cray basic_os=unicos ;; crds | unos) basic_machine=m68k-crds basic_os= ;; da30) basic_machine=m68k-da30 basic_os= ;; decstation | pmax | pmin | dec3100 | decstatn) basic_machine=mips-dec basic_os= ;; delta88) basic_machine=m88k-motorola basic_os=sysv3 ;; dicos) basic_machine=i686-pc basic_os=dicos ;; djgpp) basic_machine=i586-pc basic_os=msdosdjgpp ;; ebmon29k) basic_machine=a29k-amd basic_os=ebmon ;; es1800 | OSE68k | ose68k | ose | OSE) basic_machine=m68k-ericsson basic_os=ose ;; gmicro) basic_machine=tron-gmicro basic_os=sysv ;; go32) basic_machine=i386-pc basic_os=go32 ;; h8300hms) basic_machine=h8300-hitachi basic_os=hms ;; h8300xray) basic_machine=h8300-hitachi basic_os=xray ;; h8500hms) basic_machine=h8500-hitachi basic_os=hms ;; harris) basic_machine=m88k-harris basic_os=sysv3 ;; hp300 | hp300hpux) basic_machine=m68k-hp basic_os=hpux ;; hp300bsd) basic_machine=m68k-hp basic_os=bsd ;; hppaosf) basic_machine=hppa1.1-hp basic_os=osf ;; hppro) basic_machine=hppa1.1-hp basic_os=proelf ;; i386mach) basic_machine=i386-mach basic_os=mach ;; isi68 | isi) basic_machine=m68k-isi basic_os=sysv ;; m68knommu) basic_machine=m68k-unknown basic_os=linux ;; magnum | m3230) basic_machine=mips-mips basic_os=sysv ;; merlin) basic_machine=ns32k-utek basic_os=sysv ;; mingw64) basic_machine=x86_64-pc basic_os=mingw64 ;; mingw32) basic_machine=i686-pc basic_os=mingw32 ;; mingw32ce) basic_machine=arm-unknown basic_os=mingw32ce ;; monitor) basic_machine=m68k-rom68k basic_os=coff ;; morphos) basic_machine=powerpc-unknown basic_os=morphos ;; moxiebox) basic_machine=moxie-unknown basic_os=moxiebox ;; msdos) basic_machine=i386-pc basic_os=msdos ;; msys) basic_machine=i686-pc basic_os=msys ;; mvs) basic_machine=i370-ibm basic_os=mvs ;; nacl) basic_machine=le32-unknown basic_os=nacl ;; ncr3000) basic_machine=i486-ncr basic_os=sysv4 ;; netbsd386) basic_machine=i386-pc basic_os=netbsd ;; netwinder) basic_machine=armv4l-rebel basic_os=linux ;; news | news700 | news800 | news900) basic_machine=m68k-sony basic_os=newsos ;; news1000) basic_machine=m68030-sony basic_os=newsos ;; necv70) basic_machine=v70-nec basic_os=sysv ;; nh3000) basic_machine=m68k-harris basic_os=cxux ;; nh[45]000) basic_machine=m88k-harris basic_os=cxux ;; nindy960) basic_machine=i960-intel basic_os=nindy ;; mon960) basic_machine=i960-intel basic_os=mon960 ;; nonstopux) basic_machine=mips-compaq basic_os=nonstopux ;; os400) basic_machine=powerpc-ibm basic_os=os400 ;; OSE68000 | ose68000) basic_machine=m68000-ericsson basic_os=ose ;; os68k) basic_machine=m68k-none basic_os=os68k ;; paragon) basic_machine=i860-intel basic_os=osf ;; parisc) basic_machine=hppa-unknown basic_os=linux ;; psp) basic_machine=mipsallegrexel-sony basic_os=psp ;; pw32) basic_machine=i586-unknown basic_os=pw32 ;; rdos | rdos64) basic_machine=x86_64-pc basic_os=rdos ;; rdos32) basic_machine=i386-pc basic_os=rdos ;; rom68k) basic_machine=m68k-rom68k basic_os=coff ;; sa29200) basic_machine=a29k-amd basic_os=udi ;; sei) basic_machine=mips-sei basic_os=seiux ;; sequent) basic_machine=i386-sequent basic_os= ;; sps7) basic_machine=m68k-bull basic_os=sysv2 ;; st2000) basic_machine=m68k-tandem basic_os= ;; stratus) basic_machine=i860-stratus basic_os=sysv4 ;; sun2) basic_machine=m68000-sun basic_os= ;; sun2os3) basic_machine=m68000-sun basic_os=sunos3 ;; sun2os4) basic_machine=m68000-sun basic_os=sunos4 ;; sun3) basic_machine=m68k-sun basic_os= ;; sun3os3) basic_machine=m68k-sun basic_os=sunos3 ;; sun3os4) basic_machine=m68k-sun basic_os=sunos4 ;; sun4) basic_machine=sparc-sun basic_os= ;; sun4os3) basic_machine=sparc-sun basic_os=sunos3 ;; sun4os4) basic_machine=sparc-sun basic_os=sunos4 ;; sun4sol2) basic_machine=sparc-sun basic_os=solaris2 ;; sun386 | sun386i | roadrunner) basic_machine=i386-sun basic_os= ;; sv1) basic_machine=sv1-cray basic_os=unicos ;; symmetry) basic_machine=i386-sequent basic_os=dynix ;; t3e) basic_machine=alphaev5-cray basic_os=unicos ;; t90) basic_machine=t90-cray basic_os=unicos ;; toad1) basic_machine=pdp10-xkl basic_os=tops20 ;; tpf) basic_machine=s390x-ibm basic_os=tpf ;; udi29k) basic_machine=a29k-amd basic_os=udi ;; ultra3) basic_machine=a29k-nyu basic_os=sym1 ;; v810 | necv810) basic_machine=v810-nec basic_os=none ;; vaxv) basic_machine=vax-dec basic_os=sysv ;; vms) basic_machine=vax-dec basic_os=vms ;; vsta) basic_machine=i386-pc basic_os=vsta ;; vxworks960) basic_machine=i960-wrs basic_os=vxworks ;; vxworks68) basic_machine=m68k-wrs basic_os=vxworks ;; vxworks29k) basic_machine=a29k-wrs basic_os=vxworks ;; xbox) basic_machine=i686-pc basic_os=mingw32 ;; ymp) basic_machine=ymp-cray basic_os=unicos ;; *) basic_machine=$1 basic_os= ;; esac ;; esac # Decode 1-component or ad-hoc basic machines case $basic_machine in # Here we handle the default manufacturer of certain CPU types. It is in # some cases the only manufacturer, in others, it is the most popular. w89k) cpu=hppa1.1 vendor=winbond ;; op50n) cpu=hppa1.1 vendor=oki ;; op60c) cpu=hppa1.1 vendor=oki ;; ibm*) cpu=i370 vendor=ibm ;; orion105) cpu=clipper vendor=highlevel ;; mac | mpw | mac-mpw) cpu=m68k vendor=apple ;; pmac | pmac-mpw) cpu=powerpc vendor=apple ;; # Recognize the various machine names and aliases which stand # for a CPU type and a company and sometimes even an OS. 3b1 | 7300 | 7300-att | att-7300 | pc7300 | safari | unixpc) cpu=m68000 vendor=att ;; 3b*) cpu=we32k vendor=att ;; bluegene*) cpu=powerpc vendor=ibm basic_os=cnk ;; decsystem10* | dec10*) cpu=pdp10 vendor=dec basic_os=tops10 ;; decsystem20* | dec20*) cpu=pdp10 vendor=dec basic_os=tops20 ;; delta | 3300 | delta-motorola | 3300-motorola | motorola-delta | motorola-3300) cpu=m68k vendor=motorola ;; # This used to be dpx2*, but that gets the RS6000-based # DPX/20 and the x86-based DPX/2-100 wrong. See # https://oldskool.silicium.org/stations/bull_dpx20.htm # https://www.feb-patrimoine.com/english/bull_dpx2.htm # https://www.feb-patrimoine.com/english/unix_and_bull.htm dpx2 | dpx2[23]00 | dpx2[23]xx) cpu=m68k vendor=bull ;; dpx2100 | dpx21xx) cpu=i386 vendor=bull ;; dpx20) cpu=rs6000 vendor=bull ;; encore | umax | mmax) cpu=ns32k vendor=encore ;; elxsi) cpu=elxsi vendor=elxsi basic_os=${basic_os:-bsd} ;; fx2800) cpu=i860 vendor=alliant ;; genix) cpu=ns32k vendor=ns ;; h3050r* | hiux*) cpu=hppa1.1 vendor=hitachi basic_os=hiuxwe2 ;; hp3k9[0-9][0-9] | hp9[0-9][0-9]) cpu=hppa1.0 vendor=hp ;; hp9k2[0-9][0-9] | hp9k31[0-9]) cpu=m68000 vendor=hp ;; hp9k3[2-9][0-9]) cpu=m68k vendor=hp ;; hp9k6[0-9][0-9] | hp6[0-9][0-9]) cpu=hppa1.0 vendor=hp ;; hp9k7[0-79][0-9] | hp7[0-79][0-9]) cpu=hppa1.1 vendor=hp ;; hp9k78[0-9] | hp78[0-9]) # FIXME: really hppa2.0-hp cpu=hppa1.1 vendor=hp ;; hp9k8[67]1 | hp8[67]1 | hp9k80[24] | hp80[24] | hp9k8[78]9 | hp8[78]9 | hp9k893 | hp893) # FIXME: really hppa2.0-hp cpu=hppa1.1 vendor=hp ;; hp9k8[0-9][13679] | hp8[0-9][13679]) cpu=hppa1.1 vendor=hp ;; hp9k8[0-9][0-9] | hp8[0-9][0-9]) cpu=hppa1.0 vendor=hp ;; i*86v32) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc basic_os=sysv32 ;; i*86v4*) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc basic_os=sysv4 ;; i*86v) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc basic_os=sysv ;; i*86sol2) cpu=`echo "$1" | sed -e 's/86.*/86/'` vendor=pc basic_os=solaris2 ;; j90 | j90-cray) cpu=j90 vendor=cray basic_os=${basic_os:-unicos} ;; iris | iris4d) cpu=mips vendor=sgi case $basic_os in irix*) ;; *) basic_os=irix4 ;; esac ;; miniframe) cpu=m68000 vendor=convergent ;; *mint | mint[0-9]* | *MiNT | *MiNT[0-9]*) cpu=m68k vendor=atari basic_os=mint ;; news-3600 | risc-news) cpu=mips vendor=sony basic_os=newsos ;; next | m*-next) cpu=m68k vendor=next ;; np1) cpu=np1 vendor=gould ;; op50n-* | op60c-*) cpu=hppa1.1 vendor=oki basic_os=proelf ;; pa-hitachi) cpu=hppa1.1 vendor=hitachi basic_os=hiuxwe2 ;; pbd) cpu=sparc vendor=tti ;; pbb) cpu=m68k vendor=tti ;; pc532) cpu=ns32k vendor=pc532 ;; pn) cpu=pn vendor=gould ;; power) cpu=power vendor=ibm ;; ps2) cpu=i386 vendor=ibm ;; rm[46]00) cpu=mips vendor=siemens ;; rtpc | rtpc-*) cpu=romp vendor=ibm ;; sde) cpu=mipsisa32 vendor=sde basic_os=${basic_os:-elf} ;; simso-wrs) cpu=sparclite vendor=wrs basic_os=vxworks ;; tower | tower-32) cpu=m68k vendor=ncr ;; vpp*|vx|vx-*) cpu=f301 vendor=fujitsu ;; w65) cpu=w65 vendor=wdc ;; w89k-*) cpu=hppa1.1 vendor=winbond basic_os=proelf ;; none) cpu=none vendor=none ;; leon|leon[3-9]) cpu=sparc vendor=$basic_machine ;; leon-*|leon[3-9]-*) cpu=sparc vendor=`echo "$basic_machine" | sed 's/-.*//'` ;; *-*) saved_IFS=$IFS IFS="-" read cpu vendor <<EOF $basic_machine EOF IFS=$saved_IFS ;; # We use 'pc' rather than 'unknown' # because (1) that's what they normally are, and # (2) the word "unknown" tends to confuse beginning users. i*86 | x86_64) cpu=$basic_machine vendor=pc ;; # These rules are duplicated from below for sake of the special case above; # i.e. things that normalized to x86 arches should also default to "pc" pc98) cpu=i386 vendor=pc ;; x64 | amd64) cpu=x86_64 vendor=pc ;; # Recognize the basic CPU types without company name. *) cpu=$basic_machine vendor=unknown ;; esac unset -v basic_machine # Decode basic machines in the full and proper CPU-Company form. case $cpu-$vendor in # Here we handle the default manufacturer of certain CPU types in canonical form. # It is in some cases the only manufacturer, in others, it is the most popular. c[12]-convex | c[12]-unknown | c3[248]-convex | c3[248]-unknown) vendor=convex basic_os=${basic_os:-bsd} ;; craynv-unknown) vendor=cray basic_os=${basic_os:-unicosmp} ;; c90-unknown | c90-cray) vendor=cray basic_os=${basic_os:-unicos} ;; fx80-unknown) vendor=alliant ;; romp-unknown) vendor=ibm ;; mmix-unknown) vendor=knuth ;; microblaze-unknown | microblazeel-unknown) vendor=xilinx ;; rs6000-unknown) vendor=ibm ;; vax-unknown) vendor=dec ;; pdp11-unknown) vendor=dec ;; we32k-unknown) vendor=att ;; cydra-unknown) vendor=cydrome ;; i370-ibm*) vendor=ibm ;; orion-unknown) vendor=highlevel ;; xps-unknown | xps100-unknown) cpu=xps100 vendor=honeywell ;; # Here we normalize CPU types with a missing or matching vendor armh-unknown | armh-alt) cpu=armv7l vendor=alt basic_os=${basic_os:-linux-gnueabihf} ;; # Normalized CPU+vendor pairs that imply an OS, if not otherwise specified m68k-isi) basic_os=${basic_os:-sysv} ;; m68k-sony) basic_os=${basic_os:-newsos} ;; m68k-tektronix) basic_os=${basic_os:-bsd} ;; m88k-harris) basic_os=${basic_os:-sysv3} ;; i386-bull | m68k-bull) basic_os=${basic_os:-sysv3} ;; rs6000-bull) basic_os=${basic_os:-bosx} ;; mips-sni) basic_os=${basic_os:-sysv4} ;; # Here we normalize CPU types irrespective of the vendor amd64-*) cpu=x86_64 ;; blackfin-*) cpu=bfin basic_os=${basic_os:-linux} ;; c54x-*) cpu=tic54x ;; c55x-*) cpu=tic55x ;; c6x-*) cpu=tic6x ;; e500v[12]-*) cpu=powerpc basic_os=${basic_os}"spe" ;; mips3*-*) cpu=mips64 ;; ms1-*) cpu=mt ;; m68knommu-*) cpu=m68k basic_os=${basic_os:-linux} ;; m9s12z-* | m68hcs12z-* | hcs12z-* | s12z-*) cpu=s12z ;; openrisc-*) cpu=or32 ;; parisc-*) cpu=hppa basic_os=${basic_os:-linux} ;; pentium-* | p5-* | k5-* | k6-* | nexgen-* | viac3-*) cpu=i586 ;; pentiumpro-* | p6-* | 6x86-* | athlon-* | athlon_*-*) cpu=i686 ;; pentiumii-* | pentium2-* | pentiumiii-* | pentium3-*) cpu=i686 ;; pentium4-*) cpu=i786 ;; ppc-* | ppcbe-*) cpu=powerpc ;; ppcle-* | powerpclittle-*) cpu=powerpcle ;; ppc64-*) cpu=powerpc64 ;; ppc64le-* | powerpc64little-*) cpu=powerpc64le ;; sb1-*) cpu=mipsisa64sb1 ;; sb1el-*) cpu=mipsisa64sb1el ;; sh5e[lb]-*) cpu=`echo "$cpu" | sed 's/^\(sh.\)e\(.\)$/\1\2e/'` ;; spur-*) cpu=spur ;; strongarm-* | thumb-*) cpu=arm ;; tx39-*) cpu=mipstx39 ;; tx39el-*) cpu=mipstx39el ;; xscale-* | xscalee[bl]-*) cpu=`echo "$cpu" | sed 's/^xscale/arm/'` ;; arm64-* | aarch64le-*) cpu=aarch64 ;; # Recognize the canonical CPU Types that limit and/or modify the # company names they are paired with. cr16-*) basic_os=${basic_os:-elf} ;; crisv32-* | etraxfs*-*) cpu=crisv32 vendor=axis ;; cris-* | etrax*-*) cpu=cris vendor=axis ;; crx-*) basic_os=${basic_os:-elf} ;; neo-tandem) cpu=neo vendor=tandem ;; nse-tandem) cpu=nse vendor=tandem ;; nsr-tandem) cpu=nsr vendor=tandem ;; nsv-tandem) cpu=nsv vendor=tandem ;; nsx-tandem) cpu=nsx vendor=tandem ;; mipsallegrexel-sony) cpu=mipsallegrexel vendor=sony ;; tile*-*) basic_os=${basic_os:-linux-gnu} ;; *) # Recognize the canonical CPU types that are allowed with any # company name. case $cpu in 1750a \ | 580 \ | [cjt]90 \ | a29k \ | aarch64 \ | aarch64_be \ | aarch64c \ | abacus \ | alpha \ | alpha64 \ | alpha64ev56 \ | alpha64ev6[78] \ | alpha64ev[4-8] \ | alpha64pca5[67] \ | alphaev56 \ | alphaev6[78] \ | alphaev[4-8] \ | alphapca5[67] \ | am33_2.0 \ | amdgcn \ | arc \ | arc32 \ | arc64 \ | arceb \ | arm \ | arm64e \ | arm64ec \ | arm[lb]e \ | arme[lb] \ | armv* \ | asmjs \ | avr \ | avr32 \ | ba \ | be32 \ | be64 \ | bfin \ | bpf \ | bs2000 \ | c30 \ | c4x \ | c8051 \ | c[123]* \ | clipper \ | craynv \ | csky \ | cydra \ | d10v \ | d30v \ | dlx \ | dsp16xx \ | e2k \ | elxsi \ | epiphany \ | f30[01] \ | f700 \ | fido \ | fr30 \ | frv \ | ft32 \ | fx80 \ | h8300 \ | h8500 \ | hexagon \ | hppa \ | hppa1.[01] \ | hppa2.0 \ | hppa2.0[nw] \ | hppa64 \ | i*86 \ | i370 \ | i860 \ | i960 \ | ia16 \ | ia64 \ | ip2k \ | iq2000 \ | javascript \ | k1om \ | kvx \ | le32 \ | le64 \ | lm32 \ | loongarch32 \ | loongarch64 \ | m32c \ | m32r \ | m32rle \ | m5200 \ | m68000 \ | m680[012346]0 \ | m6811 \ | m6812 \ | m68360 \ | m683?2 \ | m68hc11 \ | m68hc12 \ | m68hcs12x \ | m68k \ | m88110 \ | m88k \ | maxq \ | mb \ | mcore \ | mep \ | metag \ | microblaze \ | microblazeel \ | mips* \ | mmix \ | mn10200 \ | mn10300 \ | moxie \ | msp430 \ | mt \ | nanomips* \ | nds32 \ | nds32be \ | nds32le \ | nfp \ | nios \ | nios2 \ | nios2eb \ | nios2el \ | none \ | np1 \ | ns16k \ | ns32k \ | nvptx \ | open8 \ | or1k* \ | or32 \ | orion \ | pdp10 \ | pdp11 \ | picochip \ | pj \ | pjl \ | pn \ | power \ | powerpc \ | powerpc64 \ | powerpc64le \ | powerpcle \ | powerpcspe \ | pru \ | pyramid \ | riscv \ | riscv32 \ | riscv32be \ | riscv64 \ | riscv64be \ | rl78 \ | romp \ | rs6000 \ | rx \ | s390 \ | s390x \ | score \ | sh \ | sh64 \ | sh64le \ | sh[12345][lb]e \ | sh[1234] \ | sh[1234]e[lb] \ | sh[23]e \ | sh[23]ele \ | sh[24]a \ | sh[24]ae[lb] \ | sh[lb]e \ | she[lb] \ | shl \ | sparc \ | sparc64 \ | sparc64b \ | sparc64v \ | sparc86x \ | sparclet \ | sparclite \ | sparcv8 \ | sparcv9 \ | sparcv9b \ | sparcv9v \ | spu \ | sv1 \ | sx* \ | tahoe \ | thumbv7* \ | tic30 \ | tic4x \ | tic54x \ | tic55x \ | tic6x \ | tic80 \ | tron \ | ubicom32 \ | v70 \ | v810 \ | v850 \ | v850e \ | v850e1 \ | v850e2 \ | v850e2v3 \ | v850es \ | vax \ | vc4 \ | visium \ | w65 \ | wasm32 \ | wasm64 \ | we32k \ | x86 \ | x86_64 \ | xc16x \ | xgate \ | xps100 \ | xstormy16 \ | xtensa* \ | ymp \ | z80 \ | z8k) ;; *) echo "Invalid configuration '$1': machine '$cpu-$vendor' not recognized" 1>&2 exit 1 ;; esac ;; esac # Here we canonicalize certain aliases for manufacturers. case $vendor in digital*) vendor=dec ;; commodore*) vendor=cbm ;; *) ;; esac # Decode manufacturer-specific aliases for certain operating systems. if test x"$basic_os" != x then # First recognize some ad-hoc cases, or perhaps split kernel-os, or else just # set os. obj= case $basic_os in gnu/linux*) kernel=linux os=`echo "$basic_os" | sed -e 's|gnu/linux|gnu|'` ;; os2-emx) kernel=os2 os=`echo "$basic_os" | sed -e 's|os2-emx|emx|'` ;; nto-qnx*) kernel=nto os=`echo "$basic_os" | sed -e 's|nto-qnx|qnx|'` ;; *-*) saved_IFS=$IFS IFS="-" read kernel os <<EOF $basic_os EOF IFS=$saved_IFS ;; # Default OS when just kernel was specified nto*) kernel=nto os=`echo "$basic_os" | sed -e 's|nto|qnx|'` ;; linux*) kernel=linux os=`echo "$basic_os" | sed -e 's|linux|gnu|'` ;; managarm*) kernel=managarm os=`echo "$basic_os" | sed -e 's|managarm|mlibc|'` ;; *) kernel= os=$basic_os ;; esac # Now, normalize the OS (knowing we just have one component, it's not a kernel, # etc.) case $os in # First match some system type aliases that might get confused # with valid system types. # solaris* is a basic system type, with this one exception. auroraux) os=auroraux ;; bluegene*) os=cnk ;; solaris1 | solaris1.*) os=`echo "$os" | sed -e 's|solaris1|sunos4|'` ;; solaris) os=solaris2 ;; unixware*) os=sysv4.2uw ;; # The marketing names for NeXT's operating systems were # NeXTSTEP, NeXTSTEP 2, OpenSTEP 3, OpenSTEP 4. 'openstep' is # mapped to 'openstep3', but 'openstep1' and 'openstep2' are # mapped to 'nextstep' and 'nextstep2', consistent with the # treatment of SunOS/Solaris. ns | ns1 | nextstep | nextstep1 | openstep1) os=nextstep ;; ns2 | nextstep2 | openstep2) os=nextstep2 ;; ns3 | nextstep3 | openstep | openstep3) os=openstep3 ;; ns4 | nextstep4 | openstep4) os=openstep4 ;; # es1800 is here to avoid being matched by es* (a different OS) es1800*) os=ose ;; # Some version numbers need modification chorusos*) os=chorusos ;; isc) os=isc2.2 ;; sco6) os=sco5v6 ;; sco5) os=sco3.2v5 ;; sco4) os=sco3.2v4 ;; sco3.2.[4-9]*) os=`echo "$os" | sed -e 's/sco3.2./sco3.2v/'` ;; sco*v* | scout) # Don't match below ;; sco*) os=sco3.2v2 ;; psos*) os=psos ;; qnx*) os=qnx ;; hiux*) os=hiuxwe2 ;; lynx*178) os=lynxos178 ;; lynx*5) os=lynxos5 ;; lynxos*) # don't get caught up in next wildcard ;; lynx*) os=lynxos ;; mac[0-9]*) os=`echo "$os" | sed -e 's|mac|macos|'` ;; opened*) os=openedition ;; os400*) os=os400 ;; sunos5*) os=`echo "$os" | sed -e 's|sunos5|solaris2|'` ;; sunos6*) os=`echo "$os" | sed -e 's|sunos6|solaris3|'` ;; wince*) os=wince ;; utek*) os=bsd vendor=`echo "$vendor" | sed -e 's|^unknown$|tektronix|'` ;; dynix*) os=bsd ;; acis*) os=aos ;; atheos*) os=atheos ;; syllable*) os=syllable ;; 386bsd) os=bsd ;; ctix*) os=sysv vendor=`echo "$vendor" | sed -e 's|^unknown$|convergent|'` ;; uts*) os=sysv ;; nova*) kernel=rtmk os=nova ;; # Preserve the version number of sinix5. sinix5.*) os=`echo "$os" | sed -e 's|sinix|sysv|'` vendor=`echo "$vendor" | sed -e 's|^unknown$|sni|'` ;; sinix*) os=sysv4 vendor=`echo "$vendor" | sed -e 's|^unknown$|sni|'` ;; tpf*) os=tpf ;; triton*) os=sysv3 ;; oss*) os=sysv3 ;; svr4*) os=sysv4 ;; svr3) os=sysv3 ;; sysvr4) os=sysv4 ;; ose*) os=ose ;; *mint | mint[0-9]* | *MiNT | MiNT[0-9]*) os=mint ;; dicos*) os=dicos ;; pikeos*) # Until real need of OS specific support for # particular features comes up, bare metal # configurations are quite functional. case $cpu in arm*) os=eabi ;; *) os= obj=elf ;; esac ;; aout* | coff* | elf* | pe*) # These are machine code file formats, not OSes obj=$os os= ;; *) # No normalization, but not necessarily accepted, that comes below. ;; esac else # Here we handle the default operating systems that come with various machines. # The value should be what the vendor currently ships out the door with their # machine or put another way, the most popular os provided with the machine. # Note that if you're going to try to match "-MANUFACTURER" here (say, # "-sun"), then you have to tell the case statement up towards the top # that MANUFACTURER isn't an operating system. Otherwise, code above # will signal an error saying that MANUFACTURER isn't an operating # system, and we'll never get to this point. kernel= obj= case $cpu-$vendor in score-*) os= obj=elf ;; spu-*) os= obj=elf ;; *-acorn) os=riscix1.2 ;; arm*-rebel) kernel=linux os=gnu ;; arm*-semi) os= obj=aout ;; c4x-* | tic4x-*) os= obj=coff ;; c8051-*) os= obj=elf ;; clipper-intergraph) os=clix ;; hexagon-*) os= obj=elf ;; tic54x-*) os= obj=coff ;; tic55x-*) os= obj=coff ;; tic6x-*) os= obj=coff ;; # This must come before the *-dec entry. pdp10-*) os=tops20 ;; pdp11-*) os=none ;; *-dec | vax-*) os=ultrix4.2 ;; m68*-apollo) os=domain ;; i386-sun) os=sunos4.0.2 ;; m68000-sun) os=sunos3 ;; m68*-cisco) os= obj=aout ;; mep-*) os= obj=elf ;; # The -sgi and -siemens entries must be before the mips- entry # or we get the wrong os. *-sgi) os=irix ;; *-siemens) os=sysv4 ;; mips*-cisco) os= obj=elf ;; mips*-*|nanomips*-*) os= obj=elf ;; or32-*) os= obj=coff ;; # This must be before the sparc-* entry or we get the wrong os. *-tti) os=sysv3 ;; sparc-* | *-sun) os=sunos4.1.1 ;; pru-*) os= obj=elf ;; *-be) os=beos ;; *-ibm) os=aix ;; *-knuth) os=mmixware ;; *-wec) os=proelf ;; *-winbond) os=proelf ;; *-oki) os=proelf ;; *-hp) os=hpux ;; *-hitachi) os=hiuxwe2 ;; i860-* | *-att | *-ncr | *-altos | *-motorola | *-convergent) os=sysv ;; *-cbm) os=amigaos ;; *-dg) os=dgux ;; *-dolphin) os=sysv3 ;; m68k-ccur) os=rtu ;; m88k-omron*) os=luna ;; *-next) os=nextstep ;; *-sequent) os=ptx ;; *-crds) os=unos ;; *-ns) os=genix ;; i370-*) os=mvs ;; *-gould) os=sysv ;; *-highlevel) os=bsd ;; *-encore) os=bsd ;; *-masscomp) os=rtu ;; f30[01]-fujitsu | f700-fujitsu) os=uxpv ;; *-rom68k) os= obj=coff ;; *-*bug) os= obj=coff ;; *-apple) os=macos ;; *-atari*) os=mint ;; *-wrs) os=vxworks ;; *) os=none ;; esac fi # Now, validate our (potentially fixed-up) individual pieces (OS, OBJ). case $os in # Sometimes we do "kernel-libc", so those need to count as OSes. llvm* | musl* | newlib* | relibc* | uclibc*) ;; # Likewise for "kernel-abi" eabi* | gnueabi*) ;; # VxWorks passes extra cpu info in the 4th filed. simlinux | simwindows | spe) ;; # See `case $cpu-$os` validation below ghcjs) ;; # Now accept the basic system types. # Each alternative MUST end in a * to match a version number. abug \ | aix* \ | amdhsa* \ | amigados* \ | amigaos* \ | android* \ | aof* \ | aos* \ | aros* \ | atheos* \ | auroraux* \ | aux* \ | beos* \ | bitrig* \ | bme* \ | bosx* \ | bsd* \ | cegcc* \ | chorusos* \ | chorusrdb* \ | clix* \ | cloudabi* \ | cnk* \ | conix* \ | cos* \ | cxux* \ | cygwin* \ | darwin* \ | dgux* \ | dicos* \ | dnix* \ | domain* \ | dragonfly* \ | drops* \ | ebmon* \ | ecoff* \ | ekkobsd* \ | emscripten* \ | emx* \ | es* \ | fiwix* \ | freebsd* \ | fuchsia* \ | genix* \ | genode* \ | glidix* \ | gnu* \ | go32* \ | haiku* \ | hcos* \ | hiux* \ | hms* \ | hpux* \ | ieee* \ | interix* \ | ios* \ | iris* \ | irix* \ | ironclad* \ | isc* \ | its* \ | l4re* \ | libertybsd* \ | lites* \ | lnews* \ | luna* \ | lynxos* \ | mach* \ | macos* \ | magic* \ | mbr* \ | midipix* \ | midnightbsd* \ | mingw32* \ | mingw64* \ | minix* \ | mint* \ | mirbsd* \ | mks* \ | mlibc* \ | mmixware* \ | mon960* \ | morphos* \ | moss* \ | moxiebox* \ | mpeix* \ | mpw* \ | msdos* \ | msys* \ | mvs* \ | nacl* \ | netbsd* \ | netware* \ | newsos* \ | nextstep* \ | nindy* \ | nonstopux* \ | nova* \ | nsk* \ | nucleus* \ | nx6 \ | nx7 \ | oabi* \ | ohos* \ | onefs* \ | openbsd* \ | openedition* \ | openstep* \ | os108* \ | os2* \ | os400* \ | os68k* \ | os9* \ | ose* \ | osf* \ | oskit* \ | osx* \ | palmos* \ | phoenix* \ | plan9* \ | powermax* \ | powerunix* \ | proelf* \ | psos* \ | psp* \ | ptx* \ | pw32* \ | qnx* \ | rdos* \ | redox* \ | rhapsody* \ | riscix* \ | riscos* \ | rtems* \ | rtmk* \ | rtu* \ | scout* \ | secbsd* \ | sei* \ | serenity* \ | sim* \ | skyos* \ | solaris* \ | solidbsd* \ | sortix* \ | storm-chaos* \ | sunos \ | sunos[34]* \ | superux* \ | syllable* \ | sym* \ | sysv* \ | tenex* \ | tirtos* \ | toppers* \ | tops10* \ | tops20* \ | tpf* \ | tvos* \ | twizzler* \ | uclinux* \ | udi* \ | udk* \ | ultrix* \ | unicos* \ | uniplus* \ | unleashed* \ | unos* \ | uwin* \ | uxpv* \ | v88r* \ |*vms* \ | vos* \ | vsta* \ | vxsim* \ | vxworks* \ | wasi* \ | watchos* \ | wince* \ | windiss* \ | windows* \ | winnt* \ | xenix* \ | xray* \ | zephyr* \ | zvmoe* ) ;; # This one is extra strict with allowed versions sco3.2v2 | sco3.2v[4-9]* | sco5v6*) # Don't forget version if it is 3.2v4 or newer. ;; # This refers to builds using the UEFI calling convention # (which depends on the architecture) and PE file format. # Note that this is both a different calling convention and # different file format than that of GNU-EFI # (x86_64-w64-mingw32). uefi) ;; none) ;; kernel* | msvc* ) # Restricted further below ;; '') if test x"$obj" = x then echo "Invalid configuration '$1': Blank OS only allowed with explicit machine code file format" 1>&2 fi ;; *) echo "Invalid configuration '$1': OS '$os' not recognized" 1>&2 exit 1 ;; esac case $obj in aout* | coff* | elf* | pe*) ;; '') # empty is fine ;; *) echo "Invalid configuration '$1': Machine code format '$obj' not recognized" 1>&2 exit 1 ;; esac # Here we handle the constraint that a (synthetic) cpu and os are # valid only in combination with each other and nowhere else. case $cpu-$os in # The "javascript-unknown-ghcjs" triple is used by GHC; we # accept it here in order to tolerate that, but reject any # variations. javascript-ghcjs) ;; javascript-* | *-ghcjs) echo "Invalid configuration '$1': cpu '$cpu' is not valid with os '$os$obj'" 1>&2 exit 1 ;; esac # As a final step for OS-related things, validate the OS-kernel combination # (given a valid OS), if there is a kernel. case $kernel-$os-$obj in linux-gnu*- | linux-android*- | linux-dietlibc*- | linux-llvm*- \ | linux-mlibc*- | linux-musl*- | linux-newlib*- \ | linux-relibc*- | linux-uclibc*- | linux-ohos*- ) ;; uclinux-uclibc*- | uclinux-gnu*- ) ;; managarm-mlibc*- | managarm-kernel*- ) ;; windows*-msvc*-) ;; -dietlibc*- | -llvm*- | -mlibc*- | -musl*- | -newlib*- | -relibc*- \ | -uclibc*- ) # These are just libc implementations, not actual OSes, and thus # require a kernel. echo "Invalid configuration '$1': libc '$os' needs explicit kernel." 1>&2 exit 1 ;; -kernel*- ) echo "Invalid configuration '$1': '$os' needs explicit kernel." 1>&2 exit 1 ;; *-kernel*- ) echo "Invalid configuration '$1': '$kernel' does not support '$os'." 1>&2 exit 1 ;; *-msvc*- ) echo "Invalid configuration '$1': '$os' needs 'windows'." 1>&2 exit 1 ;; kfreebsd*-gnu*- | knetbsd*-gnu*- | netbsd*-gnu*- | kopensolaris*-gnu*-) ;; vxworks-simlinux- | vxworks-simwindows- | vxworks-spe-) ;; nto-qnx*-) ;; os2-emx-) ;; rtmk-nova-) ;; *-eabi*- | *-gnueabi*-) ;; none--*) # None (no kernel, i.e. freestanding / bare metal), # can be paired with an machine code file format ;; -*-) # Blank kernel with real OS is always fine. ;; --*) # Blank kernel and OS with real machine code file format is always fine. ;; *-*-*) echo "Invalid configuration '$1': Kernel '$kernel' not known to work with OS '$os'." 1>&2 exit 1 ;; esac # Here we handle the case where we know the os, and the CPU type, but not the # manufacturer. We pick the logical manufacturer. case $vendor in unknown) case $cpu-$os in *-riscix*) vendor=acorn ;; *-sunos* | *-solaris*) vendor=sun ;; *-cnk* | *-aix*) vendor=ibm ;; *-beos*) vendor=be ;; *-hpux*) vendor=hp ;; *-mpeix*) vendor=hp ;; *-hiux*) vendor=hitachi ;; *-unos*) vendor=crds ;; *-dgux*) vendor=dg ;; *-luna*) vendor=omron ;; *-genix*) vendor=ns ;; *-clix*) vendor=intergraph ;; *-mvs* | *-opened*) vendor=ibm ;; *-os400*) vendor=ibm ;; s390-* | s390x-*) vendor=ibm ;; *-ptx*) vendor=sequent ;; *-tpf*) vendor=ibm ;; *-vxsim* | *-vxworks* | *-windiss*) vendor=wrs ;; *-aux*) vendor=apple ;; *-hms*) vendor=hitachi ;; *-mpw* | *-macos*) vendor=apple ;; *-*mint | *-mint[0-9]* | *-*MiNT | *-MiNT[0-9]*) vendor=atari ;; *-vos*) vendor=stratus ;; esac ;; esac echo "$cpu-$vendor${kernel:+-$kernel}${os:+-$os}${obj:+-$obj}" exit # Local variables: # eval: (add-hook 'before-save-hook 'time-stamp) # time-stamp-start: "timestamp='" # time-stamp-format: "%:y-%02m-%02d" # time-stamp-end: "'" # End: |
Added autosetup/autosetup-find-tclsh.
> > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 | #!/bin/sh # Looks for a suitable tclsh or jimsh in the PATH # If not found, builds a bootstrap jimsh in current dir from source # Prefer $autosetup_tclsh if is set in the environment (unless ./jimsh0 works) # If an argument is given, use that as the test instead of autosetup-test-tclsh d="`dirname "$0"`" for tclsh in ./jimsh0 $autosetup_tclsh jimsh tclsh tclsh8.5 tclsh8.6 tclsh8.7; do { $tclsh "$d/${1-autosetup-test-tclsh}"; } 2>/dev/null && exit 0 done echo 1>&2 "No installed jimsh or tclsh, building local bootstrap jimsh0" for cc in ${CC_FOR_BUILD:-cc} gcc; do { $cc -o jimsh0 "$d/jimsh0.c"; } 2>/dev/null >/dev/null || continue ./jimsh0 "$d/${1-autosetup-test-tclsh}" && exit 0 done echo 1>&2 "No working C compiler found. Tried ${CC_FOR_BUILD:-cc} and gcc." echo false |
Added autosetup/autosetup-test-tclsh.
> > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 | # A small Tcl script to verify that the chosen # interpreter works. Sometimes we might e.g. pick up # an interpreter for a different arch. # Outputs the full path to the interpreter if {[catch {info version} version] == 0} { # This is Jim Tcl if {$version >= 0.72} { # Ensure that regexp works regexp (a.*?) a puts [info nameofexecutable] exit 0 } } elseif {[catch {info tclversion} version] == 0} { if {$version >= 8.5 && ![string match 8.5a* [info patchlevel]]} { puts [info nameofexecutable] exit 0 } } exit 1 |
Added autosetup/cc-db.tcl.
> > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 | # Copyright (c) 2011 WorkWare Systems http://www.workware.net.au/ # All rights reserved # @synopsis: # # The 'cc-db' module provides a knowledge-base of system idiosyncrasies. # In general, this module can always be included. use cc options {} # openbsd needs sys/types.h to detect some system headers cc-include-needs sys/socket.h sys/types.h cc-include-needs netinet/in.h sys/types.h |
Added autosetup/cc-lib.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 | # Copyright (c) 2011 WorkWare Systems http://www.workware.net.au/ # All rights reserved # @synopsis: # # Provides a library of common tests on top of the 'cc' module. use cc # @cc-check-lfs # # The equivalent of the 'AC_SYS_LARGEFILE' macro. # # defines 'HAVE_LFS' if LFS is available, # and defines '_FILE_OFFSET_BITS=64' if necessary # # Returns 1 if 'LFS' is available or 0 otherwise # proc cc-check-lfs {} { cc-check-includes sys/types.h msg-checking "Checking if -D_FILE_OFFSET_BITS=64 is needed..." set lfs 1 if {[msg-quiet cc-with {-includes sys/types.h} {cc-check-sizeof off_t}] == 8} { msg-result no } elseif {[msg-quiet cc-with {-includes sys/types.h -cflags -D_FILE_OFFSET_BITS=64} {cc-check-sizeof off_t}] == 8} { define _FILE_OFFSET_BITS 64 msg-result yes } else { set lfs 0 msg-result none } define-feature lfs $lfs return $lfs } # @cc-check-endian # # The equivalent of the 'AC_C_BIGENDIAN' macro. # # defines 'HAVE_BIG_ENDIAN' if endian is known to be big, # or 'HAVE_LITTLE_ENDIAN' if endian is known to be little. # # Returns 1 if determined, or 0 if not. # proc cc-check-endian {} { cc-check-includes sys/types.h sys/param.h set rc 0 msg-checking "Checking endian..." cc-with {-includes {sys/types.h sys/param.h}} { if {[cctest -code { #if !defined(BIG_ENDIAN) || !defined(BYTE_ORDER) #error unknown #elif BYTE_ORDER != BIG_ENDIAN #error little #endif }]} { define-feature big-endian msg-result "big" set rc 1 } elseif {[cctest -code { #if !defined(LITTLE_ENDIAN) || !defined(BYTE_ORDER) #error unknown #elif BYTE_ORDER != LITTLE_ENDIAN #error big #endif }]} { define-feature little-endian msg-result "little" set rc 1 } else { msg-result "unknown" } } return $rc } # @cc-check-flags flag ?...? # # Checks whether the given C/C++ compiler flags can be used. Defines feature # names prefixed with 'HAVE_CFLAG' and 'HAVE_CXXFLAG' respectively, and # appends working flags to '-cflags' and 'AS_CFLAGS' or 'AS_CXXFLAGS'. proc cc-check-flags {args} { set result 1 array set opts [cc-get-settings] switch -exact -- $opts(-lang) { c++ { set lang C++ set prefix CXXFLAG } c { set lang C set prefix CFLAG } default { autosetup-error "cc-check-flags failed with unknown language: $opts(-lang)" } } foreach flag $args { msg-checking "Checking whether the $lang compiler accepts $flag..." if {[cctest -cflags $flag]} { msg-result yes define-feature $prefix$flag cc-with [list -cflags [list $flag]] define-append AS_${prefix}S $flag } else { msg-result no set result 0 } } return $result } # @cc-check-standards ver ?...? # # Checks whether the C/C++ compiler accepts one of the specified '-std=$ver' # options, and appends the first working one to '-cflags' and 'AS_CFLAGS' or # 'AS_CXXFLAGS'. proc cc-check-standards {args} { array set opts [cc-get-settings] foreach std $args { if {[cc-check-flags -std=$std]} { return $std } } return "" } # Checks whether $keyword is usable as alignof proc cctest_alignof {keyword} { msg-checking "Checking for $keyword..." if {[cctest -code "int x = ${keyword}(char), y = ${keyword}('x');"]} then { msg-result ok define-feature $keyword } else { msg-result "not found" } } # @cc-check-c11 # # Checks for several C11/C++11 extensions and their alternatives. Currently # checks for '_Static_assert', '_Alignof', '__alignof__', '__alignof'. proc cc-check-c11 {} { msg-checking "Checking for _Static_assert..." if {[cctest -code { _Static_assert(1, "static assertions are available"); }]} then { msg-result ok define-feature _Static_assert } else { msg-result "not found" } cctest_alignof _Alignof cctest_alignof __alignof__ cctest_alignof __alignof } # @cc-check-alloca # # The equivalent of the 'AC_FUNC_ALLOCA' macro. # # Checks for the existence of 'alloca' # defines 'HAVE_ALLOCA' and returns 1 if it exists. proc cc-check-alloca {} { cc-check-some-feature alloca { cctest -includes alloca.h -code { alloca (2 * sizeof (int)); } } } # @cc-signal-return-type # # The equivalent of the 'AC_TYPE_SIGNAL' macro. # # defines 'RETSIGTYPE' to 'int' or 'void'. proc cc-signal-return-type {} { msg-checking "Checking return type of signal handlers..." cc-with {-includes {sys/types.h signal.h}} { if {[cctest -code {return *(signal (0, 0)) (0) == 1;}]} { set type int } else { set type void } define RETSIGTYPE $type msg-result $type } } |
Added autosetup/cc-shared.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # @synopsis: # # The 'cc-shared' module provides support for shared libraries and shared objects. # It defines the following variables: # ## SH_CFLAGS Flags to use compiling sources destined for a shared library ## SH_LDFLAGS Flags to use linking (creating) a shared library ## SH_SOPREFIX Prefix to use to set the soname when creating a shared library ## SH_SOFULLPATH Set to 1 if the shared library soname should include the full install path ## SH_SOEXT Extension for shared libs ## SH_SOEXTVER Format for versioned shared libs - %s = version ## SHOBJ_CFLAGS Flags to use compiling sources destined for a shared object ## SHOBJ_LDFLAGS Flags to use linking a shared object, undefined symbols allowed ## SHOBJ_LDFLAGS_R - as above, but all symbols must be resolved ## SH_LINKRPATH Format for setting the rpath when linking an executable, %s = path ## SH_LINKFLAGS Flags to use linking an executable which will load shared objects ## LD_LIBRARY_PATH Environment variable which specifies path to shared libraries ## STRIPLIBFLAGS Arguments to strip a dynamic library options {} # Defaults: gcc on unix define SHOBJ_CFLAGS -fPIC define SHOBJ_LDFLAGS -shared define SH_CFLAGS -fPIC define SH_LDFLAGS -shared define SH_LINKFLAGS -rdynamic define SH_LINKRPATH "-Wl,-rpath -Wl,%s" define SH_SOEXT .so define SH_SOEXTVER .so.%s define SH_SOPREFIX -Wl,-soname, define LD_LIBRARY_PATH LD_LIBRARY_PATH define STRIPLIBFLAGS --strip-unneeded # Note: This is a helpful reference for identifying the toolchain # http://sourceforge.net/apps/mediawiki/predef/index.php?title=Compilers switch -glob -- [get-define host] { *-*-darwin* { define SHOBJ_CFLAGS "-dynamic -fno-common" define SHOBJ_LDFLAGS "-bundle -undefined dynamic_lookup" define SHOBJ_LDFLAGS_R -bundle define SH_CFLAGS -dynamic define SH_LDFLAGS -dynamiclib define SH_LINKFLAGS "" define SH_SOEXT .dylib define SH_SOEXTVER .%s.dylib define SH_SOPREFIX -Wl,-install_name, define SH_SOFULLPATH define LD_LIBRARY_PATH DYLD_LIBRARY_PATH define STRIPLIBFLAGS -x } *-*-ming* - *-*-cygwin - *-*-msys { define SHOBJ_CFLAGS "" define SHOBJ_LDFLAGS -shared define SH_CFLAGS "" define SH_LDFLAGS -shared define SH_LINKRPATH "" define SH_LINKFLAGS "" define SH_SOEXT .dll define SH_SOEXTVER .dll define SH_SOPREFIX "" define LD_LIBRARY_PATH PATH } sparc* { if {[msg-quiet cc-check-decls __SUNPRO_C]} { msg-result "Found sun stdio compiler" # sun stdio compiler # XXX: These haven't been fully tested. define SHOBJ_CFLAGS -KPIC define SHOBJ_LDFLAGS "-G" define SH_CFLAGS -KPIC define SH_LINKFLAGS -Wl,-export-dynamic define SH_SOPREFIX -Wl,-h, } } *-*-solaris* { if {[msg-quiet cc-check-decls __SUNPRO_C]} { msg-result "Found sun stdio compiler" # sun stdio compiler # XXX: These haven't been fully tested. define SHOBJ_CFLAGS -KPIC define SHOBJ_LDFLAGS "-G" define SH_CFLAGS -KPIC define SH_LINKFLAGS -Wl,-export-dynamic define SH_SOPREFIX -Wl,-h, } } *-*-hpux { # XXX: These haven't been tested define SHOBJ_CFLAGS "+O3 +z" define SHOBJ_LDFLAGS -b define SH_CFLAGS +z define SH_LINKFLAGS -Wl,+s define LD_LIBRARY_PATH SHLIB_PATH } *-*-haiku { define SHOBJ_CFLAGS "" define SHOBJ_LDFLAGS -shared define SH_CFLAGS "" define SH_LDFLAGS -shared define SH_LINKFLAGS "" define SH_SOPREFIX "" define LD_LIBRARY_PATH LIBRARY_PATH } } if {![is-defined SHOBJ_LDFLAGS_R]} { define SHOBJ_LDFLAGS_R [get-define SHOBJ_LDFLAGS] } |
Added autosetup/cc.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 | # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # @synopsis: # # The 'cc' module supports checking various 'features' of the C or C++ # compiler/linker environment. Common commands are 'cc-check-includes', # 'cc-check-types', 'cc-check-functions', 'cc-with' and 'make-config-header' # # The following environment variables are used if set: # ## CC - C compiler ## CXX - C++ compiler ## CPP - C preprocessor ## CCACHE - Set to "none" to disable automatic use of ccache ## CPPFLAGS - Additional C preprocessor compiler flags (C and C++), before CFLAGS, CXXFLAGS ## CFLAGS - Additional C compiler flags ## CXXFLAGS - Additional C++ compiler flags ## LDFLAGS - Additional compiler flags during linking ## LINKFLAGS - ?How is this different from LDFLAGS? ## LIBS - Additional libraries to use (for all tests) ## CROSS - Tool prefix for cross compilation # # The following variables are defined from the corresponding # environment variables if set. # ## CC_FOR_BUILD ## LD use system options {} # Checks for the existence of the given function by linking # proc cctest_function {function} { cctest -link 1 -declare "extern void $function\(void);" -code "$function\();" } # Checks for the existence of the given type by compiling proc cctest_type {type} { cctest -code "$type _x;" } # Checks for the existence of the given type/structure member. # e.g. "struct stat.st_mtime" proc cctest_member {struct_member} { # split at the first dot regexp {^([^.]+)[.](.*)$} $struct_member -> struct member cctest -code "static $struct _s; return sizeof(_s.$member);" } # Checks for the existence of the given define by compiling # proc cctest_define {name} { cctest -code "#ifndef $name\n#error not defined\n#endif" } # Checks for the existence of the given name either as # a macro (#define) or an rvalue (such as an enum) # proc cctest_decl {name} { cctest -code "#ifndef $name\n(void)$name;\n#endif" } # @cc-check-sizeof type ... # # Checks the size of the given types (between 1 and 32, inclusive). # Defines a variable with the size determined, or 'unknown' otherwise. # e.g. for type 'long long', defines 'SIZEOF_LONG_LONG'. # Returns the size of the last type. # proc cc-check-sizeof {args} { foreach type $args { msg-checking "Checking for sizeof $type..." set size unknown # Try the most common sizes first foreach i {4 8 1 2 16 32} { if {[cctest -code "static int _x\[sizeof($type) == $i ? 1 : -1\] = { 1 };"]} { set size $i break } } msg-result $size set define [feature-define-name $type SIZEOF_] define $define $size } # Return the last result get-define $define } # Checks for each feature in $list by using the given script. # # When the script is evaluated, $each is set to the feature # being checked, and $extra is set to any additional cctest args. # # Returns 1 if all features were found, or 0 otherwise. proc cc-check-some-feature {list script} { set ret 1 foreach each $list { if {![check-feature $each $script]} { set ret 0 } } return $ret } # @cc-check-includes includes ... # # Checks that the given include files can be used. proc cc-check-includes {args} { cc-check-some-feature $args { set with {} if {[dict exists $::autosetup(cc-include-deps) $each]} { set deps [dict keys [dict get $::autosetup(cc-include-deps) $each]] msg-quiet cc-check-includes {*}$deps foreach i $deps { if {[have-feature $i]} { lappend with $i } } } if {[llength $with]} { cc-with [list -includes $with] { cctest -includes $each } } else { cctest -includes $each } } } # @cc-include-needs include required ... # # Ensures that when checking for '$include', a check is first # made for each '$required' file, and if found, it is included with '#include'. proc cc-include-needs {file args} { foreach depfile $args { dict set ::autosetup(cc-include-deps) $file $depfile 1 } } # @cc-check-types type ... # # Checks that the types exist. proc cc-check-types {args} { cc-check-some-feature $args { cctest_type $each } } # @cc-check-defines define ... # # Checks that the given preprocessor symbols are defined. proc cc-check-defines {args} { cc-check-some-feature $args { cctest_define $each } } # @cc-check-decls name ... # # Checks that each given name is either a preprocessor symbol or rvalue # such as an enum. Note that the define used is 'HAVE_DECL_xxx' # rather than 'HAVE_xxx'. proc cc-check-decls {args} { set ret 1 foreach name $args { msg-checking "Checking for $name..." set r [cctest_decl $name] define-feature "decl $name" $r if {$r} { msg-result "ok" } else { msg-result "not found" set ret 0 } } return $ret } # @cc-check-functions function ... # # Checks that the given functions exist (can be linked). proc cc-check-functions {args} { cc-check-some-feature $args { cctest_function $each } } # @cc-check-members type.member ... # # Checks that the given type/structure members exist. # A structure member is of the form 'struct stat.st_mtime'. proc cc-check-members {args} { cc-check-some-feature $args { cctest_member $each } } # @cc-check-function-in-lib function libs ?otherlibs? # # Checks that the given function can be found in one of the libs. # # First checks for no library required, then checks each of the libraries # in turn. # # If the function is found, the feature is defined and 'lib_$function' is defined # to '-l$lib' where the function was found, or "" if no library required. # In addition, '-l$lib' is prepended to the 'LIBS' define. # # If additional libraries may be needed for linking, they should be specified # with '$extralibs' as '-lotherlib1 -lotherlib2'. # These libraries are not automatically added to 'LIBS'. # # Returns 1 if found or 0 if not. # proc cc-check-function-in-lib {function libs {otherlibs {}}} { msg-checking "Checking libs for $function..." set found 0 cc-with [list -libs $otherlibs] { if {[cctest_function $function]} { msg-result "none needed" define lib_$function "" incr found } else { foreach lib $libs { cc-with [list -libs -l$lib] { if {[cctest_function $function]} { msg-result -l$lib define lib_$function -l$lib # prepend to LIBS define LIBS "-l$lib [get-define LIBS]" incr found break } } } } } define-feature $function $found if {!$found} { msg-result "no" } return $found } # @cc-check-tools tool ... # # Checks for existence of the given compiler tools, taking # into account any cross compilation prefix. # # For example, when checking for 'ar', first 'AR' is checked on the command # line and then in the environment. If not found, '${host}-ar' or # simply 'ar' is assumed depending upon whether cross compiling. # The path is searched for this executable, and if found 'AR' is defined # to the executable name. # Note that even when cross compiling, the simple 'ar' is used as a fallback, # but a warning is generated. This is necessary for some toolchains. # # It is an error if the executable is not found. # proc cc-check-tools {args} { foreach tool $args { set TOOL [string toupper $tool] set exe [get-env $TOOL [get-define cross]$tool] if {[find-executable $exe]} { define $TOOL $exe continue } if {[find-executable $tool]} { msg-result "Warning: Failed to find $exe, falling back to $tool which may be incorrect" define $TOOL $tool continue } user-error "Failed to find $exe" } } # @cc-check-progs prog ... # # Checks for existence of the given executables on the path. # # For example, when checking for 'grep', the path is searched for # the executable, 'grep', and if found 'GREP' is defined as 'grep'. # # If the executable is not found, the variable is defined as 'false'. # Returns 1 if all programs were found, or 0 otherwise. # proc cc-check-progs {args} { set failed 0 foreach prog $args { set PROG [string toupper $prog] msg-checking "Checking for $prog..." if {![find-executable $prog]} { msg-result no define $PROG false incr failed } else { msg-result ok define $PROG $prog } } expr {!$failed} } # @cc-path-progs prog ... # # Like cc-check-progs, but sets the define to the full path rather # than just the program name. # proc cc-path-progs {args} { set failed 0 foreach prog $args { set PROG [string toupper $prog] msg-checking "Checking for $prog..." set path [find-executable-path $prog] if {$path eq ""} { msg-result no define $PROG false incr failed } else { msg-result $path define $PROG $path } } expr {!$failed} } # Adds the given settings to $::autosetup(ccsettings) and # returns the old settings. # proc cc-add-settings {settings} { if {[llength $settings] % 2} { autosetup-error "settings list is missing a value: $settings" } set prev [cc-get-settings] # workaround a bug in some versions of jimsh by forcing # conversion of $prev to a list llength $prev array set new $prev foreach {name value} $settings { switch -exact -- $name { -cflags - -includes { # These are given as lists lappend new($name) {*}[list-non-empty $value] } -declare { lappend new($name) $value } -libs { # Note that new libraries are added before previous libraries set new($name) [list {*}[list-non-empty $value] {*}$new($name)] } -link - -lang - -nooutput { set new($name) $value } -source - -sourcefile - -code { # XXX: These probably are only valid directly from cctest set new($name) $value } default { autosetup-error "unknown cctest setting: $name" } } } cc-store-settings [array get new] return $prev } proc cc-store-settings {new} { set ::autosetup(ccsettings) $new } proc cc-get-settings {} { return $::autosetup(ccsettings) } # Similar to cc-add-settings, but each given setting # simply replaces the existing value. # # Returns the previous settings proc cc-update-settings {args} { set prev [cc-get-settings] cc-store-settings [dict merge $prev $args] return $prev } # @cc-with settings ?{ script }? # # Sets the given 'cctest' settings and then runs the tests in '$script'. # Note that settings such as '-lang' replace the current setting, while # those such as '-includes' are appended to the existing setting. # # If no script is given, the settings become the default for the remainder # of the 'auto.def' file. # ## cc-with {-lang c++} { ## # This will check with the C++ compiler ## cc-check-types bool ## cc-with {-includes signal.h} { ## # This will check with the C++ compiler, signal.h and any existing includes. ## ... ## } ## # back to just the C++ compiler ## } # # The '-libs' setting is special in that newer values are added *before* earlier ones. # ## cc-with {-libs {-lc -lm}} { ## cc-with {-libs -ldl} { ## cctest -libs -lsocket ... ## # libs will be in this order: -lsocket -ldl -lc -lm ## } ## } # # If you wish to invoke something like cc-check-flags but not have -cflags updated, # use the following idiom: # ## cc-with {} { ## cc-check-flags ... ## } proc cc-with {settings args} { if {[llength $args] == 0} { cc-add-settings $settings } elseif {[llength $args] > 1} { autosetup-error "usage: cc-with settings ?script?" } else { set save [cc-add-settings $settings] set rc [catch {uplevel 1 [lindex $args 0]} result info] cc-store-settings $save if {$rc != 0} { return -code [dict get $info -code] $result } return $result } } # @cctest ?settings? # # Low level C/C++ compiler checker. Compiles and or links a small C program # according to the arguments and returns 1 if OK, or 0 if not. # # Supported settings are: # ## -cflags cflags A list of flags to pass to the compiler ## -includes list A list of includes, e.g. {stdlib.h stdio.h} ## -declare code Code to declare before main() ## -link 1 Don't just compile, link too ## -lang c|c++ Use the C (default) or C++ compiler ## -libs liblist List of libraries to link, e.g. {-ldl -lm} ## -code code Code to compile in the body of main() ## -source code Compile a complete program. Ignore -includes, -declare and -code ## -sourcefile file Shorthand for -source [readfile [get-define srcdir]/$file] ## -nooutput 1 Treat any compiler output (e.g. a warning) as an error # # Unless '-source' or '-sourcefile' is specified, the C program looks like: # ## #include <firstinclude> /* same for remaining includes in the list */ ## declare-code /* any code in -declare, verbatim */ ## int main(void) { ## code /* any code in -code, verbatim */ ## return 0; ## } # # And the command line looks like: # ## CC -cflags CFLAGS CPPFLAGS conftest.c -o conftest.o ## CXX -cflags CXXFLAGS CPPFLAGS conftest.cpp -o conftest.o # # And if linking: # ## CC LDFLAGS -cflags CFLAGS conftest.c -o conftest -libs LIBS ## CXX LDFLAGS -cflags CXXFLAGS conftest.c -o conftest -libs LIBS # # Any failures are recorded in 'config.log' # proc cctest {args} { set tmp conftest__ # Easiest way to merge in the settings cc-with $args { array set opts [cc-get-settings] } if {[info exists opts(-sourcefile)]} { set opts(-source) [readfile [get-define srcdir]/$opts(-sourcefile) "#error can't find $opts(-sourcefile)"] } if {[info exists opts(-source)]} { set lines $opts(-source) } else { foreach i $opts(-includes) { if {$opts(-code) ne "" && ![feature-checked $i]} { # Compiling real code with an unchecked header file # Quickly (and silently) check for it now # Remove all -includes from settings before checking set saveopts [cc-update-settings -includes {}] msg-quiet cc-check-includes $i cc-store-settings $saveopts } if {$opts(-code) eq "" || [have-feature $i]} { lappend source "#include <$i>" } } lappend source {*}$opts(-declare) lappend source "int main(void) {" lappend source $opts(-code) lappend source "return 0;" lappend source "}" set lines [join $source \n] } # Build the command line set cmdline {} lappend cmdline {*}[get-define CCACHE] switch -exact -- $opts(-lang) { c++ { set src conftest__.cpp lappend cmdline {*}[get-define CXX] set cflags [get-define CXXFLAGS] } c { set src conftest__.c lappend cmdline {*}[get-define CC] set cflags [get-define CFLAGS] } default { autosetup-error "cctest called with unknown language: $opts(-lang)" } } if {$opts(-link)} { lappend cmdline {*}[get-define LDFLAGS] } else { lappend cflags {*}[get-define CPPFLAGS] set tmp conftest__.o lappend cmdline -c } lappend cmdline {*}$opts(-cflags) {*}[get-define cc-default-debug ""] {*}$cflags lappend cmdline $src -o $tmp if {$opts(-link)} { lappend cmdline {*}$opts(-libs) {*}[get-define LIBS] } # At this point we have the complete command line and the # complete source to be compiled. Get the result from cache if # we can if {[info exists ::cc_cache($cmdline,$lines)]} { msg-checking "(cached) " set ok $::cc_cache($cmdline,$lines) if {$::autosetup(debug)} { configlog "From cache (ok=$ok): [join $cmdline]" configlog "============" configlog $lines configlog "============" } return $ok } writefile $src $lines\n set ok 1 set err [catch {exec-with-stderr {*}$cmdline} result errinfo] if {$err || ($opts(-nooutput) && [string length $result])} { configlog "Failed: [join $cmdline]" configlog $result configlog "============" configlog "The failed code was:" configlog $lines configlog "============" set ok 0 } elseif {$::autosetup(debug)} { configlog "Compiled OK: [join $cmdline]" configlog "============" configlog $lines configlog "============" } file delete $src file delete $tmp # cache it set ::cc_cache($cmdline,$lines) $ok return $ok } # @make-autoconf-h outfile ?auto-patterns=HAVE_*? ?bare-patterns=SIZEOF_*? # # Deprecated - see 'make-config-header' proc make-autoconf-h {file {autopatterns {HAVE_*}} {barepatterns {SIZEOF_* HAVE_DECL_*}}} { user-notice "*** make-autoconf-h is deprecated -- use make-config-header instead" make-config-header $file -auto $autopatterns -bare $barepatterns } # @make-config-header outfile ?-auto patternlist? ?-bare patternlist? ?-none patternlist? ?-str patternlist? ... # # Examines all defined variables which match the given patterns # and writes an include file, '$file', which defines each of these. # Variables which match '-auto' are output as follows: # - defines which have the value '0' are ignored. # - defines which have integer values are defined as the integer value. # - any other value is defined as a string, e.g. '"value"' # Variables which match '-bare' are defined as-is. # Variables which match '-str' are defined as a string, e.g. '"value"' # Variables which match '-none' are omitted. # # Note that order is important. The first pattern that matches is selected. # Default behaviour is: # ## -bare {SIZEOF_* HAVE_DECL_*} -auto HAVE_* -none * # # If the file would be unchanged, it is not written. proc make-config-header {file args} { set guard _[string toupper [regsub -all {[^a-zA-Z0-9]} [file tail $file] _]] file mkdir [file dirname $file] set lines {} lappend lines "#ifndef $guard" lappend lines "#define $guard" # Add some defaults lappend args -bare {SIZEOF_* HAVE_DECL_*} -auto HAVE_* foreach n [lsort [dict keys [all-defines]]] { set value [get-define $n] set type [calc-define-output-type $n $args] switch -exact -- $type { -bare { # Just output the value unchanged } -none { continue } -str { set value \"[string map [list \\ \\\\ \" \\\"] $value]\" } -auto { # Automatically determine the type if {$value eq "0"} { lappend lines "/* #undef $n */" continue } if {![string is integer -strict $value]} { set value \"[string map [list \\ \\\\ \" \\\"] $value]\" } } "" { continue } default { autosetup-error "Unknown type in make-config-header: $type" } } lappend lines "#define $n $value" } lappend lines "#endif" set buf [join $lines \n] write-if-changed $file $buf { msg-result "Created $file" } } proc calc-define-output-type {name spec} { foreach {type patterns} $spec { foreach pattern $patterns { if {[string match $pattern $name]} { return $type } } } return "" } proc cc-init {} { global autosetup # Initialise some values from the environment or commandline or default settings foreach i {LDFLAGS LIBS CPPFLAGS LINKFLAGS CFLAGS} { lassign $i var default define $var [get-env $var $default] } if {[env-is-set CC]} { # Set by the user, so don't try anything else set try [list [get-env CC ""]] } else { # Try some reasonable options set try [list [get-define cross]cc [get-define cross]gcc] } define CC [find-an-executable {*}$try] if {[get-define CC] eq ""} { user-error "Could not find a C compiler. Tried: [join $try ", "]" } define CPP [get-env CPP "[get-define CC] -E"] # XXX: Could avoid looking for a C++ compiler until requested # If CXX isn't found, it is set to the empty string. if {[env-is-set CXX]} { define CXX [find-an-executable -required [get-env CXX ""]] } else { define CXX [find-an-executable [get-define cross]c++ [get-define cross]g++] } # CXXFLAGS default to CFLAGS if not specified define CXXFLAGS [get-env CXXFLAGS [get-define CFLAGS]] # May need a CC_FOR_BUILD, so look for one define CC_FOR_BUILD [find-an-executable [get-env CC_FOR_BUILD ""] cc gcc false] # These start empty and never come from the user or environment define AS_CFLAGS "" define AS_CPPFLAGS "" define AS_CXXFLAGS "" define CCACHE [find-an-executable [get-env CCACHE ccache]] # If any of these are set in the environment, propagate them to the AUTOREMAKE commandline foreach i {CC CXX CCACHE CPP CFLAGS CXXFLAGS CXXFLAGS LDFLAGS LIBS CROSS CPPFLAGS LINKFLAGS CC_FOR_BUILD LD} { if {[env-is-set $i]} { # Note: If the variable is set on the command line, get-env will return that value # so the command line will continue to override the environment define-append-argv AUTOREMAKE $i=[get-env $i ""] } } # Initial cctest settings cc-store-settings {-cflags {} -includes {} -declare {} -link 0 -lang c -libs {} -code {} -nooutput 0} set autosetup(cc-include-deps) {} msg-result "C compiler...[get-define CCACHE] [get-define CC] [get-define CFLAGS] [get-define CPPFLAGS]" if {[get-define CXX] ne "false"} { msg-result "C++ compiler...[get-define CCACHE] [get-define CXX] [get-define CXXFLAGS] [get-define CPPFLAGS]" } msg-result "Build C compiler...[get-define CC_FOR_BUILD]" # On Darwin, we prefer to use -g0 to avoid creating .dSYM directories # but some compilers may not support it, so test here. switch -glob -- [get-define host] { *-*-darwin* { if {[cctest -cflags {-g0}]} { define cc-default-debug -g0 } } } if {![cc-check-includes stdlib.h]} { user-error "Compiler does not work. See config.log" } } cc-init |
Added autosetup/find_tclconfig.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | # # Run this TCL script to find and print the pathname for the tclConfig.sh # file. Used by ../configure # if {[catch { set libdir [tcl::pkgconfig get libdir,install] }]} { puts stderr "tclsh too old: does not support tcl::pkgconfig" exit 1 } if {![file exists $libdir]} { puts stderr "tclsh reported library directory \"$libdir\" does not exist" exit 1 } if {![file exists $libdir/tclConfig.sh]} { set n1 $libdir/tcl$::tcl_version if {[file exists $n1/tclConfig.sh]} { set libdir $n1 } else { puts stderr "cannot find tclConfig.sh in either $libdir or $n1" exit 1 } } puts $libdir |
Added autosetup/jimsh0.c.
more than 10,000 changes
Added autosetup/pkg-config.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 | # Copyright (c) 2016 WorkWare Systems http://www.workware.net.au/ # All rights reserved # @synopsis: # # The 'pkg-config' module allows package information to be found via 'pkg-config'. # # If not cross-compiling, the package path should be determined automatically # by 'pkg-config'. # If cross-compiling, the default package path is the compiler sysroot. # If the C compiler doesn't support '-print-sysroot', the path can be supplied # by the '--sysroot' option or by defining 'SYSROOT'. # # 'PKG_CONFIG' may be set to use an alternative to 'pkg-config'. use cc options { sysroot:dir => "Override compiler sysroot for pkg-config search path" } # @pkg-config-init ?required? # # Initialises the 'pkg-config' system. Unless '$required' is set to 0, # it is a fatal error if a usable 'pkg-config' is not found . # # This command will normally be called automatically as required, # but it may be invoked explicitly if lack of 'pkg-config' is acceptable. # # Returns 1 if ok, or 0 if 'pkg-config' not found/usable (only if '$required' is 0). # proc pkg-config-init {{required 1}} { if {[is-defined HAVE_PKG_CONFIG]} { return [get-define HAVE_PKG_CONFIG] } set found 0 define PKG_CONFIG [get-env PKG_CONFIG pkg-config] msg-checking "Checking for pkg-config..." if {[catch {exec [get-define PKG_CONFIG] --version} version]} { msg-result "[get-define PKG_CONFIG] (not found)" if {$required} { user-error "No usable pkg-config" } } else { msg-result $version define PKG_CONFIG_VERSION $version set found 1 if {[opt-str sysroot o]} { define SYSROOT [file-normalize $o] msg-result "Using specified sysroot [get-define SYSROOT]" } elseif {[get-define build] ne [get-define host]} { if {[catch {exec-with-stderr {*}[get-define CC] -print-sysroot} result errinfo] == 0} { # Use the compiler sysroot, if there is one define SYSROOT $result msg-result "Found compiler sysroot $result" } else { configlog "[get-define CC] -print-sysroot: $result" set msg "pkg-config: Cross compiling, but no compiler sysroot and no --sysroot supplied" if {$required} { user-error $msg } else { msg-result $msg } set found 0 } } if {[is-defined SYSROOT]} { set sysroot [get-define SYSROOT] # XXX: It's possible that these should be set only when invoking pkg-config global env set env(PKG_CONFIG_DIR) "" # Supposedly setting PKG_CONFIG_LIBDIR means that PKG_CONFIG_PATH is ignored, # but it doesn't seem to work that way in practice set env(PKG_CONFIG_PATH) "" # Do we need to try /usr/local as well or instead? set env(PKG_CONFIG_LIBDIR) $sysroot/usr/lib/pkgconfig:$sysroot/usr/share/pkgconfig set env(PKG_CONFIG_SYSROOT_DIR) $sysroot } } define HAVE_PKG_CONFIG $found return $found } # @pkg-config module ?requirements? # # Use 'pkg-config' to find the given module meeting the given requirements. # e.g. # ## pkg-config pango >= 1.37.0 # # If found, returns 1 and sets 'HAVE_PKG_PANGO' to 1 along with: # ## PKG_PANGO_VERSION to the found version ## PKG_PANGO_LIBS to the required libs (--libs-only-l) ## PKG_PANGO_LDFLAGS to the required linker flags (--libs-only-L) ## PKG_PANGO_CFLAGS to the required compiler flags (--cflags) # # If not found, returns 0. # proc pkg-config {module args} { set ok [pkg-config-init] msg-checking "Checking for $module $args..." if {!$ok} { msg-result "no pkg-config" return 0 } set pkgconfig [get-define PKG_CONFIG] set ret [catch {exec $pkgconfig --modversion "$module $args"} version] configlog "$pkgconfig --modversion $module $args: $version" if {$ret} { msg-result "not found" return 0 } # Sometimes --modversion succeeds but because of dependencies it isn't usable # This seems to show up with --cflags set ret [catch {exec $pkgconfig --cflags $module} cflags] if {$ret} { msg-result "unusable ($version - see config.log)" configlog "$pkgconfig --cflags $module" configlog $cflags return 0 } msg-result $version set prefix [feature-define-name $module PKG_] define HAVE_${prefix} define ${prefix}_VERSION $version define ${prefix}_CFLAGS $cflags define ${prefix}_LIBS [exec $pkgconfig --libs-only-l $module] define ${prefix}_LDFLAGS [exec $pkgconfig --libs-only-L $module] return 1 } # @pkg-config-get module setting # # Convenience access to the results of 'pkg-config'. # # For example, '[pkg-config-get pango CFLAGS]' returns # the value of 'PKG_PANGO_CFLAGS', or '""' if not defined. proc pkg-config-get {module name} { set prefix [feature-define-name $module PKG_] get-define ${prefix}_${name} "" } # @pkg-config-get-var module variable # # Return the value of the given variable from the given pkg-config module. # The module must already have been successfully detected with pkg-config. # e.g. # ## if {[pkg-config harfbuzz >= 2.5]} { ## define harfbuzz_libdir [pkg-config-get-var harfbuzz libdir] ## } # # Returns the empty string if the variable isn't defined. proc pkg-config-get-var {module variable} { set pkgconfig [get-define PKG_CONFIG] set prefix [feature-define-name $module HAVE_PKG_] exec $pkgconfig $module --variable $variable } |
Added autosetup/proj.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 | ######################################################################## # 2024 September 25 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # * May you do good and not evil. # * May you find forgiveness for yourself and forgive others. # * May you share freely, never taking more than you give. # # # ----- @module proj.tcl ----- # @section Project-agnostic Helper APIs # # # Routines for Steve Bennett's autosetup which are common to trees # managed in and around the umbrella of the SQLite project. # # The intent is that these routines be relatively generic, independent # of a given project. # # For practical purposes, the copy of this file hosted in the SQLite # project is the "canonical" one: # # https://sqlite.org/src/file/autosetup/proj.tcl # # This file was initially derived from one used in the libfossil # project, authored by the same person who ported it here, and this is # noted here only as an indication that there are no licensing issues # despite this code having a handful of near-twins running around a # handful of third-party source trees. # # Design notes: # # - Symbols with _ separators are intended for internal use within # this file, and are not part of the API which auto.def files should # rely on. Symbols with - separators are public APIs. # # - By and large, autosetup prefers to update global state with the # results of feature checks, e.g. whether the compiler supports flag # --X. In this developer's opinion that (A) causes more confusion # than it solves[^1] and (B) adds an unnecessary layer of "voodoo" # between the autosetup user and its internals. This module, in # contrast, instead injects the results of its own tests into # well-defined variables and leaves the integration of those values # to the caller's discretion. # # [1]: As an example: testing for the -rpath flag, using # cc-check-flags, can break later checks which use # [cc-check-function-in-lib ...] because the resulting -rpath flag # implicitly becomes part of those tests. In the case of an rpath # test, downstream tests may not like the $prefix/lib path added by # the rpath test. To avoid such problems, we avoid (intentionally) # updating global state via feature tests. # # # $proj__Config is an internal-use-only array for storing whatever generic # internal stuff we need stored. # array set ::proj__Config [subst { self-tests [get-env proj.self-tests 0] verbose-assert [get-env proj.assert-verbose 0] isatty [isatty? stdout] }] # # List of dot-in files to filter in the final stages of # configuration. Some configuration steps may append to this. Each # one in this list which exists will trigger the generation of a # file with that same name, minus the ".in", in the build directory # (which differ from the source dir in out-of-tree builds). # # See: proj-dot-ins-append and proj-dot-ins-process # set ::proj__Config(dot-in-files) [list] # # @proj-warn msg # # Emits a warning message to stderr. All args are appended with a # space between each. # proc proj-warn {args} { show-notices puts stderr [join [list "WARNING:" \[ [proj-scope 1] \]: {*}$args] " "] } # # Internal impl of [proj-fatal] and [proj-error]. It must be called # using tailcall. # proc proj__faterr {failMode args} { show-notices set lvl 1 while {"-up" eq [lindex $args 0]} { set args [lassign $args -] incr lvl } if {$failMode} { puts stderr [join [list "FATAL:" \[ [proj-scope $lvl] \]: {*}$args]] exit 1 } else { error [join [list in \[ [proj-scope $lvl] \]: {*}$args]] } } # # @proj-fatal ?-up...? msg... # # Emits an error message to stderr and exits with non-0. All args are # appended with a space between each. # # The calling scope's name is used in the error message. To instead # use the name of a call higher up in the stack, use -up once for each # additional level. # proc proj-fatal {args} { tailcall proj__faterr 1 {*}$args } # # @proj-error ?-up...? msg... # # Works like proj-fatal but uses [error] intead of [exit]. # proc proj-error {args} { tailcall proj__faterr 0 {*}$args } # # @proj-assert script ?message? # # Kind of like a C assert: if uplevel of [list expr $script] is false, # a fatal error is triggered. The error message, by default, includes # the body of the failed assertion, but if $msg is set then that is # used instead. # proc proj-assert {script {msg ""}} { if {1 eq $::proj__Config(verbose-assert)} { msg-result [proj-bold "asserting: $script"] } if {![uplevel 1 [list expr $script]]} { if {"" eq $msg} { set msg $script } tailcall proj__faterr 1 "Assertion failed:" $msg } } # # @proj-bold str # # If this function believes that the current console might support # ANSI escape sequences then this returns $str wrapped in a sequence # to bold that text, else it returns $str as-is. # proc proj-bold {args} { if {$::autosetup(iswin) || !$::proj__Config(isatty)} { return [join $args] } return "\033\[1m${args}\033\[0m" } # # @proj-indented-notice ?-error? ?-notice? msg # # Takes a multi-line message and emits it with consistent indentation. # It does not perform any line-wrapping of its own. Which output # routine it uses depends on its flags, defaulting to msg-result. # For -error and -notice it uses user-notice. # # If the -notice flag it used then it emits using [user-notice], which # means its rendering will (A) go to stderr and (B) be delayed until # the next time autosetup goes to output a message. # # If the -error flag is provided then it renders the message # immediately to stderr and then exits. # # If neither -notice nor -error are used, the message will be sent to # stdout without delay. # proc proj-indented-notice {args} { set fErr "" set outFunc "msg-result" while {[llength $args] > 1} { switch -exact -- [lindex $args 0] { -error { set args [lassign $args fErr] set outFunc "user-notice" } -notice { set args [lassign $args -] set outFunc "user-notice" } default { break } } } set lines [split [join $args] \n] foreach line $lines { set line [string trimleft $line] if {"" eq $line} { $outFunc $line } else { $outFunc " $line" } } if {"" ne $fErr} { show-notices exit 1 } } # # @proj-is-cross-compiling # # Returns 1 if cross-compiling, else 0. # proc proj-is-cross-compiling {} { expr {[get-define host] ne [get-define build]} } # # @proj-strip-hash-comments value # # Expects to receive string input, which it splits on newlines, strips # out any lines which begin with any number of whitespace followed by # a '#', and returns a value containing the [append]ed results of each # remaining line with a \n between each. It does not strip out # comments which appear after the first non-whitespace character. # proc proj-strip-hash-comments {val} { set x {} foreach line [split $val \n] { if {![string match "#*" [string trimleft $line]]} { append x $line \n } } return $x } # # @proj-cflags-without-werror # # Fetches [define $var], strips out any -Werror entries, and returns # the new value. This is intended for temporarily stripping -Werror # from CFLAGS or CPPFLAGS within the scope of a [define-push] block. # proc proj-cflags-without-werror {{var CFLAGS}} { set rv {} foreach f [get-define $var ""] { switch -exact -- $f { -Werror {} default { lappend rv $f } } } join $rv " " } # # @proj-check-function-in-lib # # A proxy for cc-check-function-in-lib with the following differences: # # - Does not make any global changes to the LIBS define. # # - Strips out the -Werror flag from CFLAGS before running the test, # as these feature tests will often fail if -Werror is used. # # Returns the result of cc-check-function-in-lib (i.e. true or false). # The resulting linker flags are stored in the [define] named # lib_${function}. # proc proj-check-function-in-lib {function libs {otherlibs {}}} { set found 0 define-push {LIBS CFLAGS} { #puts "CFLAGS before=[get-define CFLAGS]" define CFLAGS [proj-cflags-without-werror] #puts "CFLAGS after =[get-define CFLAGS]" set found [cc-check-function-in-lib $function $libs $otherlibs] } return $found } # # @proj-search-for-header-dir ?-dirs LIST? ?-subdirs LIST? header # # Searches for $header in a combination of dirs and subdirs, specified # by the -dirs {LIST} and -subdirs {LIST} flags (each of which have # sane defaults). Returns either the first matching dir or an empty # string. The return value does not contain the filename part. # proc proj-search-for-header-dir {header args} { set subdirs {include} set dirs {/usr /usr/local /mingw} # Debatable: # if {![proj-is-cross-compiling]} { # lappend dirs [get-define prefix] # } while {[llength $args]} { switch -exact -- [lindex $args 0] { -dirs { set args [lassign $args - dirs] } -subdirs { set args [lassign $args - subdirs] } default { proj-error "Unhandled argument: $args" } } } foreach dir $dirs { foreach sub $subdirs { if {[file exists $dir/$sub/$header]} { return "$dir/$sub" } } } return "" } # # @proj-find-executable-path ?-v? binaryName # # Works similarly to autosetup's [find-executable-path $binName] but: # # - If the first arg is -v, it's verbose about searching, else it's quiet. # # Returns the full path to the result or an empty string. # proc proj-find-executable-path {args} { set binName $args set verbose 0 if {[lindex $args 0] eq "-v"} { set verbose 1 set args [lassign $args - binName] msg-checking "Looking for $binName ... " } set check [find-executable-path $binName] if {$verbose} { if {"" eq $check} { msg-result "not found" } else { msg-result $check } } return $check } # # @proj-bin-define binName ?defName? # # Uses [proj-find-executable-path $binName] to (verbosely) search for # a binary, sets a define (see below) to the result, and returns the # result (an empty string if not found). # # The define'd name is: If $defName is not empty, it is used as-is. If # $defName is empty then "BIN_X" is used, where X is the upper-case # form of $binName with any '-' characters replaced with '_'. # proc proj-bin-define {binName {defName {}}} { set check [proj-find-executable-path -v $binName] if {"" eq $defName} { set defName "BIN_[string toupper [string map {- _} $binName]]" } define $defName $check return $check } # # @proj-first-bin-of bin... # # Looks for the first binary found of the names passed to this # function. If a match is found, the full path to that binary is # returned, else "" is returned. # # Despite using cc-path-progs to do the search, this function clears # any define'd name that function stores for the result (because the # caller has no sensible way of knowing which result it was unless # they pass only a single argument). # proc proj-first-bin-of {args} { set rc "" foreach b $args { set u [string toupper $b] # Note that cc-path-progs defines $u to "false" if it finds no # match. if {[cc-path-progs $b]} { set rc [get-define $u] } undefine $u if {"" ne $rc} break } return $rc } # # @proj-opt-was-provided key # # Returns 1 if the user specifically provided the given configure flag # or if it was specifically set using proj-opt-set, else 0. This can # be used to distinguish between options which have a default value # and those which were explicitly provided by the user, even if the # latter is done in a way which uses the default value. # # For example, with a configure flag defined like: # # { foo-bar:=baz => {its help text} } # # This function will, when passed foo-bar, return 1 only if the user # passes --foo-bar to configure, even if that invocation would resolve # to the default value of baz. If the user does not explicitly pass in # --foo-bar (with or without a value) then this returns 0. # # Calling [proj-opt-set] is, for purposes of the above, equivalent to # explicitly passing in the flag. # # Note: unlike most functions which deal with configure --flags, this # one does not validate that $key refers to a pre-defined flag. i.e. # it accepts arbitrary keys, even those not defined via an [options] # call. [proj-opt-set] manipulates the internal list of flags, such # that new options set via that function will cause this function to # return true. (That's an unintended and unavoidable side-effect, not # specifically a feature which should be made use of.) # proc proj-opt-was-provided {key} { dict exists $::autosetup(optset) $key } # # @proj-opt-set flag ?val? # # Force-set autosetup option $flag to $val. The value can be fetched # later with [opt-val], [opt-bool], and friends. # # Returns $val. # proc proj-opt-set {flag {val 1}} { if {$flag ni $::autosetup(options)} { # We have to add this to autosetup(options) or else future calls # to [opt-bool $flag] will fail validation of $flag. lappend ::autosetup(options) $flag } dict set ::autosetup(optset) $flag $val return $val } # # @proj-opt-exists flag # # Returns 1 if the given flag has been defined as a legal configure # option, else returns 0. # proc proj-opt-exists {flag} { expr {$flag in $::autosetup(options)}; } # # @proj-val-truthy val # # Returns 1 if $val appears to be a truthy value, else returns # 0. Truthy values are any of {1 on true yes enabled} # proc proj-val-truthy {val} { expr {$val in {1 on true yes enabled}} } # # @proj-opt-truthy flag # # Returns 1 if [opt-val $flag] appears to be a truthy value or # [opt-bool $flag] is true. See proj-val-truthy. # proc proj-opt-truthy {flag} { if {[proj-val-truthy [opt-val $flag]]} { return 1 } set rc 0 catch { # opt-bool will throw if $flag is not a known boolean flag set rc [opt-bool $flag] } return $rc } # # @proj-if-opt-truthy boolFlag thenScript ?elseScript? # # If [proj-opt-truthy $flag] is true, eval $then, else eval $else. # proc proj-if-opt-truthy {boolFlag thenScript {elseScript {}}} { if {[proj-opt-truthy $boolFlag]} { uplevel 1 $thenScript } else { uplevel 1 $elseScript } } # # @proj-define-for-opt flag def ?msg? ?iftrue? ?iffalse? # # If [proj-opt-truthy $flag] then [define $def $iftrue] else [define # $def $iffalse]. If $msg is not empty, output [msg-checking $msg] and # a [msg-results ...] which corresponds to the result. Returns 1 if # the opt-truthy check passes, else 0. # proc proj-define-for-opt {flag def {msg ""} {iftrue 1} {iffalse 0}} { if {"" ne $msg} { msg-checking "$msg " } set rcMsg "" set rc 0 if {[proj-opt-truthy $flag]} { define $def $iftrue set rc 1 } else { define $def $iffalse } switch -- [proj-val-truthy [get-define $def]] { 0 { set rcMsg no } 1 { set rcMsg yes } } if {"" ne $msg} { msg-result $rcMsg } return $rc } # # @proj-opt-define-bool ?-v? optName defName ?descr? # # Checks [proj-opt-truthy $optName] and calls [define $defName X] # where X is 0 for false and 1 for true. $descr is an optional # [msg-checking] argument which defaults to $defName. Returns X. # # If args[0] is -v then the boolean semantics are inverted: if # the option is set, it gets define'd to 0, else 1. Returns the # define'd value. # proc proj-opt-define-bool {args} { set invert 0 if {[lindex $args 0] eq "-v"} { incr invert lassign $args - optName defName descr } else { lassign $args optName defName descr } if {"" eq $descr} { set descr $defName } #puts "optName=$optName defName=$defName descr=$descr" set rc 0 msg-checking "[join $descr] ... " set rc [proj-opt-truthy $optName] if {$invert} { set rc [expr {!$rc}] } msg-result $rc define $defName $rc return $rc } # # @proj-check-module-loader # # Check for module-loading APIs (libdl/libltdl)... # # Looks for libltdl or dlopen(), the latter either in -ldl or built in # to libc (as it is on some platforms). Returns 1 if found, else # 0. Either way, it `define`'s: # # - HAVE_LIBLTDL to 1 or 0 if libltdl is found/not found # - HAVE_LIBDL to 1 or 0 if dlopen() is found/not found # - LDFLAGS_MODULE_LOADER one of ("-lltdl", "-ldl", or ""), noting # that -ldl may legally be empty on some platforms even if # HAVE_LIBDL is true (indicating that dlopen() is available without # extra link flags). LDFLAGS_MODULE_LOADER also gets "-rdynamic" appended # to it because otherwise trying to open DLLs will result in undefined # symbol errors. # # Note that if it finds LIBLTDL it does not look for LIBDL, so will # report only that is has LIBLTDL. # proc proj-check-module-loader {} { msg-checking "Looking for module-loader APIs... " if {99 ne [get-define LDFLAGS_MODULE_LOADER 99]} { if {1 eq [get-define HAVE_LIBLTDL 0]} { msg-result "(cached) libltdl" return 1 } elseif {1 eq [get-define HAVE_LIBDL 0]} { msg-result "(cached) libdl" return 1 } # else: wha??? } set HAVE_LIBLTDL 0 set HAVE_LIBDL 0 set LDFLAGS_MODULE_LOADER "" set rc 0 puts "" ;# cosmetic kludge for cc-check-XXX if {[cc-check-includes ltdl.h] && [cc-check-function-in-lib lt_dlopen ltdl]} { set HAVE_LIBLTDL 1 set LDFLAGS_MODULE_LOADER "-lltdl -rdynamic" msg-result " - Got libltdl." set rc 1 } elseif {[cc-with {-includes dlfcn.h} { cctest -link 1 -declare "extern char* dlerror(void);" -code "dlerror();"}]} { msg-result " - This system can use dlopen() without -ldl." set HAVE_LIBDL 1 set LDFLAGS_MODULE_LOADER "" set rc 1 } elseif {[cc-check-includes dlfcn.h]} { set HAVE_LIBDL 1 set rc 1 if {[cc-check-function-in-lib dlopen dl]} { msg-result " - dlopen() needs libdl." set LDFLAGS_MODULE_LOADER "-ldl -rdynamic" } else { msg-result " - dlopen() not found in libdl. Assuming dlopen() is built-in." set LDFLAGS_MODULE_LOADER "-rdynamic" } } define HAVE_LIBLTDL $HAVE_LIBLTDL define HAVE_LIBDL $HAVE_LIBDL define LDFLAGS_MODULE_LOADER $LDFLAGS_MODULE_LOADER return $rc } # # @proj-no-check-module-loader # # Sets all flags which would be set by proj-check-module-loader to # empty/falsy values, as if those checks had failed to find a module # loader. Intended to be called in place of that function when # a module loader is explicitly not desired. # proc proj-no-check-module-loader {} { define HAVE_LIBDL 0 define HAVE_LIBLTDL 0 define LDFLAGS_MODULE_LOADER "" } # # @proj-file-content ?-trim? filename # # Opens the given file, reads all of its content, and returns it. If # the first arg is -trim, the contents of the file named by the second # argument are trimmed before returning them. # proc proj-file-content {args} { set trim 0 set fname $args if {"-trim" eq [lindex $args 0]} { set trim 1 lassign $args - fname } set fp [open $fname rb] set rc [read $fp] close $fp if {$trim} { return [string trim $rc] } return $rc } # # @proj-file-conent filename # # Returns the contents of the given file as an array of lines, with # the EOL stripped from each input line. # proc proj-file-content-list {fname} { set fp [open $fname rb] set rc {} while { [gets $fp line] >= 0 } { lappend rc $line } close $fp return $rc } # # @proj-file-write ?-ro? fname content # # Works like autosetup's [writefile] but explicitly uses binary mode # to avoid EOL translation on Windows. If $fname already exists, it is # overwritten, even if it's flagged as read-only. # proc proj-file-write {args} { if {"-ro" eq [lindex $args 0]} { lassign $args ro fname content } else { set ro "" lassign $args fname content } file delete -force -- $fname; # in case it's read-only set f [open $fname wb] puts -nonewline $f $content close $f if {"" ne $ro} { catch { exec chmod -w $fname #file attributes -w $fname; #jimtcl has no 'attributes' } } } # # @proj-check-compile-commands ?configFlag? # # Checks the compiler for compile_commands.json support. If passed an # argument it is assumed to be the name of an autosetup boolean config # which controls whether to run/skip this check. # # Returns 1 if supported, else 0, and defines HAVE_COMPILE_COMMANDS to # that value. Defines MAKE_COMPILATION_DB to "yes" if supported, "no" # if not. The use of MAKE_COMPILATION_DB is deprecated/discouraged: # HAVE_COMPILE_COMMANDS is preferred. # # ACHTUNG: this test has a long history of false positive results # because of compilers reacting differently to the -MJ flag. # proc proj-check-compile-commands {{configFlag {}}} { msg-checking "compile_commands.json support... " if {"" ne $configFlag && ![proj-opt-truthy $configFlag]} { msg-result "explicitly disabled" define HAVE_COMPILE_COMMANDS 0 define MAKE_COMPILATION_DB no return 0 } else { if {[cctest -lang c -cflags {/dev/null -MJ} -source {}]} { # This test reportedly incorrectly succeeds on one of # Martin G.'s older systems. drh also reports a false # positive on an unspecified older Mac system. msg-result "compiler supports compile_commands.json" define MAKE_COMPILATION_DB yes; # deprecated define HAVE_COMPILE_COMMANDS 1 return 1 } else { msg-result "compiler does not support compile_commands.json" define MAKE_COMPILATION_DB no define HAVE_COMPILE_COMMANDS 0 return 0 } } } # # @proj-touch filename # # Runs the 'touch' external command on one or more files, ignoring any # errors. # proc proj-touch {filename} { catch { exec touch {*}$filename } } # # @proj-make-from-dot-in ?-touch? infile ?outfile? # # Uses [make-template] to create makefile(-like) file(s) $outfile from # $infile but explicitly makes the output read-only, to avoid # inadvertent editing (who, me?). # # If $outfile is empty then: # # - If $infile is a 2-element list, it is assumed to be an in/out pair, # and $outfile is set from the 2nd entry in that list. Else... # # - $outfile is set to $infile stripped of its extension. # # If the first argument is -touch then the generated file is touched # to update its timestamp. This can be used as a workaround for # cases where (A) autosetup does not update the file because it was # not really modified and (B) the file *really* needs to be updated to # please the build process. # # Failures when running chmod or touch are silently ignored. # proc proj-make-from-dot-in {args} { set fIn "" set fOut "" set touch 0 if {[lindex $args 0] eq "-touch"} { set touch 1 lassign $args - fIn fOut } else { lassign $args fIn fOut } if {"" eq $fOut} { if {[llength $fIn]>1} { lassign $fIn fIn fOut } else { set fOut [file rootname $fIn] } } #puts "filenames=$filename" if {[file exists $fOut]} { catch { exec chmod u+w $fOut } } #puts "making template: $fIn ==> $fOut" #define-push {top_srcdir} { #puts "--- $fIn $fOut top_srcdir=[get-define top_srcdir]" make-template $fIn $fOut #puts "--- $fIn $fOut top_srcdir=[get-define top_srcdir]" # make-template modifies top_srcdir #} if {$touch} { proj-touch $fOut } catch { exec chmod -w $fOut #file attributes -w $f; #jimtcl has no 'attributes' } } # # @proj-check-profile-flag ?flagname? # # Checks for the boolean configure option named by $flagname. If set, # it checks if $CC seems to refer to gcc. If it does (or appears to) # then it defines CC_PROFILE_FLAG to "-pg" and returns 1, else it # defines CC_PROFILE_FLAG to "" and returns 0. # # Note that the resulting flag must be added to both CFLAGS and # LDFLAGS in order for binaries to be able to generate "gmon.out". In # order to avoid potential problems with escaping, space-containing # tokens, and interfering with autosetup's use of these vars, this # routine does not directly modify CFLAGS or LDFLAGS. # proc proj-check-profile-flag {{flagname profile}} { #puts "flagname=$flagname ?[proj-opt-truthy $flagname]?" if {[proj-opt-truthy $flagname]} { set CC [get-define CC] regsub {.*ccache *} $CC "" CC # ^^^ if CC="ccache gcc" then [exec] treats "ccache gcc" as a # single binary name and fails. So strip any leading ccache part # for this purpose. if { ![catch { exec $CC --version } msg]} { if {[string first gcc $CC] != -1} { define CC_PROFILE_FLAG "-pg" return 1 } } } define CC_PROFILE_FLAG "" return 0 } # # @proj-looks-like-windows ?key? # # Returns 1 if this appears to be a Windows environment (MinGw, # Cygwin, MSys), else returns 0. The optional argument is the name of # an autosetup define which contains platform name info, defaulting to # "host" (meaning, somewhat counterintuitively, the target system, not # the current host). The other legal value is "build" (the build # machine, i.e. the local host). If $key == "build" then some # additional checks may be performed which are not applicable when # $key == "host". # proc proj-looks-like-windows {{key host}} { global autosetup switch -glob -- [get-define $key] { *-*-ming* - *-*-cygwin - *-*-msys - *windows* { return 1 } } if {$key eq "build"} { # These apply only to the local OS, not a cross-compilation target, # as the above check potentially can. if {$::autosetup(iswin)} { return 1 } if {[find-an-executable cygpath] ne "" || $::tcl_platform(os) eq "Windows NT"} { return 1 } } return 0 } # # @proj-looks-like-mac ?key? # # Looks at either the 'host' (==compilation target platform) or # 'build' (==the being-built-on platform) define value and returns if # if that value seems to indicate that it represents a Mac platform, # else returns 0. # proc proj-looks-like-mac {{key host}} { switch -glob -- [get-define $key] { *-*-darwin* { # https://sqlite.org/forum/forumpost/7b218c3c9f207646 # There's at least one Linux out there which matches *apple*. return 1 } default { return 0 } } } # # @proj-exe-extension # # Checks autosetup's "host" and "build" defines to see if the build # host and target are Windows-esque (Cygwin, MinGW, MSys). If the # build environment is then BUILD_EXEEXT is [define]'d to ".exe", else # "". If the target, a.k.a. "host", is then TARGET_EXEEXT is # [define]'d to ".exe", else "". # proc proj-exe-extension {} { set rH "" set rB "" if {[proj-looks-like-windows host]} { set rH ".exe" } if {[proj-looks-like-windows build]} { set rB ".exe" } define BUILD_EXEEXT $rB define TARGET_EXEEXT $rH } # # @proj-dll-extension # # Works like proj-exe-extension except that it defines BUILD_DLLEXT # and TARGET_DLLEXT to one of (.so, ,dll, .dylib). # # Trivia: for .dylib files, the linker needs the -dynamiclib flag # instead of -shared. # proc proj-dll-extension {} { set inner {{key} { if {[proj-looks-like-mac $key]} { return ".dylib" } if {[proj-looks-like-windows $key]} { return ".dll" } return ".so" }} define BUILD_DLLEXT [apply $inner build] define TARGET_DLLEXT [apply $inner host] } # # @proj-lib-extension # # Static-library counterpart of proj-dll-extension. Defines # BUILD_LIBEXT and TARGET_LIBEXT to the conventional static library # extension for the being-built-on resp. the target platform. # proc proj-lib-extension {} { set inner {{key} { switch -glob -- [get-define $key] { *-*-ming* - *-*-cygwin - *-*-msys { return ".a" # ^^^ this was ".lib" until 2025-02-07. See # https://sqlite.org/forum/forumpost/02db2d4240 } default { return ".a" } } }} define BUILD_LIBEXT [apply $inner build] define TARGET_LIBEXT [apply $inner host] } # # @proj-file-extensions # # Calls all of the proj-*-extension functions. # proc proj-file-extensions {} { proj-exe-extension proj-dll-extension proj-lib-extension } # # @proj-affirm-files-exist ?-v? filename... # # Expects a list of file names. If any one of them does not exist in # the filesystem, it fails fatally with an informative message. # Returns the last file name it checks. If the first argument is -v # then it emits msg-checking/msg-result messages for each file. # proc proj-affirm-files-exist {args} { set rc "" set verbose 0 if {[lindex $args 0] eq "-v"} { set verbose 1 set args [lrange $args 1 end] } foreach f $args { if {$verbose} { msg-checking "Looking for $f ... " } if {![file exists $f]} { user-error "not found: $f" } if {$verbose} { msg-result "" } set rc $f } return rc } # # @proj-check-emsdk # # Emscripten is used for doing in-tree builds of web-based WASM stuff, # as opposed to WASI-based WASM or WASM binaries we import from other # places. This is only set up for Unix-style OSes and is untested # anywhere but Linux. Requires that the --with-emsdk flag be # registered with autosetup. # # It looks for the SDK in the location specified by --with-emsdk. # Values of "" or "auto" mean to check for the environment var EMSDK # (which gets set by the emsdk_env.sh script from the SDK) or that # same var passed to configure. # # If the given directory is found, it expects to find emsdk_env.sh in # that directory, as well as the emcc compiler somewhere under there. # # If the --with-emsdk[=DIR] flag is explicitly provided and the SDK is # not found then a fatal error is generated, otherwise failure to find # the SDK is not fatal. # # Defines the following: # # - HAVE_EMSDK = 0 or 1 (this function's return value) # - EMSDK_HOME = "" or top dir of the emsdk # - EMSDK_ENV_SH = "" or $EMSDK_HOME/emsdk_env.sh # - BIN_EMCC = "" or $EMSDK_HOME/upstream/emscripten/emcc # # Returns 1 if EMSDK_ENV_SH is found, else 0. If EMSDK_HOME is not empty # but BIN_EMCC is then emcc was not found in the EMSDK_HOME, in which # case we have to rely on the fact that sourcing $EMSDK_ENV_SH from a # shell will add emcc to the $PATH. # proc proj-check-emsdk {} { set emsdkHome [opt-val with-emsdk] define EMSDK_HOME "" define EMSDK_ENV_SH "" define BIN_EMCC "" set hadValue [llength $emsdkHome] msg-checking "Emscripten SDK? " if {$emsdkHome in {"" "auto"}} { # Check the environment. $EMSDK gets set by sourcing emsdk_env.sh. set emsdkHome [get-env EMSDK ""] } set rc 0 if {$emsdkHome ne ""} { define EMSDK_HOME $emsdkHome set emsdkEnv "$emsdkHome/emsdk_env.sh" if {[file exists $emsdkEnv]} { msg-result "$emsdkHome" define EMSDK_ENV_SH $emsdkEnv set rc 1 set emcc "$emsdkHome/upstream/emscripten/emcc" if {[file exists $emcc]} { define BIN_EMCC $emcc } } else { msg-result "emsdk_env.sh not found in $emsdkHome" } } else { msg-result "not found" } if {$hadValue && 0 == $rc} { # Fail if it was explicitly requested but not found proj-fatal "Cannot find the Emscripten SDK" } define HAVE_EMSDK $rc return $rc } # # @proj-cc-check-Wl-flag ?flag ?args?? # # Checks whether the given linker flag (and optional arguments) can be # passed from the compiler to the linker using one of these formats: # # - -Wl,flag[,arg1[,...argN]] # - -Wl,flag -Wl,arg1 ...-Wl,argN # # If so, that flag string is returned, else an empty string is # returned. # proc proj-cc-check-Wl-flag {args} { cc-with {-link 1} { # Try -Wl,flag,...args set fli "-Wl" foreach f $args { append fli ",$f" } if {[cc-check-flags $fli]} { return $fli } # Try -Wl,flag -Wl,arg1 ...-Wl,argN set fli "" foreach f $args { append fli "-Wl,$f " } if {[cc-check-flags $fli]} { return [string trim $fli] } return "" } } # # @proj-check-rpath # # Tries various approaches to handling the -rpath link-time # flag. Defines LDFLAGS_RPATH to that/those flag(s) or an empty # string. Returns 1 if it finds an option, else 0. # # By default, the rpath is set to $prefix/lib. However, if either of # --exec-prefix=... or --libdir=... are explicitly passed to # configure then [get-define libdir] is used (noting that it derives # from exec-prefix by default). # proc proj-check-rpath {} { if {[proj-opt-was-provided libdir] || [proj-opt-was-provided exec-prefix]} { set lp "[get-define libdir]" } else { set lp "[get-define prefix]/lib" } # If we _don't_ use cc-with {} here (to avoid updating the global # CFLAGS or LIBS or whatever it is that cc-check-flags updates) then # downstream tests may fail because the resulting rpath gets # implicitly injected into them. cc-with {-link 1} { if {[cc-check-flags "-rpath $lp"]} { define LDFLAGS_RPATH "-rpath $lp" } else { set wl [proj-cc-check-Wl-flag -rpath $lp] if {"" eq $wl} { set wl [proj-cc-check-Wl-flag -R$lp] } define LDFLAGS_RPATH $wl } } expr {"" ne [get-define LDFLAGS_RPATH]} } # # @proj-check-soname ?libname? # # Checks whether CC supports the -Wl,soname,lib... flag. If so, it # returns 1 and defines LDFLAGS_SONAME_PREFIX to the flag's prefix, to # which the client would need to append "libwhatever.N". If not, it # returns 0 and defines LDFLAGS_SONAME_PREFIX to an empty string. # # The libname argument is only for purposes of running the flag # compatibility test, and is not included in the resulting # LDFLAGS_SONAME_PREFIX. It is provided so that clients may # potentially avoid some end-user confusion by using their own lib's # name here (which shows up in the "checking..." output). # proc proj-check-soname {{libname "libfoo.so.0"}} { cc-with {-link 1} { if {[cc-check-flags "-Wl,-soname,${libname}"]} { define LDFLAGS_SONAME_PREFIX "-Wl,-soname," return 1 } else { define LDFLAGS_SONAME_PREFIX "" return 0 } } } # # @proj-check-fsanitize ?list-of-opts? # # Checks whether CC supports -fsanitize=X, where X is each entry of # the given list of flags. If any of those flags are supported, it # returns the string "-fsanitize=X..." where X... is a comma-separated # list of all flags from the original set which are supported. If none # of the given options are supported then it returns an empty string. # # Example: # # set f [proj-check-fsanitize {address bounds-check just-testing}] # # Will, on many systems, resolve to "-fsanitize=address,bounds-check", # but may also resolve to "-fsanitize=address". # proc proj-check-fsanitize {{opts {address bounds-strict}}} { set sup {} foreach opt $opts { # -nooutput is used because -fsanitize=hwaddress will otherwise # pass this test on x86_64, but then warn at build time that # "hwaddress is not supported for this target". cc-with {-nooutput 1} { if {[cc-check-flags "-fsanitize=$opt"]} { lappend sup $opt } } } if {[llength $sup] > 0} { return "-fsanitize=[join $sup ,]" } return "" } # # Internal helper for proj-dump-defs-json. Expects to be passed a # [define] name and the variadic $args which are passed to # proj-dump-defs-json. If it finds a pattern match for the given # $name in the various $args, it returns the type flag for that $name, # e.g. "-str" or "-bare", else returns an empty string. # proc proj-defs-type_ {name spec} { foreach {type patterns} $spec { foreach pattern $patterns { if {[string match $pattern $name]} { return $type } } } return "" } # # Internal helper for proj-defs-format_: returns a JSON-ish quoted # form of the given string-type values. It only performs the most # basic of escaping. The input must not contain any control # characters. # proc proj-quote-str_ {value} { return \"[string map [list \\ \\\\ \" \\\"] $value]\" } # # An internal impl detail of proj-dump-defs-json. Requires a data # type specifier, as used by make-config-header, and a value. Returns # the formatted value or the value $::proj__Config(defs-skip) if the caller # should skip emitting that value. # set ::proj__Config(defs-skip) "-proj-defs-format_ sentinel" proc proj-defs-format_ {type value} { switch -exact -- $type { -bare { # Just output the value unchanged } -none { set value $::proj__Config(defs-skip) } -str { set value [proj-quote-str_ $value] } -auto { # Automatically determine the type if {![string is integer -strict $value]} { set value [proj-quote-str_ $value] } } -array { set ar {} foreach v $value { set v [proj-defs-format_ -auto $v] if {$::proj__Config(defs-skip) ne $v} { lappend ar $v } } set value "\[ [join $ar {, }] \]" } "" { set value $::proj__Config(defs-skip) } default { proj-fatal "Unknown type in proj-dump-defs-json: $type" } } return $value } # # @proj-dump-defs-json outfile ...flags # # This function works almost identically to autosetup's # make-config-header but emits its output in JSON form. It is not a # fully-functional JSON emitter, and will emit broken JSON for # complicated outputs, but should be sufficient for purposes of # emitting most configure vars (numbers and simple strings). # # In addition to the formatting flags supported by make-config-header, # it also supports: # # -array {patterns...} # # Any defines matching the given patterns will be treated as a list of # values, each of which will be formatted as if it were in an -auto {...} # set, and the define will be emitted to JSON in the form: # # "ITS_NAME": [ "value1", ...valueN ] # # Achtung: if a given -array pattern contains values which themselves # contains spaces... # # define-append foo {"-DFOO=bar baz" -DBAR="baz barre"} # # will lead to: # # ["-DFOO=bar baz", "-DBAR=\"baz", "barre\""] # # Neither is especially satisfactory (and the second is useless), and # handling of such values is subject to change if any such values ever # _really_ need to be processed by our source trees. # proc proj-dump-defs-json {file args} { file mkdir [file dirname $file] set lines {} lappend args -bare {SIZEOF_* HAVE_DECL_*} -auto HAVE_* foreach n [lsort [dict keys [all-defines]]] { set type [proj-defs-type_ $n $args] set value [proj-defs-format_ $type [get-define $n]] if {$::proj__Config(defs-skip) ne $value} { lappend lines "\"$n\": ${value}" } } set buf {} lappend buf [join $lines ",\n"] write-if-changed $file $buf { msg-result "Created $file" } } # # @proj-xfer-option-aliases map # # Expects a list of pairs of configure flags which have been # registered with autosetup, in this form: # # { alias1 => canonical1 # aliasN => canonicalN ... } # # The names must not have their leading -- part and must be in the # form which autosetup will expect for passing to [opt-val NAME] and # friends. # # Comment lines are permitted in the input. # # For each pair of ALIAS and CANONICAL, if --ALIAS is provided but # --CANONICAL is not, the value of the former is copied to the # latter. If --ALIAS is not provided, this is a no-op. If both have # explicitly been provided a fatal usage error is triggered. # # Motivation: autosetup enables "hidden aliases" in [options] lists, # and elides the aliases from --help output but does no further # handling of them. For example, when --alias is a hidden alias of # --canonical and a user passes --alias=X, [opt-val canonical] returns # no value. i.e. the script must check both [opt-val alias] and # [opt-val canonical]. The intent here is that this function be # passed such mappings immediately after [options] is called, to carry # over any values from hidden aliases into their canonical names, such # that [opt-value canonical] will return X if --alias=X is passed to # configure. # # That said: autosetup's [opt-str] does support alias forms, but it # requires that the caller know all possible aliases. It's simpler, in # terms of options handling, if there's only a single canonical name # which each down-stream call of [opt-...] has to know. # proc proj-xfer-options-aliases {mapping} { foreach {hidden - canonical} [proj-strip-hash-comments $mapping] { if {[proj-opt-was-provided $hidden]} { if {[proj-opt-was-provided $canonical]} { proj-fatal "both --$canonical and its alias --$hidden were used. Use only one or the other." } else { proj-opt-set $canonical [opt-val $hidden] } } } } # # Arguable/debatable... # # When _not_ cross-compiling and CC_FOR_BUILD is _not_ explicitly # specified, force CC_FOR_BUILD to be the same as CC, so that: # # ./configure CC=clang # # will use CC_FOR_BUILD=clang, instead of cc, for building in-tree # tools. This is based off of an email discussion and is thought to # be likely to cause less confusion than seeing 'cc' invocations # when when the user passes CC=clang. # # Sidebar: if we do this before the cc package is installed, it gets # reverted by that package. Ergo, the cc package init will tell the # user "Build C compiler...cc" shortly before we tell them otherwise. # proc proj-redefine-cc-for-build {} { if {![proj-is-cross-compiling] && [get-define CC] ne [get-define CC_FOR_BUILD] && "nope" eq [get-env CC_FOR_BUILD "nope"]} { user-notice "Re-defining CC_FOR_BUILD to CC=[get-define CC]. To avoid this, explicitly pass CC_FOR_BUILD=..." define CC_FOR_BUILD [get-define CC] } } # # @proj-which-linenoise headerFile # # Attempts to determine whether the given linenoise header file is of # the "antirez" or "msteveb" flavor. It returns 2 for msteveb, else 1 # (it does not validate that the header otherwise contains the # linenoise API). # proc proj-which-linenoise {dotH} { set srcHeader [proj-file-content $dotH] if {[string match *userdata* $srcHeader]} { return 2 } else { return 1 } } # # @proj-remap-autoconf-dir-vars # # "Re-map" the autoconf-conventional --XYZdir flags into something # which is more easily overridable from a make invocation. # # Based off of notes in <https://sqlite.org/forum/forumpost/00d12a41f7>. # # Consider: # # $ ./configure --prefix=/foo # $ make install prefix=/blah # # In that make invocation, $(libdir) would, at make-time, normally be # hard-coded to /foo/lib, rather than /blah/lib. That happens because # autosetup exports conventional $prefix-based values for the numerous # autoconfig-compatible XYZdir vars at configure-time. What we would # normally want, however, is that --libdir derives from the make-time # $(prefix). The distinction between configure-time and make-time is # the significant factor there. # # This function attempts to reconcile those vars in such a way that # they will derive, at make-time, from $(prefix) in a conventional # manner unless they are explicitly overridden at configure-time, in # which case those overrides takes precedence. # # Each autoconf-relvant --XYZ flag which is explicitly passed to # configure is exported as-is, as are those which default to some # top-level system directory, e.g. /etc or /var. All which derive # from either $prefix or $exec_prefix are exported in the form of a # Makefile var reference, e.g. libdir=${exec_prefix}/lib. Ergo, if # --exec-prefix=FOO is passed to configure, libdir will still derive, # at make-time, from whatever exec_prefix is passed to make, and will # use FOO if exec_prefix is not overridden at make-time. Without this # post-processing, libdir would be cemented in as FOO/lib at # configure-time, so could be tedious to override properly via a make # invocation. # proc proj-remap-autoconf-dir-vars {} { set prefix [get-define prefix] set exec_prefix [get-define exec_prefix $prefix] # The following var derefs must be formulated such that they are # legal for use in (A) makefiles, (B) pkgconfig files, and (C) TCL's # [subst] command. i.e. they must use the form ${X}. foreach {flag makeVar makeDeref} { exec-prefix exec_prefix ${prefix} datadir datadir ${prefix}/share mandir mandir ${datadir}/man includedir includedir ${prefix}/include bindir bindir ${exec_prefix}/bin libdir libdir ${exec_prefix}/lib sbindir sbindir ${exec_prefix}/sbin sysconfdir sysconfdir /etc sharedstatedir sharedstatedir ${prefix}/com localstatedir localstatedir /var runstatedir runstatedir /run infodir infodir ${datadir}/info libexecdir libexecdir ${exec_prefix}/libexec } { if {[proj-opt-was-provided $flag]} { define $makeVar [join [opt-val $flag]] } else { define $makeVar [join $makeDeref] } # Maintenance reminder: the [join] call is to avoid {braces} # around the output when someone passes in, # e.g. --libdir=\${prefix}/foo/bar. Debian's SQLite package build # script does that. } } # # @proj-env-file flag ?default? # # If a file named .env-$flag exists, this function returns a # trimmed copy of its contents, else it returns $dflt. The intended # usage is that things like developer-specific CFLAGS preferences can # be stored in .env-CFLAGS. # proc proj-env-file {flag {dflt ""}} { set fn ".env-${flag}" if {[file readable $fn]} { return [proj-file-content -trim $fn] } return $dflt } # # @proj-get-env var ?default? # # Extracts the value of "environment" variable $var from the first of # the following places where it's defined: # # - Passed to configure as $var=... # - Exists as an environment variable # - A file named .env-$var (see [proj-env-file]) # # If none of those are set, $dflt is returned. # proc proj-get-env {var {dflt ""}} { get-env $var [proj-env-file $var $dflt] } # # @proj-scope ?lvl? # # Returns the name of the _calling_ proc from ($lvl + 1) levels up the # call stack (where the caller's level will be 1 up from _this_ # call). If $lvl would resolve to global scope "global scope" is # returned and if it would be negative then a string indicating such # is returned (as opposed to throwing an error). # proc proj-scope {{lvl 0}} { #uplevel [expr {$lvl + 1}] {lindex [info level 0] 0} set ilvl [info level] set offset [expr {$ilvl - $lvl - 1}] if { $offset < 0} { return "invalid scope ($offset)" } elseif { $offset == 0} { return "global scope" } else { return [lindex [info level $offset] 0] } } # # Deprecated name of [proj-scope]. # proc proj-current-scope {{lvl 0}} { puts stderr \ "Deprecated proj-current-scope called from [proj-scope 1]. Use proj-scope instead." proj-scope [incr lvl] } # # Converts parts of tclConfig.sh to autosetup [define]s. # # Expects to be passed the name of a value tclConfig.sh or an empty # string. It converts certain parts of that file's contents to # [define]s (see the code for the whole list). If $tclConfigSh is an # empty string then it [define]s the various vars as empty strings. # proc proj-tclConfig-sh-to-autosetup {tclConfigSh} { set shBody {} set tclVars { TCL_INCLUDE_SPEC TCL_LIBS TCL_LIB_SPEC TCL_STUB_LIB_SPEC TCL_EXEC_PREFIX TCL_PREFIX TCL_VERSION TCL_MAJOR_VERSION TCL_MINOR_VERSION TCL_PACKAGE_PATH TCL_PATCH_LEVEL TCL_SHLIB_SUFFIX } # Build a small shell script which proxies the $tclVars from # $tclConfigSh into autosetup code... lappend shBody "if test x = \"x${tclConfigSh}\"; then" foreach v $tclVars { lappend shBody "$v= ;" } lappend shBody "else . \"${tclConfigSh}\"; fi" foreach v $tclVars { lappend shBody "echo define $v {\$$v} ;" } lappend shBody "exit" set shBody [join $shBody "\n"] #puts "shBody=$shBody\n"; exit eval [exec echo $shBody | sh] } # # @proj-tweak-default-env-dirs # # This function is not useful before [use system] is called to set up # --prefix and friends. It should be called as soon after [use system] # as feasible. # # For certain target environments, if --prefix is _not_ passed in by # the user, set the prefix to an environment-specific default. For # such environments its does [define prefix ...] and [proj-opt-set # prefix ...], but it does not process vars derived from the prefix, # e.g. exec-prefix. To do so it is generally necessary to also call # proj-remap-autoconf-dir-vars late in the config process (immediately # before ".in" files are filtered). # # Similar modifications may be made for --mandir. # # Returns 1 if it modifies the environment, else 0. # proc proj-tweak-default-env-dirs {} { set rc 0 switch -glob -- [get-define host] { *-haiku { if {![proj-opt-was-provided prefix]} { set hdir /boot/home/config/non-packaged proj-opt-set prefix $hdir define prefix $hdir incr rc } if {![proj-opt-was-provided mandir]} { set hdir /boot/system/documentation/man proj-opt-set mandir $hdir define mandir $hdir incr rc } } } return $rc } # # @proj-dot-ins-append file ?fileOut ?postProcessScript?? # # Queues up an autosetup [make-template]-style file to be processed # at a later time using [proj-dot-ins-process]. # # $file is the input file. If $fileOut is empty then this function # derives $fileOut from $file, stripping both its directory and # extension parts. i.e. it defaults to writing the output to the # current directory (typically $::autosetup(builddir)). # # If $postProcessScript is not empty then, during # [proj-dot-ins-process], it will be eval'd immediately after # processing the file. In the context of that script, the vars # $dotInsIn and $dotInsOut will be set to the input and output file # names. This can be used, for example, to make the output file # executable or perform validation on its contents. # # See [proj-dot-ins-process], [proj-dot-ins-list] # proc proj-dot-ins-append {fileIn args} { set srcdir $::autosetup(srcdir) switch -exact -- [llength $args] { 0 { lappend fileIn [file rootname [file tail $fileIn]] "" } 1 { lappend fileIn [join $args] "" } 2 { lappend fileIn {*}$args } default { proj-fatal "Too many arguments: $fileIn $args" } } #puts "******* [proj-scope]: adding [llength $fileIn]-length item: $fileIn" lappend ::proj__Config(dot-in-files) $fileIn } # # @proj-dot-ins-list # # Returns the current list of [proj-dot-ins-append]'d files, noting # that each entry is a 3-element list of (inputFileName, # outputFileName, postProcessScript). # proc proj-dot-ins-list {} { return $::proj__Config(dot-in-files) } # # @proj-dot-ins-process ?-touch? ?-validate? ?-clear? # # Each file which has previously been passed to [proj-dot-ins-append] # is processed, with its passing its in-file out-file names to # [proj-make-from-dot-in]. # # The intent is that a project accumulate any number of files to # filter and delay their actual filtering until the last stage of the # configure script, calling this function at that time. # # Optional flags: # # -touch: gets passed on to [proj-make-from-dot-in] # # -validate: after processing each file, before running the file's # associated script, if any, it runs the file through # proj-validate-no-unresolved-ats, erroring out if that does. # # -clear: after processing, empty the dot-ins list. This effectively # makes proj-dot-ins-append available for re-use. # proc proj-dot-ins-process {args} { proj-parse-flags args flags { -touch "" {return "-touch"} -clear 0 {expr 1} -validate 0 {expr 1} } #puts "args=$args"; parray flags if {[llength $args] > 0} { error "Invalid argument to [proj-scope]: $args" } foreach f $::proj__Config(dot-in-files) { proj-assert {3==[llength $f]} \ "Expecting proj-dot-ins-list to be stored in 3-entry lists. Got: $f" lassign $f fIn fOut fScript #puts "DOING $fIn ==> $fOut" proj-make-from-dot-in {*}$flags(-touch) $fIn $fOut if {$flags(-validate)} { proj-validate-no-unresolved-ats $fOut } if {"" ne $fScript} { uplevel 1 [join [list set dotInsIn $fIn \; \ set dotInsOut $fOut \; \ eval \{${fScript}\} \; \ unset dotInsIn dotInsOut]] } } if {$flags(-clear)} { set ::proj__Config(dot-in-files) [list] } } # # @proj-validate-no-unresolved-ats filenames... # # For each filename given to it, it validates that the file has no # unresolved @VAR@ references. If it finds any, it produces an error # with location information. # # Exception: if a filename matches the pattern {*[Mm]ake*} AND a given # line begins with a # (not including leading whitespace) then that # line is ignored for purposes of this validation. The intent is that # @VAR@ inside of makefile comments should not (necessarily) cause # validation to fail, as it's sometimes convenient to comment out # sections during development of a configure script and its # corresponding makefile(s). # proc proj-validate-no-unresolved-ats {args} { foreach f $args { set lnno 1 set isMake [string match {*[Mm]ake*} $f] foreach line [proj-file-content-list $f] { if {!$isMake || ![string match "#*" [string trimleft $line]]} { if {[regexp {(@[A-Za-z0-9_\.]+@)} $line match]} { error "Unresolved reference to $match at line $lnno of $f" } } incr lnno } } } # # @proj-first-file-found tgtVar fileList # # Searches $fileList for an existing file. If one is found, its name # is assigned to tgtVar and 1 is returned, else tgtVar is set to "" # and 0 is returned. # proc proj-first-file-found {tgtVar fileList} { upvar $tgtVar tgt foreach f $fileList { if {[file exists $f]} { set tgt $f return 1 } } set tgt "" return 0 } # # Defines $defName to contain makefile recipe commands for re-running # the configure script with its current set of $::argv flags. This # can be used to automatically reconfigure. # proc proj-setup-autoreconfig {defName} { define $defName \ [join [list \ cd \"$::autosetup(builddir)\" \ && [get-define AUTOREMAKE "error - missing @AUTOREMAKE@"]]] } # # @prop-append-to defineName args... # # A proxy for Autosetup's [define-append]. Appends all non-empty $args # to [define-append $defineName]. # proc proj-define-append {defineName args} { foreach a $args { if {"" ne $a} { define-append $defineName {*}$a } } } # # @prod-define-amend ?-p|-prepend? ?-d|-define? defineName args... # # A proxy for Autosetup's [define-append]. # # Appends all non-empty $args to the define named by $defineName. If # one of (-p | -prepend) are used it instead prepends them, in their # given order, to $defineName. # # If -define is used then each argument is assumed to be a [define]'d # flag and [get-define X ""] is used to fetch it. # # Re. linker flags: typically, -lXYZ flags need to be in "reverse" # order, with each -lY resolving symbols for -lX's to its left. This # order is largely historical, and not relevant on all environments, # but it is technically correct and still relevant on some # environments. # # See: proj-append-to # proc proj-define-amend {args} { set defName "" set prepend 0 set isdefs 0 set xargs [list] foreach arg $args { switch -exact -- $arg { "" {} -p - -prepend { incr prepend } -d - -define { incr isdefs } default { if {"" eq $defName} { set defName $arg } else { lappend xargs $arg } } } } if {"" eq $defName} { proj-error "Missing defineName argument in call from [proj-scope 1]" } if {$isdefs} { set args $xargs set xargs [list] foreach arg $args { lappend xargs [get-define $arg ""] } set args $xargs } # puts "**** args=$args" # puts "**** xargs=$xargs" set args $xargs if {$prepend} { lappend args {*}[get-define $defName ""] define $defName [join $args]; # join to eliminate {} entries } else { proj-define-append $defName {*}$args } } # # @proj-define-to-cflag ?-list? ?-quote? ?-zero-undef? defineName... # # Treat each argument as the name of a [define] and renders it like a # CFLAGS value in one of the following forms: # # -D$name # -D$name=integer (strict integer matches only) # '-D$name=value' (without -quote) # '-D$name="value"' (with -quote) # # It treats integers as numbers and everything else as a quoted # string, noting that it does not handle strings which themselves # contain quotes. # # The -zero-undef flag causes no -D to be emitted for integer values # of 0. # # By default it returns the result as string of all -D... flags, # but if passed the -list flag it will return a list of the # individual CFLAGS. # proc proj-define-to-cflag {args} { set rv {} proj-parse-flags args flags { -list 0 {expr 1} -quote 0 {expr 1} -zero-undef 0 {expr 1} } foreach d $args { set v [get-define $d ""] set li {} if {"" eq $d} { set v "-D${d}" } elseif {[string is integer -strict $v]} { if {!$flags(-zero-undef) || $v ne "0"} { set v "-D${d}=$v" } } elseif {$flags(-quote)} { set v "'-D${d}=\"$v\"'" } else { set v "'-D${d}=$v'" } lappend rv $v } expr {$flags(-list) ? $rv : [join $rv]} } if {0} { # Turns out that autosetup's [options-add] essentially does exactly # this... # A list of lists of Autosetup [options]-format --flags definitions. # Append to this using [proj-options-add] and use # [proj-options-combine] to merge them into a single list for passing # to [options]. # set ::proj__Config(extra-options) {} # @proj-options-add list # # Adds a list of options to the pending --flag processing. It must be # in the format used by Autosetup's [options] function. # # This will have no useful effect if called from after [options] # is called. # # Use [proj-options-combine] to get a combined list of all added # options. # # PS: when writing this i wasn't aware of autosetup's [options-add], # works quite similarly. Only the timing is different. proc proj-options-add {list} { lappend ::proj__Config(extra-options) $list } # @proj-options-combine list1 ?...listN? # # Expects each argument to be a list of options compatible with # autosetup's [options] function. This function concatenates the # contents of each list into a new top-level list, stripping the outer # list part of each argument, and returning that list # # If passed no arguments, it uses the list generated by calls to # [proj-options-add]. proc proj-options-combine {args} { set rv [list] if {0 == [llength $args]} { set args $::proj__Config(extra-options) } foreach e $args { lappend rv {*}$e } return $rv } }; # proj-options-* # Internal cache for use via proj-cache-*. array set proj__Cache {} # # @proj-cache-key arg {addLevel 0} # # Helper to generate cache keys for [proj-cache-*]. # # $addLevel should almost always be 0. # # Returns a cache key for the given argument: # # integer: relative call stack levels to get the scope name of for # use as a key. [proj-scope [expr {1 + $arg + addLevel}]] is # then used to generate the key. i.e. the default of 0 uses the # calling scope's name as the key. # # Anything else: returned as-is # proc proj-cache-key {arg {addLevel 0}} { if {[string is integer -strict $arg]} { return [proj-scope [expr {$arg + $addLevel + 1}]] } return $arg } # # @proj-cache-set ?-key KEY? ?-level 0? value # # Sets a feature-check cache entry with the given key. # # See proj-cache-key for -key's and -level's semantics, noting that # this function adds one to -level for purposes of that call. proc proj-cache-set {args} { proj-parse-flags args flags { -key => 0 -level => 0 } lassign $args val set key [proj-cache-key $flags(-key) [expr {1 + $flags(-level)}]] #puts "** fcheck set $key = $val" set ::proj__Cache($key) $val } # # @proj-cache-remove ?key? ?addLevel? # # Removes an entry from the proj-cache. proc proj-cache-remove {{key 0} {addLevel 0}} { set key [proj-cache-key $key [expr {1 + $addLevel}]] set rv "" if {[info exists ::proj__Cache($key)]} { set rv $::proj__Cache($key) unset ::proj__Cache($key) } return $rv; } # # @proj-cache-check ?-key KEY? ?-level LEVEL? tgtVarName # # Checks for a feature-check cache entry with the given key. # # If the feature-check cache has a matching entry then this function # assigns its value to tgtVar and returns 1, else it assigns tgtVar to # "" and returns 0. # # See proj-cache-key for $key's and $addLevel's semantics, noting that # this function adds one to $addLevel for purposes of that call. proc proj-cache-check {args} { proj-parse-flags args flags { -key => 0 -level => 0 } lassign $args tgtVar upvar $tgtVar tgt set rc 0 set key [proj-cache-key $flags(-key) [expr {1 + $flags(-level)}]] #puts "** fcheck get key=$key" if {[info exists ::proj__Cache($key)]} { set tgt $::proj__Cache($key) incr rc } else { set tgt "" } return $rc } # # @proj-coalesce ...args # # Returns the first argument which is not empty (eq ""), or an empty # string on no match. proc proj-coalesce {args} { foreach arg $args { if {"" ne $arg} { return $arg } } return "" } # # @proj-parse-flags argvListName targetArrayName {prototype} # # A helper to parse flags from proc argument lists. # # The first argument is the name of a var holding the args to # parse. It will be overwritten, possibly with a smaller list. # # The second argument is the name of an array variable to create in # the caller's scope. # # The third argument, $prototype, is a description of how to handle # the flags. Each entry in that list must be in one of the # following forms: # # -flag defaultValue ?-literal|-call|-apply? # script|number|incr|proc-name|{apply $aLambda} # # -flag* ...as above... # # -flag => defaultValue ?-call proc-name-and-args|-apply lambdaExpr? # # -flag* => ...as above... # # :PRAGMA # # The first two forms represents a basic flag with no associated # following argument. The third and fourth forms, called arg-consuming # flags, extract the value from the following argument in $argvName # (pneumonic: => points to the next argument.). The :PRAGMA form # offers a way to configure certain aspects of this call. # # If $argv contains any given flag from $prototype, its default value # is overridden depending on several factors: # # - If the -literal flag is used, or the flag's script is a number, # value is used verbatim. # # - Else if the -call flag is used, the argument must be a proc name # and any leading arguments, e.g. {apply $myLambda}. The proc is passed # the (flag, value) as arguments (non-consuming flags will get # passed the flag's current/starting value and consuming flags will # get the next argument). Its result becomes the result of the # flag. # # - Else if -apply X is used, it's effectively shorthand for -call # {apply X}. Its argument may either be a $lambaRef or a {{f v} # {body}} construct. # # - Else if $script is one of the following values, it is treated as # the result of... # # - incr: increments the current value of the flag. # # - Else $script is eval'd to get its result value. That result # becomes the new flag value for $tgtArrayName(-flag). This # function intercepts [return $val] from eval'ing $script. Any # empty script will result in the flag having "" assigned to it. # # Unless the -flag has a trailing asterisk, e.g. -flag*, this function # assumes that each flag is unique, and using a flag more than once # causes an error to be triggered. the -flag* forms works similarly # except that may appear in $argv any number of times: # # - For non-arg-consuming flags, each invocation of -flag causes the # result of $script to overwrite the previous value. e.g. so # {-flag* {x} {incr foo}} has a default value of x, but passing in # -flag twice would change it to the result of incrementing foo # twice. This form can be used to implement, e.g., increasing # verbosity levels by passing -verbose multiple times. # # - For arg-consuming flags, the given flag starts with value X, but # if the flag is provided in $argv, the default is cleared, then # each instance of -flag causes its value to be appended to the # result, so {-flag* => {a b c}} defaults to {a b c}, but passing # in -flag y -flag z would change it to {y z}, not {a b c y z}.. # # By default, the args list is only inspected until the first argument # which is not described by $prototype. i.e. the first "non-flag" (not # counting values consumed for flags defined like -flag => default). # The :all-flags pragma (see below) can modify this behavior. # # If a "--" flag is encountered, no more arguments are inspected as # flags unless the :all-flags pragma (see below) is in effect. The # first instance of "--" is removed from the target result list but # all remaining instances of "--" are are passed through. # # Any argvName entries not described in $prototype are considered to # be "non-flags" for purposes of this function, even if they # ostensibly look like flags. # # Returns the number of flags it processed in $argvName, not counting # "--". # # Example: # ## set args [list -foo -bar {blah} -z 8 9 10 -theEnd] ## proj-parse-flags args flags { ## -foo 0 {expr 1} ## -bar => 0 ## -no-baz 1 {return 0} ## -z 0 2 ## } # # After that $flags would contain {-foo 1 -bar {blah} -no-baz 1 -z 2} # and $args would be {8 9 10 -theEnd}. # # Pragmas: # # Passing :PRAGMAS to this function may modify how it works. The # following pragmas are supported (note the leading ":"): # # :all-flags indicates that the whole input list should be scanned, # not stopping at the first non-flag or "--". # proc proj-parse-flags {argvName tgtArrayName prototype} { upvar $argvName argv upvar $tgtArrayName outFlags array set flags {}; # staging area array set blob {}; # holds markers for various per-key state and options set incrSkip 1; # 1 if we stop at the first non-flag, else 0 # Parse $prototype for flag definitions... set n [llength $prototype] set checkProtoFlag { #puts "**** checkProtoFlag #$i of $n k=$k fv=$fv" switch -exact -- $fv { -literal { proj-assert {![info exists blob(${k}.consumes)]} set blob(${k}.script) [list expr [lindex $prototype [incr i]]] } -apply { set fv [lindex $prototype [incr i]] if {2 == [llength $fv]} { # Treat this as a lambda literal set fv [list $fv] } lappend blob(${k}.call) "apply $fv" } -call { # arg is either a proc name or {apply $aLambda} set fv [lindex $prototype [incr i]] lappend blob(${k}.call) $fv } default { proj-assert {![info exists blob(${k}.consumes)]} set blob(${k}.script) $fv } } if {$i >= $n} { proj-error -up "[proj-scope]: Missing argument for $k flag" } } for {set i 0} {$i < $n} {incr i} { set k [lindex $prototype $i] #puts "**** #$i of $n k=$k" # Check for :PRAGMA... switch -exact -- $k { :all-flags { set incrSkip 0 continue } } proj-assert {[string match -* $k]} \ "Invalid argument: $k" if {[string match {*\*} $k]} { # Re-map -foo* to -foo and flag -foo as a repeatable flag set k [string map {* ""} $k] incr blob(${k}.multi) } if {[info exists flags($k)]} { proj-error -up "[proj-scope]: Duplicated prototype for flag $k" } switch -exact -- [lindex $prototype [expr {$i + 1}]] { => { # -flag => DFLT ?-subflag arg? incr i 2 if {$i >= $n} { proj-error -up "[proj-scope]: Missing argument for $k => flag" } incr blob(${k}.consumes) set vi [lindex $prototype $i] if {$vi in {-apply -call}} { proj-error -up "[proj-scope]: Missing default value for $k flag" } else { set fv [lindex $prototype [expr {$i + 1}]] if {$fv in {-apply -call}} { incr i eval $checkProtoFlag } } } default { # -flag VALUE ?flag? SCRIPT set vi [lindex $prototype [incr i]] set fv [lindex $prototype [incr i]] eval $checkProtoFlag } } #puts "**** #$i of $n k=$k vi=$vi" set flags($k) $vi } #puts "-- flags"; parray flags #puts "-- blob"; parray blob set rc 0 set rv {}; # staging area for the target argv value set skipMode 0 set n [llength $argv] # Now look for those flags in $argv... for {set i 0} {$i < $n} {incr i} { set arg [lindex $argv $i] #puts "-- [proj-scope] arg=$arg" if {$skipMode} { lappend rv $arg } elseif {"--" eq $arg} { # "--" is the conventional way to end processing of args if {[incr blob(--)] > 1} { # Elide only the first one lappend rv $arg } incr skipMode $incrSkip } elseif {[info exists flags($arg)]} { # A known flag... set isMulti [info exists blob(${arg}.multi)] incr blob(${arg}.seen) if {1 < $blob(${arg}.seen) && !$isMulti} { proj-error -up [proj-scope] "$arg flag was used multiple times" } set vMode 0; # 0=as-is, 1=eval, 2=call set isConsuming [info exists blob(${arg}.consumes)] if {$isConsuming} { incr i if {$i >= $n} { proj-error -up [proj-scope] "is missing argument for $arg flag" } set vv [lindex $argv $i] } elseif {[info exists blob(${arg}.script)]} { set vMode 1 set vv $blob(${arg}.script) } else { set vv $flags($arg) } if {[info exists blob(${arg}.call)]} { set vMode 2 set vv [concat {*}$blob(${arg}.call) $arg $vv] } elseif {$isConsuming} { proj-assert {!$vMode} # fall through } elseif {"" eq $vv || [string is double -strict $vv]} { set vMode 0 } elseif {$vv in {incr}} { set vMode 0 switch -exact $vv { incr { set xx $flags($k); incr xx; set vv $xx; unset xx } default { proj-error "Unhandled \$vv value $vv" } } } else { set vv [list eval $vv] set vMode 1 } if {$vMode} { set code [catch [list uplevel 1 $vv] vv xopt] if {$code ni {0 2}} { return {*}$xopt $vv } } if {$isConsuming && $isMulti} { if {1 == $blob(${arg}.seen)} { # On the first hit, overwrite the default with a new list. set flags($arg) [list $vv] } else { # On subsequent hits, append to the list. lappend flags($arg) $vv } } else { set flags($arg) $vv } incr rc } else { # Non-flag incr skipMode $incrSkip lappend rv $arg } } set argv $rv array set outFlags [array get flags] #puts "-- rv=$rv argv=$argv flags="; parray flags return $rc }; # proj-parse-flags # # Older (deprecated) name of proj-parse-flags. # proc proj-parse-simple-flags {args} { tailcall proj-parse-flags {*}$args } if {$::proj__Config(self-tests)} { set __ova $::proj__Config(verbose-assert); set ::proj__Config(verbose-assert) 1 puts "Running [info script] self-tests..." # proj-cache... apply {{} { #proj-warn "Test code for proj-cache" proj-assert {![proj-cache-check -key here check]} proj-assert {"here" eq [proj-cache-key here]} proj-assert {"" eq $check} proj-cache-set -key here thevalue proj-assert {[proj-cache-check -key here check]} proj-assert {"thevalue" eq $check} proj-assert {![proj-cache-check check]} #puts "*** key = ([proj-cache-key 0])" proj-assert {"" eq $check} proj-cache-set abc proj-assert {[proj-cache-check check]} proj-assert {"abc" eq $check} #parray ::proj__Cache; proj-assert {"" ne [proj-cache-remove]} proj-assert {![proj-cache-check check]} proj-assert {"" eq [proj-cache-remove]} proj-assert {"" eq $check} }} # proj-parse-flags ... apply {{} { set foo 3 set argv {-a "hi - world" -b -b -b -- -a {bye bye} -- -d -D c -a "" --} proj-parse-flags argv flags { :all-flags -a* => "gets overwritten" -b* 7 {incr foo} -d 1 0 -D 0 1 } #puts "-- argv = $argv"; parray flags; proj-assert {"-- c --" eq $argv} proj-assert {$flags(-a) eq "{hi - world} {bye bye} {}"} proj-assert {$foo == 6} proj-assert {$flags(-b) eq $foo} proj-assert {$flags(-d) == 0} proj-assert {$flags(-D) == 1} set foo 0 foreach x $flags(-a) { proj-assert {$x in {{hi - world} {bye bye} {}}} incr foo } proj-assert {3 == $foo} set argv {-a {hi world} -b -maybe -- -a {bye bye} -- -b c --} set foo 0 proj-parse-flags argv flags { -a => "aaa" -b 0 {incr foo} -maybe no -literal yes } #parray flags; puts "--- argv = $argv" proj-assert {"-a {bye bye} -- -b c --" eq $argv} proj-assert {$flags(-a) eq "hi world"} proj-assert {1 == $flags(-b)} proj-assert {"yes" eq $flags(-maybe)} set argv {-f -g -a aaa -M -M -M -L -H -A AAA a b c} set foo 0 set myLambda {{flag val} { proj-assert {$flag in {-f -g -M}} #puts "myLambda flag=$flag val=$val" incr val }} proc myNonLambda {flag val} { proj-assert {$flag in {-A -a}} #puts "myNonLambda flag=$flag val=$val" concat $val $val } proj-parse-flags argv flags { -f 0 -call {apply $myLambda} -g 2 -apply $myLambda -h 3 -apply $myLambda -H 30 33 -a => aAAAa -apply {{f v} { set v }} -A => AaaaA -call myNonLambda -B => 17 -call myNonLambda -M* 0 -apply $myLambda -L "" -literal $myLambda } rename myNonLambda "" #puts "--- argv = $argv"; parray flags proj-assert {$flags(-f) == 1} proj-assert {$flags(-g) == 3} proj-assert {$flags(-h) == 3} proj-assert {$flags(-H) == 33} proj-assert {$flags(-a) == {aaa}} proj-assert {$flags(-A) eq "AAA AAA"} proj-assert {$flags(-B) == 17} proj-assert {$flags(-M) == 3} proj-assert {$flags(-L) eq $myLambda} set argv {-touch -validate} proj-parse-flags argv flags { -touch "" {return "-touch"} -validate 0 1 } #puts "----- argv = $argv"; parray flags proj-assert {$flags(-touch) eq "-touch"} proj-assert {$flags(-validate) == 1} proj-assert {$argv eq {}} set argv {-i -i -i} proj-parse-flags argv flags { -i* 0 incr } proj-assert {3 == $flags(-i)} }} set ::proj__Config(verbose-assert) $__ova unset __ova puts "Done running [info script] self-tests." }; # proj- API self-tests |
Added autosetup/sqlite-config.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 | # This file holds functions for autosetup which are specific to the # sqlite build tree. They are in this file, instead of auto.def, so # that they can be reused in the autoconf sub-tree. This file requires # functions from proj.tcl. if {[string first " " $autosetup(srcdir)] != -1} { user-error "The pathname of the source tree\ may not contain space characters" } if {[string first " " $autosetup(builddir)] != -1} { user-error "The pathname of the build directory\ may not contain space characters" } #parray ::autosetup; exit 0 use proj # # We want the package version info to be emitted early on, but doing # so requires a bit of juggling. We have to [use system] for # --prefix=... to work and to emit the Host/Build system info, but we # don't want those to interfere with --help output. define PACKAGE_VERSION [proj-file-content -trim $::autosetup(srcdir)/VERSION] if {"--help" ni $::argv} { msg-result "Configuring SQLite version [get-define PACKAGE_VERSION]" } use system ; # Will output "Host System" and "Build System" lines if {"--help" ni $::argv} { proj-tweak-default-env-dirs msg-result "Source dir = $::autosetup(srcdir)" msg-result "Build dir = $::autosetup(builddir)" use cc cc-db cc-shared cc-lib pkg-config } # # Object for communicating certain config-time state across various # auto.def-related pieces. array set sqliteConfig [subst [proj-strip-hash-comments { # # Gets set by [sqlite-configure] (the main configure script driver). build-mode unknown # # Gets set to 1 when using jimsh for code generation. May affect # later decisions. use-jim-for-codegen 0 # # Set to 1 when cross-compiling This value may be changed by certain # build options, so it's important that config code which checks for # cross-compilation uses this var instead of # [proj-is-cross-compiling]. is-cross-compiling [proj-is-cross-compiling] # # Pass msg-debug=1 to configure to enable obnoxiously loud output # from [msg-debug]. msg-debug-enabled 0 # # Output file for --dump-defines. Intended only for build debugging # and not part of the public build interface. dump-defines-txt ./config.defines.txt # # If not empty then --dump-defines will dump not only # (dump-defines-txt) but also a JSON file named after this option's # value. dump-defines-json "" # # The list of feature --flags which the --all flag implies. This # requires special handling in a few places. # all-flag-enables {fts4 fts5 rtree geopoly session} # # Default value for the --all flag. Can hypothetically be modified # by non-canonical builds. # all-flag-default 0 }]] ######################################################################## # Processes all configure --flags for this build, run build-specific # config checks, then finalize the configure process. $buildMode must # be one of (canonical, autoconf), and others may be added in the # future. After bootstrapping, $configScript is eval'd in the caller's # scope, then post-configuration finalization is run. $configScript is # intended to hold configure code which is specific to the given # $buildMode, with the caveat that _some_ build-specific code is # encapsulated in the configuration finalization step. # # The intent is that all (or almost all) build-mode-specific # configuration goes inside the $configScript argument to this # function, and that an auto.def file contains only two commands: # # use sqlite-config # sqlite-configure BUILD_NAME { build-specific configure script } # # There are snippets of build-mode-specific decision-making in # [sqlite-configure-finalize] proc sqlite-configure {buildMode configScript} { proj-assert {$::sqliteConfig(build-mode) eq "unknown"} \ "sqlite-configure must not be called more than once" set allBuildModes {canonical autoconf} if {$buildMode ni $allBuildModes} { user-error "Invalid build mode: $buildMode. Expecting one of: $allBuildModes" } if {$::sqliteConfig(all-flag-default)} { set allFlagHelp "Disable these extensions: $::sqliteConfig(all-flag-enables)" } else { set allFlagHelp "Enable these extensions: $::sqliteConfig(all-flag-enables)" } set ::sqliteConfig(build-mode) $buildMode ######################################################################## # A gentle introduction to flags handling in autosetup # # Reference: https://msteveb.github.io/autosetup/developer/ # # All configure flags must be described in an 'options' call. The # general syntax is: # # FLAG => {Help text} # # Where FLAG can have any of the following formats: # # boolopt => "a boolean option which defaults to disabled" # boolopt2=1 => "a boolean option which defaults to enabled" # stringopt: => "an option which takes an argument, e.g. --stringopt=value" # stringopt:DESCR => As for stringopt: with a description for the value # stringopt2:=value => "an option where the argument is optional and defaults to 'value'" # optalias booltopt3 => "a boolean with a hidden alias. --optalias is not shown in --help" # # Autosetup does no small amount of specialized handling for flags, # especially booleans. Each bool-type --FLAG implicitly gets # --enable-FLAG and --disable-FLAG forms. That can lead lead to some # confusion when writing help text. For example: # # options { json=1 {Disable JSON functions} } # # The reason the help text says "disable" is because a boolean option # which defaults to true is, in the --help text, rendered as: # # --disable-json Disable JSON functions # # Whereas a bool flag which defaults to false will instead render as: # # --enable-FLAG # # Non-boolean flags, in contrast, use the names specifically given to # them in the [options] invocation. e.g. "with-tcl" is the --with-tcl # flag. # # Fetching values for flags: # # booleans: use one of: # - [opt-bool FLAG] is autosetup's built-in command for this, but we # have some convenience variants: # - [proj-opt-truthy FLAG] # - [proj-opt-if-truthy FLAG {THEN} {ELSE}] # # Non-boolean (i.e. string) flags: # - [opt-val FLAG ?default?] # - [opt-str ...] - see the docs in ./autosetup/autosetup # # [proj-opt-was-provided] can be used to determine whether a flag was # explicitly provided, which is often useful for distinguishing from # the case of a default value. ######################################################################## set allFlags { # Structure: a list of M {Z} pairs, where M is a descriptive # option group name and Z is a list of X Y pairs. X is a list of # $buildMode name(s) to which the Y flags apply, or {*} to apply # to all builds. Y is a {block} in the form expected by # autosetup's [options] command. Each block which is applicable # to $buildMode is appended to a new list before that list is # passed on to [options]. The order of each Y and sub-Y is # retained, which is significant for rendering of --help. # When writing {help text blocks}, be aware that: # # A) autosetup formats them differently if the {block} starts with # a newline: it starts left-aligned, directly under the --flag, and # the rest of the block is pasted verbatim rather than # pretty-printed. # # B) Vars and commands are NOT expanded, but we use a [subst] call # below which will replace (only) var refs. # Options for how to build the library build-modes { {canonical autoconf} { shared=1 => {Disable build of shared library} static=1 => {Disable build of static library} } {canonical} { amalgamation=1 => {Disable the amalgamation and instead build all files separately} } } # Library-level features and defaults lib-features { {*} { threadsafe=1 => {Disable mutexing} with-tempstore:=no => {Use an in-RAM database for temporary tables: never,no,yes,always} load-extension=1 => {Disable loading of external extensions} # ^^^ one of the downstream custom builds overrides the load-extension default to 0, which # confuses the --help text generator. https://github.com/msteveb/autosetup/issues/77 math=1 => {Disable math functions} json=1 => {Disable JSON functions} memsys5 => {Enable MEMSYS5} memsys3 => {Enable MEMSYS3} fts3 => {Enable the FTS3 extension} fts4 => {Enable the FTS4 extension} fts5 => {Enable the FTS5 extension} update-limit => {Enable the UPDATE/DELETE LIMIT clause} geopoly => {Enable the GEOPOLY extension} rtree => {Enable the RTREE extension} session => {Enable the SESSION extension} all=$::sqliteConfig(all-flag-default) => {$allFlagHelp} largefile=1 => {This legacy flag has no effect on the library but may influence the generated sqlite_cfg.h by adding #define HAVE_LFS} } {canonical} { column-metadata => {Enable the column metadata APIs} # ^^^ Affects how sqlite3.c is generated, so is not available in # the autoconf build. } } # Options for TCL support tcl { {canonical} { tcl=1 => {Disable components which require TCL, including all tests. This tree requires TCL for code generation but can use the in-tree copy of autosetup/jimsh0.c for that. The SQLite TCL extension and the test code require a canonical tclsh.} with-tcl:DIR => {Directory containing tclConfig.sh or a directory one level up from that, from which we can derive a directory containing tclConfig.sh. A dir name of "prefix" is equivalent to the directory specified by the --prefix flag.} with-tclsh:PATH => {Full pathname of tclsh to use. It is used for (A) trying to find tclConfig.sh and (B) all TCL-based code generation. Use --with-tcl unless you have a specific need for this flag. Warning: if its containing dir has multiple tclsh versions, it may select the wrong tclConfig.sh!} static-tclsqlite3=0 => {Statically-link tclsqlite3. This only works if TCL support is enabled and all requisite libraries are available in static form. Note that glibc is unable to fully statically link certain libraries required by tclsqlite3, so this won't work on most Linux environments.} } } # Options for line-editing modes for the CLI shell line-editing { {canonical autoconf} { readline=1 => {Disable readline support} # --with-readline-lib is a backwards-compatible alias for # --with-readline-ldflags with-readline-lib: with-readline-ldflags:=auto => {Readline LDFLAGS, e.g. -lreadline -lncurses} # --with-readline-inc is a backwards-compatible alias for # --with-readline-cflags. with-readline-inc: with-readline-cflags:=auto => {Readline CFLAGS, e.g. -I/path/to/includes} with-readline-header:PATH => {Full path to readline.h, from which --with-readline-cflags will be derived} with-linenoise:DIR => {Source directory for linenoise.c and linenoise.h} editline=0 => {Enable BSD editline support} } } # Options for ICU: International Components for Unicode icu { {*} { with-icu-ldflags:LDFLAGS => {Enable SQLITE_ENABLE_ICU and add the given linker flags for the ICU libraries. e.g. on Ubuntu systems, try '-licui18n -licuuc -licudata'.} with-icu-cflags:CFLAGS => {Apply extra CFLAGS/CPPFLAGS necessary for building with ICU. e.g. -I/usr/local/include} with-icu-config:=auto => {Enable SQLITE_ENABLE_ICU. Value must be one of: auto, pkg-config, /path/to/icu-config} icu-collations=0 => {Enable SQLITE_ENABLE_ICU_COLLATIONS. Requires --with-icu-ldflags=... or --with-icu-config} } } # Options for exotic/alternative build modes alternative-builds { {canonical autoconf} { with-wasi-sdk:=/opt/wasi-sdk => {Top-most dir of the wasi-sdk for a WASI build} } {*} { # Note that --static-cli-shell has a completely different # meaning from --static-shell in the autoconf build! # --[disable-]static-shell is a legacy flag which we can't # remove without breaking downstream builds. static-cli-shell=0 => {Statically-link the sqlite3 CLI shell. This only works if the requisite libraries are all available in static form.} } {canonical} { static-shells=0 => {Shorthand for --static-cli-shell --static-tclsqlite3} with-emsdk:=auto => {Top-most dir of the Emscripten SDK installation. Needed only by ext/wasm. Default=EMSDK env var.} amalgamation-extra-src:FILES => {Space-separated list of soure files to append as-is to the resulting sqlite3.c amalgamation file. May be provided multiple times.} } } # Options primarily for downstream packagers/package maintainers packaging { {autoconf} { # --disable-static-shell: https://sqlite.org/forum/forumpost/cc219ee704 # Note that this has a different meaning from --static-cli-shell in the # canonical build! static-shell=1 => {Link the sqlite3 shell app against the DLL instead of embedding sqlite3.c} } {canonical autoconf} { rpath=1 => {Disable use of the rpath linker flag} # soname: https://sqlite.org/src/forumpost/5a3b44f510df8ded soname:=legacy => {SONAME for libsqlite3.so. "none", or not using this flag, sets no soname. "legacy" sets it to its historical value of libsqlite3.so.0. A value matching the glob "libsqlite3.*" sets it to that literal value. Any other value is assumed to be a suffix which gets applied to "libsqlite3.so.", e.g. --soname=9.10 equates to "libsqlite3.so.9.10".} # dll-basename: https://sqlite.org/forum/forumpost/828fdfe904 dll-basename:=auto => {Specifies the base name of the resulting DLL file. If not provided, "libsqlite3" is usually assumed but on some platforms a platform-dependent default is used. On some platforms this flag gets automatically enabled if it is not provided. Use "default" to explicitly disable platform-dependent activation on such systems.} # out-implib: https://sqlite.org/forum/forumpost/0c7fc097b2 out-implib:=auto => {Enable use of --out-implib linker flag to generate an "import library" for the DLL. The output's base name is specified by this flag's value, with "auto" meaning to figure out a name automatically. On some platforms this flag gets automatically enabled if it is not provided. Use "none" to explicitly disable this feature on such platforms.} } } # Options mostly for sqlite's own development developer { {*} { # Note that using the --debug/--enable-debug flag here # requires patching autosetup/autosetup to rename its builtin # --debug to --autosetup-debug. See details in # autosetup/README.md#patching. with-debug=0 debug=0 => {Enable debug build flags. This option will impact performance by as much as 4x, as it includes large numbers of assert()s in performance-critical loops. Never use --debug for production builds.} scanstatus => {Enable the SQLITE_ENABLE_STMT_SCANSTATUS feature flag} } {canonical} { dev => {Enable dev-mode build: automatically enables certain other flags} test-status => {Enable status of tests} gcov=0 => {Enable coverage testing using gcov} linemacros => {Enable #line macros in the amalgamation} dynlink-tools => {Dynamically link libsqlite3 to certain tools which normally statically embed it} asan-fsanitize:=auto => {Comma- or space-separated list of -fsanitize flags for use with the fuzzcheck-asan tool. Only those which the compiler claims to support will actually be used. May be provided multiple times.} } {*} { dump-defines=0 => {Dump autosetup defines to $::sqliteConfig(dump-defines-txt) (for build debugging)} } } }; # $allFlags set allFlags [proj-strip-hash-comments $allFlags] # ^^^ lappend of [sqlite-custom-flags] introduces weirdness if # we delay [proj-strip-hash-comments] until after that. ######################################################################## # sqlite-custom.tcl is intended only for vendor-branch-specific # customization. See autosetup/README.md#branch-customization for # details. if {[file exists $::autosetup(libdir)/sqlite-custom.tcl]} { uplevel 1 {source $::autosetup(libdir)/sqlite-custom.tcl} } if {[llength [info proc sqlite-custom-flags]] > 0} { # sqlite-custom-flags is assumed to be imported via # autosetup/sqlite-custom.tcl. set scf [sqlite-custom-flags] if {"" ne $scf} { lappend allFlags sqlite-custom-flags $scf } } # Filter allFlags to create the set of [options] legal for this build foreach {group XY} [subst -nobackslashes -nocommands $allFlags] { foreach {X Y} $XY { if { $buildMode in $X || "*" in $X } { options-add $Y } } } #lappend opts "soname:=duplicateEntry => {x}"; #just testing if {[catch {options {}} msg xopts]} { # Workaround for <https://github.com/msteveb/autosetup/issues/73> # where [options] behaves oddly on _some_ TCL builds when it's # called from deeper than the global scope. dict incr xopts -level return {*}$xopts $msg } sqlite-configure-phase1 $buildMode uplevel 1 $configScript sqlite-configure-finalize }; # sqlite-configure ######################################################################## # Runs "phase 1" of the configure process: after initial --flags # handling but before the build-specific parts are run. $buildMode # must be the mode which was passed to [sqlite-configure]. proc sqlite-configure-phase1 {buildMode} { define PACKAGE_NAME sqlite define PACKAGE_URL {https://sqlite.org} define PACKAGE_BUGREPORT [get-define PACKAGE_URL]/forum define PACKAGE_STRING "[get-define PACKAGE_NAME] [get-define PACKAGE_VERSION]" proj-xfer-options-aliases { # Carry values from hidden --flag aliases over to their canonical # flag forms. This list must include only options which are common # to all build modes supported by [sqlite-configure]. with-readline-inc => with-readline-cflags with-readline-lib => with-readline-ldflags with-debug => debug } set ::sqliteConfig(msg-debug-enabled) [proj-val-truthy [get-env msg-debug 0]] proc-debug "msg-debug is enabled" proj-setup-autoreconfig SQLITE_AUTORECONFIG proj-file-extensions if {".exe" eq [get-define TARGET_EXEEXT]} { define SQLITE_OS_UNIX 0 define SQLITE_OS_WIN 1 } else { define SQLITE_OS_UNIX 1 define SQLITE_OS_WIN 0 } sqlite-setup-default-cflags define HAVE_LFS 0 if {[opt-bool largefile]} { # # Insofar as we can determine HAVE_LFS has no effect on the # library. Perhaps it did back in the early 2000's. The # --enable/disable-largefile flag is retained because it's # harmless, but it doesn't do anything useful. It does have # visible side-effects, though: the generated sqlite_cfg.h may (or # may not) define HAVE_LFS. cc-check-lfs } set srcdir $::autosetup(srcdir) proj-dot-ins-append $srcdir/Makefile.in if {[file exists $srcdir/sqlite3.pc.in]} { proj-dot-ins-append $srcdir/sqlite3.pc.in } }; # sqlite-configure-phase1 ######################################################################## # Performs late-stage config steps common to all supported # $::sqliteConfig(build-mode) values. proc sqlite-configure-finalize {} { sqlite-handle-rpath sqlite-handle-soname sqlite-handle-threadsafe sqlite-handle-tempstore sqlite-handle-load-extension sqlite-handle-math sqlite-handle-icu if {[proj-opt-exists readline]} { sqlite-handle-line-editing } if {[proj-opt-exists shared]} { proj-define-for-opt shared ENABLE_LIB_SHARED "Build shared library?" } if {[proj-opt-exists static]} { if {![proj-define-for-opt static ENABLE_LIB_STATIC "Build static library?"]} { # This notice really only applies to the canonical build... proj-indented-notice { NOTICE: static lib build may be implicitly re-activated by other components, e.g. some test apps. } } } sqlite-handle-env-quirks sqlite-handle-common-feature-flags sqlite-finalize-feature-flags sqlite-process-dot-in-files; # do not [define] anything after this sqlite-dump-defines } ######################################################################## # Internal config-time debugging output routine. It generates no # output unless msg-debug=1 is passed to the configure script. proc msg-debug {msg} { if {$::sqliteConfig(msg-debug-enabled)} { puts stderr [proj-bold "** DEBUG: $msg"] } } ######################################################################## # A [msg-debug] proxy which prepends the name of the current proc to # the debug message. It is not legal to call this from the global # scope. proc proc-debug {msg} { msg-debug "\[[proj-scope 1]\]: $msg" } define OPT_FEATURE_FLAGS {} ; # -DSQLITE_OMIT/ENABLE flags. define OPT_SHELL {} ; # Feature-related CFLAGS for the sqlite3 CLI app ######################################################################## # Adds $args, if not empty, to OPT_FEATURE_FLAGS. If the first arg is # -shell then it strips that arg and passes the remaining args the # sqlite-add-shell-opt in addition to adding them to # OPT_FEATURE_FLAGS. This is intended only for holding # -DSQLITE_ENABLE/OMIT/... flags, but that is not enforced here. proc sqlite-add-feature-flag {args} { set shell "" if {"-shell" eq [lindex $args 0]} { set args [lassign $args shell] } if {"" ne $args} { if {"" ne $shell} { sqlite-add-shell-opt {*}$args } define-append OPT_FEATURE_FLAGS {*}$args } } ######################################################################## # Appends $args, if not empty, to OPT_SHELL. proc sqlite-add-shell-opt {args} { if {"" ne $args} { define-append OPT_SHELL {*}$args } } ######################################################################## # Check for log(3) in libm and die with an error if it is not # found. $featureName should be the feature name which requires that # function (it's used only in error messages). defines LDFLAGS_MATH to # the required linker flags (which may be empty even if the math APIs # are found, depending on the OS). proc sqlite-affirm-have-math {featureName} { if {"" eq [get-define LDFLAGS_MATH ""]} { if {![msg-quiet proj-check-function-in-lib log m]} { user-error "Missing math APIs for $featureName" } set lfl [get-define lib_log ""] undefine lib_log if {"" ne $lfl} { user-notice "Forcing requirement of $lfl for $featureName" } define LDFLAGS_MATH $lfl } } ######################################################################## # Run checks for required binaries, like ld and ar. In the canonical # build this must come before [sqlite-handle-wasi-sdk]. proc sqlite-check-common-bins {} { cc-check-tools ld ar ; # must come before [sqlite-handle-wasi-sdk] if {"" eq [proj-bin-define install]} { proj-warn "Cannot find install binary, so 'make install' will not work." define BIN_INSTALL false } } ######################################################################## # Run checks for system-level includes and libs which are common to # both the canonical build and the "autoconf" bundle. # # For the canonical build this must come after # [sqlite-handle-wasi-sdk], as that function may change the # environment in ways which affect this. proc sqlite-check-common-system-deps {} { # Check for needed/wanted data types cc-with {-includes stdint.h} \ {cc-check-types int8_t int16_t int32_t int64_t intptr_t \ uint8_t uint16_t uint32_t uint64_t uintptr_t} # Check for needed/wanted functions cc-check-functions gmtime_r isnan localtime_r localtime_s \ strchrnul usleep utime pread pread64 pwrite pwrite64 apply {{} { set ldrt "" # Collapse funcs from librt into LDFLAGS_RT. # Some systems (ex: SunOS) require -lrt in order to use nanosleep foreach func {fdatasync nanosleep} { if {[proj-check-function-in-lib $func rt]} { set ldrt [get-define lib_${func} ""] undefine lib_${func} if {"" ne $ldrt} { break } } } define LDFLAGS_RT $ldrt }} # Check for needed/wanted headers cc-check-includes \ sys/types.h sys/stat.h dlfcn.h unistd.h \ stdlib.h malloc.h memory.h \ string.h strings.h \ inttypes.h if {[cc-check-includes zlib.h] && [proj-check-function-in-lib deflate z]} { # TODO? port over the more sophisticated zlib search from the fossil auto.def define HAVE_ZLIB 1 define LDFLAGS_ZLIB -lz sqlite-add-shell-opt -DSQLITE_HAVE_ZLIB=1 } else { define HAVE_ZLIB 0 define LDFLAGS_ZLIB "" } } ######################################################################## # Move -DSQLITE_OMIT... and -DSQLITE_ENABLE... flags from CFLAGS and # CPPFLAGS to OPT_FEATURE_FLAGS and remove them from BUILD_CFLAGS. proc sqlite-munge-cflags {} { # Move CFLAGS and CPPFLAGS entries matching -DSQLITE_OMIT* and # -DSQLITE_ENABLE* to OPT_FEATURE_FLAGS. This behavior is derived # from the legacy build and was missing the 3.48.0 release (the # initial Autosetup port). # https://sqlite.org/forum/forumpost/9801e54665afd728 # # Handling of CPPFLAGS, as well as removing ENABLE/OMIT from # CFLAGS/CPPFLAGS, was missing in the 3.49.0 release as well. # # If any configure flags for features are in conflict with # CFLAGS/CPPFLAGS-specified feature flags, all bets are off. There # are no guarantees about which one will take precedence. foreach flagDef {CFLAGS CPPFLAGS} { set tmp "" foreach cf [get-define $flagDef ""] { switch -glob -- $cf { -DSQLITE_OMIT* - -DSQLITE_ENABLE* { sqlite-add-feature-flag $cf } default { lappend tmp $cf } } } define $flagDef $tmp } # Strip all SQLITE_ENABLE/OMIT flags from BUILD_CFLAGS, # for compatibility with the legacy build. set tmp "" foreach cf [get-define BUILD_CFLAGS ""] { switch -glob -- $cf { -DSQLITE_OMIT* - -DSQLITE_ENABLE* {} default { lappend tmp $cf } } } define BUILD_CFLAGS $tmp } ######################################################################### # Set up the default CFLAGS and BUILD_CFLAGS values. proc sqlite-setup-default-cflags {} { ######################################################################## # We differentiate between two C compilers: the one used for binaries # which are to run on the build system (in autosetup it's called # CC_FOR_BUILD and in Makefile.in it's $(B.cc)) and the one used for # compiling binaries for the target system (CC a.k.a. $(T.cc)). # Normally they're the same, but they will differ when # cross-compiling. # # When cross-compiling we default to not using the -g flag, based on a # /chat discussion prompted by # https://sqlite.org/forum/forumpost/9a67df63eda9925c set defaultCFlags {-O2} if {!$::sqliteConfig(is-cross-compiling)} { lappend defaultCFlags -g } define CFLAGS [proj-get-env CFLAGS $defaultCFlags] # BUILD_CFLAGS is the CFLAGS for CC_FOR_BUILD. define BUILD_CFLAGS [proj-get-env BUILD_CFLAGS {-g}] sqlite-munge-cflags } ######################################################################## # Handle various SQLITE_ENABLE/OMIT_... feature flags. proc sqlite-handle-common-feature-flags {} { msg-result "Feature flags..." if {![opt-bool all]} { # Special handling for --disable-all foreach flag $::sqliteConfig(all-flag-enables) { if {![proj-opt-was-provided $flag]} { proj-opt-set $flag 0 } } } foreach {boolFlag featureFlag ifSetEvalThis} [proj-strip-hash-comments { all {} { # The 'all' option must be first in this list. This impl makes # an effort to only apply flags which the user did not already # apply, so that combinations like (--all --disable-geopoly) # will indeed disable geopoly. There are corner cases where # flags which depend on each other will behave in non-intuitive # ways: # # --all --disable-rtree # # Will NOT disable geopoly, though geopoly depends on rtree. # The --geopoly flag, though, will automatically re-enable # --rtree, so --disable-rtree won't actually disable anything in # that case. foreach k $::sqliteConfig(all-flag-enables) { if {![proj-opt-was-provided $k]} { proj-opt-set $k 1 } } } fts3 -DSQLITE_ENABLE_FTS3 {sqlite-affirm-have-math fts3} fts4 -DSQLITE_ENABLE_FTS4 {sqlite-affirm-have-math fts4} fts5 -DSQLITE_ENABLE_FTS5 {sqlite-affirm-have-math fts5} geopoly -DSQLITE_ENABLE_GEOPOLY {proj-opt-set rtree} rtree -DSQLITE_ENABLE_RTREE {} session {-DSQLITE_ENABLE_SESSION -DSQLITE_ENABLE_PREUPDATE_HOOK} {} update-limit -DSQLITE_ENABLE_UPDATE_DELETE_LIMIT {} memsys5 -DSQLITE_ENABLE_MEMSYS5 {} memsys3 {} { if {[opt-bool memsys5]} { proj-warn "not enabling memsys3 because memsys5 is enabled." expr 0 } else { sqlite-add-feature-flag -DSQLITE_ENABLE_MEMSYS3 } } scanstatus -DSQLITE_ENABLE_STMT_SCANSTATUS {} column-metadata -DSQLITE_ENABLE_COLUMN_METADATA {} }] { if {$boolFlag ni $::autosetup(options)} { # Skip flags which are in the canonical build but not # the autoconf bundle. continue } proj-if-opt-truthy $boolFlag { sqlite-add-feature-flag $featureFlag if {0 != [eval $ifSetEvalThis] && "all" ne $boolFlag} { msg-result " + $boolFlag" } } { if {"all" ne $boolFlag} { msg-result " - $boolFlag" } } } ######################################################################## # Invert the above loop's logic for some SQLITE_OMIT_... cases. If # config option $boolFlag is false, [sqlite-add-feature-flag # $featureFlag], where $featureFlag is intended to be # -DSQLITE_OMIT_... foreach {boolFlag featureFlag} { json -DSQLITE_OMIT_JSON } { if {[proj-opt-truthy $boolFlag]} { msg-result " + $boolFlag" } else { sqlite-add-feature-flag $featureFlag msg-result " - $boolFlag" } } } ######################################################################### # Remove duplicates from the final feature flag sets and show them to # the user. proc sqlite-finalize-feature-flags {} { set oFF [get-define OPT_FEATURE_FLAGS] if {"" ne $oFF} { define OPT_FEATURE_FLAGS [lsort -unique $oFF] msg-result "Library feature flags: [get-define OPT_FEATURE_FLAGS]" } set oFF [get-define OPT_SHELL] if {"" ne $oFF} { define OPT_SHELL [lsort -unique $oFF] msg-result "Shell options: [get-define OPT_SHELL]" } if {"" ne [set extraSrc [get-define AMALGAMATION_EXTRA_SRC ""]]} { proj-assert {"canonical" eq $::sqliteConfig(build-mode)} msg-result "Appending source files to amalgamation: $extraSrc" } if {[lsearch [get-define TARGET_DEBUG ""] -DSQLITE_DEBUG=1] > -1} { msg-result "Note: this is a debug build, so performance will suffer." } } ######################################################################## # Checks for the --debug flag and [define]s TARGET_DEBUG based on # that. TARGET_DEBUG is unused in the autoconf build but that is # arguably a bug. proc sqlite-handle-debug {} { msg-checking "SQLITE_DEBUG build? " proj-if-opt-truthy debug { define TARGET_DEBUG {-g -DSQLITE_DEBUG=1 -O0 -Wall} sqlite-add-feature-flag -DSQLITE_ENABLE_SELECTTRACE -DSQLITE_ENABLE_WHERETRACE proj-opt-set memsys5 msg-result yes } { define TARGET_DEBUG {-DNDEBUG} msg-result no } } ######################################################################## # "soname" for libsqlite3.so. See discussion at: # https://sqlite.org/src/forumpost/5a3b44f510df8ded proc sqlite-handle-soname {} { define LDFLAGS_LIBSQLITE3_SONAME "" if {[proj-opt-was-provided soname]} { set soname [join [opt-val soname] ""] } else { # Enabling soname breaks linking for the --dynlink-tools feature, # and this project has no direct use for soname, so default to # none. Package maintainers, on the other hand, like to have an # soname. set soname none } switch -exact -- $soname { none - "" { return 0 } legacy { set soname libsqlite3.so.0 } default { if {[string match libsqlite3.* $soname]} { # use it as-is } else { # Assume it's a suffix set soname "libsqlite3.so.${soname}" } } } proc-debug "soname=$soname" if {[proj-check-soname $soname]} { define LDFLAGS_LIBSQLITE3_SONAME [get-define LDFLAGS_SONAME_PREFIX]$soname msg-result "Setting SONAME using: [get-define LDFLAGS_LIBSQLITE3_SONAME]" } elseif {[proj-opt-was-provided soname]} { # --soname was explicitly requested but not available, so fail fatally proj-fatal "This environment does not support SONAME." } else { # --soname was not explicitly requested but not available, so just warn msg-result "This environment does not support SONAME." } } ######################################################################## # If --enable-threadsafe is set, this adds -DSQLITE_THREADSAFE=1 to # OPT_FEATURE_FLAGS and sets LDFLAGS_PTHREAD to the linker flags # needed for linking pthread (possibly an empty string). If # --enable-threadsafe is not set, adds -DSQLITE_THREADSAFE=0 to # OPT_FEATURE_FLAGS and sets LDFLAGS_PTHREAD to an empty string. proc sqlite-handle-threadsafe {} { msg-checking "Support threadsafe operation? " define LDFLAGS_PTHREAD "" set enable 0 proj-if-opt-truthy threadsafe { msg-result "Checking for libs..." if {[proj-check-function-in-lib pthread_create pthread] && [proj-check-function-in-lib pthread_mutexattr_init pthread]} { set enable 1 define LDFLAGS_PTHREAD [get-define lib_pthread_create] undefine lib_pthread_create undefine lib_pthread_mutexattr_init } elseif {[proj-opt-was-provided threadsafe]} { user-error "Missing required pthread libraries. Use --disable-threadsafe to disable this check." } else { msg-result "pthread support not detected" } # Recall that LDFLAGS_PTHREAD might be empty even if pthreads if # found because it's in -lc on some platforms. } { msg-result "Disabled using --disable-threadsafe" } sqlite-add-feature-flag -DSQLITE_THREADSAFE=${enable} return $enable } ######################################################################## # Handles the --with-tempstore flag. # # The test fixture likes to set SQLITE_TEMP_STORE on its own, so do # not set that feature flag unless it was explicitly provided to the # configure script. proc sqlite-handle-tempstore {} { if {[proj-opt-was-provided with-tempstore]} { set ts [opt-val with-tempstore no] set tsn 1 msg-checking "Use an in-RAM database for temporary tables? " switch -exact -- $ts { never { set tsn 0 } no { set tsn 1 } yes { set tsn 2 } always { set tsn 3 } default { user-error "Invalid --with-tempstore value '$ts'. Use one of: never, no, yes, always" } } msg-result $ts sqlite-add-feature-flag -DSQLITE_TEMP_STORE=$tsn } } ######################################################################## # Check for the Emscripten SDK for building the web-based wasm # components. The core lib and tools do not require this but ext/wasm # does. Most of the work is done via [proj-check-emsdk], then this # function adds the following defines: # # - EMCC_WRAPPER = "" or top-srcdir/tool/emcc.sh # - BIN_WASM_OPT = "" or path to wasm-opt # - BIN_WASM_STRIP = "" or path to wasm-strip # # Noting that: # # 1) Not finding the SDK is not fatal at this level, nor is failure to # find one of the related binaries. # # 2) wasm-strip is part of the wabt package: # # https://github.com/WebAssembly/wabt # # and this project requires it for production-mode builds but not dev # builds. # proc sqlite-handle-emsdk {} { define EMCC_WRAPPER "" define BIN_WASM_STRIP "" define BIN_WASM_OPT "" set srcdir $::autosetup(srcdir) if {$srcdir ne $::autosetup(builddir)} { # The EMSDK pieces require writing to the original source tree # even when doing an out-of-tree build. The ext/wasm pieces do not # support an out-of-tree build so we treat that case as if EMSDK # were not found. msg-result "Out-of tree build: not checking for EMSDK." return } set emccSh $srcdir/tool/emcc.sh set extWasmConfig $srcdir/ext/wasm/config.make if {![get-define HAVE_WASI_SDK] && [proj-check-emsdk]} { define EMCC_WRAPPER $emccSh set emsdkHome [get-define EMSDK_HOME ""] proj-assert {"" ne $emsdkHome} #define EMCC_WRAPPER ""; # just for testing proj-bin-define wasm-strip proj-bin-define bash; # ext/wasm/GNUmakefile requires bash if {[file-isexec $emsdkHome/upstream/bin/wasm-opt]} { define BIN_WASM_OPT $emsdkHome/upstream/bin/wasm-opt } else { # Maybe there's a copy in the path? proj-bin-define wasm-opt BIN_WASM_OPT } proj-dot-ins-append $emccSh.in $emccSh { catch {exec chmod u+x $dotInsOut} } proj-dot-ins-append $extWasmConfig.in $extWasmConfig } else { define EMCC_WRAPPER "" file delete -force -- $emccSh $extWasmConfig } } ######################################################################## # Internal helper for [sqlite-check-line-editing]. Returns a list of # potential locations under which readline.h might be found. # # On some environments this function may perform extra work to help # sqlite-check-line-editing figure out how to find libreadline and # friends. It will communicate those results via means other than the # result value, e.g. by modifying configure --flags. proc sqlite-get-readline-dir-list {} { # Historical note: the dirs list, except for the inclusion of # $prefix and some platform-specific dirs, originates from the # legacy configure script set dirs [list [get-define prefix]] switch -glob -- [get-define host] { *-linux-android { # Possibly termux lappend dirs /data/data/com.termux/files/usr } *-mingw32 { lappend dirs /mingw32 /mingw } *-mingw64 { lappend dirs /mingw64 /mingw } *-haiku { lappend dirs /boot/system/develop/headers if {[opt-val with-readline-ldflags] in {auto ""}} { # If the user did not supply their own --with-readline-ldflags # value, hijack that flag to inject options which are known to # work on a default Haiku installation. if {"" ne [glob -nocomplain /boot/system/lib/libreadline*]} { proj-opt-set with-readline-ldflags {-L/boot/system/lib -lreadline} } } } } lappend dirs /usr /usr/local /usr/local/readline /usr/contrib set rv {} foreach d $dirs { if {[file isdir $d]} {lappend rv $d} } #proc-debug "dirs=$rv" return $rv } ######################################################################## # sqlite-check-line-editing jumps through proverbial hoops to try to # find a working line-editing library, setting: # # - HAVE_READLINE to 0 or 1 # - HAVE_LINENOISE to 0, 1, or 2 # - HAVE_EDITLINE to 0 or 1 # # Only one of ^^^ those will be set to non-0. # # - LDFLAGS_READLINE = linker flags or empty string # # - CFLAGS_READLINE = compilation flags for clients or empty string. # # Note that LDFLAGS_READLINE and CFLAGS_READLINE may refer to # linenoise or editline, not necessarily libreadline. In some cases # it will set HAVE_READLINE=1 when it's really using editline, for # reasons described in this function's comments. # # Returns a string describing which line-editing approach to use, or # "none" if no option is available. # # Order of checks: # # 1) --with-linenoise trumps all others and skips all of the # complexities involved with the remaining options. # # 2) --editline trumps --readline # # 3) --disable-readline trumps --readline # # 4) Default to automatic search for optional readline # # 5) Try to find readline or editline. If it's not found AND the # corresponding --FEATURE flag was explicitly given, fail fatally, # else fail silently. proc sqlite-check-line-editing {} { msg-result "Checking for line-editing capability..." define HAVE_READLINE 0 define HAVE_LINENOISE 0 define HAVE_EDITLINE 0 define LDFLAGS_READLINE "" define CFLAGS_READLINE "" set failIfNotFound 0 ; # Gets set to 1 for explicit --FEATURE requests # so that we know whether to fail fatally or not # if the library is not found. set libsForReadline {readline edit} ; # -l<LIB> names to check for readline(). # The libedit check changes this. set editLibName "readline" ; # "readline" or "editline" set editLibDef "HAVE_READLINE" ; # "HAVE_READLINE" or "HAVE_EDITLINE" set dirLn [opt-val with-linenoise] if {"" ne $dirLn} { # Use linenoise from a copy of its sources (not a library)... if {![file isdir $dirLn]} { proj-fatal "--with-linenoise value is not a directory" } set lnH $dirLn/linenoise.h if {![file exists $lnH] } { proj-fatal "Cannot find linenoise.h in $dirLn" } set lnC "" set lnCOpts {linenoise-ship.c linenoise.c} foreach f $lnCOpts { if {[file exists $dirLn/$f]} { set lnC $dirLn/$f break; } } if {"" eq $lnC} { proj-fatal "Cannot find any of $lnCOpts in $dirLn" } set flavor "" set lnVal [proj-which-linenoise $lnH] switch -- $lnVal { 1 { set flavor "antirez" } 2 { set flavor "msteveb" } default { proj-fatal "Cannot determine the flavor of linenoise from $lnH" } } define CFLAGS_READLINE "-I$dirLn $lnC" define HAVE_LINENOISE $lnVal sqlite-add-shell-opt -DHAVE_LINENOISE=$lnVal if {$::sqliteConfig(use-jim-for-codegen) && 2 == $lnVal} { define-append CFLAGS_JIMSH -DUSE_LINENOISE [get-define CFLAGS_READLINE] user-notice "Adding linenoise support to jimsh." } return "linenoise ($flavor)" } elseif {[opt-bool editline]} { # libedit mimics libreadline and on some systems does not have its # own header installed (instead, that of libreadline is used). # # shell.c historically expects HAVE_EDITLINE to be set for # libedit, but it then expects to see <editline/readline.h>, which # some system's don't actually have despite having libedit. If we # end up finding <editline/readline.h> below, we will use # -DHAVE_EDITLINE=1, else we will use -DHAVE_READLINE=1. In either # case, we will link against libedit. set failIfNotFound 1 set libsForReadline {edit} set editLibName editline } elseif {![opt-bool readline]} { msg-result "Readline support explicitly disabled with --disable-readline" return "none" } elseif {[proj-opt-was-provided readline]} { # If an explicit --[enable-]readline was used, fail if it's not # found, else treat the feature as optional. set failIfNotFound 1 } # Transform with-readline-header=X to with-readline-cflags=-I... set v [opt-val with-readline-header] proj-opt-set with-readline-header "" if {"" ne $v} { if {"auto" eq $v} { proj-opt-set with-readline-cflags auto } else { set v [file dirname $v] if {[string match */readline $v]} { # Special case: if the path includes .../readline/readline.h, # set the -I to one dir up from that because our sources # #include <readline/readline.h> or <editline/readline.h>. set v [file dirname $v] } proj-opt-set with-readline-cflags "-I$v" } } # Look for readline.h set rlInc [opt-val with-readline-cflags auto] if {"auto" eq $rlInc} { set rlInc "" if {$::sqliteConfig(is-cross-compiling)} { # ^^^ this check is derived from the legacy configure script. proj-warn "Skipping check for readline.h because we're cross-compiling." } else { set dirs [sqlite-get-readline-dir-list] set subdirs [list \ include/$editLibName \ readline] if {"editline" eq $editLibName} { lappend subdirs include/readline # ^^^ editline, on some systems, does not have its own header, # and uses libreadline's header. } lappend subdirs include set rlInc [proj-search-for-header-dir readline.h \ -dirs $dirs -subdirs $subdirs] #proc-debug "rlInc=$rlInc" if {"" ne $rlInc} { if {[string match */readline $rlInc]} { set rlInc [file dirname $rlInc]; # CLI shell: #include <readline/readline.h> } elseif {[string match */editline $rlInc]} { set editLibDef HAVE_EDITLINE set rlInc [file dirname $rlInc]; # CLI shell: #include <editline/readline.h> } set rlInc "-I${rlInc}" } } } elseif {"" ne $rlInc && ![string match *-I* $rlInc]} { proj-fatal "Argument to --with-readline-cflags is intended to be CFLAGS and contain -I..." } # If readline.h was found/specified, look for lib(readline|edit)... # # This is not quite straightforward because both libreadline and # libedit typically require some other library which (according to # legacy autotools-generated tests) provides tgetent(3). On some # systems that's built into libreadline/edit, on some (most?) its in # lib[n]curses, and on some it's in libtermcap. set rlLib "" if {"" ne $rlInc} { set rlLib [opt-val with-readline-ldflags] #proc-debug "rlLib=$rlLib" if {$rlLib in {auto ""}} { set rlLib "" set libTerm "" if {[proj-check-function-in-lib tgetent "$editLibName ncurses curses termcap"]} { # ^^^ that libs list comes from the legacy configure script ^^^ set libTerm [get-define lib_tgetent] undefine lib_tgetent } if {$editLibName eq $libTerm} { set rlLib $libTerm } elseif {[proj-check-function-in-lib readline $libsForReadline $libTerm]} { set rlLib [get-define lib_readline] lappend rlLib $libTerm undefine lib_readline } } } # If we found a library, configure the build to use it... if {"" ne $rlLib} { if {"editline" eq $editLibName && "HAVE_READLINE" eq $editLibDef} { # Alert the user that, despite outward appearances, we won't be # linking to the GPL'd libreadline. Presumably that distinction is # significant for those using --editline. proj-indented-notice { NOTE: the local libedit uses <readline/readline.h> so we will compile with -DHAVE_READLINE=1 but will link with libedit. } } set rlLib [join $rlLib] set rlInc [join $rlInc] define LDFLAGS_READLINE $rlLib define CFLAGS_READLINE $rlInc proj-assert {$editLibDef in {HAVE_READLINE HAVE_EDITLINE}} proj-assert {$editLibName in {readline editline}} sqlite-add-shell-opt -D${editLibDef}=1 msg-result "Using $editLibName flags: $rlInc $rlLib" # Check whether rl_completion_matches() has a signature we can use # and disable that sub-feature if it doesn't. if {![cctest \ -cflags "$rlInc -D${editLibDef}" -libs $rlLib -nooutput 1 -source { #include <stdio.h> #ifdef HAVE_EDITLINE #include <editline/readline.h> #else #include <readline/readline.h> #endif static char * rcg(const char *z, int i){(void)z; (void)i; return 0;} int main(void) { char ** x = rl_completion_matches("one", rcg); (void)x; return 0; } }]} { proj-warn "readline-style completion disabled due to rl_completion_matches() signature mismatch" sqlite-add-shell-opt -DSQLITE_OMIT_READLINE_COMPLETION } return $editLibName } if {$failIfNotFound} { proj-fatal "Explicit --$editLibName failed to find a matching library." } return "none" }; # sqlite-check-line-editing ######################################################################## # Runs sqlite-check-line-editing and adds a message around it. In the # canonical build this must not be called before # sqlite-determine-codegen-tcl for reasons now lost to history (and # might not still be applicable). proc sqlite-handle-line-editing {} { msg-result "Line-editing support for the sqlite3 shell: [sqlite-check-line-editing]" } ######################################################################## # ICU - International Components for Unicode # # Handles these flags: # # --with-icu-ldflags=LDFLAGS # --with-icu-cflags=CFLAGS # --with-icu-config[=auto | pkg-config | /path/to/icu-config] # --enable-icu-collations # # --with-icu-config values: # # - auto: use the first one of (pkg-config, icu-config) found on the # system. # - pkg-config: use only pkg-config to determine flags # - /path/to/icu-config: use that to determine flags # # If --with-icu-config is used as neither pkg-config nor icu-config # are found, fail fatally. # # If both --with-icu-ldflags and --with-icu-config are provided, they # are cumulative. If neither are provided, icu-collations is not # honored and a warning is emitted if it is provided. # # Design note: though we could automatically enable ICU if the # icu-config binary or (pkg-config icu-io) are found, we specifically # do not. ICU is always an opt-in feature. proc sqlite-handle-icu {} { define LDFLAGS_ICU [join [opt-val with-icu-ldflags ""]] define CFLAGS_ICU [join [opt-val with-icu-cflags ""]] if {[proj-opt-was-provided with-icu-config]} { msg-result "Checking for ICU support..." set icuConfigBin [opt-val with-icu-config] set tryIcuConfigBin 1; # set to 0 if we end up using pkg-config if {$icuConfigBin in {auto pkg-config}} { if {[pkg-config-init 0] && [pkg-config icu-io]} { # Maintenance reminder: historical docs say to use both of # (icu-io, icu-uc). icu-uc lacks a required lib and icu-io has # all of them on tested OSes. set tryIcuConfigBin 0 define LDFLAGS_ICU [get-define PKG_ICU_IO_LDFLAGS] define-append LDFLAGS_ICU [get-define PKG_ICU_IO_LIBS] define CFLAGS_ICU [get-define PKG_ICU_IO_CFLAGS] } elseif {"pkg-config" eq $icuConfigBin} { proj-fatal "pkg-config cannot find package icu-io" } else { proj-assert {"auto" eq $icuConfigBin} } } if {$tryIcuConfigBin} { if {"auto" eq $icuConfigBin} { set icuConfigBin [proj-first-bin-of \ /usr/local/bin/icu-config \ /usr/bin/icu-config] if {"" eq $icuConfigBin} { proj-indented-notice -error { --with-icu-config=auto cannot find (pkg-config icu-io) or icu-config binary. On Ubuntu-like systems try: --with-icu-ldflags='-licui18n -licuuc -licudata' } } } if {[file-isexec $icuConfigBin]} { set x [exec $icuConfigBin --ldflags] if {"" eq $x} { proj-indented-notice -error \ [subst { $icuConfigBin --ldflags returned no data. On Ubuntu-like systems try: --with-icu-ldflags='-licui18n -licuuc -licudata' }] } define-append LDFLAGS_ICU $x set x [exec $icuConfigBin --cppflags] define-append CFLAGS_ICU $x } else { proj-fatal "--with-icu-config=$icuConfigBin does not refer to an executable" } } } set ldflags [define LDFLAGS_ICU [string trim [get-define LDFLAGS_ICU]]] set cflags [define CFLAGS_ICU [string trim [get-define CFLAGS_ICU]]] if {"" ne $ldflags} { sqlite-add-feature-flag -shell -DSQLITE_ENABLE_ICU msg-result "Enabling ICU support with flags: $ldflags $cflags" if {[opt-bool icu-collations]} { msg-result "Enabling ICU collations." sqlite-add-feature-flag -shell -DSQLITE_ENABLE_ICU_COLLATIONS # Recall that shell.c builds with sqlite3.c except in the case # of --disable-static-shell, a combination we do not # specifically attempt to account for. } } elseif {[opt-bool icu-collations]} { proj-warn "ignoring --enable-icu-collations because neither --with-icu-ldflags nor --with-icu-config provided any linker flags" } else { msg-result "ICU support is disabled." } }; # sqlite-handle-icu ######################################################################## # Handles the --enable-load-extension flag. Returns 1 if the support # is enabled, else 0. If support for that feature is not found, a # fatal error is triggered if --enable-load-extension is explicitly # provided, else a loud warning is instead emitted. If # --disable-load-extension is used, no check is performed. # # Makes the following environment changes: # # - defines LDFLAGS_DLOPEN to any linker flags needed for this # feature. It may legally be empty on some systems where dlopen() # is in libc. # # - If the feature is not available, adds # -DSQLITE_OMIT_LOAD_EXTENSION=1 to the feature flags list. proc sqlite-handle-load-extension {} { define LDFLAGS_DLOPEN "" set found 0 proj-if-opt-truthy load-extension { set found [proj-check-function-in-lib dlopen dl] if {$found} { define LDFLAGS_DLOPEN [get-define lib_dlopen] undefine lib_dlopen } else { if {[proj-opt-was-provided load-extension]} { # Explicit --enable-load-extension: fail if not found proj-indented-notice -error { --enable-load-extension was provided but dlopen() not found. Use --disable-load-extension to bypass this check. } } else { # It was implicitly enabled: warn if not found proj-indented-notice { WARNING: dlopen() not found, so loadable module support will be disabled. Use --disable-load-extension to bypass this check. } } } } if {$found} { msg-result "Loadable extension support enabled." } else { msg-result "Disabling loadable extension support. Use --enable-load-extension to enable them." sqlite-add-feature-flag -DSQLITE_OMIT_LOAD_EXTENSION=1 } return $found } ######################################################################## # Handles the --enable-math flag. proc sqlite-handle-math {} { proj-if-opt-truthy math { if {![proj-check-function-in-lib ceil m]} { user-error "Cannot find libm functions. Use --disable-math to bypass this." } define LDFLAGS_MATH [get-define lib_ceil] undefine lib_ceil sqlite-add-feature-flag -DSQLITE_ENABLE_MATH_FUNCTIONS msg-result "Enabling math SQL functions" } { define LDFLAGS_MATH "" msg-result "Disabling math SQL functions" } } ######################################################################## # If this OS looks like a Mac, checks for the Mac-specific # -current_version and -compatibility_version linker flags. Defines # LDFLAGS_MAC_CVERSION to an empty string and returns 0 if they're not # supported, else defines that to the linker flags and returns 1. # # We don't check this on non-Macs because this whole thing is a # libtool compatibility kludge to account for a version stamp which # libtool applied only on Mac platforms. # # Based on https://sqlite.org/forum/forumpost/9dfd5b8fd525a5d7. proc sqlite-handle-mac-cversion {} { define LDFLAGS_MAC_CVERSION "" set rc 0 if {[proj-looks-like-mac]} { cc-with {-link 1} { # These version numbers are historical libtool-defined values, not # library-defined ones if {[cc-check-flags "-Wl,-current_version,9.6.0"] && [cc-check-flags "-Wl,-compatibility_version,9.0.0"]} { define LDFLAGS_MAC_CVERSION "-Wl,-compatibility_version,9.0.0 -Wl,-current_version,9.6.0" set rc 1 } elseif {[cc-check-flags "-compatibility_version 9.0.0"] && [cc-check-flags "-current_version 9.6.0"]} { define LDFLAGS_MAC_CVERSION "-compatibility_version 9.0.0 -current_version 9.6.0" set rc 1 } } } return $rc } ######################################################################## # If this is a Mac platform, check for support for # -Wl,-install_name,... and, if it's available, define # LDFLAGS_MAC_INSTALL_NAME to a variant of that string which is # intended to expand at make-time, else set LDFLAGS_MAC_INSTALL_NAME # to an empty string. # # https://sqlite.org/forum/forumpost/5651662b8875ec0a proc sqlite-handle-mac-install-name {} { define LDFLAGS_MAC_INSTALL_NAME ""; # {-Wl,-install_name,"$(install-dir.lib)/$(libsqlite3.DLL)"} set rc 0 if {[proj-looks-like-mac]} { cc-with {-link 1} { if {[cc-check-flags "-Wl,-install_name,/usr/local/lib/libsqlite3.dylib"]} { define LDFLAGS_MAC_INSTALL_NAME {-Wl,-install_name,"$(install-dir.lib)/$(libsqlite3.DLL)"} set rc 1 } } } return $rc } ######################################################################## # Handles the --dll-basename configure flag. [define]'s # SQLITE_DLL_BASENAME to the DLL's preferred base name (minus # extension). If --dll-basename is not provided (or programmatically # set - see [sqlite-handle-env-quirks]) then this is always # "libsqlite3", otherwise it may use a different value based on the # value of [get-define host]. proc sqlite-handle-dll-basename {} { if {[proj-opt-was-provided dll-basename]} { set dn [join [opt-val dll-basename] ""] if {$dn in {none default}} { set dn libsqlite3 } } else { set dn libsqlite3 } if {$dn in {auto ""}} { switch -glob -- [get-define host] { *-*-cygwin { set dn cygsqlite3-0 } *-*-ming* { set dn libsqlite3-0 } *-*-msys { set dn msys-sqlite3-0 } default { set dn libsqlite3 } } } define SQLITE_DLL_BASENAME $dn } ######################################################################## # [define]s LDFLAGS_OUT_IMPLIB to either an empty string or to a # -Wl,... flag for the platform-specific --out-implib flag, which is # used for building an "import library .dll.a" file on some platforms # (e.g. msys2, mingw). SQLITE_OUT_IMPLIB is defined to the name of the # import lib or an empty string. Returns 1 if supported, else 0. # # The name of the import library is [define]d in SQLITE_OUT_IMPLIB. # # If the configure flag --out-implib is not used (or programmatically # set) then this simply sets the above-listed defines to empty strings # (but see [sqlite-handle-env-quirks]). If that flag is used but the # capability is not available, a fatal error is triggered. # # This feature is specifically opt-in because it's supported on far # more platforms than actually need it and enabling it causes creation # of libsqlite3.so.a files which are unnecessary in most environments. # # Added in response to: https://sqlite.org/forum/forumpost/0c7fc097b2 # # Platform notes: # # - cygwin sqlite packages historically install no .dll.a file. # # - msys2 and mingw sqlite packages historically install # /usr/lib/libsqlite3.dll.a despite the DLL being in # /usr/bin. proc sqlite-handle-out-implib {} { define LDFLAGS_OUT_IMPLIB "" define SQLITE_OUT_IMPLIB "" set rc 0 if {[proj-opt-was-provided out-implib]} { set olBaseName [join [opt-val out-implib] ""] if {$olBaseName in {auto ""}} { set olBaseName "libsqlite3" ;# [get-define SQLITE_DLL_BASENAME] # Based on discussions with mingw/msys users, the import lib # should always be called libsqlite3.dll.a even on platforms # which rename libsqlite3.dll to something else. } if {$olBaseName ne "none"} { cc-with {-link 1} { set dll "${olBaseName}[get-define TARGET_DLLEXT]" set flags [proj-cc-check-Wl-flag --out-implib ${dll}.a] if {"" ne $flags} { define LDFLAGS_OUT_IMPLIB $flags define SQLITE_OUT_IMPLIB ${dll}.a set rc 1 } } if {!$rc} { user-error "--out-implib is not supported on this platform" } } } return $rc } ######################################################################## # If the given platform identifier (defaulting to [get-define host]) # appears to be one of the Unix-on-Windows environments, returns a # brief symbolic name for that environment, else returns an empty # string. # # It does not distinguish between msys and msys2, returning msys for # both. The build does not, as of this writing, specifically support # msys v1. Similarly, this function returns "mingw" for both "mingw32" # and "mingw64". proc sqlite-env-is-unix-on-windows {{envTuple ""}} { if {"" eq $envTuple} { set envTuple [get-define host] } set name "" switch -glob -- $envTuple { *-*-cygwin { set name cygwin } *-*-ming* { set name mingw } *-*-msys { set name msys } } return $name } ######################################################################## # Performs various tweaks to the build which are only relevant on # certain platforms, e.g. Mac and "Unix on Windows" platforms (msys2, # cygwin, ...). # # 1) DLL installation: # # [define]s SQLITE_DLL_INSTALL_RULES to a symbolic name suffix for a # set of "make install" rules to use for installation of the DLL # deliverable. The makefile is tasked with providing rules named # install-dll-NAME which runs the installation for that set, as well # as providing a rule named install-dll which resolves to # install-dll-NAME (perhaps indirectly, depending on whether the DLL # is (de)activated). # # The default value is "unix-generic". # # 2) --out-implib: # # On platforms where an "import library" is conventionally used but # --out-implib was not explicitly used, automatically add that flag. # This conventionally applies only to the "Unix on Windows" # environments like msys and cygwin. # # 3) --dll-basename: # # On the same platforms addressed by --out-implib, if --dll-basename # is not explicitly specified, --dll-basename=auto is implied. proc sqlite-handle-env-quirks {} { set instName unix-generic; # name of installation rules set set autoDll 0; # true if --out-implib/--dll-basename should be implied set host [get-define host] switch -glob -- $host { *apple* - *darwin* { set instName darwin } default { set x [sqlite-env-is-unix-on-windows $host] if {"" ne $x} { set instName $x set autoDll 1 } } } define SQLITE_DLL_INSTALL_RULES $instName if {$autoDll} { if {![proj-opt-was-provided out-implib]} { # Imply --out-implib=auto proj-indented-notice [subst -nocommands -nobackslashes { NOTICE: auto-enabling --out-implib for environment [$host]. Use --out-implib=none to disable this special case or --out-implib=auto to squelch this notice. }] proj-opt-set out-implib auto } if {![proj-opt-was-provided dll-basename]} { # Imply --dll-basename=auto proj-indented-notice [subst -nocommands -nobackslashes { NOTICE: auto-enabling --dll-basename for environment [$host]. Use --dll-basename=default to disable this special case or --dll-basename=auto to squelch this notice. }] proj-opt-set dll-basename auto } } sqlite-handle-dll-basename sqlite-handle-out-implib sqlite-handle-mac-cversion sqlite-handle-mac-install-name if {[llength [info proc sqlite-custom-handle-flags]] > 0} { # sqlite-custom-handle-flags is assumed to be imported via a # client-specific import: autosetup/sqlite-custom.tcl. sqlite-custom-handle-flags } } ######################################################################## # Perform some late-stage work and generate the configure-process # output file(s). proc sqlite-process-dot-in-files {} { ######################################################################## # "Re-export" the autoconf-conventional --XYZdir flags into something # which is more easily overridable from a make invocation. See the docs # for [proj-remap-autoconf-dir-vars] for the explanation of why. # # We do this late in the config process, immediately before we export # the Makefile and other generated files, so that configure tests # which make make use of the autotools-conventional flags # (e.g. [proj-check-rpath]) may do so before we "mangle" them here. proj-remap-autoconf-dir-vars proj-dot-ins-process -validate make-config-header sqlite_cfg.h \ -bare {SIZEOF_* HAVE_DECL_*} \ -none {HAVE_CFLAG_* LDFLAGS_* SH_* SQLITE_AUTORECONFIG TARGET_* USE_GCOV TCL_*} \ -auto {HAVE_* PACKAGE_*} \ -none * proj-touch sqlite_cfg.h ; # help avoid frequent unnecessary @SQLITE_AUTORECONFIG@ } ######################################################################## # Handle --with-wasi-sdk[=DIR] # # This must be run relatively early on because it may change the # toolchain and disable a number of config options. However, in the # canonical build this must come after [sqlite-check-common-bins]. proc sqlite-handle-wasi-sdk {} { set wasiSdkDir [opt-val with-wasi-sdk] ; # ??? [lindex [opt-val with-wasi-sdk] end] define HAVE_WASI_SDK 0 if {$wasiSdkDir eq ""} { return 0 } elseif {$::sqliteConfig(is-cross-compiling)} { proj-fatal "Cannot combine --with-wasi-sdk with cross-compilation" } msg-result "Checking WASI SDK directory \[$wasiSdkDir]... " proj-affirm-files-exist -v {*}[prefix "$wasiSdkDir/bin/" {clang wasm-ld ar}] define HAVE_WASI_SDK 1 define WASI_SDK_DIR $wasiSdkDir # Disable numerous options which we know either can't work or are # not useful in this build... msg-result "Using wasi-sdk clang. Disabling CLI shell and modifying config flags:" # Boolean (--enable-/--disable-) flags which must be switched off: foreach opt { dynlink-tools editline gcov icu-collations load-extension readline shared tcl threadsafe } { if {[proj-opt-exists $opt] && [opt-bool $opt]} { # -^^^^ not all builds define all of these flags msg-result " --disable-$opt" proj-opt-set $opt 0 } } # Non-boolean flags which need to be cleared: foreach opt { with-emsdk with-icu-config with-icu-ldflags with-icu-cflags with-linenoise with-tcl } { if {[proj-opt-was-provided $opt]} { msg-result " removing --$opt" proj-opt-set $opt "" } } # Remember that we now have a discrepancy between # $::sqliteConfig(is-cross-compiling) and [proj-is-cross-compiling]. set ::sqliteConfig(is-cross-compiling) 1 # # Changing --host and --target have no effect here except to # possibly cause confusion. Autosetup has finished processing them # by this point. # # host_alias=wasm32-wasi # target=wasm32-wasi # # Merely changing CC, LD, and AR to the wasi-sdk's is enough to get # sqlite3.o building in WASM format. # define CC "${wasiSdkDir}/bin/clang" define LD "${wasiSdkDir}/bin/wasm-ld" define AR "${wasiSdkDir}/bin/ar" #define STRIP "${wasiSdkDir}/bin/strip" return 1 }; # sqlite-handle-wasi-sdk ######################################################################## # TCL... # # sqlite-check-tcl performs most of the --with-tcl and --with-tclsh # handling. Some related bits and pieces are performed before and # after that function is called. # # Important [define]'d vars: # # - HAVE_TCL indicates whether we have a tclsh suitable for building # the TCL SQLite extension and, by extension, the testing # infrastructure. This must only be 1 for environments where # tclConfig.sh can be found. # # - TCLSH_CMD is the path to the canonical tclsh or "". It never # refers to jimtcl. # # - TCL_CONFIG_SH is the path to tclConfig.sh or "". # # - TCLLIBDIR is the dir to which libtclsqlite3 gets installed. # # - BTCLSH = the path to the tcl interpreter used for in-tree code # generation. It may be jimtcl or the canonical tclsh but may not # be empty - this tree requires TCL to generated numerous # components. # # If --tcl or --with-tcl are provided but no TCL is found, this # function fails fatally. If they are not explicitly provided then # failure to find TCL is not fatal but a loud warning will be emitted. # proc sqlite-check-tcl {} { define TCLSH_CMD false ; # Significant is that it exits with non-0 define HAVE_TCL 0 ; # Will be enabled via --tcl or a successful search define TCLLIBDIR "" ; # Installation dir for TCL extension lib define TCL_CONFIG_SH ""; # full path to tclConfig.sh # Clear out all vars which would harvest from tclConfig.sh so that # the late-config validation of @VARS@ works even if --disable-tcl # is used. proj-tclConfig-sh-to-autosetup "" file delete -force ".tclenv.sh"; # ensure no stale state from previous configures. if {![opt-bool tcl]} { proj-indented-notice { NOTE: TCL is disabled via --disable-tcl. This means that none of the TCL-based components will be built, including tests and sqlite3_analyzer. } return } # TODO: document the steps this is taking. set srcdir $::autosetup(srcdir) msg-result "Checking for a suitable tcl... " proj-assert [proj-opt-truthy tcl] set use_tcl 1 set with_tclsh [opt-val with-tclsh] set with_tcl [opt-val with-tcl] if {"prefix" eq $with_tcl} { set with_tcl [get-define prefix] } proc-debug "use_tcl ${use_tcl}" proc-debug "with_tclsh=${with_tclsh}" proc-debug "with_tcl=$with_tcl" if {"" eq $with_tclsh && "" eq $with_tcl} { # If neither --with-tclsh nor --with-tcl are provided, try to find # a workable tclsh. set with_tclsh [proj-first-bin-of tclsh9.1 tclsh9.0 tclsh8.6 tclsh] proc-debug "with_tclsh=${with_tclsh}" } set doConfigLookup 1 ; # set to 0 to test the tclConfig.sh-not-found cases if {"" ne $with_tclsh} { # --with-tclsh was provided or found above. Validate it and use it # to trump any value passed via --with-tcl=DIR. if {![file-isexec $with_tclsh]} { proj-fatal "TCL shell $with_tclsh is not executable" } else { define TCLSH_CMD $with_tclsh #msg-result "Using tclsh: $with_tclsh" } if {$doConfigLookup && [catch {exec $with_tclsh $::autosetup(libdir)/find_tclconfig.tcl} result] == 0} { set with_tcl $result } if {"" ne $with_tcl && [file isdir $with_tcl]} { msg-result "$with_tclsh recommends the tclConfig.sh from $with_tcl" } else { proj-warn "$with_tclsh is unable to recommend a tclConfig.sh" set use_tcl 0 } } set cfg "" set tclSubdirs {tcl9.1 tcl9.0 tcl8.6 lib} while {$use_tcl} { if {"" ne $with_tcl} { # Ensure that we can find tclConfig.sh under ${with_tcl}/... if {$doConfigLookup} { if {[file readable "${with_tcl}/tclConfig.sh"]} { set cfg "${with_tcl}/tclConfig.sh" } else { foreach i $tclSubdirs { if {[file readable "${with_tcl}/$i/tclConfig.sh"]} { set cfg "${with_tcl}/$i/tclConfig.sh" break } } } } if {"" eq $cfg} { proj-fatal "No tclConfig.sh found under ${with_tcl}" } } else { # If we have not yet found a tclConfig.sh file, look in $libdir # which is set automatically by autosetup or via the --prefix # command-line option. See # https://sqlite.org/forum/forumpost/e04e693439a22457 set libdir [get-define libdir] if {[file readable "${libdir}/tclConfig.sh"]} { set cfg "${libdir}/tclConfig.sh" } else { foreach i $tclSubdirs { if {[file readable "${libdir}/$i/tclConfig.sh"]} { set cfg "${libdir}/$i/tclConfig.sh" break } } } if {![file readable $cfg]} { break } } msg-result "Using tclConfig.sh: $cfg" break } define TCL_CONFIG_SH $cfg # Export a subset of tclConfig.sh to the current TCL-space. If $cfg # is an empty string, this emits empty-string entries for the # various options we're interested in. proj-tclConfig-sh-to-autosetup $cfg if {"" eq $with_tclsh && $cfg ne ""} { # We have tclConfig.sh but no tclsh. Attempt to locate a tclsh # based on info from tclConfig.sh. set tclExecPrefix [get-define TCL_EXEC_PREFIX] proj-assert {"" ne $tclExecPrefix} set tryThese [list \ $tclExecPrefix/bin/tclsh[get-define TCL_VERSION] \ $tclExecPrefix/bin/tclsh ] foreach trySh $tryThese { if {[file-isexec $trySh]} { set with_tclsh $trySh break } } if {![file-isexec $with_tclsh]} { proj-warn "Cannot find a usable tclsh (tried: $tryThese) } } define TCLSH_CMD $with_tclsh if {$use_tcl} { # Set up the TCLLIBDIR # # 2024-10-28: calculation of TCLLIBDIR is now done via the shell # in main.mk (search it for T.tcl.env.sh) so that # static/hand-written makefiles which import main.mk do not have # to define that before importing main.mk. Even so, we export # TCLLIBDIR from here, which will cause the canonical makefile to # use this one rather than to re-calculate it at make-time. set tcllibdir [get-env TCLLIBDIR ""] set sq3Ver [get-define PACKAGE_VERSION] if {"" eq $tcllibdir} { # Attempt to extract TCLLIBDIR from TCL's $auto_path if {"" ne $with_tclsh && [catch {exec echo "puts stdout \$auto_path" | "$with_tclsh"} result] == 0} { foreach i $result { if {[file isdir $i]} { set tcllibdir $i/sqlite${sq3Ver} break } } } else { proj-warn "Cannot determine TCLLIBDIR." # The makefile will fail fatally in this case if a target is # invoked which requires TCLLIBDIR. } } #if {"" ne $tcllibdir} { msg-result "TCLLIBDIR = ${tcllibdir}"; } define TCLLIBDIR $tcllibdir }; # find TCLLIBDIR if {[file-isexec $with_tclsh]} { msg-result "Using tclsh: $with_tclsh" if {$cfg ne ""} { define HAVE_TCL 1 } else { proj-warn "Found tclsh but no tclConfig.sh." } } show-notices # If TCL is not found: if it was explicitly requested then fail # fatally, else just emit a warning. If we can find the APIs needed # to generate a working JimTCL then that will suffice for build-time # TCL purposes (see: proc sqlite-determine-codegen-tcl). if {![get-define HAVE_TCL] && ([proj-opt-was-provided tcl] || [proj-opt-was-provided with-tcl])} { proj-fatal "TCL support was requested but no tclConfig.sh could be found." } if {"" eq $cfg} { proj-assert {0 == [get-define HAVE_TCL]} proj-indented-notice { WARNING: Cannot find a usable tclConfig.sh file. Use --with-tcl=DIR to specify a directory where tclConfig.sh can be found. SQLite does not use TCL internally, but some optional components require TCL, including tests and sqlite3_analyzer. } } }; # sqlite-check-tcl ######################################################################## # sqlite-determine-codegen-tcl checks which TCL to use as a code # generator. By default, prefer jimsh simply because we have it # in-tree (it's part of autosetup) unless --with-tclsh=X is used, in # which case prefer X. # # Returns the human-readable name of the TCL it selects. Fails fatally # if it cannot detect a TCL appropriate for code generation. # # Defines: # # - BTCLSH = the TCL shell used for code generation. It may set this # to an unexpanded makefile var name. # # - CFLAGS_JIMSH = any flags needed for buildng a BTCLSH-compatible # jimsh. The defaults may be passed on to configure as # CFLAGS_JIMSH=... proc sqlite-determine-codegen-tcl {} { msg-result "Checking for TCL to use for code generation... " define CFLAGS_JIMSH [proj-get-env CFLAGS_JIMSH {-O1}] set cgtcl [opt-val with-tclsh jimsh] if {"jimsh" ne $cgtcl} { # When --with-tclsh=X is used, use that for all TCL purposes, # including in-tree code generation, per developer request. define BTCLSH "\$(TCLSH_CMD)" return $cgtcl } set flagsToRestore {CC CFLAGS AS_CFLAGS CPPFLAGS AS_CPPFLAGS LDFLAGS LINKFLAGS LIBS CROSS} define-push $flagsToRestore { # We have to swap CC to CC_FOR_BUILD for purposes of the various # [cc-...] tests below. Recall that --with-wasi-sdk may have # swapped out CC with one which is not appropriate for this block. # Per consulation with autosetup's creator, doing this properly # requires us to [define-push] the whole $flagsToRestore list # (plus a few others which are not relevant in this tree). # # These will get set to their previous values at the end of this # block. foreach flag $flagsToRestore {define $flag ""} define CC [get-define CC_FOR_BUILD] # These headers are technically optional for JimTCL but necessary if # we want to use it for code generation: set sysh [cc-check-includes dirent.h sys/time.h] # jimsh0.c hard-codes #define's for HAVE_DIRENT_H and # HAVE_SYS_TIME_H on the platforms it supports, so we do not # need to add -D... flags for those. We check for them here only # so that we can avoid the situation that we later, at # make-time, try to compile jimsh but it then fails due to # missing headers (i.e. fail earlier rather than later). if {$sysh && [cc-check-functions realpath]} { define-append CFLAGS_JIMSH -DHAVE_REALPATH define BTCLSH "\$(JIMSH)" set ::sqliteConfig(use-jim-for-codegen) 1 } elseif {$sysh && [cc-check-functions _fullpath]} { # _fullpath() is a Windows API. It's not entirely clear # whether we need to add {-DHAVE_SYS_TIME_H -DHAVE_DIRENT_H} # to CFLAGS_JIMSH in this case. On MinGW32 we definitely do # not want to because it already hard-codes them. On _MSC_VER # builds it does not. define-append CFLAGS_JIMSH -DHAVE__FULLPATH define BTCLSH "\$(JIMSH)" set ::sqliteConfig(use-jim-for-codegen) 1 } elseif {[file-isexec [get-define TCLSH_CMD]]} { set cgtcl [get-define TCLSH_CMD] define BTCLSH "\$(TCLSH_CMD)" } else { # One last-ditch effort to find TCLSH_CMD: use info from # tclConfig.sh to try to find a tclsh if {"" eq [get-define TCLSH_CMD]} { set tpre [get-define TCL_EXEC_PREFIX] if {"" ne $tpre} { set tv [get-define TCL_VERSION] if {[file-isexec "${tpre}/bin/tclsh${tv}"]} { define TCLSH_CMD "${tpre}/bin/tclsh${tv}" } elseif {[file-isexec "${tpre}/bin/tclsh"]} { define TCLSH_CMD "${tpre}/bin/tclsh" } } } set cgtcl [get-define TCLSH_CMD] if {![file-isexec $cgtcl]} { proj-fatal "Cannot find a tclsh to use for code generation." } define BTCLSH "\$(TCLSH_CMD)" } }; # /define-push $flagsToRestore return $cgtcl }; # sqlite-determine-codegen-tcl ######################################################################## # Runs sqlite-check-tcl and, if this is the canonical build, # sqlite-determine-codegen-tcl. proc sqlite-handle-tcl {} { sqlite-check-tcl if {"canonical" ne $::sqliteConfig(build-mode)} return msg-result "TCL for code generation: [sqlite-determine-codegen-tcl]" # Determine the base name of the Tcl extension's DLL # if {[get-define HAVE_TCL]} { if {[string match *-cygwin [get-define host]]} { set libname cyg } else { set libname lib } if {[get-define TCL_MAJOR_VERSION] > 8} { append libname tcl9 } append libname sqlite } else { set libname "" } define TCL_EXT_DLL_BASENAME $libname # The extension is added in the makefile } ######################################################################## # Handle the --enable/disable-rpath flag. proc sqlite-handle-rpath {} { # autosetup/cc-shared.tcl sets the rpath flag definition in # [get-define SH_LINKRPATH], but it does so on a per-platform basis # rather than as a compiler check. Though we should do a proper # compiler check (as proj-check-rpath does), we may want to consider # adopting its approach of clearing the rpath flags for environments # for which sqlite-env-is-unix-on-windows returns a non-empty # string. # https://sqlite.org/forum/forumpost/13cac3b56516f849 if {[proj-opt-truthy rpath]} { proj-check-rpath } else { msg-result "Disabling use of rpath." define LDFLAGS_RPATH "" } } ######################################################################## # If the --dump-defines configure flag is provided then emit a list of # all [define] values to config.defines.txt, else do nothing. proc sqlite-dump-defines {} { proj-if-opt-truthy dump-defines { make-config-header $::sqliteConfig(dump-defines-txt) \ -bare {SQLITE_OS* SQLITE_DEBUG USE_*} \ -str {BIN_* CC LD AR LDFLAG* OPT_*} \ -auto {*} # achtung: ^^^^ whichever SQLITE_OS_foo flag which is set to 0 will # get _undefined_ here unless it's part of the -bare set. if {"" ne $::sqliteConfig(dump-defines-json)} { msg-result "--dump-defines is creating $::sqliteConfig(dump-defines-json)" ######################################################################## # Dump config-defines.json... # Demonstrate (mis?)handling of spaces in JSON-export array values: # define-append OPT_FOO.list {"-DFOO=bar baz" -DBAR="baz barre"} define OPT_FEATURE_FLAGS.list [get-define OPT_FEATURE_FLAGS] define OPT_SHELL.list [get-define OPT_SHELL] set dumpDefsOpt { -bare {SIZEOF_* HAVE_DECL_*} -none {HAVE_CFLAG_* LDFLAGS_* SH_* SQLITE_AUTORECONFIG TARGET_* USE_GCOV TCL_*} -array {*.list} -auto {OPT_* PACKAGE_* HAVE_*} } # if {$::sqliteConfig(dump-defines-json-include-lowercase)} { # lappend dumpDefsOpt -none {lib_*} ; # remnants from proj-check-function-in-lib and friends # lappend dumpDefsOpt -auto {[a-z]*} # } lappend dumpDefsOpt -none * proj-dump-defs-json $::sqliteConfig(dump-defines-json) {*}$dumpDefsOpt undefine OPT_FEATURE_FLAGS.list undefine OPT_SHELL.list } } } |
Added autosetup/system.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 | # Copyright (c) 2010 WorkWare Systems http://www.workware.net.au/ # All rights reserved # @synopsis: # # This module supports common system interrogation and options # such as '--host', '--build', '--prefix', and setting 'srcdir', 'builddir', and 'EXEEXT'. # # It also support the "feature" naming convention, where searching # for a feature such as 'sys/type.h' defines 'HAVE_SYS_TYPES_H'. # # It defines the following variables, based on '--prefix' unless overridden by the user: # ## datadir ## sysconfdir ## sharedstatedir ## localstatedir ## infodir ## mandir ## includedir # # If '--prefix' is not supplied, it defaults to '/usr/local' unless 'options-defaults { prefix ... }' is used *before* # including the 'system' module. if {[is-defined defaultprefix]} { user-notice "Note: defaultprefix is deprecated. Use options-defaults to set default options" options-defaults [list prefix [get-define defaultprefix]] } options { host:host-alias => {a complete or partial cpu-vendor-opsys for the system where the application will run (defaults to the same value as --build)} build:build-alias => {a complete or partial cpu-vendor-opsys for the system where the application will be built (defaults to the result of running config.guess)} prefix:dir=/usr/local => {the target directory for the build (default: '@default@')} # These (hidden) options are supported for autoconf/automake compatibility exec-prefix: bindir: sbindir: includedir: mandir: infodir: libexecdir: datadir: libdir: sysconfdir: sharedstatedir: localstatedir: runstatedir: maintainer-mode=0 dependency-tracking=0 silent-rules=0 program-prefix: program-suffix: program-transform-name: x-includes: x-libraries: } # @check-feature name { script } # # defines feature '$name' to the return value of '$script', # which should be 1 if found or 0 if not found. # # e.g. the following will define 'HAVE_CONST' to 0 or 1. # ## check-feature const { ## cctest -code {const int _x = 0;} ## } proc check-feature {name code} { msg-checking "Checking for $name..." set r [uplevel 1 $code] define-feature $name $r if {$r} { msg-result "ok" } else { msg-result "not found" } return $r } # @have-feature name ?default=0? # # Returns the value of feature '$name' if defined, or '$default' if not. # # See 'feature-define-name' for how the "feature" name # is translated into the "define" name. # proc have-feature {name {default 0}} { get-define [feature-define-name $name] $default } # @define-feature name ?value=1? # # Sets the feature 'define' to '$value'. # # See 'feature-define-name' for how the "feature" name # is translated into the "define" name. # proc define-feature {name {value 1}} { define [feature-define-name $name] $value } # @feature-checked name # # Returns 1 if feature '$name' has been checked, whether true or not. # proc feature-checked {name} { is-defined [feature-define-name $name] } # @feature-define-name name ?prefix=HAVE_? # # Converts a "feature" name to the corresponding "define", # e.g. 'sys/stat.h' becomes 'HAVE_SYS_STAT_H'. # # Converts '*' to 'P' and all non-alphanumeric to underscore. # proc feature-define-name {name {prefix HAVE_}} { string toupper $prefix[regsub -all {[^a-zA-Z0-9]} [regsub -all {[*]} $name p] _] } # @write-if-changed filename contents ?script? # # If '$filename' doesn't exist, or it's contents are different to '$contents', # the file is written and '$script' is evaluated. # # Otherwise a "file is unchanged" message is displayed. proc write-if-changed {file buf {script {}}} { set old [readfile $file ""] if {$old eq $buf && [file exists $file]} { msg-result "$file is unchanged" } else { writefile $file $buf\n uplevel 1 $script } } # @include-file infile mapping # # The core of make-template, called recursively for each @include # directive found within that template so that this proc's result # is the fully-expanded template. # # The mapping parameter is how we expand @varname@ within the template. # We do that inline within this step only for @include directives which # can have variables in the filename arg. A separate substitution pass # happens when this recursive function returns, expanding the rest of # the variables. # proc include-file {infile mapping} { # A stack of true/false conditions, one for each nested conditional # starting with "true" set condstack {1} set result {} set linenum 0 foreach line [split [readfile $infile] \n] { incr linenum if {[regexp {^@(if|else|endif)(\s*)(.*)} $line -> condtype condspace condargs]} { if {$condtype eq "if"} { if {[string length $condspace] == 0} { autosetup-error "$infile:$linenum: Invalid expression: $line" } if {[llength $condargs] == 1} { # ABC => [get-define ABC] ni {0 ""} # !ABC => [get-define ABC] in {0 ""} lassign $condargs condvar if {[regexp {^!(.*)} $condvar -> condvar]} { set op in } else { set op ni } set condexpr "\[[list get-define $condvar]\] $op {0 {}}" } else { # Translate alphanumeric ABC into [get-define ABC] and leave the # rest of the expression untouched regsub -all {([A-Z][[:alnum:]_]*)} $condargs {[get-define \1]} condexpr } if {[catch [list expr $condexpr] condval]} { dputs $condval autosetup-error "$infile:$linenum: Invalid expression: $line" } dputs "@$condtype: $condexpr => $condval" } if {$condtype ne "if"} { if {[llength $condstack] <= 1} { autosetup-error "$infile:$linenum: Error: @$condtype missing @if" } elseif {[string length $condargs] && [string index $condargs 0] ne "#"} { autosetup-error "$infile:$linenum: Error: Extra arguments after @$condtype" } } switch -exact $condtype { if { # push condval lappend condstack $condval } else { # Toggle the last entry set condval [lpop condstack] set condval [expr {!$condval}] lappend condstack $condval } endif { if {[llength $condstack] == 0} { user-notice "$infile:$linenum: Error: @endif missing @if" } lpop condstack } } continue } # Only continue if the stack contains all "true" if {"0" in $condstack} { continue } if {[regexp {^@include\s+(.*)} $line -> filearg]} { set incfile [string map $mapping $filearg] if {[file exists $incfile]} { lappend ::autosetup(deps) [file-normalize $incfile] lappend result {*}[include-file $incfile $mapping] } else { user-error "$infile:$linenum: Include file $incfile is missing" } continue } if {[regexp {^@define\s+(\w+)\s+(.*)} $line -> var val]} { define $var $val continue } lappend result $line } return $result } # @make-template template ?outfile? # # Reads the input file '<srcdir>/$template' and writes the output file '$outfile' # (unless unchanged). # If '$outfile' is blank/omitted, '$template' should end with '.in' which # is removed to create the output file name. # # Each pattern of the form '@define@' is replaced with the corresponding # "define", if it exists, or left unchanged if not. # # The special value '@srcdir@' is substituted with the relative # path to the source directory from the directory where the output # file is created, while the special value '@top_srcdir@' is substituted # with the relative path to the top level source directory. # # Conditional sections may be specified as follows: ## @if NAME eq "value" ## lines ## @else ## lines ## @endif # # Where 'NAME' is a defined variable name and '@else' is optional. # Note that variables names *must* start with an uppercase letter. # If the expression does not match, all lines through '@endif' are ignored. # # The alternative forms may also be used: ## @if NAME (true if the variable is defined, but not empty and not "0") ## @if !NAME (opposite of the form above) ## @if <general-tcl-expression> # # In the general Tcl expression, any words beginning with an uppercase letter # are translated into [get-define NAME] # # Expressions may be nested # proc make-template {template {out {}}} { set infile [file join $::autosetup(srcdir) $template] if {![file exists $infile]} { user-error "Template $template is missing" } # Define this as late as possible define AUTODEPS $::autosetup(deps) if {$out eq ""} { if {[file ext $template] ne ".in"} { autosetup-error "make_template $template has no target file and can't guess" } set out [file rootname $template] } set outdir [file dirname $out] # Make sure the directory exists file mkdir $outdir # Set up srcdir and top_srcdir to be relative to the target dir define srcdir [relative-path [file join $::autosetup(srcdir) $outdir] $outdir] define top_srcdir [relative-path $::autosetup(srcdir) $outdir] # Build map from global defines to their values so they can be # substituted into @include file names. proc build-define-mapping {} { set mapping {} foreach {n v} [array get ::define] { lappend mapping @$n@ $v } return $mapping } set mapping [build-define-mapping] set result [include-file $infile $mapping] # Rebuild the define mapping in case we ran across @define # directives in the template or a file it @included, then # apply that mapping to the expanded template. set mapping [build-define-mapping] write-if-changed $out [string map $mapping [join $result \n]] { msg-result "Created [relative-path $out] from [relative-path $template]" } } proc system-init {} { global autosetup # build/host tuples and cross-compilation prefix opt-str build build "" define build_alias $build if {$build eq ""} { define build [config_guess] } else { define build [config_sub $build] } opt-str host host "" define host_alias $host if {$host eq ""} { define host [get-define build] set cross "" } else { define host [config_sub $host] set cross $host- } define cross [get-env CROSS $cross] # build/host _cpu, _vendor and _os foreach type {build host} { set v [get-define $type] if {![regexp {^([^-]+)-([^-]+)-(.*)$} $v -> cpu vendor os]} { user-error "Invalid canonical $type: $v" } define ${type}_cpu $cpu define ${type}_vendor $vendor define ${type}_os $os } opt-str prefix prefix /usr/local # These are for compatibility with autoconf define target [get-define host] define prefix $prefix define builddir $autosetup(builddir) define srcdir $autosetup(srcdir) define top_srcdir $autosetup(srcdir) define abs_top_srcdir [file-normalize $autosetup(srcdir)] define abs_top_builddir [file-normalize $autosetup(builddir)] # autoconf supports all of these define exec_prefix [opt-str exec-prefix exec_prefix $prefix] foreach {name defpath} { bindir /bin sbindir /sbin libexecdir /libexec libdir /lib } { define $name [opt-str $name o $exec_prefix$defpath] } foreach {name defpath} { datadir /share sharedstatedir /com infodir /share/info mandir /share/man includedir /include } { define $name [opt-str $name o $prefix$defpath] } if {$prefix ne {/usr}} { opt-str sysconfdir sysconfdir $prefix/etc } else { opt-str sysconfdir sysconfdir /etc } define sysconfdir $sysconfdir define localstatedir [opt-str localstatedir o /var] define runstatedir [opt-str runstatedir o /run] define SHELL [get-env SHELL [find-an-executable sh bash ksh]] # These could be used to generate Makefiles following some automake conventions define AM_SILENT_RULES [opt-bool silent-rules] define AM_MAINTAINER_MODE [opt-bool maintainer-mode] define AM_DEPENDENCY_TRACKING [opt-bool dependency-tracking] # Windows vs. non-Windows switch -glob -- [get-define host] { *-*-ming* - *-*-cygwin - *-*-msys { define-feature windows define EXEEXT .exe } default { define EXEEXT "" } } # Display msg-result "Host System...[get-define host]" msg-result "Build System...[get-define build]" } system-init |
Added autosetup/teaish/README.txt.
> > > > | 1 2 3 4 | The *.tcl files in this directory are part of the SQLite's "autoconf" bundle which are specific to the TEA(-ish) build. During the tarball generation process, they are copied into <TOP>/autoconf/autosetup/teaish (which itself is created as part of that process). |
Added autosetup/teaish/core.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 2406 2407 2408 2409 2410 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 | ######################################################################## # 2025 April 5 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # * May you do good and not evil. # * May you find forgiveness for yourself and forgive others. # * May you share freely, never taking more than you give. # ######################################################################## # ----- @module teaish.tcl ----- # @section TEA-ish ((TCL Extension Architecture)-ish) # # Functions in this file with a prefix of teaish__ are # private/internal APIs. Those with a prefix of teaish- are # public APIs. # # Teaish has a hard dependency on proj.tcl, and any public API members # of that module are considered legal for use by teaish extensions. # # Project home page: https://fossil.wanderinghorse.net/r/teaish use proj # # API-internal settings and shared state. array set teaish__Config [proj-strip-hash-comments { # # Teaish's version number, not to be confused with # teaish__PkgInfo(-version). # version 0.1-beta # set to 1 to enable some internal debugging output debug-enabled 0 # # 0 = don't yet have extension's pkgindex # 0x01 = found TEAISH_EXT_DIR/pkgIndex.tcl.in # 0x02 = found srcdir/pkgIndex.tcl.in # 0x10 = found TEAISH_EXT_DIR/pkgIndex.tcl (static file) # 0x20 = static-pkgIndex.tcl pragma: behave as if 0x10 # 0x100 = disabled by -tm.tcl.in # 0x200 = disabled by -tm.tcl # # Reminder: it's significant that the bottom 4 bits be # cases where teaish manages ./pkgIndex.tcl. # pkgindex-policy 0 # # The pkginit counterpart of pkgindex-policy: # # 0 = no pkginit # 0x01 = found default X.in: generate X from X.in # 0x10 = found static pkginit file X # 0x02 = user-provided X.in generates ./X. # 0x20 = user-provided static pkginit file X # # The 0x0f bits indicate that teaish is responsible for cleaning up # the (generated) pkginit file. # pkginit-policy 0 # # 0 = no tm.tcl # 0x01 = tm.tcl.in # 0x10 = static tm.tcl tm-policy 0 # # If 1+ then teaish__verbose will emit messages. # verbose 0 # # Mapping of pkginfo -flags to their TEAISH_xxx define (if any). # This must not be modified after initialization. # pkginfo-f2d { -name TEAISH_NAME -name.dist TEAISH_DIST_NAME -name.pkg TEAISH_PKGNAME -version TEAISH_VERSION -libDir TEAISH_LIBDIR_NAME -loadPrefix TEAISH_LOAD_PREFIX -vsatisfies TEAISH_VSATISFIES -pkgInit.tcl TEAISH_PKGINIT_TCL -pkgInit.tcl.in TEAISH_PKGINIT_TCL_IN -url TEAISH_URL -tm.tcl TEAISH_TM_TCL -tm.tcl.in TEAISH_TM_TCL_IN -options {} -pragmas {} } # # Queues for use with teaish-checks-queue and teaish-checks-run. # queued-checks-pre {} queued-checks-post {} # Whether or not "make dist" parts are enabled. They get enabled # when building from an extension's dir, disabled when building # elsewhere. dist-enabled 1 # Whether or not "make install" parts are enabled. By default # they are, but we have a single use case where they're # both unnecessary and unhelpful, so... install-enabled 1 # By default we enable compilation of a native extension but if the # extension has no native code or the user wants to take that over # via teaish.make.in or provide a script-only extension, we will # elide the default compilation rules if this is 0. dll-enabled 1 # Files to include in the "make dist" bundle. dist-files {} # List of source files for the extension. extension-src {} # Path to the teaish.tcl file. teaish.tcl {} # Dir where teaish.tcl is found. extension-dir {} # Whether the generates TEASH_VSATISFIES_CODE should error out on a # satisfies error. If 0, it uses return instead of error. vsatisfies-error 1 # Whether or not to allow a "full dist" - a "make dist" build which # includes both the extension and teaish. By default this is only on # if the extension dir is teaish's dir. dist-full-enabled 0 }] set teaish__Config(core-dir) $::autosetup(libdir)/teaish # # Array of info managed by teaish-pkginfo-get and friends. Has the # same set of keys as $teaish__Config(pkginfo-f2d). # array set teaish__PkgInfo {} # # Runs {*}$args if $lvl is <= the current verbosity level, else it has # no side effects. # proc teaish__verbose {lvl args} { if {$lvl <= $::teaish__Config(verbose)} { {*}$args } } # # @teaish-argv-has flags... # # Returns true if any arg in $::argv matches any of the given globs, # else returns false. # proc teaish-argv-has {args} { foreach glob $args { foreach arg $::argv { if {[string match $glob $arg]} { return 1 } } } return 0 } if {[teaish-argv-has --teaish-verbose --t-v]} { # Check this early so that we can use verbose-only messages in the # pre-options-parsing steps. set ::teaish__Config(verbose) 1 #teaish__verbose 1 msg-result "--teaish-verbose activated" } msg-quiet use system ; # Outputs "Host System" and "Build System" lines if {"--help" ni $::argv} { teaish__verbose 1 msg-result "TEA(ish) Version = $::teaish__Config(version)" teaish__verbose 1 msg-result "Source dir = $::autosetup(srcdir)" teaish__verbose 1 msg-result "Build dir = $::autosetup(builddir)" } # # @teaish-configure-core # # Main entry point for the TEA-ish configure process. auto.def's primary # (ideally only) job should be to call this. # proc teaish-configure-core {} { proj-tweak-default-env-dirs set ::teaish__Config(install-mode) [teaish-argv-has --teaish-install*] set ::teaish__Config(create-ext-mode) \ [teaish-argv-has --teaish-create-extension=* --t-c-e=*] set gotExt 0; # True if an extension config is found if {!$::teaish__Config(create-ext-mode) && !$::teaish__Config(install-mode)} { # Don't look for an extension if we're in --t-c-e or --t-i mode set gotExt [teaish__find_extension] } # # Set up the core --flags. This needs to come before teaish.tcl is # sourced so that that file can use teaish-pkginfo-set to append # options. # options-add [proj-strip-hash-comments { with-tcl:DIR => {Directory containing tclConfig.sh or a directory one level up from that, from which we can derive a directory containing tclConfig.sh. Defaults to the $TCL_HOME environment variable.} with-tclsh:PATH => {Full pathname of tclsh to use. It is used for trying to find tclConfig.sh. Warning: if its containing dir has multiple tclsh versions, it may select the wrong tclConfig.sh! Defaults to the $TCLSH environment variable.} # TEA has --with-tclinclude but it appears to only be useful for # building an extension against an uninstalled copy of TCL's own # source tree. The policy here is that either we get that info # from tclConfig.sh or we give up. # # with-tclinclude:DIR # => {Specify the directory which contains the tcl.h. This should not # normally be required, as that information comes from tclConfig.sh.} # We _generally_ want to reduce the possibility of flag collisions with # extensions, and thus use a teaish-... prefix on most flags. However, # --teaish-extension-dir is frequently needed, so... # # As of this spontaneous moment, we'll settle on using --t-A-X to # abbreviate --teaish-A...-X... flags when doing so is # unambiguous... ted: t-e-d: teaish-extension-dir:DIR => {Looks for an extension in the given directory instead of the current dir.} t-c-e: teaish-create-extension:TARGET_DIRECTORY => {Writes stub files for creating an extension. Will refuse to overwrite existing files without --teaish-force.} t-f teaish-force => {Has a context-dependent meaning (autosetup defines --force for its own use).} t-d-d teaish-dump-defines => {Dump all configure-defined vars to config.defines.txt} t-v:=0 teaish-verbose:=0 => {Enable more (often extraneous) messages from the teaish core.} t-d teaish-debug=0 => {Enable teaish-specific debug output} t-i teaish-install:=auto => {Installs a copy of teaish, including autosetup, to the target dir. When used with --teaish-create-extension=DIR, a value of "auto" (no no value) will inherit that directory.} #TODO: --teaish-install-extension:=dir as short for # --t-c-e=dir --t-i t-e-p: teaish-extension-pkginfo:pkginfo => {For use with --teaish-create-extension. If used, it must be a list of arguments for use with teaish-pkginfo-set, e.g. --teaish-extension-pkginfo="-name Foo -version 2.3"} t-v-c teaish-vsatisfies-check=1 => {Disable the configure-time "vsatisfies" check on the target tclsh.} }]; # main options. if {$gotExt} { # We found an extension. Source it... set ttcl $::teaish__Config(teaish.tcl) proj-assert {"" ne [teaish-pkginfo-get -name]} proj-assert {[file exists $ttcl]} \ "Expecting to have found teaish.(tcl|config) by now" if {[string match *.tcl $ttcl]} { uplevel 1 {source $::teaish__Config(teaish.tcl)} } else { teaish-pkginfo-set {*}[proj-file-content -trim $ttcl] } unset ttcl # Set up some default values if the extension did not set them. # This must happen _after_ it's sourced but before # teaish-configure is called. array set f2d $::teaish__Config(pkginfo-f2d) foreach {pflag key type val} { - TEAISH_CFLAGS -v "" - TEAISH_LDFLAGS -v "" - TEAISH_MAKEFILE -v "" - TEAISH_MAKEFILE_CODE -v "" - TEAISH_MAKEFILE_IN -v "" - TEAISH_PKGINDEX_TCL -v "" - TEAISH_PKGINDEX_TCL_IN -v "" - TEAISH_PKGINIT_TCL -v "" - TEAISH_PKGINIT_TCL_IN -v "" - TEAISH_PKGINIT_TCL_TAIL -v "" - TEAISH_TEST_TCL -v "" - TEAISH_TEST_TCL_IN -v "" -version - -v 0.0.0 -name.pkg - -e {set ::teaish__PkgInfo(-name)} -name.dist - -e {set ::teaish__PkgInfo(-name)} -libDir - -e { join [list \ $::teaish__PkgInfo(-name.pkg) \ $::teaish__PkgInfo(-version)] "" } -loadPrefix - -e { string totitle $::teaish__PkgInfo(-name.pkg) } -vsatisfies - -v {{Tcl 8.5-}} -pkgInit.tcl - -v "" -pkgInit.tcl.in - -v "" -url - -v "" -tm.tcl - -v "" -tm.tcl.in - -v "" } { set isPIFlag [expr {"-" ne $pflag}] if {$isPIFlag} { if {[info exists ::teaish__PkgInfo($pflag)]} { # Was already set - skip it. continue; } proj-assert {{-} eq $key} set key $f2d($pflag) } proj-assert {"" ne $key} set got [get-define $key "<nope>"] if {"<nope>" ne $got} { # Was already set - skip it. continue } switch -exact -- $type { -v {} -e { set val [eval $val] } default { proj-error "Invalid type flag: $type" } } #puts "***** defining default $pflag $key {$val} isPIFlag=$isPIFlag got=$got" define $key $val if {$isPIFlag} { set ::teaish__PkgInfo($pflag) $val } } unset isPIFlag pflag key type val array unset f2d }; # sourcing extension's teaish.tcl if {[llength [info proc teaish-options]] > 0} { # Add options defined by teaish-options, which is assumed to be # imported via [teaish-get -teaish-tcl]. set o [teaish-options] if {"" ne $o} { options-add $o } } #set opts [proj-options-combine] #lappend opts teaish-debug => {x}; #testing dupe entry handling if {[catch {options {}} msg xopts]} { # Workaround for <https://github.com/msteveb/autosetup/issues/73> # where [options] behaves oddly on _some_ TCL builds when it's # called from deeper than the global scope. dict incr xopts -level return {*}$xopts $msg } proj-xfer-options-aliases { t-c-e => teaish-create-extension t-d => teaish-debug t-d-d => teaish-dump-defines ted => teaish-extension-dir t-e-d => teaish-extension-dir t-e-p => teaish-extension-pkginfo t-f => teaish-force t-i => teaish-install t-v => teaish-verbose t-v-c => teaish-vsatisfies-check } scan [opt-val teaish-verbose 0] %d ::teaish__Config(verbose) set ::teaish__Config(debug-enabled) [opt-bool teaish-debug] set exitEarly 0 if {[proj-opt-was-provided teaish-create-extension]} { teaish__create_extension [opt-val teaish-create-extension] incr exitEarly } if {$::teaish__Config(install-mode)} { teaish__install incr exitEarly } if {$exitEarly} { file delete -force config.log return } proj-assert {1==$gotExt} "Else we cannot have gotten this far" teaish__configure_phase1 } # # Internal config-time debugging output routine. It is not legal to # call this from the global scope. # proc teaish-debug {msg} { if {$::teaish__Config(debug-enabled)} { puts stderr [proj-bold "** DEBUG: \[[proj-scope 1]\]: $msg"] } } # # Runs "phase 1" of the configuration, immediately after processing # --flags. This is what will import the client-defined teaish.tcl. # proc teaish__configure_phase1 {} { msg-result \ [join [list "Configuring build of Tcl extension" \ [proj-bold [teaish-pkginfo-get -name] \ [teaish-pkginfo-get -version]] "..."]] uplevel 1 { use cc cc-db cc-shared cc-lib; # pkg-config } teaish__check_tcl apply {{} { # # If --prefix or --exec-prefix are _not_ provided, use their # TCL_... counterpart from tclConfig.sh. Caveat: by the time we can # reach this point, autosetup's system.tcl will have already done # some non-trivial amount of work with these to create various # derived values from them, so we temporarily end up with a mishmash # of autotools-compatibility var values. That will be straightened # out in the final stage of the configure script via # [proj-remap-autoconf-dir-vars]. # foreach {flag uflag tclVar} { prefix prefix TCL_PREFIX exec-prefix exec_prefix TCL_EXEC_PREFIX } { if {![proj-opt-was-provided $flag]} { if {"exec-prefix" eq $flag} { # If --exec-prefix was not used, ensure that --exec-prefix # derives from the --prefix we may have just redefined. set v {${prefix}} } else { set v [get-define $tclVar "???"] teaish__verbose 1 msg-result "Using \$$tclVar for --$flag=$v" } proj-assert {"???" ne $v} "Expecting teaish__check_tcl to have defined $tclVar" #puts "*** $flag $uflag $tclVar = $v" proj-opt-set $flag $v define $uflag $v # ^^^ As of here, all autotools-compatibility vars which derive # from --$flag, e.g. --libdir, still derive from the default # --$flag value which was active when system.tcl was # included. So long as those flags are not explicitly passed to # the configure script, those will be straightened out via # [proj-remap-autoconf-dir-vars]. } } }}; # --[exec-]prefix defaults teaish__check_common_bins # # Set up library file names # proj-file-extensions teaish__define_pkginfo_derived * teaish-checks-run -pre if {[llength [info proc teaish-configure]] > 0} { # teaish-configure is assumed to be imported via # teaish.tcl teaish-configure } teaish-checks-run -post apply {{} { # Set up "vsatisfies" code for pkgIndex.tcl.in, # _teaish.tester.tcl.in, and for a configure-time check. We would # like to put this before [teaish-checks-run -pre] but it's # marginally conceivable that a client may need to dynamically # calculate the vsatisfies and set it via [teaish-configure]. set vs [get-define TEAISH_VSATISFIES ""] if {"" eq $vs} return set code {} set n 0 # Treat $vs as a list-of-lists {{Tcl 8.5-} {Foo 1.0- -3.0} ...} # and generate Tcl which will run package vsatisfies tests with # that info. foreach pv $vs { set n [llength $pv] if {$n < 2} { proj-error "-vsatisfies: {$pv} appears malformed. Whole list is: $vs" } set pkg [lindex $pv 0] set vcheck {} for {set i 1} {$i < $n} {incr i} { lappend vcheck [lindex $pv $i] } if {[opt-bool teaish-vsatisfies-check]} { set tclsh [get-define TCLSH_CMD] set vsat "package vsatisfies \[ package provide $pkg \] $vcheck" set vputs "puts \[ $vsat \]" #puts "*** vputs = $vputs" scan [exec echo $vputs | $tclsh] %d vvcheck if {0 == $vvcheck} { proj-fatal -up $tclsh "check failed:" $vsat } } if {$::teaish__Config(vsatisfies-error)} { set vunsat \ [list error [list Package \ $::teaish__PkgInfo(-name) $::teaish__PkgInfo(-version) \ requires $pv]] } else { set vunsat return } lappend code \ [string trim [subst -nocommands \ {if { ![package vsatisfies [package provide $pkg] $vcheck] } {\n $vunsat\n}}]] }; # foreach pv define TEAISH_VSATISFIES_CODE [join $code "\n"] }}; # vsatisfies if {[proj-looks-like-windows]} { # Without this, linking of an extension will not work on Cygwin or # Msys2. msg-result "Using USE_TCL_STUBS for Unix(ish)-on-Windows environment" teaish-cflags-add -DUSE_TCL_STUBS=1 } #define AS_LIBDIR $::autosetup(libdir) define TEAISH_TESTUTIL_TCL $::teaish__Config(core-dir)/tester.tcl apply {{} { # # Ensure we have a pkgIndex.tcl and don't have a stale generated one # when rebuilding for different --with-tcl=... values. # if {!$::teaish__Config(pkgindex-policy)} { proj-error "Cannot determine which pkgIndex.tcl to use" } if {0x300 & $::teaish__Config(pkgindex-policy)} { teaish__verbose 1 msg-result "pkgIndex disabled by -tm.tcl(.in)" } else { set tpi [proj-coalesce \ [get-define TEAISH_PKGINDEX_TCL_IN] \ [get-define TEAISH_PKGINDEX_TCL]] proj-assert {$tpi ne ""} \ "TEAISH_PKGINDEX_TCL should have been set up by now" teaish__verbose 1 msg-result "Using pkgIndex from $tpi" if {0x0f & $::teaish__Config(pkgindex-policy)} { # Don't leave stale pkgIndex.tcl laying around yet don't delete # or overwrite a user-managed static pkgIndex.tcl. file delete -force -- [get-define TEAISH_PKGINDEX_TCL] proj-dot-ins-append [get-define TEAISH_PKGINDEX_TCL_IN] } else { teaish-dist-add [file tail $tpi] } } }}; # $::teaish__Config(pkgindex-policy) # # Ensure we clean up TEAISH_PKGINIT_TCL if needed and @-process # TEAISH_PKGINIT_TCL_IN if needed. # if {0x0f & $::teaish__Config(pkginit-policy)} { file delete -force -- [get-define TEAISH_PKGINIT_TCL] proj-dot-ins-append [get-define TEAISH_PKGINIT_TCL_IN] } if {0x0f & $::teaish__Config(tm-policy)} { file delete -force -- [get-define TEAISH_TM_TCL] proj-dot-ins-append [get-define TEAISH_TM_TCL_IN] } apply {{} { # Queue up any remaining dot-in files set dotIns [list] foreach d { TEAISH_TESTER_TCL_IN TEAISH_TEST_TCL_IN TEAISH_MAKEFILE_IN } { lappend dotIns [get-define $d ""] } lappend dotIns $::autosetup(srcdir)/Makefile.in; # must be after TEAISH_MAKEFILE_IN foreach f $dotIns { if {"" ne $f} { proj-dot-ins-append $f } } }} define TEAISH_DIST_FULL \ [expr { $::teaish__Config(dist-enabled) && $::teaish__Config(dist-full-enabled) }] define TEAISH_AUTOSETUP_DIR $::teaish__Config(core-dir) define TEAISH_ENABLE_DIST $::teaish__Config(dist-enabled) define TEAISH_ENABLE_INSTALL $::teaish__Config(install-enabled) define TEAISH_ENABLE_DLL $::teaish__Config(dll-enabled) define TEAISH_TCL $::teaish__Config(teaish.tcl) define TEAISH_DIST_FILES [join $::teaish__Config(dist-files)] define TEAISH_EXT_DIR [join $::teaish__Config(extension-dir)] define TEAISH_EXT_SRC [join $::teaish__Config(extension-src)] proj-setup-autoreconfig TEAISH_AUTORECONFIG foreach f { TEAISH_CFLAGS TEAISH_LDFLAGS } { # Ensure that any of these lists are flattened define $f [join [get-define $f]] } proj-remap-autoconf-dir-vars set tdefs [teaish__defines_to_list] define TEAISH__DEFINES_MAP $tdefs; # injected into _teaish.tester.tcl # # NO [define]s after this point! # proj-dot-ins-process -validate proj-if-opt-truthy teaish-dump-defines { proj-file-write config.defines.txt $tdefs } }; # teaish__configure_phase1 # # Run checks for required binaries. # proc teaish__check_common_bins {} { if {"" eq [proj-bin-define install]} { proj-warn "Cannot find install binary, so 'make install' will not work." define BIN_INSTALL false } if {"" eq [proj-bin-define zip]} { proj-warn "Cannot find zip, so 'make dist.zip' will not work." } if {"" eq [proj-bin-define tar]} { proj-warn "Cannot find tar, so 'make dist.tgz' will not work." } } # # TCL... # # teaish__check_tcl performs most of the --with-tcl and --with-tclsh # handling. Some related bits and pieces are performed before and # after that function is called. # # Important [define]'d vars: # # - TCLSH_CMD is the path to the canonical tclsh or "". # # - TCL_CONFIG_SH is the path to tclConfig.sh or "". # # - TCLLIBDIR is the dir to which the extension library gets # - installed. # proc teaish__check_tcl {} { define TCLSH_CMD false ; # Significant is that it exits with non-0 define TCLLIBDIR "" ; # Installation dir for TCL extension lib define TCL_CONFIG_SH ""; # full path to tclConfig.sh # Clear out all vars which would harvest from tclConfig.sh so that # the late-config validation of @VARS@ works even if --disable-tcl # is used. proj-tclConfig-sh-to-autosetup "" # TODO: better document the steps this is taking. set srcdir $::autosetup(srcdir) msg-result "Checking for a suitable tcl... " set use_tcl 1 set withSh [opt-val with-tclsh [proj-get-env TCLSH]] set tclHome [opt-val with-tcl [proj-get-env TCL_HOME]] if {[string match */lib $tclHome]} { # TEA compatibility kludge: its --with-tcl wants the lib # dir containing tclConfig.sh. #proj-warn "Replacing --with-tcl=$tclHome for TEA compatibility" regsub {/lib^} $tclHome "" tclHome msg-result "NOTE: stripped /lib suffix from --with-tcl=$tclHome (a TEA-ism)" } if {0} { # This misinteracts with the $TCL_PREFIX default: it will use the # autosetup-defined --prefix default if {"prefix" eq $tclHome} { set tclHome [get-define prefix] } } teaish-debug "use_tcl ${use_tcl}" teaish-debug "withSh=${withSh}" teaish-debug "tclHome=$tclHome" if {"" eq $withSh && "" eq $tclHome} { # If neither --with-tclsh nor --with-tcl are provided, try to find # a workable tclsh. set withSh [proj-first-bin-of tclsh9.1 tclsh9.0 tclsh8.6 tclsh] teaish-debug "withSh=${withSh}" } set doConfigLookup 1 ; # set to 0 to test the tclConfig.sh-not-found cases if {"" ne $withSh} { # --with-tclsh was provided or found above. Validate it and use it # to trump any value passed via --with-tcl=DIR. if {![file-isexec $withSh]} { proj-error "TCL shell $withSh is not executable" } else { define TCLSH_CMD $withSh #msg-result "Using tclsh: $withSh" } if {$doConfigLookup && [catch {exec $withSh $::autosetup(libdir)/find_tclconfig.tcl} result] == 0} { set tclHome $result } if {"" ne $tclHome && [file isdirectory $tclHome]} { teaish__verbose 1 msg-result "$withSh recommends the tclConfig.sh from $tclHome" } else { proj-warn "$withSh is unable to recommend a tclConfig.sh" set use_tcl 0 } } set cfg "" set tclSubdirs {tcl9.1 tcl9.0 tcl8.6 tcl8.5 lib} while {$use_tcl} { if {"" ne $tclHome} { # Ensure that we can find tclConfig.sh under ${tclHome}/... if {$doConfigLookup} { if {[file readable "${tclHome}/tclConfig.sh"]} { set cfg "${tclHome}/tclConfig.sh" } else { foreach i $tclSubdirs { if {[file readable "${tclHome}/$i/tclConfig.sh"]} { set cfg "${tclHome}/$i/tclConfig.sh" break } } } } if {"" eq $cfg} { proj-error "No tclConfig.sh found under ${tclHome}" } } else { # If we have not yet found a tclConfig.sh file, look in $libdir # which is set automatically by autosetup or via the --prefix # command-line option. See # https://sqlite.org/forum/forumpost/e04e693439a22457 set libdir [get-define libdir] if {[file readable "${libdir}/tclConfig.sh"]} { set cfg "${libdir}/tclConfig.sh" } else { foreach i $tclSubdirs { if {[file readable "${libdir}/$i/tclConfig.sh"]} { set cfg "${libdir}/$i/tclConfig.sh" break } } } if {![file readable $cfg]} { break } } teaish__verbose 1 msg-result "Using tclConfig.sh = $cfg" break }; # while {$use_tcl} define TCL_CONFIG_SH $cfg # Export a subset of tclConfig.sh to the current TCL-space. If $cfg # is an empty string, this emits empty-string entries for the # various options we're interested in. proj-tclConfig-sh-to-autosetup $cfg if {"" eq $withSh && $cfg ne ""} { # We have tclConfig.sh but no tclsh. Attempt to locate a tclsh # based on info from tclConfig.sh. set tclExecPrefix [get-define TCL_EXEC_PREFIX] proj-assert {"" ne $tclExecPrefix} set tryThese [list \ $tclExecPrefix/bin/tclsh[get-define TCL_VERSION] \ $tclExecPrefix/bin/tclsh ] foreach trySh $tryThese { if {[file-isexec $trySh]} { set withSh $trySh break } } if {![file-isexec $withSh]} { proj-warn "Cannot find a usable tclsh (tried: $tryThese)" } } define TCLSH_CMD $withSh if {$use_tcl} { # Set up the TCLLIBDIR set tcllibdir [get-env TCLLIBDIR ""] set extDirName [teaish-pkginfo-get -libDir] if {"" eq $tcllibdir} { # Attempt to extract TCLLIBDIR from TCL's $auto_path if {"" ne $withSh && [catch {exec echo "puts stdout \$auto_path" | "$withSh"} result] == 0} { foreach i $result { if {[file isdirectory $i]} { set tcllibdir $i/$extDirName break } } } else { proj-error "Cannot determine TCLLIBDIR." } } define TCLLIBDIR $tcllibdir }; # find TCLLIBDIR set gotSh [file-isexec $withSh] set tmdir ""; # first tcl::tm::list entry if {$gotSh} { catch { set tmli [exec echo {puts [tcl::tm::list]} | $withSh] # Reminder: this list contains many names of dirs which do not # exist but are legitimate. If we rely only on an is-dir check, # we can end up not finding any of the many candidates. set firstDir "" foreach d $tmli { if {"" eq $firstDir && ![string match //*:* $d]} { # First non-VFS entry, e.g. not //zipfs: set firstDir $d } if {[file isdirectory $d]} { set tmdir $d break } } if {"" eq $tmdir} { set tmdir $firstDir } }; # find tcl::tm path } define TEAISH_TCL_TM_DIR $tmdir # Finally, let's wrap up... if {$gotSh} { teaish__verbose 1 msg-result "Using tclsh = $withSh" if {$cfg ne ""} { define HAVE_TCL 1 } else { proj-warn "Found tclsh but no tclConfig.sh." } if {"" eq $tmdir} { proj-warn "Did not find tcl::tm directory." } } show-notices # If TCL is not found: if it was explicitly requested then fail # fatally, else just emit a warning. If we can find the APIs needed # to generate a working JimTCL then that will suffice for build-time # TCL purposes (see: proc sqlite-determine-codegen-tcl). if {!$gotSh} { proj-error "Did not find tclsh" } elseif {"" eq $cfg} { proj-indented-notice -error { Cannot find a usable tclConfig.sh file. Use --with-tcl=DIR to specify a directory near which tclConfig.sh can be found, or --with-tclsh=/path/to/tclsh to allow the tclsh binary to locate its tclConfig.sh, with the caveat that a symlink to tclsh, or wrapper script around it, e.g. ~/bin/tclsh -> $HOME/tcl/9.0/bin/tclsh9.1, may not work because tclsh emits different library paths for the former than the latter. } } msg-result "Using Tcl [get-define TCL_VERSION] from [get-define TCL_PREFIX]." teaish__tcl_platform_quirks }; # teaish__check_tcl # # Perform last-minute platform-specific tweaks to account for quirks. # proc teaish__tcl_platform_quirks {} { define TEAISH_POSTINST_PREREQUIRE "" switch -glob -- [get-define host] { *-haiku { # Haiku's default TCLLIBDIR is "all wrong": it points to a # read-only virtual filesystem mount-point. We bend it back to # fit under $TCL_PACKAGE_PATH here. foreach {k d} { vj TCL_MAJOR_VERSION vn TCL_MINOR_VERSION pp TCL_PACKAGE_PATH ld TCLLIBDIR } { set $k [get-define $d] } if {[string match /packages/* $ld]} { set old $ld set tail [file tail $ld] if {8 == $vj} { set ld "${pp}/tcl${vj}.${vn}/${tail}" } else { proj-assert {9 == $vj} set ld "${pp}/${tail}" } define TCLLIBDIR $ld # [load foo.so], without a directory part, does not work via # automated tests on Haiku (but works when run # manually). Similarly, the post-install [package require ...] # test fails, presumably for a similar reason. We work around # the former in _teaish.tester.tcl.in. We work around the # latter by amending the post-install check's ::auto_path (in # Makefile.in). This code MUST NOT contain any single-quotes. define TEAISH_POSTINST_PREREQUIRE \ [join [list set ::auto_path \ \[ linsert \$::auto_path 0 $ld \] \; \ ]] proj-indented-notice [subst -nocommands -nobackslashes { Haiku users take note: patching target installation dir to match Tcl's home because Haiku's is not writable. Original : $old Substitute: $ld }] } } } }; # teaish__tcl_platform_quirks # # Searches $::argv and/or the build dir and/or the source dir for # teaish.tcl and friends. Fails if it cannot find teaish.tcl or if # there are other irreconcilable problems. If it returns 0 then it did # not find an extension but the --help flag was seen, in which case # that's not an error. # # This does not _load_ the extension, it primarily locates the files # which make up an extension and fills out no small amount of teaish # state related to that. # proc teaish__find_extension {} { proj-assert {!$::teaish__Config(install-mode)} teaish__verbose 1 msg-result "Looking for teaish extension..." # Helper for the foreach loop below. set checkTeaishTcl {{mustHave fid dir} { set f [file join $dir $fid] if {[file readable $f]} { file-normalize $f } elseif {$mustHave} { proj-error "Missing required $dir/$fid" } }} # # We have to handle some flags manually because the extension must # be loaded before [options] is run (so that the extension can # inject its own options). # set dirBld $::autosetup(builddir); # dir we're configuring under set dirSrc $::autosetup(srcdir); # where teaish's configure script lives set extT ""; # teaish.tcl set largv {}; # rewritten $::argv set gotHelpArg 0; # got the --help foreach arg $::argv { #puts "*** arg=$arg" switch -glob -- $arg { --ted=* - --t-e-d=* - --teaish-extension-dir=* { # Ensure that $extD refers to a directory and contains a # teaish.tcl. regexp -- {--[^=]+=(.+)} $arg - extD set extD [file-normalize $extD] if {![file isdirectory $extD]} { proj-error "--teaish-extension-dir value is not a directory: $extD" } set extT [apply $checkTeaishTcl 0 teaish.config $extD] if {"" eq $extT} { set extT [apply $checkTeaishTcl 1 teaish.tcl $extD] } set ::teaish__Config(extension-dir) $extD } --help { incr gotHelpArg lappend largv $arg } default { lappend largv $arg } } } set ::argv $largv set dirExt $::teaish__Config(extension-dir); # dir with the extension # # teaish.tcl is a TCL script which implements various # interfaces described by this framework. # # We use the first one we find in the builddir or srcdir. # if {"" eq $extT} { set flist [list] proj-assert {$dirExt eq ""} lappend flist $dirBld/teaish.tcl $dirBld/teaish.config $dirSrc/teaish.tcl if {![proj-first-file-found extT $flist]} { if {$gotHelpArg} { # Tell teaish-configure-core that the lack of extension is not # an error when --help or --teaish-install is used. return 0; } proj-indented-notice -error " Did not find any of: $flist If you are attempting an out-of-tree build, use --teaish-extension-dir=/path/to/extension" } } if {![file readable $extT]} { proj-error "extension tcl file is not readable: $extT" } set ::teaish__Config(teaish.tcl) $extT set dirExt [file dirname $extT] set ::teaish__Config(extension-dir) $dirExt set ::teaish__Config(blddir-is-extdir) [expr {$dirBld eq $dirExt}] set ::teaish__Config(dist-enabled) $::teaish__Config(blddir-is-extdir); # may change later set ::teaish__Config(dist-full-enabled) \ [expr {[file-normalize $::autosetup(srcdir)] eq [file-normalize $::teaish__Config(extension-dir)]}] set addDist {{file} { teaish-dist-add [file tail $file] }} apply $addDist $extT teaish__verbose 1 msg-result "Extension dir = [teaish-get -dir]" teaish__verbose 1 msg-result "Extension config = $extT" teaish-pkginfo-set -name [file tail [file dirname $extT]] # # teaish.make[.in] provides some of the info for the main makefile, # like which source(s) to build and their build flags. # # We use the first one of teaish.make.in or teaish.make we find in # $dirExt. # if {[proj-first-file-found extM \ [list \ $dirExt/teaish.make.in \ $dirExt/teaish.make \ ]]} { if {[string match *.in $extM]} { define TEAISH_MAKEFILE_IN $extM define TEAISH_MAKEFILE [file rootname [file tail $extM]] } else { define TEAISH_MAKEFILE_IN "" define TEAISH_MAKEFILE $extM } apply $addDist $extM teaish__verbose 1 msg-result "Extension makefile = $extM" } else { define TEAISH_MAKEFILE_IN "" define TEAISH_MAKEFILE "" } # Look for teaish.pkginit.tcl[.in] set piPolicy 0 if {[proj-first-file-found extI \ [list \ $dirExt/teaish.pkginit.tcl.in \ $dirExt/teaish.pkginit.tcl \ ]]} { if {[string match *.in $extI]} { # Generate teaish.pkginit.tcl from $extI. define TEAISH_PKGINIT_TCL_IN $extI define TEAISH_PKGINIT_TCL [file rootname [file tail $extI]] set piPolicy 0x01 } else { # Assume static $extI. define TEAISH_PKGINIT_TCL_IN "" define TEAISH_PKGINIT_TCL $extI set piPolicy 0x10 } apply $addDist $extI teaish__verbose 1 msg-result "Extension post-load init = $extI" define TEAISH_PKGINIT_TCL_TAIL \ [file tail [get-define TEAISH_PKGINIT_TCL]]; # for use in pkgIndex.tcl.in } set ::teaish__Config(pkginit-policy) $piPolicy # Look for pkgIndex.tcl[.in]... set piPolicy 0 if {[proj-first-file-found extPI $dirExt/pkgIndex.tcl.in]} { # Generate ./pkgIndex.tcl from $extPI. define TEAISH_PKGINDEX_TCL_IN $extPI define TEAISH_PKGINDEX_TCL [file rootname [file tail $extPI]] apply $addDist $extPI set piPolicy 0x01 } elseif {$dirExt ne $dirSrc && [proj-first-file-found extPI $dirSrc/pkgIndex.tcl.in]} { # Generate ./pkgIndex.tcl from $extPI. define TEAISH_PKGINDEX_TCL_IN $extPI define TEAISH_PKGINDEX_TCL [file rootname [file tail $extPI]] set piPolicy 0x02 } elseif {[proj-first-file-found extPI $dirExt/pkgIndex.tcl]} { # Assume $extPI's a static file and use it. define TEAISH_PKGINDEX_TCL_IN "" define TEAISH_PKGINDEX_TCL $extPI apply $addDist $extPI set piPolicy 0x10 } # Reminder: we have to delay removal of stale TEAISH_PKGINDEX_TCL # and the proj-dot-ins-append of TEAISH_PKGINDEX_TCL_IN until much # later in the process. set ::teaish__Config(pkgindex-policy) $piPolicy # Look for teaish.test.tcl[.in] proj-assert {"" ne $dirExt} set flist [list $dirExt/teaish.test.tcl.in $dirExt/teaish.test.tcl] if {[proj-first-file-found ttt $flist]} { if {[string match *.in $ttt]} { # Generate teaish.test.tcl from $ttt set xt [file rootname [file tail $ttt]] file delete -force -- $xt; # ensure no stale copy is used define TEAISH_TEST_TCL $xt define TEAISH_TEST_TCL_IN $ttt } else { define TEAISH_TEST_TCL $ttt define TEAISH_TEST_TCL_IN "" } apply $addDist $ttt } else { define TEAISH_TEST_TCL "" define TEAISH_TEST_TCL_IN "" } # Look for _teaish.tester.tcl[.in] set flist [list $dirExt/_teaish.tester.tcl.in $dirSrc/_teaish.tester.tcl.in] if {[proj-first-file-found ttt $flist]} { # Generate teaish.test.tcl from $ttt set xt [file rootname [file tail $ttt]] file delete -force -- $xt; # ensure no stale copy is used define TEAISH_TESTER_TCL $xt define TEAISH_TESTER_TCL_IN $ttt if {[lindex $flist 0] eq $ttt} { apply $addDist $ttt } unset ttt xt } else { if {[file exists [set ttt [file join $dirSrc _teaish.tester.tcl.in]]]} { set xt [file rootname [file tail $ttt]] define TEAISH_TESTER_TCL $xt define TEAISH_TESTER_TCL_IN $ttt } else { define TEAISH_TESTER_TCL "" define TEAISH_TESTER_TCL_IN "" } } unset flist # TEAISH_OUT_OF_EXT_TREE = 1 if we're building from a dir other # than the extension's home dir. define TEAISH_OUT_OF_EXT_TREE \ [expr {[file-normalize $::autosetup(builddir)] ne \ [file-normalize $::teaish__Config(extension-dir)]}] return 1 }; # teaish__find_extension # # @teaish-cflags-add ?-p|prepend? ?-define? cflags... # # Equivalent to [proj-define-amend TEAISH_CFLAGS {*}$args]. # proc teaish-cflags-add {args} { proj-define-amend TEAISH_CFLAGS {*}$args } # # @teaish-define-to-cflag ?flags? defineName...|{defineName...} # # Uses [proj-define-to-cflag] to expand a list of [define] keys, each # one a separate argument, to CFLAGS-style -D... form then appends # that to the current TEAISH_CFLAGS. # # It accepts these flags from proj-define-to-cflag: -quote, # -zero-undef. It does _not_ support its -list flag. # # It accepts its non-flag argument(s) in 2 forms: (1) each arg is a # single [define] key or (2) its one arg is a list of such keys. # # TODO: document teaish's well-defined (as it were) defines for this # purpose. At a bare minimum: # # - TEAISH_NAME # - TEAISH_PKGNAME # - TEAISH_VERSION # - TEAISH_LIBDIR_NAME # - TEAISH_LOAD_PREFIX # - TEAISH_URL # proc teaish-define-to-cflag {args} { set flags {} while {[string match -* [lindex $args 0]]} { set arg [lindex $args 0] switch -exact -- $arg { -quote - -zero-undef { lappend flags $arg set args [lassign $args -] } default break } } if {1 == [llength $args]} { set args [list {*}[lindex $args 0]] } #puts "***** flags=$flags args=$args" teaish-cflags-add [proj-define-to-cflag {*}$flags {*}$args] } # # @teaish-cflags-for-tea ?...CFLAGS? # # Adds several -DPACKAGE_... CFLAGS using the extension's metadata, # all as quoted strings. Those symbolic names are commonly used in # TEA-based builds, and this function is intended to simplify porting # of such builds. The -D... flags added are: # # -DPACKAGE_VERSION=... # -DPACKAGE_NAME=... # -DPACKAGE_URL=... # -DPACKAGE_STRING=... # # Any arguments are passed-on as-is to teaish-cflags-add. # proc teaish-cflags-for-tea {args} { set name $::teaish__PkgInfo(-name) set version $::teaish__PkgInfo(-version) set pstr [join [list $name $version]] teaish-cflags-add \ {*}$args \ '-DPACKAGE_VERSION="$version"' \ '-DPACKAGE_NAME="$name"' \ '-DPACKAGE_STRING="$pstr"' \ '-DPACKAGE_URL="[teaish-get -url]"' } # # @teaish-ldflags-add ?-p|-prepend? ?-define? ldflags... # # Equivalent to [proj-define-amend TEAISH_LDFLAGS {*}$args]. # # Typically, -lXYZ flags need to be in "reverse" order, with each -lY # resolving symbols for -lX's to its left. This order is largely # historical, and not relevant on all environments, but it is # technically correct and still relevant on some environments. # # See: teaish-ldflags-prepend # proc teaish-ldflags-add {args} { proj-define-amend TEAISH_LDFLAGS {*}$args } # # @teaish-ldflags-prepend args... # # Functionally equivalent to [teaish-ldflags-add -p {*}$args] # proc teaish-ldflags-prepend {args} { teaish-ldflags-add -p {*}$args } # # @teaish-src-add ?-dist? ?-dir? src-files... # # Appends all non-empty $args to the project's list of C/C++ source or # (in some cases) object files. # # If passed -dist then it also passes each filename, as-is, to # [teaish-dist-add]. # # If passed -dir then each src-file has [teaish-get -dir] prepended to # it before they're added to the list. As often as not, that will be # the desired behavior so that out-of-tree builds can find the # sources, but there are cases where it's not desired (e.g. when using # a source file from outside of the extension's dir, or when adding # object files (which are typically in the build tree)). # proc teaish-src-add {args} { set i 0 proj-parse-simple-flags args flags { -dist 0 {expr 1} -dir 0 {expr 1} } if {$flags(-dist)} { teaish-dist-add {*}$args } if {$flags(-dir)} { set xargs {} foreach arg $args { if {"" ne $arg} { lappend xargs [file join $::teaish__Config(extension-dir) $arg] } } set args $xargs } lappend ::teaish__Config(extension-src) {*}$args } # # @teaish-dist-add files-or-dirs... # # Adds the given files to the list of files to include with the "make # dist" rules. # # This is a no-op when the current build is not in the extension's # directory, as dist support is disabled in out-of-tree builds. # # It is not legal to call this until [teaish-get -dir] has been # reliably set (via teaish__find_extension). # proc teaish-dist-add {args} { if {$::teaish__Config(blddir-is-extdir)} { # ^^^ reminder: we ignore $::teaish__Config(dist-enabled) here # because the client might want to implement their own dist # rules. #proj-warn "**** args=$args" lappend ::teaish__Config(dist-files) {*}$args } } # teaish-install-add files... # Equivalent to [proj-define-apend TEAISH_INSTALL_FILES ...]. #proc teaish-install-add {args} { # proj-define-amend TEAISH_INSTALL_FILES {*}$args #} # # @teash-make-add args... # # Appends makefile code to the TEAISH_MAKEFILE_CODE define. Each # arg may be any of: # # -tab: emit a literal tab # -nl: emit a literal newline # -nltab: short for -nl -tab # -bnl: emit a backslash-escaped end-of-line # -bnltab: short for -eol -tab # # Anything else is appended verbatim. This function adds no additional # spacing between each argument nor between subsequent invocations. # Generally speaking, a series of calls to this function need to # be sure to end the series with a newline. proc teaish-make-add {args} { set out [get-define TEAISH_MAKEFILE_CODE ""] foreach a $args { switch -exact -- $a { -bnl { set a " \\\n" } -bnltab { set a " \\\n\t" } -tab { set a "\t" } -nl { set a "\n" } -nltab { set a "\n\t" } } append out $a } define TEAISH_MAKEFILE_CODE $out } # Internal helper to generate a clean/distclean rule name proc teaish__cleanup_rule {{tgt clean}} { set x [incr ::teaish__Config(teaish__cleanup_rule-counter-${tgt})] return ${tgt}-_${x}_ } # @teaish-make-obj objfile srcfile ?...args? # # Uses teaish-make-add to inject makefile rules for $objfile from # $srcfile, which is assumed to be C code which uses libtcl. Unless # -recipe is used (see below) it invokes the compiler using the # makefile-defined $(CC.tcl) which, in the default Makefile.in # template, includes any flags needed for building against the # configured Tcl. # # This always terminates the resulting code with a newline. # # Any arguments after the 2nd may be flags described below or, if no # -recipe is provided, flags for the compiler call. # # -recipe {...} # Uses the trimmed value of {...} as the recipe, prefixing it with # a single hard-tab character. # # -deps {...} # List of extra files to list as dependencies of $o. Good luck # escaping non-trivial cases properly. # # -clean # Generate cleanup rules as well. proc teaish-make-obj {o src args} { set consume 0 set clean 0 set flag "" array set flags {} set xargs {} foreach arg $args { if {$consume} { set consume 0 set flags($flag) $arg continue } switch -exact -- $arg { -clean {incr clean} -recipe - -deps { set flag $arg incr consume } default { lappend xargs $arg } } } teaish-make-add \ "# [proj-scope 1] -> [proj-scope] $o $src" -nl \ "$o: $src $::teaish__Config(teaish.tcl)" if {[info exists flags(-deps)]} { teaish-make-add " " [join $flags(-deps)] } teaish-make-add -nltab if {[info exists flags(-recipe)]} { teaish-make-add [string trim $flags(-recipe)] -nl } else { teaish-make-add [join [list \$(CC.tcl) -c $src {*}$xargs]] -nl } if {$clean} { set rule [teaish__cleanup_rule] teaish-make-add \ "clean: $rule\n$rule:\n\trm -f \"$o\"\n" } } # # @teaish-make-clean ?-r? ?-dist? ...files|{...files} # # Adds makefile rules for cleaning up the given files via the "make # clean" or (if -dist is used) "make distclean" makefile rules. The -r # flag uses "rm -fr" instead of "rm -f", so be careful with that. # # The file names are taken literally as arguments to "rm", so they may # be shell wildcards to be resolved at cleanup-time. To clean up whole # directories, pass the -r flag. Each name gets quoted in # double-quotes, so spaces in names should not be a problem (but # double-quotes in names will be). # proc teaish-make-clean {args} { if {1 == [llength $args]} { set args [list {*}[lindex $args 0]] } set tgt clean set rmflags "-f" proj-parse-simple-flags args flags { -dist 0 { set tgt distclean } -r 0 { set rmflags "-fr" } } set rule [teaish__cleanup_rule $tgt] teaish-make-add "# [proj-scope 1] -> [proj-scope]: [join $args]\n" teaish-make-add "${rule}:\n\trm ${rmflags}" foreach a $args { teaish-make-add " \"$a\"" } teaish-make-add "\n${tgt}: ${rule}\n" } # # @teaish-make-config-header filename # # Invokes autosetup's [make-config-header] and passes it $filename and # a relatively generic list of options for controlling which defined # symbols get exported. Clients which need more control over the # exports can copy/paste/customize this. # # The exported file is then passed to [proj-touch] because, in # practice, that's sometimes necessary to avoid build dependency # issues. # proc teaish-make-config-header {filename} { make-config-header $filename \ -none {HAVE_CFLAG_* LDFLAGS_* SH_* TEAISH__* TEAISH_*_CODE} \ -auto {SIZEOF_* HAVE_* TEAISH_* TCL_*} \ -none * proj-touch $filename; # help avoid frequent unnecessary auto-reconfig } # # @teaish-feature-cache-set $key value # # Sets a feature-check cache entry with the given key. # See proj-cache-set for the key's semantics. $key should # normally be 0. # proc teaish-feature-cache-set {key val} { proj-cache-set -key $key -level 1 $val } # # @teaish-feature-cache-check key tgtVarName # # Checks for a feature-check cache entry with the given key. # See proj-cache-set for the key's semantics. # # $key should also almost always be 0 but, due to a tclsh # incompatibility in 1 OS, it cannot have a default value unless it's # the second argument (but it should be the first one). # # If the feature-check cache has a matching entry then this function # assigns its value to tgtVar and returns 1, else it assigns tgtVar to # "" and returns 0. # # See proj-cache-check for $key's semantics. # proc teaish-feature-cache-check {key tgtVar} { upvar $tgtVar tgt proj-cache-check -key $key -level 1 tgt } # # @teaish-check-cached@ ?flags? msg script... # # A proxy for feature-test impls which handles caching of a feature # flag check on per-function basis, using the calling scope's name as # the cache key. # # It emits [msg-checking $msg]. If $msg is empty then it defaults to # the name of the caller's scope. The -nomsg flag suppresses the # message for non-cache-hit checks. At the end, it will [msg-result # "ok"] [msg-result "no"] unless -nostatus is used, in which case the # caller is responsible for emitting at least a newline when it's # done. The -msg-0 and -msg-1 flags can be used to change the ok/no # text. # # This function checks for a cache hit before running $script and # caching the result. If no hit is found then $script is run in the # calling scope and its result value is stored in the cache. This # routine will intercept a 'return' from $script. # # $script may be a command and its arguments, as opposed to a single # script block. # # Flags: # # -nostatus = do not emit "ok" or "no" at the end. This presumes # that either $script will emit at least one newline before # returning or the caller will account for it. Because of how this # function is typically used, -nostatus is not honored when the # response includes a cached result. # # -quiet = disable output from Autosetup's msg-checking and # msg-result for the duration of the $script check. Note that when # -quiet is in effect, Autosetup's user-notice can be used to queue # up output to appear after the check is done. Also note that # -quiet has no effect on _this_ function, only the $script part. # # -nomsg = do not emit $msg for initial check. Like -nostatus, this # flag is not honored when the response includes a cached result # because it would otherwise produce no output (which is confusing # in this context). This is useful when a check runs several other # verbose checks and they emit all the necessary info. # # -msg-0 and -msg-1 MSG = strings to show when the check has failed # resp. passed. Defaults are "no" and "ok". The 0 and 1 refer to the # result value from teaish-feature-cache-check. # # -key cachekey = set the cache context key. Only needs to be # explicit when using this function multiple times from a single # scope. See proj-cache-check and friends for details on the key # name. Its default is the name of the scope which calls this # function. # proc teaish-check-cached {args} { proj-parse-simple-flags args flags { -nostatus 0 {expr 1} -quiet 0 {expr 1} -key => 1 -nomsg 0 {expr 1} -msg-0 => no -msg-1 => ok } set args [lassign $args msg] set script [join $args] if {"" eq $msg} { set msg [proj-scope 1] } if {[teaish-feature-cache-check $flags(-key) check]} { #if {0 == $flags(-nomsg)} { msg-checking "${msg} ... (cached) " #} #if {!$flags(-nostatus)} { msg-result $flags(-msg-[expr {0 != ${check}}]) #} return $check } else { if {0 == $flags(-nomsg)} { msg-checking "${msg} ... " } if {$flags(-quiet)} { incr ::autosetup(msg-quiet) } set code [catch {uplevel 1 $script} rc xopt] if {$flags(-quiet)} { incr ::autosetup(msg-quiet) -1 } #puts "***** cached-check got code=$code rc=$rc" if {$code in {0 2}} { teaish-feature-cache-set 1 $rc if {!$flags(-nostatus)} { msg-result $flags(-msg-[expr {0 != ${rc}}]) } else { #show-notices; # causes a phantom newline because we're in a #msg-checking scope, so... if {[info exists ::autosetup(notices)]} { show-notices } } } else { #puts "**** code=$code rc=$rc xopt=$xopt" teaish-feature-cache-set 1 0 } #puts "**** code=$code rc=$rc" return {*}$xopt $rc } } # # Internal helper for teaish__defs_format_: returns a JSON-ish quoted # form of the given string-type values. # # If $asList is true then the return value is in {$value} form. If # $asList is false it only performs the most basic of escaping and # the input must not contain any control characters. # proc teaish__quote_str {asList value} { if {$asList} { return "{${value}}" } return \"[string map [list \\ \\\\ \" \\\"] $value]\" } # # Internal helper for teaish__defines_to_list. Expects to be passed # a name and the variadic $args which are passed to # teaish__defines_to_list.. If it finds a pattern match for the # given $name in the various $args, it returns the type flag for that # $name, e.g. "-str" or "-bare", else returns an empty string. # proc teaish__defs_type {name spec} { foreach {type patterns} $spec { foreach pattern $patterns { if {[string match $pattern $name]} { return $type } } } return "" } # # An internal impl detail. Requires a data type specifier, as used by # Autosetup's [make-config-header], and a value. Returns the formatted # value or the value $::teaish__Config(defs-skip) if the caller should # skip emitting that value. # # In addition to -str, -auto, etc., as defined by make-config-header, # it supports: # # -list {...} will cause non-integer values to be quoted in {...} # instead of quotes. # # -autolist {...} works like -auto {...} except that it falls back to # -list {...} type instead of -str {...} style for non-integers. # # -jsarray {...} emits the output in something which, for # conservative inputs, will be a valid JSON array. It can only # handle relatively simple values with no control characters in # them. # set teaish__Config(defs-skip) "-teaish__defs_format sentinel" proc teaish__defs_format {type value} { switch -exact -- $type { -bare { # Just output the value unchanged } -none { set value $::teaish__Config(defs-skip) } -str { set value [teaish__quote_str 0 $value] } -auto { # Automatically determine the type if {![string is integer -strict $value]} { set value [teaish__quote_str 0 $value] } } -autolist { if {![string is integer -strict $value]} { set value [teaish__quote_str 1 $value] } } -list { set value [teaish__quote_str 1 $value] } -jsarray { set ar {} foreach v $value { if {![string is integer -strict $v]} { set v [teaish__quote_str 0 $v] } if {$::teaish__Config(defs-skip) ne $v} { lappend ar $v } } set value [concat \[ [join $ar {, }] \]] } "" { # (Much later:) Why do we do this? set value $::teaish__Config(defs-skip) } default { proj-error \ "Unknown [proj-scope] -type ($type) called from" \ [proj-scope 1] } } return $value } # # Returns Tcl code in the form of code which evaluates to a list of # configure-time DEFINEs in the form {key val key2 val...}. It may # misbehave for values which are not numeric or simple strings. Some # defines are specifically filtered out of the result, either because # their irrelevant to teaish or because they may be arbitrarily large # (e.g. makefile content). # # The $args are explained in the docs for internal-use-only # [teaish__defs_format]. The default mode is -autolist. # proc teaish__defines_to_list {args} { set lines {} lappend lines "\{" set skipper $::teaish__Config(defs-skip) set args [list \ -none { TEAISH__* TEAISH_*_CODE AM_* AS_* } \ {*}$args \ -autolist *] foreach d [lsort [dict keys [all-defines]]] { set type [teaish__defs_type $d $args] set value [teaish__defs_format $type [get-define $d]] if {$skipper ne $value} { lappend lines "$d $value" } } lappend lines "\}" tailcall join $lines "\n" } # # teaish__pragma ...flags # # Offers a way to tweak how teaish's core behaves in some cases, in # particular those which require changing how the core looks for an # extension and its files. # # Accepts the following flags. Those marked with [L] are safe to use # during initial loading of tclish.tcl (recall that most teaish APIs # cannot be used until [teaish-configure] is called). # # static-pkgIndex.tcl [L]: Tells teaish that ./pkgIndex.tcl is not # a generated file, so it will not try to overwrite or delete # it. Errors out if it does not find pkgIndex.tcl in the # extension's dir. # # no-dist [L]: tells teaish to elide the 'make dist' recipe # from the generated Makefile. # # no-dll [L]: tells teaish to elide the DLL-building recipe # from the generated Makefile. # # no-vsatisfies-error [L]: tells teaish that failure to match the # -vsatisfies value should simply "return" instead of "error". # # no-tester [L]: disables automatic generation of teaish.test.tcl # even if a copy of _teaish.tester.tcl.in is found. # # no-full-dist [L]: changes the "make dist" rules to never include # a copy of teaish itself. By default it will include itself only # if the extension lives in the same directory as teaish. # # full-dist [L]: changes the "make dist" rules to always include # a copy of teaish itself. # # Emits a warning message for unknown arguments. # proc teaish__pragma {args} { foreach arg $args { switch -exact -- $arg { static-pkgIndex.tcl { if {$::teaish__Config(tm-policy)} { proj-fatal -up "Cannot use pragma $arg together with -tm.tcl or -tm.tcl.in." } set tpi [file join $::teaish__Config(extension-dir) pkgIndex.tcl] if {[file exists $tpi]} { define TEAISH_PKGINDEX_TCL_IN "" define TEAISH_PKGINDEX_TCL $tpi set ::teaish__Config(pkgindex-policy) 0x20 } else { proj-error "pragma $arg: found no package-local pkgIndex.tcl\[.in]" } } no-dist { set ::teaish__Config(dist-enabled) 0 } no-install { set ::teaish__Config(install-enabled) 0 } full-dist { set ::teaish__Config(dist-full-enabled) 1 } no-full-dist { set ::teaish__Config(dist-full-enabled) 0 } no-dll { set ::teaish__Config(dll-enabled) 0 } no-vsatisfies-error { set ::teaish__Config(vsatisfies-error) 0 } no-tester { define TEAISH_TESTER_TCL_IN "" define TEAISH_TESTER_TCL "" } default { proj-error "Unknown flag: $arg" } } } } # # @teaish-pkginfo-set ...flags # # The way to set up the initial package state. Used like: # # teaish-pkginfo-set -name foo -version 0.1.2 # # Or: # # teaish-pkginfo-set ?-vars|-subst? {-name foo -version 0.1.2} # # The latter may be easier to write for a multi-line invocation. # # For the second call form, passing the -vars flag tells it to perform # a [subst] of (only) variables in the {...} part from the calling # scope. The -subst flag will cause it to [subst] the {...} with # command substitution as well (but no backslash substitution). When # using -subst for string concatenation, e.g. with -libDir # foo[get-version-number], be sure to wrap the value in braces: # -libDir {foo[get-version-number]}. # # Each pkginfo flag corresponds to one piece of extension package # info. Teaish provides usable default values for all of these flags, # but at least the -name and -version should be set by clients. # e.g. the default -name is the directory name the extension lives in, # which may change (e.g. when building it from a "make dist" bundle). # # The flags: # # -name theName: The extension's name. It defaults to the name of the # directory containing the extension. (In TEA this would be the # PACKAGE_NAME, not to be confused with...) # # -name.pkg pkg-provide-name: The extension's name for purposes of # Tcl_PkgProvide(), [package require], and friends. It defaults to # the `-name`, and is normally the same, but some projects (like # SQLite) have a different name here than they do in their # historical TEA PACKAGE_NAME. # # -version version: The extension's package version. Defaults to # 0.0.0. # # -libDir dirName: The base name of the directory into which this # extension should be installed. It defaults to a concatenation of # `-name.pkg` and `-version`. # # -loadPrefix prefix: For use as the second argument passed to # Tcl's `load` command in the package-loading process. It defaults # to title-cased `-name.pkg` because Tcl's `load` plugin system # expects it in that form. # # -options {...}: If provided, it must be a list compatible with # Autosetup's `options-add` function. These can also be set up via # `teaish-options`. # # -vsatisfies {{...} ...}: Expects a list-of-lists of conditions # for Tcl's `package vsatisfies` command: each list entry is a # sub-list of `{PkgName Condition...}`. Teaish inserts those # checks via its default pkgIndex.tcl.in and _teaish.tester.tcl.in # templates to verify that the system's package dependencies meet # these requirements. The default value is `{{Tcl 8.5-}}` (recall # that it's a list-of-lists), as 8.5 is the minimum Tcl version # teaish will run on, but some extensions may require newer # versions or dependencies on other packages. As a special case, # if `-vsatisfies` is given a single token, e.g. `8.6-`, then it # is transformed into `{Tcl $thatToken}`, i.e. it checks the Tcl # version which the package is being run with. If given multiple # lists, each `package provides` check is run in the given # order. Failure to meet a `vsatisfies` condition triggers an # error. # # -url {...}: an optional URL for the extension. # # -pragmas {...} A list of infrequently-needed lower-level # directives which can influence teaish, including: # # static-pkgIndex.tcl: tells teaish that the client manages their # own pkgIndex.tcl, so that teaish won't try to overwrite it # using a template. # # no-dist: tells teaish to elide the "make dist" recipe from the # makefile so that the client can implement it. # # no-dll: tells teaish to elide the makefile rules which build # the DLL, as well as any templated test script and pkgIndex.tcl # references to them. The intent here is to (A) support # client-defined build rules for the DLL and (B) eventually # support script-only extensions. # # Unsupported flags or pragmas will trigger an error. # # Potential pothole: setting certain state, e.g. -version, after the # initial call requires recalculating of some [define]s. Any such # changes should be made as early as possible in teaish-configure so # that any later use of those [define]s gets recorded properly (not # with the old value). This is particularly relevant when it is not # possible to determine the -version or -name until teaish-configure # has been called, and it's updated dynamically from # teaish-configure. Notably: # # - If -version or -name are updated, -libDir will almost certainly # need to be explicitly set along with them. # # - If -name is updated, -loadPrefix probably needs to be as well. # proc teaish-pkginfo-set {args} { set doVars 0 set doCommands 0 set xargs $args set recalc {} foreach arg $args { switch -exact -- $arg { -vars { incr doVars set xargs [lassign $xargs -] } -subst { incr doVars incr doCommands set xargs [lassign $xargs -] } default { break } } } set args $xargs unset xargs if {1 == [llength $args] && [llength [lindex $args 0]] > 1} { # Transform a single {...} arg into the canonical call form set a [list {*}[lindex $args 0]] if {$doVars || $doCommands} { set sflags -nobackslashes if {!$doCommands} { lappend sflags -nocommands } set a [uplevel 1 [list subst {*}$sflags $a]] } set args $a } set sentinel "<nope>" set flagDefs [list] foreach {f d} $::teaish__Config(pkginfo-f2d) { lappend flagDefs $f => $sentinel } proj-parse-simple-flags args flags $flagDefs if {[llength $args]} { proj-error -up "Too many (or unknown) arguments to [proj-scope]: $args" } foreach {f d} $::teaish__Config(pkginfo-f2d) { if {$sentinel eq [set v $flags($f)]} continue switch -exact -- $f { -options { proj-assert {"" eq $d} options-add $v } -pragmas { teaish__pragma {*}$v } -vsatisfies { if {1 == [llength $v] && 1 == [llength [lindex $v 0]]} { # Transform X to {Tcl $X} set v [list [join [list Tcl $v]]] } define $d $v } -pkgInit.tcl - -pkgInit.tcl.in { if {0x22 & $::teaish__Config(pkginit-policy)} { proj-fatal "Cannot use -pkgInit.tcl(.in) more than once." } set x [file join $::teaish__Config(extension-dir) $v] set tTail [file tail $v] if {"-pkgInit.tcl.in" eq $f} { # Generate pkginit file X from X.in set pI 0x02 set tIn $x set tOut [file rootname $tTail] set other -pkgInit.tcl } else { # Static pkginit file X set pI 0x20 set tIn "" set tOut $x set other -pkgInit.tcl.in } set ::teaish__Config(pkginit-policy) $pI set ::teaish__PkgInfo($other) {} define TEAISH_PKGINIT_TCL_IN $tIn define TEAISH_PKGINIT_TCL $tOut define TEAISH_PKGINIT_TCL_TAIL $tTail teaish-dist-add $v set v $x } -tm.tcl - -tm.tcl.in { if {0x30 & $::teaish__Config(pkgindex-policy)} { proj-fatal "Cannot use $f together with a pkgIndex.tcl." } elseif {$::teaish__Config(tm-policy)} { proj-fatal "Cannot use -tm.tcl(.in) more than once." } set x [file join $::teaish__Config(extension-dir) $v] if {"-tm.tcl.in" eq $f} { # Generate tm file X from X.in set pT 0x02 set pI 0x100 set tIn $x set tOut [file rootname [file tail $v]] set other -tm.tcl } else { # Static tm file X set pT 0x20 set pI 0x200 set tIn "" set tOut $x set other -tm.tcl.in } set ::teaish__Config(pkgindex-policy) $pI set ::teaish__Config(tm-policy) $pT set ::teaish__PkgInfo($other) {} define TEAISH_TM_TCL_IN $tIn define TEAISH_TM_TCL $tOut define TEAISH_PKGINDEX_TCL "" define TEAISH_PKGINDEX_TCL_IN "" define TEAISH_PKGINDEX_TCL_TAIL "" teaish-dist-add $v teaish__pragma no-dll set v $x } default { proj-assert {"" ne $d} define $d $v } } set ::teaish__PkgInfo($f) $v if {$f in {-name -version -libDir -loadPrefix}} { lappend recalc $f } } if {"" ne $recalc} { teaish__define_pkginfo_derived $recalc } } # # @teaish-pkginfo-get ?arg? # # If passed no arguments, it returns the extension config info in the # same form accepted by teaish-pkginfo-set. # # If passed one -flagname arg then it returns the value of that config # option. # # Else it treats arg as the name of caller-scoped variable to # which this function assigns an array containing the configuration # state of this extension, in the same structure accepted by # teaish-pkginfo-set. In this case it returns an empty string. # proc teaish-pkginfo-get {args} { set cases {} set argc [llength $args] set rv {} switch -exact $argc { 0 { # Return a list of (-flag value) pairs lappend cases default {{ if {[info exists ::teaish__PkgInfo($flag)]} { lappend rv $flag $::teaish__PkgInfo($flag) } else { lappend rv $flag [get-define $defName] } }} } 1 { set arg $args if {[string match -* $arg]} { # Return the corresponding -flag's value lappend cases $arg {{ if {[info exists ::teaish__PkgInfo($flag)]} { return $::teaish__PkgInfo($flag) } else { return [get-define $defName] } }} } else { # Populate target with an array of (-flag value). upvar $arg tgt array set tgt {} lappend cases default {{ if {[info exists ::teaish__PkgInfo($flag)]} { set tgt($flag) $::teaish__PkgInfo($flag) } else { set tgt($flag) [get-define $defName] } }} } } default { proj-error "invalid arg count from [proj-scope 1]" } } foreach {flag defName} $::teaish__Config(pkginfo-f2d) { switch -exact -- $flag [join $cases] } if {0 == $argc} { return $rv } } # (Re)set some defines based on pkginfo state. $flags is the list of # pkginfo -flags which triggered this, or "*" for the initial call. proc teaish__define_pkginfo_derived {flags} { set all [expr {{*} in $flags}] if {$all || "-version" in $flags || "-name" in $flags} { set name $::teaish__PkgInfo(-name) ; # _not_ -name.pkg if {[info exists ::teaish__PkgInfo(-version)]} { set pkgver $::teaish__PkgInfo(-version) set libname "lib" if {[string match *-cygwin [get-define host]]} { set libname cyg } define TEAISH_DLL8_BASENAME $libname$name$pkgver define TEAISH_DLL9_BASENAME ${libname}tcl9$name$pkgver set ext [get-define TARGET_DLLEXT] define TEAISH_DLL8 [get-define TEAISH_DLL8_BASENAME]$ext define TEAISH_DLL9 [get-define TEAISH_DLL9_BASENAME]$ext } } if {$all || "-libDir" in $flags} { if {[info exists ::teaish__PkgInfo(-libDir)]} { define TCLLIBDIR \ [file dirname [get-define TCLLIBDIR]]/$::teaish__PkgInfo(-libDir) } } } # # @teaish-checks-queue -pre|-post args... # # Queues one or more arbitrary "feature test" functions to be run when # teaish-checks-run is called. $flag must be one of -pre or -post to # specify whether the tests should be run before or after # teaish-configure is run. Each additional arg is the name of a # feature-test proc. # proc teaish-checks-queue {flag args} { if {$flag ni {-pre -post}} { proj-error "illegal flag: $flag" } lappend ::teaish__Config(queued-checks${flag}) {*}$args } # # @teaish-checks-run -pre|-post # # Runs all feature checks queued using teaish-checks-queue # then cleares the queue. # proc teaish-checks-run {flag} { if {$flag ni {-pre -post}} { proj-error "illegal flag: $flag" } #puts "*** running $flag: $::teaish__Config(queued-checks${flag})" set foo 0 foreach f $::teaish__Config(queued-checks${flag}) { if {![teaish-feature-cache-check $f foo]} { set v [$f] teaish-feature-cache-set $f $v } } set ::teaish__Config(queued-checks${flag}) {} } # # A general-purpose getter for various teaish state. Requires one # flag, which determines its result value. Flags marked with [L] below # are safe for using at load-time, before teaish-pkginfo-set is called # # -dir [L]: returns the extension's directory, which may differ from # the teaish core dir or the build dir. # # -teaish-home [L]: the "home" dir of teaish itself, which may # differ from the extension dir or build dir. # # -build-dir [L]: the build directory (typically the current working # -dir). # # Any of the teaish-pkginfo-get/get flags: returns the same as # teaish-pkginfo-get. Not safe for use until teaish-pkginfo-set has # been called. # # Triggers an error if passed an unknown flag. # proc teaish-get {flag} { #-teaish.tcl {return $::teaish__Config(teaish.tcl)} switch -exact -- $flag { -dir { return $::teaish__Config(extension-dir) } -teaish-home { return $::autosetup(srcdir) } -build-dir { return $::autosetup(builddir) } default { if {[info exists ::teaish__PkgInfo($flag)]} { return $::teaish__PkgInfo($flag) } } } proj-error "Unhandled flag: $flag" } # # Handles --teaish-create-extension=TARGET-DIR # proc teaish__create_extension {dir} { set force [opt-bool teaish-force] if {"" eq $dir} { proj-error "--teaish-create-extension=X requires a directory name." } file mkdir $dir/generic set cwd [pwd] #set dir [file-normalize [file join $cwd $dir]] teaish__verbose 1 msg-result "Created dir $dir" cd $dir if {!$force} { # Ensure that we don't blindly overwrite anything foreach f { generic/teaish.c teaish.tcl teaish.make.in teaish.test.tcl } { if {[file exists $f]} { error "Cowardly refusing to overwrite $dir/$f. Use --teaish-force to overwrite." } } } set name [file tail $dir] set pkgName $name set version 0.0.1 set loadPrefix [string totitle $pkgName] set content {teaish-pkginfo-set } #puts "0 content=$content" if {[opt-str teaish-extension-pkginfo epi]} { set epi [string trim $epi] if {[string match "*\n*" $epi]} { set epi "{$epi}" } elseif {![string match "{*}" $epi]} { append content "\{" $epi "\}" } else { append content $epi } #puts "2 content=$content\nepi=$epi" } else { append content [subst -nocommands -nobackslashes {{ -name ${name} -name.pkg ${pkgName} -name.dist ${pkgName} -version ${version} -loadPrefix $loadPrefix -libDir ${name}${version} -vsatisfies {{Tcl 8.5-}} -url {} -options {} -pragmas {full-dist} }}] #puts "3 content=$content" } #puts "1 content=$content" append content "\n" { #proc teaish-options {} { # Return a list and/or use \[options-add\] to add new # configure flags. This is called before teaish's # bootstrapping is finished, so only teaish-* # APIs which are explicitly noted as being safe # early on may be used here. Any autosetup-related # APIs may be used here. # # Return an empty string if there are no options to # add or if they are added using \[options-add\]. # # If there are no options to add, this proc need # not be defined. #} # Called by teaish once bootstrapping is complete. # This function is responsible for the client-specific # parts of the configuration process. proc teaish-configure {} { teaish-src-add -dir -dist generic/teaish.c teaish-define-to-cflag -quote TEAISH_PKGNAME TEAISH_VERSION # TODO: your code goes here.. } }; # $content proj-file-write teaish.tcl $content teaish__verbose 1 msg-result "Created teaish.tcl" set content "# Teaish test script. # When this tcl script is invoked via 'make test' it will have loaded # the package, run any teaish.pkginit.tcl code, and loaded # autosetup/teaish/tester.tcl. " proj-file-write teaish.test.tcl $content teaish__verbose 1 msg-result "Created teaish.test.tcl" set content [subst -nocommands -nobackslashes { #include <tcl.h> static int ${loadPrefix}_Cmd(ClientData cdata, Tcl_Interp *interp, int objc, Tcl_Obj *const objv[]){ Tcl_SetObjResult(interp, Tcl_NewStringObj("this is the ${name} extension", -1)); return TCL_OK; } extern int DLLEXPORT ${loadPrefix}_Init(Tcl_Interp *interp){ if (Tcl_InitStubs(interp, TCL_VERSION, 0) == NULL) { return TCL_ERROR; } if (Tcl_PkgProvide(interp, TEAISH_PKGNAME, TEAISH_VERSION) == TCL_ERROR) { return TCL_ERROR; } Tcl_CreateObjCommand(interp, TEAISH_PKGNAME, ${loadPrefix}_Cmd, NULL, NULL); return TCL_OK; } }] proj-file-write generic/teaish.c $content teaish__verbose 1 msg-result "Created generic/teaish.c" set content "# teaish makefile for the ${name} extension # tx.src = \$(tx.dir)/generic/teaish.c # tx.LDFLAGS = # tx.CFLAGS = " proj-file-write teaish.make.in $content teaish__verbose 1 msg-result "Created teaish.make.in" msg-result "Created new extension \[$dir\]." cd $cwd set ::teaish__Config(install-ext-dir) $dir } # # Internal helper for teaish__install # proc teaish__install_file {f destDir force} { set dest $destDir/[file tail $f] if {[file isdirectory $f]} { file mkdir $dest } elseif {!$force && [file exists $dest]} { array set st1 [file stat $f] array set st2 [file stat $dest] if {($st1(mtime) == $st2(mtime)) && ($st1(size) == $st2(size))} { if {[file tail $f] in { pkgIndex.tcl.in _teaish.tester.tcl.in }} { # Assume they're the same. In the scope of the "make dist" # rules, this happens legitimately when an extension with a # copy of teaish installed in the same dir assumes that the # pkgIndex.tcl.in and _teaish.tester.tcl.in belong to the # extension, whereas teaish believes they belong to teaish. # So we end up with dupes of those. return } } proj-error -up "Cowardly refusing to overwrite \[$dest\]." \ "Use --teaish-force to enable overwriting." } else { # file copy -force $f $destDir; # loses +x bit # # JimTcl doesn't have [file attribute], so we can't use that here # (in the context of an autosetup configure script). exec cp -p $f $dest } } # # Installs a copy of teaish, with autosetup, to $dDest, which defaults # to the --teaish-install=X or --teash-create-extension=X dir. Won't # overwrite files unless --teaish-force is used. # proc teaish__install {{dDest ""}} { if {$dDest in {auto ""}} { set dDest [opt-val teaish-install] if {$dDest in {auto ""}} { if {[info exists ::teaish__Config(install-ext-dir)]} { set dDest $::teaish__Config(install-ext-dir) } } } set force [opt-bool teaish-force] if {$dDest in {auto ""}} { proj-error "Cannot determine installation directory." } elseif {!$force && [file exists $dDest/auto.def]} { proj-error \ "Target dir looks like it already contains teaish and/or autosetup: $dDest" \ "\nUse --teaish-force to overwrite it." } set dSrc $::autosetup(srcdir) set dAS $::autosetup(libdir) set dAST $::teaish__Config(core-dir) set dASTF $dAST/feature teaish__verbose 1 msg-result "Installing teaish to \[$dDest\]..." if {$::teaish__Config(verbose)>1} { msg-result "dSrc = $dSrc" msg-result "dAS = $dAS" msg-result "dAST = $dAST" msg-result "dASTF = $dASTF" msg-result "dDest = $dDest" } # Dest subdirs... set ddAS $dDest/autosetup set ddAST $ddAS/teaish set ddASTF $ddAST/feature foreach {srcDir destDir} [list \ $dAS $ddAS \ $dAST $ddAST \ $dASTF $ddASTF \ ] { teaish__verbose 1 msg-result "Copying files to $destDir..." file mkdir $destDir foreach f [glob -directory $srcDir *] { if {[string match {*~} $f] || [string match "#*#" [file tail $f]]} { # Editor-generated backups and emacs lock files continue } teaish__verbose 2 msg-result "\t$f" teaish__install_file $f $destDir $force } } teaish__verbose 1 msg-result "Copying files to $dDest..." foreach f { auto.def configure Makefile.in pkgIndex.tcl.in _teaish.tester.tcl.in } { teaish__verbose 2 msg-result "\t$f" teaish__install_file $dSrc/$f $dDest $force } set ::teaish__Config(install-self-dir) $dDest msg-result "Teaish $::teaish__Config(version) installed in \[$dDest\]." } |
Added autosetup/teaish/feature.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 | ######################################################################## # 2025 April 7 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # * May you do good and not evil. # * May you find forgiveness for yourself and forgive others. # * May you share freely, never taking more than you give. # ######################################################################## # ----- @module feature-tests.tcl ----- # @section TEA-ish collection of feature tests. # # Functions in this file with a prefix of teaish__ are # private/internal APIs. Those with a prefix of teaish- are # public APIs. # @teaish-check-libz # # Checks for zlib.h and the function deflate in libz. If found, # prepends -lz to the extension's ldflags and returns 1, else returns # 0. It also defines LDFLAGS_LIBZ to the libs flag. # proc teaish-check-libz {} { teaish-check-cached "Checking for libz" { set rc 0 if {[msg-quiet cc-check-includes zlib.h] && [msg-quiet proj-check-function-in-lib deflate z]} { teaish-ldflags-prepend [define LDFLAGS_LIBZ [get-define lib_deflate]] undefine lib_deflate incr rc } expr $rc } } # @teaish-check-librt ?funclist? # # Checks whether -lrt is needed for any of the given functions. If # so, appends -lrt via [teaish-ldflags-prepend] and returns 1, else # returns 0. It also defines LDFLAGS_LIBRT to the libs flag or an # empty string. # # Some systems (ex: SunOS) require -lrt in order to use nanosleep. # proc teaish-check-librt {{funclist {fdatasync nanosleep}}} { teaish-check-cached -nostatus "Checking whether ($funclist) need librt" { define LDFLAGS_LIBRT "" foreach func $funclist { if {[msg-quiet proj-check-function-in-lib $func rt]} { set ldrt [get-define lib_${func}] undefine lib_${func} if {"" ne $ldrt} { teaish-ldflags-prepend -r [define LDFLAGS_LIBRT $ldrt] msg-result $ldrt return 1 } else { msg-result "no lib needed" return 1 } } } msg-result "not found" return 0 } } # @teaish-check-stdint # # A thin proxy for [cc-with] which checks for <stdint.h> and the # various fixed-size int types it declares. It defines HAVE_STDINT_T # to 0 or 1 and (if it's 1) defines HAVE_XYZ_T for each XYZ int type # to 0 or 1, depending on whether its available. proc teaish-check-stdint {} { teaish-check-cached "Checking for stdint.h" { msg-quiet cc-with {-includes stdint.h} \ {cc-check-types int8_t int16_t int32_t int64_t intptr_t \ uint8_t uint16_t uint32_t uint64_t uintptr_t} } } # @teaish-is-mingw # # Returns 1 if building for mingw, else 0. proc teaish-is-mingw {} { return [expr { [string match *mingw* [get-define host]] && ![file exists /dev/null] }] } # @teaish-check-libdl # # Checks for whether dlopen() can be found and whether it requires # -ldl for linking. If found, returns 1, defines LDFLAGS_DLOPEN to the # linker flags (if any), and passes those flags to # teaish-ldflags-prepend. It unconditionally defines HAVE_DLOPEN to 0 # or 1 (the its return result value). proc teaish-check-dlopen {} { teaish-check-cached -nostatus "Checking for dlopen()" { set rc 0 set lfl "" if {[cc-with {-includes dlfcn.h} { cctest -link 1 -declare "extern char* dlerror(void);" -code "dlerror();"}]} { msg-result "-ldl not needed" incr rc } elseif {[cc-check-includes dlfcn.h]} { incr rc if {[cc-check-function-in-lib dlopen dl]} { set lfl [get-define lib_dlopen] undefine lib_dlopen msg-result " dlopen() needs $lfl" } else { msg-result " - dlopen() not found in libdl. Assuming dlopen() is built-in." } } else { msg-result "not found" } teaish-ldflags-prepend [define LDFLAGS_DLOPEN $lfl] define HAVE_DLOPEN $rc } } # # @teaish-check-libmath # # Handles the --enable-math flag. Returns 1 if found, else 0. # If found, it prepends -lm (if needed) to the linker flags. proc teaish-check-libmath {} { teaish-check-cached "Checking for libc math library" { set lfl "" set rc 0 if {[msg-quiet proj-check-function-in-lib ceil m]} { incr rc set lfl [get-define lib_ceil] undefine lib_ceil teaish-ldflags-prepend $lfl msg-checking "$lfl " } define LDFLAGS_LIBMATH $lfl expr $rc } } # @teaish-import-features ?-flags? feature-names... # # For each $name in feature-names... it invokes: # # use teaish/feature/$name # # to load TEAISH_AUTOSETUP_DIR/feature/$name.tcl # # By default, if a proc named teaish-check-${name}-options is defined # after sourcing a file, it is called and its result is passed to # proj-append-options. This can be suppressed with the -no-options # flag. # # Flags: # # -no-options: disables the automatic running of # teaish-check-NAME-options, # # -run: if the function teaish-check-NAME exists after importing # then it is called. This flag must not be used when calling this # function from teaish-options. This trumps both -pre and -post. # # -pre: if the function teaish-check-NAME exists after importing # then it is passed to [teaish-checks-queue -pre]. # # -post: works like -pre but instead uses[teaish-checks-queue -post]. proc teaish-import-features {args} { set pk "" set doOpt 1 proj-parse-simple-flags args flags { -no-options 0 {set doOpt 0} -run 0 {expr 1} -pre 0 {set pk -pre} -post 0 {set pk -post} } # # TODO: never import the same module more than once. The "use" # command is smart enough to not do that but we would need to # remember whether or not any teaish-check-${arg}* procs have been # called before, and skip them. # if {$flags(-run) && "" ne $pk} { proj-error "Cannot use both -run and $pk" \ " (called from [proj-scope 1])" } foreach arg $args { uplevel "use teaish/feature/$arg" if {$doOpt} { set n "teaish-check-${arg}-options" if {[llength [info proc $n]] > 0} { if {"" ne [set x [$n]]} { options-add $x } } } if {$flags(-run)} { set n "teaish-check-${arg}" if {[llength [info proc $n]] > 0} { uplevel 1 $n } } elseif {"" ne $pk} { set n "teaish-check-${arg}" if {[llength [info proc $n]] > 0} { teaish-checks-queue {*}$pk $n } } } } |
Added autosetup/teaish/tester.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 | ######################################################################## # 2025 April 5 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # * May you do good and not evil. # * May you find forgiveness for yourself and forgive others. # * May you share freely, never taking more than you give. # ######################################################################## # # Helper routines for running tests on teaish extensions # ######################################################################## # ----- @module teaish/tester.tcl ----- # # @section TEA-ish Testing APIs. # # Though these are part of the autosup dir hierarchy, they are not # intended to be run from autosetup code. Rather, they're for use # with/via teaish.tester.tcl and target canonical Tcl only, not JimTcl # (which the autosetup pieces do target). # # @test-current-scope ?lvl? # # Returns the name of the _calling_ proc from ($lvl + 1) levels up the # call stack (where the caller's level will be 1 up from _this_ # call). If $lvl would resolve to global scope "global scope" is # returned and if it would be negative then a string indicating such # is returned (as opposed to throwing an error). # proc test-current-scope {{lvl 0}} { #uplevel [expr {$lvl + 1}] {lindex [info level 0] 0} set ilvl [info level] set offset [expr {$ilvl - $lvl - 1}] if { $offset < 0} { return "invalid scope ($offset)" } elseif { $offset == 0} { return "global scope" } else { return [lindex [info level $offset] 0] } } # @test-msg # # Emits all arugments to stdout. # proc test-msg {args} { puts "$args" } # @test-warn # # Emits all arugments to stderr. # proc test-warn {args} { puts stderr "WARNING: $args" } # # @test-error msg # # Triggers a test-failed error with a string describing the calling # scope and the provided message. # proc test-fail {args} { #puts stderr "ERROR: \[[test-current-scope 1]]: $msg" #exit 1 error "FAIL: \[[test-current-scope 1]]: $args" } array set ::test__Counters {} array set ::test__Config { verbose-assert 0 verbose-affirm 0 } # Internal impl for affirm and assert. # # $args = ?-v? script {msg-on-fail ""} proc test__affert {failMode args} { if {$failMode} { set what assert } else { set what affirm } set verbose $::test__Config(verbose-$what) if {"-v" eq [lindex $args 0]} { lassign $args - script msg if {1 == [llength $args]} { # If -v is the only arg, toggle default verbose mode set ::test__Config(verbose-$what) [expr {!$::test__Config(verbose-$what)}] return } incr verbose } else { lassign $args script msg } incr ::test__Counters($what) if {![uplevel 1 [concat expr [list $script]]]} { if {"" eq $msg} { set msg $script } set txt [join [list $what # $::test__Counters($what) "failed:" $msg]] if {$failMode} { puts stderr $txt exit 1 } else { error $txt } } elseif {$verbose} { puts stderr [join [list $what # $::test__Counters($what) "passed:" $script]] } } # # @affirm ?-v? script ?msg? # # Works like a conventional assert method does, but reports failures # using [error] instead of [exit]. If -v is used, it reports passing # assertions to stderr. $script is evaluated in the caller's scope as # an argument to [expr]. # proc affirm {args} { tailcall test__affert 0 {*}$args } # # @assert ?-v? script ?msg? # # Works like [affirm] but exits on error. # proc assert {args} { tailcall test__affert 1 {*}$args } # # @test-assert testId script ?msg? # # Works like [assert] but emits $testId to stdout first. # proc test-assert {testId script {msg ""}} { puts "test $testId" tailcall test__affert 1 $script $msg } # # @test-expect testId script result # # Runs $script in the calling scope and compares its result to # $result, minus any leading or trailing whitespace. If they differ, # it triggers an [assert]. # proc test-expect {testId script result} { puts "test $testId" set x [string trim [uplevel 1 $script]] set result [string trim $result] tailcall test__affert 0 [list $x eq $result] \ "\nEXPECTED: <<$result>>\nGOT: <<$x>>" } # # @test-catch cmd ?...args? # # Runs [cmd ...args], repressing any exception except to possibly log # the failure. Returns 1 if it caught anything, 0 if it didn't. # proc test-catch {cmd args} { if {[catch { $cmd {*}$args } rc xopts]} { puts "[test-current-scope] ignoring failure of: $cmd [lindex $args 0]: $rc" return 1 } return 0 } if {![array exists ::teaish__BuildFlags]} { array set ::teaish__BuildFlags {} } # # @teaish-build-flag3 flag tgtVar ?dflt? # # If the current build has the configure-time flag named $flag set # then tgtVar is assigned its value and 1 is returned, else tgtVal is # assigned $dflt and 0 is returned. # # Caveat #1: only valid when called in the context of teaish's default # "make test" recipe, e.g. from teaish.test.tcl. It is not valid from # a teaish.tcl configure script because (A) the state it relies on # doesn't fully exist at that point and (B) that level of the API has # more direct access to the build state. This function requires that # an external script have populated its internal state, which is # normally handled via teaish.tester.tcl.in. # # Caveat #2: defines in the style of HAVE_FEATURENAME with a value of # 0 are, by long-standing configure script conventions, treated as # _undefined_ here. # proc teaish-build-flag3 {flag tgtVar {dflt ""}} { upvar $tgtVar tgt if {[info exists ::teaish__BuildFlags($flag)]} { set tgt $::teaish__BuildFlags($flag) return 1; } elseif {0==[array size ::teaish__BuildFlags]} { test-warn \ "\[[test-current-scope]] was called from " \ "[test-current-scope 1] without the build flags imported." } set tgt $dflt return 0 } # # @teaish-build-flag flag ?dflt? # # Convenience form of teaish-build-flag3 which returns the # configure-time-defined value of $flag or "" if it's not defined (or # if it's an empty string). # proc teaish-build-flag {flag {dflt ""}} { set tgt "" teaish-build-flag3 $flag tgt $dflt return $tgt } |
Deleted config.guess.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted config.sub.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to configure.
more than 10,000 changes
Deleted configure.ac.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to contrib/sqlitecon.tcl.
︙ | ︙ | |||
563 564 565 566 567 568 569 | proc sqlitecon::Cut w { if {[sqlitecon::canCut $w]==1} { sqlitecon::Copy $w $w delete sel.first sel.last } } | | | 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 | proc sqlitecon::Cut w { if {[sqlitecon::canCut $w]==1} { sqlitecon::Copy $w $w delete sel.first sel.last } } # Do a paste operation. # proc sqlitecon::Paste w { if {[sqlitecon::canCut $w]==1} { $w delete sel.first sel.last } if {[catch {selection get -displayof $w -selection CLIPBOARD} topaste] && [catch {selection get -displayof $w -selection PRIMARY} topaste]} { |
︙ | ︙ |
Added doc/compile-for-unix.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | # Notes On Compiling SQLite On All Kinds Of Unix Here are step-by-step instructions on how to build SQLite from canonical source on any modern machine that isn't Windows. These notes are tested (on 2024-10-11) on Ubuntu and on MacOS, but they are general and should work on most any modern unix platform. See the companion document ([](./compile-for-windows.md>)) for guidance on building for Windows. 1. Install a C-compiler. GCC or Clang both work fine. If you are reading this document, you've probably already done that. 2. *(Optional):* Install TCL development libraries. In this note, we'll do a private install in the $HOME/local directory, but you can make adjustments to install TCL wherever you like. This document assumes you are working with TCL version 9.0. See also the [](./tcl-extension-testing.md) document that contains more details on compiling Tcl for use with SQLite. <ol type="a"> <li>Get the TCL source archive, perhaps from <https://www.tcl.tk/software/tcltk/download.html> or <https://sqlite.org/tmp/tcl9.0.0.tar.gz>. <li>Untar the source archive. CD into the "unix/" subfolder of the source tree. <li>Run: `mkdir $HOME/local` <li>Run: `./configure --prefix=$HOME/local` <li>Run: `make install` </ol> <p> As of 2024-10-25, TCL is not longer required for many common build targets, such as "sqlite3.c" or the "sqlite3" command-line tool. So you can skip this step if that is all you want to build. TCL is still required to run "make test" and similar, or to build the TCL extension, of course. 4. Download the SQLite source tree and unpack it. CD into the toplevel directory of the source tree. 5. Run: `./configure --enable-all --with-tclsh=$HOME/local/bin/tclsh9.0` You do not need to use --with-tclsh if the tclsh you want to use is the first one on your PATH or if you are building without TCL. 6. Run the "`Makefile`" makefile with an appropriate target. Examples: <ul> <li> `make sqlite3.c` <li> `make sqlite3` <li> `make sqldiff` <li> `make sqlite3_rsync` </ul> <p>None of the targets above require TCL. TCL is needed for the following targets: <ul> <li> `make tclextension-install` <li> `make devtest` <li> `make releasetest` <li> `make sqlite3_analyzer` </ul> It is not required that you run the "tclextension-install" target prior to running tests. However, the tests will run more smoothly if you do. The version of SQLite used for the TCL extension does *not* need to correspond to the version of SQLite under test. So you can install the SQLite TCL extension once, and then use it to test many different versions of SQLite. 7. For a debugging build of the CLI, where the ".treetrace" and ".wheretrace" commands work, add the the --with-debug argument to configure. |
Added doc/compile-for-windows.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 | # Notes On Compiling SQLite On Windows 11 Below are step-by-step instructions on how to build SQLite from canonical source on a new Windows 11 PC, as of 2024-10-09. See [](./compile-for-unix.md) for a similar guide for unix-like systems, including MacOS. 1. Install Microsoft Visual Studio. The free "community edition" will work fine. Do a standard install for C++ development. SQLite only needs the "cl" compiler and the "nmake" build tool. <ul><li><b>Note:</b> VS2015 or later is required for the procedures below to all work. You *might* be able to get the build to work with earlier versions of MSVC, but in that case the TCL installation of step 3 will be required, since the "jimsh0.c" program of Autosetup that is used as a substitute for "tclsh.exe" won't compile with versions of Visual Studio prior to VS2015. In any event, building SQLite from canonical source code on Windows is not supported for earlier versions of Visual Studio.</ul> 2. Under the "Start" menu, find "All Apps" then go to "Visual Studio 20XX" and find "x64 Native Tools Command Prompt for VS 20XX". Pin that application to your task bar, as you will use it a lot. Bring up an instance of this command prompt and do all of the subsequent steps in that "x64 Native Tools" command prompt. (Or use "x86" if you want a 32-bit build.) The subsequent steps will not work in a vanilla DOS prompt. Nor will they work in PowerShell. 3. *(Optional):* Install TCL development libraries. This note assumes that you will install the TCL development libraries in the "`c:\Tcl`" directory. Make adjustments if you want TCL installed somewhere else. SQLite needs both the "tclsh90.exe" command-line tool as part of the build process, and the "tcl90.lib" and "tclstub.lib" libraries in order to run tests. This document assumes you are working with TCL version 9.0. See [](./tcl-extension-testing.md#windows) for guidance on how to compile TCL version 8.6 for use with SQLite. <ol type="a"> <li>Get the TCL source archive, perhaps from <https://www.tcl.tk/software/tcltk/download.html> or <https://sqlite.org/tmp/tcl9.0.0.tar.gz>. <li>Untar or unzip the source archive. CD into the "win/" subfolder of the source tree. <li>Run: `nmake /f makefile.vc INSTALLDIR=c:\Tcl release` <li>Run: `nmake /f makefile.vc INSTALLDIR=c:\Tcl install` <br> Notes: <ol type="i"> <li> The previous two `nmake` commands must be run separately. <li> Also, the INSTALLDIR=... argument is required on both. </ol> <li><i>Optional:</i> CD to `c:\Tcl\bin` and make a copy of `tclsh90.exe` over into just `tclsh.exe`. <li><i>Optional:</i> Add `c:\Tcl\bin` to your %PATH%. To do this, go to Settings and search for "path". Select "edit environment variables for your account" and modify your default PATH accordingly. You will need to close and reopen your command prompts after making this change. </ol> As of 2024-10-25, TCL is not longer required for many common build targets, such as "sqlite3.c" or the "sqlite3.exe" command-line tool. So you can skip this step if that is all you want to build. TCL is still required to run "make test" and similar, or to build the TCL extension, of course. 4. Download the SQLite source tree and unpack it. CD into the toplevel directory of the source tree. 5. Run the "`Makefile.msc`" makefile with an appropriate target. Examples: <ul> <li> `nmake /f makefile.msc` <li> `nmake /f makefile.msc sqlite3.c` <li> `nmake /f makefile.msc sqlite3.exe` <li> `nmake /f makefile.msc sqldiff.exe` <li> `nmake /f makefile.msc sqlite3_rsync.exe` </ul> <p>No TCL is required for the nmake targets above. But for the ones that follow, you will need a TCL installation, as described in step 3 above. If you install TCL in some directory other than C:\\Tcl, then you will also need to add the "TCLDIR=<i><dir></i>" option on the nmake command line to tell nmake where your TCL is installed. <ul> <li> `nmake /f makefile.msc tclextension-install` <li> `nmake /f makefile.msc devtest` <li> `nmake /f makefile.msc releasetest` <li> `nmake /f makefile.msc sqlite3_analyzer.exe` </ul> It is not required that you run the "tclextension-install" target prior to running tests. However, the tests will run more smoothly if you do. The version of SQLite used for the TCL extension does *not* need to correspond to the version of SQLite under test. So you can install the SQLite TCL extension once, and then use it to test many different versions of SQLite. 7. For a debugging build of the CLI, where the ".treetrace" and ".wheretrace" commands work, add the DEBUG=3 argument to nmake. Like this: <ul> <li> `nmake /f makefile.msc DEBUG=3 clean sqlite3.exe` </ul> ## 32-bit Builds Doing a 32-bit build is just like doing a 64-bit build with the following minor changes: 1. Use the "x86 Native Tools Command Prompt" instead of "x64 Native Tools Command Prompt". "**x86**" instead of "**x64**". 2. Use a different installation directory for TCL. The recommended directory is `c:\tcl32`. Thus you end up with two TCL builds: <ul> <li> `c:\tcl` ← 64-bit (the default) <li> `c:\tcl32` ← 32-bit </ul> 3. Ensure that `c:\tcl32\bin` comes before `c:\tcl\bin` on your PATH environment variable. You can achieve this using a command like: <ul> <li> `set PATH=c:\tcl32\bin;%PATH%` </ul> ## Building a DLL The command the developers use for building the deliverable DLL on the [download page](https://sqlite.org/download.html) is as follows: > ~~~~ nmake /f Makefile.msc sqlite3.dll USE_NATIVE_LIBPATHS=1 "OPTS=-DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_ENABLE_RTREE=1 -DSQLITE_ENABLE_JSON1=1 -DSQLITE_ENABLE_GEOPOLY=1 -DSQLITE_ENABLE_SESSION=1 -DSQLITE_ENABLE_PREUPDATE_HOOK=1 -DSQLITE_ENABLE_SERIALIZE=1 -DSQLITE_ENABLE_MATH_FUNCTIONS=1" ~~~~ That command generates both the sqlite3.dll and sqlite3.def files. The same command works for both 32-bit and 64-bit builds. ## Statically Linking The TCL Library Some utility programs associated with SQLite need to be linked with TCL in order to function. The [sqlite3_analyzer.exe program](https://sqlite.org/sqlanalyze.html) is an example. You can build as described above, and then enter: > ~~~~ nmake /f Makefile.msc sqlite3_analyzer.exe ~~~~ And you will end up with a working executable. However, that executable will depend on having the "tcl98.dll" library somewhere on your %PATH%. Use the following steps to build an executable that has the TCL library statically linked so that it does not depend on separate DLL: 1. Use the appropriate "Command Prompt" window - either x86 or x64, depending on whether you want a 32-bit or 64-bit executable. 2. Untar the TCL source tarball into a fresh directory. CD into the "win/" subfolder. 3. Run: `nmake /f makefile.vc OPTS=static shell` 4. CD into the "Release*" subfolder that is created (note the wildcard - the full name of the directory might vary). There you will find the "tcl90s.lib" file. Copy this file into the same directory that you put the "tcl90.lib" on your initial installation. (In this document, that directory is "`C:\Tcl32\lib`" for 32-bit builds and "`C:\Tcl\lib`" for 64-bit builds.) 5. CD into your SQLite source code directory and build the desired utility program, but add the following extra argument to the nmake command line: <blockquote><pre> STATICALLY_LINK_TCL=1 </pre></blockquote> <p>So, for example, to build a statically linked version of sqlite3_analyzer.exe, you might type: <blockquote><pre> nmake /f Makefile.msc STATICALLY_LINK_TCL=1 sqlite3_analyzer.exe </pre></blockquote> 6. After your executable is built, you can verify that it does not depend on the TCL DLL by running: <blockquote><pre> dumpbin /dependents sqlite3_analyzer.exe </pre></blockquote> |
Added doc/jsonb.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 | # The JSONB Format This document describes SQLite's JSONB binary encoding of JSON. ## 1.0 What Is JSONB? Beginning with version 3.45.0 (circa 2024-01-01), SQLite supports an alternative binary encoding of JSON which we call "JSONB". JSONB is a binary format that stored as a BLOB. The advantage of JSONB over ordinary text RFC 8259 JSON is that JSONB is both slightly smaller (by between 5% and 10% in most cases) and can be processed in less than half the number of CPU cycles. The built-in [JSON SQL functions] of SQLite can accept either ordinary text JSON or the binary JSONB encoding for any of their JSON inputs. The "JSONB" name is inspired by [PostgreSQL](https://postgresql.org), but the on-disk format for SQLite's JSONB is not the same as PostgreSQL's. The two formats have the same name, but they have wildly different internal representations and are not in any way binary compatible. The central idea behind this JSONB specification is that each element begins with a header that includes the size and type of that element. The header takes the place of punctuation such as double-quotes, curly-brackes, square-brackets, commas, and colons. Since the size and type of each element is contained in its header, the element can be read faster since it is no longer necessary to carefully scan forward looking for the closing delimiter. The payload of JSONB is the same as for corresponding text JSON. The same payload bytes occur in the same order. The only real difference between JSONB and ordinary text JSON is that JSONB includes a binary header on each element and omits delimiter and separator punctuation. ### 1.1 Internal Use Only The details of the JSONB are not intended to be visible to application developers. Application developers should look at JSONB as an opaque BLOB used internally by SQLite. Nevertheless, we want the format to be backwards compatible across all future versions of SQLite. To that end, the format is documented by this file in the source tree. But this file should be used only by SQLite core developers, not by developers of applications that only use SQLite. ## 2.0 The Purpose Of This Document JSONB is not intended as an external format to be used by applications. JSONB is designed for internal use by SQLite only. Programmers do not need to understand the JSONB format in order to use it effectively. Applications should access JSONB only through the [JSON SQL functions], not by looking at individual bytes of the BLOB. However, JSONB is intended to be portable and backwards compatible for all future versions of SQLite. In other words, you should not have to export and reimport your SQLite database files when you upgrade to a newer SQLite version. For that reason, the JSONB format needs to be well-defined. This document is therefore similar in purpose to the [SQLite database file format] document that describes the on-disk format of an SQLite database file. Applications are not expected to directly read and write the bits and bytes of SQLite database files. The SQLite database file format is carefully documented so that it can be stable and enduring. In the same way, the JSONB representation of JSON is documented here so that it too can be stable and enduring, not so that applications can read or writes individual bytes. ## 3.0 Encoding JSONB is a direct translation of the underlying text JSON. The difference is that JSONB uses a binary encoding that is faster to parse compared to the detailed syntax of text JSON. Each JSON element is encoded as a header and a payload. The header determines type of element (string, numeric, boolean, null, object, or array) and the size of the payload. The header can be between 1 and 9 bytes in size. The payload can be any size from zero bytes up to the maximum allowed BLOB size. ### 3.1 Payload Size The upper four bits of the first byte of the header determine size of the header and possibly also the size of the payload. If the upper four bits have a value between 0 and 11, then the header is exactly one byte in size and the payload size is determined by those upper four bits. If the upper four bits have a value between 12 and 15, that means that the total header size is 2, 3, 5, or 9 bytes and the payload size is unsigned big-endian integer that is contained in the subsequent bytes. The size integer is the one byte that following the initial header byte if the upper four bits are 12, two bytes if the upper bits are 13, four bytes if the upper bits are 14, and eight bytes if the upper bits are 15. The current design of SQLite does not support BLOB values larger than 2GiB, so the eight-byte variant of the payload size integer will never be used by the current code. The eight-byte payload size integer is included in the specification to allow for future expansion. The header for an element does *not* need to be in its simplest form. For example, consider the JSON numeric value "`1`". That element can be encode in five different ways: * `0x13 0x31` * `0xc3 0x01 0x31` * `0xd3 0x00 0x01 0x31` * `0xe3 0x00 0x00 0x00 0x01 0x31` * `0xf3 0x00 0x00 0x00 0x00 0x00 0x00 0x00 0x01 0x31` The shortest encoding is preferred, of course, and usually happens with primitive elements such as numbers. However the total size of an array or object might not be known exactly when the header of the element is first generated. It is convenient to reserve space for the largest possible header and then go back and fill in the correct payload size at the end. This technique can result in array or object headers that are larger than absolutely necessary. ### 3.2 Element Type The least-significant four bits of the first byte of the header (the first byte masked against 0x0f) determine element type. The following codes are used: <ol> <li type="0"><p><b>NULL</b> → The element is a JSON "null". The payload size for a true JSON NULL must must be zero. Future versions of SQLite might extend the JSONB format with elements that have a zero element type but a non-zero size. In that way, legacy versions of SQLite will interpret the element as a NULL for backwards compatibility while newer versions will interpret the element in some other way. <li value="1"><p><b>TRUE</b> → The element is a JSON "true". The payload size must be zero for a actual "true" value. Elements with type 1 and a non-zero payload size are reserved for future expansion. Legacy implementations that see an element type of 1 with a non-zero payload size should continue to interpret that element as "true" for compatibility. <li value="2"><p><b>FALSE</b> → The element is a JSON "false". The payload size must be zero for a actual "false" value. Elements with type 2 and a non-zero payload size are reserved for future expansion. Legacy implementations that see an element type of 2 with a non-zero payload size should continue to interpret that element as "false" for compatibility. <li value="3"><p><b>INT</b> → The element is a JSON integer value in the canonical RFC 8259 format, without extensions. The payload is the ASCII text representation of that numeric value. <li value="4"><p><b>INT5</b> → The element is a JSON integer value that is not in the canonical format. The payload is the ASCII text representation of that numeric value. Because the payload is in a non-standard format, it will need to be translated when the JSONB is converted into RFC 8259 text JSON. <li value="5"><p><b>FLOAT</b> → The element is a JSON floating-point value in the canonical RFC 8259 format, without extensions. The payload is the ASCII text representation of that numeric value. <li value="6"><p><b>FLOAT5</b> → The element is a JSON floating-point value that is not in the canonical format. The payload is the ASCII text representation of that numeric value. Because the payload is in a non-standard format, it will need to be translated when the JSONB is converted into RFC 8259 text JSON. <li value="7"><p><b>TEXT</b> → The element is a JSON string value that does not contain any escapes nor any characters that need to be escaped for either SQL or JSON. The payload is the UTF8 text representation of the string value. The payload does <i>not</i> include string delimiters. <li value="8"><p><b>TEXTJ</b> → The element is a JSON string value that contains RFC 8259 character escapes (such as "<tt>\n</tt>" or "<tt>\u0020</tt>"). Those escapes will need to be translated into actual UTF8 if this element is [json_extract|extracted] into SQL. The payload is the UTF8 text representation of the escaped string value. The payload does <i>not</i> include string delimiters. <li value="9"><p><b>TEXT5</b> → The element is a JSON string value that contains character escapes, including some character escapes that part of JSON5 and which are not found in the canonical RFC 8259 spec. Those escapes will need to be translated into standard JSON prior to rendering the JSON as text, or into their actual UTF8 characters if this element is [json_extract|extracted] into SQL. The payload is the UTF8 text representation of the escaped string value. The payload does <i>not</i> include string delimiters. <li value="10"><p><b>TEXTRAW</b> → The element is a JSON string value that contains UTF8 characters that need to be escaped if this string is rendered into standard JSON text. The payload does <i>not</i> include string delimiters. <li value="11"><p><b>ARRAY</b> → The element is a JSON array. The payload contains JSONB elements that comprise values contained within the array. <li value="12"><p><b>OBJECT</b> → The element is a JSON object. The payload contains pairs of JSONB elements that comprise entries for the JSON object. The first element in each pair must be a string (types 7 through 10). The second element of each pair may be any types, including nested arrays or objects. <li value="13"><p><b>RESERVED-13</b> → Reserved for future expansion. Legacy implements that encounter this element type should raise an error. <li value="14"><p><b>RESERVED-14</b> → Reserved for future expansion. Legacy implements that encounter this element type should raise an error. <li value="15"><p><b>RESERVED-15</b> → Reserved for future expansion. Legacy implements that encounter this element type should raise an error. </ol> Element types outside the range of 0 to 12 are reserved for future expansion. The current implement raises an error if see an element type other than those listed above. However, future versions of SQLite might use of the three remaining element types to implement indexing or similar optimizations, to speed up lookup against large JSON arrays and/or objects. ### 3.3 Design Rationale For Element Types A key goal of JSONB is that it should be quick to translate to and from text JSON and/or be constructed from SQL values. When converting from text into JSONB, we do not want the converter subroutine to burn CPU cycles converting elements values into some standard format which might never be used. Format conversion is "lazy" - it is deferred until actually needed. This has implications for the JSONB format design: 1. Numeric values are stored as text, not a numbers. The values are a direct copy of the text JSON values from which they are derived. 2. There are multiple element types depending on the details of value formats. For example, INT is used for pure RFC-8259 integer literals and INT5 exists for JSON5 extensions such as hexadecimal notation. FLOAT is used for pure RFC-8259 floating point literals and FLOAT5 is used for JSON5 extensions. There are four different representations of strings, depending on where the string came from and how special characters within the string are escaped. A second goal of JSONB is that it should be capable of serving as the "parse tree" for JSON when a JSON value is being processed by the various [JSON SQL functions] built into SQLite. Before JSONB was developed, operations such [json_replace()] and [json_patch()] and similar worked in three stages: 1. Translate the text JSON into a internal format that is easier to scan and edit. 2. Perform the requested operation on the JSON. 3. Translate the internal format back into text. JSONB seeks to serve as the internal format directly - bypassing the first and third stages of that process. Since most of the CPU cycles are spent on the first and third stages, that suggests that JSONB processing will be much faster than text JSON processing. So when processing JSONB, only the second stage of the three-stage process is required. But when processing text JSON, it is still necessary to do stages one and three. If JSONB is to be used as the internal binary representation, this is yet another reason to store numeric values as text. Storing numbers as text minimizes the amount of conversion work needed for stages one and three. This is also why there are four different representations of text in JSONB. Different text representations are used for text coming from different sources (RFC-8259 JSON, JSON5, or SQL string values) and conversions only happen if and when they are actually needed. ### 3.4 Valid JSONB BLOBs A valid JSONB BLOB consists of a single JSON element. The element must exactly fill the BLOB. This one element is often a JSON object or array and those usually contain additional elements as its payload, but the element can be a primitive value such a string, number, boolean, or null. When the built-in JSON functions are attempting to determine if a BLOB argument is a JSONB or just a random BLOB, they look at the header of the outer element to see that it is well-formed and that the element completely fills the BLOB. If these conditions are met, then the BLOB is accepted as a JSONB value. |
Changes to doc/lemon.html.
︙ | ︙ | |||
318 319 320 321 322 323 324 | <a href="#pcode"><tt>%code</tt> directives</a>, then the parse object can be allocated from the stack rather than from the heap. These are the steps: <ul> <li> Declare a local variable of type "yyParser" <li> Initialize the variable using ParseInit() | | | 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 | <a href="#pcode"><tt>%code</tt> directives</a>, then the parse object can be allocated from the stack rather than from the heap. These are the steps: <ul> <li> Declare a local variable of type "yyParser" <li> Initialize the variable using ParseInit() <li> Pass a pointer to the variable in calls to Parse() <li> Deallocate substructure in the parse variable using ParseFinalize(). </ul> <p>The following code illustrates how this is done: <pre> ParseFile(){ |
︙ | ︙ | |||
679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 | <li><tt><a href='#default_destructor'>%default_destructor</a></tt> <li><tt><a href='#default_type'>%default_type</a></tt> <li><tt><a href='#destructor'>%destructor</a></tt> <li><tt><a href='#pifdef'>%else</a></tt> <li><tt><a href='#pifdef'>%endif</a></tt> <li><tt><a href='#extraarg'>%extra_argument</a></tt> <li><tt><a href='#pfallback'>%fallback</a></tt> <li><tt><a href='#pifdef'>%if</a></tt> <li><tt><a href='#pifdef'>%ifdef</a></tt> <li><tt><a href='#pifdef'>%ifndef</a></tt> <li><tt><a href='#pinclude'>%include</a></tt> <li><tt><a href='#pleft'>%left</a></tt> <li><tt><a href='#pname'>%name</a></tt> <li><tt><a href='#pnonassoc'>%nonassoc</a></tt> <li><tt><a href='#parse_accept'>%parse_accept</a></tt> <li><tt><a href='#parse_failure'>%parse_failure</a></tt> <li><tt><a href='#pright'>%right</a></tt> <li><tt><a href='#stack_overflow'>%stack_overflow</a></tt> <li><tt><a href='#stack_size'>%stack_size</a></tt> <li><tt><a href='#start_symbol'>%start_symbol</a></tt> <li><tt><a href='#syntax_error'>%syntax_error</a></tt> <li><tt><a href='#token'>%token</a></tt> <li><tt><a href='#token_class'>%token_class</a></tt> <li><tt><a href='#token_destructor'>%token_destructor</a></tt> | > > | 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 | <li><tt><a href='#default_destructor'>%default_destructor</a></tt> <li><tt><a href='#default_type'>%default_type</a></tt> <li><tt><a href='#destructor'>%destructor</a></tt> <li><tt><a href='#pifdef'>%else</a></tt> <li><tt><a href='#pifdef'>%endif</a></tt> <li><tt><a href='#extraarg'>%extra_argument</a></tt> <li><tt><a href='#pfallback'>%fallback</a></tt> <li><tt><a href='#reallc'>%free</a></tt> <li><tt><a href='#pifdef'>%if</a></tt> <li><tt><a href='#pifdef'>%ifdef</a></tt> <li><tt><a href='#pifdef'>%ifndef</a></tt> <li><tt><a href='#pinclude'>%include</a></tt> <li><tt><a href='#pleft'>%left</a></tt> <li><tt><a href='#pname'>%name</a></tt> <li><tt><a href='#pnonassoc'>%nonassoc</a></tt> <li><tt><a href='#parse_accept'>%parse_accept</a></tt> <li><tt><a href='#parse_failure'>%parse_failure</a></tt> <li><tt><a href='#pright'>%right</a></tt> <li><tt><a href='#reallc'>%realloc</a></tt> <li><tt><a href='#stack_overflow'>%stack_overflow</a></tt> <li><tt><a href='#stack_size'>%stack_size</a></tt> <li><tt><a href='#start_symbol'>%start_symbol</a></tt> <li><tt><a href='#syntax_error'>%syntax_error</a></tt> <li><tt><a href='#token'>%token</a></tt> <li><tt><a href='#token_class'>%token_class</a></tt> <li><tt><a href='#token_destructor'>%token_destructor</a></tt> |
︙ | ︙ | |||
840 841 842 843 844 845 846 | <h4>4.4.7 The <tt>%fallback</tt> directive</h4> <p>The <tt>%fallback</tt> directive specifies an alternative meaning for one or more tokens. The alternative meaning is tried if the original token would have generated a syntax error.</p> <p>The <tt>%fallback</tt> directive was added to support robust parsing of SQL | | | 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 | <h4>4.4.7 The <tt>%fallback</tt> directive</h4> <p>The <tt>%fallback</tt> directive specifies an alternative meaning for one or more tokens. The alternative meaning is tried if the original token would have generated a syntax error.</p> <p>The <tt>%fallback</tt> directive was added to support robust parsing of SQL syntax in <a href='https://sqlite.org/'>SQLite</a>. The SQL language contains a large assortment of keywords, each of which appears as a different token to the language parser. SQL contains so many keywords that it can be difficult for programmers to keep up with them all. Programmers will, therefore, sometimes mistakenly use an obscure language keyword for an identifier. The <tt>%fallback</tt> directive provides a mechanism to tell the parser: "If you are unable to parse this keyword, try treating it as an identifier instead."</p> |
︙ | ︙ | |||
875 876 877 878 879 880 881 | just not as general. Each of these directives must begin at the left margin. No whitespace is allowed between the "%" and the directive name.</p> <p>Grammar text in between "<tt>%ifdef MACRO</tt>" and the next nested "<tt>%endif</tt>" is ignored unless the "-DMACRO" command-line option is used. Grammar text | | | 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 | just not as general. Each of these directives must begin at the left margin. No whitespace is allowed between the "%" and the directive name.</p> <p>Grammar text in between "<tt>%ifdef MACRO</tt>" and the next nested "<tt>%endif</tt>" is ignored unless the "-DMACRO" command-line option is used. Grammar text between "<tt>%ifndef MACRO</tt>" and the next nested "<tt>%endif</tt>" is included except when the "-DMACRO" command-line option is used.<p> <p>The text in between "<tt>%if</tt> <i>CONDITIONAL</i>" and its corresponding <tt>%endif</tt> is included only if <i>CONDITIONAL</i> is true. The CONDITION is one or more macro names, optionally connected using the "||" and "&&" binary operators, the "!" unary operator, and grouped using balanced parentheses. Each term is true if the |
︙ | ︙ | |||
1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 | period. This directive specifies that the identified token should match any input token.</p> <p>When the generated parser has the choice of matching an input against the wildcard token and some other token, the other token is always used. The wildcard token is only matched if there are no alternatives.</p> <a id='errors'></a> <h2>5.0 Error Processing</h2> <p>After extensive experimentation over several years, it has been discovered that the error recovery strategy used by yacc is about as good as it gets. And so that is what Lemon uses.</p> | > > > > > > > > > > > > > > > | 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 | period. This directive specifies that the identified token should match any input token.</p> <p>When the generated parser has the choice of matching an input against the wildcard token and some other token, the other token is always used. The wildcard token is only matched if there are no alternatives.</p> <a id='reallc'></a> <h4>4.4.26 The <tt>%realloc</tt> and <tt>%free</tt> directives</h4> <p>The <tt>%realloc</tt> and <tt>%free</tt> directives defines function that allocate and free heap memory. The signatures of these functions should be the same as the realloc() and free() functions from the standard C library. <p>If both of these functions are defined then these functions are used to allocate and free memory for supplemental parser stack space, if the initial parse stack space is exceeded. The initial parser stack size is specified by either <tt>%stack_size</tt> or the -DYYSTACKDEPTH compile-time flag. <a id='errors'></a> <h2>5.0 Error Processing</h2> <p>After extensive experimentation over several years, it has been discovered that the error recovery strategy used by yacc is about as good as it gets. And so that is what Lemon uses.</p> |
︙ | ︙ | |||
1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 | <p>If the parser pops its stack until the stack is empty, and it still is unable to shift the error symbol, then the <tt><a href='#parse_failure'>%parse_failure</a></tt> routine is invoked and the parser resets itself to its start state, ready to begin parsing a new file. This is what will happen at the very first syntax error, of course, if there are no instances of the "error" non-terminal in your grammar.</p> <a id='history'></a> <h2>6.0 History of Lemon</h2> <p>Lemon was originally written by Richard Hipp sometime in the late 1980s on a Sun4 Workstation using K&R C. | > | | | 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 | <p>If the parser pops its stack until the stack is empty, and it still is unable to shift the error symbol, then the <tt><a href='#parse_failure'>%parse_failure</a></tt> routine is invoked and the parser resets itself to its start state, ready to begin parsing a new file. This is what will happen at the very first syntax error, of course, if there are no instances of the "error" non-terminal in your grammar.</p> <a id='history'></a> <h2>6.0 History of Lemon</h2> <p>Lemon was originally written by Richard Hipp sometime in the late 1980s on a Sun4 Workstation using K&R C. There was a companion LL(1) parser generator program named "Lime". The Lime source code has been lost.</p> <p>The lemon.c source file was originally many separate files that were compiled together to generate the "lemon" executable. Sometime in the 1990s, the individual source code files were combined together into the current single large "lemon.c" source file. You can still see traces of original filenames in the code.</p> |
︙ | ︙ |
Changes to doc/pager-invariants.txt.
︙ | ︙ | |||
41 42 43 44 45 46 47 | (6) If a master journal file is used, then all writes to the database file are synced prior to the master journal being deleted. *** Definition: Two databases (or the same database at two points it time) are said to be "logically equivalent" if they give the same answer to all queries. Note in particular the content of freelist leaf | | | 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | (6) If a master journal file is used, then all writes to the database file are synced prior to the master journal being deleted. *** Definition: Two databases (or the same database at two points it time) are said to be "logically equivalent" if they give the same answer to all queries. Note in particular the content of freelist leaf pages can be changed arbitrarily without effecting the logical equivalence of the database. (7) At any time, if any subset, including the empty set and the total set, of the unsynced changes to a rollback journal are removed and the journal is rolled back, the resulting database file will be logical equivalent to the database file at the beginning of the transaction. |
︙ | ︙ |
Added doc/tcl-extension-testing.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 | # Test Procedures For The SQLite TCL Extension ## 1.0 Background The SQLite TCL extension logic (in the "[tclsqlite.c](/file/src/tclsqlite.c)" source file) is statically linked into "textfixture" executable which is the program used to do most of the testing associated with "make test", "make devtest", and/or "make releasetest". So the functionality of the SQLite TCL extension is thoroughly vetted during normal testing. The procedures below are designed to test the loadable extension aspect of the SQLite TCL extension, and in particular to verify that the "make tclextension-install" build target works and that an ordinary tclsh can subsequently run "package require sqlite3". This procedure can also be used as a template for how to set up a local TCL+SQLite development environment. In other words, it can be be used as a guide on how to compile per-user copies of Tcl that are used to develop, test, and debug SQLite. In that case, perhaps make minor changes to the procedure such as: * Make TCLBUILD directory is permanent. * Enable debugging symbols on the Tcl library build. * Reduce the optimization level to -O0 for easier debugging. * Also compile "wish" to go with each "tclsh". <a id="unix"></a> ## 2.0 Testing On Unix-like Systems (Including Mac) See also the [](./compile-for-unix.md) document which provides another perspective on how to compile SQLite on unix-like systems. ### 2.1 Setup <ol type="1"> <li value="1"> [Fossil][] installed. <li> Check out source code and set environment variables: <ol type="a"> <li> **TCLSOURCE** → The top-level directory of a [Fossil][] check-out of the [TCL source tree][tcl-fossil]. <li> **SQLITESOURCE** → A Fossil check-out of the SQLite source tree. <li> **TCLHOME** → A directory that does not exist at the start of the test and which will be deleted at the end of the test, and that will contain the test builds of the TCL libraries and the SQLite TCL Extensions. It is the top-most installation directory, i.e. the one provided to Tcl's `./configure --prefix=/path/to/tcl`. <li> **TCLVERSION** → The `X.Y`-form version of Tcl being used: 8.6, 9.0, 9.1... </ol> </ol> ### 2.2 Testing TCL 8.x and 9.x on unix From a checked-out copy of [the core Tcl tree][tcl-fossil] <ol type="1"> <li value="3">`TCLVERSION=8.6` <br> ↑ A version of your choice. This process has been tested with values of 8.6, 9.0, and 9.1 (as of 2025-04-16). The out-of-life version 8.5 fails some of `make devtest` for undetermined reasons. <li>`TCLHOME=$HOME/tcl/$TCLVERSION` <li>`TCLSOURCE=/path/to/tcl/checkout` <li>`SQLITESOURCE=/path/to/sqlite/checkout` <li>`rm -fr $TCLHOME` <br> ↑ Ensure that no stale Tcl installation is laying around. <li> `cd $TCLSOURCE` <li> `fossil up core-8-6-branch` <br> ↑ The branch corresponding to `$TCLVERSION`, e.g. `core-9-0-branch` or `trunk`. <li> `fossil clean -x` <li> `cd unix` <li> `./configure --prefix=$TCLHOME --disable-shared` <br> ↑ The `--disable-shared` is to avoid the need to set `LD_LIBRARY_PATH` when using this Tcl build. <li> `make install` <li> `cd $SQLITESOURCE` <li> `fossil clean -x` <li> `./configure --with-tcl=$TCLHOME --all` <li> `make tclextension-install` <br> ↑ Verify extension installed at `$TCLHOME/lib/tcl${TCLVERSION}/sqlite<SQLITE_VERSION>`. <li> `make tclextension-list` <br> ↑ Verify TCL extension correctly installed. <li> `make tclextension-verify` <br> ↑ Verify that the correct version is installed. <li> `$TCLHOME/bin/tclsh[89].[0-9] test/testrunner.tcl release --explain` <br> ↑ Verify thousands of lines of output with no errors. Or consider running "devtest" without --explain instead of "release". </ol> ### 2.3 Cleanup <ol type="1"> <li value="29"> `rm -rf $TCLHOME` </ol> <a id="windows"></a> ## 3.0 Testing On Windows See also the [](./compile-for-windows.md) document which provides another perspective on how to compile SQLite on Windows. ### 3.1 Setup for Windows (These docs are not as up-to-date as the Unix docs, above.) <ol type="1"> <li value="1"> [Fossil][] installed. <li> Unix-like command-line tools installed. Example: [unxutils](https://unxutils.sourceforge.net/) <li> [Visual Studio](https://visualstudio.microsoft.com/vs/community/) installed. VS2015 or later required. <li> Check out source code and set environment variables. <ol type="a"> <li> **TCLSOURCE** → The top-level directory of a Fossil check-out of the TCL source tree. <li> **SQLITESOURCE** → A Fossil check-out of the SQLite source tree. <li> **TCLBUILD** → A directory that does not exist at the start of the test and which will be deleted at the end of the test, and that will contain the test builds of the TCL libraries and the SQLite TCL Extensions. <li> **ORIGINALPATH** → The original value of %PATH%. In other words, set as follows: `set ORIGINALPATH %PATH%` </ol> </ol> ### 3.2 Testing TCL 8.6 on Windows <ol type="1"> <li value="5"> `mkdir %TCLBUILD%\tcl86` <li> `cd %TCLSOURCE%\win` <li> `fossil up core-8-6-16` <br> ↑ Or some other version of Tcl8.6. <li> `fossil clean -x` <li> `set INSTALLDIR=%TCLBUILD%\tcl86` <li> `nmake /f makefile.vc release` <br> ⇅ You *must* invoke the "release" and "install" targets using separate "nmake" commands or tclsh86t.exe won't be installed. <li> `nmake /f makefile.vc install` <li> `cd %SQLITESOURCE%` <li> `fossil clean -x` <li> `set TCLDIR=%TCLBUILD%\tcl86` <li> `set PATH=%TCLBUILD%\tcl86\bin;%ORIGINALPATH%` <li> `set TCLSH_CMD=%TCLBUILD%\tcl86\bin\tclsh86t.exe` <li> `nmake /f Makefile.msc tclextension-install` <br> ↑ Verify extension installed at %TCLBUILD%\\tcl86\\lib\\tcl8.6\\sqlite3.* <li> `nmake /f Makefile.msc tclextension-verify` <li>`tclsh86t test/testrunner.tcl release --explain` <br> ↑ Verify thousands of lines of output with no errors. Or consider running "devtest" without --explain instead of "release". </ol> ### 3.3 Testing TCL 9.0 on Windows <ol> <li value="20"> `mkdir %TCLBUILD%\tcl90` <li> `cd %TCLSOURCE%\win` <li> `fossil up core-9-0-0` <br> ↑ Or some other version of Tcl9 <li> `fossil clean -x` <li> `set INSTALLDIR=%TCLBUILD%\tcl90` <li> `nmake /f makefile.vc release` <br> ⇅ You *must* invoke the "release" and "install" targets using separate "nmake" commands or tclsh90.exe won't be installed. <li> `nmake /f makefile.vc install` <li> `cd %SQLITESOURCE%` <li> `fossil clean -x` <li> `set TCLDIR=%TCLBUILD%\tcl90` <li> `set PATH=%TCLBUILD%\tcl90\bin;%ORIGINALPATH%` <li> `set TCLSH_CMD=%TCLBUILD%\tcl90\bin\tclsh90.exe` <li> `nmake /f Makefile.msc tclextension-install` <br> ↑ Verify extension installed at %TCLBUILD%\\tcl90\\lib\\sqlite3.* <li> `nmake /f Makefile.msc tclextension-verify` <li> `tclsh90 test/testrunner.tcl release --explain` <br> ↑ Verify thousands of lines of output with no errors. Or consider running "devtest" without --explain instead of "release". </ol> ### 3.4 Cleanup <ol type="1"> <li value="35"> `rm -rf %TCLBUILD%` </ol> ## 4.0 Testing the TEA(ish) Build (unix only) This part requires following the setup instructions for Unix systems, at the top of this document. The former TEA, now TEA(ish), build of this extension uses the same code as the builds described above but is provided in a form more convenient for downstream Tcl users. It lives in `autoconf/tea` and, as part of the `autoconf` bundle, _cannot be tested directly from the canonical tree_. Instead it has to be packaged. ### 4.1 Teaish Setup Follow the same Tcl- and environment-related related setup described in the first section of this document, up to and including the installation of Tcl (unless, of course, it was already installed using those same instructions). ### 4.2 Teaish Testing <ol> <li>`cd $SQLITESOURCE` <li>Run either `make snapshot-tarball` or `make amalgamation-tarball` ↑ Those steps will leave behind a temp dir called `mkpkg_tmp_dir`, under which the extension is most readily reached. It can optionally be extracted from the generated tarball, but that tarball was generated from this dir, and reusing this dir is a time saver during development. <li> `cd mkpkg_tmp/tea` <li> `./configure --with-tcl=$TCLHOME` <li> `make test install` <br> ↑ Should run to completion without any errors. <li> `make uninstall` <br> ↑ Will uninstall the extension. This _can_ be run in the same invocation as the `install` target, but only if the `-j#` make flag is _not_ used. If it is, the install/uninstall steps will race and make a mess of things. Parallel builds do not help in this build, anyway, as there's only a single C file to compile. </ol> When actively developing and testing the teaish build, which requires going through the tarball generation, there's a caveat about the `mkpkg_tmp_dir` dir: it will be deleted every time a tarball is built, the shell console which is parked in that directory for testing needs to add `cd $PWD &&` to the start of the build commands, like: > ``` [user@host:.../mkpkg_tmp_dir/tea]$ \ cd $PWD && ./configure CFLAGS=-O0 --with-tcl=$TCLHOME \ && make test install uninstall ``` ### 4.3 Teaish Cleanup <ol type="1"> <li> `rm -rf $TCLHOME` <li> `cd $SQLITESOURCE; rm -fr mkpkg_tmp_dir; fossil clean -x` </ol> [Fossil]: https://fossil-scm.org/home [tcl-fossil]: https://core.tcl-lang.org/tcl |
Added doc/testrunner.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 | # The testrunner.tcl Script <ul type=none> <li> 1. <a href=#overview>Overview</a> <li> 2. <a href=#binary_tests>Binary Tests</a> <ul type=none> <li> 2.1. <a href=#organization_tests>Organization of Tcl Tests</a> <li> 2.2. <a href=#run_tests>Commands to Run Tests</a> <li> 2.3. <a href=#binary_test_failures>Investigating Binary Test Failures</a> </ul> <li> 3. <a href=#source_code_tests>Source Tests</a> <ul type=none> <li> 3.1. <a href=#commands_to_run_tests>Commands to Run SQLite Tests</a> <li> 3.2. <a href=#zipvfs_tests>Running ZipVFS Tests</a> <li> 3.3. <a href=#source_code_test_failures>Investigating Source Code Test Failures</a> <li> 3.4. <a href=#fuzzdb>External Fuzzcheck Databases</a> </ul> <li> 4. <a href=#testrunner_options>Extra testrunner.tcl Options</a> <li> 5. <a href=#cpu_cores>Controlling CPU Core Utilization</a> </ul> <a name=overview></a> # 1. Overview The testrunner.tcl program is a Tcl script used to run multiple SQLite tests in parallel, thus reducing testing time on multi-core machines. It supports the following types of tests: * Tcl test scripts. * Fuzzcheck tests, including using an external fuzzcheck database. * Tests run with `make` commands. Examples: - `make devtest` - `make releasetest` - `make sdevtest` - `make testrunner` The testrunner.tcl program stores output of all tests and builds run in log file **testrunner.log**, created in the current working directory. Search this file to find details of errors. Suggested search commands: * `grep "^!" testrunner.log` * `grep failed testrunner.log` The testrunner.tcl program also populates SQLite database **testrunner.db**. This database contains details of all tests run, running and to be run. A useful query might be: ``` SELECT * FROM script WHERE state='failed' ``` You can get a summary of errors in a prior run by invoking commands like these: ``` tclsh $(TESTDIR)/testrunner.tcl errors tclsh $(TESTDIR)/testrunner.tcl errors -v ``` Running the command: ``` tclsh $(TESTDIR)/testrunner.tcl status ``` in the directory containing the testrunner.db database runs various queries to produce a succinct report on the state of a running testrunner.tcl script. A good way to keep and eye on test progress is to run either of the two following commands: ``` watch tclsh $(TESTDIR)/testrunner.tcl status tclsh $(TESTDIR)/testrunner.tcl status -d 2 ``` Both of the commands above accomplish about the same thing, but the second one has the advantage of not requiring "watch" to be installed on your system. Sometimes testrunner.tcl uses the `testfixture` binary that it is run with to run tests (see "Binary Tests" below). Sometimes it builds testfixture and other binaries in specific configurations to test (see "Source Tests"). <a name=binary_tests></a> # 2. Binary Tests The commands described in this section all run various combinations of the Tcl test scripts using the `testfixture` binary used to run the testrunner.tcl script (i.e. they do not invoke the compiler to build new binaries, or the `make` command to run tests that are not Tcl scripts). The procedure to run these tests is therefore: 1. Build the "testfixture" (or "testfixture.exe" for windows) binary using whatever method seems convenient. 2. Test the binary built in step 1 by running testrunner.tcl with it, perhaps with various options. The following sub-sections describe the various options that can be passed to testrunner.tcl to test binary testfixture builds. <a name=organization_tests></a> ## 2.1. Organization of Tcl Tests Tcl tests are stored in files that match the pattern *\*.test*. They are found in both the $TOP/test/ directory, and in the various sub-directories of the $TOP/ext/ directory of the source tree. Not all *\*.test* files contain Tcl tests - a handful are Tcl scripts designed to invoke other *\*.test* files. The **veryquick** set of tests is a subset of all Tcl test scripts in the source tree. In includes most tests, but excludes some that are very slow. Almost all fault-injection tests (those that test the response of the library to OOM or IO errors) are excluded. It is defined in source file *test/permutations.test*. The **full** set of tests includes all Tcl test scripts in the source tree. To run a "full" test is to run all Tcl test scripts that can be found in the source tree. File *permutations.test* defines various test "permutations". A permutation consists of: * A subset of Tcl test scripts, and * Runtime configuration to apply before running each test script (e.g. enabling auto-vacuum, or disable lookaside). Running **all** tests is to run all tests in the full test set, plus a dozen or so permutations. The specific permutations that are run as part of "all" are defined in file *testrunner_data.tcl*. <a name=run_tests></a> ## 2.2. Commands to Run Tests To run the "veryquick" test set, use either of the following: ``` ./testfixture $TESTDIR/testrunner.tcl ./testfixture $TESTDIR/testrunner.tcl veryquick ``` To run the "full" test suite: ``` ./testfixture $TESTDIR/testrunner.tcl full ``` To run the subset of the "full" test suite for which the test file name matches a specified pattern (e.g. all tests that start with "fts5"), either of: ``` ./testfixture $TESTDIR/testrunner.tcl fts5% ./testfixture $TESTDIR/testrunner.tcl 'fts5*' ``` Strictly speaking, for a test to be run the pattern must match the script filename, not including the directory, using the rules of Tcl's \[string match\] command. Except that before the matching is done, any "%" characters specified as part of the pattern are transformed to "\*". To run "all" tests (full + permutations): ``` ./testfixture $TESTDIR/testrunner.tcl all ``` <a name=binary_test_failures></a> ## 2.3. Investigating Binary Test Failures If a test fails, testrunner.tcl reports name of the Tcl test script and, if applicable, the name of the permutation, to stdout. This information can also be retrieved from either *testrunner.log* or *testrunner.db*. If there is no permutation, the individual test script may be run with: ``` ./testfixture $PATH_TO_SCRIPT ``` Or, if the failure occured as part of a permutation: ``` ./testfixture $TESTDIR/testrunner.tcl $PERMUTATION $PATH_TO_SCRIPT ``` TODO: An example instead of "$PERMUTATION" and $PATH\_TO\_SCRIPT? <a name=source_code_tests></a> # 3. Source Code Tests The commands described in this section invoke the C compiler to build binaries from the source tree, then use those binaries to run Tcl and other tests. The advantages of this are that: * it is possible to test multiple build configurations with a single command, and * it ensures that tests are always run using binaries created with the same set of compiler options. The testrunner.tcl commands described in this section may be run using either a *testfixture* (or testfixture.exe) build, or with any other Tcl shell that supports SQLite 3.31.1 or newer via "package require sqlite3". TODO: ./configure + Makefile.msc build systems. <a name=commands_to_run_tests></a> ## 3.1. Commands to Run SQLite Tests The **mdevtest** command is equivalent to running the veryquick tests and the `make fuzztest` target once for each of two --enable-all builds - one with debugging enabled and one without: ``` tclsh $TESTDIR/testrunner.tcl mdevtest ``` In other words, it is equivalent to running: ``` $TOP/configure --enable-all --enable-debug make fuzztest make testfixture ./testfixture $TOP/test/testrunner.tcl veryquick # Then, after removing files created by the tests above: $TOP/configure --enable-all OPTS="-O0" make fuzztest make testfixture ./testfixture $TOP/test/testrunner.tcl veryquick ``` The **sdevtest** command is identical to the mdevtest command, except that the second of the two builds is a sanitizer build. Specifically, this means that OPTS="-fsanitize=address,undefined" is specified instead of OPTS="-O0": ``` tclsh $TESTDIR/testrunner.tcl sdevtest ``` The **release** command runs lots of tests under lots of builds. It runs different combinations of builds and tests depending on whether it is run on Linux, Windows or OSX. Refer to *testrunner\_data.tcl* for the details of the specific tests run. ``` tclsh $TESTDIR/testrunner.tcl release ``` As with <a href=#source code tests>source code tests</a>, one or more patterns may be appended to any of the above commands (mdevtest, sdevtest or release). Pattern matching is used for both Tcl tests and fuzz tests. ``` tclsh $TESTDIR/testrunner.tcl release rtree% ``` <a name=zipvfs_tests></a> ## 3.2. Running ZipVFS Tests testrunner.tcl can build a zipvfs-enabled testfixture and use it to run tests from the Zipvfs project with the following command: ``` tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS ``` This can be combined with any of "mdevtest", "sdevtest" or "release" to test both SQLite and Zipvfs with a single command: ``` tclsh $TESTDIR/testrunner.tcl --zipvfs $PATH_TO_ZIPVFS mdevtest ``` <a name=source_code_test_failures></a> ## 3.3. Investigating Source Code Test Failures Investigating a test failure that occurs during source code testing is a two step process: 1. Recreating the build configuration in which the test failed, and 2. Re-running the actual test. To recreate a build configuration, use the testrunner.tcl **script** command to create a build script. A build script is a bash script on Linux or OSX, or a dos \*.bat file on windows. For example: ``` # Create a script that recreates build configuration "Device-One" on # Linux or OSX: tclsh $TESTDIR/testrunner.tcl script Device-One > make.sh # Create a script that recreates build configuration "Have-Not" on Windows: tclsh $TESTDIR/testrunner.tcl script Have-Not > make.bat ``` The generated bash or \*.bat file script accepts a single argument - a makefile target to build. This may be used either to run a `make` command test directly, or else to build a testfixture (or testfixture.exe) binary with which to run a Tcl test script, as <a href=#binary_test_failures>described above</a>. <a name=fuzzdb></a> ## 3.4 External Fuzzcheck Databases Testrunner.tcl will also run fuzzcheck against an external (out of tree) database, for example fuzzcheck databases generated by dbsqlfuzz. To do this, simply add the "`--fuzzdb` *FILENAME*" command-line option or set the FUZZDB environment variable to the name of the external database. For large external databases, testrunner.tcl will automatically use the "`--slice`" command-line option of fuzzcheck to divide the work up into multiple jobs, to increase parallelism. Thus, for example, to run a full releasetest including an external dbsqlfuzz database, run a command like one of these: ``` tclsh test/testrunner.tcl releasetest --fuzzdb ../fuzz/20250415.db FUZZDB=../fuzz/20250415.db make releasetest nmake /f Makefile.msc FUZZDB=../fuzz/20250415.db releasetest ``` The patternlist option to testrunner.tcl will match against fuzzcheck databases. So if you want to run *only* tests involving the external database, you can use a command something like this: ``` tclsh test/testrunner.tcl releasetest 20250415 --fuzzdb ../fuzz/20250415.db ``` <a name=testrunner_options></a> # 4. Extra testrunner.tcl Options The testrunner.tcl script options in this section may be used with both source code and binary tests. The **--buildonly** option instructs testrunner.tcl just to build the binaries required by a test, not to run any actual tests. For example: ``` # Build binaries required by release test. tclsh $TESTDIR/testrunner.tcl --buildonly release" ``` The **--dryrun** option prevents testrunner.tcl from building any binaries or running any tests. Instead, it just writes the shell commands that it would normally execute into the testrunner.log file. Example: ``` # Log the shell commmands that make up the mdevtest test. tclsh $TESTDIR/testrunner.tcl --dryrun mdevtest" ``` The **--explain** option is similar to --dryrun in that it prevents testrunner.tcl from building any binaries or running any tests. The difference is that --explain prints on standard output a human-readable summary of all the builds and tests that would have been run. ``` # Show what builds and tests would have been run tclsh $TESTDIR/testrunner.tcl --explain mdevtest ``` The **--status** option uses VT100 escape sequences to display the test status full-screen. This is similar to running "`watch test/testrunner status`" in a separate window, just more convenient. Unfortunately, this option does not work correctly on Windows, due to the sketchy implementation of VT100 escapes on the Windows console. <a name=cpu_cores></a> # 5. Controlling CPU Core Utilization When running either binary or source code tests, testrunner.tcl reports the number of jobs it intends to use to stdout. e.g. ``` $ ./testfixture $TESTDIR/testrunner.tcl splitting work across 16 jobs ... more output ... ``` By default, testfixture.tcl attempts to set the number of jobs to the number of real cores on the machine. This can be overridden using the "--jobs" (or -j) switch: ``` $ ./testfixture $TESTDIR/testrunner.tcl --jobs 8 splitting work across 8 jobs ... more output ... ``` The number of jobs may also be changed while an instance of testrunner.tcl is running by exucuting the following command from the directory containing the testrunner.log and testrunner.db files: ``` $ ./testfixture $TESTDIR/testrunner.tcl njob $NEW_NUMBER_OF_JOBS ``` |
Changes to doc/vfs-shm.txt.
1 2 | The 5 states of an historical rollback lock as implemented by the xLock, xUnlock, and xCheckReservedLock methods of the sqlite3_io_methods | | | 1 2 3 4 5 6 7 8 9 10 | The 5 states of an historical rollback lock as implemented by the xLock, xUnlock, and xCheckReservedLock methods of the sqlite3_io_methods object are: UNLOCKED SHARED RESERVED PENDING EXCLUSIVE |
︙ | ︙ | |||
54 55 56 57 58 59 60 | RECOVER - Held during wal-index recovery. Used to prevent a race if multiple clients try to recover a wal-index at the same time. A particular lock manager implementation may coalesce one or more of the wal-index locking states, though with a reduction in concurrency. | | | 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 | RECOVER - Held during wal-index recovery. Used to prevent a race if multiple clients try to recover a wal-index at the same time. A particular lock manager implementation may coalesce one or more of the wal-index locking states, though with a reduction in concurrency. For example, an implementation might implement only exclusive locking, in which case all states would be equivalent to CHECKPOINT, meaning that only one reader or one writer or one checkpointer could be active at a time. Or, an implementation might combine READ and READ_FULL into a single state equivalent to READ, meaning that a writer could coexist with a reader, but no reader or writers could coexist with a checkpointer. |
︙ | ︙ |
Changes to doc/wal-lock.md.
︙ | ︙ | |||
8 9 10 11 12 13 14 | Blocking locks may be advantageous as (a) waiting database clients do not need to continuously poll the database lock, and (b) using blocking locks facilitates transfer of OS priority between processes when a high priority process is blocked by a lower priority one. Only read/write clients use blocking locks. Clients that have read-only access | | | 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | Blocking locks may be advantageous as (a) waiting database clients do not need to continuously poll the database lock, and (b) using blocking locks facilitates transfer of OS priority between processes when a high priority process is blocked by a lower priority one. Only read/write clients use blocking locks. Clients that have read-only access to the \*-shm file never use blocking locks. Threads or processes that access a single database at a time never deadlock as a result of blocking database locks. But it is of course possible for threads that lock multiple databases simultaneously to do so. In most cases the OS will detect the deadlock and return an error. ## Wal Recovery |
︙ | ︙ |
Changes to ext/README.md.
1 2 | ## Loadable Extensions | | | 1 2 3 4 5 6 7 8 | ## Loadable Extensions Various [loadable extensions](https://sqlite.org/loadext.html) for SQLite are found in subfolders. Most subfolders are dedicated to a single loadable extension (for example FTS5, or RTREE). But the misc/ subfolder contains a collection of smaller single-file extensions. |
Deleted ext/async/README.txt.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/async/sqlite3async.c.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Deleted ext/async/sqlite3async.h.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to ext/expert/expert1.test.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 | # 2009 Nov 11 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # The focus of this file is testing the CLI shell tool. Specifically, # the ".recommend" command. # # # Test plan: | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 | # 2009 Nov 11 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # TESTRUNNER: shell # # The focus of this file is testing the CLI shell tool. Specifically, # the ".recommend" command. # # # Test plan: |
︙ | ︙ | |||
397 398 399 400 401 402 403 404 405 406 407 408 409 410 | } { SELECT * FROM t1 ORDER BY "index"; } { CREATE INDEX t1_idx_01a7214e ON t1('index'); SCAN t1 USING COVERING INDEX t1_idx_01a7214e } } proc do_candidates_test {tn sql res} { set res [squish [string trim $res]] set expert [sqlite3_expert_new db] $expert sql $sql | > > > > > > > > > > > > > | 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 | } { SELECT * FROM t1 ORDER BY "index"; } { CREATE INDEX t1_idx_01a7214e ON t1('index'); SCAN t1 USING COVERING INDEX t1_idx_01a7214e } ifcapable fts5 { do_setup_rec_test $tn.20.0 { CREATE VIRTUAL TABLE ft USING fts5(a); CREATE TABLE t1(x, y); } { SELECT * FROM ft, t1 WHERE a=x } { CREATE INDEX t1_idx_00000078 ON t1(x); SCAN ft VIRTUAL TABLE INDEX 0: SEARCH t1 USING INDEX t1_idx_00000078 (x=?) } } } proc do_candidates_test {tn sql res} { set res [squish [string trim $res]] set expert [sqlite3_expert_new db] $expert sql $sql |
︙ | ︙ | |||
423 424 425 426 427 428 429 430 431 432 433 434 435 436 | CREATE TABLE t2(c, d); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t1 SELECT (i-1)/50, (i-1)/20 FROM s; WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t2 SELECT (i-1)/20, (i-1)/5 FROM s; } do_candidates_test 5.1 { SELECT * FROM t1,t2 WHERE (b=? OR a=?) AND (c=? OR d=?) } { CREATE INDEX t1_idx_00000062 ON t1(b); -- stat1: 100 20 CREATE INDEX t1_idx_00000061 ON t1(a); -- stat1: 100 50 CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 | > > | 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 | CREATE TABLE t2(c, d); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t1 SELECT (i-1)/50, (i-1)/20 FROM s; WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) INSERT INTO t2 SELECT (i-1)/20, (i-1)/5 FROM s; CREATE INDEX i1 ON t1( lower(a) ); } do_candidates_test 5.1 { SELECT * FROM t1,t2 WHERE (b=? OR a=?) AND (c=? OR d=?) } { CREATE INDEX t1_idx_00000062 ON t1(b); -- stat1: 100 20 CREATE INDEX t1_idx_00000061 ON t1(a); -- stat1: 100 50 CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 |
︙ | ︙ | |||
452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 | CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 CREATE INDEX t2_idx_00000064 ON t2(d); -- stat1: 100 5 CREATE INDEX t2_idx_0001295b ON t2(c, d); -- stat1: 100 20 5 ANALYZE; SELECT * FROM sqlite_stat1 ORDER BY 1, 2; } { t1 t1_idx_00000061 {100 50} t1 t1_idx_00000062 {100 20} t1 t1_idx_000123a7 {100 50 17} t2 t2_idx_00000063 {100 20} t2 t2_idx_00000064 {100 5} t2 t2_idx_0001295b {100 20 5} } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 | CREATE INDEX t2_idx_00000063 ON t2(c); -- stat1: 100 20 CREATE INDEX t2_idx_00000064 ON t2(d); -- stat1: 100 5 CREATE INDEX t2_idx_0001295b ON t2(c, d); -- stat1: 100 20 5 ANALYZE; SELECT * FROM sqlite_stat1 ORDER BY 1, 2; } { t1 i1 {100 50} t1 t1_idx_00000061 {100 50} t1 t1_idx_00000062 {100 20} t1 t1_idx_000123a7 {100 50 17} t2 t2_idx_00000063 {100 20} t2 t2_idx_00000064 {100 5} t2 t2_idx_0001295b {100 20 5} } do_catchsql_test 5.4 { SELECT sqlite_expert_rem(123, 123); } {1 {no such function: sqlite_expert_rem}} do_catchsql_test 5.5 { SELECT sqlite_expert_sample(); } {1 {no such function: sqlite_expert_sample}} if 0 { do_test expert1-6.0 { catchcmd :memory: { .expert select base64(''); .expert select name from pragma_collation_list order by name collate uint; } } {0 {(no new indexes) SCAN CONSTANT ROW (no new indexes) SCAN pragma_collation_list VIRTUAL TABLE INDEX 0: USE TEMP B-TREE FOR ORDER BY }} } do_execsql_test 6.0 { CREATE TABLE x1(a, b, c, d); CREATE INDEX x1ab ON x1(a, lower(b)); CREATE INDEX x1dcba ON x1(d, b+c, a); } do_candidates_test 6.1 { SELECT * FROM x1 WHERE b=? ORDER BY a; } { CREATE INDEX x1_idx_0001267f ON x1(b, a); CREATE INDEX x1_idx_00000062 ON x1(b); } #------------------------------------------------------------------------- ifcapable fts5 { reset_db do_execsql_test 7.0 { CREATE VIRTUAL TABLE ft USING fts5(a); CREATE TABLE t1(x, y); } do_candidates_test 7.1 { SELECT * FROM ft, t1 WHERE a=x } { CREATE INDEX t1_idx_00000078 ON t1(x); } register_tcl_module db proc vtab_command {method args} { global G switch -- $method { xConnect { return "CREATE TABLE t1(a, b, c);" } xBestIndex { return [list] } xFilter { return [list sql "SELECT rowid, * FROM t0"] } } return {} } do_execsql_test 7.2 { CREATE TABLE t0(a, b, c); INSERT INTO t0 VALUES(1, 2, 3), (11, 22, 33); CREATE VIRTUAL TABLE t2 USING tcl(vtab_command); } do_execsql_test 7.3 { SELECT * FROM t2 } { 1 2 3 11 22 33 } do_candidates_test 7.4 { SELECT * FROM ft, t1 WHERE a=x } { CREATE INDEX t1_idx_00000078 ON t1(x); } do_test 7.5 { set expert [sqlite3_expert_new db] list [catch { $expert sql "SELECT * FROM ft, t2 WHERE b=1" } msg] $msg } {1 {no such table: t2}} $expert destroy reset_db do_execsql_test 7.6 { BEGIN TRANSACTION; CREATE TABLE IF NOT EXISTS 'bfts_idx_data'(id INTEGER PRIMARY KEY, block BLOB); CREATE TABLE IF NOT EXISTS 'fts_idx_data'(id INTEGER PRIMARY KEY, block BLOB); INSERT INTO fts_idx_data VALUES(1,X''); INSERT INTO fts_idx_data VALUES(10,X'00000000ff000001000000'); CREATE TABLE IF NOT EXISTS 'fts_idx_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID; CREATE TABLE IF NOT EXISTS 'fts_idx_docsize'(id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER); CREATE TABLE IF NOT EXISTS 'fts_idx_config'(k PRIMARY KEY, v) WITHOUT ROWID; INSERT INTO fts_idx_config VALUES('version',4); PRAGMA writable_schema=ON; INSERT INTO sqlite_schema(type,name,tbl_name,rootpage,sql)VALUES('table','fts_idx','fts_idx',0,'CREATE VIRTUAL TABLE fts_idx USING fts5(Title, Description, Channel, Tags, content='''', contentless_delete=1)'); CREATE TABLE f(x BLOB, y); COMMIT; PRAGMA writable_schema = RESET; } do_candidates_test 7.4 { SELECT * FROM fts_idx, f WHERE x = fts_idx.Channel } { CREATE INDEX f_idx_00000078 ON f(x); } } finish_test |
Changes to ext/expert/sqlite3expert.c.
︙ | ︙ | |||
28 29 30 31 32 33 34 | #else # define ALWAYS(X) (X) # define NEVER(X) (X) #endif #endif /* !defined(SQLITE_AMALGAMATION) */ | | | 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | #else # define ALWAYS(X) (X) # define NEVER(X) (X) #endif #endif /* !defined(SQLITE_AMALGAMATION) */ #ifndef SQLITE_OMIT_VIRTUALTABLE typedef sqlite3_int64 i64; typedef sqlite3_uint64 u64; typedef struct IdxColumn IdxColumn; typedef struct IdxConstraint IdxConstraint; typedef struct IdxScan IdxScan; |
︙ | ︙ | |||
622 623 624 625 626 627 628 | (void)idxStr; (void)argc; (void)argv; rc = sqlite3_finalize(pCsr->pData); pCsr->pData = 0; if( rc==SQLITE_OK ){ rc = idxPrintfPrepareStmt(pExpert->db, &pCsr->pData, &pVtab->base.zErrMsg, | | | 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 | (void)idxStr; (void)argc; (void)argv; rc = sqlite3_finalize(pCsr->pData); pCsr->pData = 0; if( rc==SQLITE_OK ){ rc = idxPrintfPrepareStmt(pExpert->db, &pCsr->pData, &pVtab->base.zErrMsg, "SELECT * FROM main.%Q WHERE sqlite_expert_sample()", pVtab->pTab->zName ); } if( rc==SQLITE_OK ){ rc = expertNext(cur); } return rc; |
︙ | ︙ | |||
658 659 660 661 662 663 664 665 666 667 668 669 670 671 | 0, /* xRollback - rollback transaction */ 0, /* xFindFunction - function overloading */ 0, /* xRename - rename the table */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ }; return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p); } /* ** End of virtual table implementation. *************************************************************************/ | > | 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 | 0, /* xRollback - rollback transaction */ 0, /* xFindFunction - function overloading */ 0, /* xRename - rename the table */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0, /* xIntegrity */ }; return sqlite3_create_module(p->dbv, "expert", &expertModule, (void*)p); } /* ** End of virtual table implementation. *************************************************************************/ |
︙ | ︙ | |||
1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 | pEnd = pFirst; pFirst = p->pWrite; } return rc; } static int idxCreateVtabSchema(sqlite3expert *p, char **pzErrmsg){ int rc = idxRegisterVtab(p); sqlite3_stmt *pSchema = 0; /* For each table in the main db schema: ** ** 1) Add an entry to the p->pTable list, and ** 2) Create the equivalent virtual table in dbv. */ rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg, | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > | > | | > > > > > | > | | | 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 | pEnd = pFirst; pFirst = p->pWrite; } return rc; } /* ** This function tests if the schema of the main database of database handle ** db contains an object named zTab. Assuming no error occurs, output parameter ** (*pbContains) is set to true if zTab exists, or false if it does not. ** ** Or, if an error occurs, an SQLite error code is returned. The final value ** of (*pbContains) is undefined in this case. */ static int expertDbContainsObject( sqlite3 *db, const char *zTab, int *pbContains /* OUT: True if object exists */ ){ const char *zSql = "SELECT 1 FROM sqlite_schema WHERE name = ?"; sqlite3_stmt *pSql = 0; int rc = SQLITE_OK; int ret = 0; rc = sqlite3_prepare_v2(db, zSql, -1, &pSql, 0); if( rc==SQLITE_OK ){ sqlite3_bind_text(pSql, 1, zTab, -1, SQLITE_STATIC); if( SQLITE_ROW==sqlite3_step(pSql) ){ ret = 1; } rc = sqlite3_finalize(pSql); } *pbContains = ret; return rc; } /* ** Execute SQL command zSql using database handle db. If no error occurs, ** set (*pzErr) to NULL and return SQLITE_OK. ** ** If an error does occur, return an SQLite error code and set (*pzErr) to ** point to a buffer containing an English language error message. Except, ** if the error message begins with "no such module:", then ignore the ** error and return as if the SQL statement had succeeded. ** ** This is used to copy as much of the database schema as possible while ** ignoring any errors related to missing virtual table modules. */ static int expertSchemaSql(sqlite3 *db, const char *zSql, char **pzErr){ int rc = SQLITE_OK; char *zErr = 0; rc = sqlite3_exec(db, zSql, 0, 0, &zErr); if( rc!=SQLITE_OK && zErr ){ int nErr = STRLEN(zErr); if( nErr>=15 && memcmp(zErr, "no such module:", 15)==0 ){ sqlite3_free(zErr); rc = SQLITE_OK; zErr = 0; } } *pzErr = zErr; return rc; } static int idxCreateVtabSchema(sqlite3expert *p, char **pzErrmsg){ int rc = idxRegisterVtab(p); sqlite3_stmt *pSchema = 0; /* For each table in the main db schema: ** ** 1) Add an entry to the p->pTable list, and ** 2) Create the equivalent virtual table in dbv. */ rc = idxPrepareStmt(p->db, &pSchema, pzErrmsg, "SELECT type, name, sql, 1, " " substr(sql,1,14)=='create virtual' COLLATE nocase " "FROM sqlite_schema " "WHERE type IN ('table','view') AND " " substr(name,1,7)!='sqlite_' COLLATE nocase " " UNION ALL " "SELECT type, name, sql, 2, 0 FROM sqlite_schema " "WHERE type = 'trigger'" " AND tbl_name IN(SELECT name FROM sqlite_schema WHERE type = 'view') " "ORDER BY 4, 5 DESC, 1" ); while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSchema) ){ const char *zType = (const char*)sqlite3_column_text(pSchema, 0); const char *zName = (const char*)sqlite3_column_text(pSchema, 1); const char *zSql = (const char*)sqlite3_column_text(pSchema, 2); int bVirtual = sqlite3_column_int(pSchema, 4); int bExists = 0; if( zType==0 || zName==0 ) continue; rc = expertDbContainsObject(p->dbv, zName, &bExists); if( rc || bExists ) continue; if( zType[0]=='v' || zType[1]=='r' || bVirtual ){ /* A view. Or a trigger on a view. */ if( zSql ) rc = expertSchemaSql(p->dbv, zSql, pzErrmsg); }else{ IdxTable *pTab; rc = idxGetTableInfo(p->db, zName, &pTab, pzErrmsg); if( rc==SQLITE_OK && ALWAYS(pTab!=0) ){ int i; char *zInner = 0; char *zOuter = 0; pTab->pNext = p->pTable; p->pTable = pTab; /* The statement the vtab will pass to sqlite3_declare_vtab() */ |
︙ | ︙ | |||
1495 1496 1497 1498 1499 1500 1501 | int nByte; /* Bytes of space allocated at z */ int n; /* Size of buffer z */ char *z; /* SQLITE_TEXT/BLOB value */ } aSlot[1]; }; /* | | | | 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 | int nByte; /* Bytes of space allocated at z */ int n; /* Size of buffer z */ char *z; /* SQLITE_TEXT/BLOB value */ } aSlot[1]; }; /* ** Implementation of scalar function sqlite_expert_rem(). */ static void idxRemFunc( sqlite3_context *pCtx, int argc, sqlite3_value **argv ){ struct IdxRemCtx *p = (struct IdxRemCtx*)sqlite3_user_data(pCtx); struct IdxRemSlot *pSlot; int iSlot; assert( argc==2 ); iSlot = sqlite3_value_int(argv[0]); assert( iSlot<p->nSlot ); pSlot = &p->aSlot[iSlot]; switch( pSlot->eType ){ case SQLITE_NULL: /* no-op */ break; |
︙ | ︙ | |||
1618 1619 1620 1621 1622 1623 1624 1625 | /* Formulate the query text */ sqlite3_bind_text(pIndexXInfo, 1, zIdx, -1, SQLITE_STATIC); while( SQLITE_OK==rc && SQLITE_ROW==sqlite3_step(pIndexXInfo) ){ const char *zComma = zCols==0 ? "" : ", "; const char *zName = (const char*)sqlite3_column_text(pIndexXInfo, 0); const char *zColl = (const char*)sqlite3_column_text(pIndexXInfo, 1); zCols = idxAppendText(&rc, zCols, | > > > > > > > | | 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 | /* Formulate the query text */ sqlite3_bind_text(pIndexXInfo, 1, zIdx, -1, SQLITE_STATIC); while( SQLITE_OK==rc && SQLITE_ROW==sqlite3_step(pIndexXInfo) ){ const char *zComma = zCols==0 ? "" : ", "; const char *zName = (const char*)sqlite3_column_text(pIndexXInfo, 0); const char *zColl = (const char*)sqlite3_column_text(pIndexXInfo, 1); if( zName==0 ){ /* This index contains an expression. Ignore it. */ sqlite3_free(zCols); sqlite3_free(zOrder); return sqlite3_reset(pIndexXInfo); } zCols = idxAppendText(&rc, zCols, "%sx.%Q IS sqlite_expert_rem(%d, x.%Q) COLLATE %s", zComma, zName, nCol, zName, zColl ); zOrder = idxAppendText(&rc, zOrder, "%s%d", zComma, ++nCol); } sqlite3_reset(pIndexXInfo); if( rc==SQLITE_OK ){ if( p->iSample==100 ){ zQuery = sqlite3_mprintf( |
︙ | ︙ | |||
1752 1753 1754 1755 1756 1757 1758 | if( rc==SQLITE_OK ){ int nByte = sizeof(struct IdxRemCtx) + (sizeof(struct IdxRemSlot) * nMax); pCtx = (struct IdxRemCtx*)idxMalloc(&rc, nByte); } if( rc==SQLITE_OK ){ sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv); | | | | | | 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 | if( rc==SQLITE_OK ){ int nByte = sizeof(struct IdxRemCtx) + (sizeof(struct IdxRemSlot) * nMax); pCtx = (struct IdxRemCtx*)idxMalloc(&rc, nByte); } if( rc==SQLITE_OK ){ sqlite3 *dbrem = (p->iSample==100 ? p->db : p->dbv); rc = sqlite3_create_function(dbrem, "sqlite_expert_rem", 2, SQLITE_UTF8, (void*)pCtx, idxRemFunc, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(p->db, "sqlite_expert_sample", 0, SQLITE_UTF8, (void*)&samplectx, idxSampleFunc, 0, 0 ); } if( rc==SQLITE_OK ){ pCtx->nSlot = nMax+1; rc = idxPrepareStmt(p->dbm, &pAllIndex, pzErr, zAllIndex); } |
︙ | ︙ | |||
1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 | sqlite3_free(pCtx); } if( rc==SQLITE_OK ){ rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0); } sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0); return rc; } /* ** Allocate a new sqlite3expert object. */ sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){ int rc = SQLITE_OK; sqlite3expert *pNew; | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 | sqlite3_free(pCtx); } if( rc==SQLITE_OK ){ rc = sqlite3_exec(p->dbm, "ANALYZE sqlite_schema", 0, 0, 0); } sqlite3_create_function(p->db, "sqlite_expert_rem", 2, SQLITE_UTF8, 0,0,0,0); sqlite3_create_function(p->db, "sqlite_expert_sample", 0,SQLITE_UTF8,0,0,0,0); sqlite3_exec(p->db, "DROP TABLE IF EXISTS temp."UNIQUE_TABLE_NAME,0,0,0); return rc; } /* ** Define and possibly pretend to use a useless collation sequence. ** This pretense allows expert to accept SQL using custom collations. */ int dummyCompare(void *up1, int up2, const void *up3, int up4, const void *up5){ (void)up1; (void)up2; (void)up3; (void)up4; (void)up5; assert(0); /* VDBE should never be run. */ return 0; } /* And a callback to register above upon actual need */ void useDummyCS(void *up1, sqlite3 *db, int etr, const char *zName){ (void)up1; sqlite3_create_collation_v2(db, zName, etr, 0, dummyCompare, 0); } #if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \ && !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS) /* ** dummy functions for no-op implementation of UDFs during expert's work */ void dummyUDF(sqlite3_context *up1, int up2, sqlite3_value **up3){ (void)up1; (void)up2; (void)up3; assert(0); /* VDBE should never be run. */ } void dummyUDFvalue(sqlite3_context *up1){ (void)up1; assert(0); /* VDBE should never be run. */ } /* ** Register UDFs from user database with another. */ int registerUDFs(sqlite3 *dbSrc, sqlite3 *dbDst){ sqlite3_stmt *pStmt; int rc = sqlite3_prepare_v2(dbSrc, "SELECT name,type,enc,narg,flags " "FROM pragma_function_list() " "WHERE builtin==0", -1, &pStmt, 0); if( rc==SQLITE_OK ){ while( SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){ int nargs = sqlite3_column_int(pStmt,3); int flags = sqlite3_column_int(pStmt,4); const char *name = (char*)sqlite3_column_text(pStmt,0); const char *type = (char*)sqlite3_column_text(pStmt,1); const char *enc = (char*)sqlite3_column_text(pStmt,2); if( name==0 || type==0 || enc==0 ){ /* no-op. Only happens on OOM */ }else{ int ienc = SQLITE_UTF8; int rcf = SQLITE_ERROR; if( strcmp(enc,"utf16le")==0 ) ienc = SQLITE_UTF16LE; else if( strcmp(enc,"utf16be")==0 ) ienc = SQLITE_UTF16BE; ienc |= (flags & (SQLITE_DETERMINISTIC|SQLITE_DIRECTONLY)); if( strcmp(type,"w")==0 ){ rcf = sqlite3_create_window_function(dbDst,name,nargs,ienc,0, dummyUDF,dummyUDFvalue,0,0,0); }else if( strcmp(type,"a")==0 ){ rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0, 0,dummyUDF,dummyUDFvalue); }else if( strcmp(type,"s")==0 ){ rcf = sqlite3_create_function(dbDst,name,nargs,ienc,0, dummyUDF,0,0); } if( rcf!=SQLITE_OK ){ rc = rcf; break; } } } sqlite3_finalize(pStmt); if( rc==SQLITE_DONE ) rc = SQLITE_OK; } return rc; } #endif /* ** Allocate a new sqlite3expert object. */ sqlite3expert *sqlite3_expert_new(sqlite3 *db, char **pzErrmsg){ int rc = SQLITE_OK; sqlite3expert *pNew; |
︙ | ︙ | |||
1840 1841 1842 1843 1844 1845 1846 | } if( rc==SQLITE_OK ){ rc = sqlite3_open(":memory:", &pNew->dbm); if( rc==SQLITE_OK ){ sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0); } } | | > > > > > > > > > > > > > > > | < > > > > > | > | | 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 | } if( rc==SQLITE_OK ){ rc = sqlite3_open(":memory:", &pNew->dbm); if( rc==SQLITE_OK ){ sqlite3_db_config(pNew->dbm, SQLITE_DBCONFIG_TRIGGER_EQP, 1, (int*)0); } } /* Allow custom collations to be dealt with through prepare. */ if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbm,0,useDummyCS); if( rc==SQLITE_OK ) rc = sqlite3_collation_needed(pNew->dbv,0,useDummyCS); #if !defined(SQLITE_OMIT_SCHEMA_PRAGMAS) \ && !defined(SQLITE_OMIT_INTROSPECTION_PRAGMAS) /* Register UDFs from database [db] with [dbm] and [dbv]. */ if( rc==SQLITE_OK ){ rc = registerUDFs(pNew->db, pNew->dbm); } if( rc==SQLITE_OK ){ rc = registerUDFs(pNew->db, pNew->dbv); } #endif /* Copy the entire schema of database [db] into [dbm]. */ if( rc==SQLITE_OK ){ sqlite3_stmt *pSql = 0; rc = idxPrintfPrepareStmt(pNew->db, &pSql, pzErrmsg, "SELECT sql, name, substr(sql,1,14)=='create virtual' COLLATE nocase" " FROM sqlite_schema WHERE substr(name,1,7)!='sqlite_' COLLATE nocase" " ORDER BY 3 DESC, rowid" ); while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pSql) ){ const char *zSql = (const char*)sqlite3_column_text(pSql, 0); const char *zName = (const char*)sqlite3_column_text(pSql, 1); int bExists = 0; rc = expertDbContainsObject(pNew->dbm, zName, &bExists); if( rc==SQLITE_OK && zSql && bExists==0 ){ rc = expertSchemaSql(pNew->dbm, zSql, pzErrmsg); } } idxFinalize(&rc, pSql); } /* Create the vtab schema */ if( rc==SQLITE_OK ){ rc = idxCreateVtabSchema(pNew, pzErrmsg); } /* Register the auth callback with dbv */ if( rc==SQLITE_OK ){ sqlite3_set_authorizer(pNew->dbv, idxAuthCallback, (void*)pNew); } /* If an error has occurred, free the new object and return NULL. Otherwise, ** return the new sqlite3expert handle. */ if( rc!=SQLITE_OK ){ sqlite3_expert_destroy(pNew); pNew = 0; } return pNew; } |
︙ | ︙ | |||
1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 | int rc = SQLITE_OK; const char *zStmt = zSql; if( p->bRun ) return SQLITE_MISUSE; while( rc==SQLITE_OK && zStmt && zStmt[0] ){ sqlite3_stmt *pStmt = 0; rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt); if( rc==SQLITE_OK ){ if( pStmt ){ IdxStatement *pNew; const char *z = sqlite3_sql(pStmt); int n = STRLEN(z); pNew = (IdxStatement*)idxMalloc(&rc, sizeof(IdxStatement) + n+1); | > > > > | 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 | int rc = SQLITE_OK; const char *zStmt = zSql; if( p->bRun ) return SQLITE_MISUSE; while( rc==SQLITE_OK && zStmt && zStmt[0] ){ sqlite3_stmt *pStmt = 0; /* Ensure that the provided statement compiles against user's DB. */ rc = idxPrepareStmt(p->db, &pStmt, pzErr, zStmt); if( rc!=SQLITE_OK ) break; sqlite3_finalize(pStmt); rc = sqlite3_prepare_v2(p->dbv, zStmt, -1, &pStmt, &zStmt); if( rc==SQLITE_OK ){ if( pStmt ){ IdxStatement *pNew; const char *z = sqlite3_sql(pStmt); int n = STRLEN(z); pNew = (IdxStatement*)idxMalloc(&rc, sizeof(IdxStatement) + n+1); |
︙ | ︙ |
Changes to ext/expert/test_expert.c.
︙ | ︙ | |||
12 13 14 15 16 17 18 | */ #if defined(SQLITE_TEST) #include "sqlite3expert.h" #include <assert.h> #include <string.h> | < < < < | < < < < | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 | */ #if defined(SQLITE_TEST) #include "sqlite3expert.h" #include <assert.h> #include <string.h> #include "tclsqlite.h" #ifndef SQLITE_OMIT_VIRTUALTABLE /* ** Extract an sqlite3* db handle from the object passed as the second ** argument. If successful, set *pDb to point to the db handle and return ** TCL_OK. Otherwise, return TCL_ERROR. */ static int dbHandleFromObj(Tcl_Interp *interp, Tcl_Obj *pObj, sqlite3 **pDb){ Tcl_CmdInfo info; if( 0==Tcl_GetCommandInfo(interp, Tcl_GetString(pObj), &info) ){ Tcl_AppendResult(interp, "no such handle: ", Tcl_GetString(pObj), NULL); return TCL_ERROR; } *pDb = *(sqlite3 **)info.objClientData; return TCL_OK; } |
︙ | ︙ |
Changes to ext/fts3/README.syntax.
︙ | ︙ | |||
58 59 60 61 62 63 64 | case is treated as an ordinary token. For example, the following query: <col> MATCH 'engineering NEAR consultancy' matches rows that contain both the "engineering" and "consultancy" tokens in the same column with not more than 10 other words between them. It does not matter which of the two terms occurs first in the document, only that | | | | | 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | case is treated as an ordinary token. For example, the following query: <col> MATCH 'engineering NEAR consultancy' matches rows that contain both the "engineering" and "consultancy" tokens in the same column with not more than 10 other words between them. It does not matter which of the two terms occurs first in the document, only that they be separated by only 10 tokens or less. The user may also specify a different required proximity by adding "/N" immediately after the NEAR operator, where N is an integer. For example: <col> MATCH 'engineering NEAR/5 consultancy' searches for a row containing an instance of each specified token separated by not more than 5 other tokens. More than one NEAR operator can be used in as sequence. For example this query: <col> MATCH 'reliable NEAR/2 engineering NEAR/5 consultancy' searches for a row that contains an instance of the token "reliable" separated by not more than two tokens from an instance of "engineering", which is in turn separated by not more than 5 other tokens from an instance of the term "consultancy". Phrases enclosed in quotes may also be used as arguments to the NEAR operator. Similar to the NEAR operator, one or more tokens or phrases may be separated by OR operators. In this case, only one of the specified tokens or phrases must appear in the document. For example, the query: |
︙ | ︙ | |||
142 143 144 145 146 147 148 | 1) Parenthesis are supported. When using the enhanced query syntax, parenthesis may be used to overcome the built-in precedence of the supplied binary operators. For example, the following query: <col> MATCH '(hello world) OR (simple example)' matches documents that contain both "hello" and "world", and documents | | | 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 | 1) Parenthesis are supported. When using the enhanced query syntax, parenthesis may be used to overcome the built-in precedence of the supplied binary operators. For example, the following query: <col> MATCH '(hello world) OR (simple example)' matches documents that contain both "hello" and "world", and documents that contain both "simple" and "example". It is not possible to formulate such a query using the standard syntax. 2) Instead of separating tokens and phrases by whitespace, an AND operator may be explicitly specified. This does not change query processing at all, but may be used to improve readability. For example, the following query is handled identically to the one above: |
︙ | ︙ | |||
170 171 172 173 174 175 176 | <col> MATCH 'example NOT simple' As for all other operators, the NOT operator must be specified in upper case. Otherwise it will be treated as a regular token. 4) Unlike in the standard syntax, where the OR operator has a higher precedence than the implicit AND operator, when using the enhanced | | | 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | <col> MATCH 'example NOT simple' As for all other operators, the NOT operator must be specified in upper case. Otherwise it will be treated as a regular token. 4) Unlike in the standard syntax, where the OR operator has a higher precedence than the implicit AND operator, when using the enhanced syntax implicit and explicit AND operators have a higher precedence than OR operators. Using the enhanced syntax, the following two queries are equivalent: <col> MATCH 'sqlite fantastic OR impressive' <col> MATCH '(sqlite AND fantastic) OR impressive' however, when using the standard syntax, the query: |
︙ | ︙ |
Changes to ext/fts3/fts3.c.
︙ | ︙ | |||
83 84 85 86 87 88 89 | ** } ** varint POS_END; (marks end of positions for this document. ** } ** ** Here, array { X } means zero or more occurrences of X, adjacent in ** memory. A "position" is an index of a token in the token stream ** generated by the tokenizer. Note that POS_END and POS_COLUMN occur | | | 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | ** } ** varint POS_END; (marks end of positions for this document. ** } ** ** Here, array { X } means zero or more occurrences of X, adjacent in ** memory. A "position" is an index of a token in the token stream ** generated by the tokenizer. Note that POS_END and POS_COLUMN occur ** in the same logical place as the position element, and act as sentinels ** ending a position list array. POS_END is 0. POS_COLUMN is 1. ** The positions numbers are not stored literally but rather as two more ** than the difference from the prior position, or the just the position plus ** 2 for the first position. Example: ** ** label: A B C D E F G H I J K ** value: 123 5 9 1 1 14 35 0 234 72 0 |
︙ | ︙ | |||
291 292 293 294 295 296 297 | #include "fts3Int.h" #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE) # define SQLITE_CORE 1 #endif | < < < < < < | 291 292 293 294 295 296 297 298 299 300 301 302 303 304 | #include "fts3Int.h" #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE) # define SQLITE_CORE 1 #endif #include "fts3.h" #ifndef SQLITE_CORE # include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #endif |
︙ | ︙ | |||
636 637 638 639 640 641 642 643 644 645 646 647 648 649 | int rc; /* Return code */ char *zSql; /* SQL statement passed to declare_vtab() */ char *zCols; /* List of user defined columns */ const char *zLanguageid; zLanguageid = (p->zLanguageid ? p->zLanguageid : "__langid"); sqlite3_vtab_config(p->db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1); /* Create a list of user columns for the virtual table */ zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]); for(i=1; zCols && i<p->nColumn; i++){ zCols = sqlite3_mprintf("%z%Q, ", zCols, p->azColumn[i]); } | > | 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 | int rc; /* Return code */ char *zSql; /* SQL statement passed to declare_vtab() */ char *zCols; /* List of user defined columns */ const char *zLanguageid; zLanguageid = (p->zLanguageid ? p->zLanguageid : "__langid"); sqlite3_vtab_config(p->db, SQLITE_VTAB_CONSTRAINT_SUPPORT, 1); sqlite3_vtab_config(p->db, SQLITE_VTAB_INNOCUOUS); /* Create a list of user columns for the virtual table */ zCols = sqlite3_mprintf("%Q, ", p->azColumn[0]); for(i=1; zCols && i<p->nColumn; i++){ zCols = sqlite3_mprintf("%z%Q, ", zCols, p->azColumn[i]); } |
︙ | ︙ | |||
2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 | /* Never set both isSaveLeft and isExact for the same invocation. */ assert( isSaveLeft==0 || isExact==0 ); assert_fts3_nc( p!=0 && *p1!=0 && *p2!=0 ); if( *p1==POS_COLUMN ){ p1++; p1 += fts3GetVarint32(p1, &iCol1); } if( *p2==POS_COLUMN ){ p2++; p2 += fts3GetVarint32(p2, &iCol2); } while( 1 ){ if( iCol1==iCol2 ){ char *pSave = p; sqlite3_int64 iPrev = 0; sqlite3_int64 iPos1 = 0; | > > > > > | 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 | /* Never set both isSaveLeft and isExact for the same invocation. */ assert( isSaveLeft==0 || isExact==0 ); assert_fts3_nc( p!=0 && *p1!=0 && *p2!=0 ); if( *p1==POS_COLUMN ){ p1++; p1 += fts3GetVarint32(p1, &iCol1); /* iCol1==0 indicates corruption. Column 0 does not have a POS_COLUMN ** entry, so this is actually end-of-doclist. */ if( iCol1==0 ) return 0; } if( *p2==POS_COLUMN ){ p2++; p2 += fts3GetVarint32(p2, &iCol2); /* As above, iCol2==0 indicates corruption. */ if( iCol2==0 ) return 0; } while( 1 ){ if( iCol1==iCol2 ){ char *pSave = p; sqlite3_int64 iPrev = 0; sqlite3_int64 iPos1 = 0; |
︙ | ︙ | |||
2629 2630 2631 2632 2633 2634 2635 | ** be larger in the output than it was in the input (since the delta value ** may be a larger positive integer than the actual docid). ** ** The space required to store the output is therefore the sum of the ** sizes of the two inputs, plus enough space for exactly one of the input ** docids to grow. ** | | | 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 | ** be larger in the output than it was in the input (since the delta value ** may be a larger positive integer than the actual docid). ** ** The space required to store the output is therefore the sum of the ** sizes of the two inputs, plus enough space for exactly one of the input ** docids to grow. ** ** A symmetric argument may be made if the doclists are in descending ** order. */ aOut = sqlite3_malloc64((i64)n1+n2+FTS3_VARINT_MAX-1+FTS3_BUFFER_PADDING); if( !aOut ) return SQLITE_NOMEM; p = aOut; fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); |
︙ | ︙ | |||
3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 | ** PendingTermsFlush() in in case that changes. */ assert( p->nPendingData==0 ); if( rc==SQLITE_OK ){ rc = sqlite3Fts3PendingTermsFlush(p); } if( p->zContentTbl==0 ){ fts3DbExec(&rc, db, "ALTER TABLE %Q.'%q_content' RENAME TO '%q_content';", p->zDb, p->zName, zName ); } | > > | 3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 3899 3900 | ** PendingTermsFlush() in in case that changes. */ assert( p->nPendingData==0 ); if( rc==SQLITE_OK ){ rc = sqlite3Fts3PendingTermsFlush(p); } p->bIgnoreSavepoint = 1; if( p->zContentTbl==0 ){ fts3DbExec(&rc, db, "ALTER TABLE %Q.'%q_content' RENAME TO '%q_content';", p->zDb, p->zName, zName ); } |
︙ | ︙ | |||
3912 3913 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 | "ALTER TABLE %Q.'%q_segments' RENAME TO '%q_segments';", p->zDb, p->zName, zName ); fts3DbExec(&rc, db, "ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';", p->zDb, p->zName, zName ); return rc; } /* ** The xSavepoint() method. ** ** Flush the contents of the pending-terms table to disk. */ static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ int rc = SQLITE_OK; | > > | | | | > | > > > > > > > > > > | > > > > > | < < | | | > | | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 3925 3926 3927 3928 3929 3930 3931 3932 3933 3934 3935 3936 3937 3938 3939 3940 3941 3942 3943 3944 3945 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 4015 4016 4017 4018 4019 4020 4021 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 | "ALTER TABLE %Q.'%q_segments' RENAME TO '%q_segments';", p->zDb, p->zName, zName ); fts3DbExec(&rc, db, "ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';", p->zDb, p->zName, zName ); p->bIgnoreSavepoint = 0; return rc; } /* ** The xSavepoint() method. ** ** Flush the contents of the pending-terms table to disk. */ static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ int rc = SQLITE_OK; Fts3Table *pTab = (Fts3Table*)pVtab; assert( pTab->inTransaction ); assert( pTab->mxSavepoint<=iSavepoint ); TESTONLY( pTab->mxSavepoint = iSavepoint ); if( pTab->bIgnoreSavepoint==0 ){ if( fts3HashCount(&pTab->aIndex[0].hPending)>0 ){ char *zSql = sqlite3_mprintf("INSERT INTO %Q.%Q(%Q) VALUES('flush')", pTab->zDb, pTab->zName, pTab->zName ); if( zSql ){ pTab->bIgnoreSavepoint = 1; rc = sqlite3_exec(pTab->db, zSql, 0, 0, 0); pTab->bIgnoreSavepoint = 0; sqlite3_free(zSql); }else{ rc = SQLITE_NOMEM; } } if( rc==SQLITE_OK ){ pTab->iSavepoint = iSavepoint+1; } } return rc; } /* ** The xRelease() method. ** ** This is a no-op. */ static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts3Table *pTab = (Fts3Table*)pVtab; assert( pTab->inTransaction ); assert( pTab->mxSavepoint >= iSavepoint ); TESTONLY( pTab->mxSavepoint = iSavepoint-1 ); pTab->iSavepoint = iSavepoint; return SQLITE_OK; } /* ** The xRollbackTo() method. ** ** Discard the contents of the pending terms table. */ static int fts3RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts3Table *pTab = (Fts3Table*)pVtab; UNUSED_PARAMETER(iSavepoint); assert( pTab->inTransaction ); TESTONLY( pTab->mxSavepoint = iSavepoint ); if( (iSavepoint+1)<=pTab->iSavepoint ){ sqlite3Fts3PendingTermsClear(pTab); } return SQLITE_OK; } /* ** Return true if zName is the extension on one of the shadow tables used ** by this module. */ static int fts3ShadowName(const char *zName){ static const char *azName[] = { "content", "docsize", "segdir", "segments", "stat", }; unsigned int i; for(i=0; i<sizeof(azName)/sizeof(azName[0]); i++){ if( sqlite3_stricmp(zName, azName[i])==0 ) return 1; } return 0; } /* ** Implementation of the xIntegrity() method on the FTS3/FTS4 virtual ** table. */ static int fts3IntegrityMethod( sqlite3_vtab *pVtab, /* The virtual table to be checked */ const char *zSchema, /* Name of schema in which pVtab lives */ const char *zTabname, /* Name of the pVTab table */ int isQuick, /* True if this is a quick_check */ char **pzErr /* Write error message here */ ){ Fts3Table *p = (Fts3Table*)pVtab; int rc = SQLITE_OK; int bOk = 0; UNUSED_PARAMETER(isQuick); rc = sqlite3Fts3IntegrityCheck(p, &bOk); assert( rc!=SQLITE_CORRUPT_VTAB ); if( rc==SQLITE_ERROR || (rc&0xFF)==SQLITE_CORRUPT ){ *pzErr = sqlite3_mprintf("unable to validate the inverted index for" " FTS%d table %s.%s: %s", p->bFts4 ? 4 : 3, zSchema, zTabname, sqlite3_errstr(rc)); if( *pzErr ) rc = SQLITE_OK; }else if( rc==SQLITE_OK && bOk==0 ){ *pzErr = sqlite3_mprintf("malformed inverted index for FTS%d table %s.%s", p->bFts4 ? 4 : 3, zSchema, zTabname); if( *pzErr==0 ) rc = SQLITE_NOMEM; } sqlite3Fts3SegmentsClose(p); return rc; } static const sqlite3_module fts3Module = { /* iVersion */ 4, /* xCreate */ fts3CreateMethod, /* xConnect */ fts3ConnectMethod, /* xBestIndex */ fts3BestIndexMethod, /* xDisconnect */ fts3DisconnectMethod, /* xDestroy */ fts3DestroyMethod, /* xOpen */ fts3OpenMethod, /* xClose */ fts3CloseMethod, |
︙ | ︙ | |||
4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 4012 4013 4014 | /* xRollback */ fts3RollbackMethod, /* xFindFunction */ fts3FindFunctionMethod, /* xRename */ fts3RenameMethod, /* xSavepoint */ fts3SavepointMethod, /* xRelease */ fts3ReleaseMethod, /* xRollbackTo */ fts3RollbackToMethod, /* xShadowName */ fts3ShadowName, }; /* ** This function is registered as the module destructor (called when an ** FTS3 enabled database connection is closed). It frees the memory ** allocated for the tokenizer hash table. */ | > | 4056 4057 4058 4059 4060 4061 4062 4063 4064 4065 4066 4067 4068 4069 4070 | /* xRollback */ fts3RollbackMethod, /* xFindFunction */ fts3FindFunctionMethod, /* xRename */ fts3RenameMethod, /* xSavepoint */ fts3SavepointMethod, /* xRelease */ fts3ReleaseMethod, /* xRollbackTo */ fts3RollbackToMethod, /* xShadowName */ fts3ShadowName, /* xIntegrity */ fts3IntegrityMethod, }; /* ** This function is registered as the module destructor (called when an ** FTS3 enabled database connection is closed). It frees the memory ** allocated for the tokenizer hash table. */ |
︙ | ︙ | |||
4372 4373 4374 4375 4376 4377 4378 | nDistance = nMaxUndeferred - iPrev; }else{ p1 = pPhrase->doclist.pList; p2 = aPoslist; nDistance = iPrev - nMaxUndeferred; } | | | 4428 4429 4430 4431 4432 4433 4434 4435 4436 4437 4438 4439 4440 4441 4442 | nDistance = nMaxUndeferred - iPrev; }else{ p1 = pPhrase->doclist.pList; p2 = aPoslist; nDistance = iPrev - nMaxUndeferred; } aOut = (char *)sqlite3Fts3MallocZero(((i64)nPoslist)+FTS3_BUFFER_PADDING); if( !aOut ){ sqlite3_free(aPoslist); return SQLITE_NOMEM; } pPhrase->doclist.pList = aOut; assert( p1 && p2 ); |
︙ | ︙ | |||
4671 4672 4673 4674 4675 4676 4677 | /* ** The phrase iterator passed as the second argument: ** ** * features at least one token that uses an incremental doclist, and ** ** * does not contain any deferred tokens. ** | | | 4727 4728 4729 4730 4731 4732 4733 4734 4735 4736 4737 4738 4739 4740 4741 | /* ** The phrase iterator passed as the second argument: ** ** * features at least one token that uses an incremental doclist, and ** ** * does not contain any deferred tokens. ** ** Advance it to the next matching document in the database and populate ** the Fts3Doclist.pList and nList fields. ** ** If there is no "next" entry and no error occurs, then *pbEof is set to ** 1 before returning. Otherwise, if no error occurs and the iterator is ** successfully advanced, *pbEof is set to 0. ** ** If an error occurs, return an SQLite error code. Otherwise, return |
︙ | ︙ | |||
5457 5458 5459 5460 5461 5462 5463 | /* Allocate temporary working space. */ for(p=pExpr; p->pLeft; p=p->pLeft){ assert( p->pRight->pPhrase->doclist.nList>0 ); nTmp += p->pRight->pPhrase->doclist.nList; } nTmp += p->pPhrase->doclist.nList; | | | 5513 5514 5515 5516 5517 5518 5519 5520 5521 5522 5523 5524 5525 5526 5527 | /* Allocate temporary working space. */ for(p=pExpr; p->pLeft; p=p->pLeft){ assert( p->pRight->pPhrase->doclist.nList>0 ); nTmp += p->pRight->pPhrase->doclist.nList; } nTmp += p->pPhrase->doclist.nList; aTmp = sqlite3_malloc64(nTmp*2 + FTS3_VARINT_MAX); if( !aTmp ){ *pRc = SQLITE_NOMEM; res = 0; }else{ char *aPoslist = p->pPhrase->doclist.pList; int nToken = p->pPhrase->nToken; |
︙ | ︙ | |||
5678 5679 5680 5681 5682 5683 5684 | pCsr->isEof = 1; } return rc; } /* | | | 5734 5735 5736 5737 5738 5739 5740 5741 5742 5743 5744 5745 5746 5747 5748 | pCsr->isEof = 1; } return rc; } /* ** Restart iteration for expression pExpr so that the next call to ** fts3EvalNext() visits the first row. Do not allow incremental ** loading or merging of phrase doclists for this iteration. ** ** If *pRc is other than SQLITE_OK when this function is called, it is ** a no-op. If an error occurs within this function, *pRc is set to an ** SQLite error code before returning. */ |
︙ | ︙ | |||
5720 5721 5722 5723 5724 5725 5726 5727 5728 5729 5730 5731 5732 5733 | pExpr->bEof = 0; pExpr->bStart = 0; fts3EvalRestart(pCsr, pExpr->pLeft, pRc); fts3EvalRestart(pCsr, pExpr->pRight, pRc); } } /* ** After allocating the Fts3Expr.aMI[] array for each phrase in the ** expression rooted at pExpr, the cursor iterates through all rows matched ** by pExpr, calling this function for each row. This function increments ** the values in Fts3Expr.aMI[] according to the position-list currently ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase | > > > > > > > > > > > > > > > > > > | 5776 5777 5778 5779 5780 5781 5782 5783 5784 5785 5786 5787 5788 5789 5790 5791 5792 5793 5794 5795 5796 5797 5798 5799 5800 5801 5802 5803 5804 5805 5806 5807 | pExpr->bEof = 0; pExpr->bStart = 0; fts3EvalRestart(pCsr, pExpr->pLeft, pRc); fts3EvalRestart(pCsr, pExpr->pRight, pRc); } } /* ** Expression node pExpr is an MSR phrase. This function restarts pExpr ** so that it is a regular phrase query, not an MSR. SQLITE_OK is returned ** if successful, or an SQLite error code otherwise. */ int sqlite3Fts3MsrCancel(Fts3Cursor *pCsr, Fts3Expr *pExpr){ int rc = SQLITE_OK; if( pExpr->bEof==0 ){ i64 iDocid = pExpr->iDocid; fts3EvalRestart(pCsr, pExpr, &rc); while( rc==SQLITE_OK && pExpr->iDocid!=iDocid ){ fts3EvalNextRow(pCsr, pExpr, &rc); if( pExpr->bEof ) rc = FTS_CORRUPT_VTAB; } } return rc; } /* ** After allocating the Fts3Expr.aMI[] array for each phrase in the ** expression rooted at pExpr, the cursor iterates through all rows matched ** by pExpr, calling this function for each row. This function increments ** the values in Fts3Expr.aMI[] according to the position-list currently ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase |
︙ | ︙ | |||
6108 6109 6110 6111 6112 6113 6114 | */ #ifdef SQLITE_DEBUG int sqlite3Fts3Corrupt(){ return SQLITE_CORRUPT_VTAB; } #endif | | | 6182 6183 6184 6185 6186 6187 6188 6189 6190 6191 6192 6193 6194 6195 6196 | */ #ifdef SQLITE_DEBUG int sqlite3Fts3Corrupt(){ return SQLITE_CORRUPT_VTAB; } #endif #if !defined(SQLITE_CORE) /* ** Initialize API pointer table, if required. */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_fts3_init( |
︙ | ︙ |
Changes to ext/fts3/fts3Int.h.
︙ | ︙ | |||
9 10 11 12 13 14 15 16 17 18 19 20 21 22 | ** May you share freely, never taking more than you give. ** ****************************************************************************** ** */ #ifndef _FTSINT_H #define _FTSINT_H #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) # define NDEBUG 1 #endif /* FTS3/FTS4 require virtual tables */ #ifdef SQLITE_OMIT_VIRTUALTABLE | > > > > > > > | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | ** May you share freely, never taking more than you give. ** ****************************************************************************** ** */ #ifndef _FTSINT_H #define _FTSINT_H #include <assert.h> #include <stdlib.h> #include <stddef.h> #include <stdio.h> #include <string.h> #include <stdarg.h> #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) # define NDEBUG 1 #endif /* FTS3/FTS4 require virtual tables */ #ifdef SQLITE_OMIT_VIRTUALTABLE |
︙ | ︙ | |||
197 198 199 200 201 202 203 204 205 206 207 208 209 210 | # define TESTONLY(X) #endif #define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) #define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) #define deliberate_fall_through #endif /* SQLITE_AMALGAMATION */ #ifdef SQLITE_DEBUG int sqlite3Fts3Corrupt(void); # define FTS_CORRUPT_VTAB sqlite3Fts3Corrupt() #else | > > > > > > > > > > > > > | 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 | # define TESTONLY(X) #endif #define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) #define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) #define deliberate_fall_through /* ** Macros needed to provide flexible arrays in a portable way */ #ifndef offsetof # define offsetof(ST,M) ((size_t)((char*)&((ST*)0)->M - (char*)0)) #endif #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) # define FLEXARRAY #else # define FLEXARRAY 1 #endif #endif /* SQLITE_AMALGAMATION */ #ifdef SQLITE_DEBUG int sqlite3Fts3Corrupt(void); # define FTS_CORRUPT_VTAB sqlite3Fts3Corrupt() #else |
︙ | ︙ | |||
261 262 263 264 265 266 267 268 269 270 271 272 273 274 | u8 bHasStat; /* True if %_stat table exists (2==unknown) */ u8 bHasDocsize; /* True if %_docsize table exists */ u8 bDescIdx; /* True if doclists are in reverse order */ u8 bIgnoreSavepoint; /* True to ignore xSavepoint invocations */ int nPgsz; /* Page size for host database */ char *zSegmentsTbl; /* Name of %_segments table */ sqlite3_blob *pSegments; /* Blob handle open on %_segments table */ /* ** The following array of hash tables is used to buffer pending index ** updates during transactions. All pending updates buffered at any one ** time must share a common language-id (see the FTS4 langid= feature). ** The current language id is stored in variable iPrevLangid. ** | > | 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 | u8 bHasStat; /* True if %_stat table exists (2==unknown) */ u8 bHasDocsize; /* True if %_docsize table exists */ u8 bDescIdx; /* True if doclists are in reverse order */ u8 bIgnoreSavepoint; /* True to ignore xSavepoint invocations */ int nPgsz; /* Page size for host database */ char *zSegmentsTbl; /* Name of %_segments table */ sqlite3_blob *pSegments; /* Blob handle open on %_segments table */ int iSavepoint; /* ** The following array of hash tables is used to buffer pending index ** updates during transactions. All pending updates buffered at any one ** time must share a common language-id (see the FTS4 langid= feature). ** The current language id is stored in variable iPrevLangid. ** |
︙ | ︙ | |||
301 302 303 304 305 306 307 | ** verifying the operation of the SQLite core. */ int inTransaction; /* True after xBegin but before xCommit/xRollback */ int mxSavepoint; /* Largest valid xSavepoint integer */ #endif #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) | | | 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 | ** verifying the operation of the SQLite core. */ int inTransaction; /* True after xBegin but before xCommit/xRollback */ int mxSavepoint; /* Largest valid xSavepoint integer */ #endif #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) /* True to disable the incremental doclist optimization. This is controlled ** by special insert command 'test-no-incr-doclist'. */ int bNoIncrDoclist; /* Number of segments in a level */ int nMergeCount; #endif }; |
︙ | ︙ | |||
353 354 355 356 357 358 359 | #define FTS3_EVAL_FILTER 0 #define FTS3_EVAL_NEXT 1 #define FTS3_EVAL_MATCHINFO 2 /* ** The Fts3Cursor.eSearch member is always set to one of the following. | | | 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 | #define FTS3_EVAL_FILTER 0 #define FTS3_EVAL_NEXT 1 #define FTS3_EVAL_MATCHINFO 2 /* ** The Fts3Cursor.eSearch member is always set to one of the following. ** Actually, Fts3Cursor.eSearch can be greater than or equal to ** FTS3_FULLTEXT_SEARCH. If so, then Fts3Cursor.eSearch - 2 is the index ** of the column to be searched. For example, in ** ** CREATE VIRTUAL TABLE ex1 USING fts3(a,b,c,d); ** SELECT docid FROM ex1 WHERE b MATCH 'one two three'; ** ** Because the LHS of the MATCH operator is 2nd column "b", |
︙ | ︙ | |||
426 427 428 429 430 431 432 | i64 iOrDocid; /* Variables below this point are populated by fts3_expr.c when parsing ** a MATCH expression. Everything above is part of the evaluation phase. */ int nToken; /* Number of tokens in the phrase */ int iColumn; /* Index of column this phrase must match */ | | > > > > | 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 | i64 iOrDocid; /* Variables below this point are populated by fts3_expr.c when parsing ** a MATCH expression. Everything above is part of the evaluation phase. */ int nToken; /* Number of tokens in the phrase */ int iColumn; /* Index of column this phrase must match */ Fts3PhraseToken aToken[FLEXARRAY]; /* One for each token in the phrase */ }; /* Size (in bytes) of an Fts3Phrase object large enough to hold N tokens */ #define SZ_FTS3PHRASE(N) \ (offsetof(Fts3Phrase,aToken)+(N)*sizeof(Fts3PhraseToken)) /* ** A tree of these objects forms the RHS of a MATCH operator. ** ** If Fts3Expr.eType is FTSQUERY_PHRASE and isLoaded is true, then aDoclist ** points to a malloced buffer, size nDoclist bytes, containing the results ** of this phrase query in FTS3 doclist format. As usual, the initial |
︙ | ︙ | |||
635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 | int sqlite3Fts3MsrIncrStart( Fts3Table*, Fts3MultiSegReader*, int, const char*, int); int sqlite3Fts3MsrIncrNext( Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *); int sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol, char **); int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *); int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr); /* fts3_tokenize_vtab.c */ int sqlite3Fts3InitTok(sqlite3*, Fts3Hash *, void(*xDestroy)(void*)); /* fts3_unicode2.c (functions generated by parsing unicode text files) */ #ifndef SQLITE_DISABLE_FTS3_UNICODE int sqlite3FtsUnicodeFold(int, int); int sqlite3FtsUnicodeIsalnum(int); int sqlite3FtsUnicodeIsdiacritic(int); #endif int sqlite3Fts3ExprIterate(Fts3Expr*, int (*x)(Fts3Expr*,int,void*), void*); #endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */ #endif /* _FTSINT_H */ | > > > | 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 | int sqlite3Fts3MsrIncrStart( Fts3Table*, Fts3MultiSegReader*, int, const char*, int); int sqlite3Fts3MsrIncrNext( Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *); int sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol, char **); int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *); int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr); int sqlite3Fts3MsrCancel(Fts3Cursor*, Fts3Expr*); /* fts3_tokenize_vtab.c */ int sqlite3Fts3InitTok(sqlite3*, Fts3Hash *, void(*xDestroy)(void*)); /* fts3_unicode2.c (functions generated by parsing unicode text files) */ #ifndef SQLITE_DISABLE_FTS3_UNICODE int sqlite3FtsUnicodeFold(int, int); int sqlite3FtsUnicodeIsalnum(int); int sqlite3FtsUnicodeIsdiacritic(int); #endif int sqlite3Fts3ExprIterate(Fts3Expr*, int (*x)(Fts3Expr*,int,void*), void*); int sqlite3Fts3IntegrityCheck(Fts3Table *p, int *pbOk); #endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */ #endif /* _FTSINT_H */ |
Changes to ext/fts3/fts3_aux.c.
︙ | ︙ | |||
541 542 543 544 545 546 547 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ | | > | 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 /* xIntegrity */ }; int rc; /* Return code */ rc = sqlite3_create_module(db, "fts4aux", &fts3aux_module, 0); return rc; } |
︙ | ︙ |
Changes to ext/fts3/fts3_expr.c.
︙ | ︙ | |||
156 157 158 159 160 161 162 163 164 165 166 167 168 169 | } /* ** Function getNextNode(), which is called by fts3ExprParse(), may itself ** call fts3ExprParse(). So this forward declaration is required. */ static int fts3ExprParse(ParseContext *, const char *, int, Fts3Expr **, int *); /* ** Extract the next token from buffer z (length n) using the tokenizer ** and other information (column names etc.) in pParse. Create an Fts3Expr ** structure of type FTSQUERY_PHRASE containing a phrase consisting of this ** single token and set *ppExpr to point to it. If the end of the buffer is ** reached before a token is found, set *ppExpr to zero. It is the | > > > > > > > > > > > > > > > > > | 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | } /* ** Function getNextNode(), which is called by fts3ExprParse(), may itself ** call fts3ExprParse(). So this forward declaration is required. */ static int fts3ExprParse(ParseContext *, const char *, int, Fts3Expr **, int *); /* ** Search buffer z[], size n, for a '"' character. Or, if enable_parenthesis ** is defined, search for '(' and ')' as well. Return the index of the first ** such character in the buffer. If there is no such character, return -1. */ static int findBarredChar(const char *z, int n){ int ii; for(ii=0; ii<n; ii++){ if( (z[ii]=='"') || (sqlite3_fts3_enable_parentheses && (z[ii]=='(' || z[ii]==')')) ){ return ii; } } return -1; } /* ** Extract the next token from buffer z (length n) using the tokenizer ** and other information (column names etc.) in pParse. Create an Fts3Expr ** structure of type FTSQUERY_PHRASE containing a phrase consisting of this ** single token and set *ppExpr to point to it. If the end of the buffer is ** reached before a token is found, set *ppExpr to zero. It is the |
︙ | ︙ | |||
181 182 183 184 185 186 187 | int *pnConsumed /* OUT: Number of bytes consumed */ ){ sqlite3_tokenizer *pTokenizer = pParse->pTokenizer; sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; int rc; sqlite3_tokenizer_cursor *pCursor; Fts3Expr *pRet = 0; | < < < < < < < | | > > > > > > > > > > > | | | 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 | int *pnConsumed /* OUT: Number of bytes consumed */ ){ sqlite3_tokenizer *pTokenizer = pParse->pTokenizer; sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; int rc; sqlite3_tokenizer_cursor *pCursor; Fts3Expr *pRet = 0; *pnConsumed = n; rc = sqlite3Fts3OpenTokenizer(pTokenizer, pParse->iLangid, z, n, &pCursor); if( rc==SQLITE_OK ){ const char *zToken; int nToken = 0, iStart = 0, iEnd = 0, iPosition = 0; sqlite3_int64 nByte; /* total space to allocate */ rc = pModule->xNext(pCursor, &zToken, &nToken, &iStart, &iEnd, &iPosition); if( rc==SQLITE_OK ){ /* Check that this tokenization did not gobble up any " characters. Or, ** if enable_parenthesis is true, that it did not gobble up any ** open or close parenthesis characters either. If it did, call ** getNextToken() again, but pass only that part of the input buffer ** up to the first such character. */ int iBarred = findBarredChar(z, iEnd); if( iBarred>=0 ){ pModule->xClose(pCursor); return getNextToken(pParse, iCol, z, iBarred, ppExpr, pnConsumed); } nByte = sizeof(Fts3Expr) + SZ_FTS3PHRASE(1) + nToken; pRet = (Fts3Expr *)sqlite3Fts3MallocZero(nByte); if( !pRet ){ rc = SQLITE_NOMEM; }else{ pRet->eType = FTSQUERY_PHRASE; pRet->pPhrase = (Fts3Phrase *)&pRet[1]; pRet->pPhrase->nToken = 1; pRet->pPhrase->iColumn = iCol; pRet->pPhrase->aToken[0].n = nToken; pRet->pPhrase->aToken[0].z = (char*)&pRet->pPhrase->aToken[1]; memcpy(pRet->pPhrase->aToken[0].z, zToken, nToken); if( iEnd<n && z[iEnd]=='*' ){ pRet->pPhrase->aToken[0].isPrefix = 1; iEnd++; } |
︙ | ︙ | |||
232 233 234 235 236 237 238 | }else{ break; } } } *pnConsumed = iEnd; | | > > > > | 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 | }else{ break; } } } *pnConsumed = iEnd; }else if( n && rc==SQLITE_DONE ){ int iBarred = findBarredChar(z, n); if( iBarred>=0 ){ *pnConsumed = iBarred; } rc = SQLITE_OK; } pModule->xClose(pCursor); } *ppExpr = pRet; |
︙ | ︙ | |||
279 280 281 282 283 284 285 | ){ sqlite3_tokenizer *pTokenizer = pParse->pTokenizer; sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; int rc; Fts3Expr *p = 0; sqlite3_tokenizer_cursor *pCursor = 0; char *zTemp = 0; | | | | 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 | ){ sqlite3_tokenizer *pTokenizer = pParse->pTokenizer; sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; int rc; Fts3Expr *p = 0; sqlite3_tokenizer_cursor *pCursor = 0; char *zTemp = 0; i64 nTemp = 0; const int nSpace = sizeof(Fts3Expr) + SZ_FTS3PHRASE(1); int nToken = 0; /* The final Fts3Expr data structure, including the Fts3Phrase, ** Fts3PhraseToken structures token buffers are all stored as a single ** allocation so that the expression can be freed with a single call to ** sqlite3_free(). Setting this up requires a two pass approach. ** |
︙ | ︙ | |||
315 316 317 318 319 320 321 | const char *zByte; int nByte = 0, iBegin = 0, iEnd = 0, iPos = 0; rc = pModule->xNext(pCursor, &zByte, &nByte, &iBegin, &iEnd, &iPos); if( rc==SQLITE_OK ){ Fts3PhraseToken *pToken; p = fts3ReallocOrFree(p, nSpace + ii*sizeof(Fts3PhraseToken)); | < < | > > > < < < | > > > > < < < < < < | > | > > | | | 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 | const char *zByte; int nByte = 0, iBegin = 0, iEnd = 0, iPos = 0; rc = pModule->xNext(pCursor, &zByte, &nByte, &iBegin, &iEnd, &iPos); if( rc==SQLITE_OK ){ Fts3PhraseToken *pToken; p = fts3ReallocOrFree(p, nSpace + ii*sizeof(Fts3PhraseToken)); zTemp = fts3ReallocOrFree(zTemp, nTemp + nByte); if( !zTemp || !p ){ rc = SQLITE_NOMEM; goto getnextstring_out; } assert( nToken==ii ); pToken = &((Fts3Phrase *)(&p[1]))->aToken[ii]; memset(pToken, 0, sizeof(Fts3PhraseToken)); memcpy(&zTemp[nTemp], zByte, nByte); nTemp += nByte; pToken->n = nByte; pToken->isPrefix = (iEnd<nInput && zInput[iEnd]=='*'); pToken->bFirst = (iBegin>0 && zInput[iBegin-1]=='^'); nToken = ii+1; } } } if( rc==SQLITE_DONE ){ int jj; char *zBuf = 0; p = fts3ReallocOrFree(p, nSpace + nToken*sizeof(Fts3PhraseToken) + nTemp); if( !p ){ rc = SQLITE_NOMEM; goto getnextstring_out; } memset(p, 0, (char *)&(((Fts3Phrase *)&p[1])->aToken[0])-(char *)p); p->eType = FTSQUERY_PHRASE; p->pPhrase = (Fts3Phrase *)&p[1]; p->pPhrase->iColumn = pParse->iDefaultCol; p->pPhrase->nToken = nToken; zBuf = (char *)&p->pPhrase->aToken[nToken]; assert( nTemp==0 || zTemp ); if( zTemp ){ memcpy(zBuf, zTemp, nTemp); } for(jj=0; jj<p->pPhrase->nToken; jj++){ p->pPhrase->aToken[jj].z = zBuf; zBuf += p->pPhrase->aToken[jj].n; } rc = SQLITE_OK; } getnextstring_out: if( pCursor ){ pModule->xClose(pCursor); } sqlite3_free(zTemp); if( rc!=SQLITE_OK ){ sqlite3_free(p); p = 0; } *ppExpr = p; return rc; } /* ** The output variable *ppExpr is populated with an allocated Fts3Expr ** structure, or set to 0 if the end of the input buffer is reached. ** ** Returns an SQLite error code. SQLITE_OK if everything works, SQLITE_NOMEM |
︙ | ︙ | |||
654 655 656 657 658 659 660 | p = pPrev; }else{ int eType = p->eType; isPhrase = (eType==FTSQUERY_PHRASE || p->pLeft); /* The isRequirePhrase variable is set to true if a phrase or ** an expression contained in parenthesis is required. If a | | | 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 | p = pPrev; }else{ int eType = p->eType; isPhrase = (eType==FTSQUERY_PHRASE || p->pLeft); /* The isRequirePhrase variable is set to true if a phrase or ** an expression contained in parenthesis is required. If a ** binary operator (AND, OR, NOT or NEAR) is encountered when ** isRequirePhrase is set, this is a syntax error. */ if( !isPhrase && isRequirePhrase ){ sqlite3Fts3ExprFree(p); rc = SQLITE_ERROR; goto exprparse_out; } |
︙ | ︙ | |||
1236 1237 1238 1239 1240 1241 1242 | }else{ rc = fts3ExprParseUnbalanced( pTokenizer, 0, azCol, 0, nCol, nCol, zExpr, nExpr, &pExpr ); } if( rc!=SQLITE_OK && rc!=SQLITE_NOMEM ){ | < | 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 | }else{ rc = fts3ExprParseUnbalanced( pTokenizer, 0, azCol, 0, nCol, nCol, zExpr, nExpr, &pExpr ); } if( rc!=SQLITE_OK && rc!=SQLITE_NOMEM ){ sqlite3_result_error(context, "Error parsing expression", -1); }else if( rc==SQLITE_NOMEM || !(zBuf = exprToString(pExpr, 0)) ){ sqlite3_result_error_nomem(context); }else{ sqlite3_result_text(context, zBuf, -1, SQLITE_TRANSIENT); sqlite3_free(zBuf); } |
︙ | ︙ |
Changes to ext/fts3/fts3_hash.c.
︙ | ︙ | |||
183 184 185 186 187 188 189 | pH->first = pNew; } pEntry->count++; pEntry->chain = pNew; } | | | 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 | pH->first = pNew; } pEntry->count++; pEntry->chain = pNew; } /* Resize the hash table so that it contains "new_size" buckets. ** "new_size" must be a power of 2. The hash table might fail ** to resize if sqliteMalloc() fails. ** ** Return non-zero if a memory allocation error occurs. */ static int fts3Rehash(Fts3Hash *pH, int new_size){ struct _fts3ht *new_ht; /* The new hash table */ |
︙ | ︙ |
Changes to ext/fts3/fts3_porter.c.
︙ | ︙ | |||
252 253 254 255 256 257 258 | z[0]!='w' && z[0]!='x' && z[0]!='y' && isVowel(z+1) && isConsonant(z+2); } /* ** If the word ends with zFrom and xCond() is true for the stem | | | 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 | z[0]!='w' && z[0]!='x' && z[0]!='y' && isVowel(z+1) && isConsonant(z+2); } /* ** If the word ends with zFrom and xCond() is true for the stem ** of the word that precedes the zFrom ending, then change the ** ending to zTo. ** ** The input word *pz and zFrom are both in reverse order. zTo ** is in normal order. ** ** Return TRUE if zFrom matches. Return FALSE if zFrom does not ** match. Not that TRUE is returned even if xCond() fails and |
︙ | ︙ |
Changes to ext/fts3/fts3_snippet.c.
︙ | ︙ | |||
104 105 106 107 108 109 110 | ** for details. */ struct MatchinfoBuffer { u8 aRef[3]; int nElem; int bGlobal; /* Set if global data is loaded */ char *zMatchinfo; | | > > > > | 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 | ** for details. */ struct MatchinfoBuffer { u8 aRef[3]; int nElem; int bGlobal; /* Set if global data is loaded */ char *zMatchinfo; u32 aMI[FLEXARRAY]; }; /* Size (in bytes) of a MatchinfoBuffer sufficient for N elements */ #define SZ_MATCHINFOBUFFER(N) \ (offsetof(MatchinfoBuffer,aMI)+(((N)+1)/2)*sizeof(u64)) /* ** The snippet() and offsets() functions both return text values. An instance ** of the following structure is used to accumulate those values while the ** functions are running. See fts3StringAppend() for details. */ |
︙ | ︙ | |||
131 132 133 134 135 136 137 | /* ** Allocate a two-slot MatchinfoBuffer object. */ static MatchinfoBuffer *fts3MIBufferNew(size_t nElem, const char *zMatchinfo){ MatchinfoBuffer *pRet; sqlite3_int64 nByte = sizeof(u32) * (2*(sqlite3_int64)nElem + 1) | | | | | | | | | | | | 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 | /* ** Allocate a two-slot MatchinfoBuffer object. */ static MatchinfoBuffer *fts3MIBufferNew(size_t nElem, const char *zMatchinfo){ MatchinfoBuffer *pRet; sqlite3_int64 nByte = sizeof(u32) * (2*(sqlite3_int64)nElem + 1) + SZ_MATCHINFOBUFFER(1); sqlite3_int64 nStr = strlen(zMatchinfo); pRet = sqlite3Fts3MallocZero(nByte + nStr+1); if( pRet ){ pRet->aMI[0] = (u8*)(&pRet->aMI[1]) - (u8*)pRet; pRet->aMI[1+nElem] = pRet->aMI[0] + sizeof(u32)*((int)nElem+1); pRet->nElem = (int)nElem; pRet->zMatchinfo = ((char*)pRet) + nByte; memcpy(pRet->zMatchinfo, zMatchinfo, nStr+1); pRet->aRef[0] = 1; } return pRet; } static void fts3MIBufferFree(void *p){ MatchinfoBuffer *pBuf = (MatchinfoBuffer*)((u8*)p - ((u32*)p)[-1]); assert( (u32*)p==&pBuf->aMI[1] || (u32*)p==&pBuf->aMI[pBuf->nElem+2] ); if( (u32*)p==&pBuf->aMI[1] ){ pBuf->aRef[1] = 0; }else{ pBuf->aRef[2] = 0; } if( pBuf->aRef[0]==0 && pBuf->aRef[1]==0 && pBuf->aRef[2]==0 ){ sqlite3_free(pBuf); } } static void (*fts3MIBufferAlloc(MatchinfoBuffer *p, u32 **paOut))(void*){ void (*xRet)(void*) = 0; u32 *aOut = 0; if( p->aRef[1]==0 ){ p->aRef[1] = 1; aOut = &p->aMI[1]; xRet = fts3MIBufferFree; } else if( p->aRef[2]==0 ){ p->aRef[2] = 1; aOut = &p->aMI[p->nElem+2]; xRet = fts3MIBufferFree; }else{ aOut = (u32*)sqlite3_malloc64(p->nElem * sizeof(u32)); if( aOut ){ xRet = sqlite3_free; if( p->bGlobal ) memcpy(aOut, &p->aMI[1], p->nElem*sizeof(u32)); } } *paOut = aOut; return xRet; } static void fts3MIBufferSetGlobal(MatchinfoBuffer *p){ p->bGlobal = 1; memcpy(&p->aMI[2+p->nElem], &p->aMI[1], p->nElem*sizeof(u32)); } /* ** Free a MatchinfoBuffer object allocated using fts3MIBufferNew() */ void sqlite3Fts3MIBufferFree(MatchinfoBuffer *p){ if( p ){ |
︙ | ︙ | |||
394 395 396 397 398 399 400 401 402 403 404 405 406 407 | iEnd = pPhrase->iHead; } } if( iEnd==0x7FFFFFFF ){ return 1; } pIter->iCurrent = iStart = iEnd - pIter->nSnippet + 1; for(i=0; i<pIter->nPhrase; i++){ SnippetPhrase *pPhrase = &pIter->aPhrase[i]; fts3SnippetAdvance(&pPhrase->pHead, &pPhrase->iHead, iEnd+1); fts3SnippetAdvance(&pPhrase->pTail, &pPhrase->iTail, iStart); } } | > | 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 | iEnd = pPhrase->iHead; } } if( iEnd==0x7FFFFFFF ){ return 1; } assert( pIter->nSnippet>=0 ); pIter->iCurrent = iStart = iEnd - pIter->nSnippet + 1; for(i=0; i<pIter->nPhrase; i++){ SnippetPhrase *pPhrase = &pIter->aPhrase[i]; fts3SnippetAdvance(&pPhrase->pHead, &pPhrase->iHead, iEnd+1); fts3SnippetAdvance(&pPhrase->pTail, &pPhrase->iTail, iStart); } } |
︙ | ︙ | |||
442 443 444 445 446 447 448 | if( (mCover|mCovered)&mPhrase ){ iScore++; }else{ iScore += 1000; } mCover |= mPhrase; | | | 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 | if( (mCover|mCovered)&mPhrase ){ iScore++; }else{ iScore += 1000; } mCover |= mPhrase; for(j=0; j<pPhrase->nToken && j<pIter->nSnippet; j++){ mHighlight |= (mPos>>j); } if( 0==(*pCsr & 0x0FE) ) break; fts3GetDeltaPosition(&pCsr, &iCsr); } } |
︙ | ︙ | |||
606 607 608 609 610 611 612 | int nAppend /* Size of zAppend in bytes (or -1) */ ){ if( nAppend<0 ){ nAppend = (int)strlen(zAppend); } /* If there is insufficient space allocated at StrBuffer.z, use realloc() | | | 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 | int nAppend /* Size of zAppend in bytes (or -1) */ ){ if( nAppend<0 ){ nAppend = (int)strlen(zAppend); } /* If there is insufficient space allocated at StrBuffer.z, use realloc() ** to grow the buffer until so that it is big enough to accommodate the ** appended data. */ if( pStr->n+nAppend+1>=pStr->nAlloc ){ sqlite3_int64 nAlloc = pStr->nAlloc+(sqlite3_int64)nAppend+100; char *zNew = sqlite3_realloc64(pStr->z, nAlloc); if( !zNew ){ return SQLITE_NOMEM; |
︙ | ︙ | |||
1018 1019 1020 1021 1022 1023 1024 | case FTS3_MATCHINFO_AVGLENGTH: case FTS3_MATCHINFO_LENGTH: case FTS3_MATCHINFO_LCS: nVal = pInfo->nCol; break; case FTS3_MATCHINFO_LHITS: | | | | | 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 | case FTS3_MATCHINFO_AVGLENGTH: case FTS3_MATCHINFO_LENGTH: case FTS3_MATCHINFO_LCS: nVal = pInfo->nCol; break; case FTS3_MATCHINFO_LHITS: nVal = (size_t)pInfo->nCol * pInfo->nPhrase; break; case FTS3_MATCHINFO_LHITS_BM: nVal = (size_t)pInfo->nPhrase * ((pInfo->nCol + 31) / 32); break; default: assert( cArg==FTS3_MATCHINFO_HITS ); nVal = (size_t)pInfo->nCol * pInfo->nPhrase * 3; break; } return nVal; } static int fts3MatchinfoSelectDoctotal( |
︙ | ︙ | |||
1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 | pT->iOff = nTerm-iTerm-1; pT->pList = pList; pT->iPos = iPos; } return rc; } /* ** Implementation of offsets() function. */ void sqlite3Fts3Offsets( sqlite3_context *pCtx, /* SQLite function call context */ Fts3Cursor *pCsr /* Cursor object */ | > > > > > > > > > > > > > > > > | 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 | pT->iOff = nTerm-iTerm-1; pT->pList = pList; pT->iPos = iPos; } return rc; } /* ** If expression pExpr is a phrase expression that uses an MSR query, ** restart it as a regular, non-incremental query. Return SQLITE_OK ** if successful, or an SQLite error code otherwise. */ static int fts3ExprRestartIfCb(Fts3Expr *pExpr, int iPhrase, void *ctx){ TermOffsetCtx *p = (TermOffsetCtx*)ctx; int rc = SQLITE_OK; UNUSED_PARAMETER(iPhrase); if( pExpr->pPhrase && pExpr->pPhrase->bIncr ){ rc = sqlite3Fts3MsrCancel(p->pCsr, pExpr); pExpr->pPhrase->bIncr = 0; } return rc; } /* ** Implementation of offsets() function. */ void sqlite3Fts3Offsets( sqlite3_context *pCtx, /* SQLite function call context */ Fts3Cursor *pCsr /* Cursor object */ |
︙ | ︙ | |||
1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 | sCtx.aTerm = (TermOffset *)sqlite3Fts3MallocZero(sizeof(TermOffset)*nToken); if( 0==sCtx.aTerm ){ rc = SQLITE_NOMEM; goto offsets_out; } sCtx.iDocid = pCsr->iPrevId; sCtx.pCsr = pCsr; /* Loop through the table columns, appending offset information to ** string-buffer res for each column. */ for(iCol=0; iCol<pTab->nColumn; iCol++){ sqlite3_tokenizer_cursor *pC; /* Tokenizer cursor */ const char *ZDUMMY; /* Dummy argument used with xNext() */ | > > > > > > | 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 | sCtx.aTerm = (TermOffset *)sqlite3Fts3MallocZero(sizeof(TermOffset)*nToken); if( 0==sCtx.aTerm ){ rc = SQLITE_NOMEM; goto offsets_out; } sCtx.iDocid = pCsr->iPrevId; sCtx.pCsr = pCsr; /* If a query restart will be required, do it here, rather than later of ** after pointers to poslist buffers that may be invalidated by a restart ** have been saved. */ rc = sqlite3Fts3ExprIterate(pCsr->pExpr, fts3ExprRestartIfCb, (void*)&sCtx); if( rc!=SQLITE_OK ) goto offsets_out; /* Loop through the table columns, appending offset information to ** string-buffer res for each column. */ for(iCol=0; iCol<pTab->nColumn; iCol++){ sqlite3_tokenizer_cursor *pC; /* Tokenizer cursor */ const char *ZDUMMY; /* Dummy argument used with xNext() */ |
︙ | ︙ |
Changes to ext/fts3/fts3_term.c.
︙ | ︙ | |||
74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 | int iIndex = 0; UNUSED_PARAMETER(pCtx); if( argc==5 ){ iIndex = atoi(argv[4]); argc--; } /* The user should specify a single argument - the name of an fts3 table. */ if( argc!=4 ){ sqlite3Fts3ErrMsg(pzErr, "wrong number of arguments to fts4term constructor" ); return SQLITE_ERROR; } zDb = argv[1]; nDb = (int)strlen(zDb); zFts3 = argv[3]; nFts3 = (int)strlen(zFts3); rc = sqlite3_declare_vtab(db, FTS3_TERMS_SCHEMA); if( rc!=SQLITE_OK ) return rc; | > > | | < | > > > > > > | 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | int iIndex = 0; UNUSED_PARAMETER(pCtx); if( argc==5 ){ iIndex = atoi(argv[4]); argc--; } *ppVtab = 0; /* The user should specify a single argument - the name of an fts3 table. */ if( argc!=4 ){ sqlite3Fts3ErrMsg(pzErr, "wrong number of arguments to fts4term constructor" ); return SQLITE_ERROR; } zDb = argv[1]; nDb = (int)strlen(zDb); zFts3 = argv[3]; nFts3 = (int)strlen(zFts3); rc = sqlite3_declare_vtab(db, FTS3_TERMS_SCHEMA); if( rc!=SQLITE_OK ) return rc; nByte = sizeof(Fts3termTable); p = (Fts3termTable *)sqlite3Fts3MallocZero(nByte); if( !p ) return SQLITE_NOMEM; p->pFts3Tab = (Fts3Table*)sqlite3Fts3MallocZero( sizeof(Fts3Table) + nDb + nFts3 + 2 ); if( p->pFts3Tab==0 ){ sqlite3_free(p); return SQLITE_NOMEM; } p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1]; p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1]; p->pFts3Tab->db = db; p->pFts3Tab->nIndex = iIndex+1; p->iIndex = iIndex; memcpy((char *)p->pFts3Tab->zDb, zDb, nDb); |
︙ | ︙ | |||
126 127 128 129 130 131 132 133 134 135 136 137 138 139 | int i; /* Free any prepared statements held */ for(i=0; i<SizeofArray(pFts3->aStmt); i++){ sqlite3_finalize(pFts3->aStmt[i]); } sqlite3_free(pFts3->zSegmentsTbl); sqlite3_free(p); return SQLITE_OK; } #define FTS4AUX_EQ_CONSTRAINT 1 #define FTS4AUX_GE_CONSTRAINT 2 #define FTS4AUX_LE_CONSTRAINT 4 | > | 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 | int i; /* Free any prepared statements held */ for(i=0; i<SizeofArray(pFts3->aStmt); i++){ sqlite3_finalize(pFts3->aStmt[i]); } sqlite3_free(pFts3->zSegmentsTbl); sqlite3_free(pFts3); sqlite3_free(p); return SQLITE_OK; } #define FTS4AUX_EQ_CONSTRAINT 1 #define FTS4AUX_GE_CONSTRAINT 2 #define FTS4AUX_LE_CONSTRAINT 4 |
︙ | ︙ | |||
358 359 360 361 362 363 364 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ | | > | 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 /* xIntegrity */ }; int rc; /* Return code */ rc = sqlite3_create_module(db, "fts4term", &fts3term_module, 0); return rc; } #endif #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) */ |
Changes to ext/fts3/fts3_test.c.
︙ | ︙ | |||
14 15 16 17 18 19 20 | ** testing. It contains a Tcl command that can be used to test if a document ** matches an FTS NEAR expression. ** ** As of March 2012, it also contains a version 1 tokenizer used for testing ** that the sqlite3_tokenizer_module.xLanguage() method is invoked correctly. */ | < < < | < < < < | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | ** testing. It contains a Tcl command that can be used to test if a document ** matches an FTS NEAR expression. ** ** As of March 2012, it also contains a version 1 tokenizer used for testing ** that the sqlite3_tokenizer_module.xLanguage() method is invoked correctly. */ #include "tclsqlite.h" #include <string.h> #include <assert.h> #if defined(SQLITE_TEST) #if defined(SQLITE_ENABLE_FTS3) || defined(SQLITE_ENABLE_FTS4) /* Required so that the "ifdef SQLITE_ENABLE_FTS3" below works */ |
︙ | ︙ | |||
163 164 165 166 167 168 169 | NearPhrase *aPhrase = 0; NearDocument doc = {0, 0}; Tcl_Obj **apDocToken; Tcl_Obj *pRet; Tcl_Obj *pPhrasecount = 0; Tcl_Obj **apExprToken; | | > | 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 | NearPhrase *aPhrase = 0; NearDocument doc = {0, 0}; Tcl_Obj **apDocToken; Tcl_Obj *pRet; Tcl_Obj *pPhrasecount = 0; Tcl_Obj **apExprToken; Tcl_Size nExprToken; Tcl_Size nn; UNUSED_PARAMETER(clientData); /* Must have 3 or more arguments. */ if( objc<3 || (objc%2)==0 ){ Tcl_WrongNumArgs(interp, 1, objv, "DOCUMENT EXPR ?OPTION VALUE?..."); rc = TCL_ERROR; |
︙ | ︙ | |||
197 198 199 200 201 202 203 | switch( aOpt[iOpt].eOpt ){ case NM_PHRASECOUNTS: pPhrasecount = objv[ii+1]; break; } } | | > | > | | | | | > | | 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 | switch( aOpt[iOpt].eOpt ){ case NM_PHRASECOUNTS: pPhrasecount = objv[ii+1]; break; } } rc = Tcl_ListObjGetElements(interp, objv[1], &nn, &apDocToken); doc.nToken = (int)nn; if( rc!=TCL_OK ) goto near_match_out; doc.aToken = (NearToken *)ckalloc(doc.nToken*sizeof(NearToken)); for(ii=0; ii<doc.nToken; ii++){ doc.aToken[ii].z = Tcl_GetStringFromObj(apDocToken[ii], &nn); doc.aToken[ii].n = (int)nn; } rc = Tcl_ListObjGetElements(interp, objv[2], &nExprToken, &apExprToken); if( rc!=TCL_OK ) goto near_match_out; nPhrase = (int)(nExprToken + 1) / 2; aPhrase = (NearPhrase *)ckalloc(nPhrase * sizeof(NearPhrase)); memset(aPhrase, 0, nPhrase * sizeof(NearPhrase)); for(ii=0; ii<nPhrase; ii++){ Tcl_Obj *pPhrase = apExprToken[ii*2]; Tcl_Obj **apToken; Tcl_Size nToken; int jj; rc = Tcl_ListObjGetElements(interp, pPhrase, &nToken, &apToken); if( rc!=TCL_OK ) goto near_match_out; if( nToken>NM_MAX_TOKEN ){ Tcl_AppendResult(interp, "Too many tokens in phrase", NULL); rc = TCL_ERROR; goto near_match_out; } for(jj=0; jj<(int)nToken; jj++){ NearToken *pT = &aPhrase[ii].aToken[jj]; pT->z = Tcl_GetStringFromObj(apToken[jj], &nn); pT->n = (int)nn; } aPhrase[ii].nToken = (int)nToken; } for(ii=1; ii<nPhrase; ii++){ Tcl_Obj *pNear = apExprToken[2*ii-1]; int nNear; rc = Tcl_GetIntFromObj(interp, pNear, &nNear); if( rc!=TCL_OK ) goto near_match_out; aPhrase[ii].nNear = nNear; |
︙ | ︙ | |||
546 547 548 549 550 551 552 | if( rc!=TCL_OK ) return rc; nByte = sqlite3Fts3PutVarint(aBuf, w); nByte2 = sqlite3Fts3GetVarint(aBuf, &w2); if( w!=w2 || nByte!=nByte2 ){ char *zErr = sqlite3_mprintf("error testing %lld", w); Tcl_ResetResult(interp); | | | | 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 | if( rc!=TCL_OK ) return rc; nByte = sqlite3Fts3PutVarint(aBuf, w); nByte2 = sqlite3Fts3GetVarint(aBuf, &w2); if( w!=w2 || nByte!=nByte2 ){ char *zErr = sqlite3_mprintf("error testing %lld", w); Tcl_ResetResult(interp); Tcl_AppendResult(interp, zErr, NULL); return TCL_ERROR; } if( w<=2147483647 && w>=0 ){ int i; nByte2 = fts3GetVarint32(aBuf, &i); if( (int)w!=i || nByte!=nByte2 ){ char *zErr = sqlite3_mprintf("error testing %lld (32-bit)", w); Tcl_ResetResult(interp); Tcl_AppendResult(interp, zErr, NULL); return TCL_ERROR; } } #endif UNUSED_PARAMETER(clientData); return TCL_OK; |
︙ | ︙ |
Changes to ext/fts3/fts3_tokenize_vtab.c.
︙ | ︙ | |||
342 343 344 345 346 347 348 | Fts3tokTable *pTab = (Fts3tokTable *)(pCursor->pVtab); UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); fts3tokResetCursor(pCsr); if( idxNum==1 ){ const char *zByte = (const char *)sqlite3_value_text(apVal[0]); | | | 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 | Fts3tokTable *pTab = (Fts3tokTable *)(pCursor->pVtab); UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); fts3tokResetCursor(pCsr); if( idxNum==1 ){ const char *zByte = (const char *)sqlite3_value_text(apVal[0]); sqlite3_int64 nByte = sqlite3_value_bytes(apVal[0]); pCsr->zInput = sqlite3_malloc64(nByte+1); if( pCsr->zInput==0 ){ rc = SQLITE_NOMEM; }else{ if( nByte>0 ) memcpy(pCsr->zInput, zByte, nByte); pCsr->zInput[nByte] = 0; rc = pTab->pMod->xOpen(pTab->pTok, pCsr->zInput, nByte, &pCsr->pCsr); |
︙ | ︙ | |||
441 442 443 444 445 446 447 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ | | > | 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 /* xIntegrity */ }; int rc; /* Return code */ rc = sqlite3_create_module_v2( db, "fts3tokenize", &fts3tok_module, (void*)pHash, xDestroy ); return rc; } #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) */ |
Changes to ext/fts3/fts3_tokenizer.c.
︙ | ︙ | |||
222 223 224 225 226 227 228 | sqlite3_free(zCopy); return rc; } #ifdef SQLITE_TEST | < < < | < | 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 | sqlite3_free(zCopy); return rc; } #ifdef SQLITE_TEST #include "tclsqlite.h" #include <string.h> /* ** Implementation of a special SQL scalar function for testing tokenizers ** designed to be used in concert with the Tcl testing framework. This ** function must be called with two or more arguments: ** |
︙ | ︙ |
Changes to ext/fts3/fts3_write.c.
︙ | ︙ | |||
2663 2664 2665 2666 2667 2668 2669 | ** trying to resize the buffer, return SQLITE_NOMEM. */ static int fts3MsrBufferData( Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ char *pList, i64 nList ){ | | | | > > | 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 | ** trying to resize the buffer, return SQLITE_NOMEM. */ static int fts3MsrBufferData( Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ char *pList, i64 nList ){ if( (nList+FTS3_NODE_PADDING)>pMsr->nBuffer ){ char *pNew; int nNew = nList*2 + FTS3_NODE_PADDING; pNew = (char *)sqlite3_realloc64(pMsr->aBuffer, nNew); if( !pNew ) return SQLITE_NOMEM; pMsr->aBuffer = pNew; pMsr->nBuffer = nNew; } assert( nList>0 ); memcpy(pMsr->aBuffer, pList, nList); memset(&pMsr->aBuffer[nList], 0, FTS3_NODE_PADDING); return SQLITE_OK; } int sqlite3Fts3MsrIncrNext( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ sqlite3_int64 *piDocid, /* OUT: Docid value */ |
︙ | ︙ | |||
3319 3320 3321 3322 3323 3324 3325 | int rc = SQLITE_OK; int i; for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){ rc = fts3SegmentMerge(p, p->iPrevLangid, i, FTS3_SEGCURSOR_PENDING); if( rc==SQLITE_DONE ) rc = SQLITE_OK; } | < | 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 | int rc = SQLITE_OK; int i; for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){ rc = fts3SegmentMerge(p, p->iPrevLangid, i, FTS3_SEGCURSOR_PENDING); if( rc==SQLITE_DONE ) rc = SQLITE_OK; } /* Determine the auto-incr-merge setting if unknown. If enabled, ** estimate the number of leaf blocks of content to be written */ if( rc==SQLITE_OK && p->bHasStat && p->nAutoincrmerge==0xff && p->nLeafAdd>0 ){ |
︙ | ︙ | |||
3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 | if( p->nAutoincrmerge==1 ) p->nAutoincrmerge = 8; }else if( rc==SQLITE_DONE ){ p->nAutoincrmerge = 0; } rc = sqlite3_reset(pStmt); } } return rc; } /* ** Encode N integers as varints into a blob. */ static void fts3EncodeIntArray( | > > > > | 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 | if( p->nAutoincrmerge==1 ) p->nAutoincrmerge = 8; }else if( rc==SQLITE_DONE ){ p->nAutoincrmerge = 0; } rc = sqlite3_reset(pStmt); } } if( rc==SQLITE_OK ){ sqlite3Fts3PendingTermsClear(p); } return rc; } /* ** Encode N integers as varints into a blob. */ static void fts3EncodeIntArray( |
︙ | ︙ | |||
3947 3948 3949 3950 3951 3952 3953 | ** If the size of the value in blob pPrev is zero, then this is the first ** term written to the node. Otherwise, pPrev contains a copy of the ** previous term. Before this function returns, it is updated to contain a ** copy of zTerm/nTerm. ** ** It is assumed that the buffer associated with pNode is already large ** enough to accommodate the new entry. The buffer associated with pPrev | | | 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 | ** If the size of the value in blob pPrev is zero, then this is the first ** term written to the node. Otherwise, pPrev contains a copy of the ** previous term. Before this function returns, it is updated to contain a ** copy of zTerm/nTerm. ** ** It is assumed that the buffer associated with pNode is already large ** enough to accommodate the new entry. The buffer associated with pPrev ** is extended by this function if required. ** ** If an error (i.e. OOM condition) occurs, an SQLite error code is ** returned. Otherwise, SQLITE_OK. */ static int fts3AppendToNode( Blob *pNode, /* Current node image to append to */ Blob *pPrev, /* Buffer containing previous term written */ |
︙ | ︙ | |||
3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 3982 3983 3984 3985 | /* Node must have already been started. There must be a doclist for a ** leaf node, and there must not be a doclist for an internal node. */ assert( pNode->n>0 ); assert_fts3_nc( (pNode->a[0]=='\0')==(aDoclist!=0) ); blobGrowBuffer(pPrev, nTerm, &rc); if( rc!=SQLITE_OK ) return rc; nPrefix = fts3PrefixCompress(pPrev->a, pPrev->n, zTerm, nTerm); nSuffix = nTerm - nPrefix; if( nSuffix<=0 ) return FTS_CORRUPT_VTAB; memcpy(pPrev->a, zTerm, nTerm); pPrev->n = nTerm; | > > | 3977 3978 3979 3980 3981 3982 3983 3984 3985 3986 3987 3988 3989 3990 3991 3992 | /* Node must have already been started. There must be a doclist for a ** leaf node, and there must not be a doclist for an internal node. */ assert( pNode->n>0 ); assert_fts3_nc( (pNode->a[0]=='\0')==(aDoclist!=0) ); blobGrowBuffer(pPrev, nTerm, &rc); if( rc!=SQLITE_OK ) return rc; assert( pPrev!=0 ); assert( pPrev->a!=0 ); nPrefix = fts3PrefixCompress(pPrev->a, pPrev->n, zTerm, nTerm); nSuffix = nTerm - nPrefix; if( nSuffix<=0 ) return FTS_CORRUPT_VTAB; memcpy(pPrev->a, zTerm, nTerm); pPrev->n = nTerm; |
︙ | ︙ | |||
4028 4029 4030 4031 4032 4033 4034 | if(nSuffix<=0 ) return FTS_CORRUPT_VTAB; nSpace = sqlite3Fts3VarintLen(nPrefix); nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix; nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist; /* If the current block is not empty, and if adding this term/doclist | | > | > | > > | 4035 4036 4037 4038 4039 4040 4041 4042 4043 4044 4045 4046 4047 4048 4049 4050 4051 4052 4053 4054 4055 | if(nSuffix<=0 ) return FTS_CORRUPT_VTAB; nSpace = sqlite3Fts3VarintLen(nPrefix); nSpace += sqlite3Fts3VarintLen(nSuffix) + nSuffix; nSpace += sqlite3Fts3VarintLen(nDoclist) + nDoclist; /* If the current block is not empty, and if adding this term/doclist ** to the current block would make it larger than Fts3Table.nNodeSize bytes, ** and if there is still room for another leaf page, write this block out to ** the database. */ if( pLeaf->block.n>0 && (pLeaf->block.n + nSpace)>p->nNodeSize && pLeaf->iBlock < (pWriter->iStart + pWriter->nLeafEst) ){ rc = fts3WriteSegment(p, pLeaf->iBlock, pLeaf->block.a, pLeaf->block.n); pWriter->nWork++; /* Add the current term to the parent node. The term added to the ** parent must: ** ** a) be greater than the largest term on the leaf node just written |
︙ | ︙ | |||
4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 | memcpy(pNode->block.a, aRoot, nRoot); pNode->block.n = nRoot; memset(&pNode->block.a[nRoot], 0, FTS3_NODE_PADDING); } for(i=nHeight; i>=0 && rc==SQLITE_OK; i--){ NodeReader reader; pNode = &pWriter->aNodeWriter[i]; if( pNode->block.a){ rc = nodeReaderInit(&reader, pNode->block.a, pNode->block.n); while( reader.aNode && rc==SQLITE_OK ) rc = nodeReaderNext(&reader); blobGrowBuffer(&pNode->key, reader.term.n, &rc); if( rc==SQLITE_OK ){ assert_fts3_nc( reader.term.n>0 || reader.aNode==0 ); if( reader.term.n>0 ){ memcpy(pNode->key.a, reader.term.a, reader.term.n); } pNode->key.n = reader.term.n; if( i>0 ){ char *aBlock = 0; int nBlock = 0; pNode = &pWriter->aNodeWriter[i-1]; pNode->iBlock = reader.iChild; rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock,0); blobGrowBuffer(&pNode->block, MAX(nBlock, p->nNodeSize)+FTS3_NODE_PADDING, &rc | > | | 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 4362 4363 4364 4365 4366 4367 4368 4369 4370 4371 4372 4373 4374 4375 4376 4377 4378 4379 4380 4381 4382 4383 4384 4385 4386 4387 | memcpy(pNode->block.a, aRoot, nRoot); pNode->block.n = nRoot; memset(&pNode->block.a[nRoot], 0, FTS3_NODE_PADDING); } for(i=nHeight; i>=0 && rc==SQLITE_OK; i--){ NodeReader reader; memset(&reader, 0, sizeof(reader)); pNode = &pWriter->aNodeWriter[i]; if( pNode->block.a){ rc = nodeReaderInit(&reader, pNode->block.a, pNode->block.n); while( reader.aNode && rc==SQLITE_OK ) rc = nodeReaderNext(&reader); blobGrowBuffer(&pNode->key, reader.term.n, &rc); if( rc==SQLITE_OK ){ assert_fts3_nc( reader.term.n>0 || reader.aNode==0 ); if( reader.term.n>0 ){ memcpy(pNode->key.a, reader.term.a, reader.term.n); } pNode->key.n = reader.term.n; if( i>0 ){ char *aBlock = 0; int nBlock = 0; pNode = &pWriter->aNodeWriter[i-1]; pNode->iBlock = reader.iChild; rc = sqlite3Fts3ReadBlock(p, reader.iChild, &aBlock, &nBlock,0); blobGrowBuffer(&pNode->block, MAX(nBlock, p->nNodeSize)+FTS3_NODE_PADDING, &rc ); if( rc==SQLITE_OK ){ memcpy(pNode->block.a, aBlock, nBlock); pNode->block.n = nBlock; memset(&pNode->block.a[nBlock], 0, FTS3_NODE_PADDING); } sqlite3_free(aBlock); } |
︙ | ︙ | |||
5211 5212 5213 5214 5215 5216 5217 | int *pRc /* OUT: Return code */ ){ Fts3SegFilter filter; Fts3MultiSegReader csr; int rc; u64 cksum = 0; | | | 5223 5224 5225 5226 5227 5228 5229 5230 5231 5232 5233 5234 5235 5236 5237 | int *pRc /* OUT: Return code */ ){ Fts3SegFilter filter; Fts3MultiSegReader csr; int rc; u64 cksum = 0; if( *pRc ) return 0; memset(&filter, 0, sizeof(filter)); memset(&csr, 0, sizeof(csr)); filter.flags = FTS3_SEGMENT_REQUIRE_POS|FTS3_SEGMENT_IGNORE_EMPTY; filter.flags |= FTS3_SEGMENT_SCAN; rc = sqlite3Fts3SegReaderCursor( |
︙ | ︙ | |||
5278 5279 5280 5281 5282 5283 5284 | ** content table. If no error occurs and the contents do match, set *pbOk ** to true and return SQLITE_OK. Or if the contents do not match, set *pbOk ** to false before returning. ** ** If an error occurs (e.g. an OOM or IO error), return an SQLite error ** code. The final value of *pbOk is undefined in this case. */ | | | 5290 5291 5292 5293 5294 5295 5296 5297 5298 5299 5300 5301 5302 5303 5304 | ** content table. If no error occurs and the contents do match, set *pbOk ** to true and return SQLITE_OK. Or if the contents do not match, set *pbOk ** to false before returning. ** ** If an error occurs (e.g. an OOM or IO error), return an SQLite error ** code. The final value of *pbOk is undefined in this case. */ int sqlite3Fts3IntegrityCheck(Fts3Table *p, int *pbOk){ int rc = SQLITE_OK; /* Return code */ u64 cksum1 = 0; /* Checksum based on FTS index contents */ u64 cksum2 = 0; /* Checksum based on %_content contents */ sqlite3_stmt *pAllLangid = 0; /* Statement to return all language-ids */ /* This block calculates the checksum according to the FTS index. */ rc = fts3SqlStmt(p, SQL_SELECT_ALL_LANGID, &pAllLangid, 0); |
︙ | ︙ | |||
5356 5357 5358 5359 5360 5361 5362 | } } } sqlite3_finalize(pStmt); } | > > | > > > | 5368 5369 5370 5371 5372 5373 5374 5375 5376 5377 5378 5379 5380 5381 5382 5383 5384 5385 5386 5387 | } } } sqlite3_finalize(pStmt); } if( rc==SQLITE_CORRUPT_VTAB ){ rc = SQLITE_OK; *pbOk = 0; }else{ *pbOk = (rc==SQLITE_OK && cksum1==cksum2); } return rc; } /* ** Run the integrity-check. If no error occurs and the current contents of ** the FTS index are correct, return SQLITE_OK. Or, if the contents of the ** FTS index are incorrect, return SQLITE_CORRUPT_VTAB. |
︙ | ︙ | |||
5396 5397 5398 5399 5400 5401 5402 | ** passed. */ static int fts3DoIntegrityCheck( Fts3Table *p /* FTS3 table handle */ ){ int rc; int bOk = 0; | | | 5413 5414 5415 5416 5417 5418 5419 5420 5421 5422 5423 5424 5425 5426 5427 | ** passed. */ static int fts3DoIntegrityCheck( Fts3Table *p /* FTS3 table handle */ ){ int rc; int bOk = 0; rc = sqlite3Fts3IntegrityCheck(p, &bOk); if( rc==SQLITE_OK && bOk==0 ) rc = FTS_CORRUPT_VTAB; return rc; } /* ** Handle a 'special' INSERT of the form: ** |
︙ | ︙ | |||
5426 5427 5428 5429 5430 5431 5432 5433 | rc = fts3DoRebuild(p); }else if( nVal==15 && 0==sqlite3_strnicmp(zVal, "integrity-check", 15) ){ rc = fts3DoIntegrityCheck(p); }else if( nVal>6 && 0==sqlite3_strnicmp(zVal, "merge=", 6) ){ rc = fts3DoIncrmerge(p, &zVal[6]); }else if( nVal>10 && 0==sqlite3_strnicmp(zVal, "automerge=", 10) ){ rc = fts3DoAutoincrmerge(p, &zVal[10]); #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) | > > > | < > | 5443 5444 5445 5446 5447 5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 5461 5462 5463 5464 5465 5466 5467 5468 5469 5470 5471 5472 5473 5474 5475 5476 5477 5478 5479 5480 | rc = fts3DoRebuild(p); }else if( nVal==15 && 0==sqlite3_strnicmp(zVal, "integrity-check", 15) ){ rc = fts3DoIntegrityCheck(p); }else if( nVal>6 && 0==sqlite3_strnicmp(zVal, "merge=", 6) ){ rc = fts3DoIncrmerge(p, &zVal[6]); }else if( nVal>10 && 0==sqlite3_strnicmp(zVal, "automerge=", 10) ){ rc = fts3DoAutoincrmerge(p, &zVal[10]); }else if( nVal==5 && 0==sqlite3_strnicmp(zVal, "flush", 5) ){ rc = sqlite3Fts3PendingTermsFlush(p); } #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST) else{ int v; if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){ v = atoi(&zVal[9]); if( v>=24 && v<=p->nPgsz-35 ) p->nNodeSize = v; rc = SQLITE_OK; }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){ v = atoi(&zVal[11]); if( v>=64 && v<=FTS3_MAX_PENDING_DATA ) p->nMaxPendingData = v; rc = SQLITE_OK; }else if( nVal>21 && 0==sqlite3_strnicmp(zVal,"test-no-incr-doclist=",21) ){ p->bNoIncrDoclist = atoi(&zVal[21]); rc = SQLITE_OK; }else if( nVal>11 && 0==sqlite3_strnicmp(zVal,"mergecount=",11) ){ v = atoi(&zVal[11]); if( v>=4 && v<=FTS3_MERGE_COUNT && (v&1)==0 ) p->nMergeCount = v; rc = SQLITE_OK; } } #endif return rc; } #ifndef SQLITE_DISABLE_FTS4_DEFERRED /* ** Delete all cached deferred doclists. Deferred doclists are cached ** (allocated) by the sqlite3Fts3CacheDeferredDoclists() function. |
︙ | ︙ | |||
5595 5596 5597 5598 5599 5600 5601 | return SQLITE_OK; } #endif /* ** SQLite value pRowid contains the rowid of a row that may or may not be ** present in the FTS3 table. If it is, delete it and adjust the contents | | | 5615 5616 5617 5618 5619 5620 5621 5622 5623 5624 5625 5626 5627 5628 5629 | return SQLITE_OK; } #endif /* ** SQLite value pRowid contains the rowid of a row that may or may not be ** present in the FTS3 table. If it is, delete it and adjust the contents ** of subsidiary data structures accordingly. */ static int fts3DeleteByRowid( Fts3Table *p, sqlite3_value *pRowid, int *pnChng, /* IN/OUT: Decrement if row is deleted */ u32 *aSzDel ){ |
︙ | ︙ |
Changes to ext/fts3/unicode/mkunicode.tcl.
︙ | ︙ | |||
624 625 626 627 628 629 630 631 632 633 634 635 636 637 | $caseC $caseL $caseM $caseN $caseP $caseS $caseZ } return 0; } }] set nRepeat 0 set first [lindex $lMap 0 0] | > > > | 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 | $caseC $caseL $caseM $caseN $caseP $caseS $caseZ default: return 1; } return 0; } }] set nRepeat 0 set first [lindex $lMap 0 0] |
︙ | ︙ | |||
886 887 888 889 890 891 892 | puts " if( r3 ) printf(\"categories(): Problem with code %d\\n\",code);" puts " else printf(\"categories(): test passed\\n\");" } puts " return (r1 || r2 || r3);" puts "\}" } | | | 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 | puts " if( r3 ) printf(\"categories(): Problem with code %d\\n\",code);" puts " else printf(\"categories(): test passed\\n\");" } puts " return (r1 || r2 || r3);" puts "\}" } # Process the command line arguments. Exit early if they are not to # our liking. # proc usage {} { puts -nonewline stderr "Usage: $::argv0 ?-test? ?-fts5? " puts stderr "<CaseFolding.txt file> <UnicodeData.txt file>" exit 1 } |
︙ | ︙ |
Changes to ext/fts5/extract_api_docs.tcl.
︙ | ︙ | |||
78 79 80 81 82 83 84 | if {$current_header!=""} { append current_doc "\n" } } else { if {$current_doc != ""} { lappend res $current_header $current_doc set current_doc "" } set subject n/a | | | 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | if {$current_header!=""} { append current_doc "\n" } } else { if {$current_doc != ""} { lappend res $current_header $current_doc set current_doc "" } set subject n/a regexp {^ *([[:alnum:]_]*)} $line -> subject if {[lsearch $names $subject]>=0} { set current_header $subject } else { set current_header [string trim $line] } } } |
︙ | ︙ | |||
103 104 105 106 107 108 109 110 111 | set res "<dl>\n" foreach line [split [string trim $docs] "\n"] { regexp {[*][*](.*)} $line -> line if {[regexp {^ ?x.*:} $line]} { append res "<dt><b>$line</b></dt><dd><p style=margin-top:0>\n" continue } if {[regexp {SYNONYM SUPPORT} $line]} { | > > > | | 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 | set res "<dl>\n" foreach line [split [string trim $docs] "\n"] { regexp {[*][*](.*)} $line -> line if {[regexp {^ ?x.*:} $line]} { append res "<dt><b>$line</b></dt><dd><p style=margin-top:0>\n" continue } if {[regexp {FTS5_TOKENIZER} $line]} { set line </dl><p> } if {[regexp {SYNONYM SUPPORT} $line]} { set line "<h3>Synonym Support</h3>" } if {[string trim $line] == ""} { append res "<p>\n" } else { append res "$line\n" } } |
︙ | ︙ | |||
219 220 221 222 223 224 225 226 227 | fts5_extension { output [get_fts5_struct $data "typedef.*Fts5ExtensionApi" "^.;"] } Fts5ExtensionApi { set struct [get_fts5_struct $data "^struct Fts5ExtensionApi" "^.;"] set map [list] foreach {k v} [get_struct_members $data] { if {[string match x* $k]==0} continue | > | > | 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 | fts5_extension { output [get_fts5_struct $data "typedef.*Fts5ExtensionApi" "^.;"] } Fts5ExtensionApi { set struct [get_fts5_struct $data "^struct Fts5ExtensionApi" "^.;"] set map [list] set lKey [list] foreach {k v} [get_struct_members $data] { if {[string match x* $k]==0} continue lappend lKey $k } foreach k [lsort -decr $lKey] { lappend map $k "<a href=#$k>$k</a>" } output [string map $map $struct] } api { get_api_docs $data } |
︙ | ︙ |
Changes to ext/fts5/fts5.h.
︙ | ︙ | |||
51 52 53 54 55 56 57 | const unsigned char *b; }; /* ** EXTENSION API FUNCTIONS ** ** xUserData(pFts): | | | | 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | const unsigned char *b; }; /* ** EXTENSION API FUNCTIONS ** ** xUserData(pFts): ** Return a copy of the pUserData pointer passed to the xCreateFunction() ** API when the extension function was registered. ** ** xColumnTotalSize(pFts, iCol, pnToken): ** If parameter iCol is less than zero, set output variable *pnToken ** to the total number of tokens in the FTS5 table. Or, if iCol is ** non-negative but less than the number of columns in the table, return ** the total number of tokens in column iCol, considering all rows in ** the FTS5 table. |
︙ | ︙ | |||
84 85 86 87 88 89 90 | ** an OOM condition or IO error), an appropriate SQLite error code is ** returned. ** ** This function may be quite inefficient if used with an FTS5 table ** created with the "columnsize=0" option. ** ** xColumnText: | > > > | | > > | | | > | | | | 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 | ** an OOM condition or IO error), an appropriate SQLite error code is ** returned. ** ** This function may be quite inefficient if used with an FTS5 table ** created with the "columnsize=0" option. ** ** xColumnText: ** If parameter iCol is less than zero, or greater than or equal to the ** number of columns in the table, SQLITE_RANGE is returned. ** ** Otherwise, this function attempts to retrieve the text of column iCol of ** the current document. If successful, (*pz) is set to point to a buffer ** containing the text in utf-8 encoding, (*pn) is set to the size in bytes ** (not characters) of the buffer and SQLITE_OK is returned. Otherwise, ** if an error occurs, an SQLite error code is returned and the final values ** of (*pz) and (*pn) are undefined. ** ** xPhraseCount: ** Returns the number of phrases in the current query expression. ** ** xPhraseSize: ** If parameter iCol is less than zero, or greater than or equal to the ** number of phrases in the current query, as returned by xPhraseCount, ** 0 is returned. Otherwise, this function returns the number of tokens in ** phrase iPhrase of the query. Phrases are numbered starting from zero. ** ** xInstCount: ** Set *pnInst to the total number of occurrences of all phrases within ** the query within the current row. Return SQLITE_OK if successful, or ** an error code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. If the FTS5 table is created ** with either "detail=none" or "detail=column" and "content=" option ** (i.e. if it is a contentless table), then this API always returns 0. ** ** xInst: ** Query for the details of phrase match iIdx within the current row. ** Phrase matches are numbered starting from zero, so the iIdx argument ** should be greater than or equal to zero and smaller than the value ** output by xInstCount(). If iIdx is less than zero or greater than ** or equal to the value returned by xInstCount(), SQLITE_RANGE is returned. ** ** Otherwise, output parameter *piPhrase is set to the phrase number, *piCol ** to the column in which it occurs and *piOff the token offset of the ** first token of the phrase. SQLITE_OK is returned if successful, or an ** error code (i.e. SQLITE_NOMEM) if an error occurs. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. ** ** xRowid: ** Returns the rowid of the current row. ** |
︙ | ︙ | |||
142 143 144 145 146 147 148 149 150 151 152 153 154 155 | ** current query is executed. Any column filter that applies to ** phrase iPhrase of the current query is included in $p. For each ** row visited, the callback function passed as the fourth argument ** is invoked. The context and API objects passed to the callback ** function may be used to access the properties of each matched row. ** Invoking Api.xUserData() returns a copy of the pointer passed as ** the third argument to pUserData. ** ** If the callback function returns any value other than SQLITE_OK, the ** query is abandoned and the xQueryPhrase function returns immediately. ** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. ** Otherwise, the error code is propagated upwards. ** ** If the query runs to completion without incident, SQLITE_OK is returned. | > > > > | 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 | ** current query is executed. Any column filter that applies to ** phrase iPhrase of the current query is included in $p. For each ** row visited, the callback function passed as the fourth argument ** is invoked. The context and API objects passed to the callback ** function may be used to access the properties of each matched row. ** Invoking Api.xUserData() returns a copy of the pointer passed as ** the third argument to pUserData. ** ** If parameter iPhrase is less than zero, or greater than or equal to ** the number of phrases in the query, as returned by xPhraseCount(), ** this function returns SQLITE_RANGE. ** ** If the callback function returns any value other than SQLITE_OK, the ** query is abandoned and the xQueryPhrase function returns immediately. ** If the returned value is SQLITE_DONE, xQueryPhrase returns SQLITE_OK. ** Otherwise, the error code is propagated upwards. ** ** If the query runs to completion without incident, SQLITE_OK is returned. |
︙ | ︙ | |||
224 225 226 227 228 229 230 231 232 233 234 235 236 237 | ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. If the FTS5 table is created ** with either "detail=none" or "detail=column" and "content=" option ** (i.e. if it is a contentless table), then this API always iterates ** through an empty set (all calls to xPhraseFirst() set iCol to -1). ** ** xPhraseNext() ** See xPhraseFirst above. ** ** xPhraseFirstColumn() ** This function and xPhraseNextColumn() are similar to the xPhraseFirst() ** and xPhraseNext() APIs described above. The difference is that instead ** of iterating through all instances of a phrase in the current row, these | > > > > | 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. If the FTS5 table is created ** with either "detail=none" or "detail=column" and "content=" option ** (i.e. if it is a contentless table), then this API always iterates ** through an empty set (all calls to xPhraseFirst() set iCol to -1). ** ** In all cases, matches are visited in (column ASC, offset ASC) order. ** i.e. all those in column 0, sorted by offset, followed by those in ** column 1, etc. ** ** xPhraseNext() ** See xPhraseFirst above. ** ** xPhraseFirstColumn() ** This function and xPhraseNextColumn() are similar to the xPhraseFirst() ** and xPhraseNext() APIs described above. The difference is that instead ** of iterating through all instances of a phrase in the current row, these |
︙ | ︙ | |||
257 258 259 260 261 262 263 264 265 | ** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext ** (or xInst/xInstCount). The chief advantage of this API is that it is ** significantly more efficient than those alternatives when used with ** "detail=column" tables. ** ** xPhraseNextColumn() ** See xPhraseFirstColumn above. */ struct Fts5ExtensionApi { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 | ** xPhraseFirstColumn() may also be obtained using xPhraseFirst/xPhraseNext ** (or xInst/xInstCount). The chief advantage of this API is that it is ** significantly more efficient than those alternatives when used with ** "detail=column" tables. ** ** xPhraseNextColumn() ** See xPhraseFirstColumn above. ** ** xQueryToken(pFts5, iPhrase, iToken, ppToken, pnToken) ** This is used to access token iToken of phrase iPhrase of the current ** query. Before returning, output parameter *ppToken is set to point ** to a buffer containing the requested token, and *pnToken to the ** size of this buffer in bytes. ** ** If iPhrase or iToken are less than zero, or if iPhrase is greater than ** or equal to the number of phrases in the query as reported by ** xPhraseCount(), or if iToken is equal to or greater than the number of ** tokens in the phrase, SQLITE_RANGE is returned and *ppToken and *pnToken are both zeroed. ** ** The output text is not a copy of the query text that specified the ** token. It is the output of the tokenizer module. For tokendata=1 ** tables, this includes any embedded 0x00 and trailing data. ** ** xInstToken(pFts5, iIdx, iToken, ppToken, pnToken) ** This is used to access token iToken of phrase hit iIdx within the ** current row. If iIdx is less than zero or greater than or equal to the ** value returned by xInstCount(), SQLITE_RANGE is returned. Otherwise, ** output variable (*ppToken) is set to point to a buffer containing the ** matching document token, and (*pnToken) to the size of that buffer in ** bytes. ** ** The output text is not a copy of the document text that was tokenized. ** It is the output of the tokenizer module. For tokendata=1 tables, this ** includes any embedded 0x00 and trailing data. ** ** This API may be slow in some cases if the token identified by parameters ** iIdx and iToken matched a prefix token in the query. In most cases, the ** first call to this API for each prefix token in the query is forced ** to scan the portion of the full-text index that matches the prefix ** token to collect the extra data required by this API. If the prefix ** token matches a large number of token instances in the document set, ** this may be a performance problem. ** ** If the user knows in advance that a query may use this API for a ** prefix token, FTS5 may be configured to collect all required data as part ** of the initial querying of the full-text index, avoiding the second scan ** entirely. This also causes prefix queries that do not use this API to ** run more slowly and use more memory. FTS5 may be configured in this way ** either on a per-table basis using the [FTS5 insttoken | 'insttoken'] ** option, or on a per-query basis using the ** [fts5_insttoken | fts5_insttoken()] user function. ** ** This API can be quite slow if used with an FTS5 table created with the ** "detail=none" or "detail=column" option. ** ** xColumnLocale(pFts5, iIdx, pzLocale, pnLocale) ** If parameter iCol is less than zero, or greater than or equal to the ** number of columns in the table, SQLITE_RANGE is returned. ** ** Otherwise, this function attempts to retrieve the locale associated ** with column iCol of the current row. Usually, there is no associated ** locale, and output parameters (*pzLocale) and (*pnLocale) are set ** to NULL and 0, respectively. However, if the fts5_locale() function ** was used to associate a locale with the value when it was inserted ** into the fts5 table, then (*pzLocale) is set to point to a nul-terminated ** buffer containing the name of the locale in utf-8 encoding. (*pnLocale) ** is set to the size in bytes of the buffer, not including the ** nul-terminator. ** ** If successful, SQLITE_OK is returned. Or, if an error occurs, an ** SQLite error code is returned. The final value of the output parameters ** is undefined in this case. ** ** xTokenize_v2: ** Tokenize text using the tokenizer belonging to the FTS5 table. This ** API is the same as the xTokenize() API, except that it allows a tokenizer ** locale to be specified. */ struct Fts5ExtensionApi { int iVersion; /* Currently always set to 4 */ void *(*xUserData)(Fts5Context*); int (*xColumnCount)(Fts5Context*); int (*xRowCount)(Fts5Context*, sqlite3_int64 *pnRow); int (*xColumnTotalSize)(Fts5Context*, int iCol, sqlite3_int64 *pnToken); |
︙ | ︙ | |||
294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 | void *(*xGetAuxdata)(Fts5Context*, int bClear); int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*); void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff); int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*); void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol); }; /* ** CUSTOM AUXILIARY FUNCTIONS *************************************************************************/ /************************************************************************* ** CUSTOM TOKENIZERS ** ** Applications may also register custom tokenizer types. A tokenizer ** is registered by providing fts5 with a populated instance of the ** following structure. All structure methods must be defined, setting ** any member of the fts5_tokenizer struct to NULL leads to undefined ** behaviour. The structure methods are expected to function as follows: ** ** xCreate: ** This function is used to allocate and initialize a tokenizer instance. ** A tokenizer instance is required to actually tokenize text. ** ** The first argument passed to this function is a copy of the (void*) | > > > > > > > > > > > > > > > > | | 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 | void *(*xGetAuxdata)(Fts5Context*, int bClear); int (*xPhraseFirst)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*, int*); void (*xPhraseNext)(Fts5Context*, Fts5PhraseIter*, int *piCol, int *piOff); int (*xPhraseFirstColumn)(Fts5Context*, int iPhrase, Fts5PhraseIter*, int*); void (*xPhraseNextColumn)(Fts5Context*, Fts5PhraseIter*, int *piCol); /* Below this point are iVersion>=3 only */ int (*xQueryToken)(Fts5Context*, int iPhrase, int iToken, const char **ppToken, int *pnToken ); int (*xInstToken)(Fts5Context*, int iIdx, int iToken, const char**, int*); /* Below this point are iVersion>=4 only */ int (*xColumnLocale)(Fts5Context*, int iCol, const char **pz, int *pn); int (*xTokenize_v2)(Fts5Context*, const char *pText, int nText, /* Text to tokenize */ const char *pLocale, int nLocale, /* Locale to pass to tokenizer */ void *pCtx, /* Context passed to xToken() */ int (*xToken)(void*, int, const char*, int, int, int) /* Callback */ ); }; /* ** CUSTOM AUXILIARY FUNCTIONS *************************************************************************/ /************************************************************************* ** CUSTOM TOKENIZERS ** ** Applications may also register custom tokenizer types. A tokenizer ** is registered by providing fts5 with a populated instance of the ** following structure. All structure methods must be defined, setting ** any member of the fts5_tokenizer struct to NULL leads to undefined ** behaviour. The structure methods are expected to function as follows: ** ** xCreate: ** This function is used to allocate and initialize a tokenizer instance. ** A tokenizer instance is required to actually tokenize text. ** ** The first argument passed to this function is a copy of the (void*) ** pointer provided by the application when the fts5_tokenizer_v2 object ** was registered with FTS5 (the third argument to xCreateTokenizer()). ** The second and third arguments are an array of nul-terminated strings ** containing the tokenizer arguments, if any, specified following the ** tokenizer name as part of the CREATE VIRTUAL TABLE statement used ** to create the FTS5 table. ** ** The final argument is an output variable. If successful, (*ppOut) |
︙ | ︙ | |||
338 339 340 341 342 343 344 | ** ** xTokenize: ** This function is expected to tokenize the nText byte string indicated ** by argument pText. pText may or may not be nul-terminated. The first ** argument passed to this function is a pointer to an Fts5Tokenizer object ** returned by an earlier call to xCreate(). ** | | | 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 | ** ** xTokenize: ** This function is expected to tokenize the nText byte string indicated ** by argument pText. pText may or may not be nul-terminated. The first ** argument passed to this function is a pointer to an Fts5Tokenizer object ** returned by an earlier call to xCreate(). ** ** The third argument indicates the reason that FTS5 is requesting ** tokenization of the supplied text. This is always one of the following ** four values: ** ** <ul><li> <b>FTS5_TOKENIZE_DOCUMENT</b> - A document is being inserted into ** or removed from the FTS table. The tokenizer is being invoked to ** determine the set of tokens to add to (or delete from) the ** FTS index. |
︙ | ︙ | |||
361 362 363 364 365 366 367 368 369 370 371 372 373 374 | ** returned by the tokenizer will be treated as a token prefix. ** ** <li> <b>FTS5_TOKENIZE_AUX</b> - The tokenizer is being invoked to ** satisfy an fts5_api.xTokenize() request made by an auxiliary ** function. Or an fts5_api.xColumnSize() request made by the same ** on a columnsize=0 database. ** </ul> ** ** For each token in the input string, the supplied callback xToken() must ** be invoked. The first argument to it should be a copy of the pointer ** passed as the second argument to xTokenize(). The third and fourth ** arguments are a pointer to a buffer containing the token text, and the ** size of the token in bytes. The 4th and 5th arguments are the byte offsets ** of the first byte of and first byte immediately following the text from | > > > > > > > | 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 | ** returned by the tokenizer will be treated as a token prefix. ** ** <li> <b>FTS5_TOKENIZE_AUX</b> - The tokenizer is being invoked to ** satisfy an fts5_api.xTokenize() request made by an auxiliary ** function. Or an fts5_api.xColumnSize() request made by the same ** on a columnsize=0 database. ** </ul> ** ** The sixth and seventh arguments passed to xTokenize() - pLocale and ** nLocale - are a pointer to a buffer containing the locale to use for ** tokenization (e.g. "en_US") and its size in bytes, respectively. The ** pLocale buffer is not nul-terminated. pLocale may be passed NULL (in ** which case nLocale is always 0) to indicate that the tokenizer should ** use its default locale. ** ** For each token in the input string, the supplied callback xToken() must ** be invoked. The first argument to it should be a copy of the pointer ** passed as the second argument to xTokenize(). The third and fourth ** arguments are a pointer to a buffer containing the token text, and the ** size of the token in bytes. The 4th and 5th arguments are the byte offsets ** of the first byte of and first byte immediately following the text from |
︙ | ︙ | |||
384 385 386 387 388 389 390 391 392 393 394 395 396 397 | ** If an xToken() callback returns any value other than SQLITE_OK, then ** the tokenization should be abandoned and the xTokenize() method should ** immediately return a copy of the xToken() return value. Or, if the ** input buffer is exhausted, xTokenize() should return SQLITE_OK. Finally, ** if an error occurs with the xTokenize() implementation itself, it ** may abandon the tokenization and return any error code other than ** SQLITE_OK or SQLITE_DONE. ** ** SYNONYM SUPPORT ** ** Custom tokenizers may also support synonyms. Consider a case in which a ** user wishes to query for a phrase such as "first place". Using the ** built-in tokenizers, the FTS5 query 'first + place' will match instances ** of "first place" within the document set, but not alternative forms | > > > > > > > > > > > > > > > > > > > > > > > > | 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 | ** If an xToken() callback returns any value other than SQLITE_OK, then ** the tokenization should be abandoned and the xTokenize() method should ** immediately return a copy of the xToken() return value. Or, if the ** input buffer is exhausted, xTokenize() should return SQLITE_OK. Finally, ** if an error occurs with the xTokenize() implementation itself, it ** may abandon the tokenization and return any error code other than ** SQLITE_OK or SQLITE_DONE. ** ** If the tokenizer is registered using an fts5_tokenizer_v2 object, ** then the xTokenize() method has two additional arguments - pLocale ** and nLocale. These specify the locale that the tokenizer should use ** for the current request. If pLocale and nLocale are both 0, then the ** tokenizer should use its default locale. Otherwise, pLocale points to ** an nLocale byte buffer containing the name of the locale to use as utf-8 ** text. pLocale is not nul-terminated. ** ** FTS5_TOKENIZER ** ** There is also an fts5_tokenizer object. This is an older, deprecated, ** version of fts5_tokenizer_v2. It is similar except that: ** ** <ul> ** <li> There is no "iVersion" field, and ** <li> The xTokenize() method does not take a locale argument. ** </ul> ** ** Legacy fts5_tokenizer tokenizers must be registered using the ** legacy xCreateTokenizer() function, instead of xCreateTokenizer_v2(). ** ** Tokenizer implementations registered using either API may be retrieved ** using both xFindTokenizer() and xFindTokenizer_v2(). ** ** SYNONYM SUPPORT ** ** Custom tokenizers may also support synonyms. Consider a case in which a ** user wishes to query for a phrase such as "first place". Using the ** built-in tokenizers, the FTS5 query 'first + place' will match instances ** of "first place" within the document set, but not alternative forms |
︙ | ︙ | |||
488 489 490 491 492 493 494 | ** provide synonyms for prefixes). However, a non-prefix query like '1st' ** will match against "1st" and "first". This method does not require ** extra disk space, as no extra entries are added to the FTS index. ** On the other hand, it may require more CPU cycles to run MATCH queries, ** as separate queries of the FTS index are required for each synonym. ** ** When using methods (2) or (3), it is important that the tokenizer only | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | > > > > > > > > > > > > > > > > > > > | 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 | ** provide synonyms for prefixes). However, a non-prefix query like '1st' ** will match against "1st" and "first". This method does not require ** extra disk space, as no extra entries are added to the FTS index. ** On the other hand, it may require more CPU cycles to run MATCH queries, ** as separate queries of the FTS index are required for each synonym. ** ** When using methods (2) or (3), it is important that the tokenizer only ** provide synonyms when tokenizing document text (method (3)) or query ** text (method (2)), not both. Doing so will not cause any errors, but is ** inefficient. */ typedef struct Fts5Tokenizer Fts5Tokenizer; typedef struct fts5_tokenizer_v2 fts5_tokenizer_v2; struct fts5_tokenizer_v2 { int iVersion; /* Currently always 2 */ int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut); void (*xDelete)(Fts5Tokenizer*); int (*xTokenize)(Fts5Tokenizer*, void *pCtx, int flags, /* Mask of FTS5_TOKENIZE_* flags */ const char *pText, int nText, const char *pLocale, int nLocale, int (*xToken)( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iStart, /* Byte offset of token within input text */ int iEnd /* Byte offset of end of token within input text */ ) ); }; /* ** New code should use the fts5_tokenizer_v2 type to define tokenizer ** implementations. The following type is included for legacy applications ** that still use it. */ typedef struct fts5_tokenizer fts5_tokenizer; struct fts5_tokenizer { int (*xCreate)(void*, const char **azArg, int nArg, Fts5Tokenizer **ppOut); void (*xDelete)(Fts5Tokenizer*); int (*xTokenize)(Fts5Tokenizer*, void *pCtx, int flags, /* Mask of FTS5_TOKENIZE_* flags */ const char *pText, int nText, int (*xToken)( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iStart, /* Byte offset of token within input text */ int iEnd /* Byte offset of end of token within input text */ ) ); }; /* Flags that may be passed as the third argument to xTokenize() */ #define FTS5_TOKENIZE_QUERY 0x0001 #define FTS5_TOKENIZE_PREFIX 0x0002 #define FTS5_TOKENIZE_DOCUMENT 0x0004 #define FTS5_TOKENIZE_AUX 0x0008 /* Flags that may be passed by the tokenizer implementation back to FTS5 ** as the third argument to the supplied xToken callback. */ #define FTS5_TOKEN_COLOCATED 0x0001 /* Same position as prev. token */ /* ** END OF CUSTOM TOKENIZERS *************************************************************************/ /************************************************************************* ** FTS5 EXTENSION REGISTRATION API */ typedef struct fts5_api fts5_api; struct fts5_api { int iVersion; /* Currently always set to 3 */ /* Create a new tokenizer */ int (*xCreateTokenizer)( fts5_api *pApi, const char *zName, void *pUserData, fts5_tokenizer *pTokenizer, void (*xDestroy)(void*) ); /* Find an existing tokenizer */ int (*xFindTokenizer)( fts5_api *pApi, const char *zName, void **ppUserData, fts5_tokenizer *pTokenizer ); /* Create a new auxiliary function */ int (*xCreateFunction)( fts5_api *pApi, const char *zName, void *pUserData, fts5_extension_function xFunction, void (*xDestroy)(void*) ); /* APIs below this point are only available if iVersion>=3 */ /* Create a new tokenizer */ int (*xCreateTokenizer_v2)( fts5_api *pApi, const char *zName, void *pUserData, fts5_tokenizer_v2 *pTokenizer, void (*xDestroy)(void*) ); /* Find an existing tokenizer */ int (*xFindTokenizer_v2)( fts5_api *pApi, const char *zName, void **ppUserData, fts5_tokenizer_v2 **ppTokenizer ); }; /* ** END OF REGISTRATION API *************************************************************************/ #ifdef __cplusplus } /* end of the 'extern "C"' block */ #endif #endif /* _FTS5_H */ |
Changes to ext/fts5/fts5Int.h.
︙ | ︙ | |||
16 17 18 19 20 21 22 23 24 25 26 27 28 29 | #include "fts5.h" #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> #ifndef SQLITE_AMALGAMATION typedef unsigned char u8; typedef unsigned int u32; typedef unsigned short u16; typedef short i16; | > | 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | #include "fts5.h" #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> #include <stddef.h> #ifndef SQLITE_AMALGAMATION typedef unsigned char u8; typedef unsigned int u32; typedef unsigned short u16; typedef short i16; |
︙ | ︙ | |||
54 55 56 57 58 59 60 61 62 63 64 65 66 67 | #define MAX(x,y) (((x) > (y)) ? (x) : (y)) /* ** Constants for the largest and smallest possible 64-bit signed integers. */ # define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) # define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) #endif /* Truncate very long tokens to this many bytes. Hard limit is ** (65536-1-1-4-9)==65521 bytes. The limiting factor is the 16-bit offset ** field that occurs at the start of each leaf page (see fts5_index.c). */ #define FTS5_MAX_TOKEN_SIZE 32768 | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | #define MAX(x,y) (((x) > (y)) ? (x) : (y)) /* ** Constants for the largest and smallest possible 64-bit signed integers. */ # define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) # define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) /* The uptr type is an unsigned integer large enough to hold a pointer */ #if defined(HAVE_STDINT_H) typedef uintptr_t uptr; #elif SQLITE_PTRSIZE==4 typedef u32 uptr; #else typedef u64 uptr; #endif #ifdef SQLITE_4_BYTE_ALIGNED_MALLOC # define EIGHT_BYTE_ALIGNMENT(X) ((((uptr)(X) - (uptr)0)&3)==0) #else # define EIGHT_BYTE_ALIGNMENT(X) ((((uptr)(X) - (uptr)0)&7)==0) #endif /* ** Macros needed to provide flexible arrays in a portable way */ #ifndef offsetof # define offsetof(ST,M) ((size_t)((char*)&((ST*)0)->M - (char*)0)) #endif #if defined(__STDC_VERSION__) && (__STDC_VERSION__ >= 199901L) # define FLEXARRAY #else # define FLEXARRAY 1 #endif #endif /* Truncate very long tokens to this many bytes. Hard limit is ** (65536-1-1-4-9)==65521 bytes. The limiting factor is the 16-bit offset ** field that occurs at the start of each leaf page (see fts5_index.c). */ #define FTS5_MAX_TOKEN_SIZE 32768 |
︙ | ︙ | |||
127 128 129 130 131 132 133 | ** then an object of the following type is used to record the set of columns. ** Each entry in the aiCol[] array is a column that may be matched. ** ** This object is used by fts5_expr.c and fts5_index.c. */ struct Fts5Colset { int nCol; | | | > > > > > > > > > > > > > > > > > | 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | ** then an object of the following type is used to record the set of columns. ** Each entry in the aiCol[] array is a column that may be matched. ** ** This object is used by fts5_expr.c and fts5_index.c. */ struct Fts5Colset { int nCol; int aiCol[FLEXARRAY]; }; /* Size (int bytes) of a complete Fts5Colset object with N columns. */ #define SZ_FTS5COLSET(N) (sizeof(i64)*((N+2)/2)) /************************************************************************** ** Interface to code in fts5_config.c. fts5_config.c contains contains code ** to parse the arguments passed to the CREATE VIRTUAL TABLE statement. */ typedef struct Fts5Config Fts5Config; typedef struct Fts5TokenizerConfig Fts5TokenizerConfig; struct Fts5TokenizerConfig { Fts5Tokenizer *pTok; fts5_tokenizer_v2 *pApi2; fts5_tokenizer *pApi1; const char **azArg; int nArg; int ePattern; /* FTS_PATTERN_XXX constant */ const char *pLocale; /* Current locale to use */ int nLocale; /* Size of pLocale in bytes */ }; /* ** An instance of the following structure encodes all information that can ** be gleaned from the CREATE VIRTUAL TABLE statement. ** ** And all information loaded from the %_config table. ** ** nAutomerge: ** The minimum number of segments that an auto-merge operation should ** attempt to merge together. A value of 1 sets the object to use the ** compile time default. Zero disables auto-merge altogether. ** ** bContentlessDelete: ** True if the contentless_delete option was present in the CREATE ** VIRTUAL TABLE statement. ** ** zContent: ** ** zContentRowid: ** The value of the content_rowid= option, if one was specified. Or ** the string "rowid" otherwise. This text is not quoted - if it is ** used as part of an SQL statement it needs to be quoted appropriately. ** |
︙ | ︙ | |||
173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | ** ** bPrefixIndex: ** This is only used for debugging. If set to false, any prefix indexes ** are ignored. This value is configured using: ** ** INSERT INTO tbl(tbl, rank) VALUES('prefix-index', $bPrefixIndex); ** */ struct Fts5Config { sqlite3 *db; /* Database handle */ char *zDb; /* Database holding FTS index (e.g. "main") */ char *zName; /* Name of FTS index */ int nCol; /* Number of columns */ char **azCol; /* Column names */ u8 *abUnindexed; /* True for unindexed columns */ int nPrefix; /* Number of prefix indexes */ int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */ int eContent; /* An FTS5_CONTENT value */ char *zContent; /* content table */ char *zContentRowid; /* "content_rowid=" option value */ int bColumnsize; /* "columnsize=" option value (dflt==1) */ int eDetail; /* FTS5_DETAIL_XXX value */ char *zContentExprlist; | > > > > > > > | < | > > > > | > > | > | | | > | 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 | ** ** bPrefixIndex: ** This is only used for debugging. If set to false, any prefix indexes ** are ignored. This value is configured using: ** ** INSERT INTO tbl(tbl, rank) VALUES('prefix-index', $bPrefixIndex); ** ** bLocale: ** Set to true if locale=1 was specified when the table was created. */ struct Fts5Config { sqlite3 *db; /* Database handle */ Fts5Global *pGlobal; /* Global fts5 object for handle db */ char *zDb; /* Database holding FTS index (e.g. "main") */ char *zName; /* Name of FTS index */ int nCol; /* Number of columns */ char **azCol; /* Column names */ u8 *abUnindexed; /* True for unindexed columns */ int nPrefix; /* Number of prefix indexes */ int *aPrefix; /* Sizes in bytes of nPrefix prefix indexes */ int eContent; /* An FTS5_CONTENT value */ int bContentlessDelete; /* "contentless_delete=" option (dflt==0) */ int bContentlessUnindexed; /* "contentless_unindexed=" option (dflt=0) */ char *zContent; /* content table */ char *zContentRowid; /* "content_rowid=" option value */ int bColumnsize; /* "columnsize=" option value (dflt==1) */ int bTokendata; /* "tokendata=" option value (dflt==0) */ int bLocale; /* "locale=" option value (dflt==0) */ int eDetail; /* FTS5_DETAIL_XXX value */ char *zContentExprlist; Fts5TokenizerConfig t; int bLock; /* True when table is preparing statement */ /* Values loaded from the %_config table */ int iVersion; /* fts5 file format 'version' */ int iCookie; /* Incremented when %_config is modified */ int pgsz; /* Approximate page size used in %_data */ int nAutomerge; /* 'automerge' setting */ int nCrisisMerge; /* Maximum allowed segments per level */ int nUsermerge; /* 'usermerge' setting */ int nHashSize; /* Bytes of memory for in-memory hash */ char *zRank; /* Name of rank function */ char *zRankArgs; /* Arguments to rank function */ int bSecureDelete; /* 'secure-delete' */ int nDeleteMerge; /* 'deletemerge' */ int bPrefixInsttoken; /* 'prefix-insttoken' */ /* If non-NULL, points to sqlite3_vtab.base.zErrmsg. Often NULL. */ char **pzErrmsg; #ifdef SQLITE_DEBUG int bPrefixIndex; /* True to use prefix-indexes */ #endif }; /* Current expected value of %_config table 'version' field. And ** the expected version if the 'secure-delete' option has ever been ** set on the table. */ #define FTS5_CURRENT_VERSION 4 #define FTS5_CURRENT_VERSION_SECUREDELETE 5 #define FTS5_CONTENT_NORMAL 0 #define FTS5_CONTENT_NONE 1 #define FTS5_CONTENT_EXTERNAL 2 #define FTS5_CONTENT_UNINDEXED 3 #define FTS5_DETAIL_FULL 0 #define FTS5_DETAIL_NONE 1 #define FTS5_DETAIL_COLUMNS 2 #define FTS5_PATTERN_NONE 0 #define FTS5_PATTERN_LIKE 65 /* matches SQLITE_INDEX_CONSTRAINT_LIKE */ |
︙ | ︙ | |||
251 252 253 254 255 256 257 258 259 260 261 262 263 264 | /* Load the contents of the %_config table */ int sqlite3Fts5ConfigLoad(Fts5Config*, int); /* Set the value of a single config attribute */ int sqlite3Fts5ConfigSetValue(Fts5Config*, const char*, sqlite3_value*, int*); int sqlite3Fts5ConfigParseRank(const char*, char**, char**); /* ** End of interface to code in fts5_config.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_buffer.c. | > > | 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 | /* Load the contents of the %_config table */ int sqlite3Fts5ConfigLoad(Fts5Config*, int); /* Set the value of a single config attribute */ int sqlite3Fts5ConfigSetValue(Fts5Config*, const char*, sqlite3_value*, int*); int sqlite3Fts5ConfigParseRank(const char*, char**, char**); void sqlite3Fts5ConfigErrmsg(Fts5Config *pConfig, const char *zFmt, ...); /* ** End of interface to code in fts5_config.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_buffer.c. |
︙ | ︙ | |||
296 297 298 299 300 301 302 | sqlite3Fts5BufferSize((pRc),(pBuf),(nn)+(pBuf)->n) \ ) /* Write and decode big-endian 32-bit integer values */ void sqlite3Fts5Put32(u8*, int); int sqlite3Fts5Get32(const u8*); | | | 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 | sqlite3Fts5BufferSize((pRc),(pBuf),(nn)+(pBuf)->n) \ ) /* Write and decode big-endian 32-bit integer values */ void sqlite3Fts5Put32(u8*, int); int sqlite3Fts5Get32(const u8*); #define FTS5_POS2COLUMN(iPos) (int)((iPos >> 32) & 0x7FFFFFFF) #define FTS5_POS2OFFSET(iPos) (int)(iPos & 0x7FFFFFFF) typedef struct Fts5PoslistReader Fts5PoslistReader; struct Fts5PoslistReader { /* Variables used only by sqlite3Fts5PoslistIterXXX() functions. */ const u8 *a; /* Position list to iterate through */ int n; /* Size of buffer at a[] in bytes */ |
︙ | ︙ | |||
369 370 371 372 373 374 375 | }; #define sqlite3Fts5IterEof(x) ((x)->bEof) /* ** Values used as part of the flags argument passed to IndexQuery(). */ | | | | | | | > > > | 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 | }; #define sqlite3Fts5IterEof(x) ((x)->bEof) /* ** Values used as part of the flags argument passed to IndexQuery(). */ #define FTS5INDEX_QUERY_PREFIX 0x0001 /* Prefix query */ #define FTS5INDEX_QUERY_DESC 0x0002 /* Docs in descending rowid order */ #define FTS5INDEX_QUERY_TEST_NOIDX 0x0004 /* Do not use prefix index */ #define FTS5INDEX_QUERY_SCAN 0x0008 /* Scan query (fts5vocab) */ /* The following are used internally by the fts5_index.c module. They are ** defined here only to make it easier to avoid clashes with the flags ** above. */ #define FTS5INDEX_QUERY_SKIPEMPTY 0x0010 #define FTS5INDEX_QUERY_NOOUTPUT 0x0020 #define FTS5INDEX_QUERY_SKIPHASH 0x0040 #define FTS5INDEX_QUERY_NOTOKENDATA 0x0080 #define FTS5INDEX_QUERY_SCANONETERM 0x0100 /* ** Create/destroy an Fts5Index object. */ int sqlite3Fts5IndexOpen(Fts5Config *pConfig, int bCreate, Fts5Index**, char**); int sqlite3Fts5IndexClose(Fts5Index *p); |
︙ | ︙ | |||
447 448 449 450 451 452 453 454 455 456 457 458 459 460 | */ const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*); int sqlite3Fts5IterNextScan(Fts5IndexIter*); void *sqlite3Fts5StructureRef(Fts5Index*); void sqlite3Fts5StructureRelease(void*); int sqlite3Fts5StructureTest(Fts5Index*, void*); /* ** Insert or remove data to or from the index. Each time a document is ** added to or removed from the index, this function is called one or more ** times. ** ** For an insert, it must be called once for each token in the new document. | > > > > > > > > > > > | 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 | */ const char *sqlite3Fts5IterTerm(Fts5IndexIter*, int*); int sqlite3Fts5IterNextScan(Fts5IndexIter*); void *sqlite3Fts5StructureRef(Fts5Index*); void sqlite3Fts5StructureRelease(void*); int sqlite3Fts5StructureTest(Fts5Index*, void*); /* ** Used by xInstToken(): */ int sqlite3Fts5IterToken( Fts5IndexIter *pIndexIter, const char *pToken, int nToken, i64 iRowid, int iCol, int iOff, const char **ppOut, int *pnOut ); /* ** Insert or remove data to or from the index. Each time a document is ** added to or removed from the index, this function is called one or more ** times. ** ** For an insert, it must be called once for each token in the new document. |
︙ | ︙ | |||
521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 | int sqlite3Fts5IndexReinit(Fts5Index *p); int sqlite3Fts5IndexOptimize(Fts5Index *p); int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge); int sqlite3Fts5IndexReset(Fts5Index *p); int sqlite3Fts5IndexLoadConfig(Fts5Index *p); /* ** End of interface to code in fts5_index.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_varint.c. */ int sqlite3Fts5GetVarint32(const unsigned char *p, u32 *v); int sqlite3Fts5GetVarintLen(u32 iVal); u8 sqlite3Fts5GetVarint(const unsigned char*, u64*); int sqlite3Fts5PutVarint(unsigned char *p, u64 v); | > > > > > > > > > > | | 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 | int sqlite3Fts5IndexReinit(Fts5Index *p); int sqlite3Fts5IndexOptimize(Fts5Index *p); int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge); int sqlite3Fts5IndexReset(Fts5Index *p); int sqlite3Fts5IndexLoadConfig(Fts5Index *p); int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin); int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid); void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter*); /* Used to populate hash tables for xInstToken in detail=none/column mode. */ int sqlite3Fts5IndexIterWriteTokendata( Fts5IndexIter*, const char*, int, i64 iRowid, int iCol, int iOff ); /* ** End of interface to code in fts5_index.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_varint.c. */ int sqlite3Fts5GetVarint32(const unsigned char *p, u32 *v); int sqlite3Fts5GetVarintLen(u32 iVal); u8 sqlite3Fts5GetVarint(const unsigned char*, u64*); int sqlite3Fts5PutVarint(unsigned char *p, u64 v); #define fts5GetVarint32(a,b) sqlite3Fts5GetVarint32(a,(u32*)&(b)) #define fts5GetVarint sqlite3Fts5GetVarint #define fts5FastGetVarint32(a, iOff, nVal) { \ nVal = (a)[iOff++]; \ if( nVal & 0x80 ){ \ iOff--; \ iOff += fts5GetVarint32(&(a)[iOff], nVal); \ |
︙ | ︙ | |||
564 565 566 567 568 569 570 | typedef struct Fts5Table Fts5Table; struct Fts5Table { sqlite3_vtab base; /* Base class used by SQLite core */ Fts5Config *pConfig; /* Virtual table configuration */ Fts5Index *pIndex; /* Full-text index */ }; | | < < < < < < > > > > > > > > | 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 | typedef struct Fts5Table Fts5Table; struct Fts5Table { sqlite3_vtab base; /* Base class used by SQLite core */ Fts5Config *pConfig; /* Virtual table configuration */ Fts5Index *pIndex; /* Full-text index */ }; int sqlite3Fts5LoadTokenizer(Fts5Config *pConfig); Fts5Table *sqlite3Fts5TableFromCsrid(Fts5Global*, i64); int sqlite3Fts5FlushToDisk(Fts5Table*); void sqlite3Fts5ClearLocale(Fts5Config *pConfig); void sqlite3Fts5SetLocale(Fts5Config *pConfig, const char *pLoc, int nLoc); int sqlite3Fts5IsLocaleValue(Fts5Config *pConfig, sqlite3_value *pVal); int sqlite3Fts5DecodeLocaleValue(sqlite3_value *pVal, const char **ppText, int *pnText, const char **ppLoc, int *pnLoc ); /* ** End of interface to code in fts5.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_hash.c. |
︙ | ︙ | |||
605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 | ); /* ** Empty (but do not delete) a hash table. */ void sqlite3Fts5HashClear(Fts5Hash*); int sqlite3Fts5HashQuery( Fts5Hash*, /* Hash table to query */ int nPre, const char *pTerm, int nTerm, /* Query term */ void **ppObj, /* OUT: Pointer to doclist for pTerm */ int *pnDoclist /* OUT: Size of doclist in bytes */ ); int sqlite3Fts5HashScanInit( Fts5Hash*, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ); void sqlite3Fts5HashScanNext(Fts5Hash*); int sqlite3Fts5HashScanEof(Fts5Hash*); void sqlite3Fts5HashScanEntry(Fts5Hash *, const char **pzTerm, /* OUT: term (nul-terminated) */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ); /* ** End of interface to code in fts5_hash.c. **************************************************************************/ /************************************************************************** | > > > > > > > | 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 | ); /* ** Empty (but do not delete) a hash table. */ void sqlite3Fts5HashClear(Fts5Hash*); /* ** Return true if the hash is empty, false otherwise. */ int sqlite3Fts5HashIsEmpty(Fts5Hash*); int sqlite3Fts5HashQuery( Fts5Hash*, /* Hash table to query */ int nPre, const char *pTerm, int nTerm, /* Query term */ void **ppObj, /* OUT: Pointer to doclist for pTerm */ int *pnDoclist /* OUT: Size of doclist in bytes */ ); int sqlite3Fts5HashScanInit( Fts5Hash*, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ); void sqlite3Fts5HashScanNext(Fts5Hash*); int sqlite3Fts5HashScanEof(Fts5Hash*); void sqlite3Fts5HashScanEntry(Fts5Hash *, const char **pzTerm, /* OUT: term (nul-terminated) */ int *pnTerm, /* OUT: Size of term in bytes */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ); /* ** End of interface to code in fts5_hash.c. **************************************************************************/ /************************************************************************** |
︙ | ︙ | |||
648 649 650 651 652 653 654 | int sqlite3Fts5StorageOpen(Fts5Config*, Fts5Index*, int, Fts5Storage**, char**); int sqlite3Fts5StorageClose(Fts5Storage *p); int sqlite3Fts5StorageRename(Fts5Storage*, const char *zName); int sqlite3Fts5DropAll(Fts5Config*); int sqlite3Fts5CreateTable(Fts5Config*, const char*, const char*, int, char **); | | | | 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 | int sqlite3Fts5StorageOpen(Fts5Config*, Fts5Index*, int, Fts5Storage**, char**); int sqlite3Fts5StorageClose(Fts5Storage *p); int sqlite3Fts5StorageRename(Fts5Storage*, const char *zName); int sqlite3Fts5DropAll(Fts5Config*); int sqlite3Fts5CreateTable(Fts5Config*, const char*, const char*, int, char **); int sqlite3Fts5StorageDelete(Fts5Storage *p, i64, sqlite3_value**, int); int sqlite3Fts5StorageContentInsert(Fts5Storage *p, int, sqlite3_value**, i64*); int sqlite3Fts5StorageIndexInsert(Fts5Storage *p, sqlite3_value**, i64); int sqlite3Fts5StorageIntegrity(Fts5Storage *p, int iArg); int sqlite3Fts5StorageStmt(Fts5Storage *p, int eStmt, sqlite3_stmt**, char**); void sqlite3Fts5StorageStmtRelease(Fts5Storage *p, int eStmt, sqlite3_stmt*); |
︙ | ︙ | |||
674 675 676 677 678 679 680 681 682 683 684 685 686 687 | int sqlite3Fts5StorageDeleteAll(Fts5Storage *p); int sqlite3Fts5StorageRebuild(Fts5Storage *p); int sqlite3Fts5StorageOptimize(Fts5Storage *p); int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge); int sqlite3Fts5StorageReset(Fts5Storage *p); /* ** End of interface to code in fts5_storage.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_expr.c. | > > > | 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 | int sqlite3Fts5StorageDeleteAll(Fts5Storage *p); int sqlite3Fts5StorageRebuild(Fts5Storage *p); int sqlite3Fts5StorageOptimize(Fts5Storage *p); int sqlite3Fts5StorageMerge(Fts5Storage *p, int nMerge); int sqlite3Fts5StorageReset(Fts5Storage *p); void sqlite3Fts5StorageReleaseDeleteRow(Fts5Storage*); int sqlite3Fts5StorageFindDeleteRow(Fts5Storage *p, i64 iDel); /* ** End of interface to code in fts5_storage.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_expr.c. |
︙ | ︙ | |||
745 746 747 748 749 750 751 752 753 754 755 756 757 758 | Fts5Config*, Fts5Expr*, Fts5PoslistPopulator*, int, const char*, int ); void sqlite3Fts5ExprCheckPoslists(Fts5Expr*, i64); int sqlite3Fts5ExprClonePhrase(Fts5Expr*, int, Fts5Expr**); int sqlite3Fts5ExprPhraseCollist(Fts5Expr *, int, const u8 **, int *); /******************************************* ** The fts5_expr.c API above this point is used by the other hand-written ** C code in this module. The interfaces below this point are called by ** the parser code in fts5parse.y. */ void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...); | > > > > | 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 | Fts5Config*, Fts5Expr*, Fts5PoslistPopulator*, int, const char*, int ); void sqlite3Fts5ExprCheckPoslists(Fts5Expr*, i64); int sqlite3Fts5ExprClonePhrase(Fts5Expr*, int, Fts5Expr**); int sqlite3Fts5ExprPhraseCollist(Fts5Expr *, int, const u8 **, int *); int sqlite3Fts5ExprQueryToken(Fts5Expr*, int, int, const char**, int*); int sqlite3Fts5ExprInstToken(Fts5Expr*, i64, int, int, int, int, const char**, int*); void sqlite3Fts5ExprClearTokens(Fts5Expr*); /******************************************* ** The fts5_expr.c API above this point is used by the other hand-written ** C code in this module. The interfaces below this point are called by ** the parser code in fts5parse.y. */ void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...); |
︙ | ︙ | |||
822 823 824 825 826 827 828 829 830 831 832 833 834 835 | */ int sqlite3Fts5TokenizerInit(fts5_api*); int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ); /* ** End of interface to code in fts5_tokenizer.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_vocab.c. */ | > | 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 | */ int sqlite3Fts5TokenizerInit(fts5_api*); int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ); int sqlite3Fts5TokenizerPreload(Fts5TokenizerConfig*); /* ** End of interface to code in fts5_tokenizer.c. **************************************************************************/ /************************************************************************** ** Interface to code in fts5_vocab.c. */ |
︙ | ︙ |
Changes to ext/fts5/fts5_aux.c.
︙ | ︙ | |||
106 107 108 109 110 111 112 | /************************************************************************* ** Start of highlight() implementation. */ typedef struct HighlightContext HighlightContext; struct HighlightContext { | | < > > > > | > | 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 | /************************************************************************* ** Start of highlight() implementation. */ typedef struct HighlightContext HighlightContext; struct HighlightContext { /* Constant parameters to fts5HighlightCb() */ int iRangeStart; /* First token to include */ int iRangeEnd; /* If non-zero, last token to include */ const char *zOpen; /* Opening highlight */ const char *zClose; /* Closing highlight */ const char *zIn; /* Input text */ int nIn; /* Size of input text in bytes */ /* Variables modified by fts5HighlightCb() */ CInstIter iter; /* Coalesced Instance Iterator */ int iPos; /* Current token offset in zIn[] */ int iOff; /* Have copied up to this offset in zIn[] */ int bOpen; /* True if highlight is open */ char *zOut; /* Output value */ }; /* ** Append text to the HighlightContext output string - p->zOut. Argument ** z points to a buffer containing n bytes of text to append. If n is ** negative, everything up until the first '\0' is appended to the output. |
︙ | ︙ | |||
147 148 149 150 151 152 153 | ** Tokenizer callback used by implementation of highlight() function. */ static int fts5HighlightCb( void *pContext, /* Pointer to HighlightContext object */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Buffer containing token */ int nToken, /* Size of token in bytes */ | | | | > > > > > > > > > > > > > > > > > | > > | > < > | > > | | < > > > > > | 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 | ** Tokenizer callback used by implementation of highlight() function. */ static int fts5HighlightCb( void *pContext, /* Pointer to HighlightContext object */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Buffer containing token */ int nToken, /* Size of token in bytes */ int iStartOff, /* Start byte offset of token */ int iEndOff /* End byte offset of token */ ){ HighlightContext *p = (HighlightContext*)pContext; int rc = SQLITE_OK; int iPos; UNUSED_PARAM2(pToken, nToken); if( tflags & FTS5_TOKEN_COLOCATED ) return SQLITE_OK; iPos = p->iPos++; if( p->iRangeEnd>=0 ){ if( iPos<p->iRangeStart || iPos>p->iRangeEnd ) return SQLITE_OK; if( p->iRangeStart && iPos==p->iRangeStart ) p->iOff = iStartOff; } /* If the parenthesis is open, and this token is not part of the current ** phrase, and the starting byte offset of this token is past the point ** that has currently been copied into the output buffer, close the ** parenthesis. */ if( p->bOpen && (iPos<=p->iter.iStart || p->iter.iStart<0) && iStartOff>p->iOff ){ fts5HighlightAppend(&rc, p, p->zClose, -1); p->bOpen = 0; } /* If this is the start of a new phrase, and the highlight is not open: ** ** * copy text from the input up to the start of the phrase, and ** * open the highlight. */ if( iPos==p->iter.iStart && p->bOpen==0 ){ fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iStartOff - p->iOff); fts5HighlightAppend(&rc, p, p->zOpen, -1); p->iOff = iStartOff; p->bOpen = 1; } if( iPos==p->iter.iEnd ){ if( p->bOpen==0 ){ assert( p->iRangeEnd>=0 ); fts5HighlightAppend(&rc, p, p->zOpen, -1); p->bOpen = 1; } fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff); p->iOff = iEndOff; if( rc==SQLITE_OK ){ rc = fts5CInstIterNext(&p->iter); } } if( iPos==p->iRangeEnd ){ if( p->bOpen ){ if( p->iter.iStart>=0 && iPos>=p->iter.iStart ){ fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff); p->iOff = iEndOff; } fts5HighlightAppend(&rc, p, p->zClose, -1); p->bOpen = 0; } fts5HighlightAppend(&rc, p, &p->zIn[p->iOff], iEndOff - p->iOff); p->iOff = iEndOff; } return rc; } /* ** Implementation of highlight() function. */ static void fts5HighlightFunction( const Fts5ExtensionApi *pApi, /* API offered by current FTS version */ Fts5Context *pFts, /* First arg to pass to pApi functions */ |
︙ | ︙ | |||
217 218 219 220 221 222 223 224 | return; } iCol = sqlite3_value_int(apVal[0]); memset(&ctx, 0, sizeof(HighlightContext)); ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]); ctx.zClose = (const char*)sqlite3_value_text(apVal[2]); rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn); | > | > > | > > > > > | > > > > > | 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 | return; } iCol = sqlite3_value_int(apVal[0]); memset(&ctx, 0, sizeof(HighlightContext)); ctx.zOpen = (const char*)sqlite3_value_text(apVal[1]); ctx.zClose = (const char*)sqlite3_value_text(apVal[2]); ctx.iRangeEnd = -1; rc = pApi->xColumnText(pFts, iCol, &ctx.zIn, &ctx.nIn); if( rc==SQLITE_RANGE ){ sqlite3_result_text(pCtx, "", -1, SQLITE_STATIC); rc = SQLITE_OK; }else if( ctx.zIn ){ const char *pLoc = 0; /* Locale of column iCol */ int nLoc = 0; /* Size of pLoc in bytes */ if( rc==SQLITE_OK ){ rc = fts5CInstIterInit(pApi, pFts, iCol, &ctx.iter); } if( rc==SQLITE_OK ){ rc = pApi->xColumnLocale(pFts, iCol, &pLoc, &nLoc); } if( rc==SQLITE_OK ){ rc = pApi->xTokenize_v2( pFts, ctx.zIn, ctx.nIn, pLoc, nLoc, (void*)&ctx, fts5HighlightCb ); } if( ctx.bOpen ){ fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1); } fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff); if( rc==SQLITE_OK ){ sqlite3_result_text(pCtx, (const char*)ctx.zOut, -1, SQLITE_TRANSIENT); } sqlite3_free(ctx.zOut); |
︙ | ︙ | |||
402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 | } nCol = pApi->xColumnCount(pFts); memset(&ctx, 0, sizeof(HighlightContext)); iCol = sqlite3_value_int(apVal[0]); ctx.zOpen = fts5ValueToText(apVal[1]); ctx.zClose = fts5ValueToText(apVal[2]); zEllips = fts5ValueToText(apVal[3]); nToken = sqlite3_value_int(apVal[4]); iBestCol = (iCol>=0 ? iCol : 0); nPhrase = pApi->xPhraseCount(pFts); aSeen = sqlite3_malloc(nPhrase); if( aSeen==0 ){ rc = SQLITE_NOMEM; } if( rc==SQLITE_OK ){ rc = pApi->xInstCount(pFts, &nInst); } memset(&sFinder, 0, sizeof(Fts5SFinder)); for(i=0; i<nCol; i++){ if( iCol<0 || iCol==i ){ int nDoc; int nDocsize; int ii; sFinder.iPos = 0; sFinder.nFirst = 0; rc = pApi->xColumnText(pFts, i, &sFinder.zDoc, &nDoc); if( rc!=SQLITE_OK ) break; | > > > > > | | | 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 | } nCol = pApi->xColumnCount(pFts); memset(&ctx, 0, sizeof(HighlightContext)); iCol = sqlite3_value_int(apVal[0]); ctx.zOpen = fts5ValueToText(apVal[1]); ctx.zClose = fts5ValueToText(apVal[2]); ctx.iRangeEnd = -1; zEllips = fts5ValueToText(apVal[3]); nToken = sqlite3_value_int(apVal[4]); iBestCol = (iCol>=0 ? iCol : 0); nPhrase = pApi->xPhraseCount(pFts); aSeen = sqlite3_malloc(nPhrase); if( aSeen==0 ){ rc = SQLITE_NOMEM; } if( rc==SQLITE_OK ){ rc = pApi->xInstCount(pFts, &nInst); } memset(&sFinder, 0, sizeof(Fts5SFinder)); for(i=0; i<nCol; i++){ if( iCol<0 || iCol==i ){ const char *pLoc = 0; /* Locale of column iCol */ int nLoc = 0; /* Size of pLoc in bytes */ int nDoc; int nDocsize; int ii; sFinder.iPos = 0; sFinder.nFirst = 0; rc = pApi->xColumnText(pFts, i, &sFinder.zDoc, &nDoc); if( rc!=SQLITE_OK ) break; rc = pApi->xColumnLocale(pFts, i, &pLoc, &nLoc); if( rc!=SQLITE_OK ) break; rc = pApi->xTokenize_v2(pFts, sFinder.zDoc, nDoc, pLoc, nLoc, (void*)&sFinder, fts5SentenceFinderCb ); if( rc!=SQLITE_OK ) break; rc = pApi->xColumnSize(pFts, i, &nDocsize); if( rc!=SQLITE_OK ) break; for(ii=0; rc==SQLITE_OK && ii<nInst; ii++){ int ip, ic, io; |
︙ | ︙ | |||
484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 | if( rc==SQLITE_OK ){ rc = pApi->xColumnText(pFts, iBestCol, &ctx.zIn, &ctx.nIn); } if( rc==SQLITE_OK && nColSize==0 ){ rc = pApi->xColumnSize(pFts, iBestCol, &nColSize); } if( ctx.zIn ){ if( rc==SQLITE_OK ){ rc = fts5CInstIterInit(pApi, pFts, iBestCol, &ctx.iter); } ctx.iRangeStart = iBestStart; ctx.iRangeEnd = iBestStart + nToken - 1; if( iBestStart>0 ){ fts5HighlightAppend(&rc, &ctx, zEllips, -1); } /* Advance iterator ctx.iter so that it points to the first coalesced ** phrase instance at or following position iBestStart. */ while( ctx.iter.iStart>=0 && ctx.iter.iStart<iBestStart && rc==SQLITE_OK ){ rc = fts5CInstIterNext(&ctx.iter); } if( rc==SQLITE_OK ){ | > > > > > > | > > > > > | 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 | if( rc==SQLITE_OK ){ rc = pApi->xColumnText(pFts, iBestCol, &ctx.zIn, &ctx.nIn); } if( rc==SQLITE_OK && nColSize==0 ){ rc = pApi->xColumnSize(pFts, iBestCol, &nColSize); } if( ctx.zIn ){ const char *pLoc = 0; /* Locale of column iBestCol */ int nLoc = 0; /* Bytes in pLoc */ if( rc==SQLITE_OK ){ rc = fts5CInstIterInit(pApi, pFts, iBestCol, &ctx.iter); } ctx.iRangeStart = iBestStart; ctx.iRangeEnd = iBestStart + nToken - 1; if( iBestStart>0 ){ fts5HighlightAppend(&rc, &ctx, zEllips, -1); } /* Advance iterator ctx.iter so that it points to the first coalesced ** phrase instance at or following position iBestStart. */ while( ctx.iter.iStart>=0 && ctx.iter.iStart<iBestStart && rc==SQLITE_OK ){ rc = fts5CInstIterNext(&ctx.iter); } if( rc==SQLITE_OK ){ rc = pApi->xColumnLocale(pFts, iBestCol, &pLoc, &nLoc); } if( rc==SQLITE_OK ){ rc = pApi->xTokenize_v2( pFts, ctx.zIn, ctx.nIn, pLoc, nLoc, (void*)&ctx,fts5HighlightCb ); } if( ctx.bOpen ){ fts5HighlightAppend(&rc, &ctx, ctx.zClose, -1); } if( ctx.iRangeEnd>=(nColSize-1) ){ fts5HighlightAppend(&rc, &ctx, &ctx.zIn[ctx.iOff], ctx.nIn - ctx.iOff); }else{ fts5HighlightAppend(&rc, &ctx, zEllips, -1); } } |
︙ | ︙ | |||
604 605 606 607 608 609 610 | ** IDF = log( (N - nHit + 0.5) / (nHit + 0.5) ) ** ** where "N" is the total number of documents in the set and nHit ** is the number that contain at least one instance of the phrase ** under consideration. ** ** The problem with this is that if (N < 2*nHit), the IDF is | | | 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 | ** IDF = log( (N - nHit + 0.5) / (nHit + 0.5) ) ** ** where "N" is the total number of documents in the set and nHit ** is the number that contain at least one instance of the phrase ** under consideration. ** ** The problem with this is that if (N < 2*nHit), the IDF is ** negative. Which is undesirable. So the minimum allowable IDF is ** (1e-6) - roughly the same as a term that appears in just over ** half of set of 5,000,000 documents. */ double idf = log( (nRow - nHit + 0.5) / (nHit + 0.5) ); if( idf<=0.0 ) idf = 1e-6; p->aIDF[i] = idf; } } |
︙ | ︙ | |||
682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 | ); } sqlite3_result_double(pCtx, -1.0 * score); }else{ sqlite3_result_error_code(pCtx, rc); } } int sqlite3Fts5AuxInit(fts5_api *pApi){ struct Builtin { const char *zFunc; /* Function name (nul-terminated) */ void *pUserData; /* User-data pointer */ fts5_extension_function xFunc;/* Callback function */ void (*xDestroy)(void*); /* Destructor function */ } aBuiltin [] = { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | > | 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 | ); } sqlite3_result_double(pCtx, -1.0 * score); }else{ sqlite3_result_error_code(pCtx, rc); } } /* ** Implementation of fts5_get_locale() function. */ static void fts5GetLocaleFunction( const Fts5ExtensionApi *pApi, /* API offered by current FTS version */ Fts5Context *pFts, /* First arg to pass to pApi functions */ sqlite3_context *pCtx, /* Context for returning result/error */ int nVal, /* Number of values in apVal[] array */ sqlite3_value **apVal /* Array of trailing arguments */ ){ int iCol = 0; int eType = 0; int rc = SQLITE_OK; const char *zLocale = 0; int nLocale = 0; /* xColumnLocale() must be available */ assert( pApi->iVersion>=4 ); if( nVal!=1 ){ const char *z = "wrong number of arguments to function fts5_get_locale()"; sqlite3_result_error(pCtx, z, -1); return; } eType = sqlite3_value_numeric_type(apVal[0]); if( eType!=SQLITE_INTEGER ){ const char *z = "non-integer argument passed to function fts5_get_locale()"; sqlite3_result_error(pCtx, z, -1); return; } iCol = sqlite3_value_int(apVal[0]); if( iCol<0 || iCol>=pApi->xColumnCount(pFts) ){ sqlite3_result_error_code(pCtx, SQLITE_RANGE); return; } rc = pApi->xColumnLocale(pFts, iCol, &zLocale, &nLocale); if( rc!=SQLITE_OK ){ sqlite3_result_error_code(pCtx, rc); return; } sqlite3_result_text(pCtx, zLocale, nLocale, SQLITE_TRANSIENT); } int sqlite3Fts5AuxInit(fts5_api *pApi){ struct Builtin { const char *zFunc; /* Function name (nul-terminated) */ void *pUserData; /* User-data pointer */ fts5_extension_function xFunc;/* Callback function */ void (*xDestroy)(void*); /* Destructor function */ } aBuiltin [] = { { "snippet", 0, fts5SnippetFunction, 0 }, { "highlight", 0, fts5HighlightFunction, 0 }, { "bm25", 0, fts5Bm25Function, 0 }, { "fts5_get_locale", 0, fts5GetLocaleFunction, 0 }, }; int rc = SQLITE_OK; /* Return code */ int i; /* To iterate through builtin functions */ for(i=0; rc==SQLITE_OK && i<ArraySize(aBuiltin); i++){ rc = pApi->xCreateFunction(pApi, aBuiltin[i].zFunc, |
︙ | ︙ |
Changes to ext/fts5/fts5_buffer.c.
︙ | ︙ | |||
64 65 66 67 68 69 70 71 72 73 74 75 76 77 | int *pRc, Fts5Buffer *pBuf, u32 nData, const u8 *pData ){ if( nData ){ if( fts5BufferGrow(pRc, pBuf, nData) ) return; memcpy(&pBuf->p[pBuf->n], pData, nData); pBuf->n += nData; } } /* ** Append the nul-terminated string zStr to the buffer pBuf. This function | > | 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | int *pRc, Fts5Buffer *pBuf, u32 nData, const u8 *pData ){ if( nData ){ if( fts5BufferGrow(pRc, pBuf, nData) ) return; assert( pBuf->p!=0 ); memcpy(&pBuf->p[pBuf->n], pData, nData); pBuf->n += nData; } } /* ** Append the nul-terminated string zStr to the buffer pBuf. This function |
︙ | ︙ | |||
165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | int sqlite3Fts5PoslistNext64( const u8 *a, int n, /* Buffer containing poslist */ int *pi, /* IN/OUT: Offset within a[] */ i64 *piOff /* IN/OUT: Current offset */ ){ int i = *pi; if( i>=n ){ /* EOF */ *piOff = -1; return 1; }else{ i64 iOff = *piOff; u32 iVal; fts5FastGetVarint32(a, i, iVal); if( iVal<=1 ){ if( iVal==0 ){ *pi = i; return 0; } fts5FastGetVarint32(a, i, iVal); | > > | 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 | int sqlite3Fts5PoslistNext64( const u8 *a, int n, /* Buffer containing poslist */ int *pi, /* IN/OUT: Offset within a[] */ i64 *piOff /* IN/OUT: Current offset */ ){ int i = *pi; assert( a!=0 || i==0 ); if( i>=n ){ /* EOF */ *piOff = -1; return 1; }else{ i64 iOff = *piOff; u32 iVal; assert( a!=0 ); fts5FastGetVarint32(a, i, iVal); if( iVal<=1 ){ if( iVal==0 ){ *pi = i; return 0; } fts5FastGetVarint32(a, i, iVal); |
︙ | ︙ | |||
301 302 303 304 305 306 307 | ** Return true if character 't' may be part of an FTS5 bareword, or false ** otherwise. Characters that may be part of barewords: ** ** * All non-ASCII characters, ** * The 52 upper and lower case ASCII characters, and ** * The 10 integer ASCII characters. ** * The underscore character "_" (0x5F). | | | 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 | ** Return true if character 't' may be part of an FTS5 bareword, or false ** otherwise. Characters that may be part of barewords: ** ** * All non-ASCII characters, ** * The 52 upper and lower case ASCII characters, and ** * The 10 integer ASCII characters. ** * The underscore character "_" (0x5F). ** * The unicode "substitute" character (0x1A). */ int sqlite3Fts5IsBareword(char t){ u8 aBareword[128] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 0x00 .. 0x0F */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 1, 0, 0, 0, 0, 0, /* 0x10 .. 0x1F */ 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 0x20 .. 0x2F */ 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 0, 0, 0, 0, 0, 0, /* 0x30 .. 0x3F */ |
︙ | ︙ |
Changes to ext/fts5/fts5_config.c.
︙ | ︙ | |||
18 19 20 21 22 23 24 25 26 27 28 29 30 31 | #define FTS5_DEFAULT_PAGE_SIZE 4050 #define FTS5_DEFAULT_AUTOMERGE 4 #define FTS5_DEFAULT_USERMERGE 4 #define FTS5_DEFAULT_CRISISMERGE 16 #define FTS5_DEFAULT_HASHSIZE (1024*1024) /* Maximum allowed page size */ #define FTS5_MAX_PAGE_SIZE (64*1024) static int fts5_iswhitespace(char x){ return (x==' '); } | > > | 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 | #define FTS5_DEFAULT_PAGE_SIZE 4050 #define FTS5_DEFAULT_AUTOMERGE 4 #define FTS5_DEFAULT_USERMERGE 4 #define FTS5_DEFAULT_CRISISMERGE 16 #define FTS5_DEFAULT_HASHSIZE (1024*1024) #define FTS5_DEFAULT_DELETE_AUTOMERGE 10 /* default 10% */ /* Maximum allowed page size */ #define FTS5_MAX_PAGE_SIZE (64*1024) static int fts5_iswhitespace(char x){ return (x==' '); } |
︙ | ︙ | |||
228 229 230 231 232 233 234 | ** ** If successful, object pConfig is updated and SQLITE_OK returned. If ** an error occurs, an SQLite error code is returned and an error message ** may be left in *pzErr. It is the responsibility of the caller to ** eventually free any such error message using sqlite3_free(). */ static int fts5ConfigParseSpecial( | < > | 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | ** ** If successful, object pConfig is updated and SQLITE_OK returned. If ** an error occurs, an SQLite error code is returned and an error message ** may be left in *pzErr. It is the responsibility of the caller to ** eventually free any such error message using sqlite3_free(). */ static int fts5ConfigParseSpecial( Fts5Config *pConfig, /* Configuration object to update */ const char *zCmd, /* Special command to parse */ const char *zArg, /* Argument to parse */ char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; int nCmd = (int)strlen(zCmd); if( sqlite3_strnicmp("prefix", zCmd, nCmd)==0 ){ const int nByte = sizeof(int) * FTS5_MAX_PREFIX_INDEXES; const char *p; int bFirst = 1; if( pConfig->aPrefix==0 ){ pConfig->aPrefix = sqlite3Fts5MallocZero(&rc, nByte); if( rc ) return rc; |
︙ | ︙ | |||
292 293 294 295 296 297 298 | assert( pConfig->nPrefix<=FTS5_MAX_PREFIX_INDEXES ); return rc; } if( sqlite3_strnicmp("tokenize", zCmd, nCmd)==0 ){ const char *p = (const char*)zArg; sqlite3_int64 nArg = strlen(zArg) + 1; | | < < | > | | 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 | assert( pConfig->nPrefix<=FTS5_MAX_PREFIX_INDEXES ); return rc; } if( sqlite3_strnicmp("tokenize", zCmd, nCmd)==0 ){ const char *p = (const char*)zArg; sqlite3_int64 nArg = strlen(zArg) + 1; char **azArg = sqlite3Fts5MallocZero(&rc, (sizeof(char*) + 2) * nArg); if( azArg ){ char *pSpace = (char*)&azArg[nArg]; if( pConfig->t.azArg ){ *pzErr = sqlite3_mprintf("multiple tokenize=... directives"); rc = SQLITE_ERROR; }else{ for(nArg=0; p && *p; nArg++){ const char *p2 = fts5ConfigSkipWhitespace(p); if( *p2=='\'' ){ p = fts5ConfigSkipLiteral(p2); |
︙ | ︙ | |||
320 321 322 323 324 325 326 | p = fts5ConfigSkipWhitespace(p); } } if( p==0 ){ *pzErr = sqlite3_mprintf("parse error in tokenize directive"); rc = SQLITE_ERROR; }else{ | < | > | < < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < < < < < < < < < < | 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 | p = fts5ConfigSkipWhitespace(p); } } if( p==0 ){ *pzErr = sqlite3_mprintf("parse error in tokenize directive"); rc = SQLITE_ERROR; }else{ pConfig->t.azArg = (const char**)azArg; pConfig->t.nArg = nArg; azArg = 0; } } } sqlite3_free(azArg); return rc; } if( sqlite3_strnicmp("content", zCmd, nCmd)==0 ){ if( pConfig->eContent!=FTS5_CONTENT_NORMAL ){ *pzErr = sqlite3_mprintf("multiple content=... directives"); rc = SQLITE_ERROR; }else{ if( zArg[0] ){ pConfig->eContent = FTS5_CONTENT_EXTERNAL; pConfig->zContent = sqlite3Fts5Mprintf(&rc, "%Q.%Q", pConfig->zDb,zArg); }else{ pConfig->eContent = FTS5_CONTENT_NONE; } } return rc; } if( sqlite3_strnicmp("contentless_delete", zCmd, nCmd)==0 ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){ *pzErr = sqlite3_mprintf("malformed contentless_delete=... directive"); rc = SQLITE_ERROR; }else{ pConfig->bContentlessDelete = (zArg[0]=='1'); } return rc; } if( sqlite3_strnicmp("contentless_unindexed", zCmd, nCmd)==0 ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){ *pzErr = sqlite3_mprintf("malformed contentless_delete=... directive"); rc = SQLITE_ERROR; }else{ pConfig->bContentlessUnindexed = (zArg[0]=='1'); } return rc; } if( sqlite3_strnicmp("content_rowid", zCmd, nCmd)==0 ){ if( pConfig->zContentRowid ){ *pzErr = sqlite3_mprintf("multiple content_rowid=... directives"); rc = SQLITE_ERROR; }else{ pConfig->zContentRowid = sqlite3Fts5Strndup(&rc, zArg, -1); } return rc; } if( sqlite3_strnicmp("columnsize", zCmd, nCmd)==0 ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){ *pzErr = sqlite3_mprintf("malformed columnsize=... directive"); rc = SQLITE_ERROR; }else{ pConfig->bColumnsize = (zArg[0]=='1'); } return rc; } if( sqlite3_strnicmp("locale", zCmd, nCmd)==0 ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){ *pzErr = sqlite3_mprintf("malformed locale=... directive"); rc = SQLITE_ERROR; }else{ pConfig->bLocale = (zArg[0]=='1'); } return rc; } if( sqlite3_strnicmp("detail", zCmd, nCmd)==0 ){ const Fts5Enum aDetail[] = { { "none", FTS5_DETAIL_NONE }, { "full", FTS5_DETAIL_FULL }, { "columns", FTS5_DETAIL_COLUMNS }, { 0, 0 } }; if( (rc = fts5ConfigSetEnum(aDetail, zArg, &pConfig->eDetail)) ){ *pzErr = sqlite3_mprintf("malformed detail=... directive"); } return rc; } if( sqlite3_strnicmp("tokendata", zCmd, nCmd)==0 ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1]!='\0' ){ *pzErr = sqlite3_mprintf("malformed tokendata=... directive"); rc = SQLITE_ERROR; }else{ pConfig->bTokendata = (zArg[0]=='1'); } return rc; } *pzErr = sqlite3_mprintf("unrecognized option: \"%.*s\"", nCmd, zCmd); return SQLITE_ERROR; } /* ** Gobble up the first bareword or quoted word from the input buffer zIn. ** Return a pointer to the character immediately following the last in ** the gobbled word if successful, or a NULL pointer otherwise (failed ** to find close-quote character). ** ** Before returning, set pzOut to point to a new buffer containing a |
︙ | ︙ | |||
455 456 457 458 459 460 461 | return zRet; } static int fts5ConfigParseColumn( Fts5Config *p, char *zCol, char *zArg, | | > > > > > > > > | > > > > > > > > > | 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 | return zRet; } static int fts5ConfigParseColumn( Fts5Config *p, char *zCol, char *zArg, char **pzErr, int *pbUnindexed ){ int rc = SQLITE_OK; if( 0==sqlite3_stricmp(zCol, FTS5_RANK_NAME) || 0==sqlite3_stricmp(zCol, FTS5_ROWID_NAME) ){ *pzErr = sqlite3_mprintf("reserved fts5 column name: %s", zCol); rc = SQLITE_ERROR; }else if( zArg ){ if( 0==sqlite3_stricmp(zArg, "unindexed") ){ p->abUnindexed[p->nCol] = 1; *pbUnindexed = 1; }else{ *pzErr = sqlite3_mprintf("unrecognized column option: %s", zArg); rc = SQLITE_ERROR; } } p->azCol[p->nCol++] = zCol; return rc; } /* ** Populate the Fts5Config.zContentExprlist string. */ static int fts5ConfigMakeExprlist(Fts5Config *p){ int i; int rc = SQLITE_OK; Fts5Buffer buf = {0, 0, 0}; sqlite3Fts5BufferAppendPrintf(&rc, &buf, "T.%Q", p->zContentRowid); if( p->eContent!=FTS5_CONTENT_NONE ){ assert( p->eContent==FTS5_CONTENT_EXTERNAL || p->eContent==FTS5_CONTENT_NORMAL || p->eContent==FTS5_CONTENT_UNINDEXED ); for(i=0; i<p->nCol; i++){ if( p->eContent==FTS5_CONTENT_EXTERNAL ){ sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.%Q", p->azCol[i]); }else if( p->eContent==FTS5_CONTENT_NORMAL || p->abUnindexed[i] ){ sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.c%d", i); }else{ sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", NULL"); } } } if( p->eContent==FTS5_CONTENT_NORMAL && p->bLocale ){ for(i=0; i<p->nCol; i++){ if( p->abUnindexed[i]==0 ){ sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", T.l%d", i); }else{ sqlite3Fts5BufferAppendPrintf(&rc, &buf, ", NULL"); } } } assert( p->zContentExprlist==0 ); p->zContentExprlist = (char*)buf.p; return rc; |
︙ | ︙ | |||
524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 | Fts5Config **ppOut, /* OUT: Results of parse */ char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; /* Return code */ Fts5Config *pRet; /* New object to return */ int i; sqlite3_int64 nByte; *ppOut = pRet = (Fts5Config*)sqlite3_malloc(sizeof(Fts5Config)); if( pRet==0 ) return SQLITE_NOMEM; memset(pRet, 0, sizeof(Fts5Config)); pRet->db = db; pRet->iCookie = -1; nByte = nArg * (sizeof(char*) + sizeof(u8)); pRet->azCol = (char**)sqlite3Fts5MallocZero(&rc, nByte); pRet->abUnindexed = pRet->azCol ? (u8*)&pRet->azCol[nArg] : 0; pRet->zDb = sqlite3Fts5Strndup(&rc, azArg[1], -1); pRet->zName = sqlite3Fts5Strndup(&rc, azArg[2], -1); pRet->bColumnsize = 1; pRet->eDetail = FTS5_DETAIL_FULL; #ifdef SQLITE_DEBUG pRet->bPrefixIndex = 1; #endif if( rc==SQLITE_OK && sqlite3_stricmp(pRet->zName, FTS5_RANK_NAME)==0 ){ *pzErr = sqlite3_mprintf("reserved fts5 table name: %s", pRet->zName); rc = SQLITE_ERROR; } for(i=3; rc==SQLITE_OK && i<nArg; i++){ const char *zOrig = azArg[i]; const char *z; char *zOne = 0; char *zTwo = 0; int bOption = 0; int bMustBeCol = 0; | > > > | 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 | Fts5Config **ppOut, /* OUT: Results of parse */ char **pzErr /* OUT: Error message */ ){ int rc = SQLITE_OK; /* Return code */ Fts5Config *pRet; /* New object to return */ int i; sqlite3_int64 nByte; int bUnindexed = 0; /* True if there are one or more UNINDEXED */ *ppOut = pRet = (Fts5Config*)sqlite3_malloc(sizeof(Fts5Config)); if( pRet==0 ) return SQLITE_NOMEM; memset(pRet, 0, sizeof(Fts5Config)); pRet->pGlobal = pGlobal; pRet->db = db; pRet->iCookie = -1; nByte = nArg * (sizeof(char*) + sizeof(u8)); pRet->azCol = (char**)sqlite3Fts5MallocZero(&rc, nByte); pRet->abUnindexed = pRet->azCol ? (u8*)&pRet->azCol[nArg] : 0; pRet->zDb = sqlite3Fts5Strndup(&rc, azArg[1], -1); pRet->zName = sqlite3Fts5Strndup(&rc, azArg[2], -1); pRet->bColumnsize = 1; pRet->eDetail = FTS5_DETAIL_FULL; #ifdef SQLITE_DEBUG pRet->bPrefixIndex = 1; #endif if( rc==SQLITE_OK && sqlite3_stricmp(pRet->zName, FTS5_RANK_NAME)==0 ){ *pzErr = sqlite3_mprintf("reserved fts5 table name: %s", pRet->zName); rc = SQLITE_ERROR; } assert( (pRet->abUnindexed && pRet->azCol) || rc!=SQLITE_OK ); for(i=3; rc==SQLITE_OK && i<nArg; i++){ const char *zOrig = azArg[i]; const char *z; char *zOne = 0; char *zTwo = 0; int bOption = 0; int bMustBeCol = 0; |
︙ | ︙ | |||
575 576 577 578 579 580 581 | if( rc==SQLITE_OK ){ if( z==0 ){ *pzErr = sqlite3_mprintf("parse error in \"%s\"", zOrig); rc = SQLITE_ERROR; }else{ if( bOption ){ | | | > > > > > > > > > > | > > > > > > > > > > | > | > | > > > > > > | | | > > > | 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 | if( rc==SQLITE_OK ){ if( z==0 ){ *pzErr = sqlite3_mprintf("parse error in \"%s\"", zOrig); rc = SQLITE_ERROR; }else{ if( bOption ){ rc = fts5ConfigParseSpecial(pRet, ALWAYS(zOne)?zOne:"", zTwo?zTwo:"", pzErr ); }else{ rc = fts5ConfigParseColumn(pRet, zOne, zTwo, pzErr, &bUnindexed); zOne = 0; } } } sqlite3_free(zOne); sqlite3_free(zTwo); } /* We only allow contentless_delete=1 if the table is indeed contentless. */ if( rc==SQLITE_OK && pRet->bContentlessDelete && pRet->eContent!=FTS5_CONTENT_NONE ){ *pzErr = sqlite3_mprintf( "contentless_delete=1 requires a contentless table" ); rc = SQLITE_ERROR; } /* We only allow contentless_delete=1 if columnsize=0 is not present. ** ** This restriction may be removed at some point. */ if( rc==SQLITE_OK && pRet->bContentlessDelete && pRet->bColumnsize==0 ){ *pzErr = sqlite3_mprintf( "contentless_delete=1 is incompatible with columnsize=0" ); rc = SQLITE_ERROR; } /* We only allow contentless_unindexed=1 if the table is actually a ** contentless one. */ if( rc==SQLITE_OK && pRet->bContentlessUnindexed && pRet->eContent!=FTS5_CONTENT_NONE ){ *pzErr = sqlite3_mprintf( "contentless_unindexed=1 requires a contentless table" ); rc = SQLITE_ERROR; } /* If no zContent option was specified, fill in the default values. */ if( rc==SQLITE_OK && pRet->zContent==0 ){ const char *zTail = 0; assert( pRet->eContent==FTS5_CONTENT_NORMAL || pRet->eContent==FTS5_CONTENT_NONE ); if( pRet->eContent==FTS5_CONTENT_NORMAL ){ zTail = "content"; }else if( bUnindexed && pRet->bContentlessUnindexed ){ pRet->eContent = FTS5_CONTENT_UNINDEXED; zTail = "content"; }else if( pRet->bColumnsize ){ zTail = "docsize"; } if( zTail ){ pRet->zContent = sqlite3Fts5Mprintf( |
︙ | ︙ | |||
639 640 641 642 643 644 645 | /* ** Free the configuration object passed as the only argument. */ void sqlite3Fts5ConfigFree(Fts5Config *pConfig){ if( pConfig ){ int i; | | > | > > | > > | 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 | /* ** Free the configuration object passed as the only argument. */ void sqlite3Fts5ConfigFree(Fts5Config *pConfig){ if( pConfig ){ int i; if( pConfig->t.pTok ){ if( pConfig->t.pApi1 ){ pConfig->t.pApi1->xDelete(pConfig->t.pTok); }else{ pConfig->t.pApi2->xDelete(pConfig->t.pTok); } } sqlite3_free((char*)pConfig->t.azArg); sqlite3_free(pConfig->zDb); sqlite3_free(pConfig->zName); for(i=0; i<pConfig->nCol; i++){ sqlite3_free(pConfig->azCol[i]); } sqlite3_free(pConfig->azCol); sqlite3_free(pConfig->aPrefix); |
︙ | ︙ | |||
716 717 718 719 720 721 722 | int sqlite3Fts5Tokenize( Fts5Config *pConfig, /* FTS5 Configuration object */ int flags, /* FTS5_TOKENIZE_* flags */ const char *pText, int nText, /* Text to tokenize */ void *pCtx, /* Context passed to xToken() */ int (*xToken)(void*, int, const char*, int, int, int) /* Callback */ ){ | > | > > > > | > | | > > > > > > > > | 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 | int sqlite3Fts5Tokenize( Fts5Config *pConfig, /* FTS5 Configuration object */ int flags, /* FTS5_TOKENIZE_* flags */ const char *pText, int nText, /* Text to tokenize */ void *pCtx, /* Context passed to xToken() */ int (*xToken)(void*, int, const char*, int, int, int) /* Callback */ ){ int rc = SQLITE_OK; if( pText ){ if( pConfig->t.pTok==0 ){ rc = sqlite3Fts5LoadTokenizer(pConfig); } if( rc==SQLITE_OK ){ if( pConfig->t.pApi1 ){ rc = pConfig->t.pApi1->xTokenize( pConfig->t.pTok, pCtx, flags, pText, nText, xToken ); }else{ rc = pConfig->t.pApi2->xTokenize(pConfig->t.pTok, pCtx, flags, pText, nText, pConfig->t.pLocale, pConfig->t.nLocale, xToken ); } } } return rc; } /* ** Argument pIn points to the first character in what is expected to be ** a comma-separated list of SQL literals followed by a ')' character. ** If it actually is this, return a pointer to the ')'. Otherwise, return ** NULL to indicate a parse error. |
︙ | ︙ | |||
884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 | *pbBadkey = 1; }else{ if( nCrisisMerge<=1 ) nCrisisMerge = FTS5_DEFAULT_CRISISMERGE; if( nCrisisMerge>=FTS5_MAX_SEGMENT ) nCrisisMerge = FTS5_MAX_SEGMENT-1; pConfig->nCrisisMerge = nCrisisMerge; } } else if( 0==sqlite3_stricmp(zKey, "rank") ){ const char *zIn = (const char*)sqlite3_value_text(pVal); char *zRank; char *zRankArgs; rc = sqlite3Fts5ConfigParseRank(zIn, &zRank, &zRankArgs); if( rc==SQLITE_OK ){ sqlite3_free(pConfig->zRank); sqlite3_free(pConfig->zRankArgs); pConfig->zRank = zRank; pConfig->zRankArgs = zRankArgs; }else if( rc==SQLITE_ERROR ){ rc = SQLITE_OK; *pbBadkey = 1; } }else{ *pbBadkey = 1; } return rc; } /* | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 | *pbBadkey = 1; }else{ if( nCrisisMerge<=1 ) nCrisisMerge = FTS5_DEFAULT_CRISISMERGE; if( nCrisisMerge>=FTS5_MAX_SEGMENT ) nCrisisMerge = FTS5_MAX_SEGMENT-1; pConfig->nCrisisMerge = nCrisisMerge; } } else if( 0==sqlite3_stricmp(zKey, "deletemerge") ){ int nVal = -1; if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ nVal = sqlite3_value_int(pVal); }else{ *pbBadkey = 1; } if( nVal<0 ) nVal = FTS5_DEFAULT_DELETE_AUTOMERGE; if( nVal>100 ) nVal = 0; pConfig->nDeleteMerge = nVal; } else if( 0==sqlite3_stricmp(zKey, "rank") ){ const char *zIn = (const char*)sqlite3_value_text(pVal); char *zRank; char *zRankArgs; rc = sqlite3Fts5ConfigParseRank(zIn, &zRank, &zRankArgs); if( rc==SQLITE_OK ){ sqlite3_free(pConfig->zRank); sqlite3_free(pConfig->zRankArgs); pConfig->zRank = zRank; pConfig->zRankArgs = zRankArgs; }else if( rc==SQLITE_ERROR ){ rc = SQLITE_OK; *pbBadkey = 1; } } else if( 0==sqlite3_stricmp(zKey, "secure-delete") ){ int bVal = -1; if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ bVal = sqlite3_value_int(pVal); } if( bVal<0 ){ *pbBadkey = 1; }else{ pConfig->bSecureDelete = (bVal ? 1 : 0); } } else if( 0==sqlite3_stricmp(zKey, "insttoken") ){ int bVal = -1; if( SQLITE_INTEGER==sqlite3_value_numeric_type(pVal) ){ bVal = sqlite3_value_int(pVal); } if( bVal<0 ){ *pbBadkey = 1; }else{ pConfig->bPrefixInsttoken = (bVal ? 1 : 0); } }else{ *pbBadkey = 1; } return rc; } /* |
︙ | ︙ | |||
921 922 923 924 925 926 927 928 929 930 931 932 933 934 | /* Set default values */ pConfig->pgsz = FTS5_DEFAULT_PAGE_SIZE; pConfig->nAutomerge = FTS5_DEFAULT_AUTOMERGE; pConfig->nUsermerge = FTS5_DEFAULT_USERMERGE; pConfig->nCrisisMerge = FTS5_DEFAULT_CRISISMERGE; pConfig->nHashSize = FTS5_DEFAULT_HASHSIZE; zSql = sqlite3Fts5Mprintf(&rc, zSelect, pConfig->zDb, pConfig->zName); if( zSql ){ rc = sqlite3_prepare_v2(pConfig->db, zSql, -1, &p, 0); sqlite3_free(zSql); } | > | 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 | /* Set default values */ pConfig->pgsz = FTS5_DEFAULT_PAGE_SIZE; pConfig->nAutomerge = FTS5_DEFAULT_AUTOMERGE; pConfig->nUsermerge = FTS5_DEFAULT_USERMERGE; pConfig->nCrisisMerge = FTS5_DEFAULT_CRISISMERGE; pConfig->nHashSize = FTS5_DEFAULT_HASHSIZE; pConfig->nDeleteMerge = FTS5_DEFAULT_DELETE_AUTOMERGE; zSql = sqlite3Fts5Mprintf(&rc, zSelect, pConfig->zDb, pConfig->zName); if( zSql ){ rc = sqlite3_prepare_v2(pConfig->db, zSql, -1, &p, 0); sqlite3_free(zSql); } |
︙ | ︙ | |||
943 944 945 946 947 948 949 | int bDummy = 0; sqlite3Fts5ConfigSetValue(pConfig, zK, pVal, &bDummy); } } rc = sqlite3_finalize(p); } | > | > > | < < | | | | > > > > > > > > > > > > > > > > > > > > > > > > | 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 | int bDummy = 0; sqlite3Fts5ConfigSetValue(pConfig, zK, pVal, &bDummy); } } rc = sqlite3_finalize(p); } if( rc==SQLITE_OK && iVersion!=FTS5_CURRENT_VERSION && iVersion!=FTS5_CURRENT_VERSION_SECUREDELETE ){ rc = SQLITE_ERROR; sqlite3Fts5ConfigErrmsg(pConfig, "invalid fts5 file format " "(found %d, expected %d or %d) - run 'rebuild'", iVersion, FTS5_CURRENT_VERSION, FTS5_CURRENT_VERSION_SECUREDELETE ); }else{ pConfig->iVersion = iVersion; } if( rc==SQLITE_OK ){ pConfig->iCookie = iCookie; } return rc; } /* ** Set (*pConfig->pzErrmsg) to point to an sqlite3_malloc()ed buffer ** containing the error message created using printf() style formatting ** string zFmt and its trailing arguments. */ void sqlite3Fts5ConfigErrmsg(Fts5Config *pConfig, const char *zFmt, ...){ va_list ap; /* ... printf arguments */ char *zMsg = 0; va_start(ap, zFmt); zMsg = sqlite3_vmprintf(zFmt, ap); if( pConfig->pzErrmsg ){ assert( *pConfig->pzErrmsg==0 ); *pConfig->pzErrmsg = zMsg; }else{ sqlite3_free(zMsg); } va_end(ap); } |
Changes to ext/fts5/fts5_expr.c.
︙ | ︙ | |||
13 14 15 16 17 18 19 20 21 22 23 24 25 26 | */ #include "fts5Int.h" #include "fts5parse.h" /* ** All token types in the generated fts5parse.h file are greater than 0. */ #define FTS5_EOF 0 #define FTS5_LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) | > > > > | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 | */ #include "fts5Int.h" #include "fts5parse.h" #ifndef SQLITE_FTS5_MAX_EXPR_DEPTH # define SQLITE_FTS5_MAX_EXPR_DEPTH 256 #endif /* ** All token types in the generated fts5parse.h file are greater than 0. */ #define FTS5_EOF 0 #define FTS5_LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) |
︙ | ︙ | |||
46 47 48 49 50 51 52 | int bDesc; /* Iterate in descending rowid order */ int nPhrase; /* Number of phrases in expression */ Fts5ExprPhrase **apExprPhrase; /* Pointers to phrase objects */ }; /* ** eType: | | > > > > > > > > > > | > > > > | > > | > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 | int bDesc; /* Iterate in descending rowid order */ int nPhrase; /* Number of phrases in expression */ Fts5ExprPhrase **apExprPhrase; /* Pointers to phrase objects */ }; /* ** eType: ** Expression node type. Usually one of: ** ** FTS5_AND (nChild, apChild valid) ** FTS5_OR (nChild, apChild valid) ** FTS5_NOT (nChild, apChild valid) ** FTS5_STRING (pNear valid) ** FTS5_TERM (pNear valid) ** ** An expression node with eType==0 may also exist. It always matches zero ** rows. This is created when a phrase containing no tokens is parsed. ** e.g. "". ** ** iHeight: ** Distance from this node to furthest leaf. This is always 0 for nodes ** of type FTS5_STRING and FTS5_TERM. For all other nodes it is one ** greater than the largest child value. */ struct Fts5ExprNode { int eType; /* Node type */ int bEof; /* True at EOF */ int bNomatch; /* True if entry is not a match */ int iHeight; /* Distance to tree leaf nodes */ /* Next method for this node. */ int (*xNext)(Fts5Expr*, Fts5ExprNode*, int, i64); i64 iRowid; /* Current rowid */ Fts5ExprNearset *pNear; /* For FTS5_STRING - cluster of phrases */ /* Child nodes. For a NOT node, this array always contains 2 entries. For ** AND or OR nodes, it contains 2 or more entries. */ int nChild; /* Number of child nodes */ Fts5ExprNode *apChild[FLEXARRAY]; /* Array of child nodes */ }; /* Size (in bytes) of an Fts5ExprNode object that holds up to N children */ #define SZ_FTS5EXPRNODE(N) \ (offsetof(Fts5ExprNode,apChild) + (N)*sizeof(Fts5ExprNode*)) #define Fts5NodeIsString(p) ((p)->eType==FTS5_TERM || (p)->eType==FTS5_STRING) /* ** Invoke the xNext method of an Fts5ExprNode object. This macro should be ** used as if it has the same signature as the xNext() methods themselves. */ #define fts5ExprNodeNext(a,b,c,d) (b)->xNext((a), (b), (c), (d)) /* ** An instance of the following structure represents a single search term ** or term prefix. */ struct Fts5ExprTerm { u8 bPrefix; /* True for a prefix term */ u8 bFirst; /* True if token must be first in column */ char *pTerm; /* Term data */ int nQueryTerm; /* Effective size of term in bytes */ int nFullTerm; /* Size of term in bytes incl. tokendata */ Fts5IndexIter *pIter; /* Iterator for this term */ Fts5ExprTerm *pSynonym; /* Pointer to first in list of synonyms */ }; /* ** A phrase. One or more terms that must appear in a contiguous sequence ** within a document for it to match. */ struct Fts5ExprPhrase { Fts5ExprNode *pNode; /* FTS5_STRING node this phrase is part of */ Fts5Buffer poslist; /* Current position list */ int nTerm; /* Number of entries in aTerm[] */ Fts5ExprTerm aTerm[FLEXARRAY]; /* Terms that make up this phrase */ }; /* Size (in bytes) of an Fts5ExprPhrase object that holds up to N terms */ #define SZ_FTS5EXPRPHRASE(N) \ (offsetof(Fts5ExprPhrase,aTerm) + (N)*sizeof(Fts5ExprTerm)) /* ** One or more phrases that must appear within a certain token distance of ** each other within each matching document. */ struct Fts5ExprNearset { int nNear; /* NEAR parameter */ Fts5Colset *pColset; /* Columns to search (NULL -> all columns) */ int nPhrase; /* Number of entries in aPhrase[] array */ Fts5ExprPhrase *apPhrase[FLEXARRAY]; /* Array of phrase pointers */ }; /* Size (in bytes) of an Fts5ExprNearset object covering up to N phrases */ #define SZ_FTS5EXPRNEARSET(N) \ (offsetof(Fts5ExprNearset,apPhrase)+(N)*sizeof(Fts5ExprPhrase*)) /* ** Parse context. */ struct Fts5Parse { Fts5Config *pConfig; char *zErr; int rc; int nPhrase; /* Size of apPhrase array */ Fts5ExprPhrase **apPhrase; /* Array of all phrases */ Fts5ExprNode *pExpr; /* Result of a successful parse */ int bPhraseToAnd; /* Convert "a+b" to "a AND b" */ }; /* ** Check that the Fts5ExprNode.iHeight variables are set correctly in ** the expression tree passed as the only argument. */ #ifndef NDEBUG static void assert_expr_depth_ok(int rc, Fts5ExprNode *p){ if( rc==SQLITE_OK ){ if( p->eType==FTS5_TERM || p->eType==FTS5_STRING || p->eType==0 ){ assert( p->iHeight==0 ); }else{ int ii; int iMaxChild = 0; for(ii=0; ii<p->nChild; ii++){ Fts5ExprNode *pChild = p->apChild[ii]; iMaxChild = MAX(iMaxChild, pChild->iHeight); assert_expr_depth_ok(SQLITE_OK, pChild); } assert( p->iHeight==iMaxChild+1 ); } } } #else # define assert_expr_depth_ok(rc, p) #endif void sqlite3Fts5ParseError(Fts5Parse *pParse, const char *zFmt, ...){ va_list ap; va_start(ap, zFmt); if( pParse->rc==SQLITE_OK ){ assert( pParse->zErr==0 ); pParse->zErr = sqlite3_vmprintf(zFmt, ap); pParse->rc = SQLITE_ERROR; |
︙ | ︙ | |||
240 241 242 243 244 245 246 247 248 249 | sParse.pConfig = pConfig; do { t = fts5ExprGetToken(&sParse, &z, &token); sqlite3Fts5Parser(pEngine, t, token, &sParse); }while( sParse.rc==SQLITE_OK && t!=FTS5_EOF ); sqlite3Fts5ParserFree(pEngine, fts5ParseFree); /* If the LHS of the MATCH expression was a user column, apply the ** implicit column-filter. */ | > > > | | < < < < < < < | < > | > > > > > > > > > > > > > > > > | 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 | sParse.pConfig = pConfig; do { t = fts5ExprGetToken(&sParse, &z, &token); sqlite3Fts5Parser(pEngine, t, token, &sParse); }while( sParse.rc==SQLITE_OK && t!=FTS5_EOF ); sqlite3Fts5ParserFree(pEngine, fts5ParseFree); assert( sParse.pExpr || sParse.rc!=SQLITE_OK ); assert_expr_depth_ok(sParse.rc, sParse.pExpr); /* If the LHS of the MATCH expression was a user column, apply the ** implicit column-filter. */ if( sParse.rc==SQLITE_OK && iCol<pConfig->nCol ){ int n = SZ_FTS5COLSET(1); Fts5Colset *pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&sParse.rc, n); if( pColset ){ pColset->nCol = 1; pColset->aiCol[0] = iCol; sqlite3Fts5ParseSetColset(&sParse, sParse.pExpr, pColset); } } assert( sParse.rc!=SQLITE_OK || sParse.zErr==0 ); if( sParse.rc==SQLITE_OK ){ *ppNew = pNew = sqlite3_malloc(sizeof(Fts5Expr)); if( pNew==0 ){ sParse.rc = SQLITE_NOMEM; sqlite3Fts5ParseNodeFree(sParse.pExpr); }else{ pNew->pRoot = sParse.pExpr; pNew->pIndex = 0; pNew->pConfig = pConfig; pNew->apExprPhrase = sParse.apPhrase; pNew->nPhrase = sParse.nPhrase; pNew->bDesc = 0; sParse.apPhrase = 0; } }else{ sqlite3Fts5ParseNodeFree(sParse.pExpr); } sqlite3_free(sParse.apPhrase); if( 0==*pzErr ){ *pzErr = sParse.zErr; }else{ sqlite3_free(sParse.zErr); } return sParse.rc; } /* ** Assuming that buffer z is at least nByte bytes in size and contains a ** valid utf-8 string, return the number of characters in the string. */ static int fts5ExprCountChar(const char *z, int nByte){ int nRet = 0; int ii; for(ii=0; ii<nByte; ii++){ if( (z[ii] & 0xC0)!=0x80 ) nRet++; } return nRet; } /* ** This function is only called when using the special 'trigram' tokenizer. ** Argument zText contains the text of a LIKE or GLOB pattern matched ** against column iCol. This function creates and compiles an FTS5 MATCH ** expression that will match a superset of the rows matched by the LIKE or ** GLOB. If successful, SQLITE_OK is returned. Otherwise, an SQLite error |
︙ | ︙ | |||
322 323 324 325 326 327 328 | aSpec[2] = '['; } while( i<=nText ){ if( i==nText || zText[i]==aSpec[0] || zText[i]==aSpec[1] || zText[i]==aSpec[2] ){ | | > | 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 | aSpec[2] = '['; } while( i<=nText ){ if( i==nText || zText[i]==aSpec[0] || zText[i]==aSpec[1] || zText[i]==aSpec[2] ){ if( fts5ExprCountChar(&zText[iFirst], i-iFirst)>=3 ){ int jj; zExpr[iOut++] = '"'; for(jj=iFirst; jj<i; jj++){ zExpr[iOut++] = zText[jj]; if( zText[jj]=='"' ) zExpr[iOut++] = '"'; } zExpr[iOut++] = '"'; |
︙ | ︙ | |||
389 390 391 392 393 394 395 | } } int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2){ Fts5Parse sParse; memset(&sParse, 0, sizeof(sParse)); | | | 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 | } } int sqlite3Fts5ExprAnd(Fts5Expr **pp1, Fts5Expr *p2){ Fts5Parse sParse; memset(&sParse, 0, sizeof(sParse)); if( *pp1 && p2 ){ Fts5Expr *p1 = *pp1; int nPhrase = p1->nPhrase + p2->nPhrase; p1->pRoot = sqlite3Fts5ParseNode(&sParse, FTS5_AND, p1->pRoot, p2->pRoot,0); p2->pRoot = 0; if( sParse.rc==SQLITE_OK ){ |
︙ | ︙ | |||
414 415 416 417 418 419 420 | } p1->nPhrase = nPhrase; p1->apExprPhrase = ap; } } sqlite3_free(p2->apExprPhrase); sqlite3_free(p2); | | | 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 | } p1->nPhrase = nPhrase; p1->apExprPhrase = ap; } } sqlite3_free(p2->apExprPhrase); sqlite3_free(p2); }else if( p2 ){ *pp1 = p2; } return sParse.rc; } /* |
︙ | ︙ | |||
912 913 914 915 916 917 918 | for(p=pTerm; p; p=p->pSynonym){ int rc; if( p->pIter ){ sqlite3Fts5IterClose(p->pIter); p->pIter = 0; } rc = sqlite3Fts5IndexQuery( | | | 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 | for(p=pTerm; p; p=p->pSynonym){ int rc; if( p->pIter ){ sqlite3Fts5IterClose(p->pIter); p->pIter = 0; } rc = sqlite3Fts5IndexQuery( pExpr->pIndex, p->pTerm, p->nQueryTerm, (pTerm->bPrefix ? FTS5INDEX_QUERY_PREFIX : 0) | (pExpr->bDesc ? FTS5INDEX_QUERY_DESC : 0), pNear->pColset, &p->pIter ); assert( (rc==SQLITE_OK)==(p->pIter!=0) ); if( rc!=SQLITE_OK ) return rc; |
︙ | ︙ | |||
1068 1069 1070 1071 1072 1073 1074 | if( fts5ExprSynonymAdvanceto(pTerm, bDesc, &iLast, &rc) ){ pNode->bNomatch = 0; pNode->bEof = 1; return rc; } }else{ Fts5IndexIter *pIter = pPhrase->aTerm[j].pIter; | | | 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 | if( fts5ExprSynonymAdvanceto(pTerm, bDesc, &iLast, &rc) ){ pNode->bNomatch = 0; pNode->bEof = 1; return rc; } }else{ Fts5IndexIter *pIter = pPhrase->aTerm[j].pIter; if( pIter->iRowid==iLast ) continue; bMatch = 0; if( fts5ExprAdvanceto(pIter, bDesc, &iLast, &rc, &pNode->bEof) ){ return rc; } } } } |
︙ | ︙ | |||
1549 1550 1551 1552 1553 1554 1555 | static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){ if( pPhrase ){ int i; for(i=0; i<pPhrase->nTerm; i++){ Fts5ExprTerm *pSyn; Fts5ExprTerm *pNext; Fts5ExprTerm *pTerm = &pPhrase->aTerm[i]; | | | 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 | static void fts5ExprPhraseFree(Fts5ExprPhrase *pPhrase){ if( pPhrase ){ int i; for(i=0; i<pPhrase->nTerm; i++){ Fts5ExprTerm *pSyn; Fts5ExprTerm *pNext; Fts5ExprTerm *pTerm = &pPhrase->aTerm[i]; sqlite3_free(pTerm->pTerm); sqlite3Fts5IterClose(pTerm->pIter); for(pSyn=pTerm->pSynonym; pSyn; pSyn=pNext){ pNext = pSyn->pSynonym; sqlite3Fts5IterClose(pSyn->pIter); fts5BufferFree((Fts5Buffer*)&pSyn[1]); sqlite3_free(pSyn); } |
︙ | ︙ | |||
1590 1591 1592 1593 1594 1595 1596 | Fts5ExprNearset *pNear, /* Existing nearset, or NULL */ Fts5ExprPhrase *pPhrase /* Recently parsed phrase */ ){ const int SZALLOC = 8; Fts5ExprNearset *pRet = 0; if( pParse->rc==SQLITE_OK ){ | < < < | | | 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 | Fts5ExprNearset *pNear, /* Existing nearset, or NULL */ Fts5ExprPhrase *pPhrase /* Recently parsed phrase */ ){ const int SZALLOC = 8; Fts5ExprNearset *pRet = 0; if( pParse->rc==SQLITE_OK ){ if( pNear==0 ){ sqlite3_int64 nByte; nByte = SZ_FTS5EXPRNEARSET(SZALLOC+1); pRet = sqlite3_malloc64(nByte); if( pRet==0 ){ pParse->rc = SQLITE_NOMEM; }else{ memset(pRet, 0, (size_t)nByte); } }else if( (pNear->nPhrase % SZALLOC)==0 ){ int nNew = pNear->nPhrase + SZALLOC; sqlite3_int64 nByte; nByte = SZ_FTS5EXPRNEARSET(nNew+1); pRet = (Fts5ExprNearset*)sqlite3_realloc64(pNear, nByte); if( pRet==0 ){ pParse->rc = SQLITE_NOMEM; } }else{ pRet = pNear; } |
︙ | ︙ | |||
1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 | } return pRet; } typedef struct TokenCtx TokenCtx; struct TokenCtx { Fts5ExprPhrase *pPhrase; int rc; }; /* ** Callback for tokenizing terms used by ParseTerm(). */ static int fts5ParseTokenize( | > | 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 | } return pRet; } typedef struct TokenCtx TokenCtx; struct TokenCtx { Fts5ExprPhrase *pPhrase; Fts5Config *pConfig; int rc; }; /* ** Callback for tokenizing terms used by ParseTerm(). */ static int fts5ParseTokenize( |
︙ | ︙ | |||
1680 1681 1682 1683 1684 1685 1686 | Fts5ExprTerm *pSyn; sqlite3_int64 nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1; pSyn = (Fts5ExprTerm*)sqlite3_malloc64(nByte); if( pSyn==0 ){ rc = SQLITE_NOMEM; }else{ memset(pSyn, 0, (size_t)nByte); | | > > > > | | | | > > > > | 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 | Fts5ExprTerm *pSyn; sqlite3_int64 nByte = sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer) + nToken+1; pSyn = (Fts5ExprTerm*)sqlite3_malloc64(nByte); if( pSyn==0 ){ rc = SQLITE_NOMEM; }else{ memset(pSyn, 0, (size_t)nByte); pSyn->pTerm = ((char*)pSyn) + sizeof(Fts5ExprTerm) + sizeof(Fts5Buffer); pSyn->nFullTerm = pSyn->nQueryTerm = nToken; if( pCtx->pConfig->bTokendata ){ pSyn->nQueryTerm = (int)strlen(pSyn->pTerm); } memcpy(pSyn->pTerm, pToken, nToken); pSyn->pSynonym = pPhrase->aTerm[pPhrase->nTerm-1].pSynonym; pPhrase->aTerm[pPhrase->nTerm-1].pSynonym = pSyn; } }else{ Fts5ExprTerm *pTerm; if( pPhrase==0 || (pPhrase->nTerm % SZALLOC)==0 ){ Fts5ExprPhrase *pNew; int nNew = SZALLOC + (pPhrase ? pPhrase->nTerm : 0); pNew = (Fts5ExprPhrase*)sqlite3_realloc64(pPhrase, SZ_FTS5EXPRPHRASE(nNew+1) ); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ if( pPhrase==0 ) memset(pNew, 0, SZ_FTS5EXPRPHRASE(1)); pCtx->pPhrase = pPhrase = pNew; pNew->nTerm = nNew - SZALLOC; } } if( rc==SQLITE_OK ){ pTerm = &pPhrase->aTerm[pPhrase->nTerm++]; memset(pTerm, 0, sizeof(Fts5ExprTerm)); pTerm->pTerm = sqlite3Fts5Strndup(&rc, pToken, nToken); pTerm->nFullTerm = pTerm->nQueryTerm = nToken; if( pCtx->pConfig->bTokendata && rc==SQLITE_OK ){ pTerm->nQueryTerm = (int)strlen(pTerm->pTerm); } } } pCtx->rc = rc; return rc; } |
︙ | ︙ | |||
1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 | Fts5Config *pConfig = pParse->pConfig; TokenCtx sCtx; /* Context object passed to callback */ int rc; /* Tokenize return code */ char *z = 0; memset(&sCtx, 0, sizeof(TokenCtx)); sCtx.pPhrase = pAppend; rc = fts5ParseStringFromToken(pToken, &z); if( rc==SQLITE_OK ){ int flags = FTS5_TOKENIZE_QUERY | (bPrefix ? FTS5_TOKENIZE_PREFIX : 0); int n; sqlite3Fts5Dequote(z); n = (int)strlen(z); | > | 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 | Fts5Config *pConfig = pParse->pConfig; TokenCtx sCtx; /* Context object passed to callback */ int rc; /* Tokenize return code */ char *z = 0; memset(&sCtx, 0, sizeof(TokenCtx)); sCtx.pPhrase = pAppend; sCtx.pConfig = pConfig; rc = fts5ParseStringFromToken(pToken, &z); if( rc==SQLITE_OK ){ int flags = FTS5_TOKENIZE_QUERY | (bPrefix ? FTS5_TOKENIZE_PREFIX : 0); int n; sqlite3Fts5Dequote(z); n = (int)strlen(z); |
︙ | ︙ | |||
1800 1801 1802 1803 1804 1805 1806 | } pParse->nPhrase++; } if( sCtx.pPhrase==0 ){ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ | | > | | | > > | | > | < | < > | | > | | > | | | | < | < | | | | | | | | | | | > | 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 | } pParse->nPhrase++; } if( sCtx.pPhrase==0 ){ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ sCtx.pPhrase = sqlite3Fts5MallocZero(&pParse->rc, SZ_FTS5EXPRPHRASE(1)); }else if( sCtx.pPhrase->nTerm ){ sCtx.pPhrase->aTerm[sCtx.pPhrase->nTerm-1].bPrefix = (u8)bPrefix; } assert( pParse->apPhrase!=0 ); pParse->apPhrase[pParse->nPhrase-1] = sCtx.pPhrase; } return sCtx.pPhrase; } /* ** Create a new FTS5 expression by cloning phrase iPhrase of the ** expression passed as the second argument. */ int sqlite3Fts5ExprClonePhrase( Fts5Expr *pExpr, int iPhrase, Fts5Expr **ppNew ){ int rc = SQLITE_OK; /* Return code */ Fts5ExprPhrase *pOrig = 0; /* The phrase extracted from pExpr */ Fts5Expr *pNew = 0; /* Expression to return via *ppNew */ TokenCtx sCtx = {0,0,0}; /* Context object for fts5ParseTokenize */ if( !pExpr || iPhrase<0 || iPhrase>=pExpr->nPhrase ){ rc = SQLITE_RANGE; }else{ pOrig = pExpr->apExprPhrase[iPhrase]; pNew = (Fts5Expr*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5Expr)); } if( rc==SQLITE_OK ){ pNew->apExprPhrase = (Fts5ExprPhrase**)sqlite3Fts5MallocZero(&rc, sizeof(Fts5ExprPhrase*)); } if( rc==SQLITE_OK ){ pNew->pRoot = (Fts5ExprNode*)sqlite3Fts5MallocZero(&rc, SZ_FTS5EXPRNODE(1)); } if( rc==SQLITE_OK ){ pNew->pRoot->pNear = (Fts5ExprNearset*)sqlite3Fts5MallocZero(&rc, SZ_FTS5EXPRNEARSET(2)); } if( rc==SQLITE_OK && ALWAYS(pOrig!=0) ){ Fts5Colset *pColsetOrig = pOrig->pNode->pNear->pColset; if( pColsetOrig ){ sqlite3_int64 nByte; Fts5Colset *pColset; nByte = SZ_FTS5COLSET(pColsetOrig->nCol); pColset = (Fts5Colset*)sqlite3Fts5MallocZero(&rc, nByte); if( pColset ){ memcpy(pColset, pColsetOrig, (size_t)nByte); } pNew->pRoot->pNear->pColset = pColset; } } if( rc==SQLITE_OK ){ if( pOrig->nTerm ){ int i; /* Used to iterate through phrase terms */ sCtx.pConfig = pExpr->pConfig; for(i=0; rc==SQLITE_OK && i<pOrig->nTerm; i++){ int tflags = 0; Fts5ExprTerm *p; for(p=&pOrig->aTerm[i]; p && rc==SQLITE_OK; p=p->pSynonym){ rc = fts5ParseTokenize((void*)&sCtx,tflags,p->pTerm,p->nFullTerm,0,0); tflags = FTS5_TOKEN_COLOCATED; } if( rc==SQLITE_OK ){ sCtx.pPhrase->aTerm[i].bPrefix = pOrig->aTerm[i].bPrefix; sCtx.pPhrase->aTerm[i].bFirst = pOrig->aTerm[i].bFirst; } } }else{ /* This happens when parsing a token or quoted phrase that contains ** no token characters at all. (e.g ... MATCH '""'). */ sCtx.pPhrase = sqlite3Fts5MallocZero(&rc, SZ_FTS5EXPRPHRASE(1)); } } if( rc==SQLITE_OK && ALWAYS(sCtx.pPhrase) ){ /* All the allocations succeeded. Put the expression object together. */ pNew->pIndex = pExpr->pIndex; pNew->pConfig = pExpr->pConfig; pNew->nPhrase = 1; |
︙ | ︙ | |||
1935 1936 1937 1938 1939 1940 1941 | char c = (char)p->p[i]; if( c<'0' || c>'9' ){ sqlite3Fts5ParseError( pParse, "expected integer, got \"%.*s\"", p->n, p->p ); return; } | | > | 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 | char c = (char)p->p[i]; if( c<'0' || c>'9' ){ sqlite3Fts5ParseError( pParse, "expected integer, got \"%.*s\"", p->n, p->p ); return; } if( nNear<214748363 ) nNear = nNear * 10 + (p->p[i] - '0'); /* ^^^^^^^^^^^^^^^--- Prevent integer overflow */ } }else{ nNear = FTS5_DEFAULT_NEARDIST; } pNear->nNear = nNear; } } |
︙ | ︙ | |||
1964 1965 1966 1967 1968 1969 1970 | ){ int nCol = p ? p->nCol : 0; /* Num. columns already in colset object */ Fts5Colset *pNew; /* New colset object to return */ assert( pParse->rc==SQLITE_OK ); assert( iCol>=0 && iCol<pParse->pConfig->nCol ); | | | 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 | ){ int nCol = p ? p->nCol : 0; /* Num. columns already in colset object */ Fts5Colset *pNew; /* New colset object to return */ assert( pParse->rc==SQLITE_OK ); assert( iCol>=0 && iCol<pParse->pConfig->nCol ); pNew = sqlite3_realloc64(p, SZ_FTS5COLSET(nCol+1)); if( pNew==0 ){ pParse->rc = SQLITE_NOMEM; }else{ int *aiCol = pNew->aiCol; int i, j; for(i=0; i<nCol; i++){ if( aiCol[i]==iCol ) return pNew; |
︙ | ︙ | |||
1999 2000 2001 2002 2003 2004 2005 | ** as the second argument before returning. */ Fts5Colset *sqlite3Fts5ParseColsetInvert(Fts5Parse *pParse, Fts5Colset *p){ Fts5Colset *pRet; int nCol = pParse->pConfig->nCol; pRet = (Fts5Colset*)sqlite3Fts5MallocZero(&pParse->rc, | | | 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 | ** as the second argument before returning. */ Fts5Colset *sqlite3Fts5ParseColsetInvert(Fts5Parse *pParse, Fts5Colset *p){ Fts5Colset *pRet; int nCol = pParse->pConfig->nCol; pRet = (Fts5Colset*)sqlite3Fts5MallocZero(&pParse->rc, SZ_FTS5COLSET(nCol+1) ); if( pRet ){ int i; int iOld = 0; for(i=0; i<nCol; i++){ if( iOld>=p->nCol || p->aiCol[iOld]!=i ){ pRet->aiCol[pRet->nCol++] = i; |
︙ | ︙ | |||
2060 2061 2062 2063 2064 2065 2066 | ** Otherwise, a copy of (*pOrig) is made into memory obtained from ** sqlite3Fts5MallocZero() and a pointer to it returned. If the allocation ** fails, (*pRc) is set to SQLITE_NOMEM and NULL is returned. */ static Fts5Colset *fts5CloneColset(int *pRc, Fts5Colset *pOrig){ Fts5Colset *pRet; if( pOrig ){ | | | 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 | ** Otherwise, a copy of (*pOrig) is made into memory obtained from ** sqlite3Fts5MallocZero() and a pointer to it returned. If the allocation ** fails, (*pRc) is set to SQLITE_NOMEM and NULL is returned. */ static Fts5Colset *fts5CloneColset(int *pRc, Fts5Colset *pOrig){ Fts5Colset *pRet; if( pOrig ){ sqlite3_int64 nByte = SZ_FTS5COLSET(pOrig->nCol); pRet = (Fts5Colset*)sqlite3Fts5MallocZero(pRc, nByte); if( pRet ){ memcpy(pRet, pOrig, (size_t)nByte); } }else{ pRet = 0; } |
︙ | ︙ | |||
2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 | default: assert( pNode->eType==FTS5_NOT ); { pNode->xNext = fts5ExprNodeNext_NOT; break; }; } } static void fts5ExprAddChildren(Fts5ExprNode *p, Fts5ExprNode *pSub){ if( p->eType!=FTS5_NOT && pSub->eType==p->eType ){ int nByte = sizeof(Fts5ExprNode*) * pSub->nChild; memcpy(&p->apChild[p->nChild], pSub->apChild, nByte); p->nChild += pSub->nChild; sqlite3_free(pSub); }else{ p->apChild[p->nChild++] = pSub; } } /* ** This function is used when parsing LIKE or GLOB patterns against ** trigram indexes that specify either detail=column or detail=none. ** It converts a phrase: ** | > > > > > > > | 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 | default: assert( pNode->eType==FTS5_NOT ); { pNode->xNext = fts5ExprNodeNext_NOT; break; }; } } /* ** Add pSub as a child of p. */ static void fts5ExprAddChildren(Fts5ExprNode *p, Fts5ExprNode *pSub){ int ii = p->nChild; if( p->eType!=FTS5_NOT && pSub->eType==p->eType ){ int nByte = sizeof(Fts5ExprNode*) * pSub->nChild; memcpy(&p->apChild[p->nChild], pSub->apChild, nByte); p->nChild += pSub->nChild; sqlite3_free(pSub); }else{ p->apChild[p->nChild++] = pSub; } for( ; ii<p->nChild; ii++){ p->iHeight = MAX(p->iHeight, p->apChild[ii]->iHeight + 1); } } /* ** This function is used when parsing LIKE or GLOB patterns against ** trigram indexes that specify either detail=column or detail=none. ** It converts a phrase: ** |
︙ | ︙ | |||
2221 2222 2223 2224 2225 2226 2227 | int ii; int nByte; Fts5ExprNode *pRet; assert( pNear->nPhrase==1 ); assert( pParse->bPhraseToAnd ); | | > | > > | < < > > | 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 | int ii; int nByte; Fts5ExprNode *pRet; assert( pNear->nPhrase==1 ); assert( pParse->bPhraseToAnd ); nByte = SZ_FTS5EXPRNODE(nTerm+1); pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte); if( pRet ){ pRet->eType = FTS5_AND; pRet->nChild = nTerm; pRet->iHeight = 1; fts5ExprAssignXNext(pRet); pParse->nPhrase--; for(ii=0; ii<nTerm; ii++){ Fts5ExprPhrase *pPhrase = (Fts5ExprPhrase*)sqlite3Fts5MallocZero( &pParse->rc, SZ_FTS5EXPRPHRASE(1) ); if( pPhrase ){ if( parseGrowPhraseArray(pParse) ){ fts5ExprPhraseFree(pPhrase); }else{ Fts5ExprTerm *p = &pNear->apPhrase[0]->aTerm[ii]; Fts5ExprTerm *pTo = &pPhrase->aTerm[0]; pParse->apPhrase[pParse->nPhrase++] = pPhrase; pPhrase->nTerm = 1; pTo->pTerm = sqlite3Fts5Strndup(&pParse->rc, p->pTerm, p->nFullTerm); pTo->nQueryTerm = p->nQueryTerm; pTo->nFullTerm = p->nFullTerm; pRet->apChild[ii] = sqlite3Fts5ParseNode(pParse, FTS5_STRING, 0, 0, sqlite3Fts5ParseNearset(pParse, 0, pPhrase) ); } } } |
︙ | ︙ | |||
2297 2298 2299 2300 2301 2302 2303 | nChild = 2; }else if( eType==FTS5_AND || eType==FTS5_OR ){ nChild = 2; if( pLeft->eType==eType ) nChild += pLeft->nChild-1; if( pRight->eType==eType ) nChild += pRight->nChild-1; } | | | 2384 2385 2386 2387 2388 2389 2390 2391 2392 2393 2394 2395 2396 2397 2398 | nChild = 2; }else if( eType==FTS5_AND || eType==FTS5_OR ){ nChild = 2; if( pLeft->eType==eType ) nChild += pLeft->nChild-1; if( pRight->eType==eType ) nChild += pRight->nChild-1; } nByte = SZ_FTS5EXPRNODE(nChild); pRet = (Fts5ExprNode*)sqlite3Fts5MallocZero(&pParse->rc, nByte); if( pRet ){ pRet->eType = eType; pRet->pNear = pNear; fts5ExprAssignXNext(pRet); if( eType==FTS5_STRING ){ |
︙ | ︙ | |||
2324 2325 2326 2327 2328 2329 2330 | || pPhrase->nTerm>1 || (pPhrase->nTerm>0 && pPhrase->aTerm[0].bFirst) ){ sqlite3Fts5ParseError(pParse, "fts5: %s queries are not supported (detail!=full)", pNear->nPhrase==1 ? "phrase": "NEAR" ); | | > > > > > > > > > > > > | 2411 2412 2413 2414 2415 2416 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 | || pPhrase->nTerm>1 || (pPhrase->nTerm>0 && pPhrase->aTerm[0].bFirst) ){ sqlite3Fts5ParseError(pParse, "fts5: %s queries are not supported (detail!=full)", pNear->nPhrase==1 ? "phrase": "NEAR" ); sqlite3Fts5ParseNodeFree(pRet); pRet = 0; pNear = 0; assert( pLeft==0 && pRight==0 ); } } }else{ assert( pNear==0 ); fts5ExprAddChildren(pRet, pLeft); fts5ExprAddChildren(pRet, pRight); pLeft = pRight = 0; if( pRet->iHeight>SQLITE_FTS5_MAX_EXPR_DEPTH ){ sqlite3Fts5ParseError(pParse, "fts5 expression tree is too large (maximum depth %d)", SQLITE_FTS5_MAX_EXPR_DEPTH ); sqlite3Fts5ParseNodeFree(pRet); pRet = 0; } } } } } if( pRet==0 ){ assert( pParse->rc!=SQLITE_OK ); |
︙ | ︙ | |||
2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 | || pLeft->eType==FTS5_TERM || pLeft->eType==FTS5_EOF || pLeft->eType==FTS5_AND ); assert( pRight->eType==FTS5_STRING || pRight->eType==FTS5_TERM || pRight->eType==FTS5_EOF ); if( pLeft->eType==FTS5_AND ){ pPrev = pLeft->apChild[pLeft->nChild-1]; }else{ pPrev = pLeft; } assert( pPrev->eType==FTS5_STRING || pPrev->eType==FTS5_TERM || pPrev->eType==FTS5_EOF ); if( pRight->eType==FTS5_EOF ){ assert( pParse->apPhrase[pParse->nPhrase-1]==pRight->pNear->apPhrase[0] ); sqlite3Fts5ParseNodeFree(pRight); pRet = pLeft; pParse->nPhrase--; } else if( pPrev->eType==FTS5_EOF ){ Fts5ExprPhrase **ap; | > > > | 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 | || pLeft->eType==FTS5_TERM || pLeft->eType==FTS5_EOF || pLeft->eType==FTS5_AND ); assert( pRight->eType==FTS5_STRING || pRight->eType==FTS5_TERM || pRight->eType==FTS5_EOF || (pRight->eType==FTS5_AND && pParse->bPhraseToAnd) ); if( pLeft->eType==FTS5_AND ){ pPrev = pLeft->apChild[pLeft->nChild-1]; }else{ pPrev = pLeft; } assert( pPrev->eType==FTS5_STRING || pPrev->eType==FTS5_TERM || pPrev->eType==FTS5_EOF ); if( pRight->eType==FTS5_EOF ){ assert( pParse->apPhrase!=0 ); assert( pParse->nPhrase>0 ); assert( pParse->apPhrase[pParse->nPhrase-1]==pRight->pNear->apPhrase[0] ); sqlite3Fts5ParseNodeFree(pRight); pRet = pLeft; pParse->nPhrase--; } else if( pPrev->eType==FTS5_EOF ){ Fts5ExprPhrase **ap; |
︙ | ︙ | |||
2409 2410 2411 2412 2413 2414 2415 | pRet = sqlite3Fts5ParseNode(pParse, FTS5_AND, pLeft, pRight, 0); } } return pRet; } | | | | > | | 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 | pRet = sqlite3Fts5ParseNode(pParse, FTS5_AND, pLeft, pRight, 0); } } return pRet; } #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) static char *fts5ExprTermPrint(Fts5ExprTerm *pTerm){ sqlite3_int64 nByte = 0; Fts5ExprTerm *p; char *zQuoted; /* Determine the maximum amount of space required. */ for(p=pTerm; p; p=p->pSynonym){ nByte += pTerm->nQueryTerm * 2 + 3 + 2; } zQuoted = sqlite3_malloc64(nByte); if( zQuoted ){ int i = 0; for(p=pTerm; p; p=p->pSynonym){ char *zIn = p->pTerm; char *zEnd = &zIn[p->nQueryTerm]; zQuoted[i++] = '"'; while( zIn<zEnd ){ if( *zIn=='"' ) zQuoted[i++] = '"'; zQuoted[i++] = *zIn++; } zQuoted[i++] = '"'; if( p->pSynonym ) zQuoted[i++] = '|'; } if( pTerm->bPrefix ){ |
︙ | ︙ | |||
2504 2505 2506 2507 2508 2509 2510 | if( zRet==0 ) return 0; for(i=0; i<pNear->nPhrase; i++){ Fts5ExprPhrase *pPhrase = pNear->apPhrase[i]; zRet = fts5PrintfAppend(zRet, " {"); for(iTerm=0; zRet && iTerm<pPhrase->nTerm; iTerm++){ | | | > > > > | 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 | if( zRet==0 ) return 0; for(i=0; i<pNear->nPhrase; i++){ Fts5ExprPhrase *pPhrase = pNear->apPhrase[i]; zRet = fts5PrintfAppend(zRet, " {"); for(iTerm=0; zRet && iTerm<pPhrase->nTerm; iTerm++){ Fts5ExprTerm *p = &pPhrase->aTerm[iTerm]; zRet = fts5PrintfAppend(zRet, "%s%.*s", iTerm==0?"":" ", p->nQueryTerm, p->pTerm ); if( pPhrase->aTerm[iTerm].bPrefix ){ zRet = fts5PrintfAppend(zRet, "*"); } } if( zRet ) zRet = fts5PrintfAppend(zRet, "}"); if( zRet==0 ) return 0; } }else if( pExpr->eType==0 ){ zRet = sqlite3_mprintf("{}"); }else{ char const *zOp = 0; int i; switch( pExpr->eType ){ case FTS5_AND: zOp = "AND"; break; case FTS5_NOT: zOp = "NOT"; break; default: |
︙ | ︙ | |||
2776 2777 2778 2779 2780 2781 2782 | int iCode; int bRemoveDiacritics = 0; iCode = sqlite3_value_int(apVal[0]); if( nArg==2 ) bRemoveDiacritics = sqlite3_value_int(apVal[1]); sqlite3_result_int(pCtx, sqlite3Fts5UnicodeFold(iCode, bRemoveDiacritics)); } } | | | | 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 | int iCode; int bRemoveDiacritics = 0; iCode = sqlite3_value_int(apVal[0]); if( nArg==2 ) bRemoveDiacritics = sqlite3_value_int(apVal[1]); sqlite3_result_int(pCtx, sqlite3Fts5UnicodeFold(iCode, bRemoveDiacritics)); } } #endif /* if SQLITE_TEST || SQLITE_FTS5_DEBUG */ /* ** This is called during initialization to register the fts5_expr() scalar ** UDF with the SQLite handle passed as the only argument. */ int sqlite3Fts5ExprInit(Fts5Global *pGlobal, sqlite3 *db){ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) struct Fts5ExprFunc { const char *z; void (*x)(sqlite3_context*,int,sqlite3_value**); } aFunc[] = { { "fts5_expr", fts5ExprFunctionHr }, { "fts5_expr_tcl", fts5ExprFunctionTcl }, { "fts5_isalnum", fts5ExprIsAlnum }, |
︙ | ︙ | |||
2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 | static int fts5ExprColsetTest(Fts5Colset *pColset, int iCol){ int i; for(i=0; i<pColset->nCol; i++){ if( pColset->aiCol[i]==iCol ) return 1; } return 0; } static int fts5ExprPopulatePoslistsCb( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iUnused1, /* Byte offset of token within input text */ int iUnused2 /* Byte offset of end of token within input text */ ){ Fts5ExprCtx *p = (Fts5ExprCtx*)pCtx; Fts5Expr *pExpr = p->pExpr; int i; UNUSED_PARAM2(iUnused1, iUnused2); | > > > > > > > > > > > > > | > > > | | < | | > > > > > > > | 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 3071 3072 3073 | static int fts5ExprColsetTest(Fts5Colset *pColset, int iCol){ int i; for(i=0; i<pColset->nCol; i++){ if( pColset->aiCol[i]==iCol ) return 1; } return 0; } /* ** pToken is a buffer nToken bytes in size that may or may not contain ** an embedded 0x00 byte. If it does, return the number of bytes in ** the buffer before the 0x00. If it does not, return nToken. */ static int fts5QueryTerm(const char *pToken, int nToken){ int ii; for(ii=0; ii<nToken && pToken[ii]; ii++){} return ii; } static int fts5ExprPopulatePoslistsCb( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iUnused1, /* Byte offset of token within input text */ int iUnused2 /* Byte offset of end of token within input text */ ){ Fts5ExprCtx *p = (Fts5ExprCtx*)pCtx; Fts5Expr *pExpr = p->pExpr; int i; int nQuery = nToken; i64 iRowid = pExpr->pRoot->iRowid; UNUSED_PARAM2(iUnused1, iUnused2); if( nQuery>FTS5_MAX_TOKEN_SIZE ) nQuery = FTS5_MAX_TOKEN_SIZE; if( pExpr->pConfig->bTokendata ){ nQuery = fts5QueryTerm(pToken, nQuery); } if( (tflags & FTS5_TOKEN_COLOCATED)==0 ) p->iOff++; for(i=0; i<pExpr->nPhrase; i++){ Fts5ExprTerm *pT; if( p->aPopulator[i].bOk==0 ) continue; for(pT=&pExpr->apExprPhrase[i]->aTerm[0]; pT; pT=pT->pSynonym){ if( (pT->nQueryTerm==nQuery || (pT->nQueryTerm<nQuery && pT->bPrefix)) && memcmp(pT->pTerm, pToken, pT->nQueryTerm)==0 ){ int rc = sqlite3Fts5PoslistWriterAppend( &pExpr->apExprPhrase[i]->poslist, &p->aPopulator[i].writer, p->iOff ); if( rc==SQLITE_OK && (pExpr->pConfig->bTokendata || pT->bPrefix) ){ int iCol = p->iOff>>32; int iTokOff = p->iOff & 0x7FFFFFFF; rc = sqlite3Fts5IndexIterWriteTokendata( pT->pIter, pToken, nToken, iRowid, iCol, iTokOff ); } if( rc ) return rc; break; } } } return SQLITE_OK; } |
︙ | ︙ | |||
2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 | } } static int fts5ExprCheckPoslists(Fts5ExprNode *pNode, i64 iRowid){ pNode->iRowid = iRowid; pNode->bEof = 0; switch( pNode->eType ){ case FTS5_TERM: case FTS5_STRING: return (pNode->pNear->apPhrase[0]->poslist.n>0); case FTS5_AND: { int i; for(i=0; i<pNode->nChild; i++){ | > | 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 | } } static int fts5ExprCheckPoslists(Fts5ExprNode *pNode, i64 iRowid){ pNode->iRowid = iRowid; pNode->bEof = 0; switch( pNode->eType ){ case 0: case FTS5_TERM: case FTS5_STRING: return (pNode->pNear->apPhrase[0]->poslist.n>0); case FTS5_AND: { int i; for(i=0; i<pNode->nChild; i++){ |
︙ | ︙ | |||
3065 3066 3067 3068 3069 3070 3071 | }else{ *ppCollist = 0; *pnCollist = 0; } return rc; } | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 | }else{ *ppCollist = 0; *pnCollist = 0; } return rc; } /* ** Does the work of the fts5_api.xQueryToken() API method. */ int sqlite3Fts5ExprQueryToken( Fts5Expr *pExpr, int iPhrase, int iToken, const char **ppOut, int *pnOut ){ Fts5ExprPhrase *pPhrase = 0; if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){ return SQLITE_RANGE; } pPhrase = pExpr->apExprPhrase[iPhrase]; if( iToken<0 || iToken>=pPhrase->nTerm ){ return SQLITE_RANGE; } *ppOut = pPhrase->aTerm[iToken].pTerm; *pnOut = pPhrase->aTerm[iToken].nFullTerm; return SQLITE_OK; } /* ** Does the work of the fts5_api.xInstToken() API method. */ int sqlite3Fts5ExprInstToken( Fts5Expr *pExpr, i64 iRowid, int iPhrase, int iCol, int iOff, int iToken, const char **ppOut, int *pnOut ){ Fts5ExprPhrase *pPhrase = 0; Fts5ExprTerm *pTerm = 0; int rc = SQLITE_OK; if( iPhrase<0 || iPhrase>=pExpr->nPhrase ){ return SQLITE_RANGE; } pPhrase = pExpr->apExprPhrase[iPhrase]; if( iToken<0 || iToken>=pPhrase->nTerm ){ return SQLITE_RANGE; } pTerm = &pPhrase->aTerm[iToken]; if( pExpr->pConfig->bTokendata || pTerm->bPrefix ){ rc = sqlite3Fts5IterToken( pTerm->pIter, pTerm->pTerm, pTerm->nQueryTerm, iRowid, iCol, iOff+iToken, ppOut, pnOut ); }else{ *ppOut = pTerm->pTerm; *pnOut = pTerm->nFullTerm; } return rc; } /* ** Clear the token mappings for all Fts5IndexIter objects managed by ** the expression passed as the only argument. */ void sqlite3Fts5ExprClearTokens(Fts5Expr *pExpr){ int ii; for(ii=0; ii<pExpr->nPhrase; ii++){ Fts5ExprTerm *pT; for(pT=&pExpr->apExprPhrase[ii]->aTerm[0]; pT; pT=pT->pSynonym){ sqlite3Fts5IndexIterClearTokendata(pT->pIter); } } } |
Changes to ext/fts5/fts5_hash.c.
︙ | ︙ | |||
16 17 18 19 20 21 22 | #include "fts5Int.h" typedef struct Fts5HashEntry Fts5HashEntry; /* ** This file contains the implementation of an in-memory hash table used | | | | | | > > > > > | 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 | #include "fts5Int.h" typedef struct Fts5HashEntry Fts5HashEntry; /* ** This file contains the implementation of an in-memory hash table used ** to accumulate "term -> doclist" content before it is flushed to a level-0 ** segment. */ struct Fts5Hash { int eDetail; /* Copy of Fts5Config.eDetail */ int *pnByte; /* Pointer to bytes counter */ int nEntry; /* Number of entries currently in hash */ int nSlot; /* Size of aSlot[] array */ Fts5HashEntry *pScan; /* Current ordered scan item */ Fts5HashEntry **aSlot; /* Array of hash slots */ }; /* ** Each entry in the hash table is represented by an object of the ** following type. Each object, its key, and its current data are stored ** in a single memory allocation. The key immediately follows the object ** in memory. The position list data immediately follows the key data ** in memory. ** ** The key is Fts5HashEntry.nKey bytes in size. It consists of a single ** byte identifying the index (either the main term index or a prefix-index), ** followed by the term data. For example: "0token". There is no ** nul-terminator - in this case nKey=6. ** ** The data that follows the key is in a similar, but not identical format ** to the doclist data stored in the database. It is: ** ** * Rowid, as a varint ** * Position list, without 0x00 terminator. ** * Size of previous position list and rowid, as a 4 byte |
︙ | ︙ | |||
68 69 70 71 72 73 74 | u8 bContent; /* Set content-flag (detail=none mode) */ i16 iCol; /* Column of last value written */ int iPos; /* Position of last value written */ i64 iRowid; /* Rowid of last value written */ }; /* | | | 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | u8 bContent; /* Set content-flag (detail=none mode) */ i16 iCol; /* Column of last value written */ int iPos; /* Position of last value written */ i64 iRowid; /* Rowid of last value written */ }; /* ** Equivalent to: ** ** char *fts5EntryKey(Fts5HashEntry *pEntry){ return zKey; } */ #define fts5EntryKey(p) ( ((char *)(&(p)[1])) ) /* |
︙ | ︙ | |||
170 171 172 173 174 175 176 | memset(apNew, 0, nNew*sizeof(Fts5HashEntry*)); for(i=0; i<pHash->nSlot; i++){ while( apOld[i] ){ unsigned int iHash; Fts5HashEntry *p = apOld[i]; apOld[i] = p->pHashNext; | | < | 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 | memset(apNew, 0, nNew*sizeof(Fts5HashEntry*)); for(i=0; i<pHash->nSlot; i++){ while( apOld[i] ){ unsigned int iHash; Fts5HashEntry *p = apOld[i]; apOld[i] = p->pHashNext; iHash = fts5HashKey(nNew, (u8*)fts5EntryKey(p), p->nKey); p->pHashNext = apNew[iHash]; apNew[iHash] = p; } } sqlite3_free(apOld); pHash->nSlot = nNew; |
︙ | ︙ | |||
255 256 257 258 259 260 261 | bNew = (pHash->eDetail==FTS5_DETAIL_FULL); /* Attempt to locate an existing hash entry */ iHash = fts5HashKey2(pHash->nSlot, (u8)bByte, (const u8*)pToken, nToken); for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ char *zKey = fts5EntryKey(p); if( zKey[0]==bByte | | | 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 | bNew = (pHash->eDetail==FTS5_DETAIL_FULL); /* Attempt to locate an existing hash entry */ iHash = fts5HashKey2(pHash->nSlot, (u8)bByte, (const u8*)pToken, nToken); for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ char *zKey = fts5EntryKey(p); if( zKey[0]==bByte && p->nKey==nToken+1 && memcmp(&zKey[1], pToken, nToken)==0 ){ break; } } /* If an existing hash entry cannot be found, create a new one. */ |
︙ | ︙ | |||
285 286 287 288 289 290 291 | if( !p ) return SQLITE_NOMEM; memset(p, 0, sizeof(Fts5HashEntry)); p->nAlloc = (int)nByte; zKey = fts5EntryKey(p); zKey[0] = bByte; memcpy(&zKey[1], pToken, nToken); assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) ); | | | | 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 | if( !p ) return SQLITE_NOMEM; memset(p, 0, sizeof(Fts5HashEntry)); p->nAlloc = (int)nByte; zKey = fts5EntryKey(p); zKey[0] = bByte; memcpy(&zKey[1], pToken, nToken); assert( iHash==fts5HashKey(pHash->nSlot, (u8*)zKey, nToken+1) ); p->nKey = nToken+1; zKey[nToken+1] = '\0'; p->nData = nToken+1 + sizeof(Fts5HashEntry); p->pHashNext = pHash->aSlot[iHash]; pHash->aSlot[iHash] = p; pHash->nEntry++; /* Add the first rowid field to the hash-entry */ p->nData += sqlite3Fts5PutVarint(&((u8*)p)[p->nData], iRowid); p->iRowid = iRowid; |
︙ | ︙ | |||
404 405 406 407 408 409 410 | if( p1==0 ){ *ppOut = p2; p2 = 0; }else if( p2==0 ){ *ppOut = p1; p1 = 0; }else{ | < > | > > > | > | > | | < < | 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 | if( p1==0 ){ *ppOut = p2; p2 = 0; }else if( p2==0 ){ *ppOut = p1; p1 = 0; }else{ char *zKey1 = fts5EntryKey(p1); char *zKey2 = fts5EntryKey(p2); int nMin = MIN(p1->nKey, p2->nKey); int cmp = memcmp(zKey1, zKey2, nMin); if( cmp==0 ){ cmp = p1->nKey - p2->nKey; } assert( cmp!=0 ); if( cmp>0 ){ /* p2 is smaller */ *ppOut = p2; ppOut = &p2->pScanNext; p2 = p2->pScanNext; }else{ /* p1 is smaller */ *ppOut = p1; ppOut = &p1->pScanNext; p1 = p1->pScanNext; } *ppOut = 0; } } return pRet; } /* ** Link all tokens from hash table iHash into a list in sorted order. The ** tokens are not removed from the hash table. */ static int fts5HashEntrySort( Fts5Hash *pHash, const char *pTerm, int nTerm, /* Query prefix, if any */ Fts5HashEntry **ppSorted ){ const int nMergeSlot = 32; |
︙ | ︙ | |||
453 454 455 456 457 458 459 | if( !ap ) return SQLITE_NOMEM; memset(ap, 0, sizeof(Fts5HashEntry*) * nMergeSlot); for(iSlot=0; iSlot<pHash->nSlot; iSlot++){ Fts5HashEntry *pIter; for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){ if( pTerm==0 | | < | 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 | if( !ap ) return SQLITE_NOMEM; memset(ap, 0, sizeof(Fts5HashEntry*) * nMergeSlot); for(iSlot=0; iSlot<pHash->nSlot; iSlot++){ Fts5HashEntry *pIter; for(pIter=pHash->aSlot[iSlot]; pIter; pIter=pIter->pHashNext){ if( pTerm==0 || (pIter->nKey>=nTerm && 0==memcmp(fts5EntryKey(pIter), pTerm, nTerm)) ){ Fts5HashEntry *pEntry = pIter; pEntry->pScanNext = 0; for(i=0; ap[i]; i++){ pEntry = fts5HashEntryMerge(pEntry, ap[i]); ap[i] = 0; } ap[i] = pEntry; } } } pList = 0; for(i=0; i<nMergeSlot; i++){ pList = fts5HashEntryMerge(pList, ap[i]); } sqlite3_free(ap); *ppSorted = pList; return SQLITE_OK; } /* ** Query the hash table for a doclist associated with term pTerm/nTerm. |
︙ | ︙ | |||
493 494 495 496 497 498 499 | ){ unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm); char *zKey = 0; Fts5HashEntry *p; for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ zKey = fts5EntryKey(p); | < | | | 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 | ){ unsigned int iHash = fts5HashKey(pHash->nSlot, (const u8*)pTerm, nTerm); char *zKey = 0; Fts5HashEntry *p; for(p=pHash->aSlot[iHash]; p; p=p->pHashNext){ zKey = fts5EntryKey(p); if( nTerm==p->nKey && memcmp(zKey, pTerm, nTerm)==0 ) break; } if( p ){ int nHashPre = sizeof(Fts5HashEntry) + nTerm; int nList = p->nData - nHashPre; u8 *pRet = (u8*)(*ppOut = sqlite3_malloc64(nPre + nList + 10)); if( pRet ){ Fts5HashEntry *pFaux = (Fts5HashEntry*)&pRet[nPre-nHashPre]; memcpy(&pRet[nPre], &((u8*)p)[nHashPre], nList); nList += fts5HashAddPoslistSize(pHash, p, pFaux); *pnDoclist = nList; |
︙ | ︙ | |||
524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 | int sqlite3Fts5HashScanInit( Fts5Hash *p, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ){ return fts5HashEntrySort(p, pTerm, nTerm, &p->pScan); } void sqlite3Fts5HashScanNext(Fts5Hash *p){ assert( !sqlite3Fts5HashScanEof(p) ); p->pScan = p->pScan->pScanNext; } int sqlite3Fts5HashScanEof(Fts5Hash *p){ return (p->pScan==0); } void sqlite3Fts5HashScanEntry( Fts5Hash *pHash, const char **pzTerm, /* OUT: term (nul-terminated) */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ){ Fts5HashEntry *p; if( (p = pHash->pScan) ){ char *zKey = fts5EntryKey(p); | > > > > > > > > > > > > > > > > > > > > > > > | > | | > | 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 | int sqlite3Fts5HashScanInit( Fts5Hash *p, /* Hash table to query */ const char *pTerm, int nTerm /* Query prefix */ ){ return fts5HashEntrySort(p, pTerm, nTerm, &p->pScan); } #ifdef SQLITE_DEBUG static int fts5HashCount(Fts5Hash *pHash){ int nEntry = 0; int ii; for(ii=0; ii<pHash->nSlot; ii++){ Fts5HashEntry *p = 0; for(p=pHash->aSlot[ii]; p; p=p->pHashNext){ nEntry++; } } return nEntry; } #endif /* ** Return true if the hash table is empty, false otherwise. */ int sqlite3Fts5HashIsEmpty(Fts5Hash *pHash){ assert( pHash->nEntry==fts5HashCount(pHash) ); return pHash->nEntry==0; } void sqlite3Fts5HashScanNext(Fts5Hash *p){ assert( !sqlite3Fts5HashScanEof(p) ); p->pScan = p->pScan->pScanNext; } int sqlite3Fts5HashScanEof(Fts5Hash *p){ return (p->pScan==0); } void sqlite3Fts5HashScanEntry( Fts5Hash *pHash, const char **pzTerm, /* OUT: term (nul-terminated) */ int *pnTerm, /* OUT: Size of term in bytes */ const u8 **ppDoclist, /* OUT: pointer to doclist */ int *pnDoclist /* OUT: size of doclist in bytes */ ){ Fts5HashEntry *p; if( (p = pHash->pScan) ){ char *zKey = fts5EntryKey(p); int nTerm = p->nKey; fts5HashAddPoslistSize(pHash, p, 0); *pzTerm = zKey; *pnTerm = nTerm; *ppDoclist = (const u8*)&zKey[nTerm]; *pnDoclist = p->nData - (sizeof(Fts5HashEntry) + nTerm); }else{ *pzTerm = 0; *pnTerm = 0; *ppDoclist = 0; *pnDoclist = 0; } } |
Changes to ext/fts5/fts5_index.c.
︙ | ︙ | |||
50 51 52 53 54 55 56 57 58 59 60 61 62 63 | #define FTS5_MAIN_PREFIX '0' #if FTS5_MAX_PREFIX_INDEXES > 31 # error "FTS5_MAX_PREFIX_INDEXES is too large" #endif /* ** Details: ** ** The %_data table managed by this module, ** ** CREATE TABLE %_data(id INTEGER PRIMARY KEY, block BLOB); ** | > > > > > > > > > > > > > > > > > > > > | | | < < | > > | > > > > > > | 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 | #define FTS5_MAIN_PREFIX '0' #if FTS5_MAX_PREFIX_INDEXES > 31 # error "FTS5_MAX_PREFIX_INDEXES is too large" #endif #define FTS5_MAX_LEVEL 64 /* ** There are two versions of the format used for the structure record: ** ** 1. the legacy format, that may be read by all fts5 versions, and ** ** 2. the V2 format, which is used by contentless_delete=1 databases. ** ** Both begin with a 4-byte "configuration cookie" value. Then, a legacy ** format structure record contains a varint - the number of levels in ** the structure. Whereas a V2 structure record contains the constant ** 4 bytes [0xff 0x00 0x00 0x01]. This is unambiguous as the value of a ** varint has to be at least 16256 to begin with "0xFF". And the default ** maximum number of levels is 64. ** ** See below for more on structure record formats. */ #define FTS5_STRUCTURE_V2 "\xFF\x00\x00\x01" /* ** Details: ** ** The %_data table managed by this module, ** ** CREATE TABLE %_data(id INTEGER PRIMARY KEY, block BLOB); ** ** , contains the following 6 types of records. See the comments surrounding ** the FTS5_*_ROWID macros below for a description of how %_data rowids are ** assigned to each fo them. ** ** 1. Structure Records: ** ** The set of segments that make up an index - the index structure - are ** recorded in a single record within the %_data table. The record consists ** of a single 32-bit configuration cookie value followed by a list of ** SQLite varints. ** ** If the structure record is a V2 record, the configuration cookie is ** followed by the following 4 bytes: [0xFF 0x00 0x00 0x01]. ** ** Next, the record continues with three varints: ** ** + number of levels, ** + total number of segments on all levels, ** + value of write counter. ** ** Then, for each level from 0 to nMax: ** ** + number of input segments in ongoing merge. ** + total number of segments in level. ** + for each segment from oldest to newest: ** + segment id (always > 0) ** + first leaf page number (often 1, always greater than 0) ** + final leaf page number ** ** Then, for V2 structures only: ** ** + lower origin counter value, ** + upper origin counter value, ** + the number of tombstone hash pages. ** ** 2. The Averages Record: ** ** A single record within the %_data table. The data is a list of varints. ** The first value is the number of rows in the index. Then, for each column ** from left to right, the total number of tokens in the column for all ** rows of the table. ** |
︙ | ︙ | |||
201 202 203 204 205 206 207 208 209 210 211 212 213 214 | ** * Page number of first child page. As a varint. ** ** * Copy of first rowid on page indicated by previous field. As a varint. ** ** * A list of delta-encoded varints - the first rowid on each subsequent ** child page. ** */ /* ** Rowids for the averages and structure records in the %_data table. */ #define FTS5_AVERAGES_ROWID 1 /* Rowid used for the averages record */ #define FTS5_STRUCTURE_ROWID 10 /* The structure record */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 | ** * Page number of first child page. As a varint. ** ** * Copy of first rowid on page indicated by previous field. As a varint. ** ** * A list of delta-encoded varints - the first rowid on each subsequent ** child page. ** ** 6. Tombstone Hash Page ** ** These records are only ever present in contentless_delete=1 tables. ** There are zero or more of these associated with each segment. They ** are used to store the tombstone rowids for rows contained in the ** associated segments. ** ** The set of nHashPg tombstone hash pages associated with a single ** segment together form a single hash table containing tombstone rowids. ** To find the page of the hash on which a key might be stored: ** ** iPg = (rowid % nHashPg) ** ** Then, within page iPg, which has nSlot slots: ** ** iSlot = (rowid / nHashPg) % nSlot ** ** Each tombstone hash page begins with an 8 byte header: ** ** 1-byte: Key-size (the size in bytes of each slot). Either 4 or 8. ** 1-byte: rowid-0-tombstone flag. This flag is only valid on the ** first tombstone hash page for each segment (iPg=0). If set, ** the hash table contains rowid 0. If clear, it does not. ** Rowid 0 is handled specially. ** 2-bytes: unused. ** 4-bytes: Big-endian integer containing number of entries on page. ** ** Following this are nSlot 4 or 8 byte slots (depending on the key-size ** in the first byte of the page header). The number of slots may be ** determined based on the size of the page record and the key-size: ** ** nSlot = (nByte - 8) / key-size */ /* ** Rowids for the averages and structure records in the %_data table. */ #define FTS5_AVERAGES_ROWID 1 /* Rowid used for the averages record */ #define FTS5_STRUCTURE_ROWID 10 /* The structure record */ |
︙ | ︙ | |||
234 235 236 237 238 239 240 241 242 243 244 245 246 247 | ((i64)(dlidx) << (FTS5_DATA_PAGE_B + FTS5_DATA_HEIGHT_B)) + \ ((i64)(height) << (FTS5_DATA_PAGE_B)) + \ ((i64)(pgno)) \ ) #define FTS5_SEGMENT_ROWID(segid, pgno) fts5_dri(segid, 0, 0, pgno) #define FTS5_DLIDX_ROWID(segid, height, pgno) fts5_dri(segid, 1, height, pgno) #ifdef SQLITE_DEBUG int sqlite3Fts5Corrupt() { return SQLITE_CORRUPT_VTAB; } #endif /* | > | 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 | ((i64)(dlidx) << (FTS5_DATA_PAGE_B + FTS5_DATA_HEIGHT_B)) + \ ((i64)(height) << (FTS5_DATA_PAGE_B)) + \ ((i64)(pgno)) \ ) #define FTS5_SEGMENT_ROWID(segid, pgno) fts5_dri(segid, 0, 0, pgno) #define FTS5_DLIDX_ROWID(segid, height, pgno) fts5_dri(segid, 1, height, pgno) #define FTS5_TOMBSTONE_ROWID(segid,ipg) fts5_dri(segid+(1<<16), 0, 0, ipg) #ifdef SQLITE_DEBUG int sqlite3Fts5Corrupt() { return SQLITE_CORRUPT_VTAB; } #endif /* |
︙ | ︙ | |||
260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 | typedef struct Fts5PageWriter Fts5PageWriter; typedef struct Fts5SegIter Fts5SegIter; typedef struct Fts5DoclistIter Fts5DoclistIter; typedef struct Fts5SegWriter Fts5SegWriter; typedef struct Fts5Structure Fts5Structure; typedef struct Fts5StructureLevel Fts5StructureLevel; typedef struct Fts5StructureSegment Fts5StructureSegment; struct Fts5Data { u8 *p; /* Pointer to buffer containing record */ int nn; /* Size of record in bytes */ int szLeaf; /* Size of leaf without page-index */ }; /* ** One object per %_data table. */ struct Fts5Index { Fts5Config *pConfig; /* Virtual table configuration */ char *zDataTbl; /* Name of %_data table */ int nWorkUnit; /* Leaf pages in a "unit" of work */ /* ** Variables related to the accumulation of tokens and doclists within the ** in-memory hash tables before they are flushed to disk. */ Fts5Hash *pHash; /* Hash table for in-memory data */ int nPendingData; /* Current bytes of pending data */ i64 iWriteRowid; /* Rowid for current doc being written */ int bDelete; /* Current write is a delete */ /* Error state. */ int rc; /* Current error code */ /* State used by the fts5DataXXX() functions. */ sqlite3_blob *pReader; /* RO incr-blob open on %_data table */ sqlite3_stmt *pWriter; /* "INSERT ... %_data VALUES(?,?)" */ sqlite3_stmt *pDeleter; /* "DELETE FROM %_data ... id>=? AND id<=?" */ sqlite3_stmt *pIdxWriter; /* "INSERT ... %_idx VALUES(?,?,?,?)" */ sqlite3_stmt *pIdxDeleter; /* "DELETE FROM %_idx WHERE segid=?" */ sqlite3_stmt *pIdxSelect; int nRead; /* Total number of blocks read */ sqlite3_stmt *pDataVersion; i64 iStructVersion; /* data_version when pStruct read */ Fts5Structure *pStruct; /* Current db structure (or NULL) */ }; struct Fts5DoclistIter { u8 *aEof; /* Pointer to 1 byte past end of doclist */ /* Output variables. aPoslist==0 at EOF */ i64 iRowid; u8 *aPoslist; int nPoslist; int nSize; }; /* ** The contents of the "structure" record for each index are represented ** using an Fts5Structure record in memory. Which uses instances of the ** other Fts5StructureXXX types as components. */ struct Fts5StructureSegment { int iSegid; /* Segment id */ int pgnoFirst; /* First leaf page number in segment */ int pgnoLast; /* Last leaf page number in segment */ }; struct Fts5StructureLevel { int nMerge; /* Number of segments in incr-merge */ int nSeg; /* Total number of segments on level */ Fts5StructureSegment *aSeg; /* Array of segments. aSeg[0] is oldest. */ }; struct Fts5Structure { int nRef; /* Object reference count */ u64 nWriteCounter; /* Total leaves written to level 0 */ int nSegment; /* Total segments in this structure */ int nLevel; /* Number of levels in this index */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > | 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 | typedef struct Fts5PageWriter Fts5PageWriter; typedef struct Fts5SegIter Fts5SegIter; typedef struct Fts5DoclistIter Fts5DoclistIter; typedef struct Fts5SegWriter Fts5SegWriter; typedef struct Fts5Structure Fts5Structure; typedef struct Fts5StructureLevel Fts5StructureLevel; typedef struct Fts5StructureSegment Fts5StructureSegment; typedef struct Fts5TokenDataIter Fts5TokenDataIter; typedef struct Fts5TokenDataMap Fts5TokenDataMap; typedef struct Fts5TombstoneArray Fts5TombstoneArray; struct Fts5Data { u8 *p; /* Pointer to buffer containing record */ int nn; /* Size of record in bytes */ int szLeaf; /* Size of leaf without page-index */ }; /* ** One object per %_data table. ** ** nContentlessDelete: ** The number of contentless delete operations since the most recent ** call to fts5IndexFlush() or fts5IndexDiscardData(). This is tracked ** so that extra auto-merge work can be done by fts5IndexFlush() to ** account for the delete operations. */ struct Fts5Index { Fts5Config *pConfig; /* Virtual table configuration */ char *zDataTbl; /* Name of %_data table */ int nWorkUnit; /* Leaf pages in a "unit" of work */ /* ** Variables related to the accumulation of tokens and doclists within the ** in-memory hash tables before they are flushed to disk. */ Fts5Hash *pHash; /* Hash table for in-memory data */ int nPendingData; /* Current bytes of pending data */ i64 iWriteRowid; /* Rowid for current doc being written */ int bDelete; /* Current write is a delete */ int nContentlessDelete; /* Number of contentless delete ops */ int nPendingRow; /* Number of INSERT in hash table */ /* Error state. */ int rc; /* Current error code */ int flushRc; /* State used by the fts5DataXXX() functions. */ sqlite3_blob *pReader; /* RO incr-blob open on %_data table */ sqlite3_stmt *pWriter; /* "INSERT ... %_data VALUES(?,?)" */ sqlite3_stmt *pDeleter; /* "DELETE FROM %_data ... id>=? AND id<=?" */ sqlite3_stmt *pIdxWriter; /* "INSERT ... %_idx VALUES(?,?,?,?)" */ sqlite3_stmt *pIdxDeleter; /* "DELETE FROM %_idx WHERE segid=?" */ sqlite3_stmt *pIdxSelect; sqlite3_stmt *pIdxNextSelect; int nRead; /* Total number of blocks read */ sqlite3_stmt *pDeleteFromIdx; sqlite3_stmt *pDataVersion; i64 iStructVersion; /* data_version when pStruct read */ Fts5Structure *pStruct; /* Current db structure (or NULL) */ }; struct Fts5DoclistIter { u8 *aEof; /* Pointer to 1 byte past end of doclist */ /* Output variables. aPoslist==0 at EOF */ i64 iRowid; u8 *aPoslist; int nPoslist; int nSize; }; /* ** The contents of the "structure" record for each index are represented ** using an Fts5Structure record in memory. Which uses instances of the ** other Fts5StructureXXX types as components. ** ** nOriginCntr: ** This value is set to non-zero for structure records created for ** contentlessdelete=1 tables only. In that case it represents the ** origin value to apply to the next top-level segment created. */ struct Fts5StructureSegment { int iSegid; /* Segment id */ int pgnoFirst; /* First leaf page number in segment */ int pgnoLast; /* Last leaf page number in segment */ /* contentlessdelete=1 tables only: */ u64 iOrigin1; u64 iOrigin2; int nPgTombstone; /* Number of tombstone hash table pages */ u64 nEntryTombstone; /* Number of tombstone entries that "count" */ u64 nEntry; /* Number of rows in this segment */ }; struct Fts5StructureLevel { int nMerge; /* Number of segments in incr-merge */ int nSeg; /* Total number of segments on level */ Fts5StructureSegment *aSeg; /* Array of segments. aSeg[0] is oldest. */ }; struct Fts5Structure { int nRef; /* Object reference count */ u64 nWriteCounter; /* Total leaves written to level 0 */ u64 nOriginCntr; /* Origin value for next top-level segment */ int nSegment; /* Total segments in this structure */ int nLevel; /* Number of levels in this index */ Fts5StructureLevel aLevel[FLEXARRAY]; /* Array of nLevel level objects */ }; /* Size (in bytes) of an Fts5Structure object holding up to N levels */ #define SZ_FTS5STRUCTURE(N) \ (offsetof(Fts5Structure,aLevel) + (N)*sizeof(Fts5StructureLevel)) /* ** An object of type Fts5SegWriter is used to write to segments. */ struct Fts5PageWriter { int pgno; /* Page number for this page */ int iPrevPgidx; /* Previous value written into pgidx */ Fts5Buffer buf; /* Buffer containing leaf data */ |
︙ | ︙ | |||
388 389 390 391 392 393 394 | ** ** iLeafPgno: ** Current leaf page number within segment. ** ** iLeafOffset: ** Byte offset within the current leaf that is the first byte of the ** position list data (one byte passed the position-list size field). | < < < | 479 480 481 482 483 484 485 486 487 488 489 490 491 492 | ** ** iLeafPgno: ** Current leaf page number within segment. ** ** iLeafOffset: ** Byte offset within the current leaf that is the first byte of the ** position list data (one byte passed the position-list size field). ** ** pLeaf: ** Buffer containing current leaf page data. Set to NULL at EOF. ** ** iTermLeafPgno, iTermLeafOffset: ** Leaf page number containing the last term read from the segment. And ** the offset immediately following the term data. |
︙ | ︙ | |||
420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 | ** ** For each rowid on the page corresponding to the current term, the ** corresponding aRowidOffset[] entry is set to the byte offset of the ** start of the "position-list-size" field within the page. ** ** iTermIdx: ** Index of current term on iTermLeafPgno. */ struct Fts5SegIter { Fts5StructureSegment *pSeg; /* Segment to iterate through */ int flags; /* Mask of configuration flags */ int iLeafPgno; /* Current leaf page number */ Fts5Data *pLeaf; /* Current leaf data */ Fts5Data *pNextLeaf; /* Leaf page (iLeafPgno+1) */ i64 iLeafOffset; /* Byte offset within current leaf */ /* Next method */ void (*xNext)(Fts5Index*, Fts5SegIter*, int*); /* The page and offset from which the current term was read. The offset ** is the offset of the first rowid in the current doclist. */ int iTermLeafPgno; | > > > > > > > > | 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 | ** ** For each rowid on the page corresponding to the current term, the ** corresponding aRowidOffset[] entry is set to the byte offset of the ** start of the "position-list-size" field within the page. ** ** iTermIdx: ** Index of current term on iTermLeafPgno. ** ** apTombstone/nTombstone: ** These are used for contentless_delete=1 tables only. When the cursor ** is first allocated, the apTombstone[] array is allocated so that it ** is large enough for all tombstones hash pages associated with the ** segment. The pages themselves are loaded lazily from the database as ** they are required. */ struct Fts5SegIter { Fts5StructureSegment *pSeg; /* Segment to iterate through */ int flags; /* Mask of configuration flags */ int iLeafPgno; /* Current leaf page number */ Fts5Data *pLeaf; /* Current leaf data */ Fts5Data *pNextLeaf; /* Leaf page (iLeafPgno+1) */ i64 iLeafOffset; /* Byte offset within current leaf */ Fts5TombstoneArray *pTombArray; /* Array of tombstone pages */ /* Next method */ void (*xNext)(Fts5Index*, Fts5SegIter*, int*); /* The page and offset from which the current term was read. The offset ** is the offset of the first rowid in the current doclist. */ int iTermLeafPgno; |
︙ | ︙ | |||
454 455 456 457 458 459 460 461 462 463 464 465 466 467 | /* Variables populated based on current entry. */ Fts5Buffer term; /* Current term */ i64 iRowid; /* Current rowid */ int nPos; /* Number of bytes in current position list */ u8 bDel; /* True if the delete flag is set */ }; /* ** Argument is a pointer to an Fts5Data structure that contains a ** leaf page. */ #define ASSERT_SZLEAF_OK(x) assert( \ (x)->szLeaf==(x)->nn || (x)->szLeaf==fts5GetU16(&(x)->p[2]) \ ) | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 | /* Variables populated based on current entry. */ Fts5Buffer term; /* Current term */ i64 iRowid; /* Current rowid */ int nPos; /* Number of bytes in current position list */ u8 bDel; /* True if the delete flag is set */ }; static int fts5IndexCorruptRowid(Fts5Index *pIdx, i64 iRowid){ pIdx->rc = FTS5_CORRUPT; sqlite3Fts5ConfigErrmsg(pIdx->pConfig, "fts5: corruption found reading blob %lld from table \"%s\"", iRowid, pIdx->pConfig->zName ); return SQLITE_CORRUPT_VTAB; } #define FTS5_CORRUPT_ROWID(pIdx, iRowid) fts5IndexCorruptRowid(pIdx, iRowid) static int fts5IndexCorruptIter(Fts5Index *pIdx, Fts5SegIter *pIter){ pIdx->rc = FTS5_CORRUPT; sqlite3Fts5ConfigErrmsg(pIdx->pConfig, "fts5: corruption on page %d, segment %d, table \"%s\"", pIter->iLeafPgno, pIter->pSeg->iSegid, pIdx->pConfig->zName ); return SQLITE_CORRUPT_VTAB; } #define FTS5_CORRUPT_ITER(pIdx, pIter) fts5IndexCorruptIter(pIdx, pIter) static int fts5IndexCorruptIdx(Fts5Index *pIdx){ pIdx->rc = FTS5_CORRUPT; sqlite3Fts5ConfigErrmsg(pIdx->pConfig, "fts5: corruption in table \"%s\"", pIdx->pConfig->zName ); return SQLITE_CORRUPT_VTAB; } #define FTS5_CORRUPT_IDX(pIdx) fts5IndexCorruptIdx(pIdx) /* ** Array of tombstone pages. Reference counted. */ struct Fts5TombstoneArray { int nRef; /* Number of pointers to this object */ int nTombstone; Fts5Data *apTombstone[FLEXARRAY]; /* Array of tombstone pages */ }; /* Size (in bytes) of an Fts5TombstoneArray holding up to N tombstones */ #define SZ_FTS5TOMBSTONEARRAY(N) \ (offsetof(Fts5TombstoneArray,apTombstone)+(N)*sizeof(Fts5Data*)) /* ** Argument is a pointer to an Fts5Data structure that contains a ** leaf page. */ #define ASSERT_SZLEAF_OK(x) assert( \ (x)->szLeaf==(x)->nn || (x)->szLeaf==fts5GetU16(&(x)->p[2]) \ ) |
︙ | ︙ | |||
498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 | ** ** aFirst[1] contains the index in aSeg[] of the iterator that points to ** the smallest key overall. aFirst[0] is unused. ** ** poslist: ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. */ struct Fts5Iter { Fts5IndexIter base; /* Base class containing output vars */ Fts5Index *pIndex; /* Index that owns this iterator */ Fts5Buffer poslist; /* Buffer containing current poslist */ Fts5Colset *pColset; /* Restrict matches to these columns */ /* Invoked to set output variables. */ void (*xSetOutputs)(Fts5Iter*, Fts5SegIter*); int nSeg; /* Size of aSeg[] array */ int bRev; /* True to iterate in reverse order */ u8 bSkipEmpty; /* True to skip deleted entries */ i64 iSwitchRowid; /* Firstest rowid of other than aFirst[1] */ Fts5CResult *aFirst; /* Current merge state (see above) */ | > > > > > > > | > > | 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 | ** ** aFirst[1] contains the index in aSeg[] of the iterator that points to ** the smallest key overall. aFirst[0] is unused. ** ** poslist: ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. ** ** pColset: ** If not NULL, points to an object containing a set of column indices. ** Only matches that occur in one of these columns will be returned. ** The Fts5Iter does not own the Fts5Colset object, and so it is not ** freed when the iterator is closed - it is owned by the upper layer. */ struct Fts5Iter { Fts5IndexIter base; /* Base class containing output vars */ Fts5TokenDataIter *pTokenDataIter; Fts5Index *pIndex; /* Index that owns this iterator */ Fts5Buffer poslist; /* Buffer containing current poslist */ Fts5Colset *pColset; /* Restrict matches to these columns */ /* Invoked to set output variables. */ void (*xSetOutputs)(Fts5Iter*, Fts5SegIter*); int nSeg; /* Size of aSeg[] array */ int bRev; /* True to iterate in reverse order */ u8 bSkipEmpty; /* True to skip deleted entries */ i64 iSwitchRowid; /* Firstest rowid of other than aFirst[1] */ Fts5CResult *aFirst; /* Current merge state (see above) */ Fts5SegIter aSeg[FLEXARRAY]; /* Array of segment iterators */ }; /* Size (in bytes) of an Fts5Iter object holding up to N segment iterators */ #define SZ_FTS5ITER(N) (offsetof(Fts5Iter,aSeg)+(N)*sizeof(Fts5SegIter)) /* ** An instance of the following type is used to iterate through the contents ** of a doclist-index record. ** ** pData: ** Record containing the doclist-index data. |
︙ | ︙ | |||
545 546 547 548 549 550 551 | /* Output variables */ int iLeafPgno; /* Page number of current leaf page */ i64 iRowid; /* First rowid on leaf iLeafPgno */ }; struct Fts5DlidxIter { int nLvl; int iSegid; | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 | /* Output variables */ int iLeafPgno; /* Page number of current leaf page */ i64 iRowid; /* First rowid on leaf iLeafPgno */ }; struct Fts5DlidxIter { int nLvl; int iSegid; Fts5DlidxLvl aLvl[FLEXARRAY]; }; /* Size (in bytes) of an Fts5DlidxIter object with up to N levels */ #define SZ_FTS5DLIDXITER(N) \ (offsetof(Fts5DlidxIter,aLvl)+(N)*sizeof(Fts5DlidxLvl)) static void fts5PutU16(u8 *aOut, u16 iVal){ aOut[0] = (iVal>>8); aOut[1] = (iVal&0xFF); } static u16 fts5GetU16(const u8 *aIn){ return ((u16)aIn[0] << 8) + aIn[1]; } /* ** The only argument points to a buffer at least 8 bytes in size. This ** function interprets the first 8 bytes of the buffer as a 64-bit big-endian ** unsigned integer and returns the result. */ static u64 fts5GetU64(u8 *a){ return ((u64)a[0] << 56) + ((u64)a[1] << 48) + ((u64)a[2] << 40) + ((u64)a[3] << 32) + ((u64)a[4] << 24) + ((u64)a[5] << 16) + ((u64)a[6] << 8) + ((u64)a[7] << 0); } /* ** The only argument points to a buffer at least 4 bytes in size. This ** function interprets the first 4 bytes of the buffer as a 32-bit big-endian ** unsigned integer and returns the result. */ static u32 fts5GetU32(const u8 *a){ return ((u32)a[0] << 24) + ((u32)a[1] << 16) + ((u32)a[2] << 8) + ((u32)a[3] << 0); } /* ** Write iVal, formated as a 64-bit big-endian unsigned integer, to the ** buffer indicated by the first argument. */ static void fts5PutU64(u8 *a, u64 iVal){ a[0] = ((iVal >> 56) & 0xFF); a[1] = ((iVal >> 48) & 0xFF); a[2] = ((iVal >> 40) & 0xFF); a[3] = ((iVal >> 32) & 0xFF); a[4] = ((iVal >> 24) & 0xFF); a[5] = ((iVal >> 16) & 0xFF); a[6] = ((iVal >> 8) & 0xFF); a[7] = ((iVal >> 0) & 0xFF); } /* ** Write iVal, formated as a 32-bit big-endian unsigned integer, to the ** buffer indicated by the first argument. */ static void fts5PutU32(u8 *a, u32 iVal){ a[0] = ((iVal >> 24) & 0xFF); a[1] = ((iVal >> 16) & 0xFF); a[2] = ((iVal >> 8) & 0xFF); a[3] = ((iVal >> 0) & 0xFF); } /* ** Allocate and return a buffer at least nByte bytes in size. ** ** If an OOM error is encountered, return NULL and set the error code in ** the Fts5Index handle passed as the first argument. */ |
︙ | ︙ | |||
613 614 615 616 617 618 619 | fts5GetVarint32(&pLeaf->p[pLeaf->szLeaf], ret); return ret; } /* ** Close the read-only blob handle, if it is open. */ | | > | > | 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 | fts5GetVarint32(&pLeaf->p[pLeaf->szLeaf], ret); return ret; } /* ** Close the read-only blob handle, if it is open. */ static void fts5IndexCloseReader(Fts5Index *p){ if( p->pReader ){ int rc; sqlite3_blob *pReader = p->pReader; p->pReader = 0; rc = sqlite3_blob_close(pReader); if( p->rc==SQLITE_OK ) p->rc = rc; } } /* ** Retrieve a record from the %_data table. ** ** If an error occurs, NULL is returned and an error left in the |
︙ | ︙ | |||
642 643 644 645 646 647 648 | ** is required. */ sqlite3_blob *pBlob = p->pReader; p->pReader = 0; rc = sqlite3_blob_reopen(pBlob, iRowid); assert( p->pReader==0 ); p->pReader = pBlob; if( rc!=SQLITE_OK ){ | | | > | | | 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 | ** is required. */ sqlite3_blob *pBlob = p->pReader; p->pReader = 0; rc = sqlite3_blob_reopen(pBlob, iRowid); assert( p->pReader==0 ); p->pReader = pBlob; if( rc!=SQLITE_OK ){ fts5IndexCloseReader(p); } if( rc==SQLITE_ABORT ) rc = SQLITE_OK; } /* If the blob handle is not open at this point, open it and seek ** to the requested entry. */ if( p->pReader==0 && rc==SQLITE_OK ){ Fts5Config *pConfig = p->pConfig; rc = sqlite3_blob_open(pConfig->db, pConfig->zDb, p->zDataTbl, "block", iRowid, 0, &p->pReader ); } /* If either of the sqlite3_blob_open() or sqlite3_blob_reopen() calls ** above returned SQLITE_ERROR, return SQLITE_CORRUPT_VTAB instead. ** All the reasons those functions might return SQLITE_ERROR - missing ** table, missing row, non-blob/text in block column - indicate ** backing store corruption. */ if( rc==SQLITE_ERROR ) rc = FTS5_CORRUPT_ROWID(p, iRowid); if( rc==SQLITE_OK ){ u8 *aOut = 0; /* Read blob data into this buffer */ int nByte = sqlite3_blob_bytes(p->pReader); int szData = (sizeof(Fts5Data) + 7) & ~7; sqlite3_int64 nAlloc = szData + nByte + FTS5_DATA_PADDING; pRet = (Fts5Data*)sqlite3_malloc64(nAlloc); if( pRet ){ pRet->nn = nByte; aOut = pRet->p = (u8*)pRet + szData; }else{ rc = SQLITE_NOMEM; } if( rc==SQLITE_OK ){ rc = sqlite3_blob_read(p->pReader, aOut, nByte, 0); } |
︙ | ︙ | |||
693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 | } } p->rc = rc; p->nRead++; } assert( (pRet==0)==(p->rc!=SQLITE_OK) ); return pRet; } /* ** Release a reference to data record returned by an earlier call to ** fts5DataRead(). */ static void fts5DataRelease(Fts5Data *pData){ sqlite3_free(pData); } static Fts5Data *fts5LeafRead(Fts5Index *p, i64 iRowid){ Fts5Data *pRet = fts5DataRead(p, iRowid); if( pRet ){ if( pRet->nn<4 || pRet->szLeaf>pRet->nn ){ | > | | > > > > | 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 | } } p->rc = rc; p->nRead++; } assert( (pRet==0)==(p->rc!=SQLITE_OK) ); assert( pRet==0 || EIGHT_BYTE_ALIGNMENT( pRet->p ) ); return pRet; } /* ** Release a reference to data record returned by an earlier call to ** fts5DataRead(). */ static void fts5DataRelease(Fts5Data *pData){ sqlite3_free(pData); } static Fts5Data *fts5LeafRead(Fts5Index *p, i64 iRowid){ Fts5Data *pRet = fts5DataRead(p, iRowid); if( pRet ){ if( pRet->nn<4 || pRet->szLeaf>pRet->nn ){ FTS5_CORRUPT_ROWID(p, iRowid); fts5DataRelease(pRet); pRet = 0; } } return pRet; } static int fts5IndexPrepareStmt( Fts5Index *p, sqlite3_stmt **ppStmt, char *zSql ){ if( p->rc==SQLITE_OK ){ if( zSql ){ int rc = sqlite3_prepare_v3(p->pConfig->db, zSql, -1, SQLITE_PREPARE_PERSISTENT|SQLITE_PREPARE_NO_VTAB, ppStmt, 0); /* If this prepare() call fails with SQLITE_ERROR, then one of the ** %_idx or %_data tables has been removed or modified. Call this ** corruption. */ p->rc = (rc==SQLITE_ERROR ? SQLITE_CORRUPT : rc); }else{ p->rc = SQLITE_NOMEM; } } sqlite3_free(zSql); return p->rc; } |
︙ | ︙ | |||
784 785 786 787 788 789 790 | sqlite3_step(p->pDeleter); p->rc = sqlite3_reset(p->pDeleter); } /* ** Remove all records associated with segment iSegid. */ | | > > > > > > > | 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 | sqlite3_step(p->pDeleter); p->rc = sqlite3_reset(p->pDeleter); } /* ** Remove all records associated with segment iSegid. */ static void fts5DataRemoveSegment(Fts5Index *p, Fts5StructureSegment *pSeg){ int iSegid = pSeg->iSegid; i64 iFirst = FTS5_SEGMENT_ROWID(iSegid, 0); i64 iLast = FTS5_SEGMENT_ROWID(iSegid+1, 0)-1; fts5DataDelete(p, iFirst, iLast); if( pSeg->nPgTombstone ){ i64 iTomb1 = FTS5_TOMBSTONE_ROWID(iSegid, 0); i64 iTomb2 = FTS5_TOMBSTONE_ROWID(iSegid, pSeg->nPgTombstone-1); fts5DataDelete(p, iTomb1, iTomb2); } if( p->pIdxDeleter==0 ){ Fts5Config *pConfig = p->pConfig; fts5IndexPrepareStmt(p, &p->pIdxDeleter, sqlite3_mprintf( "DELETE FROM '%q'.'%q_idx' WHERE segid=?", pConfig->zDb, pConfig->zName )); } |
︙ | ︙ | |||
846 847 848 849 850 851 852 | ** ** This function is a no-op if (*pRc) is not SQLITE_OK when it is called. If ** an error occurs, (*pRc) is set to an SQLite error code before returning. */ static void fts5StructureMakeWritable(int *pRc, Fts5Structure **pp){ Fts5Structure *p = *pp; if( *pRc==SQLITE_OK && p->nRef>1 ){ | | | 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 | ** ** This function is a no-op if (*pRc) is not SQLITE_OK when it is called. If ** an error occurs, (*pRc) is set to an SQLite error code before returning. */ static void fts5StructureMakeWritable(int *pRc, Fts5Structure **pp){ Fts5Structure *p = *pp; if( *pRc==SQLITE_OK && p->nRef>1 ){ i64 nByte = SZ_FTS5STRUCTURE(p->nLevel); Fts5Structure *pNew; pNew = (Fts5Structure*)sqlite3Fts5MallocZero(pRc, nByte); if( pNew ){ int i; memcpy(pNew, p, nByte); for(i=0; i<p->nLevel; i++) pNew->aLevel[i].aSeg = 0; for(i=0; i<p->nLevel; i++){ |
︙ | ︙ | |||
898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 | int rc = SQLITE_OK; int i = 0; int iLvl; int nLevel = 0; int nSegment = 0; sqlite3_int64 nByte; /* Bytes of space to allocate at pRet */ Fts5Structure *pRet = 0; /* Structure object to return */ /* Grab the cookie value */ if( piCookie ) *piCookie = sqlite3Fts5Get32(pData); i = 4; /* Read the total number of levels and segments from the start of the ** structure record. */ i += fts5GetVarint32(&pData[i], nLevel); i += fts5GetVarint32(&pData[i], nSegment); if( nLevel>FTS5_MAX_SEGMENT || nLevel<0 || nSegment>FTS5_MAX_SEGMENT || nSegment<0 ){ return FTS5_CORRUPT; } | > > > > > > > > | < < < | 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 | int rc = SQLITE_OK; int i = 0; int iLvl; int nLevel = 0; int nSegment = 0; sqlite3_int64 nByte; /* Bytes of space to allocate at pRet */ Fts5Structure *pRet = 0; /* Structure object to return */ int bStructureV2 = 0; /* True for FTS5_STRUCTURE_V2 */ u64 nOriginCntr = 0; /* Largest origin value seen so far */ /* Grab the cookie value */ if( piCookie ) *piCookie = sqlite3Fts5Get32(pData); i = 4; /* Check if this is a V2 structure record. Set bStructureV2 if it is. */ if( 0==memcmp(&pData[i], FTS5_STRUCTURE_V2, 4) ){ i += 4; bStructureV2 = 1; } /* Read the total number of levels and segments from the start of the ** structure record. */ i += fts5GetVarint32(&pData[i], nLevel); i += fts5GetVarint32(&pData[i], nSegment); if( nLevel>FTS5_MAX_SEGMENT || nLevel<0 || nSegment>FTS5_MAX_SEGMENT || nSegment<0 ){ return FTS5_CORRUPT; } nByte = SZ_FTS5STRUCTURE(nLevel); pRet = (Fts5Structure*)sqlite3Fts5MallocZero(&rc, nByte); if( pRet ){ pRet->nRef = 1; pRet->nLevel = nLevel; pRet->nSegment = nSegment; i += sqlite3Fts5GetVarint(&pData[i], &pRet->nWriteCounter); |
︙ | ︙ | |||
949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 | pLvl->nSeg = nTotal; for(iSeg=0; iSeg<nTotal; iSeg++){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; if( i>=nData ){ rc = FTS5_CORRUPT; break; } i += fts5GetVarint32(&pData[i], pSeg->iSegid); i += fts5GetVarint32(&pData[i], pSeg->pgnoFirst); i += fts5GetVarint32(&pData[i], pSeg->pgnoLast); if( pSeg->pgnoLast<pSeg->pgnoFirst ){ rc = FTS5_CORRUPT; break; } } if( iLvl>0 && pLvl[-1].nMerge && nTotal==0 ) rc = FTS5_CORRUPT; if( iLvl==nLevel-1 && pLvl->nMerge ) rc = FTS5_CORRUPT; } } if( nSegment!=0 && rc==SQLITE_OK ) rc = FTS5_CORRUPT; if( rc!=SQLITE_OK ){ fts5StructureRelease(pRet); pRet = 0; } } *ppOut = pRet; return rc; } /* ** Add a level to the Fts5Structure.aLevel[] array of structure object ** (*ppStruct). */ static void fts5StructureAddLevel(int *pRc, Fts5Structure **ppStruct){ fts5StructureMakeWritable(pRc, ppStruct); if( *pRc==SQLITE_OK ){ Fts5Structure *pStruct = *ppStruct; int nLevel = pStruct->nLevel; | > > > > > > > > > > > > > | < < < | 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 | pLvl->nSeg = nTotal; for(iSeg=0; iSeg<nTotal; iSeg++){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; if( i>=nData ){ rc = FTS5_CORRUPT; break; } assert( pSeg!=0 ); i += fts5GetVarint32(&pData[i], pSeg->iSegid); i += fts5GetVarint32(&pData[i], pSeg->pgnoFirst); i += fts5GetVarint32(&pData[i], pSeg->pgnoLast); if( bStructureV2 ){ i += fts5GetVarint(&pData[i], &pSeg->iOrigin1); i += fts5GetVarint(&pData[i], &pSeg->iOrigin2); i += fts5GetVarint32(&pData[i], pSeg->nPgTombstone); i += fts5GetVarint(&pData[i], &pSeg->nEntryTombstone); i += fts5GetVarint(&pData[i], &pSeg->nEntry); nOriginCntr = MAX(nOriginCntr, pSeg->iOrigin2); } if( pSeg->pgnoLast<pSeg->pgnoFirst ){ rc = FTS5_CORRUPT; break; } } if( iLvl>0 && pLvl[-1].nMerge && nTotal==0 ) rc = FTS5_CORRUPT; if( iLvl==nLevel-1 && pLvl->nMerge ) rc = FTS5_CORRUPT; } } if( nSegment!=0 && rc==SQLITE_OK ) rc = FTS5_CORRUPT; if( bStructureV2 ){ pRet->nOriginCntr = nOriginCntr+1; } if( rc!=SQLITE_OK ){ fts5StructureRelease(pRet); pRet = 0; } } *ppOut = pRet; return rc; } /* ** Add a level to the Fts5Structure.aLevel[] array of structure object ** (*ppStruct). */ static void fts5StructureAddLevel(int *pRc, Fts5Structure **ppStruct){ fts5StructureMakeWritable(pRc, ppStruct); assert( (ppStruct!=0 && (*ppStruct)!=0) || (*pRc)!=SQLITE_OK ); if( *pRc==SQLITE_OK ){ Fts5Structure *pStruct = *ppStruct; int nLevel = pStruct->nLevel; sqlite3_int64 nByte = SZ_FTS5STRUCTURE(nLevel+2); pStruct = sqlite3_realloc64(pStruct, nByte); if( pStruct ){ memset(&pStruct->aLevel[nLevel], 0, sizeof(Fts5StructureLevel)); pStruct->nLevel++; *ppStruct = pStruct; }else{ |
︙ | ︙ | |||
1042 1043 1044 1045 1046 1047 1048 | Fts5Data *pData; pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID); if( p->rc==SQLITE_OK ){ /* TODO: Do we need this if the leaf-index is appended? Probably... */ memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING); p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet); | > | | > > > > > | 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 | Fts5Data *pData; pData = fts5DataRead(p, FTS5_STRUCTURE_ROWID); if( p->rc==SQLITE_OK ){ /* TODO: Do we need this if the leaf-index is appended? Probably... */ memset(&pData->p[pData->nn], 0, FTS5_DATA_PADDING); p->rc = fts5StructureDecode(pData->p, pData->nn, &iCookie, &pRet); if( p->rc==SQLITE_OK ){ if( (pConfig->pgsz==0 || pConfig->iCookie!=iCookie) ){ p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie); } }else if( p->rc==SQLITE_CORRUPT_VTAB ){ sqlite3Fts5ConfigErrmsg(p->pConfig, "fts5: corrupt structure record for table \"%s\"", p->pConfig->zName ); } fts5DataRelease(pData); if( p->rc!=SQLITE_OK ){ fts5StructureRelease(pRet); pRet = 0; } } |
︙ | ︙ | |||
1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 | ** error has already occurred, this function is a no-op. */ static void fts5StructureWrite(Fts5Index *p, Fts5Structure *pStruct){ if( p->rc==SQLITE_OK ){ Fts5Buffer buf; /* Buffer to serialize record into */ int iLvl; /* Used to iterate through levels */ int iCookie; /* Cookie value to store */ assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) ); memset(&buf, 0, sizeof(Fts5Buffer)); /* Append the current configuration cookie */ iCookie = p->pConfig->iCookie; if( iCookie<0 ) iCookie = 0; | > | > > > > | | | > > > > > > > | 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 | ** error has already occurred, this function is a no-op. */ static void fts5StructureWrite(Fts5Index *p, Fts5Structure *pStruct){ if( p->rc==SQLITE_OK ){ Fts5Buffer buf; /* Buffer to serialize record into */ int iLvl; /* Used to iterate through levels */ int iCookie; /* Cookie value to store */ int nHdr = (pStruct->nOriginCntr>0 ? (4+4+9+9+9) : (4+9+9)); assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) ); memset(&buf, 0, sizeof(Fts5Buffer)); /* Append the current configuration cookie */ iCookie = p->pConfig->iCookie; if( iCookie<0 ) iCookie = 0; if( 0==sqlite3Fts5BufferSize(&p->rc, &buf, nHdr) ){ sqlite3Fts5Put32(buf.p, iCookie); buf.n = 4; if( pStruct->nOriginCntr>0 ){ fts5BufferSafeAppendBlob(&buf, FTS5_STRUCTURE_V2, 4); } fts5BufferSafeAppendVarint(&buf, pStruct->nLevel); fts5BufferSafeAppendVarint(&buf, pStruct->nSegment); fts5BufferSafeAppendVarint(&buf, (i64)pStruct->nWriteCounter); } for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ int iSeg; /* Used to iterate through segments */ Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl]; fts5BufferAppendVarint(&p->rc, &buf, pLvl->nMerge); fts5BufferAppendVarint(&p->rc, &buf, pLvl->nSeg); assert( pLvl->nMerge<=pLvl->nSeg ); for(iSeg=0; iSeg<pLvl->nSeg; iSeg++){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; fts5BufferAppendVarint(&p->rc, &buf, pSeg->iSegid); fts5BufferAppendVarint(&p->rc, &buf, pSeg->pgnoFirst); fts5BufferAppendVarint(&p->rc, &buf, pSeg->pgnoLast); if( pStruct->nOriginCntr>0 ){ fts5BufferAppendVarint(&p->rc, &buf, pSeg->iOrigin1); fts5BufferAppendVarint(&p->rc, &buf, pSeg->iOrigin2); fts5BufferAppendVarint(&p->rc, &buf, pSeg->nPgTombstone); fts5BufferAppendVarint(&p->rc, &buf, pSeg->nEntryTombstone); fts5BufferAppendVarint(&p->rc, &buf, pSeg->nEntry); } } } fts5DataWrite(p, FTS5_STRUCTURE_ROWID, buf.p, buf.n); fts5BufferFree(&buf); } } |
︙ | ︙ | |||
1342 1343 1344 1345 1346 1347 1348 | }else{ int iOff; for(iOff=pLvl->iOff; iOff<pData->nn; iOff++){ if( pData->p[iOff] ) break; } if( iOff<pData->nn ){ | | | | 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 | }else{ int iOff; for(iOff=pLvl->iOff; iOff<pData->nn; iOff++){ if( pData->p[iOff] ) break; } if( iOff<pData->nn ){ u64 iVal; pLvl->iLeafPgno += (iOff - pLvl->iOff) + 1; iOff += fts5GetVarint(&pData->p[iOff], &iVal); pLvl->iRowid += iVal; pLvl->iOff = iOff; }else{ pLvl->bEof = 1; } } |
︙ | ︙ | |||
1437 1438 1439 1440 1441 1442 1443 | int iOff = pLvl->iOff; assert( pLvl->bEof==0 ); if( iOff<=pLvl->iFirstOff ){ pLvl->bEof = 1; }else{ u8 *a = pLvl->pData->p; | | | > | | | < < < < < < < < | < < < < | | > | < < < | < < < < < | | < | > | > | 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 | int iOff = pLvl->iOff; assert( pLvl->bEof==0 ); if( iOff<=pLvl->iFirstOff ){ pLvl->bEof = 1; }else{ u8 *a = pLvl->pData->p; pLvl->iOff = 0; fts5DlidxLvlNext(pLvl); while( 1 ){ int nZero = 0; int ii = pLvl->iOff; u64 delta = 0; while( a[ii]==0 ){ nZero++; ii++; } ii += sqlite3Fts5GetVarint(&a[ii], &delta); if( ii>=iOff ) break; pLvl->iLeafPgno += nZero+1; pLvl->iRowid += delta; pLvl->iOff = ii; } } return pLvl->bEof; } static int fts5DlidxIterPrevR(Fts5Index *p, Fts5DlidxIter *pIter, int iLvl){ Fts5DlidxLvl *pLvl = &pIter->aLvl[iLvl]; |
︙ | ︙ | |||
1529 1530 1531 1532 1533 1534 1535 | int iLeafPg /* Leaf page number to load dlidx for */ ){ Fts5DlidxIter *pIter = 0; int i; int bDone = 0; for(i=0; p->rc==SQLITE_OK && bDone==0; i++){ | | | 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 | int iLeafPg /* Leaf page number to load dlidx for */ ){ Fts5DlidxIter *pIter = 0; int i; int bDone = 0; for(i=0; p->rc==SQLITE_OK && bDone==0; i++){ sqlite3_int64 nByte = SZ_FTS5DLIDXITER(i+1); Fts5DlidxIter *pNew; pNew = (Fts5DlidxIter*)sqlite3_realloc64(pIter, nByte); if( pNew==0 ){ p->rc = SQLITE_NOMEM; }else{ i64 iRowid = FTS5_DLIDX_ROWID(iSegid, i, iLeafPg); |
︙ | ︙ | |||
1668 1669 1670 1671 1672 1673 1674 | } static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ i64 iOff = pIter->iLeafOffset; ASSERT_SZLEAF_OK(pIter->pLeaf); | | | | 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 | } static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ i64 iOff = pIter->iLeafOffset; ASSERT_SZLEAF_OK(pIter->pLeaf); while( iOff>=pIter->pLeaf->szLeaf ){ fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ){ if( p->rc==SQLITE_OK ) FTS5_CORRUPT_ITER(p, pIter); return; } iOff = 4; a = pIter->pLeaf->p; } iOff += sqlite3Fts5GetVarint(&a[iOff], (u64*)&pIter->iRowid); pIter->iLeafOffset = iOff; |
︙ | ︙ | |||
1703 1704 1705 1706 1707 1708 1709 | static void fts5SegIterLoadTerm(Fts5Index *p, Fts5SegIter *pIter, int nKeep){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ i64 iOff = pIter->iLeafOffset; /* Offset to read at */ int nNew; /* Bytes of new data */ iOff += fts5GetVarint32(&a[iOff], nNew); if( iOff+nNew>pIter->pLeaf->szLeaf || nKeep>pIter->term.n || nNew==0 ){ | | | 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 | static void fts5SegIterLoadTerm(Fts5Index *p, Fts5SegIter *pIter, int nKeep){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ i64 iOff = pIter->iLeafOffset; /* Offset to read at */ int nNew; /* Bytes of new data */ iOff += fts5GetVarint32(&a[iOff], nNew); if( iOff+nNew>pIter->pLeaf->szLeaf || nKeep>pIter->term.n || nNew==0 ){ FTS5_CORRUPT_ITER(p, pIter); return; } pIter->term.n = nKeep; fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]); assert( pIter->term.n<=pIter->term.nSpace ); iOff += nNew; pIter->iTermLeafOffset = iOff; |
︙ | ︙ | |||
1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 | pIter->xNext = fts5SegIterNext_Reverse; }else if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){ pIter->xNext = fts5SegIterNext_None; }else{ pIter->xNext = fts5SegIterNext; } } /* ** Initialize the iterator object pIter to iterate through the entries in ** segment pSeg. The iterator is left pointing to the first entry when ** this function returns. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If | > > > > > > > > > > > > > > > > > > > | 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 | pIter->xNext = fts5SegIterNext_Reverse; }else if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){ pIter->xNext = fts5SegIterNext_None; }else{ pIter->xNext = fts5SegIterNext; } } /* ** Allocate a tombstone hash page array object (pIter->pTombArray) for ** the iterator passed as the second argument. If an OOM error occurs, ** leave an error in the Fts5Index object. */ static void fts5SegIterAllocTombstone(Fts5Index *p, Fts5SegIter *pIter){ const int nTomb = pIter->pSeg->nPgTombstone; if( nTomb>0 ){ int nByte = SZ_FTS5TOMBSTONEARRAY(nTomb+1); Fts5TombstoneArray *pNew; pNew = (Fts5TombstoneArray*)sqlite3Fts5MallocZero(&p->rc, nByte); if( pNew ){ pNew->nTombstone = nTomb; pNew->nRef = 1; pIter->pTombArray = pNew; } } } /* ** Initialize the iterator object pIter to iterate through the entries in ** segment pSeg. The iterator is left pointing to the first entry when ** this function returns. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If |
︙ | ︙ | |||
1767 1768 1769 1770 1771 1772 1773 | } if( p->rc==SQLITE_OK ){ memset(pIter, 0, sizeof(*pIter)); fts5SegIterSetNext(p, pIter); pIter->pSeg = pSeg; pIter->iLeafPgno = pSeg->pgnoFirst-1; | > | > | > | 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 | } if( p->rc==SQLITE_OK ){ memset(pIter, 0, sizeof(*pIter)); fts5SegIterSetNext(p, pIter); pIter->pSeg = pSeg; pIter->iLeafPgno = pSeg->pgnoFirst-1; do { fts5SegIterNextPage(p, pIter); }while( p->rc==SQLITE_OK && pIter->pLeaf && pIter->pLeaf->nn==4 ); } if( p->rc==SQLITE_OK && pIter->pLeaf ){ pIter->iLeafOffset = 4; assert( pIter->pLeaf!=0 ); assert_nc( pIter->pLeaf->nn>4 ); assert_nc( fts5LeafFirstTermOff(pIter->pLeaf)==4 ); pIter->iPgidxOff = pIter->pLeaf->szLeaf+1; fts5SegIterLoadTerm(p, pIter, 0); fts5SegIterLoadNPos(p, pIter); fts5SegIterAllocTombstone(p, pIter); } } /* ** This function is only ever called on iterators created by calls to ** Fts5IndexQuery() with the FTS5INDEX_QUERY_DESC flag set. ** |
︙ | ︙ | |||
1876 1877 1878 1879 1880 1881 1882 | pIter->iLeafOffset = pIter->iTermLeafOffset; } }else{ int iRowidOff; iRowidOff = fts5LeafFirstRowidOff(pNew); if( iRowidOff ){ if( iRowidOff>=pNew->szLeaf ){ | | | 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 | pIter->iLeafOffset = pIter->iTermLeafOffset; } }else{ int iRowidOff; iRowidOff = fts5LeafFirstRowidOff(pNew); if( iRowidOff ){ if( iRowidOff>=pNew->szLeaf ){ FTS5_CORRUPT_ITER(p, pIter); }else{ pIter->pLeaf = pNew; pIter->iLeafOffset = iRowidOff; } } } |
︙ | ︙ | |||
1964 1965 1966 1967 1968 1969 1970 | assert( (pIter->flags & FTS5_SEGITER_REVERSE)==0 ); assert( p->pConfig->eDetail==FTS5_DETAIL_NONE ); ASSERT_SZLEAF_OK(pIter->pLeaf); iOff = pIter->iLeafOffset; /* Next entry is on the next page */ | | | > | | | 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 | assert( (pIter->flags & FTS5_SEGITER_REVERSE)==0 ); assert( p->pConfig->eDetail==FTS5_DETAIL_NONE ); ASSERT_SZLEAF_OK(pIter->pLeaf); iOff = pIter->iLeafOffset; /* Next entry is on the next page */ while( pIter->pSeg && iOff>=pIter->pLeaf->szLeaf ){ fts5SegIterNextPage(p, pIter); if( p->rc || pIter->pLeaf==0 ) return; pIter->iRowid = 0; iOff = 4; } if( iOff<pIter->iEndofDoclist ){ /* Next entry is on the current page */ u64 iDelta; iOff += sqlite3Fts5GetVarint(&pIter->pLeaf->p[iOff], (u64*)&iDelta); pIter->iLeafOffset = iOff; pIter->iRowid += iDelta; }else if( (pIter->flags & FTS5_SEGITER_ONETERM)==0 ){ if( pIter->pSeg ){ int nKeep = 0; if( iOff!=fts5LeafFirstTermOff(pIter->pLeaf) ){ iOff += fts5GetVarint32(&pIter->pLeaf->p[iOff], nKeep); } pIter->iLeafOffset = iOff; fts5SegIterLoadTerm(p, pIter, nKeep); }else{ const u8 *pList = 0; const char *zTerm = 0; int nTerm = 0; int nList; sqlite3Fts5HashScanNext(p->pHash); sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &nTerm, &pList, &nList); if( pList==0 ) goto next_none_eof; pIter->pLeaf->p = (u8*)pList; pIter->pLeaf->nn = nList; pIter->pLeaf->szLeaf = nList; pIter->iEndofDoclist = nList; sqlite3Fts5BufferSet(&p->rc,&pIter->term, nTerm, (u8*)zTerm); pIter->iLeafOffset = fts5GetVarint(pList, (u64*)&pIter->iRowid); } if( pbNewTerm ) *pbNewTerm = 1; }else{ goto next_none_eof; } |
︙ | ︙ | |||
2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 | assert_nc( iDelta>0 ); } pIter->iLeafOffset = iOff; }else if( pIter->pSeg==0 ){ const u8 *pList = 0; const char *zTerm = 0; int nList = 0; assert( (pIter->flags & FTS5_SEGITER_ONETERM) || pbNewTerm ); if( 0==(pIter->flags & FTS5_SEGITER_ONETERM) ){ sqlite3Fts5HashScanNext(p->pHash); | > | | < | 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 | assert_nc( iDelta>0 ); } pIter->iLeafOffset = iOff; }else if( pIter->pSeg==0 ){ const u8 *pList = 0; const char *zTerm = 0; int nTerm = 0; int nList = 0; assert( (pIter->flags & FTS5_SEGITER_ONETERM) || pbNewTerm ); if( 0==(pIter->flags & FTS5_SEGITER_ONETERM) ){ sqlite3Fts5HashScanNext(p->pHash); sqlite3Fts5HashScanEntry(p->pHash, &zTerm, &nTerm, &pList, &nList); } if( pList==0 ){ fts5DataRelease(pIter->pLeaf); pIter->pLeaf = 0; }else{ pIter->pLeaf->p = (u8*)pList; pIter->pLeaf->nn = nList; pIter->pLeaf->szLeaf = nList; pIter->iEndofDoclist = nList+1; sqlite3Fts5BufferSet(&p->rc, &pIter->term, nTerm, (u8*)zTerm); pIter->iLeafOffset = fts5GetVarint(pList, (u64*)&pIter->iRowid); *pbNewTerm = 1; } }else{ iOff = 0; /* Next entry is not on the current page */ while( iOff==0 ){ |
︙ | ︙ | |||
2109 2110 2111 2112 2113 2114 2115 | ); pIter->iLeafOffset = iOff; pIter->iEndofDoclist = iOff; bNewTerm = 1; } assert_nc( iOff<pLeaf->szLeaf ); if( iOff>pLeaf->szLeaf ){ | | | 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 | ); pIter->iLeafOffset = iOff; pIter->iEndofDoclist = iOff; bNewTerm = 1; } assert_nc( iOff<pLeaf->szLeaf ); if( iOff>pLeaf->szLeaf ){ FTS5_CORRUPT_ITER(p, pIter); return; } } } /* Check if the iterator is now at EOF. If so, return early. */ if( pIter->pLeaf ){ |
︙ | ︙ | |||
2157 2158 2159 2160 2161 2162 2163 | ** the doclist. */ static void fts5SegIterReverse(Fts5Index *p, Fts5SegIter *pIter){ Fts5DlidxIter *pDlidx = pIter->pDlidx; Fts5Data *pLast = 0; int pgnoLast = 0; | | | 2417 2418 2419 2420 2421 2422 2423 2424 2425 2426 2427 2428 2429 2430 2431 | ** the doclist. */ static void fts5SegIterReverse(Fts5Index *p, Fts5SegIter *pIter){ Fts5DlidxIter *pDlidx = pIter->pDlidx; Fts5Data *pLast = 0; int pgnoLast = 0; if( pDlidx && p->pConfig->iVersion==FTS5_CURRENT_VERSION ){ int iSegid = pIter->pSeg->iSegid; pgnoLast = fts5DlidxIterPgno(pDlidx); pLast = fts5LeafRead(p, FTS5_SEGMENT_ROWID(iSegid, pgnoLast)); }else{ Fts5Data *pLeaf = pIter->pLeaf; /* Current leaf data */ /* Currently, Fts5SegIter.iLeafOffset points to the first byte of |
︙ | ︙ | |||
2219 2220 2221 2222 2223 2224 2225 | if( pLast ){ int iOff; fts5DataRelease(pIter->pLeaf); pIter->pLeaf = pLast; pIter->iLeafPgno = pgnoLast; iOff = fts5LeafFirstRowidOff(pLast); if( iOff>pLast->szLeaf ){ | | | 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 | if( pLast ){ int iOff; fts5DataRelease(pIter->pLeaf); pIter->pLeaf = pLast; pIter->iLeafPgno = pgnoLast; iOff = fts5LeafFirstRowidOff(pLast); if( iOff>pLast->szLeaf ){ FTS5_CORRUPT_ITER(p, pIter); return; } iOff += fts5GetVarint(&pLast->p[iOff], (u64*)&pIter->iRowid); pIter->iLeafOffset = iOff; if( fts5LeafIsTermless(pLast) ){ pIter->iEndofDoclist = pLast->nn+1; |
︙ | ︙ | |||
2298 2299 2300 2301 2302 2303 2304 | assert( p->rc==SQLITE_OK ); iPgidx = (u32)pIter->pLeaf->szLeaf; iPgidx += fts5GetVarint32(&a[iPgidx], iTermOff); iOff = iTermOff; if( iOff>n ){ | | | 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 | assert( p->rc==SQLITE_OK ); iPgidx = (u32)pIter->pLeaf->szLeaf; iPgidx += fts5GetVarint32(&a[iPgidx], iTermOff); iOff = iTermOff; if( iOff>n ){ FTS5_CORRUPT_ITER(p, pIter); return; } while( 1 ){ /* Figure out how many new bytes are in this term */ fts5FastGetVarint32(a, iOff, nNew); |
︙ | ︙ | |||
2341 2342 2343 2344 2345 2346 2347 | } iPgidx += fts5GetVarint32(&a[iPgidx], nKeep); iTermOff += nKeep; iOff = iTermOff; if( iOff>=n ){ | | | 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 | } iPgidx += fts5GetVarint32(&a[iPgidx], nKeep); iTermOff += nKeep; iOff = iTermOff; if( iOff>=n ){ FTS5_CORRUPT_ITER(p, pIter); return; } /* Read the nKeep field of the next term. */ fts5FastGetVarint32(a, iOff, nKeep); } |
︙ | ︙ | |||
2363 2364 2365 2366 2367 2368 2369 | fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ) return; a = pIter->pLeaf->p; if( fts5LeafIsTermless(pIter->pLeaf)==0 ){ iPgidx = (u32)pIter->pLeaf->szLeaf; iPgidx += fts5GetVarint32(&pIter->pLeaf->p[iPgidx], iOff); if( iOff<4 || (i64)iOff>=pIter->pLeaf->szLeaf ){ | | | | 2623 2624 2625 2626 2627 2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 2638 2639 2640 2641 2642 2643 2644 2645 2646 2647 2648 2649 2650 2651 2652 | fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ) return; a = pIter->pLeaf->p; if( fts5LeafIsTermless(pIter->pLeaf)==0 ){ iPgidx = (u32)pIter->pLeaf->szLeaf; iPgidx += fts5GetVarint32(&pIter->pLeaf->p[iPgidx], iOff); if( iOff<4 || (i64)iOff>=pIter->pLeaf->szLeaf ){ FTS5_CORRUPT_ITER(p, pIter); return; }else{ nKeep = 0; iTermOff = iOff; n = (u32)pIter->pLeaf->nn; iOff += fts5GetVarint32(&a[iOff], nNew); break; } } }while( 1 ); } search_success: if( (i64)iOff+nNew>n || nNew<1 ){ FTS5_CORRUPT_ITER(p, pIter); return; } pIter->iLeafOffset = iOff + nNew; pIter->iTermLeafOffset = pIter->iLeafOffset; pIter->iTermLeafPgno = pIter->iLeafPgno; fts5BufferSet(&p->rc, &pIter->term, nKeep, pTerm); |
︙ | ︙ | |||
2463 2464 2465 2466 2467 2468 2469 | pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); if( pIter->pLeaf ){ fts5LeafSeek(p, bGe, pIter, pTerm, nTerm); } | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 | pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); if( pIter->pLeaf ){ fts5LeafSeek(p, bGe, pIter, pTerm, nTerm); } if( p->rc==SQLITE_OK && (bGe==0 || (flags & FTS5INDEX_QUERY_SCANONETERM)) ){ pIter->flags |= FTS5_SEGITER_ONETERM; if( pIter->pLeaf ){ if( flags & FTS5INDEX_QUERY_DESC ){ pIter->flags |= FTS5_SEGITER_REVERSE; } if( bDlidx ){ fts5SegIterLoadDlidx(p, pIter); } if( flags & FTS5INDEX_QUERY_DESC ){ fts5SegIterReverse(p, pIter); } } } fts5SegIterSetNext(p, pIter); if( 0==(flags & FTS5INDEX_QUERY_SCANONETERM) ){ fts5SegIterAllocTombstone(p, pIter); } /* Either: ** ** 1) an error has occurred, or ** 2) the iterator points to EOF, or ** 3) the iterator points to an entry with term (pTerm/nTerm), or ** 4) the FTS5INDEX_QUERY_SCAN flag was set and the iterator points ** to an entry with a term greater than or equal to (pTerm/nTerm). */ assert_nc( p->rc!=SQLITE_OK /* 1 */ || pIter->pLeaf==0 /* 2 */ || fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)==0 /* 3 */ || (bGe && fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)>0) /* 4 */ ); } /* ** SQL used by fts5SegIterNextInit() to find the page to open. */ static sqlite3_stmt *fts5IdxNextStmt(Fts5Index *p){ if( p->pIdxNextSelect==0 ){ Fts5Config *pConfig = p->pConfig; fts5IndexPrepareStmt(p, &p->pIdxNextSelect, sqlite3_mprintf( "SELECT pgno FROM '%q'.'%q_idx' WHERE " "segid=? AND term>? ORDER BY term ASC LIMIT 1", pConfig->zDb, pConfig->zName )); } return p->pIdxNextSelect; } /* ** This is similar to fts5SegIterSeekInit(), except that it initializes ** the segment iterator to point to the first term following the page ** with pToken/nToken on it. */ static void fts5SegIterNextInit( Fts5Index *p, const char *pTerm, int nTerm, Fts5StructureSegment *pSeg, /* Description of segment */ Fts5SegIter *pIter /* Object to populate */ ){ int iPg = -1; /* Page of segment to open */ int bDlidx = 0; sqlite3_stmt *pSel = 0; /* SELECT to find iPg */ pSel = fts5IdxNextStmt(p); if( pSel ){ assert( p->rc==SQLITE_OK ); sqlite3_bind_int(pSel, 1, pSeg->iSegid); sqlite3_bind_blob(pSel, 2, pTerm, nTerm, SQLITE_STATIC); if( sqlite3_step(pSel)==SQLITE_ROW ){ i64 val = sqlite3_column_int64(pSel, 0); iPg = (int)(val>>1); bDlidx = (val & 0x0001); } p->rc = sqlite3_reset(pSel); sqlite3_bind_null(pSel, 2); if( p->rc ) return; } memset(pIter, 0, sizeof(*pIter)); pIter->pSeg = pSeg; pIter->flags |= FTS5_SEGITER_ONETERM; if( iPg>=0 ){ pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); fts5SegIterSetNext(p, pIter); } if( pIter->pLeaf ){ const u8 *a = pIter->pLeaf->p; int iTermOff = 0; pIter->iPgidxOff = pIter->pLeaf->szLeaf; pIter->iPgidxOff += fts5GetVarint32(&a[pIter->iPgidxOff], iTermOff); pIter->iLeafOffset = iTermOff; fts5SegIterLoadTerm(p, pIter, 0); fts5SegIterLoadNPos(p, pIter); if( bDlidx ) fts5SegIterLoadDlidx(p, pIter); assert( p->rc!=SQLITE_OK || fts5BufferCompareBlob(&pIter->term, (const u8*)pTerm, nTerm)>0 ); } } /* ** Initialize the object pIter to point to term pTerm/nTerm within the ** in-memory hash table. If there is no such term in the hash-table, the ** iterator is set to EOF. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If ** an error has already occurred when this function is called, it is a no-op. |
︙ | ︙ | |||
2521 2522 2523 2524 2525 2526 2527 | assert( p->pHash ); assert( p->rc==SQLITE_OK ); if( pTerm==0 || (flags & FTS5INDEX_QUERY_SCAN) ){ const u8 *pList = 0; p->rc = sqlite3Fts5HashScanInit(p->pHash, (const char*)pTerm, nTerm); | | < > > > > > > > > | 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 | assert( p->pHash ); assert( p->rc==SQLITE_OK ); if( pTerm==0 || (flags & FTS5INDEX_QUERY_SCAN) ){ const u8 *pList = 0; p->rc = sqlite3Fts5HashScanInit(p->pHash, (const char*)pTerm, nTerm); sqlite3Fts5HashScanEntry(p->pHash, (const char**)&z, &n, &pList, &nList); if( pList ){ pLeaf = fts5IdxMalloc(p, sizeof(Fts5Data)); if( pLeaf ){ pLeaf->p = (u8*)pList; } } /* The call to sqlite3Fts5HashScanInit() causes the hash table to ** fill the size field of all existing position lists. This means they ** can no longer be appended to. Since the only scenario in which they ** can be appended to is if the previous operation on this table was ** a DELETE, by clearing the Fts5Index.bDelete flag we can avoid this ** possibility altogether. */ p->bDelete = 0; }else{ p->rc = sqlite3Fts5HashQuery(p->pHash, sizeof(Fts5Data), (const char*)pTerm, nTerm, (void**)&pLeaf, &nList ); if( pLeaf ){ pLeaf->p = (u8*)&pLeaf[1]; } |
︙ | ︙ | |||
2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 | }else{ fts5SegIterLoadNPos(p, pIter); } } fts5SegIterSetNext(p, pIter); } /* ** Zero the iterator passed as the only argument. */ static void fts5SegIterClear(Fts5SegIter *pIter){ fts5BufferFree(&pIter->term); fts5DataRelease(pIter->pLeaf); fts5DataRelease(pIter->pNextLeaf); fts5DlidxIterFree(pIter->pDlidx); sqlite3_free(pIter->aRowidOffset); memset(pIter, 0, sizeof(Fts5SegIter)); } #ifdef SQLITE_DEBUG | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 | }else{ fts5SegIterLoadNPos(p, pIter); } } fts5SegIterSetNext(p, pIter); } /* ** Array ap[] contains n elements. Release each of these elements using ** fts5DataRelease(). Then free the array itself using sqlite3_free(). */ static void fts5IndexFreeArray(Fts5Data **ap, int n){ if( ap ){ int ii; for(ii=0; ii<n; ii++){ fts5DataRelease(ap[ii]); } sqlite3_free(ap); } } /* ** Decrement the ref-count of the object passed as the only argument. If it ** reaches 0, free it and its contents. */ static void fts5TombstoneArrayDelete(Fts5TombstoneArray *p){ if( p ){ p->nRef--; if( p->nRef<=0 ){ int ii; for(ii=0; ii<p->nTombstone; ii++){ fts5DataRelease(p->apTombstone[ii]); } sqlite3_free(p); } } } /* ** Zero the iterator passed as the only argument. */ static void fts5SegIterClear(Fts5SegIter *pIter){ fts5BufferFree(&pIter->term); fts5DataRelease(pIter->pLeaf); fts5DataRelease(pIter->pNextLeaf); fts5TombstoneArrayDelete(pIter->pTombArray); fts5DlidxIterFree(pIter->pDlidx); sqlite3_free(pIter->aRowidOffset); memset(pIter, 0, sizeof(Fts5SegIter)); } #ifdef SQLITE_DEBUG |
︙ | ︙ | |||
2699 2700 2701 2702 2703 2704 2705 | }else{ int res = fts5BufferCompare(&p1->term, &p2->term); if( res==0 ){ assert_nc( i2>i1 ); assert_nc( i2!=0 ); pRes->bTermEq = 1; if( p1->iRowid==p2->iRowid ){ | < | > | < < | > > > > | | < < | | | | | | > > | 3074 3075 3076 3077 3078 3079 3080 3081 3082 3083 3084 3085 3086 3087 3088 3089 3090 3091 3092 3093 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 | }else{ int res = fts5BufferCompare(&p1->term, &p2->term); if( res==0 ){ assert_nc( i2>i1 ); assert_nc( i2!=0 ); pRes->bTermEq = 1; if( p1->iRowid==p2->iRowid ){ return i2; } res = ((p1->iRowid > p2->iRowid)==pIter->bRev) ? -1 : +1; } assert( res!=0 ); if( res<0 ){ iRes = i1; }else{ iRes = i2; } } pRes->iFirst = (u16)iRes; return 0; } /* ** Move the seg-iter so that it points to the first rowid on page iLeafPgno. ** It is an error if leaf iLeafPgno does not exist. Unless the db is ** a 'secure-delete' db, if it contains no rowids then this is also an error. */ static void fts5SegIterGotoPage( Fts5Index *p, /* FTS5 backend object */ Fts5SegIter *pIter, /* Iterator to advance */ int iLeafPgno ){ assert( iLeafPgno>pIter->iLeafPgno ); if( iLeafPgno>pIter->pSeg->pgnoLast ){ FTS5_CORRUPT_IDX(p); }else{ fts5DataRelease(pIter->pNextLeaf); pIter->pNextLeaf = 0; pIter->iLeafPgno = iLeafPgno-1; while( p->rc==SQLITE_OK ){ int iOff; fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ) break; iOff = fts5LeafFirstRowidOff(pIter->pLeaf); if( iOff>0 ){ u8 *a = pIter->pLeaf->p; int n = pIter->pLeaf->szLeaf; if( iOff<4 || iOff>=n ){ FTS5_CORRUPT_IDX(p); }else{ iOff += fts5GetVarint(&a[iOff], (u64*)&pIter->iRowid); pIter->iLeafOffset = iOff; fts5SegIterLoadNPos(p, pIter); } break; } } } } /* ** Advance the iterator passed as the second argument until it is at or |
︙ | ︙ | |||
2807 2808 2809 2810 2811 2812 2813 | if( bMove && p->rc==SQLITE_OK ) pIter->xNext(p, pIter, 0); if( pIter->pLeaf==0 ) break; if( bRev==0 && pIter->iRowid>=iMatch ) break; if( bRev!=0 && pIter->iRowid<=iMatch ) break; bMove = 1; }while( p->rc==SQLITE_OK ); } | < | 3184 3185 3186 3187 3188 3189 3190 3191 3192 3193 3194 3195 3196 3197 | if( bMove && p->rc==SQLITE_OK ) pIter->xNext(p, pIter, 0); if( pIter->pLeaf==0 ) break; if( bRev==0 && pIter->iRowid>=iMatch ) break; if( bRev!=0 && pIter->iRowid<=iMatch ) break; bMove = 1; }while( p->rc==SQLITE_OK ); } /* ** Free the iterator object passed as the second argument. */ static void fts5MultiIterFree(Fts5Iter *pIter){ if( pIter ){ int i; |
︙ | ︙ | |||
2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 | ** Set the pIter->bEof variable based on the state of the sub-iterators. */ static void fts5MultiIterSetEof(Fts5Iter *pIter){ Fts5SegIter *pSeg = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; pIter->base.bEof = pSeg->pLeaf==0; pIter->iSwitchRowid = pSeg->iRowid; } /* ** Move the iterator to the next entry. ** ** If an error occurs, an error code is left in Fts5Index.rc. It is not ** considered an error if the iterator reaches EOF, or if it is already at ** EOF when this function is called. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 | ** Set the pIter->bEof variable based on the state of the sub-iterators. */ static void fts5MultiIterSetEof(Fts5Iter *pIter){ Fts5SegIter *pSeg = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; pIter->base.bEof = pSeg->pLeaf==0; pIter->iSwitchRowid = pSeg->iRowid; } /* ** The argument to this macro must be an Fts5Data structure containing a ** tombstone hash page. This macro returns the key-size of the hash-page. */ #define TOMBSTONE_KEYSIZE(pPg) (pPg->p[0]==4 ? 4 : 8) #define TOMBSTONE_NSLOT(pPg) \ ((pPg->nn > 16) ? ((pPg->nn-8) / TOMBSTONE_KEYSIZE(pPg)) : 1) /* ** Query a single tombstone hash table for rowid iRowid. Return true if ** it is found or false otherwise. The tombstone hash table is one of ** nHashTable tables. */ static int fts5IndexTombstoneQuery( Fts5Data *pHash, /* Hash table page to query */ int nHashTable, /* Number of pages attached to segment */ u64 iRowid /* Rowid to query hash for */ ){ const int szKey = TOMBSTONE_KEYSIZE(pHash); const int nSlot = TOMBSTONE_NSLOT(pHash); int iSlot = (iRowid / nHashTable) % nSlot; int nCollide = nSlot; if( iRowid==0 ){ return pHash->p[1]; }else if( szKey==4 ){ u32 *aSlot = (u32*)&pHash->p[8]; while( aSlot[iSlot] ){ if( fts5GetU32((u8*)&aSlot[iSlot])==iRowid ) return 1; if( nCollide--==0 ) break; iSlot = (iSlot+1)%nSlot; } }else{ u64 *aSlot = (u64*)&pHash->p[8]; while( aSlot[iSlot] ){ if( fts5GetU64((u8*)&aSlot[iSlot])==iRowid ) return 1; if( nCollide--==0 ) break; iSlot = (iSlot+1)%nSlot; } } return 0; } /* ** Return true if the iterator passed as the only argument points ** to an segment entry for which there is a tombstone. Return false ** if there is no tombstone or if the iterator is already at EOF. */ static int fts5MultiIterIsDeleted(Fts5Iter *pIter){ int iFirst = pIter->aFirst[1].iFirst; Fts5SegIter *pSeg = &pIter->aSeg[iFirst]; Fts5TombstoneArray *pArray = pSeg->pTombArray; if( pSeg->pLeaf && pArray ){ /* Figure out which page the rowid might be present on. */ int iPg = ((u64)pSeg->iRowid) % pArray->nTombstone; assert( iPg>=0 ); /* If tombstone hash page iPg has not yet been loaded from the ** database, load it now. */ if( pArray->apTombstone[iPg]==0 ){ pArray->apTombstone[iPg] = fts5DataRead(pIter->pIndex, FTS5_TOMBSTONE_ROWID(pSeg->pSeg->iSegid, iPg) ); if( pArray->apTombstone[iPg]==0 ) return 0; } return fts5IndexTombstoneQuery( pArray->apTombstone[iPg], pArray->nTombstone, pSeg->iRowid ); } return 0; } /* ** Move the iterator to the next entry. ** ** If an error occurs, an error code is left in Fts5Index.rc. It is not ** considered an error if the iterator reaches EOF, or if it is already at ** EOF when this function is called. |
︙ | ︙ | |||
2937 2938 2939 2940 2941 2942 2943 | fts5MultiIterSetEof(pIter); pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst]; if( pSeg->pLeaf==0 ) return; } fts5AssertMultiIterSetup(p, pIter); assert( pSeg==&pIter->aSeg[pIter->aFirst[1].iFirst] && pSeg->pLeaf ); | | > > | 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 | fts5MultiIterSetEof(pIter); pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst]; if( pSeg->pLeaf==0 ) return; } fts5AssertMultiIterSetup(p, pIter); assert( pSeg==&pIter->aSeg[pIter->aFirst[1].iFirst] && pSeg->pLeaf ); if( (pIter->bSkipEmpty==0 || pSeg->nPos) && 0==fts5MultiIterIsDeleted(pIter) ){ pIter->xSetOutputs(pIter, pSeg); return; } bUseFrom = 0; } } |
︙ | ︙ | |||
2969 2970 2971 2972 2973 2974 2975 | ){ fts5MultiIterAdvanced(p, pIter, iFirst, 1); fts5MultiIterSetEof(pIter); *pbNewTerm = 1; } fts5AssertMultiIterSetup(p, pIter); | | > > | | < | 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 | ){ fts5MultiIterAdvanced(p, pIter, iFirst, 1); fts5MultiIterSetEof(pIter); *pbNewTerm = 1; } fts5AssertMultiIterSetup(p, pIter); }while( (fts5MultiIterIsEmpty(p, pIter) || fts5MultiIterIsDeleted(pIter)) && (p->rc==SQLITE_OK) ); } } static void fts5IterSetOutputs_Noop(Fts5Iter *pUnused1, Fts5SegIter *pUnused2){ UNUSED_PARAM2(pUnused1, pUnused2); } static Fts5Iter *fts5MultiIterAlloc( Fts5Index *p, /* FTS5 backend to iterate within */ int nSeg ){ Fts5Iter *pNew; i64 nSlot; /* Power of two >= nSeg */ for(nSlot=2; nSlot<nSeg; nSlot=nSlot*2); pNew = fts5IdxMalloc(p, SZ_FTS5ITER(nSlot) + /* pNew + pNew->aSeg[] */ sizeof(Fts5CResult) * nSlot /* pNew->aFirst[] */ ); if( pNew ){ pNew->nSeg = nSlot; pNew->aFirst = (Fts5CResult*)&pNew->aSeg[nSlot]; pNew->pIndex = p; pNew->xSetOutputs = fts5IterSetOutputs_Noop; |
︙ | ︙ | |||
3139 3140 3141 3142 3143 3144 3145 | while( 1 ){ xChunk(p, pCtx, pChunk, nChunk); nRem -= nChunk; fts5DataRelease(pData); if( nRem<=0 ){ break; }else if( pSeg->pSeg==0 ){ | | | 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 | while( 1 ){ xChunk(p, pCtx, pChunk, nChunk); nRem -= nChunk; fts5DataRelease(pData); if( nRem<=0 ){ break; }else if( pSeg->pSeg==0 ){ FTS5_CORRUPT_IDX(p); return; }else{ pgno++; pData = fts5LeafRead(p, FTS5_SEGMENT_ROWID(pSeg->pSeg->iSegid, pgno)); if( pData==0 ) break; pChunk = &pData->p[4]; nChunk = MIN(nRem, pData->szLeaf - 4); |
︙ | ︙ | |||
3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 | }else{ pIter->xSetOutputs = fts5IterSetOutputs_Col; } } } } /* ** Allocate a new Fts5Iter object. ** ** The new object will be used to iterate through data in structure pStruct. ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel ** is zero or greater, data from the first nSegment segments on level iLevel | > > > > > > > > > > > > > > > > > > > > > > > > > > | 3885 3886 3887 3888 3889 3890 3891 3892 3893 3894 3895 3896 3897 3898 3899 3900 3901 3902 3903 3904 3905 3906 3907 3908 3909 3910 3911 3912 3913 3914 3915 3916 3917 3918 3919 3920 3921 3922 3923 3924 | }else{ pIter->xSetOutputs = fts5IterSetOutputs_Col; } } } } /* ** All the component segment-iterators of pIter have been set up. This ** functions finishes setup for iterator pIter itself. */ static void fts5MultiIterFinishSetup(Fts5Index *p, Fts5Iter *pIter){ int iIter; for(iIter=pIter->nSeg-1; iIter>0; iIter--){ int iEq; if( (iEq = fts5MultiIterDoCompare(pIter, iIter)) ){ Fts5SegIter *pSeg = &pIter->aSeg[iEq]; if( p->rc==SQLITE_OK ) pSeg->xNext(p, pSeg, 0); fts5MultiIterAdvanced(p, pIter, iEq, iIter); } } fts5MultiIterSetEof(pIter); fts5AssertMultiIterSetup(p, pIter); if( (pIter->bSkipEmpty && fts5MultiIterIsEmpty(p, pIter)) || fts5MultiIterIsDeleted(pIter) ){ fts5MultiIterNext(p, pIter, 0, 0); }else if( pIter->base.bEof==0 ){ Fts5SegIter *pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst]; pIter->xSetOutputs(pIter, pSeg); } } /* ** Allocate a new Fts5Iter object. ** ** The new object will be used to iterate through data in structure pStruct. ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel ** is zero or greater, data from the first nSegment segments on level iLevel |
︙ | ︙ | |||
3462 3463 3464 3465 3466 3467 3468 | assert( (pTerm==0 && nTerm==0) || iLevel<0 ); /* Allocate space for the new multi-seg-iterator. */ if( p->rc==SQLITE_OK ){ if( iLevel<0 ){ assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) ); nSeg = pStruct->nSegment; | | | | 3946 3947 3948 3949 3950 3951 3952 3953 3954 3955 3956 3957 3958 3959 3960 3961 3962 3963 3964 3965 3966 3967 3968 3969 3970 3971 3972 3973 3974 3975 3976 3977 3978 3979 3980 3981 | assert( (pTerm==0 && nTerm==0) || iLevel<0 ); /* Allocate space for the new multi-seg-iterator. */ if( p->rc==SQLITE_OK ){ if( iLevel<0 ){ assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) ); nSeg = pStruct->nSegment; nSeg += (p->pHash && 0==(flags & FTS5INDEX_QUERY_SKIPHASH)); }else{ nSeg = MIN(pStruct->aLevel[iLevel].nSeg, nSegment); } } *ppOut = pNew = fts5MultiIterAlloc(p, nSeg); if( pNew==0 ){ assert( p->rc!=SQLITE_OK ); goto fts5MultiIterNew_post_check; } pNew->bRev = (0!=(flags & FTS5INDEX_QUERY_DESC)); pNew->bSkipEmpty = (0!=(flags & FTS5INDEX_QUERY_SKIPEMPTY)); pNew->pColset = pColset; if( (flags & FTS5INDEX_QUERY_NOOUTPUT)==0 ){ fts5IterSetOutputCb(&p->rc, pNew); } /* Initialize each of the component segment iterators. */ if( p->rc==SQLITE_OK ){ if( iLevel<0 ){ Fts5StructureLevel *pEnd = &pStruct->aLevel[pStruct->nLevel]; if( p->pHash && 0==(flags & FTS5INDEX_QUERY_SKIPHASH) ){ /* Add a segment iterator for the current contents of the hash table. */ Fts5SegIter *pIter = &pNew->aSeg[iIter++]; fts5SegIterHashInit(p, pTerm, nTerm, flags, pIter); } for(pLvl=&pStruct->aLevel[0]; pLvl<pEnd; pLvl++){ for(iSeg=pLvl->nSeg-1; iSeg>=0; iSeg--){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; |
︙ | ︙ | |||
3508 3509 3510 3511 3512 3513 3514 | for(iSeg=nSeg-1; iSeg>=0; iSeg--){ fts5SegIterInit(p, &pLvl->aSeg[iSeg], &pNew->aSeg[iIter++]); } } assert( iIter==nSeg ); } | | < < < < < < < < | < < < < < < < < < | 3992 3993 3994 3995 3996 3997 3998 3999 4000 4001 4002 4003 4004 4005 4006 4007 4008 4009 4010 4011 | for(iSeg=nSeg-1; iSeg>=0; iSeg--){ fts5SegIterInit(p, &pLvl->aSeg[iSeg], &pNew->aSeg[iIter++]); } } assert( iIter==nSeg ); } /* If the above was successful, each component iterator now points ** to the first entry in its segment. In this case initialize the ** aFirst[] array. Or, if an error has occurred, free the iterator ** object and set the output variable to NULL. */ if( p->rc==SQLITE_OK ){ fts5MultiIterFinishSetup(p, pNew); }else{ fts5MultiIterFree(pNew); *ppOut = 0; } fts5MultiIterNew_post_check: assert( (*ppOut)!=0 || p->rc!=SQLITE_OK ); |
︙ | ︙ | |||
3555 3556 3557 3558 3559 3560 3561 | int bDesc, /* True for descending rowid order */ Fts5Iter **ppOut /* New object */ ){ Fts5Iter *pNew; pNew = fts5MultiIterAlloc(p, 2); if( pNew ){ Fts5SegIter *pIter = &pNew->aSeg[1]; | < | 4022 4023 4024 4025 4026 4027 4028 4029 4030 4031 4032 4033 4034 4035 | int bDesc, /* True for descending rowid order */ Fts5Iter **ppOut /* New object */ ){ Fts5Iter *pNew; pNew = fts5MultiIterAlloc(p, 2); if( pNew ){ Fts5SegIter *pIter = &pNew->aSeg[1]; pIter->flags = FTS5_SEGITER_ONETERM; if( pData->szLeaf>0 ){ pIter->pLeaf = pData; pIter->iLeafOffset = fts5GetVarint(pData->p, (u64*)&pIter->iRowid); pIter->iEndofDoclist = pData->nn; pNew->aFirst[1].iFirst = 1; if( bDesc ){ |
︙ | ︙ | |||
3702 3703 3704 3705 3706 3707 3708 3709 3710 3711 3712 3713 3714 3715 3716 | ** Discard all data currently cached in the hash-tables. */ static void fts5IndexDiscardData(Fts5Index *p){ assert( p->pHash || p->nPendingData==0 ); if( p->pHash ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; } } /* ** Return the size of the prefix, in bytes, that buffer ** (pNew/<length-unknown>) shares with buffer (pOld/nOld). ** ** Buffer (pNew/<length-unknown>) is guaranteed to be greater | > > > | 4168 4169 4170 4171 4172 4173 4174 4175 4176 4177 4178 4179 4180 4181 4182 4183 4184 4185 | ** Discard all data currently cached in the hash-tables. */ static void fts5IndexDiscardData(Fts5Index *p){ assert( p->pHash || p->nPendingData==0 ); if( p->pHash ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; p->nPendingRow = 0; p->flushRc = SQLITE_OK; } p->nContentlessDelete = 0; } /* ** Return the size of the prefix, in bytes, that buffer ** (pNew/<length-unknown>) shares with buffer (pOld/nOld). ** ** Buffer (pNew/<length-unknown>) is guaranteed to be greater |
︙ | ︙ | |||
3916 3917 3918 3919 3920 3921 3922 | pDlidx->bPrevValid = 0; pDlidx->pgno++; }else{ bDone = 1; } if( pDlidx->bPrevValid ){ | | | 4385 4386 4387 4388 4389 4390 4391 4392 4393 4394 4395 4396 4397 4398 4399 | pDlidx->bPrevValid = 0; pDlidx->pgno++; }else{ bDone = 1; } if( pDlidx->bPrevValid ){ iVal = (u64)iRowid - (u64)pDlidx->iPrev; }else{ i64 iPgno = (i==0 ? pWriter->writer.pgno : pDlidx[-1].pgno); assert( pDlidx->buf.n==0 ); sqlite3Fts5BufferAppendVarint(&p->rc, &pDlidx->buf, !bDone); sqlite3Fts5BufferAppendVarint(&p->rc, &pDlidx->buf, iPgno); iVal = iRowid; } |
︙ | ︙ | |||
4103 4104 4105 4106 4107 4108 4109 | const u8 *aData, int nData ){ Fts5PageWriter *pPage = &pWriter->writer; const u8 *a = aData; int n = nData; | | | 4572 4573 4574 4575 4576 4577 4578 4579 4580 4581 4582 4583 4584 4585 4586 | const u8 *aData, int nData ){ Fts5PageWriter *pPage = &pWriter->writer; const u8 *a = aData; int n = nData; assert( p->pConfig->pgsz>0 || p->rc!=SQLITE_OK ); while( p->rc==SQLITE_OK && (pPage->buf.n + pPage->pgidx.n + n)>=p->pConfig->pgsz ){ int nReq = p->pConfig->pgsz - pPage->buf.n - pPage->pgidx.n; int nCopy = 0; while( nCopy<nReq ){ i64 dummy; |
︙ | ︙ | |||
4231 4232 4233 4234 4235 4236 4237 | pData = fts5LeafRead(p, iLeafRowid); if( pData ){ if( iOff>pData->szLeaf ){ /* This can occur if the pages that the segments occupy overlap - if ** a single page has been assigned to more than one segment. In ** this case a prior iteration of this loop may have corrupted the ** segment currently being trimmed. */ | | | | 4700 4701 4702 4703 4704 4705 4706 4707 4708 4709 4710 4711 4712 4713 4714 4715 4716 4717 4718 4719 4720 4721 | pData = fts5LeafRead(p, iLeafRowid); if( pData ){ if( iOff>pData->szLeaf ){ /* This can occur if the pages that the segments occupy overlap - if ** a single page has been assigned to more than one segment. In ** this case a prior iteration of this loop may have corrupted the ** segment currently being trimmed. */ FTS5_CORRUPT_ROWID(p, iLeafRowid); }else{ fts5BufferZero(&buf); fts5BufferGrow(&p->rc, &buf, pData->nn); fts5BufferAppendBlob(&p->rc, &buf, sizeof(aHdr), aHdr); fts5BufferAppendVarint(&p->rc, &buf, pSeg->term.n); fts5BufferAppendBlob(&p->rc, &buf, pSeg->term.n, pSeg->term.p); fts5BufferAppendBlob(&p->rc, &buf,pData->szLeaf-iOff,&pData->p[iOff]); if( p->rc==SQLITE_OK ){ /* Set the szLeaf field */ fts5PutU16(&buf.p[2], (u16)buf.n); } /* Set up the new page-index array */ fts5BufferAppendVarint(&p->rc, &buf, 4); |
︙ | ︙ | |||
4339 4340 4341 4342 4343 4344 4345 4346 4347 4348 4349 4350 4351 4352 | pLvlOut->nSeg++; pSeg->pgnoFirst = 1; pSeg->iSegid = iSegid; pStruct->nSegment++; /* Read input from all segments in the input level */ nInput = pLvl->nSeg; } bOldest = (pLvlOut->nSeg==1 && pStruct->nLevel==iLvl+2); assert( iLvl>=0 ); for(fts5MultiIterNew(p, pStruct, flags, 0, 0, 0, iLvl, nInput, &pIter); fts5MultiIterEof(p, pIter)==0; fts5MultiIterNext(p, pIter, 0, 0) | > > > > > > | 4808 4809 4810 4811 4812 4813 4814 4815 4816 4817 4818 4819 4820 4821 4822 4823 4824 4825 4826 4827 | pLvlOut->nSeg++; pSeg->pgnoFirst = 1; pSeg->iSegid = iSegid; pStruct->nSegment++; /* Read input from all segments in the input level */ nInput = pLvl->nSeg; /* Set the range of origins that will go into the output segment. */ if( pStruct->nOriginCntr>0 ){ pSeg->iOrigin1 = pLvl->aSeg[0].iOrigin1; pSeg->iOrigin2 = pLvl->aSeg[pLvl->nSeg-1].iOrigin2; } } bOldest = (pLvlOut->nSeg==1 && pStruct->nLevel==iLvl+2); assert( iLvl>=0 ); for(fts5MultiIterNew(p, pStruct, flags, 0, 0, 0, iLvl, nInput, &pIter); fts5MultiIterEof(p, pIter)==0; fts5MultiIterNext(p, pIter, 0, 0) |
︙ | ︙ | |||
4398 4399 4400 4401 4402 4403 4404 4405 | fts5WriteFinish(p, &writer, &pSeg->pgnoLast); assert( pIter!=0 || p->rc!=SQLITE_OK ); if( fts5MultiIterEof(p, pIter) ){ int i; /* Remove the redundant segments from the %_data table */ for(i=0; i<nInput; i++){ | > > > | | 4873 4874 4875 4876 4877 4878 4879 4880 4881 4882 4883 4884 4885 4886 4887 4888 4889 4890 4891 | fts5WriteFinish(p, &writer, &pSeg->pgnoLast); assert( pIter!=0 || p->rc!=SQLITE_OK ); if( fts5MultiIterEof(p, pIter) ){ int i; /* Remove the redundant segments from the %_data table */ assert( pSeg->nEntry==0 ); for(i=0; i<nInput; i++){ Fts5StructureSegment *pOld = &pLvl->aSeg[i]; pSeg->nEntry += (pOld->nEntry - pOld->nEntryTombstone); fts5DataRemoveSegment(p, pOld); } /* Remove the redundant segments from the input level */ if( pLvl->nSeg!=nInput ){ int nMove = (pLvl->nSeg - nInput) * sizeof(Fts5StructureSegment); memmove(pLvl->aSeg, &pLvl->aSeg[nInput], nMove); } |
︙ | ︙ | |||
4424 4425 4426 4427 4428 4429 4430 4431 4432 4433 4434 4435 4436 4437 4438 4439 4440 4441 4442 4443 4444 4445 4446 4447 4448 4449 4450 | pLvl->nMerge = nInput; } fts5MultiIterFree(pIter); fts5BufferFree(&term); if( pnRem ) *pnRem -= writer.nLeafWritten; } /* ** Do up to nPg pages of automerge work on the index. ** ** Return true if any changes were actually made, or false otherwise. */ static int fts5IndexMerge( Fts5Index *p, /* FTS5 backend object */ Fts5Structure **ppStruct, /* IN/OUT: Current structure of index */ int nPg, /* Pages of work to do */ int nMin /* Minimum number of segments to merge */ ){ int nRem = nPg; int bRet = 0; Fts5Structure *pStruct = *ppStruct; while( nRem>0 && p->rc==SQLITE_OK ){ int iLvl; /* To iterate through levels */ int iBestLvl = 0; /* Level offering the most input segments */ int nBest = 0; /* Number of input segments on best level */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | | < < < | < < | < > > | 4902 4903 4904 4905 4906 4907 4908 4909 4910 4911 4912 4913 4914 4915 4916 4917 4918 4919 4920 4921 4922 4923 4924 4925 4926 4927 4928 4929 4930 4931 4932 4933 4934 4935 4936 4937 4938 4939 4940 4941 4942 4943 4944 4945 4946 4947 4948 4949 4950 4951 4952 4953 4954 4955 4956 4957 4958 4959 4960 4961 4962 4963 4964 4965 4966 4967 4968 4969 4970 4971 4972 4973 4974 4975 4976 4977 4978 4979 4980 4981 4982 4983 4984 4985 4986 4987 4988 4989 4990 4991 4992 4993 4994 4995 4996 4997 4998 4999 5000 5001 5002 5003 5004 5005 5006 | pLvl->nMerge = nInput; } fts5MultiIterFree(pIter); fts5BufferFree(&term); if( pnRem ) *pnRem -= writer.nLeafWritten; } /* ** If this is not a contentless_delete=1 table, or if the 'deletemerge' ** configuration option is set to 0, then this function always returns -1. ** Otherwise, it searches the structure object passed as the second argument ** for a level suitable for merging due to having a large number of ** tombstones in the tombstone hash. If one is found, its index is returned. ** Otherwise, if there is no suitable level, -1. */ static int fts5IndexFindDeleteMerge(Fts5Index *p, Fts5Structure *pStruct){ Fts5Config *pConfig = p->pConfig; int iRet = -1; if( pConfig->bContentlessDelete && pConfig->nDeleteMerge>0 ){ int ii; int nBest = 0; for(ii=0; ii<pStruct->nLevel; ii++){ Fts5StructureLevel *pLvl = &pStruct->aLevel[ii]; i64 nEntry = 0; i64 nTomb = 0; int iSeg; for(iSeg=0; iSeg<pLvl->nSeg; iSeg++){ nEntry += pLvl->aSeg[iSeg].nEntry; nTomb += pLvl->aSeg[iSeg].nEntryTombstone; } assert_nc( nEntry>0 || pLvl->nSeg==0 ); if( nEntry>0 ){ int nPercent = (nTomb * 100) / nEntry; if( nPercent>=pConfig->nDeleteMerge && nPercent>nBest ){ iRet = ii; nBest = nPercent; } } /* If pLvl is already the input level to an ongoing merge, look no ** further for a merge candidate. The caller should be allowed to ** continue merging from pLvl first. */ if( pLvl->nMerge ) break; } } return iRet; } /* ** Do up to nPg pages of automerge work on the index. ** ** Return true if any changes were actually made, or false otherwise. */ static int fts5IndexMerge( Fts5Index *p, /* FTS5 backend object */ Fts5Structure **ppStruct, /* IN/OUT: Current structure of index */ int nPg, /* Pages of work to do */ int nMin /* Minimum number of segments to merge */ ){ int nRem = nPg; int bRet = 0; Fts5Structure *pStruct = *ppStruct; while( nRem>0 && p->rc==SQLITE_OK ){ int iLvl; /* To iterate through levels */ int iBestLvl = 0; /* Level offering the most input segments */ int nBest = 0; /* Number of input segments on best level */ /* Set iBestLvl to the level to read input segments from. Or to -1 if ** there is no level suitable to merge segments from. */ assert( pStruct->nLevel>0 ); for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl]; if( pLvl->nMerge ){ if( pLvl->nMerge>nBest ){ iBestLvl = iLvl; nBest = nMin; } break; } if( pLvl->nSeg>nBest ){ nBest = pLvl->nSeg; iBestLvl = iLvl; } } if( nBest<nMin ){ iBestLvl = fts5IndexFindDeleteMerge(p, pStruct); } if( iBestLvl<0 ) break; bRet = 1; fts5IndexMergeLevel(p, &pStruct, iBestLvl, &nRem); if( p->rc==SQLITE_OK && pStruct->aLevel[iBestLvl].nMerge==0 ){ fts5StructurePromote(p, iBestLvl+1, pStruct); } if( nMin==1 ) nMin = 2; } *ppStruct = pStruct; return bRet; } /* ** A total of nLeaf leaf pages of data has just been flushed to a level-0 |
︙ | ︙ | |||
4516 4517 4518 4519 4520 4521 4522 | static void fts5IndexCrisismerge( Fts5Index *p, /* FTS5 backend object */ Fts5Structure **ppStruct /* IN/OUT: Current structure of index */ ){ const int nCrisis = p->pConfig->nCrisisMerge; Fts5Structure *pStruct = *ppStruct; | > | < < | | | | | | | > > > > > > > > > | 5033 5034 5035 5036 5037 5038 5039 5040 5041 5042 5043 5044 5045 5046 5047 5048 5049 5050 5051 5052 5053 5054 5055 5056 5057 5058 5059 5060 5061 5062 5063 5064 5065 5066 5067 5068 5069 5070 5071 | static void fts5IndexCrisismerge( Fts5Index *p, /* FTS5 backend object */ Fts5Structure **ppStruct /* IN/OUT: Current structure of index */ ){ const int nCrisis = p->pConfig->nCrisisMerge; Fts5Structure *pStruct = *ppStruct; if( pStruct && pStruct->nLevel>0 ){ int iLvl = 0; while( p->rc==SQLITE_OK && pStruct->aLevel[iLvl].nSeg>=nCrisis ){ fts5IndexMergeLevel(p, &pStruct, iLvl, 0); assert( p->rc!=SQLITE_OK || pStruct->nLevel>(iLvl+1) ); fts5StructurePromote(p, iLvl+1, pStruct); iLvl++; } *ppStruct = pStruct; } } static int fts5IndexReturn(Fts5Index *p){ int rc = p->rc; p->rc = SQLITE_OK; return rc; } /* ** Close the read-only blob handle, if it is open. */ void sqlite3Fts5IndexCloseReader(Fts5Index *p){ fts5IndexCloseReader(p); fts5IndexReturn(p); } typedef struct Fts5FlushCtx Fts5FlushCtx; struct Fts5FlushCtx { Fts5Index *pIdx; Fts5SegWriter writer; }; |
︙ | ︙ | |||
4558 4559 4560 4561 4562 4563 4564 4565 4566 4567 4568 4569 4570 4571 4572 4573 4574 4575 4576 4577 4578 4579 4580 4581 | int i = fts5GetVarint32(&aBuf[ret], dummy); if( (ret + i) > nMax ) break; ret += i; } } return ret; } /* ** Flush the contents of in-memory hash table iHash to a new level-0 ** segment on disk. Also update the corresponding structure record. ** ** If an error occurs, set the Fts5Index.rc error code. If an error has ** already occurred, this function is a no-op. */ static void fts5FlushOneHash(Fts5Index *p){ Fts5Hash *pHash = p->pHash; Fts5Structure *pStruct; int iSegid; int pgnoLast = 0; /* Last leaf page number in segment */ /* Obtain a reference to the index structure and allocate a new segment-id ** for the new level-0 segment. */ pStruct = fts5StructureRead(p); | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < > > | | | > | | | | | | | | | | | | | | | | | | | | | | > | | | | | > | | > | | | | | | > | | | | | | | | > | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | < | > | | > | > | | | | | | | | | | | | | | | | | > > > > > | > | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | < | | > > | | | | | | | | | | | > > > > > > | | | | | > > | > > > > | > > > | > > > | > | | | | > | | | | > > > > | > | > | | 5083 5084 5085 5086 5087 5088 5089 5090 5091 5092 5093 5094 5095 5096 5097 5098 5099 5100 5101 5102 5103 5104 5105 5106 5107 5108 5109 5110 5111 5112 5113 5114 5115 5116 5117 5118 5119 5120 5121 5122 5123 5124 5125 5126 5127 5128 5129 5130 5131 5132 5133 5134 5135 5136 5137 5138 5139 5140 5141 5142 5143 5144 5145 5146 5147 5148 5149 5150 5151 5152 5153 5154 5155 5156 5157 5158 5159 5160 5161 5162 5163 5164 5165 5166 5167 5168 5169 5170 5171 5172 5173 5174 5175 5176 5177 5178 5179 5180 5181 5182 5183 5184 5185 5186 5187 5188 5189 5190 5191 5192 5193 5194 5195 5196 5197 5198 5199 5200 5201 5202 5203 5204 5205 5206 5207 5208 5209 5210 5211 5212 5213 5214 5215 5216 5217 5218 5219 5220 5221 5222 5223 5224 5225 5226 5227 5228 5229 5230 5231 5232 5233 5234 5235 5236 5237 5238 5239 5240 5241 5242 5243 5244 5245 5246 5247 5248 5249 5250 5251 5252 5253 5254 5255 5256 5257 5258 5259 5260 5261 5262 5263 5264 5265 5266 5267 5268 5269 5270 5271 5272 5273 5274 5275 5276 5277 5278 5279 5280 5281 5282 5283 5284 5285 5286 5287 5288 5289 5290 5291 5292 5293 5294 5295 5296 5297 5298 5299 5300 5301 5302 5303 5304 5305 5306 5307 5308 5309 5310 5311 5312 5313 5314 5315 5316 5317 5318 5319 5320 5321 5322 5323 5324 5325 5326 5327 5328 5329 5330 5331 5332 5333 5334 5335 5336 5337 5338 5339 5340 5341 5342 5343 5344 5345 5346 5347 5348 5349 5350 5351 5352 5353 5354 5355 5356 5357 5358 5359 5360 5361 5362 5363 5364 5365 5366 5367 5368 5369 5370 5371 5372 5373 5374 5375 5376 5377 5378 5379 5380 5381 5382 5383 5384 5385 5386 5387 5388 5389 5390 5391 5392 5393 5394 5395 5396 5397 5398 5399 5400 5401 5402 5403 5404 5405 5406 5407 5408 5409 5410 5411 5412 5413 5414 5415 5416 5417 5418 5419 5420 5421 5422 5423 5424 5425 5426 5427 5428 5429 5430 5431 5432 5433 5434 5435 5436 5437 5438 5439 5440 5441 5442 5443 5444 5445 5446 5447 5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 5461 5462 5463 5464 5465 5466 5467 5468 5469 5470 5471 5472 5473 5474 5475 5476 5477 5478 5479 5480 5481 5482 5483 5484 5485 5486 5487 5488 5489 5490 5491 5492 5493 5494 5495 5496 5497 5498 5499 5500 5501 5502 5503 5504 5505 5506 5507 5508 5509 5510 5511 5512 5513 5514 5515 5516 5517 5518 5519 5520 5521 5522 5523 5524 5525 5526 5527 5528 5529 5530 5531 5532 5533 5534 5535 5536 5537 5538 5539 5540 5541 5542 5543 5544 5545 5546 5547 5548 5549 5550 5551 5552 5553 5554 5555 5556 5557 5558 5559 5560 5561 5562 5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 5590 5591 5592 5593 5594 5595 5596 5597 5598 5599 5600 5601 5602 5603 5604 5605 5606 5607 5608 5609 5610 5611 5612 5613 5614 5615 5616 5617 5618 5619 5620 5621 5622 5623 5624 5625 5626 5627 5628 5629 5630 5631 5632 5633 5634 5635 5636 5637 5638 5639 5640 5641 5642 5643 5644 5645 5646 5647 5648 5649 5650 5651 5652 5653 5654 5655 5656 5657 5658 5659 5660 5661 5662 5663 5664 5665 5666 5667 5668 5669 5670 5671 5672 5673 5674 5675 5676 5677 5678 5679 5680 5681 5682 5683 5684 5685 5686 5687 5688 5689 5690 5691 5692 5693 5694 5695 5696 5697 5698 5699 5700 5701 5702 5703 5704 5705 5706 5707 5708 5709 5710 5711 5712 5713 5714 5715 5716 5717 5718 5719 5720 5721 5722 5723 5724 5725 5726 5727 5728 5729 5730 5731 5732 5733 5734 5735 5736 5737 5738 5739 5740 5741 5742 5743 5744 5745 5746 5747 5748 5749 5750 5751 5752 5753 5754 5755 5756 5757 5758 5759 5760 5761 5762 5763 5764 5765 5766 5767 5768 5769 5770 5771 5772 5773 5774 5775 5776 5777 5778 5779 5780 5781 5782 5783 5784 5785 5786 5787 5788 5789 5790 5791 5792 5793 5794 5795 5796 5797 5798 5799 5800 5801 5802 5803 5804 5805 5806 5807 5808 5809 5810 5811 5812 5813 5814 5815 5816 5817 5818 5819 5820 5821 5822 5823 5824 5825 5826 5827 5828 5829 5830 5831 5832 5833 5834 5835 5836 5837 5838 5839 5840 5841 5842 5843 5844 5845 5846 5847 5848 5849 5850 5851 5852 5853 5854 5855 5856 5857 5858 5859 5860 5861 5862 5863 5864 5865 | int i = fts5GetVarint32(&aBuf[ret], dummy); if( (ret + i) > nMax ) break; ret += i; } } return ret; } /* ** Execute the SQL statement: ** ** DELETE FROM %_idx WHERE (segid, (pgno/2)) = ($iSegid, $iPgno); ** ** This is used when a secure-delete operation removes the last term ** from a segment leaf page. In that case the %_idx entry is removed ** too. This is done to ensure that if all instances of a token are ** removed from an fts5 database in secure-delete mode, no trace of ** the token itself remains in the database. */ static void fts5SecureDeleteIdxEntry( Fts5Index *p, /* FTS5 backend object */ int iSegid, /* Id of segment to delete entry for */ int iPgno /* Page number within segment */ ){ if( iPgno!=1 ){ assert( p->pConfig->iVersion==FTS5_CURRENT_VERSION_SECUREDELETE ); if( p->pDeleteFromIdx==0 ){ fts5IndexPrepareStmt(p, &p->pDeleteFromIdx, sqlite3_mprintf( "DELETE FROM '%q'.'%q_idx' WHERE (segid, (pgno/2)) = (?1, ?2)", p->pConfig->zDb, p->pConfig->zName )); } if( p->rc==SQLITE_OK ){ sqlite3_bind_int(p->pDeleteFromIdx, 1, iSegid); sqlite3_bind_int(p->pDeleteFromIdx, 2, iPgno); sqlite3_step(p->pDeleteFromIdx); p->rc = sqlite3_reset(p->pDeleteFromIdx); } } } /* ** This is called when a secure-delete operation removes a position-list ** that overflows onto segment page iPgno of segment pSeg. This function ** rewrites node iPgno, and possibly one or more of its right-hand peers, ** to remove this portion of the position list. ** ** Output variable (*pbLastInDoclist) is set to true if the position-list ** removed is followed by a new term or the end-of-segment, or false if ** it is followed by another rowid/position list. */ static void fts5SecureDeleteOverflow( Fts5Index *p, Fts5StructureSegment *pSeg, int iPgno, int *pbLastInDoclist ){ const int bDetailNone = (p->pConfig->eDetail==FTS5_DETAIL_NONE); int pgno; Fts5Data *pLeaf = 0; assert( iPgno!=1 ); *pbLastInDoclist = 1; for(pgno=iPgno; p->rc==SQLITE_OK && pgno<=pSeg->pgnoLast; pgno++){ i64 iRowid = FTS5_SEGMENT_ROWID(pSeg->iSegid, pgno); int iNext = 0; u8 *aPg = 0; pLeaf = fts5DataRead(p, iRowid); if( pLeaf==0 ) break; aPg = pLeaf->p; iNext = fts5GetU16(&aPg[0]); if( iNext!=0 ){ *pbLastInDoclist = 0; } if( iNext==0 && pLeaf->szLeaf!=pLeaf->nn ){ fts5GetVarint32(&aPg[pLeaf->szLeaf], iNext); } if( iNext==0 ){ /* The page contains no terms or rowids. Replace it with an empty ** page and move on to the right-hand peer. */ const u8 aEmpty[] = {0x00, 0x00, 0x00, 0x04}; assert_nc( bDetailNone==0 || pLeaf->nn==4 ); if( bDetailNone==0 ) fts5DataWrite(p, iRowid, aEmpty, sizeof(aEmpty)); fts5DataRelease(pLeaf); pLeaf = 0; }else if( bDetailNone ){ break; }else if( iNext>=pLeaf->szLeaf || pLeaf->nn<pLeaf->szLeaf || iNext<4 ){ FTS5_CORRUPT_ROWID(p, iRowid); break; }else{ int nShift = iNext - 4; int nPg; int nIdx = 0; u8 *aIdx = 0; /* Unless the current page footer is 0 bytes in size (in which case ** the new page footer will be as well), allocate and populate a ** buffer containing the new page footer. Set stack variables aIdx ** and nIdx accordingly. */ if( pLeaf->nn>pLeaf->szLeaf ){ int iFirst = 0; int i1 = pLeaf->szLeaf; int i2 = 0; i1 += fts5GetVarint32(&aPg[i1], iFirst); if( iFirst<iNext ){ FTS5_CORRUPT_ROWID(p, iRowid); break; } aIdx = sqlite3Fts5MallocZero(&p->rc, (pLeaf->nn-pLeaf->szLeaf)+2); if( aIdx==0 ) break; i2 = sqlite3Fts5PutVarint(aIdx, iFirst-nShift); if( i1<pLeaf->nn ){ memcpy(&aIdx[i2], &aPg[i1], pLeaf->nn-i1); i2 += (pLeaf->nn-i1); } nIdx = i2; } /* Modify the contents of buffer aPg[]. Set nPg to the new size ** in bytes. The new page is always smaller than the old. */ nPg = pLeaf->szLeaf - nShift; memmove(&aPg[4], &aPg[4+nShift], nPg-4); fts5PutU16(&aPg[2], nPg); if( fts5GetU16(&aPg[0]) ) fts5PutU16(&aPg[0], 4); if( nIdx>0 ){ memcpy(&aPg[nPg], aIdx, nIdx); nPg += nIdx; } sqlite3_free(aIdx); /* Write the new page to disk and exit the loop */ assert( nPg>4 || fts5GetU16(aPg)==0 ); fts5DataWrite(p, iRowid, aPg, nPg); break; } } fts5DataRelease(pLeaf); } /* ** Completely remove the entry that pSeg currently points to from ** the database. */ static void fts5DoSecureDelete( Fts5Index *p, Fts5SegIter *pSeg ){ const int bDetailNone = (p->pConfig->eDetail==FTS5_DETAIL_NONE); int iSegid = pSeg->pSeg->iSegid; u8 *aPg = pSeg->pLeaf->p; int nPg = pSeg->pLeaf->nn; int iPgIdx = pSeg->pLeaf->szLeaf; u64 iDelta = 0; int iNextOff = 0; int iOff = 0; int nIdx = 0; u8 *aIdx = 0; int bLastInDoclist = 0; int iIdx = 0; int iStart = 0; int iDelKeyOff = 0; /* Offset of deleted key, if any */ nIdx = nPg-iPgIdx; aIdx = sqlite3Fts5MallocZero(&p->rc, ((i64)nIdx)+16); if( p->rc ) return; memcpy(aIdx, &aPg[iPgIdx], nIdx); /* At this point segment iterator pSeg points to the entry ** this function should remove from the b-tree segment. ** ** In detail=full or detail=column mode, pSeg->iLeafOffset is the ** offset of the first byte in the position-list for the entry to ** remove. Immediately before this comes two varints that will also ** need to be removed: ** ** + the rowid or delta rowid value for the entry, and ** + the size of the position list in bytes. ** ** Or, in detail=none mode, there is a single varint prior to ** pSeg->iLeafOffset - the rowid or delta rowid value. ** ** This block sets the following variables: ** ** iStart: ** The offset of the first byte of the rowid or delta-rowid ** value for the doclist entry being removed. ** ** iDelta: ** The value of the rowid or delta-rowid value for the doclist ** entry being removed. ** ** iNextOff: ** The offset of the next entry following the position list ** for the one being removed. If the position list for this ** entry overflows onto the next leaf page, this value will be ** greater than pLeaf->szLeaf. */ { int iSOP; /* Start-Of-Position-list */ if( pSeg->iLeafPgno==pSeg->iTermLeafPgno ){ iStart = pSeg->iTermLeafOffset; }else{ iStart = fts5GetU16(&aPg[0]); } iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta); assert_nc( iSOP<=pSeg->iLeafOffset ); if( bDetailNone ){ while( iSOP<pSeg->iLeafOffset ){ if( aPg[iSOP]==0x00 ) iSOP++; if( aPg[iSOP]==0x00 ) iSOP++; iStart = iSOP; iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta); } iNextOff = iSOP; if( iNextOff<pSeg->iEndofDoclist && aPg[iNextOff]==0x00 ) iNextOff++; if( iNextOff<pSeg->iEndofDoclist && aPg[iNextOff]==0x00 ) iNextOff++; }else{ int nPos = 0; iSOP += fts5GetVarint32(&aPg[iSOP], nPos); while( iSOP<pSeg->iLeafOffset ){ iStart = iSOP + (nPos/2); iSOP = iStart + fts5GetVarint(&aPg[iStart], &iDelta); iSOP += fts5GetVarint32(&aPg[iSOP], nPos); } assert_nc( iSOP==pSeg->iLeafOffset ); iNextOff = pSeg->iLeafOffset + pSeg->nPos; } } iOff = iStart; /* If the position-list for the entry being removed flows over past ** the end of this page, delete the portion of the position-list on the ** next page and beyond. ** ** Set variable bLastInDoclist to true if this entry happens ** to be the last rowid in the doclist for its term. */ if( iNextOff>=iPgIdx ){ int pgno = pSeg->iLeafPgno+1; fts5SecureDeleteOverflow(p, pSeg->pSeg, pgno, &bLastInDoclist); iNextOff = iPgIdx; } if( pSeg->bDel==0 ){ if( iNextOff!=iPgIdx ){ /* Loop through the page-footer. If iNextOff (offset of the ** entry following the one we are removing) is equal to the ** offset of a key on this page, then the entry is the last ** in its doclist. */ int iKeyOff = 0; for(iIdx=0; iIdx<nIdx; /* no-op */){ u32 iVal = 0; iIdx += fts5GetVarint32(&aIdx[iIdx], iVal); iKeyOff += iVal; if( iKeyOff==iNextOff ){ bLastInDoclist = 1; } } } /* If this is (a) the first rowid on a page and (b) is not followed by ** another position list on the same page, set the "first-rowid" field ** of the header to 0. */ if( fts5GetU16(&aPg[0])==iStart && (bLastInDoclist || iNextOff==iPgIdx) ){ fts5PutU16(&aPg[0], 0); } } if( pSeg->bDel ){ iOff += sqlite3Fts5PutVarint(&aPg[iOff], iDelta); aPg[iOff++] = 0x01; }else if( bLastInDoclist==0 ){ if( iNextOff!=iPgIdx ){ u64 iNextDelta = 0; iNextOff += fts5GetVarint(&aPg[iNextOff], &iNextDelta); iOff += sqlite3Fts5PutVarint(&aPg[iOff], iDelta + iNextDelta); } }else if( pSeg->iLeafPgno==pSeg->iTermLeafPgno && iStart==pSeg->iTermLeafOffset ){ /* The entry being removed was the only position list in its ** doclist. Therefore the term needs to be removed as well. */ int iKey = 0; int iKeyOff = 0; /* Set iKeyOff to the offset of the term that will be removed - the ** last offset in the footer that is not greater than iStart. */ for(iIdx=0; iIdx<nIdx; iKey++){ u32 iVal = 0; iIdx += fts5GetVarint32(&aIdx[iIdx], iVal); if( (iKeyOff+iVal)>(u32)iStart ) break; iKeyOff += iVal; } assert_nc( iKey>=1 ); /* Set iDelKeyOff to the value of the footer entry to remove from ** the page. */ iDelKeyOff = iOff = iKeyOff; if( iNextOff!=iPgIdx ){ /* This is the only position-list associated with the term, and there ** is another term following it on this page. So the subsequent term ** needs to be moved to replace the term associated with the entry ** being removed. */ int nPrefix = 0; int nSuffix = 0; int nPrefix2 = 0; int nSuffix2 = 0; iDelKeyOff = iNextOff; iNextOff += fts5GetVarint32(&aPg[iNextOff], nPrefix2); iNextOff += fts5GetVarint32(&aPg[iNextOff], nSuffix2); if( iKey!=1 ){ iKeyOff += fts5GetVarint32(&aPg[iKeyOff], nPrefix); } iKeyOff += fts5GetVarint32(&aPg[iKeyOff], nSuffix); nPrefix = MIN(nPrefix, nPrefix2); nSuffix = (nPrefix2 + nSuffix2) - nPrefix; if( (iKeyOff+nSuffix)>iPgIdx || (iNextOff+nSuffix2)>iPgIdx ){ FTS5_CORRUPT_IDX(p); }else{ if( iKey!=1 ){ iOff += sqlite3Fts5PutVarint(&aPg[iOff], nPrefix); } iOff += sqlite3Fts5PutVarint(&aPg[iOff], nSuffix); if( nPrefix2>pSeg->term.n ){ FTS5_CORRUPT_IDX(p); }else if( nPrefix2>nPrefix ){ memcpy(&aPg[iOff], &pSeg->term.p[nPrefix], nPrefix2-nPrefix); iOff += (nPrefix2-nPrefix); } memmove(&aPg[iOff], &aPg[iNextOff], nSuffix2); iOff += nSuffix2; iNextOff += nSuffix2; } } }else if( iStart==4 ){ int iPgno; assert_nc( pSeg->iLeafPgno>pSeg->iTermLeafPgno ); /* The entry being removed may be the only position list in ** its doclist. */ for(iPgno=pSeg->iLeafPgno-1; iPgno>pSeg->iTermLeafPgno; iPgno-- ){ Fts5Data *pPg = fts5DataRead(p, FTS5_SEGMENT_ROWID(iSegid, iPgno)); int bEmpty = (pPg && pPg->nn==4); fts5DataRelease(pPg); if( bEmpty==0 ) break; } if( iPgno==pSeg->iTermLeafPgno ){ i64 iId = FTS5_SEGMENT_ROWID(iSegid, pSeg->iTermLeafPgno); Fts5Data *pTerm = fts5DataRead(p, iId); if( pTerm && pTerm->szLeaf==pSeg->iTermLeafOffset ){ u8 *aTermIdx = &pTerm->p[pTerm->szLeaf]; int nTermIdx = pTerm->nn - pTerm->szLeaf; int iTermIdx = 0; int iTermOff = 0; while( 1 ){ u32 iVal = 0; int nByte = fts5GetVarint32(&aTermIdx[iTermIdx], iVal); iTermOff += iVal; if( (iTermIdx+nByte)>=nTermIdx ) break; iTermIdx += nByte; } nTermIdx = iTermIdx; memmove(&pTerm->p[iTermOff], &pTerm->p[pTerm->szLeaf], nTermIdx); fts5PutU16(&pTerm->p[2], iTermOff); fts5DataWrite(p, iId, pTerm->p, iTermOff+nTermIdx); if( nTermIdx==0 ){ fts5SecureDeleteIdxEntry(p, iSegid, pSeg->iTermLeafPgno); } } fts5DataRelease(pTerm); } } /* Assuming no error has occurred, this block does final edits to the ** leaf page before writing it back to disk. Input variables are: ** ** nPg: Total initial size of leaf page. ** iPgIdx: Initial offset of page footer. ** ** iOff: Offset to move data to ** iNextOff: Offset to move data from */ if( p->rc==SQLITE_OK ){ const int nMove = nPg - iNextOff; /* Number of bytes to move */ int nShift = iNextOff - iOff; /* Distance to move them */ int iPrevKeyOut = 0; int iKeyIn = 0; memmove(&aPg[iOff], &aPg[iNextOff], nMove); iPgIdx -= nShift; nPg = iPgIdx; fts5PutU16(&aPg[2], iPgIdx); for(iIdx=0; iIdx<nIdx; /* no-op */){ u32 iVal = 0; iIdx += fts5GetVarint32(&aIdx[iIdx], iVal); iKeyIn += iVal; if( iKeyIn!=iDelKeyOff ){ int iKeyOut = (iKeyIn - (iKeyIn>iOff ? nShift : 0)); nPg += sqlite3Fts5PutVarint(&aPg[nPg], iKeyOut - iPrevKeyOut); iPrevKeyOut = iKeyOut; } } if( iPgIdx==nPg && nIdx>0 && pSeg->iLeafPgno!=1 ){ fts5SecureDeleteIdxEntry(p, iSegid, pSeg->iLeafPgno); } assert_nc( nPg>4 || fts5GetU16(aPg)==0 ); fts5DataWrite(p, FTS5_SEGMENT_ROWID(iSegid,pSeg->iLeafPgno), aPg, nPg); } sqlite3_free(aIdx); } /* ** This is called as part of flushing a delete to disk in 'secure-delete' ** mode. It edits the segments within the database described by argument ** pStruct to remove the entries for term zTerm, rowid iRowid. ** ** Return SQLITE_OK if successful, or an SQLite error code if an error ** has occurred. Any error code is also stored in the Fts5Index handle. */ static int fts5FlushSecureDelete( Fts5Index *p, Fts5Structure *pStruct, const char *zTerm, int nTerm, i64 iRowid ){ const int f = FTS5INDEX_QUERY_SKIPHASH; Fts5Iter *pIter = 0; /* Used to find term instance */ /* If the version number has not been set to SECUREDELETE, do so now. */ if( p->pConfig->iVersion!=FTS5_CURRENT_VERSION_SECUREDELETE ){ Fts5Config *pConfig = p->pConfig; sqlite3_stmt *pStmt = 0; fts5IndexPrepareStmt(p, &pStmt, sqlite3_mprintf( "REPLACE INTO %Q.'%q_config' VALUES ('version', %d)", pConfig->zDb, pConfig->zName, FTS5_CURRENT_VERSION_SECUREDELETE )); if( p->rc==SQLITE_OK ){ int rc; sqlite3_step(pStmt); rc = sqlite3_finalize(pStmt); if( p->rc==SQLITE_OK ) p->rc = rc; pConfig->iCookie++; pConfig->iVersion = FTS5_CURRENT_VERSION_SECUREDELETE; } } fts5MultiIterNew(p, pStruct, f, 0, (const u8*)zTerm, nTerm, -1, 0, &pIter); if( fts5MultiIterEof(p, pIter)==0 ){ i64 iThis = fts5MultiIterRowid(pIter); if( iThis<iRowid ){ fts5MultiIterNextFrom(p, pIter, iRowid); } if( p->rc==SQLITE_OK && fts5MultiIterEof(p, pIter)==0 && iRowid==fts5MultiIterRowid(pIter) ){ Fts5SegIter *pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst]; fts5DoSecureDelete(p, pSeg); } } fts5MultiIterFree(pIter); return p->rc; } /* ** Flush the contents of in-memory hash table iHash to a new level-0 ** segment on disk. Also update the corresponding structure record. ** ** If an error occurs, set the Fts5Index.rc error code. If an error has ** already occurred, this function is a no-op. */ static void fts5FlushOneHash(Fts5Index *p){ Fts5Hash *pHash = p->pHash; Fts5Structure *pStruct; int iSegid; int pgnoLast = 0; /* Last leaf page number in segment */ /* Obtain a reference to the index structure and allocate a new segment-id ** for the new level-0 segment. */ pStruct = fts5StructureRead(p); fts5StructureInvalidate(p); if( sqlite3Fts5HashIsEmpty(pHash)==0 ){ iSegid = fts5AllocateSegid(p, pStruct); if( iSegid ){ const int pgsz = p->pConfig->pgsz; int eDetail = p->pConfig->eDetail; int bSecureDelete = p->pConfig->bSecureDelete; Fts5StructureSegment *pSeg; /* New segment within pStruct */ Fts5Buffer *pBuf; /* Buffer in which to assemble leaf page */ Fts5Buffer *pPgidx; /* Buffer in which to assemble pgidx */ Fts5SegWriter writer; fts5WriteInit(p, &writer, iSegid); pBuf = &writer.writer.buf; pPgidx = &writer.writer.pgidx; /* fts5WriteInit() should have initialized the buffers to (most likely) ** the maximum space required. */ assert( p->rc || pBuf->nSpace>=(pgsz + FTS5_DATA_PADDING) ); assert( p->rc || pPgidx->nSpace>=(pgsz + FTS5_DATA_PADDING) ); /* Begin scanning through hash table entries. This loop runs once for each ** term/doclist currently stored within the hash table. */ if( p->rc==SQLITE_OK ){ p->rc = sqlite3Fts5HashScanInit(pHash, 0, 0); } while( p->rc==SQLITE_OK && 0==sqlite3Fts5HashScanEof(pHash) ){ const char *zTerm; /* Buffer containing term */ int nTerm; /* Size of zTerm in bytes */ const u8 *pDoclist; /* Pointer to doclist for this term */ int nDoclist; /* Size of doclist in bytes */ /* Get the term and doclist for this entry. */ sqlite3Fts5HashScanEntry(pHash, &zTerm, &nTerm, &pDoclist, &nDoclist); if( bSecureDelete==0 ){ fts5WriteAppendTerm(p, &writer, nTerm, (const u8*)zTerm); if( p->rc!=SQLITE_OK ) break; assert( writer.bFirstRowidInPage==0 ); } if( !bSecureDelete && pgsz>=(pBuf->n + pPgidx->n + nDoclist + 1) ){ /* The entire doclist will fit on the current leaf. */ fts5BufferSafeAppendBlob(pBuf, pDoclist, nDoclist); }else{ int bTermWritten = !bSecureDelete; i64 iRowid = 0; i64 iPrev = 0; int iOff = 0; /* The entire doclist will not fit on this leaf. The following ** loop iterates through the poslists that make up the current ** doclist. */ while( p->rc==SQLITE_OK && iOff<nDoclist ){ u64 iDelta = 0; iOff += fts5GetVarint(&pDoclist[iOff], &iDelta); iRowid += iDelta; /* If in secure delete mode, and if this entry in the poslist is ** in fact a delete, then edit the existing segments directly ** using fts5FlushSecureDelete(). */ if( bSecureDelete ){ if( eDetail==FTS5_DETAIL_NONE ){ if( iOff<nDoclist && pDoclist[iOff]==0x00 && !fts5FlushSecureDelete(p, pStruct, zTerm, nTerm, iRowid) ){ iOff++; if( iOff<nDoclist && pDoclist[iOff]==0x00 ){ iOff++; nDoclist = 0; }else{ continue; } } }else if( (pDoclist[iOff] & 0x01) && !fts5FlushSecureDelete(p, pStruct, zTerm, nTerm, iRowid) ){ if( p->rc!=SQLITE_OK || pDoclist[iOff]==0x01 ){ iOff++; continue; } } } if( p->rc==SQLITE_OK && bTermWritten==0 ){ fts5WriteAppendTerm(p, &writer, nTerm, (const u8*)zTerm); bTermWritten = 1; assert( p->rc!=SQLITE_OK || writer.bFirstRowidInPage==0 ); } if( writer.bFirstRowidInPage ){ fts5PutU16(&pBuf->p[0], (u16)pBuf->n); /* first rowid on page */ pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iRowid); writer.bFirstRowidInPage = 0; fts5WriteDlidxAppend(p, &writer, iRowid); }else{ u64 iRowidDelta = (u64)iRowid - (u64)iPrev; pBuf->n += sqlite3Fts5PutVarint(&pBuf->p[pBuf->n], iRowidDelta); } if( p->rc!=SQLITE_OK ) break; assert( pBuf->n<=pBuf->nSpace ); iPrev = iRowid; if( eDetail==FTS5_DETAIL_NONE ){ if( iOff<nDoclist && pDoclist[iOff]==0 ){ pBuf->p[pBuf->n++] = 0; iOff++; if( iOff<nDoclist && pDoclist[iOff]==0 ){ pBuf->p[pBuf->n++] = 0; iOff++; } } if( (pBuf->n + pPgidx->n)>=pgsz ){ fts5WriteFlushLeaf(p, &writer); } }else{ int bDel = 0; int nPos = 0; int nCopy = fts5GetPoslistSize(&pDoclist[iOff], &nPos, &bDel); if( bDel && bSecureDelete ){ fts5BufferAppendVarint(&p->rc, pBuf, nPos*2); iOff += nCopy; nCopy = nPos; }else{ nCopy += nPos; } if( (pBuf->n + pPgidx->n + nCopy) <= pgsz ){ /* The entire poslist will fit on the current leaf. So copy ** it in one go. */ fts5BufferSafeAppendBlob(pBuf, &pDoclist[iOff], nCopy); }else{ /* The entire poslist will not fit on this leaf. So it needs ** to be broken into sections. The only qualification being ** that each varint must be stored contiguously. */ const u8 *pPoslist = &pDoclist[iOff]; int iPos = 0; while( p->rc==SQLITE_OK ){ int nSpace = pgsz - pBuf->n - pPgidx->n; int n = 0; if( (nCopy - iPos)<=nSpace ){ n = nCopy - iPos; }else{ n = fts5PoslistPrefix(&pPoslist[iPos], nSpace); } assert( n>0 ); fts5BufferSafeAppendBlob(pBuf, &pPoslist[iPos], n); iPos += n; if( (pBuf->n + pPgidx->n)>=pgsz ){ fts5WriteFlushLeaf(p, &writer); } if( iPos>=nCopy ) break; } } iOff += nCopy; } } } /* TODO2: Doclist terminator written here. */ /* pBuf->p[pBuf->n++] = '\0'; */ assert( pBuf->n<=pBuf->nSpace ); if( p->rc==SQLITE_OK ) sqlite3Fts5HashScanNext(pHash); } fts5WriteFinish(p, &writer, &pgnoLast); assert( p->rc!=SQLITE_OK || bSecureDelete || pgnoLast>0 ); if( pgnoLast>0 ){ /* Update the Fts5Structure. It is written back to the database by the ** fts5StructureRelease() call below. */ if( pStruct->nLevel==0 ){ fts5StructureAddLevel(&p->rc, &pStruct); } fts5StructureExtendLevel(&p->rc, pStruct, 0, 1, 0); if( p->rc==SQLITE_OK ){ pSeg = &pStruct->aLevel[0].aSeg[ pStruct->aLevel[0].nSeg++ ]; pSeg->iSegid = iSegid; pSeg->pgnoFirst = 1; pSeg->pgnoLast = pgnoLast; if( pStruct->nOriginCntr>0 ){ pSeg->iOrigin1 = pStruct->nOriginCntr; pSeg->iOrigin2 = pStruct->nOriginCntr; pSeg->nEntry = p->nPendingRow; pStruct->nOriginCntr++; } pStruct->nSegment++; } fts5StructurePromote(p, 0, pStruct); } } } fts5IndexAutomerge(p, &pStruct, pgnoLast + p->nContentlessDelete); fts5IndexCrisismerge(p, &pStruct); fts5StructureWrite(p, pStruct); fts5StructureRelease(pStruct); } /* ** Flush any data stored in the in-memory hash tables to the database. */ static void fts5IndexFlush(Fts5Index *p){ /* Unless it is empty, flush the hash table to disk */ if( p->flushRc ){ p->rc = p->flushRc; return; } if( p->nPendingData || p->nContentlessDelete ){ assert( p->pHash ); fts5FlushOneHash(p); if( p->rc==SQLITE_OK ){ sqlite3Fts5HashClear(p->pHash); p->nPendingData = 0; p->nPendingRow = 0; p->nContentlessDelete = 0; }else if( p->nPendingData || p->nContentlessDelete ){ p->flushRc = p->rc; } } } static Fts5Structure *fts5IndexOptimizeStruct( Fts5Index *p, Fts5Structure *pStruct ){ Fts5Structure *pNew = 0; sqlite3_int64 nByte = SZ_FTS5STRUCTURE(1); int nSeg = pStruct->nSegment; int i; /* Figure out if this structure requires optimization. A structure does ** not require optimization if either: ** ** 1. it consists of fewer than two segments, or ** 2. all segments are on the same level, or ** 3. all segments except one are currently inputs to a merge operation. ** ** In the first case, if there are no tombstone hash pages, return NULL. In ** the second, increment the ref-count on *pStruct and return a copy of the ** pointer to it. */ if( nSeg==0 ) return 0; for(i=0; i<pStruct->nLevel; i++){ int nThis = pStruct->aLevel[i].nSeg; int nMerge = pStruct->aLevel[i].nMerge; if( nThis>0 && (nThis==nSeg || (nThis==nSeg-1 && nMerge==nThis)) ){ if( nSeg==1 && nThis==1 && pStruct->aLevel[i].aSeg[0].nPgTombstone==0 ){ return 0; } fts5StructureRef(pStruct); return pStruct; } assert( pStruct->aLevel[i].nMerge<=nThis ); } nByte += (((i64)pStruct->nLevel)+1) * sizeof(Fts5StructureLevel); assert( nByte==SZ_FTS5STRUCTURE(pStruct->nLevel+2) ); pNew = (Fts5Structure*)sqlite3Fts5MallocZero(&p->rc, nByte); if( pNew ){ Fts5StructureLevel *pLvl; nByte = nSeg * sizeof(Fts5StructureSegment); pNew->nLevel = MIN(pStruct->nLevel+1, FTS5_MAX_LEVEL); pNew->nRef = 1; pNew->nWriteCounter = pStruct->nWriteCounter; pNew->nOriginCntr = pStruct->nOriginCntr; pLvl = &pNew->aLevel[pNew->nLevel-1]; pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(&p->rc, nByte); if( pLvl->aSeg ){ int iLvl, iSeg; int iSegOut = 0; /* Iterate through all segments, from oldest to newest. Add them to ** the new Fts5Level object so that pLvl->aSeg[0] is the oldest ** segment in the data structure. */ |
︙ | ︙ | |||
4797 4798 4799 4800 4801 4802 4803 4804 4805 4806 4807 4808 4809 4810 4811 | int sqlite3Fts5IndexOptimize(Fts5Index *p){ Fts5Structure *pStruct; Fts5Structure *pNew = 0; assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); pStruct = fts5StructureRead(p); fts5StructureInvalidate(p); if( pStruct ){ pNew = fts5IndexOptimizeStruct(p, pStruct); } fts5StructureRelease(pStruct); | > > | 5881 5882 5883 5884 5885 5886 5887 5888 5889 5890 5891 5892 5893 5894 5895 5896 5897 | int sqlite3Fts5IndexOptimize(Fts5Index *p){ Fts5Structure *pStruct; Fts5Structure *pNew = 0; assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); assert( p->rc!=SQLITE_OK || p->nContentlessDelete==0 ); pStruct = fts5StructureRead(p); assert( p->rc!=SQLITE_OK || pStruct!=0 ); fts5StructureInvalidate(p); if( pStruct ){ pNew = fts5IndexOptimizeStruct(p, pStruct); } fts5StructureRelease(pStruct); |
︙ | ︙ | |||
4826 4827 4828 4829 4830 4831 4832 | } /* ** This is called to implement the special "VALUES('merge', $nMerge)" ** INSERT command. */ int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge){ | | > > > | | 5912 5913 5914 5915 5916 5917 5918 5919 5920 5921 5922 5923 5924 5925 5926 5927 5928 5929 5930 5931 5932 5933 5934 5935 5936 5937 | } /* ** This is called to implement the special "VALUES('merge', $nMerge)" ** INSERT command. */ int sqlite3Fts5IndexMerge(Fts5Index *p, int nMerge){ Fts5Structure *pStruct = 0; fts5IndexFlush(p); pStruct = fts5StructureRead(p); if( pStruct ){ int nMin = p->pConfig->nUsermerge; fts5StructureInvalidate(p); if( nMerge<0 ){ Fts5Structure *pNew = fts5IndexOptimizeStruct(p, pStruct); fts5StructureRelease(pStruct); pStruct = pNew; nMin = 1; nMerge = nMerge*-1; } if( pStruct && pStruct->nLevel ){ if( fts5IndexMerge(p, &pStruct, nMerge, nMin) ){ fts5StructureWrite(p, pStruct); } } |
︙ | ︙ | |||
5121 5122 5123 5124 5125 5126 5127 | nTmp += pSave->iter.nPoslist + 10; nMerge++; fts5PrefixMergerInsertByPosition(&pHead, pSave); pSave = pNext; } if( pHead==0 || pHead->pNext==0 ){ | | | 6210 6211 6212 6213 6214 6215 6216 6217 6218 6219 6220 6221 6222 6223 6224 | nTmp += pSave->iter.nPoslist + 10; nMerge++; fts5PrefixMergerInsertByPosition(&pHead, pSave); pSave = pNext; } if( pHead==0 || pHead->pNext==0 ){ FTS5_CORRUPT_IDX(p); break; } /* See the earlier comment in this function for an explanation of why ** corrupt input position lists might cause the output to consume ** at most nMerge*10 bytes of unexpected space. */ if( sqlite3Fts5BufferSize(&p->rc, &tmp, nTmp+nMerge*10) ){ |
︙ | ︙ | |||
5158 5159 5160 5161 5162 5163 5164 | } nTail = pHead->iter.nPoslist - pHead->iOff; /* WRITEPOSLISTSIZE */ assert_nc( tmp.n+nTail<=nTmp ); assert( tmp.n+nTail<=nTmp+nMerge*10 ); if( tmp.n+nTail>nTmp-FTS5_DATA_ZERO_PADDING ){ | | | 6247 6248 6249 6250 6251 6252 6253 6254 6255 6256 6257 6258 6259 6260 6261 | } nTail = pHead->iter.nPoslist - pHead->iOff; /* WRITEPOSLISTSIZE */ assert_nc( tmp.n+nTail<=nTmp ); assert( tmp.n+nTail<=nTmp+nMerge*10 ); if( tmp.n+nTail>nTmp-FTS5_DATA_ZERO_PADDING ){ if( p->rc==SQLITE_OK ) FTS5_CORRUPT_IDX(p); break; } fts5BufferSafeAppendVarint(&out, (tmp.n+nTail) * 2); fts5BufferSafeAppendBlob(&out, tmp.p, tmp.n); if( nTail>0 ){ fts5BufferSafeAppendBlob(&out, &pHead->aPos[pHead->iOff], nTail); } |
︙ | ︙ | |||
5193 5194 5195 5196 5197 5198 5199 5200 5201 5202 5203 5204 5205 5206 | fts5BufferFree(p1); fts5BufferFree(&tmp); memset(&out.p[out.n], 0, FTS5_DATA_ZERO_PADDING); *p1 = out; } static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ int iIdx, /* Index to scan for data */ u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset, /* Restrict matches to these columns */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > | > > | | > > > > > > > > | | < | | | | | | | > | | < < < < < < > > | > > < < | < < < < < < < < < < | | < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | | | | < | > > | | | > > > > > | | > > > | | > > > | | < | | > | > > > > | | 6282 6283 6284 6285 6286 6287 6288 6289 6290 6291 6292 6293 6294 6295 6296 6297 6298 6299 6300 6301 6302 6303 6304 6305 6306 6307 6308 6309 6310 6311 6312 6313 6314 6315 6316 6317 6318 6319 6320 6321 6322 6323 6324 6325 6326 6327 6328 6329 6330 6331 6332 6333 6334 6335 6336 6337 6338 6339 6340 6341 6342 6343 6344 6345 6346 6347 6348 6349 6350 6351 6352 6353 6354 6355 6356 6357 6358 6359 6360 6361 6362 6363 6364 6365 6366 6367 6368 6369 6370 6371 6372 6373 6374 6375 6376 6377 6378 6379 6380 6381 6382 6383 6384 6385 6386 6387 6388 6389 6390 6391 6392 6393 6394 6395 6396 6397 6398 6399 6400 6401 6402 6403 6404 6405 6406 6407 6408 6409 6410 6411 6412 6413 6414 6415 6416 6417 6418 6419 6420 6421 6422 6423 6424 6425 6426 6427 6428 6429 6430 6431 6432 6433 6434 6435 6436 6437 6438 6439 6440 6441 6442 6443 6444 6445 6446 6447 6448 6449 6450 6451 6452 6453 6454 6455 6456 6457 6458 6459 6460 6461 6462 6463 6464 6465 6466 6467 6468 6469 6470 6471 6472 6473 6474 6475 6476 6477 6478 6479 6480 6481 6482 6483 6484 6485 6486 6487 6488 6489 6490 6491 6492 6493 6494 6495 6496 6497 6498 6499 6500 6501 6502 6503 6504 6505 6506 6507 6508 6509 6510 6511 6512 6513 6514 6515 6516 6517 6518 6519 6520 6521 6522 6523 6524 6525 6526 6527 6528 6529 6530 6531 6532 6533 6534 6535 6536 6537 6538 6539 6540 6541 6542 6543 6544 6545 6546 6547 6548 6549 6550 6551 6552 6553 6554 6555 6556 6557 6558 6559 6560 6561 6562 6563 6564 6565 6566 6567 6568 6569 6570 6571 6572 6573 6574 6575 6576 6577 6578 6579 6580 6581 6582 6583 6584 6585 6586 6587 6588 6589 6590 6591 6592 6593 6594 6595 6596 6597 6598 6599 6600 6601 6602 6603 6604 6605 6606 6607 6608 6609 6610 6611 6612 6613 6614 6615 6616 6617 6618 6619 6620 6621 6622 6623 6624 6625 6626 6627 6628 6629 6630 6631 6632 6633 6634 6635 6636 6637 6638 6639 6640 6641 6642 6643 6644 6645 6646 6647 6648 6649 6650 6651 6652 6653 6654 6655 6656 6657 6658 6659 6660 6661 6662 6663 6664 6665 6666 6667 6668 6669 6670 6671 6672 6673 6674 6675 6676 6677 6678 6679 6680 6681 6682 6683 6684 6685 6686 6687 6688 6689 6690 6691 6692 6693 6694 6695 6696 6697 6698 6699 6700 6701 6702 6703 6704 6705 6706 6707 6708 6709 6710 6711 6712 6713 6714 6715 6716 6717 6718 6719 6720 6721 6722 6723 6724 6725 6726 6727 6728 6729 6730 6731 6732 6733 6734 6735 6736 6737 6738 6739 6740 6741 6742 6743 6744 6745 6746 6747 6748 6749 6750 6751 6752 6753 6754 6755 6756 6757 6758 6759 6760 6761 6762 6763 6764 6765 6766 6767 6768 6769 6770 6771 6772 6773 6774 6775 6776 6777 6778 6779 6780 6781 6782 6783 6784 6785 6786 6787 6788 6789 6790 6791 6792 6793 6794 6795 6796 6797 6798 6799 6800 6801 6802 6803 6804 6805 6806 6807 6808 6809 6810 6811 6812 6813 6814 6815 6816 6817 6818 6819 6820 6821 6822 6823 6824 6825 6826 6827 6828 6829 6830 6831 6832 6833 6834 6835 6836 6837 6838 6839 | fts5BufferFree(p1); fts5BufferFree(&tmp); memset(&out.p[out.n], 0, FTS5_DATA_ZERO_PADDING); *p1 = out; } /* ** Iterate through a range of entries in the FTS index, invoking the xVisit ** callback for each of them. ** ** Parameter pToken points to an nToken buffer containing an FTS index term ** (i.e. a document term with the preceding 1 byte index identifier - ** FTS5_MAIN_PREFIX or similar). If bPrefix is true, then the call visits ** all entries for terms that have pToken/nToken as a prefix. If bPrefix ** is false, then only entries with pToken/nToken as the entire key are ** visited. ** ** If the current table is a tokendata=1 table, then if bPrefix is true then ** each index term is treated separately. However, if bPrefix is false, then ** all index terms corresponding to pToken/nToken are collapsed into a single ** term before the callback is invoked. ** ** The callback invoked for each entry visited is specified by paramter xVisit. ** Each time it is invoked, it is passed a pointer to the Fts5Index object, ** a copy of the 7th paramter to this function (pCtx) and a pointer to the ** iterator that indicates the current entry. If the current entry is the ** first with a new term (i.e. different from that of the previous entry, ** including the very first term), then the final two parameters are passed ** a pointer to the term and its size in bytes, respectively. If the current ** entry is not the first associated with its term, these two parameters ** are passed 0. ** ** If parameter pColset is not NULL, then it is used to filter entries before ** the callback is invoked. */ static int fts5VisitEntries( Fts5Index *p, /* Fts5 index object */ Fts5Colset *pColset, /* Columns filter to apply, or NULL */ u8 *pToken, /* Buffer containing token */ int nToken, /* Size of buffer pToken in bytes */ int bPrefix, /* True for a prefix scan */ void (*xVisit)(Fts5Index*, void *pCtx, Fts5Iter *pIter, const u8*, int), void *pCtx /* Passed as second argument to xVisit() */ ){ const int flags = (bPrefix ? FTS5INDEX_QUERY_SCAN : 0) | FTS5INDEX_QUERY_SKIPEMPTY | FTS5INDEX_QUERY_NOOUTPUT; Fts5Iter *p1 = 0; /* Iterator used to gather data from index */ int bNewTerm = 1; Fts5Structure *pStruct = fts5StructureRead(p); fts5MultiIterNew(p, pStruct, flags, pColset, pToken, nToken, -1, 0, &p1); fts5IterSetOutputCb(&p->rc, p1); for( /* no-op */ ; fts5MultiIterEof(p, p1)==0; fts5MultiIterNext2(p, p1, &bNewTerm) ){ Fts5SegIter *pSeg = &p1->aSeg[ p1->aFirst[1].iFirst ]; int nNew = 0; const u8 *pNew = 0; p1->xSetOutputs(p1, pSeg); if( p->rc ) break; if( bNewTerm ){ nNew = pSeg->term.n; pNew = pSeg->term.p; if( nNew<nToken || memcmp(pToken, pNew, nToken) ) break; } xVisit(p, pCtx, p1, pNew, nNew); } fts5MultiIterFree(p1); fts5StructureRelease(pStruct); return p->rc; } /* ** Usually, a tokendata=1 iterator (struct Fts5TokenDataIter) accumulates an ** array of these for each row it visits (so all iRowid fields are the same). ** Or, for an iterator used by an "ORDER BY rank" query, it accumulates an ** array of these for the entire query (in which case iRowid fields may take ** a variety of values). ** ** Each instance in the array indicates the iterator (and therefore term) ** associated with position iPos of rowid iRowid. This is used by the ** xInstToken() API. ** ** iRowid: ** Rowid for the current entry. ** ** iPos: ** Position of current entry within row. In the usual ((iCol<<32)+iOff) ** format (e.g. see macros FTS5_POS2COLUMN() and FTS5_POS2OFFSET()). ** ** iIter: ** If the Fts5TokenDataIter iterator that the entry is part of is ** actually an iterator (i.e. with nIter>0, not just a container for ** Fts5TokenDataMap structures), then this variable is an index into ** the apIter[] array. The corresponding term is that which the iterator ** at apIter[iIter] currently points to. ** ** Or, if the Fts5TokenDataIter iterator is just a container object ** (nIter==0), then iIter is an index into the term.p[] buffer where ** the term is stored. ** ** nByte: ** In the case where iIter is an index into term.p[], this variable ** is the size of the term in bytes. If iIter is an index into apIter[], ** this variable is unused. */ struct Fts5TokenDataMap { i64 iRowid; /* Row this token is located in */ i64 iPos; /* Position of token */ int iIter; /* Iterator token was read from */ int nByte; /* Length of token in bytes (or 0) */ }; /* ** An object used to supplement Fts5Iter for tokendata=1 iterators. ** ** This object serves two purposes. The first is as a container for an array ** of Fts5TokenDataMap structures, which are used to find the token required ** when the xInstToken() API is used. This is done by the nMapAlloc, nMap and ** aMap[] variables. */ struct Fts5TokenDataIter { int nMapAlloc; /* Allocated size of aMap[] in entries */ int nMap; /* Number of valid entries in aMap[] */ Fts5TokenDataMap *aMap; /* Array of (rowid+pos -> token) mappings */ /* The following are used for prefix-queries only. */ Fts5Buffer terms; /* The following are used for other full-token tokendata queries only. */ int nIter; int nIterAlloc; Fts5PoslistReader *aPoslistReader; int *aPoslistToIter; Fts5Iter *apIter[FLEXARRAY]; }; /* Size in bytes of an Fts5TokenDataIter object holding up to N iterators */ #define SZ_FTS5TOKENDATAITER(N) \ (offsetof(Fts5TokenDataIter,apIter) + (N)*sizeof(Fts5Iter)) /* ** The two input arrays - a1[] and a2[] - are in sorted order. This function ** merges the two arrays together and writes the result to output array ** aOut[]. aOut[] is guaranteed to be large enough to hold the result. ** ** Duplicate entries are copied into the output. So the size of the output ** array is always (n1+n2) entries. */ static void fts5TokendataMerge( Fts5TokenDataMap *a1, int n1, /* Input array 1 */ Fts5TokenDataMap *a2, int n2, /* Input array 2 */ Fts5TokenDataMap *aOut /* Output array */ ){ int i1 = 0; int i2 = 0; assert( n1>=0 && n2>=0 ); while( i1<n1 || i2<n2 ){ Fts5TokenDataMap *pOut = &aOut[i1+i2]; if( i2>=n2 || (i1<n1 && ( a1[i1].iRowid<a2[i2].iRowid || (a1[i1].iRowid==a2[i2].iRowid && a1[i1].iPos<=a2[i2].iPos) ))){ memcpy(pOut, &a1[i1], sizeof(Fts5TokenDataMap)); i1++; }else{ memcpy(pOut, &a2[i2], sizeof(Fts5TokenDataMap)); i2++; } } } /* ** Append a mapping to the token-map belonging to object pT. */ static void fts5TokendataIterAppendMap( Fts5Index *p, Fts5TokenDataIter *pT, int iIter, int nByte, i64 iRowid, i64 iPos ){ if( p->rc==SQLITE_OK ){ if( pT->nMap==pT->nMapAlloc ){ int nNew = pT->nMapAlloc ? pT->nMapAlloc*2 : 64; int nAlloc = nNew * sizeof(Fts5TokenDataMap); Fts5TokenDataMap *aNew; aNew = (Fts5TokenDataMap*)sqlite3_realloc(pT->aMap, nAlloc); if( aNew==0 ){ p->rc = SQLITE_NOMEM; return; } pT->aMap = aNew; pT->nMapAlloc = nNew; } pT->aMap[pT->nMap].iRowid = iRowid; pT->aMap[pT->nMap].iPos = iPos; pT->aMap[pT->nMap].iIter = iIter; pT->aMap[pT->nMap].nByte = nByte; pT->nMap++; } } /* ** Sort the contents of the pT->aMap[] array. ** ** The sorting algorithm requires a malloc(). If this fails, an error code ** is left in Fts5Index.rc before returning. */ static void fts5TokendataIterSortMap(Fts5Index *p, Fts5TokenDataIter *pT){ Fts5TokenDataMap *aTmp = 0; int nByte = pT->nMap * sizeof(Fts5TokenDataMap); aTmp = (Fts5TokenDataMap*)sqlite3Fts5MallocZero(&p->rc, nByte); if( aTmp ){ Fts5TokenDataMap *a1 = pT->aMap; Fts5TokenDataMap *a2 = aTmp; i64 nHalf; for(nHalf=1; nHalf<pT->nMap; nHalf=nHalf*2){ int i1; for(i1=0; i1<pT->nMap; i1+=(nHalf*2)){ int n1 = MIN(nHalf, pT->nMap-i1); int n2 = MIN(nHalf, pT->nMap-i1-n1); fts5TokendataMerge(&a1[i1], n1, &a1[i1+n1], n2, &a2[i1]); } SWAPVAL(Fts5TokenDataMap*, a1, a2); } if( a1!=pT->aMap ){ memcpy(pT->aMap, a1, pT->nMap*sizeof(Fts5TokenDataMap)); } sqlite3_free(aTmp); #ifdef SQLITE_DEBUG { int ii; for(ii=1; ii<pT->nMap; ii++){ Fts5TokenDataMap *p1 = &pT->aMap[ii-1]; Fts5TokenDataMap *p2 = &pT->aMap[ii]; assert( p1->iRowid<p2->iRowid || (p1->iRowid==p2->iRowid && p1->iPos<=p2->iPos) ); } } #endif } } /* ** Delete an Fts5TokenDataIter structure and its contents. */ static void fts5TokendataIterDelete(Fts5TokenDataIter *pSet){ if( pSet ){ int ii; for(ii=0; ii<pSet->nIter; ii++){ fts5MultiIterFree(pSet->apIter[ii]); } fts5BufferFree(&pSet->terms); sqlite3_free(pSet->aPoslistReader); sqlite3_free(pSet->aMap); sqlite3_free(pSet); } } /* ** fts5VisitEntries() context object used by fts5SetupPrefixIterTokendata() ** to pass data to prefixIterSetupTokendataCb(). */ typedef struct TokendataSetupCtx TokendataSetupCtx; struct TokendataSetupCtx { Fts5TokenDataIter *pT; /* Object being populated with mappings */ int iTermOff; /* Offset of current term in terms.p[] */ int nTermByte; /* Size of current term in bytes */ }; /* ** fts5VisitEntries() callback used by fts5SetupPrefixIterTokendata(). This ** callback adds an entry to the Fts5TokenDataIter.aMap[] array for each ** position in the current position-list. It doesn't matter that some of ** these may be out of order - they will be sorted later. */ static void prefixIterSetupTokendataCb( Fts5Index *p, void *pCtx, Fts5Iter *p1, const u8 *pNew, int nNew ){ TokendataSetupCtx *pSetup = (TokendataSetupCtx*)pCtx; int iPosOff = 0; i64 iPos = 0; if( pNew ){ pSetup->nTermByte = nNew-1; pSetup->iTermOff = pSetup->pT->terms.n; fts5BufferAppendBlob(&p->rc, &pSetup->pT->terms, nNew-1, pNew+1); } while( 0==sqlite3Fts5PoslistNext64( p1->base.pData, p1->base.nData, &iPosOff, &iPos ) ){ fts5TokendataIterAppendMap(p, pSetup->pT, pSetup->iTermOff, pSetup->nTermByte, p1->base.iRowid, iPos ); } } /* ** Context object passed by fts5SetupPrefixIter() to fts5VisitEntries(). */ typedef struct PrefixSetupCtx PrefixSetupCtx; struct PrefixSetupCtx { void (*xMerge)(Fts5Index*, Fts5Buffer*, int, Fts5Buffer*); void (*xAppend)(Fts5Index*, u64, Fts5Iter*, Fts5Buffer*); i64 iLastRowid; int nMerge; Fts5Buffer *aBuf; int nBuf; Fts5Buffer doclist; TokendataSetupCtx *pTokendata; }; /* ** fts5VisitEntries() callback used by fts5SetupPrefixIter() */ static void prefixIterSetupCb( Fts5Index *p, void *pCtx, Fts5Iter *p1, const u8 *pNew, int nNew ){ PrefixSetupCtx *pSetup = (PrefixSetupCtx*)pCtx; const int nMerge = pSetup->nMerge; if( p1->base.nData>0 ){ if( p1->base.iRowid<=pSetup->iLastRowid && pSetup->doclist.n>0 ){ int i; for(i=0; p->rc==SQLITE_OK && pSetup->doclist.n; i++){ int i1 = i*nMerge; int iStore; assert( i1+nMerge<=pSetup->nBuf ); for(iStore=i1; iStore<i1+nMerge; iStore++){ if( pSetup->aBuf[iStore].n==0 ){ fts5BufferSwap(&pSetup->doclist, &pSetup->aBuf[iStore]); fts5BufferZero(&pSetup->doclist); break; } } if( iStore==i1+nMerge ){ pSetup->xMerge(p, &pSetup->doclist, nMerge, &pSetup->aBuf[i1]); for(iStore=i1; iStore<i1+nMerge; iStore++){ fts5BufferZero(&pSetup->aBuf[iStore]); } } } pSetup->iLastRowid = 0; } pSetup->xAppend( p, (u64)p1->base.iRowid-(u64)pSetup->iLastRowid, p1, &pSetup->doclist ); pSetup->iLastRowid = p1->base.iRowid; } if( pSetup->pTokendata ){ prefixIterSetupTokendataCb(p, (void*)pSetup->pTokendata, p1, pNew, nNew); } } static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ int iIdx, /* Index to scan for data */ u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset, /* Restrict matches to these columns */ Fts5Iter **ppIter /* OUT: New iterator */ ){ Fts5Structure *pStruct; PrefixSetupCtx s; TokendataSetupCtx s2; memset(&s, 0, sizeof(s)); memset(&s2, 0, sizeof(s2)); s.nMerge = 1; s.iLastRowid = 0; s.nBuf = 32; if( iIdx==0 && p->pConfig->eDetail==FTS5_DETAIL_FULL && p->pConfig->bPrefixInsttoken ){ s.pTokendata = &s2; s2.pT = (Fts5TokenDataIter*)fts5IdxMalloc(p, SZ_FTS5TOKENDATAITER(1)); } if( p->pConfig->eDetail==FTS5_DETAIL_NONE ){ s.xMerge = fts5MergeRowidLists; s.xAppend = fts5AppendRowid; }else{ s.nMerge = FTS5_MERGE_NLIST-1; s.nBuf = s.nMerge*8; /* Sufficient to merge (16^8)==(2^32) lists */ s.xMerge = fts5MergePrefixLists; s.xAppend = fts5AppendPoslist; } s.aBuf = (Fts5Buffer*)fts5IdxMalloc(p, sizeof(Fts5Buffer)*s.nBuf); pStruct = fts5StructureRead(p); assert( p->rc!=SQLITE_OK || (s.aBuf && pStruct) ); if( p->rc==SQLITE_OK ){ void *pCtx = (void*)&s; int i; Fts5Data *pData; /* If iIdx is non-zero, then it is the number of a prefix-index for ** prefixes 1 character longer than the prefix being queried for. That ** index contains all the doclists required, except for the one ** corresponding to the prefix itself. That one is extracted from the ** main term index here. */ if( iIdx!=0 ){ pToken[0] = FTS5_MAIN_PREFIX; fts5VisitEntries(p, pColset, pToken, nToken, 0, prefixIterSetupCb, pCtx); } pToken[0] = FTS5_MAIN_PREFIX + iIdx; fts5VisitEntries(p, pColset, pToken, nToken, 1, prefixIterSetupCb, pCtx); assert( (s.nBuf%s.nMerge)==0 ); for(i=0; i<s.nBuf; i+=s.nMerge){ int iFree; if( p->rc==SQLITE_OK ){ s.xMerge(p, &s.doclist, s.nMerge, &s.aBuf[i]); } for(iFree=i; iFree<i+s.nMerge; iFree++){ fts5BufferFree(&s.aBuf[iFree]); } } pData = fts5IdxMalloc(p, sizeof(*pData) + ((i64)s.doclist.n)+FTS5_DATA_ZERO_PADDING); assert( pData!=0 || p->rc!=SQLITE_OK ); if( pData ){ pData->p = (u8*)&pData[1]; pData->nn = pData->szLeaf = s.doclist.n; if( s.doclist.n ) memcpy(pData->p, s.doclist.p, s.doclist.n); fts5MultiIterNew2(p, pData, bDesc, ppIter); } assert( (*ppIter)!=0 || p->rc!=SQLITE_OK ); if( p->rc==SQLITE_OK && s.pTokendata ){ fts5TokendataIterSortMap(p, s2.pT); (*ppIter)->pTokenDataIter = s2.pT; s2.pT = 0; } } fts5TokendataIterDelete(s2.pT); fts5BufferFree(&s.doclist); fts5StructureRelease(pStruct); sqlite3_free(s.aBuf); } /* ** Indicate that all subsequent calls to sqlite3Fts5IndexWrite() pertain ** to the document with rowid iRowid. */ int sqlite3Fts5IndexBeginWrite(Fts5Index *p, int bDelete, i64 iRowid){ assert( p->rc==SQLITE_OK ); /* Allocate the hash table if it has not already been allocated */ if( p->pHash==0 ){ p->rc = sqlite3Fts5HashNew(p->pConfig, &p->pHash, &p->nPendingData); } /* Flush the hash table to disk if required */ if( iRowid<p->iWriteRowid || (iRowid==p->iWriteRowid && p->bDelete==0) || (p->nPendingData > p->pConfig->nHashSize) ){ fts5IndexFlush(p); } p->iWriteRowid = iRowid; p->bDelete = bDelete; if( bDelete==0 ){ p->nPendingRow++; } return fts5IndexReturn(p); } /* ** Commit data to disk. */ int sqlite3Fts5IndexSync(Fts5Index *p){ assert( p->rc==SQLITE_OK ); fts5IndexFlush(p); fts5IndexCloseReader(p); return fts5IndexReturn(p); } /* ** Discard any data stored in the in-memory hash tables. Do not write it ** to the database. Additionally, assume that the contents of the %_data ** table may have changed on disk. So any in-memory caches of %_data ** records must be invalidated. */ int sqlite3Fts5IndexRollback(Fts5Index *p){ fts5IndexCloseReader(p); fts5IndexDiscardData(p); fts5StructureInvalidate(p); return fts5IndexReturn(p); } /* ** The %_data table is completely empty when this function is called. This ** function populates it with the initial structure objects for each index, ** and the initial version of the "averages" record (a zero-byte blob). */ int sqlite3Fts5IndexReinit(Fts5Index *p){ Fts5Structure *pTmp; u8 tmpSpace[SZ_FTS5STRUCTURE(1)]; fts5StructureInvalidate(p); fts5IndexDiscardData(p); pTmp = (Fts5Structure*)tmpSpace; memset(pTmp, 0, SZ_FTS5STRUCTURE(1)); if( p->pConfig->bContentlessDelete ){ pTmp->nOriginCntr = 1; } fts5DataWrite(p, FTS5_AVERAGES_ROWID, (const u8*)"", 0); fts5StructureWrite(p, pTmp); return fts5IndexReturn(p); } /* ** Open a new Fts5Index handle. If the bCreate argument is true, create ** and initialize the underlying %_data table. ** |
︙ | ︙ | |||
5448 5449 5450 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 5461 5462 | assert( p->pReader==0 ); fts5StructureInvalidate(p); sqlite3_finalize(p->pWriter); sqlite3_finalize(p->pDeleter); sqlite3_finalize(p->pIdxWriter); sqlite3_finalize(p->pIdxDeleter); sqlite3_finalize(p->pIdxSelect); sqlite3_finalize(p->pDataVersion); sqlite3Fts5HashFree(p->pHash); sqlite3_free(p->zDataTbl); sqlite3_free(p); } return rc; } | > > | 6887 6888 6889 6890 6891 6892 6893 6894 6895 6896 6897 6898 6899 6900 6901 6902 6903 | assert( p->pReader==0 ); fts5StructureInvalidate(p); sqlite3_finalize(p->pWriter); sqlite3_finalize(p->pDeleter); sqlite3_finalize(p->pIdxWriter); sqlite3_finalize(p->pIdxDeleter); sqlite3_finalize(p->pIdxSelect); sqlite3_finalize(p->pIdxNextSelect); sqlite3_finalize(p->pDataVersion); sqlite3_finalize(p->pDeleteFromIdx); sqlite3Fts5HashFree(p->pHash); sqlite3_free(p->zDataTbl); sqlite3_free(p); } return rc; } |
︙ | ︙ | |||
5541 5542 5543 5544 5545 5546 5547 5548 5549 5550 5551 5552 5553 5554 | nByte ); } } return rc; } /* ** Open a new iterator to iterate though all rowid that match the ** specified token or token prefix. */ int sqlite3Fts5IndexQuery( Fts5Index *p, /* FTS index to query */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 6982 6983 6984 6985 6986 6987 6988 6989 6990 6991 6992 6993 6994 6995 6996 6997 6998 6999 7000 7001 7002 7003 7004 7005 7006 7007 7008 7009 7010 7011 7012 7013 7014 7015 7016 7017 7018 7019 7020 7021 7022 7023 7024 7025 7026 7027 7028 7029 7030 7031 7032 7033 7034 7035 7036 7037 7038 7039 7040 7041 7042 7043 7044 7045 7046 7047 7048 7049 7050 7051 7052 7053 7054 7055 7056 7057 7058 7059 7060 7061 7062 7063 7064 7065 7066 7067 7068 7069 7070 7071 7072 7073 7074 7075 7076 7077 7078 7079 7080 7081 7082 7083 7084 7085 7086 7087 7088 7089 7090 7091 7092 7093 7094 7095 7096 7097 7098 7099 7100 7101 7102 7103 7104 7105 7106 7107 7108 7109 7110 7111 7112 7113 7114 7115 7116 7117 7118 7119 7120 7121 7122 7123 7124 7125 7126 7127 7128 7129 7130 7131 7132 7133 7134 7135 7136 7137 7138 7139 7140 7141 7142 7143 7144 7145 7146 7147 7148 7149 7150 7151 7152 7153 7154 7155 7156 7157 7158 7159 7160 7161 7162 7163 7164 7165 7166 7167 7168 7169 7170 7171 7172 7173 7174 7175 7176 7177 7178 7179 7180 7181 7182 7183 7184 7185 7186 7187 7188 7189 7190 7191 7192 7193 7194 7195 7196 7197 7198 7199 7200 7201 7202 7203 7204 7205 7206 7207 7208 7209 7210 7211 7212 7213 7214 7215 7216 7217 7218 7219 7220 7221 7222 7223 7224 7225 7226 7227 7228 7229 7230 7231 7232 7233 7234 7235 7236 7237 7238 7239 7240 7241 7242 7243 7244 7245 7246 7247 7248 7249 7250 7251 7252 7253 7254 7255 7256 7257 7258 7259 7260 7261 7262 7263 7264 7265 7266 7267 7268 7269 7270 7271 7272 7273 7274 7275 7276 7277 7278 7279 7280 7281 7282 7283 7284 7285 7286 7287 7288 7289 7290 7291 7292 7293 7294 7295 7296 7297 7298 7299 7300 7301 7302 7303 7304 7305 7306 7307 7308 7309 7310 7311 7312 7313 7314 7315 7316 7317 7318 7319 7320 7321 7322 7323 7324 7325 7326 7327 7328 7329 7330 7331 7332 7333 7334 7335 7336 7337 7338 7339 7340 7341 7342 7343 7344 7345 7346 7347 7348 7349 7350 7351 7352 7353 7354 7355 7356 7357 7358 7359 7360 7361 7362 7363 7364 7365 7366 7367 7368 7369 7370 7371 7372 7373 7374 7375 7376 | nByte ); } } return rc; } /* ** pToken points to a buffer of size nToken bytes containing a search ** term, including the index number at the start, used on a tokendata=1 ** table. This function returns true if the term in buffer pBuf matches ** token pToken/nToken. */ static int fts5IsTokendataPrefix( Fts5Buffer *pBuf, const u8 *pToken, int nToken ){ return ( pBuf->n>=nToken && 0==memcmp(pBuf->p, pToken, nToken) && (pBuf->n==nToken || pBuf->p[nToken]==0x00) ); } /* ** Ensure the segment-iterator passed as the only argument points to EOF. */ static void fts5SegIterSetEOF(Fts5SegIter *pSeg){ fts5DataRelease(pSeg->pLeaf); pSeg->pLeaf = 0; } static void fts5IterClose(Fts5IndexIter *pIndexIter){ if( pIndexIter ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5Index *pIndex = pIter->pIndex; fts5TokendataIterDelete(pIter->pTokenDataIter); fts5MultiIterFree(pIter); fts5IndexCloseReader(pIndex); } } /* ** This function appends iterator pAppend to Fts5TokenDataIter pIn and ** returns the result. */ static Fts5TokenDataIter *fts5AppendTokendataIter( Fts5Index *p, /* Index object (for error code) */ Fts5TokenDataIter *pIn, /* Current Fts5TokenDataIter struct */ Fts5Iter *pAppend /* Append this iterator */ ){ Fts5TokenDataIter *pRet = pIn; if( p->rc==SQLITE_OK ){ if( pIn==0 || pIn->nIter==pIn->nIterAlloc ){ int nAlloc = pIn ? pIn->nIterAlloc*2 : 16; int nByte = SZ_FTS5TOKENDATAITER(nAlloc+1); Fts5TokenDataIter *pNew = (Fts5TokenDataIter*)sqlite3_realloc(pIn, nByte); if( pNew==0 ){ p->rc = SQLITE_NOMEM; }else{ if( pIn==0 ) memset(pNew, 0, nByte); pRet = pNew; pNew->nIterAlloc = nAlloc; } } } if( p->rc ){ fts5IterClose((Fts5IndexIter*)pAppend); }else{ pRet->apIter[pRet->nIter++] = pAppend; } assert( pRet==0 || pRet->nIter<=pRet->nIterAlloc ); return pRet; } /* ** The iterator passed as the only argument must be a tokendata=1 iterator ** (pIter->pTokenDataIter!=0). This function sets the iterator output ** variables (pIter->base.*) according to the contents of the current ** row. */ static void fts5IterSetOutputsTokendata(Fts5Iter *pIter){ int ii; int nHit = 0; i64 iRowid = SMALLEST_INT64; int iMin = 0; Fts5TokenDataIter *pT = pIter->pTokenDataIter; pIter->base.nData = 0; pIter->base.pData = 0; for(ii=0; ii<pT->nIter; ii++){ Fts5Iter *p = pT->apIter[ii]; if( p->base.bEof==0 ){ if( nHit==0 || p->base.iRowid<iRowid ){ iRowid = p->base.iRowid; nHit = 1; pIter->base.pData = p->base.pData; pIter->base.nData = p->base.nData; iMin = ii; }else if( p->base.iRowid==iRowid ){ nHit++; } } } if( nHit==0 ){ pIter->base.bEof = 1; }else{ int eDetail = pIter->pIndex->pConfig->eDetail; pIter->base.bEof = 0; pIter->base.iRowid = iRowid; if( nHit==1 && eDetail==FTS5_DETAIL_FULL ){ fts5TokendataIterAppendMap(pIter->pIndex, pT, iMin, 0, iRowid, -1); }else if( nHit>1 && eDetail!=FTS5_DETAIL_NONE ){ int nReader = 0; int nByte = 0; i64 iPrev = 0; /* Allocate array of iterators if they are not already allocated. */ if( pT->aPoslistReader==0 ){ pT->aPoslistReader = (Fts5PoslistReader*)sqlite3Fts5MallocZero( &pIter->pIndex->rc, pT->nIter * (sizeof(Fts5PoslistReader) + sizeof(int)) ); if( pT->aPoslistReader==0 ) return; pT->aPoslistToIter = (int*)&pT->aPoslistReader[pT->nIter]; } /* Populate an iterator for each poslist that will be merged */ for(ii=0; ii<pT->nIter; ii++){ Fts5Iter *p = pT->apIter[ii]; if( iRowid==p->base.iRowid ){ pT->aPoslistToIter[nReader] = ii; sqlite3Fts5PoslistReaderInit( p->base.pData, p->base.nData, &pT->aPoslistReader[nReader++] ); nByte += p->base.nData; } } /* Ensure the output buffer is large enough */ if( fts5BufferGrow(&pIter->pIndex->rc, &pIter->poslist, nByte+nHit*10) ){ return; } /* Ensure the token-mapping is large enough */ if( eDetail==FTS5_DETAIL_FULL && pT->nMapAlloc<(pT->nMap + nByte) ){ int nNew = (pT->nMapAlloc + nByte) * 2; Fts5TokenDataMap *aNew = (Fts5TokenDataMap*)sqlite3_realloc( pT->aMap, nNew*sizeof(Fts5TokenDataMap) ); if( aNew==0 ){ pIter->pIndex->rc = SQLITE_NOMEM; return; } pT->aMap = aNew; pT->nMapAlloc = nNew; } pIter->poslist.n = 0; while( 1 ){ i64 iMinPos = LARGEST_INT64; /* Find smallest position */ iMin = 0; for(ii=0; ii<nReader; ii++){ Fts5PoslistReader *pReader = &pT->aPoslistReader[ii]; if( pReader->bEof==0 ){ if( pReader->iPos<iMinPos ){ iMinPos = pReader->iPos; iMin = ii; } } } /* If all readers were at EOF, break out of the loop. */ if( iMinPos==LARGEST_INT64 ) break; sqlite3Fts5PoslistSafeAppend(&pIter->poslist, &iPrev, iMinPos); sqlite3Fts5PoslistReaderNext(&pT->aPoslistReader[iMin]); if( eDetail==FTS5_DETAIL_FULL ){ pT->aMap[pT->nMap].iPos = iMinPos; pT->aMap[pT->nMap].iIter = pT->aPoslistToIter[iMin]; pT->aMap[pT->nMap].iRowid = iRowid; pT->nMap++; } } pIter->base.pData = pIter->poslist.p; pIter->base.nData = pIter->poslist.n; } } } /* ** The iterator passed as the only argument must be a tokendata=1 iterator ** (pIter->pTokenDataIter!=0). This function advances the iterator. If ** argument bFrom is false, then the iterator is advanced to the next ** entry. Or, if bFrom is true, it is advanced to the first entry with ** a rowid of iFrom or greater. */ static void fts5TokendataIterNext(Fts5Iter *pIter, int bFrom, i64 iFrom){ int ii; Fts5TokenDataIter *pT = pIter->pTokenDataIter; Fts5Index *pIndex = pIter->pIndex; for(ii=0; ii<pT->nIter; ii++){ Fts5Iter *p = pT->apIter[ii]; if( p->base.bEof==0 && (p->base.iRowid==pIter->base.iRowid || (bFrom && p->base.iRowid<iFrom)) ){ fts5MultiIterNext(pIndex, p, bFrom, iFrom); while( bFrom && p->base.bEof==0 && p->base.iRowid<iFrom && pIndex->rc==SQLITE_OK ){ fts5MultiIterNext(pIndex, p, 0, 0); } } } if( pIndex->rc==SQLITE_OK ){ fts5IterSetOutputsTokendata(pIter); } } /* ** If the segment-iterator passed as the first argument is at EOF, then ** set pIter->term to a copy of buffer pTerm. */ static void fts5TokendataSetTermIfEof(Fts5Iter *pIter, Fts5Buffer *pTerm){ if( pIter && pIter->aSeg[0].pLeaf==0 ){ fts5BufferSet(&pIter->pIndex->rc, &pIter->aSeg[0].term, pTerm->n, pTerm->p); } } /* ** This function sets up an iterator to use for a non-prefix query on a ** tokendata=1 table. */ static Fts5Iter *fts5SetupTokendataIter( Fts5Index *p, /* FTS index to query */ const u8 *pToken, /* Buffer containing query term */ int nToken, /* Size of buffer pToken in bytes */ Fts5Colset *pColset /* Colset to filter on */ ){ Fts5Iter *pRet = 0; Fts5TokenDataIter *pSet = 0; Fts5Structure *pStruct = 0; const int flags = FTS5INDEX_QUERY_SCANONETERM | FTS5INDEX_QUERY_SCAN; Fts5Buffer bSeek = {0, 0, 0}; Fts5Buffer *pSmall = 0; fts5IndexFlush(p); pStruct = fts5StructureRead(p); while( p->rc==SQLITE_OK ){ Fts5Iter *pPrev = pSet ? pSet->apIter[pSet->nIter-1] : 0; Fts5Iter *pNew = 0; Fts5SegIter *pNewIter = 0; Fts5SegIter *pPrevIter = 0; int iLvl, iSeg, ii; pNew = fts5MultiIterAlloc(p, pStruct->nSegment); if( pSmall ){ fts5BufferSet(&p->rc, &bSeek, pSmall->n, pSmall->p); fts5BufferAppendBlob(&p->rc, &bSeek, 1, (const u8*)"\0"); }else{ fts5BufferSet(&p->rc, &bSeek, nToken, pToken); } if( p->rc ){ fts5IterClose((Fts5IndexIter*)pNew); break; } pNewIter = &pNew->aSeg[0]; pPrevIter = (pPrev ? &pPrev->aSeg[0] : 0); for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ for(iSeg=pStruct->aLevel[iLvl].nSeg-1; iSeg>=0; iSeg--){ Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg]; int bDone = 0; if( pPrevIter ){ if( fts5BufferCompare(pSmall, &pPrevIter->term) ){ memcpy(pNewIter, pPrevIter, sizeof(Fts5SegIter)); memset(pPrevIter, 0, sizeof(Fts5SegIter)); bDone = 1; }else if( pPrevIter->iEndofDoclist>pPrevIter->pLeaf->szLeaf ){ fts5SegIterNextInit(p,(const char*)bSeek.p,bSeek.n-1,pSeg,pNewIter); bDone = 1; } } if( bDone==0 ){ fts5SegIterSeekInit(p, bSeek.p, bSeek.n, flags, pSeg, pNewIter); } if( pPrevIter ){ if( pPrevIter->pTombArray ){ pNewIter->pTombArray = pPrevIter->pTombArray; pNewIter->pTombArray->nRef++; } }else{ fts5SegIterAllocTombstone(p, pNewIter); } pNewIter++; if( pPrevIter ) pPrevIter++; if( p->rc ) break; } } fts5TokendataSetTermIfEof(pPrev, pSmall); pNew->bSkipEmpty = 1; pNew->pColset = pColset; fts5IterSetOutputCb(&p->rc, pNew); /* Loop through all segments in the new iterator. Find the smallest ** term that any segment-iterator points to. Iterator pNew will be ** used for this term. Also, set any iterator that points to a term that ** does not match pToken/nToken to point to EOF */ pSmall = 0; for(ii=0; ii<pNew->nSeg; ii++){ Fts5SegIter *pII = &pNew->aSeg[ii]; if( 0==fts5IsTokendataPrefix(&pII->term, pToken, nToken) ){ fts5SegIterSetEOF(pII); } if( pII->pLeaf && (!pSmall || fts5BufferCompare(pSmall, &pII->term)>0) ){ pSmall = &pII->term; } } /* If pSmall is still NULL at this point, then the new iterator does ** not point to any terms that match the query. So delete it and break ** out of the loop - all required iterators have been collected. */ if( pSmall==0 ){ fts5IterClose((Fts5IndexIter*)pNew); break; } /* Append this iterator to the set and continue. */ pSet = fts5AppendTokendataIter(p, pSet, pNew); } if( p->rc==SQLITE_OK && pSet ){ int ii; for(ii=0; ii<pSet->nIter; ii++){ Fts5Iter *pIter = pSet->apIter[ii]; int iSeg; for(iSeg=0; iSeg<pIter->nSeg; iSeg++){ pIter->aSeg[iSeg].flags |= FTS5_SEGITER_ONETERM; } fts5MultiIterFinishSetup(p, pIter); } } if( p->rc==SQLITE_OK ){ pRet = fts5MultiIterAlloc(p, 0); } if( pRet ){ pRet->nSeg = 0; pRet->pTokenDataIter = pSet; if( pSet ){ fts5IterSetOutputsTokendata(pRet); }else{ pRet->base.bEof = 1; } }else{ fts5TokendataIterDelete(pSet); } fts5StructureRelease(pStruct); fts5BufferFree(&bSeek); return pRet; } /* ** Open a new iterator to iterate though all rowid that match the ** specified token or token prefix. */ int sqlite3Fts5IndexQuery( Fts5Index *p, /* FTS index to query */ |
︙ | ︙ | |||
5563 5564 5565 5566 5567 5568 5569 5570 5571 5572 5573 5574 5575 5576 5577 | /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || flags==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){ int iIdx = 0; /* Index to search */ int iPrefixIdx = 0; /* +1 prefix index */ if( nToken>0 ) memcpy(&buf.p[1], pToken, nToken); /* Figure out which index to search and set iIdx accordingly. If this ** is a prefix query for which there is no prefix index, set iIdx to ** greater than pConfig->nPrefix to indicate that the query will be ** satisfied by scanning multiple terms in the main index. ** ** If the QUERY_TEST_NOIDX flag was specified, then this must be a | > > > > > > > > > > > | 7385 7386 7387 7388 7389 7390 7391 7392 7393 7394 7395 7396 7397 7398 7399 7400 7401 7402 7403 7404 7405 7406 7407 7408 7409 7410 | /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || flags==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferSize(&p->rc, &buf, nToken+1)==0 ){ int iIdx = 0; /* Index to search */ int iPrefixIdx = 0; /* +1 prefix index */ int bTokendata = pConfig->bTokendata; assert( buf.p!=0 ); if( nToken>0 ) memcpy(&buf.p[1], pToken, nToken); /* The NOTOKENDATA flag is set when each token in a tokendata=1 table ** should be treated individually, instead of merging all those with ** a common prefix into a single entry. This is used, for example, by ** queries performed as part of an integrity-check, or by the fts5vocab ** module. */ if( flags & (FTS5INDEX_QUERY_NOTOKENDATA|FTS5INDEX_QUERY_SCAN) ){ bTokendata = 0; } /* Figure out which index to search and set iIdx accordingly. If this ** is a prefix query for which there is no prefix index, set iIdx to ** greater than pConfig->nPrefix to indicate that the query will be ** satisfied by scanning multiple terms in the main index. ** ** If the QUERY_TEST_NOIDX flag was specified, then this must be a |
︙ | ︙ | |||
5590 5591 5592 5593 5594 5595 5596 | for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){ int nIdxChar = pConfig->aPrefix[iIdx-1]; if( nIdxChar==nChar ) break; if( nIdxChar==nChar+1 ) iPrefixIdx = iIdx; } } | > > > | | | | > > > > | > | 7423 7424 7425 7426 7427 7428 7429 7430 7431 7432 7433 7434 7435 7436 7437 7438 7439 7440 7441 7442 7443 7444 7445 7446 7447 7448 7449 7450 7451 7452 7453 7454 7455 7456 7457 7458 7459 7460 7461 7462 7463 7464 7465 7466 7467 7468 7469 7470 7471 7472 7473 7474 7475 7476 7477 7478 7479 7480 7481 7482 7483 7484 7485 7486 7487 7488 7489 7490 7491 7492 | for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){ int nIdxChar = pConfig->aPrefix[iIdx-1]; if( nIdxChar==nChar ) break; if( nIdxChar==nChar+1 ) iPrefixIdx = iIdx; } } if( bTokendata && iIdx==0 ){ buf.p[0] = FTS5_MAIN_PREFIX; pRet = fts5SetupTokendataIter(p, buf.p, nToken+1, pColset); }else if( iIdx<=pConfig->nPrefix ){ /* Straight index lookup */ Fts5Structure *pStruct = fts5StructureRead(p); buf.p[0] = (u8)(FTS5_MAIN_PREFIX + iIdx); if( pStruct ){ fts5MultiIterNew(p, pStruct, flags | FTS5INDEX_QUERY_SKIPEMPTY, pColset, buf.p, nToken+1, -1, 0, &pRet ); fts5StructureRelease(pStruct); } }else{ /* Scan multiple terms in the main index for a prefix query. */ int bDesc = (flags & FTS5INDEX_QUERY_DESC)!=0; fts5SetupPrefixIter(p, bDesc, iPrefixIdx, buf.p, nToken+1, pColset,&pRet); if( pRet==0 ){ assert( p->rc!=SQLITE_OK ); }else{ assert( pRet->pColset==0 ); fts5IterSetOutputCb(&p->rc, pRet); if( p->rc==SQLITE_OK ){ Fts5SegIter *pSeg = &pRet->aSeg[pRet->aFirst[1].iFirst]; if( pSeg->pLeaf ) pRet->xSetOutputs(pRet, pSeg); } } } if( p->rc ){ fts5IterClose((Fts5IndexIter*)pRet); pRet = 0; fts5IndexCloseReader(p); } *ppIter = (Fts5IndexIter*)pRet; sqlite3Fts5BufferFree(&buf); } return fts5IndexReturn(p); } /* ** Return true if the iterator passed as the only argument is at EOF. */ /* ** Move to the next matching rowid. */ int sqlite3Fts5IterNext(Fts5IndexIter *pIndexIter){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; assert( pIter->pIndex->rc==SQLITE_OK ); if( pIter->nSeg==0 ){ assert( pIter->pTokenDataIter ); fts5TokendataIterNext(pIter, 0, 0); }else{ fts5MultiIterNext(pIter->pIndex, pIter, 0, 0); } return fts5IndexReturn(pIter->pIndex); } /* ** Move to the next matching term/rowid. Used by the fts5vocab module. */ int sqlite3Fts5IterNextScan(Fts5IndexIter *pIndexIter){ |
︙ | ︙ | |||
5670 5671 5672 5673 5674 5675 5676 | /* ** Move to the next matching rowid that occurs at or after iMatch. The ** definition of "at or after" depends on whether this iterator iterates ** in ascending or descending rowid order. */ int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIndexIter, i64 iMatch){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; | > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < | | | | 7511 7512 7513 7514 7515 7516 7517 7518 7519 7520 7521 7522 7523 7524 7525 7526 7527 7528 7529 7530 7531 7532 7533 7534 7535 7536 7537 7538 7539 7540 7541 7542 7543 7544 7545 7546 7547 7548 7549 7550 7551 7552 7553 7554 7555 7556 7557 7558 7559 7560 7561 7562 7563 7564 7565 7566 7567 7568 7569 7570 7571 7572 7573 7574 7575 7576 7577 7578 7579 7580 7581 7582 7583 7584 7585 7586 7587 7588 7589 7590 7591 7592 7593 7594 7595 7596 7597 7598 7599 7600 7601 7602 7603 7604 7605 7606 7607 7608 7609 7610 7611 7612 7613 7614 7615 7616 7617 7618 7619 7620 7621 7622 7623 7624 7625 7626 7627 7628 7629 7630 7631 7632 7633 7634 7635 7636 7637 7638 7639 7640 7641 7642 7643 7644 7645 7646 7647 7648 7649 7650 7651 7652 7653 7654 7655 7656 7657 7658 7659 7660 7661 7662 7663 7664 7665 7666 7667 7668 7669 7670 7671 7672 7673 7674 7675 7676 7677 7678 7679 7680 7681 7682 7683 7684 7685 7686 7687 7688 7689 7690 7691 7692 7693 7694 7695 7696 7697 7698 7699 7700 7701 7702 7703 7704 7705 7706 7707 7708 7709 7710 7711 7712 7713 7714 7715 7716 7717 7718 7719 7720 7721 7722 7723 | /* ** Move to the next matching rowid that occurs at or after iMatch. The ** definition of "at or after" depends on whether this iterator iterates ** in ascending or descending rowid order. */ int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIndexIter, i64 iMatch){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; if( pIter->nSeg==0 ){ assert( pIter->pTokenDataIter ); fts5TokendataIterNext(pIter, 1, iMatch); }else{ fts5MultiIterNextFrom(pIter->pIndex, pIter, iMatch); } return fts5IndexReturn(pIter->pIndex); } /* ** Return the current term. */ const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIndexIter, int *pn){ int n; const char *z = (const char*)fts5MultiIterTerm((Fts5Iter*)pIndexIter, &n); assert_nc( z || n<=1 ); *pn = n-1; return (z ? &z[1] : 0); } /* ** pIter is a prefix query. This function populates pIter->pTokenDataIter ** with an Fts5TokenDataIter object containing mappings for all rows ** matched by the query. */ static int fts5SetupPrefixIterTokendata( Fts5Iter *pIter, const char *pToken, /* Token prefix to search for */ int nToken /* Size of pToken in bytes */ ){ Fts5Index *p = pIter->pIndex; Fts5Buffer token = {0, 0, 0}; TokendataSetupCtx ctx; memset(&ctx, 0, sizeof(ctx)); fts5BufferGrow(&p->rc, &token, nToken+1); assert( token.p!=0 || p->rc!=SQLITE_OK ); ctx.pT = (Fts5TokenDataIter*)sqlite3Fts5MallocZero(&p->rc, SZ_FTS5TOKENDATAITER(1)); if( p->rc==SQLITE_OK ){ /* Fill in the token prefix to search for */ token.p[0] = FTS5_MAIN_PREFIX; memcpy(&token.p[1], pToken, nToken); token.n = nToken+1; fts5VisitEntries( p, 0, token.p, token.n, 1, prefixIterSetupTokendataCb, (void*)&ctx ); fts5TokendataIterSortMap(p, ctx.pT); } if( p->rc==SQLITE_OK ){ pIter->pTokenDataIter = ctx.pT; }else{ fts5TokendataIterDelete(ctx.pT); } fts5BufferFree(&token); return fts5IndexReturn(p); } /* ** This is used by xInstToken() to access the token at offset iOff, column ** iCol of row iRowid. The token is returned via output variables *ppOut ** and *pnOut. The iterator passed as the first argument must be a tokendata=1 ** iterator (pIter->pTokenDataIter!=0). ** ** pToken/nToken: */ int sqlite3Fts5IterToken( Fts5IndexIter *pIndexIter, const char *pToken, int nToken, i64 iRowid, int iCol, int iOff, const char **ppOut, int *pnOut ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5TokenDataIter *pT = pIter->pTokenDataIter; i64 iPos = (((i64)iCol)<<32) + iOff; Fts5TokenDataMap *aMap = 0; int i1 = 0; int i2 = 0; int iTest = 0; assert( pT || (pToken && pIter->nSeg>0) ); if( pT==0 ){ int rc = fts5SetupPrefixIterTokendata(pIter, pToken, nToken); if( rc!=SQLITE_OK ) return rc; pT = pIter->pTokenDataIter; } i2 = pT->nMap; aMap = pT->aMap; while( i2>i1 ){ iTest = (i1 + i2) / 2; if( aMap[iTest].iRowid<iRowid ){ i1 = iTest+1; }else if( aMap[iTest].iRowid>iRowid ){ i2 = iTest; }else{ if( aMap[iTest].iPos<iPos ){ if( aMap[iTest].iPos<0 ){ break; } i1 = iTest+1; }else if( aMap[iTest].iPos>iPos ){ i2 = iTest; }else{ break; } } } if( i2>i1 ){ if( pIter->nSeg==0 ){ Fts5Iter *pMap = pT->apIter[aMap[iTest].iIter]; *ppOut = (const char*)pMap->aSeg[0].term.p+1; *pnOut = pMap->aSeg[0].term.n-1; }else{ Fts5TokenDataMap *p = &aMap[iTest]; *ppOut = (const char*)&pT->terms.p[p->iIter]; *pnOut = aMap[iTest].nByte; } } return SQLITE_OK; } /* ** Clear any existing entries from the token-map associated with the ** iterator passed as the only argument. */ void sqlite3Fts5IndexIterClearTokendata(Fts5IndexIter *pIndexIter){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; if( pIter && pIter->pTokenDataIter && (pIter->nSeg==0 || pIter->pIndex->pConfig->eDetail!=FTS5_DETAIL_FULL) ){ pIter->pTokenDataIter->nMap = 0; } } /* ** Set a token-mapping for the iterator passed as the first argument. This ** is used in detail=column or detail=none mode when a token is requested ** using the xInstToken() API. In this case the caller tokenizers the ** current row and configures the token-mapping via multiple calls to this ** function. */ int sqlite3Fts5IndexIterWriteTokendata( Fts5IndexIter *pIndexIter, const char *pToken, int nToken, i64 iRowid, int iCol, int iOff ){ Fts5Iter *pIter = (Fts5Iter*)pIndexIter; Fts5TokenDataIter *pT = pIter->pTokenDataIter; Fts5Index *p = pIter->pIndex; i64 iPos = (((i64)iCol)<<32) + iOff; assert( p->pConfig->eDetail!=FTS5_DETAIL_FULL ); assert( pIter->pTokenDataIter || pIter->nSeg>0 ); if( pIter->nSeg>0 ){ /* This is a prefix term iterator. */ if( pT==0 ){ pT = (Fts5TokenDataIter*)sqlite3Fts5MallocZero(&p->rc, SZ_FTS5TOKENDATAITER(1)); pIter->pTokenDataIter = pT; } if( pT ){ fts5TokendataIterAppendMap(p, pT, pT->terms.n, nToken, iRowid, iPos); fts5BufferAppendBlob(&p->rc, &pT->terms, nToken, (const u8*)pToken); } }else{ int ii; for(ii=0; ii<pT->nIter; ii++){ Fts5Buffer *pTerm = &pT->apIter[ii]->aSeg[0].term; if( nToken==pTerm->n-1 && memcmp(pToken, pTerm->p+1, nToken)==0 ) break; } if( ii<pT->nIter ){ fts5TokendataIterAppendMap(p, pT, ii, 0, iRowid, iPos); } } return fts5IndexReturn(p); } /* ** Close an iterator opened by an earlier call to sqlite3Fts5IndexQuery(). */ void sqlite3Fts5IterClose(Fts5IndexIter *pIndexIter){ if( pIndexIter ){ Fts5Index *pIndex = ((Fts5Iter*)pIndexIter)->pIndex; fts5IterClose(pIndexIter); fts5IndexReturn(pIndex); } } /* ** Read and decode the "averages" record from the database. ** ** Parameter anSize must point to an array of size nCol, where nCol is |
︙ | ︙ | |||
5775 5776 5777 5778 5779 5780 5781 5782 5783 5784 5785 5786 5787 5788 | int sqlite3Fts5IndexLoadConfig(Fts5Index *p){ Fts5Structure *pStruct; pStruct = fts5StructureRead(p); fts5StructureRelease(pStruct); return fts5IndexReturn(p); } /************************************************************************* ************************************************************************** ** Below this point is the implementation of the integrity-check ** functionality. */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 7791 7792 7793 7794 7795 7796 7797 7798 7799 7800 7801 7802 7803 7804 7805 7806 7807 7808 7809 7810 7811 7812 7813 7814 7815 7816 7817 7818 7819 7820 7821 7822 7823 7824 7825 7826 7827 7828 7829 7830 7831 7832 7833 7834 7835 7836 7837 7838 7839 7840 7841 7842 7843 7844 7845 7846 7847 7848 7849 7850 7851 7852 7853 7854 7855 7856 7857 7858 7859 7860 7861 7862 7863 7864 7865 7866 7867 7868 7869 7870 7871 7872 7873 7874 7875 7876 7877 7878 7879 7880 7881 7882 7883 7884 7885 7886 7887 7888 7889 7890 7891 7892 7893 7894 7895 7896 7897 7898 7899 7900 7901 7902 7903 7904 7905 7906 7907 7908 7909 7910 7911 7912 7913 7914 7915 7916 7917 7918 7919 7920 7921 7922 7923 7924 7925 7926 7927 7928 7929 7930 7931 7932 7933 7934 7935 7936 7937 7938 7939 7940 7941 7942 7943 7944 7945 7946 7947 7948 7949 7950 7951 7952 7953 7954 7955 7956 7957 7958 7959 7960 7961 7962 7963 7964 7965 7966 7967 7968 7969 7970 7971 7972 7973 7974 7975 7976 7977 7978 7979 7980 7981 7982 7983 7984 7985 7986 7987 7988 7989 7990 7991 7992 7993 7994 7995 7996 7997 7998 7999 8000 8001 8002 8003 8004 8005 8006 8007 8008 8009 8010 8011 8012 8013 8014 8015 8016 8017 8018 8019 8020 8021 8022 8023 8024 8025 8026 8027 8028 8029 8030 8031 8032 8033 8034 8035 8036 8037 8038 8039 8040 8041 8042 8043 8044 8045 8046 8047 8048 8049 8050 8051 8052 8053 8054 8055 8056 8057 8058 8059 8060 8061 8062 8063 8064 8065 8066 8067 8068 8069 8070 8071 8072 8073 8074 8075 8076 8077 8078 8079 8080 8081 8082 8083 8084 8085 8086 8087 8088 8089 8090 8091 8092 8093 8094 8095 8096 8097 8098 8099 8100 8101 8102 8103 8104 8105 8106 8107 8108 8109 8110 8111 8112 8113 8114 8115 8116 8117 8118 8119 8120 8121 8122 8123 8124 8125 8126 8127 8128 8129 8130 8131 8132 8133 8134 8135 8136 8137 8138 8139 8140 8141 8142 8143 8144 8145 | int sqlite3Fts5IndexLoadConfig(Fts5Index *p){ Fts5Structure *pStruct; pStruct = fts5StructureRead(p); fts5StructureRelease(pStruct); return fts5IndexReturn(p); } /* ** Retrieve the origin value that will be used for the segment currently ** being accumulated in the in-memory hash table when it is flushed to ** disk. If successful, SQLITE_OK is returned and (*piOrigin) set to ** the queried value. Or, if an error occurs, an error code is returned ** and the final value of (*piOrigin) is undefined. */ int sqlite3Fts5IndexGetOrigin(Fts5Index *p, i64 *piOrigin){ Fts5Structure *pStruct; pStruct = fts5StructureRead(p); if( pStruct ){ *piOrigin = pStruct->nOriginCntr; fts5StructureRelease(pStruct); } return fts5IndexReturn(p); } /* ** Buffer pPg contains a page of a tombstone hash table - one of nPg pages ** associated with the same segment. This function adds rowid iRowid to ** the hash table. The caller is required to guarantee that there is at ** least one free slot on the page. ** ** If parameter bForce is false and the hash table is deemed to be full ** (more than half of the slots are occupied), then non-zero is returned ** and iRowid not inserted. Or, if bForce is true or if the hash table page ** is not full, iRowid is inserted and zero returned. */ static int fts5IndexTombstoneAddToPage( Fts5Data *pPg, int bForce, int nPg, u64 iRowid ){ const int szKey = TOMBSTONE_KEYSIZE(pPg); const int nSlot = TOMBSTONE_NSLOT(pPg); const int nElem = fts5GetU32(&pPg->p[4]); int iSlot = (iRowid / nPg) % nSlot; int nCollide = nSlot; if( szKey==4 && iRowid>0xFFFFFFFF ) return 2; if( iRowid==0 ){ pPg->p[1] = 0x01; return 0; } if( bForce==0 && nElem>=(nSlot/2) ){ return 1; } fts5PutU32(&pPg->p[4], nElem+1); if( szKey==4 ){ u32 *aSlot = (u32*)&pPg->p[8]; while( aSlot[iSlot] ){ iSlot = (iSlot + 1) % nSlot; if( nCollide--==0 ) return 0; } fts5PutU32((u8*)&aSlot[iSlot], (u32)iRowid); }else{ u64 *aSlot = (u64*)&pPg->p[8]; while( aSlot[iSlot] ){ iSlot = (iSlot + 1) % nSlot; if( nCollide--==0 ) return 0; } fts5PutU64((u8*)&aSlot[iSlot], iRowid); } return 0; } /* ** This function attempts to build a new hash containing all the keys ** currently in the tombstone hash table for segment pSeg. The new ** hash will be stored in the nOut buffers passed in array apOut[]. ** All pages of the new hash use key-size szKey (4 or 8). ** ** Return 0 if the hash is successfully rebuilt into the nOut pages. ** Or non-zero if it is not (because one page became overfull). In this ** case the caller should retry with a larger nOut parameter. ** ** Parameter pData1 is page iPg1 of the hash table being rebuilt. */ static int fts5IndexTombstoneRehash( Fts5Index *p, Fts5StructureSegment *pSeg, /* Segment to rebuild hash of */ Fts5Data *pData1, /* One page of current hash - or NULL */ int iPg1, /* Which page of the current hash is pData1 */ int szKey, /* 4 or 8, the keysize */ int nOut, /* Number of output pages */ Fts5Data **apOut /* Array of output hash pages */ ){ int ii; int res = 0; /* Initialize the headers of all the output pages */ for(ii=0; ii<nOut; ii++){ apOut[ii]->p[0] = szKey; fts5PutU32(&apOut[ii]->p[4], 0); } /* Loop through the current pages of the hash table. */ for(ii=0; res==0 && ii<pSeg->nPgTombstone; ii++){ Fts5Data *pData = 0; /* Page ii of the current hash table */ Fts5Data *pFree = 0; /* Free this at the end of the loop */ if( iPg1==ii ){ pData = pData1; }else{ pFree = pData = fts5DataRead(p, FTS5_TOMBSTONE_ROWID(pSeg->iSegid, ii)); } if( pData ){ int szKeyIn = TOMBSTONE_KEYSIZE(pData); int nSlotIn = (pData->nn - 8) / szKeyIn; int iIn; for(iIn=0; iIn<nSlotIn; iIn++){ u64 iVal = 0; /* Read the value from slot iIn of the input page into iVal. */ if( szKeyIn==4 ){ u32 *aSlot = (u32*)&pData->p[8]; if( aSlot[iIn] ) iVal = fts5GetU32((u8*)&aSlot[iIn]); }else{ u64 *aSlot = (u64*)&pData->p[8]; if( aSlot[iIn] ) iVal = fts5GetU64((u8*)&aSlot[iIn]); } /* If iVal is not 0 at this point, insert it into the new hash table */ if( iVal ){ Fts5Data *pPg = apOut[(iVal % nOut)]; res = fts5IndexTombstoneAddToPage(pPg, 0, nOut, iVal); if( res ) break; } } /* If this is page 0 of the old hash, copy the rowid-0-flag from the ** old hash to the new. */ if( ii==0 ){ apOut[0]->p[1] = pData->p[1]; } } fts5DataRelease(pFree); } return res; } /* ** This is called to rebuild the hash table belonging to segment pSeg. ** If parameter pData1 is not NULL, then one page of the existing hash table ** has already been loaded - pData1, which is page iPg1. The key-size for ** the new hash table is szKey (4 or 8). ** ** If successful, the new hash table is not written to disk. Instead, ** output parameter (*pnOut) is set to the number of pages in the new ** hash table, and (*papOut) to point to an array of buffers containing ** the new page data. ** ** If an error occurs, an error code is left in the Fts5Index object and ** both output parameters set to 0 before returning. */ static void fts5IndexTombstoneRebuild( Fts5Index *p, Fts5StructureSegment *pSeg, /* Segment to rebuild hash of */ Fts5Data *pData1, /* One page of current hash - or NULL */ int iPg1, /* Which page of the current hash is pData1 */ int szKey, /* 4 or 8, the keysize */ int *pnOut, /* OUT: Number of output pages */ Fts5Data ***papOut /* OUT: Output hash pages */ ){ const int MINSLOT = 32; int nSlotPerPage = MAX(MINSLOT, (p->pConfig->pgsz - 8) / szKey); int nSlot = 0; /* Number of slots in each output page */ int nOut = 0; /* Figure out how many output pages (nOut) and how many slots per ** page (nSlot). There are three possibilities: ** ** 1. The hash table does not yet exist. In this case the new hash ** table will consist of a single page with MINSLOT slots. ** ** 2. The hash table exists but is currently a single page. In this ** case an attempt is made to grow the page to accommodate the new ** entry. The page is allowed to grow up to nSlotPerPage (see above) ** slots. ** ** 3. The hash table already consists of more than one page, or of ** a single page already so large that it cannot be grown. In this ** case the new hash consists of (nPg*2+1) pages of nSlotPerPage ** slots each, where nPg is the current number of pages in the ** hash table. */ if( pSeg->nPgTombstone==0 ){ /* Case 1. */ nOut = 1; nSlot = MINSLOT; }else if( pSeg->nPgTombstone==1 ){ /* Case 2. */ int nElem = (int)fts5GetU32(&pData1->p[4]); assert( pData1 && iPg1==0 ); nOut = 1; nSlot = MAX(nElem*4, MINSLOT); if( nSlot>nSlotPerPage ) nOut = 0; } if( nOut==0 ){ /* Case 3. */ nOut = (pSeg->nPgTombstone * 2 + 1); nSlot = nSlotPerPage; } /* Allocate the required array and output pages */ while( 1 ){ int res = 0; int ii = 0; int szPage = 0; Fts5Data **apOut = 0; /* Allocate space for the new hash table */ assert( nSlot>=MINSLOT ); apOut = (Fts5Data**)sqlite3Fts5MallocZero(&p->rc, sizeof(Fts5Data*) * nOut); szPage = 8 + nSlot*szKey; for(ii=0; ii<nOut; ii++){ Fts5Data *pNew = (Fts5Data*)sqlite3Fts5MallocZero(&p->rc, sizeof(Fts5Data)+szPage ); if( pNew ){ pNew->nn = szPage; pNew->p = (u8*)&pNew[1]; apOut[ii] = pNew; } } /* Rebuild the hash table. */ if( p->rc==SQLITE_OK ){ res = fts5IndexTombstoneRehash(p, pSeg, pData1, iPg1, szKey, nOut, apOut); } if( res==0 ){ if( p->rc ){ fts5IndexFreeArray(apOut, nOut); apOut = 0; nOut = 0; } *pnOut = nOut; *papOut = apOut; break; } /* If control flows to here, it was not possible to rebuild the hash ** table. Free all buffers and then try again with more pages. */ assert( p->rc==SQLITE_OK ); fts5IndexFreeArray(apOut, nOut); nSlot = nSlotPerPage; nOut = nOut*2 + 1; } } /* ** Add a tombstone for rowid iRowid to segment pSeg. */ static void fts5IndexTombstoneAdd( Fts5Index *p, Fts5StructureSegment *pSeg, u64 iRowid ){ Fts5Data *pPg = 0; int iPg = -1; int szKey = 0; int nHash = 0; Fts5Data **apHash = 0; p->nContentlessDelete++; if( pSeg->nPgTombstone>0 ){ iPg = iRowid % pSeg->nPgTombstone; pPg = fts5DataRead(p, FTS5_TOMBSTONE_ROWID(pSeg->iSegid,iPg)); if( pPg==0 ){ assert( p->rc!=SQLITE_OK ); return; } if( 0==fts5IndexTombstoneAddToPage(pPg, 0, pSeg->nPgTombstone, iRowid) ){ fts5DataWrite(p, FTS5_TOMBSTONE_ROWID(pSeg->iSegid,iPg), pPg->p, pPg->nn); fts5DataRelease(pPg); return; } } /* Have to rebuild the hash table. First figure out the key-size (4 or 8). */ szKey = pPg ? TOMBSTONE_KEYSIZE(pPg) : 4; if( iRowid>0xFFFFFFFF ) szKey = 8; /* Rebuild the hash table */ fts5IndexTombstoneRebuild(p, pSeg, pPg, iPg, szKey, &nHash, &apHash); assert( p->rc==SQLITE_OK || (nHash==0 && apHash==0) ); /* If all has succeeded, write the new rowid into one of the new hash ** table pages, then write them all out to disk. */ if( nHash ){ int ii = 0; fts5IndexTombstoneAddToPage(apHash[iRowid % nHash], 1, nHash, iRowid); for(ii=0; ii<nHash; ii++){ i64 iTombstoneRowid = FTS5_TOMBSTONE_ROWID(pSeg->iSegid, ii); fts5DataWrite(p, iTombstoneRowid, apHash[ii]->p, apHash[ii]->nn); } pSeg->nPgTombstone = nHash; fts5StructureWrite(p, p->pStruct); } fts5DataRelease(pPg); fts5IndexFreeArray(apHash, nHash); } /* ** Add iRowid to the tombstone list of the segment or segments that contain ** rows from origin iOrigin. Return SQLITE_OK if successful, or an SQLite ** error code otherwise. */ int sqlite3Fts5IndexContentlessDelete(Fts5Index *p, i64 iOrigin, i64 iRowid){ Fts5Structure *pStruct; pStruct = fts5StructureRead(p); if( pStruct ){ int bFound = 0; /* True after pSeg->nEntryTombstone incr. */ int iLvl; for(iLvl=pStruct->nLevel-1; iLvl>=0; iLvl--){ int iSeg; for(iSeg=pStruct->aLevel[iLvl].nSeg-1; iSeg>=0; iSeg--){ Fts5StructureSegment *pSeg = &pStruct->aLevel[iLvl].aSeg[iSeg]; if( pSeg->iOrigin1<=(u64)iOrigin && pSeg->iOrigin2>=(u64)iOrigin ){ if( bFound==0 ){ pSeg->nEntryTombstone++; bFound = 1; } fts5IndexTombstoneAdd(p, pSeg, iRowid); } } } fts5StructureRelease(pStruct); } return fts5IndexReturn(p); } /************************************************************************* ************************************************************************** ** Below this point is the implementation of the integrity-check ** functionality. */ |
︙ | ︙ | |||
5858 5859 5860 5861 5862 5863 5864 | int n, /* Size of index key in bytes */ int flags, /* Flags for Fts5IndexQuery */ u64 *pCksum /* IN/OUT: Checksum value */ ){ int eDetail = p->pConfig->eDetail; u64 cksum = *pCksum; Fts5IndexIter *pIter = 0; | | > > | 8215 8216 8217 8218 8219 8220 8221 8222 8223 8224 8225 8226 8227 8228 8229 8230 8231 | int n, /* Size of index key in bytes */ int flags, /* Flags for Fts5IndexQuery */ u64 *pCksum /* IN/OUT: Checksum value */ ){ int eDetail = p->pConfig->eDetail; u64 cksum = *pCksum; Fts5IndexIter *pIter = 0; int rc = sqlite3Fts5IndexQuery( p, z, n, (flags | FTS5INDEX_QUERY_NOTOKENDATA), 0, &pIter ); while( rc==SQLITE_OK && ALWAYS(pIter!=0) && 0==sqlite3Fts5IterEof(pIter) ){ i64 rowid = pIter->iRowid; if( eDetail==FTS5_DETAIL_NONE ){ cksum ^= sqlite3Fts5IndexEntryCksum(rowid, 0, 0, iIdx, z, n); }else{ |
︙ | ︙ | |||
5880 5881 5882 5883 5884 5885 5886 | cksum ^= sqlite3Fts5IndexEntryCksum(rowid, iCol, iOff, iIdx, z, n); } } if( rc==SQLITE_OK ){ rc = sqlite3Fts5IterNext(pIter); } } | | | 8239 8240 8241 8242 8243 8244 8245 8246 8247 8248 8249 8250 8251 8252 8253 | cksum ^= sqlite3Fts5IndexEntryCksum(rowid, iCol, iOff, iIdx, z, n); } } if( rc==SQLITE_OK ){ rc = sqlite3Fts5IterNext(pIter); } } fts5IterClose(pIter); *pCksum = cksum; return rc; } /* ** Check if buffer z[], size n bytes, contains as series of valid utf-8 |
︙ | ︙ | |||
6017 6018 6019 6020 6021 6022 6023 | int i; /* Now check that the iter.nEmpty leaves following the current leaf ** (a) exist and (b) contain no terms. */ for(i=iFirst; p->rc==SQLITE_OK && i<=iLast; i++){ Fts5Data *pLeaf = fts5DataRead(p, FTS5_SEGMENT_ROWID(pSeg->iSegid, i)); if( pLeaf ){ | | | > > > | | | | | | | > | 8376 8377 8378 8379 8380 8381 8382 8383 8384 8385 8386 8387 8388 8389 8390 8391 8392 8393 8394 8395 8396 8397 8398 8399 8400 8401 8402 8403 8404 8405 8406 8407 8408 8409 8410 8411 8412 8413 8414 8415 8416 8417 8418 8419 8420 8421 8422 8423 8424 8425 8426 8427 8428 8429 8430 8431 8432 8433 8434 8435 8436 8437 8438 8439 8440 8441 8442 8443 8444 8445 8446 8447 8448 8449 8450 8451 8452 8453 8454 8455 | int i; /* Now check that the iter.nEmpty leaves following the current leaf ** (a) exist and (b) contain no terms. */ for(i=iFirst; p->rc==SQLITE_OK && i<=iLast; i++){ Fts5Data *pLeaf = fts5DataRead(p, FTS5_SEGMENT_ROWID(pSeg->iSegid, i)); if( pLeaf ){ if( !fts5LeafIsTermless(pLeaf) || (i>=iNoRowid && 0!=fts5LeafFirstRowidOff(pLeaf)) ){ FTS5_CORRUPT_ROWID(p, FTS5_SEGMENT_ROWID(pSeg->iSegid, i)); } } fts5DataRelease(pLeaf); } } static void fts5IntegrityCheckPgidx(Fts5Index *p, i64 iRowid, Fts5Data *pLeaf){ i64 iTermOff = 0; int ii; Fts5Buffer buf1 = {0,0,0}; Fts5Buffer buf2 = {0,0,0}; ii = pLeaf->szLeaf; while( ii<pLeaf->nn && p->rc==SQLITE_OK ){ int res; i64 iOff; int nIncr; ii += fts5GetVarint32(&pLeaf->p[ii], nIncr); iTermOff += nIncr; iOff = iTermOff; if( iOff>=pLeaf->szLeaf ){ FTS5_CORRUPT_ROWID(p, iRowid); }else if( iTermOff==nIncr ){ int nByte; iOff += fts5GetVarint32(&pLeaf->p[iOff], nByte); if( (iOff+nByte)>pLeaf->szLeaf ){ FTS5_CORRUPT_ROWID(p, iRowid); }else{ fts5BufferSet(&p->rc, &buf1, nByte, &pLeaf->p[iOff]); } }else{ int nKeep, nByte; iOff += fts5GetVarint32(&pLeaf->p[iOff], nKeep); iOff += fts5GetVarint32(&pLeaf->p[iOff], nByte); if( nKeep>buf1.n || (iOff+nByte)>pLeaf->szLeaf ){ FTS5_CORRUPT_ROWID(p, iRowid); }else{ buf1.n = nKeep; fts5BufferAppendBlob(&p->rc, &buf1, nByte, &pLeaf->p[iOff]); } if( p->rc==SQLITE_OK ){ res = fts5BufferCompare(&buf1, &buf2); if( res<=0 ) FTS5_CORRUPT_ROWID(p, iRowid); } } fts5BufferSet(&p->rc, &buf2, buf1.n, buf1.p); } fts5BufferFree(&buf1); fts5BufferFree(&buf2); } static void fts5IndexIntegrityCheckSegment( Fts5Index *p, /* FTS5 backend object */ Fts5StructureSegment *pSeg /* Segment to check internal consistency */ ){ Fts5Config *pConfig = p->pConfig; int bSecureDelete = (pConfig->iVersion==FTS5_CURRENT_VERSION_SECUREDELETE); sqlite3_stmt *pStmt = 0; int rc2; int iIdxPrevLeaf = pSeg->pgnoFirst-1; int iDlidxPrevLeaf = pSeg->pgnoLast; if( pSeg->pgnoFirst==0 ) return; |
︙ | ︙ | |||
6114 6115 6116 6117 6118 6119 6120 | if( pLeaf==0 ) break; /* Check that the leaf contains at least one term, and that it is equal ** to or larger than the split-key in zIdxTerm. Also check that if there ** is also a rowid pointer within the leaf page header, it points to a ** location before the term. */ if( pLeaf->nn<=pLeaf->szLeaf ){ | > > > > > > > > > > | > > | | | | 8477 8478 8479 8480 8481 8482 8483 8484 8485 8486 8487 8488 8489 8490 8491 8492 8493 8494 8495 8496 8497 8498 8499 8500 8501 8502 8503 8504 8505 8506 8507 8508 8509 8510 8511 8512 8513 8514 8515 8516 8517 8518 8519 8520 8521 | if( pLeaf==0 ) break; /* Check that the leaf contains at least one term, and that it is equal ** to or larger than the split-key in zIdxTerm. Also check that if there ** is also a rowid pointer within the leaf page header, it points to a ** location before the term. */ if( pLeaf->nn<=pLeaf->szLeaf ){ if( nIdxTerm==0 && pConfig->iVersion==FTS5_CURRENT_VERSION_SECUREDELETE && pLeaf->nn==pLeaf->szLeaf && pLeaf->nn==4 ){ /* special case - the very first page in a segment keeps its %_idx ** entry even if all the terms are removed from it by secure-delete ** operations. */ }else{ FTS5_CORRUPT_ROWID(p, iRow); } }else{ int iOff; /* Offset of first term on leaf */ int iRowidOff; /* Offset of first rowid on leaf */ int nTerm; /* Size of term on leaf in bytes */ int res; /* Comparison of term and split-key */ iOff = fts5LeafFirstTermOff(pLeaf); iRowidOff = fts5LeafFirstRowidOff(pLeaf); if( iRowidOff>=iOff || iOff>=pLeaf->szLeaf ){ FTS5_CORRUPT_ROWID(p, iRow); }else{ iOff += fts5GetVarint32(&pLeaf->p[iOff], nTerm); res = fts5Memcmp(&pLeaf->p[iOff], zIdxTerm, MIN(nTerm, nIdxTerm)); if( res==0 ) res = nTerm - nIdxTerm; if( res<0 ) FTS5_CORRUPT_ROWID(p, iRow); } fts5IntegrityCheckPgidx(p, iRow, pLeaf); } fts5DataRelease(pLeaf); if( p->rc ) break; /* Now check that the iter.nEmpty leaves following the current leaf ** (a) exist and (b) contain no terms. */ fts5IndexIntegrityCheckEmpty( |
︙ | ︙ | |||
6162 6163 6164 6165 6166 6167 6168 | ){ /* Check any rowid-less pages that occur before the current leaf. */ for(iPg=iPrevLeaf+1; iPg<fts5DlidxIterPgno(pDlidx); iPg++){ iKey = FTS5_SEGMENT_ROWID(iSegid, iPg); pLeaf = fts5DataRead(p, iKey); if( pLeaf ){ | | | | > > | > | 8537 8538 8539 8540 8541 8542 8543 8544 8545 8546 8547 8548 8549 8550 8551 8552 8553 8554 8555 8556 8557 8558 8559 8560 8561 8562 8563 8564 8565 8566 8567 8568 8569 8570 8571 8572 | ){ /* Check any rowid-less pages that occur before the current leaf. */ for(iPg=iPrevLeaf+1; iPg<fts5DlidxIterPgno(pDlidx); iPg++){ iKey = FTS5_SEGMENT_ROWID(iSegid, iPg); pLeaf = fts5DataRead(p, iKey); if( pLeaf ){ if( fts5LeafFirstRowidOff(pLeaf)!=0 ) FTS5_CORRUPT_ROWID(p, iKey); fts5DataRelease(pLeaf); } } iPrevLeaf = fts5DlidxIterPgno(pDlidx); /* Check that the leaf page indicated by the iterator really does ** contain the rowid suggested by the same. */ iKey = FTS5_SEGMENT_ROWID(iSegid, iPrevLeaf); pLeaf = fts5DataRead(p, iKey); if( pLeaf ){ i64 iRowid; int iRowidOff = fts5LeafFirstRowidOff(pLeaf); ASSERT_SZLEAF_OK(pLeaf); if( iRowidOff>=pLeaf->szLeaf ){ FTS5_CORRUPT_ROWID(p, iKey); }else if( bSecureDelete==0 || iRowidOff>0 ){ i64 iDlRowid = fts5DlidxIterRowid(pDlidx); fts5GetVarint(&pLeaf->p[iRowidOff], (u64*)&iRowid); if( iRowid<iDlRowid || (bSecureDelete==0 && iRowid!=iDlRowid) ){ FTS5_CORRUPT_ROWID(p, iKey); } } fts5DataRelease(pLeaf); } } iDlidxPrevLeaf = iPg; fts5DlidxIterFree(pDlidx); |
︙ | ︙ | |||
6294 6295 6296 6297 6298 6299 6300 | cksum2 ^= sqlite3Fts5IndexEntryCksum(iRowid, iCol, iTokOff, -1, z, n); } } } fts5TestTerm(p, &term, 0, 0, cksum2, &cksum3); fts5MultiIterFree(pIter); | | > > > > > | > > | > > | | | | | | > > | | | > > > > > > | | | 8672 8673 8674 8675 8676 8677 8678 8679 8680 8681 8682 8683 8684 8685 8686 8687 8688 8689 8690 8691 8692 8693 8694 8695 8696 8697 8698 8699 8700 8701 8702 8703 8704 8705 8706 8707 8708 8709 8710 8711 8712 8713 8714 8715 8716 8717 8718 8719 8720 8721 8722 8723 8724 8725 8726 8727 8728 8729 8730 8731 8732 8733 8734 8735 8736 8737 8738 8739 8740 8741 8742 8743 8744 8745 8746 8747 8748 8749 8750 8751 8752 8753 8754 8755 8756 8757 8758 8759 8760 8761 8762 8763 8764 8765 8766 8767 8768 8769 8770 8771 8772 8773 8774 8775 8776 8777 8778 8779 8780 8781 8782 8783 8784 8785 8786 8787 8788 | cksum2 ^= sqlite3Fts5IndexEntryCksum(iRowid, iCol, iTokOff, -1, z, n); } } } fts5TestTerm(p, &term, 0, 0, cksum2, &cksum3); fts5MultiIterFree(pIter); if( p->rc==SQLITE_OK && bUseCksum && cksum!=cksum2 ){ p->rc = FTS5_CORRUPT; sqlite3Fts5ConfigErrmsg(p->pConfig, "fts5: checksum mismatch for table \"%s\"", p->pConfig->zName ); } fts5StructureRelease(pStruct); #ifdef SQLITE_DEBUG fts5BufferFree(&term); #endif fts5BufferFree(&poslist); return fts5IndexReturn(p); } /************************************************************************* ************************************************************************** ** Below this point is the implementation of the fts5_decode() scalar ** function only. */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** Decode a segment-data rowid from the %_data table. This function is ** the opposite of macro FTS5_SEGMENT_ROWID(). */ static void fts5DecodeRowid( i64 iRowid, /* Rowid from %_data table */ int *pbTombstone, /* OUT: Tombstone hash flag */ int *piSegid, /* OUT: Segment id */ int *pbDlidx, /* OUT: Dlidx flag */ int *piHeight, /* OUT: Height */ int *piPgno /* OUT: Page number */ ){ *piPgno = (int)(iRowid & (((i64)1 << FTS5_DATA_PAGE_B) - 1)); iRowid >>= FTS5_DATA_PAGE_B; *piHeight = (int)(iRowid & (((i64)1 << FTS5_DATA_HEIGHT_B) - 1)); iRowid >>= FTS5_DATA_HEIGHT_B; *pbDlidx = (int)(iRowid & 0x0001); iRowid >>= FTS5_DATA_DLI_B; *piSegid = (int)(iRowid & (((i64)1 << FTS5_DATA_ID_B) - 1)); iRowid >>= FTS5_DATA_ID_B; *pbTombstone = (int)(iRowid & 0x0001); } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) static void fts5DebugRowid(int *pRc, Fts5Buffer *pBuf, i64 iKey){ int iSegid, iHeight, iPgno, bDlidx, bTomb; /* Rowid components */ fts5DecodeRowid(iKey, &bTomb, &iSegid, &bDlidx, &iHeight, &iPgno); if( iSegid==0 ){ if( iKey==FTS5_AVERAGES_ROWID ){ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{averages} "); }else{ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{structure}"); } } else{ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "{%s%ssegid=%d h=%d pgno=%d}", bDlidx ? "dlidx " : "", bTomb ? "tombstone " : "", iSegid, iHeight, iPgno ); } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) static void fts5DebugStructure( int *pRc, /* IN/OUT: error code */ Fts5Buffer *pBuf, Fts5Structure *p ){ int iLvl, iSeg; /* Iterate through levels, segments */ for(iLvl=0; iLvl<p->nLevel; iLvl++){ Fts5StructureLevel *pLvl = &p->aLevel[iLvl]; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {lvl=%d nMerge=%d nSeg=%d", iLvl, pLvl->nMerge, pLvl->nSeg ); for(iSeg=0; iSeg<pLvl->nSeg; iSeg++){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {id=%d leaves=%d..%d", pSeg->iSegid, pSeg->pgnoFirst, pSeg->pgnoLast ); if( pSeg->iOrigin1>0 ){ sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " origin=%lld..%lld", pSeg->iOrigin1, pSeg->iOrigin2 ); } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "}"); } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "}"); } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** This is part of the fts5_decode() debugging aid. ** ** Arguments pBlob/nBlob contain a serialized Fts5Structure object. This ** function appends a human-readable representation of the same object ** to the buffer passed as the second argument. */ |
︙ | ︙ | |||
6404 6405 6406 6407 6408 6409 6410 | *pRc = rc; return; } fts5DebugStructure(pRc, pBuf, p); fts5StructureRelease(p); } | | | | 8799 8800 8801 8802 8803 8804 8805 8806 8807 8808 8809 8810 8811 8812 8813 8814 8815 | *pRc = rc; return; } fts5DebugStructure(pRc, pBuf, p); fts5StructureRelease(p); } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** This is part of the fts5_decode() debugging aid. ** ** Arguments pBlob/nBlob contain an "averages" record. This function ** appends a human-readable representation of record to the buffer passed ** as the second argument. */ |
︙ | ︙ | |||
6429 6430 6431 6432 6433 6434 6435 | while( i<nBlob ){ u64 iVal; i += sqlite3Fts5GetVarint(&pBlob[i], &iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "%s%d", zSpace, (int)iVal); zSpace = " "; } } | | | | | | 8824 8825 8826 8827 8828 8829 8830 8831 8832 8833 8834 8835 8836 8837 8838 8839 8840 8841 8842 8843 8844 8845 8846 8847 8848 8849 8850 8851 8852 8853 8854 8855 8856 8857 8858 8859 | while( i<nBlob ){ u64 iVal; i += sqlite3Fts5GetVarint(&pBlob[i], &iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, "%s%d", zSpace, (int)iVal); zSpace = " "; } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** Buffer (a/n) is assumed to contain a list of serialized varints. Read ** each varint and append its string representation to buffer pBuf. Return ** after either the input buffer is exhausted or a 0 value is read. ** ** The return value is the number of bytes read from the input buffer. */ static int fts5DecodePoslist(int *pRc, Fts5Buffer *pBuf, const u8 *a, int n){ int iOff = 0; while( iOff<n ){ int iVal; iOff += fts5GetVarint32(&a[iOff], iVal); sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %d", iVal); } return iOff; } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** The start of buffer (a/n) contains the start of a doclist. The doclist ** may or may not finish within the buffer. This function appends a text ** representation of the part of the doclist that is present to buffer ** pBuf. ** ** The return value is the number of bytes read from the input buffer. |
︙ | ︙ | |||
6483 6484 6485 6486 6487 6488 6489 | iDocid += iDelta; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " id=%lld", iDocid); } } return iOff; } | | | | 8878 8879 8880 8881 8882 8883 8884 8885 8886 8887 8888 8889 8890 8891 8892 8893 8894 | iDocid += iDelta; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " id=%lld", iDocid); } } return iOff; } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** This function is part of the fts5_decode() debugging function. It is ** only ever used with detail=none tables. ** ** Buffer (pData/nData) contains a doclist in the format used by detail=none ** tables. This function appends a human-readable version of that list to ** buffer pBuf. |
︙ | ︙ | |||
6526 6527 6528 6529 6530 6531 6532 | zApp = "*"; } } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp); } } | | > > > > | > > > > > > > > > > > > > > > | | > > > > > > > > > > > > > > > > > > > > > > | 8921 8922 8923 8924 8925 8926 8927 8928 8929 8930 8931 8932 8933 8934 8935 8936 8937 8938 8939 8940 8941 8942 8943 8944 8945 8946 8947 8948 8949 8950 8951 8952 8953 8954 8955 8956 8957 8958 8959 8960 8961 8962 8963 8964 8965 8966 8967 8968 8969 8970 8971 8972 8973 8974 8975 8976 8977 8978 8979 8980 8981 8982 8983 8984 8985 8986 8987 8988 8989 8990 8991 8992 8993 8994 8995 8996 8997 8998 8999 9000 9001 9002 9003 9004 9005 9006 9007 9008 9009 9010 9011 9012 9013 9014 9015 9016 9017 9018 9019 9020 9021 9022 9023 9024 9025 9026 9027 9028 | zApp = "*"; } } sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " %lld%s", iRowid, zApp); } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) static void fts5BufferAppendTerm(int *pRc, Fts5Buffer *pBuf, Fts5Buffer *pTerm){ int ii; fts5BufferGrow(pRc, pBuf, pTerm->n*2 + 1); if( *pRc==SQLITE_OK ){ for(ii=0; ii<pTerm->n; ii++){ if( pTerm->p[ii]==0x00 ){ pBuf->p[pBuf->n++] = '\\'; pBuf->p[pBuf->n++] = '0'; }else{ pBuf->p[pBuf->n++] = pTerm->p[ii]; } } pBuf->p[pBuf->n] = 0x00; } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** The implementation of user-defined scalar function fts5_decode(). */ static void fts5DecodeFunction( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args (always 2) */ sqlite3_value **apVal /* Function arguments */ ){ i64 iRowid; /* Rowid for record being decoded */ int iSegid,iHeight,iPgno,bDlidx;/* Rowid components */ int bTomb; const u8 *aBlob; int n; /* Record to decode */ u8 *a = 0; Fts5Buffer s; /* Build up text to return here */ int rc = SQLITE_OK; /* Return code */ sqlite3_int64 nSpace = 0; int eDetailNone = (sqlite3_user_data(pCtx)!=0); assert( nArg==2 ); UNUSED_PARAM(nArg); memset(&s, 0, sizeof(Fts5Buffer)); iRowid = sqlite3_value_int64(apVal[0]); /* Make a copy of the second argument (a blob) in aBlob[]. The aBlob[] ** copy is followed by FTS5_DATA_ZERO_PADDING 0x00 bytes, which prevents ** buffer overreads even if the record is corrupt. */ n = sqlite3_value_bytes(apVal[1]); aBlob = sqlite3_value_blob(apVal[1]); nSpace = ((i64)n) + FTS5_DATA_ZERO_PADDING; a = (u8*)sqlite3Fts5MallocZero(&rc, nSpace); if( a==0 ) goto decode_out; if( n>0 ) memcpy(a, aBlob, n); fts5DecodeRowid(iRowid, &bTomb, &iSegid, &bDlidx, &iHeight, &iPgno); fts5DebugRowid(&rc, &s, iRowid); if( bDlidx ){ Fts5Data dlidx; Fts5DlidxLvl lvl; dlidx.p = a; dlidx.nn = n; memset(&lvl, 0, sizeof(Fts5DlidxLvl)); lvl.pData = &dlidx; lvl.iLeafPgno = iPgno; for(fts5DlidxLvlNext(&lvl); lvl.bEof==0; fts5DlidxLvlNext(&lvl)){ sqlite3Fts5BufferAppendPrintf(&rc, &s, " %d(%lld)", lvl.iLeafPgno, lvl.iRowid ); } }else if( bTomb ){ u32 nElem = fts5GetU32(&a[4]); int szKey = (aBlob[0]==4 || aBlob[0]==8) ? aBlob[0] : 8; int nSlot = (n - 8) / szKey; int ii; sqlite3Fts5BufferAppendPrintf(&rc, &s, " nElem=%d", (int)nElem); if( aBlob[1] ){ sqlite3Fts5BufferAppendPrintf(&rc, &s, " 0"); } for(ii=0; ii<nSlot; ii++){ u64 iVal = 0; if( szKey==4 ){ u32 *aSlot = (u32*)&aBlob[8]; if( aSlot[ii] ) iVal = fts5GetU32((u8*)&aSlot[ii]); }else{ u64 *aSlot = (u64*)&aBlob[8]; if( aSlot[ii] ) iVal = fts5GetU64((u8*)&aSlot[ii]); } if( iVal!=0 ){ sqlite3Fts5BufferAppendPrintf(&rc, &s, " %lld", (i64)iVal); } } }else if( iSegid==0 ){ if( iRowid==FTS5_AVERAGES_ROWID ){ fts5DecodeAverages(&rc, &s, a, n); }else{ fts5DecodeStructure(&rc, &s, a, n); } |
︙ | ︙ | |||
6605 6606 6607 6608 6609 6610 6611 | iPgidxOff += fts5GetVarint32(&a[iPgidxOff], iTermOff); }else{ iTermOff = szLeaf; } fts5DecodeRowidList(&rc, &s, &a[4], iTermOff-4); iOff = iTermOff; | | | | < | > > | > | 9041 9042 9043 9044 9045 9046 9047 9048 9049 9050 9051 9052 9053 9054 9055 9056 9057 9058 9059 9060 9061 9062 9063 9064 9065 9066 9067 9068 9069 9070 9071 9072 9073 9074 9075 9076 9077 9078 | iPgidxOff += fts5GetVarint32(&a[iPgidxOff], iTermOff); }else{ iTermOff = szLeaf; } fts5DecodeRowidList(&rc, &s, &a[4], iTermOff-4); iOff = iTermOff; while( iOff<szLeaf && rc==SQLITE_OK ){ int nAppend; /* Read the term data for the next term*/ iOff += fts5GetVarint32(&a[iOff], nAppend); term.n = nKeep; fts5BufferAppendBlob(&rc, &term, nAppend, &a[iOff]); sqlite3Fts5BufferAppendPrintf(&rc, &s, " term="); fts5BufferAppendTerm(&rc, &s, &term); iOff += nAppend; /* Figure out where the doclist for this term ends */ if( iPgidxOff<n ){ int nIncr; iPgidxOff += fts5GetVarint32(&a[iPgidxOff], nIncr); iTermOff += nIncr; }else{ iTermOff = szLeaf; } if( iTermOff>szLeaf ){ rc = FTS5_CORRUPT; }else{ fts5DecodeRowidList(&rc, &s, &a[iOff], iTermOff-iOff); } iOff = iTermOff; if( iOff<szLeaf ){ iOff += fts5GetVarint32(&a[iOff], nKeep); } } fts5BufferFree(&term); |
︙ | ︙ | |||
6719 6720 6721 6722 6723 6724 6725 | if( iOff+nByte>n ){ rc = FTS5_CORRUPT; break; } fts5BufferAppendBlob(&rc, &term, nByte, &a[iOff]); iOff += nByte; | | | < | | | 9157 9158 9159 9160 9161 9162 9163 9164 9165 9166 9167 9168 9169 9170 9171 9172 9173 9174 9175 9176 9177 9178 9179 9180 9181 9182 9183 9184 9185 9186 9187 9188 9189 9190 | if( iOff+nByte>n ){ rc = FTS5_CORRUPT; break; } fts5BufferAppendBlob(&rc, &term, nByte, &a[iOff]); iOff += nByte; sqlite3Fts5BufferAppendPrintf(&rc, &s, " term="); fts5BufferAppendTerm(&rc, &s, &term); iOff += fts5DecodeDoclist(&rc, &s, &a[iOff], iEnd-iOff); } fts5BufferFree(&term); } decode_out: sqlite3_free(a); if( rc==SQLITE_OK ){ sqlite3_result_text(pCtx, (const char*)s.p, s.n, SQLITE_TRANSIENT); }else{ sqlite3_result_error_code(pCtx, rc); } fts5BufferFree(&s); } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) /* ** The implementation of user-defined scalar function fts5_rowid(). */ static void fts5RowidFunction( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args (always 2) */ sqlite3_value **apVal /* Function arguments */ |
︙ | ︙ | |||
6773 6774 6775 6776 6777 6778 6779 | }else{ sqlite3_result_error(pCtx, "first arg to fts5_rowid() must be 'segment'" , -1 ); } } } | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 9210 9211 9212 9213 9214 9215 9216 9217 9218 9219 9220 9221 9222 9223 9224 9225 9226 9227 9228 9229 9230 9231 9232 9233 9234 9235 9236 9237 9238 9239 9240 9241 9242 9243 9244 9245 9246 9247 9248 9249 9250 9251 9252 9253 9254 9255 9256 9257 9258 9259 9260 9261 9262 9263 9264 9265 9266 9267 9268 9269 9270 9271 9272 9273 9274 9275 9276 9277 9278 9279 9280 9281 9282 9283 9284 9285 9286 9287 9288 9289 9290 9291 9292 9293 9294 9295 9296 9297 9298 9299 9300 9301 9302 9303 9304 9305 9306 9307 9308 9309 9310 9311 9312 9313 9314 9315 9316 9317 9318 9319 9320 9321 9322 9323 9324 9325 9326 9327 9328 9329 9330 9331 9332 9333 9334 9335 9336 9337 9338 9339 9340 9341 9342 9343 9344 9345 9346 9347 9348 9349 9350 9351 9352 9353 9354 9355 9356 9357 9358 9359 9360 9361 9362 9363 9364 9365 9366 9367 9368 9369 9370 9371 9372 9373 9374 9375 9376 9377 9378 9379 9380 9381 9382 9383 9384 9385 9386 9387 9388 9389 9390 9391 9392 9393 9394 9395 9396 9397 9398 9399 9400 9401 9402 9403 9404 9405 9406 9407 9408 9409 9410 9411 9412 9413 9414 9415 9416 9417 9418 9419 9420 9421 9422 9423 9424 9425 9426 9427 9428 9429 9430 9431 9432 9433 9434 9435 9436 9437 9438 9439 9440 9441 9442 9443 9444 9445 9446 9447 9448 9449 9450 9451 9452 9453 9454 9455 9456 9457 9458 9459 9460 9461 9462 9463 9464 9465 9466 9467 9468 9469 9470 9471 9472 9473 9474 9475 9476 9477 9478 9479 9480 9481 9482 9483 9484 9485 9486 9487 9488 9489 9490 9491 9492 9493 9494 9495 9496 9497 9498 9499 9500 9501 9502 9503 9504 9505 9506 9507 9508 9509 9510 | }else{ sqlite3_result_error(pCtx, "first arg to fts5_rowid() must be 'segment'" , -1 ); } } } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) typedef struct Fts5StructVtab Fts5StructVtab; struct Fts5StructVtab { sqlite3_vtab base; }; typedef struct Fts5StructVcsr Fts5StructVcsr; struct Fts5StructVcsr { sqlite3_vtab_cursor base; Fts5Structure *pStruct; int iLevel; int iSeg; int iRowid; }; /* ** Create a new fts5_structure() table-valued function. */ static int fts5structConnectMethod( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ Fts5StructVtab *pNew = 0; int rc = SQLITE_OK; rc = sqlite3_declare_vtab(db, "CREATE TABLE xyz(" "level, segment, merge, segid, leaf1, leaf2, loc1, loc2, " "npgtombstone, nentrytombstone, nentry, struct HIDDEN);" ); if( rc==SQLITE_OK ){ pNew = sqlite3Fts5MallocZero(&rc, sizeof(*pNew)); } *ppVtab = (sqlite3_vtab*)pNew; return rc; } /* ** We must have a single struct=? constraint that will be passed through ** into the xFilter method. If there is no valid struct=? constraint, ** then return an SQLITE_CONSTRAINT error. */ static int fts5structBestIndexMethod( sqlite3_vtab *tab, sqlite3_index_info *pIdxInfo ){ int i; int rc = SQLITE_CONSTRAINT; struct sqlite3_index_constraint *p; pIdxInfo->estimatedCost = (double)100; pIdxInfo->estimatedRows = 100; pIdxInfo->idxNum = 0; for(i=0, p=pIdxInfo->aConstraint; i<pIdxInfo->nConstraint; i++, p++){ if( p->usable==0 ) continue; if( p->op==SQLITE_INDEX_CONSTRAINT_EQ && p->iColumn==11 ){ rc = SQLITE_OK; pIdxInfo->aConstraintUsage[i].omit = 1; pIdxInfo->aConstraintUsage[i].argvIndex = 1; break; } } return rc; } /* ** This method is the destructor for bytecodevtab objects. */ static int fts5structDisconnectMethod(sqlite3_vtab *pVtab){ Fts5StructVtab *p = (Fts5StructVtab*)pVtab; sqlite3_free(p); return SQLITE_OK; } /* ** Constructor for a new bytecodevtab_cursor object. */ static int fts5structOpenMethod(sqlite3_vtab *p, sqlite3_vtab_cursor **ppCsr){ int rc = SQLITE_OK; Fts5StructVcsr *pNew = 0; pNew = sqlite3Fts5MallocZero(&rc, sizeof(*pNew)); *ppCsr = (sqlite3_vtab_cursor*)pNew; return SQLITE_OK; } /* ** Destructor for a bytecodevtab_cursor. */ static int fts5structCloseMethod(sqlite3_vtab_cursor *cur){ Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur; fts5StructureRelease(pCsr->pStruct); sqlite3_free(pCsr); return SQLITE_OK; } /* ** Advance a bytecodevtab_cursor to its next row of output. */ static int fts5structNextMethod(sqlite3_vtab_cursor *cur){ Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur; Fts5Structure *p = pCsr->pStruct; assert( pCsr->pStruct ); pCsr->iSeg++; pCsr->iRowid++; while( pCsr->iLevel<p->nLevel && pCsr->iSeg>=p->aLevel[pCsr->iLevel].nSeg ){ pCsr->iLevel++; pCsr->iSeg = 0; } if( pCsr->iLevel>=p->nLevel ){ fts5StructureRelease(pCsr->pStruct); pCsr->pStruct = 0; } return SQLITE_OK; } /* ** Return TRUE if the cursor has been moved off of the last ** row of output. */ static int fts5structEofMethod(sqlite3_vtab_cursor *cur){ Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur; return pCsr->pStruct==0; } static int fts5structRowidMethod( sqlite3_vtab_cursor *cur, sqlite_int64 *piRowid ){ Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur; *piRowid = pCsr->iRowid; return SQLITE_OK; } /* ** Return values of columns for the row at which the bytecodevtab_cursor ** is currently pointing. */ static int fts5structColumnMethod( sqlite3_vtab_cursor *cur, /* The cursor */ sqlite3_context *ctx, /* First argument to sqlite3_result_...() */ int i /* Which column to return */ ){ Fts5StructVcsr *pCsr = (Fts5StructVcsr*)cur; Fts5Structure *p = pCsr->pStruct; Fts5StructureSegment *pSeg = &p->aLevel[pCsr->iLevel].aSeg[pCsr->iSeg]; switch( i ){ case 0: /* level */ sqlite3_result_int(ctx, pCsr->iLevel); break; case 1: /* segment */ sqlite3_result_int(ctx, pCsr->iSeg); break; case 2: /* merge */ sqlite3_result_int(ctx, pCsr->iSeg < p->aLevel[pCsr->iLevel].nMerge); break; case 3: /* segid */ sqlite3_result_int(ctx, pSeg->iSegid); break; case 4: /* leaf1 */ sqlite3_result_int(ctx, pSeg->pgnoFirst); break; case 5: /* leaf2 */ sqlite3_result_int(ctx, pSeg->pgnoLast); break; case 6: /* origin1 */ sqlite3_result_int64(ctx, pSeg->iOrigin1); break; case 7: /* origin2 */ sqlite3_result_int64(ctx, pSeg->iOrigin2); break; case 8: /* npgtombstone */ sqlite3_result_int(ctx, pSeg->nPgTombstone); break; case 9: /* nentrytombstone */ sqlite3_result_int64(ctx, pSeg->nEntryTombstone); break; case 10: /* nentry */ sqlite3_result_int64(ctx, pSeg->nEntry); break; } return SQLITE_OK; } /* ** Initialize a cursor. ** ** idxNum==0 means show all subprograms ** idxNum==1 means show only the main bytecode and omit subprograms. */ static int fts5structFilterMethod( sqlite3_vtab_cursor *pVtabCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ Fts5StructVcsr *pCsr = (Fts5StructVcsr *)pVtabCursor; int rc = SQLITE_OK; const u8 *aBlob = 0; int nBlob = 0; assert( argc==1 ); fts5StructureRelease(pCsr->pStruct); pCsr->pStruct = 0; nBlob = sqlite3_value_bytes(argv[0]); aBlob = (const u8*)sqlite3_value_blob(argv[0]); rc = fts5StructureDecode(aBlob, nBlob, 0, &pCsr->pStruct); if( rc==SQLITE_OK ){ pCsr->iLevel = 0; pCsr->iRowid = 0; pCsr->iSeg = -1; rc = fts5structNextMethod(pVtabCursor); } return rc; } #endif /* SQLITE_TEST || SQLITE_FTS5_DEBUG */ /* ** This is called as part of registering the FTS5 module with database ** connection db. It registers several user-defined scalar functions useful ** with FTS5. ** ** If successful, SQLITE_OK is returned. If an error occurs, some other ** SQLite error code is returned instead. */ int sqlite3Fts5IndexInit(sqlite3 *db){ #if defined(SQLITE_TEST) || defined(SQLITE_FTS5_DEBUG) int rc = sqlite3_create_function( db, "fts5_decode", 2, SQLITE_UTF8, 0, fts5DecodeFunction, 0, 0 ); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_decode_none", 2, SQLITE_UTF8, (void*)db, fts5DecodeFunction, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_rowid", -1, SQLITE_UTF8, 0, fts5RowidFunction, 0, 0 ); } if( rc==SQLITE_OK ){ static const sqlite3_module fts5structure_module = { 0, /* iVersion */ 0, /* xCreate */ fts5structConnectMethod, /* xConnect */ fts5structBestIndexMethod, /* xBestIndex */ fts5structDisconnectMethod, /* xDisconnect */ 0, /* xDestroy */ fts5structOpenMethod, /* xOpen */ fts5structCloseMethod, /* xClose */ fts5structFilterMethod, /* xFilter */ fts5structNextMethod, /* xNext */ fts5structEofMethod, /* xEof */ fts5structColumnMethod, /* xColumn */ fts5structRowidMethod, /* xRowid */ 0, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 /* xIntegrity */ }; rc = sqlite3_create_module(db, "fts5_structure", &fts5structure_module, 0); } return rc; #else return SQLITE_OK; UNUSED_PARAM(db); #endif } |
︙ | ︙ |
Changes to ext/fts5/fts5_main.c.
︙ | ︙ | |||
79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | fts5_api api; /* User visible part of object (see fts5.h) */ sqlite3 *db; /* Associated database connection */ i64 iNextId; /* Used to allocate unique cursor ids */ Fts5Auxiliary *pAux; /* First in list of all aux. functions */ Fts5TokenizerModule *pTok; /* First in list of all tokenizer modules */ Fts5TokenizerModule *pDfltTok; /* Default tokenizer module */ Fts5Cursor *pCsr; /* First in list of all open cursors */ }; /* ** Each auxiliary function registered with the FTS5 module is represented ** by an object of the following type. All such objects are stored as part ** of the Fts5Global.pAux list. */ struct Fts5Auxiliary { Fts5Global *pGlobal; /* Global context for this function */ char *zFunc; /* Function name (nul-terminated) */ void *pUserData; /* User-data pointer */ fts5_extension_function xFunc; /* Callback function */ void (*xDestroy)(void*); /* Destructor function */ Fts5Auxiliary *pNext; /* Next registered auxiliary function */ }; /* ** Each tokenizer module registered with the FTS5 module is represented ** by an object of the following type. All such objects are stored as part ** of the Fts5Global.pTok list. */ struct Fts5TokenizerModule { char *zName; /* Name of tokenizer */ void *pUserData; /* User pointer passed to xCreate() */ | > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > | 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 | fts5_api api; /* User visible part of object (see fts5.h) */ sqlite3 *db; /* Associated database connection */ i64 iNextId; /* Used to allocate unique cursor ids */ Fts5Auxiliary *pAux; /* First in list of all aux. functions */ Fts5TokenizerModule *pTok; /* First in list of all tokenizer modules */ Fts5TokenizerModule *pDfltTok; /* Default tokenizer module */ Fts5Cursor *pCsr; /* First in list of all open cursors */ u32 aLocaleHdr[4]; }; /* ** Size of header on fts5_locale() values. And macro to access a buffer ** containing a copy of the header from an Fts5Config pointer. */ #define FTS5_LOCALE_HDR_SIZE ((int)sizeof( ((Fts5Global*)0)->aLocaleHdr )) #define FTS5_LOCALE_HDR(pConfig) ((const u8*)(pConfig->pGlobal->aLocaleHdr)) #define FTS5_INSTTOKEN_SUBTYPE 73 /* ** Each auxiliary function registered with the FTS5 module is represented ** by an object of the following type. All such objects are stored as part ** of the Fts5Global.pAux list. */ struct Fts5Auxiliary { Fts5Global *pGlobal; /* Global context for this function */ char *zFunc; /* Function name (nul-terminated) */ void *pUserData; /* User-data pointer */ fts5_extension_function xFunc; /* Callback function */ void (*xDestroy)(void*); /* Destructor function */ Fts5Auxiliary *pNext; /* Next registered auxiliary function */ }; /* ** Each tokenizer module registered with the FTS5 module is represented ** by an object of the following type. All such objects are stored as part ** of the Fts5Global.pTok list. ** ** bV2Native: ** True if the tokenizer was registered using xCreateTokenizer_v2(), false ** for xCreateTokenizer(). If this variable is true, then x2 is populated ** with the routines as supplied by the caller and x1 contains synthesized ** wrapper routines. In this case the user-data pointer passed to ** x1.xCreate should be a pointer to the Fts5TokenizerModule structure, ** not a copy of pUserData. ** ** Of course, if bV2Native is false, then x1 contains the real routines and ** x2 the synthesized ones. In this case a pointer to the Fts5TokenizerModule ** object should be passed to x2.xCreate. ** ** The synthesized wrapper routines are necessary for xFindTokenizer(_v2) ** calls. */ struct Fts5TokenizerModule { char *zName; /* Name of tokenizer */ void *pUserData; /* User pointer passed to xCreate() */ int bV2Native; /* True if v2 native tokenizer */ fts5_tokenizer x1; /* Tokenizer functions */ fts5_tokenizer_v2 x2; /* V2 tokenizer functions */ void (*xDestroy)(void*); /* Destructor function */ Fts5TokenizerModule *pNext; /* Next registered tokenizer module */ }; struct Fts5FullTable { Fts5Table p; /* Public class members from fts5Int.h */ Fts5Storage *pStorage; /* Document store */ Fts5Global *pGlobal; /* Global (connection wide) data */ Fts5Cursor *pSortCsr; /* Sort data from this cursor */ int iSavepoint; /* Successful xSavepoint()+1 */ #ifdef SQLITE_DEBUG struct Fts5TransactionState ts; #endif }; struct Fts5MatchPhrase { Fts5Buffer *pPoslist; /* Pointer to current poslist */ |
︙ | ︙ | |||
137 138 139 140 141 142 143 | ** byte of the position list for the corresponding phrase. */ struct Fts5Sorter { sqlite3_stmt *pStmt; i64 iRowid; /* Current rowid */ const u8 *aPoslist; /* Position lists for current row */ int nIdx; /* Number of entries in aIdx[] */ | | > > | 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | ** byte of the position list for the corresponding phrase. */ struct Fts5Sorter { sqlite3_stmt *pStmt; i64 iRowid; /* Current rowid */ const u8 *aPoslist; /* Position lists for current row */ int nIdx; /* Number of entries in aIdx[] */ int aIdx[FLEXARRAY]; /* Offsets into aPoslist for current row */ }; /* Size (int bytes) of an Fts5Sorter object with N indexes */ #define SZ_FTS5SORTER(N) (offsetof(Fts5Sorter,nIdx)+((N+2)/2)*sizeof(i64)) /* ** Virtual-table cursor object. ** ** iSpecial: ** If this is a 'special' query (refer to function fts5SpecialMatch()), ** then this variable contains the result of the query. |
︙ | ︙ | |||
189 190 191 192 193 194 195 | sqlite3_value **apRankArg; /* Array of trailing arguments */ sqlite3_stmt *pRankArgStmt; /* Origin of objects in apRankArg[] */ /* Auxiliary data storage */ Fts5Auxiliary *pAux; /* Currently executing extension function */ Fts5Auxdata *pAuxdata; /* First in linked list of saved aux-data */ | | | 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 | sqlite3_value **apRankArg; /* Array of trailing arguments */ sqlite3_stmt *pRankArgStmt; /* Origin of objects in apRankArg[] */ /* Auxiliary data storage */ Fts5Auxiliary *pAux; /* Currently executing extension function */ Fts5Auxdata *pAuxdata; /* First in linked list of saved aux-data */ /* Cache used by auxiliary API functions xInst() and xInstCount() */ Fts5PoslistReader *aInstIter; /* One for each phrase */ int nInstAlloc; /* Size of aInst[] array (entries / 3) */ int nInstCount; /* Number of phrase instances */ int *aInst; /* 3 integers per phrase instance */ }; /* |
︙ | ︙ | |||
300 301 302 303 304 305 306 | } } #else # define fts5CheckTransactionState(x,y,z) #endif /* | | > > | > > | > > | 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 | } } #else # define fts5CheckTransactionState(x,y,z) #endif /* ** Return true if pTab is a contentless table. If parameter bIncludeUnindexed ** is true, this includes contentless tables that store UNINDEXED columns ** only. */ static int fts5IsContentless(Fts5FullTable *pTab, int bIncludeUnindexed){ int eContent = pTab->p.pConfig->eContent; return ( eContent==FTS5_CONTENT_NONE || (bIncludeUnindexed && eContent==FTS5_CONTENT_UNINDEXED) ); } /* ** Delete a virtual table handle allocated by fts5InitVtab(). */ static void fts5FreeVtab(Fts5FullTable *pTab){ if( pTab ){ |
︙ | ︙ | |||
371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 | /* Allocate the new vtab object and parse the configuration */ pTab = (Fts5FullTable*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5FullTable)); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigParse(pGlobal, db, argc, azConfig, &pConfig, pzErr); assert( (rc==SQLITE_OK && *pzErr==0) || pConfig==0 ); } if( rc==SQLITE_OK ){ pTab->p.pConfig = pConfig; pTab->pGlobal = pGlobal; } /* Open the index sub-system */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexOpen(pConfig, bCreate, &pTab->p.pIndex, pzErr); } | > > > > | 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 | /* Allocate the new vtab object and parse the configuration */ pTab = (Fts5FullTable*)sqlite3Fts5MallocZero(&rc, sizeof(Fts5FullTable)); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigParse(pGlobal, db, argc, azConfig, &pConfig, pzErr); assert( (rc==SQLITE_OK && *pzErr==0) || pConfig==0 ); } if( rc==SQLITE_OK ){ pConfig->pzErrmsg = pzErr; pTab->p.pConfig = pConfig; pTab->pGlobal = pGlobal; if( bCreate || sqlite3Fts5TokenizerPreload(&pConfig->t) ){ rc = sqlite3Fts5LoadTokenizer(pConfig); } } /* Open the index sub-system */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexOpen(pConfig, bCreate, &pTab->p.pIndex, pzErr); } |
︙ | ︙ | |||
394 395 396 397 398 399 400 | /* Call sqlite3_declare_vtab() */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigDeclareVtab(pConfig); } /* Load the initial configuration */ if( rc==SQLITE_OK ){ | > > | | | > > | < > | 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 | /* Call sqlite3_declare_vtab() */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigDeclareVtab(pConfig); } /* Load the initial configuration */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigLoad(pTab->p.pConfig, pTab->p.pConfig->iCookie-1); } if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_NORMAL ){ rc = sqlite3_vtab_config(db, SQLITE_VTAB_CONSTRAINT_SUPPORT, (int)1); } if( rc==SQLITE_OK ){ rc = sqlite3_vtab_config(db, SQLITE_VTAB_INNOCUOUS); } if( pConfig ) pConfig->pzErrmsg = 0; if( rc!=SQLITE_OK ){ fts5FreeVtab(pTab); pTab = 0; }else if( bCreate ){ fts5CheckTransactionState(pTab, FTS5_BEGIN, 0); } *ppVTab = (sqlite3_vtab*)pTab; |
︙ | ︙ | |||
468 469 470 471 472 473 474 | static int fts5UsePatternMatch( Fts5Config *pConfig, struct sqlite3_index_constraint *p ){ assert( FTS5_PATTERN_GLOB==SQLITE_INDEX_CONSTRAINT_GLOB ); assert( FTS5_PATTERN_LIKE==SQLITE_INDEX_CONSTRAINT_LIKE ); | | | | 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 | static int fts5UsePatternMatch( Fts5Config *pConfig, struct sqlite3_index_constraint *p ){ assert( FTS5_PATTERN_GLOB==SQLITE_INDEX_CONSTRAINT_GLOB ); assert( FTS5_PATTERN_LIKE==SQLITE_INDEX_CONSTRAINT_LIKE ); if( pConfig->t.ePattern==FTS5_PATTERN_GLOB && p->op==FTS5_PATTERN_GLOB ){ return 1; } if( pConfig->t.ePattern==FTS5_PATTERN_LIKE && (p->op==FTS5_PATTERN_LIKE || p->op==FTS5_PATTERN_GLOB) ){ return 1; } return 0; } |
︙ | ︙ | |||
518 519 520 521 522 523 524 | ** Equality constraint against the rowid: "=" ** A < or <= against the rowid: "<" ** A > or >= against the rowid: ">" ** ** This function ensures that there is at most one "r" or "=". And that if ** there exists an "=" then there is no "<" or ">". ** | | > < | | 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 | ** Equality constraint against the rowid: "=" ** A < or <= against the rowid: "<" ** A > or >= against the rowid: ">" ** ** This function ensures that there is at most one "r" or "=". And that if ** there exists an "=" then there is no "<" or ">". ** ** If an unusable MATCH operator is present in the WHERE clause, then ** SQLITE_CONSTRAINT is returned. ** ** Costs are assigned as follows: ** ** a) If a MATCH operator is present, the cost depends on the other ** constraints also present. As follows: ** ** * No other constraints: cost=1000.0 ** * One rowid range constraint: cost=750.0 ** * Both rowid range constraints: cost=500.0 |
︙ | ︙ | |||
554 555 556 557 558 559 560 | char *idxStr; int iIdxStr = 0; int iCons = 0; int bSeenEq = 0; int bSeenGt = 0; int bSeenLt = 0; | | | 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 | char *idxStr; int iIdxStr = 0; int iCons = 0; int bSeenEq = 0; int bSeenGt = 0; int bSeenLt = 0; int nSeenMatch = 0; int bSeenRank = 0; assert( SQLITE_INDEX_CONSTRAINT_EQ<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_GT<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_LE<SQLITE_INDEX_CONSTRAINT_MATCH ); assert( SQLITE_INDEX_CONSTRAINT_GE<SQLITE_INDEX_CONSTRAINT_MATCH ); |
︙ | ︙ | |||
585 586 587 588 589 590 591 | int iCol = p->iColumn; if( p->op==SQLITE_INDEX_CONSTRAINT_MATCH || (p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol>=nCol) ){ /* A MATCH operator or equivalent */ if( p->usable==0 || iCol<0 ){ /* As there exists an unusable MATCH constraint this is an | | < < | | | > | 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 | int iCol = p->iColumn; if( p->op==SQLITE_INDEX_CONSTRAINT_MATCH || (p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol>=nCol) ){ /* A MATCH operator or equivalent */ if( p->usable==0 || iCol<0 ){ /* As there exists an unusable MATCH constraint this is an ** unusable plan. Return SQLITE_CONSTRAINT. */ idxStr[iIdxStr] = 0; return SQLITE_CONSTRAINT; }else{ if( iCol==nCol+1 ){ if( bSeenRank ) continue; idxStr[iIdxStr++] = 'r'; bSeenRank = 1; }else{ nSeenMatch++; idxStr[iIdxStr++] = 'M'; sqlite3_snprintf(6, &idxStr[iIdxStr], "%d", iCol); idxStr += strlen(&idxStr[iIdxStr]); assert( idxStr[iIdxStr]=='\0' ); } pInfo->aConstraintUsage[i].argvIndex = ++iCons; pInfo->aConstraintUsage[i].omit = 1; } }else if( p->usable ){ if( iCol>=0 && iCol<nCol && fts5UsePatternMatch(pConfig, p) ){ assert( p->op==FTS5_PATTERN_LIKE || p->op==FTS5_PATTERN_GLOB ); idxStr[iIdxStr++] = p->op==FTS5_PATTERN_LIKE ? 'L' : 'G'; sqlite3_snprintf(6, &idxStr[iIdxStr], "%d", iCol); idxStr += strlen(&idxStr[iIdxStr]); pInfo->aConstraintUsage[i].argvIndex = ++iCons; assert( idxStr[iIdxStr]=='\0' ); nSeenMatch++; }else if( bSeenEq==0 && p->op==SQLITE_INDEX_CONSTRAINT_EQ && iCol<0 ){ idxStr[iIdxStr++] = '='; bSeenEq = 1; pInfo->aConstraintUsage[i].argvIndex = ++iCons; } } } |
︙ | ︙ | |||
643 644 645 646 647 648 649 | bSeenGt = 1; } } } } idxStr[iIdxStr] = '\0'; | | > > > | | | | | | > > > | | 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 | bSeenGt = 1; } } } } idxStr[iIdxStr] = '\0'; /* Set idxFlags flags for the ORDER BY clause ** ** Note that tokendata=1 tables cannot currently handle "ORDER BY rowid DESC". */ if( pInfo->nOrderBy==1 ){ int iSort = pInfo->aOrderBy[0].iColumn; if( iSort==(pConfig->nCol+1) && nSeenMatch>0 ){ idxFlags |= FTS5_BI_ORDER_RANK; }else if( iSort==-1 && (!pInfo->aOrderBy[0].desc || !pConfig->bTokendata) ){ idxFlags |= FTS5_BI_ORDER_ROWID; } if( BitFlagTest(idxFlags, FTS5_BI_ORDER_RANK|FTS5_BI_ORDER_ROWID) ){ pInfo->orderByConsumed = 1; if( pInfo->aOrderBy[0].desc ){ idxFlags |= FTS5_BI_ORDER_DESC; } } } /* Calculate the estimated cost based on the flags set in idxFlags. */ if( bSeenEq ){ pInfo->estimatedCost = nSeenMatch ? 1000.0 : 10.0; if( nSeenMatch==0 ) fts5SetUniqueFlag(pInfo); }else if( bSeenLt && bSeenGt ){ pInfo->estimatedCost = nSeenMatch ? 5000.0 : 250000.0; }else if( bSeenLt || bSeenGt ){ pInfo->estimatedCost = nSeenMatch ? 7500.0 : 750000.0; }else{ pInfo->estimatedCost = nSeenMatch ? 10000.0 : 1000000.0; } for(i=1; i<nSeenMatch; i++){ pInfo->estimatedCost *= 0.4; } pInfo->idxNum = idxFlags; return SQLITE_OK; } static int fts5NewTransaction(Fts5FullTable *pTab){ |
︙ | ︙ | |||
899 900 901 902 903 904 905 906 907 908 909 910 911 912 | Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc; assert( (pCsr->ePlan<3)== (pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE) ); assert( !CsrFlagTest(pCsr, FTS5CSR_EOF) ); if( pCsr->ePlan<3 ){ int bSkip = 0; if( (rc = fts5CursorReseek(pCsr, &bSkip)) || bSkip ) return rc; rc = sqlite3Fts5ExprNext(pCsr->pExpr, pCsr->iLastRowid); CsrFlagSet(pCsr, sqlite3Fts5ExprEof(pCsr->pExpr)); fts5CsrNewrow(pCsr); | > > > > > > > > > > | 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 | Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc; assert( (pCsr->ePlan<3)== (pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE) ); assert( !CsrFlagTest(pCsr, FTS5CSR_EOF) ); /* If this cursor uses FTS5_PLAN_MATCH and this is a tokendata=1 table, ** clear any token mappings accumulated at the fts5_index.c level. In ** other cases, specifically FTS5_PLAN_SOURCE and FTS5_PLAN_SORTED_MATCH, ** we need to retain the mappings for the entire query. */ if( pCsr->ePlan==FTS5_PLAN_MATCH && ((Fts5Table*)pCursor->pVtab)->pConfig->bTokendata ){ sqlite3Fts5ExprClearTokens(pCsr->pExpr); } if( pCsr->ePlan<3 ){ int bSkip = 0; if( (rc = fts5CursorReseek(pCsr, &bSkip)) || bSkip ) return rc; rc = sqlite3Fts5ExprNext(pCsr->pExpr, pCsr->iLastRowid); CsrFlagSet(pCsr, sqlite3Fts5ExprEof(pCsr->pExpr)); fts5CsrNewrow(pCsr); |
︙ | ︙ | |||
934 935 936 937 938 939 940 941 942 943 944 945 946 947 | if( rc!=SQLITE_OK ){ pCursor->pVtab->zErrMsg = sqlite3_mprintf( "%s", sqlite3_errmsg(pConfig->db) ); } }else{ rc = SQLITE_OK; } break; } } } return rc; | > | 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 | if( rc!=SQLITE_OK ){ pCursor->pVtab->zErrMsg = sqlite3_mprintf( "%s", sqlite3_errmsg(pConfig->db) ); } }else{ rc = SQLITE_OK; CsrFlagSet(pCsr, FTS5CSR_REQUIRE_DOCSIZE); } break; } } } return rc; |
︙ | ︙ | |||
963 964 965 966 967 968 969 | zSql = sqlite3_vmprintf(zFmt, ap); if( zSql==0 ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare_v3(pConfig->db, zSql, -1, SQLITE_PREPARE_PERSISTENT, &pRet, 0); if( rc!=SQLITE_OK ){ | | | 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 | zSql = sqlite3_vmprintf(zFmt, ap); if( zSql==0 ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare_v3(pConfig->db, zSql, -1, SQLITE_PREPARE_PERSISTENT, &pRet, 0); if( rc!=SQLITE_OK ){ sqlite3Fts5ConfigErrmsg(pConfig, "%s", sqlite3_errmsg(pConfig->db)); } sqlite3_free(zSql); } va_end(ap); *ppStmt = pRet; return rc; |
︙ | ︙ | |||
987 988 989 990 991 992 993 | int nPhrase; sqlite3_int64 nByte; int rc; const char *zRank = pCsr->zRank; const char *zRankArgs = pCsr->zRankArgs; nPhrase = sqlite3Fts5ExprPhraseCount(pCsr->pExpr); | | | 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 | int nPhrase; sqlite3_int64 nByte; int rc; const char *zRank = pCsr->zRank; const char *zRankArgs = pCsr->zRankArgs; nPhrase = sqlite3Fts5ExprPhraseCount(pCsr->pExpr); nByte = SZ_FTS5SORTER(nPhrase); pSorter = (Fts5Sorter*)sqlite3_malloc64(nByte); if( pSorter==0 ) return SQLITE_NOMEM; memset(pSorter, 0, (size_t)nByte); pSorter->nIdx = nPhrase; /* TODO: It would be better to have some system for reusing statement ** handles here, rather than preparing a new one for each query. But that |
︙ | ︙ | |||
1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 | int eType = sqlite3_value_numeric_type(pVal); if( eType==SQLITE_INTEGER ){ return sqlite3_value_int64(pVal); } } return iDefault; } /* ** This is the xFilter interface for the virtual table. See ** the virtual table xFilter method documentation for additional ** information. ** ** There are three possible query strategies: | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 | int eType = sqlite3_value_numeric_type(pVal); if( eType==SQLITE_INTEGER ){ return sqlite3_value_int64(pVal); } } return iDefault; } /* ** Set the error message on the virtual table passed as the first argument. */ static void fts5SetVtabError(Fts5FullTable *p, const char *zFormat, ...){ va_list ap; /* ... printf arguments */ va_start(ap, zFormat); sqlite3_free(p->p.base.zErrMsg); p->p.base.zErrMsg = sqlite3_vmprintf(zFormat, ap); va_end(ap); } /* ** Arrange for subsequent calls to sqlite3Fts5Tokenize() to use the locale ** specified by pLocale/nLocale. The buffer indicated by pLocale must remain ** valid until after the final call to sqlite3Fts5Tokenize() that will use ** the locale. */ static void sqlite3Fts5SetLocale( Fts5Config *pConfig, const char *zLocale, int nLocale ){ Fts5TokenizerConfig *pT = &pConfig->t; pT->pLocale = zLocale; pT->nLocale = nLocale; } /* ** Clear any locale configured by an earlier call to sqlite3Fts5SetLocale(). */ void sqlite3Fts5ClearLocale(Fts5Config *pConfig){ sqlite3Fts5SetLocale(pConfig, 0, 0); } /* ** Return true if the value passed as the only argument is an ** fts5_locale() value. */ int sqlite3Fts5IsLocaleValue(Fts5Config *pConfig, sqlite3_value *pVal){ int ret = 0; if( sqlite3_value_type(pVal)==SQLITE_BLOB ){ /* Call sqlite3_value_bytes() after sqlite3_value_blob() in this case. ** If the blob was created using zeroblob(), then sqlite3_value_blob() ** may call malloc(). If this malloc() fails, then the values returned ** by both value_blob() and value_bytes() will be 0. If value_bytes() were ** called first, then the NULL pointer returned by value_blob() might ** be dereferenced. */ const u8 *pBlob = sqlite3_value_blob(pVal); int nBlob = sqlite3_value_bytes(pVal); if( nBlob>FTS5_LOCALE_HDR_SIZE && 0==memcmp(pBlob, FTS5_LOCALE_HDR(pConfig), FTS5_LOCALE_HDR_SIZE) ){ ret = 1; } } return ret; } /* ** Value pVal is guaranteed to be an fts5_locale() value, according to ** sqlite3Fts5IsLocaleValue(). This function extracts the text and locale ** from the value and returns them separately. ** ** If successful, SQLITE_OK is returned and (*ppText) and (*ppLoc) set ** to point to buffers containing the text and locale, as utf-8, ** respectively. In this case output parameters (*pnText) and (*pnLoc) are ** set to the sizes in bytes of these two buffers. ** ** Or, if an error occurs, then an SQLite error code is returned. The final ** value of the four output parameters is undefined in this case. */ int sqlite3Fts5DecodeLocaleValue( sqlite3_value *pVal, const char **ppText, int *pnText, const char **ppLoc, int *pnLoc ){ const char *p = sqlite3_value_blob(pVal); int n = sqlite3_value_bytes(pVal); int nLoc = 0; assert( sqlite3_value_type(pVal)==SQLITE_BLOB ); assert( n>FTS5_LOCALE_HDR_SIZE ); for(nLoc=FTS5_LOCALE_HDR_SIZE; p[nLoc]; nLoc++){ if( nLoc==(n-1) ){ return SQLITE_MISMATCH; } } *ppLoc = &p[FTS5_LOCALE_HDR_SIZE]; *pnLoc = nLoc - FTS5_LOCALE_HDR_SIZE; *ppText = &p[nLoc+1]; *pnText = n - nLoc - 1; return SQLITE_OK; } /* ** Argument pVal is the text of a full-text search expression. It may or ** may not have been wrapped by fts5_locale(). This function extracts ** the text of the expression, and sets output variable (*pzText) to ** point to a nul-terminated buffer containing the expression. ** ** If pVal was an fts5_locale() value, then sqlite3Fts5SetLocale() is called ** to set the tokenizer to use the specified locale. ** ** If output variable (*pbFreeAndReset) is set to true, then the caller ** is required to (a) call sqlite3Fts5ClearLocale() to reset the tokenizer ** locale, and (b) call sqlite3_free() to free (*pzText). */ static int fts5ExtractExprText( Fts5Config *pConfig, /* Fts5 configuration */ sqlite3_value *pVal, /* Value to extract expression text from */ char **pzText, /* OUT: nul-terminated buffer of text */ int *pbFreeAndReset /* OUT: Free (*pzText) and clear locale */ ){ int rc = SQLITE_OK; if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ const char *pText = 0; int nText = 0; const char *pLoc = 0; int nLoc = 0; rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc); *pzText = sqlite3Fts5Mprintf(&rc, "%.*s", nText, pText); if( rc==SQLITE_OK ){ sqlite3Fts5SetLocale(pConfig, pLoc, nLoc); } *pbFreeAndReset = 1; }else{ *pzText = (char*)sqlite3_value_text(pVal); *pbFreeAndReset = 0; } return rc; } /* ** This is the xFilter interface for the virtual table. See ** the virtual table xFilter method documentation for additional ** information. ** ** There are three possible query strategies: |
︙ | ︙ | |||
1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 | int bOrderByRank; /* True if ORDER BY rank */ sqlite3_value *pRank = 0; /* rank MATCH ? expression (or NULL) */ sqlite3_value *pRowidEq = 0; /* rowid = ? expression (or NULL) */ sqlite3_value *pRowidLe = 0; /* rowid <= ? expression (or NULL) */ sqlite3_value *pRowidGe = 0; /* rowid >= ? expression (or NULL) */ int iCol; /* Column on LHS of MATCH operator */ char **pzErrmsg = pConfig->pzErrmsg; int i; int iIdxStr = 0; Fts5Expr *pExpr = 0; | > | < < < < < < | 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 | int bOrderByRank; /* True if ORDER BY rank */ sqlite3_value *pRank = 0; /* rank MATCH ? expression (or NULL) */ sqlite3_value *pRowidEq = 0; /* rowid = ? expression (or NULL) */ sqlite3_value *pRowidLe = 0; /* rowid <= ? expression (or NULL) */ sqlite3_value *pRowidGe = 0; /* rowid >= ? expression (or NULL) */ int iCol; /* Column on LHS of MATCH operator */ char **pzErrmsg = pConfig->pzErrmsg; int bPrefixInsttoken = pConfig->bPrefixInsttoken; int i; int iIdxStr = 0; Fts5Expr *pExpr = 0; assert( pConfig->bLock==0 ); if( pCsr->ePlan ){ fts5FreeCursorComponents(pCsr); memset(&pCsr->ePlan, 0, sizeof(Fts5Cursor) - ((u8*)&pCsr->ePlan-(u8*)pCsr)); } assert( pCsr->pStmt==0 ); assert( pCsr->pExpr==0 ); |
︙ | ︙ | |||
1251 1252 1253 1254 1255 1256 1257 | /* Decode the arguments passed through to this function. */ for(i=0; i<nVal; i++){ switch( idxStr[iIdxStr++] ){ case 'r': pRank = apVal[i]; break; case 'M': { | | > > > > > > > > > | > | > > > > > | 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 | /* Decode the arguments passed through to this function. */ for(i=0; i<nVal; i++){ switch( idxStr[iIdxStr++] ){ case 'r': pRank = apVal[i]; break; case 'M': { char *zText = 0; int bFreeAndReset = 0; int bInternal = 0; rc = fts5ExtractExprText(pConfig, apVal[i], &zText, &bFreeAndReset); if( rc!=SQLITE_OK ) goto filter_out; if( zText==0 ) zText = ""; if( sqlite3_value_subtype(apVal[i])==FTS5_INSTTOKEN_SUBTYPE ){ pConfig->bPrefixInsttoken = 1; } iCol = 0; do{ iCol = iCol*10 + (idxStr[iIdxStr]-'0'); iIdxStr++; }while( idxStr[iIdxStr]>='0' && idxStr[iIdxStr]<='9' ); if( zText[0]=='*' ){ /* The user has issued a query of the form "MATCH '*...'". This ** indicates that the MATCH expression is not a full text query, ** but a request for an internal parameter. */ rc = fts5SpecialMatch(pTab, pCsr, &zText[1]); bInternal = 1; }else{ char **pzErr = &pTab->p.base.zErrMsg; rc = sqlite3Fts5ExprNew(pConfig, 0, iCol, zText, &pExpr, pzErr); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ExprAnd(&pCsr->pExpr, pExpr); pExpr = 0; } } if( bFreeAndReset ){ sqlite3_free(zText); sqlite3Fts5ClearLocale(pConfig); } if( bInternal || rc!=SQLITE_OK ) goto filter_out; break; } case 'L': case 'G': { int bGlob = (idxStr[iIdxStr-1]=='G'); const char *zText = (const char*)sqlite3_value_text(apVal[i]); |
︙ | ︙ | |||
1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 | pCsr->iFirstRowid = fts5GetRowidLimit(pRowidLe, LARGEST_INT64); pCsr->iLastRowid = fts5GetRowidLimit(pRowidGe, SMALLEST_INT64); }else{ pCsr->iLastRowid = fts5GetRowidLimit(pRowidLe, LARGEST_INT64); pCsr->iFirstRowid = fts5GetRowidLimit(pRowidGe, SMALLEST_INT64); } if( pTab->pSortCsr ){ /* If pSortCsr is non-NULL, then this call is being made as part of ** processing for a "... MATCH <expr> ORDER BY rank" query (ePlan is ** set to FTS5_PLAN_SORTED_MATCH). pSortCsr is the cursor that will ** return results to the user for this query. The current cursor ** (pCursor) is used to execute the query issued by function ** fts5CursorFirstSorted() above. */ | > > > | 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 | pCsr->iFirstRowid = fts5GetRowidLimit(pRowidLe, LARGEST_INT64); pCsr->iLastRowid = fts5GetRowidLimit(pRowidGe, SMALLEST_INT64); }else{ pCsr->iLastRowid = fts5GetRowidLimit(pRowidLe, LARGEST_INT64); pCsr->iFirstRowid = fts5GetRowidLimit(pRowidGe, SMALLEST_INT64); } rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex); if( rc!=SQLITE_OK ) goto filter_out; if( pTab->pSortCsr ){ /* If pSortCsr is non-NULL, then this call is being made as part of ** processing for a "... MATCH <expr> ORDER BY rank" query (ePlan is ** set to FTS5_PLAN_SORTED_MATCH). pSortCsr is the cursor that will ** return results to the user for this query. The current cursor ** (pCursor) is used to execute the query issued by function ** fts5CursorFirstSorted() above. */ |
︙ | ︙ | |||
1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 | pCsr->iLastRowid = pTab->pSortCsr->iLastRowid; pCsr->iFirstRowid = pTab->pSortCsr->iFirstRowid; } pCsr->ePlan = FTS5_PLAN_SOURCE; pCsr->pExpr = pTab->pSortCsr->pExpr; rc = fts5CursorFirst(pTab, pCsr, bDesc); }else if( pCsr->pExpr ){ rc = fts5CursorParseRank(pConfig, pCsr, pRank); if( rc==SQLITE_OK ){ if( bOrderByRank ){ pCsr->ePlan = FTS5_PLAN_SORTED_MATCH; rc = fts5CursorFirstSorted(pTab, pCsr, bDesc); }else{ pCsr->ePlan = FTS5_PLAN_MATCH; rc = fts5CursorFirst(pTab, pCsr, bDesc); } } }else if( pConfig->zContent==0 ){ | > < | < | 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 | pCsr->iLastRowid = pTab->pSortCsr->iLastRowid; pCsr->iFirstRowid = pTab->pSortCsr->iFirstRowid; } pCsr->ePlan = FTS5_PLAN_SOURCE; pCsr->pExpr = pTab->pSortCsr->pExpr; rc = fts5CursorFirst(pTab, pCsr, bDesc); }else if( pCsr->pExpr ){ assert( rc==SQLITE_OK ); rc = fts5CursorParseRank(pConfig, pCsr, pRank); if( rc==SQLITE_OK ){ if( bOrderByRank ){ pCsr->ePlan = FTS5_PLAN_SORTED_MATCH; rc = fts5CursorFirstSorted(pTab, pCsr, bDesc); }else{ pCsr->ePlan = FTS5_PLAN_MATCH; rc = fts5CursorFirst(pTab, pCsr, bDesc); } } }else if( pConfig->zContent==0 ){ fts5SetVtabError(pTab,"%s: table does not support scanning",pConfig->zName); rc = SQLITE_ERROR; }else{ /* This is either a full-table scan (ePlan==FTS5_PLAN_SCAN) or a lookup ** by rowid (ePlan==FTS5_PLAN_ROWID). */ pCsr->ePlan = (pRowidEq ? FTS5_PLAN_ROWID : FTS5_PLAN_SCAN); rc = sqlite3Fts5StorageStmt( pTab->pStorage, fts5StmtType(pCsr), &pCsr->pStmt, &pTab->p.base.zErrMsg |
︙ | ︙ | |||
1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 | rc = fts5NextMethod(pCursor); } } filter_out: sqlite3Fts5ExprFree(pExpr); pConfig->pzErrmsg = pzErrmsg; return rc; } /* ** This is the xEof method of the virtual table. SQLite calls this ** routine to find out if it has reached the end of a result set. */ static int fts5EofMethod(sqlite3_vtab_cursor *pCursor){ Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; return (CsrFlagTest(pCsr, FTS5CSR_EOF) ? 1 : 0); } /* ** Return the rowid that the cursor currently points to. */ static i64 fts5CursorRowid(Fts5Cursor *pCsr){ assert( pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SORTED_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE ); if( pCsr->pSorter ){ return pCsr->pSorter->iRowid; }else{ return sqlite3Fts5ExprRowid(pCsr->pExpr); } } /* ** This is the xRowid method. The SQLite core calls this routine to ** retrieve the rowid for the current row of the result set. fts5 ** exposes %_content.rowid as the rowid for the virtual table. The ** rowid should be written to *pRowid. */ static int fts5RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int ePlan = pCsr->ePlan; assert( CsrFlagTest(pCsr, FTS5CSR_EOF)==0 ); | > > > > > < | | < | < < < | < < < < < > | 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 | rc = fts5NextMethod(pCursor); } } filter_out: sqlite3Fts5ExprFree(pExpr); pConfig->pzErrmsg = pzErrmsg; pConfig->bPrefixInsttoken = bPrefixInsttoken; return rc; } /* ** This is the xEof method of the virtual table. SQLite calls this ** routine to find out if it has reached the end of a result set. */ static int fts5EofMethod(sqlite3_vtab_cursor *pCursor){ Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; return (CsrFlagTest(pCsr, FTS5CSR_EOF) ? 1 : 0); } /* ** Return the rowid that the cursor currently points to. */ static i64 fts5CursorRowid(Fts5Cursor *pCsr){ assert( pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SORTED_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE || pCsr->ePlan==FTS5_PLAN_SCAN || pCsr->ePlan==FTS5_PLAN_ROWID ); if( pCsr->pSorter ){ return pCsr->pSorter->iRowid; }else if( pCsr->ePlan>=FTS5_PLAN_SCAN ){ return sqlite3_column_int64(pCsr->pStmt, 0); }else{ return sqlite3Fts5ExprRowid(pCsr->pExpr); } } /* ** This is the xRowid method. The SQLite core calls this routine to ** retrieve the rowid for the current row of the result set. fts5 ** exposes %_content.rowid as the rowid for the virtual table. The ** rowid should be written to *pRowid. */ static int fts5RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int ePlan = pCsr->ePlan; assert( CsrFlagTest(pCsr, FTS5CSR_EOF)==0 ); if( ePlan==FTS5_PLAN_SPECIAL ){ *pRowid = 0; }else{ *pRowid = fts5CursorRowid(pCsr); } return SQLITE_OK; } /* ** If the cursor requires seeking (bSeekRequired flag is set), seek it. ** Return SQLITE_OK if no error occurs, or an SQLite error code otherwise. ** ** If argument bErrormsg is true and an error occurs, an error message may ** be left in sqlite3_vtab.zErrMsg. |
︙ | ︙ | |||
1477 1478 1479 1480 1481 1482 1483 1484 | if( rc==SQLITE_ROW ){ rc = SQLITE_OK; CsrFlagClear(pCsr, FTS5CSR_REQUIRE_CONTENT); }else{ rc = sqlite3_reset(pCsr->pStmt); if( rc==SQLITE_OK ){ rc = FTS5_CORRUPT; }else if( pTab->pConfig->pzErrmsg ){ | > > > > > | < < < < < < < < | 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 | if( rc==SQLITE_ROW ){ rc = SQLITE_OK; CsrFlagClear(pCsr, FTS5CSR_REQUIRE_CONTENT); }else{ rc = sqlite3_reset(pCsr->pStmt); if( rc==SQLITE_OK ){ rc = FTS5_CORRUPT; fts5SetVtabError((Fts5FullTable*)pTab, "fts5: missing row %lld from content table %s", fts5CursorRowid(pCsr), pTab->pConfig->zContent ); }else if( pTab->pConfig->pzErrmsg ){ fts5SetVtabError((Fts5FullTable*)pTab, "%s", sqlite3_errmsg(pTab->pConfig->db) ); } } } return rc; } /* ** This function is called to handle an FTS INSERT command. In other words, ** an INSERT statement of the form: ** ** INSERT INTO fts(fts) VALUES($pCmd) ** INSERT INTO fts(fts, rank) VALUES($pCmd, $pVal) ** |
︙ | ︙ | |||
1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 | Fts5FullTable *pTab, /* Fts5 table object */ const char *zCmd, /* Text inserted into table-name column */ sqlite3_value *pVal /* Value inserted into rank column */ ){ Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; int bError = 0; if( 0==sqlite3_stricmp("delete-all", zCmd) ){ if( pConfig->eContent==FTS5_CONTENT_NORMAL ){ fts5SetVtabError(pTab, "'delete-all' may only be used with a " "contentless or external content fts5 table" ); rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5StorageDeleteAll(pTab->pStorage); } }else if( 0==sqlite3_stricmp("rebuild", zCmd) ){ | > > | > > > > > | > > > > > > > | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 | Fts5FullTable *pTab, /* Fts5 table object */ const char *zCmd, /* Text inserted into table-name column */ sqlite3_value *pVal /* Value inserted into rank column */ ){ Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; int bError = 0; int bLoadConfig = 0; if( 0==sqlite3_stricmp("delete-all", zCmd) ){ if( pConfig->eContent==FTS5_CONTENT_NORMAL ){ fts5SetVtabError(pTab, "'delete-all' may only be used with a " "contentless or external content fts5 table" ); rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5StorageDeleteAll(pTab->pStorage); } bLoadConfig = 1; }else if( 0==sqlite3_stricmp("rebuild", zCmd) ){ if( fts5IsContentless(pTab, 1) ){ fts5SetVtabError(pTab, "'rebuild' may not be used with a contentless fts5 table" ); rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5StorageRebuild(pTab->pStorage); } bLoadConfig = 1; }else if( 0==sqlite3_stricmp("optimize", zCmd) ){ rc = sqlite3Fts5StorageOptimize(pTab->pStorage); }else if( 0==sqlite3_stricmp("merge", zCmd) ){ int nMerge = sqlite3_value_int(pVal); rc = sqlite3Fts5StorageMerge(pTab->pStorage, nMerge); }else if( 0==sqlite3_stricmp("integrity-check", zCmd) ){ int iArg = sqlite3_value_int(pVal); rc = sqlite3Fts5StorageIntegrity(pTab->pStorage, iArg); #ifdef SQLITE_DEBUG }else if( 0==sqlite3_stricmp("prefix-index", zCmd) ){ pConfig->bPrefixIndex = sqlite3_value_int(pVal); #endif }else if( 0==sqlite3_stricmp("flush", zCmd) ){ rc = sqlite3Fts5FlushToDisk(&pTab->p); }else{ rc = sqlite3Fts5FlushToDisk(&pTab->p); if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5ConfigSetValue(pTab->p.pConfig, zCmd, pVal, &bError); } if( rc==SQLITE_OK ){ if( bError ){ rc = SQLITE_ERROR; }else{ rc = sqlite3Fts5StorageConfigValue(pTab->pStorage, zCmd, pVal, 0); } } } if( rc==SQLITE_OK && bLoadConfig ){ pTab->p.pConfig->iCookie--; rc = sqlite3Fts5IndexLoadConfig(pTab->p.pIndex); } return rc; } static int fts5SpecialDelete( Fts5FullTable *pTab, sqlite3_value **apVal ){ int rc = SQLITE_OK; int eType1 = sqlite3_value_type(apVal[1]); if( eType1==SQLITE_INTEGER ){ sqlite3_int64 iDel = sqlite3_value_int64(apVal[1]); rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, &apVal[2], 0); } return rc; } static void fts5StorageInsert( int *pRc, Fts5FullTable *pTab, sqlite3_value **apVal, i64 *piRowid ){ int rc = *pRc; if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageContentInsert(pTab->pStorage, 0, apVal, piRowid); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageIndexInsert(pTab->pStorage, apVal, *piRowid); } *pRc = rc; } /* ** ** This function is called when the user attempts an UPDATE on a contentless ** table. Parameter bRowidModified is true if the UPDATE statement modifies ** the rowid value. Parameter apVal[] contains the new values for each user ** defined column of the fts5 table. pConfig is the configuration object of the ** table being updated (guaranteed to be contentless). The contentless_delete=1 ** and contentless_unindexed=1 options may or may not be set. ** ** This function returns SQLITE_OK if the UPDATE can go ahead, or an SQLite ** error code if it cannot. In this case an error message is also loaded into ** pConfig. Output parameter (*pbContent) is set to true if the caller should ** update the %_content table only - not the FTS index or any other shadow ** table. This occurs when an UPDATE modifies only UNINDEXED columns of the ** table. ** ** An UPDATE may proceed if: ** ** * The only columns modified are UNINDEXED columns, or ** ** * The contentless_delete=1 option was specified and all of the indexed ** columns (not a subset) have been modified. */ static int fts5ContentlessUpdate( Fts5Config *pConfig, sqlite3_value **apVal, int bRowidModified, int *pbContent ){ int ii; int bSeenIndex = 0; /* Have seen modified indexed column */ int bSeenIndexNC = 0; /* Have seen unmodified indexed column */ int rc = SQLITE_OK; for(ii=0; ii<pConfig->nCol; ii++){ if( pConfig->abUnindexed[ii]==0 ){ if( sqlite3_value_nochange(apVal[ii]) ){ bSeenIndexNC++; }else{ bSeenIndex++; } } } if( bSeenIndex==0 && bRowidModified==0 ){ *pbContent = 1; }else{ if( bSeenIndexNC || pConfig->bContentlessDelete==0 ){ rc = SQLITE_ERROR; sqlite3Fts5ConfigErrmsg(pConfig, (pConfig->bContentlessDelete ? "%s a subset of columns on fts5 contentless-delete table: %s" : "%s contentless fts5 table: %s") , "cannot UPDATE", pConfig->zName ); } } return rc; } /* ** This function is the implementation of the xUpdate callback used by ** FTS3 virtual tables. It is invoked by SQLite each time a row is to be ** inserted, updated or deleted. ** ** A delete specifies a single argument - the rowid of the row to remove. |
︙ | ︙ | |||
1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 | assert( pVtab->zErrMsg==0 ); assert( nArg==1 || nArg==(2+pConfig->nCol+2) ); assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER || sqlite3_value_type(apVal[0])==SQLITE_NULL ); assert( pTab->p.pConfig->pzErrmsg==0 ); pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg; /* Put any active cursors into REQUIRE_SEEK state. */ fts5TripCursors(pTab); eType0 = sqlite3_value_type(apVal[0]); if( eType0==SQLITE_NULL && sqlite3_value_type(apVal[2+pConfig->nCol])!=SQLITE_NULL ){ /* A "special" INSERT op. These are handled separately. */ const char *z = (const char*)sqlite3_value_text(apVal[2+pConfig->nCol]); if( pConfig->eContent!=FTS5_CONTENT_NORMAL && 0==sqlite3_stricmp("delete", z) ){ | > > > > > > > > > > > | > | | | > > | | | < | | | < < < | | > > > | > > > > > | > | | > > | > | | > > > > > > > > > > | > > > | | | | > > > | > > > > > > > > > | > > < | > > | < > | | 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 1924 1925 1926 1927 1928 1929 1930 1931 1932 1933 1934 1935 1936 1937 1938 1939 1940 1941 1942 1943 1944 1945 1946 1947 1948 1949 1950 1951 1952 1953 1954 1955 1956 1957 1958 1959 1960 1961 1962 1963 1964 1965 1966 1967 1968 1969 1970 1971 1972 1973 1974 1975 1976 1977 1978 1979 1980 1981 1982 1983 1984 1985 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 | assert( pVtab->zErrMsg==0 ); assert( nArg==1 || nArg==(2+pConfig->nCol+2) ); assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER || sqlite3_value_type(apVal[0])==SQLITE_NULL ); assert( pTab->p.pConfig->pzErrmsg==0 ); if( pConfig->pgsz==0 ){ rc = sqlite3Fts5ConfigLoad(pTab->p.pConfig, pTab->p.pConfig->iCookie); if( rc!=SQLITE_OK ) return rc; } pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg; /* Put any active cursors into REQUIRE_SEEK state. */ fts5TripCursors(pTab); eType0 = sqlite3_value_type(apVal[0]); if( eType0==SQLITE_NULL && sqlite3_value_type(apVal[2+pConfig->nCol])!=SQLITE_NULL ){ /* A "special" INSERT op. These are handled separately. */ const char *z = (const char*)sqlite3_value_text(apVal[2+pConfig->nCol]); if( pConfig->eContent!=FTS5_CONTENT_NORMAL && 0==sqlite3_stricmp("delete", z) ){ if( pConfig->bContentlessDelete ){ fts5SetVtabError(pTab, "'delete' may not be used with a contentless_delete=1 table" ); rc = SQLITE_ERROR; }else{ rc = fts5SpecialDelete(pTab, apVal); } }else{ rc = fts5SpecialInsert(pTab, z, apVal[2 + pConfig->nCol + 1]); } }else{ /* A regular INSERT, UPDATE or DELETE statement. The trick here is that ** any conflict on the rowid value must be detected before any ** modifications are made to the database file. There are 4 cases: ** ** 1) DELETE ** 2) UPDATE (rowid not modified) ** 3) UPDATE (rowid modified) ** 4) INSERT ** ** Cases 3 and 4 may violate the rowid constraint. */ int eConflict = SQLITE_ABORT; if( pConfig->eContent==FTS5_CONTENT_NORMAL || pConfig->bContentlessDelete ){ eConflict = sqlite3_vtab_on_conflict(pConfig->db); } assert( eType0==SQLITE_INTEGER || eType0==SQLITE_NULL ); assert( nArg!=1 || eType0==SQLITE_INTEGER ); /* DELETE */ if( nArg==1 ){ /* It is only possible to DELETE from a contentless table if the ** contentless_delete=1 flag is set. */ if( fts5IsContentless(pTab, 1) && pConfig->bContentlessDelete==0 ){ fts5SetVtabError(pTab, "cannot DELETE from contentless fts5 table: %s", pConfig->zName ); rc = SQLITE_ERROR; }else{ i64 iDel = sqlite3_value_int64(apVal[0]); /* Rowid to delete */ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iDel, 0, 0); } } /* INSERT or UPDATE */ else{ int eType1 = sqlite3_value_numeric_type(apVal[1]); /* It is an error to write an fts5_locale() value to a table without ** the locale=1 option. */ if( pConfig->bLocale==0 ){ int ii; for(ii=0; ii<pConfig->nCol; ii++){ sqlite3_value *pVal = apVal[ii+2]; if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ fts5SetVtabError(pTab, "fts5_locale() requires locale=1"); rc = SQLITE_MISMATCH; goto update_out; } } } if( eType0!=SQLITE_INTEGER ){ /* An INSERT statement. If the conflict-mode is REPLACE, first remove ** the current entry (if any). */ if( eConflict==SQLITE_REPLACE && eType1==SQLITE_INTEGER ){ i64 iNew = sqlite3_value_int64(apVal[1]); /* Rowid to delete */ rc = sqlite3Fts5StorageDelete(pTab->pStorage, iNew, 0, 0); } fts5StorageInsert(&rc, pTab, apVal, pRowid); } /* UPDATE */ else{ Fts5Storage *pStorage = pTab->pStorage; i64 iOld = sqlite3_value_int64(apVal[0]); /* Old rowid */ i64 iNew = sqlite3_value_int64(apVal[1]); /* New rowid */ int bContent = 0; /* Content only update */ /* If this is a contentless table (including contentless_unindexed=1 ** tables), check if the UPDATE may proceed. */ if( fts5IsContentless(pTab, 1) ){ rc = fts5ContentlessUpdate(pConfig, &apVal[2], iOld!=iNew, &bContent); if( rc!=SQLITE_OK ) goto update_out; } if( eType1!=SQLITE_INTEGER ){ rc = SQLITE_MISMATCH; }else if( iOld!=iNew ){ assert( bContent==0 ); if( eConflict==SQLITE_REPLACE ){ rc = sqlite3Fts5StorageDelete(pStorage, iOld, 0, 1); if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageDelete(pStorage, iNew, 0, 0); } fts5StorageInsert(&rc, pTab, apVal, pRowid); }else{ rc = sqlite3Fts5StorageFindDeleteRow(pStorage, iOld); if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageContentInsert(pStorage, 0, apVal, pRowid); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageDelete(pStorage, iOld, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageIndexInsert(pStorage, apVal, *pRowid); } } }else if( bContent ){ /* This occurs when an UPDATE on a contentless table affects *only* ** UNINDEXED columns. This is a no-op for contentless_unindexed=0 ** tables, or a write to the %_content table only for =1 tables. */ assert( fts5IsContentless(pTab, 1) ); rc = sqlite3Fts5StorageFindDeleteRow(pStorage, iOld); if( rc==SQLITE_OK ){ rc = sqlite3Fts5StorageContentInsert(pStorage, 1, apVal, pRowid); } }else{ rc = sqlite3Fts5StorageDelete(pStorage, iOld, 0, 1); fts5StorageInsert(&rc, pTab, apVal, pRowid); } sqlite3Fts5StorageReleaseDeleteRow(pStorage); } } } update_out: sqlite3Fts5IndexCloseReader(pTab->p.pIndex); pTab->p.pConfig->pzErrmsg = 0; return rc; } /* ** Implementation of xSync() method. */ static int fts5SyncMethod(sqlite3_vtab *pVtab){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; fts5CheckTransactionState(pTab, FTS5_SYNC, 0); pTab->p.pConfig->pzErrmsg = &pTab->p.base.zErrMsg; rc = sqlite3Fts5FlushToDisk(&pTab->p); pTab->p.pConfig->pzErrmsg = 0; return rc; } /* ** Implementation of xBegin() method. */ static int fts5BeginMethod(sqlite3_vtab *pVtab){ int rc = fts5NewTransaction((Fts5FullTable*)pVtab); if( rc==SQLITE_OK ){ fts5CheckTransactionState((Fts5FullTable*)pVtab, FTS5_BEGIN, 0); } return rc; } /* ** Implementation of xCommit() method. This is a no-op. The contents of ** the pending-terms hash-table have already been flushed into the database ** by fts5SyncMethod(). */ |
︙ | ︙ | |||
1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 | ** hash-table. Any changes made to the database are reverted by SQLite. */ static int fts5RollbackMethod(sqlite3_vtab *pVtab){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; fts5CheckTransactionState(pTab, FTS5_ROLLBACK, 0); rc = sqlite3Fts5StorageRollback(pTab->pStorage); return rc; } static int fts5CsrPoslist(Fts5Cursor*, int, const u8**, int*); static void *fts5ApiUserData(Fts5Context *pCtx){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | > | 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 | ** hash-table. Any changes made to the database are reverted by SQLite. */ static int fts5RollbackMethod(sqlite3_vtab *pVtab){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; fts5CheckTransactionState(pTab, FTS5_ROLLBACK, 0); rc = sqlite3Fts5StorageRollback(pTab->pStorage); pTab->p.pConfig->pgsz = 0; return rc; } static int fts5CsrPoslist(Fts5Cursor*, int, const u8**, int*); static void *fts5ApiUserData(Fts5Context *pCtx){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; |
︙ | ︙ | |||
1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 | static int fts5ApiRowCount(Fts5Context *pCtx, i64 *pnRow){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); return sqlite3Fts5StorageRowCount(pTab->pStorage, pnRow); } static int fts5ApiTokenize( Fts5Context *pCtx, const char *pText, int nText, void *pUserData, int (*xToken)(void*, int, const char*, int, int, int) ){ | > > > > > > > > > > > > > > > > > > > > > > > > > > > < < < | < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < < | | > > > > > > > | | | | > > > > > > > > | < > > > > | > | > > | | | | | | | > > > > | 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 | static int fts5ApiRowCount(Fts5Context *pCtx, i64 *pnRow){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5FullTable *pTab = (Fts5FullTable*)(pCsr->base.pVtab); return sqlite3Fts5StorageRowCount(pTab->pStorage, pnRow); } /* ** Implementation of xTokenize_v2() API. */ static int fts5ApiTokenize_v2( Fts5Context *pCtx, const char *pText, int nText, const char *pLoc, int nLoc, void *pUserData, int (*xToken)(void*, int, const char*, int, int, int) ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab); int rc = SQLITE_OK; sqlite3Fts5SetLocale(pTab->pConfig, pLoc, nLoc); rc = sqlite3Fts5Tokenize(pTab->pConfig, FTS5_TOKENIZE_AUX, pText, nText, pUserData, xToken ); sqlite3Fts5SetLocale(pTab->pConfig, 0, 0); return rc; } /* ** Implementation of xTokenize() API. This is just xTokenize_v2() with NULL/0 ** passed as the locale. */ static int fts5ApiTokenize( Fts5Context *pCtx, const char *pText, int nText, void *pUserData, int (*xToken)(void*, int, const char*, int, int, int) ){ return fts5ApiTokenize_v2(pCtx, pText, nText, 0, 0, pUserData, xToken); } static int fts5ApiPhraseCount(Fts5Context *pCtx){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; return sqlite3Fts5ExprPhraseCount(pCsr->pExpr); } static int fts5ApiPhraseSize(Fts5Context *pCtx, int iPhrase){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; return sqlite3Fts5ExprPhraseSize(pCsr->pExpr, iPhrase); } /* ** Argument pStmt is an SQL statement of the type used by Fts5Cursor. This ** function extracts the text value of column iCol of the current row. ** Additionally, if there is an associated locale, it invokes ** sqlite3Fts5SetLocale() to configure the tokenizer. In all cases the caller ** should invoke sqlite3Fts5ClearLocale() to clear the locale at some point ** after this function returns. ** ** If successful, (*ppText) is set to point to a buffer containing the text ** value as utf-8 and SQLITE_OK returned. (*pnText) is set to the size of that ** buffer in bytes. It is not guaranteed to be nul-terminated. If an error ** occurs, an SQLite error code is returned. The final values of the two ** output parameters are undefined in this case. */ static int fts5TextFromStmt( Fts5Config *pConfig, sqlite3_stmt *pStmt, int iCol, const char **ppText, int *pnText ){ sqlite3_value *pVal = sqlite3_column_value(pStmt, iCol+1); const char *pLoc = 0; int nLoc = 0; int rc = SQLITE_OK; if( pConfig->bLocale && pConfig->eContent==FTS5_CONTENT_EXTERNAL && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ rc = sqlite3Fts5DecodeLocaleValue(pVal, ppText, pnText, &pLoc, &nLoc); }else{ *ppText = (const char*)sqlite3_value_text(pVal); *pnText = sqlite3_value_bytes(pVal); if( pConfig->bLocale && pConfig->eContent==FTS5_CONTENT_NORMAL ){ pLoc = (const char*)sqlite3_column_text(pStmt, iCol+1+pConfig->nCol); nLoc = sqlite3_column_bytes(pStmt, iCol+1+pConfig->nCol); } } sqlite3Fts5SetLocale(pConfig, pLoc, nLoc); return rc; } static int fts5ApiColumnText( Fts5Context *pCtx, int iCol, const char **pz, int *pn ){ int rc = SQLITE_OK; Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5Table *pTab = (Fts5Table*)(pCsr->base.pVtab); assert( pCsr->ePlan!=FTS5_PLAN_SPECIAL ); if( iCol<0 || iCol>=pTab->pConfig->nCol ){ rc = SQLITE_RANGE; }else if( fts5IsContentless((Fts5FullTable*)(pCsr->base.pVtab), 0) ){ *pz = 0; *pn = 0; }else{ rc = fts5SeekCursor(pCsr, 0); if( rc==SQLITE_OK ){ rc = fts5TextFromStmt(pTab->pConfig, pCsr->pStmt, iCol, pz, pn); sqlite3Fts5ClearLocale(pTab->pConfig); } } return rc; } /* ** This is called by various API functions - xInst, xPhraseFirst, ** xPhraseFirstColumn etc. - to obtain the position list for phrase iPhrase ** of the current row. This function works for both detail=full tables (in ** which case the position-list was read from the fts index) or for other ** detail= modes if the row content is available. */ static int fts5CsrPoslist( Fts5Cursor *pCsr, /* Fts5 cursor object */ int iPhrase, /* Phrase to find position list for */ const u8 **pa, /* OUT: Pointer to position list buffer */ int *pn /* OUT: Size of (*pa) in bytes */ ){ Fts5Config *pConfig = ((Fts5Table*)(pCsr->base.pVtab))->pConfig; int rc = SQLITE_OK; int bLive = (pCsr->pSorter==0); if( iPhrase<0 || iPhrase>=sqlite3Fts5ExprPhraseCount(pCsr->pExpr) ){ rc = SQLITE_RANGE; }else if( pConfig->eDetail!=FTS5_DETAIL_FULL && fts5IsContentless((Fts5FullTable*)pCsr->base.pVtab, 1) ){ *pa = 0; *pn = 0; return SQLITE_OK; }else if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_POSLIST) ){ if( pConfig->eDetail!=FTS5_DETAIL_FULL ){ Fts5PoslistPopulator *aPopulator; int i; aPopulator = sqlite3Fts5ExprClearPoslists(pCsr->pExpr, bLive); if( aPopulator==0 ) rc = SQLITE_NOMEM; if( rc==SQLITE_OK ){ rc = fts5SeekCursor(pCsr, 0); } for(i=0; i<pConfig->nCol && rc==SQLITE_OK; i++){ const char *z = 0; int n = 0; rc = fts5TextFromStmt(pConfig, pCsr->pStmt, i, &z, &n); if( rc==SQLITE_OK ){ rc = sqlite3Fts5ExprPopulatePoslists( pConfig, pCsr->pExpr, aPopulator, i, z, n ); } sqlite3Fts5ClearLocale(pConfig); } sqlite3_free(aPopulator); if( pCsr->pSorter ){ sqlite3Fts5ExprCheckPoslists(pCsr->pExpr, pCsr->pSorter->iRowid); } } CsrFlagClear(pCsr, FTS5CSR_REQUIRE_POSLIST); } if( rc==SQLITE_OK ){ if( pCsr->pSorter && pConfig->eDetail==FTS5_DETAIL_FULL ){ Fts5Sorter *pSorter = pCsr->pSorter; int i1 = (iPhrase==0 ? 0 : pSorter->aIdx[iPhrase-1]); *pn = pSorter->aIdx[iPhrase] - i1; *pa = &pSorter->aPoslist[i1]; }else{ *pn = sqlite3Fts5ExprPoslist(pCsr->pExpr, iPhrase, pa); } }else{ *pa = 0; *pn = 0; } return rc; } /* ** Ensure that the Fts5Cursor.nInstCount and aInst[] variables are populated |
︙ | ︙ | |||
1965 1966 1967 1968 1969 1970 1971 | } } aInst = &pCsr->aInst[3 * (nInst-1)]; aInst[0] = iBest; aInst[1] = FTS5_POS2COLUMN(aIter[iBest].iPos); aInst[2] = FTS5_POS2OFFSET(aIter[iBest].iPos); | > | | 2390 2391 2392 2393 2394 2395 2396 2397 2398 2399 2400 2401 2402 2403 2404 2405 | } } aInst = &pCsr->aInst[3 * (nInst-1)]; aInst[0] = iBest; aInst[1] = FTS5_POS2COLUMN(aIter[iBest].iPos); aInst[2] = FTS5_POS2OFFSET(aIter[iBest].iPos); assert( aInst[1]>=0 ); if( aInst[1]>=nCol ){ rc = FTS5_CORRUPT; break; } sqlite3Fts5PoslistReaderNext(&aIter[iBest]); } } |
︙ | ︙ | |||
2003 2004 2005 2006 2007 2008 2009 | Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) ){ if( iIdx<0 || iIdx>=pCsr->nInstCount ){ rc = SQLITE_RANGE; | < < < < < < | 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 | Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) ){ if( iIdx<0 || iIdx>=pCsr->nInstCount ){ rc = SQLITE_RANGE; }else{ *piPhrase = pCsr->aInst[iIdx*3]; *piCol = pCsr->aInst[iIdx*3 + 1]; *piOff = pCsr->aInst[iIdx*3 + 2]; } } return rc; |
︙ | ︙ | |||
2049 2050 2051 2052 2053 2054 2055 | Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_DOCSIZE) ){ if( pConfig->bColumnsize ){ i64 iRowid = fts5CursorRowid(pCsr); rc = sqlite3Fts5StorageDocsize(pTab->pStorage, iRowid, pCsr->aColumnSize); | | > | | | | | > | 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 | Fts5Config *pConfig = pTab->p.pConfig; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_DOCSIZE) ){ if( pConfig->bColumnsize ){ i64 iRowid = fts5CursorRowid(pCsr); rc = sqlite3Fts5StorageDocsize(pTab->pStorage, iRowid, pCsr->aColumnSize); }else if( !pConfig->zContent || pConfig->eContent==FTS5_CONTENT_UNINDEXED ){ int i; for(i=0; i<pConfig->nCol; i++){ if( pConfig->abUnindexed[i]==0 ){ pCsr->aColumnSize[i] = -1; } } }else{ int i; rc = fts5SeekCursor(pCsr, 0); for(i=0; rc==SQLITE_OK && i<pConfig->nCol; i++){ if( pConfig->abUnindexed[i]==0 ){ const char *z = 0; int n = 0; pCsr->aColumnSize[i] = 0; rc = fts5TextFromStmt(pConfig, pCsr->pStmt, i, &z, &n); if( rc==SQLITE_OK ){ rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_AUX, z, n, (void*)&pCsr->aColumnSize[i], fts5ColumnSizeCb ); } sqlite3Fts5ClearLocale(pConfig); } } } CsrFlagClear(pCsr, FTS5CSR_REQUIRE_DOCSIZE); } if( iCol<0 ){ int i; |
︙ | ︙ | |||
2148 2149 2150 2151 2152 2153 2154 | } } return pRet; } static void fts5ApiPhraseNext( | | < > > > > | | 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 | } } return pRet; } static void fts5ApiPhraseNext( Fts5Context *pCtx, Fts5PhraseIter *pIter, int *piCol, int *piOff ){ if( pIter->a>=pIter->b ){ *piCol = -1; *piOff = -1; }else{ int iVal; pIter->a += fts5GetVarint32(pIter->a, iVal); if( iVal==1 ){ /* Avoid returning a (*piCol) value that is too large for the table, ** even if the position-list is corrupt. The caller might not be ** expecting it. */ int nCol = ((Fts5Table*)(((Fts5Cursor*)pCtx)->base.pVtab))->pConfig->nCol; pIter->a += fts5GetVarint32(pIter->a, iVal); *piCol = (iVal>=nCol ? nCol-1 : iVal); *piOff = 0; pIter->a += fts5GetVarint32(pIter->a, iVal); } *piOff += (iVal-2); } } |
︙ | ︙ | |||
2263 2264 2265 2266 2267 2268 2269 | } } } return rc; } | > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > | 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 | } } } return rc; } /* ** xQueryToken() API implemenetation. */ static int fts5ApiQueryToken( Fts5Context* pCtx, int iPhrase, int iToken, const char **ppOut, int *pnOut ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; return sqlite3Fts5ExprQueryToken(pCsr->pExpr, iPhrase, iToken, ppOut, pnOut); } /* ** xInstToken() API implemenetation. */ static int fts5ApiInstToken( Fts5Context *pCtx, int iIdx, int iToken, const char **ppOut, int *pnOut ){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST)==0 || SQLITE_OK==(rc = fts5CacheInstArray(pCsr)) ){ if( iIdx<0 || iIdx>=pCsr->nInstCount ){ rc = SQLITE_RANGE; }else{ int iPhrase = pCsr->aInst[iIdx*3]; int iCol = pCsr->aInst[iIdx*3 + 1]; int iOff = pCsr->aInst[iIdx*3 + 2]; i64 iRowid = fts5CursorRowid(pCsr); rc = sqlite3Fts5ExprInstToken( pCsr->pExpr, iRowid, iPhrase, iCol, iOff, iToken, ppOut, pnOut ); } } return rc; } static int fts5ApiQueryPhrase(Fts5Context*, int, void*, int(*)(const Fts5ExtensionApi*, Fts5Context*, void*) ); /* ** The xColumnLocale() API. */ static int fts5ApiColumnLocale( Fts5Context *pCtx, int iCol, const char **pzLocale, int *pnLocale ){ int rc = SQLITE_OK; Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; Fts5Config *pConfig = ((Fts5Table*)(pCsr->base.pVtab))->pConfig; *pzLocale = 0; *pnLocale = 0; assert( pCsr->ePlan!=FTS5_PLAN_SPECIAL ); if( iCol<0 || iCol>=pConfig->nCol ){ rc = SQLITE_RANGE; }else if( pConfig->abUnindexed[iCol]==0 && 0==fts5IsContentless((Fts5FullTable*)pCsr->base.pVtab, 1) && pConfig->bLocale ){ rc = fts5SeekCursor(pCsr, 0); if( rc==SQLITE_OK ){ const char *zDummy = 0; int nDummy = 0; rc = fts5TextFromStmt(pConfig, pCsr->pStmt, iCol, &zDummy, &nDummy); if( rc==SQLITE_OK ){ *pzLocale = pConfig->t.pLocale; *pnLocale = pConfig->t.nLocale; } sqlite3Fts5ClearLocale(pConfig); } } return rc; } static const Fts5ExtensionApi sFts5Api = { 4, /* iVersion */ fts5ApiUserData, fts5ApiColumnCount, fts5ApiRowCount, fts5ApiColumnTotalSize, fts5ApiTokenize, fts5ApiPhraseCount, fts5ApiPhraseSize, fts5ApiInstCount, fts5ApiInst, fts5ApiRowid, fts5ApiColumnText, fts5ApiColumnSize, fts5ApiQueryPhrase, fts5ApiSetAuxdata, fts5ApiGetAuxdata, fts5ApiPhraseFirst, fts5ApiPhraseNext, fts5ApiPhraseFirstColumn, fts5ApiPhraseNextColumn, fts5ApiQueryToken, fts5ApiInstToken, fts5ApiColumnLocale, fts5ApiTokenize_v2 }; /* ** Implementation of API function xQueryPhrase(). */ static int fts5ApiQueryPhrase( Fts5Context *pCtx, |
︙ | ︙ | |||
2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 | Fts5Auxiliary *pAux, Fts5Cursor *pCsr, sqlite3_context *context, int argc, sqlite3_value **argv ){ assert( pCsr->pAux==0 ); pCsr->pAux = pAux; pAux->xFunc(&sFts5Api, (Fts5Context*)pCsr, context, argc, argv); pCsr->pAux = 0; } static Fts5Cursor *fts5CursorFromCsrid(Fts5Global *pGlobal, i64 iCsrId){ Fts5Cursor *pCsr; for(pCsr=pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){ if( pCsr->iCsrId==iCsrId ) break; } return pCsr; } static void fts5ApiCallback( sqlite3_context *context, int argc, sqlite3_value **argv ){ Fts5Auxiliary *pAux; Fts5Cursor *pCsr; i64 iCsrId; assert( argc>=1 ); pAux = (Fts5Auxiliary*)sqlite3_user_data(context); iCsrId = sqlite3_value_int64(argv[0]); pCsr = fts5CursorFromCsrid(pAux->pGlobal, iCsrId); | > > > > > > > > > > > > > > > > | | < < > > > | 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 | Fts5Auxiliary *pAux, Fts5Cursor *pCsr, sqlite3_context *context, int argc, sqlite3_value **argv ){ assert( pCsr->pAux==0 ); assert( pCsr->ePlan!=FTS5_PLAN_SPECIAL ); pCsr->pAux = pAux; pAux->xFunc(&sFts5Api, (Fts5Context*)pCsr, context, argc, argv); pCsr->pAux = 0; } static Fts5Cursor *fts5CursorFromCsrid(Fts5Global *pGlobal, i64 iCsrId){ Fts5Cursor *pCsr; for(pCsr=pGlobal->pCsr; pCsr; pCsr=pCsr->pNext){ if( pCsr->iCsrId==iCsrId ) break; } return pCsr; } /* ** Parameter zFmt is a printf() style formatting string. This function ** formats it using the trailing arguments and returns the result as ** an error message to the context passed as the first argument. */ static void fts5ResultError(sqlite3_context *pCtx, const char *zFmt, ...){ char *zErr = 0; va_list ap; va_start(ap, zFmt); zErr = sqlite3_vmprintf(zFmt, ap); sqlite3_result_error(pCtx, zErr, -1); sqlite3_free(zErr); va_end(ap); } static void fts5ApiCallback( sqlite3_context *context, int argc, sqlite3_value **argv ){ Fts5Auxiliary *pAux; Fts5Cursor *pCsr; i64 iCsrId; assert( argc>=1 ); pAux = (Fts5Auxiliary*)sqlite3_user_data(context); iCsrId = sqlite3_value_int64(argv[0]); pCsr = fts5CursorFromCsrid(pAux->pGlobal, iCsrId); if( pCsr==0 || (pCsr->ePlan==0 || pCsr->ePlan==FTS5_PLAN_SPECIAL) ){ fts5ResultError(context, "no such cursor: %lld", iCsrId); }else{ sqlite3_vtab *pTab = pCsr->base.pVtab; fts5ApiInvoke(pAux, pCsr, context, argc-1, &argv[1]); sqlite3_free(pTab->zErrMsg); pTab->zErrMsg = 0; } } /* ** Given cursor id iId, return a pointer to the corresponding Fts5Table ** object. Or NULL If the cursor id does not exist. |
︙ | ︙ | |||
2490 2491 2492 2493 2494 2495 2496 | if( iCol==pConfig->nCol ){ /* User is requesting the value of the special column with the same name ** as the table. Return the cursor integer id number. This value is only ** useful in that it may be passed as the first argument to an FTS5 ** auxiliary function. */ sqlite3_result_int64(pCtx, pCsr->iCsrId); }else if( iCol==pConfig->nCol+1 ){ | < > | > | | | > > > > > > > > > > > > > | | > > | | > > | 3019 3020 3021 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 3049 3050 3051 3052 3053 3054 3055 3056 3057 3058 3059 3060 3061 3062 3063 3064 3065 3066 3067 3068 3069 3070 | if( iCol==pConfig->nCol ){ /* User is requesting the value of the special column with the same name ** as the table. Return the cursor integer id number. This value is only ** useful in that it may be passed as the first argument to an FTS5 ** auxiliary function. */ sqlite3_result_int64(pCtx, pCsr->iCsrId); }else if( iCol==pConfig->nCol+1 ){ /* The value of the "rank" column. */ if( pCsr->ePlan==FTS5_PLAN_SOURCE ){ fts5PoslistBlob(pCtx, pCsr); }else if( pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SORTED_MATCH ){ if( pCsr->pRank || SQLITE_OK==(rc = fts5FindRankFunction(pCsr)) ){ fts5ApiInvoke(pCsr->pRank, pCsr, pCtx, pCsr->nRankArg, pCsr->apRankArg); } } }else{ if( !sqlite3_vtab_nochange(pCtx) && pConfig->eContent!=FTS5_CONTENT_NONE ){ pConfig->pzErrmsg = &pTab->p.base.zErrMsg; rc = fts5SeekCursor(pCsr, 1); if( rc==SQLITE_OK ){ sqlite3_value *pVal = sqlite3_column_value(pCsr->pStmt, iCol+1); if( pConfig->bLocale && pConfig->eContent==FTS5_CONTENT_EXTERNAL && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ const char *z = 0; int n = 0; rc = fts5TextFromStmt(pConfig, pCsr->pStmt, iCol, &z, &n); if( rc==SQLITE_OK ){ sqlite3_result_text(pCtx, z, n, SQLITE_TRANSIENT); } sqlite3Fts5ClearLocale(pConfig); }else{ sqlite3_result_value(pCtx, pVal); } } pConfig->pzErrmsg = 0; } } return rc; } /* ** This routine implements the xFindFunction method for the FTS3 ** virtual table. |
︙ | ︙ | |||
2547 2548 2549 2550 2551 2552 2553 2554 | /* ** Implementation of FTS5 xRename method. Rename an fts5 table. */ static int fts5RenameMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ const char *zName /* New name of table */ ){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; | > | > > > | | | > > > > | > | > | > > > > > | > > | > > | 3094 3095 3096 3097 3098 3099 3100 3101 3102 3103 3104 3105 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 3144 3145 3146 3147 3148 3149 3150 3151 3152 3153 3154 3155 3156 3157 3158 3159 3160 3161 3162 3163 3164 3165 3166 3167 3168 | /* ** Implementation of FTS5 xRename method. Rename an fts5 table. */ static int fts5RenameMethod( sqlite3_vtab *pVtab, /* Virtual table handle */ const char *zName /* New name of table */ ){ int rc; Fts5FullTable *pTab = (Fts5FullTable*)pVtab; rc = sqlite3Fts5StorageRename(pTab->pStorage, zName); return rc; } int sqlite3Fts5FlushToDisk(Fts5Table *pTab){ fts5TripCursors((Fts5FullTable*)pTab); return sqlite3Fts5StorageSync(((Fts5FullTable*)pTab)->pStorage); } /* ** The xSavepoint() method. ** ** Flush the contents of the pending-terms table to disk. */ static int fts5SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc = SQLITE_OK; fts5CheckTransactionState(pTab, FTS5_SAVEPOINT, iSavepoint); rc = sqlite3Fts5FlushToDisk((Fts5Table*)pVtab); if( rc==SQLITE_OK ){ pTab->iSavepoint = iSavepoint+1; } return rc; } /* ** The xRelease() method. ** ** This is a no-op. */ static int fts5ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc = SQLITE_OK; fts5CheckTransactionState(pTab, FTS5_RELEASE, iSavepoint); if( (iSavepoint+1)<pTab->iSavepoint ){ rc = sqlite3Fts5FlushToDisk(&pTab->p); if( rc==SQLITE_OK ){ pTab->iSavepoint = iSavepoint; } } return rc; } /* ** The xRollbackTo() method. ** ** Discard the contents of the pending terms table. */ static int fts5RollbackToMethod(sqlite3_vtab *pVtab, int iSavepoint){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc = SQLITE_OK; fts5CheckTransactionState(pTab, FTS5_ROLLBACKTO, iSavepoint); fts5TripCursors(pTab); if( (iSavepoint+1)<=pTab->iSavepoint ){ pTab->p.pConfig->pgsz = 0; rc = sqlite3Fts5StorageRollback(pTab->pStorage); } return rc; } /* ** Register a new auxiliary function with global context pGlobal. */ static int fts5CreateAux( fts5_api *pApi, /* Global context (one per db handle) */ |
︙ | ︙ | |||
2628 2629 2630 2631 2632 2633 2634 2635 2636 2637 | }else{ rc = SQLITE_NOMEM; } } return rc; } /* ** Register a new tokenizer. This is the implementation of the | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < | < < < | | > < < < < | | | < | < | < < < < > > > > > | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | > > > > | > > > > | < | | < < < | | | > > | > > > > > > > | | > | > | | | | | > > > > > > > > > > > | 3194 3195 3196 3197 3198 3199 3200 3201 3202 3203 3204 3205 3206 3207 3208 3209 3210 3211 3212 3213 3214 3215 3216 3217 3218 3219 3220 3221 3222 3223 3224 3225 3226 3227 3228 3229 3230 3231 3232 3233 3234 3235 3236 3237 3238 3239 3240 3241 3242 3243 3244 3245 3246 3247 3248 3249 3250 3251 3252 3253 3254 3255 3256 3257 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 3276 3277 3278 3279 3280 3281 3282 3283 3284 3285 3286 3287 3288 3289 3290 3291 3292 3293 3294 3295 3296 3297 3298 3299 3300 3301 3302 3303 3304 3305 3306 3307 3308 3309 3310 3311 3312 3313 3314 3315 3316 3317 3318 3319 3320 3321 3322 3323 3324 3325 3326 3327 3328 3329 3330 3331 3332 3333 3334 3335 3336 3337 3338 3339 3340 3341 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 3357 3358 3359 3360 3361 3362 3363 3364 3365 3366 3367 3368 3369 3370 3371 3372 3373 3374 3375 3376 3377 3378 3379 3380 3381 3382 3383 3384 3385 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 3408 3409 3410 3411 3412 3413 3414 3415 3416 3417 3418 3419 3420 3421 3422 3423 3424 3425 3426 3427 3428 3429 3430 3431 3432 3433 3434 3435 3436 3437 3438 3439 3440 3441 3442 3443 3444 3445 3446 3447 3448 3449 3450 3451 3452 3453 3454 3455 3456 3457 3458 3459 3460 3461 3462 3463 3464 3465 3466 3467 3468 3469 3470 3471 3472 3473 3474 3475 3476 3477 3478 3479 3480 3481 3482 3483 3484 3485 3486 3487 3488 3489 3490 3491 3492 3493 3494 3495 3496 3497 3498 3499 3500 3501 3502 3503 3504 3505 3506 3507 3508 3509 3510 3511 3512 3513 3514 3515 3516 3517 3518 3519 3520 3521 3522 3523 3524 3525 3526 3527 3528 3529 3530 3531 3532 3533 3534 3535 3536 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 3552 3553 3554 3555 3556 3557 3558 3559 3560 3561 3562 3563 3564 3565 | }else{ rc = SQLITE_NOMEM; } } return rc; } /* ** This function is used by xCreateTokenizer_v2() and xCreateTokenizer(). ** It allocates and partially populates a new Fts5TokenizerModule object. ** The new object is already linked into the Fts5Global context before ** returning. ** ** If successful, SQLITE_OK is returned and a pointer to the new ** Fts5TokenizerModule object returned via output parameter (*ppNew). All ** that is required is for the caller to fill in the methods in ** Fts5TokenizerModule.x1 and x2, and to set Fts5TokenizerModule.bV2Native ** as appropriate. ** ** If an error occurs, an SQLite error code is returned and the final value ** of (*ppNew) undefined. */ static int fts5NewTokenizerModule( Fts5Global *pGlobal, /* Global context (one per db handle) */ const char *zName, /* Name of new function */ void *pUserData, /* User data for aux. function */ void(*xDestroy)(void*), /* Destructor for pUserData */ Fts5TokenizerModule **ppNew ){ int rc = SQLITE_OK; Fts5TokenizerModule *pNew; sqlite3_int64 nName; /* Size of zName and its \0 terminator */ sqlite3_int64 nByte; /* Bytes of space to allocate */ nName = strlen(zName) + 1; nByte = sizeof(Fts5TokenizerModule) + nName; *ppNew = pNew = (Fts5TokenizerModule*)sqlite3Fts5MallocZero(&rc, nByte); if( pNew ){ pNew->zName = (char*)&pNew[1]; memcpy(pNew->zName, zName, nName); pNew->pUserData = pUserData; pNew->xDestroy = xDestroy; pNew->pNext = pGlobal->pTok; pGlobal->pTok = pNew; if( pNew->pNext==0 ){ pGlobal->pDfltTok = pNew; } } return rc; } /* ** An instance of this type is used as the Fts5Tokenizer object for ** wrapper tokenizers - those that provide access to a v1 tokenizer via ** the fts5_tokenizer_v2 API, and those that provide access to a v2 tokenizer ** via the fts5_tokenizer API. */ typedef struct Fts5VtoVTokenizer Fts5VtoVTokenizer; struct Fts5VtoVTokenizer { int bV2Native; /* True if v2 native tokenizer */ fts5_tokenizer x1; /* Tokenizer functions */ fts5_tokenizer_v2 x2; /* V2 tokenizer functions */ Fts5Tokenizer *pReal; }; /* ** Create a wrapper tokenizer. The context argument pCtx points to the ** Fts5TokenizerModule object. */ static int fts5VtoVCreate( void *pCtx, const char **azArg, int nArg, Fts5Tokenizer **ppOut ){ Fts5TokenizerModule *pMod = (Fts5TokenizerModule*)pCtx; Fts5VtoVTokenizer *pNew = 0; int rc = SQLITE_OK; pNew = (Fts5VtoVTokenizer*)sqlite3Fts5MallocZero(&rc, sizeof(*pNew)); if( rc==SQLITE_OK ){ pNew->x1 = pMod->x1; pNew->x2 = pMod->x2; pNew->bV2Native = pMod->bV2Native; if( pMod->bV2Native ){ rc = pMod->x2.xCreate(pMod->pUserData, azArg, nArg, &pNew->pReal); }else{ rc = pMod->x1.xCreate(pMod->pUserData, azArg, nArg, &pNew->pReal); } if( rc!=SQLITE_OK ){ sqlite3_free(pNew); pNew = 0; } } *ppOut = (Fts5Tokenizer*)pNew; return rc; } /* ** Delete an Fts5VtoVTokenizer wrapper tokenizer. */ static void fts5VtoVDelete(Fts5Tokenizer *pTok){ Fts5VtoVTokenizer *p = (Fts5VtoVTokenizer*)pTok; if( p ){ if( p->bV2Native ){ p->x2.xDelete(p->pReal); }else{ p->x1.xDelete(p->pReal); } sqlite3_free(p); } } /* ** xTokenizer method for a wrapper tokenizer that offers the v1 interface ** (no support for locales). */ static int fts5V1toV2Tokenize( Fts5Tokenizer *pTok, void *pCtx, int flags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ Fts5VtoVTokenizer *p = (Fts5VtoVTokenizer*)pTok; assert( p->bV2Native ); return p->x2.xTokenize(p->pReal, pCtx, flags, pText, nText, 0, 0, xToken); } /* ** xTokenizer method for a wrapper tokenizer that offers the v2 interface ** (with locale support). */ static int fts5V2toV1Tokenize( Fts5Tokenizer *pTok, void *pCtx, int flags, const char *pText, int nText, const char *pLocale, int nLocale, int (*xToken)(void*, int, const char*, int, int, int) ){ Fts5VtoVTokenizer *p = (Fts5VtoVTokenizer*)pTok; assert( p->bV2Native==0 ); UNUSED_PARAM2(pLocale,nLocale); return p->x1.xTokenize(p->pReal, pCtx, flags, pText, nText, xToken); } /* ** Register a new tokenizer. This is the implementation of the ** fts5_api.xCreateTokenizer_v2() method. */ static int fts5CreateTokenizer_v2( fts5_api *pApi, /* Global context (one per db handle) */ const char *zName, /* Name of new function */ void *pUserData, /* User data for aux. function */ fts5_tokenizer_v2 *pTokenizer, /* Tokenizer implementation */ void(*xDestroy)(void*) /* Destructor for pUserData */ ){ Fts5Global *pGlobal = (Fts5Global*)pApi; int rc = SQLITE_OK; if( pTokenizer->iVersion>2 ){ rc = SQLITE_ERROR; }else{ Fts5TokenizerModule *pNew = 0; rc = fts5NewTokenizerModule(pGlobal, zName, pUserData, xDestroy, &pNew); if( pNew ){ pNew->x2 = *pTokenizer; pNew->bV2Native = 1; pNew->x1.xCreate = fts5VtoVCreate; pNew->x1.xTokenize = fts5V1toV2Tokenize; pNew->x1.xDelete = fts5VtoVDelete; } } return rc; } /* ** The fts5_api.xCreateTokenizer() method. */ static int fts5CreateTokenizer( fts5_api *pApi, /* Global context (one per db handle) */ const char *zName, /* Name of new function */ void *pUserData, /* User data for aux. function */ fts5_tokenizer *pTokenizer, /* Tokenizer implementation */ void(*xDestroy)(void*) /* Destructor for pUserData */ ){ Fts5TokenizerModule *pNew = 0; int rc = SQLITE_OK; rc = fts5NewTokenizerModule( (Fts5Global*)pApi, zName, pUserData, xDestroy, &pNew ); if( pNew ){ pNew->x1 = *pTokenizer; pNew->x2.xCreate = fts5VtoVCreate; pNew->x2.xTokenize = fts5V2toV1Tokenize; pNew->x2.xDelete = fts5VtoVDelete; } return rc; } /* ** Search the global context passed as the first argument for a tokenizer ** module named zName. If found, return a pointer to the Fts5TokenizerModule ** object. Otherwise, return NULL. */ static Fts5TokenizerModule *fts5LocateTokenizer( Fts5Global *pGlobal, /* Global (one per db handle) object */ const char *zName /* Name of tokenizer module to find */ ){ Fts5TokenizerModule *pMod = 0; if( zName==0 ){ pMod = pGlobal->pDfltTok; }else{ for(pMod=pGlobal->pTok; pMod; pMod=pMod->pNext){ if( sqlite3_stricmp(zName, pMod->zName)==0 ) break; } } return pMod; } /* ** Find a tokenizer. This is the implementation of the ** fts5_api.xFindTokenizer_v2() method. */ static int fts5FindTokenizer_v2( fts5_api *pApi, /* Global context (one per db handle) */ const char *zName, /* Name of tokenizer */ void **ppUserData, fts5_tokenizer_v2 **ppTokenizer /* Populate this object */ ){ int rc = SQLITE_OK; Fts5TokenizerModule *pMod; pMod = fts5LocateTokenizer((Fts5Global*)pApi, zName); if( pMod ){ if( pMod->bV2Native ){ *ppUserData = pMod->pUserData; }else{ *ppUserData = (void*)pMod; } *ppTokenizer = &pMod->x2; }else{ *ppTokenizer = 0; *ppUserData = 0; rc = SQLITE_ERROR; } return rc; } /* ** Find a tokenizer. This is the implementation of the ** fts5_api.xFindTokenizer() method. */ static int fts5FindTokenizer( fts5_api *pApi, /* Global context (one per db handle) */ const char *zName, /* Name of new function */ void **ppUserData, fts5_tokenizer *pTokenizer /* Populate this object */ ){ int rc = SQLITE_OK; Fts5TokenizerModule *pMod; pMod = fts5LocateTokenizer((Fts5Global*)pApi, zName); if( pMod ){ if( pMod->bV2Native==0 ){ *ppUserData = pMod->pUserData; }else{ *ppUserData = (void*)pMod; } *pTokenizer = pMod->x1; }else{ memset(pTokenizer, 0, sizeof(*pTokenizer)); *ppUserData = 0; rc = SQLITE_ERROR; } return rc; } /* ** Attempt to instantiate the tokenizer. */ int sqlite3Fts5LoadTokenizer(Fts5Config *pConfig){ const char **azArg = pConfig->t.azArg; const int nArg = pConfig->t.nArg; Fts5TokenizerModule *pMod = 0; int rc = SQLITE_OK; pMod = fts5LocateTokenizer(pConfig->pGlobal, nArg==0 ? 0 : azArg[0]); if( pMod==0 ){ assert( nArg>0 ); rc = SQLITE_ERROR; sqlite3Fts5ConfigErrmsg(pConfig, "no such tokenizer: %s", azArg[0]); }else{ int (*xCreate)(void*, const char**, int, Fts5Tokenizer**) = 0; if( pMod->bV2Native ){ xCreate = pMod->x2.xCreate; pConfig->t.pApi2 = &pMod->x2; }else{ pConfig->t.pApi1 = &pMod->x1; xCreate = pMod->x1.xCreate; } rc = xCreate(pMod->pUserData, (azArg?&azArg[1]:0), (nArg?nArg-1:0), &pConfig->t.pTok ); if( rc!=SQLITE_OK ){ if( rc!=SQLITE_NOMEM ){ sqlite3Fts5ConfigErrmsg(pConfig, "error in tokenizer constructor"); } }else if( pMod->bV2Native==0 ){ pConfig->t.ePattern = sqlite3Fts5TokenizerPattern( pMod->x1.xCreate, pConfig->t.pTok ); } } if( rc!=SQLITE_OK ){ pConfig->t.pApi1 = 0; pConfig->t.pApi2 = 0; pConfig->t.pTok = 0; } return rc; } /* ** xDestroy callback passed to sqlite3_create_module(). This is invoked ** when the db handle is being closed. Free memory associated with ** tokenizers and aux functions registered with this db handle. */ static void fts5ModuleDestroy(void *pCtx){ Fts5TokenizerModule *pTok, *pNextTok; Fts5Auxiliary *pAux, *pNextAux; Fts5Global *pGlobal = (Fts5Global*)pCtx; for(pAux=pGlobal->pAux; pAux; pAux=pNextAux){ pNextAux = pAux->pNext; if( pAux->xDestroy ) pAux->xDestroy(pAux->pUserData); sqlite3_free(pAux); } for(pTok=pGlobal->pTok; pTok; pTok=pNextTok){ pNextTok = pTok->pNext; if( pTok->xDestroy ) pTok->xDestroy(pTok->pUserData); sqlite3_free(pTok); } sqlite3_free(pGlobal); } /* ** Implementation of the fts5() function used by clients to obtain the ** API pointer. */ static void fts5Fts5Func( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apArg /* Function arguments */ ){ Fts5Global *pGlobal = (Fts5Global*)sqlite3_user_data(pCtx); fts5_api **ppApi; |
︙ | ︙ | |||
2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 | int nArg, /* Number of args */ sqlite3_value **apUnused /* Function arguments */ ){ assert( nArg==0 ); UNUSED_PARAM2(nArg, apUnused); sqlite3_result_text(pCtx, "--FTS5-SOURCE-ID--", -1, SQLITE_TRANSIENT); } /* ** Return true if zName is the extension on one of the shadow tables used ** by this module. */ static int fts5ShadowName(const char *zName){ static const char *azName[] = { "config", "content", "data", "docsize", "idx" }; unsigned int i; for(i=0; i<sizeof(azName)/sizeof(azName[0]); i++){ if( sqlite3_stricmp(zName, azName[i])==0 ) return 1; } return 0; } static int fts5Init(sqlite3 *db){ static const sqlite3_module fts5Mod = { | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | 3577 3578 3579 3580 3581 3582 3583 3584 3585 3586 3587 3588 3589 3590 3591 3592 3593 3594 3595 3596 3597 3598 3599 3600 3601 3602 3603 3604 3605 3606 3607 3608 3609 3610 3611 3612 3613 3614 3615 3616 3617 3618 3619 3620 3621 3622 3623 3624 3625 3626 3627 3628 3629 3630 3631 3632 3633 3634 3635 3636 3637 3638 3639 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654 3655 3656 3657 3658 3659 3660 3661 3662 3663 3664 3665 3666 3667 3668 3669 3670 3671 3672 3673 3674 3675 3676 3677 3678 3679 3680 3681 3682 3683 3684 3685 3686 3687 3688 3689 3690 3691 3692 3693 3694 3695 3696 3697 3698 3699 3700 3701 3702 3703 3704 3705 3706 3707 3708 3709 3710 3711 3712 3713 3714 3715 3716 3717 3718 3719 3720 3721 3722 3723 | int nArg, /* Number of args */ sqlite3_value **apUnused /* Function arguments */ ){ assert( nArg==0 ); UNUSED_PARAM2(nArg, apUnused); sqlite3_result_text(pCtx, "--FTS5-SOURCE-ID--", -1, SQLITE_TRANSIENT); } /* ** Implementation of fts5_locale(LOCALE, TEXT) function. ** ** If parameter LOCALE is NULL, or a zero-length string, then a copy of ** TEXT is returned. Otherwise, both LOCALE and TEXT are interpreted as ** text, and the value returned is a blob consisting of: ** ** * The 4 bytes 0x00, 0xE0, 0xB2, 0xEb (FTS5_LOCALE_HEADER). ** * The LOCALE, as utf-8 text, followed by ** * 0x00, followed by ** * The TEXT, as utf-8 text. ** ** There is no final nul-terminator following the TEXT value. */ static void fts5LocaleFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apArg /* Function arguments */ ){ const char *zLocale = 0; int nLocale = 0; const char *zText = 0; int nText = 0; assert( nArg==2 ); UNUSED_PARAM(nArg); zLocale = (const char*)sqlite3_value_text(apArg[0]); nLocale = sqlite3_value_bytes(apArg[0]); zText = (const char*)sqlite3_value_text(apArg[1]); nText = sqlite3_value_bytes(apArg[1]); if( zLocale==0 || zLocale[0]=='\0' ){ sqlite3_result_text(pCtx, zText, nText, SQLITE_TRANSIENT); }else{ Fts5Global *p = (Fts5Global*)sqlite3_user_data(pCtx); u8 *pBlob = 0; u8 *pCsr = 0; int nBlob = 0; nBlob = FTS5_LOCALE_HDR_SIZE + nLocale + 1 + nText; pBlob = (u8*)sqlite3_malloc(nBlob); if( pBlob==0 ){ sqlite3_result_error_nomem(pCtx); return; } pCsr = pBlob; memcpy(pCsr, (const u8*)p->aLocaleHdr, FTS5_LOCALE_HDR_SIZE); pCsr += FTS5_LOCALE_HDR_SIZE; memcpy(pCsr, zLocale, nLocale); pCsr += nLocale; (*pCsr++) = 0x00; if( zText ) memcpy(pCsr, zText, nText); assert( &pCsr[nText]==&pBlob[nBlob] ); sqlite3_result_blob(pCtx, pBlob, nBlob, sqlite3_free); } } /* ** Implementation of fts5_insttoken() function. */ static void fts5InsttokenFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apArg /* Function arguments */ ){ assert( nArg==1 ); (void)nArg; sqlite3_result_value(pCtx, apArg[0]); sqlite3_result_subtype(pCtx, FTS5_INSTTOKEN_SUBTYPE); } /* ** Return true if zName is the extension on one of the shadow tables used ** by this module. */ static int fts5ShadowName(const char *zName){ static const char *azName[] = { "config", "content", "data", "docsize", "idx" }; unsigned int i; for(i=0; i<sizeof(azName)/sizeof(azName[0]); i++){ if( sqlite3_stricmp(zName, azName[i])==0 ) return 1; } return 0; } /* ** Run an integrity check on the FTS5 data structures. Return a string ** if anything is found amiss. Return a NULL pointer if everything is ** OK. */ static int fts5IntegrityMethod( sqlite3_vtab *pVtab, /* the FTS5 virtual table to check */ const char *zSchema, /* Name of schema in which this table lives */ const char *zTabname, /* Name of the table itself */ int isQuick, /* True if this is a quick-check */ char **pzErr /* Write error message here */ ){ Fts5FullTable *pTab = (Fts5FullTable*)pVtab; int rc; assert( pzErr!=0 && *pzErr==0 ); UNUSED_PARAM(isQuick); assert( pTab->p.pConfig->pzErrmsg==0 ); pTab->p.pConfig->pzErrmsg = pzErr; rc = sqlite3Fts5StorageIntegrity(pTab->pStorage, 0); if( *pzErr==0 && rc!=SQLITE_OK ){ if( (rc&0xff)==SQLITE_CORRUPT ){ *pzErr = sqlite3_mprintf("malformed inverted index for FTS5 table %s.%s", zSchema, zTabname); rc = (*pzErr) ? SQLITE_OK : SQLITE_NOMEM; }else{ *pzErr = sqlite3_mprintf("unable to validate the inverted index for" " FTS5 table %s.%s: %s", zSchema, zTabname, sqlite3_errstr(rc)); } }else if( (rc&0xff)==SQLITE_CORRUPT ){ rc = SQLITE_OK; } sqlite3Fts5IndexCloseReader(pTab->p.pIndex); pTab->p.pConfig->pzErrmsg = 0; return rc; } static int fts5Init(sqlite3 *db){ static const sqlite3_module fts5Mod = { /* iVersion */ 4, /* xCreate */ fts5CreateMethod, /* xConnect */ fts5ConnectMethod, /* xBestIndex */ fts5BestIndexMethod, /* xDisconnect */ fts5DisconnectMethod, /* xDestroy */ fts5DestroyMethod, /* xOpen */ fts5OpenMethod, /* xClose */ fts5CloseMethod, |
︙ | ︙ | |||
2833 2834 2835 2836 2837 2838 2839 | /* xCommit */ fts5CommitMethod, /* xRollback */ fts5RollbackMethod, /* xFindFunction */ fts5FindFunctionMethod, /* xRename */ fts5RenameMethod, /* xSavepoint */ fts5SavepointMethod, /* xRelease */ fts5ReleaseMethod, /* xRollbackTo */ fts5RollbackToMethod, | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > | | | 3732 3733 3734 3735 3736 3737 3738 3739 3740 3741 3742 3743 3744 3745 3746 3747 3748 3749 3750 3751 3752 3753 3754 3755 3756 3757 3758 3759 3760 3761 3762 3763 3764 3765 3766 3767 3768 3769 3770 3771 3772 3773 3774 3775 3776 3777 3778 3779 3780 3781 3782 3783 3784 3785 3786 3787 3788 3789 3790 3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 3806 3807 3808 3809 3810 3811 3812 3813 3814 3815 3816 | /* xCommit */ fts5CommitMethod, /* xRollback */ fts5RollbackMethod, /* xFindFunction */ fts5FindFunctionMethod, /* xRename */ fts5RenameMethod, /* xSavepoint */ fts5SavepointMethod, /* xRelease */ fts5ReleaseMethod, /* xRollbackTo */ fts5RollbackToMethod, /* xShadowName */ fts5ShadowName, /* xIntegrity */ fts5IntegrityMethod }; int rc; Fts5Global *pGlobal = 0; pGlobal = (Fts5Global*)sqlite3_malloc(sizeof(Fts5Global)); if( pGlobal==0 ){ rc = SQLITE_NOMEM; }else{ void *p = (void*)pGlobal; memset(pGlobal, 0, sizeof(Fts5Global)); pGlobal->db = db; pGlobal->api.iVersion = 3; pGlobal->api.xCreateFunction = fts5CreateAux; pGlobal->api.xCreateTokenizer = fts5CreateTokenizer; pGlobal->api.xFindTokenizer = fts5FindTokenizer; pGlobal->api.xCreateTokenizer_v2 = fts5CreateTokenizer_v2; pGlobal->api.xFindTokenizer_v2 = fts5FindTokenizer_v2; /* Initialize pGlobal->aLocaleHdr[] to a 128-bit pseudo-random vector. ** The constants below were generated randomly. */ sqlite3_randomness(sizeof(pGlobal->aLocaleHdr), pGlobal->aLocaleHdr); pGlobal->aLocaleHdr[0] ^= 0xF924976D; pGlobal->aLocaleHdr[1] ^= 0x16596E13; pGlobal->aLocaleHdr[2] ^= 0x7C80BEAA; pGlobal->aLocaleHdr[3] ^= 0x9B03A67F; assert( sizeof(pGlobal->aLocaleHdr)==16 ); rc = sqlite3_create_module_v2(db, "fts5", &fts5Mod, p, fts5ModuleDestroy); if( rc==SQLITE_OK ) rc = sqlite3Fts5IndexInit(db); if( rc==SQLITE_OK ) rc = sqlite3Fts5ExprInit(pGlobal, db); if( rc==SQLITE_OK ) rc = sqlite3Fts5AuxInit(&pGlobal->api); if( rc==SQLITE_OK ) rc = sqlite3Fts5TokenizerInit(&pGlobal->api); if( rc==SQLITE_OK ) rc = sqlite3Fts5VocabInit(pGlobal, db); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5", 1, SQLITE_UTF8, p, fts5Fts5Func, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_source_id", 0, SQLITE_UTF8|SQLITE_DETERMINISTIC|SQLITE_INNOCUOUS, p, fts5SourceIdFunc, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_locale", 2, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_RESULT_SUBTYPE|SQLITE_SUBTYPE, p, fts5LocaleFunc, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_insttoken", 1, SQLITE_UTF8|SQLITE_INNOCUOUS|SQLITE_RESULT_SUBTYPE, p, fts5InsttokenFunc, 0, 0 ); } } /* If SQLITE_FTS5_ENABLE_TEST_MI is defined, assume that the file ** fts5_test_mi.c is compiled and linked into the executable. And call ** its entry point to enable the matchinfo() demo. */ #ifdef SQLITE_FTS5_ENABLE_TEST_MI if( rc==SQLITE_OK ){ extern int sqlite3Fts5TestRegisterMatchinfoAPI(fts5_api*); rc = sqlite3Fts5TestRegisterMatchinfoAPI(&pGlobal->api); } #endif return rc; } /* |
︙ | ︙ |
Changes to ext/fts5/fts5_storage.c.
︙ | ︙ | |||
12 13 14 15 16 17 18 19 20 21 22 23 24 | ** */ #include "fts5Int.h" struct Fts5Storage { Fts5Config *pConfig; Fts5Index *pIndex; int bTotalsValid; /* True if nTotalRow/aTotalSize[] are valid */ i64 nTotalRow; /* Total number of rows in FTS table */ i64 *aTotalSize; /* Total sizes of each column */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | | | | | | | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 | ** */ #include "fts5Int.h" /* ** pSavedRow: ** SQL statement FTS5_STMT_LOOKUP2 is a copy of FTS5_STMT_LOOKUP, it ** does a by-rowid lookup to retrieve a single row from the %_content ** table or equivalent external-content table/view. ** ** However, FTS5_STMT_LOOKUP2 is only used when retrieving the original ** values for a row being UPDATEd. In that case, the SQL statement is ** not reset and pSavedRow is set to point at it. This is so that the ** insert operation that follows the delete may access the original ** row values for any new values for which sqlite3_value_nochange() returns ** true. i.e. if the user executes: ** ** CREATE VIRTUAL TABLE ft USING fts5(a, b, c, locale=1); ** ... ** UPDATE fts SET a=?, b=? WHERE rowid=?; ** ** then the value passed to the xUpdate() method of this table as the ** new.c value is an sqlite3_value_nochange() value. So in this case it ** must be read from the saved row stored in Fts5Storage.pSavedRow. ** ** This is necessary - using sqlite3_value_nochange() instead of just having ** SQLite pass the original value back via xUpdate() - so as not to discard ** any locale information associated with such values. ** */ struct Fts5Storage { Fts5Config *pConfig; Fts5Index *pIndex; int bTotalsValid; /* True if nTotalRow/aTotalSize[] are valid */ i64 nTotalRow; /* Total number of rows in FTS table */ i64 *aTotalSize; /* Total sizes of each column */ sqlite3_stmt *pSavedRow; sqlite3_stmt *aStmt[12]; }; #if FTS5_STMT_SCAN_ASC!=0 # error "FTS5_STMT_SCAN_ASC mismatch" #endif #if FTS5_STMT_SCAN_DESC!=1 # error "FTS5_STMT_SCAN_DESC mismatch" #endif #if FTS5_STMT_LOOKUP!=2 # error "FTS5_STMT_LOOKUP mismatch" #endif #define FTS5_STMT_LOOKUP2 3 #define FTS5_STMT_INSERT_CONTENT 4 #define FTS5_STMT_REPLACE_CONTENT 5 #define FTS5_STMT_DELETE_CONTENT 6 #define FTS5_STMT_REPLACE_DOCSIZE 7 #define FTS5_STMT_DELETE_DOCSIZE 8 #define FTS5_STMT_LOOKUP_DOCSIZE 9 #define FTS5_STMT_REPLACE_CONFIG 10 #define FTS5_STMT_SCAN 11 /* ** Prepare the two insert statements - Fts5Storage.pInsertContent and ** Fts5Storage.pInsertDocsize - if they have not already been prepared. ** Return SQLITE_OK if successful, or an SQLite error code if an error ** occurs. */ |
︙ | ︙ | |||
69 70 71 72 73 74 75 76 77 78 79 | assert( eStmt>=0 && eStmt<ArraySize(p->aStmt) ); if( p->aStmt[eStmt]==0 ){ const char *azStmt[] = { "SELECT %s FROM %s T WHERE T.%Q >= ? AND T.%Q <= ? ORDER BY T.%Q ASC", "SELECT %s FROM %s T WHERE T.%Q <= ? AND T.%Q >= ? ORDER BY T.%Q DESC", "SELECT %s FROM %s T WHERE T.%Q=?", /* LOOKUP */ "INSERT INTO %Q.'%q_content' VALUES(%s)", /* INSERT_CONTENT */ "REPLACE INTO %Q.'%q_content' VALUES(%s)", /* REPLACE_CONTENT */ "DELETE FROM %Q.'%q_content' WHERE id=?", /* DELETE_CONTENT */ | > | | > > > < | > > > | > > > > > > > > > | > > > | | | | > > | | | > | > > > > > | > > > > > > | > > > > > | 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 | assert( eStmt>=0 && eStmt<ArraySize(p->aStmt) ); if( p->aStmt[eStmt]==0 ){ const char *azStmt[] = { "SELECT %s FROM %s T WHERE T.%Q >= ? AND T.%Q <= ? ORDER BY T.%Q ASC", "SELECT %s FROM %s T WHERE T.%Q <= ? AND T.%Q >= ? ORDER BY T.%Q DESC", "SELECT %s FROM %s T WHERE T.%Q=?", /* LOOKUP */ "SELECT %s FROM %s T WHERE T.%Q=?", /* LOOKUP2 */ "INSERT INTO %Q.'%q_content' VALUES(%s)", /* INSERT_CONTENT */ "REPLACE INTO %Q.'%q_content' VALUES(%s)", /* REPLACE_CONTENT */ "DELETE FROM %Q.'%q_content' WHERE id=?", /* DELETE_CONTENT */ "REPLACE INTO %Q.'%q_docsize' VALUES(?,?%s)", /* REPLACE_DOCSIZE */ "DELETE FROM %Q.'%q_docsize' WHERE id=?", /* DELETE_DOCSIZE */ "SELECT sz%s FROM %Q.'%q_docsize' WHERE id=?", /* LOOKUP_DOCSIZE */ "REPLACE INTO %Q.'%q_config' VALUES(?,?)", /* REPLACE_CONFIG */ "SELECT %s FROM %s AS T", /* SCAN */ }; Fts5Config *pC = p->pConfig; char *zSql = 0; assert( ArraySize(azStmt)==ArraySize(p->aStmt) ); switch( eStmt ){ case FTS5_STMT_SCAN: zSql = sqlite3_mprintf(azStmt[eStmt], pC->zContentExprlist, pC->zContent ); break; case FTS5_STMT_SCAN_ASC: case FTS5_STMT_SCAN_DESC: zSql = sqlite3_mprintf(azStmt[eStmt], pC->zContentExprlist, pC->zContent, pC->zContentRowid, pC->zContentRowid, pC->zContentRowid ); break; case FTS5_STMT_LOOKUP: case FTS5_STMT_LOOKUP2: zSql = sqlite3_mprintf(azStmt[eStmt], pC->zContentExprlist, pC->zContent, pC->zContentRowid ); break; case FTS5_STMT_INSERT_CONTENT: case FTS5_STMT_REPLACE_CONTENT: { char *zBind = 0; int i; assert( pC->eContent==FTS5_CONTENT_NORMAL || pC->eContent==FTS5_CONTENT_UNINDEXED ); /* Add bindings for the "c*" columns - those that store the actual ** table content. If eContent==NORMAL, then there is one binding ** for each column. Or, if eContent==UNINDEXED, then there are only ** bindings for the UNINDEXED columns. */ for(i=0; rc==SQLITE_OK && i<(pC->nCol+1); i++){ if( !i || pC->eContent==FTS5_CONTENT_NORMAL || pC->abUnindexed[i-1] ){ zBind = sqlite3Fts5Mprintf(&rc, "%z%s?%d", zBind, zBind?",":"",i+1); } } /* Add bindings for any "l*" columns. Only non-UNINDEXED columns ** require these. */ if( pC->bLocale && pC->eContent==FTS5_CONTENT_NORMAL ){ for(i=0; rc==SQLITE_OK && i<pC->nCol; i++){ if( pC->abUnindexed[i]==0 ){ zBind = sqlite3Fts5Mprintf(&rc, "%z,?%d", zBind, pC->nCol+i+2); } } } zSql = sqlite3Fts5Mprintf(&rc, azStmt[eStmt], pC->zDb, pC->zName,zBind); sqlite3_free(zBind); break; } case FTS5_STMT_REPLACE_DOCSIZE: zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName, (pC->bContentlessDelete ? ",?" : "") ); break; case FTS5_STMT_LOOKUP_DOCSIZE: zSql = sqlite3_mprintf(azStmt[eStmt], (pC->bContentlessDelete ? ",origin" : ""), pC->zDb, pC->zName ); break; default: zSql = sqlite3_mprintf(azStmt[eStmt], pC->zDb, pC->zName); break; } if( zSql==0 ){ rc = SQLITE_NOMEM; }else{ int f = SQLITE_PREPARE_PERSISTENT; if( eStmt>FTS5_STMT_LOOKUP2 ) f |= SQLITE_PREPARE_NO_VTAB; p->pConfig->bLock++; rc = sqlite3_prepare_v3(pC->db, zSql, -1, f, &p->aStmt[eStmt], 0); p->pConfig->bLock--; sqlite3_free(zSql); if( rc!=SQLITE_OK && pzErrMsg ){ *pzErrMsg = sqlite3_mprintf("%s", sqlite3_errmsg(pC->db)); } if( rc==SQLITE_ERROR && eStmt>FTS5_STMT_LOOKUP2 && eStmt<FTS5_STMT_SCAN ){ /* One of the internal tables - not the %_content table - is missing. ** This counts as a corrupted table. */ rc = SQLITE_CORRUPT; } } } *ppStmt = p->aStmt[eStmt]; sqlite3_reset(*ppStmt); return rc; |
︙ | ︙ | |||
293 294 295 296 297 298 299 | memset(p, 0, (size_t)nByte); p->aTotalSize = (i64*)&p[1]; p->pConfig = pConfig; p->pIndex = pIndex; if( bCreate ){ | | > > | > > > | | > > > > > > > > > > > > > | < < | 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 | memset(p, 0, (size_t)nByte); p->aTotalSize = (i64*)&p[1]; p->pConfig = pConfig; p->pIndex = pIndex; if( bCreate ){ if( pConfig->eContent==FTS5_CONTENT_NORMAL || pConfig->eContent==FTS5_CONTENT_UNINDEXED ){ int nDefn = 32 + pConfig->nCol*10; char *zDefn = sqlite3_malloc64(32 + (sqlite3_int64)pConfig->nCol * 20); if( zDefn==0 ){ rc = SQLITE_NOMEM; }else{ int i; int iOff; sqlite3_snprintf(nDefn, zDefn, "id INTEGER PRIMARY KEY"); iOff = (int)strlen(zDefn); for(i=0; i<pConfig->nCol; i++){ if( pConfig->eContent==FTS5_CONTENT_NORMAL || pConfig->abUnindexed[i] ){ sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", c%d", i); iOff += (int)strlen(&zDefn[iOff]); } } if( pConfig->bLocale ){ for(i=0; i<pConfig->nCol; i++){ if( pConfig->abUnindexed[i]==0 ){ sqlite3_snprintf(nDefn-iOff, &zDefn[iOff], ", l%d", i); iOff += (int)strlen(&zDefn[iOff]); } } } rc = sqlite3Fts5CreateTable(pConfig, "content", zDefn, 0, pzErr); } sqlite3_free(zDefn); } if( rc==SQLITE_OK && pConfig->bColumnsize ){ const char *zCols = "id INTEGER PRIMARY KEY, sz BLOB"; if( pConfig->bContentlessDelete ){ zCols = "id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER"; } rc = sqlite3Fts5CreateTable(pConfig, "docsize", zCols, 0, pzErr); } if( rc==SQLITE_OK ){ rc = sqlite3Fts5CreateTable( pConfig, "config", "k PRIMARY KEY, v", 1, pzErr ); } if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 | UNUSED_PARAM2(iUnused1, iUnused2); if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE; if( (tflags & FTS5_TOKEN_COLOCATED)==0 || pCtx->szCol==0 ){ pCtx->szCol++; } return sqlite3Fts5IndexWrite(pIdx, pCtx->iCol, pCtx->szCol-1, pToken, nToken); } /* ** If a row with rowid iDel is present in the %_content table, add the ** delete-markers to the FTS index necessary to delete it. Do not actually ** remove the %_content row at this time though. */ static int fts5StorageDeleteFromIndex( Fts5Storage *p, i64 iDel, | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | > > > > > > > > | | | | | > < > > | | > > > | > > > | > > | > > > > > > > > > > | | < | > > | > > > > > | | | | | | | | > > > > > > > | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 | UNUSED_PARAM2(iUnused1, iUnused2); if( nToken>FTS5_MAX_TOKEN_SIZE ) nToken = FTS5_MAX_TOKEN_SIZE; if( (tflags & FTS5_TOKEN_COLOCATED)==0 || pCtx->szCol==0 ){ pCtx->szCol++; } return sqlite3Fts5IndexWrite(pIdx, pCtx->iCol, pCtx->szCol-1, pToken, nToken); } /* ** This function is used as part of an UPDATE statement that modifies the ** rowid of a row. In that case, this function is called first to set ** Fts5Storage.pSavedRow to point to a statement that may be used to ** access the original values of the row being deleted - iDel. ** ** SQLITE_OK is returned if successful, or an SQLite error code otherwise. ** It is not considered an error if row iDel does not exist. In this case ** pSavedRow is not set and SQLITE_OK returned. */ int sqlite3Fts5StorageFindDeleteRow(Fts5Storage *p, i64 iDel){ int rc = SQLITE_OK; sqlite3_stmt *pSeek = 0; assert( p->pSavedRow==0 ); rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP+1, &pSeek, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int64(pSeek, 1, iDel); if( sqlite3_step(pSeek)!=SQLITE_ROW ){ rc = sqlite3_reset(pSeek); }else{ p->pSavedRow = pSeek; } } return rc; } /* ** If a row with rowid iDel is present in the %_content table, add the ** delete-markers to the FTS index necessary to delete it. Do not actually ** remove the %_content row at this time though. ** ** If parameter bSaveRow is true, then Fts5Storage.pSavedRow is left ** pointing to a statement (FTS5_STMT_LOOKUP2) that may be used to access ** the original values of the row being deleted. This is used by UPDATE ** statements. */ static int fts5StorageDeleteFromIndex( Fts5Storage *p, i64 iDel, sqlite3_value **apVal, int bSaveRow /* True to set pSavedRow */ ){ Fts5Config *pConfig = p->pConfig; sqlite3_stmt *pSeek = 0; /* SELECT to read row iDel from %_data */ int rc = SQLITE_OK; /* Return code */ int rc2; /* sqlite3_reset() return code */ int iCol; Fts5InsertCtx ctx; assert( bSaveRow==0 || apVal==0 ); assert( bSaveRow==0 || bSaveRow==1 ); assert( FTS5_STMT_LOOKUP2==FTS5_STMT_LOOKUP+1 ); if( apVal==0 ){ if( p->pSavedRow && bSaveRow ){ pSeek = p->pSavedRow; p->pSavedRow = 0; }else{ rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP+bSaveRow, &pSeek, 0); if( rc!=SQLITE_OK ) return rc; sqlite3_bind_int64(pSeek, 1, iDel); if( sqlite3_step(pSeek)!=SQLITE_ROW ){ return sqlite3_reset(pSeek); } } } ctx.pStorage = p; ctx.iCol = -1; for(iCol=1; rc==SQLITE_OK && iCol<=pConfig->nCol; iCol++){ if( pConfig->abUnindexed[iCol-1]==0 ){ sqlite3_value *pVal = 0; sqlite3_value *pFree = 0; const char *pText = 0; int nText = 0; const char *pLoc = 0; int nLoc = 0; assert( pSeek==0 || apVal==0 ); assert( pSeek!=0 || apVal!=0 ); if( pSeek ){ pVal = sqlite3_column_value(pSeek, iCol); }else{ pVal = apVal[iCol-1]; } if( pConfig->bLocale && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc); }else{ if( sqlite3_value_type(pVal)!=SQLITE_TEXT ){ /* Make a copy of the value to work with. This is because the call ** to sqlite3_value_text() below forces the type of the value to ** SQLITE_TEXT, and we may need to use it again later. */ pFree = pVal = sqlite3_value_dup(pVal); if( pVal==0 ){ rc = SQLITE_NOMEM; } } if( rc==SQLITE_OK ){ pText = (const char*)sqlite3_value_text(pVal); nText = sqlite3_value_bytes(pVal); if( pConfig->bLocale && pSeek ){ pLoc = (const char*)sqlite3_column_text(pSeek, iCol+pConfig->nCol); nLoc = sqlite3_column_bytes(pSeek, iCol + pConfig->nCol); } } } if( rc==SQLITE_OK ){ sqlite3Fts5SetLocale(pConfig, pLoc, nLoc); ctx.szCol = 0; rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, pText, nText, (void*)&ctx, fts5StorageInsertCallback ); p->aTotalSize[iCol-1] -= (i64)ctx.szCol; if( rc==SQLITE_OK && p->aTotalSize[iCol-1]<0 ){ rc = FTS5_CORRUPT; } sqlite3Fts5ClearLocale(pConfig); } sqlite3_value_free(pFree); } } if( rc==SQLITE_OK && p->nTotalRow<1 ){ rc = FTS5_CORRUPT; }else{ p->nTotalRow--; } if( rc==SQLITE_OK && bSaveRow ){ assert( p->pSavedRow==0 ); p->pSavedRow = pSeek; }else{ rc2 = sqlite3_reset(pSeek); if( rc==SQLITE_OK ) rc = rc2; } return rc; } /* ** Reset any saved statement pSavedRow. Zero pSavedRow as well. This ** should be called by the xUpdate() method of the fts5 table before ** returning from any operation that may have set Fts5Storage.pSavedRow. */ void sqlite3Fts5StorageReleaseDeleteRow(Fts5Storage *pStorage){ assert( pStorage->pSavedRow==0 || pStorage->pSavedRow==pStorage->aStmt[FTS5_STMT_LOOKUP2] ); sqlite3_reset(pStorage->pSavedRow); pStorage->pSavedRow = 0; } /* ** This function is called to process a DELETE on a contentless_delete=1 ** table. It adds the tombstone required to delete the entry with rowid ** iDel. If successful, SQLITE_OK is returned. Or, if an error occurs, ** an SQLite error code. */ static int fts5StorageContentlessDelete(Fts5Storage *p, i64 iDel){ i64 iOrigin = 0; sqlite3_stmt *pLookup = 0; int rc = SQLITE_OK; assert( p->pConfig->bContentlessDelete ); assert( p->pConfig->eContent==FTS5_CONTENT_NONE || p->pConfig->eContent==FTS5_CONTENT_UNINDEXED ); /* Look up the origin of the document in the %_docsize table. Store ** this in stack variable iOrigin. */ rc = fts5StorageGetStmt(p, FTS5_STMT_LOOKUP_DOCSIZE, &pLookup, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int64(pLookup, 1, iDel); if( SQLITE_ROW==sqlite3_step(pLookup) ){ iOrigin = sqlite3_column_int64(pLookup, 1); } rc = sqlite3_reset(pLookup); } if( rc==SQLITE_OK && iOrigin!=0 ){ rc = sqlite3Fts5IndexContentlessDelete(p->pIndex, iOrigin, iDel); } return rc; } /* ** Insert a record into the %_docsize table. Specifically, do: ** ** INSERT OR REPLACE INTO %_docsize(id, sz) VALUES(iRowid, pBuf); ** ** If there is no %_docsize table (as happens if the columnsize=0 option ** is specified when the FTS5 table is created), this function is a no-op. */ static int fts5StorageInsertDocsize( Fts5Storage *p, /* Storage module to write to */ i64 iRowid, /* id value */ Fts5Buffer *pBuf /* sz value */ ){ int rc = SQLITE_OK; if( p->pConfig->bColumnsize ){ sqlite3_stmt *pReplace = 0; rc = fts5StorageGetStmt(p, FTS5_STMT_REPLACE_DOCSIZE, &pReplace, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int64(pReplace, 1, iRowid); if( p->pConfig->bContentlessDelete ){ i64 iOrigin = 0; rc = sqlite3Fts5IndexGetOrigin(p->pIndex, &iOrigin); sqlite3_bind_int64(pReplace, 3, iOrigin); } } if( rc==SQLITE_OK ){ sqlite3_bind_blob(pReplace, 2, pBuf->p, pBuf->n, SQLITE_STATIC); sqlite3_step(pReplace); rc = sqlite3_reset(pReplace); sqlite3_bind_null(pReplace, 2); } } return rc; |
︙ | ︙ | |||
522 523 524 525 526 527 528 | return rc; } /* ** Remove a row from the FTS table. */ | | > > > > > > > > > > > > > > > > > > | > | > > | 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 | return rc; } /* ** Remove a row from the FTS table. */ int sqlite3Fts5StorageDelete( Fts5Storage *p, /* Storage object */ i64 iDel, /* Rowid to delete from table */ sqlite3_value **apVal, /* Optional - values to remove from index */ int bSaveRow /* If true, set pSavedRow for deleted row */ ){ Fts5Config *pConfig = p->pConfig; int rc; sqlite3_stmt *pDel = 0; assert( pConfig->eContent!=FTS5_CONTENT_NORMAL || apVal==0 ); rc = fts5StorageLoadTotals(p, 1); /* Delete the index records */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 1, iDel); } if( rc==SQLITE_OK ){ if( p->pConfig->bContentlessDelete ){ rc = fts5StorageContentlessDelete(p, iDel); if( rc==SQLITE_OK && bSaveRow && p->pConfig->eContent==FTS5_CONTENT_UNINDEXED ){ rc = sqlite3Fts5StorageFindDeleteRow(p, iDel); } }else{ rc = fts5StorageDeleteFromIndex(p, iDel, apVal, bSaveRow); } } /* Delete the %_docsize record */ if( rc==SQLITE_OK && pConfig->bColumnsize ){ rc = fts5StorageGetStmt(p, FTS5_STMT_DELETE_DOCSIZE, &pDel, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int64(pDel, 1, iDel); sqlite3_step(pDel); rc = sqlite3_reset(pDel); } } /* Delete the %_content record */ if( pConfig->eContent==FTS5_CONTENT_NORMAL || pConfig->eContent==FTS5_CONTENT_UNINDEXED ){ if( rc==SQLITE_OK ){ rc = fts5StorageGetStmt(p, FTS5_STMT_DELETE_CONTENT, &pDel, 0); } if( rc==SQLITE_OK ){ sqlite3_bind_int64(pDel, 1, iDel); sqlite3_step(pDel); rc = sqlite3_reset(pDel); |
︙ | ︙ | |||
578 579 580 581 582 583 584 | "DELETE FROM %Q.'%q_data';" "DELETE FROM %Q.'%q_idx';", pConfig->zDb, pConfig->zName, pConfig->zDb, pConfig->zName ); if( rc==SQLITE_OK && pConfig->bColumnsize ){ rc = fts5ExecPrintf(pConfig->db, 0, | | > > > > > | | 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 | "DELETE FROM %Q.'%q_data';" "DELETE FROM %Q.'%q_idx';", pConfig->zDb, pConfig->zName, pConfig->zDb, pConfig->zName ); if( rc==SQLITE_OK && pConfig->bColumnsize ){ rc = fts5ExecPrintf(pConfig->db, 0, "DELETE FROM %Q.'%q_docsize';", pConfig->zDb, pConfig->zName ); } if( rc==SQLITE_OK && pConfig->eContent==FTS5_CONTENT_UNINDEXED ){ rc = fts5ExecPrintf(pConfig->db, 0, "DELETE FROM %Q.'%q_content';", pConfig->zDb, pConfig->zName ); } /* Reinitialize the %_data table. This call creates the initial structure ** and averages records. */ if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexReinit(p->pIndex); |
︙ | ︙ | |||
609 610 611 612 613 614 615 | ctx.pStorage = p; rc = sqlite3Fts5StorageDeleteAll(p); if( rc==SQLITE_OK ){ rc = fts5StorageLoadTotals(p, 1); } if( rc==SQLITE_OK ){ | | > > > > > > > > > > > > > > > | | > > > > > | | | | | | > > | 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 | ctx.pStorage = p; rc = sqlite3Fts5StorageDeleteAll(p); if( rc==SQLITE_OK ){ rc = fts5StorageLoadTotals(p, 1); } if( rc==SQLITE_OK ){ rc = fts5StorageGetStmt(p, FTS5_STMT_SCAN, &pScan, pConfig->pzErrmsg); } while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pScan) ){ i64 iRowid = sqlite3_column_int64(pScan, 0); sqlite3Fts5BufferZero(&buf); rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){ ctx.szCol = 0; if( pConfig->abUnindexed[ctx.iCol]==0 ){ int nText = 0; /* Size of pText in bytes */ const char *pText = 0; /* Pointer to buffer containing text value */ int nLoc = 0; /* Size of pLoc in bytes */ const char *pLoc = 0; /* Pointer to buffer containing text value */ sqlite3_value *pVal = sqlite3_column_value(pScan, ctx.iCol+1); if( pConfig->eContent==FTS5_CONTENT_EXTERNAL && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc); }else{ pText = (const char*)sqlite3_value_text(pVal); nText = sqlite3_value_bytes(pVal); if( pConfig->bLocale ){ int iCol = ctx.iCol + 1 + pConfig->nCol; pLoc = (const char*)sqlite3_column_text(pScan, iCol); nLoc = sqlite3_column_bytes(pScan, iCol); } } if( rc==SQLITE_OK ){ sqlite3Fts5SetLocale(pConfig, pLoc, nLoc); rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, pText, nText, (void*)&ctx, fts5StorageInsertCallback ); sqlite3Fts5ClearLocale(pConfig); } } sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol); p->aTotalSize[ctx.iCol] += (i64)ctx.szCol; } p->nTotalRow++; if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
693 694 695 696 697 698 699 700 701 702 703 704 705 706 | } /* ** Insert a new row into the FTS content table. */ int sqlite3Fts5StorageContentInsert( Fts5Storage *p, sqlite3_value **apVal, i64 *piRowid ){ Fts5Config *pConfig = p->pConfig; int rc = SQLITE_OK; /* Insert the new row into the %_content table. */ | > | > > > > > | > > > > > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > | 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 | } /* ** Insert a new row into the FTS content table. */ int sqlite3Fts5StorageContentInsert( Fts5Storage *p, int bReplace, /* True to use REPLACE instead of INSERT */ sqlite3_value **apVal, i64 *piRowid ){ Fts5Config *pConfig = p->pConfig; int rc = SQLITE_OK; /* Insert the new row into the %_content table. */ if( pConfig->eContent!=FTS5_CONTENT_NORMAL && pConfig->eContent!=FTS5_CONTENT_UNINDEXED ){ if( sqlite3_value_type(apVal[1])==SQLITE_INTEGER ){ *piRowid = sqlite3_value_int64(apVal[1]); }else{ rc = fts5StorageNewRowid(p, piRowid); } }else{ sqlite3_stmt *pInsert = 0; /* Statement to write %_content table */ int i; /* Counter variable */ assert( FTS5_STMT_INSERT_CONTENT+1==FTS5_STMT_REPLACE_CONTENT ); assert( bReplace==0 || bReplace==1 ); rc = fts5StorageGetStmt(p, FTS5_STMT_INSERT_CONTENT+bReplace, &pInsert, 0); if( pInsert ) sqlite3_clear_bindings(pInsert); /* Bind the rowid value */ sqlite3_bind_value(pInsert, 1, apVal[1]); /* Loop through values for user-defined columns. i=2 is the leftmost ** user-defined column. As is column 1 of pSavedRow. */ for(i=2; rc==SQLITE_OK && i<=pConfig->nCol+1; i++){ int bUnindexed = pConfig->abUnindexed[i-2]; if( pConfig->eContent==FTS5_CONTENT_NORMAL || bUnindexed ){ sqlite3_value *pVal = apVal[i]; if( sqlite3_value_nochange(pVal) && p->pSavedRow ){ /* This is an UPDATE statement, and user-defined column (i-2) was not ** modified. Retrieve the value from Fts5Storage.pSavedRow. */ pVal = sqlite3_column_value(p->pSavedRow, i-1); if( pConfig->bLocale && bUnindexed==0 ){ sqlite3_bind_value(pInsert, pConfig->nCol + i, sqlite3_column_value(p->pSavedRow, pConfig->nCol + i - 1) ); } }else if( sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ const char *pText = 0; const char *pLoc = 0; int nText = 0; int nLoc = 0; assert( pConfig->bLocale ); rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc); if( rc==SQLITE_OK ){ sqlite3_bind_text(pInsert, i, pText, nText, SQLITE_TRANSIENT); if( bUnindexed==0 ){ int iLoc = pConfig->nCol + i; sqlite3_bind_text(pInsert, iLoc, pLoc, nLoc, SQLITE_TRANSIENT); } } continue; } rc = sqlite3_bind_value(pInsert, i, pVal); } } if( rc==SQLITE_OK ){ sqlite3_step(pInsert); rc = sqlite3_reset(pInsert); } *piRowid = sqlite3_last_insert_rowid(pConfig->db); } |
︙ | ︙ | |||
746 747 748 749 750 751 752 | if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); } for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){ ctx.szCol = 0; if( pConfig->abUnindexed[ctx.iCol]==0 ){ | > > > > > > > > > > > > > > > > > > > > | | > > > > | | < < | | > > | 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 | if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexBeginWrite(p->pIndex, 0, iRowid); } for(ctx.iCol=0; rc==SQLITE_OK && ctx.iCol<pConfig->nCol; ctx.iCol++){ ctx.szCol = 0; if( pConfig->abUnindexed[ctx.iCol]==0 ){ int nText = 0; /* Size of pText in bytes */ const char *pText = 0; /* Pointer to buffer containing text value */ int nLoc = 0; /* Size of pText in bytes */ const char *pLoc = 0; /* Pointer to buffer containing text value */ sqlite3_value *pVal = apVal[ctx.iCol+2]; if( p->pSavedRow && sqlite3_value_nochange(pVal) ){ pVal = sqlite3_column_value(p->pSavedRow, ctx.iCol+1); if( pConfig->eContent==FTS5_CONTENT_NORMAL && pConfig->bLocale ){ int iCol = ctx.iCol + 1 + pConfig->nCol; pLoc = (const char*)sqlite3_column_text(p->pSavedRow, iCol); nLoc = sqlite3_column_bytes(p->pSavedRow, iCol); } }else{ pVal = apVal[ctx.iCol+2]; } if( pConfig->bLocale && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ rc = sqlite3Fts5DecodeLocaleValue(pVal, &pText, &nText, &pLoc, &nLoc); }else{ pText = (const char*)sqlite3_value_text(pVal); nText = sqlite3_value_bytes(pVal); } if( rc==SQLITE_OK ){ sqlite3Fts5SetLocale(pConfig, pLoc, nLoc); rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, pText, nText, (void*)&ctx, fts5StorageInsertCallback ); sqlite3Fts5ClearLocale(pConfig); } } sqlite3Fts5BufferAppendVarint(&rc, &buf, ctx.szCol); p->aTotalSize[ctx.iCol] += (i64)ctx.szCol; } p->nTotalRow++; /* Write the %_docsize record */ |
︙ | ︙ | |||
917 918 919 920 921 922 923 | if( pConfig->bColumnsize ){ rc = sqlite3Fts5StorageDocsize(p, ctx.iRowid, aColSize); } if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_NONE ){ rc = sqlite3Fts5TermsetNew(&ctx.pTermset); } for(i=0; rc==SQLITE_OK && i<pConfig->nCol; i++){ | | > > > > > > > > > > > > > > > > > > > > > > | | > | | | > | < | | | | | | | > | > > > > | > > > | | | | | | > | 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 | if( pConfig->bColumnsize ){ rc = sqlite3Fts5StorageDocsize(p, ctx.iRowid, aColSize); } if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_NONE ){ rc = sqlite3Fts5TermsetNew(&ctx.pTermset); } for(i=0; rc==SQLITE_OK && i<pConfig->nCol; i++){ if( pConfig->abUnindexed[i]==0 ){ const char *pText = 0; int nText = 0; const char *pLoc = 0; int nLoc = 0; sqlite3_value *pVal = sqlite3_column_value(pScan, i+1); if( pConfig->eContent==FTS5_CONTENT_EXTERNAL && sqlite3Fts5IsLocaleValue(pConfig, pVal) ){ rc = sqlite3Fts5DecodeLocaleValue( pVal, &pText, &nText, &pLoc, &nLoc ); }else{ if( pConfig->eContent==FTS5_CONTENT_NORMAL && pConfig->bLocale ){ int iCol = i + 1 + pConfig->nCol; pLoc = (const char*)sqlite3_column_text(pScan, iCol); nLoc = sqlite3_column_bytes(pScan, iCol); } pText = (const char*)sqlite3_value_text(pVal); nText = sqlite3_value_bytes(pVal); } ctx.iCol = i; ctx.szCol = 0; if( rc==SQLITE_OK && pConfig->eDetail==FTS5_DETAIL_COLUMNS ){ rc = sqlite3Fts5TermsetNew(&ctx.pTermset); } if( rc==SQLITE_OK ){ sqlite3Fts5SetLocale(pConfig, pLoc, nLoc); rc = sqlite3Fts5Tokenize(pConfig, FTS5_TOKENIZE_DOCUMENT, pText, nText, (void*)&ctx, fts5StorageIntegrityCallback ); sqlite3Fts5ClearLocale(pConfig); } /* If this is not a columnsize=0 database, check that the number ** of tokens in the value matches the aColSize[] value read from ** the %_docsize table. */ if( rc==SQLITE_OK && pConfig->bColumnsize && ctx.szCol!=aColSize[i] ){ rc = FTS5_CORRUPT; } aTotalSize[i] += ctx.szCol; if( pConfig->eDetail==FTS5_DETAIL_COLUMNS ){ sqlite3Fts5TermsetFree(ctx.pTermset); ctx.pTermset = 0; } } } sqlite3Fts5TermsetFree(ctx.pTermset); ctx.pTermset = 0; if( rc!=SQLITE_OK ) break; } |
︙ | ︙ | |||
1120 1121 1122 1123 1124 1125 1126 | ** Flush any data currently held in-memory to disk. */ int sqlite3Fts5StorageSync(Fts5Storage *p){ int rc = SQLITE_OK; i64 iLastRowid = sqlite3_last_insert_rowid(p->pConfig->db); if( p->bTotalsValid ){ rc = fts5StorageSaveTotals(p); | > | > | 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 | ** Flush any data currently held in-memory to disk. */ int sqlite3Fts5StorageSync(Fts5Storage *p){ int rc = SQLITE_OK; i64 iLastRowid = sqlite3_last_insert_rowid(p->pConfig->db); if( p->bTotalsValid ){ rc = fts5StorageSaveTotals(p); if( rc==SQLITE_OK ){ p->bTotalsValid = 0; } } if( rc==SQLITE_OK ){ rc = sqlite3Fts5IndexSync(p->pIndex); } sqlite3_set_last_insert_rowid(p->pConfig->db, iLastRowid); return rc; } |
︙ | ︙ |
Changes to ext/fts5/fts5_tcl.c.
︙ | ︙ | |||
10 11 12 13 14 15 16 | ** ****************************************************************************** ** */ #ifdef SQLITE_TEST | < < < | < < < < > | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | ** ****************************************************************************** ** */ #ifdef SQLITE_TEST #include "tclsqlite.h" #ifdef SQLITE_ENABLE_FTS5 #include "fts5.h" #include <string.h> #include <assert.h> #include <stdlib.h> #ifdef SQLITE_DEBUG extern int sqlite3_fts5_may_be_corrupt; #endif extern int sqlite3Fts5TestRegisterMatchinfo(sqlite3*); extern int sqlite3Fts5TestRegisterTok(sqlite3*, fts5_api*); |
︙ | ︙ | |||
99 100 101 102 103 104 105 | return TCL_ERROR; }else{ sqlite3_stmt *pStmt = 0; fts5_api *pApi = 0; rc = sqlite3_prepare_v2(db, "SELECT fts5(?1)", -1, &pStmt, 0); if( rc!=SQLITE_OK ){ | | | | 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 | return TCL_ERROR; }else{ sqlite3_stmt *pStmt = 0; fts5_api *pApi = 0; rc = sqlite3_prepare_v2(db, "SELECT fts5(?1)", -1, &pStmt, 0); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0); return TCL_ERROR; } sqlite3_bind_pointer(pStmt, 1, (void*)&pApi, "fts5_api_ptr", 0); sqlite3_step(pStmt); if( sqlite3_finalize(pStmt)!=SQLITE_OK ){ Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0); return TCL_ERROR; } *ppDb = db; *ppApi = pApi; } |
︙ | ︙ | |||
240 241 242 243 244 245 246 247 248 249 250 251 252 253 | { "xQueryPhrase", 2, "PHRASE SCRIPT" }, /* 11 */ { "xSetAuxdata", 1, "VALUE" }, /* 12 */ { "xGetAuxdata", 1, "CLEAR" }, /* 13 */ { "xSetAuxdataInt", 1, "INTEGER" }, /* 14 */ { "xGetAuxdataInt", 1, "CLEAR" }, /* 15 */ { "xPhraseForeach", 4, "IPHRASE COLVAR OFFVAR SCRIPT" }, /* 16 */ { "xPhraseColumnForeach", 3, "IPHRASE COLVAR SCRIPT" }, /* 17 */ { 0, 0, 0} }; int rc; int iSub = 0; F5tApi *p = (F5tApi*)clientData; | > > > > | 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | { "xQueryPhrase", 2, "PHRASE SCRIPT" }, /* 11 */ { "xSetAuxdata", 1, "VALUE" }, /* 12 */ { "xGetAuxdata", 1, "CLEAR" }, /* 13 */ { "xSetAuxdataInt", 1, "INTEGER" }, /* 14 */ { "xGetAuxdataInt", 1, "CLEAR" }, /* 15 */ { "xPhraseForeach", 4, "IPHRASE COLVAR OFFVAR SCRIPT" }, /* 16 */ { "xPhraseColumnForeach", 3, "IPHRASE COLVAR SCRIPT" }, /* 17 */ { "xQueryToken", 2, "IPHRASE ITERM" }, /* 18 */ { "xInstToken", 2, "IDX ITERM" }, /* 19 */ { "xColumnLocale", 1, "COL" }, /* 20 */ { 0, 0, 0} }; int rc; int iSub = 0; F5tApi *p = (F5tApi*)clientData; |
︙ | ︙ | |||
290 291 292 293 294 295 296 | rc = p->pApi->xColumnTotalSize(p->pFts, iCol, &nSize); if( rc==SQLITE_OK ){ Tcl_SetObjResult(interp, Tcl_NewWideIntObj(nSize)); } break; } CASE(3, "xTokenize") { | | | | 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 | rc = p->pApi->xColumnTotalSize(p->pFts, iCol, &nSize); if( rc==SQLITE_OK ){ Tcl_SetObjResult(interp, Tcl_NewWideIntObj(nSize)); } break; } CASE(3, "xTokenize") { Tcl_Size nText; char *zText = Tcl_GetStringFromObj(objv[2], &nText); F5tFunction ctx; ctx.interp = interp; ctx.pScript = objv[3]; rc = p->pApi->xTokenize(p->pFts, zText, (int)nText, &ctx, xTokenizeCb); if( rc==SQLITE_OK ){ Tcl_ResetResult(interp); } return rc; } CASE(4, "xPhraseCount") { int nPhrase; |
︙ | ︙ | |||
391 392 393 394 395 396 397 | Tcl_ResetResult(interp); } break; } CASE(12, "xSetAuxdata") { F5tAuxData *pData = (F5tAuxData*)sqlite3_malloc(sizeof(F5tAuxData)); if( pData==0 ){ | | | 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 | Tcl_ResetResult(interp); } break; } CASE(12, "xSetAuxdata") { F5tAuxData *pData = (F5tAuxData*)sqlite3_malloc(sizeof(F5tAuxData)); if( pData==0 ){ Tcl_AppendResult(interp, "out of memory", (char*)0); return TCL_ERROR; } pData->pObj = objv[2]; Tcl_IncrRefCount(pData->pObj); rc = p->pApi->xSetAuxdata(p->pFts, pData, xSetAuxdataDestructor); break; } |
︙ | ︙ | |||
451 452 453 454 455 456 457 | if( Tcl_GetIntFromObj(interp, objv[2], &iPhrase) ) return TCL_ERROR; zColvar = Tcl_GetString(objv[3]); zOffvar = Tcl_GetString(objv[4]); rc = p->pApi->xPhraseFirst(p->pFts, iPhrase, &iter, &iCol, &iOff); if( rc!=SQLITE_OK ){ | | | 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 | if( Tcl_GetIntFromObj(interp, objv[2], &iPhrase) ) return TCL_ERROR; zColvar = Tcl_GetString(objv[3]); zOffvar = Tcl_GetString(objv[4]); rc = p->pApi->xPhraseFirst(p->pFts, iPhrase, &iter, &iCol, &iOff); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, sqlite3ErrName(rc), (char*)0); return TCL_ERROR; } for( ;iCol>=0; p->pApi->xPhraseNext(p->pFts, &iter, &iCol, &iOff) ){ Tcl_SetVar2Ex(interp, zColvar, 0, Tcl_NewIntObj(iCol), 0); Tcl_SetVar2Ex(interp, zOffvar, 0, Tcl_NewIntObj(iOff), 0); rc = Tcl_EvalObjEx(interp, pScript, 0); if( rc==TCL_CONTINUE ) rc = TCL_OK; |
︙ | ︙ | |||
495 496 497 498 499 500 501 502 503 504 505 506 507 508 | if( rc==TCL_BREAK ) rc = TCL_OK; break; } } break; } default: assert( 0 ); break; } #undef CASE | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 | if( rc==TCL_BREAK ) rc = TCL_OK; break; } } break; } CASE(18, "xQueryToken") { const char *pTerm = 0; int nTerm = 0; int iPhrase = 0; int iTerm = 0; if( Tcl_GetIntFromObj(interp, objv[2], &iPhrase) ) return TCL_ERROR; if( Tcl_GetIntFromObj(interp, objv[3], &iTerm) ) return TCL_ERROR; rc = p->pApi->xQueryToken(p->pFts, iPhrase, iTerm, &pTerm, &nTerm); if( rc==SQLITE_OK ){ Tcl_SetObjResult(interp, Tcl_NewStringObj(pTerm, nTerm)); } break; } CASE(19, "xInstToken") { const char *pTerm = 0; int nTerm = 0; int iIdx = 0; int iTerm = 0; if( Tcl_GetIntFromObj(interp, objv[2], &iIdx) ) return TCL_ERROR; if( Tcl_GetIntFromObj(interp, objv[3], &iTerm) ) return TCL_ERROR; rc = p->pApi->xInstToken(p->pFts, iIdx, iTerm, &pTerm, &nTerm); if( rc==SQLITE_OK ){ Tcl_SetObjResult(interp, Tcl_NewStringObj(pTerm, nTerm)); } break; } CASE(20, "xColumnLocale") { const char *z = 0; int n = 0; int iCol; if( Tcl_GetIntFromObj(interp, objv[2], &iCol) ){ return TCL_ERROR; } rc = p->pApi->xColumnLocale(p->pFts, iCol, &z, &n); if( rc==SQLITE_OK && z ){ Tcl_SetObjResult(interp, Tcl_NewStringObj(z, n)); } break; } default: assert( 0 ); break; } #undef CASE |
︙ | ︙ | |||
566 567 568 569 570 571 572 | Tcl_DecrRefCount(pEval); Tcl_DeleteCommand(p->interp, zCmd); if( rc!=TCL_OK ){ sqlite3_result_error(pCtx, Tcl_GetStringResult(p->interp), -1); }else{ Tcl_Obj *pVar = Tcl_GetObjResult(p->interp); | < > | | > > | | | 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 | Tcl_DecrRefCount(pEval); Tcl_DeleteCommand(p->interp, zCmd); if( rc!=TCL_OK ){ sqlite3_result_error(pCtx, Tcl_GetStringResult(p->interp), -1); }else{ Tcl_Obj *pVar = Tcl_GetObjResult(p->interp); const char *zType = (pVar->typePtr ? pVar->typePtr->name : ""); char c = zType[0]; if( c=='b' && strcmp(zType,"bytearray")==0 && pVar->bytes==0 ){ /* Only return a BLOB type if the Tcl variable is a bytearray and ** has no string representation. */ Tcl_Size nn; unsigned char *data = Tcl_GetByteArrayFromObj(pVar, &nn); sqlite3_result_blob(pCtx, data, (int)nn, SQLITE_TRANSIENT); }else if( c=='b' && strcmp(zType,"boolean")==0 ){ int n; Tcl_GetIntFromObj(0, pVar, &n); sqlite3_result_int(pCtx, n); }else if( c=='d' && strcmp(zType,"double")==0 ){ double r; Tcl_GetDoubleFromObj(0, pVar, &r); sqlite3_result_double(pCtx, r); }else if( (c=='w' && strcmp(zType,"wideInt")==0) || (c=='i' && strcmp(zType,"int")==0) ){ Tcl_WideInt v; Tcl_GetWideIntFromObj(0, pVar, &v); sqlite3_result_int64(pCtx, v); }else{ Tcl_Size nn; unsigned char *data = (unsigned char *)Tcl_GetStringFromObj(pVar, &nn); sqlite3_result_text(pCtx, (char*)data, (int)nn, SQLITE_TRANSIENT); } } } static void xF5tDestroy(void *pCtx){ F5tFunction *p = (F5tFunction*)pCtx; Tcl_DecrRefCount(p->pScript); |
︙ | ︙ | |||
634 635 636 637 638 639 640 | pCtx->pScript = pScript; Tcl_IncrRefCount(pScript); rc = pApi->xCreateFunction( pApi, zName, (void*)pCtx, xF5tFunction, xF5tDestroy ); if( rc!=SQLITE_OK ){ | | | 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 | pCtx->pScript = pScript; Tcl_IncrRefCount(pScript); rc = pApi->xCreateFunction( pApi, zName, (void*)pCtx, xF5tFunction, xF5tDestroy ); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (char*)0); return TCL_ERROR; } return TCL_OK; } typedef struct F5tTokenizeCtx F5tTokenizeCtx; |
︙ | ︙ | |||
680 681 682 683 684 685 686 | */ static int SQLITE_TCLAPI f5tTokenize( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ | > | | | | | | | | > > > > > > > > > > > | | > | < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | > > | > > > | 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 | */ static int SQLITE_TCLAPI f5tTokenize( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ char *pCopy = 0; char *zText = 0; Tcl_Size nText = 0; sqlite3 *db = 0; fts5_api *pApi = 0; Fts5Tokenizer *pTok = 0; fts5_tokenizer tokenizer; Tcl_Obj *pRet = 0; void *pUserdata; int rc; Tcl_Size nArg; const char **azArg; F5tTokenizeCtx ctx; if( objc!=4 && objc!=5 ){ Tcl_WrongNumArgs(interp, 1, objv, "?-subst? DB NAME TEXT"); return TCL_ERROR; } if( objc==5 ){ char *zOpt = Tcl_GetString(objv[1]); if( strcmp("-subst", zOpt) ){ Tcl_AppendResult(interp, "unrecognized option: ", zOpt, (char*)0); return TCL_ERROR; } } if( f5tDbAndApi(interp, objv[objc-3], &db, &pApi) ) return TCL_ERROR; if( Tcl_SplitList(interp, Tcl_GetString(objv[objc-2]), &nArg, &azArg) ){ return TCL_ERROR; } if( nArg==0 ){ Tcl_AppendResult(interp, "no such tokenizer: ", (char*)0); Tcl_Free((void*)azArg); return TCL_ERROR; } zText = Tcl_GetStringFromObj(objv[objc-1], &nText); rc = pApi->xFindTokenizer(pApi, azArg[0], &pUserdata, &tokenizer); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "no such tokenizer: ", azArg[0], (char*)0); return TCL_ERROR; } rc = tokenizer.xCreate(pUserdata, &azArg[1], (int)(nArg-1), &pTok); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "error in tokenizer.xCreate()", (char*)0); return TCL_ERROR; } if( nText>0 ){ pCopy = sqlite3_malloc(nText); if( pCopy==0 ){ tokenizer.xDelete(pTok); Tcl_AppendResult(interp, "error in sqlite3_malloc()", (char*)0); return TCL_ERROR; }else{ memcpy(pCopy, zText, nText); } } pRet = Tcl_NewObj(); Tcl_IncrRefCount(pRet); ctx.bSubst = (objc==5); ctx.pRet = pRet; ctx.zInput = pCopy; rc = tokenizer.xTokenize( pTok, (void*)&ctx, FTS5_TOKENIZE_DOCUMENT, pCopy,(int)nText, xTokenizeCb2 ); tokenizer.xDelete(pTok); sqlite3_free(pCopy); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "error in tokenizer.xTokenize()", (char*)0); Tcl_DecrRefCount(pRet); return TCL_ERROR; } Tcl_Free((void*)azArg); Tcl_SetObjResult(interp, pRet); Tcl_DecrRefCount(pRet); return TCL_OK; } /************************************************************************* ** Start of tokenizer wrapper. */ typedef struct F5tTokenizerContext F5tTokenizerContext; typedef struct F5tTokenizerCb F5tTokenizerCb; typedef struct F5tTokenizerModule F5tTokenizerModule; typedef struct F5tTokenizerInstance F5tTokenizerInstance; struct F5tTokenizerContext { void *pCtx; int (*xToken)(void*, int, const char*, int, int, int); F5tTokenizerInstance *pInst; }; struct F5tTokenizerModule { Tcl_Interp *interp; Tcl_Obj *pScript; void *pParentCtx; fts5_tokenizer_v2 parent_v2; fts5_tokenizer parent; F5tTokenizerContext *pContext; }; /* ** zLocale: ** Within a call to xTokenize_v2(), pLocale/nLocale store the locale ** passed to the call by fts5. This can be retrieved by a Tcl tokenize ** script using [sqlite3_fts5_locale]. */ struct F5tTokenizerInstance { Tcl_Interp *interp; Tcl_Obj *pScript; F5tTokenizerModule *pModule; Fts5Tokenizer *pParent; F5tTokenizerContext *pContext; const char *pLocale; int nLocale; }; static int f5tTokenizerCreate( void *pCtx, const char **azArg, int nArg, Fts5Tokenizer **ppOut ){ Fts5Tokenizer *pParent = 0; F5tTokenizerModule *pMod = (F5tTokenizerModule*)pCtx; Tcl_Obj *pEval; int rc = TCL_OK; int i; assert( pMod->parent_v2.xCreate==0 || pMod->parent.xCreate==0 ); if( pMod->parent_v2.xCreate ){ rc = pMod->parent_v2.xCreate(pMod->pParentCtx, 0, 0, &pParent); } if( pMod->parent.xCreate ){ rc = pMod->parent.xCreate(pMod->pParentCtx, 0, 0, &pParent); } pEval = Tcl_DuplicateObj(pMod->pScript); Tcl_IncrRefCount(pEval); for(i=0; rc==TCL_OK && i<nArg; i++){ Tcl_Obj *pObj = Tcl_NewStringObj(azArg[i], -1); rc = Tcl_ListObjAppendElement(pMod->interp, pEval, pObj); } if( rc==TCL_OK ){ rc = Tcl_EvalObjEx(pMod->interp, pEval, TCL_GLOBAL_ONLY); } Tcl_DecrRefCount(pEval); if( rc==TCL_OK ){ F5tTokenizerInstance *pInst; pInst = (F5tTokenizerInstance*)ckalloc(sizeof(F5tTokenizerInstance)); memset(pInst, 0, sizeof(F5tTokenizerInstance)); pInst->interp = pMod->interp; pInst->pScript = Tcl_GetObjResult(pMod->interp); pInst->pContext = pMod->pContext; pInst->pParent = pParent; pInst->pModule = pMod; Tcl_IncrRefCount(pInst->pScript); *ppOut = (Fts5Tokenizer*)pInst; } return rc; } static void f5tTokenizerDelete(Fts5Tokenizer *p){ F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p; if( pInst ){ if( pInst->pParent ){ if( pInst->pModule->parent_v2.xDelete ){ pInst->pModule->parent_v2.xDelete(pInst->pParent); }else{ pInst->pModule->parent.xDelete(pInst->pParent); } } Tcl_DecrRefCount(pInst->pScript); ckfree((char *)pInst); } } static int f5tTokenizerReallyTokenize( Fts5Tokenizer *p, void *pCtx, int flags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p; F5tTokenizerInstance *pOldInst = 0; void *pOldCtx; int (*xOldToken)(void*, int, const char*, int, int, int); Tcl_Obj *pEval; int rc; const char *zFlags; pOldCtx = pInst->pContext->pCtx; xOldToken = pInst->pContext->xToken; pOldInst = pInst->pContext->pInst; pInst->pContext->pCtx = pCtx; pInst->pContext->xToken = xToken; pInst->pContext->pInst = pInst; assert( flags==FTS5_TOKENIZE_DOCUMENT || flags==FTS5_TOKENIZE_AUX || flags==FTS5_TOKENIZE_QUERY || flags==(FTS5_TOKENIZE_QUERY | FTS5_TOKENIZE_PREFIX) ); |
︙ | ︙ | |||
874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 | Tcl_ListObjAppendElement(pInst->interp, pEval, Tcl_NewStringObj(zFlags, -1)); Tcl_ListObjAppendElement(pInst->interp, pEval, Tcl_NewStringObj(pText,nText)); rc = Tcl_EvalObjEx(pInst->interp, pEval, TCL_GLOBAL_ONLY); Tcl_DecrRefCount(pEval); pInst->pContext->pCtx = pOldCtx; pInst->pContext->xToken = xOldToken; return rc; } /* ** sqlite3_fts5_token ?-colocated? TEXT START END */ static int SQLITE_TCLAPI f5tTokenizerReturn( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ F5tTokenizerContext *p = (F5tTokenizerContext*)clientData; int iStart; int iEnd; | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 | Tcl_ListObjAppendElement(pInst->interp, pEval, Tcl_NewStringObj(zFlags, -1)); Tcl_ListObjAppendElement(pInst->interp, pEval, Tcl_NewStringObj(pText,nText)); rc = Tcl_EvalObjEx(pInst->interp, pEval, TCL_GLOBAL_ONLY); Tcl_DecrRefCount(pEval); pInst->pContext->pCtx = pOldCtx; pInst->pContext->xToken = xOldToken; pInst->pContext->pInst = pOldInst; return rc; } typedef struct CallbackCtx CallbackCtx; struct CallbackCtx { Fts5Tokenizer *p; void *pCtx; int flags; int (*xToken)(void*, int, const char*, int, int, int); }; static int f5tTokenizeCallback( void *pCtx, int tflags, const char *z, int n, int iStart, int iEnd ){ CallbackCtx *p = (CallbackCtx*)pCtx; return f5tTokenizerReallyTokenize(p->p, p->pCtx, p->flags, z, n, p->xToken); } static int f5tTokenizerTokenize_v2( Fts5Tokenizer *p, void *pCtx, int flags, const char *pText, int nText, const char *pLoc, int nLoc, int (*xToken)(void*, int, const char*, int, int, int) ){ int rc = SQLITE_OK; F5tTokenizerInstance *pInst = (F5tTokenizerInstance*)p; pInst->pLocale = pLoc; pInst->nLocale = nLoc; if( pInst->pParent ){ CallbackCtx ctx; ctx.p = p; ctx.pCtx = pCtx; ctx.flags = flags; ctx.xToken = xToken; if( pInst->pModule->parent_v2.xTokenize ){ rc = pInst->pModule->parent_v2.xTokenize( pInst->pParent, (void*)&ctx, flags, pText, nText, pLoc, nLoc, f5tTokenizeCallback ); }else{ rc = pInst->pModule->parent.xTokenize( pInst->pParent, (void*)&ctx, flags, pText, nText, f5tTokenizeCallback ); } }else{ rc = f5tTokenizerReallyTokenize(p, pCtx, flags, pText, nText, xToken); } pInst->pLocale = 0; pInst->nLocale = 0; return rc; } static int f5tTokenizerTokenize( Fts5Tokenizer *p, void *pCtx, int flags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ return f5tTokenizerTokenize_v2(p, pCtx, flags, pText, nText, 0, 0, xToken); } /* ** sqlite3_fts5_locale */ static int SQLITE_TCLAPI f5tTokenizerLocale( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ F5tTokenizerContext *p = (F5tTokenizerContext*)clientData; if( objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, ""); return TCL_ERROR; } if( p->xToken==0 ){ Tcl_AppendResult(interp, "sqlite3_fts5_locale may only be used by tokenizer callback", (char*)0 ); return TCL_ERROR; } Tcl_SetObjResult(interp, Tcl_NewStringObj(p->pInst->pLocale, p->pInst->nLocale) ); return TCL_OK; } /* ** sqlite3_fts5_token ?-colocated? TEXT START END */ static int SQLITE_TCLAPI f5tTokenizerReturn( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ F5tTokenizerContext *p = (F5tTokenizerContext*)clientData; int iStart; int iEnd; Tcl_Size nToken; int tflags = 0; char *zToken; int rc; if( objc==5 ){ Tcl_Size nArg; char *zArg = Tcl_GetStringFromObj(objv[1], &nArg); if( nArg<=10 && nArg>=2 && memcmp("-colocated", zArg, nArg)==0 ){ tflags |= FTS5_TOKEN_COLOCATED; }else{ goto usage; } }else if( objc!=4 ){ goto usage; } zToken = Tcl_GetStringFromObj(objv[objc-3], &nToken); if( Tcl_GetIntFromObj(interp, objv[objc-2], &iStart) || Tcl_GetIntFromObj(interp, objv[objc-1], &iEnd) ){ return TCL_ERROR; } if( p->xToken==0 ){ Tcl_AppendResult(interp, "sqlite3_fts5_token may only be used by tokenizer callback", (char*)0 ); return TCL_ERROR; } rc = p->xToken(p->pCtx, tflags, zToken, (int)nToken, iStart, iEnd); Tcl_SetResult(interp, (char*)sqlite3ErrName(rc), TCL_VOLATILE); return rc==SQLITE_OK ? TCL_OK : TCL_ERROR; usage: Tcl_WrongNumArgs(interp, 1, objv, "?-colocated? TEXT START END"); return TCL_ERROR; } |
︙ | ︙ | |||
962 963 964 965 966 967 968 | Tcl_Obj *CONST objv[] ){ F5tTokenizerContext *pContext = (F5tTokenizerContext*)clientData; sqlite3 *db; fts5_api *pApi; char *zName; Tcl_Obj *pScript; | < | > > > > | | | > > > > > | | > > > > > > > > > > > > > | > > > > | > > > > | > > > | > > > > > > | > > > | > > > > > > > > > > > > > > > > > > | > > > > > > > | > > > > > > > > > > > | > > > | 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 | Tcl_Obj *CONST objv[] ){ F5tTokenizerContext *pContext = (F5tTokenizerContext*)clientData; sqlite3 *db; fts5_api *pApi; char *zName; Tcl_Obj *pScript; F5tTokenizerModule *pMod; int rc = SQLITE_OK; int bV2 = 0; /* True to use _v2 API */ int iVersion = 2; /* Value for _v2.iVersion */ const char *zParent = 0; /* Name of parent tokenizer, if any */ int ii = 0; if( objc<4 ){ Tcl_WrongNumArgs(interp, 1, objv, "?OPTIONS? DB NAME SCRIPT"); return TCL_ERROR; } /* Parse any options. Set stack variables bV2 and zParent. */ for(ii=1; ii<objc-3; ii++){ int iOpt = 0; const char *azOpt[] = { "-v2", "-parent", "-version", 0 }; if( Tcl_GetIndexFromObj(interp, objv[ii], azOpt, "OPTION", 0, &iOpt) ){ return TCL_ERROR; } switch( iOpt ){ case 0: /* -v2 */ { bV2 = 1; break; } case 1: /* -parent */ { ii++; if( ii==objc-3 ){ Tcl_AppendResult( interp, "option requires an argument: -parent", (char*)0 ); return TCL_ERROR; } zParent = Tcl_GetString(objv[ii]); break; } case 2: /* -version */ { ii++; if( ii==objc-3 ){ Tcl_AppendResult( interp, "option requires an argument: -version", (char*)0 ); return TCL_ERROR; } if( Tcl_GetIntFromObj(interp, objv[ii], &iVersion) ){ return TCL_ERROR; } break; } default: assert( 0 ); break; } } if( f5tDbAndApi(interp, objv[objc-3], &db, &pApi) ){ return TCL_ERROR; } zName = Tcl_GetString(objv[objc-2]); pScript = objv[objc-1]; pMod = (F5tTokenizerModule*)ckalloc(sizeof(F5tTokenizerModule)); memset(pMod, 0, sizeof(F5tTokenizerModule)); pMod->interp = interp; pMod->pScript = pScript; Tcl_IncrRefCount(pScript); pMod->pContext = pContext; if( zParent ){ if( bV2 ){ fts5_tokenizer_v2 *pParent = 0; rc = pApi->xFindTokenizer_v2(pApi, zParent, &pMod->pParentCtx, &pParent); if( rc==SQLITE_OK ){ memcpy(&pMod->parent_v2, pParent, sizeof(fts5_tokenizer_v2)); pMod->parent_v2.xDelete(0); } }else{ rc = pApi->xFindTokenizer(pApi, zParent, &pMod->pParentCtx,&pMod->parent); if( rc==SQLITE_OK ){ pMod->parent.xDelete(0); } } } if( rc==SQLITE_OK ){ void *pModCtx = (void*)pMod; if( bV2==0 ){ fts5_tokenizer t; t.xCreate = f5tTokenizerCreate; t.xTokenize = f5tTokenizerTokenize; t.xDelete = f5tTokenizerDelete; rc = pApi->xCreateTokenizer(pApi, zName, pModCtx, &t, f5tDelTokenizer); }else{ fts5_tokenizer_v2 t2; memset(&t2, 0, sizeof(t2)); t2.iVersion = iVersion; t2.xCreate = f5tTokenizerCreate; t2.xTokenize = f5tTokenizerTokenize_v2; t2.xDelete = f5tTokenizerDelete; rc = pApi->xCreateTokenizer_v2(pApi, zName, pModCtx, &t2,f5tDelTokenizer); } } if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, ( bV2 ? "error in fts5_api.xCreateTokenizer_v2()" : "error in fts5_api.xCreateTokenizer()" ), (char*)0); return TCL_ERROR; } return TCL_OK; } static void SQLITE_TCLAPI xF5tFree(ClientData clientData){ |
︙ | ︙ | |||
1044 1045 1046 1047 1048 1049 1050 | static int SQLITE_TCLAPI f5tTokenHash( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ char *z; | | | | 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 | static int SQLITE_TCLAPI f5tTokenHash( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ char *z; Tcl_Size n; unsigned int iVal; int nSlot; if( objc!=3 ){ Tcl_WrongNumArgs(interp, 1, objv, "NSLOT TOKEN"); return TCL_ERROR; } if( Tcl_GetIntFromObj(interp, objv[1], &nSlot) ){ return TCL_ERROR; } z = Tcl_GetStringFromObj(objv[2], &n); iVal = f5t_fts5HashKey(nSlot, z, (int)n); Tcl_SetObjResult(interp, Tcl_NewIntObj(iVal)); return TCL_OK; } static int SQLITE_TCLAPI f5tRegisterMatchinfo( void * clientData, Tcl_Interp *interp, |
︙ | ︙ | |||
1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 | rc = sqlite3Fts5TestRegisterTok(db, pApi); if( rc!=SQLITE_OK ){ Tcl_SetResult(interp, (char*)sqlite3ErrName(rc), TCL_VOLATILE); return TCL_ERROR; } return TCL_OK; } /* ** Entry point. */ int Fts5tcl_Init(Tcl_Interp *interp){ static struct Cmd { char *zName; Tcl_ObjCmdProc *xProc; int bTokenizeCtx; } aCmd[] = { { "sqlite3_fts5_create_tokenizer", f5tCreateTokenizer, 1 }, { "sqlite3_fts5_token", f5tTokenizerReturn, 1 }, { "sqlite3_fts5_tokenize", f5tTokenize, 0 }, { "sqlite3_fts5_create_function", f5tCreateFunction, 0 }, { "sqlite3_fts5_may_be_corrupt", f5tMayBeCorrupt, 0 }, { "sqlite3_fts5_token_hash", f5tTokenHash, 0 }, { "sqlite3_fts5_register_matchinfo", f5tRegisterMatchinfo, 0 }, | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > | 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 | rc = sqlite3Fts5TestRegisterTok(db, pApi); if( rc!=SQLITE_OK ){ Tcl_SetResult(interp, (char*)sqlite3ErrName(rc), TCL_VOLATILE); return TCL_ERROR; } return TCL_OK; } typedef struct OriginTextCtx OriginTextCtx; struct OriginTextCtx { sqlite3 *db; fts5_api *pApi; }; typedef struct OriginTextTokenizer OriginTextTokenizer; struct OriginTextTokenizer { Fts5Tokenizer *pTok; /* Underlying tokenizer object */ fts5_tokenizer tokapi; /* API implementation for pTok */ }; /* ** Delete the OriginTextCtx object indicated by the only argument. */ static void f5tOrigintextTokenizerDelete(void *pCtx){ OriginTextCtx *p = (OriginTextCtx*)pCtx; ckfree((char*)p); } static int f5tOrigintextCreate( void *pCtx, const char **azArg, int nArg, Fts5Tokenizer **ppOut ){ OriginTextCtx *p = (OriginTextCtx*)pCtx; OriginTextTokenizer *pTok = 0; void *pTokCtx = 0; int rc = SQLITE_OK; pTok = (OriginTextTokenizer*)sqlite3_malloc(sizeof(OriginTextTokenizer)); if( pTok==0 ){ rc = SQLITE_NOMEM; }else if( nArg<1 ){ rc = SQLITE_ERROR; }else{ /* Locate the underlying tokenizer */ rc = p->pApi->xFindTokenizer(p->pApi, azArg[0], &pTokCtx, &pTok->tokapi); } /* Create the new tokenizer instance */ if( rc==SQLITE_OK ){ rc = pTok->tokapi.xCreate(pTokCtx, &azArg[1], nArg-1, &pTok->pTok); } if( rc!=SQLITE_OK ){ sqlite3_free(pTok); pTok = 0; } *ppOut = (Fts5Tokenizer*)pTok; return rc; } static void f5tOrigintextDelete(Fts5Tokenizer *pTokenizer){ OriginTextTokenizer *p = (OriginTextTokenizer*)pTokenizer; if( p->pTok ){ p->tokapi.xDelete(p->pTok); } sqlite3_free(p); } typedef struct OriginTextCb OriginTextCb; struct OriginTextCb { void *pCtx; const char *pText; int nText; int (*xToken)(void *, int, const char *, int, int, int); char *aBuf; /* Buffer to use */ int nBuf; /* Allocated size of aBuf[] */ }; static int xOriginToken( void *pCtx, /* Copy of 2nd argument to xTokenize() */ int tflags, /* Mask of FTS5_TOKEN_* flags */ const char *pToken, /* Pointer to buffer containing token */ int nToken, /* Size of token in bytes */ int iStart, /* Byte offset of token within input text */ int iEnd /* Byte offset of end of token within input */ ){ OriginTextCb *p = (OriginTextCb*)pCtx; int ret = 0; if( nToken==(iEnd-iStart) && 0==memcmp(pToken, &p->pText[iStart], nToken) ){ /* Token exactly matches document text. Pass it through as is. */ ret = p->xToken(p->pCtx, tflags, pToken, nToken, iStart, iEnd); }else{ int nReq = nToken + 1 + (iEnd-iStart); if( nReq>p->nBuf ){ sqlite3_free(p->aBuf); p->aBuf = sqlite3_malloc(nReq*2); if( p->aBuf==0 ) return SQLITE_NOMEM; p->nBuf = nReq*2; } memcpy(p->aBuf, pToken, nToken); p->aBuf[nToken] = '\0'; memcpy(&p->aBuf[nToken+1], &p->pText[iStart], iEnd-iStart); ret = p->xToken(p->pCtx, tflags, p->aBuf, nReq, iStart, iEnd); } return ret; } static int f5tOrigintextTokenize( Fts5Tokenizer *pTokenizer, void *pCtx, int flags, /* Mask of FTS5_TOKENIZE_* flags */ const char *pText, int nText, int (*xToken)(void *, int, const char *, int, int, int) ){ OriginTextTokenizer *p = (OriginTextTokenizer*)pTokenizer; OriginTextCb cb; int ret; memset(&cb, 0, sizeof(cb)); cb.pCtx = pCtx; cb.pText = pText; cb.nText = nText; cb.xToken = xToken; ret = p->tokapi.xTokenize(p->pTok,(void*)&cb,flags,pText,nText,xOriginToken); sqlite3_free(cb.aBuf); return ret; } /* ** sqlite3_fts5_register_origintext DB ** ** Description... */ static int SQLITE_TCLAPI f5tRegisterOriginText( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3 *db = 0; fts5_api *pApi = 0; int rc; fts5_tokenizer tok = {0, 0, 0}; OriginTextCtx *pCtx = 0; if( objc!=2 ){ Tcl_WrongNumArgs(interp, 1, objv, "DB"); return TCL_ERROR; } if( f5tDbAndApi(interp, objv[1], &db, &pApi) ) return TCL_ERROR; pCtx = (OriginTextCtx*)ckalloc(sizeof(OriginTextCtx)); pCtx->db = db; pCtx->pApi = pApi; tok.xCreate = f5tOrigintextCreate; tok.xDelete = f5tOrigintextDelete; tok.xTokenize = f5tOrigintextTokenize; rc = pApi->xCreateTokenizer( pApi, "origintext", (void*)pCtx, &tok, f5tOrigintextTokenizerDelete ); Tcl_ResetResult(interp); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (void*)0); return TCL_ERROR; } return TCL_OK; } /* ** This function is used to DROP an fts5 table. It works even if the data ** structures fts5 stores within the database are corrupt, which sometimes ** prevents a straight "DROP TABLE" command from succeeding. ** ** The first parameter is the database handle to use for the DROP TABLE ** operation. The second is the name of the database to drop the fts5 table ** from (i.e. "main", "temp" or the name of an attached database). The ** third parameter is the name of the fts5 table to drop. ** ** SQLITE_OK is returned if the table is successfully dropped. Or, if an ** error occurs, an SQLite error code. */ static int sqlite3_fts5_drop_corrupt_table( sqlite3 *db, /* Database handle */ const char *zDb, /* Database name ("main", "temp" etc.) */ const char *zTab /* Name of fts5 table to drop */ ){ int rc = SQLITE_OK; int bDef = 0; rc = sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, -1, &bDef); if( rc==SQLITE_OK ){ char *zScript = sqlite3_mprintf( "DELETE FROM %Q.'%q_data';" "DELETE FROM %Q.'%q_config';" "INSERT INTO %Q.'%q_data' VALUES(10, X'0000000000');" "INSERT INTO %Q.'%q_config' VALUES('version', 4);" "DROP TABLE %Q.'%q';", zDb, zTab, zDb, zTab, zDb, zTab, zDb, zTab, zDb, zTab ); if( zScript==0 ){ rc = SQLITE_NOMEM; }else{ if( bDef ) sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 0, 0); rc = sqlite3_exec(db, zScript, 0, 0, 0); if( bDef ) sqlite3_db_config(db, SQLITE_DBCONFIG_DEFENSIVE, 1, 0); sqlite3_free(zScript); } } return rc; } /* ** sqlite3_fts5_drop_corrupt_table DB DATABASE TABLE ** ** Description... */ static int SQLITE_TCLAPI f5tDropCorruptTable( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3 *db = 0; const char *zDb = 0; const char *zTab = 0; int rc = SQLITE_OK; if( objc!=4 ){ Tcl_WrongNumArgs(interp, 1, objv, "DB DATABASE TABLE"); return TCL_ERROR; } if( f5tDbPointer(interp, objv[1], &db) ){ return TCL_ERROR; } zDb = Tcl_GetString(objv[2]); zTab = Tcl_GetString(objv[3]); rc = sqlite3_fts5_drop_corrupt_table(db, zDb, zTab); if( rc!=SQLITE_OK ){ Tcl_AppendResult(interp, "error: ", sqlite3_errmsg(db), (void*)0); return TCL_ERROR; } return TCL_OK; } /* ** Free a buffer returned to SQLite by the str() function. */ void f5tFree(void *p){ char *x = (char *)p; ckfree(&x[-8]); } /* ** Implementation of str(). */ void f5tStrFunc(sqlite3_context *pCtx, int nArg, sqlite3_value **apArg){ const char *zText = 0; assert( nArg==1 ); zText = (const char*)sqlite3_value_text(apArg[0]); if( zText ){ sqlite3_int64 nText = strlen(zText); char *zCopy = (char*)ckalloc(nText+8); if( zCopy==0 ){ sqlite3_result_error_nomem(pCtx); }else{ zCopy += 8; memcpy(zCopy, zText, nText); sqlite3_result_text64(pCtx, zCopy, nText, f5tFree, SQLITE_UTF8); } } } /* ** sqlite3_fts5_register_str DB ** ** Register the str() function with database handle DB. str() interprets ** its only argument as text and returns a copy of the value in a ** non-nul-terminated buffer. */ static int SQLITE_TCLAPI f5tRegisterStr( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3 *db = 0; if( objc!=2 ){ Tcl_WrongNumArgs(interp, 1, objv, "DB"); return TCL_ERROR; } if( f5tDbPointer(interp, objv[1], &db) ){ return TCL_ERROR; } sqlite3_create_function(db, "str", 1, SQLITE_UTF8, 0, f5tStrFunc, 0, 0); return TCL_OK; } /* ** Entry point. */ int Fts5tcl_Init(Tcl_Interp *interp){ static struct Cmd { char *zName; Tcl_ObjCmdProc *xProc; int bTokenizeCtx; } aCmd[] = { { "sqlite3_fts5_create_tokenizer", f5tCreateTokenizer, 1 }, { "sqlite3_fts5_token", f5tTokenizerReturn, 1 }, { "sqlite3_fts5_locale", f5tTokenizerLocale, 1 }, { "sqlite3_fts5_tokenize", f5tTokenize, 0 }, { "sqlite3_fts5_create_function", f5tCreateFunction, 0 }, { "sqlite3_fts5_may_be_corrupt", f5tMayBeCorrupt, 0 }, { "sqlite3_fts5_token_hash", f5tTokenHash, 0 }, { "sqlite3_fts5_register_matchinfo", f5tRegisterMatchinfo, 0 }, { "sqlite3_fts5_register_fts5tokenize", f5tRegisterTok, 0 }, { "sqlite3_fts5_register_origintext",f5tRegisterOriginText, 0 }, { "sqlite3_fts5_drop_corrupt_table", f5tDropCorruptTable, 0 }, { "sqlite3_fts5_register_str", f5tRegisterStr, 0 } }; int i; F5tTokenizerContext *pContext; pContext = (F5tTokenizerContext*)ckalloc(sizeof(F5tTokenizerContext)); memset(pContext, 0, sizeof(*pContext)); |
︙ | ︙ |
Changes to ext/fts5/fts5_test_mi.c.
︙ | ︙ | |||
10 11 12 13 14 15 16 | ** ****************************************************************************** ** ** This file contains test code only, it is not included in release ** versions of FTS5. It contains the implementation of an FTS5 auxiliary ** function very similar to the FTS4 function matchinfo(): ** | | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | ** ****************************************************************************** ** ** This file contains test code only, it is not included in release ** versions of FTS5. It contains the implementation of an FTS5 auxiliary ** function very similar to the FTS4 function matchinfo(): ** ** https://sqlite.org/fts3.html#matchinfo ** ** Known differences are that: ** ** 1) this function uses the FTS5 definition of "matchable phrase", which ** excludes any phrases that are part of an expression sub-tree that ** does not match the current row. This comes up for MATCH queries ** such as: |
︙ | ︙ | |||
389 390 391 392 393 394 395 | }else{ /* No errors has occured, so return a copy of the array of integers. */ int nByte = p->nRet * sizeof(u32); sqlite3_result_blob(pCtx, (void*)p->aRet, nByte, SQLITE_TRANSIENT); } } | > > > | | < < < < < < < | > > > > > > > > > > > > > > > > | 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 | }else{ /* No errors has occured, so return a copy of the array of integers. */ int nByte = p->nRet * sizeof(u32); sqlite3_result_blob(pCtx, (void*)p->aRet, nByte, SQLITE_TRANSIENT); } } /* ** Register "matchinfo" with global API object pApi. */ int sqlite3Fts5TestRegisterMatchinfoAPI(fts5_api *pApi){ int rc; /* If fts5_api_from_db() returned NULL, then either FTS5 is not registered ** with this database handle, or an error (OOM perhaps?) has occurred. ** ** Also check that the fts5_api object is version 2 or newer. */ if( pApi==0 || pApi->iVersion<2 ){ return SQLITE_ERROR; } /* Register the implementation of matchinfo() */ rc = pApi->xCreateFunction(pApi, "matchinfo", 0, fts5MatchinfoFunc, 0); return rc; } /* ** Register "matchinfo" with database handle db. */ int sqlite3Fts5TestRegisterMatchinfo(sqlite3 *db){ int rc; /* Return code */ fts5_api *pApi; /* FTS5 API functions */ /* Extract the FTS5 API pointer from the database handle. The ** fts5_api_from_db() function above is copied verbatim from the ** FTS5 documentation. Refer there for details. */ rc = fts5_api_from_db(db, &pApi); if( rc!=SQLITE_OK ) return rc; return sqlite3Fts5TestRegisterMatchinfoAPI(pApi); } #endif /* SQLITE_ENABLE_FTS5 */ |
Changes to ext/fts5/fts5_test_tok.c.
︙ | ︙ | |||
468 469 470 471 472 473 474 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ | | > | 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 | 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0 /* xIntegrity */ }; int rc; /* Return code */ rc = sqlite3_create_module(db, "fts5tokenize", &fts5tok_module, (void*)pApi); return rc; } |
︙ | ︙ |
Changes to ext/fts5/fts5_tokenize.c.
︙ | ︙ | |||
194 195 196 197 198 199 200 | 0x00, 0x01, 0x02, 0x03, 0x00, 0x01, 0x00, 0x00, }; #define READ_UTF8(zIn, zTerm, c) \ c = *(zIn++); \ if( c>=0xc0 ){ \ c = sqlite3Utf8Trans1[c-0xc0]; \ | | | 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 | 0x00, 0x01, 0x02, 0x03, 0x00, 0x01, 0x00, 0x00, }; #define READ_UTF8(zIn, zTerm, c) \ c = *(zIn++); \ if( c>=0xc0 ){ \ c = sqlite3Utf8Trans1[c-0xc0]; \ while( zIn<zTerm && (*zIn & 0xc0)==0x80 ){ \ c = (c<<6) + (0x3f & *(zIn++)); \ } \ if( c<0x80 \ || (c&0xFFFFF800)==0xD800 \ || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \ } |
︙ | ︙ | |||
224 225 226 227 228 229 230 231 232 233 234 235 236 237 | *zOut++ = 0x80 + (unsigned char)((c>>12) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)((c>>6) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)(c & 0x3F); \ } \ } #endif /* ifndef SQLITE_AMALGAMATION */ typedef struct Unicode61Tokenizer Unicode61Tokenizer; struct Unicode61Tokenizer { unsigned char aTokenChar[128]; /* ASCII range token characters */ char *aFold; /* Buffer to fold text into */ int nFold; /* Size of aFold[] in bytes */ int eRemoveDiacritic; /* True if remove_diacritics=1 is set */ | > > > > > > | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 | *zOut++ = 0x80 + (unsigned char)((c>>12) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)((c>>6) & 0x3F); \ *zOut++ = 0x80 + (unsigned char)(c & 0x3F); \ } \ } #endif /* ifndef SQLITE_AMALGAMATION */ #define FTS5_SKIP_UTF8(zIn) { \ if( ((unsigned char)(*(zIn++)))>=0xc0 ){ \ while( (((unsigned char)*zIn) & 0xc0)==0x80 ){ zIn++; } \ } \ } typedef struct Unicode61Tokenizer Unicode61Tokenizer; struct Unicode61Tokenizer { unsigned char aTokenChar[128]; /* ASCII range token characters */ char *aFold; /* Buffer to fold text into */ int nFold; /* Size of aFold[] in bytes */ int eRemoveDiacritic; /* True if remove_diacritics=1 is set */ |
︙ | ︙ | |||
376 377 378 379 380 381 382 | /* Search for a "categories" argument */ for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ if( 0==sqlite3_stricmp(azArg[i], "categories") ){ zCat = azArg[i+1]; } } | < | 382 383 384 385 386 387 388 389 390 391 392 393 394 395 | /* Search for a "categories" argument */ for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ if( 0==sqlite3_stricmp(azArg[i], "categories") ){ zCat = azArg[i+1]; } } if( rc==SQLITE_OK ){ rc = unicodeSetCategories(p, zCat); } for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ const char *zArg = azArg[i+1]; if( 0==sqlite3_stricmp(azArg[i], "remove_diacritics") ){ |
︙ | ︙ | |||
406 407 408 409 410 411 412 | }else if( 0==sqlite3_stricmp(azArg[i], "categories") ){ /* no-op */ }else{ rc = SQLITE_ERROR; } } | < | 411 412 413 414 415 416 417 418 419 420 421 422 423 424 | }else if( 0==sqlite3_stricmp(azArg[i], "categories") ){ /* no-op */ }else{ rc = SQLITE_ERROR; } } }else{ rc = SQLITE_NOMEM; } if( rc!=SQLITE_OK ){ fts5UnicodeDelete((Fts5Tokenizer*)p); p = 0; } |
︙ | ︙ | |||
545 546 547 548 549 550 551 | /* Any tokens larger than this (in bytes) are passed through without ** stemming. */ #define FTS5_PORTER_MAX_TOKEN 64 typedef struct PorterTokenizer PorterTokenizer; struct PorterTokenizer { | | | > | | > | | 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 | /* Any tokens larger than this (in bytes) are passed through without ** stemming. */ #define FTS5_PORTER_MAX_TOKEN 64 typedef struct PorterTokenizer PorterTokenizer; struct PorterTokenizer { fts5_tokenizer_v2 tokenizer_v2; /* Parent tokenizer module */ Fts5Tokenizer *pTokenizer; /* Parent tokenizer instance */ char aBuf[FTS5_PORTER_MAX_TOKEN + 64]; }; /* ** Delete a "porter" tokenizer. */ static void fts5PorterDelete(Fts5Tokenizer *pTok){ if( pTok ){ PorterTokenizer *p = (PorterTokenizer*)pTok; if( p->pTokenizer ){ p->tokenizer_v2.xDelete(p->pTokenizer); } sqlite3_free(p); } } /* ** Create a "porter" tokenizer. */ static int fts5PorterCreate( void *pCtx, const char **azArg, int nArg, Fts5Tokenizer **ppOut ){ fts5_api *pApi = (fts5_api*)pCtx; int rc = SQLITE_OK; PorterTokenizer *pRet; void *pUserdata = 0; const char *zBase = "unicode61"; fts5_tokenizer_v2 *pV2 = 0; if( nArg>0 ){ zBase = azArg[0]; } pRet = (PorterTokenizer*)sqlite3_malloc(sizeof(PorterTokenizer)); if( pRet ){ memset(pRet, 0, sizeof(PorterTokenizer)); rc = pApi->xFindTokenizer_v2(pApi, zBase, &pUserdata, &pV2); }else{ rc = SQLITE_NOMEM; } if( rc==SQLITE_OK ){ int nArg2 = (nArg>0 ? nArg-1 : 0); const char **az2 = (nArg2 ? &azArg[1] : 0); memcpy(&pRet->tokenizer_v2, pV2, sizeof(fts5_tokenizer_v2)); rc = pRet->tokenizer_v2.xCreate(pUserdata, az2, nArg2, &pRet->pTokenizer); } if( rc!=SQLITE_OK ){ fts5PorterDelete((Fts5Tokenizer*)pRet); pRet = 0; } *ppOut = (Fts5Tokenizer*)pRet; |
︙ | ︙ | |||
1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 | ** Tokenize using the porter tokenizer. */ static int fts5PorterTokenize( Fts5Tokenizer *pTokenizer, void *pCtx, int flags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int nToken, int iStart, int iEnd) ){ PorterTokenizer *p = (PorterTokenizer*)pTokenizer; PorterContext sCtx; sCtx.xToken = xToken; sCtx.pCtx = pCtx; sCtx.aBuf = p->aBuf; | > | | > | | | > > > > | > > | | | | | | | | > > > | > > > | | | > > > > > | | | > > > | | > > > > > > > > > > > > > > > > > > > > > > | | | | < < | | < < < | < | | < < < < | | < < | > | | | > > > > > > > > > > > > | 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 | ** Tokenize using the porter tokenizer. */ static int fts5PorterTokenize( Fts5Tokenizer *pTokenizer, void *pCtx, int flags, const char *pText, int nText, const char *pLoc, int nLoc, int (*xToken)(void*, int, const char*, int nToken, int iStart, int iEnd) ){ PorterTokenizer *p = (PorterTokenizer*)pTokenizer; PorterContext sCtx; sCtx.xToken = xToken; sCtx.pCtx = pCtx; sCtx.aBuf = p->aBuf; return p->tokenizer_v2.xTokenize( p->pTokenizer, (void*)&sCtx, flags, pText, nText, pLoc, nLoc, fts5PorterCb ); } /************************************************************************** ** Start of trigram implementation. */ typedef struct TrigramTokenizer TrigramTokenizer; struct TrigramTokenizer { int bFold; /* True to fold to lower-case */ int iFoldParam; /* Parameter to pass to Fts5UnicodeFold() */ }; /* ** Free a trigram tokenizer. */ static void fts5TriDelete(Fts5Tokenizer *p){ sqlite3_free(p); } /* ** Allocate a trigram tokenizer. */ static int fts5TriCreate( void *pUnused, const char **azArg, int nArg, Fts5Tokenizer **ppOut ){ int rc = SQLITE_OK; TrigramTokenizer *pNew = 0; UNUSED_PARAM(pUnused); if( nArg%2 ){ rc = SQLITE_ERROR; }else{ int i; pNew = (TrigramTokenizer*)sqlite3_malloc(sizeof(*pNew)); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ pNew->bFold = 1; pNew->iFoldParam = 0; for(i=0; rc==SQLITE_OK && i<nArg; i+=2){ const char *zArg = azArg[i+1]; if( 0==sqlite3_stricmp(azArg[i], "case_sensitive") ){ if( (zArg[0]!='0' && zArg[0]!='1') || zArg[1] ){ rc = SQLITE_ERROR; }else{ pNew->bFold = (zArg[0]=='0'); } }else if( 0==sqlite3_stricmp(azArg[i], "remove_diacritics") ){ if( (zArg[0]!='0' && zArg[0]!='1' && zArg[0]!='2') || zArg[1] ){ rc = SQLITE_ERROR; }else{ pNew->iFoldParam = (zArg[0]!='0') ? 2 : 0; } }else{ rc = SQLITE_ERROR; } } if( pNew->iFoldParam!=0 && pNew->bFold==0 ){ rc = SQLITE_ERROR; } if( rc!=SQLITE_OK ){ fts5TriDelete((Fts5Tokenizer*)pNew); pNew = 0; } } } *ppOut = (Fts5Tokenizer*)pNew; return rc; } /* ** Trigram tokenizer tokenize routine. */ static int fts5TriTokenize( Fts5Tokenizer *pTok, void *pCtx, int unusedFlags, const char *pText, int nText, int (*xToken)(void*, int, const char*, int, int, int) ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; int rc = SQLITE_OK; char aBuf[32]; char *zOut = aBuf; int ii; const unsigned char *zIn = (const unsigned char*)pText; const unsigned char *zEof = (zIn ? &zIn[nText] : 0); u32 iCode = 0; int aStart[3]; /* Input offset of each character in aBuf[] */ UNUSED_PARAM(unusedFlags); /* Populate aBuf[] with the characters for the first trigram. */ for(ii=0; ii<3; ii++){ do { aStart[ii] = zIn - (const unsigned char*)pText; if( zIn>=zEof ) return SQLITE_OK; READ_UTF8(zIn, zEof, iCode); if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam); }while( iCode==0 ); WRITE_UTF8(zOut, iCode); } /* At the start of each iteration of this loop: ** ** aBuf: Contains 3 characters. The 3 characters of the next trigram. ** zOut: Points to the byte following the last character in aBuf. ** aStart[3]: Contains the byte offset in the input text corresponding ** to the start of each of the three characters in the buffer. */ assert( zIn<=zEof ); while( 1 ){ int iNext; /* Start of character following current tri */ const char *z1; /* Read characters from the input up until the first non-diacritic */ do { iNext = zIn - (const unsigned char*)pText; if( zIn>=zEof ){ iCode = 0; break; } READ_UTF8(zIn, zEof, iCode); if( p->bFold ) iCode = sqlite3Fts5UnicodeFold(iCode, p->iFoldParam); }while( iCode==0 ); /* Pass the current trigram back to fts5 */ rc = xToken(pCtx, 0, aBuf, zOut-aBuf, aStart[0], iNext); if( iCode==0 || rc!=SQLITE_OK ) break; /* Remove the first character from buffer aBuf[]. Append the character ** with codepoint iCode. */ z1 = aBuf; FTS5_SKIP_UTF8(z1); memmove(aBuf, z1, zOut - z1); zOut -= (z1 - aBuf); WRITE_UTF8(zOut, iCode); /* Update the aStart[] array */ aStart[0] = aStart[1]; aStart[1] = aStart[2]; aStart[2] = iNext; } return rc; } /* ** Argument xCreate is a pointer to a constructor function for a tokenizer. |
︙ | ︙ | |||
1376 1377 1378 1379 1380 1381 1382 | */ int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ){ if( xCreate==fts5TriCreate ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; | > | > > > > > > > > > > > < > > > > > > > > > > > | > > | 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 | */ int sqlite3Fts5TokenizerPattern( int (*xCreate)(void*, const char**, int, Fts5Tokenizer**), Fts5Tokenizer *pTok ){ if( xCreate==fts5TriCreate ){ TrigramTokenizer *p = (TrigramTokenizer*)pTok; if( p->iFoldParam==0 ){ return p->bFold ? FTS5_PATTERN_LIKE : FTS5_PATTERN_GLOB; } } return FTS5_PATTERN_NONE; } /* ** Return true if the tokenizer described by p->azArg[] is the trigram ** tokenizer. This tokenizer needs to be loaded before xBestIndex is ** called for the first time in order to correctly handle LIKE/GLOB. */ int sqlite3Fts5TokenizerPreload(Fts5TokenizerConfig *p){ return (p->nArg>=1 && 0==sqlite3_stricmp(p->azArg[0], "trigram")); } /* ** Register all built-in tokenizers with FTS5. */ int sqlite3Fts5TokenizerInit(fts5_api *pApi){ struct BuiltinTokenizer { const char *zName; fts5_tokenizer x; } aBuiltin[] = { { "unicode61", {fts5UnicodeCreate, fts5UnicodeDelete, fts5UnicodeTokenize}}, { "ascii", {fts5AsciiCreate, fts5AsciiDelete, fts5AsciiTokenize }}, { "trigram", {fts5TriCreate, fts5TriDelete, fts5TriTokenize}}, }; int rc = SQLITE_OK; /* Return code */ int i; /* To iterate through builtin functions */ for(i=0; rc==SQLITE_OK && i<ArraySize(aBuiltin); i++){ rc = pApi->xCreateTokenizer(pApi, aBuiltin[i].zName, (void*)pApi, &aBuiltin[i].x, 0 ); } if( rc==SQLITE_OK ){ fts5_tokenizer_v2 sPorter = { 2, fts5PorterCreate, fts5PorterDelete, fts5PorterTokenize }; rc = pApi->xCreateTokenizer_v2(pApi, "porter", (void*)pApi, &sPorter, 0 ); } return rc; } |
Changes to ext/fts5/fts5_unicode2.c.
︙ | ︙ | |||
360 361 362 363 364 365 366 367 368 369 370 371 372 373 | aArray[27] = 1; aArray[28] = 1; aArray[29] = 1; break; default: return 1; } break; } return 0; } static u16 aFts5UnicodeBlock[] = { 0, 1471, 1753, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1763, 1765, | > > > | 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 | aArray[27] = 1; aArray[28] = 1; aArray[29] = 1; break; default: return 1; } break; default: return 1; } return 0; } static u16 aFts5UnicodeBlock[] = { 0, 1471, 1753, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1760, 1763, 1765, |
︙ | ︙ | |||
771 772 773 774 775 776 777 | for(; i<128 && i<n; i++){ aAscii[i] = (u8)bToken; } iTbl++; } aAscii[0] = 0; /* 0x00 is never a token character */ } | < | 774 775 776 777 778 779 780 | for(; i<128 && i<n; i++){ aAscii[i] = (u8)bToken; } iTbl++; } aAscii[0] = 0; /* 0x00 is never a token character */ } |
Changes to ext/fts5/fts5_vocab.c.
︙ | ︙ | |||
60 61 62 63 64 65 66 67 68 69 70 71 72 73 | int bEof; /* True if this cursor is at EOF */ Fts5IndexIter *pIter; /* Term/rowid iterator object */ void *pStruct; /* From sqlite3Fts5StructureRef() */ int nLeTerm; /* Size of zLeTerm in bytes */ char *zLeTerm; /* (term <= $zLeTerm) paramater, or NULL */ /* These are used by 'col' tables only */ int iCol; i64 *aCnt; i64 *aDoc; /* Output values used by all tables. */ | > | 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 | int bEof; /* True if this cursor is at EOF */ Fts5IndexIter *pIter; /* Term/rowid iterator object */ void *pStruct; /* From sqlite3Fts5StructureRef() */ int nLeTerm; /* Size of zLeTerm in bytes */ char *zLeTerm; /* (term <= $zLeTerm) paramater, or NULL */ int colUsed; /* Copy of sqlite3_index_info.colUsed */ /* These are used by 'col' tables only */ int iCol; i64 *aCnt; i64 *aDoc; /* Output values used by all tables. */ |
︙ | ︙ | |||
86 87 88 89 90 91 92 | #define FTS5_VOCAB_COL_SCHEMA "term, col, doc, cnt" #define FTS5_VOCAB_ROW_SCHEMA "term, doc, cnt" #define FTS5_VOCAB_INST_SCHEMA "term, doc, col, offset" /* ** Bits for the mask used as the idxNum value by xBestIndex/xFilter. */ | | | | > > | 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | #define FTS5_VOCAB_COL_SCHEMA "term, col, doc, cnt" #define FTS5_VOCAB_ROW_SCHEMA "term, doc, cnt" #define FTS5_VOCAB_INST_SCHEMA "term, doc, col, offset" /* ** Bits for the mask used as the idxNum value by xBestIndex/xFilter. */ #define FTS5_VOCAB_TERM_EQ 0x0100 #define FTS5_VOCAB_TERM_GE 0x0200 #define FTS5_VOCAB_TERM_LE 0x0400 #define FTS5_VOCAB_COLUSED_MASK 0xFF /* ** Translate a string containing an fts5vocab table type to an ** FTS5_VOCAB_XXX constant. If successful, set *peType to the output ** value and return SQLITE_OK. Otherwise, set *pzErr to an error message ** and return SQLITE_ERROR. |
︙ | ︙ | |||
186 187 188 189 190 191 192 | bDb = (argc==6 && strlen(argv[1])==4 && memcmp("temp", argv[1], 4)==0); if( argc!=5 && bDb==0 ){ *pzErr = sqlite3_mprintf("wrong number of vtable arguments"); rc = SQLITE_ERROR; }else{ | | | | | 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 | bDb = (argc==6 && strlen(argv[1])==4 && memcmp("temp", argv[1], 4)==0); if( argc!=5 && bDb==0 ){ *pzErr = sqlite3_mprintf("wrong number of vtable arguments"); rc = SQLITE_ERROR; }else{ i64 nByte; /* Bytes of space to allocate */ const char *zDb = bDb ? argv[3] : argv[1]; const char *zTab = bDb ? argv[4] : argv[3]; const char *zType = bDb ? argv[5] : argv[4]; i64 nDb = strlen(zDb)+1; i64 nTab = strlen(zTab)+1; int eType = 0; rc = fts5VocabTableType(zType, pzErr, &eType); if( rc==SQLITE_OK ){ assert( eType>=0 && eType<ArraySize(azSchema) ); rc = sqlite3_declare_vtab(db, azSchema[eType]); } |
︙ | ︙ | |||
265 266 267 268 269 270 271 | sqlite3_vtab *pUnused, sqlite3_index_info *pInfo ){ int i; int iTermEq = -1; int iTermGe = -1; int iTermLe = -1; | | > > | 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 | sqlite3_vtab *pUnused, sqlite3_index_info *pInfo ){ int i; int iTermEq = -1; int iTermGe = -1; int iTermLe = -1; int idxNum = (int)pInfo->colUsed; int nArg = 0; UNUSED_PARAM(pUnused); assert( (pInfo->colUsed & FTS5_VOCAB_COLUSED_MASK)==pInfo->colUsed ); for(i=0; i<pInfo->nConstraint; i++){ struct sqlite3_index_constraint *p = &pInfo->aConstraint[i]; if( p->usable==0 ) continue; if( p->iColumn==0 ){ /* term column */ if( p->op==SQLITE_INDEX_CONSTRAINT_EQ ) iTermEq = i; if( p->op==SQLITE_INDEX_CONSTRAINT_LE ) iTermLe = i; |
︙ | ︙ | |||
361 362 363 364 365 366 367 | if( rc==SQLITE_OK ){ if( pFts5==0 ){ rc = sqlite3_finalize(pStmt); pStmt = 0; if( rc==SQLITE_OK ){ pVTab->zErrMsg = sqlite3_mprintf( "no such fts5 table: %s.%s", pTab->zFts5Db, pTab->zFts5Tbl | | | 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 | if( rc==SQLITE_OK ){ if( pFts5==0 ){ rc = sqlite3_finalize(pStmt); pStmt = 0; if( rc==SQLITE_OK ){ pVTab->zErrMsg = sqlite3_mprintf( "no such fts5 table: %s.%s", pTab->zFts5Db, pTab->zFts5Tbl ); rc = SQLITE_ERROR; } }else{ rc = sqlite3Fts5FlushToDisk(pFts5); } } |
︙ | ︙ | |||
521 522 523 524 525 526 527 | int iOff = 0; /* Current offset within position list */ pPos = pCsr->pIter->pData; nPos = pCsr->pIter->nData; switch( pTab->eType ){ case FTS5_VOCAB_ROW: | > > | | > > > > > > > | > | 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 | int iOff = 0; /* Current offset within position list */ pPos = pCsr->pIter->pData; nPos = pCsr->pIter->nData; switch( pTab->eType ){ case FTS5_VOCAB_ROW: /* Do not bother counting the number of instances if the "cnt" ** column is not being read (according to colUsed). */ if( eDetail==FTS5_DETAIL_FULL && (pCsr->colUsed & 0x04) ){ while( iPos<nPos ){ u32 ii; fts5FastGetVarint32(pPos, iPos, ii); if( ii==1 ){ /* New column in the position list */ fts5FastGetVarint32(pPos, iPos, ii); }else{ /* An instance - increment pCsr->aCnt[] */ pCsr->aCnt[0]++; } } } pCsr->aDoc[0]++; break; case FTS5_VOCAB_COL: if( eDetail==FTS5_DETAIL_FULL ){ |
︙ | ︙ | |||
621 622 623 624 625 626 627 628 629 630 631 | UNUSED_PARAM2(zUnused, nUnused); fts5VocabResetCursor(pCsr); if( idxNum & FTS5_VOCAB_TERM_EQ ) pEq = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_GE ) pGe = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_LE ) pLe = apVal[iVal++]; if( pEq ){ zTerm = (const char *)sqlite3_value_text(pEq); nTerm = sqlite3_value_bytes(pEq); | > | | 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 | UNUSED_PARAM2(zUnused, nUnused); fts5VocabResetCursor(pCsr); if( idxNum & FTS5_VOCAB_TERM_EQ ) pEq = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_GE ) pGe = apVal[iVal++]; if( idxNum & FTS5_VOCAB_TERM_LE ) pLe = apVal[iVal++]; pCsr->colUsed = (idxNum & FTS5_VOCAB_COLUSED_MASK); if( pEq ){ zTerm = (const char *)sqlite3_value_text(pEq); nTerm = sqlite3_value_bytes(pEq); f = FTS5INDEX_QUERY_NOTOKENDATA; }else{ if( pGe ){ zTerm = (const char *)sqlite3_value_text(pGe); nTerm = sqlite3_value_bytes(pGe); } if( pLe ){ const char *zCopy = (const char *)sqlite3_value_text(pLe); |
︙ | ︙ | |||
779 780 781 782 783 784 785 | /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, | | > | 795 796 797 798 799 800 801 802 803 804 805 806 807 808 | /* xCommit */ 0, /* xRollback */ 0, /* xFindFunction */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ 0, /* xShadowName */ 0, /* xIntegrity */ 0 }; void *p = (void*)pGlobal; return sqlite3_create_module_v2(db, "fts5vocab", &fts5Vocab, p, 0); } |
Changes to ext/fts5/test/fts5_common.tcl.
︙ | ︙ | |||
46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | lappend res $i.$c.$o } } #set res sort_poslist $res } proc fts5_test_collist {cmd} { set res [list] for {set i 0} {$i < [$cmd xPhraseCount]} {incr i} { $cmd xPhraseColumnForeach $i c { lappend res $i.$c } } set res } proc fts5_test_columnsize {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnSize $i] } set res } proc fts5_test_columntext {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnText $i] } set res } proc fts5_test_columntotalsize {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnTotalSize $i] } set res | > > > > > > > > > > > > > > > > > > > > > > > > > | 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | lappend res $i.$c.$o } } #set res sort_poslist $res } proc fts5_test_insttoken {cmd iInst iToken} { $cmd xInstToken $iInst $iToken } proc fts5_test_collist {cmd} { set res [list] for {set i 0} {$i < [$cmd xPhraseCount]} {incr i} { $cmd xPhraseColumnForeach $i c { lappend res $i.$c } } set res } proc fts5_collist {cmd iPhrase} { set res [list] $cmd xPhraseColumnForeach $iPhrase c { lappend res $c } set res } proc fts5_test_columnsize {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnSize $i] } set res } proc fts5_columntext {cmd iCol} { $cmd xColumnText $iCol } proc fts5_columnlocale {cmd iCol} { $cmd xColumnLocale $iCol } proc fts5_test_columntext {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnText $i] } set res } proc fts5_test_columnlocale {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnLocale $i] } set res } proc fts5_test_columntotalsize {cmd} { set res [list] for {set i 0} {$i < [$cmd xColumnCount]} {incr i} { lappend res [$cmd xColumnTotalSize $i] } set res |
︙ | ︙ | |||
99 100 101 102 103 104 105 106 107 108 109 110 111 112 | } set res } proc fts5_test_rowcount {cmd} { $cmd xRowCount } proc test_queryphrase_cb {cnt cmd} { upvar $cnt L for {set i 0} {$i < [$cmd xInstCount]} {incr i} { foreach {ip ic io} [$cmd xInst $i] break set A($ic) 1 } | > > > > | 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 | } set res } proc fts5_test_rowcount {cmd} { $cmd xRowCount } proc fts5_test_rowid {cmd} { $cmd xRowid } proc test_queryphrase_cb {cnt cmd} { upvar $cnt L for {set i 0} {$i < [$cmd xInstCount]} {incr i} { foreach {ip ic io} [$cmd xInst $i] break set A($ic) 1 } |
︙ | ︙ | |||
120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 | set cnt [list] for {set j 0} {$j < [$cmd xColumnCount]} {incr j} { lappend cnt 0 } $cmd xQueryPhrase $i [list test_queryphrase_cb cnt] lappend res $cnt } set res } proc fts5_test_phrasecount {cmd} { $cmd xPhraseCount } proc fts5_test_all {cmd} { set res [list] lappend res columnsize [fts5_test_columnsize $cmd] lappend res columntext [fts5_test_columntext $cmd] lappend res columntotalsize [fts5_test_columntotalsize $cmd] lappend res poslist [fts5_test_poslist $cmd] lappend res tokenize [fts5_test_tokenize $cmd] lappend res rowcount [fts5_test_rowcount $cmd] set res } proc fts5_aux_test_functions {db} { foreach f { fts5_test_columnsize fts5_test_columntext fts5_test_columntotalsize fts5_test_poslist fts5_test_poslist2 fts5_test_collist fts5_test_tokenize fts5_test_rowcount fts5_test_all fts5_test_queryphrase fts5_test_phrasecount } { sqlite3_fts5_create_function $db $f $f } } proc fts5_segcount {tbl} { set N 0 | > > > > > > > > > > > > > > | 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 | set cnt [list] for {set j 0} {$j < [$cmd xColumnCount]} {incr j} { lappend cnt 0 } $cmd xQueryPhrase $i [list test_queryphrase_cb cnt] lappend res $cnt } set res } proc fts5_queryphrase {cmd iPhrase} { set cnt [list] for {set j 0} {$j < [$cmd xColumnCount]} {incr j} { lappend cnt 0 } $cmd xQueryPhrase $iPhrase [list test_queryphrase_cb cnt] set cnt } proc fts5_test_phrasecount {cmd} { $cmd xPhraseCount } proc fts5_test_all {cmd} { set res [list] lappend res columnsize [fts5_test_columnsize $cmd] lappend res columntext [fts5_test_columntext $cmd] lappend res columntotalsize [fts5_test_columntotalsize $cmd] lappend res poslist [fts5_test_poslist $cmd] lappend res tokenize [fts5_test_tokenize $cmd] lappend res rowcount [fts5_test_rowcount $cmd] set res } proc fts5_aux_test_functions {db} { foreach f { fts5_test_columnsize fts5_test_columntext fts5_test_columnlocale fts5_test_columntotalsize fts5_test_poslist fts5_test_poslist2 fts5_test_collist fts5_test_insttoken fts5_test_tokenize fts5_test_rowcount fts5_test_rowid fts5_test_all fts5_test_queryphrase fts5_test_phrasecount fts5_columntext fts5_columnlocale fts5_queryphrase fts5_collist } { sqlite3_fts5_create_function $db $f $f } } proc fts5_segcount {tbl} { set N 0 |
︙ | ︙ | |||
434 435 436 437 438 439 440 441 442 443 444 445 446 447 | error "not in foreach_detail_mode {...} block" } } proc detail_is_none {} { detail_check ; expr {$::detail == "none"} } proc detail_is_col {} { detail_check ; expr {$::detail == "col" } } proc detail_is_full {} { detail_check ; expr {$::detail == "full"} } #------------------------------------------------------------------------- # Convert a poslist of the type returned by fts5_test_poslist() to a # collist as returned by fts5_test_collist(). # proc fts5_poslist2collist {poslist} { set res [list] | > > > > > > > > > > > > > > | 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 | error "not in foreach_detail_mode {...} block" } } proc detail_is_none {} { detail_check ; expr {$::detail == "none"} } proc detail_is_col {} { detail_check ; expr {$::detail == "col" } } proc detail_is_full {} { detail_check ; expr {$::detail == "full"} } proc foreach_tokenizer_mode {prefix script} { set saved $::testprefix foreach {d mapping} { "" {} "-origintext" {, tokenize="origintext unicode61", tokendata=1} } { set s [string map [list %TOKENIZER% $mapping] $script] set ::testprefix "$prefix$d" reset_db sqlite3_fts5_register_origintext db uplevel $s } set ::testprefix $saved } #------------------------------------------------------------------------- # Convert a poslist of the type returned by fts5_test_poslist() to a # collist as returned by fts5_test_collist(). # proc fts5_poslist2collist {poslist} { set res [list] |
︙ | ︙ | |||
590 591 592 593 594 595 596 597 598 599 600 601 602 603 | nearset_rf $aCol {*}$args if {[lsearch $args -col]>=0} { set ::expr_not_ok 1 } list } #------------------------------------------------------------------------- # Code for a simple Tcl tokenizer that supports synonyms at query time. # proc tclnum_tokenize {mode tflags text} { foreach {w iStart iEnd} [fts5_tokenize_split $text] { sqlite3_fts5_token $w $iStart $iEnd | > > > > | 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 | nearset_rf $aCol {*}$args if {[lsearch $args -col]>=0} { set ::expr_not_ok 1 } list } proc dump {tname} { execsql_pp "SELECT * FROM ${tname}_idx" execsql_pp "SELECT id, quote(block), fts5_decode(id,block) FROM ${tname}_data" } #------------------------------------------------------------------------- # Code for a simple Tcl tokenizer that supports synonyms at query time. # proc tclnum_tokenize {mode tflags text} { foreach {w iStart iEnd} [fts5_tokenize_split $text] { sqlite3_fts5_token $w $iStart $iEnd |
︙ | ︙ |
Changes to ext/fts5/test/fts5aa.test.
︙ | ︙ | |||
18 19 20 21 22 23 24 25 26 27 28 29 30 31 | # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $::testprefix { do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, c); SELECT name, sql FROM sqlite_master; } { t1 {CREATE VIRTUAL TABLE t1 USING fts5(a, b, c)} t1_data {CREATE TABLE 't1_data'(id INTEGER PRIMARY KEY, block BLOB)} | > | 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $::testprefix { foreach_tokenizer_mode $::testprefix { do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, c); SELECT name, sql FROM sqlite_master; } { t1 {CREATE VIRTUAL TABLE t1 USING fts5(a, b, c)} t1_data {CREATE TABLE 't1_data'(id INTEGER PRIMARY KEY, block BLOB)} |
︙ | ︙ | |||
40 41 42 43 44 45 46 | SELECT name, sql FROM sqlite_master; } {} #------------------------------------------------------------------------- # do_execsql_test 2.0 { | | > > | > | > > | | 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | SELECT name, sql FROM sqlite_master; } {} #------------------------------------------------------------------------- # do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%); } do_execsql_test 2.1 { INSERT INTO t1 VALUES('a b c', 'd e f'); } do_test 2.2 { execsql { SELECT fts5_decode(id, block) FROM t1_data WHERE id==10 } } {/{{structure} {lvl=0 nMerge=0 nSeg=1 {id=[0123456789]* leaves=1..1}}}/} foreach w {a b c d e f} { do_execsql_test 2.3.$w.asc { SELECT rowid FROM t1 WHERE t1 MATCH $w; } {1} do_execsql_test 2.3.$w.desc { SELECT rowid FROM t1 WHERE t1 MATCH $w ORDER BY rowid DESC; } {1} } do_execsql_test 2.4 { INSERT INTO t1(t1) VALUES('integrity-check'); PRAGMA integrity_check; PRAGMA integrity_check(t1); } {ok ok} #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%); } foreach {i x y} { 1 {g f d b f} {h h e i a} 2 {f i g j e} {i j c f f} 3 {e e i f a} {e h f d f} 4 {h j f j i} {h a c f j} 5 {d b j c g} {f e i b e} 6 {a j a e e} {j d f d e} 7 {g i j c h} {j d h c a} 8 {j j i d d} {e e d f b} 9 {c j j d c} {h j i f g} 10 {b f h i a} {c f b b j} } { do_execsql_test 3.$i.1 { INSERT INTO t1 VALUES($x, $y) } do_execsql_test 3.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') } do_execsql_test 3.$i.3 { PRAGMA integrity_check(t1) } ok if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } foreach {i x y} { 1 {g f d b f} {h h e i a} 2 {f i g j e} {i j c f f} 3 {e e i f a} {e h f d f} 4 {h j f j i} {h a c f j} |
︙ | ︙ | |||
114 115 116 117 118 119 120 121 | do_execsql_test 4.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') } if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db do_execsql_test 5.0 { | > | | > | | 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 | do_execsql_test 4.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check') } if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 5.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } foreach {i x y} { 1 {dd abc abc abc abcde} {aaa dd ddd ddd aab} 2 {dd aab d aaa b} {abcde c aaa aaa aaa} 3 {abcde dd b b dd} {abc abc d abc ddddd} 4 {aaa abcde dddd dddd abcde} {abc b b abcde abc} 5 {aab dddd d dddd c} {ddd abcde dddd abcde c} 6 {ddd dd b aab abcde} {d ddddd dddd c abc} 7 {d ddddd ddd c abcde} {c aab d abcde ddd} 8 {abcde aaa aab c c} {ddd c dddd b aaa} 9 {abcde aab ddddd c aab} {dddd dddd b c dd} 10 {ddd abcde dddd dd c} {dddd c c d abcde} } { do_execsql_test 5.$i.1 { INSERT INTO t1 VALUES($x, $y) } do_execsql_test 5.$i.2 { PRAGMA integrity_check(t1) } ok if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 6.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } do_execsql_test 6.1 { INSERT INTO t1(rowid, x, y) VALUES(22, 'a b c', 'c b a'); REPLACE INTO t1(rowid, x, y) VALUES(22, 'd e f', 'f e d'); } |
︙ | ︙ | |||
174 175 176 177 178 179 180 181 182 183 184 185 186 187 | 22 {l l l} {l l l} 23 {x y z} {x y z} } #------------------------------------------------------------------------- # reset_db expr srand(0) do_execsql_test 7.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y,z); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } proc doc {} { | > | 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 | 22 {l l l} {l l l} 23 {x y z} {x y z} } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db expr srand(0) do_execsql_test 7.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y,z); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } proc doc {} { |
︙ | ︙ | |||
215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 | } {} if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db do_execsql_test 8.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, prefix="1,2,3"); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } do_execsql_test 8.1 { INSERT INTO t1 VALUES('the quick brown fox'); INSERT INTO t1(t1) VALUES('integrity-check'); } #------------------------------------------------------------------------- # reset_db expr srand(0) do_execsql_test 9.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y,z, prefix="1,2,3"); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } | > > | 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 | } {} if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 8.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, prefix="1,2,3"); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } do_execsql_test 8.1 { INSERT INTO t1 VALUES('the quick brown fox'); INSERT INTO t1(t1) VALUES('integrity-check'); } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db expr srand(0) do_execsql_test 9.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y,z, prefix="1,2,3"); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } |
︙ | ︙ | |||
273 274 275 276 277 278 279 280 | if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db do_execsql_test 10.0 { | > | | 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 | if {[set_test_counter errors]} break } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 10.0 { CREATE VIRTUAL TABLE t1 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%); } set d10 { 1 {g f d b f} {h h e i a} 2 {f i g j e} {i j c f f} 3 {e e i f a} {e h f d f} 4 {h j f j i} {h a c f j} 5 {d b j c g} {f e i b e} |
︙ | ︙ | |||
307 308 309 310 311 312 313 | do_execsql_test 10.4.1 { DELETE FROM t1 } do_execsql_test 10.4.2 { INSERT INTO t1(t1) VALUES('integrity-check') } #------------------------------------------------------------------------- # do_catchsql_test 11.1 { | | | | | > | | 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 | do_execsql_test 10.4.1 { DELETE FROM t1 } do_execsql_test 10.4.2 { INSERT INTO t1(t1) VALUES('integrity-check') } #------------------------------------------------------------------------- # do_catchsql_test 11.1 { CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rank, detail=%DETAIL% %TOKENIZER%); } {1 {reserved fts5 column name: rank}} do_catchsql_test 11.2 { CREATE VIRTUAL TABLE rank USING fts5(a, b, c, detail=%DETAIL% %TOKENIZER%); } {1 {reserved fts5 table name: rank}} do_catchsql_test 11.3 { CREATE VIRTUAL TABLE t2 USING fts5(a, b, c, rowid, detail=%DETAIL% %TOKENIZER%); } {1 {reserved fts5 column name: rowid}} #------------------------------------------------------------------------- # do_execsql_test 12.1 { CREATE VIRTUAL TABLE t2 USING fts5(x,y, detail=%DETAIL% %TOKENIZER%); } {} do_catchsql_test 12.2 { SELECT t2 FROM t2 WHERE t2 MATCH '*stuff' } {1 {unknown special query: stuff}} do_test 12.3 { set res [db eval { SELECT t2 FROM t2 WHERE t2 MATCH '* reads ' }] string is integer $res } {1} #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 13.1 { CREATE VIRTUAL TABLE t1 USING fts5(x, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1(rowid, x) VALUES(1, 'o n e'), (2, 't w o'); } {} do_execsql_test 13.2 { SELECT rowid FROM t1 WHERE t1 MATCH 'o'; } {1 2} |
︙ | ︙ | |||
358 359 360 361 362 363 364 365 | do_execsql_test 13.6 { SELECT rowid FROM t1 WHERE t1 MATCH '""'; } {} #------------------------------------------------------------------------- # reset_db do_execsql_test 14.1 { | > | | 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 | do_execsql_test 13.6 { SELECT rowid FROM t1 WHERE t1 MATCH '""'; } {} #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 14.1 { CREATE VIRTUAL TABLE t1 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); WITH d(x,y) AS ( SELECT NULL, 'xyz xyz xyz xyz xyz xyz' UNION ALL SELECT NULL, 'xyz xyz xyz xyz xyz xyz' FROM d ) INSERT INTO t1 SELECT * FROM d LIMIT 200; |
︙ | ︙ | |||
410 411 412 413 414 415 416 | } sqlite3_db_config db DEFENSIVE 0 do_execsql_test 15.1 { UPDATE t1_content SET c1 = 'xyz xyz xyz xyz xyz abc' WHERE rowid = 1; } do_catchsql_test 15.2 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | | > | > | > | | 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 | } sqlite3_db_config db DEFENSIVE 0 do_execsql_test 15.1 { UPDATE t1_content SET c1 = 'xyz xyz xyz xyz xyz abc' WHERE rowid = 1; } do_catchsql_test 15.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {1 {fts5: checksum mismatch for table "t1"}} #------------------------------------------------------------------------- # do_execsql_test 16.1 { CREATE VIRTUAL TABLE n1 USING fts5(a); INSERT INTO n1 VALUES('a b c d'); } proc funk {} { db eval { UPDATE n1_config SET v=50 WHERE k='version' } set fd [db incrblob main n1_data block 10] fconfigure $fd -translation binary # puts -nonewline $fd "\x44\x45" close $fd } db func funk funk # This test case corrupts the structure record within the first invocation # of function funk(). Which used to cause the bm25() function to throw an # exception. But since bm25() can now used the cached structure record, # it never sees the corruption introduced by funk() and so the following # statement no longer fails. # do_catchsql_test 16.2 { SELECT funk(), format('%g',bm25(n1)), funk() FROM n1 WHERE n1 MATCH 'a+b+c+d' } {0 {{} -1e-06 {}}} # {1 {SQL logic error}} #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 17.1 { CREATE VIRTUAL TABLE b2 USING fts5(x, detail=%DETAIL% %TOKENIZER%); INSERT INTO b2 VALUES('a'); INSERT INTO b2 VALUES('b'); INSERT INTO b2 VALUES('c'); } do_test 17.2 { set res [list] db eval { SELECT * FROM b2 ORDER BY rowid ASC } { lappend res [execsql { SELECT * FROM b2 ORDER BY rowid ASC }] } set res } {{a b c} {a b c} {a b c}} if {[string match n* %DETAIL%]==0} { reset_db sqlite3_fts5_register_origintext db do_execsql_test 17.3 { CREATE VIRTUAL TABLE c2 USING fts5(x, y, detail=%DETAIL% %TOKENIZER%); INSERT INTO c2 VALUES('x x x', 'x x x'); SELECT rowid FROM c2 WHERE c2 MATCH 'y:x'; } {1} } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db do_execsql_test 17.1 { CREATE VIRTUAL TABLE uio USING fts5(ttt, detail=%DETAIL% %TOKENIZER%); INSERT INTO uio VALUES(NULL); INSERT INTO uio SELECT NULL FROM uio; INSERT INTO uio SELECT NULL FROM uio; INSERT INTO uio SELECT NULL FROM uio; INSERT INTO uio SELECT NULL FROM uio; INSERT INTO uio SELECT NULL FROM uio; INSERT INTO uio SELECT NULL FROM uio; |
︙ | ︙ | |||
517 518 519 520 521 522 523 | do_execsql_test 17.9 { SELECT min(rowid), max(rowid), count(*) FROM uio WHERE rowid < 10; } {-9223372036854775808 9 10} #-------------------------------------------------------------------- # do_execsql_test 18.1 { | | | > | > | | | | | > > > | > | 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 | do_execsql_test 17.9 { SELECT min(rowid), max(rowid), count(*) FROM uio WHERE rowid < 10; } {-9223372036854775808 9 10} #-------------------------------------------------------------------- # do_execsql_test 18.1 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, detail=%DETAIL% %TOKENIZER%); CREATE VIRTUAL TABLE t2 USING fts5(c, d, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1 VALUES('abc*', NULL); INSERT INTO t2 VALUES(1, 'abcdefg'); } do_execsql_test 18.2 { SELECT t1.rowid, t2.rowid FROM t1, t2 WHERE t2 MATCH t1.a AND t1.rowid = t2.c } {1 1} do_execsql_test 18.3 { SELECT t1.rowid, t2.rowid FROM t2, t1 WHERE t2 MATCH t1.a AND t1.rowid = t2.c } {1 1} #-------------------------------------------------------------------- # fts5 table in the temp schema. # reset_db sqlite3_fts5_register_origintext db do_execsql_test 19.0 { CREATE VIRTUAL TABLE temp.t1 USING fts5(x, detail=%DETAIL% %TOKENIZER%); INSERT INTO t1 VALUES('x y z'); INSERT INTO t1 VALUES('w x 1'); SELECT rowid FROM t1 WHERE t1 MATCH 'x'; } {1 2} #-------------------------------------------------------------------- # Test that 6 and 7 byte varints can be read. # reset_db sqlite3_fts5_register_origintext db do_execsql_test 20.0 { CREATE VIRTUAL TABLE temp.tmp USING fts5(x, detail=%DETAIL% %TOKENIZER%); } set ::ids [list \ 0 [expr 1<<36] [expr 2<<36] [expr 1<<43] [expr 2<<43] ] do_test 20.1 { foreach id $::ids { execsql { INSERT INTO tmp(rowid, x) VALUES($id, 'x y z') } } execsql { SELECT rowid FROM tmp WHERE tmp MATCH 'y' } } $::ids #-------------------------------------------------------------------- # Test that a DROP TABLE may be executed within a transaction that # writes to an FTS5 table. # do_execsql_test 21.0 { CREATE TEMP TABLE t8(a, b); CREATE VIRTUAL TABLE ft USING fts5(x, detail=%DETAIL% %TOKENIZER%); } do_execsql_test 21.1 { BEGIN; INSERT INTO ft VALUES('a b c'); DROP TABLE t8; COMMIT; } do_execsql_test 22.0 { CREATE VIRTUAL TABLE t9 USING fts5(x, detail=%DETAIL% %TOKENIZER%); INSERT INTO t9(rowid, x) VALUES(2, 'bbb'); BEGIN; INSERT INTO t9(rowid, x) VALUES(1, 'aaa'); DELETE FROM t9 WHERE rowid = 2; INSERT INTO t9(rowid, x) VALUES(3, 'bbb'); COMMIT; } do_execsql_test 22.1 { SELECT rowid FROM t9('a*') } {1} #------------------------------------------------------------------------- do_execsql_test 23.0 { CREATE VIRTUAL TABLE t10 USING fts5(x, detail=%DETAIL% %TOKENIZER%); CREATE TABLE t11(x); } do_execsql_test 23.1 { SELECT * FROM t11, t10 WHERE t11.x = t10.x AND t10.rowid IS NULL; } do_execsql_test 23.2 { SELECT * FROM t11, t10 WHERE t10.rowid IS NULL; } #------------------------------------------------------------------------- do_execsql_test 24.0 { CREATE VIRTUAL TABLE t12 USING fts5(x, detail=%DETAIL% %TOKENIZER%); INSERT INTO t12 VALUES('aaaa'); } do_execsql_test 24.1 { BEGIN; DELETE FROM t12 WHERE rowid=1; SELECT * FROM t12('aaaa'); INSERT INTO t12 VALUES('aaaa'); END; } execsql_pp { SELECT rowid, hex(block) FROM t12_data } do_execsql_test 24.2 { INSERT INTO t12(t12) VALUES('integrity-check'); } do_execsql_test 24.3 { SELECT * FROM t12('aaaa'); } {aaaa} #------------------------------------------------------------------------- do_execsql_test 25.0 { CREATE VIRTUAL TABLE t13 USING fts5(x, detail=%DETAIL% %TOKENIZER%); } do_execsql_test 25.1 { BEGIN; INSERT INTO t13 VALUES('AAAA'); SELECT * FROM t13('BBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBBB*'); END; } } } expand_all_sql db finish_test |
Changes to ext/fts5/test/fts5ab.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # focus of this script is testing the FTS5 module. # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ab | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # focus of this script is testing the FTS5 module. # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ab # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ | |||
176 177 178 179 180 181 182 | if {[detail_is_full]} { do_execsql_test 4.4 { SELECT rowid FROM s1 WHERE s1 MATCH '"a x"' } {1 2} } | | > > > > | 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 | if {[detail_is_full]} { do_execsql_test 4.4 { SELECT rowid FROM s1 WHERE s1 MATCH '"a x"' } {1 2} } do_execsql_test 4.5.1 { SELECT rowid FROM s1 WHERE s1 MATCH 'a AND x' } {1 2} do_execsql_test 4.5.2 { SELECT rowid FROM s1 WHERE s1 MATCH 'a x' } {1 2} #------------------------------------------------------------------------- # Check that a special case of segment promotion works. The case is where # a new segment is written to level L, but the oldest segment within level # (L-2) is larger than it. |
︙ | ︙ |
Changes to ext/fts5/test/fts5ac.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # focus of this script is testing the FTS5 module. # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ac | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # focus of this script is testing the FTS5 module. # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ac # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ |
Changes to ext/fts5/test/fts5ad.test.
︙ | ︙ | |||
14 15 16 17 18 19 20 | # More specifically, the focus is on testing prefix queries, both with and # without prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ad | | | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | # More specifically, the focus is on testing prefix queries, both with and # without prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ad # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ |
Changes to ext/fts5/test/fts5ae.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # focus of this script is testing the FTS5 module. # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ae | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # focus of this script is testing the FTS5 module. # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ae # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ |
Changes to ext/fts5/test/fts5af.test.
︙ | ︙ | |||
14 15 16 17 18 19 20 | # More specifically, the tests in this file focus on the built-in # snippet() function. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5af | | | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | # More specifically, the tests in this file focus on the built-in # snippet() function. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5af # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ | |||
188 189 190 191 192 193 194 195 196 | SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ""'); } {{[x a a a a a...}} do_execsql_test 5.6 { SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ' || x'DB'); } {{[x a a a a a...}} } ;# foreach_detail_mode finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 | SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ""'); } {{[x a a a a a...}} do_execsql_test 5.6 { SELECT snippet(p1, 0, '[', NULL, '...', 6) FROM p1('x OR ' || x'DB'); } {{[x a a a a a...}} } ;# foreach_detail_mode reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE t1 USING fts5(colA, colB); INSERT INTO t1 VALUES('A B C', 'D E F'); } do_execsql_test 6.1 { SELECT colA, colB, snippet(t1,0,'[', ']','...',1) FROM t1 WHERE t1 MATCH 'B'; } {{A B C} {D E F} ...[B]...} breakpoint do_execsql_test 6.2 { SELECT colA, colB, snippet(t1, 1,'[',']','...',2) FROM t1 WHERE t1 MATCH 'B'; } {{A B C} {D E F} {D E...}} do_execsql_test 6.3 { SELECT colA, colB, snippet(t1, 1,'[',']','...',1) FROM t1 WHERE t1 MATCH 'B'; } {{A B C} {D E F} {D...}} do_execsql_test 6.1 { SELECT colA, colB, snippet(t1,0,'[', ']','...',1) FROM t1 WHERE t1 MATCH 'A'; } {{A B C} {D E F} [A]...} breakpoint do_execsql_test 6.2 { SELECT colA, colB, snippet(t1, 1,'[',']','...',2) FROM t1 WHERE t1 MATCH 'A'; } {{A B C} {D E F} {D E...}} do_execsql_test 6.3 { SELECT colA, colB, snippet(t1, 1,'[',']','...',1) FROM t1 WHERE t1 MATCH 'A'; } {{A B C} {D E F} {D...}} finish_test |
Changes to ext/fts5/test/fts5ag.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ag | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ag # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # This file attempts to verify that the extension APIs work with |
︙ | ︙ |
Changes to ext/fts5/test/fts5ah.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # focus of this script is testing the FTS5 module. # # TESTRUNNER: slow source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ah | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # focus of this script is testing the FTS5 module. # # TESTRUNNER: slow source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ah # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ | |||
159 160 161 162 163 164 165 166 167 168 169 170 | do_execsql_test 1.8.1 { SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND +rowid < 'text'; } {10000} do_execsql_test 1.8.2 { SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid < 'text'; } {10000} } ;# foreach_detail_mode #db eval {SELECT rowid, fts5_decode(rowid, block) aS r FROM t1_data} {puts $r} finish_test | > > > > > > > > > > > | 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | do_execsql_test 1.8.1 { SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND +rowid < 'text'; } {10000} do_execsql_test 1.8.2 { SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid < 'text'; } {10000} do_execsql_test 1.8.3 { SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid<5000 AND rowid < 'text'; } {4999} do_execsql_test 1.8.4 { SELECT count(*) FROM t1 WHERE t1 MATCH 'x' AND rowid>5000 AND rowid > 'text'; } {0} do_catchsql_test 1.9 { SELECT * FROM t1('*xy'); } {1 {unknown special query: xy}} } ;# foreach_detail_mode #db eval {SELECT rowid, fts5_decode(rowid, block) aS r FROM t1_data} {puts $r} finish_test |
Changes to ext/fts5/test/fts5ai.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # # Specifically, it tests transactions and savepoints # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ai | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # # Specifically, it tests transactions and savepoints # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ai # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ |
Changes to ext/fts5/test/fts5aj.test.
︙ | ︙ | |||
15 16 17 18 19 20 21 | # constant, the FTS index does not grow indefinitely as rows are inserted # and deleted, # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5aj | | | 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 | # constant, the FTS index does not grow indefinitely as rows are inserted # and deleted, # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5aj # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc doc {} { set dict [list a b c d e f g h i j k l m n o p q r s t u v w x y z] |
︙ | ︙ |
Changes to ext/fts5/test/fts5ak.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # # Specifically, the auxiliary function "highlight". # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ak | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # # Specifically, the auxiliary function "highlight". # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ak # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ | |||
149 150 151 152 153 154 155 156 157 | } { {a b c x c d e} {a b c c d e} {a b c d e} } } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > | 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 | } { {a b c x c d e} {a b c c d e} {a b c d e} } } # 2023-04-06 https://sqlite.org/forum/forumpost/cae4367d9b # # This is not a test of FTS5, but rather a test of the of what happens to # prepared statements that encounter SQLITE_SCHEMA while other prepared # statements are running. The original problem POC used FTS5, and so # is seems reasonable to put the test here. # # The vdbeaux24.test module in TH3 also tests this same behavior but # without requiring FTS5 or an other extension. # reset_db db null NULL do_execsql_test 4.0 { CREATE TABLE t5(a PRIMARY KEY); INSERT INTO t5 VALUES(0); CREATE VIRTUAL TABLE t6 USING fts5(0); DELETE FROM t6; CREATE TABLE t7(x); WITH cte(a) AS ( SELECT a FROM t5 WHERE ((0,0) IN (SELECT 0, LAG(0) OVER (PARTITION BY 0) FROM t6), 0) < (a,0) ) SELECT max(a) FROM cte; } NULL finish_test |
Changes to ext/fts5/test/fts5al.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # # Specifically, this function tests the %_config table. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5al | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # # Specifically, this function tests the %_config table. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5al # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ | |||
288 289 290 291 292 293 294 295 296 297 298 299 | do_catchsql_test 4.4.3 { SELECT *, rank FROM t3 WHERE t3 MATCH 'a' AND rank MATCH 'xyz(3)' } {1 {no such function: xyz}} do_catchsql_test 4.4.4 { SELECT *, rank FROM t3 WHERE t3 MATCH 'a' AND rank MATCH NULL } {1 {parse error in rank function: }} } ;# foreach_detail_mode finish_test | > > > > > > > > > > | 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 | do_catchsql_test 4.4.3 { SELECT *, rank FROM t3 WHERE t3 MATCH 'a' AND rank MATCH 'xyz(3)' } {1 {no such function: xyz}} do_catchsql_test 4.4.4 { SELECT *, rank FROM t3 WHERE t3 MATCH 'a' AND rank MATCH NULL } {1 {parse error in rank function: }} # Check that the second and subsequent rank= constraints are ignored. # do_catchsql_test 4.3.3 { SELECT *, rank FROM t3 WHERE t3 MATCH 'a' AND rank MATCH 'nosuch()' AND rank MATCH 'rowidmod(3)' ORDER BY rank ASC } {1 {unable to use function MATCH in the requested context}} } ;# foreach_detail_mode finish_test |
Changes to ext/fts5/test/fts5alter.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # The tests in this file focus on renaming FTS5 tables using the # "ALTER TABLE ... RENAME TO ..." command # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5alter | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # The tests in this file focus on renaming FTS5 tables using the # "ALTER TABLE ... RENAME TO ..." command # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5alter # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # Test renaming regular, contentless and columnsize=0 FTS5 tables. |
︙ | ︙ |
Changes to ext/fts5/test/fts5auto.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # This file contains automatically generated tests for various types # of MATCH expressions. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5auto | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # This file contains automatically generated tests for various types # of MATCH expressions. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5auto # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set data { -4026076 |
︙ | ︙ |
Changes to ext/fts5/test/fts5aux.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on the auxiliary function APIs. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5aux | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on the auxiliary function APIs. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5aux # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc inst {cmd i} { $cmd xInst $i |
︙ | ︙ | |||
303 304 305 306 307 308 309 | SELECT group_concat(firstcol(t1), '.') FROM t1 } {1 {unable to use function firstcol in the requested context}} do_catchsql_test 10.1.4 { SELECT group_concat(firstcol(t1), '.') FROM t1 GROUP BY rowid } {1 {unable to use function firstcol in the requested context}} | > > > > | > > > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 | SELECT group_concat(firstcol(t1), '.') FROM t1 } {1 {unable to use function firstcol in the requested context}} do_catchsql_test 10.1.4 { SELECT group_concat(firstcol(t1), '.') FROM t1 GROUP BY rowid } {1 {unable to use function firstcol in the requested context}} #------------------------------------------------------------------------- # Test that xInstCount() works from within an xPhraseQuery() callback. # reset_db proc xCallback {cmd} { incr ::hitcount [$cmd xInstCount] return SQLITE_OK } proc fts5_hitcount {cmd} { set ::hitcount 0 $cmd xQueryPhrase 0 xCallback return $::hitcount } sqlite3_fts5_create_function db fts5_hitcount fts5_hitcount do_execsql_test 11.1 { CREATE VIRTUAL TABLE x1 USING fts5(z); INSERT INTO x1 VALUES('one two three'); INSERT INTO x1 VALUES('one two one three one'); INSERT INTO x1 VALUES('one two three'); } do_execsql_test 11.2 { SELECT fts5_hitcount(x1) FROM x1('one') LIMIT 1; } {5} #------------------------------------------------------------------------- # Test that xColumnText returns SQLITE_RANGE when it should. # reset_db fts5_aux_test_functions db do_execsql_test 12.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, c); INSERT INTO t1 VALUES('one', 'two', 'three'); INSERT INTO t1 VALUES('one', 'one', 'one'); INSERT INTO t1 VALUES('two', 'two', 'two'); INSERT INTO t1 VALUES('three', 'three', 'three'); } do_catchsql_test 12.1.1 { SELECT fts5_columntext(t1, -1) FROM t1('two'); } {1 SQLITE_RANGE} do_catchsql_test 12.1.2 { SELECT fts5_columntext(t1, 3) FROM t1('two'); } {1 SQLITE_RANGE} do_catchsql_test 12.1.2 { SELECT fts5_columntext(t1, 1) FROM t1('one AND two'); } {0 two} do_catchsql_test 12.2.1 { SELECT fts5_queryphrase(t1, -1) FROM t1('one AND two'); } {1 SQLITE_RANGE} do_catchsql_test 12.2.2 { SELECT fts5_queryphrase(t1, 2) FROM t1('one AND two'); } {1 SQLITE_RANGE} do_catchsql_test 12.2.3 { SELECT fts5_queryphrase(t1, 1) FROM t1('one AND two'); } {0 {{1 2 1}}} do_catchsql_test 12.3.1 { SELECT fts5_collist(t1, -1) FROM t1('one AND two'); } {1 SQLITE_RANGE} do_catchsql_test 12.3.2 { SELECT fts5_collist(t1, 2) FROM t1('one AND two'); } {1 SQLITE_RANGE} do_catchsql_test 12.3.3 { SELECT fts5_collist(t1, 1) FROM t1('one AND two'); } {0 1} #------------------------------------------------------------------------- reset_db do_execsql_test 13.1 { CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=ascii); INSERT INTO t1 VALUES('a b c'), ('d e f'); PRAGMA integrity_check; } {ok} do_catchsql_test 13.2 { SELECT highlight(t1, 0, '[', ']') FROM t1 } {0 {{a b c} {d e f}}} do_execsql_test 13.3 { PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=blah)' WHERE name = 't1'; } db close sqlite3 db test.db do_catchsql_test 13.4 { SELECT highlight(t1, 0, '[', ']') FROM t1 } {1 {SQL logic error}} finish_test |
Added ext/fts5/test/fts5aux2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | # 2024 June 24 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focusing on the auxiliary function APIs. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5aux # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(a, b); INSERT INTO x1 VALUES('a b', 'c d'); INSERT INTO x1 VALUES('d e', 'a b'); INSERT INTO x1 VALUES('a b', 'e f'); INSERT INTO x1 VALUES('d e', 'c d'); } fts5_aux_test_functions db do_execsql_test 1.1 { SELECT fts5_test_all(x1) FROM x1 WHERE rowid=2 } [list [list {*}{ columnsize {2 2} columntext {{d e} {a b}} columntotalsize {8 8} poslist {} tokenize {{d e} {a b}} rowcount 4 }]] do_execsql_test 1.2 { SELECT fts5_test_columntext(x1) FROM x1 } { {{a b} {c d}} {{d e} {a b}} {{a b} {e f}} {{d e} {c d}} } do_execsql_test 1.3 { SELECT fts5_test_rowid(x1) FROM x1 } { 1 2 3 4 } do_execsql_test 1.4 { SELECT fts5_test_phrasecount(x1) FROM x1 } { 0 0 0 0 } do_catchsql_test 1.5 { SELECT fts5_queryphrase(x1, 0) FROM x1 } {1 SQLITE_RANGE} do_execsql_test 1.6 { SELECT fts5_test_rowcount(x1) FROM x1 } {4 4 4 4} finish_test |
Changes to ext/fts5/test/fts5auxdata.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on the fts5 xSetAuxdata() and xGetAuxdata() APIs. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5auxdata | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on the fts5 xSetAuxdata() and xGetAuxdata() APIs. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5auxdata # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE f1 USING fts5(a, b); |
︙ | ︙ |
Added ext/fts5/test/fts5bigid.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 | # 2023 May 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5bigid # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set nRow 20000 proc do_ascdesc_test {tn query} { set ::lAsc [db eval { SELECT rowid FROM x1($query) }] set ::lDesc [db eval { SELECT rowid FROM x1($query) ORDER BY rowid DESC }] do_test $tn.1 { lsort -integer $::lAsc } $::lAsc do_test $tn.2 { lsort -integer -decr $::lDesc } $::lDesc do_test $tn.3 { lsort -integer $::lDesc } $::lAsc } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(a); } do_test 1.1 { for {set ii 0} {$ii < $nRow} {incr ii} { db eval { REPLACE INTO x1(rowid, a) VALUES(random(), 'movement at the station'); } } } {} do_ascdesc_test 1.2 "the" do_execsql_test 1.3 { DELETE FROM x1 } do_test 1.4 { for {set ii 0} {$ii < $nRow} {incr ii} { db eval { INSERT INTO x1(rowid, a) VALUES( $ii + 0x6FFFFFFFFFFFFFFF, 'movement at the station' ); } } } {} do_ascdesc_test 1.5 "movement" finish_test |
Changes to ext/fts5/test/fts5bigpl.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # 4 or 5 byte position-list size varints. Because of the amount of memory # required, these tests only run on 64-bit platforms. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5bigpl | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # 4 or 5 byte position-list size varints. Because of the amount of memory # required, these tests only run on 64-bit platforms. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5bigpl # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } if { $tcl_platform(wordSize)<8 } { finish_test |
︙ | ︙ |
Added ext/fts5/test/fts5blob.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 | # 2024 July 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file verifies that: # # * blob values may be written to locale=0 tables. # # * blob values - other than fts5_locale() values - may not be written # to locale=0 tables. This is an SQLITE_MISMATCH error # # * blob values may be returned by queries on the external-content table # of a locale=0 table. # # * blob values not may be returned by queries on the external-content # table of a locale=1 table, apart from fts5_locale() blobs. This is an # SQLITE_MISMATCH error. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5blob # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } # Test that blobs may be stored in normal locale=0 tables. # foreach {tn enc} { 1 utf8 2 utf16 } { reset_db fts5_aux_test_functions db execsql "PRAGMA encoding = $enc" execsql " CREATE VIRTUAL TABLE t1 USING fts5(x, y); " do_execsql_test 1.$tn.0 { CREATE VIRTUAL TABLE tt USING fts5vocab('t1', 'instance'); INSERT INTO t1(rowid, x, y) VALUES(1, 555, X'0000000041424320444546'); INSERT INTO t1(rowid, x, y) VALUES(2, 666, X'41424300444546'); INSERT INTO t1(rowid, x, y) VALUES(3, 777, 'xyz'); } do_execsql_test 1.$tn.1 { SELECT rowid, quote(x), quote(y) FROM t1 } { 1 555 X'0000000041424320444546' 2 666 X'41424300444546' 3 777 'xyz' } do_execsql_test 1.$tn.2 { DELETE FROM t1 WHERE rowid=2; DELETE FROM t1 WHERE rowid=1; } do_execsql_test 1.$tn.3 { PRAGMA integrity_check; } {ok} } #-------------------------------------------------------------------------- # Test that a blob may be stored and retrieved in an unindexed column of # a regular table with locale=1. # reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, y UNINDEXED, locale=1); INSERT INTO t1(rowid, x, y) VALUES(12, 'twelve', X'0000000041424320444546'); } do_execsql_test 2.1 { select rowid, x, quote(y) FROM t1 } { 12 twelve X'0000000041424320444546' } #-------------------------------------------------------------------------- # Test that blobs may not be written to any type of table with locale=1 # set. Except, they may be written to UNINDEXED columns. # reset_db do_execsql_test 3.0 { CREATE TABLE t1(a, b); CREATE VIRTUAL TABLE x1 USING fts5(a, b, locale=1); CREATE VIRTUAL TABLE x2 USING fts5(a, b, locale=1, content=t2); CREATE VIRTUAL TABLE x3 USING fts5(a, b, locale=1, content=); } do_catchsql_test 3.1 { INSERT INTO x1(rowid, a, b) VALUES(113, 'hello world', X'123456'); } {0 {}} do_catchsql_test 3.2 { INSERT INTO x2(rowid, a, b) VALUES(113, 'hello world', X'123456'); } {0 {}} do_catchsql_test 3.3 { INSERT INTO x3(rowid, a, b) VALUES(113, 'hello world', X'123456'); } {0 {}} #-------------------------------------------------------------------------- # Test that fts5_locale() values may not be written to any type of table # without locale=1 set. Even to an UNINDEXED column. # reset_db do_execsql_test 3.0 { CREATE TABLE t1(a, b); CREATE VIRTUAL TABLE x1 USING fts5(a, b); CREATE VIRTUAL TABLE x2 USING fts5(a, b, content=t2); CREATE VIRTUAL TABLE x3 USING fts5(a, b, content=); CREATE VIRTUAL TABLE x4 USING fts5(a, b, c UNINDEXED); } do_catchsql_test 3.1 { INSERT INTO x1(rowid, a, b) VALUES(113, 'hello world', fts5_locale('en_AU', 'abc')); } {1 {fts5_locale() requires locale=1}} do_catchsql_test 3.2 { INSERT INTO x2(rowid, a, b) VALUES(113, 'hello world', fts5_locale('en_AU', 'abc')); } {1 {fts5_locale() requires locale=1}} do_catchsql_test 3.3 { INSERT INTO x3(rowid, a, b) VALUES(113, 'hello world', fts5_locale('en_AU', 'abc')); } {1 {fts5_locale() requires locale=1}} do_catchsql_test 3.4 { INSERT INTO x4(rowid, a, b, c) VALUES(113, 'hello world', 'yesno', fts5_locale('en_AU', 'abc')); } {1 {fts5_locale() requires locale=1}} #------------------------------------------------------------------------- # reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); } foreach {tn sql} { 1 { INSERT INTO x1(rowid, x) VALUES(4.5, 'abcd') } 2 { INSERT INTO x1(rowid, x) VALUES('xyz', 'abcd') } 3 { INSERT INTO x1(rowid, x) VALUES(X'001122', 'abcd') } } { do_catchsql_test 4.1.$tn $sql {1 {datatype mismatch}} } finish_test |
Changes to ext/fts5/test/fts5cat.test.
︙ | ︙ | |||
50 51 52 53 54 55 56 57 58 59 | do_execsql_test 1.4 { SELECT * FROM t3t } {สนามกีฬา 1 1} do_execsql_test 1.5 { SELECT * FROM t4t } {สนามกีฬา 1 1} finish_test | > > > > > > > > > > > > > > > > > | 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | do_execsql_test 1.4 { SELECT * FROM t3t } {สนามกีฬา 1 1} do_execsql_test 1.5 { SELECT * FROM t4t } {สนามกีฬา 1 1} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 " CREATE VIRTUAL TABLE x1 USING fts5(c, tokenize=\"unicode61 categories ' \t'\"); " do_catchsql_test 2.1 " CREATE VIRTUAL TABLE x2 USING fts5(c, tokenize=\"unicode61 categories 'N*\t\tMYZ'\"); " {1 {error in tokenizer constructor}} do_catchsql_test 2.2 " CREATE VIRTUAL TABLE x2 USING fts5(c, tokenize=\"unicode61 categories 'N*\t\tXYZ'\"); " {1 {error in tokenizer constructor}} finish_test |
Changes to ext/fts5/test/fts5circref.test.
︙ | ︙ | |||
68 69 70 71 72 73 74 | END; } { INSERT INTO tt(a) VALUES('one two three'); } } { db_restore_and_reopen do_execsql_test 1.1.$tn.1 $schema | | | 68 69 70 71 72 73 74 75 76 77 78 79 80 | END; } { INSERT INTO tt(a) VALUES('one two three'); } } { db_restore_and_reopen do_execsql_test 1.1.$tn.1 $schema do_catchsql_test 1.1.$tn.2 $sql {1 {database disk image is malformed}} db close } finish_test |
Changes to ext/fts5/test/fts5colset.test.
︙ | ︙ | |||
75 76 77 78 79 80 81 | do_execsql_test 3.$tn " SELECT rowid FROM t1 WHERE $w " $res } do_catchsql_test 4.1 { SELECT * FROM t1 WHERE rowid MATCH 'a' | | | 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | do_execsql_test 3.$tn " SELECT rowid FROM t1 WHERE $w " $res } do_catchsql_test 4.1 { SELECT * FROM t1 WHERE rowid MATCH 'a' } {1 {no query solution}} } #------------------------------------------------------------------------- # Confirm that the expression parser creates the same expression tree # for: # # {a b} : (abc AND def) |
︙ | ︙ |
Changes to ext/fts5/test/fts5columnsize.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on fts5 tables with the columnsize=0 option. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5columnsize | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on fts5 tables with the columnsize=0 option. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5columnsize # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # Check that the option can be parsed and that the %_docsize table is |
︙ | ︙ |
Changes to ext/fts5/test/fts5config.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # This file focuses on the code in fts5_config.c, which is largely concerned # with parsing the various configuration and CREATE TABLE options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5config | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # This file focuses on the code in fts5_config.c, which is largely concerned # with parsing the various configuration and CREATE TABLE options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5config # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # Try different types of quote characters. |
︙ | ︙ |
Changes to ext/fts5/test/fts5conflict.test.
︙ | ︙ | |||
60 61 62 63 64 65 66 67 68 | UPDATE tbl SET b = '1 2 x' WHERE rowid=10; REPLACE INTO tbl VALUES(1, '4 5 6', '3 2 1'); DELETE FROM tbl WHERE a=100; INSERT INTO fts_idx(fts_idx) VALUES('integrity-check'); } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | UPDATE tbl SET b = '1 2 x' WHERE rowid=10; REPLACE INTO tbl VALUES(1, '4 5 6', '3 2 1'); DELETE FROM tbl WHERE a=100; INSERT INTO fts_idx(fts_idx) VALUES('integrity-check'); } #------------------------------------------------------------------------- # Tests for OR IGNORE conflict handling. # reset_db foreach_detail_mode $::testprefix { do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(xyz, detail=%DETAIL%); BEGIN; INSERT INTO t1(rowid, xyz) VALUES(13, 'thirteen documents'); INSERT INTO t1(rowid, xyz) VALUES(14, 'fourteen documents'); INSERT INTO t1(rowid, xyz) VALUES(15, 'fifteen documents'); COMMIT; } set db_cksum [cksum] foreach {tn sql} { 1 { INSERT OR IGNORE INTO t1(rowid, xyz) VALUES(14, 'new text'); } 2 { UPDATE OR IGNORE t1 SET rowid=13 WHERE rowid=15; } 3 { INSERT OR IGNORE INTO t1(rowid, xyz) SELECT 13, 'some text' UNION ALL SELECT 14, 'some text' UNION ALL SELECT 15, 'some text' } } { do_execsql_test 3.1.$tn.1 $sql do_test 3.1.$tn.2 { cksum } $db_cksum } } finish_test |
Changes to ext/fts5/test/fts5content.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5content | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5content # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # Contentless tables |
︙ | ︙ | |||
288 289 290 291 292 293 294 295 296 297 | } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.4 { SELECT count(*) FROM t1; } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.5 { SELECT * FROM t1('abc') ORDER BY rank; } {1 {recursively defined fts5 content table}} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 | } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.4 { SELECT count(*) FROM t1; } {1 {recursively defined fts5 content table}} do_catchsql_test 7.2.5 { SELECT * FROM t1('abc') ORDER BY rank; } {1 {recursively defined fts5 content table}} #--------------------------------------------------------------------------- # Check that if the content table is a view, and that view contains an # error, a reasonable error message is returned if the user tries to # read from the view via the fts5 table. # reset_db do_execsql_test 8.1 { CREATE VIEW a1 AS SELECT 1 AS r, text_value(1) AS t UNION ALL SELECT 2 AS r, text_value(2) AS t; CREATE VIRTUAL TABLE t1 USING fts5(t, content='a1', content_rowid='r'); } foreach {tn sql} { 1 "SELECT * FROM t1" 2 "INSERT INTO t1(t1) VALUES('rebuild')" 3 "SELECT * FROM t1 WHERE rowid=1" } { do_catchsql_test 8.2.$tn $sql {1 {no such function: text_value}} } proc text_value {i} { if {$i==1} { return "one" } if {$i==2} { return "two" } return "many" } db func text_value text_value do_execsql_test 8.3.1 { SELECT * FROM t1 } {one two} do_execsql_test 8.3.2 { INSERT INTO t1(t1) VALUES('rebuild') } do_execsql_test 8.3.3 { SELECT * FROM t1 WHERE rowid=1 } {one} do_execsql_test 8.3.4 { SELECT rowid FROM t1('two') } {2} #------------------------------------------------------------------------- reset_db do_execsql_test 9.1 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b); INSERT INTO t1 VALUES(1, 'one two three'); INSERT INTO t1 VALUES(2, 'one two three'); CREATE VIRTUAL TABLE ft USING fts5(b, content=t1, content_rowid=a); INSERT INTO ft(ft) VALUES('rebuild'); } do_execsql_test 9.2 { SELECT rowid, b FROM ft('two'); } { 1 {one two three} 2 {one two three} } do_execsql_test 9.3 { DELETE FROM t1 WHERE a=2; } do_catchsql_test 9.4 { SELECT rowid FROM ft('two'); } {0 {1 2}} do_catchsql_test 9.5 { SELECT * FROM ft('two'); } {1 {fts5: missing row 2 from content table 'main'.'t1'}} fts5_aux_test_functions db do_catchsql_test 9.6 { SELECT rowid, fts5_columntext(ft, 0) FROM ft('two'); } {1 SQLITE_CORRUPT_VTAB} finish_test |
Added ext/fts5/test/fts5contentless.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 | # 2014 Dec 20 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5contentless # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } # Check that it is not possible to specify "contentless_delete=1" for # anything other than a contentless table. # set res(0) {0 {}} set res(1) {1 {contentless_delete=1 requires a contentless table}} foreach {tn sql bError} { 1 "(a, b, contentless_delete=1)" 1 2 "(a, b, contentless_delete=1, content=abc)" 1 3 "(a, b, contentless_delete=1, content=)" 0 4 "(content=, contentless_delete=1, a)" 0 5 "(content='', contentless_delete=1, hello)" 0 } { execsql { BEGIN } do_catchsql_test 1.$tn "CREATE VIRTUAL TABLE t1 USING fts5 $sql" $res($bError) execsql { ROLLBACK } } # Check that it is not possible to specify "contentless_delete=1" # along with columnsize=1. # set res(0) {0 {}} set res(1) {1 {contentless_delete=1 is incompatible with columnsize=0}} foreach {tn sql bError} { 2 "(a, b, content='', contentless_delete=1, columnsize=0)" 1 } { execsql { BEGIN } do_catchsql_test 1.$tn "CREATE VIRTUAL TABLE t1 USING fts5 $sql" $res($bError) execsql { ROLLBACK } } # Check that if contentless_delete=1 is specified, then the "origin" # column is added to the %_docsize table. reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE x1 USING fts5(c, content=''); CREATE VIRTUAL TABLE x2 USING fts5(c, content='', contentless_delete=1); } do_execsql_test 3.1 { SELECT sql FROM sqlite_schema WHERE name IN ('x1_docsize', 'x2_docsize'); } { {CREATE TABLE 'x1_docsize'(id INTEGER PRIMARY KEY, sz BLOB)} {CREATE TABLE 'x2_docsize'(id INTEGER PRIMARY KEY, sz BLOB, origin INTEGER)} } do_execsql_test 3.2.1 { SELECT hex(block) FROM x1_data WHERE id=10 } {00000000000000} do_execsql_test 3.2.2 { SELECT hex(block) FROM x2_data WHERE id=10 } {00000000FF000001000000} do_execsql_test 3.3 { INSERT INTO x2 VALUES('first text'); INSERT INTO x2 VALUES('second text'); } do_execsql_test 3.4 { SELECT id, origin FROM x2_docsize } {1 1 2 2} do_execsql_test 3.5 { SELECT level, segment, loc1, loc2 FROM fts5_structure( (SELECT block FROM x2_data WHERE id=10) ) } { 0 0 1 1 0 1 2 2 } do_execsql_test 3.6 { INSERT INTO x2(x2) VALUES('optimize'); } do_execsql_test 3.7 { SELECT level, segment, loc1, loc2 FROM fts5_structure( (SELECT block FROM x2_data WHERE id=10) ) } { 1 0 1 2 } do_execsql_test 3.8 { DELETE FROM x2 WHERE rowid=2; } do_execsql_test 3.9 { SELECT rowid FROM x2('text') } {1} #-------------------------------------------------------------------------- reset_db proc document {n} { set vocab [list A B C D E F G H I J K L M N O P Q R S T U V W X Y Z] set ret [list] for {set ii 0} {$ii < $n} {incr ii} { lappend ret [lindex $vocab [expr int(rand()*[llength $vocab])]] } set ret } set nRow 1000 do_execsql_test 4.0 { CREATE TABLE t1(x); CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); INSERT INTO ft(ft, rank) VALUES('pgsz', 100); } do_test 4.1 { for {set ii 0} {$ii < $nRow} {incr ii} { set doc [document 6] execsql { INSERT INTO t1 VALUES($doc); INSERT INTO ft VALUES($doc); } } } {} foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} { set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}] set L2 [execsql {SELECT rowid FROM ft($v)}] do_test 4.2.$v { set L1 } $L2 } do_test 4.3 { for {set ii 1} {$ii < $nRow} {incr ii 2} { execsql { DELETE FROM ft WHERE rowid=$ii; DELETE FROM t1 WHERE rowid=$ii; } } } {} foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} { set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}] set L2 [execsql {SELECT rowid FROM ft($v)}] do_test 4.4.$v { set L1 } $L2 } do_execsql_test 4.5 { INSERT INTO ft(ft) VALUES('optimize'); } {} foreach v {A B C D E F G H I J K L M N O P Q R S T U V W X Y Z} { set L1 [execsql {SELECT rowid FROM t1 WHERE x LIKE '%'||$v||'%'}] set L2 [execsql {SELECT rowid FROM ft($v)}] do_test 4.6.$v { set L1 } $L2 } #execsql_pp { SELECT fts5_decode(id, block) FROM ft_data } #------------------------------------------------------------------------- reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); INSERT INTO ft(rowid, x) VALUES(1, 'one two three'); INSERT INTO ft(rowid, x) VALUES(2, 'one two four'); INSERT INTO ft(rowid, x) VALUES(3, 'one two five'); INSERT INTO ft(rowid, x) VALUES(4, 'one two seven'); INSERT INTO ft(rowid, x) VALUES(5, 'one two eight'); } do_execsql_test 5.1 { DELETE FROM ft WHERE rowid=2 } do_execsql_test 5.2 { SELECT rowid FROM ft } {1 3 4 5} do_catchsql_test 5.3 { UPDATE ft SET x='four six' WHERE rowid=3 } {0 {}} do_execsql_test 5.4 { SELECT rowid FROM ft('one'); } {1 4 5} do_execsql_test 5.5 { REPLACE INTO ft(rowid, x) VALUES(3, 'four six'); SELECT rowid FROM ft('one'); } {1 4 5} do_execsql_test 5.6 { REPLACE INTO ft(rowid, x) VALUES(6, 'one two eleven'); SELECT rowid FROM ft('one'); } {1 4 5 6} #------------------------------------------------------------------------- reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); INSERT INTO ft(rowid, x) VALUES(1, 'one two three'); INSERT INTO ft(rowid, x) VALUES(2, 'one two four'); } do_test 6.1 { db eval { SELECT rowid FROM ft('one two') } { if {$rowid==1} { db eval { INSERT INTO ft(rowid, x) VALUES(3, 'one two four') } } } } {} #------------------------------------------------------------------------- reset_db do_execsql_test 7.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); } set lRowid [list -450 0 1 2 42] do_test 7.1 { execsql BEGIN foreach r $lRowid { execsql { INSERT INTO ft(rowid, x) VALUES($r, 'one one one'); } } execsql COMMIT } {} do_test 7.2 { execsql BEGIN foreach r $lRowid { execsql { REPLACE INTO ft(rowid, x) VALUES($r, 'two two two'); } } execsql COMMIT } {} do_execsql_test 7.3 { SELECT rowid FROM ft('one'); } {} do_execsql_test 7.4 { SELECT rowid FROM ft('two'); } $lRowid #------------------------------------------------------------------------- reset_db do_execsql_test 8.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); INSERT INTO ft VALUES('hello world'); INSERT INTO ft VALUES('one two three'); } do_catchsql_test 8.1 { INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, 'hello world'); } {1 {'delete' may not be used with a contentless_delete=1 table}} do_execsql_test 8.2 { BEGIN; INSERT INTO ft(rowid, x) VALUES(3, 'four four four'); DELETE FROM ft WHERE rowid=3; COMMIT; SELECT rowid FROM ft('four'); } {} #------------------------------------------------------------------------- reset_db do_execsql_test 9.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=0); INSERT INTO ft VALUES('hello world'); INSERT INTO ft VALUES('one two three'); } do_catchsql_test 9.1 { INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, 'hello world'); } {0 {}} do_catchsql_test 9.2 { CREATE VIRTUAL TABLE ft2 USING fts5(x, content='', contentless_delete=2); } {1 {malformed contentless_delete=... directive}} do_catchsql_test 9.3 { CREATE VIRTUAL TABLE ft2 USING fts5(x, content='', contentless_delete=11); } {1 {malformed contentless_delete=... directive}} finish_test |
Added ext/fts5/test/fts5contentless2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 | # 2023 July 19 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5contentless2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc vocab {} { list aaa bbb ccc ddd eee fff ggg hhh iii jjj kkk lll mmm nnn ooo ppp } proc document {nToken} { set doc [list] set vocab [vocab] for {set ii 0} {$ii < $nToken} {incr ii} { lappend doc [lindex $vocab [expr int(rand()*[llength $vocab])]] } set doc } db func document document proc contains {doc token} { expr {[lsearch $doc $token]>=0} } db func contains contains proc do_compare_tables_test {tn} { uplevel [list do_test $tn { foreach v [vocab] { set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $v) }] set l2 [execsql { SELECT rowid FROM t2($v) }] if {$l1!=$l2} { error "1: query mismatch ($l1) ($l2)" } set w "[string range $v 0 1]*" set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $w) }] set l2 [execsql { SELECT rowid FROM t2($w) }] if {$l1!=$l2} { error "2: query mismatch ($l1) ($l2)" } set w "[string range $v 0 0]*" set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $w) }] set l2 [execsql { SELECT rowid FROM t2($w) }] if {$l1!=$l2} { error "2: query mismatch ($l1) ($l2)" } set l1 [execsql { SELECT rowid FROM t1 WHERE contains(doc, $v) ORDER BY rowid DESC }] set l2 [execsql { SELECT rowid FROM t2($v) ORDER BY rowid DESC }] if {$l1!=$l2} { error "1: query mismatch ($l1) ($l2)" } } set {} {} } {}] } proc lshuffle {in} { set L [list] set ret [list] foreach elem $in { lappend L [list [expr rand()] $elem] } foreach pair [lsort -index 0 $L] { lappend ret [lindex $pair 1] } set ret } expr srand(0) do_execsql_test 1.0 { CREATE VIRTUAL TABLE t2 USING fts5( doc, prefix=2, content=, contentless_delete=1 ); CREATE TABLE t1(doc); CREATE TRIGGER tr1 AFTER DELETE ON t1 BEGIN DELETE FROM t2 WHERE rowid = old.rowid; END; } set SMALLEST64 -9223372036854775808 set LARGEST64 9223372036854775807 foreach {tn r1 r2} { 1 0 50 2 $SMALLEST64 $SMALLEST64+50 3 $LARGEST64-50 $LARGEST64 4 -50 -1 } { set r1 [expr $r1] set r2 [expr $r2] do_test 1.1.$tn { execsql BEGIN for {set ii $r1} {$ii <= $r2} {incr ii} { execsql { INSERT INTO t1(rowid, doc) VALUES ($ii, document(8)); } } execsql COMMIT } {} } do_test 1.2 { db eval { SELECT rowid, doc FROM t1 } { execsql { INSERT INTO t2(rowid, doc) VALUES($rowid, $doc) } } } {} foreach {tn rowid} { 1 $SMALLEST64 2 0 3 -5 4 -30 5 $LARGEST64 6 $LARGEST64-1 } { set rowid [expr $rowid] do_execsql_test 1.3.$tn.1 { DELETE FROM t1 WHERE rowid=$rowid } do_compare_tables_test 1.3.$tn.2 } set iTest 1 foreach r [lshuffle [execsql {SELECT rowid FROM t1}]] { if {($iTest % 50)==0} { execsql { INSERT INTO t2(t2) VALUES('optimize') } } if {($iTest % 5)==0} { execsql { INSERT INTO t2(t2, rank) VALUES('merge', 5) } } do_execsql_test 1.4.$iTest.1($r) { DELETE FROM t1 WHERE rowid=$r } do_compare_tables_test 1.4.$iTest.2 incr iTest } do_execsql_test 1.5 { SELECT * FROM t1 } {} #------------------------------------------------------------------------- reset_db db func document document do_execsql_test 2.0 { CREATE VIRTUAL TABLE t2 USING fts5(doc, content=, contentless_delete=1); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000 ) INSERT INTO t2(rowid, doc) SELECT i, i || ' ' || i FROM s; } do_execsql_test 2.1 { BEGIN; DELETE FROM t2 WHERE rowid=32; DELETE FROM t2 WHERE rowid=64; DELETE FROM t2 WHERE rowid=96; DELETE FROM t2 WHERE rowid=128; DELETE FROM t2 WHERE rowid=160; DELETE FROM t2 WHERE rowid=192; COMMIT; } do_execsql_test 2.2 { SELECT * FROM t2('128'); } {} #------------------------------------------------------------------------- foreach {tn step} { 1 3 2 7 3 15 } { set step [expr $step] reset_db db func document document do_execsql_test 3.$tn.0 { CREATE VIRTUAL TABLE t2 USING fts5(doc, content=, contentless_delete=1); INSERT INTO t2(t2, rank) VALUES('pgsz', 100); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000 ) INSERT INTO t2(rowid, doc) SELECT i, i || ' ' || i FROM s; } do_execsql_test 3.$tn.1 { DELETE FROM t2 WHERE (rowid % $step)==0 } do_execsql_test 3.$tn.2 { SELECT * FROM t2( $step * 5 ) } {} } finish_test |
Added ext/fts5/test/fts5contentless3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 | # 2023 July 21 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5contentless3 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1); BEGIN; INSERT INTO ft VALUES('one one one'); INSERT INTO ft VALUES('two two two'); INSERT INTO ft VALUES('three three three'); INSERT INTO ft VALUES('four four four'); INSERT INTO ft VALUES('five five five'); INSERT INTO ft VALUES('six six six'); INSERT INTO ft VALUES('seven seven seven'); INSERT INTO ft VALUES('eight eight eight'); INSERT INTO ft VALUES('nine nine nine'); COMMIT; DELETE FROM ft WHERE rowid=3; } proc myhex {hex} { binary decode hex $hex } db func myhex myhex do_execsql_test 1.1 { UPDATE ft_data SET block = myhex('04000000 00000001' || '01020304 01020304 01020304 01020304' || '01020304 01020304 01020304 01020304' ) WHERE id = (SELECT max(id) FROM ft_data); } do_execsql_test 1.2 { DELETE FROM ft WHERE rowid=1 } do_execsql_test 1.3 { SELECT rowid FROM ft('two'); } {2} do_execsql_test 1.3 { UPDATE ft_data SET block = myhex('08000000 00000001' || '0000000001020304 0000000001020304 0000000001020304 0000000001020304' || '0000000001020304 0000000001020304 0000000001020304 0000000001020304' ) WHERE id = (SELECT max(id) FROM ft_data); } do_execsql_test 1.4 { SELECT rowid FROM ft('two'); } {2} do_execsql_test 1.5 { DELETE FROM ft WHERE rowid=4 } do_execsql_test 1.6 { UPDATE ft_data SET block = myhex('04000000 00000000') WHERE id = (SELECT max(id) FROM ft_data); } do_execsql_test 1.7 { SELECT rowid FROM ft('two'); } {2} do_execsql_test 1.8 { UPDATE ft_data SET block = myhex('04000000 00000000') WHERE id = (SELECT max(id) FROM ft_data); } do_execsql_test 1.9 { DELETE FROM ft WHERE rowid=8 } {} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1); INSERT INTO ft VALUES('one one one'); INSERT INTO ft VALUES('two two two'); INSERT INTO ft VALUES('three three three'); INSERT INTO ft VALUES('four four four'); INSERT INTO ft VALUES('five five five'); INSERT INTO ft VALUES('six six six'); INSERT INTO ft VALUES('seven seven seven'); INSERT INTO ft VALUES('eight eight eight'); INSERT INTO ft VALUES('nine nine nine'); } do_execsql_test 2.1 { INSERT INTO ft(ft) VALUES('optimize'); } do_execsql_test 2.2 { SELECT count(*) FROM ft_data } {3} do_execsql_test 2.3 { DELETE FROM ft WHERE rowid=5 } do_execsql_test 2.4 { SELECT count(*) FROM ft_data } {4} # Check that an 'optimize' works (rewrites the index) if there is a single # segment with one or more tombstone hash pages. do_execsql_test 2.5 { INSERT INTO ft(ft) VALUES('optimize'); } do_execsql_test 2.6 { SELECT count(*) FROM ft_data } {3} # Check that an 'optimize' is a no-op if there is a single segment # and no tombstone hash pages. do_execsql_test 2.7 { INSERT INTO ft(ft) VALUES('optimize'); SELECT rowid FROM ft_data; } [db eval {SELECT rowid FROM ft_data}] #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content=, contentless_delete=1); INSERT INTO ft(ft, rank) VALUES('pgsz', 64); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000 ) INSERT INTO ft(rowid, x) SELECT i, i||' '||i||' '||i||' '||i FROM s; INSERT INTO ft(ft) VALUES('optimize'); } do_execsql_test 3.1 { SELECT count(*) FROM ft_data } {200} do_execsql_test 3.2 { DELETE FROM ft WHERE (rowid % 50)==0; SELECT count(*) FROM ft_data; } {203} do_execsql_test 3.3 { INSERT INTO ft(ft, rank) VALUES('merge', 500); SELECT rowid FROM ft_data; } [db eval {SELECT rowid FROM ft_data}] do_execsql_test 3.4 { INSERT INTO ft(ft, rank) VALUES('merge', -1000); SELECT count(*) FROM ft_data; } {197} do_execsql_test 3.5 { DELETE FROM ft WHERE (rowid % 50)==1; SELECT count(*) FROM ft_data; } {200} do_execsql_test 3.6 { SELECT level, segment, npgtombstone FROM fts5_structure( (SELECT block FROM ft_data WHERE id=10) ) } {1 0 3} do_test 3.6 { while 1 { set nChange [db total_changes] execsql { INSERT INTO ft(ft, rank) VALUES('merge', -5) } if {([db total_changes] - $nChange)<2} break } } {} do_execsql_test 3.7 { SELECT level, segment, npgtombstone FROM fts5_structure( (SELECT block FROM ft_data WHERE id=10) ) } {2 0 0} finish_test |
Added ext/fts5/test/fts5contentless4.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 | # 2023 July 21 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5contentless4 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc document {n} { set vocab [list A B C D E F G H I J K L M N O P Q R S T U V W X Y Z] set ret [list] for {set ii 0} {$ii < $n} {incr ii} { lappend ret [lindex $vocab [expr int(rand()*[llength $vocab])]] } set ret } db func document document do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); INSERT INTO ft(ft, rank) VALUES('pgsz', 240); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000 ) INSERT INTO ft SELECT document(12) FROM s; } do_execsql_test 1.1 { INSERT INTO ft(ft) VALUES('optimize'); } do_execsql_test 1.2 { SELECT level, segment, nentry, nentrytombstone FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {0 0 1000 0} do_execsql_test 1.3 { DELETE FROM ft WHERE rowid < 50 } do_execsql_test 1.4 { SELECT level, segment, nentry, nentrytombstone FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {0 0 1000 49} do_execsql_test 1.5 { DELETE FROM ft WHERE rowid < 1000 } do_execsql_test 1.6 { SELECT level, segment, nentry, nentrytombstone FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {1 0 1 0} #-------------------------------------------------------------------------- reset_db db func document document do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); } do_test 2.1 { for {set ii 0} {$ii < 5000} {incr ii} { execsql { INSERT INTO ft VALUES( document(12) ) } } } {} do_execsql_test 2.2 { SELECT sum(nentry) - sum(nentrytombstone) FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {5000} for {set ii 5000} {$ii >= 0} {incr ii -100} { do_execsql_test 2.3.$ii { DELETE FROM ft WHERE rowid > $ii } do_execsql_test 2.3.$ii.2 { SELECT CAST((total(nentry) - total(nentrytombstone)) AS integer) FROM fts5_structure( (SELECT block FROM ft_data WHERE id=10) ) } $ii } execsql_pp { SELECT * FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } do_test 2.4 { for {set ii 0} {$ii < 5000} {incr ii} { execsql { INSERT INTO ft VALUES( document(12) ) } } } {} for {set ii 1} {$ii <= 5000} {incr ii 10} { do_execsql_test 2.3.$ii { DELETE FROM ft WHERE rowid = $ii; INSERT INTO ft VALUES( document(12) ); INSERT INTO ft(ft, rank) VALUES('merge', -10); } do_execsql_test 2.3.$ii.2 { SELECT CAST((total(nentry) - total(nentrytombstone)) AS integer) FROM fts5_structure( (SELECT block FROM ft_data WHERE id=10) ) } 5000 } #------------------------------------------------------------------------- reset_db db func document document do_execsql_test 3.0 { CREATE VIRTUAL TABLE ft USING fts5(x, content='', contentless_delete=1); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO ft SELECT document(12) FROM s; } do_catchsql_test 3.1 { INSERT INTO ft(ft, rank) VALUES('deletemerge', 'text'); } {1 {SQL logic error}} do_catchsql_test 3.2 { INSERT INTO ft(ft, rank) VALUES('deletemerge', 50); } {0 {}} do_execsql_test 3.3 { SELECT * FROM ft_config WHERE k='deletemerge' } {deletemerge 50} do_catchsql_test 3.4 { INSERT INTO ft(ft, rank) VALUES('deletemerge', 101); } {0 {}} do_execsql_test 3.5 { SELECT * FROM ft_config WHERE k='deletemerge' } {deletemerge 101} do_execsql_test 3.6 { DELETE FROM ft WHERE rowid<95 } do_execsql_test 3.7 { SELECT nentrytombstone, nentry FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {94 100} do_execsql_test 3.8 { DELETE FROM ft WHERE rowid=95 } do_execsql_test 3.9 { SELECT nentrytombstone, nentry FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {95 100} do_execsql_test 3.10 { DELETE FROM ft; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO ft SELECT document(12) FROM s; INSERT INTO ft(ft, rank) VALUES('deletemerge', 50); } do_execsql_test 3.11 { DELETE FROM ft WHERE rowid<95 } do_execsql_test 3.12 { SELECT nentrytombstone, nentry FROM fts5_structure(( SELECT block FROM ft_data WHERE id=10 )) } {0 6} #------------------------------------------------------------------------- reset_db db func document document do_execsql_test 4.0 { CREATE VIRTUAL TABLE x1 USING fts5(x, content='', contentless_delete=1); INSERT INTO x1(x1, rank) VALUES('usermerge', 16); INSERT INTO x1(x1, rank) VALUES('deletemerge', 40); INSERT INTO x1 VALUES('one'); INSERT INTO x1 VALUES('two'); INSERT INTO x1 VALUES('three'); INSERT INTO x1 VALUES('four'); INSERT INTO x1 VALUES('five'); INSERT INTO x1 VALUES('six'); INSERT INTO x1 VALUES('seven'); INSERT INTO x1 VALUES('eight'); INSERT INTO x1 VALUES('nine'); INSERT INTO x1 VALUES('ten'); } do_execsql_test 4.1 { SELECT level, segment FROM fts5_structure(( SELECT block FROM x1_data WHERE id=10 )) } { 0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 } for {set ii 1} {$ii < 4} {incr ii} { do_execsql_test 4.2.$ii { DELETE FROM x1 WHERE rowid = $ii; INSERT INTO x1(x1, rank) VALUES('merge', 5); SELECT level, segment FROM fts5_structure(( SELECT block FROM x1_data WHERE id=10 )) } { 0 0 0 1 0 2 0 3 0 4 0 5 0 6 0 7 0 8 0 9 } } do_execsql_test 4.3 { DELETE FROM x1 WHERE rowid = $ii; INSERT INTO x1(x1, rank) VALUES('merge', 5); SELECT level, segment, nentry FROM fts5_structure(( SELECT block FROM x1_data WHERE id=10 )) } { 1 0 6 } finish_test |
Added ext/fts5/test/fts5contentless5.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | # 2023 August 7 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests for the content= and content_rowid= options. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5contentless5 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } unset -nocomplain res do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, c, content='', contentless_delete=1); INSERT INTO t1 VALUES('A', 'B', 'C'); INSERT INTO t1 VALUES('D', 'E', 'F'); INSERT INTO t1 VALUES('G', 'H', 'I'); } do_execsql_test 1.01 { CREATE TABLE t2(x, y); INSERT INTO t2 VALUES('x', 'y'); } # explain_i "UPDATE t1 SET a='a' WHERE t1.rowid=1" #breakpoint #explain_i "UPDATE t1 SET a='a' FROM t2 WHERE t1.rowid=1 AND b IS NULL" #breakpoint #explain_i "UPDATE t1 SET a='a' WHERE b IS NULL AND rowid=?" foreach {tn up err} { 1 "UPDATE t1 SET a='a', b='b', c='c' WHERE rowid=1" 0 2 "UPDATE t1 SET a='a', b='b' WHERE rowid=1" 1 3 "UPDATE t1 SET b='b', c='c' WHERE rowid=1" 1 4 "UPDATE t1 SET a='a', c='c' WHERE rowid=1" 1 5 "UPDATE t1 SET a='a', c='c' WHERE t1.rowid=1 AND b IS NULL" 1 6 "UPDATE t1 SET a='a' FROM t2 WHERE t1.rowid=1" 1 7 "UPDATE t1 SET a='a', b='b', c='c' FROM t2 WHERE t1.rowid=1" 0 } { set res(0) {0 {}} set res(1) {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: t1}} do_catchsql_test 1.$tn $up $res($err) } #------------------------------------------------------------------------- reset_db proc random {n} { expr {abs(int(rand()*$n))} } proc select_one {list} { set n [llength $list] lindex $list [random $n] } proc vocab {} { list abc def ghi jkl mno pqr stu vwx yza } proc term {} { select_one [vocab] } proc document {} { set nTerm [expr [random 3] + 7] set doc "" for {set ii 0} {$ii < $nTerm} {incr ii} { lappend doc [term] } set doc } db func document document do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft USING fts5(a, contentless_delete=1, content=''); INSERT INTO ft(ft, rank) VALUES('pgsz', 64); } do_test 2.1 { for {set ii 1} {$ii < 12} {incr ii} { db transaction { for {set jj 0} {$jj < 10} {incr jj} { set doc [document] execsql { INSERT INTO ft VALUES($doc); } } } } } {} do_test 2.2 { foreach r [db eval {SELECT rowid FROM ft}] { execsql { DELETE FROM ft WHERE rowid=$r } } } {} set doc [document] do_execsql_test 2.3 { INSERT INTO ft VALUES($doc) } finish_test |
Changes to ext/fts5/test/fts5corrupt.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # This file tests that the FTS5 'integrity-check' command detects # inconsistencies (corruption) in the on-disk backing tables. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # This file tests that the FTS5 'integrity-check' command detects # inconsistencies (corruption) in the on-disk backing tables. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); |
︙ | ︙ | |||
43 44 45 46 47 48 49 | sqlite3_db_config db DEFENSIVE 0 do_test 1.3 { execsql { DELETE FROM t1_data WHERE rowid = fts5_rowid('segment', $segid, 4); } catchsql { INSERT INTO t1(t1) VALUES('integrity-check') } | | > > > | | 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | sqlite3_db_config db DEFENSIVE 0 do_test 1.3 { execsql { DELETE FROM t1_data WHERE rowid = fts5_rowid('segment', $segid, 4); } catchsql { INSERT INTO t1(t1) VALUES('integrity-check') } } {1 {fts5: corruption found reading blob 137438953476 from table "t1"}} do_execsql_test 1.3b { PRAGMA integrity_check(t1); } {{fts5: corruption found reading blob 137438953476 from table "t1"}} do_test 1.4 { db_restore_and_reopen sqlite3_db_config db DEFENSIVE 0 execsql { UPDATE t1_data set block = X'00000000' || substr(block, 5) WHERE rowid = fts5_rowid('segment', $segid, 4); } catchsql { INSERT INTO t1(t1) VALUES('integrity-check') } } {1 {fts5: corruption found reading blob 137438953476 from table "t1"}} db_restore_and_reopen #db eval {SELECT rowid, fts5_decode(rowid, block) aS r FROM t1_data} {puts $r} #-------------------------------------------------------------------- # |
︙ | ︙ | |||
91 92 93 94 95 96 97 98 99 100 | do_execsql_test 3.1 { SELECT * FROM t3 WHERE t3 MATCH 'o' } {{one o} {three o} {five o}} sqlite3_db_config db DEFENSIVE 0 do_catchsql_test 3.1 { DELETE FROM t3_content WHERE rowid = 3; SELECT * FROM t3 WHERE t3 MATCH 'o'; } {1 {database disk image is malformed}} finish_test | > > > > > > > > > > > > > > > > > > > > > | 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | do_execsql_test 3.1 { SELECT * FROM t3 WHERE t3 MATCH 'o' } {{one o} {three o} {five o}} sqlite3_db_config db DEFENSIVE 0 do_catchsql_test 3.1 { DELETE FROM t3_content WHERE rowid = 3; SELECT * FROM t3 WHERE t3 MATCH 'o'; } {1 {fts5: missing row 3 from content table 'main'.'t3_content'}} #-------------------------------------------------------------------- # reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE t2 USING fts5(x); INSERT INTO t2 VALUES('one two three'); INSERT INTO t2 VALUES('four five six'); INSERT INTO t2 VALUES('seven eight nine'); INSERT INTO t2 VALUES('ten eleven twelve'); } do_execsql_test 4.1 { SELECT hex(block) FROM t2_data WHERE id=1; } {040C} do_execsql_test 4.2 { UPDATE t2_data SET block = X'0402' WHERE id=1 } breakpoint do_catchsql_test 4.3 { DELETE FROM t2 WHERE rowid=3 } {1 {database disk image is malformed}} finish_test |
Changes to ext/fts5/test/fts5corrupt2.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt2 | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 # Create a simple FTS5 table containing 100 documents. Each document |
︙ | ︙ | |||
96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | # corruption. It may not report the db as corrupt because truncating the # final leaf to some sizes may create a valid leaf page. # set lrowid [db one {SELECT max(rowid) FROM t1_data WHERE (rowid & $mask)=0}] set nbyte [db one {SELECT length(block) FROM t1_data WHERE rowid=$lrowid}] set all [db eval {SELECT rowid FROM t1}] sqlite3_db_config db DEFENSIVE 0 for {set i [expr $nbyte-2]} {$i>=0} {incr i -1} { do_execsql_test 2.$i.1 { BEGIN; UPDATE t1_data SET block = substr(block, 1, $i) WHERE rowid=$lrowid; } do_catchsql_test 2.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check'); | > | | | 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | # corruption. It may not report the db as corrupt because truncating the # final leaf to some sizes may create a valid leaf page. # set lrowid [db one {SELECT max(rowid) FROM t1_data WHERE (rowid & $mask)=0}] set nbyte [db one {SELECT length(block) FROM t1_data WHERE rowid=$lrowid}] set all [db eval {SELECT rowid FROM t1}] sqlite3_db_config db DEFENSIVE 0 unset -nocomplain res for {set i [expr $nbyte-2]} {$i>=0} {incr i -1} { do_execsql_test 2.$i.1 { BEGIN; UPDATE t1_data SET block = substr(block, 1, $i) WHERE rowid=$lrowid; } do_catchsql_test 2.$i.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/1.*fts5: corruption.*/} do_test 2.$i.3 { set res [catchsql {SELECT rowid FROM t1 WHERE t1 MATCH 'x*'}] expr { [string match {*fts5: corruption*} $res] || $res=="0 {$all}" } } 1 do_execsql_test 2.$i.4 { ROLLBACK; INSERT INTO t1(t1) VALUES('integrity-check'); |
︙ | ︙ | |||
148 149 150 151 152 153 154 | foreach rowid [db eval {SELECT rowid FROM x3_data WHERE rowid>10}] { if {$rowid & $mask} continue incr tn2 do_test 3.$tn.$tn2.1 { execsql BEGIN set fd [db incrblob main x3_data block $rowid] | | | | > > > | 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 | foreach rowid [db eval {SELECT rowid FROM x3_data WHERE rowid>10}] { if {$rowid & $mask} continue incr tn2 do_test 3.$tn.$tn2.1 { execsql BEGIN set fd [db incrblob main x3_data block $rowid] fconfigure $fd -translation binary set existing [read $fd [string length $hdr]] seek $fd 0 puts -nonewline $fd $hdr close $fd set res [catchsql {SELECT rowid FROM x3 WHERE x3 MATCH 'x AND a'}] if {[string match {*fts5: corruption*} $res]} {incr nCorrupt} set {} 1 } {1} if {($tn2 % 10)==0 && $existing != $hdr} { do_test 3.$tn.$tn2.2 { catchsql { INSERT INTO x3(x3) VALUES('integrity-check') } } {/.*fts5: corruption.*/} do_execsql_test 3.$tn.$tn2.3 { PRAGMA integrity_check(x3); } {/.*fts5: corruption.*/} } execsql ROLLBACK } do_test 3.$tn.x { expr $nCorrupt>0 } 1 } |
︙ | ︙ | |||
201 202 203 204 205 206 207 | UPDATE x4_data SET block = substr(block, 1, length(block)-$nCut) WHERE id = $rowid; } set res [catchsql { SELECT rowid FROM x4 WHERE x4 MATCH 'a' ORDER BY 1 DESC }] | | | 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 | UPDATE x4_data SET block = substr(block, 1, length(block)-$nCut) WHERE id = $rowid; } set res [catchsql { SELECT rowid FROM x4 WHERE x4 MATCH 'a' ORDER BY 1 DESC }] if {[string match {*fts5: corruption*} $res]} {incr nCorrupt} set {} 1 } {1} execsql ROLLBACK } # do_test 4.$tn.x { expr $nCorrupt>0 } 1 |
︙ | ︙ | |||
231 232 233 234 235 236 237 | foreach rowid [db eval {SELECT rowid FROM x5_data WHERE rowid>10}] { if {$rowid & $mask} continue incr tn2 do_test 5.$tn.$tn2 { execsql BEGIN set fd [db incrblob main x5_data block $rowid] | | | 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 | foreach rowid [db eval {SELECT rowid FROM x5_data WHERE rowid>10}] { if {$rowid & $mask} continue incr tn2 do_test 5.$tn.$tn2 { execsql BEGIN set fd [db incrblob main x5_data block $rowid] fconfigure $fd -translation binary puts -nonewline $fd $hdr close $fd catchsql { INSERT INTO x5(x5) VALUES('integrity-check') } set {} {} } {} |
︙ | ︙ |
Changes to ext/fts5/test/fts5corrupt3.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt3 | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt3 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 database_may_be_corrupt |
︙ | ︙ | |||
98 99 100 101 102 103 104 | incr i do_test $tn.$i { db eval BEGIN db eval {DELETE FROM t1_data WHERE rowid = $::rowid} list [ catch { db eval {SELECT rowid FROM t1 WHERE t1 MATCH 'x*'} } msg ] $msg | | | 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | incr i do_test $tn.$i { db eval BEGIN db eval {DELETE FROM t1_data WHERE rowid = $::rowid} list [ catch { db eval {SELECT rowid FROM t1 WHERE t1 MATCH 'x*'} } msg ] $msg } {/.*fts5: corruption.*/} catch { db eval ROLLBACK } } } do_3_test 3.2 do_execsql_test 3.3 { |
︙ | ︙ | |||
269 270 271 272 273 274 275 | do_execsql_test 6.1.1 { UPDATE t1_data SET block = X'000000180630626262626201020201056161616161010203040A' WHERE rowid>100; } do_catchsql_test 6.1.2 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | | | | | | 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 | do_execsql_test 6.1.1 { UPDATE t1_data SET block = X'000000180630626262626201020201056161616161010203040A' WHERE rowid>100; } do_catchsql_test 6.1.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corruption.*/} #------- reset_db sqlite3_db_config db DEFENSIVE 0 do_execsql_test 6.2.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); INSERT INTO t1 VALUES('aa bb cc dd ee'); SELECT pgno, quote(term) FROM t1_idx; } {2 X'' 4 X'3064'} do_execsql_test 6.2.1 { UPDATE t1_idx SET term = X'3065' WHERE pgno=4; } do_catchsql_test 6.2.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corruption.*/} #------- reset_db sqlite3_db_config db DEFENSIVE 0 do_execsql_test 6.3.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); INSERT INTO t1 VALUES('abc abcdef abcdefghi'); SELECT quote(block) FROM t1_data WHERE id>100; } {X'0000001C043061626301020204036465660102030703676869010204040808'} do_execsql_test 6.3.1 { BEGIN; UPDATE t1_data SET block = X'0000001C043061626301020204036465660102035003676869010204040808' ------------------------------------------^^--------------------- WHERE id>100; } do_catchsql_test 6.3.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corruption.*/} do_execsql_test 6.3.3 { ROLLBACK; BEGIN; UPDATE t1_data SET block = X'0000001C043061626301020204036465660102030750676869010204040808' --------------------------------------------^^------------------- WHERE id>100; } do_catchsql_test 6.3.3 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corruption.*/} do_execsql_test 6.3.4 { ROLLBACK; BEGIN; UPDATE t1_data SET block = X'0000001C043061626301020204036465660102030707676869010204040850' --------------------------------------------------------------^^- WHERE id>100; } do_catchsql_test 6.3.5 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corruption.*/} do_execsql_test 6.3.6 { ROLLBACK; BEGIN; UPDATE t1_data SET block = X'0000001C503061626301020204036465660102030707676869010204040808' ----------^^----------------------------------------------------- WHERE id>100; } do_catchsql_test 6.3.5 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corruption.*/} #------------------------------------------------------------------------ # reset_db proc rnddoc {n} { set map [list a b c d] |
︙ | ︙ | |||
370 371 372 373 374 375 376 | sqlite3_db_config db DEFENSIVE 0 do_test 7.1 { foreach i [db eval { SELECT rowid FROM t5_data WHERE rowid>100 }] { db eval BEGIN db eval {DELETE FROM t5_data WHERE rowid = $i} set r [catchsql { INSERT INTO t5(t5) VALUES('integrity-check')} ] | | | 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 | sqlite3_db_config db DEFENSIVE 0 do_test 7.1 { foreach i [db eval { SELECT rowid FROM t5_data WHERE rowid>100 }] { db eval BEGIN db eval {DELETE FROM t5_data WHERE rowid = $i} set r [catchsql { INSERT INTO t5(t5) VALUES('integrity-check')} ] if {![string match {*fts5: corruption*} $r]} { error $r } db eval ROLLBACK } } {} #------------------------------------------------------------------------ # Corruption within the structure record. # |
︙ | ︙ | |||
395 396 397 398 399 400 401 | append blob "06" ;# write counter append blob "0002" ;# first level has 0 segments merging, 2 other. append blob "450108" ;# first segment execsql "REPLACE INTO t1_data VALUES(10, X'$blob')" } {} do_catchsql_test 9.1.2 { SELECT * FROM t1('one AND two'); | | | | 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 | append blob "06" ;# write counter append blob "0002" ;# first level has 0 segments merging, 2 other. append blob "450108" ;# first segment execsql "REPLACE INTO t1_data VALUES(10, X'$blob')" } {} do_catchsql_test 9.1.2 { SELECT * FROM t1('one AND two'); } {/.*fts5: corrupt.*/} do_test 9.2.1 { set blob "12345678" ;# cookie append blob "0205" ;# 2 levels, total of 5 segments append blob "06" ;# write counter append blob "0001" ;# first level has 0 segments merging, 1 other. append blob "450108" ;# first segment execsql "REPLACE INTO t1_data VALUES(10, X'$blob')" } {} do_catchsql_test 9.2.2 { SELECT * FROM t1('one AND two'); } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 10.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 32768 pagesize 4096 filename c9.db |
︙ | ︙ | |||
493 494 495 496 497 498 499 | | 32: 65 6e 74 29 0d 00 00 00 03 0f bd 00 0f e8 0f ef ent)............ | 48: 0f bd 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | end c9.db }] } {} do_catchsql_test 10.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; | | | 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 | | 32: 65 6e 74 29 0d 00 00 00 03 0f bd 00 0f e8 0f ef ent)............ | 48: 0f bd 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | end c9.db }] } {} do_catchsql_test 10.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- # reset_db do_test 11.0 { sqlite3 db {} db deserialize [decode_hexdb { |
︙ | ︙ | |||
674 675 676 677 678 679 680 | | 0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00 ................ | 4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c ..........rebuil | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c2.db }]} {} | | | | | | 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 | | 0: 0d 00 00 00 03 0f d6 00 0f f4 0f e1 0f d6 00 00 ................ | 4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c ..........rebuil | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c2.db }]} {} do_catchsql_test 12.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; } {/.*fts5: corrupt.*/} do_catchsql_test 12.2 { INSERT INTO t1(t1, rank) VALUES('merge', 500); } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- # reset_db do_test 13.0 { sqlite3 db {} db deserialize [decode_hexdb { |
︙ | ︙ | |||
866 867 868 869 870 871 872 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c14b.db }]} {} do_catchsql_test 14.1 { INSERT INTO t1(t1) VALUES('optimize'); | | | 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c14b.db }]} {} do_catchsql_test 14.1 { INSERT INTO t1(t1) VALUES('optimize'); } {/.*fts5: corrupt.*/} #--------------------------------------------------------------------------- # reset_db do_test 15.0 { sqlite3 db {} db deserialize [decode_hexdb { |
︙ | ︙ | |||
1036 1037 1038 1039 1040 1041 1042 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c17.db }]} {} do_catchsql_test 16.1 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c17.db }]} {} do_catchsql_test 16.1 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 17.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename c18.db |
︙ | ︙ | |||
1122 1123 1124 1125 1126 1127 1128 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c18.db }]} {} do_catchsql_test 17.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; | | | 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c18.db }]} {} do_catchsql_test 17.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 18.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename c19b.db |
︙ | ︙ | |||
1431 1432 1433 1434 1435 1436 1437 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c19b.db }]} {} do_catchsql_test 18.1 { INSERT INTO t1(t1) VALUES('optimize'); | | | 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c19b.db }]} {} do_catchsql_test 18.1 { INSERT INTO t1(t1) VALUES('optimize'); } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 19.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename c20b.db |
︙ | ︙ | |||
1542 1543 1544 1545 1546 1547 1548 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 86 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 .eck....optimize | end c20b.db }]} {} do_catchsql_test 19.1 { INSERT INTO t1(t1) VALUES('optimize'); | | | 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 86 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 .eck....optimize | end c20b.db }]} {} do_catchsql_test 19.1 { INSERT INTO t1(t1) VALUES('optimize'); } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 20.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename crash-cf347c523f793c.db |
︙ | ︙ | |||
1626 1627 1628 1629 1630 1631 1632 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end crash-cf347c523f793c.db }]} {} do_catchsql_test 20.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; | | | 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end crash-cf347c523f793c.db }]} {} do_catchsql_test 20.1 { SELECT * FROM t1 WHERE t1 MATCH 'abandon'; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 21.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename c22b.db |
︙ | ︙ | |||
1760 1761 1762 1763 1764 1765 1766 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c22b.db }]} {} do_catchsql_test 21.1 { DELETE FROM t1 WHERE t1 MATCH 'ab*ndon'; | | | 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c22b.db }]} {} do_catchsql_test 21.1 { DELETE FROM t1 WHERE t1 MATCH 'ab*ndon'; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- # reset_db do_test 22.0 { sqlite3 db {} db deserialize [decode_hexdb { |
︙ | ︙ | |||
2096 2097 2098 2099 2100 2101 2102 | | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c22b.db }]} {} do_catchsql_test 22.1 { INSERT INTO t1(t1) VALUES('optimize'); | | | 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 | | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c22b.db }]} {} do_catchsql_test 22.1 { INSERT INTO t1(t1) VALUES('optimize'); } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 23.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename c24b.db |
︙ | ︙ | |||
2207 2208 2209 2210 2211 2212 2213 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c24b.db }]} {} do_catchsql_test 23.1 { INSERT INTO t1(t1) VALUES('optimize'); | | | 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end c24b.db }]} {} do_catchsql_test 23.1 { INSERT INTO t1(t1) VALUES('optimize'); } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 24.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 32768 pagesize 4096 filename crash-b87dfef02880fe.db |
︙ | ︙ | |||
2425 2426 2427 2428 2429 2430 2431 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 uild....optimize | end crash-b87dfef02880fe.db }]} {} do_catchsql_test 24.1 { UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thread*'; | | | 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 uild....optimize | end crash-b87dfef02880fe.db }]} {} do_catchsql_test 24.1 { UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thread*'; } {/.*fts5: corrupt.*/} do_catchsql_test 24.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {1 {database disk image is malformed}} #-------------------------------------------------------------------------- reset_db |
︙ | ︙ | |||
2514 2515 2516 2517 2518 2519 2520 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end crash-e3b1b19e4d4bcc.db }]} {} do_catchsql_test 25.1 { INSERT INTO t1(t1) VALUES('rebuild'); | | | 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end crash-e3b1b19e4d4bcc.db }]} {} do_catchsql_test 25.1 { INSERT INTO t1(t1) VALUES('rebuild'); } {/.*fts5: corrupt.*/} do_execsql_test 25.2 { PRAGMA page_size=512; } #-------------------------------------------------------------------------- reset_db |
︙ | ︙ | |||
3007 3008 3009 3010 3011 3012 3013 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 uild....optimize | end timeout-2ca5b0658c98.db }]} {} do_catchsql_test 27.1 { DELETE FROM t1 WHERE a MATCH 'fts*'; | | | 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 uild....optimize | end timeout-2ca5b0658c98.db }]} {} do_catchsql_test 27.1 { DELETE FROM t1 WHERE a MATCH 'fts*'; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 28.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 40960 pagesize 4096 filename crash-e2d47e0624a42c.db |
︙ | ︙ | |||
3696 3697 3698 3699 3700 3701 3702 | }]} {} do_catchsql_test 32.1 { SELECT snippet(t1, -1, '.', '..', '[', ']'), highlight(t1, 2, '[', ']') FROM t1('g + h') WHERE rank MATCH 'bm25(1.0, 1.0)' ORDER BY rank; | | | 3696 3697 3698 3699 3700 3701 3702 3703 3704 3705 3706 3707 3708 3709 3710 | }]} {} do_catchsql_test 32.1 { SELECT snippet(t1, -1, '.', '..', '[', ']'), highlight(t1, 2, '[', ']') FROM t1('g + h') WHERE rank MATCH 'bm25(1.0, 1.0)' ORDER BY rank; } {/.*fts5: corrupt.*/} do_catchsql_test 32.2 { SELECT * FROM t3; } {1 {database disk image is malformed}} do_catchsql_test 32.3 { SELECT * FROM t4; |
︙ | ︙ | |||
4263 4264 4265 4266 4267 4268 4269 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 uild....optimize | end crash-ae135cb10977c7.db }]} {} do_catchsql_test 35.1 { SELECT * FROM t1 WHERE t1 MATCH 'e*'; | | | 4263 4264 4265 4266 4267 4268 4269 4270 4271 4272 4273 4274 4275 4276 4277 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 uild....optimize | end crash-ae135cb10977c7.db }]} {} do_catchsql_test 35.1 { SELECT * FROM t1 WHERE t1 MATCH 'e*'; } {1 {fts5: missing row 14 from content table 'main'.'t1_content'}} #------------------------------------------------------------------------- reset_db do_test 36.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 24576 pagesize 4096 filename crash-a6651222df1bd1.db |
︙ | ︙ | |||
5347 5348 5349 5350 5351 5352 5353 | INSERT INTO t1_data VALUES(274877906945,X'00000183023030250601011d010331c2ba250601010d0101342506010121010135250601011e02036ec2ba250601012b0101382506010112010161250a0101021a1d02016f2506010111010162250601013201026377250601012f010166250801011f0c010167250601012701026863250601010f02026473250601013002016b2506010133020175250601010e010169250601012c0204386ec2be250601012001016a250601010401056bc2b2cebc250601010901016c25060101150203cebc71250601011301036dd18a250601010c01016f25060101260102706425060101240101712506010122010173250a010116040d02016f2506010134010175250801011b14020161250601010b010376c2aa25060101100202d7ac250601010601017725060101030201752506010114010179250a0101190e050202357a250601010701017a250601012e0102c2aa250801011c100201b3250601010a0202ba6225060101310203be656625060101080103c5a77425060101050102de8e250601011704080a08080a080a080809090809090808080b080c080a0a0809080a0809080a0908080a09080a08090a0a'); INSERT INTO t1_idx VALUES(1,X'',2); INSERT INTO t1_idx VALUES(2,X'',2); } do_catchsql_test 41.1 { INSERT INTO t1(t1) VALUES('optimize'); | | | 5347 5348 5349 5350 5351 5352 5353 5354 5355 5356 5357 5358 5359 5360 5361 | INSERT INTO t1_data VALUES(274877906945,X'00000183023030250601011d010331c2ba250601010d0101342506010121010135250601011e02036ec2ba250601012b0101382506010112010161250a0101021a1d02016f2506010111010162250601013201026377250601012f010166250801011f0c010167250601012701026863250601010f02026473250601013002016b2506010133020175250601010e010169250601012c0204386ec2be250601012001016a250601010401056bc2b2cebc250601010901016c25060101150203cebc71250601011301036dd18a250601010c01016f25060101260102706425060101240101712506010122010173250a010116040d02016f2506010134010175250801011b14020161250601010b010376c2aa25060101100202d7ac250601010601017725060101030201752506010114010179250a0101190e050202357a250601010701017a250601012e0102c2aa250801011c100201b3250601010a0202ba6225060101310203be656625060101080103c5a77425060101050102de8e250601011704080a08080a080a080809090809090808080b080c080a0a0809080a0809080a0908080a09080a08090a0a'); INSERT INTO t1_idx VALUES(1,X'',2); INSERT INTO t1_idx VALUES(2,X'',2); } do_catchsql_test 41.1 { INSERT INTO t1(t1) VALUES('optimize'); } {/.*fts5: corrupt.*/} do_catchsql_test 41.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db |
︙ | ︙ | |||
5569 5570 5571 5572 5573 5574 5575 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 5d 69 7a 65 uild....opti]ize | end 8cfba7fbb67e48de92c6.db }]} {} do_catchsql_test 42.1 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | 5569 5570 5571 5572 5573 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 5d 69 7a 65 uild....opti]ize | end 8cfba7fbb67e48de92c6.db }]} {} do_catchsql_test 42.1 { INSERT INTO t1(t1) VALUES('integrity-check'); } {1 {fts5: checksum mismatch for table "t1"}} #------------------------------------------------------------------------- reset_db do_test 43.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 24576 pagesize 4096 filename 89028ffd2c29b679e250.db |
︙ | ︙ | |||
5809 5810 5811 5812 5813 5814 5815 | INSERT INTO t1_docsize VALUES(1,X'030003'); INSERT INTO t1_docsize VALUES(2,X'030003'); INSERT INTO t1_docsize VALUES(3,X'030003'); } {} do_catchsql_test 44.2 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | 5809 5810 5811 5812 5813 5814 5815 5816 5817 5818 5819 5820 5821 5822 5823 | INSERT INTO t1_docsize VALUES(1,X'030003'); INSERT INTO t1_docsize VALUES(2,X'030003'); INSERT INTO t1_docsize VALUES(3,X'030003'); } {} do_catchsql_test 44.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corrupt.*/} do_catchsql_test 44.3 { SELECT snippet(t1, -1, '.', '..', '', 2 ) FROM t1('g h') ORDER BY rank; } {0 {{.g.. .h..} {.g.. h} {.g.. .h..}}} #-------------------------------------------------------------------------- reset_db |
︙ | ︙ | |||
6913 6914 6915 6916 6917 6918 6919 | PRAGMA writable_schema=ON; CREATE VIRTUAL TABLE t1 USING fts5(a,b,c); CREATE TABLE IF NOT EXISTS 't1_data'(id INTEGER PRIMARY KEY, block BLOB); REPLACE INTO t1_data VALUES(1,X'2eb1182424'); REPLACE INTO t1_data VALUES(10,X'000000000102080002010101020107'); INSERT INTO t1_data VALUES(137438953473,X'0000032b0230300102060102060102061f0203010203010203010832303136303630390102070102070102070101340102050102050102050101350102040102040102040207303030303030301c023d010204010204010662696e6172790306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020108636f6d70696c657201020201020201020201066462737461740702030102030102030204656275670402020102020102020107656e61626c6507020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020202087874656e73696f6e1f02040102040102040104667473340a02030102030102030401350d020301020301020301036763630102030102030102030206656f706f6c7910020301020301020301056a736f6e3113020301020301020301046c6f61641f020301020301020301036d61781c02020102020102020205656d6f72791c020301020301020304047379733516020301020301020301066e6f6361736502060102020306010202030601020213060102020306010202030601020203060102020306010202030601020203060102020306010202030601020201046f6d69741f0202010202010202010572747265651902030102030102030402696d010601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202010a7468726561647361666522020201020201020201047674616207020401020401020401017801060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102ad060101020106010102010601010201060101020106010101010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020415130c0c124413110f47130efc0e11100f0e100f440f1040150f'); INSERT INTO t1_data VALUES(274877906945,X'00000e96023030011a042319320d3b123d812b5a31120110446e581b66814a05010a4537814274010e8102815c810f3d0104846d01081581204401103741043c59416b44010a404655265301103f73811a11114213010a821235820f020135030484320201360104816a020162020484550302390301710a04824a020166030483690201670704837d0201690404822602016a0504825c02026b620504817502016f0904810d0303e79c88060482760201700a04826302017204048155020373c2be050481130201770204846202027962050482710202c2ba010482140203e58496070483330204e8b2b879010483710101310110545c0c814b0e3a6501082c815d5b011a2a0e2f0d765c3d686014061d0d0112810733112c2e82141101048313010e5c6f632e813e42010c811882370548010e19158146822f1f01104d364a708146135a010a237b0a55210201610904841703027678090481270201620304810002026374060484660202657a0704827602016601048351090483540301660704814b02016b03025f0304c582caba0204816602016c01025f0302cebc0904843e02016e0804821802016f0a04817503016f070483100201720304822c020484380201740404842e0102460201770104812f0204836c02027a6f040483110202cebc02048267040161020484650205d5bd62cebc0604845b0204f2a580880204842a0206f38184a179670502750204f696a3aa0a04814601013201063330390110812281378114600d010c03716c5e822d010e81226b542a814d010a72740f83000108813a1e0b010c5681046f812c010c07814a777328011664244219531b1a2f811e4a010c4d81557c7f1b0201300704810702013307048230010484050202367807048175020239710804832502016204026b0204814d020363306108048262020265650504817602026667070483150201690704832f0301360a04814d02016c08024702016d0304843e0303cfb2630204814002016e0804837503016203048416030370c2be0304821b02016f0604834b0201700504816b030175070210020273660604841c020676c2bac2b2640604830a02017704027d02017808048141010482700201790504811d0202c2ba0502470206ca8d73ecbab9010483340204f09e9ab504048367010133010c3e04814f82250114812e814b2e0411811305010c811337811e6e010c82085e2b0e5d010c61812054811c01148122451b0781050c813d010c17823762643e011e080c1720814a10364306143b0d33260112810f0c2a810c816b13010a8163810e470201370404811102056176c2aa36050481530202646e0904846202026a730404827402016b0a020c02036f616b0404830a0201750504820d02017605025e0201770a04820702027a73050482460202c2ba0604824a020483330203cba434040483200203cebc790304847e01013401124181442c1d091f81580108601d8336011081320a2b8125820001123b0b81158116811f070110078112817a817308010e6682410d810e2601122d0d6413378147351e01105081021d3525812d01128246510a622204054101105c1b620e81302b05020130020483480104822702013102027e020132030483270201350304844802023770030207020261710604823f0201640802570204830a020265770304831f020168070483210201690204825b02016f020481280704835402057037e18b8d0904810802017304048439030172020481440303c2bd6b0a02630201760202490804815e0201770304816e050483550201780704816902017a070483280207c2b2f093aabc780a0482240301b303024e0301bc0604837a0202cb800204834a0202cebc04048201040484410203d3ad770a04814f0202d5a508048371010135011630817e0f81040d2c041552010c813d3b7e8115010c40692182693a01121f810d810d0a32814701101d1d1f642281742e01068229240110811231810a387f4c01100f50810f8165810d0114811f26443152593c104a010e641e1a3357820e020132030481540201340204815402023778040207020163060481020204815b020164020483010201670a0481540304f0948f870904811a0201690704835502016d0604832203027b01022803017a0a025102016e020484260404816002026f690502680301720104834e02017208024f02021d020475e69c8e0504814c0202767602025f0302de870a04837b020178090483200104835003026a72090484400201790504837204022302017a0104836b0202c2aa0a0481070303bcc2b3090484370203c7866403027501013601087c158303011212814305813e7b0e090118141a1c49713a211e0c74630f010a59826d8113011203328166037781561a01101d7f1d2a1f822533010e820e070f7b40160110811f40292c813226010a2d20824a32010a81418158670201300304840a0201660404817d02016d0804826902016e080626817f0104820b02016f0404825a02017003023b020272750804840c0301760a04844d02017403025b020175080484200202766e060482360303c2ba6c030482220202c2aa0204810e0301b20204835703048421040484240301bc0802410202d2a1010482630204e1b18f3704048354010137010c08816337812f01101382211532424d39010881248123010e7724810267815f011081236029813e273301101b7b29812a5b813b01128150810324814b220b01060c8417010e165b6c81708117010a1782346f6c0201380804816803016b0604840102013908023f0201610204816c0201630302760201640304832604023e0204833902016502048203020266770804821a0201670a04830002026964080484500304f29e9eb70802250201700204811b0201710904832d0201730304826b05048403020174010481000205776a62c2b2050482630201790202260206c2b2eaaeb464050485020301b3080482480303bac7af0a0484550203c695650804822a0202ceb90a0481170202dbae05023f010138011819814a2703390a61090c6912011a21181304812523811b5f164e050114110f35128123423f810c010c817573817c03010e7182590c812b0401142503597e6e0e2f3a3759011252813a811a2b75091a010882162a31010e17450a81048279010858658208020231670a048205020361c2b906023b02016301048236020164050482520201650904833d0201670904811b0201690604825002026a7a0604837c02016e0204832002017002026d0302c789010481020201720504835e0201740604810002017502020a0702630302676306025f0303e4a0a70102640203786a75010484440201790104841402047ac2be72070481340207c2ba3766673576090482790301bd07048142010481600202c7b309027a0401740604823b0202d2bf0304830f0204e989a6300a02600204f4bd91b60702120101390106518369010e19254641823711010c258267288121010e817c810d2b17250110810a578133812f4c0110415681067b288121010e0881208119347101140b8131543c8100343d1101088203813e01100d742e3230820f3802013006048107020134080481440202356501021a02013808048147020162020482230201630304833c030162050483390206656cf093b5bd070484140202677303048502020769e3ad9669c2b90304847402016b0804836402016c0404841f02016d010481250904825202016e06025402016f0704842a0201700a04834a0201720a0483530201750304822e020676f097b18374030482140201780604833d0202c2b9010481550301be070483720204826f0202ca80060481630202d5a80504833f0101610114551047810e130a78660c011a364611206c0b13080705733d5501240f08070c090b0c20813d1471042e4351131e011204412d814f0913104201263036110d060b1f811a301b0f4e1a29092f181c012808071e221a2a81075b320503065a0f140c1a0a26011c07231d0e6f3715063b760c6b091501121111303e3a71566d6d010e0867814d816a0c01181e18240d41724d221b3f384b0201300204830b060483080302c2b30204837f0201310a026e020134050481560301730902690201360406827c0f0201370204825d06020a0201390904815b0304f3bfb2a70a04822c0201610404810e020262660604841a03017608025b02016309088112825f020164090482310201650a0484480302396f01025d02026774010482090302df9b06048321020168070661813303016f03048248020169030483610504814001048401010483460301300704824203016a0204824402016a0504813303021d03017209048412030277380804824502016c0104814c02036ec2ba0804835702016f0204811d030176020238020270360a04840d02017107048201030469ca99690602350201720304812d0104845e02017304022c02017409022b020175050484140302caaf0402410201760404831f020177010484180704845203026f6d04024b02017a030482660202c2aa0204810a0301b30a04817b0302bd76020483780302be6a0302440202c58207025e0202c69901027a0302ad77010483200206c993f099b183070484270203caa1660204841a0204f29788ac0804831d0204f4ba9f950504843c010162011c0e33810216341c2413042130780501184d373e53131f2f052907423e010c830e3781390e011c1320461f81041b811b041e15243d011e241b10816c310b130c3133033b0741011a11816d3139100c13140b395848011c580e411a06304306810a3138330d011a441707092c70140c1643813920010c73653581374f010c826c81210f0402013803024f07048172020161020481650204810c020162030483470301370404813602016307048379020483280201640304815501048176010481060201650104812a0104841f0201660504821f0201670302700201680804846403016b0a04831d0301700904845502016908025c02016a010483560904827602026b6306023402016c0404832602016d040484410204825702016e0504831603027831020482160302c2be0504827d02026f6a05048121020171030232020483220304845402023a0201720204845c0304846e03013607048224020174060484480201780704844303016f0604814f045807070a0707070707080709070709070808090a5c07080708080b07060a06080707070b0a0b0808070b0a0b0a55070b08080a0908080707060709070709070706080c060b07070c0a66070b08080609070607080c0909660b06070707080a0807070b0b0707080a0b07070d0607080c0908630707070b07070a070d060b0707090a07080b080a070809085f0707070c0706080706070809080f06080a5807070607060e070807080907070b070b060c0709090807690808070707070708070608070709070809070a0d0b07070809095b070707070707070c080d07070b06070707070c07080b0808811a0b08060706080a070a07080609070707080808071307070a0708070907060807090b06060707070b0707080708070707080c090a0a81080a0b07070b0f0b0706070707060b07070b07080808110b070707'); | < | 6913 6914 6915 6916 6917 6918 6919 6920 6921 6922 6923 6924 6925 6926 | PRAGMA writable_schema=ON; CREATE VIRTUAL TABLE t1 USING fts5(a,b,c); CREATE TABLE IF NOT EXISTS 't1_data'(id INTEGER PRIMARY KEY, block BLOB); REPLACE INTO t1_data VALUES(1,X'2eb1182424'); REPLACE INTO t1_data VALUES(10,X'000000000102080002010101020107'); INSERT INTO t1_data VALUES(137438953473,X'0000032b0230300102060102060102061f0203010203010203010832303136303630390102070102070102070101340102050102050102050101350102040102040102040207303030303030301c023d010204010204010662696e6172790306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020108636f6d70696c657201020201020201020201066462737461740702030102030102030204656275670402020102020102020107656e61626c6507020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020201020202087874656e73696f6e1f02040102040102040104667473340a02030102030102030401350d020301020301020301036763630102030102030102030206656f706f6c7910020301020301020301056a736f6e3113020301020301020301046c6f61641f020301020301020301036d61781c02020102020102020205656d6f72791c020301020301020304047379733516020301020301020301066e6f6361736502060102020306010202030601020213060102020306010202030601020203060102020306010202030601020203060102020306010202030601020201046f6d69741f0202010202010202010572747265651902030102030102030402696d010601020203060102020306010202030601020203060102020306010202030601020203060102020306010202030601020203060102020306010202010a7468726561647361666522020201020201020201047674616207020401020401020401017801060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102ad060101020106010102010601010201060101020106010101010601010201060101020106010102010601010201060101020106010102010601010201060101020106010102010601010201060101020415130c0c124413110f47130efc0e11100f0e100f440f1040150f'); INSERT INTO t1_data VALUES(274877906945,X'00000e96023030011a042319320d3b123d812b5a31120110446e581b66814a05010a4537814274010e8102815c810f3d0104846d01081581204401103741043c59416b44010a404655265301103f73811a11114213010a821235820f020135030484320201360104816a020162020484550302390301710a04824a020166030483690201670704837d0201690404822602016a0504825c02026b620504817502016f0904810d0303e79c88060482760201700a04826302017204048155020373c2be050481130201770204846202027962050482710202c2ba010482140203e58496070483330204e8b2b879010483710101310110545c0c814b0e3a6501082c815d5b011a2a0e2f0d765c3d686014061d0d0112810733112c2e82141101048313010e5c6f632e813e42010c811882370548010e19158146822f1f01104d364a708146135a010a237b0a55210201610904841703027678090481270201620304810002026374060484660202657a0704827602016601048351090483540301660704814b02016b03025f0304c582caba0204816602016c01025f0302cebc0904843e02016e0804821802016f0a04817503016f070483100201720304822c020484380201740404842e0102460201770104812f0204836c02027a6f040483110202cebc02048267040161020484650205d5bd62cebc0604845b0204f2a580880204842a0206f38184a179670502750204f696a3aa0a04814601013201063330390110812281378114600d010c03716c5e822d010e81226b542a814d010a72740f83000108813a1e0b010c5681046f812c010c07814a777328011664244219531b1a2f811e4a010c4d81557c7f1b0201300704810702013307048230010484050202367807048175020239710804832502016204026b0204814d020363306108048262020265650504817602026667070483150201690704832f0301360a04814d02016c08024702016d0304843e0303cfb2630204814002016e0804837503016203048416030370c2be0304821b02016f0604834b0201700504816b030175070210020273660604841c020676c2bac2b2640604830a02017704027d02017808048141010482700201790504811d0202c2ba0502470206ca8d73ecbab9010483340204f09e9ab504048367010133010c3e04814f82250114812e814b2e0411811305010c811337811e6e010c82085e2b0e5d010c61812054811c01148122451b0781050c813d010c17823762643e011e080c1720814a10364306143b0d33260112810f0c2a810c816b13010a8163810e470201370404811102056176c2aa36050481530202646e0904846202026a730404827402016b0a020c02036f616b0404830a0201750504820d02017605025e0201770a04820702027a73050482460202c2ba0604824a020483330203cba434040483200203cebc790304847e01013401124181442c1d091f81580108601d8336011081320a2b8125820001123b0b81158116811f070110078112817a817308010e6682410d810e2601122d0d6413378147351e01105081021d3525812d01128246510a622204054101105c1b620e81302b05020130020483480104822702013102027e020132030483270201350304844802023770030207020261710604823f0201640802570204830a020265770304831f020168070483210201690204825b02016f020481280704835402057037e18b8d0904810802017304048439030172020481440303c2bd6b0a02630201760202490804815e0201770304816e050483550201780704816902017a070483280207c2b2f093aabc780a0482240301b303024e0301bc0604837a0202cb800204834a0202cebc04048201040484410203d3ad770a04814f0202d5a508048371010135011630817e0f81040d2c041552010c813d3b7e8115010c40692182693a01121f810d810d0a32814701101d1d1f642281742e01068229240110811231810a387f4c01100f50810f8165810d0114811f26443152593c104a010e641e1a3357820e020132030481540201340204815402023778040207020163060481020204815b020164020483010201670a0481540304f0948f870904811a0201690704835502016d0604832203027b01022803017a0a025102016e020484260404816002026f690502680301720104834e02017208024f02021d020475e69c8e0504814c0202767602025f0302de870a04837b020178090483200104835003026a72090484400201790504837204022302017a0104836b0202c2aa0a0481070303bcc2b3090484370203c7866403027501013601087c158303011212814305813e7b0e090118141a1c49713a211e0c74630f010a59826d8113011203328166037781561a01101d7f1d2a1f822533010e820e070f7b40160110811f40292c813226010a2d20824a32010a81418158670201300304840a0201660404817d02016d0804826902016e080626817f0104820b02016f0404825a02017003023b020272750804840c0301760a04844d02017403025b020175080484200202766e060482360303c2ba6c030482220202c2aa0204810e0301b20204835703048421040484240301bc0802410202d2a1010482630204e1b18f3704048354010137010c08816337812f01101382211532424d39010881248123010e7724810267815f011081236029813e273301101b7b29812a5b813b01128150810324814b220b01060c8417010e165b6c81708117010a1782346f6c0201380804816803016b0604840102013908023f0201610204816c0201630302760201640304832604023e0204833902016502048203020266770804821a0201670a04830002026964080484500304f29e9eb70802250201700204811b0201710904832d0201730304826b05048403020174010481000205776a62c2b2050482630201790202260206c2b2eaaeb464050485020301b3080482480303bac7af0a0484550203c695650804822a0202ceb90a0481170202dbae05023f010138011819814a2703390a61090c6912011a21181304812523811b5f164e050114110f35128123423f810c010c817573817c03010e7182590c812b0401142503597e6e0e2f3a3759011252813a811a2b75091a010882162a31010e17450a81048279010858658208020231670a048205020361c2b906023b02016301048236020164050482520201650904833d0201670904811b0201690604825002026a7a0604837c02016e0204832002017002026d0302c789010481020201720504835e0201740604810002017502020a0702630302676306025f0303e4a0a70102640203786a75010484440201790104841402047ac2be72070481340207c2ba3766673576090482790301bd07048142010481600202c7b309027a0401740604823b0202d2bf0304830f0204e989a6300a02600204f4bd91b60702120101390106518369010e19254641823711010c258267288121010e817c810d2b17250110810a578133812f4c0110415681067b288121010e0881208119347101140b8131543c8100343d1101088203813e01100d742e3230820f3802013006048107020134080481440202356501021a02013808048147020162020482230201630304833c030162050483390206656cf093b5bd070484140202677303048502020769e3ad9669c2b90304847402016b0804836402016c0404841f02016d010481250904825202016e06025402016f0704842a0201700a04834a0201720a0483530201750304822e020676f097b18374030482140201780604833d0202c2b9010481550301be070483720204826f0202ca80060481630202d5a80504833f0101610114551047810e130a78660c011a364611206c0b13080705733d5501240f08070c090b0c20813d1471042e4351131e011204412d814f0913104201263036110d060b1f811a301b0f4e1a29092f181c012808071e221a2a81075b320503065a0f140c1a0a26011c07231d0e6f3715063b760c6b091501121111303e3a71566d6d010e0867814d816a0c01181e18240d41724d221b3f384b0201300204830b060483080302c2b30204837f0201310a026e020134050481560301730902690201360406827c0f0201370204825d06020a0201390904815b0304f3bfb2a70a04822c0201610404810e020262660604841a03017608025b02016309088112825f020164090482310201650a0484480302396f01025d02026774010482090302df9b06048321020168070661813303016f03048248020169030483610504814001048401010483460301300704824203016a0204824402016a0504813303021d03017209048412030277380804824502016c0104814c02036ec2ba0804835702016f0204811d030176020238020270360a04840d02017107048201030469ca99690602350201720304812d0104845e02017304022c02017409022b020175050484140302caaf0402410201760404831f020177010484180704845203026f6d04024b02017a030482660202c2aa0204810a0301b30a04817b0302bd76020483780302be6a0302440202c58207025e0202c69901027a0302ad77010483200206c993f099b183070484270203caa1660204841a0204f29788ac0804831d0204f4ba9f950504843c010162011c0e33810216341c2413042130780501184d373e53131f2f052907423e010c830e3781390e011c1320461f81041b811b041e15243d011e241b10816c310b130c3133033b0741011a11816d3139100c13140b395848011c580e411a06304306810a3138330d011a441707092c70140c1643813920010c73653581374f010c826c81210f0402013803024f07048172020161020481650204810c020162030483470301370404813602016307048379020483280201640304815501048176010481060201650104812a0104841f0201660504821f0201670302700201680804846403016b0a04831d0301700904845502016908025c02016a010483560904827602026b6306023402016c0404832602016d040484410204825702016e0504831603027831020482160302c2be0504827d02026f6a05048121020171030232020483220304845402023a0201720204845c0304846e03013607048224020174060484480201780704844303016f0604814f045807070a0707070707080709070709070808090a5c07080708080b07060a06080707070b0a0b0808070b0a0b0a55070b08080a0908080707060709070709070706080c060b07070c0a66070b08080609070607080c0909660b06070707080a0807070b0b0707080a0b07070d0607080c0908630707070b07070a070d060b0707090a07080b080a070809085f0707070c0706080706070809080f06080a5807070607060e070807080907070b070b060c0709090807690808070707070708070608070709070809070a0d0b07070809095b070707070707070c080d07070b06070707070c07080b0808811a0b08060706080a070a07080609070707080808071307070a0708070907060807090b06060707070b0707080708070707080c090a0a81080a0b07070b0f0b0706070707060b07070b07080808110b070707'); INSERT INTO t1_data VALUES(274877906947,X'00080e7f073c23110a1a18392f66090524183704276d6703306a320404824e030164080483520305c2bd7ac2bc0604815a0201360704833202016106021f020482400201630304822a0708817e8204030173040483500201640404824803016d0804824a03017709023002016606048367020268680a04815802016902088339811804027f0302656e0704834e0303d5a5370604816702036a3366090484470303c2ba660904826e02016b0904837c02016c07021403026c610604835802016d0204816802016e0104831202016f0104822f020270720602060201710704822202017206048174020273690204824602017409020c020175090482140201760a0482720301660404824403016a090484290201790404845703025d0203c2bc33040484620301bd0304824c020484540202c78607022403019a010482380202ca87070484390202d39d030485050203e184940404831b0203e6a881060483480203e8b18c0a04816d0203ee8d850104814801016b0110467257393c81272c011a053e815d3b190517064524521f011c3823590a8115372004313b1f3216011a5a20780b102d0804426916112c011a182f810781082d12137026161501221a180516811611051c131207811515173501180320112581062e05621c1407011c2d0e0617811522062208065a21520114582841621e6c203f1e2001161647411a272533815b1c2602013009048309020232630104835a0301720104817f0201330604836f0302ddb5080482560202347a07048102020135020483460104827b02043678ca800a04835f0201370404814b0104846002016103048246010482220301700204833f0201620404824d060481150201650304824f02016606088110834c0201670604821d0303c2be66010481790201680404843b030176050482270201690a04830e02016e0904844202016f040481010301630304822f020270640204822f03016e0704845802027177090482710206736ec2b2796a0104832e0306dab1d485377004048304020174050481700201750a0212020378627604048164030173080483190201790704833d0204823a02017a0506820e67030178070484530202c2b2060481500104823f020483030301b3020484310301bc04027e0402caaf0a026a0301be040482590204842e0202540202c79f0804824d0202cfa30804815a0204f29a92970204823301016c01140f63351a0a653b650d22011c09117a3e1538123537046a15043101141310082f49052f772b0c011c11121781583c2a5010133228241301287f3e0a2b1244080503060a100f413b4f0d070e2a01103e4e1f04814e7b1601183d0404052877111f230f811d01123a100f053e5c076910011a031732102381243d1b1727507301180e5d273e810803812e0f192a02013301048271070481330204821d020134080263020135060481280201610104830a0201640604826d020165060483050201670204841c0504841c0304841b0201680a04845602016a0104811c01024f030481080204813102016b0204837008024502016d0404836c04068207780301670704842302016f0404821203016d040484490301720404837e0201700104821d03048407030165050483050201720a04811602017307023502017407020503016f080484240302c2b90504821b020175090484090201770a088119822503026d6905048300020178030482680604812a0201790104830c0204833a0303d9a06806022002017a060482600203c2aa33030481560301b904020a0301bd0504820f0202d0b90904817c0202d3820202200202daa9080482030203de966e02024b0202df9d080484350204f098b0a20604845e01016d01220304456608322258060a031d4c38340f090112310c070e4238626e6601124a318109030513812f0118240d561e533742188113101b01160b24444b224d44814d4806011c05774e483410330d23541b28090401141f29062581131e221b6d011e81053a037a03320b0e4c24360d2310011a0e321d3c141825111d54637a1c0114093d3c2e58571a35293a0201350104840a03017701048330020136060217020138030483370201610a0482650201620504815e0201630704827701048201020164080483690201660804846703016904048113020167070483080201680504837d05022c0302cdb10a04815f0201690104833a0404824302026a360a04823b02016b0a04813502016d0504831a0204833803021a02026e360404825e02016f080484140201720304844b0404816603056ff09d899b0304823f020275390204816e0301780a04824202017604088308812703027902027770050482040201790104827e040482750204812902017a060483030304c2bdc2b30104836f0203c2aa62040484040301b903021c0302bd6b090484300301be0704814d0202c99402025603049a65656b090484020202ca92090482060203d19a730504844a0203d49f690804836e0202dfa8020482710204f09180860704822901016e011a0c0b8104243647521f43231f36011a2e1b33432c3d0b414905054d17011010573a6c0a816c1801160e063582340a5239050b06011a4481063d1b67250f2044200839012044591d1857291214135814101a1b361d011225067e8147111a4a4301166b13362e17195f3812186f01141c465b032b290406373301182a152a2281300f8107054e3f02023274080481770305c2bacf8168020481450201340604832f0201350704842e02013605020e0201380404841d0201610404810d020483750201620304812b020484230301610804834503026c6a0304816d0201630102380305613577337405048359020165040482720201660904826202066736f094b0af0a0482250201680104811f02016b0304847202016c0404822403016f0904822c0302c2b301025002016d0504817b01023f02016e020483090802040303e7bda10804832d02036f6b740404811402017005048419020484220202716506026303026b760904830a020172080482430304706c73620504825f02017308048413020174080481070201760104827f0204836e020477e7b89a0104840e02017a030483700206c2aa35657065050482740301b30804842b04046cc2be78090481040301b903020d0301bc010484260904813f0203c7a5620302330203c99f36050481010301a30704815b0202ca8b090483250202cdbb0604820a0202cebc0102170401380304842b0207eca2a6f29c87950904824001016f01221d17052b58101241060e3a201f1021633a0114816919811c142443100801280426080e2620042a812c531a490e121707131710011273432e493347811a340112195f671f46721c325e0118380c052b812822478107600b0116021c21821b2019263433040126021b05351b2a286b05181f071b5628111a330a012014533e073d0c0e5469141d1e2734050901220318051b44412803632e0642370e0a3a2b020131070481770201320a04812f0202346e04022b020136030483590304f09a81b60404834702016105048210020162030205030167010268020163010481540604820202048300020264310902420201650804834b02016703048247030365c6b602048205030573f098b890030481450201690204832802016a0a04826703016c0104825e02016b0604815e03016c03048334030677c2aa74c2bc09023d02016c0304823903027777060484540303df866c0104815b02016d0204811f0303796f7704020302016e0204814d07024a02016f0902680201700604840a0104831a0204835a020172070484440201730a023703026b660604830e030278790304815b0201750904822402017704025b020178030482350307f4b2a3896a343407026b0201790902720302633409020402017a020484590302dea004025f0202c2b20204816b050481200202de900402160204ee85a5770204822c0101700114143a0d391a60812d4e09011a2f313104201c372c3a3411321b011a268140144226334145050d1c4d01164e081f20671f088107237901186b123c1f6d07261e2b732e210116511116342a3d32376e083001106882257a0a17141101163039192b0c05812d735f3b01262a3e0841030b17181411051e0a18530e272b6d01182f322b260e24581d5381050f02013104048353020132050482370301690204843e02013302020d0201340104841f0201350a048139020137050482770201380204833a02016105025a0504832f0201620304836d020163050484100304832003016c030481290201640304837e020482490304822b010482290201660a04827002046964dc960204833102016c0704814502016d010482000201700104817e02037176760904821f020473eb91a708048152020174090482770201750404831e01063a825d03017a0904826a020276730a0254020177080260030277630104815c020178070481220202020301720804841202017a0204834c0202c2aa040484010301ba080482580202c6a3020481320203cdbf690502790202ce90070483140301bc030481470205d1a371cebc060481590203d2976a0404830c0203dfba6e0604814b01017101163732393b8120422f054b0e010e030b211d815d1c01165641757c080d81311d090e0112816581542d2313054301224e07121706516606080e39102d231c4b39010a2d81402d5e011a132527428114080d6e1111721c011a814a1a341538251023100d1c4c011e22182622623712411e38162a182d3b01142b67611981470f1f1f250201310a04824e0202336207048217020238730204815a0303cf886d06025a02033962620404833803016f0a04814e020161060483140302726d050483450301790904810c02036376690504811c02016403024d020165010483280802550301650904827f0304ebb8b561070482340201670302670301660804810e020168040482340201690704844d0302616404020f02016a030481060301700704827802016b070481240104814b0302c2bd0504816102016c0604837f03017a0404837902036dc2b90804810002016e0904821602016f0304812c03016401024b02017103048233060483600303e5848e04023a020172040481050305f3978aa06c070481151708070b070a0d0707070607080c080909090706080707070707070806070707070a090b070708080909090981140708070708080b0a0b0b070b070907090707070707070807080c0c070609070b0807100706070e08080a81110f06070707070f07120a0c070707070b0707060607080709080b0b080709060708070808080a810f0707060707070b070707070a080b08070e08070b0b08070c080f070a09060807070a080909080a810b080b070706070b0b0708060b07070c07070707070a0a09090b0708070a07070b0a070c070a060b080907080807070d8123070707070a0706060f070707090b07070707070b07080907080a060f070608080706070c060707070c070a810f07070706070707070a070b0713070a070707090a070c070706080a07070807080808070b0909810607080808090707080709060a070a060707070707070b080707090707060b0807'); INSERT INTO t1_data VALUES(274877906948,X'00000e8a0330717304048359030134050481100203756371070482190201760704817b0301770804821d0201770204844d0201780204836c0404826103017504026a0202c2aa020482420301b305022c040267390602570301b9080481540301bc0102290206c99cf6b5aa80080481430202cebc02026e02048120010172011a1c2f15158108048125463f251d010811811539011412423105812181171549011847284a30234e5b33042632120118351e8113817d0f2b220d111901264f104a211004061d0a2a0b35121a0a2118341f011c81160a1b030d2a0610243e445f0c011c6f0c1e3b1768141e322717500b140110537f810169811625011a492847203e210f532c16480627020135020481780302caae0104811a020261330904846c0201620804812d0201630404814a0201650704837503017301048276020168050213020169010484540604842b0302796f0504833302016a0304831b02016b0604701302016d0604815302016e070483630204815202027071020481520201710104835b02037266700704843002027362040481490201740904817d020175020481040304f59e9c9407048218020176060484060204776dc2aa020483070201780504812503016601048159020279790704840502027a730904826c030178030482740202c2aa04023d010483540301bd06026b0203cab877010483290202cdbf060482410202cebc04025c0401690204827f04016c0904840e0202cf880104835d0203dfbe6c01025b0204f0aeb7b2030481680205f1a7b5bb390504826a010173011a22810d12415003071f81181839011a3220221511546d810012052b57011a0c4274300d154e81111f041e10011e293f4213051b2276560817312811170120092136122418370e4e782b3912080f3201262b3b340f222b0c09142a0822116a135c1c130c0114320c4e385a0d0415075f01163543340f06362381133c0c012224180981742048191d110e180e180d310f011a20632450281f043027114b034e0203316a61030482160201360104826e0201380a0484570201610104844d0504814a0201620904820f020164010481630304810403027761040258020165080481550302c9b70204827a0204677367690102660201680104831b020169020483760301720a04814502016a04020c02016b04023405023c030269630504840502016c08048463020170030482670202716d080482000201720304835c03016507024d020174030482520504821503016505048207020175040208030137060481710201760a04833d0201780404832302027a6f080481200202c2ba0204812c0301bd0404821e0205c3a66865730702540202c7890804816f0205cebcc7af730504815c0202d2930202540202db89020481160203e8b8a00304825d0205f0958db331070481620305989b8569780a04813d0204f69299a5020210010174011a7b3829100a4e511f1a281c17140114812626032c372634234c01140a520e815a810815200501123e4f3531042d57615b011a041f3e64070f1f1913274a20770114811d0f5d743e0634161c01162c2782130c1b810520280d01164a513110480b402b810d13011e522d08042c1146137012201e810512011a290903182c05301e5d811944290201390a04836d0301610104836e02016208027803036cc2bc07048261020263640a0481480301730704813602016408022b020165050484310301720702260301780402500203666d73070484470201670104825b02016a040481590304836702016c0304835601025e02016d060484110301340204813702016f04021802017006068336280201710304813507025902017201026508022903016d0604812f0201740304827e03016a010482440201750404834b02017604021a010484150504836a02057773c2b2380602520201780204823e0302cebc080484040303d2956403048171020179020484240204813c0202c2b303048307020484410301be0204816f030484250203c798680104843d03019c030482570204e19ea86a020482350302a0950a0482280204e5a4bc780304810e0101750126090a35030a03220a1731630f31252f0c4b1e31011e39200e3715282a03103b56090f6b1501121d4916246e6d460d6501162609380406361e816d203f011a22166008124f58202e182025150114390f3a25713f0e3f715c011a5a11191123466025710c313312011e3c191326811c1444055f1f5109051201143b106f1181000d068155012043381381020d81080d0603171824260a0201300404844a02013207024203026b390204833803066eeebabb35660604842102013308020302023f02013503048243020436716b66040481440201380a04843d020261690704836002016203048209030484670201630a04841d020264790104822b0201650502340302c2bd060484300201670202620201680704810d020169010484430104843402026b67040481540306eea3ad77c2aa0a04836702016c020258040482270104830a02016d0404824102036e716a0604843e02016f0104832a04020c0204836803036530650a04817402017005025b0301630604843d020171070275020273720404827002017504048133010484120301370102270201760104814f03026203016d0904844802017701048375080481220301660202330301700a04827f020378c2b2030484710202796806023c0301730704813d02017a0404815f0104817202048407010483390202c2b204026802048254040266640204831e0301b3090482230302b963060482010401750a0483290306ba35f2999dac09020f0203ca926e090483350203cdb4780302350202cebc0a020f0204d7a7696d010483100206f097bc996d71040481480101760114185b2258291610821c0e01160272173107154f5b813722011a81020c200e1826250d39811f07011a7911152a2a45131504422c81070120050d3f5b23342e3e4139032a3813042d0116592d1c15630c0c0a814649011c1a362f5c4a35511f0804033e372b01102981262a352e8205010e0b4b6282388106011e26810a2d125f361a12170d1721311e0201300204832b0201320104811b050483790201350204832202016404024c0202657a090483710202666307022703016c07048362030277750604842f0201670304843d0104844c0203686f7a07027202016a0102430204847502046ce0a2b20704810302016d0204827b02026e330804826c02016f0104835502017102027606026c020172060483490202736505048371020174030484130302387602025d02017502048345020376346b0904825c0201770904814b0303c2bd720804812b020178040483600201790804816402017a08026e0202c2b90704811c0301bd0504821e0204cdbcc7a108021a0202cfb8080481490204f09f96a50204842101017701180207232d37812d0c045c4a0a011a06163b3408171c52213a26592201206d08581605811a171e0c0a1347104914011282181324082b73320f01122f6e811d2c3d410a44011e551414206a092f133f333d150a3e0f011e235b170e37060627471b13373b3e27011a0e1e816b270c10102d53381045011a060e1e254d044932651234691e011a158138300a04810c0a8121071802013003027707048214020131080481370201320a04835502013304048271020538ceb369650802310201390a0481440202616c0502570302c7a104026c0201620804811c02016301048168020241020164040481560104820b020165080484460102400202686c060232020269670604827302016a0108810c826e0704824e02026b6c0604816902016c0304831c02016d0304811903016407022b02016e01022e0604845a030237780702040301710804815102016f0104842903017a0704826902017003048445020482080202713002024e0201720804822003016202022302017303048111010482790204812d03067479c39f66700a0210020174020229020175090484430301690604820402017606020202017a01021b0104843a0202c2b90604842b0301bd030484570202c69b0504815d0202c8a30a02240202ceb8090484690301bc0404832f0202df85020481230101780116812b0a16810e4b045a3b2a01205b1305811134092f62072343100f0f05011e5734152612030b4c4134123009361601121781653207780a6a0d01164a25210824138107738139011481341f088158060c8133010e5920193a4c2331011a0510358101231a1b3609702732011a2f07631610033436810256174c011a1342040a58110721378139101602033067750802720201320604834303017109048244020133090481700203366f79010483520201390404810902016301048411050483420201640a048432020265310704832b02026774040481000302d5b2010259020369756c0504832902016c07048365030233700904824102016d010482670404834c0504830c02016e0104831604048120030169090481750303eba6990104835f02016f0604834c030379c2ba0a0481560201700404815e06048256020174040482370201760a04820d0201770604811c02017a0404812e06020b0202c2b9030481660301bd0a04816c0301be030483620206c99b6d7777750304835b0206ceb0646b66610402490202d8bf030482250206e8bfbc626964080482510204f09c9a9e0404830f010179010e2081335661371c01220e4e2718124f0d0649812b0b0a063b040b011402741d1235810805211a011409161d732b8106325f6a01182e330325068107703728302b011e3723081c0d0a3f810c183e061b067f0106834a12011a044030185a1e810704220a0541011245602b0e421441817801144b03811a1a29614e224b02013003048139020132050485050304f09caba6010482230201350104822708048413020138020483430201610402230201620902440108812181070301300304815e020263710404831002016403048226010483660604823c020165010484510201670104816407048418030334c2b20a0481120201690404814c02016a010481500904810f02036b75610304836402016d01027902016f0304817b03056f6373cebc0a0483010201700104817d020171050482680104843b0302383203048128040807090707070b0608060707060c0b810e07080807070707060b080707070b0807090807070a070a07070808070b06090807070708080a0b81230907070b070b0707080907070706090807070807060b07060707070808070a080b0708090b0b09810a0707060908070607060609070b0a070706080a09070707070e0a0708090b0c0b09070a080a811a0706080c09070a07080b0708060806070b080c0e07090e09060706080b060a070b0607090707130b080708070b0908070a0c810d070b0706080707080b080a0a070807090708070707090709070706080709080a81170a0707070a070707070a0b0a07080d080707060a070707070b0707060f0b060707060a08070807080708810d0807070709070b070808070907080f0b070907090b0707070a0807070c0b080c0a810107070a0b07060c07080f070b09070b0906070b070b'); INSERT INTO t1_data VALUES(274877906949,X'00000e5c033079720404826c0404833002021b03026f6b020482100201740904842e010484150303c2b36f0204840e0201760704826b02017708048273020578f48ba5b50a0481400201790904845902017a050483280304c2bac2bd0404841e0203c2aa680904843a0301bc070481100301be0304820304016a050481630202ca8103027b0202cb860604840d0204ceb56e370204832e0302bc740202520202cf8d07027e0205d8ae39c2aa0104813c0204e887b3770404816a0204f1bfb0970204832f01017a0118101e282f07045961813a193e011e69162f0d2b051c060f084460063053011e06810c20330d0733815c220515220c011210290a7e07810c3a18011a1f2f064a19155212472781047c010a123e45825501166c6062182718167131092f0112331b812c0b6e81470b01184f3a230d45261e271c36111701140a8128456b291248391a0201300104840d0204812502013401048318020137070483070201380904823b03016704026402013907023403016e060481380201610204812f0404824504045243020162050481150104824003016a0804827a020163070484590204817903017a02048209020164020484200203653077030482610302737601048424020166070481730201670304841e0504840e03016b0504825302016901048235010484400404841002016a010483680404845f02046bc2b97407023302026c7a0704812d02027161070209030378cebc0a021f020172040483750404815f0201730304813c030131020482040201740704824d0302793901027502017604048423020177040620813103048313020179030484540204833c04022503017a0904826302017a050483530203c2b2660404840f0307ba6272f397bd92010483070301be020482760202c6b9050482410203c9a3650604812c0202cbae020482180202d38c0704812f0204f096adb7070483490204f6a69c8b030484390102c2aa010c815e81147969010c811c827f0417010e81077a4c03815f010a2e8100820c010a810d148140010c0a7481201a13010c0b831525323d010a8206358129010c21637a33812701083c8340390301300504820a0304326939770404841c03016107048334030165030484150302696c0704810103016c0704812a03027178090481620301730504830e0301750504846303017802021c0301790304836803017a0604834e040135030482650302cdbf010483330304ceb23169070484090303e4849a0504817d0201b201088219744601082210832c010629846001121c8137182211816232010a81668122690108817281080110816a433581102f3e010e7481001b3481190106814e1b010e3646823a810e070301310504840b030161010481720301660604822f0301680804821003016a0704844803016b0104845b03016c0504823403026f700802340301710804826004026d3106024a03017509068458070301770a04836c04016b01021f0301790a0482790302c2b20a02270401b3040482130303ceae6e010484330302df9e040482100201b3010881656e750106820b50010e81434a27048153010a812b068122010843810c0401048211010a50817d812e010c811a8163810801061e832c010c811f81418101030163030481210301640504831102048104030482440301680a048202040271300404841303066b6576cebc6a0104843603026f7a08026d0303756a7808023e030176040483610301770704814403027973020484360302c2b906020c0401bc010484490401bd0804835c0302d38c0304846f0304eaae96750a0484020201b9010c3b824018322d010e0468315a5c817901067d583401060b810e010865118169010c0e07826b813a01066f8265010c3b8239633852010e61161e7030821b01068241210301350804843d0301620302480301690402610403e7a1910704812103016d080269030673c2bac2b36c090483070301760504821d02048454030278350702620302c2ba010483320304ca897a750804813b0401b5070482350303cebc6d050483120302d199020483020402ad660604840c0302d2a1010482550302daa30202340201ba01026d01085119826a010c2f5e82008110010882028221010481090108821b812d01068128460208810b8264010c810971812d440301310304820e0301320504825e05048221030333756d010482720301340502700301640604813d03016607020c0301690204823a03016b0104812203016c0704842103016d0304835f03016e0a04813c03016f0304834d030173060482660401710604810f0301740104843c0303d48174040482690201bc010482690108813c83290114090f816045242e148111010a810616822701048456010e2c81167a638115010a3a3b83204001067a8367010c8114127a2265010a824f7f5c230301300604833e0302616f020483560301640104827d090481280301650502150301660804841a03016804048221030169030481120301720104827803017606023d04016e020481700302c2aa0904836c0401ba090484100201bd01088240224f010843813674020a81185068620106822a44010e14154a8101825e010c19814a1b826c010c81221f81651b010a815a4d812c01082d7281160301300804843c030231300304836703013509048353030365c2bc0304814d0302667504027a0301670604831503016c09025d03016d03048178030172020483620301730104816b0402c2b30404823203017401027204016a08048451030175090237040177090481690301770302190301780a04823d03027a350a0481260302c2aa060481620401b3090481090304f098a1a30a04825d0201be010a0d730b816f010c821d81236c2b010c6f8208358119010681236801087d4e831e0108823c8235010667794b010683165e010e05812e3d3c820d010c81148123817403043531c2ba0204824c03083875c2bceba7957a03020803016302020804036dcebc0304814e0301640304843704033379790504823203016b08023603016d070481090304825804013707027a03016f080484530301710a04842203017809025e0304c2aa6135010481460401b90a04836b0401bc0704814f0306c6ab66e3afb9080482660103c39f350304821f0201b0050481110301680602030201b8040481310604810403016c0a024803017108024b0201be06048323030482650301300504812d0304f48990ae060482740103c491680a0483700301760a0481290201a7010484090202b177020482380201b3030482280102c5800602090301360104826a02018202048173040483780301370104814102018b090482730201930504810f0102c680020483790404830c020183020228020185030484400704820b02028d7a0a04821c0201920404835e02019507048437010484070301760204836102019a020214080483100303d795680504820602019e030481050201a30404820a05048436030177070484290201a50704843b0201a8060482690304816f03027735020483630202ab79050484430201b6060481050201b904024003016d070481130201bd090484350201bf06048361030163020481580102c781040484560201830604833a020286650404842a0301750a04821a020189070483370303e0b9b30904836602018c0104816c04020602021d030171050484570201960a04842d03016b0204821903016c04021e03016e0204845002019a0a04844402019d0704835103013103021702039f77750704827e0201a1090483340201a301022b0404831d0304616363750502520302cfb2040483720201a50a0481530202ad760902310201af0104811d0908816281390301640404843c0201b3080483240201bb030483480201bd040484290304840a0203bf646a0404843f03017a040482450102c89d05020f0201a1090483550201a3060483550201ab0904813203017a04024a0201ad030482210201b10204824302025d0303656577030483600202b46e010482240201b6090484410204bce39f9d020484570102c9870604813202018b0804843603017a06027102018d080481130204840c02018f0104836309026202029164020484430201930504816d0201970304843402039c6c73070483700201a0070483470201a203048262030364d7960a0484010201a80304832a060484390202a96a080481170303c69532090481440201ac040214020481180202ae770604832c0201b0090482300201b40804845a0201b5070482110302716f010482160201ba0504837b0201bb02048149030165050483250201bc07088105833002067f824d0302d19f0402330201bd04024d03017a0904814f0102ca800a048321020181040481580201820604821e020383693707048417020287630704840002018a0904834c0104835c02018b06022f02018c0704814702018d0704810a0104831a020490e7b38308048423020191020482510201920304832f030135030483710201950304833207026802019902026b02019b0904832a02019d09024c02019f0802210201a4010484080202a5710a0481680203a777670804831e0301790902100201ae0402700104834e0201af040484650202b673080484190201b705048221010483520201b808048465030167010484400104cb81cab90104834802018602048317020689777568cf920804842602018b0404817b020191050484200202a0720304833e0201a3040625827f0201a4030485080102cdb1070881408205020483700201b30404840e0202b735080482640201b80504815a0201b90a0482370206bb31cebc6c730304842f030133070483010103ce80370304826d0201900202770201ae0504844e0201b1050484080201b20904815f0201b3040483430201b40702600201b50804823b0201b90304846e0302c9a1070483300201ba0404844d0104847e0201bb050482780201bc010882378223010e811105814d8134010a8142823f2d0106811c52010a6e1814816b02088168824b010a438137812d011019060c6b812f811c010a81314e811b03033366660404825503013502048263030238620904820703016303048120040f080b0907070b07070a0907070707080a07070b0a0a81060b0707070606070f0b070b07070908070b070f0b090807080b07070707070c0e0707090d07080908080a0a50070a0707080708070706070707080a094d07070707070707070707080706070707090844070f07080c0708070708070707080a4707060609060c0b07080a07090808080737070b09060706070707070707070707094807080b0607070707060708074107080709070706070707080607060706070808070a460a0d06090709060b060707060a07070c0907060b06060b070a090707080707070b0707070c060b08070b070a09070b07070b08080706070707070807080707090d070707060707070609070a090807070d0707070b09070707070706070a0908070a0807060b0a080707090707090b08090a08070707080707070e07060708070709080b06070b0a0707070a06070606070809060a07080b07070a070c07070808070e070807070c07090607070707060707080b0743090708'); INSERT INTO t1_data VALUES(274877906950,X'00040e1807020e0830cebc66f09f97b60604840703016b06021003016c0804835103016d0a0484050301710602560301730304822d030275770904834e0301770204824d0302797707022e0302c2b2070482640402bc6d0a025e0303c3a6370602530305cebc7768690a04823f0201be0504816f0301780a0482780201bf0a04833a0305f09e919a790a0483440103cf80340604827e0201810404823e0201830304843a0504837f0301780704834c02018407048139020186030881038372030178010231020187010481100201880204835b0204845a0104825a0201890204825c0604831802018a0704841502028c340304845e02018e0602180301390204833c0203936e770104820f0201940704823702039765630304847b020199080484110301710704821402029b650704827102019d0304845802019f0504845c0303e7ba880502690202a163090484160201a3060483020301680102470201a90a0482480201ab0104837f030482570201af050482190201b20a0481020201b30504842903026f64040483650203b873780704826d0201bb0704831c030177020481410102d0b0070482560202b17a050484460201b2040483400201b40402390202b671020484220202b777090483510201ba030481750202bb6301021e0202bc620a0484080201bd0204815d0201be060482470201bf040483340102d1800904825002028470050482290202876f06022202048872c5820104826202028b680504845d02018c0a027302018d0a0205020197090483210202986b0a02780201a3030481580201a70604835e020483670201a901048379060483780201ad0a04816b0201af010482310201b1080482020201b3060484610202b5640804814c03026e690304847c0201bb080482710201bd020482020201bf06048435010482590107d28b37c2ba61640104831702018d0704810b020393c5890204836e0201950602070305f6a5989a74040484450201970104813702029f790a04843f0202a57407027f0201a90504835b0202ab750502490202b9760a04827d0301780304826e0202bb6f060481780203bd7130050483630103d3866f09023602028a6c0604825302018c0304845302028e650404833a02028f6f0104821302039d356d090484320201a5020481130404813f0201ab0704835b0201b90a0482380201bd0904833c0201bf060484450302307604027f0102d491050483300201930a024603036bd291070483060201970704826702079b72c2b3356f730904844f02019d010482610201a10404814d0202a36b0502250203a5c7b3080483780201ab02023d0103d5a1640102260201a206026a0301660804813c0201a5030481180201a7050483360201a9040483510201aa060482020201ab0504811e0201bb090481680202bd740702160202be700304846d0102d6840a027d02068534f0988d910a0483730102d78f0404832804048462020390766601027802019202027405024b0201940a04834d020298770904815702039a6c6b0304846802049c73616e080483420201b0010481050202b565050481290206b67376c2aa640a0483280201b80704830e0102d89d0202150802720201a60602480202b073090481480201ba0204836f0102d9a50202670201b40a04822a030337c2b20a04820f0102da800604831c0201840604842802038636380804811e020188090484190201920504815402019a0304830c0201a301023d0201a6040481460201a8020484100201af0104842a0204825f0201b50304827c0201bb0804830f0105dbb878c2bc0502580107dc97e69eaf6764020259020299610104812902029d680502110202a0690a0483270203a56372030483540202a96a0204842c0205acf094abac0704821c0102dd92020483410203946d790304814f020395d7ac090228020296300404830302039b397809020a02029c770504833d0202a2700504821a0201ab010483400202b0760a04813a0201b2040483130203b3c7a10104833d0204b4e8a5a40604816f0201b90604830d0201ba090483630102de800904810a020281740a02380201880704826802018f030481670201900202790203916e710404833502019206048418020399ddba080481240202a3630a02250202b86e070481670204830d0201b90604811f0201bc070481530102df820102390201830304830102038e367901024f02039134620a0482080201920104814b020198030484790202a06b060481240201ba0802090201be08022f03023670010482170202bf6f01022d0103e0a1960304831d0302bb75060483700202a2900804817c0202a49e050485010202a6b2040482030202aa80040481720202abb30304843c0301ba0204833b0203aca677010481120206ae9f613566730504841e0202afa001020a0302a979080484060202b1a1010483730203b2a46a070482460301ba020484580203b39b610804834a0202b6a60304813b0202bfae0404837c0103e185930304845f02038889780a04826802028a85070484620301b90304826c020298a3070483270203af8962050484620202b5b7070481170202b8970602190202bd8f0a0482620103e2848f04021b0203afa861020484010301b70704824a0103e3828f05048214020284a50304812f0202909e0904824d020292b907026302049ba0336604020602029c8e0204840d0202ad860202750302a16a0802180202ae880602040202b480010483150202b7b1090483780202bdbf070482050103e480ab06020f0202889605024f020293b9030483310204968fdc93080483110202a295010484150202a69a0804844d0202a78e0a02080202aa9b0a04825f0203b2b8670402720203b5b764060482550204babacf87070481270202bd9b010481780206be9a6878696805021e0103e58bba0304821202049eab6a630a04827702029faf08024d0202a2b50a02400202a3af0704827c0203a4bd6f0504840c0202ad9d030485010202b1be040483780202b5b9060484500103e689b10404844602028abc0704811902038e98690802320203968b35090482680203979971010483060202b6a4030484050301be0a04833e0103e780a601048171020282be0304824d020286b105021b02028f960a048266020291be0804831c020299900204815502029ea70702400202a0ba040481410204a69e3870030481220202a9800602310202afb8020482170203b1af6d0802140203b5a06a0702740104e88c9a6c06048426020295bc05048326020296af0304846b02059cad72d4aa060484780205baa97364680404837b0103e983840104843902029a8a010481490203a1ac770504812c0301ba050483410202af8c0302620202b1af0302500103ea8fa60a023e020297b50904812e020298a90704825702039ea76b0304847f0202a59e080481460202a68e080482590204a9b1c6a8010481350203b38d75040482750203b89e6f0a0484030103eb828f0704824e020384837109048178020397b6690104821a02039e9e330704837f0203a68b73030484440203b980320804822c0103ec8fa80204814302039588760a04826902029bb80a024e0203afb6630802710203b4a268060483000103ed8cb50104811602029982060481550104ee8085760304834502028a870904845702028f8b04027602039da470030483040202a1a20504813c0202a8ae0504837f0202af9d0402570205b1b773d8b4080482120202bd880904832c0202be930104827b0104ef88ae680704835d0205a8ab63c2be06022e0203aaa16c080481090204adb779660804823f0202b8ae080484220203bb9165030481340106f08798b1636a0604824f0203908ea80404826203029f880402530303b4ad740a04833302039184b6090484650302acb3080483340302b1b703048272020a9281a23578f098b1893001048210030291a50404816803029d8f070483200302afa5080484440302b49f020481010303bd833603026a020393839a07024c0304a3ad6e300504842d0303abb16d0502660302adae0904830e02039481a406048164030386a16c070483530302b2a8050484320302bfb9080483170204958eb8720304832b0302b1ba090481200302babe05048209020496888b6708048358030293b4050483400304b4927a6f050483770204978b93320104810a03048fb9c7890704822803029ba702022a0302a39d060484490302a4be0a0484530302adaa0a026b0302af92040483700302b9bc0404821b02049884b56404048352030486b1716f0904827a030296b80904811d03049d9e797a060483390302a5a00a04844a0303b5856d0a04822e0302bcbd040483310303bd91740704845102039983a20a04816503039d8a330a02140303b9826b0404814e02069a8b8438687403026b0302a599010481740302a6ac0704840f0303b6896d0404831a0302b785010484370302bfac08027d02059b86b770610404845203028fbd020483550303a0be75090483620302a295040482390302b6970404813002039c97bb0a04841103059d8976c2be0604824e0302afbc05023c02039d87b40504825903028daf03048172030292be040482310302a0ab0502290303b2a16b03025d02039e929d050483040302a69f060484150302aa8607025502049f9f8b66020483160302a3be060481120203a8afad0904835e0204acb1af640204845b0204b282a47a040481610203baa68a010482490204bb9f8366060483130203bfa48f0704817c0104f18191920802220204828cad360a0482730203859c9a050481670204888eb23408025f020492ba9c3202026502049bb0bf7008021f02039f94a60104835c0203a6b496030482760203a98c960504824e0203acb7b8030483240203afa0b703023d0206b3a38c683972060483340203bbaaad0902070105f282848f790804833a020384b79f0a048170020590a98c616a0804843b02049a92be730904822f02039da18b080482530203a28f8f06048309070d06070706070807070807080b0707070b09070b07070906070f0b07080607090709070708070708080706070b070707080907070808070608080707080707070808070a0806060707070b0b07070707080807070b0d0709060b0708070707080708090808070808090b0707070707080609070d07070709060806070707070707070708070c0c08090708090a07080c070a0608070707090807090707070607070b07070a0c08070809080b08090808080808070807090a07070807070706090709070c070707070809070708060608070908080808080807090c07080809070908080909080708090807080809070908080709080707070808080807080a0808070808090a080b090a070708090808080908080909080709080708080807080a070808080a08080b0b0908090707070808080908080a0909090909090909090907080909080a0807090808070b08080a0a090a08090c090709090808100808080808080a0808090908080a08080a080a0a0a0708080708080a0a080a080908090908090b08080908070b08090808090b0709080807080908070a08090a0a090a09090a0909090909090909080c080b090b0a09'); INSERT INTO t1_data VALUES(274877906951,X'000002020630f2abab907a060481130204b381af77040481790203bea6a6070484250108f380b5a33633737701048229020383be9f0404837102048495bf73070481060204938c9a6e0904823502039eb39201022302049fbb9f6c0a0481500204a3909e620404837d0204a9ac9a75020483110203bd8c850a048111030293ad0a048247030294b9040483550203bea4ac0702220105f483b5ab66030482440302bda306048141020584b2bb643804048142020586b1a8677a0204831d0205878c916c770804821f02038da1bf0704824302038f81bd0304825102039e91b20302420203b29b9403021f0203b3a1b10704825a0203b5afb4040481210204b98da0780a02060104f582b1b00404811002038f9fbc010482200302b8a406048419020394b7b80404834c02039bb5b005022b0203aa88b9060481390203aba48f0604842a0205af969438680602360203b88d8903026c0104f68386ad0904846702049aae8f79030483520203a8989a0604836c0203b18cb4070482260203b9a98d050483730203bebc870104824c0104f78283840804816703028db20302600303a8b46106020b020389b2a80104835002048e809339010481190205929193d0b409048253020393aebc0204816202059787b96f690704817202079b949dc78934620404817f02039db28b020483750205a288a7c7a3080482780203a5948d0204811a040b0a090e090a0a080a0a0a090808080b080b0b0b090908080909090a0908090809090a080a0a090909090a0708090a0b090b0d090b'); CREATE TABLE IF NOT EXISTS 't1_idx'(segid, term, pgno, PRIMARY KEY(segid, term)) WITHOUT ROWID; INSERT INTO t1_idx VALUES(2,X'',2); |
︙ | ︙ | |||
6972 6973 6974 6975 6976 6977 6978 | INSERT INTO t2 VALUES('integrity-check'); PRAGMA writable_schema=OFF; COMMIT; } {} do_catchsql_test 51.1 { SELECT max(rowid)==0 FROM t1('e*'); | | | 6971 6972 6973 6974 6975 6976 6977 6978 6979 6980 6981 6982 6983 6984 6985 | INSERT INTO t2 VALUES('integrity-check'); PRAGMA writable_schema=OFF; COMMIT; } {} do_catchsql_test 51.1 { SELECT max(rowid)==0 FROM t1('e*'); } {/.*fts5: corrupt.*/} #-------------------------------------------------------------------------- reset_db do_test 52.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 40960 pagesize 4096 filename crash-2b92f77ddfe191.db |
︙ | ︙ | |||
8748 8749 8750 8751 8752 8753 8754 | | 4048: 00 00 00 00 00 00 5d 03 00 00 00 00 00 00 00 00 ......]......... | end crash-c77b90b929dc92.db }]} {} do_catchsql_test 60.2 { SELECT (matchinfo(t1,591)) FROM t1 WHERE t1 MATCH 'e*e' | | | 8747 8748 8749 8750 8751 8752 8753 8754 8755 8756 8757 8758 8759 8760 8761 | | 4048: 00 00 00 00 00 00 5d 03 00 00 00 00 00 00 00 00 ......]......... | end crash-c77b90b929dc92.db }]} {} do_catchsql_test 60.2 { SELECT (matchinfo(t1,591)) FROM t1 WHERE t1 MATCH 'e*e' } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- do_test 61.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 28672 pagesize 4096 filename crash-e5fa281edabddf.db |
︙ | ︙ | |||
8954 8955 8956 8957 8958 8959 8960 | CREATE VIRTUAL TABLE t3 USING fts5vocab('t1'(),'col' ); } {/*malformed database schema*/} do_catchsql_test 61.2 { SELECT * FROM t3 ORDER BY rowid; } {/*malformed database schema*/} | < | 8953 8954 8955 8956 8957 8958 8959 8960 8961 8962 8963 8964 8965 8966 | CREATE VIRTUAL TABLE t3 USING fts5vocab('t1'(),'col' ); } {/*malformed database schema*/} do_catchsql_test 61.2 { SELECT * FROM t3 ORDER BY rowid; } {/*malformed database schema*/} #------------------------------------------------------------------------- do_test 62.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 28672 pagesize 4096 filename crash-44942694542e1e.db | page 1 offset 0 |
︙ | ︙ | |||
9770 9771 9772 9773 9774 9775 9776 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end crash-37cecb4e784e9f.db }]} {} do_catchsql_test 66.1 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | 9768 9769 9770 9771 9772 9773 9774 9775 9776 9777 9778 9779 9780 9781 9782 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end crash-37cecb4e784e9f.db }]} {} do_catchsql_test 66.1 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- # reset_db do_test 67.0 { sqlite3 db {} db deserialize [decode_hexdb { |
︙ | ︙ | |||
10320 10321 10322 10323 10324 10325 10326 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 00 00 00 00 00 00 ity-check....... | end crash-31c462b8b665d0.db }]} {} do_catchsql_test 69.2 { SELECT * FROM t1 WHERE a MATCH 'fx*' | | | 10318 10319 10320 10321 10322 10323 10324 10325 10326 10327 10328 10329 10330 10331 10332 | | 4064: 69 74 79 2d 63 68 65 63 6b 09 00 00 00 00 00 00 ity-check....... | end crash-31c462b8b665d0.db }]} {} do_catchsql_test 69.2 { SELECT * FROM t1 WHERE a MATCH 'fx*' } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 70.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb |
︙ | ︙ | |||
10503 10504 10505 10506 10507 10508 10509 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end sql025294.txt.db }]} {} do_catchsql_test 71.2 { INSERT INTO t1(t1) VALUES('integrity-check'); | | | 10501 10502 10503 10504 10505 10506 10507 10508 10509 10510 10511 10512 10513 10514 10515 | | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 69 7a 65 heck....optimize | end sql025294.txt.db }]} {} do_catchsql_test 71.2 { INSERT INTO t1(t1) VALUES('integrity-check'); } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 72.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb |
︙ | ︙ | |||
10630 10631 10632 10633 10634 10635 10636 | | end crash-77b86d070d0ac6.db }]} {} do_catchsql_test 72.1 { INSERT INTO ttt(ttt) VALUES('integrity-check'); } {1 {database disk image is malformed}} | | | | 10628 10629 10630 10631 10632 10633 10634 10635 10636 10637 10638 10639 10640 10641 10642 10643 10644 | | end crash-77b86d070d0ac6.db }]} {} do_catchsql_test 72.1 { INSERT INTO ttt(ttt) VALUES('integrity-check'); } {1 {database disk image is malformed}} do_catchsql_test 72.2 { SELECT 1 FROM ttt('e* NOT ee*'); } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 73.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb |
︙ | ︙ | |||
10758 10759 10760 10761 10762 10763 10764 | | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-b02ca2cc4d7dda.db }]} {} do_catchsql_test 73.1 { SELECT snippet(ttt,ttt, NOT 54 ), * FROM ttt('e* NOT ee*e* NOT ee* NOT ee*e* NOT e*') ; | | > | 10756 10757 10758 10759 10760 10761 10762 10763 10764 10765 10766 10767 10768 10769 10770 10771 10772 10773 10774 10775 10776 | | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-b02ca2cc4d7dda.db }]} {} do_catchsql_test 73.1 { SELECT snippet(ttt,ttt, NOT 54 ), * FROM ttt('e* NOT ee*e* NOT ee* NOT ee*e* NOT e*') ; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 74.0 { sqlite3 db {} sqlite3_fts5_register_matchinfo db db deserialize [decode_hexdb { | size 106496 pagesize 4096 filename x.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 01 00 00 00 1a .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 04 ................ | 80: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 01 ................ |
︙ | ︙ | |||
14583 14584 14585 14586 14587 14588 14589 | | 4032: b8 64 0a 02 34 02 03 b2 9a 9e 05 04 83 16 02 04 .d..4........... | 4048: b4 a7 93 36 06 04 83 7d 03 02 ba 9c 02 04 83 07 ...6............ | 4064: 04 09 09 0a 0a 0c 0b 08 09 09 08 09 08 09 0a 0a ................ | 4080: 0a 09 0a 0a 08 0a 08 09 0a 09 09 09 09 09 09 0a ................ | end x.db }]} {} | | | > > > > > | 14582 14583 14584 14585 14586 14587 14588 14589 14590 14591 14592 14593 14594 14595 14596 14597 14598 14599 14600 14601 14602 14603 14604 14605 14606 14607 14608 | | 4032: b8 64 0a 02 34 02 03 b2 9a 9e 05 04 83 16 02 04 .d..4........... | 4048: b4 a7 93 36 06 04 83 7d 03 02 ba 9c 02 04 83 07 ...6............ | 4064: 04 09 09 0a 0a 0c 0b 08 09 09 08 09 08 09 0a 0a ................ | 4080: 0a 09 0a 0a 08 0a 08 09 0a 09 09 09 09 09 09 0a ................ | end x.db }]} {} do_catchsql_test 74.0.5 { SELECT matchinfo(1,2); } {1 {unable to use function matchinfo in the requested context}} do_catchsql_test 74.1 { SELECT rowid, quote(matchinfo(t1,'pxyb<s')) FROM t1 WHERE t1 MATCH 'e*'; } {1 {unrecognized matchinfo flag: <}} #------------------------------------------------------------------------- reset_db do_test 75.0 { sqlite3 db {} sqlite3_fts5_register_matchinfo db db deserialize [decode_hexdb { | size 32768 pagesize 4096 filename crash-033d665d5caa8d.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 00 .....@ ........ | 96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36 ...............6 | 112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 00 00 00 00 00 00 ...k............ |
︙ | ︙ | |||
14787 14788 14789 14790 14791 14792 14793 | | page 8 offset 28672 | 4048: 00 00 00 00 00 00 5d 03 02 2b 69 6e 74 00 00 00 ......]..+int... | end crash-033d665d5caa8d.db }]} {} do_catchsql_test 75.1 { SELECT rowid, quote(matchinfo(t1,'pcxybs')) FROM t1 WHERE t1 MATCH 'e*'; | | | 14791 14792 14793 14794 14795 14796 14797 14798 14799 14800 14801 14802 14803 14804 14805 | | page 8 offset 28672 | 4048: 00 00 00 00 00 00 5d 03 02 2b 69 6e 74 00 00 00 ......]..+int... | end crash-033d665d5caa8d.db }]} {} do_catchsql_test 75.1 { SELECT rowid, quote(matchinfo(t1,'pcxybs')) FROM t1 WHERE t1 MATCH 'e*'; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 76.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 40960 pagesize 4096 filename crash-03b68c01d30713.db |
︙ | ︙ | |||
15514 15515 15516 15517 15518 15519 15520 15521 15522 15523 15524 15525 | | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 00 00 00 00 ........vers.... | end crash-f928a9c1ec68dd.db }]} {} do_catchsql_test 80.1 { SELECT snippet(rowid, -1, '.', '..', '[', '(]'),snippet(rowid, -1, '.', '.', '', '(]'), highlight(t1, 29, 1 , '') FROM t1('g+ h') WHERE rank MATCH 'bm25(1.0, 10)' ORDER BY NOT (SELECT 1 FROM t1('g+ æ') WHERE rank MATCH 'bm25(1.0, 10)' ORDER BY rank); } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 15518 15519 15520 15521 15522 15523 15524 15525 15526 15527 15528 15529 15530 15531 15532 15533 15534 15535 15536 15537 15538 15539 15540 15541 15542 15543 15544 15545 15546 15547 15548 15549 15550 15551 15552 15553 15554 15555 15556 15557 15558 15559 15560 15561 15562 15563 15564 15565 15566 15567 15568 15569 15570 15571 15572 15573 15574 15575 15576 15577 15578 15579 15580 15581 15582 15583 15584 15585 15586 15587 15588 15589 15590 15591 15592 15593 15594 15595 15596 15597 15598 15599 15600 15601 15602 15603 15604 15605 15606 15607 15608 15609 15610 15611 15612 15613 15614 15615 15616 15617 15618 15619 15620 15621 15622 15623 15624 15625 15626 15627 15628 15629 15630 15631 15632 15633 15634 15635 15636 15637 15638 15639 15640 15641 15642 15643 15644 15645 15646 15647 15648 15649 15650 15651 15652 15653 15654 15655 15656 15657 15658 15659 15660 15661 15662 15663 15664 15665 15666 15667 15668 15669 15670 15671 15672 15673 15674 15675 15676 15677 15678 15679 15680 15681 15682 15683 15684 15685 15686 15687 15688 15689 15690 15691 15692 15693 15694 15695 15696 15697 15698 15699 15700 15701 15702 15703 15704 15705 15706 15707 15708 15709 15710 15711 15712 15713 15714 15715 15716 15717 15718 15719 15720 15721 15722 15723 15724 15725 15726 15727 15728 15729 15730 15731 15732 15733 15734 15735 15736 15737 15738 15739 15740 15741 15742 15743 15744 15745 15746 15747 15748 15749 15750 15751 15752 15753 15754 15755 15756 15757 15758 15759 15760 15761 15762 15763 15764 15765 15766 15767 15768 15769 15770 15771 15772 15773 15774 15775 15776 15777 15778 15779 15780 15781 15782 15783 15784 15785 15786 15787 15788 15789 15790 15791 15792 15793 15794 15795 15796 15797 15798 15799 15800 15801 15802 15803 15804 15805 15806 15807 15808 15809 15810 15811 15812 15813 15814 15815 15816 15817 15818 15819 15820 15821 15822 15823 15824 15825 15826 15827 15828 15829 15830 15831 15832 15833 15834 15835 15836 15837 15838 15839 15840 15841 15842 15843 15844 15845 15846 15847 15848 15849 15850 15851 15852 15853 15854 15855 15856 15857 15858 15859 15860 15861 15862 15863 15864 15865 15866 15867 15868 15869 15870 15871 15872 15873 15874 15875 15876 15877 15878 15879 15880 15881 15882 15883 15884 15885 15886 15887 15888 15889 15890 15891 15892 15893 15894 15895 15896 15897 15898 15899 15900 15901 15902 15903 15904 15905 15906 15907 15908 15909 15910 15911 15912 15913 15914 15915 15916 | | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 00 00 00 00 ........vers.... | end crash-f928a9c1ec68dd.db }]} {} do_catchsql_test 80.1 { SELECT snippet(rowid, -1, '.', '..', '[', '(]'),snippet(rowid, -1, '.', '.', '', '(]'), highlight(t1, 29, 1 , '') FROM t1('g+ h') WHERE rank MATCH 'bm25(1.0, 10)' ORDER BY NOT (SELECT 1 FROM t1('g+ æ') WHERE rank MATCH 'bm25(1.0, 10)' ORDER BY rank); } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 81.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 40960 pagesize 4096 filename crash-44e8035a976422.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04 ................ | 48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ | 96: 00 00 00 00 0d 00 00 00 0d 0b 6e 00 0f a3 0f 4c ..........n....L | 112: 0e e1 0e 81 0e 24 0d cc 0d 72 0d 1b 0c b0 0c 50 .....$...r.....P | 128: 0b f8 0b b3 0b 6e 00 00 00 00 00 00 00 00 00 00 .....n.......... | 2912: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 43 0d ..............C. | 2928: 06 17 11 11 08 75 74 61 62 6c 65 74 34 74 34 43 .....utablet4t4C | 2944: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 REATE VIRTUAL TA | 2960: 42 4c 45 20 74 34 20 55 53 49 4e 47 20 66 74 73 BLE t4 USING fts | 2976: 35 76 6f 63 61 62 28 27 74 32 27 2c 20 27 72 6f 5vocab('t2', 'ro | 2992: 77 27 29 43 0c 06 17 11 11 08 75 74 61 62 6c 65 w')C......utable | 3008: 74 33 74 33 43 52 45 41 54 45 20 56 49 52 54 55 t3t3CREATE VIRTU | 3024: 41 4c 20 54 41 42 4c 45 20 74 33 20 55 53 49 4e AL TABLE t3 USIN | 3040: 47 20 66 74 73 35 76 6f 63 61 62 28 27 74 31 27 G fts5vocab('t1' | 3056: 2c 20 27 72 6f 77 27 29 56 0b 06 17 1f 1f 01 7d , 'row')V....... | 3072: 74 61 62 6c 65 74 32 5f 63 6f 6e 66 69 67 74 32 tablet2_configt2 | 3088: 5f 63 6f 6e 66 69 67 0a 43 52 45 41 54 45 20 54 _config.CREATE T | 3104: 41 42 4c 45 20 27 74 32 5f 63 6f 6e 66 69 67 27 ABLE 't2_config' | 3120: 28 6b 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 (k PRIMARY KEY, | 3136: 76 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 v) WITHOUT ROWID | 3152: 5e 0a 07 17 21 21 01 81 07 74 61 62 6c 65 74 32 ^...!!...tablet2 | 3168: 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 6f 6e 74 65 _contentt2_conte | 3184: 6e 74 09 43 52 45 41 54 45 20 54 41 42 4c 45 20 nt.CREATE TABLE | 3200: 27 74 32 5f 63 6f 6e 74 65 6e 74 27 28 69 64 20 't2_content'(id | 3216: 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 INTEGER PRIMARY | 3232: 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 29 KEY, c0, c1, c2) | 3248: 69 09 07 17 19 19 01 81 2d 74 61 62 6c 65 74 32 i.......-tablet2 | 3264: 5f 69 64 78 74 32 5f 69 64 78 08 43 52 45 41 54 _idxt2_idx.CREAT | 3280: 45 20 54 41 42 4c 45 20 27 74 32 5f 69 64 78 27 E TABLE 't2_idx' | 3296: 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 67 (segid, term, pg | 3312: 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 28 no, PRIMARY KEY( | 3328: 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 49 segid, term)) WI | 3344: 54 48 4f 55 54 20 52 4f 57 49 44 55 08 07 17 1b THOUT ROWIDU.... | 3360: 1b 01 81 01 74 61 62 6c 65 74 32 5f 64 61 74 61 ....tablet2_data | 3376: 74 32 5f 64 61 74 61 07 43 52 45 41 54 45 20 54 t2_data.CREATE T | 3392: 41 42 4c 45 20 27 74 32 5f 64 61 74 61 27 28 69 ABLE 't2_data'(i | 3408: 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 d INTEGER PRIMAR | 3424: 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c 4f Y KEY, block BLO | 3440: 42 29 58 07 07 17 11 11 08 81 1d 74 61 62 6c 65 B)X........table | 3456: 74 32 74 32 43 52 45 41 54 45 20 56 49 52 54 55 t2t2CREATE VIRTU | 3472: 41 4c 20 54 41 42 4c 45 20 74 32 20 55 53 49 4e AL TABLE t2 USIN | 3488: 47 20 66 74 73 35 28 27 61 27 2c 5b 62 5d 2c 22 G fts5('a',[b],. | 3504: 63 22 2c 64 65 74 61 69 6c 3d 6e 6f 6e 65 2c 63 c.,detail=none,c | 3520: 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 56 06 06 17 olumnsize=0)V... | 3536: 1f 1f 01 7d 74 61 62 6c 65 74 31 5f 63 6f 6e 66 ....tablet1_conf | 3552: 69 67 74 31 5f 63 6f 6e 66 69 67 06 43 52 45 41 igt1_config.CREA | 3568: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f 6e TE TABLE 't1_con | 3584: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b fig'(k PRIMARY K | 3600: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52 EY, v) WITHOUT R | 3616: 4f 57 49 44 5b 05 07 17 21 21 01 81 01 74 61 62 OWID[...!!...tab | 3632: 6c 65 74 31 5f 64 6f 63 73 69 7a 65 74 31 5f 64 let1_docsizet1_d | 3648: 6f 63 73 69 7a 65 05 43 52 45 41 54 45 20 54 41 ocsize.CREATE TA | 3664: 42 4c 45 20 27 74 31 5f 64 6f 63 73 69 7a 65 27 BLE 't1_docsize' | 3680: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d (id INTEGER PRIM | 3696: 41 52 59 20 4b 45 59 2c 20 73 7a 20 42 4c 4f 42 ARY KEY, sz BLOB | 3712: 29 5e 04 07 17 21 21 01 81 07 74 61 62 6c 65 74 )^...!!...tablet | 3728: 31 5f 63 6f 6e 74 65 6e 74 74 31 5f 63 6f 6e 74 1_contentt1_cont | 3744: 65 6e 74 04 43 52 45 41 54 45 20 54 41 42 4c 45 ent.CREATE TABLE | 3760: 20 27 74 31 5f 63 6f 6e 74 65 6e 74 27 28 69 64 't1_content'(id | 3776: 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 52 59 INTEGER PRIMARY | 3792: 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c 20 63 32 KEY, c0, c1, c2 | 3808: 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 74 )i.......-tablet | 3824: 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 41 1_idxt1_idx.CREA | 3840: 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 78 TE TABLE 't1_idx | 3856: 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 70 '(segid, term, p | 3872: 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 59 gno, PRIMARY KEY | 3888: 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 57 (segid, term)) W | 3904: 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 17 ITHOUT ROWIDU... | 3920: 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 74 .....tablet1_dat | 3936: 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 20 at1_data.CREATE | 3952: 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 28 TABLE 't1_data'( | 3968: 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d 41 id INTEGER PRIMA | 3984: 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 4c RY KEY, block BL | 4000: 4f 42 29 5b 01 07 17 11 11 08 81 23 74 61 62 6c OB)[.......#tabl | 4016: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54 et1t1CREATE VIRT | 4032: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49 UAL TABLE t1 USI | 4048: 4e 47 20 66 74 73 35 28 61 2c 62 20 75 6e 69 6e NG fts5(a,b unin | 4064: 64 65 78 65 64 2c 63 2c 74 6f 6b 65 6e 69 7a 65 dexed,c,tokenize | 4080: 3d 22 70 6f 72 74 65 72 20 61 73 63 69 69 22 29 =.porter ascii.) | page 2 offset 4096 | 0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c ..h............| | 16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .*.............. | 3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 .......0........ | 3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80 ..........7..... | 3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01 ...t.....0a..... | 3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01 b.....c.....g... | 3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01 ....h.......i... | 3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00 ............*... | 3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80 ............%... | 3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02 .....P.....0g... | 3984: 01 02 02 01 01 68 02 08 03 8d 02 03 01 01 6a 42 .....h........jB | 4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 80 01 03 ........7....... | 4016: 00 74 00 00 00 2e 02 30 61 01 12 02 01 01 62 01 .t.....0a.....b. | 4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02 ....c.....g..... | 4048: 01 01 68 01 05 01 02 03 01 01 69 01 06 01 02 04 ..h.......i..... | 4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00 ................ | 4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 3 offset 8192 | 0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c ................ | 4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02 ................ | page 4 offset 12288 | 0: 0d 00 00 00 03 0f be 00 0f ea 00 00 00 00 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03 ................ | 4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67 .....a b cg h ig | 4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69 h i.......g h i | 4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17 a b cg h i...... | 4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69 .a b cd e fg h i | page 5 offset 16384 | 0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03 ................ | 4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03 ................ | page 6 offset 20480 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 7 offset 24576 | 0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00 ................ | 3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84 ..............A. | 4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61 ............4.0a | 4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01 .....b.....c.... | 4032: e6 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01 .d...e...f...g.. | 4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06 ...h.....i...... | 4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f ................ | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 8 offset 28672 | 0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 9 offset 32768 | 0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03 ................ | 4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67 .....a b cg h ig | 4048: 20 68 21 69 14 02 05 00 17 17 17 67 20 68 20 69 h!i.......g h i | 4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17 a b cg h i...... | 4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69 .a b cd e fg h i | page 10 offset 36864 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-44e8035a976422.db }]} {} do_catchsql_test 81.2 { UPDATE t1 SET b=zeroblob(299); } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 82.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 32768 pagesize 4096 filename c0.txt.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08 .....@ ........ | 32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04 ................ | 96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36 ...............6 | 112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00 ...k............ | 3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06 WITHOUT ROWID[. | 3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21 Y, sz BLOB)^...! | 3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74 !...tablet1_cont | 3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52 entt1_content.CR | 3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 EATE TABLE 't1_c | 3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 ontent'(id INTEG | 3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19 c0, c1, c2)i.... | 3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 ...-tablet1_idxt | 3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42 1_idx.CREATE TAB | 3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69 LE 't1_idx'(segi | 3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 d, term, pgno, P | 3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64 RIMARY KEY(segid | 3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54 , term)) WITHOUT | 3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74 ROWIDU........t | 3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61 ablet1_datat1_da | 3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 ta.CREATE TABLE | 3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54 't1_data'(id INT | 3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 EGER PRIMARY KEY | 3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06 , block BLOB)8.. | 3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52 ...._tablet1t1CR | 4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 EATE VIRTUAL TAB | 4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 LE t1 USING fts5 | 4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00 (a,b,c)......... | page 3 offset 8192 | 0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00 ................ | 3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18 .....J.......... | 3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06 ...+.00......... | 3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30 ...........20160 | 3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01 609...........4. | 3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02 ..........5..... | 3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04 ......0000000... | 3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06 ........binary.. | 3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70 ............comp | 3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 iler...........d | 3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 bstat........... | 3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 ebug...........e | 3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 nable........... | 3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 ................ | 3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 ................ | 3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 ................ | 3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 ......xtension.. | 3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03 .........fts4... | 3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01 ........5....... | 3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03 ....gcc......... | 3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02 ..eopoly........ | 3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02 ...json1........ | 3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03 ...load......... | 3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05 ..max........... | 3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04 emory........... | 3648: 73 79 73 35 16 02 03 01 02 02 03 01 03 01 06 6e sys5...........n | 3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03 ocase........... | 3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 ................ | 3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02 ...omit......... | 3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03 ..rtree......... | 3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06 ..im............ | 3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56 ..threadsafe.W4V | 3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02 .d.F...vtab..... | 3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01 ......x......... | 3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02 ................ | 3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 3936: 01 02 01 06 01 01 10 01 06 01 01 02 01 06 01 01 ................ | 3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 ................ | 3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 ................ | 4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c ................ | 4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f .D...G.......... | 4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f D..@.......$Z$$. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 4 offset 12288 | 0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 5 offset 16384 | 0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74 ....$..........t | 16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5 .a.N./.......... | 32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5 ...t.[.@.$...... | 48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb .......h.O.5.... | 64: 0c da 0c b9 0c 99 0c 78 0c 57 00 00 00 00 00 00 .......x.W...... | 3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f ...........$..%. | 3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49 .THREADSAFE=0XBI | 3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41 NARY.#..%..THREA | 3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22 DSAFE=0XNOCASE.. | 3136: 05 00 25 0f 17 54 48 52 45 41 44 53 31 46 45 3d ..%..THREADS1FE= | 3152: 30 58 52 64 52 49 4d 1f 21 05 00 33 0f 19 4f 4d 0XRdRIM.!..3..OM | 3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f IT LOAD EXTENSIO | 3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f NXBINARY. ..3..O | 3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 MIT LOAD EXTENSI | 3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17 ONXNOCASE....3.. | 3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 OMIT LOAD EXTENS | 3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19 IONXRTRIM....3.. | 3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30 MAX MEMORY=50000 | 3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f 000XBINARY....3. | 3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 .MAX MEMORY=5000 | 3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33 0000XNOCASE....3 | 3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 ..MAX MEMORY=500 | 3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25 00000XRTRIM....% | 3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42 ..ENABLE RTREEXB | 3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17 LE RTREEXNOCASE. | 3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52 ...%..ENABLE RTR | 3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45 EEXRTRIM....)..E | 3440: 4e 41 42 4b 45 20 4d 45 4d 53 59 53 35 58 42 49 NABKE MEMSYS5XBI | 3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c NARY....)..ENABL | 3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45 B`-EMSYS5XNOCASE | 3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45 ....)..ENABLE ME | 3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25 MSYS5XRTRIM....% | 3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42 ..ENABLE JSON1XB | 3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17 LE JSON1XNOCASE. | 3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f ...%..ENABLE JSO | 3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45 N1XRTRIM....)..E | 3600: 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49 4e 41 BLE GEOPOLYXBINA | 3616: 52 59 1a 11 05 00 39 0f 19 45 4e 41 42 4c 45 2e RY....9..ENABLE. | 3632: 41 40 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 40 A@GEOPOLYXNOCAS@ | 3648: 4f 4c 59 58 55 19 10 05 00 29 0f 17 45 4e 41 42 OLYXU....)..ENAB | 3664: 4c 45 20 47 45 4f 52 54 52 49 4d 17 0f 05 00 23 LE GEORTRIM....# | 3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49 ..ENABLE FTS5XBI | 3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c NARY....#..ENABL | 3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05 E FTS5XNOCASE... | 3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58 .#..ENABLE FTS5X | 3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42 RTRIM....#..ENAB | 3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b LE FTS4XBINARY.. | 3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34 ..#..ENABLE FTS4 | 3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e XNOCASE....#..EN | 3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e ABLE FTS4XRTRIM. | 3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e TAT VTABXBINARY. | 3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d TAT VT$.HNOCASE. | 3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06 TAT VTABXRTRIM.. | 3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52 .....DEBUGXBINAR | 3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f Y.......DEBUGXNO | 3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47 CASE.......DEBUG | 3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d XRTRIM'...C..COM | 3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20 PILER=gcc-5.4.0 | 4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27 20160609XBINARY' | 4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67 ...C..COMPILER=g | 4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30 cc-5.4.0 2016060 | 4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43 9XNOCASE&...C..C | 4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e OMPILER=gcc-5.4. | 4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d 0 20160609XRTRIM | page 6 offset 20480 | 0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0 ....$........... | 16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0 ................ | 32: 1f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60 .........x.p.h.` | 48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20 .X.P.H.@.8.0.(. | 64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 00 00 ................ | 3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01 .$.......#...... | 3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01 .........!...... | 3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01 . .............. | 3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01 ................ | 3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01 ................ | 3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01 ................ | 3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01 ................ | 3920: 06 15 f3 00 12 02 01 01 06 15 03 00 12 02 01 01 ................ | 3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01 ................ | 3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01 ................ | 3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01 ................ | 3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01 ................ | 4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01 ................ | 4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01 ................ | 4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01 ................ | 4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01 ................ | 4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01 ................ | 4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01 ................ | page 7 offset 24576 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 8 offset 28672 | 0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00 ................ | 4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72 .........+integr | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 00 00 00 00 uild....opti.... | end c0.txt.db }]} {} do_execsql_test 82.2 { UPDATE t1 SET b=quote(zeroblob(current_date)) WHERE t1 MATCH 't*'; PRAGMA writable_schema=ON; UPDATE sqlite_schema SET sql='SELECT * FROM t1' WHERE rowid=6; INSERT INTO t1(t1,rank) VALUES('secure-delete',1); } do_catchsql_test 82.3 { CREATE VIRTUAL TABLE IF NOT EXISTS t USING rtree(x,c); } {1 {Too few columns for an rtree table}} do_catchsql_test 82.4 { BEGIN; REPLACE INTO t1(rowid,b,a,rowid) VALUES(x'44023b9eb002d28b0ee90c',1,2,3); SAVEPOINT b; } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test |
Changes to ext/fts5/test/fts5corrupt4.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt4 | | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt4 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 do_execsql_test 1.0 { |
︙ | ︙ |
Changes to ext/fts5/test/fts5corrupt5.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file tests that FTS5 handles corrupt databases (i.e. internal # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] | | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # # This file tests that FTS5 handles corrupt databases (i.e. internal # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt5 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 database_may_be_corrupt |
︙ | ︙ | |||
233 234 235 236 237 238 239 | | 4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00 ................ | end crash-0f47112aa7520c.db }] } {} do_catchsql_test 1.1 { SELECT * FROM t1('R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC; | | | 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 | | 4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00 ................ | end crash-0f47112aa7520c.db }] } {} do_catchsql_test 1.1 { SELECT * FROM t1('R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- # reset_db do_test 2.0 { sqlite3 db {} db deserialize [decode_hexdb { |
︙ | ︙ | |||
446 447 448 449 450 451 452 | | 0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00 ................ | end sql047467.txt.db }]} {} do_catchsql_test 2.1 { SELECT * FROM t1('R*R*R*R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC; | | | 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 | | 0: 0d 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 03 03 02 01 03 03 02 02 01 00 00 00 00 00 ................ | end sql047467.txt.db }]} {} do_catchsql_test 2.1 { SELECT * FROM t1('R*R*R*R*') WHERE (a,b)<=(current_date,0) ORDER BY rowid DESC; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 3.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb |
︙ | ︙ | |||
565 566 567 568 569 570 571 | | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-c69fcaceff1e50.db }]} {} do_catchsql_test 3.1 { UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thra*T'; | | | 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 | | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-c69fcaceff1e50.db }]} {} do_catchsql_test 3.1 { UPDATE t1 SET b=quote(zeroblob(200)) WHERE a MATCH 'thra*T'; } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 4.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb |
︙ | ︙ | |||
788 789 790 791 792 793 794 795 796 797 798 | } {1 {database disk image is malformed}} do_catchsql_test 4.4 { REPLACE INTO t1(rowid,b,a,rowid) VALUES(0,1,2,3); } {1 {database disk image is malformed}} do_catchsql_test 4.5 { REPLACE INTO t1(rowid,a,b,rowid) VALUES(200,1,2,3); } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 1143 1144 1145 1146 1147 1148 1149 1150 1151 1152 1153 1154 1155 1156 1157 1158 1159 1160 1161 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 1215 1216 1217 1218 1219 1220 1221 1222 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 1277 1278 1279 1280 1281 1282 1283 1284 1285 1286 1287 1288 1289 1290 1291 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 1307 1308 1309 1310 1311 1312 1313 1314 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 1335 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 1352 1353 1354 1355 1356 1357 1358 1359 1360 1361 1362 1363 1364 1365 1366 1367 1368 1369 1370 1371 1372 1373 1374 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 1390 1391 1392 1393 1394 1395 1396 1397 1398 1399 1400 1401 1402 1403 1404 1405 1406 1407 1408 1409 1410 1411 1412 1413 1414 1415 1416 1417 1418 1419 1420 1421 1422 1423 1424 1425 1426 1427 1428 1429 1430 1431 1432 1433 1434 1435 1436 1437 1438 1439 1440 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 1456 1457 1458 1459 1460 1461 1462 1463 1464 1465 1466 1467 1468 1469 1470 1471 1472 1473 1474 1475 1476 1477 1478 1479 1480 1481 1482 1483 1484 1485 1486 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 1538 1539 1540 1541 1542 1543 1544 1545 1546 1547 1548 1549 1550 1551 1552 1553 1554 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 1587 1588 1589 1590 1591 1592 1593 1594 1595 1596 1597 1598 1599 1600 1601 1602 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 1634 1635 1636 1637 1638 1639 1640 1641 1642 1643 1644 1645 1646 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 1664 1665 1666 1667 1668 1669 1670 1671 1672 1673 1674 1675 1676 1677 1678 1679 1680 1681 1682 1683 1684 1685 1686 1687 1688 1689 1690 1691 1692 1693 1694 1695 1696 1697 1698 1699 1700 1701 1702 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 1723 1724 1725 1726 1727 1728 1729 1730 1731 1732 1733 1734 1735 1736 1737 1738 1739 1740 1741 1742 1743 1744 1745 1746 1747 1748 1749 1750 1751 1752 1753 1754 1755 1756 1757 1758 1759 1760 1761 1762 1763 1764 1765 1766 1767 1768 1769 1770 1771 1772 1773 1774 1775 1776 1777 1778 1779 1780 1781 1782 1783 1784 1785 1786 1787 1788 1789 1790 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 1806 1807 1808 1809 1810 1811 1812 1813 1814 1815 1816 1817 1818 1819 1820 1821 1822 1823 1824 1825 1826 1827 1828 1829 1830 1831 1832 1833 1834 1835 1836 1837 1838 1839 1840 1841 1842 1843 1844 1845 1846 1847 1848 1849 1850 1851 1852 1853 1854 1855 1856 1857 1858 1859 1860 1861 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 1894 1895 1896 1897 1898 1899 1900 1901 1902 1903 1904 1905 1906 1907 1908 1909 1910 1911 1912 1913 1914 1915 1916 1917 1918 1919 1920 1921 1922 1923 | } {1 {database disk image is malformed}} do_catchsql_test 4.4 { REPLACE INTO t1(rowid,b,a,rowid) VALUES(0,1,2,3); } {1 {database disk image is malformed}} do_catchsql_test 4.5 { REPLACE INTO t1(rowid,a,b,rowid) VALUES(200,1,2,3); } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 5.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 28672 pagesize 4096 filename crash-0c6d3451d11597.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07 .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 07 00 00 00 04 ................ | 96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d ...............m | 112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00 .....N.......... | 3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05 WITHOUT ROWID[. | 3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21 Y, sz BLOB)U...! | 3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65 !.wtablet1_conte | 3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45 ntt1_content.CRE | 3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f ATE TABLE 't1_co | 3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45 ntent'(id INTEGE | 3824: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 63 R PRIMARY KEY, c | 3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 0)i.......-table | 3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 t1_idxt1_idx.CRE | 3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 ATE TABLE 't1_id | 3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 x'(segid, term, | 3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 pgno, PRIMARY KE | 3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 Y(segid, term)) | 3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 WITHOUT ROWIDU.. | 3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 ......tablet1_da | 3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 tat1_data.CREATE | 3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 TABLE 't1_data' | 4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d (id INTEGER PRIM | 4016: 41 52 b9 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 AR. KEY, block B | 4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c LOB):......ctabl | 4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54 et1t1CREATE VIRT | 4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49 UAL TABLE t1 USI | 4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29 NG fts5(content) | page 2 offset 4096 | 0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd 00 00 ................ | 16: 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80 .............$.. | 4032: 80 80 80 01 03 00 4e 00 00 00 1e 06 30 61 62 61 ......N.....0aba | 4048: 63 6b 01 02 02 04 02 66 74 02 02 02 04 04 6e 64 ck.....ft.....nd | 4064: 6f 6e 03 02 02 04 0a 07 05 01 03 00 10 03 03 0f on.............. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 11 ...$............ | page 3 offset 8192 | 0: 0a 00 00 00 01 0f 00 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 4 offset 12288 | 0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00 ................ | 4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00 .....abandon.... | 4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b .abaft.....aback | page 5 offset 16384 | 0: 0d 00 00 00 03 0f ee 00 0f fa 0f f4 0f ee 00 00 ................ | 16: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03 ................ | 4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01 ................ | page 6 offset 20480 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 7 offset 24576 | 0: 0d 00 00 10 03 0f d6 00 0f f4 10 e1 0f d6 00 00 ................ | 16: 00 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c ..........rebuil | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 00 00 00 heck....optim... | end crash-0c6d3451d11597.db }]} {} do_execsql_test 5.1 { INSERT INTO t1(t1,rank) VALUES('secure-delete',1); } do_catchsql_test 5.4 { UPDATE t1 SET content=randomblob(500); } {/.*fts5: corrupt.*/} #------------------------------------------------------------------------- reset_db do_test 6.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 32768 pagesize 4096 filename crash-42fa37b694d45a.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07 .....@ ........ | 96: 00 00 00 00 0d 00 00 00 07 0d d2 00 0f c4 0f 6d ...............m | 112: 0f 02 0e ab 0e 4e 0d f6 0d d2 00 00 00 00 00 00 .....N.......... | 3536: 00 00 22 07 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3552: 32 74 32 07 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3568: 20 74 32 28 78 29 56 06 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3584: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3600: 6f 6e 66 69 67 06 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3616: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3632: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3648: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 05 WITHOUT ROWID[. | 3664: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3680: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3696: 05 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3712: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3728: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3744: 59 2c 20 73 7a 20 42 4c 4f 42 29 55 04 06 17 21 Y, sz BLOB)U...! | 3760: 21 01 77 74 61 62 6c 65 74 31 5f 63 6f 6e 74 65 !.wtablet1_conte | 3776: 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 04 43 52 45 ntt1_content.CRE | 3792: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 6f ATE TABLE 't1_co | 3808: 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 45 ntent'(id INTEGE | 3824: 52 20 50 52 49 4d 41 52 49 20 4b 45 59 2c 20 63 R PRIMARI KEY, c | 3840: 30 29 69 03 07 17 19 19 01 81 2d 74 61 62 6c 65 0)i.......-table | 3856: 74 31 5f 69 64 78 74 31 5f 69 64 78 03 43 52 45 t1_idxt1_idx.CRE | 3872: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 69 64 ATE TABLE 't1_id | 3888: 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d 2c 20 x'(segid, term, | 3904: 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 4b 45 pgno, PRIMARY KE | 3920: 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 29 20 Y(segid, term)) | 3936: 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 02 07 WITHOUT ROWIDU.. | 3952: 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 5f 64 61 ......tablet1_da | 3968: 74 61 74 31 5f 64 61 74 61 02 43 52 45 41 54 45 tat1_data.CREATE | 3984: 20 54 41 42 4c 45 20 27 74 31 5f 64 61 74 61 27 TABLE 't1_data' | 4000: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d (id INTEGER PRIM | 4016: 41 52 b9 20 4b 45 59 2c 20 62 6c 6f 63 6b 20 42 AR. KEY, block B | 4032: 4c 4f 42 29 3a 01 06 17 11 11 08 63 74 61 62 6c LOB):......ctabl | 4048: 65 74 31 74 31 43 52 45 41 54 45 20 56 49 52 54 et1t1CREATE VIRT | 4064: 55 41 4c 20 54 41 42 4c 45 20 74 31 20 55 53 49 UAL TABLE t1 USI | 4080: 4e 47 20 66 74 73 35 28 63 6f 6e 74 65 6e 74 29 NG fts5(content) | page 2 offset 4096 | 0: 0d 00 00 00 03 0f bd 00 0f e8 0f ef 0f bd f0 00 ................ | 16: 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 24 84 80 .............$.. | 4032: 80 80 80 01 03 00 4e 00 10 00 1e 06 30 61 62 61 ......N.....0aba | 4048: 63 6c 01 02 02 04 02 66 74 02 5f 02 04 04 6e 64 cl.....ft._...nd | 4064: 6f 6e 02 02 02 04 0a 07 05 01 03 00 10 03 03 0f on.............. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 11 ...$............ | page 3 offset 8192 | 0: 0a 00 00 00 01 0f 00 01 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 4 offset 12288 | 0: 0d 00 00 00 03 0f e0 00 0f f6 0f ec 0f e0 00 00 ................ | 4064: 0a 03 03 00 1b 61 62 61 6e 64 6f 6e 08 02 03 00 .....abandon.... | 4080: 17 61 62 61 66 74 08 01 03 00 17 61 62 61 63 6b .abaft.....aback | page 5 offset 16384 | 0: 0d 00 00 00 03 0f ee 00 0f fa 0f 00 00 00 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 04 03 ................ | 4080: 03 00 0e 01 04 02 03 00 0e 01 04 01 03 00 0e 01 ................ | page 6 offset 20480 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 7 offset 24576 | 0: 0d 00 00 10 03 0f d6 00 0f 00 00 00 00 00 00 00 ................ | 4048: 00 00 00 00 00 00 09 03 02 1b 72 65 62 75 69 6c ..........rebuil | 4064: 64 11 02 02 2b 69 6e 74 65 67 72 69 74 79 2d 63 d...+integrity-c | 4080: 68 65 63 6b 0a 01 02 1d 6f 70 74 69 6d 00 00 00 heck....optim... | page 8 offset 28672 | 0: 00 00 01 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | end crash-42fa37b694d45a.db }]} {} do_execsql_test 6.1 { INSERT INTO t1(t1,rank) VALUES('secure-delete',1); } do_catchsql_test 6.2 { UPDATE t1 SET content=randomblob(500) WHERE t1; } {1 {constraint failed}} #------------------------------------------------------------------------- reset_db do_test 7.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 40960 pagesize 4096 filename crash-d8b4a99207c10b.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 0a .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 0d 00 00 00 04 ................ | 48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ | 96: 00 00 00 00 0d 00 00 00 0d 0b 62 00 0f 97 0f 40 ..........b....@ | 112: 0e d5 0e 75 0e 18 0d c0 0d 66 0d 0f 0c a4 0c 44 ...u.....f.....D | 128: 0b ec 0b a7 0b 62 00 00 00 00 00 00 00 00 00 00 .....b.......... | 2912: 00 00 43 0d 06 17 11 11 08 75 74 61 62 6c 65 74 ..C......utablet | 2928: 34 74 34 43 52 45 41 54 45 20 56 49 52 54 55 41 4t4CREATE VIRTUA | 2944: 4c 20 54 41 42 4c 45 20 74 34 20 55 53 49 4e 47 L TABLE t4 USING | 2960: 20 66 74 73 35 76 6f 63 61 62 28 27 74 32 27 2c fts5vocab('t2', | 2976: 20 27 72 6f 77 27 29 43 0c 06 17 11 11 08 75 74 'row')C......ut | 2992: 61 62 6c 65 74 33 74 33 43 52 45 41 54 45 20 56 ablet3t3CREATE V | 3008: 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 33 20 IRTUAL TABLE t3 | 3024: 55 53 49 4e 47 20 66 74 73 35 76 6f 63 61 62 28 USING fts5vocab( | 3040: 27 74 31 27 2c 20 27 72 6f 77 27 29 56 0b 06 17 't1', 'row')V... | 3056: 1f 1f 01 7d 74 61 62 6c 65 74 32 5f 63 6f 6e 66 ....tablet2_conf | 3072: 69 67 74 32 5f 63 6f 6e 66 69 67 0a 43 52 45 41 igt2_config.CREA | 3088: 54 45 20 54 41 42 4c 45 20 27 74 32 5f 63 6f 6e TE TABLE 't2_con | 3104: 66 69 67 27 28 6b 20 50 52 49 4d 41 52 59 20 4b fig'(k PRIMARY K | 3120: 45 59 2c 20 76 29 20 57 49 54 48 4f 55 54 20 52 EY, v) WITHOUT R | 3136: 4f 57 49 44 5e 0a 07 17 21 21 01 81 07 74 61 62 OWID^...!!...tab | 3152: 6c 65 74 32 5f 63 6f 6e 74 65 6e 74 74 32 5f 63 let2_contentt2_c | 3168: 6f 6e 74 65 6e 74 09 43 52 45 41 54 45 20 54 41 ontent.CREATE TA | 3184: 42 4c 45 20 27 74 32 5f 63 6f 6e 74 65 6e 74 27 BLE 't2_content' | 3200: 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 4d (id INTEGER PRIM | 3216: 41 52 59 20 4b 45 59 2c 20 63 30 2c 20 63 31 2c ARY KEY, c0, c1, | 3232: 20 63 32 29 69 09 07 17 19 19 01 81 2d 74 61 62 c2)i.......-tab | 3248: 6c 65 74 32 5f 69 64 78 74 32 5f 69 64 78 08 43 let2_idxt2_idx.C | 3264: 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 32 5f REATE TABLE 't2_ | 3280: 69 64 78 27 28 73 65 67 69 64 2c 20 74 65 72 6d idx'(segid, term | 3296: 2c 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 20 , pgno, PRIMARY | 3312: 4b 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d 29 KEY(segid, term) | 3328: 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 55 ) WITHOUT ROWIDU | 3344: 08 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 32 5f ........tablet2_ | 3360: 64 61 74 61 74 32 5f 64 61 74 61 07 43 52 45 41 datat2_data.CREA | 3376: 54 45 20 54 41 42 4c 45 20 27 74 32 5f 64 61 74 TE TABLE 't2_dat | 3392: 61 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 a'(id INTEGER PR | 3408: 49 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 6b IMARY KEY, block | 3424: 20 42 4c 4f 42 29 58 07 07 17 11 11 08 81 1d 74 BLOB)X........t | 3440: 61 62 6c 65 74 32 74 32 43 52 45 41 54 45 20 56 ablet2t2CREATE V | 3456: 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 32 20 IRTUAL TABLE t2 | 3472: 55 53 49 4e 47 20 66 74 73 35 28 27 61 27 2c 5b USING fts5('a',[ | 3488: 62 5d 2c 22 63 22 2c 64 65 74 61 69 6c 3d 6e 6f b],.c.,detail=no | 3504: 6e 65 2c 63 6f 6c 75 6d 6e 73 69 7a 65 3d 30 29 ne,columnsize=0) | 3520: 56 06 06 17 1f 1f 01 7d 74 61 62 6c 65 74 31 5f V.......tablet1_ | 3536: 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 69 67 06 configt1_config. | 3552: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3568: 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 49 4d 41 _config'(k PRIMA | 3584: 52 59 20 4b 45 59 2c 20 76 29 20 57 49 54 48 4f RY KEY, v) WITHO | 3600: 55 54 20 52 4f 57 49 44 5b 05 07 17 21 21 01 81 UT ROWID[...!!.. | 3616: 01 74 61 62 6c 65 74 31 5f 64 6f 63 73 69 7a 65 .tablet1_docsize | 3632: 74 31 5f 64 6f 63 73 69 7a 65 05 43 52 45 41 54 t1_docsize.CREAT | 3648: 45 20 54 41 42 4c 45 20 27 74 31 5f 64 6f 63 73 E TABLE 't1_docs | 3664: 69 7a 65 27 28 69 64 20 49 4e 54 45 47 45 52 20 ize'(id INTEGER | 3680: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 73 7a 20 PRIMARY KEY, sz | 3696: 42 4c 4f 42 29 5e 04 07 17 21 21 01 81 07 74 61 BLOB)^...!!...ta | 3712: 62 6c 65 74 31 5f 63 6f 6e 74 65 6e 74 74 31 5f blet1_contentt1_ | 3728: 63 6f 6e 74 65 6e 74 04 43 52 45 41 54 45 20 54 content.CREATE T | 3744: 41 42 4c 45 20 27 74 31 5f 63 6f 6e 74 65 6e 74 ABLE 't1_content | 3760: 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 52 49 '(id INTEGER PRI | 3776: 4d 41 52 59 20 4b 45 59 2c 20 63 30 2c 20 63 31 MARY KEY, c0, c1 | 3792: 2c 20 63 32 29 69 03 07 17 19 19 01 81 2d 74 61 , c2)i.......-ta | 3808: 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 64 78 03 blet1_idxt1_idx. | 3824: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3840: 5f 69 64 78 27 28 73 65 67 69 64 2c 20 74 65 72 _idx'(segid, ter | 3856: 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 m, pgno, PRIMARY | 3872: 20 4b 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d KEY(segid, term | 3888: 29 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 )) WITHOUT ROWID | 3904: 55 02 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 U........tablet1 | 3920: 5f 64 61 74 61 74 31 5f 64 61 74 61 02 43 52 45 _datat1_data.CRE | 3936: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 61 ATE TABLE 't1_da | 3952: 74 61 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 ta'(id INTEGER P | 3968: 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 RIMARY KEY, bloc | 3984: 6b 20 42 4c 4f 42 29 67 01 07 17 11 11 08 81 3b k BLOB)g.......; | 4000: 74 61 62 6c 65 74 31 74 31 43 52 45 41 54 45 20 tablet1t1CREATE | 4016: 56 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 31 VIRTUAL TABLE t1 | 4032: 20 55 53 49 4e 47 20 66 74 73 35 28 61 2c 62 20 USING fts5(a,b | 4048: 75 6e 69 6e 64 65 78 65 64 2c 63 2c 74 6f 6b 65 unindexed,c,toke | 4064: 6e 69 7a 65 3d 22 70 6f 72 74 65 72 20 61 73 63 nize=.porter asc | 4080: 69 69 22 2c 74 6f 6b 65 6e 64 61 74 61 3d 31 29 ii.,tokendata=1) | page 2 offset 4096 | 0: 0d 0f 68 00 05 0f 13 00 0f e6 0f 13 0f a8 0f 7c ..h............| | 16: 0f 2a 00 00 00 00 00 00 00 00 00 00 00 00 00 00 .*.............. | 3856: 00 00 00 15 0a 03 00 30 00 00 00 00 01 03 03 00 .......0........ | 3872: 03 01 01 01 02 01 01 03 01 01 37 8c 80 80 80 80 ..........7..... | 3888: 01 03 00 74 00 00 00 2e 02 30 61 03 02 02 01 01 ...t.....0a..... | 3904: 62 03 02 03 01 01 63 03 02 04 01 01 67 03 06 01 b.....c.....g... | 3920: 02 02 01 01 68 03 06 01 02 03 01 01 69 03 06 01 ....h.......i... | 3936: 02 04 04 06 06 06 08 08 0f ef 00 14 2a 00 00 00 ............*... | 3952: 00 01 02 02 00 02 01 01 01 02 01 01 25 88 80 80 ............%... | 3968: 80 80 01 03 00 50 00 00 00 1f 02 30 67 02 08 02 .....P.....0g... | 3984: 01 02 02 01 01 68 02 08 03 01 02 03 01 01 69 02 .....h........i. | 4000: 08 04 01 02 04 04 09 09 37 84 80 80 80 7f f1 03 ........7....... | 4016: 00 74 00 00 00 2e 02 30 61 01 02 02 01 01 62 01 .t.....0a.....b. | 4032: 02 03 01 01 63 01 02 04 01 01 67 01 06 01 02 02 ....c.....g..... | 4048: 01 01 68 01 06 01 02 03 01 01 69 01 06 01 02 04 ..h.......i..... | 4064: 04 06 06 06 08 08 07 01 03 00 14 03 09 00 09 00 ................ | 4080: 00 00 11 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 3 offset 8192 | 0: 0a 00 00 00 03 0f ec 00 0f fa 0f f3 0f ec 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 00 00 00 00 06 04 01 0c ................ | 4080: 01 03 02 06 04 01 0c 01 02 02 05 04 09 0c 01 02 ................ | page 4 offset 12288 | 0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03 ................ | 4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67 .....a b cg h ig | 4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69 h i.......g h i | 4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17 a b cg h i...... | 4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69 .a b cd e fg h i | page 5 offset 16384 | 0: 0d 00 00 00 03 0f e8 00 0f f8 0f f0 0f e8 00 00 ................ | 4064: 00 00 00 00 00 00 00 00 06 03 03 00 12 03 00 03 ................ | 4080: 06 02 03 00 12 03 00 03 06 01 03 00 12 03 00 03 ................ | page 6 offset 20480 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 7 offset 24576 | 0: 0d 00 00 00 03 0f 9e 00 0f e6 0f ef 0f 9e 00 00 ................ | 3984: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 41 84 ..............A. | 4000: 80 80 80 80 01 04 00 81 06 00 00 00 34 02 30 61 ............4.0a | 4016: 01 01 01 01 01 62 01 01 01 01 01 63 01 01 01 01 .....b.....c.... | 4032: 01 64 01 01 01 65 01 01 01 66 01 01 01 67 01 01 .d...e...f...g.. | 4048: 01 01 01 68 01 01 01 01 01 69 01 01 01 04 06 06 ...h.....i...... | 4064: 06 04 04 04 06 06 07 01 03 00 14 03 09 09 09 0f ................ | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 8 offset 28672 | 0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 9 offset 32768 | 0: 0d 00 00 00 03 0f be 00 0f ea 0f d4 0f be 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 14 03 ................ | 4032: 05 00 17 17 17 61 20 62 20 63 67 20 68 20 69 67 .....a b cg h ig | 4048: 20 68 20 69 14 02 05 00 17 17 17 67 20 68 20 69 h i.......g h i | 4064: 61 20 62 20 63 67 20 68 20 69 14 01 05 00 17 17 a b cg h i...... | 4080: 17 61 20 62 20 63 64 20 65 20 66 67 20 68 20 69 .a b cd e fg h i | page 10 offset 36864 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-d8b4a99207c10b.db }]} {} do_catchsql_test 7.1 { SELECT snippet(t1, -1, '.', '..', '[', ']'), highlight(t1, 2, '[', ']') FROM t1('g + h') WHERE rank MATCH 'bm25(1.0, 1.0)' ORDER BY rank; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 8.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 20480 pagesize 4096 filename crash-d57c01958e48ab.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 05 .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 05 00 00 00 04 ................ | 48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ | 96: 00 00 00 00 0d 00 00 00 05 0e 10 00 0f 97 0f 40 ...............@ | 112: 0e d5 0e 68 0e 10 01 00 00 00 00 00 00 00 00 00 ...h............ | 3600: 56 05 06 17 1f 1f 01 7d 74 61 62 6c 65 74 31 5f V.......tablet1_ | 3616: 63 6f 6e 66 69 67 74 31 5f 63 6f 6e 66 69 67 05 configt1_config. | 3632: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3648: 5f 63 6f 6e 66 69 67 27 28 6b 20 50 52 49 4d 41 _config'(k PRIMA | 3664: 52 59 20 4b 45 59 2c 20 76 29 20 57 49 54 48 4f RY KEY, v) WITHO | 3680: 55 54 20 52 4f 57 49 44 6b 04 07 17 21 21 01 81 UT ROWIDk...!!.. | 3696: 21 74 61 62 6c 65 74 31 5f 64 6f 63 73 69 7a 65 !tablet1_docsize | 3712: 74 31 5f 64 6f 63 73 69 7a 65 04 43 52 45 41 54 t1_docsize.CREAT | 3728: 45 20 54 41 42 4c 45 20 27 74 31 5f 64 6f 63 73 E TABLE 't1_docs | 3744: 69 7a 65 27 28 69 64 20 49 4e 54 45 47 45 52 20 ize'(id INTEGER | 3760: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 73 7a 20 PRIMARY KEY, sz | 3776: 42 4c 4f 42 2c 20 6f 72 69 67 69 6e 20 49 4e 54 BLOB, origin INT | 3792: 45 47 45 52 29 69 03 07 17 19 19 01 81 2d 74 61 EGER)i.......-ta | 3808: 62 6c 65 74 31 5f 69 64 78 74 31 5f 69 64 78 03 blet1_idxt1_idx. | 3824: 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 CREATE TABLE 't1 | 3840: 5f 69 64 78 27 28 73 65 67 69 64 2c 20 74 65 72 _idx'(segid, ter | 3856: 6d 2c 20 70 67 6e 6f 2c 20 50 52 49 4d 41 52 59 m, pgno, PRIMARY | 3872: 20 4b 45 59 28 73 65 67 69 64 2c 20 74 65 72 6d KEY(segid, term | 3888: 29 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 )) WITHOUT ROWID | 3904: 55 02 07 17 1b 1b 01 81 01 74 61 62 6c 65 74 31 U........tablet1 | 3920: 5f 64 61 74 61 74 31 5f 64 61 74 61 02 43 52 45 _datat1_data.CRE | 3936: 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 64 61 ATE TABLE 't1_da | 3952: 74 61 27 28 69 64 20 49 4e 54 45 47 45 52 20 50 ta'(id INTEGER P | 3968: 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 6c 6f 63 RIMARY KEY, bloc | 3984: 6b 20 42 4c 4f 42 29 67 01 07 17 11 11 08 81 3b k BLOB)g.......; | 4000: 74 61 62 6c 65 74 31 74 31 43 52 45 41 54 45 20 tablet1t1CREATE | 4016: 56 49 52 54 55 41 4c 20 54 41 42 4c 45 20 74 31 VIRTUAL TABLE t1 | 4032: 20 55 53 49 4e 47 20 66 74 73 35 28 61 2c 20 62 USING fts5(a, b | 4048: 2c 20 63 6f 6e 74 65 6e 74 3d 27 27 2c 20 63 6f , content='', co | 4064: 6e 74 65 6e 74 6c 65 73 73 5f 64 65 6c 65 74 65 ntentless_delete | 4080: 3d 31 2c 20 74 6f 6b 65 6e 64 61 74 61 3d 31 29 =1, tokendata=1) | page 2 offset 4096 | 0: 0d 0f eb 00 03 0e 17 00 0f e2 0e 17 0e 31 00 00 .............1.. | 16: 01 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 3600: 00 00 00 00 00 00 00 18 0a 03 00 36 00 00 00 00 ...........6.... | 3616: ff 00 00 01 01 01 01 00 01 01 01 01 01 01 00 00 ................ | 3632: 07 83 29 84 80 80 80 80 01 04 00 86 56 00 00 01 ..).........V... | 3648: 96 04 30 61 61 61 01 02 02 01 04 02 04 01 08 02 ..0aaa.......... | 3664: 04 04 04 01 10 02 04 04 04 04 04 04 04 01 20 02 .............. . | 3680: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 01 ................ | 3696: 40 02 04 04 04 04 04 04 04 04 04 04 04 04 04 04 @............... | 3712: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 ................ | 3728: 04 01 81 00 02 04 04 04 04 04 04 04 04 04 04 04 ................ | 3744: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 ................ | 3760: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 ................ | 3776: 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 04 ................ | 3792: 04 04 04 04 02 02 62 63 01 06 01 01 02 01 03 62 ......bc.......b | 3808: 62 62 02 02 03 01 04 03 06 01 08 03 06 06 06 01 bb.............. | 3824: 10 03 06 06 06 06 06 06 06 01 20 03 06 06 06 06 .......... ..... | 3840: 06 06 06 06 06 06 06 06 06 06 06 01 40 03 06 06 ............@... | 3856: 06 06 06 06 06 06 06 06 06 06 06 06 06 06 06 06 ................ | 3872: 06 06 06 06 06 06 06 06 06 06 16 06 06 02 02 63 ...............c | 3888: 64 02 06 01 01 02 01 03 63 63 63 03 02 05 01 04 d.......ccc..... | 3904: 05 0a 01 08 05 0a 0a 0a 01 10 05 0a 0a 0a 0a 0a ................ | 3920: 0a 0a 01 20 05 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a 0a ... ............ | 3936: 0a 0a 0a 0a 02 02 64 65 03 06 01 01 02 01 03 64 ......de.......d | 3952: 64 64 04 02 09 01 04 09 12 01 08 09 12 12 12 01 dd.............. | 3968: 10 09 12 12 12 12 12 12 12 02 02 65 66 04 06 01 ...........ef... | 3984: 01 02 01 03 65 65 65 05 02 11 01 04 11 22 01 08 ....eee......... | 4000: 11 22 22 22 02 02 66 67 05 06 01 01 02 01 03 66 ......fg.......f | 4016: 56 66 06 02 21 01 04 21 42 02 02 67 68 06 06 01 Vf..!..!B..gh... | 4032: 01 02 cb 03 67 67 67 07 02 41 02 02 68 69 07 06 ....ggg..A..hi.. | 4048: 01 01 02 04 81 13 09 50 09 2e 09 1c 09 12 09 0c .......P........ | 4064: 09 08 07 01 03 00 14 07 81 77 07 00 00 00 15 22 .........w...... | 4080: 00 00 00 00 ff 00 00 01 00 00 00 00 00 00 05 0c ................ | page 3 offset 8192 | 0: 0a 00 00 00 01 0f fa 00 0f fa 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 4 offset 12288 | 0: 0d 00 00 00 07 0f c8 00 0f f8 0f f0 0f e8 0f e0 ................ | 16: 0f d8 0f d0 0f c8 00 00 00 00 00 00 00 00 00 00 ................ | 4032: 00 00 00 00 00 00 00 00 06 07 04 00 10 09 7f 01 ................ | 4048: 06 06 04 00 10 09 3f 01 06 05 04 00 10 09 1f 01 ......?......... | 4064: 06 04 04 00 10 09 0f 01 06 03 04 00 10 09 07 01 ................ | 4080: 06 02 04 00 10 09 03 01 06 01 04 00 10 09 01 01 ................ | page 5 offset 16384 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | end crash-d57c01958e48ab.db }]} {} do_catchsql_test 8.1 { SELECT rowid FROM t1('a* NOT ý') ; } {0 {1 2 3 4 5 6 7}} #------------------------------------------------------------------------- reset_db do_test 9.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 32768 pagesize 4096 filename crash-c76a16c24c8ba6.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08 .....@ ........ | 32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04 ................ | 96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36 ...............6 | 112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00 ...k............ | 3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06 WITHOUT ROWID[. | 3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21 Y, sz BLOB)^...! | 3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74 !...tablet1_cont | 3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52 entt1_content.CR | 3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 EATE TABLE 't1_c | 3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 ontent'(id INTEG | 3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19 c0, c1, c2)i.... | 3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 ...-tablet1_idxt | 3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42 1_idx.CREATE TAB | 3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69 LE 't1_idx'(segi | 3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 d, term, pgno, P | 3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64 RIMARY KEY(segid | 3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54 , term)) WITHOUT | 3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74 ROWIDU........t | 3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61 ablet1_datat1_da | 3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 ta.CREATE TABLE | 3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54 't1_data'(id INT | 3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 EGER PRIMARY KEY | 3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06 , block BLOB)8.. | 3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52 ...._tablet1t1CR | 4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 EATE VIRTUAL TAB | 4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 LE t1 USING fts5 | 4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00 (a,b,c)......... | page 3 offset 8192 | 0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00 ................ | 3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18 .....J.......... | 3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06 ...+.00......... | 3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30 ...........20160 | 3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01 609...........4. | 3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02 ..........5..... | 3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04 ......0000000... | 3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06 ........binary.. | 3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70 ............comp | 3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 iler...........d | 3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 bstat........... | 3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 ebug...........e | 3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 nable........... | 3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 ................ | 3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 ................ | 3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 ................ | 3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 ......xtension.. | 3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03 .........fts4... | 3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01 ........5....... | 3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03 ....gcc......... | 3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02 ..eopoly........ | 3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02 ...json1........ | 3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03 ...load......... | 3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05 ..max........... | 3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04 emory........... | 3648: 73 79 73 35 16 02 03 01 02 03 01 02 03 01 06 6e sys5...........n | 3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03 ocase........... | 3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 ................ | 3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02 ...omit......... | 3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03 ..rtree......... | 3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06 ..im............ | 3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56 ..threadsafe.W4V | 3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02 .d.F...vtab..... | 3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01 ......x......... | 3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02 ................ | 3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 3936: 01 02 01 06 01 01 10 01 06 01 01 02 01 06 01 01 ................ | 3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 ................ | 3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 ................ | 4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c ................ | 4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f .D...G.......... | 4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f D..@.......$Z$$. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 4 offset 12288 | 0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 5 offset 16384 | 0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74 ....$..........t | 16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5 .a.N./.......... | 32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5 ...t.[.@.$...... | 48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb .......h.O.5.... | 64: 0c da 0c b9 0c 99 0c 78 0c 57 0c 3e 0c 24 0c 0a .......x.W.>.$.. | 3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f ...........$..%. | 3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49 .THREADSAFE=0XBI | 3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41 NARY.#..%..THREA | 3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22 DSAFE=0XNOCASE.. | 3136: 05 00 25 0f 17 54 48 52 45 41 44 53 31 46 45 3d ..%..THREADS1FE= | 3152: 30 58 52 64 52 49 4d 1f 21 05 00 33 0f 19 4f 4d 0XRdRIM.!..3..OM | 3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f IT LOAD EXTENSIO | 3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f NXBINARY. ..3..O | 3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 MIT LOAD EXTENSI | 3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17 ONXNOCASE....3.. | 3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 OMIT LOAD EXTENS | 3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19 IONXRTRIM....3.. | 3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30 MAX MEMORY=50000 | 3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f 000XBINARY....3. | 3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 .MAX MEMORY=5000 | 3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33 0000XNOCASE....3 | 3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 ..MAX MEMORY=500 | 3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25 00000XRTRIM....% | 3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42 ..ENABLE RTREEXB | 3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17 LE RTREEXNOCASE. | 3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52 ...%..ENABLE RTR | 3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45 EEXRTRIM....)..E | 3440: 4e 41 42 4b 45 20 4d 45 4d 53 59 53 35 58 42 49 NABKE MEMSYS5XBI | 3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c NARY....)..ENABL | 3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45 B`-EMSYS5XNOCASE | 3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 45 ....)..ENABLE ME | 3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25 MSYS5XRTRIM....% | 3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42 ..ENABLE JSON1XB | 3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17 LE JSON1XNOCASE. | 3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f ...%..ENABLE JSO | 3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45 N1XRTRIM....)..E | 3600: 4e 41 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49 NABLE GEOPOLYXBI | 3616: 4e 41 52 59 1a 11 05 00 39 0f 19 45 4e 41 42 4c NARY....9..ENABL | 3632: 45 20 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 45 E GEOPOLYXNOCASE | 3648: 19 10 05 00 29 0f 17 45 4e 41 42 4c 45 20 47 45 ....)..ENABLE GE | 3664: 4f 50 4f 4c 59 58 52 54 52 49 4d 17 0f 05 00 23 OPOLYXRTRIM....# | 3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49 ..ENABLE FTS5XBI | 3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c NARY....#..ENABL | 3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05 E FTS5XNOCASE... | 3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58 .#..ENABLE FTS5X | 3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42 RTRIM....#..ENAB | 3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b LE FTS4XBINARY.. | 3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34 ..#..ENABLE FTS4 | 3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e XNOCASE....#..EN | 3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e ABLE FTS4XRTRIM. | 3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e TAT VTABXBINARY. | 3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d TAT VT$.HNOCASE. | 3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06 TAT VTABXRTRIM.. | 3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52 .....DEBUGXBINAR | 3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f Y.......DEBUGXNO | 3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47 CASE.......DEBUG | 3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d XRTRIM'...C..COM | 3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20 PILER=gcc-5.4.0 | 4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27 20160609XBINARY' | 4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3c 67 ...C..COMPILER<g | 4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30 cc-5.4.0 2016060 | 4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43 9XNOCASE&...C..C | 4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e OMPILER=gcc-5.4. | 4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d 0 20160609XRTRIM | page 6 offset 20480 | 0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0 ....$........... | 16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0 ................ | 32: 1f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60 .........x.p.h.` | 48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20 .X.P.H.@.8.0.(. | 64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 0e e0 ................ | 3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01 .$.......#...... | 3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01 .........!...... | 3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01 . .............. | 3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01 ................ | 3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01 ................ | 3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01 ................ | 3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01 ................ | 3920: 06 15 f3 00 12 02 01 01 06 15 03 00 12 02 01 01 ................ | 3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01 ................ | 3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01 ................ | 3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01 ................ | 3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01 ................ | 4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01 ................ | 4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01 ................ | 4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01 ................ | 4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01 ................ | 4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01 ................ | 4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01 ................ | page 7 offset 24576 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 8 offset 28672 | 0: 0d 00 00 00 03 0f d6 00 0f f4 0f e9 0f d6 00 00 ................ | 4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72 .........+integr | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 00 00 00 00 uild....opti.... | end crash-c76a16c24c8ba6.db }]} {} #.testctrl prng_seed 1 db #.testctrl internal_functions #.testctrl json_selfcheck on # do_execsql_test 9.1 { UPDATE t1 SET b=quote(zeroblob(current_date)) WHERE t1 MATCH 't*'; SAVEPOINT a; UPDATE t1 SET b=quote(zeroblob(current_date)) WHERE t1 MATCH 't*'; INSERT INTO t1(t1,rank) VALUES('secure-delete',1); } do_catchsql_test 9.2 { DELETE FROM t1; } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 10.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 32768 pagesize 4096 filename crash-b06f016068bcea.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08 .....@ ........ | 32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04 ................ | 96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36 ...............6 | 112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00 ...k............ | 3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06 WITHOUT ROWID[. | 3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21 Y, sz BLOB)^...! | 3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74 !...tablet1_cont | 3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52 entt1_content.CR | 3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 EATE TABLE 't1_c | 3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 ontent'(id INTEG | 3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19 c0, c1, c2)i.... | 3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 ...-tablet1_idxt | 3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42 1_idx.CREATE TAB | 3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69 LE 't1_idx'(segi | 3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 d, term, pgno, P | 3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64 RIMARY KEY(segid | 3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54 , term)) WITHOUT | 3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74 ROWIDU........t | 3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61 ablet1_datat1_da | 3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 ta.CREATE TABLE | 3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54 't1_data'(id INT | 3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 EGER PRIMARY KEY | 3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06 , block BLOB)8.. | 3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52 ...._tablet1t1CR | 4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 EATE VIRTUAL TAB | 4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 LE t1 USING fts5 | 4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00 (a,b,c)......... | page 3 offset 8192 | 0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00 ................ | 3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18 .....J.......... | 3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06 ...+.00......... | 3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30 ...........20160 | 3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01 609...........4. | 3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02 ..........5..... | 3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04 ......0000000... | 3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06 ........binary.. | 3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70 ............comp | 3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 iler...........d | 3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 bstat........... | 3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 ebug...........e | 3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 nable........... | 3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 ................ | 3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 ................ | 3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 ................ | 3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 ......xtension.. | 3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03 .........fts4... | 3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01 ........5....... | 3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03 ....gcc......... | 3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02 ..eopoly........ | 3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02 ...json1........ | 3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03 ...load......... | 3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05 ..max........... | 3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04 emory........... | 3648: 73 79 73 35 16 02 03 01 02 02 03 01 03 01 06 6e sys5...........n | 3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03 ocase........... | 3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 ................ | 3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02 ...omit......... | 3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03 ..rtree......... | 3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06 ..im............ | 3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56 ..threadsafe.W4V | 3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02 .d.F...vtab..... | 3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01 ......x......... | 3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02 ................ | 3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 3936: 01 02 01 06 01 01 10 01 06 01 01 02 01 06 01 01 ................ | 3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 ................ | 3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 ................ | 4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c ................ | 4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f .D...G.......... | 4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f D..@.......$Z$$. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 4 offset 12288 | 0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 5 offset 16384 | 0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74 ....$..........t | 16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5 .a.N./.......... | 32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5 ...t.[.@.$...... | 48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb .......h.O.5.... | 64: 0c da 0c b9 0c 99 0c 00 00 00 00 00 00 00 00 00 ................ | 3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f ...........$..%. | 3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49 .THREADSAFE=0XBI | 3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41 NARY.#..%..THREA | 3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22 DSAFE=0XNOCASE.. | 3136: 05 00 25 0f 17 54 48 52 45 41 44 53 31 46 45 3d ..%..THREADS1FE= | 3152: 30 58 52 64 52 49 4d 1f 21 05 00 33 0f 19 4f 4d 0XRdRIM.!..3..OM | 3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f IT LOAD EXTENSIO | 3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f NXBINARY. ..3..O | 3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 MIT LOAD EXTENSI | 3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17 ONXNOCASE....3.. | 3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 OMIT LOAD EXTENS | 3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19 IONXRTRIM....3.. | 3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30 MAX MEMORY=50000 | 3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f 000XBINARY....3. | 3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 .MAX MEMORY=5000 | 3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33 0000XNOCASE....3 | 3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 ..MAX MEMORY=500 | 3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25 00000XRTRIM....% | 3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42 ..ENABLE RTREEXB | 3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17 LE RTREEXNOCASE. | 3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52 ...%..ENABLE RTR | 3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45 EEXRTRIM....)..E | 3440: 4e 41 42 4b 45 20 4d 45 4d 53 59 53 35 58 42 49 NABKE MEMSYS5XBI | 3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c NARY....)..ENABL | 3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45 B`-EMSYS5XNOCASE | 3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 46 ....)..ENABLE MF | 3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25 MSYS5XRTRIM....% | 3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42 ..ENABLE JSON1XB | 3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17 LE JSON1XNOCASE. | 3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f ...%..ENABLE JSO | 3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45 N1XRTRIM....)..E | 3600: 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49 4e 41 BLE GEOPOLYXBINA | 3616: 52 59 1a 11 05 00 39 0f 19 45 4e 41 42 4c 45 2e RY....9..ENABLE. | 3632: 41 40 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 40 A@GEOPOLYXNOCAS@ | 3648: 4f 4c 59 58 55 09 10 05 00 29 0f 17 45 4e 41 42 OLYXU....)..ENAB | 3664: 4c 45 20 47 45 4f 52 54 52 49 4d 17 0f 05 00 23 LE GEORTRIM....# | 3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49 ..ENABLE FTS5XBI | 3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c NARY....#..ENABL | 3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05 E FTS5XNOCASE... | 3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58 .#..ENABLE FTS5X | 3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42 RTRIM....#..ENAB | 3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b LE FTS4XBINARY.. | 3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34 ..#..ENABLE FTS4 | 3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e XNOCASE....#..EN | 3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e ABLE FTS4XRTRIM. | 3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e TAT VTABXBINARY. | 3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d TAT VT$.HNOCASE. | 3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06 TAT VTABXRTRIM.. | 3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52 .....DEBUGXBINAR | 3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f Y.......DEBUGXNO | 3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47 CASE.......DEBUG | 3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d XRTRIM'...C..COM | 3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20 PILER=gcc-5.4.0 | 4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27 20160609XBINARY' | 4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67 ...C..COMPILER=g | 4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30 cc-5.4.0 2016060 | 4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43 9XNOCASE&...C..C | 4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e OMPILER=gcc-5.4. | 4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d 0 20160609XRTRIM | page 6 offset 20480 | 0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0 ....$........... | 16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0 ................ | 32: 1f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60 .........x.p.h.` | 48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20 .X.P.H.@.8.0.(. | 64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 00 00 ................ | 3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01 .$.......#...... | 3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01 .........!...... | 3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01 . .............. | 3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01 ................ | 3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01 ................ | 3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01 ................ | 3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01 ................ | 3920: 06 15 f3 00 12 02 01 01 06 15 03 00 12 02 01 01 ................ | 3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01 ................ | 3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01 ................ | 3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01 ................ | 3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01 ................ | 4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01 ................ | 4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01 ................ | 4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01 ................ | 4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01 ................ | 4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01 ................ | 4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01 ................ | page 7 offset 24576 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 8 offset 28672 | 0: 0d 00 00 00 03 0f d6 00 0f f4 0f 00 00 00 00 00 ................ | 4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72 .........+integr | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 70 74 69 00 00 00 00 uild....opti.... | end crash-b06f016068bcea.db .testctrl prng_seed 1 db .testctrl internal_functions .testctrl json_selfcheck on }]} {} do_execsql_test 10.1 { UPDATE t1 SET b=quote(zeroblob(current_date)) WHERE t1 MATCH 't*'; } do_catchsql_test 10.2 { BEGIN; INSERT INTO t1(t1,rank) VALUES('secure-delete',1); REPLACE INTO t1(b,a,rowid) VALUES(1,2,3); } {0 {}} do_catchsql_test 10.3 { COMMIT } {1 {database disk image is malformed}} do_catchsql_test 10.4 { REPLACE INTO t1(b,a,rowid) VALUES(1,2,3); } {1 {database disk image is malformed}} #------------------------------------------------------------------------- reset_db do_test 11.0 { sqlite3 db {} db deserialize [decode_hexdb { .open --hexdb | size 32768 pagesize 4096 filename crash-3d05232c78871b.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 08 .....@ ........ | 32: 00 00 00 02 00 00 00 01 00 00 00 09 00 00 00 04 ................ | 96: 00 00 00 00 0d 0f c7 00 07 0d 92 00 0f 8d 0f 36 ...............6 | 112: 0e cb 0e 6b 0e 0e 0d b6 0d 92 0d 92 00 00 00 00 ...k............ | 3472: 00 00 22 08 06 17 11 11 01 31 74 61 62 6c 65 74 .........1tablet | 3488: 32 74 32 08 43 52 45 41 54 45 20 54 41 42 4c 45 2t2.CREATE TABLE | 3504: 20 74 32 28 78 29 56 07 06 17 1f 1f 01 7d 74 61 t2(x)V.......ta | 3520: 62 6c 65 74 31 5f 63 6f 6e 66 69 67 74 31 5f 63 blet1_configt1_c | 3536: 6f 6e 66 69 67 07 43 52 45 41 54 45 20 54 41 42 onfig.CREATE TAB | 3552: 4c 45 20 27 74 31 5f 63 6f 6e 66 69 67 27 28 6b LE 't1_config'(k | 3568: 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 76 29 PRIMARY KEY, v) | 3584: 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 5b 06 WITHOUT ROWID[. | 3600: 07 17 21 21 01 81 01 74 61 62 6c 65 74 31 5f 64 ..!!...tablet1_d | 3616: 6f 63 73 69 7a 65 74 31 5f 64 6f 63 73 69 7a 65 ocsizet1_docsize | 3632: 06 43 52 45 41 54 45 20 54 41 42 4c 45 20 27 74 .CREATE TABLE 't | 3648: 31 5f 64 6f 63 73 69 7a 65 27 28 69 64 20 49 4e 1_docsize'(id IN | 3664: 54 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 TEGER PRIMARY KE | 3680: 59 2c 20 73 7a 20 42 4c 4f 42 29 5e 05 07 17 21 Y, sz BLOB)^...! | 3696: 21 01 81 07 74 61 62 6c 65 74 31 5f 63 6f 6e 74 !...tablet1_cont | 3712: 65 6e 74 74 31 5f 63 6f 6e 74 65 6e 74 05 43 52 entt1_content.CR | 3728: 45 41 54 45 20 54 41 42 4c 45 20 27 74 31 5f 63 EATE TABLE 't1_c | 3744: 6f 6e 74 65 6e 74 27 28 69 64 20 49 4e 54 45 47 ontent'(id INTEG | 3760: 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 ER PRIMARY KEY, | 3776: 63 30 2c 20 63 31 2c 20 63 32 29 69 04 07 17 19 c0, c1, c2)i.... | 3792: 19 01 81 2d 74 61 62 6c 65 74 31 5f 69 64 78 74 ...-tablet1_idxt | 3808: 31 5f 69 64 78 04 43 52 45 41 54 45 20 54 41 42 1_idx.CREATE TAB | 3824: 4c 45 20 27 74 31 5f 69 64 78 27 28 73 65 67 69 LE 't1_idx'(segi | 3840: 64 2c 20 74 65 72 6d 2c 20 70 67 6e 6f 2c 20 50 d, term, pgno, P | 3856: 52 49 4d 41 52 59 20 4b 45 59 28 73 65 67 69 64 RIMARY KEY(segid | 3872: 2c 20 74 65 72 6d 29 29 20 57 49 54 48 4f 55 54 , term)) WITHOUT | 3888: 20 52 4f 57 49 44 55 03 07 17 1b 1b 01 81 01 74 ROWIDU........t | 3904: 61 62 6c 65 74 31 5f 64 61 74 61 74 31 5f 64 61 ablet1_datat1_da | 3920: 74 61 03 43 52 45 41 54 45 20 54 41 42 4c 45 20 ta.CREATE TABLE | 3936: 27 74 31 5f 64 61 74 61 27 28 69 64 20 49 4e 54 't1_data'(id INT | 3952: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 EGER PRIMARY KEY | 3968: 2c 20 62 6c 6f 63 6b 20 42 4c 4f 42 29 38 02 06 , block BLOB)8.. | 3984: 17 11 11 08 5f 74 61 62 6c 65 74 31 74 31 43 52 ...._tablet1t1CR | 4000: 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 42 EATE VIRTUAL TAB | 4016: 4c 45 20 74 31 20 55 53 49 4e 47 20 66 74 73 35 LE t1 USING fts5 | 4032: 28 61 2c 62 2c 63 29 00 00 00 00 00 00 00 00 00 (a,b,c)......... | page 3 offset 8192 | 0: 0d 00 00 00 03 0c 94 00 0f e6 0f ef 0c 94 00 00 ................ | 3216: 00 00 00 00 86 4a 84 80 80 80 80 01 04 00 8d 18 .....J.......... | 3232: 00 00 03 2b 02 30 30 01 02 06 01 02 06 01 02 06 ...+.00......... | 3248: 1f 02 03 01 02 03 01 02 03 01 08 32 30 31 36 30 ...........20160 | 3264: 36 30 39 01 02 07 01 02 07 01 02 07 01 01 34 01 609...........4. | 3280: 02 05 01 02 05 01 02 05 01 01 35 01 02 04 01 02 ..........5..... | 3296: 04 01 02 04 02 07 30 30 30 30 30 30 30 1c 02 04 ......0000000... | 3312: 01 02 04 01 02 04 01 06 62 69 6e 61 72 79 03 06 ........binary.. | 3328: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3344: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3360: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3376: 03 06 01 02 02 03 06 01 02 02 01 08 63 6f 6d 70 ............comp | 3392: 69 6c 65 72 01 02 02 01 02 02 01 02 02 01 06 64 iler...........d | 3408: 62 73 74 61 74 07 02 03 01 02 03 01 02 03 02 04 bstat........... | 3424: 65 62 75 67 04 02 02 01 02 02 01 02 02 01 06 65 ebug...........e | 3440: 6e 61 62 6c 65 07 02 02 01 02 02 01 02 02 01 02 nable........... | 3456: 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 ................ | 3472: 01 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 ................ | 3488: 02 02 01 02 02 01 02 02 01 02 02 01 02 02 01 02 ................ | 3504: 02 01 02 02 02 08 78 74 65 6e 73 69 6f 6e 1f 02 ......xtension.. | 3520: 04 01 02 04 01 02 04 01 04 66 74 73 34 0a 02 03 .........fts4... | 3536: 01 02 03 01 02 03 04 01 35 0d 02 03 01 02 03 01 ........5....... | 3552: 02 03 01 03 67 63 63 01 02 03 01 02 03 01 02 03 ....gcc......... | 3568: 02 06 65 6f 70 6f 6c 79 10 02 03 01 02 03 01 02 ..eopoly........ | 3584: 03 01 05 6a 73 6f 6e 31 13 02 03 01 02 03 01 02 ...json1........ | 3600: 03 01 04 6c 6f 61 64 1f 02 03 01 02 03 01 02 03 ...load......... | 3616: 01 03 6d 61 78 1c 02 02 01 02 02 01 02 02 02 05 ..max........... | 3632: 65 6d 6f 72 79 1c 02 03 01 02 03 01 02 03 04 04 emory........... | 3648: 73 79 73 35 16 02 03 01 02 02 03 01 03 01 06 6e sys5...........n | 3664: 6f 63 61 73 65 02 06 01 02 02 03 06 01 02 02 03 ocase........... | 3680: 06 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 ................ | 3696: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3712: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3728: 02 01 04 6f 6d 69 74 1f 02 02 01 02 02 01 02 02 ...omit......... | 3744: 01 05 72 74 72 65 65 19 02 03 01 02 03 01 02 03 ..rtree......... | 3760: 04 02 69 6d 01 06 01 02 02 03 06 01 02 02 03 06 ..im............ | 3776: 01 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 ................ | 3792: 02 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 ................ | 3808: 02 03 06 01 02 02 03 06 01 02 02 03 06 01 02 02 ................ | 3824: 01 0a 74 68 72 65 61 64 73 61 66 65 03 57 34 56 ..threadsafe.W4V | 3840: 94 64 91 46 85 84 04 76 74 61 62 07 02 04 01 02 .d.F...vtab..... | 3856: 04 01 02 04 01 01 78 01 06 01 01 02 01 06 01 01 ......x......... | 3872: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 10 02 ................ | 3888: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3904: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 3920: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 3936: 01 02 01 06 01 01 10 01 06 01 01 02 01 06 01 01 ................ | 3952: 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 ................ | 3968: 01 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 ................ | 3984: 06 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 ................ | 4000: 01 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 ................ | 4016: 01 02 01 06 01 01 02 01 06 01 01 02 01 06 01 01 ................ | 4032: 02 01 06 01 01 02 01 06 01 01 02 04 15 13 0c 0c ................ | 4048: 12 44 13 11 0f 47 13 0f 0c 0e 11 10 0f 0e 10 0f .D...G.......... | 4064: 44 0f 10 40 15 0f 07 01 03 00 14 24 5a 24 24 0f D..@.......$Z$$. | 4080: 0a 03 00 24 00 00 00 00 01 01 01 00 01 01 01 01 ...$............ | page 4 offset 12288 | 0: 0a 00 00 00 01 0f fa 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 05 04 09 0c 01 02 ................ | page 5 offset 16384 | 0: 0d 00 00 00 24 0c 0a 00 0f d8 0f af 0f 86 0f 74 ....$..........t | 16: 0f 61 0f 4e 0f 2f 0f 0f 0e ef 0e d7 0e be 0e a5 .a.N./.......... | 32: 0e 8d 0e 74 0e 5b 0e 40 0e 24 0e 08 0d ef 0d d5 ...t.[.@.$...... | 48: 0d bb 0d a0 0d 84 0d 68 0d 4f 0d 35 0d 1b 0c fb .......h.O.5.... | 64: 0c da 0c b9 0c 99 0c 00 00 00 00 00 00 00 00 00 ................ | 3072: 00 00 00 00 00 00 00 00 00 00 18 24 05 00 25 0f ...........$..%. | 3088: 19 54 48 52 45 41 44 53 41 46 45 3d 30 58 42 49 .THREADSAFE=0XBI | 3104: 4e 41 52 59 18 23 05 00 25 0f 19 54 48 52 45 41 NARY.#..%..THREA | 3120: 44 53 41 46 45 3d 30 58 4e 4f 43 41 53 45 17 22 DSAFE=0XNOCASE.. | 3136: 05 00 25 0f 17 54 48 52 45 41 44 53 31 46 45 3d ..%..THREADS1FE= | 3152: 30 58 52 64 52 49 4d 1f 21 05 00 33 0f 19 4f 4d 0XRdRIM.!..3..OM | 3168: 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 4f IT LOAD EXTENSIO | 3184: 4e 58 42 49 4e 41 52 59 1f 20 05 00 33 0f 19 4f NXBINARY. ..3..O | 3200: 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 49 MIT LOAD EXTENSI | 3216: 4f 4e 58 4e 4f 43 41 53 45 1e 1f 05 00 33 0f 17 ONXNOCASE....3.. | 3232: 4f 4d 49 54 20 4c 4f 41 44 20 45 58 54 45 4e 53 OMIT LOAD EXTENS | 3248: 49 4f 4e 58 52 54 52 49 4d 1f 1e 05 00 33 0f 19 IONXRTRIM....3.. | 3264: 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 30 MAX MEMORY=50000 | 3280: 30 30 30 58 42 49 4e 41 52 59 1f 1d 05 00 33 0f 000XBINARY....3. | 3296: 19 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 30 .MAX MEMORY=5000 | 3312: 30 30 30 30 58 4e 4f 43 41 53 45 1e 1c 05 00 33 0000XNOCASE....3 | 3328: 0f 17 4d 41 58 20 4d 45 4d 4f 52 59 3d 35 30 30 ..MAX MEMORY=500 | 3344: 30 30 30 30 30 58 52 54 52 49 4d 18 1b 05 00 25 00000XRTRIM....% | 3360: 0f 19 45 4e 41 42 4c 45 20 52 54 52 45 45 58 42 ..ENABLE RTREEXB | 3376: 49 4e 41 52 59 18 1a 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3392: 4c 45 20 52 54 52 45 45 58 4e 4f 43 41 53 45 17 LE RTREEXNOCASE. | 3408: 19 05 00 25 0f 17 45 4e 41 42 4c 45 20 52 54 52 ...%..ENABLE RTR | 3424: 45 45 58 52 54 52 49 4d 1a 18 05 00 29 0f 19 45 EEXRTRIM....)..E | 3440: 4e 41 42 4b 45 20 4d 45 4d 53 59 53 35 58 42 49 NABKE MEMSYS5XBI | 3456: 4e 41 52 59 1a 17 05 00 29 0f 19 45 4e 41 42 4c NARY....)..ENABL | 3472: 42 60 2d 45 4d 53 59 53 35 58 4e 4f 43 41 53 45 B`-EMSYS5XNOCASE | 3488: 19 16 05 00 29 0f 17 45 4e 41 42 4c 45 20 4d 46 ....)..ENABLE MF | 3504: 4d 53 59 53 35 58 52 54 52 49 4d 18 15 05 00 25 MSYS5XRTRIM....% | 3520: 0f 19 45 4e 41 42 4c 45 20 4a 53 4f 4e 31 58 42 ..ENABLE JSON1XB | 3536: 49 4e 41 52 59 18 14 05 00 25 0f 19 45 4e 41 42 INARY....%..ENAB | 3552: 4c 45 20 4a 53 4f 4e 31 58 4e 4f 43 41 53 45 17 LE JSON1XNOCASE. | 3568: 13 05 00 25 0f 17 45 4e 41 42 4c 45 20 4a 53 4f ...%..ENABLE JSO | 3584: 4e 31 58 52 54 52 49 4d 1a 12 05 00 29 0f 19 45 N1XRTRIM....)..E | 3600: 42 4c 45 20 47 45 4f 50 4f 4c 59 58 42 49 4e 41 BLE GEOPOLYXBINA | 3616: 52 59 1a 11 05 00 39 0f 19 45 4e 41 42 4c 45 2e RY....9..ENABLE. | 3632: 41 40 47 45 4f 50 4f 4c 59 58 4e 4f 43 41 53 40 A@GEOPOLYXNOCAS@ | 3648: 4f 4c 59 58 55 09 10 05 00 29 0f 17 45 4e 41 42 OLYXU....)..ENAB | 3664: 4c 45 20 47 45 4f 52 54 52 49 4d 17 0f 05 00 23 LE GEORTRIM....# | 3680: 0f 19 45 4e 41 42 4c 45 20 46 54 53 35 58 42 49 ..ENABLE FTS5XBI | 3696: 4e 41 52 59 17 0e 05 00 23 0f 19 45 4e 41 42 4c NARY....#..ENABL | 3712: 45 20 46 54 53 35 58 4e 4f 43 41 53 45 16 0d 05 E FTS5XNOCASE... | 3728: 00 23 0f 17 45 4e 41 42 4c 45 20 46 54 53 35 58 .#..ENABLE FTS5X | 3744: 52 54 52 49 4d 17 0c 05 00 23 0f 19 45 4e 41 42 RTRIM....#..ENAB | 3760: 4c 45 20 46 54 53 34 58 42 49 4e 41 52 59 17 0b LE FTS4XBINARY.. | 3776: 05 00 23 0f 19 45 4e 41 42 4c 45 20 46 54 53 34 ..#..ENABLE FTS4 | 3792: 58 4e 4f 43 41 53 45 16 0a 05 00 23 0f 17 45 4e XNOCASE....#..EN | 3808: 41 42 4c 45 20 46 54 53 34 58 52 54 52 49 4d 1e ABLE FTS4XRTRIM. | 3824: 09 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3840: 54 41 54 20 56 54 41 42 58 42 49 4e 41 52 59 1e TAT VTABXBINARY. | 3856: 08 05 00 31 0f 19 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3872: 54 41 54 20 56 54 24 15 48 4e 4f 43 41 53 45 1d TAT VT$.HNOCASE. | 3888: 07 05 00 31 0f 17 45 4e 41 42 4c 45 20 44 42 53 ...1..ENABLE DBS | 3904: 54 41 54 20 56 54 41 42 58 52 54 52 49 4d 11 06 TAT VTABXRTRIM.. | 3920: 05 00 17 0f 19 44 45 42 55 47 58 42 49 4e 41 52 .....DEBUGXBINAR | 3936: 59 11 05 05 00 17 0f 19 44 45 42 55 47 58 4e 4f Y.......DEBUGXNO | 3952: 43 41 53 45 10 04 05 00 17 0f 17 44 45 42 55 47 CASE.......DEBUG | 3968: 58 52 54 52 49 4d 27 03 05 00 43 0f 19 43 4f 4d XRTRIM'...C..COM | 3984: 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e 30 20 PILER=gcc-5.4.0 | 4000: 32 30 31 36 30 36 30 39 58 42 49 4e 41 52 59 27 20160609XBINARY' | 4016: 02 05 00 43 0f 19 43 4f 4d 50 49 4c 45 52 3d 67 ...C..COMPILER=g | 4032: 63 63 2d 35 2e 34 2e 30 20 32 30 31 36 30 36 30 cc-5.4.0 2016060 | 4048: 39 58 4e 4f 43 41 53 45 26 01 05 00 43 0f 17 43 9XNOCASE&...C..C | 4064: 4f 4d 50 49 4c 45 52 3d 67 63 63 2d 35 2e 34 2e OMPILER=gcc-5.4. | 4080: 30 20 32 30 31 36 30 36 30 39 58 52 54 52 49 4d 0 20160609XRTRIM | page 6 offset 20480 | 0: 0d 00 00 00 24 0e e0 00 0f f8 0f f0 0f e8 0f e0 ....$........... | 16: 0f d8 0f d0 0f c8 0f c0 0f b8 0f b0 0f a8 0f a0 ................ | 32: 1f 98 0f 90 0f 88 0f 80 0f 78 0f 70 0f 68 0f 60 .........x.p.h.` | 48: 0f 58 0f 50 0f 48 0f 40 0f 38 0f 30 0f 28 0f 20 .X.P.H.@.8.0.(. | 64: 0f 18 0f 10 0f 08 0f 00 0e f8 0e f0 0e e8 00 00 ................ | 3808: 06 24 03 00 12 02 01 01 06 23 03 00 12 02 01 01 .$.......#...... | 3824: 06 22 03 00 12 02 01 01 06 21 03 00 12 03 01 01 .........!...... | 3840: 06 20 03 00 12 03 01 01 06 1f 03 00 12 03 01 01 . .............. | 3856: 06 1e 03 00 12 03 01 01 06 1d 03 00 12 03 01 01 ................ | 3872: 06 1c 03 00 12 03 01 01 06 1b 03 00 12 02 01 01 ................ | 3888: 06 1a 03 00 12 02 01 01 06 19 03 00 12 02 01 01 ................ | 3904: 06 18 03 00 12 02 01 01 06 17 03 00 12 02 01 01 ................ | 3920: 06 15 f3 00 12 02 01 01 06 15 03 00 12 02 01 01 ................ | 3936: 06 14 03 00 12 02 01 01 06 13 03 00 12 02 01 01 ................ | 3952: 06 12 03 00 12 02 01 01 06 11 03 00 12 02 01 01 ................ | 3968: 06 10 03 00 12 02 01 01 06 0f 03 00 12 02 01 01 ................ | 3984: 06 0e 03 00 12 02 01 01 06 0d 03 00 12 02 01 01 ................ | 4000: 06 0c 03 00 12 02 01 01 06 0b 03 00 12 02 01 01 ................ | 4016: 06 0a 03 00 12 02 01 01 06 09 03 00 12 03 01 01 ................ | 4032: 06 08 03 00 12 03 01 01 06 07 03 00 12 03 01 01 ................ | 4048: 06 06 03 00 12 01 01 01 06 05 03 00 12 01 01 01 ................ | 4064: 06 04 03 00 12 01 01 01 06 03 03 00 12 06 01 01 ................ | 4080: 06 02 03 00 12 06 01 01 06 01 03 00 12 06 01 01 ................ | page 7 offset 24576 | 0: 0a 00 00 00 01 0f f4 00 0f f4 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 0b 03 1b 01 76 65 72 73 69 6f 6e 04 ........version. | page 8 offset 28672 | 0: 0d 00 00 00 03 0f d6 00 0f f4 0f 00 00 00 00 00 ................ | 4048: 00 00 00 00 00 00 11 03 02 2b 69 6e 74 65 67 72 .........+integr | 4064: 69 74 79 2d 63 68 65 63 6b 09 02 02 1b 72 65 62 ity-check....reb | 4080: 75 69 6c 64 0a 01 02 1d 6f 00 00 00 00 00 00 00 uild....o....... | end crash-3d05232c78871b.db }]} {} do_execsql_test 11.1 { UPDATE t1 SET b=quote(zeroblob('2025-01-09')) WHERE t1 MATCH 't*'; INSERT INTO t1(t1,rank) VALUES('secure-delete',1); } do_catchsql_test 11.2 { BEGIN; REPLACE INTO t1(rowid,b,a,rowid) VALUES(x'44023b9eb002d28b0ee90c',1,2,3); PRAGMA writable_schema=RESET; INSERT INTO t1(t1) SELECT x FROM t2; ROLLBACK; } {1 {database disk image is malformed}} do_catchsql_test 11.3 { REPLACE INTO t1(rowid,b,a,rowid) VALUES(x'44023b9eb002d28b0ee90c',1,2,3); } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test |
Changes to ext/fts5/test/fts5corrupt6.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt6 | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # inconsistencies in the backing tables) correctly. In this case # "correctly" means without crashing. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt6 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 database_may_be_corrupt |
︙ | ︙ |
Added ext/fts5/test/fts5corrupt7.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | # 2023 April 30 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt7 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_may_be_corrupt 1 do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } set doc [string repeat "a b " 30] do_execsql_test 1.1 { BEGIN; INSERT INTO t1(rowid, x) VALUES(123, $doc); INSERT INTO t1(rowid, x) VALUES(124, $doc); COMMIT; } execsql_pp { SELECT id, fts5_decode(id, block), quote(block) FROM t1_data } set rows [db eval { SELECT rowid FROM t1_data }] db_save_and_close foreach r $rows { db_restore_and_reopen proc edit_block {b} { binary scan $b c* in set out [lreplace $in 0 1 255 255] binary format c* $out } db func edit_block edit_block do_execsql_test 1.2.$r.1 { UPDATE t1_data SET block = edit_block(block) WHERE rowid=$r; } do_execsql_test 1.2.$r.2 { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } do_test 1.2.$r.3 { catchsql { DELETE FROM t1 WHERE rowid=123; } catchsql { DELETE FROM t1 WHERE rowid=124; } set {} {} } {} db close } foreach r $rows { set r 137438953475 db_restore_and_reopen proc edit_block {b} { binary scan $b c* in set out [lreplace $in end end 127] binary format c* $out } db func edit_block edit_block do_execsql_test 1.2.$r.1 { UPDATE t1_data SET block = edit_block(block) WHERE rowid=$r; } do_execsql_test 1.2.$r.2 { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } do_test 1.2.$r.3 { catchsql { DELETE FROM t1 WHERE rowid=124; } catchsql { DELETE FROM t1 WHERE rowid=123; } set {} {} } {} db close } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); BEGIN; INSERT INTO t1 VALUES('abc'); INSERT INTO t1 VALUES('b d d d'); COMMIT; INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } execsql_pp { SELECT id, quote(block) FROM t1_data } do_execsql_test 2.1 { SELECT quote(block) FROM t1_data WHERE id > 10; } {X'0000001A04306162630102020101620202020101640206030303040806'} do_execsql_test 2.2 { UPDATE t1_data SET block=X'0000001A04306162630102025501620202020101640206030303040806' WHERE id>10 } do_catchsql_test 2.3 { DELETE FROM t1 WHERE rowid = 1 } {/.*fts5: corrupt.*/} finish_test |
Added ext/fts5/test/fts5corrupt8.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 | # 2024 Aug 28 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5corrupt8 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); } do_execsql_test 1.1 { UPDATE t1_data SET block='hello world' WHERE id=10 } db close sqlite3 db test.db do_catchsql_test 1.2 { SELECT * FROM t1 } {1 {fts5: corrupt structure record for table "t1"}} do_catchsql_test 1.3 { DROP TABLE t1 } {0 {}} do_execsql_test 1.4 { SELECT * FROM sqlite_schema } do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); } do_execsql_test 2.1 { UPDATE t1_config SET v=555 WHERE k='version' } db close sqlite3 db test.db do_catchsql_test 2.2 { SELECT * FROM t1 } {1 {invalid fts5 file format (found 555, expected 4 or 5) - run 'rebuild'}} do_catchsql_test 2.3 { DROP TABLE t1 } {1 {invalid fts5 file format (found 555, expected 4 or 5) - run 'rebuild'}} do_test 2.4 { sqlite3_fts5_drop_corrupt_table db main t1 } {} do_execsql_test 2.5 { SELECT * FROM sqlite_schema } do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); } do_execsql_test 3.1 { DELETE FROM t1_config; } db close sqlite3 db test.db do_catchsql_test 3.2 { SELECT * FROM t1 } {1 {invalid fts5 file format (found 0, expected 4 or 5) - run 'rebuild'}} do_catchsql_test 3.3 { DROP TABLE t1 } {1 {invalid fts5 file format (found 0, expected 4 or 5) - run 'rebuild'}} do_test 3.4 { sqlite3_db_config db DEFENSIVE 1 } {1} do_test 3.5 { sqlite3_fts5_drop_corrupt_table db main t1 } {} do_test 3.6 { sqlite3_db_config db DEFENSIVE -1 } {1} do_execsql_test 3.7 { SELECT * FROM sqlite_schema } finish_test |
Changes to ext/fts5/test/fts5delete.test.
︙ | ︙ | |||
108 109 110 111 112 113 114 115 116 117 | do_execsql_test 3.3 { INSERT INTO tx(tx, rowid, a, b, c, d) VALUES('delete', 1, 'abc def', NULL, 'a b c', NULL); } do_execsql_test 3.4 { INSERT INTO tx(tx) VALUES('integrity-check'); } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | do_execsql_test 3.3 { INSERT INTO tx(tx, rowid, a, b, c, d) VALUES('delete', 1, 'abc def', NULL, 'a b c', NULL); } do_execsql_test 3.4 { INSERT INTO tx(tx) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE ft1 USING fts5(a, b UNINDEXED, content='', contentless_unindexed=1 ); CREATE VIRTUAL TABLE ft2 USING fts5(a, b UNINDEXED, content='', contentless_unindexed=1, contentless_delete=1 ); INSERT INTO ft1(rowid, a, b) VALUES(1, 'one', 'i'); INSERT INTO ft1(rowid, a, b) VALUES(2, 'two', 'ii'); INSERT INTO ft1(rowid, a, b) VALUES(3, 'three', 'iii'); INSERT INTO ft2(rowid, a, b) VALUES(1, 'one', 'i'); INSERT INTO ft2(rowid, a, b) VALUES(2, 'two', 'ii'); INSERT INTO ft2(rowid, a, b) VALUES(3, 'three', 'iii'); } do_catchsql_test 4.1 { DELETE FROM ft1 WHERE rowid=2 } {1 {cannot DELETE from contentless fts5 table: ft1}} do_catchsql_test 4.2 { DELETE FROM ft2 WHERE rowid=2 } {0 {}} do_catchsql_test 4.3 { INSERT INTO ft1(ft1, rowid, a) VALUES('delete', 2, 'two'); } {0 {}} do_catchsql_test 4.2 { INSERT INTO ft2(ft2, rowid, a) VALUES('delete', 2, 'two'); } {1 {'delete' may not be used with a contentless_delete=1 table}} do_execsql_test 4.3 { SELECT rowid, * FROM ft1; } { 1 {} i 3 {} iii } do_execsql_test 4.4 { SELECT rowid, * FROM ft2; } { 1 {} i 3 {} iii } do_execsql_test 4.5 { SELECT * FROM ft1_content } {1 i 3 iii} do_execsql_test 4.6 { SELECT * FROM ft2_content } {1 i 3 iii} finish_test |
Changes to ext/fts5/test/fts5dlidx.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This test is focused on uses of doclist-index records. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5dlidx | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This test is focused on uses of doclist-index records. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5dlidx # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } if { $tcl_platform(wordSize)<8 } { finish_test |
︙ | ︙ |
Changes to ext/fts5/test/fts5doclist.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This test is focused on edge cases in the doclist format. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5doclist | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This test is focused on edge cases in the doclist format. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5doclist # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- |
︙ | ︙ |
Changes to ext/fts5/test/fts5ea.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # Test the fts5 expression parser directly using the fts5_expr() SQL # test function. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ea | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # Test the fts5 expression parser directly using the fts5_expr() SQL # test function. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ea # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc do_syntax_error_test {tn expr err} { set ::se_expr $expr |
︙ | ︙ |
Changes to ext/fts5/test/fts5eb.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5eb | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5eb # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc do_syntax_error_test {tn expr err} { set ::se_expr $expr |
︙ | ︙ | |||
82 83 84 85 86 87 88 | } do_execsql_test 3.0 { CREATE VIRTUAL TABLE e1 USING fts5(text, tokenize = 'porter unicode61'); INSERT INTO e1 VALUES ('just a few words with a / inside'); } do_execsql_test 3.1 { | | | > > > | 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | } do_execsql_test 3.0 { CREATE VIRTUAL TABLE e1 USING fts5(text, tokenize = 'porter unicode61'); INSERT INTO e1 VALUES ('just a few words with a / inside'); } do_execsql_test 3.1 { SELECT rowid, format('%g',bm25(e1)) FROM e1 WHERE e1 MATCH '"just"' ORDER BY rank; } {1 -1e-06} do_execsql_test 3.2 { SELECT rowid FROM e1 WHERE e1 MATCH '"/" OR "just"' } 1 do_execsql_test 3.3 { SELECT rowid, format('%g',bm25(e1)) FROM e1 WHERE e1 MATCH '"/" OR "just"' ORDER BY rank; } {1 -1e-06} do_execsql_test 3.4 " SELECT fts5_expr_tcl('e AND \" \"'); " {{AND [nearset -- {e}] [{}]}} finish_test |
Added ext/fts5/test/fts5expr.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | # 2024 August 8 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5expr # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(a); INSERT INTO x1(rowid, a) VALUES (113, 'fts5 expr test'); } do_execsql_test 1.1 { SELECT rowid FROM x1('expr'); } {113} for {set ii 0} {$ii < 300} {incr ii} { set expr "expr " append expr [string repeat "NOT abcd " $ii] if {$ii<257} { set res {0 113} } else { set res {1 {fts5 expression tree is too large (maximum depth 256)}} } do_catchsql_test 1.1.$ii { SELECT rowid FROM x1($expr) } $res } do_execsql_test 1.2 { SELECT rowid FROM x1 WHERE a MATCH '"..."' } {} finish_test |
Changes to ext/fts5/test/fts5fault4.test.
︙ | ︙ | |||
86 87 88 89 90 91 92 | } set ::res [db eval {SELECT rowid, x1 FROM x1 WHERE x1 MATCH '*reads'}] do_faultsim_test 4 -faults oom-* -body { db eval {SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads'} } -test { | | | 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | } set ::res [db eval {SELECT rowid, x1 FROM x1 WHERE x1 MATCH '*reads'}] do_faultsim_test 4 -faults oom-* -body { db eval {SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads'} } -test { faultsim_test_result {0 {0 {} 2}} } #------------------------------------------------------------------------- # An OOM within a query that uses a custom rank function. # reset_db do_execsql_test 5.0 { |
︙ | ︙ |
Changes to ext/fts5/test/fts5fault6.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5fault6 | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # This file is focused on OOM errors. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5fault6 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- |
︙ | ︙ |
Changes to ext/fts5/test/fts5fault8.test.
︙ | ︙ | |||
53 54 55 56 57 58 59 | } -test { faultsim_test_result {0 {1 3}} {1 SQLITE_NOMEM} } } } ;# foreach_detail_mode... | < | 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | } -test { faultsim_test_result {0 {1 3}} {1 SQLITE_NOMEM} } } } ;# foreach_detail_mode... do_execsql_test 4.0 { CREATE VIRTUAL TABLE x2 USING fts5(a); INSERT INTO x2(x2, rank) VALUES('crisismerge', 2); INSERT INTO x2(x2, rank) VALUES('pgsz', 32); INSERT INTO x2 VALUES('a b c d'); INSERT INTO x2 VALUES('e f g h'); INSERT INTO x2 VALUES('i j k l'); |
︙ | ︙ | |||
75 76 77 78 79 80 81 82 83 84 | do_faultsim_test 4 -faults oom-* -prep { faultsim_restore_and_reopen } -body { execsql { INSERT INTO x2(x2) VALUES('optimize') } } -test { faultsim_test_result {0 {}} {1 SQLITE_NOMEM} } finish_test | > > > > > > > > > > > > > | 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | do_faultsim_test 4 -faults oom-* -prep { faultsim_restore_and_reopen } -body { execsql { INSERT INTO x2(x2) VALUES('optimize') } } -test { faultsim_test_result {0 {}} {1 SQLITE_NOMEM} } set TMPDBERROR {1 {unable to open a temporary database file for storing temporary tables}} do_faultsim_test 5 -faults oom-t* -prep { faultsim_restore_and_reopen execsql { PRAGMA temp_store = memory } } -body { execsql { PRAGMA integrity_check } } -test { if {[string match {*error code=7*} $testresult]==0} { faultsim_test_result {0 ok} {1 SQLITE_NOMEM} $::TMPDBERROR } } finish_test |
Added ext/fts5/test/fts5faultF.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 | # 2023 July 20 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # # This file is focused on OOM errors. Particularly those that may occur # when using contentless_delete=1 databases. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5faultF # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } faultsim_save_and_close do_faultsim_test 1 -prep { faultsim_restore_and_reopen } -body { execsql { CREATE VIRTUAL TABLE t1 USING fts5(x, y, content=, contentless_delete=1) } } -test { faultsim_test_result {0 {}} {1 {vtable constructor failed: t1}} } reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1); BEGIN; INSERT INTO t1(rowid, doc) VALUES(1, 'a b c d'); INSERT INTO t1(rowid, doc) VALUES(2, 'a b c d'); INSERT INTO t1(rowid, doc) VALUES(3, 'a b c d'); INSERT INTO t1(rowid, doc) VALUES(4, 'a b c d'); COMMIT; DELETE FROM t1 WHERE rowid IN (2, 4); } do_faultsim_test 2 -prep { sqlite3 db test.db execsql { SELECT rowid FROM t1 } } -body { execsql { SELECT rowid FROM t1('b c'); } } -test { faultsim_test_result {0 {1 3}} } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1); BEGIN; INSERT INTO t1(rowid, doc) VALUES(1, 'a b c d'); INSERT INTO t1(rowid, doc) VALUES(2, 'a b c d'); INSERT INTO t1(rowid, doc) VALUES(3, 'a b c d'); INSERT INTO t1(rowid, doc) VALUES(4, 'a b c d'); COMMIT; } faultsim_save_and_close do_faultsim_test 3 -prep { faultsim_restore_and_reopen execsql { SELECT rowid FROM t1 } } -body { execsql { INSERT INTO t1(rowid, doc) VALUES(5, 'a b c d'); } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(doc, content=, contentless_delete=1); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<1000 ) INSERT INTO t1(rowid, doc) SELECT i, 'a b c d' FROM s; } do_execsql_test 4.1 { DELETE FROM t1 WHERE rowid <= 25 } faultsim_save_and_close do_faultsim_test 4 -faults oom-t* -prep { faultsim_restore_and_reopen execsql { SELECT rowid FROM t1 } } -body { execsql { DELETE FROM t1 WHERE rowid < 100 } } -test { faultsim_test_result {0 {}} } finish_test |
Added ext/fts5/test/fts5faultG.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 | # 2010 June 15 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5faultG # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set ::testprefix fts5faultG do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a); INSERT INTO t1 VALUES('test renaming the table'); INSERT INTO t1 VALUES(' after it has been written'); INSERT INTO t1 VALUES(' actually other stuff instead'); } faultsim_save_and_close do_faultsim_test 1 -faults oom* -prep { faultsim_restore_and_reopen execsql { BEGIN; DELETE FROM t1 WHERE rowid=2; } } -body { execsql { DELETE FROM t1; } } -test { catchsql { COMMIT } faultsim_integrity_check faultsim_test_result {0 {}} } reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, content=, contentless_delete=1); BEGIN; INSERT INTO t1 VALUES('here''s some text'); INSERT INTO t1 VALUES('useful stuff, text'); INSERT INTO t1 VALUES('what would we do without text!'); COMMIT; } faultsim_save_and_close do_faultsim_test 2 -faults oom* -prep { faultsim_restore_and_reopen execsql { BEGIN; DELETE FROM t1 WHERE rowid=2; } } -body { execsql { INSERT INTO t1(t1) VALUES('optimize'); } } -test { faultsim_integrity_check faultsim_test_result {0 {}} } finish_test |
Added ext/fts5/test/fts5faultH.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 | # 2010 June 15 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5faultG # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set ::testprefix fts5faultH sqlite3_fts5_register_origintext db do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5( x, tokenize="origintext unicode61", tokendata=1 ); BEGIN; INSERT INTO t1 VALUES('oNe tWo thRee'); INSERT INTO t1 VALUES('One Two Three'); INSERT INTO t1 VALUES('onE twO threE'); COMMIT; BEGIN; INSERT INTO t1 VALUES('one two three'); INSERT INTO t1 VALUES('one two three'); INSERT INTO t1 VALUES('one two three'); COMMIT; } do_faultsim_test 1 -faults oom* -prep { } -body { execsql { SELECT rowid FROM t1('three'); } } -test { faultsim_integrity_check faultsim_test_result {0 {1 2 3 4 5 6}} } reset_db sqlite3_fts5_register_origintext db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5( x, tokenize="origintext unicode61", tokendata=1 ); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); BEGIN; INSERT INTO t1(rowid, x) VALUES(10, 'aaa bbb BBB'); INSERT INTO t1(rowid, x) VALUES(12, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(13, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(14, 'bbb BBB bbb'); INSERT INTO t1(rowid, x) VALUES(15, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(16, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(17, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(18, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(19, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(20, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(21, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(22, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(23, 'bbb bbb bbb'); INSERT INTO t1(rowid, x) VALUES(24, 'aaa bbb BBB'); COMMIT; } do_faultsim_test 2 -faults oom* -prep { } -body { execsql { SELECT rowid FROM t1('BBB AND AAA'); } } -test { faultsim_integrity_check faultsim_test_result {0 {10 24}} } reset_db sqlite3_fts5_register_origintext db do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5( x, tokenize="origintext unicode61", tokendata=1 ); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); INSERT INTO t1(rowid, x) VALUES(9, 'bbb Bbb BBB'); BEGIN; INSERT INTO t1(rowid, x) VALUES(10, 'aaa bbb BBB'); INSERT INTO t1(rowid, x) VALUES(11, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(12, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(13, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(14, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(15, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(16, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(17, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(18, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(19, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(20, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(21, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(22, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(23, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(24, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(25, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(26, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(27, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(28, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(29, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(30, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(31, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(32, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(33, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(34, 'bbb Bbb BBB'); INSERT INTO t1(rowid, x) VALUES(35, 'aaa bbb BBB'); COMMIT; } do_faultsim_test 3.1 -faults oom* -prep { } -body { execsql { SELECT rowid FROM t1('BBB AND AAA'); } } -test { faultsim_integrity_check faultsim_test_result {0 {10 35}} } do_faultsim_test 3.2 -faults oom* -prep { } -body { execsql { SELECT count(*) FROM t1('BBB'); } } -test { faultsim_integrity_check faultsim_test_result {0 27} } finish_test |
Added ext/fts5/test/fts5faultI.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 | # 2010 June 15 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5faultI # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set ::testprefix fts5faultI do_execsql_test 1.0 { PRAGMA encoding = utf16; CREATE VIRTUAL TABLE t1 USING fts5(x, locale=1); INSERT INTO t1 VALUES('origintext unicode61 ascii porter trigram'); } faultsim_save_and_close faultsim_restore_and_reopen do_faultsim_test 1 -faults oom* -prep { } -body { execsql { SELECT rowid FROM t1(fts5_locale('en_US', 'origintext')); } } -test { faultsim_test_result {0 1} } do_faultsim_test 2 -faults oom* -prep { faultsim_restore_and_reopen execsql { SELECT * FROM t1('ascii'); } } -body { execsql { UPDATE t1 SET rowid=rowid+1; } } -test { faultsim_test_result {0 {}} } fts5_aux_test_functions db do_faultsim_test 3 -faults oom* -prep { } -body { execsql { SELECT fts5_columnlocale(t1, 0) FROM t1('unicode*'); } } -test { faultsim_test_result {0 {{}}} {1 SQLITE_NOMEM} } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE w1 USING fts5(a); } faultsim_save_and_close do_faultsim_test 4 -faults oom* -prep { faultsim_restore_and_reopen execsql { BEGIN; INSERT INTO w1 VALUES('token token token'); } } -body { execsql { INSERT INTO w1(w1, rank) VALUES('rank', 'bm25()'); } } -test { faultsim_test_result {0 {}} } do_faultsim_test 5 -faults oom* -prep { faultsim_restore_and_reopen execsql { BEGIN; INSERT INTO w1 VALUES('one'); SAVEPOINT one; INSERT INTO w1 VALUES('two'); ROLLBACK TO one; } } -body { execsql { INSERT INTO w1 VALUES('string'); } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE w1 USING fts5(a); INSERT INTO w1 VALUES('one two three'); } fts5_aux_test_functions db do_faultsim_test 5 -faults oom* -prep { } -body { execsql { SELECT fts5_test_insttoken(w1, 0, 0) FROM w1('two'); } } -test { faultsim_test_result {0 two} {1 SQLITE_NOMEM} } #------------------------------------------------------------------------- reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE w1 USING fts5(a); INSERT INTO w1 VALUES('one two three'); } fts5_aux_test_functions db faultsim_save_and_close do_faultsim_test 6 -faults oom* -prep { faultsim_restore_and_reopen db eval { BEGIN; INSERT INTO w1 VALUES('four five six'); SAVEPOINT abc; INSERT INTO w1 VALUES('seven eight nine'); SAVEPOINT def; INSERT INTO w1 VALUES('ten eleven twelve'); } } -body { execsql { RELEASE abc; } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- reset_db do_execsql_test 7.0 { CREATE VIRTUAL TABLE w1 USING fts5(a); INSERT INTO w1 VALUES('one two three'); INSERT INTO w1 VALUES('three two one'); DELETE FROM w1_content WHERE rowid=1; } faultsim_save_and_close do_faultsim_test 7 -faults oom* -prep { faultsim_restore_and_reopen db eval { SELECT * FROM w1 } } -body { execsql { PRAGMA integrity_check; } } -test { } #------------------------------------------------------------------------- reset_db fts5_tclnum_register db fts5_aux_test_functions db do_execsql_test 8.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize = "tclnum query", detail=columns ); INSERT INTO ft VALUES('one two three i ii iii'); INSERT INTO ft VALUES('four five six iv v vi'); INSERT INTO ft VALUES('eight nine ten viii ix x'); } {} do_faultsim_test 8.1 -faults oom* -prep { } -body { execsql { SELECT fts5_test_collist (ft) FROM ft('one two'); } } -test { faultsim_test_result {0 {{0.0 1.0}}} {1 {SQL logic error}} {1 SQLITE_NOMEM} } do_faultsim_test 8.2 -faults oom* -prep { } -body { execsql { SELECT rowid FROM ft('one two') ORDER BY rank; } } -test { faultsim_test_result {0 1} {1 {SQL logic error}} {1 SQLITE_NOMEM} } #------------------------------------------------------------------------- reset_db do_execsql_test 9.0 { CREATE VIRTUAL TABLE ft USING fts5(x); INSERT INTO ft VALUES('one two three i ii iii'); INSERT INTO ft VALUES('four five six iv v vi'); INSERT INTO ft VALUES('eight nine ten viii ix x'); } {} faultsim_save_and_close do_faultsim_test 9.1 -faults oom* -prep { faultsim_restore_and_reopen } -body { execsql { UPDATE ft SET rowid=4 WHERE rowid=1 } } -test { faultsim_test_result {0 {}} } do_faultsim_test 9.2 -faults oom* -prep { faultsim_restore_and_reopen } -body { execsql { SELECT rowid FROM ft WHERE x MATCH 'one AND two AND three' } } -test { faultsim_test_result {0 1} } #------------------------------------------------------------------------- reset_db do_execsql_test 10.0 { CREATE VIRTUAL TABLE ft USING fts5(x, locale=1); INSERT INTO ft VALUES(fts5_locale('hello', 'one two three i ii iii')); INSERT INTO ft VALUES('four five six iv v vi'); INSERT INTO ft VALUES('eight nine ten viii ix x'); } {} do_execsql_test 10.1 { SELECT fts5_get_locale(ft, 0) FROM ft WHERE x MATCH 'one AND two AND three' } {hello} faultsim_save_and_close do_faultsim_test 10.1 -faults oom* -prep { faultsim_restore_and_reopen } -body { execsql { SELECT fts5_get_locale(ft, 0) FROM ft WHERE x MATCH 'one AND two AND three' } } -test { faultsim_test_result {0 hello} } breakpoint faultsim_save_and_close do_faultsim_test 10.2 -faults oom-t* -prep { faultsim_restore_and_reopen } -body { execsql { INSERT INTO ft VALUES(zeroblob(10000)); } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- reset_db do_execsql_test 11.0 { CREATE VIRTUAL TABLE f1 USING fts5(content); CREATE TABLE g1(id, content); INSERT INTO g1 VALUES(30000, 'a b c'); INSERT INTO g1 VALUES(40000, 'd e f'); } faultsim_save_and_close do_faultsim_test 11 -faults oom* -prep { faultsim_restore_and_reopen } -body { execsql { INSERT INTO f1(rowid, content) SELECT id, content FROM g1; } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- reset_db ifcapable foreignkey { do_execsql_test 12.0 { CREATE VIRTUAL TABLE f1 USING fts5(content); CREATE TABLE p1(a INTEGER PRIMARY KEY); CREATE TABLE c1(b REFERENCES p1 DEFERRABLE INITIALLY DEFERRED); } faultsim_save_and_close do_faultsim_test 11 -faults oom* -prep { faultsim_restore_and_reopen execsql { PRAGMA foreign_keys = 1; BEGIN; INSERT INTO c1 VALUES(123); SAVEPOINT xyz; } } -body { execsql { INSERT INTO f1 VALUES('a b c'); ROLLBACK TO xyz; COMMIT; } } -test { execsql { SELECT 123 } faultsim_test_result \ {1 {FOREIGN KEY constraint failed}} \ {1 {out of memory}} \ {1 {constraint failed}} } } #------------------------------------------------------------------------- reset_db do_execsql_test 13.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b); INSERT INTO t1 VALUES('abc def', X'123456'); } faultsim_save_and_close do_faultsim_test 13 -faults oom* -prep { faultsim_restore_and_reopen } -body { execsql { UPDATE t1 SET a='def abc' } } -test { faultsim_test_result {0 {}} } finish_test |
Changes to ext/fts5/test/fts5first.test.
︙ | ︙ | |||
18 19 20 21 22 23 24 25 26 27 28 29 30 31 | } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(a, b); } foreach {tn expr ok} { 1 {^abc} 1 2 {^abc + def} 1 3 {^ "abc def"} 1 4 {^"abc def"} 1 5 {abc ^def} 1 6 {abc + ^def} 0 | > | 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(a, b); } unset -nocomplain res foreach {tn expr ok} { 1 {^abc} 1 2 {^abc + def} 1 3 {^ "abc def"} 1 4 {^"abc def"} 1 5 {abc ^def} 1 6 {abc + ^def} 0 |
︙ | ︙ |
Changes to ext/fts5/test/fts5full.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # segid to use. In practice this can only really happen when automerge and # crisismerge are both disabled. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5full | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # segid to use. In practice this can only really happen when automerge and # crisismerge are both disabled. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5full # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x8 USING fts5(i); |
︙ | ︙ |
Changes to ext/fts5/test/fts5hash.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # The tests in this file are focused on the code in fts5_hash.c. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5hash | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # The tests in this file are focused on the code in fts5_hash.c. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5hash # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # Return a list of tokens (a vocabulary) that all share the same hash |
︙ | ︙ |
Changes to ext/fts5/test/fts5integrity.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file contains tests focused on the integrity-check procedure. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5integrity | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file contains tests focused on the integrity-check procedure. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5integrity # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE xx USING fts5(x); |
︙ | ︙ | |||
73 74 75 76 77 78 79 80 81 82 83 84 85 86 | sqlite3_db_config db DEFENSIVE 0 do_catchsql_test 4.2 { BEGIN; UPDATE aa_docsize SET sz = X'44' WHERE rowid = 3; INSERT INTO aa(aa) VALUES('integrity-check'); } {1 {database disk image is malformed}} do_catchsql_test 4.3 { ROLLBACK; BEGIN; UPDATE aa_data SET block = X'44' WHERE rowid = 1; INSERT INTO aa(aa) VALUES('integrity-check'); } {1 {database disk image is malformed}} | > > > | 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | sqlite3_db_config db DEFENSIVE 0 do_catchsql_test 4.2 { BEGIN; UPDATE aa_docsize SET sz = X'44' WHERE rowid = 3; INSERT INTO aa(aa) VALUES('integrity-check'); } {1 {database disk image is malformed}} do_execsql_test 4.2.1 { PRAGMA integrity_check(aa); } {{malformed inverted index for FTS5 table main.aa}} do_catchsql_test 4.3 { ROLLBACK; BEGIN; UPDATE aa_data SET block = X'44' WHERE rowid = 1; INSERT INTO aa(aa) VALUES('integrity-check'); } {1 {database disk image is malformed}} |
︙ | ︙ | |||
146 147 148 149 150 151 152 153 154 155 156 157 158 159 | INSERT INTO gg(gg) VALUES('optimize'); } do_execsql_test 5.3 { INSERT INTO gg(gg) VALUES('integrity-check'); } do_test 5.4.1 { set ok 0 for {set i 0} {$i < 10000} {incr i} { set T [format %.5d $i] set res [db eval { SELECT rowid FROM gg($T) ORDER BY rowid ASC }] set res2 [db eval { SELECT rowid FROM gg($T) ORDER BY rowid DESC }] if {$res == [lsort -integer $res2]} { incr ok } | > | 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 | INSERT INTO gg(gg) VALUES('optimize'); } do_execsql_test 5.3 { INSERT INTO gg(gg) VALUES('integrity-check'); } unset -nocomplain res do_test 5.4.1 { set ok 0 for {set i 0} {$i < 10000} {incr i} { set T [format %.5d $i] set res [db eval { SELECT rowid FROM gg($T) ORDER BY rowid ASC }] set res2 [db eval { SELECT rowid FROM gg($T) ORDER BY rowid DESC }] if {$res == [lsort -integer $res2]} { incr ok } |
︙ | ︙ | |||
312 313 314 315 316 317 318 319 320 | } {0 {}} do_catchsql_test 10.5.2 { INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1); } {0 {}} do_catchsql_test 10.5.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } {0 {}} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 | } {0 {}} do_catchsql_test 10.5.2 { INSERT INTO vt0(vt0, rank) VALUES('integrity-check', 1); } {0 {}} do_catchsql_test 10.5.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } {0 {}} reset_db proc slang {in} {return [string map {th d e eh} $in]} db function slang -deterministic -innocuous slang do_execsql_test 11.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT, c TEXT AS (slang(b))); INSERT INTO t1(b) VALUES('the quick fox jumps over the lazy brown dog'); SELECT c FROM t1; } {{deh quick fox jumps ovehr deh lazy brown dog}} do_execsql_test 11.1 { CREATE VIRTUAL TABLE t2 USING fts5(content="t1", c); INSERT INTO t2(t2) VALUES('rebuild'); SELECT rowid FROM t2 WHERE t2 MATCH 'deh'; } {1} do_execsql_test 11.2 { PRAGMA integrity_check(t2); } {ok} db close sqlite3 db test.db # FIX ME? # # FTS5 integrity-check does not care if the content table is unreadable or # does not exist. It only looks for internal inconsistencies in the # inverted index. # do_execsql_test 11.3 { PRAGMA integrity_check(t2); } {ok} do_execsql_test 11.4 { DROP TABLE t1; PRAGMA integrity_check(t2); } {ok} #------------------------------------------------------------------- reset_db do_execsql_test 12.1 { CREATE VIRTUAL TABLE x1 USING fts5(a, b); INSERT INTO x1 VALUES('one', 'two'); INSERT INTO x1 VALUES('three', 'four'); INSERT INTO x1 VALUES('five', 'six'); } do_execsql_test 12.2 { PRAGMA integrity_check } {ok} db close sqlite3 db test.db -readonly 1 explain_i { PRAGMA integrity_check } do_execsql_test 12.3 { PRAGMA integrity_check } {ok} #------------------------------------------------------------------- reset_db do_execsql_test 13.1 { CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=ascii); INSERT INTO t1 VALUES('a b c'), ('d e f'); PRAGMA integrity_check; } {ok} db close sqlite3 db test.db do_catchsql_test 13.2 { PRAGMA integrity_check; } {0 ok} do_execsql_test 13.3 { PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=blah)' WHERE name = 't1'; } db close sqlite3 db test.db breakpoint do_catchsql_test 13.4 { PRAGMA integrity_check; } {1 {SQL logic error}} finish_test |
Added ext/fts5/test/fts5integrity2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | # 2024 September 3 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file contains tests focused on the integrity-check procedure. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5integrity2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 2.0 { CREATE VIRTUAL TABLE t2 USING fts5(a, detail='none'); BEGIN; INSERT INTO t2(rowid, a) VALUES(-1, 'hello world'); INSERT INTO t2(rowid, a) VALUES(9223372036854775807, 'hello world'); COMMIT; } do_execsql_test 2.1 { SELECT rowid FROM t2('hello AND world'); } {-1 9223372036854775807} #------------------------------------------------------------------------- do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, detail='none'); CREATE TABLE r1(r); WITH c(x) AS (VALUES(1) UNION SELECT x<<1 FROM c) INSERT INTO r1(r) SELECT -1-x FROM c; INSERT INTO t1(rowid, a) SELECT r, 'abc' FROM r1; } do_execsql_test 2.1 { PRAGMA integrity_check; } {ok} do_execsql_test 2.2 { SELECT rowid FROM t1('abc') ORDER BY +rowid; } [db eval {SELECT r FROM r1 ORDER BY r}] finish_test |
Changes to ext/fts5/test/fts5interrupt.test.
︙ | ︙ | |||
29 30 31 32 33 34 35 36 37 38 39 40 41 42 | proc progress_handler {args} { incr ::progress_handler_delay -1 if {$::progress_handler_delay<=0} { return 1 } return 0 } foreach {tn sql} { 1 { INSERT INTO t1(rowid, a) VALUES(0, 'z z z z') } 2 { COMMIT } } { set bDone 0 for {set i 1} {$bDone==0} {incr i} { do_test 1.$tn.$i { | > | 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 | proc progress_handler {args} { incr ::progress_handler_delay -1 if {$::progress_handler_delay<=0} { return 1 } return 0 } unset -nocomplain res foreach {tn sql} { 1 { INSERT INTO t1(rowid, a) VALUES(0, 'z z z z') } 2 { COMMIT } } { set bDone 0 for {set i 1} {$bDone==0} {incr i} { do_test 1.$tn.$i { |
︙ | ︙ | |||
60 61 62 63 64 65 66 | } set {} {} } {} } } finish_test | < | 61 62 63 64 65 66 67 | } set {} {} } {} } } finish_test |
Changes to ext/fts5/test/fts5lastrowid.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests of the last_insert_rowid functionality with fts5. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5lastrowid | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests of the last_insert_rowid functionality with fts5. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5lastrowid # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(str); |
︙ | ︙ |
Added ext/fts5/test/fts5limits.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | # 2023 May 16 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5limits return_if_no_fts5 do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(x); } # Default limit for expression depth is 256 # foreach {tn nRepeat op bErr} { 1 200 AND 0 2 200 NOT 0 3 200 OR 0 4 260 AND 0 5 260 NOT 1 6 260 OR 0 } { set L [string repeat "abc " $nRepeat] set Q [join $L " $op "] set res {0 {}} if {$bErr} { set res "1 {fts5 expression tree is too large (maximum depth 256)}" } do_catchsql_test 1.$tn { SELECT * FROM ft($Q) } $res } finish_test |
Added ext/fts5/test/fts5locale.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 | # 2014 Dec 20 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focusing on the built-in fts5 tokenizers. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5locale # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc transform_token {locale token} { switch -- $locale { reverse { set ret "" foreach c [split $token ""] { set ret "$c$ret" } set token $ret } default { # no-op } } set token } proc tcl_create {args} { return "tcl_tokenize" } proc tcl_tokenize {tflags text} { set iToken 1 set bSkip 0 if {[sqlite3_fts5_locale]=="second"} { set bSkip 1 } foreach {w iStart iEnd} [fts5_tokenize_split $text] { incr iToken if {(($iToken) % ($bSkip + 1))} continue set w [transform_token [sqlite3_fts5_locale] $w] sqlite3_fts5_token $w $iStart $iEnd } } #------------------------------------------------------------------------- # Check that queries can have a locale attached to them. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=tcl); INSERT INTO t1 VALUES('abc'); INSERT INTO t1 VALUES('cba'); } {} do_execsql_test 1.1 { SELECT rowid, a FROM t1( fts5_locale('en_US', 'abc') ); } {1 abc} do_execsql_test 1.2 { SELECT rowid, a FROM t1( fts5_locale('reverse', 'abc') ); } {2 cba} #------------------------------------------------------------------------- # Test that the locale= option exists and seems to accept values. And # that fts5_locale() values may only be inserted into an internal-content # table if the locale=1 option was specified. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 2.1 { CREATE VIRTUAL TABLE b1 USING fts5(x, y, locale=1, tokenize=tcl); CREATE VIRTUAL TABLE b2 USING fts5(x, y, locale=0, tokenize=tcl); CREATE VIRTUAL TABLE ttt USING fts5vocab('b1', instance); } do_catchsql_test 2.2.1 { CREATE VIRTUAL TABLE b3 USING fts5(x, y, locale=2); } {1 {malformed locale=... directive}} do_catchsql_test 2.2.2 { CREATE VIRTUAL TABLE b3 USING fts5(x, y, locale=111); } {1 {malformed locale=... directive}} do_catchsql_test 2.3 { INSERT INTO b1(b1, rank) VALUES('locale', 0); } {1 {SQL logic error}} do_execsql_test 2.4.1 { INSERT INTO b1 VALUES('abc', 'one two three'); } do_execsql_test 2.4.2 { INSERT INTO b1 VALUES('def', fts5_locale('reverse', 'four five six')); } do_execsql_test 2.5 { INSERT INTO b2 VALUES('abc', 'one two three'); } do_catchsql_test 2.6 { INSERT INTO b2 VALUES('def', fts5_locale('reverse', 'four five six')); } {1 {fts5_locale() requires locale=1}} do_execsql_test 2.7 { SELECT rowid FROM b1('one') } {1} do_execsql_test 2.8 { SELECT rowid FROM b1('four') } {} do_execsql_test 2.9 { SELECT rowid FROM b1('ruof') } 2 do_execsql_test 2.10 { SELECT rowid FROM b1(fts5_locale('reverse', 'five'))} 2 do_execsql_test 2.11 { SELECT x, quote(y) FROM b1 } { abc {'one two three'} def {'four five six'} } do_execsql_test 2.12 { SELECT quote(y) FROM b1('ruof') } { {'four five six'} } do_execsql_test 2.13 { INSERT INTO b1(b1) VALUES('integrity-check'); } do_execsql_test 2.14 { INSERT INTO b1(b1) VALUES('rebuild'); } do_execsql_test 2.15 { INSERT INTO b1(b1) VALUES('integrity-check'); } do_execsql_test 2.16 { DELETE FROM b1 WHERE rowid=2 } do_execsql_test 2.17 { INSERT INTO b1(b1) VALUES('integrity-check'); } do_execsql_test 2.18 { INSERT INTO b1(rowid, x, y) VALUES( test_setsubtype(45, 76), 'abc def', 'def abc' ); } #------------------------------------------------------------------------- # Test the 'delete' command with contentless tables. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 3.1 { CREATE VIRTUAL TABLE c1 USING fts5(x, content=, tokenize=tcl, locale=1); CREATE VIRTUAL TABLE c2 USING fts5vocab('c1', instance); INSERT INTO c1 VALUES('hello world'); INSERT INTO c1 VALUES( fts5_locale('reverse', 'one two three') ); } do_execsql_test 3.2 { SELECT DISTINCT term FROM c2 ORDER BY 1 } { eerht eno hello owt world } do_execsql_test 3.3 { INSERT INTO c1(c1, rowid, x) VALUES('delete', 2, fts5_locale('reverse', 'one two three') ); } do_execsql_test 3.4 { SELECT DISTINCT term FROM c2 ORDER BY 1 } { hello world } #------------------------------------------------------------------------- # Test that an UPDATE that updates a subset of the columns does not # magically discard the locale from those columns not updated. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 4.1 { CREATE VIRTUAL TABLE d1 USING fts5(x, y, locale=1, tokenize=tcl); CREATE VIRTUAL TABLE d2 USING fts5vocab('d1', instance); INSERT INTO d1(rowid, x, y) VALUES(1, 'abc', 'def'); INSERT INTO d1(rowid, x, y) VALUES(2, 'ghi', fts5_locale('reverse', 'hello')); } do_execsql_test 4.2 { SELECT DISTINCT term FROM d2 ORDER BY 1 } { abc def ghi olleh } do_execsql_test 4.3 { UPDATE d1 SET x='jkl' WHERE rowid=2; } do_execsql_test 4.4 { SELECT DISTINCT term FROM d2 ORDER BY 1 } { abc def jkl olleh } do_execsql_test 4.5 { SELECT rowid, * FROM d1 } { 1 abc def 2 jkl hello } do_execsql_test 4.6 { UPDATE d1 SET rowid=4 WHERE rowid=2 } do_execsql_test 4.7 { SELECT rowid, * FROM d1 } { 1 abc def 4 jkl hello } fts5_aux_test_functions db do_execsql_test 4.8.1 { SELECT fts5_test_columntext(d1) FROM d1('jkl') } {{jkl hello}} do_execsql_test 4.8.2 { SELECT fts5_test_columntext(d1) FROM d1(fts5_locale('reverse', 'hello')) } {{jkl hello}} do_execsql_test 4.9 { SELECT fts5_test_columnlocale(d1) FROM d1(fts5_locale('reverse', 'hello')) } {{{} reverse}} do_execsql_test 4.10 { SELECT fts5_test_columnlocale(d1) FROM d1 } { {{} {}} {{} reverse} } #------------------------------------------------------------------------- # Test that if an fts5_locale() value is written to an UNINDEXED # column it is stored as text. This is so that blobs and other values # can also be stored as is. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 5.1 { CREATE VIRTUAL TABLE t1 USING fts5( x, y UNINDEXED, locale=1, tokenize=tcl ); INSERT INTO t1(rowid, x, y) VALUES(111, fts5_locale('reverse', 'one two three'), fts5_locale('reverse', 'four five six') ); } do_execsql_test 5.2 { SELECT rowid, x, y FROM t1 } { 111 {one two three} {four five six} } do_execsql_test 5.3 { SELECT typeof(c0), typeof(c1), typeof(l0) FROM t1_content } { text text text } #------------------------------------------------------------------------- foreach {tn opt} { 1 {} 2 {, columnsize=0} } { reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 6.$tn.1 " CREATE VIRTUAL TABLE y1 USING fts5(t, locale=1, tokenize=tcl $opt); " do_execsql_test 6.$tn.2 { INSERT INTO y1(rowid, t) VALUES (1, fts5_locale('second', 'the city of London')), (2, fts5_locale('second', 'shall have all the old')), (3, fts5_locale('second', 'Liberties and Customs')), (4, fts5_locale('second', 'which it hath been used to have')); } fts5_aux_test_functions db do_execsql_test 6.$tn.3 { SELECT fts5_test_columnsize(y1) FROM y1 } { 2 3 2 4 } do_execsql_test 6.$tn.4 { SELECT rowid, fts5_test_columnsize(y1) FROM y1('shall'); } { 2 3 } do_execsql_test 6.$tn.5 { SELECT rowid, fts5_test_columnsize(y1) FROM y1('shall'); } { 2 3 } do_execsql_test 6.$tn.6 { SELECT rowid, fts5_test_columnsize(y1) FROM y1('have'); } { 4 4 } do_execsql_test 6.$tn.7 { SELECT rowid, highlight(y1, 0, '[', ']') FROM y1('have'); } { 4 {which it hath been used to [have]} } do_execsql_test 6.$tn.8 { SELECT rowid, highlight(y1, 0, '[', ']'), snippet(y1, 0, '[', ']', '...', 10) FROM y1('Liberties + Customs'); } { 3 {[Liberties and Customs]} {[Liberties and Customs]} } } #------------------------------------------------------------------------- reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); } do_catchsql_test 6.1 { INSERT INTO x1(rowid, x) VALUES(123, fts5_locale('en_AU', 'hello world')); } {1 {fts5_locale() requires locale=1}} do_execsql_test 6.2 { SELECT typeof( fts5_locale(NULL, 'xyz') ), typeof( fts5_locale('', 'abc') ); } {text text} #-------------------------------------------------------------------------- # Test that fts5_locale() works with external-content tables. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 7.1 { CREATE TABLE t1(ii INTEGER PRIMARY KEY, bb BLOB, tt TEXT, locale TEXT); CREATE VIEW v1 AS SELECT ii AS rowid, bb, fts5_locale(locale, tt) AS tt FROM t1; CREATE VIRTUAL TABLE ft USING fts5( bb, tt, locale=1, tokenize=tcl, content=v1 ); INSERT INTO t1 VALUES(1, NULL, 'one two three', NULL); INSERT INTO t1 VALUES(2, '7800616263', 'four five six', 'reverse'); INSERT INTO t1 VALUES(3, '000000007800616263', 'seven eight nine', 'second'); } do_execsql_test 7.2 { INSERT INTO ft(ft) VALUES('rebuild'); INSERT INTO ft(ft) VALUES('integrity-check'); } do_execsql_test 7.3 { SELECT rowid, quote(bb), quote(tt) FROM ft } { 1 NULL {'one two three'} 2 '7800616263' {'four five six'} 3 '000000007800616263' {'seven eight nine'} } do_execsql_test 7.4 { SELECT rowid FROM ft('six'); } do_execsql_test 7.5 { SELECT rowid FROM ft(fts5_locale('reverse','six')); } 2 fts5_aux_test_functions db do_execsql_test 7.6 { SELECT fts5_test_columnlocale(ft) FROM ft; } { {{} {}} {{} reverse} {{} second} } #------------------------------------------------------------------------- # Test that the porter tokenizer works with locales. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 8.1 { CREATE VIRTUAL TABLE ft USING fts5(tt, locale=1, tokenize="porter tcl"); CREATE VIRTUAL TABLE vocab USING fts5vocab('ft', instance); INSERT INTO ft(rowid, tt) VALUES (111, fts5_locale('second', 'the porter tokenizer is a wrapper tokenizer')), (222, fts5_locale('reverse', 'This value may also be set')); } do_execsql_test 8.1 { SELECT DISTINCT term FROM vocab ORDER BY 1 } { a eb eulav osla sihT te the token yam } #------------------------------------------------------------------------- # Test that position-lists (used by xInst, xPhraseFirst etc.) work with # locales and modes other than detail=full. # foreach {tn detail} { 1 detail=full 2 detail=none 3 detail=column } { reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create do_execsql_test 9.$tn.0 " CREATE VIRTUAL TABLE ft USING fts5(tt, locale=1, tokenize=tcl, $detail); " do_execsql_test 9.$tn.1 { CREATE VIRTUAL TABLE vocab USING fts5vocab('ft', instance); INSERT INTO ft(rowid, tt) VALUES (-1, fts5_locale('second', 'it is an ancient mariner')); } do_execsql_test 9.$tn.2 { SELECT DISTINCT term FROM vocab } {an it mariner} do_execsql_test 9.$tn.3 { SELECT highlight(ft, 0, '[', ']') FROM ft('mariner') } {{it is an ancient [mariner]}} } #------------------------------------------------------------------------- # Check some corrupt fts5_locale() blob formats are detected. # foreach_detail_mode $::testprefix { reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create fts5_aux_test_functions db do_execsql_test 10.1 { CREATE TABLE x1(ii INTEGER PRIMARY KEY, x); CREATE VIRTUAL TABLE ft USING fts5(x, content=x1, content_rowid=ii, locale=1, detail=%DETAIL%, columnsize=0 ); CREATE VIRTUAL TABLE ft2 USING fts5( x, locale=1, detail=%DETAIL%, columnsize=0 ); } foreach {tn v} { 1 X'001152' 2 X'0011223344' 3 X'00E0B2EB68656c6c6f' 4 X'00E0B2EB0068656c6c6f' } { do_execsql_test 10.2.$tn.0 { INSERT INTO ft(ft) VALUES('delete-all') } do_execsql_test 10.2.$tn.1 { DELETE FROM x1; } do_execsql_test 10.2.$tn.2 " INSERT INTO x1 VALUES(NULL, $v) " do_catchsql_test 10.2.$tn.3 { INSERT INTO ft(ft) VALUES('rebuild'); } {0 {}} do_catchsql_test 10.2.$tn.4 " SELECT * FROM ft( test_setsubtype($v, 76) ); " {1 {fts5: syntax error near ""}} do_execsql_test 10.2.$tn.5 { INSERT INTO ft(rowid, x) VALUES(1, 'hello world'); } if {"%DETAIL%"=="full"} { do_execsql_test 10.2.$tn.6 { SELECT fts5_test_poslist(ft) FROM ft('world'); } {0.0.1} do_execsql_test 10.2.$tn.7.1 { SELECT fts5_test_columnsize(ft) FROM ft('world'); } {1} do_execsql_test 10.2.$tn.7.2 { SELECT fts5_test_columnlocale(ft) FROM ft('world'); } {{{}}} } do_catchsql_test 10.2.$tn.8 { SELECT count(*) FROM ft('hello') } {0 1} do_catchsql_test 10.2.$tn.9 { PRAGMA integrity_check; } {0 ok} do_execsql_test 10.2.$tn.10 { DELETE FROM x1; INSERT INTO x1(ii, x) VALUES(1, 'hello world'); } do_catchsql_test 10.2.$tn.11 " INSERT INTO ft(ft, rowid, x) VALUES('delete', 1, test_setsubtype($v,76) ) " {0 {}} do_catchsql_test 10.2.$tn.12 " INSERT INTO ft(rowid, x) VALUES(2, test_setsubtype($v,76) ) " {0 {}} do_execsql_test 10.2.$tn.13 { INSERT INTO ft2(rowid, x) VALUES(1, 'hello world'); } do_execsql_test 10.2.$tn.14 "UPDATE ft2_content SET c0=$v" do_catchsql_test 10.2.$tn.15 { PRAGMA integrity_check; } {0 {{malformed inverted index for FTS5 table main.ft2}}} do_execsql_test 10.2.$tn.16 { DELETE FROM ft2_content; INSERT INTO ft2(ft2) VALUES('rebuild'); } } } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create fts5_aux_test_functions db do_execsql_test 11.0 { CREATE VIRTUAL TABLE x1 USING fts5(abc, locale=1); INSERT INTO x1(rowid, abc) VALUES(123, fts5_locale('en_US', 'one two three')); } do_catchsql_test 11.1 { SELECT fts5_columnlocale(x1, -1) FROM x1('two'); } {1 SQLITE_RANGE} do_catchsql_test 11.2 { SELECT fts5_columnlocale(x1, 1) FROM x1('two'); } {1 SQLITE_RANGE} #------------------------------------------------------------------------- # reset_db do_test 12.0 { list [catch { sqlite3_fts5_create_tokenizer -v2 -version 3 db tcl tcl_create } msg] $msg } {1 {error in fts5_api.xCreateTokenizer_v2()}} #------------------------------------------------------------------------- # Tests for auxiliary function fts5_get_locale(). # reset_db # Check that if the table does not support locale=1, fts5_get_locale() # always returns NULL. do_execsql_test 13.1.0 { CREATE VIRTUAL TABLE nolocale USING fts5(a, b); INSERT INTO nolocale VALUES('one two three', 'four five six'); INSERT INTO nolocale VALUES('three two one', 'seven eight nine'); } do_execsql_test 13.1.1 { SELECT fts5_get_locale(nolocale, 0) IS NULL FROM nolocale; } {1 1} do_execsql_test 13.1.2 { SELECT fts5_get_locale(nolocale, 1) IS NULL FROM nolocale('one + two'); } {1} do_execsql_test 13.1.3 { SELECT fts5_get_locale(nolocale, 0) IS NULL FROM nolocale('one AND two'); } {1 1} do_execsql_test 13.1.4 { SELECT fts5_get_locale(nolocale, 1) IS NULL FROM nolocale('three AND two') ORDER BY rank } {1 1} do_catchsql_test 13.1.5 { SELECT fts5_get_locale(nolocale, 2) IS NULL FROM nolocale('three AND two'); } {1 {column index out of range}} do_catchsql_test 13.1.6 { SELECT fts5_get_locale(nolocale, -1) IS NULL FROM nolocale('three AND two'); } {1 {column index out of range}} do_catchsql_test 13.1.7 { SELECT fts5_get_locale(nolocale) IS NULL FROM nolocale('three AND two'); } {1 {wrong number of arguments to function fts5_get_locale()}} do_catchsql_test 13.1.8 { SELECT fts5_get_locale(nolocale, 0, 0) IS NULL FROM nolocale('three AND two'); } {1 {wrong number of arguments to function fts5_get_locale()}} do_catchsql_test 13.1.9 { SELECT fts5_get_locale(nolocale, 'text') FROM nolocale('three AND two'); } {1 {non-integer argument passed to function fts5_get_locale()}} # Check that if the table does support locale=1, fts5_get_locale() # returns the locale of the identified row/column. do_execsql_test 13.2.0 { CREATE VIRTUAL TABLE ft USING fts5(a, b, locale=1); INSERT INTO ft VALUES( fts5_locale('th_TH', 'one two three'), 'four five six seven' ); INSERT INTO ft VALUES( 'three two one', fts5_locale('en_AU', 'seven eight nine') ); } do_execsql_test 13.2.1 { SELECT quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) FROM ft } { 'th_TH' NULL NULL 'en_AU' } do_execsql_test 13.2.2 { SELECT quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) FROM ft('one AND three') } { 'th_TH' NULL NULL 'en_AU' } do_execsql_test 13.2.3 { SELECT quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) FROM ft('one AND three') ORDER BY rank } { NULL 'en_AU' 'th_TH' NULL } do_execsql_test 13.2.4 { SELECT quote(fts5_get_locale(ft, 0)), quote(fts5_get_locale(ft, 1)) FROM ft('one AND three') ORDER BY rowid } { 'th_TH' NULL NULL 'en_AU' } do_execsql_test 13.2.5 { SELECT quote(fts5_get_locale(ft, '0')), quote(fts5_get_locale(ft, 1)) FROM ft('one AND three') ORDER BY rowid } { 'th_TH' NULL NULL 'en_AU' } do_catchsql_test 13.2.6 { SELECT quote(fts5_get_locale(ft, '0.0')), quote(fts5_get_locale(ft, 1)) FROM ft('one AND three') ORDER BY rowid } {1 {non-integer argument passed to function fts5_get_locale()}} do_catchsql_test 13.2.7 { SELECT quote(fts5_get_locale(ft, 0.0)), quote(fts5_get_locale(ft, 1)) FROM ft('one AND three') ORDER BY rowid } {1 {non-integer argument passed to function fts5_get_locale()}} #------------------------------------------------------------------------- # Check that UPDATE statements that may affect more than one row work. # reset_db do_execsql_test 14.1 { CREATE VIRTUAL TABLE ft USING fts5(a, b, locale=1); } do_execsql_test 14.2 { INSERT INTO ft VALUES('hello', 'world'); } do_execsql_test 14.3 { UPDATE ft SET b = fts5_locale('en_AU', 'world'); } do_execsql_test 14.4 { INSERT INTO ft VALUES(X'abcd', X'1234'); } {} do_execsql_test 14.5 { SELECT quote(a), quote(b) FROM ft } {'hello' 'world' X'ABCD' X'1234'} do_execsql_test 14.6 { DELETE FROM ft; INSERT INTO ft VALUES(NULL, 'null'); INSERT INTO ft VALUES(123, 'int'); INSERT INTO ft VALUES(345.0, 'real'); INSERT INTO ft VALUES('abc', 'text'); INSERT INTO ft VALUES(fts5_locale('abc', 'def'), 'text'); SELECT a, typeof(a), b FROM ft } { {} null null 123 integer int 345.0 real real abc text text def text text } do_execsql_test 14.7 { SELECT quote(c0), typeof(c0) FROM ft_content } { NULL null 123 integer 345.0 real 'abc' text 'def' text } #------------------------------------------------------------------------- # Check that inserting UNINDEXED columns between indexed columns of a # locale=1 table does not cause a problem. # reset_db sqlite3_fts5_create_tokenizer -v2 db tcl tcl_create fts5_aux_test_functions db do_execsql_test 15.1 { CREATE VIRTUAL TABLE ft USING fts5(a, b UNINDEXED, c, locale=1, tokenize=tcl); } do_execsql_test 15.2 { INSERT INTO ft VALUES('one', 'two', 'three'); INSERT INTO ft VALUES('one', 'two', fts5_locale('loc', 'three')); } do_execsql_test 15.3 { SELECT c2, l2 FROM ft_content } {three {} three loc} do_execsql_test 15.4 { SELECT c, fts5_columnlocale(ft, 2) FROM ft } {three {} three loc} finish_test |
Changes to ext/fts5/test/fts5matchinfo.test.
︙ | ︙ | |||
513 514 515 516 517 518 519 | SELECT quote(matchinfo(t1, 'n')) FROM t1 LIMIT 1; } $res fts5_aux_test_functions db do_execsql_test 15.3 { SELECT fts5_test_all(t1) FROM t1 LIMIT 1; } { | | > > > > > > > > > > > > > > > > > > > > > > > > | 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 | SELECT quote(matchinfo(t1, 'n')) FROM t1 LIMIT 1; } $res fts5_aux_test_functions db do_execsql_test 15.3 { SELECT fts5_test_all(t1) FROM t1 LIMIT 1; } { {columnsize {1 1} columntext {c d} columntotalsize {2 2} poslist {} tokenize {c d} rowcount 2} } #------------------------------------------------------------------------- # reset_db do_execsql_test 16.0 { CREATE TABLE t1(x); BEGIN EXCLUSIVE; } do_test 16.1 { set rc [catch { sqlite3 db2 test.db db2 eval {SELECT * FROM t1} } errmsg] lappend rc $errmsg } {1 {database is locked}} do_execsql_test 16.2 { ROLLBACK; } do_test 16.3 { catchsql { SELECT * FROM t1 } db2 } {0 {}} finish_test |
Changes to ext/fts5/test/fts5merge.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Test that focus on incremental merges of segments. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5merge | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Test that focus on incremental merges of segments. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5merge # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } db func repeat [list string repeat] |
︙ | ︙ |
Changes to ext/fts5/test/fts5misc.test.
︙ | ︙ | |||
31 32 33 34 35 36 37 | do_catchsql_test 1.1.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*')); } {1 {unknown special query: }} do_catchsql_test 1.2.1 { SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*id'); | | | | | | > > > > > | 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 | do_catchsql_test 1.1.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*')); } {1 {unknown special query: }} do_catchsql_test 1.2.1 { SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*id'); } {1 {no such cursor: 4}} do_catchsql_test 1.2.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*id')); } {1 {no such cursor: 6}} do_catchsql_test 1.3.1 { SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads'); } {1 {no such cursor: 0}} do_catchsql_test 1.3.2 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads')); } {1 {no such cursor: 0}} db close sqlite3 db test.db do_catchsql_test 1.3.3 { SELECT a FROM t1 WHERE rank = (SELECT highlight(t1, 4, '<b>', '</b>') FROM t1('*reads')); } {1 {no such cursor: 0}} fts5_aux_test_functions db do_catchsql_test 1.3.4 { SELECT fts5_columntext(t1) FROM t1('*reads'); } {1 {no such cursor: 0}} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE TABLE t0(c0); CREATE VIRTUAL TABLE vt0 USING fts5(c0); } |
︙ | ︙ | |||
87 88 89 90 91 92 93 | do_execsql_test 2.2.1 { CREATE TABLE t0(c0); CREATE VIRTUAL TABLE vt0 USING fts5(c0); BEGIN TRANSACTION; INSERT INTO vt0(c0) VALUES ('xyz'); } | < | 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | do_execsql_test 2.2.1 { CREATE TABLE t0(c0); CREATE VIRTUAL TABLE vt0 USING fts5(c0); BEGIN TRANSACTION; INSERT INTO vt0(c0) VALUES ('xyz'); } do_execsql_test 2.2.2 { ALTER TABLE t0 RENAME TO t1; } do_execsql_test 2.2.3 { INSERT INTO vt0(vt0) VALUES('integrity-check'); } do_execsql_test 2.2.4 { |
︙ | ︙ | |||
325 326 327 328 329 330 331 | #------------------------------------------------------------------------- # Forum post https://sqlite.org/forum/forumpost/21127c1160 # reset_db sqlite3_db_config db DEFENSIVE 1 | | > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 | #------------------------------------------------------------------------- # Forum post https://sqlite.org/forum/forumpost/21127c1160 # reset_db sqlite3_db_config db DEFENSIVE 1 do_execsql_test 13.1.0 { CREATE TABLE a (id INTEGER PRIMARY KEY, name TEXT); CREATE VIRTUAL TABLE b USING fts5(name); CREATE TRIGGER a_trigger AFTER INSERT ON a BEGIN INSERT INTO b (name) VALUES ('foo'); END; } do_test 13.1.1 { set ::STMT [ sqlite3_prepare db "INSERT INTO a VALUES (1, 'foo') RETURNING id;" -1 dummy ] sqlite3_step $::STMT } {SQLITE_ROW} do_test 13.1.2 { sqlite3_finalize $::STMT } {SQLITE_OK} do_test 13.1.3 { sqlite3_errmsg db } {not an error} reset_db sqlite3_db_config db DEFENSIVE 1 do_execsql_test 13.2.0 { BEGIN; CREATE TABLE a (id INTEGER PRIMARY KEY, name TEXT); CREATE VIRTUAL TABLE b USING fts5(name); CREATE TRIGGER a_trigger AFTER INSERT ON a BEGIN INSERT INTO b (name) VALUES ('foo'); END; } do_test 13.2.1 { set ::STMT [ sqlite3_prepare db "INSERT INTO a VALUES (1, 'foo') RETURNING id;" -1 dummy ] sqlite3_step $::STMT } {SQLITE_ROW} do_test 13.2.2 { sqlite3_finalize $::STMT } {SQLITE_OK} do_test 13.2.3 { sqlite3_errmsg db } {not an error} #------------------------------------------------------------------------- reset_db db close sqlite3 db test.db -uri 1 |
︙ | ︙ | |||
394 395 396 397 398 399 400 | do_test 15.2 { list [catch { db2 eval COMMIT } msg] $msg } {1 {database is locked}} do_execsql_test -db db2 15.3 { SAVEPOINT one; } {} do_execsql_test 15.4 END | | > > | 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 | do_test 15.2 { list [catch { db2 eval COMMIT } msg] $msg } {1 {database is locked}} do_execsql_test -db db2 15.3 { SAVEPOINT one; } {} do_execsql_test 15.4 END do_test 15.5 { list [catch { db2 eval COMMIT } msg] $msg } {0 {}} db2 close #------------------------------------------------------------------------- reset_db forcedelete test.db2 sqlite3 db2 test.db do_execsql_test 16.0 { |
︙ | ︙ | |||
438 439 440 441 442 443 444 445 446 447 | do_execsql_test 16.5 { COMMIT } do_execsql_test -db db2 16.6 { SELECT * FROM x1 } {abc def} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 | do_execsql_test 16.5 { COMMIT } do_execsql_test -db db2 16.6 { SELECT * FROM x1 } {abc def} db2 close #------------------------------------------------------------------------- reset_db do_execsql_test 17.1 { CREATE VIRTUAL TABLE ft USING fts5(x, tokenize="unicode61 separators 'X'"); } do_execsql_test 17.2 { SELECT 0 FROM ft WHERE ft MATCH 'X' AND ft MATCH 'X' } do_execsql_test 17.3 { SELECT 0 FROM ft('X') } do_execsql_test 17.4 { CREATE VIRTUAL TABLE t0 USING fts5(c0, t="trigram"); INSERT INTO t0 VALUES('assertionfaultproblem'); } do_execsql_test 17.5 { SELECT 0 FROM t0(0) WHERE c0 GLOB 0; } {} do_execsql_test 17.5 { SELECT c0 FROM t0 WHERE c0 GLOB '*f*'; } {assertionfaultproblem} do_execsql_test 17.5 { SELECT c0 FROM t0 WHERE c0 GLOB '*faul*'; } {assertionfaultproblem} #------------------------------------------------------------------------- reset_db do_execsql_test 18.0 { BEGIN; CREATE VIRTUAL TABLE t1 USING fts5(text); ALTER TABLE t1 RENAME TO t2; } do_execsql_test 18.1 { DROP TABLE t2; } do_execsql_test 18.2 { COMMIT; } #------------------------------------------------------------------------- reset_db do_execsql_test 19.0 { CREATE VIRTUAL TABLE t1 USING fts5(text); CREATE TABLE t2(text); BEGIN; INSERT INTO t1 VALUES('one'); INSERT INTO t1 VALUES('two'); INSERT INTO t1 VALUES('three'); INSERT INTO t1 VALUES('one'); INSERT INTO t1 VALUES('two'); INSERT INTO t1 VALUES('three'); SAVEPOINT one; INSERT INTO t2 VALUES('one'); INSERT INTO t2 VALUES('two'); INSERT INTO t2 VALUES('three'); ROLLBACK TO one; COMMIT; } #------------------------------------------------------------------------- reset_db do_execsql_test 20.0 { CREATE VIRTUAL TABLE x1 USING fts5(a); INSERT INTO x1(rowid, a) VALUES (1, 'a b c d'), (2, 'x b c d'), (3, 'x y z d'), (4, 'a y c x'); } do_execsql_test 20.1 { SELECT rowid FROM x1 WHERE x1 MATCH 'a' AND x1 MATCH 'b'; } {1} do_execsql_test 20.2 { SELECT rowid FROM x1 WHERE x1 MATCH 'a' AND x1 MATCH 'y'; } {4} do_execsql_test 20.3 { SELECT rowid FROM x1 WHERE x1 MATCH 'a' OR x1 MATCH 'y'; } {1 4 3} do_execsql_test 20.4 { SELECT rowid FROM x1 WHERE x1 MATCH 'a' OR (x1 MATCH 'y' AND x1 MATCH 'd'); } {1 4 3} do_execsql_test 20.5 { SELECT rowid FROM x1 WHERE x1 MATCH 'z' OR (x1 MATCH 'a' AND x1 MATCH 'd'); } {3 1} #------------------------------------------------------------------------- reset_db do_execsql_test 21.0 { CREATE TABLE t1(ii INTEGER, x TEXT, y TEXT); CREATE VIRTUAL TABLE xyz USING fts5(content_rowid=ii, content=t1, x, y); INSERT INTO t1 VALUES(1, 'one', 'i'); INSERT INTO t1 VALUES(2, 'two', 'ii'); INSERT INTO t1 VALUES(3, 'tree', 'iii'); INSERT INTO xyz(xyz) VALUES('rebuild'); } do_execsql_test 21.1 { UPDATE xyz SET y='TWO' WHERE rowid=2; UPDATE t1 SET y='TWO' WHERE ii=2; } do_execsql_test 21.2 { PRAGMA integrity_check } {ok} sqlite3_db_config db DEFENSIVE 1 do_execsql_test 21.3 { CREATE TABLE xyz_notashadow(x, y); DROP TABLE xyz_notashadow; } sqlite3_db_config db DEFENSIVE 0 #------------------------------------------------------------------------- reset_db do_execsql_test 22.0 { SELECT fts5(NULL); } {{}} do_execsql_test 22.1 { SELECT count(*) FROM ( SELECT fts5_source_id() ) } {1} execsql_pp { SELECT fts5_source_id() } #------------------------------------------------------------------------- reset_db do_execsql_test 23.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); INSERT INTO x1 VALUES('one + two + three'); INSERT INTO x1 VALUES('one + xyz + three'); INSERT INTO x1 VALUES('xyz + two + xyz'); } do_execsql_test 23.1 { SELECT rowid FROM x1('one + two + three'); } {1} do_execsql_test 23.2 { SELECT rowid FROM x1('^".." AND one'); } {} do_execsql_test 23.3 { SELECT rowid FROM x1('abc NEAR ".." NEAR def'); } {} #------------------------------------------------------------------------- reset_db do_execsql_test 24.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, detail='none'); INSERT INTO t1(a) VALUES('a'); } do_execsql_test 24.2 { SELECT rank FROM ( SELECT rank FROM t1('a NOT "" NOT def') ) ORDER BY 1; } {-1e-06} do_execsql_test 24.3 { SELECT rank FROM ( SELECT rank FROM t1('a NOT � NOT def') ) ORDER BY 1; } {-1e-06} do_execsql_test 24.4 { SELECT rank FROM ( SELECT rank FROM t1('a NOT "" NOT def') ); } {-1e-06} #------------------------------------------------------------------------- reset_db fts5_aux_test_functions db do_execsql_test 25.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, detail='none', content=''); INSERT INTO t1(a) VALUES('a b c'); } do_execsql_test 25.0 { SELECT fts5_test_poslist(t1) FROM t1('b') ORDER BY rank; } {{}} #------------------------------------------------------------------------- reset_db do_execsql_test 26.0 { PRAGMA foreign_keys = ON; CREATE TABLE t1(x INTEGER PRIMARY KEY); CREATE TABLE t2(y INTEGER PRIMARY KEY, z INTEGER REFERENCES t1(x) DEFERRABLE INITIALLY DEFERRED ); CREATE VIRTUAL TABLE t3 USING fts5(a, b, content='', tokendata=1); } do_execsql_test 26.1 { BEGIN; INSERT INTO t2 VALUES(1,111); INSERT INTO t3 VALUES(3,3); PRAGMA defer_foreign_keys=ON; DELETE FROM t2 WHERE y+1; COMMIT; } finish_test |
Changes to ext/fts5/test/fts5near.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focused on the NEAR operator. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5near | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focused on the NEAR operator. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5near # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc do_near_test {tn doc near res} { uplevel [list do_execsql_test $tn " |
︙ | ︙ |
Changes to ext/fts5/test/fts5optimize.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5optimize | | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5optimize # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } # # 1.* - Warm body tests for index optimization using ('optimize') |
︙ | ︙ |
Added ext/fts5/test/fts5optimize2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | # 2014 Dec 20 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # TESTRUNNER: superslow # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5optimize2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set nLoop 2500 do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); } do_test 1.1 { for {set ii 0} {$ii < $nLoop} {incr ii} { execsql { INSERT INTO t1 VALUES('abc def ghi'); INSERT INTO t1 VALUES('jkl mno pqr'); INSERT INTO t1(t1) VALUES('optimize'); } } } {} do_execsql_test 1.2 { SELECT count(*) FROM t1('mno') } $nLoop finish_test |
Added ext/fts5/test/fts5optimize3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 | # 2023 Aug 27 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # TESTRUNNER: superslow # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5optimize2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set nLoop 2500 do_execsql_test 1.0 { CREATE VIRTUAL TABLE t2 USING fts5(x); INSERT INTO t2(t2, rank) VALUES('pgsz', 32); } do_test 1.1 { for {set ii 0} {$ii < $nLoop} {incr ii} { execsql { INSERT INTO t2 VALUES('abc def ghi'); INSERT INTO t2 VALUES('jkl mno pqr'); INSERT INTO t2(t2, rank) VALUES('merge', -1); } } } {} do_execsql_test 1.2 { SELECT count(*) FROM t2('mno') } $nLoop finish_test |
Added ext/fts5/test/fts5origintext.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 | # 2014 Jan 08 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5origintext # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { foreach {tn insttoken} { 1 0 2 1 } { reset_db sqlite3_fts5_register_origintext db do_execsql_test $tn.1.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", detail=%DETAIL% ); INSERT INTO ft(ft, rank) VALUES('insttoken', $insttoken); CREATE VIRTUAL TABLE vocab USING fts5vocab(ft, instance); } do_execsql_test $tn.1.1 { INSERT INTO ft VALUES('Hello world'); } do_execsql_test $tn.1.2 { INSERT INTO ft(ft) VALUES('integrity-check'); } proc b {x} { string map [list "\0" "."] $x } db func b b do_execsql_test $tn.1.3 { select b(term) from vocab; } { hello.Hello world } do_execsql_test $tn.1.4 { SELECT rowid FROM ft('Hello'); } {1} #------------------------------------------------------------------------- reset_db # Return a random integer between 0 and n-1. # proc random {n} { expr {abs(int(rand()*$n))} } proc select_one {list} { set n [llength $list] lindex $list [random $n] } proc term {} { set first_letter { a b c d e f g h i j k l m n o p q r s t u v w x y z A B C D E F G H I J K L M N O P Q R S T U V W X Y Z } set term [select_one $first_letter] append term [random 100] } proc document {} { set nTerm [expr [random 5] + 5] set doc "" for {set ii 0} {$ii < $nTerm} {incr ii} { lappend doc [term] } set doc } db func document document sqlite3_fts5_register_origintext db do_execsql_test $tn.2.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", detail=%DETAIL% ); INSERT INTO ft(ft, rank) VALUES('insttoken', $insttoken); INSERT INTO ft(ft, rank) VALUES('pgsz', 128); CREATE VIRTUAL TABLE vocab USING fts5vocab(ft, instance); } do_test $tn.2.1 { for {set ii 0} {$ii < 500} {incr ii} { execsql { INSERT INTO ft VALUES( document() ) } } } {} do_execsql_test $tn.2.2 { INSERT INTO ft(ft) VALUES('integrity-check'); } do_execsql_test $tn.2.3 { INSERT INTO ft(ft, rank) VALUES('merge', 16); } do_execsql_test $tn.2.4 { INSERT INTO ft(ft) VALUES('integrity-check'); } do_execsql_test $tn.2.5 { INSERT INTO ft(ft) VALUES('optimize'); } #------------------------------------------------------------------------- reset_db sqlite3_fts5_register_origintext db do_execsql_test $tn.3.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", detail=%DETAIL% ); INSERT INTO ft(ft, rank) VALUES('insttoken', $insttoken); CREATE VIRTUAL TABLE vocab USING fts5vocab(ft, instance); INSERT INTO ft(rowid, x) VALUES(1, 'hello'); INSERT INTO ft(rowid, x) VALUES(2, 'Hello'); INSERT INTO ft(rowid, x) VALUES(3, 'HELLO'); } #proc b {x} { string map [list "\0" "."] $x } #db func b b #execsql_pp { SELECT b(term) FROM vocab } do_execsql_test $tn.3.1.1 { SELECT rowid FROM ft('hello') } 1 do_execsql_test $tn.3.1.2 { SELECT rowid FROM ft('Hello') } 2 do_execsql_test $tn.3.1.3 { SELECT rowid FROM ft('HELLO') } 3 do_execsql_test $tn.3.2 { CREATE VIRTUAL TABLE ft2 USING fts5(x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL% ); INSERT INTO ft2(ft2, rank) VALUES('insttoken', $insttoken); CREATE VIRTUAL TABLE vocab2 USING fts5vocab(ft2, instance); INSERT INTO ft2(rowid, x) VALUES(1, 'hello'); INSERT INTO ft2(rowid, x) VALUES(2, 'Hello'); INSERT INTO ft2(rowid, x) VALUES(3, 'HELLO'); INSERT INTO ft2(rowid, x) VALUES(10, 'helloooo'); } #proc b {x} { string map [list "\0" "."] $x } #db func b b #execsql_pp { SELECT b(term) FROM vocab } do_execsql_test $tn.3.3.1 { SELECT rowid FROM ft2('hello') } {1 2 3} do_execsql_test $tn.3.3.2 { SELECT rowid FROM ft2('Hello') } {1 2 3} do_execsql_test $tn.3.3.3 { SELECT rowid FROM ft2('HELLO') } {1 2 3} do_execsql_test $tn.3.3.4 { SELECT rowid FROM ft2('hello*') } {1 2 3 10} do_execsql_test $tn.3.3.5.1 { SELECT rowid FROM ft2('HELLO') ORDER BY rowid DESC} { 3 2 1 } do_execsql_test $tn.3.3.5.2 { SELECT rowid FROM ft2('HELLO') ORDER BY +rowid DESC} { 3 2 1 } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db proc querytoken {cmd iPhrase iToken} { set txt [$cmd xQueryToken $iPhrase $iToken] string map [list "\0" "."] $txt } sqlite3_fts5_create_function db querytoken querytoken do_execsql_test $tn.4.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize='origintext unicode61', tokendata=1, detail=%DETAIL% ); INSERT INTO ft(ft, rank) VALUES('insttoken', $insttoken); INSERT INTO ft VALUES('one two three four'); } do_execsql_test $tn.4.1 { SELECT rowid, querytoken(ft, 0, 0) FROM ft('TwO') } {1 two.TwO} do_execsql_test $tn.4.2 { SELECT rowid, querytoken(ft, 0, 0) FROM ft('one TWO ThreE') } {1 one} do_execsql_test $tn.4.3 { SELECT rowid, querytoken(ft, 1, 0) FROM ft('one TWO ThreE') } {1 two.TWO} if {"%DETAIL%"=="full"} { # Phrase queries are only supported for detail=full. # do_execsql_test $tn.4.4 { SELECT rowid, querytoken(ft, 0, 2) FROM ft('"one TWO ThreE"') } {1 three.ThreE} do_catchsql_test $tn.4.5 { SELECT rowid, querytoken(ft, 0, 3) FROM ft('"one TWO ThreE"') } {1 SQLITE_RANGE} do_catchsql_test $tn.4.6 { SELECT rowid, querytoken(ft, 1, 0) FROM ft('"one TWO ThreE"') } {1 SQLITE_RANGE} do_catchsql_test $tn.4.7 { SELECT rowid, querytoken(ft, -1, 0) FROM ft('"one TWO ThreE"') } {1 SQLITE_RANGE} } #------------------------------------------------------------------------- # reset_db sqlite3_fts5_register_origintext db proc insttoken {cmd iIdx iToken} { set txt [$cmd xInstToken $iIdx $iToken] string map [list "\0" "."] $txt } sqlite3_fts5_create_function db insttoken insttoken fts5_aux_test_functions db do_execsql_test $tn.5.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize='origintext unicode61', tokendata=1, detail=%DETAIL% ); INSERT INTO ft(ft, rank) VALUES('insttoken', $insttoken); INSERT INTO ft VALUES('one ONE One oNe oNE one'); } do_execsql_test $tn.5.1 { SELECT insttoken(ft, 0, 0), insttoken(ft, 1, 0), insttoken(ft, 2, 0), insttoken(ft, 3, 0), insttoken(ft, 4, 0), insttoken(ft, 5, 0) FROM ft('one'); } { one one.ONE one.One one.oNe one.oNE one } do_execsql_test $tn.5.2 { SELECT insttoken(ft, 0, 0), insttoken(ft, 1, 0), insttoken(ft, 2, 0), insttoken(ft, 3, 0), insttoken(ft, 4, 0), insttoken(ft, 5, 0) FROM ft('on*'); } { one one.ONE one.One one.oNe one.oNE one } do_execsql_test $tn.5.3 { SELECT insttoken(ft, 0, 0), insttoken(ft, 1, 0), insttoken(ft, 2, 0), insttoken(ft, 3, 0), insttoken(ft, 4, 0), insttoken(ft, 5, 0) FROM ft(fts5_insttoken('on*')); } { one one.ONE one.One one.oNe one.oNE one } do_execsql_test $tn.5.4 { SELECT insttoken(ft, 1, 0) FROM ft('one'); } { one.ONE } do_execsql_test $tn.5.5 { SELECT fts5_test_poslist(ft) FROM ft('one'); } { {0.0.0 0.0.1 0.0.2 0.0.3 0.0.4 0.0.5} } #------------------------------------------------------------------------- # Test the xInstToken() API with: # # * a non tokendata=1 table. # * prefix queries. # reset_db sqlite3_fts5_register_origintext db do_execsql_test $tn.6.0 { CREATE VIRTUAL TABLE ft USING fts5( x, y, tokenize='origintext unicode61', detail=%DETAIL%, tokendata=0 ); INSERT INTO ft(ft, rank) VALUES('insttoken', $insttoken); INSERT INTO ft VALUES('One Two', 'Three two'); INSERT INTO ft VALUES('three Three', 'one One'); } proc tokens {cmd} { set ret [list] for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} { set txt [$cmd xInstToken $iTok 0] set txt [string map [list "\0" "."] $txt] lappend ret $txt } set ret } sqlite3_fts5_create_function db tokens tokens do_execsql_test $tn.6.1 { SELECT rowid, tokens(ft) FROM ft('One'); } {1 one.One 2 one.One} do_execsql_test $tn.6.2 { SELECT rowid, tokens(ft) FROM ft('on*'); } {1 one.One 2 {one one.One}} do_execsql_test $tn.6.3 { SELECT rowid, tokens(ft) FROM ft('Three*'); } {1 three.Three 2 three.Three} fts5_aux_test_functions db do_catchsql_test $tn.6.4 { SELECT fts5_test_insttoken(ft, -1, 0) FROM ft('one'); } {1 SQLITE_RANGE} do_catchsql_test $tn.6.5 { SELECT fts5_test_insttoken(ft, 1, 0) FROM ft('one'); } {1 SQLITE_RANGE} do_catchsql_test $tn.6.6 { CREATE VIRTUAL TABLE ft2 USING fts5(x, tokendata=2); } {1 {malformed tokendata=... directive}} do_catchsql_test $tn.6.7 { CREATE VIRTUAL TABLE ft2 USING fts5(x, content='', tokendata=11); } {1 {malformed tokendata=... directive}} } } finish_test |
Added ext/fts5/test/fts5origintext2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | # 2014 Jan 08 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5origintext2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_register_origintext db do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", tokendata=1 ); } do_execsql_test 1.1 { BEGIN; INSERT INTO ft VALUES('Hello'); INSERT INTO ft VALUES('hello'); INSERT INTO ft VALUES('HELLO'); INSERT INTO ft VALUES('today'); INSERT INTO ft VALUES('today'); INSERT INTO ft VALUES('today'); INSERT INTO ft VALUES('World'); INSERT INTO ft VALUES('world'); INSERT INTO ft VALUES('WORLD'); COMMIT; } do_execsql_test 1.2 { SELECT rowid FROM ft('hello'); } {1 2 3} do_execsql_test 1.3 { SELECT rowid FROM ft('today'); } {4 5 6} do_execsql_test 1.4 { SELECT rowid FROM ft('world'); } {7 8 9} do_execsql_test 1.5 { SELECT count(*) FROM ft_data } 3 do_execsql_test 1.6 { DELETE FROM ft; INSERT INTO ft(ft, rank) VALUES('pgsz', 64); BEGIN; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO ft SELECT 'Hello Hello Hello Hello Hello Hello Hello' FROM s; INSERT INTO ft VALUES ('hELLO hELLO hELLO'); INSERT INTO ft VALUES('today today today today today today today'); INSERT INTO ft VALUES('today today today today today today today'); INSERT INTO ft VALUES('today today today today today today today'); INSERT INTO ft VALUES('today today today today today today today'); INSERT INTO ft VALUES('today today today today today today today'); INSERT INTO ft VALUES('today today today today today today today'); INSERT INTO ft VALUES('World World World World World World World'); INSERT INTO ft VALUES('world world world world world world world'); INSERT INTO ft VALUES('WORLD WORLD WORLD WORLD WORLD WORLD WORLD'); INSERT INTO ft VALUES('World World World World World World World'); INSERT INTO ft VALUES('world world world world world world world'); INSERT INTO ft VALUES('WORLD WORLD WORLD WORLD WORLD WORLD WORLD'); COMMIT; } do_execsql_test 1.7 { SELECT count(*) FROM ft_data; } 23 do_execsql_test 1.8 { SELECT rowid FROM ft('hello') WHERE rowid>100; } {101} do_execsql_test 1.9 { DELETE FROM ft; INSERT INTO ft(ft) VALUES('optimize'); SELECT count(*) FROM ft_data; } {2} do_execsql_test 1.10 { BEGIN; INSERT INTO ft VALUES('Hello'); INSERT INTO ft VALUES('hello'); INSERT INTO ft VALUES('HELLO'); INSERT INTO ft VALUES('today'); INSERT INTO ft VALUES('today'); INSERT INTO ft VALUES('today'); INSERT INTO ft VALUES('World'); INSERT INTO ft VALUES('world'); INSERT INTO ft VALUES('WORLD'); } do_execsql_test 1.11 { SELECT rowid FROM ft('hello'); } {1 2 3} do_execsql_test 1.12 { SELECT rowid FROM ft('today'); } {4 5 6} do_execsql_test 1.13 { SELECT rowid FROM ft('world'); } {7 8 9} do_execsql_test 1.14 { SELECT rowid FROM ft('hello') ORDER BY rank; } {1 2 3} #------------------------------------------------------------------------ reset_db sqlite3_fts5_register_origintext db proc tokens {cmd} { set ret [list] for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} { set txt [$cmd xInstToken $iTok 0] set txt [string map [list "\0" "."] $txt] lappend ret $txt } set ret } sqlite3_fts5_create_function db tokens tokens do_execsql_test 2.0 { CREATE VIRTUAL TABLE x1 USING fts5( v, tokenize="origintext unicode61", tokendata=1, detail=none ); INSERT INTO x1 VALUES('xxx Xxx XXX yyy YYY yyy'); INSERT INTO x1 VALUES('xxx yyy xxx yyy yyy yyy'); } do_execsql_test 2.1 { SELECT tokens(x1) FROM x1('xxx'); } { {xxx xxx.Xxx xxx.XXX} {xxx xxx} } do_execsql_test 2.2 { UPDATE x1_content SET c0 = 'xxx xxX xxx yyy yyy yyy' WHERE id=1; } do_execsql_test 2.3 { SELECT tokens(x1) FROM x1('xxx'); } { {xxx {} xxx} {xxx xxx} } finish_test |
Added ext/fts5/test/fts5origintext3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 | # 2023 November 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5origintext3 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { foreach {tn insttoken} { 1 0 2 1 } { reset_db sqlite3_fts5_register_origintext db fts5_aux_test_functions db proc insttoken {cmd iIdx iToken} { set txt [$cmd xInstToken $iIdx $iToken] string map [list "\0" "."] $txt } sqlite3_fts5_create_function db insttoken insttoken do_execsql_test $tn.1.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL% ); } do_execsql_test $tn.1.0.1 { INSERT INTO ft(ft, rank) VALUES('insttoken', 1); } do_execsql_test $tn.1.1 { INSERT INTO ft VALUES('Hello world HELLO WORLD hello'); } do_execsql_test $tn.1.2 { SELECT fts5_test_poslist(ft) FROM ft('hello'); } {{0.0.0 0.0.2 0.0.4}} do_execsql_test $tn.1.3 { SELECT insttoken(ft, 0, 0), insttoken(ft, 1, 0), insttoken(ft, 2, 0) FROM ft('hello'); } {hello.Hello hello.HELLO hello} do_execsql_test $tn.1.3.1 { SELECT insttoken(ft, 0, 0), insttoken(ft, 1, 0), insttoken(ft, 2, 0) FROM ft('hel*'); } {hello.Hello hello.HELLO hello} do_execsql_test $tn.1.4 { SELECT insttoken(ft, 0, 0), insttoken(ft, 1, 0), insttoken(ft, 2, 0) FROM ft('hello') ORDER BY rank; } {hello.Hello hello.HELLO hello} do_execsql_test $tn.1.5 { CREATE VIRTUAL TABLE ft2 USING fts5( x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL% ); INSERT INTO ft2(rowid, x) VALUES(1, 'ONE one two three ONE'); INSERT INTO ft2(rowid, x) VALUES(2, 'TWO one two three TWO'); INSERT INTO ft2(rowid, x) VALUES(3, 'THREE one two three THREE'); } do_execsql_test $tn.1.6 { SELECT insttoken(ft2, 0, 0), rowid FROM ft2('three') ORDER BY rank; } {three.THREE 3 three 1 three 2} do_execsql_test $tn.1.7 { INSERT INTO ft2(rowid, x) VALUES(10, 'aaa bbb BBB'); INSERT INTO ft2(rowid, x) VALUES(12, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(13, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(14, 'bbb BBB bbb'); INSERT INTO ft2(rowid, x) VALUES(15, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(16, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(17, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(18, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(19, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(20, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(21, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(22, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(23, 'bbb bbb bbb'); INSERT INTO ft2(rowid, x) VALUES(24, 'aaa bbb BBB'); } do_execsql_test $tn.1.8 { SELECT rowid FROM ft2('aaa AND bbb'); } {10 24} do_execsql_test $tn.1.9 { SELECT rowid FROM ft2('bbb AND aaa'); } {10 24} do_execsql_test $tn.2.0 { CREATE VIRTUAL TABLE ft3 USING fts5( x, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%, prefix=2 ); } do_execsql_test $tn.2.1 { INSERT INTO ft3(rowid, x) VALUES(1, 'one'); INSERT INTO ft3(rowid, x) VALUES(2, 'ONE'); INSERT INTO ft3(rowid, x) VALUES(3, 'ONT'); INSERT INTO ft3(rowid, x) VALUES(4, 'on'); INSERT INTO ft3(rowid, x) VALUES(5, 'On'); } do_execsql_test $tn.2.2 { SELECT rowid FROM ft3('on*'); } {1 2 3 4 5} do_execsql_test $tn.2.3 { SELECT rowid, insttoken(ft3, 0, 0) FROM ft3('on*'); } {1 one 2 one.ONE 3 ont.ONT 4 on 5 on.On} } } finish_test |
Added ext/fts5/test/fts5origintext4.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 | # 2023 November 22 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5origintext4 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } # The tests below verify that a doclist-index is used to limit the number # of pages loaded into the cache. It does this by querying sqlite3_db_status() # for the amount of memory used by the pager cache. # # memsubsys1 effectively limits the page-cache to 24 pages. Which masks # the effect tested by the tests in this file. And "mmap" prevents the # cache from being used, also preventing these tests from working. # if {[permutation]=="memsubsys1" || [permutation]=="mmap"} { finish_test return } sqlite3_fts5_register_origintext db do_execsql_test 1.0 { PRAGMA page_size = 4096; CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", tokendata=1 ); } do_execsql_test 1.1 { BEGIN; INSERT INTO ft SELECT 'the first thing'; WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<90000 ) INSERT INTO ft SELECT 'The second thing' FROM s; INSERT INTO ft SELECT 'the first thing'; COMMIT; INSERT INTO ft(ft) VALUES('optimize'); } foreach {tn sql expr} { 1 { SELECT rowid FROM ft('the') } {$mem > 250000} 2 { SELECT rowid FROM ft('first') } {$mem < 50000} 3 { SELECT rowid FROM ft('the first') } {$mem < 50000} } { db close sqlite3 db test.db sqlite3_fts5_register_origintext db execsql $sql do_test 1.2.$tn { set mem [lindex [sqlite3_db_status db CACHE_USED 0] 1] expr $expr } 1 } proc b {x} { string map [list "\0" "."] $x } db func b b # execsql_pp { SELECT segid, b(term), pgno from ft_idx } finish_test |
Added ext/fts5/test/fts5origintext5.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 | # 2023 Dec 04 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests for tables that use both tokendata=1 and contentless_delete=1. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5origintext # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } # Return a random integer between 0 and n-1. # proc random {n} { expr {abs(int(rand()*$n))} } # Select an element of the list passed as the only argument at random and # return it. # proc select_one {list} { set n [llength $list] lindex $list [random $n] } # Given a term that consists entirely of alphabet characters, return all # permutations of the term using upper and lower case characters. e.g. # # "abc" -> {CBA cBA CbA cbA CBa cBa Cba cba} # proc casify {term {lRet {{}}}} { if {$term==""} { return $lRet } set t [string range $term 1 end] set f1 [string toupper [string range $term 0 0]] set f2 [string tolower [string range $term 0 0]] set ret [list] foreach x $lRet { lappend ret "$x$f1" lappend ret "$x$f2" } return [casify $t $ret] } proc vocab {} { list abc def ghi jkl mno pqr stu vwx yza } # Return a random 3 letter term. # proc term {} { if {[info exists ::expanded_vocab]==0} { foreach v [vocab] { lappend ::expanded_vocab {*}[casify $v] } } select_one $::expanded_vocab } # Return a document - between 3 and 10 terms. # proc document {} { set nTerm [expr [random 3] + 7] set doc "" for {set ii 0} {$ii < $nTerm} {incr ii} { lappend doc [term] } set doc } db func document document #------------------------------------------------------------------------- expr srand(6) set NDOC 200 set NLOOP 50 sqlite3_fts5_register_origintext db proc tokens {cmd} { set ret [list] for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} { set txt [$cmd xInstToken $iTok 0] set txt [string map [list "\0" "."] $txt] lappend ret $txt } set ret } sqlite3_fts5_create_function db tokens tokens proc rankfunc {cmd} { $cmd xRowid } sqlite3_fts5_create_function db rankfunc rankfunc proc ctrl_tokens {term args} { set ret [list] set term [string tolower $term] foreach doc $args { foreach a $doc { if {[string tolower $a]==$term} { if {$a==$term} { lappend ret $a } else { lappend ret [string tolower $a].$a } } } } set ret } db func ctrl_tokens ctrl_tokens proc do_all_vocab_test {tn} { foreach ::v [concat [vocab] nnn] { set answer [execsql { SELECT id, ctrl_tokens($::v, x) FROM ctrl WHERE x LIKE '%' || $::v || '%' }] do_execsql_test $tn.$::v.1 { SELECT rowid, tokens(ft) FROM ft($::v) } $answer do_execsql_test $tn.$::v.2 { SELECT rowid, tokens(ft) FROM ft($::v) ORDER BY rank } $answer } } do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5( x, tokenize="origintext unicode61", content=, contentless_delete=1, tokendata=1 ); CREATE TABLE ctrl(id INTEGER PRIMARY KEY, x TEXT); INSERT INTO ft(ft, rank) VALUES('pgsz', 64); INSERT INTO ft(ft, rank) VALUES('rank', 'rankfunc()'); } do_test 1.1 { for {set ii 0} {$ii < $NDOC} {incr ii} { set doc [document] execsql { INSERT INTO ft(rowid, x) VALUES($ii, $doc); INSERT INTO ctrl(id, x) VALUES($ii, $doc); } } } {} #execsql_pp { SELECT * FROM ctrl } #execsql_pp { SELECT * FROM ft } #fts5_aux_test_functions db #execsql_pp { SELECT rowid, tokens(ft), fts5_test_poslist(ft) FROM ft('ghi'); } do_all_vocab_test 1.2 for {set ii 0} {$ii < $NLOOP} {incr ii} { set lRowid [execsql { SELECT id FROM ctrl WHERE random() % 2 }] foreach r $lRowid { execsql { DELETE FROM ft WHERE rowid = $r } execsql { DELETE FROM ctrl WHERE rowid = $r } set doc [document] execsql { INSERT INTO ft(rowid, x) VALUES($r, $doc) } execsql { INSERT INTO ctrl(id, x) VALUES($r, $doc) } } do_all_vocab_test 1.3.$ii } #------------------------------------------------------------------------- do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft2 USING fts5( x, y, tokenize="origintext unicode61", content=, contentless_delete=1, tokendata=1 ); CREATE TABLE ctrl2(id INTEGER PRIMARY KEY, x TEXT, y TEXT); INSERT INTO ft2(ft2, rank) VALUES('pgsz', 64); INSERT INTO ft2(ft2, rank) VALUES('rank', 'rankfunc()'); } do_test 2.1 { for {set ii 0} {$ii < $NDOC} {incr ii} { set doc1 [document] set doc2 [document] execsql { INSERT INTO ft2(rowid, x, y) VALUES($ii, $doc, $doc2); INSERT INTO ctrl2(id, x, y) VALUES($ii, $doc, $doc2); } } } {} proc do_all_vocab_test2 {tn} { foreach ::v [vocab] { set answer [execsql { SELECT id, ctrl_tokens($::v, x, y) FROM ctrl2 WHERE x LIKE '%' || $::v || '%' OR y LIKE '%' || $::v || '%'; }] do_execsql_test $tn.$::v.1 { SELECT rowid, tokens(ft2) FROM ft2($::v) } $answer do_execsql_test $tn.$::v.2 { SELECT rowid, tokens(ft2) FROM ft2($::v) ORDER BY rank } $answer } } do_all_vocab_test2 2.2 for {set ii 0} {$ii < $NLOOP} {incr ii} { set lRowid [execsql { SELECT id FROM ctrl2 WHERE random() % 2 }] foreach r $lRowid { execsql { DELETE FROM ft2 WHERE rowid = $r } execsql { DELETE FROM ctrl2 WHERE rowid = $r } set doc1 [document] set doc2 [document] execsql { INSERT INTO ft2(rowid, x, y) VALUES($r, $doc, $doc1) } execsql { INSERT INTO ctrl2(id, x, y) VALUES($r, $doc, $doc2) } } do_all_vocab_test 2.3.$ii } #------------------------------------------------------------------------- unset -nocomplain ::expanded_vocab proc vocab {} { list abcde fghij klmno } proc do_all_vocab_test3 {tn} { foreach ::v [concat [vocab] nnn] { set answer [execsql { SELECT rowid, ctrl_tokens($::v, w) FROM ctrl3 WHERE w LIKE '%' || $::v || '%' }] do_execsql_test $tn.$::v.1 { SELECT rowid, tokens(ft3) FROM ft3($::v) } $answer do_execsql_test $tn.$::v.2 { SELECT rowid, tokens(ft3) FROM ft3($::v) ORDER BY rank } $answer } } do_execsql_test 3.0 { CREATE VIRTUAL TABLE ft3 USING fts5( w, tokenize="origintext unicode61", content=, contentless_delete=1, tokendata=1 ); INSERT INTO ft3(ft3, rank) VALUES('rank', 'rankfunc()'); CREATE TABLE ctrl3(w); } do_execsql_test 3.1 { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<2 ) INSERT INTO ctrl3 SELECT document() FROM s; INSERT INTO ft3(rowid, w) SELECT rowid, w FROM ctrl3; } do_all_vocab_test3 3.2 finish_test |
Added ext/fts5/test/fts5origintext6.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | # 2014 Jan 08 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5origintext6 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc insert_data {tbl} { db eval " INSERT INTO $tbl (rowid, x, y) VALUES (1, 'ChH BDd HhG efc BjJ BGi GBG FdD','ciJ AFf ADf fBJ fhC GFI JEH fcA'), (2, 'deg AIG Fie jII cCd Hbf igF fEE','GeA Ija gJg EDc HFi DDI dCf aDd'), (3, 'IJC hga deC Jfa Aeg hfh CcH dfb','ajD hgC Jaf IfH CHe jIG AjD adF'), (4, 'FiH GJH IDA AiG bBc CGG Eih bIH','hHg JaH aii IHE Ggd gcH gji CGc'), (5, 'ceg CAd jFI GAB BGg EeC IdH acG','bBC eIG ifH eDE Adj bjb GCj ebA'), (6, 'Eac Fbh aFF Eea jeG EIj HCc JJH','hbd giE Gfe eiI dEF abE cJf cAb'), (7, 'dic hAc jEC AiG FEF jHc HiD HBI','aEd ebE Gfi AJG EBA faj GiG jjE'), (8, 'Fca iEe EgE jjJ gce ijf EGc EBi','gaI dhH bFg CFc HeC CjI Jfg ccH'), (9, 'cfd iaa HCf iHJ HjG ffh ABb ibi','CfG bia Dai eii Ejg Jeg fCg hDb'), (10, 'Jjf hJC IID HJj bGB EbJ cgg eBj','jci jhi JAF jIg Bei Bcd cAC AJd'), (11, 'egG Cdi bFf fEB hfH jDH jia Efd','FAd eCg fAi aiC baC eJG acF iGE'), (12, 'Ada Gde CJI ADG gJA Cbb ccF iAB','eAE ajC FBB ccd Jgh fJg ieg hGE'), (13, 'gBb fDG Jdd HdD fiJ Bed Cig iGg','heC FeI iaj gdg ebB giC HaD FIe'), (14, 'FiI iDd Ffe igI bgB EJf FHG hDF','cjC AeI abf Fah cbJ ffH jEb aib'), (15, 'jaF hBI jIH Gdh FEc Fij hgj jFh','dGA ADH feh AAI AfJ DbC gBi hGH'), (16, 'gjH BGg iGj aFE CAH edI idf HEH','hIf DDg fjB hGi cHF BCH FjG Bgd'), (17, 'iaI JGH hji gcj Dda eeG jDd CBi','cHg jeh caG gIc feF ihG hgJ Abj'), (18, 'jHI iDB eFf AiH EFB CDb IAj GbC','Ghe dEI gdI jai gib dAG BIa djb'), (19, 'abI fHG Ccf aAc FDa fiC agF bdB','afi hde IgE bGF cfg DHD diE aca'), (20, 'IFh eDJ jfh cDg dde JGJ GAf fIJ','IBa EfH faE aeI FIF baJ FGj EIH'), (21, 'Dee bFC bBA dEI CEj aJI ghA dCH','hBA ddA HJh dfj egI Dij dFE bGE'), (22, 'JFE BCj FgA afc Jda FGD iHJ HDh','eAI jHe BHD Gah bbD Bgj gbh eGB'), (23, 'edE CJE FjG aFI edA Cea FId iFe','ABG jcA ddj EEc Dcg hAI agA biA'), (24, 'AgE cfc eef cGh aFB DcH efJ hcH','eGF HaB diG fgi bdc iGJ FGJ fFB'), (25, 'aCa AgI GhC DDI hGJ Hgc Gcg bbG','iID Fga jHa jIj idj DFD bAC AFJ'), (26, 'gjC JGh Fge faa eCA iGG gHE Gai','bDi hFE BbI DHD Adb Fgi hCa Hij'), (27, 'Eji jEI jhF DFC afH cDh AGc dHA','IDe GcA ChF DIb Bif HfH agD DGh'), (28, 'gDD AEE Dfg ICf Cbi JdE jgH eEi','eEb dBG FDE jgf cAI FaJ jaA cDd'), (29, 'cbe Gec hgB Egi bca dHg bAJ jBf','EFB DgD GJc fDb EeE bBA GFC Hbe'), (30, 'Adc eHB afI hDc Bhh baE hcJ BBd','JAH deg bcF Dab Bgj Gbb JHi FIB'), (31, 'agF dIj AJJ Hfg cCG hED Igc fHC','JEf eia dHf Ggc Agj geD bEE Gei'), (32, 'DAd cCe cbJ FjG gJe gba dJA GCf','eAf hFc bGE ABI hHA IcE abF CCE'), (33, 'fFh jJe DhJ cDJ EBi AfD eFI IhG','fEG GCc Bjd EFF ggg CFe EHd ciB'), (34, 'Ejb BjI eAF HaD eEJ FaG Eda AHC','Iah hgD EJG fdD cIE Daj IFf eJh'), (35, 'aHG eCe FjA djJ dAJ jiJ IaE GGB','Acg iEF JfB FIC Eei ggj dic Iii'), (36, 'Fdb EDF GaF JjB ehH IgC hgi DCG','cag DHI Fah hAJ bbh egG Hia hgJ'), (37, 'HGg icC JEC AFJ Ddh dhi hfC Ich','fEg bED Bff hCJ EiA cIf bfG cGA'), (38, 'aEJ jGI BCi FaA ebA BHj cIJ GcC','dCH ADd bGB cFE AgF geD cbG jIc'), (39, 'JFB bBi heA BFA hgB Ahj EIE CgI','EIJ JFG FJE GeA Hdg HeH ACh GiA'), (40, 'agB DDC CED igC Dfc DhI eiC fHi','dAB dcg iJF cej Fcc cAc AfB Fdd'), (41, 'BdF DHj Ege hcG DEd eFa dCf gBb','FBG ChB cej iGd Hbh fCc Ibe Abh'), (42, 'Bgc DjI cbC jGD bdb hHB IJA IJH','heg cii abb IGf eDe hJc dii fcE'), (43, 'fhf ECa FiA aDh Jbf CiB Jhe ajD','GFE bIF aeD gDE BIE Jea DfC BEc'), (44, 'GjE dBj DbJ ICF aDh EEH Ejb jFb','dJj aEc IBg bEG Faf fjA hjf FAF'), (45, 'BfA efd IIJ AHG dDF eGg dIJ Gcb','Bfj jeb Ahc dAE ACH Dfb ieb dhC'), (46, 'Ibj ege geC dJh CIi hbD EAG fGA','DEb BFe Bjg FId Fhg HeF JAc BbE'), (47, 'dhB afC hgG bEJ aIe Cbe iEE JCD','bdg Ajc FGA jbh Jge iAj fIA jbE'), (48, 'egH iDi bfH iiI hGC jFF Hfd AHB','bjE Beb iCc haB gIH Dea bga dfd'), (49, 'jgf chc jGc Baj HBb jdE hgh heI','FFB aBd iEB EIG HGf Bbj EIi JbI'), (50, 'jhe EGi ajA fbH geh EHe FdC bij','jDE bBC gbH HeE dcH iBH IFE AHi'), (51, 'aCb JiD cgJ Bjj iAI Hbe IAF FhH','ijf bhE Jdf FED dCH bbG HcJ ebH'); " } foreach_detail_mode $testprefix { foreach external {0 1 2} { reset_db proc tokens {cmd} { set ret [list] for {set iTok 0} {$iTok < [$cmd xInstCount]} {incr iTok} { set txt [$cmd xInstToken $iTok 0] set txt [string map [list "\0" "."] $txt] lappend ret $txt } set ret } sqlite3_fts5_create_function db tokens tokens sqlite3_fts5_register_origintext db set E(0) internal set E(1) external set E(2) contentless set e $E($external) db eval { CREATE TABLE ex(x, y) } switch -- $external { 0 { do_execsql_test 1.$e.0 { CREATE VIRTUAL TABLE ft USING fts5( x, y, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL% ); } } 1 { do_execsql_test 1.$e.0 { CREATE VIRTUAL TABLE ft USING fts5( x, y, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%, content=ex ); } } 2 { do_execsql_test 1.$e.0 { CREATE VIRTUAL TABLE ft USING fts5( x, y, tokenize="origintext unicode61", tokendata=1, detail=%DETAIL%, content= ); } } } insert_data ex insert_data ft proc prefixquery {prefix bInst bYOnly} { set ret [list] db eval { SELECT rowid, x, y FROM ex ORDER BY rowid } { set row [list] set bSeen 0 set T [concat $x $y] if {$bYOnly} { set T $y } foreach w $T { if {[string match -nocase $prefix $w]} { set bSeen 1 if {$bInst} { set v [string tolower $w] if {$w != $v} { append v ".$w" } lappend row $v } } } if {$bSeen} { lappend ret $rowid lappend ret $row } } set ret } proc do_prefixquery_test {tn prefix} { set bInst [expr {$::e!="contentless" || "%DETAIL%"=="full"}] set expect [prefixquery $prefix $bInst 0] set expect2 [prefixquery $prefix $bInst 1] uplevel [list do_execsql_test $tn.1 " SELECT rowid, tokens(ft) FROM ft('$prefix') " $expect] uplevel [list do_execsql_test $tn.2 " SELECT rowid, tokens(ft) FROM ft(fts5_insttoken('$prefix')) " $expect] db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 1) } uplevel [list do_execsql_test $tn.3 " SELECT rowid, tokens(ft) FROM ft('$prefix') " $expect] db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 0) } if {"%DETAIL%"!="none"} { uplevel [list do_execsql_test $tn.4 " SELECT rowid, tokens(ft) FROM ft('y: $prefix') " $expect2] uplevel [list do_execsql_test $tn.5 " SELECT rowid, tokens(ft) FROM ft(fts5_insttoken('y: $prefix')) " $expect2] db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 1) } uplevel [list do_execsql_test $tn.6 " SELECT rowid, tokens(ft) FROM ft('y: $prefix') " $expect2] db eval { INSERT INTO ft(ft, rank) VALUES('insttoken', 0) } } } do_prefixquery_test 1.$e.1 a* do_prefixquery_test 1.$e.2 b* do_prefixquery_test 1.$e.3 c* do_prefixquery_test 1.$e.4 d* do_prefixquery_test 1.$e.5 e* do_prefixquery_test 1.$e.6 f* do_prefixquery_test 1.$e.7 g* do_prefixquery_test 1.$e.8 h* do_prefixquery_test 1.$e.9 i* do_prefixquery_test 1.$e.10 j* }} finish_test |
Changes to ext/fts5/test/fts5phrase.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5phrase | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5phrase # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t3 USING fts5(a, b, c); |
︙ | ︙ | |||
89 90 91 92 93 94 95 | 6 {a b} "b h" 7 {a c} "b h" 8 {c a} "b h" 9 {c} "i e" 10 {b} "i e" 11 {a} "i e" } { | < > > > > > > | | | | > | 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | 6 {a b} "b h" 7 {a c} "b h" 8 {c a} "b h" 9 {c} "i e" 10 {b} "i e" 11 {a} "i e" } { set where [list] foreach c $cols { lappend where "pmatch($c, '$tokens')" } set where [join $where " OR "] foreach fts [list \ "{$cols}:[join $tokens +]" \ "{$cols}:NEAR([join $tokens +])" \ "{$cols}:NEAR([join $tokens +],1)" \ "{$cols}:NEAR([join $tokens +],111)" \ ] { set res [db eval "SELECT rowid FROM t3 WHERE $where"] do_execsql_test "1.$tn.$fts->([llength $res] rows)" { SELECT rowid FROM t3($fts) } $res } } do_execsql_test 2.0 { SELECT rowid, highlight(t3, 0, '*', '*'), highlight(t3, 1, '*', '*'), highlight(t3, 2, '*', '*') |
︙ | ︙ |
Changes to ext/fts5/test/fts5plan.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file focuses on testing the planner (xBestIndex function). # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5plan | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file focuses on testing the planner (xBestIndex function). # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5plan # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE TABLE t1(x, y); |
︙ | ︙ |
Changes to ext/fts5/test/fts5porter.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # # http://tartarus.org/martin/PorterStemmer/ # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5porter | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # # http://tartarus.org/martin/PorterStemmer/ # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5porter # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set test_vocab { a a aaron aaron |
︙ | ︙ |
Changes to ext/fts5/test/fts5porter2.test.
︙ | ︙ | |||
14 15 16 17 18 19 20 | # These are extra tests added to those in fts5porter.test in order to # improve test coverage of the porter stemmer implementation. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5porter2 | | | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | # These are extra tests added to those in fts5porter.test in order to # improve test coverage of the porter stemmer implementation. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5porter2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } set test_vocab { tion tion |
︙ | ︙ |
Changes to ext/fts5/test/fts5prefix.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file contains tests focused on prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5prefix | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file contains tests focused on prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5prefix # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE xx USING fts5(x, prefix=1); |
︙ | ︙ |
Changes to ext/fts5/test/fts5prefix2.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file contains tests focused on prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5prefix2 | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file contains tests focused on prefix indexes. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5prefix2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach p {3 2 1} { reset_db |
︙ | ︙ | |||
47 48 49 50 51 52 53 54 55 56 57 | INSERT INTO t2 VALUES('to'); INSERT INTO t2 VALUES('tommy'); } do_execsql_test 2.1 { SELECT * FROM t2('to*'); } {top to tommy} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | INSERT INTO t2 VALUES('to'); INSERT INTO t2 VALUES('tommy'); } do_execsql_test 2.1 { SELECT * FROM t2('to*'); } {top to tommy} #------------------------------------------------------------------------- foreach {tn newrowid} { 1 122 2 123 3 124 } { reset_db do_execsql_test 3.$tn.0 { CREATE VIRTUAL TABLE t12 USING fts5(x); INSERT INTO t12(rowid, x) VALUES(123, 'wwww'); } do_execsql_test 3.$tn.1 { BEGIN; DELETE FROM t12 WHERE rowid=123; SELECT * FROM t12('wwww*'); INSERT INTO t12(rowid, x) VALUES($newrowid, 'wwww'); SELECT * FROM t12('wwww*'); END; } {wwww} do_execsql_test 3.$tn.2 { INSERT INTO t12(t12) VALUES('integrity-check'); } do_execsql_test 3.$tn.3 { SELECT rowid FROM t12('wwww*'); } $newrowid } finish_test finish_test |
Changes to ext/fts5/test/fts5query.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5query | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5query # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } for {set tn 1 ; set pgsz 64} {$tn<32} {incr tn; incr pgsz 16} { reset_db |
︙ | ︙ |
Changes to ext/fts5/test/fts5rank.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file focuses on testing queries that use the "rank" column. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5rank | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This file focuses on testing queries that use the "rank" column. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5rank # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- |
︙ | ︙ | |||
176 177 178 179 180 181 182 183 | } do_execsql_test 6.1 { SELECT * FROM "My.Table" WHERE Text MATCH 'table' ORDER BY rank; } { {table table table} {the table names.} {rank on an fts5 table} } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > | 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 | } do_execsql_test 6.1 { SELECT * FROM "My.Table" WHERE Text MATCH 'table' ORDER BY rank; } { {table table table} {the table names.} {rank on an fts5 table} } #------------------------------------------------------------------------- # forum post: https://sqlite.org/forum/forumpost/a2dd636330 # reset_db do_execsql_test 1.0 { CREATE VIRTUAL TABLE t USING fts5 (a, b); INSERT INTO t (a, b) VALUES ('data1', 'sentence1'), ('data2', 'sentence2'); INSERT INTO t(t, rank) VALUES ('rank', 'bm25(10.0,1.0)'); } sqlite3 db2 test.db do_execsql_test -db db2 1.1 { SELECT *, rank<0.0 FROM t('data*') ORDER BY RANK; } {data1 sentence1 1 data2 sentence2 1} do_execsql_test 1.2 { INSERT INTO t(t, rank) VALUES ('rank', 'bm25(10.0,1.0)'); } do_execsql_test -db db2 1.3 { SELECT *, rank<0.0 FROM t('data*') ORDER BY RANK; } {data1 sentence1 1 data2 sentence2 1} db2 close finish_test |
Changes to ext/fts5/test/fts5rebuild.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5rebuild | | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5rebuild # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.1 { CREATE VIRTUAL TABLE f1 USING fts5(a, b); |
︙ | ︙ | |||
42 43 44 45 46 47 48 | sqlite3_db_config db DEFENSIVE 0 do_execsql_test 1.5 { DELETE FROM f1_data; } {} do_catchsql_test 1.6 { INSERT INTO f1(f1) VALUES('integrity-check'); | | | 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | sqlite3_db_config db DEFENSIVE 0 do_execsql_test 1.5 { DELETE FROM f1_data; } {} do_catchsql_test 1.6 { INSERT INTO f1(f1) VALUES('integrity-check'); } {/.*fts5: corrupt.*/} do_execsql_test 1.7 { INSERT INTO f1(f1) VALUES('rebuild'); INSERT INTO f1(f1) VALUES('integrity-check'); } {} |
︙ | ︙ |
Changes to ext/fts5/test/fts5restart.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This file focuses on testing the planner (xBestIndex function). # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5restart | | > | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 | # # This file focuses on testing the planner (xBestIndex function). # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5restart # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE f1 USING fts5(ff); } #------------------------------------------------------------------------- # Run the 'optimize' command. Check that it does not disturb ongoing # full-text queries. # unset -nocomplain lRowid do_test 1.1 { for {set i 1} {$i < 1000} {incr i} { execsql { INSERT INTO f1 VALUES('a b c d e') } lappend lRowid $i } } {} |
︙ | ︙ |
Changes to ext/fts5/test/fts5rowid.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests of the scalar fts5_rowid() and fts5_decode() functions. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5rowid | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests of the scalar fts5_rowid() and fts5_decode() functions. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5rowid # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_catchsql_test 1.1 { SELECT fts5_rowid() |
︙ | ︙ |
Changes to ext/fts5/test/fts5savepoint.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5savepoint | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #*********************************************************************** # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5savepoint # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(c); |
︙ | ︙ | |||
44 45 46 47 48 49 50 | DROP TABLE ft2_idx; BEGIN; INSERT INTO ft2 VALUES('a'); INSERT INTO ft1 VALUES('a'); SAVEPOINT two; INSERT INTO ft1 VALUES('b'); COMMIT; | | | 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 | DROP TABLE ft2_idx; BEGIN; INSERT INTO ft2 VALUES('a'); INSERT INTO ft1 VALUES('a'); SAVEPOINT two; INSERT INTO ft1 VALUES('b'); COMMIT; } {1 {database disk image is malformed}} reset_db ifcapable fts3 { do_execsql_test 3.0 { CREATE VIRTUAL TABLE vt0 USING fts5(c0); CREATE VIRTUAL TABLE vt1 USING fts4(c0); INSERT INTO vt1(c0) VALUES(0); |
︙ | ︙ | |||
67 68 69 70 71 72 73 | UPDATE vt1 SET c0 = 0; INSERT INTO vt1(vt1) VALUES('automerge=1'); UPDATE vt1 SET c0 = 0; } do_catchsql_test 3.2 { DROP TABLE vt1; | | | 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | UPDATE vt1 SET c0 = 0; INSERT INTO vt1(vt1) VALUES('automerge=1'); UPDATE vt1 SET c0 = 0; } do_catchsql_test 3.2 { DROP TABLE vt1; } {0 {}} do_execsql_test 3.3 { SAVEPOINT x; INSERT INTO vt0 VALUES('x'); COMMIT; INSERT INTO vt0(vt0) VALUES('integrity-check'); } } finish_test |
Added ext/fts5/test/fts5secure.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 | # 2023 Feb 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5secure proc dump {tname} { execsql_pp "SELECT * FROM ${tname}_idx" execsql_pp "SELECT id, quote(block), fts5_decode(id,block) FROM ${tname}_data" } do_execsql_test 0.0 { CREATE VIRTUAL TABLE t1 USING fts5(ab); CREATE VIRTUAL TABLE v1 USING fts5vocab('t1', 'instance'); INSERT INTO t1(rowid, ab) VALUES (0,'abc'), (1,'abc'), (2,'abc'), (3,'abc'), (4,'def'); } do_execsql_test 0.1 { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } do_execsql_test 0.2 { DELETE FROM t1 WHERE rowid=2; } do_execsql_test 0.3 { SELECT count(*) FROM t1_data } 3 do_execsql_test 0.4 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 0.5 { DELETE FROM t1 WHERE rowid=3; } do_execsql_test 0.6 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 0.7 { DELETE FROM t1 WHERE rowid=0; } do_execsql_test 0.8 { INSERT INTO t1(t1) VALUES('integrity-check'); } #---------------------------------- do_execsql_test 1.0 { CREATE VIRTUAL TABLE t2 USING fts5(ab); INSERT INTO t2(rowid, ab) VALUES (5, 'key'), (6, 'value'); INSERT INTO t2(t2, rank) VALUES('secure-delete', 1); } #execsql_pp { SELECT id, quote(block) FROM t1_data } #execsql_pp { SELECT segid, quote(term), pgno FROM t1_idx } do_execsql_test 1.1 { DELETE FROM t2 WHERE rowid = 5; } do_execsql_test 1.2 { INSERT INTO t2(t2) VALUES('integrity-check'); } do_execsql_test 1.3 { DELETE FROM t2 WHERE rowid = 6; } do_execsql_test 1.4 { INSERT INTO t2(t2) VALUES('integrity-check'); } do_execsql_test 1.5 { SELECT * FROM t2('value'); SELECT * FROM t2('v*'); } do_execsql_test 1.6 { SELECT * FROM t2('value') ORDER BY rowid DESC; SELECT * FROM t2('v*') ORDER BY rowid DESC; } execsql_pp { SELECT id, quote(block) FROM t2_data; } #---------------------------------- do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft USING fts5(ab); CREATE VIRTUAL TABLE vocab USING fts5vocab('ft', 'instance'); INSERT INTO ft(rowid, ab) VALUES (1, 'one'), (2, 'two'), (3, 'three'), (4, 'four'), (5, 'one one'), (6, 'one two'), (7, 'one three'), (8, 'one four'), (9, 'two one'), (10, 'two two'), (11, 'two three'), (12, 'two four'), (13, 'three one'), (14, 'three two'), (15, 'three three'), (16, 'three four'); } do_execsql_test 2.1 { SELECT count(*) FROM ft_data; } {3} do_execsql_test 2.2 { INSERT INTO ft(ft, rank) VALUES('secure-delete', 1); } do_execsql_test 2.3 { DELETE FROM ft WHERE rowid=9; } do_execsql_test 2.4 { INSERT INTO ft(ft) VALUES('integrity-check'); } do_execsql_test 2.5 { DELETE FROM ft WHERE ab LIKE '%two%' } do_execsql_test 2.6 { INSERT INTO ft(ft) VALUES('integrity-check'); } do_execsql_test 2.7 { SELECT count(*) FROM ft_data; } {3} #---------------------------------- reset_db set ::vocab { one two three four five six seven eight nine ten eleven twelve thirteen fourteen fifteen sixteen seventeen eighteen nineteen twenty } proc rnddoc {} { set nVocab [llength $::vocab] set ret [list] for {set ii 0} {$ii < 8} {incr ii} { lappend ret [lindex $::vocab [expr int(abs(rand()) * $nVocab)]] } set ret } proc contains {list val} { expr {[lsearch $list $val]>=0} } foreach {tn pgsz} { 2 64 1 1000 } { reset_db db function rnddoc rnddoc db function contains contains expr srand(1) do_execsql_test 3.$tn.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('pgsz', $pgsz); WITH s(i) AS ( VALUES(1) UNION SELECT i+1 FROM s WHERE i<20 ) INSERT INTO t1 SELECT rnddoc() FROM s; } do_execsql_test 3.$tn.1 { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } foreach {rowid} { 6 16 3 4 9 14 13 7 20 15 19 10 11 2 5 18 17 1 12 8 } { do_execsql_test 3.$tn.2.$rowid { DELETE FROM t1 WHERE rowid=$rowid; } do_execsql_test 3.$tn.2.$rowid.ic { INSERT INTO t1(t1) VALUES('integrity-check'); } foreach v $::vocab { do_execsql_test 3.$tn.2.$rowid.q.$v { SELECT rowid FROM t1($v) } [db eval {SELECT rowid FROM t1 WHERE contains(x, $v)}] do_execsql_test 3.$tn.2.$rowid.q.$v.DESC { SELECT rowid FROM t1($v) ORDER BY 1 DESC } [db eval {SELECT rowid FROM t1 WHERE contains(x, $v) ORDER BY 1 DESC}] } } } do_execsql_test 3.3 { INSERT INTO t1(x) VALUES('optimize'); INSERT INTO t1(t1) VALUES('optimize'); SELECT count(*) FROM t1_data; } {3} #---------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } set L1 [string repeat abcdefghij 10] set L2 [string repeat 1234567890 10] do_execsql_test 4.1 { INSERT INTO t1 VALUES('aa' || $L1 || ' ' || $L2); } do_execsql_test 4.2 { DELETE FROM t1 WHERE rowid=1 } do_execsql_test 4.3 { INSERT INTO t1(t1) VALUES('integrity-check'); } #---------------------------------- reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } set doc "aa [string repeat {abc } 60]" do_execsql_test 5.1 { BEGIN; INSERT INTO t1 VALUES($doc); INSERT INTO t1 VALUES('aa abc'); COMMIT; } do_execsql_test 5.2 { DELETE FROM t1 WHERE rowid = 1; } do_execsql_test 5.3 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 5.4 { SELECT rowid FROM t1('abc'); } 2 do_execsql_test 5.5 { SELECT rowid FROM t1('aa'); } 2 #------------------------------------------------------------------------- # Tests for the bug fixed by https://sqlite.org/src/info/4b60a1c3 # reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE fts USING fts5(content); INSERT INTO fts(fts, rank) VALUES ('secure-delete', 1); INSERT INTO fts(rowid, content) VALUES (3407, 'profile profile profile profile profile profile profile profile pull pulling pulling really'); DELETE FROM fts WHERE rowid IS 3407; INSERT INTO fts(fts) VALUES ('integrity-check'); } foreach {tn detail} { 1 full 2 column 3 none } { do_execsql_test 6.1.$detail " DROP TABLE IF EXISTS t1; CREATE VIRTUAL TABLE t1 USING fts5(x, detail=$detail); " do_execsql_test 6.2.$detail { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } for {set ii 1} {$ii < 100} {incr ii} { do_execsql_test 6.3.$detail.$ii.1 { BEGIN; INSERT INTO t1(rowid, x) VALUES(10, 'word1'); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<CAST($ii AS integer) ) INSERT INTO t1(x) SELECT 'word3' FROM s; COMMIT; INSERT INTO t1(t1) VALUES('optimize'); } do_execsql_test 6.3.$detail.$ii.2 { DELETE FROM t1 WHERE rowid=10; INSERT INTO t1(t1) VALUES ('integrity-check'); } do_execsql_test 6.3.$detail.$ii.3 { DELETE FROM t1; } do_execsql_test 6.3.$detail.$ii.4 { BEGIN; INSERT INTO t1(rowid, x) VALUES(10, 'tokenA'); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<CAST($ii AS integer) ) INSERT INTO t1(x) SELECT group_concat('tokenB ') FROM s; COMMIT; INSERT INTO t1(t1) VALUES('optimize'); } do_execsql_test 6.3.$detail.$ii.5 { DELETE FROM t1 WHERE rowid=10; INSERT INTO t1(t1) VALUES ('integrity-check'); } do_execsql_test 6.3.$detail.$ii.6 { DELETE FROM t1; } } } finish_test |
Added ext/fts5/test/fts5secure2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | # 2023 Feb 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5secure2 do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(col); INSERT INTO ft VALUES('data for the table'); INSERT INTO ft VALUES('more of the same'); INSERT INTO ft VALUES('and extra data'); } do_execsql_test 1.1 { SELECT * FROM ft_config } {version 4} do_execsql_test 1.2 { INSERT INTO ft(ft, rank) VALUES('secure-delete', 1); SELECT * FROM ft_config; } {secure-delete 1 version 4} do_execsql_test 1.3 { INSERT INTO ft(ft, rank) VALUES('secure-delete', 1); SELECT * FROM ft_config; } {secure-delete 1 version 4} do_execsql_test 1.4 { DELETE FROM ft WHERE rowid=2; SELECT * FROM ft_config; } {secure-delete 1 version 5} do_execsql_test 1.5 { SELECT rowid, col FROM ft('data'); } {1 {data for the table} 3 {and extra data}} db close sqlite3 db test.db do_execsql_test 1.6 { SELECT rowid, col FROM ft('data'); } {1 {data for the table} 3 {and extra data}} #------------------------------------------------------------------------ reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft USING fts5(col); INSERT INTO ft VALUES('one zero one one zero'); INSERT INTO ft(ft, rank) VALUES('secure-delete', 1); } do_execsql_test 2.1 { SELECT count(*) FROM ft_data WHERE block=X'00000004'; } {0} do_execsql_test 2.2 { UPDATE ft SET col = 'zero one zero zero one' WHERE rowid=1; } do_execsql_test 2.3 { SELECT count(*) FROM ft_data WHERE block=X'00000004'; } {1} do_execsql_test 2.4 { INSERT INTO ft VALUES('one zero zero one'); DELETE FROM ft WHERE rowid=1; } do_execsql_test 2.5 { SELECT count(*) FROM ft_data WHERE block=X'00000004'; } {2} finish_test |
Added ext/fts5/test/fts5secure3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 | # 2023 Feb 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # # TESTRUNNER: slow # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5secure3 do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(col); INSERT INTO ft VALUES('data for the table'); INSERT INTO ft VALUES('more of the same'); INSERT INTO ft VALUES('and extra data'); INSERT INTO ft(ft, rank) VALUES('secure-delete', 1); } do_execsql_test 1.1 { BEGIN; INSERT INTO ft(rowid, col) VALUES(0, 'the next data'); DELETE FROM ft WHERE rowid=1; DELETE FROM ft WHERE rowid=2; INSERT INTO ft(rowid, col) VALUES(6, 'with some more of the same data'); COMMIT; } do_execsql_test 1.2 { INSERT INTO ft(ft) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); BEGIN; INSERT INTO t1 VALUES('the start'); } do_test 2.1 { for {set i 0} {$i < 1000} {incr i} { execsql { INSERT INTO t1 VALUES('the ' || hex(randomblob(3))) } } execsql { INSERT INTO t1 VALUES('the end'); COMMIT; } } {} do_execsql_test 2.2 { DELETE FROM t1 WHERE rowid BETWEEN 2 AND 1000; } do_execsql_test 2.3 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 2.6 { INSERT INTO t1(rowid, x) VALUES(500, 'middle'); INSERT INTO t1(rowid, x) VALUES(501, 'value'); SELECT * FROM t1('the middle'); } do_execsql_test 2.7 { INSERT INTO t1(t1) VALUES('optimize'); } do_execsql_test 2.8 { SELECT count(*) FROM t1_data } 4 #execsql_pp { SELECT id, quote(block), fts5_decode(id, block) FROM t1_data; } #------------------------------------------------------------------------- # Tests with large/small rowid values. # foreach {tn cfg} { 1 "" 2 "INSERT INTO fff(fff, rank) VALUES('secure-delete', 1)" } { reset_db expr srand(0) set vocab { Popper Poppins Popsicle Porfirio Porrima Porsche Porter Portia Portland Portsmouth Portugal Portuguese Poseidon Post PostgreSQL Potemkin Potomac Potsdam Pottawatomie Potter Potts Pound Poussin Powell PowerPC PowerPoint Powers Powhatan Poznan Prada Prado Praetorian Prague Praia Prakrit Pratchett Pratt Pravda Praxiteles Preakness Precambrian Preminger Premyslid Prensa Prentice Pres Presbyterian Presbyterianism } proc newdoc {} { for {set i 0} {$i<8} {incr i} { lappend ret [lindex $::vocab [expr int(abs(rand()) * [llength $::vocab])]] } set ret } db func newdoc newdoc proc random {} { set res [expr { int(rand() * 0x7FFFFFFFFFFFFFFF) }] if { int(rand() * 2) } { set res [expr $res*-1] } return $res } db func random random do_execsql_test 3.$tn.0 { CREATE VIRTUAL TABLE fff USING fts5(y); INSERT INTO fff(fff, rank) VALUES('pgsz', 64); WITH s(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM s WHERE x<1000 ) INSERT INTO fff(rowid, y) SELECT random() , newdoc() FROM s; WITH s(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM s WHERE x<1000 ) INSERT INTO fff(rowid, y) SELECT random() , newdoc() FROM s; WITH s(x) AS ( VALUES(1) UNION ALL SELECT x+1 FROM s WHERE x<1000 ) INSERT INTO fff(rowid, y) SELECT random() , newdoc() FROM s; } execsql $cfg proc lshuffle {in} { set out [list] while {[llength $in]>0} { set idx [expr int(abs(rand()) * [llength $in])] lappend out [lindex $in $idx] set in [lreplace $in $idx $idx] } set out } #dump fff set iTest 1 foreach ii [lshuffle [db eval {SELECT rowid FROM fff}]] { #if {$iTest==1} { dump fff } #if {$iTest==1} { breakpoint } do_execsql_test 3.$tn.1.$iTest.$ii { DELETE FROM fff WHERE rowid=$ii; } #if {$iTest==1} { dump fff } if {($iTest % 20)==0} { do_execsql_test 3.$tn.1.$iTest.$ii.ic { INSERT INTO fff(fff) VALUES('integrity-check'); } } #if {$iTest==1} { break } incr iTest } } #execsql_pp { SELECT rowid FROM fff('post') ORDER BY rowid ASC } #breakpoint #execsql_pp { # SELECT rowid FROM fff('post') ORDER BY rowid DESC #} # #dump fff finish_test |
Added ext/fts5/test/fts5secure4.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 | # 2023 April 14 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] return_if_no_fts5 set ::testprefix fts5secure4 #------------------------------------------------------------------------- # Test using the 'delete' command to attempt to delete a token that # is not present in the index in secure-delete mode. # do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b, content=x1); CREATE TABLE x1(rowid INTEGER PRIMARY KEY, a, b); INSERT INTO x1 VALUES (1, 'hello world', 'today xyz'), (2, 'not the day', 'crunch crumble and chomp'), (3, 'one', 'two'); INSERT INTO t1(t1) VALUES('rebuild'); } do_execsql_test 1.1 { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } do_execsql_test 1.2 { INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 4, 'nosuchtoken', ''); } do_execsql_test 1.3 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 1.4 { INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 1, 'crunch', ''); } do_execsql_test 1.5 { INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 3, 'crunch', ''); } do_execsql_test 1.6 { INSERT INTO t1(t1) VALUES('integrity-check'); } do_execsql_test 1.7 { CREATE VIRTUAL TABLE y1 USING fts5(xx, prefix='1,2'); INSERT INTO y1(y1, rank) VALUES('pgsz', 64); INSERT INTO y1(y1, rank) VALUES('secure-delete', 1); } do_execsql_test 1.8 { BEGIN; INSERT INTO y1(rowid, xx) VALUES(1, 'abc def'); INSERT INTO y1(rowid, xx) VALUES(2, 'reallyreallylongtoken'); COMMIT; } do_execsql_test 1.9 { DELETE FROM y1 WHERE rowid=1; INSERT INTO y1(y1) VALUES('integrity-check'); } do_execsql_test 1.10 { CREATE VIRTUAL TABLE w1 USING fts5(ww, content=""); INSERT INTO w1(rowid, ww) VALUES(123, ''); } do_catchsql_test 1.11 { INSERT INTO w1(w1, rowid, ww) VALUES('delete', 123, 'xyz'); } {1 {database disk image is malformed}} do_catchsql_test 1.12 { DROP TABLE w1; CREATE VIRTUAL TABLE w1 USING fts5(ww, content=""); INSERT INTO w1(rowid, ww) VALUES(123, ''); DELETE FROM w1_data WHERE id>10; INSERT INTO w1(w1, rowid, ww) VALUES('delete', 123, 'xyz'); } {1 {database disk image is malformed}} #------------------------------------------------------------------------- # Test using secure-delete with detail=none or detail=col. # foreach {tn d} {1 full 2 none 3 column} { reset_db do_execsql_test 2.$tn.1 " CREATE VIRTUAL TABLE x1 USING fts5(xx, yy, zz, detail=$d, prefix='10,20'); INSERT INTO x1(x1, rank) VALUES('pgsz', 64); INSERT INTO x1(x1, rank) VALUES('secure-delete', 1); " do_execsql_test 2.$tn.2 { BEGIN; INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c'); INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c'); INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c'); INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c'); INSERT INTO x1(xx, yy, zz) VALUES('a b c', 'd e f', 'a b c'); COMMIT; INSERT INTO x1(x1) VALUES('integrity-check'); } do_execsql_test 2.$tn.3 { DELETE FROM x1 WHERE rowid IN (2, 4, 6); INSERT INTO x1(x1) VALUES('integrity-check'); } do_execsql_test 2.$tn.4 { DELETE FROM x1 WHERE rowid IN (1, 3, 5); INSERT INTO x1(x1) VALUES('integrity-check'); } do_execsql_test 2.$tn.5 { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100 ) INSERT INTO x1 SELECT 'seems to be', 'used brew to', 'everything is working' FROM s UNION ALL SELECT 'used brew to', 'everything is working', 'seems to be' FROM s UNION ALL SELECT 'everything is working', 'seems to be', 'used brew to' FROM s UNION ALL SELECT 'abc', 'zzz', 'a b c d' UNION ALL SELECT 'z', 'z', 'z' FROM s } do_test 2.$tn.6 { for {set i 300} {$i > 200} {incr i -1} { execsql { DELETE FROM x1 WHERE rowid=$i; INSERT INTO x1(x1) VALUES('integrity-check'); } } } {} do_test 2.$tn.7 { for {set i 1} {$i < 100} {incr i} { execsql { DELETE FROM x1 WHERE rowid=$i; INSERT INTO x1(x1) VALUES('integrity-check'); } } } {} do_test 2.$tn.8 { foreach i [db eval {SELECT rowid FROM x1}] { execsql { DELETE FROM x1 WHERE rowid=$i; INSERT INTO x1(x1) VALUES('integrity-check'); } } } {} do_execsql_test 2.$tn.9 { SELECT * FROM x1 } {} } finish_test |
Added ext/fts5/test/fts5secure5.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | # 2023 April 14 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] return_if_no_fts5 set ::testprefix fts5secure5 return_if_no_fts5 proc dump {} { execsql_pp { SELECT id, quote(block), fts5_decode_none(id, block) FROM ft1_data } } do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none); INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1); } do_execsql_test 1.1 { BEGIN; INSERT INTO ft1(rowid, a) VALUES(1, 'abcd'); INSERT INTO ft1(rowid, a) VALUES(2, 'abcd'); INSERT INTO ft1(rowid, a) VALUES(3, 'abcd'); COMMIT; } do_execsql_test 1.2 { DELETE FROM ft1 WHERE rowid=1; } do_execsql_test 1.3 { INSERT INTO ft1(ft1) VALUES('integrity-check'); } do_execsql_test 1.4 { DELETE FROM ft1 WHERE rowid=3; } do_execsql_test 1.5 { INSERT INTO ft1(ft1) VALUES('integrity-check'); } do_execsql_test 1.6 { DELETE FROM ft1 WHERE rowid=3; } do_execsql_test 1.7 { INSERT INTO ft1(ft1) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none); INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1); } do_execsql_test 2.1 { BEGIN; INSERT INTO ft1(rowid, a) VALUES(1, 'abcd one'); INSERT INTO ft1(rowid, a) VALUES(2, 'abcd two'); INSERT INTO ft1(rowid, a) VALUES(3, 'abcd two'); INSERT INTO ft1(rowid, a) VALUES(4, 'abcd two'); INSERT INTO ft1(rowid, a) VALUES(5, 'abcd three'); COMMIT; } do_execsql_test 2.2a { DELETE FROM ft1 WHERE rowid=3; } do_execsql_test 2.2b { INSERT INTO ft1(ft1) VALUES('integrity-check'); } do_execsql_test 2.3a { DELETE FROM ft1 WHERE rowid=2; } do_execsql_test 2.3b { INSERT INTO ft1(ft1) VALUES('integrity-check'); } do_execsql_test 2.4a { DELETE FROM ft1 WHERE rowid=4; } do_execsql_test 2.4b { INSERT INTO ft1(ft1) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none, prefix=1); INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1); INSERT INTO ft1(ft1, rank) VALUES('pgsz', 64); } do_execsql_test 3.1 { BEGIN; INSERT INTO ft1(a) VALUES('c'); COMMIT; } do_execsql_test 3.2 { DELETE FROM ft1 WHERE rowid IN (1); INSERT INTO ft1(ft1) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE ft1 USING fts5(a, detail=none); INSERT INTO ft1(ft1, rank) VALUES('secure-delete', 1); INSERT INTO ft1(ft1, rank) VALUES('pgsz', 64); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<500 ) INSERT INTO ft1 SELECT 'abcdefg' FROM s; } do_test 4.1 { for {set i 500} {$i > 0} {incr i -1} { execsql { DELETE FROM ft1 WHERE rowid=$i } execsql { INSERT INTO ft1(ft1) VALUES('integrity-check') } } } {} finish_test |
Added ext/fts5/test/fts5secure6.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 | # 2023 Feb 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5secure6 db progress 1 progress_handler set ::PHC 0 proc progress_handler {args} { incr ::PHC # if {($::PHC % 100000)==0} breakpoint return 0 } proc setup {} { db eval { DROP TABLE IF EXISTS t1; CREATE VIRTUAL TABLE t1 USING fts5(x); WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<1000 ) INSERT INTO t1 SELECT 'a b c d e f g h i j k' FROM s; } } foreach {tn sd} { 1 0 2 1 } { setup do_execsql_test 1.$tn.0 { INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd) } set PHC 0 do_execsql_test 1.$tn.1 { DELETE FROM t1; } set phc($tn) $PHC } do_test 1.3 { expr $phc(1)*5 < $phc(2) } {1} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd) } do_execsql_test 2.1 { BEGIN; INSERT INTO t1(rowid, x) VALUES(-100000, 'abc def ghi'); INSERT INTO t1(rowid, x) VALUES(-99999, 'abc def ghi'); INSERT INTO t1(rowid, x) VALUES(9223372036854775800, 'abc def ghi'); COMMIT; } do_execsql_test 2.2 { SELECT rowid FROM t1('def') } {-100000 -99999 9223372036854775800} #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(t1, rank) VALUES('secure-delete', $sd) } do_execsql_test 3.1 { BEGIN; INSERT INTO t1(rowid, x) VALUES(51869, 'when whenever where weress what turn'), (51871, 'to were'); COMMIT; } do_execsql_test 3.2 { DELETE FROM t1 WHERE rowid=51871; INSERT INTO t1(t1) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1(rowid, x) VALUES(10, 'one two'); } do_execsql_test 4.1 { UPDATE t1 SET x = 'one three' WHERE rowid=10; INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } do_execsql_test 4.2 { DELETE FROM t1 WHERE rowid=10; } do_execsql_test 4.3 { INSERT INTO t1(t1) VALUES('integrity-check'); } #------------------------------------------------------------------------- reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE t1 USING fts5(content); INSERT INTO t1(t1,rank) VALUES('secure-delete',1); INSERT INTO t1 VALUES('active'),('boomer'),('atom'),('atomic'), ('alpha channel backup abandon test aback boomer atom alpha active'); DELETE FROM t1 WHERE t1 MATCH 'abandon'; } do_execsql_test 5.1 { INSERT INTO t1(t1) VALUES('rebuild'); } do_execsql_test 5.2 { DELETE FROM t1 WHERE rowid NOTNULL<5; } db close sqlite3 db test.db do_execsql_test 5.3 { PRAGMA integrity_check; } {ok} finish_test |
Added ext/fts5/test/fts5secure7.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 | # 2023 Feb 17 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # # TESTRUNNER: slow # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5secure7 set NVOCAB 500 set NDOC [expr 1000] set NREP 100 set nDeletePerRep [expr 5] set VOCAB [list] proc select_one {list} { set n [llength $list] lindex $list [expr {abs(int(rand()*$n))}] } proc init_vocab {} { set L [split "abcdefghijklmnopqrstuvwxyz" {}] set nL [llength $L] for {set i 0} {$i < $::NVOCAB} {incr i} { set n [expr {6 + int(rand()*8)}] set word "" for {set j 0} {$j < $n} {incr j} { append word [select_one $L] } lappend ::VOCAB $word } } proc get_word {} { select_one $::VOCAB } proc get_document {nWord} { set ret [list] for {set i 0} {$i < $nWord} {incr i} { lappend ret [get_word] } return $ret } init_vocab db func document [list get_document 12] do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(body); INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } do_execsql_test 1.1 { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$NDOC ) INSERT INTO t1 SELECT document() FROM s; } for {set iRep 0} {$iRep < $NREP} {incr iRep} { set lRowid [db eval {SELECT rowid FROM t1}] for {set iDel 0} {$iDel < $nDeletePerRep} {incr iDel} { set idx [select_one $lRowid] db eval { DELETE FROM t1 WHERE rowid=$idx } } db eval { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$nDeletePerRep ) INSERT INTO t1 SELECT document() FROM s; } do_execsql_test 1.2.$iRep { INSERT INTO t1(t1) VALUES('integrity-check'); } } reset_db db func document [list get_document 12] do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(body); INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); INSERT INTO t1(t1, rank) VALUES('pgsz', 128); } do_execsql_test 2.1 { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<$NDOC ) INSERT INTO t1 SELECT document() FROM s; } for {set ii 0} {$ii < $NDOC} {incr ii} { set lRowid [db eval {SELECT rowid FROM t1}] set idx [select_one $lRowid] db eval { DELETE FROM t1 WHERE rowid=$idx } do_execsql_test 2.2.$ii { INSERT INTO t1(t1) VALUES('integrity-check'); } } finish_test |
Added ext/fts5/test/fts5secure8.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 | # 2023 Nov 23 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5secure8 proc sql_repeat {txt n} { string repeat $txt $n } db func repeat sql_repeat do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(x); INSERT INTO ft(ft, rank) VALUES('pgsz', 64); INSERT INTO ft(rowid, x) VALUES(100, 'hello world'); INSERT INTO ft(rowid, x) VALUES(200, 'one day'); BEGIN; INSERT INTO ft(rowid, x) VALUES(45, 'one two three'); UPDATE ft SET x = repeat('hello world ', 500) WHERE rowid=100; COMMIT } do_execsql_test 1.1 { INSERT INTO ft(ft, rank) VALUES('secure-delete', 1); DELETE FROM ft WHERE rowid=100; } do_execsql_test 1.2 { PRAGMA integrity_check; } {ok} do_execsql_test 2.0 { CREATE VIRTUAL TABLE xyz USING fts5 ( name, content='' ); INSERT INTO xyz(xyz, rank) VALUES('secure-delete', 1); INSERT INTO xyz (rowid, name) VALUES(1, 'A'); INSERT INTO xyz (rowid, name) VALUES(2, 'A'); INSERT INTO xyz(xyz, rowid, name) VALUES('delete', 2, 'A'); } do_execsql_test 2.1 { pragma quick_check; } {ok} do_catchsql_test 2.2 { INSERT INTO xyz(xyz, rank) VALUES('secure-delete', 'hello world'); } {1 {SQL logic error}} finish_test |
Added ext/fts5/test/fts5securefault.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 | # 2023 April 14 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] source $testdir/malloc_common.tcl set testprefix fts5securefault # If SQLITE_ENABLE_FTS5 is not defined, omit this file. return_if_no_fts5 do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(rowid, ab) VALUES (0, 'abc'), (1, 'abc'), (2, 'abc'), (3, 'abc'), (4, 'def'); } faultsim_save_and_close do_faultsim_test 1.1 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { DELETE FROM t1 WHERE rowid=2 } } -test { faultsim_test_result {0 {}} } do_faultsim_test 1.2 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { DELETE FROM t1 WHERE rowid IN(0, 1, 2, 3, 4) } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- # reset_db set big [string repeat abcdefghij 5] set big2 [string repeat klmnopqrst 5] set doc "$big $big2" do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<4 ) INSERT INTO t1(rowid, ab) SELECT i, $doc FROM s; } faultsim_save_and_close do_faultsim_test 2.1 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { DELETE FROM t1 WHERE rowid = 3 } execsql { DELETE FROM t1 WHERE rowid = 4 } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- # reset_db set big [string repeat abcdefghij 5] set big2 [string repeat klmnopqrst 5] set doc "$big $big2" do_execsql_test 3.0 { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<25 ) INSERT INTO t1(rowid, ab) SELECT i, $doc FROM s; INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); DELETE FROM t1 WHERE rowid BETWEEN 3 AND 23; } faultsim_save_and_close do_faultsim_test 3.1 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { DELETE FROM t1 WHERE rowid = 24 } execsql { DELETE FROM t1 WHERE rowid = 25 } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- # reset_db set doc [string repeat "tok " 400] do_execsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); INSERT INTO t1(rowid, ab) VALUES(1, $doc), (2, $doc), (3, $doc); } faultsim_save_and_close do_faultsim_test 4.1 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { DELETE FROM t1 WHERE rowid = 2 } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- # reset_db set doc1 [string repeat "abc " 10] set doc2 [string repeat "def " 10] do_test 5.0 { execsql { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); BEGIN; } for {set i 0} {$i < 50} {incr i} { execsql { INSERT INTO t1(rowid, ab) VALUES($i, 'abcdefg'); } } execsql { INSERT INTO t1(rowid, ab) VALUES(105, 'def'); COMMIT; } } {} faultsim_save_and_close do_faultsim_test 5.1 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { DELETE FROM t1 WHERE rowid = 105 } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- # reset_db do_test 6.0 { execsql { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(t1, rank) VALUES('pgsz', 64); BEGIN; INSERT INTO t1(rowid, ab) VALUES(1, 'abcdefg'); INSERT INTO t1(rowid, ab) VALUES(2, 'abcdefg'); INSERT INTO t1(rowid, ab) VALUES(3, 'abcdefg'); COMMIT; } } {} faultsim_save_and_close do_faultsim_test 6.1 -faults oom* -prep { faultsim_restore_and_reopen execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } -body { execsql { UPDATE t1 SET ab='abcdefg' WHERE rowid=2; } } -test { faultsim_test_result {0 {}} } #------------------------------------------------------------------------- # reset_db do_test 7.0 { execsql { CREATE VIRTUAL TABLE t1 USING fts5(ab); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); INSERT INTO t1(t1, rank) VALUES('secure-delete', 1); } } {} faultsim_save_and_close do_faultsim_test 7.1 -faults oom* -prep { faultsim_restore_and_reopen set big1 "[string repeat x 50] [string repeat y 50] [string repeat z 50]" execsql { BEGIN; INSERT INTO t1 VALUES($big1); } } -body { execsql { COMMIT } } -test { faultsim_test_result {0 {}} } finish_test |
Changes to ext/fts5/test/fts5simple.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5simple | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5simple # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } if 1 { |
︙ | ︙ | |||
346 347 348 349 350 351 352 | WITH ii(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM ii WHERE i<10 ) INSERT INTO x1 SELECT rnddoc(5) FROM ii; } do_execsql_test 14.4 { SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads' | | | 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 | WITH ii(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM ii WHERE i<10 ) INSERT INTO x1 SELECT rnddoc(5) FROM ii; } do_execsql_test 14.4 { SELECT rowid, x, x1 FROM x1 WHERE x1 MATCH '*reads' } {0 {} 2} #------------------------------------------------------------------------- reset_db do_execsql_test 15.0 { CREATE VIRTUAL TABLE x2 USING fts5(x, prefix=1); INSERT INTO x2 VALUES('ab'); } |
︙ | ︙ | |||
475 476 477 478 479 480 481 482 483 | INSERT INTO x1(x) VALUES('x y z'); INSERT INTO x1(x) VALUES('c b a'); INSERT INTO x1(x) VALUES('z y x'); } do_catchsql_test 22.1 {SELECT * FROM x1('')} {1 {fts5: syntax error near ""}} do_catchsql_test 22.2 {SELECT * FROM x1(NULL)} {1 {fts5: syntax error near ""}} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 | INSERT INTO x1(x) VALUES('x y z'); INSERT INTO x1(x) VALUES('c b a'); INSERT INTO x1(x) VALUES('z y x'); } do_catchsql_test 22.1 {SELECT * FROM x1('')} {1 {fts5: syntax error near ""}} do_catchsql_test 22.2 {SELECT * FROM x1(NULL)} {1 {fts5: syntax error near ""}} #------------------------------------------------------------------------- reset_db do_execsql_test 23.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); SELECT count(*) FROM x1_data; } {2} do_execsql_test 23.1 { BEGIN; INSERT INTO x1 VALUES('a b c d'); INSERT INTO x1 VALUES('a b c d'); INSERT INTO x1 VALUES('a b c d'); } do_execsql_test 23.2 { SELECT count(*) FROM x1_data; } {2} do_execsql_test 23.3 { INSERT INTO x1(x1) VALUES('flush'); SELECT count(*) FROM x1_data; } {3} do_execsql_test 23.4 { ROLLBACK; SELECT count(*) FROM x1_data; } {2} finish_test |
Changes to ext/fts5/test/fts5simple2.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5simple2 | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5simple2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, detail=none); |
︙ | ︙ | |||
339 340 341 342 343 344 345 | CREATE VIRTUAL TABLE t2 USING fts5(x, y); BEGIN; INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); COMMIT; } | > | > | 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 | CREATE VIRTUAL TABLE t2 USING fts5(x, y); BEGIN; INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); COMMIT; } do_execsql_test 17.1 { SELECT * FROM t2('y:a*') WHERE rowid BETWEEN 10 AND 20 } do_execsql_test 17.2 { BEGIN; INSERT INTO t2 VALUES('a aa aaa', 'b bb bbb'); SELECT * FROM t2('y:a*') WHERE rowid BETWEEN 10 AND 20 ; } do_execsql_test 17.3 { COMMIT |
︙ | ︙ |
Changes to ext/fts5/test/fts5simple3.test.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5simple3 | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 | # #************************************************************************* # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5simple3 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } fts5_aux_test_functions db |
︙ | ︙ | |||
77 78 79 80 81 82 83 | INSERT INTO x3 VALUES('a b c'); INSERT INTO x3 VALUES('c b a'); INSERT INTO x3 VALUES('o t t'); SELECT * FROM x3('x OR y OR z'); } #------------------------------------------------------------------------- | | | 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 | INSERT INTO x3 VALUES('a b c'); INSERT INTO x3 VALUES('c b a'); INSERT INTO x3 VALUES('o t t'); SELECT * FROM x3('x OR y OR z'); } #------------------------------------------------------------------------- # Test that a crash occurring when the second or subsequent tokens in a # phrase matched zero rows has been fixed. # do_execsql_test 4.0 { CREATE VIRTUAL TABLE t1 USING fts5(x); INSERT INTO t1 VALUES('ab'); INSERT INTO t1 VALUES('cd'); INSERT INTO t1 VALUES('ab cd'); |
︙ | ︙ |
Changes to ext/fts5/test/fts5synonym.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on custom tokenizers that support synonyms. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5synonym | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on custom tokenizers that support synonyms. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5synonym # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc tcl_create {args} { return "tcl_tokenize" } |
︙ | ︙ |
Changes to ext/fts5/test/fts5synonym2.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on custom tokenizers that support synonyms. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5synonym2 | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on custom tokenizers that support synonyms. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5synonym2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach tok {query document} { foreach_detail_mode $testprefix { |
︙ | ︙ | |||
38 39 40 41 42 43 44 | } } list [sort_poslist $PL] $CL } sqlite3_fts5_create_function db fts5_test_bothlist fts5_test_bothlist | | | 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | } } list [sort_poslist $PL] $CL } sqlite3_fts5_create_function db fts5_test_bothlist fts5_test_bothlist proc fts5_rowid {cmd} { expr [$cmd xRowid] } sqlite3_fts5_create_function db fts5_rowid fts5_rowid do_execsql_test 1.$tok.0.1 " CREATE VIRTUAL TABLE ss USING fts5(a, b, tokenize='tclnum $tok', detail=%DETAIL%); INSERT INTO ss(ss, rank) VALUES('rank', 'fts5_rowid()'); " |
︙ | ︙ | |||
118 119 120 121 122 123 124 125 126 127 128 129 130 131 | 3.1 "b:one AND a:two" 3.2 "b:one OR a:two" 3.3 "a:one OR b:1 OR {a b} : i" 4.1 "NEAR(one two, 2)" 4.2 "NEAR(one two three, 2)" 4.3 "NEAR(eight nine, 1) OR NEAR(six seven, 1)" } { if {[fts5_expr_ok $expr ss]==0} { do_test 1.$tok.$tn.OMITTED { list } [list] continue } set res [fts5_query_data $expr ss ASC ::tclnum_syn] | > > > | 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 | 3.1 "b:one AND a:two" 3.2 "b:one OR a:two" 3.3 "a:one OR b:1 OR {a b} : i" 4.1 "NEAR(one two, 2)" 4.2 "NEAR(one two three, 2)" 4.3 "NEAR(eight nine, 1) OR NEAR(six seven, 1)" 5.1 "one + two" 5.2 "1 + two" } { if {[fts5_expr_ok $expr ss]==0} { do_test 1.$tok.$tn.OMITTED { list } [list] continue } set res [fts5_query_data $expr ss ASC ::tclnum_syn] |
︙ | ︙ |
Added ext/fts5/test/fts5tokendata.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 | # 2014 Jan 08 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focused on phrase queries. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5tokendata # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { sqlite3_fts5_register_origintext db fts5_aux_test_functions db proc b {x} { string map [list "\0" "."] $x } db func b b do_execsql_test 1.0 { CREATE VIRTUAL TABLE ft USING fts5(a, b, tokendata=1, tokenize="origintext unicode61", detail=%DETAIL% ); CREATE VIRTUAL TABLE vocab USING fts5vocab(ft, instance); } do_execsql_test 1.1 { INSERT INTO ft(rowid, a, b) VALUES (1, 'Pedagog Pedal Pedant', 'Peculier Day Today'), (2, 'Pedant pedantic pecked', 'Peck Penalize Pen'); INSERT INTO ft(rowid, a, b) VALUES (3, 'Penalty Pence Penciled', 'One Two Three'), (4, 'Pedant Pedal Pedant', 'Peculier Day Today'); } do_execsql_test 1.2 { SELECT DISTINCT b(term) FROM vocab } { day.Day one.One peck.Peck pecked peculier.Peculier pedagog.Pedagog pedal.Pedal pedant.Pedant pedantic pen.Pen penalize.Penalize penalty.Penalty pence.Pence penciled.Penciled three.Three today.Today two.Two } do_execsql_test 1.3.1 { SELECT rowid FROM ft('pe*') } { 1 2 3 4 } do_execsql_test 1.3.2 { SELECT rowid FROM ft('pe*') ORDER BY rowid DESC } { 4 3 2 1 } if {"%DETAIL%"!="none"} { do_execsql_test 1.3.3 { SELECT rowid FROM ft WHERE a MATCH 'pe*' ORDER BY rowid DESC } { 4 3 2 1 } } do_execsql_test 1.4 { SELECT rowid, b( fts5_test_insttoken(ft, 0, 0) ) FROM ft('pedant') } { 1 pedant.Pedant 2 pedant.Pedant 4 pedant.Pedant } do_execsql_test 1.5 { SELECT rowid, b( fts5_test_insttoken(ft, 0, 0) ) FROM ft('pe*') } { 1 pedagog.Pedagog 2 pedant.Pedant 3 penalty.Penalty 4 pedant.Pedant } do_execsql_test 1.6 { SELECT rowid, fts5_test_poslist(ft) FROM ft('pe*') } { 1 {0.0.0 0.0.1 0.0.2 0.1.0} 2 {0.0.0 0.0.1 0.0.2 0.1.0 0.1.1 0.1.2} 3 {0.0.0 0.0.1 0.0.2} 4 {0.0.0 0.0.1 0.0.2 0.1.0} } } finish_test |
Changes to ext/fts5/test/fts5tokenizer.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on the built-in fts5 tokenizers. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5tokenizer | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on the built-in fts5 tokenizers. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5tokenizer # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { |
︙ | ︙ | |||
296 297 298 299 300 301 302 | do_execsql_test 9.4.1 { SELECT * FROM t1('"abc xyz" *'); } {} do_test 9.4.2 { set ::flags } {prefixquery} set ::flags [list] do_execsql_test 9.5.1 { SELECT * FROM t1('"abc xyz*"'); } {} do_test 9.5.2 { set ::flags } {query} | > | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 | do_execsql_test 9.4.1 { SELECT * FROM t1('"abc xyz" *'); } {} do_test 9.4.2 { set ::flags } {prefixquery} set ::flags [list] do_execsql_test 9.5.1 { SELECT * FROM t1('"abc xyz*"'); } {} do_test 9.5.2 { set ::flags } {query} #------------------------------------------------------------------------- # reset_db do_execsql_test 10.1 { CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize=unicode61); PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize="unicode61 error");' WHERE name = 'x1'; } db close sqlite3 db test.db do_catchsql_test 10.2 { SELECT * FROM x1('abc'); } {1 {error in tokenizer constructor}} do_catchsql_test 10.3 { INSERT INTO x1 VALUES('abc'); } {1 {error in tokenizer constructor}} do_execsql_test 10.4 { PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize="nosuch error");' WHERE name = 'x1'; } db close sqlite3 db test.db do_catchsql_test 10.5 { SELECT * FROM x1('abc'); } {1 {no such tokenizer: nosuch}} do_catchsql_test 10.6 { INSERT INTO x1 VALUES('abc'); } {1 {no such tokenizer: nosuch}} do_execsql_test 10.7 { DROP TABLE x1; SELECT * FROM sqlite_schema; } reset_db do_execsql_test 10.8 { CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize=unicode61); INSERT INTO x1 VALUES('a b c'), ('d e f'), ('a b c'); CREATE VIRTUAL TABLE x1v USING fts5vocab(x1, row); PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE VIRTUAL TABLE x1 USING fts5(x, tokenize=simplify);' WHERE name = 'x1'; } do_execsql_test 10.9 { SELECT * FROM x1v } { a 2 2 b 2 2 c 2 2 d 1 1 e 1 1 f 1 1 } db close sqlite3 db test.db do_execsql_test 10.10 { SELECT * FROM x1v } { a 2 2 b 2 2 c 2 2 d 1 1 e 1 1 f 1 1 } finish_test |
Added ext/fts5/test/fts5tokenizer2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 | # 2023 Nov 03 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focusing on the built-in fts5 tokenizers. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5tokenizer2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } sqlite3_fts5_create_tokenizer db tst get_tst_tokenizer proc get_tst_tokenizer {args} { return "tst_tokenizer" } proc tst_tokenizer {flags txt} { set token "" set lTok [list] foreach c [split $txt {}] { if {$token==""} { append token $c } else { set t1 [string is upper $token] set t2 [string is upper $c] if {$t1!=$t2} { lappend lTok $token set token "" } append token $c } } if {$token!=""} { lappend lTok $token } set iOff 0 foreach t $lTok { set n [string length $t] sqlite3_fts5_token $t $iOff [expr $iOff+$n] incr iOff $n } } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(t, tokenize=tst); } do_execsql_test 1.1 { INSERT INTO t1 VALUES('AAdontBBmess'); } do_execsql_test 1.2 { SELECT snippet(t1, 0, '>', '<', '...', 4) FROM t1('BB'); } {AAdont>BB<mess} do_execsql_test 1.3 { SELECT highlight(t1, 0, '>', '<') FROM t1('BB'); } {AAdont>BB<mess} do_execsql_test 1.4 { SELECT highlight(t1, 0, '>', '<') FROM t1('AA'); } {>AA<dontBBmess} do_execsql_test 1.5 { SELECT highlight(t1, 0, '>', '<') FROM t1('dont'); } {AA>dont<BBmess} do_execsql_test 1.6 { SELECT highlight(t1, 0, '>', '<') FROM t1('mess'); } {AAdontBB>mess<} do_execsql_test 1.7 { SELECT highlight(t1, 0, '>', '<') FROM t1('BB mess'); } {AAdont>BBmess<} # 2024-08-06 https://sqlite.org/forum/forumpost/171bcc2bcd # Error handling of tokenize= arguments. # foreach {n tkz} { 1 {ascii none} 2 {unicode61 none} 3 {porter none} 4 {trigram none} 5 {ascii none 0} 6 {unicode61 none 0} 7 {porter none 0} 8 {trigram none 0} } { db eval {DROP TABLE IF EXISTS t2;} do_catchsql_test 2.$n " DROP TABLE IF EXISTS t2; CREATE VIRTUAL TABLE t2 USING fts5(a,b,c,tokenize='$tkz'); " {1 {error in tokenizer constructor}} } finish_test |
Added ext/fts5/test/fts5tokenizer3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | # 2024 Aug 10 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # Tests focusing on the built-in fts5 tokenizers. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5tokenizer3 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc get_sod {args} { return "split_on_dot" } proc get_lowercase {args} { return "lowercase" } proc lowercase {flags txt} { set n [string length $txt] sqlite3_fts5_token [string tolower $txt] 0 $n return 0 } proc split_on_dot {flags txt} { set iOff 0 foreach t [split $txt "."] { set n [string length $txt] sqlite3_fts5_token $t $iOff [expr $iOff+$n] incr iOff [expr {$n+1}] } return "" } foreach {tn script} { 1 { sqlite3_fts5_create_tokenizer db lowercase get_lowercase sqlite3_fts5_create_tokenizer -parent lowercase db split_on_dot get_sod } 2 { sqlite3_fts5_create_tokenizer -v2 db lowercase get_lowercase sqlite3_fts5_create_tokenizer -parent lowercase db split_on_dot get_sod } 3 { sqlite3_fts5_create_tokenizer db lowercase get_lowercase sqlite3_fts5_create_tokenizer -v2 -parent lowercase db split_on_dot get_sod } 4 { sqlite3_fts5_create_tokenizer -v2 db lowercase get_lowercase sqlite3_fts5_create_tokenizer -v2 -parent lowercase db split_on_dot get_sod } } { reset_db eval $script do_execsql_test 1.$tn.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, tokenize=split_on_dot); CREATE VIRTUAL TABLE t1vocab USING fts5vocab(t1, instance); INSERT INTO t1 VALUES('ABC.Def.ghi'); } do_execsql_test 1.$tn.1 { SELECT term FROM t1vocab ORDER BY 1 } {abc def ghi} } finish_test |
Changes to ext/fts5/test/fts5trigram.test.
︙ | ︙ | |||
51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 | 2 {cDef%} {} 3 {%f%} 1 4 {%f_h%} 1 5 {%f_g%} {} 6 {abc%klm} 1 7 {ABCDEFG%} 1 8 {%รุงเ%} 2 } { do_execsql_test 1.3.$tn { SELECT rowid FROM t1 WHERE y LIKE $like } $res } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize="trigram case_sensitive 1"); INSERT INTO t1 VALUES('abcdefghijklm'); INSERT INTO t1 VALUES('กรุงเทพมหานคร'); } foreach {tn s res} { 1 abc "(abc)defghijklm" 2 defgh "abc(defgh)ijklm" 3 abcdefghijklm "(abcdefghijklm)" 4 กรุ "(กรุ)งเทพมหานคร" 5 งเทพมห "กรุ(งเทพมห)านคร" | > > > > > | 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | 2 {cDef%} {} 3 {%f%} 1 4 {%f_h%} 1 5 {%f_g%} {} 6 {abc%klm} 1 7 {ABCDEFG%} 1 8 {%รุงเ%} 2 9 {%งเ%} 2 10 {%"งเ"%} {} } { do_execsql_test 1.3.$tn { SELECT rowid FROM t1 WHERE y LIKE $like } $res } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize="trigram case_sensitive 1"); INSERT INTO t1 VALUES('abcdefghijklm'); INSERT INTO t1 VALUES('กรุงเทพมหานคร'); } do_catchsql_test 2.0.1 { CREATE VIRTUAL TABLE t2 USING fts5(z, tokenize='trigram case_sensitive'); } {1 {error in tokenizer constructor}} foreach {tn s res} { 1 abc "(abc)defghijklm" 2 defgh "abc(defgh)ijklm" 3 abcdefghijklm "(abcdefghijklm)" 4 กรุ "(กรุ)งเทพมหานคร" 5 งเทพมห "กรุ(งเทพมห)านคร" |
︙ | ︙ | |||
192 193 194 195 196 197 198 199 200 | } {VIRTUAL TABLE INDEX 0:G0} do_eqp_test 6.3 { SELECT * FROM ci1 WHERE x LIKE ? } {{SCAN ci1 VIRTUAL TABLE INDEX 0:}} do_eqp_test 6.4 { SELECT * FROM ci1 WHERE x GLOB ? } {VIRTUAL TABLE INDEX 0:G0} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 | } {VIRTUAL TABLE INDEX 0:G0} do_eqp_test 6.3 { SELECT * FROM ci1 WHERE x LIKE ? } {{SCAN ci1 VIRTUAL TABLE INDEX 0:}} do_eqp_test 6.4 { SELECT * FROM ci1 WHERE x GLOB ? } {VIRTUAL TABLE INDEX 0:G0} do_eqp_test 6.5 { SELECT * FROM ci1 WHERE x < ? } {{SCAN ci1 VIRTUAL TABLE INDEX 0:}} do_eqp_test 6.6 { SELECT * FROM ci0 WHERE x < ? } {{SCAN ci0 VIRTUAL TABLE INDEX 0:}} reset_db do_execsql_test 7.0 { CREATE VIRTUAL TABLE f USING FTS5(filename, tokenize="trigram"); INSERT INTO f (rowid, filename) VALUES (10, "giraffe.png"), (20, "жираф.png"), (30, "cat.png"), (40, "кот.png"), (50, "misic-ðµ-.mp3"); } do_execsql_test 7.1 { SELECT rowid FROM f WHERE +filename GLOB '*ир*'; } {20} do_execsql_test 7.2 { SELECT rowid FROM f WHERE filename GLOB '*ир*'; } {20} #------------------------------------------------------------------------- reset_db do_execsql_test 8.0 { CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize=trigram); INSERT INTO t1 VALUES('abcdefghijklm'); } foreach {tn match res} { 1 "abc ghi" "(abc)def(ghi)jklm" 2 "def ghi" "abc(defghi)jklm" 3 "efg ghi" "abcd(efghi)jklm" 4 "efghi" "abcd(efghi)jklm" 5 "abcd jklm" "(abcd)efghi(jklm)" 6 "ijkl jklm" "abcdefgh(ijklm)" 7 "ijk ijkl hijk" "abcdefg(hijkl)m" } { do_execsql_test 8.1.$tn { SELECT highlight(t1, 0, '(', ')') FROM t1($match) } $res } do_execsql_test 8.2 { CREATE VIRTUAL TABLE ft2 USING fts5(a, tokenize="trigram"); INSERT INTO ft2 VALUES('abc x cde'); INSERT INTO ft2 VALUES('abc cde'); INSERT INTO ft2 VALUES('abcde'); } do_execsql_test 8.3 { SELECT highlight(ft2, 0, '[', ']') FROM ft2 WHERE ft2 MATCH 'abc AND cde'; } { {[abc] x [cde]} {[abc] [cde]} {[abcde]} } #------------------------------------------------------------------------- reset_db do_execsql_test 9.0 { CREATE VIRTUAL TABLE t1 USING fts5( a0, a1, a2, a3, a4, a5, a6, a7, a8, a9, a10, a11, a12, tokenize=trigram ); INSERT INTO t1(rowid, a12) VALUES(111, 'thats a tricky case though'); INSERT INTO t1(rowid, a12) VALUES(222, 'the query planner cannot do'); } do_execsql_test 9.1 { SELECT rowid FROM t1 WHERE a12 LIKE '%tricky%' } {111} do_execsql_test 9.2 { SELECT rowid FROM t1 WHERE a12 LIKE '%tricky%' AND a12 LIKE '%case%' } {111} do_execsql_test 9.3 { SELECT rowid FROM t1 WHERE a12 LIKE NULL } {} #------------------------------------------------------------------------- reset_db do_execsql_test 10.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, tokenize=trigram); } do_test 10.1 { foreach {val} { "abc \UFFjkl\UFF" "abc \UFFFjkl\UFFF" "abc \UFFFFjkl\UFFFF" "abc \UFFFFFjkl\UFFFFF" "\UFFjkl\UFF abc" "\UFFFjkl\UFFF abc" "\UFFFFjkl\UFFFF abc" "\UFFFFFjkl\UFFFFF abc" "\U10001jkl\U10001 abc" } { execsql { INSERT INTO t1 VALUES( $val ) } } } {} do_test 10.2 { foreach {val} { X'E18000626320646566' X'61EDA0806320646566' X'61EDA0806320646566' X'61EFBFBE6320646566' X'76686920E18000626320646566' X'7668692061EDA0806320646566' X'7668692061EDA0806320646566' X'7668692061EFBFBE6320646566' } { execsql " INSERT INTO t1 VALUES( $val ) " } } {} do_test 10.3 { set a [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0x62}] set b [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0xBF 0x62}] set c [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0xBF 0xBF 0x62}] set d [binary format c* {0x61 0xF7 0xBF 0xBF 0xBF 0xBF 0xBF 0xBF 0x62}] execsql { INSERT INTO t1 VALUES($a); INSERT INTO t1 VALUES($b); INSERT INTO t1 VALUES($c); INSERT INTO t1 VALUES($d); INSERT INTO t1 VALUES('abcd' || $a); INSERT INTO t1 VALUES('abcd' || $b); INSERT INTO t1 VALUES('abcd' || $c); INSERT INTO t1 VALUES('abcd' || $d); } } {} do_execsql_test 11.0 { CREATE VIRTUAL TABLE t4 USING fts5(y, tokenize=trigram); } sqlite3_fts5_register_str db do_execsql_test 11.1 { INSERT INTO t4 VALUES( str('') ); } do_test 12.0 { sqlite3_fts5_tokenize db trigram "abcd" } {abc 0 3 bcd 1 4} do_test 12.1 { sqlite3_fts5_tokenize db trigram "a" } {} do_test 12.2 { sqlite3_fts5_tokenize db trigram "" } {} finish_test |
Added ext/fts5/test/fts5trigram2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 | # 2023 October 24 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # # Tests for the fts5 "trigram" tokenizer. # source [file join [file dirname [info script]] fts5_common.tcl] ifcapable !fts5 { finish_test ; return } set ::testprefix fts5trigram2 do_execsql_test 1.0 " CREATE VIRTUAL TABLE t1 USING fts5(y, tokenize='trigram remove_diacritics 1'); INSERT INTO t1 VALUES('abc\u0303defghijklm'); INSERT INTO t1 VALUES('a\u0303b\u0303c\u0303defghijklm'); " do_catchsql_test 1.0.1 { CREATE VIRTUAL TABLE t2 USING fts5(z, tokenize='trigram remove_diacritics'); } {1 {error in tokenizer constructor}} do_execsql_test 1.1 { SELECT highlight(t1, 0, '(', ')') FROM t1('abc'); } [list \ "(abc\u0303)defghijklm" \ "(a\u0303b\u0303c\u0303)defghijklm" \ ] do_execsql_test 1.2 { SELECT highlight(t1, 0, '(', ')') FROM t1('bcde'); } [list \ "a(bc\u0303de)fghijklm" \ "a\u0303(b\u0303c\u0303de)fghijklm" \ ] do_execsql_test 1.3 { SELECT highlight(t1, 0, '(', ')') FROM t1('cdef'); } [list \ "ab(c\u0303def)ghijklm" \ "a\u0303b\u0303(c\u0303def)ghijklm" \ ] do_execsql_test 1.4 { SELECT highlight(t1, 0, '(', ')') FROM t1('def'); } [list \ "abc\u0303(def)ghijklm" \ "a\u0303b\u0303c\u0303(def)ghijklm" \ ] #------------------------------------------------------------------------- do_catchsql_test 2.0 { CREATE VIRTUAL TABLE t2 USING fts5( z, tokenize='trigram case_sensitive 1 remove_diacritics 1' ); } {1 {error in tokenizer constructor}} do_execsql_test 2.1 { CREATE VIRTUAL TABLE t2 USING fts5( z, tokenize='trigram case_sensitive 0 remove_diacritics 1' ); } do_execsql_test 2.2 " INSERT INTO t2 VALUES('\u00E3bcdef'); INSERT INTO t2 VALUES('b\u00E3cdef'); INSERT INTO t2 VALUES('bc\u00E3def'); INSERT INTO t2 VALUES('bcd\u00E3ef'); " do_execsql_test 2.3 { SELECT highlight(t2, 0, '(', ')') FROM t2('abc'); } "(\u00E3bc)def" do_execsql_test 2.4 { SELECT highlight(t2, 0, '(', ')') FROM t2('bac'); } "(b\u00E3c)def" do_execsql_test 2.5 { SELECT highlight(t2, 0, '(', ')') FROM t2('bca'); } "(bc\u00E3)def" do_execsql_test 2.6 " SELECT highlight(t2, 0, '(', ')') FROM t2('\u00E3bc'); " "(\u00E3bc)def" #------------------------------------------------------------------------- do_execsql_test 3.0 { CREATE VIRTUAL TABLE t3 USING fts5( z, tokenize='trigram remove_diacritics 1' ); } {} do_execsql_test 3.1 " INSERT INTO t3 VALUES ('\u0303abc\u0303'); " do_execsql_test 3.2 { SELECT highlight(t3, 0, '(', ')') FROM t3('abc'); } "\u0303(abc\u0303)" #------------------------------------------------------------------------- do_execsql_test 4.0 { CREATE VIRTUAL TABLE t4 USING fts5(z, tokenize=trigram); } {} do_execsql_test 4.1 { INSERT INTO t4 VALUES('ABCD'); INSERT INTO t4 VALUES('DEFG'); } {} db close sqlite3 db test.db do_eqp_test 4.1 { SELECT rowid FROM t4 WHERE z LIKE '%abc%' } {VIRTUAL TABLE INDEX 0:L0} do_execsql_test 4.2 { SELECT rowid FROM t4 WHERE z LIKE '%abc%' } {1} #------------------------------------------------------------------------- reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE t5 USING fts5( c1, tokenize='trigram', detail='none' ); INSERT INTO t5(rowid, c1) VALUES(1, 'abc_____xyx_yxz'); INSERT INTO t5(rowid, c1) VALUES(2, 'abc_____xyxz'); INSERT INTO t5(rowid, c1) VALUES(3, 'ac_____xyxz'); } {} do_execsql_test 5.1 { SELECT rowid FROM t5 WHERE c1 LIKE 'abc%xyxz' } {2} finish_test |
Changes to ext/fts5/test/fts5ubsan.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # This test is focused on edge cases that cause ubsan errors. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ubsan | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # This test is focused on edge cases that cause ubsan errors. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5ubsan # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE x1 USING fts5(x); |
︙ | ︙ |
Changes to ext/fts5/test/fts5unicode.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # Tests focusing on the fts5 tokenizers # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unicode | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # Tests focusing on the fts5 tokenizers # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unicode # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc tokenize_test {tn tokenizer input output} { uplevel [list do_test $tn [subst -nocommands { |
︙ | ︙ | |||
56 57 58 59 60 61 62 63 64 65 66 67 68 69 | " {t1 t2} #------------------------------------------------------------------------- # Check that codepoints that require 4 bytes to store in utf-8 (those that # require 17 or more bits to store). # set A [db one {SELECT char(0x1F75E)}] ;# Type So set B [db one {SELECT char(0x1F5FD)}] ;# Type So set C [db one {SELECT char(0x2F802)}] ;# Type Lo set D [db one {SELECT char(0x2F808)}] ;# Type Lo do_execsql_test 3.0 " CREATE VIRTUAL TABLE xyz USING fts5(x, | > | 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | " {t1 t2} #------------------------------------------------------------------------- # Check that codepoints that require 4 bytes to store in utf-8 (those that # require 17 or more bits to store). # unset -nocomplain A B C D set A [db one {SELECT char(0x1F75E)}] ;# Type So set B [db one {SELECT char(0x1F5FD)}] ;# Type So set C [db one {SELECT char(0x2F802)}] ;# Type Lo set D [db one {SELECT char(0x2F808)}] ;# Type Lo do_execsql_test 3.0 " CREATE VIRTUAL TABLE xyz USING fts5(x, |
︙ | ︙ |
Changes to ext/fts5/test/fts5unicode2.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # # This is a modified copy of FTS4 test file "fts4_unicode.test". # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unicode2 | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # # This is a modified copy of FTS4 test file "fts4_unicode.test". # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unicode2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc do_unicode_token_test {tn input res} { uplevel [list do_test $tn [list \ |
︙ | ︙ | |||
112 113 114 115 116 117 118 119 120 121 122 123 124 125 | Improvements to the handling of CSV inputs in the command-line shell } { Fix a bug introduced in version 3.7.10 that might cause a LEFT JOIN to be incorrectly converted into an INNER JOIN if the WHERE clause indexable terms connected by OR. }] set map(a) [list "\u00C4" "\u00E4"] ; # LATIN LETTER A WITH DIAERESIS set map(e) [list "\u00CB" "\u00EB"] ; # LATIN LETTER E WITH DIAERESIS set map(i) [list "\u00CF" "\u00EF"] ; # LATIN LETTER I WITH DIAERESIS set map(o) [list "\u00D6" "\u00F6"] ; # LATIN LETTER O WITH DIAERESIS set map(u) [list "\u00DC" "\u00FC"] ; # LATIN LETTER U WITH DIAERESIS set map(y) [list "\u0178" "\u00FF"] ; # LATIN LETTER Y WITH DIAERESIS set map(h) [list "\u1E26" "\u1E27"] ; # LATIN LETTER H WITH DIAERESIS | > | 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | Improvements to the handling of CSV inputs in the command-line shell } { Fix a bug introduced in version 3.7.10 that might cause a LEFT JOIN to be incorrectly converted into an INNER JOIN if the WHERE clause indexable terms connected by OR. }] unset -nocomplain map set map(a) [list "\u00C4" "\u00E4"] ; # LATIN LETTER A WITH DIAERESIS set map(e) [list "\u00CB" "\u00EB"] ; # LATIN LETTER E WITH DIAERESIS set map(i) [list "\u00CF" "\u00EF"] ; # LATIN LETTER I WITH DIAERESIS set map(o) [list "\u00D6" "\u00F6"] ; # LATIN LETTER O WITH DIAERESIS set map(u) [list "\u00DC" "\u00FC"] ; # LATIN LETTER U WITH DIAERESIS set map(y) [list "\u0178" "\u00FF"] ; # LATIN LETTER Y WITH DIAERESIS set map(h) [list "\u1E26" "\u1E27"] ; # LATIN LETTER H WITH DIAERESIS |
︙ | ︙ | |||
466 467 468 469 470 471 472 | SELECT rowid FROM t4 WHERE t4 MATCH 'o' ORDER BY rowid ASC; } {1 3} do_execsql_test 8.2.3 { SELECT rowid FROM t4 WHERE t4 MATCH 'a' ORDER BY rowid ASC; } {2 4} #------------------------------------------------------------------------- | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < | < < | < < < < < | < < < < | < < < < < < | < < | < < < < < < < < < < < | < > | < > | | | | < < < < | | 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 | SELECT rowid FROM t4 WHERE t4 MATCH 'o' ORDER BY rowid ASC; } {1 3} do_execsql_test 8.2.3 { SELECT rowid FROM t4 WHERE t4 MATCH 'a' ORDER BY rowid ASC; } {2 4} #------------------------------------------------------------------------- foreach {tn val bErr} { 1 0 0 2 1 0 3 2 0 4 3 1 5 11 1 } { reset_db set aRes(0) {0 {}} set aRes(1) {1 {error in tokenizer constructor}} set res $aRes($bErr) do_catchsql_test 9.1.$tn " CREATE VIRTUAL TABLE bl USING fts5( s, tokenize='trigram remove_diacritics $val' ); " $res } finish_test |
Changes to ext/fts5/test/fts5unicode3.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #*********************************************************************** # # Tests focusing on the fts5 tokenizers # source [file join [file dirname [info script]] fts5_common.tcl] | | | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 | #*********************************************************************** # # Tests focusing on the fts5 tokenizers # source [file join [file dirname [info script]] fts5_common.tcl] # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } proc fts3_unicode_path {file} { file join .. [file dirname [info script]] .. .. fts3 unicode $file |
︙ | ︙ |
Changes to ext/fts5/test/fts5unicode4.test.
︙ | ︙ | |||
10 11 12 13 14 15 16 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unicode4 | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 | #*********************************************************************** # # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unicode4 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE sss USING fts5(a, prefix=3); } do_execsql_test 1.1 { INSERT INTO sss VALUES('まりや'); } foreach {tn enc tok} { 1 utf-8 ascii 2 utf-16 ascii 3 utf-8 unicode61 4 utf-16 unicode61 } { reset_db do_execsql_test 1.$tn.0 " PRAGMA encoding = '$enc'; CREATE VIRTUAL TABLE vt2 USING fts5(c0, c1, tokenize=$tok); " do_execsql_test 1.$tn.1 { INSERT INTO vt2(c0, c1) VALUES ('bhal', x'17db'); } do_execsql_test 1.$tn.2 { UPDATE vt2 SET c0='bhal'; } do_execsql_test 1.$tn.3 { INSERT INTO vt2(vt2) VALUES('integrity-check') } do_execsql_test 1.$tn.4 { SELECT quote(c1) FROM vt2 } {X'17DB'} } finish_test |
Changes to ext/fts5/test/fts5unindexed.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # The tests in this file focus on "unindexed" columns. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unindexed | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # The tests in this file focus on "unindexed" columns. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unindexed # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.1 { |
︙ | ︙ |
Added ext/fts5/test/fts5unindexed2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 | # 2024 Sep 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # The tests in this file focus on "unindexed" columns in contentless # tables. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5unindexed2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.1 { CREATE VIRTUAL TABLE t1 USING fts5( a, b UNINDEXED, content=, contentless_unindexed=1 ); } {} do_execsql_test 1.2 { INSERT INTO t1 VALUES('abc def', 'ghi jkl'); } do_execsql_test 1.3 { SELECT rowid, a, b FROM t1 } {1 {} {ghi jkl}} do_execsql_test 1.4 { INSERT INTO t1(rowid, a, b) VALUES(11, 'hello world', 'one two three'); } do_execsql_test 1.5 { INSERT INTO t1(t1, rowid, a, b) VALUES('delete', 1, 'abc def', 'ghi jkl'); } do_execsql_test 1.6 { SELECT rowid, a, b FROM t1 } { 11 {} {one two three} } do_execsql_test 1.7 { PRAGMA integrity_check } {ok} do_execsql_test 1.8 { INSERT INTO t1(rowid, a, b) VALUES(12, 'abc def', 'ghi jkl'); } do_execsql_test 1.9 { SELECT rowid, a, b FROM t1('def') } {12 {} {ghi jkl}} do_execsql_test 1.10 { SELECT rowid, a, b FROM t1('def OR hello') ORDER BY rank } {11 {} {one two three} 12 {} {ghi jkl}} do_execsql_test 1.11 { SELECT rowid, a, b FROM t1 WHERE rowid=11 } {11 {} {one two three}} do_execsql_test 1.12 { SELECT rowid, a, b FROM t1 } {11 {} {one two three} 12 {} {ghi jkl}} fts5_aux_test_functions db do_execsql_test 1.12.2 { SELECT rowid, fts5_test_columntext(t1) FROM t1('def OR hello') } {11 {{} {one two three}} 12 {{} {ghi jkl}}} do_execsql_test 1.13 { INSERT INTO t1(t1) VALUES('delete-all'); } do_execsql_test 1.14 { SELECT rowid, a, b FROM t1 } do_execsql_test 1.15 { PRAGMA integrity_check } {ok} do_execsql_test 2.0 { CREATE VIRTUAL TABLE t4 USING fts5( x, y UNINDEXED, z, columnsize=0, content='', contentless_unindexed=1 ); } do_execsql_test 2.1 { INSERT INTO t4(rowid, x, y, z) VALUES(1, 'a a', 'b b b', 'c'); } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE VIRTUAL TABLE x1 USING fts5( a UNINDEXED, b, c UNINDEXED, d, content=, contentless_delete=1, contentless_unindexed=1 ); } do_execsql_test 3.1 { INSERT INTO x1(rowid, a, b, c, d) VALUES(131, 'aaa', 'bbb', 'ccc', 'ddd'); } do_execsql_test 3.2 { SELECT * FROM x1 } {aaa {} ccc {}} do_execsql_test 3.3 { INSERT INTO x1(rowid, a, b, c, d) VALUES(1000, 'AAA', 'BBB', 'CCC', 'DDD'); } do_execsql_test 3.4 { SELECT rowid, * FROM x1 } { 131 aaa {} ccc {} 1000 AAA {} CCC {} } do_execsql_test 3.5 { DELETE FROM x1 WHERE rowid=131; SELECT rowid, * FROM x1 } { 1000 AAA {} CCC {} } do_execsql_test 3.6 { INSERT INTO x1(rowid, a, b, c, d) VALUES(112, 'aaa', 'bbb', 'ccc', 'ddd'); SELECT rowid, * FROM x1 } { 112 aaa {} ccc {} 1000 AAA {} CCC {} } do_execsql_test 3.7 { UPDATE x1 SET b='hello', d='world', rowid=1120 WHERE rowid=112 } do_execsql_test 3.8 { SELECT rowid, * FROM x1 } { 1000 AAA {} CCC {} 1120 aaa {} ccc {} } do_execsql_test 3.9 { SELECT rowid, * FROM x1('hello'); } { 1120 aaa {} ccc {} } do_execsql_test 3.9 { SELECT rowid, * FROM x1('bbb'); } { 1000 AAA {} CCC {} } fts5_aux_test_functions db do_execsql_test 3.10 { SELECT rowid, fts5_test_columntext(x1) FROM x1('b*') } {1000 {AAA {} CCC {}}} #------------------------------------------------------------------------- # Check that if contentless_unindexed=1 is not specified, the values # of UNINDEXED columns are not stored in the database. # # Also check that contentless_unindexed=1 is not allowed unless the table # is actually contentless. # reset_db do_execsql_test 4.0 { CREATE VIRTUAL TABLE ft USING fts5(a, b, c UNINDEXED, content=''); INSERT INTO ft VALUES('one', 'two', 'three'); SELECT rowid, * FROM ft; } {1 {} {} {}} do_execsql_test 4.1 { SELECT name FROM sqlite_schema ORDER BY 1 } { ft ft_config ft_data ft_docsize ft_idx } do_catchsql_test 4.2 { CREATE VIRTUAL TABLE ft2 USING fts5( a, b, c UNINDEXED, contentless_unindexed=1 ); } {1 {contentless_unindexed=1 requires a contentless table}} do_catchsql_test 4.3 { DELETE FROM ft WHERE rowid=1 } {1 {cannot DELETE from contentless fts5 table: ft}} #------------------------------------------------------------------------- # Check that the usual restrictions on contentless tables apply to # contentless_unindexed=1 tables. # reset_db do_execsql_test 5.0 { CREATE VIRTUAL TABLE ft USING fts5( a, b UNINDEXED, c, content='', contentless_unindexed=1 ); INSERT INTO ft VALUES('one', 'two', 'three'); INSERT INTO ft VALUES('four', 'five', 'six'); INSERT INTO ft VALUES('seven', 'eight', 'nine'); SELECT rowid, * FROM ft; } { 1 {} two {} 2 {} five {} 3 {} eight {} } do_execsql_test 5.1 { PRAGMA integrity_check } {ok} do_catchsql_test 5.2 { DELETE FROM ft WHERE rowid=2 } {1 {cannot DELETE from contentless fts5 table: ft}} do_execsql_test 5.3 { SELECT rowid, * FROM ft('six') } { 2 {} five {} } do_catchsql_test 5.4 { UPDATE ft SET a='x', b='y', c='z' WHERE rowid=3 } {1 {cannot UPDATE contentless fts5 table: ft}} fts5_aux_test_functions db do_execsql_test 5.5 { SELECT fts5_test_columntext(ft) FROM ft WHERE rowid=3 } { {{} eight {}} } do_execsql_test 5.6 { SELECT fts5_test_columntext(ft) FROM ft('three'); } { {{} two {}} } #------------------------------------------------------------------------- # Check that it is possible to UPDATE a contentless_unindexed=1 table # if the only columns being modified are UNINDEXED. # # If the contentless_unindexed=1 table is also contentless_delete=1, then # it is also possible to update indexed columns - but only if *all* indexed # columns are updated. # reset_db do_execsql_test 6.0 { CREATE VIRTUAL TABLE ft1 USING fts5(a, b UNINDEXED, c UNINDEXED, d, contentless_unindexed=1, content='' ); INSERT INTO ft1(rowid, a, b, c, d) VALUES (100, 'x y', 'b1', 'c1', 'a b'), (200, 'c d', 'b2', 'c2', 'a b'), (300, 'e f', 'b3', 'c3', 'a b'); } do_execsql_test 6.1 { UPDATE ft1 SET b='b1.1', c='c1.1' WHERE rowid=100; } do_execsql_test 6.2 { UPDATE ft1 SET b='b2.1' WHERE rowid=200; } do_execsql_test 6.3 { UPDATE ft1 SET c='c3.1' WHERE rowid=300; } do_execsql_test 6.4 { SELECT rowid, a, b, c, d FROM ft1 } { 100 {} b1.1 c1.1 {} 200 {} b2.1 c2 {} 300 {} b3 c3.1 {} } finish_test |
Added ext/fts5/test/fts5update2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 | # 2024 Sep 27 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #************************************************************************* # This file implements regression tests for SQLite library. The # focus of this script is testing the FTS5 module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5update2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } #------------------------------------------------------------------------- # Test that the various types of UPDATE statement are handled correctly # by different table types. # foreach_detail_mode $testprefix { foreach {tn cu} { 1 0 2 1 } { reset_db do_execsql_test 1.$tn.1 " CREATE VIRTUAL TABLE ft1 USING fts5(a, b UNINDEXED, c UNINDEXED, d, content='', contentless_unindexed=$cu, detail=%DETAIL% ); CREATE VIRTUAL TABLE ft2 USING fts5(a, b UNINDEXED, c UNINDEXED, d, content='', contentless_unindexed=$cu, contentless_delete=1, detail=%DETAIL% ); " do_execsql_test 1.$tn.2 { INSERT INTO ft1(rowid, a, b, c, d) VALUES(1, 'a1', 'b1', 'c1', 'd1'); INSERT INTO ft1(rowid, a, b, c, d) VALUES(2, 'a2', 'b2', 'c2', 'd2'); INSERT INTO ft1(rowid, a, b, c, d) VALUES(3, 'a3', 'b3', 'c3', 'd3'); INSERT INTO ft2(rowid, a, b, c, d) VALUES(1, 'a1', 'b1', 'c1', 'd1'); INSERT INTO ft2(rowid, a, b, c, d) VALUES(2, 'a2', 'b2', 'c2', 'd2'); INSERT INTO ft2(rowid, a, b, c, d) VALUES(3, 'a3', 'b3', 'c3', 'd3'); } # It should be possible to update a subset of the UNINDEXED columns of # a contentless table. Regardless of whether or not contentless_unindexed=1 # or contentless_delete=1 is set. do_execsql_test 1.$tn.3 { UPDATE ft1 SET b=b||'.1'; UPDATE ft2 SET b=b||'.1'; } do_execsql_test 1.$tn.4 { UPDATE ft1 SET b=b||'.2', c=c||'.2'; UPDATE ft2 SET b=b||'.2', c=c||'.2'; } set res(0) { 1 {} {} {} {} 2 {} {} {} {} 3 {} {} {} {} } set res(1) { 1 {} b1.1.2 c1.2 {} 2 {} b2.1.2 c2.2 {} 3 {} b3.1.2 c3.2 {} } do_execsql_test 1.$tn.5 { SELECT rowid, * FROM ft2 } $res($cu) do_execsql_test 1.6.1 { SELECT rowid FROM ft1('a2') } {2} do_execsql_test 1.6.2 { SELECT rowid FROM ft2('a2') } {2} # It should be possible to update all indexed columns (but no other subset) # if the contentless_delete=1 option is set, as it is for "ft2". do_execsql_test 1.$tn.7 { UPDATE ft2 SET a='a22', d='d22' WHERE rowid=2; } do_execsql_test 1.$tn.8 { SELECT rowid FROM ft2('a22 AND d22') } {2} do_execsql_test 1.$tn.9 { UPDATE ft2 SET a='a33', d='d33', b='b3' WHERE rowid=3; } set res(1) { 1 {} b1.1.2 c1.2 {} 2 {} b2.1.2 c2.2 {} 3 {} b3 c3.2 {} } do_execsql_test 1.$tn.10 { SELECT rowid, * FROM ft2 } $res($cu) do_catchsql_test 1.$tn.11 { UPDATE ft2 SET a='a11' WHERE rowid=1 } {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: ft2}} do_catchsql_test 1.$tn.12 { UPDATE ft2 SET d='d11' WHERE rowid=1 } {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: ft2}} # It is not possible to update the values of indexed columns if # contentless_delete=1 is not set. do_catchsql_test 1.$tn.13 { UPDATE ft1 SET a='a11' WHERE rowid=1 } {1 {cannot UPDATE contentless fts5 table: ft1}} do_catchsql_test 1.$tn.14 { UPDATE ft1 SET d='d11' WHERE rowid=1 } {1 {cannot UPDATE contentless fts5 table: ft1}} # It should be possible to update the rowid if contentless_delete=1 is # set and all indexed columns are updated. do_execsql_test 1.$tn.15 { UPDATE ft2 SET a='aXone', d='dXone', rowid=11 WHERE rowid=1 } set res(0) { 2 {} {} {} {} 3 {} {} {} {} 11 {} {} {} {} } set res(1) { 2 {} b2.1.2 c2.2 {} 3 {} b3 c3.2 {} 11 {} b1.1.2 c1.2 {} } do_execsql_test 1.$tn.16 { SELECT rowid, * FROM ft2 } $res($cu) # Should not be possible to update the rowid of a contentless_delete=1 # table if no indexed columns are updated. do_catchsql_test 1.$tn.17 { UPDATE ft2 SET rowid=12 WHERE rowid=11 } {1 {cannot UPDATE a subset of columns on fts5 contentless-delete table: ft2}} do_catchsql_test 1.$tn.18 { UPDATE ft1 SET rowid=12 WHERE rowid=1 } {1 {cannot UPDATE contentless fts5 table: ft1}} do_execsql_test 1.$tn.19 { UPDATE ft2 SET a='aXtwo', d='dXtwo', c='newval', rowid=12 WHERE rowid=2 } {} set res(0) { 3 {} {} {} {} 11 {} {} {} {} 12 {} {} {} {} } set res(1) { 3 {} b3 c3.2 {} 11 {} b1.1.2 c1.2 {} 12 {} b2.1.2 newval {} } do_execsql_test 1.$tn.20 { SELECT rowid, * FROM ft2 } $res($cu) do_execsql_test 1.$tn.21 { SELECT rowid, * FROM ft2('aXtwo AND dXtwo') } [lrange $res($cu) 10 end] }} ;# end of [foreach_detail_mode] loop finish_test |
Changes to ext/fts5/test/fts5version.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # The tests in this file focus on testing that unrecognized file-format # versions are detected and reported. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5version | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # The tests in this file focus on testing that unrecognized file-format # versions are detected and reported. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5version # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.1 { |
︙ | ︙ | |||
34 35 36 37 38 39 40 | do_execsql_test 1.3 { SELECT rowid FROM t1 WHERE t1 MATCH 'a'; } {1} sqlite3_db_config db DEFENSIVE 0 do_execsql_test 1.4 { | | | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 | do_execsql_test 1.3 { SELECT rowid FROM t1 WHERE t1 MATCH 'a'; } {1} sqlite3_db_config db DEFENSIVE 0 do_execsql_test 1.4 { UPDATE t1_config set v=6 WHERE k='version'; } do_test 1.5 { db close sqlite3 db test.db catchsql { SELECT * FROM t1 WHERE t1 MATCH 'a' } } {1 {invalid fts5 file format (found 6, expected 4 or 5) - run 'rebuild'}} do_test 1.6 { db close sqlite3 db test.db catchsql { INSERT INTO t1 VALUES('x y z') } } {1 {invalid fts5 file format (found 6, expected 4 or 5) - run 'rebuild'}} do_test 1.7 { sqlite3_db_config db DEFENSIVE 0 execsql { DELETE FROM t1_config WHERE k='version' } db close sqlite3 db test.db catchsql { SELECT * FROM t1 WHERE t1 MATCH 'a' } } {1 {invalid fts5 file format (found 0, expected 4 or 5) - run 'rebuild'}} do_test 1.8 { sqlite3_db_config db DEFENSIVE 0 execsql { INSERT INTO t1_config VALUES('version', 4) } execsql { INSERT INTO t1(t1, rank) VALUES('secure-delete', 1) } } {} do_execsql_test 1.10 { SELECT * FROM t1_config } {secure-delete 1 version 4} do_execsql_test 1.11 { INSERT INTO t1(rowid, one) VALUES(123, 'one two three'); DELETE FROM t1 WHERE rowid=123; SELECT * FROM t1_config } {secure-delete 1 version 5} do_execsql_test 1.11 { INSERT INTO t1(t1) VALUES('rebuild'); SELECT * FROM t1_config } {secure-delete 1 version 4} do_execsql_test 1.12 { SELECT * FROM t1_config } {secure-delete 1 version 4} #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE VIRTUAL TABLE xyz USING fts5(x); INSERT INTO xyz(rowid, x) VALUES (1, 'one document'), (2, 'two document'), (3, 'three document'), (4, 'four document'), (5, 'five document'), (6, 'six document'); INSERT INTO xyz(xyz, rank) VALUES('secure-delete', 1); SELECT v FROM xyz_config WHERE k='version'; } {4} do_execsql_test 2.1 { BEGIN; INSERT INTO xyz(rowid, x) VALUES(7, 'seven document'); SAVEPOINT one; DELETE FROM xyz WHERE rowid = 4; } do_execsql_test 2.2 { SELECT v FROM xyz_config WHERE k='version'; } {4} do_execsql_test 2.3 { ROLLBACK TO one; SELECT v FROM xyz_config WHERE k='version'; } {4} do_execsql_test 2.4 { DELETE FROM xyz WHERE rowid = 3; COMMIT; SELECT v FROM xyz_config WHERE k='version'; } {5} finish_test |
Changes to ext/fts5/test/fts5vocab.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # The tests in this file focus on testing the fts5vocab module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5vocab | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # The tests in this file focus on testing the fts5vocab module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5vocab # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } foreach_detail_mode $testprefix { |
︙ | ︙ | |||
509 510 511 512 513 514 515 516 517 518 519 520 521 522 | INSERT INTO ft(a) VALUES('4 5 6'); INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); } do_test 10.6 { set res [list] db eval { SELECT rowid FROM ft('4') } x { db eval { SELECT * FROM t2 } lappend res $x(rowid) } db eval COMMIT | > | 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 | INSERT INTO ft(a) VALUES('4 5 6'); INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); INSERT INTO ft(a) VALUES('1 2 3'); INSERT INTO ft(a) VALUES('4 5 6'); } unset -nocomplain x res do_test 10.6 { set res [list] db eval { SELECT rowid FROM ft('4') } x { db eval { SELECT * FROM t2 } lappend res $x(rowid) } db eval COMMIT |
︙ | ︙ |
Changes to ext/fts5/test/fts5vocab2.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # # The tests in this file focus on testing the fts5vocab module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5vocab2 | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # # The tests in this file focus on testing the fts5vocab module. # source [file join [file dirname [info script]] fts5_common.tcl] set testprefix fts5vocab2 # If SQLITE_ENABLE_FTS5 is not defined, omit this file. ifcapable !fts5 { finish_test return } do_execsql_test 1.0 { CREATE VIRTUAL TABLE t1 USING fts5(a, b); |
︙ | ︙ | |||
276 277 278 279 280 281 282 283 284 285 286 | } {1 {query aborted}} do_catchsql_test 5.2 { DELETE FROM t1 WHERE rowid>100; INSERT INTO t1 SELECT randomblob(3000) FROM v1 } {1 {query aborted}} finish_test | > > > > > > > > > > > > > > > > > > > > > > > > | 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 | } {1 {query aborted}} do_catchsql_test 5.2 { DELETE FROM t1 WHERE rowid>100; INSERT INTO t1 SELECT randomblob(3000) FROM v1 } {1 {query aborted}} #------------------------------------------------------------------------- reset_db sqlite3_fts5_may_be_corrupt 1 do_execsql_test 6.0 { BEGIN TRANSACTION; CREATE VIRTUAL TABLE t1 USING fts5(a,b unindexed,c,tokenize="porter ascii",tokendata=1); REPLACE INTO t1_data VALUES(1,X'03090009'); REPLACE INTO t1_data VALUES(10,X'000000000103030003010101020101030101'); REPLACE INTO t1_data VALUES(137438953473,X'0000002e023061010202010162010203010163010204010167010601020201016801060102030101690106010204040606060808'); REPLACE INTO t1_data VALUES(274877906945,X'0000001f013067020802010202010168020803010203010169020804010204040909'); REPLACE INTO t1_data VALUES(412316860417,X'0000002e023061030202010162030203010163030204010167030601020201016803060102030101690306010204040606060808'); COMMIT; } do_execsql_test 6.1 { CREATE VIRTUAL TABLE t3 USING fts5vocab('t1', 'row'); } do_catchsql_test 6.2 { SELECT * FROM t3; } {1 {database disk image is malformed}} sqlite3_fts5_may_be_corrupt 0 finish_test |
Changes to ext/fts5/tool/mkfts5c.tcl.
1 2 3 4 | #!/bin/sh # restart with tclsh \ exec tclsh "$0" "$@" | | | > > > > > > > > > > > > > > > > > > > > > > > > > > | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 | #!/bin/sh # restart with tclsh \ exec tclsh "$0" "$@" set srcdir [file dirname [file dirname [file normalize [info script]]]] set G(src) [string map [list %dir% $srcdir] { %dir%/fts5.h %dir%/fts5Int.h fts5parse.h fts5parse.c %dir%/fts5_aux.c %dir%/fts5_buffer.c %dir%/fts5_config.c %dir%/fts5_expr.c %dir%/fts5_hash.c %dir%/fts5_index.c %dir%/fts5_main.c %dir%/fts5_storage.c %dir%/fts5_tokenize.c %dir%/fts5_unicode2.c %dir%/fts5_varint.c %dir%/fts5_vocab.c }] set G(hdr) { /* ** This, the "fts5.c" source file, is a composite file that is itself ** assembled from the following files: ** ** fts5.h ** fts5Int.h ** fts5parse.h <--- Generated from fts5parse.y by Lemon ** fts5parse.c <--- Generated from fts5parse.y by Lemon ** fts5_aux.c ** fts5_buffer.c ** fts5_config.c ** fts5_expr.c ** fts5_hash.c ** fts5_index.c ** fts5_main.c ** fts5_storage.c ** fts5_tokenize.c ** fts5_unicode2.c ** fts5_varint.c ** fts5_vocab.c */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5) #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) # define NDEBUG 1 #endif #if defined(NDEBUG) && defined(SQLITE_DEBUG) # undef NDEBUG #endif #ifdef HAVE_STDINT_H #include <stdint.h> #endif #ifdef HAVE_INTTYPES_H #include <inttypes.h> #endif } set G(footer) { /* Here ends the fts5.c composite file. */ #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS5) */ } #------------------------------------------------------------------------- # Read and return the entire contents of text file $zFile from disk. # proc readfile {zFile} { |
︙ | ︙ |
Changes to ext/icu/README.txt.
|
| < | | | | | | | | | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 | This directory contains source code for the SQLite "ICU" extension, an integration of the "International Components for Unicode" library with SQLite. Documentation follows. 1. Features 1.1 SQL Scalars upper() and lower() 1.2 Unicode Aware LIKE Operator 1.3 ICU Collation Sequences 1.4 SQL REGEXP Operator 2. Compilation and Usage 3. Bugs, Problems and Security Issues 3.1 The "case_sensitive_like" Pragma 3.2 The SQLITE_MAX_LIKE_PATTERN_LENGTH Macro 3.3 Collation Sequence Security Issue 1. FEATURES 1.1 SQL Scalars upper() and lower() SQLite's built-in implementations of these two functions only provide case mapping for the 26 letters used in the English language. The ICU based functions provided by this extension provide case mapping, where defined, for the full range of unicode characters. ICU provides two types of case mapping, "general" case mapping and "language specific". Refer to ICU documentation for the differences between the two. Specifically: http://www.icu-project.org/userguide/caseMappings.html http://www.icu-project.org/userguide/posix.html#case_mappings To utilise "general" case mapping, the upper() or lower() scalar functions are invoked with one argument: upper('abc') -> 'ABC' lower('ABC') -> 'abc' To access ICU "language specific" case mapping, upper() or lower() should be invoked with two arguments. The second argument is the name of the locale to use. Passing an empty string ("") or SQL NULL value as the second argument is the same as invoking the 1 argument version of upper() or lower(): lower('I', 'en_us') -> 'i' lower('I', 'tr_tr') -> 'ı' (small dotless i) 1.2 Unicode Aware LIKE Operator Similarly to the upper() and lower() functions, the built-in SQLite LIKE operator understands case equivalence for the 26 letters of the English language alphabet. The implementation of LIKE included in this extension uses the ICU function u_foldCase() to provide case independent comparisons for the full range of unicode characters. The U_FOLD_CASE_DEFAULT flag is passed to u_foldCase(), meaning the dotless 'I' character used in the Turkish language is considered to be in the same equivalence class as the dotted 'I' character used by many languages (including English). 1.3 ICU Collation Sequences A special SQL scalar function, icu_load_collation() is provided that may be used to register ICU collation sequences with SQLite. It is always called with exactly two arguments, the ICU locale identifying the collation sequence to ICU, and the name of the SQLite collation sequence to create. For example, to create an SQLite collation sequence named "turkish" using Turkish language sorting rules, the SQL statement: SELECT icu_load_collation('tr_TR', 'turkish'); Or, for Australian English: SELECT icu_load_collation('en_AU', 'australian'); The identifiers "turkish" and "australian" may then be used as collation sequence identifiers in SQL statements: CREATE TABLE aust_turkish_penpals( australian_penpal_name TEXT COLLATE australian, turkish_penpal_name TEXT COLLATE turkish ); 1.4 SQL REGEXP Operator This extension provides an implementation of the SQL binary comparision operator "REGEXP", based on the regular expression functions provided by the ICU library. The syntax of the operator is as described in SQLite documentation: |
︙ | ︙ | |||
112 113 114 115 116 117 118 | 2 COMPILATION AND USAGE The easiest way to compile and use the ICU extension is to build and use it as a dynamically loadable SQLite extension. To do this using gcc on *nix: | | > > > > > | 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 | 2 COMPILATION AND USAGE The easiest way to compile and use the ICU extension is to build and use it as a dynamically loadable SQLite extension. To do this using gcc on *nix: gcc -fPIC -shared icu.c `pkg-config --libs --cflags icu-io` \ -o libSqliteIcu.so You may need to add "-I" flags so that gcc can find sqlite3ext.h and sqlite3.h. The resulting shared lib, libSqliteIcu.so, may be loaded into sqlite in the same way as any other dynamically loadable extension. As of version 3.48, it can be enabled in the canonical build process by passing one of --with-icu-config or --with-icu-ldflags to the configure script, optionally together with --enable-icu-collations. See the configure --help for more details. 3 BUGS, PROBLEMS AND SECURITY ISSUES 3.1 The "case_sensitive_like" Pragma This extension does not work well with the "case_sensitive_like" |
︙ | ︙ | |||
140 141 142 143 144 145 146 | 3.2 The SQLITE_MAX_LIKE_PATTERN_LENGTH Macro Passing very long patterns to the built-in SQLite LIKE operator can cause excessive CPU usage. To curb this problem, SQLite defines the SQLITE_MAX_LIKE_PATTERN_LENGTH macro as the maximum length of a pattern in bytes (irrespective of encoding). The default value is defined in internal header file "limits.h". | | | | | 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 | 3.2 The SQLITE_MAX_LIKE_PATTERN_LENGTH Macro Passing very long patterns to the built-in SQLite LIKE operator can cause excessive CPU usage. To curb this problem, SQLite defines the SQLITE_MAX_LIKE_PATTERN_LENGTH macro as the maximum length of a pattern in bytes (irrespective of encoding). The default value is defined in internal header file "limits.h". The ICU extension LIKE implementation suffers from the same problem and uses the same solution. However, since the ICU extension code does not include the SQLite file "limits.h", modifying the default value therein does not affect the ICU extension. The default value of SQLITE_MAX_LIKE_PATTERN_LENGTH used by the ICU extension LIKE operator is 50000, defined in source file "icu.c". 3.3 Collation Sequence Security Internally, SQLite assumes that indices stored in database files are sorted according to the collation sequence indicated by the SQL schema. Changing the definition of a collation sequence after an index has been built is therefore equivalent to database corruption. The SQLite library is well tested for robustness in the fact of database corruption. Database corruption may well lead to incorrect answers, but should not cause memory errors. |
Changes to ext/icu/icu.c.
︙ | ︙ | |||
467 468 469 470 471 472 473 | sqlite3 *db = (sqlite3 *)sqlite3_user_data(p); UErrorCode status = U_ZERO_ERROR; const char *zLocale; /* Locale identifier - (eg. "jp_JP") */ const char *zName; /* SQL Collation sequence name (eg. "japanese") */ UCollator *pUCollator; /* ICU library collation object */ int rc; /* Return code from sqlite3_create_collation_x() */ | | > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > | 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 | sqlite3 *db = (sqlite3 *)sqlite3_user_data(p); UErrorCode status = U_ZERO_ERROR; const char *zLocale; /* Locale identifier - (eg. "jp_JP") */ const char *zName; /* SQL Collation sequence name (eg. "japanese") */ UCollator *pUCollator; /* ICU library collation object */ int rc; /* Return code from sqlite3_create_collation_x() */ assert(nArg==2 || nArg==3); (void)nArg; /* Unused parameter */ zLocale = (const char *)sqlite3_value_text(apArg[0]); zName = (const char *)sqlite3_value_text(apArg[1]); if( !zLocale || !zName ){ return; } pUCollator = ucol_open(zLocale, &status); if( !U_SUCCESS(status) ){ icuFunctionError(p, "ucol_open", status); return; } assert(p); if(nArg==3){ const char *zOption = (const char*)sqlite3_value_text(apArg[2]); static const struct { const char *zName; UColAttributeValue val; } aStrength[] = { { "PRIMARY", UCOL_PRIMARY }, { "SECONDARY", UCOL_SECONDARY }, { "TERTIARY", UCOL_TERTIARY }, { "DEFAULT", UCOL_DEFAULT_STRENGTH }, { "QUARTERNARY", UCOL_QUATERNARY }, { "IDENTICAL", UCOL_IDENTICAL }, }; unsigned int i; for(i=0; i<sizeof(aStrength)/sizeof(aStrength[0]); i++){ if( sqlite3_stricmp(zOption,aStrength[i].zName)==0 ){ ucol_setStrength(pUCollator, aStrength[i].val); break; } } if( i>=sizeof(aStrength)/sizeof(aStrength[0]) ){ sqlite3_str *pStr = sqlite3_str_new(sqlite3_context_db_handle(p)); sqlite3_str_appendf(pStr, "unknown collation strength \"%s\" - should be one of:", zOption); for(i=0; i<sizeof(aStrength)/sizeof(aStrength[0]); i++){ sqlite3_str_appendf(pStr, " %s", aStrength[i].zName); } sqlite3_result_error(p, sqlite3_str_value(pStr), -1); sqlite3_free(sqlite3_str_finish(pStr)); return; } } rc = sqlite3_create_collation_v2(db, zName, SQLITE_UTF16, (void *)pUCollator, icuCollationColl, icuCollationDel ); if( rc!=SQLITE_OK ){ ucol_close(pUCollator); sqlite3_result_error(p, "Error registering collation function", -1); } |
︙ | ︙ | |||
505 506 507 508 509 510 511 512 513 514 515 516 517 518 | const char *zName; /* Function name */ unsigned char nArg; /* Number of arguments */ unsigned int enc; /* Optimal text encoding */ unsigned char iContext; /* sqlite3_user_data() context */ void (*xFunc)(sqlite3_context*,int,sqlite3_value**); } scalars[] = { {"icu_load_collation",2,SQLITE_UTF8|SQLITE_DIRECTONLY,1, icuLoadCollation}, #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) {"regexp", 2, SQLITE_ANY|SQLITEICU_EXTRAFLAGS, 0, icuRegexpFunc}, {"lower", 1, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"lower", 2, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"upper", 1, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"upper", 2, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"lower", 1, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, | > | 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 | const char *zName; /* Function name */ unsigned char nArg; /* Number of arguments */ unsigned int enc; /* Optimal text encoding */ unsigned char iContext; /* sqlite3_user_data() context */ void (*xFunc)(sqlite3_context*,int,sqlite3_value**); } scalars[] = { {"icu_load_collation",2,SQLITE_UTF8|SQLITE_DIRECTONLY,1, icuLoadCollation}, {"icu_load_collation",3,SQLITE_UTF8|SQLITE_DIRECTONLY,1, icuLoadCollation}, #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_ICU) {"regexp", 2, SQLITE_ANY|SQLITEICU_EXTRAFLAGS, 0, icuRegexpFunc}, {"lower", 1, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"lower", 2, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, {"upper", 1, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"upper", 2, SQLITE_UTF16|SQLITEICU_EXTRAFLAGS, 1, icuCaseFunc16}, {"lower", 1, SQLITE_UTF8|SQLITEICU_EXTRAFLAGS, 0, icuCaseFunc16}, |
︙ | ︙ | |||
534 535 536 537 538 539 540 | p->xFunc, 0, 0 ); } return rc; } | | | 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 | p->xFunc, 0, 0 ); } return rc; } #ifndef SQLITE_CORE #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_icu_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi |
︙ | ︙ |
Added ext/intck/intck1.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 | # 2008 Feb 19 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # The focus of this file is testing the incremental integrity check # (intck) extension. # source [file join [file dirname [info script]] intck_common.tcl] set testprefix intck1 return_if_no_intck foreach {tn sql} { 1 "CREATE TABLE t1(a PRIMARY KEY, b)" 2 "CREATE TABLE t2(a PRIMARY KEY, b) WITHOUT ROWID " 3 "CREATE TABLE t3(a PRIMARY KEY, b) WITHOUT rowID;" 4 "CREATE TABLE t4(a PRIMARY KEY, ROWID)" 5 {CREATE TABLE t5(a PRIMARY KEY, ROWID) WITHOUT ROWID } } { do_test 1.1.$tn { db eval $sql set {} {} } {} } set space " \n\v\t\r\f" do_execsql_test 1.2 { SELECT name, (rtrim(sql, $space) LIKE '%rowid') FROM sqlite_schema WHERE type='table' ORDER BY 1 } { t1 0 t2 1 t3 1 t4 0 t5 1 } do_execsql_test 1.3 { CREATE TABLE x1(a COLLATE nocase, b INTEGER, c BLOB); INSERT INTO x1 VALUES('lEtTeRs', 1234, 1234); } do_execsql_test 1.3.1 { WITH wrapper(c1, c2, c3) AS ( SELECT a, b, c FROM x1 ) SELECT * FROM wrapper WHERE c1='letters'; } {lEtTeRs 1234 1234} do_execsql_test 1.3.2 { WITH wrapper(c1, c2, c3) AS ( SELECT a, b, c FROM x1 ) SELECT * FROM wrapper WHERE c2='1234'; } {lEtTeRs 1234 1234} do_execsql_test 1.3.2 { WITH wrapper(c1, c2, c3) AS ( SELECT a, b, c FROM x1 ) SELECT * FROM wrapper WHERE c3='1234'; } {} do_execsql_test 1.4 { CREATE TABLE z1(a, b); CREATE INDEX z1ab ON z1(a+b COLLATE nocase); } do_execsql_test 1.4.1 { SELECT * FROM z1 INDEXED BY z1ab } do_catchsql_test 1.5.1 { CREATE INDEX z1b ON z1(b ASC NULLS LAST); } {1 {unsupported use of NULLS LAST}} do_catchsql_test 1.5.2 { CREATE INDEX z1b ON z1(b DESC NULLS LAST); } {1 {unsupported use of NULLS LAST}} do_catchsql_test 1.5.3 { CREATE INDEX z1b ON z1(b ASC NULLS FIRST); } {1 {unsupported use of NULLS FIRST}} do_catchsql_test 1.5.4 { CREATE INDEX z1b ON z1(b DESC NULLS FIRST); } {1 {unsupported use of NULLS FIRST}} reset_db do_execsql_test 1.6.1 { CREATE TABLE t1(i INTEGER PRIMARY KEY, b, c); CREATE INDEX i1 ON t1(b); ANALYZE; INSERT INTO sqlite_stat1 VALUES('t1', 'i1', '10000 10000'); ANALYZE sqlite_schema; } {} do_eqp_test 1.6.2 { SELECT 1 FROM t1 INDEXED BY i1 WHERE (b, i) IS (?, ?); } {SEARCH} #------------------------------------------------------------------------- reset_db do_test 2.0 { set ic [sqlite3_intck db main] $ic close } {} do_execsql_test 2.1 { CREATE TABLE t1(a, b); INSERT INTO t1 VALUES(1, 2); INSERT INTO t1 VALUES(3, 4); CREATE INDEX i1 ON t1(a COLLATE nocase); CREATE INDEX i2 ON t1(b, a); CREATE INDEX i3 ON t1(b + a COLLATE nocase) WHERE a!=1; } do_intck_test 2.2 { } # Delete a row from each of the i1 and i2 indexes using the imposter # table interface. # do_test 2.3 { db eval {SELECT name, rootpage FROM sqlite_schema} { set R($name) $rootpage } sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 1 $R(i1) db eval { CREATE TABLE imp1(a PRIMARY KEY, rowid) WITHOUT ROWID; } sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 1 $R(i2) db eval { CREATE TABLE imp2(b, a, rowid, PRIMARY KEY(b, a)) WITHOUT ROWID; } sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 0 0 db eval { DELETE FROM imp1 WHERE rowid=1; DELETE FROM imp2 WHERE rowid=2; } db close sqlite3 db test.db } {} do_intck_test 2.4 { {entry (1,1) missing from index i1} {entry (4,3,2) missing from index i2} } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE TABLE x1(a, b, c, PRIMARY KEY(c, b)) WITHOUT ROWID; CREATE INDEX x1a ON x1(a COLLATE nocase); INSERT INTO x1 VALUES(1, 2, 'three'); INSERT INTO x1 VALUES(4, 5, 'six'); INSERT INTO x1 VALUES(7, 8, 'nine'); } do_intck_test 3.1 { } do_test 3.2 { db eval {SELECT name, rootpage FROM sqlite_schema} { set R($name) $rootpage } sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 1 $R(x1a) db eval { CREATE TABLE imp1(c, b, a, PRIMARY KEY(c, b)) WITHOUT ROWID } sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER db main 0 0 db eval { DELETE FROM imp1 WHERE a=5; } execsql_pp { } db close sqlite3 db test.db } {} do_intck_test 3.3 { {entry (4,'six',5) missing from index x1a} } #------------------------------------------------------------------------- reset_db do_execsql_test 4.0 { CREATE TABLE www(x, y, z); CREATE INDEX w1 ON www( (x+1), z ); INSERT INTO www VALUES(1, 1, 1), (2, 2, 2); } do_intck_test 4.1 { } #------------------------------------------------------------------------- reset_db do_execsql_test 5.0 { CREATE TABLE t1(a, b); CREATE INDEX i1 ON t1(a COLLATE NOCASE); INSERT INTO t1 VALUES(1, 1); INSERT INTO t1 VALUES(2, 2); } do_test 5.1 { set ic [sqlite3_intck db nosuchdb] $ic step } {SQLITE_ERROR} do_test 5.2 { $ic close set ic [sqlite3_intck db {}] while {[$ic step]=="SQLITE_OK"} {} set res [$ic error] $ic close set res } {SQLITE_OK {}} do_test 5.3 { test_do_intck db "main" } {} do_test 5.4 { set ret {} set ic [sqlite3_intck db main] db eval [$ic test_sql t1] { if {$error_message!=""} { lappend ret $error_message } } $ic close set ret } {} do_test 5.5 { set ret {} set ic [sqlite3_intck db main] db eval [$ic test_sql {}] { if {$error_message!=""} { lappend ret $error_message } } $ic close set ret } {} db cache flush do_test 5.6 { set ret {} set ic [sqlite3_intck db main] $ic step db eval [$ic test_sql {}] { if {$error_message!=""} { lappend ret $error_message } } $ic close set ret } {} #------------------------------------------------------------------------- reset_db do_execsql_test 6.0 { CREATE TABLE t1(x, y, PRIMARY KEY(x)) WITHOUT ROWID; CREATE INDEX i1 ON t1(y, x); INSERT INTO t1 VALUES(X'0000', X'1111'); } do_intck_test 6.1 {} do_execsql_test 6.2.1 { PRAGMA writable_schema = 1; UPDATE sqlite_schema SET sql = 'CREATE INDEX i1' WHERE name='i1'; } {} do_intck_test 6.2.2 {} do_execsql_test 6.3.1 { UPDATE sqlite_schema SET sql = 'CREATE INDEX i1(y' WHERE name='i1'; } {} do_intck_test 6.3.2 {} do_execsql_test 6.4.1 { UPDATE sqlite_schema SET sql = 'CREATE INDEX i1(y) hello world' WHERE name='i1'; } {} do_intck_test 6.4.2 {} do_execsql_test 6.5.1 { UPDATE sqlite_schema SET sql = 'CREATE INDEX i1(y, x) WHERE 1 ' WHERE name='i1'; } {} do_intck_test 6.5.2 {} do_execsql_test 6.6.1 { UPDATE sqlite_schema SET sql = 'CREATE INDEX i1( , ) WHERE 1 ' WHERE name='i1'; } {} do_test 6.7.2 { set ic [sqlite3_intck db main] $ic step } {SQLITE_ERROR} do_test 6.5.3 { $ic error } {SQLITE_ERROR {near "AS": syntax error}} $ic close do_execsql_test 6.6.1 { UPDATE sqlite_schema SET sql = 'CREATE INDEX i1([y' WHERE name='i1'; } {} do_intck_test 6.6.2 {} #------------------------------------------------------------------------- reset_db do_execsql_test 7.0 { CREATE TABLE x1("1", "22", "3333", four); CREATE INDEX i1 ON x1( "1" , "22", NULL); INSERT INTO x1 VALUES(1, 22, 3333, NULL); INSERT INTO x1 VALUES(1, 22, 3333, NULL); } do_execsql_test 7.1 " CREATE INDEX i2 ON x1( \"1\"\r\n\t ) " do_execsql_test 7.2 { CREATE INDEX i3 ON x1( "22" || 'abc''def' || `1` ) } do_execsql_test 7.3 { CREATE INDEX i4 ON x1( [22] + [1] ) } do_execsql_test 7.4 { CREATE INDEX i5 ON x1( four||'hello' ) } do_intck_test 7.5 {} finish_test |
Added ext/intck/intck2.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 | # 2024 Feb 19 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # The focus of this file is testing the incremental integrity check # (intck) extension. # source [file join [file dirname [info script]] intck_common.tcl] set testprefix intck2 return_if_no_intck do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT); INSERT INTO t1 VALUES(1, 'one'); INSERT INTO t1 VALUES(2, 'two'); INSERT INTO t1 VALUES(3, 'three'); CREATE INDEX i1 ON t1(b); } proc imposter_edit {obj create sql} { sqlite3 xdb test.db set pgno [xdb one {SELECT rootpage FROM sqlite_schema WHERE name=$obj}] sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER xdb main 1 $pgno xdb eval $create sqlite3_test_control SQLITE_TESTCTRL_IMPOSTER xdb main 0 0 xdb eval $sql xdb close } imposter_edit i1 { CREATE TABLE imp(b, a, PRIMARY KEY(b)) WITHOUT ROWID; } { DELETE FROM imp WHERE b='two'; INSERT INTO imp(b, a) VALUES('four', 4); } do_intck_test 1.1 { {surplus entry ('four',4) in index i1} {entry ('two',2) missing from index i1} } #------------------------------------------------------------------------- reset_db do_execsql_test 2.0 { CREATE TABLE x1(a, b, "c d"); CREATE INDEX x1a ON x1(a COLLATE nocase DESC , b ASC); CREATE INDEX x1b ON x1( a || b || ' "''" ' COLLATE binary ASC ); CREATE INDEX x1c ON x1( format('%s', a)ASC, format('%d', "c d" ) ); INSERT INTO x1 VALUES('one', 2, 3); INSERT INTO x1 VALUES('One', 4, 5); INSERT INTO x1 VALUES('ONE', 6, 7); INSERT INTO x1 VALUES(NULL, NULL, NULL); } do_intck_test 2.1 {} imposter_edit x1 { CREATE TABLE imp(a, b, c); } { DELETE FROM imp WHERE c=7; } do_intck_test 2.2 { {surplus entry ('ONE',6,3) in index x1a} {surplus entry ('ONE6 "''" ',3) in index x1b} {surplus entry ('ONE','7',3) in index x1c} } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { CREATE TABLE x1(a, b, c); CREATE INDEX x1all ON x1(a DESC, b ASC, c DESC); INSERT INTO x1 VALUES(2, 1, 2); INSERT INTO x1 VALUES(2, 1, 1); INSERT INTO x1 VALUES(2, 2, 2); INSERT INTO x1 VALUES(2, 2, 1); INSERT INTO x1 VALUES(1, 1, 2); INSERT INTO x1 VALUES(1, 1, 1); INSERT INTO x1 VALUES(1, 2, 2); INSERT INTO x1 VALUES(1, 2, 1); } do_intck_test 3.1 { } imposter_edit x1 { CREATE TABLE imp(a, b, c); } { DELETE FROM imp WHERE 1; } db close sqlite3 db test.db do_intck_test 3.2 { {surplus entry (2,1,2,1) in index x1all} {surplus entry (2,1,1,2) in index x1all} {surplus entry (2,2,2,3) in index x1all} {surplus entry (2,2,1,4) in index x1all} {surplus entry (1,1,2,5) in index x1all} {surplus entry (1,1,1,6) in index x1all} {surplus entry (1,2,2,7) in index x1all} {surplus entry (1,2,1,8) in index x1all} } do_execsql_test 3.3 { DELETE FROM x1; INSERT INTO x1 VALUES(NULL, NULL, NULL); INSERT INTO x1 VALUES(NULL, NULL, NULL); INSERT INTO x1 VALUES(NULL, NULL, NULL); INSERT INTO x1 VALUES(NULL, NULL, NULL); } do_intck_test 3.4 { } imposter_edit x1 { CREATE TABLE imp(a, b, c); } { DELETE FROM imp WHERE 1; INSERT INTO imp(rowid) VALUES(-123); INSERT INTO imp(rowid) VALUES(456); } db close sqlite3 db test.db do_intck_test 3.5 { {entry (NULL,NULL,NULL,-123) missing from index x1all} {entry (NULL,NULL,NULL,456) missing from index x1all} {surplus entry (NULL,NULL,NULL,1) in index x1all} {surplus entry (NULL,NULL,NULL,2) in index x1all} {surplus entry (NULL,NULL,NULL,3) in index x1all} {surplus entry (NULL,NULL,NULL,4) in index x1all} } reset_db do_execsql_test 3.6 { CREATE TABLE w1(a PRIMARY KEY, b, c); INSERT INTO w1 VALUES(1.0, NULL, NULL); INSERT INTO w1 VALUES(33.0, NULL, NULL); INSERT INTO w1 VALUES(100.0, NULL, NULL); CREATE INDEX w1bc ON w1(b, c); } do_intck_test 3.7 { } imposter_edit w1 { CREATE TABLE imp(a, b, c); } { DELETE FROM imp WHERE a=33; INSERT INTO imp(a) VALUES(1234.5); INSERT INTO imp(a) VALUES(-1234.5); } do_intck_test 3.8 { {surplus entry (33.0,2) in index sqlite_autoindex_w1_1} {entry (1234.5,4) missing from index sqlite_autoindex_w1_1} {entry (NULL,NULL,4) missing from index w1bc} {entry (-1234.5,5) missing from index sqlite_autoindex_w1_1} {entry (NULL,NULL,5) missing from index w1bc} {surplus entry (NULL,NULL,2) in index w1bc} } finish_test |
Added ext/intck/intck_common.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 | # 2024 Feb 18 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # if {![info exists testdir]} { set testdir [file join [file dirname [info script]] .. .. test] } source $testdir/tester.tcl ifcapable !vtab||!pragma { proc return_if_no_intck {} { finish_test return -code return } return } else { proc return_if_no_intck {} {} } proc do_intck {db {bSuspend 0}} { set ic [sqlite3_intck $db main] set ret [list] while {"SQLITE_OK"==[$ic step]} { set msg [$ic message] if {$msg!=""} { lappend ret $msg } if {$bSuspend} { $ic unlock #puts "SQL: [$ic test_sql {}]" #execsql_pp "EXPLAIN query plan [$ic test_sql {}]" #explain_i [$ic test_sql {}] } } set err [$ic error] if {[lindex $err 0]!="SQLITE_OK"} { error $err } $ic close return $ret } proc intck_sql {db tbl} { set ic [sqlite3_intck $db main] set sql [$ic test_sql $tbl] $ic close return $sql } proc do_intck_test {tn expect} { uplevel [list do_test $tn.a [list do_intck db] [list {*}$expect]] uplevel [list do_test $tn.b [list do_intck db 1] [list {*}$expect]] } |
Added ext/intck/intckbusy.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | # 2024 February 24 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] intck_common.tcl] set testprefix intckbusy return_if_no_intck do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(2, 'two', 'three'); INSERT INTO t1 VALUES(3, NULL, NULL); CREATE INDEX i1 ON t1(b, c); } sqlite3 db2 test.db do_execsql_test -db db2 1.1 { BEGIN EXCLUSIVE; INSERT INTO t1 VALUES(4, 5, 6); } do_test 1.2 { set ic [sqlite3_intck db main] $ic step } {SQLITE_BUSY} do_test 1.3 { $ic unlock } {SQLITE_BUSY} do_test 1.4 { $ic error } {SQLITE_BUSY {database is locked}} do_test 1.4 { $ic close } {} finish_test |
Added ext/intck/intckcorrupt.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 | # 2024 Feb 21 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # The focus of this file is testing the intck extensions response # to corruption at the b-tree level. # source [file join [file dirname [info script]] intck_common.tcl] set testprefix intckcorrupt return_if_no_intck #------------------------------------------------------------------------- reset_db do_test 1.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 356352 pagesize 4096 filename crash-acaae0347204ae.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 d0 00 00 00 .....@ ........ | 32: 40 00 ea 00 00 00 00 00 00 40 00 00 00 40 00 00 @........@...@.. | 96: 00 00 00 00 0d 00 00 00 04 0e 9c 00 0f ad 0f 4f ...............O | 112: 0e fc 0e 9c 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 3728: 00 00 00 00 00 00 00 00 00 00 00 00 5e 04 07 17 ............^... | 3744: 1f 1f 01 81 0b 74 61 62 6c 65 74 31 5f 70 61 72 .....tablet1_par | 3760: 65 6e 74 74 31 5f 70 61 72 65 6e 74 04 43 52 45 entt1_parent.CRE | 3776: 41 54 45 20 54 41 42 4c 45 20 22 74 31 5f 70 61 ATE TABLE .t1_pa | 3792: 72 65 6e 74 22 28 6e 6f 64 65 6e 6f 20 49 4e 54 rent.(nodeno INT | 3808: 45 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 EGER PRIMARY KEY | 3824: 2c 70 61 72 65 6e 74 6e 6f 64 65 29 51 03 06 17 ,parentnode)Q... | 3840: 1b 1b 01 7b 74 61 62 6c 65 74 31 5f 6e 6f 64 65 ....tablet1_node | 3856: 74 31 5f 6e 6f 64 65 03 43 52 45 41 54 45 20 54 t1_node.CREATE T | 3872: 41 42 4c 45 20 22 74 31 5f 6e 6f 64 65 22 28 6e ABLE .t1_node.(n | 3888: 6f 64 65 6e 6f 20 49 4e 54 45 47 45 52 20 50 52 odeno INTEGER PR | 3904: 49 4d 41 52 59 20 4b 45 59 2c 64 61 74 61 29 5c IMARY KEY,data). | 3920: 02 07 17 1d 1d 01 81 0b 74 61 62 6c 65 74 31 5f ........tablet1_ | 3936: 72 6f 77 69 64 74 31 5f 72 6f 77 69 64 02 43 52 rowidt1_rowid.CR | 3952: 45 41 54 45 20 54 41 42 4c 45 20 22 74 31 5f 72 EATE TABLE .t1_r | 3968: 6f 77 69 64 22 28 72 6f 77 69 64 20 49 4e 54 45 owid.(rowid INTE | 3984: 47 45 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c GER PRIMARY KEY, | 4000: 6e 6f 64 65 6e 6f 2c 61 30 2c 61 31 29 51 01 07 nodeno,a0,a1)Q.. | 4016: 17 11 11 08 81 0f 74 61 62 6c 65 74 31 74 31 43 ......tablet1t1C | 4032: 52 45 41 54 45 20 56 49 52 54 55 41 4c 20 54 41 REATE VIRTUAL TA | 4048: 42 4c 45 20 74 31 20 55 53 49 4e 47 20 72 74 72 BLE t1 USING rtr | 4064: 65 65 28 69 64 2c 78 30 20 50 52 49 4d 41 52 59 ee(id,x0 PRIMARY | 4080: 20 4b 45 59 2c 70 61 72 65 6e 74 6e 6f 64 65 29 KEY,parentnode) | page 2 offset 4096 | 0: 51 03 06 17 1b 1b 01 7b 74 61 62 6c 65 74 31 5f Q.......tablet1_ | 16: 6e 6f 64 65 74 31 5f 6e 6f 64 65 03 43 52 45 41 nodet1_node.CREA | 32: 54 45 20 54 41 42 4c 45 20 22 74 31 5f 6e 6f 64 TE TABLE .t1_nod | 48: 65 22 28 6e 6f 64 65 6e 6f 20 49 4e 54 45 47 45 e.(nodeno INTEGE | 64: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 64 61 R PRIMARY KEY,da | 80: 74 61 29 5c 02 07 17 1d 1d 01 81 0b 74 61 62 6c ta).........tabl | 96: 65 74 31 5f 72 6f 77 69 64 74 31 5f 72 6f 77 69 et1_rowidt1_rowi | 112: 64 02 43 52 45 41 54 45 20 54 41 42 4c 45 00 00 d.CREATE TABLE.. | 128: 01 0a 02 00 00 00 01 0e 0d 00 00 00 00 24 0e 0d .............$.. | 144: 0c 1a 06 85 50 46 60 27 70 08 00 00 00 00 00 00 ....PF`'p....... | 3824: 00 00 00 00 00 00 00 0d 0e 05 00 09 1d 00 74 6f ..............to | 3840: 79 20 68 61 6c 66 10 0d 05 00 09 23 00 62 6f 74 y half.....#.bot | 3856: 74 6f 6d 20 68 61 6c 66 0f 0c 05 00 09 21 00 72 tom half.....!.r | 3872: 69 67 68 74 20 68 61 6c 66 0e 0b 05 00 09 1f 00 ight half....... | 3888: 6c 65 66 74 20 43 15 f6 e6 f6 46 50 34 35 24 54 left C....FP45$T | 3904: 15 44 52 05 44 14 24 c4 52 02 27 43 15 f6 e6 f6 .DR.D.$.R.'C.... | 3920: 46 52 22 8e 6f 64 65 6e 6f 20 49 4e 54 45 47 45 FR..odeno INTEGE | 3936: 52 20 50 52 49 4d 41 52 59 20 4b 45 59 2c 64 61 R PRIMARY KEY,da | 3952: 74 61 29 5c 02 07 17 1d 1d 01 81 0b 74 61 62 6c ta).........tabl | 3968: 65 74 31 5f 72 6f 74 74 6f 6d 20 65 64 67 65 0f et1_rottom edge. | 3984: 07 05 00 09 21 00 72 69 67 68 74 20 65 64 67 65 ....!.right edge | 4000: 0e 06 05 00 09 1f 00 6c 65 66 74 20 65 64 67 65 .......left edge | 4016: 0b 05 05 00 09 19 00 63 65 6e 74 65 72 17 04 05 .......center... | 4032: 00 09 31 00 75 70 70 65 72 2d 72 69 67 68 74 20 ..1.upper-right | 4048: 63 6f 72 6e 65 72 17 03 05 00 09 31 00 6c 6f 77 corner.....1.low | 4064: 65 72 2d 72 69 67 68 74 20 63 6f 72 6e 65 72 16 er-right corner. | 4080: 02 05 00 09 2f 00 75 70 70 65 72 2d 6c 65 66 74 ..../.upper-left | page 3 offset 8192 | 0: 20 63 6f 72 6e 65 72 16 01 05 00 09 2f 01 8c 6f corner...../..o | 16: 77 65 72 2d 6c 53 51 4c 69 74 65 20 66 6f 72 6d wer-lSQLite form | 32: 61 74 20 33 00 10 00 01 01 00 40 20 20 00 00 00 at 3......@ ... | 48: 00 00 00 00 2f 00 00 0d eb 13 00 00 00 03 00 00 ..../........... | 64: 00 04 00 00 00 00 00 00 00 06 00 00 00 01 00 00 ................ | 80: 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 00 ................ | page 6 offset 20480 | 128: 00 00 00 00 00 00 00 00 97 3d 04 ae 7c 01 00 00 .........=..|... | 624: 00 00 00 00 00 00 21 97 3d 04 ae 7c 01 00 00 00 ......!.=..|.... | 1120: 00 00 00 00 00 20 97 3d 04 ae 7c 01 00 00 00 00 ..... .=..|..... | 1616: 00 00 00 00 1f 97 3d 04 ae 7c 01 00 00 00 00 00 ......=..|...... | 2112: 00 00 00 1e 97 3d 04 ae 7c 01 00 00 00 00 00 00 .....=..|....... | 2608: 00 00 1d 97 d3 d0 4a e7 c0 00 00 00 00 00 00 00 ......J......... | 3088: 00 00 00 00 00 00 00 00 00 00 00 00 01 f3 00 00 ................ | 3600: 23 97 3d 04 ae 7c 01 00 00 00 00 00 00 00 00 00 #.=..|.......... | 4080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 26 ...............& | page 8 offset 28672 | 0: 0d 00 00 00 01 04 30 00 04 30 00 00 00 00 00 00 ......0..0...... | 1072: 97 4d 1e 14 00 ae 7c 00 00 00 00 00 00 00 00 00 .M....|......... | 1088: 00 00 00 00 00 00 01 00 00 00 00 00 00 00 00 00 ................ | 4080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 03 ................ | page 10 offset 36864 | 0: 0d 00 00 00 01 04 30 00 04 30 00 00 00 00 00 00 ......0..0...... | 1072: 9a ee c1 80 fd 78 1f ce 1b ae eb b4 00 00 00 00 .....x.......... | 1088: 13 20 ff 20 00 70 00 00 00 60 50 00 00 00 11 e0 . . .p...`P..... | 1104: 00 00 00 70 00 00 00 60 50 05 35 14 c6 97 46 52 ...p...`P.5...FR | 1120: 06 66 f7 26 d6 17 42 03 30 01 00 00 10 10 04 02 .f.&..B.0....... | 1136: 02 00 00 00 00 00 00 00 00 40 00 00 00 00 00 00 .........@...... | 1152: 00 00 00 00 00 40 00 00 00 40 00 00 00 00 00 00 .....@...@...... | 4080: 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 05 ................ | page 12 offset 45056 | 0: 0d 00 00 00 01 04 30 00 04 30 e1 b4 30 97 4d 46 ......0..0..0.MF | 16: 14 00 ae 7c 00 00 00 00 00 00 00 03 00 00 43 00 ...|..........C. | page 47 offset 188416 | 2512: 00 00 00 00 00 00 00 00 be 00 00 00 00 00 00 00 ................ | page 87 offset 352256 | 2512: 00 00 00 00 00 00 00 00 aa 00 00 00 00 00 00 00 ................ | end crash-acaae0347204ae.db }]} {} do_intck_test 1.1 { {corruption found while reading database schema} } #------------------------------------------------------------------------- reset_db do_test 2.0 { sqlite3 db {} db deserialize [decode_hexdb { | size 28672 pagesize 4096 filename crash-3afa1ca9e9c1bd.db | page 1 offset 0 | 0: 53 51 4c 69 74 65 20 66 6f 72 6d 61 74 20 33 00 SQLite format 3. | 16: 10 00 01 01 00 40 20 20 00 00 00 00 00 00 00 07 .....@ ........ | 32: 00 00 00 00 00 00 00 00 00 00 00 06 00 00 00 04 ................ | 48: 00 00 00 00 00 00 00 00 00 00 00 01 00 00 00 00 ................ | 96: 00 00 00 00 0d 00 00 00 06 0e 88 00 0f b8 0f 6d ...............m | 112: 0f 3a 0f 0b 0e d5 0e 88 01 00 00 00 00 00 00 00 .:.............. | 3712: 00 00 00 00 00 00 00 00 4b 06 06 17 25 25 01 5b ........K...%%.[ | 3728: 74 61 62 6c 65 73 71 6c 69 74 65 5f 73 74 61 74 tablesqlite_stat | 3744: 31 73 71 6c 69 74 65 5f 73 74 61 74 31 07 43 52 1sqlite_stat1.CR | 3760: 45 41 54 45 20 54 41 42 4c 45 20 73 71 6c 69 74 EATE TABLE sqlit | 3776: 65 5f 73 74 61 74 31 28 74 62 6c 2c 69 64 78 2c e_stat1(tbl,idx, | 3792: 73 74 61 74 29 34 05 06 17 13 11 01 53 69 6e 64 stat)4......Sind | 3808: 65 78 63 31 63 63 31 06 43 52 45 41 54 45 20 55 exc1cc1.CREATE U | 3824: 4e 49 51 55 45 20 49 4e 44 45 58 20 63 31 63 20 NIQUE INDEX c1c | 3840: 4f 4e 20 63 31 28 63 2c 20 62 29 2d 04 06 17 13 ON c1(c, b)-.... | 3856: 11 01 45 69 6e 64 65 78 63 31 64 63 31 05 43 52 ..Eindexc1dc1.CR | 3872: 45 41 54 45 20 49 4e 44 45 58 20 63 31 64 20 4f EATE INDEX c1d O | 3888: 4e 20 63 31 28 64 2c 20 62 29 31 03 06 17 13 11 N c1(d, b)1..... | 3904: 01 4d 69 6e 64 65 78 62 31 63 62 31 05 43 52 45 .Mindexb1cb1.CRE | 3920: 41 54 45 20 55 4e 49 51 55 45 20 49 4e 44 45 58 ATE UNIQUE INDEX | 3936: 20 62 31 63 20 4f 4e 20 62 31 28 63 29 49 02 06 b1c ON b1(c)I.. | 3952: 17 11 11 0f 7f 74 61 62 6c 65 63 31 63 31 03 43 .....tablec1c1.C | 3968: 52 45 41 54 45 20 54 41 42 4c 45 20 63 31 28 61 REATE TABLE c1(a | 3984: 20 49 4e 54 20 50 52 49 4d 41 52 59 20 4b 45 59 INT PRIMARY KEY | 4000: 2c 20 62 2c 20 63 2c 20 64 29 20 57 49 54 48 4f , b, c, d) WITHO | 4016: 55 54 20 52 4f 57 49 44 46 01 06 17 11 11 01 79 UT ROWIDF......y | 4032: 74 61 62 6c 65 62 31 62 31 02 43 52 45 41 54 45 tableb1b1.CREATE | 4048: 20 54 41 42 4c 45 20 62 31 28 61 20 49 4e 54 20 TABLE b1(a INT | 4064: 50 52 49 4d 41 52 59 20 4b 45 59 2c 20 62 2c 20 PRIMARY KEY, b, | 4080: 63 29 20 57 49 54 48 4f 55 54 20 52 4f 57 49 44 c) WITHOUT ROWID | page 2 offset 4096 | 0: 0a 00 00 00 07 0f ca 00 0f fa 0f f2 0f ea 0f e2 ................ | 16: 0f da 00 00 00 01 00 00 00 00 00 00 00 00 00 00 ................ | 4032: 00 00 00 00 00 00 00 00 00 00 07 04 01 0f 01 06 ................ | 4048: 67 07 07 04 01 0f 01 06 66 06 07 04 01 0f 01 05 g.......f....... | 4064: 65 05 07 04 01 0f 01 04 64 04 07 04 01 0f 01 03 e.......d....... | 4080: 63 03 07 04 01 0f 01 02 62 0f 05 04 09 0f 09 61 c.......b......a | page 3 offset 8192 | 0: 0a 00 00 00 07 0f bd 00 0f f9 0f ef 0f e5 0f db ................ | 16: 0f d1 0f c7 0f bd 00 00 00 00 01 00 00 00 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 00 00 00 00 00 09 05 01 ................ | 4032: 0f 01 01 07 61 07 07 09 05 01 0f 01 01 06 61 06 ....a.........a. | 4048: 06 09 05 01 0f 01 01 05 61 05 05 09 05 01 0f 01 ........a....... | 4064: 01 04 61 04 04 09 05 01 0f 01 01 03 61 03 03 09 ..a.........a... | 4080: 05 01 0f 01 01 02 61 0f 02 06 05 09 0f 09 09 61 ......a........a | page 4 offset 12288 | 0: 0a 00 00 00 07 0f d8 00 0f fc 0f f0 0f ea 0f e4 ................ | 16: 0f de 0f d8 0f f6 00 00 00 00 00 00 00 00 00 00 ................ | 4048: 00 00 00 00 00 00 00 00 05 03 01 01 07 07 05 03 ................ | 4064: 01 01 06 06 05 03 01 01 05 05 05 03 01 01 04 04 ................ | 4080: 05 03 01 01 03 03 05 03 01 01 0f 02 03 03 09 09 ................ | page 5 offset 16384 | 0: 0a 00 00 00 07 0f ca 00 0f fa 0f f2 0f ea 0f 00 ................ | 4032: 00 00 00 00 00 00 00 00 00 00 07 04 01 0f 01 07 ................ | 4048: 61 07 07 04 01 0f 01 06 61 06 07 04 01 0f 01 05 a.......a....... | 4064: 61 05 07 04 01 1f 01 04 61 04 07 04 01 0f 01 03 a.......a....... | 4080: 61 03 07 04 01 0f 01 02 61 02 05 04 09 0f 09 61 a.......a......a | page 6 offset 20480 | 0: 0a 00 00 00 07 0f ca 00 0f fa 0f ea 0f e2 00 00 ................ | 4032: 00 00 00 00 00 00 00 00 00 00 07 04 01 0f 01 07 ................ | 4048: 61 07 07 04 01 0f 01 06 61 06 07 04 01 0f 01 05 a.......a....... | 4064: 61 05 07 04 01 0f 01 04 61 04 07 04 01 0f 01 03 a.......a....... | 4080: 61 03 07 04 01 0f 01 0f 61 02 05 04 09 0f 09 61 a.......a......a | page 7 offset 24576 | 0: 0d 00 00 00 05 0f 1c 00 0f f0 0f e0 0f d3 0f c5 ................ | 16: 0f b8 00 00 00 00 00 00 00 00 00 00 00 00 00 00 ................ | 4016: 00 00 00 00 00 00 00 00 0b 05 04 11 11 13 62 31 ..............b1 | 4032: 62 31 37 20 31 0c 04 04 11 13 13 62 31 62 31 63 b17 1......b1b1c | 4048: 37 20 31 0b 03 04 11 11 13 63 31 63 31 37 20 31 7 1......c1c17 1 | 4064: 0e 02 04 11 13 07 63 31 63 31 64 37 20 31 20 31 ......c1c1d7 1 1 | 4080: 0e 01 04 11 13 17 63 31 63 31 63 37 20 31 00 00 ......c1c1c7 1.. | end crash-3afa1ca9e9c1bd.db }]} {} do_intck_test 2.1 { {corruption found while reading database schema} } #------------------------------------------------------------------------- reset_db do_execsql_test 3.0 { PRAGMA page_size = 1024; CREATE TABLE t1(a, b); CREATE INDEX i1 ON t1(a); INSERT INTO t1 VALUES(1, 1), (2, 2), (3, 3); } do_test 3.1 { set pgno [db one {SELECT rootpage FROM sqlite_schema WHERE name='t1'}] db close hexio_write test.db [expr ($pgno-1)*1024] 0000 } {2} sqlite3 db test.db do_intck_test 3.2 { {corruption found while scanning database object i1} {corruption found while scanning database object t1} } finish_test |
Added ext/intck/intckfault.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 | # 2024 February 24 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # source [file join [file dirname [info script]] intck_common.tcl] set testprefix intckfault return_if_no_intck do_execsql_test 1.0 { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c); INSERT INTO t1 VALUES(1, 2, 3); INSERT INTO t1 VALUES(2, 'two', 'three'); INSERT INTO t1 VALUES(3, NULL, NULL); CREATE INDEX i1 ON t1(b, c); } do_faultsim_test 1 -faults oom-t* -prep { } -body { set ::ic [sqlite3_intck db main] set nStep 0 while {"SQLITE_OK"==[$::ic step]} { incr nStep if {$nStep==3} { $::ic unlock } } set res [$::ic error] $::ic close set res } -test { catch { $::ic close } faultsim_test_result {0 {SQLITE_OK {}}} {0 {SQLITE_NOMEM {}}} {0 {SQLITE_NOMEM {out of memory}}} } finish_test |
Added ext/intck/sqlite3intck.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 | /* ** 2024-02-08 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* */ #include "sqlite3intck.h" #include <string.h> #include <assert.h> #include <stdio.h> #include <stdlib.h> /* ** nKeyVal: ** The number of values that make up the 'key' for the current pCheck ** statement. ** ** rc: ** Error code returned by most recent sqlite3_intck_step() or ** sqlite3_intck_unlock() call. This is set to SQLITE_DONE when ** the integrity-check operation is finished. ** ** zErr: ** If the object has entered the error state, this is the error message. ** Is freed using sqlite3_free() when the object is deleted. ** ** zTestSql: ** The value returned by the most recent call to sqlite3_intck_testsql(). ** Each call to testsql() frees the previous zTestSql value (using ** sqlite3_free()) and replaces it with the new value it will return. */ struct sqlite3_intck { sqlite3 *db; const char *zDb; /* Copy of zDb parameter to _open() */ char *zObj; /* Current object. Or NULL. */ sqlite3_stmt *pCheck; /* Current check statement */ char *zKey; int nKeyVal; char *zMessage; int bCorruptSchema; int rc; /* Error code */ char *zErr; /* Error message */ char *zTestSql; /* Returned by sqlite3_intck_test_sql() */ }; /* ** Some error has occurred while using database p->db. Save the error message ** and error code currently held by the database handle in p->rc and p->zErr. */ static void intckSaveErrmsg(sqlite3_intck *p){ p->rc = sqlite3_errcode(p->db); sqlite3_free(p->zErr); p->zErr = sqlite3_mprintf("%s", sqlite3_errmsg(p->db)); } /* ** If the handle passed as the first argument is already in the error state, ** then this function is a no-op (returns NULL immediately). Otherwise, if an ** error occurs within this function, it leaves an error in said handle. ** ** Otherwise, this function attempts to prepare SQL statement zSql and ** return the resulting statement handle to the user. */ static sqlite3_stmt *intckPrepare(sqlite3_intck *p, const char *zSql){ sqlite3_stmt *pRet = 0; if( p->rc==SQLITE_OK ){ p->rc = sqlite3_prepare_v2(p->db, zSql, -1, &pRet, 0); if( p->rc!=SQLITE_OK ){ intckSaveErrmsg(p); assert( pRet==0 ); } } return pRet; } /* ** If the handle passed as the first argument is already in the error state, ** then this function is a no-op (returns NULL immediately). Otherwise, if an ** error occurs within this function, it leaves an error in said handle. ** ** Otherwise, this function treats argument zFmt as a printf() style format ** string. It formats it according to the trailing arguments and then ** attempts to prepare the results and return the resulting prepared ** statement. */ static sqlite3_stmt *intckPrepareFmt(sqlite3_intck *p, const char *zFmt, ...){ sqlite3_stmt *pRet = 0; va_list ap; char *zSql = 0; va_start(ap, zFmt); zSql = sqlite3_vmprintf(zFmt, ap); if( p->rc==SQLITE_OK && zSql==0 ){ p->rc = SQLITE_NOMEM; } pRet = intckPrepare(p, zSql); sqlite3_free(zSql); va_end(ap); return pRet; } /* ** Finalize SQL statement pStmt. If an error occurs and the handle passed ** as the first argument does not already contain an error, store the ** error in the handle. */ static void intckFinalize(sqlite3_intck *p, sqlite3_stmt *pStmt){ int rc = sqlite3_finalize(pStmt); if( p->rc==SQLITE_OK && rc!=SQLITE_OK ){ intckSaveErrmsg(p); } } /* ** If there is already an error in handle p, return it. Otherwise, call ** sqlite3_step() on the statement handle and return that value. */ static int intckStep(sqlite3_intck *p, sqlite3_stmt *pStmt){ if( p->rc ) return p->rc; return sqlite3_step(pStmt); } /* ** Execute SQL statement zSql. There is no way to obtain any results ** returned by the statement. This function uses the sqlite3_intck error ** code convention. */ static void intckExec(sqlite3_intck *p, const char *zSql){ sqlite3_stmt *pStmt = 0; pStmt = intckPrepare(p, zSql); intckStep(p, pStmt); intckFinalize(p, pStmt); } /* ** A wrapper around sqlite3_mprintf() that uses the sqlite3_intck error ** code convention. */ static char *intckMprintf(sqlite3_intck *p, const char *zFmt, ...){ va_list ap; char *zRet = 0; va_start(ap, zFmt); zRet = sqlite3_vmprintf(zFmt, ap); if( p->rc==SQLITE_OK ){ if( zRet==0 ){ p->rc = SQLITE_NOMEM; } }else{ sqlite3_free(zRet); zRet = 0; } return zRet; } /* ** This is used by sqlite3_intck_unlock() to save the vector key value ** required to restart the current pCheck query as a nul-terminated string ** in p->zKey. */ static void intckSaveKey(sqlite3_intck *p){ int ii; char *zSql = 0; sqlite3_stmt *pStmt = 0; sqlite3_stmt *pXinfo = 0; const char *zDir = 0; assert( p->pCheck ); assert( p->zKey==0 ); pXinfo = intckPrepareFmt(p, "SELECT group_concat(desc, '') FROM %Q.sqlite_schema s, " "pragma_index_xinfo(%Q, %Q) " "WHERE s.type='index' AND s.name=%Q", p->zDb, p->zObj, p->zDb, p->zObj ); if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXinfo) ){ zDir = (const char*)sqlite3_column_text(pXinfo, 0); } if( zDir==0 ){ /* Object is a table, not an index. This is the easy case,as there are ** no DESC columns or NULL values in a primary key. */ const char *zSep = "SELECT '(' || "; for(ii=0; ii<p->nKeyVal; ii++){ zSql = intckMprintf(p, "%z%squote(?)", zSql, zSep); zSep = " || ', ' || "; } zSql = intckMprintf(p, "%z || ')'", zSql); }else{ /* Object is an index. */ assert( p->nKeyVal>1 ); for(ii=p->nKeyVal; ii>0; ii--){ int bLastIsDesc = zDir[ii-1]=='1'; int bLastIsNull = sqlite3_column_type(p->pCheck, ii)==SQLITE_NULL; const char *zLast = sqlite3_column_name(p->pCheck, ii); char *zLhs = 0; char *zRhs = 0; char *zWhere = 0; if( bLastIsNull ){ if( bLastIsDesc ) continue; zWhere = intckMprintf(p, "'%s IS NOT NULL'", zLast); }else{ const char *zOp = bLastIsDesc ? "<" : ">"; zWhere = intckMprintf(p, "'%s %s ' || quote(?%d)", zLast, zOp, ii); } if( ii>1 ){ const char *zLhsSep = ""; const char *zRhsSep = ""; int jj; for(jj=0; jj<ii-1; jj++){ const char *zAlias = (const char*)sqlite3_column_name(p->pCheck,jj+1); zLhs = intckMprintf(p, "%z%s%s", zLhs, zLhsSep, zAlias); zRhs = intckMprintf(p, "%z%squote(?%d)", zRhs, zRhsSep, jj+1); zLhsSep = ","; zRhsSep = " || ',' || "; } zWhere = intckMprintf(p, "'(%z) IS (' || %z || ') AND ' || %z", zLhs, zRhs, zWhere); } zWhere = intckMprintf(p, "'WHERE ' || %z", zWhere); zSql = intckMprintf(p, "%z%s(quote( %z ) )", zSql, (zSql==0 ? "VALUES" : ",\n "), zWhere ); } zSql = intckMprintf(p, "WITH wc(q) AS (\n%z\n)" "SELECT 'VALUES' || group_concat('(' || q || ')', ',\n ') FROM wc" , zSql ); } pStmt = intckPrepare(p, zSql); if( p->rc==SQLITE_OK ){ for(ii=0; ii<p->nKeyVal; ii++){ sqlite3_bind_value(pStmt, ii+1, sqlite3_column_value(p->pCheck, ii+1)); } if( SQLITE_ROW==sqlite3_step(pStmt) ){ p->zKey = intckMprintf(p,"%s",(const char*)sqlite3_column_text(pStmt, 0)); } intckFinalize(p, pStmt); } sqlite3_free(zSql); intckFinalize(p, pXinfo); } /* ** Find the next database object (table or index) to check. If successful, ** set sqlite3_intck.zObj to point to a nul-terminated buffer containing ** the object's name before returning. */ static void intckFindObject(sqlite3_intck *p){ sqlite3_stmt *pStmt = 0; char *zPrev = p->zObj; p->zObj = 0; assert( p->rc==SQLITE_OK ); assert( p->pCheck==0 ); pStmt = intckPrepareFmt(p, "WITH tables(table_name) AS (" " SELECT name" " FROM %Q.sqlite_schema WHERE (type='table' OR type='index') AND rootpage" " UNION ALL " " SELECT 'sqlite_schema'" ")" "SELECT table_name FROM tables " "WHERE ?1 IS NULL OR table_name%s?1 " "ORDER BY 1" , p->zDb, (p->zKey ? ">=" : ">") ); if( p->rc==SQLITE_OK ){ sqlite3_bind_text(pStmt, 1, zPrev, -1, SQLITE_TRANSIENT); if( sqlite3_step(pStmt)==SQLITE_ROW ){ p->zObj = intckMprintf(p,"%s",(const char*)sqlite3_column_text(pStmt, 0)); } } intckFinalize(p, pStmt); /* If this is a new object, ensure the previous key value is cleared. */ if( sqlite3_stricmp(p->zObj, zPrev) ){ sqlite3_free(p->zKey); p->zKey = 0; } sqlite3_free(zPrev); } /* ** Return the size in bytes of the first token in nul-terminated buffer z. ** For the purposes of this call, a token is either: ** ** * a quoted SQL string, * * a contiguous series of ascii alphabet characters, or * * any other single byte. */ static int intckGetToken(const char *z){ char c = z[0]; int iRet = 1; if( c=='\'' || c=='"' || c=='`' ){ while( 1 ){ if( z[iRet]==c ){ iRet++; if( z[iRet]!=c ) break; } iRet++; } } else if( c=='[' ){ while( z[iRet++]!=']' && z[iRet] ); } else if( (c>='A' && c<='Z') || (c>='a' && c<='z') ){ while( (z[iRet]>='A' && z[iRet]<='Z') || (z[iRet]>='a' && z[iRet]<='z') ){ iRet++; } } return iRet; } /* ** Return true if argument c is an ascii whitespace character. */ static int intckIsSpace(char c){ return (c==' ' || c=='\t' || c=='\n' || c=='\r'); } /* ** Argument z points to the text of a CREATE INDEX statement. This function ** identifies the part of the text that contains either the index WHERE ** clause (if iCol<0) or the iCol'th column of the index. ** ** If (iCol<0), the identified fragment does not include the "WHERE" keyword, ** only the expression that follows it. If (iCol>=0) then the identified ** fragment does not include any trailing sort-order keywords - "ASC" or ** "DESC". ** ** If the CREATE INDEX statement does not contain the requested field or ** clause, NULL is returned and (*pnByte) is set to 0. Otherwise, a pointer to ** the identified fragment is returned and output parameter (*pnByte) set ** to its size in bytes. */ static const char *intckParseCreateIndex(const char *z, int iCol, int *pnByte){ int iOff = 0; int iThisCol = 0; int iStart = 0; int nOpen = 0; const char *zRet = 0; int nRet = 0; int iEndOfCol = 0; /* Skip forward until the first "(" token */ while( z[iOff]!='(' ){ iOff += intckGetToken(&z[iOff]); if( z[iOff]=='\0' ) return 0; } assert( z[iOff]=='(' ); nOpen = 1; iOff++; iStart = iOff; while( z[iOff] ){ const char *zToken = &z[iOff]; int nToken = 0; /* Check if this is the end of the current column - either a "," or ")" ** when nOpen==1. */ if( nOpen==1 ){ if( z[iOff]==',' || z[iOff]==')' ){ if( iCol==iThisCol ){ int iEnd = iEndOfCol ? iEndOfCol : iOff; nRet = (iEnd - iStart); zRet = &z[iStart]; break; } iStart = iOff+1; while( intckIsSpace(z[iStart]) ) iStart++; iThisCol++; } if( z[iOff]==')' ) break; } if( z[iOff]=='(' ) nOpen++; if( z[iOff]==')' ) nOpen--; nToken = intckGetToken(zToken); if( (nToken==3 && 0==sqlite3_strnicmp(zToken, "ASC", nToken)) || (nToken==4 && 0==sqlite3_strnicmp(zToken, "DESC", nToken)) ){ iEndOfCol = iOff; }else if( 0==intckIsSpace(zToken[0]) ){ iEndOfCol = 0; } iOff += nToken; } /* iStart is now the byte offset of 1 byte passed the final ')' in the ** CREATE INDEX statement. Try to find a WHERE clause to return. */ while( zRet==0 && z[iOff] ){ int n = intckGetToken(&z[iOff]); if( n==5 && 0==sqlite3_strnicmp(&z[iOff], "where", 5) ){ zRet = &z[iOff+5]; nRet = (int)strlen(zRet); } iOff += n; } /* Trim any whitespace from the start and end of the returned string. */ if( zRet ){ while( intckIsSpace(zRet[0]) ){ nRet--; zRet++; } while( nRet>0 && intckIsSpace(zRet[nRet-1]) ) nRet--; } *pnByte = nRet; return zRet; } /* ** User-defined SQL function wrapper for intckParseCreateIndex(): ** ** SELECT parse_create_index(<sql>, <icol>); */ static void intckParseCreateIndexFunc( sqlite3_context *pCtx, int nVal, sqlite3_value **apVal ){ const char *zSql = (const char*)sqlite3_value_text(apVal[0]); int idx = sqlite3_value_int(apVal[1]); const char *zRes = 0; int nRes = 0; assert( nVal==2 ); if( zSql ){ zRes = intckParseCreateIndex(zSql, idx, &nRes); } sqlite3_result_text(pCtx, zRes, nRes, SQLITE_TRANSIENT); } /* ** Return true if sqlite3_intck.db has automatic indexes enabled, false ** otherwise. */ static int intckGetAutoIndex(sqlite3_intck *p){ int bRet = 0; sqlite3_stmt *pStmt = 0; pStmt = intckPrepare(p, "PRAGMA automatic_index"); if( SQLITE_ROW==intckStep(p, pStmt) ){ bRet = sqlite3_column_int(pStmt, 0); } intckFinalize(p, pStmt); return bRet; } /* ** Return true if zObj is an index, or false otherwise. */ static int intckIsIndex(sqlite3_intck *p, const char *zObj){ int bRet = 0; sqlite3_stmt *pStmt = 0; pStmt = intckPrepareFmt(p, "SELECT 1 FROM %Q.sqlite_schema WHERE name=%Q AND type='index'", p->zDb, zObj ); if( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){ bRet = 1; } intckFinalize(p, pStmt); return bRet; } /* ** Return a pointer to a nul-terminated buffer containing the SQL statement ** used to check database object zObj (a table or index) for corruption. ** If parameter zPrev is not NULL, then it must be a string containing the ** vector key required to restart the check where it left off last time. ** If pnKeyVal is not NULL, then (*pnKeyVal) is set to the number of ** columns in the vector key value for the specified object. ** ** This function uses the sqlite3_intck error code convention. */ static char *intckCheckObjectSql( sqlite3_intck *p, /* Integrity check object */ const char *zObj, /* Object (table or index) to scan */ const char *zPrev, /* Restart key vector, if any */ int *pnKeyVal /* OUT: Number of key-values for this scan */ ){ char *zRet = 0; sqlite3_stmt *pStmt = 0; int bAutoIndex = 0; int bIsIndex = 0; const char *zCommon = /* Relation without_rowid also contains just one row. Column "b" is ** set to true if the table being examined is a WITHOUT ROWID table, ** or false otherwise. */ ", without_rowid(b) AS (" " SELECT EXISTS (" " SELECT 1 FROM tabname, pragma_index_list(tab, db) AS l" " WHERE origin='pk' " " AND NOT EXISTS (SELECT 1 FROM sqlite_schema WHERE name=l.name)" " )" ")" "" /* Table idx_cols contains 1 row for each column in each index on the ** table being checked. Columns are: ** ** idx_name: Name of the index. ** idx_ispk: True if this index is the PK of a WITHOUT ROWID table. ** col_name: Name of indexed column, or NULL for index on expression. ** col_expr: Indexed expression, including COLLATE clause. ** col_alias: Alias used for column in 'intck_wrapper' table. */ ", idx_cols(idx_name, idx_ispk, col_name, col_expr, col_alias) AS (" " SELECT l.name, (l.origin=='pk' AND w.b), i.name, COALESCE((" " SELECT parse_create_index(sql, i.seqno) FROM " " sqlite_schema WHERE name = l.name" " ), format('\"%w\"', i.name) || ' COLLATE ' || quote(i.coll))," " 'c' || row_number() OVER ()" " FROM " " tabname t," " without_rowid w," " pragma_index_list(t.tab, t.db) l," " pragma_index_xinfo(l.name) i" " WHERE i.key" " UNION ALL" " SELECT '', 1, '_rowid_', '_rowid_', 'r1' FROM without_rowid WHERE b=0" ")" "" "" /* ** For a PK declared as "PRIMARY KEY(a, b) ... WITHOUT ROWID", where ** the intck_wrapper aliases of "a" and "b" are "c1" and "c2": ** ** o_pk: "o.c1, o.c2" ** i_pk: "i.'a', i.'b'" ** ... ** n_pk: 2 */ ", tabpk(db, tab, idx, o_pk, i_pk, q_pk, eq_pk, ps_pk, pk_pk, n_pk) AS (" " WITH pkfields(f, a) AS (" " SELECT i.col_name, i.col_alias FROM idx_cols i WHERE i.idx_ispk" " )" " SELECT t.db, t.tab, t.idx, " " group_concat(a, ', '), " " group_concat('i.'||quote(f), ', '), " " group_concat('quote(o.'||a||')', ' || '','' || '), " " format('(%s)==(%s)'," " group_concat('o.'||a, ', '), " " group_concat(format('\"%w\"', f), ', ')" " )," " group_concat('%s', ',')," " group_concat('quote('||a||')', ', '), " " count(*)" " FROM tabname t, pkfields" ")" "" ", idx(name, match_expr, partial, partial_alias, idx_ps, idx_idx) AS (" " SELECT idx_name," " format('(%s,%s) IS (%s,%s)', " " group_concat(i.col_expr, ', '), i_pk," " group_concat('o.'||i.col_alias, ', '), o_pk" " ), " " parse_create_index(" " (SELECT sql FROM sqlite_schema WHERE name=idx_name), -1" " )," " 'cond' || row_number() OVER ()" " , group_concat('%s', ',')" " , group_concat('quote('||i.col_alias||')', ', ')" " FROM tabpk t, " " without_rowid w," " idx_cols i" " WHERE i.idx_ispk==0 " " GROUP BY idx_name" ")" "" ", wrapper_with(s) AS (" " SELECT 'intck_wrapper AS (\n SELECT\n ' || (" " WITH f(a, b) AS (" " SELECT col_expr, col_alias FROM idx_cols" " UNION ALL " " SELECT partial, partial_alias FROM idx WHERE partial IS NOT NULL" " )" " SELECT group_concat(format('%s AS %s', a, b), ',\n ') FROM f" " )" " || format('\n FROM %Q.%Q ', t.db, t.tab)" /* If the object being checked is a table, append "NOT INDEXED". ** Otherwise, append "INDEXED BY <index>", and then, if the index ** is a partial index " WHERE <condition>". */ " || CASE WHEN t.idx IS NULL THEN " " 'NOT INDEXED'" " ELSE" " format('INDEXED BY %Q%s', t.idx, ' WHERE '||i.partial)" " END" " || '\n)'" " FROM tabname t LEFT JOIN idx i ON (i.name=t.idx)" ")" "" ; bAutoIndex = intckGetAutoIndex(p); if( bAutoIndex ) intckExec(p, "PRAGMA automatic_index = 0"); bIsIndex = intckIsIndex(p, zObj); if( bIsIndex ){ pStmt = intckPrepareFmt(p, /* Table idxname contains a single row. The first column, "db", contains ** the name of the db containing the table (e.g. "main") and the second, ** "tab", the name of the table itself. */ "WITH tabname(db, tab, idx) AS (" " SELECT %Q, (SELECT tbl_name FROM %Q.sqlite_schema WHERE name=%Q), %Q " ")" "" ", whereclause(w_c) AS (%s)" "" "%s" /* zCommon */ "" ", case_statement(c) AS (" " SELECT " " 'CASE WHEN (' || group_concat(col_alias, ', ') || ', 1) IS (\n' " " || ' SELECT ' || group_concat(col_expr, ', ') || ', 1 FROM '" " || format('%%Q.%%Q NOT INDEXED WHERE %%s\n', t.db, t.tab, p.eq_pk)" " || ' )\n THEN NULL\n '" " || 'ELSE format(''surplus entry ('" " || group_concat('%%s', ',') || ',' || p.ps_pk" " || ') in index ' || t.idx || ''', ' " " || group_concat('quote('||i.col_alias||')', ', ') || ', ' || p.pk_pk" " || ')'" " || '\n END AS error_message'" " FROM tabname t, tabpk p, idx_cols i WHERE i.idx_name=t.idx" ")" "" ", thiskey(k, n) AS (" " SELECT group_concat(i.col_alias, ', ') || ', ' || p.o_pk, " " count(*) + p.n_pk " " FROM tabpk p, idx_cols i WHERE i.idx_name=p.idx" ")" "" ", main_select(m, n) AS (" " SELECT format(" " 'WITH %%s\n' ||" " ', idx_checker AS (\n' ||" " ' SELECT %%s,\n' ||" " ' %%s\n' || " " ' FROM intck_wrapper AS o\n' ||" " ')\n'," " ww.s, c, t.k" " ), t.n" " FROM case_statement, wrapper_with ww, thiskey t" ")" "SELECT m || " " group_concat('SELECT * FROM idx_checker ' || w_c, ' UNION ALL '), n" " FROM " "main_select, whereclause " , p->zDb, p->zDb, zObj, zObj , zPrev ? zPrev : "VALUES('')", zCommon ); }else{ pStmt = intckPrepareFmt(p, /* Table tabname contains a single row. The first column, "db", contains ** the name of the db containing the table (e.g. "main") and the second, ** "tab", the name of the table itself. */ "WITH tabname(db, tab, idx, prev) AS (SELECT %Q, %Q, NULL, %Q)" "" "%s" /* zCommon */ /* expr(e) contains one row for each index on table zObj. Value e ** is set to an expression that evaluates to NULL if the required ** entry is present in the index, or an error message otherwise. */ ", expr(e, p) AS (" " SELECT format('CASE WHEN EXISTS \n" " (SELECT 1 FROM %%Q.%%Q AS i INDEXED BY %%Q WHERE %%s%%s)\n" " THEN NULL\n" " ELSE format(''entry (%%s,%%s) missing from index %%s'', %%s, %%s)\n" " END\n'" " , t.db, t.tab, i.name, i.match_expr, ' AND (' || partial || ')'," " i.idx_ps, t.ps_pk, i.name, i.idx_idx, t.pk_pk)," " CASE WHEN partial IS NULL THEN NULL ELSE i.partial_alias END" " FROM tabpk t, idx i" ")" ", numbered(ii, cond, e) AS (" " SELECT 0, 'n.ii=0', 'NULL'" " UNION ALL " " SELECT row_number() OVER ()," " '(n.ii='||row_number() OVER ()||COALESCE(' AND '||p||')', ')'), e" " FROM expr" ")" ", counter_with(w) AS (" " SELECT 'WITH intck_counter(ii) AS (\n ' || " " group_concat('SELECT '||ii, ' UNION ALL\n ') " " || '\n)' FROM numbered" ")" "" ", case_statement(c) AS (" " SELECT 'CASE ' || " " group_concat(format('\n WHEN %%s THEN (%%s)', cond, e), '') ||" " '\nEND AS error_message'" " FROM numbered" ")" "" /* This table contains a single row consisting of a single value - ** the text of an SQL expression that may be used by the main SQL ** statement to output an SQL literal that can be used to resume ** the scan if it is suspended. e.g. for a rowid table, an expression ** like: ** ** format('(%d,%d)', _rowid_, n.ii) */ ", thiskey(k, n) AS (" " SELECT o_pk || ', ii', n_pk+1 FROM tabpk" ")" "" ", whereclause(w_c) AS (" " SELECT CASE WHEN prev!='' THEN " " '\nWHERE (' || o_pk ||', n.ii) > ' || prev" " ELSE ''" " END" " FROM tabpk, tabname" ")" "" ", main_select(m, n) AS (" " SELECT format(" " '%%s, %%s\nSELECT %%s,\n%%s\nFROM intck_wrapper AS o" ", intck_counter AS n%%s\nORDER BY %%s', " " w, ww.s, c, thiskey.k, whereclause.w_c, t.o_pk" " ), thiskey.n" " FROM case_statement, tabpk t, counter_with, " " wrapper_with ww, thiskey, whereclause" ")" "SELECT m, n FROM main_select", p->zDb, zObj, zPrev, zCommon ); } while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){ zRet = intckMprintf(p, "%s", (const char*)sqlite3_column_text(pStmt, 0)); if( pnKeyVal ){ *pnKeyVal = sqlite3_column_int(pStmt, 1); } } intckFinalize(p, pStmt); if( bAutoIndex ) intckExec(p, "PRAGMA automatic_index = 1"); return zRet; } /* ** Open a new integrity-check object. */ int sqlite3_intck_open( sqlite3 *db, /* Database handle to operate on */ const char *zDbArg, /* "main", "temp" etc. */ sqlite3_intck **ppOut /* OUT: New integrity-check handle */ ){ sqlite3_intck *pNew = 0; int rc = SQLITE_OK; const char *zDb = zDbArg ? zDbArg : "main"; int nDb = (int)strlen(zDb); pNew = (sqlite3_intck*)sqlite3_malloc(sizeof(*pNew) + nDb + 1); if( pNew==0 ){ rc = SQLITE_NOMEM; }else{ memset(pNew, 0, sizeof(*pNew)); pNew->db = db; pNew->zDb = (const char*)&pNew[1]; memcpy(&pNew[1], zDb, nDb+1); rc = sqlite3_create_function(db, "parse_create_index", 2, SQLITE_UTF8, 0, intckParseCreateIndexFunc, 0, 0 ); if( rc!=SQLITE_OK ){ sqlite3_intck_close(pNew); pNew = 0; } } *ppOut = pNew; return rc; } /* ** Free the integrity-check object. */ void sqlite3_intck_close(sqlite3_intck *p){ if( p ){ sqlite3_finalize(p->pCheck); sqlite3_create_function( p->db, "parse_create_index", 1, SQLITE_UTF8, 0, 0, 0, 0 ); sqlite3_free(p->zObj); sqlite3_free(p->zKey); sqlite3_free(p->zTestSql); sqlite3_free(p->zErr); sqlite3_free(p->zMessage); sqlite3_free(p); } } /* ** Step the integrity-check object. */ int sqlite3_intck_step(sqlite3_intck *p){ if( p->rc==SQLITE_OK ){ if( p->zMessage ){ sqlite3_free(p->zMessage); p->zMessage = 0; } if( p->bCorruptSchema ){ p->rc = SQLITE_DONE; }else if( p->pCheck==0 ){ intckFindObject(p); if( p->rc==SQLITE_OK ){ if( p->zObj ){ char *zSql = 0; zSql = intckCheckObjectSql(p, p->zObj, p->zKey, &p->nKeyVal); p->pCheck = intckPrepare(p, zSql); sqlite3_free(zSql); sqlite3_free(p->zKey); p->zKey = 0; }else{ p->rc = SQLITE_DONE; } }else if( p->rc==SQLITE_CORRUPT ){ p->rc = SQLITE_OK; p->zMessage = intckMprintf(p, "%s", "corruption found while reading database schema" ); p->bCorruptSchema = 1; } } if( p->pCheck ){ assert( p->rc==SQLITE_OK ); if( sqlite3_step(p->pCheck)==SQLITE_ROW ){ /* Normal case, do nothing. */ }else{ intckFinalize(p, p->pCheck); p->pCheck = 0; p->nKeyVal = 0; if( p->rc==SQLITE_CORRUPT ){ p->rc = SQLITE_OK; p->zMessage = intckMprintf(p, "corruption found while scanning database object %s", p->zObj ); } } } } return p->rc; } /* ** Return a message describing the corruption encountered by the most recent ** call to sqlite3_intck_step(), or NULL if no corruption was encountered. */ const char *sqlite3_intck_message(sqlite3_intck *p){ assert( p->pCheck==0 || p->zMessage==0 ); if( p->zMessage ){ return p->zMessage; } if( p->pCheck ){ return (const char*)sqlite3_column_text(p->pCheck, 0); } return 0; } /* ** Return the error code and message. */ int sqlite3_intck_error(sqlite3_intck *p, const char **pzErr){ if( pzErr ) *pzErr = p->zErr; return (p->rc==SQLITE_DONE ? SQLITE_OK : p->rc); } /* ** Close any read transaction the integrity-check object is holding open ** on the database. */ int sqlite3_intck_unlock(sqlite3_intck *p){ if( p->rc==SQLITE_OK && p->pCheck ){ assert( p->zKey==0 && p->nKeyVal>0 ); intckSaveKey(p); intckFinalize(p, p->pCheck); p->pCheck = 0; } return p->rc; } /* ** Return the SQL statement used to check object zObj. Or, if zObj is ** NULL, the current SQL statement. */ const char *sqlite3_intck_test_sql(sqlite3_intck *p, const char *zObj){ sqlite3_free(p->zTestSql); if( zObj ){ p->zTestSql = intckCheckObjectSql(p, zObj, 0, 0); }else{ if( p->zObj ){ p->zTestSql = intckCheckObjectSql(p, p->zObj, p->zKey, 0); }else{ sqlite3_free(p->zTestSql); p->zTestSql = 0; } } return p->zTestSql; } |
Added ext/intck/sqlite3intck.h.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 | /* ** 2024-02-08 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* */ /* ** Incremental Integrity-Check Extension ** ------------------------------------- ** ** This module contains code to check whether or not an SQLite database ** is well-formed or corrupt. This is the same task as performed by SQLite's ** built-in "PRAGMA integrity_check" command. This module differs from ** "PRAGMA integrity_check" in that: ** ** + It is less thorough - this module does not detect certain types ** of corruption that are detected by the PRAGMA command. However, ** it does detect all kinds of corruption that are likely to cause ** errors in SQLite applications. ** ** + It is slower. Sometimes up to three times slower. ** ** + It allows integrity-check operations to be split into multiple ** transactions, so that the database does not need to be read-locked ** for the duration of the integrity-check. ** ** One way to use the API to run integrity-check on the "main" database ** of handle db is: ** ** int rc = SQLITE_OK; ** sqlite3_intck *p = 0; ** ** sqlite3_intck_open(db, "main", &p); ** while( SQLITE_OK==sqlite3_intck_step(p) ){ ** const char *zMsg = sqlite3_intck_message(p); ** if( zMsg ) printf("corruption: %s\n", zMsg); ** } ** rc = sqlite3_intck_error(p, &zErr); ** if( rc!=SQLITE_OK ){ ** printf("error occured (rc=%d), (errmsg=%s)\n", rc, zErr); ** } ** sqlite3_intck_close(p); ** ** Usually, the sqlite3_intck object opens a read transaction within the ** first call to sqlite3_intck_step() and holds it open until the ** integrity-check is complete. However, if sqlite3_intck_unlock() is ** called, the read transaction is ended and a new read transaction opened ** by the subsequent call to sqlite3_intck_step(). */ #ifndef _SQLITE_INTCK_H #define _SQLITE_INTCK_H #include "sqlite3.h" #ifdef __cplusplus extern "C" { #endif /* ** An ongoing incremental integrity-check operation is represented by an ** opaque pointer of the following type. */ typedef struct sqlite3_intck sqlite3_intck; /* ** Open a new incremental integrity-check object. If successful, populate ** output variable (*ppOut) with the new object handle and return SQLITE_OK. ** Or, if an error occurs, set (*ppOut) to NULL and return an SQLite error ** code (e.g. SQLITE_NOMEM). ** ** The integrity-check will be conducted on database zDb (which must be "main", ** "temp", or the name of an attached database) of database handle db. Once ** this function has been called successfully, the caller should not use ** database handle db until the integrity-check object has been destroyed ** using sqlite3_intck_close(). */ int sqlite3_intck_open( sqlite3 *db, /* Database handle */ const char *zDb, /* Database name ("main", "temp" etc.) */ sqlite3_intck **ppOut /* OUT: New sqlite3_intck handle */ ); /* ** Close and release all resources associated with a handle opened by an ** earlier call to sqlite3_intck_open(). The results of using an ** integrity-check handle after it has been passed to this function are ** undefined. */ void sqlite3_intck_close(sqlite3_intck *pCk); /* ** Do the next step of the integrity-check operation specified by the handle ** passed as the only argument. This function returns SQLITE_DONE if the ** integrity-check operation is finished, or an SQLite error code if ** an error occurs, or SQLITE_OK if no error occurs but the integrity-check ** is not finished. It is not considered an error if database corruption ** is encountered. ** ** Following a successful call to sqlite3_intck_step() (one that returns ** SQLITE_OK), sqlite3_intck_message() returns a non-NULL value if ** corruption was detected in the db. ** ** If an error occurs and a value other than SQLITE_OK or SQLITE_DONE is ** returned, then the integrity-check handle is placed in an error state. ** In this state all subsequent calls to sqlite3_intck_step() or ** sqlite3_intck_unlock() will immediately return the same error. The ** sqlite3_intck_error() method may be used to obtain an English language ** error message in this case. */ int sqlite3_intck_step(sqlite3_intck *pCk); /* ** If the previous call to sqlite3_intck_step() encountered corruption ** within the database, then this function returns a pointer to a buffer ** containing a nul-terminated string describing the corruption in ** English. If the previous call to sqlite3_intck_step() did not encounter ** corruption, or if there was no previous call, this function returns ** NULL. */ const char *sqlite3_intck_message(sqlite3_intck *pCk); /* ** Close any read-transaction opened by an earlier call to ** sqlite3_intck_step(). Any subsequent call to sqlite3_intck_step() will ** open a new transaction. Return SQLITE_OK if successful, or an SQLite error ** code otherwise. ** ** If an error occurs, then the integrity-check handle is placed in an error ** state. In this state all subsequent calls to sqlite3_intck_step() or ** sqlite3_intck_unlock() will immediately return the same error. The ** sqlite3_intck_error() method may be used to obtain an English language ** error message in this case. */ int sqlite3_intck_unlock(sqlite3_intck *pCk); /* ** If an error has occurred in an earlier call to sqlite3_intck_step() ** or sqlite3_intck_unlock(), then this method returns the associated ** SQLite error code. Additionally, if pzErr is not NULL, then (*pzErr) ** may be set to point to a nul-terminated string containing an English ** language error message. Or, if no error message is available, to ** NULL. ** ** If no error has occurred within sqlite3_intck_step() or ** sqlite_intck_unlock() calls on the handle passed as the first argument, ** then SQLITE_OK is returned and (*pzErr) set to NULL. */ int sqlite3_intck_error(sqlite3_intck *pCk, const char **pzErr); /* ** This API is used for testing only. It returns the full-text of an SQL ** statement used to test object zObj, which may be a table or index. ** The returned buffer is valid until the next call to either this function ** or sqlite3_intck_close() on the same sqlite3_intck handle. */ const char *sqlite3_intck_test_sql(sqlite3_intck *pCk, const char *zObj); #ifdef __cplusplus } /* end of the 'extern "C"' block */ #endif #endif /* ifndef _SQLITE_INTCK_H */ |
Added ext/intck/test_intck.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 | /* ** 2010 August 28 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** Code for testing all sorts of SQLite interfaces. This code ** is not included in the SQLite library. */ #include "sqlite3.h" #include "sqlite3intck.h" #include "tclsqlite.h" #include <string.h> #include <assert.h> /* In test1.c */ int getDbPointer(Tcl_Interp *interp, const char *zA, sqlite3 **ppDb); const char *sqlite3ErrName(int); typedef struct TestIntck TestIntck; struct TestIntck { sqlite3_intck *intck; }; static int testIntckCmd( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ struct Subcmd { const char *zName; int nArg; const char *zExpect; } aCmd[] = { {"close", 0, ""}, /* 0 */ {"step", 0, ""}, /* 1 */ {"message", 0, ""}, /* 2 */ {"error", 0, ""}, /* 3 */ {"unlock", 0, ""}, /* 4 */ {"test_sql", 1, ""}, /* 5 */ {0 , 0} }; int rc = TCL_OK; int iIdx = -1; TestIntck *p = (TestIntck*)clientData; if( objc<2 ){ Tcl_WrongNumArgs(interp, 1, objv, "SUB-COMMAND ..."); return TCL_ERROR; } rc = Tcl_GetIndexFromObjStruct( interp, objv[1], aCmd, sizeof(aCmd[0]), "SUB-COMMAND", 0, &iIdx ); if( rc ) return rc; if( objc!=2+aCmd[iIdx].nArg ){ Tcl_WrongNumArgs(interp, 2, objv, aCmd[iIdx].zExpect); return TCL_ERROR; } switch( iIdx ){ case 0: assert( 0==strcmp("close", aCmd[iIdx].zName) ); { Tcl_DeleteCommand(interp, Tcl_GetStringFromObj(objv[0], 0)); break; } case 1: assert( 0==strcmp("step", aCmd[iIdx].zName) ); { rc = sqlite3_intck_step(p->intck); Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); break; } case 2: assert( 0==strcmp("message", aCmd[iIdx].zName) ); { const char *z = sqlite3_intck_message(p->intck); Tcl_SetObjResult(interp, Tcl_NewStringObj(z ? z : "", -1)); break; } case 3: assert( 0==strcmp("error", aCmd[iIdx].zName) ); { const char *zErr = 0; Tcl_Obj *pRes; rc = sqlite3_intck_error(p->intck, 0); pRes = Tcl_NewObj(); Tcl_ListObjAppendElement( interp, pRes, Tcl_NewStringObj(sqlite3ErrName(rc), -1) ); sqlite3_intck_error(p->intck, &zErr); Tcl_ListObjAppendElement( interp, pRes, Tcl_NewStringObj(zErr ? zErr : 0, -1) ); Tcl_SetObjResult(interp, pRes); break; } case 4: assert( 0==strcmp("unlock", aCmd[iIdx].zName) ); { rc = sqlite3_intck_unlock(p->intck); Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); break; } case 5: assert( 0==strcmp("test_sql", aCmd[iIdx].zName) ); { const char *zObj = Tcl_GetString(objv[2]); const char *zSql = sqlite3_intck_test_sql(p->intck, zObj[0] ? zObj : 0); Tcl_SetObjResult(interp, Tcl_NewStringObj(zSql, -1)); break; } } return TCL_OK; } /* ** Destructor for commands created by test_sqlite3_intck(). */ static void testIntckFree(void *clientData){ TestIntck *p = (TestIntck*)clientData; sqlite3_intck_close(p->intck); ckfree(p); } /* ** tclcmd: sqlite3_intck DB DBNAME */ static int test_sqlite3_intck( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ char zName[64]; int iName = 0; Tcl_CmdInfo info; TestIntck *p = 0; sqlite3 *db = 0; const char *zDb = 0; int rc = SQLITE_OK; if( objc!=3 ){ Tcl_WrongNumArgs(interp, 1, objv, "DB DBNAME"); return TCL_ERROR; } p = (TestIntck*)ckalloc(sizeof(TestIntck)); memset(p, 0, sizeof(TestIntck)); if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ){ return TCL_ERROR; } zDb = Tcl_GetString(objv[2]); if( zDb[0]=='\0' ) zDb = 0; rc = sqlite3_intck_open(db, zDb, &p->intck); if( rc!=SQLITE_OK ){ ckfree(p); Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3_errstr(rc), -1)); return TCL_ERROR; } do { sprintf(zName, "intck%d", iName++); }while( Tcl_GetCommandInfo(interp, zName, &info)!=0 ); Tcl_CreateObjCommand(interp, zName, testIntckCmd, (void*)p, testIntckFree); Tcl_SetObjResult(interp, Tcl_NewStringObj(zName, -1)); return TCL_OK; } /* ** tclcmd: test_do_intck DB DBNAME */ static int test_do_intck( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3 *db = 0; const char *zDb = 0; int rc = SQLITE_OK; sqlite3_intck *pCk = 0; Tcl_Obj *pRet = 0; const char *zErr = 0; if( objc!=3 ){ Tcl_WrongNumArgs(interp, 1, objv, "DB DBNAME"); return TCL_ERROR; } if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ){ return TCL_ERROR; } zDb = Tcl_GetString(objv[2]); pRet = Tcl_NewObj(); Tcl_IncrRefCount(pRet); rc = sqlite3_intck_open(db, zDb, &pCk); if( rc==SQLITE_OK ){ while( sqlite3_intck_step(pCk)==SQLITE_OK ){ const char *zMsg = sqlite3_intck_message(pCk); if( zMsg ){ Tcl_ListObjAppendElement(interp, pRet, Tcl_NewStringObj(zMsg, -1)); } } rc = sqlite3_intck_error(pCk, &zErr); } if( rc!=SQLITE_OK ){ if( zErr ){ Tcl_SetObjResult(interp, Tcl_NewStringObj(zErr, -1)); }else{ Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1)); } }else{ Tcl_SetObjResult(interp, pRet); } Tcl_DecrRefCount(pRet); sqlite3_intck_close(pCk); sqlite3_intck_close(0); return rc ? TCL_ERROR : TCL_OK; } int Sqlitetestintck_Init(Tcl_Interp *interp){ Tcl_CreateObjCommand(interp, "sqlite3_intck", test_sqlite3_intck, 0, 0); Tcl_CreateObjCommand(interp, "test_do_intck", test_do_intck, 0, 0); return TCL_OK; } |
Added ext/jni/GNUmakefile.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 | # Quick-and-dirty makefile to bootstrap the sqlite3-jni project. This # build assumes a Linux-like system. default: all JAVA_HOME ?= $(HOME)/jdk/current # e.g. /usr/lib/jvm/default-javajava-19-openjdk-amd64 JDK_HOME ?= $(JAVA_HOME) # ^^^ JDK_HOME is not as widely used as JAVA_HOME bin.jar := $(JDK_HOME)/bin/jar bin.java := $(JDK_HOME)/bin/java bin.javac := $(JDK_HOME)/bin/javac bin.javadoc := $(JDK_HOME)/bin/javadoc ifeq (,$(wildcard $(JDK_HOME))) $(error set JDK_HOME to the top-most dir of your JDK installation.) endif MAKEFILE := $(lastword $(MAKEFILE_LIST)) $(MAKEFILE): package.jar := sqlite3-jni.jar dir.top := ../.. dir.tool := ../../tool dir.jni := $(patsubst %/,%,$(dir $(MAKEFILE))) dir.src := $(dir.jni)/src dir.src.c := $(dir.src)/c dir.bld := $(dir.jni)/bld dir.bld.c := $(dir.bld) dir.src.jni := $(dir.src)/org/sqlite/jni dir.src.capi := $(dir.src.jni)/capi dir.src.fts5 := $(dir.src.jni)/fts5 dir.tests := $(dir.src)/tests mkdir ?= mkdir -p $(dir.bld.c): $(mkdir) $@ javac.flags ?= -Xlint:unchecked -Xlint:deprecation java.flags ?= javac.flags += -encoding utf8 # -------------^^^^^^^^^^^^^^ required for Windows builds jnicheck ?= 1 ifeq (1,$(jnicheck)) java.flags += -Xcheck:jni endif classpath := $(dir.src) CLEAN_FILES := $(package.jar) DISTCLEAN_FILES := $(dir.jni)/*~ $(dir.src.c)/*~ $(dir.src.jni)/*~ sqlite3-jni.h := $(dir.src.c)/sqlite3-jni.h .NOTPARALLEL: $(sqlite3-jni.h) CApi.java := $(dir.src.capi)/CApi.java SQLTester.java := $(dir.src.capi)/SQLTester.java CApi.class := $(CApi.java:.java=.class) SQLTester.class := $(SQLTester.java:.java=.class) ######################################################################## # The future of FTS5 customization in this API is as yet unclear. # The pieces are all in place, and are all thin proxies so not much # complexity, but some semantic changes were required in porting # which are largely untested. # # Reminder: this flag influences the contents of $(sqlite3-jni.h), # which is checked in. Please do not check in changes to that file in # which the fts5 APIs have been stripped unless that feature is # intended to be stripped for good. enable.fts5 ?= 1 ifeq (,$(wildcard $(dir.tests)/*)) enable.tester := 0 else enable.tester := 1 endif # bin.version-info = binary to output various sqlite3 version info # building the distribution zip file. bin.version-info := $(dir.top)/version-info .NOTPARALLEL: $(bin.version-info) $(bin.version-info): $(dir.tool)/version-info.c $(sqlite3.h) $(dir.top)/Makefile $(MAKE) -C $(dir.top) version-info # Be explicit about which Java files to compile so that we can work on # in-progress files without requiring them to be in a compilable state. JAVA_FILES.main := $(patsubst %,$(dir.src.jni)/annotation/%,\ Experimental.java \ NotNull.java \ Nullable.java \ ) $(patsubst %,$(dir.src.capi)/%,\ AbstractCollationCallback.java \ AggregateFunction.java \ AuthorizerCallback.java \ AutoExtensionCallback.java \ BusyHandlerCallback.java \ CollationCallback.java \ CollationNeededCallback.java \ CommitHookCallback.java \ ConfigLogCallback.java \ ConfigSqlLogCallback.java \ NativePointerHolder.java \ OutputPointer.java \ PrepareMultiCallback.java \ PreupdateHookCallback.java \ ProgressHandlerCallback.java \ ResultCode.java \ RollbackHookCallback.java \ ScalarFunction.java \ SQLFunction.java \ CallbackProxy.java \ CApi.java \ TableColumnMetadata.java \ TraceV2Callback.java \ UpdateHookCallback.java \ ValueHolder.java \ WindowFunction.java \ XDestroyCallback.java \ sqlite3.java \ sqlite3_blob.java \ sqlite3_context.java \ sqlite3_stmt.java \ sqlite3_value.java \ ) $(patsubst %,$(dir.src.jni)/wrapper1/%,\ AggregateFunction.java \ ScalarFunction.java \ SqlFunction.java \ Sqlite.java \ SqliteException.java \ ValueHolder.java \ WindowFunction.java \ ) JAVA_FILES.unittest := $(patsubst %,$(dir.src.jni)/%,\ capi/Tester1.java \ wrapper1/Tester2.java \ ) ifeq (1,$(enable.fts5)) JAVA_FILES.unittest += $(patsubst %,$(dir.src.fts5)/%,\ TesterFts5.java \ ) JAVA_FILES.main += $(patsubst %,$(dir.src.fts5)/%,\ fts5_api.java \ fts5_extension_function.java \ fts5_tokenizer.java \ Fts5.java \ Fts5Context.java \ Fts5ExtensionApi.java \ Fts5PhraseIter.java \ Fts5Tokenizer.java \ XTokenizeCallback.java \ ) endif JAVA_FILES.tester := $(SQLTester.java) JAVA_FILES.package.info := \ $(dir.src.jni)/package-info.java \ $(dir.src.jni)/annotation/package-info.java CLASS_FILES.main := $(JAVA_FILES.main:.java=.class) CLASS_FILES.unittest := $(JAVA_FILES.unittest:.java=.class) CLASS_FILES.tester := $(JAVA_FILES.tester:.java=.class) JAVA_FILES += $(JAVA_FILES.main) $(JAVA_FILES.unittest) ifeq (1,$(enable.tester)) JAVA_FILES += $(JAVA_FILES.tester) endif CLASS_FILES := define CLASSFILE_DEPS all: $(1).class $(1).class: $(1).java CLASS_FILES += $(1).class endef $(foreach B,$(basename \ $(JAVA_FILES.main) $(JAVA_FILES.unittest) $(JAVA_FILES.tester)),\ $(eval $(call CLASSFILE_DEPS,$(B)))) $(CLASS_FILES): $(MAKEFILE) $(bin.javac) $(javac.flags) -h $(dir.bld.c) -cp $(classpath) $(JAVA_FILES) #.PHONY: classfiles ######################################################################## # Set up sqlite3.c and sqlite3.h... # # To build with SEE (https://sqlite.org/see), either put sqlite3-see.c # in the top of this build tree or pass # sqlite3.c=PATH_TO_sqlite3-see.c to the build. Note that only # encryption modules with no 3rd-party dependencies will currently # work here: AES256-OFB, AES128-OFB, and AES128-CCM. Not # coincidentally, those 3 modules are included in the sqlite3-see.c # bundle. # # A custom sqlite3.c must not have any spaces in its name. # $(sqlite3.canonical.c) must point to the sqlite3.c in # the sqlite3 canonical source tree, as that source file # is required for certain utility and test code. sqlite3.canonical.c := $(firstword $(wildcard $(dir.src.c)/sqlite3.c) $(dir.top)/sqlite3.c) sqlite3.canonical.h := $(firstword $(wildcard $(dir.src.c)/sqlite3.h) $(dir.top)/sqlite3.h) sqlite3.c := $(sqlite3.canonical.c) sqlite3.h := $(sqlite3.canonical.h) #ifeq (,$(shell grep sqlite3_activate_see $(sqlite3.c) 2>/dev/null)) # SQLITE_C_IS_SEE := 0 #else # SQLITE_C_IS_SEE := 1 # $(info This is an SEE build.) #endif .NOTPARALLEL: $(sqlite3.h) $(sqlite3.h): $(MAKE) -C $(dir.top) sqlite3.c $(sqlite3.c): $(sqlite3.h) opt.threadsafe ?= 1 opt.fatal-oom ?= 1 opt.debug ?= 1 opt.metrics ?= 1 SQLITE_OPT = \ -DSQLITE_THREADSAFE=$(opt.threadsafe) \ -DSQLITE_TEMP_STORE=2 \ -DSQLITE_USE_URI=1 \ -DSQLITE_OMIT_LOAD_EXTENSION \ -DSQLITE_OMIT_DEPRECATED \ -DSQLITE_OMIT_SHARED_CACHE \ -DSQLITE_C=$(sqlite3.c) \ -DSQLITE_JNI_FATAL_OOM=$(opt.fatal-oom) \ -DSQLITE_JNI_ENABLE_METRICS=$(opt.metrics) opt.extras ?= 1 ifeq (1,$(opt.extras)) SQLITE_OPT += -DSQLITE_ENABLE_RTREE \ -DSQLITE_ENABLE_EXPLAIN_COMMENTS \ -DSQLITE_ENABLE_STMTVTAB \ -DSQLITE_ENABLE_DBPAGE_VTAB \ -DSQLITE_ENABLE_DBSTAT_VTAB \ -DSQLITE_ENABLE_BYTECODE_VTAB \ -DSQLITE_ENABLE_OFFSET_SQL_FUNC \ -DSQLITE_ENABLE_PREUPDATE_HOOK \ -DSQLITE_ENABLE_NORMALIZE \ -DSQLITE_ENABLE_SQLLOG \ -DSQLITE_ENABLE_COLUMN_METADATA endif ifeq (1,$(opt.debug)) SQLITE_OPT += -DSQLITE_DEBUG -g -DDEBUG -UNDEBUG else SQLITE_OPT += -Os endif ifeq (1,$(enable.fts5)) SQLITE_OPT += -DSQLITE_ENABLE_FTS5 endif sqlite3-jni.c := $(dir.src.c)/sqlite3-jni.c sqlite3-jni.o := $(dir.bld.c)/sqlite3-jni.o sqlite3-jni.h := $(dir.src.c)/sqlite3-jni.h package.dll := $(dir.bld.c)/libsqlite3-jni.so # All javac-generated .h files must be listed in $(sqlite3-jni.h.in): sqlite3-jni.h.in := # $(java.with.jni) lists all Java files which contain JNI decls: java.with.jni := define ADD_JNI_H sqlite3-jni.h.in += $$(dir.bld.c)/org_sqlite_jni$(3)_$(2).h java.with.jni += $(1)/$(2).java $$(dir.bld.c)/org_sqlite_jni$(3)_$(2).h: $(1)/$(2).java endef # Invoke ADD_JNI_H once for each Java file which includes JNI # declarations: $(eval $(call ADD_JNI_H,$(dir.src.capi),CApi,_capi)) $(eval $(call ADD_JNI_H,$(dir.src.capi),SQLTester,_capi)) ifeq (1,$(enable.fts5)) $(eval $(call ADD_JNI_H,$(dir.src.fts5),Fts5ExtensionApi,_fts5)) $(eval $(call ADD_JNI_H,$(dir.src.fts5),fts5_api,_fts5)) $(eval $(call ADD_JNI_H,$(dir.src.fts5),fts5_tokenizer,_fts5)) endif $(sqlite3-jni.h.in): $(dir.bld.c) #package.dll.cfiles := package.dll.cflags = \ -std=c99 \ -fPIC \ -I. \ -I$(dir $(sqlite3.h)) \ -I$(dir.src.c) \ -I$(JDK_HOME)/include \ $(patsubst %,-I%,$(patsubst %.h,,$(wildcard $(JDK_HOME)/include/*))) \ -Wall # The gross $(patsubst...) above is to include the platform-specific # subdir which lives under $(JDK_HOME)/include and is a required # include path for client-level code. # # Using (-Wall -Wextra) triggers an untennable number of # gcc warnings from sqlite3.c for mundane things like # unused parameters. ######################################################################## ifeq (1,$(enable.tester)) package.dll.cflags += -DSQLITE_JNI_ENABLE_SQLTester endif $(sqlite3-jni.h): $(sqlite3-jni.h.in) $(MAKEFILE) @cat $(sqlite3-jni.h.in) > $@.tmp @if cmp $@ $@.tmp >/dev/null; then \ rm -f $@.tmp; \ echo "$@ not modified"; \ else \ mv $@.tmp $@; \ echo "Updated $@"; \ fi @if [ x1 != x$(enable.fts5) ]; then \ echo "*** REMINDER:"; \ echo "*** enable.fts5=0, so please do not check in changes to $@."; \ fi $(package.dll): $(sqlite3-jni.h) $(sqlite3.c) $(sqlite3.h) $(package.dll): $(sqlite3-jni.c) $(MAKEFILE) $(CC) $(package.dll.cflags) $(SQLITE_OPT) \ $(sqlite3-jni.c) -shared -o $@ all: $(package.dll) .PHONY: test test-one Tester1.flags ?= Tester2.flags ?= test.flags.jvm = -ea -Djava.library.path=$(dir.bld.c) \ $(java.flags) -cp $(classpath) test.deps := $(CLASS_FILES) $(package.dll) test-one: $(test.deps) $(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 $(Tester1.flags) $(bin.java) $(test.flags.jvm) org.sqlite.jni.wrapper1.Tester2 $(Tester2.flags) test-sqllog: $(test.deps) @echo "Testing with -sqllog..." $(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 $(Tester1.flags) -sqllog test-mt: $(test.deps) @echo "Testing in multi-threaded mode:"; $(bin.java) $(test.flags.jvm) org.sqlite.jni.capi.Tester1 \ -t 7 -r 50 -shuffle $(Tester1.flags) $(bin.java) $(test.flags.jvm) org.sqlite.jni.wrapper1.Tester2 \ -t 7 -r 50 -shuffle $(Tester2.flags) test: test-one test-mt tests: test test-sqllog tester.scripts := $(sort $(wildcard $(dir.src)/tests/*.test)) tester.flags ?= # --verbose .PHONY: tester tester-local tester-ext ifeq (1,$(enable.tester)) tester-local: $(CLASS_FILES.tester) $(package.dll) $(bin.java) -ea -Djava.library.path=$(dir.bld.c) \ $(java.flags) -cp $(classpath) \ org.sqlite.jni.capi.SQLTester $(tester.flags) $(tester.scripts) tester: tester-local else tester: @echo "SQLTester support is disabled." endif tester.extdir.default := $(dir.tests)/ext tester.extdir ?= $(tester.extdir.default) tester.extern-scripts := $(wildcard $(tester.extdir)/*.test) ifneq (,$(tester.extern-scripts)) tester-ext: $(bin.java) -ea -Djava.library.path=$(dir.bld.c) \ $(java.flags) -cp $(classpath) \ org.sqlite.jni.capi.SQLTester $(tester.flags) $(tester.extern-scripts) else tester-ext: @echo "******************************************************"; \ echo "*** Include the out-of-tree test suite in the 'tester'"; \ echo "*** target by either symlinking its directory to"; \ echo "*** $(tester.extdir.default) or passing it to make"; \ echo "*** as tester.extdir=/path/to/that/dir."; \ echo "******************************************************"; endif tester-ext: tester-local tester: tester-ext tests: tester ######################################################################## # Build each SQLITE_THREADMODE variant and run all tests against them. multitest: clean define MULTIOPT multitest: multitest-$(1) multitest-$(1): $$(MAKE) opt.debug=$$(opt.debug) $(patsubst %,opt.%,$(2)) \ tests clean enable.fts5=1 endef $(eval $(call MULTIOPT,01,threadsafe=0 oom=1)) $(eval $(call MULTIOPT,00,threadsafe=0 oom=0)) $(eval $(call MULTIOPT,11,threadsafe=1 oom=1)) $(eval $(call MULTIOPT,10,threadsafe=1 oom=0)) $(eval $(call MULTIOPT,21,threadsafe=2 oom=1)) $(eval $(call MULTIOPT,20,threadsafe=2 oom=0)) ######################################################################## # jar bundle... package.jar.in := $(abspath $(dir.src)/jar.in) CLEAN_FILES += $(package.jar.in) JAVA_FILES.jar := $(JAVA_FILES.main) $(JAVA_FILES.unittest) $(JAVA_FILES.package.info) CLASS_FILES.jar := $(filter-out %/package-info.class,$(JAVA_FILES.jar:.java=.class)) $(package.jar.in): $(package.dll) $(MAKEFILE) ls -1 \ $(dir.src.jni)/*/*.java $(dir.src.jni)/*/*.class \ | sed -e 's,^$(dir.src)/,,' | sort > $@ $(package.jar): $(CLASS_FILES.jar) $(MAKEFILE) $(package.jar.in) @rm -f $(dir.src)/c/*~ $(dir.src.jni)/*~ cd $(dir.src); $(bin.jar) -cfe ../$@ org.sqlite.jni.capi.Tester1 @$(package.jar.in) @ls -la $@ @echo "To use this jar you will need the -Djava.library.path=DIR/CONTAINING/libsqlite3-jni.so flag." @echo "e.g. java -Djava.library.path=bld -jar $@" jar: $(package.jar) run-jar: $(package.jar) $(package.dll) $(bin.java) -Djava.library.path=$(dir.bld) -jar $(package.jar) $(run-jar.flags) ######################################################################## # javadoc... dir.doc := $(dir.jni)/javadoc doc.index := $(dir.doc)/index.html javadoc.exclude := -exclude org.sqlite.jni.fts5 # ^^^^ 2023-09-13: elide the fts5 parts from the public docs for # the time being, as it's not clear where the Java bindings for # those bits are going. # javadoc.exclude += -exclude org.sqlite.jni.capi # ^^^^ exclude the capi API only for certain builds (TBD) $(doc.index): $(JAVA_FILES.main) $(MAKEFILE) @if [ -d $(dir.doc) ]; then rm -fr $(dir.doc)/*; fi $(bin.javadoc) -cp $(classpath) -d $(dir.doc) -quiet \ -subpackages org.sqlite.jni $(javadoc.exclude) @echo "javadoc output is in $@" .PHONY: doc javadoc docserve .FORCE: doc doc: $(doc.index) javadoc: $(doc.index) # Force rebuild of docs redoc: @rm -f $(doc.index) @$(MAKE) doc docserve: $(doc.index) cd $(dir.doc) && althttpd -max-age 1 -page index.html ######################################################################## # Clean up... CLEAN_FILES += $(dir.bld.c)/* \ $(dir.src.jni)/*.class \ $(dir.src.jni)/*/*.class \ $(package.dll) \ hs_err_pid*.log .PHONY: clean distclean clean: -rm -f $(CLEAN_FILES) distclean: clean -rm -f $(DISTCLEAN_FILES) -rm -fr $(dir.bld.c) $(dir.doc) ######################################################################## # distribution bundle rules... ifeq (,$(filter snapshot,$(MAKECMDGOALS))) dist-name-prefix := sqlite-jni else dist-name-prefix := sqlite-jni-snapshot-$(shell /usr/bin/date +%Y%m%d) endif dist-name := $(dist-name-prefix)-TEMP dist-dir.top := $(dist-name) dist-dir.src := $(dist-dir.top)/src dist.top.extras := \ README.md .PHONY: dist snapshot dist: \ $(bin.version-info) $(sqlite3.canonical.c) \ $(package.jar) $(MAKEFILE) @echo "Making end-user deliverables..." @echo "****************************************************************************"; \ echo "*** WARNING: be sure to build this with JDK8 (javac 1.8) for compatibility."; \ echo "*** reasons!"; $$($(bin.javac) -version); \ echo "****************************************************************************" @rm -fr $(dist-dir.top) @mkdir -p $(dist-dir.src) @cp -p $(dist.top.extras) $(dist-dir.top)/. @cp -p jar-dist.make $(dist-dir.top)/Makefile @cp -p $(dir.src.c)/*.[ch] $(dist-dir.src)/. @cp -p $(sqlite3.canonical.c) $(sqlite3.canonical.h) $(dist-dir.src)/. @set -e; \ vnum=$$($(bin.version-info) --download-version); \ vjar=$$($(bin.version-info) --version); \ vdir=$(dist-name-prefix)-$$vnum; \ arczip=$$vdir.zip; \ cp -p $(package.jar) $(dist-dir.top)/sqlite3-jni-$${vjar}.jar; \ echo "Making $$arczip ..."; \ rm -fr $$arczip $$vdir; \ mv $(dist-dir.top) $$vdir; \ zip -qr $$arczip $$vdir; \ rm -fr $$vdir; \ ls -la $$arczip; \ set +e; \ unzip -lv $$arczip || echo "Missing unzip app? Not fatal." snapshot: dist .PHONY: dist-clean clean: dist-clean dist-clean: rm -fr $(dist-name) $(wildcard sqlite-jni-*.zip) |
Added ext/jni/README.md.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 | SQLite3 via JNI ======================================================================== This directory houses a Java Native Interface (JNI) binding for the sqlite3 API. If you are reading this from the distribution ZIP file, links to resources in the canonical source tree will note work. The canonical copy of this file can be browsed at: <https://sqlite.org/src/doc/trunk/ext/jni/README.md> Technical support is available in the forum: <https://sqlite.org/forum> > **FOREWARNING:** this subproject is very much in development and subject to any number of changes. Please do not rely on any information about its API until this disclaimer is removed. The JNI bindings released with version 3.43 are a "tech preview." Once finalized, strong backward compatibility guarantees will apply. Project goals/requirements: - A [1-to-1(-ish) mapping of the C API](#1to1ish) to Java via JNI, insofar as cross-language semantics allow for. A closely-related goal is that [the C documentation](https://sqlite.org/c3ref/intro.html) should be usable as-is, insofar as possible, for the JNI binding. - Support Java as far back as version 8 (2014). - Environment-independent. Should work everywhere both Java and SQLite3 do. - No 3rd-party dependencies beyond the JDK. That includes no build-level dependencies for specific IDEs and toolchains. We welcome the addition of build files for arbitrary environments insofar as they neither interfere with each other nor become a maintenance burden for the sqlite developers. Non-goals: - Creation of high-level OO wrapper APIs. Clients are free to create them off of the C-style API. - Virtual tables are unlikely to be supported due to the amount of glue code needed to fit them into Java. - Support for mixed-mode operation, where client code accesses SQLite both via the Java-side API and the C API via their own native code. Such cases would be a minefield of potential mis-interactions between this project's JNI bindings and mixed-mode client code. Hello World ----------------------------------------------------------------------- ```java import org.sqlite.jni.*; import static org.sqlite.jni.CApi.*; ... final sqlite3 db = sqlite3_open(":memory:"); try { final int rc = sqlite3_errcode(db); if( 0 != rc ){ if( null != db ){ System.out.print("Error opening db: "+sqlite3_errmsg(db)); }else{ System.out.print("Error opening db: rc="+rc); } ... handle error ... } // ... else use the db ... }finally{ // ALWAYS close databases using sqlite3_close() or sqlite3_close_v2() // when done with them. All of their active statement handles must // first have been passed to sqlite3_finalize(). sqlite3_close_v2(db); } ``` Building ======================================================================== The canonical builds assumes a Linux-like environment and requires: - GNU Make - A JDK supporting Java 8 or higher - A modern C compiler. gcc and clang should both work. Put simply: ```console $ export JAVA_HOME=/path/to/jdk/root $ make $ make test $ make clean ``` The jar distribution can be created with `make jar`, but note that it does not contain the binary DLL file. A different DLL is needed for each target platform. <a id='1to1ish'></a> One-to-One(-ish) Mapping to C ======================================================================== This JNI binding aims to provide as close to a 1-to-1 experience with the C API as cross-language semantics allow. Interface changes are necessarily made where cross-language semantics do not allow a 1-to-1, and judiciously made where a 1-to-1 mapping would be unduly cumbersome to use in Java. In all cases, this binding makes every effort to provide semantics compatible with the C API documentation even if the interface to those semantics is slightly different. Any cases which deviate from those semantics (either removing or adding semantics) are clearly documented. Where it makes sense to do so for usability, Java-side overloads are provided which accept or return data in alternative forms or provide sensible default argument values. In all such cases they are thin proxies around the corresponding C APIs and do not introduce new semantics. In a few cases, Java-specific capabilities have been added in new APIs, all of which have "_java" somewhere in their names. Examples include: - `sqlite3_result_java_object()` - `sqlite3_column_java_object()` - `sqlite3_value_java_object()` which, as one might surmise, collectively enable the passing of arbitrary Java objects from user-defined SQL functions through to the caller. Golden Rule: Garbage Collection Cannot Free SQLite Resources ------------------------------------------------------------------------ It is important that all databases and prepared statement handles get cleaned up by client code. A database cannot be closed if it has open statement handles. `sqlite3_close()` fails if the db cannot be closed whereas `sqlite3_close_v2()` recognizes that case and marks the db as a "zombie," pending finalization when the library detects that all pending statements have been closed. Be aware that Java garbage collection _cannot_ close a database or finalize a prepared statement. Those things require explicit API calls. Classes for which it is sensible support Java's `AutoCloseable` interface so can be used with try-with-resources constructs. Golden Rule #2: _Never_ Throw from Callbacks (Unless...) ------------------------------------------------------------------------ All routines in this API, barring explicitly documented exceptions, retain C-like semantics. For example, they are not permitted to throw or propagate exceptions and must return error information (if any) via result codes or `null`. The only cases where the C-style APIs may throw is through client-side misuse, e.g. passing in a null where it may cause a `NullPointerException`. The APIs clearly mark function parameters which should not be null, but does not generally actively defend itself against such misuse. Some C-style APIs explicitly accept `null` as a no-op for usability's sake, and some of the JNI APIs deliberately return an error code, instead of segfaulting, when passed a `null`. Client-defined callbacks _must never throw exceptions_ unless _very explicitly documented_ as being throw-safe. Exceptions are generally reserved for higher-level bindings which are constructed to specifically deal with them and ensure that they do not leak C-level resources. In some cases, callback handlers are permitted to throw, in which cases they get translated to C-level result codes and/or messages. If a callback which is not permitted to throw throws, its exception may trigger debug output but will otherwise be suppressed. The reason some callbacks are permitted to throw and others not is because all such callbacks act as proxies for C function callback interfaces and some of those interfaces have no error-reporting mechanism. Those which are capable of propagating errors back through the library convert exceptions from callbacks into corresponding C-level error information. Those which cannot propagate errors necessarily suppress any exceptions in order to maintain the C-style semantics of the APIs. Unwieldy Constructs are Re-mapped ------------------------------------------------------------------------ Some constructs, when modelled 1-to-1 from C to Java, are unduly clumsy to work with in Java because they try to shoehorn C's way of doing certain things into Java's wildly different ways. The following subsections cover those, starting with a verbose explanation and demonstration of where such changes are "really necessary"... ### Custom Collations A prime example of where interface changes for Java are necessary for usability is [registration of a custom collation](https://sqlite.org/c3ref/create_collation.html): ```c // C: int sqlite3_create_collation(sqlite3 * db, const char * name, int eTextRep, void *pUserData, int (*xCompare)(void*,int,void const *,int,void const *)); int sqlite3_create_collation_v2(sqlite3 * db, const char * name, int eTextRep, void *pUserData, int (*xCompare)(void*,int,void const *,int,void const *), void (*xDestroy)(void*)); ``` The `pUserData` object is optional client-defined state for the `xCompare()` and/or `xDestroy()` callback functions, both of which are passed that object as their first argument. That data is passed around "externally" in C because that's how C models the world. If we were to bind that part as-is to Java, the result would be awkward to use (^Yes, we tried this.): ```java // Java: int sqlite3_create_collation(sqlite3 db, String name, int eTextRep, Object pUserData, xCompareType xCompare); int sqlite3_create_collation_v2(sqlite3 db, String name, int eTextRep, Object pUserData, xCompareType xCompare, xDestroyType xDestroy); ``` The awkwardness comes from (A) having two distinctly different objects for callbacks and (B) having their internal state provided separately, which is ill-fitting in Java. For the sake of usability, C APIs which follow that pattern use a slightly different Java interface: ```java int sqlite3_create_collation(sqlite3 db, String name, int eTextRep, SomeCallbackType collation); ``` Where the `Collation` class has an abstract `call()` method and no-op `xDestroy()` method which can be overridden if needed, leading to a much more Java-esque usage: ```java int rc = sqlite3_create_collation(db, "mycollation", SQLITE_UTF8, new SomeCallbackType(){ // Required comparison function: @Override public int call(byte[] lhs, byte[] rhs){ ... } // Optional finalizer function: @Override public void xDestroy(){ ... } // Optional local state: private String localState1 = "This is local state. There are many like it, but this one is mine."; private MyStateType localState2 = new MyStateType(); ... }); ``` Noting that: - It is possible to bind in call-scope-local state via closures, if desired, as opposed to packing it into the Collation object. - No capabilities of the C API are lost or unduly obscured via the above API reshaping, so power users need not make any compromises. - In the specific example above, `sqlite3_create_collation_v2()` becomes superfluous because the provided interface effectively provides both the v1 and v2 interfaces, the difference being that overriding the `xDestroy()` method effectively gives it v2 semantics. ### User-defined SQL Functions (a.k.a. UDFs) The [`sqlite3_create_function()`](https://sqlite.org/c3ref/create_function.html) family of APIs make heavy use of function pointers to provide client-defined callbacks, necessitating interface changes in the JNI binding. The Java API has only one core function-registration function: ```java int sqlite3_create_function(sqlite3 db, String funcName, int nArgs, int encoding, SQLFunction func); ``` > Design question: does the encoding argument serve any purpose in Java? That's as-yet undetermined. If not, it will be removed. `SQLFunction` is not used directly, but is instead instantiated via one of its three subclasses: - `ScalarFunction` implements simple scalar functions using but a single callback. - `AggregateFunction` implements aggregate functions using two callbacks. - `WindowFunction` implements window functions using four callbacks. Search [`Tester1.java`](/file/ext/jni/src/org/sqlite/jni/capi/Tester1.java) for `SQLFunction` for how it's used. Reminder: see the disclaimer at the top of this document regarding the in-flux nature of this API. ### And so on... Various APIs which accept callbacks, e.g. `sqlite3_trace_v2()` and `sqlite3_update_hook()`, use interfaces similar to those shown above. Despite the changes in signature, the JNI layer makes every effort to provide the same semantics as the C API documentation suggests. |
Added ext/jni/jar-dist.make.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | #!/this/is/make #^^^^ help emacs out # # This is a POSIX-make-compatible makefile for building the sqlite3 # JNI library from "dist" zip file. It must be edited to set the # proper top-level JDK directory and, depending on the platform, add a # platform-specific -I directory. It should build as-is with any # 2020s-era version of gcc or clang. It requires JDK version 8 or # higher and that JAVA_HOME points to the top-most installation # directory of that JDK. On Ubuntu-style systems the JDK is typically # installed under /usr/lib/jvm/java-VERSION-PLATFORM. default: all JAVA_HOME = /usr/lib/jvm/java-1.8.0-openjdk-amd64 CFLAGS = \ -fPIC \ -Isrc \ -I$(JAVA_HOME)/include \ -I$(JAVA_HOME)/include/linux \ -I$(JAVA_HOME)/include/apple \ -I$(JAVA_HOME)/include/bsd \ -Wall SQLITE_OPT = \ -DSQLITE_ENABLE_RTREE \ -DSQLITE_ENABLE_EXPLAIN_COMMENTS \ -DSQLITE_ENABLE_STMTVTAB \ -DSQLITE_ENABLE_DBPAGE_VTAB \ -DSQLITE_ENABLE_DBSTAT_VTAB \ -DSQLITE_ENABLE_BYTECODE_VTAB \ -DSQLITE_ENABLE_OFFSET_SQL_FUNC \ -DSQLITE_OMIT_LOAD_EXTENSION \ -DSQLITE_OMIT_DEPRECATED \ -DSQLITE_OMIT_SHARED_CACHE \ -DSQLITE_THREADSAFE=1 \ -DSQLITE_TEMP_STORE=2 \ -DSQLITE_USE_URI=1 \ -DSQLITE_ENABLE_FTS5 \ -DSQLITE_DEBUG sqlite3-jni.dll = libsqlite3-jni.so $(sqlite3-jni.dll): @echo "************************************************************************"; \ echo "*** If this fails to build, be sure to edit this makefile ***"; \ echo "*** to configure it for your system. ***"; \ echo "************************************************************************" $(CC) $(CFLAGS) $(SQLITE_OPT) \ src/sqlite3-jni.c -shared -o $@ @echo "Now try running it with: make test" test.flags = -Djava.library.path=. sqlite3-jni-*.jar test: $(sqlite3-jni.dll) java -jar $(test.flags) java -jar $(test.flags) -t 7 -r 10 -shuffle clean: -rm -f $(sqlite3-jni.dll) all: $(sqlite3-jni.dll) |
Added ext/jni/src/c/sqlite3-jni.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > &g |