SQLite

All files named ”ext/fts3/fts3.c”
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

History for ext/fts3/fts3.c

2025-03-27
18:22
[4f02858ab8] part of check-in [c5206cee83] Rearrange #includes in FTS3 to avoid a harmless compiler warning. (check-in: [c5206cee83] user: drh branch: trunk, size: 209040)
2025-02-27
21:17
[fd2a8642fa] part of check-in [f50c21484d] Approximately 100 typo corrections spanning the whole tree, submitted via forum post 0db9827f0464bc33 and individually audited and verified. Affects only code comments, innocuous test strings, error message text in tool (not library) code, and configure-level help text. (check-in: [f50c21484d] user: stephan branch: trunk, size: 209159)
2025-02-25
20:55
[6f0a91f065] part of check-in [af928818a0] Approximately 50 typo fixes, spanning the whole tree, contributed via forum post 006c8fa165083ac3 and individually checked for correctness. Affects only code comments, docs, and a single line of debug output in a test app. (check-in: [af928818a0] user: stephan branch: trunk, size: 209159)
2025-02-17
18:09
[1da0265e87] part of check-in [e846743a87] Additional changes making it easier to prove that integer overflow does not occur. No problems found. (check-in: [e846743a87] user: drh branch: trunk, size: 209160)
2025-02-03
23:19
[b840ee915a] part of check-in [75f3d87448] Fix a potential UAF in FTS3. (check-in: [75f3d87448] user: drh branch: trunk, size: 209153)
2024-11-09
18:17
[27f09d3e00] part of check-in [17bc57fa16] Fix a case in fts3 where a corrupt database record was not being handled correctly. (check-in: [17bc57fa16] user: drh branch: branch-3.47, size: 208593)
17:54
[9f8ce82bbf] part of check-in [cbcb53759b] Fix a case in fts3 where a corrupt database record was not being handled correctly. (check-in: [cbcb53759b] user: dan branch: trunk, size: 208602)
2024-11-07
18:06
[56c091b9c5] part of check-in [521b873be7] Fix an OOB write that could occur in fts3 when dealing with corrupt database records. (check-in: [521b873be7] user: drh branch: branch-3.47, size: 208357)
17:34
[070d6fec09] part of check-in [108863ec79] Fix an OOB write that could occur in fts3 when dealing with corrupt database records. (check-in: [108863ec79] user: dan branch: trunk, size: 208366)
2024-10-24
12:06
[59ae8c42ed] part of check-in [d4816e534a] Fix an inconsistency in the way SQLITE_CORE is used in fts3.c. (check-in: [d4816e534a] user: dan branch: trunk, size: 208348)
2024-02-01
15:21
[c922380b62] part of check-in [1bdb8cbaf7] Ensure the fts3 xIntegrity method correctly returns error codes unrelated to corruption or missing SQL elements. (check-in: [1bdb8cbaf7] user: dan branch: trunk, size: 208339)
2024-01-16
16:14
[fd64a58847] part of check-in [bb1fe53a97] Fix harmless "unused parameter" compiler warning in the new fts3IntegrityMethod implementation. (check-in: [bb1fe53a97] user: drh branch: trunk, size: 208250)
16:05
[16366d9e42] part of check-in [b855886c4c] Ensure that the xIntegrity methods of fts3 and fts5 work on read-only databases. (check-in: [b855886c4c] user: dan branch: trunk, size: 208221)
2023-10-25
10:37
[d01dfb641f] part of check-in [bc8afa3f15] Enhance the new xIntegrity method of the sqlite3_module object with new parameters that provide the name of the table being checked and a flag to indicate a "quick_check". Based on feedback in forum post 965c0d02ea. (check-in: [bc8afa3f15] user: drh branch: trunk, size: 208438)
2023-10-23
13:46
[7e23ea4675] part of check-in [df69abc082] Improved error messages for the xIntegrity method in FTS3/4 and FTS5. (check-in: [df69abc082] user: drh branch: trunk, size: 208076)
2023-10-21
18:12
[c409b5f921] part of check-in [7f41d7006d] Fix a problem allowing a COMMIT following an OOM to cause fts3/4 corruption. (check-in: [7f41d7006d] user: dan branch: trunk, size: 207783)
2023-10-12
18:46
[89298f42a0] part of check-in [f34c533b6c] Enable SQLITE_VTAB_INNOCUOUS for FTS3, FTS5, and RTREE. (check-in: [f34c533b6c] user: drh branch: trunk, size: 207300)
2023-09-06
14:00
[4095c97f29] part of check-in [f1d4024a8c] Change the xIntegrity virtual table method signature so that it returns an integer error code and writes the error message into a parameter. (check-in: [f1d4024a8c] user: drh branch: vtab-integrity-check, size: 207245)
12:52
[3393dcffc2] part of check-in [17bede8cde] Add the xIntegrity method to the sqlite3_module object. Implement this method in RTREE, FTS3/4, and FTS5 so that "PRAGMA integrity_check" also verifies the correctness of shadow tables associated with those virtual tables. (check-in: [17bede8cde] user: drh branch: vtab-integrity-check, size: 207206)
2023-01-25
21:28
[66d2ced306] part of check-in [68d02d3c2a] Fix a harmless compiler warning in FTS3. (check-in: [68d02d3c2a] user: drh branch: trunk, size: 206583)
15:45
[4b75c00b6c] part of check-in [de4690a10a] Fix a problem with fts3 auxiliary functions and one or more NEAR expressions ORed together. (check-in: [de4690a10a] user: dan branch: trunk, size: 206554)
13:42
[a613b81b20] part of check-in [a8c91c132f] Fix another problem with fts3/4 auxiliary functions and NEAR expressions that consist entirely of deferred tokens. (check-in: [a8c91c132f] user: dan branch: trunk, size: 206566)
2023-01-24
11:24
[36e4555d96] part of check-in [39bfae4c46] Fix a problem with using fts3 auxiliary functions with expressions like "E AND ...", where E is a NEAR expression that consists entirely of deferred tokens. (check-in: [39bfae4c46] user: dan branch: trunk, size: 206144)
2022-09-27
01:53
[46c116ee08] part of check-in [5517bc5098] Make the legacy FTS3 code more robust against integer overflow during memory allocation. (check-in: [5517bc5098] user: drh branch: trunk, size: 206123)
2022-08-05
17:16
[b20bd58399] part of check-in [98729c5fa6] Avoid having fts3 read uninitialized values when processing deferred tokens. (check-in: [98729c5fa6] user: dan branch: trunk, size: 206109)
2022-07-22
20:24
[4aa9e9e8ba] part of check-in [57beb700c0] Add experimental 'xShadowName2' method for virtual table modules. (check-in: [57beb700c0] user: mistachkin branch: xShadowName2, size: 206252)
2022-07-15
15:08
[8cd361bd86] part of check-in [d74f6f6d51] Fix a memory leak in fts3 that could occur when processing a corrupt database. (check-in: [d74f6f6d51] user: dan branch: trunk, size: 206084)
11:34
[9ba24e484d] part of check-in [b072851be1] Update some faulty assert() statements in fts3. (check-in: [b072851be1] user: dan branch: trunk, size: 205890)
2022-07-11
14:26
[5577b9e88b] part of check-in [d0bfe5c574] Fix a problem in fts3 to do with deferred tokens and OR expressions. (check-in: [d0bfe5c574] user: dan branch: trunk, size: 206005)
14:14
[a45ff63c0a] part of check-in [d95bbc93c0] Fix a problem in fts3 to do with deferred tokens and OR expressions. (check-in: [d95bbc93c0] user: dan branch: mistake, size: 206037)
2021-12-30
11:13
[6bfdc94137] part of check-in [35e2b27af7] Change an assert() in FTS3 into assert_fts3_nc(), since it is a harmless condition that can occur when processing a corrupt database file. (check-in: [35e2b27af7] user: drh branch: branch-3.37, size: 205483)
2021-12-25
23:59
[6634a3854e] part of check-in [c03205aee9] Change an assert() in FTS3 into assert_fts3_nc(), since it is a harmless condition that can occur when processing a corrupt database file. (check-in: [c03205aee9] user: drh branch: build-bug, size: 205941)
2021-12-16
15:29
[f27e890395] part of check-in [a0a8d6c9f0] Fix fts3 and fts4 so that they can still be used even if the other is dropped by a call to sqlite3_drop_modules(). (check-in: [a0a8d6c9f0] user: dan branch: trunk, size: 205933)
2021-11-05
12:34
[da41de2162] part of check-in [029c59cdf9] Use the FTS_CORRUPT_VTAB macro rather than a direct call to sqlite3Fts3Corrupt() in [2fad2a89527757b3] so that the build works in all cases. (check-in: [029c59cdf9] user: drh branch: trunk, size: 205475)
2021-11-04
18:04
[27c9167817] part of check-in [2fad2a8952] Fix a crash that could occur in fts3 when processing a corrupt database. (check-in: [2fad2a8952] user: dan branch: trunk, size: 205479)
2021-10-20
11:40
[a28ef00701] part of check-in [7b7d31a615] Fix a case in the fts3 offsets() function where a corrupt database record could lead to dereferencing an uninitialized pointer. (check-in: [7b7d31a615] user: dan branch: trunk, size: 205384)
2021-10-04
18:21
[957237610d] part of check-in [1c67f957fc] Fix harmless static analyzer warnings in sessions, rtree, fts3 and fts5. Add the -DSQLITE_OMIT_AUXILIARY_SAFETY_CHECKS compile-time option to cause ALWAYS() and NEVER() macros to be omitted from the build. (check-in: [1c67f957fc] user: drh branch: fix-warnings, size: 205368)
2021-05-05
11:47
[95f55e2455] part of check-in [a0bf931bd7] Fix an undefined-integer-overflow problem in fts3.c. (check-in: [a0bf931bd7] user: dan branch: trunk, size: 205552)
2021-04-16
17:09
[1d80d0a1e5] part of check-in [fac3ff857c] Fix another undefined behaviour in fts3 that could follow an OOM error. (check-in: [fac3ff857c] user: dan branch: trunk, size: 205530)
2021-04-07
12:59
[8912fedb1b] part of check-in [dd55357833] In FTS3, if the xBegin method fails to start a transaction due to an OOM error, then make sure that the virtual table is left in a consistent state. dbsqlfuzz 85ef48423fda4eef79a551654445d8ef3537a590. (check-in: [dd55357833] user: drh branch: trunk, size: 205538)
2021-03-29
20:04
[d91986eb52] part of check-in [004559544e] Add "#ifdef SQLITE_DEBUG" blocks around test variables sqlite3_fts5_may_be_corrupt and sqlite3_fts3_may_be_corrupt. (check-in: [004559544e] user: dan branch: trunk, size: 205484)
2021-03-06
13:28
[7b44934822] part of check-in [609d949963] Fix a crash in fts3 that could occur when running a NEAR query on a corrupt database. (check-in: [609d949963] user: dan branch: trunk, size: 205457)
2020-08-10
14:18
[4809e0b05a] part of check-in [9d670a3183] Fix harmless compiler warnings that surface in newer versions of GCC. (check-in: [9d670a3183] user: drh branch: trunk, size: 205419)
2020-07-29
16:18
[9ae4acb998] part of check-in [a80ae2c98b] Dozens and dozens of typo fixes in comments. This change adds no value to the end product and is disruptive, so it is questionable whether or not it will ever land on trunk. (check-in: [a80ae2c98b] user: drh branch: typos, size: 205194)
2020-07-23
13:45
[a571f1edf8] part of check-in [28515bbbae] Fix another case where a corrupt record could cause an assert() to fail in fts3. (check-in: [28515bbbae] user: dan branch: trunk, size: 205195)
2020-07-10
11:12
[b8ed676b37] part of check-in [5124732370] Fix a broken assert() in fts3 that could fail when handling corrupt records. (check-in: [5124732370] user: dan branch: trunk, size: 205138)
2020-06-29
13:33
[5ffabd0d13] part of check-in [6e0ffa2053] Fix another fts3 problem with processing corrupt records. (check-in: [6e0ffa2053] user: dan branch: trunk, size: 205130)
2020-06-17
17:27
[2e97a41c7b] part of check-in [ea71fb7fab] Fix an assert() in fts3 that could fail when processing a corrupt record. (check-in: [ea71fb7fab] user: drh branch: branch-3.32, size: 205024)
14:54
[b0e864379f] part of check-in [4adc0a1b0d] Fix an assert() in fts3 that could fail when processing a corrupt record. (check-in: [4adc0a1b0d] user: dan branch: trunk, size: 205097)
12:37
[a93003c89b] part of check-in [d55b8e7993] Merge miscellaneous fixes from trunk into the 3.32 branch. (check-in: [d55b8e7993] user: drh branch: branch-3.32, size: 205016)
2020-06-16
14:06
[ad34168273] part of check-in [a58a6d6fb2] Fix an assert() in fts3 that can fail when processing a corrupt record. (check-in: [a58a6d6fb2] user: dan branch: trunk, size: 205089)
2020-06-12
15:17
[acc51e4378] part of check-in [9a4a40c45f] Fix a buffer overread in fts3 that could occur when decoding a corrupted record. (check-in: [9a4a40c45f] user: dan branch: trunk, size: 205081)
2020-06-08
14:43
[7a12745b02] part of check-in [d48af4d2cf] Fix a case where a corrupted fts3 record could cause an assert() failure, or spurious SQLITE_NOMEM error in builds with assert() disabled. (check-in: [d48af4d2cf] user: dan branch: trunk, size: 205033)
2020-05-16
17:26
[45f5774987] part of check-in [0d69f76f08] Fix a use-after-free bug in the fts3 snippet() function. (check-in: [0d69f76f08] user: dan branch: trunk, size: 204960)
2020-05-07
19:55
[176d51ed5a] part of check-in [cb299a090c] Fix a failing assert() in fts3 triggered by a corrupt database. (check-in: [cb299a090c] user: dan branch: trunk, size: 204916)
2020-04-22
11:11
[de2cc136cc] part of check-in [e256f85289] Fix an integer overflow in fts3 causing a usan error. (check-in: [e256f85289] user: dan branch: trunk, size: 204875)
2020-03-03
01:16
[2a9dd45200] part of check-in [cc99447ac9] Improved detection of corruption in the interior nodes of a segment btree in FTS3/4. (check-in: [cc99447ac9] user: drh branch: trunk, size: 204519)
2020-01-17
14:18
[b3192f3596] part of check-in [cbcbb1e5a9] Import all FTS3/4 enhancements and fixes that exist on the latest trunk (3.31.0-beta) that do not require extensive change to the SQLite core into the 3.22 branch. Basically, the 3.31.0 FTS3 sources are copied into 3.22.0, with minor changes to work around core enhancements that are not available in 3.22.0. (check-in: [cbcbb1e5a9] user: drh branch: branch-3.22, size: 203999)
2019-12-21
20:43
[52c09f4593] part of check-in [70815e273f] Fix a spurious report of corruption from the fts3/4 integrity-check triggered by using the languageid option. (check-in: [70815e273f] user: dan branch: trunk, size: 204425)
2019-12-20
14:08
[41acb66ad7] part of check-in [2bbd014c90] Fix harmless compiler warnings. (check-in: [2bbd014c90] user: drh branch: trunk, size: 204398)
2019-12-14
14:15
[a059387255] part of check-in [3b873029ef] Fix problems in fts3 with processing corrupt records and undefined integer overflows. (check-in: [3b873029ef] user: dan branch: trunk, size: 204395)
2019-12-03
03:31
[9a69143c94] part of check-in [85d95abec4] Avoid a harmless zero offset of a null pointer in FTS3, so as to not provoke unnecessary warnings from run-time checkers. (check-in: [85d95abec4] user: drh branch: trunk, size: 204519)
2019-11-26
02:03
[3fd0b9c984] part of check-in [5d9a369301] Fix an assert() in fts3 that could fail due to corrupt database records. (check-in: [5d9a369301] user: dan branch: trunk, size: 204441)
2019-11-21
14:20
[f2e6743c76] part of check-in [2eb997327c] Prevent direct and indirect recursive content= options in fts3/4 and fts5. (check-in: [2eb997327c] user: dan branch: trunk, size: 204433)
2019-11-20
13:31
[9c2ac0443f] part of check-in [a0f6d526ba] Further improve detection of corrupt records in fts3. Also fix an error with check-in [dfcf081d842629a0] (check-in: [a0f6d526ba] user: dan branch: trunk, size: 204020)
2019-11-18
14:04
[a4c64bf2d3] part of check-in [6b67eba54e] Improvements to detection of corruption in the %_stat shadow table of FTS4. (check-in: [6b67eba54e] user: drh branch: trunk, size: 204020)
11:14
[48e9359e0b] part of check-in [dfcf081d84] Detect and prevent infinite recursion in fts3SelectLeaf() due to a malformed FTS3 btree. (check-in: [dfcf081d84] user: drh branch: trunk, size: 203927)
2019-11-17
02:41
[493e747000] part of check-in [1e44968788] Better detection of corruption in the %_stat and %_docsize shadow tables of FTS3. (check-in: [1e44968788] user: drh branch: trunk, size: 203771)
2019-11-16
23:47
[ad760e49a6] part of check-in [51525f9c32] More improvements to shadow table corruption detection in FTS3. (check-in: [51525f9c32] user: drh branch: trunk, size: 203134)
21:40
[6184d5748b] part of check-in [04b2873be5] Improved detection of corrupt shadow tables in FTS3. Enable the debugging special-inserts for FTS3 for both SQLITE_DEBUG and SQLITE_TEST. (check-in: [04b2873be5] user: drh branch: trunk, size: 203041)
2019-11-01
13:37
[c75aa32b23] part of check-in [7c52f5478f] Fix an fts3 assert() that could fail when operating on a database containing corrupt records. (check-in: [7c52f5478f] user: dan branch: trunk, size: 203010)
2019-09-28
11:19
[a01da95e84] part of check-in [752679aea5] In FTS3/4, the poslist end marker must be larger than any other possible poslist value, even on a corrupt poslist. (check-in: [752679aea5] user: drh branch: trunk, size: 202958)
2019-09-19
13:51
[32e913e667] part of check-in [601ce9532d] Fix an assert() in fts3 that could fail when accessing a corrupt database. (check-in: [601ce9532d] user: dan branch: trunk, size: 202909)
2019-09-18
17:22
[91a5b302a1] part of check-in [40e29a47d1] Add extra tests for the handling of corrupt records in fts3. (check-in: [40e29a47d1] user: dan branch: trunk, size: 202863)
2019-03-08
13:27
[33958a8c05] part of check-in [fcc24edde4] Fix another segfault in fts3 that could occur with corrupt records. (check-in: [fcc24edde4] user: dan branch: trunk, size: 202781)
2019-01-28
16:50
[5da1329ccf] part of check-in [a9faf90339] Fix a buffer overread in fts3 that could occur when accessing a corrupt database. (check-in: [a9faf90339] user: dan branch: trunk, size: 202767)
2019-01-26
19:09
[014e2bef60] part of check-in [b8dd2d67b5] Fix a broken assert() in fts3. (check-in: [b8dd2d67b5] user: dan branch: rollback-abort, size: 202704)
2019-01-25
14:48
[229d0d6e08] part of check-in [3498908cd7] Add an assert_fts3_nc() macro to fts3 - for assert() conditions that are only true when it is guaranteed that the fts3 database is not corrupt. (check-in: [3498908cd7] user: dan branch: trunk, size: 202746)
2019-01-24
17:41
[c8b68202dd] part of check-in [d0d5689371] Fix a buffer overread in fts3 that could occur in a prefix query on a corrupted database. (check-in: [d0d5689371] user: dan branch: trunk, size: 202404)
2019-01-21
17:57
[560cc692cf] part of check-in [505ed9a478] Avoid deferencing a freed pointer following an OOM or SQLITE_CORRUPT error in the fts3 xDestroy method. (check-in: [505ed9a478] user: dan branch: trunk, size: 202328)
16:12
[63602028ef] part of check-in [6c33a303eb] Remove a faulty assert() from fts3. (check-in: [6c33a303eb] user: dan branch: trunk, size: 202375)
2019-01-18
21:17
[90352639eb] part of check-in [fbd681dce2] Fix a memory leak introduced by [55c5d72a]. (check-in: [fbd681dce2] user: dan branch: trunk, size: 202395)
21:03
[ec300b0af6] part of check-in [55c5d72af9] Fix an infinite loop caused by a corrupt database in fts3. Also an undefined left-shift in fts5. (check-in: [55c5d72af9] user: dan branch: trunk, size: 202326)
2019-01-17
03:43
[230bfe6509] part of check-in [10ffc1fe00] Ensure that the variable-length integer decoder in FTS3 never tries to left-shift a negative number. (check-in: [10ffc1fe00] user: drh branch: trunk, size: 202146)
2019-01-15
16:14
[9f441b5c3e] part of check-in [e54efd60c2] Fix a buffer overread in fts3 caused by a corrupt record. (check-in: [e54efd60c2] user: dan branch: trunk, size: 202124)
2019-01-12
00:12
[5131338642] part of check-in [567be3bb1e] Improved shadow table corruption detection in the matchinfo() function of FTS3. (check-in: [567be3bb1e] user: drh branch: trunk, size: 202108)
2019-01-11
21:34
[6b5029db69] part of check-in [2d7b1d1d41] Fix a segfault in fts3 prompted by a corrupted database. (check-in: [2d7b1d1d41] user: dan branch: trunk, size: 202060)
2019-01-08
20:02
[4636bd255f] part of check-in [ca67f2ec0e] Use 64-bit math to compute the sizes of memory allocations in extensions. (check-in: [ca67f2ec0e] user: drh branch: trunk, size: 202051)
2018-12-26
11:39
[5adba0be37] part of check-in [2fa63a8be6] Avoid a left-shift of a negative value (undefined behaviour) when dealing with a corrupt database in fts3. Cherrypick of [b851d12474]. (check-in: [2fa63a8be6] user: dan branch: branch-3.22, size: 201469)
2018-12-22
09:39
[6cf87a0f51] part of check-in [b851d12474] Avoid a left-shift of a negative value (undefined behaviour) when dealing with a corrupt database in fts3. (check-in: [b851d12474] user: dan branch: trunk, size: 201982)
2018-12-19
17:05
[cac1ce084b] part of check-in [553a923c82] Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. Cherrypick from commit [8201f4e1] on branch-3.18. (check-in: [553a923c82] user: dan branch: branch-3.19, size: 201188)
16:03
[2e76a808d0] part of check-in [4cb67252d3] Add interfaces sqlite3_bind_pointer(), sqlite3_result_pointer(), and sqlite3_value_pointer() used to safely move pointer values through SQL without exposing underlying memory address information. Cherrypick from commit [8201f4e1] on branch-3.18. (check-in: [4cb67252d3] user: dan branch: branch-3.9, size: 200914)
14:44
[d890fb31ad] part of check-in [de07814857] Fix a crash that can follow an OOM in fts3 on this branch. (check-in: [de07814857] user: dan branch: branch-3.19, size: 201360)
01:57
[ba4d2e512a] part of check-in [882ef4e39b] Add extra defenses against strategically corrupt databases to fts3/4. (check-in: [882ef4e39b] user: drh branch: branch-3.9, size: 201102)
01:44
[db4466fa7f] part of check-in [4f315e4a3d] Add extra defenses against strategically corrupt databases to fts3/4. (check-in: [4f315e4a3d] user: drh branch: branch-3.19, size: 201303)
01:38
[10aa75a096] part of check-in [4bb21d8205] Add extra defenses against strategically corrupt databases to fts3/4. (check-in: [4bb21d8205] user: drh branch: branch-3.18, size: 201754)
01:30
[829e2943ac] part of check-in [c255889bd9] Add extra defenses against strategically corrupt databases to fts3/4. (check-in: [c255889bd9] user: drh branch: branch-3.22, size: 201419)
2018-11-27
09:57
[65b8489e35] part of check-in [6f4b0d0494] Add an "#ifndef SQLITE_DISABLE_FTS4_DEFERRED" block around function fts3EvalDeferredPhrase() in fts3 in order to silence a compiler warning. (check-in: [6f4b0d0494] user: dan branch: trunk, size: 201932)
2018-11-05
23:01
[eddaba472f] part of check-in [31942b3dd3] Initial code to make shadow tables read-only to ordinary SQL. The now xShadowName method is added to the sqlite3_module object and is used to identify potential shadow tables. The SQLITE_PREPARE_SHADOW argument to sqlite3_prepare_v3() is defined. It is designed to permit writing to shadow tables, but is currently an unused placeholder. (check-in: [31942b3dd3] user: drh branch: read-only-shadow, size: 201853)
2018-11-03
16:51
[012384f01d] part of check-in [d44318f590] Add extra defenses against strategically corrupt databases to fts3/4. (check-in: [d44318f590] user: dan branch: trunk, size: 201427)
2018-09-10
16:38
[53020b61da] part of check-in [564fa75195] Fix a broken assert() in fts3.c. (check-in: [564fa75195] user: dan branch: trunk, size: 201395)
2018-04-27
18:05
[b577921c7e] part of check-in [576a8f69ae] Update test script fts3expr4.test so that it always creates fts3 tokenizers in the "en_US" locality. (check-in: [576a8f69ae] user: dan branch: trunk, size: 201394)
2017-07-27
14:04
[53413c372c] part of check-in [49cf31da82] A demonstration of how FTS3 cursor pointers can be passed from the MATCH operator to the snippet() function securely and opaquely without having to use the sqlite3_result_pointer() and sqlite3_value_pointer() interfaces. (check-in: [49cf31da82] user: drh branch: pass-pointer-as-blob, size: 201554)
03:48
[f1c58503bc] part of check-in [3d9e841f60] Add a destructor argument to sqlite3_bind_pointer() and sqlite3_result_pointer(). (check-in: [3d9e841f60] user: drh branch: pointer-with-destructor, size: 201387)
2017-07-21
07:56
[8825caa44d] part of check-in [8201f4e1c5] Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This changes is a cherry-pick of [f0f49224] via [69906880cee] and is ultimately inspired by check-in [72de49f2]. (check-in: [8201f4e1c5] user: drh branch: branch-3.18, size: 201722)
07:45
[c936a1563f] part of check-in [69906880ce] Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This changes is a cherry-pick of [f0f49224] with version-specific edits. That check-in was inspired by check-in [72de49f2]. (check-in: [69906880ce] user: drh branch: branch-3.9.2, size: 200931)
04:08
[ca29618d3b] part of check-in [5a3022e081] Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This is a cherry-pick of [f0f49224] which was in turn inspired by [72de49f2]. (check-in: [5a3022e081] user: drh branch: branch-3.8.10, size: 199987)
03:23
[48fed98ac4] part of check-in [a66a5b397b] Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This is a cherry-pick of [f0f492245e95], which is in turn inspired by check-in [72de49f2]. (check-in: [a66a5b397b] user: drh branch: branch-3.8.6, size: 197270)
03:09
[eca471e5a9] part of check-in [f0f492245e] Add new interfaces sqlite3_result_pointer(), and sqlite3_value_pointer() and use them to transfer the eponymous FTS3 column pointer to the snippet() and offsets() routines. This changes is inspired by check-in [72de49f2] but is new implementation, not a cherry-pick. (check-in: [f0f492245e] user: drh branch: branch-3.7.11, size: 176353)
2017-07-17
09:30
[839018ad38] part of check-in [f4f7196b87] Use integer handles for fts3 cursors for a small performance improvement. (check-in: [f4f7196b87] user: dan branch: fts3-int-cursor, size: 202040)
00:40
[dfda8bb464] part of check-in [211cce04e9] Add an experimental "pointer type" parameter to sqlite3_bind_pointer(), sqlite3_result_pointer(), and sqlite3_value_pointer(). The pointer type is a string that must compare equal using strcmp() or else the pointer comes through as a NULL. (check-in: [211cce04e9] user: drh branch: pointer-types, size: 201384)
2017-07-15
10:44
[127b3f966c] part of check-in [d6a44b352d] Add new interfaces sqlite3_bind_pointer(), sqlite3_result_pointer(), and sqlite3_value_pointer() used to safely move pointer values through SQL without exposing underlying memory address information. (check-in: [d6a44b352d] user: drh branch: branch-3.17, size: 201505)
2017-07-13
17:48
[02fbd22153] part of check-in [0fa2e170e5] Update the FTS3 extension to use sqlite3_result_pointer() and sqlite3_value_pointer() for passing FTS3 cursor objects from the table into functions such as snippet(). (check-in: [0fa2e170e5] user: drh branch: bind-pointer, size: 201356)
2017-07-05
18:48
[b47d3ba5c0] part of check-in [b6b14ab6c8] Have fts3 virtual table cursors free internal resources when they reach EOF, instead of waiting until the xClose method is called. (check-in: [b6b14ab6c8] user: dan branch: trunk, size: 201464)
2017-06-28
17:29
[45ea4573f1] part of check-in [942c3ef8cd] Instead of the new sqlite3_prepare_v3() interface, provide the SQLITE_DBCONFIG_PREPARE_FLAGS interface which sets the flags on the single next call to sqlite3_prepare_v2() or its cousins. (check-in: [942c3ef8cd] user: drh branch: dbconfig-prepare-flags, size: 201447)
2017-06-01
00:54
[6c5543ce77] part of check-in [4a25c58833] Add interfaces sqlite3_prepare_v3() and sqlite3_prepare16_v3() with the extra prepFlags argument. Add the SQLITE_PREPARE_PERSISTENT option as one bit in that argument. (check-in: [4a25c58833] user: drh branch: prepare_v3, size: 201316)
2017-05-02
18:00
[88ea5c444f] part of check-in [430f539cbb] Fix the fts3EvalAverageDocsize() routine so that it returns errors from sqlite3_reset() rather than always returning SQLITE_OK. (check-in: [430f539cbb] user: drh branch: trunk, size: 201271)
2017-04-19
13:25
[10fc22119e] part of check-in [6b21d0fdeb] Further improvements to coverage of fts3 module. (check-in: [6b21d0fdeb] user: dan branch: trunk, size: 201283)
07:33
[aba99577b6] part of check-in [ea8a0d2ce0] Further modifications and test cases to improve test coverage of fts3. (check-in: [ea8a0d2ce0] user: dan branch: trunk, size: 201438)
2017-04-18
13:50
[43dbd76489] part of check-in [dc2a48020a] Use sqlite3_table_column_metadata() instead of a SELECT statement to check for the existence of a %_stat table in fts3. This leads to smaller and easier to test code. (check-in: [dc2a48020a] user: dan branch: trunk, size: 201455)
05:49
[1f7f2b23b1] part of check-in [e47fdb493b] Refactor the fts3ColumnMethod() function so that all branches can be covered. (check-in: [e47fdb493b] user: dan branch: trunk, size: 201697)
2017-04-17
16:07
[1db0b7086d] part of check-in [352413eed4] Further improvements to test coverage in fts3. (check-in: [352413eed4] user: dan branch: trunk, size: 201860)
13:38
[a05d3acd67] part of check-in [1a08a836fa] Improve coverage of code in fts3. (check-in: [1a08a836fa] user: dan branch: trunk, size: 201791)
2017-03-20
19:35
[f0d5de1bc2] part of check-in [49b93d972d] Avoid passing NULL as the second argument to memcpy() in fts3. (check-in: [49b93d972d] user: dan branch: trunk, size: 201861)
18:53
[df964dbcc7] part of check-in [16a8e84fa7] Fix some problems in fts3 found by address-sanitizer. (check-in: [16a8e84fa7] user: dan branch: trunk, size: 201838)
2017-02-27
14:52
[95c7041ea7] part of check-in [fe41bb5632] Add an sqlite3_set_last_insert_rowid() method. Use it to work around fts4 and fts5 modifying the last-insert-rowid unintuitively from within commit processing. (check-in: [fe41bb5632] user: dan branch: set-last-insert-rowid, size: 201771)
2017-02-08
19:10
[c4d7eecb12] part of check-in [9962c10a5c] Avoid preparing a SELECT statement each time an UPDATE or DELETE by docid is executed against an fts3 table. (check-in: [9962c10a5c] user: dan branch: fts3-seekstmt-cache, size: 201660)
2017-02-01
22:43
[5a8cafedff] part of check-in [997f765bc6] Fix harmless compiler warnings seen with MSVC. (check-in: [997f765bc6] user: mistachkin branch: msvcWarn, size: 201086)
2015-09-29
11:57
[e028eb1343] part of check-in [f61203bc0f] Change the name of the new sqlite3_index_info.flags field to "idxFlags". Add documentation for the same to sqlite.h.in. (check-in: [f61203bc0f] user: dan branch: vtab-onepass, size: 201070)
2015-09-28
15:23
[52232f39ba] part of check-in [fffab4f70f] Update fts3 to use the onepass strategy for delete operations. (check-in: [fffab4f70f] user: dan branch: vtab-onepass, size: 201067)
2015-07-25
19:26
[b04b0c5776] part of check-in [ee348b1275] Remove a duplicated "if" condition from fts3.c. (check-in: [ee348b1275] user: dan branch: trunk, size: 200507)
2015-07-08
17:59
[d2f7981f4d] part of check-in [372c1db247] Fix two problems that could cause fts3 auxiliary functions to occasionally misbehave if used with match expressions that contain both OR and NEAR. (check-in: [372c1db247] user: dan branch: trunk, size: 200562)
2015-05-25
10:57
[a95de5190c] part of check-in [f1e942a1dd] Fix an fts4 matchinf() problem triggered by deferred tokens that are part of phrases that are part of AND expressions. (check-in: [f1e942a1dd] user: dan branch: trunk, size: 200093)
2015-05-21
02:24
[64d50e38c2] part of check-in [000197cc4e] Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. Cherrypick [ad4b19d2ac0889a] (check-in: [000197cc4e] user: drh branch: branch-3.7.11, size: 176492)
01:04
[a2a545a38d] part of check-in [0f0694e424] Do not allow virtual table constructors to be called recursively. Cherrypick [0a72726da21581ab] (check-in: [0f0694e424] user: drh branch: branch-3.7.11, size: 176414)
2015-05-20
20:34
[47ca4807af] part of check-in [7d7d633c71] Fix a problem causing the fts3 integrity-check to fail if run inside a transaction. Cherrypick of [3b925189a75e]. (check-in: [7d7d633c71] user: dan branch: branch-3.8.6, size: 197409)
19:50
[31686f9b19] part of check-in [023a29baf0] Do not allow virtual table constructors to be called recursively. Cherrypick of [0a72726da215] (check-in: [023a29baf0] user: dan branch: branch-3.8.6, size: 197401)
19:32
[d41ed0df22] part of check-in [ff7be4db84] Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. Cherrypick of [ad4b19d2ac0889a2]. (check-in: [ff7be4db84] user: dan branch: branch-3.8.6, size: 197250)
2015-05-05
19:37
[341e9d9a3c] part of check-in [dddd7e1829] Optimizations for the matchinfo() function, particularly the 'y' flag. (check-in: [dddd7e1829] user: dan branch: fts3-matchinfo-y, size: 200146)
2015-05-01
14:07
[2fb98467f4] part of check-in [605347e087] Use the sqlite3Fts3ErrMsg() function to set error message text in FTS3, and to ensure that error messages are not overwritten and thus leaked. (check-in: [605347e087] user: drh branch: trunk, size: 200126)
2015-04-23
11:52
[81f9ed55ad] part of check-in [3b925189a7] Fix a problem causing the fts3 integrity-check to fail if run inside a transaction. (check-in: [3b925189a7] user: dan branch: trunk, size: 199896)
2015-04-21
12:06
[1b198ddb76] part of check-in [caba5be0b2] Fix an fts4 problem to do with the handling of phrases that contain zero tokens. (check-in: [caba5be0b2] user: dan branch: trunk, size: 199888)
2015-04-17
20:51
[29300a76fa] part of check-in [284c1623f8] Fix a problem affecting some fts3 UPDATE and DELETE statements on order=DESC tables. (check-in: [284c1623f8] user: dan branch: trunk, size: 199850)
2015-04-16
20:10
[cb5019020e] part of check-in [55d10baf0b] Fix a boundry-value condition in the phrase poslist extractor of FTS3/4. (check-in: [55d10baf0b] user: drh branch: trunk, size: 198926)
2015-04-15
09:16
[4bd7528987] part of check-in [bd06eeb8d0] Remove an incorrect assert() statement from sqlite3Fts3Dequote(). (check-in: [bd06eeb8d0] user: dan branch: trunk, size: 198883)
08:20
[ed6624d7c5] part of check-in [eafd0a1e3f] Fix a problem causing an assert() to fail if a snippet containing 0 tokens was requested from fts3. (check-in: [eafd0a1e3f] user: dan branch: trunk, size: 198707)
2015-04-10
07:55
[57d863c3bd] part of check-in [0a72726da2] Do not allow virtual table constructors to be called recursively. (check-in: [0a72726da2] user: dan branch: trunk, size: 198624)
2015-04-06
09:05
[23bd9d37a7] part of check-in [0cdf502885] Fix a problem with fts3 prefix terms within phrase queries on "order=DESC" tables with a mix of negative and positive rowids. (check-in: [0cdf502885] user: dan branch: fts3-prefix-query-fix, size: 198473)
2015-03-26
14:29
[d3f6f0e95d] part of check-in [1688f60c08] Fix a harmless compiler warning in FTS3. (check-in: [1688f60c08] user: drh branch: trunk, size: 197195)
2015-03-21
20:50
[2a1cf23133] part of check-in [235157de51] Add a missing "int" on a constant declaration. (check-in: [235157de51] user: drh branch: trunk, size: 197195)
12:22
[e2c7e61d67] part of check-in [e3e2346496] Avoid an integer overflow in fts3 causing gcc 4.7.1 with -O2 to behave counter-intuitively (perhaps because the behaviour is undefined). Add an "ifcapable trace" to a test in shell4.test. (check-in: [e3e2346496] user: dan branch: trunk, size: 197191)
2015-03-19
15:52
[022915f305] part of check-in [ad4b19d2ac] Silently ignore any attempt to add a prefix index for prefixes zero bytes in size to an fts3/4 table. Or any prefix index size so large that it overflows a 32-bit signed integer. (check-in: [ad4b19d2ac] user: dan branch: trunk, size: 197064)
2015-01-29
11:52
[56a78f7e65] part of check-in [85dc12625d] Optimize range constraints on the rowid column of fts3/4 tables even if there is no MATCH clause in the query. (check-in: [85dc12625d] user: dan branch: trunk, size: 196986)
2015-01-27
21:24
[3b2f792afc] part of check-in [e7d2ec048c] Fix harmless compiler warnings. (check-in: [e7d2ec048c] user: mistachkin branch: trunk, size: 196722)
18:43
[845f20440d] part of check-in [245e873045] Improve the performance of fts3/4 queries that use the OR operator and at least one auxiliary fts function. (check-in: [245e873045] user: dan branch: trunk, size: 196803)
2015-01-12
18:38
[5c46481650] part of check-in [2b8eecbfe7] Fix some harmless compiler warnings. (check-in: [2b8eecbfe7] user: mistachkin branch: msvcW4, size: 197236)
2014-10-09
14:00
[8b6cceb3e0] part of check-in [fb8da82411] Fix a memory leak associated with the FTS4 matchinfo() function. (check-in: [fb8da82411] user: drh branch: trunk, size: 197224)
2014-10-06
12:41
[66f39c425f] part of check-in [418f3c9ad2] Fix a harmless compiler warning inside an assert() in FTS4. (check-in: [418f3c9ad2] user: drh branch: trunk, size: 197190)
2014-07-03
12:18
[2f5e925bdb] part of check-in [0cc0230ae9] Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE. (check-in: [0cc0230ae9] user: dan branch: trunk, size: 197172)
2014-05-27
19:18
[20bc65862c] part of check-in [20b75460c0] Fix harmless compiler warnings. (check-in: [20b75460c0] user: drh branch: trunk, size: 197172)
2014-05-26
16:40
[0a9813c01c] part of check-in [d90c4964fc] Fix a problem in FTS4 where columns with names that are prefixes of any notindexed column were also being (incorrectly) marked as not indexed. For example in "CREATE ... t1(abc, bc, abcd, notindexed=abcd)", both abc and abcd were being treated as notindexed. (check-in: [d90c4964fc] user: dan branch: trunk, size: 197168)
2014-05-14
15:58
[e83f894cf1] part of check-in [21491a9bc6] Fix various problems to do with segment promotion. Add test file fts4growth2.test, containing tests to check that the FTS index does not grow indefinitely as the table is updated. Allow the user to configure the number of segments merged simultaneously by the automerge option. (check-in: [21491a9bc6] user: dan branch: fts4-experimental, size: 197134)
2014-03-12
19:20
[41b1920b9a] part of check-in [c67a52c356] Changes to FTS to ensure that it does not access the database from within the xConnect method. (check-in: [c67a52c356] user: dan branch: trunk, size: 197084)
2014-01-04
19:58
[3fe91e36a0] part of check-in [d6fcfc8890] Add the usual "fts3" prefix to new static method setEstimatedRows() in fts3.c. This fixes a problem when compiling the amalgamation, as the r-tree module also contains a static method named setEstimatedRows. (check-in: [d6fcfc8890] user: dan branch: trunk, size: 196225)
14:42
[2af2cb2e74] part of check-in [24f84b3813] Instead of having the planner ignore plans with unusable MATCH constraints, have FTS assign extremely high costs to such plans in order to discourage the planner from using them. (check-in: [24f84b3813] user: dan branch: avoid-unusable-match, size: 196217)
2013-11-19
18:17
[1e667eacb3] part of check-in [6cc023bb29] Fix a harmless MSVC compiler warning. (check-in: [6cc023bb29] user: drh branch: trunk, size: 195222)
2013-11-18
11:20
[3ae281c073] part of check-in [8d002740bf] Fix harmless compiler warnings from clang scan-build. (check-in: [8d002740bf] user: drh branch: trunk, size: 195215)
2013-11-12
21:10
[dceaa50798] part of check-in [ddacd10105] Fix harmless compiler warning. (check-in: [ddacd10105] user: mistachkin branch: trunk, size: 195040)
17:46
[7eb3b2935f] part of check-in [aa7ba302ed] When possible, have FTS use 32-bit operations to read varints from the database. (check-in: [aa7ba302ed] user: dan branch: trunk, size: 195051)
2013-10-14
20:30
[f25ae5729d] part of check-in [0bf438fc30] Fix a crash in FTS incremental phrase processing that can occur if the second or subsequent token is much more common in the dataset than the first. (check-in: [0bf438fc30] user: dan branch: trunk, size: 194494)
2013-10-12
02:33
[dcb90d12ff] part of check-in [4b130f88fb] Fix harmless compiler warning. (check-in: [4b130f88fb] user: mistachkin branch: trunk, size: 194483)
2013-10-11
15:05
[1864684d2f] part of check-in [7df06684ab] Fix various harmless compiler warnings. Change the "warnings.sh" script to work with STAT4 instead of STAT3. (check-in: [7df06684ab] user: drh branch: trunk, size: 194478)
2013-10-07
21:49
[543cbd7322] part of check-in [36d64dc36f] Fix compilation issue with MSVC. (check-in: [36d64dc36f] user: mistachkin branch: trunk, size: 194478)
2013-10-03
19:27
[6d277a3ff6] part of check-in [f6819c5f33] Allow FTS4 multi-token phrases to use a combination of in-memory and incrementally loaded doclists. This allows phrases to (partially) benefit from incremental doclists without disabling the deferred token optimization. (check-in: [f6819c5f33] user: dan branch: fts4-docid-range-constraints, size: 194479)
2013-10-02
08:04
[a6b015359e] part of check-in [bc3a2ed5fb] Add a test to check that the new multi-token phrase optimization is actually helping. (check-in: [bc3a2ed5fb] user: dan branch: fts4-docid-range-constraints, size: 192860)
2013-10-01
20:02
[e781a0147f] part of check-in [ea543f081d] Allow multi-token phrases to load doclists from the database incrementally. This allows queries that feature such phrases to benefit from the "docid<?" optimization. (check-in: [ea543f081d] user: dan branch: fts4-docid-range-constraints, size: 192745)
2013-09-30
18:16
[00a56dc5ce] part of check-in [e294a9c7c5] Merge trunk changes with this branch. (check-in: [e294a9c7c5] user: dan branch: fts4-docid-range-constraints, size: 188213)
18:14
[e1240ab6f5] part of check-in [fa0f2f0e3e] Fix a performance problem in the FTS4 auxiliary functions triggered by an OR clause in the full-text query. (check-in: [fa0f2f0e3e] user: dan branch: trunk, size: 185441)
11:42
[2dd0f461e0] part of check-in [6622424a3a] Have fts4 full-text queries consider "docid<?" and similar constraints. (check-in: [6622424a3a] user: dan branch: fts4-docid-range-constraints, size: 186342)
2013-07-09
12:36
[0d6311cd43] part of check-in [1e39f85077] Modify several extensions to use the new exported function naming. Fix some shared library compilation issues. (check-in: [1e39f85077] user: drh branch: trunk, size: 183570)
2013-07-05
10:46
[1370ea15ac] part of check-in [9676280a93] Make a couple of implicit casts explicit to fix compiler warnings. (check-in: [9676280a93] user: dan branch: trunk, size: 183532)
2013-07-04
23:53
[cafcfaf8fc] part of check-in [f2ab874782] Modify several extensions to use the new exported function naming. Fix some shared library compilation issues. (check-in: [f2ab874782] user: mistachkin branch: extRefactor, size: 183558)
2013-06-26
11:43
[6dbb5c4241] part of check-in [0fe31f60ca] Cut over the next generation query planner. Increase the version number to 3.8.0. (check-in: [0fe31f60ca] user: drh branch: trunk, size: 183520)
2013-06-21
17:30
[3cb4afd0ac] part of check-in [8ff2b8f594] Add the "notindexed" option to fts4. (check-in: [8ff2b8f594] user: dan branch: fts4-notindexed, size: 183519)
2013-06-20
18:32
[e88a755aef] part of check-in [81527768ef] Fix some issues related to ORDER BY and fts tables with a non-zero languageid_bits setting. (check-in: [81527768ef] user: dan branch: fts-languageid-bits, size: 183194)
16:22
[6b8ca2c8e5] part of check-in [b1df00f3f1] Add extra tests for fts with a non-zero languageid_bits setting. Fix querying by docid with the same. (check-in: [b1df00f3f1] user: dan branch: fts-languageid-bits, size: 183063)
11:48
[b0c6f12259] part of check-in [8dc261b765] Add tests (and a fix) for large and small fts docid values with various languageid_bits settings. (check-in: [8dc261b765] user: dan branch: fts-languageid-bits, size: 183032)
2013-06-19
20:13
[2d86f9b356] part of check-in [d36d7e6833] Add the languageid_bits= option to fts. Still some problems to work out. (check-in: [d36d7e6833] user: dan branch: fts-languageid-bits, size: 183032)
2013-06-04
13:07
[760fecd100] part of check-in [8d9f29f847] Increase the estimated full-scan cost for an FTS3 table. (check-in: [8d9f29f847] user: drh branch: nextgen-query-plan-exp, size: 182255)
2013-05-27
10:11
[931b3c83ab] part of check-in [3cd2da42e9] When writing to an FTS table, take an exclusive shared-cache lock on the %_segdir table before writing. Otherwise, an xCommit() call may report an SQLITE_LOCKED error. (check-in: [3cd2da42e9] user: dan branch: trunk, size: 182254)
2013-05-07
12:16
[4bc160e6ff] part of check-in [0ba67b64de] Modify the fts3tokenize table implementation so that it does not use the SQL function fts3_tokenizer. The user may have installed an authorizer callback that prohibits this. (check-in: [0ba67b64de] user: dan branch: trunk, size: 182324)
2013-04-29
18:07
[5c3d44d167] part of check-in [f480b1fe60] Improve the error message issued when an FTS query exceeds the maximum allowable tree depth. (check-in: [f480b1fe60] user: dan branch: trunk, size: 182307)
2013-04-22
15:30
[784aadfb4c] part of check-in [c5a4b21a15] Add virtual table module 'fts3tokenize' to fts3. fts3tokenize provides SQL access to FTS tokenizer implementations. (check-in: [c5a4b21a15] user: dan branch: trunk, size: 182411)
2013-03-21
21:20
[0eaedfc6d2] part of check-in [6f6e2d5094] Many spelling fixes in comments. No changes to code. (check-in: [6f6e2d5094] user: mistachkin branch: trunk, size: 182346)
2012-10-25
19:25
[a867cafae0] part of check-in [33f4f9817e] Fix a bug in fts3 causing it to return SQLITE_NOMEM when NEAR, OR and some auxiliary functions were used together. Also a segfault in vtab.c that could follow an OOM condition. (check-in: [33f4f9817e] user: dan branch: trunk, size: 182344)
2012-08-20
17:24
[ab90126ee0] part of check-in [e799222f3b] Add the SQLITE_DISABLE_FTS4_DEFERRED compile time option. (check-in: [e799222f3b] user: dan branch: trunk, size: 182239)
2012-06-06
19:51
[bb3107c0e4] part of check-in [eccd6b6580] Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61. (check-in: [eccd6b6580] user: dan branch: trunk, size: 182142)
2012-05-26
18:28
[41824d0db7] part of check-in [e71495a817] If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer. (check-in: [e71495a817] user: dan branch: fts4-unicode, size: 182142)
14:54
[aff512cbc4] part of check-in [8f3e60aa22] Change the name of the "unicode" tokenizer to "unicode61" to emphasize that the case folding and separator-character identification routines are based on unicode version 6.1. (check-in: [8f3e60aa22] user: dan branch: fts4-unicode, size: 181968)
2012-05-25
17:50
[81c7726429] part of check-in [0c13570ec7] Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. (check-in: [0c13570ec7] user: dan branch: fts4-unicode, size: 181965)
2012-05-10
17:43
[a7adf6747d] part of check-in [4d3e1673b3] Have the FTS auxiliary functions correctly handle terms that appear in non-matching branches of the FTS expression. Fix for [bdc6bbbb38]. (check-in: [4d3e1673b3] user: dan branch: trunk, size: 181729)
2012-05-04
02:58
[5718d78a09] part of check-in [930115693a] Add an experimental tokenizer to FTS3/4: one that transliterates latin, greek, cyrillic, and hebrew characters into pure ascii. (check-in: [930115693a] user: drh branch: translit-tokenizer, size: 178962)
2012-03-30
00:00
[111626ce72] part of check-in [b451c0f97f] Fix compiler warnings on GCC and MSVC and fix a C89-ism that broke the build for MSVC. (check-in: [b451c0f97f] user: drh branch: trunk, size: 178665)
2012-03-27
15:00
[f41f52a24f] part of check-in [5c447e226a] Remove the fts3merge.test script in favour of changing the fts4merge.test script so that it runs tests using both fts4 and fts3. Fix some problems with incr-merge and FTS3 tables. (check-in: [5c447e226a] user: dan branch: fts4-incr-merge, size: 178660)
00:38
[5e2d74de50] part of check-in [fb8aacdd8f] Minor correct errors in the file format description for FTS3/4 contained in the fts3.c header comment. (check-in: [fb8aacdd8f] user: drh branch: fts4-incr-merge, size: 178610)
2012-03-26
14:36
[f2dd409e14] part of check-in [a18c103121] Fix FTS3 so that it works even without SQLITE_DEBUG. (check-in: [a18c103121] user: drh branch: fts4-incr-merge, size: 178556)
10:36
[a36f2add4c] part of check-in [40fc880474] Add an experimental integrity-check function to FTS. (check-in: [40fc880474] user: dan branch: fts4-incr-merge, size: 178568)
2012-03-24
17:29
[95409b4980] part of check-in [cc051fc0b2] Enable fts3 tables to use incremental merge by automatically creating the %_stat table when it is needed. (check-in: [cc051fc0b2] user: drh branch: fts4-incr-merge, size: 178655)
16:43
[10988e19ba] part of check-in [af55ca5fc6] Add a comment to fts3SyncMethod() to justify the nMinMerge=64 constant. (check-in: [af55ca5fc6] user: dan branch: fts4-incr-merge, size: 178082)
16:11
[eb3a57bb79] part of check-in [67a0cffc9d] Remove the Fts3Table.mxLevel variable. (check-in: [67a0cffc9d] user: dan branch: fts4-incr-merge, size: 176929)
14:45
[442f0ccf5c] part of check-in [0d841c957c] Modify the way the number of leaves written and the maximum relative level are calculated in the auto-incr-merge code. (check-in: [0d841c957c] user: dan branch: fts4-auto-incr-merge, size: 176932)
02:20
[ef3d7af4a6] part of check-in [ed69434cd8] An attempt at automatic incremental merging for FTS4. (check-in: [ed69434cd8] user: drh branch: fts4-auto-incr-merge, size: 176597)
2012-03-23
18:26
[c47bb6fe7a] part of check-in [ed7c17ea16] Fix a spurious SQLITE_CONSTRAINT error that may be returned by an incr-merge operation. (check-in: [ed7c17ea16] user: dan branch: fts4-incr-merge, size: 176353)
2012-03-19
14:57
[a62e09140c] part of check-in [bc03d99a78] Fix one more compiler warning missed by the previous check-in. (check-in: [bc03d99a78] user: drh branch: trunk, size: 176263)
14:51
[cb5b77609e] part of check-in [4fd68647c8] Suppress harmless compiler warnings on windows in FTS4 and RTREE. (check-in: [4fd68647c8] user: drh branch: trunk, size: 176258)
14:28
[bd94c66281] part of check-in [7dd97f12cd] Avoid a compiler warning (an incorrect compiler warning, at that) in vs2010. (check-in: [7dd97f12cd] user: drh branch: trunk, size: 176188)
2012-03-05
15:33
[806632fd00] part of check-in [99a9073b5e] Merge the fts4-languageid branch with the trunk. (check-in: [99a9073b5e] user: dan branch: trunk, size: 176185)
2012-03-03
18:46
[fcda9a9ff7] part of check-in [f8e9c445dd] Add the xLanguageid method to sqlite3_fts3_tokenizer versions 1 and greater. (check-in: [f8e9c445dd] user: dan branch: fts4-languageid, size: 176349)
2012-03-02
20:34
[96ae9e273a] part of check-in [74d224b144] Remove an obsolete comment from FTS3. No code changes. (check-in: [74d224b144] user: drh branch: trunk, size: 173516)
19:53
[fd89caa416] part of check-in [22491e7bc3] Fix problems with combining content= and languageid= in a single fts4 table. (check-in: [22491e7bc3] user: dan branch: fts4-languageid, size: 176335)
2012-03-01
19:44
[93a8eb6e6e] part of check-in [bea257f70f] Add the "languageid=" option to fts4. This code is still largely untested and alsmost certainly buggy. (check-in: [bea257f70f] user: dan branch: fts4-languageid, size: 175766)
2012-01-25
16:29
[4cf7b8e5bb] part of check-in [c05c3fd20d] Fix an FTS problem triggered by querying for an N character prefix using an N+1 character prefix index after rows have been deleted from the FTS table. Fix for [edb497982c]. (check-in: [c05c3fd20d] user: dan branch: trunk, size: 173680)
2011-11-04
10:37
[bd570b99f1] part of check-in [e6f825748a] Add a missing va_end() macro to fts3.c. (check-in: [e6f825748a] user: dan branch: trunk, size: 173631)
2011-10-19
18:21
[246ef2d0ce] part of check-in [1af4a25631] Fix harmless compiler warnings in FTS4. (check-in: [1af4a25631] user: drh branch: trunk, size: 173615)
16:20
[9c125699ba] part of check-in [8a4077057d] Merge the fts4-content branch with the trunk. (check-in: [8a4077057d] user: dan branch: trunk, size: 173577)
15:52
[e8ee5c78a3] part of check-in [df36ac9481] Have FTS3 ignore ^ prefixes. The ^ syntax is only supported on FTS4 tables. (check-in: [df36ac9481] user: dan branch: fts4-content, size: 173359)
11:57
[064b660a11] part of check-in [9b58c59eb4] Change the way ^ tokens work in FTS so that the filtering is done as part of reading the FTS index instead of waiting until an entire doclist has been retrieved and then filtering it. (check-in: [9b58c59eb4] user: dan branch: fts4-content, size: 173346)
09:40
[708122f0ed] part of check-in [2c03b24f4c] Fix a problem in FTS to do with ^ tokens and the snippet() function. (check-in: [2c03b24f4c] user: dan branch: fts4-content, size: 173576)
2011-10-18
19:39
[e12a151b5f] part of check-in [63ac33c860] If a token within an FTS query is prefixed with a '^' character, it must be the first token in a column of data to match. (check-in: [63ac33c860] user: dan branch: fts4-content, size: 173552)
12:49
[15e1725f3d] part of check-in [f9b5b21708] Cherrypick patch [3126754c72] from the trunk into the content= branch. (check-in: [f9b5b21708] user: dan branch: fts4-content, size: 171753)
12:44
[19f3694514] part of check-in [3126754c72] Fix a floating-point exception that can occur when an FTS4 query contains a large number of tokens connected by AND or NEAR operators. (check-in: [3126754c72] user: dan branch: trunk, size: 165665)
2011-10-15
00:16
[d02bb1b321] part of check-in [39408702a9] Added the tool/warnings-clang.sh script. Changes so that there are no warnings with either gcc or clang even including FTS4 and RTREE and both with and without SQLITE_THREADSAFE=0. (check-in: [39408702a9] user: drh branch: trunk, size: 165444)
2011-10-14
22:57
[9dd7e3edfe] part of check-in [b3324f6cc2] Fix a few minor and harmless clang warnings in FTS3 and RTREE. (check-in: [b3324f6cc2] user: drh branch: trunk, size: 165427)
2011-10-13
17:16
[0ace6b45d6] part of check-in [7e24645be2] Handle updating the only row of an FTS table correctly. Fix for [9fd058691]. (check-in: [7e24645be2] user: dan branch: trunk, size: 165337)
2011-10-05
15:11
[f2ed0ae669] part of check-in [289ee43179] Change FTS4 so that if both the content=xxx option and column names are specified, the virtual table assumes that the named columns correspond to columns of table xxx. (check-in: [289ee43179] user: dan branch: fts4-content, size: 171532)
2011-10-04
19:41
[6d6f3d331e] part of check-in [0f439944ab] Improve test coverage of fts3.c. (check-in: [0f439944ab] user: dan branch: fts4-content, size: 170826)
16:37
[df14905642] part of check-in [be86c7061b] Add tests to check that modifying the schema of an FTS content table does not cause a crash in the FTS module. Also disable the deferred token optimization for content=xxx FTS tables. (check-in: [be86c7061b] user: dan branch: fts4-content, size: 170553)
11:22
[626fcf477e] part of check-in [1d27ea741f] Add experimental 'content' option to FTS4. (check-in: [1d27ea741f] user: dan branch: fts4-content, size: 170158)
2011-09-13
19:08
[195e4da669] part of check-in [07788c0f7f] Allocate the correct size for the output buffer in fts3DoclistOrMerge(). Fix for [56be976859]. (check-in: [07788c0f7f] user: dan branch: trunk, size: 165226)
2011-06-28
11:58
[f45ad45053] part of check-in [91daea7d2e] Add a fix and tests for the FTS deferred token logic. (check-in: [91daea7d2e] user: dan branch: fts3-changes, size: 163543)
2011-06-27
11:15
[70127bf57c] part of check-in [2c4bbd90e2] Changes to improve the selection of deferred tokens within phrases. (check-in: [2c4bbd90e2] user: dan branch: fts3-changes, size: 162725)
2011-06-23
17:09
[c25723c7e9] part of check-in [8230d83120] Fix some of the code issues (missing comments etc.) in the new FTS code. (check-in: [8230d83120] user: dan branch: fts3-changes, size: 162566)
00:59
[ca77603749] part of check-in [ba39382ef5] If "PRAGMA page_size" commands are not authorized, the FTS module will assume a page size of 1024. (check-in: [ba39382ef5] user: drh branch: trunk, size: 143667)
2011-06-20
20:50
[1c8a45dfd6] part of check-in [81e6f151d3] Fix a couple of compiler issues on Windows due to warning clean up. (check-in: [81e6f151d3] user: shaneh branch: trunk, size: 143583)
19:00
[f63086fc75] part of check-in [ed2dda9329] More compiler warning fixes. (check-in: [ed2dda9329] user: drh branch: trunk, size: 143583)
17:24
[e9c4b57121] part of check-in [fe62179efd] Fix various harmless compiler warnings in FTS3 and RTREE. (check-in: [fe62179efd] user: drh branch: trunk, size: 143573)
2011-06-17
17:37
[f919a79664] part of check-in [8f939723f7] Avoid loading doclists for infrequent terms that are part of phrases twice. (check-in: [8f939723f7] user: dan branch: trunk, size: 143569)
2011-06-16
16:06
[78b02b5f01] part of check-in [051c756c36] Fix a problem with NEAR queries executed inside a transaction that writes the FTS table. (check-in: [051c756c36] user: dan branch: trunk, size: 142882)
00:54
[014e06f585] part of check-in [a0b43a320e] Changes to #ifdefs so that the build goes correctly if the only FTS macro defined is SQLITE_ENABLE_FTS4. (check-in: [a0b43a320e] user: drh branch: trunk, size: 142856)
2011-06-15
12:43
[6c7b588761] part of check-in [3899f3b95e] Fix a couple of compiler warnings. (check-in: [3899f3b95e] user: drh branch: broken-build, size: 142857)
2011-06-14
09:00
[ae3ae91d20] part of check-in [04907fbade] Fix another bug caused by NEAR/matchinfo/order=DESC interaction. (check-in: [04907fbade] user: dan branch: fts3-prefix-search, size: 142858)
07:22
[5d59c015d6] part of check-in [135ce30f62] Merge recent trunk changes into fts3-prefix-search branch. (check-in: [135ce30f62] user: dan branch: fts3-prefix-search, size: 142910)
07:14
[f4f6527312] part of check-in [06de3f2cbc] Remove unused parameters from internal fts3 function. (check-in: [06de3f2cbc] user: dan branch: fts3-prefix-search, size: 142518)
2011-06-13
17:00
[2a48945d19] part of check-in [b9fb69e55b] Add tests for deferred tokens to fts3auto.test. Fix a problem with OR queries and deferred tokens. (check-in: [b9fb69e55b] user: dan branch: fts3-prefix-search, size: 142008)
09:11
[e71dafb1f3] part of check-in [5f6b87f420] Fix a bug exposed by combining matchinfo(), NEAR and "ORDER BY rowid DESC". (check-in: [5f6b87f420] user: dan branch: fts3-prefix-search, size: 141836)
2011-06-09
10:48
[5df3b57975] part of check-in [4e8dd19eef] Fix problems to do with using both OR and NEAR operators in a single expression. (check-in: [4e8dd19eef] user: dan branch: fts3-prefix-search, size: 141755)
2011-06-08
18:39
[b44083cafb] part of check-in [3972a787df] Fix various issues to do with deferred tokens, NEAR expressions and matchinfo(). (check-in: [3972a787df] user: dan branch: fts3-prefix-search, size: 141145)
2011-06-07
18:35
[9d2d2cab4d] part of check-in [9d10a6846b] Have NEAR queries use incremental merging. Fix issues surrounding the deferred token optimization. (check-in: [9d10a6846b] user: dan branch: fts3-prefix-search, size: 143648)
2011-06-06
18:14
[d0970d9fa0] part of check-in [567dd84359] Merge the latest trunk changes into the fts3-prefix-search branch. (check-in: [567dd84359] user: drh branch: fts3-prefix-search, size: 137056)
14:51
[aafc143b59] part of check-in [89f2f482e0] Modify fts3rnd.test to run tests for both "ORDER BY docid ASC" and "ORDER BY docid DESC" with both order=ASC and order=DESC FTS tables. Fixes for some bugs found. (check-in: [89f2f482e0] user: dan branch: fts3-prefix-search, size: 136933)
06:55
[ea03065c8f] part of check-in [0425138a23] Clean up the code for processing FTS4 options a bit. (check-in: [0425138a23] user: dan branch: fts3-prefix-search, size: 136339)
2011-06-04
20:13
[31f73cae1d] part of check-in [650e1a79ed] Remove some unreachable code. (check-in: [650e1a79ed] user: dan branch: fts3-prefix-search, size: 135716)
20:04
[432c902c69] part of check-in [f6a0193f5a] Allow the "order=DESC" and "order=ASC" parameters in FTS4 "CREATE VIRTUAL TABLE" statements. Tables created with "order=DESC" store all doclists in descending order, which allows optimizations normally applied to "ORDER BY docid ASC" queries to be used with "ORDER BY docid DESC" queries instead. (check-in: [f6a0193f5a] user: dan branch: fts3-prefix-search, size: 142120)
2011-06-03
18:00
[3183cf6aa7] part of check-in [a4c7e28208] FTS changes: Remove unreachable code. Fix bugs. When processing a large doclist incrementally, read from disk incrementally too. (check-in: [a4c7e28208] user: dan branch: fts3-prefix-search, size: 135253)
2011-06-02
19:57
[f92b6e0241] part of check-in [28149a7882] Changes to improve performance and support LIMIT clauses on fts3 tables. This branch is unstable for now. (check-in: [28149a7882] user: dan branch: fts3-prefix-search, size: 156042)
2011-05-28
15:57
[eb59cdd89e] part of check-in [84097a4c75] Minor changes made while planning a larger change. (check-in: [84097a4c75] user: dan branch: fts3-prefix-search, size: 133250)
2011-05-25
23:18
[b3a10a1a32] part of check-in [a4755e7088] Fix some minor and harmless compiler warnings. (check-in: [a4755e7088] user: drh branch: trunk, size: 128741)
19:17
[b44e0abc7a] part of check-in [cc83991caa] If a prefix index of size N is not present, use a prefix index of size N+1 along with the terms index for queries for prefixes of length N. (check-in: [cc83991caa] user: dan branch: fts3-prefix-search, size: 132878)
18:47
[925360f6c0] part of check-in [f0f0a03db2] Merge trunk changes into experimental fts3-prefix-search branch. (check-in: [f0f0a03db2] user: dan branch: fts3-prefix-search, size: 131731)
18:34
[9ebda4cd40] part of check-in [be59bf4940] Change fts4 so that the prefix= parameter is passes a comma-separated list of integers. For each integer N, a separate index of all prefixes of length N bytes is created. (check-in: [be59bf4940] user: dan branch: fts3-prefix-search, size: 131750)
01:16
[0a2ae110f2] part of check-in [92f26a8b8f] Changes to savepoint in virtual tables for simpler and more consistent operation. (check-in: [92f26a8b8f] user: drh branch: trunk, size: 128618)
2011-05-24
18:49
[4a48bfef34] part of check-in [b5bdc63989] If the fts4 option prefix=1 is specified, have the fts4 module maintain an index of prefixes as well as terms. (check-in: [b5bdc63989] user: dan branch: fts3-prefix-search, size: 129763)
15:36
[e7600e285b] part of check-in [651ef24249] Do not invoke the xRollbackTo or xRelease methods of a virtual table without having first invoked an appropriate xSavepoint method. Add assert() statements to FTS3/4 to verify that this is happening in all cases. (check-in: [651ef24249] user: drh branch: trunk, size: 128637)
2011-05-17
15:56
[18c2a079ba] part of check-in [8844e8bfb8] Add extended return code SQLITE_CORRUPT_VTAB. Returned when the tcontents of the sqlite tables used internally by a virtual table module are invalid or inconsistent. (check-in: [8844e8bfb8] user: dan branch: trunk, size: 127722)
2011-05-06
00:19
[0077bd0739] part of check-in [0f4954e979] Increase the iVersion for sqlite3_module needed for the xSavepoint, xRelease, and xRollbackTo methods from 1 to 2. This is so that it will conform to published documentation. (check-in: [0f4954e979] user: drh branch: trunk, size: 127707)
2011-05-04
17:23
[73d6718dba] part of check-in [8f9666af5f] Merge vtab-conflict branch with trunk. (check-in: [8f9666af5f] user: dan branch: trunk, size: 127707)
16:30
[84dc693d12] part of check-in [1a11335970] Fix a couple of compiler warnings in the FTS code. (check-in: [1a11335970] user: dan branch: vtab-conflict, size: 127708)
15:41
[18bf7e4eca] part of check-in [95e09b20e9] Fix a performance problem in queries that use "ORDER BY rowid DESC" and one or more FTS auxiliary functions. (check-in: [95e09b20e9] user: dan branch: vtab-conflict, size: 127323)
12:52
[47e4f4da59] part of check-in [13395121e3] Optimize "ORDER BY rowid/docid DESC/ASC" clauses on FTS tables. (check-in: [13395121e3] user: dan branch: vtab-conflict, size: 126240)
2011-04-26
19:21
[ce37973c86] part of check-in [fb4a355871] Extra tests for fts3. And fixes for conflict-handling related problems in fts3. (check-in: [fb4a355871] user: dan branch: vtab-conflict, size: 124536)
2011-04-25
18:49
[94fa15fc9d] part of check-in [6d2633a6d0] Add support for on conflict clauses to fts3/fts4. (check-in: [6d2633a6d0] user: dan branch: vtab-conflict, size: 123952)
2011-02-03
12:48
[5653c5654a] part of check-in [80225abe79] Extra tests for fts4 compress/uncompress hooks. Fix some minor problems with the same. (check-in: [80225abe79] user: dan branch: trunk, size: 123884)
2011-02-02
17:30
[f93f5d614e] part of check-in [386701ded2] Optimize handling of equality and range constraints on the "term" column of an fts4aux table. (check-in: [386701ded2] user: dan branch: trunk, size: 122882)
2011-02-01
16:34
[871600c065] part of check-in [b010ddcc52] Add virtual table module "fts4aux", used to inspect the full-text index of an fts4 table directly. Also add the "compress" and "uncompress" fts4 options. (check-in: [b010ddcc52] user: dan branch: fts4aux, size: 122683)
2010-12-10
17:06
[28ada7d1c7] part of check-in [fa37d8eb7c] Fix a performance regression in matchinfo(). (check-in: [fa37d8eb7c] user: dan branch: trunk, size: 118524)
2010-12-04
18:20
[44f998657e] part of check-in [507027b70f] Fix a memory leak in fts3 that can occur in an OOM condition. (check-in: [507027b70f] user: dan branch: trunk, size: 118328)
2010-12-02
17:39
[bb233da09f] part of check-in [099195b148] Fix a bug causing fts3 to incorrectly return SQLITE_CORRUPT. (check-in: [099195b148] user: dan branch: trunk, size: 118179)
2010-12-01
15:36
[c354e8a411] part of check-in [c7771c0b22] Fix some warnings under MSVC in fts3 module. (check-in: [c7771c0b22] user: shaneh branch: trunk, size: 118180)
2010-11-26
16:49
[bae65cf771] part of check-in [c412f61229] Fix various compiler warnings. (check-in: [c412f61229] user: drh branch: trunk, size: 118152)
16:31
[7fda91bbe8] part of check-in [3c3d076b42] Fix an uninitialized variable in fts3.c. (check-in: [3c3d076b42] user: dan branch: trunk, size: 118228)
2010-11-24
15:02
[0e207ae800] part of check-in [ed61fd20ad] Fix crashes that can occur when queries are run on an FTS4 table containing zero rows. (check-in: [ed61fd20ad] user: dan branch: fts3-experimental, size: 118228)
11:51
[916091c7ae] part of check-in [ae40b34cf7] Remove some unused code from fts3. Add tests to fts3matchinfo.test. (check-in: [ae40b34cf7] user: dan branch: fts3-experimental, size: 118130)
2010-11-23
19:16
[591b6fdb56] part of check-in [9cf0f2b76b] Experimental changes to fts3 function matchinfo(). (check-in: [9cf0f2b76b] user: dan branch: fts3-experimental, size: 118091)
2010-11-18
13:52
[7e01fd82fa] part of check-in [76681870a4] Restrict the scope of the fts3ExprCost() subroutine inside of FTS3. (check-in: [76681870a4] user: drh branch: trunk, size: 117936)
13:47
[98e6f7c094] part of check-in [6c4f1d5c24] Fix compiler warnings. (check-in: [6c4f1d5c24] user: drh branch: trunk, size: 117929)
2010-11-02
17:41
[9da9f1d5bc] part of check-in [31989b18f5] Changes to allow FTS4 tables to be created without the underlying %_docsize table (in order to save space). (check-in: [31989b18f5] user: dan branch: trunk, size: 118942)
2010-10-30
15:21
[48c37f640f] part of check-in [b770290561] Test cases and minor changes to make fts3 more robust in the face of a corrupt database. (check-in: [b770290561] user: dan branch: trunk, size: 116744)
2010-10-26
07:14
[e78273903c] part of check-in [7a2f286400] More coverage tests for fts3.c. (check-in: [7a2f286400] user: dan branch: experimental, size: 115965)
2010-10-25
19:01
[989ab87071] part of check-in [918b609290] Further improvements to coverage of fts3.c. Fixes for bugs revealed by the same. (check-in: [918b609290] user: dan branch: experimental, size: 115995)
12:47
[8a3b4144b6] part of check-in [a8b1d99899] Test coverage improvements for fts3.c. (check-in: [a8b1d99899] user: dan branch: experimental, size: 115917)
2010-10-23
19:07
[b7442369ab] part of check-in [deb80eac91] Fixes for the matchinfo() function related to FTS4 common token handling. (check-in: [deb80eac91] user: dan branch: experimental, size: 116388)
2010-10-22
16:44
[e2f031ea6b] part of check-in [1c9c70fec3] Add missing comments and fix compiler warnings in new FTS3/4 code. Other minor fixes too. (check-in: [1c9c70fec3] user: dan branch: experimental, size: 115406)
2010-10-21
15:49
[f423181b76] part of check-in [fd1e5cade0] Merge trunk changes into experimental branch. (check-in: [fd1e5cade0] user: dan branch: experimental, size: 109410)
2010-10-20
18:56
[ce7bcd1f42] part of check-in [d0a450ce78] Updates to FTS4 to improve performance and make more accurate cost estimates for prefix terms. (check-in: [d0a450ce78] user: dan branch: experimental, size: 109382)
2010-10-19
14:08
[9d4ccf3b7b] part of check-in [5ae0ba447a] Experimental changes to fts4 to try to selectively avoid loading very large doclists. (check-in: [5ae0ba447a] user: dan branch: experimental, size: 107042)
2010-09-17
15:28
[03be86c59a] part of check-in [018e82c775] Have all FTS3 queries obtain a read or write table-lock at the shared-cache level before doing anything else. (check-in: [018e82c775] user: dan branch: trunk, size: 92319)
2010-08-24
16:59
[f6436b3bcd] part of check-in [d1c875320a] Fixes to allow fts3 tables to be renamed mid-transaction. (check-in: [d1c875320a] user: dan branch: trunk, size: 92250)
2010-08-09
14:26
[e818310c47] part of check-in [88bf901627] Fix two comment typos that were reported on the mailing list. (check-in: [88bf901627] user: drh branch: trunk, size: 92200)
2010-07-19
11:16
[9dec342fa1] part of check-in [d692434b49] Re-introduce the prefix-search optimization of [feef1b15d6], which was lost in a reorganization of FTS3 code. (check-in: [d692434b49] user: dan branch: trunk, size: 92199)
2010-07-02
18:15
[51948505e0] part of check-in [26bc27e3f4] Fix a couple more compiler warnings under MSVC. (check-in: [26bc27e3f4] user: shaneh branch: mistake, size: 90068)
2010-03-23
18:24
[20c55bed37] part of check-in [892e286709] More commenting and documentation enhancements in FTS3. (check-in: [892e286709] user: drh branch: trunk, size: 89989)
15:46
[e3b543c61c] part of check-in [3e4a008217] Fix up obsolete comments in FTS3 to conform to the latest nomenclature. Add new comments to better explain FTS3 operation. (check-in: [3e4a008217] user: drh branch: trunk, size: 90107)
2010-03-18
16:34
[2d2ee8cb1a] part of check-in [39b34ac4a7] Mark internal routines as "static" in FTS3. Ticket [b1552d0fa7]. (check-in: [39b34ac4a7] user: drh branch: trunk, size: 86266)
2010-03-15
20:53
[2bb2045d14] part of check-in [bea9258643] Improved error message when the right-hand operand of MATCH in FTS3 is not a valid search expression. Ticket [170872f1c3]. (check-in: [bea9258643] user: drh branch: trunk, size: 86259)
19:27
[54d3c2273c] part of check-in [f186b6a619] Fix FTS3 so that it does not run illegal SQL and cause a spurious log message. Ticket [42d45a693e6]. (check-in: [f186b6a619] user: drh branch: trunk, size: 86079)
2010-03-09
12:35
[dea6740e64] part of check-in [87a6a9970b] Disable FTS4 (which is incomplete and not ready for publication) to prevent accidental use. (check-in: [87a6a9970b] user: drh branch: trunk, size: 85869)
2010-03-05
17:27
[15ca4d9baf] part of check-in [6e3e014af9] Fix a one byte buffer overread in fts3. (check-in: [6e3e014af9] user: dan branch: trunk, size: 85762)
2010-02-03
19:55
[c7a20cf4c2] part of check-in [1b6e6094c8] Add a new full-text search variant that tracks the total number of documents and document sizes, to make ranking search results easier. Currently called FTS4. (check-in: [1b6e6094c8] user: drh branch: trunk, size: 85460)
2010-01-11
12:00
[699abb6cc1] part of check-in [a2b1183d9e] Modify snippets code to run more efficiently. And to avoid a bug relating to snippets based on full-text queries that contain duplicate terms. (check-in: [a2b1183d9e] user: dan branch: trunk, size: 83349)
2010-01-08
23:01
[451eb6554f] part of check-in [e424a03073] Update comments in fts3.c to more accurately describe the doclist format. (check-in: [e424a03073] user: drh branch: trunk, size: 83342)
2010-01-07
10:54
[7b1969f6b9] part of check-in [3b5ccd2682] Fixes to problems in FTS3 snippet() function found by th3 tests. (check-in: [3b5ccd2682] user: dan branch: trunk, size: 82277)
2010-01-06
17:19
[04e95afa45] part of check-in [8a208223a7] Change the fts3 snippet function to return (hopefully) more relevant snippets in less time. (check-in: [8a208223a7] user: dan branch: trunk, size: 82220)
2010-01-02
19:02
[15fb87c1f0] part of check-in [51f7ee8440] Add experimental implementation of FTS3 functions matchinfo() and snippet() (not enabled by default). (check-in: [51f7ee8440] user: dan branch: trunk, size: 81828)
2009-12-31
19:06
[ac757a4561] part of check-in [6cf76c2ae2] Changes to remove warnings in MSVC build. (check-in: [6cf76c2ae2] user: shaneh branch: trunk, size: 86624)
2009-12-29
10:32
[439bda67fb] part of check-in [a3b50e4f80] Fix a bug triggered by optimizing an FTS3 table when there are no segments on disk but pending terms in the hash table. (check-in: [a3b50e4f80] user: dan branch: trunk, size: 86594)
2009-12-22
18:56
[0a09deb552] part of check-in [37a1de02d1] Add the experimental FTS3 matchinfo() function. Provides details of the match that may be used for result ranking and other purposes. (check-in: [37a1de02d1] user: dan branch: trunk, size: 86236)
2009-12-20
15:00
[9c4ab6e74b] part of check-in [956de051f4] Fix a problem in FTS3 phrase queries. Add tests to check that it really is fixed. (check-in: [956de051f4] user: dan branch: trunk, size: 76803)
2009-12-14
15:17
[682882c15f] part of check-in [82b069998e] Handle an unhandled OOM error code that occurs in fts3 with SQLITE_TEST defined. (check-in: [82b069998e] user: dan branch: trunk, size: 76750)
14:49
[6ae94945dd] part of check-in [a730a05b39] Fix a broken assert() in fts3_write.c. Also fix a couple of compiler warnings in fts3 code. (check-in: [a730a05b39] user: dan branch: trunk, size: 76716)
2009-12-12
19:15
[ea601b21f6] part of check-in [06b72b0073] Tests to cover a few extra branches in fts3.c. (check-in: [06b72b0073] user: dan branch: trunk, size: 76726)
09:51
[e1828210a5] part of check-in [8fcb0478c8] Add coverage test cases for fts3. (check-in: [8fcb0478c8] user: dan branch: trunk, size: 76739)
2009-12-11
16:03
[c059f8e743] part of check-in [87fc0ce151] Change the fts3 test interface used to configure the advisory node size parameter. (check-in: [87fc0ce151] user: dan branch: trunk, size: 76626)
2009-12-10
18:38
[b79e729c85] part of check-in [83a80efe8a] Remove some unused macros and types from fts3.c. (check-in: [83a80efe8a] user: dan branch: trunk, size: 76968)
18:29
[2542d22247] part of check-in [6cbbae8499] Fix handling of "WHERE col MATCH ? AND docid = ?" clauses in fts3. (check-in: [6cbbae8499] user: dan branch: trunk, size: 78588)
18:20
[a52050eb59] part of check-in [61efff4142] Fix an OOM related problem in the snippet() and offsets() functions of fts3. (check-in: [61efff4142] user: dan branch: trunk, size: 78485)
16:04
[f72d7fdb5c] part of check-in [48c0db0eb2] Changes to fts3 to avoid flushing data to disk within a SELECT statement. (check-in: [48c0db0eb2] user: dan branch: trunk, size: 78148)
2009-12-09
14:39
[6e24afa93e] part of check-in [56b6432f86] Improve test coverage of fts3.c. (check-in: [56b6432f86] user: dan branch: trunk, size: 77849)
05:30
[cb9bd52e20] part of check-in [a9038306c3] Mark fts3ReallocOrFree and fts3InitVtab as static. Ticket [ff44d82f3b]. (check-in: [a9038306c3] user: dan branch: trunk, size: 77729)
2009-12-08
19:05
[ece6d5b17b] part of check-in [f0eac4175a] Add tests to improve coverage of fts3. Associated bugfixes. (check-in: [f0eac4175a] user: dan branch: trunk, size: 77722)
2009-12-07
16:18
[684a55d603] part of check-in [fad43d290f] Move some SQLITE_TEST code down to avoid mixing code and variable declarations. Fix a test for high-order bit handling in sqlite3Fts3InitTokenizer(). (check-in: [fad43d290f] user: shaneh branch: trunk, size: 77665)
12:34
[43d8cfd021] part of check-in [5811df3f04] Add some tests for OR, AND and NOT operations to fts3rnd.test. Add tests to check that errors are returned when bad arguments are passed to fts3 functions snippet, offsets and optimize. Minor fix for the same (check-in: [5811df3f04] user: dan branch: trunk, size: 77665)
2009-12-05
14:29
[f4152f2777] part of check-in [3bb13a0652] Fix another bug in 3-way NEAR queries. (check-in: [3bb13a0652] user: dan branch: trunk, size: 77306)
11:37
[7d344410bf] part of check-in [23dc6fb5b2] Fix some problems with FTS3 and 3-way NEAR queries. (check-in: [23dc6fb5b2] user: dan branch: trunk, size: 77103)
2009-12-04
19:07
[54ad3e8f03] part of check-in [507890a913] Fix a problem involving a 3-way NEAR query. (check-in: [507890a913] user: dan branch: trunk, size: 75216)
14:11
[da95f0191c] part of check-in [69c21ee46a] Modify [2ad1461f25] to avoid leaving a prepared statement in "active" state following an OOM error in FTS3. (check-in: [69c21ee46a] user: dan branch: trunk, size: 74883)
13:43
[fb2459bc19] part of check-in [2ad1461f25] Fix an FTS3 problem where an OOM error was not being propagated back out to the top-level interface. (check-in: [2ad1461f25] user: drh branch: trunk, size: 74397)
2009-12-03
17:36
[b15d44a46f] part of check-in [75863c2d55] Fix an incorrect assert() in fts3.c. Add further fts3 tests. (check-in: [75863c2d55] user: dan branch: trunk, size: 74357)
06:26
[8352dc3506] part of check-in [37495b55ff] Updates to FTS3 to correct compiler warnings under MSVC. (check-in: [37495b55ff] user: shaneh branch: trunk, size: 73485)
2009-12-01
13:57
[eca3828c5e] part of check-in [58113932d9] Changes to the TCL interface header to allow it to be compiled independently from the amalgamation. (check-in: [58113932d9] user: drh branch: trunk, size: 73087)
2009-11-28
17:23
[8e966349c9] part of check-in [31eed4f8f9] Change FTS3 to detect when the RHS of the MATCH opertor encounters an OOM during string format conversion and report back an SQLITE_NOMEM error. (check-in: [31eed4f8f9] user: drh branch: trunk, size: 73371)
15:35
[591a849341] part of check-in [a9cba7ea0a] Add a test case for creating an FTS3 table with no module arguments or opening/closing brackets in the CREATE VIRTUAL TABLE statement. (check-in: [a9cba7ea0a] user: dan branch: trunk, size: 73272)
12:40
[524b080a1a] part of check-in [eada284bc1] Fix a bug introduced by the fts3 refactoring (segfault when creating a table with zero module args). Also a fix to handle an OOM error. (check-in: [eada284bc1] user: dan branch: trunk, size: 73165)
2009-11-27
12:14
[c2776bb39a] part of check-in [97d3324160] Remove a C++ism accidentally added to fts3.c. (check-in: [97d3324160] user: dan branch: trunk, size: 72918)
2009-11-21
03:03
[6396a59d3f] part of check-in [b6402b2065] Add some missing comments to fts3 files. No source code changes. (check-in: [b6402b2065] user: dan branch: trunk, size: 72902)
2009-11-20
02:24
[4990e02c64] part of check-in [b456eacbbb] Minor optimizations to fts3 code. (check-in: [b456eacbbb] user: dan branch: trunk, size: 72190)
2009-11-19
18:28
[e7383c7440] part of check-in [7cd178a72a] Fix a performance regression introduced while reworking the fts3 code. (check-in: [7cd178a72a] user: dan branch: trunk, size: 71778)
00:15
[3251d3c5e2] part of check-in [fa0998e19d] Fix problems introduced into fts3 as part of the refactoring. (check-in: [fa0998e19d] user: dan branch: fts3-refactor, size: 71670)
2009-11-18
15:35
[2aa2c3f7b4] part of check-in [2fe579e778] Add some missing comments and fix some other issues in fts3 code. (check-in: [2fe579e778] user: dan branch: fts3-refactor, size: 71657)
2009-11-17
12:52
[a72c19fa62] part of check-in [45c051e786] Improvements to the way fts3 reads the full-text index. (check-in: [45c051e786] user: dan branch: fts3-refactor, size: 72418)
2009-11-16
16:36
[4d701e31cb] part of check-in [f29c8fcade] Add a few extra coverage test cases for fts3. (check-in: [f29c8fcade] user: dan branch: fts3-refactor, size: 72493)
2009-11-14
18:41
[1de6c9d40f] part of check-in [c27d46b33e] Further OOM testing for fts3 code. Add Tcl code implementing an integrity-check for fts3. (check-in: [c27d46b33e] user: dan branch: fts3-refactor, size: 72438)
2009-11-13
17:36
[835061e6c5] part of check-in [30a92f1132] Start reworking fts3 code to match the rest of SQLite (code conventions, malloc-failure handling etc.). (check-in: [30a92f1132] user: dan branch: fts3-refactor, size: 72213)
2009-03-05
04:20
[35bfa67d9c] part of check-in [6404afa0c5] Corrected typos and misspellings. Ticket #3702. (CVS 6336) (check-in: [6404afa0c5] user: shane branch: trunk, size: 226039)
2009-01-14
18:59
[da5420cab3] part of check-in [b0f066630c] Fix typos in comments in FTS3 implementation. (CVS 6178) (check-in: [b0f066630c] user: drh branch: trunk, size: 226037)
2009-01-02
01:10
[4e0d3b1b8a] part of check-in [d44c84c0f7] Do not display matches against the right-hand side of a NOT operator in the output of the FTS snippet() or offsets() functions. (CVS 6097) (check-in: [d44c84c0f7] user: drh branch: trunk, size: 226036)
2008-12-17
15:18
[3aa6aef1ea] part of check-in [7389b9ecb8] Modify fts3 to support a more complex expression syntax that allows parenthesis. The new syntax is not entirely backwards compatible, so is disabled by default. Use -DSQLITE_ENABLE_FTS3_PARENTHESIS to enable it. (CVS 6034) (check-in: [7389b9ecb8] user: danielk1977 branch: trunk, size: 224631)
2008-09-12
18:25
[e67453b6ac] part of check-in [8e9b955311] Fix the NEAR connector in FTS3 so that it can take ranges in excess of 9. The maximum range is now 32767. (CVS 5695) (check-in: [8e9b955311] user: drh branch: trunk, size: 232065)
2008-07-29
01:13
[fa37048c36] part of check-in [a08a5f2b12] Re-used prepared statement from fts3 cursor. Previously, each call to fulltextFilter() finalized any existing prepared statement and prepared a new one. In the case where idxNum has not changed, simply reseting the statement suffices. This provides an order of magnitude speedup in incoming joins against docid. (CVS 5489) (check-in: [a08a5f2b12] user: shess branch: trunk, size: 232214)
2008-07-15
21:32
[14ac2e3788] part of check-in [b22e187bc2] Implement optimize() function. This merges all segments in the fts index into a single segment, including dropping delete cookies. (CVS 5417) (check-in: [b22e187bc2] user: shess branch: trunk, size: 231398)
2008-07-14
20:43
[c4037314d3] part of check-in [8b872e4260] Delete all fts3 index data the table becomes empty. Previously, deleting all rows from an fts3 table would leave a bunch of index data describing the terms of the original data, plus deletions of those terms, perhaps with some amount of it merged together so the deletions knocked out the originals. Even when all rows were deleted that original data would hang out, though eventually it would mostly be overwritten if new data contained the same set of terms. (CVS 5413) (check-in: [8b872e4260] user: shess branch: trunk, size: 220403)
2008-07-03
19:53
[192a65d39c] part of check-in [a48e3d95f7] fts3 functions for testing scripts. These are a first step towards being able to write test script which verify that fts3 is internally building indices in the expected way. Both new functions are only defined if fts3.c is compiled with SQLITE_TEST defined, as when building testfixture. These functions are not intended to be part of the exposed fts3 API.

dump_terms() generates a TEXT result of all the terms in the index (or a specified segment), sorted and joined with spaces.

dump_doclist() generates a TEXT representation of the doclist associated with a given term in the index (or a specified segment). (CVS 5340) (check-in: [a48e3d95f7] user: shess branch: trunk, size: 218633)

2008-04-12
13:06
[9f52ff22ca] part of check-in [062bf5d44d] Remove all instances of sprintf() from the FTS modules. Ticket #3049. (CVS 4996) (check-in: [062bf5d44d] user: drh branch: trunk, size: 204423)
2008-02-01
15:34
[34da63e761] part of check-in [62ede6699d] Minor fixes to FTS3 so that it works better when appended to the end of the amalgamation. (CVS 4769) (check-in: [62ede6699d] user: drh branch: trunk, size: 204398)
2007-12-13
21:54
[7d0bdfa46f] part of check-in [4e91a267fe] Change all instances of "it's" in comments to either "its" or "it is", as appropriate, in case the comments are ever again read by a pedantic grammarian. Ticket #2840. (CVS 4629) (check-in: [4e91a267fe] user: drh branch: trunk, size: 204399)
2007-12-07
23:47
[0992fca534] part of check-in [feef1b15d6] Change prefix search from O(N*M) to O(NlogM). The previous code linearly merged the doclists, so as the accumulated list got large, things got slow (the M term, a fucntion of the number of documents in the index). This change does pairwise merges until a single doclist remains. A test search of 't*' against a database of RFC text improves from 1m16s to 4.75s. (CVS 4599) (check-in: [feef1b15d6] user: shess branch: trunk, size: 204394)
2007-11-24
00:41
[b95b4b6221] part of check-in [0fc61f99b5] The FTS3 amalgamation can now be appended to the SQLite amalgamation to generate a single source file that contains both components. (CVS 4558) (check-in: [0fc61f99b5] user: drh branch: trunk, size: 200877)
2007-11-23
17:31
[2ea2aead0d] part of check-in [460af6bb66] Changes fts3 to use only sqlite3_malloc() and not system malloc. Ticket #2762. (CVS 4554) (check-in: [460af6bb66] user: drh branch: trunk, size: 200527)
2007-11-16
00:23
[2e77742df5] part of check-in [75cb46f82a] Don't do anything when input doclists are both empty. Ticket #2774 (CVS 4546) (check-in: [75cb46f82a] user: shess branch: trunk, size: 200030)
2007-10-24
23:24
[80b5eb9081] part of check-in [deb8f56d3a] Be a bit more susicious of invalid results from the tokenizer. (CVS 4514) (check-in: [deb8f56d3a] user: shess branch: trunk, size: 199999)
2007-10-22
18:02
[577d671469] part of check-in [aef7720e0b] Add the NEAR operator to fts3. (CVS 4502) (check-in: [aef7720e0b] user: danielk1977 branch: trunk, size: 199895)
2007-09-13
18:16
[6b390b5054] part of check-in [fec6567a0f] Drop the forced error from fts3.c and add forced errors to fts2.c and fts1.c. (CVS 4427) (check-in: [fec6567a0f] user: shess branch: trunk, size: 189346)
18:14
[6d266d7250] part of check-in [c8d2345200] Add an implicit (HIDDEN) docid column. This works as an alias to rowid, similar to how things work in SQLite tables with INTEGER PRIMARY KEY. Add tests to verify operation. (CVS 4426) (check-in: [c8d2345200] user: shess branch: trunk, size: 189520)
18:12
[fa62f39818] part of check-in [ca669eaf1b] Mark the table-named column HIDDEN. Add tests to make sure it's working as expected. (CVS 4425) (check-in: [ca669eaf1b] user: shess branch: trunk, size: 188104)
2007-08-28
20:36
[f85d5d8fdf] part of check-in [6c617bd89f] Fix memory leak of InteriorReader.term. Comes up when doing queries against large segments. (CVS 4315) (check-in: [6c617bd89f] user: shess branch: trunk, size: 188097)
2007-08-23
20:28
[c77071d54e] part of check-in [76f1e18ebc] Make comments and variable naming more consistent WRT rowid versus docid/blockid. This should have no code impact. (CVS 4281) (check-in: [76f1e18ebc] user: shess branch: trunk, size: 188060)
20:23
[2098c9b085] part of check-in [6eb2d74a8c] Fix fts3 to not have the VACUUM bug from fts2. %_content.docid is an alias to fix the rowid for documents, %_segments.blockid is an alias to fix the rowid for segment blocks. Unit test for the problem. (CVS 4280) (check-in: [6eb2d74a8c] user: shess branch: trunk, size: 187769)
2007-08-20
17:37
Added: [64ebaf649e] part of check-in [216c91d2fc] Copy fts2 to fts3, renaming, and replacing references to fts2 with fts3, including capitalization variants. (CVS 4249) (check-in: [216c91d2fc] user: shess branch: trunk, size: 187003)