Index: ext/fts3/README.tokenizers ================================================================== --- ext/fts3/README.tokenizers +++ ext/fts3/README.tokenizers @@ -50,12 +50,14 @@ encoded as a blob. Or, if no such tokenizer exists, an SQL exception (error) is raised. SECURITY: If the fts3 extension is used in an environment where potentially malicious users may execute arbitrary SQL (i.e. gears), they should be - prevented from invoking the fts3_tokenizer() function, possibly using the - authorisation callback. + prevented from invoking the fts3_tokenizer() function. The + fts3_tokenizer() function is disabled by default. It is only enabled + by SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER. Do not enable it in + security sensitive environments. See "Sample code" below for an example of calling the fts3_tokenizer() function from C code. 3. ICU Library Tokenizers Index: ext/fts3/fts3_tokenizer.c ================================================================== --- ext/fts3/fts3_tokenizer.c +++ ext/fts3/fts3_tokenizer.c @@ -104,11 +104,13 @@ sqlite3_result_error(context, zErr, -1); sqlite3_free(zErr); return; } } - sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT); + if( fts3TokenizerEnabled(context) ){ + sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT); + } } int sqlite3Fts3IsIdChar(char c){ static const char isFtsIdChar[] = { 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, /* 0x */ Index: src/select.c ================================================================== --- src/select.c +++ src/select.c @@ -5304,11 +5304,11 @@ ** Update the accumulator memory cells for an aggregate based on ** the current cursor position. ** ** If regAcc is non-zero and there are no min() or max() aggregates ** in pAggInfo, then only populate the pAggInfo->nAccumulator accumulator -** registers i register regAcc contains 0. The caller will take care +** registers if register regAcc contains 0. The caller will take care ** of setting and clearing regAcc. */ static void updateAccumulator(Parse *pParse, int regAcc, AggInfo *pAggInfo){ Vdbe *v = pParse->pVdbe; int i; Index: src/sqlite.h.in ================================================================== --- src/sqlite.h.in +++ src/sqlite.h.in @@ -2085,12 +2085,12 @@ ** following this call. The second parameter may be a NULL pointer, in ** which case the trigger setting is not reported back. ** ** [[SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER]] **
SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER
-**
^This option is used to enable or disable the two-argument -** version of the [fts3_tokenizer()] function which is part of the +**
^This option is used to enable or disable the +** [fts3_tokenizer()] function which is part of the ** [FTS3] full-text search engine extension. ** There should be two additional arguments. ** The first argument is an integer which is 0 to disable fts3_tokenizer() or ** positive to enable fts3_tokenizer() or negative to leave the setting ** unchanged. Index: test/fts3atoken.test ================================================================== --- test/fts3atoken.test +++ test/fts3atoken.test @@ -105,10 +105,11 @@ #-------------------------------------------------------------------------- # Test cases fts3atoken-3.* test the three built-in tokenizers with a # simple input string via the built-in test function. This is as much # to test the test function as the tokenizer implementations. # +sqlite3_db_config db SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1 do_test fts3atoken-3.1 { execsql { SELECT fts3_tokenizer_test('simple', 'I don''t see how'); } } {{0 i I 1 don don 2 t t 3 see see 4 how how}}