/ Check-in [f270245a]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge recent changes from trunk.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | apple-osx
Files: files | file ages | folders
SHA3-256:f270245a7b7b7d21a5f8d2a07eef0e5a56d15fb0c69024c666b047c3f3eb0eb9
User & Date: drh 2019-03-01 21:37:52
Context
2019-03-19
20:48
Merge recent enhancements from trunk. check-in: 000f4398 user: drh tags: apple-osx
2019-03-01
21:37
Merge recent changes from trunk. check-in: f270245a user: drh tags: apple-osx
21:33
The fts3_tokenizer() function returns NULL if the SQLITE_DBCONFIG_ENABLE_FTS_TOKENIZER setting is disabled, which is is by default. check-in: f5732f4c user: drh tags: trunk
18:27
Merge the latest enhancements from trunk. check-in: e64ded7b user: drh tags: apple-osx
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to ext/fts3/README.tokenizers.

    48     48     returned. If only one argument is passed, a pointer to the tokenizer
    49     49     implementation currently registered as <tokenizer-name> is returned,
    50     50     encoded as a blob. Or, if no such tokenizer exists, an SQL exception
    51     51     (error) is raised.
    52     52   
    53     53     SECURITY: If the fts3 extension is used in an environment where potentially
    54     54       malicious users may execute arbitrary SQL (i.e. gears), they should be
    55         -    prevented from invoking the fts3_tokenizer() function, possibly using the
    56         -    authorisation callback.
           55  +    prevented from invoking the fts3_tokenizer() function.  The
           56  +    fts3_tokenizer() function is disabled by default. It is only enabled
           57  +    by SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER. Do not enable it in
           58  +    security sensitive environments.
    57     59   
    58     60     See "Sample code" below for an example of calling the fts3_tokenizer()
    59     61     function from C code.
    60     62   
    61     63   3. ICU Library Tokenizers
    62     64   
    63     65     If this extension is compiled with the SQLITE_ENABLE_ICU pre-processor 

Changes to ext/fts3/fts3_tokenizer.c.

   102    102       if( !pPtr ){
   103    103         char *zErr = sqlite3_mprintf("unknown tokenizer: %s", zName);
   104    104         sqlite3_result_error(context, zErr, -1);
   105    105         sqlite3_free(zErr);
   106    106         return;
   107    107       }
   108    108     }
   109         -  sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
          109  +  if( fts3TokenizerEnabled(context) ){
          110  +    sqlite3_result_blob(context, (void *)&pPtr, sizeof(pPtr), SQLITE_TRANSIENT);
          111  +  }
   110    112   }
   111    113   
   112    114   int sqlite3Fts3IsIdChar(char c){
   113    115     static const char isFtsIdChar[] = {
   114    116         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 0x */
   115    117         0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 1x */
   116    118         0, 0, 0, 0, 1, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0,  /* 2x */

Changes to src/select.c.

  5302   5302   
  5303   5303   /*
  5304   5304   ** Update the accumulator memory cells for an aggregate based on
  5305   5305   ** the current cursor position.
  5306   5306   **
  5307   5307   ** If regAcc is non-zero and there are no min() or max() aggregates
  5308   5308   ** in pAggInfo, then only populate the pAggInfo->nAccumulator accumulator
  5309         -** registers i register regAcc contains 0. The caller will take care
         5309  +** registers if register regAcc contains 0. The caller will take care
  5310   5310   ** of setting and clearing regAcc.
  5311   5311   */
  5312   5312   static void updateAccumulator(Parse *pParse, int regAcc, AggInfo *pAggInfo){
  5313   5313     Vdbe *v = pParse->pVdbe;
  5314   5314     int i;
  5315   5315     int regHit = 0;
  5316   5316     int addrHitTest = 0;

Changes to src/sqlite.h.in.

  2083   2083   ** The second parameter is a pointer to an integer into which
  2084   2084   ** is written 0 or 1 to indicate whether triggers are disabled or enabled
  2085   2085   ** following this call.  The second parameter may be a NULL pointer, in
  2086   2086   ** which case the trigger setting is not reported back. </dd>
  2087   2087   **
  2088   2088   ** [[SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER]]
  2089   2089   ** <dt>SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER</dt>
  2090         -** <dd> ^This option is used to enable or disable the two-argument
  2091         -** version of the [fts3_tokenizer()] function which is part of the
         2090  +** <dd> ^This option is used to enable or disable the
         2091  +** [fts3_tokenizer()] function which is part of the
  2092   2092   ** [FTS3] full-text search engine extension.
  2093   2093   ** There should be two additional arguments.
  2094   2094   ** The first argument is an integer which is 0 to disable fts3_tokenizer() or
  2095   2095   ** positive to enable fts3_tokenizer() or negative to leave the setting
  2096   2096   ** unchanged.
  2097   2097   ** The second parameter is a pointer to an integer into which
  2098   2098   ** is written 0 or 1 to indicate whether fts3_tokenizer is disabled or enabled

Changes to test/fts3atoken.test.

   103    103   } {1 {unknown tokenizer: nosuchtokenizer}}
   104    104   
   105    105   #--------------------------------------------------------------------------
   106    106   # Test cases fts3atoken-3.* test the three built-in tokenizers with a
   107    107   # simple input string via the built-in test function. This is as much
   108    108   # to test the test function as the tokenizer implementations.
   109    109   #
          110  +sqlite3_db_config db SQLITE_DBCONFIG_ENABLE_FTS3_TOKENIZER 1
   110    111   do_test fts3atoken-3.1 {
   111    112     execsql {
   112    113       SELECT fts3_tokenizer_test('simple', 'I don''t see how');
   113    114     }
   114    115   } {{0 i I 1 don don 2 t t 3 see see 4 how how}}
   115    116   do_test fts3atoken-3.2 {
   116    117     execsql {