Documentation Source Text

Check-in [0585b3df1c]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update the FTS5 extension documentation to show the new fts5_api access technique. Update the change log to talk about the backwards incompatible changes in extensions flowing out of sqlite3_bind_pointer(). Update the search extension to use the new FTS5 extensions technique.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 0585b3df1c81112e92acaf600f802a62df2be77795754af3680f327e087c1538
User & Date: drh 2017-07-17 16:00:37.043
Context
2017-07-20
00:15
Update the size and performance graph. (check-in: d35ef36eeb user: drh tags: trunk)
2017-07-17
16:00
Update the FTS5 extension documentation to show the new fts5_api access technique. Update the change log to talk about the backwards incompatible changes in extensions flowing out of sqlite3_bind_pointer(). Update the search extension to use the new FTS5 extensions technique. (check-in: 0585b3df1c user: drh tags: trunk)
12:52
Add the integrity_check false-positive bug-fix to the change log. (check-in: 9b65b351e5 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/changes.in.
21
22
23
24
25
26
27
28







29
30
31
32
33
34

35
36
37
38
39
40
41
  incr nChng
}

chng {2017-07-20 (3.20.0)} {
<li> Update the text of error messages returned by [sqlite3_errmsg()] for some
     error codes.
<li> Add new interfaces [sqlite3_bind_pointer()], [sqlite3_result_pointer()], and
     [sqlite3_value_pointer()].  Update the [carray(PTR,N)] and 







     [https://www.sqlite.org/src/file/ext/misc/remember.c | remember(V,PTR)]
     extensions to require the use of [sqlite3_bind_pointer()] to set their
     pointer values.
<li> Added the [SQLITE_STMT virtual table] extension.
<li> Added the [COMPLETION extension] - designed to suggest
     tab-completions for interactive user interfaces.

<li> Added the [sqlite3_prepare_v3()] and [sqlite3_prepare16_v3()] interfaces
     with the extra "prepFlags" parameters.
<li> Provide the [SQLITE_PREPARE_PERSISTENT] flag [sqlite3_prepare_v3()] and
     use it to limit [lookaside memory] misuse by [FTS3], [FTS5], and the
     [R-Tree extension].
<li> Added the [PRAGMA secure_delete=FAST] command.  When secure_delete is
     set to FAST, old content is overwritten with zeros as long as that does







|
>
>
>
>
>
>
>
|
|
|


|
>







21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
  incr nChng
}

chng {2017-07-20 (3.20.0)} {
<li> Update the text of error messages returned by [sqlite3_errmsg()] for some
     error codes.
<li> Add new interfaces [sqlite3_bind_pointer()], [sqlite3_result_pointer()], and
     [sqlite3_value_pointer()].
<li> Backwards-incompatible changes to some extensions in order to take 
     advantage of the improved security offered by the new [sqlite3_bind_pointer()]
     interface:
     <ul>
     <li> [Extending FTS5] &rarr; requires [sqlite3_bind_pointer()] to find
          the fts5_api pointer.
     <li> [carray(PTR,N)] &rarr; requires [sqlite3_bind_pointer()] to set the PTR parameter.
     <li> [https://www.sqlite.org/src/file/ext/misc/remember.c|remember(V,PTR)]
          &rarr; requires [sqlite3_bind_pointer()] to set the PTR parameter.
     </ul>
<li> Added the [SQLITE_STMT virtual table] extension.
<li> Added the [COMPLETION extension] - designed to suggest
     tab-completions for interactive user interfaces.  This is a work in progress.
     Expect further enhancements in future releases.
<li> Added the [sqlite3_prepare_v3()] and [sqlite3_prepare16_v3()] interfaces
     with the extra "prepFlags" parameters.
<li> Provide the [SQLITE_PREPARE_PERSISTENT] flag [sqlite3_prepare_v3()] and
     use it to limit [lookaside memory] misuse by [FTS3], [FTS5], and the
     [R-Tree extension].
<li> Added the [PRAGMA secure_delete=FAST] command.  When secure_delete is
     set to FAST, old content is overwritten with zeros as long as that does
Changes to pages/fts5.in.
1397
1398
1399
1400
1401
1402
1403
1404
1405

1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418

1419
1420
1421
1422
1423
1424
1425
1426
1427





1428
1429
1430
1431
1432
1433
1434
document are all implemented using the publicly available API described
below.

<p> Before a new auxiliary function or tokenizer implementation may be 
registered with FTS5, an application must obtain a pointer to the "fts5_api"
structure. There is one fts5_api structure for each database connection with
which the FTS5 extension is registered. To obtain the pointer, the application
invokes the SQL user-defined function fts5(), which returns a blob value
containing the pointer to the fts5_api structure for the connection. The

following example code demonstrates the technique:

<codeblock>
  <i>/*
  ** Return a pointer to the fts5_api pointer for database connection db.
  ** If an error occurs, return NULL and leave an error in the database 
  ** handle (accessible using sqlite3_errcode()/errmsg()).
  */</i>
  fts5_api *fts5_api_from_db(sqlite3 *db){
    fts5_api *pRet = 0;
    sqlite3_stmt *pStmt = 0;

    if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5()", -1, &pStmt, 0)

     && SQLITE_ROW==sqlite3_step(pStmt) 
     && sizeof(pRet)==sqlite3_column_bytes(pStmt, 0)
    ){
      memcpy(&pRet, sqlite3_column_blob(pStmt, 0), sizeof(pRet));
    }
    sqlite3_finalize(pStmt);
    return pRet;
  }
</codeblock>






<p> The fts5_api structure is defined as follows. It exposes three methods, 
one each for registering new auxiliary functions and tokenizers, and one for
retrieving existing tokenizer. The latter is intended to facilitate the
implementation of "tokenizer wrappers" similar to the built-in
porter tokenizer.








|
|
>
|











|
>
|
<
<
<





>
>
>
>
>







1397
1398
1399
1400
1401
1402
1403
1404
1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421



1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
document are all implemented using the publicly available API described
below.

<p> Before a new auxiliary function or tokenizer implementation may be 
registered with FTS5, an application must obtain a pointer to the "fts5_api"
structure. There is one fts5_api structure for each database connection with
which the FTS5 extension is registered. To obtain the pointer, the application
invokes the SQL user-defined function fts5() with a single argument.  That
argument must be set to a pointer to a pointer to an fts5_api object
using the [sqlite3_bind_pointer()] interface.
The following example code demonstrates the technique:

<codeblock>
  <i>/*
  ** Return a pointer to the fts5_api pointer for database connection db.
  ** If an error occurs, return NULL and leave an error in the database 
  ** handle (accessible using sqlite3_errcode()/errmsg()).
  */</i>
  fts5_api *fts5_api_from_db(sqlite3 *db){
    fts5_api *pRet = 0;
    sqlite3_stmt *pStmt = 0;

    if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5(?1)", -1, &pStmt, 0) ){
      sqlite3_bind_pointer(pStmt, (void*)&pRet, "fts5_api_ptr");
      sqlite3_step(pStmt);



    }
    sqlite3_finalize(pStmt);
    return pRet;
  }
</codeblock>

<p><b>Backwards Compatibility Warning:</b>
Prior to SQLite version 3.20.0 ([dateof:3.20.0]), the fts5() worked slightly
differently.  Older applications that extend FTS5 must be revised to use 
the new technique shown above.

<p> The fts5_api structure is defined as follows. It exposes three methods, 
one each for registering new auxiliary functions and tokenizers, and one for
retrieving existing tokenizer. The latter is intended to facilitate the
implementation of "tokenizer wrappers" similar to the built-in
porter tokenizer.

Changes to search/fts5ext.c.
30
31
32
33
34
35
36
37

38
39
40
41
42
43
44
45
46
47
48
** Return a pointer to the fts5_api pointer for database connection db.
** If an error occurs, return NULL and leave an error in the database 
** handle (accessible using sqlite3_errcode()/errmsg()).
*/
fts5_api *fts5_api_from_db(sqlite3 *db){
  fts5_api *pRet = 0;
  sqlite3_stmt *pStmt = 0;
  if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5()", -1, &pStmt, 0)

   && SQLITE_ROW==sqlite3_step(pStmt) 
   && sizeof(pRet)==sqlite3_column_bytes(pStmt, 0)
  ){
    memcpy(&pRet, sqlite3_column_blob(pStmt, 0), sizeof(pRet));
  }
  sqlite3_finalize(pStmt);
  return pRet;
}
/************************************************************************/

/*







|
>
|
<
<
<







30
31
32
33
34
35
36
37
38
39



40
41
42
43
44
45
46
** Return a pointer to the fts5_api pointer for database connection db.
** If an error occurs, return NULL and leave an error in the database 
** handle (accessible using sqlite3_errcode()/errmsg()).
*/
fts5_api *fts5_api_from_db(sqlite3 *db){
  fts5_api *pRet = 0;
  sqlite3_stmt *pStmt = 0;
  if( SQLITE_OK==sqlite3_prepare(db, "SELECT fts5(?1)", -1, &pStmt, 0) ){
    sqlite3_bind_pointer(pStmt, 1, (void*)&pRet, "fts5_api_ptr");
    sqlite3_step(pStmt);



  }
  sqlite3_finalize(pStmt);
  return pRet;
}
/************************************************************************/

/*