Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Fix errors in comments in fts5.h. |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | trunk |
| Files: | files | file ages | folders |
| SHA3-256: |
4002790d9418289f3e090c694f651a89 |
| User & Date: | dan 2018-12-15 07:12:34 |
Context
|
2018-12-17
| ||
| 22:19 | Move variable declaration to address compilation issue (C89). (check-in: d64f248d user: mistachkin tags: trunk) | |
|
2018-12-15
| ||
| 07:12 | Fix errors in comments in fts5.h. (check-in: 4002790d user: dan tags: trunk) | |
|
2018-12-14
| ||
| 18:11 | Fix harmless compiler warnings. (check-in: 27221c69 user: drh tags: trunk) | |
Changes
Changes to ext/fts5/fts5.h.
| ︙ | ︙ | |||
410 411 412 413 414 415 416 | ** same token for inputs "first" and "1st". Say that token is in ** fact "first", so that when the user inserts the document "I won ** 1st place" entries are added to the index for tokens "i", "won", ** "first" and "place". If the user then queries for '1st + place', ** the tokenizer substitutes "first" for "1st" and the query works ** as expected. ** | | | | | | | 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 | ** same token for inputs "first" and "1st". Say that token is in ** fact "first", so that when the user inserts the document "I won ** 1st place" entries are added to the index for tokens "i", "won", ** "first" and "place". If the user then queries for '1st + place', ** the tokenizer substitutes "first" for "1st" and the query works ** as expected. ** ** <li> By querying the index for all synonyms of each query term ** separately. In this case, when tokenizing query text, the ** tokenizer may provide multiple synonyms for a single term ** within the document. FTS5 then queries the index for each ** synonym individually. For example, faced with the query: ** ** <codeblock> ** ... MATCH 'first place'</codeblock> ** ** the tokenizer offers both "1st" and "first" as synonyms for the ** first token in the MATCH query and FTS5 effectively runs a query ** similar to: |
| ︙ | ︙ | |||
438 439 440 441 442 443 444 | ** Using this method, when tokenizing document text, the tokenizer ** provides multiple synonyms for each token. So that when a ** document such as "I won first place" is tokenized, entries are ** added to the FTS index for "i", "won", "first", "1st" and ** "place". ** ** This way, even if the tokenizer does not provide synonyms | | | 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 | ** Using this method, when tokenizing document text, the tokenizer ** provides multiple synonyms for each token. So that when a ** document such as "I won first place" is tokenized, entries are ** added to the FTS index for "i", "won", "first", "1st" and ** "place". ** ** This way, even if the tokenizer does not provide synonyms ** when tokenizing query text (it should not - to do so would be ** inefficient), it doesn't matter if the user queries for ** 'first + place' or '1st + place', as there are entries in the ** FTS index corresponding to both forms of the first token. ** </ol> ** ** Whether it is parsing document or query text, any call to xToken that ** specifies a <i>tflags</i> argument with the FTS5_TOKEN_COLOCATED bit |
| ︙ | ︙ |