/ File History
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

History of ext/fts3/fts3_unicode.c

2019-01-08
20:02
Use 64-bit math to compute the sizes of memory allocations in extensions. file: [4b9af615] check-in: [ca67f2ec] user: drh branch: trunk, size: 12680
2018-12-03
16:14
Add the "remove_diacritics=2" option to the unicode61 tokenizer in both FTS5 and FTS3/4. file: [b1902e9a] check-in: [06177f3f] user: dan branch: trunk, size: 12677
2017-03-20
18:53
Fix some problems in fts3 found by address-sanitizer. file: [525a3bd9] check-in: [16a8e84f] user: dan branch: trunk, size: 12572
2014-08-11
14:21
Fix harmless compiler warnings. file: [a93f5edc] check-in: [52b03f04] user: drh branch: trunk, size: 12499
2014-07-19
15:44
Fix some harmess compiler warnings in the FTS3 Unicode module. file: [e80eef8a] check-in: [c01caea5] user: mistachkin branch: toolWarnings, size: 12495
2014-07-03
12:18
Change fts3/4 so that the "unicode61" is included in builds by default. It may now be excluded by defining SQLITE_DISABLE_FTS3_UNICODE. file: [0f598fa4] check-in: [0cc0230a] user: dan branch: trunk, size: 12468
2013-03-21
21:20
Many spelling fixes in comments. No changes to code. file: [92391b4b] check-in: [6f6e2d50] user: mistachkin branch: trunk, size: 12469
2012-06-07
15:53
Add the "tokenchars=" and "separators=" options, for customizing the set of characters considered to be token separators, to the unicode61 tokenizer. file: [49e36e6b] check-in: [e56fb462] user: dan branch: trunk, size: 12470
2012-06-06
19:51
Disable FTS unicode61 by default. It is enabled by specifying compile time option SQLITE_ENABLE_FTS4_UNICODE61. file: [b9660ab4] check-in: [eccd6b65] user: dan branch: trunk, size: 8723
19:30
Have the FTS unicode61 strip out diacritics when tokenizing text. This can be disabled by specifying the tokenizer option "remove_diacritics=0". file: [a3c1b078] check-in: [790f76a5] user: dan branch: trunk, size: 8722
2012-05-26
18:28
If SQLITE_DISABLE_FTS3_UNICODE is defined, do not build the "unicode61" tokenizer. file: [76b6f6fe] check-in: [e71495a8] user: dan branch: fts4-unicode, size: 8112
2012-05-25
19:50
Add special fast paths to sqlite3FtsUnicodeTolower() and Isalnum() for codepoints in the ASCII range. file: [f487c6ef] check-in: [cf7b25d4] user: dan branch: fts4-unicode, size: 8027
17:50
Add an experimental tokenizer to fts4 - "unicode". This tokenizer works in the same way except that it understands unicode "simple case folding" and recognizes all characters not classified as "Letters" or "Numbers" by unicode as token separators. file: [033ee5d1] check-in: [0c13570e] user: dan branch: fts4-unicode, size: 8001 Added