Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Add examples of creating fts3tokenize tables that use tokenizers declared with multiple arguments to fts3.in. |
|---|---|
| Timelines: | family | ancestors | descendants | both | trunk |
| Files: | files | file ages | folders |
| SHA1: |
23c364befb1473853d18ba36dc253622 |
| User & Date: | dan 2013-09-18 11:31:13 |
Context
|
2013-09-19
| ||
| 13:22 | Updates to the TH3 description. check-in: ecd17d4f41 user: drh tags: trunk | |
|
2013-09-18
| ||
| 11:31 | Add examples of creating fts3tokenize tables that use tokenizers declared with multiple arguments to fts3.in. check-in: 23c364befb user: dan tags: trunk | |
|
2013-09-16
| ||
| 13:32 | Update the date and time function documentation to guarantee that 'now' is stable within a single sqlite3_step() call. check-in: a2842a55e9 user: drh tags: trunk | |
Changes
Changes to pages/fts3.in.
2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 |
of the fts3tokenize virtual table:
<codeblock>
CREATE VIRTUAL TABLE tok1 USING fts3tokenize('porter');
</codeblock>
<p>The name of the desired tokenizer should be substituted in place of
'porter' in the example, of course. Once the virtual table is created,
it can be queried as follows:
<codeblock>
SELECT token, start, end, position
FROM tok1
WHERE input='This is a test sentence.';
</codeblock>
|
| > > > > > > > > > > > > > | |
2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 |
of the fts3tokenize virtual table:
<codeblock>
CREATE VIRTUAL TABLE tok1 USING fts3tokenize('porter');
</codeblock>
<p>The name of the desired tokenizer should be substituted in place of
'porter' in the example, of course. If the tokenizer requires one or
more arguments, they should be separated by commas in the fts3tokenize
declaration (even though they are separated by spaces in declarations
of regular fts4 tables). The following creates fts4 and fts3tokenize
tables that use the same tokenizer:
<codeblock>
CREATE VIRTUAL TABLE text1 USING fts4(tokenize=icu en_AU);
CREATE VIRTUAL TABLE tokens1 USING fts3tokenize(icu, en_AU);
CREATE VIRTUAL TABLE text2 USING fts4(tokenize=unicode61 "tokenchars=@." "separators=123");
CREATE VIRTUAL TABLE tokens2 USING fts3tokenize(unicode61, "tokenchars=@.", "separators=123");
</codeblock>
<p>
Once the virtual table is created, it can be queried as follows:
<codeblock>
SELECT token, start, end, position
FROM tok1
WHERE input='This is a test sentence.';
</codeblock>
|