/ All files named "ext/fts1/fts1_tokenizer1.c"

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

History for ext/fts1/fts1_tokenizer1.c

[fd00d1fe] part of check-in [3f9a6661] Fix ticket #2439: the FTS1 and FTS2 extensions use the non-standard, unportable and highly deprecated <malloc.h> header on all platforms except Apple Mac OS X. The <malloc.h> actually is never required on any OS with an at least partly POSIX-conforming API as the malloc(3) & friends functions officially live in <stdlib.h> since over 10 years. Under some platform like FreeBSD the inclusion of <malloc.h> since a few years even causes an "#error" and this way a build failure. So, just get rid of the bad <malloc.h> usage in FTS1 and FTS2 extensions at all and stick with <stdlib.h> there only. (CVS 4191) (check-in: [3f9a6661] user: rse branch: trunk, size: 6176)
[df09e638] part of check-in [fbc53da8] Out-of-memory cleanup in tokenizers. Handle NULL return from malloc/calloc/realloc appropriately, and use sizeof(var) instead of sizeof(type) to make certain that we don't get a mismatch between them as the code rots. (CVS 3693) (check-in: [fbc53da8] user: shess branch: trunk, size: 6233)
[98c2bb9f] part of check-in [936b06aa] Add a Porter stemmer option to the FTS1 module. (CVS 3452) (check-in: [936b06aa] user: drh branch: trunk, size: 6141)
[3c969058] part of check-in [165645d3] Avoid segfaults when inserted NULL values into FTS1. (CVS 3447) (check-in: [165645d3] user: drh branch: trunk, size: 5018)
[79877389] part of check-in [366a70b0] Allow virtual tables to contain multiple full-text-indexed columns. Added a magic column "_all" which can be used for querying all columns in a table at once.

For now, each posting list stores position/offset information for multiple columns. We may implement separate posting lists for separate columns at some future point. (CVS 3408) (check-in: [366a70b0] user: adamd branch: trunk, size: 4938)

[1155942b] part of check-in [098cbafc] Convert static variables into constants in the FTS module. (CVS 3385) (check-in: [098cbafc] user: drh branch: trunk, size: 4931)
[03a9af15] part of check-in [2631ceae] Make tokenizer not rely on nul-terminated text. Instead of using strcspn() and a nul-terminated delimiter list, I just flagged delimiters in an array and wrote things inline. Submitting this for review separately because it's pretty standalone. (CVS 3378) (check-in: [2631ceae] user: shess branch: trunk, size: 4912)
Added: [13789776] part of check-in [e1891f0d] Refactor the FTS1 module so that its name is "fts1" instead of "fulltext", so that all symbols with external linkage begin with "sqlite3Fts1", and so that all filenames begin with "fts1". (CVS 3377) (check-in: [e1891f0d] user: drh branch: trunk, size: 5255)