Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
| Comment: | Make hi-bit characters delimiters. This is a stopgap until the tokenizer and fulltext.c recognize UTF-8 correctly. (CVS 3370) |
|---|---|
| Downloads: | Tarball | ZIP archive | SQL archive |
| Timelines: | family | ancestors | descendants | both | trunk |
| Files: | files | file ages | folders |
| SHA1: | ca850d3d80f67672172d11392fcdf60b |
| User & Date: | shess 2006-08-28 20:08:57 |
Context
|
2006-08-28
| ||
| 23:46 | Make static some symbols which shouldn't have been exported. (CVS 3371) check-in: 58006e38 user: shess tags: trunk | |
| 20:08 | Make hi-bit characters delimiters. This is a stopgap until the tokenizer and fulltext.c recognize UTF-8 correctly. (CVS 3370) check-in: ca850d3d user: shess tags: trunk | |
|
2006-08-27
| ||
| 14:10 | Add sqlite3_malloc and sqlite3_realloc to the sqlite3.def file. Ticket #1943. (CVS 3369) check-in: 4a74838e user: drh tags: trunk | |
Changes
Changes to ext/fts1/simple_tokenizer.c.
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
** else we need to reindex. One solution would be a meta-table to
** track such information in the database, then we'd only want this
** information on the initial create.
*/
if( argc>1 ){
t->zDelim = string_dup(argv[1]);
} else {
/* Build a string of non-alphanumeric ASCII characters */
char zDelim[128]; /* nul-terminated, so nul not a member */
int i, j;
for(i=1, j=0; i<0x80; i++){
if( !isalnum(i) ){
zDelim[j++] = i;
}
}
zDelim[j++] = '\0';
assert( j<=sizeof(zDelim) );
t->zDelim = string_dup(zDelim);
}
|
| | | | |
57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 |
** else we need to reindex. One solution would be a meta-table to
** track such information in the database, then we'd only want this
** information on the initial create.
*/
if( argc>1 ){
t->zDelim = string_dup(argv[1]);
} else {
/* Build a string excluding alphanumeric ASCII characters */
char zDelim[256]; /* nul-terminated, so nul not a member */
int i, j;
for(i=1, j=0; i<0x100; i++){
if( i>=0x80 || !isalnum(i) ){
zDelim[j++] = i;
}
}
zDelim[j++] = '\0';
assert( j<=sizeof(zDelim) );
t->zDelim = string_dup(zDelim);
}
|