Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Update the SQLite sources to the version 3.7.7 release candidate. |
---|---|
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
a0ab881592bffafef4c49eecc89ec009 |
User & Date: | drh 2011-06-23 18:46:11.437 |
Context
2011-08-26
| ||
13:29 | Update the built-in SQLite to the latest trunk build. check-in: c70e29bd81 user: drh tags: trunk | |
2011-06-23
| ||
18:46 | Update the SQLite sources to the version 3.7.7 release candidate. check-in: a0ab881592 user: drh tags: trunk | |
17:33 | Fixes to evidence marks. check-in: b1158b0aff user: drh tags: trunk | |
Changes
Changes to src/sqlite3.c.
︙ | ︙ | |||
648 649 650 651 652 653 654 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.7.7" #define SQLITE_VERSION_NUMBER 3007007 | | | 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.7.7" #define SQLITE_VERSION_NUMBER 3007007 #define SQLITE_SOURCE_ID "2011-06-23 17:29:33 b61a76a53af04f731fe7617f7b6b4fb2aef6587b" /* ** CAPI3REF: Run-Time Library Version Numbers ** KEYWORDS: sqlite3_version, sqlite3_sourceid ** ** These interfaces provide the same information as the [SQLITE_VERSION], ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros |
︙ | ︙ | |||
849 850 851 852 853 854 855 | ** ** ^The sqlite3_exec() interface runs zero or more UTF-8 encoded, ** semicolon-separate SQL statements passed into its 2nd argument, ** in the context of the [database connection] passed in as its 1st ** argument. ^If the callback function of the 3rd argument to ** sqlite3_exec() is not NULL, then it is invoked for each result row ** coming out of the evaluated SQL statements. ^The 4th argument to | | | 849 850 851 852 853 854 855 856 857 858 859 860 861 862 863 | ** ** ^The sqlite3_exec() interface runs zero or more UTF-8 encoded, ** semicolon-separate SQL statements passed into its 2nd argument, ** in the context of the [database connection] passed in as its 1st ** argument. ^If the callback function of the 3rd argument to ** sqlite3_exec() is not NULL, then it is invoked for each result row ** coming out of the evaluated SQL statements. ^The 4th argument to ** sqlite3_exec() is relayed through to the 1st argument of each ** callback invocation. ^If the callback pointer to sqlite3_exec() ** is NULL, then no callback is ever invoked and result rows are ** ignored. ** ** ^If an error occurs while evaluating the SQL statements passed into ** sqlite3_exec(), then execution of the current statement stops and ** subsequent statements are skipped. ^If the 5th parameter to sqlite3_exec() |
︙ | ︙ | |||
998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 | #define SQLITE_IOERR_SHMLOCK (SQLITE_IOERR | (20<<8)) #define SQLITE_IOERR_SHMMAP (SQLITE_IOERR | (21<<8)) #define SQLITE_IOERR_SEEK (SQLITE_IOERR | (22<<8)) #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) #define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) /* ** CAPI3REF: Flags For File Open Operations ** ** These bit values are intended for use in the ** 3rd parameter to the [sqlite3_open_v2()] interface and ** in the 4th parameter to the [sqlite3_vfs.xOpen] method. | > > | 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 | #define SQLITE_IOERR_SHMLOCK (SQLITE_IOERR | (20<<8)) #define SQLITE_IOERR_SHMMAP (SQLITE_IOERR | (21<<8)) #define SQLITE_IOERR_SEEK (SQLITE_IOERR | (22<<8)) #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) #define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) #define SQLITE_READONLY_RECOVERY (SQLITE_READONLY | (1<<8)) #define SQLITE_READONLY_CANTLOCK (SQLITE_READONLY | (2<<8)) /* ** CAPI3REF: Flags For File Open Operations ** ** These bit values are intended for use in the ** 3rd parameter to the [sqlite3_open_v2()] interface and ** in the 4th parameter to the [sqlite3_vfs.xOpen] method. |
︙ | ︙ | |||
1439 1440 1441 1442 1443 1444 1445 | ** of good-quality randomness into zOut. The return value is ** the actual number of bytes of randomness obtained. ** The xSleep() method causes the calling thread to sleep for at ** least the number of microseconds given. ^The xCurrentTime() ** method returns a Julian Day Number for the current date and time as ** a floating point value. ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian | | | 1441 1442 1443 1444 1445 1446 1447 1448 1449 1450 1451 1452 1453 1454 1455 | ** of good-quality randomness into zOut. The return value is ** the actual number of bytes of randomness obtained. ** The xSleep() method causes the calling thread to sleep for at ** least the number of microseconds given. ^The xCurrentTime() ** method returns a Julian Day Number for the current date and time as ** a floating point value. ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian ** Day Number multiplied by 86400000 (the number of milliseconds in ** a 24-hour day). ** ^SQLite will use the xCurrentTimeInt64() method to get the current ** date and time if that method is available (if iVersion is 2 or ** greater and the function pointer is not NULL) and will fall back ** to xCurrentTime() if xCurrentTimeInt64() is unavailable. ** ** ^The xSetSystemCall(), xGetSystemCall(), and xNestSystemCall() interfaces |
︙ | ︙ | |||
1877 1878 1879 1880 1881 1882 1883 | ** ^SQLite will never require a scratch buffer that is more than 6 ** times the database page size. ^If SQLite needs needs additional ** scratch memory beyond what is provided by this configuration option, then ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd> ** ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt> ** <dd> ^This option specifies a static memory buffer that SQLite can use for | | | 1879 1880 1881 1882 1883 1884 1885 1886 1887 1888 1889 1890 1891 1892 1893 | ** ^SQLite will never require a scratch buffer that is more than 6 ** times the database page size. ^If SQLite needs needs additional ** scratch memory beyond what is provided by this configuration option, then ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd> ** ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt> ** <dd> ^This option specifies a static memory buffer that SQLite can use for ** the database page cache with the default page cache implementation. ** This configuration should not be used if an application-define page ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option. ** There are three arguments to this option: A pointer to 8-byte aligned ** memory, the size of each page buffer (sz), and the number of pages (N). ** The sz argument should be the size of the largest database page ** (a power of two between 512 and 32768) plus a little extra for each ** page header. ^The page header size is 20 to 40 bytes depending on |
︙ | ︙ | |||
2975 2976 2977 2978 2979 2980 2981 | ** automatically deleted as soon as the database connection is closed. ** ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3> ** ** ^If [URI filename] interpretation is enabled, and the filename argument ** begins with "file:", then the filename is interpreted as a URI. ^URI ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is | | | | 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 | ** automatically deleted as soon as the database connection is closed. ** ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3> ** ** ^If [URI filename] interpretation is enabled, and the filename argument ** begins with "file:", then the filename is interpreted as a URI. ^URI ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is ** set in the fourth argument to sqlite3_open_v2(), or if it has ** been enabled globally using the [SQLITE_CONFIG_URI] option with the ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option. ** As of SQLite version 3.7.7, URI filename interpretation is turned off ** by default, but future releases of SQLite might enable URI filename ** interpretation by default. See "[URI filenames]" for additional ** information. ** ** URI filenames are parsed according to RFC 3986. ^If the URI contains an ** authority, then it must be either an empty string or the string ** "localhost". ^If the authority is not an empty string or "localhost", an ** error is returned to the caller. ^The fragment component of a URI, if ** present, is ignored. |
︙ | ︙ | |||
3799 3800 3801 3802 3803 3804 3805 | ** [SQLITE_DONE], [SQLITE_ROW], [SQLITE_ERROR], or [SQLITE_MISUSE]. ** ^With the "v2" interface, any of the other [result codes] or ** [extended result codes] might be returned as well. ** ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the ** database locks it needs to do its job. ^If the statement is a [COMMIT] ** or occurs outside of an explicit transaction, then you can retry the | | | 3801 3802 3803 3804 3805 3806 3807 3808 3809 3810 3811 3812 3813 3814 3815 | ** [SQLITE_DONE], [SQLITE_ROW], [SQLITE_ERROR], or [SQLITE_MISUSE]. ** ^With the "v2" interface, any of the other [result codes] or ** [extended result codes] might be returned as well. ** ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the ** database locks it needs to do its job. ^If the statement is a [COMMIT] ** or occurs outside of an explicit transaction, then you can retry the ** statement. If the statement is not a [COMMIT] and occurs within an ** explicit transaction then you should rollback the transaction before ** continuing. ** ** ^[SQLITE_DONE] means that the statement has finished executing ** successfully. sqlite3_step() should not be called again on this virtual ** machine without first calling [sqlite3_reset()] to reset the virtual ** machine back to its initial state. |
︙ | ︙ | |||
4078 4079 4080 4081 4082 4083 4084 | SQLITE_API int sqlite3_column_type(sqlite3_stmt*, int iCol); SQLITE_API sqlite3_value *sqlite3_column_value(sqlite3_stmt*, int iCol); /* ** CAPI3REF: Destroy A Prepared Statement Object ** ** ^The sqlite3_finalize() function is called to delete a [prepared statement]. | | | 4080 4081 4082 4083 4084 4085 4086 4087 4088 4089 4090 4091 4092 4093 4094 | SQLITE_API int sqlite3_column_type(sqlite3_stmt*, int iCol); SQLITE_API sqlite3_value *sqlite3_column_value(sqlite3_stmt*, int iCol); /* ** CAPI3REF: Destroy A Prepared Statement Object ** ** ^The sqlite3_finalize() function is called to delete a [prepared statement]. ** ^If the most recent evaluation of the statement encountered no errors ** or if the statement is never been evaluated, then sqlite3_finalize() returns ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then ** sqlite3_finalize(S) returns the appropriate [error code] or ** [extended error code]. ** ** ^The sqlite3_finalize(S) routine can be called at any point during ** the life cycle of [prepared statement] S: |
︙ | ︙ | |||
5992 5993 5994 5995 5996 5997 5998 | ** ^The implementation is not required to provided versions of these ** routines that actually work. If the implementation does not provide working ** versions of these routines, it should at least provide stubs that always ** return true so that one does not get spurious assertion failures. ** ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then ** the routine should return 1. This seems counter-intuitive since | | | 5994 5995 5996 5997 5998 5999 6000 6001 6002 6003 6004 6005 6006 6007 6008 | ** ^The implementation is not required to provided versions of these ** routines that actually work. If the implementation does not provide working ** versions of these routines, it should at least provide stubs that always ** return true so that one does not get spurious assertion failures. ** ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then ** the routine should return 1. This seems counter-intuitive since ** clearly the mutex cannot be held if it does not exist. But ** the reason the mutex does not exist is because the build is not ** using mutexes. And we do not want the assert() containing the ** call to sqlite3_mutex_held() to fail, so a non-zero return is ** the appropriate thing to do. ^The sqlite3_mutex_notheld() ** interface should also return 1 when given a NULL pointer. */ #ifndef NDEBUG |
︙ | ︙ | |||
6115 6116 6117 6118 6119 6120 6121 | #define SQLITE_TESTCTRL_ASSERT 12 #define SQLITE_TESTCTRL_ALWAYS 13 #define SQLITE_TESTCTRL_RESERVE 14 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 #define SQLITE_TESTCTRL_ISKEYWORD 16 #define SQLITE_TESTCTRL_PGHDRSZ 17 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18 | > | | 6117 6118 6119 6120 6121 6122 6123 6124 6125 6126 6127 6128 6129 6130 6131 6132 | #define SQLITE_TESTCTRL_ASSERT 12 #define SQLITE_TESTCTRL_ALWAYS 13 #define SQLITE_TESTCTRL_RESERVE 14 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 #define SQLITE_TESTCTRL_ISKEYWORD 16 #define SQLITE_TESTCTRL_PGHDRSZ 17 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18 #define SQLITE_TESTCTRL_LOCALTIME_FAULT 19 #define SQLITE_TESTCTRL_LAST 19 /* ** CAPI3REF: SQLite Runtime Status ** ** ^This interface is used to retrieve runtime status information ** about the performance of SQLite, and optionally to reset various ** highwater marks. ^The first argument is an integer code for |
︙ | ︙ | |||
6501 6502 6503 6504 6505 6506 6507 | ** stored in the cache, both pinned and unpinned. ** ** [[the xFetch() page cache methods]] ** The xFetch() method locates a page in the cache and returns a pointer to ** the page, or a NULL pointer. ** A "page", in this context, means a buffer of szPage bytes aligned at an ** 8-byte boundary. The page to be fetched is determined by the key. ^The | | | 6504 6505 6506 6507 6508 6509 6510 6511 6512 6513 6514 6515 6516 6517 6518 | ** stored in the cache, both pinned and unpinned. ** ** [[the xFetch() page cache methods]] ** The xFetch() method locates a page in the cache and returns a pointer to ** the page, or a NULL pointer. ** A "page", in this context, means a buffer of szPage bytes aligned at an ** 8-byte boundary. The page to be fetched is determined by the key. ^The ** minimum key value is 1. After it has been retrieved using xFetch, the page ** is considered to be "pinned". ** ** If the requested page is already in the page cache, then the page cache ** implementation must return a pointer to the page buffer with its content ** intact. If the requested page is not already in the cache, then the ** cache implementation should use the value of the createFlag ** parameter to help it determined what action to take: |
︙ | ︙ | |||
8479 8480 8481 8482 8483 8484 8485 8486 8487 8488 8489 8490 8491 8492 8493 8494 8495 8496 8497 8498 | SQLITE_PRIVATE int sqlite3VdbeAddOp0(Vdbe*,int); SQLITE_PRIVATE int sqlite3VdbeAddOp1(Vdbe*,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int); SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp); SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1); SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2); SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3); SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u8 P5); SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr); SQLITE_PRIVATE void sqlite3VdbeChangeToNoop(Vdbe*, int addr, int N); SQLITE_PRIVATE void sqlite3VdbeChangeP4(Vdbe*, int addr, const char *zP4, int N); SQLITE_PRIVATE void sqlite3VdbeUsesBtree(Vdbe*, int); SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetOp(Vdbe*, int); SQLITE_PRIVATE int sqlite3VdbeMakeLabel(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeRunOnlyOnce(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeDelete(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeDeleteObject(sqlite3*,Vdbe*); | > | > | 8482 8483 8484 8485 8486 8487 8488 8489 8490 8491 8492 8493 8494 8495 8496 8497 8498 8499 8500 8501 8502 8503 8504 8505 8506 8507 8508 8509 8510 8511 8512 8513 8514 8515 8516 8517 8518 8519 | SQLITE_PRIVATE int sqlite3VdbeAddOp0(Vdbe*,int); SQLITE_PRIVATE int sqlite3VdbeAddOp1(Vdbe*,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp2(Vdbe*,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp3(Vdbe*,int,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOp4(Vdbe*,int,int,int,int,const char *zP4,int); SQLITE_PRIVATE int sqlite3VdbeAddOp4Int(Vdbe*,int,int,int,int,int); SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe*, int nOp, VdbeOpList const *aOp); SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe*,int,char*); SQLITE_PRIVATE void sqlite3VdbeChangeP1(Vdbe*, int addr, int P1); SQLITE_PRIVATE void sqlite3VdbeChangeP2(Vdbe*, int addr, int P2); SQLITE_PRIVATE void sqlite3VdbeChangeP3(Vdbe*, int addr, int P3); SQLITE_PRIVATE void sqlite3VdbeChangeP5(Vdbe*, u8 P5); SQLITE_PRIVATE void sqlite3VdbeJumpHere(Vdbe*, int addr); SQLITE_PRIVATE void sqlite3VdbeChangeToNoop(Vdbe*, int addr, int N); SQLITE_PRIVATE void sqlite3VdbeChangeP4(Vdbe*, int addr, const char *zP4, int N); SQLITE_PRIVATE void sqlite3VdbeUsesBtree(Vdbe*, int); SQLITE_PRIVATE VdbeOp *sqlite3VdbeGetOp(Vdbe*, int); SQLITE_PRIVATE int sqlite3VdbeMakeLabel(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeRunOnlyOnce(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeDelete(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeDeleteObject(sqlite3*,Vdbe*); SQLITE_PRIVATE void sqlite3VdbeMakeReady(Vdbe*,Parse*); SQLITE_PRIVATE int sqlite3VdbeFinalize(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeResolveLabel(Vdbe*, int); SQLITE_PRIVATE int sqlite3VdbeCurrentAddr(Vdbe*); #ifdef SQLITE_DEBUG SQLITE_PRIVATE int sqlite3VdbeAssertMayAbort(Vdbe *, int); SQLITE_PRIVATE void sqlite3VdbeTrace(Vdbe*,FILE*); #endif SQLITE_PRIVATE void sqlite3VdbeResetStepResult(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeRewind(Vdbe*); SQLITE_PRIVATE int sqlite3VdbeReset(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeSetNumCols(Vdbe*,int); SQLITE_PRIVATE int sqlite3VdbeSetColName(Vdbe*, int, int, const char *, void(*)(void*)); SQLITE_PRIVATE void sqlite3VdbeCountChanges(Vdbe*); SQLITE_PRIVATE sqlite3 *sqlite3VdbeDb(Vdbe*); SQLITE_PRIVATE void sqlite3VdbeSetSql(Vdbe*, const char *z, int n, int); SQLITE_PRIVATE void sqlite3VdbeSwap(Vdbe*,Vdbe*); |
︙ | ︙ | |||
9277 9278 9279 9280 9281 9282 9283 | ** Schema objects are automatically deallocated when the last Btree that ** references them is destroyed. The TEMP Schema is manually freed by ** sqlite3_close(). * ** A thread must be holding a mutex on the corresponding Btree in order ** to access Schema content. This implies that the thread must also be ** holding a mutex on the sqlite3 connection pointer that owns the Btree. | | | 9282 9283 9284 9285 9286 9287 9288 9289 9290 9291 9292 9293 9294 9295 9296 | ** Schema objects are automatically deallocated when the last Btree that ** references them is destroyed. The TEMP Schema is manually freed by ** sqlite3_close(). * ** A thread must be holding a mutex on the corresponding Btree in order ** to access Schema content. This implies that the thread must also be ** holding a mutex on the sqlite3 connection pointer that owns the Btree. ** For a TEMP Schema, only the connection mutex is required. */ struct Schema { int schema_cookie; /* Database schema version number for this file */ int iGeneration; /* Generation counter. Incremented with each change */ Hash tblHash; /* All tables indexed by name */ Hash idxHash; /* All (named) indices indexed by name */ Hash trigHash; /* All triggers indexed by name */ |
︙ | ︙ | |||
9550 9551 9552 9553 9554 9555 9556 9557 9558 9559 9560 9561 9562 9563 | #define SQLITE_QueryFlattener 0x01 /* Disable query flattening */ #define SQLITE_ColumnCache 0x02 /* Disable the column cache */ #define SQLITE_IndexSort 0x04 /* Disable indexes for sorting */ #define SQLITE_IndexSearch 0x08 /* Disable indexes for searching */ #define SQLITE_IndexCover 0x10 /* Disable index covering table */ #define SQLITE_GroupByOrder 0x20 /* Disable GROUPBY cover of ORDERBY */ #define SQLITE_FactorOutConst 0x40 /* Disable factoring out constants */ #define SQLITE_OptMask 0xff /* Mask of all disablable opts */ /* ** Possible values for the sqlite.magic field. ** The numbers are obtained at random and have no special meaning, other ** than being distinct from one another. */ | > | 9555 9556 9557 9558 9559 9560 9561 9562 9563 9564 9565 9566 9567 9568 9569 | #define SQLITE_QueryFlattener 0x01 /* Disable query flattening */ #define SQLITE_ColumnCache 0x02 /* Disable the column cache */ #define SQLITE_IndexSort 0x04 /* Disable indexes for sorting */ #define SQLITE_IndexSearch 0x08 /* Disable indexes for searching */ #define SQLITE_IndexCover 0x10 /* Disable index covering table */ #define SQLITE_GroupByOrder 0x20 /* Disable GROUPBY cover of ORDERBY */ #define SQLITE_FactorOutConst 0x40 /* Disable factoring out constants */ #define SQLITE_IdxRealAsInt 0x80 /* Store REAL as INT in indices */ #define SQLITE_OptMask 0xff /* Mask of all disablable opts */ /* ** Possible values for the sqlite.magic field. ** The numbers are obtained at random and have no special meaning, other ** than being distinct from one another. */ |
︙ | ︙ | |||
10825 10826 10827 10828 10829 10830 10831 | u8 disableTriggers; /* True to disable triggers */ double nQueryLoop; /* Estimated number of iterations of a query */ /* Above is constant between recursions. Below is reset before and after ** each recursion */ int nVar; /* Number of '?' variables seen in the SQL so far */ | < | | | 10831 10832 10833 10834 10835 10836 10837 10838 10839 10840 10841 10842 10843 10844 10845 10846 | u8 disableTriggers; /* True to disable triggers */ double nQueryLoop; /* Estimated number of iterations of a query */ /* Above is constant between recursions. Below is reset before and after ** each recursion */ int nVar; /* Number of '?' variables seen in the SQL so far */ int nzVar; /* Number of available slots in azVar[] */ char **azVar; /* Pointers to names of parameters */ Vdbe *pReprepare; /* VM being reprepared (sqlite3Reprepare()) */ int nAlias; /* Number of aliased result set columns */ int nAliasAlloc; /* Number of allocated slots for aAlias[] */ int *aAlias; /* Register used to hold aliased result */ u8 explain; /* True if the EXPLAIN flag is found on the query */ Token sNameToken; /* Token with unqualified schema object name */ Token sLastToken; /* The last token parsed */ |
︙ | ︙ | |||
11048 11049 11050 11051 11052 11053 11054 11055 11056 11057 11058 11059 11060 11061 | int isMutexInit; /* True after mutexes are initialized */ int isMallocInit; /* True after malloc is initialized */ int isPCacheInit; /* True after malloc is initialized */ sqlite3_mutex *pInitMutex; /* Mutex used by sqlite3_initialize() */ int nRefInitMutex; /* Number of users of pInitMutex */ void (*xLog)(void*,int,const char*); /* Function for logging */ void *pLogArg; /* First argument to xLog() */ }; /* ** Context pointer passed down through the tree-walk. */ struct Walker { int (*xExprCallback)(Walker*, Expr*); /* Callback for expressions */ | > | 11053 11054 11055 11056 11057 11058 11059 11060 11061 11062 11063 11064 11065 11066 11067 | int isMutexInit; /* True after mutexes are initialized */ int isMallocInit; /* True after malloc is initialized */ int isPCacheInit; /* True after malloc is initialized */ sqlite3_mutex *pInitMutex; /* Mutex used by sqlite3_initialize() */ int nRefInitMutex; /* Number of users of pInitMutex */ void (*xLog)(void*,int,const char*); /* Function for logging */ void *pLogArg; /* First argument to xLog() */ int bLocaltimeFault; /* True to fail localtime() calls */ }; /* ** Context pointer passed down through the tree-walk. */ struct Walker { int (*xExprCallback)(Walker*, Expr*); /* Callback for expressions */ |
︙ | ︙ | |||
11475 11476 11477 11478 11479 11480 11481 | SQLITE_PRIVATE int sqlite3FixExprList(DbFixer*, ExprList*); SQLITE_PRIVATE int sqlite3FixTriggerStep(DbFixer*, TriggerStep*); SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8); SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*); SQLITE_PRIVATE int sqlite3Atoi(const char*); SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar); SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte); | | | 11481 11482 11483 11484 11485 11486 11487 11488 11489 11490 11491 11492 11493 11494 11495 | SQLITE_PRIVATE int sqlite3FixExprList(DbFixer*, ExprList*); SQLITE_PRIVATE int sqlite3FixTriggerStep(DbFixer*, TriggerStep*); SQLITE_PRIVATE int sqlite3AtoF(const char *z, double*, int, u8); SQLITE_PRIVATE int sqlite3GetInt32(const char *, int*); SQLITE_PRIVATE int sqlite3Atoi(const char*); SQLITE_PRIVATE int sqlite3Utf16ByteLen(const void *pData, int nChar); SQLITE_PRIVATE int sqlite3Utf8CharLen(const char *pData, int nByte); SQLITE_PRIVATE u32 sqlite3Utf8Read(const u8*, const u8**); /* ** Routines to read and write variable-length integers. These used to ** be defined locally, but now we use the varint routines in the util.c ** file. Code should use the MACRO forms below, as the Varint32 versions ** are coded to assume the single byte case is already handled (which ** the MACRO form does). |
︙ | ︙ | |||
12008 12009 12010 12011 12012 12013 12014 12015 12016 12017 12018 12019 12020 12021 | 0, /* isMutexInit */ 0, /* isMallocInit */ 0, /* isPCacheInit */ 0, /* pInitMutex */ 0, /* nRefInitMutex */ 0, /* xLog */ 0, /* pLogArg */ }; /* ** Hash table for global functions - functions common to all ** database connections. After initialization, this table is ** read-only. | > | 12014 12015 12016 12017 12018 12019 12020 12021 12022 12023 12024 12025 12026 12027 12028 | 0, /* isMutexInit */ 0, /* isMallocInit */ 0, /* isPCacheInit */ 0, /* pInitMutex */ 0, /* nRefInitMutex */ 0, /* xLog */ 0, /* pLogArg */ 0, /* bLocaltimeFault */ }; /* ** Hash table for global functions - functions common to all ** database connections. After initialization, this table is ** read-only. |
︙ | ︙ | |||
12760 12761 12762 12763 12764 12765 12766 12767 12768 12769 12770 | u32 magic; /* Magic number for sanity checking */ char *zErrMsg; /* Error message written here */ Vdbe *pPrev,*pNext; /* Linked list of VDBEs with the same Vdbe.db */ VdbeCursor **apCsr; /* One element of this array for each open cursor */ Mem *aVar; /* Values for the OP_Variable opcode. */ char **azVar; /* Name of variables */ ynVar nVar; /* Number of entries in aVar[] */ u32 cacheCtr; /* VdbeCursor row cache generation counter */ int pc; /* The program counter */ int rc; /* Value to return */ u8 errorAction; /* Recovery action to do in case of an error */ | > < | 12767 12768 12769 12770 12771 12772 12773 12774 12775 12776 12777 12778 12779 12780 12781 12782 12783 12784 12785 | u32 magic; /* Magic number for sanity checking */ char *zErrMsg; /* Error message written here */ Vdbe *pPrev,*pNext; /* Linked list of VDBEs with the same Vdbe.db */ VdbeCursor **apCsr; /* One element of this array for each open cursor */ Mem *aVar; /* Values for the OP_Variable opcode. */ char **azVar; /* Name of variables */ ynVar nVar; /* Number of entries in aVar[] */ ynVar nzVar; /* Number of entries in azVar[] */ u32 cacheCtr; /* VdbeCursor row cache generation counter */ int pc; /* The program counter */ int rc; /* Value to return */ u8 errorAction; /* Recovery action to do in case of an error */ u8 explain; /* True if EXPLAIN present on SQL command */ u8 changeCntOn; /* True to update the change-counter */ u8 expired; /* True if the VM needs to be recompiled */ u8 runOnlyOnce; /* Automatically expire on reset */ u8 minWriteFileFormat; /* Minimum file format for writable database files */ u8 inVtabMethod; /* See comments above */ u8 usesStmtJournal; /* True if uses a statement journal */ |
︙ | ︙ | |||
13158 13159 13160 13161 13162 13163 13164 | ** Willmann-Bell, Inc ** Richmond, Virginia (USA) */ #include <time.h> #ifndef SQLITE_OMIT_DATETIME_FUNCS | < < < < < < < < < < < < < < < < | 13165 13166 13167 13168 13169 13170 13171 13172 13173 13174 13175 13176 13177 13178 | ** Willmann-Bell, Inc ** Richmond, Virginia (USA) */ #include <time.h> #ifndef SQLITE_OMIT_DATETIME_FUNCS /* ** A structure for holding a single date and time. */ typedef struct DateTime DateTime; struct DateTime { sqlite3_int64 iJD; /* The julian day number times 86400000 */ |
︙ | ︙ | |||
13518 13519 13520 13521 13522 13523 13524 13525 13526 13527 | ** Clear the YMD and HMS and the TZ */ static void clearYMD_HMS_TZ(DateTime *p){ p->validYMD = 0; p->validHMS = 0; p->validTZ = 0; } #ifndef SQLITE_OMIT_LOCALTIME /* | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < | > > > > | > > > > > > > > > < < < | | < | | < < < < < < | | | | | | < < < < < < < < < < < < < < < > | 13509 13510 13511 13512 13513 13514 13515 13516 13517 13518 13519 13520 13521 13522 13523 13524 13525 13526 13527 13528 13529 13530 13531 13532 13533 13534 13535 13536 13537 13538 13539 13540 13541 13542 13543 13544 13545 13546 13547 13548 13549 13550 13551 13552 13553 13554 13555 13556 13557 13558 13559 13560 13561 13562 13563 13564 13565 13566 13567 13568 13569 13570 13571 13572 13573 13574 13575 13576 13577 13578 13579 13580 13581 13582 13583 13584 13585 13586 13587 13588 13589 13590 13591 13592 13593 13594 13595 13596 13597 13598 13599 13600 13601 13602 13603 13604 13605 13606 13607 13608 13609 13610 13611 13612 13613 13614 13615 13616 13617 13618 13619 13620 13621 13622 13623 13624 13625 13626 13627 13628 13629 13630 13631 13632 13633 13634 | ** Clear the YMD and HMS and the TZ */ static void clearYMD_HMS_TZ(DateTime *p){ p->validYMD = 0; p->validHMS = 0; p->validTZ = 0; } /* ** On recent Windows platforms, the localtime_s() function is available ** as part of the "Secure CRT". It is essentially equivalent to ** localtime_r() available under most POSIX platforms, except that the ** order of the parameters is reversed. ** ** See http://msdn.microsoft.com/en-us/library/a442x3ye(VS.80).aspx. ** ** If the user has not indicated to use localtime_r() or localtime_s() ** already, check for an MSVC build environment that provides ** localtime_s(). */ #if !defined(HAVE_LOCALTIME_R) && !defined(HAVE_LOCALTIME_S) && \ defined(_MSC_VER) && defined(_CRT_INSECURE_DEPRECATE) #define HAVE_LOCALTIME_S 1 #endif #ifndef SQLITE_OMIT_LOCALTIME /* ** The following routine implements the rough equivalent of localtime_r() ** using whatever operating-system specific localtime facility that ** is available. This routine returns 0 on success and ** non-zero on any kind of error. ** ** If the sqlite3GlobalConfig.bLocaltimeFault variable is true then this ** routine will always fail. */ static int osLocaltime(time_t *t, struct tm *pTm){ int rc; #if (!defined(HAVE_LOCALTIME_R) || !HAVE_LOCALTIME_R) \ && (!defined(HAVE_LOCALTIME_S) || !HAVE_LOCALTIME_S) struct tm *pX; sqlite3_mutex *mutex = sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_MASTER); sqlite3_mutex_enter(mutex); pX = localtime(t); #ifndef SQLITE_OMIT_BUILTIN_TEST if( sqlite3GlobalConfig.bLocaltimeFault ) pX = 0; #endif if( pX ) *pTm = *pX; sqlite3_mutex_leave(mutex); rc = pX==0; #else #ifndef SQLITE_OMIT_BUILTIN_TEST if( sqlite3GlobalConfig.bLocaltimeFault ) return 1; #endif #if defined(HAVE_LOCALTIME_R) && HAVE_LOCALTIME_R rc = localtime_r(t, pTm)==0; #else rc = localtime_s(pTm, t); #endif /* HAVE_LOCALTIME_R */ #endif /* HAVE_LOCALTIME_R || HAVE_LOCALTIME_S */ return rc; } #endif /* SQLITE_OMIT_LOCALTIME */ #ifndef SQLITE_OMIT_LOCALTIME /* ** Compute the difference (in milliseconds) between localtime and UTC ** (a.k.a. GMT) for the time value p where p is in UTC. If no error occurs, ** return this value and set *pRc to SQLITE_OK. ** ** Or, if an error does occur, set *pRc to SQLITE_ERROR. The returned value ** is undefined in this case. */ static sqlite3_int64 localtimeOffset( DateTime *p, /* Date at which to calculate offset */ sqlite3_context *pCtx, /* Write error here if one occurs */ int *pRc /* OUT: Error code. SQLITE_OK or ERROR */ ){ DateTime x, y; time_t t; struct tm sLocal; /* Initialize the contents of sLocal to avoid a compiler warning. */ memset(&sLocal, 0, sizeof(sLocal)); x = *p; computeYMD_HMS(&x); if( x.Y<1971 || x.Y>=2038 ){ x.Y = 2000; x.M = 1; x.D = 1; x.h = 0; x.m = 0; x.s = 0.0; } else { int s = (int)(x.s + 0.5); x.s = s; } x.tz = 0; x.validJD = 0; computeJD(&x); t = (time_t)(x.iJD/1000 - 21086676*(i64)10000); if( osLocaltime(&t, &sLocal) ){ sqlite3_result_error(pCtx, "local time unavailable", -1); *pRc = SQLITE_ERROR; return 0; } y.Y = sLocal.tm_year + 1900; y.M = sLocal.tm_mon + 1; y.D = sLocal.tm_mday; y.h = sLocal.tm_hour; y.m = sLocal.tm_min; y.s = sLocal.tm_sec; y.validYMD = 1; y.validHMS = 1; y.validJD = 0; y.validTZ = 0; computeJD(&y); *pRc = SQLITE_OK; return y.iJD - x.iJD; } #endif /* SQLITE_OMIT_LOCALTIME */ /* ** Process a modifier to a date-time stamp. The modifiers are ** as follows: |
︙ | ︙ | |||
13609 13610 13611 13612 13613 13614 13615 | ** start of week ** start of day ** weekday N ** unixepoch ** localtime ** utc ** | | > > > | | < | > | | | < > | 13644 13645 13646 13647 13648 13649 13650 13651 13652 13653 13654 13655 13656 13657 13658 13659 13660 13661 13662 13663 13664 13665 13666 13667 13668 13669 13670 13671 13672 13673 13674 13675 13676 13677 13678 13679 13680 13681 13682 13683 13684 13685 13686 13687 13688 13689 13690 13691 13692 13693 13694 13695 13696 13697 13698 13699 13700 13701 13702 13703 13704 13705 13706 13707 13708 13709 13710 | ** start of week ** start of day ** weekday N ** unixepoch ** localtime ** utc ** ** Return 0 on success and 1 if there is any kind of error. If the error ** is in a system call (i.e. localtime()), then an error message is written ** to context pCtx. If the error is an unrecognized modifier, no error is ** written to pCtx. */ static int parseModifier(sqlite3_context *pCtx, const char *zMod, DateTime *p){ int rc = 1; int n; double r; char *z, zBuf[30]; z = zBuf; for(n=0; n<ArraySize(zBuf)-1 && zMod[n]; n++){ z[n] = (char)sqlite3UpperToLower[(u8)zMod[n]]; } z[n] = 0; switch( z[0] ){ #ifndef SQLITE_OMIT_LOCALTIME case 'l': { /* localtime ** ** Assuming the current time value is UTC (a.k.a. GMT), shift it to ** show local time. */ if( strcmp(z, "localtime")==0 ){ computeJD(p); p->iJD += localtimeOffset(p, pCtx, &rc); clearYMD_HMS_TZ(p); } break; } #endif case 'u': { /* ** unixepoch ** ** Treat the current value of p->iJD as the number of ** seconds since 1970. Convert to a real julian day number. */ if( strcmp(z, "unixepoch")==0 && p->validJD ){ p->iJD = (p->iJD + 43200)/86400 + 21086676*(i64)10000000; clearYMD_HMS_TZ(p); rc = 0; } #ifndef SQLITE_OMIT_LOCALTIME else if( strcmp(z, "utc")==0 ){ sqlite3_int64 c1; computeJD(p); c1 = localtimeOffset(p, pCtx, &rc); if( rc==SQLITE_OK ){ p->iJD -= c1; clearYMD_HMS_TZ(p); p->iJD += c1 - localtimeOffset(p, pCtx, &rc); } } #endif break; } case 'w': { /* ** weekday N |
︙ | ︙ | |||
13839 13840 13841 13842 13843 13844 13845 | }else{ z = sqlite3_value_text(argv[0]); if( !z || parseDateOrTime(context, (char*)z, p) ){ return 1; } } for(i=1; i<argc; i++){ | | | < | 13877 13878 13879 13880 13881 13882 13883 13884 13885 13886 13887 13888 13889 13890 13891 13892 | }else{ z = sqlite3_value_text(argv[0]); if( !z || parseDateOrTime(context, (char*)z, p) ){ return 1; } } for(i=1; i<argc; i++){ z = sqlite3_value_text(argv[i]); if( z==0 || parseModifier(context, (char*)z, p) ) return 1; } return 0; } /* ** The following routines implement the various date and time functions |
︙ | ︙ | |||
20082 20083 20084 20085 20086 20087 20088 | while( zIn!=zTerm && (*zIn & 0xc0)==0x80 ){ \ c = (c<<6) + (0x3f & *(zIn++)); \ } \ if( c<0x80 \ || (c&0xFFFFF800)==0xD800 \ || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \ } | | | 20119 20120 20121 20122 20123 20124 20125 20126 20127 20128 20129 20130 20131 20132 20133 | while( zIn!=zTerm && (*zIn & 0xc0)==0x80 ){ \ c = (c<<6) + (0x3f & *(zIn++)); \ } \ if( c<0x80 \ || (c&0xFFFFF800)==0xD800 \ || (c&0xFFFFFFFE)==0xFFFE ){ c = 0xFFFD; } \ } SQLITE_PRIVATE u32 sqlite3Utf8Read( const unsigned char *zIn, /* First byte of UTF-8 character */ const unsigned char **pzNext /* Write first byte past UTF-8 char here */ ){ unsigned int c; /* Same as READ_UTF8() above but without the zTerm parameter. ** For this routine, we assume the UTF8 string is always zero-terminated. |
︙ | ︙ | |||
24359 24360 24361 24362 24363 24364 24365 24366 24367 24368 24369 24370 24371 24372 | # endif #endif /* SQLITE_ENABLE_LOCKING_STYLE */ #if defined(__APPLE__) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS) # include <sys/mount.h> #endif /* ** Allowed values of unixFile.fsFlags */ #define SQLITE_FSFLAGS_IS_MSDOS 0x1 /* ** If we are to be thread-safe, include the pthreads header and define | > > > > | 24396 24397 24398 24399 24400 24401 24402 24403 24404 24405 24406 24407 24408 24409 24410 24411 24412 24413 | # endif #endif /* SQLITE_ENABLE_LOCKING_STYLE */ #if defined(__APPLE__) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS) # include <sys/mount.h> #endif #ifdef HAVE_UTIME # include <utime.h> #endif /* ** Allowed values of unixFile.fsFlags */ #define SQLITE_FSFLAGS_IS_MSDOS 0x1 /* ** If we are to be thread-safe, include the pthreads header and define |
︙ | ︙ | |||
26364 26365 26366 26367 26368 26369 26370 | /* If we have any lock, then the lock file already exists. All we have ** to do is adjust our internal record of the lock level. */ if( pFile->eFileLock > NO_LOCK ){ pFile->eFileLock = eFileLock; | < > > > | 26405 26406 26407 26408 26409 26410 26411 26412 26413 26414 26415 26416 26417 26418 26419 26420 26421 26422 | /* If we have any lock, then the lock file already exists. All we have ** to do is adjust our internal record of the lock level. */ if( pFile->eFileLock > NO_LOCK ){ pFile->eFileLock = eFileLock; /* Always update the timestamp on the old file */ #ifdef HAVE_UTIME utime(zLockFile, NULL); #else utimes(zLockFile, NULL); #endif return SQLITE_OK; } /* grab an exclusive lock */ fd = robust_open(zLockFile,O_RDONLY|O_CREAT|O_EXCL,0600); |
︙ | ︙ | |||
27962 27963 27964 27965 27966 27967 27968 | */ struct unixShmNode { unixInodeInfo *pInode; /* unixInodeInfo that owns this SHM node */ sqlite3_mutex *mutex; /* Mutex to access this object */ char *zFilename; /* Name of the mmapped file */ int h; /* Open file descriptor */ int szRegion; /* Size of shared-memory regions */ | | > | 28005 28006 28007 28008 28009 28010 28011 28012 28013 28014 28015 28016 28017 28018 28019 28020 | */ struct unixShmNode { unixInodeInfo *pInode; /* unixInodeInfo that owns this SHM node */ sqlite3_mutex *mutex; /* Mutex to access this object */ char *zFilename; /* Name of the mmapped file */ int h; /* Open file descriptor */ int szRegion; /* Size of shared-memory regions */ u16 nRegion; /* Size of array apRegion */ u8 isReadonly; /* True if read-only */ char **apRegion; /* Array of mapped shared-memory regions */ int nRef; /* Number of unixShm objects pointing to this */ unixShm *pFirst; /* All unixShm objects pointing to this */ #ifdef SQLITE_DEBUG u8 exclMask; /* Mask of exclusive locks held */ u8 sharedMask; /* Mask of shared locks held */ u8 nextShmId; /* Next available unixShm.id value */ |
︙ | ︙ | |||
28209 28210 28211 28212 28213 28214 28215 | goto shm_open_err; } if( pInode->bProcessLock==0 ){ pShmNode->h = robust_open(zShmFilename, O_RDWR|O_CREAT, (sStat.st_mode & 0777)); if( pShmNode->h<0 ){ | > > > > > > > > | | > | 28253 28254 28255 28256 28257 28258 28259 28260 28261 28262 28263 28264 28265 28266 28267 28268 28269 28270 28271 28272 28273 28274 28275 28276 28277 | goto shm_open_err; } if( pInode->bProcessLock==0 ){ pShmNode->h = robust_open(zShmFilename, O_RDWR|O_CREAT, (sStat.st_mode & 0777)); if( pShmNode->h<0 ){ const char *zRO; zRO = sqlite3_uri_parameter(pDbFd->zPath, "readonly_shm"); if( zRO && sqlite3GetBoolean(zRO) ){ pShmNode->h = robust_open(zShmFilename, O_RDONLY, (sStat.st_mode & 0777)); pShmNode->isReadonly = 1; } if( pShmNode->h<0 ){ rc = unixLogError(SQLITE_CANTOPEN_BKPT, "open", zShmFilename); goto shm_open_err; } } /* Check to see if another process is holding the dead-man switch. ** If not, truncate the file to zero length. */ rc = SQLITE_OK; if( unixShmSystemLock(pShmNode, F_WRLCK, UNIX_SHM_DMS, 1)==SQLITE_OK ){ |
︙ | ︙ | |||
28349 28350 28351 28352 28353 28354 28355 | rc = SQLITE_IOERR_NOMEM; goto shmpage_out; } pShmNode->apRegion = apNew; while(pShmNode->nRegion<=iRegion){ void *pMem; if( pShmNode->h>=0 ){ | | > | 28402 28403 28404 28405 28406 28407 28408 28409 28410 28411 28412 28413 28414 28415 28416 28417 | rc = SQLITE_IOERR_NOMEM; goto shmpage_out; } pShmNode->apRegion = apNew; while(pShmNode->nRegion<=iRegion){ void *pMem; if( pShmNode->h>=0 ){ pMem = mmap(0, szRegion, pShmNode->isReadonly ? PROT_READ : PROT_READ|PROT_WRITE, MAP_SHARED, pShmNode->h, pShmNode->nRegion*szRegion ); if( pMem==MAP_FAILED ){ rc = unixLogError(SQLITE_IOERR_SHMMAP, "mmap", pShmNode->zFilename); goto shmpage_out; } }else{ |
︙ | ︙ | |||
28375 28376 28377 28378 28379 28380 28381 28382 28383 28384 28385 28386 28387 28388 | shmpage_out: if( pShmNode->nRegion>iRegion ){ *pp = pShmNode->apRegion[iRegion]; }else{ *pp = 0; } sqlite3_mutex_leave(pShmNode->mutex); return rc; } /* ** Change the lock state for a shared-memory segment. ** | > | 28429 28430 28431 28432 28433 28434 28435 28436 28437 28438 28439 28440 28441 28442 28443 | shmpage_out: if( pShmNode->nRegion>iRegion ){ *pp = pShmNode->apRegion[iRegion]; }else{ *pp = 0; } if( pShmNode->isReadonly && rc==SQLITE_OK ) rc = SQLITE_READONLY; sqlite3_mutex_leave(pShmNode->mutex); return rc; } /* ** Change the lock state for a shared-memory segment. ** |
︙ | ︙ | |||
32237 32238 32239 32240 32241 32242 32243 | if( nRem>0 ){ pFile->lastErrno = GetLastError(); rc = 1; } } if( rc ){ | | > | 32292 32293 32294 32295 32296 32297 32298 32299 32300 32301 32302 32303 32304 32305 32306 32307 | if( nRem>0 ){ pFile->lastErrno = GetLastError(); rc = 1; } } if( rc ){ if( ( pFile->lastErrno==ERROR_HANDLE_DISK_FULL ) || ( pFile->lastErrno==ERROR_DISK_FULL )){ return SQLITE_FULL; } return winLogError(SQLITE_IOERR_WRITE, "winWrite", pFile->zPath); } return SQLITE_OK; } |
︙ | ︙ | |||
34872 34873 34874 34875 34876 34877 34878 34879 34880 34881 34882 34883 34884 34885 | ); pCache->pSynced = pPg; if( !pPg ){ for(pPg=pCache->pDirtyTail; pPg && pPg->nRef; pPg=pPg->pDirtyPrev); } if( pPg ){ int rc; rc = pCache->xStress(pCache->pStress, pPg); if( rc!=SQLITE_OK && rc!=SQLITE_BUSY ){ return rc; } } pPage = sqlite3GlobalConfig.pcache.xFetch(pCache->pCache, pgno, 2); | > > > > > > > | 34928 34929 34930 34931 34932 34933 34934 34935 34936 34937 34938 34939 34940 34941 34942 34943 34944 34945 34946 34947 34948 | ); pCache->pSynced = pPg; if( !pPg ){ for(pPg=pCache->pDirtyTail; pPg && pPg->nRef; pPg=pPg->pDirtyPrev); } if( pPg ){ int rc; #ifdef SQLITE_LOG_CACHE_SPILL sqlite3_log(SQLITE_FULL, "spill page %d making room for %d - cache used: %d/%d", pPg->pgno, pgno, sqlite3GlobalConfig.pcache.xPagecount(pCache->pCache), pCache->nMax); #endif rc = pCache->xStress(pCache->pStress, pPg); if( rc!=SQLITE_OK && rc!=SQLITE_BUSY ){ return rc; } } pPage = sqlite3GlobalConfig.pcache.xFetch(pCache->pCache, pgno, 2); |
︙ | ︙ | |||
35782 35783 35784 35785 35786 35787 35788 | pCache = (PCache1 *)sqlite3_malloc(sz); if( pCache ){ memset(pCache, 0, sz); if( separateCache ){ pGroup = (PGroup*)&pCache[1]; pGroup->mxPinned = 10; }else{ | | | 35845 35846 35847 35848 35849 35850 35851 35852 35853 35854 35855 35856 35857 35858 35859 | pCache = (PCache1 *)sqlite3_malloc(sz); if( pCache ){ memset(pCache, 0, sz); if( separateCache ){ pGroup = (PGroup*)&pCache[1]; pGroup->mxPinned = 10; }else{ pGroup = &pcache1.grp; } pCache->pGroup = pGroup; pCache->szPage = szPage; pCache->bPurgeable = (bPurgeable ? 1 : 0); if( bPurgeable ){ pCache->nMin = 10; pcache1EnterMutex(pGroup); |
︙ | ︙ | |||
43976 43977 43978 43979 43980 43981 43982 | int nWiData; /* Size of array apWiData */ volatile u32 **apWiData; /* Pointer to wal-index content in memory */ u32 szPage; /* Database page size */ i16 readLock; /* Which read lock is being held. -1 for none */ u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */ u8 writeLock; /* True if in a write transaction */ u8 ckptLock; /* True if holding a checkpoint lock */ | | > > > > > > > | 44039 44040 44041 44042 44043 44044 44045 44046 44047 44048 44049 44050 44051 44052 44053 44054 44055 44056 44057 44058 44059 44060 44061 44062 44063 44064 44065 44066 44067 44068 44069 44070 44071 44072 44073 44074 44075 | int nWiData; /* Size of array apWiData */ volatile u32 **apWiData; /* Pointer to wal-index content in memory */ u32 szPage; /* Database page size */ i16 readLock; /* Which read lock is being held. -1 for none */ u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */ u8 writeLock; /* True if in a write transaction */ u8 ckptLock; /* True if holding a checkpoint lock */ u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */ WalIndexHdr hdr; /* Wal-index header for current transaction */ const char *zWalName; /* Name of WAL file */ u32 nCkpt; /* Checkpoint sequence counter in the wal-header */ #ifdef SQLITE_DEBUG u8 lockError; /* True if a locking error has occurred */ #endif }; /* ** Candidate values for Wal.exclusiveMode. */ #define WAL_NORMAL_MODE 0 #define WAL_EXCLUSIVE_MODE 1 #define WAL_HEAPMEMORY_MODE 2 /* ** Possible values for WAL.readOnly */ #define WAL_RDWR 0 /* Normal read/write connection */ #define WAL_RDONLY 1 /* The WAL file is readonly */ #define WAL_SHM_RDONLY 2 /* The SHM file is readonly */ /* ** Each page of the wal-index mapping contains a hash-table made up of ** an array of HASHTABLE_NSLOT elements of the following type. */ typedef u16 ht_slot; /* |
︙ | ︙ | |||
44085 44086 44087 44088 44089 44090 44091 44092 44093 44094 44095 44096 44097 44098 | if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){ pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ); if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM; }else{ rc = sqlite3OsShmMap(pWal->pDbFd, iPage, WALINDEX_PGSZ, pWal->writeLock, (void volatile **)&pWal->apWiData[iPage] ); } } *ppPage = pWal->apWiData[iPage]; assert( iPage==0 || *ppPage || rc!=SQLITE_OK ); return rc; } | > > > > | 44155 44156 44157 44158 44159 44160 44161 44162 44163 44164 44165 44166 44167 44168 44169 44170 44171 44172 | if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){ pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ); if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM; }else{ rc = sqlite3OsShmMap(pWal->pDbFd, iPage, WALINDEX_PGSZ, pWal->writeLock, (void volatile **)&pWal->apWiData[iPage] ); if( rc==SQLITE_READONLY ){ pWal->readOnly |= WAL_SHM_RDONLY; rc = SQLITE_OK; } } } *ppPage = pWal->apWiData[iPage]; assert( iPage==0 || *ppPage || rc!=SQLITE_OK ); return rc; } |
︙ | ︙ | |||
44832 44833 44834 44835 44836 44837 44838 | pRet->zWalName = zWalName; pRet->exclusiveMode = (bNoShm ? WAL_HEAPMEMORY_MODE: WAL_NORMAL_MODE); /* Open file handle on the write-ahead log file. */ flags = (SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|SQLITE_OPEN_WAL); rc = sqlite3OsOpen(pVfs, zWalName, pRet->pWalFd, flags, &flags); if( rc==SQLITE_OK && flags&SQLITE_OPEN_READONLY ){ | | | 44906 44907 44908 44909 44910 44911 44912 44913 44914 44915 44916 44917 44918 44919 44920 | pRet->zWalName = zWalName; pRet->exclusiveMode = (bNoShm ? WAL_HEAPMEMORY_MODE: WAL_NORMAL_MODE); /* Open file handle on the write-ahead log file. */ flags = (SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|SQLITE_OPEN_WAL); rc = sqlite3OsOpen(pVfs, zWalName, pRet->pWalFd, flags, &flags); if( rc==SQLITE_OK && flags&SQLITE_OPEN_READONLY ){ pRet->readOnly = WAL_RDONLY; } if( rc!=SQLITE_OK ){ walIndexClose(pRet, 0); sqlite3OsClose(pRet->pWalFd); sqlite3_free(pRet); }else{ |
︙ | ︙ | |||
45473 45474 45475 45476 45477 45478 45479 | */ badHdr = (page0 ? walIndexTryHdr(pWal, pChanged) : 1); /* If the first attempt failed, it might have been due to a race ** with a writer. So get a WRITE lock and try again. */ assert( badHdr==0 || pWal->writeLock==0 ); | > > > > > > | | | | | | | | | | | | | | | > | 45547 45548 45549 45550 45551 45552 45553 45554 45555 45556 45557 45558 45559 45560 45561 45562 45563 45564 45565 45566 45567 45568 45569 45570 45571 45572 45573 45574 45575 45576 45577 45578 45579 45580 45581 45582 | */ badHdr = (page0 ? walIndexTryHdr(pWal, pChanged) : 1); /* If the first attempt failed, it might have been due to a race ** with a writer. So get a WRITE lock and try again. */ assert( badHdr==0 || pWal->writeLock==0 ); if( badHdr ){ if( pWal->readOnly & WAL_SHM_RDONLY ){ if( SQLITE_OK==(rc = walLockShared(pWal, WAL_WRITE_LOCK)) ){ walUnlockShared(pWal, WAL_WRITE_LOCK); rc = SQLITE_READONLY_RECOVERY; } }else if( SQLITE_OK==(rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1)) ){ pWal->writeLock = 1; if( SQLITE_OK==(rc = walIndexPage(pWal, 0, &page0)) ){ badHdr = walIndexTryHdr(pWal, pChanged); if( badHdr ){ /* If the wal-index header is still malformed even while holding ** a WRITE lock, it can only mean that the header is corrupted and ** needs to be reconstructed. So run recovery to do exactly that. */ rc = walIndexRecover(pWal); *pChanged = 1; } } pWal->writeLock = 0; walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1); } } /* If the header is read successfully, check the version number to make ** sure the wal-index was not constructed with some future format that ** this version of SQLite cannot understand. */ if( badHdr==0 && pWal->hdr.iVersion!=WALINDEX_MAX_VERSION ){ |
︙ | ︙ | |||
45674 45675 45676 45677 45678 45679 45680 | assert( thisMark!=READMARK_NOT_USED ); mxReadMark = thisMark; mxI = i; } } /* There was once an "if" here. The extra "{" is to preserve indentation. */ { | > | > | | | 45755 45756 45757 45758 45759 45760 45761 45762 45763 45764 45765 45766 45767 45768 45769 45770 45771 45772 45773 45774 45775 45776 45777 45778 45779 45780 45781 45782 45783 45784 45785 45786 | assert( thisMark!=READMARK_NOT_USED ); mxReadMark = thisMark; mxI = i; } } /* There was once an "if" here. The extra "{" is to preserve indentation. */ { if( (pWal->readOnly & WAL_SHM_RDONLY)==0 && (mxReadMark<pWal->hdr.mxFrame || mxI==0) ){ for(i=1; i<WAL_NREADER; i++){ rc = walLockExclusive(pWal, WAL_READ_LOCK(i), 1); if( rc==SQLITE_OK ){ mxReadMark = pInfo->aReadMark[i] = pWal->hdr.mxFrame; mxI = i; walUnlockExclusive(pWal, WAL_READ_LOCK(i), 1); break; }else if( rc!=SQLITE_BUSY ){ return rc; } } } if( mxI==0 ){ assert( rc==SQLITE_BUSY || (pWal->readOnly & WAL_SHM_RDONLY)!=0 ); return rc==SQLITE_BUSY ? WAL_RETRY : SQLITE_READONLY_CANTLOCK; } rc = walLockShared(pWal, WAL_READ_LOCK(mxI)); if( rc ){ return rc==SQLITE_BUSY ? WAL_RETRY : rc; } /* Now that the read-lock has been obtained, check that neither the |
︙ | ︙ | |||
46331 46332 46333 46334 46335 46336 46337 46338 46339 46340 46341 46342 46343 46344 | int rc; /* Return code */ int isChanged = 0; /* True if a new wal-index header is loaded */ int eMode2 = eMode; /* Mode to pass to walCheckpoint() */ assert( pWal->ckptLock==0 ); assert( pWal->writeLock==0 ); WALTRACE(("WAL%p: checkpoint begins\n", pWal)); rc = walLockExclusive(pWal, WAL_CKPT_LOCK, 1); if( rc ){ /* Usually this is SQLITE_BUSY meaning that another thread or process ** is already running a checkpoint, or maybe a recovery. But it might ** also be SQLITE_IOERR. */ return rc; | > | 46414 46415 46416 46417 46418 46419 46420 46421 46422 46423 46424 46425 46426 46427 46428 | int rc; /* Return code */ int isChanged = 0; /* True if a new wal-index header is loaded */ int eMode2 = eMode; /* Mode to pass to walCheckpoint() */ assert( pWal->ckptLock==0 ); assert( pWal->writeLock==0 ); if( pWal->readOnly ) return SQLITE_READONLY; WALTRACE(("WAL%p: checkpoint begins\n", pWal)); rc = walLockExclusive(pWal, WAL_CKPT_LOCK, 1); if( rc ){ /* Usually this is SQLITE_BUSY meaning that another thread or process ** is already running a checkpoint, or maybe a recovery. But it might ** also be SQLITE_IOERR. */ return rc; |
︙ | ︙ | |||
48280 48281 48282 48283 48284 48285 48286 48287 48288 48289 48290 48291 48292 48293 | ** the page, 1 means the second cell, and so forth) return a pointer ** to the cell content. ** ** This routine works only for pages that do not contain overflow cells. */ #define findCell(P,I) \ ((P)->aData + ((P)->maskPage & get2byte(&(P)->aData[(P)->cellOffset+2*(I)]))) /* ** This a more complex version of findCell() that works for ** pages that do contain overflow cells. */ static u8 *findOverflowCell(MemPage *pPage, int iCell){ int i; | > > | 48364 48365 48366 48367 48368 48369 48370 48371 48372 48373 48374 48375 48376 48377 48378 48379 | ** the page, 1 means the second cell, and so forth) return a pointer ** to the cell content. ** ** This routine works only for pages that do not contain overflow cells. */ #define findCell(P,I) \ ((P)->aData + ((P)->maskPage & get2byte(&(P)->aData[(P)->cellOffset+2*(I)]))) #define findCellv2(D,M,O,I) (D+(M&get2byte(D+(O+2*(I))))) /* ** This a more complex version of findCell() that works for ** pages that do contain overflow cells. */ static u8 *findOverflowCell(MemPage *pPage, int iCell){ int i; |
︙ | ︙ | |||
51874 51875 51876 51877 51878 51879 51880 | if( pCur->eState==CURSOR_INVALID ){ *pRes = -1; assert( pCur->apPage[pCur->iPage]->nCell==0 ); return SQLITE_OK; } assert( pCur->apPage[0]->intKey || pIdxKey ); for(;;){ | | | | < > | 51960 51961 51962 51963 51964 51965 51966 51967 51968 51969 51970 51971 51972 51973 51974 51975 51976 51977 51978 51979 51980 51981 51982 51983 51984 51985 51986 51987 51988 51989 51990 51991 51992 51993 51994 51995 51996 51997 | if( pCur->eState==CURSOR_INVALID ){ *pRes = -1; assert( pCur->apPage[pCur->iPage]->nCell==0 ); return SQLITE_OK; } assert( pCur->apPage[0]->intKey || pIdxKey ); for(;;){ int lwr, upr, idx; Pgno chldPg; MemPage *pPage = pCur->apPage[pCur->iPage]; int c; /* pPage->nCell must be greater than zero. If this is the root-page ** the cursor would have been INVALID above and this for(;;) loop ** not run. If this is not the root-page, then the moveToChild() routine ** would have already detected db corruption. Similarly, pPage must ** be the right kind (index or table) of b-tree page. Otherwise ** a moveToChild() or moveToRoot() call would have detected corruption. */ assert( pPage->nCell>0 ); assert( pPage->intKey==(pIdxKey==0) ); lwr = 0; upr = pPage->nCell-1; if( biasRight ){ pCur->aiIdx[pCur->iPage] = (u16)(idx = upr); }else{ pCur->aiIdx[pCur->iPage] = (u16)(idx = (upr+lwr)/2); } for(;;){ u8 *pCell; /* Pointer to current cell in pPage */ assert( idx==pCur->aiIdx[pCur->iPage] ); pCur->info.nSize = 0; pCell = findCell(pPage, idx) + pPage->childPtrSize; if( pPage->intKey ){ i64 nCellKey; if( pPage->hasData ){ u32 dummy; pCell += getVarint32(pCell, dummy); |
︙ | ︙ | |||
51980 51981 51982 51983 51984 51985 51986 | lwr = idx+1; }else{ upr = idx-1; } if( lwr>upr ){ break; } | | | 52066 52067 52068 52069 52070 52071 52072 52073 52074 52075 52076 52077 52078 52079 52080 | lwr = idx+1; }else{ upr = idx-1; } if( lwr>upr ){ break; } pCur->aiIdx[pCur->iPage] = (u16)(idx = (lwr+upr)/2); } assert( lwr==upr+1 ); assert( pPage->isInit ); if( pPage->leaf ){ chldPg = 0; }else if( lwr>=pPage->nCell ){ chldPg = get4byte(&pPage->aData[pPage->hdrOffset+8]); |
︙ | ︙ | |||
52813 52814 52815 52816 52817 52818 52819 | ** The cell content is not freed or deallocated. It is assumed that ** the cell content has been copied someplace else. This routine just ** removes the reference to the cell from pPage. ** ** "sz" must be the number of bytes in the cell. */ static void dropCell(MemPage *pPage, int idx, int sz, int *pRC){ | < > | 52899 52900 52901 52902 52903 52904 52905 52906 52907 52908 52909 52910 52911 52912 52913 52914 52915 52916 | ** The cell content is not freed or deallocated. It is assumed that ** the cell content has been copied someplace else. This routine just ** removes the reference to the cell from pPage. ** ** "sz" must be the number of bytes in the cell. */ static void dropCell(MemPage *pPage, int idx, int sz, int *pRC){ u32 pc; /* Offset to cell content of cell being deleted */ u8 *data; /* pPage->aData */ u8 *ptr; /* Used to move bytes around within data[] */ u8 *endPtr; /* End of loop */ int rc; /* The return code */ int hdr; /* Beginning of the header. 0 most pages. 100 page 1 */ if( *pRC ) return; assert( idx>=0 && idx<pPage->nCell ); assert( sz==cellSize(pPage, idx) ); |
︙ | ︙ | |||
52841 52842 52843 52844 52845 52846 52847 | return; } rc = freeSpace(pPage, pc, sz); if( rc ){ *pRC = rc; return; } | | > | > | | 52927 52928 52929 52930 52931 52932 52933 52934 52935 52936 52937 52938 52939 52940 52941 52942 52943 52944 52945 | return; } rc = freeSpace(pPage, pc, sz); if( rc ){ *pRC = rc; return; } endPtr = &data[pPage->cellOffset + 2*pPage->nCell - 2]; assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */ while( ptr<endPtr ){ *(u16*)ptr = *(u16*)&ptr[2]; ptr += 2; } pPage->nCell--; put2byte(&data[hdr+3], pPage->nCell); pPage->nFree += 2; } /* |
︙ | ︙ | |||
52883 52884 52885 52886 52887 52888 52889 52890 52891 52892 52893 52894 52895 52896 | int idx = 0; /* Where to write new cell content in data[] */ int j; /* Loop counter */ int end; /* First byte past the last cell pointer in data[] */ int ins; /* Index in data[] where new cell pointer is inserted */ int cellOffset; /* Address of first cell pointer in data[] */ u8 *data; /* The content of the whole page */ u8 *ptr; /* Used for moving information around in data[] */ int nSkip = (iChild ? 4 : 0); if( *pRC ) return; assert( i>=0 && i<=pPage->nCell+pPage->nOverflow ); assert( pPage->nCell<=MX_CELL(pPage->pBt) && MX_CELL(pPage->pBt)<=10921 ); | > | 52971 52972 52973 52974 52975 52976 52977 52978 52979 52980 52981 52982 52983 52984 52985 | int idx = 0; /* Where to write new cell content in data[] */ int j; /* Loop counter */ int end; /* First byte past the last cell pointer in data[] */ int ins; /* Index in data[] where new cell pointer is inserted */ int cellOffset; /* Address of first cell pointer in data[] */ u8 *data; /* The content of the whole page */ u8 *ptr; /* Used for moving information around in data[] */ u8 *endPtr; /* End of the loop */ int nSkip = (iChild ? 4 : 0); if( *pRC ) return; assert( i>=0 && i<=pPage->nCell+pPage->nOverflow ); assert( pPage->nCell<=MX_CELL(pPage->pBt) && MX_CELL(pPage->pBt)<=10921 ); |
︙ | ︙ | |||
52933 52934 52935 52936 52937 52938 52939 | assert( idx+sz <= (int)pPage->pBt->usableSize ); pPage->nCell++; pPage->nFree -= (u16)(2 + sz); memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip); if( iChild ){ put4byte(&data[idx], iChild); } | | > > | > | | 53022 53023 53024 53025 53026 53027 53028 53029 53030 53031 53032 53033 53034 53035 53036 53037 53038 53039 53040 53041 | assert( idx+sz <= (int)pPage->pBt->usableSize ); pPage->nCell++; pPage->nFree -= (u16)(2 + sz); memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip); if( iChild ){ put4byte(&data[idx], iChild); } ptr = &data[end]; endPtr = &data[ins]; assert( (SQLITE_PTR_TO_INT(ptr)&1)==0 ); /* ptr is always 2-byte aligned */ while( ptr>endPtr ){ *(u16*)ptr = *(u16*)&ptr[-2]; ptr -= 2; } put2byte(&data[ins], idx); put2byte(&data[pPage->hdrOffset+3], pPage->nCell); #ifndef SQLITE_OMIT_AUTOVACUUM if( pPage->pBt->autoVacuum ){ /* The cell may contain a pointer to an overflow page. If so, write ** the entry for the overflow page into the pointer map. |
︙ | ︙ | |||
52980 52981 52982 52983 52984 52985 52986 52987 | /* Check that the page has just been zeroed by zeroPage() */ assert( pPage->nCell==0 ); assert( get2byteNotZero(&data[hdr+5])==nUsable ); pCellptr = &data[pPage->cellOffset + nCell*2]; cellbody = nUsable; for(i=nCell-1; i>=0; i--){ pCellptr -= 2; | > | | | 53072 53073 53074 53075 53076 53077 53078 53079 53080 53081 53082 53083 53084 53085 53086 53087 53088 53089 53090 | /* Check that the page has just been zeroed by zeroPage() */ assert( pPage->nCell==0 ); assert( get2byteNotZero(&data[hdr+5])==nUsable ); pCellptr = &data[pPage->cellOffset + nCell*2]; cellbody = nUsable; for(i=nCell-1; i>=0; i--){ u16 sz = aSize[i]; pCellptr -= 2; cellbody -= sz; put2byte(pCellptr, cellbody); memcpy(&data[cellbody], apCell[i], sz); } put2byte(&data[hdr+3], nCell); put2byte(&data[hdr+5], cellbody); pPage->nFree -= (nCell*2 + nUsable - cellbody); pPage->nCell = (u16)nCell; } |
︙ | ︙ | |||
53437 53438 53439 53440 53441 53442 53443 | ** process of being overwritten. */ MemPage *pOld = apCopy[i] = (MemPage*)&aSpace1[pBt->pageSize + k*i]; memcpy(pOld, apOld[i], sizeof(MemPage)); pOld->aData = (void*)&pOld[1]; memcpy(pOld->aData, apOld[i]->aData, pBt->pageSize); limit = pOld->nCell+pOld->nOverflow; | > | | | | | | > > > > > > > > > > > | 53530 53531 53532 53533 53534 53535 53536 53537 53538 53539 53540 53541 53542 53543 53544 53545 53546 53547 53548 53549 53550 53551 53552 53553 53554 53555 53556 53557 53558 53559 53560 53561 | ** process of being overwritten. */ MemPage *pOld = apCopy[i] = (MemPage*)&aSpace1[pBt->pageSize + k*i]; memcpy(pOld, apOld[i], sizeof(MemPage)); pOld->aData = (void*)&pOld[1]; memcpy(pOld->aData, apOld[i]->aData, pBt->pageSize); limit = pOld->nCell+pOld->nOverflow; if( pOld->nOverflow>0 ){ for(j=0; j<limit; j++){ assert( nCell<nMaxCells ); apCell[nCell] = findOverflowCell(pOld, j); szCell[nCell] = cellSizePtr(pOld, apCell[nCell]); nCell++; } }else{ u8 *aData = pOld->aData; u16 maskPage = pOld->maskPage; u16 cellOffset = pOld->cellOffset; for(j=0; j<limit; j++){ assert( nCell<nMaxCells ); apCell[nCell] = findCellv2(aData, maskPage, cellOffset, j); szCell[nCell] = cellSizePtr(pOld, apCell[nCell]); nCell++; } } if( i<nOld-1 && !leafData){ u16 sz = (u16)szNew[i]; u8 *pTemp; assert( nCell<nMaxCells ); szCell[nCell] = sz; pTemp = &aSpace1[iSpace1]; iSpace1 += sz; |
︙ | ︙ | |||
57591 57592 57593 57594 57595 57596 57597 | pOp->opcode = (u8)op; pOp->p5 = 0; pOp->p1 = p1; pOp->p2 = p2; pOp->p3 = p3; pOp->p4.p = 0; pOp->p4type = P4_NOTUSED; | < < < < < < < | 57696 57697 57698 57699 57700 57701 57702 57703 57704 57705 57706 57707 57708 57709 | pOp->opcode = (u8)op; pOp->p5 = 0; pOp->p1 = p1; pOp->p2 = p2; pOp->p3 = p3; pOp->p4.p = 0; pOp->p4type = P4_NOTUSED; #ifdef SQLITE_DEBUG pOp->zComment = 0; if( sqlite3VdbeAddopTrace ) sqlite3VdbePrintOp(0, i, &p->aOp[i]); #endif #ifdef VDBE_PROFILE pOp->cycles = 0; pOp->cnt = 0; |
︙ | ︙ | |||
57635 57636 57637 57638 57639 57640 57641 57642 57643 57644 57645 57646 57647 57648 | const char *zP4, /* The P4 operand */ int p4type /* P4 operand type */ ){ int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3); sqlite3VdbeChangeP4(p, addr, zP4, p4type); return addr; } /* ** Add an opcode that includes the p4 value as an integer. */ SQLITE_PRIVATE int sqlite3VdbeAddOp4Int( Vdbe *p, /* Add the opcode to this VM */ int op, /* The new opcode */ | > > > > > > > > > > > > > > | 57733 57734 57735 57736 57737 57738 57739 57740 57741 57742 57743 57744 57745 57746 57747 57748 57749 57750 57751 57752 57753 57754 57755 57756 57757 57758 57759 57760 | const char *zP4, /* The P4 operand */ int p4type /* P4 operand type */ ){ int addr = sqlite3VdbeAddOp3(p, op, p1, p2, p3); sqlite3VdbeChangeP4(p, addr, zP4, p4type); return addr; } /* ** Add an OP_ParseSchema opcode. This routine is broken out from ** sqlite3VdbeAddOp4() since it needs to also local all btrees. ** ** The zWhere string must have been obtained from sqlite3_malloc(). ** This routine will take ownership of the allocated memory. */ SQLITE_PRIVATE void sqlite3VdbeAddParseSchemaOp(Vdbe *p, int iDb, char *zWhere){ int j; int addr = sqlite3VdbeAddOp3(p, OP_ParseSchema, iDb, 0, 0); sqlite3VdbeChangeP4(p, addr, zWhere, P4_DYNAMIC); for(j=0; j<p->db->nDb; j++) sqlite3VdbeUsesBtree(p, j); } /* ** Add an opcode that includes the p4 value as an integer. */ SQLITE_PRIVATE int sqlite3VdbeAddOp4Int( Vdbe *p, /* Add the opcode to this VM */ int op, /* The new opcode */ |
︙ | ︙ | |||
58826 58827 58828 58829 58830 58831 58832 | }else{ *pnByte += nByte; } return pBuf; } /* | < < | < | < < < < < < < < < < | < < < < < < < < > | < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < < < | | < | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > > > > > | | | | | | | | < < < < < < < < < < | < < < < < < < < | < < < < < < | 58938 58939 58940 58941 58942 58943 58944 58945 58946 58947 58948 58949 58950 58951 58952 58953 58954 58955 58956 58957 58958 58959 58960 58961 58962 58963 58964 58965 58966 58967 58968 58969 58970 58971 58972 58973 58974 58975 58976 58977 58978 58979 58980 58981 58982 58983 58984 58985 58986 58987 58988 58989 58990 58991 58992 58993 58994 58995 58996 58997 58998 58999 59000 59001 59002 59003 59004 59005 59006 59007 59008 59009 59010 59011 59012 59013 59014 59015 59016 59017 59018 59019 59020 59021 59022 59023 59024 59025 59026 59027 59028 59029 59030 59031 59032 59033 59034 59035 59036 59037 59038 59039 59040 59041 59042 59043 59044 59045 59046 59047 59048 59049 59050 59051 59052 59053 59054 59055 59056 59057 59058 59059 59060 59061 59062 59063 59064 59065 59066 59067 59068 59069 59070 59071 59072 59073 59074 59075 59076 59077 59078 59079 59080 59081 59082 59083 59084 59085 59086 59087 59088 59089 59090 59091 59092 59093 59094 59095 59096 59097 59098 59099 59100 59101 59102 59103 59104 59105 59106 59107 | }else{ *pnByte += nByte; } return pBuf; } /* ** Rewind the VDBE back to the beginning in preparation for ** running it. */ SQLITE_PRIVATE void sqlite3VdbeRewind(Vdbe *p){ #if defined(SQLITE_DEBUG) || defined(VDBE_PROFILE) int i; #endif assert( p!=0 ); assert( p->magic==VDBE_MAGIC_INIT ); /* There should be at least one opcode. */ assert( p->nOp>0 ); /* Set the magic to VDBE_MAGIC_RUN sooner rather than later. */ p->magic = VDBE_MAGIC_RUN; #ifdef SQLITE_DEBUG for(i=1; i<p->nMem; i++){ assert( p->aMem[i].db==p->db ); } #endif p->pc = -1; p->rc = SQLITE_OK; p->errorAction = OE_Abort; p->magic = VDBE_MAGIC_RUN; p->nChange = 0; p->cacheCtr = 1; p->minWriteFileFormat = 255; p->iStatement = 0; p->nFkConstraint = 0; #ifdef VDBE_PROFILE for(i=0; i<p->nOp; i++){ p->aOp[i].cnt = 0; p->aOp[i].cycles = 0; } #endif } /* ** Prepare a virtual machine for execution for the first time after ** creating the virtual machine. This involves things such ** as allocating stack space and initializing the program counter. ** After the VDBE has be prepped, it can be executed by one or more ** calls to sqlite3VdbeExec(). ** ** This function may be called exact once on a each virtual machine. ** After this routine is called the VM has been "packaged" and is ready ** to run. After this routine is called, futher calls to ** sqlite3VdbeAddOp() functions are prohibited. This routine disconnects ** the Vdbe from the Parse object that helped generate it so that the ** the Vdbe becomes an independent entity and the Parse object can be ** destroyed. ** ** Use the sqlite3VdbeRewind() procedure to restore a virtual machine back ** to its initial state after it has been run. */ SQLITE_PRIVATE void sqlite3VdbeMakeReady( Vdbe *p, /* The VDBE */ Parse *pParse /* Parsing context */ ){ sqlite3 *db; /* The database connection */ int nVar; /* Number of parameters */ int nMem; /* Number of VM memory registers */ int nCursor; /* Number of cursors required */ int nArg; /* Number of arguments in subprograms */ int n; /* Loop counter */ u8 *zCsr; /* Memory available for allocation */ u8 *zEnd; /* First byte past allocated memory */ int nByte; /* How much extra memory is needed */ assert( p!=0 ); assert( p->nOp>0 ); assert( pParse!=0 ); assert( p->magic==VDBE_MAGIC_INIT ); db = p->db; assert( db->mallocFailed==0 ); nVar = pParse->nVar; nMem = pParse->nMem; nCursor = pParse->nTab; nArg = pParse->nMaxArg; /* For each cursor required, also allocate a memory cell. Memory ** cells (nMem+1-nCursor)..nMem, inclusive, will never be used by ** the vdbe program. Instead they are used to allocate space for ** VdbeCursor/BtCursor structures. The blob of memory associated with ** cursor 0 is stored in memory cell nMem. Memory cell (nMem-1) ** stores the blob of memory associated with cursor 1, etc. ** ** See also: allocateCursor(). */ nMem += nCursor; /* Allocate space for memory registers, SQL variables, VDBE cursors and ** an array to marshal SQL function arguments in. */ zCsr = (u8*)&p->aOp[p->nOp]; /* Memory avaliable for allocation */ zEnd = (u8*)&p->aOp[p->nOpAlloc]; /* First byte past end of zCsr[] */ resolveP2Values(p, &nArg); p->usesStmtJournal = (u8)(pParse->isMultiWrite && pParse->mayAbort); if( pParse->explain && nMem<10 ){ nMem = 10; } memset(zCsr, 0, zEnd-zCsr); zCsr += (zCsr - (u8*)0)&7; assert( EIGHT_BYTE_ALIGNMENT(zCsr) ); /* Memory for registers, parameters, cursor, etc, is allocated in two ** passes. On the first pass, we try to reuse unused space at the ** end of the opcode array. If we are unable to satisfy all memory ** requirements by reusing the opcode array tail, then the second ** pass will fill in the rest using a fresh allocation. ** ** This two-pass approach that reuses as much memory as possible from ** the leftover space at the end of the opcode array can significantly ** reduce the amount of memory held by a prepared statement. */ do { nByte = 0; p->aMem = allocSpace(p->aMem, nMem*sizeof(Mem), &zCsr, zEnd, &nByte); p->aVar = allocSpace(p->aVar, nVar*sizeof(Mem), &zCsr, zEnd, &nByte); p->apArg = allocSpace(p->apArg, nArg*sizeof(Mem*), &zCsr, zEnd, &nByte); p->azVar = allocSpace(p->azVar, nVar*sizeof(char*), &zCsr, zEnd, &nByte); p->apCsr = allocSpace(p->apCsr, nCursor*sizeof(VdbeCursor*), &zCsr, zEnd, &nByte); if( nByte ){ p->pFree = sqlite3DbMallocZero(db, nByte); } zCsr = p->pFree; zEnd = &zCsr[nByte]; }while( nByte && !db->mallocFailed ); p->nCursor = (u16)nCursor; if( p->aVar ){ p->nVar = (ynVar)nVar; for(n=0; n<nVar; n++){ p->aVar[n].flags = MEM_Null; p->aVar[n].db = db; } } if( p->azVar ){ p->nzVar = pParse->nzVar; memcpy(p->azVar, pParse->azVar, p->nzVar*sizeof(p->azVar[0])); memset(pParse->azVar, 0, pParse->nzVar*sizeof(pParse->azVar[0])); } if( p->aMem ){ p->aMem--; /* aMem[] goes from 1..nMem */ p->nMem = nMem; /* not from 0..nMem-1 */ for(n=1; n<=nMem; n++){ p->aMem[n].flags = MEM_Null; p->aMem[n].db = db; } } p->explain = pParse->explain; sqlite3VdbeRewind(p); } /* ** Close a VDBE cursor and release all the resources that cursor ** happens to hold. */ SQLITE_PRIVATE void sqlite3VdbeFreeCursor(Vdbe *p, VdbeCursor *pCx){ |
︙ | ︙ | |||
59834 59835 59836 59837 59838 59839 59840 59841 59842 59843 59844 59845 59846 59847 59848 59849 59850 59851 59852 59853 59854 59855 | ** Free all memory associated with the Vdbe passed as the second argument. ** The difference between this function and sqlite3VdbeDelete() is that ** VdbeDelete() also unlinks the Vdbe from the list of VMs associated with ** the database connection. */ SQLITE_PRIVATE void sqlite3VdbeDeleteObject(sqlite3 *db, Vdbe *p){ SubProgram *pSub, *pNext; assert( p->db==0 || p->db==db ); releaseMemArray(p->aVar, p->nVar); releaseMemArray(p->aColName, p->nResColumn*COLNAME_N); for(pSub=p->pProgram; pSub; pSub=pNext){ pNext = pSub->pNext; vdbeFreeOpArray(db, pSub->aOp, pSub->nOp); sqlite3DbFree(db, pSub); } vdbeFreeOpArray(db, p->aOp, p->nOp); sqlite3DbFree(db, p->aLabel); sqlite3DbFree(db, p->aColName); sqlite3DbFree(db, p->zSql); sqlite3DbFree(db, p->pFree); sqlite3DbFree(db, p); } | > > | 59967 59968 59969 59970 59971 59972 59973 59974 59975 59976 59977 59978 59979 59980 59981 59982 59983 59984 59985 59986 59987 59988 59989 59990 | ** Free all memory associated with the Vdbe passed as the second argument. ** The difference between this function and sqlite3VdbeDelete() is that ** VdbeDelete() also unlinks the Vdbe from the list of VMs associated with ** the database connection. */ SQLITE_PRIVATE void sqlite3VdbeDeleteObject(sqlite3 *db, Vdbe *p){ SubProgram *pSub, *pNext; int i; assert( p->db==0 || p->db==db ); releaseMemArray(p->aVar, p->nVar); releaseMemArray(p->aColName, p->nResColumn*COLNAME_N); for(pSub=p->pProgram; pSub; pSub=pNext){ pNext = pSub->pNext; vdbeFreeOpArray(db, pSub->aOp, pSub->nOp); sqlite3DbFree(db, pSub); } for(i=p->nzVar-1; i>=0; i--) sqlite3DbFree(db, p->azVar[i]); vdbeFreeOpArray(db, p->aOp, p->nOp); sqlite3DbFree(db, p->aLabel); sqlite3DbFree(db, p->aColName); sqlite3DbFree(db, p->zSql); sqlite3DbFree(db, p->pFree); sqlite3DbFree(db, p); } |
︙ | ︙ | |||
60287 60288 60289 60290 60291 60292 60293 | u = 0; while( idx<szHdr && u<p->nField && d<=nKey ){ u32 serial_type; idx += getVarint32(&aKey[idx], serial_type); pMem->enc = pKeyInfo->enc; pMem->db = pKeyInfo->db; | | > > | 60422 60423 60424 60425 60426 60427 60428 60429 60430 60431 60432 60433 60434 60435 60436 60437 60438 60439 60440 60441 60442 60443 60444 60445 60446 60447 60448 60449 60450 60451 60452 60453 60454 60455 60456 60457 60458 60459 60460 60461 60462 60463 60464 60465 | u = 0; while( idx<szHdr && u<p->nField && d<=nKey ){ u32 serial_type; idx += getVarint32(&aKey[idx], serial_type); pMem->enc = pKeyInfo->enc; pMem->db = pKeyInfo->db; /* pMem->flags = 0; // sqlite3VdbeSerialGet() will set this for us */ pMem->zMalloc = 0; d += sqlite3VdbeSerialGet(&aKey[d], serial_type, pMem); pMem++; u++; } assert( u<=pKeyInfo->nField + 1 ); p->nField = u; return (void*)p; } /* ** This routine destroys a UnpackedRecord object. */ SQLITE_PRIVATE void sqlite3VdbeDeleteUnpackedRecord(UnpackedRecord *p){ #ifdef SQLITE_DEBUG int i; Mem *pMem; assert( p!=0 ); assert( p->flags & UNPACKED_NEED_DESTROY ); for(i=0, pMem=p->aMem; i<p->nField; i++, pMem++){ /* The unpacked record is always constructed by the ** sqlite3VdbeUnpackRecord() function above, which makes all ** strings and blobs static. And none of the elements are ** ever transformed, so there is never anything to delete. */ if( NEVER(pMem->zMalloc) ) sqlite3VdbeMemRelease(pMem); } #endif if( p->flags & UNPACKED_NEED_FREE ){ sqlite3DbFree(p->pKeyInfo->db, p); } } /* ** This function compares the two table rows or index records |
︙ | ︙ | |||
60750 60751 60752 60753 60754 60755 60756 | int rc; if( pStmt==0 ){ rc = SQLITE_OK; }else{ Vdbe *v = (Vdbe*)pStmt; sqlite3_mutex_enter(v->db->mutex); rc = sqlite3VdbeReset(v); | | | 60887 60888 60889 60890 60891 60892 60893 60894 60895 60896 60897 60898 60899 60900 60901 | int rc; if( pStmt==0 ){ rc = SQLITE_OK; }else{ Vdbe *v = (Vdbe*)pStmt; sqlite3_mutex_enter(v->db->mutex); rc = sqlite3VdbeReset(v); sqlite3VdbeRewind(v); assert( (rc & (v->db->errMask))==rc ); rc = sqlite3ApiExit(v->db, rc); sqlite3_mutex_leave(v->db->mutex); } return rc; } |
︙ | ︙ | |||
61107 61108 61109 61110 61111 61112 61113 61114 61115 61116 61117 61118 61119 61120 61121 61122 61123 61124 61125 61126 61127 61128 61129 61130 61131 | ** caller. Set the error code in the database handle to the same value. */ rc = db->errCode = p->rc; } return (rc&db->errMask); } /* ** This is the top-level implementation of sqlite3_step(). Call ** sqlite3Step() to do most of the work. If a schema error occurs, ** call sqlite3Reprepare() and try again. */ SQLITE_API int sqlite3_step(sqlite3_stmt *pStmt){ int rc = SQLITE_OK; /* Result from sqlite3Step() */ int rc2 = SQLITE_OK; /* Result from sqlite3Reprepare() */ Vdbe *v = (Vdbe*)pStmt; /* the prepared statement */ int cnt = 0; /* Counter to prevent infinite loop of reprepares */ sqlite3 *db; /* The database connection */ if( vdbeSafetyNotNull(v) ){ return SQLITE_MISUSE_BKPT; } db = v->db; sqlite3_mutex_enter(db->mutex); while( (rc = sqlite3Step(v))==SQLITE_SCHEMA | > > > > > > > > | | 61244 61245 61246 61247 61248 61249 61250 61251 61252 61253 61254 61255 61256 61257 61258 61259 61260 61261 61262 61263 61264 61265 61266 61267 61268 61269 61270 61271 61272 61273 61274 61275 61276 61277 61278 61279 61280 61281 61282 61283 61284 | ** caller. Set the error code in the database handle to the same value. */ rc = db->errCode = p->rc; } return (rc&db->errMask); } /* ** The maximum number of times that a statement will try to reparse ** itself before giving up and returning SQLITE_SCHEMA. */ #ifndef SQLITE_MAX_SCHEMA_RETRY # define SQLITE_MAX_SCHEMA_RETRY 5 #endif /* ** This is the top-level implementation of sqlite3_step(). Call ** sqlite3Step() to do most of the work. If a schema error occurs, ** call sqlite3Reprepare() and try again. */ SQLITE_API int sqlite3_step(sqlite3_stmt *pStmt){ int rc = SQLITE_OK; /* Result from sqlite3Step() */ int rc2 = SQLITE_OK; /* Result from sqlite3Reprepare() */ Vdbe *v = (Vdbe*)pStmt; /* the prepared statement */ int cnt = 0; /* Counter to prevent infinite loop of reprepares */ sqlite3 *db; /* The database connection */ if( vdbeSafetyNotNull(v) ){ return SQLITE_MISUSE_BKPT; } db = v->db; sqlite3_mutex_enter(db->mutex); while( (rc = sqlite3Step(v))==SQLITE_SCHEMA && cnt++ < SQLITE_MAX_SCHEMA_RETRY && (rc2 = rc = sqlite3Reprepare(v))==SQLITE_OK ){ sqlite3_reset(pStmt); v->expired = 0; } if( rc2!=SQLITE_OK && ALWAYS(v->isPrepareV2) && ALWAYS(db->pErr) ){ /* This case occurs after failing to recompile an sql statement. ** The error message from the SQL compiler has already been loaded |
︙ | ︙ | |||
61815 61816 61817 61818 61819 61820 61821 | ** This routine is added to support DBD::SQLite. */ SQLITE_API int sqlite3_bind_parameter_count(sqlite3_stmt *pStmt){ Vdbe *p = (Vdbe*)pStmt; return p ? p->nVar : 0; } | < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | | 61960 61961 61962 61963 61964 61965 61966 61967 61968 61969 61970 61971 61972 61973 61974 61975 61976 61977 61978 61979 61980 61981 61982 61983 61984 61985 61986 61987 61988 61989 61990 61991 61992 61993 61994 61995 61996 61997 61998 61999 | ** This routine is added to support DBD::SQLite. */ SQLITE_API int sqlite3_bind_parameter_count(sqlite3_stmt *pStmt){ Vdbe *p = (Vdbe*)pStmt; return p ? p->nVar : 0; } /* ** Return the name of a wildcard parameter. Return NULL if the index ** is out of range or if the wildcard is unnamed. ** ** The result is always UTF-8. */ SQLITE_API const char *sqlite3_bind_parameter_name(sqlite3_stmt *pStmt, int i){ Vdbe *p = (Vdbe*)pStmt; if( p==0 || i<1 || i>p->nzVar ){ return 0; } return p->azVar[i-1]; } /* ** Given a wildcard parameter name, return the index of the variable ** with that name. If there is no variable with the given name, ** return 0. */ SQLITE_PRIVATE int sqlite3VdbeParameterIndex(Vdbe *p, const char *zName, int nName){ int i; if( p==0 ){ return 0; } if( zName ){ for(i=0; i<p->nzVar; i++){ const char *z = p->azVar[i]; if( z && memcmp(z,zName,nName)==0 && z[nName]==0 ){ return i+1; } } } return 0; |
︙ | ︙ | |||
63192 63193 63194 63195 63196 63197 63198 63199 63200 63201 63202 63203 63204 63205 | int i; sqlite_int64 rowid; Mem **apArg; Mem *pX; } cm; struct OP_Trace_stack_vars { char *zTrace; } cn; } u; /* End automatically generated code ********************************************************************/ assert( p->magic==VDBE_MAGIC_RUN ); /* sqlite3_step() verifies this */ sqlite3VdbeEnter(p); | > | 63309 63310 63311 63312 63313 63314 63315 63316 63317 63318 63319 63320 63321 63322 63323 | int i; sqlite_int64 rowid; Mem **apArg; Mem *pX; } cm; struct OP_Trace_stack_vars { char *zTrace; char *z; } cn; } u; /* End automatically generated code ********************************************************************/ assert( p->magic==VDBE_MAGIC_RUN ); /* sqlite3_step() verifies this */ sqlite3VdbeEnter(p); |
︙ | ︙ | |||
63425 63426 63427 63428 63429 63430 63431 | REGISTER_TRACE(pOp->p1, pIn1); pc = u.aa.pcDest; break; } /* Opcode: HaltIfNull P1 P2 P3 P4 * ** | | | 63543 63544 63545 63546 63547 63548 63549 63550 63551 63552 63553 63554 63555 63556 63557 | REGISTER_TRACE(pOp->p1, pIn1); pc = u.aa.pcDest; break; } /* Opcode: HaltIfNull P1 P2 P3 P4 * ** ** Check the value in register P3. If it is NULL then Halt using ** parameter P1, P2, and P4 as if this were a Halt instruction. If the ** value in register P3 is not NULL, then this routine is a no-op. */ case OP_HaltIfNull: { /* in3 */ pIn3 = &aMem[pOp->p3]; if( (pIn3->flags & MEM_Null)==0 ) break; /* Fall through into OP_Halt */ |
︙ | ︙ | |||
63618 63619 63620 63621 63622 63623 63624 63625 63626 63627 63628 63629 63630 63631 | */ case OP_Variable: { /* out2-prerelease */ #if 0 /* local variables moved into u.ab */ Mem *pVar; /* Value being transferred */ #endif /* local variables moved into u.ab */ assert( pOp->p1>0 && pOp->p1<=p->nVar ); u.ab.pVar = &p->aVar[pOp->p1 - 1]; if( sqlite3VdbeMemTooBig(u.ab.pVar) ){ goto too_big; } sqlite3VdbeMemShallowCopy(pOut, u.ab.pVar, MEM_Static); UPDATE_MAX_BLOBSIZE(pOut); break; | > | 63736 63737 63738 63739 63740 63741 63742 63743 63744 63745 63746 63747 63748 63749 63750 | */ case OP_Variable: { /* out2-prerelease */ #if 0 /* local variables moved into u.ab */ Mem *pVar; /* Value being transferred */ #endif /* local variables moved into u.ab */ assert( pOp->p1>0 && pOp->p1<=p->nVar ); assert( pOp->p4.z==0 || pOp->p4.z==p->azVar[pOp->p1-1] ); u.ab.pVar = &p->aVar[pOp->p1 - 1]; if( sqlite3VdbeMemTooBig(u.ab.pVar) ){ goto too_big; } sqlite3VdbeMemShallowCopy(pOut, u.ab.pVar, MEM_Static); UPDATE_MAX_BLOBSIZE(pOut); break; |
︙ | ︙ | |||
64038 64039 64040 64041 64042 64043 64044 64045 64046 64047 64048 64049 64050 64051 64052 64053 64054 | assert( pOp[-1].p4type==P4_COLLSEQ ); assert( pOp[-1].opcode==OP_CollSeq ); u.ag.ctx.pColl = pOp[-1].p4.pColl; } db->lastRowid = lastRowid; (*u.ag.ctx.pFunc->xFunc)(&u.ag.ctx, u.ag.n, u.ag.apVal); /* IMP: R-24505-23230 */ lastRowid = db->lastRowid; if( db->mallocFailed ){ /* Even though a malloc() has failed, the implementation of the ** user function may have called an sqlite3_result_XXX() function ** to return a value. The following call releases any resources ** associated with such a value. */ sqlite3VdbeMemRelease(&u.ag.ctx.s); goto no_mem; } | > > > > > > > > > > < < < < < < < < < | 64157 64158 64159 64160 64161 64162 64163 64164 64165 64166 64167 64168 64169 64170 64171 64172 64173 64174 64175 64176 64177 64178 64179 64180 64181 64182 64183 64184 64185 64186 64187 64188 64189 64190 | assert( pOp[-1].p4type==P4_COLLSEQ ); assert( pOp[-1].opcode==OP_CollSeq ); u.ag.ctx.pColl = pOp[-1].p4.pColl; } db->lastRowid = lastRowid; (*u.ag.ctx.pFunc->xFunc)(&u.ag.ctx, u.ag.n, u.ag.apVal); /* IMP: R-24505-23230 */ lastRowid = db->lastRowid; /* If any auxiliary data functions have been called by this user function, ** immediately call the destructor for any non-static values. */ if( u.ag.ctx.pVdbeFunc ){ sqlite3VdbeDeleteAuxData(u.ag.ctx.pVdbeFunc, pOp->p1); pOp->p4.pVdbeFunc = u.ag.ctx.pVdbeFunc; pOp->p4type = P4_VDBEFUNC; } if( db->mallocFailed ){ /* Even though a malloc() has failed, the implementation of the ** user function may have called an sqlite3_result_XXX() function ** to return a value. The following call releases any resources ** associated with such a value. */ sqlite3VdbeMemRelease(&u.ag.ctx.s); goto no_mem; } /* If the function returned an error, throw an exception */ if( u.ag.ctx.isError ){ sqlite3SetString(&p->zErrMsg, db, "%s", sqlite3_value_text(&u.ag.ctx.s)); rc = u.ag.ctx.isError; } /* Copy the result of the function into register P3 */ |
︙ | ︙ | |||
64360 64361 64362 64363 64364 64365 64366 | ** This works just like the Lt opcode except that the jump is taken if ** the operands in registers P1 and P3 are not equal. See the Lt opcode for ** additional information. ** ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either ** true or false and is never NULL. If both operands are NULL then the result ** of comparison is false. If either operand is NULL then the result is true. | | | | 64480 64481 64482 64483 64484 64485 64486 64487 64488 64489 64490 64491 64492 64493 64494 64495 64496 64497 64498 64499 64500 64501 64502 64503 64504 64505 64506 | ** This works just like the Lt opcode except that the jump is taken if ** the operands in registers P1 and P3 are not equal. See the Lt opcode for ** additional information. ** ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either ** true or false and is never NULL. If both operands are NULL then the result ** of comparison is false. If either operand is NULL then the result is true. ** If neither operand is NULL the result is the same as it would be if ** the SQLITE_NULLEQ flag were omitted from P5. */ /* Opcode: Eq P1 P2 P3 P4 P5 ** ** This works just like the Lt opcode except that the jump is taken if ** the operands in registers P1 and P3 are equal. ** See the Lt opcode for additional information. ** ** If SQLITE_NULLEQ is set in P5 then the result of comparison is always either ** true or false and is never NULL. If both operands are NULL then the result ** of comparison is true. If either operand is NULL then the result is false. ** If neither operand is NULL the result is the same as it would be if ** the SQLITE_NULLEQ flag were omitted from P5. */ /* Opcode: Le P1 P2 P3 P4 P5 ** ** This works just like the Lt opcode except that the jump is taken if ** the content of register P3 is less than or equal to the content of ** register P1. See the Lt opcode for additional information. |
︙ | ︙ | |||
64410 64411 64412 64413 64414 64415 64416 | u16 flags3; /* Copy of initial value of pIn3->flags */ #endif /* local variables moved into u.ai */ pIn1 = &aMem[pOp->p1]; pIn3 = &aMem[pOp->p3]; u.ai.flags1 = pIn1->flags; u.ai.flags3 = pIn3->flags; | | | | 64530 64531 64532 64533 64534 64535 64536 64537 64538 64539 64540 64541 64542 64543 64544 64545 64546 64547 64548 64549 64550 64551 64552 | u16 flags3; /* Copy of initial value of pIn3->flags */ #endif /* local variables moved into u.ai */ pIn1 = &aMem[pOp->p1]; pIn3 = &aMem[pOp->p3]; u.ai.flags1 = pIn1->flags; u.ai.flags3 = pIn3->flags; if( (u.ai.flags1 | u.ai.flags3)&MEM_Null ){ /* One or both operands are NULL */ if( pOp->p5 & SQLITE_NULLEQ ){ /* If SQLITE_NULLEQ is set (which will only happen if the operator is ** OP_Eq or OP_Ne) then take the jump or not depending on whether ** or not both operands are null. */ assert( pOp->opcode==OP_Eq || pOp->opcode==OP_Ne ); u.ai.res = (u.ai.flags1 & u.ai.flags3 & MEM_Null)==0; }else{ /* SQLITE_NULLEQ is clear and at least one operand is NULL, ** then the result is always NULL. ** The jump is taken if the SQLITE_JUMPIFNULL bit is set. */ if( pOp->p5 & SQLITE_STOREP2 ){ pOut = &aMem[pOp->p2]; |
︙ | ︙ | |||
64657 64658 64659 64660 64661 64662 64663 | sqlite3VdbeMemSetInt64(pOut, ~sqlite3VdbeIntValue(pIn1)); } break; } /* Opcode: If P1 P2 P3 * * ** | | | | 64777 64778 64779 64780 64781 64782 64783 64784 64785 64786 64787 64788 64789 64790 64791 64792 64793 64794 64795 64796 64797 | sqlite3VdbeMemSetInt64(pOut, ~sqlite3VdbeIntValue(pIn1)); } break; } /* Opcode: If P1 P2 P3 * * ** ** Jump to P2 if the value in register P1 is true. The value ** is considered true if it is numeric and non-zero. If the value ** in P1 is NULL then take the jump if P3 is true. */ /* Opcode: IfNot P1 P2 P3 * * ** ** Jump to P2 if the value in register P1 is False. The value ** is considered true if it has a numeric value of zero. If the value ** in P1 is NULL then take the jump if P3 is true. */ case OP_If: /* jump, in1 */ case OP_IfNot: { /* jump, in1 */ #if 0 /* local variables moved into u.al */ int c; |
︙ | ︙ | |||
65244 65245 65246 65247 65248 65249 65250 65251 65252 65253 65254 65255 65256 65257 65258 65259 65260 65261 65262 65263 65264 65265 | */ sqlite3SetString(&p->zErrMsg, db, "cannot open savepoint - " "SQL statements in progress"); rc = SQLITE_BUSY; }else{ u.aq.nName = sqlite3Strlen30(u.aq.zName); /* This call is Ok even if this savepoint is actually a transaction ** savepoint (and therefore should not prompt xSavepoint()) callbacks. ** If this is a transaction savepoint being opened, it is guaranteed ** that the db->aVTrans[] array is empty. */ assert( db->autoCommit==0 || db->nVTrans==0 ); rc = sqlite3VtabSavepoint(db, SAVEPOINT_BEGIN, db->nStatement+db->nSavepoint); if( rc!=SQLITE_OK ) goto abort_due_to_error; /* Create a new savepoint structure. */ u.aq.pNew = sqlite3DbMallocRaw(db, sizeof(Savepoint)+u.aq.nName+1); if( u.aq.pNew ){ u.aq.pNew->zName = (char *)&u.aq.pNew[1]; memcpy(u.aq.pNew->zName, u.aq.zName, u.aq.nName+1); | > > | 65364 65365 65366 65367 65368 65369 65370 65371 65372 65373 65374 65375 65376 65377 65378 65379 65380 65381 65382 65383 65384 65385 65386 65387 | */ sqlite3SetString(&p->zErrMsg, db, "cannot open savepoint - " "SQL statements in progress"); rc = SQLITE_BUSY; }else{ u.aq.nName = sqlite3Strlen30(u.aq.zName); #ifndef SQLITE_OMIT_VIRTUALTABLE /* This call is Ok even if this savepoint is actually a transaction ** savepoint (and therefore should not prompt xSavepoint()) callbacks. ** If this is a transaction savepoint being opened, it is guaranteed ** that the db->aVTrans[] array is empty. */ assert( db->autoCommit==0 || db->nVTrans==0 ); rc = sqlite3VtabSavepoint(db, SAVEPOINT_BEGIN, db->nStatement+db->nSavepoint); if( rc!=SQLITE_OK ) goto abort_due_to_error; #endif /* Create a new savepoint structure. */ u.aq.pNew = sqlite3DbMallocRaw(db, sizeof(Savepoint)+u.aq.nName+1); if( u.aq.pNew ){ u.aq.pNew->zName = (char *)&u.aq.pNew[1]; memcpy(u.aq.pNew->zName, u.aq.zName, u.aq.nName+1); |
︙ | ︙ | |||
66315 66316 66317 66318 66319 66320 66321 | } } break; } /* Opcode: NotExists P1 P2 P3 * * ** | | | 66437 66438 66439 66440 66441 66442 66443 66444 66445 66446 66447 66448 66449 66450 66451 | } } break; } /* Opcode: NotExists P1 P2 P3 * * ** ** Use the content of register P3 as an integer key. If a record ** with that key does not exist in table of P1, then jump to P2. ** If the record does exist, then fall through. The cursor is left ** pointing to the record if it exists. ** ** The difference between this operation and NotFound is that this ** operation assumes the key is an integer and that P1 is a table whereas ** NotFound assumes key is a blob constructed from MakeRecord and |
︙ | ︙ | |||
66393 66394 66395 66396 66397 66398 66399 | ** The record number is not previously used as a key in the database ** table that cursor P1 points to. The new record number is written ** written to register P2. ** ** If P3>0 then P3 is a register in the root frame of this VDBE that holds ** the largest previously generated record number. No new record numbers are ** allowed to be less than this value. When this value reaches its maximum, | | | 66515 66516 66517 66518 66519 66520 66521 66522 66523 66524 66525 66526 66527 66528 66529 | ** The record number is not previously used as a key in the database ** table that cursor P1 points to. The new record number is written ** written to register P2. ** ** If P3>0 then P3 is a register in the root frame of this VDBE that holds ** the largest previously generated record number. No new record numbers are ** allowed to be less than this value. When this value reaches its maximum, ** an SQLITE_FULL error is generated. The P3 register is updated with the ' ** generated record number. This P3 mechanism is used to help implement the ** AUTOINCREMENT feature. */ case OP_NewRowid: { /* out2-prerelease */ #if 0 /* local variables moved into u.be */ i64 v; /* The new rowid */ VdbeCursor *pC; /* Cursor of table to get the new rowid */ |
︙ | ︙ | |||
67035 67036 67037 67038 67039 67040 67041 | } u.bm.pC->rowidIsValid = 0; break; } /* Opcode: IdxInsert P1 P2 P3 * P5 ** | | | 67157 67158 67159 67160 67161 67162 67163 67164 67165 67166 67167 67168 67169 67170 67171 | } u.bm.pC->rowidIsValid = 0; break; } /* Opcode: IdxInsert P1 P2 P3 * P5 ** ** Register P2 holds an SQL index key made using the ** MakeRecord instructions. This opcode writes that key ** into the index P1. Data for the entry is nil. ** ** P3 is a flag that provides a hint to the b-tree layer that this ** insert is likely to be an append. ** ** This instruction only works for indices. The equivalent instruction |
︙ | ︙ | |||
68637 68638 68639 68640 68641 68642 68643 68644 68645 | ** ** If tracing is enabled (by the sqlite3_trace()) interface, then ** the UTF-8 string contained in P4 is emitted on the trace callback. */ case OP_Trace: { #if 0 /* local variables moved into u.cn */ char *zTrace; #endif /* local variables moved into u.cn */ | > | < < | | | | | > > | | < | 68759 68760 68761 68762 68763 68764 68765 68766 68767 68768 68769 68770 68771 68772 68773 68774 68775 68776 68777 68778 68779 68780 68781 68782 68783 68784 68785 68786 68787 | ** ** If tracing is enabled (by the sqlite3_trace()) interface, then ** the UTF-8 string contained in P4 is emitted on the trace callback. */ case OP_Trace: { #if 0 /* local variables moved into u.cn */ char *zTrace; char *z; #endif /* local variables moved into u.cn */ if( db->xTrace && (u.cn.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql))!=0 ){ u.cn.z = sqlite3VdbeExpandSql(p, u.cn.zTrace); db->xTrace(db->pTraceArg, u.cn.z); sqlite3DbFree(db, u.cn.z); } #ifdef SQLITE_DEBUG if( (db->flags & SQLITE_SqlTrace)!=0 && (u.cn.zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql))!=0 ){ sqlite3DebugPrintf("SQL-trace: %s\n", u.cn.zTrace); } #endif /* SQLITE_DEBUG */ break; } #endif /* Opcode: Noop * * * * * ** |
︙ | ︙ | |||
69076 69077 69078 69079 69080 69081 69082 | ** always return an SQL NULL. This is useful because it means ** we can invoke OP_Column to fill in the vdbe cursors type ** and offset cache without causing any IO. */ sqlite3VdbeChangeP4(v, 3+flags, SQLITE_INT_TO_PTR(pTab->nCol+1),P4_INT32); sqlite3VdbeChangeP2(v, 7, pTab->nCol); if( !db->mallocFailed ){ | > > > | | 69198 69199 69200 69201 69202 69203 69204 69205 69206 69207 69208 69209 69210 69211 69212 69213 69214 69215 | ** always return an SQL NULL. This is useful because it means ** we can invoke OP_Column to fill in the vdbe cursors type ** and offset cache without causing any IO. */ sqlite3VdbeChangeP4(v, 3+flags, SQLITE_INT_TO_PTR(pTab->nCol+1),P4_INT32); sqlite3VdbeChangeP2(v, 7, pTab->nCol); if( !db->mallocFailed ){ pParse->nVar = 1; pParse->nMem = 1; pParse->nTab = 1; sqlite3VdbeMakeReady(v, pParse); } } pBlob->flags = flags; pBlob->iCol = iCol; pBlob->db = db; sqlite3BtreeLeaveAll(db); |
︙ | ︙ | |||
71643 71644 71645 71646 71647 71648 71649 | z = pExpr->u.zToken; assert( z!=0 ); assert( z[0]!=0 ); if( z[1]==0 ){ /* Wildcard of the form "?". Assign the next variable number */ assert( z[0]=='?' ); pExpr->iColumn = (ynVar)(++pParse->nVar); | > > > | | | | | | | | | | | | | > | | | | | | | | | | < < | | < < | | | | < | > | < | > | < > | > | < | | | | 71768 71769 71770 71771 71772 71773 71774 71775 71776 71777 71778 71779 71780 71781 71782 71783 71784 71785 71786 71787 71788 71789 71790 71791 71792 71793 71794 71795 71796 71797 71798 71799 71800 71801 71802 71803 71804 71805 71806 71807 71808 71809 71810 71811 71812 71813 71814 71815 71816 71817 71818 71819 71820 71821 71822 71823 71824 71825 71826 71827 71828 | z = pExpr->u.zToken; assert( z!=0 ); assert( z[0]!=0 ); if( z[1]==0 ){ /* Wildcard of the form "?". Assign the next variable number */ assert( z[0]=='?' ); pExpr->iColumn = (ynVar)(++pParse->nVar); }else{ ynVar x = 0; u32 n = sqlite3Strlen30(z); if( z[0]=='?' ){ /* Wildcard of the form "?nnn". Convert "nnn" to an integer and ** use it as the variable number */ i64 i; int bOk = 0==sqlite3Atoi64(&z[1], &i, n-1, SQLITE_UTF8); pExpr->iColumn = x = (ynVar)i; testcase( i==0 ); testcase( i==1 ); testcase( i==db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER]-1 ); testcase( i==db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ); if( bOk==0 || i<1 || i>db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ){ sqlite3ErrorMsg(pParse, "variable number must be between ?1 and ?%d", db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER]); x = 0; } if( i>pParse->nVar ){ pParse->nVar = (int)i; } }else{ /* Wildcards like ":aaa", "$aaa" or "@aaa". Reuse the same variable ** number as the prior appearance of the same name, or if the name ** has never appeared before, reuse the same variable number */ ynVar i; for(i=0; i<pParse->nzVar; i++){ if( pParse->azVar[i] && memcmp(pParse->azVar[i],z,n+1)==0 ){ pExpr->iColumn = x = (ynVar)i+1; break; } } if( x==0 ) x = pExpr->iColumn = (ynVar)(++pParse->nVar); } if( x>0 ){ if( x>pParse->nzVar ){ char **a; a = sqlite3DbRealloc(db, pParse->azVar, x*sizeof(a[0])); if( a==0 ) return; /* Error reported through db->mallocFailed */ pParse->azVar = a; memset(&a[pParse->nzVar], 0, (x-pParse->nzVar)*sizeof(a[0])); pParse->nzVar = x; } if( z[0]!='?' || pParse->azVar[x-1]==0 ){ sqlite3DbFree(db, pParse->azVar[x-1]); pParse->azVar[x-1] = sqlite3DbStrNDup(db, z, n); } } } if( !pParse->nErr && pParse->nVar>db->aLimit[SQLITE_LIMIT_VARIABLE_NUMBER] ){ sqlite3ErrorMsg(pParse, "too many SQL variables"); } } |
︙ | ︙ | |||
73433 73434 73435 73436 73437 73438 73439 | #endif case TK_VARIABLE: { assert( !ExprHasProperty(pExpr, EP_IntValue) ); assert( pExpr->u.zToken!=0 ); assert( pExpr->u.zToken[0]!=0 ); sqlite3VdbeAddOp2(v, OP_Variable, pExpr->iColumn, target); if( pExpr->u.zToken[1]!=0 ){ | > > | | 73558 73559 73560 73561 73562 73563 73564 73565 73566 73567 73568 73569 73570 73571 73572 73573 73574 | #endif case TK_VARIABLE: { assert( !ExprHasProperty(pExpr, EP_IntValue) ); assert( pExpr->u.zToken!=0 ); assert( pExpr->u.zToken[0]!=0 ); sqlite3VdbeAddOp2(v, OP_Variable, pExpr->iColumn, target); if( pExpr->u.zToken[1]!=0 ){ assert( pExpr->u.zToken[0]=='?' || strcmp(pExpr->u.zToken, pParse->azVar[pExpr->iColumn-1])==0 ); sqlite3VdbeChangeP4(v, -1, pParse->azVar[pExpr->iColumn-1], P4_STATIC); } break; } case TK_REGISTER: { inReg = pExpr->iTable; break; } |
︙ | ︙ | |||
75203 75204 75205 75206 75207 75208 75209 | /* Drop the table and index from the internal schema. */ sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0); /* Reload the table, index and permanent trigger schemas. */ zWhere = sqlite3MPrintf(pParse->db, "tbl_name=%Q", zName); if( !zWhere ) return; | | | | 75330 75331 75332 75333 75334 75335 75336 75337 75338 75339 75340 75341 75342 75343 75344 75345 75346 75347 75348 75349 75350 75351 | /* Drop the table and index from the internal schema. */ sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0); /* Reload the table, index and permanent trigger schemas. */ zWhere = sqlite3MPrintf(pParse->db, "tbl_name=%Q", zName); if( !zWhere ) return; sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere); #ifndef SQLITE_OMIT_TRIGGER /* Now, if the table is not stored in the temp database, reload any temp ** triggers. Don't use IN(...) in case SQLITE_OMIT_SUBQUERY is defined. */ if( (zWhere=whereTempTriggers(pParse, pTab))!=0 ){ sqlite3VdbeAddParseSchemaOp(v, 1, zWhere); } #endif } /* ** Parameter zName is the name of a table that is about to be altered ** (either with ALTER TABLE ... RENAME TO or ALTER TABLE ... ADD COLUMN). |
︙ | ︙ | |||
77404 77405 77406 77407 77408 77409 77410 | FILE *trace = (db->flags & SQLITE_VdbeTrace)!=0 ? stdout : 0; sqlite3VdbeTrace(v, trace); #endif assert( pParse->iCacheLevel==0 ); /* Disables and re-enables match */ /* A minimum of one cursor is required if autoincrement is used * See ticket [a696379c1f08866] */ if( pParse->pAinc!=0 && pParse->nTab==0 ) pParse->nTab = 1; | | < < | 77531 77532 77533 77534 77535 77536 77537 77538 77539 77540 77541 77542 77543 77544 77545 | FILE *trace = (db->flags & SQLITE_VdbeTrace)!=0 ? stdout : 0; sqlite3VdbeTrace(v, trace); #endif assert( pParse->iCacheLevel==0 ); /* Disables and re-enables match */ /* A minimum of one cursor is required if autoincrement is used * See ticket [a696379c1f08866] */ if( pParse->pAinc!=0 && pParse->nTab==0 ) pParse->nTab = 1; sqlite3VdbeMakeReady(v, pParse); pParse->rc = SQLITE_DONE; pParse->colNamesSet = 0; }else{ pParse->rc = SQLITE_ERROR; } pParse->nTab = 0; pParse->nMem = 0; |
︙ | ︙ | |||
78825 78826 78827 78828 78829 78830 78831 | pDb->zName ); } } #endif /* Reparse everything to update our internal data structures */ | | | | 78950 78951 78952 78953 78954 78955 78956 78957 78958 78959 78960 78961 78962 78963 78964 78965 | pDb->zName ); } } #endif /* Reparse everything to update our internal data structures */ sqlite3VdbeAddParseSchemaOp(v, iDb, sqlite3MPrintf(db, "tbl_name='%q'", p->zName)); } /* Add the table to the in-memory representation of the database. */ if( db->init.busy ){ Table *pOld; |
︙ | ︙ | |||
80023 80024 80025 80026 80027 80028 80029 | /* Fill the index with data and reparse the schema. Code an OP_Expire ** to invalidate all pre-compiled statements. */ if( pTblName ){ sqlite3RefillIndex(pParse, pIndex, iMem); sqlite3ChangeCookie(pParse, iDb); | | | < | 80148 80149 80150 80151 80152 80153 80154 80155 80156 80157 80158 80159 80160 80161 80162 80163 | /* Fill the index with data and reparse the schema. Code an OP_Expire ** to invalidate all pre-compiled statements. */ if( pTblName ){ sqlite3RefillIndex(pParse, pIndex, iMem); sqlite3ChangeCookie(pParse, iDb); sqlite3VdbeAddParseSchemaOp(v, iDb, sqlite3MPrintf(db, "name='%q' AND type='index'", pIndex->zName)); sqlite3VdbeAddOp1(v, OP_Expire, 0); } } /* When adding an index to the list of indices for a table, make ** sure all indices labeled OE_Replace come after all those labeled ** OE_Ignore. This is necessary for the correct constraint check |
︙ | ︙ | |||
81823 81824 81825 81826 81827 81828 81829 81830 81831 81832 81833 81834 81835 81836 | /* Delete the row */ #ifndef SQLITE_OMIT_VIRTUALTABLE if( IsVirtual(pTab) ){ const char *pVTab = (const char *)sqlite3GetVTable(db, pTab); sqlite3VtabMakeWritable(pParse, pTab); sqlite3VdbeAddOp4(v, OP_VUpdate, 0, 1, iRowid, pVTab, P4_VTAB); sqlite3MayAbort(pParse); }else #endif { int count = (pParse->nested==0); /* True to count changes */ sqlite3GenerateRowDelete(pParse, pTab, iCur, iRowid, count, pTrigger, OE_Default); } | > | 81947 81948 81949 81950 81951 81952 81953 81954 81955 81956 81957 81958 81959 81960 81961 | /* Delete the row */ #ifndef SQLITE_OMIT_VIRTUALTABLE if( IsVirtual(pTab) ){ const char *pVTab = (const char *)sqlite3GetVTable(db, pTab); sqlite3VtabMakeWritable(pParse, pTab); sqlite3VdbeAddOp4(v, OP_VUpdate, 0, 1, iRowid, pVTab, P4_VTAB); sqlite3VdbeChangeP5(v, OE_Abort); sqlite3MayAbort(pParse); }else #endif { int count = (pParse->nested==0); /* True to count changes */ sqlite3GenerateRowDelete(pParse, pTab, iCur, iRowid, count, pTrigger, OE_Default); } |
︙ | ︙ | |||
82057 82058 82059 82060 82061 82062 82063 82064 | sqlite3VdbeAddOp2(v, OP_SCopy, regBase+nCol, regBase+j); }else{ sqlite3VdbeAddOp3(v, OP_Column, iCur, idx, regBase+j); sqlite3ColumnDefault(v, pTab, idx, -1); } } if( doMakeRec ){ sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol+1, regOut); | > > > > > > | | 82182 82183 82184 82185 82186 82187 82188 82189 82190 82191 82192 82193 82194 82195 82196 82197 82198 82199 82200 82201 82202 82203 | sqlite3VdbeAddOp2(v, OP_SCopy, regBase+nCol, regBase+j); }else{ sqlite3VdbeAddOp3(v, OP_Column, iCur, idx, regBase+j); sqlite3ColumnDefault(v, pTab, idx, -1); } } if( doMakeRec ){ const char *zAff; if( pTab->pSelect || (pParse->db->flags & SQLITE_IdxRealAsInt)!=0 ){ zAff = 0; }else{ zAff = sqlite3IndexAffinityStr(v, pIdx); } sqlite3VdbeAddOp3(v, OP_MakeRecord, regBase, nCol+1, regOut); sqlite3VdbeChangeP4(v, -1, zAff, P4_TRANSIENT); } sqlite3ReleaseTempRange(pParse, regBase, nCol+1); return regBase; } /************** End of delete.c **********************************************/ /************** Begin file func.c ********************************************/ |
︙ | ︙ | |||
82570 82571 82572 82573 82574 82575 82576 | /* ** For LIKE and GLOB matching on EBCDIC machines, assume that every ** character is exactly one byte in size. Also, all characters are ** able to participate in upper-case-to-lower-case mappings in EBCDIC ** whereas only characters less than 0x80 do in ASCII. */ #if defined(SQLITE_EBCDIC) | | | | | 82701 82702 82703 82704 82705 82706 82707 82708 82709 82710 82711 82712 82713 82714 82715 82716 82717 82718 | /* ** For LIKE and GLOB matching on EBCDIC machines, assume that every ** character is exactly one byte in size. Also, all characters are ** able to participate in upper-case-to-lower-case mappings in EBCDIC ** whereas only characters less than 0x80 do in ASCII. */ #if defined(SQLITE_EBCDIC) # define sqlite3Utf8Read(A,C) (*(A++)) # define GlogUpperToLower(A) A = sqlite3UpperToLower[A] #else # define GlogUpperToLower(A) if( !((A)&~0x7f) ){ A = sqlite3UpperToLower[A]; } #endif static const struct compareInfo globInfo = { '*', '?', '[', 0 }; /* The correct SQL-92 behavior is for the LIKE operator to ignore ** case. Thus 'a' LIKE 'A' would be true. */ static const struct compareInfo likeInfoNorm = { '%', '_', 0, 1 }; /* If SQLITE_CASE_SENSITIVE_LIKE is defined, then the LIKE operator |
︙ | ︙ | |||
82616 82617 82618 82619 82620 82621 82622 | ** ** abc[*]xyz Matches "abc*xyz" only */ static int patternCompare( const u8 *zPattern, /* The glob pattern */ const u8 *zString, /* The string to compare against the glob */ const struct compareInfo *pInfo, /* Information about how to do the compare */ | | | | 82747 82748 82749 82750 82751 82752 82753 82754 82755 82756 82757 82758 82759 82760 82761 82762 82763 | ** ** abc[*]xyz Matches "abc*xyz" only */ static int patternCompare( const u8 *zPattern, /* The glob pattern */ const u8 *zString, /* The string to compare against the glob */ const struct compareInfo *pInfo, /* Information about how to do the compare */ u32 esc /* The escape character */ ){ u32 c, c2; int invert; int seen; u8 matchOne = pInfo->matchOne; u8 matchAll = pInfo->matchAll; u8 matchSet = pInfo->matchSet; u8 noCase = pInfo->noCase; int prevEscape = 0; /* True if the previous character was 'escape' */ |
︙ | ︙ | |||
82672 82673 82674 82675 82676 82677 82678 | } return 0; }else if( !prevEscape && c==matchOne ){ if( sqlite3Utf8Read(zString, &zString)==0 ){ return 0; } }else if( c==matchSet ){ | | | 82803 82804 82805 82806 82807 82808 82809 82810 82811 82812 82813 82814 82815 82816 82817 | } return 0; }else if( !prevEscape && c==matchOne ){ if( sqlite3Utf8Read(zString, &zString)==0 ){ return 0; } }else if( c==matchSet ){ u32 prior_c = 0; assert( esc==0 ); /* This only occurs for GLOB, not LIKE */ seen = 0; invert = 0; c = sqlite3Utf8Read(zString, &zString); if( c==0 ) return 0; c2 = sqlite3Utf8Read(zPattern, &zPattern); if( c2=='^' ){ |
︙ | ︙ | |||
82748 82749 82750 82751 82752 82753 82754 | */ static void likeFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ const unsigned char *zA, *zB; | | | 82879 82880 82881 82882 82883 82884 82885 82886 82887 82888 82889 82890 82891 82892 82893 | */ static void likeFunc( sqlite3_context *context, int argc, sqlite3_value **argv ){ const unsigned char *zA, *zB; u32 escape = 0; int nPat; sqlite3 *db = sqlite3_context_db_handle(context); zB = sqlite3_value_text(argv[0]); zA = sqlite3_value_text(argv[1]); /* Limit the length of the LIKE or GLOB pattern to avoid problems |
︙ | ︙ | |||
84059 84060 84061 84062 84063 84064 84065 | for(i=0; i<nCol; i++){ sqlite3VdbeAddOp2(v, OP_Copy, aiCol[i]+1+regData, regTemp+i); } /* If the parent table is the same as the child table, and we are about ** to increment the constraint-counter (i.e. this is an INSERT operation), ** then check if the row being inserted matches itself. If so, do not | | > > > > > > > > > > > > | 84190 84191 84192 84193 84194 84195 84196 84197 84198 84199 84200 84201 84202 84203 84204 84205 84206 84207 84208 84209 84210 84211 84212 84213 84214 84215 84216 84217 84218 84219 84220 84221 84222 | for(i=0; i<nCol; i++){ sqlite3VdbeAddOp2(v, OP_Copy, aiCol[i]+1+regData, regTemp+i); } /* If the parent table is the same as the child table, and we are about ** to increment the constraint-counter (i.e. this is an INSERT operation), ** then check if the row being inserted matches itself. If so, do not ** increment the constraint-counter. ** ** If any of the parent-key values are NULL, then the row cannot match ** itself. So set JUMPIFNULL to make sure we do the OP_Found if any ** of the parent-key values are NULL (at this point it is known that ** none of the child key values are). */ if( pTab==pFKey->pFrom && nIncr==1 ){ int iJump = sqlite3VdbeCurrentAddr(v) + nCol + 1; for(i=0; i<nCol; i++){ int iChild = aiCol[i]+1+regData; int iParent = pIdx->aiColumn[i]+1+regData; assert( aiCol[i]!=pTab->iPKey ); if( pIdx->aiColumn[i]==pTab->iPKey ){ /* The parent key is a composite key that includes the IPK column */ iParent = regData; } sqlite3VdbeAddOp3(v, OP_Ne, iChild, iJump, iParent); sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL); } sqlite3VdbeAddOp2(v, OP_Goto, 0, iOk); } sqlite3VdbeAddOp3(v, OP_MakeRecord, regTemp, nCol, regRec); sqlite3VdbeChangeP4(v, -1, sqlite3IndexAffinityStr(v,pIdx), P4_TRANSIENT); sqlite3VdbeAddOp4Int(v, OP_Found, iCur, iOk, regRec, 0); |
︙ | ︙ | |||
87943 87944 87945 87946 87947 87948 87949 | ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** This file contains code used to implement the PRAGMA command. */ | < < < < | 88086 88087 88088 88089 88090 88091 88092 88093 88094 88095 88096 88097 88098 88099 | ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** This file contains code used to implement the PRAGMA command. */ /* ** Interpret the given string as a safety level. Return 0 for OFF, ** 1 for ON or NORMAL and 2 for FULL. Return 1 for an empty or ** unrecognized string argument. ** ** Note that the values returned are one less that the values that ** should be passed into sqlite3BtreeSetSafetyLevel(). The is done |
︙ | ︙ | |||
87982 87983 87984 87985 87986 87987 87988 87989 87990 87991 87992 87993 87994 87995 | /* ** Interpret the given string as a boolean value. */ SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z){ return getSafetyLevel(z)&1; } /* ** Interpret the given string as a locking mode value. */ static int getLockingMode(const char *z){ if( z ){ if( 0==sqlite3StrICmp(z, "exclusive") ) return PAGER_LOCKINGMODE_EXCLUSIVE; | > > > > > > | 88121 88122 88123 88124 88125 88126 88127 88128 88129 88130 88131 88132 88133 88134 88135 88136 88137 88138 88139 88140 | /* ** Interpret the given string as a boolean value. */ SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z){ return getSafetyLevel(z)&1; } /* The sqlite3GetBoolean() function is used by other modules but the ** remainder of this file is specific to PRAGMA processing. So omit ** the rest of the file if PRAGMAs are omitted from the build. */ #if !defined(SQLITE_OMIT_PRAGMA) /* ** Interpret the given string as a locking mode value. */ static int getLockingMode(const char *z){ if( z ){ if( 0==sqlite3StrICmp(z, "exclusive") ) return PAGER_LOCKINGMODE_EXCLUSIVE; |
︙ | ︙ | |||
95283 95284 95285 95286 95287 95288 95289 | z = sqlite3DbStrNDup(db, (char*)pAll->z, pAll->n); sqlite3NestedParse(pParse, "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')", db->aDb[iDb].zName, SCHEMA_TABLE(iDb), zName, pTrig->table, z); sqlite3DbFree(db, z); sqlite3ChangeCookie(pParse, iDb); | | | < | 95428 95429 95430 95431 95432 95433 95434 95435 95436 95437 95438 95439 95440 95441 95442 95443 | z = sqlite3DbStrNDup(db, (char*)pAll->z, pAll->n); sqlite3NestedParse(pParse, "INSERT INTO %Q.%s VALUES('trigger',%Q,%Q,0,'CREATE TRIGGER %q')", db->aDb[iDb].zName, SCHEMA_TABLE(iDb), zName, pTrig->table, z); sqlite3DbFree(db, z); sqlite3ChangeCookie(pParse, iDb); sqlite3VdbeAddParseSchemaOp(v, iDb, sqlite3MPrintf(db, "type='trigger' AND name='%q'", zName)); } if( db->init.busy ){ Trigger *pLink = pTrig; Hash *pHash = &db->aDb[iDb].pSchema->trigHash; assert( sqlite3SchemaMutexHeld(db, iDb, 0) ); pTrig = sqlite3HashInsert(pHash, zName, sqlite3Strlen30(zName), pTrig); |
︙ | ︙ | |||
96339 96340 96341 96342 96343 96344 96345 | for(nIdx=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, nIdx++){} if( nIdx>0 ){ aRegIdx = sqlite3DbMallocRaw(db, sizeof(Index*) * nIdx ); if( aRegIdx==0 ) goto update_cleanup; } for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){ int reg; | | | 96483 96484 96485 96486 96487 96488 96489 96490 96491 96492 96493 96494 96495 96496 96497 | for(nIdx=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, nIdx++){} if( nIdx>0 ){ aRegIdx = sqlite3DbMallocRaw(db, sizeof(Index*) * nIdx ); if( aRegIdx==0 ) goto update_cleanup; } for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){ int reg; if( hasFK || chngRowid ){ reg = ++pParse->nMem; }else{ reg = 0; for(i=0; i<pIdx->nColumn; i++){ if( aXRef[pIdx->aiColumn[i]]>=0 ){ reg = ++pParse->nMem; break; |
︙ | ︙ | |||
97494 97495 97496 97497 97498 97499 97500 | ); sqlite3DbFree(db, zStmt); v = sqlite3GetVdbe(pParse); sqlite3ChangeCookie(pParse, iDb); sqlite3VdbeAddOp2(v, OP_Expire, 0, 0); zWhere = sqlite3MPrintf(db, "name='%q' AND type='table'", pTab->zName); | | | 97638 97639 97640 97641 97642 97643 97644 97645 97646 97647 97648 97649 97650 97651 97652 | ); sqlite3DbFree(db, zStmt); v = sqlite3GetVdbe(pParse); sqlite3ChangeCookie(pParse, iDb); sqlite3VdbeAddOp2(v, OP_Expire, 0, 0); zWhere = sqlite3MPrintf(db, "name='%q' AND type='table'", pTab->zName); sqlite3VdbeAddParseSchemaOp(v, iDb, zWhere); sqlite3VdbeAddOp4(v, OP_VCreate, iDb, 0, 0, pTab->zName, sqlite3Strlen30(pTab->zName) + 1); } /* If we are rereading the sqlite_master table create the in-memory ** record of the table. The xConnect() method is not called until ** the first time the virtual table is used in an SQL statement. This |
︙ | ︙ | |||
107221 107222 107223 107224 107225 107226 107227 | return i; } #ifndef SQLITE_OMIT_BLOB_LITERAL case 'x': case 'X': { testcase( z[0]=='x' ); testcase( z[0]=='X' ); if( z[1]=='\'' ){ *tokenType = TK_BLOB; | < | > | > | < < | | 107365 107366 107367 107368 107369 107370 107371 107372 107373 107374 107375 107376 107377 107378 107379 107380 107381 107382 107383 107384 | return i; } #ifndef SQLITE_OMIT_BLOB_LITERAL case 'x': case 'X': { testcase( z[0]=='x' ); testcase( z[0]=='X' ); if( z[1]=='\'' ){ *tokenType = TK_BLOB; for(i=2; sqlite3Isxdigit(z[i]); i++){} if( z[i]!='\'' || i%2 ){ *tokenType = TK_ILLEGAL; while( z[i] && z[i]!='\'' ){ i++; } } if( z[i] ) i++; return i; } /* Otherwise fall through to the next case */ } #endif default: { if( !IdChar(*z) ){ |
︙ | ︙ | |||
107280 107281 107282 107283 107284 107285 107286 | if( pEngine==0 ){ db->mallocFailed = 1; return SQLITE_NOMEM; } assert( pParse->pNewTable==0 ); assert( pParse->pNewTrigger==0 ); assert( pParse->nVar==0 ); | | < | | 107423 107424 107425 107426 107427 107428 107429 107430 107431 107432 107433 107434 107435 107436 107437 107438 | if( pEngine==0 ){ db->mallocFailed = 1; return SQLITE_NOMEM; } assert( pParse->pNewTable==0 ); assert( pParse->pNewTrigger==0 ); assert( pParse->nVar==0 ); assert( pParse->nzVar==0 ); assert( pParse->azVar==0 ); enableLookaside = db->lookaside.bEnabled; if( db->lookaside.pStart ) db->lookaside.bEnabled = 1; while( !db->mallocFailed && zSql[i]!=0 ){ assert( i>=0 ); pParse->sLastToken.z = &zSql[i]; pParse->sLastToken.n = sqlite3GetToken((unsigned char*)&zSql[i],&tokenType); i += pParse->sLastToken.n; |
︙ | ︙ | |||
107376 107377 107378 107379 107380 107381 107382 | ** structure built up in pParse->pNewTable. The calling code (see vtab.c) ** will take responsibility for freeing the Table structure. */ sqlite3DeleteTable(db, pParse->pNewTable); } sqlite3DeleteTrigger(db, pParse->pNewTrigger); | > | | 107518 107519 107520 107521 107522 107523 107524 107525 107526 107527 107528 107529 107530 107531 107532 107533 | ** structure built up in pParse->pNewTable. The calling code (see vtab.c) ** will take responsibility for freeing the Table structure. */ sqlite3DeleteTable(db, pParse->pNewTable); } sqlite3DeleteTrigger(db, pParse->pNewTrigger); for(i=pParse->nzVar-1; i>=0; i--) sqlite3DbFree(db, pParse->azVar[i]); sqlite3DbFree(db, pParse->azVar); sqlite3DbFree(db, pParse->aAlias); while( pParse->pAinc ){ AutoincInfo *p = pParse->pAinc; pParse->pAinc = p->pNext; sqlite3DbFree(db, p); } while( pParse->pZombieTab ){ |
︙ | ︙ | |||
109708 109709 109710 109711 109712 109713 109714 | if( nOpt==3 && memcmp("vfs", zOpt, 3)==0 ){ zVfs = zVal; }else{ struct OpenMode { const char *z; int mode; } *aMode = 0; | | | | | 109851 109852 109853 109854 109855 109856 109857 109858 109859 109860 109861 109862 109863 109864 109865 109866 109867 | if( nOpt==3 && memcmp("vfs", zOpt, 3)==0 ){ zVfs = zVal; }else{ struct OpenMode { const char *z; int mode; } *aMode = 0; char *zModeType = 0; int mask = 0; int limit = 0; if( nOpt==5 && memcmp("cache", zOpt, 5)==0 ){ static struct OpenMode aCacheMode[] = { { "shared", SQLITE_OPEN_SHAREDCACHE }, { "private", SQLITE_OPEN_PRIVATECACHE }, { 0, 0 } }; |
︙ | ︙ | |||
110673 110674 110675 110676 110677 110678 110679 110680 110681 110682 110683 110684 110685 110686 | ppNew = va_arg(ap, void**); pFree = va_arg(ap, void*); if( sz ) *ppNew = sqlite3ScratchMalloc(sz); sqlite3ScratchFree(pFree); break; } } va_end(ap); #endif /* SQLITE_OMIT_BUILTIN_TEST */ return rc; } /* | > > > > > > > > > > > | 110816 110817 110818 110819 110820 110821 110822 110823 110824 110825 110826 110827 110828 110829 110830 110831 110832 110833 110834 110835 110836 110837 110838 110839 110840 | ppNew = va_arg(ap, void**); pFree = va_arg(ap, void*); if( sz ) *ppNew = sqlite3ScratchMalloc(sz); sqlite3ScratchFree(pFree); break; } /* sqlite3_test_control(SQLITE_TESTCTRL_LOCALTIME_FAULT, int onoff); ** ** If parameter onoff is non-zero, configure the wrappers so that all ** subsequent calls to localtime() and variants fail. If onoff is zero, ** undo this setting. */ case SQLITE_TESTCTRL_LOCALTIME_FAULT: { sqlite3GlobalConfig.bLocaltimeFault = va_arg(ap, int); break; } } va_end(ap); #endif /* SQLITE_OMIT_BUILTIN_TEST */ return rc; } /* |
︙ | ︙ | |||
111330 111331 111332 111333 111334 111335 111336 | ** older data. ** ** TODO(shess) Provide a VACUUM type operation to clear out all ** deletions and duplications. This would basically be a forced merge ** into a single segment. */ | < < < < < < < > > > > > > > > > > | 111484 111485 111486 111487 111488 111489 111490 111491 111492 111493 111494 111495 111496 111497 111498 111499 111500 111501 111502 111503 111504 111505 111506 111507 111508 111509 111510 111511 111512 111513 111514 111515 111516 111517 111518 111519 111520 111521 111522 111523 111524 111525 111526 111527 111528 111529 | ** older data. ** ** TODO(shess) Provide a VACUUM type operation to clear out all ** deletions and duplications. This would basically be a forced merge ** into a single segment. */ /************** Include fts3Int.h in the middle of fts3.c ********************/ /************** Begin file fts3Int.h *****************************************/ /* ** 2009 Nov 12 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** */ #ifndef _FTSINT_H #define _FTSINT_H #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) # define NDEBUG 1 #endif /* ** FTS4 is really an extension for FTS3. It is enabled using the ** SQLITE_ENABLE_FTS3 macro. But to avoid confusion we also all ** the SQLITE_ENABLE_FTS4 macro to serve as an alisse for SQLITE_ENABLE_FTS3. */ #if defined(SQLITE_ENABLE_FTS4) && !defined(SQLITE_ENABLE_FTS3) # define SQLITE_ENABLE_FTS3 #endif #ifdef SQLITE_ENABLE_FTS3 /************** Include fts3_tokenizer.h in the middle of fts3Int.h **********/ /************** Begin file fts3_tokenizer.h **********************************/ /* ** 2006 July 10 ** ** The author disclaims copyright to this source code. ** |
︙ | ︙ | |||
111657 111658 111659 111660 111661 111662 111663 111664 111665 111666 111667 111668 111669 111670 111671 111672 111673 111674 111675 111676 | /* ** Macro to return the number of elements in an array. SQLite has a ** similar macro called ArraySize(). Use a different name to avoid ** a collision when building an amalgamation with built-in FTS3. */ #define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0]))) /* ** Maximum length of a varint encoded integer. The varint format is different ** from that used by SQLite, so the maximum length is 10, not 9. */ #define FTS3_VARINT_MAX 10 /* ** The testcase() macro is only used by the amalgamation. If undefined, ** make it a no-op. */ #ifndef testcase # define testcase(X) #endif | > > > > > > > > > > > > > > > > > > > > > > > | 111814 111815 111816 111817 111818 111819 111820 111821 111822 111823 111824 111825 111826 111827 111828 111829 111830 111831 111832 111833 111834 111835 111836 111837 111838 111839 111840 111841 111842 111843 111844 111845 111846 111847 111848 111849 111850 111851 111852 111853 111854 111855 111856 | /* ** Macro to return the number of elements in an array. SQLite has a ** similar macro called ArraySize(). Use a different name to avoid ** a collision when building an amalgamation with built-in FTS3. */ #define SizeofArray(X) ((int)(sizeof(X)/sizeof(X[0]))) #ifndef MIN # define MIN(x,y) ((x)<(y)?(x):(y)) #endif /* ** Maximum length of a varint encoded integer. The varint format is different ** from that used by SQLite, so the maximum length is 10, not 9. */ #define FTS3_VARINT_MAX 10 /* ** FTS4 virtual tables may maintain multiple indexes - one index of all terms ** in the document set and zero or more prefix indexes. All indexes are stored ** as one or more b+-trees in the %_segments and %_segdir tables. ** ** It is possible to determine which index a b+-tree belongs to based on the ** value stored in the "%_segdir.level" column. Given this value L, the index ** that the b+-tree belongs to is (L<<10). In other words, all b+-trees with ** level values between 0 and 1023 (inclusive) belong to index 0, all levels ** between 1024 and 2047 to index 1, and so on. ** ** It is considered impossible for an index to use more than 1024 levels. In ** theory though this may happen, but only after at least ** (FTS3_MERGE_COUNT^1024) separate flushes of the pending-terms tables. */ #define FTS3_SEGDIR_MAXLEVEL 1024 #define FTS3_SEGDIR_MAXLEVEL_STR "1024" /* ** The testcase() macro is only used by the amalgamation. If undefined, ** make it a no-op. */ #ifndef testcase # define testcase(X) #endif |
︙ | ︙ | |||
111734 111735 111736 111737 111738 111739 111740 111741 111742 111743 | typedef struct Fts3Table Fts3Table; typedef struct Fts3Cursor Fts3Cursor; typedef struct Fts3Expr Fts3Expr; typedef struct Fts3Phrase Fts3Phrase; typedef struct Fts3PhraseToken Fts3PhraseToken; typedef struct Fts3SegFilter Fts3SegFilter; typedef struct Fts3DeferredToken Fts3DeferredToken; typedef struct Fts3SegReader Fts3SegReader; | > | | > > > | > > > > > > > > > > | | | < | 111914 111915 111916 111917 111918 111919 111920 111921 111922 111923 111924 111925 111926 111927 111928 111929 111930 111931 111932 111933 111934 111935 111936 111937 111938 111939 111940 111941 111942 111943 111944 111945 111946 111947 111948 111949 111950 111951 111952 111953 111954 111955 111956 111957 111958 111959 111960 111961 111962 111963 111964 111965 111966 111967 111968 111969 111970 111971 111972 111973 111974 111975 111976 111977 111978 111979 111980 111981 111982 111983 111984 111985 111986 111987 | typedef struct Fts3Table Fts3Table; typedef struct Fts3Cursor Fts3Cursor; typedef struct Fts3Expr Fts3Expr; typedef struct Fts3Phrase Fts3Phrase; typedef struct Fts3PhraseToken Fts3PhraseToken; typedef struct Fts3Doclist Fts3Doclist; typedef struct Fts3SegFilter Fts3SegFilter; typedef struct Fts3DeferredToken Fts3DeferredToken; typedef struct Fts3SegReader Fts3SegReader; typedef struct Fts3MultiSegReader Fts3MultiSegReader; /* ** A connection to a fulltext index is an instance of the following ** structure. The xCreate and xConnect methods create an instance ** of this structure and xDestroy and xDisconnect free that instance. ** All other methods receive a pointer to the structure as one of their ** arguments. */ struct Fts3Table { sqlite3_vtab base; /* Base class used by SQLite core */ sqlite3 *db; /* The database connection */ const char *zDb; /* logical database name */ const char *zName; /* virtual table name */ int nColumn; /* number of named columns in virtual table */ char **azColumn; /* column names. malloced */ sqlite3_tokenizer *pTokenizer; /* tokenizer for inserts and queries */ /* Precompiled statements used by the implementation. Each of these ** statements is run and reset within a single virtual table API call. */ sqlite3_stmt *aStmt[27]; char *zReadExprlist; char *zWriteExprlist; int nNodeSize; /* Soft limit for node size */ u8 bHasStat; /* True if %_stat table exists */ u8 bHasDocsize; /* True if %_docsize table exists */ u8 bDescIdx; /* True if doclists are in reverse order */ int nPgsz; /* Page size for host database */ char *zSegmentsTbl; /* Name of %_segments table */ sqlite3_blob *pSegments; /* Blob handle open on %_segments table */ /* TODO: Fix the first paragraph of this comment. ** ** The following hash table is used to buffer pending index updates during ** transactions. Variable nPendingData estimates the memory size of the ** pending data, including hash table overhead, but not malloc overhead. ** When nPendingData exceeds nMaxPendingData, the buffer is flushed ** automatically. Variable iPrevDocid is the docid of the most recently ** inserted record. ** ** A single FTS4 table may have multiple full-text indexes. For each index ** there is an entry in the aIndex[] array. Index 0 is an index of all the ** terms that appear in the document set. Each subsequent index in aIndex[] ** is an index of prefixes of a specific length. */ int nIndex; /* Size of aIndex[] */ struct Fts3Index { int nPrefix; /* Prefix length (0 for main terms index) */ Fts3Hash hPending; /* Pending terms table for this index */ } *aIndex; int nMaxPendingData; /* Max pending data before flush to disk */ int nPendingData; /* Current bytes of pending data */ sqlite_int64 iPrevDocid; /* Docid of most recently inserted document */ #if defined(SQLITE_DEBUG) /* State variables used for validating that the transaction control ** methods of the virtual table are called at appropriate times. These ** values do not contribution to the FTS computation; they are used for ** verifying the SQLite core. */ |
︙ | ︙ | |||
111811 111812 111813 111814 111815 111816 111817 | Fts3Expr *pExpr; /* Parsed MATCH query string */ int nPhrase; /* Number of matchable phrases in query */ Fts3DeferredToken *pDeferred; /* Deferred search tokens, if any */ sqlite3_int64 iPrevId; /* Previous id read from aDoclist */ char *pNextId; /* Pointer into the body of aDoclist */ char *aDoclist; /* List of docids for full-text queries */ int nDoclist; /* Size of buffer at aDoclist */ | | > | 112004 112005 112006 112007 112008 112009 112010 112011 112012 112013 112014 112015 112016 112017 112018 112019 112020 112021 | Fts3Expr *pExpr; /* Parsed MATCH query string */ int nPhrase; /* Number of matchable phrases in query */ Fts3DeferredToken *pDeferred; /* Deferred search tokens, if any */ sqlite3_int64 iPrevId; /* Previous id read from aDoclist */ char *pNextId; /* Pointer into the body of aDoclist */ char *aDoclist; /* List of docids for full-text queries */ int nDoclist; /* Size of buffer at aDoclist */ u8 bDesc; /* True to sort in descending order */ int eEvalmode; /* An FTS3_EVAL_XX constant */ int nRowAvg; /* Average size of database rows, in pages */ sqlite3_int64 nDoc; /* Documents in table */ int isMatchinfoNeeded; /* True when aMatchinfo[] needs filling in */ u32 *aMatchinfo; /* Information about most recent match */ int nMatchinfo; /* Number of elements in aMatchinfo[] */ char *zMatchinfo; /* Matchinfo specification */ }; |
︙ | ︙ | |||
111844 111845 111846 111847 111848 111849 111850 111851 111852 111853 111854 111855 | ** indicating that all columns should be searched, ** then eSearch would be set to FTS3_FULLTEXT_SEARCH+4. */ #define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */ #define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */ #define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */ /* ** A "phrase" is a sequence of one or more tokens that must match in ** sequence. A single token is the base case and the most common case. ** For a sequence of tokens contained in double-quotes (i.e. "one two three") ** nToken will be the number of tokens in the string. | > > > > > > > > > > > > < < < < < < | | > > > > > > > > | > > < | | | | | | | > > > > > > > > | > > | | | < | | 112038 112039 112040 112041 112042 112043 112044 112045 112046 112047 112048 112049 112050 112051 112052 112053 112054 112055 112056 112057 112058 112059 112060 112061 112062 112063 112064 112065 112066 112067 112068 112069 112070 112071 112072 112073 112074 112075 112076 112077 112078 112079 112080 112081 112082 112083 112084 112085 112086 112087 112088 112089 112090 112091 112092 112093 112094 112095 112096 112097 112098 112099 112100 112101 112102 112103 112104 112105 112106 112107 112108 112109 112110 112111 112112 112113 112114 112115 112116 112117 112118 112119 112120 112121 112122 112123 112124 112125 112126 112127 112128 112129 112130 112131 | ** indicating that all columns should be searched, ** then eSearch would be set to FTS3_FULLTEXT_SEARCH+4. */ #define FTS3_FULLSCAN_SEARCH 0 /* Linear scan of %_content table */ #define FTS3_DOCID_SEARCH 1 /* Lookup by rowid on %_content table */ #define FTS3_FULLTEXT_SEARCH 2 /* Full-text index search */ struct Fts3Doclist { char *aAll; /* Array containing doclist (or NULL) */ int nAll; /* Size of a[] in bytes */ char *pNextDocid; /* Pointer to next docid */ sqlite3_int64 iDocid; /* Current docid (if pList!=0) */ int bFreeList; /* True if pList should be sqlite3_free()d */ char *pList; /* Pointer to position list following iDocid */ int nList; /* Length of position list */ } doclist; /* ** A "phrase" is a sequence of one or more tokens that must match in ** sequence. A single token is the base case and the most common case. ** For a sequence of tokens contained in double-quotes (i.e. "one two three") ** nToken will be the number of tokens in the string. */ struct Fts3PhraseToken { char *z; /* Text of the token */ int n; /* Number of bytes in buffer z */ int isPrefix; /* True if token ends with a "*" character */ /* Variables above this point are populated when the expression is ** parsed (by code in fts3_expr.c). Below this point the variables are ** used when evaluating the expression. */ Fts3DeferredToken *pDeferred; /* Deferred token object for this token */ Fts3MultiSegReader *pSegcsr; /* Segment-reader for this token */ }; struct Fts3Phrase { /* Cache of doclist for this phrase. */ Fts3Doclist doclist; int bIncr; /* True if doclist is loaded incrementally */ int iDoclistToken; /* Variables below this point are populated by fts3_expr.c when parsing ** a MATCH expression. Everything above is part of the evaluation phase. */ int nToken; /* Number of tokens in the phrase */ int iColumn; /* Index of column this phrase must match */ Fts3PhraseToken aToken[1]; /* One entry for each token in the phrase */ }; /* ** A tree of these objects forms the RHS of a MATCH operator. ** ** If Fts3Expr.eType is FTSQUERY_PHRASE and isLoaded is true, then aDoclist ** points to a malloced buffer, size nDoclist bytes, containing the results ** of this phrase query in FTS3 doclist format. As usual, the initial ** "Length" field found in doclists stored on disk is omitted from this ** buffer. ** ** Variable aMI is used only for FTSQUERY_NEAR nodes to store the global ** matchinfo data. If it is not NULL, it points to an array of size nCol*3, ** where nCol is the number of columns in the queried FTS table. The array ** is populated as follows: ** ** aMI[iCol*3 + 0] = Undefined ** aMI[iCol*3 + 1] = Number of occurrences ** aMI[iCol*3 + 2] = Number of rows containing at least one instance ** ** The aMI array is allocated using sqlite3_malloc(). It should be freed ** when the expression node is. */ struct Fts3Expr { int eType; /* One of the FTSQUERY_XXX values defined below */ int nNear; /* Valid if eType==FTSQUERY_NEAR */ Fts3Expr *pParent; /* pParent->pLeft==this or pParent->pRight==this */ Fts3Expr *pLeft; /* Left operand */ Fts3Expr *pRight; /* Right operand */ Fts3Phrase *pPhrase; /* Valid if eType==FTSQUERY_PHRASE */ /* The following are used by the fts3_eval.c module. */ sqlite3_int64 iDocid; /* Current docid */ u8 bEof; /* True this expression is at EOF already */ u8 bStart; /* True if iDocid is valid */ u8 bDeferred; /* True if this expression is entirely deferred */ u32 *aMI; }; /* ** Candidate values for Fts3Query.eType. Note that the order of the first ** four values is in order of precedence when parsing expressions. For ** example, the following: ** |
︙ | ︙ | |||
111927 111928 111929 111930 111931 111932 111933 | /* fts3_write.c */ SQLITE_PRIVATE int sqlite3Fts3UpdateMethod(sqlite3_vtab*,int,sqlite3_value**,sqlite3_int64*); SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *); SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *); SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *); SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(int, sqlite3_int64, sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**); | | > < | | < > | | | | | > | | > > > | | > > < | < < | 112145 112146 112147 112148 112149 112150 112151 112152 112153 112154 112155 112156 112157 112158 112159 112160 112161 112162 112163 112164 112165 112166 112167 112168 112169 112170 112171 112172 112173 112174 112175 112176 112177 112178 112179 112180 112181 112182 112183 112184 112185 112186 112187 112188 112189 112190 112191 112192 112193 112194 112195 112196 112197 112198 112199 112200 112201 112202 112203 112204 112205 112206 112207 112208 112209 112210 112211 112212 112213 112214 112215 112216 112217 112218 112219 112220 112221 112222 112223 112224 112225 112226 112227 112228 112229 112230 112231 112232 | /* fts3_write.c */ SQLITE_PRIVATE int sqlite3Fts3UpdateMethod(sqlite3_vtab*,int,sqlite3_value**,sqlite3_int64*); SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *); SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *); SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *); SQLITE_PRIVATE int sqlite3Fts3SegReaderNew(int, sqlite3_int64, sqlite3_int64, sqlite3_int64, const char *, int, Fts3SegReader**); SQLITE_PRIVATE int sqlite3Fts3SegReaderPending( Fts3Table*,int,const char*,int,int,Fts3SegReader**); SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *); SQLITE_PRIVATE int sqlite3Fts3AllSegdirs(Fts3Table*, int, int, sqlite3_stmt **); SQLITE_PRIVATE int sqlite3Fts3ReadLock(Fts3Table *); SQLITE_PRIVATE int sqlite3Fts3ReadBlock(Fts3Table*, sqlite3_int64, char **, int*, int*); SQLITE_PRIVATE int sqlite3Fts3SelectDoctotal(Fts3Table *, sqlite3_stmt **); SQLITE_PRIVATE int sqlite3Fts3SelectDocsize(Fts3Table *, sqlite3_int64, sqlite3_stmt **); SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *); SQLITE_PRIVATE int sqlite3Fts3DeferToken(Fts3Cursor *, Fts3PhraseToken *, int); SQLITE_PRIVATE int sqlite3Fts3CacheDeferredDoclists(Fts3Cursor *); SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *); SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *); /* Special values interpreted by sqlite3SegReaderCursor() */ #define FTS3_SEGCURSOR_PENDING -1 #define FTS3_SEGCURSOR_ALL -2 SQLITE_PRIVATE int sqlite3Fts3SegReaderStart(Fts3Table*, Fts3MultiSegReader*, Fts3SegFilter*); SQLITE_PRIVATE int sqlite3Fts3SegReaderStep(Fts3Table *, Fts3MultiSegReader *); SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish(Fts3MultiSegReader *); SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor( Fts3Table *, int, int, const char *, int, int, int, Fts3MultiSegReader *); /* Flags allowed as part of the 4th argument to SegmentReaderIterate() */ #define FTS3_SEGMENT_REQUIRE_POS 0x00000001 #define FTS3_SEGMENT_IGNORE_EMPTY 0x00000002 #define FTS3_SEGMENT_COLUMN_FILTER 0x00000004 #define FTS3_SEGMENT_PREFIX 0x00000008 #define FTS3_SEGMENT_SCAN 0x00000010 /* Type passed as 4th argument to SegmentReaderIterate() */ struct Fts3SegFilter { const char *zTerm; int nTerm; int iCol; int flags; }; struct Fts3MultiSegReader { /* Used internally by sqlite3Fts3SegReaderXXX() calls */ Fts3SegReader **apSegment; /* Array of Fts3SegReader objects */ int nSegment; /* Size of apSegment array */ int nAdvance; /* How many seg-readers to advance */ Fts3SegFilter *pFilter; /* Pointer to filter object */ char *aBuffer; /* Buffer to merge doclists in */ int nBuffer; /* Allocated size of aBuffer[] in bytes */ int iColFilter; /* If >=0, filter for this column */ int bRestart; /* Used by fts3.c only. */ int nCost; /* Cost of running iterator */ int bLookup; /* True if a lookup of a single entry. */ /* Output values. Valid only after Fts3SegReaderStep() returns SQLITE_ROW. */ char *zTerm; /* Pointer to term buffer */ int nTerm; /* Size of zTerm in bytes */ char *aDoclist; /* Pointer to doclist buffer */ int nDoclist; /* Size of aDoclist[] in bytes */ }; /* fts3.c */ SQLITE_PRIVATE int sqlite3Fts3PutVarint(char *, sqlite3_int64); SQLITE_PRIVATE int sqlite3Fts3GetVarint(const char *, sqlite_int64 *); SQLITE_PRIVATE int sqlite3Fts3GetVarint32(const char *, int *); SQLITE_PRIVATE int sqlite3Fts3VarintLen(sqlite3_uint64); SQLITE_PRIVATE void sqlite3Fts3Dequote(char *); SQLITE_PRIVATE void sqlite3Fts3DoclistPrev(int,char*,int,char**,sqlite3_int64*,int*,u8*); SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats(Fts3Cursor *, Fts3Expr *, u32 *); /* fts3_tokenizer.c */ SQLITE_PRIVATE const char *sqlite3Fts3NextToken(const char *, int *); SQLITE_PRIVATE int sqlite3Fts3InitHashTable(sqlite3 *, Fts3Hash *, const char *); SQLITE_PRIVATE int sqlite3Fts3InitTokenizer(Fts3Hash *pHash, const char *, sqlite3_tokenizer **, char ** ); |
︙ | ︙ | |||
112027 112028 112029 112030 112031 112032 112033 112034 112035 112036 112037 112038 112039 112040 112041 112042 112043 112044 | SQLITE_PRIVATE int sqlite3Fts3ExprInitTestInterface(sqlite3 *db); SQLITE_PRIVATE int sqlite3Fts3InitTerm(sqlite3 *db); #endif /* fts3_aux.c */ SQLITE_PRIVATE int sqlite3Fts3InitAux(sqlite3 *db); #endif /* _FTSINT_H */ /************** End of fts3Int.h *********************************************/ /************** Continuing where we left off in fts3.c ***********************/ #ifndef SQLITE_CORE SQLITE_EXTENSION_INIT1 #endif /* | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 112248 112249 112250 112251 112252 112253 112254 112255 112256 112257 112258 112259 112260 112261 112262 112263 112264 112265 112266 112267 112268 112269 112270 112271 112272 112273 112274 112275 112276 112277 112278 112279 112280 112281 112282 112283 112284 112285 112286 112287 112288 112289 112290 112291 112292 112293 112294 | SQLITE_PRIVATE int sqlite3Fts3ExprInitTestInterface(sqlite3 *db); SQLITE_PRIVATE int sqlite3Fts3InitTerm(sqlite3 *db); #endif /* fts3_aux.c */ SQLITE_PRIVATE int sqlite3Fts3InitAux(sqlite3 *db); SQLITE_PRIVATE int sqlite3Fts3TermSegReaderCursor( Fts3Cursor *pCsr, /* Virtual table cursor handle */ const char *zTerm, /* Term to query for */ int nTerm, /* Size of zTerm in bytes */ int isPrefix, /* True for a prefix search */ Fts3MultiSegReader **ppSegcsr /* OUT: Allocated seg-reader cursor */ ); SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *); SQLITE_PRIVATE int sqlite3Fts3EvalStart(Fts3Cursor *, Fts3Expr *, int); SQLITE_PRIVATE int sqlite3Fts3EvalNext(Fts3Cursor *pCsr); SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart( Fts3Table*, Fts3MultiSegReader*, int, const char*, int); SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext( Fts3Table *, Fts3MultiSegReader *, sqlite3_int64 *, char **, int *); SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist(Fts3Cursor *, Fts3Expr *, int iCol); SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *); SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr); SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList(Fts3DeferredToken *, char **, int *); #endif /* SQLITE_ENABLE_FTS3 */ #endif /* _FTSINT_H */ /************** End of fts3Int.h *********************************************/ /************** Continuing where we left off in fts3.c ***********************/ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #if defined(SQLITE_ENABLE_FTS3) && !defined(SQLITE_CORE) # define SQLITE_CORE 1 #endif #ifndef SQLITE_CORE SQLITE_EXTENSION_INIT1 #endif /* |
︙ | ︙ | |||
112147 112148 112149 112150 112151 112152 112153 | *pVal += iVal; } /* ** When this function is called, *pp points to the first byte following a ** varint that is part of a doclist (or position-list, or any other list ** of varints). This function moves *pp to point to the start of that varint, | | | | < < < < < < < < < < < < < < | 112397 112398 112399 112400 112401 112402 112403 112404 112405 112406 112407 112408 112409 112410 112411 112412 112413 112414 112415 112416 112417 112418 112419 112420 112421 112422 112423 112424 112425 112426 112427 112428 112429 112430 112431 112432 | *pVal += iVal; } /* ** When this function is called, *pp points to the first byte following a ** varint that is part of a doclist (or position-list, or any other list ** of varints). This function moves *pp to point to the start of that varint, ** and sets *pVal by the varint value. ** ** Argument pStart points to the first byte of the doclist that the ** varint is part of. */ static void fts3GetReverseVarint( char **pp, char *pStart, sqlite3_int64 *pVal ){ sqlite3_int64 iVal; char *p = *pp; /* Pointer p now points at the first byte past the varint we are ** interested in. So, unless the doclist is corrupt, the 0x80 bit is ** clear on character p[-1]. */ for(p = (*pp)-2; p>=pStart && *p&0x80; p--); p++; *pp = p; sqlite3Fts3GetVarint(p, &iVal); *pVal = iVal; } /* ** The xDisconnect() virtual table method. */ static int fts3DisconnectMethod(sqlite3_vtab *pVtab){ Fts3Table *p = (Fts3Table *)pVtab; |
︙ | ︙ | |||
112383 112384 112385 112386 112387 112388 112389 112390 112391 112392 112393 112394 112395 112396 | rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare(p->db, zSql, -1, &pStmt, 0); if( rc==SQLITE_OK ){ sqlite3_step(pStmt); p->nPgsz = sqlite3_column_int(pStmt, 0); rc = sqlite3_finalize(pStmt); } } assert( p->nPgsz>0 || rc!=SQLITE_OK ); sqlite3_free(zSql); *pRc = rc; } } | > > > | 112619 112620 112621 112622 112623 112624 112625 112626 112627 112628 112629 112630 112631 112632 112633 112634 112635 | rc = SQLITE_NOMEM; }else{ rc = sqlite3_prepare(p->db, zSql, -1, &pStmt, 0); if( rc==SQLITE_OK ){ sqlite3_step(pStmt); p->nPgsz = sqlite3_column_int(pStmt, 0); rc = sqlite3_finalize(pStmt); }else if( rc==SQLITE_AUTH ){ p->nPgsz = 1024; rc = SQLITE_OK; } } assert( p->nPgsz>0 || rc!=SQLITE_OK ); sqlite3_free(zSql); *pRc = rc; } } |
︙ | ︙ | |||
112555 112556 112557 112558 112559 112560 112561 112562 112563 112564 112565 112566 112567 112568 | fts3Appendf(pRc, &zRet, "?"); for(i=0; i<p->nColumn; i++){ fts3Appendf(pRc, &zRet, ",%s(?)", zFunction); } sqlite3_free(zFree); return zRet; } /* ** This function is the implementation of both the xConnect and xCreate ** methods of the FTS3 virtual table. ** ** The argv[] array contains the following: ** | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 112794 112795 112796 112797 112798 112799 112800 112801 112802 112803 112804 112805 112806 112807 112808 112809 112810 112811 112812 112813 112814 112815 112816 112817 112818 112819 112820 112821 112822 112823 112824 112825 112826 112827 112828 112829 112830 112831 112832 112833 112834 112835 112836 112837 112838 112839 112840 112841 112842 112843 112844 112845 112846 112847 112848 112849 112850 112851 112852 112853 112854 112855 112856 112857 112858 112859 | fts3Appendf(pRc, &zRet, "?"); for(i=0; i<p->nColumn; i++){ fts3Appendf(pRc, &zRet, ",%s(?)", zFunction); } sqlite3_free(zFree); return zRet; } static int fts3GobbleInt(const char **pp, int *pnOut){ const char *p = *pp; int nInt = 0; for(p=*pp; p[0]>='0' && p[0]<='9'; p++){ nInt = nInt * 10 + (p[0] - '0'); } if( p==*pp ) return SQLITE_ERROR; *pnOut = nInt; *pp = p; return SQLITE_OK; } static int fts3PrefixParameter( const char *zParam, /* ABC in prefix=ABC parameter to parse */ int *pnIndex, /* OUT: size of *apIndex[] array */ struct Fts3Index **apIndex, /* OUT: Array of indexes for this table */ struct Fts3Index **apFree /* OUT: Free this with sqlite3_free() */ ){ struct Fts3Index *aIndex; int nIndex = 1; if( zParam && zParam[0] ){ const char *p; nIndex++; for(p=zParam; *p; p++){ if( *p==',' ) nIndex++; } } aIndex = sqlite3_malloc(sizeof(struct Fts3Index) * nIndex); *apIndex = *apFree = aIndex; *pnIndex = nIndex; if( !aIndex ){ return SQLITE_NOMEM; } memset(aIndex, 0, sizeof(struct Fts3Index) * nIndex); if( zParam ){ const char *p = zParam; int i; for(i=1; i<nIndex; i++){ int nPrefix; if( fts3GobbleInt(&p, &nPrefix) ) return SQLITE_ERROR; aIndex[i].nPrefix = nPrefix; p++; } } return SQLITE_OK; } /* ** This function is the implementation of both the xConnect and xCreate ** methods of the FTS3 virtual table. ** ** The argv[] array contains the following: ** |
︙ | ︙ | |||
112588 112589 112590 112591 112592 112593 112594 | int iCol; /* Column index */ int nString = 0; /* Bytes required to hold all column names */ int nCol = 0; /* Number of columns in the FTS table */ char *zCsr; /* Space for holding column names */ int nDb; /* Bytes required to hold database name */ int nName; /* Bytes required to hold table name */ int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */ | < > > > > > > > > | | | 112879 112880 112881 112882 112883 112884 112885 112886 112887 112888 112889 112890 112891 112892 112893 112894 112895 112896 112897 112898 112899 112900 112901 112902 112903 112904 112905 | int iCol; /* Column index */ int nString = 0; /* Bytes required to hold all column names */ int nCol = 0; /* Number of columns in the FTS table */ char *zCsr; /* Space for holding column names */ int nDb; /* Bytes required to hold database name */ int nName; /* Bytes required to hold table name */ int isFts4 = (argv[0][3]=='4'); /* True for FTS4, false for FTS3 */ const char **aCol; /* Array of column names */ sqlite3_tokenizer *pTokenizer = 0; /* Tokenizer for this table */ int nIndex; /* Size of aIndex[] array */ struct Fts3Index *aIndex; /* Array of indexes for this table */ struct Fts3Index *aFree = 0; /* Free this before returning */ /* The results of parsing supported FTS4 key=value options: */ int bNoDocsize = 0; /* True to omit %_docsize table */ int bDescIdx = 0; /* True to store descending indexes */ char *zPrefix = 0; /* Prefix parameter value (or NULL) */ char *zCompress = 0; /* compress=? parameter (or NULL) */ char *zUncompress = 0; /* uncompress=? parameter (or NULL) */ assert( strlen(argv[0])==4 ); assert( (sqlite3_strnicmp(argv[0], "fts4", 4)==0 && isFts4) || (sqlite3_strnicmp(argv[0], "fts3", 4)==0 && !isFts4) ); nDb = (int)strlen(argv[1]) + 1; |
︙ | ︙ | |||
112634 112635 112636 112637 112638 112639 112640 112641 112642 | && 0==sqlite3Fts3IsIdChar(z[8]) ){ rc = sqlite3Fts3InitTokenizer(pHash, &z[9], &pTokenizer, pzErr); } /* Check if it is an FTS4 special argument. */ else if( isFts4 && fts3IsSpecialColumn(z, &nKey, &zVal) ){ if( !zVal ){ rc = SQLITE_NOMEM; | > > > > > > > > > > > > > | > > > > | > > | < | > > > | | | > > | > > > > > > > > | | > | > > | | > | > > > > | | | > > > > | > | 112932 112933 112934 112935 112936 112937 112938 112939 112940 112941 112942 112943 112944 112945 112946 112947 112948 112949 112950 112951 112952 112953 112954 112955 112956 112957 112958 112959 112960 112961 112962 112963 112964 112965 112966 112967 112968 112969 112970 112971 112972 112973 112974 112975 112976 112977 112978 112979 112980 112981 112982 112983 112984 112985 112986 112987 112988 112989 112990 112991 112992 112993 112994 112995 112996 112997 112998 112999 113000 113001 113002 113003 113004 113005 113006 113007 113008 113009 113010 113011 | && 0==sqlite3Fts3IsIdChar(z[8]) ){ rc = sqlite3Fts3InitTokenizer(pHash, &z[9], &pTokenizer, pzErr); } /* Check if it is an FTS4 special argument. */ else if( isFts4 && fts3IsSpecialColumn(z, &nKey, &zVal) ){ struct Fts4Option { const char *zOpt; int nOpt; char **pzVar; } aFts4Opt[] = { { "matchinfo", 9, 0 }, /* 0 -> MATCHINFO */ { "prefix", 6, 0 }, /* 1 -> PREFIX */ { "compress", 8, 0 }, /* 2 -> COMPRESS */ { "uncompress", 10, 0 }, /* 3 -> UNCOMPRESS */ { "order", 5, 0 } /* 4 -> ORDER */ }; int iOpt; if( !zVal ){ rc = SQLITE_NOMEM; }else{ for(iOpt=0; iOpt<SizeofArray(aFts4Opt); iOpt++){ struct Fts4Option *pOp = &aFts4Opt[iOpt]; if( nKey==pOp->nOpt && !sqlite3_strnicmp(z, pOp->zOpt, pOp->nOpt) ){ break; } } if( iOpt==SizeofArray(aFts4Opt) ){ *pzErr = sqlite3_mprintf("unrecognized parameter: %s", z); rc = SQLITE_ERROR; }else{ switch( iOpt ){ case 0: /* MATCHINFO */ if( strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "fts3", 4) ){ *pzErr = sqlite3_mprintf("unrecognized matchinfo: %s", zVal); rc = SQLITE_ERROR; } bNoDocsize = 1; break; case 1: /* PREFIX */ sqlite3_free(zPrefix); zPrefix = zVal; zVal = 0; break; case 2: /* COMPRESS */ sqlite3_free(zCompress); zCompress = zVal; zVal = 0; break; case 3: /* UNCOMPRESS */ sqlite3_free(zUncompress); zUncompress = zVal; zVal = 0; break; case 4: /* ORDER */ if( (strlen(zVal)!=3 || sqlite3_strnicmp(zVal, "asc", 3)) && (strlen(zVal)!=4 || sqlite3_strnicmp(zVal, "desc", 3)) ){ *pzErr = sqlite3_mprintf("unrecognized order: %s", zVal); rc = SQLITE_ERROR; } bDescIdx = (zVal[0]=='d' || zVal[0]=='D'); break; } } sqlite3_free(zVal); } } /* Otherwise, the argument is a column name. */ else { nString += (int)(strlen(z) + 1); aCol[nCol++] = z; } |
︙ | ︙ | |||
112679 112680 112681 112682 112683 112684 112685 | if( pTokenizer==0 ){ rc = sqlite3Fts3InitTokenizer(pHash, "simple", &pTokenizer, pzErr); if( rc!=SQLITE_OK ) goto fts3_init_out; } assert( pTokenizer ); | > > > > | > > | > < > > > > > > | | > | | | 113021 113022 113023 113024 113025 113026 113027 113028 113029 113030 113031 113032 113033 113034 113035 113036 113037 113038 113039 113040 113041 113042 113043 113044 113045 113046 113047 113048 113049 113050 113051 113052 113053 113054 113055 113056 113057 113058 113059 113060 113061 113062 113063 113064 113065 113066 113067 113068 113069 113070 113071 113072 113073 113074 113075 113076 113077 113078 113079 113080 113081 113082 113083 113084 113085 113086 | if( pTokenizer==0 ){ rc = sqlite3Fts3InitTokenizer(pHash, "simple", &pTokenizer, pzErr); if( rc!=SQLITE_OK ) goto fts3_init_out; } assert( pTokenizer ); rc = fts3PrefixParameter(zPrefix, &nIndex, &aIndex, &aFree); if( rc==SQLITE_ERROR ){ assert( zPrefix ); *pzErr = sqlite3_mprintf("error parsing prefix parameter: %s", zPrefix); } if( rc!=SQLITE_OK ) goto fts3_init_out; /* Allocate and populate the Fts3Table structure. */ nByte = sizeof(Fts3Table) + /* Fts3Table */ nCol * sizeof(char *) + /* azColumn */ nIndex * sizeof(struct Fts3Index) + /* aIndex */ nName + /* zName */ nDb + /* zDb */ nString; /* Space for azColumn strings */ p = (Fts3Table*)sqlite3_malloc(nByte); if( p==0 ){ rc = SQLITE_NOMEM; goto fts3_init_out; } memset(p, 0, nByte); p->db = db; p->nColumn = nCol; p->nPendingData = 0; p->azColumn = (char **)&p[1]; p->pTokenizer = pTokenizer; p->nMaxPendingData = FTS3_MAX_PENDING_DATA; p->bHasDocsize = (isFts4 && bNoDocsize==0); p->bHasStat = isFts4; p->bDescIdx = bDescIdx; TESTONLY( p->inTransaction = -1 ); TESTONLY( p->mxSavepoint = -1 ); p->aIndex = (struct Fts3Index *)&p->azColumn[nCol]; memcpy(p->aIndex, aIndex, sizeof(struct Fts3Index) * nIndex); p->nIndex = nIndex; for(i=0; i<nIndex; i++){ fts3HashInit(&p->aIndex[i].hPending, FTS3_HASH_STRING, 1); } /* Fill in the zName and zDb fields of the vtab structure. */ zCsr = (char *)&p->aIndex[nIndex]; p->zName = zCsr; memcpy(zCsr, argv[2], nName); zCsr += nName; p->zDb = zCsr; memcpy(zCsr, argv[1], nDb); zCsr += nDb; /* Fill in the azColumn array */ for(iCol=0; iCol<nCol; iCol++){ char *z; int n = 0; z = (char *)sqlite3Fts3NextToken(aCol[iCol], &n); memcpy(zCsr, z, n); zCsr[n] = '\0'; sqlite3Fts3Dequote(zCsr); p->azColumn[iCol] = zCsr; zCsr += n+1; assert( zCsr <= &((char *)p)[nByte] ); |
︙ | ︙ | |||
112744 112745 112746 112747 112748 112749 112750 | ** database. TODO: For xConnect(), it could verify that said tables exist. */ if( isCreate ){ rc = fts3CreateTables(p); } /* Figure out the page-size for the database. This is required in order to | | < < > > > > | 113099 113100 113101 113102 113103 113104 113105 113106 113107 113108 113109 113110 113111 113112 113113 113114 113115 113116 113117 113118 113119 113120 113121 113122 113123 113124 113125 113126 113127 113128 113129 113130 113131 113132 113133 | ** database. TODO: For xConnect(), it could verify that said tables exist. */ if( isCreate ){ rc = fts3CreateTables(p); } /* Figure out the page-size for the database. This is required in order to ** estimate the cost of loading large doclists from the database. */ fts3DatabasePageSize(&rc, p); p->nNodeSize = p->nPgsz-35; /* Declare the table schema to SQLite. */ fts3DeclareVtab(&rc, p); fts3_init_out: sqlite3_free(zPrefix); sqlite3_free(aFree); sqlite3_free(zCompress); sqlite3_free(zUncompress); sqlite3_free((void *)aCol); if( rc!=SQLITE_OK ){ if( p ){ fts3DisconnectMethod((sqlite3_vtab *)p); }else if( pTokenizer ){ pTokenizer->pModule->xDestroy(pTokenizer); } }else{ assert( p->pSegments==0 ); *ppVTab = &p->base; } return rc; } /* ** The xConnect() and xCreate() methods for the virtual table. All the |
︙ | ︙ | |||
112860 112861 112862 112863 112864 112865 112866 | struct sqlite3_index_orderby *pOrder = &pInfo->aOrderBy[0]; if( pOrder->iColumn<0 || pOrder->iColumn==p->nColumn+1 ){ if( pOrder->desc ){ pInfo->idxStr = "DESC"; }else{ pInfo->idxStr = "ASC"; } | < | | | > > | 113217 113218 113219 113220 113221 113222 113223 113224 113225 113226 113227 113228 113229 113230 113231 113232 113233 113234 113235 | struct sqlite3_index_orderby *pOrder = &pInfo->aOrderBy[0]; if( pOrder->iColumn<0 || pOrder->iColumn==p->nColumn+1 ){ if( pOrder->desc ){ pInfo->idxStr = "DESC"; }else{ pInfo->idxStr = "ASC"; } pInfo->orderByConsumed = 1; } } assert( p->pSegments==0 ); return SQLITE_OK; } /* ** Implementation of xOpen method. */ static int fts3OpenMethod(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCsr){ |
︙ | ︙ | |||
112899 112900 112901 112902 112903 112904 112905 112906 112907 112908 112909 112910 112911 112912 112913 112914 112915 112916 | Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); sqlite3_finalize(pCsr->pStmt); sqlite3Fts3ExprFree(pCsr->pExpr); sqlite3Fts3FreeDeferredTokens(pCsr); sqlite3_free(pCsr->aDoclist); sqlite3_free(pCsr->aMatchinfo); sqlite3_free(pCsr); return SQLITE_OK; } /* ** Position the pCsr->pStmt statement so that it is on the row ** of the %_content table that contains the last match. Return ** SQLITE_OK on success. */ static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){ if( pCsr->isRequireSeek ){ | > < > | 113257 113258 113259 113260 113261 113262 113263 113264 113265 113266 113267 113268 113269 113270 113271 113272 113273 113274 113275 113276 113277 113278 113279 113280 113281 113282 113283 113284 | Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); sqlite3_finalize(pCsr->pStmt); sqlite3Fts3ExprFree(pCsr->pExpr); sqlite3Fts3FreeDeferredTokens(pCsr); sqlite3_free(pCsr->aDoclist); sqlite3_free(pCsr->aMatchinfo); assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); sqlite3_free(pCsr); return SQLITE_OK; } /* ** Position the pCsr->pStmt statement so that it is on the row ** of the %_content table that contains the last match. Return ** SQLITE_OK on success. */ static int fts3CursorSeek(sqlite3_context *pContext, Fts3Cursor *pCsr){ if( pCsr->isRequireSeek ){ sqlite3_bind_int64(pCsr->pStmt, 1, pCsr->iPrevId); pCsr->isRequireSeek = 0; if( SQLITE_ROW==sqlite3_step(pCsr->pStmt) ){ return SQLITE_OK; }else{ int rc = sqlite3_reset(pCsr->pStmt); if( rc==SQLITE_OK ){ /* If no row was found and no error has occured, then the %_content ** table is missing a row that is present in the full-text index. |
︙ | ︙ | |||
113092 113093 113094 113095 113096 113097 113098 | assert( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) ); if( rc==SQLITE_OK && iHeight>1 ){ char *zBlob = 0; /* Blob read from %_segments table */ int nBlob; /* Size of zBlob in bytes */ if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){ | | | | 113451 113452 113453 113454 113455 113456 113457 113458 113459 113460 113461 113462 113463 113464 113465 113466 113467 113468 113469 113470 113471 113472 113473 113474 113475 | assert( !piLeaf2 || !piLeaf || rc!=SQLITE_OK || (*piLeaf<=*piLeaf2) ); if( rc==SQLITE_OK && iHeight>1 ){ char *zBlob = 0; /* Blob read from %_segments table */ int nBlob; /* Size of zBlob in bytes */ if( piLeaf && piLeaf2 && (*piLeaf!=*piLeaf2) ){ rc = sqlite3Fts3ReadBlock(p, *piLeaf, &zBlob, &nBlob, 0); if( rc==SQLITE_OK ){ rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, 0); } sqlite3_free(zBlob); piLeaf = 0; zBlob = 0; } if( rc==SQLITE_OK ){ rc = sqlite3Fts3ReadBlock(p, piLeaf?*piLeaf:*piLeaf2, &zBlob, &nBlob, 0); } if( rc==SQLITE_OK ){ rc = fts3SelectLeaf(p, zTerm, nTerm, zBlob, nBlob, piLeaf, piLeaf2); } sqlite3_free(zBlob); } |
︙ | ︙ | |||
113478 113479 113480 113481 113482 113483 113484 | } *p++ = 0x00; *pp = p; return 1; } /* | | > > > > > > > > > > > > < < < < < < | | | | | | | | | | | | | | | | | | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | < | | | > > > > | | < < | 113837 113838 113839 113840 113841 113842 113843 113844 113845 113846 113847 113848 113849 113850 113851 113852 113853 113854 113855 113856 113857 113858 113859 113860 113861 113862 113863 113864 113865 113866 113867 113868 113869 113870 113871 113872 113873 113874 113875 113876 113877 113878 113879 113880 113881 113882 113883 113884 113885 113886 113887 113888 113889 113890 113891 113892 113893 113894 113895 113896 113897 113898 113899 113900 113901 113902 113903 113904 113905 113906 113907 113908 113909 113910 113911 113912 113913 113914 113915 113916 113917 113918 113919 113920 113921 113922 113923 113924 113925 113926 113927 113928 113929 113930 113931 113932 113933 113934 113935 113936 113937 113938 113939 113940 113941 113942 113943 113944 113945 113946 113947 113948 113949 113950 113951 113952 113953 113954 113955 113956 113957 113958 113959 113960 113961 113962 113963 113964 113965 113966 113967 113968 113969 113970 113971 113972 113973 113974 113975 113976 113977 113978 113979 113980 113981 113982 113983 113984 113985 113986 113987 113988 113989 113990 113991 113992 113993 113994 113995 113996 113997 113998 113999 114000 114001 114002 114003 114004 114005 114006 114007 114008 114009 114010 114011 114012 114013 114014 114015 114016 114017 114018 114019 114020 114021 114022 114023 114024 114025 114026 114027 114028 114029 114030 114031 114032 114033 114034 114035 114036 114037 114038 114039 114040 114041 114042 114043 114044 114045 114046 114047 114048 114049 114050 114051 114052 114053 114054 114055 114056 114057 114058 114059 114060 114061 114062 114063 114064 114065 114066 114067 114068 114069 114070 114071 114072 114073 114074 114075 114076 114077 114078 114079 114080 114081 114082 114083 114084 114085 114086 | } *p++ = 0x00; *pp = p; return 1; } /* ** Merge two position-lists as required by the NEAR operator. The argument ** position lists correspond to the left and right phrases of an expression ** like: ** ** "phrase 1" NEAR "phrase number 2" ** ** Position list *pp1 corresponds to the left-hand side of the NEAR ** expression and *pp2 to the right. As usual, the indexes in the position ** lists are the offsets of the last token in each phrase (tokens "1" and "2" ** in the example above). ** ** The output position list - written to *pp - is a copy of *pp2 with those ** entries that are not sufficiently NEAR entries in *pp1 removed. */ static int fts3PoslistNearMerge( char **pp, /* Output buffer */ char *aTmp, /* Temporary buffer space */ int nRight, /* Maximum difference in token positions */ int nLeft, /* Maximum difference in token positions */ char **pp1, /* IN/OUT: Left input list */ char **pp2 /* IN/OUT: Right input list */ ){ char *p1 = *pp1; char *p2 = *pp2; char *pTmp1 = aTmp; char *pTmp2; char *aTmp2; int res = 1; fts3PoslistPhraseMerge(&pTmp1, nRight, 0, 0, pp1, pp2); aTmp2 = pTmp2 = pTmp1; *pp1 = p1; *pp2 = p2; fts3PoslistPhraseMerge(&pTmp2, nLeft, 1, 0, pp2, pp1); if( pTmp1!=aTmp && pTmp2!=aTmp2 ){ fts3PoslistMerge(pp, &aTmp, &aTmp2); }else if( pTmp1!=aTmp ){ fts3PoslistCopy(pp, &aTmp); }else if( pTmp2!=aTmp2 ){ fts3PoslistCopy(pp, &aTmp2); }else{ res = 0; } return res; } /* ** A pointer to an instance of this structure is used as the context ** argument to sqlite3Fts3SegReaderIterate() */ typedef struct TermSelect TermSelect; struct TermSelect { int isReqPos; char *aaOutput[16]; /* Malloc'd output buffer */ int anOutput[16]; /* Size of output in bytes */ }; static void fts3GetDeltaVarint3( char **pp, char *pEnd, int bDescIdx, sqlite3_int64 *pVal ){ if( *pp>=pEnd ){ *pp = 0; }else{ sqlite3_int64 iVal; *pp += sqlite3Fts3GetVarint(*pp, &iVal); if( bDescIdx ){ *pVal -= iVal; }else{ *pVal += iVal; } } } static void fts3PutDeltaVarint3( char **pp, /* IN/OUT: Output pointer */ int bDescIdx, /* True for descending docids */ sqlite3_int64 *piPrev, /* IN/OUT: Previous value written to list */ int *pbFirst, /* IN/OUT: True after first int written */ sqlite3_int64 iVal /* Write this value to the list */ ){ sqlite3_int64 iWrite; if( bDescIdx==0 || *pbFirst==0 ){ iWrite = iVal - *piPrev; }else{ iWrite = *piPrev - iVal; } assert( *pbFirst || *piPrev==0 ); assert( *pbFirst==0 || iWrite>0 ); *pp += sqlite3Fts3PutVarint(*pp, iWrite); *piPrev = iVal; *pbFirst = 1; } #define COMPARE_DOCID(i1, i2) ((bDescIdx?-1:1) * (i1-i2)) static int fts3DoclistOrMerge( int bDescIdx, /* True if arguments are desc */ char *a1, int n1, /* First doclist */ char *a2, int n2, /* Second doclist */ char **paOut, int *pnOut /* OUT: Malloc'd doclist */ ){ sqlite3_int64 i1 = 0; sqlite3_int64 i2 = 0; sqlite3_int64 iPrev = 0; char *pEnd1 = &a1[n1]; char *pEnd2 = &a2[n2]; char *p1 = a1; char *p2 = a2; char *p; char *aOut; int bFirstOut = 0; *paOut = 0; *pnOut = 0; aOut = sqlite3_malloc(n1+n2); if( !aOut ) return SQLITE_NOMEM; p = aOut; fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2); while( p1 || p2 ){ sqlite3_int64 iDiff = COMPARE_DOCID(i1, i2); if( p2 && p1 && iDiff==0 ){ fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i1); fts3PoslistMerge(&p, &p1, &p2); fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); }else if( !p2 || (p1 && iDiff<0) ){ fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i1); fts3PoslistCopy(&p, &p1); fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); }else{ fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i2); fts3PoslistCopy(&p, &p2); fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); } } *paOut = aOut; *pnOut = (p-aOut); return SQLITE_OK; } static void fts3DoclistPhraseMerge( int bDescIdx, /* True if arguments are desc */ int nDist, /* Distance from left to right (1=adjacent) */ char *aLeft, int nLeft, /* Left doclist */ char *aRight, int *pnRight /* IN/OUT: Right/output doclist */ ){ sqlite3_int64 i1 = 0; sqlite3_int64 i2 = 0; sqlite3_int64 iPrev = 0; char *pEnd1 = &aLeft[nLeft]; char *pEnd2 = &aRight[*pnRight]; char *p1 = aLeft; char *p2 = aRight; char *p; int bFirstOut = 0; char *aOut = aRight; assert( nDist>0 ); p = aOut; fts3GetDeltaVarint3(&p1, pEnd1, 0, &i1); fts3GetDeltaVarint3(&p2, pEnd2, 0, &i2); while( p1 && p2 ){ sqlite3_int64 iDiff = COMPARE_DOCID(i1, i2); if( iDiff==0 ){ char *pSave = p; sqlite3_int64 iPrevSave = iPrev; int bFirstOutSave = bFirstOut; fts3PutDeltaVarint3(&p, bDescIdx, &iPrev, &bFirstOut, i1); if( 0==fts3PoslistPhraseMerge(&p, nDist, 0, 1, &p1, &p2) ){ p = pSave; iPrev = iPrevSave; bFirstOut = bFirstOutSave; } fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); }else if( iDiff<0 ){ fts3PoslistCopy(0, &p1); fts3GetDeltaVarint3(&p1, pEnd1, bDescIdx, &i1); }else{ fts3PoslistCopy(0, &p2); fts3GetDeltaVarint3(&p2, pEnd2, bDescIdx, &i2); } } *pnRight = p - aOut; } /* ** Merge all doclists in the TermSelect.aaOutput[] array into a single ** doclist stored in TermSelect.aaOutput[0]. If successful, delete all ** other doclists (except the aaOutput[0] one) and return SQLITE_OK. ** ** If an OOM error occurs, return SQLITE_NOMEM. In this case it is ** the responsibility of the caller to free any doclists left in the ** TermSelect.aaOutput[] array. */ static int fts3TermSelectMerge(Fts3Table *p, TermSelect *pTS){ char *aOut = 0; int nOut = 0; int i; /* Loop through the doclists in the aaOutput[] array. Merge them all ** into a single doclist. */ for(i=0; i<SizeofArray(pTS->aaOutput); i++){ if( pTS->aaOutput[i] ){ if( !aOut ){ aOut = pTS->aaOutput[i]; nOut = pTS->anOutput[i]; pTS->aaOutput[i] = 0; }else{ int nNew; char *aNew; int rc = fts3DoclistOrMerge(p->bDescIdx, pTS->aaOutput[i], pTS->anOutput[i], aOut, nOut, &aNew, &nNew ); if( rc!=SQLITE_OK ){ sqlite3_free(aOut); return rc; } sqlite3_free(pTS->aaOutput[i]); sqlite3_free(aOut); pTS->aaOutput[i] = 0; aOut = aNew; nOut = nNew; } } |
︙ | ︙ | |||
113781 113782 113783 113784 113785 113786 113787 | UNUSED_PARAMETER(p); UNUSED_PARAMETER(zTerm); UNUSED_PARAMETER(nTerm); if( pTS->aaOutput[0]==0 ){ /* If this is the first term selected, copy the doclist to the output | | < < < < < | > > > | | | | < < | | < < < | | | | | | | | | | | | > > > > | > | < < < > | | | < | < < < < < < < < | < < | < | > | > | > | < < < < | < < < < < | | | | < < > | > | | | < < < < < < < | | | < | > > < < < < < < < < < < < < | | | < < > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > | | | > > | > > > | | | > | > | > > > | > > > > | > > > > > > > > | > | | 114108 114109 114110 114111 114112 114113 114114 114115 114116 114117 114118 114119 114120 114121 114122 114123 114124 114125 114126 114127 114128 114129 114130 114131 114132 114133 114134 114135 114136 114137 114138 114139 114140 114141 114142 114143 114144 114145 114146 114147 114148 114149 114150 114151 114152 114153 114154 114155 114156 114157 114158 114159 114160 114161 114162 114163 114164 114165 114166 114167 114168 114169 114170 114171 114172 114173 114174 114175 114176 114177 114178 114179 114180 114181 114182 114183 114184 114185 114186 114187 114188 114189 114190 114191 114192 114193 114194 114195 114196 114197 114198 114199 114200 114201 114202 114203 114204 114205 114206 114207 114208 114209 114210 114211 114212 114213 114214 114215 114216 114217 114218 114219 114220 114221 114222 114223 114224 114225 114226 114227 114228 114229 114230 114231 114232 114233 114234 114235 114236 114237 114238 114239 114240 114241 114242 114243 114244 114245 114246 114247 114248 114249 114250 114251 114252 114253 114254 114255 114256 114257 114258 114259 114260 114261 114262 114263 114264 114265 114266 114267 114268 114269 114270 114271 114272 114273 114274 114275 114276 114277 114278 114279 114280 114281 114282 114283 114284 114285 114286 114287 114288 114289 114290 114291 114292 114293 114294 114295 114296 114297 114298 114299 114300 114301 114302 114303 114304 114305 114306 114307 114308 114309 114310 114311 114312 114313 114314 114315 114316 114317 114318 114319 114320 114321 114322 114323 114324 114325 114326 114327 114328 114329 114330 114331 114332 114333 114334 114335 114336 114337 114338 114339 114340 114341 114342 114343 114344 114345 114346 114347 114348 114349 114350 114351 114352 | UNUSED_PARAMETER(p); UNUSED_PARAMETER(zTerm); UNUSED_PARAMETER(nTerm); if( pTS->aaOutput[0]==0 ){ /* If this is the first term selected, copy the doclist to the output ** buffer using memcpy(). */ pTS->aaOutput[0] = sqlite3_malloc(nDoclist); pTS->anOutput[0] = nDoclist; if( pTS->aaOutput[0] ){ memcpy(pTS->aaOutput[0], aDoclist, nDoclist); }else{ return SQLITE_NOMEM; } }else{ char *aMerge = aDoclist; int nMerge = nDoclist; int iOut; for(iOut=0; iOut<SizeofArray(pTS->aaOutput); iOut++){ if( pTS->aaOutput[iOut]==0 ){ assert( iOut>0 ); pTS->aaOutput[iOut] = aMerge; pTS->anOutput[iOut] = nMerge; break; }else{ char *aNew; int nNew; int rc = fts3DoclistOrMerge(p->bDescIdx, aMerge, nMerge, pTS->aaOutput[iOut], pTS->anOutput[iOut], &aNew, &nNew ); if( rc!=SQLITE_OK ){ if( aMerge!=aDoclist ) sqlite3_free(aMerge); return rc; } if( aMerge!=aDoclist ) sqlite3_free(aMerge); sqlite3_free(pTS->aaOutput[iOut]); pTS->aaOutput[iOut] = 0; aMerge = aNew; nMerge = nNew; if( (iOut+1)==SizeofArray(pTS->aaOutput) ){ pTS->aaOutput[iOut] = aMerge; pTS->anOutput[iOut] = nMerge; } } } } return SQLITE_OK; } /* ** Append SegReader object pNew to the end of the pCsr->apSegment[] array. */ static int fts3SegReaderCursorAppend( Fts3MultiSegReader *pCsr, Fts3SegReader *pNew ){ if( (pCsr->nSegment%16)==0 ){ Fts3SegReader **apNew; int nByte = (pCsr->nSegment + 16)*sizeof(Fts3SegReader*); apNew = (Fts3SegReader **)sqlite3_realloc(pCsr->apSegment, nByte); if( !apNew ){ sqlite3Fts3SegReaderFree(pNew); return SQLITE_NOMEM; } pCsr->apSegment = apNew; } pCsr->apSegment[pCsr->nSegment++] = pNew; return SQLITE_OK; } static int fts3SegReaderCursor( Fts3Table *p, /* FTS3 table handle */ int iIndex, /* Index to search (from 0 to p->nIndex-1) */ int iLevel, /* Level of segments to scan */ const char *zTerm, /* Term to query for */ int nTerm, /* Size of zTerm in bytes */ int isPrefix, /* True for a prefix search */ int isScan, /* True to scan from zTerm to EOF */ Fts3MultiSegReader *pCsr /* Cursor object to populate */ ){ int rc = SQLITE_OK; int rc2; sqlite3_stmt *pStmt = 0; /* If iLevel is less than 0 and this is not a scan, include a seg-reader ** for the pending-terms. If this is a scan, then this call must be being ** made by an fts4aux module, not an FTS table. In this case calling ** Fts3SegReaderPending might segfault, as the data structures used by ** fts4aux are not completely populated. So it's easiest to filter these ** calls out here. */ if( iLevel<0 && p->aIndex ){ Fts3SegReader *pSeg = 0; rc = sqlite3Fts3SegReaderPending(p, iIndex, zTerm, nTerm, isPrefix, &pSeg); if( rc==SQLITE_OK && pSeg ){ rc = fts3SegReaderCursorAppend(pCsr, pSeg); } } if( iLevel!=FTS3_SEGCURSOR_PENDING ){ if( rc==SQLITE_OK ){ rc = sqlite3Fts3AllSegdirs(p, iIndex, iLevel, &pStmt); } while( rc==SQLITE_OK && SQLITE_ROW==(rc = sqlite3_step(pStmt)) ){ Fts3SegReader *pSeg = 0; /* Read the values returned by the SELECT into local variables. */ sqlite3_int64 iStartBlock = sqlite3_column_int64(pStmt, 1); sqlite3_int64 iLeavesEndBlock = sqlite3_column_int64(pStmt, 2); sqlite3_int64 iEndBlock = sqlite3_column_int64(pStmt, 3); int nRoot = sqlite3_column_bytes(pStmt, 4); char const *zRoot = sqlite3_column_blob(pStmt, 4); /* If zTerm is not NULL, and this segment is not stored entirely on its ** root node, the range of leaves scanned can be reduced. Do this. */ if( iStartBlock && zTerm ){ sqlite3_int64 *pi = (isPrefix ? &iLeavesEndBlock : 0); rc = fts3SelectLeaf(p, zTerm, nTerm, zRoot, nRoot, &iStartBlock, pi); if( rc!=SQLITE_OK ) goto finished; if( isPrefix==0 && isScan==0 ) iLeavesEndBlock = iStartBlock; } rc = sqlite3Fts3SegReaderNew(pCsr->nSegment+1, iStartBlock, iLeavesEndBlock, iEndBlock, zRoot, nRoot, &pSeg ); if( rc!=SQLITE_OK ) goto finished; rc = fts3SegReaderCursorAppend(pCsr, pSeg); } } finished: rc2 = sqlite3_reset(pStmt); if( rc==SQLITE_DONE ) rc = rc2; return rc; } /* ** Set up a cursor object for iterating through a full-text index or a ** single level therein. */ SQLITE_PRIVATE int sqlite3Fts3SegReaderCursor( Fts3Table *p, /* FTS3 table handle */ int iIndex, /* Index to search (from 0 to p->nIndex-1) */ int iLevel, /* Level of segments to scan */ const char *zTerm, /* Term to query for */ int nTerm, /* Size of zTerm in bytes */ int isPrefix, /* True for a prefix search */ int isScan, /* True to scan from zTerm to EOF */ Fts3MultiSegReader *pCsr /* Cursor object to populate */ ){ assert( iIndex>=0 && iIndex<p->nIndex ); assert( iLevel==FTS3_SEGCURSOR_ALL || iLevel==FTS3_SEGCURSOR_PENDING || iLevel>=0 ); assert( iLevel<FTS3_SEGDIR_MAXLEVEL ); assert( FTS3_SEGCURSOR_ALL<0 && FTS3_SEGCURSOR_PENDING<0 ); assert( isPrefix==0 || isScan==0 ); /* "isScan" is only set to true by the ft4aux module, an ordinary ** full-text tables. */ assert( isScan==0 || p->aIndex==0 ); memset(pCsr, 0, sizeof(Fts3MultiSegReader)); return fts3SegReaderCursor( p, iIndex, iLevel, zTerm, nTerm, isPrefix, isScan, pCsr ); } static int fts3SegReaderCursorAddZero( Fts3Table *p, const char *zTerm, int nTerm, Fts3MultiSegReader *pCsr ){ return fts3SegReaderCursor(p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0,pCsr); } SQLITE_PRIVATE int sqlite3Fts3TermSegReaderCursor( Fts3Cursor *pCsr, /* Virtual table cursor handle */ const char *zTerm, /* Term to query for */ int nTerm, /* Size of zTerm in bytes */ int isPrefix, /* True for a prefix search */ Fts3MultiSegReader **ppSegcsr /* OUT: Allocated seg-reader cursor */ ){ Fts3MultiSegReader *pSegcsr; /* Object to allocate and return */ int rc = SQLITE_NOMEM; /* Return code */ pSegcsr = sqlite3_malloc(sizeof(Fts3MultiSegReader)); if( pSegcsr ){ int i; int bFound = 0; /* True once an index has been found */ Fts3Table *p = (Fts3Table *)pCsr->base.pVtab; if( isPrefix ){ for(i=1; bFound==0 && i<p->nIndex; i++){ if( p->aIndex[i].nPrefix==nTerm ){ bFound = 1; rc = sqlite3Fts3SegReaderCursor( p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 0, 0, pSegcsr); pSegcsr->bLookup = 1; } } for(i=1; bFound==0 && i<p->nIndex; i++){ if( p->aIndex[i].nPrefix==nTerm+1 ){ bFound = 1; rc = sqlite3Fts3SegReaderCursor( p, i, FTS3_SEGCURSOR_ALL, zTerm, nTerm, 1, 0, pSegcsr ); if( rc==SQLITE_OK ){ rc = fts3SegReaderCursorAddZero(p, zTerm, nTerm, pSegcsr); } } } } if( bFound==0 ){ rc = sqlite3Fts3SegReaderCursor( p, 0, FTS3_SEGCURSOR_ALL, zTerm, nTerm, isPrefix, 0, pSegcsr ); pSegcsr->bLookup = !isPrefix; } } *ppSegcsr = pSegcsr; return rc; } static void fts3SegReaderCursorFree(Fts3MultiSegReader *pSegcsr){ sqlite3Fts3SegReaderFinish(pSegcsr); sqlite3_free(pSegcsr); } /* ** This function retreives the doclist for the specified term (or term ** prefix) from the database. |
︙ | ︙ | |||
114012 114013 114014 114015 114016 114017 114018 | Fts3PhraseToken *pTok, /* Token to query for */ int iColumn, /* Column to query (or -ve for all columns) */ int isReqPos, /* True to include position lists in output */ int *pnOut, /* OUT: Size of buffer at *ppOut */ char **ppOut /* OUT: Malloced result buffer */ ){ int rc; /* Return code */ | | | 114363 114364 114365 114366 114367 114368 114369 114370 114371 114372 114373 114374 114375 114376 114377 | Fts3PhraseToken *pTok, /* Token to query for */ int iColumn, /* Column to query (or -ve for all columns) */ int isReqPos, /* True to include position lists in output */ int *pnOut, /* OUT: Size of buffer at *ppOut */ char **ppOut /* OUT: Malloced result buffer */ ){ int rc; /* Return code */ Fts3MultiSegReader *pSegcsr; /* Seg-reader cursor for this term */ TermSelect tsc; /* Context object for fts3TermSelectCb() */ Fts3SegFilter filter; /* Segment term filter configuration */ pSegcsr = pTok->pSegcsr; memset(&tsc, 0, sizeof(TermSelect)); tsc.isReqPos = isReqPos; |
︙ | ︙ | |||
114038 114039 114040 114041 114042 114043 114044 | ){ rc = fts3TermSelectCb(p, (void *)&tsc, pSegcsr->zTerm, pSegcsr->nTerm, pSegcsr->aDoclist, pSegcsr->nDoclist ); } if( rc==SQLITE_OK ){ | | | 114389 114390 114391 114392 114393 114394 114395 114396 114397 114398 114399 114400 114401 114402 114403 | ){ rc = fts3TermSelectCb(p, (void *)&tsc, pSegcsr->zTerm, pSegcsr->nTerm, pSegcsr->aDoclist, pSegcsr->nDoclist ); } if( rc==SQLITE_OK ){ rc = fts3TermSelectMerge(p, &tsc); } if( rc==SQLITE_OK ){ *ppOut = tsc.aaOutput[0]; *pnOut = tsc.anOutput[0]; }else{ int i; for(i=0; i<SizeofArray(tsc.aaOutput); i++){ |
︙ | ︙ | |||
114088 114089 114090 114091 114092 114093 114094 | } } } return nDoc; } | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < | < < < | | | < | < < < | < | < | < < < < < < | < < | < | | 114439 114440 114441 114442 114443 114444 114445 114446 114447 114448 114449 114450 114451 114452 114453 114454 114455 114456 114457 114458 114459 114460 114461 114462 114463 114464 114465 114466 114467 114468 114469 114470 114471 114472 114473 114474 114475 114476 114477 114478 | } } } return nDoc; } /* ** Advance the cursor to the next row in the %_content table that ** matches the search criteria. For a MATCH search, this will be ** the next row that matches. For a full-table scan, this will be ** simply the next row in the %_content table. For a docid lookup, ** this routine simply sets the EOF flag. ** ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned ** even if we reach end-of-file. The fts3EofMethod() will be called ** subsequently to determine whether or not an EOF was hit. */ static int fts3NextMethod(sqlite3_vtab_cursor *pCursor){ int rc; Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; if( pCsr->eSearch==FTS3_DOCID_SEARCH || pCsr->eSearch==FTS3_FULLSCAN_SEARCH ){ if( SQLITE_ROW!=sqlite3_step(pCsr->pStmt) ){ pCsr->isEof = 1; rc = sqlite3_reset(pCsr->pStmt); }else{ pCsr->iPrevId = sqlite3_column_int64(pCsr->pStmt, 0); rc = SQLITE_OK; } }else{ rc = sqlite3Fts3EvalNext((Fts3Cursor *)pCursor); } assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); return rc; } /* ** This is the xFilter interface for the virtual table. See ** the virtual table xFilter method documentation for additional ** information. |
︙ | ︙ | |||
114813 114814 114815 114816 114817 114818 114819 | static int fts3FilterMethod( sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ int idxNum, /* Strategy index */ const char *idxStr, /* Unused */ int nVal, /* Number of elements in apVal */ sqlite3_value **apVal /* Arguments for the indexing scheme */ ){ | < < < < | > > > > > > > > | < | > | > > | > > | < > | < < | | > | | < < < < | < < < < < | < < < < < < < | < | < < < < < < < < | < < | < > > | 114491 114492 114493 114494 114495 114496 114497 114498 114499 114500 114501 114502 114503 114504 114505 114506 114507 114508 114509 114510 114511 114512 114513 114514 114515 114516 114517 114518 114519 114520 114521 114522 114523 114524 114525 114526 114527 114528 114529 114530 114531 114532 114533 114534 114535 114536 114537 114538 114539 114540 114541 114542 114543 114544 114545 114546 114547 114548 114549 114550 114551 114552 114553 114554 114555 114556 114557 114558 114559 114560 114561 114562 114563 114564 114565 114566 114567 114568 114569 114570 114571 114572 114573 114574 114575 114576 114577 114578 114579 114580 114581 114582 114583 114584 114585 114586 114587 114588 114589 114590 114591 114592 114593 114594 114595 114596 114597 114598 114599 114600 114601 114602 114603 114604 114605 114606 114607 114608 114609 114610 114611 114612 114613 114614 114615 114616 114617 114618 114619 114620 114621 114622 114623 114624 114625 114626 114627 114628 114629 114630 114631 114632 114633 114634 114635 114636 114637 114638 114639 114640 | static int fts3FilterMethod( sqlite3_vtab_cursor *pCursor, /* The cursor used for this query */ int idxNum, /* Strategy index */ const char *idxStr, /* Unused */ int nVal, /* Number of elements in apVal */ sqlite3_value **apVal /* Arguments for the indexing scheme */ ){ int rc; char *zSql; /* SQL statement used to access %_content */ Fts3Table *p = (Fts3Table *)pCursor->pVtab; Fts3Cursor *pCsr = (Fts3Cursor *)pCursor; UNUSED_PARAMETER(idxStr); UNUSED_PARAMETER(nVal); assert( idxNum>=0 && idxNum<=(FTS3_FULLTEXT_SEARCH+p->nColumn) ); assert( nVal==0 || nVal==1 ); assert( (nVal==0)==(idxNum==FTS3_FULLSCAN_SEARCH) ); assert( p->pSegments==0 ); /* In case the cursor has been used before, clear it now. */ sqlite3_finalize(pCsr->pStmt); sqlite3_free(pCsr->aDoclist); sqlite3Fts3ExprFree(pCsr->pExpr); memset(&pCursor[1], 0, sizeof(Fts3Cursor)-sizeof(sqlite3_vtab_cursor)); if( idxStr ){ pCsr->bDesc = (idxStr[0]=='D'); }else{ pCsr->bDesc = p->bDescIdx; } pCsr->eSearch = (i16)idxNum; if( idxNum!=FTS3_DOCID_SEARCH && idxNum!=FTS3_FULLSCAN_SEARCH ){ int iCol = idxNum-FTS3_FULLTEXT_SEARCH; const char *zQuery = (const char *)sqlite3_value_text(apVal[0]); if( zQuery==0 && sqlite3_value_type(apVal[0])!=SQLITE_NULL ){ return SQLITE_NOMEM; } rc = sqlite3Fts3ExprParse(p->pTokenizer, p->azColumn, p->nColumn, iCol, zQuery, -1, &pCsr->pExpr ); if( rc!=SQLITE_OK ){ if( rc==SQLITE_ERROR ){ static const char *zErr = "malformed MATCH expression: [%s]"; p->base.zErrMsg = sqlite3_mprintf(zErr, zQuery); } return rc; } rc = sqlite3Fts3ReadLock(p); if( rc!=SQLITE_OK ) return rc; rc = sqlite3Fts3EvalStart(pCsr, pCsr->pExpr, 1); sqlite3Fts3SegmentsClose(p); if( rc!=SQLITE_OK ) return rc; pCsr->pNextId = pCsr->aDoclist; pCsr->iPrevId = 0; } /* Compile a SELECT statement for this cursor. For a full-table-scan, the ** statement loops through all rows of the %_content table. For a ** full-text query or docid lookup, the statement retrieves a single ** row by docid. */ if( idxNum==FTS3_FULLSCAN_SEARCH ){ const char *zSort = (pCsr->bDesc ? "DESC" : "ASC"); const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x ORDER BY docid %s"; zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName, zSort); }else{ const char *zTmpl = "SELECT %s FROM %Q.'%q_content' AS x WHERE docid = ?"; zSql = sqlite3_mprintf(zTmpl, p->zReadExprlist, p->zDb, p->zName); } if( !zSql ) return SQLITE_NOMEM; rc = sqlite3_prepare_v2(p->db, zSql, -1, &pCsr->pStmt, 0); sqlite3_free(zSql); if( rc!=SQLITE_OK ) return rc; if( idxNum==FTS3_DOCID_SEARCH ){ rc = sqlite3_bind_value(pCsr->pStmt, 1, apVal[0]); if( rc!=SQLITE_OK ) return rc; } return fts3NextMethod(pCursor); } /* ** This is the xEof method of the virtual table. SQLite calls this ** routine to find out if it has reached the end of a result set. */ static int fts3EofMethod(sqlite3_vtab_cursor *pCursor){ return ((Fts3Cursor *)pCursor)->isEof; } /* ** This is the xRowid method. The SQLite core calls this routine to ** retrieve the rowid for the current row of the result set. fts3 ** exposes %_content.docid as the rowid for the virtual table. The ** rowid should be written to *pRowid. */ static int fts3RowidMethod(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ Fts3Cursor *pCsr = (Fts3Cursor *) pCursor; *pRowid = pCsr->iPrevId; return SQLITE_OK; } /* ** This is the xColumn method, called by SQLite to request a value from ** the row that the supplied cursor currently points to. */ static int fts3ColumnMethod( sqlite3_vtab_cursor *pCursor, /* Cursor to retrieve value from */ sqlite3_context *pContext, /* Context for sqlite3_result_xxx() calls */ int iCol /* Index of column to read value from */ ){ int rc = SQLITE_OK; /* Return Code */ Fts3Cursor *pCsr = (Fts3Cursor *) pCursor; Fts3Table *p = (Fts3Table *)pCursor->pVtab; /* The column value supplied by SQLite must be in range. */ assert( iCol>=0 && iCol<=p->nColumn+1 ); if( iCol==p->nColumn+1 ){ /* This call is a request for the "docid" column. Since "docid" is an ** alias for "rowid", use the xRowid() method to obtain the value. */ sqlite3_result_int64(pContext, pCsr->iPrevId); }else if( iCol==p->nColumn ){ /* The extra column whose name is the same as the table. ** Return a blob which is a pointer to the cursor. */ sqlite3_result_blob(pContext, &pCsr, sizeof(pCsr), SQLITE_TRANSIENT); }else{ rc = fts3CursorSeek(0, pCsr); if( rc==SQLITE_OK ){ sqlite3_result_value(pContext, sqlite3_column_value(pCsr->pStmt, iCol+1)); } } assert( ((Fts3Table *)pCsr->base.pVtab)->pSegments==0 ); return rc; } /* ** This function is the implementation of the xUpdate callback used by ** FTS3 virtual tables. It is invoked by SQLite each time a row is to be ** inserted, updated or deleted. |
︙ | ︙ | |||
114999 115000 115001 115002 115003 115004 115005 | return rc; } /* ** Implementation of xBegin() method. This is a no-op. */ static int fts3BeginMethod(sqlite3_vtab *pVtab){ | < > > < > > < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | > > < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 114658 114659 114660 114661 114662 114663 114664 114665 114666 114667 114668 114669 114670 114671 114672 114673 114674 114675 114676 114677 114678 114679 114680 114681 114682 114683 114684 114685 114686 114687 114688 114689 114690 114691 114692 114693 114694 114695 114696 114697 114698 114699 114700 114701 114702 114703 114704 114705 114706 114707 114708 114709 114710 114711 114712 114713 114714 114715 114716 114717 114718 114719 114720 114721 114722 114723 114724 114725 114726 114727 114728 114729 | return rc; } /* ** Implementation of xBegin() method. This is a no-op. */ static int fts3BeginMethod(sqlite3_vtab *pVtab){ TESTONLY( Fts3Table *p = (Fts3Table*)pVtab ); UNUSED_PARAMETER(pVtab); assert( p->pSegments==0 ); assert( p->nPendingData==0 ); assert( p->inTransaction!=1 ); TESTONLY( p->inTransaction = 1 ); TESTONLY( p->mxSavepoint = -1; ); return SQLITE_OK; } /* ** Implementation of xCommit() method. This is a no-op. The contents of ** the pending-terms hash-table have already been flushed into the database ** by fts3SyncMethod(). */ static int fts3CommitMethod(sqlite3_vtab *pVtab){ TESTONLY( Fts3Table *p = (Fts3Table*)pVtab ); UNUSED_PARAMETER(pVtab); assert( p->nPendingData==0 ); assert( p->inTransaction!=0 ); assert( p->pSegments==0 ); TESTONLY( p->inTransaction = 0 ); TESTONLY( p->mxSavepoint = -1; ); return SQLITE_OK; } /* ** Implementation of xRollback(). Discard the contents of the pending-terms ** hash-table. Any changes made to the database are reverted by SQLite. */ static int fts3RollbackMethod(sqlite3_vtab *pVtab){ Fts3Table *p = (Fts3Table*)pVtab; sqlite3Fts3PendingTermsClear(p); assert( p->inTransaction!=0 ); TESTONLY( p->inTransaction = 0 ); TESTONLY( p->mxSavepoint = -1; ); return SQLITE_OK; } /* ** When called, *ppPoslist must point to the byte immediately following the ** end of a position-list. i.e. ( (*ppPoslist)[-1]==POS_END ). This function ** moves *ppPoslist so that it instead points to the first byte of the ** same position list. */ static void fts3ReversePoslist(char *pStart, char **ppPoslist){ char *p = &(*ppPoslist)[-2]; char c; while( p>pStart && (c=*p--)==0 ); while( p>pStart && (*p & 0x80) | c ){ c = *p--; } if( p>pStart ){ p = &p[2]; } while( *p++&0x80 ); *ppPoslist = p; } /* ** Helper function used by the implementation of the overloaded snippet(), ** offsets() and optimize() SQL functions. ** ** If the value passed as the third argument is a blob of size ** sizeof(Fts3Cursor*), then the blob contents are copied to the ** output variable *ppCsr and SQLITE_OK is returned. Otherwise, an error |
︙ | ︙ | |||
115388 115389 115390 115391 115392 115393 115394 | "ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';", p->zDb, p->zName, zName ); return rc; } static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ | < | | | | | 114947 114948 114949 114950 114951 114952 114953 114954 114955 114956 114957 114958 114959 114960 114961 114962 114963 114964 114965 | "ALTER TABLE %Q.'%q_segdir' RENAME TO '%q_segdir';", p->zDb, p->zName, zName ); return rc; } static int fts3SavepointMethod(sqlite3_vtab *pVtab, int iSavepoint){ UNUSED_PARAMETER(iSavepoint); assert( ((Fts3Table *)pVtab)->inTransaction ); assert( ((Fts3Table *)pVtab)->mxSavepoint < iSavepoint ); TESTONLY( ((Fts3Table *)pVtab)->mxSavepoint = iSavepoint ); return fts3SyncMethod(pVtab); } static int fts3ReleaseMethod(sqlite3_vtab *pVtab, int iSavepoint){ TESTONLY( Fts3Table *p = (Fts3Table*)pVtab ); UNUSED_PARAMETER(iSavepoint); UNUSED_PARAMETER(pVtab); assert( p->inTransaction ); assert( p->mxSavepoint >= iSavepoint ); |
︙ | ︙ | |||
115564 115565 115566 115567 115568 115569 115570 115571 115572 115573 115574 115575 115576 115577 115578 115579 115580 115581 115582 115583 115584 115585 115586 115587 | const sqlite3_api_routines *pApi ){ SQLITE_EXTENSION_INIT2(pApi) return sqlite3Fts3Init(db); } #endif #endif /************** End of fts3.c ************************************************/ /************** Begin file fts3_aux.c ****************************************/ /* ** 2011 Jan 27 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < | | 115122 115123 115124 115125 115126 115127 115128 115129 115130 115131 115132 115133 115134 115135 115136 115137 115138 115139 115140 115141 115142 115143 115144 115145 115146 115147 115148 115149 115150 115151 115152 115153 115154 115155 115156 115157 115158 115159 115160 115161 115162 115163 115164 115165 115166 115167 115168 115169 115170 115171 115172 115173 115174 115175 115176 115177 115178 115179 115180 115181 115182 115183 115184 115185 115186 115187 115188 115189 115190 115191 115192 115193 115194 115195 115196 115197 115198 115199 115200 115201 115202 115203 115204 115205 115206 115207 115208 115209 115210 115211 115212 115213 115214 115215 115216 115217 115218 115219 115220 115221 115222 115223 115224 115225 115226 115227 115228 115229 115230 115231 115232 115233 115234 115235 115236 115237 115238 115239 115240 115241 115242 115243 115244 115245 115246 115247 115248 115249 115250 115251 115252 115253 115254 115255 115256 115257 115258 115259 115260 115261 115262 115263 115264 115265 115266 115267 115268 115269 115270 115271 115272 115273 115274 115275 115276 115277 115278 115279 115280 115281 115282 115283 115284 115285 115286 115287 115288 115289 115290 115291 115292 115293 115294 115295 115296 115297 115298 115299 115300 115301 115302 115303 115304 115305 115306 115307 115308 115309 115310 115311 115312 115313 115314 115315 115316 115317 115318 115319 115320 115321 115322 115323 115324 115325 115326 115327 115328 115329 115330 115331 115332 115333 115334 115335 115336 115337 115338 115339 115340 115341 115342 115343 115344 115345 115346 115347 115348 115349 115350 115351 115352 115353 115354 115355 115356 115357 115358 115359 115360 115361 115362 115363 115364 115365 115366 115367 115368 115369 115370 115371 115372 115373 115374 115375 115376 115377 115378 115379 115380 115381 115382 115383 115384 115385 115386 115387 115388 115389 115390 115391 115392 115393 115394 115395 115396 115397 115398 115399 115400 115401 115402 115403 115404 115405 115406 115407 115408 115409 115410 115411 115412 115413 115414 115415 115416 115417 115418 115419 115420 115421 115422 115423 115424 115425 115426 115427 115428 115429 115430 115431 115432 115433 115434 115435 115436 115437 115438 115439 115440 115441 115442 115443 115444 115445 115446 115447 115448 115449 115450 115451 115452 115453 115454 115455 115456 115457 115458 115459 115460 115461 115462 115463 115464 115465 115466 115467 115468 115469 115470 115471 115472 115473 115474 115475 115476 115477 115478 115479 115480 115481 115482 115483 115484 115485 115486 115487 115488 115489 115490 115491 115492 115493 115494 115495 115496 115497 115498 115499 115500 115501 115502 115503 115504 115505 115506 115507 115508 115509 115510 115511 115512 115513 115514 115515 115516 115517 115518 115519 115520 115521 115522 115523 115524 115525 115526 115527 115528 115529 115530 115531 115532 115533 115534 115535 115536 115537 115538 115539 115540 115541 115542 115543 115544 115545 115546 115547 115548 115549 115550 115551 115552 115553 115554 115555 115556 115557 115558 115559 115560 115561 115562 115563 115564 115565 115566 115567 115568 115569 115570 115571 115572 115573 115574 115575 115576 115577 115578 115579 115580 115581 115582 115583 115584 115585 115586 115587 115588 115589 115590 115591 115592 115593 115594 115595 115596 115597 115598 115599 115600 115601 115602 115603 115604 115605 115606 115607 115608 115609 115610 115611 115612 115613 115614 115615 115616 115617 115618 115619 115620 115621 115622 115623 115624 115625 115626 115627 115628 115629 115630 115631 115632 115633 115634 115635 115636 115637 115638 115639 115640 115641 115642 115643 115644 115645 115646 115647 115648 115649 115650 115651 115652 115653 115654 115655 115656 115657 115658 115659 115660 115661 115662 115663 115664 115665 115666 115667 115668 115669 115670 115671 115672 115673 115674 115675 115676 115677 115678 115679 115680 115681 115682 115683 115684 115685 115686 115687 115688 115689 115690 115691 115692 115693 115694 115695 115696 115697 115698 115699 115700 115701 115702 115703 115704 115705 115706 115707 115708 115709 115710 115711 115712 115713 115714 115715 115716 115717 115718 115719 115720 115721 115722 115723 115724 115725 115726 115727 115728 115729 115730 115731 115732 115733 115734 115735 115736 115737 115738 115739 115740 115741 115742 115743 115744 115745 115746 115747 115748 115749 115750 115751 115752 115753 115754 115755 115756 115757 115758 115759 115760 115761 115762 115763 115764 115765 115766 115767 115768 115769 115770 115771 115772 115773 115774 115775 115776 115777 115778 115779 115780 115781 115782 115783 115784 115785 115786 115787 115788 115789 115790 115791 115792 115793 115794 115795 115796 115797 115798 115799 115800 115801 115802 115803 115804 115805 115806 115807 115808 115809 115810 115811 115812 115813 115814 115815 115816 115817 115818 115819 115820 115821 115822 115823 115824 115825 115826 115827 115828 115829 115830 115831 115832 115833 115834 115835 115836 115837 115838 115839 115840 115841 115842 115843 115844 115845 115846 115847 115848 115849 115850 115851 115852 115853 115854 115855 115856 115857 115858 115859 115860 115861 115862 115863 115864 115865 115866 115867 115868 115869 115870 115871 115872 115873 115874 115875 115876 115877 115878 115879 115880 115881 115882 115883 115884 115885 115886 115887 115888 115889 115890 115891 115892 115893 115894 115895 115896 115897 115898 115899 115900 115901 115902 115903 115904 115905 115906 115907 115908 115909 115910 115911 115912 115913 115914 115915 115916 115917 115918 115919 115920 115921 115922 115923 115924 115925 115926 115927 115928 115929 115930 115931 115932 115933 115934 115935 115936 115937 115938 115939 115940 115941 115942 115943 115944 115945 115946 115947 115948 115949 115950 115951 115952 115953 115954 115955 115956 115957 115958 115959 115960 115961 115962 115963 115964 115965 115966 115967 115968 115969 115970 115971 115972 115973 115974 115975 115976 115977 115978 115979 115980 115981 115982 115983 115984 115985 115986 115987 115988 115989 115990 115991 115992 115993 115994 115995 115996 115997 115998 115999 116000 116001 116002 116003 116004 116005 116006 116007 116008 116009 116010 116011 116012 116013 116014 116015 116016 116017 116018 116019 116020 116021 116022 116023 116024 116025 116026 116027 116028 116029 116030 116031 116032 116033 116034 116035 116036 116037 116038 116039 116040 116041 116042 116043 116044 116045 116046 116047 116048 116049 116050 116051 116052 116053 116054 116055 116056 116057 116058 116059 116060 116061 116062 116063 116064 116065 116066 116067 116068 116069 116070 116071 116072 116073 116074 116075 116076 116077 116078 116079 116080 116081 116082 116083 116084 116085 116086 116087 116088 116089 116090 116091 116092 116093 116094 116095 116096 116097 116098 116099 116100 116101 116102 116103 116104 116105 116106 116107 116108 116109 116110 116111 116112 116113 116114 116115 116116 116117 116118 116119 116120 116121 116122 116123 116124 116125 116126 116127 116128 116129 116130 116131 116132 116133 116134 116135 116136 116137 116138 116139 116140 116141 116142 116143 116144 116145 116146 116147 116148 116149 116150 116151 116152 116153 116154 116155 116156 116157 116158 116159 116160 116161 116162 116163 116164 116165 116166 116167 116168 116169 116170 116171 116172 116173 116174 116175 116176 116177 116178 116179 116180 116181 116182 116183 116184 116185 116186 116187 116188 116189 116190 116191 116192 116193 116194 116195 116196 116197 116198 116199 116200 116201 116202 116203 116204 116205 116206 116207 116208 116209 116210 116211 116212 116213 116214 116215 116216 116217 116218 116219 116220 116221 116222 116223 116224 116225 116226 116227 116228 116229 116230 116231 116232 116233 116234 116235 116236 116237 116238 116239 116240 116241 116242 116243 116244 116245 116246 116247 116248 116249 116250 116251 116252 116253 116254 116255 116256 116257 116258 116259 116260 116261 116262 116263 116264 116265 116266 116267 116268 116269 116270 116271 116272 116273 116274 116275 116276 116277 116278 116279 116280 116281 116282 116283 116284 116285 116286 116287 116288 116289 116290 116291 116292 116293 116294 116295 116296 116297 116298 116299 116300 116301 116302 116303 116304 116305 116306 116307 116308 116309 116310 116311 116312 116313 116314 116315 116316 116317 116318 116319 116320 116321 116322 116323 116324 116325 116326 116327 116328 116329 116330 116331 116332 116333 116334 116335 116336 116337 116338 116339 116340 116341 116342 116343 116344 116345 116346 116347 116348 116349 116350 116351 116352 116353 116354 116355 116356 116357 116358 116359 116360 116361 116362 116363 116364 116365 116366 116367 116368 116369 116370 116371 116372 116373 116374 116375 116376 116377 116378 116379 116380 116381 116382 116383 116384 116385 116386 116387 116388 116389 116390 116391 116392 116393 116394 116395 116396 116397 116398 116399 116400 116401 116402 116403 116404 116405 116406 116407 116408 116409 116410 116411 116412 116413 116414 116415 116416 116417 116418 116419 116420 116421 116422 116423 116424 116425 116426 116427 116428 116429 116430 116431 116432 116433 116434 116435 116436 116437 116438 116439 116440 116441 116442 116443 116444 116445 116446 116447 116448 116449 116450 116451 116452 116453 116454 116455 116456 116457 116458 116459 116460 116461 116462 116463 116464 116465 116466 116467 116468 | const sqlite3_api_routines *pApi ){ SQLITE_EXTENSION_INIT2(pApi) return sqlite3Fts3Init(db); } #endif /* ** Allocate an Fts3MultiSegReader for each token in the expression headed ** by pExpr. ** ** An Fts3SegReader object is a cursor that can seek or scan a range of ** entries within a single segment b-tree. An Fts3MultiSegReader uses multiple ** Fts3SegReader objects internally to provide an interface to seek or scan ** within the union of all segments of a b-tree. Hence the name. ** ** If the allocated Fts3MultiSegReader just seeks to a single entry in a ** segment b-tree (if the term is not a prefix or it is a prefix for which ** there exists prefix b-tree of the right length) then it may be traversed ** and merged incrementally. Otherwise, it has to be merged into an in-memory ** doclist and then traversed. */ static void fts3EvalAllocateReaders( Fts3Cursor *pCsr, Fts3Expr *pExpr, int *pnToken, /* OUT: Total number of tokens in phrase. */ int *pnOr, /* OUT: Total number of OR nodes in expr. */ int *pRc ){ if( pExpr && SQLITE_OK==*pRc ){ if( pExpr->eType==FTSQUERY_PHRASE ){ int i; int nToken = pExpr->pPhrase->nToken; *pnToken += nToken; for(i=0; i<nToken; i++){ Fts3PhraseToken *pToken = &pExpr->pPhrase->aToken[i]; int rc = sqlite3Fts3TermSegReaderCursor(pCsr, pToken->z, pToken->n, pToken->isPrefix, &pToken->pSegcsr ); if( rc!=SQLITE_OK ){ *pRc = rc; return; } } assert( pExpr->pPhrase->iDoclistToken==0 ); pExpr->pPhrase->iDoclistToken = -1; }else{ *pnOr += (pExpr->eType==FTSQUERY_OR); fts3EvalAllocateReaders(pCsr, pExpr->pLeft, pnToken, pnOr, pRc); fts3EvalAllocateReaders(pCsr, pExpr->pRight, pnToken, pnOr, pRc); } } } static void fts3EvalPhraseMergeToken( Fts3Table *pTab, Fts3Phrase *p, int iToken, char *pList, int nList ){ assert( iToken!=p->iDoclistToken ); if( pList==0 ){ sqlite3_free(p->doclist.aAll); p->doclist.aAll = 0; p->doclist.nAll = 0; } else if( p->iDoclistToken<0 ){ p->doclist.aAll = pList; p->doclist.nAll = nList; } else if( p->doclist.aAll==0 ){ sqlite3_free(pList); } else { char *pLeft; char *pRight; int nLeft; int nRight; int nDiff; if( p->iDoclistToken<iToken ){ pLeft = p->doclist.aAll; nLeft = p->doclist.nAll; pRight = pList; nRight = nList; nDiff = iToken - p->iDoclistToken; }else{ pRight = p->doclist.aAll; nRight = p->doclist.nAll; pLeft = pList; nLeft = nList; nDiff = p->iDoclistToken - iToken; } fts3DoclistPhraseMerge(pTab->bDescIdx, nDiff, pLeft, nLeft, pRight,&nRight); sqlite3_free(pLeft); p->doclist.aAll = pRight; p->doclist.nAll = nRight; } if( iToken>p->iDoclistToken ) p->iDoclistToken = iToken; } static int fts3EvalPhraseLoad( Fts3Cursor *pCsr, Fts3Phrase *p ){ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; int iToken; int rc = SQLITE_OK; for(iToken=0; rc==SQLITE_OK && iToken<p->nToken; iToken++){ Fts3PhraseToken *pToken = &p->aToken[iToken]; assert( pToken->pDeferred==0 || pToken->pSegcsr==0 ); if( pToken->pSegcsr ){ int nThis = 0; char *pThis = 0; rc = fts3TermSelect(pTab, pToken, p->iColumn, 1, &nThis, &pThis); if( rc==SQLITE_OK ){ fts3EvalPhraseMergeToken(pTab, p, iToken, pThis, nThis); } } assert( pToken->pSegcsr==0 ); } return rc; } static int fts3EvalDeferredPhrase(Fts3Cursor *pCsr, Fts3Phrase *pPhrase){ int iToken; int rc = SQLITE_OK; int nMaxUndeferred = pPhrase->iDoclistToken; char *aPoslist = 0; int nPoslist = 0; int iPrev = -1; assert( pPhrase->doclist.bFreeList==0 ); for(iToken=0; rc==SQLITE_OK && iToken<pPhrase->nToken; iToken++){ Fts3PhraseToken *pToken = &pPhrase->aToken[iToken]; Fts3DeferredToken *pDeferred = pToken->pDeferred; if( pDeferred ){ char *pList; int nList; rc = sqlite3Fts3DeferredTokenList(pDeferred, &pList, &nList); if( rc!=SQLITE_OK ) return rc; if( pList==0 ){ sqlite3_free(aPoslist); pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; return SQLITE_OK; }else if( aPoslist==0 ){ aPoslist = pList; nPoslist = nList; }else{ char *aOut = pList; char *p1 = aPoslist; char *p2 = aOut; assert( iPrev>=0 ); fts3PoslistPhraseMerge(&aOut, iToken-iPrev, 0, 1, &p1, &p2); sqlite3_free(aPoslist); aPoslist = pList; nPoslist = aOut - aPoslist; if( nPoslist==0 ){ sqlite3_free(aPoslist); pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; return SQLITE_OK; } } iPrev = iToken; } } if( iPrev>=0 ){ if( nMaxUndeferred<0 ){ pPhrase->doclist.pList = aPoslist; pPhrase->doclist.nList = nPoslist; pPhrase->doclist.iDocid = pCsr->iPrevId; pPhrase->doclist.bFreeList = 1; }else{ int nDistance; char *p1; char *p2; char *aOut; if( nMaxUndeferred>iPrev ){ p1 = aPoslist; p2 = pPhrase->doclist.pList; nDistance = nMaxUndeferred - iPrev; }else{ p1 = pPhrase->doclist.pList; p2 = aPoslist; nDistance = iPrev - nMaxUndeferred; } aOut = (char *)sqlite3_malloc(nPoslist+8); if( !aOut ){ sqlite3_free(aPoslist); return SQLITE_NOMEM; } pPhrase->doclist.pList = aOut; if( fts3PoslistPhraseMerge(&aOut, nDistance, 0, 1, &p1, &p2) ){ pPhrase->doclist.bFreeList = 1; pPhrase->doclist.nList = (aOut - pPhrase->doclist.pList); }else{ sqlite3_free(aOut); pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; } sqlite3_free(aPoslist); } } return SQLITE_OK; } /* ** This function is called for each Fts3Phrase in a full-text query ** expression to initialize the mechanism for returning rows. Once this ** function has been called successfully on an Fts3Phrase, it may be ** used with fts3EvalPhraseNext() to iterate through the matching docids. */ static int fts3EvalPhraseStart(Fts3Cursor *pCsr, int bOptOk, Fts3Phrase *p){ int rc; Fts3PhraseToken *pFirst = &p->aToken[0]; Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; if( pCsr->bDesc==pTab->bDescIdx && bOptOk==1 && p->nToken==1 && pFirst->pSegcsr && pFirst->pSegcsr->bLookup ){ /* Use the incremental approach. */ int iCol = (p->iColumn >= pTab->nColumn ? -1 : p->iColumn); rc = sqlite3Fts3MsrIncrStart( pTab, pFirst->pSegcsr, iCol, pFirst->z, pFirst->n); p->bIncr = 1; }else{ /* Load the full doclist for the phrase into memory. */ rc = fts3EvalPhraseLoad(pCsr, p); p->bIncr = 0; } assert( rc!=SQLITE_OK || p->nToken<1 || p->aToken[0].pSegcsr==0 || p->bIncr ); return rc; } /* ** This function is used to iterate backwards (from the end to start) ** through doclists. */ SQLITE_PRIVATE void sqlite3Fts3DoclistPrev( int bDescIdx, /* True if the doclist is desc */ char *aDoclist, /* Pointer to entire doclist */ int nDoclist, /* Length of aDoclist in bytes */ char **ppIter, /* IN/OUT: Iterator pointer */ sqlite3_int64 *piDocid, /* IN/OUT: Docid pointer */ int *pnList, /* IN/OUT: List length pointer */ u8 *pbEof /* OUT: End-of-file flag */ ){ char *p = *ppIter; assert( nDoclist>0 ); assert( *pbEof==0 ); assert( p || *piDocid==0 ); assert( !p || (p>aDoclist && p<&aDoclist[nDoclist]) ); if( p==0 ){ sqlite3_int64 iDocid = 0; char *pNext = 0; char *pDocid = aDoclist; char *pEnd = &aDoclist[nDoclist]; int iMul = 1; while( pDocid<pEnd ){ sqlite3_int64 iDelta; pDocid += sqlite3Fts3GetVarint(pDocid, &iDelta); iDocid += (iMul * iDelta); pNext = pDocid; fts3PoslistCopy(0, &pDocid); while( pDocid<pEnd && *pDocid==0 ) pDocid++; iMul = (bDescIdx ? -1 : 1); } *pnList = pEnd - pNext; *ppIter = pNext; *piDocid = iDocid; }else{ int iMul = (bDescIdx ? -1 : 1); sqlite3_int64 iDelta; fts3GetReverseVarint(&p, aDoclist, &iDelta); *piDocid -= (iMul * iDelta); if( p==aDoclist ){ *pbEof = 1; }else{ char *pSave = p; fts3ReversePoslist(aDoclist, &p); *pnList = (pSave - p); } *ppIter = p; } } /* ** Attempt to move the phrase iterator to point to the next matching docid. ** If an error occurs, return an SQLite error code. Otherwise, return ** SQLITE_OK. ** ** If there is no "next" entry and no error occurs, then *pbEof is set to ** 1 before returning. Otherwise, if no error occurs and the iterator is ** successfully advanced, *pbEof is set to 0. */ static int fts3EvalPhraseNext( Fts3Cursor *pCsr, Fts3Phrase *p, u8 *pbEof ){ int rc = SQLITE_OK; Fts3Doclist *pDL = &p->doclist; Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; if( p->bIncr ){ assert( p->nToken==1 ); assert( pDL->pNextDocid==0 ); rc = sqlite3Fts3MsrIncrNext(pTab, p->aToken[0].pSegcsr, &pDL->iDocid, &pDL->pList, &pDL->nList ); if( rc==SQLITE_OK && !pDL->pList ){ *pbEof = 1; } }else if( pCsr->bDesc!=pTab->bDescIdx && pDL->nAll ){ sqlite3Fts3DoclistPrev(pTab->bDescIdx, pDL->aAll, pDL->nAll, &pDL->pNextDocid, &pDL->iDocid, &pDL->nList, pbEof ); pDL->pList = pDL->pNextDocid; }else{ char *pIter; /* Used to iterate through aAll */ char *pEnd = &pDL->aAll[pDL->nAll]; /* 1 byte past end of aAll */ if( pDL->pNextDocid ){ pIter = pDL->pNextDocid; }else{ pIter = pDL->aAll; } if( pIter>=pEnd ){ /* We have already reached the end of this doclist. EOF. */ *pbEof = 1; }else{ sqlite3_int64 iDelta; pIter += sqlite3Fts3GetVarint(pIter, &iDelta); if( pTab->bDescIdx==0 || pDL->pNextDocid==0 ){ pDL->iDocid += iDelta; }else{ pDL->iDocid -= iDelta; } pDL->pList = pIter; fts3PoslistCopy(0, &pIter); pDL->nList = (pIter - pDL->pList); /* pIter now points just past the 0x00 that terminates the position- ** list for document pDL->iDocid. However, if this position-list was ** edited in place by fts3EvalNearTrim2(), then pIter may not actually ** point to the start of the next docid value. The following line deals ** with this case by advancing pIter past the zero-padding added by ** fts3EvalNearTrim2(). */ while( pIter<pEnd && *pIter==0 ) pIter++; pDL->pNextDocid = pIter; assert( pIter>=&pDL->aAll[pDL->nAll] || *pIter ); *pbEof = 0; } } return rc; } static void fts3EvalStartReaders( Fts3Cursor *pCsr, Fts3Expr *pExpr, int bOptOk, int *pRc ){ if( pExpr && SQLITE_OK==*pRc ){ if( pExpr->eType==FTSQUERY_PHRASE ){ int i; int nToken = pExpr->pPhrase->nToken; for(i=0; i<nToken; i++){ if( pExpr->pPhrase->aToken[i].pDeferred==0 ) break; } pExpr->bDeferred = (i==nToken); *pRc = fts3EvalPhraseStart(pCsr, bOptOk, pExpr->pPhrase); }else{ fts3EvalStartReaders(pCsr, pExpr->pLeft, bOptOk, pRc); fts3EvalStartReaders(pCsr, pExpr->pRight, bOptOk, pRc); pExpr->bDeferred = (pExpr->pLeft->bDeferred && pExpr->pRight->bDeferred); } } } typedef struct Fts3TokenAndCost Fts3TokenAndCost; struct Fts3TokenAndCost { Fts3Phrase *pPhrase; /* The phrase the token belongs to */ int iToken; /* Position of token in phrase */ Fts3PhraseToken *pToken; /* The token itself */ Fts3Expr *pRoot; int nOvfl; int iCol; /* The column the token must match */ }; static void fts3EvalTokenCosts( Fts3Cursor *pCsr, Fts3Expr *pRoot, Fts3Expr *pExpr, Fts3TokenAndCost **ppTC, Fts3Expr ***ppOr, int *pRc ){ if( *pRc==SQLITE_OK && pExpr ){ if( pExpr->eType==FTSQUERY_PHRASE ){ Fts3Phrase *pPhrase = pExpr->pPhrase; int i; for(i=0; *pRc==SQLITE_OK && i<pPhrase->nToken; i++){ Fts3TokenAndCost *pTC = (*ppTC)++; pTC->pPhrase = pPhrase; pTC->iToken = i; pTC->pRoot = pRoot; pTC->pToken = &pPhrase->aToken[i]; pTC->iCol = pPhrase->iColumn; *pRc = sqlite3Fts3MsrOvfl(pCsr, pTC->pToken->pSegcsr, &pTC->nOvfl); } }else if( pExpr->eType!=FTSQUERY_NOT ){ if( pExpr->eType==FTSQUERY_OR ){ pRoot = pExpr->pLeft; **ppOr = pRoot; (*ppOr)++; } fts3EvalTokenCosts(pCsr, pRoot, pExpr->pLeft, ppTC, ppOr, pRc); if( pExpr->eType==FTSQUERY_OR ){ pRoot = pExpr->pRight; **ppOr = pRoot; (*ppOr)++; } fts3EvalTokenCosts(pCsr, pRoot, pExpr->pRight, ppTC, ppOr, pRc); } } } static int fts3EvalAverageDocsize(Fts3Cursor *pCsr, int *pnPage){ if( pCsr->nRowAvg==0 ){ /* The average document size, which is required to calculate the cost ** of each doclist, has not yet been determined. Read the required ** data from the %_stat table to calculate it. ** ** Entry 0 of the %_stat table is a blob containing (nCol+1) FTS3 ** varints, where nCol is the number of columns in the FTS3 table. ** The first varint is the number of documents currently stored in ** the table. The following nCol varints contain the total amount of ** data stored in all rows of each column of the table, from left ** to right. */ int rc; Fts3Table *p = (Fts3Table*)pCsr->base.pVtab; sqlite3_stmt *pStmt; sqlite3_int64 nDoc = 0; sqlite3_int64 nByte = 0; const char *pEnd; const char *a; rc = sqlite3Fts3SelectDoctotal(p, &pStmt); if( rc!=SQLITE_OK ) return rc; a = sqlite3_column_blob(pStmt, 0); assert( a ); pEnd = &a[sqlite3_column_bytes(pStmt, 0)]; a += sqlite3Fts3GetVarint(a, &nDoc); while( a<pEnd ){ a += sqlite3Fts3GetVarint(a, &nByte); } if( nDoc==0 || nByte==0 ){ sqlite3_reset(pStmt); return SQLITE_CORRUPT_VTAB; } pCsr->nDoc = nDoc; pCsr->nRowAvg = (int)(((nByte / nDoc) + p->nPgsz) / p->nPgsz); assert( pCsr->nRowAvg>0 ); rc = sqlite3_reset(pStmt); if( rc!=SQLITE_OK ) return rc; } *pnPage = pCsr->nRowAvg; return SQLITE_OK; } static int fts3EvalSelectDeferred( Fts3Cursor *pCsr, Fts3Expr *pRoot, Fts3TokenAndCost *aTC, int nTC ){ int nDocSize = 0; int nDocEst = 0; int rc = SQLITE_OK; Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; int ii; int nOvfl = 0; int nTerm = 0; for(ii=0; ii<nTC; ii++){ if( aTC[ii].pRoot==pRoot ){ nOvfl += aTC[ii].nOvfl; nTerm++; } } if( nOvfl==0 || nTerm<2 ) return SQLITE_OK; rc = fts3EvalAverageDocsize(pCsr, &nDocSize); for(ii=0; ii<nTerm && rc==SQLITE_OK; ii++){ int jj; Fts3TokenAndCost *pTC = 0; for(jj=0; jj<nTC; jj++){ if( aTC[jj].pToken && aTC[jj].pRoot==pRoot && (!pTC || aTC[jj].nOvfl<pTC->nOvfl) ){ pTC = &aTC[jj]; } } assert( pTC ); /* At this point pTC points to the cheapest remaining token. */ if( ii==0 ){ if( pTC->nOvfl ){ nDocEst = (pTC->nOvfl * pTab->nPgsz + pTab->nPgsz) / 10; }else{ Fts3PhraseToken *pToken = pTC->pToken; int nList = 0; char *pList = 0; rc = fts3TermSelect(pTab, pToken, pTC->iCol, 1, &nList, &pList); assert( rc==SQLITE_OK || pList==0 ); if( rc==SQLITE_OK ){ nDocEst = fts3DoclistCountDocids(1, pList, nList); fts3EvalPhraseMergeToken(pTab, pTC->pPhrase, pTC->iToken,pList,nList); } } }else{ if( pTC->nOvfl>=(nDocEst*nDocSize) ){ Fts3PhraseToken *pToken = pTC->pToken; rc = sqlite3Fts3DeferToken(pCsr, pToken, pTC->iCol); fts3SegReaderCursorFree(pToken->pSegcsr); pToken->pSegcsr = 0; } nDocEst = 1 + (nDocEst/4); } pTC->pToken = 0; } return rc; } SQLITE_PRIVATE int sqlite3Fts3EvalStart(Fts3Cursor *pCsr, Fts3Expr *pExpr, int bOptOk){ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; int rc = SQLITE_OK; int nToken = 0; int nOr = 0; /* Allocate a MultiSegReader for each token in the expression. */ fts3EvalAllocateReaders(pCsr, pExpr, &nToken, &nOr, &rc); /* Call fts3EvalPhraseStart() on all phrases in the expression. TODO: ** This call will eventually also be responsible for determining which ** tokens are 'deferred' until the document text is loaded into memory. ** ** Each token in each phrase is dealt with using one of the following ** three strategies: ** ** 1. Entire doclist loaded into memory as part of the ** fts3EvalStartReaders() call. ** ** 2. Doclist loaded into memory incrementally, as part of each ** sqlite3Fts3EvalNext() call. ** ** 3. Token doclist is never loaded. Instead, documents are loaded into ** memory and scanned for the token as part of the sqlite3Fts3EvalNext() ** call. This is known as a "deferred" token. */ /* If bOptOk is true, check if there are any tokens that should be deferred. */ if( rc==SQLITE_OK && bOptOk && nToken>1 && pTab->bHasStat ){ Fts3TokenAndCost *aTC; Fts3Expr **apOr; aTC = (Fts3TokenAndCost *)sqlite3_malloc( sizeof(Fts3TokenAndCost) * nToken + sizeof(Fts3Expr *) * nOr * 2 ); apOr = (Fts3Expr **)&aTC[nToken]; if( !aTC ){ rc = SQLITE_NOMEM; }else{ int ii; Fts3TokenAndCost *pTC = aTC; Fts3Expr **ppOr = apOr; fts3EvalTokenCosts(pCsr, 0, pExpr, &pTC, &ppOr, &rc); nToken = pTC-aTC; nOr = ppOr-apOr; if( rc==SQLITE_OK ){ rc = fts3EvalSelectDeferred(pCsr, 0, aTC, nToken); for(ii=0; rc==SQLITE_OK && ii<nOr; ii++){ rc = fts3EvalSelectDeferred(pCsr, apOr[ii], aTC, nToken); } } sqlite3_free(aTC); } } fts3EvalStartReaders(pCsr, pExpr, bOptOk, &rc); return rc; } static void fts3EvalZeroPoslist(Fts3Phrase *pPhrase){ if( pPhrase->doclist.bFreeList ){ sqlite3_free(pPhrase->doclist.pList); } pPhrase->doclist.pList = 0; pPhrase->doclist.nList = 0; pPhrase->doclist.bFreeList = 0; } static int fts3EvalNearTrim2( int nNear, char *aTmp, /* Temporary space to use */ char **paPoslist, /* IN/OUT: Position list */ int *pnToken, /* IN/OUT: Tokens in phrase of *paPoslist */ Fts3Phrase *pPhrase /* The phrase object to trim the doclist of */ ){ int nParam1 = nNear + pPhrase->nToken; int nParam2 = nNear + *pnToken; int nNew; char *p2; char *pOut; int res; assert( pPhrase->doclist.pList ); p2 = pOut = pPhrase->doclist.pList; res = fts3PoslistNearMerge( &pOut, aTmp, nParam1, nParam2, paPoslist, &p2 ); if( res ){ nNew = (pOut - pPhrase->doclist.pList) - 1; assert( pPhrase->doclist.pList[nNew]=='\0' ); assert( nNew<=pPhrase->doclist.nList && nNew>0 ); memset(&pPhrase->doclist.pList[nNew], 0, pPhrase->doclist.nList - nNew); pPhrase->doclist.nList = nNew; *paPoslist = pPhrase->doclist.pList; *pnToken = pPhrase->nToken; } return res; } static int fts3EvalNearTest(Fts3Expr *pExpr, int *pRc){ int res = 1; /* The following block runs if pExpr is the root of a NEAR query. ** For example, the query: ** ** "w" NEAR "x" NEAR "y" NEAR "z" ** ** which is represented in tree form as: ** ** | ** +--NEAR--+ <-- root of NEAR query ** | | ** +--NEAR--+ "z" ** | | ** +--NEAR--+ "y" ** | | ** "w" "x" ** ** The right-hand child of a NEAR node is always a phrase. The ** left-hand child may be either a phrase or a NEAR node. There are ** no exceptions to this. */ if( *pRc==SQLITE_OK && pExpr->eType==FTSQUERY_NEAR && pExpr->bEof==0 && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR) ){ Fts3Expr *p; int nTmp = 0; /* Bytes of temp space */ char *aTmp; /* Temp space for PoslistNearMerge() */ /* Allocate temporary working space. */ for(p=pExpr; p->pLeft; p=p->pLeft){ nTmp += p->pRight->pPhrase->doclist.nList; } nTmp += p->pPhrase->doclist.nList; aTmp = sqlite3_malloc(nTmp*2); if( !aTmp ){ *pRc = SQLITE_NOMEM; res = 0; }else{ char *aPoslist = p->pPhrase->doclist.pList; int nToken = p->pPhrase->nToken; for(p=p->pParent;res && p && p->eType==FTSQUERY_NEAR; p=p->pParent){ Fts3Phrase *pPhrase = p->pRight->pPhrase; int nNear = p->nNear; res = fts3EvalNearTrim2(nNear, aTmp, &aPoslist, &nToken, pPhrase); } aPoslist = pExpr->pRight->pPhrase->doclist.pList; nToken = pExpr->pRight->pPhrase->nToken; for(p=pExpr->pLeft; p && res; p=p->pLeft){ int nNear = p->pParent->nNear; Fts3Phrase *pPhrase = ( p->eType==FTSQUERY_NEAR ? p->pRight->pPhrase : p->pPhrase ); res = fts3EvalNearTrim2(nNear, aTmp, &aPoslist, &nToken, pPhrase); } } sqlite3_free(aTmp); } return res; } /* ** This macro is used by the fts3EvalNext() function. The two arguments are ** 64-bit docid values. If the current query is "ORDER BY docid ASC", then ** the macro returns (i1 - i2). Or if it is "ORDER BY docid DESC", then ** it returns (i2 - i1). This allows the same code to be used for merging ** doclists in ascending or descending order. */ #define DOCID_CMP(i1, i2) ((pCsr->bDesc?-1:1) * (i1-i2)) static void fts3EvalNext( Fts3Cursor *pCsr, Fts3Expr *pExpr, int *pRc ){ if( *pRc==SQLITE_OK ){ assert( pExpr->bEof==0 ); pExpr->bStart = 1; switch( pExpr->eType ){ case FTSQUERY_NEAR: case FTSQUERY_AND: { Fts3Expr *pLeft = pExpr->pLeft; Fts3Expr *pRight = pExpr->pRight; assert( !pLeft->bDeferred || !pRight->bDeferred ); if( pLeft->bDeferred ){ fts3EvalNext(pCsr, pRight, pRc); pExpr->iDocid = pRight->iDocid; pExpr->bEof = pRight->bEof; }else if( pRight->bDeferred ){ fts3EvalNext(pCsr, pLeft, pRc); pExpr->iDocid = pLeft->iDocid; pExpr->bEof = pLeft->bEof; }else{ fts3EvalNext(pCsr, pLeft, pRc); fts3EvalNext(pCsr, pRight, pRc); while( !pLeft->bEof && !pRight->bEof && *pRc==SQLITE_OK ){ sqlite3_int64 iDiff = DOCID_CMP(pLeft->iDocid, pRight->iDocid); if( iDiff==0 ) break; if( iDiff<0 ){ fts3EvalNext(pCsr, pLeft, pRc); }else{ fts3EvalNext(pCsr, pRight, pRc); } } pExpr->iDocid = pLeft->iDocid; pExpr->bEof = (pLeft->bEof || pRight->bEof); } break; } case FTSQUERY_OR: { Fts3Expr *pLeft = pExpr->pLeft; Fts3Expr *pRight = pExpr->pRight; sqlite3_int64 iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid); assert( pLeft->bStart || pLeft->iDocid==pRight->iDocid ); assert( pRight->bStart || pLeft->iDocid==pRight->iDocid ); if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){ fts3EvalNext(pCsr, pLeft, pRc); }else if( pLeft->bEof || (pRight->bEof==0 && iCmp>0) ){ fts3EvalNext(pCsr, pRight, pRc); }else{ fts3EvalNext(pCsr, pLeft, pRc); fts3EvalNext(pCsr, pRight, pRc); } pExpr->bEof = (pLeft->bEof && pRight->bEof); iCmp = DOCID_CMP(pLeft->iDocid, pRight->iDocid); if( pRight->bEof || (pLeft->bEof==0 && iCmp<0) ){ pExpr->iDocid = pLeft->iDocid; }else{ pExpr->iDocid = pRight->iDocid; } break; } case FTSQUERY_NOT: { Fts3Expr *pLeft = pExpr->pLeft; Fts3Expr *pRight = pExpr->pRight; if( pRight->bStart==0 ){ fts3EvalNext(pCsr, pRight, pRc); assert( *pRc!=SQLITE_OK || pRight->bStart ); } fts3EvalNext(pCsr, pLeft, pRc); if( pLeft->bEof==0 ){ while( !*pRc && !pRight->bEof && DOCID_CMP(pLeft->iDocid, pRight->iDocid)>0 ){ fts3EvalNext(pCsr, pRight, pRc); } } pExpr->iDocid = pLeft->iDocid; pExpr->bEof = pLeft->bEof; break; } default: { Fts3Phrase *pPhrase = pExpr->pPhrase; fts3EvalZeroPoslist(pPhrase); *pRc = fts3EvalPhraseNext(pCsr, pPhrase, &pExpr->bEof); pExpr->iDocid = pPhrase->doclist.iDocid; break; } } } } static int fts3EvalDeferredTest(Fts3Cursor *pCsr, Fts3Expr *pExpr, int *pRc){ int bHit = 1; if( *pRc==SQLITE_OK ){ switch( pExpr->eType ){ case FTSQUERY_NEAR: case FTSQUERY_AND: bHit = ( fts3EvalDeferredTest(pCsr, pExpr->pLeft, pRc) && fts3EvalDeferredTest(pCsr, pExpr->pRight, pRc) && fts3EvalNearTest(pExpr, pRc) ); /* If the NEAR expression does not match any rows, zero the doclist for ** all phrases involved in the NEAR. This is because the snippet(), ** offsets() and matchinfo() functions are not supposed to recognize ** any instances of phrases that are part of unmatched NEAR queries. ** For example if this expression: ** ** ... MATCH 'a OR (b NEAR c)' ** ** is matched against a row containing: ** ** 'a b d e' ** ** then any snippet() should ony highlight the "a" term, not the "b" ** (as "b" is part of a non-matching NEAR clause). */ if( bHit==0 && pExpr->eType==FTSQUERY_NEAR && (pExpr->pParent==0 || pExpr->pParent->eType!=FTSQUERY_NEAR) ){ Fts3Expr *p; for(p=pExpr; p->pPhrase==0; p=p->pLeft){ if( p->pRight->iDocid==pCsr->iPrevId ){ fts3EvalZeroPoslist(p->pRight->pPhrase); } } if( p->iDocid==pCsr->iPrevId ){ fts3EvalZeroPoslist(p->pPhrase); } } break; case FTSQUERY_OR: { int bHit1 = fts3EvalDeferredTest(pCsr, pExpr->pLeft, pRc); int bHit2 = fts3EvalDeferredTest(pCsr, pExpr->pRight, pRc); bHit = bHit1 || bHit2; break; } case FTSQUERY_NOT: bHit = ( fts3EvalDeferredTest(pCsr, pExpr->pLeft, pRc) && !fts3EvalDeferredTest(pCsr, pExpr->pRight, pRc) ); break; default: { if( pCsr->pDeferred && (pExpr->iDocid==pCsr->iPrevId || pExpr->bDeferred) ){ Fts3Phrase *pPhrase = pExpr->pPhrase; assert( pExpr->bDeferred || pPhrase->doclist.bFreeList==0 ); if( pExpr->bDeferred ){ fts3EvalZeroPoslist(pPhrase); } *pRc = fts3EvalDeferredPhrase(pCsr, pPhrase); bHit = (pPhrase->doclist.pList!=0); pExpr->iDocid = pCsr->iPrevId; }else{ bHit = (pExpr->bEof==0 && pExpr->iDocid==pCsr->iPrevId); } break; } } } return bHit; } /* ** Return 1 if both of the following are true: ** ** 1. *pRc is SQLITE_OK when this function returns, and ** ** 2. After scanning the current FTS table row for the deferred tokens, ** it is determined that the row does not match the query. ** ** Or, if no error occurs and it seems the current row does match the FTS ** query, return 0. */ static int fts3EvalLoadDeferred(Fts3Cursor *pCsr, int *pRc){ int rc = *pRc; int bMiss = 0; if( rc==SQLITE_OK ){ if( pCsr->pDeferred ){ rc = fts3CursorSeek(0, pCsr); if( rc==SQLITE_OK ){ rc = sqlite3Fts3CacheDeferredDoclists(pCsr); } } bMiss = (0==fts3EvalDeferredTest(pCsr, pCsr->pExpr, &rc)); sqlite3Fts3FreeDeferredDoclists(pCsr); *pRc = rc; } return (rc==SQLITE_OK && bMiss); } /* ** Advance to the next document that matches the FTS expression in ** Fts3Cursor.pExpr. */ SQLITE_PRIVATE int sqlite3Fts3EvalNext(Fts3Cursor *pCsr){ int rc = SQLITE_OK; /* Return Code */ Fts3Expr *pExpr = pCsr->pExpr; assert( pCsr->isEof==0 ); if( pExpr==0 ){ pCsr->isEof = 1; }else{ do { if( pCsr->isRequireSeek==0 ){ sqlite3_reset(pCsr->pStmt); } assert( sqlite3_data_count(pCsr->pStmt)==0 ); fts3EvalNext(pCsr, pExpr, &rc); pCsr->isEof = pExpr->bEof; pCsr->isRequireSeek = 1; pCsr->isMatchinfoNeeded = 1; pCsr->iPrevId = pExpr->iDocid; }while( pCsr->isEof==0 && fts3EvalLoadDeferred(pCsr, &rc) ); } return rc; } /* ** Restart interation for expression pExpr so that the next call to ** sqlite3Fts3EvalNext() visits the first row. Do not allow incremental ** loading or merging of phrase doclists for this iteration. ** ** If *pRc is other than SQLITE_OK when this function is called, it is ** a no-op. If an error occurs within this function, *pRc is set to an ** SQLite error code before returning. */ static void fts3EvalRestart( Fts3Cursor *pCsr, Fts3Expr *pExpr, int *pRc ){ if( pExpr && *pRc==SQLITE_OK ){ Fts3Phrase *pPhrase = pExpr->pPhrase; if( pPhrase ){ fts3EvalZeroPoslist(pPhrase); if( pPhrase->bIncr ){ assert( pPhrase->nToken==1 ); assert( pPhrase->aToken[0].pSegcsr ); sqlite3Fts3MsrIncrRestart(pPhrase->aToken[0].pSegcsr); *pRc = fts3EvalPhraseStart(pCsr, 0, pPhrase); } pPhrase->doclist.pNextDocid = 0; pPhrase->doclist.iDocid = 0; } pExpr->iDocid = 0; pExpr->bEof = 0; pExpr->bStart = 0; fts3EvalRestart(pCsr, pExpr->pLeft, pRc); fts3EvalRestart(pCsr, pExpr->pRight, pRc); } } /* ** After allocating the Fts3Expr.aMI[] array for each phrase in the ** expression rooted at pExpr, the cursor iterates through all rows matched ** by pExpr, calling this function for each row. This function increments ** the values in Fts3Expr.aMI[] according to the position-list currently ** found in Fts3Expr.pPhrase->doclist.pList for each of the phrase ** expression nodes. */ static void fts3EvalUpdateCounts(Fts3Expr *pExpr){ if( pExpr ){ Fts3Phrase *pPhrase = pExpr->pPhrase; if( pPhrase && pPhrase->doclist.pList ){ int iCol = 0; char *p = pPhrase->doclist.pList; assert( *p ); while( 1 ){ u8 c = 0; int iCnt = 0; while( 0xFE & (*p | c) ){ if( (c&0x80)==0 ) iCnt++; c = *p++ & 0x80; } /* aMI[iCol*3 + 1] = Number of occurrences ** aMI[iCol*3 + 2] = Number of rows containing at least one instance */ pExpr->aMI[iCol*3 + 1] += iCnt; pExpr->aMI[iCol*3 + 2] += (iCnt>0); if( *p==0x00 ) break; p++; p += sqlite3Fts3GetVarint32(p, &iCol); } } fts3EvalUpdateCounts(pExpr->pLeft); fts3EvalUpdateCounts(pExpr->pRight); } } /* ** Expression pExpr must be of type FTSQUERY_PHRASE. ** ** If it is not already allocated and populated, this function allocates and ** populates the Fts3Expr.aMI[] array for expression pExpr. If pExpr is part ** of a NEAR expression, then it also allocates and populates the same array ** for all other phrases that are part of the NEAR expression. ** ** SQLITE_OK is returned if the aMI[] array is successfully allocated and ** populated. Otherwise, if an error occurs, an SQLite error code is returned. */ static int fts3EvalGatherStats( Fts3Cursor *pCsr, /* Cursor object */ Fts3Expr *pExpr /* FTSQUERY_PHRASE expression */ ){ int rc = SQLITE_OK; /* Return code */ assert( pExpr->eType==FTSQUERY_PHRASE ); if( pExpr->aMI==0 ){ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; Fts3Expr *pRoot; /* Root of NEAR expression */ Fts3Expr *p; /* Iterator used for several purposes */ sqlite3_int64 iPrevId = pCsr->iPrevId; sqlite3_int64 iDocid; u8 bEof; /* Find the root of the NEAR expression */ pRoot = pExpr; while( pRoot->pParent && pRoot->pParent->eType==FTSQUERY_NEAR ){ pRoot = pRoot->pParent; } iDocid = pRoot->iDocid; bEof = pRoot->bEof; assert( pRoot->bStart ); /* Allocate space for the aMSI[] array of each FTSQUERY_PHRASE node */ for(p=pRoot; p; p=p->pLeft){ Fts3Expr *pE = (p->eType==FTSQUERY_PHRASE?p:p->pRight); assert( pE->aMI==0 ); pE->aMI = (u32 *)sqlite3_malloc(pTab->nColumn * 3 * sizeof(u32)); if( !pE->aMI ) return SQLITE_NOMEM; memset(pE->aMI, 0, pTab->nColumn * 3 * sizeof(u32)); } fts3EvalRestart(pCsr, pRoot, &rc); while( pCsr->isEof==0 && rc==SQLITE_OK ){ do { /* Ensure the %_content statement is reset. */ if( pCsr->isRequireSeek==0 ) sqlite3_reset(pCsr->pStmt); assert( sqlite3_data_count(pCsr->pStmt)==0 ); /* Advance to the next document */ fts3EvalNext(pCsr, pRoot, &rc); pCsr->isEof = pRoot->bEof; pCsr->isRequireSeek = 1; pCsr->isMatchinfoNeeded = 1; pCsr->iPrevId = pRoot->iDocid; }while( pCsr->isEof==0 && pRoot->eType==FTSQUERY_NEAR && fts3EvalLoadDeferred(pCsr, &rc) ); if( rc==SQLITE_OK && pCsr->isEof==0 ){ fts3EvalUpdateCounts(pRoot); } } pCsr->isEof = 0; pCsr->iPrevId = iPrevId; if( bEof ){ pRoot->bEof = bEof; }else{ /* Caution: pRoot may iterate through docids in ascending or descending ** order. For this reason, even though it seems more defensive, the ** do loop can not be written: ** ** do {...} while( pRoot->iDocid<iDocid && rc==SQLITE_OK ); */ fts3EvalRestart(pCsr, pRoot, &rc); do { fts3EvalNext(pCsr, pRoot, &rc); assert( pRoot->bEof==0 ); }while( pRoot->iDocid!=iDocid && rc==SQLITE_OK ); fts3EvalLoadDeferred(pCsr, &rc); } } return rc; } /* ** This function is used by the matchinfo() module to query a phrase ** expression node for the following information: ** ** 1. The total number of occurrences of the phrase in each column of ** the FTS table (considering all rows), and ** ** 2. For each column, the number of rows in the table for which the ** column contains at least one instance of the phrase. ** ** If no error occurs, SQLITE_OK is returned and the values for each column ** written into the array aiOut as follows: ** ** aiOut[iCol*3 + 1] = Number of occurrences ** aiOut[iCol*3 + 2] = Number of rows containing at least one instance ** ** Caveats: ** ** * If a phrase consists entirely of deferred tokens, then all output ** values are set to the number of documents in the table. In other ** words we assume that very common tokens occur exactly once in each ** column of each row of the table. ** ** * If a phrase contains some deferred tokens (and some non-deferred ** tokens), count the potential occurrence identified by considering ** the non-deferred tokens instead of actual phrase occurrences. ** ** * If the phrase is part of a NEAR expression, then only phrase instances ** that meet the NEAR constraint are included in the counts. */ SQLITE_PRIVATE int sqlite3Fts3EvalPhraseStats( Fts3Cursor *pCsr, /* FTS cursor handle */ Fts3Expr *pExpr, /* Phrase expression */ u32 *aiOut /* Array to write results into (see above) */ ){ Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; int rc = SQLITE_OK; int iCol; if( pExpr->bDeferred && pExpr->pParent->eType!=FTSQUERY_NEAR ){ assert( pCsr->nDoc>0 ); for(iCol=0; iCol<pTab->nColumn; iCol++){ aiOut[iCol*3 + 1] = (u32)pCsr->nDoc; aiOut[iCol*3 + 2] = (u32)pCsr->nDoc; } }else{ rc = fts3EvalGatherStats(pCsr, pExpr); if( rc==SQLITE_OK ){ assert( pExpr->aMI ); for(iCol=0; iCol<pTab->nColumn; iCol++){ aiOut[iCol*3 + 1] = pExpr->aMI[iCol*3 + 1]; aiOut[iCol*3 + 2] = pExpr->aMI[iCol*3 + 2]; } } } return rc; } /* ** The expression pExpr passed as the second argument to this function ** must be of type FTSQUERY_PHRASE. ** ** The returned value is either NULL or a pointer to a buffer containing ** a position-list indicating the occurrences of the phrase in column iCol ** of the current row. ** ** More specifically, the returned buffer contains 1 varint for each ** occurence of the phrase in the column, stored using the normal (delta+2) ** compression and is terminated by either an 0x01 or 0x00 byte. For example, ** if the requested column contains "a b X c d X X" and the position-list ** for 'X' is requested, the buffer returned may contain: ** ** 0x04 0x05 0x03 0x01 or 0x04 0x05 0x03 0x00 ** ** This function works regardless of whether or not the phrase is deferred, ** incremental, or neither. */ SQLITE_PRIVATE char *sqlite3Fts3EvalPhrasePoslist( Fts3Cursor *pCsr, /* FTS3 cursor object */ Fts3Expr *pExpr, /* Phrase to return doclist for */ int iCol /* Column to return position list for */ ){ Fts3Phrase *pPhrase = pExpr->pPhrase; Fts3Table *pTab = (Fts3Table *)pCsr->base.pVtab; char *pIter = pPhrase->doclist.pList; int iThis; assert( iCol>=0 && iCol<pTab->nColumn ); if( !pIter || pExpr->bEof || pExpr->iDocid!=pCsr->iPrevId || (pPhrase->iColumn<pTab->nColumn && pPhrase->iColumn!=iCol) ){ return 0; } assert( pPhrase->doclist.nList>0 ); if( *pIter==0x01 ){ pIter++; pIter += sqlite3Fts3GetVarint32(pIter, &iThis); }else{ iThis = 0; } while( iThis<iCol ){ fts3ColumnlistCopy(0, &pIter); if( *pIter==0x00 ) return 0; pIter++; pIter += sqlite3Fts3GetVarint32(pIter, &iThis); } return ((iCol==iThis)?pIter:0); } /* ** Free all components of the Fts3Phrase structure that were allocated by ** the eval module. Specifically, this means to free: ** ** * the contents of pPhrase->doclist, and ** * any Fts3MultiSegReader objects held by phrase tokens. */ SQLITE_PRIVATE void sqlite3Fts3EvalPhraseCleanup(Fts3Phrase *pPhrase){ if( pPhrase ){ int i; sqlite3_free(pPhrase->doclist.aAll); fts3EvalZeroPoslist(pPhrase); memset(&pPhrase->doclist, 0, sizeof(Fts3Doclist)); for(i=0; i<pPhrase->nToken; i++){ fts3SegReaderCursorFree(pPhrase->aToken[i].pSegcsr); pPhrase->aToken[i].pSegcsr = 0; } } } #endif /************** End of fts3.c ************************************************/ /************** Begin file fts3_aux.c ****************************************/ /* ** 2011 Jan 27 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ****************************************************************************** ** */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) typedef struct Fts3auxTable Fts3auxTable; typedef struct Fts3auxCursor Fts3auxCursor; struct Fts3auxTable { sqlite3_vtab base; /* Base class used by SQLite core */ Fts3Table *pFts3Tab; }; struct Fts3auxCursor { sqlite3_vtab_cursor base; /* Base class used by SQLite core */ Fts3MultiSegReader csr; /* Must be right after "base" */ Fts3SegFilter filter; char *zStop; int nStop; /* Byte-length of string zStop */ int isEof; /* True if cursor is at EOF */ sqlite3_int64 iRowid; /* Current rowid */ int iCol; /* Current value of 'col' column */ |
︙ | ︙ | |||
115663 115664 115665 115666 115667 115668 115669 115670 115671 115672 115673 115674 115675 115676 | if( !p ) return SQLITE_NOMEM; memset(p, 0, nByte); p->pFts3Tab = (Fts3Table *)&p[1]; p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1]; p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1]; p->pFts3Tab->db = db; memcpy((char *)p->pFts3Tab->zDb, zDb, nDb); memcpy((char *)p->pFts3Tab->zName, zFts3, nFts3); sqlite3Fts3Dequote((char *)p->pFts3Tab->zName); *ppVtab = (sqlite3_vtab *)p; return SQLITE_OK; | > | 116522 116523 116524 116525 116526 116527 116528 116529 116530 116531 116532 116533 116534 116535 116536 | if( !p ) return SQLITE_NOMEM; memset(p, 0, nByte); p->pFts3Tab = (Fts3Table *)&p[1]; p->pFts3Tab->zDb = (char *)&p->pFts3Tab[1]; p->pFts3Tab->zName = &p->pFts3Tab->zDb[nDb+1]; p->pFts3Tab->db = db; p->pFts3Tab->nIndex = 1; memcpy((char *)p->pFts3Tab->zDb, zDb, nDb); memcpy((char *)p->pFts3Tab->zName, zFts3, nFts3); sqlite3Fts3Dequote((char *)p->pFts3Tab->zName); *ppVtab = (sqlite3_vtab *)p; return SQLITE_OK; |
︙ | ︙ | |||
115943 115944 115945 115946 115947 115948 115949 | if( idxNum&FTS4AUX_LE_CONSTRAINT ){ int iIdx = (idxNum&FTS4AUX_GE_CONSTRAINT) ? 1 : 0; pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iIdx])); pCsr->nStop = sqlite3_value_bytes(apVal[iIdx]); if( pCsr->zStop==0 ) return SQLITE_NOMEM; } | | | 116803 116804 116805 116806 116807 116808 116809 116810 116811 116812 116813 116814 116815 116816 116817 | if( idxNum&FTS4AUX_LE_CONSTRAINT ){ int iIdx = (idxNum&FTS4AUX_GE_CONSTRAINT) ? 1 : 0; pCsr->zStop = sqlite3_mprintf("%s", sqlite3_value_text(apVal[iIdx])); pCsr->nStop = sqlite3_value_bytes(apVal[iIdx]); if( pCsr->zStop==0 ) return SQLITE_NOMEM; } rc = sqlite3Fts3SegReaderCursor(pFts3, 0, FTS3_SEGCURSOR_ALL, pCsr->filter.zTerm, pCsr->filter.nTerm, 0, isScan, &pCsr->csr ); if( rc==SQLITE_OK ){ rc = sqlite3Fts3SegReaderStart(pFts3, &pCsr->csr, &pCsr->filter); } if( rc==SQLITE_OK ) rc = fts3auxNextMethod(pCursor); |
︙ | ︙ | |||
116122 116123 116124 116125 116126 116127 116128 116129 116130 116131 116132 116133 116134 116135 116136 116137 116138 116139 116140 116141 | /* ** Default span for NEAR operators. */ #define SQLITE_FTS3_DEFAULT_NEAR_PARAM 10 typedef struct ParseContext ParseContext; struct ParseContext { sqlite3_tokenizer *pTokenizer; /* Tokenizer module */ const char **azCol; /* Array of column names for fts3 table */ int nCol; /* Number of entries in azCol[] */ int iDefaultCol; /* Default column to query */ sqlite3_context *pCtx; /* Write error message here */ int nNest; /* Number of nested brackets */ }; /* ** This function is equivalent to the standard isspace() function. ** | > > > > > > > > > | 116982 116983 116984 116985 116986 116987 116988 116989 116990 116991 116992 116993 116994 116995 116996 116997 116998 116999 117000 117001 117002 117003 117004 117005 117006 117007 117008 117009 117010 | /* ** Default span for NEAR operators. */ #define SQLITE_FTS3_DEFAULT_NEAR_PARAM 10 /* ** isNot: ** This variable is used by function getNextNode(). When getNextNode() is ** called, it sets ParseContext.isNot to true if the 'next node' is a ** FTSQUERY_PHRASE with a unary "-" attached to it. i.e. "mysql" in the ** FTS3 query "sqlite -mysql". Otherwise, ParseContext.isNot is set to ** zero. */ typedef struct ParseContext ParseContext; struct ParseContext { sqlite3_tokenizer *pTokenizer; /* Tokenizer module */ const char **azCol; /* Array of column names for fts3 table */ int nCol; /* Number of entries in azCol[] */ int iDefaultCol; /* Default column to query */ int isNot; /* True if getNextNode() sees a unary - */ sqlite3_context *pCtx; /* Write error message here */ int nNest; /* Number of nested brackets */ }; /* ** This function is equivalent to the standard isspace() function. ** |
︙ | ︙ | |||
116213 116214 116215 116216 116217 116218 116219 | memcpy(pRet->pPhrase->aToken[0].z, zToken, nToken); if( iEnd<n && z[iEnd]=='*' ){ pRet->pPhrase->aToken[0].isPrefix = 1; iEnd++; } if( !sqlite3_fts3_enable_parentheses && iStart>0 && z[iStart-1]=='-' ){ | | | 117082 117083 117084 117085 117086 117087 117088 117089 117090 117091 117092 117093 117094 117095 117096 | memcpy(pRet->pPhrase->aToken[0].z, zToken, nToken); if( iEnd<n && z[iEnd]=='*' ){ pRet->pPhrase->aToken[0].isPrefix = 1; iEnd++; } if( !sqlite3_fts3_enable_parentheses && iStart>0 && z[iStart-1]=='-' ){ pParse->isNot = 1; } } nConsumed = iEnd; } pModule->xClose(pCursor); } |
︙ | ︙ | |||
116265 116266 116267 116268 116269 116270 116271 116272 116273 116274 116275 116276 | sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; int rc; Fts3Expr *p = 0; sqlite3_tokenizer_cursor *pCursor = 0; char *zTemp = 0; int nTemp = 0; rc = pModule->xOpen(pTokenizer, zInput, nInput, &pCursor); if( rc==SQLITE_OK ){ int ii; pCursor->pTokenizer = pTokenizer; for(ii=0; rc==SQLITE_OK; ii++){ | > > > > > > > > > > > > > > > > > > > > > > | | | | | | < | | > | < < | > | | | < | | < < < < | > > > | | < < | | > > > > | | < | | < | | < | | < < < | 117134 117135 117136 117137 117138 117139 117140 117141 117142 117143 117144 117145 117146 117147 117148 117149 117150 117151 117152 117153 117154 117155 117156 117157 117158 117159 117160 117161 117162 117163 117164 117165 117166 117167 117168 117169 117170 117171 117172 117173 117174 117175 117176 117177 117178 117179 117180 117181 117182 117183 117184 117185 117186 117187 117188 117189 117190 117191 117192 117193 117194 117195 117196 117197 117198 117199 117200 117201 117202 117203 117204 117205 117206 117207 117208 117209 117210 117211 117212 117213 117214 117215 117216 117217 117218 117219 117220 117221 117222 117223 | sqlite3_tokenizer_module const *pModule = pTokenizer->pModule; int rc; Fts3Expr *p = 0; sqlite3_tokenizer_cursor *pCursor = 0; char *zTemp = 0; int nTemp = 0; const int nSpace = sizeof(Fts3Expr) + sizeof(Fts3Phrase); int nToken = 0; /* The final Fts3Expr data structure, including the Fts3Phrase, ** Fts3PhraseToken structures token buffers are all stored as a single ** allocation so that the expression can be freed with a single call to ** sqlite3_free(). Setting this up requires a two pass approach. ** ** The first pass, in the block below, uses a tokenizer cursor to iterate ** through the tokens in the expression. This pass uses fts3ReallocOrFree() ** to assemble data in two dynamic buffers: ** ** Buffer p: Points to the Fts3Expr structure, followed by the Fts3Phrase ** structure, followed by the array of Fts3PhraseToken ** structures. This pass only populates the Fts3PhraseToken array. ** ** Buffer zTemp: Contains copies of all tokens. ** ** The second pass, in the block that begins "if( rc==SQLITE_DONE )" below, ** appends buffer zTemp to buffer p, and fills in the Fts3Expr and Fts3Phrase ** structures. */ rc = pModule->xOpen(pTokenizer, zInput, nInput, &pCursor); if( rc==SQLITE_OK ){ int ii; pCursor->pTokenizer = pTokenizer; for(ii=0; rc==SQLITE_OK; ii++){ const char *zByte; int nByte, iBegin, iEnd, iPos; rc = pModule->xNext(pCursor, &zByte, &nByte, &iBegin, &iEnd, &iPos); if( rc==SQLITE_OK ){ Fts3PhraseToken *pToken; p = fts3ReallocOrFree(p, nSpace + ii*sizeof(Fts3PhraseToken)); if( !p ) goto no_mem; zTemp = fts3ReallocOrFree(zTemp, nTemp + nByte); if( !zTemp ) goto no_mem; assert( nToken==ii ); pToken = &((Fts3Phrase *)(&p[1]))->aToken[ii]; memset(pToken, 0, sizeof(Fts3PhraseToken)); memcpy(&zTemp[nTemp], zByte, nByte); nTemp += nByte; pToken->n = nByte; pToken->isPrefix = (iEnd<nInput && zInput[iEnd]=='*'); nToken = ii+1; } } pModule->xClose(pCursor); pCursor = 0; } if( rc==SQLITE_DONE ){ int jj; char *zBuf = 0; p = fts3ReallocOrFree(p, nSpace + nToken*sizeof(Fts3PhraseToken) + nTemp); if( !p ) goto no_mem; memset(p, 0, (char *)&(((Fts3Phrase *)&p[1])->aToken[0])-(char *)p); p->eType = FTSQUERY_PHRASE; p->pPhrase = (Fts3Phrase *)&p[1]; p->pPhrase->iColumn = pParse->iDefaultCol; p->pPhrase->nToken = nToken; zBuf = (char *)&p->pPhrase->aToken[nToken]; memcpy(zBuf, zTemp, nTemp); sqlite3_free(zTemp); for(jj=0; jj<p->pPhrase->nToken; jj++){ p->pPhrase->aToken[jj].z = zBuf; zBuf += p->pPhrase->aToken[jj].n; } rc = SQLITE_OK; } *ppExpr = p; return rc; no_mem: |
︙ | ︙ | |||
116381 116382 116383 116384 116385 116386 116387 116388 116389 116390 116391 116392 116393 116394 | int iCol; int iColLen; int rc; Fts3Expr *pRet = 0; const char *zInput = z; int nInput = n; /* Skip over any whitespace before checking for a keyword, an open or ** close bracket, or a quoted string. */ while( nInput>0 && fts3isspace(*zInput) ){ nInput--; zInput++; | > > | 117265 117266 117267 117268 117269 117270 117271 117272 117273 117274 117275 117276 117277 117278 117279 117280 | int iCol; int iColLen; int rc; Fts3Expr *pRet = 0; const char *zInput = z; int nInput = n; pParse->isNot = 0; /* Skip over any whitespace before checking for a keyword, an open or ** close bracket, or a quoted string. */ while( nInput>0 && fts3isspace(*zInput) ){ nInput--; zInput++; |
︙ | ︙ | |||
116600 116601 116602 116603 116604 116605 116606 | Fts3Expr *p = 0; int nByte = 0; rc = getNextNode(pParse, zIn, nIn, &p, &nByte); if( rc==SQLITE_OK ){ int isPhrase; if( !sqlite3_fts3_enable_parentheses | | < | 117486 117487 117488 117489 117490 117491 117492 117493 117494 117495 117496 117497 117498 117499 117500 117501 117502 117503 117504 117505 117506 117507 117508 117509 117510 117511 117512 117513 117514 117515 117516 117517 | Fts3Expr *p = 0; int nByte = 0; rc = getNextNode(pParse, zIn, nIn, &p, &nByte); if( rc==SQLITE_OK ){ int isPhrase; if( !sqlite3_fts3_enable_parentheses && p->eType==FTSQUERY_PHRASE && pParse->isNot ){ /* Create an implicit NOT operator. */ Fts3Expr *pNot = fts3MallocZero(sizeof(Fts3Expr)); if( !pNot ){ sqlite3Fts3ExprFree(p); rc = SQLITE_NOMEM; goto exprparse_out; } pNot->eType = FTSQUERY_NOT; pNot->pRight = p; if( pNotBranch ){ pNot->pLeft = pNotBranch; } pNotBranch = pNot; p = pPrev; }else{ int eType = p->eType; isPhrase = (eType==FTSQUERY_PHRASE || p->pLeft); /* The isRequirePhrase variable is set to true if a phrase or ** an expression contained in parenthesis is required. If a ** binary operator (AND, OR, NOT or NEAR) is encounted when ** isRequirePhrase is set, this is a syntax error. */ |
︙ | ︙ | |||
116781 116782 116783 116784 116785 116786 116787 116788 116789 | } /* ** Free a parsed fts3 query expression allocated by sqlite3Fts3ExprParse(). */ SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *p){ if( p ){ sqlite3Fts3ExprFree(p->pLeft); sqlite3Fts3ExprFree(p->pRight); | > > | | 117666 117667 117668 117669 117670 117671 117672 117673 117674 117675 117676 117677 117678 117679 117680 117681 117682 117683 117684 | } /* ** Free a parsed fts3 query expression allocated by sqlite3Fts3ExprParse(). */ SQLITE_PRIVATE void sqlite3Fts3ExprFree(Fts3Expr *p){ if( p ){ assert( p->eType==FTSQUERY_PHRASE || p->pPhrase==0 ); sqlite3Fts3ExprFree(p->pLeft); sqlite3Fts3ExprFree(p->pRight); sqlite3Fts3EvalPhraseCleanup(p->pPhrase); sqlite3_free(p->aMI); sqlite3_free(p); } } /**************************************************************************** ***************************************************************************** ** Everything after this point is just test code. |
︙ | ︙ | |||
116840 116841 116842 116843 116844 116845 116846 | */ static char *exprToString(Fts3Expr *pExpr, char *zBuf){ switch( pExpr->eType ){ case FTSQUERY_PHRASE: { Fts3Phrase *pPhrase = pExpr->pPhrase; int i; zBuf = sqlite3_mprintf( | | | 117727 117728 117729 117730 117731 117732 117733 117734 117735 117736 117737 117738 117739 117740 117741 | */ static char *exprToString(Fts3Expr *pExpr, char *zBuf){ switch( pExpr->eType ){ case FTSQUERY_PHRASE: { Fts3Phrase *pPhrase = pExpr->pPhrase; int i; zBuf = sqlite3_mprintf( "%zPHRASE %d 0", zBuf, pPhrase->iColumn); for(i=0; zBuf && i<pPhrase->nToken; i++){ zBuf = sqlite3_mprintf("%z %.*s%s", zBuf, pPhrase->aToken[i].n, pPhrase->aToken[i].z, (pPhrase->aToken[i].isPrefix?"+":"") ); } return zBuf; |
︙ | ︙ | |||
117384 117385 117386 117387 117388 117389 117390 | ** * The FTS3 module is being built as an extension ** (in which case SQLITE_CORE is not defined), or ** ** * The FTS3 module is being built into the core of ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined). */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) | < | 118271 118272 118273 118274 118275 118276 118277 118278 118279 118280 118281 118282 118283 118284 | ** * The FTS3 module is being built as an extension ** (in which case SQLITE_CORE is not defined), or ** ** * The FTS3 module is being built into the core of ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined). */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) /* ** Class derived from sqlite3_tokenizer */ typedef struct porter_tokenizer { |
︙ | ︙ | |||
118027 118028 118029 118030 118031 118032 118033 | ** ** * The FTS3 module is being built as an extension ** (in which case SQLITE_CORE is not defined), or ** ** * The FTS3 module is being built into the core of ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined). */ | < < > > | 118913 118914 118915 118916 118917 118918 118919 118920 118921 118922 118923 118924 118925 118926 118927 118928 118929 118930 118931 | ** ** * The FTS3 module is being built as an extension ** (in which case SQLITE_CORE is not defined), or ** ** * The FTS3 module is being built into the core of ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined). */ #ifndef SQLITE_CORE SQLITE_EXTENSION_INIT1 #endif #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) /* ** Implementation of the SQL scalar function for accessing the underlying ** hash table. This function may be called as follows: ** ** SELECT <function-name>(<key-name>); |
︙ | ︙ | |||
118156 118157 118158 118159 118160 118161 118162 | Fts3Hash *pHash, /* Tokenizer hash table */ const char *zArg, /* Tokenizer name */ sqlite3_tokenizer **ppTok, /* OUT: Tokenizer (if applicable) */ char **pzErr /* OUT: Set to malloced error message */ ){ int rc; char *z = (char *)zArg; | | | 119042 119043 119044 119045 119046 119047 119048 119049 119050 119051 119052 119053 119054 119055 119056 | Fts3Hash *pHash, /* Tokenizer hash table */ const char *zArg, /* Tokenizer name */ sqlite3_tokenizer **ppTok, /* OUT: Tokenizer (if applicable) */ char **pzErr /* OUT: Set to malloced error message */ ){ int rc; char *z = (char *)zArg; int n = 0; char *zCopy; char *zEnd; /* Pointer to nul-term of zCopy */ sqlite3_tokenizer_module *m; zCopy = sqlite3_mprintf("%s", zArg); if( !zCopy ) return SQLITE_NOMEM; zEnd = &zCopy[strlen(zCopy)]; |
︙ | ︙ | |||
118521 118522 118523 118524 118525 118526 118527 | ** * The FTS3 module is being built into the core of ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined). */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) | < | 119407 119408 119409 119410 119411 119412 119413 119414 119415 119416 119417 119418 119419 119420 | ** * The FTS3 module is being built into the core of ** SQLite (in which case SQLITE_ENABLE_FTS3 is defined). */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) typedef struct simple_tokenizer { sqlite3_tokenizer base; char delim[128]; /* flag ASCII delimiters */ } simple_tokenizer; typedef struct simple_tokenizer_cursor { sqlite3_tokenizer_cursor base; |
︙ | ︙ | |||
118758 118759 118760 118761 118762 118763 118764 118765 118766 118767 118768 118769 | ** ** This means that if we have a pointer into a buffer containing node data, ** it is always safe to read up to two varints from it without risking an ** overread, even if the node data is corrupted. */ #define FTS3_NODE_PADDING (FTS3_VARINT_MAX*2) typedef struct PendingList PendingList; typedef struct SegmentNode SegmentNode; typedef struct SegmentWriter SegmentWriter; /* | > > > > > > > > > > > > > > > > > > > > > > > > > > > < < | > | 119643 119644 119645 119646 119647 119648 119649 119650 119651 119652 119653 119654 119655 119656 119657 119658 119659 119660 119661 119662 119663 119664 119665 119666 119667 119668 119669 119670 119671 119672 119673 119674 119675 119676 119677 119678 119679 119680 119681 119682 119683 119684 119685 119686 119687 119688 119689 119690 | ** ** This means that if we have a pointer into a buffer containing node data, ** it is always safe to read up to two varints from it without risking an ** overread, even if the node data is corrupted. */ #define FTS3_NODE_PADDING (FTS3_VARINT_MAX*2) /* ** Under certain circumstances, b-tree nodes (doclists) can be loaded into ** memory incrementally instead of all at once. This can be a big performance ** win (reduced IO and CPU) if SQLite stops calling the virtual table xNext() ** method before retrieving all query results (as may happen, for example, ** if a query has a LIMIT clause). ** ** Incremental loading is used for b-tree nodes FTS3_NODE_CHUNK_THRESHOLD ** bytes and larger. Nodes are loaded in chunks of FTS3_NODE_CHUNKSIZE bytes. ** The code is written so that the hard lower-limit for each of these values ** is 1. Clearly such small values would be inefficient, but can be useful ** for testing purposes. ** ** If this module is built with SQLITE_TEST defined, these constants may ** be overridden at runtime for testing purposes. File fts3_test.c contains ** a Tcl interface to read and write the values. */ #ifdef SQLITE_TEST int test_fts3_node_chunksize = (4*1024); int test_fts3_node_chunk_threshold = (4*1024)*4; # define FTS3_NODE_CHUNKSIZE test_fts3_node_chunksize # define FTS3_NODE_CHUNK_THRESHOLD test_fts3_node_chunk_threshold #else # define FTS3_NODE_CHUNKSIZE (4*1024) # define FTS3_NODE_CHUNK_THRESHOLD (FTS3_NODE_CHUNKSIZE*4) #endif typedef struct PendingList PendingList; typedef struct SegmentNode SegmentNode; typedef struct SegmentWriter SegmentWriter; /* ** An instance of the following data structure is used to build doclists ** incrementally. See function fts3PendingListAppend() for details. */ struct PendingList { int nData; char *aData; int nSpace; sqlite3_int64 iLastDocid; sqlite3_int64 iLastCol; |
︙ | ︙ | |||
118796 118797 118798 118799 118800 118801 118802 | ** a contiguous set of segment b-tree leaf nodes. Although the details of ** this structure are only manipulated by code in this file, opaque handles ** of type Fts3SegReader* are also used by code in fts3.c to iterate through ** terms when querying the full-text index. See functions: ** ** sqlite3Fts3SegReaderNew() ** sqlite3Fts3SegReaderFree() | < > > > | > > > | 119707 119708 119709 119710 119711 119712 119713 119714 119715 119716 119717 119718 119719 119720 119721 119722 119723 119724 119725 119726 119727 119728 119729 119730 119731 119732 119733 119734 119735 119736 119737 119738 119739 119740 119741 119742 119743 119744 119745 119746 119747 119748 119749 119750 119751 119752 119753 119754 119755 119756 119757 119758 119759 | ** a contiguous set of segment b-tree leaf nodes. Although the details of ** this structure are only manipulated by code in this file, opaque handles ** of type Fts3SegReader* are also used by code in fts3.c to iterate through ** terms when querying the full-text index. See functions: ** ** sqlite3Fts3SegReaderNew() ** sqlite3Fts3SegReaderFree() ** sqlite3Fts3SegReaderIterate() ** ** Methods used to manipulate Fts3SegReader structures: ** ** fts3SegReaderNext() ** fts3SegReaderFirstDocid() ** fts3SegReaderNextDocid() */ struct Fts3SegReader { int iIdx; /* Index within level, or 0x7FFFFFFF for PT */ sqlite3_int64 iStartBlock; /* Rowid of first leaf block to traverse */ sqlite3_int64 iLeafEndBlock; /* Rowid of final leaf block to traverse */ sqlite3_int64 iEndBlock; /* Rowid of final block in segment (or 0) */ sqlite3_int64 iCurrentBlock; /* Current leaf block (or 0) */ char *aNode; /* Pointer to node data (or NULL) */ int nNode; /* Size of buffer at aNode (or 0) */ int nPopulate; /* If >0, bytes of buffer aNode[] loaded */ sqlite3_blob *pBlob; /* If not NULL, blob handle to read node */ Fts3HashElem **ppNextElem; /* Variables set by fts3SegReaderNext(). These may be read directly ** by the caller. They are valid from the time SegmentReaderNew() returns ** until SegmentReaderNext() returns something other than SQLITE_OK ** (i.e. SQLITE_DONE). */ int nTerm; /* Number of bytes in current term */ char *zTerm; /* Pointer to current term */ int nTermAlloc; /* Allocated size of zTerm buffer */ char *aDoclist; /* Pointer to doclist of current entry */ int nDoclist; /* Size of doclist in current entry */ /* The following variables are used by fts3SegReaderNextDocid() to iterate ** through the current doclist (aDoclist/nDoclist). */ char *pOffsetList; int nOffsetList; /* For descending pending seg-readers only */ sqlite3_int64 iDocid; }; #define fts3SegReaderIsPending(p) ((p)->ppNextElem!=0) #define fts3SegReaderIsRootOnly(p) ((p)->aNode==(char *)&(p)[1]) /* |
︙ | ︙ | |||
118867 118868 118869 118870 118871 118872 118873 118874 118875 118876 118877 118878 118879 118880 | ** the interior part of the segment b+-tree structures (everything except ** the leaf nodes). These functions and type are only ever used by code ** within the fts3SegWriterXXX() family of functions described above. ** ** fts3NodeAddTerm() ** fts3NodeWrite() ** fts3NodeFree() */ struct SegmentNode { SegmentNode *pParent; /* Parent node (or NULL for root node) */ SegmentNode *pRight; /* Pointer to right-sibling */ SegmentNode *pLeftmost; /* Pointer to left-most node of this depth */ int nEntry; /* Number of terms written to node so far */ char *zTerm; /* Pointer to previous term buffer */ | > > > > > > > > | 119783 119784 119785 119786 119787 119788 119789 119790 119791 119792 119793 119794 119795 119796 119797 119798 119799 119800 119801 119802 119803 119804 | ** the interior part of the segment b+-tree structures (everything except ** the leaf nodes). These functions and type are only ever used by code ** within the fts3SegWriterXXX() family of functions described above. ** ** fts3NodeAddTerm() ** fts3NodeWrite() ** fts3NodeFree() ** ** When a b+tree is written to the database (either as a result of a merge ** or the pending-terms table being flushed), leaves are written into the ** database file as soon as they are completely populated. The interior of ** the tree is assembled in memory and written out only once all leaves have ** been populated and stored. This is Ok, as the b+-tree fanout is usually ** very large, meaning that the interior of the tree consumes relatively ** little memory. */ struct SegmentNode { SegmentNode *pParent; /* Parent node (or NULL for root node) */ SegmentNode *pRight; /* Pointer to right-sibling */ SegmentNode *pLeftmost; /* Pointer to left-most node of this depth */ int nEntry; /* Number of terms written to node so far */ char *zTerm; /* Pointer to previous term buffer */ |
︙ | ︙ | |||
118897 118898 118899 118900 118901 118902 118903 | #define SQL_DELETE_ALL_STAT 6 #define SQL_SELECT_CONTENT_BY_ROWID 7 #define SQL_NEXT_SEGMENT_INDEX 8 #define SQL_INSERT_SEGMENTS 9 #define SQL_NEXT_SEGMENTS_ID 10 #define SQL_INSERT_SEGDIR 11 #define SQL_SELECT_LEVEL 12 | | | | > > > > > | 119821 119822 119823 119824 119825 119826 119827 119828 119829 119830 119831 119832 119833 119834 119835 119836 119837 119838 119839 119840 119841 119842 119843 119844 119845 119846 119847 119848 119849 119850 | #define SQL_DELETE_ALL_STAT 6 #define SQL_SELECT_CONTENT_BY_ROWID 7 #define SQL_NEXT_SEGMENT_INDEX 8 #define SQL_INSERT_SEGMENTS 9 #define SQL_NEXT_SEGMENTS_ID 10 #define SQL_INSERT_SEGDIR 11 #define SQL_SELECT_LEVEL 12 #define SQL_SELECT_LEVEL_RANGE 13 #define SQL_SELECT_LEVEL_COUNT 14 #define SQL_SELECT_SEGDIR_MAX_LEVEL 15 #define SQL_DELETE_SEGDIR_LEVEL 16 #define SQL_DELETE_SEGMENTS_RANGE 17 #define SQL_CONTENT_INSERT 18 #define SQL_DELETE_DOCSIZE 19 #define SQL_REPLACE_DOCSIZE 20 #define SQL_SELECT_DOCSIZE 21 #define SQL_SELECT_DOCTOTAL 22 #define SQL_REPLACE_DOCTOTAL 23 #define SQL_SELECT_ALL_PREFIX_LEVEL 24 #define SQL_DELETE_ALL_TERMS_SEGDIR 25 #define SQL_DELETE_SEGDIR_RANGE 26 /* ** This function is used to obtain an SQLite prepared statement handle ** for the statement identified by the second argument. If successful, ** *pp is set to the requested statement handle and SQLITE_OK returned. ** Otherwise, an SQLite error code is returned and *pp is set to 0. ** |
︙ | ︙ | |||
118944 118945 118946 118947 118948 118949 118950 | /* 10 */ "SELECT coalesce((SELECT max(blockid) FROM %Q.'%q_segments') + 1, 1)", /* 11 */ "INSERT INTO %Q.'%q_segdir' VALUES(?,?,?,?,?,?)", /* Return segments in order from oldest to newest.*/ /* 12 */ "SELECT idx, start_block, leaves_end_block, end_block, root " "FROM %Q.'%q_segdir' WHERE level = ? ORDER BY idx ASC", /* 13 */ "SELECT idx, start_block, leaves_end_block, end_block, root " | > | | > > > > > | 119873 119874 119875 119876 119877 119878 119879 119880 119881 119882 119883 119884 119885 119886 119887 119888 119889 119890 119891 119892 119893 119894 119895 119896 119897 119898 119899 119900 119901 119902 119903 119904 119905 | /* 10 */ "SELECT coalesce((SELECT max(blockid) FROM %Q.'%q_segments') + 1, 1)", /* 11 */ "INSERT INTO %Q.'%q_segdir' VALUES(?,?,?,?,?,?)", /* Return segments in order from oldest to newest.*/ /* 12 */ "SELECT idx, start_block, leaves_end_block, end_block, root " "FROM %Q.'%q_segdir' WHERE level = ? ORDER BY idx ASC", /* 13 */ "SELECT idx, start_block, leaves_end_block, end_block, root " "FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?" "ORDER BY level DESC, idx ASC", /* 14 */ "SELECT count(*) FROM %Q.'%q_segdir' WHERE level = ?", /* 15 */ "SELECT max(level) FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?", /* 16 */ "DELETE FROM %Q.'%q_segdir' WHERE level = ?", /* 17 */ "DELETE FROM %Q.'%q_segments' WHERE blockid BETWEEN ? AND ?", /* 18 */ "INSERT INTO %Q.'%q_content' VALUES(%s)", /* 19 */ "DELETE FROM %Q.'%q_docsize' WHERE docid = ?", /* 20 */ "REPLACE INTO %Q.'%q_docsize' VALUES(?,?)", /* 21 */ "SELECT size FROM %Q.'%q_docsize' WHERE docid=?", /* 22 */ "SELECT value FROM %Q.'%q_stat' WHERE id=0", /* 23 */ "REPLACE INTO %Q.'%q_stat' VALUES(0,?)", /* 24 */ "", /* 25 */ "", /* 26 */ "DELETE FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ?", }; int rc = SQLITE_OK; sqlite3_stmt *pStmt; assert( SizeofArray(azSql)==SizeofArray(p->aStmt) ); assert( eStmt<SizeofArray(azSql) && eStmt>=0 ); |
︙ | ︙ | |||
119112 119113 119114 119115 119116 119117 119118 | ** ** 0: idx ** 1: start_block ** 2: leaves_end_block ** 3: end_block ** 4: root */ | | > > > > > > > > > > > | > > > > > | > > | 120047 120048 120049 120050 120051 120052 120053 120054 120055 120056 120057 120058 120059 120060 120061 120062 120063 120064 120065 120066 120067 120068 120069 120070 120071 120072 120073 120074 120075 120076 120077 120078 120079 120080 120081 120082 120083 120084 120085 120086 | ** ** 0: idx ** 1: start_block ** 2: leaves_end_block ** 3: end_block ** 4: root */ SQLITE_PRIVATE int sqlite3Fts3AllSegdirs( Fts3Table *p, /* FTS3 table */ int iIndex, /* Index for p->aIndex[] */ int iLevel, /* Level to select */ sqlite3_stmt **ppStmt /* OUT: Compiled statement */ ){ int rc; sqlite3_stmt *pStmt = 0; assert( iLevel==FTS3_SEGCURSOR_ALL || iLevel>=0 ); assert( iLevel<FTS3_SEGDIR_MAXLEVEL ); assert( iIndex>=0 && iIndex<p->nIndex ); if( iLevel<0 ){ /* "SELECT * FROM %_segdir WHERE level BETWEEN ? AND ? ORDER BY ..." */ rc = fts3SqlStmt(p, SQL_SELECT_LEVEL_RANGE, &pStmt, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int(pStmt, 1, iIndex*FTS3_SEGDIR_MAXLEVEL); sqlite3_bind_int(pStmt, 2, (iIndex+1)*FTS3_SEGDIR_MAXLEVEL-1); } }else{ /* "SELECT * FROM %_segdir WHERE level = ? ORDER BY ..." */ rc = fts3SqlStmt(p, SQL_SELECT_LEVEL, &pStmt, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int(pStmt, 1, iLevel+iIndex*FTS3_SEGDIR_MAXLEVEL); } } *ppStmt = pStmt; return rc; } /* |
︙ | ︙ | |||
119233 119234 119235 119236 119237 119238 119239 119240 119241 119242 119243 119244 119245 119246 | *pRc = rc; if( p!=*pp ){ *pp = p; return 1; } return 0; } /* ** Tokenize the nul-terminated string zText and add all tokens to the ** pending-terms hash-table. The docid used is that currently stored in ** p->iPrevDocid, and the column is specified by argument iCol. ** ** If successful, SQLITE_OK is returned. Otherwise, an SQLite error code. | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 120186 120187 120188 120189 120190 120191 120192 120193 120194 120195 120196 120197 120198 120199 120200 120201 120202 120203 120204 120205 120206 120207 120208 120209 120210 120211 120212 120213 120214 120215 120216 120217 120218 120219 120220 120221 120222 120223 120224 120225 120226 120227 120228 120229 120230 120231 120232 120233 120234 120235 120236 120237 120238 120239 120240 | *pRc = rc; if( p!=*pp ){ *pp = p; return 1; } return 0; } /* ** Free a PendingList object allocated by fts3PendingListAppend(). */ static void fts3PendingListDelete(PendingList *pList){ sqlite3_free(pList); } /* ** Add an entry to one of the pending-terms hash tables. */ static int fts3PendingTermsAddOne( Fts3Table *p, int iCol, int iPos, Fts3Hash *pHash, /* Pending terms hash table to add entry to */ const char *zToken, int nToken ){ PendingList *pList; int rc = SQLITE_OK; pList = (PendingList *)fts3HashFind(pHash, zToken, nToken); if( pList ){ p->nPendingData -= (pList->nData + nToken + sizeof(Fts3HashElem)); } if( fts3PendingListAppend(&pList, p->iPrevDocid, iCol, iPos, &rc) ){ if( pList==fts3HashInsert(pHash, zToken, nToken, pList) ){ /* Malloc failed while inserting the new entry. This can only ** happen if there was no previous entry for this token. */ assert( 0==fts3HashFind(pHash, zToken, nToken) ); sqlite3_free(pList); rc = SQLITE_NOMEM; } } if( rc==SQLITE_OK ){ p->nPendingData += (pList->nData + nToken + sizeof(Fts3HashElem)); } return rc; } /* ** Tokenize the nul-terminated string zText and add all tokens to the ** pending-terms hash-table. The docid used is that currently stored in ** p->iPrevDocid, and the column is specified by argument iCol. ** ** If successful, SQLITE_OK is returned. Otherwise, an SQLite error code. |
︙ | ︙ | |||
119282 119283 119284 119285 119286 119287 119288 | } pCsr->pTokenizer = pTokenizer; xNext = pModule->xNext; while( SQLITE_OK==rc && SQLITE_OK==(rc = xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos)) ){ | < | | < < < | < < < < | | < | < > > | > > | > > | 120276 120277 120278 120279 120280 120281 120282 120283 120284 120285 120286 120287 120288 120289 120290 120291 120292 120293 120294 120295 120296 120297 120298 120299 120300 120301 120302 120303 120304 120305 120306 120307 120308 120309 120310 120311 120312 120313 | } pCsr->pTokenizer = pTokenizer; xNext = pModule->xNext; while( SQLITE_OK==rc && SQLITE_OK==(rc = xNext(pCsr, &zToken, &nToken, &iStart, &iEnd, &iPos)) ){ int i; if( iPos>=nWord ) nWord = iPos+1; /* Positions cannot be negative; we use -1 as a terminator internally. ** Tokens must have a non-zero length. */ if( iPos<0 || !zToken || nToken<=0 ){ rc = SQLITE_ERROR; break; } /* Add the term to the terms index */ rc = fts3PendingTermsAddOne( p, iCol, iPos, &p->aIndex[0].hPending, zToken, nToken ); /* Add the term to each of the prefix indexes that it is not too ** short for. */ for(i=1; rc==SQLITE_OK && i<p->nIndex; i++){ struct Fts3Index *pIndex = &p->aIndex[i]; if( nToken<pIndex->nPrefix ) continue; rc = fts3PendingTermsAddOne( p, iCol, iPos, &pIndex->hPending, zToken, pIndex->nPrefix ); } } pModule->xClose(pCsr); *pnWord = nWord; return (rc==SQLITE_DONE ? SQLITE_OK : rc); } |
︙ | ︙ | |||
119339 119340 119341 119342 119343 119344 119345 | if( rc!=SQLITE_OK ) return rc; } p->iPrevDocid = iDocid; return SQLITE_OK; } /* | | > > | > | | > | | > | 120329 120330 120331 120332 120333 120334 120335 120336 120337 120338 120339 120340 120341 120342 120343 120344 120345 120346 120347 120348 120349 120350 120351 120352 120353 120354 120355 | if( rc!=SQLITE_OK ) return rc; } p->iPrevDocid = iDocid; return SQLITE_OK; } /* ** Discard the contents of the pending-terms hash tables. */ SQLITE_PRIVATE void sqlite3Fts3PendingTermsClear(Fts3Table *p){ int i; for(i=0; i<p->nIndex; i++){ Fts3HashElem *pElem; Fts3Hash *pHash = &p->aIndex[i].hPending; for(pElem=fts3HashFirst(pHash); pElem; pElem=fts3HashNext(pElem)){ PendingList *pList = (PendingList *)fts3HashData(pElem); fts3PendingListDelete(pList); } fts3HashClear(pHash); } p->nPendingData = 0; } /* ** This function is called by the xUpdate() method as part of an INSERT ** operation. It adds entries for each term in the new record to the ** pendingTerms hash table. |
︙ | ︙ | |||
119502 119503 119504 119505 119506 119507 119508 | *pRC = rc; } /* ** Forward declaration to account for the circular dependency between ** functions fts3SegmentMerge() and fts3AllocateSegdirIdx(). */ | | | > > > > > | | | 120497 120498 120499 120500 120501 120502 120503 120504 120505 120506 120507 120508 120509 120510 120511 120512 120513 120514 120515 120516 120517 120518 120519 120520 120521 120522 120523 120524 120525 120526 120527 120528 120529 120530 120531 120532 120533 120534 120535 120536 120537 120538 120539 120540 120541 120542 120543 120544 120545 120546 120547 120548 120549 120550 120551 120552 120553 120554 120555 | *pRC = rc; } /* ** Forward declaration to account for the circular dependency between ** functions fts3SegmentMerge() and fts3AllocateSegdirIdx(). */ static int fts3SegmentMerge(Fts3Table *, int, int); /* ** This function allocates a new level iLevel index in the segdir table. ** Usually, indexes are allocated within a level sequentially starting ** with 0, so the allocated index is one greater than the value returned ** by: ** ** SELECT max(idx) FROM %_segdir WHERE level = :iLevel ** ** However, if there are already FTS3_MERGE_COUNT indexes at the requested ** level, they are merged into a single level (iLevel+1) segment and the ** allocated index is 0. ** ** If successful, *piIdx is set to the allocated index slot and SQLITE_OK ** returned. Otherwise, an SQLite error code is returned. */ static int fts3AllocateSegdirIdx( Fts3Table *p, int iIndex, /* Index for p->aIndex */ int iLevel, int *piIdx ){ int rc; /* Return Code */ sqlite3_stmt *pNextIdx; /* Query for next idx at level iLevel */ int iNext = 0; /* Result of query pNextIdx */ /* Set variable iNext to the next available segdir index at level iLevel. */ rc = fts3SqlStmt(p, SQL_NEXT_SEGMENT_INDEX, &pNextIdx, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int(pNextIdx, 1, iIndex*FTS3_SEGDIR_MAXLEVEL + iLevel); if( SQLITE_ROW==sqlite3_step(pNextIdx) ){ iNext = sqlite3_column_int(pNextIdx, 0); } rc = sqlite3_reset(pNextIdx); } if( rc==SQLITE_OK ){ /* If iNext is FTS3_MERGE_COUNT, indicating that level iLevel is already ** full, merge all segments in level iLevel into a single iLevel+1 ** segment and allocate (newly freed) index 0 at level iLevel. Otherwise, ** if iNext is less than FTS3_MERGE_COUNT, allocate index iNext. */ if( iNext>=FTS3_MERGE_COUNT ){ rc = fts3SegmentMerge(p, iIndex, iLevel); *piIdx = 0; }else{ *piIdx = iNext; } } return rc; |
︙ | ︙ | |||
119582 119583 119584 119585 119586 119587 119588 | ** method (xFilter etc.) that may directly or indirectly call this function ** must call sqlite3Fts3SegmentsClose() before returning. */ SQLITE_PRIVATE int sqlite3Fts3ReadBlock( Fts3Table *p, /* FTS3 table handle */ sqlite3_int64 iBlockid, /* Access the row with blockid=$iBlockid */ char **paBlob, /* OUT: Blob data in malloc'd buffer */ | | > > > > > > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > < > > | > > > > > > | > > > > > | < > > > > | | | > > > > > > > > > > > | | > > > | > | | | > < < < < < | > > > | | < | | | | | | | > > | | < < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > | | | | > | > > | | < < < < < < < | | | | > > | < | < < < < | > > > > | | | < | | < < < < < < < < < < < < < < < < < | | < < < < < < < < < < < < < < < < < < < < < < < < < < | | | < | | | | | > | 120582 120583 120584 120585 120586 120587 120588 120589 120590 120591 120592 120593 120594 120595 120596 120597 120598 120599 120600 120601 120602 120603 120604 120605 120606 120607 120608 120609 120610 120611 120612 120613 120614 120615 120616 120617 120618 120619 120620 120621 120622 120623 120624 120625 120626 120627 120628 120629 120630 120631 120632 120633 120634 120635 120636 120637 120638 120639 120640 120641 120642 120643 120644 120645 120646 120647 120648 120649 120650 120651 120652 120653 120654 120655 120656 120657 120658 120659 120660 120661 120662 120663 120664 120665 120666 120667 120668 120669 120670 120671 120672 120673 120674 120675 120676 120677 120678 120679 120680 120681 120682 120683 120684 120685 120686 120687 120688 120689 120690 120691 120692 120693 120694 120695 120696 120697 120698 120699 120700 120701 120702 120703 120704 120705 120706 120707 120708 120709 120710 120711 120712 120713 120714 120715 120716 120717 120718 120719 120720 120721 120722 120723 120724 120725 120726 120727 120728 120729 120730 120731 120732 120733 120734 120735 120736 120737 120738 120739 120740 120741 120742 120743 120744 120745 120746 120747 120748 120749 120750 120751 120752 120753 120754 120755 120756 120757 120758 120759 120760 120761 120762 120763 120764 120765 120766 120767 120768 120769 120770 120771 120772 120773 120774 120775 120776 120777 120778 120779 120780 120781 120782 120783 120784 120785 120786 120787 120788 120789 120790 120791 120792 120793 120794 120795 120796 120797 120798 120799 120800 120801 120802 120803 120804 120805 120806 120807 120808 120809 120810 120811 120812 120813 120814 120815 120816 120817 120818 120819 120820 120821 120822 120823 120824 120825 120826 120827 120828 120829 120830 120831 120832 120833 120834 120835 120836 120837 120838 120839 120840 120841 120842 120843 120844 120845 120846 120847 120848 120849 120850 120851 120852 120853 120854 120855 120856 120857 120858 120859 120860 120861 120862 120863 120864 120865 120866 120867 120868 120869 120870 120871 120872 120873 120874 120875 120876 120877 120878 120879 120880 120881 120882 120883 120884 120885 120886 120887 120888 120889 120890 120891 120892 120893 120894 120895 120896 120897 120898 120899 120900 120901 120902 120903 120904 120905 120906 120907 120908 120909 120910 120911 120912 120913 120914 120915 120916 120917 120918 120919 120920 120921 120922 120923 120924 120925 120926 120927 120928 120929 120930 120931 120932 120933 120934 120935 120936 120937 120938 120939 120940 120941 120942 120943 120944 120945 120946 120947 120948 120949 120950 120951 120952 120953 120954 120955 120956 120957 120958 120959 120960 120961 120962 120963 120964 120965 120966 120967 120968 | ** method (xFilter etc.) that may directly or indirectly call this function ** must call sqlite3Fts3SegmentsClose() before returning. */ SQLITE_PRIVATE int sqlite3Fts3ReadBlock( Fts3Table *p, /* FTS3 table handle */ sqlite3_int64 iBlockid, /* Access the row with blockid=$iBlockid */ char **paBlob, /* OUT: Blob data in malloc'd buffer */ int *pnBlob, /* OUT: Size of blob data */ int *pnLoad /* OUT: Bytes actually loaded */ ){ int rc; /* Return code */ /* pnBlob must be non-NULL. paBlob may be NULL or non-NULL. */ assert( pnBlob); if( p->pSegments ){ rc = sqlite3_blob_reopen(p->pSegments, iBlockid); }else{ if( 0==p->zSegmentsTbl ){ p->zSegmentsTbl = sqlite3_mprintf("%s_segments", p->zName); if( 0==p->zSegmentsTbl ) return SQLITE_NOMEM; } rc = sqlite3_blob_open( p->db, p->zDb, p->zSegmentsTbl, "block", iBlockid, 0, &p->pSegments ); } if( rc==SQLITE_OK ){ int nByte = sqlite3_blob_bytes(p->pSegments); *pnBlob = nByte; if( paBlob ){ char *aByte = sqlite3_malloc(nByte + FTS3_NODE_PADDING); if( !aByte ){ rc = SQLITE_NOMEM; }else{ if( pnLoad && nByte>(FTS3_NODE_CHUNK_THRESHOLD) ){ nByte = FTS3_NODE_CHUNKSIZE; *pnLoad = nByte; } rc = sqlite3_blob_read(p->pSegments, aByte, nByte, 0); memset(&aByte[nByte], 0, FTS3_NODE_PADDING); if( rc!=SQLITE_OK ){ sqlite3_free(aByte); aByte = 0; } } *paBlob = aByte; } } return rc; } /* ** Close the blob handle at p->pSegments, if it is open. See comments above ** the sqlite3Fts3ReadBlock() function for details. */ SQLITE_PRIVATE void sqlite3Fts3SegmentsClose(Fts3Table *p){ sqlite3_blob_close(p->pSegments); p->pSegments = 0; } static int fts3SegReaderIncrRead(Fts3SegReader *pReader){ int nRead; /* Number of bytes to read */ int rc; /* Return code */ nRead = MIN(pReader->nNode - pReader->nPopulate, FTS3_NODE_CHUNKSIZE); rc = sqlite3_blob_read( pReader->pBlob, &pReader->aNode[pReader->nPopulate], nRead, pReader->nPopulate ); if( rc==SQLITE_OK ){ pReader->nPopulate += nRead; memset(&pReader->aNode[pReader->nPopulate], 0, FTS3_NODE_PADDING); if( pReader->nPopulate==pReader->nNode ){ sqlite3_blob_close(pReader->pBlob); pReader->pBlob = 0; pReader->nPopulate = 0; } } return rc; } static int fts3SegReaderRequire(Fts3SegReader *pReader, char *pFrom, int nByte){ int rc = SQLITE_OK; assert( !pReader->pBlob || (pFrom>=pReader->aNode && pFrom<&pReader->aNode[pReader->nNode]) ); while( pReader->pBlob && rc==SQLITE_OK && (pFrom - pReader->aNode + nByte)>pReader->nPopulate ){ rc = fts3SegReaderIncrRead(pReader); } return rc; } /* ** Move the iterator passed as the first argument to the next term in the ** segment. If successful, SQLITE_OK is returned. If there is no next term, ** SQLITE_DONE. Otherwise, an SQLite error code. */ static int fts3SegReaderNext( Fts3Table *p, Fts3SegReader *pReader, int bIncr ){ int rc; /* Return code of various sub-routines */ char *pNext; /* Cursor variable */ int nPrefix; /* Number of bytes in term prefix */ int nSuffix; /* Number of bytes in term suffix */ if( !pReader->aDoclist ){ pNext = pReader->aNode; }else{ pNext = &pReader->aDoclist[pReader->nDoclist]; } if( !pNext || pNext>=&pReader->aNode[pReader->nNode] ){ if( fts3SegReaderIsPending(pReader) ){ Fts3HashElem *pElem = *(pReader->ppNextElem); if( pElem==0 ){ pReader->aNode = 0; }else{ PendingList *pList = (PendingList *)fts3HashData(pElem); pReader->zTerm = (char *)fts3HashKey(pElem); pReader->nTerm = fts3HashKeysize(pElem); pReader->nNode = pReader->nDoclist = pList->nData + 1; pReader->aNode = pReader->aDoclist = pList->aData; pReader->ppNextElem++; assert( pReader->aNode ); } return SQLITE_OK; } if( !fts3SegReaderIsRootOnly(pReader) ){ sqlite3_free(pReader->aNode); sqlite3_blob_close(pReader->pBlob); pReader->pBlob = 0; } pReader->aNode = 0; /* If iCurrentBlock>=iLeafEndBlock, this is an EOF condition. All leaf ** blocks have already been traversed. */ assert( pReader->iCurrentBlock<=pReader->iLeafEndBlock ); if( pReader->iCurrentBlock>=pReader->iLeafEndBlock ){ return SQLITE_OK; } rc = sqlite3Fts3ReadBlock( p, ++pReader->iCurrentBlock, &pReader->aNode, &pReader->nNode, (bIncr ? &pReader->nPopulate : 0) ); if( rc!=SQLITE_OK ) return rc; assert( pReader->pBlob==0 ); if( bIncr && pReader->nPopulate<pReader->nNode ){ pReader->pBlob = p->pSegments; p->pSegments = 0; } pNext = pReader->aNode; } assert( !fts3SegReaderIsPending(pReader) ); rc = fts3SegReaderRequire(pReader, pNext, FTS3_VARINT_MAX*2); if( rc!=SQLITE_OK ) return rc; /* Because of the FTS3_NODE_PADDING bytes of padding, the following is ** safe (no risk of overread) even if the node data is corrupted. */ pNext += sqlite3Fts3GetVarint32(pNext, &nPrefix); pNext += sqlite3Fts3GetVarint32(pNext, &nSuffix); if( nPrefix<0 || nSuffix<=0 || &pNext[nSuffix]>&pReader->aNode[pReader->nNode] ){ return SQLITE_CORRUPT_VTAB; } if( nPrefix+nSuffix>pReader->nTermAlloc ){ int nNew = (nPrefix+nSuffix)*2; char *zNew = sqlite3_realloc(pReader->zTerm, nNew); if( !zNew ){ return SQLITE_NOMEM; } pReader->zTerm = zNew; pReader->nTermAlloc = nNew; } rc = fts3SegReaderRequire(pReader, pNext, nSuffix+FTS3_VARINT_MAX); if( rc!=SQLITE_OK ) return rc; memcpy(&pReader->zTerm[nPrefix], pNext, nSuffix); pReader->nTerm = nPrefix+nSuffix; pNext += nSuffix; pNext += sqlite3Fts3GetVarint32(pNext, &pReader->nDoclist); pReader->aDoclist = pNext; pReader->pOffsetList = 0; /* Check that the doclist does not appear to extend past the end of the ** b-tree node. And that the final byte of the doclist is 0x00. If either ** of these statements is untrue, then the data structure is corrupt. */ if( &pReader->aDoclist[pReader->nDoclist]>&pReader->aNode[pReader->nNode] || (pReader->nPopulate==0 && pReader->aDoclist[pReader->nDoclist-1]) ){ return SQLITE_CORRUPT_VTAB; } return SQLITE_OK; } /* ** Set the SegReader to point to the first docid in the doclist associated ** with the current term. */ static int fts3SegReaderFirstDocid(Fts3Table *pTab, Fts3SegReader *pReader){ int rc = SQLITE_OK; assert( pReader->aDoclist ); assert( !pReader->pOffsetList ); if( pTab->bDescIdx && fts3SegReaderIsPending(pReader) ){ u8 bEof = 0; pReader->iDocid = 0; pReader->nOffsetList = 0; sqlite3Fts3DoclistPrev(0, pReader->aDoclist, pReader->nDoclist, &pReader->pOffsetList, &pReader->iDocid, &pReader->nOffsetList, &bEof ); }else{ rc = fts3SegReaderRequire(pReader, pReader->aDoclist, FTS3_VARINT_MAX); if( rc==SQLITE_OK ){ int n = sqlite3Fts3GetVarint(pReader->aDoclist, &pReader->iDocid); pReader->pOffsetList = &pReader->aDoclist[n]; } } return rc; } /* ** Advance the SegReader to point to the next docid in the doclist ** associated with the current term. ** ** If arguments ppOffsetList and pnOffsetList are not NULL, then ** *ppOffsetList is set to point to the first column-offset list ** in the doclist entry (i.e. immediately past the docid varint). ** *pnOffsetList is set to the length of the set of column-offset ** lists, not including the nul-terminator byte. For example: */ static int fts3SegReaderNextDocid( Fts3Table *pTab, Fts3SegReader *pReader, /* Reader to advance to next docid */ char **ppOffsetList, /* OUT: Pointer to current position-list */ int *pnOffsetList /* OUT: Length of *ppOffsetList in bytes */ ){ int rc = SQLITE_OK; char *p = pReader->pOffsetList; char c = 0; assert( p ); if( pTab->bDescIdx && fts3SegReaderIsPending(pReader) ){ /* A pending-terms seg-reader for an FTS4 table that uses order=desc. ** Pending-terms doclists are always built up in ascending order, so ** we have to iterate through them backwards here. */ u8 bEof = 0; if( ppOffsetList ){ *ppOffsetList = pReader->pOffsetList; *pnOffsetList = pReader->nOffsetList - 1; } sqlite3Fts3DoclistPrev(0, pReader->aDoclist, pReader->nDoclist, &p, &pReader->iDocid, &pReader->nOffsetList, &bEof ); if( bEof ){ pReader->pOffsetList = 0; }else{ pReader->pOffsetList = p; } }else{ char *pEnd = &pReader->aDoclist[pReader->nDoclist]; /* Pointer p currently points at the first byte of an offset list. The ** following block advances it to point one byte past the end of ** the same offset list. */ while( 1 ){ /* The following line of code (and the "p++" below the while() loop) is ** normally all that is required to move pointer p to the desired ** position. The exception is if this node is being loaded from disk ** incrementally and pointer "p" now points to the first byte passed ** the populated part of pReader->aNode[]. */ while( *p | c ) c = *p++ & 0x80; assert( *p==0 ); if( pReader->pBlob==0 || p<&pReader->aNode[pReader->nPopulate] ) break; rc = fts3SegReaderIncrRead(pReader); if( rc!=SQLITE_OK ) return rc; } p++; /* If required, populate the output variables with a pointer to and the ** size of the previous offset-list. */ if( ppOffsetList ){ *ppOffsetList = pReader->pOffsetList; *pnOffsetList = (int)(p - pReader->pOffsetList - 1); } while( p<pEnd && *p==0 ) p++; /* If there are no more entries in the doclist, set pOffsetList to ** NULL. Otherwise, set Fts3SegReader.iDocid to the next docid and ** Fts3SegReader.pOffsetList to point to the next offset list before ** returning. */ if( p>=pEnd ){ pReader->pOffsetList = 0; }else{ rc = fts3SegReaderRequire(pReader, p, FTS3_VARINT_MAX); if( rc==SQLITE_OK ){ sqlite3_int64 iDelta; pReader->pOffsetList = p + sqlite3Fts3GetVarint(p, &iDelta); if( pTab->bDescIdx ){ pReader->iDocid -= iDelta; }else{ pReader->iDocid += iDelta; } } } } return SQLITE_OK; } SQLITE_PRIVATE int sqlite3Fts3MsrOvfl( Fts3Cursor *pCsr, Fts3MultiSegReader *pMsr, int *pnOvfl ){ Fts3Table *p = (Fts3Table*)pCsr->base.pVtab; int nOvfl = 0; int ii; int rc = SQLITE_OK; int pgsz = p->nPgsz; assert( p->bHasStat ); assert( pgsz>0 ); for(ii=0; rc==SQLITE_OK && ii<pMsr->nSegment; ii++){ Fts3SegReader *pReader = pMsr->apSegment[ii]; if( !fts3SegReaderIsPending(pReader) && !fts3SegReaderIsRootOnly(pReader) ){ sqlite3_int64 jj; for(jj=pReader->iStartBlock; jj<=pReader->iLeafEndBlock; jj++){ int nBlob; rc = sqlite3Fts3ReadBlock(p, jj, 0, &nBlob, 0); if( rc!=SQLITE_OK ) break; if( (nBlob+35)>pgsz ){ nOvfl += (nBlob + 34)/pgsz; } } } } *pnOvfl = nOvfl; return rc; } /* ** Free all allocations associated with the iterator passed as the ** second argument. */ SQLITE_PRIVATE void sqlite3Fts3SegReaderFree(Fts3SegReader *pReader){ if( pReader && !fts3SegReaderIsPending(pReader) ){ sqlite3_free(pReader->zTerm); if( !fts3SegReaderIsRootOnly(pReader) ){ sqlite3_free(pReader->aNode); sqlite3_blob_close(pReader->pBlob); } } sqlite3_free(pReader); } /* ** Allocate a new SegReader object. |
︙ | ︙ | |||
119957 119958 119959 119960 119961 119962 119963 119964 119965 119966 119967 119968 | } return c; } /* ** This function is used to allocate an Fts3SegReader that iterates through ** a subset of the terms stored in the Fts3Table.pendingTerms array. */ SQLITE_PRIVATE int sqlite3Fts3SegReaderPending( Fts3Table *p, /* Virtual table handle */ const char *zTerm, /* Term to search for */ int nTerm, /* Size of buffer zTerm */ | > > > > > > > > > > > > > > > > | > > | | > > > | | | 121031 121032 121033 121034 121035 121036 121037 121038 121039 121040 121041 121042 121043 121044 121045 121046 121047 121048 121049 121050 121051 121052 121053 121054 121055 121056 121057 121058 121059 121060 121061 121062 121063 121064 121065 121066 121067 121068 121069 121070 121071 121072 121073 121074 121075 121076 121077 121078 121079 121080 121081 121082 121083 121084 121085 121086 121087 121088 121089 121090 121091 121092 121093 121094 121095 121096 121097 121098 121099 121100 121101 121102 121103 121104 121105 121106 121107 121108 121109 121110 121111 121112 121113 121114 121115 121116 121117 121118 121119 121120 121121 121122 121123 121124 121125 121126 121127 121128 121129 121130 121131 121132 121133 | } return c; } /* ** This function is used to allocate an Fts3SegReader that iterates through ** a subset of the terms stored in the Fts3Table.pendingTerms array. ** ** If the isPrefixIter parameter is zero, then the returned SegReader iterates ** through each term in the pending-terms table. Or, if isPrefixIter is ** non-zero, it iterates through each term and its prefixes. For example, if ** the pending terms hash table contains the terms "sqlite", "mysql" and ** "firebird", then the iterator visits the following 'terms' (in the order ** shown): ** ** f fi fir fire fireb firebi firebir firebird ** m my mys mysq mysql ** s sq sql sqli sqlit sqlite ** ** Whereas if isPrefixIter is zero, the terms visited are: ** ** firebird mysql sqlite */ SQLITE_PRIVATE int sqlite3Fts3SegReaderPending( Fts3Table *p, /* Virtual table handle */ int iIndex, /* Index for p->aIndex */ const char *zTerm, /* Term to search for */ int nTerm, /* Size of buffer zTerm */ int bPrefix, /* True for a prefix iterator */ Fts3SegReader **ppReader /* OUT: SegReader for pending-terms */ ){ Fts3SegReader *pReader = 0; /* Fts3SegReader object to return */ Fts3HashElem **aElem = 0; /* Array of term hash entries to scan */ int nElem = 0; /* Size of array at aElem */ int rc = SQLITE_OK; /* Return Code */ Fts3Hash *pHash; pHash = &p->aIndex[iIndex].hPending; if( bPrefix ){ int nAlloc = 0; /* Size of allocated array at aElem */ Fts3HashElem *pE = 0; /* Iterator variable */ for(pE=fts3HashFirst(pHash); pE; pE=fts3HashNext(pE)){ char *zKey = (char *)fts3HashKey(pE); int nKey = fts3HashKeysize(pE); if( nTerm==0 || (nKey>=nTerm && 0==memcmp(zKey, zTerm, nTerm)) ){ if( nElem==nAlloc ){ Fts3HashElem **aElem2; nAlloc += 16; aElem2 = (Fts3HashElem **)sqlite3_realloc( aElem, nAlloc*sizeof(Fts3HashElem *) ); if( !aElem2 ){ rc = SQLITE_NOMEM; nElem = 0; break; } aElem = aElem2; } aElem[nElem++] = pE; } } /* If more than one term matches the prefix, sort the Fts3HashElem ** objects in term order using qsort(). This uses the same comparison ** callback as is used when flushing terms to disk. */ if( nElem>1 ){ qsort(aElem, nElem, sizeof(Fts3HashElem *), fts3CompareElemByTerm); } }else{ /* The query is a simple term lookup that matches at most one term in ** the index. All that is required is a straight hash-lookup. */ Fts3HashElem *pE = fts3HashFindElem(pHash, zTerm, nTerm); if( pE ){ aElem = &pE; nElem = 1; } } if( nElem>0 ){ int nByte = sizeof(Fts3SegReader) + (nElem+1)*sizeof(Fts3HashElem *); pReader = (Fts3SegReader *)sqlite3_malloc(nByte); if( !pReader ){ rc = SQLITE_NOMEM; }else{ memset(pReader, 0, nByte); pReader->iIdx = 0x7FFFFFFF; pReader->ppNextElem = (Fts3HashElem **)&pReader[1]; memcpy(pReader->ppNextElem, aElem, nElem*sizeof(Fts3HashElem *)); } } if( bPrefix ){ sqlite3_free(aElem); } *ppReader = pReader; return rc; } /* |
︙ | ︙ | |||
120084 120085 120086 120087 120088 120089 120090 120091 120092 120093 120094 120095 120096 120097 | int rc = (pLhs->pOffsetList==0)-(pRhs->pOffsetList==0); if( rc==0 ){ if( pLhs->iDocid==pRhs->iDocid ){ rc = pRhs->iIdx - pLhs->iIdx; }else{ rc = (pLhs->iDocid > pRhs->iDocid) ? 1 : -1; } } assert( pLhs->aNode && pRhs->aNode ); return rc; } /* ** Compare the term that the Fts3SegReader object passed as the first argument | > > > > > > > > > > > > | 121179 121180 121181 121182 121183 121184 121185 121186 121187 121188 121189 121190 121191 121192 121193 121194 121195 121196 121197 121198 121199 121200 121201 121202 121203 121204 | int rc = (pLhs->pOffsetList==0)-(pRhs->pOffsetList==0); if( rc==0 ){ if( pLhs->iDocid==pRhs->iDocid ){ rc = pRhs->iIdx - pLhs->iIdx; }else{ rc = (pLhs->iDocid > pRhs->iDocid) ? 1 : -1; } } assert( pLhs->aNode && pRhs->aNode ); return rc; } static int fts3SegReaderDoclistCmpRev(Fts3SegReader *pLhs, Fts3SegReader *pRhs){ int rc = (pLhs->pOffsetList==0)-(pRhs->pOffsetList==0); if( rc==0 ){ if( pLhs->iDocid==pRhs->iDocid ){ rc = pRhs->iIdx - pLhs->iIdx; }else{ rc = (pLhs->iDocid < pRhs->iDocid) ? 1 : -1; } } assert( pLhs->aNode && pRhs->aNode ); return rc; } /* ** Compare the term that the Fts3SegReader object passed as the first argument |
︙ | ︙ | |||
120636 120637 120638 120639 120640 120641 120642 | } rc = sqlite3_reset(pStmt); } return rc; } /* | < | > > | | > > > > > > > | > > < | > | 121743 121744 121745 121746 121747 121748 121749 121750 121751 121752 121753 121754 121755 121756 121757 121758 121759 121760 121761 121762 121763 121764 121765 121766 121767 121768 121769 121770 121771 121772 121773 121774 121775 121776 121777 121778 121779 121780 121781 121782 121783 121784 121785 121786 121787 121788 121789 121790 121791 121792 121793 121794 121795 121796 121797 121798 121799 121800 121801 | } rc = sqlite3_reset(pStmt); } return rc; } /* ** Set *pnMax to the largest segment level in the database for the index ** iIndex. ** ** Segment levels are stored in the 'level' column of the %_segdir table. ** ** Return SQLITE_OK if successful, or an SQLite error code if not. */ static int fts3SegmentMaxLevel(Fts3Table *p, int iIndex, int *pnMax){ sqlite3_stmt *pStmt; int rc; assert( iIndex>=0 && iIndex<p->nIndex ); /* Set pStmt to the compiled version of: ** ** SELECT max(level) FROM %Q.'%q_segdir' WHERE level BETWEEN ? AND ? ** ** (1024 is actually the value of macro FTS3_SEGDIR_PREFIXLEVEL_STR). */ rc = fts3SqlStmt(p, SQL_SELECT_SEGDIR_MAX_LEVEL, &pStmt, 0); if( rc!=SQLITE_OK ) return rc; sqlite3_bind_int(pStmt, 1, iIndex*FTS3_SEGDIR_MAXLEVEL); sqlite3_bind_int(pStmt, 2, (iIndex+1)*FTS3_SEGDIR_MAXLEVEL - 1); if( SQLITE_ROW==sqlite3_step(pStmt) ){ *pnMax = sqlite3_column_int(pStmt, 0); } return sqlite3_reset(pStmt); } /* ** This function is used after merging multiple segments into a single large ** segment to delete the old, now redundant, segment b-trees. Specifically, ** it: ** ** 1) Deletes all %_segments entries for the segments associated with ** each of the SegReader objects in the array passed as the third ** argument, and ** ** 2) deletes all %_segdir entries with level iLevel, or all %_segdir ** entries regardless of level if (iLevel<0). ** ** SQLITE_OK is returned if successful, otherwise an SQLite error code. */ static int fts3DeleteSegdir( Fts3Table *p, /* Virtual table handle */ int iIndex, /* Index for p->aIndex */ int iLevel, /* Level of %_segdir entries to delete */ Fts3SegReader **apSegment, /* Array of SegReader objects */ int nReader /* Size of array apSegment */ ){ int rc; /* Return Code */ int i; /* Iterator variable */ sqlite3_stmt *pDelete; /* SQL statement to delete rows */ |
︙ | ︙ | |||
120693 120694 120695 120696 120697 120698 120699 120700 | rc = sqlite3_reset(pDelete); } } if( rc!=SQLITE_OK ){ return rc; } if( iLevel==FTS3_SEGCURSOR_ALL ){ | > | > | | > < | | > > > > | | | < | 121810 121811 121812 121813 121814 121815 121816 121817 121818 121819 121820 121821 121822 121823 121824 121825 121826 121827 121828 121829 121830 121831 121832 121833 121834 121835 121836 121837 121838 121839 121840 121841 | rc = sqlite3_reset(pDelete); } } if( rc!=SQLITE_OK ){ return rc; } assert( iLevel>=0 || iLevel==FTS3_SEGCURSOR_ALL ); if( iLevel==FTS3_SEGCURSOR_ALL ){ rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_RANGE, &pDelete, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int(pDelete, 1, iIndex*FTS3_SEGDIR_MAXLEVEL); sqlite3_bind_int(pDelete, 2, (iIndex+1) * FTS3_SEGDIR_MAXLEVEL - 1); } }else{ rc = fts3SqlStmt(p, SQL_DELETE_SEGDIR_LEVEL, &pDelete, 0); if( rc==SQLITE_OK ){ sqlite3_bind_int(pDelete, 1, iIndex*FTS3_SEGDIR_MAXLEVEL + iLevel); } } if( rc==SQLITE_OK ){ sqlite3_step(pDelete); rc = sqlite3_reset(pDelete); } return rc; } /* ** When this function is called, buffer *ppList (size *pnList bytes) contains ** a position list that may (or may not) feature multiple columns. This |
︙ | ︙ | |||
120753 120754 120755 120756 120757 120758 120759 | p += sqlite3Fts3GetVarint32(p, &iCurrent); } *ppList = pList; *pnList = nList; } | > > > > > > > > > > > > > > > > > > > > > > > > | > > > > > > > > > > > > > > > > > | > | > > > > > > > > > > > > > > > > | > > > > > | > > | | > > > > > > > > > > > > > > > | > > > > > > > > > > > | < < | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | > > > | | 121875 121876 121877 121878 121879 121880 121881 121882 121883 121884 121885 121886 121887 121888 121889 121890 121891 121892 121893 121894 121895 121896 121897 121898 121899 121900 121901 121902 121903 121904 121905 121906 121907 121908 121909 121910 121911 121912 121913 121914 121915 121916 121917 121918 121919 121920 121921 121922 121923 121924 121925 121926 121927 121928 121929 121930 121931 121932 121933 121934 121935 121936 121937 121938 121939 121940 121941 121942 121943 121944 121945 121946 121947 121948 121949 121950 121951 121952 121953 121954 121955 121956 121957 121958 121959 121960 121961 121962 121963 121964 121965 121966 121967 121968 121969 121970 121971 121972 121973 121974 121975 121976 121977 121978 121979 121980 121981 121982 121983 121984 121985 121986 121987 121988 121989 121990 121991 121992 121993 121994 121995 121996 121997 121998 121999 122000 122001 122002 122003 122004 122005 122006 122007 122008 122009 122010 122011 122012 122013 122014 122015 122016 122017 122018 122019 122020 122021 122022 122023 122024 122025 122026 122027 122028 122029 122030 122031 122032 122033 122034 122035 122036 122037 122038 122039 122040 122041 122042 122043 122044 122045 122046 122047 122048 122049 122050 122051 122052 122053 122054 122055 122056 122057 122058 122059 122060 122061 122062 122063 122064 122065 122066 122067 122068 122069 122070 122071 122072 122073 122074 122075 122076 122077 122078 122079 122080 122081 122082 122083 122084 122085 122086 122087 122088 122089 122090 122091 122092 122093 122094 122095 122096 122097 122098 122099 122100 122101 122102 122103 122104 122105 122106 122107 122108 122109 122110 122111 122112 122113 122114 122115 122116 122117 122118 122119 122120 122121 | p += sqlite3Fts3GetVarint32(p, &iCurrent); } *ppList = pList; *pnList = nList; } /* ** Cache data in the Fts3MultiSegReader.aBuffer[] buffer (overwriting any ** existing data). Grow the buffer if required. ** ** If successful, return SQLITE_OK. Otherwise, if an OOM error is encountered ** trying to resize the buffer, return SQLITE_NOMEM. */ static int fts3MsrBufferData( Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ char *pList, int nList ){ if( nList>pMsr->nBuffer ){ char *pNew; pMsr->nBuffer = nList*2; pNew = (char *)sqlite3_realloc(pMsr->aBuffer, pMsr->nBuffer); if( !pNew ) return SQLITE_NOMEM; pMsr->aBuffer = pNew; } memcpy(pMsr->aBuffer, pList, nList); return SQLITE_OK; } SQLITE_PRIVATE int sqlite3Fts3MsrIncrNext( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pMsr, /* Multi-segment-reader handle */ sqlite3_int64 *piDocid, /* OUT: Docid value */ char **paPoslist, /* OUT: Pointer to position list */ int *pnPoslist /* OUT: Size of position list in bytes */ ){ int nMerge = pMsr->nAdvance; Fts3SegReader **apSegment = pMsr->apSegment; int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = ( p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp ); if( nMerge==0 ){ *paPoslist = 0; return SQLITE_OK; } while( 1 ){ Fts3SegReader *pSeg; pSeg = pMsr->apSegment[0]; if( pSeg->pOffsetList==0 ){ *paPoslist = 0; break; }else{ int rc; char *pList; int nList; int j; sqlite3_int64 iDocid = apSegment[0]->iDocid; rc = fts3SegReaderNextDocid(p, apSegment[0], &pList, &nList); j = 1; while( rc==SQLITE_OK && j<nMerge && apSegment[j]->pOffsetList && apSegment[j]->iDocid==iDocid ){ rc = fts3SegReaderNextDocid(p, apSegment[j], 0, 0); j++; } if( rc!=SQLITE_OK ) return rc; fts3SegReaderSort(pMsr->apSegment, nMerge, j, xCmp); if( pMsr->iColFilter>=0 ){ fts3ColumnFilter(pMsr->iColFilter, &pList, &nList); } if( nList>0 ){ if( fts3SegReaderIsPending(apSegment[0]) ){ rc = fts3MsrBufferData(pMsr, pList, nList+1); if( rc!=SQLITE_OK ) return rc; *paPoslist = pMsr->aBuffer; assert( (pMsr->aBuffer[nList] & 0xFE)==0x00 ); }else{ *paPoslist = pList; } *piDocid = iDocid; *pnPoslist = nList; break; } } } return SQLITE_OK; } static int fts3SegReaderStart( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pCsr, /* Cursor object */ const char *zTerm, /* Term searched for (or NULL) */ int nTerm /* Length of zTerm in bytes */ ){ int i; int nSeg = pCsr->nSegment; /* If the Fts3SegFilter defines a specific term (or term prefix) to search ** for, then advance each segment iterator until it points to a term of ** equal or greater value than the specified term. This prevents many ** unnecessary merge/sort operations for the case where single segment ** b-tree leaf nodes contain more than one term. */ for(i=0; pCsr->bRestart==0 && i<pCsr->nSegment; i++){ Fts3SegReader *pSeg = pCsr->apSegment[i]; do { int rc = fts3SegReaderNext(p, pSeg, 0); if( rc!=SQLITE_OK ) return rc; }while( zTerm && fts3SegReaderTermCmp(pSeg, zTerm, nTerm)<0 ); } fts3SegReaderSort(pCsr->apSegment, nSeg, nSeg, fts3SegReaderCmp); return SQLITE_OK; } SQLITE_PRIVATE int sqlite3Fts3SegReaderStart( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pCsr, /* Cursor object */ Fts3SegFilter *pFilter /* Restrictions on range of iteration */ ){ pCsr->pFilter = pFilter; return fts3SegReaderStart(p, pCsr, pFilter->zTerm, pFilter->nTerm); } SQLITE_PRIVATE int sqlite3Fts3MsrIncrStart( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pCsr, /* Cursor object */ int iCol, /* Column to match on. */ const char *zTerm, /* Term to iterate through a doclist for */ int nTerm /* Number of bytes in zTerm */ ){ int i; int rc; int nSegment = pCsr->nSegment; int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = ( p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp ); assert( pCsr->pFilter==0 ); assert( zTerm && nTerm>0 ); /* Advance each segment iterator until it points to the term zTerm/nTerm. */ rc = fts3SegReaderStart(p, pCsr, zTerm, nTerm); if( rc!=SQLITE_OK ) return rc; /* Determine how many of the segments actually point to zTerm/nTerm. */ for(i=0; i<nSegment; i++){ Fts3SegReader *pSeg = pCsr->apSegment[i]; if( !pSeg->aNode || fts3SegReaderTermCmp(pSeg, zTerm, nTerm) ){ break; } } pCsr->nAdvance = i; /* Advance each of the segments to point to the first docid. */ for(i=0; i<pCsr->nAdvance; i++){ rc = fts3SegReaderFirstDocid(p, pCsr->apSegment[i]); if( rc!=SQLITE_OK ) return rc; } fts3SegReaderSort(pCsr->apSegment, i, i, xCmp); assert( iCol<0 || iCol<p->nColumn ); pCsr->iColFilter = iCol; return SQLITE_OK; } /* ** This function is called on a MultiSegReader that has been started using ** sqlite3Fts3MsrIncrStart(). One or more calls to MsrIncrNext() may also ** have been made. Calling this function puts the MultiSegReader in such ** a state that if the next two calls are: ** ** sqlite3Fts3SegReaderStart() ** sqlite3Fts3SegReaderStep() ** ** then the entire doclist for the term is available in ** MultiSegReader.aDoclist/nDoclist. */ SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr){ int i; /* Used to iterate through segment-readers */ assert( pCsr->zTerm==0 ); assert( pCsr->nTerm==0 ); assert( pCsr->aDoclist==0 ); assert( pCsr->nDoclist==0 ); pCsr->nAdvance = 0; pCsr->bRestart = 1; for(i=0; i<pCsr->nSegment; i++){ pCsr->apSegment[i]->pOffsetList = 0; pCsr->apSegment[i]->nOffsetList = 0; pCsr->apSegment[i]->iDocid = 0; } return SQLITE_OK; } SQLITE_PRIVATE int sqlite3Fts3SegReaderStep( Fts3Table *p, /* Virtual table handle */ Fts3MultiSegReader *pCsr /* Cursor object */ ){ int rc = SQLITE_OK; int isIgnoreEmpty = (pCsr->pFilter->flags & FTS3_SEGMENT_IGNORE_EMPTY); int isRequirePos = (pCsr->pFilter->flags & FTS3_SEGMENT_REQUIRE_POS); int isColFilter = (pCsr->pFilter->flags & FTS3_SEGMENT_COLUMN_FILTER); int isPrefix = (pCsr->pFilter->flags & FTS3_SEGMENT_PREFIX); int isScan = (pCsr->pFilter->flags & FTS3_SEGMENT_SCAN); Fts3SegReader **apSegment = pCsr->apSegment; int nSegment = pCsr->nSegment; Fts3SegFilter *pFilter = pCsr->pFilter; int (*xCmp)(Fts3SegReader *, Fts3SegReader *) = ( p->bDescIdx ? fts3SegReaderDoclistCmpRev : fts3SegReaderDoclistCmp ); if( pCsr->nSegment==0 ) return SQLITE_OK; do { int nMerge; int i; /* Advance the first pCsr->nAdvance entries in the apSegment[] array ** forward. Then sort the list in order of current term again. */ for(i=0; i<pCsr->nAdvance; i++){ rc = fts3SegReaderNext(p, apSegment[i], 0); if( rc!=SQLITE_OK ) return rc; } fts3SegReaderSort(apSegment, nSegment, pCsr->nAdvance, fts3SegReaderCmp); pCsr->nAdvance = 0; /* If all the seg-readers are at EOF, we're finished. return SQLITE_OK. */ assert( rc==SQLITE_OK ); |
︙ | ︙ | |||
120849 120850 120851 120852 120853 120854 120855 | && apSegment[nMerge]->nTerm==pCsr->nTerm && 0==memcmp(pCsr->zTerm, apSegment[nMerge]->zTerm, pCsr->nTerm) ){ nMerge++; } assert( isIgnoreEmpty || (isRequirePos && !isColFilter) ); | | > > > > > > | > | > | | | | | > > > > > > > > > > > > | | < < | > | | 122146 122147 122148 122149 122150 122151 122152 122153 122154 122155 122156 122157 122158 122159 122160 122161 122162 122163 122164 122165 122166 122167 122168 122169 122170 122171 122172 122173 122174 122175 122176 122177 122178 122179 122180 122181 122182 122183 122184 122185 122186 122187 122188 122189 122190 122191 122192 122193 122194 122195 122196 122197 122198 122199 122200 122201 122202 122203 122204 122205 122206 122207 122208 122209 122210 122211 122212 122213 122214 122215 122216 122217 122218 122219 122220 122221 122222 122223 122224 122225 122226 122227 122228 122229 122230 122231 122232 122233 122234 122235 122236 122237 122238 122239 122240 122241 122242 122243 122244 122245 122246 122247 122248 122249 122250 122251 122252 | && apSegment[nMerge]->nTerm==pCsr->nTerm && 0==memcmp(pCsr->zTerm, apSegment[nMerge]->zTerm, pCsr->nTerm) ){ nMerge++; } assert( isIgnoreEmpty || (isRequirePos && !isColFilter) ); if( nMerge==1 && !isIgnoreEmpty && (p->bDescIdx==0 || fts3SegReaderIsPending(apSegment[0])==0) ){ pCsr->nDoclist = apSegment[0]->nDoclist; if( fts3SegReaderIsPending(apSegment[0]) ){ rc = fts3MsrBufferData(pCsr, apSegment[0]->aDoclist, pCsr->nDoclist); pCsr->aDoclist = pCsr->aBuffer; }else{ pCsr->aDoclist = apSegment[0]->aDoclist; } if( rc==SQLITE_OK ) rc = SQLITE_ROW; }else{ int nDoclist = 0; /* Size of doclist */ sqlite3_int64 iPrev = 0; /* Previous docid stored in doclist */ /* The current term of the first nMerge entries in the array ** of Fts3SegReader objects is the same. The doclists must be merged ** and a single term returned with the merged doclist. */ for(i=0; i<nMerge; i++){ fts3SegReaderFirstDocid(p, apSegment[i]); } fts3SegReaderSort(apSegment, nMerge, nMerge, xCmp); while( apSegment[0]->pOffsetList ){ int j; /* Number of segments that share a docid */ char *pList; int nList; int nByte; sqlite3_int64 iDocid = apSegment[0]->iDocid; fts3SegReaderNextDocid(p, apSegment[0], &pList, &nList); j = 1; while( j<nMerge && apSegment[j]->pOffsetList && apSegment[j]->iDocid==iDocid ){ fts3SegReaderNextDocid(p, apSegment[j], 0, 0); j++; } if( isColFilter ){ fts3ColumnFilter(pFilter->iCol, &pList, &nList); } if( !isIgnoreEmpty || nList>0 ){ /* Calculate the 'docid' delta value to write into the merged ** doclist. */ sqlite3_int64 iDelta; if( p->bDescIdx && nDoclist>0 ){ iDelta = iPrev - iDocid; }else{ iDelta = iDocid - iPrev; } assert( iDelta>0 || (nDoclist==0 && iDelta==iDocid) ); assert( nDoclist>0 || iDelta==iDocid ); nByte = sqlite3Fts3VarintLen(iDelta) + (isRequirePos?nList+1:0); if( nDoclist+nByte>pCsr->nBuffer ){ char *aNew; pCsr->nBuffer = (nDoclist+nByte)*2; aNew = sqlite3_realloc(pCsr->aBuffer, pCsr->nBuffer); if( !aNew ){ return SQLITE_NOMEM; } pCsr->aBuffer = aNew; } nDoclist += sqlite3Fts3PutVarint(&pCsr->aBuffer[nDoclist], iDelta); iPrev = iDocid; if( isRequirePos ){ memcpy(&pCsr->aBuffer[nDoclist], pList, nList); nDoclist += nList; pCsr->aBuffer[nDoclist++] = '\0'; } } fts3SegReaderSort(apSegment, nMerge, j, xCmp); } if( nDoclist>0 ){ pCsr->aDoclist = pCsr->aBuffer; pCsr->nDoclist = nDoclist; rc = SQLITE_ROW; } } pCsr->nAdvance = nMerge; }while( rc==SQLITE_OK ); return rc; } SQLITE_PRIVATE void sqlite3Fts3SegReaderFinish( Fts3MultiSegReader *pCsr /* Cursor object */ ){ if( pCsr ){ int i; for(i=0; i<pCsr->nSegment; i++){ sqlite3Fts3SegReaderFree(pCsr->apSegment[i]); } sqlite3_free(pCsr->apSegment); |
︙ | ︙ | |||
120949 120950 120951 120952 120953 120954 120955 | ** currently present in the database. ** ** If this function is called with iLevel<0, but there is only one ** segment in the database, SQLITE_DONE is returned immediately. ** Otherwise, if successful, SQLITE_OK is returned. If an error occurs, ** an SQLite error code is returned. */ | | | | > > > > > > > > | | | < | > > > > > | < | > | > | > | | > | > > > | > > > > | 122265 122266 122267 122268 122269 122270 122271 122272 122273 122274 122275 122276 122277 122278 122279 122280 122281 122282 122283 122284 122285 122286 122287 122288 122289 122290 122291 122292 122293 122294 122295 122296 122297 122298 122299 122300 122301 122302 122303 122304 122305 122306 122307 122308 122309 122310 122311 122312 122313 122314 122315 122316 122317 122318 122319 122320 122321 122322 122323 122324 122325 122326 122327 122328 122329 122330 122331 122332 122333 122334 122335 122336 122337 122338 122339 122340 122341 122342 122343 122344 122345 122346 122347 122348 122349 122350 122351 122352 122353 122354 122355 122356 122357 122358 122359 122360 122361 122362 122363 122364 | ** currently present in the database. ** ** If this function is called with iLevel<0, but there is only one ** segment in the database, SQLITE_DONE is returned immediately. ** Otherwise, if successful, SQLITE_OK is returned. If an error occurs, ** an SQLite error code is returned. */ static int fts3SegmentMerge(Fts3Table *p, int iIndex, int iLevel){ int rc; /* Return code */ int iIdx = 0; /* Index of new segment */ int iNewLevel = 0; /* Level/index to create new segment at */ SegmentWriter *pWriter = 0; /* Used to write the new, merged, segment */ Fts3SegFilter filter; /* Segment term filter condition */ Fts3MultiSegReader csr; /* Cursor to iterate through level(s) */ int bIgnoreEmpty = 0; /* True to ignore empty segments */ assert( iLevel==FTS3_SEGCURSOR_ALL || iLevel==FTS3_SEGCURSOR_PENDING || iLevel>=0 ); assert( iLevel<FTS3_SEGDIR_MAXLEVEL ); assert( iIndex>=0 && iIndex<p->nIndex ); rc = sqlite3Fts3SegReaderCursor(p, iIndex, iLevel, 0, 0, 1, 0, &csr); if( rc!=SQLITE_OK || csr.nSegment==0 ) goto finished; if( iLevel==FTS3_SEGCURSOR_ALL ){ /* This call is to merge all segments in the database to a single ** segment. The level of the new segment is equal to the the numerically ** greatest segment level currently present in the database for this ** index. The idx of the new segment is always 0. */ if( csr.nSegment==1 ){ rc = SQLITE_DONE; goto finished; } rc = fts3SegmentMaxLevel(p, iIndex, &iNewLevel); bIgnoreEmpty = 1; }else if( iLevel==FTS3_SEGCURSOR_PENDING ){ iNewLevel = iIndex * FTS3_SEGDIR_MAXLEVEL; rc = fts3AllocateSegdirIdx(p, iIndex, 0, &iIdx); }else{ /* This call is to merge all segments at level iLevel. find the next ** available segment index at level iLevel+1. The call to ** fts3AllocateSegdirIdx() will merge the segments at level iLevel+1 to ** a single iLevel+2 segment if necessary. */ rc = fts3AllocateSegdirIdx(p, iIndex, iLevel+1, &iIdx); iNewLevel = iIndex * FTS3_SEGDIR_MAXLEVEL + iLevel+1; } if( rc!=SQLITE_OK ) goto finished; assert( csr.nSegment>0 ); assert( iNewLevel>=(iIndex*FTS3_SEGDIR_MAXLEVEL) ); assert( iNewLevel<((iIndex+1)*FTS3_SEGDIR_MAXLEVEL) ); memset(&filter, 0, sizeof(Fts3SegFilter)); filter.flags = FTS3_SEGMENT_REQUIRE_POS; filter.flags |= (bIgnoreEmpty ? FTS3_SEGMENT_IGNORE_EMPTY : 0); rc = sqlite3Fts3SegReaderStart(p, &csr, &filter); while( SQLITE_OK==rc ){ rc = sqlite3Fts3SegReaderStep(p, &csr); if( rc!=SQLITE_ROW ) break; rc = fts3SegWriterAdd(p, &pWriter, 1, csr.zTerm, csr.nTerm, csr.aDoclist, csr.nDoclist); } if( rc!=SQLITE_OK ) goto finished; assert( pWriter ); if( iLevel!=FTS3_SEGCURSOR_PENDING ){ rc = fts3DeleteSegdir(p, iIndex, iLevel, csr.apSegment, csr.nSegment); if( rc!=SQLITE_OK ) goto finished; } rc = fts3SegWriterFlush(p, pWriter, iNewLevel, iIdx); finished: fts3SegWriterFree(pWriter); sqlite3Fts3SegReaderFinish(&csr); return rc; } /* ** Flush the contents of pendingTerms to level 0 segments. */ SQLITE_PRIVATE int sqlite3Fts3PendingTermsFlush(Fts3Table *p){ int rc = SQLITE_OK; int i; for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){ rc = fts3SegmentMerge(p, i, FTS3_SEGCURSOR_PENDING); if( rc==SQLITE_DONE ) rc = SQLITE_OK; } sqlite3Fts3PendingTermsClear(p); return rc; } /* ** Encode N integers as varints into a blob. */ static void fts3EncodeIntArray( int N, /* The number of integers to encode */ |
︙ | ︙ | |||
121162 121163 121164 121165 121166 121167 121168 121169 121170 121171 121172 121173 121174 121175 121176 121177 121178 121179 121180 121181 121182 121183 121184 121185 | return; } sqlite3_bind_blob(pStmt, 1, pBlob, nBlob, SQLITE_STATIC); sqlite3_step(pStmt); *pRC = sqlite3_reset(pStmt); sqlite3_free(a); } /* ** Handle a 'special' INSERT of the form: ** ** "INSERT INTO tbl(tbl) VALUES(<expr>)" ** ** Argument pVal contains the result of <expr>. Currently the only ** meaningful value to insert is the text 'optimize'. */ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){ int rc; /* Return Code */ const char *zVal = (const char *)sqlite3_value_text(pVal); int nVal = sqlite3_value_bytes(pVal); if( !zVal ){ return SQLITE_NOMEM; }else if( nVal==8 && 0==sqlite3_strnicmp(zVal, "optimize", 8) ){ | > > > > > > > > > > > > > > > > > | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < < | | 122500 122501 122502 122503 122504 122505 122506 122507 122508 122509 122510 122511 122512 122513 122514 122515 122516 122517 122518 122519 122520 122521 122522 122523 122524 122525 122526 122527 122528 122529 122530 122531 122532 122533 122534 122535 122536 122537 122538 122539 122540 122541 122542 122543 122544 122545 122546 122547 122548 122549 122550 122551 122552 122553 122554 122555 122556 122557 122558 122559 122560 122561 122562 122563 122564 122565 122566 122567 122568 122569 122570 122571 122572 122573 122574 122575 122576 122577 122578 122579 122580 122581 122582 122583 122584 122585 | return; } sqlite3_bind_blob(pStmt, 1, pBlob, nBlob, SQLITE_STATIC); sqlite3_step(pStmt); *pRC = sqlite3_reset(pStmt); sqlite3_free(a); } static int fts3DoOptimize(Fts3Table *p, int bReturnDone){ int i; int bSeenDone = 0; int rc = SQLITE_OK; for(i=0; rc==SQLITE_OK && i<p->nIndex; i++){ rc = fts3SegmentMerge(p, i, FTS3_SEGCURSOR_ALL); if( rc==SQLITE_DONE ){ bSeenDone = 1; rc = SQLITE_OK; } } sqlite3Fts3SegmentsClose(p); sqlite3Fts3PendingTermsClear(p); return (rc==SQLITE_OK && bReturnDone && bSeenDone) ? SQLITE_DONE : rc; } /* ** Handle a 'special' INSERT of the form: ** ** "INSERT INTO tbl(tbl) VALUES(<expr>)" ** ** Argument pVal contains the result of <expr>. Currently the only ** meaningful value to insert is the text 'optimize'. */ static int fts3SpecialInsert(Fts3Table *p, sqlite3_value *pVal){ int rc; /* Return Code */ const char *zVal = (const char *)sqlite3_value_text(pVal); int nVal = sqlite3_value_bytes(pVal); if( !zVal ){ return SQLITE_NOMEM; }else if( nVal==8 && 0==sqlite3_strnicmp(zVal, "optimize", 8) ){ rc = fts3DoOptimize(p, 0); #ifdef SQLITE_TEST }else if( nVal>9 && 0==sqlite3_strnicmp(zVal, "nodesize=", 9) ){ p->nNodeSize = atoi(&zVal[9]); rc = SQLITE_OK; }else if( nVal>11 && 0==sqlite3_strnicmp(zVal, "maxpending=", 9) ){ p->nMaxPendingData = atoi(&zVal[11]); rc = SQLITE_OK; #endif }else{ rc = SQLITE_ERROR; } return rc; } /* ** Delete all cached deferred doclists. Deferred doclists are cached ** (allocated) by the sqlite3Fts3CacheDeferredDoclists() function. */ SQLITE_PRIVATE void sqlite3Fts3FreeDeferredDoclists(Fts3Cursor *pCsr){ Fts3DeferredToken *pDef; for(pDef=pCsr->pDeferred; pDef; pDef=pDef->pNext){ fts3PendingListDelete(pDef->pList); pDef->pList = 0; } } /* ** Free all entries in the pCsr->pDeffered list. Entries are added to ** this list using sqlite3Fts3DeferToken(). */ SQLITE_PRIVATE void sqlite3Fts3FreeDeferredTokens(Fts3Cursor *pCsr){ Fts3DeferredToken *pDef; Fts3DeferredToken *pNext; for(pDef=pCsr->pDeferred; pDef; pDef=pNext){ pNext = pDef->pNext; fts3PendingListDelete(pDef->pList); sqlite3_free(pDef); } pCsr->pDeferred = 0; } /* ** Generate deferred-doclists for all tokens in the pCsr->pDeferred list |
︙ | ︙ | |||
121323 121324 121325 121326 121327 121328 121329 121330 121331 121332 121333 121334 121335 121336 | rc = fts3PendingListAppendVarint(&pDef->pList, 0); } } } return rc; } /* ** Add an entry for token pToken to the pCsr->pDeferred list. */ SQLITE_PRIVATE int sqlite3Fts3DeferToken( Fts3Cursor *pCsr, /* Fts3 table cursor */ Fts3PhraseToken *pToken, /* Token to defer */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > | 122635 122636 122637 122638 122639 122640 122641 122642 122643 122644 122645 122646 122647 122648 122649 122650 122651 122652 122653 122654 122655 122656 122657 122658 122659 122660 122661 122662 122663 122664 122665 122666 122667 122668 122669 122670 122671 122672 122673 122674 122675 | rc = fts3PendingListAppendVarint(&pDef->pList, 0); } } } return rc; } SQLITE_PRIVATE int sqlite3Fts3DeferredTokenList( Fts3DeferredToken *p, char **ppData, int *pnData ){ char *pRet; int nSkip; sqlite3_int64 dummy; *ppData = 0; *pnData = 0; if( p->pList==0 ){ return SQLITE_OK; } pRet = (char *)sqlite3_malloc(p->pList->nData); if( !pRet ) return SQLITE_NOMEM; nSkip = sqlite3Fts3GetVarint(p->pList->aData, &dummy); *pnData = p->pList->nData - nSkip; *ppData = pRet; memcpy(pRet, &p->pList->aData[nSkip], *pnData); return SQLITE_OK; } /* ** Add an entry for token pToken to the pCsr->pDeferred list. */ SQLITE_PRIVATE int sqlite3Fts3DeferToken( Fts3Cursor *pCsr, /* Fts3 table cursor */ Fts3PhraseToken *pToken, /* Token to defer */ |
︙ | ︙ | |||
121398 121399 121400 121401 121402 121403 121404 | sqlite3_value **apVal, /* Array of arguments */ sqlite_int64 *pRowid /* OUT: The affected (or effected) rowid */ ){ Fts3Table *p = (Fts3Table *)pVtab; int rc = SQLITE_OK; /* Return Code */ int isRemove = 0; /* True for an UPDATE or DELETE */ sqlite3_int64 iRemove = 0; /* Rowid removed by UPDATE or DELETE */ | | | > | > > > | 122737 122738 122739 122740 122741 122742 122743 122744 122745 122746 122747 122748 122749 122750 122751 122752 122753 122754 122755 122756 122757 122758 122759 122760 122761 122762 122763 122764 122765 122766 122767 122768 122769 122770 122771 122772 122773 122774 122775 | sqlite3_value **apVal, /* Array of arguments */ sqlite_int64 *pRowid /* OUT: The affected (or effected) rowid */ ){ Fts3Table *p = (Fts3Table *)pVtab; int rc = SQLITE_OK; /* Return Code */ int isRemove = 0; /* True for an UPDATE or DELETE */ sqlite3_int64 iRemove = 0; /* Rowid removed by UPDATE or DELETE */ u32 *aSzIns = 0; /* Sizes of inserted documents */ u32 *aSzDel; /* Sizes of deleted documents */ int nChng = 0; /* Net change in number of documents */ int bInsertDone = 0; assert( p->pSegments==0 ); /* Check for a "special" INSERT operation. One of the form: ** ** INSERT INTO xyz(xyz) VALUES('command'); */ if( nArg>1 && sqlite3_value_type(apVal[0])==SQLITE_NULL && sqlite3_value_type(apVal[p->nColumn+2])!=SQLITE_NULL ){ rc = fts3SpecialInsert(p, apVal[p->nColumn+2]); goto update_out; } /* Allocate space to hold the change in document sizes */ aSzIns = sqlite3_malloc( sizeof(aSzIns[0])*(p->nColumn+1)*2 ); if( aSzIns==0 ){ rc = SQLITE_NOMEM; goto update_out; } aSzDel = &aSzIns[p->nColumn+1]; memset(aSzIns, 0, sizeof(aSzIns[0])*(p->nColumn+1)*2); /* If this is an INSERT operation, or an UPDATE that modifies the rowid ** value, then this operation requires constraint handling. ** ** If the on-conflict mode is REPLACE, this means that the existing row |
︙ | ︙ | |||
121468 121469 121470 121471 121472 121473 121474 | }else{ rc = fts3InsertData(p, apVal, pRowid); bInsertDone = 1; } } } if( rc!=SQLITE_OK ){ | | < | 122811 122812 122813 122814 122815 122816 122817 122818 122819 122820 122821 122822 122823 122824 122825 | }else{ rc = fts3InsertData(p, apVal, pRowid); bInsertDone = 1; } } } if( rc!=SQLITE_OK ){ goto update_out; } /* If this is a DELETE or UPDATE operation, remove the old record. */ if( sqlite3_value_type(apVal[0])!=SQLITE_NULL ){ assert( sqlite3_value_type(apVal[0])==SQLITE_INTEGER ); rc = fts3DeleteByRowid(p, apVal[0], &nChng, aSzDel); isRemove = 1; |
︙ | ︙ | |||
121502 121503 121504 121505 121506 121507 121508 121509 121510 121511 121512 121513 121514 121515 121516 121517 121518 121519 121520 121521 121522 | nChng++; } if( p->bHasStat ){ fts3UpdateDocTotals(&rc, p, aSzIns, aSzDel, nChng); } sqlite3_free(aSzIns); sqlite3Fts3SegmentsClose(p); return rc; } /* ** Flush any data in the pending-terms hash table to disk. If successful, ** merge all segments in the database (including the new segment, if ** there was any data to flush) into a single segment. */ SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *p){ int rc; rc = sqlite3_exec(p->db, "SAVEPOINT fts3", 0, 0, 0); if( rc==SQLITE_OK ){ | > | | | | < < | 122844 122845 122846 122847 122848 122849 122850 122851 122852 122853 122854 122855 122856 122857 122858 122859 122860 122861 122862 122863 122864 122865 122866 122867 122868 122869 122870 122871 122872 122873 122874 122875 122876 | nChng++; } if( p->bHasStat ){ fts3UpdateDocTotals(&rc, p, aSzIns, aSzDel, nChng); } update_out: sqlite3_free(aSzIns); sqlite3Fts3SegmentsClose(p); return rc; } /* ** Flush any data in the pending-terms hash table to disk. If successful, ** merge all segments in the database (including the new segment, if ** there was any data to flush) into a single segment. */ SQLITE_PRIVATE int sqlite3Fts3Optimize(Fts3Table *p){ int rc; rc = sqlite3_exec(p->db, "SAVEPOINT fts3", 0, 0, 0); if( rc==SQLITE_OK ){ rc = fts3DoOptimize(p, 1); if( rc==SQLITE_OK || rc==SQLITE_DONE ){ int rc2 = sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0); if( rc2!=SQLITE_OK ) rc = rc2; }else{ sqlite3_exec(p->db, "ROLLBACK TO fts3", 0, 0, 0); sqlite3_exec(p->db, "RELEASE fts3", 0, 0, 0); } } sqlite3Fts3SegmentsClose(p); return rc; |
︙ | ︙ | |||
121710 121711 121712 121713 121714 121715 121716 | int (*x)(Fts3Expr*,int,void*), /* Callback function to invoke for phrases */ void *pCtx /* Second argument to pass to callback */ ){ int iPhrase = 0; /* Variable used as the phrase counter */ return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx); } | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < > | < < < < < < < < | 123051 123052 123053 123054 123055 123056 123057 123058 123059 123060 123061 123062 123063 123064 123065 123066 123067 123068 123069 123070 123071 123072 123073 123074 123075 123076 123077 123078 | int (*x)(Fts3Expr*,int,void*), /* Callback function to invoke for phrases */ void *pCtx /* Second argument to pass to callback */ ){ int iPhrase = 0; /* Variable used as the phrase counter */ return fts3ExprIterate2(pExpr, &iPhrase, x, pCtx); } /* ** This is an fts3ExprIterate() callback used while loading the doclists ** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also ** fts3ExprLoadDoclists(). */ static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){ int rc = SQLITE_OK; Fts3Phrase *pPhrase = pExpr->pPhrase; LoadDoclistCtx *p = (LoadDoclistCtx *)ctx; UNUSED_PARAMETER(iPhrase); p->nPhrase++; p->nToken += pPhrase->nToken; return rc; } /* ** Load the doclists for each phrase in the query associated with FTS3 cursor ** pCsr. |
︙ | ︙ | |||
121949 121950 121951 121952 121953 121954 121955 | static int fts3SnippetFindPositions(Fts3Expr *pExpr, int iPhrase, void *ctx){ SnippetIter *p = (SnippetIter *)ctx; SnippetPhrase *pPhrase = &p->aPhrase[iPhrase]; char *pCsr; pPhrase->nToken = pExpr->pPhrase->nToken; | | | 123238 123239 123240 123241 123242 123243 123244 123245 123246 123247 123248 123249 123250 123251 123252 | static int fts3SnippetFindPositions(Fts3Expr *pExpr, int iPhrase, void *ctx){ SnippetIter *p = (SnippetIter *)ctx; SnippetPhrase *pPhrase = &p->aPhrase[iPhrase]; char *pCsr; pPhrase->nToken = pExpr->pPhrase->nToken; pCsr = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol); if( pCsr ){ int iFirst = 0; pPhrase->pList = pCsr; fts3GetDeltaPosition(&pCsr, &iFirst); pPhrase->pHead = pCsr; pPhrase->pTail = pCsr; pPhrase->iHead = iFirst; |
︙ | ︙ | |||
122306 122307 122308 122309 122310 122311 122312 | if( !c ) nEntry++; } *ppCollist = pEnd; return nEntry; } | < < < < < < < < < < < < < < < < < < < < | 123595 123596 123597 123598 123599 123600 123601 123602 123603 123604 123605 123606 123607 123608 | if( !c ) nEntry++; } *ppCollist = pEnd; return nEntry; } /* ** fts3ExprIterate() callback used to collect the "global" matchinfo stats ** for a single query. ** ** fts3ExprIterate() callback to load the 'global' elements of a ** FTS3_MATCHINFO_HITS matchinfo array. The global stats are those elements ** of the matchinfo array that are constant for all rows returned by the |
︙ | ︙ | |||
122359 122360 122361 122362 122363 122364 122365 | */ static int fts3ExprGlobalHitsCb( Fts3Expr *pExpr, /* Phrase expression node */ int iPhrase, /* Phrase number (numbered from zero) */ void *pCtx /* Pointer to MatchInfo structure */ ){ MatchInfo *p = (MatchInfo *)pCtx; | | < < < | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | < < | < > > | | 123628 123629 123630 123631 123632 123633 123634 123635 123636 123637 123638 123639 123640 123641 123642 123643 123644 123645 123646 123647 123648 123649 123650 123651 123652 123653 123654 123655 123656 123657 123658 123659 123660 123661 123662 123663 123664 123665 123666 123667 | */ static int fts3ExprGlobalHitsCb( Fts3Expr *pExpr, /* Phrase expression node */ int iPhrase, /* Phrase number (numbered from zero) */ void *pCtx /* Pointer to MatchInfo structure */ ){ MatchInfo *p = (MatchInfo *)pCtx; return sqlite3Fts3EvalPhraseStats( p->pCursor, pExpr, &p->aMatchinfo[3*iPhrase*p->nCol] ); } /* ** fts3ExprIterate() callback used to collect the "local" part of the ** FTS3_MATCHINFO_HITS array. The local stats are those elements of the ** array that are different for each row returned by the query. */ static int fts3ExprLocalHitsCb( Fts3Expr *pExpr, /* Phrase expression node */ int iPhrase, /* Phrase number */ void *pCtx /* Pointer to MatchInfo structure */ ){ MatchInfo *p = (MatchInfo *)pCtx; int iStart = iPhrase * p->nCol * 3; int i; for(i=0; i<p->nCol; i++){ char *pCsr; pCsr = sqlite3Fts3EvalPhrasePoslist(p->pCursor, pExpr, i); if( pCsr ){ p->aMatchinfo[iStart+i*3] = fts3ColumnlistCount(&pCsr); }else{ p->aMatchinfo[iStart+i*3] = 0; } } return SQLITE_OK; } static int fts3MatchinfoCheck( |
︙ | ︙ | |||
122510 122511 122512 122513 122514 122515 122516 | ** iterating through a multi-column position-list corresponding to the ** hits for a single phrase on a single row in order to calculate the ** values for a matchinfo() FTS3_MATCHINFO_LCS request. */ typedef struct LcsIterator LcsIterator; struct LcsIterator { Fts3Expr *pExpr; /* Pointer to phrase expression */ | < | | 123739 123740 123741 123742 123743 123744 123745 123746 123747 123748 123749 123750 123751 123752 123753 123754 | ** iterating through a multi-column position-list corresponding to the ** hits for a single phrase on a single row in order to calculate the ** values for a matchinfo() FTS3_MATCHINFO_LCS request. */ typedef struct LcsIterator LcsIterator; struct LcsIterator { Fts3Expr *pExpr; /* Pointer to phrase expression */ int iPosOffset; /* Tokens count up to end of this phrase */ char *pRead; /* Cursor used to iterate through aDoclist */ int iPos; /* Current position */ }; /* ** If LcsIterator.iCol is set to the following value, the iterator has ** finished iterating through all offsets for all columns. */ |
︙ | ︙ | |||
122543 122544 122545 122546 122547 122548 122549 | */ static int fts3LcsIteratorAdvance(LcsIterator *pIter){ char *pRead = pIter->pRead; sqlite3_int64 iRead; int rc = 0; pRead += sqlite3Fts3GetVarint(pRead, &iRead); | | | < < < < < < < | 123771 123772 123773 123774 123775 123776 123777 123778 123779 123780 123781 123782 123783 123784 123785 123786 123787 123788 | */ static int fts3LcsIteratorAdvance(LcsIterator *pIter){ char *pRead = pIter->pRead; sqlite3_int64 iRead; int rc = 0; pRead += sqlite3Fts3GetVarint(pRead, &iRead); if( iRead==0 || iRead==1 ){ pRead = 0; rc = 1; }else{ pIter->iPos += (int)(iRead-2); } pIter->pRead = pRead; return rc; } |
︙ | ︙ | |||
122585 122586 122587 122588 122589 122590 122591 122592 122593 122594 122595 | /* Allocate and populate the array of LcsIterator objects. The array ** contains one element for each matchable phrase in the query. **/ aIter = sqlite3_malloc(sizeof(LcsIterator) * pCsr->nPhrase); if( !aIter ) return SQLITE_NOMEM; memset(aIter, 0, sizeof(LcsIterator) * pCsr->nPhrase); (void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter); for(i=0; i<pInfo->nPhrase; i++){ LcsIterator *pIter = &aIter[i]; nToken -= pIter->pExpr->pPhrase->nToken; pIter->iPosOffset = nToken; | > < < < < < < < < < < > > > > | | | | | < < < | | 123806 123807 123808 123809 123810 123811 123812 123813 123814 123815 123816 123817 123818 123819 123820 123821 123822 123823 123824 123825 123826 123827 123828 123829 123830 123831 123832 123833 123834 123835 123836 123837 123838 123839 123840 123841 123842 123843 123844 123845 123846 123847 | /* Allocate and populate the array of LcsIterator objects. The array ** contains one element for each matchable phrase in the query. **/ aIter = sqlite3_malloc(sizeof(LcsIterator) * pCsr->nPhrase); if( !aIter ) return SQLITE_NOMEM; memset(aIter, 0, sizeof(LcsIterator) * pCsr->nPhrase); (void)fts3ExprIterate(pCsr->pExpr, fts3MatchinfoLcsCb, (void*)aIter); for(i=0; i<pInfo->nPhrase; i++){ LcsIterator *pIter = &aIter[i]; nToken -= pIter->pExpr->pPhrase->nToken; pIter->iPosOffset = nToken; } for(iCol=0; iCol<pInfo->nCol; iCol++){ int nLcs = 0; /* LCS value for this column */ int nLive = 0; /* Number of iterators in aIter not at EOF */ for(i=0; i<pInfo->nPhrase; i++){ LcsIterator *pIt = &aIter[i]; pIt->pRead = sqlite3Fts3EvalPhrasePoslist(pCsr, pIt->pExpr, iCol); if( pIt->pRead ){ pIt->iPos = pIt->iPosOffset; fts3LcsIteratorAdvance(&aIter[i]); nLive++; } } while( nLive>0 ){ LcsIterator *pAdv = 0; /* The iterator to advance by one position */ int nThisLcs = 0; /* LCS for the current iterator positions */ for(i=0; i<pInfo->nPhrase; i++){ LcsIterator *pIter = &aIter[i]; if( pIter->pRead==0 ){ /* This iterator is already at EOF for this column. */ nThisLcs = 0; }else{ if( pAdv==0 || pIter->iPos<pAdv->iPos ){ pAdv = pIter; } if( nThisLcs==0 || pIter->iPos==pIter[-1].iPos ){ |
︙ | ︙ | |||
122686 122687 122688 122689 122690 122691 122692 | case FTS3_MATCHINFO_NCOL: if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol; break; case FTS3_MATCHINFO_NDOC: if( bGlobal ){ | | | 123899 123900 123901 123902 123903 123904 123905 123906 123907 123908 123909 123910 123911 123912 123913 | case FTS3_MATCHINFO_NCOL: if( bGlobal ) pInfo->aMatchinfo[0] = pInfo->nCol; break; case FTS3_MATCHINFO_NDOC: if( bGlobal ){ sqlite3_int64 nDoc = 0; rc = fts3MatchinfoSelectDoctotal(pTab, &pSelect, &nDoc, 0); pInfo->aMatchinfo[0] = (u32)nDoc; } break; case FTS3_MATCHINFO_AVGLENGTH: if( bGlobal ){ |
︙ | ︙ | |||
122960 122961 122962 122963 122964 122965 122966 | TermOffsetCtx *p = (TermOffsetCtx *)ctx; int nTerm; /* Number of tokens in phrase */ int iTerm; /* For looping through nTerm phrase terms */ char *pList; /* Pointer to position list for phrase */ int iPos = 0; /* First position in position-list */ UNUSED_PARAMETER(iPhrase); | | | 124173 124174 124175 124176 124177 124178 124179 124180 124181 124182 124183 124184 124185 124186 124187 | TermOffsetCtx *p = (TermOffsetCtx *)ctx; int nTerm; /* Number of tokens in phrase */ int iTerm; /* For looping through nTerm phrase terms */ char *pList; /* Pointer to position list for phrase */ int iPos = 0; /* First position in position-list */ UNUSED_PARAMETER(iPhrase); pList = sqlite3Fts3EvalPhrasePoslist(p->pCsr, pExpr, p->iCol); nTerm = pExpr->pPhrase->nToken; if( pList ){ fts3GetDeltaPosition(&pList, &iPos); assert( iPos>=0 ); } for(iTerm=0; iTerm<nTerm; iTerm++){ |
︙ | ︙ | |||
124581 124582 124583 124584 124585 124586 124587 | /* ** Return the N-dimensional volumn of the cell stored in *p. */ static float cellArea(Rtree *pRtree, RtreeCell *p){ float area = 1.0; int ii; for(ii=0; ii<(pRtree->nDim*2); ii+=2){ | | | | 125794 125795 125796 125797 125798 125799 125800 125801 125802 125803 125804 125805 125806 125807 125808 125809 125810 125811 125812 125813 125814 125815 125816 125817 125818 125819 125820 125821 | /* ** Return the N-dimensional volumn of the cell stored in *p. */ static float cellArea(Rtree *pRtree, RtreeCell *p){ float area = 1.0; int ii; for(ii=0; ii<(pRtree->nDim*2); ii+=2){ area = (float)(area * (DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii]))); } return area; } /* ** Return the margin length of cell p. The margin length is the sum ** of the objects size in each dimension. */ static float cellMargin(Rtree *pRtree, RtreeCell *p){ float margin = 0.0; int ii; for(ii=0; ii<(pRtree->nDim*2); ii+=2){ margin += (float)(DCOORD(p->aCoord[ii+1]) - DCOORD(p->aCoord[ii])); } return margin; } /* ** Store the union of cells p1 and p2 in p1. */ |
︙ | ︙ | |||
124679 124680 124681 124682 124683 124684 124685 | x1 = MAX(DCOORD(p->aCoord[jj]), DCOORD(aCell[ii].aCoord[jj])); x2 = MIN(DCOORD(p->aCoord[jj+1]), DCOORD(aCell[ii].aCoord[jj+1])); if( x2<x1 ){ o = 0.0; break; }else{ | | | | | | 125892 125893 125894 125895 125896 125897 125898 125899 125900 125901 125902 125903 125904 125905 125906 125907 125908 125909 125910 125911 125912 125913 125914 125915 125916 125917 125918 125919 125920 125921 125922 125923 125924 125925 125926 125927 125928 125929 125930 | x1 = MAX(DCOORD(p->aCoord[jj]), DCOORD(aCell[ii].aCoord[jj])); x2 = MIN(DCOORD(p->aCoord[jj+1]), DCOORD(aCell[ii].aCoord[jj+1])); if( x2<x1 ){ o = 0.0; break; }else{ o = o * (float)(x2-x1); } } overlap += o; } } return overlap; } #endif #if VARIANT_RSTARTREE_CHOOSESUBTREE static float cellOverlapEnlargement( Rtree *pRtree, RtreeCell *p, RtreeCell *pInsert, RtreeCell *aCell, int nCell, int iExclude ){ double before; double after; before = cellOverlap(pRtree, p, aCell, nCell, iExclude); cellUnion(pRtree, p, pInsert); after = cellOverlap(pRtree, p, aCell, nCell, iExclude); return (float)(after-before); } #endif /* ** This function implements the ChooseLeaf algorithm from Gutman[84]. ** ChooseSubTree in r*tree terminology. |
︙ | ︙ | |||
124725 124726 124727 124728 124729 124730 124731 | int rc; int ii; RtreeNode *pNode; rc = nodeAcquire(pRtree, 1, 0, &pNode); for(ii=0; rc==SQLITE_OK && ii<(pRtree->iDepth-iHeight); ii++){ int iCell; | | | | | | 125938 125939 125940 125941 125942 125943 125944 125945 125946 125947 125948 125949 125950 125951 125952 125953 125954 125955 125956 | int rc; int ii; RtreeNode *pNode; rc = nodeAcquire(pRtree, 1, 0, &pNode); for(ii=0; rc==SQLITE_OK && ii<(pRtree->iDepth-iHeight); ii++){ int iCell; sqlite3_int64 iBest = 0; float fMinGrowth = 0.0; float fMinArea = 0.0; float fMinOverlap = 0.0; int nCell = NCELL(pNode); RtreeCell cell; RtreeNode *pChild; RtreeCell *aCell = 0; |
︙ | ︙ | |||
125159 125160 125161 125162 125163 125164 125165 | RtreeCell *pBboxLeft, RtreeCell *pBboxRight ){ int **aaSorted; int *aSpare; int ii; | | | | | 126372 126373 126374 126375 126376 126377 126378 126379 126380 126381 126382 126383 126384 126385 126386 126387 126388 | RtreeCell *pBboxLeft, RtreeCell *pBboxRight ){ int **aaSorted; int *aSpare; int ii; int iBestDim = 0; int iBestSplit = 0; float fBestMargin = 0.0; int nByte = (pRtree->nDim+1)*(sizeof(int*)+nCell*sizeof(int)); aaSorted = (int **)sqlite3_malloc(nByte); if( !aaSorted ){ return SQLITE_NOMEM; } |
︙ | ︙ | |||
125183 125184 125185 125186 125187 125188 125189 | aaSorted[ii][jj] = jj; } SortByDimension(pRtree, aaSorted[ii], nCell, ii, aCell, aSpare); } for(ii=0; ii<pRtree->nDim; ii++){ float margin = 0.0; | | | | | 126396 126397 126398 126399 126400 126401 126402 126403 126404 126405 126406 126407 126408 126409 126410 126411 126412 | aaSorted[ii][jj] = jj; } SortByDimension(pRtree, aaSorted[ii], nCell, ii, aCell, aSpare); } for(ii=0; ii<pRtree->nDim; ii++){ float margin = 0.0; float fBestOverlap = 0.0; float fBestArea = 0.0; int iBestLeft = 0; int nLeft; for( nLeft=RTREE_MINCELLS(pRtree); nLeft<=(nCell-RTREE_MINCELLS(pRtree)); nLeft++ ){ |
︙ | ︙ | |||
125500 125501 125502 125503 125504 125505 125506 | } static int deleteCell(Rtree *, RtreeNode *, int, int); static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){ int rc; int rc2; | | | 126713 126714 126715 126716 126717 126718 126719 126720 126721 126722 126723 126724 126725 126726 126727 | } static int deleteCell(Rtree *, RtreeNode *, int, int); static int removeNode(Rtree *pRtree, RtreeNode *pNode, int iHeight){ int rc; int rc2; RtreeNode *pParent = 0; int iCell; assert( pNode->nRef==1 ); /* Remove the entry in the parent cell. */ rc = nodeParentIndex(pRtree, pNode, &iCell); if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
125648 125649 125650 125651 125652 125653 125654 | if( ii==(nCell-1) ){ memcpy(&aCell[ii], pCell, sizeof(RtreeCell)); }else{ nodeGetCell(pRtree, pNode, ii, &aCell[ii]); } aOrder[ii] = ii; for(iDim=0; iDim<pRtree->nDim; iDim++){ | | | | | | | 126861 126862 126863 126864 126865 126866 126867 126868 126869 126870 126871 126872 126873 126874 126875 126876 126877 126878 126879 126880 126881 126882 126883 126884 126885 126886 126887 | if( ii==(nCell-1) ){ memcpy(&aCell[ii], pCell, sizeof(RtreeCell)); }else{ nodeGetCell(pRtree, pNode, ii, &aCell[ii]); } aOrder[ii] = ii; for(iDim=0; iDim<pRtree->nDim; iDim++){ aCenterCoord[iDim] += (float)DCOORD(aCell[ii].aCoord[iDim*2]); aCenterCoord[iDim] += (float)DCOORD(aCell[ii].aCoord[iDim*2+1]); } } for(iDim=0; iDim<pRtree->nDim; iDim++){ aCenterCoord[iDim] = (float)(aCenterCoord[iDim]/((float)nCell*2.0)); } for(ii=0; ii<nCell; ii++){ aDistance[ii] = 0.0; for(iDim=0; iDim<pRtree->nDim; iDim++){ float coord = (float)(DCOORD(aCell[ii].aCoord[iDim*2+1]) - DCOORD(aCell[ii].aCoord[iDim*2])); aDistance[ii] += (coord-aCenterCoord[iDim])*(coord-aCenterCoord[iDim]); } } SortByDistance(aOrder, nCell, aDistance, aSpare); nodeZero(pRtree, pNode); |
︙ | ︙ | |||
125759 125760 125761 125762 125763 125764 125765 | RtreeNode *pInsert; RtreeCell cell; nodeGetCell(pRtree, pNode, ii, &cell); /* Find a node to store this cell in. pNode->iNode currently contains ** the height of the sub-tree headed by the cell. */ | | | | 126972 126973 126974 126975 126976 126977 126978 126979 126980 126981 126982 126983 126984 126985 126986 126987 126988 126989 | RtreeNode *pInsert; RtreeCell cell; nodeGetCell(pRtree, pNode, ii, &cell); /* Find a node to store this cell in. pNode->iNode currently contains ** the height of the sub-tree headed by the cell. */ rc = ChooseLeaf(pRtree, &cell, (int)pNode->iNode, &pInsert); if( rc==SQLITE_OK ){ int rc2; rc = rtreeInsertCell(pRtree, pInsert, &cell, (int)pNode->iNode); rc2 = nodeRelease(pRtree, pInsert); if( rc==SQLITE_OK ){ rc = rc2; } } } return rc; |
︙ | ︙ | |||
126151 126152 126153 126154 126155 126156 126157 | sqlite3 *db, /* Database handle */ Rtree *pRtree, /* Rtree handle */ int isCreate /* True for xCreate, false for xConnect */ ){ int rc; char *zSql; if( isCreate ){ | | | 127364 127365 127366 127367 127368 127369 127370 127371 127372 127373 127374 127375 127376 127377 127378 | sqlite3 *db, /* Database handle */ Rtree *pRtree, /* Rtree handle */ int isCreate /* True for xCreate, false for xConnect */ ){ int rc; char *zSql; if( isCreate ){ int iPageSize = 0; zSql = sqlite3_mprintf("PRAGMA %Q.page_size", pRtree->zDb); rc = getIntFromStmt(db, zSql, &iPageSize); if( rc==SQLITE_OK ){ pRtree->iNodeSize = iPageSize-64; if( (4+pRtree->nBytesPerCell*RTREE_MAXCELLS)<pRtree->iNodeSize ){ pRtree->iNodeSize = 4+pRtree->nBytesPerCell*RTREE_MAXCELLS; } |
︙ | ︙ | |||
126954 126955 126956 126957 126958 126959 126960 | ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** This file implements a tokenizer for fts3 based on the ICU library. | < < < | 128167 128168 128169 128170 128171 128172 128173 128174 128175 128176 128177 128178 128179 128180 128181 | ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** This file implements a tokenizer for fts3 based on the ICU library. */ #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) #ifdef SQLITE_ENABLE_ICU #include <unicode/ubrk.h> #include <unicode/utf16.h> |
︙ | ︙ |
Changes to src/sqlite3.h.
︙ | ︙ | |||
105 106 107 108 109 110 111 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.7.7" #define SQLITE_VERSION_NUMBER 3007007 | | | 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | ** ** See also: [sqlite3_libversion()], ** [sqlite3_libversion_number()], [sqlite3_sourceid()], ** [sqlite_version()] and [sqlite_source_id()]. */ #define SQLITE_VERSION "3.7.7" #define SQLITE_VERSION_NUMBER 3007007 #define SQLITE_SOURCE_ID "2011-06-23 17:29:33 b61a76a53af04f731fe7617f7b6b4fb2aef6587b" /* ** CAPI3REF: Run-Time Library Version Numbers ** KEYWORDS: sqlite3_version, sqlite3_sourceid ** ** These interfaces provide the same information as the [SQLITE_VERSION], ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros |
︙ | ︙ | |||
306 307 308 309 310 311 312 | ** ** ^The sqlite3_exec() interface runs zero or more UTF-8 encoded, ** semicolon-separate SQL statements passed into its 2nd argument, ** in the context of the [database connection] passed in as its 1st ** argument. ^If the callback function of the 3rd argument to ** sqlite3_exec() is not NULL, then it is invoked for each result row ** coming out of the evaluated SQL statements. ^The 4th argument to | | | 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 | ** ** ^The sqlite3_exec() interface runs zero or more UTF-8 encoded, ** semicolon-separate SQL statements passed into its 2nd argument, ** in the context of the [database connection] passed in as its 1st ** argument. ^If the callback function of the 3rd argument to ** sqlite3_exec() is not NULL, then it is invoked for each result row ** coming out of the evaluated SQL statements. ^The 4th argument to ** sqlite3_exec() is relayed through to the 1st argument of each ** callback invocation. ^If the callback pointer to sqlite3_exec() ** is NULL, then no callback is ever invoked and result rows are ** ignored. ** ** ^If an error occurs while evaluating the SQL statements passed into ** sqlite3_exec(), then execution of the current statement stops and ** subsequent statements are skipped. ^If the 5th parameter to sqlite3_exec() |
︙ | ︙ | |||
455 456 457 458 459 460 461 462 463 464 465 466 467 468 | #define SQLITE_IOERR_SHMLOCK (SQLITE_IOERR | (20<<8)) #define SQLITE_IOERR_SHMMAP (SQLITE_IOERR | (21<<8)) #define SQLITE_IOERR_SEEK (SQLITE_IOERR | (22<<8)) #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) #define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) /* ** CAPI3REF: Flags For File Open Operations ** ** These bit values are intended for use in the ** 3rd parameter to the [sqlite3_open_v2()] interface and ** in the 4th parameter to the [sqlite3_vfs.xOpen] method. | > > | 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 | #define SQLITE_IOERR_SHMLOCK (SQLITE_IOERR | (20<<8)) #define SQLITE_IOERR_SHMMAP (SQLITE_IOERR | (21<<8)) #define SQLITE_IOERR_SEEK (SQLITE_IOERR | (22<<8)) #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) #define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) #define SQLITE_READONLY_RECOVERY (SQLITE_READONLY | (1<<8)) #define SQLITE_READONLY_CANTLOCK (SQLITE_READONLY | (2<<8)) /* ** CAPI3REF: Flags For File Open Operations ** ** These bit values are intended for use in the ** 3rd parameter to the [sqlite3_open_v2()] interface and ** in the 4th parameter to the [sqlite3_vfs.xOpen] method. |
︙ | ︙ | |||
896 897 898 899 900 901 902 | ** of good-quality randomness into zOut. The return value is ** the actual number of bytes of randomness obtained. ** The xSleep() method causes the calling thread to sleep for at ** least the number of microseconds given. ^The xCurrentTime() ** method returns a Julian Day Number for the current date and time as ** a floating point value. ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian | | | 898 899 900 901 902 903 904 905 906 907 908 909 910 911 912 | ** of good-quality randomness into zOut. The return value is ** the actual number of bytes of randomness obtained. ** The xSleep() method causes the calling thread to sleep for at ** least the number of microseconds given. ^The xCurrentTime() ** method returns a Julian Day Number for the current date and time as ** a floating point value. ** ^The xCurrentTimeInt64() method returns, as an integer, the Julian ** Day Number multiplied by 86400000 (the number of milliseconds in ** a 24-hour day). ** ^SQLite will use the xCurrentTimeInt64() method to get the current ** date and time if that method is available (if iVersion is 2 or ** greater and the function pointer is not NULL) and will fall back ** to xCurrentTime() if xCurrentTimeInt64() is unavailable. ** ** ^The xSetSystemCall(), xGetSystemCall(), and xNestSystemCall() interfaces |
︙ | ︙ | |||
1334 1335 1336 1337 1338 1339 1340 | ** ^SQLite will never require a scratch buffer that is more than 6 ** times the database page size. ^If SQLite needs needs additional ** scratch memory beyond what is provided by this configuration option, then ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd> ** ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt> ** <dd> ^This option specifies a static memory buffer that SQLite can use for | | | 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 | ** ^SQLite will never require a scratch buffer that is more than 6 ** times the database page size. ^If SQLite needs needs additional ** scratch memory beyond what is provided by this configuration option, then ** [sqlite3_malloc()] will be used to obtain the memory needed.</dd> ** ** [[SQLITE_CONFIG_PAGECACHE]] <dt>SQLITE_CONFIG_PAGECACHE</dt> ** <dd> ^This option specifies a static memory buffer that SQLite can use for ** the database page cache with the default page cache implementation. ** This configuration should not be used if an application-define page ** cache implementation is loaded using the SQLITE_CONFIG_PCACHE option. ** There are three arguments to this option: A pointer to 8-byte aligned ** memory, the size of each page buffer (sz), and the number of pages (N). ** The sz argument should be the size of the largest database page ** (a power of two between 512 and 32768) plus a little extra for each ** page header. ^The page header size is 20 to 40 bytes depending on |
︙ | ︙ | |||
2432 2433 2434 2435 2436 2437 2438 | ** automatically deleted as soon as the database connection is closed. ** ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3> ** ** ^If [URI filename] interpretation is enabled, and the filename argument ** begins with "file:", then the filename is interpreted as a URI. ^URI ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is | | | | 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 | ** automatically deleted as soon as the database connection is closed. ** ** [[URI filenames in sqlite3_open()]] <h3>URI Filenames</h3> ** ** ^If [URI filename] interpretation is enabled, and the filename argument ** begins with "file:", then the filename is interpreted as a URI. ^URI ** filename interpretation is enabled if the [SQLITE_OPEN_URI] flag is ** set in the fourth argument to sqlite3_open_v2(), or if it has ** been enabled globally using the [SQLITE_CONFIG_URI] option with the ** [sqlite3_config()] method or by the [SQLITE_USE_URI] compile-time option. ** As of SQLite version 3.7.7, URI filename interpretation is turned off ** by default, but future releases of SQLite might enable URI filename ** interpretation by default. See "[URI filenames]" for additional ** information. ** ** URI filenames are parsed according to RFC 3986. ^If the URI contains an ** authority, then it must be either an empty string or the string ** "localhost". ^If the authority is not an empty string or "localhost", an ** error is returned to the caller. ^The fragment component of a URI, if ** present, is ignored. |
︙ | ︙ | |||
3256 3257 3258 3259 3260 3261 3262 | ** [SQLITE_DONE], [SQLITE_ROW], [SQLITE_ERROR], or [SQLITE_MISUSE]. ** ^With the "v2" interface, any of the other [result codes] or ** [extended result codes] might be returned as well. ** ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the ** database locks it needs to do its job. ^If the statement is a [COMMIT] ** or occurs outside of an explicit transaction, then you can retry the | | | 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 | ** [SQLITE_DONE], [SQLITE_ROW], [SQLITE_ERROR], or [SQLITE_MISUSE]. ** ^With the "v2" interface, any of the other [result codes] or ** [extended result codes] might be returned as well. ** ** ^[SQLITE_BUSY] means that the database engine was unable to acquire the ** database locks it needs to do its job. ^If the statement is a [COMMIT] ** or occurs outside of an explicit transaction, then you can retry the ** statement. If the statement is not a [COMMIT] and occurs within an ** explicit transaction then you should rollback the transaction before ** continuing. ** ** ^[SQLITE_DONE] means that the statement has finished executing ** successfully. sqlite3_step() should not be called again on this virtual ** machine without first calling [sqlite3_reset()] to reset the virtual ** machine back to its initial state. |
︙ | ︙ | |||
3535 3536 3537 3538 3539 3540 3541 | SQLITE_API int sqlite3_column_type(sqlite3_stmt*, int iCol); SQLITE_API sqlite3_value *sqlite3_column_value(sqlite3_stmt*, int iCol); /* ** CAPI3REF: Destroy A Prepared Statement Object ** ** ^The sqlite3_finalize() function is called to delete a [prepared statement]. | | | 3537 3538 3539 3540 3541 3542 3543 3544 3545 3546 3547 3548 3549 3550 3551 | SQLITE_API int sqlite3_column_type(sqlite3_stmt*, int iCol); SQLITE_API sqlite3_value *sqlite3_column_value(sqlite3_stmt*, int iCol); /* ** CAPI3REF: Destroy A Prepared Statement Object ** ** ^The sqlite3_finalize() function is called to delete a [prepared statement]. ** ^If the most recent evaluation of the statement encountered no errors ** or if the statement is never been evaluated, then sqlite3_finalize() returns ** SQLITE_OK. ^If the most recent evaluation of statement S failed, then ** sqlite3_finalize(S) returns the appropriate [error code] or ** [extended error code]. ** ** ^The sqlite3_finalize(S) routine can be called at any point during ** the life cycle of [prepared statement] S: |
︙ | ︙ | |||
5449 5450 5451 5452 5453 5454 5455 | ** ^The implementation is not required to provided versions of these ** routines that actually work. If the implementation does not provide working ** versions of these routines, it should at least provide stubs that always ** return true so that one does not get spurious assertion failures. ** ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then ** the routine should return 1. This seems counter-intuitive since | | | 5451 5452 5453 5454 5455 5456 5457 5458 5459 5460 5461 5462 5463 5464 5465 | ** ^The implementation is not required to provided versions of these ** routines that actually work. If the implementation does not provide working ** versions of these routines, it should at least provide stubs that always ** return true so that one does not get spurious assertion failures. ** ** ^If the argument to sqlite3_mutex_held() is a NULL pointer then ** the routine should return 1. This seems counter-intuitive since ** clearly the mutex cannot be held if it does not exist. But ** the reason the mutex does not exist is because the build is not ** using mutexes. And we do not want the assert() containing the ** call to sqlite3_mutex_held() to fail, so a non-zero return is ** the appropriate thing to do. ^The sqlite3_mutex_notheld() ** interface should also return 1 when given a NULL pointer. */ #ifndef NDEBUG |
︙ | ︙ | |||
5572 5573 5574 5575 5576 5577 5578 | #define SQLITE_TESTCTRL_ASSERT 12 #define SQLITE_TESTCTRL_ALWAYS 13 #define SQLITE_TESTCTRL_RESERVE 14 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 #define SQLITE_TESTCTRL_ISKEYWORD 16 #define SQLITE_TESTCTRL_PGHDRSZ 17 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18 | > | | 5574 5575 5576 5577 5578 5579 5580 5581 5582 5583 5584 5585 5586 5587 5588 5589 | #define SQLITE_TESTCTRL_ASSERT 12 #define SQLITE_TESTCTRL_ALWAYS 13 #define SQLITE_TESTCTRL_RESERVE 14 #define SQLITE_TESTCTRL_OPTIMIZATIONS 15 #define SQLITE_TESTCTRL_ISKEYWORD 16 #define SQLITE_TESTCTRL_PGHDRSZ 17 #define SQLITE_TESTCTRL_SCRATCHMALLOC 18 #define SQLITE_TESTCTRL_LOCALTIME_FAULT 19 #define SQLITE_TESTCTRL_LAST 19 /* ** CAPI3REF: SQLite Runtime Status ** ** ^This interface is used to retrieve runtime status information ** about the performance of SQLite, and optionally to reset various ** highwater marks. ^The first argument is an integer code for |
︙ | ︙ | |||
5958 5959 5960 5961 5962 5963 5964 | ** stored in the cache, both pinned and unpinned. ** ** [[the xFetch() page cache methods]] ** The xFetch() method locates a page in the cache and returns a pointer to ** the page, or a NULL pointer. ** A "page", in this context, means a buffer of szPage bytes aligned at an ** 8-byte boundary. The page to be fetched is determined by the key. ^The | | | 5961 5962 5963 5964 5965 5966 5967 5968 5969 5970 5971 5972 5973 5974 5975 | ** stored in the cache, both pinned and unpinned. ** ** [[the xFetch() page cache methods]] ** The xFetch() method locates a page in the cache and returns a pointer to ** the page, or a NULL pointer. ** A "page", in this context, means a buffer of szPage bytes aligned at an ** 8-byte boundary. The page to be fetched is determined by the key. ^The ** minimum key value is 1. After it has been retrieved using xFetch, the page ** is considered to be "pinned". ** ** If the requested page is already in the page cache, then the page cache ** implementation must return a pointer to the page buffer with its content ** intact. If the requested page is not already in the cache, then the ** cache implementation should use the value of the createFlag ** parameter to help it determined what action to take: |
︙ | ︙ |