sqllogictest
Check-in [d3e28e8083]
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update to the SQLite 3.7.5 beta.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: d3e28e8083920a6ef672a66a97c69b06ba0bd302
User & Date: drh 2011-01-25 20:41:49
Context
2011-04-11
20:51
Add Windows equiv or run-all.sh. check-in: 4a969aec8c user: shaneh tags: trunk
2011-01-25
20:41
Update to the SQLite 3.7.5 beta. check-in: d3e28e8083 user: drh tags: trunk
2010-12-07
03:38
More changes in support of evidence testing. check-in: fb75130b8f user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to src/sqlite3.c.

     1      1   /******************************************************************************
     2      2   ** This file is an amalgamation of many separate C source files from SQLite
     3         -** version 3.7.4.  By combining all the individual C code files into this 
            3  +** version 3.7.5.  By combining all the individual C code files into this 
     4      4   ** single large file, the entire code can be compiled as a one translation
     5      5   ** unit.  This allows many compilers to do optimizations that would not be
     6      6   ** possible if the files were compiled separately.  Performance improvements
     7      7   ** of 5% or more are commonly seen when SQLite is compiled as a single
     8      8   ** translation unit.
     9      9   **
    10     10   ** This file is all you need to compile SQLite.  To use SQLite in other
................................................................................
   646    646   ** string contains the date and time of the check-in (UTC) and an SHA1
   647    647   ** hash of the entire source tree.
   648    648   **
   649    649   ** See also: [sqlite3_libversion()],
   650    650   ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
   651    651   ** [sqlite_version()] and [sqlite_source_id()].
   652    652   */
   653         -#define SQLITE_VERSION        "3.7.4"
   654         -#define SQLITE_VERSION_NUMBER 3007004
   655         -#define SQLITE_SOURCE_ID      "2010-12-06 21:09:59 fabcb6b95e1d4059d1e6c6183f65846f6cbd5749"
          653  +#define SQLITE_VERSION        "3.7.5"
          654  +#define SQLITE_VERSION_NUMBER 3007005
          655  +#define SQLITE_SOURCE_ID      "2011-01-25 18:30:51 c17703ec1e604934f8bd5b1f66f34b19d17a6d1f"
   656    656   
   657    657   /*
   658    658   ** CAPI3REF: Run-Time Library Version Numbers
   659    659   ** KEYWORDS: sqlite3_version, sqlite3_sourceid
   660    660   **
   661    661   ** These interfaces provide the same information as the [SQLITE_VERSION],
   662    662   ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros
................................................................................
  1254   1254   ** The [SQLITE_FCNTL_CHUNK_SIZE] opcode is used to request that the VFS
  1255   1255   ** extends and truncates the database file in chunks of a size specified
  1256   1256   ** by the user. The fourth argument to [sqlite3_file_control()] should 
  1257   1257   ** point to an integer (type int) containing the new chunk-size to use
  1258   1258   ** for the nominated database. Allocating database file space in large
  1259   1259   ** chunks (say 1MB at a time), may reduce file-system fragmentation and
  1260   1260   ** improve performance on some systems.
         1261  +**
         1262  +** The [SQLITE_FCNTL_FILE_POINTER] opcode is used to obtain a pointer
         1263  +** to the [sqlite3_file] object associated with a particular database
         1264  +** connection.  See the [sqlite3_file_control()] documentation for
         1265  +** additional information.
         1266  +**
         1267  +** ^(The [SQLITE_FCNTL_SYNC_OMITTED] opcode is generated internally by
         1268  +** SQLite and sent to all VFSes in place of a call to the xSync method
         1269  +** when the database connection has [PRAGMA synchronous] set to OFF.)^
         1270  +** Some specialized VFSes need this signal in order to operate correctly
         1271  +** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most 
         1272  +** VFSes do not need this signal and should silently ignore this opcode.
         1273  +** Applications should not call [sqlite3_file_control()] with this
         1274  +** opcode as doing so may disrupt the operation of the specilized VFSes
         1275  +** that do require it.  
  1261   1276   */
  1262   1277   #define SQLITE_FCNTL_LOCKSTATE        1
  1263   1278   #define SQLITE_GET_LOCKPROXYFILE      2
  1264   1279   #define SQLITE_SET_LOCKPROXYFILE      3
  1265   1280   #define SQLITE_LAST_ERRNO             4
  1266   1281   #define SQLITE_FCNTL_SIZE_HINT        5
  1267   1282   #define SQLITE_FCNTL_CHUNK_SIZE       6
  1268   1283   #define SQLITE_FCNTL_FILE_POINTER     7
         1284  +#define SQLITE_FCNTL_SYNC_OMITTED     8
  1269   1285   
  1270   1286   
  1271   1287   /*
  1272   1288   ** CAPI3REF: Mutex Handle
  1273   1289   **
  1274   1290   ** The mutex module within SQLite defines [sqlite3_mutex] to be an
  1275   1291   ** abstract type for a mutex object.  The SQLite core never looks
................................................................................
  2381   2397   ** ^The sqlite3_mprintf() and sqlite3_vmprintf() routines write their
  2382   2398   ** results into memory obtained from [sqlite3_malloc()].
  2383   2399   ** The strings returned by these two routines should be
  2384   2400   ** released by [sqlite3_free()].  ^Both routines return a
  2385   2401   ** NULL pointer if [sqlite3_malloc()] is unable to allocate enough
  2386   2402   ** memory to hold the resulting string.
  2387   2403   **
  2388         -** ^(In sqlite3_snprintf() routine is similar to "snprintf()" from
         2404  +** ^(The sqlite3_snprintf() routine is similar to "snprintf()" from
  2389   2405   ** the standard C library.  The result is written into the
  2390   2406   ** buffer supplied as the second parameter whose size is given by
  2391   2407   ** the first parameter. Note that the order of the
  2392   2408   ** first two parameters is reversed from snprintf().)^  This is an
  2393   2409   ** historical accident that cannot be fixed without breaking
  2394   2410   ** backwards compatibility.  ^(Note also that sqlite3_snprintf()
  2395   2411   ** returns a pointer to its buffer instead of the number of
................................................................................
  2399   2415   ** now without breaking compatibility.
  2400   2416   **
  2401   2417   ** ^As long as the buffer size is greater than zero, sqlite3_snprintf()
  2402   2418   ** guarantees that the buffer is always zero-terminated.  ^The first
  2403   2419   ** parameter "n" is the total size of the buffer, including space for
  2404   2420   ** the zero terminator.  So the longest string that can be completely
  2405   2421   ** written will be n-1 characters.
         2422  +**
         2423  +** ^The sqlite3_vsnprintf() routine is a varargs version of sqlite3_snprintf().
  2406   2424   **
  2407   2425   ** These routines all implement some additional formatting
  2408   2426   ** options that are useful for constructing SQL statements.
  2409   2427   ** All of the usual printf() formatting options apply.  In addition, there
  2410   2428   ** is are "%q", "%Q", and "%z" options.
  2411   2429   **
  2412   2430   ** ^(The %q option works like %s in that it substitutes a null-terminated
................................................................................
  2463   2481   ** ^(The "%z" formatting option works like "%s" but with the
  2464   2482   ** addition that after the string has been read and copied into
  2465   2483   ** the result, [sqlite3_free()] is called on the input string.)^
  2466   2484   */
  2467   2485   SQLITE_API char *sqlite3_mprintf(const char*,...);
  2468   2486   SQLITE_API char *sqlite3_vmprintf(const char*, va_list);
  2469   2487   SQLITE_API char *sqlite3_snprintf(int,char*,const char*, ...);
         2488  +SQLITE_API char *sqlite3_vsnprintf(int,char*,const char*, va_list);
  2470   2489   
  2471   2490   /*
  2472   2491   ** CAPI3REF: Memory Allocation Subsystem
  2473   2492   **
  2474   2493   ** The SQLite core uses these three routines for all of its own
  2475   2494   ** internal memory allocation needs. "Core" in the previous sentence
  2476   2495   ** does not include operating-system specific VFS implementation.  The
................................................................................
  2840   2859   **
  2841   2860   ** ^(<dt>[SQLITE_OPEN_READWRITE]</dt>
  2842   2861   ** <dd>The database is opened for reading and writing if possible, or reading
  2843   2862   ** only if the file is write protected by the operating system.  In either
  2844   2863   ** case the database must already exist, otherwise an error is returned.</dd>)^
  2845   2864   **
  2846   2865   ** ^(<dt>[SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]</dt>
  2847         -** <dd>The database is opened for reading and writing, and is creates it if
         2866  +** <dd>The database is opened for reading and writing, and is created if
  2848   2867   ** it does not already exist. This is the behavior that is always used for
  2849   2868   ** sqlite3_open() and sqlite3_open16().</dd>)^
  2850   2869   ** </dl>
  2851   2870   **
  2852   2871   ** If the 3rd parameter to sqlite3_open_v2() is not one of the
  2853   2872   ** combinations shown above or one of the combinations shown above combined
  2854   2873   ** with the [SQLITE_OPEN_NOMUTEX], [SQLITE_OPEN_FULLMUTEX],
................................................................................
  3189   3208   ** compiled using either [sqlite3_prepare_v2()] or [sqlite3_prepare16_v2()].
  3190   3209   */
  3191   3210   SQLITE_API const char *sqlite3_sql(sqlite3_stmt *pStmt);
  3192   3211   
  3193   3212   /*
  3194   3213   ** CAPI3REF: Determine If An SQL Statement Writes The Database
  3195   3214   **
  3196         -** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if
  3197         -** the [prepared statement] X is [SELECT] statement and false (zero) if
  3198         -** X is an [INSERT], [UPDATE], [DELETE], CREATE, DROP, [ANALYZE],
  3199         -** [ALTER], or [REINDEX] statement.
  3200         -** If X is a NULL pointer or any other kind of statement, including but
  3201         -** not limited to [ATTACH], [DETACH], [COMMIT], [ROLLBACK], [RELEASE],
  3202         -** [SAVEPOINT], [PRAGMA], or [VACUUM] the result of sqlite3_stmt_readonly(X) is
  3203         -** undefined.
         3215  +** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if 
         3216  +** and only if the [prepared statement] X makes no direct changes to
         3217  +** the content of the database file.
         3218  +**
         3219  +** Note that [application-defined SQL functions] or
         3220  +** [virtual tables] might change the database indirectly as a side effect.  
         3221  +** ^(For example, if an application defines a function "eval()" that 
         3222  +** calls [sqlite3_exec()], then the following SQL statement would
         3223  +** change the database file through side-effects:
         3224  +**
         3225  +** <blockquote><pre>
         3226  +**    SELECT eval('DELETE FROM t1') FROM t2;
         3227  +** </pre></blockquote>
         3228  +**
         3229  +** But because the [SELECT] statement does not change the database file
         3230  +** directly, sqlite3_stmt_readonly() would still return true.)^
         3231  +**
         3232  +** ^Transaction control statements such as [BEGIN], [COMMIT], [ROLLBACK],
         3233  +** [SAVEPOINT], and [RELEASE] cause sqlite3_stmt_readonly() to return true,
         3234  +** since the statements themselves do not actually modify the database but
         3235  +** rather they control the timing of when other statements modify the 
         3236  +** database.  ^The [ATTACH] and [DETACH] statements also cause
         3237  +** sqlite3_stmt_readonly() to return true since, while those statements
         3238  +** change the configuration of a database connection, they do not make 
         3239  +** changes to the content of the database files on disk.
  3204   3240   */
  3205   3241   SQLITE_API int sqlite3_stmt_readonly(sqlite3_stmt *pStmt);
  3206   3242   
  3207   3243   /*
  3208   3244   ** CAPI3REF: Dynamically Typed Value Object
  3209   3245   ** KEYWORDS: {protected sqlite3_value} {unprotected sqlite3_value}
  3210   3246   **
................................................................................
  3590   3626   ** [SQLITE_MISUSE] means that the this routine was called inappropriately.
  3591   3627   ** Perhaps it was called on a [prepared statement] that has
  3592   3628   ** already been [sqlite3_finalize | finalized] or on one that had
  3593   3629   ** previously returned [SQLITE_ERROR] or [SQLITE_DONE].  Or it could
  3594   3630   ** be the case that the same database connection is being used by two or
  3595   3631   ** more threads at the same moment in time.
  3596   3632   **
  3597         -** For all versions of SQLite up to and including 3.6.23.1, it was required
  3598         -** after sqlite3_step() returned anything other than [SQLITE_ROW] that
  3599         -** [sqlite3_reset()] be called before any subsequent invocation of
  3600         -** sqlite3_step().  Failure to invoke [sqlite3_reset()] in this way would
  3601         -** result in an [SQLITE_MISUSE] return from sqlite3_step().  But after
  3602         -** version 3.6.23.1, sqlite3_step() began calling [sqlite3_reset()] 
  3603         -** automatically in this circumstance rather than returning [SQLITE_MISUSE].  
         3633  +** For all versions of SQLite up to and including 3.6.23.1, a call to
         3634  +** [sqlite3_reset()] was required after sqlite3_step() returned anything
         3635  +** other than [SQLITE_ROW] before any subsequent invocation of
         3636  +** sqlite3_step().  Failure to reset the prepared statement using 
         3637  +** [sqlite3_reset()] would result in an [SQLITE_MISUSE] return from
         3638  +** sqlite3_step().  But after version 3.6.23.1, sqlite3_step() began
         3639  +** calling [sqlite3_reset()] automatically in this circumstance rather
         3640  +** than returning [SQLITE_MISUSE].  This is not considered a compatibility
         3641  +** break because any application that ever receives an SQLITE_MISUSE error
         3642  +** is broken by definition.  The [SQLITE_OMIT_AUTORESET] compile-time option
         3643  +** can be used to restore the legacy behavior.
  3604   3644   **
  3605   3645   ** <b>Goofy Interface Alert:</b> In the legacy interface, the sqlite3_step()
  3606   3646   ** API always returns a generic error code, [SQLITE_ERROR], following any
  3607   3647   ** error other than [SQLITE_BUSY] and [SQLITE_MISUSE].  You must call
  3608   3648   ** [sqlite3_reset()] or [sqlite3_finalize()] in order to find one of the
  3609   3649   ** specific [error codes] that better describes the error.
  3610   3650   ** We admit that this is a goofy design.  The problem has been fixed
................................................................................
  3933   3973   ** will pick the one that involves the least amount of data conversion.
  3934   3974   ** If there is only a single implementation which does not care what text
  3935   3975   ** encoding is used, then the fourth argument should be [SQLITE_ANY].
  3936   3976   **
  3937   3977   ** ^(The fifth parameter is an arbitrary pointer.  The implementation of the
  3938   3978   ** function can gain access to this pointer using [sqlite3_user_data()].)^
  3939   3979   **
  3940         -** ^The seventh, eighth and ninth parameters, xFunc, xStep and xFinal, are
         3980  +** ^The sixth, seventh and eighth parameters, xFunc, xStep and xFinal, are
  3941   3981   ** pointers to C-language functions that implement the SQL function or
  3942   3982   ** aggregate. ^A scalar SQL function requires an implementation of the xFunc
  3943   3983   ** callback only; NULL pointers must be passed as the xStep and xFinal
  3944   3984   ** parameters. ^An aggregate SQL function requires an implementation of xStep
  3945   3985   ** and xFinal and NULL pointer must be passed for xFunc. ^To delete an existing
  3946   3986   ** SQL function or aggregate, pass NULL poiners for all three function
  3947   3987   ** callbacks.
  3948   3988   **
  3949         -** ^(If the tenth parameter to sqlite3_create_function_v2() is not NULL,
         3989  +** ^(If the ninth parameter to sqlite3_create_function_v2() is not NULL,
  3950   3990   ** then it is destructor for the application data pointer. 
  3951   3991   ** The destructor is invoked when the function is deleted, either by being
  3952   3992   ** overloaded or when the database connection closes.)^
  3953   3993   ** ^The destructor is also invoked if the call to
  3954   3994   ** sqlite3_create_function_v2() fails.
  3955   3995   ** ^When the destructor callback of the tenth parameter is invoked, it
  3956   3996   ** is passed a single argument which is a copy of the application data 
................................................................................
  4046   4086   ** The C-language implementation of SQL functions and aggregates uses
  4047   4087   ** this set of interface routines to access the parameter values on
  4048   4088   ** the function or aggregate.
  4049   4089   **
  4050   4090   ** The xFunc (for scalar functions) or xStep (for aggregates) parameters
  4051   4091   ** to [sqlite3_create_function()] and [sqlite3_create_function16()]
  4052   4092   ** define callbacks that implement the SQL functions and aggregates.
  4053         -** The 4th parameter to these callbacks is an array of pointers to
         4093  +** The 3rd parameter to these callbacks is an array of pointers to
  4054   4094   ** [protected sqlite3_value] objects.  There is one [sqlite3_value] object for
  4055   4095   ** each parameter to the SQL function.  These routines are used to
  4056   4096   ** extract values from the [sqlite3_value] objects.
  4057   4097   **
  4058   4098   ** These routines work only with [protected sqlite3_value] objects.
  4059   4099   ** Any attempt to use these routines on an [unprotected sqlite3_value]
  4060   4100   ** object results in undefined behavior.
................................................................................
  5774   5814   #define SQLITE_MUTEX_RECURSIVE        1
  5775   5815   #define SQLITE_MUTEX_STATIC_MASTER    2
  5776   5816   #define SQLITE_MUTEX_STATIC_MEM       3  /* sqlite3_malloc() */
  5777   5817   #define SQLITE_MUTEX_STATIC_MEM2      4  /* NOT USED */
  5778   5818   #define SQLITE_MUTEX_STATIC_OPEN      4  /* sqlite3BtreeOpen() */
  5779   5819   #define SQLITE_MUTEX_STATIC_PRNG      5  /* sqlite3_random() */
  5780   5820   #define SQLITE_MUTEX_STATIC_LRU       6  /* lru page list */
  5781         -#define SQLITE_MUTEX_STATIC_LRU2      7  /* lru page list */
         5821  +#define SQLITE_MUTEX_STATIC_LRU2      7  /* NOT USED */
         5822  +#define SQLITE_MUTEX_STATIC_PMEM      7  /* sqlite3PageMalloc() */
  5782   5823   
  5783   5824   /*
  5784   5825   ** CAPI3REF: Retrieve the mutex for a database connection
  5785   5826   **
  5786   5827   ** ^This interface returns a pointer the [sqlite3_mutex] object that 
  5787   5828   ** serializes access to the [database connection] given in the argument
  5788   5829   ** when the [threading mode] is Serialized.
................................................................................
  5925   5966   ** <dd>This parameter records the largest memory allocation request
  5926   5967   ** handed to [sqlite3_malloc()] or [sqlite3_realloc()] (or their
  5927   5968   ** internal equivalents).  Only the value returned in the
  5928   5969   ** *pHighwater parameter to [sqlite3_status()] is of interest.  
  5929   5970   ** The value written into the *pCurrent parameter is undefined.</dd>)^
  5930   5971   **
  5931   5972   ** ^(<dt>SQLITE_STATUS_MALLOC_COUNT</dt>
  5932         -** <dd>This parameter records the number of separate memory allocations.</dd>)^
         5973  +** <dd>This parameter records the number of separate memory allocations
         5974  +** currently checked out.</dd>)^
  5933   5975   **
  5934   5976   ** ^(<dt>SQLITE_STATUS_PAGECACHE_USED</dt>
  5935   5977   ** <dd>This parameter returns the number of pages used out of the
  5936   5978   ** [pagecache memory allocator] that was configured using 
  5937   5979   ** [SQLITE_CONFIG_PAGECACHE].  The
  5938   5980   ** value returned is in pages, not in bytes.</dd>)^
  5939   5981   **
................................................................................
  6030   6072   ** The [sqlite3_db_status()] interface will return a non-zero error code
  6031   6073   ** if a discontinued or unsupported verb is invoked.
  6032   6074   **
  6033   6075   ** <dl>
  6034   6076   ** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_USED</dt>
  6035   6077   ** <dd>This parameter returns the number of lookaside memory slots currently
  6036   6078   ** checked out.</dd>)^
         6079  +**
         6080  +** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_HIT</dt>
         6081  +** <dd>This parameter returns the number malloc attempts that were 
         6082  +** satisfied using lookaside memory. Only the high-water value is meaningful;
         6083  +** the current value is always zero.
         6084  +** checked out.</dd>)^
         6085  +**
         6086  +** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE</dt>
         6087  +** <dd>This parameter returns the number malloc attempts that might have
         6088  +** been satisfied using lookaside memory but failed due to the amount of
         6089  +** memory requested being larger than the lookaside slot size.
         6090  +** Only the high-water value is meaningful;
         6091  +** the current value is always zero.
         6092  +** checked out.</dd>)^
         6093  +**
         6094  +** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL</dt>
         6095  +** <dd>This parameter returns the number malloc attempts that might have
         6096  +** been satisfied using lookaside memory but failed due to all lookaside
         6097  +** memory already being in use.
         6098  +** Only the high-water value is meaningful;
         6099  +** the current value is always zero.
         6100  +** checked out.</dd>)^
  6037   6101   **
  6038   6102   ** ^(<dt>SQLITE_DBSTATUS_CACHE_USED</dt>
  6039   6103   ** <dd>This parameter returns the approximate number of of bytes of heap
  6040   6104   ** memory used by all pager caches associated with the database connection.)^
  6041   6105   ** ^The highwater mark associated with SQLITE_DBSTATUS_CACHE_USED is always 0.
  6042   6106   **
  6043   6107   ** ^(<dt>SQLITE_DBSTATUS_SCHEMA_USED</dt>
................................................................................
  6053   6117   ** <dd>This parameter returns the approximate number of of bytes of heap
  6054   6118   ** and lookaside memory used by all prepared statements associated with
  6055   6119   ** the database connection.)^
  6056   6120   ** ^The highwater mark associated with SQLITE_DBSTATUS_STMT_USED is always 0.
  6057   6121   ** </dd>
  6058   6122   ** </dl>
  6059   6123   */
  6060         -#define SQLITE_DBSTATUS_LOOKASIDE_USED     0
  6061         -#define SQLITE_DBSTATUS_CACHE_USED         1
  6062         -#define SQLITE_DBSTATUS_SCHEMA_USED        2
  6063         -#define SQLITE_DBSTATUS_STMT_USED          3
  6064         -#define SQLITE_DBSTATUS_MAX                3   /* Largest defined DBSTATUS */
         6124  +#define SQLITE_DBSTATUS_LOOKASIDE_USED       0
         6125  +#define SQLITE_DBSTATUS_CACHE_USED           1
         6126  +#define SQLITE_DBSTATUS_SCHEMA_USED          2
         6127  +#define SQLITE_DBSTATUS_STMT_USED            3
         6128  +#define SQLITE_DBSTATUS_LOOKASIDE_HIT        4
         6129  +#define SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE  5
         6130  +#define SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL  6
         6131  +#define SQLITE_DBSTATUS_MAX                  6   /* Largest defined DBSTATUS */
  6065   6132   
  6066   6133   
  6067   6134   /*
  6068   6135   ** CAPI3REF: Prepared Statement Status
  6069   6136   **
  6070   6137   ** ^(Each prepared statement maintains various
  6071   6138   ** [SQLITE_STMTSTATUS_SORT | counters] that measure the number
................................................................................
  6185   6252   **
  6186   6253   ** ^SQLite invokes the xCreate() method to construct a new cache instance.
  6187   6254   ** SQLite will typically create one cache instance for each open database file,
  6188   6255   ** though this is not guaranteed. ^The
  6189   6256   ** first parameter, szPage, is the size in bytes of the pages that must
  6190   6257   ** be allocated by the cache.  ^szPage will not be a power of two.  ^szPage
  6191   6258   ** will the page size of the database file that is to be cached plus an
  6192         -** increment (here called "R") of about 100 or 200.  SQLite will use the
         6259  +** increment (here called "R") of less than 250.  SQLite will use the
  6193   6260   ** extra R bytes on each page to store metadata about the underlying
  6194   6261   ** database page on disk.  The value of R depends
  6195   6262   ** on the SQLite version, the target platform, and how SQLite was compiled.
  6196         -** ^R is constant for a particular build of SQLite.  ^The second argument to
         6263  +** ^(R is constant for a particular build of SQLite. Except, there are two
         6264  +** distinct values of R when SQLite is compiled with the proprietary
         6265  +** ZIPVFS extension.)^  ^The second argument to
  6197   6266   ** xCreate(), bPurgeable, is true if the cache being created will
  6198   6267   ** be used to cache database pages of a file stored on disk, or
  6199   6268   ** false if it is used for an in-memory database. The cache implementation
  6200   6269   ** does not have to do anything special based with the value of bPurgeable;
  6201   6270   ** it is purely advisory.  ^On a cache where bPurgeable is false, SQLite will
  6202   6271   ** never invoke xUnpin() except to deliberately delete a page.
  6203   6272   ** ^In other words, calls to xUnpin() on a cache with bPurgeable set to
................................................................................
  6221   6290   ** 8-byte boundary. The page to be fetched is determined by the key. ^The
  6222   6291   ** mimimum key value is 1.  After it has been retrieved using xFetch, the page 
  6223   6292   ** is considered to be "pinned".
  6224   6293   **
  6225   6294   ** If the requested page is already in the page cache, then the page cache
  6226   6295   ** implementation must return a pointer to the page buffer with its content
  6227   6296   ** intact.  If the requested page is not already in the cache, then the
  6228         -** behavior of the cache implementation should use the value of the createFlag
         6297  +** cache implementation should use the value of the createFlag
  6229   6298   ** parameter to help it determined what action to take:
  6230   6299   **
  6231   6300   ** <table border=1 width=85% align=center>
  6232   6301   ** <tr><th> createFlag <th> Behaviour when page is not already in cache
  6233   6302   ** <tr><td> 0 <td> Do not allocate a new page.  Return NULL.
  6234   6303   ** <tr><td> 1 <td> Allocate a new page if it easy and convenient to do so.
  6235   6304   **                 Otherwise return NULL.
................................................................................
  6305   6374   **
  6306   6375   ** The backup API copies the content of one database into another.
  6307   6376   ** It is useful either for creating backups of databases or
  6308   6377   ** for copying in-memory databases to or from persistent files. 
  6309   6378   **
  6310   6379   ** See Also: [Using the SQLite Online Backup API]
  6311   6380   **
  6312         -** ^Exclusive access is required to the destination database for the 
  6313         -** duration of the operation. ^However the source database is only
  6314         -** read-locked while it is actually being read; it is not locked
  6315         -** continuously for the entire backup operation. ^Thus, the backup may be
  6316         -** performed on a live source database without preventing other users from
         6381  +** ^SQLite holds a write transaction open on the destination database file
         6382  +** for the duration of the backup operation.
         6383  +** ^The source database is read-locked only while it is being read;
         6384  +** it is not locked continuously for the entire backup operation.
         6385  +** ^Thus, the backup may be performed on a live source database without
         6386  +** preventing other database connections from
  6317   6387   ** reading or writing to the source database while the backup is underway.
  6318   6388   ** 
  6319   6389   ** ^(To perform a backup operation: 
  6320   6390   **   <ol>
  6321   6391   **     <li><b>sqlite3_backup_init()</b> is called once to initialize the
  6322   6392   **         backup, 
  6323   6393   **     <li><b>sqlite3_backup_step()</b> is called one or more times to transfer 
................................................................................
  6336   6406   ** ^The database name is "main" for the main database, "temp" for the
  6337   6407   ** temporary database, or the name specified after the AS keyword in
  6338   6408   ** an [ATTACH] statement for an attached database.
  6339   6409   ** ^The S and M arguments passed to 
  6340   6410   ** sqlite3_backup_init(D,N,S,M) identify the [database connection]
  6341   6411   ** and database name of the source database, respectively.
  6342   6412   ** ^The source and destination [database connections] (parameters S and D)
  6343         -** must be different or else sqlite3_backup_init(D,N,S,M) will file with
         6413  +** must be different or else sqlite3_backup_init(D,N,S,M) will fail with
  6344   6414   ** an error.
  6345   6415   **
  6346   6416   ** ^If an error occurs within sqlite3_backup_init(D,N,S,M), then NULL is
  6347         -** returned and an error code and error message are store3d in the
         6417  +** returned and an error code and error message are stored in the
  6348   6418   ** destination [database connection] D.
  6349   6419   ** ^The error code and message for the failed call to sqlite3_backup_init()
  6350   6420   ** can be retrieved using the [sqlite3_errcode()], [sqlite3_errmsg()], and/or
  6351   6421   ** [sqlite3_errmsg16()] functions.
  6352   6422   ** ^A successful call to sqlite3_backup_init() returns a pointer to an
  6353   6423   ** [sqlite3_backup] object.
  6354   6424   ** ^The [sqlite3_backup] object may be used with the sqlite3_backup_step() and
................................................................................
  6357   6427   **
  6358   6428   ** <b>sqlite3_backup_step()</b>
  6359   6429   **
  6360   6430   ** ^Function sqlite3_backup_step(B,N) will copy up to N pages between 
  6361   6431   ** the source and destination databases specified by [sqlite3_backup] object B.
  6362   6432   ** ^If N is negative, all remaining source pages are copied. 
  6363   6433   ** ^If sqlite3_backup_step(B,N) successfully copies N pages and there
  6364         -** are still more pages to be copied, then the function resturns [SQLITE_OK].
         6434  +** are still more pages to be copied, then the function returns [SQLITE_OK].
  6365   6435   ** ^If sqlite3_backup_step(B,N) successfully finishes copying all pages
  6366   6436   ** from source to destination, then it returns [SQLITE_DONE].
  6367   6437   ** ^If an error occurs while running sqlite3_backup_step(B,N),
  6368   6438   ** then an [error code] is returned. ^As well as [SQLITE_OK] and
  6369   6439   ** [SQLITE_DONE], a call to sqlite3_backup_step() may return [SQLITE_READONLY],
  6370   6440   ** [SQLITE_NOMEM], [SQLITE_BUSY], [SQLITE_LOCKED], or an
  6371   6441   ** [SQLITE_IOERR_ACCESS | SQLITE_IOERR_XXX] extended error code.
  6372   6442   **
  6373   6443   ** ^(The sqlite3_backup_step() might return [SQLITE_READONLY] if
  6374   6444   ** <ol>
  6375   6445   ** <li> the destination database was opened read-only, or
  6376   6446   ** <li> the destination database is using write-ahead-log journaling
  6377   6447   ** and the destination and source page sizes differ, or
  6378         -** <li> The destination database is an in-memory database and the
         6448  +** <li> the destination database is an in-memory database and the
  6379   6449   ** destination and source page sizes differ.
  6380   6450   ** </ol>)^
  6381   6451   **
  6382   6452   ** ^If sqlite3_backup_step() cannot obtain a required file-system lock, then
  6383   6453   ** the [sqlite3_busy_handler | busy-handler function]
  6384   6454   ** is invoked (if one is specified). ^If the 
  6385   6455   ** busy-handler returns non-zero before the lock is available, then 
................................................................................
  6702   6772   ** using [sqlite3_wal_hook()] disables the automatic checkpoint mechanism
  6703   6773   ** configured by this function.
  6704   6774   **
  6705   6775   ** ^The [wal_autocheckpoint pragma] can be used to invoke this interface
  6706   6776   ** from SQL.
  6707   6777   **
  6708   6778   ** ^Every new [database connection] defaults to having the auto-checkpoint
  6709         -** enabled with a threshold of 1000 pages.  The use of this interface
         6779  +** enabled with a threshold of 1000 or [SQLITE_DEFAULT_WAL_AUTOCHECKPOINT]
         6780  +** pages.  The use of this interface
  6710   6781   ** is only necessary if the default setting is found to be suboptimal
  6711   6782   ** for a particular application.
  6712   6783   */
  6713   6784   SQLITE_API int sqlite3_wal_autocheckpoint(sqlite3 *db, int N);
  6714   6785   
  6715   6786   /*
  6716   6787   ** CAPI3REF: Checkpoint a database
................................................................................
  8867   8938   */
  8868   8939   struct Lookaside {
  8869   8940     u16 sz;                 /* Size of each buffer in bytes */
  8870   8941     u8 bEnabled;            /* False to disable new lookaside allocations */
  8871   8942     u8 bMalloced;           /* True if pStart obtained from sqlite3_malloc() */
  8872   8943     int nOut;               /* Number of buffers currently checked out */
  8873   8944     int mxOut;              /* Highwater mark for nOut */
         8945  +  int anStat[3];          /* 0: hits.  1: size misses.  2: full misses */
  8874   8946     LookasideSlot *pFree;   /* List of available buffers */
  8875   8947     void *pStart;           /* First byte of available memory space */
  8876   8948     void *pEnd;             /* First byte past end of available space */
  8877   8949   };
  8878   8950   struct LookasideSlot {
  8879   8951     LookasideSlot *pNext;    /* Next buffer in the list of free buffers */
  8880   8952   };
................................................................................
  8945   9017       u8 orphanTrigger;           /* Last statement is orphaned TEMP trigger */
  8946   9018     } init;
  8947   9019     int nExtension;               /* Number of loaded extensions */
  8948   9020     void **aExtension;            /* Array of shared library handles */
  8949   9021     struct Vdbe *pVdbe;           /* List of active virtual machines */
  8950   9022     int activeVdbeCnt;            /* Number of VDBEs currently executing */
  8951   9023     int writeVdbeCnt;             /* Number of active VDBEs that are writing */
         9024  +  int vdbeExecCnt;              /* Number of nested calls to VdbeExec() */
  8952   9025     void (*xTrace)(void*,const char*);        /* Trace function */
  8953   9026     void *pTraceArg;                          /* Argument to the trace function */
  8954   9027     void (*xProfile)(void*,const char*,u64);  /* Profiling function */
  8955   9028     void *pProfileArg;                        /* Argument to profile function */
  8956   9029     void *pCommitArg;                 /* Argument to xCommitCallback() */   
  8957   9030     int (*xCommitCallback)(void*);    /* Invoked at every commit. */
  8958   9031     void *pRollbackArg;               /* Argument to xRollbackCallback() */   
................................................................................
 11716  11789     "OMIT_AUTOINCREMENT",
 11717  11790   #endif
 11718  11791   #ifdef SQLITE_OMIT_AUTOINIT
 11719  11792     "OMIT_AUTOINIT",
 11720  11793   #endif
 11721  11794   #ifdef SQLITE_OMIT_AUTOMATIC_INDEX
 11722  11795     "OMIT_AUTOMATIC_INDEX",
        11796  +#endif
        11797  +#ifdef SQLITE_OMIT_AUTORESET
        11798  +  "OMIT_AUTORESET",
 11723  11799   #endif
 11724  11800   #ifdef SQLITE_OMIT_AUTOVACUUM
 11725  11801     "OMIT_AUTOVACUUM",
 11726  11802   #endif
 11727  11803   #ifdef SQLITE_OMIT_BETWEEN_OPTIMIZATION
 11728  11804     "OMIT_BETWEEN_OPTIMIZATION",
 11729  11805   #endif
................................................................................
 11992  12068   ** The cursor can seek to a BTree entry with a particular key, or
 11993  12069   ** loop over all entries of the Btree.  You can also insert new BTree
 11994  12070   ** entries or retrieve the key or data from the entry that the cursor
 11995  12071   ** is currently pointing to.
 11996  12072   ** 
 11997  12073   ** Every cursor that the virtual machine has open is represented by an
 11998  12074   ** instance of the following structure.
 11999         -**
 12000         -** If the VdbeCursor.isTriggerRow flag is set it means that this cursor is
 12001         -** really a single row that represents the NEW or OLD pseudo-table of
 12002         -** a row trigger.  The data for the row is stored in VdbeCursor.pData and
 12003         -** the rowid is in VdbeCursor.iKey.
 12004  12075   */
 12005  12076   struct VdbeCursor {
 12006  12077     BtCursor *pCursor;    /* The cursor structure of the backend */
        12078  +  Btree *pBt;           /* Separate file holding temporary table */
        12079  +  KeyInfo *pKeyInfo;    /* Info about index keys needed by index cursors */
 12007  12080     int iDb;              /* Index of cursor database in db->aDb[] (or -1) */
 12008         -  i64 lastRowid;        /* Last rowid from a Next or NextIdx operation */
        12081  +  int pseudoTableReg;   /* Register holding pseudotable content. */
        12082  +  int nField;           /* Number of fields in the header */
 12009  12083     Bool zeroed;          /* True if zeroed out and ready for reuse */
 12010  12084     Bool rowidIsValid;    /* True if lastRowid is valid */
 12011  12085     Bool atFirst;         /* True if pointing to first entry */
 12012  12086     Bool useRandomRowid;  /* Generate new record numbers semi-randomly */
 12013  12087     Bool nullRow;         /* True if pointing to a row with no data */
 12014  12088     Bool deferredMoveto;  /* A call to sqlite3BtreeMoveto() is needed */
 12015  12089     Bool isTable;         /* True if a table requiring integer keys */
 12016  12090     Bool isIndex;         /* True if an index containing keys only - no data */
 12017  12091     Bool isOrdered;       /* True if the underlying table is BTREE_UNORDERED */
 12018         -  i64 movetoTarget;     /* Argument to the deferred sqlite3BtreeMoveto() */
 12019         -  Btree *pBt;           /* Separate file holding temporary table */
 12020         -  int pseudoTableReg;   /* Register holding pseudotable content. */
 12021         -  KeyInfo *pKeyInfo;    /* Info about index keys needed by index cursors */
 12022         -  int nField;           /* Number of fields in the header */
 12023         -  i64 seqCount;         /* Sequence counter */
 12024  12092     sqlite3_vtab_cursor *pVtabCursor;  /* The cursor for a virtual table */
 12025  12093     const sqlite3_module *pModule;     /* Module for cursor pVtabCursor */
        12094  +  i64 seqCount;         /* Sequence counter */
        12095  +  i64 movetoTarget;     /* Argument to the deferred sqlite3BtreeMoveto() */
        12096  +  i64 lastRowid;        /* Last rowid from a Next or NextIdx operation */
 12026  12097   
 12027  12098     /* Result of last sqlite3BtreeMoveto() done by an OP_NotExists or 
 12028  12099     ** OP_IsUnique opcode on this cursor. */
 12029  12100     int seekResult;
 12030  12101   
 12031  12102     /* Cached information about the header for the data record that the
 12032  12103     ** cursor is currently pointing to.  Only valid if cacheStatus matches
................................................................................
 12090  12161   ** A value for VdbeCursor.cacheValid that means the cache is always invalid.
 12091  12162   */
 12092  12163   #define CACHE_STALE 0
 12093  12164   
 12094  12165   /*
 12095  12166   ** Internally, the vdbe manipulates nearly all SQL values as Mem
 12096  12167   ** structures. Each Mem struct may cache multiple representations (string,
 12097         -** integer etc.) of the same value.  A value (and therefore Mem structure)
 12098         -** has the following properties:
 12099         -**
 12100         -** Each value has a manifest type. The manifest type of the value stored
 12101         -** in a Mem struct is returned by the MemType(Mem*) macro. The type is
 12102         -** one of SQLITE_NULL, SQLITE_INTEGER, SQLITE_REAL, SQLITE_TEXT or
 12103         -** SQLITE_BLOB.
        12168  +** integer etc.) of the same value.
 12104  12169   */
 12105  12170   struct Mem {
        12171  +  sqlite3 *db;        /* The associated database connection */
        12172  +  char *z;            /* String or BLOB value */
        12173  +  double r;           /* Real value */
 12106  12174     union {
 12107         -    i64 i;              /* Integer value. */
        12175  +    i64 i;              /* Integer value used when MEM_Int is set in flags */
 12108  12176       int nZero;          /* Used when bit MEM_Zero is set in flags */
 12109  12177       FuncDef *pDef;      /* Used only when flags==MEM_Agg */
 12110  12178       RowSet *pRowSet;    /* Used only when flags==MEM_RowSet */
 12111  12179       VdbeFrame *pFrame;  /* Used when flags==MEM_Frame */
 12112  12180     } u;
 12113         -  double r;           /* Real value */
 12114         -  sqlite3 *db;        /* The associated database connection */
 12115         -  char *z;            /* String or BLOB value */
 12116  12181     int n;              /* Number of characters in string value, excluding '\0' */
 12117  12182     u16 flags;          /* Some combination of MEM_Null, MEM_Str, MEM_Dyn, etc. */
 12118  12183     u8  type;           /* One of SQLITE_NULL, SQLITE_TEXT, SQLITE_INTEGER, etc */
 12119  12184     u8  enc;            /* SQLITE_UTF8, SQLITE_UTF16BE, SQLITE_UTF16LE */
 12120  12185   #ifdef SQLITE_DEBUG
 12121  12186     Mem *pScopyFrom;    /* This Mem is a shallow copy of pScopyFrom */
 12122  12187     void *pFiller;      /* So that sizeof(Mem) is a multiple of 8 */
................................................................................
 12132  12197   ** No other flags may be set in this case.
 12133  12198   **
 12134  12199   ** If the MEM_Str flag is set then Mem.z points at a string representation.
 12135  12200   ** Usually this is encoded in the same unicode encoding as the main
 12136  12201   ** database (see below for exceptions). If the MEM_Term flag is also
 12137  12202   ** set, then the string is nul terminated. The MEM_Int and MEM_Real 
 12138  12203   ** flags may coexist with the MEM_Str flag.
 12139         -**
 12140         -** Multiple of these values can appear in Mem.flags.  But only one
 12141         -** at a time can appear in Mem.type.
 12142  12204   */
 12143  12205   #define MEM_Null      0x0001   /* Value is NULL */
 12144  12206   #define MEM_Str       0x0002   /* Value is a string */
 12145  12207   #define MEM_Int       0x0004   /* Value is an integer */
 12146  12208   #define MEM_Real      0x0008   /* Value is a real number */
 12147  12209   #define MEM_Blob      0x0010   /* Value is a BLOB */
 12148  12210   #define MEM_RowSet    0x0020   /* Value is a RowSet object */
................................................................................
 12217  12279     VdbeFunc *pVdbeFunc;  /* Auxilary data, if created. */
 12218  12280     Mem s;                /* The return value is stored here */
 12219  12281     Mem *pMem;            /* Memory cell used to store aggregate context */
 12220  12282     int isError;          /* Error code returned by the function. */
 12221  12283     CollSeq *pColl;       /* Collating sequence */
 12222  12284   };
 12223  12285   
 12224         -/*
 12225         -** A Set structure is used for quick testing to see if a value
 12226         -** is part of a small set.  Sets are used to implement code like
 12227         -** this:
 12228         -**            x.y IN ('hi','hoo','hum')
 12229         -*/
 12230         -typedef struct Set Set;
 12231         -struct Set {
 12232         -  Hash hash;             /* A set is just a hash table */
 12233         -  HashElem *prev;        /* Previously accessed hash elemen */
 12234         -};
 12235         -
 12236  12286   /*
 12237  12287   ** An instance of the virtual machine.  This structure contains the complete
 12238  12288   ** state of the virtual machine.
 12239  12289   **
 12240         -** The "sqlite3_stmt" structure pointer that is returned by sqlite3_compile()
        12290  +** The "sqlite3_stmt" structure pointer that is returned by sqlite3_prepare()
 12241  12291   ** is really a pointer to an instance of this structure.
 12242  12292   **
 12243  12293   ** The Vdbe.inVtabMethod variable is set to non-zero for the duration of
 12244  12294   ** any virtual table method invocations made by the vdbe program. It is
 12245  12295   ** set to 2 for xDestroy method calls and 1 for all other methods. This
 12246  12296   ** variable is used for two purposes: to allow xDestroy methods to execute
 12247  12297   ** "DROP TABLE" statements and to prevent some nasty side effects of
 12248  12298   ** malloc failure when SQLite is invoked recursively by a virtual table 
 12249  12299   ** method function.
 12250  12300   */
 12251  12301   struct Vdbe {
 12252  12302     sqlite3 *db;            /* The database connection that owns this statement */
 12253         -  Vdbe *pPrev,*pNext;     /* Linked list of VDBEs with the same Vdbe.db */
 12254         -  int nOp;                /* Number of instructions in the program */
 12255         -  int nOpAlloc;           /* Number of slots allocated for aOp[] */
 12256  12303     Op *aOp;                /* Space to hold the virtual machine's program */
 12257         -  int nLabel;             /* Number of labels used */
 12258         -  int nLabelAlloc;        /* Number of slots allocated in aLabel[] */
 12259         -  int *aLabel;            /* Space to hold the labels */
        12304  +  Mem *aMem;              /* The memory locations */
 12260  12305     Mem **apArg;            /* Arguments to currently executing user function */
 12261  12306     Mem *aColName;          /* Column names to return */
 12262  12307     Mem *pResultSet;        /* Pointer to an array of results */
        12308  +  int nMem;               /* Number of memory locations currently allocated */
        12309  +  int nOp;                /* Number of instructions in the program */
        12310  +  int nOpAlloc;           /* Number of slots allocated for aOp[] */
        12311  +  int nLabel;             /* Number of labels used */
        12312  +  int nLabelAlloc;        /* Number of slots allocated in aLabel[] */
        12313  +  int *aLabel;            /* Space to hold the labels */
 12263  12314     u16 nResColumn;         /* Number of columns in one row of the result set */
 12264  12315     u16 nCursor;            /* Number of slots in apCsr[] */
        12316  +  u32 magic;              /* Magic number for sanity checking */
        12317  +  char *zErrMsg;          /* Error message written here */
        12318  +  Vdbe *pPrev,*pNext;     /* Linked list of VDBEs with the same Vdbe.db */
 12265  12319     VdbeCursor **apCsr;     /* One element of this array for each open cursor */
 12266         -  u8 errorAction;         /* Recovery action to do in case of an error */
 12267         -  u8 okVar;               /* True if azVar[] has been initialized */
 12268         -  ynVar nVar;             /* Number of entries in aVar[] */
 12269  12320     Mem *aVar;              /* Values for the OP_Variable opcode. */
 12270  12321     char **azVar;           /* Name of variables */
 12271         -  u32 magic;              /* Magic number for sanity checking */
 12272         -  int nMem;               /* Number of memory locations currently allocated */
 12273         -  Mem *aMem;              /* The memory locations */
        12322  +  ynVar nVar;             /* Number of entries in aVar[] */
 12274  12323     u32 cacheCtr;           /* VdbeCursor row cache generation counter */
 12275  12324     int pc;                 /* The program counter */
 12276  12325     int rc;                 /* Value to return */
 12277         -  char *zErrMsg;          /* Error message written here */
        12326  +  u8 errorAction;         /* Recovery action to do in case of an error */
        12327  +  u8 okVar;               /* True if azVar[] has been initialized */
 12278  12328     u8 explain;             /* True if EXPLAIN present on SQL command */
 12279  12329     u8 changeCntOn;         /* True to update the change-counter */
 12280  12330     u8 expired;             /* True if the VM needs to be recompiled */
 12281  12331     u8 runOnlyOnce;         /* Automatically expire on reset */
 12282  12332     u8 minWriteFileFormat;  /* Minimum file format for writable database files */
 12283  12333     u8 inVtabMethod;        /* See comments above */
 12284  12334     u8 usesStmtJournal;     /* True if uses a statement journal */
 12285  12335     u8 readOnly;            /* True for read-only statements */
 12286  12336     u8 isPrepareV2;         /* True if prepared with prepare_v2() */
 12287  12337     int nChange;            /* Number of db changes made since last reset */
 12288  12338     int btreeMask;          /* Bitmask of db->aDb[] entries referenced */
 12289         -  i64 startTime;          /* Time when query started - used for profiling */
 12290         -  BtreeMutexArray aMutex; /* An array of Btree used here and needing locks */
        12339  +  int iStatement;         /* Statement number (or 0 if has not opened stmt) */
 12291  12340     int aCounter[3];        /* Counters used by sqlite3_stmt_status() */
 12292         -  char *zSql;             /* Text of the SQL statement that generated this */
 12293         -  void *pFree;            /* Free this when deleting the vdbe */
        12341  +  BtreeMutexArray aMutex; /* An array of Btree used here and needing locks */
        12342  +#ifndef SQLITE_OMIT_TRACE
        12343  +  i64 startTime;          /* Time when query started - used for profiling */
        12344  +#endif
 12294  12345     i64 nFkConstraint;      /* Number of imm. FK constraints this VM */
 12295  12346     i64 nStmtDefCons;       /* Number of def. constraints when stmt started */
 12296         -  int iStatement;         /* Statement number (or 0 if has not opened stmt) */
        12347  +  char *zSql;             /* Text of the SQL statement that generated this */
        12348  +  void *pFree;            /* Free this when deleting the vdbe */
 12297  12349   #ifdef SQLITE_DEBUG
 12298  12350     FILE *trace;            /* Write an execution trace here, if not NULL */
 12299  12351   #endif
 12300  12352     VdbeFrame *pFrame;      /* Parent frame */
 12301  12353     VdbeFrame *pDelFrame;   /* List of frame objects to free on VM reset */
 12302  12354     int nFrame;             /* Number of frames in pFrame list */
 12303  12355     u32 expmask;            /* Binding to these vars invalidates VM */
................................................................................
 12498  12550         *pCurrent = db->lookaside.nOut;
 12499  12551         *pHighwater = db->lookaside.mxOut;
 12500  12552         if( resetFlag ){
 12501  12553           db->lookaside.mxOut = db->lookaside.nOut;
 12502  12554         }
 12503  12555         break;
 12504  12556       }
        12557  +
        12558  +    case SQLITE_DBSTATUS_LOOKASIDE_HIT:
        12559  +    case SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE:
        12560  +    case SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL: {
        12561  +      testcase( op==SQLITE_DBSTATUS_LOOKASIDE_HIT );
        12562  +      testcase( op==SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE );
        12563  +      testcase( op==SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL );
        12564  +      assert( (op-SQLITE_DBSTATUS_LOOKASIDE_HIT)>=0 );
        12565  +      assert( (op-SQLITE_DBSTATUS_LOOKASIDE_HIT)<3 );
        12566  +      *pCurrent = 0;
        12567  +      *pHighwater = db->lookaside.anStat[op - SQLITE_DBSTATUS_LOOKASIDE_HIT];
        12568  +      if( resetFlag ){
        12569  +        db->lookaside.anStat[op - SQLITE_DBSTATUS_LOOKASIDE_HIT] = 0;
        12570  +      }
        12571  +      break;
        12572  +    }
 12505  12573   
 12506  12574       /* 
 12507  12575       ** Return an approximation for the amount of memory currently used
 12508  12576       ** by all pagers associated with the given database connection.  The
 12509  12577       ** highwater mark is meaningless and is returned as zero.
 12510  12578       */
 12511  12579       case SQLITE_DBSTATUS_CACHE_USED: {
................................................................................
 18060  18128     assert( db==0 || db->pnBytesFreed==0 );
 18061  18129   #ifndef SQLITE_OMIT_LOOKASIDE
 18062  18130     if( db ){
 18063  18131       LookasideSlot *pBuf;
 18064  18132       if( db->mallocFailed ){
 18065  18133         return 0;
 18066  18134       }
 18067         -    if( db->lookaside.bEnabled && n<=db->lookaside.sz
 18068         -         && (pBuf = db->lookaside.pFree)!=0 ){
 18069         -      db->lookaside.pFree = pBuf->pNext;
 18070         -      db->lookaside.nOut++;
 18071         -      if( db->lookaside.nOut>db->lookaside.mxOut ){
 18072         -        db->lookaside.mxOut = db->lookaside.nOut;
        18135  +    if( db->lookaside.bEnabled ){
        18136  +      if( n>db->lookaside.sz ){
        18137  +        db->lookaside.anStat[1]++;
        18138  +      }else if( (pBuf = db->lookaside.pFree)==0 ){
        18139  +        db->lookaside.anStat[2]++;
        18140  +      }else{
        18141  +        db->lookaside.pFree = pBuf->pNext;
        18142  +        db->lookaside.nOut++;
        18143  +        db->lookaside.anStat[0]++;
        18144  +        if( db->lookaside.nOut>db->lookaside.mxOut ){
        18145  +          db->lookaside.mxOut = db->lookaside.nOut;
        18146  +        }
        18147  +        return (void*)pBuf;
 18073  18148         }
 18074         -      return (void*)pBuf;
 18075  18149       }
 18076  18150     }
 18077  18151   #else
 18078  18152     if( db && db->mallocFailed ){
 18079  18153       return 0;
 18080  18154     }
 18081  18155   #endif
................................................................................
 18979  19053       if( !p->useMalloc ){
 18980  19054         p->tooBig = 1;
 18981  19055         N = p->nAlloc - p->nChar - 1;
 18982  19056         if( N<=0 ){
 18983  19057           return;
 18984  19058         }
 18985  19059       }else{
        19060  +      char *zOld = (p->zText==p->zBase ? 0 : p->zText);
 18986  19061         i64 szNew = p->nChar;
 18987  19062         szNew += N + 1;
 18988  19063         if( szNew > p->mxAlloc ){
 18989  19064           sqlite3StrAccumReset(p);
 18990  19065           p->tooBig = 1;
 18991  19066           return;
 18992  19067         }else{
 18993  19068           p->nAlloc = (int)szNew;
 18994  19069         }
 18995  19070         if( p->useMalloc==1 ){
 18996         -        zNew = sqlite3DbMallocRaw(p->db, p->nAlloc );
        19071  +        zNew = sqlite3DbRealloc(p->db, zOld, p->nAlloc);
 18997  19072         }else{
 18998         -        zNew = sqlite3_malloc(p->nAlloc);
        19073  +        zNew = sqlite3_realloc(zOld, p->nAlloc);
 18999  19074         }
 19000  19075         if( zNew ){
 19001         -        memcpy(zNew, p->zText, p->nChar);
 19002         -        sqlite3StrAccumReset(p);
        19076  +        if( zOld==0 ) memcpy(zNew, p->zText, p->nChar);
 19003  19077           p->zText = zNew;
 19004  19078         }else{
 19005  19079           p->mallocFailed = 1;
 19006  19080           sqlite3StrAccumReset(p);
 19007  19081           return;
 19008  19082         }
 19009  19083       }
................................................................................
 19150  19224   }
 19151  19225   
 19152  19226   /*
 19153  19227   ** sqlite3_snprintf() works like snprintf() except that it ignores the
 19154  19228   ** current locale settings.  This is important for SQLite because we
 19155  19229   ** are not able to use a "," as the decimal point in place of "." as
 19156  19230   ** specified by some locales.
        19231  +**
        19232  +** Oops:  The first two arguments of sqlite3_snprintf() are backwards
        19233  +** from the snprintf() standard.  Unfortunately, it is too late to change
        19234  +** this without breaking compatibility, so we just have to live with the
        19235  +** mistake.
        19236  +**
        19237  +** sqlite3_vsnprintf() is the varargs version.
 19157  19238   */
        19239  +SQLITE_API char *sqlite3_vsnprintf(int n, char *zBuf, const char *zFormat, va_list ap){
        19240  +  StrAccum acc;
        19241  +  if( n<=0 ) return zBuf;
        19242  +  sqlite3StrAccumInit(&acc, zBuf, n, 0);
        19243  +  acc.useMalloc = 0;
        19244  +  sqlite3VXPrintf(&acc, 0, zFormat, ap);
        19245  +  return sqlite3StrAccumFinish(&acc);
        19246  +}
 19158  19247   SQLITE_API char *sqlite3_snprintf(int n, char *zBuf, const char *zFormat, ...){
 19159  19248     char *z;
 19160  19249     va_list ap;
 19161         -  StrAccum acc;
 19162         -
 19163         -  if( n<=0 ){
 19164         -    return zBuf;
 19165         -  }
 19166         -  sqlite3StrAccumInit(&acc, zBuf, n, 0);
 19167         -  acc.useMalloc = 0;
 19168  19250     va_start(ap,zFormat);
 19169         -  sqlite3VXPrintf(&acc, 0, zFormat, ap);
        19251  +  z = sqlite3_vsnprintf(n, zBuf, zFormat, ap);
 19170  19252     va_end(ap);
 19171         -  z = sqlite3StrAccumFinish(&acc);
 19172  19253     return z;
 19173  19254   }
 19174  19255   
 19175  19256   /*
 19176  19257   ** This is the routine that actually formats the sqlite3_log() message.
 19177  19258   ** We house it in a separate routine from sqlite3_log() to avoid using
 19178  19259   ** stack space on small-stack systems when logging is disabled.
................................................................................
 22893  22974   */
 22894  22975   #include <sys/types.h>
 22895  22976   #include <sys/stat.h>
 22896  22977   #include <fcntl.h>
 22897  22978   #include <unistd.h>
 22898  22979   #include <sys/time.h>
 22899  22980   #include <errno.h>
        22981  +#ifndef SQLITE_OMIT_WAL
 22900  22982   #include <sys/mman.h>
        22983  +#endif
 22901  22984   
 22902  22985   #if SQLITE_ENABLE_LOCKING_STYLE
 22903  22986   # include <sys/ioctl.h>
 22904  22987   # if OS_VXWORKS
 22905  22988   #  include <semaphore.h>
 22906  22989   #  include <limits.h>
 22907  22990   # else
................................................................................
 28381  28464   /* Not always defined in the headers as it ought to be */
 28382  28465   extern int gethostuuid(uuid_t id, const struct timespec *wait);
 28383  28466   
 28384  28467   /* get the host ID via gethostuuid(), pHostID must point to PROXY_HOSTIDLEN 
 28385  28468   ** bytes of writable memory.
 28386  28469   */
 28387  28470   static int proxyGetHostID(unsigned char *pHostID, int *pError){
 28388         -  struct timespec timeout = {1, 0}; /* 1 sec timeout */
 28389         -  
 28390  28471     assert(PROXY_HOSTIDLEN == sizeof(uuid_t));
 28391  28472     memset(pHostID, 0, PROXY_HOSTIDLEN);
 28392  28473   #if defined(__MAX_OS_X_VERSION_MIN_REQUIRED)\
 28393  28474                  && __MAC_OS_X_VERSION_MIN_REQUIRED<1050
 28394         -  if( gethostuuid(pHostID, &timeout) ){
 28395         -    int err = errno;
 28396         -    if( pError ){
 28397         -      *pError = err;
        28475  +  {
        28476  +    static const struct timespec timeout = {1, 0}; /* 1 sec timeout */
        28477  +    if( gethostuuid(pHostID, &timeout) ){
        28478  +      int err = errno;
        28479  +      if( pError ){
        28480  +        *pError = err;
        28481  +      }
        28482  +      return SQLITE_IOERR;
 28398  28483       }
 28399         -    return SQLITE_IOERR;
 28400  28484     }
 28401  28485   #endif
 28402  28486   #ifdef SQLITE_TEST
 28403  28487     /* simulate multiple hosts by creating unique hostid file paths */
 28404  28488     if( sqlite3_hostid_num != 0){
 28405  28489       pHostID[0] = (char)(pHostID[0] + (char)(sqlite3_hostid_num & 0xFF));
 28406  28490     }
................................................................................
 33319  33403   ** these two features are available.
 33320  33404   */
 33321  33405   
 33322  33406   
 33323  33407   typedef struct PCache1 PCache1;
 33324  33408   typedef struct PgHdr1 PgHdr1;
 33325  33409   typedef struct PgFreeslot PgFreeslot;
        33410  +typedef struct PGroup PGroup;
        33411  +
        33412  +/* Each page cache (or PCache) belongs to a PGroup.  A PGroup is a set 
        33413  +** of one or more PCaches that are able to recycle each others unpinned
        33414  +** pages when they are under memory pressure.  A PGroup is an instance of
        33415  +** the following object.
        33416  +**
        33417  +** This page cache implementation works in one of two modes:
        33418  +**
        33419  +**   (1)  Every PCache is the sole member of its own PGroup.  There is
        33420  +**        one PGroup per PCache.
        33421  +**
        33422  +**   (2)  There is a single global PGroup that all PCaches are a member
        33423  +**        of.
        33424  +**
        33425  +** Mode 1 uses more memory (since PCache instances are not able to rob
        33426  +** unused pages from other PCaches) but it also operates without a mutex,
        33427  +** and is therefore often faster.  Mode 2 requires a mutex in order to be
        33428  +** threadsafe, but is able recycle pages more efficient.
        33429  +**
        33430  +** For mode (1), PGroup.mutex is NULL.  For mode (2) there is only a single
        33431  +** PGroup which is the pcache1.grp global variable and its mutex is
        33432  +** SQLITE_MUTEX_STATIC_LRU.
        33433  +*/
        33434  +struct PGroup {
        33435  +  sqlite3_mutex *mutex;          /* MUTEX_STATIC_LRU or NULL */
        33436  +  int nMaxPage;                  /* Sum of nMax for purgeable caches */
        33437  +  int nMinPage;                  /* Sum of nMin for purgeable caches */
        33438  +  int mxPinned;                  /* nMaxpage + 10 - nMinPage */
        33439  +  int nCurrentPage;              /* Number of purgeable pages allocated */
        33440  +  PgHdr1 *pLruHead, *pLruTail;   /* LRU list of unpinned pages */
        33441  +};
 33326  33442   
 33327  33443   /* Each page cache is an instance of the following object.  Every
 33328  33444   ** open database file (including each in-memory database and each
 33329  33445   ** temporary or transient database) has a single page cache which
 33330  33446   ** is an instance of this object.
 33331  33447   **
 33332  33448   ** Pointers to structures of this type are cast and returned as 
 33333  33449   ** opaque sqlite3_pcache* handles.
 33334  33450   */
 33335  33451   struct PCache1 {
 33336  33452     /* Cache configuration parameters. Page size (szPage) and the purgeable
 33337  33453     ** flag (bPurgeable) are set when the cache is created. nMax may be 
 33338  33454     ** modified at any time by a call to the pcache1CacheSize() method.
 33339         -  ** The global mutex must be held when accessing nMax.
        33455  +  ** The PGroup mutex must be held when accessing nMax.
 33340  33456     */
        33457  +  PGroup *pGroup;                     /* PGroup this cache belongs to */
 33341  33458     int szPage;                         /* Size of allocated pages in bytes */
 33342  33459     int bPurgeable;                     /* True if cache is purgeable */
 33343  33460     unsigned int nMin;                  /* Minimum number of pages reserved */
 33344  33461     unsigned int nMax;                  /* Configured "cache_size" value */
        33462  +  unsigned int mxPinned;              /* nMax*9/10 */
 33345  33463   
 33346  33464     /* Hash table of all pages. The following variables may only be accessed
 33347         -  ** when the accessor is holding the global mutex (see pcache1EnterMutex() 
 33348         -  ** and pcache1LeaveMutex()).
        33465  +  ** when the accessor is holding the PGroup mutex.
 33349  33466     */
 33350  33467     unsigned int nRecyclable;           /* Number of pages in the LRU list */
 33351  33468     unsigned int nPage;                 /* Total number of pages in apHash */
 33352  33469     unsigned int nHash;                 /* Number of slots in apHash[] */
 33353  33470     PgHdr1 **apHash;                    /* Hash table for fast lookup by key */
 33354  33471   
 33355  33472     unsigned int iMaxKey;               /* Largest key seen since xTruncate() */
................................................................................
 33377  33494     PgFreeslot *pNext;  /* Next free slot */
 33378  33495   };
 33379  33496   
 33380  33497   /*
 33381  33498   ** Global data used by this cache.
 33382  33499   */
 33383  33500   static SQLITE_WSD struct PCacheGlobal {
 33384         -  sqlite3_mutex *mutex;               /* static mutex MUTEX_STATIC_LRU */
        33501  +  PGroup grp;                    /* The global PGroup for mode (2) */
 33385  33502   
 33386         -  int nMaxPage;                       /* Sum of nMaxPage for purgeable caches */
 33387         -  int nMinPage;                       /* Sum of nMinPage for purgeable caches */
 33388         -  int nCurrentPage;                   /* Number of purgeable pages allocated */
 33389         -  PgHdr1 *pLruHead, *pLruTail;        /* LRU list of unpinned pages */
 33390         -
 33391         -  /* Variables related to SQLITE_CONFIG_PAGECACHE settings. */
 33392         -  int szSlot;                         /* Size of each free slot */
 33393         -  int nSlot;                          /* The number of pcache slots */
 33394         -  int nFreeSlot;                      /* Number of unused pcache slots */
 33395         -  int nReserve;                       /* Try to keep nFreeSlot above this */
 33396         -  void *pStart, *pEnd;                /* Bounds of pagecache malloc range */
 33397         -  PgFreeslot *pFree;                  /* Free page blocks */
 33398         -  int isInit;                         /* True if initialized */
        33503  +  /* Variables related to SQLITE_CONFIG_PAGECACHE settings.  The
        33504  +  ** szSlot, nSlot, pStart, pEnd, nReserve, and isInit values are all
        33505  +  ** fixed at sqlite3_initialize() time and do not require mutex protection.
        33506  +  ** The nFreeSlot and pFree values do require mutex protection.
        33507  +  */
        33508  +  int isInit;                    /* True if initialized */
        33509  +  int szSlot;                    /* Size of each free slot */
        33510  +  int nSlot;                     /* The number of pcache slots */
        33511  +  int nReserve;                  /* Try to keep nFreeSlot above this */
        33512  +  void *pStart, *pEnd;           /* Bounds of pagecache malloc range */
        33513  +  /* Above requires no mutex.  Use mutex below for variable that follow. */
        33514  +  sqlite3_mutex *mutex;          /* Mutex for accessing the following: */
        33515  +  int nFreeSlot;                 /* Number of unused pcache slots */
        33516  +  PgFreeslot *pFree;             /* Free page blocks */
        33517  +  /* The following value requires a mutex to change.  We skip the mutex on
        33518  +  ** reading because (1) most platforms read a 32-bit integer atomically and
        33519  +  ** (2) even if an incorrect value is read, no great harm is done since this
        33520  +  ** is really just an optimization. */
        33521  +  int bUnderPressure;            /* True if low on PAGECACHE memory */
 33399  33522   } pcache1_g;
 33400  33523   
 33401  33524   /*
 33402  33525   ** All code in this file should access the global structure above via the
 33403  33526   ** alias "pcache1". This ensures that the WSD emulation is used when
 33404  33527   ** compiling for systems that do not support real WSD.
 33405  33528   */
................................................................................
 33417  33540   **
 33418  33541   **   assert( PGHDR1_TO_PAGE(PAGE_TO_PGHDR1(pCache, X))==X );
 33419  33542   */
 33420  33543   #define PGHDR1_TO_PAGE(p)    (void*)(((char*)p) - p->pCache->szPage)
 33421  33544   #define PAGE_TO_PGHDR1(c, p) (PgHdr1*)(((char*)p) + c->szPage)
 33422  33545   
 33423  33546   /*
 33424         -** Macros to enter and leave the global LRU mutex.
        33547  +** Macros to enter and leave the PCache LRU mutex.
 33425  33548   */
 33426         -#define pcache1EnterMutex() sqlite3_mutex_enter(pcache1.mutex)
 33427         -#define pcache1LeaveMutex() sqlite3_mutex_leave(pcache1.mutex)
        33549  +#define pcache1EnterMutex(X) sqlite3_mutex_enter((X)->mutex)
        33550  +#define pcache1LeaveMutex(X) sqlite3_mutex_leave((X)->mutex)
 33428  33551   
 33429  33552   /******************************************************************************/
 33430  33553   /******** Page Allocation/SQLITE_CONFIG_PCACHE Related Functions **************/
 33431  33554   
 33432  33555   /*
 33433  33556   ** This function is called during initialization if a static buffer is 
 33434  33557   ** supplied to use for the page-cache by passing the SQLITE_CONFIG_PAGECACHE
 33435  33558   ** verb to sqlite3_config(). Parameter pBuf points to an allocation large
 33436  33559   ** enough to contain 'n' buffers of 'sz' bytes each.
        33560  +**
        33561  +** This routine is called from sqlite3_initialize() and so it is guaranteed
        33562  +** to be serialized already.  There is no need for further mutexing.
 33437  33563   */
 33438  33564   SQLITE_PRIVATE void sqlite3PCacheBufferSetup(void *pBuf, int sz, int n){
 33439  33565     if( pcache1.isInit ){
 33440  33566       PgFreeslot *p;
 33441  33567       sz = ROUNDDOWN8(sz);
 33442  33568       pcache1.szSlot = sz;
 33443  33569       pcache1.nSlot = pcache1.nFreeSlot = n;
 33444  33570       pcache1.nReserve = n>90 ? 10 : (n/10 + 1);
 33445  33571       pcache1.pStart = pBuf;
 33446  33572       pcache1.pFree = 0;
        33573  +    pcache1.bUnderPressure = 0;
 33447  33574       while( n-- ){
 33448  33575         p = (PgFreeslot*)pBuf;
 33449  33576         p->pNext = pcache1.pFree;
 33450  33577         pcache1.pFree = p;
 33451  33578         pBuf = (void*)&((char*)pBuf)[sz];
 33452  33579       }
 33453  33580       pcache1.pEnd = pBuf;
................................................................................
 33455  33582   }
 33456  33583   
 33457  33584   /*
 33458  33585   ** Malloc function used within this file to allocate space from the buffer
 33459  33586   ** configured using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no 
 33460  33587   ** such buffer exists or there is no space left in it, this function falls 
 33461  33588   ** back to sqlite3Malloc().
        33589  +**
        33590  +** Multiple threads can run this routine at the same time.  Global variables
        33591  +** in pcache1 need to be protected via mutex.
 33462  33592   */
 33463  33593   static void *pcache1Alloc(int nByte){
 33464         -  void *p;
 33465         -  assert( sqlite3_mutex_held(pcache1.mutex) );
        33594  +  void *p = 0;
        33595  +  assert( sqlite3_mutex_notheld(pcache1.grp.mutex) );
 33466  33596     sqlite3StatusSet(SQLITE_STATUS_PAGECACHE_SIZE, nByte);
 33467         -  if( nByte<=pcache1.szSlot && pcache1.pFree ){
 33468         -    assert( pcache1.isInit );
        33597  +  if( nByte<=pcache1.szSlot ){
        33598  +    sqlite3_mutex_enter(pcache1.mutex);
 33469  33599       p = (PgHdr1 *)pcache1.pFree;
 33470         -    pcache1.pFree = pcache1.pFree->pNext;
 33471         -    pcache1.nFreeSlot--;
 33472         -    assert( pcache1.nFreeSlot>=0 );
 33473         -    sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_USED, 1);
 33474         -  }else{
 33475         -
 33476         -    /* Allocate a new buffer using sqlite3Malloc. Before doing so, exit the
 33477         -    ** global pcache mutex and unlock the pager-cache object pCache. This is 
 33478         -    ** so that if the attempt to allocate a new buffer causes the the 
 33479         -    ** configured soft-heap-limit to be breached, it will be possible to
 33480         -    ** reclaim memory from this pager-cache.
        33600  +    if( p ){
        33601  +      pcache1.pFree = pcache1.pFree->pNext;
        33602  +      pcache1.nFreeSlot--;
        33603  +      pcache1.bUnderPressure = pcache1.nFreeSlot<pcache1.nReserve;
        33604  +      assert( pcache1.nFreeSlot>=0 );
        33605  +      sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_USED, 1);
        33606  +    }
        33607  +    sqlite3_mutex_leave(pcache1.mutex);
        33608  +  }
        33609  +  if( p==0 ){
        33610  +    /* Memory is not available in the SQLITE_CONFIG_PAGECACHE pool.  Get
        33611  +    ** it from sqlite3Malloc instead.
 33481  33612       */
 33482         -    pcache1LeaveMutex();
 33483  33613       p = sqlite3Malloc(nByte);
 33484         -    pcache1EnterMutex();
 33485  33614       if( p ){
 33486  33615         int sz = sqlite3MallocSize(p);
 33487  33616         sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_OVERFLOW, sz);
 33488  33617       }
 33489  33618       sqlite3MemdebugSetType(p, MEMTYPE_PCACHE);
 33490  33619     }
 33491  33620     return p;
 33492  33621   }
 33493  33622   
 33494  33623   /*
 33495  33624   ** Free an allocated buffer obtained from pcache1Alloc().
 33496  33625   */
 33497  33626   static void pcache1Free(void *p){
 33498         -  assert( sqlite3_mutex_held(pcache1.mutex) );
 33499  33627     if( p==0 ) return;
 33500  33628     if( p>=pcache1.pStart && p<pcache1.pEnd ){
 33501  33629       PgFreeslot *pSlot;
        33630  +    sqlite3_mutex_enter(pcache1.mutex);
 33502  33631       sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_USED, -1);
 33503  33632       pSlot = (PgFreeslot*)p;
 33504  33633       pSlot->pNext = pcache1.pFree;
 33505  33634       pcache1.pFree = pSlot;
 33506  33635       pcache1.nFreeSlot++;
        33636  +    pcache1.bUnderPressure = pcache1.nFreeSlot<pcache1.nReserve;
 33507  33637       assert( pcache1.nFreeSlot<=pcache1.nSlot );
        33638  +    sqlite3_mutex_leave(pcache1.mutex);
 33508  33639     }else{
 33509  33640       int iSize;
 33510  33641       assert( sqlite3MemdebugHasType(p, MEMTYPE_PCACHE) );
 33511  33642       sqlite3MemdebugSetType(p, MEMTYPE_HEAP);
 33512  33643       iSize = sqlite3MallocSize(p);
 33513  33644       sqlite3StatusAdd(SQLITE_STATUS_PAGECACHE_OVERFLOW, -iSize);
 33514  33645       sqlite3_free(p);
................................................................................
 33516  33647   }
 33517  33648   
 33518  33649   #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT
 33519  33650   /*
 33520  33651   ** Return the size of a pcache allocation
 33521  33652   */
 33522  33653   static int pcache1MemSize(void *p){
 33523         -  assert( sqlite3_mutex_held(pcache1.mutex) );
 33524  33654     if( p>=pcache1.pStart && p<pcache1.pEnd ){
 33525  33655       return pcache1.szSlot;
 33526  33656     }else{
 33527  33657       int iSize;
 33528  33658       assert( sqlite3MemdebugHasType(p, MEMTYPE_PCACHE) );
 33529  33659       sqlite3MemdebugSetType(p, MEMTYPE_HEAP);
 33530  33660       iSize = sqlite3MallocSize(p);
................................................................................
 33540  33670   static PgHdr1 *pcache1AllocPage(PCache1 *pCache){
 33541  33671     int nByte = sizeof(PgHdr1) + pCache->szPage;
 33542  33672     void *pPg = pcache1Alloc(nByte);
 33543  33673     PgHdr1 *p;
 33544  33674     if( pPg ){
 33545  33675       p = PAGE_TO_PGHDR1(pCache, pPg);
 33546  33676       if( pCache->bPurgeable ){
 33547         -      pcache1.nCurrentPage++;
        33677  +      pCache->pGroup->nCurrentPage++;
 33548  33678       }
 33549  33679     }else{
 33550  33680       p = 0;
 33551  33681     }
 33552  33682     return p;
 33553  33683   }
 33554  33684   
................................................................................
 33557  33687   **
 33558  33688   ** The pointer is allowed to be NULL, which is prudent.  But it turns out
 33559  33689   ** that the current implementation happens to never call this routine
 33560  33690   ** with a NULL pointer, so we mark the NULL test with ALWAYS().
 33561  33691   */
 33562  33692   static void pcache1FreePage(PgHdr1 *p){
 33563  33693     if( ALWAYS(p) ){
 33564         -    if( p->pCache->bPurgeable ){
 33565         -      pcache1.nCurrentPage--;
        33694  +    PCache1 *pCache = p->pCache;
        33695  +    if( pCache->bPurgeable ){
        33696  +      pCache->pGroup->nCurrentPage--;
 33566  33697       }
 33567  33698       pcache1Free(PGHDR1_TO_PAGE(p));
 33568  33699     }
 33569  33700   }
 33570  33701   
 33571  33702   /*
 33572  33703   ** Malloc function used by SQLite to obtain space from the buffer configured
 33573  33704   ** using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no such buffer
 33574  33705   ** exists, this function falls back to sqlite3Malloc().
 33575  33706   */
 33576  33707   SQLITE_PRIVATE void *sqlite3PageMalloc(int sz){
 33577         -  void *p;
 33578         -  pcache1EnterMutex();
 33579         -  p = pcache1Alloc(sz);
 33580         -  pcache1LeaveMutex();
 33581         -  return p;
        33708  +  return pcache1Alloc(sz);
 33582  33709   }
 33583  33710   
 33584  33711   /*
 33585  33712   ** Free an allocated buffer obtained from sqlite3PageMalloc().
 33586  33713   */
 33587  33714   SQLITE_PRIVATE void sqlite3PageFree(void *p){
 33588         -  pcache1EnterMutex();
 33589  33715     pcache1Free(p);
 33590         -  pcache1LeaveMutex();
 33591  33716   }
 33592  33717   
 33593  33718   
 33594  33719   /*
 33595  33720   ** Return true if it desirable to avoid allocating a new page cache
 33596  33721   ** entry.
 33597  33722   **
................................................................................
 33604  33729   **
 33605  33730   ** Or, the heap is used for all page cache memory put the heap is
 33606  33731   ** under memory pressure, then again it is desirable to avoid
 33607  33732   ** allocating a new page cache entry in order to avoid stressing
 33608  33733   ** the heap even further.
 33609  33734   */
 33610  33735   static int pcache1UnderMemoryPressure(PCache1 *pCache){
 33611         -  assert( sqlite3_mutex_held(pcache1.mutex) );
 33612  33736     if( pcache1.nSlot && pCache->szPage<=pcache1.szSlot ){
 33613         -    return pcache1.nFreeSlot<pcache1.nReserve;
        33737  +    return pcache1.bUnderPressure;
 33614  33738     }else{
 33615  33739       return sqlite3HeapNearlyFull();
 33616  33740     }
 33617  33741   }
 33618  33742   
 33619  33743   /******************************************************************************/
 33620  33744   /******** General Implementation Functions ************************************/
 33621  33745   
 33622  33746   /*
 33623  33747   ** This function is used to resize the hash table used by the cache passed
 33624  33748   ** as the first argument.
 33625  33749   **
 33626         -** The global mutex must be held when this function is called.
        33750  +** The PCache mutex must be held when this function is called.
 33627  33751   */
 33628  33752   static int pcache1ResizeHash(PCache1 *p){
 33629  33753     PgHdr1 **apNew;
 33630  33754     unsigned int nNew;
 33631  33755     unsigned int i;
 33632  33756   
 33633         -  assert( sqlite3_mutex_held(pcache1.mutex) );
        33757  +  assert( sqlite3_mutex_held(p->pGroup->mutex) );
 33634  33758   
 33635  33759     nNew = p->nHash*2;
 33636  33760     if( nNew<256 ){
 33637  33761       nNew = 256;
 33638  33762     }
 33639  33763   
 33640         -  pcache1LeaveMutex();
        33764  +  pcache1LeaveMutex(p->pGroup);
 33641  33765     if( p->nHash ){ sqlite3BeginBenignMalloc(); }
 33642  33766     apNew = (PgHdr1 **)sqlite3_malloc(sizeof(PgHdr1 *)*nNew);
 33643  33767     if( p->nHash ){ sqlite3EndBenignMalloc(); }
 33644         -  pcache1EnterMutex();
        33768  +  pcache1EnterMutex(p->pGroup);
 33645  33769     if( apNew ){
 33646  33770       memset(apNew, 0, sizeof(PgHdr1 *)*nNew);
 33647  33771       for(i=0; i<p->nHash; i++){
 33648  33772         PgHdr1 *pPage;
 33649  33773         PgHdr1 *pNext = p->apHash[i];
 33650  33774         while( (pPage = pNext)!=0 ){
 33651  33775           unsigned int h = pPage->iKey % nNew;
................................................................................
 33660  33784     }
 33661  33785   
 33662  33786     return (p->apHash ? SQLITE_OK : SQLITE_NOMEM);
 33663  33787   }
 33664  33788   
 33665  33789   /*
 33666  33790   ** This function is used internally to remove the page pPage from the 
 33667         -** global LRU list, if is part of it. If pPage is not part of the global
        33791  +** PGroup LRU list, if is part of it. If pPage is not part of the PGroup
 33668  33792   ** LRU list, then this function is a no-op.
 33669  33793   **
 33670         -** The global mutex must be held when this function is called.
        33794  +** The PGroup mutex must be held when this function is called.
        33795  +**
        33796  +** If pPage is NULL then this routine is a no-op.
 33671  33797   */
 33672  33798   static void pcache1PinPage(PgHdr1 *pPage){
 33673         -  assert( sqlite3_mutex_held(pcache1.mutex) );
 33674         -  if( pPage && (pPage->pLruNext || pPage==pcache1.pLruTail) ){
        33799  +  PCache1 *pCache;
        33800  +  PGroup *pGroup;
        33801  +
        33802  +  if( pPage==0 ) return;
        33803  +  pCache = pPage->pCache;
        33804  +  pGroup = pCache->pGroup;
        33805  +  assert( sqlite3_mutex_held(pGroup->mutex) );
        33806  +  if( pPage->pLruNext || pPage==pGroup->pLruTail ){
 33675  33807       if( pPage->pLruPrev ){
 33676  33808         pPage->pLruPrev->pLruNext = pPage->pLruNext;
 33677  33809       }
 33678  33810       if( pPage->pLruNext ){
 33679  33811         pPage->pLruNext->pLruPrev = pPage->pLruPrev;
 33680  33812       }
 33681         -    if( pcache1.pLruHead==pPage ){
 33682         -      pcache1.pLruHead = pPage->pLruNext;
        33813  +    if( pGroup->pLruHead==pPage ){
        33814  +      pGroup->pLruHead = pPage->pLruNext;
 33683  33815       }
 33684         -    if( pcache1.pLruTail==pPage ){
 33685         -      pcache1.pLruTail = pPage->pLruPrev;
        33816  +    if( pGroup->pLruTail==pPage ){
        33817  +      pGroup->pLruTail = pPage->pLruPrev;
 33686  33818       }
 33687  33819       pPage->pLruNext = 0;
 33688  33820       pPage->pLruPrev = 0;
 33689  33821       pPage->pCache->nRecyclable--;
 33690  33822     }
 33691  33823   }
 33692  33824   
 33693  33825   
 33694  33826   /*
 33695  33827   ** Remove the page supplied as an argument from the hash table 
 33696  33828   ** (PCache1.apHash structure) that it is currently stored in.
 33697  33829   **
 33698         -** The global mutex must be held when this function is called.
        33830  +** The PGroup mutex must be held when this function is called.
 33699  33831   */
 33700  33832   static void pcache1RemoveFromHash(PgHdr1 *pPage){
 33701  33833     unsigned int h;
 33702  33834     PCache1 *pCache = pPage->pCache;
 33703  33835     PgHdr1 **pp;
 33704  33836   
        33837  +  assert( sqlite3_mutex_held(pCache->pGroup->mutex) );
 33705  33838     h = pPage->iKey % pCache->nHash;
 33706  33839     for(pp=&pCache->apHash[h]; (*pp)!=pPage; pp=&(*pp)->pNext);
 33707  33840     *pp = (*pp)->pNext;
 33708  33841   
 33709  33842     pCache->nPage--;
 33710  33843   }
 33711  33844   
 33712  33845   /*
 33713         -** If there are currently more than pcache.nMaxPage pages allocated, try
 33714         -** to recycle pages to reduce the number allocated to pcache.nMaxPage.
        33846  +** If there are currently more than nMaxPage pages allocated, try
        33847  +** to recycle pages to reduce the number allocated to nMaxPage.
 33715  33848   */
 33716         -static void pcache1EnforceMaxPage(void){
 33717         -  assert( sqlite3_mutex_held(pcache1.mutex) );
 33718         -  while( pcache1.nCurrentPage>pcache1.nMaxPage && pcache1.pLruTail ){
 33719         -    PgHdr1 *p = pcache1.pLruTail;
        33849  +static void pcache1EnforceMaxPage(PGroup *pGroup){
        33850  +  assert( sqlite3_mutex_held(pGroup->mutex) );
        33851  +  while( pGroup->nCurrentPage>pGroup->nMaxPage && pGroup->pLruTail ){
        33852  +    PgHdr1 *p = pGroup->pLruTail;
        33853  +    assert( p->pCache->pGroup==pGroup );
 33720  33854       pcache1PinPage(p);
 33721  33855       pcache1RemoveFromHash(p);
 33722  33856       pcache1FreePage(p);
 33723  33857     }
 33724  33858   }
 33725  33859   
 33726  33860   /*
 33727  33861   ** Discard all pages from cache pCache with a page number (key value) 
 33728  33862   ** greater than or equal to iLimit. Any pinned pages that meet this 
 33729  33863   ** criteria are unpinned before they are discarded.
 33730  33864   **
 33731         -** The global mutex must be held when this function is called.
        33865  +** The PCache mutex must be held when this function is called.
 33732  33866   */
 33733  33867   static void pcache1TruncateUnsafe(
 33734         -  PCache1 *pCache, 
 33735         -  unsigned int iLimit 
        33868  +  PCache1 *pCache,             /* The cache to truncate */
        33869  +  unsigned int iLimit          /* Drop pages with this pgno or larger */
 33736  33870   ){
 33737         -  TESTONLY( unsigned int nPage = 0; )      /* Used to assert pCache->nPage is correct */
        33871  +  TESTONLY( unsigned int nPage = 0; )  /* To assert pCache->nPage is correct */
 33738  33872     unsigned int h;
 33739         -  assert( sqlite3_mutex_held(pcache1.mutex) );
        33873  +  assert( sqlite3_mutex_held(pCache->pGroup->mutex) );
 33740  33874     for(h=0; h<pCache->nHash; h++){
 33741  33875       PgHdr1 **pp = &pCache->apHash[h]; 
 33742  33876       PgHdr1 *pPage;
 33743  33877       while( (pPage = *pp)!=0 ){
 33744  33878         if( pPage->iKey>=iLimit ){
 33745  33879           pCache->nPage--;
 33746  33880           *pp = pPage->pNext;
................................................................................
 33762  33896   ** Implementation of the sqlite3_pcache.xInit method.
 33763  33897   */
 33764  33898   static int pcache1Init(void *NotUsed){
 33765  33899     UNUSED_PARAMETER(NotUsed);
 33766  33900     assert( pcache1.isInit==0 );
 33767  33901     memset(&pcache1, 0, sizeof(pcache1));
 33768  33902     if( sqlite3GlobalConfig.bCoreMutex ){
 33769         -    pcache1.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_LRU);
        33903  +    pcache1.grp.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_LRU);
        33904  +    pcache1.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_PMEM);
 33770  33905     }
        33906  +  pcache1.grp.mxPinned = 10;
 33771  33907     pcache1.isInit = 1;
 33772  33908     return SQLITE_OK;
 33773  33909   }
 33774  33910   
 33775  33911   /*
 33776  33912   ** Implementation of the sqlite3_pcache.xShutdown method.
 33777  33913   ** Note that the static mutex allocated in xInit does 
................................................................................
 33785  33921   
 33786  33922   /*
 33787  33923   ** Implementation of the sqlite3_pcache.xCreate method.
 33788  33924   **
 33789  33925   ** Allocate a new cache.
 33790  33926   */
 33791  33927   static sqlite3_pcache *pcache1Create(int szPage, int bPurgeable){
 33792         -  PCache1 *pCache;
        33928  +  PCache1 *pCache;      /* The newly created page cache */
        33929  +  PGroup *pGroup;       /* The group the new page cache will belong to */
        33930  +  int sz;               /* Bytes of memory required to allocate the new cache */
 33793  33931   
 33794         -  pCache = (PCache1 *)sqlite3_malloc(sizeof(PCache1));
        33932  +  /*
        33933  +  ** The seperateCache variable is true if each PCache has its own private
        33934  +  ** PGroup.  In other words, separateCache is true for mode (1) where no
        33935  +  ** mutexing is required.
        33936  +  **
        33937  +  **   *  Always use a unified cache (mode-2) if ENABLE_MEMORY_MANAGEMENT
        33938  +  **
        33939  +  **   *  Always use a unified cache in single-threaded applications
        33940  +  **
        33941  +  **   *  Otherwise (if multi-threaded and ENABLE_MEMORY_MANAGEMENT is off)
        33942  +  **      use separate caches (mode-1)
        33943  +  */
        33944  +#if defined(SQLITE_ENABLE_MEMORY_MANAGEMENT) || SQLITE_THREADSAFE==0
        33945  +  const int separateCache = 0;
        33946  +#else
        33947  +  int separateCache = sqlite3GlobalConfig.bCoreMutex>0;
        33948  +#endif
        33949  +
        33950  +  sz = sizeof(PCache1) + sizeof(PGroup)*separateCache;
        33951  +  pCache = (PCache1 *)sqlite3_malloc(sz);
 33795  33952     if( pCache ){
 33796         -    memset(pCache, 0, sizeof(PCache1));
        33953  +    memset(pCache, 0, sz);
        33954  +    if( separateCache ){
        33955  +      pGroup = (PGroup*)&pCache[1];
        33956  +      pGroup->mxPinned = 10;
        33957  +    }else{
        33958  +      pGroup = &pcache1_g.grp;
        33959  +    }
        33960  +    pCache->pGroup = pGroup;
 33797  33961       pCache->szPage = szPage;
 33798  33962       pCache->bPurgeable = (bPurgeable ? 1 : 0);
 33799  33963       if( bPurgeable ){
 33800  33964         pCache->nMin = 10;
 33801         -      pcache1EnterMutex();
 33802         -      pcache1.nMinPage += pCache->nMin;
 33803         -      pcache1LeaveMutex();
        33965  +      pcache1EnterMutex(pGroup);
        33966  +      pGroup->nMinPage += pCache->nMin;
        33967  +      pGroup->mxPinned = pGroup->nMaxPage + 10 - pGroup->nMinPage;
        33968  +      pcache1LeaveMutex(pGroup);
 33804  33969       }
 33805  33970     }
 33806  33971     return (sqlite3_pcache *)pCache;
 33807  33972   }
 33808  33973   
 33809  33974   /*
 33810  33975   ** Implementation of the sqlite3_pcache.xCachesize method. 
 33811  33976   **
 33812  33977   ** Configure the cache_size limit for a cache.
 33813  33978   */
 33814  33979   static void pcache1Cachesize(sqlite3_pcache *p, int nMax){
 33815  33980     PCache1 *pCache = (PCache1 *)p;
 33816  33981     if( pCache->bPurgeable ){
 33817         -    pcache1EnterMutex();
 33818         -    pcache1.nMaxPage += (nMax - pCache->nMax);
        33982  +    PGroup *pGroup = pCache->pGroup;
        33983  +    pcache1EnterMutex(pGroup);
        33984  +    pGroup->nMaxPage += (nMax - pCache->nMax);
        33985  +    pGroup->mxPinned = pGroup->nMaxPage + 10 - pGroup->nMinPage;
 33819  33986       pCache->nMax = nMax;
 33820         -    pcache1EnforceMaxPage();
 33821         -    pcache1LeaveMutex();
        33987  +    pCache->mxPinned = nMax*9/10;
        33988  +    pcache1EnforceMaxPage(pGroup);
        33989  +    pcache1LeaveMutex(pGroup);
 33822  33990     }
 33823  33991   }
 33824  33992   
 33825  33993   /*
 33826  33994   ** Implementation of the sqlite3_pcache.xPagecount method. 
 33827  33995   */
 33828  33996   static int pcache1Pagecount(sqlite3_pcache *p){
 33829  33997     int n;
 33830         -  pcache1EnterMutex();
 33831         -  n = ((PCache1 *)p)->nPage;
 33832         -  pcache1LeaveMutex();
        33998  +  PCache1 *pCache = (PCache1*)p;
        33999  +  pcache1EnterMutex(pCache->pGroup);
        34000  +  n = pCache->nPage;
        34001  +  pcache1LeaveMutex(pCache->pGroup);
 33833  34002     return n;
 33834  34003   }
 33835  34004   
 33836  34005   /*
 33837  34006   ** Implementation of the sqlite3_pcache.xFetch method. 
 33838  34007   **
 33839  34008   ** Fetch a page by key value.
................................................................................
 33884  34053   **      then attempt to recycle a page from the LRU list. If it is the right
 33885  34054   **      size, return the recycled buffer. Otherwise, free the buffer and
 33886  34055   **      proceed to step 5. 
 33887  34056   **
 33888  34057   **   5. Otherwise, allocate and return a new page buffer.
 33889  34058   */
 33890  34059   static void *pcache1Fetch(sqlite3_pcache *p, unsigned int iKey, int createFlag){
 33891         -  unsigned int nPinned;
        34060  +  int nPinned;
 33892  34061     PCache1 *pCache = (PCache1 *)p;
        34062  +  PGroup *pGroup;
 33893  34063     PgHdr1 *pPage = 0;
 33894  34064   
 33895  34065     assert( pCache->bPurgeable || createFlag!=1 );
 33896         -  pcache1EnterMutex();
 33897         -  if( createFlag==1 ) sqlite3BeginBenignMalloc();
        34066  +  assert( pCache->bPurgeable || pCache->nMin==0 );
        34067  +  assert( pCache->bPurgeable==0 || pCache->nMin==10 );
        34068  +  assert( pCache->nMin==0 || pCache->bPurgeable );
        34069  +  pcache1EnterMutex(pGroup = pCache->pGroup);
 33898  34070   
 33899         -  /* Search the hash table for an existing entry. */
        34071  +  /* Step 1: Search the hash table for an existing entry. */
 33900  34072     if( pCache->nHash>0 ){
 33901  34073       unsigned int h = iKey % pCache->nHash;
 33902  34074       for(pPage=pCache->apHash[h]; pPage&&pPage->iKey!=iKey; pPage=pPage->pNext);
 33903  34075     }
 33904  34076   
        34077  +  /* Step 2: Abort if no existing page is found and createFlag is 0 */
 33905  34078     if( pPage || createFlag==0 ){
 33906  34079       pcache1PinPage(pPage);
 33907  34080       goto fetch_out;
 33908  34081     }
 33909  34082   
 33910         -  /* Step 3 of header comment. */
        34083  +  /* The pGroup local variable will normally be initialized by the
        34084  +  ** pcache1EnterMutex() macro above.  But if SQLITE_MUTEX_OMIT is defined,
        34085  +  ** then pcache1EnterMutex() is a no-op, so we have to initialize the
        34086  +  ** local variable here.  Delaying the initialization of pGroup is an
        34087  +  ** optimization:  The common case is to exit the module before reaching
        34088  +  ** this point.
        34089  +  */
        34090  +#ifdef SQLITE_MUTEX_OMIT
        34091  +  pGroup = pCache->pGroup;
        34092  +#endif
        34093  +
        34094  +
        34095  +  /* Step 3: Abort if createFlag is 1 but the cache is nearly full */
 33911  34096     nPinned = pCache->nPage - pCache->nRecyclable;
        34097  +  assert( pGroup->mxPinned == pGroup->nMaxPage + 10 - pGroup->nMinPage );
        34098  +  assert( pCache->mxPinned == pCache->nMax*9/10 );
 33912  34099     if( createFlag==1 && (
 33913         -        nPinned>=(pcache1.nMaxPage+pCache->nMin-pcache1.nMinPage)
 33914         -     || nPinned>=(pCache->nMax * 9 / 10)
        34100  +        nPinned>=pGroup->mxPinned
        34101  +     || nPinned>=(int)pCache->mxPinned
 33915  34102        || pcache1UnderMemoryPressure(pCache)
 33916  34103     )){
 33917  34104       goto fetch_out;
 33918  34105     }
 33919  34106   
 33920  34107     if( pCache->nPage>=pCache->nHash && pcache1ResizeHash(pCache) ){
 33921  34108       goto fetch_out;
 33922  34109     }
 33923  34110   
 33924         -  /* Step 4. Try to recycle a page buffer if appropriate. */
 33925         -  if( pCache->bPurgeable && pcache1.pLruTail && (
        34111  +  /* Step 4. Try to recycle a page. */
        34112  +  if( pCache->bPurgeable && pGroup->pLruTail && (
 33926  34113            (pCache->nPage+1>=pCache->nMax)
 33927         -      || pcache1.nCurrentPage>=pcache1.nMaxPage
        34114  +      || pGroup->nCurrentPage>=pGroup->nMaxPage
 33928  34115         || pcache1UnderMemoryPressure(pCache)
 33929  34116     )){
 33930         -    pPage = pcache1.pLruTail;
        34117  +    PCache1 *pOtherCache;
        34118  +    pPage = pGroup->pLruTail;
 33931  34119       pcache1RemoveFromHash(pPage);
 33932  34120       pcache1PinPage(pPage);
 33933         -    if( pPage->pCache->szPage!=pCache->szPage ){
        34121  +    if( (pOtherCache = pPage->pCache)->szPage!=pCache->szPage ){
 33934  34122         pcache1FreePage(pPage);
 33935  34123         pPage = 0;
 33936  34124       }else{
 33937         -      pcache1.nCurrentPage -= (pPage->pCache->bPurgeable - pCache->bPurgeable);
        34125  +      pGroup->nCurrentPage -= 
        34126  +               (pOtherCache->bPurgeable - pCache->bPurgeable);
 33938  34127       }
 33939  34128     }
 33940  34129   
 33941  34130     /* Step 5. If a usable page buffer has still not been found, 
 33942  34131     ** attempt to allocate a new one. 
 33943  34132     */
 33944  34133     if( !pPage ){
        34134  +    if( createFlag==1 ) sqlite3BeginBenignMalloc();
        34135  +    pcache1LeaveMutex(pGroup);
 33945  34136       pPage = pcache1AllocPage(pCache);
        34137  +    pcache1EnterMutex(pGroup);
        34138  +    if( createFlag==1 ) sqlite3EndBenignMalloc();
 33946  34139     }
 33947  34140   
 33948  34141     if( pPage ){
 33949  34142       unsigned int h = iKey % pCache->nHash;
 33950  34143       pCache->nPage++;
 33951  34144       pPage->iKey = iKey;
 33952  34145       pPage->pNext = pCache->apHash[h];
................................................................................
 33957  34150       pCache->apHash[h] = pPage;
 33958  34151     }
 33959  34152   
 33960  34153   fetch_out:
 33961  34154     if( pPage && iKey>pCache->iMaxKey ){
 33962  34155       pCache->iMaxKey = iKey;
 33963  34156     }
 33964         -  if( createFlag==1 ) sqlite3EndBenignMalloc();
 33965         -  pcache1LeaveMutex();
        34157  +  pcache1LeaveMutex(pGroup);
 33966  34158     return (pPage ? PGHDR1_TO_PAGE(pPage) : 0);
 33967  34159   }
 33968  34160   
 33969  34161   
 33970  34162   /*
 33971  34163   ** Implementation of the sqlite3_pcache.xUnpin method.
 33972  34164   **
 33973  34165   ** Mark a page as unpinned (eligible for asynchronous recycling).
 33974  34166   */
 33975  34167   static void pcache1Unpin(sqlite3_pcache *p, void *pPg, int reuseUnlikely){
 33976  34168     PCache1 *pCache = (PCache1 *)p;
 33977  34169     PgHdr1 *pPage = PAGE_TO_PGHDR1(pCache, pPg);
        34170  +  PGroup *pGroup = pCache->pGroup;
 33978  34171    
 33979  34172     assert( pPage->pCache==pCache );
 33980         -  pcache1EnterMutex();
        34173  +  pcache1EnterMutex(pGroup);
 33981  34174   
 33982  34175     /* It is an error to call this function if the page is already 
 33983         -  ** part of the global LRU list.
        34176  +  ** part of the PGroup LRU list.
 33984  34177     */
 33985  34178     assert( pPage->pLruPrev==0 && pPage->pLruNext==0 );
 33986         -  assert( pcache1.pLruHead!=pPage && pcache1.pLruTail!=pPage );
        34179  +  assert( pGroup->pLruHead!=pPage && pGroup->pLruTail!=pPage );
 33987  34180   
 33988         -  if( reuseUnlikely || pcache1.nCurrentPage>pcache1.nMaxPage ){
        34181  +  if( reuseUnlikely || pGroup->nCurrentPage>pGroup->nMaxPage ){
 33989  34182       pcache1RemoveFromHash(pPage);
 33990  34183       pcache1FreePage(pPage);
 33991  34184     }else{
 33992         -    /* Add the page to the global LRU list. Normally, the page is added to
 33993         -    ** the head of the list (last page to be recycled). However, if the 
 33994         -    ** reuseUnlikely flag passed to this function is true, the page is added
 33995         -    ** to the tail of the list (first page to be recycled).
 33996         -    */
 33997         -    if( pcache1.pLruHead ){
 33998         -      pcache1.pLruHead->pLruPrev = pPage;
 33999         -      pPage->pLruNext = pcache1.pLruHead;
 34000         -      pcache1.pLruHead = pPage;
        34185  +    /* Add the page to the PGroup LRU list. */
        34186  +    if( pGroup->pLruHead ){
        34187  +      pGroup->pLruHead->pLruPrev = pPage;
        34188  +      pPage->pLruNext = pGroup->pLruHead;
        34189  +      pGroup->pLruHead = pPage;
 34001  34190       }else{
 34002         -      pcache1.pLruTail = pPage;
 34003         -      pcache1.pLruHead = pPage;
        34191  +      pGroup->pLruTail = pPage;
        34192  +      pGroup->pLruHead = pPage;
 34004  34193       }
 34005  34194       pCache->nRecyclable++;
 34006  34195     }
 34007  34196   
 34008         -  pcache1LeaveMutex();
        34197  +  pcache1LeaveMutex(pCache->pGroup);
 34009  34198   }
 34010  34199   
 34011  34200   /*
 34012  34201   ** Implementation of the sqlite3_pcache.xRekey method. 
 34013  34202   */
 34014  34203   static void pcache1Rekey(
 34015  34204     sqlite3_pcache *p,
................................................................................
 34020  34209     PCache1 *pCache = (PCache1 *)p;
 34021  34210     PgHdr1 *pPage = PAGE_TO_PGHDR1(pCache, pPg);
 34022  34211     PgHdr1 **pp;
 34023  34212     unsigned int h; 
 34024  34213     assert( pPage->iKey==iOld );
 34025  34214     assert( pPage->pCache==pCache );
 34026  34215   
 34027         -  pcache1EnterMutex();
        34216  +  pcache1EnterMutex(pCache->pGroup);
 34028  34217   
 34029  34218     h = iOld%pCache->nHash;
 34030  34219     pp = &pCache->apHash[h];
 34031  34220     while( (*pp)!=pPage ){
 34032  34221       pp = &(*pp)->pNext;
 34033  34222     }
 34034  34223     *pp = pPage->pNext;
................................................................................
 34037  34226     pPage->iKey = iNew;
 34038  34227     pPage->pNext = pCache->apHash[h];
 34039  34228     pCache->apHash[h] = pPage;
 34040  34229     if( iNew>pCache->iMaxKey ){
 34041  34230       pCache->iMaxKey = iNew;
 34042  34231     }
 34043  34232   
 34044         -  pcache1LeaveMutex();
        34233  +  pcache1LeaveMutex(pCache->pGroup);
 34045  34234   }
 34046  34235   
 34047  34236   /*
 34048  34237   ** Implementation of the sqlite3_pcache.xTruncate method. 
 34049  34238   **
 34050  34239   ** Discard all unpinned pages in the cache with a page number equal to
 34051  34240   ** or greater than parameter iLimit. Any pinned pages with a page number
 34052  34241   ** equal to or greater than iLimit are implicitly unpinned.
 34053  34242   */
 34054  34243   static void pcache1Truncate(sqlite3_pcache *p, unsigned int iLimit){
 34055  34244     PCache1 *pCache = (PCache1 *)p;
 34056         -  pcache1EnterMutex();
        34245  +  pcache1EnterMutex(pCache->pGroup);
 34057  34246     if( iLimit<=pCache->iMaxKey ){
 34058  34247       pcache1TruncateUnsafe(pCache, iLimit);
 34059  34248       pCache->iMaxKey = iLimit-1;
 34060  34249     }
 34061         -  pcache1LeaveMutex();
        34250  +  pcache1LeaveMutex(pCache->pGroup);
 34062  34251   }
 34063  34252   
 34064  34253   /*
 34065  34254   ** Implementation of the sqlite3_pcache.xDestroy method. 
 34066  34255   **
 34067  34256   ** Destroy a cache allocated using pcache1Create().
 34068  34257   */
 34069  34258   static void pcache1Destroy(sqlite3_pcache *p){
 34070  34259     PCache1 *pCache = (PCache1 *)p;
        34260  +  PGroup *pGroup = pCache->pGroup;
 34071  34261     assert( pCache->bPurgeable || (pCache->nMax==0 && pCache->nMin==0) );
 34072         -  pcache1EnterMutex();
        34262  +  pcache1EnterMutex(pGroup);
 34073  34263     pcache1TruncateUnsafe(pCache, 0);
 34074         -  pcache1.nMaxPage -= pCache->nMax;
 34075         -  pcache1.nMinPage -= pCache->nMin;
 34076         -  pcache1EnforceMaxPage();
 34077         -  pcache1LeaveMutex();
        34264  +  pGroup->nMaxPage -= pCache->nMax;
        34265  +  pGroup->nMinPage -= pCache->nMin;
        34266  +  pGroup->mxPinned = pGroup->nMaxPage + 10 - pGroup->nMinPage;
        34267  +  pcache1EnforceMaxPage(pGroup);
        34268  +  pcache1LeaveMutex(pGroup);
 34078  34269     sqlite3_free(pCache->apHash);
 34079  34270     sqlite3_free(pCache);
 34080  34271   }
 34081  34272   
 34082  34273   /*
 34083  34274   ** This function is called during initialization (sqlite3_initialize()) to
 34084  34275   ** install the default pluggable cache module, assuming the user has not
................................................................................
 34109  34300   **
 34110  34301   ** nReq is the number of bytes of memory required. Once this much has
 34111  34302   ** been released, the function returns. The return value is the total number 
 34112  34303   ** of bytes of memory released.
 34113  34304   */
 34114  34305   SQLITE_PRIVATE int sqlite3PcacheReleaseMemory(int nReq){
 34115  34306     int nFree = 0;
        34307  +  assert( sqlite3_mutex_notheld(pcache1.grp.mutex) );
        34308  +  assert( sqlite3_mutex_notheld(pcache1.mutex) );
 34116  34309     if( pcache1.pStart==0 ){
 34117  34310       PgHdr1 *p;
 34118         -    pcache1EnterMutex();
 34119         -    while( (nReq<0 || nFree<nReq) && ((p=pcache1.pLruTail)!=0) ){
        34311  +    pcache1EnterMutex(&pcache1.grp);
        34312  +    while( (nReq<0 || nFree<nReq) && ((p=pcache1.grp.pLruTail)!=0) ){
 34120  34313         nFree += pcache1MemSize(PGHDR1_TO_PAGE(p));
 34121  34314         pcache1PinPage(p);
 34122  34315         pcache1RemoveFromHash(p);
 34123  34316         pcache1FreePage(p);
 34124  34317       }
 34125         -    pcache1LeaveMutex();
        34318  +    pcache1LeaveMutex(&pcache1.grp);
 34126  34319     }
 34127  34320     return nFree;
 34128  34321   }
 34129  34322   #endif /* SQLITE_ENABLE_MEMORY_MANAGEMENT */
 34130  34323   
 34131  34324   #ifdef SQLITE_TEST
 34132  34325   /*
................................................................................
 34137  34330     int *pnCurrent,      /* OUT: Total number of pages cached */
 34138  34331     int *pnMax,          /* OUT: Global maximum cache size */
 34139  34332     int *pnMin,          /* OUT: Sum of PCache1.nMin for purgeable caches */
 34140  34333     int *pnRecyclable    /* OUT: Total number of pages available for recycling */
 34141  34334   ){
 34142  34335     PgHdr1 *p;
 34143  34336     int nRecyclable = 0;
 34144         -  for(p=pcache1.pLruHead; p; p=p->pLruNext){
        34337  +  for(p=pcache1.grp.pLruHead; p; p=p->pLruNext){
 34145  34338       nRecyclable++;
 34146  34339     }
 34147         -  *pnCurrent = pcache1.nCurrentPage;
 34148         -  *pnMax = pcache1.nMaxPage;
 34149         -  *pnMin = pcache1.nMinPage;
        34340  +  *pnCurrent = pcache1.grp.nCurrentPage;
        34341  +  *pnMax = pcache1.grp.nMaxPage;
        34342  +  *pnMin = pcache1.grp.nMinPage;
 34150  34343     *pnRecyclable = nRecyclable;
 34151  34344   }
 34152  34345   #endif
 34153  34346   
 34154  34347   /************** End of pcache1.c *********************************************/
 34155  34348   /************** Begin file rowset.c ******************************************/
 34156  34349   /*
................................................................................
 37439  37632     pPager->changeCountDone = pPager->tempFile;
 37440  37633   
 37441  37634     if( rc==SQLITE_OK ){
 37442  37635       zMaster = pPager->pTmpSpace;
 37443  37636       rc = readMasterJournal(pPager->jfd, zMaster, pPager->pVfs->mxPathname+1);
 37444  37637       testcase( rc!=SQLITE_OK );
 37445  37638     }
 37446         -  if( rc==SQLITE_OK && !pPager->noSync 
        37639  +  if( rc==SQLITE_OK
 37447  37640      && (pPager->eState>=PAGER_WRITER_DBMOD || pPager->eState==PAGER_OPEN)
 37448  37641     ){
 37449         -    rc = sqlite3OsSync(pPager->fd, pPager->syncFlags);
        37642  +    rc = sqlite3PagerSync(pPager);
 37450  37643     }
 37451  37644     if( rc==SQLITE_OK ){
 37452  37645       rc = pager_end_transaction(pPager, zMaster[0]!='\0');
 37453  37646       testcase( rc!=SQLITE_OK );
 37454  37647     }
 37455  37648     if( rc==SQLITE_OK && zMaster[0] && res ){
 37456  37649       /* If there was a master journal and this routine will return success,
................................................................................
 37605  37798       rc = pagerUndoCallback((void *)pPager, pList->pgno);
 37606  37799       pList = pNext;
 37607  37800     }
 37608  37801   
 37609  37802     return rc;
 37610  37803   }
 37611  37804   
        37805  +
        37806  +/*
        37807  +** Update the value of the change-counter at offsets 24 and 92 in
        37808  +** the header and the sqlite version number at offset 96.
        37809  +**
        37810  +** This is an unconditional update.  See also the pager_incr_changecounter()
        37811  +** routine which only updates the change-counter if the update is actually
        37812  +** needed, as determined by the pPager->changeCountDone state variable.
        37813  +*/
        37814  +static void pager_write_changecounter(PgHdr *pPg){
        37815  +  u32 change_counter;
        37816  +
        37817  +  /* Increment the value just read and write it back to byte 24. */
        37818  +  change_counter = sqlite3Get4byte((u8*)pPg->pPager->dbFileVers)+1;
        37819  +  put32bits(((char*)pPg->pData)+24, change_counter);
        37820  +
        37821  +  /* Also store the SQLite version number in bytes 96..99 and in
        37822  +  ** bytes 92..95 store the change counter for which the version number
        37823  +  ** is valid. */
        37824  +  put32bits(((char*)pPg->pData)+92, change_counter);
        37825  +  put32bits(((char*)pPg->pData)+96, SQLITE_VERSION_NUMBER);
        37826  +}
        37827  +
 37612  37828   /*
 37613  37829   ** This function is a wrapper around sqlite3WalFrames(). As well as logging
 37614  37830   ** the contents of the list of pages headed by pList (connected by pDirty),
 37615  37831   ** this function notifies any active backup processes that the pages have
 37616         -** changed. 
        37832  +** changed.
        37833  +**
        37834  +** The list of pages passed into this routine is always sorted by page number.
        37835  +** Hence, if page 1 appears anywhere on the list, it will be the first page.
 37617  37836   */ 
 37618  37837   static int pagerWalFrames(
 37619  37838     Pager *pPager,                  /* Pager object */
 37620  37839     PgHdr *pList,                   /* List of frames to log */
 37621  37840     Pgno nTruncate,                 /* Database size after this commit */
 37622  37841     int isCommit,                   /* True if this is a commit */
 37623  37842     int syncFlags                   /* Flags to pass to OsSync() (or 0) */
 37624  37843   ){
 37625  37844     int rc;                         /* Return code */
        37845  +#if defined(SQLITE_DEBUG) || defined(SQLITE_CHECK_PAGES)
        37846  +  PgHdr *p;                       /* For looping over pages */
        37847  +#endif
 37626  37848   
 37627  37849     assert( pPager->pWal );
        37850  +#ifdef SQLITE_DEBUG
        37851  +  /* Verify that the page list is in accending order */
        37852  +  for(p=pList; p && p->pDirty; p=p->pDirty){
        37853  +    assert( p->pgno < p->pDirty->pgno );
        37854  +  }
        37855  +#endif
        37856  +
        37857  +  if( pList->pgno==1 ) pager_write_changecounter(pList);
 37628  37858     rc = sqlite3WalFrames(pPager->pWal, 
 37629  37859         pPager->pageSize, pList, nTruncate, isCommit, syncFlags
 37630  37860     );
 37631  37861     if( rc==SQLITE_OK && pPager->pBackup ){
 37632  37862       PgHdr *p;
 37633  37863       for(p=pList; p; p=p->pDirty){
 37634  37864         sqlite3BackupUpdate(pPager->pBackup, p->pgno, (u8 *)p->pData);
 37635  37865       }
 37636  37866     }
 37637  37867   
 37638  37868   #ifdef SQLITE_CHECK_PAGES
 37639         -  {
 37640         -    PgHdr *p;
 37641         -    for(p=pList; p; p=p->pDirty) pager_set_pagehash(p);
        37869  +  for(p=pList; p; p=p->pDirty){
        37870  +    pager_set_pagehash(p);
 37642  37871     }
 37643  37872   #endif
 37644  37873   
 37645  37874     return rc;
 37646  37875   }
 37647  37876   
 37648  37877   /*
................................................................................
 38659  38888       ** set (set by sqlite3PagerDontWrite()).
 38660  38889       */
 38661  38890       if( pgno<=pPager->dbSize && 0==(pList->flags&PGHDR_DONT_WRITE) ){
 38662  38891         i64 offset = (pgno-1)*(i64)pPager->pageSize;   /* Offset to write */
 38663  38892         char *pData;                                   /* Data to write */    
 38664  38893   
 38665  38894         assert( (pList->flags&PGHDR_NEED_SYNC)==0 );
        38895  +      if( pList->pgno==1 ) pager_write_changecounter(pList);
 38666  38896   
 38667  38897         /* Encode the database */
 38668  38898         CODEC2(pPager, pList->pData, pgno, 6, return SQLITE_NOMEM, pData);
 38669  38899   
 38670  38900         /* Write out the page data. */
 38671  38901         rc = sqlite3OsWrite(pPager->fd, pData, pPager->pageSize, offset);
 38672  38902   
................................................................................
 40179  40409       pager_set_pagehash(pPg);
 40180  40410     }
 40181  40411   }
 40182  40412   
 40183  40413   /*
 40184  40414   ** This routine is called to increment the value of the database file 
 40185  40415   ** change-counter, stored as a 4-byte big-endian integer starting at 
 40186         -** byte offset 24 of the pager file.
        40416  +** byte offset 24 of the pager file.  The secondary change counter at
        40417  +** 92 is also updated, as is the SQLite version number at offset 96.
        40418  +**
        40419  +** But this only happens if the pPager->changeCountDone flag is false.
        40420  +** To avoid excess churning of page 1, the update only happens once.
        40421  +** See also the pager_write_changecounter() routine that does an 
        40422  +** unconditional update of the change counters.
 40187  40423   **
 40188  40424   ** If the isDirectMode flag is zero, then this is done by calling 
 40189  40425   ** sqlite3PagerWrite() on page 1, then modifying the contents of the
 40190  40426   ** page data. In this case the file will be updated when the current
 40191  40427   ** transaction is committed.
 40192  40428   **
 40193  40429   ** The isDirectMode flag may only be non-zero if the library was compiled
................................................................................
 40220  40456     UNUSED_PARAMETER(isDirectMode);
 40221  40457   #else
 40222  40458   # define DIRECT_MODE isDirectMode
 40223  40459   #endif
 40224  40460   
 40225  40461     if( !pPager->changeCountDone && pPager->dbSize>0 ){
 40226  40462       PgHdr *pPgHdr;                /* Reference to page 1 */
 40227         -    u32 change_counter;           /* Initial value of change-counter field */
 40228  40463   
 40229  40464       assert( !pPager->tempFile && isOpen(pPager->fd) );
 40230  40465   
 40231  40466       /* Open page 1 of the file for writing. */
 40232  40467       rc = sqlite3PagerGet(pPager, 1, &pPgHdr);
 40233  40468       assert( pPgHdr==0 || rc==SQLITE_OK );
 40234  40469   
................................................................................
 40238  40473       ** above is always successful - hence the ALWAYS on rc==SQLITE_OK.
 40239  40474       */
 40240  40475       if( !DIRECT_MODE && ALWAYS(rc==SQLITE_OK) ){
 40241  40476         rc = sqlite3PagerWrite(pPgHdr);
 40242  40477       }
 40243  40478   
 40244  40479       if( rc==SQLITE_OK ){
 40245         -      /* Increment the value just read and write it back to byte 24. */
 40246         -      change_counter = sqlite3Get4byte((u8*)pPager->dbFileVers);
 40247         -      change_counter++;
 40248         -      put32bits(((char*)pPgHdr->pData)+24, change_counter);
 40249         -
 40250         -      /* Also store the SQLite version number in bytes 96..99 and in
 40251         -      ** bytes 92..95 store the change counter for which the version number
 40252         -      ** is valid. */
 40253         -      put32bits(((char*)pPgHdr->pData)+92, change_counter);
 40254         -      put32bits(((char*)pPgHdr->pData)+96, SQLITE_VERSION_NUMBER);
        40480  +      /* Actually do the update of the change counter */
        40481  +      pager_write_changecounter(pPgHdr);
 40255  40482   
 40256  40483         /* If running in direct mode, write the contents of page 1 to the file. */
 40257  40484         if( DIRECT_MODE ){
 40258  40485           const void *zBuf;
 40259  40486           assert( pPager->dbFileSize>0 );
 40260  40487           CODEC2(pPager, pPgHdr->pData, 1, 6, rc=SQLITE_NOMEM, zBuf);
 40261  40488           if( rc==SQLITE_OK ){
................................................................................
 40279  40506   ** Sync the database file to disk. This is a no-op for in-memory databases
 40280  40507   ** or pages with the Pager.noSync flag set.
 40281  40508   **
 40282  40509   ** If successful, or if called on a pager for which it is a no-op, this
 40283  40510   ** function returns SQLITE_OK. Otherwise, an IO error code is returned.
 40284  40511   */
 40285  40512   SQLITE_PRIVATE int sqlite3PagerSync(Pager *pPager){
 40286         -  int rc;                              /* Return code */
 40287         -  assert( !MEMDB );
 40288         -  if( pPager->noSync ){
 40289         -    rc = SQLITE_OK;
 40290         -  }else{
        40513  +  int rc = SQLITE_OK;
        40514  +  if( !pPager->noSync ){
        40515  +    assert( !MEMDB );
 40291  40516       rc = sqlite3OsSync(pPager->fd, pPager->syncFlags);
        40517  +  }else if( isOpen(pPager->fd) ){
        40518  +    assert( !MEMDB );
        40519  +    sqlite3OsFileControl(pPager->fd, SQLITE_FCNTL_SYNC_OMITTED, (void *)&rc);
 40292  40520     }
 40293  40521     return rc;
 40294  40522   }
 40295  40523   
 40296  40524   /*
 40297  40525   ** This function may only be called while a write-transaction is active in
 40298  40526   ** rollback. If the connection is in WAL mode, this call is a no-op. 
................................................................................
 40503  40731           Pgno nNew = pPager->dbSize - (pPager->dbSize==PAGER_MJ_PGNO(pPager));
 40504  40732           assert( pPager->eState==PAGER_WRITER_DBMOD );
 40505  40733           rc = pager_truncate(pPager, nNew);
 40506  40734           if( rc!=SQLITE_OK ) goto commit_phase_one_exit;
 40507  40735         }
 40508  40736     
 40509  40737         /* Finally, sync the database file. */
 40510         -      if( !pPager->noSync && !noSync ){
 40511         -        rc = sqlite3OsSync(pPager->fd, pPager->syncFlags);
        40738  +      if( !noSync ){
        40739  +        rc = sqlite3PagerSync(pPager);
 40512  40740         }
 40513  40741         IOTRACE(("DBSYNC %p\n", pPager))
 40514  40742       }
 40515  40743     }
 40516  40744   
 40517  40745   commit_phase_one_exit:
 40518  40746     if( rc==SQLITE_OK && !pagerUseWal(pPager) ){
................................................................................
 40616  40844   
 40617  40845     if( pagerUseWal(pPager) ){
 40618  40846       int rc2;
 40619  40847       rc = sqlite3PagerSavepoint(pPager, SAVEPOINT_ROLLBACK, -1);
 40620  40848       rc2 = pager_end_transaction(pPager, pPager->setMaster);
 40621  40849       if( rc==SQLITE_OK ) rc = rc2;
 40622  40850     }else if( !isOpen(pPager->jfd) || pPager->eState==PAGER_WRITER_LOCKED ){
        40851  +    int eState = pPager->eState;
 40623  40852       rc = pager_end_transaction(pPager, 0);
        40853  +    if( !MEMDB && eState>PAGER_WRITER_LOCKED ){
        40854  +      /* This can happen using journal_mode=off. Move the pager to the error 
        40855  +      ** state to indicate that the contents of the cache may not be trusted.
        40856  +      ** Any active readers will get SQLITE_ABORT.
        40857  +      */
        40858  +      pPager->errCode = SQLITE_ABORT;
        40859  +      pPager->eState = PAGER_ERROR;
        40860  +      return rc;
        40861  +    }
 40624  40862     }else{
 40625  40863       rc = pager_playback(pPager, 0);
 40626  40864     }
 40627  40865   
 40628  40866     assert( pPager->eState==PAGER_READER || rc!=SQLITE_OK );
 40629  40867     assert( rc==SQLITE_OK || rc==SQLITE_FULL || (rc&0xFF)==SQLITE_IOERR );
 40630  40868   
................................................................................
 41886  42124   **   walIteratorNext() - Step an iterator,
 41887  42125   **   walIteratorFree() - Free an iterator.
 41888  42126   **
 41889  42127   ** This functionality is used by the checkpoint code (see walCheckpoint()).
 41890  42128   */
 41891  42129   struct WalIterator {
 41892  42130     int iPrior;                     /* Last result returned from the iterator */
 41893         -  int nSegment;                   /* Size of the aSegment[] array */
        42131  +  int nSegment;                   /* Number of entries in aSegment[] */
 41894  42132     struct WalSegment {
 41895  42133       int iNext;                    /* Next slot in aIndex[] not yet returned */
 41896  42134       ht_slot *aIndex;              /* i0, i1, i2... such that aPgno[iN] ascend */
 41897  42135       u32 *aPgno;                   /* Array of page numbers. */
 41898         -    int nEntry;                   /* Max size of aPgno[] and aIndex[] arrays */
        42136  +    int nEntry;                   /* Nr. of entries in aPgno[] and aIndex[] */
 41899  42137       int iZero;                    /* Frame number associated with aPgno[0] */
 41900         -  } aSegment[1];                  /* One for every 32KB page in the WAL */
        42138  +  } aSegment[1];                  /* One for every 32KB page in the wal-index */
 41901  42139   };
 41902  42140   
 41903  42141   /*
 41904  42142   ** Define the parameters of the hash tables in the wal-index file. There
 41905  42143   ** is a hash-table following every HASHTABLE_NPAGE page numbers in the
 41906  42144   ** wal-index.
 41907  42145   **
................................................................................
 42757  42995   
 42758  42996     *piPage = p->iPrior = iRet;
 42759  42997     return (iRet==0xFFFFFFFF);
 42760  42998   }
 42761  42999   
 42762  43000   /*
 42763  43001   ** This function merges two sorted lists into a single sorted list.
        43002  +**
        43003  +** aLeft[] and aRight[] are arrays of indices.  The sort key is
        43004  +** aContent[aLeft[]] and aContent[aRight[]].  Upon entry, the following
        43005  +** is guaranteed for all J<K:
        43006  +**
        43007  +**        aContent[aLeft[J]] < aContent[aLeft[K]]
        43008  +**        aContent[aRight[J]] < aContent[aRight[K]]
        43009  +**
        43010  +** This routine overwrites aRight[] with a new (probably longer) sequence
        43011  +** of indices such that the aRight[] contains every index that appears in
        43012  +** either aLeft[] or the old aRight[] and such that the second condition
        43013  +** above is still met.
        43014  +**
        43015  +** The aContent[aLeft[X]] values will be unique for all X.  And the
        43016  +** aContent[aRight[X]] values will be unique too.  But there might be
        43017  +** one or more combinations of X and Y such that
        43018  +**
        43019  +**      aLeft[X]!=aRight[Y]  &&  aContent[aLeft[X]] == aContent[aRight[Y]]
        43020  +**
        43021  +** When that happens, omit the aLeft[X] and use the aRight[Y] index.
 42764  43022   */
 42765  43023   static void walMerge(
 42766         -  u32 *aContent,                  /* Pages in wal */
        43024  +  const u32 *aContent,            /* Pages in wal - keys for the sort */
 42767  43025     ht_slot *aLeft,                 /* IN: Left hand input list */
 42768  43026     int nLeft,                      /* IN: Elements in array *paLeft */
 42769  43027     ht_slot **paRight,              /* IN/OUT: Right hand input list */
 42770  43028     int *pnRight,                   /* IN/OUT: Elements in *paRight */
 42771  43029     ht_slot *aTmp                   /* Temporary buffer */
 42772  43030   ){
 42773  43031     int iLeft = 0;                  /* Current index in aLeft */
................................................................................
 42799  43057   
 42800  43058     *paRight = aLeft;
 42801  43059     *pnRight = iOut;
 42802  43060     memcpy(aLeft, aTmp, sizeof(aTmp[0])*iOut);
 42803  43061   }
 42804  43062   
 42805  43063   /*
 42806         -** Sort the elements in list aList, removing any duplicates.
        43064  +** Sort the elements in list aList using aContent[] as the sort key.
        43065  +** Remove elements with duplicate keys, preferring to keep the
        43066  +** larger aList[] values.
        43067  +**
        43068  +** The aList[] entries are indices into aContent[].  The values in
        43069  +** aList[] are to be sorted so that for all J<K:
        43070  +**
        43071  +**      aContent[aList[J]] < aContent[aList[K]]
        43072  +**
        43073  +** For any X and Y such that
        43074  +**
        43075  +**      aContent[aList[X]] == aContent[aList[Y]]
        43076  +**
        43077  +** Keep the larger of the two values aList[X] and aList[Y] and discard
        43078  +** the smaller.
 42807  43079   */
 42808  43080   static void walMergesort(
 42809         -  u32 *aContent,                  /* Pages in wal */
        43081  +  const u32 *aContent,            /* Pages in wal */
 42810  43082     ht_slot *aBuffer,               /* Buffer of at least *pnList items to use */
 42811  43083     ht_slot *aList,                 /* IN/OUT: List to sort */
 42812  43084     int *pnList                     /* IN/OUT: Number of elements in aList[] */
 42813  43085   ){
 42814  43086     struct Sublist {
 42815  43087       int nList;                    /* Number of elements in aList */
 42816  43088       ht_slot *aList;               /* Pointer to sub-list content */
................................................................................
 42867  43139   static void walIteratorFree(WalIterator *p){
 42868  43140     sqlite3ScratchFree(p);
 42869  43141   }
 42870  43142   
 42871  43143   /*
 42872  43144   ** Construct a WalInterator object that can be used to loop over all 
 42873  43145   ** pages in the WAL in ascending order. The caller must hold the checkpoint
        43146  +** lock.
 42874  43147   **
 42875  43148   ** On success, make *pp point to the newly allocated WalInterator object
 42876  43149   ** return SQLITE_OK. Otherwise, return an error code. If this routine
 42877  43150   ** returns an error, the value of *pp is undefined.
 42878  43151   **
 42879  43152   ** The calling routine should invoke walIteratorFree() to destroy the
 42880  43153   ** WalIterator object when it has finished with it.
................................................................................
 43001  43274     u32 mxPage;                     /* Max database page to write */
 43002  43275     int i;                          /* Loop counter */
 43003  43276     volatile WalCkptInfo *pInfo;    /* The checkpoint status information */
 43004  43277   
 43005  43278     szPage = (pWal->hdr.szPage&0xfe00) + ((pWal->hdr.szPage&0x0001)<<16);
 43006  43279     testcase( szPage<=32768 );
 43007  43280     testcase( szPage>=65536 );
 43008         -  if( pWal->hdr.mxFrame==0 ) return SQLITE_OK;
        43281  +  pInfo = walCkptInfo(pWal);
        43282  +  if( pInfo->nBackfill>=pWal->hdr.mxFrame ) return SQLITE_OK;
 43009  43283   
 43010  43284     /* Allocate the iterator */
 43011  43285     rc = walIteratorInit(pWal, &pIter);
 43012  43286     if( rc!=SQLITE_OK ){
 43013  43287       return rc;
 43014  43288     }
 43015  43289     assert( pIter );
................................................................................
 43023  43297     /* Compute in mxSafeFrame the index of the last frame of the WAL that is
 43024  43298     ** safe to write into the database.  Frames beyond mxSafeFrame might
 43025  43299     ** overwrite database pages that are in use by active readers and thus
 43026  43300     ** cannot be backfilled from the WAL.
 43027  43301     */
 43028  43302     mxSafeFrame = pWal->hdr.mxFrame;
 43029  43303     mxPage = pWal->hdr.nPage;
 43030         -  pInfo = walCkptInfo(pWal);
 43031  43304     for(i=1; i<WAL_NREADER; i++){
 43032  43305       u32 y = pInfo->aReadMark[i];
 43033  43306       if( mxSafeFrame>=y ){
 43034  43307         assert( y<=pWal->hdr.mxFrame );
 43035  43308         rc = walLockExclusive(pWal, WAL_READ_LOCK(i), 1);
 43036  43309         if( rc==SQLITE_OK ){
 43037  43310           pInfo->aReadMark[i] = READMARK_NOT_USED;
................................................................................
 44613  44886     u8 secureDelete;      /* True if secure_delete is enabled */
 44614  44887     u8 initiallyEmpty;    /* Database is empty at start of transaction */
 44615  44888     u8 openFlags;         /* Flags to sqlite3BtreeOpen() */
 44616  44889   #ifndef SQLITE_OMIT_AUTOVACUUM
 44617  44890     u8 autoVacuum;        /* True if auto-vacuum is enabled */
 44618  44891     u8 incrVacuum;        /* True if incr-vacuum is enabled */
 44619  44892   #endif
        44893  +  u8 inTransaction;     /* Transaction state */
        44894  +  u8 doNotUseWAL;       /* If true, do not open write-ahead-log file */
 44620  44895     u16 maxLocal;         /* Maximum local payload in non-LEAFDATA tables */
 44621  44896     u16 minLocal;         /* Minimum local payload in non-LEAFDATA tables */
 44622  44897     u16 maxLeaf;          /* Maximum local payload in a LEAFDATA table */
 44623  44898     u16 minLeaf;          /* Minimum local payload in a LEAFDATA table */
 44624         -  u8 inTransaction;     /* Transaction state */
 44625         -  u8 doNotUseWAL;       /* If true, do not open write-ahead-log file */
 44626  44899     u32 pageSize;         /* Total number of bytes on a page */
 44627  44900     u32 usableSize;       /* Number of usable bytes on each page */
 44628  44901     int nTransaction;     /* Number of open transactions (read + write) */
 44629  44902     u32 nPage;            /* Number of pages in the database */
 44630  44903     void *pSchema;        /* Pointer to space allocated by sqlite3BtreeSchema() */
 44631  44904     void (*xFreeSchema)(void*);  /* Destructor for BtShared.pSchema */
 44632  44905     sqlite3_mutex *mutex; /* Non-recursive mutex required to access this struct */
................................................................................
 44645  44918   /*
 44646  44919   ** An instance of the following structure is used to hold information
 44647  44920   ** about a cell.  The parseCellPtr() function fills in this structure
 44648  44921   ** based on information extract from the raw disk page.
 44649  44922   */
 44650  44923   typedef struct CellInfo CellInfo;
 44651  44924   struct CellInfo {
 44652         -  u8 *pCell;     /* Pointer to the start of cell content */
 44653  44925     i64 nKey;      /* The key for INTKEY tables, or number of bytes in key */
        44926  +  u8 *pCell;     /* Pointer to the start of cell content */
 44654  44927     u32 nData;     /* Number of bytes of data */
 44655  44928     u32 nPayload;  /* Total amount of payload */
 44656  44929     u16 nHeader;   /* Size of the cell content header in bytes */
 44657  44930     u16 nLocal;    /* Amount of payload held locally */
 44658  44931     u16 iOverflow; /* Offset to overflow page number.  Zero if no overflow */
 44659  44932     u16 nSize;     /* Size of the cell content on the main b-tree page */
 44660  44933   };
................................................................................
 44688  44961     Btree *pBtree;            /* The Btree to which this cursor belongs */
 44689  44962     BtShared *pBt;            /* The BtShared this cursor points to */
 44690  44963     BtCursor *pNext, *pPrev;  /* Forms a linked list of all cursors */
 44691  44964     struct KeyInfo *pKeyInfo; /* Argument passed to comparison function */
 44692  44965     Pgno pgnoRoot;            /* The root page of this tree */
 44693  44966     sqlite3_int64 cachedRowid; /* Next rowid cache.  0 means not valid */
 44694  44967     CellInfo info;            /* A parse of the cell we are pointing at */
        44968  +  i64 nKey;        /* Size of pKey, or last integer key */
        44969  +  void *pKey;      /* Saved key that was cursor's last known position */
        44970  +  int skipNext;    /* Prev() is noop if negative. Next() is noop if positive */
 44695  44971     u8 wrFlag;                /* True if writable */
 44696  44972     u8 atLast;                /* Cursor pointing to the last entry */
 44697  44973     u8 validNKey;             /* True if info.nKey is valid */
 44698  44974     u8 eState;                /* One of the CURSOR_XXX constants (see below) */
 44699         -  void *pKey;      /* Saved key that was cursor's last known position */
 44700         -  i64 nKey;        /* Size of pKey, or last integer key */
 44701         -  int skipNext;    /* Prev() is noop if negative. Next() is noop if positive */
 44702  44975   #ifndef SQLITE_OMIT_INCRBLOB
 44703         -  u8 isIncrblobHandle;      /* True if this cursor is an incr. io handle */
 44704  44976     Pgno *aOverflow;          /* Cache of overflow page locations */
        44977  +  u8 isIncrblobHandle;      /* True if this cursor is an incr. io handle */
 44705  44978   #endif
 44706  44979     i16 iPage;                            /* Index of current page in apPage */
 44707         -  MemPage *apPage[BTCURSOR_MAX_DEPTH];  /* Pages from root to current page */
 44708  44980     u16 aiIdx[BTCURSOR_MAX_DEPTH];        /* Current index in apPage[i] */
        44981  +  MemPage *apPage[BTCURSOR_MAX_DEPTH];  /* Pages from root to current page */
 44709  44982   };
 44710  44983   
 44711  44984   /*
 44712  44985   ** Potential values for BtCursor.eState.
 44713  44986   **
 44714  44987   ** CURSOR_VALID:
 44715  44988   **   Cursor points to a valid entry. getPayload() etc. may be called.
................................................................................
 46105  46378     pInfo->nHeader = n;
 46106  46379     testcase( nPayload==pPage->maxLocal );
 46107  46380     testcase( nPayload==pPage->maxLocal+1 );
 46108  46381     if( likely(nPayload<=pPage->maxLocal) ){
 46109  46382       /* This is the (easy) common case where the entire payload fits
 46110  46383       ** on the local page.  No overflow is required.
 46111  46384       */
 46112         -    int nSize;          /* Total size of cell content in bytes */
 46113         -    nSize = nPayload + n;
        46385  +    if( (pInfo->nSize = (u16)(n+nPayload))<4 ) pInfo->nSize = 4;
 46114  46386       pInfo->nLocal = (u16)nPayload;
 46115  46387       pInfo->iOverflow = 0;
 46116         -    if( (nSize & ~3)==0 ){
 46117         -      nSize = 4;        /* Minimum cell size is 4 */
 46118         -    }
 46119         -    pInfo->nSize = (u16)nSize;
 46120  46388     }else{
 46121  46389       /* If the payload will not fit completely on the local page, we have
 46122  46390       ** to decide how much to store locally and how much to spill onto
 46123  46391       ** overflow pages.  The strategy is to minimize the amount of unused
 46124  46392       ** space on overflow pages while keeping the amount of local storage
 46125  46393       ** in between minLocal and maxLocal.
 46126  46394       **
................................................................................
 47573  47841         pBt->usableSize = usableSize;
 47574  47842         pBt->pageSize = pageSize;
 47575  47843         freeTempSpace(pBt);
 47576  47844         rc = sqlite3PagerSetPagesize(pBt->pPager, &pBt->pageSize,
 47577  47845                                      pageSize-usableSize);
 47578  47846         return rc;
 47579  47847       }
 47580         -    if( nPageHeader>nPageFile ){
        47848  +    if( (pBt->db->flags & SQLITE_RecoveryMode)==0 && nPageHeader>nPageFile ){
 47581  47849         rc = SQLITE_CORRUPT_BKPT;
 47582  47850         goto page1_init_failed;
 47583  47851       }
 47584  47852       if( usableSize<480 ){
 47585  47853         goto page1_init_failed;
 47586  47854       }
 47587  47855       pBt->pageSize = pageSize;
................................................................................
 53444  53712     if( i<0 ){
 53445  53713       sqlite3Error(pErrorDb, SQLITE_ERROR, "unknown database %s", zDb);
 53446  53714       return 0;
 53447  53715     }
 53448  53716   
 53449  53717     return pDb->aDb[i].pBt;
 53450  53718   }
        53719  +
        53720  +/*
        53721  +** Attempt to set the page size of the destination to match the page size
        53722  +** of the source.
        53723  +*/
        53724  +static int setDestPgsz(sqlite3_backup *p){
        53725  +  int rc;
        53726  +  rc = sqlite3BtreeSetPageSize(p->pDest,sqlite3BtreeGetPageSize(p->pSrc),-1,0);
        53727  +  return rc;
        53728  +}
 53451  53729   
 53452  53730   /*
 53453  53731   ** Create an sqlite3_backup process to copy the contents of zSrcDb from
 53454  53732   ** connection handle pSrcDb to zDestDb in pDestDb. If successful, return
 53455  53733   ** a pointer to the new sqlite3_backup object.
 53456  53734   **
 53457  53735   ** If an error occurs, NULL is returned and an error code and error message
................................................................................
 53498  53776       p->pSrc = findBtree(pDestDb, pSrcDb, zSrcDb);
 53499  53777       p->pDest = findBtree(pDestDb, pDestDb, zDestDb);
 53500  53778       p->pDestDb = pDestDb;
 53501  53779       p->pSrcDb = pSrcDb;
 53502  53780       p->iNext = 1;
 53503  53781       p->isAttached = 0;
 53504  53782   
 53505         -    if( 0==p->pSrc || 0==p->pDest ){
 53506         -      /* One (or both) of the named databases did not exist. An error has
 53507         -      ** already been written into the pDestDb handle. All that is left
 53508         -      ** to do here is free the sqlite3_backup structure.
        53783  +    if( 0==p->pSrc || 0==p->pDest || setDestPgsz(p)==SQLITE_NOMEM ){
        53784  +      /* One (or both) of the named databases did not exist or an OOM
        53785  +      ** error was hit.  The error has already been written into the
        53786  +      ** pDestDb handle.  All that is left to do here is free the
        53787  +      ** sqlite3_backup structure.
 53509  53788         */
 53510  53789         sqlite3_free(p);
 53511  53790         p = 0;
 53512  53791       }
 53513  53792     }
 53514  53793     if( p ){
 53515  53794       p->pSrc->nBackup++;
................................................................................
 53758  54037           **
 53759  54038           **   * Data stored on the pages immediately following the 
 53760  54039           **     pending-byte page in the source database may need to be
 53761  54040           **     copied into the destination database.
 53762  54041           */
 53763  54042           const i64 iSize = (i64)pgszSrc * (i64)nSrcPage;
 53764  54043           sqlite3_file * const pFile = sqlite3PagerFile(pDestPager);
        54044  +        i64 iOff;
        54045  +        i64 iEnd;
 53765  54046   
 53766  54047           assert( pFile );
 53767  54048           assert( (i64)nDestTruncate*(i64)pgszDest >= iSize || (
 53768  54049                 nDestTruncate==(int)(PENDING_BYTE_PAGE(p->pDest->pBt)-1)
 53769  54050              && iSize>=PENDING_BYTE && iSize<=PENDING_BYTE+pgszDest
 53770  54051           ));
 53771         -        if( SQLITE_OK==(rc = sqlite3PagerCommitPhaseOne(pDestPager, 0, 1))
 53772         -         && SQLITE_OK==(rc = backupTruncateFile(pFile, iSize))
 53773         -         && SQLITE_OK==(rc = sqlite3PagerSync(pDestPager))
        54052  +
        54053  +        /* This call ensures that all data required to recreate the original
        54054  +        ** database has been stored in the journal for pDestPager and the
        54055  +        ** journal synced to disk. So at this point we may safely modify
        54056  +        ** the database file in any way, knowing that if a power failure
        54057  +        ** occurs, the original database will be reconstructed from the 
        54058  +        ** journal file.  */
        54059  +        rc = sqlite3PagerCommitPhaseOne(pDestPager, 0, 1);
        54060  +
        54061  +        /* Write the extra pages and truncate the database file as required. */
        54062  +        iEnd = MIN(PENDING_BYTE + pgszDest, iSize);
        54063  +        for(
        54064  +          iOff=PENDING_BYTE+pgszSrc; 
        54065  +          rc==SQLITE_OK && iOff<iEnd; 
        54066  +          iOff+=pgszSrc
 53774  54067           ){
 53775         -          i64 iOff;
 53776         -          i64 iEnd = MIN(PENDING_BYTE + pgszDest, iSize);
 53777         -          for(
 53778         -            iOff=PENDING_BYTE+pgszSrc; 
 53779         -            rc==SQLITE_OK && iOff<iEnd; 
 53780         -            iOff+=pgszSrc
 53781         -          ){
 53782         -            PgHdr *pSrcPg = 0;
 53783         -            const Pgno iSrcPg = (Pgno)((iOff/pgszSrc)+1);
 53784         -            rc = sqlite3PagerGet(pSrcPager, iSrcPg, &pSrcPg);
 53785         -            if( rc==SQLITE_OK ){
 53786         -              u8 *zData = sqlite3PagerGetData(pSrcPg);
 53787         -              rc = sqlite3OsWrite(pFile, zData, pgszSrc, iOff);
 53788         -            }
 53789         -            sqlite3PagerUnref(pSrcPg);
        54068  +          PgHdr *pSrcPg = 0;
        54069  +          const Pgno iSrcPg = (Pgno)((iOff/pgszSrc)+1);
        54070  +          rc = sqlite3PagerGet(pSrcPager, iSrcPg, &pSrcPg);
        54071  +          if( rc==SQLITE_OK ){
        54072  +            u8 *zData = sqlite3PagerGetData(pSrcPg);
        54073  +            rc = sqlite3OsWrite(pFile, zData, pgszSrc, iOff);
 53790  54074             }
        54075  +          sqlite3PagerUnref(pSrcPg);
        54076  +        }
        54077  +        if( rc==SQLITE_OK ){
        54078  +          rc = backupTruncateFile(pFile, iSize);
        54079  +        }
        54080  +
        54081  +        /* Sync the database file to disk. */
        54082  +        if( rc==SQLITE_OK ){
        54083  +          rc = sqlite3PagerSync(pDestPager);
 53791  54084           }
 53792  54085         }else{
 53793  54086           rc = sqlite3PagerCommitPhaseOne(pDestPager, 0, 0);
 53794  54087         }
 53795  54088     
 53796  54089         /* Finish committing the transaction to the destination database. */
 53797  54090         if( SQLITE_OK==rc
................................................................................
 55546  55839     p->readOnly = 1;
 55547  55840     for(pOp=p->aOp, i=p->nOp-1; i>=0; i--, pOp++){
 55548  55841       u8 opcode = pOp->opcode;
 55549  55842   
 55550  55843       pOp->opflags = sqlite3OpcodeProperty[opcode];
 55551  55844       if( opcode==OP_Function || opcode==OP_AggStep ){
 55552  55845         if( pOp->p5>nMaxArgs ) nMaxArgs = pOp->p5;
 55553         -    }else if( opcode==OP_Transaction && pOp->p2!=0 ){
        55846  +    }else if( (opcode==OP_Transaction && pOp->p2!=0) || opcode==OP_Vacuum ){
 55554  55847         p->readOnly = 0;
 55555  55848   #ifndef SQLITE_OMIT_VIRTUALTABLE
 55556  55849       }else if( opcode==OP_VUpdate ){
 55557  55850         if( pOp->p2>nMaxArgs ) nMaxArgs = pOp->p2;
 55558  55851       }else if( opcode==OP_VFilter ){
 55559  55852         int n;
 55560  55853         assert( p->nOp - i >= 3 );
................................................................................
 58643  58936   static int sqlite3Step(Vdbe *p){
 58644  58937     sqlite3 *db;
 58645  58938     int rc;
 58646  58939   
 58647  58940     assert(p);
 58648  58941     if( p->magic!=VDBE_MAGIC_RUN ){
 58649  58942       /* We used to require that sqlite3_reset() be called before retrying
 58650         -    ** sqlite3_step() after any error.  But after 3.6.23, we changed this
 58651         -    ** so that sqlite3_reset() would be called automatically instead of
 58652         -    ** throwing the error.
        58943  +    ** sqlite3_step() after any error or after SQLITE_DONE.  But beginning
        58944  +    ** with version 3.7.0, we changed this so that sqlite3_reset() would
        58945  +    ** be called automatically instead of throwing the SQLITE_MISUSE error.
        58946  +    ** This "automatic-reset" change is not technically an incompatibility, 
        58947  +    ** since any application that receives an SQLITE_MISUSE is broken by
        58948  +    ** definition.
        58949  +    **
        58950  +    ** Nevertheless, some published applications that were originally written
        58951  +    ** for version 3.6.23 or earlier do in fact depend on SQLITE_MISUSE 
        58952  +    ** returns, and the so were broken by the automatic-reset change.  As a
        58953  +    ** a work-around, the SQLITE_OMIT_AUTORESET compile-time restores the
        58954  +    ** legacy behavior of returning SQLITE_MISUSE for cases where the 
        58955  +    ** previous sqlite3_step() returned something other than a SQLITE_LOCKED
        58956  +    ** or SQLITE_BUSY error.
 58653  58957       */
        58958  +#ifdef SQLITE_OMIT_AUTORESET
        58959  +    if( p->rc==SQLITE_BUSY || p->rc==SQLITE_LOCKED ){
        58960  +      sqlite3_reset((sqlite3_stmt*)p);
        58961  +    }else{
        58962  +      return SQLITE_MISUSE_BKPT;
        58963  +    }
        58964  +#else
 58654  58965       sqlite3_reset((sqlite3_stmt*)p);
        58966  +#endif
 58655  58967     }
 58656  58968   
 58657  58969     /* Check that malloc() has not failed. If it has, return early. */
 58658  58970     db = p->db;
 58659  58971     if( db->mallocFailed ){
 58660  58972       p->rc = SQLITE_NOMEM;
 58661  58973       return SQLITE_NOMEM;
................................................................................
 58689  59001     }
 58690  59002   #ifndef SQLITE_OMIT_EXPLAIN
 58691  59003     if( p->explain ){
 58692  59004       rc = sqlite3VdbeList(p);
 58693  59005     }else
 58694  59006   #endif /* SQLITE_OMIT_EXPLAIN */
 58695  59007     {
        59008  +    db->vdbeExecCnt++;
 58696  59009       rc = sqlite3VdbeExec(p);
        59010  +    db->vdbeExecCnt--;
 58697  59011     }
 58698  59012   
 58699  59013   #ifndef SQLITE_OMIT_TRACE
 58700  59014     /* Invoke the profile callback if there is one
 58701  59015     */
 58702  59016     if( rc!=SQLITE_ROW && db->xProfile && !db->init.busy && p->zSql ){
 58703  59017       sqlite3_int64 iNow;
................................................................................
 58983  59297       ** this assert() from failing, when building with SQLITE_DEBUG defined
 58984  59298       ** using gcc, force nullMem to be 8-byte aligned using the magical
 58985  59299       ** __attribute__((aligned(8))) macro.  */
 58986  59300       static const Mem nullMem 
 58987  59301   #if defined(SQLITE_DEBUG) && defined(__GNUC__)
 58988  59302         __attribute__((aligned(8))) 
 58989  59303   #endif
 58990         -      = {{0}, (double)0, 0, "", 0, MEM_Null, SQLITE_NULL, 0, 0, 0 };
        59304  +      = {0, "", (double)0, {0}, 0, MEM_Null, SQLITE_NULL, 0, 0, 0 };
 58991  59305   
 58992  59306       if( pVm && ALWAYS(pVm->db) ){
 58993  59307         sqlite3_mutex_enter(pVm->db->mutex);
 58994  59308         sqlite3Error(pVm->db, SQLITE_RANGE, 0);
 58995  59309       }
 58996  59310       pOut = (Mem*)&nullMem;
 58997  59311     }
................................................................................
 59646  59960       nTotal += n;
 59647  59961       zSql += n;
 59648  59962     }
 59649  59963     return nTotal;
 59650  59964   }
 59651  59965   
 59652  59966   /*
 59653         -** Return a pointer to a string in memory obtained form sqlite3DbMalloc() which
 59654         -** holds a copy of zRawSql but with host parameters expanded to their
 59655         -** current bindings.
        59967  +** This function returns a pointer to a nul-terminated string in memory
        59968  +** obtained from sqlite3DbMalloc(). If sqlite3.vdbeExecCnt is 1, then the
        59969  +** string contains a copy of zRawSql but with host parameters expanded to 
        59970  +** their current bindings. Or, if sqlite3.vdbeExecCnt is greater than 1, 
        59971  +** then the returned string holds a copy of zRawSql with "-- " prepended
        59972  +** to each line of text.
 59656  59973   **
 59657  59974   ** The calling function is responsible for making sure the memory returned
 59658  59975   ** is eventually freed.
 59659  59976   **
 59660  59977   ** ALGORITHM:  Scan the input string looking for host parameters in any of
 59661  59978   ** these forms:  ?, ?N, $A, @A, :A.  Take care to avoid text within
 59662  59979   ** string literals, quoted identifier names, and comments.  For text forms,
................................................................................
 59679  59996     StrAccum out;            /* Accumulate the output here */
 59680  59997     char zBase[100];         /* Initial working space */
 59681  59998   
 59682  59999     db = p->db;
 59683  60000     sqlite3StrAccumInit(&out, zBase, sizeof(zBase), 
 59684  60001                         db->aLimit[SQLITE_LIMIT_LENGTH]);
 59685  60002     out.db = db;
 59686         -  while( zRawSql[0] ){
 59687         -    n = findNextHostParameter(zRawSql, &nToken);
 59688         -    assert( n>0 );
 59689         -    sqlite3StrAccumAppend(&out, zRawSql, n);
 59690         -    zRawSql += n;
 59691         -    assert( zRawSql[0] || nToken==0 );
 59692         -    if( nToken==0 ) break;
 59693         -    if( zRawSql[0]=='?' ){
 59694         -      if( nToken>1 ){
 59695         -        assert( sqlite3Isdigit(zRawSql[1]) );
 59696         -        sqlite3GetInt32(&zRawSql[1], &idx);
 59697         -      }else{
 59698         -        idx = nextIndex;
 59699         -      }
 59700         -    }else{
 59701         -      assert( zRawSql[0]==':' || zRawSql[0]=='$' || zRawSql[0]=='@' );
 59702         -      testcase( zRawSql[0]==':' );
 59703         -      testcase( zRawSql[0]=='$' );
 59704         -      testcase( zRawSql[0]=='@' );
 59705         -      idx = sqlite3VdbeParameterIndex(p, zRawSql, nToken);
 59706         -      assert( idx>0 );
 59707         -    }
 59708         -    zRawSql += nToken;
 59709         -    nextIndex = idx + 1;
 59710         -    assert( idx>0 && idx<=p->nVar );
 59711         -    pVar = &p->aVar[idx-1];
 59712         -    if( pVar->flags & MEM_Null ){
 59713         -      sqlite3StrAccumAppend(&out, "NULL", 4);
 59714         -    }else if( pVar->flags & MEM_Int ){
 59715         -      sqlite3XPrintf(&out, "%lld", pVar->u.i);
 59716         -    }else if( pVar->flags & MEM_Real ){
 59717         -      sqlite3XPrintf(&out, "%!.15g", pVar->r);
 59718         -    }else if( pVar->flags & MEM_Str ){
 59719         -#ifndef SQLITE_OMIT_UTF16
 59720         -      u8 enc = ENC(db);
 59721         -      if( enc!=SQLITE_UTF8 ){
 59722         -        Mem utf8;
 59723         -        memset(&utf8, 0, sizeof(utf8));
 59724         -        utf8.db = db;
 59725         -        sqlite3VdbeMemSetStr(&utf8, pVar->z, pVar->n, enc, SQLITE_STATIC);
 59726         -        sqlite3VdbeChangeEncoding(&utf8, SQLITE_UTF8);
 59727         -        sqlite3XPrintf(&out, "'%.*q'", utf8.n, utf8.z);
 59728         -        sqlite3VdbeMemRelease(&utf8);
 59729         -      }else
 59730         -#endif
 59731         -      {
 59732         -        sqlite3XPrintf(&out, "'%.*q'", pVar->n, pVar->z);
 59733         -      }
 59734         -    }else if( pVar->flags & MEM_Zero ){
 59735         -      sqlite3XPrintf(&out, "zeroblob(%d)", pVar->u.nZero);
 59736         -    }else{
 59737         -      assert( pVar->flags & MEM_Blob );
 59738         -      sqlite3StrAccumAppend(&out, "x'", 2);
 59739         -      for(i=0; i<pVar->n; i++){
 59740         -        sqlite3XPrintf(&out, "%02x", pVar->z[i]&0xff);
 59741         -      }
 59742         -      sqlite3StrAccumAppend(&out, "'", 1);
        60003  +  if( db->vdbeExecCnt>1 ){
        60004  +    while( *zRawSql ){
        60005  +      const char *zStart = zRawSql;
        60006  +      while( *(zRawSql++)!='\n' && *zRawSql );
        60007  +      sqlite3StrAccumAppend(&out, "-- ", 3);
        60008  +      sqlite3StrAccumAppend(&out, zStart, zRawSql-zStart);
        60009  +    }
        60010  +  }else{
        60011  +    while( zRawSql[0] ){
        60012  +      n = findNextHostParameter(zRawSql, &nToken);
        60013  +      assert( n>0 );
        60014  +      sqlite3StrAccumAppend(&out, zRawSql, n);
        60015  +      zRawSql += n;
        60016  +      assert( zRawSql[0] || nToken==0 );
        60017  +      if( nToken==0 ) break;
        60018  +      if( zRawSql[0]=='?' ){
        60019  +        if( nToken>1 ){
        60020  +          assert( sqlite3Isdigit(zRawSql[1]) );
        60021  +          sqlite3GetInt32(&zRawSql[1], &idx);
        60022  +        }else{
        60023  +          idx = nextIndex;
        60024  +        }
        60025  +      }else{
        60026  +        assert( zRawSql[0]==':' || zRawSql[0]=='$' || zRawSql[0]=='@' );
        60027  +        testcase( zRawSql[0]==':' );
        60028  +        testcase( zRawSql[0]=='$' );
        60029  +        testcase( zRawSql[0]=='@' );
        60030  +        idx = sqlite3VdbeParameterIndex(p, zRawSql, nToken);
        60031  +        assert( idx>0 );
        60032  +      }
        60033  +      zRawSql += nToken;
        60034  +      nextIndex = idx + 1;
        60035  +      assert( idx>0 && idx<=p->nVar );
        60036  +      pVar = &p->aVar[idx-1];
        60037  +      if( pVar->flags & MEM_Null ){
        60038  +        sqlite3StrAccumAppend(&out, "NULL", 4);
        60039  +      }else if( pVar->flags & MEM_Int ){
        60040  +        sqlite3XPrintf(&out, "%lld", pVar->u.i);
        60041  +      }else if( pVar->flags & MEM_Real ){
        60042  +        sqlite3XPrintf(&out, "%!.15g", pVar->r);
        60043  +      }else if( pVar->flags & MEM_Str ){
        60044  +#ifndef SQLITE_OMIT_UTF16
        60045  +        u8 enc = ENC(db);
        60046  +        if( enc!=SQLITE_UTF8 ){
        60047  +          Mem utf8;
        60048  +          memset(&utf8, 0, sizeof(utf8));
        60049  +          utf8.db = db;
        60050  +          sqlite3VdbeMemSetStr(&utf8, pVar->z, pVar->n, enc, SQLITE_STATIC);
        60051  +          sqlite3VdbeChangeEncoding(&utf8, SQLITE_UTF8);
        60052  +          sqlite3XPrintf(&out, "'%.*q'", utf8.n, utf8.z);
        60053  +          sqlite3VdbeMemRelease(&utf8);
        60054  +        }else
        60055  +#endif
        60056  +        {
        60057  +          sqlite3XPrintf(&out, "'%.*q'", pVar->n, pVar->z);
        60058  +        }
        60059  +      }else if( pVar->flags & MEM_Zero ){
        60060  +        sqlite3XPrintf(&out, "zeroblob(%d)", pVar->u.nZero);
        60061  +      }else{
        60062  +        assert( pVar->flags & MEM_Blob );
        60063  +        sqlite3StrAccumAppend(&out, "x'", 2);
        60064  +        for(i=0; i<pVar->n; i++){
        60065  +          sqlite3XPrintf(&out, "%02x", pVar->z[i]&0xff);
        60066  +        }
        60067  +        sqlite3StrAccumAppend(&out, "'", 1);
        60068  +      }
 59743  60069       }
 59744  60070     }
 59745  60071     return sqlite3StrAccumFinish(&out);
 59746  60072   }
 59747  60073   
 59748  60074   #endif /* #ifndef SQLITE_OMIT_TRACE */
 59749  60075   
................................................................................
 73430  73756       ** the index b-tree.  */
 73431  73757       endOfLoop = sqlite3VdbeMakeLabel(v);
 73432  73758       sqlite3VdbeAddOp2(v, OP_Rewind, iIdxCur, endOfLoop);
 73433  73759       topOfLoop = sqlite3VdbeCurrentAddr(v);
 73434  73760       sqlite3VdbeAddOp2(v, OP_AddImm, iMem, 1);
 73435  73761   
 73436  73762       for(i=0; i<nCol; i++){
        73763  +      CollSeq *pColl;
 73437  73764         sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regCol);
 73438         -#ifdef SQLITE_ENABLE_STAT2
 73439  73765         if( i==0 ){
        73766  +#ifdef SQLITE_ENABLE_STAT2
 73440  73767           /* Check if the record that cursor iIdxCur points to contains a
 73441  73768           ** value that should be stored in the sqlite_stat2 table. If so,
 73442  73769           ** store it.  */
 73443  73770           int ne = sqlite3VdbeAddOp3(v, OP_Ne, regRecno, 0, regSamplerecno);
 73444  73771           assert( regTabname+1==regIdxname 
 73445  73772                && regTabname+2==regSampleno
 73446  73773                && regTabname+3==regCol
................................................................................
 73461  73788           sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_INDEX_SAMPLES, regTemp2);
 73462  73789           sqlite3VdbeAddOp3(v, OP_Subtract, regSampleno, regTemp2, regTemp2);
 73463  73790           sqlite3VdbeAddOp3(v, OP_Divide, regTemp2, regTemp, regTemp);
 73464  73791           sqlite3VdbeAddOp3(v, OP_Add, regSamplerecno, regTemp, regSamplerecno);
 73465  73792   
 73466  73793           sqlite3VdbeJumpHere(v, ne);
 73467  73794           sqlite3VdbeAddOp2(v, OP_AddImm, regRecno, 1);
 73468         -      }
 73469  73795   #endif
 73470  73796   
 73471         -      sqlite3VdbeAddOp3(v, OP_Ne, regCol, 0, iMem+nCol+i+1);
 73472         -      /**** TODO:  add collating sequence *****/
 73473         -      sqlite3VdbeChangeP5(v, SQLITE_JUMPIFNULL);
        73797  +        /* Always record the very first row */
        73798  +        sqlite3VdbeAddOp1(v, OP_IfNot, iMem+1);
        73799  +      }
        73800  +      assert( pIdx->azColl!=0 );
        73801  +      assert( pIdx->azColl[i]!=0 );
        73802  +      pColl = sqlite3LocateCollSeq(pParse, pIdx->azColl[i]);
        73803  +      sqlite3VdbeAddOp4(v, OP_Ne, regCol, 0, iMem+nCol+i+1,
        73804  +                       (char*)pColl, P4_COLLSEQ);
        73805  +      sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
 73474  73806       }
 73475  73807       if( db->mallocFailed ){
 73476  73808         /* If a malloc failure has occurred, then the result of the expression 
 73477  73809         ** passed as the second argument to the call to sqlite3VdbeJumpHere() 
 73478  73810         ** below may be negative. Which causes an assert() to fail (or an
 73479  73811         ** out-of-bounds write if SQLITE_DEBUG is not defined).  */
 73480  73812         return;
 73481  73813       }
 73482  73814       sqlite3VdbeAddOp2(v, OP_Goto, 0, endOfLoop);
 73483  73815       for(i=0; i<nCol; i++){
 73484         -      sqlite3VdbeJumpHere(v, sqlite3VdbeCurrentAddr(v)-(nCol*2));
        73816  +      int addr2 = sqlite3VdbeCurrentAddr(v) - (nCol*2);
        73817  +      if( i==0 ){
        73818  +        sqlite3VdbeJumpHere(v, addr2-1);  /* Set jump dest for the OP_IfNot */
        73819  +      }
        73820  +      sqlite3VdbeJumpHere(v, addr2);      /* Set jump dest for the OP_Ne */
 73485  73821         sqlite3VdbeAddOp2(v, OP_AddImm, iMem+i+1, 1);
 73486  73822         sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, iMem+nCol+i+1);
 73487  73823       }
 73488  73824   
 73489  73825       /* End of the analysis loop. */
 73490  73826       sqlite3VdbeResolveLabel(v, endOfLoop);
 73491  73827       sqlite3VdbeAddOp2(v, OP_Next, iIdxCur, topOfLoop);
................................................................................
 73823  74159       }else{
 73824  74160         rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
 73825  74161         sqlite3DbFree(db, zSql);
 73826  74162       }
 73827  74163   
 73828  74164       if( rc==SQLITE_OK ){
 73829  74165         while( sqlite3_step(pStmt)==SQLITE_ROW ){
 73830         -        char *zIndex = (char *)sqlite3_column_text(pStmt, 0);
 73831         -        Index *pIdx = sqlite3FindIndex(db, zIndex, sInfo.zDatabase);
        74166  +        char *zIndex;   /* Index name */
        74167  +        Index *pIdx;    /* Pointer to the index object */
        74168  +
        74169  +        zIndex = (char *)sqlite3_column_text(pStmt, 0);
        74170  +        pIdx = zIndex ? sqlite3FindIndex(db, zIndex, sInfo.zDatabase) : 0;
 73832  74171           if( pIdx ){
 73833  74172             int iSample = sqlite3_column_int(pStmt, 1);
 73834  74173             if( iSample<SQLITE_INDEX_SAMPLES && iSample>=0 ){
 73835  74174               int eType = sqlite3_column_type(pStmt, 2);
 73836  74175   
 73837  74176               if( pIdx->aSample==0 ){
 73838  74177                 static const int sz = sizeof(IndexSample)*SQLITE_INDEX_SAMPLES;
................................................................................
 80911  81250   SQLITE_PRIVATE void sqlite3RegisterLikeFunctions(sqlite3 *db, int caseSensitive){
 80912  81251     struct compareInfo *pInfo;
 80913  81252     if( caseSensitive ){
 80914  81253       pInfo = (struct compareInfo*)&likeInfoAlt;
 80915  81254     }else{
 80916  81255       pInfo = (struct compareInfo*)&likeInfoNorm;
 80917  81256     }
 80918         -  sqlite3CreateFunc(db, "like", 2, SQLITE_ANY, pInfo, likeFunc, 0, 0, 0);
 80919         -  sqlite3CreateFunc(db, "like", 3, SQLITE_ANY, pInfo, likeFunc, 0, 0, 0);
 80920         -  sqlite3CreateFunc(db, "glob", 2, SQLITE_ANY, 
        81257  +  sqlite3CreateFunc(db, "like", 2, SQLITE_UTF8, pInfo, likeFunc, 0, 0, 0);
        81258  +  sqlite3CreateFunc(db, "like", 3, SQLITE_UTF8, pInfo, likeFunc, 0, 0, 0);
        81259  +  sqlite3CreateFunc(db, "glob", 2, SQLITE_UTF8, 
 80921  81260         (struct compareInfo*)&globInfo, likeFunc, 0, 0, 0);
 80922  81261     setLikeOptFlag(db, "glob", SQLITE_FUNC_LIKE | SQLITE_FUNC_CASE);
 80923  81262     setLikeOptFlag(db, "like", 
 80924  81263         caseSensitive ? (SQLITE_FUNC_LIKE | SQLITE_FUNC_CASE) : SQLITE_FUNC_LIKE);
 80925  81264   }
 80926  81265   
 80927  81266   /*
................................................................................
 87592  87931   
 87593  87932   #ifndef SQLITE_OMIT_UTF16
 87594  87933   /*
 87595  87934   ** Compile the UTF-16 encoded SQL statement zSql into a statement handle.
 87596  87935   */
 87597  87936   static int sqlite3Prepare16(
 87598  87937     sqlite3 *db,              /* Database handle. */ 
 87599         -  const void *zSql,         /* UTF-8 encoded SQL statement. */
        87938  +  const void *zSql,         /* UTF-16 encoded SQL statement. */
 87600  87939     int nBytes,               /* Length of zSql in bytes. */
 87601  87940     int saveSqlFlag,          /* True to save SQL text into the sqlite3_stmt */
 87602  87941     sqlite3_stmt **ppStmt,    /* OUT: A pointer to the prepared statement */
 87603  87942     const void **pzTail       /* OUT: End of parsed string */
 87604  87943   ){
 87605  87944     /* This function currently works by first transforming the UTF-16
 87606  87945     ** encoded string to UTF-8, then invoking sqlite3_prepare(). The
................................................................................
 87642  87981   ** and so if a schema change occurs, SQLITE_SCHEMA is returned by
 87643  87982   ** sqlite3_step().  In the new version, the original SQL text is retained
 87644  87983   ** and the statement is automatically recompiled if an schema change
 87645  87984   ** occurs.
 87646  87985   */
 87647  87986   SQLITE_API int sqlite3_prepare16(
 87648  87987     sqlite3 *db,              /* Database handle. */ 
 87649         -  const void *zSql,         /* UTF-8 encoded SQL statement. */
        87988  +  const void *zSql,         /* UTF-16 encoded SQL statement. */
 87650  87989     int nBytes,               /* Length of zSql in bytes. */
 87651  87990     sqlite3_stmt **ppStmt,    /* OUT: A pointer to the prepared statement */
 87652  87991     const void **pzTail       /* OUT: End of parsed string */
 87653  87992   ){
 87654  87993     int rc;
 87655  87994     rc = sqlite3Prepare16(db,zSql,nBytes,0,ppStmt,pzTail);
 87656  87995     assert( rc==SQLITE_OK || ppStmt==0 || *ppStmt==0 );  /* VERIFY: F13021 */
 87657  87996     return rc;
 87658  87997   }
 87659  87998   SQLITE_API int sqlite3_prepare16_v2(
 87660  87999     sqlite3 *db,              /* Database handle. */ 
 87661         -  const void *zSql,         /* UTF-8 encoded SQL statement. */
        88000  +  const void *zSql,         /* UTF-16 encoded SQL statement. */
 87662  88001     int nBytes,               /* Length of zSql in bytes. */
 87663  88002     sqlite3_stmt **ppStmt,    /* OUT: A pointer to the prepared statement */
 87664  88003     const void **pzTail       /* OUT: End of parsed string */
 87665  88004   ){
 87666  88005     int rc;
 87667  88006     rc = sqlite3Prepare16(db,zSql,nBytes,1,ppStmt,pzTail);
 87668  88007     assert( rc==SQLITE_OK || ppStmt==0 || *ppStmt==0 );  /* VERIFY: F13021 */
................................................................................
 97663  98002   #ifdef SQLITE_ENABLE_STAT2
 97664  98003   static int valueFromExpr(
 97665  98004     Parse *pParse, 
 97666  98005     Expr *pExpr, 
 97667  98006     u8 aff, 
 97668  98007     sqlite3_value **pp
 97669  98008   ){
 97670         -  /* The evalConstExpr() function will have already converted any TK_VARIABLE
 97671         -  ** expression involved in an comparison into a TK_REGISTER. */
 97672         -  assert( pExpr->op!=TK_VARIABLE );
 97673         -  if( pExpr->op==TK_REGISTER && pExpr->op2==TK_VARIABLE ){
        98009  +  if( pExpr->op==TK_VARIABLE
        98010  +   || (pExpr->op==TK_REGISTER && pExpr->op2==TK_VARIABLE)
        98011  +  ){
 97674  98012       int iVar = pExpr->iColumn;
 97675  98013       sqlite3VdbeSetVarmask(pParse->pVdbe, iVar); /* IMP: R-23257-02778 */
 97676  98014       *pp = sqlite3VdbeGetValue(pParse->pReprepare, iVar, aff);
 97677  98015       return SQLITE_OK;
 97678  98016     }
 97679  98017     return sqlite3ValueFromExpr(pParse->db, pExpr, SQLITE_UTF8, aff, pp);
 97680  98018   }
................................................................................
106280 106618   #endif
106281 106619   #ifdef SQLITE_ENABLE_LOAD_EXTENSION
106282 106620                    | SQLITE_LoadExtension
106283 106621   #endif
106284 106622   #if SQLITE_DEFAULT_RECURSIVE_TRIGGERS
106285 106623                    | SQLITE_RecTriggers
106286 106624   #endif
       106625  +#if defined(SQLITE_DEFAULT_FOREIGN_KEYS) && SQLITE_DEFAULT_FOREIGN_KEYS
       106626  +                 | SQLITE_ForeignKeys
       106627  +#endif
106287 106628         ;
106288 106629     sqlite3HashInit(&db->aCollSeq);
106289 106630   #ifndef SQLITE_OMIT_VIRTUALTABLE
106290 106631     sqlite3HashInit(&db->aModule);
106291 106632   #endif
106292 106633   
106293 106634     db->pVfs = sqlite3_vfs_find(zVfs);
................................................................................
111117 111458     Fts3Expr *pExpr,                /* Access this expressions doclist */
111118 111459     sqlite3_int64 iDocid,           /* Docid associated with requested pos-list */
111119 111460     int iCol                        /* Column of requested pos-list */
111120 111461   ){
111121 111462     assert( pExpr->isLoaded );
111122 111463     if( pExpr->aDoclist ){
111123 111464       char *pEnd = &pExpr->aDoclist[pExpr->nDoclist];
111124         -    char *pCsr = pExpr->pCurrent;
       111465  +    char *pCsr;
111125 111466   
       111467  +    if( pExpr->pCurrent==0 ){
       111468  +      pExpr->pCurrent = pExpr->aDoclist;
       111469  +      pExpr->iCurrent = 0;
       111470  +      pExpr->pCurrent += sqlite3Fts3GetVarint(pExpr->pCurrent,&pExpr->iCurrent);
       111471  +    }
       111472  +    pCsr = pExpr->pCurrent;
111126 111473       assert( pCsr );
       111474  +
111127 111475       while( pCsr<pEnd ){
111128 111476         if( pExpr->iCurrent<iDocid ){
111129 111477           fts3PoslistCopy(0, &pCsr);
111130 111478           if( pCsr<pEnd ){
111131 111479             fts3GetDeltaVarint(&pCsr, &pExpr->iCurrent);
111132 111480           }
111133 111481           pExpr->pCurrent = pCsr;
................................................................................
115308 115656         ** varints, where nCol is the number of columns in the FTS3 table.
115309 115657         ** The first varint is the number of documents currently stored in
115310 115658         ** the table. The following nCol varints contain the total amount of
115311 115659         ** data stored in all rows of each column of the table, from left
115312 115660         ** to right.
115313 115661         */
115314 115662         sqlite3_stmt *pStmt;
115315         -      rc = fts3SqlStmt(p, SQL_SELECT_DOCTOTAL, &pStmt, 0);
       115663  +      sqlite3_int64 nDoc = 0;
       115664  +      sqlite3_int64 nByte = 0;
       115665  +      const char *a;
       115666  +      rc = sqlite3Fts3SelectDoctotal(p, &pStmt);
115316 115667         if( rc ) return rc;
115317         -      if( sqlite3_step(pStmt)==SQLITE_ROW ){
115318         -        sqlite3_int64 nDoc = 0;
115319         -        sqlite3_int64 nByte = 0;
115320         -        const char *a = sqlite3_column_blob(pStmt, 0);
115321         -        if( a ){
115322         -          const char *pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
115323         -          a += sqlite3Fts3GetVarint(a, &nDoc);
115324         -          while( a<pEnd ){
115325         -            a += sqlite3Fts3GetVarint(a, &nByte);
115326         -          }
       115668  +      a = sqlite3_column_blob(pStmt, 0);
       115669  +      if( a ){
       115670  +        const char *pEnd = &a[sqlite3_column_bytes(pStmt, 0)];
       115671  +        a += sqlite3Fts3GetVarint(a, &nDoc);
       115672  +        while( a<pEnd ){
       115673  +          a += sqlite3Fts3GetVarint(a, &nByte);
115327 115674           }
       115675  +      }
       115676  +      if( nDoc==0 || nByte==0 ){
       115677  +        sqlite3_reset(pStmt);
       115678  +        return SQLITE_CORRUPT;
       115679  +      }
115328 115680   
115329         -        pCsr->nRowAvg = (int)(((nByte / nDoc) + pgsz - 1) / pgsz);
115330         -      }
       115681  +      pCsr->nRowAvg = (int)(((nByte / nDoc) + pgsz) / pgsz);
       115682  +      assert( pCsr->nRowAvg>0 ); 
115331 115683         rc = sqlite3_reset(pStmt);
115332         -      if( rc!=SQLITE_OK || pCsr->nRowAvg==0 ) return rc;
       115684  +      if( rc!=SQLITE_OK ) return rc;
115333 115685       }
115334 115686   
115335 115687       /* Assume that a blob flows over onto overflow pages if it is larger
115336 115688       ** than (pgsz-35) bytes in size (the file-format documentation
115337 115689       ** confirms this).
115338 115690       */
115339 115691       for(iBlock=pReader->iStartBlock; iBlock<=pReader->iLeafEndBlock; iBlock++){
................................................................................
116446 116798             fts3ColumnFilter(pFilter->iCol, &pList, &nList);
116447 116799           }
116448 116800   
116449 116801           if( !isIgnoreEmpty || nList>0 ){
116450 116802             nByte = sqlite3Fts3VarintLen(iDocid-iPrev) + (isRequirePos?nList+1:0);
116451 116803             if( nDoclist+nByte>nAlloc ){
116452 116804               char *aNew;
116453         -            nAlloc = nDoclist+nByte*2;
       116805  +            nAlloc = (nDoclist+nByte)*2;
116454 116806               aNew = sqlite3_realloc(aBuffer, nAlloc);
116455 116807               if( !aNew ){
116456 116808                 rc = SQLITE_NOMEM;
116457 116809                 goto finished;
116458 116810               }
116459 116811               aBuffer = aNew;
116460 116812             }
................................................................................
117323 117675   }
117324 117676   
117325 117677   /*
117326 117678   ** This is an fts3ExprIterate() callback used while loading the doclists
117327 117679   ** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
117328 117680   ** fts3ExprLoadDoclists().
117329 117681   */
117330         -static int fts3ExprLoadDoclistsCb1(Fts3Expr *pExpr, int iPhrase, void *ctx){
       117682  +static int fts3ExprLoadDoclistsCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
117331 117683     int rc = SQLITE_OK;
117332 117684     LoadDoclistCtx *p = (LoadDoclistCtx *)ctx;
117333 117685   
117334 117686     UNUSED_PARAMETER(iPhrase);
117335 117687   
117336 117688     p->nPhrase++;
117337 117689     p->nToken += pExpr->pPhrase->nToken;
................................................................................
117343 117695         rc = fts3ExprNearTrim(pExpr);
117344 117696       }
117345 117697     }
117346 117698   
117347 117699     return rc;
117348 117700   }
117349 117701   
117350         -/*
117351         -** This is an fts3ExprIterate() callback used while loading the doclists
117352         -** for each phrase into Fts3Expr.aDoclist[]/nDoclist. See also
117353         -** fts3ExprLoadDoclists().
117354         -*/
117355         -static int fts3ExprLoadDoclistsCb2(Fts3Expr *pExpr, int iPhrase, void *ctx){
117356         -  UNUSED_PARAMETER(iPhrase);
117357         -  UNUSED_PARAMETER(ctx);
117358         -  if( pExpr->aDoclist ){
117359         -    pExpr->pCurrent = pExpr->aDoclist;
117360         -    pExpr->iCurrent = 0;
117361         -    pExpr->pCurrent += sqlite3Fts3GetVarint(pExpr->pCurrent, &pExpr->iCurrent);
117362         -  }
117363         -  return SQLITE_OK;
117364         -}
117365         -
117366 117702   /*
117367 117703   ** Load the doclists for each phrase in the query associated with FTS3 cursor
117368 117704   ** pCsr. 
117369 117705   **
117370 117706   ** If pnPhrase is not NULL, then *pnPhrase is set to the number of matchable 
117371 117707   ** phrases in the expression (all phrases except those directly or 
117372 117708   ** indirectly descended from the right-hand-side of a NOT operator). If 
................................................................................
117377 117713     Fts3Cursor *pCsr,               /* Fts3 cursor for current query */
117378 117714     int *pnPhrase,                  /* OUT: Number of phrases in query */
117379 117715     int *pnToken                    /* OUT: Number of tokens in query */
117380 117716   ){
117381 117717     int rc;                         /* Return Code */
117382 117718     LoadDoclistCtx sCtx = {0,0,0};  /* Context for fts3ExprIterate() */
117383 117719     sCtx.pCsr = pCsr;
117384         -  rc = fts3ExprIterate(pCsr->pExpr, fts3ExprLoadDoclistsCb1, (void *)&sCtx);
117385         -  if( rc==SQLITE_OK ){
117386         -    (void)fts3ExprIterate(pCsr->pExpr, fts3ExprLoadDoclistsCb2, 0);
117387         -  }
       117720  +  rc = fts3ExprIterate(pCsr->pExpr, fts3ExprLoadDoclistsCb, (void *)&sCtx);
117388 117721     if( pnPhrase ) *pnPhrase = sCtx.nPhrase;
117389 117722     if( pnToken ) *pnToken = sCtx.nToken;
117390 117723     return rc;
117391 117724   }
117392 117725   
117393 117726   static int fts3ExprPhraseCountCb(Fts3Expr *pExpr, int iPhrase, void *ctx){
117394 117727     (*(int *)ctx)++;
................................................................................
118072 118405     sqlite3_int64 nDoc;
118073 118406   
118074 118407     if( !*ppStmt ){
118075 118408       int rc = sqlite3Fts3SelectDoctotal(pTab, ppStmt);
118076 118409       if( rc!=SQLITE_OK ) return rc;
118077 118410     }
118078 118411     pStmt = *ppStmt;
       118412  +  assert( sqlite3_data_count(pStmt)==1 );
118079 118413   
118080 118414     a = sqlite3_column_blob(pStmt, 0);
118081 118415     a += sqlite3Fts3GetVarint(a, &nDoc);
118082 118416     *pnDoc = (u32)nDoc;
118083 118417   
118084 118418     if( paLen ) *paLen = a;
118085 118419     return SQLITE_OK;

Changes to src/sqlite3.h.

   103    103   ** string contains the date and time of the check-in (UTC) and an SHA1
   104    104   ** hash of the entire source tree.
   105    105   **
   106    106   ** See also: [sqlite3_libversion()],
   107    107   ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
   108    108   ** [sqlite_version()] and [sqlite_source_id()].
   109    109   */
   110         -#define SQLITE_VERSION        "3.7.4"
   111         -#define SQLITE_VERSION_NUMBER 3007004
   112         -#define SQLITE_SOURCE_ID      "2010-12-06 21:09:59 fabcb6b95e1d4059d1e6c6183f65846f6cbd5749"
          110  +#define SQLITE_VERSION        "3.7.5"
          111  +#define SQLITE_VERSION_NUMBER 3007005
          112  +#define SQLITE_SOURCE_ID      "2011-01-25 18:30:51 c17703ec1e604934f8bd5b1f66f34b19d17a6d1f"
   113    113   
   114    114   /*
   115    115   ** CAPI3REF: Run-Time Library Version Numbers
   116    116   ** KEYWORDS: sqlite3_version, sqlite3_sourceid
   117    117   **
   118    118   ** These interfaces provide the same information as the [SQLITE_VERSION],
   119    119   ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros
................................................................................
   711    711   ** The [SQLITE_FCNTL_CHUNK_SIZE] opcode is used to request that the VFS
   712    712   ** extends and truncates the database file in chunks of a size specified
   713    713   ** by the user. The fourth argument to [sqlite3_file_control()] should 
   714    714   ** point to an integer (type int) containing the new chunk-size to use
   715    715   ** for the nominated database. Allocating database file space in large
   716    716   ** chunks (say 1MB at a time), may reduce file-system fragmentation and
   717    717   ** improve performance on some systems.
          718  +**
          719  +** The [SQLITE_FCNTL_FILE_POINTER] opcode is used to obtain a pointer
          720  +** to the [sqlite3_file] object associated with a particular database
          721  +** connection.  See the [sqlite3_file_control()] documentation for
          722  +** additional information.
          723  +**
          724  +** ^(The [SQLITE_FCNTL_SYNC_OMITTED] opcode is generated internally by
          725  +** SQLite and sent to all VFSes in place of a call to the xSync method
          726  +** when the database connection has [PRAGMA synchronous] set to OFF.)^
          727  +** Some specialized VFSes need this signal in order to operate correctly
          728  +** when [PRAGMA synchronous | PRAGMA synchronous=OFF] is set, but most 
          729  +** VFSes do not need this signal and should silently ignore this opcode.
          730  +** Applications should not call [sqlite3_file_control()] with this
          731  +** opcode as doing so may disrupt the operation of the specilized VFSes
          732  +** that do require it.  
   718    733   */
   719    734   #define SQLITE_FCNTL_LOCKSTATE        1
   720    735   #define SQLITE_GET_LOCKPROXYFILE      2
   721    736   #define SQLITE_SET_LOCKPROXYFILE      3
   722    737   #define SQLITE_LAST_ERRNO             4
   723    738   #define SQLITE_FCNTL_SIZE_HINT        5
   724    739   #define SQLITE_FCNTL_CHUNK_SIZE       6
   725    740   #define SQLITE_FCNTL_FILE_POINTER     7
          741  +#define SQLITE_FCNTL_SYNC_OMITTED     8
   726    742   
   727    743   
   728    744   /*
   729    745   ** CAPI3REF: Mutex Handle
   730    746   **
   731    747   ** The mutex module within SQLite defines [sqlite3_mutex] to be an
   732    748   ** abstract type for a mutex object.  The SQLite core never looks
................................................................................
  1838   1854   ** ^The sqlite3_mprintf() and sqlite3_vmprintf() routines write their
  1839   1855   ** results into memory obtained from [sqlite3_malloc()].
  1840   1856   ** The strings returned by these two routines should be
  1841   1857   ** released by [sqlite3_free()].  ^Both routines return a
  1842   1858   ** NULL pointer if [sqlite3_malloc()] is unable to allocate enough
  1843   1859   ** memory to hold the resulting string.
  1844   1860   **
  1845         -** ^(In sqlite3_snprintf() routine is similar to "snprintf()" from
         1861  +** ^(The sqlite3_snprintf() routine is similar to "snprintf()" from
  1846   1862   ** the standard C library.  The result is written into the
  1847   1863   ** buffer supplied as the second parameter whose size is given by
  1848   1864   ** the first parameter. Note that the order of the
  1849   1865   ** first two parameters is reversed from snprintf().)^  This is an
  1850   1866   ** historical accident that cannot be fixed without breaking
  1851   1867   ** backwards compatibility.  ^(Note also that sqlite3_snprintf()
  1852   1868   ** returns a pointer to its buffer instead of the number of
................................................................................
  1856   1872   ** now without breaking compatibility.
  1857   1873   **
  1858   1874   ** ^As long as the buffer size is greater than zero, sqlite3_snprintf()
  1859   1875   ** guarantees that the buffer is always zero-terminated.  ^The first
  1860   1876   ** parameter "n" is the total size of the buffer, including space for
  1861   1877   ** the zero terminator.  So the longest string that can be completely
  1862   1878   ** written will be n-1 characters.
         1879  +**
         1880  +** ^The sqlite3_vsnprintf() routine is a varargs version of sqlite3_snprintf().
  1863   1881   **
  1864   1882   ** These routines all implement some additional formatting
  1865   1883   ** options that are useful for constructing SQL statements.
  1866   1884   ** All of the usual printf() formatting options apply.  In addition, there
  1867   1885   ** is are "%q", "%Q", and "%z" options.
  1868   1886   **
  1869   1887   ** ^(The %q option works like %s in that it substitutes a null-terminated
................................................................................
  1920   1938   ** ^(The "%z" formatting option works like "%s" but with the
  1921   1939   ** addition that after the string has been read and copied into
  1922   1940   ** the result, [sqlite3_free()] is called on the input string.)^
  1923   1941   */
  1924   1942   SQLITE_API char *sqlite3_mprintf(const char*,...);
  1925   1943   SQLITE_API char *sqlite3_vmprintf(const char*, va_list);
  1926   1944   SQLITE_API char *sqlite3_snprintf(int,char*,const char*, ...);
         1945  +SQLITE_API char *sqlite3_vsnprintf(int,char*,const char*, va_list);
  1927   1946   
  1928   1947   /*
  1929   1948   ** CAPI3REF: Memory Allocation Subsystem
  1930   1949   **
  1931   1950   ** The SQLite core uses these three routines for all of its own
  1932   1951   ** internal memory allocation needs. "Core" in the previous sentence
  1933   1952   ** does not include operating-system specific VFS implementation.  The
................................................................................
  2297   2316   **
  2298   2317   ** ^(<dt>[SQLITE_OPEN_READWRITE]</dt>
  2299   2318   ** <dd>The database is opened for reading and writing if possible, or reading
  2300   2319   ** only if the file is write protected by the operating system.  In either
  2301   2320   ** case the database must already exist, otherwise an error is returned.</dd>)^
  2302   2321   **
  2303   2322   ** ^(<dt>[SQLITE_OPEN_READWRITE] | [SQLITE_OPEN_CREATE]</dt>
  2304         -** <dd>The database is opened for reading and writing, and is creates it if
         2323  +** <dd>The database is opened for reading and writing, and is created if
  2305   2324   ** it does not already exist. This is the behavior that is always used for
  2306   2325   ** sqlite3_open() and sqlite3_open16().</dd>)^
  2307   2326   ** </dl>
  2308   2327   **
  2309   2328   ** If the 3rd parameter to sqlite3_open_v2() is not one of the
  2310   2329   ** combinations shown above or one of the combinations shown above combined
  2311   2330   ** with the [SQLITE_OPEN_NOMUTEX], [SQLITE_OPEN_FULLMUTEX],
................................................................................
  2646   2665   ** compiled using either [sqlite3_prepare_v2()] or [sqlite3_prepare16_v2()].
  2647   2666   */
  2648   2667   SQLITE_API const char *sqlite3_sql(sqlite3_stmt *pStmt);
  2649   2668   
  2650   2669   /*
  2651   2670   ** CAPI3REF: Determine If An SQL Statement Writes The Database
  2652   2671   **
  2653         -** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if
  2654         -** the [prepared statement] X is [SELECT] statement and false (zero) if
  2655         -** X is an [INSERT], [UPDATE], [DELETE], CREATE, DROP, [ANALYZE],
  2656         -** [ALTER], or [REINDEX] statement.
  2657         -** If X is a NULL pointer or any other kind of statement, including but
  2658         -** not limited to [ATTACH], [DETACH], [COMMIT], [ROLLBACK], [RELEASE],
  2659         -** [SAVEPOINT], [PRAGMA], or [VACUUM] the result of sqlite3_stmt_readonly(X) is
  2660         -** undefined.
         2672  +** ^The sqlite3_stmt_readonly(X) interface returns true (non-zero) if 
         2673  +** and only if the [prepared statement] X makes no direct changes to
         2674  +** the content of the database file.
         2675  +**
         2676  +** Note that [application-defined SQL functions] or
         2677  +** [virtual tables] might change the database indirectly as a side effect.  
         2678  +** ^(For example, if an application defines a function "eval()" that 
         2679  +** calls [sqlite3_exec()], then the following SQL statement would
         2680  +** change the database file through side-effects:
         2681  +**
         2682  +** <blockquote><pre>
         2683  +**    SELECT eval('DELETE FROM t1') FROM t2;
         2684  +** </pre></blockquote>
         2685  +**
         2686  +** But because the [SELECT] statement does not change the database file
         2687  +** directly, sqlite3_stmt_readonly() would still return true.)^
         2688  +**
         2689  +** ^Transaction control statements such as [BEGIN], [COMMIT], [ROLLBACK],
         2690  +** [SAVEPOINT], and [RELEASE] cause sqlite3_stmt_readonly() to return true,
         2691  +** since the statements themselves do not actually modify the database but
         2692  +** rather they control the timing of when other statements modify the 
         2693  +** database.  ^The [ATTACH] and [DETACH] statements also cause
         2694  +** sqlite3_stmt_readonly() to return true since, while those statements
         2695  +** change the configuration of a database connection, they do not make 
         2696  +** changes to the content of the database files on disk.
  2661   2697   */
  2662   2698   SQLITE_API int sqlite3_stmt_readonly(sqlite3_stmt *pStmt);
  2663   2699   
  2664   2700   /*
  2665   2701   ** CAPI3REF: Dynamically Typed Value Object
  2666   2702   ** KEYWORDS: {protected sqlite3_value} {unprotected sqlite3_value}
  2667   2703   **
................................................................................
  3047   3083   ** [SQLITE_MISUSE] means that the this routine was called inappropriately.
  3048   3084   ** Perhaps it was called on a [prepared statement] that has
  3049   3085   ** already been [sqlite3_finalize | finalized] or on one that had
  3050   3086   ** previously returned [SQLITE_ERROR] or [SQLITE_DONE].  Or it could
  3051   3087   ** be the case that the same database connection is being used by two or
  3052   3088   ** more threads at the same moment in time.
  3053   3089   **
  3054         -** For all versions of SQLite up to and including 3.6.23.1, it was required
  3055         -** after sqlite3_step() returned anything other than [SQLITE_ROW] that
  3056         -** [sqlite3_reset()] be called before any subsequent invocation of
  3057         -** sqlite3_step().  Failure to invoke [sqlite3_reset()] in this way would
  3058         -** result in an [SQLITE_MISUSE] return from sqlite3_step().  But after
  3059         -** version 3.6.23.1, sqlite3_step() began calling [sqlite3_reset()] 
  3060         -** automatically in this circumstance rather than returning [SQLITE_MISUSE].  
         3090  +** For all versions of SQLite up to and including 3.6.23.1, a call to
         3091  +** [sqlite3_reset()] was required after sqlite3_step() returned anything
         3092  +** other than [SQLITE_ROW] before any subsequent invocation of
         3093  +** sqlite3_step().  Failure to reset the prepared statement using 
         3094  +** [sqlite3_reset()] would result in an [SQLITE_MISUSE] return from
         3095  +** sqlite3_step().  But after version 3.6.23.1, sqlite3_step() began
         3096  +** calling [sqlite3_reset()] automatically in this circumstance rather
         3097  +** than returning [SQLITE_MISUSE].  This is not considered a compatibility
         3098  +** break because any application that ever receives an SQLITE_MISUSE error
         3099  +** is broken by definition.  The [SQLITE_OMIT_AUTORESET] compile-time option
         3100  +** can be used to restore the legacy behavior.
  3061   3101   **
  3062   3102   ** <b>Goofy Interface Alert:</b> In the legacy interface, the sqlite3_step()
  3063   3103   ** API always returns a generic error code, [SQLITE_ERROR], following any
  3064   3104   ** error other than [SQLITE_BUSY] and [SQLITE_MISUSE].  You must call
  3065   3105   ** [sqlite3_reset()] or [sqlite3_finalize()] in order to find one of the
  3066   3106   ** specific [error codes] that better describes the error.
  3067   3107   ** We admit that this is a goofy design.  The problem has been fixed
................................................................................
  3390   3430   ** will pick the one that involves the least amount of data conversion.
  3391   3431   ** If there is only a single implementation which does not care what text
  3392   3432   ** encoding is used, then the fourth argument should be [SQLITE_ANY].
  3393   3433   **
  3394   3434   ** ^(The fifth parameter is an arbitrary pointer.  The implementation of the
  3395   3435   ** function can gain access to this pointer using [sqlite3_user_data()].)^
  3396   3436   **
  3397         -** ^The seventh, eighth and ninth parameters, xFunc, xStep and xFinal, are
         3437  +** ^The sixth, seventh and eighth parameters, xFunc, xStep and xFinal, are
  3398   3438   ** pointers to C-language functions that implement the SQL function or
  3399   3439   ** aggregate. ^A scalar SQL function requires an implementation of the xFunc
  3400   3440   ** callback only; NULL pointers must be passed as the xStep and xFinal
  3401   3441   ** parameters. ^An aggregate SQL function requires an implementation of xStep
  3402   3442   ** and xFinal and NULL pointer must be passed for xFunc. ^To delete an existing
  3403   3443   ** SQL function or aggregate, pass NULL poiners for all three function
  3404   3444   ** callbacks.
  3405   3445   **
  3406         -** ^(If the tenth parameter to sqlite3_create_function_v2() is not NULL,
         3446  +** ^(If the ninth parameter to sqlite3_create_function_v2() is not NULL,
  3407   3447   ** then it is destructor for the application data pointer. 
  3408   3448   ** The destructor is invoked when the function is deleted, either by being
  3409   3449   ** overloaded or when the database connection closes.)^
  3410   3450   ** ^The destructor is also invoked if the call to
  3411   3451   ** sqlite3_create_function_v2() fails.
  3412   3452   ** ^When the destructor callback of the tenth parameter is invoked, it
  3413   3453   ** is passed a single argument which is a copy of the application data 
................................................................................
  3503   3543   ** The C-language implementation of SQL functions and aggregates uses
  3504   3544   ** this set of interface routines to access the parameter values on
  3505   3545   ** the function or aggregate.
  3506   3546   **
  3507   3547   ** The xFunc (for scalar functions) or xStep (for aggregates) parameters
  3508   3548   ** to [sqlite3_create_function()] and [sqlite3_create_function16()]
  3509   3549   ** define callbacks that implement the SQL functions and aggregates.
  3510         -** The 4th parameter to these callbacks is an array of pointers to
         3550  +** The 3rd parameter to these callbacks is an array of pointers to
  3511   3551   ** [protected sqlite3_value] objects.  There is one [sqlite3_value] object for
  3512   3552   ** each parameter to the SQL function.  These routines are used to
  3513   3553   ** extract values from the [sqlite3_value] objects.
  3514   3554   **
  3515   3555   ** These routines work only with [protected sqlite3_value] objects.
  3516   3556   ** Any attempt to use these routines on an [unprotected sqlite3_value]
  3517   3557   ** object results in undefined behavior.
................................................................................
  5231   5271   #define SQLITE_MUTEX_RECURSIVE        1
  5232   5272   #define SQLITE_MUTEX_STATIC_MASTER    2
  5233   5273   #define SQLITE_MUTEX_STATIC_MEM       3  /* sqlite3_malloc() */
  5234   5274   #define SQLITE_MUTEX_STATIC_MEM2      4  /* NOT USED */
  5235   5275   #define SQLITE_MUTEX_STATIC_OPEN      4  /* sqlite3BtreeOpen() */
  5236   5276   #define SQLITE_MUTEX_STATIC_PRNG      5  /* sqlite3_random() */
  5237   5277   #define SQLITE_MUTEX_STATIC_LRU       6  /* lru page list */
  5238         -#define SQLITE_MUTEX_STATIC_LRU2      7  /* lru page list */
         5278  +#define SQLITE_MUTEX_STATIC_LRU2      7  /* NOT USED */
         5279  +#define SQLITE_MUTEX_STATIC_PMEM      7  /* sqlite3PageMalloc() */
  5239   5280   
  5240   5281   /*
  5241   5282   ** CAPI3REF: Retrieve the mutex for a database connection
  5242   5283   **
  5243   5284   ** ^This interface returns a pointer the [sqlite3_mutex] object that 
  5244   5285   ** serializes access to the [database connection] given in the argument
  5245   5286   ** when the [threading mode] is Serialized.
................................................................................
  5382   5423   ** <dd>This parameter records the largest memory allocation request
  5383   5424   ** handed to [sqlite3_malloc()] or [sqlite3_realloc()] (or their
  5384   5425   ** internal equivalents).  Only the value returned in the
  5385   5426   ** *pHighwater parameter to [sqlite3_status()] is of interest.  
  5386   5427   ** The value written into the *pCurrent parameter is undefined.</dd>)^
  5387   5428   **
  5388   5429   ** ^(<dt>SQLITE_STATUS_MALLOC_COUNT</dt>
  5389         -** <dd>This parameter records the number of separate memory allocations.</dd>)^
         5430  +** <dd>This parameter records the number of separate memory allocations
         5431  +** currently checked out.</dd>)^
  5390   5432   **
  5391   5433   ** ^(<dt>SQLITE_STATUS_PAGECACHE_USED</dt>
  5392   5434   ** <dd>This parameter returns the number of pages used out of the
  5393   5435   ** [pagecache memory allocator] that was configured using 
  5394   5436   ** [SQLITE_CONFIG_PAGECACHE].  The
  5395   5437   ** value returned is in pages, not in bytes.</dd>)^
  5396   5438   **
................................................................................
  5487   5529   ** The [sqlite3_db_status()] interface will return a non-zero error code
  5488   5530   ** if a discontinued or unsupported verb is invoked.
  5489   5531   **
  5490   5532   ** <dl>
  5491   5533   ** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_USED</dt>
  5492   5534   ** <dd>This parameter returns the number of lookaside memory slots currently
  5493   5535   ** checked out.</dd>)^
         5536  +**
         5537  +** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_HIT</dt>
         5538  +** <dd>This parameter returns the number malloc attempts that were 
         5539  +** satisfied using lookaside memory. Only the high-water value is meaningful;
         5540  +** the current value is always zero.
         5541  +** checked out.</dd>)^
         5542  +**
         5543  +** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE</dt>
         5544  +** <dd>This parameter returns the number malloc attempts that might have
         5545  +** been satisfied using lookaside memory but failed due to the amount of
         5546  +** memory requested being larger than the lookaside slot size.
         5547  +** Only the high-water value is meaningful;
         5548  +** the current value is always zero.
         5549  +** checked out.</dd>)^
         5550  +**
         5551  +** ^(<dt>SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL</dt>
         5552  +** <dd>This parameter returns the number malloc attempts that might have
         5553  +** been satisfied using lookaside memory but failed due to all lookaside
         5554  +** memory already being in use.
         5555  +** Only the high-water value is meaningful;
         5556  +** the current value is always zero.
         5557  +** checked out.</dd>)^
  5494   5558   **
  5495   5559   ** ^(<dt>SQLITE_DBSTATUS_CACHE_USED</dt>
  5496   5560   ** <dd>This parameter returns the approximate number of of bytes of heap
  5497   5561   ** memory used by all pager caches associated with the database connection.)^
  5498   5562   ** ^The highwater mark associated with SQLITE_DBSTATUS_CACHE_USED is always 0.
  5499   5563   **
  5500   5564   ** ^(<dt>SQLITE_DBSTATUS_SCHEMA_USED</dt>
................................................................................
  5510   5574   ** <dd>This parameter returns the approximate number of of bytes of heap
  5511   5575   ** and lookaside memory used by all prepared statements associated with
  5512   5576   ** the database connection.)^
  5513   5577   ** ^The highwater mark associated with SQLITE_DBSTATUS_STMT_USED is always 0.
  5514   5578   ** </dd>
  5515   5579   ** </dl>
  5516   5580   */
  5517         -#define SQLITE_DBSTATUS_LOOKASIDE_USED     0
  5518         -#define SQLITE_DBSTATUS_CACHE_USED         1
  5519         -#define SQLITE_DBSTATUS_SCHEMA_USED        2
  5520         -#define SQLITE_DBSTATUS_STMT_USED          3
  5521         -#define SQLITE_DBSTATUS_MAX                3   /* Largest defined DBSTATUS */
         5581  +#define SQLITE_DBSTATUS_LOOKASIDE_USED       0
         5582  +#define SQLITE_DBSTATUS_CACHE_USED           1
         5583  +#define SQLITE_DBSTATUS_SCHEMA_USED          2
         5584  +#define SQLITE_DBSTATUS_STMT_USED            3
         5585  +#define SQLITE_DBSTATUS_LOOKASIDE_HIT        4
         5586  +#define SQLITE_DBSTATUS_LOOKASIDE_MISS_SIZE  5
         5587  +#define SQLITE_DBSTATUS_LOOKASIDE_MISS_FULL  6
         5588  +#define SQLITE_DBSTATUS_MAX                  6   /* Largest defined DBSTATUS */
  5522   5589   
  5523   5590   
  5524   5591   /*
  5525   5592   ** CAPI3REF: Prepared Statement Status
  5526   5593   **
  5527   5594   ** ^(Each prepared statement maintains various
  5528   5595   ** [SQLITE_STMTSTATUS_SORT | counters] that measure the number
................................................................................
  5642   5709   **
  5643   5710   ** ^SQLite invokes the xCreate() method to construct a new cache instance.
  5644   5711   ** SQLite will typically create one cache instance for each open database file,
  5645   5712   ** though this is not guaranteed. ^The
  5646   5713   ** first parameter, szPage, is the size in bytes of the pages that must
  5647   5714   ** be allocated by the cache.  ^szPage will not be a power of two.  ^szPage
  5648   5715   ** will the page size of the database file that is to be cached plus an
  5649         -** increment (here called "R") of about 100 or 200.  SQLite will use the
         5716  +** increment (here called "R") of less than 250.  SQLite will use the
  5650   5717   ** extra R bytes on each page to store metadata about the underlying
  5651   5718   ** database page on disk.  The value of R depends
  5652   5719   ** on the SQLite version, the target platform, and how SQLite was compiled.
  5653         -** ^R is constant for a particular build of SQLite.  ^The second argument to
         5720  +** ^(R is constant for a particular build of SQLite. Except, there are two
         5721  +** distinct values of R when SQLite is compiled with the proprietary
         5722  +** ZIPVFS extension.)^  ^The second argument to
  5654   5723   ** xCreate(), bPurgeable, is true if the cache being created will
  5655   5724   ** be used to cache database pages of a file stored on disk, or
  5656   5725   ** false if it is used for an in-memory database. The cache implementation
  5657   5726   ** does not have to do anything special based with the value of bPurgeable;
  5658   5727   ** it is purely advisory.  ^On a cache where bPurgeable is false, SQLite will
  5659   5728   ** never invoke xUnpin() except to deliberately delete a page.
  5660   5729   ** ^In other words, calls to xUnpin() on a cache with bPurgeable set to
................................................................................
  5678   5747   ** 8-byte boundary. The page to be fetched is determined by the key. ^The
  5679   5748   ** mimimum key value is 1.  After it has been retrieved using xFetch, the page 
  5680   5749   ** is considered to be "pinned".
  5681   5750   **
  5682   5751   ** If the requested page is already in the page cache, then the page cache
  5683   5752   ** implementation must return a pointer to the page buffer with its content
  5684   5753   ** intact.  If the requested page is not already in the cache, then the
  5685         -** behavior of the cache implementation should use the value of the createFlag
         5754  +** cache implementation should use the value of the createFlag
  5686   5755   ** parameter to help it determined what action to take:
  5687   5756   **
  5688   5757   ** <table border=1 width=85% align=center>
  5689   5758   ** <tr><th> createFlag <th> Behaviour when page is not already in cache
  5690   5759   ** <tr><td> 0 <td> Do not allocate a new page.  Return NULL.
  5691   5760   ** <tr><td> 1 <td> Allocate a new page if it easy and convenient to do so.
  5692   5761   **                 Otherwise return NULL.
................................................................................
  5762   5831   **
  5763   5832   ** The backup API copies the content of one database into another.
  5764   5833   ** It is useful either for creating backups of databases or
  5765   5834   ** for copying in-memory databases to or from persistent files. 
  5766   5835   **
  5767   5836   ** See Also: [Using the SQLite Online Backup API]
  5768   5837   **
  5769         -** ^Exclusive access is required to the destination database for the 
  5770         -** duration of the operation. ^However the source database is only
  5771         -** read-locked while it is actually being read; it is not locked
  5772         -** continuously for the entire backup operation. ^Thus, the backup may be
  5773         -** performed on a live source database without preventing other users from
         5838  +** ^SQLite holds a write transaction open on the destination database file
         5839  +** for the duration of the backup operation.
         5840  +** ^The source database is read-locked only while it is being read;
         5841  +** it is not locked continuously for the entire backup operation.
         5842  +** ^Thus, the backup may be performed on a live source database without
         5843  +** preventing other database connections from
  5774   5844   ** reading or writing to the source database while the backup is underway.
  5775   5845   ** 
  5776   5846   ** ^(To perform a backup operation: 
  5777   5847   **   <ol>
  5778   5848   **     <li><b>sqlite3_backup_init()</b> is called once to initialize the
  5779   5849   **         backup, 
  5780   5850   **     <li><b>sqlite3_backup_step()</b> is called one or more times to transfer 
................................................................................
  5793   5863   ** ^The database name is "main" for the main database, "temp" for the
  5794   5864   ** temporary database, or the name specified after the AS keyword in
  5795   5865   ** an [ATTACH] statement for an attached database.
  5796   5866   ** ^The S and M arguments passed to 
  5797   5867   ** sqlite3_backup_init(D,N,S,M) identify the [database connection]
  5798   5868   ** and database name of the source database, respectively.
  5799   5869   ** ^The source and destination [database connections] (parameters S and D)
  5800         -** must be different or else sqlite3_backup_init(D,N,S,M) will file with
         5870  +** must be different or else sqlite3_backup_init(D,N,S,M) will fail with
  5801   5871   ** an error.
  5802   5872   **
  5803   5873   ** ^If an error occurs within sqlite3_backup_init(D,N,S,M), then NULL is
  5804         -** returned and an error code and error message are store3d in the
         5874  +** returned and an error code and error message are stored in the
  5805   5875   ** destination [database connection] D.
  5806   5876   ** ^The error code and message for the failed call to sqlite3_backup_init()
  5807   5877   ** can be retrieved using the [sqlite3_errcode()], [sqlite3_errmsg()], and/or
  5808   5878   ** [sqlite3_errmsg16()] functions.
  5809   5879   ** ^A successful call to sqlite3_backup_init() returns a pointer to an
  5810   5880   ** [sqlite3_backup] object.
  5811   5881   ** ^The [sqlite3_backup] object may be used with the sqlite3_backup_step() and
................................................................................
  5814   5884   **
  5815   5885   ** <b>sqlite3_backup_step()</b>
  5816   5886   **
  5817   5887   ** ^Function sqlite3_backup_step(B,N) will copy up to N pages between 
  5818   5888   ** the source and destination databases specified by [sqlite3_backup] object B.
  5819   5889   ** ^If N is negative, all remaining source pages are copied. 
  5820   5890   ** ^If sqlite3_backup_step(B,N) successfully copies N pages and there
  5821         -** are still more pages to be copied, then the function resturns [SQLITE_OK].
         5891  +** are still more pages to be copied, then the function returns [SQLITE_OK].
  5822   5892   ** ^If sqlite3_backup_step(B,N) successfully finishes copying all pages
  5823   5893   ** from source to destination, then it returns [SQLITE_DONE].
  5824   5894   ** ^If an error occurs while running sqlite3_backup_step(B,N),
  5825   5895   ** then an [error code] is returned. ^As well as [SQLITE_OK] and
  5826   5896   ** [SQLITE_DONE], a call to sqlite3_backup_step() may return [SQLITE_READONLY],
  5827   5897   ** [SQLITE_NOMEM], [SQLITE_BUSY], [SQLITE_LOCKED], or an
  5828   5898   ** [SQLITE_IOERR_ACCESS | SQLITE_IOERR_XXX] extended error code.
  5829   5899   **
  5830   5900   ** ^(The sqlite3_backup_step() might return [SQLITE_READONLY] if
  5831   5901   ** <ol>
  5832   5902   ** <li> the destination database was opened read-only, or
  5833   5903   ** <li> the destination database is using write-ahead-log journaling
  5834   5904   ** and the destination and source page sizes differ, or
  5835         -** <li> The destination database is an in-memory database and the
         5905  +** <li> the destination database is an in-memory database and the
  5836   5906   ** destination and source page sizes differ.
  5837   5907   ** </ol>)^
  5838   5908   **
  5839   5909   ** ^If sqlite3_backup_step() cannot obtain a required file-system lock, then
  5840   5910   ** the [sqlite3_busy_handler | busy-handler function]
  5841   5911   ** is invoked (if one is specified). ^If the 
  5842   5912   ** busy-handler returns non-zero before the lock is available, then 
................................................................................
  6159   6229   ** using [sqlite3_wal_hook()] disables the automatic checkpoint mechanism
  6160   6230   ** configured by this function.
  6161   6231   **
  6162   6232   ** ^The [wal_autocheckpoint pragma] can be used to invoke this interface
  6163   6233   ** from SQL.
  6164   6234   **
  6165   6235   ** ^Every new [database connection] defaults to having the auto-checkpoint
  6166         -** enabled with a threshold of 1000 pages.  The use of this interface
         6236  +** enabled with a threshold of 1000 or [SQLITE_DEFAULT_WAL_AUTOCHECKPOINT]
         6237  +** pages.  The use of this interface
  6167   6238   ** is only necessary if the default setting is found to be suboptimal
  6168   6239   ** for a particular application.
  6169   6240   */
  6170   6241   SQLITE_API int sqlite3_wal_autocheckpoint(sqlite3 *db, int N);
  6171   6242   
  6172   6243   /*
  6173   6244   ** CAPI3REF: Checkpoint a database