SQLite Archiver
Check-in [95ee3b3708]
Not logged in

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update the built-in SQLite to version 3.8.6.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1:95ee3b370851674d0bb74084611ece714b9f38ad
User & Date: drh 2014-08-15 19:01:24
Context
2014-12-10
22:06
Add a missing "closedir()". check-in: 15adeb2f9a user: drh tags: trunk
2014-08-15
19:01
Update the built-in SQLite to version 3.8.6. check-in: 95ee3b3708 user: drh tags: trunk
2014-06-14
17:20
In sqlarfs (the Fuse FS driver for SQLite archives) make sure it always runs in the foreground. And after it stops, clean up prepared statements and memory allocations and close the database connection before exiting. check-in: ed4e6af2e8 user: drh tags: trunk
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to sqlite3.c.

     1      1   /******************************************************************************
     2      2   ** This file is an amalgamation of many separate C source files from SQLite
     3         -** version 3.8.5.  By combining all the individual C code files into this 
            3  +** version 3.8.6.  By combining all the individual C code files into this 
     4      4   ** single large file, the entire code can be compiled as a single translation
     5      5   ** unit.  This allows many compilers to do optimizations that would not be
     6      6   ** possible if the files were compiled separately.  Performance improvements
     7      7   ** of 5% or more are commonly seen when SQLite is compiled as a single
     8      8   ** translation unit.
     9      9   **
    10     10   ** This file is all you need to compile SQLite.  To use SQLite in other
................................................................................
   218    218   ** string contains the date and time of the check-in (UTC) and an SHA1
   219    219   ** hash of the entire source tree.
   220    220   **
   221    221   ** See also: [sqlite3_libversion()],
   222    222   ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
   223    223   ** [sqlite_version()] and [sqlite_source_id()].
   224    224   */
   225         -#define SQLITE_VERSION        "3.8.5"
   226         -#define SQLITE_VERSION_NUMBER 3008005
   227         -#define SQLITE_SOURCE_ID      "2014-06-02 11:26:33 9f18b303cd1bc5779d82669884f802c7889b4947"
          225  +#define SQLITE_VERSION        "3.8.6"
          226  +#define SQLITE_VERSION_NUMBER 3008006
          227  +#define SQLITE_SOURCE_ID      "2014-08-15 11:46:33 9491ba7d738528f168657adb43a198238abde19e"
   228    228   
   229    229   /*
   230    230   ** CAPI3REF: Run-Time Library Version Numbers
   231    231   ** KEYWORDS: sqlite3_version, sqlite3_sourceid
   232    232   **
   233    233   ** These interfaces provide the same information as the [SQLITE_VERSION],
   234    234   ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros
................................................................................
   380    380   #endif
   381    381   
   382    382   /*
   383    383   ** CAPI3REF: Closing A Database Connection
   384    384   **
   385    385   ** ^The sqlite3_close() and sqlite3_close_v2() routines are destructors
   386    386   ** for the [sqlite3] object.
   387         -** ^Calls to sqlite3_close() and sqlite3_close_v2() return SQLITE_OK if
          387  +** ^Calls to sqlite3_close() and sqlite3_close_v2() return [SQLITE_OK] if
   388    388   ** the [sqlite3] object is successfully destroyed and all associated
   389    389   ** resources are deallocated.
   390    390   **
   391    391   ** ^If the database connection is associated with unfinalized prepared
   392    392   ** statements or unfinished sqlite3_backup objects then sqlite3_close()
   393    393   ** will leave the database connection open and return [SQLITE_BUSY].
   394    394   ** ^If sqlite3_close_v2() is called with unfinalized prepared statements
   395         -** and unfinished sqlite3_backups, then the database connection becomes
          395  +** and/or unfinished sqlite3_backups, then the database connection becomes
   396    396   ** an unusable "zombie" which will automatically be deallocated when the
   397    397   ** last prepared statement is finalized or the last sqlite3_backup is
   398    398   ** finished.  The sqlite3_close_v2() interface is intended for use with
   399    399   ** host languages that are garbage collected, and where the order in which
   400    400   ** destructors are called is arbitrary.
   401    401   **
   402    402   ** Applications should [sqlite3_finalize | finalize] all [prepared statements],
   403    403   ** [sqlite3_blob_close | close] all [BLOB handles], and 
   404    404   ** [sqlite3_backup_finish | finish] all [sqlite3_backup] objects associated
   405    405   ** with the [sqlite3] object prior to attempting to close the object.  ^If
   406    406   ** sqlite3_close_v2() is called on a [database connection] that still has
   407    407   ** outstanding [prepared statements], [BLOB handles], and/or
   408         -** [sqlite3_backup] objects then it returns SQLITE_OK but the deallocation
          408  +** [sqlite3_backup] objects then it returns [SQLITE_OK] and the deallocation
   409    409   ** of resources is deferred until all [prepared statements], [BLOB handles],
   410    410   ** and [sqlite3_backup] objects are also destroyed.
   411    411   **
   412    412   ** ^If an [sqlite3] object is destroyed while a transaction is open,
   413    413   ** the transaction is automatically rolled back.
   414    414   **
   415    415   ** The C parameter to [sqlite3_close(C)] and [sqlite3_close_v2(C)]
................................................................................
   497    497     int (*callback)(void*,int,char**,char**),  /* Callback function */
   498    498     void *,                                    /* 1st argument to callback */
   499    499     char **errmsg                              /* Error msg written here */
   500    500   );
   501    501   
   502    502   /*
   503    503   ** CAPI3REF: Result Codes
   504         -** KEYWORDS: SQLITE_OK {error code} {error codes}
   505         -** KEYWORDS: {result code} {result codes}
          504  +** KEYWORDS: {result code definitions}
   506    505   **
   507    506   ** Many SQLite functions return an integer result code from the set shown
   508    507   ** here in order to indicate success or failure.
   509    508   **
   510    509   ** New error codes may be added in future versions of SQLite.
   511    510   **
   512         -** See also: [SQLITE_IOERR_READ | extended result codes],
   513         -** [sqlite3_vtab_on_conflict()] [SQLITE_ROLLBACK | result codes].
          511  +** See also: [extended result code definitions]
   514    512   */
   515    513   #define SQLITE_OK           0   /* Successful result */
   516    514   /* beginning-of-error-codes */
   517    515   #define SQLITE_ERROR        1   /* SQL error or missing database */
   518    516   #define SQLITE_INTERNAL     2   /* Internal logic error in SQLite */
   519    517   #define SQLITE_PERM         3   /* Access permission denied */
   520    518   #define SQLITE_ABORT        4   /* Callback routine requested an abort */
................................................................................
   544    542   #define SQLITE_WARNING     28   /* Warnings from sqlite3_log() */
   545    543   #define SQLITE_ROW         100  /* sqlite3_step() has another row ready */
   546    544   #define SQLITE_DONE        101  /* sqlite3_step() has finished executing */
   547    545   /* end-of-error-codes */
   548    546   
   549    547   /*
   550    548   ** CAPI3REF: Extended Result Codes
   551         -** KEYWORDS: {extended error code} {extended error codes}
   552         -** KEYWORDS: {extended result code} {extended result codes}
          549  +** KEYWORDS: {extended result code definitions}
   553    550   **
   554         -** In its default configuration, SQLite API routines return one of 26 integer
   555         -** [SQLITE_OK | result codes].  However, experience has shown that many of
          551  +** In its default configuration, SQLite API routines return one of 30 integer
          552  +** [result codes].  However, experience has shown that many of
   556    553   ** these result codes are too coarse-grained.  They do not provide as
   557    554   ** much information about problems as programmers might like.  In an effort to
   558    555   ** address this, newer versions of SQLite (version 3.3.8 and later) include
   559    556   ** support for additional result codes that provide more detailed information
   560         -** about errors. The extended result codes are enabled or disabled
          557  +** about errors. These [extended result codes] are enabled or disabled
   561    558   ** on a per database connection basis using the
   562         -** [sqlite3_extended_result_codes()] API.
   563         -**
   564         -** Some of the available extended result codes are listed here.
   565         -** One may expect the number of extended result codes will increase
   566         -** over time.  Software that uses extended result codes should expect
   567         -** to see new result codes in future releases of SQLite.
   568         -**
   569         -** The SQLITE_OK result code will never be extended.  It will always
   570         -** be exactly zero.
          559  +** [sqlite3_extended_result_codes()] API.  Or, the extended code for
          560  +** the most recent error can be obtained using
          561  +** [sqlite3_extended_errcode()].
   571    562   */
   572    563   #define SQLITE_IOERR_READ              (SQLITE_IOERR | (1<<8))
   573    564   #define SQLITE_IOERR_SHORT_READ        (SQLITE_IOERR | (2<<8))
   574    565   #define SQLITE_IOERR_WRITE             (SQLITE_IOERR | (3<<8))
   575    566   #define SQLITE_IOERR_FSYNC             (SQLITE_IOERR | (4<<8))
   576    567   #define SQLITE_IOERR_DIR_FSYNC         (SQLITE_IOERR | (5<<8))
   577    568   #define SQLITE_IOERR_TRUNCATE          (SQLITE_IOERR | (6<<8))
................................................................................
   796    787   ** integer opcode.  The third argument is a generic pointer intended to
   797    788   ** point to a structure that may contain arguments or space in which to
   798    789   ** write return values.  Potential uses for xFileControl() might be
   799    790   ** functions to enable blocking locks with timeouts, to change the
   800    791   ** locking strategy (for example to use dot-file locks), to inquire
   801    792   ** about the status of a lock, or to break stale locks.  The SQLite
   802    793   ** core reserves all opcodes less than 100 for its own use.
   803         -** A [SQLITE_FCNTL_LOCKSTATE | list of opcodes] less than 100 is available.
          794  +** A [file control opcodes | list of opcodes] less than 100 is available.
   804    795   ** Applications that define a custom xFileControl method should use opcodes
   805    796   ** greater than 100 to avoid conflicts.  VFS implementations should
   806    797   ** return [SQLITE_NOTFOUND] for file control opcodes that they do not
   807    798   ** recognize.
   808    799   **
   809    800   ** The xSectorSize() method returns the sector size of the
   810    801   ** device that underlies the file.  The sector size is the
................................................................................
   869    860     int (*xUnfetch)(sqlite3_file*, sqlite3_int64 iOfst, void *p);
   870    861     /* Methods above are valid for version 3 */
   871    862     /* Additional methods may be added in future releases */
   872    863   };
   873    864   
   874    865   /*
   875    866   ** CAPI3REF: Standard File Control Opcodes
          867  +** KEYWORDS: {file control opcodes} {file control opcode}
   876    868   **
   877    869   ** These integer constants are opcodes for the xFileControl method
   878    870   ** of the [sqlite3_io_methods] object and for the [sqlite3_file_control()]
   879    871   ** interface.
   880    872   **
   881    873   ** The [SQLITE_FCNTL_LOCKSTATE] opcode is used for debugging.  This
   882    874   ** opcode causes the xFileControl method to write the current state of
................................................................................
  2148   2140   */
  2149   2141   SQLITE_API int sqlite3_complete(const char *sql);
  2150   2142   SQLITE_API int sqlite3_complete16(const void *sql);
  2151   2143   
  2152   2144   /*
  2153   2145   ** CAPI3REF: Register A Callback To Handle SQLITE_BUSY Errors
  2154   2146   **
  2155         -** ^This routine sets a callback function that might be invoked whenever
  2156         -** an attempt is made to open a database table that another thread
  2157         -** or process has locked.
         2147  +** ^The sqlite3_busy_handler(D,X,P) routine sets a callback function X
         2148  +** that might be invoked with argument P whenever
         2149  +** an attempt is made to access a database table associated with
         2150  +** [database connection] D when another thread
         2151  +** or process has the table locked.
         2152  +** The sqlite3_busy_handler() interface is used to implement
         2153  +** [sqlite3_busy_timeout()] and [PRAGMA busy_timeout].
  2158   2154   **
  2159         -** ^If the busy callback is NULL, then [SQLITE_BUSY] or [SQLITE_IOERR_BLOCKED]
         2155  +** ^If the busy callback is NULL, then [SQLITE_BUSY]
  2160   2156   ** is returned immediately upon encountering the lock.  ^If the busy callback
  2161   2157   ** is not NULL, then the callback might be invoked with two arguments.
  2162   2158   **
  2163   2159   ** ^The first argument to the busy handler is a copy of the void* pointer which
  2164   2160   ** is the third argument to sqlite3_busy_handler().  ^The second argument to
  2165   2161   ** the busy handler callback is the number of times that the busy handler has
  2166         -** been invoked for this locking event.  ^If the
         2162  +** been invoked for the same locking event.  ^If the
  2167   2163   ** busy callback returns 0, then no additional attempts are made to
  2168         -** access the database and [SQLITE_BUSY] or [SQLITE_IOERR_BLOCKED] is returned.
         2164  +** access the database and [SQLITE_BUSY] is returned
         2165  +** to the application.
  2169   2166   ** ^If the callback returns non-zero, then another attempt
  2170         -** is made to open the database for reading and the cycle repeats.
         2167  +** is made to access the database and the cycle repeats.
  2171   2168   **
  2172   2169   ** The presence of a busy handler does not guarantee that it will be invoked
  2173   2170   ** when there is lock contention. ^If SQLite determines that invoking the busy
  2174   2171   ** handler could result in a deadlock, it will go ahead and return [SQLITE_BUSY]
  2175         -** or [SQLITE_IOERR_BLOCKED] instead of invoking the busy handler.
         2172  +** to the application instead of invoking the 
         2173  +** busy handler.
  2176   2174   ** Consider a scenario where one process is holding a read lock that
  2177   2175   ** it is trying to promote to a reserved lock and
  2178   2176   ** a second process is holding a reserved lock that it is trying
  2179   2177   ** to promote to an exclusive lock.  The first process cannot proceed
  2180   2178   ** because it is blocked by the second and the second process cannot
  2181   2179   ** proceed because it is blocked by the first.  If both processes
  2182   2180   ** invoke the busy handlers, neither will make any progress.  Therefore,
  2183   2181   ** SQLite returns [SQLITE_BUSY] for the first process, hoping that this
  2184   2182   ** will induce the first process to release its read lock and allow
  2185   2183   ** the second process to proceed.
  2186   2184   **
  2187   2185   ** ^The default busy callback is NULL.
  2188   2186   **
  2189         -** ^The [SQLITE_BUSY] error is converted to [SQLITE_IOERR_BLOCKED]
  2190         -** when SQLite is in the middle of a large transaction where all the
  2191         -** changes will not fit into the in-memory cache.  SQLite will
  2192         -** already hold a RESERVED lock on the database file, but it needs
  2193         -** to promote this lock to EXCLUSIVE so that it can spill cache
  2194         -** pages into the database file without harm to concurrent
  2195         -** readers.  ^If it is unable to promote the lock, then the in-memory
  2196         -** cache will be left in an inconsistent state and so the error
  2197         -** code is promoted from the relatively benign [SQLITE_BUSY] to
  2198         -** the more severe [SQLITE_IOERR_BLOCKED].  ^This error code promotion
  2199         -** forces an automatic rollback of the changes.  See the
  2200         -** <a href="/cvstrac/wiki?p=CorruptionFollowingBusyError">
  2201         -** CorruptionFollowingBusyError</a> wiki page for a discussion of why
  2202         -** this is important.
  2203         -**
  2204   2187   ** ^(There can only be a single busy handler defined for each
  2205   2188   ** [database connection].  Setting a new busy handler clears any
  2206   2189   ** previously set handler.)^  ^Note that calling [sqlite3_busy_timeout()]
  2207         -** will also set or clear the busy handler.
         2190  +** or evaluating [PRAGMA busy_timeout=N] will change the
         2191  +** busy handler and thus clear any previously set busy handler.
  2208   2192   **
  2209   2193   ** The busy callback should not take any actions which modify the
  2210         -** database connection that invoked the busy handler.  Any such actions
         2194  +** database connection that invoked the busy handler.  In other words,
         2195  +** the busy handler is not reentrant.  Any such actions
  2211   2196   ** result in undefined behavior.
  2212   2197   ** 
  2213   2198   ** A busy handler must not close the database connection
  2214   2199   ** or [prepared statement] that invoked the busy handler.
  2215   2200   */
  2216   2201   SQLITE_API int sqlite3_busy_handler(sqlite3*, int(*)(void*,int), void*);
  2217   2202   
................................................................................
  2219   2204   ** CAPI3REF: Set A Busy Timeout
  2220   2205   **
  2221   2206   ** ^This routine sets a [sqlite3_busy_handler | busy handler] that sleeps
  2222   2207   ** for a specified amount of time when a table is locked.  ^The handler
  2223   2208   ** will sleep multiple times until at least "ms" milliseconds of sleeping
  2224   2209   ** have accumulated.  ^After at least "ms" milliseconds of sleeping,
  2225   2210   ** the handler returns 0 which causes [sqlite3_step()] to return
  2226         -** [SQLITE_BUSY] or [SQLITE_IOERR_BLOCKED].
         2211  +** [SQLITE_BUSY].
  2227   2212   **
  2228   2213   ** ^Calling this routine with an argument less than or equal to zero
  2229   2214   ** turns off all busy handlers.
  2230   2215   **
  2231   2216   ** ^(There can only be a single busy handler for a particular
  2232   2217   ** [database connection] any any given moment.  If another busy handler
  2233   2218   ** was defined  (using [sqlite3_busy_handler()]) prior to calling
  2234   2219   ** this routine, that other busy handler is cleared.)^
         2220  +**
         2221  +** See also:  [PRAGMA busy_timeout]
  2235   2222   */
  2236   2223   SQLITE_API int sqlite3_busy_timeout(sqlite3*, int ms);
  2237   2224   
  2238   2225   /*
  2239   2226   ** CAPI3REF: Convenience Routines For Running Queries
  2240   2227   **
  2241   2228   ** This is a legacy interface that is preserved for backwards compatibility.
................................................................................
  2629   2616   **
  2630   2617   ** The [sqlite3_set_authorizer | authorizer callback function] must
  2631   2618   ** return either [SQLITE_OK] or one of these two constants in order
  2632   2619   ** to signal SQLite whether or not the action is permitted.  See the
  2633   2620   ** [sqlite3_set_authorizer | authorizer documentation] for additional
  2634   2621   ** information.
  2635   2622   **
  2636         -** Note that SQLITE_IGNORE is also used as a [SQLITE_ROLLBACK | return code]
  2637         -** from the [sqlite3_vtab_on_conflict()] interface.
         2623  +** Note that SQLITE_IGNORE is also used as a [conflict resolution mode]
         2624  +** returned from the [sqlite3_vtab_on_conflict()] interface.
  2638   2625   */
  2639   2626   #define SQLITE_DENY   1   /* Abort the SQL statement with an error */
  2640   2627   #define SQLITE_IGNORE 2   /* Don't allow access, but don't generate an error */
  2641   2628   
  2642   2629   /*
  2643   2630   ** CAPI3REF: Authorizer Action Codes
  2644   2631   **
................................................................................
  4816   4803   **
  4817   4804   ** ^(If this global variable is made to point to a string which is
  4818   4805   ** the name of a folder (a.k.a. directory), then all temporary files
  4819   4806   ** created by SQLite when using a built-in [sqlite3_vfs | VFS]
  4820   4807   ** will be placed in that directory.)^  ^If this variable
  4821   4808   ** is a NULL pointer, then SQLite performs a search for an appropriate
  4822   4809   ** temporary file directory.
         4810  +**
         4811  +** Applications are strongly discouraged from using this global variable.
         4812  +** It is required to set a temporary folder on Windows Runtime (WinRT).
         4813  +** But for all other platforms, it is highly recommended that applications
         4814  +** neither read nor write this variable.  This global variable is a relic
         4815  +** that exists for backwards compatibility of legacy applications and should
         4816  +** be avoided in new projects.
  4823   4817   **
  4824   4818   ** It is not safe to read or modify this variable in more than one
  4825   4819   ** thread at a time.  It is not safe to read or modify this variable
  4826   4820   ** if a [database connection] is being used at the same time in a separate
  4827   4821   ** thread.
  4828   4822   ** It is intended that this variable be set once
  4829   4823   ** as part of process initialization and before any SQLite interface
................................................................................
  4835   4829   ** the [temp_store_directory pragma] always assumes that any string
  4836   4830   ** that this variable points to is held in memory obtained from 
  4837   4831   ** [sqlite3_malloc] and the pragma may attempt to free that memory
  4838   4832   ** using [sqlite3_free].
  4839   4833   ** Hence, if this variable is modified directly, either it should be
  4840   4834   ** made NULL or made to point to memory obtained from [sqlite3_malloc]
  4841   4835   ** or else the use of the [temp_store_directory pragma] should be avoided.
         4836  +** Except when requested by the [temp_store_directory pragma], SQLite
         4837  +** does not free the memory that sqlite3_temp_directory points to.  If
         4838  +** the application wants that memory to be freed, it must do
         4839  +** so itself, taking care to only do so after all [database connection]
         4840  +** objects have been destroyed.
  4842   4841   **
  4843   4842   ** <b>Note to Windows Runtime users:</b>  The temporary directory must be set
  4844   4843   ** prior to calling [sqlite3_open] or [sqlite3_open_v2].  Otherwise, various
  4845   4844   ** features that require the use of temporary files may fail.  Here is an
  4846   4845   ** example of how to do this using C++ with the Windows Runtime:
  4847   4846   **
  4848   4847   ** <blockquote><pre>
................................................................................
  5969   5968   ** to sqlite3_mutex_alloc() is one of these integer constants:
  5970   5969   **
  5971   5970   ** <ul>
  5972   5971   ** <li>  SQLITE_MUTEX_FAST
  5973   5972   ** <li>  SQLITE_MUTEX_RECURSIVE
  5974   5973   ** <li>  SQLITE_MUTEX_STATIC_MASTER
  5975   5974   ** <li>  SQLITE_MUTEX_STATIC_MEM
  5976         -** <li>  SQLITE_MUTEX_STATIC_MEM2
         5975  +** <li>  SQLITE_MUTEX_STATIC_OPEN
  5977   5976   ** <li>  SQLITE_MUTEX_STATIC_PRNG
  5978   5977   ** <li>  SQLITE_MUTEX_STATIC_LRU
  5979         -** <li>  SQLITE_MUTEX_STATIC_LRU2
         5978  +** <li>  SQLITE_MUTEX_STATIC_PMEM
         5979  +** <li>  SQLITE_MUTEX_STATIC_APP1
         5980  +** <li>  SQLITE_MUTEX_STATIC_APP2
  5980   5981   ** </ul>)^
  5981   5982   **
  5982   5983   ** ^The first two constants (SQLITE_MUTEX_FAST and SQLITE_MUTEX_RECURSIVE)
  5983   5984   ** cause sqlite3_mutex_alloc() to create
  5984   5985   ** a new mutex.  ^The new mutex is recursive when SQLITE_MUTEX_RECURSIVE
  5985   5986   ** is used but not necessarily so when SQLITE_MUTEX_FAST is used.
  5986   5987   ** The mutex implementation does not need to make a distinction
................................................................................
  6176   6177   #define SQLITE_MUTEX_STATIC_MEM       3  /* sqlite3_malloc() */
  6177   6178   #define SQLITE_MUTEX_STATIC_MEM2      4  /* NOT USED */
  6178   6179   #define SQLITE_MUTEX_STATIC_OPEN      4  /* sqlite3BtreeOpen() */
  6179   6180   #define SQLITE_MUTEX_STATIC_PRNG      5  /* sqlite3_random() */
  6180   6181   #define SQLITE_MUTEX_STATIC_LRU       6  /* lru page list */
  6181   6182   #define SQLITE_MUTEX_STATIC_LRU2      7  /* NOT USED */
  6182   6183   #define SQLITE_MUTEX_STATIC_PMEM      7  /* sqlite3PageMalloc() */
         6184  +#define SQLITE_MUTEX_STATIC_APP1      8  /* For use by application */
         6185  +#define SQLITE_MUTEX_STATIC_APP2      9  /* For use by application */
         6186  +#define SQLITE_MUTEX_STATIC_APP3     10  /* For use by application */
  6183   6187   
  6184   6188   /*
  6185   6189   ** CAPI3REF: Retrieve the mutex for a database connection
  6186   6190   **
  6187   6191   ** ^This interface returns a pointer the [sqlite3_mutex] object that 
  6188   6192   ** serializes access to the [database connection] given in the argument
  6189   6193   ** when the [threading mode] is Serialized.
................................................................................
  6271   6275   #define SQLITE_TESTCTRL_ISKEYWORD               16
  6272   6276   #define SQLITE_TESTCTRL_SCRATCHMALLOC           17
  6273   6277   #define SQLITE_TESTCTRL_LOCALTIME_FAULT         18
  6274   6278   #define SQLITE_TESTCTRL_EXPLAIN_STMT            19
  6275   6279   #define SQLITE_TESTCTRL_NEVER_CORRUPT           20
  6276   6280   #define SQLITE_TESTCTRL_VDBE_COVERAGE           21
  6277   6281   #define SQLITE_TESTCTRL_BYTEORDER               22
  6278         -#define SQLITE_TESTCTRL_LAST                    22
         6282  +#define SQLITE_TESTCTRL_ISINIT                  23
         6283  +#define SQLITE_TESTCTRL_LAST                    23
  6279   6284   
  6280   6285   /*
  6281   6286   ** CAPI3REF: SQLite Runtime Status
  6282   6287   **
  6283   6288   ** ^This interface is used to retrieve runtime status information
  6284   6289   ** about the performance of SQLite, and optionally to reset various
  6285   6290   ** highwater marks.  ^The first argument is an integer code for
................................................................................
  7254   7259   ** ^The callback registered by this function replaces any existing callback
  7255   7260   ** registered using [sqlite3_wal_hook()].  ^Likewise, registering a callback
  7256   7261   ** using [sqlite3_wal_hook()] disables the automatic checkpoint mechanism
  7257   7262   ** configured by this function.
  7258   7263   **
  7259   7264   ** ^The [wal_autocheckpoint pragma] can be used to invoke this interface
  7260   7265   ** from SQL.
         7266  +**
         7267  +** ^Checkpoints initiated by this mechanism are
         7268  +** [sqlite3_wal_checkpoint_v2|PASSIVE].
  7261   7269   **
  7262   7270   ** ^Every new [database connection] defaults to having the auto-checkpoint
  7263   7271   ** enabled with a threshold of 1000 or [SQLITE_DEFAULT_WAL_AUTOCHECKPOINT]
  7264   7272   ** pages.  The use of this interface
  7265   7273   ** is only necessary if the default setting is found to be suboptimal
  7266   7274   ** for a particular application.
  7267   7275   */
................................................................................
  7271   7279   ** CAPI3REF: Checkpoint a database
  7272   7280   **
  7273   7281   ** ^The [sqlite3_wal_checkpoint(D,X)] interface causes database named X
  7274   7282   ** on [database connection] D to be [checkpointed].  ^If X is NULL or an
  7275   7283   ** empty string, then a checkpoint is run on all databases of
  7276   7284   ** connection D.  ^If the database connection D is not in
  7277   7285   ** [WAL | write-ahead log mode] then this interface is a harmless no-op.
         7286  +** ^The [sqlite3_wal_checkpoint(D,X)] interface initiates a
         7287  +** [sqlite3_wal_checkpoint_v2|PASSIVE] checkpoint.
         7288  +** Use the [sqlite3_wal_checkpoint_v2()] interface to get a FULL
         7289  +** or RESET checkpoint.
  7278   7290   **
  7279   7291   ** ^The [wal_checkpoint pragma] can be used to invoke this interface
  7280   7292   ** from SQL.  ^The [sqlite3_wal_autocheckpoint()] interface and the
  7281   7293   ** [wal_autocheckpoint pragma] can be used to cause this interface to be
  7282   7294   ** run whenever the WAL reaches a certain size threshold.
  7283   7295   **
  7284   7296   ** See also: [sqlite3_wal_checkpoint_v2()]
................................................................................
  7293   7305   ** eMode parameter:
  7294   7306   **
  7295   7307   ** <dl>
  7296   7308   ** <dt>SQLITE_CHECKPOINT_PASSIVE<dd>
  7297   7309   **   Checkpoint as many frames as possible without waiting for any database 
  7298   7310   **   readers or writers to finish. Sync the db file if all frames in the log
  7299   7311   **   are checkpointed. This mode is the same as calling 
  7300         -**   sqlite3_wal_checkpoint(). The busy-handler callback is never invoked.
         7312  +**   sqlite3_wal_checkpoint(). The [sqlite3_busy_handler|busy-handler callback]
         7313  +**   is never invoked.
  7301   7314   **
  7302   7315   ** <dt>SQLITE_CHECKPOINT_FULL<dd>
  7303         -**   This mode blocks (calls the busy-handler callback) until there is no
         7316  +**   This mode blocks (it invokes the
         7317  +**   [sqlite3_busy_handler|busy-handler callback]) until there is no
  7304   7318   **   database writer and all readers are reading from the most recent database
  7305   7319   **   snapshot. It then checkpoints all frames in the log file and syncs the
  7306   7320   **   database file. This call blocks database writers while it is running,
  7307   7321   **   but not database readers.
  7308   7322   **
  7309   7323   ** <dt>SQLITE_CHECKPOINT_RESTART<dd>
  7310   7324   **   This mode works the same way as SQLITE_CHECKPOINT_FULL, except after 
  7311         -**   checkpointing the log file it blocks (calls the busy-handler callback)
         7325  +**   checkpointing the log file it blocks (calls the 
         7326  +**   [sqlite3_busy_handler|busy-handler callback])
  7312   7327   **   until all readers are reading from the database file only. This ensures 
  7313   7328   **   that the next client to write to the database file restarts the log file 
  7314   7329   **   from the beginning. This call blocks database writers while it is running,
  7315   7330   **   but not database readers.
  7316   7331   ** </dl>
  7317   7332   **
  7318   7333   ** If pnLog is not NULL, then *pnLog is set to the total number of frames in
................................................................................
  7442   7457   ** of the SQL statement that triggered the call to the [xUpdate] method of the
  7443   7458   ** [virtual table].
  7444   7459   */
  7445   7460   SQLITE_API int sqlite3_vtab_on_conflict(sqlite3 *);
  7446   7461   
  7447   7462   /*
  7448   7463   ** CAPI3REF: Conflict resolution modes
         7464  +** KEYWORDS: {conflict resolution mode}
  7449   7465   **
  7450   7466   ** These constants are returned by [sqlite3_vtab_on_conflict()] to
  7451   7467   ** inform a [virtual table] implementation what the [ON CONFLICT] mode
  7452   7468   ** is for the SQL statement being evaluated.
  7453   7469   **
  7454   7470   ** Note that the [SQLITE_IGNORE] constant is also used as a potential
  7455   7471   ** return value from the [sqlite3_set_authorizer()] callback and that
................................................................................
  9283   9299   #define OP_IfNot          45
  9284   9300   #define OP_Column         46 /* synopsis: r[P3]=PX                         */
  9285   9301   #define OP_Affinity       47 /* synopsis: affinity(r[P1@P2])               */
  9286   9302   #define OP_MakeRecord     48 /* synopsis: r[P3]=mkrec(r[P1@P2])            */
  9287   9303   #define OP_Count          49 /* synopsis: r[P2]=count()                    */
  9288   9304   #define OP_ReadCookie     50
  9289   9305   #define OP_SetCookie      51
  9290         -#define OP_OpenRead       52 /* synopsis: root=P2 iDb=P3                   */
  9291         -#define OP_OpenWrite      53 /* synopsis: root=P2 iDb=P3                   */
  9292         -#define OP_OpenAutoindex  54 /* synopsis: nColumn=P2                       */
  9293         -#define OP_OpenEphemeral  55 /* synopsis: nColumn=P2                       */
  9294         -#define OP_SorterOpen     56
  9295         -#define OP_OpenPseudo     57 /* synopsis: P3 columns in r[P2]              */
  9296         -#define OP_Close          58
  9297         -#define OP_SeekLT         59
  9298         -#define OP_SeekLE         60
  9299         -#define OP_SeekGE         61
  9300         -#define OP_SeekGT         62
  9301         -#define OP_Seek           63 /* synopsis: intkey=r[P2]                     */
  9302         -#define OP_NoConflict     64 /* synopsis: key=r[P3@P4]                     */
  9303         -#define OP_NotFound       65 /* synopsis: key=r[P3@P4]                     */
  9304         -#define OP_Found          66 /* synopsis: key=r[P3@P4]                     */
  9305         -#define OP_NotExists      67 /* synopsis: intkey=r[P3]                     */
  9306         -#define OP_Sequence       68 /* synopsis: r[P2]=cursor[P1].ctr++           */
  9307         -#define OP_NewRowid       69 /* synopsis: r[P2]=rowid                      */
  9308         -#define OP_Insert         70 /* synopsis: intkey=r[P3] data=r[P2]          */
         9306  +#define OP_ReopenIdx      52 /* synopsis: root=P2 iDb=P3                   */
         9307  +#define OP_OpenRead       53 /* synopsis: root=P2 iDb=P3                   */
         9308  +#define OP_OpenWrite      54 /* synopsis: root=P2 iDb=P3                   */
         9309  +#define OP_OpenAutoindex  55 /* synopsis: nColumn=P2                       */
         9310  +#define OP_OpenEphemeral  56 /* synopsis: nColumn=P2                       */
         9311  +#define OP_SorterOpen     57
         9312  +#define OP_OpenPseudo     58 /* synopsis: P3 columns in r[P2]              */
         9313  +#define OP_Close          59
         9314  +#define OP_SeekLT         60 /* synopsis: key=r[P3@P4]                     */
         9315  +#define OP_SeekLE         61 /* synopsis: key=r[P3@P4]                     */
         9316  +#define OP_SeekGE         62 /* synopsis: key=r[P3@P4]                     */
         9317  +#define OP_SeekGT         63 /* synopsis: key=r[P3@P4]                     */
         9318  +#define OP_Seek           64 /* synopsis: intkey=r[P2]                     */
         9319  +#define OP_NoConflict     65 /* synopsis: key=r[P3@P4]                     */
         9320  +#define OP_NotFound       66 /* synopsis: key=r[P3@P4]                     */
         9321  +#define OP_Found          67 /* synopsis: key=r[P3@P4]                     */
         9322  +#define OP_NotExists      68 /* synopsis: intkey=r[P3]                     */
         9323  +#define OP_Sequence       69 /* synopsis: r[P2]=cursor[P1].ctr++           */
         9324  +#define OP_NewRowid       70 /* synopsis: r[P2]=rowid                      */
  9309   9325   #define OP_Or             71 /* same as TK_OR, synopsis: r[P3]=(r[P1] || r[P2]) */
  9310   9326   #define OP_And            72 /* same as TK_AND, synopsis: r[P3]=(r[P1] && r[P2]) */
  9311         -#define OP_InsertInt      73 /* synopsis: intkey=P3 data=r[P2]             */
  9312         -#define OP_Delete         74
  9313         -#define OP_ResetCount     75
         9327  +#define OP_Insert         73 /* synopsis: intkey=r[P3] data=r[P2]          */
         9328  +#define OP_InsertInt      74 /* synopsis: intkey=P3 data=r[P2]             */
         9329  +#define OP_Delete         75
  9314   9330   #define OP_IsNull         76 /* same as TK_ISNULL, synopsis: if r[P1]==NULL goto P2 */
  9315   9331   #define OP_NotNull        77 /* same as TK_NOTNULL, synopsis: if r[P1]!=NULL goto P2 */
  9316   9332   #define OP_Ne             78 /* same as TK_NE, synopsis: if r[P1]!=r[P3] goto P2 */
  9317   9333   #define OP_Eq             79 /* same as TK_EQ, synopsis: if r[P1]==r[P3] goto P2 */
  9318   9334   #define OP_Gt             80 /* same as TK_GT, synopsis: if r[P1]>r[P3] goto P2 */
  9319   9335   #define OP_Le             81 /* same as TK_LE, synopsis: if r[P1]<=r[P3] goto P2 */
  9320   9336   #define OP_Lt             82 /* same as TK_LT, synopsis: if r[P1]<r[P3] goto P2 */
  9321   9337   #define OP_Ge             83 /* same as TK_GE, synopsis: if r[P1]>=r[P3] goto P2 */
  9322         -#define OP_SorterCompare  84 /* synopsis: if key(P1)!=rtrim(r[P3],P4) goto P2 */
         9338  +#define OP_ResetCount     84
  9323   9339   #define OP_BitAnd         85 /* same as TK_BITAND, synopsis: r[P3]=r[P1]&r[P2] */
  9324   9340   #define OP_BitOr          86 /* same as TK_BITOR, synopsis: r[P3]=r[P1]|r[P2] */
  9325   9341   #define OP_ShiftLeft      87 /* same as TK_LSHIFT, synopsis: r[P3]=r[P2]<<r[P1] */
  9326   9342   #define OP_ShiftRight     88 /* same as TK_RSHIFT, synopsis: r[P3]=r[P2]>>r[P1] */
  9327   9343   #define OP_Add            89 /* same as TK_PLUS, synopsis: r[P3]=r[P1]+r[P2] */
  9328   9344   #define OP_Subtract       90 /* same as TK_MINUS, synopsis: r[P3]=r[P2]-r[P1] */
  9329   9345   #define OP_Multiply       91 /* same as TK_STAR, synopsis: r[P3]=r[P1]*r[P2] */
  9330   9346   #define OP_Divide         92 /* same as TK_SLASH, synopsis: r[P3]=r[P2]/r[P1] */
  9331   9347   #define OP_Remainder      93 /* same as TK_REM, synopsis: r[P3]=r[P2]%r[P1] */
  9332   9348   #define OP_Concat         94 /* same as TK_CONCAT, synopsis: r[P3]=r[P2]+r[P1] */
  9333         -#define OP_SorterData     95 /* synopsis: r[P2]=data                       */
         9349  +#define OP_SorterCompare  95 /* synopsis: if key(P1)!=trim(r[P3],P4) goto P2 */
  9334   9350   #define OP_BitNot         96 /* same as TK_BITNOT, synopsis: r[P1]= ~r[P1] */
  9335   9351   #define OP_String8        97 /* same as TK_STRING, synopsis: r[P2]='P4'    */
  9336         -#define OP_RowKey         98 /* synopsis: r[P2]=key                        */
  9337         -#define OP_RowData        99 /* synopsis: r[P2]=data                       */
  9338         -#define OP_Rowid         100 /* synopsis: r[P2]=rowid                      */
  9339         -#define OP_NullRow       101
  9340         -#define OP_Last          102
  9341         -#define OP_SorterSort    103
  9342         -#define OP_Sort          104
  9343         -#define OP_Rewind        105
  9344         -#define OP_SorterInsert  106
  9345         -#define OP_IdxInsert     107 /* synopsis: key=r[P2]                        */
  9346         -#define OP_IdxDelete     108 /* synopsis: key=r[P2@P3]                     */
  9347         -#define OP_IdxRowid      109 /* synopsis: r[P2]=rowid                      */
  9348         -#define OP_IdxLE         110 /* synopsis: key=r[P3@P4]                     */
  9349         -#define OP_IdxGT         111 /* synopsis: key=r[P3@P4]                     */
  9350         -#define OP_IdxLT         112 /* synopsis: key=r[P3@P4]                     */
  9351         -#define OP_IdxGE         113 /* synopsis: key=r[P3@P4]                     */
  9352         -#define OP_Destroy       114
  9353         -#define OP_Clear         115
  9354         -#define OP_ResetSorter   116
  9355         -#define OP_CreateIndex   117 /* synopsis: r[P2]=root iDb=P1                */
  9356         -#define OP_CreateTable   118 /* synopsis: r[P2]=root iDb=P1                */
  9357         -#define OP_ParseSchema   119
  9358         -#define OP_LoadAnalysis  120
  9359         -#define OP_DropTable     121
  9360         -#define OP_DropIndex     122
  9361         -#define OP_DropTrigger   123
  9362         -#define OP_IntegrityCk   124
  9363         -#define OP_RowSetAdd     125 /* synopsis: rowset(P1)=r[P2]                 */
  9364         -#define OP_RowSetRead    126 /* synopsis: r[P3]=rowset(P1)                 */
  9365         -#define OP_RowSetTest    127 /* synopsis: if r[P3] in rowset(P1) goto P2   */
  9366         -#define OP_Program       128
  9367         -#define OP_Param         129
  9368         -#define OP_FkCounter     130 /* synopsis: fkctr[P1]+=P2                    */
  9369         -#define OP_FkIfZero      131 /* synopsis: if fkctr[P1]==0 goto P2          */
  9370         -#define OP_MemMax        132 /* synopsis: r[P1]=max(r[P1],r[P2])           */
         9352  +#define OP_SorterData     98 /* synopsis: r[P2]=data                       */
         9353  +#define OP_RowKey         99 /* synopsis: r[P2]=key                        */
         9354  +#define OP_RowData       100 /* synopsis: r[P2]=data                       */
         9355  +#define OP_Rowid         101 /* synopsis: r[P2]=rowid                      */
         9356  +#define OP_NullRow       102
         9357  +#define OP_Last          103
         9358  +#define OP_SorterSort    104
         9359  +#define OP_Sort          105
         9360  +#define OP_Rewind        106
         9361  +#define OP_SorterInsert  107
         9362  +#define OP_IdxInsert     108 /* synopsis: key=r[P2]                        */
         9363  +#define OP_IdxDelete     109 /* synopsis: key=r[P2@P3]                     */
         9364  +#define OP_IdxRowid      110 /* synopsis: r[P2]=rowid                      */
         9365  +#define OP_IdxLE         111 /* synopsis: key=r[P3@P4]                     */
         9366  +#define OP_IdxGT         112 /* synopsis: key=r[P3@P4]                     */
         9367  +#define OP_IdxLT         113 /* synopsis: key=r[P3@P4]                     */
         9368  +#define OP_IdxGE         114 /* synopsis: key=r[P3@P4]                     */
         9369  +#define OP_Destroy       115
         9370  +#define OP_Clear         116
         9371  +#define OP_ResetSorter   117
         9372  +#define OP_CreateIndex   118 /* synopsis: r[P2]=root iDb=P1                */
         9373  +#define OP_CreateTable   119 /* synopsis: r[P2]=root iDb=P1                */
         9374  +#define OP_ParseSchema   120
         9375  +#define OP_LoadAnalysis  121
         9376  +#define OP_DropTable     122
         9377  +#define OP_DropIndex     123
         9378  +#define OP_DropTrigger   124
         9379  +#define OP_IntegrityCk   125
         9380  +#define OP_RowSetAdd     126 /* synopsis: rowset(P1)=r[P2]                 */
         9381  +#define OP_RowSetRead    127 /* synopsis: r[P3]=rowset(P1)                 */
         9382  +#define OP_RowSetTest    128 /* synopsis: if r[P3] in rowset(P1) goto P2   */
         9383  +#define OP_Program       129
         9384  +#define OP_Param         130
         9385  +#define OP_FkCounter     131 /* synopsis: fkctr[P1]+=P2                    */
         9386  +#define OP_FkIfZero      132 /* synopsis: if fkctr[P1]==0 goto P2          */
  9371   9387   #define OP_Real          133 /* same as TK_FLOAT, synopsis: r[P2]=P4       */
  9372         -#define OP_IfPos         134 /* synopsis: if r[P1]>0 goto P2               */
  9373         -#define OP_IfNeg         135 /* synopsis: if r[P1]<0 goto P2               */
  9374         -#define OP_IfZero        136 /* synopsis: r[P1]+=P3, if r[P1]==0 goto P2   */
  9375         -#define OP_AggFinal      137 /* synopsis: accum=r[P1] N=P2                 */
  9376         -#define OP_IncrVacuum    138
  9377         -#define OP_Expire        139
  9378         -#define OP_TableLock     140 /* synopsis: iDb=P1 root=P2 write=P3          */
  9379         -#define OP_VBegin        141
  9380         -#define OP_VCreate       142
         9388  +#define OP_MemMax        134 /* synopsis: r[P1]=max(r[P1],r[P2])           */
         9389  +#define OP_IfPos         135 /* synopsis: if r[P1]>0 goto P2               */
         9390  +#define OP_IfNeg         136 /* synopsis: r[P1]+=P3, if r[P1]<0 goto P2    */
         9391  +#define OP_IfZero        137 /* synopsis: r[P1]+=P3, if r[P1]==0 goto P2   */
         9392  +#define OP_AggFinal      138 /* synopsis: accum=r[P1] N=P2                 */
         9393  +#define OP_IncrVacuum    139
         9394  +#define OP_Expire        140
         9395  +#define OP_TableLock     141 /* synopsis: iDb=P1 root=P2 write=P3          */
         9396  +#define OP_VBegin        142
  9381   9397   #define OP_ToText        143 /* same as TK_TO_TEXT                         */
  9382   9398   #define OP_ToBlob        144 /* same as TK_TO_BLOB                         */
  9383   9399   #define OP_ToNumeric     145 /* same as TK_TO_NUMERIC                      */
  9384   9400   #define OP_ToInt         146 /* same as TK_TO_INT                          */
  9385   9401   #define OP_ToReal        147 /* same as TK_TO_REAL                         */
  9386         -#define OP_VDestroy      148
  9387         -#define OP_VOpen         149
  9388         -#define OP_VColumn       150 /* synopsis: r[P3]=vcolumn(P2)                */
  9389         -#define OP_VNext         151
  9390         -#define OP_VRename       152
  9391         -#define OP_Pagecount     153
  9392         -#define OP_MaxPgcnt      154
  9393         -#define OP_Init          155 /* synopsis: Start at P2                      */
  9394         -#define OP_Noop          156
  9395         -#define OP_Explain       157
         9402  +#define OP_VCreate       148
         9403  +#define OP_VDestroy      149
         9404  +#define OP_VOpen         150
         9405  +#define OP_VColumn       151 /* synopsis: r[P3]=vcolumn(P2)                */
         9406  +#define OP_VNext         152
         9407  +#define OP_VRename       153
         9408  +#define OP_Pagecount     154
         9409  +#define OP_MaxPgcnt      155
         9410  +#define OP_Init          156 /* synopsis: Start at P2                      */
         9411  +#define OP_Noop          157
         9412  +#define OP_Explain       158
  9396   9413   
  9397   9414   
  9398   9415   /* Properties such as "out2" or "jump" that are specified in
  9399   9416   ** comments following the "case" for each opcode in the vdbe.c
  9400   9417   ** are encoded into bitvectors as follows:
  9401   9418   */
  9402   9419   #define OPFLG_JUMP            0x0001  /* jump:  P2 holds jmp target */
................................................................................
  9410   9427   /*   0 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x01, 0x01, 0x01,\
  9411   9428   /*   8 */ 0x01, 0x01, 0x00, 0x00, 0x02, 0x00, 0x01, 0x00,\
  9412   9429   /*  16 */ 0x01, 0x01, 0x04, 0x24, 0x01, 0x04, 0x05, 0x10,\
  9413   9430   /*  24 */ 0x00, 0x02, 0x02, 0x02, 0x02, 0x00, 0x02, 0x02,\
  9414   9431   /*  32 */ 0x00, 0x00, 0x20, 0x00, 0x00, 0x04, 0x05, 0x04,\
  9415   9432   /*  40 */ 0x00, 0x00, 0x01, 0x01, 0x05, 0x05, 0x00, 0x00,\
  9416   9433   /*  48 */ 0x00, 0x02, 0x02, 0x10, 0x00, 0x00, 0x00, 0x00,\
  9417         -/*  56 */ 0x00, 0x00, 0x00, 0x11, 0x11, 0x11, 0x11, 0x08,\
  9418         -/*  64 */ 0x11, 0x11, 0x11, 0x11, 0x02, 0x02, 0x00, 0x4c,\
         9434  +/*  56 */ 0x00, 0x00, 0x00, 0x00, 0x11, 0x11, 0x11, 0x11,\
         9435  +/*  64 */ 0x08, 0x11, 0x11, 0x11, 0x11, 0x02, 0x02, 0x4c,\
  9419   9436   /*  72 */ 0x4c, 0x00, 0x00, 0x00, 0x05, 0x05, 0x15, 0x15,\
  9420   9437   /*  80 */ 0x15, 0x15, 0x15, 0x15, 0x00, 0x4c, 0x4c, 0x4c,\
  9421   9438   /*  88 */ 0x4c, 0x4c, 0x4c, 0x4c, 0x4c, 0x4c, 0x4c, 0x00,\
  9422         -/*  96 */ 0x24, 0x02, 0x00, 0x00, 0x02, 0x00, 0x01, 0x01,\
  9423         -/* 104 */ 0x01, 0x01, 0x08, 0x08, 0x00, 0x02, 0x01, 0x01,\
  9424         -/* 112 */ 0x01, 0x01, 0x02, 0x00, 0x00, 0x02, 0x02, 0x00,\
  9425         -/* 120 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, 0x45, 0x15,\
  9426         -/* 128 */ 0x01, 0x02, 0x00, 0x01, 0x08, 0x02, 0x05, 0x05,\
  9427         -/* 136 */ 0x05, 0x00, 0x01, 0x00, 0x00, 0x00, 0x00, 0x04,\
  9428         -/* 144 */ 0x04, 0x04, 0x04, 0x04, 0x00, 0x00, 0x00, 0x01,\
  9429         -/* 152 */ 0x00, 0x02, 0x02, 0x01, 0x00, 0x00,}
         9439  +/*  96 */ 0x24, 0x02, 0x00, 0x00, 0x00, 0x02, 0x00, 0x01,\
         9440  +/* 104 */ 0x01, 0x01, 0x01, 0x08, 0x08, 0x00, 0x02, 0x01,\
         9441  +/* 112 */ 0x01, 0x01, 0x01, 0x02, 0x00, 0x00, 0x02, 0x02,\
         9442  +/* 120 */ 0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x0c, 0x45,\
         9443  +/* 128 */ 0x15, 0x01, 0x02, 0x00, 0x01, 0x02, 0x08, 0x05,\
         9444  +/* 136 */ 0x05, 0x05, 0x00, 0x01, 0x00, 0x00, 0x00, 0x04,\
         9445  +/* 144 */ 0x04, 0x04, 0x04, 0x04, 0x00, 0x00, 0x00, 0x00,\
         9446  +/* 152 */ 0x01, 0x00, 0x02, 0x02, 0x01, 0x00, 0x00,}
  9430   9447   
  9431   9448   /************** End of opcodes.h *********************************************/
  9432   9449   /************** Continuing where we left off in vdbe.h ***********************/
  9433   9450   
  9434   9451   /*
  9435   9452   ** Prototypes for the VDBE interface.  See comments on the implementation
  9436   9453   ** for a description of what each of these routines does.
................................................................................
  9477   9494   SQLITE_PRIVATE void sqlite3VdbeSwap(Vdbe*,Vdbe*);
  9478   9495   SQLITE_PRIVATE VdbeOp *sqlite3VdbeTakeOpArray(Vdbe*, int*, int*);
  9479   9496   SQLITE_PRIVATE sqlite3_value *sqlite3VdbeGetBoundValue(Vdbe*, int, u8);
  9480   9497   SQLITE_PRIVATE void sqlite3VdbeSetVarmask(Vdbe*, int);
  9481   9498   #ifndef SQLITE_OMIT_TRACE
  9482   9499   SQLITE_PRIVATE   char *sqlite3VdbeExpandSql(Vdbe*, const char*);
  9483   9500   #endif
         9501  +SQLITE_PRIVATE int sqlite3MemCompare(const Mem*, const Mem*, const CollSeq*);
  9484   9502   
  9485   9503   SQLITE_PRIVATE void sqlite3VdbeRecordUnpack(KeyInfo*,int,const void*,UnpackedRecord*);
  9486   9504   SQLITE_PRIVATE int sqlite3VdbeRecordCompare(int,const void*,UnpackedRecord*,int);
  9487   9505   SQLITE_PRIVATE UnpackedRecord *sqlite3VdbeAllocUnpackedRecord(KeyInfo *, char *, int, char **);
  9488   9506   
  9489   9507   typedef int (*RecordCompare)(int,const void*,UnpackedRecord*,int);
  9490   9508   SQLITE_PRIVATE RecordCompare sqlite3VdbeFindCompare(UnpackedRecord*);
................................................................................
 10328  10346     Hash tblHash;        /* All tables indexed by name */
 10329  10347     Hash idxHash;        /* All (named) indices indexed by name */
 10330  10348     Hash trigHash;       /* All triggers indexed by name */
 10331  10349     Hash fkeyHash;       /* All foreign keys by referenced table name */
 10332  10350     Table *pSeqTab;      /* The sqlite_sequence table used by AUTOINCREMENT */
 10333  10351     u8 file_format;      /* Schema format version for this file */
 10334  10352     u8 enc;              /* Text encoding used by this database */
 10335         -  u16 flags;           /* Flags associated with this schema */
        10353  +  u16 schemaFlags;     /* Flags associated with this schema */
 10336  10354     int cache_size;      /* Number of pages to use in the cache */
 10337  10355   };
 10338  10356   
 10339  10357   /*
 10340  10358   ** These macros can be used to test, set, or clear bits in the 
 10341  10359   ** Db.pSchema->flags field.
 10342  10360   */
 10343         -#define DbHasProperty(D,I,P)     (((D)->aDb[I].pSchema->flags&(P))==(P))
 10344         -#define DbHasAnyProperty(D,I,P)  (((D)->aDb[I].pSchema->flags&(P))!=0)
 10345         -#define DbSetProperty(D,I,P)     (D)->aDb[I].pSchema->flags|=(P)
 10346         -#define DbClearProperty(D,I,P)   (D)->aDb[I].pSchema->flags&=~(P)
        10361  +#define DbHasProperty(D,I,P)     (((D)->aDb[I].pSchema->schemaFlags&(P))==(P))
        10362  +#define DbHasAnyProperty(D,I,P)  (((D)->aDb[I].pSchema->schemaFlags&(P))!=0)
        10363  +#define DbSetProperty(D,I,P)     (D)->aDb[I].pSchema->schemaFlags|=(P)
        10364  +#define DbClearProperty(D,I,P)   (D)->aDb[I].pSchema->schemaFlags&=~(P)
 10347  10365   
 10348  10366   /*
 10349  10367   ** Allowed values for the DB.pSchema->flags field.
 10350  10368   **
 10351  10369   ** The DB_SchemaLoaded flag is set after the database schema has been
 10352  10370   ** read into internal hash tables.
 10353  10371   **
................................................................................
 10929  10947   #endif
 10930  10948     LogEst nRowLogEst;   /* Estimated rows in table - from sqlite_stat1 table */
 10931  10949     int tnum;            /* Root BTree node for this table (see note above) */
 10932  10950     i16 iPKey;           /* If not negative, use aCol[iPKey] as the primary key */
 10933  10951     i16 nCol;            /* Number of columns in this table */
 10934  10952     u16 nRef;            /* Number of pointers to this Table */
 10935  10953     LogEst szTabRow;     /* Estimated size of each table row in bytes */
        10954  +#ifdef SQLITE_ENABLE_COSTMULT
        10955  +  LogEst costMult;     /* Cost multiplier for using this table */
        10956  +#endif
 10936  10957     u8 tabFlags;         /* Mask of TF_* values */
 10937  10958     u8 keyConf;          /* What to do in case of uniqueness conflict on iPKey */
 10938  10959   #ifndef SQLITE_OMIT_ALTERTABLE
 10939  10960     int addColOffset;    /* Offset in CREATE TABLE stmt to add a new column */
 10940  10961   #endif
 10941  10962   #ifndef SQLITE_OMIT_VIRTUALTABLE
 10942  10963     int nModuleArg;      /* Number of arguments to the module */
................................................................................
 11169  11190   #define SQLITE_IDXTYPE_APPDEF      0   /* Created using CREATE INDEX */
 11170  11191   #define SQLITE_IDXTYPE_UNIQUE      1   /* Implements a UNIQUE constraint */
 11171  11192   #define SQLITE_IDXTYPE_PRIMARYKEY  2   /* Is the PRIMARY KEY for the table */
 11172  11193   
 11173  11194   /* Return true if index X is a PRIMARY KEY index */
 11174  11195   #define IsPrimaryKeyIndex(X)  ((X)->idxType==SQLITE_IDXTYPE_PRIMARYKEY)
 11175  11196   
        11197  +/* Return true if index X is a UNIQUE index */
        11198  +#define IsUniqueIndex(X)      ((X)->onError!=OE_None)
        11199  +
 11176  11200   /*
 11177  11201   ** Each sample stored in the sqlite_stat3 table is represented in memory 
 11178  11202   ** using a structure of this type.  See documentation at the top of the
 11179  11203   ** analyze.c source file for additional information.
 11180  11204   */
 11181  11205   struct IndexSample {
 11182  11206     void *p;          /* Pointer to sampled record */
................................................................................
 11588  11612   #define WHERE_FORCE_TABLE      0x0020 /* Do not use an index-only search */
 11589  11613   #define WHERE_ONETABLE_ONLY    0x0040 /* Only code the 1st table in pTabList */
 11590  11614   #define WHERE_AND_ONLY         0x0080 /* Don't use indices for OR terms */
 11591  11615   #define WHERE_GROUPBY          0x0100 /* pOrderBy is really a GROUP BY */
 11592  11616   #define WHERE_DISTINCTBY       0x0200 /* pOrderby is really a DISTINCT clause */
 11593  11617   #define WHERE_WANT_DISTINCT    0x0400 /* All output needs to be distinct */
 11594  11618   #define WHERE_SORTBYGROUP      0x0800 /* Support sqlite3WhereIsSorted() */
        11619  +#define WHERE_REOPEN_IDX       0x1000 /* Try to use OP_ReopenIdx */
 11595  11620   
 11596  11621   /* Allowed return values from sqlite3WhereIsDistinct()
 11597  11622   */
 11598  11623   #define WHERE_DISTINCT_NOOP      0  /* DISTINCT keyword not used */
 11599  11624   #define WHERE_DISTINCT_UNIQUE    1  /* No duplicates */
 11600  11625   #define WHERE_DISTINCT_ORDERED   2  /* All duplicates are adjacent */
 11601  11626   #define WHERE_DISTINCT_UNORDERED 3  /* Duplicates are scattered */
................................................................................
 11844  11869     u32 aColmask[2];        /* Masks of old.*, new.* columns accessed */
 11845  11870   };
 11846  11871   
 11847  11872   /*
 11848  11873   ** The yDbMask datatype for the bitmask of all attached databases.
 11849  11874   */
 11850  11875   #if SQLITE_MAX_ATTACHED>30
 11851         -  typedef sqlite3_uint64 yDbMask;
        11876  +  typedef unsigned char yDbMask[(SQLITE_MAX_ATTACHED+9)/8];
        11877  +# define DbMaskTest(M,I)    (((M)[(I)/8]&(1<<((I)&7)))!=0)
        11878  +# define DbMaskZero(M)      memset((M),0,sizeof(M))
        11879  +# define DbMaskSet(M,I)     (M)[(I)/8]|=(1<<((I)&7))
        11880  +# define DbMaskAllZero(M)   sqlite3DbMaskAllZero(M)
        11881  +# define DbMaskNonZero(M)   (sqlite3DbMaskAllZero(M)==0)
 11852  11882   #else
 11853  11883     typedef unsigned int yDbMask;
        11884  +# define DbMaskTest(M,I)    (((M)&(((yDbMask)1)<<(I)))!=0)
        11885  +# define DbMaskZero(M)      (M)=0
        11886  +# define DbMaskSet(M,I)     (M)|=(((yDbMask)1)<<(I))
        11887  +# define DbMaskAllZero(M)   (M)==0
        11888  +# define DbMaskNonZero(M)   (M)!=0
 11854  11889   #endif
 11855  11890   
 11856  11891   /*
 11857  11892   ** An SQL parser context.  A copy of this structure is passed through
 11858  11893   ** the parser and down into all the parser action routine in order to
 11859  11894   ** carry around information that is global to the entire parse.
 11860  11895   **
................................................................................
 12519  12554   
 12520  12555   #if !defined(SQLITE_OMIT_VIEW) || !defined(SQLITE_OMIT_VIRTUALTABLE)
 12521  12556   SQLITE_PRIVATE   int sqlite3ViewGetColumnNames(Parse*,Table*);
 12522  12557   #else
 12523  12558   # define sqlite3ViewGetColumnNames(A,B) 0
 12524  12559   #endif
 12525  12560   
        12561  +#if SQLITE_MAX_ATTACHED>30
        12562  +SQLITE_PRIVATE   int sqlite3DbMaskAllZero(yDbMask);
        12563  +#endif
 12526  12564   SQLITE_PRIVATE void sqlite3DropTable(Parse*, SrcList*, int, int);
 12527  12565   SQLITE_PRIVATE void sqlite3CodeDropTable(Parse*, Table*, int, int);
 12528  12566   SQLITE_PRIVATE void sqlite3DeleteTable(sqlite3*, Table*);
 12529  12567   #ifndef SQLITE_OMIT_AUTOINCREMENT
 12530  12568   SQLITE_PRIVATE   void sqlite3AutoincrementBegin(Parse *pParse);
 12531  12569   SQLITE_PRIVATE   void sqlite3AutoincrementEnd(Parse *pParse);
 12532  12570   #else
................................................................................
 12769  12807   
 12770  12808   SQLITE_PRIVATE const char *sqlite3IndexAffinityStr(Vdbe *, Index *);
 12771  12809   SQLITE_PRIVATE void sqlite3TableAffinity(Vdbe*, Table*, int);
 12772  12810   SQLITE_PRIVATE char sqlite3CompareAffinity(Expr *pExpr, char aff2);
 12773  12811   SQLITE_PRIVATE int sqlite3IndexAffinityOk(Expr *pExpr, char idx_affinity);
 12774  12812   SQLITE_PRIVATE char sqlite3ExprAffinity(Expr *pExpr);
 12775  12813   SQLITE_PRIVATE int sqlite3Atoi64(const char*, i64*, int, u8);
        12814  +SQLITE_PRIVATE int sqlite3DecOrHexToI64(const char*, i64*);
 12776  12815   SQLITE_PRIVATE void sqlite3Error(sqlite3*, int, const char*,...);
 12777  12816   SQLITE_PRIVATE void *sqlite3HexToBlob(sqlite3*, const char *z, int n);
 12778  12817   SQLITE_PRIVATE u8 sqlite3HexToInt(int h);
 12779  12818   SQLITE_PRIVATE int sqlite3TwoPartName(Parse *, Token *, Token *, Token **);
 12780  12819   
 12781  12820   #if defined(SQLITE_TEST) 
 12782  12821   SQLITE_PRIVATE const char *sqlite3ErrName(int);
................................................................................
 12798  12837   SQLITE_PRIVATE int sqlite3MulInt64(i64*,i64);
 12799  12838   SQLITE_PRIVATE int sqlite3AbsInt32(int);
 12800  12839   #ifdef SQLITE_ENABLE_8_3_NAMES
 12801  12840   SQLITE_PRIVATE void sqlite3FileSuffix3(const char*, char*);
 12802  12841   #else
 12803  12842   # define sqlite3FileSuffix3(X,Y)
 12804  12843   #endif
 12805         -SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z,int);
        12844  +SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z,u8);
 12806  12845   
 12807  12846   SQLITE_PRIVATE const void *sqlite3ValueText(sqlite3_value*, u8);
 12808  12847   SQLITE_PRIVATE int sqlite3ValueBytes(sqlite3_value*, u8);
 12809  12848   SQLITE_PRIVATE void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8, 
 12810  12849                           void(*)(void*));
 12811  12850   SQLITE_PRIVATE void sqlite3ValueSetNull(sqlite3_value*);
 12812  12851   SQLITE_PRIVATE void sqlite3ValueFree(sqlite3_value*);
................................................................................
 12883  12922   
 12884  12923   SQLITE_PRIVATE void sqlite3BackupRestart(sqlite3_backup *);
 12885  12924   SQLITE_PRIVATE void sqlite3BackupUpdate(sqlite3_backup *, Pgno, const u8 *);
 12886  12925   
 12887  12926   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
 12888  12927   SQLITE_PRIVATE void sqlite3AnalyzeFunctions(void);
 12889  12928   SQLITE_PRIVATE int sqlite3Stat4ProbeSetValue(Parse*,Index*,UnpackedRecord**,Expr*,u8,int,int*);
        12929  +SQLITE_PRIVATE int sqlite3Stat4ValueFromExpr(Parse*, Expr*, u8, sqlite3_value**);
 12890  12930   SQLITE_PRIVATE void sqlite3Stat4ProbeFree(UnpackedRecord*);
        12931  +SQLITE_PRIVATE int sqlite3Stat4Column(sqlite3*, const void*, int, int, sqlite3_value**);
 12891  12932   #endif
 12892  12933   
 12893  12934   /*
 12894  12935   ** The interface to the LEMON-generated parser
 12895  12936   */
 12896  12937   SQLITE_PRIVATE void *sqlite3ParserAlloc(void*(*)(size_t));
 12897  12938   SQLITE_PRIVATE void sqlite3ParserFree(void*, void(*)(void*));
................................................................................
 13020  13061   SQLITE_PRIVATE   void sqlite3BeginBenignMalloc(void);
 13021  13062   SQLITE_PRIVATE   void sqlite3EndBenignMalloc(void);
 13022  13063   #else
 13023  13064     #define sqlite3BeginBenignMalloc()
 13024  13065     #define sqlite3EndBenignMalloc()
 13025  13066   #endif
 13026  13067   
 13027         -#define IN_INDEX_ROWID           1
 13028         -#define IN_INDEX_EPH             2
 13029         -#define IN_INDEX_INDEX_ASC       3
 13030         -#define IN_INDEX_INDEX_DESC      4
 13031         -SQLITE_PRIVATE int sqlite3FindInIndex(Parse *, Expr *, int*);
        13068  +/*
        13069  +** Allowed return values from sqlite3FindInIndex()
        13070  +*/
        13071  +#define IN_INDEX_ROWID        1   /* Search the rowid of the table */
        13072  +#define IN_INDEX_EPH          2   /* Search an ephemeral b-tree */
        13073  +#define IN_INDEX_INDEX_ASC    3   /* Existing index ASCENDING */
        13074  +#define IN_INDEX_INDEX_DESC   4   /* Existing index DESCENDING */
        13075  +#define IN_INDEX_NOOP         5   /* No table available. Use comparisons */
        13076  +/*
        13077  +** Allowed flags for the 3rd parameter to sqlite3FindInIndex().
        13078  +*/
        13079  +#define IN_INDEX_NOOP_OK     0x0001  /* OK to return IN_INDEX_NOOP */
        13080  +#define IN_INDEX_MEMBERSHIP  0x0002  /* IN operator used for membership test */
        13081  +#define IN_INDEX_LOOP        0x0004  /* IN operator used as a loop */
        13082  +SQLITE_PRIVATE int sqlite3FindInIndex(Parse *, Expr *, u32, int*);
 13032  13083   
 13033  13084   #ifdef SQLITE_ENABLE_ATOMIC_WRITE
 13034  13085   SQLITE_PRIVATE   int sqlite3JournalOpen(sqlite3_vfs *, const char *, sqlite3_file *, int, int);
 13035  13086   SQLITE_PRIVATE   int sqlite3JournalSize(sqlite3_vfs *);
 13036  13087   SQLITE_PRIVATE   int sqlite3JournalCreate(sqlite3_file *);
 13037  13088   SQLITE_PRIVATE   int sqlite3JournalExists(sqlite3_file *p);
 13038  13089   #else
................................................................................
 13871  13922     BtCursor *pCursor;    /* The cursor structure of the backend */
 13872  13923     Btree *pBt;           /* Separate file holding temporary table */
 13873  13924     KeyInfo *pKeyInfo;    /* Info about index keys needed by index cursors */
 13874  13925     int seekResult;       /* Result of previous sqlite3BtreeMoveto() */
 13875  13926     int pseudoTableReg;   /* Register holding pseudotable content. */
 13876  13927     i16 nField;           /* Number of fields in the header */
 13877  13928     u16 nHdrParsed;       /* Number of header fields parsed so far */
        13929  +#ifdef SQLITE_DEBUG
        13930  +  u8 seekOp;            /* Most recent seek operation on this cursor */
        13931  +#endif
 13878  13932     i8 iDb;               /* Index of cursor database in db->aDb[] (or -1) */
 13879  13933     u8 nullRow;           /* True if pointing to a row with no data */
 13880  13934     u8 rowidIsValid;      /* True if lastRowid is valid */
 13881  13935     u8 deferredMoveto;    /* A call to sqlite3BtreeMoveto() is needed */
 13882  13936     Bool isEphemeral:1;   /* True for an ephemeral table */
 13883  13937     Bool useRandomRowid:1;/* Generate new record numbers semi-randomly */
 13884  13938     Bool isTable:1;       /* True if a table requiring integer keys */
 13885  13939     Bool isOrdered:1;     /* True if the underlying table is BTREE_UNORDERED */
        13940  +  Pgno pgnoRoot;        /* Root page of the open btree cursor */
 13886  13941     sqlite3_vtab_cursor *pVtabCursor;  /* The cursor for a virtual table */
 13887  13942     i64 seqCount;         /* Sequence counter */
 13888  13943     i64 movetoTarget;     /* Argument to the deferred sqlite3BtreeMoveto() */
 13889  13944     i64 lastRowid;        /* Rowid being deleted by OP_Delete */
 13890  13945     VdbeSorter *pSorter;  /* Sorter object for OP_SorterOpen cursors */
 13891  13946   
 13892  13947     /* Cached information about the header for the data record that the
................................................................................
 14197  14252   SQLITE_PRIVATE u32 sqlite3VdbeSerialPut(unsigned char*, Mem*, u32);
 14198  14253   SQLITE_PRIVATE u32 sqlite3VdbeSerialGet(const unsigned char*, u32, Mem*);
 14199  14254   SQLITE_PRIVATE void sqlite3VdbeDeleteAuxData(Vdbe*, int, int);
 14200  14255   
 14201  14256   int sqlite2BtreeKeyCompare(BtCursor *, const void *, int, int, int *);
 14202  14257   SQLITE_PRIVATE int sqlite3VdbeIdxKeyCompare(VdbeCursor*,UnpackedRecord*,int*);
 14203  14258   SQLITE_PRIVATE int sqlite3VdbeIdxRowid(sqlite3*, BtCursor *, i64 *);
 14204         -SQLITE_PRIVATE int sqlite3MemCompare(const Mem*, const Mem*, const CollSeq*);
 14205  14259   SQLITE_PRIVATE int sqlite3VdbeExec(Vdbe*);
 14206  14260   SQLITE_PRIVATE int sqlite3VdbeList(Vdbe*);
 14207  14261   SQLITE_PRIVATE int sqlite3VdbeHalt(Vdbe*);
 14208  14262   SQLITE_PRIVATE int sqlite3VdbeChangeEncoding(Mem *, int);
 14209  14263   SQLITE_PRIVATE int sqlite3VdbeMemTooBig(Mem*);
 14210  14264   SQLITE_PRIVATE int sqlite3VdbeMemCopy(Mem*, const Mem*);
 14211  14265   SQLITE_PRIVATE void sqlite3VdbeMemShallowCopy(Mem*, const Mem*, int);
................................................................................
 18392  18446   }
 18393  18447   
 18394  18448   /*
 18395  18449   ** Retrieve a pointer to a static mutex or allocate a new dynamic one.
 18396  18450   */
 18397  18451   SQLITE_API sqlite3_mutex *sqlite3_mutex_alloc(int id){
 18398  18452   #ifndef SQLITE_OMIT_AUTOINIT
 18399         -  if( sqlite3_initialize() ) return 0;
        18453  +  if( id<=SQLITE_MUTEX_RECURSIVE && sqlite3_initialize() ) return 0;
 18400  18454   #endif
 18401  18455     return sqlite3GlobalConfig.mutex.xMutexAlloc(id);
 18402  18456   }
 18403  18457   
 18404  18458   SQLITE_PRIVATE sqlite3_mutex *sqlite3MutexAlloc(int id){
 18405  18459     if( !sqlite3GlobalConfig.bCoreMutex ){
 18406  18460       return 0;
................................................................................
 18573  18627   
 18574  18628   /*
 18575  18629   ** The sqlite3_mutex_alloc() routine allocates a new
 18576  18630   ** mutex and returns a pointer to it.  If it returns NULL
 18577  18631   ** that means that a mutex could not be allocated. 
 18578  18632   */
 18579  18633   static sqlite3_mutex *debugMutexAlloc(int id){
 18580         -  static sqlite3_debug_mutex aStatic[6];
        18634  +  static sqlite3_debug_mutex aStatic[SQLITE_MUTEX_STATIC_APP3 - 1];
 18581  18635     sqlite3_debug_mutex *pNew = 0;
 18582  18636     switch( id ){
 18583  18637       case SQLITE_MUTEX_FAST:
 18584  18638       case SQLITE_MUTEX_RECURSIVE: {
 18585  18639         pNew = sqlite3Malloc(sizeof(*pNew));
 18586  18640         if( pNew ){
 18587  18641           pNew->id = id;
................................................................................
 18770  18824   ** to sqlite3_mutex_alloc() is one of these integer constants:
 18771  18825   **
 18772  18826   ** <ul>
 18773  18827   ** <li>  SQLITE_MUTEX_FAST
 18774  18828   ** <li>  SQLITE_MUTEX_RECURSIVE
 18775  18829   ** <li>  SQLITE_MUTEX_STATIC_MASTER
 18776  18830   ** <li>  SQLITE_MUTEX_STATIC_MEM
 18777         -** <li>  SQLITE_MUTEX_STATIC_MEM2
        18831  +** <li>  SQLITE_MUTEX_STATIC_OPEN
 18778  18832   ** <li>  SQLITE_MUTEX_STATIC_PRNG
 18779  18833   ** <li>  SQLITE_MUTEX_STATIC_LRU
 18780  18834   ** <li>  SQLITE_MUTEX_STATIC_PMEM
        18835  +** <li>  SQLITE_MUTEX_STATIC_APP1
        18836  +** <li>  SQLITE_MUTEX_STATIC_APP2
        18837  +** <li>  SQLITE_MUTEX_STATIC_APP3
 18781  18838   ** </ul>
 18782  18839   **
 18783  18840   ** The first two constants cause sqlite3_mutex_alloc() to create
 18784  18841   ** a new mutex.  The new mutex is recursive when SQLITE_MUTEX_RECURSIVE
 18785  18842   ** is used but not necessarily so when SQLITE_MUTEX_FAST is used.
 18786  18843   ** The mutex implementation does not need to make a distinction
 18787  18844   ** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does
................................................................................
 18802  18859   ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc()
 18803  18860   ** returns a different mutex on every call.  But for the static 
 18804  18861   ** mutex types, the same mutex is returned on every call that has
 18805  18862   ** the same type number.
 18806  18863   */
 18807  18864   static sqlite3_mutex *pthreadMutexAlloc(int iType){
 18808  18865     static sqlite3_mutex staticMutexes[] = {
        18866  +    SQLITE3_MUTEX_INITIALIZER,
        18867  +    SQLITE3_MUTEX_INITIALIZER,
        18868  +    SQLITE3_MUTEX_INITIALIZER,
 18809  18869       SQLITE3_MUTEX_INITIALIZER,
 18810  18870       SQLITE3_MUTEX_INITIALIZER,
 18811  18871       SQLITE3_MUTEX_INITIALIZER,
 18812  18872       SQLITE3_MUTEX_INITIALIZER,
 18813  18873       SQLITE3_MUTEX_INITIALIZER,
 18814  18874       SQLITE3_MUTEX_INITIALIZER
 18815  18875     };
................................................................................
 19037  19097   ** a legal notice, here is a blessing:
 19038  19098   **
 19039  19099   **    May you do good and not evil.
 19040  19100   **    May you find forgiveness for yourself and forgive others.
 19041  19101   **    May you share freely, never taking more than you give.
 19042  19102   **
 19043  19103   *************************************************************************
 19044         -** This file contains the C functions that implement mutexes for win32
        19104  +** This file contains the C functions that implement mutexes for Win32.
 19045  19105   */
 19046  19106   
 19047  19107   #if SQLITE_OS_WIN
        19108  +/*
        19109  +** Include code that is common to all os_*.c files
        19110  +*/
        19111  +/************** Include os_common.h in the middle of mutex_w32.c *************/
        19112  +/************** Begin file os_common.h ***************************************/
        19113  +/*
        19114  +** 2004 May 22
        19115  +**
        19116  +** The author disclaims copyright to this source code.  In place of
        19117  +** a legal notice, here is a blessing:
        19118  +**
        19119  +**    May you do good and not evil.
        19120  +**    May you find forgiveness for yourself and forgive others.
        19121  +**    May you share freely, never taking more than you give.
        19122  +**
        19123  +******************************************************************************
        19124  +**
        19125  +** This file contains macros and a little bit of code that is common to
        19126  +** all of the platform-specific files (os_*.c) and is #included into those
        19127  +** files.
        19128  +**
        19129  +** This file should be #included by the os_*.c files only.  It is not a
        19130  +** general purpose header file.
        19131  +*/
        19132  +#ifndef _OS_COMMON_H_
        19133  +#define _OS_COMMON_H_
        19134  +
        19135  +/*
        19136  +** At least two bugs have slipped in because we changed the MEMORY_DEBUG
        19137  +** macro to SQLITE_DEBUG and some older makefiles have not yet made the
        19138  +** switch.  The following code should catch this problem at compile-time.
        19139  +*/
        19140  +#ifdef MEMORY_DEBUG
        19141  +# error "The MEMORY_DEBUG macro is obsolete.  Use SQLITE_DEBUG instead."
        19142  +#endif
        19143  +
        19144  +#if defined(SQLITE_TEST) && defined(SQLITE_DEBUG)
        19145  +# ifndef SQLITE_DEBUG_OS_TRACE
        19146  +#   define SQLITE_DEBUG_OS_TRACE 0
        19147  +# endif
        19148  +  int sqlite3OSTrace = SQLITE_DEBUG_OS_TRACE;
        19149  +# define OSTRACE(X)          if( sqlite3OSTrace ) sqlite3DebugPrintf X
        19150  +#else
        19151  +# define OSTRACE(X)
        19152  +#endif
        19153  +
        19154  +/*
        19155  +** Macros for performance tracing.  Normally turned off.  Only works
        19156  +** on i486 hardware.
        19157  +*/
        19158  +#ifdef SQLITE_PERFORMANCE_TRACE
        19159  +
        19160  +/* 
        19161  +** hwtime.h contains inline assembler code for implementing 
        19162  +** high-performance timing routines.
        19163  +*/
        19164  +/************** Include hwtime.h in the middle of os_common.h ****************/
        19165  +/************** Begin file hwtime.h ******************************************/
        19166  +/*
        19167  +** 2008 May 27
        19168  +**
        19169  +** The author disclaims copyright to this source code.  In place of
        19170  +** a legal notice, here is a blessing:
        19171  +**
        19172  +**    May you do good and not evil.
        19173  +**    May you find forgiveness for yourself and forgive others.
        19174  +**    May you share freely, never taking more than you give.
        19175  +**
        19176  +******************************************************************************
        19177  +**
        19178  +** This file contains inline asm code for retrieving "high-performance"
        19179  +** counters for x86 class CPUs.
        19180  +*/
        19181  +#ifndef _HWTIME_H_
        19182  +#define _HWTIME_H_
        19183  +
        19184  +/*
        19185  +** The following routine only works on pentium-class (or newer) processors.
        19186  +** It uses the RDTSC opcode to read the cycle count value out of the
        19187  +** processor and returns that value.  This can be used for high-res
        19188  +** profiling.
        19189  +*/
        19190  +#if (defined(__GNUC__) || defined(_MSC_VER)) && \
        19191  +      (defined(i386) || defined(__i386__) || defined(_M_IX86))
        19192  +
        19193  +  #if defined(__GNUC__)
        19194  +
        19195  +  __inline__ sqlite_uint64 sqlite3Hwtime(void){
        19196  +     unsigned int lo, hi;
        19197  +     __asm__ __volatile__ ("rdtsc" : "=a" (lo), "=d" (hi));
        19198  +     return (sqlite_uint64)hi << 32 | lo;
        19199  +  }
        19200  +
        19201  +  #elif defined(_MSC_VER)
        19202  +
        19203  +  __declspec(naked) __inline sqlite_uint64 __cdecl sqlite3Hwtime(void){
        19204  +     __asm {
        19205  +        rdtsc
        19206  +        ret       ; return value at EDX:EAX
        19207  +     }
        19208  +  }
        19209  +
        19210  +  #endif
        19211  +
        19212  +#elif (defined(__GNUC__) && defined(__x86_64__))
        19213  +
        19214  +  __inline__ sqlite_uint64 sqlite3Hwtime(void){
        19215  +      unsigned long val;
        19216  +      __asm__ __volatile__ ("rdtsc" : "=A" (val));
        19217  +      return val;
        19218  +  }
        19219  + 
        19220  +#elif (defined(__GNUC__) && defined(__ppc__))
        19221  +
        19222  +  __inline__ sqlite_uint64 sqlite3Hwtime(void){
        19223  +      unsigned long long retval;
        19224  +      unsigned long junk;
        19225  +      __asm__ __volatile__ ("\n\
        19226  +          1:      mftbu   %1\n\
        19227  +                  mftb    %L0\n\
        19228  +                  mftbu   %0\n\
        19229  +                  cmpw    %0,%1\n\
        19230  +                  bne     1b"
        19231  +                  : "=r" (retval), "=r" (junk));
        19232  +      return retval;
        19233  +  }
        19234  +
        19235  +#else
        19236  +
        19237  +  #error Need implementation of sqlite3Hwtime() for your platform.
        19238  +
        19239  +  /*
        19240  +  ** To compile without implementing sqlite3Hwtime() for your platform,
        19241  +  ** you can remove the above #error and use the following
        19242  +  ** stub function.  You will lose timing support for many
        19243  +  ** of the debugging and testing utilities, but it should at
        19244  +  ** least compile and run.
        19245  +  */
        19246  +SQLITE_PRIVATE   sqlite_uint64 sqlite3Hwtime(void){ return ((sqlite_uint64)0); }
        19247  +
        19248  +#endif
        19249  +
        19250  +#endif /* !defined(_HWTIME_H_) */
        19251  +
        19252  +/************** End of hwtime.h **********************************************/
        19253  +/************** Continuing where we left off in os_common.h ******************/
        19254  +
        19255  +static sqlite_uint64 g_start;
        19256  +static sqlite_uint64 g_elapsed;
        19257  +#define TIMER_START       g_start=sqlite3Hwtime()
        19258  +#define TIMER_END         g_elapsed=sqlite3Hwtime()-g_start
        19259  +#define TIMER_ELAPSED     g_elapsed
        19260  +#else
        19261  +#define TIMER_START
        19262  +#define TIMER_END
        19263  +#define TIMER_ELAPSED     ((sqlite_uint64)0)
        19264  +#endif
        19265  +
        19266  +/*
        19267  +** If we compile with the SQLITE_TEST macro set, then the following block
        19268  +** of code will give us the ability to simulate a disk I/O error.  This
        19269  +** is used for testing the I/O recovery logic.
        19270  +*/
        19271  +#ifdef SQLITE_TEST
        19272  +SQLITE_API int sqlite3_io_error_hit = 0;            /* Total number of I/O Errors */
        19273  +SQLITE_API int sqlite3_io_error_hardhit = 0;        /* Number of non-benign errors */
        19274  +SQLITE_API int sqlite3_io_error_pending = 0;        /* Count down to first I/O error */
        19275  +SQLITE_API int sqlite3_io_error_persist = 0;        /* True if I/O errors persist */
        19276  +SQLITE_API int sqlite3_io_error_benign = 0;         /* True if errors are benign */
        19277  +SQLITE_API int sqlite3_diskfull_pending = 0;
        19278  +SQLITE_API int sqlite3_diskfull = 0;
        19279  +#define SimulateIOErrorBenign(X) sqlite3_io_error_benign=(X)
        19280  +#define SimulateIOError(CODE)  \
        19281  +  if( (sqlite3_io_error_persist && sqlite3_io_error_hit) \
        19282  +       || sqlite3_io_error_pending-- == 1 )  \
        19283  +              { local_ioerr(); CODE; }
        19284  +static void local_ioerr(){
        19285  +  IOTRACE(("IOERR\n"));
        19286  +  sqlite3_io_error_hit++;
        19287  +  if( !sqlite3_io_error_benign ) sqlite3_io_error_hardhit++;
        19288  +}
        19289  +#define SimulateDiskfullError(CODE) \
        19290  +   if( sqlite3_diskfull_pending ){ \
        19291  +     if( sqlite3_diskfull_pending == 1 ){ \
        19292  +       local_ioerr(); \
        19293  +       sqlite3_diskfull = 1; \
        19294  +       sqlite3_io_error_hit = 1; \
        19295  +       CODE; \
        19296  +     }else{ \
        19297  +       sqlite3_diskfull_pending--; \
        19298  +     } \
        19299  +   }
        19300  +#else
        19301  +#define SimulateIOErrorBenign(X)
        19302  +#define SimulateIOError(A)
        19303  +#define SimulateDiskfullError(A)
        19304  +#endif
        19305  +
        19306  +/*
        19307  +** When testing, keep a count of the number of open files.
        19308  +*/
        19309  +#ifdef SQLITE_TEST
        19310  +SQLITE_API int sqlite3_open_file_count = 0;
        19311  +#define OpenCounter(X)  sqlite3_open_file_count+=(X)
        19312  +#else
        19313  +#define OpenCounter(X)
        19314  +#endif
        19315  +
        19316  +#endif /* !defined(_OS_COMMON_H_) */
        19317  +
        19318  +/************** End of os_common.h *******************************************/
        19319  +/************** Continuing where we left off in mutex_w32.c ******************/
        19320  +
 19048  19321   /*
 19049  19322   ** Include the header file for the Windows VFS.
 19050  19323   */
 19051  19324   /************** Include os_win.h in the middle of mutex_w32.c ****************/
 19052  19325   /************** Begin file os_win.h ******************************************/
 19053  19326   /*
 19054  19327   ** 2013 November 25
................................................................................
 19120  19393   
 19121  19394   /************** End of os_win.h **********************************************/
 19122  19395   /************** Continuing where we left off in mutex_w32.c ******************/
 19123  19396   #endif
 19124  19397   
 19125  19398   /*
 19126  19399   ** The code in this file is only used if we are compiling multithreaded
 19127         -** on a win32 system.
        19400  +** on a Win32 system.
 19128  19401   */
 19129  19402   #ifdef SQLITE_MUTEX_W32
 19130  19403   
 19131  19404   /*
 19132  19405   ** Each recursive mutex is an instance of the following structure.
 19133  19406   */
 19134  19407   struct sqlite3_mutex {
 19135  19408     CRITICAL_SECTION mutex;    /* Mutex controlling the lock */
 19136  19409     int id;                    /* Mutex type */
 19137  19410   #ifdef SQLITE_DEBUG
 19138  19411     volatile int nRef;         /* Number of enterances */
 19139  19412     volatile DWORD owner;      /* Thread holding this mutex */
 19140         -  int trace;                 /* True to trace changes */
        19413  +  volatile int trace;        /* True to trace changes */
 19141  19414   #endif
 19142  19415   };
        19416  +
        19417  +/*
        19418  +** These are the initializer values used when declaring a "static" mutex
        19419  +** on Win32.  It should be noted that all mutexes require initialization
        19420  +** on the Win32 platform.
        19421  +*/
 19143  19422   #define SQLITE_W32_MUTEX_INITIALIZER { 0 }
        19423  +
 19144  19424   #ifdef SQLITE_DEBUG
 19145         -#define SQLITE3_MUTEX_INITIALIZER { SQLITE_W32_MUTEX_INITIALIZER, 0, 0L, (DWORD)0, 0 }
        19425  +#define SQLITE3_MUTEX_INITIALIZER { SQLITE_W32_MUTEX_INITIALIZER, 0, \
        19426  +                                    0L, (DWORD)0, 0 }
 19146  19427   #else
 19147  19428   #define SQLITE3_MUTEX_INITIALIZER { SQLITE_W32_MUTEX_INITIALIZER, 0 }
 19148  19429   #endif
 19149  19430   
 19150         -/*
 19151         -** Return true (non-zero) if we are running under WinNT, Win2K, WinXP,
 19152         -** or WinCE.  Return false (zero) for Win95, Win98, or WinME.
 19153         -**
 19154         -** Here is an interesting observation:  Win95, Win98, and WinME lack
 19155         -** the LockFileEx() API.  But we can still statically link against that
 19156         -** API as long as we don't call it win running Win95/98/ME.  A call to
 19157         -** this routine is used to determine if the host is Win95/98/ME or
 19158         -** WinNT/2K/XP so that we will know whether or not we can safely call
 19159         -** the LockFileEx() API.
 19160         -**
 19161         -** mutexIsNT() is only used for the TryEnterCriticalSection() API call,
 19162         -** which is only available if your application was compiled with 
 19163         -** _WIN32_WINNT defined to a value >= 0x0400.  Currently, the only
 19164         -** call to TryEnterCriticalSection() is #ifdef'ed out, so #ifdef 
 19165         -** this out as well.
 19166         -*/
 19167         -#if 0
 19168         -#if SQLITE_OS_WINCE || SQLITE_OS_WINRT
 19169         -# define mutexIsNT()  (1)
 19170         -#else
 19171         -  static int mutexIsNT(void){
 19172         -    static int osType = 0;
 19173         -    if( osType==0 ){
 19174         -      OSVERSIONINFO sInfo;
 19175         -      sInfo.dwOSVersionInfoSize = sizeof(sInfo);
 19176         -      GetVersionEx(&sInfo);
 19177         -      osType = sInfo.dwPlatformId==VER_PLATFORM_WIN32_NT ? 2 : 1;
 19178         -    }
 19179         -    return osType==2;
 19180         -  }
 19181         -#endif /* SQLITE_OS_WINCE || SQLITE_OS_WINRT */
 19182         -#endif
 19183         -
 19184  19431   #ifdef SQLITE_DEBUG
 19185  19432   /*
 19186  19433   ** The sqlite3_mutex_held() and sqlite3_mutex_notheld() routine are
 19187  19434   ** intended for use only inside assert() statements.
 19188  19435   */
 19189  19436   static int winMutexHeld(sqlite3_mutex *p){
 19190  19437     return p->nRef!=0 && p->owner==GetCurrentThreadId();
 19191  19438   }
        19439  +
 19192  19440   static int winMutexNotheld2(sqlite3_mutex *p, DWORD tid){
 19193  19441     return p->nRef==0 || p->owner!=tid;
 19194  19442   }
        19443  +
 19195  19444   static int winMutexNotheld(sqlite3_mutex *p){
 19196         -  DWORD tid = GetCurrentThreadId(); 
        19445  +  DWORD tid = GetCurrentThreadId();
 19197  19446     return winMutexNotheld2(p, tid);
 19198  19447   }
 19199  19448   #endif
 19200  19449   
 19201         -
 19202  19450   /*
 19203  19451   ** Initialize and deinitialize the mutex subsystem.
 19204  19452   */
 19205         -static sqlite3_mutex winMutex_staticMutexes[6] = {
        19453  +static sqlite3_mutex winMutex_staticMutexes[] = {
        19454  +  SQLITE3_MUTEX_INITIALIZER,
        19455  +  SQLITE3_MUTEX_INITIALIZER,
        19456  +  SQLITE3_MUTEX_INITIALIZER,
 19206  19457     SQLITE3_MUTEX_INITIALIZER,
 19207  19458     SQLITE3_MUTEX_INITIALIZER,
 19208  19459     SQLITE3_MUTEX_INITIALIZER,
 19209  19460     SQLITE3_MUTEX_INITIALIZER,
 19210  19461     SQLITE3_MUTEX_INITIALIZER,
 19211  19462     SQLITE3_MUTEX_INITIALIZER
 19212  19463   };
        19464  +
 19213  19465   static int winMutex_isInit = 0;
 19214         -/* As winMutexInit() and winMutexEnd() are called as part
 19215         -** of the sqlite3_initialize and sqlite3_shutdown()
 19216         -** processing, the "interlocked" magic is probably not
 19217         -** strictly necessary.
        19466  +static int winMutex_isNt = -1; /* <0 means "need to query" */
        19467  +
        19468  +/* As the winMutexInit() and winMutexEnd() functions are called as part
        19469  +** of the sqlite3_initialize() and sqlite3_shutdown() processing, the
        19470  +** "interlocked" magic used here is probably not strictly necessary.
 19218  19471   */
 19219         -static LONG winMutex_lock = 0;
        19472  +static LONG volatile winMutex_lock = 0;
 19220  19473   
        19474  +SQLITE_API int sqlite3_win32_is_nt(void); /* os_win.c */
 19221  19475   SQLITE_API void sqlite3_win32_sleep(DWORD milliseconds); /* os_win.c */
 19222  19476   
 19223         -static int winMutexInit(void){ 
        19477  +static int winMutexInit(void){
 19224  19478     /* The first to increment to 1 does actual initialization */
 19225  19479     if( InterlockedCompareExchange(&winMutex_lock, 1, 0)==0 ){
 19226  19480       int i;
 19227  19481       for(i=0; i<ArraySize(winMutex_staticMutexes); i++){
 19228  19482   #if SQLITE_OS_WINRT
 19229  19483         InitializeCriticalSectionEx(&winMutex_staticMutexes[i].mutex, 0, 0);
 19230  19484   #else
 19231  19485         InitializeCriticalSection(&winMutex_staticMutexes[i].mutex);
 19232  19486   #endif
 19233  19487       }
 19234  19488       winMutex_isInit = 1;
 19235  19489     }else{
 19236         -    /* Someone else is in the process of initing the static mutexes */
        19490  +    /* Another thread is (in the process of) initializing the static
        19491  +    ** mutexes */
 19237  19492       while( !winMutex_isInit ){
 19238  19493         sqlite3_win32_sleep(1);
 19239  19494       }
 19240  19495     }
 19241         -  return SQLITE_OK; 
        19496  +  return SQLITE_OK;
 19242  19497   }
 19243  19498   
 19244         -static int winMutexEnd(void){ 
 19245         -  /* The first to decrement to 0 does actual shutdown 
        19499  +static int winMutexEnd(void){
        19500  +  /* The first to decrement to 0 does actual shutdown
 19246  19501     ** (which should be the last to shutdown.) */
 19247  19502     if( InterlockedCompareExchange(&winMutex_lock, 0, 1)==1 ){
 19248  19503       if( winMutex_isInit==1 ){
 19249  19504         int i;
 19250  19505         for(i=0; i<ArraySize(winMutex_staticMutexes); i++){
 19251  19506           DeleteCriticalSection(&winMutex_staticMutexes[i].mutex);
 19252  19507         }
 19253  19508         winMutex_isInit = 0;
 19254  19509       }
 19255  19510     }
 19256         -  return SQLITE_OK; 
        19511  +  return SQLITE_OK;
 19257  19512   }
 19258  19513   
 19259  19514   /*
 19260  19515   ** The sqlite3_mutex_alloc() routine allocates a new
 19261  19516   ** mutex and returns a pointer to it.  If it returns NULL
 19262  19517   ** that means that a mutex could not be allocated.  SQLite
 19263  19518   ** will unwind its stack and return an error.  The argument
................................................................................
 19264  19519   ** to sqlite3_mutex_alloc() is one of these integer constants:
 19265  19520   **
 19266  19521   ** <ul>
 19267  19522   ** <li>  SQLITE_MUTEX_FAST
 19268  19523   ** <li>  SQLITE_MUTEX_RECURSIVE
 19269  19524   ** <li>  SQLITE_MUTEX_STATIC_MASTER
 19270  19525   ** <li>  SQLITE_MUTEX_STATIC_MEM
 19271         -** <li>  SQLITE_MUTEX_STATIC_MEM2
        19526  +** <li>  SQLITE_MUTEX_STATIC_OPEN
 19272  19527   ** <li>  SQLITE_MUTEX_STATIC_PRNG
 19273  19528   ** <li>  SQLITE_MUTEX_STATIC_LRU
 19274  19529   ** <li>  SQLITE_MUTEX_STATIC_PMEM
        19530  +** <li>  SQLITE_MUTEX_STATIC_APP1
        19531  +** <li>  SQLITE_MUTEX_STATIC_APP2
        19532  +** <li>  SQLITE_MUTEX_STATIC_APP3
 19275  19533   ** </ul>
 19276  19534   **
 19277  19535   ** The first two constants cause sqlite3_mutex_alloc() to create
 19278  19536   ** a new mutex.  The new mutex is recursive when SQLITE_MUTEX_RECURSIVE
 19279  19537   ** is used but not necessarily so when SQLITE_MUTEX_FAST is used.
 19280  19538   ** The mutex implementation does not need to make a distinction
 19281  19539   ** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does
................................................................................
 19290  19548   ** may add additional static mutexes.  Static mutexes are for internal
 19291  19549   ** use by SQLite only.  Applications that use SQLite mutexes should
 19292  19550   ** use only the dynamic mutexes returned by SQLITE_MUTEX_FAST or
 19293  19551   ** SQLITE_MUTEX_RECURSIVE.
 19294  19552   **
 19295  19553   ** Note that if one of the dynamic mutex parameters (SQLITE_MUTEX_FAST
 19296  19554   ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc()
 19297         -** returns a different mutex on every call.  But for the static 
        19555  +** returns a different mutex on every call.  But for the static
 19298  19556   ** mutex types, the same mutex is returned on every call that has
 19299  19557   ** the same type number.
 19300  19558   */
 19301  19559   static sqlite3_mutex *winMutexAlloc(int iType){
 19302  19560     sqlite3_mutex *p;
 19303  19561   
 19304  19562     switch( iType ){
 19305  19563       case SQLITE_MUTEX_FAST:
 19306  19564       case SQLITE_MUTEX_RECURSIVE: {
 19307  19565         p = sqlite3MallocZero( sizeof(*p) );
 19308         -      if( p ){  
        19566  +      if( p ){
 19309  19567   #ifdef SQLITE_DEBUG
 19310  19568           p->id = iType;
        19569  +#ifdef SQLITE_WIN32_MUTEX_TRACE_DYNAMIC
        19570  +        p->trace = 1;
        19571  +#endif
 19311  19572   #endif
 19312  19573   #if SQLITE_OS_WINRT
 19313  19574           InitializeCriticalSectionEx(&p->mutex, 0, 0);
 19314  19575   #else
 19315  19576           InitializeCriticalSection(&p->mutex);
 19316  19577   #endif
 19317  19578         }
 19318  19579         break;
 19319  19580       }
 19320  19581       default: {
 19321         -      assert( winMutex_isInit==1 );
 19322  19582         assert( iType-2 >= 0 );
 19323  19583         assert( iType-2 < ArraySize(winMutex_staticMutexes) );
        19584  +      assert( winMutex_isInit==1 );
 19324  19585         p = &winMutex_staticMutexes[iType-2];
 19325  19586   #ifdef SQLITE_DEBUG
 19326  19587         p->id = iType;
        19588  +#ifdef SQLITE_WIN32_MUTEX_TRACE_STATIC
        19589  +      p->trace = 1;
        19590  +#endif
 19327  19591   #endif
 19328  19592         break;
 19329  19593       }
 19330  19594     }
 19331  19595     return p;
 19332  19596   }
 19333  19597   
................................................................................
 19335  19599   /*
 19336  19600   ** This routine deallocates a previously
 19337  19601   ** allocated mutex.  SQLite is careful to deallocate every
 19338  19602   ** mutex that it allocates.
 19339  19603   */
 19340  19604   static void winMutexFree(sqlite3_mutex *p){
 19341  19605     assert( p );
        19606  +#ifdef SQLITE_DEBUG
 19342  19607     assert( p->nRef==0 && p->owner==0 );
 19343  19608     assert( p->id==SQLITE_MUTEX_FAST || p->id==SQLITE_MUTEX_RECURSIVE );
        19609  +#endif
        19610  +  assert( winMutex_isInit==1 );
 19344  19611     DeleteCriticalSection(&p->mutex);
 19345  19612     sqlite3_free(p);
 19346  19613   }
 19347  19614   
 19348  19615   /*
 19349  19616   ** The sqlite3_mutex_enter() and sqlite3_mutex_try() routines attempt
 19350  19617   ** to enter a mutex.  If another thread is already within the mutex,
................................................................................
 19353  19620   ** upon successful entry.  Mutexes created using SQLITE_MUTEX_RECURSIVE can
 19354  19621   ** be entered multiple times by the same thread.  In such cases the,
 19355  19622   ** mutex must be exited an equal number of times before another thread
 19356  19623   ** can enter.  If the same thread tries to enter any other kind of mutex
 19357  19624   ** more than once, the behavior is undefined.
 19358  19625   */
 19359  19626   static void winMutexEnter(sqlite3_mutex *p){
        19627  +#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
        19628  +  DWORD tid = GetCurrentThreadId();
        19629  +#endif
 19360  19630   #ifdef SQLITE_DEBUG
 19361         -  DWORD tid = GetCurrentThreadId(); 
        19631  +  assert( p );
 19362  19632     assert( p->id==SQLITE_MUTEX_RECURSIVE || winMutexNotheld2(p, tid) );
        19633  +#else
        19634  +  assert( p );
 19363  19635   #endif
        19636  +  assert( winMutex_isInit==1 );
 19364  19637     EnterCriticalSection(&p->mutex);
 19365  19638   #ifdef SQLITE_DEBUG
 19366  19639     assert( p->nRef>0 || p->owner==0 );
 19367         -  p->owner = tid; 
        19640  +  p->owner = tid;
 19368  19641     p->nRef++;
 19369  19642     if( p->trace ){
 19370         -    printf("enter mutex %p (%d) with nRef=%d\n", p, p->trace, p->nRef);
        19643  +    OSTRACE(("ENTER-MUTEX tid=%lu, mutex=%p (%d), nRef=%d\n",
        19644  +             tid, p, p->trace, p->nRef));
 19371  19645     }
 19372  19646   #endif
 19373  19647   }
        19648  +
 19374  19649   static int winMutexTry(sqlite3_mutex *p){
 19375         -#ifndef NDEBUG
 19376         -  DWORD tid = GetCurrentThreadId(); 
        19650  +#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
        19651  +  DWORD tid = GetCurrentThreadId();
 19377  19652   #endif
 19378  19653     int rc = SQLITE_BUSY;
        19654  +  assert( p );
 19379  19655     assert( p->id==SQLITE_MUTEX_RECURSIVE || winMutexNotheld2(p, tid) );
 19380  19656     /*
 19381  19657     ** The sqlite3_mutex_try() routine is very rarely used, and when it
 19382  19658     ** is used it is merely an optimization.  So it is OK for it to always
 19383         -  ** fail.  
        19659  +  ** fail.
 19384  19660     **
 19385  19661     ** The TryEnterCriticalSection() interface is only available on WinNT.
 19386  19662     ** And some windows compilers complain if you try to use it without
 19387  19663     ** first doing some #defines that prevent SQLite from building on Win98.
 19388  19664     ** For that reason, we will omit this optimization for now.  See
 19389  19665     ** ticket #2685.
 19390  19666     */
 19391         -#if 0
 19392         -  if( mutexIsNT() && TryEnterCriticalSection(&p->mutex) ){
        19667  +#if defined(_WIN32_WINNT) && _WIN32_WINNT >= 0x0400
        19668  +  assert( winMutex_isInit==1 );
        19669  +  assert( winMutex_isNt>=-1 && winMutex_isNt<=1 );
        19670  +  if( winMutex_isNt<0 ){
        19671  +    winMutex_isNt = sqlite3_win32_is_nt();
        19672  +  }
        19673  +  assert( winMutex_isNt==0 || winMutex_isNt==1 );
        19674  +  if( winMutex_isNt && TryEnterCriticalSection(&p->mutex) ){
        19675  +#ifdef SQLITE_DEBUG
 19393  19676       p->owner = tid;
 19394  19677       p->nRef++;
        19678  +#endif
 19395  19679       rc = SQLITE_OK;
 19396  19680     }
 19397  19681   #else
 19398  19682     UNUSED_PARAMETER(p);
 19399  19683   #endif
 19400  19684   #ifdef SQLITE_DEBUG
 19401         -  if( rc==SQLITE_OK && p->trace ){
 19402         -    printf("try mutex %p (%d) with nRef=%d\n", p, p->trace, p->nRef);
        19685  +  if( p->trace ){
        19686  +    OSTRACE(("TRY-MUTEX tid=%lu, mutex=%p (%d), owner=%lu, nRef=%d, rc=%s\n",
        19687  +             tid, p, p->trace, p->owner, p->nRef, sqlite3ErrName(rc)));
 19403  19688     }
 19404  19689   #endif
 19405  19690     return rc;
 19406  19691   }
 19407  19692   
 19408  19693   /*
 19409  19694   ** The sqlite3_mutex_leave() routine exits a mutex that was
 19410  19695   ** previously entered by the same thread.  The behavior
 19411  19696   ** is undefined if the mutex is not currently entered or
 19412  19697   ** is not currently allocated.  SQLite will never do either.
 19413  19698   */
 19414  19699   static void winMutexLeave(sqlite3_mutex *p){
 19415         -#ifndef NDEBUG
        19700  +#if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
 19416  19701     DWORD tid = GetCurrentThreadId();
        19702  +#endif
        19703  +  assert( p );
        19704  +#ifdef SQLITE_DEBUG
 19417  19705     assert( p->nRef>0 );
 19418  19706     assert( p->owner==tid );
 19419  19707     p->nRef--;
 19420  19708     if( p->nRef==0 ) p->owner = 0;
 19421  19709     assert( p->nRef==0 || p->id==SQLITE_MUTEX_RECURSIVE );
 19422  19710   #endif
        19711  +  assert( winMutex_isInit==1 );
 19423  19712     LeaveCriticalSection(&p->mutex);
 19424  19713   #ifdef SQLITE_DEBUG
 19425  19714     if( p->trace ){
 19426         -    printf("leave mutex %p (%d) with nRef=%d\n", p, p->trace, p->nRef);
        19715  +    OSTRACE(("LEAVE-MUTEX tid=%lu, mutex=%p (%d), nRef=%d\n",
        19716  +             tid, p, p->trace, p->nRef));
 19427  19717     }
 19428  19718   #endif
 19429  19719   }
 19430  19720   
 19431  19721   SQLITE_PRIVATE sqlite3_mutex_methods const *sqlite3DefaultMutex(void){
 19432  19722     static const sqlite3_mutex_methods sMutex = {
 19433  19723       winMutexInit,
................................................................................
 19441  19731       winMutexHeld,
 19442  19732       winMutexNotheld
 19443  19733   #else
 19444  19734       0,
 19445  19735       0
 19446  19736   #endif
 19447  19737     };
 19448         -
 19449  19738     return &sMutex;
 19450  19739   }
        19740  +
 19451  19741   #endif /* SQLITE_MUTEX_W32 */
 19452  19742   
 19453  19743   /************** End of mutex_w32.c *******************************************/
 19454  19744   /************** Begin file malloc.c ******************************************/
 19455  19745   /*
 19456  19746   ** 2001 September 15
 19457  19747   **
................................................................................
 21560  21850   **     0xd800 and 0xe000 then it is rendered as 0xfffd.
 21561  21851   **
 21562  21852   **  *  Bytes in the range of 0x80 through 0xbf which occur as the first
 21563  21853   **     byte of a character are interpreted as single-byte characters
 21564  21854   **     and rendered as themselves even though they are technically
 21565  21855   **     invalid characters.
 21566  21856   **
 21567         -**  *  This routine accepts an infinite number of different UTF8 encodings
 21568         -**     for unicode values 0x80 and greater.  It do not change over-length
        21857  +**  *  This routine accepts over-length UTF8 encodings
        21858  +**     for unicode values 0x80 and greater.  It does not change over-length
 21569  21859   **     encodings to 0xfffd as some systems recommend.
 21570  21860   */
 21571  21861   #define READ_UTF8(zIn, zTerm, c)                           \
 21572  21862     c = *(zIn++);                                            \
 21573  21863     if( c>=0xc0 ){                                           \
 21574  21864       c = sqlite3Utf8Trans1[c-0xc0];                         \
 21575  21865       while( zIn!=zTerm && (*zIn & 0xc0)==0x80 ){            \
................................................................................
 22418  22708       testcase( c==(-1) );
 22419  22709       testcase( c==0 );
 22420  22710       testcase( c==(+1) );
 22421  22711     }
 22422  22712     return c;
 22423  22713   }
 22424  22714   
 22425         -
 22426  22715   /*
 22427         -** Convert zNum to a 64-bit signed integer.
        22716  +** Convert zNum to a 64-bit signed integer.  zNum must be decimal. This
        22717  +** routine does *not* accept hexadecimal notation.
 22428  22718   **
 22429  22719   ** If the zNum value is representable as a 64-bit twos-complement 
 22430  22720   ** integer, then write that value into *pNum and return 0.
 22431  22721   **
 22432  22722   ** If zNum is exactly 9223372036854775808, return 2.  This special
 22433  22723   ** case is broken out because while 9223372036854775808 cannot be a 
 22434  22724   ** signed 64-bit integer, its negative -9223372036854775808 can be.
................................................................................
 22507  22797         /* zNum is exactly 9223372036854775808.  Fits if negative.  The
 22508  22798         ** special case 2 overflow if positive */
 22509  22799         assert( u-1==LARGEST_INT64 );
 22510  22800         return neg ? 0 : 2;
 22511  22801       }
 22512  22802     }
 22513  22803   }
        22804  +
        22805  +/*
        22806  +** Transform a UTF-8 integer literal, in either decimal or hexadecimal,
        22807  +** into a 64-bit signed integer.  This routine accepts hexadecimal literals,
        22808  +** whereas sqlite3Atoi64() does not.
        22809  +**
        22810  +** Returns:
        22811  +**
        22812  +**     0    Successful transformation.  Fits in a 64-bit signed integer.
        22813  +**     1    Integer too large for a 64-bit signed integer or is malformed
        22814  +**     2    Special case of 9223372036854775808
        22815  +*/
        22816  +SQLITE_PRIVATE int sqlite3DecOrHexToI64(const char *z, i64 *pOut){
        22817  +#ifndef SQLITE_OMIT_HEX_INTEGER
        22818  +  if( z[0]=='0'
        22819  +   && (z[1]=='x' || z[1]=='X')
        22820  +   && sqlite3Isxdigit(z[2])
        22821  +  ){
        22822  +    u64 u = 0;
        22823  +    int i, k;
        22824  +    for(i=2; z[i]=='0'; i++){}
        22825  +    for(k=i; sqlite3Isxdigit(z[k]); k++){
        22826  +      u = u*16 + sqlite3HexToInt(z[k]);
        22827  +    }
        22828  +    memcpy(pOut, &u, 8);
        22829  +    return (z[k]==0 && k-i<=16) ? 0 : 1;
        22830  +  }else
        22831  +#endif /* SQLITE_OMIT_HEX_INTEGER */
        22832  +  {
        22833  +    return sqlite3Atoi64(z, pOut, sqlite3Strlen30(z), SQLITE_UTF8);
        22834  +  }
        22835  +}
 22514  22836   
 22515  22837   /*
 22516  22838   ** If zNum represents an integer that will fit in 32-bits, then set
 22517  22839   ** *pValue to that integer and return true.  Otherwise return false.
        22840  +**
        22841  +** This routine accepts both decimal and hexadecimal notation for integers.
 22518  22842   **
 22519  22843   ** Any non-numeric characters that following zNum are ignored.
 22520  22844   ** This is different from sqlite3Atoi64() which requires the
 22521  22845   ** input number to be zero-terminated.
 22522  22846   */
 22523  22847   SQLITE_PRIVATE int sqlite3GetInt32(const char *zNum, int *pValue){
 22524  22848     sqlite_int64 v = 0;
................................................................................
 22526  22850     int neg = 0;
 22527  22851     if( zNum[0]=='-' ){
 22528  22852       neg = 1;
 22529  22853       zNum++;
 22530  22854     }else if( zNum[0]=='+' ){
 22531  22855       zNum++;
 22532  22856     }
 22533         -  while( zNum[0]=='0' ) zNum++;
        22857  +#ifndef SQLITE_OMIT_HEX_INTEGER
        22858  +  else if( zNum[0]=='0'
        22859  +        && (zNum[1]=='x' || zNum[1]=='X')
        22860  +        && sqlite3Isxdigit(zNum[2])
        22861  +  ){
        22862  +    u32 u = 0;
        22863  +    zNum += 2;
        22864  +    while( zNum[0]=='0' ) zNum++;
        22865  +    for(i=0; sqlite3Isxdigit(zNum[i]) && i<8; i++){
        22866  +      u = u*16 + sqlite3HexToInt(zNum[i]);
        22867  +    }
        22868  +    if( (u&0x80000000)==0 && sqlite3Isxdigit(zNum[i])==0 ){
        22869  +      memcpy(pValue, &u, 4);
        22870  +      return 1;
        22871  +    }else{
        22872  +      return 0;
        22873  +    }
        22874  +  }
        22875  +#endif
 22534  22876     for(i=0; i<11 && (c = zNum[i] - '0')>=0 && c<=9; i++){
 22535  22877       v = v*10 + c;
 22536  22878     }
 22537  22879   
 22538  22880     /* The longest decimal representation of a 32 bit integer is 10 digits:
 22539  22881     **
 22540  22882     **             1234567890
................................................................................
 23602  23944        /*  45 */ "IfNot"            OpHelp(""),
 23603  23945        /*  46 */ "Column"           OpHelp("r[P3]=PX"),
 23604  23946        /*  47 */ "Affinity"         OpHelp("affinity(r[P1@P2])"),
 23605  23947        /*  48 */ "MakeRecord"       OpHelp("r[P3]=mkrec(r[P1@P2])"),
 23606  23948        /*  49 */ "Count"            OpHelp("r[P2]=count()"),
 23607  23949        /*  50 */ "ReadCookie"       OpHelp(""),
 23608  23950        /*  51 */ "SetCookie"        OpHelp(""),
 23609         -     /*  52 */ "OpenRead"         OpHelp("root=P2 iDb=P3"),
 23610         -     /*  53 */ "OpenWrite"        OpHelp("root=P2 iDb=P3"),
 23611         -     /*  54 */ "OpenAutoindex"    OpHelp("nColumn=P2"),
 23612         -     /*  55 */ "OpenEphemeral"    OpHelp("nColumn=P2"),
 23613         -     /*  56 */ "SorterOpen"       OpHelp(""),
 23614         -     /*  57 */ "OpenPseudo"       OpHelp("P3 columns in r[P2]"),
 23615         -     /*  58 */ "Close"            OpHelp(""),
 23616         -     /*  59 */ "SeekLT"           OpHelp(""),
 23617         -     /*  60 */ "SeekLE"           OpHelp(""),
 23618         -     /*  61 */ "SeekGE"           OpHelp(""),
 23619         -     /*  62 */ "SeekGT"           OpHelp(""),
 23620         -     /*  63 */ "Seek"             OpHelp("intkey=r[P2]"),
 23621         -     /*  64 */ "NoConflict"       OpHelp("key=r[P3@P4]"),
 23622         -     /*  65 */ "NotFound"         OpHelp("key=r[P3@P4]"),
 23623         -     /*  66 */ "Found"            OpHelp("key=r[P3@P4]"),
 23624         -     /*  67 */ "NotExists"        OpHelp("intkey=r[P3]"),
 23625         -     /*  68 */ "Sequence"         OpHelp("r[P2]=cursor[P1].ctr++"),
 23626         -     /*  69 */ "NewRowid"         OpHelp("r[P2]=rowid"),
 23627         -     /*  70 */ "Insert"           OpHelp("intkey=r[P3] data=r[P2]"),
        23951  +     /*  52 */ "ReopenIdx"        OpHelp("root=P2 iDb=P3"),
        23952  +     /*  53 */ "OpenRead"         OpHelp("root=P2 iDb=P3"),
        23953  +     /*  54 */ "OpenWrite"        OpHelp("root=P2 iDb=P3"),
        23954  +     /*  55 */ "OpenAutoindex"    OpHelp("nColumn=P2"),
        23955  +     /*  56 */ "OpenEphemeral"    OpHelp("nColumn=P2"),
        23956  +     /*  57 */ "SorterOpen"       OpHelp(""),
        23957  +     /*  58 */ "OpenPseudo"       OpHelp("P3 columns in r[P2]"),
        23958  +     /*  59 */ "Close"            OpHelp(""),
        23959  +     /*  60 */ "SeekLT"           OpHelp("key=r[P3@P4]"),
        23960  +     /*  61 */ "SeekLE"           OpHelp("key=r[P3@P4]"),
        23961  +     /*  62 */ "SeekGE"           OpHelp("key=r[P3@P4]"),
        23962  +     /*  63 */ "SeekGT"           OpHelp("key=r[P3@P4]"),
        23963  +     /*  64 */ "Seek"             OpHelp("intkey=r[P2]"),
        23964  +     /*  65 */ "NoConflict"       OpHelp("key=r[P3@P4]"),
        23965  +     /*  66 */ "NotFound"         OpHelp("key=r[P3@P4]"),
        23966  +     /*  67 */ "Found"            OpHelp("key=r[P3@P4]"),
        23967  +     /*  68 */ "NotExists"        OpHelp("intkey=r[P3]"),
        23968  +     /*  69 */ "Sequence"         OpHelp("r[P2]=cursor[P1].ctr++"),
        23969  +     /*  70 */ "NewRowid"         OpHelp("r[P2]=rowid"),
 23628  23970        /*  71 */ "Or"               OpHelp("r[P3]=(r[P1] || r[P2])"),
 23629  23971        /*  72 */ "And"              OpHelp("r[P3]=(r[P1] && r[P2])"),
 23630         -     /*  73 */ "InsertInt"        OpHelp("intkey=P3 data=r[P2]"),
 23631         -     /*  74 */ "Delete"           OpHelp(""),
 23632         -     /*  75 */ "ResetCount"       OpHelp(""),
        23972  +     /*  73 */ "Insert"           OpHelp("intkey=r[P3] data=r[P2]"),
        23973  +     /*  74 */ "InsertInt"        OpHelp("intkey=P3 data=r[P2]"),
        23974  +     /*  75 */ "Delete"           OpHelp(""),
 23633  23975        /*  76 */ "IsNull"           OpHelp("if r[P1]==NULL goto P2"),
 23634  23976        /*  77 */ "NotNull"          OpHelp("if r[P1]!=NULL goto P2"),
 23635  23977        /*  78 */ "Ne"               OpHelp("if r[P1]!=r[P3] goto P2"),
 23636  23978        /*  79 */ "Eq"               OpHelp("if r[P1]==r[P3] goto P2"),
 23637  23979        /*  80 */ "Gt"               OpHelp("if r[P1]>r[P3] goto P2"),
 23638  23980        /*  81 */ "Le"               OpHelp("if r[P1]<=r[P3] goto P2"),
 23639  23981        /*  82 */ "Lt"               OpHelp("if r[P1]<r[P3] goto P2"),
 23640  23982        /*  83 */ "Ge"               OpHelp("if r[P1]>=r[P3] goto P2"),
 23641         -     /*  84 */ "SorterCompare"    OpHelp("if key(P1)!=rtrim(r[P3],P4) goto P2"),
        23983  +     /*  84 */ "ResetCount"       OpHelp(""),
 23642  23984        /*  85 */ "BitAnd"           OpHelp("r[P3]=r[P1]&r[P2]"),
 23643  23985        /*  86 */ "BitOr"            OpHelp("r[P3]=r[P1]|r[P2]"),
 23644  23986        /*  87 */ "ShiftLeft"        OpHelp("r[P3]=r[P2]<<r[P1]"),
 23645  23987        /*  88 */ "ShiftRight"       OpHelp("r[P3]=r[P2]>>r[P1]"),
 23646  23988        /*  89 */ "Add"              OpHelp("r[P3]=r[P1]+r[P2]"),
 23647  23989        /*  90 */ "Subtract"         OpHelp("r[P3]=r[P2]-r[P1]"),
 23648  23990        /*  91 */ "Multiply"         OpHelp("r[P3]=r[P1]*r[P2]"),
 23649  23991        /*  92 */ "Divide"           OpHelp("r[P3]=r[P2]/r[P1]"),
 23650  23992        /*  93 */ "Remainder"        OpHelp("r[P3]=r[P2]%r[P1]"),
 23651  23993        /*  94 */ "Concat"           OpHelp("r[P3]=r[P2]+r[P1]"),
 23652         -     /*  95 */ "SorterData"       OpHelp("r[P2]=data"),
        23994  +     /*  95 */ "SorterCompare"    OpHelp("if key(P1)!=trim(r[P3],P4) goto P2"),
 23653  23995        /*  96 */ "BitNot"           OpHelp("r[P1]= ~r[P1]"),
 23654  23996        /*  97 */ "String8"          OpHelp("r[P2]='P4'"),
 23655         -     /*  98 */ "RowKey"           OpHelp("r[P2]=key"),
 23656         -     /*  99 */ "RowData"          OpHelp("r[P2]=data"),
 23657         -     /* 100 */ "Rowid"            OpHelp("r[P2]=rowid"),
 23658         -     /* 101 */ "NullRow"          OpHelp(""),
 23659         -     /* 102 */ "Last"             OpHelp(""),
 23660         -     /* 103 */ "SorterSort"       OpHelp(""),
 23661         -     /* 104 */ "Sort"             OpHelp(""),
 23662         -     /* 105 */ "Rewind"           OpHelp(""),
 23663         -     /* 106 */ "SorterInsert"     OpHelp(""),
 23664         -     /* 107 */ "IdxInsert"        OpHelp("key=r[P2]"),
 23665         -     /* 108 */ "IdxDelete"        OpHelp("key=r[P2@P3]"),
 23666         -     /* 109 */ "IdxRowid"         OpHelp("r[P2]=rowid"),
 23667         -     /* 110 */ "IdxLE"            OpHelp("key=r[P3@P4]"),
 23668         -     /* 111 */ "IdxGT"            OpHelp("key=r[P3@P4]"),
 23669         -     /* 112 */ "IdxLT"            OpHelp("key=r[P3@P4]"),
 23670         -     /* 113 */ "IdxGE"            OpHelp("key=r[P3@P4]"),
 23671         -     /* 114 */ "Destroy"          OpHelp(""),
 23672         -     /* 115 */ "Clear"            OpHelp(""),
 23673         -     /* 116 */ "ResetSorter"      OpHelp(""),
 23674         -     /* 117 */ "CreateIndex"      OpHelp("r[P2]=root iDb=P1"),
 23675         -     /* 118 */ "CreateTable"      OpHelp("r[P2]=root iDb=P1"),
 23676         -     /* 119 */ "ParseSchema"      OpHelp(""),
 23677         -     /* 120 */ "LoadAnalysis"     OpHelp(""),
 23678         -     /* 121 */ "DropTable"        OpHelp(""),
 23679         -     /* 122 */ "DropIndex"        OpHelp(""),
 23680         -     /* 123 */ "DropTrigger"      OpHelp(""),
 23681         -     /* 124 */ "IntegrityCk"      OpHelp(""),
 23682         -     /* 125 */ "RowSetAdd"        OpHelp("rowset(P1)=r[P2]"),
 23683         -     /* 126 */ "RowSetRead"       OpHelp("r[P3]=rowset(P1)"),
 23684         -     /* 127 */ "RowSetTest"       OpHelp("if r[P3] in rowset(P1) goto P2"),
 23685         -     /* 128 */ "Program"          OpHelp(""),
 23686         -     /* 129 */ "Param"            OpHelp(""),
 23687         -     /* 130 */ "FkCounter"        OpHelp("fkctr[P1]+=P2"),
 23688         -     /* 131 */ "FkIfZero"         OpHelp("if fkctr[P1]==0 goto P2"),
 23689         -     /* 132 */ "MemMax"           OpHelp("r[P1]=max(r[P1],r[P2])"),
        23997  +     /*  98 */ "SorterData"       OpHelp("r[P2]=data"),
        23998  +     /*  99 */ "RowKey"           OpHelp("r[P2]=key"),
        23999  +     /* 100 */ "RowData"          OpHelp("r[P2]=data"),
        24000  +     /* 101 */ "Rowid"            OpHelp("r[P2]=rowid"),
        24001  +     /* 102 */ "NullRow"          OpHelp(""),
        24002  +     /* 103 */ "Last"             OpHelp(""),
        24003  +     /* 104 */ "SorterSort"       OpHelp(""),
        24004  +     /* 105 */ "Sort"             OpHelp(""),
        24005  +     /* 106 */ "Rewind"           OpHelp(""),
        24006  +     /* 107 */ "SorterInsert"     OpHelp(""),
        24007  +     /* 108 */ "IdxInsert"        OpHelp("key=r[P2]"),
        24008  +     /* 109 */ "IdxDelete"        OpHelp("key=r[P2@P3]"),
        24009  +     /* 110 */ "IdxRowid"         OpHelp("r[P2]=rowid"),
        24010  +     /* 111 */ "IdxLE"            OpHelp("key=r[P3@P4]"),
        24011  +     /* 112 */ "IdxGT"            OpHelp("key=r[P3@P4]"),
        24012  +     /* 113 */ "IdxLT"            OpHelp("key=r[P3@P4]"),
        24013  +     /* 114 */ "IdxGE"            OpHelp("key=r[P3@P4]"),
        24014  +     /* 115 */ "Destroy"          OpHelp(""),
        24015  +     /* 116 */ "Clear"            OpHelp(""),
        24016  +     /* 117 */ "ResetSorter"      OpHelp(""),
        24017  +     /* 118 */ "CreateIndex"      OpHelp("r[P2]=root iDb=P1"),
        24018  +     /* 119 */ "CreateTable"      OpHelp("r[P2]=root iDb=P1"),
        24019  +     /* 120 */ "ParseSchema"      OpHelp(""),
        24020  +     /* 121 */ "LoadAnalysis"     OpHelp(""),
        24021  +     /* 122 */ "DropTable"        OpHelp(""),
        24022  +     /* 123 */ "DropIndex"        OpHelp(""),
        24023  +     /* 124 */ "DropTrigger"      OpHelp(""),
        24024  +     /* 125 */ "IntegrityCk"      OpHelp(""),
        24025  +     /* 126 */ "RowSetAdd"        OpHelp("rowset(P1)=r[P2]"),
        24026  +     /* 127 */ "RowSetRead"       OpHelp("r[P3]=rowset(P1)"),
        24027  +     /* 128 */ "RowSetTest"       OpHelp("if r[P3] in rowset(P1) goto P2"),
        24028  +     /* 129 */ "Program"          OpHelp(""),
        24029  +     /* 130 */ "Param"            OpHelp(""),
        24030  +     /* 131 */ "FkCounter"        OpHelp("fkctr[P1]+=P2"),
        24031  +     /* 132 */ "FkIfZero"         OpHelp("if fkctr[P1]==0 goto P2"),
 23690  24032        /* 133 */ "Real"             OpHelp("r[P2]=P4"),
 23691         -     /* 134 */ "IfPos"            OpHelp("if r[P1]>0 goto P2"),
 23692         -     /* 135 */ "IfNeg"            OpHelp("if r[P1]<0 goto P2"),
 23693         -     /* 136 */ "IfZero"           OpHelp("r[P1]+=P3, if r[P1]==0 goto P2"),
 23694         -     /* 137 */ "AggFinal"         OpHelp("accum=r[P1] N=P2"),
 23695         -     /* 138 */ "IncrVacuum"       OpHelp(""),
 23696         -     /* 139 */ "Expire"           OpHelp(""),
 23697         -     /* 140 */ "TableLock"        OpHelp("iDb=P1 root=P2 write=P3"),
 23698         -     /* 141 */ "VBegin"           OpHelp(""),
 23699         -     /* 142 */ "VCreate"          OpHelp(""),
        24033  +     /* 134 */ "MemMax"           OpHelp("r[P1]=max(r[P1],r[P2])"),
        24034  +     /* 135 */ "IfPos"            OpHelp("if r[P1]>0 goto P2"),
        24035  +     /* 136 */ "IfNeg"            OpHelp("r[P1]+=P3, if r[P1]<0 goto P2"),
        24036  +     /* 137 */ "IfZero"           OpHelp("r[P1]+=P3, if r[P1]==0 goto P2"),
        24037  +     /* 138 */ "AggFinal"         OpHelp("accum=r[P1] N=P2"),
        24038  +     /* 139 */ "IncrVacuum"       OpHelp(""),
        24039  +     /* 140 */ "Expire"           OpHelp(""),
        24040  +     /* 141 */ "TableLock"        OpHelp("iDb=P1 root=P2 write=P3"),
        24041  +     /* 142 */ "VBegin"           OpHelp(""),
 23700  24042        /* 143 */ "ToText"           OpHelp(""),
 23701  24043        /* 144 */ "ToBlob"           OpHelp(""),
 23702  24044        /* 145 */ "ToNumeric"        OpHelp(""),
 23703  24045        /* 146 */ "ToInt"            OpHelp(""),
 23704  24046        /* 147 */ "ToReal"           OpHelp(""),
 23705         -     /* 148 */ "VDestroy"         OpHelp(""),
 23706         -     /* 149 */ "VOpen"            OpHelp(""),
 23707         -     /* 150 */ "VColumn"          OpHelp("r[P3]=vcolumn(P2)"),
 23708         -     /* 151 */ "VNext"            OpHelp(""),
 23709         -     /* 152 */ "VRename"          OpHelp(""),
 23710         -     /* 153 */ "Pagecount"        OpHelp(""),
 23711         -     /* 154 */ "MaxPgcnt"         OpHelp(""),
 23712         -     /* 155 */ "Init"             OpHelp("Start at P2"),
 23713         -     /* 156 */ "Noop"             OpHelp(""),
 23714         -     /* 157 */ "Explain"          OpHelp(""),
        24047  +     /* 148 */ "VCreate"          OpHelp(""),
        24048  +     /* 149 */ "VDestroy"         OpHelp(""),
        24049  +     /* 150 */ "VOpen"            OpHelp(""),
        24050  +     /* 151 */ "VColumn"          OpHelp("r[P3]=vcolumn(P2)"),
        24051  +     /* 152 */ "VNext"            OpHelp(""),
        24052  +     /* 153 */ "VRename"          OpHelp(""),
        24053  +     /* 154 */ "Pagecount"        OpHelp(""),
        24054  +     /* 155 */ "MaxPgcnt"         OpHelp(""),
        24055  +     /* 156 */ "Init"             OpHelp("Start at P2"),
        24056  +     /* 157 */ "Noop"             OpHelp(""),
        24057  +     /* 158 */ "Explain"          OpHelp(""),
 23715  24058     };
 23716  24059     return azName[i];
 23717  24060   }
 23718  24061   #endif
 23719  24062   
 23720  24063   /************** End of opcodes.c *********************************************/
 23721  24064   /************** Begin file os_unix.c *****************************************/
................................................................................
 23810  24153   #include <sys/stat.h>
 23811  24154   #include <fcntl.h>
 23812  24155   #include <unistd.h>
 23813  24156   /* #include <time.h> */
 23814  24157   #include <sys/time.h>
 23815  24158   #include <errno.h>
 23816  24159   #if !defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0
 23817         -#include <sys/mman.h>
        24160  +# include <sys/mman.h>
 23818  24161   #endif
 23819  24162   
 23820         -
 23821         -#if SQLITE_ENABLE_LOCKING_STYLE
        24163  +#if SQLITE_ENABLE_LOCKING_STYLE || OS_VXWORKS
 23822  24164   # include <sys/ioctl.h>
 23823  24165   # if OS_VXWORKS
 23824  24166   #  include <semaphore.h>
 23825  24167   #  include <limits.h>
 23826  24168   # else
 23827  24169   #  include <sys/file.h>
 23828  24170   #  include <sys/param.h>
................................................................................
 24242  24584   
 24243  24585   /*
 24244  24586   ** On some systems, calls to fchown() will trigger a message in a security
 24245  24587   ** log if they come from non-root processes.  So avoid calling fchown() if
 24246  24588   ** we are not running as root.
 24247  24589   */
 24248  24590   static int posixFchown(int fd, uid_t uid, gid_t gid){
        24591  +#if OS_VXWORKS
        24592  +  return 0;
        24593  +#else
 24249  24594     return geteuid() ? 0 : fchown(fd,uid,gid);
        24595  +#endif
 24250  24596   }
 24251  24597   
 24252  24598   /* Forward reference */
 24253  24599   static int openDirectory(const char*, int*);
 24254  24600   static int unixGetpagesize(void);
 24255  24601   
 24256  24602   /*
................................................................................
 24298  24644   
 24299  24645     { "fcntl",        (sqlite3_syscall_ptr)fcntl,      0  },
 24300  24646   #define osFcntl     ((int(*)(int,int,...))aSyscall[7].pCurrent)
 24301  24647   
 24302  24648     { "read",         (sqlite3_syscall_ptr)read,       0  },
 24303  24649   #define osRead      ((ssize_t(*)(int,void*,size_t))aSyscall[8].pCurrent)
 24304  24650   
 24305         -#if defined(USE_PREAD) || SQLITE_ENABLE_LOCKING_STYLE
        24651  +#if defined(USE_PREAD) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS)
 24306  24652     { "pread",        (sqlite3_syscall_ptr)pread,      0  },
 24307  24653   #else
 24308  24654     { "pread",        (sqlite3_syscall_ptr)0,          0  },
 24309  24655   #endif
 24310  24656   #define osPread     ((ssize_t(*)(int,void*,size_t,off_t))aSyscall[9].pCurrent)
 24311  24657   
 24312  24658   #if defined(USE_PREAD64)
................................................................................
 24315  24661     { "pread64",      (sqlite3_syscall_ptr)0,          0  },
 24316  24662   #endif
 24317  24663   #define osPread64   ((ssize_t(*)(int,void*,size_t,off_t))aSyscall[10].pCurrent)
 24318  24664   
 24319  24665     { "write",        (sqlite3_syscall_ptr)write,      0  },
 24320  24666   #define osWrite     ((ssize_t(*)(int,const void*,size_t))aSyscall[11].pCurrent)
 24321  24667   
 24322         -#if defined(USE_PREAD) || SQLITE_ENABLE_LOCKING_STYLE
        24668  +#if defined(USE_PREAD) || (SQLITE_ENABLE_LOCKING_STYLE && !OS_VXWORKS)
 24323  24669     { "pwrite",       (sqlite3_syscall_ptr)pwrite,     0  },
 24324  24670   #else
 24325  24671     { "pwrite",       (sqlite3_syscall_ptr)0,          0  },
 24326  24672   #endif
 24327  24673   #define osPwrite    ((ssize_t(*)(int,const void*,size_t,off_t))\
 24328  24674                       aSyscall[12].pCurrent)
 24329  24675   
................................................................................
 24369  24715   
 24370  24716   #if HAVE_MREMAP
 24371  24717     { "mremap",       (sqlite3_syscall_ptr)mremap,          0 },
 24372  24718   #else
 24373  24719     { "mremap",       (sqlite3_syscall_ptr)0,               0 },
 24374  24720   #endif
 24375  24721   #define osMremap ((void*(*)(void*,size_t,size_t,int,...))aSyscall[23].pCurrent)
 24376         -#endif
 24377         -
 24378  24722     { "getpagesize",  (sqlite3_syscall_ptr)unixGetpagesize, 0 },
 24379  24723   #define osGetpagesize ((int(*)(void))aSyscall[24].pCurrent)
        24724  +
        24725  +#endif
 24380  24726   
 24381  24727   }; /* End of the overrideable system calls */
 24382  24728   
 24383  24729   /*
 24384  24730   ** This is the xSetSystemCall() method of sqlite3_vfs for all of the
 24385  24731   ** "unix" VFSes.  Return SQLITE_OK opon successfully updating the
 24386  24732   ** system call pointer, or SQLITE_NOTFOUND if there is no configurable
................................................................................
 24685  25031           (sqliteIOErr == SQLITE_IOERR_CHECKRESERVEDLOCK) ){
 24686  25032         return SQLITE_BUSY;
 24687  25033       }
 24688  25034       /* else fall through */
 24689  25035     case EPERM: 
 24690  25036       return SQLITE_PERM;
 24691  25037       
 24692         -  /* EDEADLK is only possible if a call to fcntl(F_SETLKW) is made. And
 24693         -  ** this module never makes such a call. And the code in SQLite itself 
 24694         -  ** asserts that SQLITE_IOERR_BLOCKED is never returned. For these reasons
 24695         -  ** this case is also commented out. If the system does set errno to EDEADLK,
 24696         -  ** the default SQLITE_IOERR_XXX code will be returned. */
 24697         -#if 0
 24698         -  case EDEADLK:
 24699         -    return SQLITE_IOERR_BLOCKED;
 24700         -#endif
 24701         -    
 24702  25038   #if EOPNOTSUPP!=ENOTSUP
 24703  25039     case EOPNOTSUPP: 
 24704  25040       /* something went terribly awry, unless during file system support 
 24705  25041        * introspection, in which it actually means what it says */
 24706  25042   #endif
 24707  25043   #ifdef ENOTSUP
 24708  25044     case ENOTSUP: 
................................................................................
 25227  25563     return SQLITE_OK;
 25228  25564   }
 25229  25565   
 25230  25566   /*
 25231  25567   ** Return TRUE if pFile has been renamed or unlinked since it was first opened.
 25232  25568   */
 25233  25569   static int fileHasMoved(unixFile *pFile){
        25570  +#if OS_VXWORKS
        25571  +  return pFile->pInode!=0 && pFile->pId!=pFile->pInode->fileId.pId;
        25572  +#else
 25234  25573     struct stat buf;
 25235  25574     return pFile->pInode!=0 &&
 25236         -         (osStat(pFile->zPath, &buf)!=0 || buf.st_ino!=pFile->pInode->fileId.ino);
        25575  +      (osStat(pFile->zPath, &buf)!=0 || buf.st_ino!=pFile->pInode->fileId.ino);
        25576  +#endif
 25237  25577   }
 25238  25578   
 25239  25579   
 25240  25580   /*
 25241  25581   ** Check a unixFile that is a database.  Verify the following:
 25242  25582   **
 25243  25583   ** (1) There is exactly one hard link on the file
................................................................................
 25842  26182     if( pFile->pId ){
 25843  26183       if( pFile->ctrlFlags & UNIXFILE_DELETE ){
 25844  26184         osUnlink(pFile->pId->zCanonicalName);
 25845  26185       }
 25846  26186       vxworksReleaseFileId(pFile->pId);
 25847  26187       pFile->pId = 0;
 25848  26188     }
        26189  +#endif
        26190  +#ifdef SQLITE_UNLINK_AFTER_CLOSE
        26191  +  if( pFile->ctrlFlags & UNIXFILE_DELETE ){
        26192  +    osUnlink(pFile->zPath);
        26193  +    sqlite3_free(*(char**)&pFile->zPath);
        26194  +    pFile->zPath = 0;
        26195  +  }
 25849  26196   #endif
 25850  26197     OSTRACE(("CLOSE   %-3d\n", pFile->h));
 25851  26198     OpenCounter(-1);
 25852  26199     sqlite3_free(pFile->pUnused);
 25853  26200     memset(pFile, 0, sizeof(unixFile));
 25854  26201     return SQLITE_OK;
 25855  26202   }
................................................................................
 26365  26712     if( pFile->eFileLock>SHARED_LOCK ){
 26366  26713       reserved = 1;
 26367  26714     }
 26368  26715     
 26369  26716     /* Otherwise see if some other process holds it. */
 26370  26717     if( !reserved ){
 26371  26718       sem_t *pSem = pFile->pInode->pSem;
 26372         -    struct stat statBuf;
 26373  26719   
 26374  26720       if( sem_trywait(pSem)==-1 ){
 26375  26721         int tErrno = errno;
 26376  26722         if( EAGAIN != tErrno ){
 26377  26723           rc = sqliteErrorFromPosixError(tErrno, SQLITE_IOERR_CHECKRESERVEDLOCK);
 26378  26724           pFile->lastErrno = tErrno;
 26379  26725         } else {
................................................................................
 26418  26764   ** access the file.
 26419  26765   **
 26420  26766   ** This routine will only increase a lock.  Use the sqlite3OsUnlock()
 26421  26767   ** routine to lower a locking level.
 26422  26768   */
 26423  26769   static int semLock(sqlite3_file *id, int eFileLock) {
 26424  26770     unixFile *pFile = (unixFile*)id;
 26425         -  int fd;
 26426  26771     sem_t *pSem = pFile->pInode->pSem;
 26427  26772     int rc = SQLITE_OK;
 26428  26773   
 26429  26774     /* if we already have a lock, it is exclusive.  
 26430  26775     ** Just adjust level and punt on outta here. */
 26431  26776     if (pFile->eFileLock > NO_LOCK) {
 26432  26777       pFile->eFileLock = eFileLock;
................................................................................
 27881  28226   #endif
 27882  28227     if( p->ctrlFlags & UNIXFILE_PSOW ){
 27883  28228       rc |= SQLITE_IOCAP_POWERSAFE_OVERWRITE;
 27884  28229     }
 27885  28230     return rc;
 27886  28231   }
 27887  28232   
        28233  +#if !defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0
        28234  +
        28235  +/*
        28236  +** Return the system page size.
        28237  +**
        28238  +** This function should not be called directly by other code in this file. 
        28239  +** Instead, it should be called via macro osGetpagesize().
        28240  +*/
        28241  +static int unixGetpagesize(void){
        28242  +#if defined(_BSD_SOURCE)
        28243  +  return getpagesize();
        28244  +#else
        28245  +  return (int)sysconf(_SC_PAGESIZE);
        28246  +#endif
        28247  +}
        28248  +
        28249  +#endif /* !defined(SQLITE_OMIT_WAL) || SQLITE_MAX_MMAP_SIZE>0 */
        28250  +
 27888  28251   #ifndef SQLITE_OMIT_WAL
 27889         -
 27890  28252   
 27891  28253   /*
 27892  28254   ** Object used to represent an shared memory buffer.  
 27893  28255   **
 27894  28256   ** When multiple threads all reference the same wal-index, each thread
 27895  28257   ** has its own unixShm object, but they all point to a single instance
 27896  28258   ** of this unixShmNode object.  In other words, each wal-index is opened
................................................................................
 28033  28395              pShmNode->sharedMask, pShmNode->exclMask));
 28034  28396     }
 28035  28397   #endif
 28036  28398   
 28037  28399     return rc;        
 28038  28400   }
 28039  28401   
 28040         -/*
 28041         -** Return the system page size.
 28042         -**
 28043         -** This function should not be called directly by other code in this file. 
 28044         -** Instead, it should be called via macro osGetpagesize().
 28045         -*/
 28046         -static int unixGetpagesize(void){
 28047         -#if defined(_BSD_SOURCE)
 28048         -  return getpagesize();
 28049         -#else
 28050         -  return (int)sysconf(_SC_PAGESIZE);
 28051         -#endif
 28052         -}
 28053         -
 28054  28402   /*
 28055  28403   ** Return the minimum number of 32KB shm regions that should be mapped at
 28056  28404   ** a time, assuming that each mapping must be an integer multiple of the
 28057  28405   ** current system page-size.
 28058  28406   **
 28059  28407   ** Usually, this is 1. The exception seems to be systems that are configured
 28060  28408   ** to use 64KB pages - in this case each mapping must cover at least two
................................................................................
 29696  30044       p->pUnused->fd = fd;
 29697  30045       p->pUnused->flags = flags;
 29698  30046     }
 29699  30047   
 29700  30048     if( isDelete ){
 29701  30049   #if OS_VXWORKS
 29702  30050       zPath = zName;
        30051  +#elif defined(SQLITE_UNLINK_AFTER_CLOSE)
        30052  +    zPath = sqlite3_mprintf("%s", zName);
        30053  +    if( zPath==0 ){
        30054  +      robust_close(p, fd, __LINE__);
        30055  +      return SQLITE_NOMEM;
        30056  +    }
 29703  30057   #else
 29704  30058       osUnlink(zName);
 29705  30059   #endif
 29706  30060     }
 29707  30061   #if SQLITE_ENABLE_LOCKING_STYLE
 29708  30062     else{
 29709  30063       p->openFlags = openFlags;
................................................................................
 29796  30150     const char *zPath,        /* Name of file to be deleted */
 29797  30151     int dirSync               /* If true, fsync() directory after deleting file */
 29798  30152   ){
 29799  30153     int rc = SQLITE_OK;
 29800  30154     UNUSED_PARAMETER(NotUsed);
 29801  30155     SimulateIOError(return SQLITE_IOERR_DELETE);
 29802  30156     if( osUnlink(zPath)==(-1) ){
 29803         -    if( errno==ENOENT ){
        30157  +    if( errno==ENOENT
        30158  +#if OS_VXWORKS
        30159  +        || errno==0x380003
        30160  +#endif
        30161  +    ){
 29804  30162         rc = SQLITE_IOERR_DELETE_NOENT;
 29805  30163       }else{
 29806  30164         rc = unixLogError(SQLITE_IOERR_DELETE, "unlink", zPath);
 29807  30165       }
 29808  30166       return rc;
 29809  30167     }
 29810  30168   #ifndef SQLITE_DISABLE_DIRSYNC
................................................................................
 31693  32051   #endif
 31694  32052   
 31695  32053   #ifndef NTDDI_WINBLUE
 31696  32054   #  define NTDDI_WINBLUE                     0x06030000
 31697  32055   #endif
 31698  32056   
 31699  32057   /*
 31700         -** Check if the GetVersionEx[AW] functions should be considered deprecated
 31701         -** and avoid using them in that case.  It should be noted here that if the
 31702         -** value of the SQLITE_WIN32_GETVERSIONEX pre-processor macro is zero
 31703         -** (whether via this block or via being manually specified), that implies
 31704         -** the underlying operating system will always be based on the Windows NT
 31705         -** Kernel.
        32058  +** Check to see if the GetVersionEx[AW] functions are deprecated on the
        32059  +** target system.  GetVersionEx was first deprecated in Win8.1.
 31706  32060   */
 31707  32061   #ifndef SQLITE_WIN32_GETVERSIONEX
 31708  32062   #  if defined(NTDDI_VERSION) && NTDDI_VERSION >= NTDDI_WINBLUE
 31709         -#    define SQLITE_WIN32_GETVERSIONEX   0
        32063  +#    define SQLITE_WIN32_GETVERSIONEX   0   /* GetVersionEx() is deprecated */
 31710  32064   #  else
 31711         -#    define SQLITE_WIN32_GETVERSIONEX   1
        32065  +#    define SQLITE_WIN32_GETVERSIONEX   1   /* GetVersionEx() is current */
 31712  32066   #  endif
 31713  32067   #endif
 31714  32068   
 31715  32069   /*
 31716  32070   ** This constant should already be defined (in the "WinDef.h" SDK file).
 31717  32071   */
 31718  32072   #ifndef MAX_PATH
................................................................................
 31776  32130   #endif
 31777  32131   
 31778  32132   /*
 31779  32133   ** This macro is used when a local variable is set to a value that is
 31780  32134   ** [sometimes] not used by the code (e.g. via conditional compilation).
 31781  32135   */
 31782  32136   #ifndef UNUSED_VARIABLE_VALUE
 31783         -#  define UNUSED_VARIABLE_VALUE(x) (void)(x)
        32137  +#  define UNUSED_VARIABLE_VALUE(x)      (void)(x)
 31784  32138   #endif
 31785  32139   
 31786  32140   /*
 31787  32141   ** Returns the character that should be used as the directory separator.
 31788  32142   */
 31789  32143   #ifndef winGetDirSep
 31790  32144   #  define winGetDirSep()                '\\'
................................................................................
 31825  32179   WINBASEAPI BOOL WINAPI UnmapViewOfFile(LPCVOID);
 31826  32180   #endif /* SQLITE_WIN32_FILEMAPPING_API && !defined(SQLITE_OMIT_WAL) */
 31827  32181   
 31828  32182   /*
 31829  32183   ** Some Microsoft compilers lack this definition.
 31830  32184   */
 31831  32185   #ifndef INVALID_FILE_ATTRIBUTES
 31832         -# define INVALID_FILE_ATTRIBUTES ((DWORD)-1) 
        32186  +# define INVALID_FILE_ATTRIBUTES ((DWORD)-1)
 31833  32187   #endif
 31834  32188   
 31835  32189   #ifndef FILE_FLAG_MASK
 31836  32190   # define FILE_FLAG_MASK          (0xFF3C0000)
 31837  32191   #endif
 31838  32192   
 31839  32193   #ifndef FILE_ATTRIBUTE_MASK
................................................................................
 31875  32229   #ifndef SQLITE_OMIT_WAL
 31876  32230     winShm *pShm;           /* Instance of shared memory on this file */
 31877  32231   #endif
 31878  32232     const char *zPath;      /* Full pathname of this file */
 31879  32233     int szChunk;            /* Chunk size configured by FCNTL_CHUNK_SIZE */
 31880  32234   #if SQLITE_OS_WINCE
 31881  32235     LPWSTR zDeleteOnClose;  /* Name of file to delete when closing */
 31882         -  HANDLE hMutex;          /* Mutex used to control access to shared lock */  
        32236  +  HANDLE hMutex;          /* Mutex used to control access to shared lock */
 31883  32237     HANDLE hShared;         /* Shared memory segment used for locking */
 31884  32238     winceLock local;        /* Locks obtained by this instance of winFile */
 31885  32239     winceLock *shared;      /* Global shared lock memory for the file  */
 31886  32240   #endif
 31887  32241   #if SQLITE_MAX_MMAP_SIZE>0
 31888  32242     int nFetchOut;                /* Number of outstanding xFetch references */
 31889  32243     HANDLE hMap;                  /* Handle for accessing memory mapping */
................................................................................
 32035  32389   ** 1:   Operating system is Win9x.
 32036  32390   ** 2:   Operating system is WinNT.
 32037  32391   **
 32038  32392   ** In order to facilitate testing on a WinNT system, the test fixture
 32039  32393   ** can manually set this value to 1 to emulate Win98 behavior.
 32040  32394   */
 32041  32395   #ifdef SQLITE_TEST
 32042         -SQLITE_API int sqlite3_os_type = 0;
 32043         -#elif !SQLITE_OS_WINCE && !SQLITE_OS_WINRT && \
 32044         -      defined(SQLITE_WIN32_HAS_ANSI) && defined(SQLITE_WIN32_HAS_WIDE)
 32045         -static int sqlite3_os_type = 0;
        32396  +SQLITE_API LONG volatile sqlite3_os_type = 0;
        32397  +#else
        32398  +static LONG volatile sqlite3_os_type = 0;
 32046  32399   #endif
 32047  32400   
 32048  32401   #ifndef SYSCALL
 32049  32402   #  define SYSCALL sqlite3_syscall_ptr
 32050  32403   #endif
 32051  32404   
 32052  32405   /*
................................................................................
 32669  33022   #else
 32670  33023     { "CreateFileMappingFromApp", (SYSCALL)0,                      0 },
 32671  33024   #endif
 32672  33025   
 32673  33026   #define osCreateFileMappingFromApp ((HANDLE(WINAPI*)(HANDLE, \
 32674  33027           LPSECURITY_ATTRIBUTES,ULONG,ULONG64,LPCWSTR))aSyscall[75].pCurrent)
 32675  33028   
        33029  +/*
        33030  +** NOTE: On some sub-platforms, the InterlockedCompareExchange "function"
        33031  +**       is really just a macro that uses a compiler intrinsic (e.g. x64).
        33032  +**       So do not try to make this is into a redefinable interface.
        33033  +*/
        33034  +#if defined(InterlockedCompareExchange)
        33035  +  { "InterlockedCompareExchange", (SYSCALL)0,                    0 },
        33036  +
        33037  +#define osInterlockedCompareExchange InterlockedCompareExchange
        33038  +#else
        33039  +  { "InterlockedCompareExchange", (SYSCALL)InterlockedCompareExchange, 0 },
        33040  +
        33041  +#define osInterlockedCompareExchange ((LONG(WINAPI*)(LONG volatile*, \
        33042  +        LONG,LONG))aSyscall[76].pCurrent)
        33043  +#endif /* defined(InterlockedCompareExchange) */
        33044  +
 32676  33045   }; /* End of the overrideable system calls */
 32677  33046   
 32678  33047   /*
 32679  33048   ** This is the xSetSystemCall() method of sqlite3_vfs for all of the
 32680  33049   ** "win32" VFSes.  Return SQLITE_OK opon successfully updating the
 32681  33050   ** system call pointer, or SQLITE_NOTFOUND if there is no configurable
 32682  33051   ** system call named zName.
................................................................................
 32919  33288   #if !defined(SQLITE_WIN32_GETVERSIONEX) || !SQLITE_WIN32_GETVERSIONEX
 32920  33289   # define osIsNT()  (1)
 32921  33290   #elif SQLITE_OS_WINCE || SQLITE_OS_WINRT || !defined(SQLITE_WIN32_HAS_ANSI)
 32922  33291   # define osIsNT()  (1)
 32923  33292   #elif !defined(SQLITE_WIN32_HAS_WIDE)
 32924  33293   # define osIsNT()  (0)
 32925  33294   #else
 32926         -  static int osIsNT(void){
 32927         -    if( sqlite3_os_type==0 ){
 32928         -#if defined(NTDDI_VERSION) && NTDDI_VERSION >= NTDDI_WIN8
 32929         -      OSVERSIONINFOW sInfo;
 32930         -      sInfo.dwOSVersionInfoSize = sizeof(sInfo);
 32931         -      osGetVersionExW(&sInfo);
 32932         -#else
 32933         -      OSVERSIONINFOA sInfo;
 32934         -      sInfo.dwOSVersionInfoSize = sizeof(sInfo);
 32935         -      osGetVersionExA(&sInfo);
 32936         -#endif
 32937         -      sqlite3_os_type = sInfo.dwPlatformId==VER_PLATFORM_WIN32_NT ? 2 : 1;
 32938         -    }
 32939         -    return sqlite3_os_type==2;
 32940         -  }
 32941         -#endif
        33295  +# define osIsNT()  ((sqlite3_os_type==2) || sqlite3_win32_is_nt())
        33296  +#endif
        33297  +
        33298  +/*
        33299  +** This function determines if the machine is running a version of Windows
        33300  +** based on the NT kernel.
        33301  +*/
        33302  +SQLITE_API int sqlite3_win32_is_nt(void){
        33303  +#if defined(SQLITE_WIN32_GETVERSIONEX) && SQLITE_WIN32_GETVERSIONEX
        33304  +  if( osInterlockedCompareExchange(&sqlite3_os_type, 0, 0)==0 ){
        33305  +#if !SQLITE_OS_WINRT && defined(SQLITE_WIN32_HAS_WIDE) && \
        33306  +        defined(NTDDI_VERSION) && NTDDI_VERSION >= NTDDI_WIN8
        33307  +    OSVERSIONINFOW sInfo;
        33308  +    sInfo.dwOSVersionInfoSize = sizeof(sInfo);
        33309  +    osGetVersionExW(&sInfo);
        33310  +    osInterlockedCompareExchange(&sqlite3_os_type,
        33311  +        (sInfo.dwPlatformId == VER_PLATFORM_WIN32_NT) ? 2 : 1, 0);
        33312  +#elif defined(SQLITE_WIN32_HAS_ANSI)
        33313  +    OSVERSIONINFOA sInfo;
        33314  +    sInfo.dwOSVersionInfoSize = sizeof(sInfo);
        33315  +    osGetVersionExA(&sInfo);
        33316  +    osInterlockedCompareExchange(&sqlite3_os_type,
        33317  +        (sInfo.dwPlatformId == VER_PLATFORM_WIN32_NT) ? 2 : 1, 0);
        33318  +#endif
        33319  +  }
        33320  +  return osInterlockedCompareExchange(&sqlite3_os_type, 2, 2)==2;
        33321  +#elif SQLITE_TEST
        33322  +  return osInterlockedCompareExchange(&sqlite3_os_type, 2, 2)==2;
        33323  +#else
        33324  +  return 1;
        33325  +#endif
        33326  +}
 32942  33327   
 32943  33328   #ifdef SQLITE_WIN32_MALLOC
 32944  33329   /*
 32945  33330   ** Allocate nBytes of memory.
 32946  33331   */
 32947  33332   static void *winMemMalloc(int nBytes){
 32948  33333     HANDLE hHeap;
................................................................................
 33142  33527   
 33143  33528   SQLITE_PRIVATE void sqlite3MemSetDefault(void){
 33144  33529     sqlite3_config(SQLITE_CONFIG_MALLOC, sqlite3MemGetWin32());
 33145  33530   }
 33146  33531   #endif /* SQLITE_WIN32_MALLOC */
 33147  33532   
 33148  33533   /*
 33149         -** Convert a UTF-8 string to Microsoft Unicode (UTF-16?). 
        33534  +** Convert a UTF-8 string to Microsoft Unicode (UTF-16?).
 33150  33535   **
 33151  33536   ** Space to hold the returned string is obtained from malloc.
 33152  33537   */
 33153  33538   static LPWSTR winUtf8ToUnicode(const char *zFilename){
 33154  33539     int nChar;
 33155  33540     LPWSTR zWideFilename;
 33156  33541   
................................................................................
 33195  33580     }
 33196  33581     return zFilename;
 33197  33582   }
 33198  33583   
 33199  33584   /*
 33200  33585   ** Convert an ANSI string to Microsoft Unicode, based on the
 33201  33586   ** current codepage settings for file apis.
 33202         -** 
        33587  +**
 33203  33588   ** Space to hold the returned string is obtained
 33204  33589   ** from sqlite3_malloc.
 33205  33590   */
 33206  33591   static LPWSTR winMbcsToUnicode(const char *zFilename){
 33207  33592     int nByte;
 33208  33593     LPWSTR zMbcsFilename;
 33209  33594     int codepage = osAreFileApisANSI() ? CP_ACP : CP_OEMCP;
................................................................................
 33269  33654     }
 33270  33655     zFilenameUtf8 = winUnicodeToUtf8(zTmpWide);
 33271  33656     sqlite3_free(zTmpWide);
 33272  33657     return zFilenameUtf8;
 33273  33658   }
 33274  33659   
 33275  33660   /*
 33276         -** Convert UTF-8 to multibyte character string.  Space to hold the 
        33661  +** Convert UTF-8 to multibyte character string.  Space to hold the
 33277  33662   ** returned string is obtained from sqlite3_malloc().
 33278  33663   */
 33279  33664   SQLITE_API char *sqlite3_win32_utf8_to_mbcs(const char *zFilename){
 33280  33665     char *zFilenameMbcs;
 33281  33666     LPWSTR zTmpWide;
 33282  33667   
 33283  33668     zTmpWide = winUtf8ToUnicode(zFilename);
................................................................................
 33409  33794   /*
 33410  33795   **
 33411  33796   ** This function - winLogErrorAtLine() - is only ever called via the macro
 33412  33797   ** winLogError().
 33413  33798   **
 33414  33799   ** This routine is invoked after an error occurs in an OS function.
 33415  33800   ** It logs a message using sqlite3_log() containing the current value of
 33416         -** error code and, if possible, the human-readable equivalent from 
        33801  +** error code and, if possible, the human-readable equivalent from
 33417  33802   ** FormatMessage.
 33418  33803   **
 33419  33804   ** The first argument passed to the macro should be the error code that
 33420         -** will be returned to SQLite (e.g. SQLITE_IOERR_DELETE, SQLITE_CANTOPEN). 
        33805  +** will be returned to SQLite (e.g. SQLITE_IOERR_DELETE, SQLITE_CANTOPEN).
 33421  33806   ** The two subsequent arguments should be the name of the OS function that
 33422  33807   ** failed and the associated file-system path, if any.
 33423  33808   */
 33424  33809   #define winLogError(a,b,c,d)   winLogErrorAtLine(a,b,c,d,__LINE__)
 33425  33810   static int winLogErrorAtLine(
 33426  33811     int errcode,                    /* SQLite error code */
 33427  33812     DWORD lastErrno,                /* Win32 last error */
................................................................................
 33444  33829     );
 33445  33830   
 33446  33831     return errcode;
 33447  33832   }
 33448  33833   
 33449  33834   /*
 33450  33835   ** The number of times that a ReadFile(), WriteFile(), and DeleteFile()
 33451         -** will be retried following a locking error - probably caused by 
        33836  +** will be retried following a locking error - probably caused by
 33452  33837   ** antivirus software.  Also the initial delay before the first retry.
 33453  33838   ** The delay increases linearly with each retry.
 33454  33839   */
 33455  33840   #ifndef SQLITE_WIN32_IOERR_RETRY
 33456  33841   # define SQLITE_WIN32_IOERR_RETRY 10
 33457  33842   #endif
 33458  33843   #ifndef SQLITE_WIN32_IOERR_RETRY_DELAY
................................................................................
 33519  33904   }
 33520  33905   
 33521  33906   /*
 33522  33907   ** Log a I/O error retry episode.
 33523  33908   */
 33524  33909   static void winLogIoerr(int nRetry){
 33525  33910     if( nRetry ){
 33526         -    sqlite3_log(SQLITE_IOERR, 
        33911  +    sqlite3_log(SQLITE_IOERR,
 33527  33912         "delayed %dms for lock/sharing conflict",
 33528  33913         winIoerrRetryDelay*nRetry*(nRetry+1)/2
 33529  33914       );
 33530  33915     }
 33531  33916   }
 33532  33917   
 33533  33918   #if SQLITE_OS_WINCE
................................................................................
 33613  33998       sqlite3_free(zName);
 33614  33999       return winLogError(SQLITE_IOERR, pFile->lastErrno,
 33615  34000                          "winceCreateLock1", zFilename);
 33616  34001     }
 33617  34002   
 33618  34003     /* Acquire the mutex before continuing */
 33619  34004     winceMutexAcquire(pFile->hMutex);
 33620         -  
 33621         -  /* Since the names of named mutexes, semaphores, file mappings etc are 
        34005  +
        34006  +  /* Since the names of named mutexes, semaphores, file mappings etc are
 33622  34007     ** case-sensitive, take advantage of that by uppercasing the mutex name
 33623  34008     ** and using that as the shared filemapping name.
 33624  34009     */
 33625  34010     osCharUpperW(zName);
 33626  34011     pFile->hShared = osCreateFileMappingW(INVALID_HANDLE_VALUE, NULL,
 33627  34012                                           PAGE_READWRITE, 0, sizeof(winceLock),
 33628         -                                        zName);  
        34013  +                                        zName);
 33629  34014   
 33630         -  /* Set a flag that indicates we're the first to create the memory so it 
        34015  +  /* Set a flag that indicates we're the first to create the memory so it
 33631  34016     ** must be zero-initialized */
 33632  34017     lastErrno = osGetLastError();
 33633  34018     if (lastErrno == ERROR_ALREADY_EXISTS){
 33634  34019       bInit = FALSE;
 33635  34020     }
 33636  34021   
 33637  34022     sqlite3_free(zName);
 33638  34023   
 33639  34024     /* If we succeeded in making the shared memory handle, map it. */
 33640  34025     if( pFile->hShared ){
 33641         -    pFile->shared = (winceLock*)osMapViewOfFile(pFile->hShared, 
        34026  +    pFile->shared = (winceLock*)osMapViewOfFile(pFile->hShared,
 33642  34027                FILE_MAP_READ|FILE_MAP_WRITE, 0, 0, sizeof(winceLock));
 33643  34028       /* If mapping failed, close the shared memory handle and erase it */
 33644  34029       if( !pFile->shared ){
 33645  34030         pFile->lastErrno = osGetLastError();
 33646  34031         winLogError(SQLITE_IOERR, pFile->lastErrno,
 33647  34032                     "winceCreateLock2", zFilename);
 33648  34033         bLogged = TRUE;
................................................................................
 33660  34045         bLogged = TRUE;
 33661  34046       }
 33662  34047       winceMutexRelease(pFile->hMutex);
 33663  34048       osCloseHandle(pFile->hMutex);
 33664  34049       pFile->hMutex = NULL;
 33665  34050       return SQLITE_IOERR;
 33666  34051     }
 33667         -  
        34052  +
 33668  34053     /* Initialize the shared memory if we're supposed to */
 33669  34054     if( bInit ){
 33670  34055       memset(pFile->shared, 0, sizeof(winceLock));
 33671  34056     }
 33672  34057   
 33673  34058     winceMutexRelease(pFile->hMutex);
 33674  34059     return SQLITE_OK;
................................................................................
 33698  34083       }
 33699  34084   
 33700  34085       /* De-reference and close our copy of the shared memory handle */
 33701  34086       osUnmapViewOfFile(pFile->shared);
 33702  34087       osCloseHandle(pFile->hShared);
 33703  34088   
 33704  34089       /* Done with the mutex */
 33705         -    winceMutexRelease(pFile->hMutex);    
        34090  +    winceMutexRelease(pFile->hMutex);
 33706  34091       osCloseHandle(pFile->hMutex);
 33707  34092       pFile->hMutex = NULL;
 33708  34093     }
 33709  34094   }
 33710  34095   
 33711         -/* 
        34096  +/*
 33712  34097   ** An implementation of the LockFile() API of Windows for CE
 33713  34098   */
 33714  34099   static BOOL winceLockFile(
 33715  34100     LPHANDLE phFile,
 33716  34101     DWORD dwFileOffsetLow,
 33717  34102     DWORD dwFileOffsetHigh,
 33718  34103     DWORD nNumberOfBytesToLockLow,
................................................................................
 33915  34300   ** Some Microsoft compilers lack this definition.
 33916  34301   */
 33917  34302   #ifndef INVALID_SET_FILE_POINTER
 33918  34303   # define INVALID_SET_FILE_POINTER ((DWORD)-1)
 33919  34304   #endif
 33920  34305   
 33921  34306   /*
 33922         -** Move the current position of the file handle passed as the first 
 33923         -** argument to offset iOffset within the file. If successful, return 0. 
        34307  +** Move the current position of the file handle passed as the first
        34308  +** argument to offset iOffset within the file. If successful, return 0.
 33924  34309   ** Otherwise, set pFile->lastErrno and return non-zero.
 33925  34310   */
 33926  34311   static int winSeekFile(winFile *pFile, sqlite3_int64 iOffset){
 33927  34312   #if !SQLITE_OS_WINRT
 33928  34313     LONG upperBits;                 /* Most sig. 32 bits of new offset */
 33929  34314     LONG lowerBits;                 /* Least sig. 32 bits of new offset */
 33930  34315     DWORD dwRet;                    /* Value returned by SetFilePointer() */
................................................................................
 33931  34316     DWORD lastErrno;                /* Value returned by GetLastError() */
 33932  34317   
 33933  34318     OSTRACE(("SEEK file=%p, offset=%lld\n", pFile->h, iOffset));
 33934  34319   
 33935  34320     upperBits = (LONG)((iOffset>>32) & 0x7fffffff);
 33936  34321     lowerBits = (LONG)(iOffset & 0xffffffff);
 33937  34322   
 33938         -  /* API oddity: If successful, SetFilePointer() returns a dword 
        34323  +  /* API oddity: If successful, SetFilePointer() returns a dword
 33939  34324     ** containing the lower 32-bits of the new file-offset. Or, if it fails,
 33940         -  ** it returns INVALID_SET_FILE_POINTER. However according to MSDN, 
 33941         -  ** INVALID_SET_FILE_POINTER may also be a valid new offset. So to determine 
 33942         -  ** whether an error has actually occurred, it is also necessary to call 
        34325  +  ** it returns INVALID_SET_FILE_POINTER. However according to MSDN,
        34326  +  ** INVALID_SET_FILE_POINTER may also be a valid new offset. So to determine
        34327  +  ** whether an error has actually occurred, it is also necessary to call
 33943  34328     ** GetLastError().
 33944  34329     */
 33945  34330     dwRet = osSetFilePointer(pFile->h, lowerBits, &upperBits, FILE_BEGIN);
 33946  34331   
 33947  34332     if( (dwRet==INVALID_SET_FILE_POINTER
 33948  34333         && ((lastErrno = osGetLastError())!=NO_ERROR)) ){
 33949  34334       pFile->lastErrno = lastErrno;
................................................................................
 34018  34403   #if SQLITE_OS_WINCE
 34019  34404   #define WINCE_DELETION_ATTEMPTS 3
 34020  34405     winceDestroyLock(pFile);
 34021  34406     if( pFile->zDeleteOnClose ){
 34022  34407       int cnt = 0;
 34023  34408       while(
 34024  34409              osDeleteFileW(pFile->zDeleteOnClose)==0
 34025         -        && osGetFileAttributesW(pFile->zDeleteOnClose)!=0xffffffff 
        34410  +        && osGetFileAttributesW(pFile->zDeleteOnClose)!=0xffffffff
 34026  34411           && cnt++ < WINCE_DELETION_ATTEMPTS
 34027  34412       ){
 34028  34413          sqlite3_win32_sleep(100);  /* Wait a little before trying again */
 34029  34414       }
 34030  34415       sqlite3_free(pFile->zDeleteOnClose);
 34031  34416     }
 34032  34417   #endif
................................................................................
 34866  35251   */
 34867  35252   static int winDeviceCharacteristics(sqlite3_file *id){
 34868  35253     winFile *p = (winFile*)id;
 34869  35254     return SQLITE_IOCAP_UNDELETABLE_WHEN_OPEN |
 34870  35255            ((p->ctrlFlags & WINFILE_PSOW)?SQLITE_IOCAP_POWERSAFE_OVERWRITE:0);
 34871  35256   }
 34872  35257   
 34873         -/* 
        35258  +/*
 34874  35259   ** Windows will only let you create file view mappings
 34875  35260   ** on allocation size granularity boundaries.
 34876  35261   ** During sqlite3_os_init() we do a GetSystemInfo()
 34877  35262   ** to get the granularity size.
 34878  35263   */
 34879  35264   static SYSTEM_INFO winSysInfo;
 34880  35265   
 34881  35266   #ifndef SQLITE_OMIT_WAL
 34882  35267   
 34883  35268   /*
 34884  35269   ** Helper functions to obtain and relinquish the global mutex. The
 34885         -** global mutex is used to protect the winLockInfo objects used by 
        35270  +** global mutex is used to protect the winLockInfo objects used by
 34886  35271   ** this file, all of which may be shared by multiple threads.
 34887  35272   **
 34888         -** Function winShmMutexHeld() is used to assert() that the global mutex 
 34889         -** is held when required. This function is only used as part of assert() 
        35273  +** Function winShmMutexHeld() is used to assert() that the global mutex
        35274  +** is held when required. This function is only used as part of assert()
 34890  35275   ** statements. e.g.
 34891  35276   **
 34892  35277   **   winShmEnterMutex()
 34893  35278   **     assert( winShmMutexHeld() );
 34894  35279   **   winShmLeaveMutex()
 34895  35280   */
 34896  35281   static void winShmEnterMutex(void){
................................................................................
 34912  35297   ** point to a single instance of this object.  In other words, each
 34913  35298   ** log-summary is opened only once per process.
 34914  35299   **
 34915  35300   ** winShmMutexHeld() must be true when creating or destroying
 34916  35301   ** this object or while reading or writing the following fields:
 34917  35302   **
 34918  35303   **      nRef
 34919         -**      pNext 
        35304  +**      pNext
 34920  35305   **
 34921  35306   ** The following fields are read-only after the object is created:
 34922         -** 
        35307  +**
 34923  35308   **      fid
 34924  35309   **      zFilename
 34925  35310   **
 34926  35311   ** Either winShmNode.mutex must be held or winShmNode.nRef==0 and
 34927  35312   ** winShmMutexHeld() is true when reading or writing any other field
 34928  35313   ** in this structure.
 34929  35314   **
................................................................................
 35011  35396       rc = winUnlockFile(&pFile->hFile.h, ofst, 0, nByte, 0);
 35012  35397     }else{
 35013  35398       /* Initialize the locking parameters */
 35014  35399       DWORD dwFlags = LOCKFILE_FAIL_IMMEDIATELY;
 35015  35400       if( lockType == _SHM_WRLCK ) dwFlags |= LOCKFILE_EXCLUSIVE_LOCK;
 35016  35401       rc = winLockFile(&pFile->hFile.h, dwFlags, ofst, 0, nByte, 0);
 35017  35402     }
 35018         -  
        35403  +
 35019  35404     if( rc!= 0 ){
 35020  35405       rc = SQLITE_OK;
 35021  35406     }else{
 35022  35407       pFile->lastErrno =  osGetLastError();
 35023  35408       rc = SQLITE_BUSY;
 35024  35409     }
 35025  35410   
................................................................................
 35107  35492     pNew = sqlite3MallocZero( sizeof(*pShmNode) + nName + 17 );
 35108  35493     if( pNew==0 ){
 35109  35494       sqlite3_free(p);
 35110  35495       return SQLITE_IOERR_NOMEM;
 35111  35496     }
 35112  35497     pNew->zFilename = (char*)&pNew[1];
 35113  35498     sqlite3_snprintf(nName+15, pNew->zFilename, "%s-shm", pDbFd->zPath);
 35114         -  sqlite3FileSuffix3(pDbFd->zPath, pNew->zFilename); 
        35499  +  sqlite3FileSuffix3(pDbFd->zPath, pNew->zFilename);
 35115  35500   
 35116  35501     /* Look to see if there is an existing winShmNode that can be used.
 35117  35502     ** If no matching winShmNode currently exists, create a new one.
 35118  35503     */
 35119  35504     winShmEnterMutex();
 35120  35505     for(pShmNode = winShmNodeList; pShmNode; pShmNode=pShmNode->pNext){
 35121  35506       /* TBD need to come up with better match here.  Perhaps
................................................................................
 35144  35529                    SQLITE_OPEN_WAL | SQLITE_OPEN_READWRITE | SQLITE_OPEN_CREATE,
 35145  35530                    0);
 35146  35531       if( SQLITE_OK!=rc ){
 35147  35532         goto shm_open_err;
 35148  35533       }
 35149  35534   
 35150  35535       /* Check to see if another process is holding the dead-man switch.
 35151         -    ** If not, truncate the file to zero length. 
        35536  +    ** If not, truncate the file to zero length.
 35152  35537       */
 35153  35538       if( winShmSystemLock(pShmNode, _SHM_WRLCK, WIN_SHM_DMS, 1)==SQLITE_OK ){
 35154  35539         rc = winTruncate((sqlite3_file *)&pShmNode->hFile, 0);
 35155  35540         if( rc!=SQLITE_OK ){
 35156  35541           rc = winLogError(SQLITE_IOERR_SHMOPEN, osGetLastError(),
 35157  35542                            "winOpenShm", pDbFd->zPath);
 35158  35543         }
................................................................................
 35173  35558     pDbFd->pShm = p;
 35174  35559     winShmLeaveMutex();
 35175  35560   
 35176  35561     /* The reference count on pShmNode has already been incremented under
 35177  35562     ** the cover of the winShmEnterMutex() mutex and the pointer from the
 35178  35563     ** new (struct winShm) object to the pShmNode has been set. All that is
 35179  35564     ** left to do is to link the new object into the linked list starting
 35180         -  ** at pShmNode->pFirst. This must be done while holding the pShmNode->mutex 
        35565  +  ** at pShmNode->pFirst. This must be done while holding the pShmNode->mutex
 35181  35566     ** mutex.
 35182  35567     */
 35183  35568     sqlite3_mutex_enter(pShmNode->mutex);
 35184  35569     p->pNext = pShmNode->pFirst;
 35185  35570     pShmNode->pFirst = p;
 35186  35571     sqlite3_mutex_leave(pShmNode->mutex);
 35187  35572     return SQLITE_OK;
................................................................................
 35193  35578     sqlite3_free(p);
 35194  35579     sqlite3_free(pNew);
 35195  35580     winShmLeaveMutex();
 35196  35581     return rc;
 35197  35582   }
 35198  35583   
 35199  35584   /*
 35200         -** Close a connection to shared-memory.  Delete the underlying 
        35585  +** Close a connection to shared-memory.  Delete the underlying
 35201  35586   ** storage if deleteFlag is true.
 35202  35587   */
 35203  35588   static int winShmUnmap(
 35204  35589     sqlite3_file *fd,          /* Database holding shared memory */
 35205  35590     int deleteFlag             /* Delete after closing if true */
 35206  35591   ){
 35207  35592     winFile *pDbFd;       /* Database holding shared-memory */
................................................................................
 35282  35667         rc = SQLITE_OK;
 35283  35668       }
 35284  35669   
 35285  35670       /* Undo the local locks */
 35286  35671       if( rc==SQLITE_OK ){
 35287  35672         p->exclMask &= ~mask;
 35288  35673         p->sharedMask &= ~mask;
 35289         -    } 
        35674  +    }
 35290  35675     }else if( flags & SQLITE_SHM_SHARED ){
 35291  35676       u16 allShared = 0;  /* Union of locks held by connections other than "p" */
 35292  35677   
 35293  35678       /* Find out which shared locks are already held by sibling connections.
 35294  35679       ** If any sibling already holds an exclusive lock, go ahead and return
 35295  35680       ** SQLITE_BUSY.
 35296  35681       */
................................................................................
 35321  35706       */
 35322  35707       for(pX=pShmNode->pFirst; pX; pX=pX->pNext){
 35323  35708         if( (pX->exclMask & mask)!=0 || (pX->sharedMask & mask)!=0 ){
 35324  35709           rc = SQLITE_BUSY;
 35325  35710           break;
 35326  35711         }
 35327  35712       }
 35328         -  
        35713  +
 35329  35714       /* Get the exclusive locks at the system level.  Then if successful
 35330  35715       ** also mark the local connection as being locked.
 35331  35716       */
 35332  35717       if( rc==SQLITE_OK ){
 35333  35718         rc = winShmSystemLock(pShmNode, _SHM_WRLCK, ofst+WIN_SHM_BASE, n);
 35334  35719         if( rc==SQLITE_OK ){
 35335  35720           assert( (p->sharedMask & mask)==0 );
................................................................................
 35341  35726     OSTRACE(("SHM-LOCK pid=%lu, id=%d, sharedMask=%03x, exclMask=%03x, rc=%s\n",
 35342  35727              osGetCurrentProcessId(), p->id, p->sharedMask, p->exclMask,
 35343  35728              sqlite3ErrName(rc)));
 35344  35729     return rc;
 35345  35730   }
 35346  35731   
 35347  35732   /*
 35348         -** Implement a memory barrier or memory fence on shared memory.  
        35733  +** Implement a memory barrier or memory fence on shared memory.
 35349  35734   **
 35350  35735   ** All loads and stores begun before the barrier must complete before
 35351  35736   ** any load or store begun after the barrier.
 35352  35737   */
 35353  35738   static void winShmBarrier(
 35354  35739     sqlite3_file *fd          /* Database holding the shared memory */
 35355  35740   ){
................................................................................
 35356  35741     UNUSED_PARAMETER(fd);
 35357  35742     /* MemoryBarrier(); // does not work -- do not know why not */
 35358  35743     winShmEnterMutex();
 35359  35744     winShmLeaveMutex();
 35360  35745   }
 35361  35746   
 35362  35747   /*
 35363         -** This function is called to obtain a pointer to region iRegion of the 
 35364         -** shared-memory associated with the database file fd. Shared-memory regions 
 35365         -** are numbered starting from zero. Each shared-memory region is szRegion 
        35748  +** This function is called to obtain a pointer to region iRegion of the
        35749  +** shared-memory associated with the database file fd. Shared-memory regions
        35750  +** are numbered starting from zero. Each shared-memory region is szRegion
 35366  35751   ** bytes in size.
 35367  35752   **
 35368  35753   ** If an error occurs, an error code is returned and *pp is set to NULL.
 35369  35754   **
 35370  35755   ** Otherwise, if the isWrite parameter is 0 and the requested shared-memory
 35371  35756   ** region has not been allocated (by any client, including one running in a
 35372         -** separate process), then *pp is set to NULL and SQLITE_OK returned. If 
 35373         -** isWrite is non-zero and the requested shared-memory region has not yet 
        35757  +** separate process), then *pp is set to NULL and SQLITE_OK returned. If
        35758  +** isWrite is non-zero and the requested shared-memory region has not yet
 35374  35759   ** been allocated, it is allocated by this function.
 35375  35760   **
 35376  35761   ** If the shared-memory region has already been allocated or is allocated by
 35377         -** this call as described above, then it is mapped into this processes 
 35378         -** address space (if it is not already), *pp is set to point to the mapped 
        35762  +** this call as described above, then it is mapped into this processes
        35763  +** address space (if it is not already), *pp is set to point to the mapped
 35379  35764   ** memory and SQLITE_OK returned.
 35380  35765   */
 35381  35766   static int winShmMap(
 35382  35767     sqlite3_file *fd,               /* Handle open on database file */
 35383  35768     int iRegion,                    /* Region to retrieve */
 35384  35769     int szRegion,                   /* Size of regions */
 35385  35770     int isWrite,                    /* True to extend file if necessary */
................................................................................
 35443  35828         goto shmpage_out;
 35444  35829       }
 35445  35830       pShmNode->aRegion = apNew;
 35446  35831   
 35447  35832       while( pShmNode->nRegion<=iRegion ){
 35448  35833         HANDLE hMap = NULL;         /* file-mapping handle */
 35449  35834         void *pMap = 0;             /* Mapped memory region */
 35450         -     
        35835  +
 35451  35836   #if SQLITE_OS_WINRT
 35452  35837         hMap = osCreateFileMappingFromApp(pShmNode->hFile.h,
 35453  35838             NULL, PAGE_READWRITE, nByte, NULL
 35454  35839         );
 35455  35840   #elif defined(SQLITE_WIN32_HAS_WIDE)
 35456         -      hMap = osCreateFileMappingW(pShmNode->hFile.h, 
        35841  +      hMap = osCreateFileMappingW(pShmNode->hFile.h,
 35457  35842             NULL, PAGE_READWRITE, 0, nByte, NULL
 35458  35843         );
 35459  35844   #elif defined(SQLITE_WIN32_HAS_ANSI)
 35460         -      hMap = osCreateFileMappingA(pShmNode->hFile.h, 
        35845  +      hMap = osCreateFileMappingA(pShmNode->hFile.h,
 35461  35846             NULL, PAGE_READWRITE, 0, nByte, NULL
 35462  35847         );
 35463  35848   #endif
 35464  35849         OSTRACE(("SHM-MAP-CREATE pid=%lu, region=%d, size=%d, rc=%s\n",
 35465  35850                  osGetCurrentProcessId(), pShmNode->nRegion, nByte,
 35466  35851                  hMap ? "ok" : "failed"));
 35467  35852         if( hMap ){
................................................................................
 35550  35935     OSTRACE(("UNMAP-FILE pid=%lu, pFile=%p, rc=SQLITE_OK\n",
 35551  35936              osGetCurrentProcessId(), pFile));
 35552  35937     return SQLITE_OK;
 35553  35938   }
 35554  35939   
 35555  35940   /*
 35556  35941   ** Memory map or remap the file opened by file-descriptor pFd (if the file
 35557         -** is already mapped, the existing mapping is replaced by the new). Or, if 
 35558         -** there already exists a mapping for this file, and there are still 
        35942  +** is already mapped, the existing mapping is replaced by the new). Or, if
        35943  +** there already exists a mapping for this file, and there are still
 35559  35944   ** outstanding xFetch() references to it, this function is a no-op.
 35560  35945   **
 35561         -** If parameter nByte is non-negative, then it is the requested size of 
 35562         -** the mapping to create. Otherwise, if nByte is less than zero, then the 
        35946  +** If parameter nByte is non-negative, then it is the requested size of
        35947  +** the mapping to create. Otherwise, if nByte is less than zero, then the
 35563  35948   ** requested size is the size of the file on disk. The actual size of the
 35564         -** created mapping is either the requested size or the value configured 
        35949  +** created mapping is either the requested size or the value configured
 35565  35950   ** using SQLITE_FCNTL_MMAP_SIZE, whichever is smaller.
 35566  35951   **
 35567  35952   ** SQLITE_OK is returned if no error occurs (even if the mapping is not
 35568  35953   ** recreated as a result of outstanding references) or an SQLite error
 35569  35954   ** code otherwise.
 35570  35955   */
 35571  35956   static int winMapfile(winFile *pFd, sqlite3_int64 nByte){
................................................................................
 35586  35971         return SQLITE_IOERR_FSTAT;
 35587  35972       }
 35588  35973     }
 35589  35974     if( nMap>pFd->mmapSizeMax ){
 35590  35975       nMap = pFd->mmapSizeMax;
 35591  35976     }
 35592  35977     nMap &= ~(sqlite3_int64)(winSysInfo.dwPageSize - 1);
 35593         - 
        35978  +
 35594  35979     if( nMap==0 && pFd->mmapSize>0 ){
 35595  35980       winUnmapfile(pFd);
 35596  35981     }
 35597  35982     if( nMap!=pFd->mmapSize ){
 35598  35983       void *pNew = 0;
 35599  35984       DWORD protect = PAGE_READONLY;
 35600  35985       DWORD flags = FILE_MAP_READ;
................................................................................
 35658  36043   ** iOff. The mapping must be valid for at least nAmt bytes.
 35659  36044   **
 35660  36045   ** If such a pointer can be obtained, store it in *pp and return SQLITE_OK.
 35661  36046   ** Or, if one cannot but no error occurs, set *pp to 0 and return SQLITE_OK.
 35662  36047   ** Finally, if an error does occur, return an SQLite error code. The final
 35663  36048   ** value of *pp is undefined in this case.
 35664  36049   **
 35665         -** If this function does return a pointer, the caller must eventually 
        36050  +** If this function does return a pointer, the caller must eventually
 35666  36051   ** release the reference by calling winUnfetch().
 35667  36052   */
 35668  36053   static int winFetch(sqlite3_file *fd, i64 iOff, int nAmt, void **pp){
 35669  36054   #if SQLITE_MAX_MMAP_SIZE>0
 35670  36055     winFile *pFd = (winFile*)fd;   /* The underlying database file */
 35671  36056   #endif
 35672  36057     *pp = 0;
................................................................................
 35693  36078   
 35694  36079     OSTRACE(("FETCH pid=%lu, pFile=%p, pp=%p, *pp=%p, rc=SQLITE_OK\n",
 35695  36080              osGetCurrentProcessId(), fd, pp, *pp));
 35696  36081     return SQLITE_OK;
 35697  36082   }
 35698  36083   
 35699  36084   /*
 35700         -** If the third argument is non-NULL, then this function releases a 
        36085  +** If the third argument is non-NULL, then this function releases a
 35701  36086   ** reference obtained by an earlier call to winFetch(). The second
 35702  36087   ** argument passed to this function must be the same as the corresponding
 35703         -** argument that was passed to the winFetch() invocation. 
        36088  +** argument that was passed to the winFetch() invocation.
 35704  36089   **
 35705         -** Or, if the third argument is NULL, then this function is being called 
 35706         -** to inform the VFS layer that, according to POSIX, any existing mapping 
        36090  +** Or, if the third argument is NULL, then this function is being called
        36091  +** to inform the VFS layer that, according to POSIX, any existing mapping
 35707  36092   ** may now be invalid and should be unmapped.
 35708  36093   */
 35709  36094   static int winUnfetch(sqlite3_file *fd, i64 iOff, void *p){
 35710  36095   #if SQLITE_MAX_MMAP_SIZE>0
 35711  36096     winFile *pFd = (winFile*)fd;   /* The underlying database file */
 35712  36097   
 35713         -  /* If p==0 (unmap the entire file) then there must be no outstanding 
        36098  +  /* If p==0 (unmap the entire file) then there must be no outstanding
 35714  36099     ** xFetch references. Or, if p!=0 (meaning it is an xFetch reference),
 35715  36100     ** then there must be at least one outstanding.  */
 35716  36101     assert( (p==0)==(pFd->nFetchOut==0) );
 35717  36102   
 35718  36103     /* If p!=0, it must match the iOff value. */
 35719  36104     assert( p==0 || p==&((u8 *)pFd->pMapRegion)[iOff] );
 35720  36105   
................................................................................
 35852  36237     size_t i, j;
 35853  36238     int nPre = sqlite3Strlen30(SQLITE_TEMP_FILE_PREFIX);
 35854  36239     int nMax, nBuf, nDir, nLen;
 35855  36240     char *zBuf;
 35856  36241   
 35857  36242     /* It's odd to simulate an io-error here, but really this is just
 35858  36243     ** using the io-error infrastructure to test that SQLite handles this
 35859         -  ** function failing. 
        36244  +  ** function failing.
 35860  36245     */
 35861  36246     SimulateIOError( return SQLITE_IOERR );
 35862  36247   
 35863  36248     /* Allocate a temporary buffer to store the fully qualified file
 35864  36249     ** name for the temporary file.  If this fails, we cannot continue.
 35865  36250     */
 35866  36251     nMax = pVfs->mxPathname; nBuf = nMax + 2;
................................................................................
 36034  36419     if( !winMakeEndInDirSep(nDir+1, zBuf) ){
 36035  36420       sqlite3_free(zBuf);
 36036  36421       OSTRACE(("TEMP-FILENAME rc=SQLITE_ERROR\n"));
 36037  36422       return winLogError(SQLITE_ERROR, 0, "winGetTempname4", 0);
 36038  36423     }
 36039  36424   
 36040  36425     /*
 36041         -  ** Check that the output buffer is large enough for the temporary file 
        36426  +  ** Check that the output buffer is large enough for the temporary file
 36042  36427     ** name in the following format:
 36043  36428     **
 36044  36429     **   "<temporary_directory>/etilqs_XXXXXXXXXXXXXXX\0\0"
 36045  36430     **
 36046  36431     ** If not, return SQLITE_ERROR.  The number 17 is used here in order to
 36047  36432     ** account for the space used by the 15 character random suffix and the
 36048  36433     ** two trailing NUL characters.  The final directory separator character
................................................................................
 36137  36522     int isDelete     = (flags & SQLITE_OPEN_DELETEONCLOSE);
 36138  36523     int isCreate     = (flags & SQLITE_OPEN_CREATE);
 36139  36524     int isReadonly   = (flags & SQLITE_OPEN_READONLY);
 36140  36525     int isReadWrite  = (flags & SQLITE_OPEN_READWRITE);
 36141  36526   
 36142  36527   #ifndef NDEBUG
 36143  36528     int isOpenJournal = (isCreate && (
 36144         -        eType==SQLITE_OPEN_MASTER_JOURNAL 
 36145         -     || eType==SQLITE_OPEN_MAIN_JOURNAL 
        36529  +        eType==SQLITE_OPEN_MASTER_JOURNAL
        36530  +     || eType==SQLITE_OPEN_MAIN_JOURNAL
 36146  36531        || eType==SQLITE_OPEN_WAL
 36147  36532     ));
 36148  36533   #endif
 36149  36534   
 36150  36535     OSTRACE(("OPEN name=%s, pFile=%p, flags=%x, pOutFlags=%p\n",
 36151  36536              zUtf8Name, id, flags, pOutFlags));
 36152  36537   
 36153         -  /* Check the following statements are true: 
        36538  +  /* Check the following statements are true:
 36154  36539     **
 36155         -  **   (a) Exactly one of the READWRITE and READONLY flags must be set, and 
        36540  +  **   (a) Exactly one of the READWRITE and READONLY flags must be set, and
 36156  36541     **   (b) if CREATE is set, then READWRITE must also be set, and
 36157  36542     **   (c) if EXCLUSIVE is set, then CREATE must also be set.
 36158  36543     **   (d) if DELETEONCLOSE is set, then CREATE must also be set.
 36159  36544     */
 36160  36545     assert((isReadonly==0 || isReadWrite==0) && (isReadWrite || isReadonly));
 36161  36546     assert(isCreate==0 || isReadWrite);
 36162  36547     assert(isExclusive==0 || isCreate);
 36163  36548     assert(isDelete==0 || isCreate);
 36164  36549   
 36165         -  /* The main DB, main journal, WAL file and master journal are never 
        36550  +  /* The main DB, main journal, WAL file and master journal are never
 36166  36551     ** automatically deleted. Nor are they ever temporary files.  */
 36167  36552     assert( (!isDelete && zName) || eType!=SQLITE_OPEN_MAIN_DB );
 36168  36553     assert( (!isDelete && zName) || eType!=SQLITE_OPEN_MAIN_JOURNAL );
 36169  36554     assert( (!isDelete && zName) || eType!=SQLITE_OPEN_MASTER_JOURNAL );
 36170  36555     assert( (!isDelete && zName) || eType!=SQLITE_OPEN_WAL );
 36171  36556   
 36172  36557     /* Assert that the upper layer has set one of the "file-type" flags. */
 36173         -  assert( eType==SQLITE_OPEN_MAIN_DB      || eType==SQLITE_OPEN_TEMP_DB 
 36174         -       || eType==SQLITE_OPEN_MAIN_JOURNAL || eType==SQLITE_OPEN_TEMP_JOURNAL 
 36175         -       || eType==SQLITE_OPEN_SUBJOURNAL   || eType==SQLITE_OPEN_MASTER_JOURNAL 
        36558  +  assert( eType==SQLITE_OPEN_MAIN_DB      || eType==SQLITE_OPEN_TEMP_DB
        36559  +       || eType==SQLITE_OPEN_MAIN_JOURNAL || eType==SQLITE_OPEN_TEMP_JOURNAL
        36560  +       || eType==SQLITE_OPEN_SUBJOURNAL   || eType==SQLITE_OPEN_MASTER_JOURNAL
 36176  36561          || eType==SQLITE_OPEN_TRANSIENT_DB || eType==SQLITE_OPEN_WAL
 36177  36562     );
 36178  36563   
 36179  36564     assert( pFile!=0 );
 36180  36565     memset(pFile, 0, sizeof(winFile));
 36181  36566     pFile->h = INVALID_HANDLE_VALUE;
 36182  36567   
................................................................................
 36183  36568   #if SQLITE_OS_WINRT
 36184  36569     if( !zUtf8Name && !sqlite3_temp_directory ){
 36185  36570       sqlite3_log(SQLITE_ERROR,
 36186  36571           "sqlite3_temp_directory variable should be set for WinRT");
 36187  36572     }
 36188  36573   #endif
 36189  36574   
 36190         -  /* If the second argument to this function is NULL, generate a 
 36191         -  ** temporary file name to use 
        36575  +  /* If the second argument to this function is NULL, generate a
        36576  +  ** temporary file name to use
 36192  36577     */
 36193  36578     if( !zUtf8Name ){
 36194  36579       assert( isDelete && !isOpenJournal );
 36195  36580       rc = winGetTempname(pVfs, &zTmpname);
 36196  36581       if( rc!=SQLITE_OK ){
 36197  36582         OSTRACE(("OPEN name=%s, rc=%s", zUtf8Name, sqlite3ErrName(rc)));
 36198  36583         return rc;
................................................................................
 36224  36609   
 36225  36610     if( isReadWrite ){
 36226  36611       dwDesiredAccess = GENERIC_READ | GENERIC_WRITE;
 36227  36612     }else{
 36228  36613       dwDesiredAccess = GENERIC_READ;
 36229  36614     }
 36230  36615   
 36231         -  /* SQLITE_OPEN_EXCLUSIVE is used to make sure that a new file is 
 36232         -  ** created. SQLite doesn't use it to indicate "exclusive access" 
        36616  +  /* SQLITE_OPEN_EXCLUSIVE is used to make sure that a new file is
        36617  +  ** created. SQLite doesn't use it to indicate "exclusive access"
 36233  36618     ** as it is usually understood.
 36234  36619     */
 36235  36620     if( isExclusive ){
 36236  36621       /* Creates a new file, only if it does not already exist. */
 36237  36622       /* If the file exists, it fails. */
 36238  36623       dwCreationDisposition = CREATE_NEW;
 36239  36624     }else if( isCreate ){
................................................................................
 36314  36699   
 36315  36700     if( h==INVALID_HANDLE_VALUE ){
 36316  36701       pFile->lastErrno = lastErrno;
 36317  36702       winLogError(SQLITE_CANTOPEN, pFile->lastErrno, "winOpen", zUtf8Name);
 36318  36703       sqlite3_free(zConverted);
 36319  36704       sqlite3_free(zTmpname);
 36320  36705       if( isReadWrite && !isExclusive ){
 36321         -      return winOpen(pVfs, zName, id, 
        36706  +      return winOpen(pVfs, zName, id,
 36322  36707            ((flags|SQLITE_OPEN_READONLY) &
 36323  36708                        ~(SQLITE_OPEN_CREATE|SQLITE_OPEN_READWRITE)),
 36324  36709            pOutFlags);
 36325  36710       }else{
 36326  36711         return SQLITE_CANTOPEN_BKPT;
 36327  36712       }
 36328  36713     }
................................................................................
 36523  36908       return SQLITE_IOERR_NOMEM;
 36524  36909     }
 36525  36910     if( osIsNT() ){
 36526  36911       int cnt = 0;
 36527  36912       WIN32_FILE_ATTRIBUTE_DATA sAttrData;
 36528  36913       memset(&sAttrData, 0, sizeof(sAttrData));
 36529  36914       while( !(rc = osGetFileAttributesExW((LPCWSTR)zConverted,
 36530         -                             GetFileExInfoStandard, 
        36915  +                             GetFileExInfoStandard,
 36531  36916                                &sAttrData)) && winRetryIoerr(&cnt, &lastErrno) ){}
 36532  36917       if( rc ){
 36533  36918         /* For an SQLITE_ACCESS_EXISTS query, treat a zero-length file
 36534  36919         ** as if it does not exist.
 36535  36920         */
 36536  36921         if(    flags==SQLITE_ACCESS_EXISTS
 36537         -          && sAttrData.nFileSizeHigh==0 
        36922  +          && sAttrData.nFileSizeHigh==0
 36538  36923             && sAttrData.nFileSizeLow==0 ){
 36539  36924           attr = INVALID_FILE_ATTRIBUTES;
 36540  36925         }else{
 36541  36926           attr = sAttrData.dwFileAttributes;
 36542  36927         }
 36543  36928       }else{
 36544  36929         winLogIoerr(cnt);
................................................................................
 36629  37014   */
 36630  37015   static int winFullPathname(
 36631  37016     sqlite3_vfs *pVfs,            /* Pointer to vfs object */
 36632  37017     const char *zRelative,        /* Possibly relative input path */
 36633  37018     int nFull,                    /* Size of output buffer in bytes */
 36634  37019     char *zFull                   /* Output buffer */
 36635  37020   ){
 36636         -  
        37021  +
 36637  37022   #if defined(__CYGWIN__)
 36638  37023     SimulateIOError( return SQLITE_ERROR );
 36639  37024     UNUSED_PARAMETER(nFull);
 36640  37025     assert( nFull>=pVfs->mxPathname );
 36641  37026     if ( sqlite3_data_directory && !winIsVerbatimPathname(zRelative) ){
 36642  37027       /*
 36643  37028       ** NOTE: We are dealing with a relative path name and the data
................................................................................
 36942  37327   /*
 36943  37328   ** Find the current time (in Universal Coordinated Time).  Write into *piNow
 36944  37329   ** the current time and date as a Julian Day number times 86_400_000.  In
 36945  37330   ** other words, write into *piNow the number of milliseconds since the Julian
 36946  37331   ** epoch of noon in Greenwich on November 24, 4714 B.C according to the
 36947  37332   ** proleptic Gregorian calendar.
 36948  37333   **
 36949         -** On success, return SQLITE_OK.  Return SQLITE_ERROR if the time and date 
        37334  +** On success, return SQLITE_OK.  Return SQLITE_ERROR if the time and date
 36950  37335   ** cannot be found.
 36951  37336   */
 36952  37337   static int winCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *piNow){
 36953         -  /* FILETIME structure is a 64-bit value representing the number of 
 36954         -     100-nanosecond intervals since January 1, 1601 (= JD 2305813.5). 
        37338  +  /* FILETIME structure is a 64-bit value representing the number of
        37339  +     100-nanosecond intervals since January 1, 1601 (= JD 2305813.5).
 36955  37340     */
 36956  37341     FILETIME ft;
 36957  37342     static const sqlite3_int64 winFiletimeEpoch = 23058135*(sqlite3_int64)8640000;
 36958  37343   #ifdef SQLITE_TEST
 36959  37344     static const sqlite3_int64 unixEpoch = 24405875*(sqlite3_int64)8640000;
 36960  37345   #endif
 36961  37346     /* 2^32 - to avoid use of LL and warnings in gcc */
 36962         -  static const sqlite3_int64 max32BitValue = 
        37347  +  static const sqlite3_int64 max32BitValue =
 36963  37348         (sqlite3_int64)2000000000 + (sqlite3_int64)2000000000 +
 36964  37349         (sqlite3_int64)294967296;
 36965  37350   
 36966  37351   #if SQLITE_OS_WINCE
 36967  37352     SYSTEMTIME time;
 36968  37353     osGetSystemTime(&time);
 36969  37354     /* if SystemTimeToFileTime() fails, it returns zero. */
................................................................................
 36971  37356       return SQLITE_ERROR;
 36972  37357     }
 36973  37358   #else
 36974  37359     osGetSystemTimeAsFileTime( &ft );
 36975  37360   #endif
 36976  37361   
 36977  37362     *piNow = winFiletimeEpoch +
 36978         -            ((((sqlite3_int64)ft.dwHighDateTime)*max32BitValue) + 
        37363  +            ((((sqlite3_int64)ft.dwHighDateTime)*max32BitValue) +
 36979  37364                  (sqlite3_int64)ft.dwLowDateTime)/(sqlite3_int64)10000;
 36980  37365   
 36981  37366   #ifdef SQLITE_TEST
 36982  37367     if( sqlite3_current_time ){
 36983  37368       *piNow = 1000*(sqlite3_int64)sqlite3_current_time + unixEpoch;
 36984  37369     }
 36985  37370   #endif
................................................................................
 37090  37475       winGetSystemCall,    /* xGetSystemCall */
 37091  37476       winNextSystemCall,   /* xNextSystemCall */
 37092  37477     };
 37093  37478   #endif
 37094  37479   
 37095  37480     /* Double-check that the aSyscall[] array has been constructed
 37096  37481     ** correctly.  See ticket [bb3a86e890c8e96ab] */
 37097         -  assert( ArraySize(aSyscall)==76 );
        37482  +  assert( ArraySize(aSyscall)==77 );
 37098  37483   
 37099  37484     /* get memory map allocation granularity */
 37100  37485     memset(&winSysInfo, 0, sizeof(SYSTEM_INFO));
 37101  37486   #if SQLITE_OS_WINRT
 37102  37487     osGetNativeSystemInfo(&winSysInfo);
 37103  37488   #else
 37104  37489     osGetSystemInfo(&winSysInfo);
................................................................................
 37108  37493   
 37109  37494     sqlite3_vfs_register(&winVfs, 1);
 37110  37495   
 37111  37496   #if defined(SQLITE_WIN32_HAS_WIDE)
 37112  37497     sqlite3_vfs_register(&winLongPathVfs, 0);
 37113  37498   #endif
 37114  37499   
 37115         -  return SQLITE_OK; 
        37500  +  return SQLITE_OK;
 37116  37501   }
 37117  37502   
 37118         -SQLITE_API int sqlite3_os_end(void){ 
        37503  +SQLITE_API int sqlite3_os_end(void){
 37119  37504   #if SQLITE_OS_WINRT
 37120  37505     if( sleepObj!=NULL ){
 37121  37506       osCloseHandle(sleepObj);
 37122  37507       sleepObj = NULL;
 37123  37508     }
 37124  37509   #endif
 37125  37510     return SQLITE_OK;
................................................................................
 49187  49572     ** during the few nanoseconds that it is holding the lock.  In that case,
 49188  49573     ** it might take longer than normal for the lock to free.
 49189  49574     **
 49190  49575     ** After 5 RETRYs, we begin calling sqlite3OsSleep().  The first few
 49191  49576     ** calls to sqlite3OsSleep() have a delay of 1 microsecond.  Really this
 49192  49577     ** is more of a scheduler yield than an actual delay.  But on the 10th
 49193  49578     ** an subsequent retries, the delays start becoming longer and longer, 
 49194         -  ** so that on the 100th (and last) RETRY we delay for 21 milliseconds.
 49195         -  ** The total delay time before giving up is less than 1 second.
        49579  +  ** so that on the 100th (and last) RETRY we delay for 323 milliseconds.
        49580  +  ** The total delay time before giving up is less than 10 seconds.
 49196  49581     */
 49197  49582     if( cnt>5 ){
 49198  49583       int nDelay = 1;                      /* Pause time in microseconds */
 49199  49584       if( cnt>100 ){
 49200  49585         VVA_ONLY( pWal->lockError = 1; )
 49201  49586         return SQLITE_PROTOCOL;
 49202  49587       }
 49203         -    if( cnt>=10 ) nDelay = (cnt-9)*238;  /* Max delay 21ms. Total delay 996ms */
        49588  +    if( cnt>=10 ) nDelay = (cnt-9)*(cnt-9)*39;
 49204  49589       sqlite3OsSleep(pWal->pVfs, nDelay);
 49205  49590     }
 49206  49591   
 49207  49592     if( !useWal ){
 49208  49593       rc = walIndexReadHdr(pWal, pChanged);
 49209  49594       if( rc==SQLITE_BUSY ){
 49210  49595         /* If there is not a recovery running in another thread or process
................................................................................
 51312  51697     }
 51313  51698   
 51314  51699     /* If the client is reading  or writing an index and the schema is
 51315  51700     ** not loaded, then it is too difficult to actually check to see if
 51316  51701     ** the correct locks are held.  So do not bother - just return true.
 51317  51702     ** This case does not come up very often anyhow.
 51318  51703     */
 51319         -  if( isIndex && (!pSchema || (pSchema->flags&DB_SchemaLoaded)==0) ){
        51704  +  if( isIndex && (!pSchema || (pSchema->schemaFlags&DB_SchemaLoaded)==0) ){
 51320  51705       return 1;
 51321  51706     }
 51322  51707   
 51323  51708     /* Figure out the root-page that the lock should be held on. For table
 51324  51709     ** b-trees, this is just the root page of the b-tree being read or
 51325  51710     ** written. For index b-trees, it is the root page of the associated
 51326  51711     ** table.  */
................................................................................
 52782  53167   */
 52783  53168   static Pgno btreePagecount(BtShared *pBt){
 52784  53169     return pBt->nPage;
 52785  53170   }
 52786  53171   SQLITE_PRIVATE u32 sqlite3BtreeLastPage(Btree *p){
 52787  53172     assert( sqlite3BtreeHoldsMutex(p) );
 52788  53173     assert( ((p->pBt->nPage)&0x8000000)==0 );
 52789         -  return (int)btreePagecount(p->pBt);
        53174  +  return btreePagecount(p->pBt);
 52790  53175   }
 52791  53176   
 52792  53177   /*
 52793  53178   ** Get a page from the pager and initialize it.  This routine is just a
 52794  53179   ** convenience wrapper around separate calls to btreeGetPage() and 
 52795  53180   ** btreeInitPage().
 52796  53181   **
................................................................................
 56891  57276     int cellOffset;   /* Address of first cell pointer in data[] */
 56892  57277     u8 *data;         /* The content of the whole page */
 56893  57278     int nSkip = (iChild ? 4 : 0);
 56894  57279   
 56895  57280     if( *pRC ) return;
 56896  57281   
 56897  57282     assert( i>=0 && i<=pPage->nCell+pPage->nOverflow );
 56898         -  assert( pPage->nCell<=MX_CELL(pPage->pBt) && MX_CELL(pPage->pBt)<=10921 );
        57283  +  assert( MX_CELL(pPage->pBt)<=10921 );
        57284  +  assert( pPage->nCell<=MX_CELL(pPage->pBt) || CORRUPT_DB );
 56899  57285     assert( pPage->nOverflow<=ArraySize(pPage->apOvfl) );
 56900  57286     assert( ArraySize(pPage->apOvfl)==ArraySize(pPage->aiOvfl) );
 56901  57287     assert( sqlite3_mutex_held(pPage->pBt->mutex) );
 56902  57288     /* The cell should normally be sized correctly.  However, when moving a
 56903  57289     ** malformed cell from a leaf page to an interior page, if the cell size
 56904  57290     ** wanted to be less than 4 but got rounded up to 4 on the leaf, then size
 56905  57291     ** might be less than 8 (leaf-size + pointer) on the interior node.  Hence
................................................................................
 61580  61966     int i;
 61581  61967     FuncDefHash *pHash = &GLOBAL(FuncDefHash, sqlite3GlobalFunctions);
 61582  61968     FuncDef *aFunc = (FuncDef*)&GLOBAL(FuncDef, aAnalyzeTableFuncs);
 61583  61969     for(i=0; i<ArraySize(aAnalyzeTableFuncs); i++){
 61584  61970       sqlite3FuncDefInsert(pHash, &aFunc[i]);
 61585  61971     }
 61586  61972   }
        61973  +
        61974  +/*
        61975  +** Attempt to extract a value from pExpr and use it to construct *ppVal.
        61976  +**
        61977  +** If pAlloc is not NULL, then an UnpackedRecord object is created for
        61978  +** pAlloc if one does not exist and the new value is added to the
        61979  +** UnpackedRecord object.
        61980  +**
        61981  +** A value is extracted in the following cases:
        61982  +**
        61983  +**  * (pExpr==0). In this case the value is assumed to be an SQL NULL,
        61984  +**
        61985  +**  * The expression is a bound variable, and this is a reprepare, or
        61986  +**
        61987  +**  * The expression is a literal value.
        61988  +**
        61989  +** On success, *ppVal is made to point to the extracted value.  The caller
        61990  +** is responsible for ensuring that the value is eventually freed.
        61991  +*/
        61992  +static int stat4ValueFromExpr(
        61993  +  Parse *pParse,                  /* Parse context */
        61994  +  Expr *pExpr,                    /* The expression to extract a value from */
        61995  +  u8 affinity,                    /* Affinity to use */
        61996  +  struct ValueNewStat4Ctx *pAlloc,/* How to allocate space.  Or NULL */
        61997  +  sqlite3_value **ppVal           /* OUT: New value object (or NULL) */
        61998  +){
        61999  +  int rc = SQLITE_OK;
        62000  +  sqlite3_value *pVal = 0;
        62001  +  sqlite3 *db = pParse->db;
        62002  +
        62003  +  /* Skip over any TK_COLLATE nodes */
        62004  +  pExpr = sqlite3ExprSkipCollate(pExpr);
        62005  +
        62006  +  if( !pExpr ){
        62007  +    pVal = valueNew(db, pAlloc);
        62008  +    if( pVal ){
        62009  +      sqlite3VdbeMemSetNull((Mem*)pVal);
        62010  +    }
        62011  +  }else if( pExpr->op==TK_VARIABLE
        62012  +        || NEVER(pExpr->op==TK_REGISTER && pExpr->op2==TK_VARIABLE)
        62013  +  ){
        62014  +    Vdbe *v;
        62015  +    int iBindVar = pExpr->iColumn;
        62016  +    sqlite3VdbeSetVarmask(pParse->pVdbe, iBindVar);
        62017  +    if( (v = pParse->pReprepare)!=0 ){
        62018  +      pVal = valueNew(db, pAlloc);
        62019  +      if( pVal ){
        62020  +        rc = sqlite3VdbeMemCopy((Mem*)pVal, &v->aVar[iBindVar-1]);
        62021  +        if( rc==SQLITE_OK ){
        62022  +          sqlite3ValueApplyAffinity(pVal, affinity, ENC(db));
        62023  +        }
        62024  +        pVal->db = pParse->db;
        62025  +      }
        62026  +    }
        62027  +  }else{
        62028  +    rc = valueFromExpr(db, pExpr, ENC(db), affinity, &pVal, pAlloc);
        62029  +  }
        62030  +
        62031  +  assert( pVal==0 || pVal->db==db );
        62032  +  *ppVal = pVal;
        62033  +  return rc;
        62034  +}
 61587  62035   
 61588  62036   /*
 61589  62037   ** This function is used to allocate and populate UnpackedRecord 
 61590  62038   ** structures intended to be compared against sample index keys stored 
 61591  62039   ** in the sqlite_stat4 table.
 61592  62040   **
 61593  62041   ** A single call to this function attempts to populates field iVal (leftmost 
................................................................................
 61620  62068     Index *pIdx,                    /* Index being probed */
 61621  62069     UnpackedRecord **ppRec,         /* IN/OUT: Probe record */
 61622  62070     Expr *pExpr,                    /* The expression to extract a value from */
 61623  62071     u8 affinity,                    /* Affinity to use */
 61624  62072     int iVal,                       /* Array element to populate */
 61625  62073     int *pbOk                       /* OUT: True if value was extracted */
 61626  62074   ){
 61627         -  int rc = SQLITE_OK;
        62075  +  int rc;
 61628  62076     sqlite3_value *pVal = 0;
 61629         -  sqlite3 *db = pParse->db;
 61630         -
 61631         -
 61632  62077     struct ValueNewStat4Ctx alloc;
        62078  +
 61633  62079     alloc.pParse = pParse;
 61634  62080     alloc.pIdx = pIdx;
 61635  62081     alloc.ppRec = ppRec;
 61636  62082     alloc.iVal = iVal;
 61637  62083   
 61638         -  /* Skip over any TK_COLLATE nodes */
 61639         -  pExpr = sqlite3ExprSkipCollate(pExpr);
 61640         -
 61641         -  if( !pExpr ){
 61642         -    pVal = valueNew(db, &alloc);
 61643         -    if( pVal ){
 61644         -      sqlite3VdbeMemSetNull((Mem*)pVal);
 61645         -    }
 61646         -  }else if( pExpr->op==TK_VARIABLE
 61647         -        || NEVER(pExpr->op==TK_REGISTER && pExpr->op2==TK_VARIABLE)
 61648         -  ){
 61649         -    Vdbe *v;
 61650         -    int iBindVar = pExpr->iColumn;
 61651         -    sqlite3VdbeSetVarmask(pParse->pVdbe, iBindVar);
 61652         -    if( (v = pParse->pReprepare)!=0 ){
 61653         -      pVal = valueNew(db, &alloc);
 61654         -      if( pVal ){
 61655         -        rc = sqlite3VdbeMemCopy((Mem*)pVal, &v->aVar[iBindVar-1]);
 61656         -        if( rc==SQLITE_OK ){
 61657         -          sqlite3ValueApplyAffinity(pVal, affinity, ENC(db));
 61658         -        }
 61659         -        pVal->db = pParse->db;
 61660         -      }
 61661         -    }
 61662         -  }else{
 61663         -    rc = valueFromExpr(db, pExpr, ENC(db), affinity, &pVal, &alloc);
 61664         -  }
        62084  +  rc = stat4ValueFromExpr(pParse, pExpr, affinity, &alloc, &pVal);
        62085  +  assert( pVal==0 || pVal->db==pParse->db );
 61665  62086     *pbOk = (pVal!=0);
        62087  +  return rc;
        62088  +}
 61666  62089   
 61667         -  assert( pVal==0 || pVal->db==db );
 61668         -  return rc;
        62090  +/*
        62091  +** Attempt to extract a value from expression pExpr using the methods
        62092  +** as described for sqlite3Stat4ProbeSetValue() above. 
        62093  +**
        62094  +** If successful, set *ppVal to point to a new value object and return 
        62095  +** SQLITE_OK. If no value can be extracted, but no other error occurs
        62096  +** (e.g. OOM), return SQLITE_OK and set *ppVal to NULL. Or, if an error
        62097  +** does occur, return an SQLite error code. The final value of *ppVal
        62098  +** is undefined in this case.
        62099  +*/
        62100  +SQLITE_PRIVATE int sqlite3Stat4ValueFromExpr(
        62101  +  Parse *pParse,                  /* Parse context */
        62102  +  Expr *pExpr,                    /* The expression to extract a value from */
        62103  +  u8 affinity,                    /* Affinity to use */
        62104  +  sqlite3_value **ppVal           /* OUT: New value object (or NULL) */
        62105  +){
        62106  +  return stat4ValueFromExpr(pParse, pExpr, affinity, 0, ppVal);
        62107  +}
        62108  +
        62109  +/*
        62110  +** Extract the iCol-th column from the nRec-byte record in pRec.  Write
        62111  +** the column value into *ppVal.  If *ppVal is initially NULL then a new
        62112  +** sqlite3_value object is allocated.
        62113  +**
        62114  +** If *ppVal is initially NULL then the caller is responsible for 
        62115  +** ensuring that the value written into *ppVal is eventually freed.
        62116  +*/
        62117  +SQLITE_PRIVATE int sqlite3Stat4Column(
        62118  +  sqlite3 *db,                    /* Database handle */
        62119  +  const void *pRec,               /* Pointer to buffer containing record */
        62120  +  int nRec,                       /* Size of buffer pRec in bytes */
        62121  +  int iCol,                       /* Column to extract */
        62122  +  sqlite3_value **ppVal           /* OUT: Extracted value */
        62123  +){
        62124  +  u32 t;                          /* a column type code */
        62125  +  int nHdr;                       /* Size of the header in the record */
        62126  +  int iHdr;                       /* Next unread header byte */
        62127  +  int iField;                     /* Next unread data byte */
        62128  +  int szField;                    /* Size of the current data field */
        62129  +  int i;                          /* Column index */
        62130  +  u8 *a = (u8*)pRec;              /* Typecast byte array */
        62131  +  Mem *pMem = *ppVal;             /* Write result into this Mem object */
        62132  +
        62133  +  assert( iCol>0 );
        62134  +  iHdr = getVarint32(a, nHdr);
        62135  +  if( nHdr>nRec || iHdr>=nHdr ) return SQLITE_CORRUPT_BKPT;
        62136  +  iField = nHdr;
        62137  +  for(i=0; i<=iCol; i++){
        62138  +    iHdr += getVarint32(&a[iHdr], t);
        62139  +    testcase( iHdr==nHdr );
        62140  +    testcase( iHdr==nHdr+1 );
        62141  +    if( iHdr>nHdr ) return SQLITE_CORRUPT_BKPT;
        62142  +    szField = sqlite3VdbeSerialTypeLen(t);
        62143  +    iField += szField;
        62144  +  }
        62145  +  testcase( iField==nRec );
        62146  +  testcase( iField==nRec+1 );
        62147  +  if( iField>nRec ) return SQLITE_CORRUPT_BKPT;
        62148  +  if( pMem==0 ){
        62149  +    pMem = *ppVal = sqlite3ValueNew(db);
        62150  +    if( pMem==0 ) return SQLITE_NOMEM;
        62151  +  }
        62152  +  sqlite3VdbeSerialGet(&a[iField-szField], t, pMem);
        62153  +  pMem->enc = ENC(db);
        62154  +  return SQLITE_OK;
 61669  62155   }
 61670  62156   
 61671  62157   /*
 61672  62158   ** Unless it is NULL, the argument must be an UnpackedRecord object returned
 61673  62159   ** by an earlier call to sqlite3Stat4ProbeSetValue(). This call deletes
 61674  62160   ** the object.
 61675  62161   */
................................................................................
 61808  62294     zTmp = pA->zSql;
 61809  62295     pA->zSql = pB->zSql;
 61810  62296     pB->zSql = zTmp;
 61811  62297     pB->isPrepareV2 = pA->isPrepareV2;
 61812  62298   }
 61813  62299   
 61814  62300   /*
 61815         -** Resize the Vdbe.aOp array so that it is at least one op larger than 
 61816         -** it was.
        62301  +** Resize the Vdbe.aOp array so that it is at least nOp elements larger 
        62302  +** than its current size. nOp is guaranteed to be less than or equal
        62303  +** to 1024/sizeof(Op).
 61817  62304   **
 61818  62305   ** If an out-of-memory error occurs while resizing the array, return
 61819         -** SQLITE_NOMEM. In this case Vdbe.aOp and Vdbe.nOpAlloc remain 
        62306  +** SQLITE_NOMEM. In this case Vdbe.aOp and Parse.nOpAlloc remain 
 61820  62307   ** unchanged (this is so that any opcodes already allocated can be 
 61821  62308   ** correctly deallocated along with the rest of the Vdbe).
 61822  62309   */
 61823         -static int growOpArray(Vdbe *v){
        62310  +static int growOpArray(Vdbe *v, int nOp){
 61824  62311     VdbeOp *pNew;
 61825  62312     Parse *p = v->pParse;
        62313  +
        62314  +  /* The SQLITE_TEST_REALLOC_STRESS compile-time option is designed to force
        62315  +  ** more frequent reallocs and hence provide more opportunities for 
        62316  +  ** simulated OOM faults.  SQLITE_TEST_REALLOC_STRESS is generally used
        62317  +  ** during testing only.  With SQLITE_TEST_REALLOC_STRESS grow the op array
        62318  +  ** by the minimum* amount required until the size reaches 512.  Normal
        62319  +  ** operation (without SQLITE_TEST_REALLOC_STRESS) is to double the current
        62320  +  ** size of the op array or add 1KB of space, whichever is smaller. */
        62321  +#ifdef SQLITE_TEST_REALLOC_STRESS
        62322  +  int nNew = (p->nOpAlloc>=512 ? p->nOpAlloc*2 : p->nOpAlloc+nOp);
        62323  +#else
 61826  62324     int nNew = (p->nOpAlloc ? p->nOpAlloc*2 : (int)(1024/sizeof(Op)));
        62325  +  UNUSED_PARAMETER(nOp);
        62326  +#endif
        62327  +
        62328  +  assert( nOp<=(1024/sizeof(Op)) );
        62329  +  assert( nNew>=(p->nOpAlloc+nOp) );
 61827  62330     pNew = sqlite3DbRealloc(p->db, v->aOp, nNew*sizeof(Op));
 61828  62331     if( pNew ){
 61829  62332       p->nOpAlloc = sqlite3DbMallocSize(p->db, pNew)/sizeof(Op);
 61830  62333       v->aOp = pNew;
 61831  62334     }
 61832  62335     return (pNew ? SQLITE_OK : SQLITE_NOMEM);
 61833  62336   }
................................................................................
 61863  62366     int i;
 61864  62367     VdbeOp *pOp;
 61865  62368   
 61866  62369     i = p->nOp;
 61867  62370     assert( p->magic==VDBE_MAGIC_INIT );
 61868  62371     assert( op>0 && op<0xff );
 61869  62372     if( p->pParse->nOpAlloc<=i ){
 61870         -    if( growOpArray(p) ){
        62373  +    if( growOpArray(p, 1) ){
 61871  62374         return 1;
 61872  62375       }
 61873  62376     }
 61874  62377     p->nOp++;
 61875  62378     pOp = &p->aOp[i];
 61876  62379     pOp->opcode = (u8)op;
 61877  62380     pOp->p5 = 0;
................................................................................
 62223  62726         pOp->p2 = aLabel[-1-pOp->p2];
 62224  62727       }
 62225  62728     }
 62226  62729     sqlite3DbFree(p->db, pParse->aLabel);
 62227  62730     pParse->aLabel = 0;
 62228  62731     pParse->nLabel = 0;
 62229  62732     *pMaxFuncArgs = nMaxArgs;
 62230         -  assert( p->bIsReader!=0 || p->btreeMask==0 );
        62733  +  assert( p->bIsReader!=0 || DbMaskAllZero(p->btreeMask) );
 62231  62734   }
 62232  62735   
 62233  62736   /*
 62234  62737   ** Return the address of the next instruction to be inserted.
 62235  62738   */
 62236  62739   SQLITE_PRIVATE int sqlite3VdbeCurrentAddr(Vdbe *p){
 62237  62740     assert( p->magic==VDBE_MAGIC_INIT );
................................................................................
 62250  62753   ** returned program.
 62251  62754   */
 62252  62755   SQLITE_PRIVATE VdbeOp *sqlite3VdbeTakeOpArray(Vdbe *p, int *pnOp, int *pnMaxArg){
 62253  62756     VdbeOp *aOp = p->aOp;
 62254  62757     assert( aOp && !p->db->mallocFailed );
 62255  62758   
 62256  62759     /* Check that sqlite3VdbeUsesBtree() was not called on this VM */
 62257         -  assert( p->btreeMask==0 );
        62760  +  assert( DbMaskAllZero(p->btreeMask) );
 62258  62761   
 62259  62762     resolveP2Values(p, pnMaxArg);
 62260  62763     *pnOp = p->nOp;
 62261  62764     p->aOp = 0;
 62262  62765     return aOp;
 62263  62766   }
 62264  62767   
................................................................................
 62265  62768   /*
 62266  62769   ** Add a whole list of operations to the operation stack.  Return the
 62267  62770   ** address of the first operation added.
 62268  62771   */
 62269  62772   SQLITE_PRIVATE int sqlite3VdbeAddOpList(Vdbe *p, int nOp, VdbeOpList const *aOp, int iLineno){
 62270  62773     int addr;
 62271  62774     assert( p->magic==VDBE_MAGIC_INIT );
 62272         -  if( p->nOp + nOp > p->pParse->nOpAlloc && growOpArray(p) ){
        62775  +  if( p->nOp + nOp > p->pParse->nOpAlloc && growOpArray(p, nOp) ){
 62273  62776       return 0;
 62274  62777     }
 62275  62778     addr = p->nOp;
 62276  62779     if( ALWAYS(nOp>0) ){
 62277  62780       int i;
 62278  62781       VdbeOpList const *pIn = aOp;
 62279  62782       for(i=0; i<nOp; i++, pIn++){
................................................................................
 62450  62953     pVdbe->pProgram = p;
 62451  62954   }
 62452  62955   
 62453  62956   /*
 62454  62957   ** Change the opcode at addr into OP_Noop
 62455  62958   */
 62456  62959   SQLITE_PRIVATE void sqlite3VdbeChangeToNoop(Vdbe *p, int addr){
 62457         -  if( p->aOp ){
        62960  +  if( addr<p->nOp ){
 62458  62961       VdbeOp *pOp = &p->aOp[addr];
 62459  62962       sqlite3 *db = p->db;
 62460  62963       freeP4(db, pOp->p4type, pOp->p4.p);
 62461  62964       memset(pOp, 0, sizeof(pOp[0]));
 62462  62965       pOp->opcode = OP_Noop;
 62463  62966       if( addr==p->nOp-1 ) p->nOp--;
 62464  62967     }
................................................................................
 62835  63338   ** attached databases that will be use.  A mask of these databases
 62836  63339   ** is maintained in p->btreeMask.  The p->lockMask value is the subset of
 62837  63340   ** p->btreeMask of databases that will require a lock.
 62838  63341   */
 62839  63342   SQLITE_PRIVATE void sqlite3VdbeUsesBtree(Vdbe *p, int i){
 62840  63343     assert( i>=0 && i<p->db->nDb && i<(int)sizeof(yDbMask)*8 );
 62841  63344     assert( i<(int)sizeof(p->btreeMask)*8 );
 62842         -  p->btreeMask |= ((yDbMask)1)<<i;
        63345  +  DbMaskSet(p->btreeMask, i);
 62843  63346     if( i!=1 && sqlite3BtreeSharable(p->db->aDb[i].pBt) ){
 62844         -    p->lockMask |= ((yDbMask)1)<<i;
        63347  +    DbMaskSet(p->lockMask, i);
 62845  63348     }
 62846  63349   }
 62847  63350   
 62848  63351   #if !defined(SQLITE_OMIT_SHARED_CACHE) && SQLITE_THREADSAFE>0
 62849  63352   /*
 62850  63353   ** If SQLite is compiled to support shared-cache mode and to be threadsafe,
 62851  63354   ** this routine obtains the mutex associated with each BtShared structure
................................................................................
 62865  63368   ** statement p will ever use.  Let N be the number of bits in p->btreeMask
 62866  63369   ** corresponding to btrees that use shared cache.  Then the runtime of
 62867  63370   ** this routine is N*N.  But as N is rarely more than 1, this should not
 62868  63371   ** be a problem.
 62869  63372   */
 62870  63373   SQLITE_PRIVATE void sqlite3VdbeEnter(Vdbe *p){
 62871  63374     int i;
 62872         -  yDbMask mask;
 62873  63375     sqlite3 *db;
 62874  63376     Db *aDb;
 62875  63377     int nDb;
 62876         -  if( p->lockMask==0 ) return;  /* The common case */
        63378  +  if( DbMaskAllZero(p->lockMask) ) return;  /* The common case */
 62877  63379     db = p->db;
 62878  63380     aDb = db->aDb;
 62879  63381     nDb = db->nDb;
 62880         -  for(i=0, mask=1; i<nDb; i++, mask += mask){
 62881         -    if( i!=1 && (mask & p->lockMask)!=0 && ALWAYS(aDb[i].pBt!=0) ){
        63382  +  for(i=0; i<nDb; i++){
        63383  +    if( i!=1 && DbMaskTest(p->lockMask,i) && ALWAYS(aDb[i].pBt!=0) ){
 62882  63384         sqlite3BtreeEnter(aDb[i].pBt);
 62883  63385       }
 62884  63386     }
 62885  63387   }
 62886  63388   #endif
 62887  63389   
 62888  63390   #if !defined(SQLITE_OMIT_SHARED_CACHE) && SQLITE_THREADSAFE>0
 62889  63391   /*
 62890  63392   ** Unlock all of the btrees previously locked by a call to sqlite3VdbeEnter().
 62891  63393   */
 62892  63394   SQLITE_PRIVATE void sqlite3VdbeLeave(Vdbe *p){
 62893  63395     int i;
 62894         -  yDbMask mask;
 62895  63396     sqlite3 *db;
 62896  63397     Db *aDb;
 62897  63398     int nDb;
 62898         -  if( p->lockMask==0 ) return;  /* The common case */
        63399  +  if( DbMaskAllZero(p->lockMask) ) return;  /* The common case */
 62899  63400     db = p->db;
 62900  63401     aDb = db->aDb;
 62901  63402     nDb = db->nDb;
 62902         -  for(i=0, mask=1; i<nDb; i++, mask += mask){
 62903         -    if( i!=1 && (mask & p->lockMask)!=0 && ALWAYS(aDb[i].pBt!=0) ){
        63403  +  for(i=0; i<nDb; i++){
        63404  +    if( i!=1 && DbMaskTest(p->lockMask,i) && ALWAYS(aDb[i].pBt!=0) ){
 62904  63405         sqlite3BtreeLeave(aDb[i].pBt);
 62905  63406       }
 62906  63407     }
 62907  63408   }
 62908  63409   #endif
 62909  63410   
 62910  63411   #if defined(VDBE_PROFILE) || defined(SQLITE_DEBUG)
................................................................................
 63867  64368   static void checkActiveVdbeCnt(sqlite3 *db){
 63868  64369     Vdbe *p;
 63869  64370     int cnt = 0;
 63870  64371     int nWrite = 0;
 63871  64372     int nRead = 0;
 63872  64373     p = db->pVdbe;
 63873  64374     while( p ){
 63874         -    if( p->magic==VDBE_MAGIC_RUN && p->pc>=0 ){
        64375  +    if( sqlite3_stmt_busy((sqlite3_stmt*)p) ){
 63875  64376         cnt++;
 63876  64377         if( p->readOnly==0 ) nWrite++;
 63877  64378         if( p->bIsReader ) nRead++;
 63878  64379       }
 63879  64380       p = p->pNext;
 63880  64381     }
 63881  64382     assert( cnt==db->nVdbeActive );
................................................................................
 64027  64528       int isSpecialError;            /* Set to true if a 'special' error */
 64028  64529   
 64029  64530       /* Lock all btrees used by the statement */
 64030  64531       sqlite3VdbeEnter(p);
 64031  64532   
 64032  64533       /* Check for one of the special errors */
 64033  64534       mrc = p->rc & 0xff;
 64034         -    assert( p->rc!=SQLITE_IOERR_BLOCKED );  /* This error no longer exists */
 64035  64535       isSpecialError = mrc==SQLITE_NOMEM || mrc==SQLITE_IOERR
 64036  64536                        || mrc==SQLITE_INTERRUPT || mrc==SQLITE_FULL;
 64037  64537       if( isSpecialError ){
 64038  64538         /* If the query was read-only and the error code is SQLITE_INTERRUPT, 
 64039  64539         ** no rollback is necessary. Otherwise, at least a savepoint 
 64040  64540         ** transaction must be rolled back to restore the database to a 
 64041  64541         ** consistent state.
................................................................................
 64512  65012   */
 64513  65013   
 64514  65014   /*
 64515  65015   ** Return the serial-type for the value stored in pMem.
 64516  65016   */
 64517  65017   SQLITE_PRIVATE u32 sqlite3VdbeSerialType(Mem *pMem, int file_format){
 64518  65018     int flags = pMem->flags;
 64519         -  int n;
        65019  +  u32 n;
 64520  65020   
 64521  65021     if( flags&MEM_Null ){
 64522  65022       return 0;
 64523  65023     }
 64524  65024     if( flags&MEM_Int ){
 64525  65025       /* Figure out whether to use 1, 2, 4, 6 or 8 bytes. */
 64526  65026   #   define MAX_6BYTE ((((i64)0x00008000)<<32)-1)
................................................................................
 64542  65042       if( u<=MAX_6BYTE ) return 5;
 64543  65043       return 6;
 64544  65044     }
 64545  65045     if( flags&MEM_Real ){
 64546  65046       return 7;
 64547  65047     }
 64548  65048     assert( pMem->db->mallocFailed || flags&(MEM_Str|MEM_Blob) );
 64549         -  n = pMem->n;
        65049  +  assert( pMem->n>=0 );
        65050  +  n = (u32)pMem->n;
 64550  65051     if( flags & MEM_Zero ){
 64551  65052       n += pMem->u.nZero;
 64552  65053     }
 64553         -  assert( n>=0 );
 64554  65054     return ((n*2) + 12 + ((flags&MEM_Str)!=0));
 64555  65055   }
 64556  65056   
 64557  65057   /*
 64558  65058   ** Return the length of the data corresponding to the supplied serial-type.
 64559  65059   */
 64560  65060   SQLITE_PRIVATE u32 sqlite3VdbeSerialTypeLen(u32 serial_type){
................................................................................
 65313  65813     assert( mem1.zMalloc==0 );
 65314  65814   
 65315  65815     /* rc==0 here means that one or both of the keys ran out of fields and
 65316  65816     ** all the fields up to that point were equal. Return the the default_rc
 65317  65817     ** value.  */
 65318  65818     assert( CORRUPT_DB 
 65319  65819          || pPKey2->default_rc==vdbeRecordCompareDebug(nKey1, pKey1, pPKey2) 
        65820  +       || pKeyInfo->db->mallocFailed
 65320  65821     );
 65321  65822     return pPKey2->default_rc;
 65322  65823   }
 65323  65824   
 65324  65825   /*
 65325  65826   ** This function is an optimized version of sqlite3VdbeRecordCompare() 
 65326  65827   ** that (a) the first field of pPKey2 is an integer, and (b) the 
................................................................................
 65478  65979       }
 65479  65980     }
 65480  65981   
 65481  65982     assert( (res==0 && vdbeRecordCompareDebug(nKey1, pKey1, pPKey2)==0)
 65482  65983          || (res<0 && vdbeRecordCompareDebug(nKey1, pKey1, pPKey2)<0)
 65483  65984          || (res>0 && vdbeRecordCompareDebug(nKey1, pKey1, pPKey2)>0)
 65484  65985          || CORRUPT_DB
        65986  +       || pPKey2->pKeyInfo->db->mallocFailed
 65485  65987     );
 65486  65988     return res;
 65487  65989   }
 65488  65990   
 65489  65991   /*
 65490  65992   ** Return a pointer to an sqlite3VdbeRecordCompare() compatible function
 65491  65993   ** suitable for comparing serialized records to the unpacked record passed
................................................................................
 67062  67564   }
 67063  67565   
 67064  67566   /*
 67065  67567   ** Return true if the prepared statement is in need of being reset.
 67066  67568   */
 67067  67569   SQLITE_API int sqlite3_stmt_busy(sqlite3_stmt *pStmt){
 67068  67570     Vdbe *v = (Vdbe*)pStmt;
 67069         -  return v!=0 && v->pc>0 && v->magic==VDBE_MAGIC_RUN;
        67571  +  return v!=0 && v->pc>=0 && v->magic==VDBE_MAGIC_RUN;
 67070  67572   }
 67071  67573   
 67072  67574   /*
 67073  67575   ** Return a pointer to the next prepared statement after pStmt associated
 67074  67576   ** with database connection pDb.  If pStmt is NULL, return the first
 67075  67577   ** prepared statement for the database connection.  Return NULL if there
 67076  67578   ** are no more.
................................................................................
 67512  68014   ** feature is used for test suite validation only and does not appear an
 67513  68015   ** production builds.
 67514  68016   **
 67515  68017   ** M is an integer, 2 or 3, that indices how many different ways the
 67516  68018   ** branch can go.  It is usually 2.  "I" is the direction the branch
 67517  68019   ** goes.  0 means falls through.  1 means branch is taken.  2 means the
 67518  68020   ** second alternative branch is taken.
        68021  +**
        68022  +** iSrcLine is the source code line (from the __LINE__ macro) that
        68023  +** generated the VDBE instruction.  This instrumentation assumes that all
        68024  +** source code is in a single file (the amalgamation).  Special values 1
        68025  +** and 2 for the iSrcLine parameter mean that this particular branch is
        68026  +** always taken or never taken, respectively.
 67519  68027   */
 67520  68028   #if !defined(SQLITE_VDBE_COVERAGE)
 67521  68029   # define VdbeBranchTaken(I,M)
 67522  68030   #else
 67523  68031   # define VdbeBranchTaken(I,M) vdbeTakeBranch(pOp->iSrcLine,I,M)
 67524  68032     static void vdbeTakeBranch(int iSrcLine, u8 I, u8 M){
 67525  68033       if( iSrcLine<=2 && ALWAYS(iSrcLine>0) ){
................................................................................
 67620  68128   /*
 67621  68129   ** Try to convert a value into a numeric representation if we can
 67622  68130   ** do so without loss of information.  In other words, if the string
 67623  68131   ** looks like a number, convert it into a number.  If it does not
 67624  68132   ** look like a number, leave it alone.
 67625  68133   */
 67626  68134   static void applyNumericAffinity(Mem *pRec){
 67627         -  if( (pRec->flags & (MEM_Real|MEM_Int))==0 ){
 67628         -    double rValue;
 67629         -    i64 iValue;
 67630         -    u8 enc = pRec->enc;
 67631         -    if( (pRec->flags&MEM_Str)==0 ) return;
 67632         -    if( sqlite3AtoF(pRec->z, &rValue, pRec->n, enc)==0 ) return;
 67633         -    if( 0==sqlite3Atoi64(pRec->z, &iValue, pRec->n, enc) ){
 67634         -      pRec->u.i = iValue;
 67635         -      pRec->flags |= MEM_Int;
 67636         -    }else{
 67637         -      pRec->r = rValue;
 67638         -      pRec->flags |= MEM_Real;
 67639         -    }
        68135  +  double rValue;
        68136  +  i64 iValue;
        68137  +  u8 enc = pRec->enc;
        68138  +  if( (pRec->flags&MEM_Str)==0 ) return;
        68139  +  if( sqlite3AtoF(pRec->z, &rValue, pRec->n, enc)==0 ) return;
        68140  +  if( 0==sqlite3Atoi64(pRec->z, &iValue, pRec->n, enc) ){
        68141  +    pRec->u.i = iValue;
        68142  +    pRec->flags |= MEM_Int;
        68143  +  }else{
        68144  +    pRec->r = rValue;
        68145  +    pRec->flags |= MEM_Real;
 67640  68146     }
 67641  68147   }
        68148  +#define ApplyNumericAffinity(X)  \
        68149  +   if(((X)->flags&(MEM_Real|MEM_Int))==0){applyNumericAffinity(X);}
 67642  68150   
 67643  68151   /*
 67644  68152   ** Processing is determine by the affinity parameter:
 67645  68153   **
 67646  68154   ** SQLITE_AFF_INTEGER:
 67647  68155   ** SQLITE_AFF_REAL:
 67648  68156   ** SQLITE_AFF_NUMERIC:
................................................................................
 67671  68179       if( 0==(pRec->flags&MEM_Str) && (pRec->flags&(MEM_Real|MEM_Int)) ){
 67672  68180         sqlite3VdbeMemStringify(pRec, enc);
 67673  68181       }
 67674  68182       pRec->flags &= ~(MEM_Real|MEM_Int);
 67675  68183     }else if( affinity!=SQLITE_AFF_NONE ){
 67676  68184       assert( affinity==SQLITE_AFF_INTEGER || affinity==SQLITE_AFF_REAL
 67677  68185                || affinity==SQLITE_AFF_NUMERIC );
 67678         -    applyNumericAffinity(pRec);
        68186  +    ApplyNumericAffinity(pRec);
 67679  68187       if( pRec->flags & MEM_Real ){
 67680  68188         sqlite3VdbeIntegerAffinity(pRec);
 67681  68189       }
 67682  68190     }
 67683  68191   }
 67684  68192   
 67685  68193   /*
................................................................................
 68252  68760     pc = (int)pIn1->u.i;
 68253  68761     pIn1->flags = MEM_Undefined;
 68254  68762     break;
 68255  68763   }
 68256  68764   
 68257  68765   /* Opcode: InitCoroutine P1 P2 P3 * *
 68258  68766   **
 68259         -** Set up register P1 so that it will OP_Yield to the co-routine
        68767  +** Set up register P1 so that it will Yield to the coroutine
 68260  68768   ** located at address P3.
 68261  68769   **
 68262         -** If P2!=0 then the co-routine implementation immediately follows
 68263         -** this opcode.  So jump over the co-routine implementation to
        68770  +** If P2!=0 then the coroutine implementation immediately follows
        68771  +** this opcode.  So jump over the coroutine implementation to
 68264  68772   ** address P2.
        68773  +**
        68774  +** See also: EndCoroutine
 68265  68775   */
 68266  68776   case OP_InitCoroutine: {     /* jump */
 68267  68777     assert( pOp->p1>0 &&  pOp->p1<=(p->nMem-p->nCursor) );
 68268  68778     assert( pOp->p2>=0 && pOp->p2<p->nOp );
 68269  68779     assert( pOp->p3>=0 && pOp->p3<p->nOp );
 68270  68780     pOut = &aMem[pOp->p1];
 68271  68781     assert( !VdbeMemDynamic(pOut) );
................................................................................
 68273  68783     pOut->flags = MEM_Int;
 68274  68784     if( pOp->p2 ) pc = pOp->p2 - 1;
 68275  68785     break;
 68276  68786   }
 68277  68787   
 68278  68788   /* Opcode:  EndCoroutine P1 * * * *
 68279  68789   **
 68280         -** The instruction at the address in register P1 is an OP_Yield.
 68281         -** Jump to the P2 parameter of that OP_Yield.
        68790  +** The instruction at the address in register P1 is a Yield.
        68791  +** Jump to the P2 parameter of that Yield.
 68282  68792   ** After the jump, register P1 becomes undefined.
        68793  +**
        68794  +** See also: InitCoroutine
 68283  68795   */
 68284  68796   case OP_EndCoroutine: {           /* in1 */
 68285  68797     VdbeOp *pCaller;
 68286  68798     pIn1 = &aMem[pOp->p1];
 68287  68799     assert( pIn1->flags==MEM_Int );
 68288  68800     assert( pIn1->u.i>=0 && pIn1->u.i<p->nOp );
 68289  68801     pCaller = &aOp[pIn1->u.i];
................................................................................
 68292  68804     pc = pCaller->p2 - 1;
 68293  68805     pIn1->flags = MEM_Undefined;
 68294  68806     break;
 68295  68807   }
 68296  68808   
 68297  68809   /* Opcode:  Yield P1 P2 * * *
 68298  68810   **
 68299         -** Swap the program counter with the value in register P1.
        68811  +** Swap the program counter with the value in register P1.  This
        68812  +** has the effect of yielding to a coroutine.
 68300  68813   **
 68301         -** If the co-routine ends with OP_Yield or OP_Return then continue
 68302         -** to the next instruction.  But if the co-routine ends with
 68303         -** OP_EndCoroutine, jump immediately to P2.
        68814  +** If the coroutine that is launched by this instruction ends with
        68815  +** Yield or Return then continue to the next instruction.  But if
        68816  +** the coroutine launched by this instruction ends with
        68817  +** EndCoroutine, then jump to P2 rather than continuing with the
        68818  +** next instruction.
        68819  +**
        68820  +** See also: InitCoroutine
 68304  68821   */
 68305  68822   case OP_Yield: {            /* in1, jump */
 68306  68823     int pcDest;
 68307  68824     pIn1 = &aMem[pOp->p1];
 68308  68825     assert( VdbeMemDynamic(pIn1)==0 );
 68309  68826     pIn1->flags = MEM_Int;
 68310  68827     pcDest = (int)pIn1->u.i;
................................................................................
 68459  68976   }
 68460  68977   #endif
 68461  68978   
 68462  68979   /* Opcode: String8 * P2 * P4 *
 68463  68980   ** Synopsis: r[P2]='P4'
 68464  68981   **
 68465  68982   ** P4 points to a nul terminated UTF-8 string. This opcode is transformed 
 68466         -** into an OP_String before it is executed for the first time.  During
        68983  +** into a String before it is executed for the first time.  During
 68467  68984   ** this transformation, the length of string P4 is computed and stored
 68468  68985   ** as the P1 parameter.
 68469  68986   */
 68470  68987   case OP_String8: {         /* same as TK_STRING, out2-prerelease */
 68471  68988     assert( pOp->p4.z!=0 );
 68472  68989     pOp->opcode = OP_String;
 68473  68990     pOp->p1 = sqlite3Strlen30(pOp->p4.z);
................................................................................
 69681  70198       sqlite3VdbeMemSetInt64(pOut, ~sqlite3VdbeIntValue(pIn1));
 69682  70199     }
 69683  70200     break;
 69684  70201   }
 69685  70202   
 69686  70203   /* Opcode: Once P1 P2 * * *
 69687  70204   **
 69688         -** Check if OP_Once flag P1 is set. If so, jump to instruction P2. Otherwise,
 69689         -** set the flag and fall through to the next instruction.  In other words,
 69690         -** this opcode causes all following opcodes up through P2 (but not including
 69691         -** P2) to run just once and to be skipped on subsequent times through the loop.
        70205  +** Check the "once" flag number P1. If it is set, jump to instruction P2. 
        70206  +** Otherwise, set the flag and fall through to the next instruction.
        70207  +** In other words, this opcode causes all following opcodes up through P2
        70208  +** (but not including P2) to run just once and to be skipped on subsequent
        70209  +** times through the loop.
        70210  +**
        70211  +** All "once" flags are initially cleared whenever a prepared statement
        70212  +** first begins to run.
 69692  70213   */
 69693  70214   case OP_Once: {             /* jump */
 69694  70215     assert( pOp->p1<p->nOnceFlag );
 69695  70216     VdbeBranchTaken(p->aOnceFlag[pOp->p1]!=0, 2);
 69696  70217     if( p->aOnceFlag[pOp->p1] ){
 69697  70218       pc = pOp->p2-1;
 69698  70219     }else{
................................................................................
 69701  70222     break;
 69702  70223   }
 69703  70224   
 69704  70225   /* Opcode: If P1 P2 P3 * *
 69705  70226   **
 69706  70227   ** Jump to P2 if the value in register P1 is true.  The value
 69707  70228   ** is considered true if it is numeric and non-zero.  If the value
 69708         -** in P1 is NULL then take the jump if P3 is non-zero.
        70229  +** in P1 is NULL then take the jump if and only if P3 is non-zero.
 69709  70230   */
 69710  70231   /* Opcode: IfNot P1 P2 P3 * *
 69711  70232   **
 69712  70233   ** Jump to P2 if the value in register P1 is False.  The value
 69713  70234   ** is considered false if it has a numeric value of zero.  If the value
 69714         -** in P1 is NULL then take the jump if P3 is zero.
        70235  +** in P1 is NULL then take the jump if and only if P3 is non-zero.
 69715  70236   */
 69716  70237   case OP_If:                 /* jump, in1 */
 69717  70238   case OP_IfNot: {            /* jump, in1 */
 69718  70239     int c;
 69719  70240     pIn1 = &aMem[pOp->p1];
 69720  70241     if( pIn1->flags & MEM_Null ){
 69721  70242       c = pOp->p3;
................................................................................
 70519  71040     Btree *pBt;
 70520  71041     int iMeta;
 70521  71042     int iGen;
 70522  71043   
 70523  71044     assert( p->bIsReader );
 70524  71045     assert( p->readOnly==0 || pOp->p2==0 );
 70525  71046     assert( pOp->p1>=0 && pOp->p1<db->nDb );
 70526         -  assert( (p->btreeMask & (((yDbMask)1)<<pOp->p1))!=0 );
        71047  +  assert( DbMaskTest(p->btreeMask, pOp->p1) );
 70527  71048     if( pOp->p2 && (db->flags & SQLITE_QueryOnly)!=0 ){
 70528  71049       rc = SQLITE_READONLY;
 70529  71050       goto abort_due_to_error;
 70530  71051     }
 70531  71052     pBt = db->aDb[pOp->p1].pBt;
 70532  71053   
 70533  71054     if( pBt ){
................................................................................
 70614  71135   
 70615  71136     assert( p->bIsReader );
 70616  71137     iDb = pOp->p1;
 70617  71138     iCookie = pOp->p3;
 70618  71139     assert( pOp->p3<SQLITE_N_BTREE_META );
 70619  71140     assert( iDb>=0 && iDb<db->nDb );
 70620  71141     assert( db->aDb[iDb].pBt!=0 );
 70621         -  assert( (p->btreeMask & (((yDbMask)1)<<iDb))!=0 );
        71142  +  assert( DbMaskTest(p->btreeMask, iDb) );
 70622  71143   
 70623  71144     sqlite3BtreeGetMeta(db->aDb[iDb].pBt, iCookie, (u32 *)&iMeta);
 70624  71145     pOut->u.i = iMeta;
 70625  71146     break;
 70626  71147   }
 70627  71148   
 70628  71149   /* Opcode: SetCookie P1 P2 P3 * *
................................................................................
 70635  71156   **
 70636  71157   ** A transaction must be started before executing this opcode.
 70637  71158   */
 70638  71159   case OP_SetCookie: {       /* in3 */
 70639  71160     Db *pDb;
 70640  71161     assert( pOp->p2<SQLITE_N_BTREE_META );
 70641  71162     assert( pOp->p1>=0 && pOp->p1<db->nDb );
 70642         -  assert( (p->btreeMask & (((yDbMask)1)<<pOp->p1))!=0 );
        71163  +  assert( DbMaskTest(p->btreeMask, pOp->p1) );
 70643  71164     assert( p->readOnly==0 );
 70644  71165     pDb = &db->aDb[pOp->p1];
 70645  71166     assert( pDb->pBt!=0 );
 70646  71167     assert( sqlite3SchemaMutexHeld(db, pOp->p1, 0) );
 70647  71168     pIn3 = &aMem[pOp->p3];
 70648  71169     sqlite3VdbeMemIntegerify(pIn3);
 70649  71170     /* See note about index shifting on OP_ReadCookie */
................................................................................
 70690  71211   **
 70691  71212   ** The P4 value may be either an integer (P4_INT32) or a pointer to
 70692  71213   ** a KeyInfo structure (P4_KEYINFO). If it is a pointer to a KeyInfo 
 70693  71214   ** structure, then said structure defines the content and collating 
 70694  71215   ** sequence of the index being opened. Otherwise, if P4 is an integer 
 70695  71216   ** value, it is set to the number of columns in the table.
 70696  71217   **
 70697         -** See also OpenWrite.
        71218  +** See also: OpenWrite, ReopenIdx
        71219  +*/
        71220  +/* Opcode: ReopenIdx P1 P2 P3 P4 P5
        71221  +** Synopsis: root=P2 iDb=P3
        71222  +**
        71223  +** The ReopenIdx opcode works exactly like ReadOpen except that it first
        71224  +** checks to see if the cursor on P1 is already open with a root page
        71225  +** number of P2 and if it is this opcode becomes a no-op.  In other words,
        71226  +** if the cursor is already open, do not reopen it.
        71227  +**
        71228  +** The ReopenIdx opcode may only be used with P5==0 and with P4 being
        71229  +** a P4_KEYINFO object.  Furthermore, the P3 value must be the same as
        71230  +** every other ReopenIdx or OpenRead for the same cursor number.
        71231  +**
        71232  +** See the OpenRead opcode documentation for additional information.
 70698  71233   */
 70699  71234   /* Opcode: OpenWrite P1 P2 P3 P4 P5
 70700  71235   ** Synopsis: root=P2 iDb=P3
 70701  71236   **
 70702  71237   ** Open a read/write cursor named P1 on the table or index whose root
 70703  71238   ** page is P2.  Or if P5!=0 use the content of register P2 to find the
 70704  71239   ** root page.
................................................................................
 70712  71247   **
 70713  71248   ** This instruction works just like OpenRead except that it opens the cursor
 70714  71249   ** in read/write mode.  For a given table, there can be one or more read-only
 70715  71250   ** cursors or a single read/write cursor but not both.
 70716  71251   **
 70717  71252   ** See also OpenRead.
 70718  71253   */
        71254  +case OP_ReopenIdx: {
        71255  +  VdbeCursor *pCur;
        71256  +
        71257  +  assert( pOp->p5==0 );
        71258  +  assert( pOp->p4type==P4_KEYINFO );
        71259  +  pCur = p->apCsr[pOp->p1];
        71260  +  if( pCur && pCur->pgnoRoot==(u32)pOp->p2 ){
        71261  +    assert( pCur->iDb==pOp->p3 );      /* Guaranteed by the code generator */
        71262  +    break;
        71263  +  }
        71264  +  /* If the cursor is not currently open or is open on a different
        71265  +  ** index, then fall through into OP_OpenRead to force a reopen */
        71266  +}
 70719  71267   case OP_OpenRead:
 70720  71268   case OP_OpenWrite: {
 70721  71269     int nField;
 70722  71270     KeyInfo *pKeyInfo;
 70723  71271     int p2;
 70724  71272     int iDb;
 70725  71273     int wrFlag;
................................................................................
 70726  71274     Btree *pX;
 70727  71275     VdbeCursor *pCur;
 70728  71276     Db *pDb;
 70729  71277   
 70730  71278     assert( (pOp->p5&(OPFLAG_P2ISREG|OPFLAG_BULKCSR))==pOp->p5 );
 70731  71279     assert( pOp->opcode==OP_OpenWrite || pOp->p5==0 );
 70732  71280     assert( p->bIsReader );
 70733         -  assert( pOp->opcode==OP_OpenRead || p->readOnly==0 );
        71281  +  assert( pOp->opcode==OP_OpenRead || pOp->opcode==OP_ReopenIdx
        71282  +          || p->readOnly==0 );
 70734  71283   
 70735  71284     if( p->expired ){
 70736  71285       rc = SQLITE_ABORT;
 70737  71286       break;
 70738  71287     }
 70739  71288   
 70740  71289     nField = 0;
 70741  71290     pKeyInfo = 0;
 70742  71291     p2 = pOp->p2;
 70743  71292     iDb = pOp->p3;
 70744  71293     assert( iDb>=0 && iDb<db->nDb );
 70745         -  assert( (p->btreeMask & (((yDbMask)1)<<iDb))!=0 );
        71294  +  assert( DbMaskTest(p->btreeMask, iDb) );
 70746  71295     pDb = &db->aDb[iDb];
 70747  71296     pX = pDb->pBt;
 70748  71297     assert( pX!=0 );
 70749  71298     if( pOp->opcode==OP_OpenWrite ){
 70750  71299       wrFlag = 1;
 70751  71300       assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
 70752  71301       if( pDb->pSchema->file_format < p->minWriteFileFormat ){
................................................................................
 70783  71332     assert( pOp->p1>=0 );
 70784  71333     assert( nField>=0 );
 70785  71334     testcase( nField==0 );  /* Table with INTEGER PRIMARY KEY and nothing else */
 70786  71335     pCur = allocateCursor(p, pOp->p1, nField, iDb, 1);
 70787  71336     if( pCur==0 ) goto no_mem;
 70788  71337     pCur->nullRow = 1;
 70789  71338     pCur->isOrdered = 1;
        71339  +  pCur->pgnoRoot = p2;
 70790  71340     rc = sqlite3BtreeCursor(pX, p2, wrFlag, pKeyInfo, pCur->pCursor);
 70791  71341     pCur->pKeyInfo = pKeyInfo;
 70792  71342     assert( OPFLAG_BULKCSR==BTREE_BULKLOAD );
 70793  71343     sqlite3BtreeCursorHints(pCur->pCursor, (pOp->p5 & OPFLAG_BULKCSR));
 70794  71344   
 70795  71345     /* Since it performs no memory allocation or IO, the only value that
 70796  71346     ** sqlite3BtreeCursor() may return is SQLITE_OK. */
................................................................................
 70937  71487   case OP_Close: {
 70938  71488     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
 70939  71489     sqlite3VdbeFreeCursor(p, p->apCsr[pOp->p1]);
 70940  71490     p->apCsr[pOp->p1] = 0;
 70941  71491     break;
 70942  71492   }
 70943  71493   
 70944         -/* Opcode: SeekGe P1 P2 P3 P4 *
        71494  +/* Opcode: SeekGE P1 P2 P3 P4 *
 70945  71495   ** Synopsis: key=r[P3@P4]
 70946  71496   **
 70947  71497   ** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), 
 70948  71498   ** use the value in register P3 as the key.  If cursor P1 refers 
 70949  71499   ** to an SQL index, then P3 is the first in an array of P4 registers 
 70950  71500   ** that are used as an unpacked index key. 
 70951  71501   **
 70952  71502   ** Reposition cursor P1 so that  it points to the smallest entry that 
 70953  71503   ** is greater than or equal to the key value. If there are no records 
 70954  71504   ** greater than or equal to the key and P2 is not zero, then jump to P2.
        71505  +**
        71506  +** This opcode leaves the cursor configured to move in forward order,
        71507  +** from the beginning toward the end.  In other words, the cursor is
        71508  +** configured to use Next, not Prev.
 70955  71509   **
 70956  71510   ** See also: Found, NotFound, SeekLt, SeekGt, SeekLe
 70957  71511   */
 70958         -/* Opcode: SeekGt P1 P2 P3 P4 *
        71512  +/* Opcode: SeekGT P1 P2 P3 P4 *
 70959  71513   ** Synopsis: key=r[P3@P4]
 70960  71514   **
 70961  71515   ** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), 
 70962  71516   ** use the value in register P3 as a key. If cursor P1 refers 
 70963  71517   ** to an SQL index, then P3 is the first in an array of P4 registers 
 70964  71518   ** that are used as an unpacked index key. 
 70965  71519   **
 70966  71520   ** Reposition cursor P1 so that  it points to the smallest entry that 
 70967  71521   ** is greater than the key value. If there are no records greater than 
 70968  71522   ** the key and P2 is not zero, then jump to P2.
        71523  +**
        71524  +** This opcode leaves the cursor configured to move in forward order,
        71525  +** from the beginning toward the end.  In other words, the cursor is
        71526  +** configured to use Next, not Prev.
 70969  71527   **
 70970  71528   ** See also: Found, NotFound, SeekLt, SeekGe, SeekLe
 70971  71529   */
 70972         -/* Opcode: SeekLt P1 P2 P3 P4 * 
        71530  +/* Opcode: SeekLT P1 P2 P3 P4 * 
 70973  71531   ** Synopsis: key=r[P3@P4]
 70974  71532   **
 70975  71533   ** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), 
 70976  71534   ** use the value in register P3 as a key. If cursor P1 refers 
 70977  71535   ** to an SQL index, then P3 is the first in an array of P4 registers 
 70978  71536   ** that are used as an unpacked index key. 
 70979  71537   **
 70980  71538   ** Reposition cursor P1 so that  it points to the largest entry that 
 70981  71539   ** is less than the key value. If there are no records less than 
 70982  71540   ** the key and P2 is not zero, then jump to P2.
        71541  +**
        71542  +** This opcode leaves the cursor configured to move in reverse order,
        71543  +** from the end toward the beginning.  In other words, the cursor is
        71544  +** configured to use Prev, not Next.
 70983  71545   **
 70984  71546   ** See also: Found, NotFound, SeekGt, SeekGe, SeekLe
 70985  71547   */
 70986         -/* Opcode: SeekLe P1 P2 P3 P4 *
        71548  +/* Opcode: SeekLE P1 P2 P3 P4 *
 70987  71549   ** Synopsis: key=r[P3@P4]
 70988  71550   **
 70989  71551   ** If cursor P1 refers to an SQL table (B-Tree that uses integer keys), 
 70990  71552   ** use the value in register P3 as a key. If cursor P1 refers 
 70991  71553   ** to an SQL index, then P3 is the first in an array of P4 registers 
 70992  71554   ** that are used as an unpacked index key. 
 70993  71555   **
 70994  71556   ** Reposition cursor P1 so that it points to the largest entry that 
 70995  71557   ** is less than or equal to the key value. If there are no records 
 70996  71558   ** less than or equal to the key and P2 is not zero, then jump to P2.
        71559  +**
        71560  +** This opcode leaves the cursor configured to move in reverse order,
        71561  +** from the end toward the beginning.  In other words, the cursor is
        71562  +** configured to use Prev, not Next.
 70997  71563   **
 70998  71564   ** See also: Found, NotFound, SeekGt, SeekGe, SeekLt
 70999  71565   */
 71000  71566   case OP_SeekLT:         /* jump, in3 */
 71001  71567   case OP_SeekLE:         /* jump, in3 */
 71002  71568   case OP_SeekGE:         /* jump, in3 */
 71003  71569   case OP_SeekGT: {       /* jump, in3 */
................................................................................
 71016  71582     assert( OP_SeekLE == OP_SeekLT+1 );
 71017  71583     assert( OP_SeekGE == OP_SeekLT+2 );
 71018  71584     assert( OP_SeekGT == OP_SeekLT+3 );
 71019  71585     assert( pC->isOrdered );
 71020  71586     assert( pC->pCursor!=0 );
 71021  71587     oc = pOp->opcode;
 71022  71588     pC->nullRow = 0;
        71589  +#ifdef SQLITE_DEBUG
        71590  +  pC->seekOp = pOp->opcode;
        71591  +#endif
 71023  71592     if( pC->isTable ){
 71024  71593       /* The input value in P3 might be of any type: integer, real, string,
 71025  71594       ** blob, or NULL.  But it needs to be an integer before we can do
 71026  71595       ** the seek, so covert it. */
 71027  71596       pIn3 = &aMem[pOp->p3];
 71028         -    applyNumericAffinity(pIn3);
        71597  +    ApplyNumericAffinity(pIn3);
 71029  71598       iKey = sqlite3VdbeIntValue(pIn3);
 71030  71599       pC->rowidIsValid = 0;
 71031  71600   
 71032  71601       /* If the P3 value could not be converted into an integer without
 71033  71602       ** loss of information, then special processing is required... */
 71034  71603       if( (pIn3->flags & MEM_Int)==0 ){
 71035  71604         if( (pIn3->flags & MEM_Real)==0 ){
................................................................................
 71170  71739   ** If P4==0 then register P3 holds a blob constructed by MakeRecord.  If
 71171  71740   ** P4>0 then register P3 is the first of P4 registers that form an unpacked
 71172  71741   ** record.
 71173  71742   **
 71174  71743   ** Cursor P1 is on an index btree.  If the record identified by P3 and P4
 71175  71744   ** is a prefix of any entry in P1 then a jump is made to P2 and
 71176  71745   ** P1 is left pointing at the matching entry.
        71746  +**
        71747  +** This operation leaves the cursor in a state where it can be
        71748  +** advanced in the forward direction.  The Next instruction will work,
        71749  +** but not the Prev instruction.
 71177  71750   **
 71178  71751   ** See also: NotFound, NoConflict, NotExists. SeekGe
 71179  71752   */
 71180  71753   /* Opcode: NotFound P1 P2 P3 P4 *
 71181  71754   ** Synopsis: key=r[P3@P4]
 71182  71755   **
 71183  71756   ** If P4==0 then register P3 holds a blob constructed by MakeRecord.  If
................................................................................
 71185  71758   ** record.
 71186  71759   ** 
 71187  71760   ** Cursor P1 is on an index btree.  If the record identified by P3 and P4
 71188  71761   ** is not the prefix of any entry in P1 then a jump is made to P2.  If P1 
 71189  71762   ** does contain an entry whose prefix matches the P3/P4 record then control
 71190  71763   ** falls through to the next instruction and P1 is left pointing at the
 71191  71764   ** matching entry.
        71765  +**
        71766  +** This operation leaves the cursor in a state where it cannot be
        71767  +** advanced in either direction.  In other words, the Next and Prev
        71768  +** opcodes do not work after this operation.
 71192  71769   **
 71193  71770   ** See also: Found, NotExists, NoConflict
 71194  71771   */
 71195  71772   /* Opcode: NoConflict P1 P2 P3 P4 *
 71196  71773   ** Synopsis: key=r[P3@P4]
 71197  71774   **
 71198  71775   ** If P4==0 then register P3 holds a blob constructed by MakeRecord.  If
................................................................................
 71204  71781   ** record are not-NULL then a check is done to determine if any row in the
 71205  71782   ** P1 index btree has a matching key prefix.  If there are no matches, jump
 71206  71783   ** immediately to P2.  If there is a match, fall through and leave the P1
 71207  71784   ** cursor pointing to the matching row.
 71208  71785   **
 71209  71786   ** This opcode is similar to OP_NotFound with the exceptions that the
 71210  71787   ** branch is always taken if any part of the search key input is NULL.
        71788  +**
        71789  +** This operation leaves the cursor in a state where it cannot be
        71790  +** advanced in either direction.  In other words, the Next and Prev
        71791  +** opcodes do not work after this operation.
 71211  71792   **
 71212  71793   ** See also: NotFound, Found, NotExists
 71213  71794   */
 71214  71795   case OP_NoConflict:     /* jump, in3 */
 71215  71796   case OP_NotFound:       /* jump, in3 */
 71216  71797   case OP_Found: {        /* jump, in3 */
 71217  71798     int alreadyExists;
................................................................................
 71227  71808     if( pOp->opcode!=OP_NoConflict ) sqlite3_found_count++;
 71228  71809   #endif
 71229  71810   
 71230  71811     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
 71231  71812     assert( pOp->p4type==P4_INT32 );
 71232  71813     pC = p->apCsr[pOp->p1];
 71233  71814     assert( pC!=0 );
        71815  +#ifdef SQLITE_DEBUG
        71816  +  pC->seekOp = pOp->opcode;
        71817  +#endif
 71234  71818     pIn3 = &aMem[pOp->p3];
 71235  71819     assert( pC->pCursor!=0 );
 71236  71820     assert( pC->isTable==0 );
 71237  71821     pFree = 0;  /* Not needed.  Only used to suppress a compiler warning. */
 71238  71822     if( pOp->p4.i>0 ){
 71239  71823       r.pKeyInfo = pC->pKeyInfo;
 71240  71824       r.nField = (u16)pOp->p4.i;
................................................................................
 71297  71881   ** keys).  P3 is an integer rowid.  If P1 does not contain a record with
 71298  71882   ** rowid P3 then jump immediately to P2.  If P1 does contain a record
 71299  71883   ** with rowid P3 then leave the cursor pointing at that record and fall
 71300  71884   ** through to the next instruction.
 71301  71885   **
 71302  71886   ** The OP_NotFound opcode performs the same operation on index btrees
 71303  71887   ** (with arbitrary multi-value keys).
        71888  +**
        71889  +** This opcode leaves the cursor in a state where it cannot be advanced
        71890  +** in either direction.  In other words, the Next and Prev opcodes will
        71891  +** not work following this opcode.
 71304  71892   **
 71305  71893   ** See also: Found, NotFound, NoConflict
 71306  71894   */
 71307  71895   case OP_NotExists: {        /* jump, in3 */
 71308  71896     VdbeCursor *pC;
 71309  71897     BtCursor *pCrsr;
 71310  71898     int res;
................................................................................
 71311  71899     u64 iKey;
 71312  71900   
 71313  71901     pIn3 = &aMem[pOp->p3];
 71314  71902     assert( pIn3->flags & MEM_Int );
 71315  71903     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
 71316  71904     pC = p->apCsr[pOp->p1];
 71317  71905     assert( pC!=0 );
        71906  +#ifdef SQLITE_DEBUG
        71907  +  pC->seekOp = 0;
        71908  +#endif
 71318  71909     assert( pC->isTable );
 71319  71910     assert( pC->pseudoTableReg==0 );
 71320  71911     pCrsr = pC->pCursor;
 71321  71912     assert( pCrsr!=0 );
 71322  71913     res = 0;
 71323  71914     iKey = pIn3->u.i;
 71324  71915     rc = sqlite3BtreeMovetoUnpacked(pCrsr, 0, iKey, 0, &res);
................................................................................
 71613  72204   /* Opcode: Delete P1 P2 * P4 *
 71614  72205   **
 71615  72206   ** Delete the record at which the P1 cursor is currently pointing.
 71616  72207   **
 71617  72208   ** The cursor will be left pointing at either the next or the previous
 71618  72209   ** record in the table. If it is left pointing at the next record, then
 71619  72210   ** the next Next instruction will be a no-op.  Hence it is OK to delete
 71620         -** a record from within an Next loop.
        72211  +** a record from within a Next loop.
 71621  72212   **
 71622  72213   ** If the OPFLAG_NCHANGE flag of P2 is set, then the row change count is
 71623  72214   ** incremented (otherwise not).
 71624  72215   **
 71625  72216   ** P1 must not be pseudo-table.  It has to be a real table with
 71626  72217   ** multiple rows.
 71627  72218   **
................................................................................
 71673  72264   case OP_ResetCount: {
 71674  72265     sqlite3VdbeSetChanges(db, p->nChange);
 71675  72266     p->nChange = 0;
 71676  72267     break;
 71677  72268   }
 71678  72269   
 71679  72270   /* Opcode: SorterCompare P1 P2 P3 P4
 71680         -** Synopsis:  if key(P1)!=rtrim(r[P3],P4) goto P2
        72271  +** Synopsis:  if key(P1)!=trim(r[P3],P4) goto P2
 71681  72272   **
 71682  72273   ** P1 is a sorter cursor. This instruction compares a prefix of the
 71683         -** the record blob in register P3 against a prefix of the entry that 
 71684         -** the sorter cursor currently points to.  The final P4 fields of both
 71685         -** the P3 and sorter record are ignored.
        72274  +** record blob in register P3 against a prefix of the entry that 
        72275  +** the sorter cursor currently points to.  Only the first P4 fields
        72276  +** of r[P3] and the sorter record are compared.
 71686  72277   **
 71687  72278   ** If either P3 or the sorter contains a NULL in one of their significant
 71688  72279   ** fields (not counting the P4 fields at the end which are ignored) then
 71689  72280   ** the comparison is assumed to be equal.
 71690  72281   **
 71691  72282   ** Fall through to next instruction if the two records compare equal to
 71692  72283   ** each other.  Jump to P2 if they are different.
 71693  72284   */
 71694  72285   case OP_SorterCompare: {
 71695  72286     VdbeCursor *pC;
 71696  72287     int res;
 71697         -  int nIgnore;
        72288  +  int nKeyCol;
 71698  72289   
 71699  72290     pC = p->apCsr[pOp->p1];
 71700  72291     assert( isSorter(pC) );
 71701  72292     assert( pOp->p4type==P4_INT32 );
 71702  72293     pIn3 = &aMem[pOp->p3];
 71703         -  nIgnore = pOp->p4.i;
 71704         -  rc = sqlite3VdbeSorterCompare(pC, pIn3, nIgnore, &res);
        72294  +  nKeyCol = pOp->p4.i;
        72295  +  rc = sqlite3VdbeSorterCompare(pC, pIn3, nKeyCol, &res);
 71705  72296     VdbeBranchTaken(res!=0,2);
 71706  72297     if( res ){
 71707  72298       pc = pOp->p2-1;
 71708  72299     }
 71709  72300     break;
 71710  72301   };
 71711  72302   
................................................................................
 71877  72468       sqlite3BtreeClearCursor(pC->pCursor);
 71878  72469     }
 71879  72470     break;
 71880  72471   }
 71881  72472   
 71882  72473   /* Opcode: Last P1 P2 * * *
 71883  72474   **
 71884         -** The next use of the Rowid or Column or Next instruction for P1 
        72475  +** The next use of the Rowid or Column or Prev instruction for P1 
 71885  72476   ** will refer to the last entry in the database table or index.
 71886  72477   ** If the table or index is empty and P2>0, then jump immediately to P2.
 71887  72478   ** If P2 is 0 or if the table or index is not empty, fall through
 71888  72479   ** to the following instruction.
        72480  +**
        72481  +** This opcode leaves the cursor configured to move in reverse order,
        72482  +** from the end toward the beginning.  In other words, the cursor is
        72483  +** configured to use Prev, not Next.
 71889  72484   */
 71890  72485   case OP_Last: {        /* jump */
 71891  72486     VdbeCursor *pC;
 71892  72487     BtCursor *pCrsr;
 71893  72488     int res;
 71894  72489   
 71895  72490     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
................................................................................
 71899  72494     res = 0;
 71900  72495     assert( pCrsr!=0 );
 71901  72496     rc = sqlite3BtreeLast(pCrsr, &res);
 71902  72497     pC->nullRow = (u8)res;
 71903  72498     pC->deferredMoveto = 0;
 71904  72499     pC->rowidIsValid = 0;
 71905  72500     pC->cacheStatus = CACHE_STALE;
        72501  +#ifdef SQLITE_DEBUG
        72502  +  pC->seekOp = OP_Last;
        72503  +#endif
 71906  72504     if( pOp->p2>0 ){
 71907  72505       VdbeBranchTaken(res!=0,2);
 71908  72506       if( res ) pc = pOp->p2 - 1;
 71909  72507     }
 71910  72508     break;
 71911  72509   }
 71912  72510   
................................................................................
 71935  72533   /* Opcode: Rewind P1 P2 * * *
 71936  72534   **
 71937  72535   ** The next use of the Rowid or Column or Next instruction for P1 
 71938  72536   ** will refer to the first entry in the database table or index.
 71939  72537   ** If the table or index is empty and P2>0, then jump immediately to P2.
 71940  72538   ** If P2 is 0 or if the table or index is not empty, fall through
 71941  72539   ** to the following instruction.
        72540  +**
        72541  +** This opcode leaves the cursor configured to move in forward order,
        72542  +** from the beginning toward the end.  In other words, the cursor is
        72543  +** configured to use Next, not Prev.
 71942  72544   */
 71943  72545   case OP_Rewind: {        /* jump */
 71944  72546     VdbeCursor *pC;
 71945  72547     BtCursor *pCrsr;
 71946  72548     int res;
 71947  72549   
 71948  72550     assert( pOp->p1>=0 && pOp->p1<p->nCursor );
 71949  72551     pC = p->apCsr[pOp->p1];
 71950  72552     assert( pC!=0 );
 71951  72553     assert( isSorter(pC)==(pOp->opcode==OP_SorterSort) );
 71952  72554     res = 1;
        72555  +#ifdef SQLITE_DEBUG
        72556  +  pC->seekOp = OP_Rewind;
        72557  +#endif
 71953  72558     if( isSorter(pC) ){
 71954  72559       rc = sqlite3VdbeSorterRewind(db, pC, &res);
 71955  72560     }else{
 71956  72561       pCrsr = pC->pCursor;
 71957  72562       assert( pCrsr );
 71958  72563       rc = sqlite3BtreeFirst(pCrsr, &res);
 71959  72564       pC->deferredMoveto = 0;
................................................................................
 71971  72576   
 71972  72577   /* Opcode: Next P1 P2 P3 P4 P5
 71973  72578   **
 71974  72579   ** Advance cursor P1 so that it points to the next key/data pair in its
 71975  72580   ** table or index.  If there are no more key/value pairs then fall through
 71976  72581   ** to the following instruction.  But if the cursor advance was successful,
 71977  72582   ** jump immediately to P2.
        72583  +**
        72584  +** The Next opcode is only valid following an SeekGT, SeekGE, or
        72585  +** OP_Rewind opcode used to position the cursor.  Next is not allowed
        72586  +** to follow SeekLT, SeekLE, or OP_Last.
 71978  72587   **
 71979  72588   ** The P1 cursor must be for a real table, not a pseudo-table.  P1 must have
 71980  72589   ** been opened prior to this opcode or the program will segfault.
 71981  72590   **
 71982  72591   ** The P3 value is a hint to the btree implementation. If P3==1, that
 71983  72592   ** means P1 is an SQL index and that this instruction could have been
 71984  72593   ** omitted if that index had been unique.  P3 is usually 0.  P3 is
................................................................................
 71990  72599   ** If P5 is positive and the jump is taken, then event counter
 71991  72600   ** number P5-1 in the prepared statement is incremented.
 71992  72601   **
 71993  72602   ** See also: Prev, NextIfOpen
 71994  72603   */
 71995  72604   /* Opcode: NextIfOpen P1 P2 P3 P4 P5
 71996  72605   **
 71997         -** This opcode works just like OP_Next except that if cursor P1 is not
        72606  +** This opcode works just like Next except that if cursor P1 is not
 71998  72607   ** open it behaves a no-op.
 71999  72608   */
 72000  72609   /* Opcode: Prev P1 P2 P3 P4 P5
 72001  72610   **
 72002  72611   ** Back up cursor P1 so that it points to the previous key/data pair in its
 72003  72612   ** table or index.  If there is no previous key/value pairs then fall through
 72004  72613   ** to the following instruction.  But if the cursor backup was successful,
 72005  72614   ** jump immediately to P2.
 72006  72615   **
        72616  +**
        72617  +** The Prev opcode is only valid following an SeekLT, SeekLE, or
        72618  +** OP_Last opcode used to position the cursor.  Prev is not allowed
        72619  +** to follow SeekGT, SeekGE, or OP_Rewind.
        72620  +**
 72007  72621   ** The P1 cursor must be for a real table, not a pseudo-table.  If P1 is
 72008  72622   ** not open then the behavior is undefined.
 72009  72623   **
 72010  72624   ** The P3 value is a hint to the btree implementation. If P3==1, that
 72011  72625   ** means P1 is an SQL index and that this instruction could have been
 72012  72626   ** omitted if that index had been unique.  P3 is usually 0.  P3 is
 72013  72627   ** always either 0 or 1.
................................................................................
 72016  72630   ** sqlite3BtreePrevious().
 72017  72631   **
 72018  72632   ** If P5 is positive and the jump is taken, then event counter
 72019  72633   ** number P5-1 in the prepared statement is incremented.
 72020  72634   */
 72021  72635   /* Opcode: PrevIfOpen P1 P2 P3 P4 P5
 72022  72636   **
 72023         -** This opcode works just like OP_Prev except that if cursor P1 is not
        72637  +** This opcode works just like Prev except that if cursor P1 is not
 72024  72638   ** open it behaves a no-op.
 72025  72639   */
 72026  72640   case OP_SorterNext: {  /* jump */
 72027  72641     VdbeCursor *pC;
 72028  72642     int res;
 72029  72643   
 72030  72644     pC = p->apCsr[pOp->p1];
................................................................................
 72047  72661     assert( pC->pCursor );
 72048  72662     assert( res==0 || (res==1 && pC->isTable==0) );
 72049  72663     testcase( res==1 );
 72050  72664     assert( pOp->opcode!=OP_Next || pOp->p4.xAdvance==sqlite3BtreeNext );
 72051  72665     assert( pOp->opcode!=OP_Prev || pOp->p4.xAdvance==sqlite3BtreePrevious );
 72052  72666     assert( pOp->opcode!=OP_NextIfOpen || pOp->p4.xAdvance==sqlite3BtreeNext );
 72053  72667     assert( pOp->opcode!=OP_PrevIfOpen || pOp->p4.xAdvance==sqlite3BtreePrevious);
        72668  +
        72669  +  /* The Next opcode is only used after SeekGT, SeekGE, and Rewind.
        72670  +  ** The Prev opcode is only used after SeekLT, SeekLE, and Last. */
        72671  +  assert( pOp->opcode!=OP_Next || pOp->opcode!=OP_NextIfOpen
        72672  +       || pC->seekOp==OP_SeekGT || pC->seekOp==OP_SeekGE
        72673  +       || pC->seekOp==OP_Rewind || pC->seekOp==OP_Found);
        72674  +  assert( pOp->opcode!=OP_Prev || pOp->opcode!=OP_PrevIfOpen
        72675  +       || pC->seekOp==OP_SeekLT || pC->seekOp==OP_SeekLE
        72676  +       || pC->seekOp==OP_Last );
        72677  +
 72054  72678     rc = pOp->p4.xAdvance(pC->pCursor, &res);
 72055  72679   next_tail:
 72056  72680     pC->cacheStatus = CACHE_STALE;
 72057  72681     VdbeBranchTaken(res==0,2);
 72058  72682     if( res==0 ){
 72059  72683       pC->nullRow = 0;
 72060  72684       pc = pOp->p2 - 1;
................................................................................
 72329  72953     pOut->flags = MEM_Null;
 72330  72954     if( iCnt>1 ){
 72331  72955       rc = SQLITE_LOCKED;
 72332  72956       p->errorAction = OE_Abort;
 72333  72957     }else{
 72334  72958       iDb = pOp->p3;
 72335  72959       assert( iCnt==1 );
 72336         -    assert( (p->btreeMask & (((yDbMask)1)<<iDb))!=0 );
        72960  +    assert( DbMaskTest(p->btreeMask, iDb) );
 72337  72961       iMoved = 0;  /* Not needed.  Only to silence a warning. */
 72338  72962       rc = sqlite3BtreeDropTable(db->aDb[iDb].pBt, pOp->p1, &iMoved);
 72339  72963       pOut->flags = MEM_Int;
 72340  72964       pOut->u.i = iMoved;
 72341  72965   #ifndef SQLITE_OMIT_AUTOVACUUM
 72342  72966       if( rc==SQLITE_OK && iMoved!=0 ){
 72343  72967         sqlite3RootPageMoved(db, iDb, iMoved, pOp->p1);
................................................................................
 72369  72993   ** See also: Destroy
 72370  72994   */
 72371  72995   case OP_Clear: {
 72372  72996     int nChange;
 72373  72997    
 72374  72998     nChange = 0;
 72375  72999     assert( p->readOnly==0 );
 72376         -  assert( (p->btreeMask & (((yDbMask)1)<<pOp->p2))!=0 );
        73000  +  assert( DbMaskTest(p->btreeMask, pOp->p2) );
 72377  73001     rc = sqlite3BtreeClearTable(
 72378  73002         db->aDb[pOp->p2].pBt, pOp->p1, (pOp->p3 ? &nChange : 0)
 72379  73003     );
 72380  73004     if( pOp->p3 ){
 72381  73005       p->nChange += nChange;
 72382  73006       if( pOp->p3>0 ){
 72383  73007         assert( memIsValid(&aMem[pOp->p3]) );
................................................................................
 72439  73063   case OP_CreateTable: {          /* out2-prerelease */
 72440  73064     int pgno;
 72441  73065     int flags;
 72442  73066     Db *pDb;
 72443  73067   
 72444  73068     pgno = 0;
 72445  73069     assert( pOp->p1>=0 && pOp->p1<db->nDb );
 72446         -  assert( (p->btreeMask & (((yDbMask)1)<<pOp->p1))!=0 );
        73070  +  assert( DbMaskTest(p->btreeMask, pOp->p1) );
 72447  73071     assert( p->readOnly==0 );
 72448  73072     pDb = &db->aDb[pOp->p1];
 72449  73073     assert( pDb->pBt!=0 );
 72450  73074     if( pOp->opcode==OP_CreateTable ){
 72451  73075       /* flags = BTREE_INTKEY; */
 72452  73076       flags = BTREE_INTKEY;
 72453  73077     }else{
................................................................................
 72527  73151   }
 72528  73152   #endif /* !defined(SQLITE_OMIT_ANALYZE) */
 72529  73153   
 72530  73154   /* Opcode: DropTable P1 * * P4 *
 72531  73155   **
 72532  73156   ** Remove the internal (in-memory) data structures that describe
 72533  73157   ** the table named P4 in database P1.  This is called after a table
 72534         -** is dropped in order to keep the internal representation of the
        73158  +** is dropped from disk (using the Destroy opcode) in order to keep 
        73159  +** the internal representation of the
 72535  73160   ** schema consistent with what is on disk.
 72536  73161   */
 72537  73162   case OP_DropTable: {
 72538  73163     sqlite3UnlinkAndDeleteTable(db, pOp->p1, pOp->p4.z);
 72539  73164     break;
 72540  73165   }
 72541  73166   
 72542  73167   /* Opcode: DropIndex P1 * * P4 *
 72543  73168   **
 72544  73169   ** Remove the internal (in-memory) data structures that describe
 72545  73170   ** the index named P4 in database P1.  This is called after an index
 72546         -** is dropped in order to keep the internal representation of the
        73171  +** is dropped from disk (using the Destroy opcode)
        73172  +** in order to keep the internal representation of the
 72547  73173   ** schema consistent with what is on disk.
 72548  73174   */
 72549  73175   case OP_DropIndex: {
 72550  73176     sqlite3UnlinkAndDeleteIndex(db, pOp->p1, pOp->p4.z);
 72551  73177     break;
 72552  73178   }
 72553  73179   
 72554  73180   /* Opcode: DropTrigger P1 * * P4 *
 72555  73181   **
 72556  73182   ** Remove the internal (in-memory) data structures that describe
 72557  73183   ** the trigger named P4 in database P1.  This is called after a trigger
 72558         -** is dropped in order to keep the internal representation of the
        73184  +** is dropped from disk (using the Destroy opcode) in order to keep 
        73185  +** the internal representation of the
 72559  73186   ** schema consistent with what is on disk.
 72560  73187   */
 72561  73188   case OP_DropTrigger: {
 72562  73189     sqlite3UnlinkAndDeleteTrigger(db, pOp->p1, pOp->p4.z);
 72563  73190     break;
 72564  73191   }
 72565  73192   
................................................................................
 72604  73231     assert( (pnErr->flags & (MEM_Str|MEM_Blob))==0 );
 72605  73232     pIn1 = &aMem[pOp->p1];
 72606  73233     for(j=0; j<nRoot; j++){
 72607  73234       aRoot[j] = (int)sqlite3VdbeIntValue(&pIn1[j]);
 72608  73235     }
 72609  73236     aRoot[j] = 0;
 72610  73237     assert( pOp->p5<db->nDb );
 72611         -  assert( (p->btreeMask & (((yDbMask)1)<<pOp->p5))!=0 );
        73238  +  assert( DbMaskTest(p->btreeMask, pOp->p5) );
 72612  73239     z = sqlite3BtreeIntegrityCheck(db->aDb[pOp->p5].pBt, aRoot, nRoot,
 72613  73240                                    (int)pnErr->u.i, &nErr);
 72614  73241     sqlite3DbFree(db, aRoot);
 72615  73242     pnErr->u.i -= nErr;
 72616  73243     sqlite3VdbeMemSetNull(pIn1);
 72617  73244     if( nErr==0 ){
 72618  73245       assert( z==0 );
................................................................................
 72966  73593     VdbeBranchTaken( pIn1->u.i>0, 2);
 72967  73594     if( pIn1->u.i>0 ){
 72968  73595        pc = pOp->p2 - 1;
 72969  73596     }
 72970  73597     break;
 72971  73598   }
 72972  73599   
 72973         -/* Opcode: IfNeg P1 P2 * * *
 72974         -** Synopsis: if r[P1]<0 goto P2
        73600  +/* Opcode: IfNeg P1 P2 P3 * *
        73601  +** Synopsis: r[P1]+=P3, if r[P1]<0 goto P2
 72975  73602   **
 72976         -** If the value of register P1 is less than zero, jump to P2. 
 72977         -**
 72978         -** It is illegal to use this instruction on a register that does
 72979         -** not contain an integer.  An assertion fault will result if you try.
        73603  +** Register P1 must contain an integer.  Add literal P3 to the value in
        73604  +** register P1 then if the value of register P1 is less than zero, jump to P2. 
 72980  73605   */
 72981  73606   case OP_IfNeg: {        /* jump, in1 */
 72982  73607     pIn1 = &aMem[pOp->p1];
 72983  73608     assert( pIn1->flags&MEM_Int );
        73609  +  pIn1->u.i += pOp->p3;
 72984  73610     VdbeBranchTaken(pIn1->u.i<0, 2);
 72985  73611     if( pIn1->u.i<0 ){
 72986  73612        pc = pOp->p2 - 1;
 72987  73613     }
 72988  73614     break;
 72989  73615   }
 72990  73616   
 72991  73617   /* Opcode: IfZero P1 P2 P3 * *
 72992  73618   ** Synopsis: r[P1]+=P3, if r[P1]==0 goto P2
 72993  73619   **
 72994  73620   ** The register P1 must contain an integer.  Add literal P3 to the
 72995  73621   ** value in register P1.  If the result is exactly 0, jump to P2. 
 72996         -**
 72997         -** It is illegal to use this instruction on a register that does
 72998         -** not contain an integer.  An assertion fault will result if you try.
 72999  73622   */
 73000  73623   case OP_IfZero: {        /* jump, in1 */
 73001  73624     pIn1 = &aMem[pOp->p1];
 73002  73625     assert( pIn1->flags&MEM_Int );
 73003  73626     pIn1->u.i += pOp->p3;
 73004  73627     VdbeBranchTaken(pIn1->u.i==0, 2);
 73005  73628     if( pIn1->u.i==0 ){
................................................................................
 73264  73887   ** the P1 database. If the vacuum has finished, jump to instruction
 73265  73888   ** P2. Otherwise, fall through to the next instruction.
 73266  73889   */
 73267  73890   case OP_IncrVacuum: {        /* jump */
 73268  73891     Btree *pBt;
 73269  73892   
 73270  73893     assert( pOp->p1>=0 && pOp->p1<db->nDb );
 73271         -  assert( (p->btreeMask & (((yDbMask)1)<<pOp->p1))!=0 );
        73894  +  assert( DbMaskTest(p->btreeMask, pOp->p1) );
 73272  73895     assert( p->readOnly==0 );
 73273  73896     pBt = db->aDb[pOp->p1].pBt;
 73274  73897     rc = sqlite3BtreeIncrVacuum(pBt);
 73275  73898     VdbeBranchTaken(rc==SQLITE_DONE,2);
 73276  73899     if( rc==SQLITE_DONE ){
 73277  73900       pc = pOp->p2 - 1;
 73278  73901       rc = SQLITE_OK;
................................................................................
 73279  73902     }
 73280  73903     break;
 73281  73904   }
 73282  73905   #endif
 73283  73906   
 73284  73907   /* Opcode: Expire P1 * * * *
 73285  73908   **
 73286         -** Cause precompiled statements to become expired. An expired statement
 73287         -** fails with an error code of SQLITE_SCHEMA if it is ever executed 
 73288         -** (via sqlite3_step()).
        73909  +** Cause precompiled statements to expire.  When an expired statement
        73910  +** is executed using sqlite3_step() it will either automatically
        73911  +** reprepare itself (if it was originally created using sqlite3_prepare_v2())
        73912  +** or it will fail with SQLITE_SCHEMA.
 73289  73913   ** 
 73290  73914   ** If P1 is 0, then all SQL statements become expired. If P1 is non-zero,
 73291         -** then only the currently executing statement is affected. 
        73915  +** then only the currently executing statement is expired.
 73292  73916   */
 73293  73917   case OP_Expire: {
 73294  73918     if( !pOp->p1 ){
 73295  73919       sqlite3ExpirePreparedStatements(db);
 73296  73920     }else{
 73297  73921       p->expired = 1;
 73298  73922     }
................................................................................
 73316  73940   ** used to generate an error message if the lock cannot be obtained.
 73317  73941   */
 73318  73942   case OP_TableLock: {
 73319  73943     u8 isWriteLock = (u8)pOp->p3;
 73320  73944     if( isWriteLock || 0==(db->flags&SQLITE_ReadUncommitted) ){
 73321  73945       int p1 = pOp->p1; 
 73322  73946       assert( p1>=0 && p1<db->nDb );
 73323         -    assert( (p->btreeMask & (((yDbMask)1)<<p1))!=0 );
        73947  +    assert( DbMaskTest(p->btreeMask, p1) );
 73324  73948       assert( isWriteLock==0 || isWriteLock==1 );
 73325  73949       rc = sqlite3BtreeLockTable(db->aDb[p1].pBt, pOp->p2, isWriteLock);
 73326  73950       if( (rc&0xFF)==SQLITE_LOCKED ){
 73327  73951         const char *z = pOp->p4.z;
 73328  73952         sqlite3SetString(&p->zErrMsg, db, "database table is locked: %s", z);
 73329  73953       }
 73330  73954     }
................................................................................
 73766  74390       sqlite3DbFree(db, z);
 73767  74391     }
 73768  74392   #ifdef SQLITE_USE_FCNTL_TRACE
 73769  74393     zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql);
 73770  74394     if( zTrace ){
 73771  74395       int i;
 73772  74396       for(i=0; i<db->nDb; i++){
 73773         -      if( (MASKBIT(i) & p->btreeMask)==0 ) continue;
        74397  +      if( DbMaskTest(p->btreeMask, i)==0 ) continue;
 73774  74398         sqlite3_file_control(db, db->aDb[i].zName, SQLITE_FCNTL_TRACE, zTrace);
 73775  74399       }
 73776  74400     }
 73777  74401   #endif /* SQLITE_USE_FCNTL_TRACE */
 73778  74402   #ifdef SQLITE_DEBUG
 73779  74403     if( (db->flags & SQLITE_SqlTrace)!=0
 73780  74404      && (zTrace = (pOp->p4.z ? pOp->p4.z : p->zSql))!=0
................................................................................
 74756  75380   ** be less than key2. Even if key2 also contains NULL values.
 74757  75381   **
 74758  75382   ** If pKey2 is passed a NULL pointer, then it is assumed that the pCsr->aSpace
 74759  75383   ** has been allocated and contains an unpacked record that is used as key2.
 74760  75384   */
 74761  75385   static void vdbeSorterCompare(
 74762  75386     const VdbeCursor *pCsr,         /* Cursor object (for pKeyInfo) */
 74763         -  int nIgnore,                    /* Ignore the last nIgnore fields */
        75387  +  int nKeyCol,                    /* Num of columns. 0 means "all" */
 74764  75388     const void *pKey1, int nKey1,   /* Left side of comparison */
 74765  75389     const void *pKey2, int nKey2,   /* Right side of comparison */
 74766  75390     int *pRes                       /* OUT: Result of comparison */
 74767  75391   ){
 74768  75392     KeyInfo *pKeyInfo = pCsr->pKeyInfo;
 74769  75393     VdbeSorter *pSorter = pCsr->pSorter;
 74770  75394     UnpackedRecord *r2 = pSorter->pUnpacked;
 74771  75395     int i;
 74772  75396   
 74773  75397     if( pKey2 ){
 74774  75398       sqlite3VdbeRecordUnpack(pKeyInfo, nKey2, pKey2, r2);
 74775  75399     }
 74776  75400   
 74777         -  if( nIgnore ){
 74778         -    r2->nField = pKeyInfo->nField - nIgnore;
 74779         -    assert( r2->nField>0 );
 74780         -    for(i=0; i<r2->nField; i++){
        75401  +  if( nKeyCol ){
        75402  +    r2->nField = nKeyCol;
        75403  +    for(i=0; i<nKeyCol; i++){
 74781  75404         if( r2->aMem[i].flags & MEM_Null ){
 74782  75405           *pRes = -1;
 74783  75406           return;
 74784  75407         }
 74785  75408       }
 74786  75409       assert( r2->default_rc==0 );
 74787  75410     }
................................................................................
 75455  76078   ** Otherwise, set *pRes to a negative, zero or positive value if the
 75456  76079   ** key in pVal is smaller than, equal to or larger than the current sorter
 75457  76080   ** key.
 75458  76081   */
 75459  76082   SQLITE_PRIVATE int sqlite3VdbeSorterCompare(
 75460  76083     const VdbeCursor *pCsr,         /* Sorter cursor */
 75461  76084     Mem *pVal,                      /* Value to compare to current sorter key */
 75462         -  int nIgnore,                    /* Ignore this many fields at the end */
        76085  +  int nKeyCol,                    /* Only compare this many fields */
 75463  76086     int *pRes                       /* OUT: Result of comparison */
 75464  76087   ){
 75465  76088     VdbeSorter *pSorter = pCsr->pSorter;
 75466  76089     void *pKey; int nKey;           /* Sorter key to compare pVal with */
 75467  76090   
 75468  76091     pKey = vdbeSorterRowkey(pSorter, &nKey);
 75469         -  vdbeSorterCompare(pCsr, nIgnore, pVal->z, pVal->n, pKey, nKey, pRes);
        76092  +  vdbeSorterCompare(pCsr, nKeyCol, pVal->z, pVal->n, pKey, nKey, pRes);
 75470  76093     return SQLITE_OK;
 75471  76094   }
 75472  76095   
 75473  76096   /************** End of vdbesort.c ********************************************/
 75474  76097   /************** Begin file journal.c *****************************************/
 75475  76098   /*
 75476  76099   ** 2007 August 22
................................................................................
 76494  77117               if( iCol==pTab->iPKey ){
 76495  77118                 iCol = -1;
 76496  77119               }
 76497  77120               break;
 76498  77121             }
 76499  77122           }
 76500  77123           if( iCol>=pTab->nCol && sqlite3IsRowid(zCol) && HasRowid(pTab) ){
 76501         -          /* IMP: R-24309-18625 */
        77124  +          /* IMP: R-51414-32910 */
 76502  77125             /* IMP: R-44911-55124 */
 76503  77126             iCol = -1;
 76504  77127           }
 76505  77128           if( iCol<pTab->nCol ){
 76506  77129             cnt++;
 76507  77130             if( iCol<0 ){
 76508  77131               pExpr->affinity = SQLITE_AFF_INTEGER;
................................................................................
 76850  77473                                         "constant between 0.0 and 1.0");
 76851  77474                 pNC->nErr++;
 76852  77475               }
 76853  77476             }else{
 76854  77477               /* EVIDENCE-OF: R-61304-29449 The unlikely(X) function is equivalent to
 76855  77478               ** likelihood(X, 0.0625).
 76856  77479               ** EVIDENCE-OF: R-01283-11636 The unlikely(X) function is short-hand for
 76857         -            ** likelihood(X,0.0625). */
 76858         -            pExpr->iTable = 62;  /* TUNING:  Default 2nd arg to unlikely() is 0.0625 */
        77480  +            ** likelihood(X,0.0625).
        77481  +            ** EVIDENCE-OF: R-36850-34127 The likely(X) function is short-hand for
        77482  +            ** likelihood(X,0.9375).
        77483  +            ** EVIDENCE-OF: R-53436-40973 The likely(X) function is equivalent to
        77484  +            ** likelihood(X,0.9375). */
        77485  +            /* TUNING: unlikely() probability is 0.0625.  likely() is 0.9375 */
        77486  +            pExpr->iTable = pDef->zName[0]=='u' ? 62 : 938;
 76859  77487             }             
 76860  77488           }
 76861  77489         }
 76862  77490   #ifndef SQLITE_OMIT_AUTHORIZATION
 76863  77491         if( pDef ){
 76864  77492           auth = sqlite3AuthCheck(pParse, SQLITE_FUNCTION, 0, pDef->zName, 0);
 76865  77493           if( auth!=SQLITE_OK ){
................................................................................
 77627  78255   ** SELECT * FROM t1 WHERE a;
 77628  78256   ** SELECT a AS b FROM t1 WHERE b;
 77629  78257   ** SELECT * FROM t1 WHERE (select a from t1);
 77630  78258   */
 77631  78259   SQLITE_PRIVATE char sqlite3ExprAffinity(Expr *pExpr){
 77632  78260     int op;
 77633  78261     pExpr = sqlite3ExprSkipCollate(pExpr);
 77634         -  if( pExpr->flags & EP_Generic ) return SQLITE_AFF_NONE;
        78262  +  if( pExpr->flags & EP_Generic ) return 0;
 77635  78263     op = pExpr->op;
 77636  78264     if( op==TK_SELECT ){
 77637  78265       assert( pExpr->flags&EP_xIsSelect );
 77638  78266       return sqlite3ExprAffinity(pExpr->x.pSelect->pEList->a[0].pExpr);
 77639  78267     }
 77640  78268   #ifndef SQLITE_OMIT_CAST
 77641  78269     if( op==TK_CAST ){
................................................................................
 78962  79590     if( op==TK_REGISTER ) op = p->op2;
 78963  79591     switch( op ){
 78964  79592       case TK_INTEGER:
 78965  79593       case TK_STRING:
 78966  79594       case TK_FLOAT:
 78967  79595       case TK_BLOB:
 78968  79596         return 0;
        79597  +    case TK_COLUMN:
        79598  +      assert( p->pTab!=0 );
        79599  +      return p->iColumn>=0 && p->pTab->aCol[p->iColumn].notNull==0;
 78969  79600       default:
 78970  79601         return 1;
 78971  79602     }
 78972  79603   }
 78973  79604   
 78974  79605   /*
 78975  79606   ** Return TRUE if the given expression is a constant which would be
................................................................................
 79069  79700   ** address of the new instruction.
 79070  79701   */
 79071  79702   SQLITE_PRIVATE int sqlite3CodeOnce(Parse *pParse){
 79072  79703     Vdbe *v = sqlite3GetVdbe(pParse);      /* Virtual machine being coded */
 79073  79704     return sqlite3VdbeAddOp1(v, OP_Once, pParse->nOnce++);
 79074  79705   }
 79075  79706   
        79707  +/*
        79708  +** Generate code that checks the left-most column of index table iCur to see if
        79709  +** it contains any NULL entries.  Cause the register at regHasNull to be set
        79710  +** to a non-NULL value if iCur contains no NULLs.  Cause register regHasNull
        79711  +** to be set to NULL if iCur contains one or more NULL values.
        79712  +*/
        79713  +static void sqlite3SetHasNullFlag(Vdbe *v, int iCur, int regHasNull){
        79714  +  int j1;
        79715  +  sqlite3VdbeAddOp2(v, OP_Integer, 0, regHasNull);
        79716  +  j1 = sqlite3VdbeAddOp1(v, OP_Rewind, iCur); VdbeCoverage(v);
        79717  +  sqlite3VdbeAddOp3(v, OP_Column, iCur, 0, regHasNull);
        79718  +  sqlite3VdbeChangeP5(v, OPFLAG_TYPEOFARG);
        79719  +  VdbeComment((v, "first_entry_in(%d)", iCur));
        79720  +  sqlite3VdbeJumpHere(v, j1);
        79721  +}
        79722  +
        79723  +
        79724  +#ifndef SQLITE_OMIT_SUBQUERY
        79725  +/*
        79726  +** The argument is an IN operator with a list (not a subquery) on the 
        79727  +** right-hand side.  Return TRUE if that list is constant.
        79728  +*/
        79729  +static int sqlite3InRhsIsConstant(Expr *pIn){
        79730  +  Expr *pLHS;
        79731  +  int res;
        79732  +  assert( !ExprHasProperty(pIn, EP_xIsSelect) );
        79733  +  pLHS = pIn->pLeft;
        79734  +  pIn->pLeft = 0;
        79735  +  res = sqlite3ExprIsConstant(pIn);
        79736  +  pIn->pLeft = pLHS;
        79737  +  return res;
        79738  +}
        79739  +#endif
        79740  +
 79076  79741   /*
 79077  79742   ** This function is used by the implementation of the IN (...) operator.
 79078  79743   ** The pX parameter is the expression on the RHS of the IN operator, which
 79079  79744   ** might be either a list of expressions or a subquery.
 79080  79745   **
 79081  79746   ** The job of this routine is to find or create a b-tree object that can
 79082  79747   ** be used either to test for membership in the RHS set or to iterate through
 79083  79748   ** all members of the RHS set, skipping duplicates.
 79084  79749   **
 79085         -** A cursor is opened on the b-tree object that the RHS of the IN operator
        79750  +** A cursor is opened on the b-tree object that is the RHS of the IN operator
 79086  79751   ** and pX->iTable is set to the index of that cursor.
 79087  79752   **
 79088  79753   ** The returned value of this function indicates the b-tree type, as follows:
 79089  79754   **
 79090  79755   **   IN_INDEX_ROWID      - The cursor was opened on a database table.
 79091  79756   **   IN_INDEX_INDEX_ASC  - The cursor was opened on an ascending index.
 79092  79757   **   IN_INDEX_INDEX_DESC - The cursor was opened on a descending index.
 79093  79758   **   IN_INDEX_EPH        - The cursor was opened on a specially created and
 79094  79759   **                         populated epheremal table.
        79760  +**   IN_INDEX_NOOP       - No cursor was allocated.  The IN operator must be
        79761  +**                         implemented as a sequence of comparisons.
 79095  79762   **
 79096  79763   ** An existing b-tree might be used if the RHS expression pX is a simple
 79097  79764   ** subquery such as:
 79098  79765   **
 79099  79766   **     SELECT <column> FROM <table>
 79100  79767   **
 79101  79768   ** If the RHS of the IN operator is a list or a more complex subquery, then
 79102  79769   ** an ephemeral table might need to be generated from the RHS and then
 79103  79770   ** pX->iTable made to point to the ephermeral table instead of an
 79104         -** existing table.  
        79771  +** existing table.
 79105  79772   **
 79106         -** If the prNotFound parameter is 0, then the b-tree will be used to iterate
 79107         -** through the set members, skipping any duplicates. In this case an
 79108         -** epheremal table must be used unless the selected <column> is guaranteed
        79773  +** The inFlags parameter must contain exactly one of the bits
        79774  +** IN_INDEX_MEMBERSHIP or IN_INDEX_LOOP.  If inFlags contains
        79775  +** IN_INDEX_MEMBERSHIP, then the generated table will be used for a
        79776  +** fast membership test.  When the IN_INDEX_LOOP bit is set, the
        79777  +** IN index will be used to loop over all values of the RHS of the
        79778  +** IN operator.
        79779  +**
        79780  +** When IN_INDEX_LOOP is used (and the b-tree will be used to iterate
        79781  +** through the set members) then the b-tree must not contain duplicates.
        79782  +** An epheremal table must be used unless the selected <column> is guaranteed
 79109  79783   ** to be unique - either because it is an INTEGER PRIMARY KEY or it
 79110  79784   ** has a UNIQUE constraint or UNIQUE index.
 79111  79785   **
 79112         -** If the prNotFound parameter is not 0, then the b-tree will be used 
 79113         -** for fast set membership tests. In this case an epheremal table must 
        79786  +** When IN_INDEX_MEMBERSHIP is used (and the b-tree will be used 
        79787  +** for fast set membership tests) then an epheremal table must 
 79114  79788   ** be used unless <column> is an INTEGER PRIMARY KEY or an index can 
 79115  79789   ** be found with <column> as its left-most column.
        79790  +**
        79791  +** If the IN_INDEX_NOOP_OK and IN_INDEX_MEMBERSHIP are both set and
        79792  +** if the RHS of the IN operator is a list (not a subquery) then this
        79793  +** routine might decide that creating an ephemeral b-tree for membership
        79794  +** testing is too expensive and return IN_INDEX_NOOP.  In that case, the
        79795  +** calling routine should implement the IN operator using a sequence
        79796  +** of Eq or Ne comparison operations.
 79116  79797   **
 79117  79798   ** When the b-tree is being used for membership tests, the calling function
 79118         -** needs to know whether or not the structure contains an SQL NULL 
 79119         -** value in order to correctly evaluate expressions like "X IN (Y, Z)".
 79120         -** If there is any chance that the (...) might contain a NULL value at
        79799  +** might need to know whether or not the RHS side of the IN operator
        79800  +** contains a NULL.  If prRhsHasNull is not a NULL pointer and 
        79801  +** if there is any chance that the (...) might contain a NULL value at
 79121  79802   ** runtime, then a register is allocated and the register number written
 79122         -** to *prNotFound. If there is no chance that the (...) contains a
 79123         -** NULL value, then *prNotFound is left unchanged.
        79803  +** to *prRhsHasNull. If there is no chance that the (...) contains a
        79804  +** NULL value, then *prRhsHasNull is left unchanged.
 79124  79805   **
 79125         -** If a register is allocated and its location stored in *prNotFound, then
 79126         -** its initial value is NULL.  If the (...) does not remain constant
 79127         -** for the duration of the query (i.e. the SELECT within the (...)
 79128         -** is a correlated subquery) then the value of the allocated register is
 79129         -** reset to NULL each time the subquery is rerun. This allows the
 79130         -** caller to use vdbe code equivalent to the following:
 79131         -**
 79132         -**   if( register==NULL ){
 79133         -**     has_null = <test if data structure contains null>
 79134         -**     register = 1
 79135         -**   }
 79136         -**
 79137         -** in order to avoid running the <test if data structure contains null>
 79138         -** test more often than is necessary.
        79806  +** If a register is allocated and its location stored in *prRhsHasNull, then
        79807  +** the value in that register will be NULL if the b-tree contains one or more
        79808  +** NULL values, and it will be some non-NULL value if the b-tree contains no
        79809  +** NULL values.
 79139  79810   */
 79140  79811   #ifndef SQLITE_OMIT_SUBQUERY
 79141         -SQLITE_PRIVATE int sqlite3FindInIndex(Parse *pParse, Expr *pX, int *prNotFound){
        79812  +SQLITE_PRIVATE int sqlite3FindInIndex(Parse *pParse, Expr *pX, u32 inFlags, int *prRhsHasNull){
 79142  79813     Select *p;                            /* SELECT to the right of IN operator */
 79143  79814     int eType = 0;                        /* Type of RHS table. IN_INDEX_* */
 79144  79815     int iTab = pParse->nTab++;            /* Cursor of the RHS table */
 79145         -  int mustBeUnique = (prNotFound==0);   /* True if RHS must be unique */
        79816  +  int mustBeUnique;                     /* True if RHS must be unique */
 79146  79817     Vdbe *v = sqlite3GetVdbe(pParse);     /* Virtual machine being coded */
 79147  79818   
 79148  79819     assert( pX->op==TK_IN );
        79820  +  mustBeUnique = (inFlags & IN_INDEX_LOOP)!=0;
 79149  79821   
 79150  79822     /* Check to see if an existing table or index can be used to
 79151  79823     ** satisfy the query.  This is preferable to generating a new 
 79152  79824     ** ephemeral table.
 79153  79825     */
 79154  79826     p = (ExprHasProperty(pX, EP_xIsSelect) ? pX->x.pSelect : 0);
 79155  79827     if( ALWAYS(pParse->nErr==0) && isCandidateForInOpt(p) ){
................................................................................
 79198  79870         ** it is not, it is not possible to use any index.
 79199  79871         */
 79200  79872         int affinity_ok = sqlite3IndexAffinityOk(pX, pTab->aCol[iCol].affinity);
 79201  79873   
 79202  79874         for(pIdx=pTab->pIndex; pIdx && eType==0 && affinity_ok; pIdx=pIdx->pNext){
 79203  79875           if( (pIdx->aiColumn[0]==iCol)
 79204  79876            && sqlite3FindCollSeq(db, ENC(db), pIdx->azColl[0], 0)==pReq
 79205         -         && (!mustBeUnique || (pIdx->nKeyCol==1 && pIdx->onError!=OE_None))
        79877  +         && (!mustBeUnique || (pIdx->nKeyCol==1 && IsUniqueIndex(pIdx)))
 79206  79878           ){
 79207  79879             int iAddr = sqlite3CodeOnce(pParse); VdbeCoverage(v);
 79208  79880             sqlite3VdbeAddOp3(v, OP_OpenRead, iTab, pIdx->tnum, iDb);
 79209  79881             sqlite3VdbeSetP4KeyInfo(pParse, pIdx);
 79210  79882             VdbeComment((v, "%s", pIdx->zName));
 79211  79883             assert( IN_INDEX_INDEX_DESC == IN_INDEX_INDEX_ASC+1 );
 79212  79884             eType = IN_INDEX_INDEX_ASC + pIdx->aSortOrder[0];
 79213  79885   
 79214         -          if( prNotFound && !pTab->aCol[iCol].notNull ){
 79215         -            *prNotFound = ++pParse->nMem;
 79216         -            sqlite3VdbeAddOp2(v, OP_Null, 0, *prNotFound);
        79886  +          if( prRhsHasNull && !pTab->aCol[iCol].notNull ){
        79887  +            *prRhsHasNull = ++pParse->nMem;
        79888  +            sqlite3SetHasNullFlag(v, iTab, *prRhsHasNull);
 79217  79889             }
 79218  79890             sqlite3VdbeJumpHere(v, iAddr);
 79219  79891           }
 79220  79892         }
 79221  79893       }
 79222  79894     }
        79895  +
        79896  +  /* If no preexisting index is available for the IN clause
        79897  +  ** and IN_INDEX_NOOP is an allowed reply
        79898  +  ** and the RHS of the IN operator is a list, not a subquery
        79899  +  ** and the RHS is not contant or has two or fewer terms,
        79900  +  ** then it is not worth creating an ephermeral table to evaluate
        79901  +  ** the IN operator so return IN_INDEX_NOOP.
        79902  +  */
        79903  +  if( eType==0
        79904  +   && (inFlags & IN_INDEX_NOOP_OK)
        79905  +   && !ExprHasProperty(pX, EP_xIsSelect)
        79906  +   && (!sqlite3InRhsIsConstant(pX) || pX->x.pList->nExpr<=2)
        79907  +  ){
        79908  +    eType = IN_INDEX_NOOP;
        79909  +  }
        79910  +     
 79223  79911   
 79224  79912     if( eType==0 ){
 79225         -    /* Could not found an existing table or index to use as the RHS b-tree.
        79913  +    /* Could not find an existing table or index to use as the RHS b-tree.
 79226  79914       ** We will have to generate an ephemeral table to do the job.
 79227  79915       */
 79228  79916       u32 savedNQueryLoop = pParse->nQueryLoop;
 79229  79917       int rMayHaveNull = 0;
 79230  79918       eType = IN_INDEX_EPH;
 79231         -    if( prNotFound ){
 79232         -      *prNotFound = rMayHaveNull = ++pParse->nMem;
 79233         -      sqlite3VdbeAddOp2(v, OP_Null, 0, *prNotFound);
 79234         -    }else{
        79919  +    if( inFlags & IN_INDEX_LOOP ){
 79235  79920         pParse->nQueryLoop = 0;
 79236  79921         if( pX->pLeft->iColumn<0 && !ExprHasProperty(pX, EP_xIsSelect) ){
 79237  79922           eType = IN_INDEX_ROWID;
 79238  79923         }
        79924  +    }else if( prRhsHasNull ){
        79925  +      *prRhsHasNull = rMayHaveNull = ++pParse->nMem;
 79239  79926       }
 79240  79927       sqlite3CodeSubselect(pParse, pX, rMayHaveNull, eType==IN_INDEX_ROWID);
 79241  79928       pParse->nQueryLoop = savedNQueryLoop;
 79242  79929     }else{
 79243  79930       pX->iTable = iTab;
 79244  79931     }
 79245  79932     return eType;
................................................................................
 79262  79949   ** to be of the form "<rowid> IN (?, ?, ?)", where <rowid> is a reference
 79263  79950   ** to some integer key column of a table B-Tree. In this case, use an
 79264  79951   ** intkey B-Tree to store the set of IN(...) values instead of the usual
 79265  79952   ** (slower) variable length keys B-Tree.
 79266  79953   **
 79267  79954   ** If rMayHaveNull is non-zero, that means that the operation is an IN
 79268  79955   ** (not a SELECT or EXISTS) and that the RHS might contains NULLs.
 79269         -** Furthermore, the IN is in a WHERE clause and that we really want
 79270         -** to iterate over the RHS of the IN operator in order to quickly locate
 79271         -** all corresponding LHS elements.  All this routine does is initialize
 79272         -** the register given by rMayHaveNull to NULL.  Calling routines will take
 79273         -** care of changing this register value to non-NULL if the RHS is NULL-free.
 79274         -**
 79275         -** If rMayHaveNull is zero, that means that the subquery is being used
 79276         -** for membership testing only.  There is no need to initialize any
 79277         -** registers to indicate the presence or absence of NULLs on the RHS.
        79956  +** All this routine does is initialize the register given by rMayHaveNull
        79957  +** to NULL.  Calling routines will take care of changing this register
        79958  +** value to non-NULL if the RHS is NULL-free.
 79278  79959   **
 79279  79960   ** For a SELECT or EXISTS operator, return the register that holds the
 79280  79961   ** result.  For IN operators or if an error occurs, the return value is 0.
 79281  79962   */
 79282  79963   #ifndef SQLITE_OMIT_SUBQUERY
 79283  79964   SQLITE_PRIVATE int sqlite3CodeSubselect(
 79284  79965     Parse *pParse,          /* Parsing context */
 79285  79966     Expr *pExpr,            /* The IN, SELECT, or EXISTS operator */
 79286         -  int rMayHaveNull,       /* Register that records whether NULLs exist in RHS */
        79967  +  int rHasNullFlag,       /* Register that records whether NULLs exist in RHS */
 79287  79968     int isRowid             /* If true, LHS of IN operator is a rowid */
 79288  79969   ){
 79289         -  int testAddr = -1;                      /* One-time test address */
        79970  +  int jmpIfDynamic = -1;                      /* One-time test address */
 79290  79971     int rReg = 0;                           /* Register storing resulting */
 79291  79972     Vdbe *v = sqlite3GetVdbe(pParse);
 79292  79973     if( NEVER(v==0) ) return 0;
 79293  79974     sqlite3ExprCachePush(pParse);
 79294  79975   
 79295  79976     /* This code must be run in its entirety every time it is encountered
 79296  79977     ** if any of the following is true:
................................................................................
 79299  79980     **    *  The right-hand side is an expression list containing variables
 79300  79981     **    *  We are inside a trigger
 79301  79982     **
 79302  79983     ** If all of the above are false, then we can run this code just once
 79303  79984     ** save the results, and reuse the same result on subsequent invocations.
 79304  79985     */
 79305  79986     if( !ExprHasProperty(pExpr, EP_VarSelect) ){
 79306         -    testAddr = sqlite3CodeOnce(pParse); VdbeCoverage(v);
        79987  +    jmpIfDynamic = sqlite3CodeOnce(pParse); VdbeCoverage(v);
 79307  79988     }
 79308  79989   
 79309  79990   #ifndef SQLITE_OMIT_EXPLAIN
 79310  79991     if( pParse->explain==2 ){
 79311  79992       char *zMsg = sqlite3MPrintf(
 79312         -        pParse->db, "EXECUTE %s%s SUBQUERY %d", testAddr>=0?"":"CORRELATED ",
        79993  +        pParse->db, "EXECUTE %s%s SUBQUERY %d", jmpIfDynamic>=0?"":"CORRELATED ",
 79313  79994           pExpr->op==TK_IN?"LIST":"SCALAR", pParse->iNextSelectId
 79314  79995       );
 79315  79996       sqlite3VdbeAddOp4(v, OP_Explain, pParse->iSelectId, 0, 0, zMsg, P4_DYNAMIC);
 79316  79997     }
 79317  79998   #endif
 79318  79999   
 79319  80000     switch( pExpr->op ){
 79320  80001       case TK_IN: {
 79321  80002         char affinity;              /* Affinity of the LHS of the IN */
 79322  80003         int addr;                   /* Address of OP_OpenEphemeral instruction */
 79323  80004         Expr *pLeft = pExpr->pLeft; /* the LHS of the IN operator */
 79324  80005         KeyInfo *pKeyInfo = 0;      /* Key information */
 79325  80006   
 79326         -      if( rMayHaveNull ){
 79327         -        sqlite3VdbeAddOp2(v, OP_Null, 0, rMayHaveNull);
 79328         -      }
 79329         -
 79330  80007         affinity = sqlite3ExprAffinity(pLeft);
 79331  80008   
 79332  80009         /* Whether this is an 'x IN(SELECT...)' or an 'x IN(<exprlist>)'
 79333  80010         ** expression it is handled the same way.  An ephemeral table is 
 79334  80011         ** filled with single-field index keys representing the results
 79335  80012         ** from the SELECT or the <exprlist>.
 79336  80013         **
................................................................................
 79348  80025   
 79349  80026         if( ExprHasProperty(pExpr, EP_xIsSelect) ){
 79350  80027           /* Case 1:     expr IN (SELECT ...)
 79351  80028           **
 79352  80029           ** Generate code to write the results of the select into the temporary
 79353  80030           ** table allocated and opened above.
 79354  80031           */
        80032  +        Select *pSelect = pExpr->x.pSelect;
 79355  80033           SelectDest dest;
 79356  80034           ExprList *pEList;
 79357  80035   
 79358  80036           assert( !isRowid );
 79359  80037           sqlite3SelectDestInit(&dest, SRT_Set, pExpr->iTable);
 79360  80038           dest.affSdst = (u8)affinity;
 79361  80039           assert( (pExpr->iTable&0x0000FFFF)==pExpr->iTable );
 79362         -        pExpr->x.pSelect->iLimit = 0;
        80040  +        pSelect->iLimit = 0;
        80041  +        testcase( pSelect->selFlags & SF_Distinct );
        80042  +        pSelect->selFlags &= ~SF_Distinct;
 79363  80043           testcase( pKeyInfo==0 ); /* Caused by OOM in sqlite3KeyInfoAlloc() */
 79364         -        if( sqlite3Select(pParse, pExpr->x.pSelect, &dest) ){
        80044  +        if( sqlite3Select(pParse, pSelect, &dest) ){
 79365  80045             sqlite3KeyInfoUnref(pKeyInfo);
 79366  80046             return 0;
 79367  80047           }
 79368         -        pEList = pExpr->x.pSelect->pEList;
        80048  +        pEList = pSelect->pEList;
 79369  80049           assert( pKeyInfo!=0 ); /* OOM will cause exit after sqlite3Select() */
 79370  80050           assert( pEList!=0 );
 79371  80051           assert( pEList->nExpr>0 );
 79372  80052           assert( sqlite3KeyInfoIsWriteable(pKeyInfo) );
 79373  80053           pKeyInfo->aColl[0] = sqlite3BinaryCompareCollSeq(pParse, pExpr->pLeft,
 79374  80054                                                            pEList->a[0].pExpr);
 79375  80055         }else if( ALWAYS(pExpr->x.pList!=0) ){
................................................................................
 79392  80072             assert( sqlite3KeyInfoIsWriteable(pKeyInfo) );
 79393  80073             pKeyInfo->aColl[0] = sqlite3ExprCollSeq(pParse, pExpr->pLeft);
 79394  80074           }
 79395  80075   
 79396  80076           /* Loop through each expression in <exprlist>. */
 79397  80077           r1 = sqlite3GetTempReg(pParse);
 79398  80078           r2 = sqlite3GetTempReg(pParse);
 79399         -        sqlite3VdbeAddOp2(v, OP_Null, 0, r2);
        80079  +        if( isRowid ) sqlite3VdbeAddOp2(v, OP_Null, 0, r2);
 79400  80080           for(i=pList->nExpr, pItem=pList->a; i>0; i--, pItem++){
 79401  80081             Expr *pE2 = pItem->pExpr;
 79402  80082             int iValToIns;
 79403  80083   
 79404  80084             /* If the expression is not constant then we will need to
 79405  80085             ** disable the test that was generated above that makes sure
 79406  80086             ** this code only executes once.  Because for a non-constant
 79407  80087             ** expression we need to rerun this code each time.
 79408  80088             */
 79409         -          if( testAddr>=0 && !sqlite3ExprIsConstant(pE2) ){
 79410         -            sqlite3VdbeChangeToNoop(v, testAddr);
 79411         -            testAddr = -1;
        80089  +          if( jmpIfDynamic>=0 && !sqlite3ExprIsConstant(pE2) ){
        80090  +            sqlite3VdbeChangeToNoop(v, jmpIfDynamic);
        80091  +            jmpIfDynamic = -1;
 79412  80092             }
 79413  80093   
 79414  80094             /* Evaluate the expression and insert it into the temp table */
 79415  80095             if( isRowid && sqlite3ExprIsInteger(pE2, &iValToIns) ){
 79416  80096               sqlite3VdbeAddOp3(v, OP_InsertInt, pExpr->iTable, r2, iValToIns);
 79417  80097             }else{
 79418  80098               r3 = sqlite3ExprCodeTarget(pParse, pE2, r1);
................................................................................
 79474  80154         }
 79475  80155         rReg = dest.iSDParm;
 79476  80156         ExprSetVVAProperty(pExpr, EP_NoReduce);
 79477  80157         break;
 79478  80158       }
 79479  80159     }
 79480  80160   
 79481         -  if( testAddr>=0 ){
 79482         -    sqlite3VdbeJumpHere(v, testAddr);
        80161  +  if( rHasNullFlag ){
        80162  +    sqlite3SetHasNullFlag(v, pExpr->iTable, rHasNullFlag);
        80163  +  }
        80164  +
        80165  +  if( jmpIfDynamic>=0 ){
        80166  +    sqlite3VdbeJumpHere(v, jmpIfDynamic);
 79483  80167     }
 79484  80168     sqlite3ExprCachePop(pParse);
 79485  80169   
 79486  80170     return rReg;
 79487  80171   }
 79488  80172   #endif /* SQLITE_OMIT_SUBQUERY */
 79489  80173   
................................................................................
 79496  80180   **
 79497  80181   ** The left-hand side (LHS) is a scalar expression.  The right-hand side (RHS)
 79498  80182   ** is an array of zero or more values.  The expression is true if the LHS is
 79499  80183   ** contained within the RHS.  The value of the expression is unknown (NULL)
 79500  80184   ** if the LHS is NULL or if the LHS is not contained within the RHS and the
 79501  80185   ** RHS contains one or more NULL values.
 79502  80186   **
 79503         -** This routine generates code will jump to destIfFalse if the LHS is not 
        80187  +** This routine generates code that jumps to destIfFalse if the LHS is not 
 79504  80188   ** contained within the RHS.  If due to NULLs we cannot determine if the LHS
 79505  80189   ** is contained in the RHS then jump to destIfNull.  If the LHS is contained
 79506  80190   ** within the RHS then fall through.
 79507  80191   */
 79508  80192   static void sqlite3ExprCodeIN(
 79509  80193     Parse *pParse,        /* Parsing and code generating context */
 79510  80194     Expr *pExpr,          /* The IN expression */
................................................................................
 79519  80203   
 79520  80204     /* Compute the RHS.   After this step, the table with cursor
 79521  80205     ** pExpr->iTable will contains the values that make up the RHS.
 79522  80206     */
 79523  80207     v = pParse->pVdbe;
 79524  80208     assert( v!=0 );       /* OOM detected prior to this routine */
 79525  80209     VdbeNoopComment((v, "begin IN expr"));
 79526         -  eType = sqlite3FindInIndex(pParse, pExpr, &rRhsHasNull);
        80210  +  eType = sqlite3FindInIndex(pParse, pExpr,
        80211  +                             IN_INDEX_MEMBERSHIP | IN_INDEX_NOOP_OK,
        80212  +                             destIfFalse==destIfNull ? 0 : &rRhsHasNull);
 79527  80213   
 79528  80214     /* Figure out the affinity to use to create a key from the results
 79529  80215     ** of the expression. affinityStr stores a static string suitable for
 79530  80216     ** P4 of OP_MakeRecord.
 79531  80217     */
 79532  80218     affinity = comparisonAffinity(pExpr);
 79533  80219   
 79534  80220     /* Code the LHS, the <expr> from "<expr> IN (...)".
 79535  80221     */
 79536  80222     sqlite3ExprCachePush(pParse);
 79537  80223     r1 = sqlite3GetTempReg(pParse);
 79538  80224     sqlite3ExprCode(pParse, pExpr->pLeft, r1);
 79539  80225   
 79540         -  /* If the LHS is NULL, then the result is either false or NULL depending
 79541         -  ** on whether the RHS is empty or not, respectively.
 79542         -  */
 79543         -  if( destIfNull==destIfFalse ){
 79544         -    /* Shortcut for the common case where the false and NULL outcomes are
 79545         -    ** the same. */
 79546         -    sqlite3VdbeAddOp2(v, OP_IsNull, r1, destIfNull); VdbeCoverage(v);
 79547         -  }else{
 79548         -    int addr1 = sqlite3VdbeAddOp1(v, OP_NotNull, r1); VdbeCoverage(v);
 79549         -    sqlite3VdbeAddOp2(v, OP_Rewind, pExpr->iTable, destIfFalse);
 79550         -    VdbeCoverage(v);
 79551         -    sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfNull);
 79552         -    sqlite3VdbeJumpHere(v, addr1);
 79553         -  }
 79554         -
 79555         -  if( eType==IN_INDEX_ROWID ){
 79556         -    /* In this case, the RHS is the ROWID of table b-tree
 79557         -    */
 79558         -    sqlite3VdbeAddOp2(v, OP_MustBeInt, r1, destIfFalse); VdbeCoverage(v);
 79559         -    sqlite3VdbeAddOp3(v, OP_NotExists, pExpr->iTable, destIfFalse, r1);
 79560         -    VdbeCoverage(v);
 79561         -  }else{
 79562         -    /* In this case, the RHS is an index b-tree.
 79563         -    */
 79564         -    sqlite3VdbeAddOp4(v, OP_Affinity, r1, 1, 0, &affinity, 1);
 79565         -
 79566         -    /* If the set membership test fails, then the result of the 
 79567         -    ** "x IN (...)" expression must be either 0 or NULL. If the set
 79568         -    ** contains no NULL values, then the result is 0. If the set 
 79569         -    ** contains one or more NULL values, then the result of the
 79570         -    ** expression is also NULL.
 79571         -    */
 79572         -    if( rRhsHasNull==0 || destIfFalse==destIfNull ){
 79573         -      /* This branch runs if it is known at compile time that the RHS
 79574         -      ** cannot contain NULL values. This happens as the result
 79575         -      ** of a "NOT NULL" constraint in the database schema.
 79576         -      **
 79577         -      ** Also run this branch if NULL is equivalent to FALSE
 79578         -      ** for this particular IN operator.
 79579         -      */
 79580         -      sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable, destIfFalse, r1, 1);
 79581         -      VdbeCoverage(v);
 79582         -    }else{
 79583         -      /* In this branch, the RHS of the IN might contain a NULL and
 79584         -      ** the presence of a NULL on the RHS makes a difference in the
 79585         -      ** outcome.
 79586         -      */
 79587         -      int j1, j2;
 79588         -
 79589         -      /* First check to see if the LHS is contained in the RHS.  If so,
 79590         -      ** then the presence of NULLs in the RHS does not matter, so jump
 79591         -      ** over all of the code that follows.
 79592         -      */
 79593         -      j1 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, r1, 1);
 79594         -      VdbeCoverage(v);
 79595         -
 79596         -      /* Here we begin generating code that runs if the LHS is not
 79597         -      ** contained within the RHS.  Generate additional code that
 79598         -      ** tests the RHS for NULLs.  If the RHS contains a NULL then
 79599         -      ** jump to destIfNull.  If there are no NULLs in the RHS then
 79600         -      ** jump to destIfFalse.
 79601         -      */
 79602         -      sqlite3VdbeAddOp2(v, OP_If, rRhsHasNull, destIfNull); VdbeCoverage(v);
 79603         -      sqlite3VdbeAddOp2(v, OP_IfNot, rRhsHasNull, destIfFalse); VdbeCoverage(v);
 79604         -      j2 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, rRhsHasNull, 1);
 79605         -      VdbeCoverage(v);
 79606         -      sqlite3VdbeAddOp2(v, OP_Integer, 0, rRhsHasNull);
        80226  +  /* If sqlite3FindInIndex() did not find or create an index that is
        80227  +  ** suitable for evaluating the IN operator, then evaluate using a
        80228  +  ** sequence of comparisons.
        80229  +  */
        80230  +  if( eType==IN_INDEX_NOOP ){
        80231  +    ExprList *pList = pExpr->x.pList;
        80232  +    CollSeq *pColl = sqlite3ExprCollSeq(pParse, pExpr->pLeft);
        80233  +    int labelOk = sqlite3VdbeMakeLabel(v);
        80234  +    int r2, regToFree;
        80235  +    int regCkNull = 0;
        80236  +    int ii;
        80237  +    assert( !ExprHasProperty(pExpr, EP_xIsSelect) );
        80238  +    if( destIfNull!=destIfFalse ){
        80239  +      regCkNull = sqlite3GetTempReg(pParse);
        80240  +      sqlite3VdbeAddOp3(v, OP_BitAnd, r1, r1, regCkNull);
        80241  +    }
        80242  +    for(ii=0; ii<pList->nExpr; ii++){
        80243  +      r2 = sqlite3ExprCodeTemp(pParse, pList->a[ii].pExpr, &regToFree);
        80244  +      if( regCkNull && sqlite3ExprCanBeNull(pList->a[ii].pExpr) ){
        80245  +        sqlite3VdbeAddOp3(v, OP_BitAnd, regCkNull, r2, regCkNull);
        80246  +      }
        80247  +      if( ii<pList->nExpr-1 || destIfNull!=destIfFalse ){
        80248  +        sqlite3VdbeAddOp4(v, OP_Eq, r1, labelOk, r2,
        80249  +                          (void*)pColl, P4_COLLSEQ);
        80250  +        VdbeCoverageIf(v, ii<pList->nExpr-1);
        80251  +        VdbeCoverageIf(v, ii==pList->nExpr-1);
        80252  +        sqlite3VdbeChangeP5(v, affinity);
        80253  +      }else{
        80254  +        assert( destIfNull==destIfFalse );
        80255  +        sqlite3VdbeAddOp4(v, OP_Ne, r1, destIfFalse, r2,
        80256  +                          (void*)pColl, P4_COLLSEQ); VdbeCoverage(v);
        80257  +        sqlite3VdbeChangeP5(v, affinity | SQLITE_JUMPIFNULL);
        80258  +      }
        80259  +      sqlite3ReleaseTempReg(pParse, regToFree);
        80260  +    }
        80261  +    if( regCkNull ){
        80262  +      sqlite3VdbeAddOp2(v, OP_IsNull, regCkNull, destIfNull); VdbeCoverage(v);
 79607  80263         sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfFalse);
 79608         -      sqlite3VdbeJumpHere(v, j2);
 79609         -      sqlite3VdbeAddOp2(v, OP_Integer, 1, rRhsHasNull);
 79610         -      sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfNull);
 79611         -
 79612         -      /* The OP_Found at the top of this branch jumps here when true, 
 79613         -      ** causing the overall IN expression evaluation to fall through.
 79614         -      */
 79615         -      sqlite3VdbeJumpHere(v, j1);
        80264  +    }
        80265  +    sqlite3VdbeResolveLabel(v, labelOk);
        80266  +    sqlite3ReleaseTempReg(pParse, regCkNull);
        80267  +  }else{
        80268  +  
        80269  +    /* If the LHS is NULL, then the result is either false or NULL depending
        80270  +    ** on whether the RHS is empty or not, respectively.
        80271  +    */
        80272  +    if( sqlite3ExprCanBeNull(pExpr->pLeft) ){
        80273  +      if( destIfNull==destIfFalse ){
        80274  +        /* Shortcut for the common case where the false and NULL outcomes are
        80275  +        ** the same. */
        80276  +        sqlite3VdbeAddOp2(v, OP_IsNull, r1, destIfNull); VdbeCoverage(v);
        80277  +      }else{
        80278  +        int addr1 = sqlite3VdbeAddOp1(v, OP_NotNull, r1); VdbeCoverage(v);
        80279  +        sqlite3VdbeAddOp2(v, OP_Rewind, pExpr->iTable, destIfFalse);
        80280  +        VdbeCoverage(v);
        80281  +        sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfNull);
        80282  +        sqlite3VdbeJumpHere(v, addr1);
        80283  +      }
        80284  +    }
        80285  +  
        80286  +    if( eType==IN_INDEX_ROWID ){
        80287  +      /* In this case, the RHS is the ROWID of table b-tree
        80288  +      */
        80289  +      sqlite3VdbeAddOp2(v, OP_MustBeInt, r1, destIfFalse); VdbeCoverage(v);
        80290  +      sqlite3VdbeAddOp3(v, OP_NotExists, pExpr->iTable, destIfFalse, r1);
        80291  +      VdbeCoverage(v);
        80292  +    }else{
        80293  +      /* In this case, the RHS is an index b-tree.
        80294  +      */
        80295  +      sqlite3VdbeAddOp4(v, OP_Affinity, r1, 1, 0, &affinity, 1);
        80296  +  
        80297  +      /* If the set membership test fails, then the result of the 
        80298  +      ** "x IN (...)" expression must be either 0 or NULL. If the set
        80299  +      ** contains no NULL values, then the result is 0. If the set 
        80300  +      ** contains one or more NULL values, then the result of the
        80301  +      ** expression is also NULL.
        80302  +      */
        80303  +      assert( destIfFalse!=destIfNull || rRhsHasNull==0 );
        80304  +      if( rRhsHasNull==0 ){
        80305  +        /* This branch runs if it is known at compile time that the RHS
        80306  +        ** cannot contain NULL values. This happens as the result
        80307  +        ** of a "NOT NULL" constraint in the database schema.
        80308  +        **
        80309  +        ** Also run this branch if NULL is equivalent to FALSE
        80310  +        ** for this particular IN operator.
        80311  +        */
        80312  +        sqlite3VdbeAddOp4Int(v, OP_NotFound, pExpr->iTable, destIfFalse, r1, 1);
        80313  +        VdbeCoverage(v);
        80314  +      }else{
        80315  +        /* In this branch, the RHS of the IN might contain a NULL and
        80316  +        ** the presence of a NULL on the RHS makes a difference in the
        80317  +        ** outcome.
        80318  +        */
        80319  +        int j1;
        80320  +  
        80321  +        /* First check to see if the LHS is contained in the RHS.  If so,
        80322  +        ** then the answer is TRUE the presence of NULLs in the RHS does
        80323  +        ** not matter.  If the LHS is not contained in the RHS, then the
        80324  +        ** answer is NULL if the RHS contains NULLs and the answer is
        80325  +        ** FALSE if the RHS is NULL-free.
        80326  +        */
        80327  +        j1 = sqlite3VdbeAddOp4Int(v, OP_Found, pExpr->iTable, 0, r1, 1);
        80328  +        VdbeCoverage(v);
        80329  +        sqlite3VdbeAddOp2(v, OP_IsNull, rRhsHasNull, destIfNull);
        80330  +        VdbeCoverage(v);
        80331  +        sqlite3VdbeAddOp2(v, OP_Goto, 0, destIfFalse);
        80332  +        sqlite3VdbeJumpHere(v, j1);
        80333  +      }
 79616  80334       }
 79617  80335     }
 79618  80336     sqlite3ReleaseTempReg(pParse, r1);
 79619  80337     sqlite3ExprCachePop(pParse);
 79620  80338     VdbeComment((v, "end IN expr"));
 79621  80339   }
 79622  80340   #endif /* SQLITE_OMIT_SUBQUERY */
................................................................................
 79669  80387       if( negFlag ) i = -i;
 79670  80388       sqlite3VdbeAddOp2(v, OP_Integer, i, iMem);
 79671  80389     }else{
 79672  80390       int c;
 79673  80391       i64 value;
 79674  80392       const char *z = pExpr->u.zToken;
 79675  80393       assert( z!=0 );
 79676         -    c = sqlite3Atoi64(z, &value, sqlite3Strlen30(z), SQLITE_UTF8);
        80394  +    c = sqlite3DecOrHexToI64(z, &value);
 79677  80395       if( c==0 || (c==2 && negFlag) ){
 79678  80396         char *zV;
 79679  80397         if( negFlag ){ value = c==2 ? SMALLEST_INT64 : -value; }
 79680  80398         zV = dup8bytes(v, (char*)&value);
 79681  80399         sqlite3VdbeAddOp4(v, OP_Int64, 0, iMem, 0, zV, P4_INT64);
 79682  80400       }else{
 79683  80401   #ifdef SQLITE_OMIT_FLOATING_POINT
 79684  80402         sqlite3ErrorMsg(pParse, "oversized integer: %s%s", negFlag ? "-" : "", z);
 79685  80403   #else
 79686         -      codeReal(v, z, negFlag, iMem);
        80404  +#ifndef SQLITE_OMIT_HEX_INTEGER
        80405  +      if( sqlite3_strnicmp(z,"0x",2)==0 ){
        80406  +        sqlite3ErrorMsg(pParse, "hex literal too big: %s", z);
        80407  +      }else
        80408  +#endif
        80409  +      {
        80410  +        codeReal(v, z, negFlag, iMem);
        80411  +      }
 79687  80412   #endif
 79688  80413       }
 79689  80414     }
 79690  80415   }
 79691  80416   
 79692  80417   /*
 79693  80418   ** Clear a cache entry.
................................................................................
 80225  80950         assert( TK_NOTNULL==OP_NotNull ); testcase( op==TK_NOTNULL );
 80226  80951         sqlite3VdbeAddOp2(v, OP_Integer, 1, target);
 80227  80952         r1 = sqlite3ExprCodeTemp(pParse, pExpr->pLeft, &regFree1);
 80228  80953         testcase( regFree1==0 );
 80229  80954         addr = sqlite3VdbeAddOp1(v, op, r1);
 80230  80955         VdbeCoverageIf(v, op==TK_ISNULL);
 80231  80956         VdbeCoverageIf(v, op==TK_NOTNULL);
 80232         -      sqlite3VdbeAddOp2(v, OP_AddImm, target, -1);
        80957  +      sqlite3VdbeAddOp2(v, OP_Integer, 0, target);
 80233  80958         sqlite3VdbeJumpHere(v, addr);
 80234  80959         break;
 80235  80960       }
 80236  80961       case TK_AGG_FUNCTION: {
 80237  80962         AggInfo *pInfo = pExpr->pAggInfo;
 80238  80963         if( pInfo==0 ){
 80239  80964           assert( !ExprHasProperty(pExpr, EP_IntValue) );
................................................................................
 80261  80986           pFarg = pExpr->x.pList;
 80262  80987         }
 80263  80988         nFarg = pFarg ? pFarg->nExpr : 0;
 80264  80989         assert( !ExprHasProperty(pExpr, EP_IntValue) );
 80265  80990         zId = pExpr->u.zToken;
 80266  80991         nId = sqlite3Strlen30(zId);
 80267  80992         pDef = sqlite3FindFunction(db, zId, nId, nFarg, enc, 0);
 80268         -      if( pDef==0 ){
        80993  +      if( pDef==0 || pDef->xFunc==0 ){
 80269  80994           sqlite3ErrorMsg(pParse, "unknown function: %.*s()", nId, zId);
 80270  80995           break;
 80271  80996         }
 80272  80997   
 80273  80998         /* Attempt a direct implementation of the built-in COALESCE() and
 80274  80999         ** IFNULL() functions.  This avoids unnecessary evalation of
 80275  81000         ** arguments past the first non-NULL argument.
................................................................................
 82927  83652     }
 82928  83653   
 82929  83654     /* Open the sqlite_stat[134] tables for writing. */
 82930  83655     for(i=0; aTable[i].zCols; i++){
 82931  83656       assert( i<ArraySize(aTable) );
 82932  83657       sqlite3VdbeAddOp4Int(v, OP_OpenWrite, iStatCur+i, aRoot[i], iDb, 3);
 82933  83658       sqlite3VdbeChangeP5(v, aCreateTbl[i]);
        83659  +    VdbeComment((v, aTable[i].zName));
 82934  83660     }
 82935  83661   }
 82936  83662   
 82937  83663   /*
 82938  83664   ** Recommended number of samples for sqlite_stat4
 82939  83665   */
 82940  83666   #ifndef SQLITE_STAT4_SAMPLES
................................................................................
 82962  83688     int iCol;                       /* If !isPSample, the reason for inclusion */
 82963  83689     u32 iHash;                      /* Tiebreaker hash */
 82964  83690   #endif
 82965  83691   };                                                    
 82966  83692   struct Stat4Accum {
 82967  83693     tRowcnt nRow;             /* Number of rows in the entire table */
 82968  83694     tRowcnt nPSample;         /* How often to do a periodic sample */
 82969         -  int nCol;                 /* Number of columns in index + rowid */
        83695  +  int nCol;                 /* Number of columns in index + pk/rowid */
        83696  +  int nKeyCol;              /* Number of index columns w/o the pk/rowid */
 82970  83697     int mxSample;             /* Maximum number of samples to accumulate */
 82971  83698     Stat4Sample current;      /* Current row as a Stat4Sample */
 82972  83699     u32 iPrn;                 /* Pseudo-random number used for sampling */
 82973  83700     Stat4Sample *aBest;       /* Array of nCol best samples */
 82974  83701     int iMin;                 /* Index in a[] of entry with minimum score */
 82975  83702     int nSample;              /* Current number of samples */
 82976  83703     int iGet;                 /* Index of current sample accessed by stat_get() */
................................................................................
 83048  83775     for(i=0; i<p->mxSample; i++) sampleClear(p->db, p->a+i);
 83049  83776     sampleClear(p->db, &p->current);
 83050  83777   #endif
 83051  83778     sqlite3DbFree(p->db, p);
 83052  83779   }
 83053  83780   
 83054  83781   /*
 83055         -** Implementation of the stat_init(N,C) SQL function. The two parameters
 83056         -** are the number of rows in the table or index (C) and the number of columns
 83057         -** in the index (N).  The second argument (C) is only used for STAT3 and STAT4.
        83782  +** Implementation of the stat_init(N,K,C) SQL function. The three parameters
        83783  +** are:
        83784  +**     N:    The number of columns in the index including the rowid/pk (note 1)
        83785  +**     K:    The number of columns in the index excluding the rowid/pk.
        83786  +**     C:    The number of rows in the index (note 2)
        83787  +**
        83788  +** Note 1:  In the special case of the covering index that implements a
        83789  +** WITHOUT ROWID table, N is the number of PRIMARY KEY columns, not the
        83790  +** total number of columns in the table.
        83791  +**
        83792  +** Note 2:  C is only used for STAT3 and STAT4.
        83793  +**
        83794  +** For indexes on ordinary rowid tables, N==K+1.  But for indexes on
        83795  +** WITHOUT ROWID tables, N=K+P where P is the number of columns in the
        83796  +** PRIMARY KEY of the table.  The covering index that implements the
        83797  +** original WITHOUT ROWID table as N==K as a special case.
 83058  83798   **
 83059  83799   ** This routine allocates the Stat4Accum object in heap memory. The return 
 83060  83800   ** value is a pointer to the the Stat4Accum object encoded as a blob (i.e. 
 83061  83801   ** the size of the blob is sizeof(void*) bytes). 
 83062  83802   */
 83063  83803   static void statInit(
 83064  83804     sqlite3_context *context,
 83065  83805     int argc,
 83066  83806     sqlite3_value **argv
 83067  83807   ){
 83068  83808     Stat4Accum *p;
 83069  83809     int nCol;                       /* Number of columns in index being sampled */
        83810  +  int nKeyCol;                    /* Number of key columns */
 83070  83811     int nColUp;                     /* nCol rounded up for alignment */
 83071  83812     int n;                          /* Bytes of space to allocate */
 83072  83813     sqlite3 *db;                    /* Database connection */
 83073  83814   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
 83074  83815     int mxSample = SQLITE_STAT4_SAMPLES;
 83075  83816   #endif
 83076  83817   
 83077  83818     /* Decode the three function arguments */
 83078  83819     UNUSED_PARAMETER(argc);
 83079  83820     nCol = sqlite3_value_int(argv[0]);
 83080         -  assert( nCol>1 );               /* >1 because it includes the rowid column */
        83821  +  assert( nCol>0 );
 83081  83822     nColUp = sizeof(tRowcnt)<8 ? (nCol+1)&~1 : nCol;
        83823  +  nKeyCol = sqlite3_value_int(argv[1]);
        83824  +  assert( nKeyCol<=nCol );
        83825  +  assert( nKeyCol>0 );
 83082  83826   
 83083  83827     /* Allocate the space required for the Stat4Accum object */
 83084  83828     n = sizeof(*p) 
 83085  83829       + sizeof(tRowcnt)*nColUp                  /* Stat4Accum.anEq */
 83086  83830       + sizeof(tRowcnt)*nColUp                  /* Stat4Accum.anDLt */
 83087  83831   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
 83088  83832       + sizeof(tRowcnt)*nColUp                  /* Stat4Accum.anLt */
................................................................................
 83096  83840       sqlite3_result_error_nomem(context);
 83097  83841       return;
 83098  83842     }
 83099  83843   
 83100  83844     p->db = db;
 83101  83845     p->nRow = 0;
 83102  83846     p->nCol = nCol;
        83847  +  p->nKeyCol = nKeyCol;
 83103  83848     p->current.anDLt = (tRowcnt*)&p[1];
 83104  83849     p->current.anEq = &p->current.anDLt[nColUp];
 83105  83850   
 83106  83851   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
 83107  83852     {
 83108  83853       u8 *pSpace;                     /* Allocated space not yet assigned */
 83109  83854       int i;                          /* Used to iterate through p->aSample[] */
 83110  83855   
 83111  83856       p->iGet = -1;
 83112  83857       p->mxSample = mxSample;
 83113         -    p->nPSample = (tRowcnt)(sqlite3_value_int64(argv[1])/(mxSample/3+1) + 1);
        83858  +    p->nPSample = (tRowcnt)(sqlite3_value_int64(argv[2])/(mxSample/3+1) + 1);
 83114  83859       p->current.anLt = &p->current.anEq[nColUp];
 83115         -    p->iPrn = nCol*0x689e962d ^ sqlite3_value_int(argv[1])*0xd0944565;
        83860  +    p->iPrn = nCol*0x689e962d ^ sqlite3_value_int(argv[2])*0xd0944565;
 83116  83861     
 83117  83862       /* Set up the Stat4Accum.a[] and aBest[] arrays */
 83118  83863       p->a = (struct Stat4Sample*)&p->current.anLt[nColUp];
 83119  83864       p->aBest = &p->a[mxSample];
 83120  83865       pSpace = (u8*)(&p->a[mxSample+nCol]);
 83121  83866       for(i=0; i<(mxSample+nCol); i++){
 83122  83867         p->a[i].anEq = (tRowcnt *)pSpace; pSpace += (sizeof(tRowcnt) * nColUp);
................................................................................
 83131  83876     }
 83132  83877   #endif
 83133  83878   
 83134  83879     /* Return a pointer to the allocated object to the caller */
 83135  83880     sqlite3_result_blob(context, p, sizeof(p), stat4Destructor);
 83136  83881   }
 83137  83882   static const FuncDef statInitFuncdef = {
 83138         -  1+IsStat34,      /* nArg */
        83883  +  2+IsStat34,      /* nArg */
 83139  83884     SQLITE_UTF8,     /* funcFlags */
 83140  83885     0,               /* pUserData */
 83141  83886     0,               /* pNext */
 83142  83887     statInit,        /* xFunc */
 83143  83888     0,               /* xStep */
 83144  83889     0,               /* xFinalize */
 83145  83890     "stat_init",     /* zName */
................................................................................
 83355  84100   ** Arguments:
 83356  84101   **
 83357  84102   **    P     Pointer to the Stat4Accum object created by stat_init()
 83358  84103   **    C     Index of left-most column to differ from previous row
 83359  84104   **    R     Rowid for the current row.  Might be a key record for
 83360  84105   **          WITHOUT ROWID tables.
 83361  84106   **
 83362         -** The SQL function always returns NULL.
        84107  +** This SQL function always returns NULL.  It's purpose it to accumulate
        84108  +** statistical data and/or samples in the Stat4Accum object about the
        84109  +** index being analyzed.  The stat_get() SQL function will later be used to
        84110  +** extract relevant information for constructing the sqlite_statN tables.
 83363  84111   **
 83364  84112   ** The R parameter is only used for STAT3 and STAT4
 83365  84113   */
 83366  84114   static void statPush(
 83367  84115     sqlite3_context *context,
 83368  84116     int argc,
 83369  84117     sqlite3_value **argv
................................................................................
 83372  84120   
 83373  84121     /* The three function arguments */
 83374  84122     Stat4Accum *p = (Stat4Accum*)sqlite3_value_blob(argv[0]);
 83375  84123     int iChng = sqlite3_value_int(argv[1]);
 83376  84124   
 83377  84125     UNUSED_PARAMETER( argc );
 83378  84126     UNUSED_PARAMETER( context );
 83379         -  assert( p->nCol>1 );        /* Includes rowid field */
        84127  +  assert( p->nCol>0 );
 83380  84128     assert( iChng<p->nCol );
 83381  84129   
 83382  84130     if( p->nRow==0 ){
 83383  84131       /* This is the first call to this function. Do initialization. */
 83384  84132       for(i=0; i<p->nCol; i++) p->current.anEq[i] = 1;
 83385  84133     }else{
 83386  84134       /* Second and subsequent calls get processed here */
................................................................................
 83449  84197   #define STAT_GET_ROWID 1          /* "rowid" column of stat[34] entry */
 83450  84198   #define STAT_GET_NEQ   2          /* "neq" column of stat[34] entry */
 83451  84199   #define STAT_GET_NLT   3          /* "nlt" column of stat[34] entry */
 83452  84200   #define STAT_GET_NDLT  4          /* "ndlt" column of stat[34] entry */
 83453  84201   
 83454  84202   /*
 83455  84203   ** Implementation of the stat_get(P,J) SQL function.  This routine is
 83456         -** used to query the results.  Content is returned for parameter J
        84204  +** used to query statistical information that has been gathered into
        84205  +** the Stat4Accum object by prior calls to stat_push().  The P parameter
        84206  +** is a BLOB which is decoded into a pointer to the Stat4Accum objects.
        84207  +** The content to returned is determined by the parameter J
 83457  84208   ** which is one of the STAT_GET_xxxx values defined above.
 83458  84209   **
 83459  84210   ** If neither STAT3 nor STAT4 are enabled, then J is always
 83460  84211   ** STAT_GET_STAT1 and is hence omitted and this routine becomes
 83461  84212   ** a one-parameter function, stat_get(P), that always returns the
 83462  84213   ** stat1 table entry information.
 83463  84214   */
................................................................................
 83500  84251       ** rows, then each estimate is computed as:
 83501  84252       **
 83502  84253       **        I = (K+D-1)/D
 83503  84254       */
 83504  84255       char *z;
 83505  84256       int i;
 83506  84257   
 83507         -    char *zRet = sqlite3MallocZero(p->nCol * 25);
        84258  +    char *zRet = sqlite3MallocZero( (p->nKeyCol+1)*25 );
 83508  84259       if( zRet==0 ){
 83509  84260         sqlite3_result_error_nomem(context);
 83510  84261         return;
 83511  84262       }
 83512  84263   
 83513  84264       sqlite3_snprintf(24, zRet, "%llu", (u64)p->nRow);
 83514  84265       z = zRet + sqlite3Strlen30(zRet);
 83515         -    for(i=0; i<(p->nCol-1); i++){
        84266  +    for(i=0; i<p->nKeyCol; i++){
 83516  84267         u64 nDistinct = p->current.anDLt[i] + 1;
 83517  84268         u64 iVal = (p->nRow + nDistinct - 1) / nDistinct;
 83518  84269         sqlite3_snprintf(24, z, " %llu", iVal);
 83519  84270         z += sqlite3Strlen30(z);
 83520  84271         assert( p->current.anEq[i] );
 83521  84272       }
 83522  84273       assert( z[0]=='\0' && z>zRet );
................................................................................
 83668  84419     iTabCur = iTab++;
 83669  84420     iIdxCur = iTab++;
 83670  84421     pParse->nTab = MAX(pParse->nTab, iTab);
 83671  84422     sqlite3OpenTable(pParse, iTabCur, iDb, pTab, OP_OpenRead);
 83672  84423     sqlite3VdbeAddOp4(v, OP_String8, 0, regTabname, 0, pTab->zName, 0);
 83673  84424   
 83674  84425     for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
 83675         -    int nCol;                     /* Number of columns indexed by pIdx */
 83676         -    int *aGotoChng;               /* Array of jump instruction addresses */
        84426  +    int nCol;                     /* Number of columns in pIdx. "N" */
 83677  84427       int addrRewind;               /* Address of "OP_Rewind iIdxCur" */
 83678         -    int addrGotoChng0;            /* Address of "Goto addr_chng_0" */
 83679  84428       int addrNextRow;              /* Address of "next_row:" */
 83680  84429       const char *zIdxName;         /* Name of the index */
        84430  +    int nColTest;                 /* Number of columns to test for changes */
 83681  84431   
 83682  84432       if( pOnlyIdx && pOnlyIdx!=pIdx ) continue;
 83683  84433       if( pIdx->pPartIdxWhere==0 ) needTableCnt = 0;
 83684         -    VdbeNoopComment((v, "Begin analysis of %s", pIdx->zName));
 83685         -    nCol = pIdx->nKeyCol;
 83686         -    aGotoChng = sqlite3DbMallocRaw(db, sizeof(int)*(nCol+1));
 83687         -    if( aGotoChng==0 ) continue;
        84434  +    if( !HasRowid(pTab) && IsPrimaryKeyIndex(pIdx) ){
        84435  +      nCol = pIdx->nKeyCol;
        84436  +      zIdxName = pTab->zName;
        84437  +      nColTest = nCol - 1;
        84438  +    }else{
        84439  +      nCol = pIdx->nColumn;
        84440  +      zIdxName = pIdx->zName;
        84441  +      nColTest = pIdx->uniqNotNull ? pIdx->nKeyCol-1 : nCol-1;
        84442  +    }
 83688  84443   
 83689  84444       /* Populate the register containing the index name. */
 83690         -    if( IsPrimaryKeyIndex(pIdx) && !HasRowid(pTab) ){
 83691         -      zIdxName = pTab->zName;
 83692         -    }else{
 83693         -      zIdxName = pIdx->zName;
 83694         -    }
 83695  84445       sqlite3VdbeAddOp4(v, OP_String8, 0, regIdxname, 0, zIdxName, 0);
        84446  +    VdbeComment((v, "Analysis for %s.%s", pTab->zName, zIdxName));
 83696  84447   
 83697  84448       /*
 83698  84449       ** Pseudo-code for loop that calls stat_push():
 83699  84450       **
 83700  84451       **   Rewind csr
 83701  84452       **   if eof(csr) goto end_of_scan;
 83702  84453       **   regChng = 0
................................................................................
 83713  84464       **
 83714  84465       **  chng_addr_0:
 83715  84466       **   regPrev(0) = idx(0)
 83716  84467       **  chng_addr_1:
 83717  84468       **   regPrev(1) = idx(1)
 83718  84469       **  ...
 83719  84470       **
 83720         -    **  chng_addr_N:
        84471  +    **  endDistinctTest:
 83721  84472       **   regRowid = idx(rowid)
 83722  84473       **   stat_push(P, regChng, regRowid)
 83723  84474       **   Next csr
 83724  84475       **   if !eof(csr) goto next_row;
 83725  84476       **
 83726  84477       **  end_of_scan:
 83727  84478       */
 83728  84479   
 83729  84480       /* Make sure there are enough memory cells allocated to accommodate 
 83730  84481       ** the regPrev array and a trailing rowid (the rowid slot is required
 83731  84482       ** when building a record to insert into the sample column of 
 83732  84483       ** the sqlite_stat4 table.  */
 83733         -    pParse->nMem = MAX(pParse->nMem, regPrev+nCol);
        84484  +    pParse->nMem = MAX(pParse->nMem, regPrev+nColTest);
 83734  84485   
 83735  84486       /* Open a read-only cursor on the index being analyzed. */
 83736  84487       assert( iDb==sqlite3SchemaToIndex(db, pIdx->pSchema) );
 83737  84488       sqlite3VdbeAddOp3(v, OP_OpenRead, iIdxCur, pIdx->tnum, iDb);
 83738  84489       sqlite3VdbeSetP4KeyInfo(pParse, pIdx);
 83739  84490       VdbeComment((v, "%s", pIdx->zName));
 83740  84491   
 83741  84492       /* Invoke the stat_init() function. The arguments are:
 83742  84493       ** 
 83743         -    **    (1) the number of columns in the index including the rowid,
 83744         -    **    (2) the number of rows in the index,
        84494  +    **    (1) the number of columns in the index including the rowid
        84495  +    **        (or for a WITHOUT ROWID table, the number of PK columns),
        84496  +    **    (2) the number of columns in the key without the rowid/pk
        84497  +    **    (3) the number of rows in the index,
 83745  84498       **
 83746         -    ** The second argument is only used for STAT3 and STAT4
        84499  +    **
        84500  +    ** The third argument is only used for STAT3 and STAT4
 83747  84501       */
 83748  84502   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
 83749         -    sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regStat4+2);
        84503  +    sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regStat4+3);
 83750  84504   #endif
 83751         -    sqlite3VdbeAddOp2(v, OP_Integer, nCol+1, regStat4+1);
        84505  +    sqlite3VdbeAddOp2(v, OP_Integer, nCol, regStat4+1);
        84506  +    sqlite3VdbeAddOp2(v, OP_Integer, pIdx->nKeyCol, regStat4+2);
 83752  84507       sqlite3VdbeAddOp3(v, OP_Function, 0, regStat4+1, regStat4);
 83753  84508       sqlite3VdbeChangeP4(v, -1, (char*)&statInitFuncdef, P4_FUNCDEF);
 83754         -    sqlite3VdbeChangeP5(v, 1+IsStat34);
        84509  +    sqlite3VdbeChangeP5(v, 2+IsStat34);
 83755  84510   
 83756  84511       /* Implementation of the following:
 83757  84512       **
 83758  84513       **   Rewind csr
 83759  84514       **   if eof(csr) goto end_of_scan;
 83760  84515       **   regChng = 0
 83761  84516       **   goto next_push_0;
 83762  84517       **
 83763  84518       */
 83764  84519       addrRewind = sqlite3VdbeAddOp1(v, OP_Rewind, iIdxCur);
 83765  84520       VdbeCoverage(v);
 83766  84521       sqlite3VdbeAddOp2(v, OP_Integer, 0, regChng);
 83767         -    addrGotoChng0 = sqlite3VdbeAddOp0(v, OP_Goto);
 83768         -
 83769         -    /*
 83770         -    **  next_row:
 83771         -    **   regChng = 0
 83772         -    **   if( idx(0) != regPrev(0) ) goto chng_addr_0
 83773         -    **   regChng = 1
 83774         -    **   if( idx(1) != regPrev(1) ) goto chng_addr_1
 83775         -    **   ...
 83776         -    **   regChng = N
 83777         -    **   goto chng_addr_N
 83778         -    */
 83779  84522       addrNextRow = sqlite3VdbeCurrentAddr(v);
 83780         -    for(i=0; i<nCol; i++){
 83781         -      char *pColl = (char*)sqlite3LocateCollSeq(pParse, pIdx->azColl[i]);
 83782         -      sqlite3VdbeAddOp2(v, OP_Integer, i, regChng);
 83783         -      sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regTemp);
 83784         -      aGotoChng[i] = 
 83785         -      sqlite3VdbeAddOp4(v, OP_Ne, regTemp, 0, regPrev+i, pColl, P4_COLLSEQ);
 83786         -      sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
 83787         -      VdbeCoverage(v);
 83788         -    }
 83789         -    sqlite3VdbeAddOp2(v, OP_Integer, nCol, regChng);
 83790         -    aGotoChng[nCol] = sqlite3VdbeAddOp0(v, OP_Goto);
 83791         -
 83792         -    /*
 83793         -    **  chng_addr_0:
 83794         -    **   regPrev(0) = idx(0)
 83795         -    **  chng_addr_1:
 83796         -    **   regPrev(1) = idx(1)
 83797         -    **  ...
 83798         -    */
 83799         -    sqlite3VdbeJumpHere(v, addrGotoChng0);
 83800         -    for(i=0; i<nCol; i++){
 83801         -      sqlite3VdbeJumpHere(v, aGotoChng[i]);
 83802         -      sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regPrev+i);
 83803         -    }
 83804         -
        84523  +
        84524  +    if( nColTest>0 ){
        84525  +      int endDistinctTest = sqlite3VdbeMakeLabel(v);
        84526  +      int *aGotoChng;               /* Array of jump instruction addresses */
        84527  +      aGotoChng = sqlite3DbMallocRaw(db, sizeof(int)*nColTest);
        84528  +      if( aGotoChng==0 ) continue;
        84529  +
        84530  +      /*
        84531  +      **  next_row:
        84532  +      **   regChng = 0
        84533  +      **   if( idx(0) != regPrev(0) ) goto chng_addr_0
        84534  +      **   regChng = 1
        84535  +      **   if( idx(1) != regPrev(1) ) goto chng_addr_1
        84536  +      **   ...
        84537  +      **   regChng = N
        84538  +      **   goto endDistinctTest
        84539  +      */
        84540  +      sqlite3VdbeAddOp0(v, OP_Goto);
        84541  +      addrNextRow = sqlite3VdbeCurrentAddr(v);
        84542  +      if( nColTest==1 && pIdx->nKeyCol==1 && IsUniqueIndex(pIdx) ){
        84543  +        /* For a single-column UNIQUE index, once we have found a non-NULL
        84544  +        ** row, we know that all the rest will be distinct, so skip 
        84545  +        ** subsequent distinctness tests. */
        84546  +        sqlite3VdbeAddOp2(v, OP_NotNull, regPrev, endDistinctTest);
        84547  +        VdbeCoverage(v);
        84548  +      }
        84549  +      for(i=0; i<nColTest; i++){
        84550  +        char *pColl = (char*)sqlite3LocateCollSeq(pParse, pIdx->azColl[i]);
        84551  +        sqlite3VdbeAddOp2(v, OP_Integer, i, regChng);
        84552  +        sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regTemp);
        84553  +        aGotoChng[i] = 
        84554  +        sqlite3VdbeAddOp4(v, OP_Ne, regTemp, 0, regPrev+i, pColl, P4_COLLSEQ);
        84555  +        sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
        84556  +        VdbeCoverage(v);
        84557  +      }
        84558  +      sqlite3VdbeAddOp2(v, OP_Integer, nColTest, regChng);
        84559  +      sqlite3VdbeAddOp2(v, OP_Goto, 0, endDistinctTest);
        84560  +  
        84561  +  
        84562  +      /*
        84563  +      **  chng_addr_0:
        84564  +      **   regPrev(0) = idx(0)
        84565  +      **  chng_addr_1:
        84566  +      **   regPrev(1) = idx(1)
        84567  +      **  ...
        84568  +      */
        84569  +      sqlite3VdbeJumpHere(v, addrNextRow-1);
        84570  +      for(i=0; i<nColTest; i++){
        84571  +        sqlite3VdbeJumpHere(v, aGotoChng[i]);
        84572  +        sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regPrev+i);
        84573  +      }
        84574  +      sqlite3VdbeResolveLabel(v, endDistinctTest);
        84575  +      sqlite3DbFree(db, aGotoChng);
        84576  +    }
        84577  +  
 83805  84578       /*
 83806  84579       **  chng_addr_N:
 83807  84580       **   regRowid = idx(rowid)            // STAT34 only
 83808  84581       **   stat_push(P, regChng, regRowid)  // 3rd parameter STAT34 only
 83809  84582       **   Next csr
 83810  84583       **   if !eof(csr) goto next_row;
 83811  84584       */
 83812         -    sqlite3VdbeJumpHere(v, aGotoChng[nCol]);
 83813  84585   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
 83814  84586       assert( regRowid==(regStat4+2) );
 83815  84587       if( HasRowid(pTab) ){
 83816  84588         sqlite3VdbeAddOp2(v, OP_IdxRowid, iIdxCur, regRowid);
 83817  84589       }else{
 83818  84590         Index *pPk = sqlite3PrimaryKeyIndex(pIdx->pTable);
 83819  84591         int j, k, regKey;
................................................................................
 83849  84621         int regSample = regStat1+3;
 83850  84622         int regCol = regStat1+4;
 83851  84623         int regSampleRowid = regCol + nCol;
 83852  84624         int addrNext;
 83853  84625         int addrIsNull;
 83854  84626         u8 seekOp = HasRowid(pTab) ? OP_NotExists : OP_NotFound;
 83855  84627   
 83856         -      pParse->nMem = MAX(pParse->nMem, regCol+nCol+1);
        84628  +      pParse->nMem = MAX(pParse->nMem, regCol+nCol);
 83857  84629   
 83858  84630         addrNext = sqlite3VdbeCurrentAddr(v);
 83859  84631         callStatGet(v, regStat4, STAT_GET_ROWID, regSampleRowid);
 83860  84632         addrIsNull = sqlite3VdbeAddOp1(v, OP_IsNull, regSampleRowid);
 83861  84633         VdbeCoverage(v);
 83862  84634         callStatGet(v, regStat4, STAT_GET_NEQ, regEq);
 83863  84635         callStatGet(v, regStat4, STAT_GET_NLT, regLt);
................................................................................
 83871  84643         sqlite3ExprCodeGetColumnOfTable(v, pTab, iTabCur, 
 83872  84644                                         pIdx->aiColumn[0], regSample);
 83873  84645   #else
 83874  84646         for(i=0; i<nCol; i++){
 83875  84647           i16 iCol = pIdx->aiColumn[i];
 83876  84648           sqlite3ExprCodeGetColumnOfTable(v, pTab, iTabCur, iCol, regCol+i);
 83877  84649         }
 83878         -      sqlite3VdbeAddOp3(v, OP_MakeRecord, regCol, nCol+1, regSample);
        84650  +      sqlite3VdbeAddOp3(v, OP_MakeRecord, regCol, nCol, regSample);
 83879  84651   #endif
 83880  84652         sqlite3VdbeAddOp3(v, OP_MakeRecord, regTabname, 6, regTemp);
 83881  84653         sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur+1, regNewRowid);
 83882  84654         sqlite3VdbeAddOp3(v, OP_Insert, iStatCur+1, regTemp, regNewRowid);
 83883  84655         sqlite3VdbeAddOp2(v, OP_Goto, 1, addrNext); /* P1==1 for end-of-loop */
 83884  84656         sqlite3VdbeJumpHere(v, addrIsNull);
 83885  84657       }
 83886  84658   #endif /* SQLITE_ENABLE_STAT3_OR_STAT4 */
 83887  84659   
 83888  84660       /* End of analysis */
 83889  84661       sqlite3VdbeJumpHere(v, addrRewind);
 83890         -    sqlite3DbFree(db, aGotoChng);
 83891  84662     }
 83892  84663   
 83893  84664   
 83894  84665     /* Create a single sqlite_stat1 entry containing NULL as the index
 83895  84666     ** name and the row count as the content.
 83896  84667     */
 83897  84668     if( pOnlyIdx==0 && needTableCnt ){
................................................................................
 83984  84755     sqlite3 *db = pParse->db;
 83985  84756     int iDb;
 83986  84757     int i;
 83987  84758     char *z, *zDb;
 83988  84759     Table *pTab;
 83989  84760     Index *pIdx;
 83990  84761     Token *pTableName;
        84762  +  Vdbe *v;
 83991  84763   
 83992  84764     /* Read the database schema. If an error occurs, leave an error message
 83993  84765     ** and code in pParse and return NULL. */
 83994  84766     assert( sqlite3BtreeHoldsAllMutexes(pParse->db) );
 83995  84767     if( SQLITE_OK!=sqlite3ReadSchema(pParse) ){
 83996  84768       return;
 83997  84769     }
................................................................................
 84031  84803           }else if( (pTab = sqlite3LocateTable(pParse, 0, z, zDb))!=0 ){
 84032  84804             analyzeTable(pParse, pTab, 0);
 84033  84805           }
 84034  84806           sqlite3DbFree(db, z);
 84035  84807         }
 84036  84808       }   
 84037  84809     }
        84810  +  v = sqlite3GetVdbe(pParse);
        84811  +  if( v ) sqlite3VdbeAddOp0(v, OP_Expire);
 84038  84812   }
 84039  84813   
 84040  84814   /*
 84041  84815   ** Used to pass information from the analyzer reader through to the
 84042  84816   ** callback routine.
 84043  84817   */
 84044  84818   typedef struct analysisInfo analysisInfo;
................................................................................
 84089  84863       if( *z==' ' ) z++;
 84090  84864     }
 84091  84865   #ifndef SQLITE_ENABLE_STAT3_OR_STAT4
 84092  84866     assert( pIndex!=0 );
 84093  84867   #else
 84094  84868     if( pIndex )
 84095  84869   #endif
 84096         -  {
 84097         -    if( strcmp(z, "unordered")==0 ){
        84870  +  while( z[0] ){
        84871  +    if( sqlite3_strglob("unordered*", z)==0 ){
 84098  84872         pIndex->bUnordered = 1;
 84099  84873       }else if( sqlite3_strglob("sz=[0-9]*", z)==0 ){
 84100         -      int v32 = 0;
 84101         -      sqlite3GetInt32(z+3, &v32);
 84102         -      pIndex->szIdxRow = sqlite3LogEst(v32);
        84874  +      pIndex->szIdxRow = sqlite3LogEst(sqlite3Atoi(z+3));
 84103  84875       }
        84876  +#ifdef SQLITE_ENABLE_COSTMULT
        84877  +    else if( sqlite3_strglob("costmult=[0-9]*",z)==0 ){
        84878  +      pIndex->pTable->costMult = sqlite3LogEst(sqlite3Atoi(z+9));
        84879  +    }
        84880  +#endif
        84881  +    while( z[0]!=0 && z[0]!=' ' ) z++;
        84882  +    while( z[0]==' ' ) z++;
 84104  84883     }
 84105  84884   }
 84106  84885   
 84107  84886   /*
 84108  84887   ** This callback is invoked once for each index when reading the
 84109  84888   ** sqlite_stat1 table.  
 84110  84889   **
................................................................................
 84137  84916       pIndex = sqlite3PrimaryKeyIndex(pTable);
 84138  84917     }else{
 84139  84918       pIndex = sqlite3FindIndex(pInfo->db, argv[1], pInfo->zDatabase);
 84140  84919     }
 84141  84920     z = argv[2];
 84142  84921   
 84143  84922     if( pIndex ){
        84923  +    pIndex->bUnordered = 0;
 84144  84924       decodeIntArray((char*)z, pIndex->nKeyCol+1, 0, pIndex->aiRowLogEst, pIndex);
 84145  84925       if( pIndex->pPartIdxWhere==0 ) pTable->nRowLogEst = pIndex->aiRowLogEst[0];
 84146  84926     }else{
 84147  84927       Index fakeIdx;
 84148  84928       fakeIdx.szIdxRow = pTable->szTabRow;
        84929  +#ifdef SQLITE_ENABLE_COSTMULT
        84930  +    fakeIdx.pTable = pTable;
        84931  +#endif
 84149  84932       decodeIntArray((char*)z, 1, 0, &pTable->nRowLogEst, &fakeIdx);
 84150  84933       pTable->szTabRow = fakeIdx.szIdxRow;
 84151  84934     }
 84152  84935   
 84153  84936     return 0;
 84154  84937   }
 84155  84938   
................................................................................
 84183  84966   ** stored in pIdx->aSample[]. 
 84184  84967   */
 84185  84968   static void initAvgEq(Index *pIdx){
 84186  84969     if( pIdx ){
 84187  84970       IndexSample *aSample = pIdx->aSample;
 84188  84971       IndexSample *pFinal = &aSample[pIdx->nSample-1];
 84189  84972       int iCol;
 84190         -    for(iCol=0; iCol<pIdx->nKeyCol; iCol++){
        84973  +    int nCol = 1;
        84974  +    if( pIdx->nSampleCol>1 ){
        84975  +      /* If this is stat4 data, then calculate aAvgEq[] values for all
        84976  +      ** sample columns except the last. The last is always set to 1, as
        84977  +      ** once the trailing PK fields are considered all index keys are
        84978  +      ** unique.  */
        84979  +      nCol = pIdx->nSampleCol-1;
        84980  +      pIdx->aAvgEq[nCol] = 1;
        84981  +    }
        84982  +    for(iCol=0; iCol<nCol; iCol++){
 84191  84983         int i;                    /* Used to iterate through samples */
 84192  84984         tRowcnt sumEq = 0;        /* Sum of the nEq values */
 84193  84985         tRowcnt nSum = 0;         /* Number of terms contributing to sumEq */
 84194  84986         tRowcnt avgEq = 0;
 84195  84987         tRowcnt nDLt = pFinal->anDLt[iCol];
 84196  84988   
 84197  84989         /* Set nSum to the number of distinct (iCol+1) field prefixes that
................................................................................
 84206  84998           }
 84207  84999         }
 84208  85000         if( nDLt>nSum ){
 84209  85001           avgEq = (pFinal->anLt[iCol] - sumEq)/(nDLt - nSum);
 84210  85002         }
 84211  85003         if( avgEq==0 ) avgEq = 1;
 84212  85004         pIdx->aAvgEq[iCol] = avgEq;
 84213         -      if( pIdx->nSampleCol==1 ) break;
 84214  85005       }
 84215  85006     }
 84216  85007   }
 84217  85008   
 84218  85009   /*
 84219  85010   ** Look up an index by name.  Or, if the name of a WITHOUT ROWID table
 84220  85011   ** is supplied instead, find the PRIMARY KEY index for that table.
................................................................................
 84265  85056     }
 84266  85057     rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0);
 84267  85058     sqlite3DbFree(db, zSql);
 84268  85059     if( rc ) return rc;
 84269  85060   
 84270  85061     while( sqlite3_step(pStmt)==SQLITE_ROW ){
 84271  85062       int nIdxCol = 1;              /* Number of columns in stat4 records */
 84272         -    int nAvgCol = 1;              /* Number of entries in Index.aAvgEq */
 84273  85063   
 84274  85064       char *zIndex;   /* Index name */
 84275  85065       Index *pIdx;    /* Pointer to the index object */
 84276  85066       int nSample;    /* Number of samples */
 84277  85067       int nByte;      /* Bytes of space required */
 84278  85068       int i;          /* Bytes of space required */
 84279  85069       tRowcnt *pSpace;
................................................................................
 84283  85073       nSample = sqlite3_column_int(pStmt, 1);
 84284  85074       pIdx = findIndexOrPrimaryKey(db, zIndex, zDb);
 84285  85075       assert( pIdx==0 || bStat3 || pIdx->nSample==0 );
 84286  85076       /* Index.nSample is non-zero at this point if data has already been
 84287  85077       ** loaded from the stat4 table. In this case ignore stat3 data.  */
 84288  85078       if( pIdx==0 || pIdx->nSample ) continue;
 84289  85079       if( bStat3==0 ){
 84290         -      nIdxCol = pIdx->nKeyCol+1;
 84291         -      nAvgCol = pIdx->nKeyCol;
        85080  +      assert( !HasRowid(pIdx->pTable) || pIdx->nColumn==pIdx->nKeyCol+1 );
        85081  +      if( !HasRowid(pIdx->pTable) && IsPrimaryKeyIndex(pIdx) ){
        85082  +        nIdxCol = pIdx->nKeyCol;
        85083  +      }else{
        85084  +        nIdxCol = pIdx->nColumn;
        85085  +      }
 84292  85086       }
 84293  85087       pIdx->nSampleCol = nIdxCol;
 84294  85088       nByte = sizeof(IndexSample) * nSample;
 84295  85089       nByte += sizeof(tRowcnt) * nIdxCol * 3 * nSample;
 84296         -    nByte += nAvgCol * sizeof(tRowcnt);     /* Space for Index.aAvgEq[] */
        85090  +    nByte += nIdxCol * sizeof(tRowcnt);     /* Space for Index.aAvgEq[] */
 84297  85091   
 84298  85092       pIdx->aSample = sqlite3DbMallocZero(db, nByte);
 84299  85093       if( pIdx->aSample==0 ){
 84300  85094         sqlite3_finalize(pStmt);
 84301  85095         return SQLITE_NOMEM;
 84302  85096       }
 84303  85097       pSpace = (tRowcnt*)&pIdx->aSample[nSample];
 84304         -    pIdx->aAvgEq = pSpace; pSpace += nAvgCol;
        85098  +    pIdx->aAvgEq = pSpace; pSpace += nIdxCol;
 84305  85099       for(i=0; i<nSample; i++){
 84306  85100         pIdx->aSample[i].anEq = pSpace; pSpace += nIdxCol;
 84307  85101         pIdx->aSample[i].anLt = pSpace; pSpace += nIdxCol;
 84308  85102         pIdx->aSample[i].anDLt = pSpace; pSpace += nIdxCol;
 84309  85103       }
 84310  85104       assert( ((u8*)pSpace)-nByte==(u8*)(pIdx->aSample) );
 84311  85105     }
................................................................................
 85406  86200                         p->zName, P4_STATIC);
 85407  86201     }
 85408  86202   }
 85409  86203   #else
 85410  86204     #define codeTableLocks(x)
 85411  86205   #endif
 85412  86206   
        86207  +/*
        86208  +** Return TRUE if the given yDbMask object is empty - if it contains no
        86209  +** 1 bits.  This routine is used by the DbMaskAllZero() and DbMaskNotZero()
        86210  +** macros when SQLITE_MAX_ATTACHED is greater than 30.
        86211  +*/
        86212  +#if SQLITE_MAX_ATTACHED>30
        86213  +SQLITE_PRIVATE int sqlite3DbMaskAllZero(yDbMask m){
        86214  +  int i;
        86215  +  for(i=0; i<sizeof(yDbMask); i++) if( m[i] ) return 0;
        86216  +  return 1;
        86217  +}
        86218  +#endif
        86219  +
 85413  86220   /*
 85414  86221   ** This routine is called after a single SQL statement has been
 85415  86222   ** parsed and a VDBE program to execute that statement has been
 85416  86223   ** prepared.  This routine puts the finishing touches on the
 85417  86224   ** VDBE program and resets the pParse structure for the next
 85418  86225   ** parse.
 85419  86226   **
................................................................................
 85442  86249   
 85443  86250       /* The cookie mask contains one bit for each database file open.
 85444  86251       ** (Bit 0 is for main, bit 1 is for temp, and so forth.)  Bits are
 85445  86252       ** set for each database that is used.  Generate code to start a
 85446  86253       ** transaction on each used database and to verify the schema cookie
 85447  86254       ** on each used database.
 85448  86255       */
 85449         -    if( db->mallocFailed==0 && (pParse->cookieMask || pParse->pConstExpr) ){
 85450         -      yDbMask mask;
        86256  +    if( db->mallocFailed==0 
        86257  +     && (DbMaskNonZero(pParse->cookieMask) || pParse->pConstExpr)
        86258  +    ){
 85451  86259         int iDb, i;
 85452  86260         assert( sqlite3VdbeGetOp(v, 0)->opcode==OP_Init );
 85453  86261         sqlite3VdbeJumpHere(v, 0);
 85454         -      for(iDb=0, mask=1; iDb<db->nDb; mask<<=1, iDb++){
 85455         -        if( (mask & pParse->cookieMask)==0 ) continue;
        86262  +      for(iDb=0; iDb<db->nDb; iDb++){
        86263  +        if( DbMaskTest(pParse->cookieMask, iDb)==0 ) continue;
 85456  86264           sqlite3VdbeUsesBtree(v, iDb);
 85457  86265           sqlite3VdbeAddOp4Int(v,
 85458  86266             OP_Transaction,                    /* Opcode */
 85459  86267             iDb,                               /* P1 */
 85460         -          (mask & pParse->writeMask)!=0,     /* P2 */
        86268  +          DbMaskTest(pParse->writeMask,iDb), /* P2 */
 85461  86269             pParse->cookieValue[iDb],          /* P3 */
 85462  86270             db->aDb[iDb].pSchema->iGeneration  /* P4 */
 85463  86271           );
 85464  86272           if( db->init.busy==0 ) sqlite3VdbeChangeP5(v, 1);
 85465  86273         }
 85466  86274   #ifndef SQLITE_OMIT_VIRTUALTABLE
 85467  86275         for(i=0; i<pParse->nVtabLock; i++){
................................................................................
 85509  86317     }else{
 85510  86318       pParse->rc = SQLITE_ERROR;
 85511  86319     }
 85512  86320     pParse->nTab = 0;
 85513  86321     pParse->nMem = 0;
 85514  86322     pParse->nSet = 0;
 85515  86323     pParse->nVar = 0;
 85516         -  pParse->cookieMask = 0;
        86324  +  DbMaskZero(pParse->cookieMask);
 85517  86325   }
 85518  86326   
 85519  86327   /*
 85520  86328   ** Run the parser and code generator recursively in order to generate
 85521  86329   ** code for the SQL statement given onto the end of the pParse context
 85522  86330   ** currently under construction.  When the parser is run recursively
 85523  86331   ** this way, the final OP_Halt is not appended and other initialization
................................................................................
 87409  88217         assert( pTable->aCol==0 );
 87410  88218         pTable->nCol = pSelTab->nCol;
 87411  88219         pTable->aCol = pSelTab->aCol;
 87412  88220         pSelTab->nCol = 0;
 87413  88221         pSelTab->aCol = 0;
 87414  88222         sqlite3DeleteTable(db, pSelTab);
 87415  88223         assert( sqlite3SchemaMutexHeld(db, 0, pTable->pSchema) );
 87416         -      pTable->pSchema->flags |= DB_UnresetViews;
        88224  +      pTable->pSchema->schemaFlags |= DB_UnresetViews;
 87417  88225       }else{
 87418  88226         pTable->nCol = 0;
 87419  88227         nErr++;
 87420  88228       }
 87421  88229       sqlite3SelectDelete(db, pSel);
 87422  88230     } else {
 87423  88231       nErr++;
................................................................................
 87986  88794     if( memRootPage<0 ) sqlite3VdbeAddOp2(v, OP_Clear, tnum, iDb);
 87987  88795     sqlite3VdbeAddOp4(v, OP_OpenWrite, iIdx, tnum, iDb, 
 87988  88796                       (char *)pKey, P4_KEYINFO);
 87989  88797     sqlite3VdbeChangeP5(v, OPFLAG_BULKCSR|((memRootPage>=0)?OPFLAG_P2ISREG:0));
 87990  88798   
 87991  88799     addr1 = sqlite3VdbeAddOp2(v, OP_SorterSort, iSorter, 0); VdbeCoverage(v);
 87992  88800     assert( pKey!=0 || db->mallocFailed || pParse->nErr );
 87993         -  if( pIndex->onError!=OE_None && pKey!=0 ){
        88801  +  if( IsUniqueIndex(pIndex) && pKey!=0 ){
 87994  88802       int j2 = sqlite3VdbeCurrentAddr(v) + 3;
 87995  88803       sqlite3VdbeAddOp2(v, OP_Goto, 0, j2);
 87996  88804       addr2 = sqlite3VdbeCurrentAddr(v);
 87997  88805       sqlite3VdbeAddOp4Int(v, OP_SorterCompare, iSorter, j2, regRecord,
 87998         -                         pKey->nField - pIndex->nKeyCol); VdbeCoverage(v);
        88806  +                         pIndex->nKeyCol); VdbeCoverage(v);
 87999  88807       sqlite3UniqueConstraint(pParse, OE_Abort, pIndex);
 88000  88808     }else{
 88001  88809       addr2 = sqlite3VdbeCurrentAddr(v);
 88002  88810     }
 88003  88811     sqlite3VdbeAddOp2(v, OP_SorterData, iSorter, regRecord);
 88004  88812     sqlite3VdbeAddOp3(v, OP_IdxInsert, iIdx, regRecord, 1);
 88005  88813     sqlite3VdbeChangeP5(v, OPFLAG_USESEEKRESULT);
................................................................................
 88383  89191       ** If there are different collating sequences or if the columns of
 88384  89192       ** the constraint occur in different orders, then the constraints are
 88385  89193       ** considered distinct and both result in separate indices.
 88386  89194       */
 88387  89195       Index *pIdx;
 88388  89196       for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
 88389  89197         int k;
 88390         -      assert( pIdx->onError!=OE_None );
        89198  +      assert( IsUniqueIndex(pIdx) );
 88391  89199         assert( pIdx->idxType!=SQLITE_IDXTYPE_APPDEF );
 88392         -      assert( pIndex->onError!=OE_None );
        89200  +      assert( IsUniqueIndex(pIndex) );
 88393  89201   
 88394  89202         if( pIdx->nKeyCol!=pIndex->nKeyCol ) continue;
 88395  89203         for(k=0; k<pIdx->nKeyCol; k++){
 88396  89204           const char *z1;
 88397  89205           const char *z2;
 88398  89206           if( pIdx->aiColumn[k]!=pIndex->aiColumn[k] ) break;
 88399  89207           z1 = pIdx->azColl[k];
................................................................................
 88576  89384     ** 6 and each subsequent value (if any) is 5.  */
 88577  89385     memcpy(&a[1], aVal, nCopy*sizeof(LogEst));
 88578  89386     for(i=nCopy+1; i<=pIdx->nKeyCol; i++){
 88579  89387       a[i] = 23;                    assert( 23==sqlite3LogEst(5) );
 88580  89388     }
 88581  89389   
 88582  89390     assert( 0==sqlite3LogEst(1) );
 88583         -  if( pIdx->onError!=OE_None ) a[pIdx->nKeyCol] = 0;
        89391  +  if( IsUniqueIndex(pIdx) ) a[pIdx->nKeyCol] = 0;
 88584  89392   }
 88585  89393   
 88586  89394   /*
 88587  89395   ** This routine will drop an existing named index.  This routine
 88588  89396   ** implements the DROP INDEX statement.
 88589  89397   */
 88590  89398   SQLITE_PRIVATE void sqlite3DropIndex(Parse *pParse, SrcList *pName, int ifExists){
................................................................................
 89136  89944   ** for database iDb.  The code to actually verify the schema cookie
 89137  89945   ** will occur at the end of the top-level VDBE and will be generated
 89138  89946   ** later, by sqlite3FinishCoding().
 89139  89947   */
 89140  89948   SQLITE_PRIVATE void sqlite3CodeVerifySchema(Parse *pParse, int iDb){
 89141  89949     Parse *pToplevel = sqlite3ParseToplevel(pParse);
 89142  89950     sqlite3 *db = pToplevel->db;
 89143         -  yDbMask mask;
 89144  89951   
 89145  89952     assert( iDb>=0 && iDb<db->nDb );
 89146  89953     assert( db->aDb[iDb].pBt!=0 || iDb==1 );
 89147  89954     assert( iDb<SQLITE_MAX_ATTACHED+2 );
 89148  89955     assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
 89149         -  mask = ((yDbMask)1)<<iDb;
 89150         -  if( (pToplevel->cookieMask & mask)==0 ){
 89151         -    pToplevel->cookieMask |= mask;
        89956  +  if( DbMaskTest(pToplevel->cookieMask, iDb)==0 ){
        89957  +    DbMaskSet(pToplevel->cookieMask, iDb);
 89152  89958       pToplevel->cookieValue[iDb] = db->aDb[iDb].pSchema->schema_cookie;
 89153  89959       if( !OMIT_TEMPDB && iDb==1 ){
 89154  89960         sqlite3OpenTempDatabase(pToplevel);
 89155  89961       }
 89156  89962     }
 89157  89963   }
 89158  89964   
................................................................................
 89183  89989   ** rollback the whole transaction.  For operations where all constraints
 89184  89990   ** can be checked before any changes are made to the database, it is never
 89185  89991   ** necessary to undo a write and the checkpoint should not be set.
 89186  89992   */
 89187  89993   SQLITE_PRIVATE void sqlite3BeginWriteOperation(Parse *pParse, int setStatement, int iDb){
 89188  89994     Parse *pToplevel = sqlite3ParseToplevel(pParse);
 89189  89995     sqlite3CodeVerifySchema(pParse, iDb);
 89190         -  pToplevel->writeMask |= ((yDbMask)1)<<iDb;
        89996  +  DbMaskSet(pToplevel->writeMask, iDb);
 89191  89997     pToplevel->isMultiWrite |= setStatement;
 89192  89998   }
 89193  89999   
 89194  90000   /*
 89195  90001   ** Indicate that the statement currently under construction might write
 89196  90002   ** more than one entry (example: deleting one row then inserting another,
 89197  90003   ** inserting multiple rows in a table, or inserting a row and index entries.)
................................................................................
 89990  90796     for(pElem=sqliteHashFirst(&temp1); pElem; pElem=sqliteHashNext(pElem)){
 89991  90797       Table *pTab = sqliteHashData(pElem);
 89992  90798       sqlite3DeleteTable(0, pTab);
 89993  90799     }
 89994  90800     sqlite3HashClear(&temp1);
 89995  90801     sqlite3HashClear(&pSchema->fkeyHash);
 89996  90802     pSchema->pSeqTab = 0;
 89997         -  if( pSchema->flags & DB_SchemaLoaded ){
        90803  +  if( pSchema->schemaFlags & DB_SchemaLoaded ){
 89998  90804       pSchema->iGeneration++;
 89999         -    pSchema->flags &= ~DB_SchemaLoaded;
        90805  +    pSchema->schemaFlags &= ~DB_SchemaLoaded;
 90000  90806     }
 90001  90807   }
 90002  90808   
 90003  90809   /*
 90004  90810   ** Find and return the schema associated with a BTree.  Create
 90005  90811   ** a new one if necessary.
 90006  90812   */
................................................................................
 90871  91677   ** a legal notice, here is a blessing:
 90872  91678   **
 90873  91679   **    May you do good and not evil.
 90874  91680   **    May you find forgiveness for yourself and forgive others.
 90875  91681   **    May you share freely, never taking more than you give.
 90876  91682   **
 90877  91683   *************************************************************************
 90878         -** This file contains the C functions that implement various SQL
 90879         -** functions of SQLite.  
 90880         -**
 90881         -** There is only one exported symbol in this file - the function
 90882         -** sqliteRegisterBuildinFunctions() found at the bottom of the file.
 90883         -** All other code has file scope.
        91684  +** This file contains the C-language implementions for many of the SQL
        91685  +** functions of SQLite.  (Some function, and in particular the date and
        91686  +** time functions, are implemented separately.)
 90884  91687   */
 90885  91688   /* #include <stdlib.h> */
 90886  91689   /* #include <assert.h> */
 90887  91690   
 90888  91691   /*
 90889  91692   ** Return the collating function associated with a function.
 90890  91693   */
................................................................................
 92551  93354       FUNCTION(coalesce,           1, 0, 0, 0                ),
 92552  93355       FUNCTION(coalesce,           0, 0, 0, 0                ),
 92553  93356       FUNCTION2(coalesce,         -1, 0, 0, noopFunc,  SQLITE_FUNC_COALESCE),
 92554  93357       FUNCTION(hex,                1, 0, 0, hexFunc          ),
 92555  93358       FUNCTION2(ifnull,            2, 0, 0, noopFunc,  SQLITE_FUNC_COALESCE),
 92556  93359       FUNCTION2(unlikely,          1, 0, 0, noopFunc,  SQLITE_FUNC_UNLIKELY),
 92557  93360       FUNCTION2(likelihood,        2, 0, 0, noopFunc,  SQLITE_FUNC_UNLIKELY),
        93361  +    FUNCTION2(likely,            1, 0, 0, noopFunc,  SQLITE_FUNC_UNLIKELY),
 92558  93362       VFUNCTION(random,            0, 0, 0, randomFunc       ),
 92559  93363       VFUNCTION(randomblob,        1, 0, 0, randomBlob       ),
 92560  93364       FUNCTION(nullif,             2, 0, 1, nullifFunc       ),
 92561  93365       FUNCTION(sqlite_version,     0, 0, 0, versionFunc      ),
 92562  93366       FUNCTION(sqlite_source_id,   0, 0, 0, sourceidFunc     ),
 92563  93367       FUNCTION(sqlite_log,         2, 0, 0, errlogFunc       ),
 92564  93368   #ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS
................................................................................
 92837  93641       assert( nCol>1 );
 92838  93642       aiCol = (int *)sqlite3DbMallocRaw(pParse->db, nCol*sizeof(int));
 92839  93643       if( !aiCol ) return 1;
 92840  93644       *paiCol = aiCol;
 92841  93645     }
 92842  93646   
 92843  93647     for(pIdx=pParent->pIndex; pIdx; pIdx=pIdx->pNext){
 92844         -    if( pIdx->nKeyCol==nCol && pIdx->onError!=OE_None ){ 
        93648  +    if( pIdx->nKeyCol==nCol && IsUniqueIndex(pIdx) ){ 
 92845  93649         /* pIdx is a UNIQUE index (or a PRIMARY KEY) and has the right number
 92846  93650         ** of columns. If each indexed column corresponds to a foreign key
 92847  93651         ** column of pFKey, then this index is a winner.  */
 92848  93652   
 92849  93653         if( zKey==0 ){
 92850  93654           /* If zKey is NULL, then this foreign key is implicitly mapped to 
 92851  93655           ** the PRIMARY KEY of table pParent. The PRIMARY KEY index may be 
................................................................................
 95863  96667        && ((pDestCol->zDflt==0)!=(pSrcCol->zDflt==0) 
 95864  96668            || (pDestCol->zDflt && strcmp(pDestCol->zDflt, pSrcCol->zDflt)!=0))
 95865  96669       ){
 95866  96670         return 0;    /* Default values must be the same for all columns */
 95867  96671       }
 95868  96672     }
 95869  96673     for(pDestIdx=pDest->pIndex; pDestIdx; pDestIdx=pDestIdx->pNext){
 95870         -    if( pDestIdx->onError!=OE_None ){
        96674  +    if( IsUniqueIndex(pDestIdx) ){
 95871  96675         destHasUniqueIdx = 1;
 95872  96676       }
 95873  96677       for(pSrcIdx=pSrc->pIndex; pSrcIdx; pSrcIdx=pSrcIdx->pNext){
 95874  96678         if( xferCompatibleIndex(pDestIdx, pSrcIdx) ) break;
 95875  96679       }
 95876  96680       if( pSrcIdx==0 ){
 95877  96681         return 0;    /* pDestIdx has no corresponding index in pSrc */
................................................................................
 96093  96897               if( !azVals[i] && sqlite3_column_type(pStmt, i)!=SQLITE_NULL ){
 96094  96898                 db->mallocFailed = 1;
 96095  96899                 goto exec_out;
 96096  96900               }
 96097  96901             }
 96098  96902           }
 96099  96903           if( xCallback(pArg, nCol, azVals, azCols) ){
        96904  +          /* EVIDENCE-OF: R-38229-40159 If the callback function to
        96905  +          ** sqlite3_exec() returns non-zero, then sqlite3_exec() will
        96906  +          ** return SQLITE_ABORT. */
 96100  96907             rc = SQLITE_ABORT;
 96101  96908             sqlite3VdbeFinalize((Vdbe *)pStmt);
 96102  96909             pStmt = 0;
 96103  96910             sqlite3Error(db, SQLITE_ABORT, 0);
 96104  96911             goto exec_out;
 96105  96912           }
 96106  96913         }
................................................................................
 97870  98677   ** if the omitFull parameter it 1.
 97871  98678   **
 97872  98679   ** Note that the values returned are one less that the values that
 97873  98680   ** should be passed into sqlite3BtreeSetSafetyLevel().  The is done
 97874  98681   ** to support legacy SQL code.  The safety level used to be boolean
 97875  98682   ** and older scripts may have used numbers 0 for OFF and 1 for ON.
 97876  98683   */
 97877         -static u8 getSafetyLevel(const char *z, int omitFull, int dflt){
        98684  +static u8 getSafetyLevel(const char *z, int omitFull, u8 dflt){
 97878  98685                                /* 123456789 123456789 */
 97879  98686     static const char zText[] = "onoffalseyestruefull";
 97880  98687     static const u8 iOffset[] = {0, 1, 2, 4, 9, 12, 16};
 97881  98688     static const u8 iLength[] = {2, 2, 3, 5, 3, 4, 4};
 97882  98689     static const u8 iValue[] =  {1, 0, 0, 0, 1, 1, 2};
 97883  98690     int i, n;
 97884  98691     if( sqlite3Isdigit(*z) ){
................................................................................
 97892  98699     }
 97893  98700     return dflt;
 97894  98701   }
 97895  98702   
 97896  98703   /*
 97897  98704   ** Interpret the given string as a boolean value.
 97898  98705   */
 97899         -SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z, int dflt){
        98706  +SQLITE_PRIVATE u8 sqlite3GetBoolean(const char *z, u8 dflt){
 97900  98707     return getSafetyLevel(z,1,dflt)!=0;
 97901  98708   }
 97902  98709   
 97903  98710   /* The sqlite3GetBoolean() function is used by other modules but the
 97904  98711   ** remainder of this file is specific to PRAGMA processing.  So omit
 97905  98712   ** the rest of the file if PRAGMAs are omitted from the build.
 97906  98713   */
................................................................................
 98438  99245     **
 98439  99246     ** Get or set the size limit on rollback journal files.
 98440  99247     */
 98441  99248     case PragTyp_JOURNAL_SIZE_LIMIT: {
 98442  99249       Pager *pPager = sqlite3BtreePager(pDb->pBt);
 98443  99250       i64 iLimit = -2;
 98444  99251       if( zRight ){
 98445         -      sqlite3Atoi64(zRight, &iLimit, sqlite3Strlen30(zRight), SQLITE_UTF8);
        99252  +      sqlite3DecOrHexToI64(zRight, &iLimit);
 98446  99253         if( iLimit<-1 ) iLimit = -1;
 98447  99254       }
 98448  99255       iLimit = sqlite3PagerJournalSizeLimit(pPager, iLimit);
 98449  99256       returnSingleInt(pParse, "journal_size_limit", iLimit);
 98450  99257       break;
 98451  99258     }
 98452  99259   
................................................................................
 98566  99373     */
 98567  99374     case PragTyp_MMAP_SIZE: {
 98568  99375       sqlite3_int64 sz;
 98569  99376   #if SQLITE_MAX_MMAP_SIZE>0
 98570  99377       assert( sqlite3SchemaMutexHeld(db, iDb, 0) );
 98571  99378       if( zRight ){
 98572  99379         int ii;
 98573         -      sqlite3Atoi64(zRight, &sz, sqlite3Strlen30(zRight), SQLITE_UTF8);
        99380  +      sqlite3DecOrHexToI64(zRight, &sz);
 98574  99381         if( sz<0 ) sz = sqlite3GlobalConfig.szMmap;
 98575  99382         if( pId2->n==0 ) db->szMmap = sz;
 98576  99383         for(ii=db->nDb-1; ii>=0; ii--){
 98577  99384           if( db->aDb[ii].pBt && (ii==iDb || pId2->n==0) ){
 98578  99385             sqlite3BtreeSetMmapLimit(db->aDb[ii].pBt, sz);
 98579  99386           }
 98580  99387         }
................................................................................
 98934  99741         sqlite3CodeVerifySchema(pParse, iDb);
 98935  99742         sqlite3VdbeSetColName(v, 0, COLNAME_NAME, "seq", SQLITE_STATIC);
 98936  99743         sqlite3VdbeSetColName(v, 1, COLNAME_NAME, "name", SQLITE_STATIC);
 98937  99744         sqlite3VdbeSetColName(v, 2, COLNAME_NAME, "unique", SQLITE_STATIC);
 98938  99745         for(pIdx=pTab->pIndex, i=0; pIdx; pIdx=pIdx->pNext, i++){
 98939  99746           sqlite3VdbeAddOp2(v, OP_Integer, i, 1);
 98940  99747           sqlite3VdbeAddOp4(v, OP_String8, 0, 2, 0, pIdx->zName, 0);
 98941         -        sqlite3VdbeAddOp2(v, OP_Integer, pIdx->onError!=OE_None, 3);
        99748  +        sqlite3VdbeAddOp2(v, OP_Integer, IsUniqueIndex(pIdx), 3);
 98942  99749           sqlite3VdbeAddOp2(v, OP_ResultRow, 1, 3);
 98943  99750         }
 98944  99751       }
 98945  99752     }
 98946  99753     break;
 98947  99754   
 98948  99755     case PragTyp_DATABASE_LIST: {
................................................................................
 99184  99991   
 99185  99992       /* Code that appears at the end of the integrity check.  If no error
 99186  99993       ** messages have been generated, output OK.  Otherwise output the
 99187  99994       ** error message
 99188  99995       */
 99189  99996       static const int iLn = VDBE_OFFSET_LINENO(2);
 99190  99997       static const VdbeOpList endCode[] = {
 99191         -      { OP_AddImm,      1, 0,        0},    /* 0 */
 99192         -      { OP_IfNeg,       1, 0,        0},    /* 1 */
 99193         -      { OP_String8,     0, 3,        0},    /* 2 */
        99998  +      { OP_IfNeg,       1, 0,        0},    /* 0 */
        99999  +      { OP_String8,     0, 3,        0},    /* 1 */
 99194 100000         { OP_ResultRow,   3, 1,        0},
 99195 100001       };
 99196 100002   
 99197 100003       int isQuick = (sqlite3Tolower(zLeft[0])=='q');
 99198 100004   
 99199 100005       /* If the PRAGMA command was of the form "PRAGMA <db>.integrity_check",
 99200 100006       ** then iDb is set to the index of the database identified by <db>.
................................................................................
 99298 100104           sqlite3VdbeAddOp2(v, OP_Integer, 0, 7);
 99299 100105           for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){
 99300 100106             sqlite3VdbeAddOp2(v, OP_Integer, 0, 8+j); /* index entries counter */
 99301 100107           }
 99302 100108           pParse->nMem = MAX(pParse->nMem, 8+j);
 99303 100109           sqlite3VdbeAddOp2(v, OP_Rewind, iDataCur, 0); VdbeCoverage(v);
 99304 100110           loopTop = sqlite3VdbeAddOp2(v, OP_AddImm, 7, 1);
       100111  +        /* Verify that all NOT NULL columns really are NOT NULL */
       100112  +        for(j=0; j<pTab->nCol; j++){
       100113  +          char *zErr;
       100114  +          int jmp2, jmp3;
       100115  +          if( j==pTab->iPKey ) continue;
       100116  +          if( pTab->aCol[j].notNull==0 ) continue;
       100117  +          sqlite3ExprCodeGetColumnOfTable(v, pTab, iDataCur, j, 3);
       100118  +          sqlite3VdbeChangeP5(v, OPFLAG_TYPEOFARG);
       100119  +          jmp2 = sqlite3VdbeAddOp1(v, OP_NotNull, 3); VdbeCoverage(v);
       100120  +          sqlite3VdbeAddOp2(v, OP_AddImm, 1, -1); /* Decrement error limit */
       100121  +          zErr = sqlite3MPrintf(db, "NULL value in %s.%s", pTab->zName,
       100122  +                              pTab->aCol[j].zName);
       100123  +          sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, zErr, P4_DYNAMIC);
       100124  +          sqlite3VdbeAddOp2(v, OP_ResultRow, 3, 1);
       100125  +          jmp3 = sqlite3VdbeAddOp1(v, OP_IfPos, 1); VdbeCoverage(v);
       100126  +          sqlite3VdbeAddOp0(v, OP_Halt);
       100127  +          sqlite3VdbeJumpHere(v, jmp2);
       100128  +          sqlite3VdbeJumpHere(v, jmp3);
       100129  +        }
       100130  +        /* Validate index entries for the current row */
 99305 100131           for(j=0, pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext, j++){
 99306         -          int jmp2, jmp3, jmp4;
       100132  +          int jmp2, jmp3, jmp4, jmp5;
       100133  +          int ckUniq = sqlite3VdbeMakeLabel(v);
 99307 100134             if( pPk==pIdx ) continue;
 99308 100135             r1 = sqlite3GenerateIndexKey(pParse, pIdx, iDataCur, 0, 0, &jmp3,
 99309 100136                                          pPrior, r1);
 99310 100137             pPrior = pIdx;
 99311 100138             sqlite3VdbeAddOp2(v, OP_AddImm, 8+j, 1);  /* increment entry count */
 99312         -          jmp2 = sqlite3VdbeAddOp4Int(v, OP_Found, iIdxCur+j, 0, r1,
       100139  +          /* Verify that an index entry exists for the current table row */
       100140  +          jmp2 = sqlite3VdbeAddOp4Int(v, OP_Found, iIdxCur+j, ckUniq, r1,
 99313 100141                                         pIdx->nColumn); VdbeCoverage(v);
 99314 100142             sqlite3VdbeAddOp2(v, OP_AddImm, 1, -1); /* Decrement error limit */
 99315 100143             sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0, "row ", P4_STATIC);
 99316 100144             sqlite3VdbeAddOp3(v, OP_Concat, 7, 3, 3);
 99317         -          sqlite3VdbeAddOp4(v, OP_String8, 0, 4, 0, " missing from index ",
 99318         -                            P4_STATIC);
       100145  +          sqlite3VdbeAddOp4(v, OP_String8, 0, 4, 0, 
       100146  +                            " missing from index ", P4_STATIC);
 99319 100147             sqlite3VdbeAddOp3(v, OP_Concat, 4, 3, 3);
 99320         -          sqlite3VdbeAddOp4(v, OP_String8, 0, 4, 0, pIdx->zName, P4_TRANSIENT);
       100148  +          jmp5 = sqlite3VdbeAddOp4(v, OP_String8, 0, 4, 0,
       100149  +                                   pIdx->zName, P4_TRANSIENT);
 99321 100150             sqlite3VdbeAddOp3(v, OP_Concat, 4, 3, 3);
 99322 100151             sqlite3VdbeAddOp2(v, OP_ResultRow, 3, 1);
 99323 100152             jmp4 = sqlite3VdbeAddOp1(v, OP_IfPos, 1); VdbeCoverage(v);
 99324 100153             sqlite3VdbeAddOp0(v, OP_Halt);
 99325         -          sqlite3VdbeJumpHere(v, jmp4);
 99326 100154             sqlite3VdbeJumpHere(v, jmp2);
       100155  +          /* For UNIQUE indexes, verify that only one entry exists with the
       100156  +          ** current key.  The entry is unique if (1) any column is NULL
       100157  +          ** or (2) the next entry has a different key */
       100158  +          if( IsUniqueIndex(pIdx) ){
       100159  +            int uniqOk = sqlite3VdbeMakeLabel(v);
       100160  +            int jmp6;
       100161  +            int kk;
       100162  +            for(kk=0; kk<pIdx->nKeyCol; kk++){
       100163  +              int iCol = pIdx->aiColumn[kk];
       100164  +              assert( iCol>=0 && iCol<pTab->nCol );
       100165  +              if( pTab->aCol[iCol].notNull ) continue;
       100166  +              sqlite3VdbeAddOp2(v, OP_IsNull, r1+kk, uniqOk);
       100167  +              VdbeCoverage(v);
       100168  +            }
       100169  +            jmp6 = sqlite3VdbeAddOp1(v, OP_Next, iIdxCur+j); VdbeCoverage(v);
       100170  +            sqlite3VdbeAddOp2(v, OP_Goto, 0, uniqOk);
       100171  +            sqlite3VdbeJumpHere(v, jmp6);
       100172  +            sqlite3VdbeAddOp4Int(v, OP_IdxGT, iIdxCur+j, uniqOk, r1,
       100173  +                                 pIdx->nKeyCol); VdbeCoverage(v);
       100174  +            sqlite3VdbeAddOp2(v, OP_AddImm, 1, -1); /* Decrement error limit */
       100175  +            sqlite3VdbeAddOp4(v, OP_String8, 0, 3, 0,
       100176  +                              "non-unique entry in index ", P4_STATIC);
       100177  +            sqlite3VdbeAddOp2(v, OP_Goto, 0, jmp5);
       100178  +            sqlite3VdbeResolveLabel(v, uniqOk);
       100179  +          }
       100180  +          sqlite3VdbeJumpHere(v, jmp4);
 99327 100181             sqlite3ResolvePartIdxLabel(pParse, jmp3);
 99328 100182           }
 99329 100183           sqlite3VdbeAddOp2(v, OP_Next, iDataCur, loopTop); VdbeCoverage(v);
 99330 100184           sqlite3VdbeJumpHere(v, loopTop-1);
 99331 100185   #ifndef SQLITE_OMIT_BTREECOUNT
 99332 100186           sqlite3VdbeAddOp4(v, OP_String8, 0, 2, 0, 
 99333 100187                        "wrong # of entries in index ", P4_STATIC);
................................................................................
 99344 100198             sqlite3VdbeAddOp3(v, OP_Concat, 3, 2, 7);
 99345 100199             sqlite3VdbeAddOp2(v, OP_ResultRow, 7, 1);
 99346 100200           }
 99347 100201   #endif /* SQLITE_OMIT_BTREECOUNT */
 99348 100202         } 
 99349 100203       }
 99350 100204       addr = sqlite3VdbeAddOpList(v, ArraySize(endCode), endCode, iLn);
 99351         -    sqlite3VdbeChangeP2(v, addr, -mxErr);
 99352         -    sqlite3VdbeJumpHere(v, addr+1);
 99353         -    sqlite3VdbeChangeP4(v, addr+2, "ok", P4_STATIC);
       100205  +    sqlite3VdbeChangeP3(v, addr, -mxErr);
       100206  +    sqlite3VdbeJumpHere(v, addr);
       100207  +    sqlite3VdbeChangeP4(v, addr+1, "ok", P4_STATIC);
 99354 100208     }
 99355 100209     break;
 99356 100210   #endif /* SQLITE_OMIT_INTEGRITY_CHECK */
 99357 100211   
 99358 100212   #ifndef SQLITE_OMIT_UTF16
 99359 100213     /*
 99360 100214     **   PRAGMA encoding
................................................................................
 99609 100463     **   PRAGMA soft_heap_limit = N
 99610 100464     **
 99611 100465     ** Call sqlite3_soft_heap_limit64(N).  Return the result.  If N is omitted,
 99612 100466     ** use -1.
 99613 100467     */
 99614 100468     case PragTyp_SOFT_HEAP_LIMIT: {
 99615 100469       sqlite3_int64 N;
 99616         -    if( zRight && sqlite3Atoi64(zRight, &N, 1000000, SQLITE_UTF8)==SQLITE_OK ){
       100470  +    if( zRight && sqlite3DecOrHexToI64(zRight, &N)==SQLITE_OK ){
 99617 100471         sqlite3_soft_heap_limit64(N);
 99618 100472       }
 99619 100473       returnSingleInt(pParse, "soft_heap_limit",  sqlite3_soft_heap_limit64(-1));
 99620 100474       break;
 99621 100475     }
 99622 100476   
 99623 100477   #if defined(SQLITE_DEBUG) || defined(SQLITE_TEST)
................................................................................
101132 101986   static void codeOffset(
101133 101987     Vdbe *v,          /* Generate code into this VM */
101134 101988     int iOffset,      /* Register holding the offset counter */
101135 101989     int iContinue     /* Jump here to skip the current record */
101136 101990   ){
101137 101991     if( iOffset>0 ){
101138 101992       int addr;
101139         -    sqlite3VdbeAddOp2(v, OP_AddImm, iOffset, -1);
101140         -    addr = sqlite3VdbeAddOp1(v, OP_IfNeg, iOffset); VdbeCoverage(v);
       101993  +    addr = sqlite3VdbeAddOp3(v, OP_IfNeg, iOffset, 0, -1); VdbeCoverage(v);
101141 101994       sqlite3VdbeAddOp2(v, OP_Goto, 0, iContinue);
101142 101995       VdbeComment((v, "skip OFFSET records"));
101143 101996       sqlite3VdbeJumpHere(v, addr);
101144 101997     }
101145 101998   }
101146 101999   
101147 102000   /*
................................................................................
101298 102151             }else{
101299 102152               sqlite3VdbeAddOp3(v, OP_Eq, regResult+i, iContinue, regPrev+i);
101300 102153               VdbeCoverage(v);
101301 102154              }
101302 102155             sqlite3VdbeChangeP4(v, -1, (const char *)pColl, P4_COLLSEQ);
101303 102156             sqlite3VdbeChangeP5(v, SQLITE_NULLEQ);
101304 102157           }
101305         -        assert( sqlite3VdbeCurrentAddr(v)==iJump );
       102158  +        assert( sqlite3VdbeCurrentAddr(v)==iJump || pParse->db->mallocFailed );
101306 102159           sqlite3VdbeAddOp3(v, OP_Copy, regResult, regPrev, nResultCol-1);
101307 102160           break;
101308 102161         }
101309 102162   
101310 102163         case WHERE_DISTINCT_UNIQUE: {
101311 102164           sqlite3VdbeChangeToNoop(v, pDistinct->addrTnct);
101312 102165           break;
................................................................................
101817 102670     if( pSort->sortFlags & SORTFLAG_UseSorter ){
101818 102671       sqlite3VdbeAddOp2(v, OP_SorterNext, iTab, addr); VdbeCoverage(v);
101819 102672     }else{
101820 102673       sqlite3VdbeAddOp2(v, OP_Next, iTab, addr); VdbeCoverage(v);
101821 102674     }
101822 102675     if( pSort->regReturn ) sqlite3VdbeAddOp1(v, OP_Return, pSort->regReturn);
101823 102676     sqlite3VdbeResolveLabel(v, addrBreak);
101824         -  if( eDest==SRT_Output || eDest==SRT_Coroutine ){
101825         -    sqlite3VdbeAddOp2(v, OP_Close, pseudoTab, 0);
101826         -  }
101827 102677   }
101828 102678   
101829 102679   /*
101830 102680   ** Return a pointer to a string containing the 'declaration type' of the
101831 102681   ** expression pExpr. The string may be treated as static by the caller.
101832 102682   **
101833 102683   ** Also try to estimate the size of the returned value and return that
................................................................................
107768 108618       if( aToOpen[iDataCur-iBaseCur] ){
107769 108619         assert( pPk!=0 );
107770 108620         sqlite3VdbeAddOp4Int(v, OP_NotFound, iDataCur, labelBreak, regKey, nKey);
107771 108621         VdbeCoverageNeverTaken(v);
107772 108622       }
107773 108623       labelContinue = labelBreak;
107774 108624       sqlite3VdbeAddOp2(v, OP_IsNull, pPk ? regKey : regOldRowid, labelBreak);
107775         -    VdbeCoverage(v);
       108625  +    VdbeCoverageIf(v, pPk==0);
       108626  +    VdbeCoverageIf(v, pPk!=0);
107776 108627     }else if( pPk ){
107777 108628       labelContinue = sqlite3VdbeMakeLabel(v);
107778 108629       sqlite3VdbeAddOp2(v, OP_Rewind, iEph, labelBreak); VdbeCoverage(v);
107779 108630       addrTop = sqlite3VdbeAddOp2(v, OP_RowKey, iEph, regKey);
107780 108631       sqlite3VdbeAddOp4Int(v, OP_NotFound, iDataCur, labelContinue, regKey, 0);
107781 108632       VdbeCoverage(v);
107782 108633     }else{
................................................................................
109778 110629   ** at the end is the choosen query plan.
109779 110630   */
109780 110631   struct WherePath {
109781 110632     Bitmask maskLoop;     /* Bitmask of all WhereLoop objects in this path */
109782 110633     Bitmask revLoop;      /* aLoop[]s that should be reversed for ORDER BY */
109783 110634     LogEst nRow;          /* Estimated number of rows generated by this path */
109784 110635     LogEst rCost;         /* Total cost of this path */
       110636  +  LogEst rUnsorted;     /* Total cost of this path ignoring sorting costs */
109785 110637     i8 isOrdered;         /* No. of ORDER BY terms satisfied. -1 for unknown */
109786 110638     WhereLoop **aLoop;    /* Array of WhereLoop objects implementing this path */
109787 110639   };
109788 110640   
109789 110641   /*
109790 110642   ** The query generator uses an array of instances of this structure to
109791 110643   ** help it analyze the subexpressions of the WHERE clause.  Each WHERE
................................................................................
110053 110905   #define WHERE_VIRTUALTABLE 0x00000400  /* WhereLoop.u.vtab is valid */
110054 110906   #define WHERE_IN_ABLE      0x00000800  /* Able to support an IN operator */
110055 110907   #define WHERE_ONEROW       0x00001000  /* Selects no more than one row */
110056 110908   #define WHERE_MULTI_OR     0x00002000  /* OR using multiple indices */
110057 110909   #define WHERE_AUTO_INDEX   0x00004000  /* Uses an ephemeral index */
110058 110910   #define WHERE_SKIPSCAN     0x00008000  /* Uses the skip-scan algorithm */
110059 110911   #define WHERE_UNQ_WANTED   0x00010000  /* WHERE_ONEROW would have been helpful*/
110060         -#define WHERE_LIKELIHOOD   0x00020000  /* A likelihood() is affecting nOut */
110061 110912   
110062 110913   /************** End of whereInt.h ********************************************/
110063 110914   /************** Continuing where we left off in where.c **********************/
110064 110915   
110065 110916   /*
110066 110917   ** Return the estimated number of output rows from a WHERE clause
110067 110918   */
................................................................................
110583 111434   
110584 111435     /* memset(pScan, 0, sizeof(*pScan)); */
110585 111436     pScan->pOrigWC = pWC;
110586 111437     pScan->pWC = pWC;
110587 111438     if( pIdx && iColumn>=0 ){
110588 111439       pScan->idxaff = pIdx->pTable->aCol[iColumn].affinity;
110589 111440       for(j=0; pIdx->aiColumn[j]!=iColumn; j++){
110590         -      if( NEVER(j>=pIdx->nKeyCol) ) return 0;
       111441  +      if( NEVER(j>pIdx->nColumn) ) return 0;
110591 111442       }
110592 111443       pScan->zCollName = pIdx->azColl[j];
110593 111444     }else{
110594 111445       pScan->idxaff = 0;
110595 111446       pScan->zCollName = 0;
110596 111447     }
110597 111448     pScan->opMask = opMask;
................................................................................
111509 112360     **      where X is a constant value. The collation sequences of the
111510 112361     **      comparison and select-list expressions must match those of the index.
111511 112362     **
111512 112363     **   3. All of those index columns for which the WHERE clause does not
111513 112364     **      contain a "col=X" term are subject to a NOT NULL constraint.
111514 112365     */
111515 112366     for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
111516         -    if( pIdx->onError==OE_None ) continue;
       112367  +    if( !IsUniqueIndex(pIdx) ) continue;
111517 112368       for(i=0; i<pIdx->nKeyCol; i++){
111518 112369         i16 iCol = pIdx->aiColumn[i];
111519 112370         if( 0==findTerm(pWC, iBase, iCol, ~(Bitmask)0, WO_EQ, pIdx) ){
111520 112371           int iIdxCol = findIndexCol(pParse, pDistinct, iBase, pIdx, i);
111521 112372           if( iIdxCol<0 || pTab->aCol[iCol].notNull==0 ){
111522 112373             break;
111523 112374           }
................................................................................
111533 112384   }
111534 112385   
111535 112386   
111536 112387   /*
111537 112388   ** Estimate the logarithm of the input value to base 2.
111538 112389   */
111539 112390   static LogEst estLog(LogEst N){
111540         -  LogEst x = sqlite3LogEst(N);
111541         -  return x>33 ? x - 33 : 0;
       112391  +  return N<=10 ? 0 : sqlite3LogEst(N) - 33;
111542 112392   }
111543 112393   
111544 112394   /*
111545 112395   ** Two routines for printing the content of an sqlite3_index_info
111546 112396   ** structure.  Used for testing and debugging only.  If neither
111547 112397   ** SQLITE_TEST or SQLITE_DEBUG are defined, then these routines
111548 112398   ** are no-ops.
................................................................................
111999 112849         iLower = 0;
112000 112850         iUpper = aSample[0].anLt[iCol];
112001 112851       }else{
112002 112852         i64 nRow0 = sqlite3LogEstToInt(pIdx->aiRowLogEst[0]);
112003 112853         iUpper = i>=pIdx->nSample ? nRow0 : aSample[i].anLt[iCol];
112004 112854         iLower = aSample[i-1].anEq[iCol] + aSample[i-1].anLt[iCol];
112005 112855       }
112006         -    aStat[1] = (pIdx->nKeyCol>iCol ? pIdx->aAvgEq[iCol] : 1);
       112856  +    aStat[1] = pIdx->aAvgEq[iCol];
112007 112857       if( iLower>=iUpper ){
112008 112858         iGap = 0;
112009 112859       }else{
112010 112860         iGap = iUpper - iLower;
112011 112861       }
112012 112862       if( roundUp ){
112013 112863         iGap = (iGap*2)/3;
................................................................................
112038 112888       }else if( (pTerm->wtFlags & TERM_VNULL)==0 ){
112039 112889         nRet -= 20;        assert( 20==sqlite3LogEst(4) );
112040 112890       }
112041 112891     }
112042 112892     return nRet;
112043 112893   }
112044 112894   
       112895  +#ifdef SQLITE_ENABLE_STAT3_OR_STAT4
       112896  +/* 
       112897  +** This function is called to estimate the number of rows visited by a
       112898  +** range-scan on a skip-scan index. For example:
       112899  +**
       112900  +**   CREATE INDEX i1 ON t1(a, b, c);
       112901  +**   SELECT * FROM t1 WHERE a=? AND c BETWEEN ? AND ?;
       112902  +**
       112903  +** Value pLoop->nOut is currently set to the estimated number of rows 
       112904  +** visited for scanning (a=? AND b=?). This function reduces that estimate 
       112905  +** by some factor to account for the (c BETWEEN ? AND ?) expression based
       112906  +** on the stat4 data for the index. this scan will be peformed multiple 
       112907  +** times (once for each (a,b) combination that matches a=?) is dealt with 
       112908  +** by the caller.
       112909  +**
       112910  +** It does this by scanning through all stat4 samples, comparing values
       112911  +** extracted from pLower and pUpper with the corresponding column in each
       112912  +** sample. If L and U are the number of samples found to be less than or
       112913  +** equal to the values extracted from pLower and pUpper respectively, and
       112914  +** N is the total number of samples, the pLoop->nOut value is adjusted
       112915  +** as follows:
       112916  +**
       112917  +**   nOut = nOut * ( min(U - L, 1) / N )
       112918  +**
       112919  +** If pLower is NULL, or a value cannot be extracted from the term, L is
       112920  +** set to zero. If pUpper is NULL, or a value cannot be extracted from it,
       112921  +** U is set to N.
       112922  +**
       112923  +** Normally, this function sets *pbDone to 1 before returning. However,
       112924  +** if no value can be extracted from either pLower or pUpper (and so the
       112925  +** estimate of the number of rows delivered remains unchanged), *pbDone
       112926  +** is left as is.
       112927  +**
       112928  +** If an error occurs, an SQLite error code is returned. Otherwise, 
       112929  +** SQLITE_OK.
       112930  +*/
       112931  +static int whereRangeSkipScanEst(
       112932  +  Parse *pParse,       /* Parsing & code generating context */
       112933  +  WhereTerm *pLower,   /* Lower bound on the range. ex: "x>123" Might be NULL */
       112934  +  WhereTerm *pUpper,   /* Upper bound on the range. ex: "x<455" Might be NULL */
       112935  +  WhereLoop *pLoop,    /* Update the .nOut value of this loop */
       112936  +  int *pbDone          /* Set to true if at least one expr. value extracted */
       112937  +){
       112938  +  Index *p = pLoop->u.btree.pIndex;
       112939  +  int nEq = pLoop->u.btree.nEq;
       112940  +  sqlite3 *db = pParse->db;
       112941  +  int nLower = -1;
       112942  +  int nUpper = p->nSample+1;
       112943  +  int rc = SQLITE_OK;
       112944  +  int iCol = p->aiColumn[nEq];
       112945  +  u8 aff = iCol>=0 ? p->pTable->aCol[iCol].affinity : SQLITE_AFF_INTEGER;
       112946  +  CollSeq *pColl;
       112947  +  
       112948  +  sqlite3_value *p1 = 0;          /* Value extracted from pLower */
       112949  +  sqlite3_value *p2 = 0;          /* Value extracted from pUpper */
       112950  +  sqlite3_value *pVal = 0;        /* Value extracted from record */
       112951  +
       112952  +  pColl = sqlite3LocateCollSeq(pParse, p->azColl[nEq]);
       112953  +  if( pLower ){
       112954  +    rc = sqlite3Stat4ValueFromExpr(pParse, pLower->pExpr->pRight, aff, &p1);
       112955  +    nLower = 0;
       112956  +  }
       112957  +  if( pUpper && rc==SQLITE_OK ){
       112958  +    rc = sqlite3Stat4ValueFromExpr(pParse, pUpper->pExpr->pRight, aff, &p2);
       112959  +    nUpper = p2 ? 0 : p->nSample;
       112960  +  }
       112961  +
       112962  +  if( p1 || p2 ){
       112963  +    int i;
       112964  +    int nDiff;
       112965  +    for(i=0; rc==SQLITE_OK && i<p->nSample; i++){
       112966  +      rc = sqlite3Stat4Column(db, p->aSample[i].p, p->aSample[i].n, nEq, &pVal);
       112967  +      if( rc==SQLITE_OK && p1 ){
       112968  +        int res = sqlite3MemCompare(p1, pVal, pColl);
       112969  +        if( res>=0 ) nLower++;
       112970  +      }
       112971  +      if( rc==SQLITE_OK && p2 ){
       112972  +        int res = sqlite3MemCompare(p2, pVal, pColl);
       112973  +        if( res>=0 ) nUpper++;
       112974  +      }
       112975  +    }
       112976  +    nDiff = (nUpper - nLower);
       112977  +    if( nDiff<=0 ) nDiff = 1;
       112978  +
       112979  +    /* If there is both an upper and lower bound specified, and the 
       112980  +    ** comparisons indicate that they are close together, use the fallback
       112981  +    ** method (assume that the scan visits 1/64 of the rows) for estimating
       112982  +    ** the number of rows visited. Otherwise, estimate the number of rows
       112983  +    ** using the method described in the header comment for this function. */
       112984  +    if( nDiff!=1 || pUpper==0 || pLower==0 ){
       112985  +      int nAdjust = (sqlite3LogEst(p->nSample) - sqlite3LogEst(nDiff));
       112986  +      pLoop->nOut -= nAdjust;
       112987  +      *pbDone = 1;
       112988  +      WHERETRACE(0x10, ("range skip-scan regions: %u..%u  adjust=%d est=%d\n",
       112989  +                           nLower, nUpper, nAdjust*-1, pLoop->nOut));
       112990  +    }
       112991  +
       112992  +  }else{
       112993  +    assert( *pbDone==0 );
       112994  +  }
       112995  +
       112996  +  sqlite3ValueFree(p1);
       112997  +  sqlite3ValueFree(p2);
       112998  +  sqlite3ValueFree(pVal);
       112999  +
       113000  +  return rc;
       113001  +}
       113002  +#endif /* SQLITE_ENABLE_STAT3_OR_STAT4 */
       113003  +
112045 113004   /*
112046 113005   ** This function is used to estimate the number of rows that will be visited
112047 113006   ** by scanning an index for a range of values. The range may have an upper
112048 113007   ** bound, a lower bound, or both. The WHERE clause terms that set the upper
112049 113008   ** and lower bounds are represented by pLower and pUpper respectively. For
112050 113009   ** example, assuming that index p is on t1(a):
112051 113010   **
................................................................................
112074 113033   ** When this function is called, *pnOut is set to the sqlite3LogEst() of the
112075 113034   ** number of rows that the index scan is expected to visit without 
112076 113035   ** considering the range constraints. If nEq is 0, this is the number of 
112077 113036   ** rows in the index. Assuming no error occurs, *pnOut is adjusted (reduced)
112078 113037   ** to account for the range contraints pLower and pUpper.
112079 113038   ** 
112080 113039   ** In the absence of sqlite_stat4 ANALYZE data, or if such data cannot be
112081         -** used, each range inequality reduces the search space by a factor of 4. 
112082         -** Hence a pair of constraints (x>? AND x<?) reduces the expected number of
112083         -** rows visited by a factor of 16.
       113040  +** used, a single range inequality reduces the search space by a factor of 4. 
       113041  +** and a pair of constraints (x>? AND x<?) reduces the expected number of
       113042  +** rows visited by a factor of 64.
112084 113043   */
112085 113044   static int whereRangeScanEst(
112086 113045     Parse *pParse,       /* Parsing & code generating context */
112087 113046     WhereLoopBuilder *pBuilder,
112088 113047     WhereTerm *pLower,   /* Lower bound on the range. ex: "x>123" Might be NULL */
112089 113048     WhereTerm *pUpper,   /* Upper bound on the range. ex: "x<455" Might be NULL */
112090 113049     WhereLoop *pLoop     /* Modify the .nOut and maybe .rRun fields */
................................................................................
112094 113053     LogEst nNew;
112095 113054   
112096 113055   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
112097 113056     Index *p = pLoop->u.btree.pIndex;
112098 113057     int nEq = pLoop->u.btree.nEq;
112099 113058   
112100 113059     if( p->nSample>0
112101         -   && nEq==pBuilder->nRecValid
112102 113060      && nEq<p->nSampleCol
112103 113061      && OptimizationEnabled(pParse->db, SQLITE_Stat3) 
112104 113062     ){
112105         -    UnpackedRecord *pRec = pBuilder->pRec;
112106         -    tRowcnt a[2];
112107         -    u8 aff;
112108         -
112109         -    /* Variable iLower will be set to the estimate of the number of rows in 
112110         -    ** the index that are less than the lower bound of the range query. The
112111         -    ** lower bound being the concatenation of $P and $L, where $P is the
112112         -    ** key-prefix formed by the nEq values matched against the nEq left-most
112113         -    ** columns of the index, and $L is the value in pLower.
112114         -    **
112115         -    ** Or, if pLower is NULL or $L cannot be extracted from it (because it
112116         -    ** is not a simple variable or literal value), the lower bound of the
112117         -    ** range is $P. Due to a quirk in the way whereKeyStats() works, even
112118         -    ** if $L is available, whereKeyStats() is called for both ($P) and 
112119         -    ** ($P:$L) and the larger of the two returned values used.
112120         -    **
112121         -    ** Similarly, iUpper is to be set to the estimate of the number of rows
112122         -    ** less than the upper bound of the range query. Where the upper bound
112123         -    ** is either ($P) or ($P:$U). Again, even if $U is available, both values
112124         -    ** of iUpper are requested of whereKeyStats() and the smaller used.
112125         -    */
112126         -    tRowcnt iLower;
112127         -    tRowcnt iUpper;
112128         -
112129         -    if( nEq==p->nKeyCol ){
112130         -      aff = SQLITE_AFF_INTEGER;
112131         -    }else{
112132         -      aff = p->pTable->aCol[p->aiColumn[nEq]].affinity;
112133         -    }
112134         -    /* Determine iLower and iUpper using ($P) only. */
112135         -    if( nEq==0 ){
112136         -      iLower = 0;
112137         -      iUpper = sqlite3LogEstToInt(p->aiRowLogEst[0]);
112138         -    }else{
112139         -      /* Note: this call could be optimized away - since the same values must 
112140         -      ** have been requested when testing key $P in whereEqualScanEst().  */
112141         -      whereKeyStats(pParse, p, pRec, 0, a);
112142         -      iLower = a[0];
112143         -      iUpper = a[0] + a[1];
112144         -    }
112145         -
112146         -    /* If possible, improve on the iLower estimate using ($P:$L). */
112147         -    if( pLower ){
112148         -      int bOk;                    /* True if value is extracted from pExpr */
112149         -      Expr *pExpr = pLower->pExpr->pRight;
112150         -      assert( (pLower->eOperator & (WO_GT|WO_GE))!=0 );
112151         -      rc = sqlite3Stat4ProbeSetValue(pParse, p, &pRec, pExpr, aff, nEq, &bOk);
112152         -      if( rc==SQLITE_OK && bOk ){
112153         -        tRowcnt iNew;
       113063  +    if( nEq==pBuilder->nRecValid ){
       113064  +      UnpackedRecord *pRec = pBuilder->pRec;
       113065  +      tRowcnt a[2];
       113066  +      u8 aff;
       113067  +
       113068  +      /* Variable iLower will be set to the estimate of the number of rows in 
       113069  +      ** the index that are less than the lower bound of the range query. The
       113070  +      ** lower bound being the concatenation of $P and $L, where $P is the
       113071  +      ** key-prefix formed by the nEq values matched against the nEq left-most
       113072  +      ** columns of the index, and $L is the value in pLower.
       113073  +      **
       113074  +      ** Or, if pLower is NULL or $L cannot be extracted from it (because it
       113075  +      ** is not a simple variable or literal value), the lower bound of the
       113076  +      ** range is $P. Due to a quirk in the way whereKeyStats() works, even
       113077  +      ** if $L is available, whereKeyStats() is called for both ($P) and 
       113078  +      ** ($P:$L) and the larger of the two returned values used.
       113079  +      **
       113080  +      ** Similarly, iUpper is to be set to the estimate of the number of rows
       113081  +      ** less than the upper bound of the range query. Where the upper bound
       113082  +      ** is either ($P) or ($P:$U). Again, even if $U is available, both values
       113083  +      ** of iUpper are requested of whereKeyStats() and the smaller used.
       113084  +      */
       113085  +      tRowcnt iLower;
       113086  +      tRowcnt iUpper;
       113087  +
       113088  +      if( nEq==p->nKeyCol ){
       113089  +        aff = SQLITE_AFF_INTEGER;
       113090  +      }else{
       113091  +        aff = p->pTable->aCol[p->aiColumn[nEq]].affinity;
       113092  +      }
       113093  +      /* Determine iLower and iUpper using ($P) only. */
       113094  +      if( nEq==0 ){
       113095  +        iLower = 0;
       113096  +        iUpper = sqlite3LogEstToInt(p->aiRowLogEst[0]);
       113097  +      }else{
       113098  +        /* Note: this call could be optimized away - since the same values must 
       113099  +        ** have been requested when testing key $P in whereEqualScanEst().  */
112154 113100           whereKeyStats(pParse, p, pRec, 0, a);
112155         -        iNew = a[0] + ((pLower->eOperator & WO_GT) ? a[1] : 0);
112156         -        if( iNew>iLower ) iLower = iNew;
112157         -        nOut--;
112158         -      }
112159         -    }
112160         -
112161         -    /* If possible, improve on the iUpper estimate using ($P:$U). */
112162         -    if( pUpper ){
112163         -      int bOk;                    /* True if value is extracted from pExpr */
112164         -      Expr *pExpr = pUpper->pExpr->pRight;
112165         -      assert( (pUpper->eOperator & (WO_LT|WO_LE))!=0 );
112166         -      rc = sqlite3Stat4ProbeSetValue(pParse, p, &pRec, pExpr, aff, nEq, &bOk);
112167         -      if( rc==SQLITE_OK && bOk ){
112168         -        tRowcnt iNew;
112169         -        whereKeyStats(pParse, p, pRec, 1, a);
112170         -        iNew = a[0] + ((pUpper->eOperator & WO_LE) ? a[1] : 0);
112171         -        if( iNew<iUpper ) iUpper = iNew;
112172         -        nOut--;
112173         -      }
112174         -    }
112175         -
112176         -    pBuilder->pRec = pRec;
112177         -    if( rc==SQLITE_OK ){
112178         -      if( iUpper>iLower ){
112179         -        nNew = sqlite3LogEst(iUpper - iLower);
112180         -      }else{
112181         -        nNew = 10;        assert( 10==sqlite3LogEst(2) );
112182         -      }
112183         -      if( nNew<nOut ){
112184         -        nOut = nNew;
112185         -      }
112186         -      pLoop->nOut = (LogEst)nOut;
112187         -      WHERETRACE(0x10, ("range scan regions: %u..%u  est=%d\n",
112188         -                         (u32)iLower, (u32)iUpper, nOut));
112189         -      return SQLITE_OK;
       113101  +        iLower = a[0];
       113102  +        iUpper = a[0] + a[1];
       113103  +      }
       113104  +
       113105  +      /* If possible, improve on the iLower estimate using ($P:$L). */
       113106  +      if( pLower ){
       113107  +        int bOk;                    /* True if value is extracted from pExpr */
       113108  +        Expr *pExpr = pLower->pExpr->pRight;
       113109  +        assert( (pLower->eOperator & (WO_GT|WO_GE))!=0 );
       113110  +        rc = sqlite3Stat4ProbeSetValue(pParse, p, &pRec, pExpr, aff, nEq, &bOk);
       113111  +        if( rc==SQLITE_OK && bOk ){
       113112  +          tRowcnt iNew;
       113113  +          whereKeyStats(pParse, p, pRec, 0, a);
       113114  +          iNew = a[0] + ((pLower->eOperator & WO_GT) ? a[1] : 0);
       113115  +          if( iNew>iLower ) iLower = iNew;
       113116  +          nOut--;
       113117  +        }
       113118  +      }
       113119  +
       113120  +      /* If possible, improve on the iUpper estimate using ($P:$U). */
       113121  +      if( pUpper ){
       113122  +        int bOk;                    /* True if value is extracted from pExpr */
       113123  +        Expr *pExpr = pUpper->pExpr->pRight;
       113124  +        assert( (pUpper->eOperator & (WO_LT|WO_LE))!=0 );
       113125  +        rc = sqlite3Stat4ProbeSetValue(pParse, p, &pRec, pExpr, aff, nEq, &bOk);
       113126  +        if( rc==SQLITE_OK && bOk ){
       113127  +          tRowcnt iNew;
       113128  +          whereKeyStats(pParse, p, pRec, 1, a);
       113129  +          iNew = a[0] + ((pUpper->eOperator & WO_LE) ? a[1] : 0);
       113130  +          if( iNew<iUpper ) iUpper = iNew;
       113131  +          nOut--;
       113132  +        }
       113133  +      }
       113134  +
       113135  +      pBuilder->pRec = pRec;
       113136  +      if( rc==SQLITE_OK ){
       113137  +        if( iUpper>iLower ){
       113138  +          nNew = sqlite3LogEst(iUpper - iLower);
       113139  +        }else{
       113140  +          nNew = 10;        assert( 10==sqlite3LogEst(2) );
       113141  +        }
       113142  +        if( nNew<nOut ){
       113143  +          nOut = nNew;
       113144  +        }
       113145  +        pLoop->nOut = (LogEst)nOut;
       113146  +        WHERETRACE(0x10, ("range scan regions: %u..%u  est=%d\n",
       113147  +                           (u32)iLower, (u32)iUpper, nOut));
       113148  +        return SQLITE_OK;
       113149  +      }
       113150  +    }else{
       113151  +      int bDone = 0;
       113152  +      rc = whereRangeSkipScanEst(pParse, pLower, pUpper, pLoop, &bDone);
       113153  +      if( bDone ) return rc;
112190 113154       }
112191 113155     }
112192 113156   #else
112193 113157     UNUSED_PARAMETER(pParse);
112194 113158     UNUSED_PARAMETER(pBuilder);
112195 113159   #endif
112196 113160     assert( pLower || pUpper );
................................................................................
112241 113205     UnpackedRecord *pRec = pBuilder->pRec;
112242 113206     u8 aff;                   /* Column affinity */
112243 113207     int rc;                   /* Subfunction return code */
112244 113208     tRowcnt a[2];             /* Statistics */
112245 113209     int bOk;
112246 113210   
112247 113211     assert( nEq>=1 );
112248         -  assert( nEq<=(p->nKeyCol+1) );
       113212  +  assert( nEq<=p->nColumn );
112249 113213     assert( p->aSample!=0 );
112250 113214     assert( p->nSample>0 );
112251 113215     assert( pBuilder->nRecValid<nEq );
112252 113216   
112253 113217     /* If values are not available for all fields of the index to the left
112254 113218     ** of this one, no estimate can be made. Return SQLITE_NOTFOUND. */
112255 113219     if( pBuilder->nRecValid<(nEq-1) ){
112256 113220       return SQLITE_NOTFOUND;
112257 113221     }
112258 113222   
112259 113223     /* This is an optimization only. The call to sqlite3Stat4ProbeSetValue()
112260 113224     ** below would return the same value.  */
112261         -  if( nEq>p->nKeyCol ){
       113225  +  if( nEq>=p->nColumn ){
112262 113226       *pnRow = 1;
112263 113227       return SQLITE_OK;
112264 113228     }
112265 113229   
112266 113230     aff = p->pTable->aCol[p->aiColumn[nEq-1]].affinity;
112267 113231     rc = sqlite3Stat4ProbeSetValue(pParse, p, &pRec, pExpr, aff, nEq-1, &bOk);
112268 113232     pBuilder->pRec = pRec;
................................................................................
112448 113412       ){
112449 113413         testcase( iEq==0 );
112450 113414         testcase( bRev );
112451 113415         bRev = !bRev;
112452 113416       }
112453 113417       assert( pX->op==TK_IN );
112454 113418       iReg = iTarget;
112455         -    eType = sqlite3FindInIndex(pParse, pX, 0);
       113419  +    eType = sqlite3FindInIndex(pParse, pX, IN_INDEX_LOOP, 0);
112456 113420       if( eType==IN_INDEX_INDEX_DESC ){
112457 113421         testcase( bRev );
112458 113422         bRev = !bRev;
112459 113423       }
112460 113424       iTab = pX->iTable;
112461 113425       sqlite3VdbeAddOp2(v, bRev ? OP_Last : OP_Rewind, iTab, 0);
112462 113426       VdbeCoverageIf(v, bRev);
................................................................................
112685 113649     if( nEq==0 && (pLoop->wsFlags & (WHERE_BTM_LIMIT|WHERE_TOP_LIMIT))==0 ){
112686 113650       return 0;
112687 113651     }
112688 113652     sqlite3StrAccumInit(&txt, 0, 0, SQLITE_MAX_LENGTH);
112689 113653     txt.db = db;
112690 113654     sqlite3StrAccumAppend(&txt, " (", 2);
112691 113655     for(i=0; i<nEq; i++){
112692         -    char *z = (i==pIndex->nKeyCol ) ? "rowid" : aCol[aiColumn[i]].zName;
       113656  +    char *z = aiColumn[i] < 0 ? "rowid" : aCol[aiColumn[i]].zName;
112693 113657       if( i>=nSkip ){
112694 113658         explainAppendTerm(&txt, i, z, "=");
112695 113659       }else{
112696 113660         if( i ) sqlite3StrAccumAppend(&txt, " AND ", 5);
112697 113661         sqlite3StrAccumAppend(&txt, "ANY(", 4);
112698 113662         sqlite3StrAccumAppendAll(&txt, z);
112699 113663         sqlite3StrAccumAppend(&txt, ")", 1);
112700 113664       }
112701 113665     }
112702 113666   
112703 113667     j = i;
112704 113668     if( pLoop->wsFlags&WHERE_BTM_LIMIT ){
112705         -    char *z = (j==pIndex->nKeyCol ) ? "rowid" : aCol[aiColumn[j]].zName;
       113669  +    char *z = aiColumn[j] < 0 ? "rowid" : aCol[aiColumn[j]].zName;
112706 113670       explainAppendTerm(&txt, i++, z, ">");
112707 113671     }
112708 113672     if( pLoop->wsFlags&WHERE_TOP_LIMIT ){
112709         -    char *z = (j==pIndex->nKeyCol ) ? "rowid" : aCol[aiColumn[j]].zName;
       113673  +    char *z = aiColumn[j] < 0 ? "rowid" : aCol[aiColumn[j]].zName;
112710 113674       explainAppendTerm(&txt, i, z, "<");
112711 113675     }
112712 113676     sqlite3StrAccumAppend(&txt, ")", 1);
112713 113677     return sqlite3StrAccumFinish(&txt);
112714 113678   }
112715 113679   
112716 113680   /*
................................................................................
113349 114313       int regReturn = ++pParse->nMem;           /* Register used with OP_Gosub */
113350 114314       int regRowset = 0;                        /* Register for RowSet object */
113351 114315       int regRowid = 0;                         /* Register holding rowid */
113352 114316       int iLoopBody = sqlite3VdbeMakeLabel(v);  /* Start of loop body */
113353 114317       int iRetInit;                             /* Address of regReturn init */
113354 114318       int untestedTerms = 0;             /* Some terms not completely tested */
113355 114319       int ii;                            /* Loop counter */
       114320  +    u16 wctrlFlags;                    /* Flags for sub-WHERE clause */
113356 114321       Expr *pAndExpr = 0;                /* An ".. AND (...)" expression */
113357 114322       Table *pTab = pTabItem->pTab;
113358 114323      
113359 114324       pTerm = pLoop->aLTerm[0];
113360 114325       assert( pTerm!=0 );
113361 114326       assert( pTerm->eOperator & WO_OR );
113362 114327       assert( (pTerm->wtFlags & TERM_ORINFO)!=0 );
................................................................................
113444 114409         }
113445 114410       }
113446 114411   
113447 114412       /* Run a separate WHERE clause for each term of the OR clause.  After
113448 114413       ** eliminating duplicates from other WHERE clauses, the action for each
113449 114414       ** sub-WHERE clause is to to invoke the main loop body as a subroutine.
113450 114415       */
       114416  +    wctrlFlags =  WHERE_OMIT_OPEN_CLOSE | WHERE_AND_ONLY |
       114417  +                  WHERE_FORCE_TABLE | WHERE_ONETABLE_ONLY;
113451 114418       for(ii=0; ii<pOrWc->nTerm; ii++){
113452 114419         WhereTerm *pOrTerm = &pOrWc->a[ii];
113453 114420         if( pOrTerm->leftCursor==iCur || (pOrTerm->eOperator & WO_AND)!=0 ){
113454 114421           WhereInfo *pSubWInfo;           /* Info for single OR-term scan */
113455 114422           Expr *pOrExpr = pOrTerm->pExpr; /* Current OR clause term */
113456 114423           int j1 = 0;                     /* Address of jump operation */
113457 114424           if( pAndExpr && !ExprHasProperty(pOrExpr, EP_FromJoin) ){
113458 114425             pAndExpr->pLeft = pOrExpr;
113459 114426             pOrExpr = pAndExpr;
113460 114427           }
113461 114428           /* Loop through table entries that match term pOrTerm. */
113462 114429           pSubWInfo = sqlite3WhereBegin(pParse, pOrTab, pOrExpr, 0, 0,
113463         -                        WHERE_OMIT_OPEN_CLOSE | WHERE_AND_ONLY |
113464         -                        WHERE_FORCE_TABLE | WHERE_ONETABLE_ONLY, iCovCur);
       114430  +                                      wctrlFlags, iCovCur);
113465 114431           assert( pSubWInfo || pParse->nErr || db->mallocFailed );
113466 114432           if( pSubWInfo ){
113467 114433             WhereLoop *pSubLoop;
113468 114434             explainOneScan(
113469 114435                 pParse, pOrTab, &pSubWInfo->a[0], iLevel, pLevel->iFrom, 0
113470 114436             );
113471 114437             /* This is the sub-WHERE clause body.  First skip over
................................................................................
113548 114514             assert( (pSubLoop->wsFlags & WHERE_AUTO_INDEX)==0 );
113549 114515             if( (pSubLoop->wsFlags & WHERE_INDEXED)!=0
113550 114516              && (ii==0 || pSubLoop->u.btree.pIndex==pCov)
113551 114517              && (HasRowid(pTab) || !IsPrimaryKeyIndex(pSubLoop->u.btree.pIndex))
113552 114518             ){
113553 114519               assert( pSubWInfo->a[0].iIdxCur==iCovCur );
113554 114520               pCov = pSubLoop->u.btree.pIndex;
       114521  +            wctrlFlags |= WHERE_REOPEN_IDX;
113555 114522             }else{
113556 114523               pCov = 0;
113557 114524             }
113558 114525   
113559 114526             /* Finish the loop through table entries that match term pOrTerm. */
113560 114527             sqlite3WhereEnd(pSubWInfo);
113561 114528           }
................................................................................
113726 114693                   p->u.vtab.idxNum, p->u.vtab.idxStr, p->u.vtab.omitMask);
113727 114694       }else{
113728 114695         z = sqlite3_mprintf("(%d,%x)", p->u.vtab.idxNum, p->u.vtab.omitMask);
113729 114696       }
113730 114697       sqlite3DebugPrintf(" %-19s", z);
113731 114698       sqlite3_free(z);
113732 114699     }
113733         -  sqlite3DebugPrintf(" f %04x N %d", p->wsFlags, p->nLTerm);
       114700  +  sqlite3DebugPrintf(" f %05x N %d", p->wsFlags, p->nLTerm);
113734 114701     sqlite3DebugPrintf(" cost %d,%d,%d\n", p->rSetup, p->rRun, p->nOut);
113735 114702   #ifdef SQLITE_ENABLE_TREE_EXPLAIN
113736 114703     /* If the 0x100 bit of wheretracing is set, then show all of the constraint
113737 114704     ** expressions in the WhereLoop.aLTerm[] array.
113738 114705     */
113739 114706     if( p->nLTerm && (sqlite3WhereTrace & 0x100)!=0 ){  /* WHERETRACE 0x100 */
113740 114707       int i;
................................................................................
113962 114929       assert( p->rSetup==0 || pTemplate->rSetup==0 
113963 114930                    || p->rSetup==pTemplate->rSetup );
113964 114931   
113965 114932       /* whereLoopAddBtree() always generates and inserts the automatic index
113966 114933       ** case first.  Hence compatible candidate WhereLoops never have a larger
113967 114934       ** rSetup. Call this SETUP-INVARIANT */
113968 114935       assert( p->rSetup>=pTemplate->rSetup );
       114936  +
       114937  +    /* Any loop using an appliation-defined index (or PRIMARY KEY or
       114938  +    ** UNIQUE constraint) with one or more == constraints is better
       114939  +    ** than an automatic index. */
       114940  +    if( (p->wsFlags & WHERE_AUTO_INDEX)!=0
       114941  +     && (pTemplate->wsFlags & WHERE_INDEXED)!=0
       114942  +     && (pTemplate->wsFlags & WHERE_COLUMN_EQ)!=0
       114943  +     && (p->prereq & pTemplate->prereq)==pTemplate->prereq
       114944  +    ){
       114945  +      break;
       114946  +    }
113969 114947   
113970 114948       /* If existing WhereLoop p is better than pTemplate, pTemplate can be
113971 114949       ** discarded.  WhereLoop p is better if:
113972 114950       **   (1)  p has no more dependencies than pTemplate, and
113973 114951       **   (2)  p has an equal or lower cost than pTemplate
113974 114952       */
113975 114953       if( (p->prereq & pTemplate->prereq)==p->prereq    /* (1)  */
................................................................................
114087 115065       /* We will be overwriting WhereLoop p[].  But before we do, first
114088 115066       ** go through the rest of the list and delete any other entries besides
114089 115067       ** p[] that are also supplated by pTemplate */
114090 115068       WhereLoop **ppTail = &p->pNextLoop;
114091 115069       WhereLoop *pToDel;
114092 115070       while( *ppTail ){
114093 115071         ppTail = whereLoopFindLesser(ppTail, pTemplate);
114094         -      if( NEVER(ppTail==0) ) break;
       115072  +      if( ppTail==0 ) break;
114095 115073         pToDel = *ppTail;
114096 115074         if( pToDel==0 ) break;
114097 115075         *ppTail = pToDel->pNextLoop;
114098 115076   #if WHERETRACE_ENABLED /* 0x8 */
114099 115077         if( sqlite3WhereTrace & 0x8 ){
114100         -        sqlite3DebugPrintf("ins-del: ");
       115078  +        sqlite3DebugPrintf("ins-del:  ");
114101 115079           whereLoopPrint(pToDel, pBuilder->pWC);
114102 115080         }
114103 115081   #endif
114104 115082         whereLoopDelete(db, pToDel);
114105 115083       }
114106 115084     }
114107 115085     whereLoopXfer(db, p, pTemplate);
................................................................................
114143 115121       }
114144 115122       if( j<0 ){
114145 115123         pLoop->nOut += (pTerm->truthProb<=0 ? pTerm->truthProb : -1);
114146 115124       }
114147 115125     }
114148 115126   }
114149 115127   
       115128  +/*
       115129  +** Adjust the cost C by the costMult facter T.  This only occurs if
       115130  +** compiled with -DSQLITE_ENABLE_COSTMULT
       115131  +*/
       115132  +#ifdef SQLITE_ENABLE_COSTMULT
       115133  +# define ApplyCostMultiplier(C,T)  C += T
       115134  +#else
       115135  +# define ApplyCostMultiplier(C,T)
       115136  +#endif
       115137  +
114150 115138   /*
114151 115139   ** We have so far matched pBuilder->pNew->u.btree.nEq terms of the 
114152 115140   ** index pIndex. Try to match one more.
114153 115141   **
114154 115142   ** When this function is called, pBuilder->pNew->nOut contains the 
114155 115143   ** number of rows expected to be visited by filtering using the nEq 
114156 115144   ** terms only. If it is modified, this value is restored before this 
................................................................................
114193 115181     }else if( pProbe->tnum<=0 || (pSrc->jointype & JT_LEFT)!=0 ){
114194 115182       opMask = WO_EQ|WO_IN|WO_GT|WO_GE|WO_LT|WO_LE;
114195 115183     }else{
114196 115184       opMask = WO_EQ|WO_IN|WO_ISNULL|WO_GT|WO_GE|WO_LT|WO_LE;
114197 115185     }
114198 115186     if( pProbe->bUnordered ) opMask &= ~(WO_GT|WO_GE|WO_LT|WO_LE);
114199 115187   
114200         -  assert( pNew->u.btree.nEq<=pProbe->nKeyCol );
114201         -  if( pNew->u.btree.nEq < pProbe->nKeyCol ){
114202         -    iCol = pProbe->aiColumn[pNew->u.btree.nEq];
114203         -  }else{
114204         -    iCol = -1;
114205         -  }
       115188  +  assert( pNew->u.btree.nEq<pProbe->nColumn );
       115189  +  iCol = pProbe->aiColumn[pNew->u.btree.nEq];
       115190  +
114206 115191     pTerm = whereScanInit(&scan, pBuilder->pWC, pSrc->iCursor, iCol,
114207 115192                           opMask, pProbe);
114208 115193     saved_nEq = pNew->u.btree.nEq;
114209 115194     saved_nSkip = pNew->u.btree.nSkip;
114210 115195     saved_nLTerm = pNew->nLTerm;
114211 115196     saved_wsFlags = pNew->wsFlags;
114212 115197     saved_prereq = pNew->prereq;
................................................................................
114281 115266         }
114282 115267         assert( nIn>0 );  /* RHS always has 2 or more terms...  The parser
114283 115268                           ** changes "x IN (?)" into "x=?". */
114284 115269   
114285 115270       }else if( eOp & (WO_EQ) ){
114286 115271         pNew->wsFlags |= WHERE_COLUMN_EQ;
114287 115272         if( iCol<0 || (nInMul==0 && pNew->u.btree.nEq==pProbe->nKeyCol-1) ){
114288         -        if( iCol>=0 && pProbe->onError==OE_None ){
       115273  +        if( iCol>=0 && !IsUniqueIndex(pProbe) ){
114289 115274             pNew->wsFlags |= WHERE_UNQ_WANTED;
114290 115275           }else{
114291 115276             pNew->wsFlags |= WHERE_ONEROW;
114292 115277           }
114293 115278         }
114294 115279       }else if( eOp & WO_ISNULL ){
114295 115280         pNew->wsFlags |= WHERE_COLUMN_NULL;
................................................................................
114325 115310   
114326 115311         assert( pNew->nOut==saved_nOut );
114327 115312         if( pTerm->truthProb<=0 && iCol>=0 ){
114328 115313           assert( (eOp & WO_IN) || nIn==0 );
114329 115314           testcase( eOp & WO_IN );
114330 115315           pNew->nOut += pTerm->truthProb;
114331 115316           pNew->nOut -= nIn;
114332         -        pNew->wsFlags |= WHERE_LIKELIHOOD;
114333 115317         }else{
114334 115318   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
114335 115319           tRowcnt nOut = 0;
114336 115320           if( nInMul==0 
114337 115321            && pProbe->nSample 
114338 115322            && pNew->u.btree.nEq<=pProbe->nSampleCol
114339 115323            && OptimizationEnabled(db, SQLITE_Stat3) 
114340 115324            && ((eOp & WO_IN)==0 || !ExprHasProperty(pTerm->pExpr, EP_xIsSelect))
114341         -         && (pNew->wsFlags & WHERE_LIKELIHOOD)==0
114342 115325           ){
114343 115326             Expr *pExpr = pTerm->pExpr;
114344 115327             if( (eOp & (WO_EQ|WO_ISNULL))!=0 ){
114345 115328               testcase( eOp & WO_EQ );
114346 115329               testcase( eOp & WO_ISNULL );
114347 115330               rc = whereEqualScanEst(pParse, pBuilder, pExpr->pRight, &nOut);
114348 115331             }else{
114349 115332               rc = whereInScanEst(pParse, pBuilder, pExpr->x.pList, &nOut);
114350 115333             }
114351         -          assert( rc!=SQLITE_OK || nOut>0 );
114352 115334             if( rc==SQLITE_NOTFOUND ) rc = SQLITE_OK;
114353 115335             if( rc!=SQLITE_OK ) break;          /* Jump out of the pTerm loop */
114354 115336             if( nOut ){
114355 115337               pNew->nOut = sqlite3LogEst(nOut);
114356 115338               if( pNew->nOut>saved_nOut ) pNew->nOut = saved_nOut;
114357 115339               pNew->nOut -= nIn;
114358 115340             }
................................................................................
114376 115358       ** seek only. Then, if this is a non-covering index, add the cost of
114377 115359       ** visiting the rows in the main table.  */
114378 115360       rCostIdx = pNew->nOut + 1 + (15*pProbe->szIdxRow)/pSrc->pTab->szTabRow;
114379 115361       pNew->rRun = sqlite3LogEstAdd(rLogSize, rCostIdx);
114380 115362       if( (pNew->wsFlags & (WHERE_IDX_ONLY|WHERE_IPK))==0 ){
114381 115363         pNew->rRun = sqlite3LogEstAdd(pNew->rRun, pNew->nOut + 16);
114382 115364       }
       115365  +    ApplyCostMultiplier(pNew->rRun, pProbe->pTable->costMult);
114383 115366   
114384 115367       nOutUnadjusted = pNew->nOut;
114385 115368       pNew->rRun += nInMul + nIn;
114386 115369       pNew->nOut += nInMul + nIn;
114387 115370       whereLoopOutputAdjust(pBuilder->pWC, pNew);
114388 115371       rc = whereLoopInsert(pBuilder, pNew);
114389 115372   
................................................................................
114390 115373       if( pNew->wsFlags & WHERE_COLUMN_RANGE ){
114391 115374         pNew->nOut = saved_nOut;
114392 115375       }else{
114393 115376         pNew->nOut = nOutUnadjusted;
114394 115377       }
114395 115378   
114396 115379       if( (pNew->wsFlags & WHERE_TOP_LIMIT)==0
114397         -     && pNew->u.btree.nEq<(pProbe->nKeyCol + (pProbe->zName!=0))
       115380  +     && pNew->u.btree.nEq<pProbe->nColumn
114398 115381       ){
114399 115382         whereLoopAddBtreeIndex(pBuilder, pSrc, pProbe, nInMul+nIn);
114400 115383       }
114401 115384       pNew->nOut = saved_nOut;
114402 115385   #ifdef SQLITE_ENABLE_STAT3_OR_STAT4
114403 115386       pBuilder->nRecValid = nRecValid;
114404 115387   #endif
................................................................................
114495 115478   **
114496 115479   **     cost = nSeek * (log(nRow) + K * nVisit)          // covering index
114497 115480   **     cost = nSeek * (log(nRow) + (K+3.0) * nVisit)    // non-covering index
114498 115481   **
114499 115482   ** Normally, nSeek is 1. nSeek values greater than 1 come about if the 
114500 115483   ** WHERE clause includes "x IN (....)" terms used in place of "x=?". Or when 
114501 115484   ** implicit "x IN (SELECT x FROM tbl)" terms are added for skip-scans.
       115485  +**
       115486  +** The estimated values (nRow, nVisit, nSeek) often contain a large amount
       115487  +** of uncertainty.  For this reason, scoring is designed to pick plans that
       115488  +** "do the least harm" if the estimates are inaccurate.  For example, a
       115489  +** log(nRow) factor is omitted from a non-covering index scan in order to
       115490  +** bias the scoring in favor of using an index, since the worst-case
       115491  +** performance of using an index is far better than the worst-case performance
       115492  +** of a full table scan.
114502 115493   */
114503 115494   static int whereLoopAddBtree(
114504 115495     WhereLoopBuilder *pBuilder, /* WHERE clause information */
114505 115496     Bitmask mExtra              /* Extra prerequesites for using this table */
114506 115497   ){
114507 115498     WhereInfo *pWInfo;          /* WHERE analysis context */
114508 115499     Index *pProbe;              /* An index we are evaluating */
................................................................................
114537 115528       /* There is no INDEXED BY clause.  Create a fake Index object in local
114538 115529       ** variable sPk to represent the rowid primary key index.  Make this
114539 115530       ** fake index the first in a chain of Index objects with all of the real
114540 115531       ** indices to follow */
114541 115532       Index *pFirst;                  /* First of real indices on the table */
114542 115533       memset(&sPk, 0, sizeof(Index));
114543 115534       sPk.nKeyCol = 1;
       115535  +    sPk.nColumn = 1;
114544 115536       sPk.aiColumn = &aiColumnPk;
114545 115537       sPk.aiRowLogEst = aiRowEstPk;
114546 115538       sPk.onError = OE_Replace;
114547 115539       sPk.pTable = pTab;
114548 115540       sPk.szIdxRow = pTab->szTabRow;
114549 115541       aiRowEstPk[0] = pTab->nRowLogEst;
114550 115542       aiRowEstPk[1] = 0;
................................................................................
114581 115573           pNew->u.btree.pIndex = 0;
114582 115574           pNew->nLTerm = 1;
114583 115575           pNew->aLTerm[0] = pTerm;
114584 115576           /* TUNING: One-time cost for computing the automatic index is
114585 115577           ** approximately 7*N*log2(N) where N is the number of rows in
114586 115578           ** the table being indexed. */
114587 115579           pNew->rSetup = rLogSize + rSize + 28;  assert( 28==sqlite3LogEst(7) );
       115580  +        ApplyCostMultiplier(pNew->rSetup, pTab->costMult);
114588 115581           /* TUNING: Each index lookup yields 20 rows in the table.  This
114589 115582           ** is more than the usual guess of 10 rows, since we have no way
114590 115583           ** of knowning how selective the index will ultimately be.  It would
114591 115584           ** not be unreasonable to make this value much larger. */
114592 115585           pNew->nOut = 43;  assert( 43==sqlite3LogEst(20) );
114593 115586           pNew->rRun = sqlite3LogEstAdd(rLogSize,pNew->nOut);
114594 115587           pNew->wsFlags = WHERE_AUTO_INDEX;
................................................................................
114622 115615         /* Integer primary key index */
114623 115616         pNew->wsFlags = WHERE_IPK;
114624 115617   
114625 115618         /* Full table scan */
114626 115619         pNew->iSortIdx = b ? iSortIdx : 0;
114627 115620         /* TUNING: Cost of full table scan is (N*3.0). */
114628 115621         pNew->rRun = rSize + 16;
       115622  +      ApplyCostMultiplier(pNew->rRun, pTab->costMult);
114629 115623         whereLoopOutputAdjust(pWC, pNew);
114630 115624         rc = whereLoopInsert(pBuilder, pNew);
114631 115625         pNew->nOut = rSize;
114632 115626         if( rc ) break;
114633 115627       }else{
114634 115628         Bitmask m;
114635 115629         if( pProbe->isCovering ){
................................................................................
114657 115651           ** between 1.1 and 3.0, depending on the relative sizes of the
114658 115652           ** index and table rows. If this is a non-covering index scan,
114659 115653           ** also add the cost of visiting table rows (N*3.0).  */
114660 115654           pNew->rRun = rSize + 1 + (15*pProbe->szIdxRow)/pTab->szTabRow;
114661 115655           if( m!=0 ){
114662 115656             pNew->rRun = sqlite3LogEstAdd(pNew->rRun, rSize+16);
114663 115657           }
114664         -
       115658  +        ApplyCostMultiplier(pNew->rRun, pTab->costMult);
114665 115659           whereLoopOutputAdjust(pWC, pNew);
114666 115660           rc = whereLoopInsert(pBuilder, pNew);
114667 115661           pNew->nOut = rSize;
114668 115662           if( rc ) break;
114669 115663         }
114670 115664       }
114671 115665   
................................................................................
115127 116121         }else if( (pIndex = pLoop->u.btree.pIndex)==0 || pIndex->bUnordered ){
115128 116122           return 0;
115129 116123         }else{
115130 116124           nKeyCol = pIndex->nKeyCol;
115131 116125           nColumn = pIndex->nColumn;
115132 116126           assert( nColumn==nKeyCol+1 || !HasRowid(pIndex->pTable) );
115133 116127           assert( pIndex->aiColumn[nColumn-1]==(-1) || !HasRowid(pIndex->pTable));
115134         -        isOrderDistinct = pIndex->onError!=OE_None;
       116128  +        isOrderDistinct = IsUniqueIndex(pIndex);
115135 116129         }
115136 116130   
115137 116131         /* Loop through all columns of the index and deal with the ones
115138 116132         ** that are not constrained by == or IN.
115139 116133         */
115140 116134         rev = revSet = 0;
115141 116135         distinctColumns = 0;
................................................................................
115295 116289     int i;
115296 116290     for(i=0; i<nLoop; i++){ zName[i] = pPath->aLoop[i]->cId; }
115297 116291     if( pLast ) zName[i++] = pLast->cId;
115298 116292     zName[i] = 0;
115299 116293     return zName;
115300 116294   }
115301 116295   #endif
       116296  +
       116297  +/*
       116298  +** Return the cost of sorting nRow rows, assuming that the keys have 
       116299  +** nOrderby columns and that the first nSorted columns are already in
       116300  +** order.
       116301  +*/
       116302  +static LogEst whereSortingCost(
       116303  +  WhereInfo *pWInfo,
       116304  +  LogEst nRow,
       116305  +  int nOrderBy,
       116306  +  int nSorted
       116307  +){
       116308  +  /* TUNING: Estimated cost of a full external sort, where N is 
       116309  +  ** the number of rows to sort is:
       116310  +  **
       116311  +  **   cost = (3.0 * N * log(N)).
       116312  +  ** 
       116313  +  ** Or, if the order-by clause has X terms but only the last Y 
       116314  +  ** terms are out of order, then block-sorting will reduce the 
       116315  +  ** sorting cost to:
       116316  +  **
       116317  +  **   cost = (3.0 * N * log(N)) * (Y/X)
       116318  +  **
       116319  +  ** The (Y/X) term is implemented using stack variable rScale
       116320  +  ** below.  */
       116321  +  LogEst rScale, rSortCost;
       116322  +  assert( nOrderBy>0 && 66==sqlite3LogEst(100) );
       116323  +  rScale = sqlite3LogEst((nOrderBy-nSorted)*100/nOrderBy) - 66;
       116324  +  rSortCost = nRow + estLog(nRow) + rScale + 16;
       116325  +
       116326  +  /* TUNING: The cost of implementing DISTINCT using a B-TREE is
       116327  +  ** similar but with a larger constant of proportionality. 
       116328  +  ** Multiply by an additional factor of 3.0.  */
       116329  +  if( pWInfo->wctrlFlags & WHERE_WANT_DISTINCT ){
       116330  +    rSortCost += 16;
       116331  +  }
       116332  +
       116333  +  return rSortCost;
       116334  +}
115302 116335   
115303 116336   /*
115304 116337   ** Given the list of WhereLoop objects at pWInfo->pLoops, this routine
115305 116338   ** attempts to find the lowest cost path that visits each WhereLoop
115306 116339   ** once.  This path is then loaded into the pWInfo->a[].pWLoop fields.
115307 116340   **
115308 116341   ** Assume that the total number of output rows that will need to be sorted
................................................................................
115317 116350     int nLoop;                /* Number of terms in the join */
115318 116351     Parse *pParse;            /* Parsing context */
115319 116352     sqlite3 *db;              /* The database connection */
115320 116353     int iLoop;                /* Loop counter over the terms of the join */
115321 116354     int ii, jj;               /* Loop counters */
115322 116355     int mxI = 0;              /* Index of next entry to replace */
115323 116356     int nOrderBy;             /* Number of ORDER BY clause terms */
115324         -  LogEst rCost;             /* Cost of a path */
115325         -  LogEst nOut;              /* Number of outputs */
115326 116357     LogEst mxCost = 0;        /* Maximum cost of a set of paths */
115327         -  LogEst mxOut = 0;         /* Maximum nOut value on the set of paths */
       116358  +  LogEst mxUnsorted = 0;    /* Maximum unsorted cost of a set of path */
115328 116359     int nTo, nFrom;           /* Number of valid entries in aTo[] and aFrom[] */
115329 116360     WherePath *aFrom;         /* All nFrom paths at the previous level */
115330 116361     WherePath *aTo;           /* The nTo best paths at the current level */
115331 116362     WherePath *pFrom;         /* An element of aFrom[] that we are working on */
115332 116363     WherePath *pTo;           /* An element of aTo[] that we are working on */
115333 116364     WhereLoop *pWLoop;        /* One of the WhereLoop objects */
115334 116365     WhereLoop **pX;           /* Used to divy up the pSpace memory */
       116366  +  LogEst *aSortCost = 0;    /* Sorting and partial sorting costs */
115335 116367     char *pSpace;             /* Temporary memory used by this routine */
       116368  +  int nSpace;               /* Bytes of space allocated at pSpace */
115336 116369   
115337 116370     pParse = pWInfo->pParse;
115338 116371     db = pParse->db;
115339 116372     nLoop = pWInfo->nLevel;
115340 116373     /* TUNING: For simple queries, only the best path is tracked.
115341 116374     ** For 2-way joins, the 5 best paths are followed.
115342 116375     ** For joins of 3 or more tables, track the 10 best paths */
115343 116376     mxChoice = (nLoop<=1) ? 1 : (nLoop==2 ? 5 : 10);
115344 116377     assert( nLoop<=pWInfo->pTabList->nSrc );
115345         -  WHERETRACE(0x002, ("---- begin solver\n"));
       116378  +  WHERETRACE(0x002, ("---- begin solver.  (nRowEst=%d)\n", nRowEst));
115346 116379   
115347         -  /* Allocate and initialize space for aTo and aFrom */
115348         -  ii = (sizeof(WherePath)+sizeof(WhereLoop*)*nLoop)*mxChoice*2;
115349         -  pSpace = sqlite3DbMallocRaw(db, ii);
       116380  +  /* If nRowEst is zero and there is an ORDER BY clause, ignore it. In this
       116381  +  ** case the purpose of this call is to estimate the number of rows returned
       116382  +  ** by the overall query. Once this estimate has been obtained, the caller
       116383  +  ** will invoke this function a second time, passing the estimate as the
       116384  +  ** nRowEst parameter.  */
       116385  +  if( pWInfo->pOrderBy==0 || nRowEst==0 ){
       116386  +    nOrderBy = 0;
       116387  +  }else{
       116388  +    nOrderBy = pWInfo->pOrderBy->nExpr;
       116389  +  }
       116390  +
       116391  +  /* Allocate and initialize space for aTo, aFrom and aSortCost[] */
       116392  +  nSpace = (sizeof(WherePath)+sizeof(WhereLoop*)*nLoop)*mxChoice*2;
       116393  +  nSpace += sizeof(LogEst) * nOrderBy;
       116394  +  pSpace = sqlite3DbMallocRaw(db, nSpace);
115350 116395     if( pSpace==0 ) return SQLITE_NOMEM;
115351 116396     aTo = (WherePath*)pSpace;
115352 116397     aFrom = aTo+mxChoice;
115353 116398     memset(aFrom, 0, sizeof(aFrom[0]));
115354 116399     pX = (WhereLoop**)(aFrom+mxChoice);
115355 116400     for(ii=mxChoice*2, pFrom=aTo; ii>0; ii--, pFrom++, pX += nLoop){
115356 116401       pFrom->aLoop = pX;
115357 116402     }
       116403  +  if( nOrderBy ){
       116404  +    /* If there is an ORDER BY clause and it is not being ignored, set up
       116405  +    ** space for the aSortCost[] array. Each element of the aSortCost array
       116406  +    ** is either zero - meaning it has not yet been initialized - or the
       116407  +    ** cost of sorting nRowEst rows of data where the first X terms of
       116408  +    ** the ORDER BY clause are already in order, where X is the array 
       116409  +    ** index.  */
       116410  +    aSortCost = (LogEst*)pX;
       116411  +    memset(aSortCost, 0, sizeof(LogEst) * nOrderBy);
       116412  +  }
       116413  +  assert( aSortCost==0 || &pSpace[nSpace]==(char*)&aSortCost[nOrderBy] );
       116414  +  assert( aSortCost!=0 || &pSpace[nSpace]==(char*)pX );
115358 116415   
115359 116416     /* Seed the search with a single WherePath containing zero WhereLoops.
115360 116417     **
115361 116418     ** TUNING: Do not let the number of iterations go above 25.  If the cost
115362 116419     ** of computing an automatic index is not paid back within the first 25
115363 116420     ** rows, then do not use the automatic index. */
115364 116421     aFrom[0].nRow = MIN(pParse->nQueryLoop, 46);  assert( 46==sqlite3LogEst(25) );
115365 116422     nFrom = 1;
115366         -
115367         -  /* Precompute the cost of sorting the final result set, if the caller
115368         -  ** to sqlite3WhereBegin() was concerned about sorting */
115369         -  if( pWInfo->pOrderBy==0 || nRowEst==0 ){
115370         -    aFrom[0].isOrdered = 0;
115371         -    nOrderBy = 0;
115372         -  }else{
115373         -    aFrom[0].isOrdered = nLoop>0 ? -1 : 1;
115374         -    nOrderBy = pWInfo->pOrderBy->nExpr;
       116423  +  assert( aFrom[0].isOrdered==0 );
       116424  +  if( nOrderBy ){
       116425  +    /* If nLoop is zero, then there are no FROM terms in the query. Since
       116426  +    ** in this case the query may return a maximum of one row, the results
       116427  +    ** are already in the requested order. Set isOrdered to nOrderBy to
       116428  +    ** indicate this. Or, if nLoop is greater than zero, set isOrdered to
       116429  +    ** -1, indicating that the result set may or may not be ordered, 
       116430  +    ** depending on the loops added to the current plan.  */
       116431  +    aFrom[0].isOrdered = nLoop>0 ? -1 : nOrderBy;
115375 116432     }
115376 116433   
115377 116434     /* Compute successively longer WherePaths using the previous generation
115378 116435     ** of WherePaths as the basis for the next.  Keep track of the mxChoice
115379 116436     ** best paths at each generation */
115380 116437     for(iLoop=0; iLoop<nLoop; iLoop++){
115381 116438       nTo = 0;
115382 116439       for(ii=0, pFrom=aFrom; ii<nFrom; ii++, pFrom++){
115383 116440         for(pWLoop=pWInfo->pLoops; pWLoop; pWLoop=pWLoop->pNextLoop){
115384         -        Bitmask maskNew;
115385         -        Bitmask revMask = 0;
115386         -        i8 isOrdered = pFrom->isOrdered;
       116441  +        LogEst nOut;                      /* Rows visited by (pFrom+pWLoop) */
       116442  +        LogEst rCost;                     /* Cost of path (pFrom+pWLoop) */
       116443  +        LogEst rUnsorted;                 /* Unsorted cost of (pFrom+pWLoop) */
       116444  +        i8 isOrdered = pFrom->isOrdered;  /* isOrdered for (pFrom+pWLoop) */
       116445  +        Bitmask maskNew;                  /* Mask of src visited by (..) */
       116446  +        Bitmask revMask = 0;              /* Mask of rev-order loops for (..) */
       116447  +
115387 116448           if( (pWLoop->prereq & ~pFrom->maskLoop)!=0 ) continue;
115388 116449           if( (pWLoop->maskSelf & pFrom->maskLoop)!=0 ) continue;
115389 116450           /* At this point, pWLoop is a candidate to be the next loop. 
115390 116451           ** Compute its cost */
115391         -        rCost = sqlite3LogEstAdd(pWLoop->rSetup,pWLoop->rRun + pFrom->nRow);
115392         -        rCost = sqlite3LogEstAdd(rCost, pFrom->rCost);
       116452  +        rUnsorted = sqlite3LogEstAdd(pWLoop->rSetup,pWLoop->rRun + pFrom->nRow);
       116453  +        rUnsorted = sqlite3LogEstAdd(rUnsorted, pFrom->rUnsorted);
115393 116454           nOut = pFrom->nRow + pWLoop->nOut;
115394 116455           maskNew = pFrom->maskLoop | pWLoop->maskSelf;
115395 116456           if( isOrdered<0 ){
115396 116457             isOrdered = wherePathSatisfiesOrderBy(pWInfo,
115397 116458                          pWInfo->pOrderBy, pFrom, pWInfo->wctrlFlags,
115398 116459                          iLoop, pWLoop, &revMask);
115399         -          if( isOrdered>=0 && isOrdered<nOrderBy ){
115400         -            /* TUNING: Estimated cost of a full external sort, where N is 
115401         -            ** the number of rows to sort is:
115402         -            **
115403         -            **   cost = (3.0 * N * log(N)).
115404         -            ** 
115405         -            ** Or, if the order-by clause has X terms but only the last Y 
115406         -            ** terms are out of order, then block-sorting will reduce the 
115407         -            ** sorting cost to:
115408         -            **
115409         -            **   cost = (3.0 * N * log(N)) * (Y/X)
115410         -            **
115411         -            ** The (Y/X) term is implemented using stack variable rScale
115412         -            ** below.  */
115413         -            LogEst rScale, rSortCost;
115414         -            assert( nOrderBy>0 && 66==sqlite3LogEst(100) );
115415         -            rScale = sqlite3LogEst((nOrderBy-isOrdered)*100/nOrderBy) - 66;
115416         -            rSortCost = nRowEst + estLog(nRowEst) + rScale + 16;
115417         -
115418         -            /* TUNING: The cost of implementing DISTINCT using a B-TREE is
115419         -            ** similar but with a larger constant of proportionality. 
115420         -            ** Multiply by an additional factor of 3.0.  */
115421         -            if( pWInfo->wctrlFlags & WHERE_WANT_DISTINCT ){
115422         -              rSortCost += 16;
115423         -            }
115424         -            WHERETRACE(0x002,
115425         -               ("---- sort cost=%-3d (%d/%d) increases cost %3d to %-3d\n",
115426         -                rSortCost, (nOrderBy-isOrdered), nOrderBy, rCost,
115427         -                sqlite3LogEstAdd(rCost,rSortCost)));
115428         -            rCost = sqlite3LogEstAdd(rCost, rSortCost);
115429         -          }
115430 116460           }else{
115431 116461             revMask = pFrom->revLoop;
115432 116462           }
115433         -        /* Check to see if pWLoop should be added to the mxChoice best so far */
       116463  +        if( isOrdered>=0 && isOrdered<nOrderBy ){
       116464  +          if( aSortCost[isOrdered]==0 ){
       116465  +            aSortCost[isOrdered] = whereSortingCost(
       116466  +                pWInfo, nRowEst, nOrderBy, isOrdered
       116467  +            );
       116468  +          }
       116469  +          rCost = sqlite3LogEstAdd(rUnsorted, aSortCost[isOrdered]);
       116470  +
       116471  +          WHERETRACE(0x002,
       116472  +              ("---- sort cost=%-3d (%d/%d) increases cost %3d to %-3d\n",
       116473  +               aSortCost[isOrdered], (nOrderBy-isOrdered), nOrderBy, 
       116474  +               rUnsorted, rCost));
       116475  +        }else{
       116476  +          rCost = rUnsorted;
       116477  +        }
       116478  +
       116479  +        /* Check to see if pWLoop should be added to the set of
       116480  +        ** mxChoice best-so-far paths.
       116481  +        **
       116482  +        ** First look for an existing path among best-so-far paths
       116483  +        ** that covers the same set of loops and has the same isOrdered
       116484  +        ** setting as the current path candidate.
       116485  +        **
       116486  +        ** The term "((pTo->isOrdered^isOrdered)&0x80)==0" is equivalent
       116487  +        ** to (pTo->isOrdered==(-1))==(isOrdered==(-1))" for the range
       116488  +        ** of legal values for isOrdered, -1..64.
       116489  +        */
115434 116490           for(jj=0, pTo=aTo; jj<nTo; jj++, pTo++){
115435 116491             if( pTo->maskLoop==maskNew
115436         -           && ((pTo->isOrdered^isOrdered)&80)==0
115437         -           && ((pTo->rCost<=rCost && pTo->nRow<=nOut) ||
115438         -                (pTo->rCost>=rCost && pTo->nRow>=nOut))
       116492  +           && ((pTo->isOrdered^isOrdered)&0x80)==0
115439 116493             ){
115440 116494               testcase( jj==nTo-1 );
115441 116495               break;
115442 116496             }
115443 116497           }
115444 116498           if( jj>=nTo ){
115445         -          if( nTo>=mxChoice && rCost>=mxCost ){
       116499  +          /* None of the existing best-so-far paths match the candidate. */
       116500  +          if( nTo>=mxChoice
       116501  +           && (rCost>mxCost || (rCost==mxCost && rUnsorted>=mxUnsorted))
       116502  +          ){
       116503  +            /* The current candidate is no better than any of the mxChoice
       116504  +            ** paths currently in the best-so-far buffer.  So discard
       116505  +            ** this candidate as not viable. */
115446 116506   #ifdef WHERETRACE_ENABLED /* 0x4 */
115447 116507               if( sqlite3WhereTrace&0x4 ){
115448 116508                 sqlite3DebugPrintf("Skip   %s cost=%-3d,%3d order=%c\n",
115449 116509                     wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
115450 116510                     isOrdered>=0 ? isOrdered+'0' : '?');
115451 116511               }
115452 116512   #endif
115453 116513               continue;
115454 116514             }
115455         -          /* Add a new Path to the aTo[] set */
       116515  +          /* If we reach this points it means that the new candidate path
       116516  +          ** needs to be added to the set of best-so-far paths. */
115456 116517             if( nTo<mxChoice ){
115457 116518               /* Increase the size of the aTo set by one */
115458 116519               jj = nTo++;
115459 116520             }else{
115460 116521               /* New path replaces the prior worst to keep count below mxChoice */
115461 116522               jj = mxI;
115462 116523             }
................................................................................
115465 116526             if( sqlite3WhereTrace&0x4 ){
115466 116527               sqlite3DebugPrintf("New    %s cost=%-3d,%3d order=%c\n",
115467 116528                   wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
115468 116529                   isOrdered>=0 ? isOrdered+'0' : '?');
115469 116530             }
115470 116531   #endif
115471 116532           }else{
115472         -          if( pTo->rCost<=rCost && pTo->nRow<=nOut ){
       116533  +          /* Control reaches here if best-so-far path pTo=aTo[jj] covers the
       116534  +          ** same set of loops and has the sam isOrdered setting as the
       116535  +          ** candidate path.  Check to see if the candidate should replace
       116536  +          ** pTo or if the candidate should be skipped */
       116537  +          if( pTo->rCost<rCost || (pTo->rCost==rCost && pTo->nRow<=nOut) ){
115473 116538   #ifdef WHERETRACE_ENABLED /* 0x4 */
115474 116539               if( sqlite3WhereTrace&0x4 ){
115475 116540                 sqlite3DebugPrintf(
115476 116541                     "Skip   %s cost=%-3d,%3d order=%c",
115477 116542                     wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
115478 116543                     isOrdered>=0 ? isOrdered+'0' : '?');
115479 116544                 sqlite3DebugPrintf("   vs %s cost=%-3d,%d order=%c\n",
115480 116545                     wherePathName(pTo, iLoop+1, 0), pTo->rCost, pTo->nRow,
115481 116546                     pTo->isOrdered>=0 ? pTo->isOrdered+'0' : '?');
115482 116547               }
115483 116548   #endif
       116549  +            /* Discard the candidate path from further consideration */
115484 116550               testcase( pTo->rCost==rCost );
115485 116551               continue;
115486 116552             }
115487 116553             testcase( pTo->rCost==rCost+1 );
115488         -          /* A new and better score for a previously created equivalent path */
       116554  +          /* Control reaches here if the candidate path is better than the
       116555  +          ** pTo path.  Replace pTo with the candidate. */
115489 116556   #ifdef WHERETRACE_ENABLED /* 0x4 */
115490 116557             if( sqlite3WhereTrace&0x4 ){
115491 116558               sqlite3DebugPrintf(
115492 116559                   "Update %s cost=%-3d,%3d order=%c",
115493 116560                   wherePathName(pFrom, iLoop, pWLoop), rCost, nOut,
115494 116561                   isOrdered>=0 ? isOrdered+'0' : '?');
115495 116562               sqlite3DebugPrintf("  was %s cost=%-3d,%3d order=%c\n",
................................................................................
115499 116566   #endif
115500 116567           }
115501 116568           /* pWLoop is a winner.  Add it to the set of best so far */
115502 116569           pTo->maskLoop = pFrom->maskLoop | pWLoop->maskSelf;
115503 116570           pTo->revLoop = revMask;
115504 116571           pTo->nRow = nOut;
115505 116572           pTo->rCost = rCost;
       116573  +        pTo->rUnsorted = rUnsorted;
115506 116574           pTo->isOrdered = isOrdered;
115507 116575           memcpy(pTo->aLoop, pFrom->aLoop, sizeof(WhereLoop*)*iLoop);
115508 116576           pTo->aLoop[iLoop] = pWLoop;
115509 116577           if( nTo>=mxChoice ){
115510 116578             mxI = 0;
115511 116579             mxCost = aTo[0].rCost;
115512         -          mxOut = aTo[0].nRow;
       116580  +          mxUnsorted = aTo[0].nRow;
115513 116581             for(jj=1, pTo=&aTo[1]; jj<mxChoice; jj++, pTo++){
115514         -            if( pTo->rCost>mxCost || (pTo->rCost==mxCost && pTo->nRow>mxOut) ){
       116582  +            if( pTo->rCost>mxCost 
       116583  +             || (pTo->rCost==mxCost && pTo->rUnsorted>mxUnsorted) 
       116584  +            ){
115515 116585                 mxCost = pTo->rCost;
115516         -              mxOut = pTo->nRow;
       116586  +              mxUnsorted = pTo->rUnsorted;
115517 116587                 mxI = jj;
115518 116588               }
115519 116589             }
115520 116590           }
115521 116591         }
115522 116592       }
115523 116593   
................................................................................
115647 116717       pLoop->u.btree.nEq = 1;
115648 116718       /* TUNING: Cost of a rowid lookup is 10 */
115649 116719       pLoop->rRun = 33;  /* 33==sqlite3LogEst(10) */
115650 116720     }else{
115651 116721       for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
115652 116722         assert( pLoop->aLTermSpace==pLoop->aLTerm );
115653 116723         assert( ArraySize(pLoop->aLTermSpace)==4 );
115654         -      if( pIdx->onError==OE_None 
       116724  +      if( !IsUniqueIndex(pIdx)
115655 116725          || pIdx->pPartIdxWhere!=0 
115656 116726          || pIdx->nKeyCol>ArraySize(pLoop->aLTermSpace) 
115657 116727         ) continue;
115658 116728         for(j=0; j<pIdx->nKeyCol; j++){
115659 116729           pTerm = findTerm(pWC, iCur, pIdx->aiColumn[j], 0, WO_EQ, pIdx);
115660 116730           if( pTerm==0 ) break;
115661 116731           pLoop->aLTerm[j] = pTerm;
................................................................................
116137 117207             iIndexCur++;
116138 117208             pJ = pJ->pNext;
116139 117209           }
116140 117210           op = OP_OpenWrite;
116141 117211           pWInfo->aiCurOnePass[1] = iIndexCur;
116142 117212         }else if( iIdxCur && (wctrlFlags & WHERE_ONETABLE_ONLY)!=0 ){
116143 117213           iIndexCur = iIdxCur;
       117214  +        if( wctrlFlags & WHERE_REOPEN_IDX ) op = OP_ReopenIdx;
116144 117215         }else{
116145 117216           iIndexCur = pParse->nTab++;
116146 117217         }
116147 117218         pLevel->iIdxCur = iIndexCur;
116148 117219         assert( pIx->pSchema==pTab->pSchema );
116149 117220         assert( iIndexCur>=0 );
116150 117221         if( op ){
................................................................................
120461 121532       case '0': case '1': case '2': case '3': case '4':
120462 121533       case '5': case '6': case '7': case '8': case '9': {
120463 121534         testcase( z[0]=='0' );  testcase( z[0]=='1' );  testcase( z[0]=='2' );
120464 121535         testcase( z[0]=='3' );  testcase( z[0]=='4' );  testcase( z[0]=='5' );
120465 121536         testcase( z[0]=='6' );  testcase( z[0]=='7' );  testcase( z[0]=='8' );
120466 121537         testcase( z[0]=='9' );
120467 121538         *tokenType = TK_INTEGER;
       121539  +#ifndef SQLITE_OMIT_HEX_INTEGER
       121540  +      if( z[0]=='0' && (z[1]=='x' || z[1]=='X') && sqlite3Isxdigit(z[2]) ){
       121541  +        for(i=3; sqlite3Isxdigit(z[i]); i++){}
       121542  +        return i;
       121543  +      }
       121544  +#endif
120468 121545         for(i=0; sqlite3Isdigit(z[i]); i++){}
120469 121546   #ifndef SQLITE_OMIT_FLOATING_POINT
120470 121547         if( z[i]=='.' ){
120471 121548           i++;
120472 121549           while( sqlite3Isdigit(z[i]) ){ i++; }
120473 121550           *tokenType = TK_FLOAT;
120474 121551         }
................................................................................
121908 122985   }
121909 122986   
121910 122987   /*
121911 122988   ** Close an existing SQLite database
121912 122989   */
121913 122990   static int sqlite3Close(sqlite3 *db, int forceZombie){
121914 122991     if( !db ){
       122992  +    /* EVIDENCE-OF: R-63257-11740 Calling sqlite3_close() or
       122993  +    ** sqlite3_close_v2() with a NULL pointer argument is a harmless no-op. */
121915 122994       return SQLITE_OK;
121916 122995     }
121917 122996     if( !sqlite3SafetyCheckSickOrOk(db) ){
121918 122997       return SQLITE_MISUSE_BKPT;
121919 122998     }
121920 122999     sqlite3_mutex_enter(db->mutex);
121921 123000   
................................................................................
122137 123216     }
122138 123217   }
122139 123218   
122140 123219   /*
122141 123220   ** Return a static string containing the name corresponding to the error code
122142 123221   ** specified in the argument.
122143 123222   */
122144         -#if defined(SQLITE_TEST)
       123223  +#if (defined(SQLITE_DEBUG) && SQLITE_OS_WIN) || defined(SQLITE_TEST)
122145 123224   SQLITE_PRIVATE const char *sqlite3ErrName(int rc){
122146 123225     const char *zName = 0;
122147 123226     int i, origRc = rc;
122148 123227     for(i=0; i<2 && zName==0; i++, rc &= 0xff){
122149 123228       switch( rc ){
122150 123229         case SQLITE_OK:                 zName = "SQLITE_OK";                break;
122151 123230         case SQLITE_ERROR:              zName = "SQLITE_ERROR";             break;
................................................................................
122172 123251         case SQLITE_IOERR_FSYNC:        zName = "SQLITE_IOERR_FSYNC";       break;
122173 123252         case SQLITE_IOERR_DIR_FSYNC:    zName = "SQLITE_IOERR_DIR_FSYNC";   break;
122174 123253         case SQLITE_IOERR_TRUNCATE:     zName = "SQLITE_IOERR_TRUNCATE";    break;
122175 123254         case SQLITE_IOERR_FSTAT:        zName = "SQLITE_IOERR_FSTAT";       break;
122176 123255         case SQLITE_IOERR_UNLOCK:       zName = "SQLITE_IOERR_UNLOCK";      break;
122177 123256         case SQLITE_IOERR_RDLOCK:       zName = "SQLITE_IOERR_RDLOCK";      break;
122178 123257         case SQLITE_IOERR_DELETE:       zName = "SQLITE_IOERR_DELETE";      break;
122179         -      case SQLITE_IOERR_BLOCKED:      zName = "SQLITE_IOERR_BLOCKED";     break;
122180 123258         case SQLITE_IOERR_NOMEM:        zName = "SQLITE_IOERR_NOMEM";       break;
122181 123259         case SQLITE_IOERR_ACCESS:       zName = "SQLITE_IOERR_ACCESS";      break;
122182 123260         case SQLITE_IOERR_CHECKRESERVEDLOCK:
122183 123261                                   zName = "SQLITE_IOERR_CHECKRESERVEDLOCK"; break;
122184 123262         case SQLITE_IOERR_LOCK:         zName = "SQLITE_IOERR_LOCK";        break;
122185 123263         case SQLITE_IOERR_CLOSE:        zName = "SQLITE_IOERR_CLOSE";       break;
122186 123264         case SQLITE_IOERR_DIR_CLOSE:    zName = "SQLITE_IOERR_DIR_CLOSE";   break;
................................................................................
123157 124235     SQLITE_MAX_COLUMN,
123158 124236     SQLITE_MAX_EXPR_DEPTH,
123159 124237     SQLITE_MAX_COMPOUND_SELECT,
123160 124238     SQLITE_MAX_VDBE_OP,
123161 124239     SQLITE_MAX_FUNCTION_ARG,
123162 124240     SQLITE_MAX_ATTACHED,
123163 124241     SQLITE_MAX_LIKE_PATTERN_LENGTH,
123164         -  SQLITE_MAX_VARIABLE_NUMBER,
       124242  +  SQLITE_MAX_VARIABLE_NUMBER,      /* IMP: R-38091-32352 */
123165 124243     SQLITE_MAX_TRIGGER_DEPTH,
123166 124244   };
123167 124245   
123168 124246   /*
123169 124247   ** Make sure the hard limits are set to reasonable values
123170 124248   */
123171 124249   #if SQLITE_MAX_LENGTH<100
................................................................................
123182 124260   #endif
123183 124261   #if SQLITE_MAX_VDBE_OP<40
123184 124262   # error SQLITE_MAX_VDBE_OP must be at least 40
123185 124263   #endif
123186 124264   #if SQLITE_MAX_FUNCTION_ARG<0 || SQLITE_MAX_FUNCTION_ARG>1000
123187 124265   # error SQLITE_MAX_FUNCTION_ARG must be between 0 and 1000
123188 124266   #endif
123189         -#if SQLITE_MAX_ATTACHED<0 || SQLITE_MAX_ATTACHED>62
123190         -# error SQLITE_MAX_ATTACHED must be between 0 and 62
       124267  +#if SQLITE_MAX_ATTACHED<0 || SQLITE_MAX_ATTACHED>125
       124268  +# error SQLITE_MAX_ATTACHED must be between 0 and 125
123191 124269   #endif
123192 124270   #if SQLITE_MAX_LIKE_PATTERN_LENGTH<1
123193 124271   # error SQLITE_MAX_LIKE_PATTERN_LENGTH must be at least 1
123194 124272   #endif
123195 124273   #if SQLITE_MAX_COLUMN>32767
123196 124274   # error SQLITE_MAX_COLUMN must not exceed 32767
123197 124275   #endif
................................................................................
124209 125287       ** is called immediately after installing the new callback and the return
124210 125288       ** value from sqlite3FaultSim(0) becomes the return from
124211 125289       ** sqlite3_test_control().
124212 125290       */
124213 125291       case SQLITE_TESTCTRL_FAULT_INSTALL: {
124214 125292         /* MSVC is picky about pulling func ptrs from va lists.
124215 125293         ** http://support.microsoft.com/kb/47961
124216         -      ** sqlite3Config.xTestCallback = va_arg(ap, int(*)(int));
       125294  +      ** sqlite3GlobalConfig.xTestCallback = va_arg(ap, int(*)(int));
124217 125295         */
124218 125296         typedef int(*TESTCALLBACKFUNC_t)(int);
124219         -      sqlite3Config.xTestCallback = va_arg(ap, TESTCALLBACKFUNC_t);
       125297  +      sqlite3GlobalConfig.xTestCallback = va_arg(ap, TESTCALLBACKFUNC_t);
124220 125298         rc = sqlite3FaultSim(0);
124221 125299         break;
124222 125300       }
124223 125301   
124224 125302       /*
124225 125303       **  sqlite3_test_control(BENIGN_MALLOC_HOOKS, xBegin, xEnd)
124226 125304       **
................................................................................
124442 125520         typedef void (*branch_callback)(void*,int,u8,u8);
124443 125521         sqlite3GlobalConfig.xVdbeBranch = va_arg(ap,branch_callback);
124444 125522         sqlite3GlobalConfig.pVdbeBranchArg = va_arg(ap,void*);
124445 125523   #endif
124446 125524         break;
124447 125525       }
124448 125526   
       125527  +    /*   sqlite3_test_control(SQLITE_TESTCTRL_ISINIT);
       125528  +    **
       125529  +    ** Return SQLITE_OK if SQLite has been initialized and SQLITE_ERROR if
       125530  +    ** not.
       125531  +    */
       125532  +    case SQLITE_TESTCTRL_ISINIT: {
       125533  +      if( sqlite3GlobalConfig.isInit==0 ) rc = SQLITE_ERROR;
       125534  +      break;
       125535  +    }
       125536  +
124449 125537     }
124450 125538     va_end(ap);
124451 125539   #endif /* SQLITE_OMIT_BUILTIN_TEST */
124452 125540     return rc;
124453 125541   }
124454 125542   
124455 125543   /*
................................................................................
124490 125578   SQLITE_API sqlite3_int64 sqlite3_uri_int64(
124491 125579     const char *zFilename,    /* Filename as passed to xOpen */
124492 125580     const char *zParam,       /* URI parameter sought */
124493 125581     sqlite3_int64 bDflt       /* return if parameter is missing */
124494 125582   ){
124495 125583     const char *z = sqlite3_uri_parameter(zFilename, zParam);
124496 125584     sqlite3_int64 v;
124497         -  if( z && sqlite3Atoi64(z, &v, sqlite3Strlen30(z), SQLITE_UTF8)==SQLITE_OK ){
       125585  +  if( z && sqlite3DecOrHexToI64(z, &v)==SQLITE_OK ){
124498 125586       bDflt = v;
124499 125587     }
124500 125588     return bDflt;
124501 125589   }
124502 125590   
124503 125591   /*
124504 125592   ** Return the Btree pointer identified by zDbName.  Return NULL if not found.
................................................................................
126021 127109   SQLITE_PRIVATE int sqlite3Fts3MsrOvfl(Fts3Cursor *, Fts3MultiSegReader *, int *);
126022 127110   SQLITE_PRIVATE int sqlite3Fts3MsrIncrRestart(Fts3MultiSegReader *pCsr);
126023 127111   
126024 127112   /* fts3_tokenize_vtab.c */
126025 127113   SQLITE_PRIVATE int sqlite3Fts3InitTok(sqlite3*, Fts3Hash *);
126026 127114   
126027 127115   /* fts3_unicode2.c (functions generated by parsing unicode text files) */
126028         -#ifdef SQLITE_ENABLE_FTS4_UNICODE61
       127116  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
126029 127117   SQLITE_PRIVATE int sqlite3FtsUnicodeFold(int, int);
126030 127118   SQLITE_PRIVATE int sqlite3FtsUnicodeIsalnum(int);
126031 127119   SQLITE_PRIVATE int sqlite3FtsUnicodeIsdiacritic(int);
126032 127120   #endif
126033 127121   
126034 127122   #endif /* !SQLITE_CORE || SQLITE_ENABLE_FTS3 */
126035 127123   #endif /* _FTSINT_H */
................................................................................
129491 130579   **
129492 130580   ** Calling sqlite3Fts3SimpleTokenizerModule() sets the value pointed
129493 130581   ** to by the argument to point to the "simple" tokenizer implementation.
129494 130582   ** And so on.
129495 130583   */
129496 130584   SQLITE_PRIVATE void sqlite3Fts3SimpleTokenizerModule(sqlite3_tokenizer_module const**ppModule);
129497 130585   SQLITE_PRIVATE void sqlite3Fts3PorterTokenizerModule(sqlite3_tokenizer_module const**ppModule);
129498         -#ifdef SQLITE_ENABLE_FTS4_UNICODE61
       130586  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
129499 130587   SQLITE_PRIVATE void sqlite3Fts3UnicodeTokenizer(sqlite3_tokenizer_module const**ppModule);
129500 130588   #endif
129501 130589   #ifdef SQLITE_ENABLE_ICU
129502 130590   SQLITE_PRIVATE void sqlite3Fts3IcuTokenizerModule(sqlite3_tokenizer_module const**ppModule);
129503 130591   #endif
129504 130592   
129505 130593   /*
................................................................................
129509 130597   ** function is called by the sqlite3_extension_init() entry point.
129510 130598   */
129511 130599   SQLITE_PRIVATE int sqlite3Fts3Init(sqlite3 *db){
129512 130600     int rc = SQLITE_OK;
129513 130601     Fts3Hash *pHash = 0;
129514 130602     const sqlite3_tokenizer_module *pSimple = 0;
129515 130603     const sqlite3_tokenizer_module *pPorter = 0;
129516         -#ifdef SQLITE_ENABLE_FTS4_UNICODE61
       130604  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
129517 130605     const sqlite3_tokenizer_module *pUnicode = 0;
129518 130606   #endif
129519 130607   
129520 130608   #ifdef SQLITE_ENABLE_ICU
129521 130609     const sqlite3_tokenizer_module *pIcu = 0;
129522 130610     sqlite3Fts3IcuTokenizerModule(&pIcu);
129523 130611   #endif
129524 130612   
129525         -#ifdef SQLITE_ENABLE_FTS4_UNICODE61
       130613  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
129526 130614     sqlite3Fts3UnicodeTokenizer(&pUnicode);
129527 130615   #endif
129528 130616   
129529 130617   #ifdef SQLITE_TEST
129530 130618     rc = sqlite3Fts3InitTerm(db);
129531 130619     if( rc!=SQLITE_OK ) return rc;
129532 130620   #endif
................................................................................
129546 130634     }
129547 130635   
129548 130636     /* Load the built-in tokenizers into the hash table */
129549 130637     if( rc==SQLITE_OK ){
129550 130638       if( sqlite3Fts3HashInsert(pHash, "simple", 7, (void *)pSimple)
129551 130639        || sqlite3Fts3HashInsert(pHash, "porter", 7, (void *)pPorter) 
129552 130640   
129553         -#ifdef SQLITE_ENABLE_FTS4_UNICODE61
       130641  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
129554 130642        || sqlite3Fts3HashInsert(pHash, "unicode61", 10, (void *)pUnicode) 
129555 130643   #endif
129556 130644   #ifdef SQLITE_ENABLE_ICU
129557 130645        || (pIcu && sqlite3Fts3HashInsert(pHash, "icu", 4, (void *)pIcu))
129558 130646   #endif
129559 130647       ){
129560 130648         rc = SQLITE_NOMEM;
................................................................................
140781 141869   
140782 141870       while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
140783 141871         i64 iDocid = sqlite3_column_int64(pStmt, 0);
140784 141872         int iLang = langidFromSelect(p, pStmt);
140785 141873         int iCol;
140786 141874   
140787 141875         for(iCol=0; rc==SQLITE_OK && iCol<p->nColumn; iCol++){
140788         -        const char *zText = (const char *)sqlite3_column_text(pStmt, iCol+1);
140789         -        int nText = sqlite3_column_bytes(pStmt, iCol+1);
140790         -        sqlite3_tokenizer_cursor *pT = 0;
140791         -
140792         -        rc = sqlite3Fts3OpenTokenizer(p->pTokenizer, iLang, zText, nText, &pT);
140793         -        while( rc==SQLITE_OK ){
140794         -          char const *zToken;       /* Buffer containing token */
140795         -          int nToken = 0;           /* Number of bytes in token */
140796         -          int iDum1 = 0, iDum2 = 0; /* Dummy variables */
140797         -          int iPos = 0;             /* Position of token in zText */
140798         -
140799         -          rc = pModule->xNext(pT, &zToken, &nToken, &iDum1, &iDum2, &iPos);
140800         -          if( rc==SQLITE_OK ){
140801         -            int i;
140802         -            cksum2 = cksum2 ^ fts3ChecksumEntry(
140803         -                zToken, nToken, iLang, 0, iDocid, iCol, iPos
140804         -            );
140805         -            for(i=1; i<p->nIndex; i++){
140806         -              if( p->aIndex[i].nPrefix<=nToken ){
140807         -                cksum2 = cksum2 ^ fts3ChecksumEntry(
140808         -                  zToken, p->aIndex[i].nPrefix, iLang, i, iDocid, iCol, iPos
140809         -                );
       141876  +        if( p->abNotindexed[iCol]==0 ){
       141877  +          const char *zText = (const char *)sqlite3_column_text(pStmt, iCol+1);
       141878  +          int nText = sqlite3_column_bytes(pStmt, iCol+1);
       141879  +          sqlite3_tokenizer_cursor *pT = 0;
       141880  +
       141881  +          rc = sqlite3Fts3OpenTokenizer(p->pTokenizer, iLang, zText, nText,&pT);
       141882  +          while( rc==SQLITE_OK ){
       141883  +            char const *zToken;       /* Buffer containing token */
       141884  +            int nToken = 0;           /* Number of bytes in token */
       141885  +            int iDum1 = 0, iDum2 = 0; /* Dummy variables */
       141886  +            int iPos = 0;             /* Position of token in zText */
       141887  +
       141888  +            rc = pModule->xNext(pT, &zToken, &nToken, &iDum1, &iDum2, &iPos);
       141889  +            if( rc==SQLITE_OK ){
       141890  +              int i;
       141891  +              cksum2 = cksum2 ^ fts3ChecksumEntry(
       141892  +                  zToken, nToken, iLang, 0, iDocid, iCol, iPos
       141893  +              );
       141894  +              for(i=1; i<p->nIndex; i++){
       141895  +                if( p->aIndex[i].nPrefix<=nToken ){
       141896  +                  cksum2 = cksum2 ^ fts3ChecksumEntry(
       141897  +                      zToken, p->aIndex[i].nPrefix, iLang, i, iDocid, iCol, iPos
       141898  +                  );
       141899  +                }
140810 141900                 }
140811 141901               }
140812 141902             }
       141903  +          if( pT ) pModule->xClose(pT);
       141904  +          if( rc==SQLITE_DONE ) rc = SQLITE_OK;
140813 141905           }
140814         -        if( pT ) pModule->xClose(pT);
140815         -        if( rc==SQLITE_DONE ) rc = SQLITE_OK;
140816 141906         }
140817 141907       }
140818 141908   
140819 141909       sqlite3_finalize(pStmt);
140820 141910     }
140821 141911   
140822 141912     *pbOk = (cksum1==cksum2);
................................................................................
142804 143894   **    May you share freely, never taking more than you give.
142805 143895   **
142806 143896   ******************************************************************************
142807 143897   **
142808 143898   ** Implementation of the "unicode" full-text-search tokenizer.
142809 143899   */
142810 143900   
142811         -#ifdef SQLITE_ENABLE_FTS4_UNICODE61
       143901  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
142812 143902   
142813 143903   #if !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3)
142814 143904   
142815 143905   /* #include <assert.h> */
142816 143906   /* #include <stdlib.h> */
142817 143907   /* #include <stdio.h> */
142818 143908   /* #include <string.h> */
................................................................................
143020 144110     pNew = (unicode_tokenizer *) sqlite3_malloc(sizeof(unicode_tokenizer));
143021 144111     if( pNew==NULL ) return SQLITE_NOMEM;
143022 144112     memset(pNew, 0, sizeof(unicode_tokenizer));
143023 144113     pNew->bRemoveDiacritic = 1;
143024 144114   
143025 144115     for(i=0; rc==SQLITE_OK && i<nArg; i++){
143026 144116       const char *z = azArg[i];
143027         -    int n = strlen(z);
       144117  +    int n = (int)strlen(z);
143028 144118   
143029 144119       if( n==19 && memcmp("remove_diacritics=1", z, 19)==0 ){
143030 144120         pNew->bRemoveDiacritic = 1;
143031 144121       }
143032 144122       else if( n==19 && memcmp("remove_diacritics=0", z, 19)==0 ){
143033 144123         pNew->bRemoveDiacritic = 0;
143034 144124       }
................................................................................
143107 144197     int *pnToken,                   /* OUT: Number of bytes at *paToken */
143108 144198     int *piStart,                   /* OUT: Starting offset of token */
143109 144199     int *piEnd,                     /* OUT: Ending offset of token */
143110 144200     int *piPos                      /* OUT: Position integer of token */
143111 144201   ){
143112 144202     unicode_cursor *pCsr = (unicode_cursor *)pC;
143113 144203     unicode_tokenizer *p = ((unicode_tokenizer *)pCsr->base.pTokenizer);
143114         -  int iCode;
       144204  +  int iCode = 0;
143115 144205     char *zOut;
143116 144206     const unsigned char *z = &pCsr->aInput[pCsr->iOff];
143117 144207     const unsigned char *zStart = z;
143118 144208     const unsigned char *zEnd;
143119 144209     const unsigned char *zTerm = &pCsr->aInput[pCsr->nInput];
143120 144210   
143121 144211     /* Scan past any delimiter characters before the start of the next token.
................................................................................
143152 144242       if( z>=zTerm ) break;
143153 144243       READ_UTF8(z, zTerm, iCode);
143154 144244     }while( unicodeIsAlnum(p, iCode) 
143155 144245          || sqlite3FtsUnicodeIsdiacritic(iCode)
143156 144246     );
143157 144247   
143158 144248     /* Set the output variables and return. */
143159         -  pCsr->iOff = (z - pCsr->aInput);
       144249  +  pCsr->iOff = (int)(z - pCsr->aInput);
143160 144250     *paToken = pCsr->zToken;
143161         -  *pnToken = zOut - pCsr->zToken;
143162         -  *piStart = (zStart - pCsr->aInput);
143163         -  *piEnd = (zEnd - pCsr->aInput);
       144251  +  *pnToken = (int)(zOut - pCsr->zToken);
       144252  +  *piStart = (int)(zStart - pCsr->aInput);
       144253  +  *piEnd = (int)(zEnd - pCsr->aInput);
143164 144254     *piPos = pCsr->iToken++;
143165 144255     return SQLITE_OK;
143166 144256   }
143167 144257   
143168 144258   /*
143169 144259   ** Set *ppModule to a pointer to the sqlite3_tokenizer_module 
143170 144260   ** structure for the unicode tokenizer.
................................................................................
143179 144269       unicodeNext,
143180 144270       0,
143181 144271     };
143182 144272     *ppModule = &module;
143183 144273   }
143184 144274   
143185 144275   #endif /* !defined(SQLITE_CORE) || defined(SQLITE_ENABLE_FTS3) */
143186         -#endif /* ifndef SQLITE_ENABLE_FTS4_UNICODE61 */
       144276  +#endif /* ifndef SQLITE_DISABLE_FTS3_UNICODE */
143187 144277   
143188 144278   /************** End of fts3_unicode.c ****************************************/
143189 144279   /************** Begin file fts3_unicode2.c ***********************************/
143190 144280   /*
143191 144281   ** 2012 May 25
143192 144282   **
143193 144283   ** The author disclaims copyright to this source code.  In place of
................................................................................
143200 144290   ******************************************************************************
143201 144291   */
143202 144292   
143203 144293   /*
143204 144294   ** DO NOT EDIT THIS MACHINE GENERATED FILE.
143205 144295   */
143206 144296   
143207         -#if defined(SQLITE_ENABLE_FTS4_UNICODE61)
       144297  +#ifndef SQLITE_DISABLE_FTS3_UNICODE
143208 144298   #if defined(SQLITE_ENABLE_FTS3) || defined(SQLITE_ENABLE_FTS4)
143209 144299   
143210 144300   /* #include <assert.h> */
143211 144301   
143212 144302   /*
143213 144303   ** Return true if the argument corresponds to a unicode codepoint
143214 144304   ** classified as either a letter or a number. Otherwise false.
................................................................................
143224 144314     ** The most significant 22 bits in each 32-bit value contain the first 
143225 144315     ** codepoint in the range. The least significant 10 bits are used to store
143226 144316     ** the size of the range (always at least 1). In other words, the value 
143227 144317     ** ((C<<22) + N) represents a range of N codepoints starting with codepoint 
143228 144318     ** C. It is not possible to represent a range larger than 1023 codepoints 
143229 144319     ** using this format.
143230 144320     */
143231         -  const static unsigned int aEntry[] = {
       144321  +  static const unsigned int aEntry[] = {
143232 144322       0x00000030, 0x0000E807, 0x00016C06, 0x0001EC2F, 0x0002AC07,
143233 144323       0x0002D001, 0x0002D803, 0x0002EC01, 0x0002FC01, 0x00035C01,
143234 144324       0x0003DC01, 0x000B0804, 0x000B480E, 0x000B9407, 0x000BB401,
143235 144325       0x000BBC81, 0x000DD401, 0x000DF801, 0x000E1002, 0x000E1C01,
143236 144326       0x000FD801, 0x00120808, 0x00156806, 0x00162402, 0x00163C01,
143237 144327       0x00164437, 0x0017CC02, 0x00180005, 0x00181816, 0x00187802,
143238 144328       0x00192C15, 0x0019A804, 0x0019C001, 0x001B5001, 0x001B580F,
................................................................................
143316 144406       0xFFFFFFFF, 0xFC00FFFF, 0xF8000001, 0xF8000001,
143317 144407     };
143318 144408   
143319 144409     if( c<128 ){
143320 144410       return ( (aAscii[c >> 5] & (1 << (c & 0x001F)))==0 );
143321 144411     }else if( c<(1<<22) ){
143322 144412       unsigned int key = (((unsigned int)c)<<10) | 0x000003FF;
143323         -    int iRes;
       144413  +    int iRes = 0;
143324 144414       int iHi = sizeof(aEntry)/sizeof(aEntry[0]) - 1;
143325 144415       int iLo = 0;
143326 144416       while( iHi>=iLo ){
143327 144417         int iTest = (iHi + iLo) / 2;
143328 144418         if( key >= aEntry[iTest] ){
143329 144419           iRes = iTest;
143330 144420           iLo = iTest+1;
................................................................................
143387 144477         iLo = iTest+1;
143388 144478       }else{
143389 144479         iHi = iTest-1;
143390 144480       }
143391 144481     }
143392 144482     assert( key>=aDia[iRes] );
143393 144483     return ((c > (aDia[iRes]>>3) + (aDia[iRes]&0x07)) ? c : (int)aChar[iRes]);
143394         -};
       144484  +}
143395 144485   
143396 144486   
143397 144487   /*
143398 144488   ** Return true if the argument interpreted as a unicode codepoint
143399 144489   ** is a diacritical modifier character.
143400 144490   */
143401 144491   SQLITE_PRIVATE int sqlite3FtsUnicodeIsdiacritic(int c){
................................................................................
143547 144637     else if( c>=66560 && c<66600 ){
143548 144638       ret = c + 40;
143549 144639     }
143550 144640   
143551 144641     return ret;
143552 144642   }
143553 144643   #endif /* defined(SQLITE_ENABLE_FTS3) || defined(SQLITE_ENABLE_FTS4) */
143554         -#endif /* !defined(SQLITE_ENABLE_FTS4_UNICODE61) */
       144644  +#endif /* !defined(SQLITE_DISABLE_FTS3_UNICODE) */
143555 144645   
143556 144646   /************** End of fts3_unicode2.c ***************************************/
143557 144647   /************** Begin file rtree.c *******************************************/
143558 144648   /*
143559 144649   ** 2001 September 15
143560 144650   **
143561 144651   ** The author disclaims copyright to this source code.  In place of
................................................................................
145084 146174     RtreeNode *pRoot = 0;
145085 146175     int ii;
145086 146176     int rc = SQLITE_OK;
145087 146177     int iCell = 0;
145088 146178   
145089 146179     rtreeReference(pRtree);
145090 146180   
       146181  +  /* Reset the cursor to the same state as rtreeOpen() leaves it in. */
145091 146182     freeCursorConstraints(pCsr);
       146183  +  sqlite3_free(pCsr->aPoint);
       146184  +  memset(pCsr, 0, sizeof(RtreeCursor));
       146185  +  pCsr->base.pVtab = (sqlite3_vtab*)pRtree;
       146186  +
145092 146187     pCsr->iStrategy = idxNum;
145093         -
145094 146188     if( idxNum==1 ){
145095 146189       /* Special case - lookup by rowid. */
145096 146190       RtreeNode *pLeaf;        /* Leaf on which the required cell resides */
145097 146191       RtreeSearchPoint *p;     /* Search point for the the leaf */
145098 146192       i64 iRowid = sqlite3_value_int64(argv[0]);
145099 146193       i64 iNode = 0;
145100 146194       rc = findLeafNode(pRtree, iRowid, &pLeaf, &iNode);

Changes to sqlite3.h.

   103    103   ** string contains the date and time of the check-in (UTC) and an SHA1
   104    104   ** hash of the entire source tree.
   105    105   **
   106    106   ** See also: [sqlite3_libversion()],
   107    107   ** [sqlite3_libversion_number()], [sqlite3_sourceid()],
   108    108   ** [sqlite_version()] and [sqlite_source_id()].
   109    109   */
   110         -#define SQLITE_VERSION        "3.8.5"
   111         -#define SQLITE_VERSION_NUMBER 3008005
   112         -#define SQLITE_SOURCE_ID      "2014-06-02 11:26:33 9f18b303cd1bc5779d82669884f802c7889b4947"
          110  +#define SQLITE_VERSION        "3.8.6"
          111  +#define SQLITE_VERSION_NUMBER 3008006
          112  +#define SQLITE_SOURCE_ID      "2014-08-15 11:46:33 9491ba7d738528f168657adb43a198238abde19e"
   113    113   
   114    114   /*
   115    115   ** CAPI3REF: Run-Time Library Version Numbers
   116    116   ** KEYWORDS: sqlite3_version, sqlite3_sourceid
   117    117   **
   118    118   ** These interfaces provide the same information as the [SQLITE_VERSION],
   119    119   ** [SQLITE_VERSION_NUMBER], and [SQLITE_SOURCE_ID] C preprocessor macros
................................................................................
   265    265   #endif
   266    266   
   267    267   /*
   268    268   ** CAPI3REF: Closing A Database Connection
   269    269   **
   270    270   ** ^The sqlite3_close() and sqlite3_close_v2() routines are destructors
   271    271   ** for the [sqlite3] object.
   272         -** ^Calls to sqlite3_close() and sqlite3_close_v2() return SQLITE_OK if
          272  +** ^Calls to sqlite3_close() and sqlite3_close_v2() return [SQLITE_OK] if
   273    273   ** the [sqlite3] object is successfully destroyed and all associated
   274    274   ** resources are deallocated.
   275    275   **
   276    276   ** ^If the database connection is associated with unfinalized prepared
   277    277   ** statements or unfinished sqlite3_backup objects then sqlite3_close()
   278    278   ** will leave the database connection open and return [SQLITE_BUSY].
   279    279   ** ^If sqlite3_close_v2() is called with unfinalized prepared statements
   280         -** and unfinished sqlite3_backups, then the database connection becomes
          280  +** and/or unfinished sqlite3_backups, then the database connection becomes
   281    281   ** an unusable "zombie" which will automatically be deallocated when the
   282    282   ** last prepared statement is finalized or the last sqlite3_backup is
   283    283   ** finished.  The sqlite3_close_v2() interface is intended for use with
   284    284   ** host languages that are garbage collected, and where the order in which
   285    285   ** destructors are called is arbitrary.
   286    286   **
   287    287   ** Applications should [sqlite3_finalize | finalize] all [prepared statements],
   288    288   ** [sqlite3_blob_close | close] all [BLOB handles], and 
   289    289   ** [sqlite3_backup_finish | finish] all [sqlite3_backup] objects associated
   290    290   ** with the [sqlite3] object prior to attempting to close the object.  ^If
   291    291   ** sqlite3_close_v2() is called on a [database connection] that still has
   292    292   ** outstanding [prepared statements], [BLOB handles], and/or
   293         -** [sqlite3_backup] objects then it returns SQLITE_OK but the deallocation
          293  +** [sqlite3_backup] objects then it returns [SQLITE_OK] and the deallocation
   294    294   ** of resources is deferred until all [prepared statements], [BLOB handles],
   295    295   ** and [sqlite3_backup] objects are also destroyed.
   296    296   **
   297    297   ** ^If an [sqlite3] object is destroyed while a transaction is open,
   298    298   ** the transaction is automatically rolled back.
   299    299   **
   300    300   ** The C parameter to [sqlite3_close(C)] and [sqlite3_close_v2(C)]
................................................................................
   382    382     int (*callback)(void*,int,char**,char**),  /* Callback function */
   383    383     void *,                                    /* 1st argument to callback */
   384    384     char **errmsg                              /* Error msg written here */
   385    385   );
   386    386   
   387    387   /*
   388    388   ** CAPI3REF: Result Codes
   389         -** KEYWORDS: SQLITE_OK {error code} {error codes}
   390         -** KEYWORDS: {result code} {result codes}
          389  +** KEYWORDS: {result code definitions}
   391    390   **
   392    391   ** Many SQLite functions return an integer result code from the set shown
   393    392   ** here in order to indicate success or failure.
   394    393   **
   395    394   ** New error codes may be added in future versions of SQLite.
   396    395   **
   397         -** See also: [SQLITE_IOERR_READ | extended result codes],
   398         -** [sqlite3_vtab_on_conflict()] [SQLITE_ROLLBACK | result codes].
          396  +** See also: [extended result code definitions]
   399    397   */
   400    398   #define SQLITE_OK           0   /* Successful result */
   401    399   /* beginning-of-error-codes */
   402    400   #define SQLITE_ERROR        1   /* SQL error or missing database */
   403    401   #define SQLITE_INTERNAL     2   /* Internal logic error in SQLite */
   404    402   #define SQLITE_PERM         3   /* Access permission denied */
   405    403   #define SQLITE_ABORT        4   /* Callback routine requested an abort */
................................................................................
   429    427   #define SQLITE_WARNING     28   /* Warnings from sqlite3_log() */
   430    428   #define SQLITE_ROW         100  /* sqlite3_step() has another row ready */
   431    429   #define SQLITE_DONE        101  /* sqlite3_step() has finished executing */
   432    430   /* end-of-error-codes */
   433    431   
   434    432   /*
   435    433   ** CAPI3REF: Extended Result Codes
   436         -** KEYWORDS: {extended error code} {extended error codes}
   437         -** KEYWORDS: {extended result code} {extended result codes}
          434  +** KEYWORDS: {extended result code definitions}
   438    435   **
   439         -** In its default configuration, SQLite API routines return one of 26 integer
   440         -** [SQLITE_OK | result codes].  However, experience has shown that many of
          436  +** In its default configuration, SQLite API routines return one of 30 integer
          437  +** [result codes].  However, experience has shown that many of
   441    438   ** these result codes are too coarse-grained.  They do not provide as
   442    439   ** much information about problems as programmers might like.  In an effort to
   443    440   ** address this, newer versions of SQLite (version 3.3.8 and later) include
   444    441   ** support for additional result codes that provide more detailed information
   445         -** about errors. The extended result codes are enabled or disabled
          442  +** about errors. These [extended result codes] are enabled or disabled
   446    443   ** on a per database connection basis using the
   447         -** [sqlite3_extended_result_codes()] API.
   448         -**
   449         -** Some of the available extended result codes are listed here.
   450         -** One may expect the number of extended result codes will increase
   451         -** over time.  Software that uses extended result codes should expect
   452         -** to see new result codes in future releases of SQLite.
   453         -**
   454         -** The SQLITE_OK result code will never be extended.  It will always
   455         -** be exactly zero.
          444  +** [sqlite3_extended_result_codes()] API.  Or, the extended code for
          445  +** the most recent error can be obtained using
          446  +** [sqlite3_extended_errcode()].
   456    447   */
   457    448   #define SQLITE_IOERR_READ              (SQLITE_IOERR | (1<<8))
   458    449   #define SQLITE_IOERR_SHORT_READ        (SQLITE_IOERR | (2<<8))
   459    450   #define SQLITE_IOERR_WRITE             (SQLITE_IOERR | (3<<8))
   460    451   #define SQLITE_IOERR_FSYNC             (SQLITE_IOERR | (4<<8))
   461    452   #define SQLITE_IOERR_DIR_FSYNC         (SQLITE_IOERR | (5<<8))
   462    453   #define SQLITE_IOERR_TRUNCATE          (SQLITE_IOERR | (6<<8))
................................................................................
   681    672   ** integer opcode.  The third argument is a generic pointer intended to
   682    673   ** point to a structure that may contain arguments or space in which to
   683    674   ** write return values.  Potential uses for xFileControl() might be
   684    675   ** functions to enable blocking locks with timeouts, to change the
   685    676   ** locking strategy (for example to use dot-file locks), to inquire
   686    677   ** about the status of a lock, or to break stale locks.  The SQLite
   687    678   ** core reserves all opcodes less than 100 for its own use.
   688         -** A [SQLITE_FCNTL_LOCKSTATE | list of opcodes] less than 100 is available.
          679  +** A [file control opcodes | list of opcodes] less than 100 is available.
   689    680   ** Applications that define a custom xFileControl method should use opcodes
   690    681   ** greater than 100 to avoid conflicts.  VFS implementations should
   691    682   ** return [SQLITE_NOTFOUND] for file control opcodes that they do not
   692    683   ** recognize.
   693    684   **
   694    685   ** The xSectorSize() method returns the sector size of the
   695    686   ** device that underlies the file.  The sector size is the
................................................................................
   754    745     int (*xUnfetch)(sqlite3_file*, sqlite3_int64 iOfst, void *p);
   755    746     /* Methods above are valid for version 3 */
   756    747     /* Additional methods may be added in future releases */
   757    748   };
   758    749   
   759    750   /*
   760    751   ** CAPI3REF: Standard File Control Opcodes
          752  +** KEYWORDS: {file control opcodes} {file control opcode}
   761    753   **
   762    754   ** These integer constants are opcodes for the xFileControl method
   763    755   ** of the [sqlite3_io_methods] object and for the [sqlite3_file_control()]
   764    756   ** interface.
   765    757   **
   766    758   ** The [SQLITE_FCNTL_LOCKSTATE] opcode is used for debugging.  This
   767    759   ** opcode causes the xFileControl method to write the current state of
................................................................................
  2033   2025   */
  2034   2026   SQLITE_API int sqlite3_complete(const char *sql);
  2035   2027   SQLITE_API int sqlite3_complete16(const void *sql);
  2036   2028   
  2037   2029   /*
  2038   2030   ** CAPI3REF: Register A Callback To Handle SQLITE_BUSY Errors
  2039   2031   **
  2040         -** ^This routine sets a callback function that might be invoked whenever
  2041         -** an attempt is made to open a database table that another thread
  2042         -** or process has locked.
         2032  +** ^The sqlite3_busy_handler(D,X,P) routine sets a callback function X
         2033  +** that might be invoked with argument P whenever
         2034  +** an attempt is made to access a database table associated with
         2035  +** [database connection] D when another thread
         2036  +** or process has the table locked.
         2037  +** The sqlite3_busy_handler() interface is used to implement
         2038  +** [sqlite3_busy_timeout()] and [PRAGMA busy_timeout].
  2043   2039   **
  2044         -** ^If the busy callback is NULL, then [SQLITE_BUSY] or [SQLITE_IOERR_BLOCKED]
         2040  +** ^If the busy callback is NULL, then [SQLITE_BUSY]
  2045   2041   ** is returned immediately upon encountering the lock.  ^If the busy callback
  2046   2042   ** is not NULL, then the callback might be invoked with two arguments.
  2047   2043   **
  2048   2044   ** ^The first argument to the busy handler is a copy of the void* pointer which
  2049   2045   ** is the third argument to sqlite3_busy_handler().  ^The second argument to
  2050   2046   ** the busy handler callback is the number of times that the busy handler has
  2051         -** been invoked for this locking event.  ^If the
         2047  +** been invoked for the same locking event.  ^If the
  2052   2048   ** busy callback returns 0, then no additional attempts are made to
  2053         -** access the database and [SQLITE_BUSY] or [SQLITE_IOERR_BLOCKED] is returned.
         2049  +** access the database and [SQLITE_BUSY] is returned
         2050  +** to the application.
  2054   2051   ** ^If the callback returns non-zero, then another attempt
  2055         -** is made to open the database for reading and the cycle repeats.
         2052  +** is made to access the database and the cycle repeats.
  2056   2053   **
  2057   2054   ** The presence of a busy handler does not guarantee that it will be invoked
  2058   2055   ** when there is lock contention. ^If SQLite determines that invoking the busy
  2059   2056   ** handler could result in a deadlock, it will go ahead and return [SQLITE_BUSY]
  2060         -** or [SQLITE_IOERR_BLOCKED] instead of invoking the busy handler.
         2057  +** to the application instead of invoking the 
         2058  +** busy handler.
  2061   2059   ** Consider a scenario where one process is holding a read lock that
  2062   2060   ** it is trying to promote to a reserved lock and
  2063   2061   ** a second process is holding a reserved lock that it is trying
  2064   2062   ** to promote to an exclusive lock.  The first process cannot proceed
  2065   2063   ** because it is blocked by the second and the second process cannot
  2066   2064   ** proceed because it is blocked by the first.  If both processes
  2067   2065   ** invoke the busy handlers, neither will make any progress.  Therefore,
  2068   2066   ** SQLite returns [SQLITE_BUSY] for the first process, hoping that this
  2069   2067   ** will induce the first process to release its read lock and allow
  2070   2068   ** the second process to proceed.
  2071   2069   **
  2072   2070   ** ^The default busy callback is NULL.
  2073   2071   **
  2074         -** ^The [SQLITE_BUSY] error is converted to [SQLITE_IOERR_BLOCKED]
  2075         -** when SQLite is in the middle of a large transaction where all the
  2076         -** changes will not fit into the in-memory cache.  SQLite will
  2077         -** already hold a RESERVED lock on the database file, but it needs
  2078         -** to promote this lock to EXCLUSIVE so that it can spill cache
  2079         -** pages into the database file without harm to concurrent
  2080         -** readers.  ^If it is unable to promote the lock, then the in-memory
  2081         -** cache will be left in an inconsistent state and so the error
  2082         -** code is promoted from the relatively benign [SQLITE_BUSY] to
  2083         -** the more severe [SQLITE_IOERR_BLOCKED].  ^This error code promotion
  2084         -** forces an automatic rollback of the changes.  See the
  2085         -** <a href="/cvstrac/wiki?p=CorruptionFollowingBusyError">
  2086         -** CorruptionFollowingBusyError</a> wiki page for a discussion of why
  2087         -** this is important.
  2088         -**
  2089   2072   ** ^(There can only be a single busy handler defined for each
  2090   2073   ** [database connection].  Setting a new busy handler clears any
  2091   2074   ** previously set handler.)^  ^Note that calling [sqlite3_busy_timeout()]
  2092         -** will also set or clear the busy handler.
         2075  +** or evaluating [PRAGMA busy_timeout=N] will change the
         2076  +** busy handler and thus clear any previously set busy handler.
  2093   2077   **
  2094   2078   ** The busy callback should not take any actions which modify the
  2095         -** database connection that invoked the busy handler.  Any such actions
         2079  +** database connection that invoked the busy handler.  In other words,
         2080  +** the busy handler is not reentrant.  Any such actions
  2096   2081   ** result in undefined behavior.
  2097   2082   ** 
  2098   2083   ** A busy handler must not close the database connection
  2099   2084   ** or [prepared statement] that invoked the busy handler.
  2100   2085   */
  2101   2086   SQLITE_API int sqlite3_busy_handler(sqlite3*, int(*)(void*,int), void*);
  2102   2087   
................................................................................
  2104   2089   ** CAPI3REF: Set A Busy Timeout
  2105   2090   **
  2106   2091   ** ^This routine sets a [sqlite3_busy_handler | busy handler] that sleeps
  2107   2092   ** for a specified amount of time when a table is locked.  ^The handler
  2108   2093   ** will sleep multiple times until at least "ms" milliseconds of sleeping
  2109   2094   ** have accumulated.  ^After at least "ms" milliseconds of sleeping,
  2110   2095   ** the handler returns 0 which causes [sqlite3_step()] to return
  2111         -** [SQLITE_BUSY] or [SQLITE_IOERR_BLOCKED].
         2096  +** [SQLITE_BUSY].
  2112   2097   **
  2113   2098   ** ^Calling this routine with an argument less than or equal to zero
  2114   2099   ** turns off all busy handlers.
  2115   2100   **
  2116   2101   ** ^(There can only be a single busy handler for a particular
  2117   2102   ** [database connection] any any given moment.  If another busy handler
  2118   2103   ** was defined  (using [sqlite3_busy_handler()]) prior to calling
  2119   2104   ** this routine, that other busy handler is cleared.)^
         2105  +**
         2106  +** See also:  [PRAGMA busy_timeout]
  2120   2107   */
  2121   2108   SQLITE_API int sqlite3_busy_timeout(sqlite3*, int ms);
  2122   2109   
  2123   2110   /*
  2124   2111   ** CAPI3REF: Convenience Routines For Running Queries
  2125   2112   **
  2126   2113   ** This is a legacy interface that is preserved for backwards compatibility.
................................................................................
  2514   2501   **
  2515   2502   ** The [sqlite3_set_authorizer | authorizer callback function] must
  2516   2503   ** return either [SQLITE_OK] or one of these two constants in order
  2517   2504   ** to signal SQLite whether or not the action is permitted.  See the
  2518   2505   ** [sqlite3_set_authorizer | authorizer documentation] for additional
  2519   2506   ** information.
  2520   2507   **
  2521         -** Note that SQLITE_IGNORE is also used as a [SQLITE_ROLLBACK | return code]
  2522         -** from the [sqlite3_vtab_on_conflict()] interface.
         2508  +** Note that SQLITE_IGNORE is also used as a [conflict resolution mode]
         2509  +** returned from the [sqlite3_vtab_on_conflict()] interface.
  2523   2510   */
  2524   2511   #define SQLITE_DENY   1   /* Abort the SQL statement with an error */
  2525   2512   #define SQLITE_IGNORE 2   /* Don't allow access, but don't generate an error */
  2526   2513   
  2527   2514   /*
  2528   2515   ** CAPI3REF: Authorizer Action Codes
  2529   2516   **
................................................................................
  4701   4688   **
  4702   4689   ** ^(If this global variable is made to point to a string which is
  4703   4690   ** the name of a folder (a.k.a. directory), then all temporary files
  4704   4691   ** created by SQLite when using a built-in [sqlite3_vfs | VFS]
  4705   4692   ** will be placed in that directory.)^  ^If this variable
  4706   4693   ** is a NULL pointer, then SQLite performs a search for an appropriate
  4707   4694   ** temporary file directory.
         4695  +**
         4696  +** Applications are strongly discouraged from using this global variable.
         4697  +** It is required to set a temporary folder on Windows Runtime (WinRT).
         4698  +** But for all other platforms, it is highly recommended that applications
         4699  +** neither read nor write this variable.  This global variable is a relic
         4700  +** that exists for backwards compatibility of legacy applications and should
         4701  +** be avoided in new projects.
  4708   4702   **
  4709   4703   ** It is not safe to read or modify this variable in more than one
  4710   4704   ** thread at a time.  It is not safe to read or modify this variable
  4711   4705   ** if a [database connection] is being used at the same time in a separate
  4712   4706   ** thread.
  4713   4707   ** It is intended that this variable be set once
  4714   4708   ** as part of process initialization and before any SQLite interface
................................................................................
  4720   4714   ** the [temp_store_directory pragma] always assumes that any string
  4721   4715   ** that this variable points to is held in memory obtained from 
  4722   4716   ** [sqlite3_malloc] and the pragma may attempt to free that memory
  4723   4717   ** using [sqlite3_free].
  4724   4718   ** Hence, if this variable is modified directly, either it should be
  4725   4719   ** made NULL or made to point to memory obtained from [sqlite3_malloc]
  4726   4720   ** or else the use of the [temp_store_directory pragma] should be avoided.
         4721  +** Except when requested by the [temp_store_directory pragma], SQLite
         4722  +** does not free the memory that sqlite3_temp_directory points to.  If
         4723  +** the application wants that memory to be freed, it must do
         4724  +** so itself, taking care to only do so after all [database connection]
         4725  +** objects have been destroyed.
  4727   4726   **
  4728   4727   ** <b>Note to Windows Runtime users:</b>  The temporary directory must be set
  4729   4728   ** prior to calling [sqlite3_open] or [sqlite3_open_v2].  Otherwise, various
  4730   4729   ** features that require the use of temporary files may fail.  Here is an
  4731   4730   ** example of how to do this using C++ with the Windows Runtime:
  4732   4731   **
  4733   4732   ** <blockquote><pre>
................................................................................
  5854   5853   ** to sqlite3_mutex_alloc() is one of these integer constants:
  5855   5854   **
  5856   5855   ** <ul>
  5857   5856   ** <li>  SQLITE_MUTEX_FAST
  5858   5857   ** <li>  SQLITE_MUTEX_RECURSIVE
  5859   5858   ** <li>  SQLITE_MUTEX_STATIC_MASTER
  5860   5859   ** <li>  SQLITE_MUTEX_STATIC_MEM
  5861         -** <li>  SQLITE_MUTEX_STATIC_MEM2
         5860  +** <li>  SQLITE_MUTEX_STATIC_OPEN
  5862   5861   ** <li>  SQLITE_MUTEX_STATIC_PRNG
  5863   5862   ** <li>  SQLITE_MUTEX_STATIC_LRU
  5864         -** <li>  SQLITE_MUTEX_STATIC_LRU2
         5863  +** <li>  SQLITE_MUTEX_STATIC_PMEM
         5864  +** <li>  SQLITE_MUTEX_STATIC_APP1
         5865  +** <li>  SQLITE_MUTEX_STATIC_APP2
  5865   5866   ** </ul>)^
  5866   5867   **
  5867   5868   ** ^The first two constants (SQLITE_MUTEX_FAST and SQLITE_MUTEX_RECURSIVE)
  5868   5869   ** cause sqlite3_mutex_alloc() to create
  5869   5870   ** a new mutex.  ^The new mutex is recursive when SQLITE_MUTEX_RECURSIVE
  5870   5871   ** is used but not necessarily so when SQLITE_MUTEX_FAST is used.
  5871   5872   ** The mutex implementation does not need to make a distinction
................................................................................
  6061   6062   #define SQLITE_MUTEX_STATIC_MEM       3  /* sqlite3_malloc() */
  6062   6063   #define SQLITE_MUTEX_STATIC_MEM2      4  /* NOT USED */
  6063   6064   #define SQLITE_MUTEX_STATIC_OPEN      4  /* sqlite3BtreeOpen() */
  6064   6065   #define SQLITE_MUTEX_STATIC_PRNG      5  /* sqlite3_random() */
  6065   6066   #define SQLITE_MUTEX_STATIC_LRU       6  /* lru page list */
  6066   6067   #define SQLITE_MUTEX_STATIC_LRU2      7  /* NOT USED */
  6067   6068   #define SQLITE_MUTEX_STATIC_PMEM      7  /* sqlite3PageMalloc() */
         6069  +#define SQLITE_MUTEX_STATIC_APP1      8  /* For use by application */
         6070  +#define SQLITE_MUTEX_STATIC_APP2      9  /* For use by application */
         6071  +#define SQLITE_MUTEX_STATIC_APP3     10  /* For use by application */
  6068   6072   
  6069   6073   /*
  6070   6074   ** CAPI3REF: Retrieve the mutex for a database connection
  6071   6075   **
  6072   6076   ** ^This interface returns a pointer the [sqlite3_mutex] object that 
  6073   6077   ** serializes access to the [database connection] given in the argument
  6074   6078   ** when the [threading mode] is Serialized.
................................................................................
  6156   6160   #define SQLITE_TESTCTRL_ISKEYWORD               16
  6157   6161   #define SQLITE_TESTCTRL_SCRATCHMALLOC           17
  6158   6162   #define SQLITE_TESTCTRL_LOCALTIME_FAULT         18
  6159   6163   #define SQLITE_TESTCTRL_EXPLAIN_STMT            19
  6160   6164   #define SQLITE_TESTCTRL_NEVER_CORRUPT           20
  6161   6165   #define SQLITE_TESTCTRL_VDBE_COVERAGE           21
  6162   6166   #define SQLITE_TESTCTRL_BYTEORDER               22
  6163         -#define SQLITE_TESTCTRL_LAST                    22
         6167  +#define SQLITE_TESTCTRL_ISINIT                  23
         6168  +#define SQLITE_TESTCTRL_LAST                    23
  6164   6169   
  6165   6170   /*
  6166   6171   ** CAPI3REF: SQLite Runtime Status
  6167   6172   **
  6168   6173   ** ^This interface is used to retrieve runtime status information
  6169   6174   ** about the performance of SQLite, and optionally to reset various
  6170   6175   ** highwater marks.  ^The first argument is an integer code for
................................................................................
  7139   7144   ** ^The callback registered by this function replaces any existing callback
  7140   7145   ** registered using [sqlite3_wal_hook()].  ^Likewise, registering a callback
  7141   7146   ** using [sqlite3_wal_hook()] disables the automatic checkpoint mechanism
  7142   7147   ** configured by this function.
  7143   7148   **
  7144   7149   ** ^The [wal_autocheckpoint pragma] can be used to invoke this interface
  7145   7150   ** from SQL.
         7151  +**
         7152  +** ^Checkpoints initiated by this mechanism are
         7153  +** [sqlite3_wal_checkpoint_v2|PASSIVE].
  7146   7154   **
  7147   7155   ** ^Every new [database connection] defaults to having the auto-checkpoint
  7148   7156   ** enabled with a threshold of 1000 or [SQLITE_DEFAULT_WAL_AUTOCHECKPOINT]
  7149   7157   ** pages.  The use of this interface
  7150   7158   ** is only necessary if the default setting is found to be suboptimal
  7151   7159   ** for a particular application.
  7152   7160   */
................................................................................
  7156   7164   ** CAPI3REF: Checkpoint a database
  7157   7165   **
  7158   7166   ** ^The [sqlite3_wal_checkpoint(D,X)] interface causes database named X
  7159   7167   ** on [database connection] D to be [checkpointed].  ^If X is NULL or an
  7160   7168   ** empty string, then a checkpoint is run on all databases of
  7161   7169   ** connection D.  ^If the database connection D is not in
  7162   7170   ** [WAL | write-ahead log mode] then this interface is a harmless no-op.
         7171  +** ^The [sqlite3_wal_checkpoint(D,X)] interface initiates a
         7172  +** [sqlite3_wal_checkpoint_v2|PASSIVE] checkpoint.
         7173  +** Use the [sqlite3_wal_checkpoint_v2()] interface to get a FULL
         7174  +** or RESET checkpoint.
  7163   7175   **
  7164   7176   ** ^The [wal_checkpoint pragma] can be used to invoke this interface
  7165   7177   ** from SQL.  ^The [sqlite3_wal_autocheckpoint()] interface and the
  7166   7178   ** [wal_autocheckpoint pragma] can be used to cause this interface to be
  7167   7179   ** run whenever the WAL reaches a certain size threshold.
  7168   7180   **
  7169   7181   ** See also: [sqlite3_wal_checkpoint_v2()]
................................................................................
  7178   7190   ** eMode parameter:
  7179   7191   **
  7180   7192   ** <dl>
  7181   7193   ** <dt>SQLITE_CHECKPOINT_PASSIVE<dd>
  7182   7194   **   Checkpoint as many frames as possible without waiting for any database 
  7183   7195   **   readers or writers to finish. Sync the db file if all frames in the log
  7184   7196   **   are checkpointed. This mode is the same as calling 
  7185         -**   sqlite3_wal_checkpoint(). The busy-handler callback is never invoked.
         7197  +**   sqlite3_wal_checkpoint(). The [sqlite3_busy_handler|busy-handler callback]
         7198  +**   is never invoked.
  7186   7199   **
  7187   7200   ** <dt>SQLITE_CHECKPOINT_FULL<dd>
  7188         -**   This mode blocks (calls the busy-handler callback) until there is no
         7201  +**   This mode blocks (it invokes the
         7202  +**   [sqlite3_busy_handler|busy-handler callback]) until there is no
  7189   7203   **   database writer and all readers are reading from the most recent database
  7190   7204   **   snapshot. It then checkpoints all frames in the log file and syncs the
  7191   7205   **   database file. This call blocks database writers while it is running,
  7192   7206   **   but not database readers.
  7193   7207   **
  7194   7208   ** <dt>SQLITE_CHECKPOINT_RESTART<dd>
  7195   7209   **   This mode works the same way as SQLITE_CHECKPOINT_FULL, except after 
  7196         -**   checkpointing the log file it blocks (calls the busy-handler callback)
         7210  +**   checkpointing the log file it blocks (calls the 
         7211  +**   [sqlite3_busy_handler|busy-handler callback])
  7197   7212   **   until all readers are reading from the database file only. This ensures 
  7198   7213   **   that the next client to write to the database file restarts the log file 
  7199   7214   **   from the beginning. This call blocks database writers while it is running,
  7200   7215   **   but not database readers.
  7201   7216   ** </dl>
  7202   7217   **
  7203   7218   ** If pnLog is not NULL, then *pnLog is set to the total number of frames in
................................................................................
  7327   7342   ** of the SQL statement that triggered the call to the [xUpdate] method of the
  7328   7343   ** [virtual table].
  7329   7344   */
  7330   7345   SQLITE_API int sqlite3_vtab_on_conflict(sqlite3 *);
  7331   7346   
  7332   7347   /*
  7333   7348   ** CAPI3REF: Conflict resolution modes
         7349  +** KEYWORDS: {conflict resolution mode}
  7334   7350   **
  7335   7351   ** These constants are returned by [sqlite3_vtab_on_conflict()] to
  7336   7352   ** inform a [virtual table] implementation what the [ON CONFLICT] mode
  7337   7353   ** is for the SQL statement being evaluated.
  7338   7354   **
  7339   7355   ** Note that the [SQLITE_IGNORE] constant is also used as a potential
  7340   7356   ** return value from the [sqlite3_set_authorizer()] callback and that