Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch stat3-trunk Excluding Merge-Ins
This is equivalent to a diff from 6d8d4e16 to 4a153f01
2011-10-03
| ||
15:30 | Merge the STAT3 changes into trunk. (check-in: 774d0842 user: drh tags: trunk) | |
2011-09-25
| ||
17:47 | If an open as read/write fails, do not try to reopen as read-only if in exclusive access mode. (Closed-Leaf check-in: 4a153f01 user: drh tags: stat3-trunk) | |
2011-09-23
| ||
16:34 | Add the -DBUILD_sqlite option to the compiler when building testfixture.exe using the configure script. (check-in: 15a13b6c user: drh tags: trunk) | |
14:40 | Merge the latest trunk changes into the stat3-trunk branch. (check-in: 0beb88a9 user: drh tags: stat3-trunk) | |
2011-09-22
| ||
15:31 | Remove a redundant (and undefined) "forcedelete" command from corruptE.test. (check-in: 6d8d4e16 user: dan tags: trunk) | |
14:56 | Remove the SQLITE_PAGECACHE_BLOCKALLOC compilation option. (check-in: 4eb46898 user: dan tags: trunk) | |
Changes to src/analyze.c.
1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | /* ** 2005 July 8 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** This file contains code associated with the ANALYZE command. */ #ifndef SQLITE_OMIT_ANALYZE #include "sqliteInt.h" /* ** This routine generates code that opens the sqlite_stat1 table for ** writing with cursor iStatCur. If the library was built with the | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | /* ** 2005 July 8 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** This file contains code associated with the ANALYZE command. ** ** The ANALYZE command gather statistics about the content of tables ** and indices. These statistics are made available to the query planner ** to help it make better decisions about how to perform queries. ** ** The following system tables are or have been supported: ** ** CREATE TABLE sqlite_stat1(tbl, idx, stat); ** CREATE TABLE sqlite_stat2(tbl, idx, sampleno, sample); ** CREATE TABLE sqlite_stat3(tbl, idx, nEq, nLt, nDLt, sample); ** ** Additional tables might be added in future releases of SQLite. ** The sqlite_stat2 table is not created or used unless the SQLite version ** is between 3.6.18 and 3.7.8, inclusive, and unless SQLite is compiled ** with SQLITE_ENABLE_STAT2. The sqlite_stat2 table is deprecated. ** The sqlite_stat2 table is superceded by sqlite_stat3, which is only ** created and used by SQLite versions 3.7.9 and later and with ** SQLITE_ENABLE_STAT3 defined. The fucntionality of sqlite_stat3 ** is a superset of sqlite_stat2. ** ** Format of sqlite_stat1: ** ** There is normally one row per index, with the index identified by the ** name in the idx column. The tbl column is the name of the table to ** which the index belongs. In each such row, the stat column will be ** a string consisting of a list of integers. The first integer in this ** list is the number of rows in the index and in the table. The second ** integer is the average number of rows in the index that have the same ** value in the first column of the index. The third integer is the average ** number of rows in the index that have the same value for the first two ** columns. The N-th integer (for N>1) is the average number of rows in ** the index which have the same value for the first N-1 columns. For ** a K-column index, there will be K+1 integers in the stat column. If ** the index is unique, then the last integer will be 1. ** ** The list of integers in the stat column can optionally be followed ** by the keyword "unordered". The "unordered" keyword, if it is present, ** must be separated from the last integer by a single space. If the ** "unordered" keyword is present, then the query planner assumes that ** the index is unordered and will not use the index for a range query. ** ** If the sqlite_stat1.idx column is NULL, then the sqlite_stat1.stat ** column contains a single integer which is the (estimated) number of ** rows in the table identified by sqlite_stat1.tbl. ** ** Format of sqlite_stat2: ** ** The sqlite_stat2 is only created and is only used if SQLite is compiled ** with SQLITE_ENABLE_STAT2 and if the SQLite version number is between ** 3.6.18 and 3.7.8. The "stat2" table contains additional information ** about the distribution of keys within an index. The index is identified by ** the "idx" column and the "tbl" column is the name of the table to which ** the index belongs. There are usually 10 rows in the sqlite_stat2 ** table for each index. ** ** The sqlite_stat2 entries for an index that have sampleno between 0 and 9 ** inclusive are samples of the left-most key value in the index taken at ** evenly spaced points along the index. Let the number of samples be S ** (10 in the standard build) and let C be the number of rows in the index. ** Then the sampled rows are given by: ** ** rownumber = (i*C*2 + C)/(S*2) ** ** For i between 0 and S-1. Conceptually, the index space is divided into ** S uniform buckets and the samples are the middle row from each bucket. ** ** The format for sqlite_stat2 is recorded here for legacy reference. This ** version of SQLite does not support sqlite_stat2. It neither reads nor ** writes the sqlite_stat2 table. This version of SQLite only supports ** sqlite_stat3. ** ** Format for sqlite_stat3: ** ** The sqlite_stat3 is an enhancement to sqlite_stat2. A new name is ** used to avoid compatibility problems. ** ** The format of the sqlite_stat3 table is similar to the format of ** the sqlite_stat2 table. There are multiple entries for each index. ** The idx column names the index and the tbl column is the table of the ** index. If the idx and tbl columns are the same, then the sample is ** of the INTEGER PRIMARY KEY. The sample column is a value taken from ** the left-most column of the index. The nEq column is the approximate ** number of entires in the index whose left-most column exactly matches ** the sample. nLt is the approximate number of entires whose left-most ** column is less than the sample. The nDLt column is the approximate ** number of distinct left-most entries in the index that are less than ** the sample. ** ** Future versions of SQLite might change to store a string containing ** multiple integers values in the nDLt column of sqlite_stat3. The first ** integer will be the number of prior index entires that are distinct in ** the left-most column. The second integer will be the number of prior index ** entries that are distinct in the first two columns. The third integer ** will be the number of prior index entries that are distinct in the first ** three columns. And so forth. With that extension, the nDLt field is ** similar in function to the sqlite_stat1.stat field. ** ** There can be an arbitrary number of sqlite_stat3 entries per index. ** The ANALYZE command will typically generate sqlite_stat3 tables ** that contain between 10 and 40 samples which are distributed across ** the key space, though not uniformly, and which include samples with ** largest possible nEq values. */ #ifndef SQLITE_OMIT_ANALYZE #include "sqliteInt.h" /* ** This routine generates code that opens the sqlite_stat1 table for ** writing with cursor iStatCur. If the library was built with the |
︙ | ︙ | |||
38 39 40 41 42 43 44 | const char *zWhereType /* Either "tbl" or "idx" */ ){ static const struct { const char *zName; const char *zCols; } aTable[] = { { "sqlite_stat1", "tbl,idx,stat" }, | | > > > > | > > > > > > > > > > > > > > > > | 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 | const char *zWhereType /* Either "tbl" or "idx" */ ){ static const struct { const char *zName; const char *zCols; } aTable[] = { { "sqlite_stat1", "tbl,idx,stat" }, #ifdef SQLITE_ENABLE_STAT3 { "sqlite_stat3", "tbl,idx,neq,nlt,ndlt,sample" }, #endif }; static const char *azToDrop[] = { "sqlite_stat2", #ifndef SQLITE_ENABLE_STAT3 "sqlite_stat3", #endif }; int aRoot[] = {0, 0}; u8 aCreateTbl[] = {0, 0}; int i; sqlite3 *db = pParse->db; Db *pDb; Vdbe *v = sqlite3GetVdbe(pParse); if( v==0 ) return; assert( sqlite3BtreeHoldsAllMutexes(db) ); assert( sqlite3VdbeDb(v)==db ); pDb = &db->aDb[iDb]; /* Drop all statistics tables that this version of SQLite does not ** understand. */ for(i=0; i<ArraySize(azToDrop); i++){ Table *pTab = sqlite3FindTable(db, azToDrop[i], pDb->zName); if( pTab ){ sqlite3CodeDropTable(pParse, pTab, iDb, 0); break; } } /* Create new statistic tables if they do not exist, or clear them ** if they do already exist. */ for(i=0; i<ArraySize(aTable); i++){ const char *zTab = aTable[i].zName; Table *pStat; if( (pStat = sqlite3FindTable(db, zTab, pDb->zName))==0 ){ /* The sqlite_stat[12] table does not exist. Create it. Note that a ** side-effect of the CREATE TABLE statement is to leave the rootpage ** of the new table in register pParse->regRoot. This is important |
︙ | ︙ | |||
85 86 87 88 89 90 91 | }else{ /* The sqlite_stat[12] table already exists. Delete all rows. */ sqlite3VdbeAddOp2(v, OP_Clear, aRoot[i], iDb); } } } | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 | }else{ /* The sqlite_stat[12] table already exists. Delete all rows. */ sqlite3VdbeAddOp2(v, OP_Clear, aRoot[i], iDb); } } } /* Open the sqlite_stat[13] tables for writing. */ for(i=0; i<ArraySize(aTable); i++){ sqlite3VdbeAddOp3(v, OP_OpenWrite, iStatCur+i, aRoot[i], iDb); sqlite3VdbeChangeP4(v, -1, (char *)3, P4_INT32); sqlite3VdbeChangeP5(v, aCreateTbl[i]); } } /* ** Recommended number of samples for sqlite_stat3 */ #ifndef SQLITE_STAT3_SAMPLES # define SQLITE_STAT3_SAMPLES 24 #endif /* ** Three SQL functions - stat3_init(), stat3_push(), and stat3_pop() - ** share an instance of the following structure to hold their state ** information. */ typedef struct Stat3Accum Stat3Accum; struct Stat3Accum { tRowcnt nRow; /* Number of rows in the entire table */ tRowcnt nPSample; /* How often to do a periodic sample */ int iMin; /* Index of entry with minimum nEq and hash */ int mxSample; /* Maximum number of samples to accumulate */ int nSample; /* Current number of samples */ u32 iPrn; /* Pseudo-random number used for sampling */ struct Stat3Sample { i64 iRowid; /* Rowid in main table of the key */ tRowcnt nEq; /* sqlite_stat3.nEq */ tRowcnt nLt; /* sqlite_stat3.nLt */ tRowcnt nDLt; /* sqlite_stat3.nDLt */ u8 isPSample; /* True if a periodic sample */ u32 iHash; /* Tiebreaker hash */ } *a; /* An array of samples */ }; #ifdef SQLITE_ENABLE_STAT3 /* ** Implementation of the stat3_init(C,S) SQL function. The two parameters ** are the number of rows in the table or index (C) and the number of samples ** to accumulate (S). ** ** This routine allocates the Stat3Accum object. ** ** The return value is the Stat3Accum object (P). */ static void stat3Init( sqlite3_context *context, int argc, sqlite3_value **argv ){ Stat3Accum *p; tRowcnt nRow; int mxSample; int n; UNUSED_PARAMETER(argc); nRow = (tRowcnt)sqlite3_value_int64(argv[0]); mxSample = sqlite3_value_int(argv[1]); n = sizeof(*p) + sizeof(p->a[0])*mxSample; p = sqlite3_malloc( n ); if( p==0 ){ sqlite3_result_error_nomem(context); return; } memset(p, 0, n); p->a = (struct Stat3Sample*)&p[1]; p->nRow = nRow; p->mxSample = mxSample; p->nPSample = p->nRow/(mxSample/3+1) + 1; sqlite3_randomness(sizeof(p->iPrn), &p->iPrn); sqlite3_result_blob(context, p, sizeof(p), sqlite3_free); } static const FuncDef stat3InitFuncdef = { 2, /* nArg */ SQLITE_UTF8, /* iPrefEnc */ 0, /* flags */ 0, /* pUserData */ 0, /* pNext */ stat3Init, /* xFunc */ 0, /* xStep */ 0, /* xFinalize */ "stat3_init", /* zName */ 0, /* pHash */ 0 /* pDestructor */ }; /* ** Implementation of the stat3_push(nEq,nLt,nDLt,rowid,P) SQL function. The ** arguments describe a single key instance. This routine makes the ** decision about whether or not to retain this key for the sqlite_stat3 ** table. ** ** The return value is NULL. */ static void stat3Push( sqlite3_context *context, int argc, sqlite3_value **argv ){ Stat3Accum *p = (Stat3Accum*)sqlite3_value_blob(argv[4]); tRowcnt nEq = sqlite3_value_int64(argv[0]); tRowcnt nLt = sqlite3_value_int64(argv[1]); tRowcnt nDLt = sqlite3_value_int64(argv[2]); i64 rowid = sqlite3_value_int64(argv[3]); u8 isPSample = 0; u8 doInsert = 0; int iMin = p->iMin; struct Stat3Sample *pSample; int i; u32 h; UNUSED_PARAMETER(context); UNUSED_PARAMETER(argc); if( nEq==0 ) return; h = p->iPrn = p->iPrn*1103515245 + 12345; if( (nLt/p->nPSample)!=((nEq+nLt)/p->nPSample) ){ doInsert = isPSample = 1; }else if( p->nSample<p->mxSample ){ doInsert = 1; }else{ if( nEq>p->a[iMin].nEq || (nEq==p->a[iMin].nEq && h>p->a[iMin].iHash) ){ doInsert = 1; } } if( !doInsert ) return; if( p->nSample==p->mxSample ){ assert( p->nSample - iMin - 1 >= 0 ); memmove(&p->a[iMin], &p->a[iMin+1], sizeof(p->a[0])*(p->nSample-iMin-1)); pSample = &p->a[p->nSample-1]; }else{ pSample = &p->a[p->nSample++]; } pSample->iRowid = rowid; pSample->nEq = nEq; pSample->nLt = nLt; pSample->nDLt = nDLt; pSample->iHash = h; pSample->isPSample = isPSample; /* Find the new minimum */ if( p->nSample==p->mxSample ){ pSample = p->a; i = 0; while( pSample->isPSample ){ i++; pSample++; assert( i<p->nSample ); } nEq = pSample->nEq; h = pSample->iHash; iMin = i; for(i++, pSample++; i<p->nSample; i++, pSample++){ if( pSample->isPSample ) continue; if( pSample->nEq<nEq || (pSample->nEq==nEq && pSample->iHash<h) ){ iMin = i; nEq = pSample->nEq; h = pSample->iHash; } } p->iMin = iMin; } } static const FuncDef stat3PushFuncdef = { 5, /* nArg */ SQLITE_UTF8, /* iPrefEnc */ 0, /* flags */ 0, /* pUserData */ 0, /* pNext */ stat3Push, /* xFunc */ 0, /* xStep */ 0, /* xFinalize */ "stat3_push", /* zName */ 0, /* pHash */ 0 /* pDestructor */ }; /* ** Implementation of the stat3_get(P,N,...) SQL function. This routine is ** used to query the results. Content is returned for the Nth sqlite_stat3 ** row where N is between 0 and S-1 and S is the number of samples. The ** value returned depends on the number of arguments. ** ** argc==2 result: rowid ** argc==3 result: nEq ** argc==4 result: nLt ** argc==5 result: nDLt */ static void stat3Get( sqlite3_context *context, int argc, sqlite3_value **argv ){ int n = sqlite3_value_int(argv[1]); Stat3Accum *p = (Stat3Accum*)sqlite3_value_blob(argv[0]); assert( p!=0 ); if( p->nSample<=n ) return; switch( argc ){ case 2: sqlite3_result_int64(context, p->a[n].iRowid); break; case 3: sqlite3_result_int64(context, p->a[n].nEq); break; case 4: sqlite3_result_int64(context, p->a[n].nLt); break; default: sqlite3_result_int64(context, p->a[n].nDLt); break; } } static const FuncDef stat3GetFuncdef = { -1, /* nArg */ SQLITE_UTF8, /* iPrefEnc */ 0, /* flags */ 0, /* pUserData */ 0, /* pNext */ stat3Get, /* xFunc */ 0, /* xStep */ 0, /* xFinalize */ "stat3_get", /* zName */ 0, /* pHash */ 0 /* pDestructor */ }; #endif /* SQLITE_ENABLE_STAT3 */ /* ** Generate code to do an analysis of all indices associated with ** a single table. */ static void analyzeOneTable( Parse *pParse, /* Parser context */ |
︙ | ︙ | |||
115 116 117 118 119 120 121 | int i; /* Loop counter */ int topOfLoop; /* The top of the loop */ int endOfLoop; /* The end of the loop */ int jZeroRows = -1; /* Jump from here if number of rows is zero */ int iDb; /* Index of database containing pTab */ int regTabname = iMem++; /* Register containing table name */ int regIdxname = iMem++; /* Register containing index name */ | > > > > > > > > | > > > > > > > | | < < < < < < < < | 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 | int i; /* Loop counter */ int topOfLoop; /* The top of the loop */ int endOfLoop; /* The end of the loop */ int jZeroRows = -1; /* Jump from here if number of rows is zero */ int iDb; /* Index of database containing pTab */ int regTabname = iMem++; /* Register containing table name */ int regIdxname = iMem++; /* Register containing index name */ int regStat1 = iMem++; /* The stat column of sqlite_stat1 */ #ifdef SQLITE_ENABLE_STAT3 int regNumEq = regStat1; /* Number of instances. Same as regStat1 */ int regNumLt = iMem++; /* Number of keys less than regSample */ int regNumDLt = iMem++; /* Number of distinct keys less than regSample */ int regSample = iMem++; /* The next sample value */ int regRowid = regSample; /* Rowid of a sample */ int regAccum = iMem++; /* Register to hold Stat3Accum object */ int regLoop = iMem++; /* Loop counter */ int regCount = iMem++; /* Number of rows in the table or index */ int regTemp1 = iMem++; /* Intermediate register */ int regTemp2 = iMem++; /* Intermediate register */ int once = 1; /* One-time initialization */ int shortJump = 0; /* Instruction address */ int iTabCur = pParse->nTab++; /* Table cursor */ #endif int regCol = iMem++; /* Content of a column in analyzed table */ int regRec = iMem++; /* Register holding completed record */ int regTemp = iMem++; /* Temporary use register */ int regNewRowid = iMem++; /* Rowid for the inserted record */ v = sqlite3GetVdbe(pParse); if( v==0 || NEVER(pTab==0) ){ return; } if( pTab->tnum==0 ){ /* Do not gather statistics on views or virtual tables */ |
︙ | ︙ | |||
161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 | sqlite3TableLock(pParse, iDb, pTab->tnum, 0, pTab->zName); iIdxCur = pParse->nTab++; sqlite3VdbeAddOp4(v, OP_String8, 0, regTabname, 0, pTab->zName, 0); for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){ int nCol; KeyInfo *pKey; if( pOnlyIdx && pOnlyIdx!=pIdx ) continue; nCol = pIdx->nColumn; pKey = sqlite3IndexKeyinfo(pParse, pIdx); if( iMem+1+(nCol*2)>pParse->nMem ){ pParse->nMem = iMem+1+(nCol*2); } /* Open a cursor to the index to be analyzed. */ assert( iDb==sqlite3SchemaToIndex(db, pIdx->pSchema) ); sqlite3VdbeAddOp4(v, OP_OpenRead, iIdxCur, pIdx->tnum, iDb, (char *)pKey, P4_KEYINFO_HANDOFF); VdbeComment((v, "%s", pIdx->zName)); /* Populate the register containing the index name. */ sqlite3VdbeAddOp4(v, OP_String8, 0, regIdxname, 0, pIdx->zName, 0); | > > > > > | | | < < | < < < | | | < < < | < < < < < | | | > > | | 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 | sqlite3TableLock(pParse, iDb, pTab->tnum, 0, pTab->zName); iIdxCur = pParse->nTab++; sqlite3VdbeAddOp4(v, OP_String8, 0, regTabname, 0, pTab->zName, 0); for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){ int nCol; KeyInfo *pKey; int addrIfNot = 0; /* address of OP_IfNot */ int *aChngAddr; /* Array of jump instruction addresses */ if( pOnlyIdx && pOnlyIdx!=pIdx ) continue; VdbeNoopComment((v, "Begin analysis of %s", pIdx->zName)); nCol = pIdx->nColumn; aChngAddr = sqlite3DbMallocRaw(db, sizeof(int)*nCol); if( aChngAddr==0 ) continue; pKey = sqlite3IndexKeyinfo(pParse, pIdx); if( iMem+1+(nCol*2)>pParse->nMem ){ pParse->nMem = iMem+1+(nCol*2); } /* Open a cursor to the index to be analyzed. */ assert( iDb==sqlite3SchemaToIndex(db, pIdx->pSchema) ); sqlite3VdbeAddOp4(v, OP_OpenRead, iIdxCur, pIdx->tnum, iDb, (char *)pKey, P4_KEYINFO_HANDOFF); VdbeComment((v, "%s", pIdx->zName)); /* Populate the register containing the index name. */ sqlite3VdbeAddOp4(v, OP_String8, 0, regIdxname, 0, pIdx->zName, 0); #ifdef SQLITE_ENABLE_STAT3 if( once ){ once = 0; sqlite3OpenTable(pParse, iTabCur, iDb, pTab, OP_OpenRead); } sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regCount); sqlite3VdbeAddOp2(v, OP_Integer, SQLITE_STAT3_SAMPLES, regTemp1); sqlite3VdbeAddOp2(v, OP_Integer, 0, regNumEq); sqlite3VdbeAddOp2(v, OP_Integer, 0, regNumLt); sqlite3VdbeAddOp2(v, OP_Integer, -1, regNumDLt); sqlite3VdbeAddOp4(v, OP_Function, 1, regCount, regAccum, (char*)&stat3InitFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 2); #endif /* SQLITE_ENABLE_STAT3 */ /* The block of memory cells initialized here is used as follows. ** ** iMem: ** The total number of rows in the table. ** ** iMem+1 .. iMem+nCol: |
︙ | ︙ | |||
232 233 234 235 236 237 238 | } /* Start the analysis loop. This loop runs through all the entries in ** the index b-tree. */ endOfLoop = sqlite3VdbeMakeLabel(v); sqlite3VdbeAddOp2(v, OP_Rewind, iIdxCur, endOfLoop); topOfLoop = sqlite3VdbeCurrentAddr(v); | | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | | > > > > > | < < < < < | | | > > > > > > > > > < > < > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | | | | | > | 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 | } /* Start the analysis loop. This loop runs through all the entries in ** the index b-tree. */ endOfLoop = sqlite3VdbeMakeLabel(v); sqlite3VdbeAddOp2(v, OP_Rewind, iIdxCur, endOfLoop); topOfLoop = sqlite3VdbeCurrentAddr(v); sqlite3VdbeAddOp2(v, OP_AddImm, iMem, 1); /* Increment row counter */ for(i=0; i<nCol; i++){ CollSeq *pColl; sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, regCol); if( i==0 ){ /* Always record the very first row */ addrIfNot = sqlite3VdbeAddOp1(v, OP_IfNot, iMem+1); } assert( pIdx->azColl!=0 ); assert( pIdx->azColl[i]!=0 ); pColl = sqlite3LocateCollSeq(pParse, pIdx->azColl[i]); aChngAddr[i] = sqlite3VdbeAddOp4(v, OP_Ne, regCol, 0, iMem+nCol+i+1, (char*)pColl, P4_COLLSEQ); sqlite3VdbeChangeP5(v, SQLITE_NULLEQ); VdbeComment((v, "jump if column %d changed", i)); #ifdef SQLITE_ENABLE_STAT3 if( i==0 ){ sqlite3VdbeAddOp2(v, OP_AddImm, regNumEq, 1); VdbeComment((v, "incr repeat count")); } #endif } sqlite3VdbeAddOp2(v, OP_Goto, 0, endOfLoop); for(i=0; i<nCol; i++){ sqlite3VdbeJumpHere(v, aChngAddr[i]); /* Set jump dest for the OP_Ne */ if( i==0 ){ sqlite3VdbeJumpHere(v, addrIfNot); /* Jump dest for OP_IfNot */ #ifdef SQLITE_ENABLE_STAT3 sqlite3VdbeAddOp4(v, OP_Function, 1, regNumEq, regTemp2, (char*)&stat3PushFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 5); sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, pIdx->nColumn, regRowid); sqlite3VdbeAddOp3(v, OP_Add, regNumEq, regNumLt, regNumLt); sqlite3VdbeAddOp2(v, OP_AddImm, regNumDLt, 1); sqlite3VdbeAddOp2(v, OP_Integer, 1, regNumEq); #endif } sqlite3VdbeAddOp2(v, OP_AddImm, iMem+i+1, 1); sqlite3VdbeAddOp3(v, OP_Column, iIdxCur, i, iMem+nCol+i+1); } sqlite3DbFree(db, aChngAddr); /* Always jump here after updating the iMem+1...iMem+1+nCol counters */ sqlite3VdbeResolveLabel(v, endOfLoop); sqlite3VdbeAddOp2(v, OP_Next, iIdxCur, topOfLoop); sqlite3VdbeAddOp1(v, OP_Close, iIdxCur); #ifdef SQLITE_ENABLE_STAT3 sqlite3VdbeAddOp4(v, OP_Function, 1, regNumEq, regTemp2, (char*)&stat3PushFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 5); sqlite3VdbeAddOp2(v, OP_Integer, -1, regLoop); shortJump = sqlite3VdbeAddOp2(v, OP_AddImm, regLoop, 1); sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regTemp1, (char*)&stat3GetFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 2); sqlite3VdbeAddOp1(v, OP_IsNull, regTemp1); sqlite3VdbeAddOp3(v, OP_NotExists, iTabCur, shortJump, regTemp1); sqlite3VdbeAddOp3(v, OP_Column, iTabCur, pIdx->aiColumn[0], regSample); sqlite3ColumnDefault(v, pTab, pIdx->aiColumn[0], regSample); sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumEq, (char*)&stat3GetFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 3); sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumLt, (char*)&stat3GetFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 4); sqlite3VdbeAddOp4(v, OP_Function, 1, regAccum, regNumDLt, (char*)&stat3GetFuncdef, P4_FUNCDEF); sqlite3VdbeChangeP5(v, 5); sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 6, regRec, "bbbbbb", 0); sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur+1, regNewRowid); sqlite3VdbeAddOp3(v, OP_Insert, iStatCur+1, regRec, regNewRowid); sqlite3VdbeAddOp2(v, OP_Goto, 0, shortJump); sqlite3VdbeJumpHere(v, shortJump+2); #endif /* Store the results in sqlite_stat1. ** ** The result is a single row of the sqlite_stat1 table. The first ** two columns are the names of the table and index. The third column ** is a string composed of a list of integer statistics about the ** index. The first integer in the list is the total number of entries ** in the index. There is one additional integer in the list for each ** column of the table. This additional integer is a guess of how many ** rows of the table the index will select. If D is the count of distinct ** values and K is the total number of rows, then the integer is computed ** as: ** ** I = (K+D-1)/D ** ** If K==0 then no entry is made into the sqlite_stat1 table. ** If K>0 then it is always the case the D>0 so division by zero ** is never possible. */ sqlite3VdbeAddOp2(v, OP_SCopy, iMem, regStat1); if( jZeroRows<0 ){ jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, iMem); } for(i=0; i<nCol; i++){ sqlite3VdbeAddOp4(v, OP_String8, 0, regTemp, 0, " ", 0); sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regStat1, regStat1); sqlite3VdbeAddOp3(v, OP_Add, iMem, iMem+i+1, regTemp); sqlite3VdbeAddOp2(v, OP_AddImm, regTemp, -1); sqlite3VdbeAddOp3(v, OP_Divide, iMem+i+1, regTemp, regTemp); sqlite3VdbeAddOp1(v, OP_ToInt, regTemp); sqlite3VdbeAddOp3(v, OP_Concat, regTemp, regStat1, regStat1); } sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0); sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regNewRowid); sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regNewRowid); sqlite3VdbeChangeP5(v, OPFLAG_APPEND); } /* If the table has no indices, create a single sqlite_stat1 entry ** containing NULL as the index name and the row count as the content. */ if( pTab->pIndex==0 ){ sqlite3VdbeAddOp3(v, OP_OpenRead, iIdxCur, pTab->tnum, iDb); VdbeComment((v, "%s", pTab->zName)); sqlite3VdbeAddOp2(v, OP_Count, iIdxCur, regStat1); sqlite3VdbeAddOp1(v, OP_Close, iIdxCur); jZeroRows = sqlite3VdbeAddOp1(v, OP_IfNot, regStat1); }else{ sqlite3VdbeJumpHere(v, jZeroRows); jZeroRows = sqlite3VdbeAddOp0(v, OP_Goto); } sqlite3VdbeAddOp2(v, OP_Null, 0, regIdxname); sqlite3VdbeAddOp4(v, OP_MakeRecord, regTabname, 3, regRec, "aaa", 0); sqlite3VdbeAddOp2(v, OP_NewRowid, iStatCur, regNewRowid); sqlite3VdbeAddOp3(v, OP_Insert, iStatCur, regRec, regNewRowid); sqlite3VdbeChangeP5(v, OPFLAG_APPEND); if( pParse->nMem<regRec ) pParse->nMem = regRec; sqlite3VdbeJumpHere(v, jZeroRows); } /* ** Generate code that will cause the most recent index analysis to ** be loaded into internal hash tables where is can be used. */ static void loadAnalysis(Parse *pParse, int iDb){ Vdbe *v = sqlite3GetVdbe(pParse); |
︙ | ︙ | |||
384 385 386 387 388 389 390 | Schema *pSchema = db->aDb[iDb].pSchema; /* Schema of database iDb */ HashElem *k; int iStatCur; int iMem; sqlite3BeginWriteOperation(pParse, 0, iDb); iStatCur = pParse->nTab; | | | 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 | Schema *pSchema = db->aDb[iDb].pSchema; /* Schema of database iDb */ HashElem *k; int iStatCur; int iMem; sqlite3BeginWriteOperation(pParse, 0, iDb); iStatCur = pParse->nTab; pParse->nTab += 3; openStatTable(pParse, iDb, iStatCur, 0, 0); iMem = pParse->nMem+1; assert( sqlite3SchemaMutexHeld(db, iDb, 0) ); for(k=sqliteHashFirst(&pSchema->tblHash); k; k=sqliteHashNext(k)){ Table *pTab = (Table*)sqliteHashData(k); analyzeOneTable(pParse, pTab, 0, iStatCur, iMem); } |
︙ | ︙ | |||
409 410 411 412 413 414 415 | int iStatCur; assert( pTab!=0 ); assert( sqlite3BtreeHoldsAllMutexes(pParse->db) ); iDb = sqlite3SchemaToIndex(pParse->db, pTab->pSchema); sqlite3BeginWriteOperation(pParse, 0, iDb); iStatCur = pParse->nTab; | | | 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 | int iStatCur; assert( pTab!=0 ); assert( sqlite3BtreeHoldsAllMutexes(pParse->db) ); iDb = sqlite3SchemaToIndex(pParse->db, pTab->pSchema); sqlite3BeginWriteOperation(pParse, 0, iDb); iStatCur = pParse->nTab; pParse->nTab += 3; if( pOnlyIdx ){ openStatTable(pParse, iDb, iStatCur, pOnlyIdx->zName, "idx"); }else{ openStatTable(pParse, iDb, iStatCur, pTab->zName, "tbl"); } analyzeOneTable(pParse, pTab, pOnlyIdx, iStatCur, pParse->nMem+1); loadAnalysis(pParse, iDb); |
︙ | ︙ | |||
514 515 516 517 518 519 520 | ** the table. */ static int analysisLoader(void *pData, int argc, char **argv, char **NotUsed){ analysisInfo *pInfo = (analysisInfo*)pData; Index *pIndex; Table *pTable; int i, c, n; | | | 866 867 868 869 870 871 872 873 874 875 876 877 878 879 880 | ** the table. */ static int analysisLoader(void *pData, int argc, char **argv, char **NotUsed){ analysisInfo *pInfo = (analysisInfo*)pData; Index *pIndex; Table *pTable; int i, c, n; tRowcnt v; const char *z; assert( argc==3 ); UNUSED_PARAMETER2(NotUsed, argc); if( argv==0 || argv[0]==0 || argv[2]==0 ){ return 0; |
︙ | ︙ | |||
557 558 559 560 561 562 563 | } /* ** If the Index.aSample variable is not NULL, delete the aSample[] array ** and its contents. */ void sqlite3DeleteIndexSamples(sqlite3 *db, Index *pIdx){ | | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | | | | | | | 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 945 946 947 948 949 950 951 952 953 954 955 956 957 958 959 960 961 962 963 964 965 966 967 968 969 970 971 972 973 974 975 976 977 978 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 994 995 996 997 998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 1084 | } /* ** If the Index.aSample variable is not NULL, delete the aSample[] array ** and its contents. */ void sqlite3DeleteIndexSamples(sqlite3 *db, Index *pIdx){ #ifdef SQLITE_ENABLE_STAT3 if( pIdx->aSample ){ int j; for(j=0; j<pIdx->nSample; j++){ IndexSample *p = &pIdx->aSample[j]; if( p->eType==SQLITE_TEXT || p->eType==SQLITE_BLOB ){ sqlite3DbFree(db, p->u.z); } } sqlite3DbFree(db, pIdx->aSample); } if( db && db->pnBytesFreed==0 ){ pIdx->nSample = 0; pIdx->aSample = 0; } #else UNUSED_PARAMETER(db); UNUSED_PARAMETER(pIdx); #endif } #ifdef SQLITE_ENABLE_STAT3 /* ** Load content from the sqlite_stat3 table into the Index.aSample[] ** arrays of all indices. */ static int loadStat3(sqlite3 *db, const char *zDb){ int rc; /* Result codes from subroutines */ sqlite3_stmt *pStmt = 0; /* An SQL statement being run */ char *zSql; /* Text of the SQL statement */ Index *pPrevIdx = 0; /* Previous index in the loop */ int idx = 0; /* slot in pIdx->aSample[] for next sample */ int eType; /* Datatype of a sample */ IndexSample *pSample; /* A slot in pIdx->aSample[] */ if( !sqlite3FindTable(db, "sqlite_stat3", zDb) ){ return SQLITE_OK; } zSql = sqlite3MPrintf(db, "SELECT idx,count(*) FROM %Q.sqlite_stat3" " GROUP BY idx", zDb); if( !zSql ){ return SQLITE_NOMEM; } rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0); sqlite3DbFree(db, zSql); if( rc ) return rc; while( sqlite3_step(pStmt)==SQLITE_ROW ){ char *zIndex; /* Index name */ Index *pIdx; /* Pointer to the index object */ int nSample; /* Number of samples */ zIndex = (char *)sqlite3_column_text(pStmt, 0); if( zIndex==0 ) continue; nSample = sqlite3_column_int(pStmt, 1); pIdx = sqlite3FindIndex(db, zIndex, zDb); if( pIdx==0 ) continue; assert( pIdx->nSample==0 ); pIdx->nSample = nSample; pIdx->aSample = sqlite3MallocZero( nSample*sizeof(IndexSample) ); pIdx->avgEq = pIdx->aiRowEst[1]; if( pIdx->aSample==0 ){ db->mallocFailed = 1; sqlite3_finalize(pStmt); return SQLITE_NOMEM; } } rc = sqlite3_finalize(pStmt); if( rc ) return rc; zSql = sqlite3MPrintf(db, "SELECT idx,neq,nlt,ndlt,sample FROM %Q.sqlite_stat3", zDb); if( !zSql ){ return SQLITE_NOMEM; } rc = sqlite3_prepare(db, zSql, -1, &pStmt, 0); sqlite3DbFree(db, zSql); if( rc ) return rc; while( sqlite3_step(pStmt)==SQLITE_ROW ){ char *zIndex; /* Index name */ Index *pIdx; /* Pointer to the index object */ int i; /* Loop counter */ tRowcnt sumEq; /* Sum of the nEq values */ zIndex = (char *)sqlite3_column_text(pStmt, 0); if( zIndex==0 ) continue; pIdx = sqlite3FindIndex(db, zIndex, zDb); if( pIdx==0 ) continue; if( pIdx==pPrevIdx ){ idx++; }else{ pPrevIdx = pIdx; idx = 0; } assert( idx<pIdx->nSample ); pSample = &pIdx->aSample[idx]; pSample->nEq = (tRowcnt)sqlite3_column_int64(pStmt, 1); pSample->nLt = (tRowcnt)sqlite3_column_int64(pStmt, 2); pSample->nDLt = (tRowcnt)sqlite3_column_int64(pStmt, 3); if( idx==pIdx->nSample-1 ){ if( pSample->nDLt>0 ){ for(i=0, sumEq=0; i<=idx-1; i++) sumEq += pIdx->aSample[i].nEq; pIdx->avgEq = (pSample->nLt - sumEq)/pSample->nDLt; } if( pIdx->avgEq<=0 ) pIdx->avgEq = 1; } eType = sqlite3_column_type(pStmt, 4); pSample->eType = (u8)eType; switch( eType ){ case SQLITE_INTEGER: { pSample->u.i = sqlite3_column_int64(pStmt, 4); break; } case SQLITE_FLOAT: { pSample->u.r = sqlite3_column_double(pStmt, 4); break; } case SQLITE_NULL: { break; } default: assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB ); { const char *z = (const char *)( (eType==SQLITE_BLOB) ? sqlite3_column_blob(pStmt, 4): sqlite3_column_text(pStmt, 4) ); int n = z ? sqlite3_column_bytes(pStmt, 4) : 0; pSample->nByte = n; if( n < 1){ pSample->u.z = 0; }else{ pSample->u.z = sqlite3Malloc(n); if( pSample->u.z==0 ){ db->mallocFailed = 1; sqlite3_finalize(pStmt); return SQLITE_NOMEM; } memcpy(pSample->u.z, z, n); } } } } return sqlite3_finalize(pStmt); } #endif /* SQLITE_ENABLE_STAT3 */ /* ** Load the content of the sqlite_stat1 and sqlite_stat3 tables. The ** contents of sqlite_stat1 are used to populate the Index.aiRowEst[] ** arrays. The contents of sqlite_stat3 are used to populate the ** Index.aSample[] arrays. ** ** If the sqlite_stat1 table is not present in the database, SQLITE_ERROR ** is returned. In this case, even if SQLITE_ENABLE_STAT3 was defined ** during compilation and the sqlite_stat3 table is present, no data is ** read from it. ** ** If SQLITE_ENABLE_STAT3 was defined during compilation and the ** sqlite_stat3 table is not present in the database, SQLITE_ERROR is ** returned. However, in this case, data is read from the sqlite_stat1 ** table (if it is present) before returning. ** ** If an OOM error occurs, this function always sets db->mallocFailed. ** This means if the caller does not care about other errors, the return ** code may be ignored. */ |
︙ | ︙ | |||
608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 | assert( db->aDb[iDb].pBt!=0 ); /* Clear any prior statistics */ assert( sqlite3SchemaMutexHeld(db, iDb, 0) ); for(i=sqliteHashFirst(&db->aDb[iDb].pSchema->idxHash);i;i=sqliteHashNext(i)){ Index *pIdx = sqliteHashData(i); sqlite3DefaultRowEst(pIdx); sqlite3DeleteIndexSamples(db, pIdx); pIdx->aSample = 0; } /* Check to make sure the sqlite_stat1 table exists */ sInfo.db = db; sInfo.zDatabase = db->aDb[iDb].zName; if( sqlite3FindTable(db, "sqlite_stat1", sInfo.zDatabase)==0 ){ return SQLITE_ERROR; } /* Load new statistics out of the sqlite_stat1 table */ zSql = sqlite3MPrintf(db, | > > | | | < < < < < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 | assert( db->aDb[iDb].pBt!=0 ); /* Clear any prior statistics */ assert( sqlite3SchemaMutexHeld(db, iDb, 0) ); for(i=sqliteHashFirst(&db->aDb[iDb].pSchema->idxHash);i;i=sqliteHashNext(i)){ Index *pIdx = sqliteHashData(i); sqlite3DefaultRowEst(pIdx); #ifdef SQLITE_ENABLE_STAT3 sqlite3DeleteIndexSamples(db, pIdx); pIdx->aSample = 0; #endif } /* Check to make sure the sqlite_stat1 table exists */ sInfo.db = db; sInfo.zDatabase = db->aDb[iDb].zName; if( sqlite3FindTable(db, "sqlite_stat1", sInfo.zDatabase)==0 ){ return SQLITE_ERROR; } /* Load new statistics out of the sqlite_stat1 table */ zSql = sqlite3MPrintf(db, "SELECT tbl,idx,stat FROM %Q.sqlite_stat1", sInfo.zDatabase); if( zSql==0 ){ rc = SQLITE_NOMEM; }else{ rc = sqlite3_exec(db, zSql, analysisLoader, &sInfo, 0); sqlite3DbFree(db, zSql); } /* Load the statistics from the sqlite_stat3 table. */ #ifdef SQLITE_ENABLE_STAT3 if( rc==SQLITE_OK ){ rc = loadStat3(db, sInfo.zDatabase); } #endif if( rc==SQLITE_NOMEM ){ db->mallocFailed = 1; } return rc; } #endif /* SQLITE_OMIT_ANALYZE */ |
Changes to src/build.c.
︙ | ︙ | |||
1986 1987 1988 1989 1990 1991 1992 | */ static void sqlite3ClearStatTables( Parse *pParse, /* The parsing context */ int iDb, /* The database number */ const char *zType, /* "idx" or "tbl" */ const char *zName /* Name of index or table */ ){ | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1986 1987 1988 1989 1990 1991 1992 1993 1994 1995 1996 1997 1998 1999 2000 2001 2002 2003 2004 2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 2041 2042 2043 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 | */ static void sqlite3ClearStatTables( Parse *pParse, /* The parsing context */ int iDb, /* The database number */ const char *zType, /* "idx" or "tbl" */ const char *zName /* Name of index or table */ ){ static const char *azStatTab[] = { "sqlite_stat1", "sqlite_stat2", "sqlite_stat3", }; int i; const char *zDbName = pParse->db->aDb[iDb].zName; for(i=0; i<ArraySize(azStatTab); i++){ if( sqlite3FindTable(pParse->db, azStatTab[i], zDbName) ){ sqlite3NestedParse(pParse, "DELETE FROM %Q.%s WHERE %s=%Q", zDbName, azStatTab[i], zType, zName ); } } } /* ** Generate code to drop a table. */ void sqlite3CodeDropTable(Parse *pParse, Table *pTab, int iDb, int isView){ Vdbe *v; sqlite3 *db = pParse->db; Trigger *pTrigger; Db *pDb = &db->aDb[iDb]; v = sqlite3GetVdbe(pParse); assert( v!=0 ); sqlite3BeginWriteOperation(pParse, 1, iDb); #ifndef SQLITE_OMIT_VIRTUALTABLE if( IsVirtual(pTab) ){ sqlite3VdbeAddOp0(v, OP_VBegin); } #endif /* Drop all triggers associated with the table being dropped. Code ** is generated to remove entries from sqlite_master and/or ** sqlite_temp_master if required. */ pTrigger = sqlite3TriggerList(pParse, pTab); while( pTrigger ){ assert( pTrigger->pSchema==pTab->pSchema || pTrigger->pSchema==db->aDb[1].pSchema ); sqlite3DropTriggerPtr(pParse, pTrigger); pTrigger = pTrigger->pNext; } #ifndef SQLITE_OMIT_AUTOINCREMENT /* Remove any entries of the sqlite_sequence table associated with ** the table being dropped. This is done before the table is dropped ** at the btree level, in case the sqlite_sequence table needs to ** move as a result of the drop (can happen in auto-vacuum mode). */ if( pTab->tabFlags & TF_Autoincrement ){ sqlite3NestedParse(pParse, "DELETE FROM %Q.sqlite_sequence WHERE name=%Q", pDb->zName, pTab->zName ); } #endif /* Drop all SQLITE_MASTER table and index entries that refer to the ** table. The program name loops through the master table and deletes ** every row that refers to a table of the same name as the one being ** dropped. Triggers are handled seperately because a trigger can be ** created in the temp database that refers to a table in another ** database. */ sqlite3NestedParse(pParse, "DELETE FROM %Q.%s WHERE tbl_name=%Q and type!='trigger'", pDb->zName, SCHEMA_TABLE(iDb), pTab->zName); if( !isView && !IsVirtual(pTab) ){ destroyTable(pParse, pTab); } /* Remove the table entry from SQLite's internal schema and modify ** the schema cookie. */ if( IsVirtual(pTab) ){ sqlite3VdbeAddOp4(v, OP_VDestroy, iDb, 0, 0, pTab->zName, 0); } sqlite3VdbeAddOp4(v, OP_DropTable, iDb, 0, 0, pTab->zName, 0); sqlite3ChangeCookie(pParse, iDb); sqliteViewResetAll(db, iDb); } /* ** This routine is called to do the work of a DROP TABLE statement. ** pName is the name of the table to be dropped. */ void sqlite3DropTable(Parse *pParse, SrcList *pName, int isView, int noErr){ Table *pTab; |
︙ | ︙ | |||
2091 2092 2093 2094 2095 2096 2097 | #endif /* Generate code to remove the table from the master table ** on disk. */ v = sqlite3GetVdbe(pParse); if( v ){ | < < | < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < | | < < < < < < < < < < < | 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 | #endif /* Generate code to remove the table from the master table ** on disk. */ v = sqlite3GetVdbe(pParse); if( v ){ sqlite3BeginWriteOperation(pParse, 1, iDb); sqlite3ClearStatTables(pParse, iDb, "tbl", pTab->zName); sqlite3FkDropTable(pParse, pName, pTab); sqlite3CodeDropTable(pParse, pTab, iDb, isView); } exit_drop_table: sqlite3SrcListDelete(db, pName); } /* ** This routine is called to create a new foreign key on the table |
︙ | ︙ | |||
2635 2636 2637 2638 2639 2640 2641 2642 | /* ** Allocate the index structure. */ nName = sqlite3Strlen30(zName); nCol = pList->nExpr; pIndex = sqlite3DbMallocZero(db, sizeof(Index) + /* Index structure */ sizeof(int)*nCol + /* Index.aiColumn */ | > < > | < | | 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 | /* ** Allocate the index structure. */ nName = sqlite3Strlen30(zName); nCol = pList->nExpr; pIndex = sqlite3DbMallocZero(db, sizeof(Index) + /* Index structure */ sizeof(tRowcnt)*(nCol+1) + /* Index.aiRowEst */ sizeof(int)*nCol + /* Index.aiColumn */ sizeof(char *)*nCol + /* Index.azColl */ sizeof(u8)*nCol + /* Index.aSortOrder */ nName + 1 + /* Index.zName */ nExtra /* Collation sequence names */ ); if( db->mallocFailed ){ goto exit_create_index; } pIndex->aiRowEst = (tRowcnt*)(&pIndex[1]); pIndex->azColl = (char**)(&pIndex->aiRowEst[nCol+1]); pIndex->aiColumn = (int *)(&pIndex->azColl[nCol]); pIndex->aSortOrder = (u8 *)(&pIndex->aiColumn[nCol]); pIndex->zName = (char *)(&pIndex->aSortOrder[nCol]); zExtra = (char *)(&pIndex->zName[nName+1]); memcpy(pIndex->zName, zName, nName+1); pIndex->pTable = pTab; pIndex->nColumn = pList->nExpr; pIndex->onError = (u8)onError; pIndex->autoIndex = (u8)(pName==0); |
︙ | ︙ | |||
2925 2926 2927 2928 2929 2930 2931 | ** aiRowEst[N]>=1 ** ** Apart from that, we have little to go on besides intuition as to ** how aiRowEst[] should be initialized. The numbers generated here ** are based on typical values found in actual indices. */ void sqlite3DefaultRowEst(Index *pIdx){ | | | | 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 | ** aiRowEst[N]>=1 ** ** Apart from that, we have little to go on besides intuition as to ** how aiRowEst[] should be initialized. The numbers generated here ** are based on typical values found in actual indices. */ void sqlite3DefaultRowEst(Index *pIdx){ tRowcnt *a = pIdx->aiRowEst; int i; tRowcnt n; assert( a!=0 ); a[0] = pIdx->pTable->nRowEst; if( a[0]<10 ) a[0] = 10; n = 10; for(i=1; i<=pIdx->nColumn; i++){ a[i] = n; if( n>5 ) n--; |
︙ | ︙ |
Changes to src/ctime.c.
︙ | ︙ | |||
112 113 114 115 116 117 118 119 120 121 122 123 124 125 | "ENABLE_OVERSIZE_CELL_CHECK", #endif #ifdef SQLITE_ENABLE_RTREE "ENABLE_RTREE", #endif #ifdef SQLITE_ENABLE_STAT2 "ENABLE_STAT2", #endif #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY "ENABLE_UNLOCK_NOTIFY", #endif #ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT "ENABLE_UPDATE_DELETE_LIMIT", #endif | > > > | 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | "ENABLE_OVERSIZE_CELL_CHECK", #endif #ifdef SQLITE_ENABLE_RTREE "ENABLE_RTREE", #endif #ifdef SQLITE_ENABLE_STAT2 "ENABLE_STAT2", #endif #ifdef SQLITE_ENABLE_STAT3 "ENABLE_STAT3", #endif #ifdef SQLITE_ENABLE_UNLOCK_NOTIFY "ENABLE_UNLOCK_NOTIFY", #endif #ifdef SQLITE_ENABLE_UPDATE_DELETE_LIMIT "ENABLE_UPDATE_DELETE_LIMIT", #endif |
︙ | ︙ |
Changes to src/os_win.c.
︙ | ︙ | |||
2611 2612 2613 2614 2615 2616 2617 | h, zName, dwDesiredAccess, h==INVALID_HANDLE_VALUE ? "failed" : "ok")); if( h==INVALID_HANDLE_VALUE ){ pFile->lastErrno = GetLastError(); winLogError(SQLITE_CANTOPEN, "winOpen", zUtf8Name); free(zConverted); | | | 2611 2612 2613 2614 2615 2616 2617 2618 2619 2620 2621 2622 2623 2624 2625 | h, zName, dwDesiredAccess, h==INVALID_HANDLE_VALUE ? "failed" : "ok")); if( h==INVALID_HANDLE_VALUE ){ pFile->lastErrno = GetLastError(); winLogError(SQLITE_CANTOPEN, "winOpen", zUtf8Name); free(zConverted); if( isReadWrite && !isExclusive ){ return winOpen(pVfs, zName, id, ((flags|SQLITE_OPEN_READONLY)&~(SQLITE_OPEN_CREATE|SQLITE_OPEN_READWRITE)), pOutFlags); }else{ return SQLITE_CANTOPEN_BKPT; } } |
︙ | ︙ |
Changes to src/sqlite.h.in.
︙ | ︙ | |||
2841 2842 2843 2844 2845 2846 2847 | ** WHERE clause might influence the choice of query plan for a statement, ** then the statement will be automatically recompiled, as if there had been ** a schema change, on the first [sqlite3_step()] call following any change ** to the [sqlite3_bind_text | bindings] of that [parameter]. ** ^The specific value of WHERE-clause [parameter] might influence the ** choice of query plan if the parameter is the left-hand side of a [LIKE] ** or [GLOB] operator or if the parameter is compared to an indexed column | | | 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 | ** WHERE clause might influence the choice of query plan for a statement, ** then the statement will be automatically recompiled, as if there had been ** a schema change, on the first [sqlite3_step()] call following any change ** to the [sqlite3_bind_text | bindings] of that [parameter]. ** ^The specific value of WHERE-clause [parameter] might influence the ** choice of query plan if the parameter is the left-hand side of a [LIKE] ** or [GLOB] operator or if the parameter is compared to an indexed column ** and the [SQLITE_ENABLE_STAT3] compile-time option is enabled. ** the ** </li> ** </ol> */ int sqlite3_prepare( sqlite3 *db, /* Database handle */ const char *zSql, /* SQL statement, UTF-8 encoded */ |
︙ | ︙ |
Changes to src/sqliteInt.h.
︙ | ︙ | |||
447 448 449 450 451 452 453 454 455 456 457 458 459 460 | ** SQLITE_MAX_U32 is a u64 constant that is the maximum u64 value ** that can be stored in a u32 without loss of data. The value ** is 0x00000000ffffffff. But because of quirks of some compilers, we ** have to specify the value in the less intuitive manner shown: */ #define SQLITE_MAX_U32 ((((u64)1)<<32)-1) /* ** Macros to determine whether the machine is big or little endian, ** evaluated at runtime. */ #ifdef SQLITE_AMALGAMATION const int sqlite3one = 1; #else | > > > > > > > > > > > > | 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 | ** SQLITE_MAX_U32 is a u64 constant that is the maximum u64 value ** that can be stored in a u32 without loss of data. The value ** is 0x00000000ffffffff. But because of quirks of some compilers, we ** have to specify the value in the less intuitive manner shown: */ #define SQLITE_MAX_U32 ((((u64)1)<<32)-1) /* ** The datatype used to store estimates of the number of rows in a ** table or index. This is an unsigned integer type. For 99.9% of ** the world, a 32-bit integer is sufficient. But a 64-bit integer ** can be used at compile-time if desired. */ #ifdef SQLITE_64BIT_STATS typedef u64 tRowcnt; /* 64-bit only if requested at compile-time */ #else typedef u32 tRowcnt; /* 32-bit is the default */ #endif /* ** Macros to determine whether the machine is big or little endian, ** evaluated at runtime. */ #ifdef SQLITE_AMALGAMATION const int sqlite3one = 1; #else |
︙ | ︙ | |||
1280 1281 1282 1283 1284 1285 1286 | struct Table { char *zName; /* Name of the table or view */ int iPKey; /* If not negative, use aCol[iPKey] as the primary key */ int nCol; /* Number of columns in this table */ Column *aCol; /* Information about each column */ Index *pIndex; /* List of SQL indexes on this table. */ int tnum; /* Root BTree node for this table (see note above) */ | | | 1292 1293 1294 1295 1296 1297 1298 1299 1300 1301 1302 1303 1304 1305 1306 | struct Table { char *zName; /* Name of the table or view */ int iPKey; /* If not negative, use aCol[iPKey] as the primary key */ int nCol; /* Number of columns in this table */ Column *aCol; /* Information about each column */ Index *pIndex; /* List of SQL indexes on this table. */ int tnum; /* Root BTree node for this table (see note above) */ tRowcnt nRowEst; /* Estimated rows in table - from sqlite_stat1 table */ Select *pSelect; /* NULL for tables. Points to definition if a view. */ u16 nRef; /* Number of pointers to this Table */ u8 tabFlags; /* Mask of TF_* values */ u8 keyConf; /* What to do in case of uniqueness conflict on iPKey */ FKey *pFKey; /* Linked list of all foreign keys in this table */ char *zColAff; /* String defining the affinity of each column */ #ifndef SQLITE_OMIT_CHECK |
︙ | ︙ | |||
1479 1480 1481 1482 1483 1484 1485 | ** algorithm to employ whenever an attempt is made to insert a non-unique ** element. */ struct Index { char *zName; /* Name of this index */ int nColumn; /* Number of columns in the table used by this index */ int *aiColumn; /* Which columns are used by this index. 1st is 0 */ | | > > > | > | > | > > > | 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 1502 1503 1504 1505 1506 1507 1508 1509 1510 1511 1512 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 1528 1529 1530 1531 1532 1533 1534 1535 1536 1537 | ** algorithm to employ whenever an attempt is made to insert a non-unique ** element. */ struct Index { char *zName; /* Name of this index */ int nColumn; /* Number of columns in the table used by this index */ int *aiColumn; /* Which columns are used by this index. 1st is 0 */ tRowcnt *aiRowEst; /* Result of ANALYZE: Est. rows selected by each column */ Table *pTable; /* The SQL table being indexed */ int tnum; /* Page containing root of this index in database file */ u8 onError; /* OE_Abort, OE_Ignore, OE_Replace, or OE_None */ u8 autoIndex; /* True if is automatically created (ex: by UNIQUE) */ u8 bUnordered; /* Use this index for == or IN queries only */ char *zColAff; /* String defining the affinity of each column */ Index *pNext; /* The next index associated with the same table */ Schema *pSchema; /* Schema containing this index */ u8 *aSortOrder; /* Array of size Index.nColumn. True==DESC, False==ASC */ char **azColl; /* Array of collation sequence names for index */ #ifdef SQLITE_ENABLE_STAT3 int nSample; /* Number of elements in aSample[] */ tRowcnt avgEq; /* Average nEq value for key values not in aSample */ IndexSample *aSample; /* Samples of the left-most key */ #endif }; /* ** Each sample stored in the sqlite_stat2 table is represented in memory ** using a structure of this type. */ struct IndexSample { union { char *z; /* Value if eType is SQLITE_TEXT or SQLITE_BLOB */ double r; /* Value if eType is SQLITE_FLOAT */ i64 i; /* Value if eType is SQLITE_INTEGER */ } u; u8 eType; /* SQLITE_NULL, SQLITE_INTEGER ... etc. */ int nByte; /* Size in byte of text or blob. */ tRowcnt nEq; /* Est. number of rows where the key equals this sample */ tRowcnt nLt; /* Est. number of rows where key is less than this sample */ tRowcnt nDLt; /* Est. number of distinct keys less than this sample */ }; /* ** Each token coming out of the lexer is an instance of ** this structure. Tokens are also used as part of an expression. ** ** Note if Token.z==0 then Token.dyn and Token.n are undefined and |
︙ | ︙ | |||
2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 | #if !defined(SQLITE_OMIT_VIEW) || !defined(SQLITE_OMIT_VIRTUALTABLE) int sqlite3ViewGetColumnNames(Parse*,Table*); #else # define sqlite3ViewGetColumnNames(A,B) 0 #endif void sqlite3DropTable(Parse*, SrcList*, int, int); void sqlite3DeleteTable(sqlite3*, Table*); #ifndef SQLITE_OMIT_AUTOINCREMENT void sqlite3AutoincrementBegin(Parse *pParse); void sqlite3AutoincrementEnd(Parse *pParse); #else # define sqlite3AutoincrementBegin(X) # define sqlite3AutoincrementEnd(X) | > | 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 | #if !defined(SQLITE_OMIT_VIEW) || !defined(SQLITE_OMIT_VIRTUALTABLE) int sqlite3ViewGetColumnNames(Parse*,Table*); #else # define sqlite3ViewGetColumnNames(A,B) 0 #endif void sqlite3DropTable(Parse*, SrcList*, int, int); void sqlite3CodeDropTable(Parse*, Table*, int, int); void sqlite3DeleteTable(sqlite3*, Table*); #ifndef SQLITE_OMIT_AUTOINCREMENT void sqlite3AutoincrementBegin(Parse *pParse); void sqlite3AutoincrementEnd(Parse *pParse); #else # define sqlite3AutoincrementBegin(X) # define sqlite3AutoincrementEnd(X) |
︙ | ︙ | |||
2968 2969 2970 2971 2972 2973 2974 | const void *sqlite3ValueText(sqlite3_value*, u8); int sqlite3ValueBytes(sqlite3_value*, u8); void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8, void(*)(void*)); void sqlite3ValueFree(sqlite3_value*); sqlite3_value *sqlite3ValueNew(sqlite3 *); char *sqlite3Utf16to8(sqlite3 *, const void*, int, u8); | | | 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 | const void *sqlite3ValueText(sqlite3_value*, u8); int sqlite3ValueBytes(sqlite3_value*, u8); void sqlite3ValueSetStr(sqlite3_value*, int, const void *,u8, void(*)(void*)); void sqlite3ValueFree(sqlite3_value*); sqlite3_value *sqlite3ValueNew(sqlite3 *); char *sqlite3Utf16to8(sqlite3 *, const void*, int, u8); #ifdef SQLITE_ENABLE_STAT3 char *sqlite3Utf8to16(sqlite3 *, u8, char *, int, int *); #endif int sqlite3ValueFromExpr(sqlite3 *, Expr *, u8, u8, sqlite3_value **); void sqlite3ValueApplyAffinity(sqlite3_value *, u8, u8); #ifndef SQLITE_AMALGAMATION extern const unsigned char sqlite3OpcodeProperty[]; extern const unsigned char sqlite3UpperToLower[]; |
︙ | ︙ |
Changes to src/test_config.c.
︙ | ︙ | |||
419 420 421 422 423 424 425 426 427 428 429 430 431 432 | #endif #ifdef SQLITE_ENABLE_STAT2 Tcl_SetVar2(interp, "sqlite_options", "stat2", "1", TCL_GLOBAL_ONLY); #else Tcl_SetVar2(interp, "sqlite_options", "stat2", "0", TCL_GLOBAL_ONLY); #endif #if !defined(SQLITE_ENABLE_LOCKING_STYLE) # if defined(__APPLE__) # define SQLITE_ENABLE_LOCKING_STYLE 1 # else # define SQLITE_ENABLE_LOCKING_STYLE 0 # endif | > > > > > > | 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 | #endif #ifdef SQLITE_ENABLE_STAT2 Tcl_SetVar2(interp, "sqlite_options", "stat2", "1", TCL_GLOBAL_ONLY); #else Tcl_SetVar2(interp, "sqlite_options", "stat2", "0", TCL_GLOBAL_ONLY); #endif #ifdef SQLITE_ENABLE_STAT3 Tcl_SetVar2(interp, "sqlite_options", "stat3", "1", TCL_GLOBAL_ONLY); #else Tcl_SetVar2(interp, "sqlite_options", "stat3", "0", TCL_GLOBAL_ONLY); #endif #if !defined(SQLITE_ENABLE_LOCKING_STYLE) # if defined(__APPLE__) # define SQLITE_ENABLE_LOCKING_STYLE 1 # else # define SQLITE_ENABLE_LOCKING_STYLE 0 # endif |
︙ | ︙ |
Changes to src/utf.c.
︙ | ︙ | |||
460 461 462 463 464 465 466 | ** is set to the length of the returned string in bytes. The call should ** arrange to call sqlite3DbFree() on the returned pointer when it is ** no longer required. ** ** If a malloc failure occurs, NULL is returned and the db.mallocFailed ** flag set. */ | | | 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 | ** is set to the length of the returned string in bytes. The call should ** arrange to call sqlite3DbFree() on the returned pointer when it is ** no longer required. ** ** If a malloc failure occurs, NULL is returned and the db.mallocFailed ** flag set. */ #ifdef SQLITE_ENABLE_STAT3 char *sqlite3Utf8to16(sqlite3 *db, u8 enc, char *z, int n, int *pnOut){ Mem m; memset(&m, 0, sizeof(m)); m.db = db; sqlite3VdbeMemSetStr(&m, z, n, SQLITE_UTF8, SQLITE_STATIC); if( sqlite3VdbeMemTranslate(&m, enc) ){ assert( db->mallocFailed ); |
︙ | ︙ |
Changes to src/vdbeaux.c.
︙ | ︙ | |||
571 572 573 574 575 576 577 | } /* ** Change the P2 operand of instruction addr so that it points to ** the address of the next instruction to be coded. */ void sqlite3VdbeJumpHere(Vdbe *p, int addr){ | | | | 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 | } /* ** Change the P2 operand of instruction addr so that it points to ** the address of the next instruction to be coded. */ void sqlite3VdbeJumpHere(Vdbe *p, int addr){ assert( addr>=0 || p->db->mallocFailed ); if( addr>=0 ) sqlite3VdbeChangeP2(p, addr, p->nOp); } /* ** If the input FuncDef structure is ephemeral, then free it. If ** the FuncDef is not ephermal, then do nothing. */ |
︙ | ︙ |
Changes to src/vdbemem.c.
︙ | ︙ | |||
1022 1023 1024 1025 1026 1027 1028 | if( !pExpr ){ *ppVal = 0; return SQLITE_OK; } op = pExpr->op; | | | | | 1022 1023 1024 1025 1026 1027 1028 1029 1030 1031 1032 1033 1034 1035 1036 1037 1038 1039 1040 | if( !pExpr ){ *ppVal = 0; return SQLITE_OK; } op = pExpr->op; /* op can only be TK_REGISTER if we have compiled with SQLITE_ENABLE_STAT3. ** The ifdef here is to enable us to achieve 100% branch test coverage even ** when SQLITE_ENABLE_STAT3 is omitted. */ #ifdef SQLITE_ENABLE_STAT3 if( op==TK_REGISTER ) op = pExpr->op2; #else if( NEVER(op==TK_REGISTER) ) op = pExpr->op2; #endif /* Handle negative integers in a single step. This is needed in the ** case when the value is -9223372036854775808. |
︙ | ︙ |
Changes to src/where.c.
︙ | ︙ | |||
114 115 116 117 118 119 120 | #define TERM_DYNAMIC 0x01 /* Need to call sqlite3ExprDelete(db, pExpr) */ #define TERM_VIRTUAL 0x02 /* Added by the optimizer. Do not code */ #define TERM_CODED 0x04 /* This term is already coded */ #define TERM_COPIED 0x08 /* Has a child */ #define TERM_ORINFO 0x10 /* Need to free the WhereTerm.u.pOrInfo object */ #define TERM_ANDINFO 0x20 /* Need to free the WhereTerm.u.pAndInfo obj */ #define TERM_OR_OK 0x40 /* Used during OR-clause processing */ | | | | 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 | #define TERM_DYNAMIC 0x01 /* Need to call sqlite3ExprDelete(db, pExpr) */ #define TERM_VIRTUAL 0x02 /* Added by the optimizer. Do not code */ #define TERM_CODED 0x04 /* This term is already coded */ #define TERM_COPIED 0x08 /* Has a child */ #define TERM_ORINFO 0x10 /* Need to free the WhereTerm.u.pOrInfo object */ #define TERM_ANDINFO 0x20 /* Need to free the WhereTerm.u.pAndInfo obj */ #define TERM_OR_OK 0x40 /* Used during OR-clause processing */ #ifdef SQLITE_ENABLE_STAT3 # define TERM_VNULL 0x80 /* Manufactured x>NULL or x<=NULL term */ #else # define TERM_VNULL 0x00 /* Disabled if not using stat3 */ #endif /* ** An instance of the following structure holds all information about a ** WHERE clause. Mostly this is a container for one or more WhereTerms. */ struct WhereClause { |
︙ | ︙ | |||
1336 1337 1338 1339 1340 1341 1342 | pTerm->nChild = 1; pTerm->wtFlags |= TERM_COPIED; pNewTerm->prereqAll = pTerm->prereqAll; } } #endif /* SQLITE_OMIT_VIRTUALTABLE */ | | | | 1336 1337 1338 1339 1340 1341 1342 1343 1344 1345 1346 1347 1348 1349 1350 1351 | pTerm->nChild = 1; pTerm->wtFlags |= TERM_COPIED; pNewTerm->prereqAll = pTerm->prereqAll; } } #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifdef SQLITE_ENABLE_STAT3 /* When sqlite_stat3 histogram data is available an operator of the ** form "x IS NOT NULL" can sometimes be evaluated more efficiently ** as "x>NULL" if x is not an INTEGER PRIMARY KEY. So construct a ** virtual term of that form. ** ** Note that the virtual term must be tagged with TERM_VNULL. This ** TERM_VNULL tag will suppress the not-null check at the beginning ** of the loop. Without the TERM_VNULL flag, the not-null check at |
︙ | ︙ | |||
1375 1376 1377 1378 1379 1380 1381 | pNewTerm->iParent = idxTerm; pTerm = &pWC->a[idxTerm]; pTerm->nChild = 1; pTerm->wtFlags |= TERM_COPIED; pNewTerm->prereqAll = pTerm->prereqAll; } } | | | 1375 1376 1377 1378 1379 1380 1381 1382 1383 1384 1385 1386 1387 1388 1389 | pNewTerm->iParent = idxTerm; pTerm = &pWC->a[idxTerm]; pTerm->nChild = 1; pTerm->wtFlags |= TERM_COPIED; pNewTerm->prereqAll = pTerm->prereqAll; } } #endif /* SQLITE_ENABLE_STAT */ /* Prevent ON clause terms of a LEFT JOIN from being used to drive ** an index for tables to the left of the join. */ pTerm->prereqRight |= extraRight; } |
︙ | ︙ | |||
2423 2424 2425 2426 2427 2428 2429 2430 | /* Try to find a more efficient access pattern by using multiple indexes ** to optimize an OR expression within the WHERE clause. */ bestOrClauseIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost); } #endif /* SQLITE_OMIT_VIRTUALTABLE */ /* | > < | < | < < < < < < < < < > > < < | < < < | | | > > > > > > > > | > | | | | | > | | | | | > > > | > | > > > | | > | > | > > > > > > > > > > > | > > > > > > > < < < < < | | | | > > | | | | > | > > > > > > > > > > > > > > > > > > > > > > > > > > | | > > | | | 2423 2424 2425 2426 2427 2428 2429 2430 2431 2432 2433 2434 2435 2436 2437 2438 2439 2440 2441 2442 2443 2444 2445 2446 2447 2448 2449 2450 2451 2452 2453 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 2469 2470 2471 2472 2473 2474 2475 2476 2477 2478 2479 2480 2481 2482 2483 2484 2485 2486 2487 2488 2489 2490 2491 2492 2493 2494 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 2555 2556 2557 2558 2559 2560 2561 2562 2563 2564 2565 2566 2567 2568 2569 2570 2571 2572 2573 2574 2575 2576 2577 2578 2579 2580 2581 2582 2583 2584 2585 2586 2587 2588 2589 2590 2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 2610 2611 2612 2613 2614 2615 2616 | /* Try to find a more efficient access pattern by using multiple indexes ** to optimize an OR expression within the WHERE clause. */ bestOrClauseIndex(pParse, pWC, pSrc, notReady, notValid, pOrderBy, pCost); } #endif /* SQLITE_OMIT_VIRTUALTABLE */ #ifdef SQLITE_ENABLE_STAT3 /* ** Estimate the location of a particular key among all keys in an ** index. Store the results in aStat as follows: ** ** aStat[0] Est. number of rows less than pVal ** aStat[1] Est. number of rows equal to pVal ** ** Return SQLITE_OK on success. */ static int whereKeyStats( Parse *pParse, /* Database connection */ Index *pIdx, /* Index to consider domain of */ sqlite3_value *pVal, /* Value to consider */ int roundUp, /* Round up if true. Round down if false */ tRowcnt *aStat /* OUT: stats written here */ ){ tRowcnt n; IndexSample *aSample; int i, eType; int isEq = 0; i64 v; double r, rS; assert( roundUp==0 || roundUp==1 ); assert( pIdx->nSample>0 ); if( pVal==0 ) return SQLITE_ERROR; n = pIdx->aiRowEst[0]; aSample = pIdx->aSample; i = 0; eType = sqlite3_value_type(pVal); if( eType==SQLITE_INTEGER ){ v = sqlite3_value_int64(pVal); r = (i64)v; for(i=0; i<pIdx->nSample; i++){ if( aSample[i].eType==SQLITE_NULL ) continue; if( aSample[i].eType>=SQLITE_TEXT ) break; if( aSample[i].eType==SQLITE_INTEGER ){ if( aSample[i].u.i>=v ){ isEq = aSample[i].u.i==v; break; } }else{ assert( aSample[i].eType==SQLITE_FLOAT ); if( aSample[i].u.r>=r ){ isEq = aSample[i].u.r==r; break; } } } }else if( eType==SQLITE_FLOAT ){ r = sqlite3_value_double(pVal); for(i=0; i<pIdx->nSample; i++){ if( aSample[i].eType==SQLITE_NULL ) continue; if( aSample[i].eType>=SQLITE_TEXT ) break; if( aSample[i].eType==SQLITE_FLOAT ){ rS = aSample[i].u.r; }else{ rS = aSample[i].u.i; } if( rS>=r ){ isEq = rS==r; break; } } }else if( eType==SQLITE_NULL ){ i = 0; if( aSample[0].eType==SQLITE_NULL ) isEq = 1; }else{ assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB ); for(i=0; i<pIdx->nSample; i++){ if( aSample[i].eType==SQLITE_TEXT || aSample[i].eType==SQLITE_BLOB ){ break; } } if( i<pIdx->nSample ){ sqlite3 *db = pParse->db; CollSeq *pColl; const u8 *z; if( eType==SQLITE_BLOB ){ z = (const u8 *)sqlite3_value_blob(pVal); pColl = db->pDfltColl; assert( pColl->enc==SQLITE_UTF8 ); }else{ pColl = sqlite3GetCollSeq(db, SQLITE_UTF8, 0, *pIdx->azColl); if( pColl==0 ){ sqlite3ErrorMsg(pParse, "no such collation sequence: %s", *pIdx->azColl); return SQLITE_ERROR; } z = (const u8 *)sqlite3ValueText(pVal, pColl->enc); if( !z ){ return SQLITE_NOMEM; } assert( z && pColl && pColl->xCmp ); } n = sqlite3ValueBytes(pVal, pColl->enc); for(; i<pIdx->nSample; i++){ int c; int eSampletype = aSample[i].eType; if( eSampletype<eType ) continue; if( eSampletype!=eType ) break; #ifndef SQLITE_OMIT_UTF16 if( pColl->enc!=SQLITE_UTF8 ){ int nSample; char *zSample = sqlite3Utf8to16( db, pColl->enc, aSample[i].u.z, aSample[i].nByte, &nSample ); if( !zSample ){ assert( db->mallocFailed ); return SQLITE_NOMEM; } c = pColl->xCmp(pColl->pUser, nSample, zSample, n, z); sqlite3DbFree(db, zSample); }else #endif { c = pColl->xCmp(pColl->pUser, aSample[i].nByte, aSample[i].u.z, n, z); } if( c>=0 ){ if( c==0 ) isEq = 1; break; } } } } /* At this point, aSample[i] is the first sample that is greater than ** or equal to pVal. Or if i==pIdx->nSample, then all samples are less ** than pVal. If aSample[i]==pVal, then isEq==1. */ if( isEq ){ assert( i<pIdx->nSample ); aStat[0] = aSample[i].nLt; aStat[1] = aSample[i].nEq; }else{ tRowcnt iLower, iUpper, iGap; if( i==0 ){ iLower = 0; iUpper = aSample[0].nLt; }else{ iUpper = i>=pIdx->nSample ? n : aSample[i].nLt; iLower = aSample[i-1].nEq + aSample[i-1].nLt; } aStat[1] = pIdx->avgEq; if( iLower>=iUpper ){ iGap = 0; }else{ iGap = iUpper - iLower; } if( roundUp ){ iGap = (iGap*2)/3; }else{ iGap = iGap/3; } aStat[0] = iLower + iGap; } return SQLITE_OK; } #endif /* SQLITE_ENABLE_STAT3 */ /* ** If expression pExpr represents a literal value, set *pp to point to ** an sqlite3_value structure containing the same value, with affinity ** aff applied to it, before returning. It is the responsibility of the ** caller to eventually release this structure by passing it to ** sqlite3ValueFree(). ** ** If the current parse is a recompile (sqlite3Reprepare()) and pExpr ** is an SQL variable that currently has a non-NULL value bound to it, ** create an sqlite3_value structure containing this value, again with ** affinity aff applied to it, instead. ** ** If neither of the above apply, set *pp to NULL. ** ** If an error occurs, return an error code. Otherwise, SQLITE_OK. */ #ifdef SQLITE_ENABLE_STAT3 static int valueFromExpr( Parse *pParse, Expr *pExpr, u8 aff, sqlite3_value **pp ){ if( pExpr->op==TK_VARIABLE |
︙ | ︙ | |||
2600 2601 2602 2603 2604 2605 2606 | ** then nEq should be passed the value 1 (as the range restricted column, ** b, is the second left-most column of the index). Or, if the query is: ** ** ... FROM t1 WHERE a > ? AND a < ? ... ** ** then nEq should be passed 0. ** | | < < | < | > | | | | | | | | < < | | | < | > > > > | > > | > > > > | | > | | < < < < < | < | < | < < | < | | < < < < | < < < < < < | | | | | < | > | | < < | < < < < < | < | | | 2650 2651 2652 2653 2654 2655 2656 2657 2658 2659 2660 2661 2662 2663 2664 2665 2666 2667 2668 2669 2670 2671 2672 2673 2674 2675 2676 2677 2678 2679 2680 2681 2682 2683 2684 2685 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 | ** then nEq should be passed the value 1 (as the range restricted column, ** b, is the second left-most column of the index). Or, if the query is: ** ** ... FROM t1 WHERE a > ? AND a < ? ... ** ** then nEq should be passed 0. ** ** The returned value is an integer divisor to reduce the estimated ** search space. A return value of 1 means that range constraints are ** no help at all. A return value of 2 means range constraints are ** expected to reduce the search space by half. And so forth... ** ** In the absence of sqlite_stat3 ANALYZE data, each range inequality ** reduces the search space by a factor of 4. Hence a single constraint (x>?) ** results in a return of 4 and a range constraint (x>? AND x<?) results ** in a return of 16. */ static int whereRangeScanEst( Parse *pParse, /* Parsing & code generating context */ Index *p, /* The index containing the range-compared column; "x" */ int nEq, /* index into p->aCol[] of the range-compared column */ WhereTerm *pLower, /* Lower bound on the range. ex: "x>123" Might be NULL */ WhereTerm *pUpper, /* Upper bound on the range. ex: "x<455" Might be NULL */ double *pRangeDiv /* OUT: Reduce search space by this divisor */ ){ int rc = SQLITE_OK; #ifdef SQLITE_ENABLE_STAT3 if( nEq==0 && p->nSample ){ sqlite3_value *pRangeVal; tRowcnt iLower = 0; tRowcnt iUpper = p->aiRowEst[0]; tRowcnt a[2]; u8 aff = p->pTable->aCol[p->aiColumn[0]].affinity; if( pLower ){ Expr *pExpr = pLower->pExpr->pRight; rc = valueFromExpr(pParse, pExpr, aff, &pRangeVal); assert( pLower->eOperator==WO_GT || pLower->eOperator==WO_GE ); if( rc==SQLITE_OK && whereKeyStats(pParse, p, pRangeVal, 0, a)==SQLITE_OK ){ iLower = a[0]; if( pLower->eOperator==WO_GT ) iLower += a[1]; } sqlite3ValueFree(pRangeVal); } if( rc==SQLITE_OK && pUpper ){ Expr *pExpr = pUpper->pExpr->pRight; rc = valueFromExpr(pParse, pExpr, aff, &pRangeVal); assert( pUpper->eOperator==WO_LT || pUpper->eOperator==WO_LE ); if( rc==SQLITE_OK && whereKeyStats(pParse, p, pRangeVal, 1, a)==SQLITE_OK ){ iUpper = a[0]; if( pUpper->eOperator==WO_LE ) iUpper += a[1]; } sqlite3ValueFree(pRangeVal); } if( rc==SQLITE_OK ){ if( iUpper<=iLower ){ *pRangeDiv = (double)p->aiRowEst[0]; }else{ *pRangeDiv = (double)p->aiRowEst[0]/(double)(iUpper - iLower); } WHERETRACE(("range scan regions: %u..%u div=%g\n", (u32)iLower, (u32)iUpper, *pRangeDiv)); return SQLITE_OK; } } #else UNUSED_PARAMETER(pParse); UNUSED_PARAMETER(p); UNUSED_PARAMETER(nEq); #endif assert( pLower || pUpper ); *pRangeDiv = (double)1; if( pLower && (pLower->wtFlags & TERM_VNULL)==0 ) *pRangeDiv *= (double)4; if( pUpper ) *pRangeDiv *= (double)4; return rc; } #ifdef SQLITE_ENABLE_STAT3 /* ** Estimate the number of rows that will be returned based on ** an equality constraint x=VALUE and where that VALUE occurs in ** the histogram data. This only works when x is the left-most ** column of an index and sqlite_stat3 histogram data is available ** for that index. When pExpr==NULL that means the constraint is ** "x IS NULL" instead of "x=VALUE". ** ** Write the estimated row count into *pnRow and return SQLITE_OK. ** If unable to make an estimate, leave *pnRow unchanged and return ** non-zero. ** ** This routine can fail if it is unable to load a collating sequence ** required for string comparison, or if unable to allocate memory ** for a UTF conversion required for comparison. The error is stored ** in the pParse structure. */ static int whereEqualScanEst( Parse *pParse, /* Parsing & code generating context */ Index *p, /* The index whose left-most column is pTerm */ Expr *pExpr, /* Expression for VALUE in the x=VALUE constraint */ double *pnRow /* Write the revised row estimate here */ ){ sqlite3_value *pRhs = 0; /* VALUE on right-hand side of pTerm */ u8 aff; /* Column affinity */ int rc; /* Subfunction return code */ tRowcnt a[2]; /* Statistics */ assert( p->aSample!=0 ); assert( p->nSample>0 ); aff = p->pTable->aCol[p->aiColumn[0]].affinity; if( pExpr ){ rc = valueFromExpr(pParse, pExpr, aff, &pRhs); if( rc ) goto whereEqualScanEst_cancel; }else{ pRhs = sqlite3ValueNew(pParse->db); } if( pRhs==0 ) return SQLITE_NOTFOUND; rc = whereKeyStats(pParse, p, pRhs, 0, a); if( rc==SQLITE_OK ){ WHERETRACE(("equality scan regions: %d\n", (int)a[1])); *pnRow = a[1]; } whereEqualScanEst_cancel: sqlite3ValueFree(pRhs); return rc; } #endif /* defined(SQLITE_ENABLE_STAT3) */ #ifdef SQLITE_ENABLE_STAT3 /* ** Estimate the number of rows that will be returned based on ** an IN constraint where the right-hand side of the IN operator ** is a list of values. Example: ** ** WHERE x IN (1,2,3,4) ** |
︙ | ︙ | |||
2771 2772 2773 2774 2775 2776 2777 | */ static int whereInScanEst( Parse *pParse, /* Parsing & code generating context */ Index *p, /* The index whose left-most column is pTerm */ ExprList *pList, /* The value list on the RHS of "x IN (v1,v2,v3,...)" */ double *pnRow /* Write the revised row estimate here */ ){ | < < < | > | < < < | < < < < < | < > | < < < < < < < < < | < < < < | < < < < < < < < < < | < < | | 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 | */ static int whereInScanEst( Parse *pParse, /* Parsing & code generating context */ Index *p, /* The index whose left-most column is pTerm */ ExprList *pList, /* The value list on the RHS of "x IN (v1,v2,v3,...)" */ double *pnRow /* Write the revised row estimate here */ ){ int rc = SQLITE_OK; /* Subfunction return code */ double nEst; /* Number of rows for a single term */ double nRowEst = (double)0; /* New estimate of the number of rows */ int i; /* Loop counter */ assert( p->aSample!=0 ); for(i=0; rc==SQLITE_OK && i<pList->nExpr; i++){ nEst = p->aiRowEst[0]; rc = whereEqualScanEst(pParse, p, pList->a[i].pExpr, &nEst); nRowEst += nEst; } if( rc==SQLITE_OK ){ if( nRowEst > p->aiRowEst[0] ) nRowEst = p->aiRowEst[0]; *pnRow = nRowEst; WHERETRACE(("IN row estimate: est=%g\n", nRowEst)); } return rc; } #endif /* defined(SQLITE_ENABLE_STAT3) */ /* ** Find the best query plan for accessing a particular table. Write the ** best query plan and its cost into the WhereCost object supplied as the ** last parameter. ** |
︙ | ︙ | |||
2871 2872 2873 2874 2875 2876 2877 | ){ int iCur = pSrc->iCursor; /* The cursor of the table to be accessed */ Index *pProbe; /* An index we are evaluating */ Index *pIdx; /* Copy of pProbe, or zero for IPK index */ int eqTermMask; /* Current mask of valid equality operators */ int idxEqTermMask; /* Index mask of valid equality operators */ Index sPk; /* A fake index object for the primary key */ | | | 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 | ){ int iCur = pSrc->iCursor; /* The cursor of the table to be accessed */ Index *pProbe; /* An index we are evaluating */ Index *pIdx; /* Copy of pProbe, or zero for IPK index */ int eqTermMask; /* Current mask of valid equality operators */ int idxEqTermMask; /* Index mask of valid equality operators */ Index sPk; /* A fake index object for the primary key */ tRowcnt aiRowEstPk[2]; /* The aiRowEst[] value for the sPk index */ int aiColumnPk = -1; /* The aColumn[] value for the sPk index */ int wsFlagMask; /* Allowed flags in pCost->plan.wsFlag */ /* Initialize the cost to a worst-case value */ memset(pCost, 0, sizeof(*pCost)); pCost->rCost = SQLITE_BIG_DBL; |
︙ | ︙ | |||
2926 2927 2928 2929 2930 2931 2932 | eqTermMask = WO_EQ|WO_IN; pIdx = 0; } /* Loop over all indices looking for the best one to use */ for(; pProbe; pIdx=pProbe=pProbe->pNext){ | | | | 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 | eqTermMask = WO_EQ|WO_IN; pIdx = 0; } /* Loop over all indices looking for the best one to use */ for(; pProbe; pIdx=pProbe=pProbe->pNext){ const tRowcnt * const aiRowEst = pProbe->aiRowEst; double cost; /* Cost of using pProbe */ double nRow; /* Estimated number of rows in result set */ double log10N = (double)1; /* base-10 logarithm of nRow (inexact) */ int rev; /* True to scan in reverse order */ int wsFlags = 0; Bitmask used = 0; /* The following variables are populated based on the properties of ** index being evaluated. They are then used to determine the expected ** cost and number of rows returned. |
︙ | ︙ | |||
2969 2970 2971 2972 2973 2974 2975 | ** ** bInEst: ** Set to true if there was at least one "x IN (SELECT ...)" term used ** in determining the value of nInMul. Note that the RHS of the ** IN operator must be a SELECT, not a value list, for this variable ** to be true. ** | | | < < | | > | < | 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 | ** ** bInEst: ** Set to true if there was at least one "x IN (SELECT ...)" term used ** in determining the value of nInMul. Note that the RHS of the ** IN operator must be a SELECT, not a value list, for this variable ** to be true. ** ** rangeDiv: ** An estimate of a divisor by which to reduce the search space due ** to inequality constraints. In the absence of sqlite_stat3 ANALYZE ** data, a single inequality reduces the search space to 1/4rd its ** original size (rangeDiv==4). Two inequalities reduce the search ** space to 1/16th of its original size (rangeDiv==16). ** ** bSort: ** Boolean. True if there is an ORDER BY clause that will require an ** external sort (i.e. scanning the index being evaluated will not ** correctly order records). ** ** bLookup: |
︙ | ︙ | |||
3001 3002 3003 3004 3005 3006 3007 | ** ** SELECT a, b FROM tbl WHERE a = 1; ** SELECT a, b, c FROM tbl WHERE a = 1; */ int nEq; /* Number of == or IN terms matching index */ int bInEst = 0; /* True if "x IN (SELECT...)" seen */ int nInMul = 1; /* Number of distinct equalities to lookup */ | | | | 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 | ** ** SELECT a, b FROM tbl WHERE a = 1; ** SELECT a, b, c FROM tbl WHERE a = 1; */ int nEq; /* Number of == or IN terms matching index */ int bInEst = 0; /* True if "x IN (SELECT...)" seen */ int nInMul = 1; /* Number of distinct equalities to lookup */ double rangeDiv = (double)1; /* Estimated reduction in search space */ int nBound = 0; /* Number of range constraints seen */ int bSort = !!pOrderBy; /* True if external sort required */ int bDist = !!pDistinct; /* True if index cannot help with DISTINCT */ int bLookup = 0; /* True if not a covering index */ WhereTerm *pTerm; /* A single term of the WHERE clause */ #ifdef SQLITE_ENABLE_STAT3 WhereTerm *pFirstTerm = 0; /* First term matching the index */ #endif /* Determine the values of nEq and nInMul */ for(nEq=0; nEq<pProbe->nColumn; nEq++){ int j = pProbe->aiColumn[nEq]; pTerm = findTerm(pWC, iCur, j, notReady, eqTermMask, pIdx); |
︙ | ︙ | |||
3031 3032 3033 3034 3035 3036 3037 | }else if( ALWAYS(pExpr->x.pList && pExpr->x.pList->nExpr) ){ /* "x IN (value, value, ...)" */ nInMul *= pExpr->x.pList->nExpr; } }else if( pTerm->eOperator & WO_ISNULL ){ wsFlags |= WHERE_COLUMN_NULL; } | | | | | 3022 3023 3024 3025 3026 3027 3028 3029 3030 3031 3032 3033 3034 3035 3036 3037 3038 3039 3040 3041 3042 3043 3044 3045 3046 3047 3048 | }else if( ALWAYS(pExpr->x.pList && pExpr->x.pList->nExpr) ){ /* "x IN (value, value, ...)" */ nInMul *= pExpr->x.pList->nExpr; } }else if( pTerm->eOperator & WO_ISNULL ){ wsFlags |= WHERE_COLUMN_NULL; } #ifdef SQLITE_ENABLE_STAT3 if( nEq==0 && pProbe->aSample ) pFirstTerm = pTerm; #endif used |= pTerm->prereqRight; } /* Determine the value of rangeDiv */ if( nEq<pProbe->nColumn && pProbe->bUnordered==0 ){ int j = pProbe->aiColumn[nEq]; if( findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE|WO_GT|WO_GE, pIdx) ){ WhereTerm *pTop = findTerm(pWC, iCur, j, notReady, WO_LT|WO_LE, pIdx); WhereTerm *pBtm = findTerm(pWC, iCur, j, notReady, WO_GT|WO_GE, pIdx); whereRangeScanEst(pParse, pProbe, nEq, pBtm, pTop, &rangeDiv); if( pTop ){ nBound = 1; wsFlags |= WHERE_TOP_LIMIT; used |= pTop->prereqRight; } if( pBtm ){ nBound++; |
︙ | ︙ | |||
3115 3116 3117 3118 3119 3120 3121 | */ nRow = (double)(aiRowEst[nEq] * nInMul); if( bInEst && nRow*2>aiRowEst[0] ){ nRow = aiRowEst[0]/2; nInMul = (int)(nRow / aiRowEst[nEq]); } | | > > | | | | 3106 3107 3108 3109 3110 3111 3112 3113 3114 3115 3116 3117 3118 3119 3120 3121 3122 3123 3124 3125 3126 3127 3128 3129 3130 3131 3132 3133 3134 3135 3136 3137 3138 3139 3140 3141 3142 3143 | */ nRow = (double)(aiRowEst[nEq] * nInMul); if( bInEst && nRow*2>aiRowEst[0] ){ nRow = aiRowEst[0]/2; nInMul = (int)(nRow / aiRowEst[nEq]); } #ifdef SQLITE_ENABLE_STAT3 /* If the constraint is of the form x=VALUE or x IN (E1,E2,...) ** and we do not think that values of x are unique and if histogram ** data is available for column x, then it might be possible ** to get a better estimate on the number of rows based on ** VALUE and how common that value is according to the histogram. */ if( nRow>(double)1 && nEq==1 && pFirstTerm!=0 && aiRowEst[1]>1 ){ assert( (pFirstTerm->eOperator & (WO_EQ|WO_ISNULL|WO_IN))!=0 ); if( pFirstTerm->eOperator & (WO_EQ|WO_ISNULL) ){ testcase( pFirstTerm->eOperator==WO_EQ ); testcase( pFirstTerm->eOperator==WO_ISNULL ); whereEqualScanEst(pParse, pProbe, pFirstTerm->pExpr->pRight, &nRow); }else if( bInEst==0 ){ assert( pFirstTerm->eOperator==WO_IN ); whereInScanEst(pParse, pProbe, pFirstTerm->pExpr->x.pList, &nRow); } } #endif /* SQLITE_ENABLE_STAT3 */ /* Adjust the number of output rows and downward to reflect rows ** that are excluded by range constraints. */ nRow = nRow/rangeDiv; if( nRow<1 ) nRow = 1; /* Experiments run on real SQLite databases show that the time needed ** to do a binary search to locate a row in a table or index is roughly ** log10(N) times the time to move from one row to the next row within ** a table or index. The actual times can vary, with the size of ** records being an important factor. Both moves and searches are |
︙ | ︙ | |||
3265 3266 3267 3268 3269 3270 3271 | } } if( nRow<2 ) nRow = 2; } WHERETRACE(( | | | | 3258 3259 3260 3261 3262 3263 3264 3265 3266 3267 3268 3269 3270 3271 3272 3273 3274 3275 | } } if( nRow<2 ) nRow = 2; } WHERETRACE(( "%s(%s): nEq=%d nInMul=%d rangeDiv=%d bSort=%d bLookup=%d wsFlags=0x%x\n" " notReady=0x%llx log10N=%.1f nRow=%.1f cost=%.1f used=0x%llx\n", pSrc->pTab->zName, (pIdx ? pIdx->zName : "ipk"), nEq, nInMul, (int)rangeDiv, bSort, bLookup, wsFlags, notReady, log10N, nRow, cost, used )); /* If this index is the best we have seen so far, then record this ** index and its cost in the pCost structure. */ if( (!pIdx || wsFlags) |
︙ | ︙ |
Changes to test/alter.test.
︙ | ︙ | |||
843 844 845 846 847 848 849 850 851 852 853 854 855 856 | #------------------------------------------------------------------------- # Test that it is not possible to use ALTER TABLE on any system table. # set system_table_list {1 sqlite_master} catchsql ANALYZE ifcapable analyze { lappend system_table_list 2 sqlite_stat1 } ifcapable stat2 { lappend system_table_list 3 sqlite_stat2 } foreach {tn tbl} $system_table_list { do_test alter-15.$tn.1 { catchsql "ALTER TABLE $tbl RENAME TO xyz" } [list 1 "table $tbl may not be altered"] do_test alter-15.$tn.2 { | > | 843 844 845 846 847 848 849 850 851 852 853 854 855 856 857 | #------------------------------------------------------------------------- # Test that it is not possible to use ALTER TABLE on any system table. # set system_table_list {1 sqlite_master} catchsql ANALYZE ifcapable analyze { lappend system_table_list 2 sqlite_stat1 } ifcapable stat2 { lappend system_table_list 3 sqlite_stat2 } ifcapable stat3 { lappend system_table_list 4 sqlite_stat3 } foreach {tn tbl} $system_table_list { do_test alter-15.$tn.1 { catchsql "ALTER TABLE $tbl RENAME TO xyz" } [list 1 "table $tbl may not be altered"] do_test alter-15.$tn.2 { |
︙ | ︙ |
Changes to test/analyze.test.
︙ | ︙ | |||
284 285 286 287 288 289 290 | sqlite3 db test.db execsql { SELECT * FROM t4 WHERE x=1234; } } {} # Verify that DROP TABLE and DROP INDEX remove entries from the | | | | | | | | | | | | 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 | sqlite3 db test.db execsql { SELECT * FROM t4 WHERE x=1234; } } {} # Verify that DROP TABLE and DROP INDEX remove entries from the # sqlite_stat1 and sqlite_stat3 tables. # do_test analyze-5.0 { execsql { DELETE FROM t3; DELETE FROM t4; INSERT INTO t3 VALUES(1,2,3,4); INSERT INTO t3 VALUES(5,6,7,8); INSERT INTO t3 SELECT a+8, b+8, c+8, d+8 FROM t3; INSERT INTO t3 SELECT a+16, b+16, c+16, d+16 FROM t3; INSERT INTO t3 SELECT a+32, b+32, c+32, d+32 FROM t3; INSERT INTO t3 SELECT a+64, b+64, c+64, d+64 FROM t3; INSERT INTO t4 SELECT a, b, c FROM t3; ANALYZE; SELECT DISTINCT idx FROM sqlite_stat1 ORDER BY 1; SELECT DISTINCT tbl FROM sqlite_stat1 ORDER BY 1; } } {t3i1 t3i2 t3i3 t4i1 t4i2 t3 t4} ifcapable stat3 { do_test analyze-5.1 { execsql { SELECT DISTINCT idx FROM sqlite_stat3 ORDER BY 1; SELECT DISTINCT tbl FROM sqlite_stat3 ORDER BY 1; } } {t3i1 t3i2 t3i3 t4i1 t4i2 t3 t4} } do_test analyze-5.2 { execsql { DROP INDEX t3i2; SELECT DISTINCT idx FROM sqlite_stat1 ORDER BY 1; SELECT DISTINCT tbl FROM sqlite_stat1 ORDER BY 1; } } {t3i1 t3i3 t4i1 t4i2 t3 t4} ifcapable stat3 { do_test analyze-5.3 { execsql { SELECT DISTINCT idx FROM sqlite_stat3 ORDER BY 1; SELECT DISTINCT tbl FROM sqlite_stat3 ORDER BY 1; } } {t3i1 t3i3 t4i1 t4i2 t3 t4} } do_test analyze-5.4 { execsql { DROP TABLE t3; SELECT DISTINCT idx FROM sqlite_stat1 ORDER BY 1; SELECT DISTINCT tbl FROM sqlite_stat1 ORDER BY 1; } } {t4i1 t4i2 t4} ifcapable stat3 { do_test analyze-5.5 { execsql { SELECT DISTINCT idx FROM sqlite_stat3 ORDER BY 1; SELECT DISTINCT tbl FROM sqlite_stat3 ORDER BY 1; } } {t4i1 t4i2 t4} } # This test corrupts the database file so it must be the last test # in the series. # |
︙ | ︙ |
Changes to test/analyze3.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # implements tests for range and LIKE constraints that use bound variables # instead of literal constant arguments. # set testdir [file dirname $argv0] source $testdir/tester.tcl | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # implements tests for range and LIKE constraints that use bound variables # instead of literal constant arguments. # set testdir [file dirname $argv0] source $testdir/tester.tcl ifcapable !stat3 { finish_test return } #---------------------------------------------------------------------- # Test Organization: # |
︙ | ︙ | |||
93 94 95 96 97 98 99 | COMMIT; ANALYZE; } } {} do_eqp_test analyze3-1.1.2 { SELECT sum(y) FROM t1 WHERE x>200 AND x<300 | | | | | | | | | | | | | | | | | 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 | COMMIT; ANALYZE; } } {} do_eqp_test analyze3-1.1.2 { SELECT sum(y) FROM t1 WHERE x>200 AND x<300 } {0 0 0 {SEARCH TABLE t1 USING INDEX i1 (x>? AND x<?) (~179 rows)}} do_eqp_test analyze3-1.1.3 { SELECT sum(y) FROM t1 WHERE x>0 AND x<1100 } {0 0 0 {SEARCH TABLE t1 USING INDEX i1 (x>? AND x<?) (~959 rows)}} do_test analyze3-1.1.4 { sf_execsql { SELECT sum(y) FROM t1 WHERE x>200 AND x<300 } } {199 0 14850} do_test analyze3-1.1.5 { set l [string range "200" 0 end] set u [string range "300" 0 end] sf_execsql { SELECT sum(y) FROM t1 WHERE x>$l AND x<$u } } {199 0 14850} do_test analyze3-1.1.6 { set l [expr int(200)] set u [expr int(300)] sf_execsql { SELECT sum(y) FROM t1 WHERE x>$l AND x<$u } } {199 0 14850} do_test analyze3-1.1.7 { sf_execsql { SELECT sum(y) FROM t1 WHERE x>0 AND x<1100 } } {2000 0 499500} do_test analyze3-1.1.8 { set l [string range "0" 0 end] set u [string range "1100" 0 end] sf_execsql { SELECT sum(y) FROM t1 WHERE x>$l AND x<$u } } {2000 0 499500} do_test analyze3-1.1.9 { set l [expr int(0)] set u [expr int(1100)] sf_execsql { SELECT sum(y) FROM t1 WHERE x>$l AND x<$u } } {2000 0 499500} # The following tests are similar to the block above. The difference is # that the indexed column has TEXT affinity in this case. In the tests # above the affinity is INTEGER. # do_test analyze3-1.2.1 { execsql { BEGIN; CREATE TABLE t2(x TEXT, y); INSERT INTO t2 SELECT * FROM t1; CREATE INDEX i2 ON t2(x); COMMIT; ANALYZE; } } {} do_eqp_test analyze3-1.2.2 { SELECT sum(y) FROM t2 WHERE x>1 AND x<2 } {0 0 0 {SEARCH TABLE t2 USING INDEX i2 (x>? AND x<?) (~196 rows)}} do_eqp_test analyze3-1.2.3 { SELECT sum(y) FROM t2 WHERE x>0 AND x<99 } {0 0 0 {SEARCH TABLE t2 USING INDEX i2 (x>? AND x<?) (~982 rows)}} do_test analyze3-1.2.4 { sf_execsql { SELECT sum(y) FROM t2 WHERE x>12 AND x<20 } } {161 0 4760} do_test analyze3-1.2.5 { set l [string range "12" 0 end] set u [string range "20" 0 end] sf_execsql {SELECT typeof($l), typeof($u), sum(y) FROM t2 WHERE x>$l AND x<$u} } {161 0 text text 4760} do_test analyze3-1.2.6 { set l [expr int(12)] set u [expr int(20)] sf_execsql {SELECT typeof($l), typeof($u), sum(y) FROM t2 WHERE x>$l AND x<$u} } {161 0 integer integer 4760} do_test analyze3-1.2.7 { sf_execsql { SELECT sum(y) FROM t2 WHERE x>0 AND x<99 } } {1981 0 490555} do_test analyze3-1.2.8 { set l [string range "0" 0 end] set u [string range "99" 0 end] sf_execsql {SELECT typeof($l), typeof($u), sum(y) FROM t2 WHERE x>$l AND x<$u} } {1981 0 text text 490555} do_test analyze3-1.2.9 { set l [expr int(0)] set u [expr int(99)] sf_execsql {SELECT typeof($l), typeof($u), sum(y) FROM t2 WHERE x>$l AND x<$u} } {1981 0 integer integer 490555} # Same tests a third time. This time, column x has INTEGER affinity and # is not the leftmost column of the table. This triggered a bug causing # SQLite to use sub-optimal query plans in 3.6.18 and earlier. # do_test analyze3-1.3.1 { execsql { BEGIN; CREATE TABLE t3(y TEXT, x INTEGER); INSERT INTO t3 SELECT y, x FROM t1; CREATE INDEX i3 ON t3(x); COMMIT; ANALYZE; } } {} do_eqp_test analyze3-1.3.2 { SELECT sum(y) FROM t3 WHERE x>200 AND x<300 } {0 0 0 {SEARCH TABLE t3 USING INDEX i3 (x>? AND x<?) (~156 rows)}} do_eqp_test analyze3-1.3.3 { SELECT sum(y) FROM t3 WHERE x>0 AND x<1100 } {0 0 0 {SEARCH TABLE t3 USING INDEX i3 (x>? AND x<?) (~989 rows)}} do_test analyze3-1.3.4 { sf_execsql { SELECT sum(y) FROM t3 WHERE x>200 AND x<300 } } {199 0 14850} do_test analyze3-1.3.5 { set l [string range "200" 0 end] set u [string range "300" 0 end] sf_execsql { SELECT sum(y) FROM t3 WHERE x>$l AND x<$u } } {199 0 14850} do_test analyze3-1.3.6 { set l [expr int(200)] set u [expr int(300)] sf_execsql { SELECT sum(y) FROM t3 WHERE x>$l AND x<$u } } {199 0 14850} do_test analyze3-1.3.7 { sf_execsql { SELECT sum(y) FROM t3 WHERE x>0 AND x<1100 } } {2000 0 499500} do_test analyze3-1.3.8 { set l [string range "0" 0 end] set u [string range "1100" 0 end] sf_execsql { SELECT sum(y) FROM t3 WHERE x>$l AND x<$u } } {2000 0 499500} do_test analyze3-1.3.9 { set l [expr int(0)] set u [expr int(1100)] sf_execsql { SELECT sum(y) FROM t3 WHERE x>$l AND x<$u } } {2000 0 499500} #------------------------------------------------------------------------- # Test that the values of bound SQL variables may be used for the LIKE # optimization. # drop_all_tables do_test analyze3-2.1 { |
︙ | ︙ | |||
244 245 246 247 248 249 250 | append t [lindex {a b c d e f g h i j} [expr ($i%10)]] execsql { INSERT INTO t1 VALUES($i, $t) } } execsql COMMIT } {} do_eqp_test analyze3-2.2 { SELECT count(a) FROM t1 WHERE b LIKE 'a%' | | | 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 | append t [lindex {a b c d e f g h i j} [expr ($i%10)]] execsql { INSERT INTO t1 VALUES($i, $t) } } execsql COMMIT } {} do_eqp_test analyze3-2.2 { SELECT count(a) FROM t1 WHERE b LIKE 'a%' } {0 0 0 {SEARCH TABLE t1 USING INDEX i1 (b>? AND b<?) (~31250 rows)}} do_eqp_test analyze3-2.3 { SELECT count(a) FROM t1 WHERE b LIKE '%a' } {0 0 0 {SCAN TABLE t1 (~500000 rows)}} do_test analyze3-2.4 { sf_execsql { SELECT count(*) FROM t1 WHERE b LIKE 'a%' } } {101 0 100} |
︙ | ︙ |
Changes to test/analyze5.test.
1 2 3 4 5 6 7 8 9 10 11 12 | # 2011 January 19 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file implements tests for SQLite library. The focus of the tests | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # 2011 January 19 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file implements tests for SQLite library. The focus of the tests # in this file is the use of the sqlite_stat3 histogram data on tables # with many repeated values and only a few distinct values. # set testdir [file dirname $argv0] source $testdir/tester.tcl ifcapable !stat3 { finish_test return } set testprefix analyze5 proc eqp {sql {db db}} { |
︙ | ︙ | |||
51 52 53 54 55 56 57 | CREATE INDEX t1u ON t1(u); -- text CREATE INDEX t1v ON t1(v); -- mixed case text CREATE INDEX t1w ON t1(w); -- integers 0, 1, 2 and a few NULLs CREATE INDEX t1x ON t1(x); -- integers 1, 2, 3 and many NULLs CREATE INDEX t1y ON t1(y); -- integers 0 and very few 1s CREATE INDEX t1z ON t1(z); -- integers 0, 1, 2, and 3 ANALYZE; | | | | < < < | | < < | | | | < < < < < < < | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 | CREATE INDEX t1u ON t1(u); -- text CREATE INDEX t1v ON t1(v); -- mixed case text CREATE INDEX t1w ON t1(w); -- integers 0, 1, 2 and a few NULLs CREATE INDEX t1x ON t1(x); -- integers 1, 2, 3 and many NULLs CREATE INDEX t1y ON t1(y); -- integers 0 and very few 1s CREATE INDEX t1z ON t1(z); -- integers 0, 1, 2, and 3 ANALYZE; SELECT sample FROM sqlite_stat3 WHERE idx='t1u' ORDER BY nlt; } } {alpha bravo charlie delta} do_test analyze5-1.1 { db eval {SELECT DISTINCT lower(sample) FROM sqlite_stat3 WHERE idx='t1v' ORDER BY 1} } {alpha bravo charlie delta} do_test analyze5-1.2 { db eval {SELECT idx, count(*) FROM sqlite_stat3 GROUP BY 1 ORDER BY 1} } {t1t 4 t1u 4 t1v 4 t1w 4 t1x 4 t1y 2 t1z 4} # Verify that range queries generate the correct row count estimates # foreach {testid where index rows} { 1 {z>=0 AND z<=0} t1z 400 2 {z>=1 AND z<=1} t1z 300 3 {z>=2 AND z<=2} t1z 175 4 {z>=3 AND z<=3} t1z 125 5 {z>=4 AND z<=4} t1z 1 6 {z>=-1 AND z<=-1} t1z 1 7 {z>1 AND z<3} t1z 175 8 {z>0 AND z<100} t1z 600 9 {z>=1 AND z<100} t1z 600 10 {z>1 AND z<100} t1z 300 11 {z>=2 AND z<100} t1z 300 12 {z>2 AND z<100} t1z 125 13 {z>=3 AND z<100} t1z 125 14 {z>3 AND z<100} t1z 1 15 {z>=4 AND z<100} t1z 1 16 {z>=-100 AND z<=-1} t1z 1 17 {z>=-100 AND z<=0} t1z 400 18 {z>=-100 AND z<0} t1z 1 19 {z>=-100 AND z<=1} t1z 700 20 {z>=-100 AND z<2} t1z 700 21 {z>=-100 AND z<=2} t1z 875 22 {z>=-100 AND z<3} t1z 875 31 {z>=0.0 AND z<=0.0} t1z 400 32 {z>=1.0 AND z<=1.0} t1z 300 33 {z>=2.0 AND z<=2.0} t1z 175 34 {z>=3.0 AND z<=3.0} t1z 125 35 {z>=4.0 AND z<=4.0} t1z 1 36 {z>=-1.0 AND z<=-1.0} t1z 1 37 {z>1.5 AND z<3.0} t1z 174 38 {z>0.5 AND z<100} t1z 599 39 {z>=1.0 AND z<100} t1z 600 40 {z>1.5 AND z<100} t1z 299 41 {z>=2.0 AND z<100} t1z 300 42 {z>2.1 AND z<100} t1z 124 43 {z>=3.0 AND z<100} t1z 125 44 {z>3.2 AND z<100} t1z 1 45 {z>=4.0 AND z<100} t1z 1 46 {z>=-100 AND z<=-1.0} t1z 1 47 {z>=-100 AND z<=0.0} t1z 400 48 {z>=-100 AND z<0.0} t1z 1 49 {z>=-100 AND z<=1.0} t1z 700 50 {z>=-100 AND z<2.0} t1z 700 51 {z>=-100 AND z<=2.0} t1z 875 52 {z>=-100 AND z<3.0} t1z 875 101 {z=-1} t1z 1 102 {z=0} t1z 400 103 {z=1} t1z 300 104 {z=2} t1z 175 105 {z=3} t1z 125 106 {z=4} t1z 1 107 {z=-10.0} t1z 1 108 {z=0.0} t1z 400 109 {z=1.0} t1z 300 110 {z=2.0} t1z 175 111 {z=3.0} t1z 125 112 {z=4.0} t1z 1 113 {z=1.5} t1z 1 114 {z=2.5} t1z 1 201 {z IN (-1)} t1z 1 202 {z IN (0)} t1z 400 203 {z IN (1)} t1z 300 204 {z IN (2)} t1z 175 205 {z IN (3)} t1z 125 206 {z IN (4)} t1z 1 207 {z IN (0.5)} t1z 1 208 {z IN (0,1)} t1z 700 209 {z IN (0,1,2)} t1z 875 210 {z IN (0,1,2,3)} {} 100 211 {z IN (0,1,2,3,4,5)} {} 100 212 {z IN (1,2)} t1z 475 213 {z IN (2,3)} t1z 300 214 {z=3 OR z=2} t1z 300 215 {z IN (-1,3)} t1z 126 216 {z=-1 OR z=3} t1z 126 300 {y=0} t1y 974 301 {y=1} t1y 26 302 {y=0.1} t1y 1 400 {x IS NULL} t1x 400 } { # Verify that the expected index is used with the expected row count do_test analyze5-1.${testid}a { set x [lindex [eqp "SELECT * FROM t1 WHERE $where"] 3] |
︙ | ︙ | |||
200 201 202 203 204 205 206 | WHERE rowid IN (SELECT rowid FROM t1 ORDER BY random() LIMIT 5); ANALYZE; } # Verify that range queries generate the correct row count estimates # foreach {testid where index rows} { | | | | | | > > | 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 | WHERE rowid IN (SELECT rowid FROM t1 ORDER BY random() LIMIT 5); ANALYZE; } # Verify that range queries generate the correct row count estimates # foreach {testid where index rows} { 500 {x IS NULL AND u='charlie'} t1u 17 501 {x=1 AND u='charlie'} t1x 1 502 {x IS NULL} t1x 995 503 {x=1} t1x 1 504 {x IS NOT NULL} t1x 2 505 {+x IS NOT NULL} {} 500 506 {upper(x) IS NOT NULL} {} 500 } { # Verify that the expected index is used with the expected row count if {$testid==50299} {breakpoint; set sqlite_where_trace 1} do_test analyze5-1.${testid}a { set x [lindex [eqp "SELECT * FROM t1 WHERE $where"] 3] set idx {} regexp {INDEX (t1.) } $x all idx regexp {~([0-9]+) rows} $x all nrow list $idx $nrow } [list $index $rows] if {$testid==50299} exit # Verify that the same result is achieved regardless of whether or not # the index is used do_test analyze5-1.${testid}b { set w2 [string map {y +y z +z} $where] set a1 [db eval "SELECT rowid FROM t1 NOT INDEXED WHERE $w2\ ORDER BY +rowid"] |
︙ | ︙ |
Changes to test/analyze6.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 | # in this file a corner-case query planner optimization involving the # join order of two tables of different sizes. # set testdir [file dirname $argv0] source $testdir/tester.tcl | | | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 | # in this file a corner-case query planner optimization involving the # join order of two tables of different sizes. # set testdir [file dirname $argv0] source $testdir/tester.tcl ifcapable !stat3 { finish_test return } set testprefix analyze6 proc eqp {sql {db db}} { |
︙ | ︙ |
Changes to test/analyze7.test.
︙ | ︙ | |||
78 79 80 81 82 83 84 | } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} do_test analyze7-3.1 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE b=123;} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b=?) (~10 rows)}} do_test analyze7-3.2.1 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE c=?;} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1cd (c=?) (~86 rows)}} | | | | | > | | | | | | > | 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 | } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} do_test analyze7-3.1 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE b=123;} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b=?) (~10 rows)}} do_test analyze7-3.2.1 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE c=?;} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1cd (c=?) (~86 rows)}} ifcapable stat3 { # If ENABLE_STAT3 is defined, SQLite comes up with a different estimated # row count for (c=2) than it does for (c=?). do_test analyze7-3.2.2 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE c=2;} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1cd (c=?) (~57 rows)}} } else { # If ENABLE_STAT3 is not defined, the expected row count for (c=2) is the # same as that for (c=?). do_test analyze7-3.2.3 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE c=2;} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1cd (c=?) (~86 rows)}} } do_test analyze7-3.3 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE a=123 AND b=123} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} ifcapable {!stat3} { do_test analyze7-3.4 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE c=123 AND b=123} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b=?) (~2 rows)}} do_test analyze7-3.5 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE a=123 AND c=123} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} } do_test analyze7-3.6 { execsql {EXPLAIN QUERY PLAN SELECT * FROM t1 WHERE c=123 AND d=123 AND b=123} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1cd (c=? AND d=?) (~1 rows)}} finish_test |
Added test/analyze8.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | # 2011 August 13 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file implements tests for SQLite library. The focus of the tests # in this file is testing the capabilities of sqlite_stat3. # set testdir [file dirname $argv0] source $testdir/tester.tcl ifcapable !stat3 { finish_test return } set testprefix analyze8 proc eqp {sql {db db}} { uplevel execsql [list "EXPLAIN QUERY PLAN $sql"] $db } # Scenario: # # Two indices. One has mostly singleton entries, but for a few # values there are hundreds of entries. The other has 10-20 # entries per value. # # Verify that the query planner chooses the first index for the singleton # entries and the second index for the others. # do_test 1.0 { db eval { CREATE TABLE t1(a,b,c,d); CREATE INDEX t1a ON t1(a); CREATE INDEX t1b ON t1(b); CREATE INDEX t1c ON t1(c); } for {set i 0} {$i<1000} {incr i} { if {$i%2==0} {set a $i} {set a [expr {($i%8)*100}]} set b [expr {$i/10}] set c [expr {$i/8}] set c [expr {$c*$c*$c}] db eval {INSERT INTO t1 VALUES($a,$b,$c,$i)} } db eval {ANALYZE} } {} # The a==100 comparison is expensive because there are many rows # with a==100. And so for those cases, choose the t1b index. # # Buf ro a==99 and a==101, there are far fewer rows so choose # the t1a index. # do_test 1.1 { eqp {SELECT * FROM t1 WHERE a=100 AND b=55} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b=?) (~2 rows)}} do_test 1.2 { eqp {SELECT * FROM t1 WHERE a=99 AND b=55} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} do_test 1.3 { eqp {SELECT * FROM t1 WHERE a=101 AND b=55} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} do_test 1.4 { eqp {SELECT * FROM t1 WHERE a=100 AND b=56} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b=?) (~2 rows)}} do_test 1.5 { eqp {SELECT * FROM t1 WHERE a=99 AND b=56} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} do_test 1.6 { eqp {SELECT * FROM t1 WHERE a=101 AND b=56} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~1 rows)}} do_test 2.1 { eqp {SELECT * FROM t1 WHERE a=100 AND b BETWEEN 50 AND 54} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b>? AND b<?) (~2 rows)}} # There are many more values of c between 0 and 100000 than there are # between 800000 and 900000. So t1c is more selective for the latter # range. # do_test 3.1 { eqp {SELECT * FROM t1 WHERE b BETWEEN 50 AND 54 AND c BETWEEN 0 AND 100000} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1b (b>? AND b<?) (~6 rows)}} do_test 3.2 { eqp {SELECT * FROM t1 WHERE b BETWEEN 50 AND 54 AND c BETWEEN 800000 AND 900000} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1c (c>? AND c<?) (~4 rows)}} do_test 3.3 { eqp {SELECT * FROM t1 WHERE a=100 AND c BETWEEN 0 AND 100000} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1a (a=?) (~63 rows)}} do_test 3.4 { eqp {SELECT * FROM t1 WHERE a=100 AND c BETWEEN 800000 AND 900000} } {0 0 0 {SEARCH TABLE t1 USING INDEX t1c (c>? AND c<?) (~2 rows)}} finish_test |
Changes to test/auth.test.
︙ | ︙ | |||
2320 2321 2322 2323 2324 2325 2326 | DROP TABLE v1chng; } } } ifcapable stat2 { set stat2 "sqlite_stat2 " } else { | > > > | > | 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 | DROP TABLE v1chng; } } } ifcapable stat2 { set stat2 "sqlite_stat2 " } else { ifcapable stat3 { set stat2 "sqlite_stat3 " } else { set stat2 "" } } do_test auth-5.2 { execsql { SELECT name FROM ( SELECT * FROM sqlite_master UNION ALL SELECT * FROM sqlite_temp_master) WHERE type='table' ORDER BY name |
︙ | ︙ |
Changes to test/dbstatus.test.
︙ | ︙ | |||
51 52 53 54 55 56 57 58 59 60 61 62 63 64 | proc lookaside {db} { expr { $::lookaside_buffer_size * [lindex [sqlite3_db_status $db SQLITE_DBSTATUS_LOOKASIDE_USED 0] 1] } } #--------------------------------------------------------------------------- # Run the dbstatus-2 and dbstatus-3 tests with several of different # lookaside buffer sizes. # foreach ::lookaside_buffer_size {0 64 120} { | > > > > > > | 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | proc lookaside {db} { expr { $::lookaside_buffer_size * [lindex [sqlite3_db_status $db SQLITE_DBSTATUS_LOOKASIDE_USED 0] 1] } } ifcapable stat3 { set STAT3 1 } else { set STAT3 0 } #--------------------------------------------------------------------------- # Run the dbstatus-2 and dbstatus-3 tests with several of different # lookaside buffer sizes. # foreach ::lookaside_buffer_size {0 64 120} { |
︙ | ︙ | |||
114 115 116 117 118 119 120 | END; } 5 { CREATE TABLE t1(a, b); CREATE TABLE t2(c, d); CREATE VIEW v1 AS SELECT * FROM t1 UNION SELECT * FROM t2; } | | | 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 | END; } 5 { CREATE TABLE t1(a, b); CREATE TABLE t2(c, d); CREATE VIEW v1 AS SELECT * FROM t1 UNION SELECT * FROM t2; } 6y { CREATE TABLE t1(a, b); CREATE INDEX i1 ON t1(a); CREATE INDEX i2 ON t1(a,b); CREATE INDEX i3 ON t1(b,b); INSERT INTO t1 VALUES(randomblob(20), randomblob(25)); INSERT INTO t1 SELECT randomblob(20), randomblob(25) FROM t1; INSERT INTO t1 SELECT randomblob(20), randomblob(25) FROM t1; |
︙ | ︙ | |||
194 195 196 197 198 199 200 | # for any reason is not counted as "schema memory". # # Additionally, in auto-vacuum mode, dropping tables and indexes causes # the page-cache to shrink. So the amount of memory freed is always # much greater than just that reported by DBSTATUS_SCHEMA_USED in this # case. # | > > > | > | 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 | # for any reason is not counted as "schema memory". # # Additionally, in auto-vacuum mode, dropping tables and indexes causes # the page-cache to shrink. So the amount of memory freed is always # much greater than just that reported by DBSTATUS_SCHEMA_USED in this # case. # # Some of the memory used for sqlite_stat3 is unaccounted for by # dbstatus. # if {[string match *x $tn] || $AUTOVACUUM || ([string match *y $tn] && $STAT3)} { do_test dbstatus-2.$tn.ax { expr {($nSchema1-$nSchema2)<=$nFree} } 1 } else { do_test dbstatus-2.$tn.a { expr {$nSchema1-$nSchema2} } $nFree } do_test dbstatus-2.$tn.b { list $nAlloc1 $nSchema1 } "$nAlloc3 $nSchema3" do_test dbstatus-2.$tn.c { list $nAlloc2 $nSchema2 } "$nAlloc4 $nSchema4" |
︙ | ︙ |
Added test/stat3.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | # 2011 August 08 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # # This file implements regression tests for SQLite library. This file # implements tests for the extra functionality provided by the ANALYZE # command when the library is compiled with SQLITE_ENABLE_STAT3 defined. # set testdir [file dirname $argv0] source $testdir/tester.tcl set testprefix stat3 # Verify that if not compiled with SQLITE_ENABLE_STAT2 that the ANALYZE # command will delete the sqlite_stat2 table. Likewise, if not compiled # with SQLITE_ENABLE_STAT3, the sqlite_stat3 table is deleted. # do_test 1.1 { db eval { PRAGMA writable_schema=ON; CREATE TABLE sqlite_stat2(tbl,idx,sampleno,sample); CREATE TABLE sqlite_stat3(tbl,idx,neq,nlt,ndlt,sample); SELECT name FROM sqlite_master ORDER BY 1; } } {sqlite_stat2 sqlite_stat3} do_test 1.2 { db close sqlite3 db test.db db eval {SELECT name FROM sqlite_master ORDER BY 1} } {sqlite_stat2 sqlite_stat3} ifcapable {stat3} { do_test 1.3 { db eval {ANALYZE; SELECT name FROM sqlite_master ORDER BY 1} } {sqlite_stat1 sqlite_stat3} } else { do_test 1.4 { db eval {ANALYZE; SELECT name FROM sqlite_master ORDER BY 1} } {sqlite_stat1} finish_test return } finish_test |
Changes to test/tkt-cbd054fa6b.test.
︙ | ︙ | |||
12 13 14 15 16 17 18 | # This file implements tests to verify that ticket [cbd054fa6b] has been # fixed. # set testdir [file dirname $argv0] source $testdir/tester.tcl | | | 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 | # This file implements tests to verify that ticket [cbd054fa6b] has been # fixed. # set testdir [file dirname $argv0] source $testdir/tester.tcl ifcapable !stat3 { finish_test return } do_test tkt-cbd05-1.1 { db eval { CREATE TABLE t1(a INTEGER PRIMARY KEY, b TEXT UNIQUE NOT NULL); |
︙ | ︙ | |||
42 43 44 45 46 47 48 | db eval { ANALYZE; } } {} do_test tkt-cbd05-1.3 { execsql { SELECT tbl,idx,group_concat(sample,' ') | | | 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | db eval { ANALYZE; } } {} do_test tkt-cbd05-1.3 { execsql { SELECT tbl,idx,group_concat(sample,' ') FROM sqlite_stat3 WHERE idx = 't1_x' GROUP BY tbl,idx } } {t1 t1_x { A B C D E F G H I}} do_test tkt-cbd05-2.1 { db eval { |
︙ | ︙ | |||
74 75 76 77 78 79 80 | db eval { ANALYZE; } } {} do_test tkt-cbd05-2.3 { execsql { SELECT tbl,idx,group_concat(sample,' ') | | | 74 75 76 77 78 79 80 81 82 83 84 85 86 87 | db eval { ANALYZE; } } {} do_test tkt-cbd05-2.3 { execsql { SELECT tbl,idx,group_concat(sample,' ') FROM sqlite_stat3 WHERE idx = 't1_x' GROUP BY tbl,idx } } {t1 t1_x { A B C D E F G H I}} finish_test |
Changes to test/unordered.test.
︙ | ︙ | |||
27 28 29 30 31 32 33 34 35 | INSERT INTO t1 SELECT a+16, b FROM t1; INSERT INTO t1 SELECT a+32, b FROM t1; INSERT INTO t1 SELECT a+64, b FROM t1; ANALYZE; } {} foreach idxmode {ordered unordered} { if {$idxmode == "unordered"} { execsql { UPDATE sqlite_stat1 SET stat = stat || ' unordered' } | > > > | | < | 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 | INSERT INTO t1 SELECT a+16, b FROM t1; INSERT INTO t1 SELECT a+32, b FROM t1; INSERT INTO t1 SELECT a+64, b FROM t1; ANALYZE; } {} foreach idxmode {ordered unordered} { catchsql { DELETE FROM sqlite_stat2 } catchsql { DELETE FROM sqlite_stat3 } if {$idxmode == "unordered"} { execsql { UPDATE sqlite_stat1 SET stat = stat || ' unordered' } } db close sqlite3 db test.db foreach {tn sql r(ordered) r(unordered)} { 1 "SELECT * FROM t1 ORDER BY a" {0 0 0 {SCAN TABLE t1 USING INDEX i1 (~128 rows)}} {0 0 0 {SCAN TABLE t1 (~128 rows)} 0 0 0 {USE TEMP B-TREE FOR ORDER BY}} 2 "SELECT * FROM t1 WHERE a >?" {0 0 0 {SEARCH TABLE t1 USING INDEX i1 (a>?) (~32 rows)}} {0 0 0 {SCAN TABLE t1 (~42 rows)}} |
︙ | ︙ |
Changes to tool/warnings.sh.
1 2 3 4 5 6 7 8 9 10 11 | #/bin/sh # # Run this script in a directory with a working makefile to check for # compiler warnings in SQLite. # rm -f sqlite3.c make sqlite3.c-debug echo '********** No optimizations. Includes FTS4 and RTREE *********' gcc -c -Wshadow -Wall -Wextra -pedantic-errors -Wno-long-long -std=c89 \ -ansi -DHAVE_STDINT_H -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_RTREE \ sqlite3.c | | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | #/bin/sh # # Run this script in a directory with a working makefile to check for # compiler warnings in SQLite. # rm -f sqlite3.c make sqlite3.c-debug echo '********** No optimizations. Includes FTS4 and RTREE *********' gcc -c -Wshadow -Wall -Wextra -pedantic-errors -Wno-long-long -std=c89 \ -ansi -DHAVE_STDINT_H -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_RTREE \ sqlite3.c echo '********** No optimizations. ENABLE_STAT3. THREADSAFE=0 *******' gcc -c -Wshadow -Wall -Wextra -pedantic-errors -Wno-long-long -std=c89 \ -ansi -DSQLITE_ENABLE_STAT3 -DSQLITE_THREADSAFE=0 \ sqlite3.c echo '********** Optimized -O3. Includes FTS4 and RTREE ************' gcc -O3 -c -Wshadow -Wall -Wextra -pedantic-errors -Wno-long-long -std=c89 \ -ansi -DHAVE_STDINT_H -DSQLITE_ENABLE_FTS4 -DSQLITE_ENABLE_RTREE \ sqlite3.c |