Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Changes In Branch fcntl_shm_read Excluding Merge-Ins
This is equivalent to a diff from f569c351 to cbec1bfe
2017-11-04
| ||
18:10 | In cases where a readonly_shm client cannot take the DMS lock on the *-shm file, have it parse the wal file and create a wal-index to access it in heap memory. (check-in: 18b26843 user: dan tags: readonly-wal-recovery) | |
2017-11-03
| ||
19:34 | Fix some comments in new code on this branch. No changes to code. (Closed-Leaf check-in: cbec1bfe user: dan tags: fcntl_shm_read) | |
17:51 | Merge latest trunk changes with this branch. (check-in: 585e0a09 user: dan tags: fcntl_shm_read) | |
17:17 | Allow readonly_shm connections to access the *-shm file using read() even if it is unable to take a DMS lock. (check-in: 9b0d5c4f user: dan tags: fcntl_shm_read) | |
2017-11-02
| ||
18:57 | Fix test cases in wal2.test broken by the locking change in the previous commit. (check-in: f569c351 user: dan tags: readonly-wal-recovery) | |
11:12 | Avoid locking shm-lock WAL_READ_LOCK(0) during recovery. Doing this allows recovery to proceed while a readonly_shm connection in unlocked mode has an ongoing read transaction. (check-in: 5190d84a user: dan tags: readonly-wal-recovery) | |
Changes to Makefile.in.
︙ | ︙ | |||
1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 | sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in $(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c sqlite3_analyzer$(TEXE): sqlite3_analyzer.c $(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS) dbdump$(TEXE): $(TOP)/ext/misc/dbdump.c sqlite3.lo $(LTLINK) -DDBDUMP_STANDALONE -o $@ \ $(TOP)/ext/misc/dbdump.c sqlite3.lo $(TLIBS) showdb$(TEXE): $(TOP)/tool/showdb.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/tool/showdb.c sqlite3.lo $(TLIBS) | > > > > > > > > > > > > > > > > | 1183 1184 1185 1186 1187 1188 1189 1190 1191 1192 1193 1194 1195 1196 1197 1198 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 | sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in $(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c sqlite3_analyzer$(TEXE): sqlite3_analyzer.c $(LTLINK) sqlite3_analyzer.c -o $@ $(LIBTCL) $(TLIBS) CHECKER_DEPS =\ $(TOP)/tool/mkccode.tcl \ sqlite3.c \ $(TOP)/src/tclsqlite.c \ $(TOP)/ext/repair/sqlite3_checker.tcl \ $(TOP)/ext/repair/checkindex.c \ $(TOP)/ext/repair/checkfreelist.c \ $(TOP)/ext/misc/btreeinfo.c \ $(TOP)/ext/repair/sqlite3_checker.c.in sqlite3_checker.c: $(CHECKER_DEPS) $(TCLSH_CMD) $(TOP)/tool/mkccode.tcl $(TOP)/ext/repair/sqlite3_checker.c.in >$@ sqlite3_checker$(TEXE): sqlite3_checker.c $(LTLINK) sqlite3_checker.c -o $@ $(LIBTCL) $(TLIBS) dbdump$(TEXE): $(TOP)/ext/misc/dbdump.c sqlite3.lo $(LTLINK) -DDBDUMP_STANDALONE -o $@ \ $(TOP)/ext/misc/dbdump.c sqlite3.lo $(TLIBS) showdb$(TEXE): $(TOP)/tool/showdb.c sqlite3.lo $(LTLINK) -o $@ $(TOP)/tool/showdb.c sqlite3.lo $(TLIBS) |
︙ | ︙ |
Changes to Makefile.msc.
︙ | ︙ | |||
1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 | # executables needed for testing # TESTPROGS = \ testfixture.exe \ $(SQLITE3EXE) \ sqlite3_analyzer.exe \ sqldiff.exe \ dbhash.exe # Databases containing fuzzer test cases # FUZZDATA = \ $(TOP)\test\fuzzdata1.db \ | > | 1487 1488 1489 1490 1491 1492 1493 1494 1495 1496 1497 1498 1499 1500 1501 | # executables needed for testing # TESTPROGS = \ testfixture.exe \ $(SQLITE3EXE) \ sqlite3_analyzer.exe \ sqlite3_checker.exe \ sqldiff.exe \ dbhash.exe # Databases containing fuzzer test cases # FUZZDATA = \ $(TOP)\test\fuzzdata1.db \ |
︙ | ︙ | |||
2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 | sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(SQLITE_TCL_DEP) $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in > $@ sqlite3_analyzer.exe: sqlite3_analyzer.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_analyzer.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) dbdump.exe: $(TOP)\ext\misc\dbdump.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -DDBDUMP_STANDALONE $(TOP)\ext\misc\dbdump.c $(SQLITE3C) \ /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) testloadext.lo: $(TOP)\src\test_loadext.c $(LTCOMPILE) $(NO_WARN) -c $(TOP)\src\test_loadext.c | > > > > > > > > > > > > > > > > > | 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 | sqlite3_analyzer.c: $(SQLITE3C) $(SQLITE3H) $(TOP)\src\tclsqlite.c $(TOP)\tool\spaceanal.tcl $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in $(SQLITE_TCL_DEP) $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\tool\sqlite3_analyzer.c.in > $@ sqlite3_analyzer.exe: sqlite3_analyzer.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_analyzer.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) CHECKER_DEPS =\ $(TOP)/tool/mkccode.tcl \ sqlite3.c \ $(TOP)/src/tclsqlite.c \ $(TOP)/ext/repair/sqlite3_checker.tcl \ $(TOP)/ext/repair/checkindex.c \ $(TOP)/ext/repair/checkfreelist.c \ $(TOP)/ext/misc/btreeinfo.c \ $(TOP)/ext/repair/sqlite3_checker.c.in sqlite3_checker.c: $(CHECKER_DEPS) $(TCLSH_CMD) $(TOP)\tool\mkccode.tcl $(TOP)\ext\repair\sqlite3_checker.c.in > $@ sqlite3_checker.exe: sqlite3_checker.c $(LIBRESOBJS) $(LTLINK) $(NO_WARN) -DBUILD_sqlite -I$(TCLINCDIR) sqlite3_checker.c \ /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) dbdump.exe: $(TOP)\ext\misc\dbdump.c $(SQLITE3C) $(SQLITE3H) $(LTLINK) $(NO_WARN) -DDBDUMP_STANDALONE $(TOP)\ext\misc\dbdump.c $(SQLITE3C) \ /link $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) $(LIBRESOBJS) $(LTLIBS) testloadext.lo: $(TOP)\src\test_loadext.c $(LTCOMPILE) $(NO_WARN) -c $(TOP)\src\test_loadext.c |
︙ | ︙ |
Changes to ext/misc/btreeinfo.c.
︙ | ︙ | |||
57 58 59 60 61 62 63 | ** ORDER BY nEntry DESC, name; ** ** Show the names of all WITHOUT ROWID tables: ** ** SELECT name FROM sqlite_btreeinfo ** WHERE type='table' AND NOT hasRowid; */ | > | > | 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 | ** ORDER BY nEntry DESC, name; ** ** Show the names of all WITHOUT ROWID tables: ** ** SELECT name FROM sqlite_btreeinfo ** WHERE type='table' AND NOT hasRowid; */ #if !defined(SQLITEINT_H) #include "sqlite3ext.h" #endif SQLITE_EXTENSION_INIT1 #include <string.h> #include <assert.h> /* Columns available in this virtual table */ #define BINFO_COLUMN_TYPE 0 #define BINFO_COLUMN_NAME 1 |
︙ | ︙ |
Changes to ext/misc/spellfix.c.
︙ | ︙ | |||
1118 1119 1120 1121 1122 1123 1124 | */ static int editDist3Install(sqlite3 *db){ int rc; EditDist3Config *pConfig = sqlite3_malloc64( sizeof(*pConfig) ); if( pConfig==0 ) return SQLITE_NOMEM; memset(pConfig, 0, sizeof(*pConfig)); rc = sqlite3_create_function_v2(db, "editdist3", | > | > | | | | 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 1130 1131 1132 1133 1134 1135 1136 1137 1138 1139 1140 1141 1142 | */ static int editDist3Install(sqlite3 *db){ int rc; EditDist3Config *pConfig = sqlite3_malloc64( sizeof(*pConfig) ); if( pConfig==0 ) return SQLITE_NOMEM; memset(pConfig, 0, sizeof(*pConfig)); rc = sqlite3_create_function_v2(db, "editdist3", 2, SQLITE_UTF8|SQLITE_DETERMINISTIC, pConfig, editDist3SqlFunc, 0, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function_v2(db, "editdist3", 3, SQLITE_UTF8|SQLITE_DETERMINISTIC, pConfig, editDist3SqlFunc, 0, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function_v2(db, "editdist3", 1, SQLITE_UTF8|SQLITE_DETERMINISTIC, pConfig, editDist3SqlFunc, 0, 0, editDist3ConfigDelete); }else{ sqlite3_free(pConfig); } return rc; } /* End configurable cost unicode edit distance routines ****************************************************************************** |
︙ | ︙ | |||
2891 2892 2893 2894 2895 2896 2897 | /* ** Register the various functions and the virtual table. */ static int spellfix1Register(sqlite3 *db){ int rc = SQLITE_OK; int i; | | > | | > | > | > | 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 2909 2910 2911 2912 2913 2914 2915 2916 2917 2918 2919 2920 2921 2922 | /* ** Register the various functions and the virtual table. */ static int spellfix1Register(sqlite3 *db){ int rc = SQLITE_OK; int i; rc = sqlite3_create_function(db, "spellfix1_translit", 1, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, transliterateSqlFunc, 0, 0); if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "spellfix1_editdist", 2, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, editdistSqlFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "spellfix1_phonehash", 1, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, phoneticHashSqlFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function(db, "spellfix1_scriptcode", 1, SQLITE_UTF8|SQLITE_DETERMINISTIC, 0, scriptCodeSqlFunc, 0, 0); } if( rc==SQLITE_OK ){ rc = sqlite3_create_module(db, "spellfix1", &spellfix1Module, 0); } if( rc==SQLITE_OK ){ rc = editDist3Install(db); |
︙ | ︙ |
Changes to ext/repair/checkfreelist.c.
︙ | ︙ | |||
162 163 164 165 166 167 168 | rc = sqlite3_prepare_v2(db, zTrunk, -1, &pTrunk, 0); if( rc!=SQLITE_OK ) return rc; sqlite3_bind_text(pTrunk, 1, zDb, -1, SQLITE_STATIC); while( rc==SQLITE_OK && sqlite3_step(pTrunk)==SQLITE_ROW ){ u32 i; u32 iTrunk = (u32)sqlite3_column_int(pTrunk, 0); const u8 *aData = (const u8*)sqlite3_column_blob(pTrunk, 1); | | | 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 | rc = sqlite3_prepare_v2(db, zTrunk, -1, &pTrunk, 0); if( rc!=SQLITE_OK ) return rc; sqlite3_bind_text(pTrunk, 1, zDb, -1, SQLITE_STATIC); while( rc==SQLITE_OK && sqlite3_step(pTrunk)==SQLITE_ROW ){ u32 i; u32 iTrunk = (u32)sqlite3_column_int(pTrunk, 0); const u8 *aData = (const u8*)sqlite3_column_blob(pTrunk, 1); u32 nData = (u32)sqlite3_column_bytes(pTrunk, 1); u32 iNext = get4byte(&aData[0]); u32 nLeaf = get4byte(&aData[4]); if( nLeaf>((nData/4)-2-6) ){ rc = checkFreelistError(pzOut, "leaf count out of range (%d) on trunk page %d", (int)nLeaf, (int)iTrunk |
︙ | ︙ |
Added ext/repair/checkindex.c.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 | /* ** 2017 October 27 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* */ #include "sqlite3ext.h" SQLITE_EXTENSION_INIT1 #ifndef SQLITE_AMALGAMATION # include <string.h> # include <stdio.h> # include <stdlib.h> # include <assert.h> # define ALWAYS(X) 1 # define NEVER(X) 0 typedef unsigned char u8; typedef unsigned short u16; typedef unsigned int u32; #define get4byte(x) ( \ ((u32)((x)[0])<<24) + \ ((u32)((x)[1])<<16) + \ ((u32)((x)[2])<<8) + \ ((u32)((x)[3])) \ ) #endif typedef struct CidxTable CidxTable; typedef struct CidxCursor CidxCursor; struct CidxTable { sqlite3_vtab base; /* Base class. Must be first */ sqlite3 *db; }; struct CidxCursor { sqlite3_vtab_cursor base; /* Base class. Must be first */ sqlite3_int64 iRowid; sqlite3_stmt *pStmt; }; typedef struct CidxColumn CidxColumn; struct CidxColumn { char *zExpr; /* Text for indexed expression */ int bDesc; /* True for DESC columns, otherwise false */ int bKey; /* Part of index, not PK */ }; typedef struct CidxIndex CidxIndex; struct CidxIndex { int nCol; /* Elements in aCol[] array */ CidxColumn aCol[1]; /* Array of indexed columns */ }; static void *cidxMalloc(int *pRc, int n){ void *pRet = 0; assert( n!=0 ); if( *pRc==SQLITE_OK ){ pRet = sqlite3_malloc(n); if( pRet ){ memset(pRet, 0, n); }else{ *pRc = SQLITE_NOMEM; } } return pRet; } static void cidxCursorError(CidxCursor *pCsr, const char *zFmt, ...){ va_list ap; va_start(ap, zFmt); assert( pCsr->base.pVtab->zErrMsg==0 ); pCsr->base.pVtab->zErrMsg = sqlite3_vmprintf(zFmt, ap); va_end(ap); } /* ** Connect to then incremental_index_check virtual table. */ static int cidxConnect( sqlite3 *db, void *pAux, int argc, const char *const*argv, sqlite3_vtab **ppVtab, char **pzErr ){ int rc = SQLITE_OK; CidxTable *pRet; #define IIC_ERRMSG 0 #define IIC_CURRENT_KEY 1 #define IIC_INDEX_NAME 2 #define IIC_AFTER_KEY 3 rc = sqlite3_declare_vtab(db, "CREATE TABLE xyz(" " errmsg TEXT, current_key TEXT," " index_name HIDDEN, after_key HIDDEN" ")" ); pRet = cidxMalloc(&rc, sizeof(CidxTable)); if( pRet ){ pRet->db = db; } *ppVtab = (sqlite3_vtab*)pRet; return rc; } /* ** Disconnect from or destroy an incremental_index_check virtual table. */ static int cidxDisconnect(sqlite3_vtab *pVtab){ CidxTable *pTab = (CidxTable*)pVtab; sqlite3_free(pTab); return SQLITE_OK; } /* ** xBestIndex method. */ static int cidxBestIndex(sqlite3_vtab *tab, sqlite3_index_info *pInfo){ int iIdxName = -1; int iAfterKey = -1; int i; for(i=0; i<pInfo->nConstraint; i++){ struct sqlite3_index_constraint *p = &pInfo->aConstraint[i]; if( p->usable==0 ) continue; if( p->op!=SQLITE_INDEX_CONSTRAINT_EQ ) continue; if( p->iColumn==IIC_INDEX_NAME ){ iIdxName = i; } if( p->iColumn==IIC_AFTER_KEY ){ iAfterKey = i; } } if( iIdxName<0 ){ pInfo->estimatedCost = 1000000000.0; }else{ pInfo->aConstraintUsage[iIdxName].argvIndex = 1; pInfo->aConstraintUsage[iIdxName].omit = 1; if( iAfterKey<0 ){ pInfo->estimatedCost = 1000000.0; }else{ pInfo->aConstraintUsage[iAfterKey].argvIndex = 2; pInfo->aConstraintUsage[iAfterKey].omit = 1; pInfo->estimatedCost = 1000.0; } } return SQLITE_OK; } /* ** Open a new btreeinfo cursor. */ static int cidxOpen(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor){ CidxCursor *pRet; int rc = SQLITE_OK; pRet = cidxMalloc(&rc, sizeof(CidxCursor)); *ppCursor = (sqlite3_vtab_cursor*)pRet; return rc; } /* ** Close a btreeinfo cursor. */ static int cidxClose(sqlite3_vtab_cursor *pCursor){ CidxCursor *pCsr = (CidxCursor*)pCursor; sqlite3_finalize(pCsr->pStmt); pCsr->pStmt = 0; sqlite3_free(pCsr); return SQLITE_OK; } /* ** Move a btreeinfo cursor to the next entry in the file. */ static int cidxNext(sqlite3_vtab_cursor *pCursor){ CidxCursor *pCsr = (CidxCursor*)pCursor; int rc = sqlite3_step(pCsr->pStmt); if( rc!=SQLITE_ROW ){ rc = sqlite3_finalize(pCsr->pStmt); pCsr->pStmt = 0; if( rc!=SQLITE_OK ){ sqlite3 *db = ((CidxTable*)pCsr->base.pVtab)->db; cidxCursorError(pCsr, "Cursor error: %s", sqlite3_errmsg(db)); } }else{ pCsr->iRowid++; rc = SQLITE_OK; } return rc; } /* We have reached EOF if previous sqlite3_step() returned ** anything other than SQLITE_ROW; */ static int cidxEof(sqlite3_vtab_cursor *pCursor){ CidxCursor *pCsr = (CidxCursor*)pCursor; return pCsr->pStmt==0; } static char *cidxMprintf(int *pRc, const char *zFmt, ...){ char *zRet = 0; va_list ap; va_start(ap, zFmt); zRet = sqlite3_vmprintf(zFmt, ap); if( *pRc==SQLITE_OK ){ if( zRet==0 ){ *pRc = SQLITE_NOMEM; } }else{ sqlite3_free(zRet); zRet = 0; } va_end(ap); return zRet; } static sqlite3_stmt *cidxPrepare( int *pRc, CidxCursor *pCsr, const char *zFmt, ... ){ sqlite3_stmt *pRet = 0; char *zSql; va_list ap; /* ... printf arguments */ va_start(ap, zFmt); zSql = sqlite3_vmprintf(zFmt, ap); if( *pRc==SQLITE_OK ){ if( zSql==0 ){ *pRc = SQLITE_NOMEM; }else{ sqlite3 *db = ((CidxTable*)pCsr->base.pVtab)->db; *pRc = sqlite3_prepare_v2(db, zSql, -1, &pRet, 0); if( *pRc!=SQLITE_OK ){ cidxCursorError(pCsr, "SQL error: %s", sqlite3_errmsg(db)); } } } sqlite3_free(zSql); va_end(ap); return pRet; } static void cidxFinalize(int *pRc, sqlite3_stmt *pStmt){ int rc = sqlite3_finalize(pStmt); if( *pRc==SQLITE_OK ) *pRc = rc; } char *cidxStrdup(int *pRc, const char *zStr){ char *zRet = 0; if( *pRc==SQLITE_OK ){ int n = (int)strlen(zStr); zRet = cidxMalloc(pRc, n+1); if( zRet ) memcpy(zRet, zStr, n+1); } return zRet; } static void cidxFreeIndex(CidxIndex *pIdx){ if( pIdx ){ int i; for(i=0; i<pIdx->nCol; i++){ sqlite3_free(pIdx->aCol[i].zExpr); } sqlite3_free(pIdx); } } static int cidx_isspace(char c){ return c==' ' || c=='\t' || c=='\r' || c=='\n'; } static int cidx_isident(char c){ return c<0 || (c>='0' && c<='9') || (c>='a' && c<='z') || (c>='A' && c<='Z') || c=='_'; } #define CIDX_PARSE_EOF 0 #define CIDX_PARSE_COMMA 1 /* "," */ #define CIDX_PARSE_OPEN 2 /* "(" */ #define CIDX_PARSE_CLOSE 3 /* ")" */ static int cidxFindNext( const char *zIn, const char **pzOut, int *pbDoNotTrim /* OUT: True if prev is -- comment */ ){ const char *z = zIn; while( 1 ){ if( z[0]=='-' && z[1]=='-' ){ z += 2; while( z[0]!='\n' ){ if( z[0]=='\0' ) return CIDX_PARSE_EOF; z++; } while( cidx_isspace(*z) ) z++; *pbDoNotTrim = 1; }else{ *pzOut = z; switch( *z ){ case '\0': return CIDX_PARSE_EOF; case '(': return CIDX_PARSE_OPEN; case ')': return CIDX_PARSE_CLOSE; case ',': return CIDX_PARSE_COMMA; case '"': case '\'': case '`': { char q = *z; z++; while( *z ){ if( *z==q ){ z++; if( *z!=q ) break; } z++; } break; } case '[': while( *z++!=']' ); break; case '/': if( z[1]=='*' ){ z += 2; while( z[0]!='*' || z[1]!='/' ){ if( z[1]=='\0' ) return CIDX_PARSE_EOF; z++; } z += 2; break; } default: z++; break; } *pbDoNotTrim = 0; } } assert( 0 ); return -1; } static int cidxParseSQL(CidxCursor *pCsr, CidxIndex *pIdx, const char *zSql){ const char *z = zSql; const char *z1; int e; int rc = SQLITE_OK; int nParen = 1; int bDoNotTrim = 0; CidxColumn *pCol = pIdx->aCol; e = cidxFindNext(z, &z, &bDoNotTrim); if( e!=CIDX_PARSE_OPEN ) goto parse_error; z1 = z+1; z++; while( nParen>0 ){ e = cidxFindNext(z, &z, &bDoNotTrim); if( e==CIDX_PARSE_EOF ) goto parse_error; if( (e==CIDX_PARSE_COMMA || e==CIDX_PARSE_CLOSE) && nParen==1 ){ const char *z2 = z; if( pCol->zExpr ) goto parse_error; if( bDoNotTrim==0 ){ while( cidx_isspace(z[-1]) ) z--; if( !sqlite3_strnicmp(&z[-3], "asc", 3) && 0==cidx_isident(z[-4]) ){ z -= 3; while( cidx_isspace(z[-1]) ) z--; }else if( !sqlite3_strnicmp(&z[-4], "desc", 4) && 0==cidx_isident(z[-5]) ){ z -= 4; while( cidx_isspace(z[-1]) ) z--; } while( cidx_isspace(z1[0]) ) z1++; } pCol->zExpr = cidxMprintf(&rc, "%.*s", z-z1, z1); pCol++; z = z1 = z2+1; } if( e==CIDX_PARSE_OPEN ) nParen++; if( e==CIDX_PARSE_CLOSE ) nParen--; z++; } return rc; parse_error: cidxCursorError(pCsr, "Parse error in: %s", zSql); return SQLITE_ERROR; } static int cidxLookupIndex( CidxCursor *pCsr, /* Cursor object */ const char *zIdx, /* Name of index to look up */ CidxIndex **ppIdx, /* OUT: Description of columns */ char **pzTab /* OUT: Table name */ ){ int rc = SQLITE_OK; char *zTab = 0; CidxIndex *pIdx = 0; sqlite3_stmt *pFindTab = 0; sqlite3_stmt *pInfo = 0; /* Find the table for this index. */ pFindTab = cidxPrepare(&rc, pCsr, "SELECT tbl_name, sql FROM sqlite_master WHERE name=%Q AND type='index'", zIdx ); if( rc==SQLITE_OK && sqlite3_step(pFindTab)==SQLITE_ROW ){ const char *zSql = (const char*)sqlite3_column_text(pFindTab, 1); zTab = cidxStrdup(&rc, (const char*)sqlite3_column_text(pFindTab, 0)); pInfo = cidxPrepare(&rc, pCsr, "PRAGMA index_xinfo(%Q)", zIdx); if( rc==SQLITE_OK ){ int nAlloc = 0; int iCol = 0; while( sqlite3_step(pInfo)==SQLITE_ROW ){ const char *zName = (const char*)sqlite3_column_text(pInfo, 2); const char *zColl = (const char*)sqlite3_column_text(pInfo, 4); CidxColumn *p; if( zName==0 ) zName = "rowid"; if( iCol==nAlloc ){ int nByte = sizeof(CidxIndex) + sizeof(CidxColumn)*(nAlloc+8); pIdx = (CidxIndex*)sqlite3_realloc(pIdx, nByte); nAlloc += 8; } p = &pIdx->aCol[iCol++]; p->bDesc = sqlite3_column_int(pInfo, 3); p->bKey = sqlite3_column_int(pInfo, 5); if( zSql==0 || p->bKey==0 ){ p->zExpr = cidxMprintf(&rc, "\"%w\" COLLATE %s",zName,zColl); }else{ p->zExpr = 0; } pIdx->nCol = iCol; } cidxFinalize(&rc, pInfo); } if( rc==SQLITE_OK && zSql ){ rc = cidxParseSQL(pCsr, pIdx, zSql); } } cidxFinalize(&rc, pFindTab); if( rc==SQLITE_OK && zTab==0 ){ rc = SQLITE_ERROR; } if( rc!=SQLITE_OK ){ sqlite3_free(zTab); cidxFreeIndex(pIdx); }else{ *pzTab = zTab; *ppIdx = pIdx; } return rc; } static int cidxDecodeAfter( CidxCursor *pCsr, int nCol, const char *zAfterKey, char ***pazAfter ){ char **azAfter; int rc = SQLITE_OK; int nAfterKey = (int)strlen(zAfterKey); azAfter = cidxMalloc(&rc, sizeof(char*)*nCol + nAfterKey+1); if( rc==SQLITE_OK ){ int i; char *zCopy = (char*)&azAfter[nCol]; char *p = zCopy; memcpy(zCopy, zAfterKey, nAfterKey+1); for(i=0; i<nCol; i++){ while( *p==' ' ) p++; /* Check NULL values */ if( *p=='N' ){ if( memcmp(p, "NULL", 4) ) goto parse_error; p += 4; } /* Check strings and blob literals */ else if( *p=='X' || *p=='\'' ){ azAfter[i] = p; if( *p=='X' ) p++; if( *p!='\'' ) goto parse_error; p++; while( 1 ){ if( *p=='\0' ) goto parse_error; if( *p=='\'' ){ p++; if( *p!='\'' ) break; } p++; } } /* Check numbers */ else{ azAfter[i] = p; while( (*p>='0' && *p<='9') || *p=='.' || *p=='+' || *p=='-' || *p=='e' || *p=='E' ){ p++; } } while( *p==' ' ) p++; if( *p!=(i==(nCol-1) ? '\0' : ',') ){ goto parse_error; } *p++ = '\0'; } } *pazAfter = azAfter; return rc; parse_error: sqlite3_free(azAfter); *pazAfter = 0; cidxCursorError(pCsr, "%s", "error parsing after value"); return SQLITE_ERROR; } static char *cidxWhere( int *pRc, CidxColumn *aCol, char **azAfter, int iGt, int bLastIsNull ){ char *zRet = 0; const char *zSep = ""; int i; for(i=0; i<iGt; i++){ zRet = cidxMprintf(pRc, "%z%s(%s) IS %s", zRet, zSep, aCol[i].zExpr, (azAfter[i] ? azAfter[i] : "NULL") ); zSep = " AND "; } if( bLastIsNull ){ zRet = cidxMprintf(pRc, "%z%s(%s) IS NULL", zRet, zSep, aCol[iGt].zExpr); } else if( azAfter[iGt] ){ zRet = cidxMprintf(pRc, "%z%s(%s) %s %s", zRet, zSep, aCol[iGt].zExpr, (aCol[iGt].bDesc ? "<" : ">"), azAfter[iGt] ); }else{ zRet = cidxMprintf(pRc, "%z%s(%s) IS NOT NULL", zRet, zSep,aCol[iGt].zExpr); } return zRet; } #define CIDX_CLIST_ALL 0 #define CIDX_CLIST_ORDERBY 1 #define CIDX_CLIST_CURRENT_KEY 2 #define CIDX_CLIST_SUBWHERE 3 #define CIDX_CLIST_SUBEXPR 4 /* ** This function returns various strings based on the contents of the ** CidxIndex structure and the eType parameter. */ static char *cidxColumnList( int *pRc, /* IN/OUT: Error code */ const char *zIdx, CidxIndex *pIdx, /* Indexed columns */ int eType /* True to include ASC/DESC */ ){ char *zRet = 0; if( *pRc==SQLITE_OK ){ const char *aDir[2] = {"", " DESC"}; int i; const char *zSep = ""; for(i=0; i<pIdx->nCol; i++){ CidxColumn *p = &pIdx->aCol[i]; assert( pIdx->aCol[i].bDesc==0 || pIdx->aCol[i].bDesc==1 ); switch( eType ){ case CIDX_CLIST_ORDERBY: zRet = cidxMprintf(pRc, "%z%s%d%s", zRet, zSep, i+1, aDir[p->bDesc]); zSep = ","; break; case CIDX_CLIST_CURRENT_KEY: zRet = cidxMprintf(pRc, "%z%squote(i%d)", zRet, zSep, i); zSep = "||','||"; break; case CIDX_CLIST_SUBWHERE: if( p->bKey==0 ){ zRet = cidxMprintf(pRc, "%z%s%s IS i.i%d", zRet, zSep, p->zExpr, i ); zSep = " AND "; } break; case CIDX_CLIST_SUBEXPR: if( p->bKey==1 ){ zRet = cidxMprintf(pRc, "%z%s%s IS i.i%d", zRet, zSep, p->zExpr, i ); zSep = " AND "; } break; default: assert( eType==CIDX_CLIST_ALL ); zRet = cidxMprintf(pRc, "%z%s(%s) AS i%d", zRet, zSep, p->zExpr, i); zSep = ", "; break; } } } return zRet; } /* ** Position a cursor back to the beginning. */ static int cidxFilter( sqlite3_vtab_cursor *pCursor, int idxNum, const char *idxStr, int argc, sqlite3_value **argv ){ int rc = SQLITE_OK; CidxCursor *pCsr = (CidxCursor*)pCursor; const char *zIdxName = 0; const char *zAfterKey = 0; if( argc>0 ){ zIdxName = (const char*)sqlite3_value_text(argv[0]); if( argc>1 ){ zAfterKey = (const char*)sqlite3_value_text(argv[1]); } } if( zIdxName ){ char *zTab = 0; char *zCurrentKey = 0; char *zOrderBy = 0; char *zSubWhere = 0; char *zSubExpr = 0; char *zSrcList = 0; char **azAfter = 0; CidxIndex *pIdx = 0; rc = cidxLookupIndex(pCsr, zIdxName, &pIdx, &zTab); zOrderBy = cidxColumnList(&rc, zIdxName, pIdx, CIDX_CLIST_ORDERBY); zCurrentKey = cidxColumnList(&rc, zIdxName, pIdx, CIDX_CLIST_CURRENT_KEY); zSubWhere = cidxColumnList(&rc, zIdxName, pIdx, CIDX_CLIST_SUBWHERE); zSubExpr = cidxColumnList(&rc, zIdxName, pIdx, CIDX_CLIST_SUBEXPR); zSrcList = cidxColumnList(&rc, zIdxName, pIdx, CIDX_CLIST_ALL); if( rc==SQLITE_OK && zAfterKey ){ rc = cidxDecodeAfter(pCsr, pIdx->nCol, zAfterKey, &azAfter); } if( rc || zAfterKey==0 ){ pCsr->pStmt = cidxPrepare(&rc, pCsr, "SELECT (SELECT %s FROM %Q AS t WHERE %s), %s " "FROM (SELECT %s FROM %Q ORDER BY %s) AS i", zSubExpr, zTab, zSubWhere, zCurrentKey, zSrcList, zTab, zOrderBy ); /* printf("SQL: %s\n", sqlite3_sql(pCsr->pStmt)); */ }else{ const char *zSep = ""; char *zSql; int i; zSql = cidxMprintf(&rc, "SELECT (SELECT %s FROM %Q WHERE %s), %s FROM (", zSubExpr, zTab, zSubWhere, zCurrentKey ); for(i=pIdx->nCol-1; i>=0; i--){ int j; if( pIdx->aCol[i].bDesc && azAfter[i]==0 ) continue; for(j=0; j<2; j++){ char *zWhere = cidxWhere(&rc, pIdx->aCol, azAfter, i, j); zSql = cidxMprintf(&rc, "%z" "%sSELECT * FROM (SELECT %s FROM %Q WHERE %z ORDER BY %s)", zSql, zSep, zSrcList, zTab, zWhere, zOrderBy ); zSep = " UNION ALL "; if( pIdx->aCol[i].bDesc==0 ) break; } } zSql = cidxMprintf(&rc, "%z) AS i", zSql); /* printf("SQL: %s\n", zSql); */ pCsr->pStmt = cidxPrepare(&rc, pCsr, "%z", zSql); } sqlite3_free(zTab); sqlite3_free(zCurrentKey); sqlite3_free(zOrderBy); sqlite3_free(zSubWhere); sqlite3_free(zSubExpr); sqlite3_free(zSrcList); cidxFreeIndex(pIdx); sqlite3_free(azAfter); } if( pCsr->pStmt ){ assert( rc==SQLITE_OK ); rc = cidxNext(pCursor); } pCsr->iRowid = 1; return rc; } /* ** Return a column value. */ static int cidxColumn( sqlite3_vtab_cursor *pCursor, sqlite3_context *ctx, int iCol ){ CidxCursor *pCsr = (CidxCursor*)pCursor; assert( iCol>=IIC_ERRMSG && iCol<=IIC_AFTER_KEY ); if( iCol==IIC_ERRMSG ){ const char *zVal = 0; if( sqlite3_column_type(pCsr->pStmt, 0)==SQLITE_INTEGER ){ if( sqlite3_column_int(pCsr->pStmt, 0)==0 ){ zVal = "row data mismatch"; } }else{ zVal = "row missing"; } sqlite3_result_text(ctx, zVal, -1, SQLITE_STATIC); }else if( iCol==IIC_CURRENT_KEY ){ sqlite3_result_value(ctx, sqlite3_column_value(pCsr->pStmt, 1)); } return SQLITE_OK; } /* Return the ROWID for the sqlite_btreeinfo table */ static int cidxRowid(sqlite3_vtab_cursor *pCursor, sqlite_int64 *pRowid){ *pRowid = 0; return SQLITE_OK; } /* ** Register the virtual table modules with the database handle passed ** as the only argument. */ static int ciInit(sqlite3 *db){ static sqlite3_module cidx_module = { 0, /* iVersion */ 0, /* xCreate */ cidxConnect, /* xConnect */ cidxBestIndex, /* xBestIndex */ cidxDisconnect, /* xDisconnect */ 0, /* xDestroy */ cidxOpen, /* xOpen - open a cursor */ cidxClose, /* xClose - close a cursor */ cidxFilter, /* xFilter - configure scan constraints */ cidxNext, /* xNext - advance a cursor */ cidxEof, /* xEof - check for end of scan */ cidxColumn, /* xColumn - read data */ cidxRowid, /* xRowid - read data */ 0, /* xUpdate */ 0, /* xBegin */ 0, /* xSync */ 0, /* xCommit */ 0, /* xRollback */ 0, /* xFindMethod */ 0, /* xRename */ 0, /* xSavepoint */ 0, /* xRelease */ 0, /* xRollbackTo */ }; return sqlite3_create_module(db, "incremental_index_check", &cidx_module, 0); } /* ** Extension load function. */ #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_checkindex_init( sqlite3 *db, char **pzErrMsg, const sqlite3_api_routines *pApi ){ SQLITE_EXTENSION_INIT2(pApi); return ciInit(db); } |
Added ext/repair/sqlite3_checker.c.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | /* ** Read an SQLite database file and analyze its space utilization. Generate ** text on standard output. */ #define TCLSH_INIT_PROC sqlite3_checker_init_proc #define SQLITE_ENABLE_DBPAGE_VTAB 1 #define SQLITE_ENABLE_JSON1 1 #undef SQLITE_THREADSAFE #define SQLITE_THREADSAFE 0 #undef SQLITE_ENABLE_COLUMN_METADATA #define SQLITE_OMIT_DECLTYPE 1 #define SQLITE_OMIT_DEPRECATED 1 #define SQLITE_OMIT_PROGRESS_CALLBACK 1 #define SQLITE_OMIT_SHARED_CACHE 1 #define SQLITE_DEFAULT_MEMSTATUS 0 #define SQLITE_MAX_EXPR_DEPTH 0 INCLUDE sqlite3.c INCLUDE $ROOT/src/tclsqlite.c INCLUDE $ROOT/ext/misc/btreeinfo.c INCLUDE $ROOT/ext/repair/checkindex.c INCLUDE $ROOT/ext/repair/checkfreelist.c /* ** Decode a pointer to an sqlite3 object. */ int getDbPointer(Tcl_Interp *interp, const char *zA, sqlite3 **ppDb){ struct SqliteDb *p; Tcl_CmdInfo cmdInfo; if( Tcl_GetCommandInfo(interp, zA, &cmdInfo) ){ p = (struct SqliteDb*)cmdInfo.objClientData; *ppDb = p->db; return TCL_OK; }else{ *ppDb = 0; return TCL_ERROR; } return TCL_OK; } /* ** sqlite3_imposter db main rootpage {CREATE TABLE...} ;# setup an imposter ** sqlite3_imposter db main ;# rm all imposters */ static int sqlite3_imposter( void *clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3 *db; const char *zSchema; int iRoot; const char *zSql; if( objc!=3 && objc!=5 ){ Tcl_WrongNumArgs(interp, 1, objv, "DB SCHEMA [ROOTPAGE SQL]"); return TCL_ERROR; } if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR; zSchema = Tcl_GetString(objv[2]); if( objc==3 ){ sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, db, zSchema, 0, 1); }else{ if( Tcl_GetIntFromObj(interp, objv[3], &iRoot) ) return TCL_ERROR; zSql = Tcl_GetString(objv[4]); sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, db, zSchema, 1, iRoot); sqlite3_exec(db, zSql, 0, 0, 0); sqlite3_test_control(SQLITE_TESTCTRL_IMPOSTER, db, zSchema, 0, 0); } return TCL_OK; } #include <stdio.h> const char *sqlite3_checker_init_proc(Tcl_Interp *interp){ Tcl_CreateObjCommand(interp, "sqlite3_imposter", (Tcl_ObjCmdProc*)sqlite3_imposter, 0, 0); sqlite3_auto_extension((void(*)(void))sqlite3_btreeinfo_init); sqlite3_auto_extension((void(*)(void))sqlite3_checkindex_init); sqlite3_auto_extension((void(*)(void))sqlite3_checkfreelist_init); return BEGIN_STRING INCLUDE $ROOT/ext/repair/sqlite3_checker.tcl END_STRING ; } |
Added ext/repair/sqlite3_checker.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 | # This TCL script is the main driver script for the sqlite3_checker utility # program. # # Special case: # # sqlite3_checker --test FILENAME ARGS # # uses FILENAME in place of this script. # if {[lindex $argv 0]=="--test" && [llength $argv]>1} { set ::argv0 [lindex $argv 1] set argv [lrange $argv 2 end] source $argv0 exit 0 } # Emulate a TCL shell # proc tclsh {} { set line {} while {![eof stdin]} { if {$line!=""} { puts -nonewline "> " } else { puts -nonewline "% " } flush stdout append line [gets stdin] if {[info complete $line]} { if {[catch {uplevel #0 $line} result]} { puts stderr "Error: $result" } elseif {$result!=""} { puts $result } set line {} } else { append line \n } } } # Do an incremental integrity check of a single index # proc check_index {idxname batchsize} { set i 0 set more 1 set nerr 0 set pct 00.0 set max [db one {SELECT nEntry FROM sqlite_btreeinfo('main') WHERE name=$idxname}] puts -nonewline "$idxname: $i of $max rows ($pct%)\r" flush stdout while {$more} { set more 0 db eval {SELECT errmsg, current_key AS key FROM incremental_index_check($idxname) WHERE after_key=$key LIMIT $batchsize} { set more 1 if {$errmsg!=""} { incr nerr puts "$idxname: key($key): $errmsg" } incr i } set x [format {%.1f} [expr {($i*100.0)/$max}]] if {$x!=$pct} { puts -nonewline "$idxname: $i of $max rows ($pct%)\r" flush stdout set pct $x } } puts "$idxname: $nerr errors out of $i entries" } # Print a usage message on standard error, then quit. # proc usage {} { set argv0 [file rootname [file tail [info nameofexecutable]]] puts stderr "Usage: $argv0 OPTIONS database-filename" puts stderr { Do sanity checking on a live SQLite3 database file specified by the "database-filename" argument. Options: --batchsize N Number of rows to check per transaction --freelist Perform a freelist check --index NAME Run a check of the index NAME --summary Print summary information about the database --table NAME Run a check of all indexes for table NAME --tclsh Run the built-in TCL interpreter (for debugging) --version Show the version number of SQLite } exit 1 } set file_to_analyze {} append argv {} set bFreelistCheck 0 set bSummary 0 set zIndex {} set zTable {} set batchsize 1000 set bAll 1 set argc [llength $argv] for {set i 0} {$i<$argc} {incr i} { set arg [lindex $argv $i] if {[regexp {^-+tclsh$} $arg]} { tclsh exit 0 } if {[regexp {^-+version$} $arg]} { sqlite3 mem :memory: puts [mem one {SELECT sqlite_version()||' '||sqlite_source_id()}] mem close exit 0 } if {[regexp {^-+freelist$} $arg]} { set bFreelistCheck 1 set bAll 0 continue } if {[regexp {^-+summary$} $arg]} { set bSummary 1 set bAll 0 continue } if {[regexp {^-+batchsize$} $arg]} { incr i if {$i>=$argc} { puts stderr "missing argument on $arg" exit 1 } set batchsize [lindex $argv $i] continue } if {[regexp {^-+index$} $arg]} { incr i if {$i>=$argc} { puts stderr "missing argument on $arg" exit 1 } set zIndex [lindex $argv $i] set bAll 0 continue } if {[regexp {^-+table$} $arg]} { incr i if {$i>=$argc} { puts stderr "missing argument on $arg" exit 1 } set zTable [lindex $argv $i] set bAll 0 continue } if {[regexp {^-} $arg]} { puts stderr "Unknown option: $arg" usage } if {$file_to_analyze!=""} { usage } else { set file_to_analyze $arg } } if {$file_to_analyze==""} usage # If a TCL script is specified on the command-line, then run that # script. # if {[file extension $file_to_analyze]==".tcl"} { source $file_to_analyze exit 0 } set root_filename $file_to_analyze regexp {^file:(//)?([^?]*)} $file_to_analyze all x1 root_filename if {![file exists $root_filename]} { puts stderr "No such file: $root_filename" exit 1 } if {![file readable $root_filename]} { puts stderr "File is not readable: $root_filename" exit 1 } if {[catch {sqlite3 db $file_to_analyze} res]} { puts stderr "Cannot open datababase $root_filename: $res" exit 1 } if {$bFreelistCheck || $bAll} { puts -nonewline "freelist-check: " flush stdout puts [db one {SELECT checkfreelist('main')}] } if {$bSummary} { set scale 0 set pgsz [db one {PRAGMA page_size}] db eval {SELECT nPage*$pgsz AS sz, name, tbl_name FROM sqlite_btreeinfo WHERE type='index' ORDER BY 1 DESC, name} { if {$scale==0} { if {$sz>10000000} { set scale 1000000.0 set unit MB } else { set scale 1000.0 set unit KB } } puts [format {%7.1f %s index %s of table %s} \ [expr {$sz/$scale}] $unit $name $tbl_name] } } if {$zIndex!=""} { check_index $zIndex $batchsize } if {$zTable!=""} { foreach idx [db eval {SELECT name FROM sqlite_master WHERE type='index' AND rootpage>0 AND tbl_name=$zTable}] { check_index $idx $batchsize } } if {$bAll} { set allidx [db eval {SELECT name FROM sqlite_btreeinfo('main') WHERE type='index' AND rootpage>0 ORDER BY nEntry}] foreach idx $allidx { check_index $idx $batchsize } } |
Added ext/repair/test/README.md.
> > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 | To run these tests, first build sqlite3_checker: > make sqlite3_checker Then run the "test.tcl" script using: > ./sqlite3_checker --test $path/test.tcl Optionally add the full pathnames of individual *.test modules |
Added ext/repair/test/checkfreelist01.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 | # 2017-10-11 set testprefix checkfreelist do_execsql_test 1.0 { PRAGMA page_size=1024; CREATE TABLE t1(a, b); } do_execsql_test 1.2 { SELECT checkfreelist('main') } {ok} do_execsql_test 1.3 { WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<10000 ) INSERT INTO t1 SELECT randomblob(400), randomblob(400) FROM s; DELETE FROM t1 WHERE rowid%3; PRAGMA freelist_count; } {6726} do_execsql_test 1.4 { SELECT checkfreelist('main') } {ok} do_execsql_test 1.5 { WITH freelist_trunk(i, d, n) AS ( SELECT 1, NULL, sqlite_readint32(data, 32) FROM sqlite_dbpage WHERE pgno=1 UNION ALL SELECT n, data, sqlite_readint32(data) FROM freelist_trunk, sqlite_dbpage WHERE pgno=n ) SELECT i FROM freelist_trunk WHERE i!=1; } { 10009 9715 9343 8969 8595 8222 7847 7474 7102 6727 6354 5982 5608 5234 4860 4487 4112 3740 3367 2992 2619 2247 1872 1499 1125 752 377 5 } do_execsql_test 1.6 { SELECT checkfreelist('main') } {ok} proc set_int {blob idx newval} { binary scan $blob I* ints lset ints $idx $newval binary format I* $ints } db func set_int set_int proc get_int {blob idx} { binary scan $blob I* ints lindex $ints $idx } db func get_int get_int do_execsql_test 1.7 { BEGIN; UPDATE sqlite_dbpage SET data = set_int(data, 1, get_int(data, 1)-1) WHERE pgno=4860; SELECT checkfreelist('main'); ROLLBACK; } {{free-list count mismatch: actual=6725 header=6726}} do_execsql_test 1.8 { BEGIN; UPDATE sqlite_dbpage SET data = set_int(data, 5, (SELECT * FROM pragma_page_count)+1) WHERE pgno=4860; SELECT checkfreelist('main'); ROLLBACK; } {{leaf page 10092 is out of range (child 3 of trunk page 4860)}} do_execsql_test 1.9 { BEGIN; UPDATE sqlite_dbpage SET data = set_int(data, 5, 0) WHERE pgno=4860; SELECT checkfreelist('main'); ROLLBACK; } {{leaf page 0 is out of range (child 3 of trunk page 4860)}} do_execsql_test 1.10 { BEGIN; UPDATE sqlite_dbpage SET data = set_int(data, get_int(data, 1)+1, 0) WHERE pgno=5; SELECT checkfreelist('main'); ROLLBACK; } {{leaf page 0 is out of range (child 247 of trunk page 5)}} do_execsql_test 1.11 { BEGIN; UPDATE sqlite_dbpage SET data = set_int(data, 1, 249) WHERE pgno=5; SELECT checkfreelist('main'); ROLLBACK; } {{leaf count out of range (249) on trunk page 5}} |
Added ext/repair/test/checkindex01.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 | # 2017-10-11 # set testprefix checkindex do_execsql_test 1.0 { CREATE TABLE t1(a, b); CREATE INDEX i1 ON t1(a); INSERT INTO t1 VALUES('one', 2); INSERT INTO t1 VALUES('two', 4); INSERT INTO t1 VALUES('three', 6); INSERT INTO t1 VALUES('four', 8); INSERT INTO t1 VALUES('five', 10); CREATE INDEX i2 ON t1(a DESC); } {} proc incr_index_check {idx nStep} { set Q { SELECT errmsg, current_key FROM incremental_index_check($idx, $after) LIMIT $nStep } set res [list] while {1} { unset -nocomplain current_key set res1 [db eval $Q] if {[llength $res1]==0} break set res [concat $res $res1] set after [lindex $res end] } return $res } proc do_index_check_test {tn idx res} { uplevel [list do_execsql_test $tn.1 " SELECT errmsg, current_key FROM incremental_index_check('$idx'); " $res] uplevel [list do_test $tn.2 "incr_index_check $idx 1" [list {*}$res]] uplevel [list do_test $tn.3 "incr_index_check $idx 2" [list {*}$res]] uplevel [list do_test $tn.4 "incr_index_check $idx 5" [list {*}$res]] } do_execsql_test 1.2 { SELECT errmsg IS NULL, current_key FROM incremental_index_check('i1'); } { 1 'five',5 1 'four',4 1 'one',1 1 'three',3 1 'two',2 } do_index_check_test 1.3 i1 { {} 'five',5 {} 'four',4 {} 'one',1 {} 'three',3 {} 'two',2 } do_index_check_test 1.4 i2 { {} 'two',2 {} 'three',3 {} 'one',1 {} 'four',4 {} 'five',5 } do_test 1.5 { set tblroot [db one { SELECT rootpage FROM sqlite_master WHERE name='t1' }] sqlite3_imposter db main $tblroot {CREATE TABLE xt1(a,b)} db eval { UPDATE xt1 SET a='six' WHERE rowid=3; DELETE FROM xt1 WHERE rowid = 5; } sqlite3_imposter db main } {} do_index_check_test 1.6 i1 { {row missing} 'five',5 {} 'four',4 {} 'one',1 {row data mismatch} 'three',3 {} 'two',2 } do_index_check_test 1.7 i2 { {} 'two',2 {row data mismatch} 'three',3 {} 'one',1 {} 'four',4 {row missing} 'five',5 } #-------------------------------------------------------------------------- do_execsql_test 2.0 { CREATE TABLE t2(a INTEGER PRIMARY KEY, b, c, d); INSERT INTO t2 VALUES(1, NULL, 1, 1); INSERT INTO t2 VALUES(2, 1, NULL, 1); INSERT INTO t2 VALUES(3, 1, 1, NULL); INSERT INTO t2 VALUES(4, 2, 2, 1); INSERT INTO t2 VALUES(5, 2, 2, 2); INSERT INTO t2 VALUES(6, 2, 2, 3); INSERT INTO t2 VALUES(7, 2, 2, 1); INSERT INTO t2 VALUES(8, 2, 2, 2); INSERT INTO t2 VALUES(9, 2, 2, 3); CREATE INDEX i3 ON t2(b, c, d); CREATE INDEX i4 ON t2(b DESC, c DESC, d DESC); CREATE INDEX i5 ON t2(d, c DESC, b); } {} do_index_check_test 2.1 i3 { {} NULL,1,1,1 {} 1,NULL,1,2 {} 1,1,NULL,3 {} 2,2,1,4 {} 2,2,1,7 {} 2,2,2,5 {} 2,2,2,8 {} 2,2,3,6 {} 2,2,3,9 } do_index_check_test 2.2 i4 { {} 2,2,3,6 {} 2,2,3,9 {} 2,2,2,5 {} 2,2,2,8 {} 2,2,1,4 {} 2,2,1,7 {} 1,1,NULL,3 {} 1,NULL,1,2 {} NULL,1,1,1 } do_index_check_test 2.3 i5 { {} NULL,1,1,3 {} 1,2,2,4 {} 1,2,2,7 {} 1,1,NULL,1 {} 1,NULL,1,2 {} 2,2,2,5 {} 2,2,2,8 {} 3,2,2,6 {} 3,2,2,9 } #-------------------------------------------------------------------------- do_execsql_test 3.0 { CREATE TABLE t3(w, x, y, z PRIMARY KEY) WITHOUT ROWID; CREATE INDEX t3wxy ON t3(w, x, y); CREATE INDEX t3wxy2 ON t3(w DESC, x DESC, y DESC); INSERT INTO t3 VALUES(NULL, NULL, NULL, 1); INSERT INTO t3 VALUES(NULL, NULL, NULL, 2); INSERT INTO t3 VALUES(NULL, NULL, NULL, 3); INSERT INTO t3 VALUES('a', NULL, NULL, 4); INSERT INTO t3 VALUES('a', NULL, NULL, 5); INSERT INTO t3 VALUES('a', NULL, NULL, 6); INSERT INTO t3 VALUES('a', 'b', NULL, 7); INSERT INTO t3 VALUES('a', 'b', NULL, 8); INSERT INTO t3 VALUES('a', 'b', NULL, 9); } {} do_index_check_test 3.1 t3wxy { {} NULL,NULL,NULL,1 {} NULL,NULL,NULL,2 {} NULL,NULL,NULL,3 {} 'a',NULL,NULL,4 {} 'a',NULL,NULL,5 {} 'a',NULL,NULL,6 {} 'a','b',NULL,7 {} 'a','b',NULL,8 {} 'a','b',NULL,9 } do_index_check_test 3.2 t3wxy2 { {} 'a','b',NULL,7 {} 'a','b',NULL,8 {} 'a','b',NULL,9 {} 'a',NULL,NULL,4 {} 'a',NULL,NULL,5 {} 'a',NULL,NULL,6 {} NULL,NULL,NULL,1 {} NULL,NULL,NULL,2 {} NULL,NULL,NULL,3 } #-------------------------------------------------------------------------- # Test with an index that uses non-default collation sequences. # do_execsql_test 4.0 { CREATE TABLE t4(a INTEGER PRIMARY KEY, c1 TEXT, c2 TEXT); INSERT INTO t4 VALUES(1, 'aaa', 'bbb'); INSERT INTO t4 VALUES(2, 'AAA', 'CCC'); INSERT INTO t4 VALUES(3, 'aab', 'ddd'); INSERT INTO t4 VALUES(4, 'AAB', 'EEE'); CREATE INDEX t4cc ON t4(c1 COLLATE nocase, c2 COLLATE nocase); } do_index_check_test 4.1 t4cc { {} 'aaa','bbb',1 {} 'AAA','CCC',2 {} 'aab','ddd',3 {} 'AAB','EEE',4 } do_test 4.2 { set tblroot [db one { SELECT rootpage FROM sqlite_master WHERE name='t4' }] sqlite3_imposter db main $tblroot \ {CREATE TABLE xt4(a INTEGER PRIMARY KEY, c1 TEXT, c2 TEXT)} db eval { UPDATE xt4 SET c1='hello' WHERE rowid=2; DELETE FROM xt4 WHERE rowid = 3; } sqlite3_imposter db main } {} do_index_check_test 4.3 t4cc { {} 'aaa','bbb',1 {row data mismatch} 'AAA','CCC',2 {row missing} 'aab','ddd',3 {} 'AAB','EEE',4 } #-------------------------------------------------------------------------- # Test an index on an expression. # do_execsql_test 5.0 { CREATE TABLE t5(x INTEGER PRIMARY KEY, y TEXT, UNIQUE(y)); INSERT INTO t5 VALUES(1, '{"x":1, "y":1}'); INSERT INTO t5 VALUES(2, '{"x":2, "y":2}'); INSERT INTO t5 VALUES(3, '{"x":3, "y":3}'); INSERT INTO t5 VALUES(4, '{"w":4, "z":4}'); INSERT INTO t5 VALUES(5, '{"x":5, "y":5}'); CREATE INDEX t5x ON t5( json_extract(y, '$.x') ); CREATE INDEX t5y ON t5( json_extract(y, '$.y') DESC ); } do_index_check_test 5.1.1 t5x { {} NULL,4 {} 1,1 {} 2,2 {} 3,3 {} 5,5 } do_index_check_test 5.1.2 t5y { {} 5,5 {} 3,3 {} 2,2 {} 1,1 {} NULL,4 } do_index_check_test 5.1.3 sqlite_autoindex_t5_1 { {} {'{"w":4, "z":4}',4} {} {'{"x":1, "y":1}',1} {} {'{"x":2, "y":2}',2} {} {'{"x":3, "y":3}',3} {} {'{"x":5, "y":5}',5} } do_test 5.2 { set tblroot [db one { SELECT rootpage FROM sqlite_master WHERE name='t5' }] sqlite3_imposter db main $tblroot \ {CREATE TABLE xt5(a INTEGER PRIMARY KEY, c1 TEXT);} db eval { UPDATE xt5 SET c1='{"x":22, "y":11}' WHERE rowid=1; DELETE FROM xt5 WHERE rowid = 4; } sqlite3_imposter db main } {} do_index_check_test 5.3.1 t5x { {row missing} NULL,4 {row data mismatch} 1,1 {} 2,2 {} 3,3 {} 5,5 } do_index_check_test 5.3.2 sqlite_autoindex_t5_1 { {row missing} {'{"w":4, "z":4}',4} {row data mismatch} {'{"x":1, "y":1}',1} {} {'{"x":2, "y":2}',2} {} {'{"x":3, "y":3}',3} {} {'{"x":5, "y":5}',5} } #------------------------------------------------------------------------- # do_execsql_test 6.0 { CREATE TABLE t6(x INTEGER PRIMARY KEY, y, z); CREATE INDEX t6x1 ON t6(y, /* one,two,three */ z); CREATE INDEX t6x2 ON t6(z, -- hello,world, y); CREATE INDEX t6x3 ON t6(z -- hello,world , y); INSERT INTO t6 VALUES(1, 2, 3); INSERT INTO t6 VALUES(4, 5, 6); } do_index_check_test 6.1 t6x1 { {} 2,3,1 {} 5,6,4 } do_index_check_test 6.2 t6x2 { {} 3,2,1 {} 6,5,4 } do_index_check_test 6.2 t6x3 { {} 3,2,1 {} 6,5,4 } |
Added ext/repair/test/test.tcl.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 | # Run this script using # # sqlite3_checker --test $thisscript $testscripts # # The $testscripts argument is optional. If omitted, all *.test files # in the same directory as $thisscript are run. # set NTEST 0 set NERR 0 # Invoke the do_test procedure to run a single test # # The $expected parameter is the expected result. The result is the return # value from the last TCL command in $cmd. # # Normally, $expected must match exactly. But if $expected is of the form # "/regexp/" then regular expression matching is used. If $expected is # "~/regexp/" then the regular expression must NOT match. If $expected is # of the form "#/value-list/" then each term in value-list must be numeric # and must approximately match the corresponding numeric term in $result. # Values must match within 10%. Or if the $expected term is A..B then the # $result term must be in between A and B. # proc do_test {name cmd expected} { if {[info exists ::testprefix]} { set name "$::testprefix$name" } incr ::NTEST puts -nonewline $name... flush stdout if {[catch {uplevel #0 "$cmd;\n"} result]} { puts -nonewline $name... puts "\nError: $result" incr ::NERR } else { set ok [expr {[string compare $result $expected]==0}] if {!$ok} { puts "\n! $name expected: \[$expected\]\n! $name got: \[$result\]" incr ::NERR } else { puts " Ok" } } flush stdout } # # do_execsql_test TESTNAME SQL RES # proc do_execsql_test {testname sql {result {}}} { uplevel [list do_test $testname [list db eval $sql] [list {*}$result]] } if {[llength $argv]==0} { set dir [file dirname $argv0] set argv [glob -nocomplain $dir/*.test] } foreach testfile $argv { file delete -force test.db sqlite3 db test.db source $testfile catch {db close} } puts "$NERR errors out of $NTEST tests" |
Changes to main.mk.
︙ | ︙ | |||
460 461 462 463 464 465 466 467 468 469 470 471 472 473 | # executables needed for testing # TESTPROGS = \ testfixture$(EXE) \ sqlite3$(EXE) \ sqlite3_analyzer$(EXE) \ sqldiff$(EXE) \ dbhash$(EXE) # Databases containing fuzzer test cases # FUZZDATA = \ $(TOP)/test/fuzzdata1.db \ | > | 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 | # executables needed for testing # TESTPROGS = \ testfixture$(EXE) \ sqlite3$(EXE) \ sqlite3_analyzer$(EXE) \ sqlite3_checker$(EXE) \ sqldiff$(EXE) \ dbhash$(EXE) # Databases containing fuzzer test cases # FUZZDATA = \ $(TOP)/test/fuzzdata1.db \ |
︙ | ︙ | |||
782 783 784 785 786 787 788 789 790 791 792 793 794 795 | sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/sqlite3_analyzer.c.in $(TOP)/tool/mkccode.tcl tclsh $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c sqlite3_analyzer$(EXE): sqlite3_analyzer.c $(TCCX) $(TCL_FLAGS) sqlite3_analyzer.c -o $@ $(LIBTCL) $(THREADLIB) dbdump$(EXE): $(TOP)/ext/misc/dbdump.c sqlite3.o $(TCCX) -DDBDUMP_STANDALONE -o dbdump$(EXE) \ $(TOP)/ext/misc/dbdump.c sqlite3.o $(THREADLIB) # Rules to build the 'testfixture' application. # TESTFIXTURE_FLAGS = -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 | > > > > > > > > > > > > > > > > | 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 | sqlite3_analyzer.c: sqlite3.c $(TOP)/src/tclsqlite.c $(TOP)/tool/spaceanal.tcl $(TOP)/tool/sqlite3_analyzer.c.in $(TOP)/tool/mkccode.tcl tclsh $(TOP)/tool/mkccode.tcl $(TOP)/tool/sqlite3_analyzer.c.in >sqlite3_analyzer.c sqlite3_analyzer$(EXE): sqlite3_analyzer.c $(TCCX) $(TCL_FLAGS) sqlite3_analyzer.c -o $@ $(LIBTCL) $(THREADLIB) CHECKER_DEPS =\ $(TOP)/tool/mkccode.tcl \ sqlite3.c \ $(TOP)/src/tclsqlite.c \ $(TOP)/ext/repair/sqlite3_checker.tcl \ $(TOP)/ext/repair/checkindex.c \ $(TOP)/ext/repair/checkfreelist.c \ $(TOP)/ext/misc/btreeinfo.c \ $(TOP)/ext/repair/sqlite3_checker.c.in sqlite3_checker.c: $(CHECKER_DEPS) tclsh $(TOP)/tool/mkccode.tcl $(TOP)/ext/repair/sqlite3_checker.c.in >$@ sqlite3_checker$(TEXE): sqlite3_checker.c $(TCCX) $(TCL_FLAGS) sqlite3_checker.c -o $@ $(LIBTCL) $(THREADLIB) dbdump$(EXE): $(TOP)/ext/misc/dbdump.c sqlite3.o $(TCCX) -DDBDUMP_STANDALONE -o dbdump$(EXE) \ $(TOP)/ext/misc/dbdump.c sqlite3.o $(THREADLIB) # Rules to build the 'testfixture' application. # TESTFIXTURE_FLAGS = -DSQLITE_TEST=1 -DSQLITE_CRASH_TEST=1 |
︙ | ︙ |
Changes to src/dbpage.c.
︙ | ︙ | |||
325 326 327 328 329 330 331 | zSchema = (const char*)sqlite3_value_text(argv[4]); iDb = zSchema ? sqlite3FindDbName(pTab->db, zSchema) : -1; if( iDb<0 ){ zErr = "no such schema"; goto update_fail; } pBt = pTab->db->aDb[iDb].pBt; | | | 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 | zSchema = (const char*)sqlite3_value_text(argv[4]); iDb = zSchema ? sqlite3FindDbName(pTab->db, zSchema) : -1; if( iDb<0 ){ zErr = "no such schema"; goto update_fail; } pBt = pTab->db->aDb[iDb].pBt; if( pgno<1 || pBt==0 || pgno>(int)sqlite3BtreeLastPage(pBt) ){ zErr = "bad page number"; goto update_fail; } szPage = sqlite3BtreeGetPageSize(pBt); if( sqlite3_value_type(argv[3])!=SQLITE_BLOB || sqlite3_value_bytes(argv[3])!=szPage ){ |
︙ | ︙ |
Changes to src/os_unix.c.
︙ | ︙ | |||
3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 | }else{ pFile->ctrlFlags |= mask; } } /* Forward declaration */ static int unixGetTempname(int nBuf, char *zBuf); /* ** Information and control of an open file handle. */ static int unixFileControl(sqlite3_file *id, int op, void *pArg){ unixFile *pFile = (unixFile*)id; switch( op ){ | > | 3791 3792 3793 3794 3795 3796 3797 3798 3799 3800 3801 3802 3803 3804 3805 | }else{ pFile->ctrlFlags |= mask; } } /* Forward declaration */ static int unixGetTempname(int nBuf, char *zBuf); static int fcntlReadShm(unixFile*,void*); /* ** Information and control of an open file handle. */ static int unixFileControl(sqlite3_file *id, int op, void *pArg){ unixFile *pFile = (unixFile*)id; switch( op ){ |
︙ | ︙ | |||
3812 3813 3814 3815 3816 3817 3818 3819 3820 3821 3822 3823 3824 3825 | return rc ? SQLITE_IOERR_COMMIT_ATOMIC : SQLITE_OK; } case SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE: { int rc = osIoctl(pFile->h, F2FS_IOC_ABORT_VOLATILE_WRITE); return rc ? SQLITE_IOERR_ROLLBACK_ATOMIC : SQLITE_OK; } #endif /* __linux__ && SQLITE_ENABLE_BATCH_ATOMIC_WRITE */ case SQLITE_FCNTL_LOCKSTATE: { *(int*)pArg = pFile->eFileLock; return SQLITE_OK; } case SQLITE_FCNTL_LAST_ERRNO: { *(int*)pArg = pFile->lastErrno; | > > > > | 3813 3814 3815 3816 3817 3818 3819 3820 3821 3822 3823 3824 3825 3826 3827 3828 3829 3830 | return rc ? SQLITE_IOERR_COMMIT_ATOMIC : SQLITE_OK; } case SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE: { int rc = osIoctl(pFile->h, F2FS_IOC_ABORT_VOLATILE_WRITE); return rc ? SQLITE_IOERR_ROLLBACK_ATOMIC : SQLITE_OK; } #endif /* __linux__ && SQLITE_ENABLE_BATCH_ATOMIC_WRITE */ case SQLITE_FCNTL_READ_SHM: { return fcntlReadShm(pFile, pArg); } case SQLITE_FCNTL_LOCKSTATE: { *(int*)pArg = pFile->eFileLock; return SQLITE_OK; } case SQLITE_FCNTL_LAST_ERRNO: { *(int*)pArg = pFile->lastErrno; |
︙ | ︙ | |||
4142 4143 4144 4145 4146 4147 4148 4149 4150 4151 4152 4153 4154 4155 | }; /* ** Constants used for locking */ #define UNIX_SHM_BASE ((22+SQLITE_SHM_NLOCK)*4) /* first lock byte */ #define UNIX_SHM_DMS (UNIX_SHM_BASE+SQLITE_SHM_NLOCK) /* deadman switch */ /* ** Apply posix advisory locks for all bytes from ofst through ofst+n-1. ** ** Locks block if the mask is exactly UNIX_SHM_C and are non-blocking ** otherwise. */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 4147 4148 4149 4150 4151 4152 4153 4154 4155 4156 4157 4158 4159 4160 4161 4162 4163 4164 4165 4166 4167 4168 4169 4170 4171 4172 4173 4174 4175 4176 4177 4178 4179 4180 4181 4182 4183 4184 4185 4186 4187 4188 4189 4190 4191 4192 4193 4194 4195 4196 4197 4198 4199 4200 4201 4202 4203 4204 4205 | }; /* ** Constants used for locking */ #define UNIX_SHM_BASE ((22+SQLITE_SHM_NLOCK)*4) /* first lock byte */ #define UNIX_SHM_DMS (UNIX_SHM_BASE+SQLITE_SHM_NLOCK) /* deadman switch */ /* ** Implementation of file-control SQLITE_FCNTL_READ_SHM. */ static int fcntlReadShm(unixFile *pFile, void *pArg){ struct ReadShmParam { void *pBuf; int nBuf; } *pParam = (struct ReadShmParam*)pArg; unixShmNode *pShmNode = pFile->pShm->pShmNode; int rc = SQLITE_OK; sqlite3_mutex_enter( pShmNode->mutex ); if( pShmNode->isUnlocked==1 ){ struct stat buf; /* Used to hold return values of fstat() */ if( osFstat(pShmNode->h, &buf) ){ rc = SQLITE_IOERR_FSTAT; }else if( 0!=lseek(pShmNode->h, 0, SEEK_SET) ){ rc = SQLITE_IOERR_READ; }else{ int nRead = buf.st_size; if( nRead==0 ){ pParam->nBuf = 0; pParam->pBuf = 0; }else{ pParam->pBuf = sqlite3_malloc(nRead); if( pParam->pBuf ){ int res = osRead(pShmNode->h, pParam->pBuf, nRead); if( res!=nRead ){ sqlite3_free(pParam->pBuf); pParam->pBuf = 0; rc = SQLITE_IOERR_READ; }else{ pParam->nBuf = nRead; } }else{ rc = SQLITE_NOMEM_BKPT; } } } }else{ rc = SQLITE_BUSY; } sqlite3_mutex_leave( pShmNode->mutex ); return rc; } /* ** Apply posix advisory locks for all bytes from ofst through ofst+n-1. ** ** Locks block if the mask is exactly UNIX_SHM_C and are non-blocking ** otherwise. */ |
︙ | ︙ |
Changes to src/sqlite.h.in.
︙ | ︙ | |||
504 505 506 507 508 509 510 | #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) #define SQLITE_BUSY_SNAPSHOT (SQLITE_BUSY | (2<<8)) #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) #define SQLITE_CANTOPEN_ISDIR (SQLITE_CANTOPEN | (2<<8)) #define SQLITE_CANTOPEN_FULLPATH (SQLITE_CANTOPEN | (3<<8)) #define SQLITE_CANTOPEN_CONVPATH (SQLITE_CANTOPEN | (4<<8)) | < | 504 505 506 507 508 509 510 511 512 513 514 515 516 517 | #define SQLITE_LOCKED_SHAREDCACHE (SQLITE_LOCKED | (1<<8)) #define SQLITE_BUSY_RECOVERY (SQLITE_BUSY | (1<<8)) #define SQLITE_BUSY_SNAPSHOT (SQLITE_BUSY | (2<<8)) #define SQLITE_CANTOPEN_NOTEMPDIR (SQLITE_CANTOPEN | (1<<8)) #define SQLITE_CANTOPEN_ISDIR (SQLITE_CANTOPEN | (2<<8)) #define SQLITE_CANTOPEN_FULLPATH (SQLITE_CANTOPEN | (3<<8)) #define SQLITE_CANTOPEN_CONVPATH (SQLITE_CANTOPEN | (4<<8)) #define SQLITE_CORRUPT_VTAB (SQLITE_CORRUPT | (1<<8)) #define SQLITE_READONLY_RECOVERY (SQLITE_READONLY | (1<<8)) #define SQLITE_READONLY_CANTLOCK (SQLITE_READONLY | (2<<8)) #define SQLITE_READONLY_ROLLBACK (SQLITE_READONLY | (3<<8)) #define SQLITE_READONLY_DBMOVED (SQLITE_READONLY | (4<<8)) #define SQLITE_ABORT_ROLLBACK (SQLITE_ABORT | (2<<8)) #define SQLITE_CONSTRAINT_CHECK (SQLITE_CONSTRAINT | (1<<8)) |
︙ | ︙ | |||
1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 | ** The [SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE] opcode causes all write ** operations since the previous successful call to ** [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] to be rolled back. ** ^This file control takes the file descriptor out of batch write mode ** so that all subsequent write operations are independent. ** ^SQLite will never invoke SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE without ** a prior successful call to [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE]. ** </ul> */ #define SQLITE_FCNTL_LOCKSTATE 1 #define SQLITE_FCNTL_GET_LOCKPROXYFILE 2 #define SQLITE_FCNTL_SET_LOCKPROXYFILE 3 #define SQLITE_FCNTL_LAST_ERRNO 4 #define SQLITE_FCNTL_SIZE_HINT 5 | > > > > > > > > > > > > > > | 1056 1057 1058 1059 1060 1061 1062 1063 1064 1065 1066 1067 1068 1069 1070 1071 1072 1073 1074 1075 1076 1077 1078 1079 1080 1081 1082 1083 | ** The [SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE] opcode causes all write ** operations since the previous successful call to ** [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE] to be rolled back. ** ^This file control takes the file descriptor out of batch write mode ** so that all subsequent write operations are independent. ** ^SQLite will never invoke SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE without ** a prior successful call to [SQLITE_FCNTL_BEGIN_ATOMIC_WRITE]. ** ** <li>[[SQLITE_FCNTL_READ_SHM]] ** The [SQLITE_FCNTL_READ_SHM] opcode may be used to read data from ** the shared-memory region used by wal mode if the previous call ** to the xShmMap method returned SQLITE_READONLY_CANTLOCK. The argument ** must point to an instance of a structure declared as: ** <blockquote><pre> ** struct ReadShmArg { void *pBuf; int nBuf }; ** </pre></blockquote>)^ ** Before returning, this operation allocates a buffer large enough for ** the entire shared-memory using sqlite3_malloc() and populates it with ** a copy thereof. ReadShmArg.pBuf is set to point to the buffer, and ** ReadShmArg to its size in bytes. It is the responsibility of the caller ** to eventually free the buffer using sqlite3_free(). ** </ul> */ #define SQLITE_FCNTL_LOCKSTATE 1 #define SQLITE_FCNTL_GET_LOCKPROXYFILE 2 #define SQLITE_FCNTL_SET_LOCKPROXYFILE 3 #define SQLITE_FCNTL_LAST_ERRNO 4 #define SQLITE_FCNTL_SIZE_HINT 5 |
︙ | ︙ | |||
1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 | #define SQLITE_FCNTL_VFS_POINTER 27 #define SQLITE_FCNTL_JOURNAL_POINTER 28 #define SQLITE_FCNTL_WIN32_GET_HANDLE 29 #define SQLITE_FCNTL_PDB 30 #define SQLITE_FCNTL_BEGIN_ATOMIC_WRITE 31 #define SQLITE_FCNTL_COMMIT_ATOMIC_WRITE 32 #define SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE 33 /* deprecated names */ #define SQLITE_GET_LOCKPROXYFILE SQLITE_FCNTL_GET_LOCKPROXYFILE #define SQLITE_SET_LOCKPROXYFILE SQLITE_FCNTL_SET_LOCKPROXYFILE #define SQLITE_LAST_ERRNO SQLITE_FCNTL_LAST_ERRNO | > | 1104 1105 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 | #define SQLITE_FCNTL_VFS_POINTER 27 #define SQLITE_FCNTL_JOURNAL_POINTER 28 #define SQLITE_FCNTL_WIN32_GET_HANDLE 29 #define SQLITE_FCNTL_PDB 30 #define SQLITE_FCNTL_BEGIN_ATOMIC_WRITE 31 #define SQLITE_FCNTL_COMMIT_ATOMIC_WRITE 32 #define SQLITE_FCNTL_ROLLBACK_ATOMIC_WRITE 33 #define SQLITE_FCNTL_READ_SHM 34 /* deprecated names */ #define SQLITE_GET_LOCKPROXYFILE SQLITE_FCNTL_GET_LOCKPROXYFILE #define SQLITE_SET_LOCKPROXYFILE SQLITE_FCNTL_SET_LOCKPROXYFILE #define SQLITE_LAST_ERRNO SQLITE_FCNTL_LAST_ERRNO |
︙ | ︙ |
Changes to src/wal.c.
︙ | ︙ | |||
451 452 453 454 455 456 457 458 459 460 461 462 463 464 | u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */ u8 writeLock; /* True if in a write transaction */ u8 ckptLock; /* True if holding a checkpoint lock */ u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */ u8 truncateOnCommit; /* True to truncate WAL file on commit */ u8 syncHeader; /* Fsync the WAL header if true */ u8 padToSectorBoundary; /* Pad transactions out to the next sector */ WalIndexHdr hdr; /* Wal-index header for current transaction */ u32 minFrame; /* Ignore wal frames before this one */ u32 iReCksum; /* On commit, recalculate checksums from here */ const char *zWalName; /* Name of WAL file */ u32 nCkpt; /* Checkpoint sequence counter in the wal-header */ #ifdef SQLITE_DEBUG u8 lockError; /* True if a locking error has occurred */ | > | 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 | u8 exclusiveMode; /* Non-zero if connection is in exclusive mode */ u8 writeLock; /* True if in a write transaction */ u8 ckptLock; /* True if holding a checkpoint lock */ u8 readOnly; /* WAL_RDWR, WAL_RDONLY, or WAL_SHM_RDONLY */ u8 truncateOnCommit; /* True to truncate WAL file on commit */ u8 syncHeader; /* Fsync the WAL header if true */ u8 padToSectorBoundary; /* Pad transactions out to the next sector */ u8 bUnlocked; /* A readonly_shm connection without DMS lock */ WalIndexHdr hdr; /* Wal-index header for current transaction */ u32 minFrame; /* Ignore wal frames before this one */ u32 iReCksum; /* On commit, recalculate checksums from here */ const char *zWalName; /* Name of WAL file */ u32 nCkpt; /* Checkpoint sequence counter in the wal-header */ #ifdef SQLITE_DEBUG u8 lockError; /* True if a locking error has occurred */ |
︙ | ︙ | |||
534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 | */ #define HASHTABLE_NPAGE_ONE (HASHTABLE_NPAGE - (WALINDEX_HDR_SIZE/sizeof(u32))) /* The wal-index is divided into pages of WALINDEX_PGSZ bytes each. */ #define WALINDEX_PGSZ ( \ sizeof(ht_slot)*HASHTABLE_NSLOT + HASHTABLE_NPAGE*sizeof(u32) \ ) /* ** Obtain a pointer to the iPage'th page of the wal-index. The wal-index ** is broken into pages of WALINDEX_PGSZ bytes. Wal-index pages are ** numbered from zero. ** ** If this call is successful, *ppPage is set to point to the wal-index ** page and SQLITE_OK is returned. If an error (an OOM or VFS error) occurs, ** then an SQLite error code is returned and *ppPage is set to 0. */ static int walIndexPage(Wal *pWal, int iPage, volatile u32 **ppPage){ int rc = SQLITE_OK; /* Enlarge the pWal->apWiData[] array if required */ | > > > > > > > > > > > > > > > > > > > > > < | < < < | | < < < < < | 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 | */ #define HASHTABLE_NPAGE_ONE (HASHTABLE_NPAGE - (WALINDEX_HDR_SIZE/sizeof(u32))) /* The wal-index is divided into pages of WALINDEX_PGSZ bytes each. */ #define WALINDEX_PGSZ ( \ sizeof(ht_slot)*HASHTABLE_NSLOT + HASHTABLE_NPAGE*sizeof(u32) \ ) /* ** Grow the pWal->apWiData[] array so that it is at least nPage entries ** in size. Return SQLITE_NOMEM if an OOM error occurs, or SQLITE_OK ** otherwise. */ static int walIndexGrowArray(Wal *pWal, int nPage){ if( nPage>pWal->nWiData ){ int nByte = sizeof(u32*)*nPage; volatile u32 **apNew; apNew = (volatile u32 **)sqlite3_realloc64((void *)pWal->apWiData, nByte); if( !apNew ){ return SQLITE_NOMEM_BKPT; } memset((void*)&apNew[pWal->nWiData], 0, sizeof(u32*)*(nPage-pWal->nWiData)); pWal->apWiData = apNew; pWal->nWiData = nPage; } return SQLITE_OK; } /* ** Obtain a pointer to the iPage'th page of the wal-index. The wal-index ** is broken into pages of WALINDEX_PGSZ bytes. Wal-index pages are ** numbered from zero. ** ** If this call is successful, *ppPage is set to point to the wal-index ** page and SQLITE_OK is returned. If an error (an OOM or VFS error) occurs, ** then an SQLite error code is returned and *ppPage is set to 0. */ static int walIndexPage(Wal *pWal, int iPage, volatile u32 **ppPage){ int rc = SQLITE_OK; /* Enlarge the pWal->apWiData[] array if required */ if( walIndexGrowArray(pWal, iPage+1) ){ *ppPage = 0; return SQLITE_NOMEM; } /* Request a pointer to the required page from the VFS */ if( pWal->apWiData[iPage]==0 ){ if( pWal->exclusiveMode==WAL_HEAPMEMORY_MODE ){ pWal->apWiData[iPage] = (u32 volatile *)sqlite3MallocZero(WALINDEX_PGSZ); if( !pWal->apWiData[iPage] ) rc = SQLITE_NOMEM_BKPT; |
︙ | ︙ | |||
1098 1099 1100 1101 1102 1103 1104 | */ static int walIndexRecover(Wal *pWal){ int rc; /* Return Code */ i64 nSize; /* Size of log file */ u32 aFrameCksum[2] = {0, 0}; int iLock; /* Lock offset to lock for checkpoint */ | | | | | | < | 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 1121 1122 1123 1124 1125 1126 1127 1128 1129 | */ static int walIndexRecover(Wal *pWal){ int rc; /* Return Code */ i64 nSize; /* Size of log file */ u32 aFrameCksum[2] = {0, 0}; int iLock; /* Lock offset to lock for checkpoint */ /* Obtain an exclusive lock on all bytes in the locking range except ** WAL_READ_LOCK(0) not already locked by the caller. The caller is ** guaranteed to have locked the WAL_WRITE_LOCK byte, and may have also ** locked the WAL_CKPT_LOCK byte. If successful, the same bytes that are ** locked here are unlocked before this function returns. */ assert( pWal->ckptLock==1 || pWal->ckptLock==0 ); assert( WAL_ALL_BUT_WRITE==WAL_WRITE_LOCK+1 ); assert( WAL_CKPT_LOCK==WAL_ALL_BUT_WRITE ); assert( pWal->writeLock ); iLock = WAL_ALL_BUT_WRITE + pWal->ckptLock; rc = walLockExclusive(pWal, iLock, WAL_READ_LOCK(0)-iLock); if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
2086 2087 2088 2089 2090 2091 2092 2093 2094 | int badHdr; /* True if a header read failed */ volatile u32 *page0; /* Chunk of wal-index containing header */ /* Ensure that page 0 of the wal-index (the page that contains the ** wal-index header) is mapped. Return early if an error occurs here. */ assert( pChanged ); rc = walIndexPage(pWal, 0, &page0); if( rc!=SQLITE_OK ){ | > < < < < < < < < > > > | | 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 | int badHdr; /* True if a header read failed */ volatile u32 *page0; /* Chunk of wal-index containing header */ /* Ensure that page 0 of the wal-index (the page that contains the ** wal-index header) is mapped. Return early if an error occurs here. */ assert( pChanged ); assert( pWal->bUnlocked==0 ); rc = walIndexPage(pWal, 0, &page0); if( rc!=SQLITE_OK ){ return rc; }; assert( page0 || pWal->writeLock==0 ); /* If the first page of the wal-index has been mapped, try to read the ** wal-index header immediately, without holding any lock. This usually ** works, but may fail if the wal-index header is corrupt or currently ** being modified by another thread or process. */ badHdr = (page0 ? walIndexTryHdr(pWal, pChanged) : 1); /* If the first attempt failed, it might have been due to a race ** with a writer. So get a WRITE lock and try again. */ assert( badHdr==0 || pWal->writeLock==0 ); if( badHdr ){ if( pWal->readOnly & WAL_SHM_RDONLY ){ if( pWal->bUnlocked ){ rc = SQLITE_READONLY_RECOVERY; } else if( SQLITE_OK==(rc = walLockShared(pWal, WAL_WRITE_LOCK)) ){ walUnlockShared(pWal, WAL_WRITE_LOCK); rc = SQLITE_READONLY_RECOVERY; } }else if( SQLITE_OK==(rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1)) ){ pWal->writeLock = 1; if( SQLITE_OK==(rc = walIndexPage(pWal, 0, &page0)) ){ badHdr = walIndexTryHdr(pWal, pChanged); |
︙ | ︙ | |||
2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 | } /* ** This is the value that walTryBeginRead returns when it needs to ** be retried. */ #define WAL_RETRY (-1) /* ** Attempt to start a read transaction. This might fail due to a race or ** other transient condition. When that happens, it returns WAL_RETRY to ** indicate to the caller that it is safe to retry immediately. ** ** On success return SQLITE_OK. On a permanent failure (such an | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 | } /* ** This is the value that walTryBeginRead returns when it needs to ** be retried. */ #define WAL_RETRY (-1) /* ** When this function is called, a call to xShmMap has just returned ** SQLITE_READONLY_CANTLOCK, indicating that the *-shm file was opened ** read-only and that there were no other connections to the database ** at that point. */ static int walBeginUnlocked(Wal *pWal, int *pChanged){ struct ReadShmParam { void *pBuf; int nBuf; } buf = {0, 0}; i64 nSize; int rc; volatile void *pDummy; assert( pWal->nWiData>0 && pWal->apWiData[0]==0 ); assert( pWal->readOnly & WAL_SHM_RDONLY ); /* Take WAL_READ_LOCK(0). This has the effect of preventing any ** live clients from running a checkpoint, but does not stop them ** from running recovery. */ rc = walLockShared(pWal, WAL_READ_LOCK(0)); if( rc!=SQLITE_OK ){ return (rc==SQLITE_BUSY ? WAL_RETRY : rc); } /* Try to map the *-shm file again. If it succeeds this time, then ** a non-readonly_shm connection has already connected to the database. ** In this case, start over with opening the transaction. ** ** The WAL_READ_LOCK(0) lock held by this client prevents a checkpoint ** from taking place. But it does not prevent the wal from being wrapped ** if a checkpoint has already taken place. This means that if another ** client is connected at this point, it may have already checkpointed ** the entire wal. In that case it would not be safe to continue with ** the unlocked transaction, as the other client may overwrite wal ** frames that this client is still using. */ rc = sqlite3OsShmMap(pWal->pDbFd, 0, WALINDEX_PGSZ, 0, &pDummy); if( rc!=SQLITE_READONLY_CANTLOCK ){ rc = (rc==SQLITE_OK ? WAL_RETRY : rc); goto begin_unlocked_out; } pWal->readLock = 0; /* Figure out how large the wal file is. If it is zero bytes in size, ** do not bother loading the *-shm file. Just read from the db file ** exclusively like any other Wal.readLock==0 client. Otherwise, ** use SQLITE_FCNTL_READ_SHM to load the contents of the *-shm file ** into heap memory. */ rc = sqlite3OsFileSize(pWal->pWalFd, &nSize); if( rc!=SQLITE_OK ) return rc; memset(&pWal->hdr, 0, sizeof(WalIndexHdr)); if( nSize>0 ){ rc = sqlite3OsFileControl(pWal->pDbFd, SQLITE_FCNTL_READ_SHM, (void*)&buf); if( rc!=SQLITE_OK ){ return rc==SQLITE_BUSY ? WAL_RETRY : rc; } if( buf.nBuf<WALINDEX_PGSZ ){ rc = SQLITE_READONLY_RECOVERY; }else{ int i; int nChunk = buf.nBuf/WALINDEX_PGSZ; if( walIndexGrowArray(pWal, nChunk) ){ rc = SQLITE_NOMEM; }else{ u8 *aBuf = (u8*)buf.pBuf; for(i=0; i<nChunk; i++){ pWal->apWiData[i] = (volatile u32*)&aBuf[i*WALINDEX_PGSZ]; } if( walIndexTryHdr(pWal, pChanged) ){ rc = SQLITE_READONLY_RECOVERY; } } } } begin_unlocked_out: if( rc!=SQLITE_OK ){ sqlite3_free(buf.pBuf); memset(pWal->apWiData, 0, pWal->nWiData*sizeof(u32*)); walUnlockShared(pWal, WAL_READ_LOCK(0)); }else{ pWal->bUnlocked = 1; *pChanged = 1; } return rc; } /* ** Attempt to start a read transaction. This might fail due to a race or ** other transient condition. When that happens, it returns WAL_RETRY to ** indicate to the caller that it is safe to retry immediately. ** ** On success return SQLITE_OK. On a permanent failure (such an |
︙ | ︙ | |||
2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 | } if( cnt>=10 ) nDelay = (cnt-9)*(cnt-9)*39; sqlite3OsSleep(pWal->pVfs, nDelay); } if( !useWal ){ rc = walIndexReadHdr(pWal, pChanged); if( rc==SQLITE_BUSY ){ /* If there is not a recovery running in another thread or process ** then convert BUSY errors to WAL_RETRY. If recovery is known to ** be running, convert BUSY to BUSY_RECOVERY. There is a race here ** which might cause WAL_RETRY to be returned even if BUSY_RECOVERY ** would be technically correct. But the race is benign since with ** WAL_RETRY this routine will be called again and will probably be | > > > > > > > > > | 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 | } if( cnt>=10 ) nDelay = (cnt-9)*(cnt-9)*39; sqlite3OsSleep(pWal->pVfs, nDelay); } if( !useWal ){ rc = walIndexReadHdr(pWal, pChanged); if( rc==SQLITE_READONLY_CANTLOCK ){ /* This is a readonly_shm connection and there are no other connections ** to the database. So the *-shm file may not be accessed using mmap. ** Try to open an "unlocked" transaction - one that loads the *-shm ** file into memory using read() - instead. */ assert( pWal->nWiData>0 && pWal->apWiData[0]==0 ); assert( pWal->readOnly & WAL_SHM_RDONLY ); return walBeginUnlocked(pWal, pChanged); } if( rc==SQLITE_BUSY ){ /* If there is not a recovery running in another thread or process ** then convert BUSY errors to WAL_RETRY. If recovery is known to ** be running, convert BUSY to BUSY_RECOVERY. There is a race here ** which might cause WAL_RETRY to be returned even if BUSY_RECOVERY ** would be technically correct. But the race is benign since with ** WAL_RETRY this routine will be called again and will probably be |
︙ | ︙ | |||
2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 | ** read-lock. */ void sqlite3WalEndReadTransaction(Wal *pWal){ sqlite3WalEndWriteTransaction(pWal); if( pWal->readLock>=0 ){ walUnlockShared(pWal, WAL_READ_LOCK(pWal->readLock)); pWal->readLock = -1; } } /* ** Search the wal file for page pgno. If found, set *piRead to the frame that ** contains the page. Otherwise, if pgno is not in the wal file, set *piRead ** to zero. | > > > > > | 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 | ** read-lock. */ void sqlite3WalEndReadTransaction(Wal *pWal){ sqlite3WalEndWriteTransaction(pWal); if( pWal->readLock>=0 ){ walUnlockShared(pWal, WAL_READ_LOCK(pWal->readLock)); pWal->readLock = -1; if( pWal->bUnlocked ){ sqlite3_free((void*)pWal->apWiData[0]); memset(pWal->apWiData, 0, sizeof(u32*)*pWal->nWiData); pWal->bUnlocked = 0; } } } /* ** Search the wal file for page pgno. If found, set *piRead to the frame that ** contains the page. Otherwise, if pgno is not in the wal file, set *piRead ** to zero. |
︙ | ︙ | |||
2622 2623 2624 2625 2626 2627 2628 | /* If the "last page" field of the wal-index header snapshot is 0, then ** no data will be read from the wal under any circumstances. Return early ** in this case as an optimization. Likewise, if pWal->readLock==0, ** then the WAL is ignored by the reader so return early, as if the ** WAL were empty. */ | | | 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 | /* If the "last page" field of the wal-index header snapshot is 0, then ** no data will be read from the wal under any circumstances. Return early ** in this case as an optimization. Likewise, if pWal->readLock==0, ** then the WAL is ignored by the reader so return early, as if the ** WAL were empty. */ if( iLast==0 || (pWal->readLock==0 && !pWal->bUnlocked) ){ *piRead = 0; return SQLITE_OK; } /* Search the hash table or tables for an entry matching page number ** pgno. Each iteration of the following for() loop searches one ** hash table (each hash table indexes up to HASHTABLE_NPAGE frames). |
︙ | ︙ |
Changes to src/whereexpr.c.
︙ | ︙ | |||
979 980 981 982 983 984 985 | Bitmask extraRight = 0; /* Extra dependencies on LEFT JOIN */ Expr *pStr1 = 0; /* RHS of LIKE/GLOB operator */ int isComplete = 0; /* RHS of LIKE/GLOB ends with wildcard */ int noCase = 0; /* uppercase equivalent to lowercase */ int op; /* Top-level operator. pExpr->op */ Parse *pParse = pWInfo->pParse; /* Parsing context */ sqlite3 *db = pParse->db; /* Database connection */ | | | 979 980 981 982 983 984 985 986 987 988 989 990 991 992 993 | Bitmask extraRight = 0; /* Extra dependencies on LEFT JOIN */ Expr *pStr1 = 0; /* RHS of LIKE/GLOB operator */ int isComplete = 0; /* RHS of LIKE/GLOB ends with wildcard */ int noCase = 0; /* uppercase equivalent to lowercase */ int op; /* Top-level operator. pExpr->op */ Parse *pParse = pWInfo->pParse; /* Parsing context */ sqlite3 *db = pParse->db; /* Database connection */ unsigned char eOp2 = 0; /* op2 value for LIKE/REGEXP/GLOB */ int nLeft; /* Number of elements on left side vector */ if( db->mallocFailed ){ return; } pTerm = &pWC->a[idxTerm]; pMaskSet = &pWInfo->sMaskSet; |
︙ | ︙ | |||
1223 1224 1225 1226 1227 1228 1229 | ** not normally optimized for ordinary tables. In other words, OP ** is one of MATCH, LIKE, GLOB, REGEXP, !=, IS, IS NOT, or NOT NULL. ** This information is used by the xBestIndex methods of ** virtual tables. The native query optimizer does not attempt ** to do anything with MATCH functions. */ if( pWC->op==TK_AND ){ | | | 1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 | ** not normally optimized for ordinary tables. In other words, OP ** is one of MATCH, LIKE, GLOB, REGEXP, !=, IS, IS NOT, or NOT NULL. ** This information is used by the xBestIndex methods of ** virtual tables. The native query optimizer does not attempt ** to do anything with MATCH functions. */ if( pWC->op==TK_AND ){ Expr *pRight = 0, *pLeft = 0; int res = isAuxiliaryVtabOperator(pExpr, &eOp2, &pLeft, &pRight); while( res-- > 0 ){ int idxNew; WhereTerm *pNewTerm; Bitmask prereqColumn, prereqExpr; prereqExpr = sqlite3WhereExprUsage(pMaskSet, pRight); |
︙ | ︙ |
Deleted test/checkfreelist.test.
|
| < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < < |
Changes to test/walro.test.
︙ | ︙ | |||
97 98 99 100 101 102 103 | do_test 1.1.13 { sql2 "INSERT INTO t1 VALUES('i', 'j')" } {} do_test 1.2.1 { code2 { db2 close } code1 { db close } list [file exists test.db-wal] [file exists test.db-shm] } {1 1} | < | | | | | 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 | do_test 1.1.13 { sql2 "INSERT INTO t1 VALUES('i', 'j')" } {} do_test 1.2.1 { code2 { db2 close } code1 { db close } list [file exists test.db-wal] [file exists test.db-shm] } {1 1} do_test 1.2.2 { code1 { sqlite3 db file:test.db?readonly_shm=1 } sql1 { SELECT * FROM t1 } } {a b c d e f g h i j} do_test 1.2.3 { code1 { db close } file attributes test.db-shm -permissions rw-r--r-- hexio_write test.db-shm 0 01020304 file attributes test.db-shm -permissions r--r--r-- code1 { sqlite3 db file:test.db?readonly_shm=1 } csql1 { SELECT * FROM t1 } } {1 {attempt to write a readonly database}} do_test 1.2.4 { code1 { sqlite3_extended_errcode db } } {SQLITE_READONLY_RECOVERY} do_test 1.2.5 { file attributes test.db-shm -permissions rw-r--r-- code2 { sqlite3 db2 test.db } sql2 "SELECT * FROM t1" } {a b c d e f g h i j} file attributes test.db-shm -permissions r--r--r-- |
︙ | ︙ | |||
152 153 154 155 156 157 158 159 160 161 162 163 164 | code2 { db2 close } file exists test.db-shm } {0} do_test 1.3.2.2 { code1 { sqlite3 db file:test.db?readonly_shm=1 } csql1 { SELECT * FROM sqlite_master } } {1 {unable to open database file}} do_test 1.3.2.3 { code1 { db close } close [open test.db-shm w] file attributes test.db-shm -permissions r--r--r-- code1 { sqlite3 db file:test.db?readonly_shm=1 } csql1 { SELECT * FROM t1 } | > > > > > | | | 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 | code2 { db2 close } file exists test.db-shm } {0} do_test 1.3.2.2 { code1 { sqlite3 db file:test.db?readonly_shm=1 } csql1 { SELECT * FROM sqlite_master } } {1 {unable to open database file}} # Update: This test used to fail (because a 0 byte *-shm file does not # contain a valid shm-header) but now succeeds (because a readonly_shm # connection ignores the *-shm file completely if the wal file is also # zero bytes in size). do_test 1.3.2.3 { code1 { db close } close [open test.db-shm w] file attributes test.db-shm -permissions r--r--r-- code1 { sqlite3 db file:test.db?readonly_shm=1 } csql1 { SELECT * FROM t1 } } {0 {a b c d e f g h i j k l}} do_test 1.3.2.4 { code1 { sqlite3_extended_errcode db } } {SQLITE_OK} #----------------------------------------------------------------------- # Test cases 1.4.* check that checkpoints and log wraps don't prevent # read-only connections from reading the database. do_test 1.4.1 { code1 { db close } forcedelete test.db-shm |
︙ | ︙ |
Changes to test/walro2.test.
︙ | ︙ | |||
58 59 60 61 62 63 64 | PRAGMA journal_mode = WAL; INSERT INTO t1 VALUES('a', 'b'); INSERT INTO t1 VALUES('c', 'd'); } file exists test.db-shm } {1} | | | > | | 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | PRAGMA journal_mode = WAL; INSERT INTO t1 VALUES('a', 'b'); INSERT INTO t1 VALUES('c', 'd'); } file exists test.db-shm } {1} do_test 1.2.1 { forcecopy test.db test.db2 forcecopy test.db-wal test.db2-wal forcecopy test.db-shm test.db2-shm code1 { sqlite3 db file:test.db2?readonly_shm=1 } } {} do_test 1.2.2 { sql1 { SELECT * FROM t1 } } {a b c d} do_test 1.3.1 { code3 { sqlite3 db3 test.db2 } sql3 { SELECT * FROM t1 } } {a b c d} do_test 1.3.2 { |
︙ | ︙ | |||
102 103 104 105 106 107 108 | code1 { sqlite3 db file:test.db2?readonly_shm=1 } sql1 { BEGIN; SELECT * FROM t1; } | | | 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 | code1 { sqlite3 db file:test.db2?readonly_shm=1 } sql1 { BEGIN; SELECT * FROM t1; } } {a b c d e f g h} do_test 2.3.1 { code3 { sqlite3 db3 test.db2 } sql3 { SELECT * FROM t1 } } {a b c d e f g h} } finish_test |