Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Merge trunk changes, including the addition of FTS5 and pcache1 performance enhancements. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | sessions |
Files: | files | file ages | folders |
SHA1: |
db4cbefb8674c6cfff27c1e918741de1 |
User & Date: | drh 2015-07-14 15:39:22.594 |
Context
2015-07-23
| ||
21:59 | Merge all changes in the latest 3.8.11 beta into the sessions branch. Changes include the rename of OTA to RBU, the WITHOUT-ROWID-OR-Skipscan fix, and improvements to pcache1. (check-in: 7f0ee77062 user: drh tags: sessions) | |
2015-07-14
| ||
15:39 | Merge trunk changes, including the addition of FTS5 and pcache1 performance enhancements. (check-in: db4cbefb86 user: drh tags: sessions) | |
14:48 | Always invoke the profile callback even if the statement does not run to completion. (check-in: 202479aa0a user: drh tags: trunk) | |
2015-07-02
| ||
18:47 | Merge in the latest enhancements from trunks, especially the use of _byteswap_ulong() and similar intrinsics on MSVC. (check-in: 85ca4409bd user: drh tags: sessions) | |
Changes
Changes to Makefile.in.
︙ | ︙ | |||
1006 1007 1008 1009 1010 1011 1012 | $(TOP)/ext/fts5/fts5_varint.c \ $(TOP)/ext/fts5/fts5_vocab.c \ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon cp $(TOP)/ext/fts5/fts5parse.y . rm -f fts5parse.h ./lemon $(OPTS) fts5parse.y | < < < > | 1006 1007 1008 1009 1010 1011 1012 1013 1014 1015 1016 1017 1018 1019 1020 1021 1022 1023 1024 1025 | $(TOP)/ext/fts5/fts5_varint.c \ $(TOP)/ext/fts5/fts5_vocab.c \ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon cp $(TOP)/ext/fts5/fts5parse.y . rm -f fts5parse.h ./lemon $(OPTS) fts5parse.y fts5parse.h: fts5parse.c fts5.c: $(FTS5_SRC) $(TCLSH_CMD) $(TOP)/ext/fts5/tool/mkfts5c.tcl cp $(TOP)/ext/fts5/fts5.h . fts5.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) -DSQLITE_CORE -c fts5.c # Rules to build the 'testfixture' application. # |
︙ | ︙ | |||
1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 1214 | rm -f shell.c sqlite3ext.h rm -f sqlite3_analyzer$(TEXE) sqlite3_analyzer.c rm -f sqlite-*-output.vsix rm -f mptester mptester.exe rm -f fuzzershell fuzzershell.exe rm -f fuzzcheck fuzzcheck.exe rm -f sqldiff sqldiff.exe distclean: clean rm -f config.log config.status libtool Makefile sqlite3.pc # # Windows section # | > | 1199 1200 1201 1202 1203 1204 1205 1206 1207 1208 1209 1210 1211 1212 1213 | rm -f shell.c sqlite3ext.h rm -f sqlite3_analyzer$(TEXE) sqlite3_analyzer.c rm -f sqlite-*-output.vsix rm -f mptester mptester.exe rm -f fuzzershell fuzzershell.exe rm -f fuzzcheck fuzzcheck.exe rm -f sqldiff sqldiff.exe rm -f fts5.c fts5.h fts5parse.* distclean: clean rm -f config.log config.status libtool Makefile sqlite3.pc # # Windows section # |
︙ | ︙ |
Changes to Makefile.msc.
︙ | ︙ | |||
1703 1704 1705 1706 1707 1708 1709 | $(TOP)\ext\fts5\fts5_varint.c \ $(TOP)\ext\fts5\fts5_vocab.c fts5parse.c: $(TOP)\ext\fts5\fts5parse.y lemon.exe copy $(TOP)\ext\fts5\fts5parse.y . del /Q fts5parse.h 2>NUL .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(OPTS) fts5parse.y | < < < < < > | 1703 1704 1705 1706 1707 1708 1709 1710 1711 1712 1713 1714 1715 1716 1717 1718 1719 1720 1721 1722 | $(TOP)\ext\fts5\fts5_varint.c \ $(TOP)\ext\fts5\fts5_vocab.c fts5parse.c: $(TOP)\ext\fts5\fts5parse.y lemon.exe copy $(TOP)\ext\fts5\fts5parse.y . del /Q fts5parse.h 2>NUL .\lemon.exe $(REQ_FEATURE_FLAGS) $(OPT_FEATURE_FLAGS) $(OPTS) fts5parse.y fts5parse.h: fts5parse.c fts5.c: $(FTS5_SRC) $(TCLSH_CMD) $(TOP)\ext\fts5\tool\mkfts5c.tcl copy $(TOP)\ext\fts5\fts5.h . fts5.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(CORE_COMPILE_OPTS) $(NO_WARN) -DSQLITE_CORE -c fts5.c fts5_ext.lo: fts5.c $(HDR) $(EXTHDR) $(LTCOMPILE) $(NO_WARN) -c fts5.c |
︙ | ︙ | |||
1866 1867 1868 1869 1870 1871 1872 | del /Q sqlite3.exe sqlite3.dll sqlite3.def 2>NUL del /Q sqlite3.c sqlite3-*.c 2>NUL del /Q sqlite3rc.h 2>NUL del /Q shell.c sqlite3ext.h 2>NUL del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL del /Q sqlite-*-output.vsix 2>NUL del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe 2>NUL | | | 1862 1863 1864 1865 1866 1867 1868 1869 1870 1871 1872 1873 1874 1875 1876 1877 1878 1879 1880 1881 1882 | del /Q sqlite3.exe sqlite3.dll sqlite3.def 2>NUL del /Q sqlite3.c sqlite3-*.c 2>NUL del /Q sqlite3rc.h 2>NUL del /Q shell.c sqlite3ext.h 2>NUL del /Q sqlite3_analyzer.exe sqlite3_analyzer.c 2>NUL del /Q sqlite-*-output.vsix 2>NUL del /Q fuzzershell.exe fuzzcheck.exe sqldiff.exe 2>NUL del /Q fts5.c fts5.h fts5parse.* 2>NUL # Dynamic link library section. # dll: sqlite3.dll sqlite3.def: libsqlite3.lib echo EXPORTS > sqlite3.def dumpbin /all libsqlite3.lib \ | $(NAWK) "/ 1 _?sqlite3_/ { sub(/^.* _?/,\"\");print }" \ | sort >> sqlite3.def sqlite3.dll: $(LIBOBJ) $(LIBRESOBJS) $(CORE_LINK_DEP) $(LD) $(LDFLAGS) $(LTLINKOPTS) $(LTLIBPATHS) /DLL $(CORE_LINK_OPTS) /OUT:$@ $(LIBOBJ) $(LIBRESOBJS) $(LTLIBS) $(TLIBS) |
Changes to ext/fts3/fts3.c.
︙ | ︙ | |||
4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 | assert( !p || (p>=aDoclist && p<=&aDoclist[nDoclist]) ); if( p==0 ){ p = aDoclist; p += sqlite3Fts3GetVarint(p, piDocid); }else{ fts3PoslistCopy(0, &p); if( p>=&aDoclist[nDoclist] ){ *pbEof = 1; }else{ sqlite3_int64 iVar; p += sqlite3Fts3GetVarint(p, &iVar); *piDocid += ((bDescIdx ? -1 : 1) * iVar); } | > | 4347 4348 4349 4350 4351 4352 4353 4354 4355 4356 4357 4358 4359 4360 4361 | assert( !p || (p>=aDoclist && p<=&aDoclist[nDoclist]) ); if( p==0 ){ p = aDoclist; p += sqlite3Fts3GetVarint(p, piDocid); }else{ fts3PoslistCopy(0, &p); while( p<&aDoclist[nDoclist] && *p==0 ) p++; if( p>=&aDoclist[nDoclist] ){ *pbEof = 1; }else{ sqlite3_int64 iVar; p += sqlite3Fts3GetVarint(p, &iVar); *piDocid += ((bDescIdx ? -1 : 1) * iVar); } |
︙ | ︙ | |||
5753 5754 5755 5756 5757 5758 5759 | iDocid = pExpr->iDocid; pIter = pPhrase->doclist.pList; if( iDocid!=pCsr->iPrevId || pExpr->bEof ){ int rc = SQLITE_OK; int bDescDoclist = pTab->bDescIdx; /* For DOCID_CMP macro */ int bOr = 0; | < > | 5754 5755 5756 5757 5758 5759 5760 5761 5762 5763 5764 5765 5766 5767 5768 5769 5770 5771 | iDocid = pExpr->iDocid; pIter = pPhrase->doclist.pList; if( iDocid!=pCsr->iPrevId || pExpr->bEof ){ int rc = SQLITE_OK; int bDescDoclist = pTab->bDescIdx; /* For DOCID_CMP macro */ int bOr = 0; u8 bTreeEof = 0; Fts3Expr *p; /* Used to iterate from pExpr to root */ Fts3Expr *pNear; /* Most senior NEAR ancestor (or pExpr) */ int bMatch; /* Check if this phrase descends from an OR expression node. If not, ** return NULL. Otherwise, the entry that corresponds to docid ** pCsr->iPrevId may lie earlier in the doclist buffer. Or, if the ** tree that the node is part of has been marked as EOF, but the node ** itself is not EOF, then it may point to an earlier entry. */ pNear = pExpr; |
︙ | ︙ | |||
5790 5791 5792 5793 5794 5795 5796 | if( bTreeEof ){ while( rc==SQLITE_OK && !pNear->bEof ){ fts3EvalNextRow(pCsr, pNear, &rc); } } if( rc!=SQLITE_OK ) return rc; | > > > > > > > > > > | | | | | | | | | | | | | | | | | | | | | | | > | > > > > | > | 5791 5792 5793 5794 5795 5796 5797 5798 5799 5800 5801 5802 5803 5804 5805 5806 5807 5808 5809 5810 5811 5812 5813 5814 5815 5816 5817 5818 5819 5820 5821 5822 5823 5824 5825 5826 5827 5828 5829 5830 5831 5832 5833 5834 5835 5836 5837 5838 5839 5840 5841 5842 5843 5844 5845 | if( bTreeEof ){ while( rc==SQLITE_OK && !pNear->bEof ){ fts3EvalNextRow(pCsr, pNear, &rc); } } if( rc!=SQLITE_OK ) return rc; bMatch = 1; for(p=pNear; p; p=p->pLeft){ u8 bEof = 0; Fts3Expr *pTest = p; Fts3Phrase *pPh; assert( pTest->eType==FTSQUERY_NEAR || pTest->eType==FTSQUERY_PHRASE ); if( pTest->eType==FTSQUERY_NEAR ) pTest = pTest->pRight; assert( pTest->eType==FTSQUERY_PHRASE ); pPh = pTest->pPhrase; pIter = pPh->pOrPoslist; iDocid = pPh->iOrDocid; if( pCsr->bDesc==bDescDoclist ){ bEof = !pPh->doclist.nAll || (pIter >= (pPh->doclist.aAll + pPh->doclist.nAll)); while( (pIter==0 || DOCID_CMP(iDocid, pCsr->iPrevId)<0 ) && bEof==0 ){ sqlite3Fts3DoclistNext( bDescDoclist, pPh->doclist.aAll, pPh->doclist.nAll, &pIter, &iDocid, &bEof ); } }else{ bEof = !pPh->doclist.nAll || (pIter && pIter<=pPh->doclist.aAll); while( (pIter==0 || DOCID_CMP(iDocid, pCsr->iPrevId)>0 ) && bEof==0 ){ int dummy; sqlite3Fts3DoclistPrev( bDescDoclist, pPh->doclist.aAll, pPh->doclist.nAll, &pIter, &iDocid, &dummy, &bEof ); } } pPh->pOrPoslist = pIter; pPh->iOrDocid = iDocid; if( bEof || iDocid!=pCsr->iPrevId ) bMatch = 0; } if( bMatch ){ pIter = pPhrase->pOrPoslist; }else{ pIter = 0; } } if( pIter==0 ) return SQLITE_OK; if( *pIter==0x01 ){ pIter++; pIter += fts3GetVarint32(pIter, &iThis); }else{ |
︙ | ︙ |
Changes to ext/fts5/extract_api_docs.tcl.
︙ | ︙ | |||
130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | array set M [get_struct_members $data] # Initialize global list D as a map from section name to documentation # text. Most (all?) section names are structure member names. # set D [get_struct_docs $data [array names M]] foreach {sub docs} $D { if {[info exists M($sub)]} { set hdr $M($sub) set link " id=$sub" } else { set link "" } | > | | | | > > > > > | | > > > | 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 | array set M [get_struct_members $data] # Initialize global list D as a map from section name to documentation # text. Most (all?) section names are structure member names. # set D [get_struct_docs $data [array names M]] output "<dl>" foreach {sub docs} $D { if {[info exists M($sub)]} { set hdr $M($sub) set link " id=$sub" } else { set link "" } #output "<hr color=#eeeee style=\"margin:1em 8.4ex 0 8.4ex;\"$link>" #set style "padding-left:6ex;font-size:1.4em;display:block" #output "<h style=\"$style\"><pre>$hdr</pre></h>" regsub -line {^ *[)]} $hdr ")" hdr output "<dt style=\"white-space:pre;font-family:monospace;font-size:120%\"" output "$link>" output "<b>$hdr</b></dt><dd>" set mode "" set margin " style=margin-top:0.1em" foreach line [split [string trim $docs] "\n"] { if {[string trim $line]==""} { if {$mode != ""} {output "</$mode>"} set mode "" } elseif {$mode == ""} { if {[regexp {^ } $line]} { set mode codeblock } else { set mode p } output "<$mode$margin>" set margin "" } output $line } if {$mode != ""} {output "</$mode>"} output "</dd>" } output "</dl>" } proc get_fts5_struct {data start end} { set res "" set bOut 0 foreach line [split $data "\n"] { if {$bOut==0} { |
︙ | ︙ |
Changes to ext/fts5/fts5Int.h.
︙ | ︙ | |||
32 33 34 35 36 37 38 39 40 41 42 43 44 45 | #define ArraySize(x) (sizeof(x) / sizeof(x[0])) #define testcase(x) #define ALWAYS(x) 1 #define NEVER(x) 0 #define MIN(x,y) (((x) < (y)) ? (x) : (y)) #endif /* ** Maximum number of prefix indexes on single FTS5 table. This must be ** less than 32. If it is set to anything large than that, an #error | > > > > > > > | 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 | #define ArraySize(x) (sizeof(x) / sizeof(x[0])) #define testcase(x) #define ALWAYS(x) 1 #define NEVER(x) 0 #define MIN(x,y) (((x) < (y)) ? (x) : (y)) #define MAX(x,y) (((x) > (y)) ? (x) : (y)) /* ** Constants for the largest and smallest possible 64-bit signed integers. */ # define LARGEST_INT64 (0xffffffff|(((i64)0x7fffffff)<<32)) # define SMALLEST_INT64 (((i64)-1) - LARGEST_INT64) #endif /* ** Maximum number of prefix indexes on single FTS5 table. This must be ** less than 32. If it is set to anything large than that, an #error |
︙ | ︙ |
Changes to ext/fts5/fts5_index.c.
︙ | ︙ | |||
290 291 292 293 294 295 296 | typedef struct Fts5BtreeIter Fts5BtreeIter; typedef struct Fts5BtreeIterLevel Fts5BtreeIterLevel; typedef struct Fts5Data Fts5Data; typedef struct Fts5DlidxIter Fts5DlidxIter; typedef struct Fts5DlidxLvl Fts5DlidxLvl; typedef struct Fts5DlidxWriter Fts5DlidxWriter; | < | 290 291 292 293 294 295 296 297 298 299 300 301 302 303 | typedef struct Fts5BtreeIter Fts5BtreeIter; typedef struct Fts5BtreeIterLevel Fts5BtreeIterLevel; typedef struct Fts5Data Fts5Data; typedef struct Fts5DlidxIter Fts5DlidxIter; typedef struct Fts5DlidxLvl Fts5DlidxLvl; typedef struct Fts5DlidxWriter Fts5DlidxWriter; typedef struct Fts5NodeIter Fts5NodeIter; typedef struct Fts5PageWriter Fts5PageWriter; typedef struct Fts5SegIter Fts5SegIter; typedef struct Fts5DoclistIter Fts5DoclistIter; typedef struct Fts5SegWriter Fts5SegWriter; typedef struct Fts5Structure Fts5Structure; typedef struct Fts5StructureLevel Fts5StructureLevel; |
︙ | ︙ | |||
344 345 346 347 348 349 350 | /* Output variables. aPoslist==0 at EOF */ i64 iRowid; u8 *aPoslist; int nPoslist; }; | < < < < < < < < < < > | 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 | /* Output variables. aPoslist==0 at EOF */ i64 iRowid; u8 *aPoslist; int nPoslist; }; /* ** The contents of the "structure" record for each index are represented ** using an Fts5Structure record in memory. Which uses instances of the ** other Fts5StructureXXX types as components. */ struct Fts5StructureSegment { int iSegid; /* Segment id */ int nHeight; /* Height of segment b-tree */ int pgnoFirst; /* First leaf page number in segment */ int pgnoLast; /* Last leaf page number in segment */ }; struct Fts5StructureLevel { int nMerge; /* Number of segments in incr-merge */ int nSeg; /* Total number of segments on level */ Fts5StructureSegment *aSeg; /* Array of segments. aSeg[0] is oldest. */ }; struct Fts5Structure { int nRef; /* Object reference count */ u64 nWriteCounter; /* Total leaves written to level 0 */ int nSegment; /* Total segments in this structure */ int nLevel; /* Number of levels in this index */ Fts5StructureLevel aLevel[0]; /* Array of nLevel level objects */ }; /* |
︙ | ︙ | |||
432 433 434 435 436 437 438 | typedef struct Fts5CResult Fts5CResult; struct Fts5CResult { u16 iFirst; /* aSeg[] index of firstest iterator */ u8 bTermEq; /* True if the terms are equal */ }; | < < < < < < < < | 422 423 424 425 426 427 428 429 430 431 432 433 434 435 | typedef struct Fts5CResult Fts5CResult; struct Fts5CResult { u16 iFirst; /* aSeg[] index of firstest iterator */ u8 bTermEq; /* True if the terms are equal */ }; /* ** Object for iterating through a single segment, visiting each term/docid ** pair in the segment. ** ** pSeg: ** The segment to iterate through. ** |
︙ | ︙ | |||
513 514 515 516 517 518 519 520 521 522 523 524 525 526 | int nPos; /* Number of bytes in current position list */ int bDel; /* True if the delete flag is set */ }; #define FTS5_SEGITER_ONETERM 0x01 #define FTS5_SEGITER_REVERSE 0x02 /* ** Object for iterating through the conents of a single internal node in ** memory. */ struct Fts5NodeIter { /* Internal. Set and managed by fts5NodeIterXXX() functions. Except, | > > > > > > > > > > > > > > > > > > > > > | 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 | int nPos; /* Number of bytes in current position list */ int bDel; /* True if the delete flag is set */ }; #define FTS5_SEGITER_ONETERM 0x01 #define FTS5_SEGITER_REVERSE 0x02 /* ** poslist: ** Used by sqlite3Fts5IterPoslist() when the poslist needs to be buffered. ** There is no way to tell if this is populated or not. */ struct Fts5IndexIter { Fts5Index *pIndex; /* Index that owns this iterator */ Fts5Structure *pStruct; /* Database structure for this iterator */ Fts5Buffer poslist; /* Buffer containing current poslist */ int nSeg; /* Size of aSeg[] array */ int bRev; /* True to iterate in reverse order */ int bSkipEmpty; /* True to skip deleted entries */ int bEof; /* True at EOF */ i64 iSwitchRowid; /* Firstest rowid of other than aFirst[1] */ Fts5CResult *aFirst; /* Current merge state (see above) */ Fts5SegIter aSeg[1]; /* Array of segment iterators */ }; /* ** Object for iterating through the conents of a single internal node in ** memory. */ struct Fts5NodeIter { /* Internal. Set and managed by fts5NodeIterXXX() functions. Except, |
︙ | ︙ | |||
599 600 601 602 603 604 605 606 607 608 609 610 611 612 | Fts5Buffer term; /* Current term */ int iLeaf; /* Leaf containing terms >= current term */ int nEmpty; /* Number of "empty" leaves following iLeaf */ int bEof; /* Set to true at EOF */ int bDlidx; /* True if there exists a dlidx */ }; static void fts5PutU16(u8 *aOut, u16 iVal){ aOut[0] = (iVal>>8); aOut[1] = (iVal&0xFF); } static u16 fts5GetU16(const u8 *aIn){ | > > > > > > > > > > > > > > > | 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 | Fts5Buffer term; /* Current term */ int iLeaf; /* Leaf containing terms >= current term */ int nEmpty; /* Number of "empty" leaves following iLeaf */ int bEof; /* Set to true at EOF */ int bDlidx; /* True if there exists a dlidx */ }; /* ** The first argument passed to this macro is a pointer to an Fts5Buffer ** object. */ #define fts5BufferSize(pBuf,n) { \ if( pBuf->nSpace<n ) { \ u8 *pNew = sqlite3_realloc(pBuf->p, n); \ if( pNew==0 ){ \ sqlite3_free(pBuf->p); \ } \ pBuf->nSpace = n; \ pBuf->p = pNew; \ } \ } static void fts5PutU16(u8 *aOut, u16 iVal){ aOut[0] = (iVal>>8); aOut[1] = (iVal&0xFF); } static u16 fts5GetU16(const u8 *aIn){ |
︙ | ︙ | |||
719 720 721 722 723 724 725 | ** backing store corruption. */ if( rc==SQLITE_ERROR ) rc = FTS5_CORRUPT; if( rc==SQLITE_OK ){ u8 *aOut; /* Read blob data into this buffer */ int nByte = sqlite3_blob_bytes(p->pReader); if( pBuf ){ | | | > | > | > > | 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 | ** backing store corruption. */ if( rc==SQLITE_ERROR ) rc = FTS5_CORRUPT; if( rc==SQLITE_OK ){ u8 *aOut; /* Read blob data into this buffer */ int nByte = sqlite3_blob_bytes(p->pReader); if( pBuf ){ fts5BufferSize(pBuf, MAX(nByte, p->pConfig->pgsz) + 20); pBuf->n = nByte; aOut = pBuf->p; if( aOut==0 ){ rc = SQLITE_NOMEM; } }else{ int nSpace = nByte + FTS5_DATA_ZERO_PADDING; pRet = (Fts5Data*)sqlite3_malloc(nSpace+sizeof(Fts5Data)); if( pRet ){ pRet->n = nByte; aOut = pRet->p = (u8*)&pRet[1]; }else{ rc = SQLITE_NOMEM; } } if( rc==SQLITE_OK ){ rc = sqlite3_blob_read(p->pReader, aOut, nByte, 0); } if( rc!=SQLITE_OK ){ |
︙ | ︙ | |||
853 854 855 856 857 858 859 | } /* ** Release a reference to an Fts5Structure object returned by an earlier ** call to fts5StructureRead() or fts5StructureDecode(). */ static void fts5StructureRelease(Fts5Structure *pStruct){ | | > > > > > | 875 876 877 878 879 880 881 882 883 884 885 886 887 888 889 890 891 892 893 894 895 896 897 898 899 900 901 | } /* ** Release a reference to an Fts5Structure object returned by an earlier ** call to fts5StructureRead() or fts5StructureDecode(). */ static void fts5StructureRelease(Fts5Structure *pStruct){ if( pStruct && 0>=(--pStruct->nRef) ){ int i; assert( pStruct->nRef==0 ); for(i=0; i<pStruct->nLevel; i++){ sqlite3_free(pStruct->aLevel[i].aSeg); } sqlite3_free(pStruct); } } static void fts5StructureRef(Fts5Structure *pStruct){ pStruct->nRef++; } /* ** Deserialize and return the structure record currently stored in serialized ** form within buffer pData/nData. ** ** The Fts5Structure.aLevel[] and each Fts5StructureLevel.aSeg[] array ** are over-allocated by one slot. This allows the structure contents |
︙ | ︙ | |||
903 904 905 906 907 908 909 910 911 912 913 914 915 916 | nByte = ( sizeof(Fts5Structure) + /* Main structure */ sizeof(Fts5StructureLevel) * (nLevel) /* aLevel[] array */ ); pRet = (Fts5Structure*)sqlite3Fts5MallocZero(&rc, nByte); if( pRet ){ pRet->nLevel = nLevel; pRet->nSegment = nSegment; i += sqlite3Fts5GetVarint(&pData[i], &pRet->nWriteCounter); for(iLvl=0; rc==SQLITE_OK && iLvl<nLevel; iLvl++){ Fts5StructureLevel *pLvl = &pRet->aLevel[iLvl]; int nTotal; | > | 930 931 932 933 934 935 936 937 938 939 940 941 942 943 944 | nByte = ( sizeof(Fts5Structure) + /* Main structure */ sizeof(Fts5StructureLevel) * (nLevel) /* aLevel[] array */ ); pRet = (Fts5Structure*)sqlite3Fts5MallocZero(&rc, nByte); if( pRet ){ pRet->nRef = 1; pRet->nLevel = nLevel; pRet->nSegment = nSegment; i += sqlite3Fts5GetVarint(&pData[i], &pRet->nWriteCounter); for(iLvl=0; rc==SQLITE_OK && iLvl<nLevel; iLvl++){ Fts5StructureLevel *pLvl = &pRet->aLevel[iLvl]; int nTotal; |
︙ | ︙ | |||
1008 1009 1010 1011 1012 1013 1014 | ** If an error occurs, NULL is returned and an error code left in the ** Fts5Index handle. If an error has already occurred when this function ** is called, it is a no-op. */ static Fts5Structure *fts5StructureRead(Fts5Index *p){ Fts5Config *pConfig = p->pConfig; Fts5Structure *pRet = 0; /* Object to return */ | < > | | > > | | | 1036 1037 1038 1039 1040 1041 1042 1043 1044 1045 1046 1047 1048 1049 1050 1051 1052 1053 1054 1055 1056 1057 1058 1059 1060 1061 1062 1063 | ** If an error occurs, NULL is returned and an error code left in the ** Fts5Index handle. If an error has already occurred when this function ** is called, it is a no-op. */ static Fts5Structure *fts5StructureRead(Fts5Index *p){ Fts5Config *pConfig = p->pConfig; Fts5Structure *pRet = 0; /* Object to return */ int iCookie; /* Configuration cookie */ Fts5Buffer buf = {0, 0, 0}; fts5DataBuffer(p, &buf, FTS5_STRUCTURE_ROWID); if( buf.p==0 ) return 0; assert( buf.nSpace>=(buf.n + FTS5_DATA_ZERO_PADDING) ); memset(&buf.p[buf.n], 0, FTS5_DATA_ZERO_PADDING); p->rc = fts5StructureDecode(buf.p, buf.n, &iCookie, &pRet); if( p->rc==SQLITE_OK && pConfig->iCookie!=iCookie ){ p->rc = sqlite3Fts5ConfigLoad(pConfig, iCookie); } fts5BufferFree(&buf); if( p->rc!=SQLITE_OK ){ fts5StructureRelease(pRet); pRet = 0; } return pRet; } |
︙ | ︙ | |||
1573 1574 1575 1576 1577 1578 1579 1580 1581 1582 1583 1584 1585 1586 | p->rc = FTS5_CORRUPT; }else{ const u8 *a = &pIter->pLeaf->p[iOff]; pIter->iLeafOffset += fts5GetPoslistSize(a, &pIter->nPos, &pIter->bDel); } } } /* ** Fts5SegIter.iLeafOffset currently points to the first byte of the ** "nSuffix" field of a term. Function parameter nKeep contains the value ** of the "nPrefix" field (if there was one - it is passed 0 if this is ** the first term in the segment). ** | > > > > > > > > > > > > > > > > > | 1603 1604 1605 1606 1607 1608 1609 1610 1611 1612 1613 1614 1615 1616 1617 1618 1619 1620 1621 1622 1623 1624 1625 1626 1627 1628 1629 1630 1631 1632 1633 | p->rc = FTS5_CORRUPT; }else{ const u8 *a = &pIter->pLeaf->p[iOff]; pIter->iLeafOffset += fts5GetPoslistSize(a, &pIter->nPos, &pIter->bDel); } } } static void fts5SegIterLoadRowid(Fts5Index *p, Fts5SegIter *pIter){ u8 *a = pIter->pLeaf->p; /* Buffer to read data from */ int iOff = pIter->iLeafOffset; if( iOff>=pIter->pLeaf->n ){ fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ){ if( p->rc==SQLITE_OK ) p->rc = FTS5_CORRUPT; return; } iOff = 4; a = pIter->pLeaf->p; } iOff += sqlite3Fts5GetVarint(&a[iOff], (u64*)&pIter->iRowid); pIter->iLeafOffset = iOff; } /* ** Fts5SegIter.iLeafOffset currently points to the first byte of the ** "nSuffix" field of a term. Function parameter nKeep contains the value ** of the "nPrefix" field (if there was one - it is passed 0 if this is ** the first term in the segment). ** |
︙ | ︙ | |||
1600 1601 1602 1603 1604 1605 1606 | iOff += fts5GetVarint32(&a[iOff], nNew); pIter->term.n = nKeep; fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]); iOff += nNew; pIter->iTermLeafOffset = iOff; pIter->iTermLeafPgno = pIter->iLeafPgno; | < < | < < | < | < < < | 1647 1648 1649 1650 1651 1652 1653 1654 1655 1656 1657 1658 1659 1660 1661 1662 1663 | iOff += fts5GetVarint32(&a[iOff], nNew); pIter->term.n = nKeep; fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]); iOff += nNew; pIter->iTermLeafOffset = iOff; pIter->iTermLeafPgno = pIter->iLeafPgno; pIter->iLeafOffset = iOff; fts5SegIterLoadRowid(p, pIter); } /* ** Initialize the iterator object pIter to iterate through the entries in ** segment pSeg. The iterator is left pointing to the first entry when ** this function returns. ** |
︙ | ︙ | |||
1752 1753 1754 1755 1756 1757 1758 | } /* ** Return true if the iterator passed as the second argument currently ** points to a delete marker. A delete marker is an entry with a 0 byte ** position-list. */ | | | 1791 1792 1793 1794 1795 1796 1797 1798 1799 1800 1801 1802 1803 1804 1805 | } /* ** Return true if the iterator passed as the second argument currently ** points to a delete marker. A delete marker is an entry with a 0 byte ** position-list. */ static int fts5MultiIterIsEmpty(Fts5Index *p, Fts5IndexIter *pIter){ Fts5SegIter *pSeg = &pIter->aSeg[pIter->aFirst[1].iFirst]; return (p->rc==SQLITE_OK && pSeg->pLeaf && pSeg->nPos==0); } /* ** Advance iterator pIter to the next entry. ** |
︙ | ︙ | |||
2005 2006 2007 2008 2009 2010 2011 2012 2013 2014 2015 2016 2017 2018 2019 2020 2021 2022 2023 2024 2025 2026 2027 2028 2029 2030 2031 2032 2033 2034 2035 2036 2037 2038 2039 2040 | iOff += fts5GetPoslistSize(&pLeaf->p[iOff], &nPos, &bDummy); iOff += nPos; } } pIter->pDlidx = fts5DlidxIterInit(p, bRev, iSeg, pIter->iTermLeafPgno); } /* ** Initialize the object pIter to point to term pTerm/nTerm within segment ** pSeg. If there is no such term in the index, the iterator is set to EOF. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If ** an error has already occurred when this function is called, it is a no-op. */ static void fts5SegIterSeekInit( Fts5Index *p, /* FTS5 backend */ const u8 *pTerm, int nTerm, /* Term to seek to */ int flags, /* Mask of FTS5INDEX_XXX flags */ Fts5StructureSegment *pSeg, /* Description of segment */ Fts5SegIter *pIter /* Object to populate */ ){ int iPg = 1; int h; int bGe = (flags & FTS5INDEX_QUERY_SCAN); int bDlidx = 0; /* True if there is a doclist-index */ Fts5Data *pLeaf; assert( bGe==0 || (flags & FTS5INDEX_QUERY_DESC)==0 ); assert( pTerm && nTerm ); memset(pIter, 0, sizeof(*pIter)); pIter->pSeg = pSeg; /* This block sets stack variable iPg to the leaf page number that may ** contain term (pTerm/nTerm), if it is present in the segment. */ for(h=pSeg->nHeight-1; h>0; h--){ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < | | | < < < < < < < < < < < < < < | < < < < < < < < < < < < | < < < < < < > > > > > > > > > > > > > > | 2044 2045 2046 2047 2048 2049 2050 2051 2052 2053 2054 2055 2056 2057 2058 2059 2060 2061 2062 2063 2064 2065 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 2088 2089 2090 2091 2092 2093 2094 2095 2096 2097 2098 2099 2100 2101 2102 2103 2104 2105 2106 2107 2108 2109 2110 2111 2112 2113 2114 2115 2116 2117 2118 2119 2120 2121 2122 2123 2124 2125 2126 2127 2128 2129 2130 2131 2132 2133 2134 2135 2136 2137 2138 2139 2140 2141 2142 2143 2144 2145 2146 2147 2148 2149 2150 2151 2152 2153 2154 2155 2156 2157 2158 2159 2160 2161 2162 2163 2164 2165 2166 2167 2168 2169 2170 2171 2172 2173 2174 2175 2176 2177 2178 2179 2180 2181 2182 2183 2184 2185 2186 2187 2188 2189 2190 2191 2192 2193 2194 2195 2196 2197 2198 2199 2200 2201 2202 2203 2204 2205 2206 2207 2208 2209 2210 2211 2212 2213 2214 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 2241 2242 2243 2244 2245 2246 2247 2248 2249 2250 2251 2252 2253 2254 2255 2256 2257 2258 2259 2260 2261 2262 2263 2264 2265 2266 2267 2268 2269 2270 2271 2272 2273 2274 2275 2276 2277 2278 2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 2307 2308 2309 2310 2311 2312 2313 2314 2315 2316 2317 2318 2319 2320 2321 2322 2323 2324 2325 2326 2327 2328 2329 2330 2331 2332 2333 2334 2335 2336 2337 2338 2339 2340 2341 2342 2343 2344 2345 2346 2347 2348 2349 2350 2351 2352 2353 2354 2355 2356 2357 2358 2359 2360 2361 2362 2363 2364 2365 2366 2367 2368 2369 2370 2371 2372 2373 2374 2375 2376 2377 2378 2379 2380 2381 2382 2383 2384 2385 2386 2387 2388 2389 2390 2391 2392 | iOff += fts5GetPoslistSize(&pLeaf->p[iOff], &nPos, &bDummy); iOff += nPos; } } pIter->pDlidx = fts5DlidxIterInit(p, bRev, iSeg, pIter->iTermLeafPgno); } #ifdef SQLITE_DEBUG static void fts5AssertNodeSeekOk( Fts5Buffer *pNode, const u8 *pTerm, int nTerm, /* Term to search for */ int iExpectPg, int bExpectDlidx ){ int bDlidx; int iPg; int rc = SQLITE_OK; Fts5NodeIter node; fts5NodeIterInit(pNode->p, pNode->n, &node); assert( node.term.n==0 ); iPg = node.iChild; bDlidx = node.bDlidx; for(fts5NodeIterNext(&rc, &node); node.aData && fts5BufferCompareBlob(&node.term, pTerm, nTerm)<=0; fts5NodeIterNext(&rc, &node) ){ iPg = node.iChild; bDlidx = node.bDlidx; } fts5NodeIterFree(&node); assert( rc!=SQLITE_OK || iPg==iExpectPg ); assert( rc!=SQLITE_OK || bDlidx==bExpectDlidx ); } #else #define fts5AssertNodeSeekOk(v,w,x,y,z) #endif /* ** Argument pNode is an internal b-tree node. This function searches ** within the node for the largest term that is smaller than or equal ** to (pTerm/nTerm). ** ** It returns the associated page number. Or, if (pTerm/nTerm) is smaller ** than all terms within the node, the leftmost child page number. ** ** Before returning, (*pbDlidx) is set to true if the last term on the ** returned child page number has a doclist-index. Or left as is otherwise. */ static int fts5NodeSeek( Fts5Buffer *pNode, /* Node to search */ const u8 *pTerm, int nTerm, /* Term to search for */ int *pbDlidx /* OUT: True if dlidx flag is set */ ){ int iPg; u8 *pPtr = pNode->p; u8 *pEnd = &pPtr[pNode->n]; int nMatch = 0; /* Number of bytes of pTerm already matched */ assert( *pbDlidx==0 ); pPtr += fts5GetVarint32(pPtr, iPg); while( pPtr<pEnd ){ int nEmpty = 0; int nKeep; int nNew; /* If there is a "no terms" record at pPtr, read it now. Store the ** number of termless pages in nEmpty. If it indicates a doclist-index, ** set (*pbDlidx) to true.*/ if( *pPtr<2 ){ *pbDlidx = (*pPtr==0x01); pPtr++; pPtr += fts5GetVarint32(pPtr, nEmpty); if( pPtr>=pEnd ) break; } /* Read the next "term" pointer. Set nKeep to the number of bytes to ** keep from the previous term, and nNew to the number of bytes of ** new data that will be appended to it. */ nKeep = (int)*pPtr++; nNew = (int)*pPtr++; if( (nKeep | nNew) & 0x0080 ){ pPtr -= 2; pPtr += fts5GetVarint32(pPtr, nKeep); pPtr += fts5GetVarint32(pPtr, nNew); } nKeep -= 2; /* Compare (pTerm/nTerm) to the current term on the node (the one described ** by nKeep/nNew). If the node term is larger, break out of the while() ** loop. ** ** Otherwise, if (pTerm/nTerm) is larger or the two terms are equal, ** leave variable nMatch set to the size of the largest prefix common to ** both terms in bytes. */ if( nKeep==nMatch ){ int nTst = MIN(nNew, nTerm-nMatch); int i; for(i=0; i<nTst; i++){ if( pTerm[nKeep+i]!=pPtr[i] ) break; } nMatch += i; assert( nMatch<=nTerm ); if( i<nNew && (nMatch==nTerm || pPtr[i] > pTerm[nMatch]) ) break; }else if( nKeep<nMatch ){ break; } iPg += 1 + nEmpty; *pbDlidx = 0; pPtr += nNew; } fts5AssertNodeSeekOk(pNode, pTerm, nTerm, iPg, *pbDlidx); return iPg; } #define fts5IndexGetVarint32(a, iOff, nVal) { \ nVal = a[iOff++]; \ if( nVal & 0x80 ){ \ iOff--; \ iOff += fts5GetVarint32(&a[iOff], nVal); \ } \ } #define fts5IndexSkipVarint(a, iOff) { \ int iEnd = iOff+9; \ while( (a[iOff++] & 0x80) && iOff<iEnd ); \ } /* ** The iterator object passed as the second argument currently contains ** no valid values except for the Fts5SegIter.pLeaf member variable. This ** function searches the leaf page for a term matching (pTerm/nTerm). ** ** If the specified term is found on the page, then the iterator is left ** pointing to it. If argument bGe is zero and the term is not found, ** the iterator is left pointing at EOF. ** ** If bGe is non-zero and the specified term is not found, then the ** iterator is left pointing to the smallest term in the segment that ** is larger than the specified term, even if this term is not on the ** current page. */ static void fts5LeafSeek( Fts5Index *p, /* Leave any error code here */ int bGe, /* True for a >= search */ Fts5SegIter *pIter, /* Iterator to seek */ const u8 *pTerm, int nTerm /* Term to search for */ ){ int iOff; const u8 *a = pIter->pLeaf->p; int n = pIter->pLeaf->n; int nMatch = 0; int nKeep = 0; int nNew = 0; assert( p->rc==SQLITE_OK ); assert( pIter->pLeaf ); iOff = fts5GetU16(&a[2]); if( iOff<4 || iOff>=n ){ p->rc = FTS5_CORRUPT; return; } while( 1 ){ int i; int nCmp; i64 rowid; /* Figure out how many new bytes are in this term */ fts5IndexGetVarint32(a, iOff, nNew); if( nKeep<nMatch ){ goto search_failed; } assert( nKeep>=nMatch ); if( nKeep==nMatch ){ nCmp = MIN(nNew, nTerm-nMatch); for(i=0; i<nCmp; i++){ if( a[iOff+i]!=pTerm[nMatch+i] ) break; } nMatch += i; if( nTerm==nMatch ){ if( i==nNew ){ goto search_success; }else{ goto search_failed; } }else if( i<nNew && a[iOff+i]>pTerm[nMatch] ){ goto search_failed; } } iOff += nNew; /* Skip past the doclist. If the end of the page is reached, bail out. */ while( 1 ){ int nPos; /* Skip past docid delta */ fts5IndexSkipVarint(a, iOff); /* Skip past position list */ fts5IndexGetVarint32(a, iOff, nPos); iOff += (nPos >> 1); if( iOff>=(n-1) ){ iOff = n; goto search_failed; } /* If this is the end of the doclist, break out of the loop */ if( a[iOff]==0x00 ){ iOff++; break; } }; /* Read the nKeep field of the next term. */ fts5IndexGetVarint32(a, iOff, nKeep); } search_failed: if( bGe==0 ){ fts5DataRelease(pIter->pLeaf); pIter->pLeaf = 0; return; }else if( iOff>=n ){ do { fts5SegIterNextPage(p, pIter); if( pIter->pLeaf==0 ) return; a = pIter->pLeaf->p; iOff = fts5GetU16(&a[2]); if( iOff ){ if( iOff<4 || iOff>=n ){ p->rc = FTS5_CORRUPT; }else{ nKeep = 0; iOff += fts5GetVarint32(&a[iOff], nNew); break; } } }while( 1 ); } search_success: pIter->iLeafOffset = iOff + nNew; pIter->iTermLeafOffset = pIter->iLeafOffset; pIter->iTermLeafPgno = pIter->iLeafPgno; fts5BufferSet(&p->rc, &pIter->term, nKeep, pTerm); fts5BufferAppendBlob(&p->rc, &pIter->term, nNew, &a[iOff]); fts5SegIterLoadRowid(p, pIter); fts5SegIterLoadNPos(p, pIter); } /* ** Initialize the object pIter to point to term pTerm/nTerm within segment ** pSeg. If there is no such term in the index, the iterator is set to EOF. ** ** If an error occurs, Fts5Index.rc is set to an appropriate error code. If ** an error has already occurred when this function is called, it is a no-op. */ static void fts5SegIterSeekInit( Fts5Index *p, /* FTS5 backend */ Fts5Buffer *pBuf, /* Buffer to use for loading pages */ const u8 *pTerm, int nTerm, /* Term to seek to */ int flags, /* Mask of FTS5INDEX_XXX flags */ Fts5StructureSegment *pSeg, /* Description of segment */ Fts5SegIter *pIter /* Object to populate */ ){ int iPg = 1; int h; int bGe = (flags & FTS5INDEX_QUERY_SCAN); int bDlidx = 0; /* True if there is a doclist-index */ Fts5Data *pLeaf; static int nCall = 0; nCall++; assert( bGe==0 || (flags & FTS5INDEX_QUERY_DESC)==0 ); assert( pTerm && nTerm ); memset(pIter, 0, sizeof(*pIter)); pIter->pSeg = pSeg; /* This block sets stack variable iPg to the leaf page number that may ** contain term (pTerm/nTerm), if it is present in the segment. */ for(h=pSeg->nHeight-1; h>0; h--){ i64 iRowid = FTS5_SEGMENT_ROWID(pSeg->iSegid, h, iPg); fts5DataBuffer(p, pBuf, iRowid); if( p->rc ) break; iPg = fts5NodeSeek(pBuf, pTerm, nTerm, &bDlidx); } if( iPg<pSeg->pgnoFirst ){ iPg = pSeg->pgnoFirst; bDlidx = 0; } pIter->iLeafPgno = iPg - 1; fts5SegIterNextPage(p, pIter); if( pIter->pLeaf ){ fts5LeafSeek(p, bGe, pIter, pTerm, nTerm); } if( p->rc==SQLITE_OK && bGe==0 ){ pIter->flags |= FTS5_SEGITER_ONETERM; if( pIter->pLeaf ){ if( flags & FTS5INDEX_QUERY_DESC ){ pIter->flags |= FTS5_SEGITER_REVERSE; } if( bDlidx ){ fts5SegIterLoadDlidx(p, pIter); } if( flags & FTS5INDEX_QUERY_DESC ){ fts5SegIterReverse(p, pIter); } } } /* Either: ** ** 1) an error has occurred, or ** 2) the iterator points to EOF, or ** 3) the iterator points to an entry with term (pTerm/nTerm), or ** 4) the FTS5INDEX_QUERY_SCAN flag was set and the iterator points ** to an entry with a term greater than or equal to (pTerm/nTerm). */ assert( p->rc!=SQLITE_OK /* 1 */ || pIter->pLeaf==0 /* 2 */ || fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)==0 /* 3 */ || (bGe && fts5BufferCompareBlob(&pIter->term, pTerm, nTerm)>0) /* 4 */ ); } /* ** Initialize the object pIter to point to term pTerm/nTerm within the ** in-memory hash table. If there is no such term in the hash-table, the ** iterator is set to EOF. ** |
︙ | ︙ | |||
2174 2175 2176 2177 2178 2179 2180 | /* ** This function is used as part of the big assert() procedure implemented by ** fts5AssertMultiIterSetup(). It ensures that the result currently stored ** in *pRes is the correct result of comparing the current positions of the ** two iterators. */ static void fts5AssertComparisonResult( | | | 2454 2455 2456 2457 2458 2459 2460 2461 2462 2463 2464 2465 2466 2467 2468 | /* ** This function is used as part of the big assert() procedure implemented by ** fts5AssertMultiIterSetup(). It ensures that the result currently stored ** in *pRes is the correct result of comparing the current positions of the ** two iterators. */ static void fts5AssertComparisonResult( Fts5IndexIter *pIter, Fts5SegIter *p1, Fts5SegIter *p2, Fts5CResult *pRes ){ int i1 = p1 - pIter->aSeg; int i2 = p2 - pIter->aSeg; |
︙ | ︙ | |||
2215 2216 2217 2218 2219 2220 2221 | /* ** This function is a no-op unless SQLITE_DEBUG is defined when this module ** is compiled. In that case, this function is essentially an assert() ** statement used to verify that the contents of the pIter->aFirst[] array ** are correct. */ | | > > > > > > > > > > > > > > > < | | | 2495 2496 2497 2498 2499 2500 2501 2502 2503 2504 2505 2506 2507 2508 2509 2510 2511 2512 2513 2514 2515 2516 2517 2518 2519 2520 2521 2522 2523 2524 2525 2526 2527 2528 2529 2530 2531 2532 2533 2534 2535 2536 2537 2538 2539 2540 2541 2542 2543 2544 2545 2546 2547 2548 2549 2550 2551 2552 2553 2554 | /* ** This function is a no-op unless SQLITE_DEBUG is defined when this module ** is compiled. In that case, this function is essentially an assert() ** statement used to verify that the contents of the pIter->aFirst[] array ** are correct. */ static void fts5AssertMultiIterSetup(Fts5Index *p, Fts5IndexIter *pIter){ if( p->rc==SQLITE_OK ){ Fts5SegIter *pFirst = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; int i; assert( (pFirst->pLeaf==0)==pIter->bEof ); /* Check that pIter->iSwitchRowid is set correctly. */ for(i=0; i<pIter->nSeg; i++){ Fts5SegIter *p1 = &pIter->aSeg[i]; assert( p1==pFirst || p1->pLeaf==0 || fts5BufferCompare(&pFirst->term, &p1->term) || p1->iRowid==pIter->iSwitchRowid || (p1->iRowid<pIter->iSwitchRowid)==pIter->bRev ); } for(i=0; i<pIter->nSeg; i+=2){ Fts5SegIter *p1 = &pIter->aSeg[i]; Fts5SegIter *p2 = &pIter->aSeg[i+1]; Fts5CResult *pRes = &pIter->aFirst[(pIter->nSeg + i) / 2]; fts5AssertComparisonResult(pIter, p1, p2, pRes); } for(i=1; i<(pIter->nSeg / 2); i+=2){ Fts5SegIter *p1 = &pIter->aSeg[ pIter->aFirst[i*2].iFirst ]; Fts5SegIter *p2 = &pIter->aSeg[ pIter->aFirst[i*2+1].iFirst ]; Fts5CResult *pRes = &pIter->aFirst[i]; fts5AssertComparisonResult(pIter, p1, p2, pRes); } } } #else # define fts5AssertMultiIterSetup(x,y) #endif /* ** Do the comparison necessary to populate pIter->aFirst[iOut]. ** ** If the returned value is non-zero, then it is the index of an entry ** in the pIter->aSeg[] array that is (a) not at EOF, and (b) pointing ** to a key that is a duplicate of another, higher priority, ** segment-iterator in the pSeg->aSeg[] array. */ static int fts5MultiIterDoCompare(Fts5IndexIter *pIter, int iOut){ int i1; /* Index of left-hand Fts5SegIter */ int i2; /* Index of right-hand Fts5SegIter */ int iRes; Fts5SegIter *p1; /* Left-hand Fts5SegIter */ Fts5SegIter *p2; /* Right-hand Fts5SegIter */ Fts5CResult *pRes = &pIter->aFirst[iOut]; |
︙ | ︙ | |||
2392 2393 2394 2395 2396 2397 2398 | } } /* ** Free the iterator object passed as the second argument. */ | | > > | > > > > > > > > > > | < > > > > > | | | | | | | | | | | > | > > | | | | | > > > > > > > > > > | > | | | | < > | | > | > > | | 2686 2687 2688 2689 2690 2691 2692 2693 2694 2695 2696 2697 2698 2699 2700 2701 2702 2703 2704 2705 2706 2707 2708 2709 2710 2711 2712 2713 2714 2715 2716 2717 2718 2719 2720 2721 2722 2723 2724 2725 2726 2727 2728 2729 2730 2731 2732 2733 2734 2735 2736 2737 2738 2739 2740 2741 2742 2743 2744 2745 2746 2747 2748 2749 2750 2751 2752 2753 2754 2755 2756 2757 2758 2759 2760 2761 2762 2763 2764 2765 2766 2767 2768 2769 2770 2771 2772 2773 2774 2775 2776 2777 2778 2779 2780 2781 2782 2783 2784 2785 2786 2787 2788 2789 2790 2791 2792 2793 2794 2795 2796 2797 2798 2799 2800 2801 2802 2803 2804 2805 2806 2807 2808 2809 2810 2811 2812 2813 2814 2815 2816 2817 2818 2819 2820 2821 2822 2823 2824 2825 2826 2827 2828 2829 2830 2831 2832 2833 2834 2835 2836 2837 2838 2839 2840 2841 2842 2843 2844 2845 2846 2847 2848 2849 2850 2851 2852 2853 2854 2855 2856 2857 2858 2859 2860 2861 2862 2863 2864 2865 2866 2867 2868 2869 2870 2871 2872 2873 2874 2875 2876 2877 2878 2879 2880 2881 2882 2883 2884 2885 2886 2887 2888 2889 2890 2891 2892 2893 2894 2895 2896 2897 2898 2899 2900 2901 2902 2903 2904 2905 2906 2907 2908 | } } /* ** Free the iterator object passed as the second argument. */ static void fts5MultiIterFree(Fts5Index *p, Fts5IndexIter *pIter){ if( pIter ){ int i; for(i=0; i<pIter->nSeg; i++){ fts5SegIterClear(&pIter->aSeg[i]); } fts5StructureRelease(pIter->pStruct); fts5BufferFree(&pIter->poslist); sqlite3_free(pIter); } } static void fts5MultiIterAdvanced( Fts5Index *p, /* FTS5 backend to iterate within */ Fts5IndexIter *pIter, /* Iterator to update aFirst[] array for */ int iChanged, /* Index of sub-iterator just advanced */ int iMinset /* Minimum entry in aFirst[] to set */ ){ int i; for(i=(pIter->nSeg+iChanged)/2; i>=iMinset && p->rc==SQLITE_OK; i=i/2){ int iEq; if( (iEq = fts5MultiIterDoCompare(pIter, i)) ){ fts5SegIterNext(p, &pIter->aSeg[iEq], 0); i = pIter->nSeg + iEq; } } } /* ** Sub-iterator iChanged of iterator pIter has just been advanced. It still ** points to the same term though - just a different rowid. This function ** attempts to update the contents of the pIter->aFirst[] accordingly. ** If it does so successfully, 0 is returned. Otherwise 1. ** ** If non-zero is returned, the caller should call fts5MultiIterAdvanced() ** on the iterator instead. That function does the same as this one, except ** that it deals with more complicated cases as well. */ static int fts5MultiIterAdvanceRowid( Fts5Index *p, /* FTS5 backend to iterate within */ Fts5IndexIter *pIter, /* Iterator to update aFirst[] array for */ int iChanged /* Index of sub-iterator just advanced */ ){ Fts5SegIter *pNew = &pIter->aSeg[iChanged]; if( pNew->iRowid==pIter->iSwitchRowid || (pNew->iRowid<pIter->iSwitchRowid)==pIter->bRev ){ int i; Fts5SegIter *pOther = &pIter->aSeg[iChanged ^ 0x0001]; pIter->iSwitchRowid = pIter->bRev ? SMALLEST_INT64 : LARGEST_INT64; for(i=(pIter->nSeg+iChanged)/2; 1; i=i/2){ Fts5CResult *pRes = &pIter->aFirst[i]; assert( pNew->pLeaf ); assert( pRes->bTermEq==0 || pOther->pLeaf ); if( pRes->bTermEq ){ if( pNew->iRowid==pOther->iRowid ){ return 1; }else if( (pOther->iRowid>pNew->iRowid)==pIter->bRev ){ pIter->iSwitchRowid = pOther->iRowid; pNew = pOther; }else if( (pOther->iRowid>pIter->iSwitchRowid)==pIter->bRev ){ pIter->iSwitchRowid = pOther->iRowid; } } pRes->iFirst = (pNew - pIter->aSeg); if( i==1 ) break; pOther = &pIter->aSeg[ pIter->aFirst[i ^ 0x0001].iFirst ]; } } return 0; } /* ** Set the pIter->bEof variable based on the state of the sub-iterators. */ static void fts5MultiIterSetEof(Fts5IndexIter *pIter){ Fts5SegIter *pSeg = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; pIter->bEof = pSeg->pLeaf==0; pIter->iSwitchRowid = pSeg->iRowid; } /* ** Move the iterator to the next entry. ** ** If an error occurs, an error code is left in Fts5Index.rc. It is not ** considered an error if the iterator reaches EOF, or if it is already at ** EOF when this function is called. */ static void fts5MultiIterNext( Fts5Index *p, Fts5IndexIter *pIter, int bFrom, /* True if argument iFrom is valid */ i64 iFrom /* Advance at least as far as this */ ){ if( p->rc==SQLITE_OK ){ int bUseFrom = bFrom; do { int iFirst = pIter->aFirst[1].iFirst; int bNewTerm = 0; Fts5SegIter *pSeg = &pIter->aSeg[iFirst]; assert( p->rc==SQLITE_OK ); if( bUseFrom && pSeg->pDlidx ){ fts5SegIterNextFrom(p, pSeg, iFrom); }else{ fts5SegIterNext(p, pSeg, &bNewTerm); } if( pSeg->pLeaf==0 || bNewTerm || fts5MultiIterAdvanceRowid(p, pIter, iFirst) ){ fts5MultiIterAdvanced(p, pIter, iFirst, 1); fts5MultiIterSetEof(pIter); } fts5AssertMultiIterSetup(p, pIter); bUseFrom = 0; }while( pIter->bSkipEmpty && fts5MultiIterIsEmpty(p, pIter) ); } } static Fts5IndexIter *fts5MultiIterAlloc( Fts5Index *p, /* FTS5 backend to iterate within */ int nSeg ){ Fts5IndexIter *pNew; int nSlot; /* Power of two >= nSeg */ for(nSlot=2; nSlot<nSeg; nSlot=nSlot*2); pNew = fts5IdxMalloc(p, sizeof(Fts5IndexIter) + /* pNew */ sizeof(Fts5SegIter) * (nSlot-1) + /* pNew->aSeg[] */ sizeof(Fts5CResult) * nSlot /* pNew->aFirst[] */ ); if( pNew ){ pNew->nSeg = nSlot; pNew->aFirst = (Fts5CResult*)&pNew->aSeg[nSlot]; pNew->pIndex = p; } return pNew; } /* ** Allocate a new Fts5IndexIter object. ** ** The new object will be used to iterate through data in structure pStruct. ** If iLevel is -ve, then all data in all segments is merged. Or, if iLevel ** is zero or greater, data from the first nSegment segments on level iLevel ** is merged. ** ** The iterator initially points to the first term/rowid entry in the ** iterated data. */ static void fts5MultiIterNew( Fts5Index *p, /* FTS5 backend to iterate within */ Fts5Structure *pStruct, /* Structure of specific index */ int bSkipEmpty, /* True to ignore delete-keys */ int flags, /* FTS5INDEX_QUERY_XXX flags */ const u8 *pTerm, int nTerm, /* Term to seek to (or NULL/0) */ int iLevel, /* Level to iterate (-1 for all) */ int nSegment, /* Number of segments to merge (iLevel>=0) */ Fts5IndexIter **ppOut /* New object */ ){ int nSeg = 0; /* Number of segment-iters in use */ int iIter = 0; /* */ int iSeg; /* Used to iterate through segments */ Fts5Buffer buf = {0,0,0}; /* Buffer used by fts5SegIterSeekInit() */ Fts5StructureLevel *pLvl; Fts5IndexIter *pNew; assert( (pTerm==0 && nTerm==0) || iLevel<0 ); /* Allocate space for the new multi-seg-iterator. */ if( p->rc==SQLITE_OK ){ if( iLevel<0 ){ assert( pStruct->nSegment==fts5StructureCountSegments(pStruct) ); nSeg = pStruct->nSegment; nSeg += (p->pHash ? 1 : 0); }else{ nSeg = MIN(pStruct->aLevel[iLevel].nSeg, nSegment); } } *ppOut = pNew = fts5MultiIterAlloc(p, nSeg); if( pNew==0 ) return; pNew->bRev = (0!=(flags & FTS5INDEX_QUERY_DESC)); pNew->bSkipEmpty = bSkipEmpty; pNew->pStruct = pStruct; fts5StructureRef(pStruct); /* Initialize each of the component segment iterators. */ if( iLevel<0 ){ Fts5StructureLevel *pEnd = &pStruct->aLevel[pStruct->nLevel]; if( p->pHash ){ /* Add a segment iterator for the current contents of the hash table. */ Fts5SegIter *pIter = &pNew->aSeg[iIter++]; fts5SegIterHashInit(p, pTerm, nTerm, flags, pIter); } for(pLvl=&pStruct->aLevel[0]; pLvl<pEnd; pLvl++){ for(iSeg=pLvl->nSeg-1; iSeg>=0; iSeg--){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; Fts5SegIter *pIter = &pNew->aSeg[iIter++]; if( pTerm==0 ){ fts5SegIterInit(p, pSeg, pIter); }else{ fts5SegIterSeekInit(p, &buf, pTerm, nTerm, flags, pSeg, pIter); } } } }else{ pLvl = &pStruct->aLevel[iLevel]; for(iSeg=nSeg-1; iSeg>=0; iSeg--){ fts5SegIterInit(p, &pLvl->aSeg[iSeg], &pNew->aSeg[iIter++]); |
︙ | ︙ | |||
2591 2592 2593 2594 2595 2596 2597 2598 2599 2600 2601 2602 2603 2604 2605 2606 2607 2608 2609 | for(iIter=pNew->nSeg-1; iIter>0; iIter--){ int iEq; if( (iEq = fts5MultiIterDoCompare(pNew, iIter)) ){ fts5SegIterNext(p, &pNew->aSeg[iEq], 0); fts5MultiIterAdvanced(p, pNew, iEq, iIter); } } fts5AssertMultiIterSetup(p, pNew); if( pNew->bSkipEmpty && fts5MultiIterIsEmpty(p, pNew) ){ fts5MultiIterNext(p, pNew, 0, 0); } }else{ fts5MultiIterFree(p, pNew); *ppOut = 0; } } /* | > > | | | > > | > > > | | | | | 2918 2919 2920 2921 2922 2923 2924 2925 2926 2927 2928 2929 2930 2931 2932 2933 2934 2935 2936 2937 2938 2939 2940 2941 2942 2943 2944 2945 2946 2947 2948 2949 2950 2951 2952 2953 2954 2955 2956 2957 2958 2959 2960 2961 2962 2963 2964 2965 2966 2967 2968 2969 2970 2971 2972 2973 2974 2975 2976 2977 2978 2979 2980 2981 2982 2983 2984 2985 2986 2987 2988 2989 2990 2991 2992 2993 2994 2995 2996 2997 2998 2999 3000 3001 3002 3003 3004 3005 3006 3007 3008 3009 3010 3011 3012 3013 3014 3015 3016 3017 3018 3019 3020 3021 3022 3023 3024 3025 3026 | for(iIter=pNew->nSeg-1; iIter>0; iIter--){ int iEq; if( (iEq = fts5MultiIterDoCompare(pNew, iIter)) ){ fts5SegIterNext(p, &pNew->aSeg[iEq], 0); fts5MultiIterAdvanced(p, pNew, iEq, iIter); } } fts5MultiIterSetEof(pNew); fts5AssertMultiIterSetup(p, pNew); if( pNew->bSkipEmpty && fts5MultiIterIsEmpty(p, pNew) ){ fts5MultiIterNext(p, pNew, 0, 0); } }else{ fts5MultiIterFree(p, pNew); *ppOut = 0; } fts5BufferFree(&buf); } /* ** Create an Fts5IndexIter that iterates through the doclist provided ** as the second argument. */ static void fts5MultiIterNew2( Fts5Index *p, /* FTS5 backend to iterate within */ Fts5Data *pData, /* Doclist to iterate through */ int bDesc, /* True for descending rowid order */ Fts5IndexIter **ppOut /* New object */ ){ Fts5IndexIter *pNew; pNew = fts5MultiIterAlloc(p, 2); if( pNew ){ Fts5SegIter *pIter = &pNew->aSeg[1]; pIter->flags = FTS5_SEGITER_ONETERM; if( pData->n>0 ){ pIter->pLeaf = pData; pIter->iLeafOffset = fts5GetVarint(pData->p, (u64*)&pIter->iRowid); pNew->aFirst[1].iFirst = 1; if( bDesc ){ pNew->bRev = 1; pIter->flags |= FTS5_SEGITER_REVERSE; fts5SegIterReverseInitPage(p, pIter); }else{ fts5SegIterLoadNPos(p, pIter); } pData = 0; }else{ pNew->bEof = 1; } *ppOut = pNew; } fts5DataRelease(pData); } /* ** Return true if the iterator is at EOF or if an error has occurred. ** False otherwise. */ static int fts5MultiIterEof(Fts5Index *p, Fts5IndexIter *pIter){ assert( p->rc || (pIter->aSeg[ pIter->aFirst[1].iFirst ].pLeaf==0)==pIter->bEof ); return (p->rc || pIter->bEof); } /* ** Return the rowid of the entry that the iterator currently points ** to. If the iterator points to EOF when this function is called the ** results are undefined. */ static i64 fts5MultiIterRowid(Fts5IndexIter *pIter){ assert( pIter->aSeg[ pIter->aFirst[1].iFirst ].pLeaf ); return pIter->aSeg[ pIter->aFirst[1].iFirst ].iRowid; } /* ** Move the iterator to the next entry at or following iMatch. */ static void fts5MultiIterNextFrom( Fts5Index *p, Fts5IndexIter *pIter, i64 iMatch ){ while( 1 ){ i64 iRowid; fts5MultiIterNext(p, pIter, 1, iMatch); if( fts5MultiIterEof(p, pIter) ) break; iRowid = fts5MultiIterRowid(pIter); if( pIter->bRev==0 && iRowid>=iMatch ) break; if( pIter->bRev!=0 && iRowid<=iMatch ) break; } } /* ** Return a pointer to a buffer containing the term associated with the ** entry that the iterator currently points to. */ static const u8 *fts5MultiIterTerm(Fts5IndexIter *pIter, int *pn){ Fts5SegIter *p = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; *pn = p->term.n; return p->term.p; } static void fts5ChunkIterate( Fts5Index *p, /* Index object */ |
︙ | ︙ | |||
3306 3307 3308 3309 3310 3311 3312 | } /* ** Iterator pIter was used to iterate through the input segments of on an ** incremental merge operation. This function is called if the incremental ** merge step has finished but the input has not been completely exhausted. */ | | | 3640 3641 3642 3643 3644 3645 3646 3647 3648 3649 3650 3651 3652 3653 3654 | } /* ** Iterator pIter was used to iterate through the input segments of on an ** incremental merge operation. This function is called if the incremental ** merge step has finished but the input has not been completely exhausted. */ static void fts5TrimSegments(Fts5Index *p, Fts5IndexIter *pIter){ int i; Fts5Buffer buf; memset(&buf, 0, sizeof(Fts5Buffer)); for(i=0; i<pIter->nSeg; i++){ Fts5SegIter *pSeg = &pIter->aSeg[i]; if( pSeg->pSeg==0 ){ /* no-op */ |
︙ | ︙ | |||
3366 3367 3368 3369 3370 3371 3372 | Fts5Structure **ppStruct, /* IN/OUT: Stucture of index */ int iLvl, /* Level to read input from */ int *pnRem /* Write up to this many output leaves */ ){ Fts5Structure *pStruct = *ppStruct; Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl]; Fts5StructureLevel *pLvlOut; | | | 3700 3701 3702 3703 3704 3705 3706 3707 3708 3709 3710 3711 3712 3713 3714 | Fts5Structure **ppStruct, /* IN/OUT: Stucture of index */ int iLvl, /* Level to read input from */ int *pnRem /* Write up to this many output leaves */ ){ Fts5Structure *pStruct = *ppStruct; Fts5StructureLevel *pLvl = &pStruct->aLevel[iLvl]; Fts5StructureLevel *pLvlOut; Fts5IndexIter *pIter = 0; /* Iterator to read input data */ int nRem = pnRem ? *pnRem : 0; /* Output leaf pages left to write */ int nInput; /* Number of input segments */ Fts5SegWriter writer; /* Writer object */ Fts5StructureSegment *pSeg; /* Output segment */ Fts5Buffer term; int bRequireDoclistTerm = 0; /* Doclist terminator (0x00) required */ int bOldest; /* True if the output segment is the oldest */ |
︙ | ︙ | |||
3840 3841 3842 3843 3844 3845 3846 3847 3848 3849 3850 3851 3852 3853 | pNew = (Fts5Structure*)sqlite3Fts5MallocZero(&p->rc, nByte); } } if( pNew ){ Fts5StructureLevel *pLvl; int nByte = nSeg * sizeof(Fts5StructureSegment); pNew->nLevel = pStruct->nLevel+1; pNew->nWriteCounter = pStruct->nWriteCounter; pLvl = &pNew->aLevel[pStruct->nLevel]; pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(&p->rc, nByte); if( pLvl->aSeg ){ int iLvl, iSeg; int iSegOut = 0; for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ | > | 4174 4175 4176 4177 4178 4179 4180 4181 4182 4183 4184 4185 4186 4187 4188 | pNew = (Fts5Structure*)sqlite3Fts5MallocZero(&p->rc, nByte); } } if( pNew ){ Fts5StructureLevel *pLvl; int nByte = nSeg * sizeof(Fts5StructureSegment); pNew->nLevel = pStruct->nLevel+1; pNew->nRef = 1; pNew->nWriteCounter = pStruct->nWriteCounter; pLvl = &pNew->aLevel[pStruct->nLevel]; pLvl->aSeg = (Fts5StructureSegment*)sqlite3Fts5MallocZero(&p->rc, nByte); if( pLvl->aSeg ){ int iLvl, iSeg; int iSegOut = 0; for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){ |
︙ | ︙ | |||
3919 3920 3921 3922 3923 3924 3925 | ** currently points to to buffer pBuf. ** ** If an error occurs, an error code is left in p->rc. It is assumed ** no error has already occurred when this function is called. */ static void fts5MultiIterPoslist( Fts5Index *p, | | | 4254 4255 4256 4257 4258 4259 4260 4261 4262 4263 4264 4265 4266 4267 4268 | ** currently points to to buffer pBuf. ** ** If an error occurs, an error code is left in p->rc. It is assumed ** no error has already occurred when this function is called. */ static void fts5MultiIterPoslist( Fts5Index *p, Fts5IndexIter *pMulti, int bSz, /* Append a size field before the data */ Fts5Buffer *pBuf ){ if( p->rc==SQLITE_OK ){ Fts5SegIter *pSeg = &pMulti->aSeg[ pMulti->aFirst[1].iFirst ]; assert( fts5MultiIterEof(p, pMulti)==0 ); |
︙ | ︙ | |||
4073 4074 4075 4076 4077 4078 4079 | } static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ const u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ | | | | 4408 4409 4410 4411 4412 4413 4414 4415 4416 4417 4418 4419 4420 4421 4422 4423 4424 4425 4426 4427 4428 4429 4430 4431 4432 4433 4434 4435 | } static void fts5SetupPrefixIter( Fts5Index *p, /* Index to read from */ int bDesc, /* True for "ORDER BY rowid DESC" */ const u8 *pToken, /* Buffer containing prefix to match */ int nToken, /* Size of buffer pToken in bytes */ Fts5IndexIter **ppIter /* OUT: New iterator */ ){ Fts5Structure *pStruct; Fts5Buffer *aBuf; const int nBuf = 32; aBuf = (Fts5Buffer*)fts5IdxMalloc(p, sizeof(Fts5Buffer)*nBuf); pStruct = fts5StructureRead(p); if( aBuf && pStruct ){ const int flags = FTS5INDEX_QUERY_SCAN; int i; i64 iLastRowid = 0; Fts5IndexIter *p1 = 0; /* Iterator used to gather data from index */ Fts5Data *pData; Fts5Buffer doclist; memset(&doclist, 0, sizeof(doclist)); for(fts5MultiIterNew(p, pStruct, 1, flags, pToken, nToken, -1, 0, &p1); fts5MultiIterEof(p, p1)==0; fts5MultiIterNext(p, p1, 0, 0) |
︙ | ︙ | |||
4129 4130 4131 4132 4133 4134 4135 | fts5MultiIterFree(p, p1); pData = fts5IdxMalloc(p, sizeof(Fts5Data) + doclist.n); if( pData ){ pData->p = (u8*)&pData[1]; pData->n = doclist.n; memcpy(pData->p, doclist.p, doclist.n); | | | 4464 4465 4466 4467 4468 4469 4470 4471 4472 4473 4474 4475 4476 4477 4478 | fts5MultiIterFree(p, p1); pData = fts5IdxMalloc(p, sizeof(Fts5Data) + doclist.n); if( pData ){ pData->p = (u8*)&pData[1]; pData->n = doclist.n; memcpy(pData->p, doclist.p, doclist.n); fts5MultiIterNew2(p, pData, bDesc, ppIter); } fts5BufferFree(&doclist); } fts5StructureRelease(pStruct); sqlite3_free(aBuf); } |
︙ | ︙ | |||
4339 4340 4341 4342 4343 4344 4345 | int sqlite3Fts5IndexQuery( Fts5Index *p, /* FTS index to query */ const char *pToken, int nToken, /* Token (or prefix) to query for */ int flags, /* Mask of FTS5INDEX_QUERY_X flags */ Fts5IndexIter **ppIter /* OUT: New iterator object */ ){ Fts5Config *pConfig = p->pConfig; | | | < | | | | | | | | | | < < < > < | < | | | | < | | | > | | | > | | < < | | > | | | < | < | > | < < | < | 4674 4675 4676 4677 4678 4679 4680 4681 4682 4683 4684 4685 4686 4687 4688 4689 4690 4691 4692 4693 4694 4695 4696 4697 4698 4699 4700 4701 4702 4703 4704 4705 4706 4707 4708 4709 4710 4711 4712 4713 4714 4715 4716 4717 4718 4719 4720 4721 4722 4723 4724 4725 4726 4727 4728 4729 4730 4731 4732 4733 4734 4735 4736 4737 4738 4739 4740 4741 4742 4743 4744 4745 4746 4747 4748 4749 4750 4751 4752 4753 4754 4755 4756 4757 4758 4759 4760 4761 4762 4763 4764 4765 4766 4767 4768 4769 4770 4771 4772 4773 4774 4775 4776 4777 4778 4779 4780 4781 4782 4783 4784 4785 4786 4787 4788 4789 4790 4791 4792 4793 4794 4795 4796 4797 4798 4799 4800 4801 4802 4803 4804 4805 4806 4807 4808 4809 4810 4811 4812 4813 4814 4815 4816 4817 4818 4819 4820 4821 4822 4823 4824 4825 4826 4827 4828 4829 4830 4831 4832 4833 4834 4835 4836 4837 4838 4839 4840 4841 4842 4843 4844 4845 4846 4847 4848 4849 4850 4851 4852 | int sqlite3Fts5IndexQuery( Fts5Index *p, /* FTS index to query */ const char *pToken, int nToken, /* Token (or prefix) to query for */ int flags, /* Mask of FTS5INDEX_QUERY_X flags */ Fts5IndexIter **ppIter /* OUT: New iterator object */ ){ Fts5Config *pConfig = p->pConfig; Fts5IndexIter *pRet = 0; int iIdx = 0; Fts5Buffer buf = {0, 0, 0}; /* If the QUERY_SCAN flag is set, all other flags must be clear. */ assert( (flags & FTS5INDEX_QUERY_SCAN)==0 || (flags & FTS5INDEX_QUERY_SCAN)==FTS5INDEX_QUERY_SCAN ); if( sqlite3Fts5BufferGrow(&p->rc, &buf, nToken+1)==0 ){ memcpy(&buf.p[1], pToken, nToken); #ifdef SQLITE_DEBUG if( flags & FTS5INDEX_QUERY_TEST_NOIDX ){ assert( flags & FTS5INDEX_QUERY_PREFIX ); iIdx = 1+pConfig->nPrefix; }else #endif if( flags & FTS5INDEX_QUERY_PREFIX ){ int nChar = fts5IndexCharlen(pToken, nToken); for(iIdx=1; iIdx<=pConfig->nPrefix; iIdx++){ if( pConfig->aPrefix[iIdx-1]==nChar ) break; } } if( iIdx<=pConfig->nPrefix ){ Fts5Structure *pStruct = fts5StructureRead(p); buf.p[0] = FTS5_MAIN_PREFIX + iIdx; if( pStruct ){ fts5MultiIterNew(p, pStruct, 1, flags, buf.p, nToken+1, -1, 0, &pRet); fts5StructureRelease(pStruct); } }else{ int bDesc = (flags & FTS5INDEX_QUERY_DESC)!=0; buf.p[0] = FTS5_MAIN_PREFIX; fts5SetupPrefixIter(p, bDesc, buf.p, nToken+1, &pRet); } if( p->rc ){ sqlite3Fts5IterClose(pRet); pRet = 0; fts5CloseReader(p); } *ppIter = pRet; sqlite3Fts5BufferFree(&buf); } return fts5IndexReturn(p); } /* ** Return true if the iterator passed as the only argument is at EOF. */ int sqlite3Fts5IterEof(Fts5IndexIter *pIter){ assert( pIter->pIndex->rc==SQLITE_OK ); return pIter->bEof; } /* ** Move to the next matching rowid. */ int sqlite3Fts5IterNext(Fts5IndexIter *pIter){ assert( pIter->pIndex->rc==SQLITE_OK ); fts5MultiIterNext(pIter->pIndex, pIter, 0, 0); return fts5IndexReturn(pIter->pIndex); } /* ** Move to the next matching term/rowid. Used by the fts5vocab module. */ int sqlite3Fts5IterNextScan(Fts5IndexIter *pIter){ Fts5Index *p = pIter->pIndex; assert( pIter->pIndex->rc==SQLITE_OK ); fts5MultiIterNext(p, pIter, 0, 0); if( p->rc==SQLITE_OK ){ Fts5SegIter *pSeg = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; if( pSeg->pLeaf && pSeg->term.p[0]!=FTS5_MAIN_PREFIX ){ fts5DataRelease(pSeg->pLeaf); pSeg->pLeaf = 0; pIter->bEof = 1; } } return fts5IndexReturn(pIter->pIndex); } /* ** Move to the next matching rowid that occurs at or after iMatch. The ** definition of "at or after" depends on whether this iterator iterates ** in ascending or descending rowid order. */ int sqlite3Fts5IterNextFrom(Fts5IndexIter *pIter, i64 iMatch){ fts5MultiIterNextFrom(pIter->pIndex, pIter, iMatch); return fts5IndexReturn(pIter->pIndex); } /* ** Return the current rowid. */ i64 sqlite3Fts5IterRowid(Fts5IndexIter *pIter){ return fts5MultiIterRowid(pIter); } /* ** Return the current term. */ const char *sqlite3Fts5IterTerm(Fts5IndexIter *pIter, int *pn){ int n; const char *z = (const char*)fts5MultiIterTerm(pIter, &n); *pn = n-1; return &z[1]; } /* ** Return a pointer to a buffer containing a copy of the position list for ** the current entry. Output variable *pn is set to the size of the buffer ** in bytes before returning. ** ** The returned position list does not include the "number of bytes" varint ** field that starts the position list on disk. */ int sqlite3Fts5IterPoslist( Fts5IndexIter *pIter, const u8 **pp, /* OUT: Pointer to position-list data */ int *pn, /* OUT: Size of position-list in bytes */ i64 *piRowid /* OUT: Current rowid */ ){ Fts5SegIter *pSeg = &pIter->aSeg[ pIter->aFirst[1].iFirst ]; assert( pIter->pIndex->rc==SQLITE_OK ); *piRowid = pSeg->iRowid; *pn = pSeg->nPos; if( pSeg->iLeafOffset+pSeg->nPos <= pSeg->pLeaf->n ){ *pp = &pSeg->pLeaf->p[pSeg->iLeafOffset]; }else{ fts5BufferZero(&pIter->poslist); fts5SegiterPoslist(pIter->pIndex, pSeg, &pIter->poslist); *pp = pIter->poslist.p; } return fts5IndexReturn(pIter->pIndex); } /* ** This function is similar to sqlite3Fts5IterPoslist(), except that it ** copies the position list into the buffer supplied as the second ** argument. */ int sqlite3Fts5IterPoslistBuffer(Fts5IndexIter *pIter, Fts5Buffer *pBuf){ Fts5Index *p = pIter->pIndex; assert( p->rc==SQLITE_OK ); fts5BufferZero(pBuf); fts5MultiIterPoslist(p, pIter, 0, pBuf); return fts5IndexReturn(p); } /* ** Close an iterator opened by an earlier call to sqlite3Fts5IndexQuery(). */ void sqlite3Fts5IterClose(Fts5IndexIter *pIter){ if( pIter ){ Fts5Index *pIndex = pIter->pIndex; fts5MultiIterFree(pIter->pIndex, pIter); fts5CloseReader(pIndex); } } /* ** Read the "averages" record into the buffer supplied as the second ** argument. Return SQLITE_OK if successful, or an SQLite error code ** if an error occurs. |
︙ | ︙ | |||
4796 4797 4798 4799 4800 4801 4802 | }else if( rc==SQLITE_OK && (pPrev->n!=n || memcmp(pPrev->p, z, n)) ){ u64 cksum3 = *pCksum; const char *zTerm = (const char*)&pPrev->p[1]; /* term sans prefix-byte */ int nTerm = pPrev->n-1; /* Size of zTerm in bytes */ int iIdx = (pPrev->p[0] - FTS5_MAIN_PREFIX); int flags = (iIdx==0 ? 0 : FTS5INDEX_QUERY_PREFIX); | < | 5122 5123 5124 5125 5126 5127 5128 5129 5130 5131 5132 5133 5134 5135 | }else if( rc==SQLITE_OK && (pPrev->n!=n || memcmp(pPrev->p, z, n)) ){ u64 cksum3 = *pCksum; const char *zTerm = (const char*)&pPrev->p[1]; /* term sans prefix-byte */ int nTerm = pPrev->n-1; /* Size of zTerm in bytes */ int iIdx = (pPrev->p[0] - FTS5_MAIN_PREFIX); int flags = (iIdx==0 ? 0 : FTS5INDEX_QUERY_PREFIX); u64 ck1 = 0; u64 ck2 = 0; /* Check that the results returned for ASC and DESC queries are ** the same. If not, call this corruption. */ rc = fts5QueryCksum(p, iIdx, zTerm, nTerm, flags, &ck1); if( rc==SQLITE_OK ){ |
︙ | ︙ | |||
4978 4979 4980 4981 4982 4983 4984 | ** checksum does not match. Return SQLITE_OK if all checks pass without ** error, or some other SQLite error code if another error (e.g. OOM) ** occurs. */ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){ u64 cksum2 = 0; /* Checksum based on contents of indexes */ Fts5Buffer poslist = {0,0,0}; /* Buffer used to hold a poslist */ | | | 5303 5304 5305 5306 5307 5308 5309 5310 5311 5312 5313 5314 5315 5316 5317 | ** checksum does not match. Return SQLITE_OK if all checks pass without ** error, or some other SQLite error code if another error (e.g. OOM) ** occurs. */ int sqlite3Fts5IndexIntegrityCheck(Fts5Index *p, u64 cksum){ u64 cksum2 = 0; /* Checksum based on contents of indexes */ Fts5Buffer poslist = {0,0,0}; /* Buffer used to hold a poslist */ Fts5IndexIter *pIter; /* Used to iterate through entire index */ Fts5Structure *pStruct; /* Index structure */ /* Used by extra internal tests only run if NDEBUG is not defined */ u64 cksum3 = 0; /* Checksum based on contents of indexes */ Fts5Buffer term = {0,0,0}; /* Buffer used to hold most recent term */ /* Load the FTS index structure */ |
︙ | ︙ | |||
5022 5023 5024 5025 5026 5027 5028 5029 5030 5031 5032 5033 5034 5035 | ){ int n; /* Size of term in bytes */ i64 iPos = 0; /* Position read from poslist */ int iOff = 0; /* Offset within poslist */ i64 iRowid = fts5MultiIterRowid(pIter); char *z = (char*)fts5MultiIterTerm(pIter, &n); poslist.n = 0; fts5MultiIterPoslist(p, pIter, 0, &poslist); while( 0==sqlite3Fts5PoslistNext64(poslist.p, poslist.n, &iOff, &iPos) ){ int iCol = FTS5_POS2COLUMN(iPos); int iTokOff = FTS5_POS2OFFSET(iPos); cksum2 ^= fts5IndexEntryCksum(iRowid, iCol, iTokOff, -1, z, n); } | > > > < < < | 5347 5348 5349 5350 5351 5352 5353 5354 5355 5356 5357 5358 5359 5360 5361 5362 5363 5364 5365 5366 5367 5368 5369 5370 | ){ int n; /* Size of term in bytes */ i64 iPos = 0; /* Position read from poslist */ int iOff = 0; /* Offset within poslist */ i64 iRowid = fts5MultiIterRowid(pIter); char *z = (char*)fts5MultiIterTerm(pIter, &n); /* If this is a new term, query for it. Update cksum3 with the results. */ fts5TestTerm(p, &term, z, n, cksum2, &cksum3); poslist.n = 0; fts5MultiIterPoslist(p, pIter, 0, &poslist); while( 0==sqlite3Fts5PoslistNext64(poslist.p, poslist.n, &iOff, &iPos) ){ int iCol = FTS5_POS2COLUMN(iPos); int iTokOff = FTS5_POS2OFFSET(iPos); cksum2 ^= fts5IndexEntryCksum(iRowid, iCol, iTokOff, -1, z, n); } } fts5TestTerm(p, &term, 0, 0, cksum2, &cksum3); fts5MultiIterFree(p, pIter); if( p->rc==SQLITE_OK && cksum!=cksum2 ) p->rc = FTS5_CORRUPT; fts5StructureRelease(pStruct); |
︙ | ︙ | |||
5129 5130 5131 5132 5133 5134 5135 | Fts5Structure *p ){ int iLvl, iSeg; /* Iterate through levels, segments */ for(iLvl=0; iLvl<p->nLevel; iLvl++){ Fts5StructureLevel *pLvl = &p->aLevel[iLvl]; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, | | | 5454 5455 5456 5457 5458 5459 5460 5461 5462 5463 5464 5465 5466 5467 5468 | Fts5Structure *p ){ int iLvl, iSeg; /* Iterate through levels, segments */ for(iLvl=0; iLvl<p->nLevel; iLvl++){ Fts5StructureLevel *pLvl = &p->aLevel[iLvl]; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {lvl=%d nMerge=%d nSeg=%d", iLvl, pLvl->nMerge, pLvl->nSeg ); for(iSeg=0; iSeg<pLvl->nSeg; iSeg++){ Fts5StructureSegment *pSeg = &pLvl->aSeg[iSeg]; sqlite3Fts5BufferAppendPrintf(pRc, pBuf, " {id=%d h=%d leaves=%d..%d}", pSeg->iSegid, pSeg->nHeight, pSeg->pgnoFirst, pSeg->pgnoLast ); |
︙ | ︙ |
Changes to ext/fts5/fts5_main.c.
︙ | ︙ | |||
215 216 217 218 219 220 221 | #define FTS5_BI_ORDER_DESC 0x0080 /* ** Values for Fts5Cursor.csrflags */ #define FTS5CSR_REQUIRE_CONTENT 0x01 #define FTS5CSR_REQUIRE_DOCSIZE 0x02 | > | | | < < < < < < < < | 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 | #define FTS5_BI_ORDER_DESC 0x0080 /* ** Values for Fts5Cursor.csrflags */ #define FTS5CSR_REQUIRE_CONTENT 0x01 #define FTS5CSR_REQUIRE_DOCSIZE 0x02 #define FTS5CSR_REQUIRE_INST 0x04 #define FTS5CSR_EOF 0x08 #define FTS5CSR_FREE_ZRANK 0x10 #define FTS5CSR_REQUIRE_RESEEK 0x20 #define BitFlagAllTest(x,y) (((x) & (y))==(y)) #define BitFlagTest(x,y) (((x) & (y))!=0) /* ** Macros to Set(), Clear() and Test() cursor flags. */ #define CsrFlagSet(pCsr, flag) ((pCsr)->csrflags |= (flag)) #define CsrFlagClear(pCsr, flag) ((pCsr)->csrflags &= ~(flag)) #define CsrFlagTest(pCsr, flag) ((pCsr)->csrflags & (flag)) |
︙ | ︙ | |||
433 434 435 436 437 438 439 | ){ return fts5InitVtab(1, db, pAux, argc, argv, ppVtab, pzErr); } /* ** The different query plans. */ | < | > > > | < < | 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 | ){ return fts5InitVtab(1, db, pAux, argc, argv, ppVtab, pzErr); } /* ** The different query plans. */ #define FTS5_PLAN_MATCH 0 /* (<tbl> MATCH ?) */ #define FTS5_PLAN_SOURCE 1 /* A source cursor for SORTED_MATCH */ #define FTS5_PLAN_SPECIAL 2 /* An internal query */ #define FTS5_PLAN_SORTED_MATCH 3 /* (<tbl> MATCH ? ORDER BY rank) */ #define FTS5_PLAN_SCAN 4 /* No usable constraint */ #define FTS5_PLAN_ROWID 5 /* (rowid = ?) */ /* ** Implementation of the xBestIndex method for FTS5 tables. Within the ** WHERE constraint, it searches for the following: ** ** 1. A MATCH constraint against the special column. ** 2. A MATCH constraint against the "rank" column. |
︙ | ︙ | |||
607 608 609 610 611 612 613 | /* ** This function is called after the cursor passed as the only argument ** is moved to point at a different row. It clears all cached data ** specific to the previous row stored by the cursor object. */ static void fts5CsrNewrow(Fts5Cursor *pCsr){ | | | | | > | | 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 | /* ** This function is called after the cursor passed as the only argument ** is moved to point at a different row. It clears all cached data ** specific to the previous row stored by the cursor object. */ static void fts5CsrNewrow(Fts5Cursor *pCsr){ CsrFlagSet(pCsr, FTS5CSR_REQUIRE_CONTENT | FTS5CSR_REQUIRE_DOCSIZE | FTS5CSR_REQUIRE_INST ); } /* ** Close the cursor. For additional information see the documentation ** on the xClose method of the virtual table interface. */ static int fts5CloseMethod(sqlite3_vtab_cursor *pCursor){ if( pCursor ){ Fts5Table *pTab = (Fts5Table*)(pCursor->pVtab); Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; Fts5Cursor **pp; Fts5Auxdata *pData; Fts5Auxdata *pNext; sqlite3_free(pCsr->aInst); if( pCsr->pStmt ){ int eStmt = fts5StmtType(pCsr); sqlite3Fts5StorageStmtRelease(pTab->pStorage, eStmt, pCsr->pStmt); } if( pCsr->pSorter ){ Fts5Sorter *pSorter = pCsr->pSorter; sqlite3_finalize(pSorter->pStmt); |
︙ | ︙ | |||
758 759 760 761 762 763 764 | ** ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned ** even if we reach end-of-file. The fts5EofMethod() will be called ** subsequently to determine whether or not an EOF was hit. */ static int fts5NextMethod(sqlite3_vtab_cursor *pCursor){ Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; | < < > > | | | | | | | | | | | | | | | | | | | | | | | | | | | | | | > | 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 | ** ** Return SQLITE_OK if nothing goes wrong. SQLITE_OK is returned ** even if we reach end-of-file. The fts5EofMethod() will be called ** subsequently to determine whether or not an EOF was hit. */ static int fts5NextMethod(sqlite3_vtab_cursor *pCursor){ Fts5Cursor *pCsr = (Fts5Cursor*)pCursor; int rc; assert( (pCsr->ePlan<2)== (pCsr->ePlan==FTS5_PLAN_MATCH || pCsr->ePlan==FTS5_PLAN_SOURCE) ); if( pCsr->ePlan<2 ){ int bSkip = 0; if( (rc = fts5CursorReseek(pCsr, &bSkip)) || bSkip ) return rc; rc = sqlite3Fts5ExprNext(pCsr->pExpr, pCsr->iLastRowid); if( sqlite3Fts5ExprEof(pCsr->pExpr) ){ CsrFlagSet(pCsr, FTS5CSR_EOF); } fts5CsrNewrow(pCsr); }else{ switch( pCsr->ePlan ){ case FTS5_PLAN_SPECIAL: { CsrFlagSet(pCsr, FTS5CSR_EOF); break; } case FTS5_PLAN_SORTED_MATCH: { rc = fts5SorterNext(pCsr); break; } default: rc = sqlite3_step(pCsr->pStmt); if( rc!=SQLITE_ROW ){ CsrFlagSet(pCsr, FTS5CSR_EOF); rc = sqlite3_reset(pCsr->pStmt); }else{ rc = SQLITE_OK; } break; } } return rc; } static int fts5CursorFirstSorted(Fts5Table *pTab, Fts5Cursor *pCsr, int bDesc){ Fts5Config *pConfig = pTab->pConfig; |
︙ | ︙ | |||
1518 1519 1520 1521 1522 1523 1524 | /* ** Ensure that the Fts5Cursor.nInstCount and aInst[] variables are populated ** correctly for the current view. Return SQLITE_OK if successful, or an ** SQLite error code otherwise. */ static int fts5CacheInstArray(Fts5Cursor *pCsr){ int rc = SQLITE_OK; | | | 1513 1514 1515 1516 1517 1518 1519 1520 1521 1522 1523 1524 1525 1526 1527 | /* ** Ensure that the Fts5Cursor.nInstCount and aInst[] variables are populated ** correctly for the current view. Return SQLITE_OK if successful, or an ** SQLite error code otherwise. */ static int fts5CacheInstArray(Fts5Cursor *pCsr){ int rc = SQLITE_OK; if( CsrFlagTest(pCsr, FTS5CSR_REQUIRE_INST) ){ Fts5PoslistReader *aIter; /* One iterator for each phrase */ int nIter; /* Number of iterators/phrases */ int nByte; nIter = sqlite3Fts5ExprPhraseCount(pCsr->pExpr); nByte = sizeof(Fts5PoslistReader) * nIter; aIter = (Fts5PoslistReader*)sqlite3Fts5MallocZero(&rc, nByte); |
︙ | ︙ | |||
1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 1574 1575 1576 | aInst = &((int*)buf.p)[3 * (nInst-1)]; aInst[0] = iBest; aInst[1] = FTS5_POS2COLUMN(aIter[iBest].iPos); aInst[2] = FTS5_POS2OFFSET(aIter[iBest].iPos); sqlite3Fts5PoslistReaderNext(&aIter[iBest]); } pCsr->aInst = (int*)buf.p; pCsr->nInstCount = nInst; sqlite3_free(aIter); } } return rc; } static int fts5ApiInstCount(Fts5Context *pCtx, int *pnInst){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; | > > | 1555 1556 1557 1558 1559 1560 1561 1562 1563 1564 1565 1566 1567 1568 1569 1570 1571 1572 1573 | aInst = &((int*)buf.p)[3 * (nInst-1)]; aInst[0] = iBest; aInst[1] = FTS5_POS2COLUMN(aIter[iBest].iPos); aInst[2] = FTS5_POS2OFFSET(aIter[iBest].iPos); sqlite3Fts5PoslistReaderNext(&aIter[iBest]); } sqlite3_free(pCsr->aInst); pCsr->aInst = (int*)buf.p; pCsr->nInstCount = nInst; sqlite3_free(aIter); CsrFlagClear(pCsr, FTS5CSR_REQUIRE_INST); } } return rc; } static int fts5ApiInstCount(Fts5Context *pCtx, int *pnInst){ Fts5Cursor *pCsr = (Fts5Cursor*)pCtx; |
︙ | ︙ | |||
2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 | Fts5Global *pGlobal = (Fts5Global*)sqlite3_user_data(pCtx); char buf[8]; assert( nArg==0 ); assert( sizeof(buf)>=sizeof(pGlobal) ); memcpy(buf, (void*)&pGlobal, sizeof(pGlobal)); sqlite3_result_blob(pCtx, buf, sizeof(pGlobal), SQLITE_TRANSIENT); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_fts5_init( sqlite3 *db, char **pzErrMsg, | > > > > > > > > > > > > | 2215 2216 2217 2218 2219 2220 2221 2222 2223 2224 2225 2226 2227 2228 2229 2230 2231 2232 2233 2234 2235 2236 2237 2238 2239 2240 | Fts5Global *pGlobal = (Fts5Global*)sqlite3_user_data(pCtx); char buf[8]; assert( nArg==0 ); assert( sizeof(buf)>=sizeof(pGlobal) ); memcpy(buf, (void*)&pGlobal, sizeof(pGlobal)); sqlite3_result_blob(pCtx, buf, sizeof(pGlobal), SQLITE_TRANSIENT); } /* ** Implementation of fts5_source_id() function. */ static void fts5SourceIdFunc( sqlite3_context *pCtx, /* Function call context */ int nArg, /* Number of args */ sqlite3_value **apVal /* Function arguments */ ){ assert( nArg==0 ); sqlite3_result_text(pCtx, "--FTS5-SOURCE-ID--", -1, SQLITE_TRANSIENT); } #ifdef _WIN32 __declspec(dllexport) #endif int sqlite3_fts5_init( sqlite3 *db, char **pzErrMsg, |
︙ | ︙ | |||
2279 2280 2281 2282 2283 2284 2285 2286 2287 2288 2289 2290 2291 2292 | if( rc==SQLITE_OK ) rc = sqlite3Fts5AuxInit(&pGlobal->api); if( rc==SQLITE_OK ) rc = sqlite3Fts5TokenizerInit(&pGlobal->api); if( rc==SQLITE_OK ) rc = sqlite3Fts5VocabInit(pGlobal, db); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5", 0, SQLITE_UTF8, p, fts5Fts5Func, 0, 0 ); } } return rc; } #ifdef _WIN32 __declspec(dllexport) | > > > > > | 2288 2289 2290 2291 2292 2293 2294 2295 2296 2297 2298 2299 2300 2301 2302 2303 2304 2305 2306 | if( rc==SQLITE_OK ) rc = sqlite3Fts5AuxInit(&pGlobal->api); if( rc==SQLITE_OK ) rc = sqlite3Fts5TokenizerInit(&pGlobal->api); if( rc==SQLITE_OK ) rc = sqlite3Fts5VocabInit(pGlobal, db); if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5", 0, SQLITE_UTF8, p, fts5Fts5Func, 0, 0 ); } if( rc==SQLITE_OK ){ rc = sqlite3_create_function( db, "fts5_source_id", 0, SQLITE_UTF8, p, fts5SourceIdFunc, 0, 0 ); } } return rc; } #ifdef _WIN32 __declspec(dllexport) |
︙ | ︙ |
Changes to ext/fts5/test/fts5_common.tcl.
︙ | ︙ | |||
120 121 122 123 124 125 126 | } } proc fts5_level_segs {tbl} { set sql "SELECT fts5_decode(rowid,block) aS r FROM ${tbl}_data WHERE rowid=10" set ret [list] foreach L [lrange [db one $sql] 1 end] { | | | | 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 | } } proc fts5_level_segs {tbl} { set sql "SELECT fts5_decode(rowid,block) aS r FROM ${tbl}_data WHERE rowid=10" set ret [list] foreach L [lrange [db one $sql] 1 end] { lappend ret [expr [llength $L] - 3] } set ret } proc fts5_level_segids {tbl} { set sql "SELECT fts5_decode(rowid,block) aS r FROM ${tbl}_data WHERE rowid=10" set ret [list] foreach L [lrange [db one $sql] 1 end] { set lvl [list] foreach S [lrange $L 3 end] { regexp {id=([1234567890]*)} $S -> segid lappend lvl $segid } lappend ret $lvl } set ret } |
︙ | ︙ |
Changes to ext/fts5/test/fts5aa.test.
︙ | ︙ | |||
45 46 47 48 49 50 51 | CREATE VIRTUAL TABLE t1 USING fts5(x,y); } do_execsql_test 2.1 { INSERT INTO t1 VALUES('a b c', 'd e f'); } do_test 2.2 { execsql { SELECT fts5_decode(id, block) FROM t1_data WHERE id==10 } | | > > | > > > > > > > > | 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 | CREATE VIRTUAL TABLE t1 USING fts5(x,y); } do_execsql_test 2.1 { INSERT INTO t1 VALUES('a b c', 'd e f'); } do_test 2.2 { execsql { SELECT fts5_decode(id, block) FROM t1_data WHERE id==10 } } {/{\(structure\) {lvl=0 nMerge=0 nSeg=1 {id=[0123456789]* h=1 leaves=1..1}}}/} foreach w {a b c d e f} { do_execsql_test 2.3.$w.asc { SELECT rowid FROM t1 WHERE t1 MATCH $w; } {1} do_execsql_test 2.3.$w.desc { SELECT rowid FROM t1 WHERE t1 MATCH $w ORDER BY rowid DESC; } {1} } do_execsql_test 2.4 { INSERT INTO t1(t1) VALUES('integrity-check'); } #------------------------------------------------------------------------- # reset_db do_execsql_test 3.0 { |
︙ | ︙ | |||
186 187 188 189 190 191 192 | set z [doc] set rowid [expr int(rand() * 100)] execsql { REPLACE INTO t1(rowid,x,y,z) VALUES($rowid, $x, $y, $z) } } execsql { INSERT INTO t1(t1) VALUES('integrity-check'); } } {} } | < < | 196 197 198 199 200 201 202 203 204 205 206 207 208 209 | set z [doc] set rowid [expr int(rand() * 100)] execsql { REPLACE INTO t1(rowid,x,y,z) VALUES($rowid, $x, $y, $z) } } execsql { INSERT INTO t1(t1) VALUES('integrity-check'); } } {} } #------------------------------------------------------------------------- # reset_db do_execsql_test 8.0 { CREATE VIRTUAL TABLE t1 USING fts5(x, prefix="1,2,3"); INSERT INTO t1(t1, rank) VALUES('pgsz', 32); |
︙ | ︙ |
Changes to ext/fts5/test/fts5ac.test.
︙ | ︙ | |||
200 201 202 203 204 205 206 207 208 209 210 211 212 213 | do_test $tn2.1.1 { foreach {id x y} $data { execsql { INSERT INTO xx(rowid, x, y) VALUES($id, $x, $y) } } execsql { INSERT INTO xx(xx) VALUES('integrity-check') } } {} #------------------------------------------------------------------------- # Test phrase queries. # foreach {tn phrase} { 1 "o" 2 "b q" | > | 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 | do_test $tn2.1.1 { foreach {id x y} $data { execsql { INSERT INTO xx(rowid, x, y) VALUES($id, $x, $y) } } execsql { INSERT INTO xx(xx) VALUES('integrity-check') } } {} #------------------------------------------------------------------------- # Test phrase queries. # foreach {tn phrase} { 1 "o" 2 "b q" |
︙ | ︙ |
Changes to ext/fts5/test/fts5ad.test.
︙ | ︙ | |||
200 201 202 203 204 205 206 | break } } if {$bMatch} { lappend ret $rowid } } return $ret } | | > | | | 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 | break } } if {$bMatch} { lappend ret $rowid } } return $ret } foreach {bAsc sql} { 1 {SELECT rowid FROM t1 WHERE t1 MATCH $prefix} 0 {SELECT rowid FROM t1 WHERE t1 MATCH $prefix ORDER BY rowid DESC} } { foreach {tn prefix} { 1 {a*} 2 {ab*} 3 {abc*} 4 {abcd*} 5 {abcde*} 6 {f*} 7 {fg*} 8 {fgh*} 9 {fghi*} 10 {fghij*} 11 {k*} 12 {kl*} 13 {klm*} 14 {klmn*} 15 {klmno*} 16 {p*} 17 {pq*} 18 {pqr*} 19 {pqrs*} 20 {pqrst*} 21 {u*} 22 {uv*} 23 {uvw*} 24 {uvwx*} 25 {uvwxy*} 26 {uvwxyz*} |
︙ | ︙ |
Changes to ext/fts5/test/fts5alter.test.
︙ | ︙ | |||
77 78 79 80 81 82 83 84 85 86 | } {-56 -22 -11} do_execsql_test 2.3 { ROLLBACK; SELECT rowid FROM yy WHERE yy MATCH 'a + b + c'; } {-56 -22} finish_test | > > > > > > > > > > > > > > > > > | 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 | } {-56 -22 -11} do_execsql_test 2.3 { ROLLBACK; SELECT rowid FROM yy WHERE yy MATCH 'a + b + c'; } {-56 -22} #------------------------------------------------------------------------- do_execsql_test 3.1 { CREATE VIRTUAL TABLE abc USING fts5(a); INSERT INTO abc(rowid, a) VALUES(1, 'a'); BEGIN; INSERT INTO abc(rowid, a) VALUES(2, 'a'); } breakpoint do_execsql_test 3.2 { SELECT rowid FROM abc WHERE abc MATCH 'a'; } {1 2} do_execsql_test 3.3 { COMMIT; SELECT rowid FROM abc WHERE abc MATCH 'a'; } {1 2} finish_test |
Changes to ext/fts5/test/fts5rowid.test.
︙ | ︙ | |||
67 68 69 70 71 72 73 | set res [db one {SELECT count(*) FROM x1_data}] do_execsql_test 2.3 { SELECT count(fts5_decode(rowid, block)) FROM x1_data; } $res do_execsql_test 2.4 { UPDATE x1_data SET block = X''; | | > > > | | | | | 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 | set res [db one {SELECT count(*) FROM x1_data}] do_execsql_test 2.3 { SELECT count(fts5_decode(rowid, block)) FROM x1_data; } $res do_execsql_test 2.4 { UPDATE x1_data SET block = X''; -- SELECT count(fts5_decode(rowid, block)) FROM x1_data; SELECT count(*) FROM x1_data; } $res do_execsql_test 2.5 { INSERT INTO x1(x1, rank) VALUES('pgsz', 1024); INSERT INTO x1(x1) VALUES('rebuild'); } set res [db one {SELECT count(*) FROM x1_data}] do_execsql_test 2.6 { SELECT count(fts5_decode(rowid, block)) FROM x1_data; } $res # This is really a corruption test... #do_execsql_test 2.7 { # UPDATE x1_data SET block = X''; # SELECT count(fts5_decode(rowid, block)) FROM x1_data; #} $res #------------------------------------------------------------------------- # Tests with very large tokens. # set strlist [list \ "[string repeat x 400]" \ "[string repeat x 300][string repeat w 100]" \ |
︙ | ︙ |
Changes to ext/fts5/test/fts5unicode2.test.
︙ | ︙ | |||
376 377 378 379 380 381 382 | proc do_isspace_test {tn tokenizer lCp} { set whitespace [format [string repeat %c [llength $lCp]] {*}$lCp] set txt "${whitespace}hello${whitespace}world${whitespace}" uplevel [list do_test $tn [list do_tokenize $tokenizer $txt] {hello world}] } set tokenizers [list unicode61] | | | 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 | proc do_isspace_test {tn tokenizer lCp} { set whitespace [format [string repeat %c [llength $lCp]] {*}$lCp] set txt "${whitespace}hello${whitespace}world${whitespace}" uplevel [list do_test $tn [list do_tokenize $tokenizer $txt] {hello world}] } set tokenizers [list unicode61] #ifcapable icu { lappend tokenizers icu } # Some tests to check that the tokenizers can both identify white-space # codepoints. All codepoints tested below are of type "Zs" in the # UnicodeData.txt file. foreach T $tokenizers { do_isspace_test 6.$T.1 $T 32 do_isspace_test 6.$T.2 $T 160 |
︙ | ︙ |
Changes to ext/fts5/test/fts5version.test.
︙ | ︙ | |||
42 43 44 45 46 47 48 | do_test 1.5 { db close sqlite3 db test.db catchsql { SELECT * FROM t1 WHERE t1 MATCH 'a' } } {1 {invalid fts5 file format (found 3, expected 2) - run 'rebuild'}} | < | 42 43 44 45 46 47 48 49 50 51 52 53 54 55 | do_test 1.5 { db close sqlite3 db test.db catchsql { SELECT * FROM t1 WHERE t1 MATCH 'a' } } {1 {invalid fts5 file format (found 3, expected 2) - run 'rebuild'}} do_test 1.6 { db close sqlite3 db test.db catchsql { INSERT INTO t1 VALUES('x y z') } } {1 {invalid fts5 file format (found 3, expected 2) - run 'rebuild'}} do_test 1.7 { |
︙ | ︙ |
Changes to ext/fts5/tool/loadfts5.tcl.
︙ | ︙ | |||
98 99 100 101 102 103 104 105 106 107 108 109 110 111 | } } } set dbfile [lindex $argv end-1] if {$O(delete)} { file delete -force $dbfile } sqlite3 db $dbfile db func loadfile loadfile db transaction { set pref "" if {$O(prefix)!=""} { set pref ", prefix='$O(prefix)'" } catch { db eval "CREATE VIRTUAL TABLE t1 USING $O(vtab) (path, content$O(tok)$pref)" | > | 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 | } } } set dbfile [lindex $argv end-1] if {$O(delete)} { file delete -force $dbfile } sqlite3 db $dbfile catch { load_static_extension db fts5 } db func loadfile loadfile db transaction { set pref "" if {$O(prefix)!=""} { set pref ", prefix='$O(prefix)'" } catch { db eval "CREATE VIRTUAL TABLE t1 USING $O(vtab) (path, content$O(tok)$pref)" |
︙ | ︙ |
Changes to ext/fts5/tool/mkfts5c.tcl.
︙ | ︙ | |||
20 21 22 23 24 25 26 | %dir%/fts5_varint.c %dir%/fts5_vocab.c fts5parse.c }] set G(hdr) { | | | > > > > > > > > > > > > > > > > > > > > | > > > > > > > | 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 | %dir%/fts5_varint.c %dir%/fts5_vocab.c fts5parse.c }] set G(hdr) { #if !defined(SQLITE_TEST) || defined(SQLITE_ENABLE_FTS5) #if !defined(NDEBUG) && !defined(SQLITE_DEBUG) # define NDEBUG 1 #endif #if defined(NDEBUG) && defined(SQLITE_DEBUG) # undef NDEBUG #endif } set G(footer) { #endif /* !defined(SQLITE_TEST) || defined(SQLITE_ENABLE_FTS5) */ } #------------------------------------------------------------------------- # Read and return the entire contents of text file $zFile from disk. # proc readfile {zFile} { set fd [open $zFile] set data [read $fd] close $fd return $data } #------------------------------------------------------------------------- # This command returns a string identifying the current sqlite version - # the equivalent of the SQLITE_SOURCE_ID string. # proc fts5_source_id {zDir} { set top [file dirname [file dirname $zDir]] set uuid [string trim [readfile [file join $top manifest.uuid]]] set L [split [readfile [file join $top manifest]]] set date [lindex $L [expr [lsearch -exact $L D]+1]] set date [string range $date 0 [string last . $date]-1] set date [string map {T { }} $date] return "fts5: $date $uuid" } proc fts5c_init {zOut} { global G set G(fd) stdout set G(fd) [open $zOut w] puts -nonewline $G(fd) $G(hdr) } proc fts5c_printfile {zIn} { global G set data [readfile $zIn] set zTail [file tail $zIn] puts $G(fd) "#line 2 \"$zTail\"" set sub_map [list --FTS5-SOURCE-ID-- [fts5_source_id $::srcdir]] if {$zTail=="fts5parse.c"} { lappend sub_map yy fts5yy YY fts5YY TOKEN FTS5TOKEN } foreach line [split $data "\n"] { if {[regexp {^#include.*fts5} $line]} continue if {[regexp {^(const )?[a-zA-Z][a-zA-Z0-9]* [*]?sqlite3Fts5} $line]} { set line "static $line" } set line [string map $sub_map $line] puts $G(fd) $line } } proc fts5c_close {} { global G puts -nonewline $G(fd) $G(footer) |
︙ | ︙ |
Changes to ext/fts5/tool/showfts5.tcl.
1 2 3 4 5 6 7 | proc usage {} { puts stderr "usage: $::argv0 database table" puts stderr "" exit 1 } | > > > > < < < < < < < < > > > > > > > > | > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 | #------------------------------------------------------------------------- # Process command line arguments. # proc usage {} { puts stderr "usage: $::argv0 database table" puts stderr "" exit 1 } if {[llength $argv]!=2} usage set database [lindex $argv 0] set tbl [lindex $argv 1] #------------------------------------------------------------------------- # Start of main program. # sqlite3 db $database catch { load_static_extension db fts5 } db eval "SELECT fts5_decode(rowid, block) AS d FROM ${tbl}_data WHERE id=10" { foreach lvl [lrange $d 1 end] { puts [lrange $lvl 0 2] foreach seg [lrange $lvl 3 end] { puts " $seg" } } } |
Changes to ext/ota/ota12.test.
︙ | ︙ | |||
162 163 164 165 166 167 168 169 170 171 172 | do_test 2.$tn.11 { sqlite3ota ota test.db ota.db while {[ota step]=="SQLITE_OK"} {} ota close } {SQLITE_DONE} } finish_test | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 | do_test 2.$tn.11 { sqlite3ota ota test.db ota.db while {[ota step]=="SQLITE_OK"} {} ota close } {SQLITE_DONE} } #------------------------------------------------------------------------- # Test that "PRAGMA data_version" works when an OTA client writes the # database. # do_multiclient_test tn { # Initialize a target (test.db) and ota (ota.db) database. # forcedelete ota.db sql1 $setup_sql # Check the initial database contains table "xx" with a single row. # Also save the current values of "PRAGMA data-version" for [db1] # and [db2]. # do_test 2.$tn.1 { list [sql1 { SELECT count(*) FROM xx }] [sql2 { SELECT count(*) FROM xx }] } {1 1} set V1 [sql1 {PRAGMA data_version}] set V2 [sql2 {PRAGMA data_version}] # Check the values of data-version have not magically changed. # do_test 2.$tn.2 { list [sql1 {PRAGMA data_version}] [sql2 {PRAGMA data_version}] } [list $V1 $V2] # Start stepping the OTA. From the point of view of [db1] and [db2], the # data-version values remain unchanged until the database contents are # modified. At which point the values are incremented. # sqlite3ota ota test.db ota.db set x 0 while {[db one {SELECT count(*) FROM xx}]==1} { do_test 2.$tn.3.[incr x] { list [sql1 {PRAGMA data_version}] [sql2 {PRAGMA data_version}] } [list $V1 $V2] ota step } do_test 2.$tn.5.1 { expr {$V1 < [sql1 {PRAGMA data_version}]} } 1 do_test 2.$tn.5.2 { expr {$V2 < [sql2 {PRAGMA data_version}]} } 1 # Check the db contents is as expected. # do_test 2.$tn.4 { list [sql1 {SELECT count(*) FROM xx}] [sql2 {SELECT count(*) FROM xx}] } {3 3} set V1 [sql1 {PRAGMA data_version}] set V2 [sql2 {PRAGMA data_version}] # Finish applying the OTA (i.e. do the incremental checkpoint). Check that # this does not cause the data-version values to change. # while {[ota step]=="SQLITE_OK"} { } ota close do_test 2.$tn.6 { list [sql1 {PRAGMA data_version}] [sql2 {PRAGMA data_version}] } [list $V1 $V2] } finish_test |
Changes to main.mk.
︙ | ︙ | |||
73 74 75 76 77 78 79 | vdbetrace.o wal.o walker.o where.o wherecode.o whereexpr.o \ utf.o vtab.o LIBOBJ += sqlite3session.o LIBOBJ += fts5.o | < < | 73 74 75 76 77 78 79 80 81 82 83 84 85 86 | vdbetrace.o wal.o walker.o where.o wherecode.o whereexpr.o \ utf.o vtab.o LIBOBJ += sqlite3session.o LIBOBJ += fts5.o # All of the source code files. # SRC = \ $(TOP)/src/alter.c \ $(TOP)/src/analyze.c \ $(TOP)/src/attach.c \ $(TOP)/src/auth.c \ |
︙ | ︙ | |||
309 310 311 312 313 314 315 | $(TOP)/ext/misc/nextchar.c \ $(TOP)/ext/misc/percentile.c \ $(TOP)/ext/misc/regexp.c \ $(TOP)/ext/misc/spellfix.c \ $(TOP)/ext/misc/totype.c \ $(TOP)/ext/misc/wholenumber.c \ $(TOP)/ext/misc/vfslog.c \ | | > | 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 | $(TOP)/ext/misc/nextchar.c \ $(TOP)/ext/misc/percentile.c \ $(TOP)/ext/misc/regexp.c \ $(TOP)/ext/misc/spellfix.c \ $(TOP)/ext/misc/totype.c \ $(TOP)/ext/misc/wholenumber.c \ $(TOP)/ext/misc/vfslog.c \ $(TOP)/ext/fts5/fts5_tcl.c \ fts5.c #TESTSRC += $(TOP)/ext/fts2/fts2_tokenizer.c #TESTSRC += $(TOP)/ext/fts3/fts3_tokenizer.c TESTSRC2 = \ $(TOP)/src/attach.c \ |
︙ | ︙ | |||
662 663 664 665 666 667 668 | $(TOP)/ext/fts5/fts5_varint.c \ $(TOP)/ext/fts5/fts5_vocab.c \ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon cp $(TOP)/ext/fts5/fts5parse.y . rm -f fts5parse.h ./lemon $(OPTS) fts5parse.y | < < < | < < < | 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 | $(TOP)/ext/fts5/fts5_varint.c \ $(TOP)/ext/fts5/fts5_vocab.c \ fts5parse.c: $(TOP)/ext/fts5/fts5parse.y lemon cp $(TOP)/ext/fts5/fts5parse.y . rm -f fts5parse.h ./lemon $(OPTS) fts5parse.y fts5parse.h: fts5parse.c fts5.c: $(FTS5_SRC) tclsh $(TOP)/ext/fts5/tool/mkfts5c.tcl cp $(TOP)/ext/fts5/fts5.h . userauth.o: $(TOP)/ext/userauth/userauth.c $(HDR) $(EXTHDR) $(TCCX) -DSQLITE_CORE -c $(TOP)/ext/userauth/userauth.c sqlite3session.o: $(TOP)/ext/session/sqlite3session.c $(HDR) $(EXTHDR) $(TCCX) -DSQLITE_CORE -c $(TOP)/ext/session/sqlite3session.c |
︙ | ︙ | |||
891 892 893 894 895 896 897 | rm -f shell.c sqlite3ext.h rm -f sqlite3_analyzer sqlite3_analyzer.exe sqlite3_analyzer.c rm -f sqlite-*-output.vsix rm -f mptester mptester.exe rm -f fuzzershell fuzzershell.exe rm -f fuzzcheck fuzzcheck.exe rm -f sqldiff sqldiff.exe | > | 884 885 886 887 888 889 890 891 | rm -f shell.c sqlite3ext.h rm -f sqlite3_analyzer sqlite3_analyzer.exe sqlite3_analyzer.c rm -f sqlite-*-output.vsix rm -f mptester mptester.exe rm -f fuzzershell fuzzershell.exe rm -f fuzzcheck fuzzcheck.exe rm -f sqldiff sqldiff.exe rm -f fts5.c fts5.h fts5parse.* |
Changes to src/btree.c.
︙ | ︙ | |||
8951 8952 8953 8954 8955 8956 8957 | u8 *pCell; /* Cell content */ u8 *pCellIdx; /* Next element of the cell pointer array */ BtShared *pBt; /* The BtShared object that owns pPage */ u32 pc; /* Address of a cell */ u32 usableSize; /* Usable size of the page */ u32 contentOffset; /* Offset to the start of the cell content area */ u32 *heap = 0; /* Min-heap used for checking cell coverage */ | | > > | 8951 8952 8953 8954 8955 8956 8957 8958 8959 8960 8961 8962 8963 8964 8965 8966 8967 8968 8969 8970 8971 8972 8973 8974 8975 8976 8977 8978 8979 8980 8981 8982 8983 8984 8985 8986 8987 | u8 *pCell; /* Cell content */ u8 *pCellIdx; /* Next element of the cell pointer array */ BtShared *pBt; /* The BtShared object that owns pPage */ u32 pc; /* Address of a cell */ u32 usableSize; /* Usable size of the page */ u32 contentOffset; /* Offset to the start of the cell content area */ u32 *heap = 0; /* Min-heap used for checking cell coverage */ u32 x, prev = 0; /* Next and previous entry on the min-heap */ const char *saved_zPfx = pCheck->zPfx; int saved_v1 = pCheck->v1; int saved_v2 = pCheck->v2; u8 savedIsInit; /* Check that the page exists */ pBt = pCheck->pBt; usableSize = pBt->usableSize; if( iPage==0 ) return 0; if( checkRef(pCheck, iPage) ) return 0; pCheck->zPfx = "Page %d: "; pCheck->v1 = iPage; if( (rc = btreeGetPage(pBt, (Pgno)iPage, &pPage, 0))!=0 ){ checkAppendMsg(pCheck, "unable to get the page. error code=%d", rc); goto end_of_check; } /* Clear MemPage.isInit to make sure the corruption detection code in ** btreeInitPage() is executed. */ savedIsInit = pPage->isInit; pPage->isInit = 0; if( (rc = btreeInitPage(pPage))!=0 ){ assert( rc==SQLITE_CORRUPT ); /* The only possible error from InitPage */ checkAppendMsg(pCheck, "btreeInitPage() returns error code %d", rc); goto end_of_check; } |
︙ | ︙ | |||
9014 9015 9016 9017 9018 9019 9020 | depth = checkTreePage(pCheck, pgno, &maxKey, maxKey); keyCanBeEqual = 0; }else{ /* For leaf pages, the coverage check will occur in the same loop ** as the other cell checks, so initialize the heap. */ heap = pCheck->heap; heap[0] = 0; | < | 9016 9017 9018 9019 9020 9021 9022 9023 9024 9025 9026 9027 9028 9029 | depth = checkTreePage(pCheck, pgno, &maxKey, maxKey); keyCanBeEqual = 0; }else{ /* For leaf pages, the coverage check will occur in the same loop ** as the other cell checks, so initialize the heap. */ heap = pCheck->heap; heap[0] = 0; } /* EVIDENCE-OF: R-02776-14802 The cell pointer array consists of K 2-byte ** integer offsets to the cell contents. */ for(i=nCell-1; i>=0 && pCheck->mxErr; i--){ CellInfo info; |
︙ | ︙ | |||
9095 9096 9097 9098 9099 9100 9101 | if( doCoverageCheck && pCheck->mxErr>0 ){ /* For leaf pages, the min-heap has already been initialized and the ** cells have already been inserted. But for internal pages, that has ** not yet been done, so do it now */ if( !pPage->leaf ){ heap = pCheck->heap; heap[0] = 0; | < | > > > > > > > > > > | < < | | 9096 9097 9098 9099 9100 9101 9102 9103 9104 9105 9106 9107 9108 9109 9110 9111 9112 9113 9114 9115 9116 9117 9118 9119 9120 9121 9122 9123 9124 9125 9126 9127 9128 9129 9130 9131 9132 9133 9134 9135 9136 9137 9138 9139 9140 9141 9142 9143 9144 9145 9146 9147 9148 9149 9150 9151 9152 9153 9154 9155 9156 9157 | if( doCoverageCheck && pCheck->mxErr>0 ){ /* For leaf pages, the min-heap has already been initialized and the ** cells have already been inserted. But for internal pages, that has ** not yet been done, so do it now */ if( !pPage->leaf ){ heap = pCheck->heap; heap[0] = 0; for(i=nCell-1; i>=0; i--){ u32 size; pc = get2byteAligned(&data[cellStart+i*2]); size = pPage->xCellSize(pPage, &data[pc]); btreeHeapInsert(heap, (pc<<16)|(pc+size-1)); } } /* Add the freeblocks to the min-heap ** ** EVIDENCE-OF: R-20690-50594 The second field of the b-tree page header ** is the offset of the first freeblock, or zero if there are no ** freeblocks on the page. */ i = get2byte(&data[hdr+1]); while( i>0 ){ int size, j; assert( (u32)i<=usableSize-4 ); /* Enforced by btreeInitPage() */ size = get2byte(&data[i+2]); assert( (u32)(i+size)<=usableSize ); /* Enforced by btreeInitPage() */ btreeHeapInsert(heap, (((u32)i)<<16)|(i+size-1)); /* EVIDENCE-OF: R-58208-19414 The first 2 bytes of a freeblock are a ** big-endian integer which is the offset in the b-tree page of the next ** freeblock in the chain, or zero if the freeblock is the last on the ** chain. */ j = get2byte(&data[i]); /* EVIDENCE-OF: R-06866-39125 Freeblocks are always connected in order of ** increasing offset. */ assert( j==0 || j>i+size ); /* Enforced by btreeInitPage() */ assert( (u32)j<=usableSize-4 ); /* Enforced by btreeInitPage() */ i = j; } /* Analyze the min-heap looking for overlap between cells and/or ** freeblocks, and counting the number of untracked bytes in nFrag. ** ** Each min-heap entry is of the form: (start_address<<16)|end_address. ** There is an implied first entry the covers the page header, the cell ** pointer index, and the gap between the cell pointer index and the start ** of cell content. ** ** The loop below pulls entries from the min-heap in order and compares ** the start_address against the previous end_address. If there is an ** overlap, that means bytes are used multiple times. If there is a gap, ** that gap is added to the fragmentation count. */ nFrag = 0; prev = contentOffset - 1; /* Implied first min-heap entry */ while( btreeHeapPull(heap,&x) ){ if( (prev&0xffff)>=(x>>16) ){ checkAppendMsg(pCheck, "Multiple uses for byte %u of page %d", x>>16, iPage); break; }else{ nFrag += (x>>16) - (prev&0xffff) - 1; prev = x; } |
︙ | ︙ | |||
9158 9159 9160 9161 9162 9163 9164 9165 9166 9167 9168 9169 9170 9171 | checkAppendMsg(pCheck, "Fragmentation of %d bytes reported as %d on page %d", nFrag, data[hdr+7], iPage); } } end_of_check: releasePage(pPage); pCheck->zPfx = saved_zPfx; pCheck->v1 = saved_v1; pCheck->v2 = saved_v2; return depth+1; } #endif /* SQLITE_OMIT_INTEGRITY_CHECK */ | > | 9166 9167 9168 9169 9170 9171 9172 9173 9174 9175 9176 9177 9178 9179 9180 | checkAppendMsg(pCheck, "Fragmentation of %d bytes reported as %d on page %d", nFrag, data[hdr+7], iPage); } } end_of_check: if( !doCoverageCheck ) pPage->isInit = savedIsInit; releasePage(pPage); pCheck->zPfx = saved_zPfx; pCheck->v1 = saved_v1; pCheck->v2 = saved_v2; return depth+1; } #endif /* SQLITE_OMIT_INTEGRITY_CHECK */ |
︙ | ︙ |
Changes to src/global.c.
︙ | ︙ | |||
182 183 184 185 186 187 188 | SQLITE_DEFAULT_MMAP_SIZE, /* szMmap */ SQLITE_MAX_MMAP_SIZE, /* mxMmap */ (void*)0, /* pScratch */ 0, /* szScratch */ 0, /* nScratch */ (void*)0, /* pPage */ 0, /* szPage */ | | | 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 | SQLITE_DEFAULT_MMAP_SIZE, /* szMmap */ SQLITE_MAX_MMAP_SIZE, /* mxMmap */ (void*)0, /* pScratch */ 0, /* szScratch */ 0, /* nScratch */ (void*)0, /* pPage */ 0, /* szPage */ SQLITE_DEFAULT_PCACHE_INITSZ, /* nPage */ 0, /* mxParserStack */ 0, /* sharedCacheEnabled */ SQLITE_SORTER_PMASZ, /* szPma */ /* All the rest should always be initialized to zero */ 0, /* isInit */ 0, /* inProgress */ 0, /* isMutexInit */ |
︙ | ︙ |
Changes to src/malloc.c.
︙ | ︙ | |||
189 190 191 192 193 194 195 | }else{ mem0.pScratchEnd = 0; sqlite3GlobalConfig.pScratch = 0; sqlite3GlobalConfig.szScratch = 0; sqlite3GlobalConfig.nScratch = 0; } if( sqlite3GlobalConfig.pPage==0 || sqlite3GlobalConfig.szPage<512 | | < | 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 | }else{ mem0.pScratchEnd = 0; sqlite3GlobalConfig.pScratch = 0; sqlite3GlobalConfig.szScratch = 0; sqlite3GlobalConfig.nScratch = 0; } if( sqlite3GlobalConfig.pPage==0 || sqlite3GlobalConfig.szPage<512 || sqlite3GlobalConfig.nPage<=0 ){ sqlite3GlobalConfig.pPage = 0; sqlite3GlobalConfig.szPage = 0; } rc = sqlite3GlobalConfig.m.xInit(sqlite3GlobalConfig.m.pAppData); if( rc!=SQLITE_OK ) memset(&mem0, 0, sizeof(mem0)); return rc; } /* |
︙ | ︙ |
Changes to src/mutex_noop.c.
︙ | ︙ | |||
103 104 105 106 107 108 109 | /* ** The sqlite3_mutex_alloc() routine allocates a new ** mutex and returns a pointer to it. If it returns NULL ** that means that a mutex could not be allocated. */ static sqlite3_mutex *debugMutexAlloc(int id){ | | | 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | /* ** The sqlite3_mutex_alloc() routine allocates a new ** mutex and returns a pointer to it. If it returns NULL ** that means that a mutex could not be allocated. */ static sqlite3_mutex *debugMutexAlloc(int id){ static sqlite3_debug_mutex aStatic[SQLITE_MUTEX_STATIC_VFS3 - 1]; sqlite3_debug_mutex *pNew = 0; switch( id ){ case SQLITE_MUTEX_FAST: case SQLITE_MUTEX_RECURSIVE: { pNew = sqlite3Malloc(sizeof(*pNew)); if( pNew ){ pNew->id = id; |
︙ | ︙ |
Changes to src/mutex_unix.c.
︙ | ︙ | |||
101 102 103 104 105 106 107 108 109 110 111 112 113 114 | ** <li> SQLITE_MUTEX_STATIC_OPEN ** <li> SQLITE_MUTEX_STATIC_PRNG ** <li> SQLITE_MUTEX_STATIC_LRU ** <li> SQLITE_MUTEX_STATIC_PMEM ** <li> SQLITE_MUTEX_STATIC_APP1 ** <li> SQLITE_MUTEX_STATIC_APP2 ** <li> SQLITE_MUTEX_STATIC_APP3 ** </ul> ** ** The first two constants cause sqlite3_mutex_alloc() to create ** a new mutex. The new mutex is recursive when SQLITE_MUTEX_RECURSIVE ** is used but not necessarily so when SQLITE_MUTEX_FAST is used. ** The mutex implementation does not need to make a distinction ** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does | > > > | 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | ** <li> SQLITE_MUTEX_STATIC_OPEN ** <li> SQLITE_MUTEX_STATIC_PRNG ** <li> SQLITE_MUTEX_STATIC_LRU ** <li> SQLITE_MUTEX_STATIC_PMEM ** <li> SQLITE_MUTEX_STATIC_APP1 ** <li> SQLITE_MUTEX_STATIC_APP2 ** <li> SQLITE_MUTEX_STATIC_APP3 ** <li> SQLITE_MUTEX_STATIC_VFS1 ** <li> SQLITE_MUTEX_STATIC_VFS2 ** <li> SQLITE_MUTEX_STATIC_VFS3 ** </ul> ** ** The first two constants cause sqlite3_mutex_alloc() to create ** a new mutex. The new mutex is recursive when SQLITE_MUTEX_RECURSIVE ** is used but not necessarily so when SQLITE_MUTEX_FAST is used. ** The mutex implementation does not need to make a distinction ** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does |
︙ | ︙ | |||
129 130 131 132 133 134 135 136 137 138 139 140 141 142 | ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc() ** returns a different mutex on every call. But for the static ** mutex types, the same mutex is returned on every call that has ** the same type number. */ static sqlite3_mutex *pthreadMutexAlloc(int iType){ static sqlite3_mutex staticMutexes[] = { SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, | > > > | 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 | ** or SQLITE_MUTEX_RECURSIVE) is used then sqlite3_mutex_alloc() ** returns a different mutex on every call. But for the static ** mutex types, the same mutex is returned on every call that has ** the same type number. */ static sqlite3_mutex *pthreadMutexAlloc(int iType){ static sqlite3_mutex staticMutexes[] = { SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, |
︙ | ︙ |
Changes to src/mutex_w32.c.
︙ | ︙ | |||
85 86 87 88 89 90 91 92 93 94 95 96 97 98 | SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER }; static int winMutex_isInit = 0; static int winMutex_isNt = -1; /* <0 means "need to query" */ /* As the winMutexInit() and winMutexEnd() functions are called as part | > > > | 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 | SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER, SQLITE3_MUTEX_INITIALIZER }; static int winMutex_isInit = 0; static int winMutex_isNt = -1; /* <0 means "need to query" */ /* As the winMutexInit() and winMutexEnd() functions are called as part |
︙ | ︙ | |||
156 157 158 159 160 161 162 163 164 165 166 167 168 169 | ** <li> SQLITE_MUTEX_STATIC_OPEN ** <li> SQLITE_MUTEX_STATIC_PRNG ** <li> SQLITE_MUTEX_STATIC_LRU ** <li> SQLITE_MUTEX_STATIC_PMEM ** <li> SQLITE_MUTEX_STATIC_APP1 ** <li> SQLITE_MUTEX_STATIC_APP2 ** <li> SQLITE_MUTEX_STATIC_APP3 ** </ul> ** ** The first two constants cause sqlite3_mutex_alloc() to create ** a new mutex. The new mutex is recursive when SQLITE_MUTEX_RECURSIVE ** is used but not necessarily so when SQLITE_MUTEX_FAST is used. ** The mutex implementation does not need to make a distinction ** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does | > > > | 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 | ** <li> SQLITE_MUTEX_STATIC_OPEN ** <li> SQLITE_MUTEX_STATIC_PRNG ** <li> SQLITE_MUTEX_STATIC_LRU ** <li> SQLITE_MUTEX_STATIC_PMEM ** <li> SQLITE_MUTEX_STATIC_APP1 ** <li> SQLITE_MUTEX_STATIC_APP2 ** <li> SQLITE_MUTEX_STATIC_APP3 ** <li> SQLITE_MUTEX_STATIC_VFS1 ** <li> SQLITE_MUTEX_STATIC_VFS2 ** <li> SQLITE_MUTEX_STATIC_VFS3 ** </ul> ** ** The first two constants cause sqlite3_mutex_alloc() to create ** a new mutex. The new mutex is recursive when SQLITE_MUTEX_RECURSIVE ** is used but not necessarily so when SQLITE_MUTEX_FAST is used. ** The mutex implementation does not need to make a distinction ** between SQLITE_MUTEX_RECURSIVE and SQLITE_MUTEX_FAST if it does |
︙ | ︙ |
Changes to src/os_unix.c.
︙ | ︙ | |||
625 626 627 628 629 630 631 | ** statements. e.g. ** ** unixEnterMutex() ** assert( unixMutexHeld() ); ** unixEnterLeave() */ static void unixEnterMutex(void){ | | | | | 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 | ** statements. e.g. ** ** unixEnterMutex() ** assert( unixMutexHeld() ); ** unixEnterLeave() */ static void unixEnterMutex(void){ sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1)); } static void unixLeaveMutex(void){ sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1)); } #ifdef SQLITE_DEBUG static int unixMutexHeld(void) { return sqlite3_mutex_held(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1)); } #endif #ifdef SQLITE_HAVE_OS_TRACE /* ** Helper function for printing out trace information from debugging |
︙ | ︙ |
Changes to src/os_win.c.
︙ | ︙ | |||
3386 3387 3388 3389 3390 3391 3392 | ** statements. e.g. ** ** winShmEnterMutex() ** assert( winShmMutexHeld() ); ** winShmLeaveMutex() */ static void winShmEnterMutex(void){ | | | | | 3386 3387 3388 3389 3390 3391 3392 3393 3394 3395 3396 3397 3398 3399 3400 3401 3402 3403 3404 3405 3406 3407 | ** statements. e.g. ** ** winShmEnterMutex() ** assert( winShmMutexHeld() ); ** winShmLeaveMutex() */ static void winShmEnterMutex(void){ sqlite3_mutex_enter(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1)); } static void winShmLeaveMutex(void){ sqlite3_mutex_leave(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1)); } #ifndef NDEBUG static int winShmMutexHeld(void) { return sqlite3_mutex_held(sqlite3MutexAlloc(SQLITE_MUTEX_STATIC_VFS1)); } #endif /* ** Object used to represent a single file opened and mmapped to provide ** shared memory. When multiple threads all reference the same ** log-summary, each thread has its own winFile object, but they all |
︙ | ︙ |
Changes to src/pcache1.c.
︙ | ︙ | |||
11 12 13 14 15 16 17 18 | ************************************************************************* ** ** This file implements the default page cache implementation (the ** sqlite3_pcache interface). It also contains part of the implementation ** of the SQLITE_CONFIG_PAGECACHE and sqlite3_release_memory() features. ** If the default page cache implementation is overridden, then neither of ** these two features are available. */ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > < | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 | ************************************************************************* ** ** This file implements the default page cache implementation (the ** sqlite3_pcache interface). It also contains part of the implementation ** of the SQLITE_CONFIG_PAGECACHE and sqlite3_release_memory() features. ** If the default page cache implementation is overridden, then neither of ** these two features are available. ** ** A Page cache line looks like this: ** ** ------------------------------------------------------------- ** | database page content | PgHdr1 | MemPage | PgHdr | ** ------------------------------------------------------------- ** ** The database page content is up front (so that buffer overreads tend to ** flow harmlessly into the PgHdr1, MemPage, and PgHdr extensions). MemPage ** is the extension added by the btree.c module containing information such ** as the database page number and how that database page is used. PgHdr ** is added by the pcache.c layer and contains information used to keep track ** of which pages are "dirty". PgHdr1 is an extension added by this ** module (pcache1.c). The PgHdr1 header is a subclass of sqlite3_pcache_page. ** PgHdr1 contains information needed to look up a page by its page number. ** The superclass sqlite3_pcache_page.pBuf points to the start of the ** database page content and sqlite3_pcache_page.pExtra points to PgHdr. ** ** The size of the extension (MemPage+PgHdr+PgHdr1) can be determined at ** runtime using sqlite3_config(SQLITE_CONFIG_PCACHE_HDRSZ, &size). The ** sizes of the extensions sum to 272 bytes on x64 for 3.8.10, but this ** size can vary according to architecture, compile-time options, and ** SQLite library version number. ** ** If SQLITE_PCACHE_SEPARATE_HEADER is defined, then the extension is obtained ** using a separate memory allocation from the database page content. This ** seeks to overcome the "clownshoe" problem (also called "internal ** fragmentation" in academic literature) of allocating a few bytes more ** than a power of two with the memory allocator rounding up to the next ** power of two, and leaving the rounded-up space unused. ** ** This module tracks pointers to PgHdr1 objects. Only pcache.c communicates ** with this module. Information is passed back and forth as PgHdr1 pointers. ** ** The pcache.c and pager.c modules deal pointers to PgHdr objects. ** The btree.c module deals with pointers to MemPage objects. ** ** SOURCE OF PAGE CACHE MEMORY: ** ** Memory for a page might come from any of three sources: ** ** (1) The general-purpose memory allocator - sqlite3Malloc() ** (2) Global page-cache memory provided using sqlite3_config() with ** SQLITE_CONFIG_PAGECACHE. ** (3) PCache-local bulk allocation. ** ** The third case is a chunk of heap memory (defaulting to 100 pages worth) ** that is allocated when the page cache is created. The size of the local ** bulk allocation can be adjusted using ** ** sqlite3_config(SQLITE_CONFIG_PCACHE, 0, 0, N). ** ** If N is positive, then N pages worth of memory are allocated using a single ** sqlite3Malloc() call and that memory is used for the first N pages allocated. ** Or if N is negative, then -1024*N bytes of memory are allocated and used ** for as many pages as can be accomodated. ** ** Only one of (2) or (3) can be used. Once the memory available to (2) or ** (3) is exhausted, subsequent allocations fail over to the general-purpose ** memory allocator (1). ** ** Earlier versions of SQLite used only methods (1) and (2). But experiments ** show that method (3) with N==100 provides about a 5% performance boost for ** common workloads. */ #include "sqliteInt.h" typedef struct PCache1 PCache1; typedef struct PgHdr1 PgHdr1; typedef struct PgFreeslot PgFreeslot; typedef struct PGroup PGroup; |
︙ | ︙ | |||
66 67 68 69 70 71 72 | struct PCache1 { /* Cache configuration parameters. Page size (szPage) and the purgeable ** flag (bPurgeable) are set when the cache is created. nMax may be ** modified at any time by a call to the pcache1Cachesize() method. ** The PGroup mutex must be held when accessing nMax. */ PGroup *pGroup; /* PGroup this cache belongs to */ | | | > > > > | | > | | 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 | struct PCache1 { /* Cache configuration parameters. Page size (szPage) and the purgeable ** flag (bPurgeable) are set when the cache is created. nMax may be ** modified at any time by a call to the pcache1Cachesize() method. ** The PGroup mutex must be held when accessing nMax. */ PGroup *pGroup; /* PGroup this cache belongs to */ int szPage; /* Size of database content section */ int szExtra; /* sizeof(MemPage)+sizeof(PgHdr) */ int szAlloc; /* Total size of one pcache line */ int bPurgeable; /* True if cache is purgeable */ unsigned int nMin; /* Minimum number of pages reserved */ unsigned int nMax; /* Configured "cache_size" value */ unsigned int n90pct; /* nMax*9/10 */ unsigned int iMaxKey; /* Largest key seen since xTruncate() */ /* Hash table of all pages. The following variables may only be accessed ** when the accessor is holding the PGroup mutex. */ unsigned int nRecyclable; /* Number of pages in the LRU list */ unsigned int nPage; /* Total number of pages in apHash */ unsigned int nHash; /* Number of slots in apHash[] */ PgHdr1 **apHash; /* Hash table for fast lookup by key */ PgHdr1 *pFree; /* List of unused pcache-local pages */ void *pBulk; /* Bulk memory used by pcache-local */ }; /* ** Each cache entry is represented by an instance of the following ** structure. Unless SQLITE_PCACHE_SEPARATE_HEADER is defined, a buffer of ** PgHdr1.pCache->szPage bytes is allocated directly before this structure ** in memory. */ struct PgHdr1 { sqlite3_pcache_page page; unsigned int iKey; /* Key value (page number) */ u8 isPinned; /* Page in use, not on the LRU list */ u8 isBulkLocal; /* This page from bulk local storage */ PgHdr1 *pNext; /* Next in hash table chain */ PCache1 *pCache; /* Cache that currently owns this page */ PgHdr1 *pLruNext; /* Next in LRU list of unpinned pages */ PgHdr1 *pLruPrev; /* Previous in LRU list of unpinned pages */ }; /* ** Free slots in the allocator used to divide up the global page cache ** buffer provided using the SQLITE_CONFIG_PAGECACHE mechanism. */ struct PgFreeslot { PgFreeslot *pNext; /* Next free slot */ }; /* ** Global data used by this cache. */ static SQLITE_WSD struct PCacheGlobal { PGroup grp; /* The global PGroup for mode (2) */ /* Variables related to SQLITE_CONFIG_PAGECACHE settings. The ** szSlot, nSlot, pStart, pEnd, nReserve, and isInit values are all ** fixed at sqlite3_initialize() time and do not require mutex protection. ** The nFreeSlot and pFree values do require mutex protection. */ int isInit; /* True if initialized */ int separateCache; /* Use a new PGroup for each PCache */ int szSlot; /* Size of each free slot */ int nSlot; /* The number of pcache slots */ int nReserve; /* Try to keep nFreeSlot above this */ void *pStart, *pEnd; /* Bounds of global page cache memory */ /* Above requires no mutex. Use mutex below for variable that follow. */ sqlite3_mutex *mutex; /* Mutex for accessing the following: */ PgFreeslot *pFree; /* Free page blocks */ int nFreeSlot; /* Number of unused pcache slots */ /* The following value requires a mutex to change. We skip the mutex on ** reading because (1) most platforms read a 32-bit integer atomically and ** (2) even if an incorrect value is read, no great harm is done since this |
︙ | ︙ | |||
169 170 171 172 173 174 175 176 177 178 179 180 181 182 | ** ** This routine is called from sqlite3_initialize() and so it is guaranteed ** to be serialized already. There is no need for further mutexing. */ void sqlite3PCacheBufferSetup(void *pBuf, int sz, int n){ if( pcache1.isInit ){ PgFreeslot *p; sz = ROUNDDOWN8(sz); pcache1.szSlot = sz; pcache1.nSlot = pcache1.nFreeSlot = n; pcache1.nReserve = n>90 ? 10 : (n/10 + 1); pcache1.pStart = pBuf; pcache1.pFree = 0; pcache1.bUnderPressure = 0; | > | 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 | ** ** This routine is called from sqlite3_initialize() and so it is guaranteed ** to be serialized already. There is no need for further mutexing. */ void sqlite3PCacheBufferSetup(void *pBuf, int sz, int n){ if( pcache1.isInit ){ PgFreeslot *p; if( pBuf==0 ) sz = n = 0; sz = ROUNDDOWN8(sz); pcache1.szSlot = sz; pcache1.nSlot = pcache1.nFreeSlot = n; pcache1.nReserve = n>90 ? 10 : (n/10 + 1); pcache1.pStart = pBuf; pcache1.pFree = 0; pcache1.bUnderPressure = 0; |
︙ | ︙ | |||
233 234 235 236 237 238 239 | } return p; } /* ** Free an allocated buffer obtained from pcache1Alloc(). */ | | | < > < | 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 | } return p; } /* ** Free an allocated buffer obtained from pcache1Alloc(). */ static void pcache1Free(void *p){ int nFreed = 0; if( p==0 ) return; if( p>=pcache1.pStart && p<pcache1.pEnd ){ PgFreeslot *pSlot; sqlite3_mutex_enter(pcache1.mutex); sqlite3StatusDown(SQLITE_STATUS_PAGECACHE_USED, 1); pSlot = (PgFreeslot*)p; pSlot->pNext = pcache1.pFree; pcache1.pFree = pSlot; pcache1.nFreeSlot++; pcache1.bUnderPressure = pcache1.nFreeSlot<pcache1.nReserve; assert( pcache1.nFreeSlot<=pcache1.nSlot ); sqlite3_mutex_leave(pcache1.mutex); }else{ assert( sqlite3MemdebugHasType(p, MEMTYPE_PCACHE) ); sqlite3MemdebugSetType(p, MEMTYPE_HEAP); #ifndef SQLITE_DISABLE_PAGECACHE_OVERFLOW_STATS nFreed = sqlite3MallocSize(p); sqlite3_mutex_enter(pcache1.mutex); sqlite3StatusDown(SQLITE_STATUS_PAGECACHE_OVERFLOW, nFreed); sqlite3_mutex_leave(pcache1.mutex); #endif sqlite3_free(p); } } #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT /* ** Return the size of a pcache allocation */ static int pcache1MemSize(void *p){ |
︙ | ︙ | |||
286 287 288 289 290 291 292 | /* ** Allocate a new page object initially associated with cache pCache. */ static PgHdr1 *pcache1AllocPage(PCache1 *pCache){ PgHdr1 *p = 0; void *pPg; | > > > > > > > | | | > | | > | | | | | | | | | > | | | > > | | | | < < < < < < > | | | > > > > > | | < | 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 | /* ** Allocate a new page object initially associated with cache pCache. */ static PgHdr1 *pcache1AllocPage(PCache1 *pCache){ PgHdr1 *p = 0; void *pPg; assert( sqlite3_mutex_held(pCache->pGroup->mutex) ); if( pCache->pFree ){ p = pCache->pFree; pCache->pFree = p->pNext; p->pNext = 0; }else{ #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT /* The group mutex must be released before pcache1Alloc() is called. This ** is because it might call sqlite3_release_memory(), which assumes that ** this mutex is not held. */ assert( pcache1.separateCache==0 ); assert( pCache->pGroup==&pcache1.grp ); pcache1LeaveMutex(pCache->pGroup); #endif #ifdef SQLITE_PCACHE_SEPARATE_HEADER pPg = pcache1Alloc(pCache->szPage); p = sqlite3Malloc(sizeof(PgHdr1) + pCache->szExtra); if( !pPg || !p ){ pcache1Free(pPg); sqlite3_free(p); pPg = 0; } #else pPg = pcache1Alloc(pCache->szAlloc); p = (PgHdr1 *)&((u8 *)pPg)[pCache->szPage]; #endif #ifdef SQLITE_ENABLE_MEMORY_MANAGEMENT pcache1EnterMutex(pCache->pGroup); #endif if( pPg==0 ) return 0; p->page.pBuf = pPg; p->page.pExtra = &p[1]; p->isBulkLocal = 0; } if( pCache->bPurgeable ){ pCache->pGroup->nCurrentPage++; } return p; } /* ** Free a page object allocated by pcache1AllocPage(). */ static void pcache1FreePage(PgHdr1 *p){ PCache1 *pCache; assert( p!=0 ); pCache = p->pCache; assert( sqlite3_mutex_held(p->pCache->pGroup->mutex) ); if( p->isBulkLocal ){ p->pNext = pCache->pFree; pCache->pFree = p; }else{ pcache1Free(p->page.pBuf); #ifdef SQLITE_PCACHE_SEPARATE_HEADER sqlite3_free(p); #endif } if( pCache->bPurgeable ){ pCache->pGroup->nCurrentPage--; } } /* ** Malloc function used by SQLite to obtain space from the buffer configured ** using sqlite3_config(SQLITE_CONFIG_PAGECACHE) option. If no such buffer ** exists, this function falls back to sqlite3Malloc(). |
︙ | ︙ | |||
533 534 535 536 537 538 539 540 541 542 543 544 545 546 | /* ** Implementation of the sqlite3_pcache.xInit method. */ static int pcache1Init(void *NotUsed){ UNUSED_PARAMETER(NotUsed); assert( pcache1.isInit==0 ); memset(&pcache1, 0, sizeof(pcache1)); #if SQLITE_THREADSAFE if( sqlite3GlobalConfig.bCoreMutex ){ pcache1.grp.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_LRU); pcache1.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_PMEM); } #endif pcache1.grp.mxPinned = 10; | > > > > > > > > > > > > > > > > > > > > > > > > > | 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 | /* ** Implementation of the sqlite3_pcache.xInit method. */ static int pcache1Init(void *NotUsed){ UNUSED_PARAMETER(NotUsed); assert( pcache1.isInit==0 ); memset(&pcache1, 0, sizeof(pcache1)); /* ** The pcache1.separateCache variable is true if each PCache has its own ** private PGroup (mode-1). pcache1.separateCache is false if the single ** PGroup in pcache1.grp is used for all page caches (mode-2). ** ** * Always use a unified cache (mode-2) if ENABLE_MEMORY_MANAGEMENT ** ** * Use a unified cache in single-threaded applications that have ** configured a start-time buffer for use as page-cache memory using ** sqlite3_config(SQLITE_CONFIG_PAGECACHE, pBuf, sz, N) with non-NULL ** pBuf argument. ** ** * Otherwise use separate caches (mode-1) */ #if defined(SQLITE_ENABLE_MEMORY_MANAGEMENT) pcache1.separateCache = 0; #elif SQLITE_THREADSAFE pcache1.separateCache = sqlite3GlobalConfig.pPage==0 || sqlite3GlobalConfig.bCoreMutex>0; #else pcache1.separateCache = sqlite3GlobalConfig.pPage==0; #endif #if SQLITE_THREADSAFE if( sqlite3GlobalConfig.bCoreMutex ){ pcache1.grp.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_LRU); pcache1.mutex = sqlite3_mutex_alloc(SQLITE_MUTEX_STATIC_PMEM); } #endif pcache1.grp.mxPinned = 10; |
︙ | ︙ | |||
568 569 570 571 572 573 574 | ** Allocate a new cache. */ static sqlite3_pcache *pcache1Create(int szPage, int szExtra, int bPurgeable){ PCache1 *pCache; /* The newly created page cache */ PGroup *pGroup; /* The group the new page cache will belong to */ int sz; /* Bytes of memory required to allocate the new cache */ | < < < < < < < < < < < < < < < < < < | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 | ** Allocate a new cache. */ static sqlite3_pcache *pcache1Create(int szPage, int szExtra, int bPurgeable){ PCache1 *pCache; /* The newly created page cache */ PGroup *pGroup; /* The group the new page cache will belong to */ int sz; /* Bytes of memory required to allocate the new cache */ assert( (szPage & (szPage-1))==0 && szPage>=512 && szPage<=65536 ); assert( szExtra < 300 ); sz = sizeof(PCache1) + sizeof(PGroup)*pcache1.separateCache; pCache = (PCache1 *)sqlite3MallocZero(sz); if( pCache ){ if( pcache1.separateCache ){ pGroup = (PGroup*)&pCache[1]; pGroup->mxPinned = 10; }else{ pGroup = &pcache1.grp; } pCache->pGroup = pGroup; pCache->szPage = szPage; pCache->szExtra = szExtra; pCache->szAlloc = szPage + szExtra + ROUND8(sizeof(PgHdr1)); pCache->bPurgeable = (bPurgeable ? 1 : 0); pcache1EnterMutex(pGroup); pcache1ResizeHash(pCache); if( bPurgeable ){ pCache->nMin = 10; pGroup->nMinPage += pCache->nMin; pGroup->mxPinned = pGroup->nMaxPage + 10 - pGroup->nMinPage; } pcache1LeaveMutex(pGroup); /* Try to initialize the local bulk pagecache line allocation if using ** separate caches and if nPage!=0 */ if( pcache1.separateCache && sqlite3GlobalConfig.nPage!=0 && sqlite3GlobalConfig.pPage==0 ){ int szBulk; char *zBulk; sqlite3BeginBenignMalloc(); if( sqlite3GlobalConfig.nPage>0 ){ szBulk = pCache->szAlloc * sqlite3GlobalConfig.nPage; }else{ szBulk = -1024*sqlite3GlobalConfig.nPage; } zBulk = pCache->pBulk = sqlite3Malloc( szBulk ); sqlite3EndBenignMalloc(); if( zBulk ){ int nBulk = sqlite3MallocSize(zBulk)/pCache->szAlloc; int i; for(i=0; i<nBulk; i++){ PgHdr1 *pX = (PgHdr1*)&zBulk[szPage]; pX->page.pBuf = zBulk; pX->page.pExtra = &pX[1]; pX->isBulkLocal = 1; pX->pNext = pCache->pFree; pCache->pFree = pX; zBulk += pCache->szAlloc; } } } if( pCache->nHash==0 ){ pcache1Destroy((sqlite3_pcache*)pCache); pCache = 0; } } return (sqlite3_pcache *)pCache; } |
︙ | ︙ | |||
703 704 705 706 707 708 709 | return 0; } if( pCache->nPage>=pCache->nHash ) pcache1ResizeHash(pCache); assert( pCache->nHash>0 && pCache->apHash ); /* Step 4. Try to recycle a page. */ | | < | | | < < < < < < | < < | 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 | return 0; } if( pCache->nPage>=pCache->nHash ) pcache1ResizeHash(pCache); assert( pCache->nHash>0 && pCache->apHash ); /* Step 4. Try to recycle a page. */ if( pCache->bPurgeable && pGroup->pLruTail && ((pCache->nPage+1>=pCache->nMax) || pcache1UnderMemoryPressure(pCache)) ){ PCache1 *pOther; pPage = pGroup->pLruTail; assert( pPage->isPinned==0 ); pcache1RemoveFromHash(pPage, 0); pcache1PinPage(pPage); pOther = pPage->pCache; if( pOther->szAlloc != pCache->szAlloc ){ pcache1FreePage(pPage); pPage = 0; }else{ pGroup->nCurrentPage -= (pOther->bPurgeable - pCache->bPurgeable); } } |
︙ | ︙ | |||
998 999 1000 1001 1002 1003 1004 1005 1006 1007 1008 1009 1010 1011 | assert( pGroup->nMaxPage >= pCache->nMax ); pGroup->nMaxPage -= pCache->nMax; assert( pGroup->nMinPage >= pCache->nMin ); pGroup->nMinPage -= pCache->nMin; pGroup->mxPinned = pGroup->nMaxPage + 10 - pGroup->nMinPage; pcache1EnforceMaxPage(pGroup); pcache1LeaveMutex(pGroup); sqlite3_free(pCache->apHash); sqlite3_free(pCache); } /* ** This function is called during initialization (sqlite3_initialize()) to ** install the default pluggable cache module, assuming the user has not | > | 1106 1107 1108 1109 1110 1111 1112 1113 1114 1115 1116 1117 1118 1119 1120 | assert( pGroup->nMaxPage >= pCache->nMax ); pGroup->nMaxPage -= pCache->nMax; assert( pGroup->nMinPage >= pCache->nMin ); pGroup->nMinPage -= pCache->nMin; pGroup->mxPinned = pGroup->nMaxPage + 10 - pGroup->nMinPage; pcache1EnforceMaxPage(pGroup); pcache1LeaveMutex(pGroup); sqlite3_free(pCache->pBulk); sqlite3_free(pCache->apHash); sqlite3_free(pCache); } /* ** This function is called during initialization (sqlite3_initialize()) to ** install the default pluggable cache module, assuming the user has not |
︙ | ︙ | |||
1053 1054 1055 1056 1057 1058 1059 | ** been released, the function returns. The return value is the total number ** of bytes of memory released. */ int sqlite3PcacheReleaseMemory(int nReq){ int nFree = 0; assert( sqlite3_mutex_notheld(pcache1.grp.mutex) ); assert( sqlite3_mutex_notheld(pcache1.mutex) ); | | | 1162 1163 1164 1165 1166 1167 1168 1169 1170 1171 1172 1173 1174 1175 1176 | ** been released, the function returns. The return value is the total number ** of bytes of memory released. */ int sqlite3PcacheReleaseMemory(int nReq){ int nFree = 0; assert( sqlite3_mutex_notheld(pcache1.grp.mutex) ); assert( sqlite3_mutex_notheld(pcache1.mutex) ); if( sqlite3GlobalConfig.nPage==0 ){ PgHdr1 *p; pcache1EnterMutex(&pcache1.grp); while( (nReq<0 || nFree<nReq) && ((p=pcache1.grp.pLruTail)!=0) ){ nFree += pcache1MemSize(p->page.pBuf); #ifdef SQLITE_PCACHE_SEPARATE_HEADER nFree += sqlite3MemSize(p); #endif |
︙ | ︙ |
Changes to src/select.c.
︙ | ︙ | |||
2066 2067 2068 2069 2070 2071 2072 | VdbeCoverage(v); } sqlite3VdbeResolveLabel(v, addrCont); /* Execute the recursive SELECT taking the single row in Current as ** the value for the recursive-table. Store the results in the Queue. */ | > > > | | | | > | 2066 2067 2068 2069 2070 2071 2072 2073 2074 2075 2076 2077 2078 2079 2080 2081 2082 2083 2084 2085 2086 2087 | VdbeCoverage(v); } sqlite3VdbeResolveLabel(v, addrCont); /* Execute the recursive SELECT taking the single row in Current as ** the value for the recursive-table. Store the results in the Queue. */ if( p->selFlags & SF_Aggregate ){ sqlite3ErrorMsg(pParse, "recursive aggregate queries not supported"); }else{ p->pPrior = 0; sqlite3Select(pParse, p, &destQueue); assert( p->pPrior==0 ); p->pPrior = pSetup; } /* Keep running the loop until the Queue is empty */ sqlite3VdbeAddOp2(v, OP_Goto, 0, addrTop); sqlite3VdbeResolveLabel(v, addrBreak); end_of_recursive_query: sqlite3ExprListDelete(pParse->db, p->pOrderBy); |
︙ | ︙ |
Changes to src/sqlite.h.in.
︙ | ︙ | |||
6287 6288 6289 6290 6291 6292 6293 6294 6295 6296 6297 6298 6299 6300 | #define SQLITE_MUTEX_STATIC_PRNG 5 /* sqlite3_random() */ #define SQLITE_MUTEX_STATIC_LRU 6 /* lru page list */ #define SQLITE_MUTEX_STATIC_LRU2 7 /* NOT USED */ #define SQLITE_MUTEX_STATIC_PMEM 7 /* sqlite3PageMalloc() */ #define SQLITE_MUTEX_STATIC_APP1 8 /* For use by application */ #define SQLITE_MUTEX_STATIC_APP2 9 /* For use by application */ #define SQLITE_MUTEX_STATIC_APP3 10 /* For use by application */ /* ** CAPI3REF: Retrieve the mutex for a database connection ** METHOD: sqlite3 ** ** ^This interface returns a pointer the [sqlite3_mutex] object that ** serializes access to the [database connection] given in the argument | > > > | 6287 6288 6289 6290 6291 6292 6293 6294 6295 6296 6297 6298 6299 6300 6301 6302 6303 | #define SQLITE_MUTEX_STATIC_PRNG 5 /* sqlite3_random() */ #define SQLITE_MUTEX_STATIC_LRU 6 /* lru page list */ #define SQLITE_MUTEX_STATIC_LRU2 7 /* NOT USED */ #define SQLITE_MUTEX_STATIC_PMEM 7 /* sqlite3PageMalloc() */ #define SQLITE_MUTEX_STATIC_APP1 8 /* For use by application */ #define SQLITE_MUTEX_STATIC_APP2 9 /* For use by application */ #define SQLITE_MUTEX_STATIC_APP3 10 /* For use by application */ #define SQLITE_MUTEX_STATIC_VFS1 11 /* For use by built-in VFS */ #define SQLITE_MUTEX_STATIC_VFS2 12 /* For use by extension VFS */ #define SQLITE_MUTEX_STATIC_VFS3 13 /* For use by application VFS */ /* ** CAPI3REF: Retrieve the mutex for a database connection ** METHOD: sqlite3 ** ** ^This interface returns a pointer the [sqlite3_mutex] object that ** serializes access to the [database connection] given in the argument |
︙ | ︙ |
Changes to src/sqliteInt.h.
︙ | ︙ | |||
505 506 507 508 509 510 511 512 513 514 515 516 517 518 | # define SQLITE_DEFAULT_WORKER_THREADS 0 #endif #if SQLITE_DEFAULT_WORKER_THREADS>SQLITE_MAX_WORKER_THREADS # undef SQLITE_MAX_WORKER_THREADS # define SQLITE_MAX_WORKER_THREADS SQLITE_DEFAULT_WORKER_THREADS #endif /* ** GCC does not define the offsetof() macro so we'll have to do it ** ourselves. */ #ifndef offsetof #define offsetof(STRUCTURE,FIELD) ((int)((char*)&((STRUCTURE*)0)->FIELD)) | > > > > > > > > > > | 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 | # define SQLITE_DEFAULT_WORKER_THREADS 0 #endif #if SQLITE_DEFAULT_WORKER_THREADS>SQLITE_MAX_WORKER_THREADS # undef SQLITE_MAX_WORKER_THREADS # define SQLITE_MAX_WORKER_THREADS SQLITE_DEFAULT_WORKER_THREADS #endif /* ** The default initial allocation for the pagecache when using separate ** pagecaches for each database connection. A positive number is the ** number of pages. A negative number N translations means that a buffer ** of -1024*N bytes is allocated and used for as many pages as it will hold. */ #ifndef SQLITE_DEFAULT_PCACHE_INITSZ # define SQLITE_DEFAULT_PCACHE_INITSZ 100 #endif /* ** GCC does not define the offsetof() macro so we'll have to do it ** ourselves. */ #ifndef offsetof #define offsetof(STRUCTURE,FIELD) ((int)((char*)&((STRUCTURE*)0)->FIELD)) |
︙ | ︙ |
Changes to src/test_mutex.c.
︙ | ︙ | |||
15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 | #include "tcl.h" #include "sqlite3.h" #include "sqliteInt.h" #include <stdlib.h> #include <assert.h> #include <string.h> /* defined in main.c */ extern const char *sqlite3ErrName(int); /* A countable mutex */ struct sqlite3_mutex { sqlite3_mutex *pReal; int eType; }; /* State variables */ static struct test_mutex_globals { | > > > > > > > > > > | | | | | | | | 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 | #include "tcl.h" #include "sqlite3.h" #include "sqliteInt.h" #include <stdlib.h> #include <assert.h> #include <string.h> #define MAX_MUTEXES (SQLITE_MUTEX_STATIC_VFS3+1) #define STATIC_MUTEXES (MAX_MUTEXES-(SQLITE_MUTEX_RECURSIVE+1)) /* defined in main.c */ extern const char *sqlite3ErrName(int); static const char *aName[MAX_MUTEXES+1] = { "fast", "recursive", "static_master", "static_mem", "static_open", "static_prng", "static_lru", "static_pmem", "static_app1", "static_app2", "static_app3", "static_vfs1", "static_vfs2", "static_vfs3", 0 }; /* A countable mutex */ struct sqlite3_mutex { sqlite3_mutex *pReal; int eType; }; /* State variables */ static struct test_mutex_globals { int isInstalled; /* True if installed */ int disableInit; /* True to cause sqlite3_initalize() to fail */ int disableTry; /* True to force sqlite3_mutex_try() to fail */ int isInit; /* True if initialized */ sqlite3_mutex_methods m; /* Interface to "real" mutex system */ int aCounter[MAX_MUTEXES]; /* Number of grabs of each type of mutex */ sqlite3_mutex aStatic[STATIC_MUTEXES]; /* The static mutexes */ } g = {0}; /* Return true if the countable mutex is currently held */ static int counterMutexHeld(sqlite3_mutex *p){ return g.m.xMutexHeld(p->pReal); } |
︙ | ︙ | |||
74 75 76 77 78 79 80 | ** Allocate a countable mutex */ static sqlite3_mutex *counterMutexAlloc(int eType){ sqlite3_mutex *pReal; sqlite3_mutex *pRet = 0; assert( g.isInit ); | | > > > > | | 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 | ** Allocate a countable mutex */ static sqlite3_mutex *counterMutexAlloc(int eType){ sqlite3_mutex *pReal; sqlite3_mutex *pRet = 0; assert( g.isInit ); assert( eType>=SQLITE_MUTEX_FAST ); assert( eType<=SQLITE_MUTEX_STATIC_VFS3 ); pReal = g.m.xMutexAlloc(eType); if( !pReal ) return 0; if( eType==SQLITE_MUTEX_FAST || eType==SQLITE_MUTEX_RECURSIVE ){ pRet = (sqlite3_mutex *)malloc(sizeof(sqlite3_mutex)); }else{ int eStaticType = eType - (MAX_MUTEXES - STATIC_MUTEXES); assert( eStaticType>=0 ); assert( eStaticType<STATIC_MUTEXES ); pRet = &g.aStatic[eStaticType]; } pRet->eType = eType; pRet->pReal = pReal; return pRet; } |
︙ | ︙ | |||
106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 | } /* ** Enter a countable mutex. Block until entry is safe. */ static void counterMutexEnter(sqlite3_mutex *p){ assert( g.isInit ); g.aCounter[p->eType]++; g.m.xMutexEnter(p->pReal); } /* ** Try to enter a mutex. Return true on success. */ static int counterMutexTry(sqlite3_mutex *p){ assert( g.isInit ); g.aCounter[p->eType]++; if( g.disableTry ) return SQLITE_BUSY; return g.m.xMutexTry(p->pReal); } /* Leave a mutex */ | > > > > | 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 | } /* ** Enter a countable mutex. Block until entry is safe. */ static void counterMutexEnter(sqlite3_mutex *p){ assert( g.isInit ); assert( p->eType>=0 ); assert( p->eType<MAX_MUTEXES ); g.aCounter[p->eType]++; g.m.xMutexEnter(p->pReal); } /* ** Try to enter a mutex. Return true on success. */ static int counterMutexTry(sqlite3_mutex *p){ assert( g.isInit ); assert( p->eType>=0 ); assert( p->eType<MAX_MUTEXES ); g.aCounter[p->eType]++; if( g.disableTry ) return SQLITE_BUSY; return g.m.xMutexTry(p->pReal); } /* Leave a mutex */ |
︙ | ︙ | |||
241 242 243 244 245 246 247 | void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ Tcl_Obj *pRet; int ii; | < < < < | | 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 | void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ Tcl_Obj *pRet; int ii; if( objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, ""); return TCL_ERROR; } pRet = Tcl_NewObj(); Tcl_IncrRefCount(pRet); for(ii=0; ii<MAX_MUTEXES; ii++){ Tcl_ListObjAppendElement(interp, pRet, Tcl_NewStringObj(aName[ii], -1)); Tcl_ListObjAppendElement(interp, pRet, Tcl_NewIntObj(g.aCounter[ii])); } Tcl_SetObjResult(interp, pRet); Tcl_DecrRefCount(pRet); return TCL_OK; |
︙ | ︙ | |||
279 280 281 282 283 284 285 | int ii; if( objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, ""); return TCL_ERROR; } | | | 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 | int ii; if( objc!=1 ){ Tcl_WrongNumArgs(interp, 1, objv, ""); return TCL_ERROR; } for(ii=0; ii<MAX_MUTEXES; ii++){ g.aCounter[ii] = 0; } return TCL_OK; } /* ** Create and free a mutex. Return the mutex pointer. The pointer |
︙ | ︙ | |||
366 367 368 369 370 371 372 373 374 375 376 377 378 379 | db = *((sqlite3 **)info.objClientData); }else{ db = (sqlite3*)sqlite3TestTextToPtr(zCmd); } assert( db ); return db; } static int test_enter_db_mutex( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 | db = *((sqlite3 **)info.objClientData); }else{ db = (sqlite3*)sqlite3TestTextToPtr(zCmd); } assert( db ); return db; } static sqlite3_mutex *getStaticMutexPointer( Tcl_Interp *pInterp, Tcl_Obj *pObj ){ int iMutex; if( Tcl_GetIndexFromObj(pInterp, pObj, aName, "mutex name", 0, &iMutex) ){ return 0; } assert( iMutex!=SQLITE_MUTEX_FAST && iMutex!=SQLITE_MUTEX_RECURSIVE ); return counterMutexAlloc(iMutex); } static int test_enter_static_mutex( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3_mutex *pMutex; if( objc!=2 ){ Tcl_WrongNumArgs(interp, 1, objv, "NAME"); return TCL_ERROR; } pMutex = getStaticMutexPointer(interp, objv[1]); if( !pMutex ){ return TCL_ERROR; } sqlite3_mutex_enter(pMutex); return TCL_OK; } static int test_leave_static_mutex( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ sqlite3_mutex *pMutex; if( objc!=2 ){ Tcl_WrongNumArgs(interp, 1, objv, "NAME"); return TCL_ERROR; } pMutex = getStaticMutexPointer(interp, objv[1]); if( !pMutex ){ return TCL_ERROR; } sqlite3_mutex_leave(pMutex); return TCL_OK; } static int test_enter_db_mutex( void * clientData, Tcl_Interp *interp, int objc, Tcl_Obj *CONST objv[] ){ |
︙ | ︙ | |||
413 414 415 416 417 418 419 420 421 422 423 424 425 426 | static struct { char *zName; Tcl_ObjCmdProc *xProc; } aCmd[] = { { "sqlite3_shutdown", (Tcl_ObjCmdProc*)test_shutdown }, { "sqlite3_initialize", (Tcl_ObjCmdProc*)test_initialize }, { "sqlite3_config", (Tcl_ObjCmdProc*)test_config }, { "enter_db_mutex", (Tcl_ObjCmdProc*)test_enter_db_mutex }, { "leave_db_mutex", (Tcl_ObjCmdProc*)test_leave_db_mutex }, { "alloc_dealloc_mutex", (Tcl_ObjCmdProc*)test_alloc_mutex }, { "install_mutex_counters", (Tcl_ObjCmdProc*)test_install_mutex_counters }, { "read_mutex_counters", (Tcl_ObjCmdProc*)test_read_mutex_counters }, | > > > | 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 | static struct { char *zName; Tcl_ObjCmdProc *xProc; } aCmd[] = { { "sqlite3_shutdown", (Tcl_ObjCmdProc*)test_shutdown }, { "sqlite3_initialize", (Tcl_ObjCmdProc*)test_initialize }, { "sqlite3_config", (Tcl_ObjCmdProc*)test_config }, { "enter_static_mutex", (Tcl_ObjCmdProc*)test_enter_static_mutex }, { "leave_static_mutex", (Tcl_ObjCmdProc*)test_leave_static_mutex }, { "enter_db_mutex", (Tcl_ObjCmdProc*)test_enter_db_mutex }, { "leave_db_mutex", (Tcl_ObjCmdProc*)test_leave_db_mutex }, { "alloc_dealloc_mutex", (Tcl_ObjCmdProc*)test_alloc_mutex }, { "install_mutex_counters", (Tcl_ObjCmdProc*)test_install_mutex_counters }, { "read_mutex_counters", (Tcl_ObjCmdProc*)test_read_mutex_counters }, |
︙ | ︙ |
Changes to src/util.c.
︙ | ︙ | |||
1078 1079 1080 1081 1082 1083 1084 | ** Read or write a four-byte big-endian integer value. */ u32 sqlite3Get4byte(const u8 *p){ #if SQLITE_BYTEORDER==4321 u32 x; memcpy(&x,p,4); return x; | | | | 1078 1079 1080 1081 1082 1083 1084 1085 1086 1087 1088 1089 1090 1091 1092 1093 1094 1095 1096 1097 1098 1099 1100 1101 1102 1103 1104 1105 1106 1107 1108 | ** Read or write a four-byte big-endian integer value. */ u32 sqlite3Get4byte(const u8 *p){ #if SQLITE_BYTEORDER==4321 u32 x; memcpy(&x,p,4); return x; #elif SQLITE_BYTEORDER==1234 && defined(__GNUC__) && GCC_VERSION>=4003000 u32 x; memcpy(&x,p,4); return __builtin_bswap32(x); #elif SQLITE_BYTEORDER==1234 && defined(_MSC_VER) && _MSC_VER>=1300 u32 x; memcpy(&x,p,4); return _byteswap_ulong(x); #else testcase( p[0]&0x80 ); return ((unsigned)p[0]<<24) | (p[1]<<16) | (p[2]<<8) | p[3]; #endif } void sqlite3Put4byte(unsigned char *p, u32 v){ #if SQLITE_BYTEORDER==4321 memcpy(p,&v,4); #elif SQLITE_BYTEORDER==1234 && defined(__GNUC__) && GCC_VERSION>=4003000 u32 x = __builtin_bswap32(v); memcpy(p,&x,4); #elif SQLITE_BYTEORDER==1234 && defined(_MSC_VER) && _MSC_VER>=1300 u32 x = _byteswap_ulong(v); memcpy(p,&x,4); #else p[0] = (u8)(v>>24); |
︙ | ︙ |
Changes to src/vdbeapi.c.
︙ | ︙ | |||
49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 | sqlite3_log(SQLITE_MISUSE, "API called with NULL prepared statement"); return 1; }else{ return vdbeSafety(p); } } /* ** The following routine destroys a virtual machine that is created by ** the sqlite3_compile() routine. The integer returned is an SQLITE_ ** success/failure code that describes the result of executing the virtual ** machine. ** ** This routine sets the error code and string returned by ** sqlite3_errcode(), sqlite3_errmsg() and sqlite3_errmsg16(). */ int sqlite3_finalize(sqlite3_stmt *pStmt){ int rc; if( pStmt==0 ){ /* IMPLEMENTATION-OF: R-57228-12904 Invoking sqlite3_finalize() on a NULL ** pointer is a harmless no-op. */ rc = SQLITE_OK; }else{ Vdbe *v = (Vdbe*)pStmt; sqlite3 *db = v->db; if( vdbeSafety(v) ) return SQLITE_MISUSE_BKPT; sqlite3_mutex_enter(db->mutex); rc = sqlite3VdbeFinalize(v); rc = sqlite3ApiExit(db, rc); sqlite3LeaveMutexAndCloseZombie(db); } return rc; } | > > > > > > > > > > > > > > > > > > > > > > > > > > | 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 | sqlite3_log(SQLITE_MISUSE, "API called with NULL prepared statement"); return 1; }else{ return vdbeSafety(p); } } #ifndef SQLITE_OMIT_TRACE /* ** Invoke the profile callback. This routine is only called if we already ** know that the profile callback is defined and needs to be invoked. */ static SQLITE_NOINLINE void invokeProfileCallback(sqlite3 *db, Vdbe *p){ sqlite3_int64 iNow; assert( p->startTime>0 ); assert( db->xProfile!=0 ); assert( db->init.busy==0 ); assert( p->zSql!=0 ); sqlite3OsCurrentTimeInt64(db->pVfs, &iNow); db->xProfile(db->pProfileArg, p->zSql, (iNow - p->startTime)*1000000); p->startTime = 0; } /* ** The checkProfileCallback(DB,P) macro checks to see if a profile callback ** is needed, and it invokes the callback if it is needed. */ # define checkProfileCallback(DB,P) \ if( ((P)->startTime)>0 ){ invokeProfileCallback(DB,P); } #else # define checkProfileCallback(DB,P) /*no-op*/ #endif /* ** The following routine destroys a virtual machine that is created by ** the sqlite3_compile() routine. The integer returned is an SQLITE_ ** success/failure code that describes the result of executing the virtual ** machine. ** ** This routine sets the error code and string returned by ** sqlite3_errcode(), sqlite3_errmsg() and sqlite3_errmsg16(). */ int sqlite3_finalize(sqlite3_stmt *pStmt){ int rc; if( pStmt==0 ){ /* IMPLEMENTATION-OF: R-57228-12904 Invoking sqlite3_finalize() on a NULL ** pointer is a harmless no-op. */ rc = SQLITE_OK; }else{ Vdbe *v = (Vdbe*)pStmt; sqlite3 *db = v->db; if( vdbeSafety(v) ) return SQLITE_MISUSE_BKPT; sqlite3_mutex_enter(db->mutex); checkProfileCallback(db, v); rc = sqlite3VdbeFinalize(v); rc = sqlite3ApiExit(db, rc); sqlite3LeaveMutexAndCloseZombie(db); } return rc; } |
︙ | ︙ | |||
90 91 92 93 94 95 96 | */ int sqlite3_reset(sqlite3_stmt *pStmt){ int rc; if( pStmt==0 ){ rc = SQLITE_OK; }else{ Vdbe *v = (Vdbe*)pStmt; | > | > | | | | 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 | */ int sqlite3_reset(sqlite3_stmt *pStmt){ int rc; if( pStmt==0 ){ rc = SQLITE_OK; }else{ Vdbe *v = (Vdbe*)pStmt; sqlite3 *db = v->db; sqlite3_mutex_enter(db->mutex); checkProfileCallback(db, v); rc = sqlite3VdbeReset(v); sqlite3VdbeRewind(v); assert( (rc & (db->errMask))==rc ); rc = sqlite3ApiExit(db, rc); sqlite3_mutex_leave(db->mutex); } return rc; } /* ** Set all the parameters in the compiled SQL statement to NULL. */ |
︙ | ︙ | |||
445 446 447 448 449 450 451 452 453 454 455 456 457 458 | rc = db->xWalCallback(db->pWalArg, db, db->aDb[i].zName, nEntry); } } } #endif return rc; } /* ** Execute the statement pStmt, either until a row of data is ready, the ** statement is completely executed or an error occurs. ** ** This routine implements the bulk of the logic behind the sqlite_step() ** API. The only thing omitted is the automatic recompile if a | > | 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 | rc = db->xWalCallback(db->pWalArg, db, db->aDb[i].zName, nEntry); } } } #endif return rc; } /* ** Execute the statement pStmt, either until a row of data is ready, the ** statement is completely executed or an error occurs. ** ** This routine implements the bulk of the logic behind the sqlite_step() ** API. The only thing omitted is the automatic recompile if a |
︙ | ︙ | |||
514 515 516 517 518 519 520 | } assert( db->nVdbeWrite>0 || db->autoCommit==0 || (db->nDeferredCons==0 && db->nDeferredImmCons==0) ); #ifndef SQLITE_OMIT_TRACE | | > > | 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 | } assert( db->nVdbeWrite>0 || db->autoCommit==0 || (db->nDeferredCons==0 && db->nDeferredImmCons==0) ); #ifndef SQLITE_OMIT_TRACE if( db->xProfile && !db->init.busy && p->zSql ){ sqlite3OsCurrentTimeInt64(db->pVfs, &p->startTime); }else{ assert( p->startTime==0 ); } #endif db->nVdbeActive++; if( p->readOnly==0 ) db->nVdbeWrite++; if( p->bIsReader ) db->nVdbeRead++; p->pc = 0; |
︙ | ︙ | |||
539 540 541 542 543 544 545 | { db->nVdbeExec++; rc = sqlite3VdbeExec(p); db->nVdbeExec--; } #ifndef SQLITE_OMIT_TRACE | | < | < < < < | 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 | { db->nVdbeExec++; rc = sqlite3VdbeExec(p); db->nVdbeExec--; } #ifndef SQLITE_OMIT_TRACE /* If the statement completed successfully, invoke the profile callback */ if( rc!=SQLITE_ROW ) checkProfileCallback(db, p); #endif if( rc==SQLITE_DONE ){ assert( p->rc==SQLITE_OK ); p->rc = doWalCallbacks(db); if( p->rc!=SQLITE_OK ){ rc = SQLITE_ERROR; |
︙ | ︙ |
Changes to src/vdbeaux.c.
︙ | ︙ | |||
3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 | ** the unnecessary initialization has a measurable negative performance ** impact, since this routine is a very high runner. And so, we choose ** to ignore the compiler warnings and leave this variable uninitialized. */ /* mem1.u.i = 0; // not needed, here to silence compiler warning */ idx1 = getVarint32(aKey1, szHdr1); d1 = szHdr1; assert( pKeyInfo->nField+pKeyInfo->nXField>=pPKey2->nField || CORRUPT_DB ); assert( pKeyInfo->aSortOrder!=0 ); assert( pKeyInfo->nField>0 ); assert( idx1<=szHdr1 || CORRUPT_DB ); do{ u32 serial_type1; | > | 3342 3343 3344 3345 3346 3347 3348 3349 3350 3351 3352 3353 3354 3355 3356 | ** the unnecessary initialization has a measurable negative performance ** impact, since this routine is a very high runner. And so, we choose ** to ignore the compiler warnings and leave this variable uninitialized. */ /* mem1.u.i = 0; // not needed, here to silence compiler warning */ idx1 = getVarint32(aKey1, szHdr1); if( szHdr1>98307 ) return SQLITE_CORRUPT; d1 = szHdr1; assert( pKeyInfo->nField+pKeyInfo->nXField>=pPKey2->nField || CORRUPT_DB ); assert( pKeyInfo->aSortOrder!=0 ); assert( pKeyInfo->nField>0 ); assert( idx1<=szHdr1 || CORRUPT_DB ); do{ u32 serial_type1; |
︙ | ︙ |
Changes to test/fts3fault.test.
︙ | ︙ | |||
13 14 15 16 17 18 19 20 21 22 23 24 25 26 | set testdir [file dirname $argv0] source $testdir/tester.tcl set ::testprefix fts3fault # If SQLITE_ENABLE_FTS3 is not defined, omit this file. ifcapable !fts3 { finish_test ; return } # Test error handling in the sqlite3Fts3Init() function. This is the # function that registers the FTS3 module and various support functions # with SQLite. # do_faultsim_test 1 -body { sqlite3 db test.db | > > | 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 | set testdir [file dirname $argv0] source $testdir/tester.tcl set ::testprefix fts3fault # If SQLITE_ENABLE_FTS3 is not defined, omit this file. ifcapable !fts3 { finish_test ; return } if 0 { # Test error handling in the sqlite3Fts3Init() function. This is the # function that registers the FTS3 module and various support functions # with SQLite. # do_faultsim_test 1 -body { sqlite3 db test.db |
︙ | ︙ | |||
153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 | execsql { CREATE VIRTUAL TABLE t1 USING fts4(a, b, matchnfo=fts3) } } -test { faultsim_test_result {1 {unrecognized parameter: matchnfo=fts3}} \ {1 {vtable constructor failed: t1}} \ {1 {SQL logic error or missing database}} } proc mit {blob} { set scan(littleEndian) i* set scan(bigEndian) I* binary scan $blob $scan($::tcl_platform(byteOrder)) r return $r } do_test 8.0 { faultsim_delete_and_reopen execsql { CREATE VIRTUAL TABLE t8 USING fts4 } execsql "INSERT INTO t8 VALUES('a b c')" execsql "INSERT INTO t8 VALUES('b b b')" execsql "INSERT INTO t8 VALUES('[string repeat {c } 50000]')" execsql "INSERT INTO t8 VALUES('d d d')" execsql "INSERT INTO t8 VALUES('e e e')" execsql "INSERT INTO t8(t8) VALUES('optimize')" faultsim_save_and_close } {} | > > > | > | 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 | execsql { CREATE VIRTUAL TABLE t1 USING fts4(a, b, matchnfo=fts3) } } -test { faultsim_test_result {1 {unrecognized parameter: matchnfo=fts3}} \ {1 {vtable constructor failed: t1}} \ {1 {SQL logic error or missing database}} } } proc mit {blob} { set scan(littleEndian) i* set scan(bigEndian) I* binary scan $blob $scan($::tcl_platform(byteOrder)) r return $r } do_test 8.0 { faultsim_delete_and_reopen execsql { CREATE VIRTUAL TABLE t8 USING fts4 } execsql "INSERT INTO t8 VALUES('a b c')" execsql "INSERT INTO t8 VALUES('b b b')" execsql "INSERT INTO t8 VALUES('[string repeat {c } 50000]')" execsql "INSERT INTO t8 VALUES('d d d')" execsql "INSERT INTO t8 VALUES('e e e')" execsql "INSERT INTO t8(t8) VALUES('optimize')" faultsim_save_and_close } {} do_faultsim_test 8.1 -faults oom-t* -prep { faultsim_restore_and_reopen db func mit mit } -body { execsql { SELECT mit(matchinfo(t8, 'x')) FROM t8 WHERE t8 MATCH 'a b c' } } -test { faultsim_test_result {0 {{1 1 1 1 4 2 1 5 5}}} } do_faultsim_test 8.2 -faults oom-t* -prep { faultsim_restore_and_reopen db func mit mit } -body { execsql { SELECT mit(matchinfo(t8, 's')) FROM t8 WHERE t8 MATCH 'a b c' } } -test { faultsim_test_result {0 3} |
︙ | ︙ |
Added test/fts3offsets.test.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 | # 2010 November 02 # # The author disclaims copyright to this source code. In place of # a legal notice, here is a blessing: # # May you do good and not evil. # May you find forgiveness for yourself and forgive others. # May you share freely, never taking more than you give. # #*********************************************************************** # set testdir [file dirname $argv0] source $testdir/tester.tcl # If SQLITE_ENABLE_FTS3 is not defined, omit this file. ifcapable !fts3 { finish_test ; return } set testprefix fts3offsets set sqlite_fts3_enable_parentheses 1 proc extract {offsets text} { set res "" set off [list] foreach {t i s n} $offsets { lappend off [list $s $n] } set off [lsort -integer -index 0 $off] set iOff 0 foreach e $off { foreach {s n} $e {} append res [string range $text $iOff $s-1] append res "(" append res [string range $text $s [expr $s+$n-1]] append res ")" set iOff [expr $s+$n] } append res [string range $text $iOff end] set res } db func extract extract do_execsql_test 1.1.0 { CREATE VIRTUAL TABLE xx USING fts3(x); INSERT INTO xx VALUES('A x x x B C x x'); INSERT INTO xx VALUES('A B C x B x x C'); INSERT INTO xx VALUES('A x x B C x x x'); } do_execsql_test 1.1.1 { SELECT oid,extract(offsets(xx), x) FROM xx WHERE xx MATCH 'a OR (b NEAR/1 c)'; } { 1 {(A) x x x (B) (C) x x} 2 {(A) (B) (C) x (B) x x C} 3 {(A) x x (B) (C) x x x} } do_execsql_test 1.2 { DELETE FROM xx; INSERT INTO xx VALUES('A x x x B C x x'); INSERT INTO xx VALUES('A x x C x x x C'); INSERT INTO xx VALUES('A x x B C x x x'); } do_execsql_test 1.2.1 { SELECT oid,extract(offsets(xx), x) FROM xx WHERE xx MATCH 'a OR (b NEAR/1 c)'; } { 1 {(A) x x x (B) (C) x x} 2 {(A) x x C x x x C} 3 {(A) x x (B) (C) x x x} } do_execsql_test 1.3 { DELETE FROM xx; INSERT INTO xx(rowid, x) VALUES(1, 'A B C'); INSERT INTO xx(rowid, x) VALUES(2, 'A x'); INSERT INTO xx(rowid, x) VALUES(3, 'A B C'); INSERT INTO xx(rowid, x) VALUES(4, 'A B C x x x x x x x B'); INSERT INTO xx(rowid, x) VALUES(5, 'A x x x x x x x x x C'); INSERT INTO xx(rowid, x) VALUES(6, 'A x x x x x x x x x x x B'); INSERT INTO xx(rowid, x) VALUES(7, 'A B C'); } do_execsql_test 1.3.1 { SELECT oid,extract(offsets(xx), x) FROM xx WHERE xx MATCH 'a OR (b NEAR/1 c)'; } { 1 {(A) (B) (C)} 2 {(A) x} 3 {(A) (B) (C)} 4 {(A) (B) (C) x x x x x x x B} 5 {(A) x x x x x x x x x C} 6 {(A) x x x x x x x x x x x B} 7 {(A) (B) (C)} } do_execsql_test 1.4 { DELETE FROM xx; INSERT INTO xx(rowid, x) VALUES(7, 'A B C'); INSERT INTO xx(rowid, x) VALUES(6, 'A x'); INSERT INTO xx(rowid, x) VALUES(5, 'A B C'); INSERT INTO xx(rowid, x) VALUES(4, 'A B C x x x x x x x B'); INSERT INTO xx(rowid, x) VALUES(3, 'A x x x x x x x x x C'); INSERT INTO xx(rowid, x) VALUES(2, 'A x x x x x x x x x x x B'); INSERT INTO xx(rowid, x) VALUES(1, 'A B C'); } do_execsql_test 1.4.1 { SELECT oid,extract(offsets(xx), x) FROM xx WHERE xx MATCH 'a OR (b NEAR/1 c)' ORDER BY docid DESC; } { 7 {(A) (B) (C)} 6 {(A) x} 5 {(A) (B) (C)} 4 {(A) (B) (C) x x x x x x x B} 3 {(A) x x x x x x x x x C} 2 {(A) x x x x x x x x x x x B} 1 {(A) (B) (C)} } set sqlite_fts3_enable_parentheses 0 finish_test |
Changes to test/memdb.test.
︙ | ︙ | |||
415 416 417 418 419 420 421 | execsql { PRAGMA auto_vacuum = full; CREATE TABLE t1(a); INSERT INTO t1 VALUES(randstr(1000,1000)); INSERT INTO t1 VALUES(randstr(1000,1000)); INSERT INTO t1 VALUES(randstr(1000,1000)); } | | < < > | | 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 | execsql { PRAGMA auto_vacuum = full; CREATE TABLE t1(a); INSERT INTO t1 VALUES(randstr(1000,1000)); INSERT INTO t1 VALUES(randstr(1000,1000)); INSERT INTO t1 VALUES(randstr(1000,1000)); } set before [db one {PRAGMA page_count}] execsql { DELETE FROM t1 } set after [db one {PRAGMA page_count}] expr {$before>$after} } {1} } } ;# ifcapable memorydb finish_test |
Changes to test/memsubsys1.test.
︙ | ︙ | |||
71 72 73 74 75 76 77 78 79 80 81 82 83 84 | set xtra_size 290 # Test 1: Both PAGECACHE and SCRATCH are shut down. # db close sqlite3_shutdown sqlite3_config_lookaside 0 0 sqlite3_initialize reset_highwater_marks build_test_db memsubsys1-1 {PRAGMA page_size=1024} do_test memsubsys1-1.3 { set pg_used [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0] 2] } 0 do_test memsubsys1-1.4 { | > | 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 | set xtra_size 290 # Test 1: Both PAGECACHE and SCRATCH are shut down. # db close sqlite3_shutdown sqlite3_config_lookaside 0 0 sqlite3_config_pagecache 0 0 sqlite3_initialize reset_highwater_marks build_test_db memsubsys1-1 {PRAGMA page_size=1024} do_test memsubsys1-1.3 { set pg_used [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0] 2] } 0 do_test memsubsys1-1.4 { |
︙ | ︙ | |||
111 112 113 114 115 116 117 118 119 120 | # Test 3: Activate PAGECACHE with 20 pages but use the wrong page size # so that PAGECACHE is not used. # db close sqlite3_shutdown sqlite3_config_pagecache [expr 512+$xtra_size] 20 sqlite3_initialize reset_highwater_marks build_test_db memsubsys1-3.1 {PRAGMA page_size=1024} | > < | 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 | # Test 3: Activate PAGECACHE with 20 pages but use the wrong page size # so that PAGECACHE is not used. # db close sqlite3_shutdown sqlite3_config_pagecache [expr 512+$xtra_size] 20 sqlite3_config singlethread sqlite3_initialize reset_highwater_marks build_test_db memsubsys1-3.1 {PRAGMA page_size=1024} do_test memsubsys1-3.1.3 { set pg_used [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0] 2] } 0 do_test memsubsys1-3.1.4 { set overflow [lindex [sqlite3_status SQLITE_STATUS_PAGECACHE_OVERFLOW 0] 2] # Note: The measured PAGECACHE_OVERFLOW is amount malloc() returns, not what # was requested. System malloc() implementations might (arbitrarily) return |
︙ | ︙ | |||
308 309 310 311 312 313 314 315 316 317 | db close sqlite3_shutdown sqlite3_config_memstatus 1 sqlite3_config_pagecache 0 0 sqlite3_config_scratch 0 0 sqlite3_config_lookaside 100 500 sqlite3_initialize autoinstall_test_functions finish_test | > | 309 310 311 312 313 314 315 316 317 318 319 | db close sqlite3_shutdown sqlite3_config_memstatus 1 sqlite3_config_pagecache 0 0 sqlite3_config_scratch 0 0 sqlite3_config_lookaside 100 500 sqlite3_config serialized sqlite3_initialize autoinstall_test_functions finish_test |
Changes to test/mutex1.test.
︙ | ︙ | |||
33 34 35 36 37 38 39 | set var($name) $value incr var(total) $value } } #------------------------------------------------------------------------- # Tests mutex1-1.* test that sqlite3_config() returns SQLITE_MISUSE if | | | | 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | set var($name) $value incr var(total) $value } } #------------------------------------------------------------------------- # Tests mutex1-1.* test that sqlite3_config() returns SQLITE_MISUSE if # is called at the wrong time. And that the first time sqlite3_initialize # is called it obtains the 'static_master' mutex 3 times and a recursive # mutex (sqlite3Config.pInitMutex) twice. Subsequent calls are no-ops # that do not require any mutexes. # do_test mutex1-1.0 { install_mutex_counters 1 } {SQLITE_MISUSE} do_test mutex1-1.1 { |
︙ | ︙ | |||
98 99 100 101 102 103 104 | # * Single-threaded mode. # ifcapable threadsafe&&shared_cache { set enable_shared_cache [sqlite3_enable_shared_cache 1] foreach {mode mutexes} { singlethread {} multithread { | > | > | > | | > | 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 | # * Single-threaded mode. # ifcapable threadsafe&&shared_cache { set enable_shared_cache [sqlite3_enable_shared_cache 1] foreach {mode mutexes} { singlethread {} multithread { fast static_app1 static_app2 static_app3 static_lru static_master static_mem static_open static_prng static_pmem static_vfs1 static_vfs2 static_vfs3 } serialized { fast recursive static_app1 static_app2 static_app3 static_lru static_master static_mem static_open static_prng static_pmem static_vfs1 static_vfs2 static_vfs3 } } { do_test mutex1.2.$mode.1 { catch {db close} sqlite3_shutdown sqlite3_config $mode |
︙ | ︙ | |||
125 126 127 128 129 130 131 | db eval { INSERT INTO abc VALUES(1, 2, 3); } } {} ifcapable !memorymanage { regsub { static_lru} $mutexes {} mutexes } | > | > > > > > > > > > > > > > > > > > > | | 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 | db eval { INSERT INTO abc VALUES(1, 2, 3); } } {} ifcapable !memorymanage { regsub { static_lru} $mutexes {} mutexes } if {$mode ne "singlethread"} { do_test mutex1.2.$mode.3 { # # NOTE: Make sure all the app and vfs mutexes get used. # enter_static_mutex static_app1 leave_static_mutex static_app1 enter_static_mutex static_app2 leave_static_mutex static_app2 enter_static_mutex static_app3 leave_static_mutex static_app3 enter_static_mutex static_vfs1 leave_static_mutex static_vfs1 enter_static_mutex static_vfs2 leave_static_mutex static_vfs2 enter_static_mutex static_vfs3 leave_static_mutex static_vfs3 } {} } do_test mutex1.2.$mode.4 { mutex_counters counters set res [list] foreach {key value} [array get counters] { if {$key ne "total" && $value > 0} { lappend res $key } } lsort $res |
︙ | ︙ |
Changes to test/pcache2.test.
︙ | ︙ | |||
20 21 22 23 24 25 26 27 28 29 30 31 32 33 | # pages are being used for cache. # do_test pcache2-1.1 { db close sqlite3_reset_auto_extension sqlite3_shutdown sqlite3_config_pagecache 6000 100 sqlite3_initialize autoinstall_test_functions sqlite3_status SQLITE_STATUS_PAGECACHE_USED 1 sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0 } {0 0 0} # Open up two database connections to separate files. | > | 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | # pages are being used for cache. # do_test pcache2-1.1 { db close sqlite3_reset_auto_extension sqlite3_shutdown sqlite3_config_pagecache 6000 100 sqlite3_config singlethread sqlite3_initialize autoinstall_test_functions sqlite3_status SQLITE_STATUS_PAGECACHE_USED 1 sqlite3_status SQLITE_STATUS_PAGECACHE_USED 0 } {0 0 0} # Open up two database connections to separate files. |
︙ | ︙ | |||
69 70 71 72 73 74 75 76 77 78 79 | } {0 13 13} db close catch {db2 close} sqlite3_reset_auto_extension sqlite3_shutdown sqlite3_config_pagecache 0 0 sqlite3_initialize autoinstall_test_functions finish_test | > | 70 71 72 73 74 75 76 77 78 79 80 81 | } {0 13 13} db close catch {db2 close} sqlite3_reset_auto_extension sqlite3_shutdown sqlite3_config_pagecache 0 0 sqlite3_config serialized sqlite3_initialize autoinstall_test_functions finish_test |
Changes to test/speedtest1.c.
︙ | ︙ | |||
11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 | " --exclusive Enable locking_mode=EXCLUSIVE\n" " --explain Like --sqlonly but with added EXPLAIN keywords\n" " --heap SZ MIN Memory allocator uses SZ bytes & min allocation MIN\n" " --incrvacuum Enable incremenatal vacuum mode\n" " --journal M Set the journal_mode to M\n" " --key KEY Set the encryption key to KEY\n" " --lookaside N SZ Configure lookaside for N slots of SZ bytes each\n" " --nosync Set PRAGMA synchronous=OFF\n" " --notnull Add NOT NULL constraints to table columns\n" " --pagesize N Set the page size to N\n" " --pcache N SZ Configure N pages of pagecache each of size SZ bytes\n" " --primarykey Use PRIMARY KEY instead of UNIQUE where appropriate\n" " --reprepare Reprepare each statement upon every invocation\n" " --scratch N SZ Configure scratch memory for N slots of SZ bytes each\n" " --sqlonly No-op. Only show the SQL that would have been run.\n" " --shrink-memory Invoke sqlite3_db_release_memory() frequently.\n" " --size N Relative test size. Default=100\n" " --stats Show statistics at the end\n" " --testset T Run test-set T\n" " --trace Turn on SQL tracing\n" " --threads N Use up to N threads for sorting\n" | > > > > | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 | " --exclusive Enable locking_mode=EXCLUSIVE\n" " --explain Like --sqlonly but with added EXPLAIN keywords\n" " --heap SZ MIN Memory allocator uses SZ bytes & min allocation MIN\n" " --incrvacuum Enable incremenatal vacuum mode\n" " --journal M Set the journal_mode to M\n" " --key KEY Set the encryption key to KEY\n" " --lookaside N SZ Configure lookaside for N slots of SZ bytes each\n" " --multithread Set multithreaded mode\n" " --nomemstat Disable memory statistics\n" " --nosync Set PRAGMA synchronous=OFF\n" " --notnull Add NOT NULL constraints to table columns\n" " --pagesize N Set the page size to N\n" " --pcache N SZ Configure N pages of pagecache each of size SZ bytes\n" " --primarykey Use PRIMARY KEY instead of UNIQUE where appropriate\n" " --reprepare Reprepare each statement upon every invocation\n" " --scratch N SZ Configure scratch memory for N slots of SZ bytes each\n" " --serialized Set serialized threading mode\n" " --singlethread Set single-threaded mode - disables all mutexing\n" " --sqlonly No-op. Only show the SQL that would have been run.\n" " --shrink-memory Invoke sqlite3_db_release_memory() frequently.\n" " --size N Relative test size. Default=100\n" " --stats Show statistics at the end\n" " --testset T Run test-set T\n" " --trace Turn on SQL tracing\n" " --threads N Use up to N threads for sorting\n" |
︙ | ︙ | |||
1169 1170 1171 1172 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 | int doIncrvac = 0; /* True for --incrvacuum */ const char *zJMode = 0; /* Journal mode */ const char *zKey = 0; /* Encryption key */ int nLook = 0, szLook = 0; /* --lookaside configuration */ int noSync = 0; /* True for --nosync */ int pageSize = 0; /* Desired page size. 0 means default */ int nPCache = 0, szPCache = 0;/* --pcache configuration */ int nScratch = 0, szScratch=0;/* --scratch configuration */ int showStats = 0; /* True for --stats */ int nThread = 0; /* --threads value */ const char *zTSet = "main"; /* Which --testset torun */ int doTrace = 0; /* True for --trace */ const char *zEncoding = 0; /* --utf16be or --utf16le */ const char *zDbName = 0; /* Name of the test database */ | > | 1173 1174 1175 1176 1177 1178 1179 1180 1181 1182 1183 1184 1185 1186 1187 | int doIncrvac = 0; /* True for --incrvacuum */ const char *zJMode = 0; /* Journal mode */ const char *zKey = 0; /* Encryption key */ int nLook = 0, szLook = 0; /* --lookaside configuration */ int noSync = 0; /* True for --nosync */ int pageSize = 0; /* Desired page size. 0 means default */ int nPCache = 0, szPCache = 0;/* --pcache configuration */ int doPCache = 0; /* True if --pcache is seen */ int nScratch = 0, szScratch=0;/* --scratch configuration */ int showStats = 0; /* True for --stats */ int nThread = 0; /* --threads value */ const char *zTSet = "main"; /* Which --testset torun */ int doTrace = 0; /* True for --trace */ const char *zEncoding = 0; /* --utf16be or --utf16le */ const char *zDbName = 0; /* Name of the test database */ |
︙ | ︙ | |||
1223 1224 1225 1226 1227 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 | if( i>=argc-1 ) fatal_error("missing argument on %s\n", argv[i]); zKey = argv[++i]; }else if( strcmp(z,"lookaside")==0 ){ if( i>=argc-2 ) fatal_error("missing arguments on %s\n", argv[i]); nLook = integerValue(argv[i+1]); szLook = integerValue(argv[i+2]); i += 2; }else if( strcmp(z,"nosync")==0 ){ noSync = 1; }else if( strcmp(z,"notnull")==0 ){ g.zNN = "NOT NULL"; #ifdef SQLITE_ENABLE_OTA }else if( strcmp(z,"ota")==0 ){ sqlite3ota_create_vfs("ota", 0); sqlite3_vfs_register(sqlite3_vfs_find("ota"), 1); #endif }else if( strcmp(z,"pagesize")==0 ){ if( i>=argc-1 ) fatal_error("missing argument on %s\n", argv[i]); pageSize = integerValue(argv[++i]); }else if( strcmp(z,"pcache")==0 ){ if( i>=argc-2 ) fatal_error("missing arguments on %s\n", argv[i]); nPCache = integerValue(argv[i+1]); szPCache = integerValue(argv[i+2]); i += 2; }else if( strcmp(z,"primarykey")==0 ){ g.zPK = "PRIMARY KEY"; }else if( strcmp(z,"reprepare")==0 ){ g.bReprepare = 1; }else if( strcmp(z,"scratch")==0 ){ if( i>=argc-2 ) fatal_error("missing arguments on %s\n", argv[i]); nScratch = integerValue(argv[i+1]); szScratch = integerValue(argv[i+2]); i += 2; }else if( strcmp(z,"sqlonly")==0 ){ g.bSqlOnly = 1; }else if( strcmp(z,"shrink-memory")==0 ){ g.bMemShrink = 1; }else if( strcmp(z,"size")==0 ){ if( i>=argc-1 ) fatal_error("missing argument on %s\n", argv[i]); g.szTest = integerValue(argv[++i]); | > > > > > > > > > | 1228 1229 1230 1231 1232 1233 1234 1235 1236 1237 1238 1239 1240 1241 1242 1243 1244 1245 1246 1247 1248 1249 1250 1251 1252 1253 1254 1255 1256 1257 1258 1259 1260 1261 1262 1263 1264 1265 1266 1267 1268 1269 1270 1271 1272 1273 1274 1275 1276 | if( i>=argc-1 ) fatal_error("missing argument on %s\n", argv[i]); zKey = argv[++i]; }else if( strcmp(z,"lookaside")==0 ){ if( i>=argc-2 ) fatal_error("missing arguments on %s\n", argv[i]); nLook = integerValue(argv[i+1]); szLook = integerValue(argv[i+2]); i += 2; }else if( strcmp(z,"multithread")==0 ){ sqlite3_config(SQLITE_CONFIG_MULTITHREAD); }else if( strcmp(z,"nomemstat")==0 ){ sqlite3_config(SQLITE_CONFIG_MEMSTATUS, 0); }else if( strcmp(z,"nosync")==0 ){ noSync = 1; }else if( strcmp(z,"notnull")==0 ){ g.zNN = "NOT NULL"; #ifdef SQLITE_ENABLE_OTA }else if( strcmp(z,"ota")==0 ){ sqlite3ota_create_vfs("ota", 0); sqlite3_vfs_register(sqlite3_vfs_find("ota"), 1); #endif }else if( strcmp(z,"pagesize")==0 ){ if( i>=argc-1 ) fatal_error("missing argument on %s\n", argv[i]); pageSize = integerValue(argv[++i]); }else if( strcmp(z,"pcache")==0 ){ if( i>=argc-2 ) fatal_error("missing arguments on %s\n", argv[i]); nPCache = integerValue(argv[i+1]); szPCache = integerValue(argv[i+2]); doPCache = 1; i += 2; }else if( strcmp(z,"primarykey")==0 ){ g.zPK = "PRIMARY KEY"; }else if( strcmp(z,"reprepare")==0 ){ g.bReprepare = 1; }else if( strcmp(z,"scratch")==0 ){ if( i>=argc-2 ) fatal_error("missing arguments on %s\n", argv[i]); nScratch = integerValue(argv[i+1]); szScratch = integerValue(argv[i+2]); i += 2; }else if( strcmp(z,"serialized")==0 ){ sqlite3_config(SQLITE_CONFIG_SERIALIZED); }else if( strcmp(z,"singlethread")==0 ){ sqlite3_config(SQLITE_CONFIG_SINGLETHREAD); }else if( strcmp(z,"sqlonly")==0 ){ g.bSqlOnly = 1; }else if( strcmp(z,"shrink-memory")==0 ){ g.bMemShrink = 1; }else if( strcmp(z,"size")==0 ){ if( i>=argc-1 ) fatal_error("missing argument on %s\n", argv[i]); g.szTest = integerValue(argv[++i]); |
︙ | ︙ | |||
1301 1302 1303 1304 1305 1306 1307 | #if SQLITE_VERSION_NUMBER>=3006001 if( nHeap>0 ){ pHeap = malloc( nHeap ); if( pHeap==0 ) fatal_error("cannot allocate %d-byte heap\n", nHeap); rc = sqlite3_config(SQLITE_CONFIG_HEAP, pHeap, nHeap, mnHeap); if( rc ) fatal_error("heap configuration failed: %d\n", rc); } | > | | | | > | 1315 1316 1317 1318 1319 1320 1321 1322 1323 1324 1325 1326 1327 1328 1329 1330 1331 1332 1333 1334 | #if SQLITE_VERSION_NUMBER>=3006001 if( nHeap>0 ){ pHeap = malloc( nHeap ); if( pHeap==0 ) fatal_error("cannot allocate %d-byte heap\n", nHeap); rc = sqlite3_config(SQLITE_CONFIG_HEAP, pHeap, nHeap, mnHeap); if( rc ) fatal_error("heap configuration failed: %d\n", rc); } if( doPCache ){ if( nPCache>0 && szPCache>0 ){ pPCache = malloc( nPCache*(sqlite3_int64)szPCache ); if( pPCache==0 ) fatal_error("cannot allocate %lld-byte pcache\n", nPCache*(sqlite3_int64)szPCache); } rc = sqlite3_config(SQLITE_CONFIG_PAGECACHE, pPCache, szPCache, nPCache); if( rc ) fatal_error("pcache configuration failed: %d\n", rc); } if( nScratch>0 && szScratch>0 ){ pScratch = malloc( nScratch*(sqlite3_int64)szScratch ); if( pScratch==0 ) fatal_error("cannot allocate %lld-byte scratch\n", nScratch*(sqlite3_int64)szScratch); |
︙ | ︙ |
Changes to test/trace.test.
︙ | ︙ | |||
164 165 166 167 168 169 170 171 172 173 174 175 176 177 | set TRACE_OUT {} execsql {SELECT * FROM t1} } {1 2 2 3 2 3} do_test trace-4.5 { set TRACE_OUT } {{SELECT * FROM t1}} catch {sqlite3_finalize $STMT} # Trigger tracing. # ifcapable trigger { do_test trace-5.1 { db eval { CREATE TRIGGER r1t1 AFTER UPDATE ON t1 BEGIN | > > > > > > > > | 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 | set TRACE_OUT {} execsql {SELECT * FROM t1} } {1 2 2 3 2 3} do_test trace-4.5 { set TRACE_OUT } {{SELECT * FROM t1}} catch {sqlite3_finalize $STMT} # 3.8.11: Profile output even if the statement is not run to completion. do_test trace-4.6 { set TRACE_OUT {} db eval {SELECT * FROM t1} {} {if {$a>=1} break} set TRACE_OUT } {{SELECT * FROM t1}} # Trigger tracing. # ifcapable trigger { do_test trace-5.1 { db eval { CREATE TRIGGER r1t1 AFTER UPDATE ON t1 BEGIN |
︙ | ︙ |
Changes to test/with1.test.
︙ | ︙ | |||
853 854 855 856 857 858 859 860 861 | # do_catchsql_test 15.1 { WITH RECURSIVE d(x) AS (VALUES(1) UNION ALL SELECT rowid+1 FROM d WHERE rowid<10) SELECT x FROM d; } {1 {no such column: rowid}} finish_test | > > > > > > > | 853 854 855 856 857 858 859 860 861 862 863 864 865 866 867 868 | # do_catchsql_test 15.1 { WITH RECURSIVE d(x) AS (VALUES(1) UNION ALL SELECT rowid+1 FROM d WHERE rowid<10) SELECT x FROM d; } {1 {no such column: rowid}} # 2015-07-05: Do not allow aggregate recursive queries # do_catchsql_test 16.1 { WITH RECURSIVE i(x) AS (VALUES(1) UNION SELECT count(*) FROM i) SELECT * FROM i; } {1 {recursive aggregate queries not supported}} finish_test |
Changes to tool/loadfts.c.
1 | /* | | > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 | /* ** 2014-07-28 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** ** This file implements a utility program that will load many disk ** files (all files under a given directory) into a FTS table. This is ** used for performance testing of FTS3, FTS4, and FTS5. */ #include <stdio.h> #include <stdlib.h> #include <ctype.h> #include <assert.h> #include <string.h> |
︙ | ︙ |