Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Fixed a few more crashes when dealing with corrupt db files. (CVS 5888) |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
f8bb34e40917e55696376d2def932a41 |
User & Date: | shane 2008-11-12 04:55:34.000 |
Context
2008-11-12
| ||
08:07 | Make sure affinities and implicit collation sequences are correctly used by comparison expressions in the select-list or having clause of an aggregate query. Ticket #3493. (CVS 5889) (check-in: b8ceaa0050 user: danielk1977 tags: trunk) | |
04:55 | Fixed a few more crashes when dealing with corrupt db files. (CVS 5888) (check-in: f8bb34e409 user: shane tags: trunk) | |
2008-11-11
| ||
22:18 | Fixed crash during an UPDATE when free cell size is corrupt. (CVS 5887) (check-in: ec18667e2d user: shane tags: trunk) | |
Changes
Changes to src/btree.c.
1 2 3 4 5 6 7 8 9 10 11 | /* ** 2004 April 6 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* | | | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 | /* ** 2004 April 6 ** ** The author disclaims copyright to this source code. In place of ** a legal notice, here is a blessing: ** ** May you do good and not evil. ** May you find forgiveness for yourself and forgive others. ** May you share freely, never taking more than you give. ** ************************************************************************* ** $Id: btree.c,v 1.532 2008/11/12 04:55:34 shane Exp $ ** ** This file implements a external (disk-based) database using BTrees. ** See the header comment on "btreeInt.h" for additional information. ** Including a description of file format and an overview of operation. */ #include "btreeInt.h" |
︙ | ︙ | |||
698 699 700 701 702 703 704 | /* ** Defragment the page given. All Cells are moved to the ** end of the page and all free space is collected into one ** big FreeBlk that occurs in between the header and cell ** pointer array and the cell content area. */ | | | 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 | /* ** Defragment the page given. All Cells are moved to the ** end of the page and all free space is collected into one ** big FreeBlk that occurs in between the header and cell ** pointer array and the cell content area. */ static int defragmentPage(MemPage *pPage){ int i; /* Loop counter */ int pc; /* Address of a i-th cell */ int addr; /* Offset of first byte after cell pointer array */ int hdr; /* Offset to the page header */ int size; /* Size of a cell */ int usableSize; /* Number of usable bytes on a page */ int cellOffset; /* Offset to the cell pointer array */ |
︙ | ︙ | |||
730 731 732 733 734 735 736 | cbrk = get2byte(&data[hdr+5]); memcpy(&temp[cbrk], &data[cbrk], usableSize - cbrk); cbrk = usableSize; for(i=0; i<nCell; i++){ u8 *pAddr; /* The i-th cell pointer */ pAddr = &data[cellOffset + i*2]; pc = get2byte(pAddr); | | > > > > > > | 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 | cbrk = get2byte(&data[hdr+5]); memcpy(&temp[cbrk], &data[cbrk], usableSize - cbrk); cbrk = usableSize; for(i=0; i<nCell; i++){ u8 *pAddr; /* The i-th cell pointer */ pAddr = &data[cellOffset + i*2]; pc = get2byte(pAddr); if (pc >= pPage->pBt->usableSize) { return SQLITE_CORRUPT_BKPT; } size = cellSizePtr(pPage, &temp[pc]); cbrk -= size; if ((cbrk < cellOffset+2*nCell) || (cbrk+size>pPage->pBt->usableSize)) { return SQLITE_CORRUPT_BKPT; } memcpy(&data[cbrk], &temp[pc], size); put2byte(pAddr, cbrk); } assert( cbrk>=cellOffset+2*nCell ); put2byte(&data[hdr+5], cbrk); data[hdr+1] = 0; data[hdr+2] = 0; data[hdr+7] = 0; addr = cellOffset+2*nCell; memset(&data[addr], 0, cbrk-addr); return SQLITE_OK; } /* ** Allocate nByte bytes of space on a page. ** ** Return the index into pPage->aData[] of the first byte of ** the new allocation. The caller guarantees that there is enough |
︙ | ︙ | |||
4554 4555 4556 4557 4558 4559 4560 | ** Remove the i-th cell from pPage. This routine effects pPage only. ** The cell content is not freed or deallocated. It is assumed that ** the cell content has been copied someplace else. This routine just ** removes the reference to the cell from pPage. ** ** "sz" must be the number of bytes in the cell. */ | | < | | > > > | 4560 4561 4562 4563 4564 4565 4566 4567 4568 4569 4570 4571 4572 4573 4574 4575 4576 4577 4578 4579 4580 4581 4582 4583 4584 4585 4586 4587 4588 4589 4590 4591 4592 4593 4594 4595 4596 4597 4598 | ** Remove the i-th cell from pPage. This routine effects pPage only. ** The cell content is not freed or deallocated. It is assumed that ** the cell content has been copied someplace else. This routine just ** removes the reference to the cell from pPage. ** ** "sz" must be the number of bytes in the cell. */ static int dropCell(MemPage *pPage, int idx, int sz){ int i; /* Loop counter */ int pc; /* Offset to cell content of cell being deleted */ u8 *data; /* pPage->aData */ u8 *ptr; /* Used to move bytes around within data[] */ assert( idx>=0 && idx<pPage->nCell ); assert( sz==cellSize(pPage, idx) ); assert( sqlite3PagerIswriteable(pPage->pDbPage) ); assert( sqlite3_mutex_held(pPage->pBt->mutex) ); data = pPage->aData; ptr = &data[pPage->cellOffset + 2*idx]; pc = get2byte(ptr); if ( pc<=10 || pc+sz>pPage->pBt->usableSize ) { return SQLITE_CORRUPT_BKPT; } freeSpace(pPage, pc, sz); for(i=idx+1; i<pPage->nCell; i++, ptr+=2){ ptr[0] = ptr[2]; ptr[1] = ptr[3]; } pPage->nCell--; put2byte(&data[pPage->hdrOffset+3], pPage->nCell); pPage->nFree += 2; return SQLITE_OK; } /* ** Insert a new cell on pPage at cell index "i". pCell points to the ** content of the cell. ** ** If the cell content will fit on the page, then put it there. If it |
︙ | ︙ | |||
4640 4641 4642 4643 4644 4645 4646 | data = pPage->aData; hdr = pPage->hdrOffset; top = get2byte(&data[hdr+5]); cellOffset = pPage->cellOffset; end = cellOffset + 2*pPage->nCell + 2; ins = cellOffset + 2*i; if( end > top - sz ){ | | > > > | > | 4648 4649 4650 4651 4652 4653 4654 4655 4656 4657 4658 4659 4660 4661 4662 4663 4664 4665 4666 4667 4668 4669 4670 4671 4672 4673 4674 | data = pPage->aData; hdr = pPage->hdrOffset; top = get2byte(&data[hdr+5]); cellOffset = pPage->cellOffset; end = cellOffset + 2*pPage->nCell + 2; ins = cellOffset + 2*i; if( end > top - sz ){ rc = defragmentPage(pPage); if( rc!=SQLITE_OK ){ return rc; } top = get2byte(&data[hdr+5]); assert( end + sz <= top ); } idx = allocateSpace(pPage, sz); assert( idx>0 ); assert( end <= get2byte(&data[hdr+5]) ); if (idx+sz > pPage->pBt->usableSize) { return SQLITE_CORRUPT_BKPT; } pPage->nCell++; pPage->nFree -= 2; memcpy(&data[idx+nSkip], pCell+nSkip, sz-nSkip); for(j=end-2, ptr=&data[j]; j>ins; j-=2, ptr-=2){ ptr[0] = ptr[-2]; ptr[1] = ptr[-1]; } |
︙ | ︙ | |||
5810 5811 5812 5813 5814 5815 5816 | oldCell = findCell(pPage, idx); if( !pPage->leaf ){ memcpy(newCell, oldCell, 4); } szOld = cellSizePtr(pPage, oldCell); rc = clearCell(pPage, oldCell); if( rc ) goto end_insert; | | > > > | 5822 5823 5824 5825 5826 5827 5828 5829 5830 5831 5832 5833 5834 5835 5836 5837 5838 5839 | oldCell = findCell(pPage, idx); if( !pPage->leaf ){ memcpy(newCell, oldCell, 4); } szOld = cellSizePtr(pPage, oldCell); rc = clearCell(pPage, oldCell); if( rc ) goto end_insert; rc = dropCell(pPage, idx, szOld); if( rc!=SQLITE_OK ) { goto end_insert; } }else if( loc<0 && pPage->nCell>0 ){ assert( pPage->leaf ); idx = ++pCur->aiIdx[pCur->iPage]; pCur->info.nSize = 0; pCur->validNKey = 0; }else{ assert( pPage->leaf ); |
︙ | ︙ | |||
6713 6714 6715 6716 6717 6718 6719 | int i, rc, depth, d2, pgno, cnt; int hdr, cellStart; int nCell; u8 *data; BtShared *pBt; int usableSize; char zContext[100]; | | | 6728 6729 6730 6731 6732 6733 6734 6735 6736 6737 6738 6739 6740 6741 6742 | int i, rc, depth, d2, pgno, cnt; int hdr, cellStart; int nCell; u8 *data; BtShared *pBt; int usableSize; char zContext[100]; char *hit = 0; sqlite3_snprintf(sizeof(zContext), zContext, "Page %d: ", iPage); /* Check that the page exists */ pBt = pCheck->pBt; usableSize = pBt->usableSize; |
︙ | ︙ | |||
6803 6804 6805 6806 6807 6808 6809 | if( hit==0 ){ pCheck->mallocFailed = 1; }else{ u16 contentOffset = get2byte(&data[hdr+5]); if (contentOffset > usableSize) { checkAppendMsg(pCheck, 0, "Corruption detected in header on page %d",iPage,0); | | | 6818 6819 6820 6821 6822 6823 6824 6825 6826 6827 6828 6829 6830 6831 6832 | if( hit==0 ){ pCheck->mallocFailed = 1; }else{ u16 contentOffset = get2byte(&data[hdr+5]); if (contentOffset > usableSize) { checkAppendMsg(pCheck, 0, "Corruption detected in header on page %d",iPage,0); goto check_page_abort; } memset(hit+contentOffset, 0, usableSize-contentOffset); memset(hit, 1, contentOffset); nCell = get2byte(&data[hdr+3]); cellStart = hdr + 12 - 4*pPage->leaf; for(i=0; i<nCell; i++){ int pc = get2byte(&data[cellStart+i*2]); |
︙ | ︙ | |||
6850 6851 6852 6853 6854 6855 6856 | } if( cnt!=data[hdr+7] ){ checkAppendMsg(pCheck, 0, "Fragmented space is %d byte reported as %d on page %d", cnt, data[hdr+7], iPage); } } | > | | 6865 6866 6867 6868 6869 6870 6871 6872 6873 6874 6875 6876 6877 6878 6879 6880 | } if( cnt!=data[hdr+7] ){ checkAppendMsg(pCheck, 0, "Fragmented space is %d byte reported as %d on page %d", cnt, data[hdr+7], iPage); } } check_page_abort: if (hit) sqlite3PageFree(hit); releasePage(pPage); return depth+1; } #endif /* SQLITE_OMIT_INTEGRITY_CHECK */ #ifndef SQLITE_OMIT_INTEGRITY_CHECK |
︙ | ︙ |
Changes to test/corruptC.test.
︙ | ︙ | |||
11 12 13 14 15 16 17 | # This file implements regression tests for SQLite library. # # This file implements tests to make sure SQLite does not crash or # segfault if it sees a corrupt database file. It creates a base # data base file, then tests that single byte corruptions in # increasingly larger quantities are handled gracefully. # | | | 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 | # This file implements regression tests for SQLite library. # # This file implements tests to make sure SQLite does not crash or # segfault if it sees a corrupt database file. It creates a base # data base file, then tests that single byte corruptions in # increasingly larger quantities are handled gracefully. # # $Id: corruptC.test,v 1.5 2008/11/12 04:55:34 shane Exp $ catch {file delete -force test.db test.db-journal test.bu} set testdir [file dirname $argv0] source $testdir/tester.tcl # Set a uniform random seed |
︙ | ︙ | |||
35 36 37 38 39 40 41 | INSERT OR IGNORE INTO t1 SELECT x*2,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*3,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*5,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*7,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*11,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*13,y FROM t1; CREATE INDEX t1i1 ON t1(x); | | | 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 | INSERT OR IGNORE INTO t1 SELECT x*2,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*3,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*5,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*7,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*11,y FROM t1; INSERT OR IGNORE INTO t1 SELECT x*13,y FROM t1; CREATE INDEX t1i1 ON t1(x); CREATE TABLE t2 AS SELECT x,2 as y FROM t1 WHERE rowid%5!=0; COMMIT; } } {} ifcapable {integrityck} { integrity_check corruptC-1.2 } |
︙ | ︙ | |||
84 85 86 87 88 89 90 | # insert corrupt byte(s) hexio_write test.db 2053 [format %02x 0x04] sqlite3 db test.db catchsql {PRAGMA integrity_check} } {0 {{*** in database main *** | | < | | > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 | # insert corrupt byte(s) hexio_write test.db 2053 [format %02x 0x04] sqlite3 db test.db catchsql {PRAGMA integrity_check} } {0 {{*** in database main *** Corruption detected in header on page 3}}} # test that a corrupt content offset size is handled (seed 5649) do_test corruptC-2.2 { db close copy_file test.bu test.db # insert corrupt byte(s) hexio_write test.db 27 [format %02x 0x08] hexio_write test.db 233 [format %02x 0x6a] hexio_write test.db 328 [format %02x 0x67] hexio_write test.db 750 [format %02x 0x1f] hexio_write test.db 1132 [format %02x 0x52] hexio_write test.db 1133 [format %02x 0x84] hexio_write test.db 1220 [format %02x 0x01] hexio_write test.db 3688 [format %02x 0xc1] hexio_write test.db 3714 [format %02x 0x58] hexio_write test.db 3746 [format %02x 0x9a] sqlite3 db test.db catchsql {UPDATE t1 SET y=1} } {1 {database disk image is malformed}} # test that a corrupt free cell size is handled (seed 13329) do_test corruptC-2.3 { db close copy_file test.bu test.db # insert corrupt byte(s) hexio_write test.db 1094 [format %02x 0x76] sqlite3 db test.db catchsql {UPDATE t1 SET y=1} } {1 {database disk image is malformed}} # test that a corrupt free cell size is handled (seed 169571) do_test corruptC-2.4 { db close copy_file test.bu test.db # insert corrupt byte(s) hexio_write test.db 3119 [format %02x 0xdf] sqlite3 db test.db catchsql {UPDATE t2 SET y='abcdef-uvwxyz'} } {1 {database disk image is malformed}} # test that a corrupt free cell size is handled (seed 169571) do_test corruptC-2.5 { db close copy_file test.bu test.db # insert corrupt byte(s) hexio_write test.db 3119 [format %02x 0xdf] hexio_write test.db 4073 [format %02x 0xbf] sqlite3 db test.db catchsql {BEGIN; UPDATE t2 SET y='abcdef-uvwxyz'; ROLLBACK;} catchsql {PRAGMA integrity_check} } {0 {{*** in database main *** On tree page 4 cell 49: invalid page number 1006653561 Corruption detected in cell 49 on page 4 Corruption detected in cell 710 on page 4 Multiple uses for byte 116 of page 4 Fragmented space is 0 byte reported as 21 on page 4}}} # test that a corrupt free cell size is handled (seed 169595) do_test corruptC-2.6 { db close copy_file test.bu test.db # insert corrupt byte(s) hexio_write test.db 619 [format %02x 0xe2] hexio_write test.db 3150 [format %02x 0xa8] sqlite3 db test.db catchsql {BEGIN; UPDATE t2 SET y='abcdef-uvwxyz'; ROLLBACK;} } {1 {database disk image is malformed}} # # now test for a series of quasi-random seeds # for {set tn 0} {$tn<=1024} {incr tn 1} { # Set a quasi-random random seed |
︙ | ︙ | |||
172 173 174 175 176 177 178 | set x {} } {} do_test corruptC-3.$tn.$i.6 { catchsql {BEGIN; UPDATE t1 SET y=1; ROLLBACK;} set x {} } {} do_test corruptC-3.$tn.$i.7 { | | | 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 | set x {} } {} do_test corruptC-3.$tn.$i.6 { catchsql {BEGIN; UPDATE t1 SET y=1; ROLLBACK;} set x {} } {} do_test corruptC-3.$tn.$i.7 { catchsql {BEGIN; UPDATE t2 SET y='abcdef-uvwxyz'; ROLLBACK;} set x {} } {} # check the integrity of the database. # once the corruption is detected, we can stop. ifcapable {integrityck} { set res [ catchsql {PRAGMA integrity_check} ] |
︙ | ︙ |