SQLite

Check-in [8a98466711]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Fix a bug in DROP TABLE that could cause SQLITE_MASTER table corruption. The root problem was that the sequence of BTree operations (Delete, Next) would not always leave the cursor pointing at the first entry after the entry that was deleted. A consequence of this error was that a DROP TABLE on a table with indices would not always remove every index associated with that table from the SQLITE_MASTER table. Subsequent attempts to open the database will fail when the index for the missing table was parsed. Changes have also been made to ignore extra indices in the SQLITE_MASTER table so that a database previously corrupted by this bug is once again readable. (CVS 316)
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 8a984667113564f2bac7412165b6ff8b7e3e8f70
User & Date: drh 2001-11-23 00:24:12.000
Context
2001-11-23
00:30
Version 2.1.2 (CVS 459) (check-in: f14835df32 user: drh tags: trunk)
00:24
Fix a bug in DROP TABLE that could cause SQLITE_MASTER table corruption. The root problem was that the sequence of BTree operations (Delete, Next) would not always leave the cursor pointing at the first entry after the entry that was deleted. A consequence of this error was that a DROP TABLE on a table with indices would not always remove every index associated with that table from the SQLITE_MASTER table. Subsequent attempts to open the database will fail when the index for the missing table was parsed. Changes have also been made to ignore extra indices in the SQLITE_MASTER table so that a database previously corrupted by this bug is once again readable. (CVS 316) (check-in: 8a98466711 user: drh tags: trunk)
2001-11-22
00:01
Fix a bug in the locking protocol. (CVS 315) (check-in: a9db1c12c5 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to VERSION.
1
2.1.1
|
1
2.1.2
Changes to src/btree.c.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/*
** 2001 September 15
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** $Id: btree.c,v 1.40 2001/11/21 02:21:12 drh Exp $
**
** This file implements a external (disk-based) database using BTrees.
** For a detailed discussion of BTrees, refer to
**
**     Donald E. Knuth, THE ART OF COMPUTER PROGRAMMING, Volume 3:
**     "Sorting And Searching", pages 473-480. Addison-Wesley
**     Publishing Company, Reading, Massachusetts.











|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/*
** 2001 September 15
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** $Id: btree.c,v 1.41 2001/11/23 00:24:12 drh Exp $
**
** This file implements a external (disk-based) database using BTrees.
** For a detailed discussion of BTrees, refer to
**
**     Donald E. Knuth, THE ART OF COMPUTER PROGRAMMING, Volume 3:
**     "Sorting And Searching", pages 473-480. Addison-Wesley
**     Publishing Company, Reading, Massachusetts.
1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
*/
int sqliteBtreeNext(BtCursor *pCur, int *pRes){
  int rc;
  if( pCur->pPage==0 ){
    if( pRes ) *pRes = 1;
    return SQLITE_ABORT;
  }
  if( pCur->bSkipNext ){
    pCur->bSkipNext = 0;
    if( pRes ) *pRes = 0;
    return SQLITE_OK;
  }
  pCur->idx++;
  if( pCur->idx>=pCur->pPage->nCell ){
    if( pCur->pPage->u.hdr.rightChild ){







|







1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
*/
int sqliteBtreeNext(BtCursor *pCur, int *pRes){
  int rc;
  if( pCur->pPage==0 ){
    if( pRes ) *pRes = 1;
    return SQLITE_ABORT;
  }
  if( pCur->bSkipNext && pCur->idx<pCur->pPage->nCell ){
    pCur->bSkipNext = 0;
    if( pRes ) *pRes = 0;
    return SQLITE_OK;
  }
  pCur->idx++;
  if( pCur->idx>=pCur->pPage->nCell ){
    if( pCur->pPage->u.hdr.rightChild ){
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203



2204

2205
2206
2207
2208
2209
2210
2211
    szNext = cellSize(pNext);
    pNext->h.leftChild = pgnoChild;
    insertCell(pPage, pCur->idx, pNext, szNext);
    rc = balance(pCur->pBt, pPage, pCur);
    if( rc ) return rc;
    pCur->bSkipNext = 1;
    dropCell(leafCur.pPage, leafCur.idx, szNext);
    rc = balance(pCur->pBt, leafCur.pPage, 0);
    releaseTempCursor(&leafCur);
  }else{
    dropCell(pPage, pCur->idx, cellSize(pCell));
    if( pCur->idx>=pPage->nCell ){
      pCur->idx = pPage->nCell-1;
      if( pCur->idx<0 ){ pCur->idx = 0; }



      pCur->bSkipNext = 0;

    }else{
      pCur->bSkipNext = 1;
    }
    rc = balance(pCur->pBt, pPage, pCur);
  }
  return rc;
}







|





|
>
>
>
|
>







2190
2191
2192
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
2215
    szNext = cellSize(pNext);
    pNext->h.leftChild = pgnoChild;
    insertCell(pPage, pCur->idx, pNext, szNext);
    rc = balance(pCur->pBt, pPage, pCur);
    if( rc ) return rc;
    pCur->bSkipNext = 1;
    dropCell(leafCur.pPage, leafCur.idx, szNext);
    rc = balance(pCur->pBt, leafCur.pPage, pCur);
    releaseTempCursor(&leafCur);
  }else{
    dropCell(pPage, pCur->idx, cellSize(pCell));
    if( pCur->idx>=pPage->nCell ){
      pCur->idx = pPage->nCell-1;
      if( pCur->idx<0 ){ 
        pCur->idx = 0;
        pCur->bSkipNext = 1;
      }else{
        pCur->bSkipNext = 0;
      }
    }else{
      pCur->bSkipNext = 1;
    }
    rc = balance(pCur->pBt, pPage, pCur);
  }
  return rc;
}
Changes to src/main.c.
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
**
*************************************************************************
** Main file for the SQLite library.  The routines in this file
** implement the programmer interface to the library.  Routines in
** other files are for internal use by SQLite and should not be
** accessed by users of the library.
**
** $Id: main.c,v 1.49 2001/11/07 16:48:27 drh Exp $
*/
#include "sqliteInt.h"
#include "os.h"

/*
** This is the callback routine for the code that initializes the
** database.  See sqliteInit() below for additional information.







|







10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
**
*************************************************************************
** Main file for the SQLite library.  The routines in this file
** implement the programmer interface to the library.  Routines in
** other files are for internal use by SQLite and should not be
** accessed by users of the library.
**
** $Id: main.c,v 1.50 2001/11/23 00:24:12 drh Exp $
*/
#include "sqliteInt.h"
#include "os.h"

/*
** This is the callback routine for the code that initializes the
** database.  See sqliteInit() below for additional information.
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
        ** or executed.  All the parser does is build the internal data
        ** structures that describe the table or index.
        */
        memset(&sParse, 0, sizeof(sParse));
        sParse.db = db;
        sParse.initFlag = 1;
        sParse.newTnum = atoi(argv[2]);
        nErr = sqliteRunParser(&sParse, argv[3], 0);
      }else{
        /* If the SQL column is blank it means this is an index that
        ** was created to be the PRIMARY KEY or to fulfill a UNIQUE
        ** constraint or a CREATE TABLE.  The index should have already
        ** been created when we processed the CREATE TABLE.  All we have
        ** to do here is record the root page.
        */







|







59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
        ** or executed.  All the parser does is build the internal data
        ** structures that describe the table or index.
        */
        memset(&sParse, 0, sizeof(sParse));
        sParse.db = db;
        sParse.initFlag = 1;
        sParse.newTnum = atoi(argv[2]);
        sqliteRunParser(&sParse, argv[3], 0);
      }else{
        /* If the SQL column is blank it means this is an index that
        ** was created to be the PRIMARY KEY or to fulfill a UNIQUE
        ** constraint or a CREATE TABLE.  The index should have already
        ** been created when we processed the CREATE TABLE.  All we have
        ** to do here is record the root page.
        */
Changes to test/btree.test.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this script is btree database backend
#
# $Id: btree.test,v 1.10 2001/09/23 02:35:53 drh Exp $


set testdir [file dirname $argv0]
source $testdir/tester.tcl

if {[info commands btree_open]!=""} {














|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this script is btree database backend
#
# $Id: btree.test,v 1.11 2001/11/23 00:24:12 drh Exp $


set testdir [file dirname $argv0]
source $testdir/tester.tcl

if {[info commands btree_open]!=""} {

993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
#
#   1.  Do some deletes from the 3-layer tree
#   2.  Commit and reopen the database
#   3.  Read every 15th entry and make sure it works
#   4.  Implement btree_sanity and put it throughout this script
#

do_test btree-10.98 {
  btree_close_cursor $::c1
  lindex [btree_pager_stats $::b1] 1
} {1}
do_test btree-10.99 {
  btree_rollback $::b1
  lindex [btree_pager_stats $::b1] 1
} {0}
btree_pager_ref_dump $::b1

do_test btree-99.1 {
  btree_close $::b1







|



|







993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
#
#   1.  Do some deletes from the 3-layer tree
#   2.  Commit and reopen the database
#   3.  Read every 15th entry and make sure it works
#   4.  Implement btree_sanity and put it throughout this script
#

do_test btree-15.98 {
  btree_close_cursor $::c1
  lindex [btree_pager_stats $::b1] 1
} {1}
do_test btree-15.99 {
  btree_rollback $::b1
  lindex [btree_pager_stats $::b1] 1
} {0}
btree_pager_ref_dump $::b1

do_test btree-99.1 {
  btree_close $::b1
Added test/btree3.test.




























































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
# 2001 November 22
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this script is btree database backend
#
# In particular, this file tests a small part of the Delete logic
# for the BTree backend.  When a row is deleted from a table, the
# cursor is suppose to be left pointing at either the previous or
# next entry in that table.  If the cursor is left pointing at the
# next entry, then the next Next operation is ignored.  So the 
# sequence of operations (Delete, Next) should always leave the
# cursor pointing at the first entry past the one that was deleted.
# This test is designed to verify that behavior.
#
# $Id: btree3.test,v 1.1 2001/11/23 00:24:12 drh Exp $


set testdir [file dirname $argv0]
source $testdir/tester.tcl

if {[info commands btree_open]!=""} {

# Open a test database.
#
file delete -force test1.bt
file delete -force test1.bt-journal
set b1 [btree_open test1.bt]
btree_begin_transaction $::b1

# Insert a few one records
#
set data {abcdefghijklmnopqrstuvwxyz0123456789}
append data $data
append data $data
append data $data
append data $data
for {set k 2} {$k<=10} {incr k} {
  for {set j 1} {$j<=$k} {incr j} {
    set jkey [format %02d $j]
    btree_clear_table $::b1 2
    set ::c1 [btree_cursor $::b1 2 1]
    for {set i 1} {$i<=$k+1} {incr i} {
      set key [format %02d $i]
      do_test btree3-$k.$j.1.$i {
        btree_insert $::c1 $::key $::data
      } {}
      # btree_tree_dump $::b1 2
    }
    do_test btree3-$k.$j.2 {
      btree_move_to $::c1 $::jkey
      btree_key $::c1
    } $::jkey
    do_test btree3-$k.$j.3 {
      btree_delete $::c1
    } {}
    do_test btree3-$k.$j.4 {
      btree_next $::c1
      btree_key $::c1
    } [format %02d [expr $j+1]]
    btree_close_cursor $::c1
  }
}

btree_rollback $::b1    
btree_pager_ref_dump $::b1
btree_close $::b1

} ;# end if( not mem: and has pager_open command );

finish_test
Changes to test/index.test.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this file is testing the CREATE INDEX statement.
#
# $Id: index.test,v 1.15 2001/11/08 00:45:22 drh Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

# Create a basic index and verify it is added to sqlite_master
#
do_test index-1.1 {













|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this file is testing the CREATE INDEX statement.
#
# $Id: index.test,v 1.16 2001/11/23 00:24:12 drh Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

# Create a basic index and verify it is added to sqlite_master
#
do_test index-1.1 {
184
185
186
187
188
189
190










191
192
193
194
195
196
197
do_test index-6.2b {
  execsql {SELECT name FROM sqlite_master WHERE type!='meta' ORDER BY name}
} {index1 test1 test2}
do_test index-6.3 {
  execsql {DROP TABLE test1}
  execsql {DROP TABLE test2}
  execsql {SELECT name FROM sqlite_master WHERE type!='meta' ORDER BY name}










} {}

# Create a primary key
#
do_test index-7.1 {
  execsql {CREATE TABLE test1(f1 int, f2 int primary key)}
  for {set i 1} {$i<20} {incr i} {







>
>
>
>
>
>
>
>
>
>







184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
do_test index-6.2b {
  execsql {SELECT name FROM sqlite_master WHERE type!='meta' ORDER BY name}
} {index1 test1 test2}
do_test index-6.3 {
  execsql {DROP TABLE test1}
  execsql {DROP TABLE test2}
  execsql {SELECT name FROM sqlite_master WHERE type!='meta' ORDER BY name}
} {}
do_test index-6.4 {
  execsql {
    CREATE TABLE test1(a,b);
    CREATE INDEX index1 ON test1(a);
    CREATE INDEX index2 ON test1(b);
    CREATE INDEX index3 ON test1(a,b);
    DROP TABLE test1;
    SELECT name FROM sqlite_master WHERE type!='meta' ORDER BY name;
  }
} {}

# Create a primary key
#
do_test index-7.1 {
  execsql {CREATE TABLE test1(f1 int, f2 int primary key)}
  for {set i 1} {$i<20} {incr i} {
Changes to test/select2.test.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28

29
30
31
32
33
34
35
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this file is testing the SELECT statement.
#
# $Id: select2.test,v 1.15 2001/11/08 00:45:22 drh Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

# Create a table with some data
#
execsql {CREATE TABLE tbl1(f1 int, f2 int)}
set f [open ./testdata1.txt w]
for {set i 0} {$i<=30} {incr i} {
  puts $f "[expr {$i%9}]\t[expr {$i%10}]"
}
close $f
execsql {COPY tbl1 FROM './testdata1.txt'}
file delete -force ./testdata1.txt


# Do a second query inside a first.
#
do_test select2-1.1 {
  set sql {SELECT DISTINCT f1 FROM tbl1 ORDER BY f1}
  set r {}
  db eval $sql data {













|














>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.  The
# focus of this file is testing the SELECT statement.
#
# $Id: select2.test,v 1.16 2001/11/23 00:24:13 drh Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

# Create a table with some data
#
execsql {CREATE TABLE tbl1(f1 int, f2 int)}
set f [open ./testdata1.txt w]
for {set i 0} {$i<=30} {incr i} {
  puts $f "[expr {$i%9}]\t[expr {$i%10}]"
}
close $f
execsql {COPY tbl1 FROM './testdata1.txt'}
file delete -force ./testdata1.txt
catch {unset data}

# Do a second query inside a first.
#
do_test select2-1.1 {
  set sql {SELECT DISTINCT f1 FROM tbl1 ORDER BY f1}
  set r {}
  db eval $sql data {
Changes to www/changes.tcl.
13
14
15
16
17
18
19
20
21
22







23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50


proc chng {date desc} {
  puts "<DT><B>$date</B></DT>"
  puts "<DD><P><UL>$desc</UL></P></DD>"
}

chng {2001 Nov ?? (2.1.2)} {
<li>Changes to support 64-bit architectures.</li>
<li>Fix a bug in the locking protocol.</li>







}

chng {2001 Nov 13 (2.1.1)} {
<li>Bug fix: Sometimes arbirary strings were passed to the callback
    function when the actual value of a column was NULL.</li>
}

chng {2001 Nov 12 (2.1.0)} {
<li>Change the format of data records so that records up to 16MB in size
    can be stored.</li>
<li>Change the format of indices to allow for better query optimization.</li>
<li>Implement the "LIMIT ... OFFSET ..." clause on SELECT statements.</li>
}

chng {2001 Nov 3 (2.0.8)} {
<li>Made selected parameters in API functions <b>const</b>. This should
    be fully backwards compatible.</li>
<li>Documentation updates</li>
<li>Simplify the design of the VDBE by restricting the number of sorters
    and lists to 1.
    In practice, no more than one sorter and one list was every used anyhow.
    </li>
}

chng {2001 Oct 21 (2.0.7)} {
<li>Any UTF-8 character or ISO8859 character can be used as part of
    an identifier.</li>
<li>Patches from Christian Werner to improve ODBC compatibility and to







|


>
>
>
>
>
>
>




















|







13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57


proc chng {date desc} {
  puts "<DT><B>$date</B></DT>"
  puts "<DD><P><UL>$desc</UL></P></DD>"
}

chng {2001 Nov 22 (2.1.2)} {
<li>Changes to support 64-bit architectures.</li>
<li>Fix a bug in the locking protocol.</li>
<li>Fix a bug that could (rarely) cause the database to become 
    unreadable after a DROP TABLE due to corruption to the SQLITE_MASTER
    table.</li>
<li>Change the code so that version 2.1.1 databases that were rendered 
    unreadable by the above bug can be read by this version of
    the library even though the SQLITE_MASTER table is (slightly)
    corrupted.</li>
}

chng {2001 Nov 13 (2.1.1)} {
<li>Bug fix: Sometimes arbirary strings were passed to the callback
    function when the actual value of a column was NULL.</li>
}

chng {2001 Nov 12 (2.1.0)} {
<li>Change the format of data records so that records up to 16MB in size
    can be stored.</li>
<li>Change the format of indices to allow for better query optimization.</li>
<li>Implement the "LIMIT ... OFFSET ..." clause on SELECT statements.</li>
}

chng {2001 Nov 3 (2.0.8)} {
<li>Made selected parameters in API functions <b>const</b>. This should
    be fully backwards compatible.</li>
<li>Documentation updates</li>
<li>Simplify the design of the VDBE by restricting the number of sorters
    and lists to 1.
    In practice, no more than one sorter and one list was ever used anyhow.
    </li>
}

chng {2001 Oct 21 (2.0.7)} {
<li>Any UTF-8 character or ISO8859 character can be used as part of
    an identifier.</li>
<li>Patches from Christian Werner to improve ODBC compatibility and to