SQLite

Check-in [80241a0502]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Check that the first byte of a potentially hot journal file is non-zero before beginning hot-journal rollback. Fix for #3751 and #3745. (CVS 6390)
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 80241a050296067937d0b0529fdf0c347358f86c
User & Date: danielk1977 2009-03-26 17:13:06.000
Context
2009-03-27
09:10
Fix an incompatibility between the test_async.c backend and change (6390). (CVS 6391) (check-in: 6762625d29 user: danielk1977 tags: trunk)
2009-03-26
17:13
Check that the first byte of a potentially hot journal file is non-zero before beginning hot-journal rollback. Fix for #3751 and #3745. (CVS 6390) (check-in: 80241a0502 user: danielk1977 tags: trunk)
14:48
Prevent the tcl test suite from throwing an exception when a threadsafe SQLite is linked against a non-threadsafe Tcl for testing. Print a warning instead. Ticket #3753. (CVS 6389) (check-in: a22e7c818b user: danielk1977 tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to src/pager.c.
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
** The pager is used to access a database disk file.  It implements
** atomic commit and rollback through the use of a journal file that
** is separate from the database file.  The pager also implements file
** locking to prevent two processes from writing the same database
** file simultaneously, or one process from reading the database while
** another is writing.
**
** @(#) $Id: pager.c,v 1.572 2009/03/23 04:33:33 danielk1977 Exp $
*/
#ifndef SQLITE_OMIT_DISKIO
#include "sqliteInt.h"

/*
** Macros for troubleshooting.  Normally turned off
*/







|







14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
** The pager is used to access a database disk file.  It implements
** atomic commit and rollback through the use of a journal file that
** is separate from the database file.  The pager also implements file
** locking to prevent two processes from writing the same database
** file simultaneously, or one process from reading the database while
** another is writing.
**
** @(#) $Id: pager.c,v 1.573 2009/03/26 17:13:06 danielk1977 Exp $
*/
#ifndef SQLITE_OMIT_DISKIO
#include "sqliteInt.h"

/*
** Macros for troubleshooting.  Normally turned off
*/
3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336

3337
3338
3339
3340
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351

3352
3353
3354
3355
3356
3357
3358
3359
3360
3361
3362
3363
3364
3365
3366
3367

3368
3369
3370
3371

3372
3373
3374





3375
3376
3377
3378
3379















3380

3381
3382
3383
3384

3385
3386
3387
3388
3389
3390
3391


/*
** This function is called after transitioning from PAGER_UNLOCK to
** PAGER_SHARED state. It tests if there is a hot journal present in
** the file-system for the given pager. A hot journal is one that 
** needs to be played back. According to this function, a hot-journal
** file exists if the following three criteria are met:
**
**   * The journal file exists in the file system, and
**   * No process holds a RESERVED or greater lock on the database file, and
**   * The database file itself is greater than 0 bytes in size.

**
** If the current size of the database file is 0 but a journal file
** exists, that is probably an old journal left over from a prior
** database with the same name. In this case the journal file is
** just deleted using OsDelete, *pExists is set to 0 and SQLITE_OK
** is returned.
**
** This routine does not open the journal file to examine its
** content.  Hence, the journal might contain the name of a master
** journal file that has been deleted, and hence not be hot.  Or
** the header of the journal might be zeroed out.  This routine
** does not discover these cases of a non-hot journal - if the
** journal file exists and is not empty this routine assumes it
** is hot.  The pager_playback() routine will discover that the
** journal file is not really hot and will no-op.

**
** If a hot-journal file is found to exist, *pExists is set to 1 and 
** SQLITE_OK returned. If no hot-journal file is present, *pExists is
** set to 0 and SQLITE_OK returned. If an IO error occurs while trying
** to determine whether or not a hot-journal file exists, the IO error
** code is returned and the value of *pExists is undefined.
*/
static int hasHotJournal(Pager *pPager, int *pExists){
  sqlite3_vfs * const pVfs = pPager->pVfs;
  int rc;                       /* Return code */
  int exists = 0;               /* True if a journal file is present */
  int locked = 0;               /* True if some process holds a RESERVED lock */

  assert( pPager!=0 );
  assert( pPager->useJournal );
  assert( isOpen(pPager->fd) );


  *pExists = 0;
  rc = sqlite3OsAccess(pVfs, pPager->zJournal, SQLITE_ACCESS_EXISTS, &exists);
  if( rc==SQLITE_OK && exists ){

    rc = sqlite3OsCheckReservedLock(pPager->fd, &locked);
    if( rc==SQLITE_OK && !locked ){
      int nPage;





      rc = sqlite3PagerPagecount(pPager, &nPage);
      if( rc==SQLITE_OK ){
       if( nPage==0 ){
          sqlite3OsDelete(pVfs, pPager->zJournal, 0);
        }else{















          *pExists = 1;

        }
      }
    }
  }

  return rc;
}

/*
** Read the content for page pPg out of the database file and into 
** pPg->pData. A shared lock or greater must be held on the database
** file before this function is called.







|



|
>







|
<
|
<
<
|
|
|
>










|
<




>




>



>
>
>
>
>


|
|

>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>




>







3325
3326
3327
3328
3329
3330
3331
3332
3333
3334
3335
3336
3337
3338
3339
3340
3341
3342
3343
3344
3345

3346


3347
3348
3349
3350
3351
3352
3353
3354
3355
3356
3357
3358
3359
3360
3361

3362
3363
3364
3365
3366
3367
3368
3369
3370
3371
3372
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413


/*
** This function is called after transitioning from PAGER_UNLOCK to
** PAGER_SHARED state. It tests if there is a hot journal present in
** the file-system for the given pager. A hot journal is one that 
** needs to be played back. According to this function, a hot-journal
** file exists if the following criteria are met:
**
**   * The journal file exists in the file system, and
**   * No process holds a RESERVED or greater lock on the database file, and
**   * The database file itself is greater than 0 bytes in size, and
**   * The first byte of the journal file exists and is not 0x00.
**
** If the current size of the database file is 0 but a journal file
** exists, that is probably an old journal left over from a prior
** database with the same name. In this case the journal file is
** just deleted using OsDelete, *pExists is set to 0 and SQLITE_OK
** is returned.
**
** This routine does not check if there is a master journal filename

** at the end of the file. If there is, and that master journal file


** does not exist, then the journal file is not really hot. In this
** case this routine will return a false-positive. The pager_playback()
** routine will discover that the journal file is not really hot and 
** will not roll it back. 
**
** If a hot-journal file is found to exist, *pExists is set to 1 and 
** SQLITE_OK returned. If no hot-journal file is present, *pExists is
** set to 0 and SQLITE_OK returned. If an IO error occurs while trying
** to determine whether or not a hot-journal file exists, the IO error
** code is returned and the value of *pExists is undefined.
*/
static int hasHotJournal(Pager *pPager, int *pExists){
  sqlite3_vfs * const pVfs = pPager->pVfs;
  int rc;                       /* Return code */
  int exists;                   /* True if a journal file is present */


  assert( pPager!=0 );
  assert( pPager->useJournal );
  assert( isOpen(pPager->fd) );
  assert( !isOpen(pPager->jfd) );

  *pExists = 0;
  rc = sqlite3OsAccess(pVfs, pPager->zJournal, SQLITE_ACCESS_EXISTS, &exists);
  if( rc==SQLITE_OK && exists ){
    int locked;                 /* True if some process holds a RESERVED lock */
    rc = sqlite3OsCheckReservedLock(pPager->fd, &locked);
    if( rc==SQLITE_OK && !locked ){
      int nPage;

      /* Check the size of the database file. If it consists of 0 pages,
      ** then delete the journal file. See the header comment above for 
      ** the reasoning here.
      */
      rc = sqlite3PagerPagecount(pPager, &nPage);
      if( rc==SQLITE_OK ){
        if( nPage==0 ){
          rc = sqlite3OsDelete(pVfs, pPager->zJournal, 0);
        }else{
          /* The journal file exists and no other connection has a reserved
          ** or greater lock on the database file. Now check that there is
          ** at least one non-zero bytes at the start of the journal file.
          ** If there is, then we consider this journal to be hot. If not, 
          ** it can be ignored.
          */
          int f = SQLITE_OPEN_READONLY|SQLITE_OPEN_MAIN_JOURNAL;
          rc = sqlite3OsOpen(pVfs, pPager->zJournal, pPager->jfd, f, &f);
          if( rc==SQLITE_OK ){
            u8 first = 0;
            rc = sqlite3OsRead(pPager->jfd, (void *)&first, 1, 0);
            if( rc==SQLITE_IOERR_SHORT_READ ){
              rc = SQLITE_OK;
            }
            sqlite3OsClose(pPager->jfd);
            *pExists = (first!=0);
          }
        }
      }
    }
  }

  return rc;
}

/*
** Read the content for page pPg out of the database file and into 
** pPg->pData. A shared lock or greater must be held on the database
** file before this function is called.
3923
3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937

  /* Open the journal file if it is not already open. */
  if( !isOpen(pPager->jfd) ){
    if( pPager->journalMode==PAGER_JOURNALMODE_MEMORY ){
      sqlite3MemJournalOpen(pPager->jfd);
    }else{
      const int flags =                   /* VFS flags to open journal file */
        SQLITE_OPEN_READWRITE|SQLITE_OPEN_EXCLUSIVE|SQLITE_OPEN_CREATE|
        (pPager->tempFile ? 
          (SQLITE_OPEN_DELETEONCLOSE|SQLITE_OPEN_TEMP_JOURNAL):
          (SQLITE_OPEN_MAIN_JOURNAL)
        );
#ifdef SQLITE_ENABLE_ATOMIC_WRITE
      rc = sqlite3JournalOpen(
          pVfs, pPager->zJournal, pPager->jfd, flags, jrnlBufferSize(pPager)







|







3945
3946
3947
3948
3949
3950
3951
3952
3953
3954
3955
3956
3957
3958
3959

  /* Open the journal file if it is not already open. */
  if( !isOpen(pPager->jfd) ){
    if( pPager->journalMode==PAGER_JOURNALMODE_MEMORY ){
      sqlite3MemJournalOpen(pPager->jfd);
    }else{
      const int flags =                   /* VFS flags to open journal file */
        SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|
        (pPager->tempFile ? 
          (SQLITE_OPEN_DELETEONCLOSE|SQLITE_OPEN_TEMP_JOURNAL):
          (SQLITE_OPEN_MAIN_JOURNAL)
        );
#ifdef SQLITE_ENABLE_ATOMIC_WRITE
      rc = sqlite3JournalOpen(
          pVfs, pPager->zJournal, pPager->jfd, flags, jrnlBufferSize(pPager)
Added test/jrnlmode2.test.








































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
# 2009 March 24
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# $Id: jrnlmode2.test,v 1.5 2009/03/26 17:13:06 danielk1977 Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

ifcapable {!pager_pragmas} {
  finish_test
  return
}

#-------------------------------------------------------------------------
# Test overview:
#
#   jrnlmode2-1.*: Demonstrate bug #3745
#   jrnlmode2-2.*: Demonstrate bug #3751
#

do_test jrnlmode2-1.1 {
  execsql {
    PRAGMA journal_mode = persist;
    CREATE TABLE t1(a, b);
    INSERT INTO t1 VALUES(1, 2);
  }
} {persist}

do_test jrnlmode2-1.2 {
  file exists test.db-journal
} {1}

do_test jrnlmode2-1.3 {
  sqlite3 db2 test.db
  execsql { SELECT * FROM t1 } db2
} {1 2}

do_test jrnlmode2-1.4 {
  execsql {
    INSERT INTO t1 VALUES(3, 4);
    BEGIN;
    SELECT * FROM t1;
  }
  execsql { PRAGMA lock_status }
} {main shared temp closed}

do_test jrnlmode2-1.5 {
  file exists test.db-journal
} {1}

do_test jrnlmode2-1.6 {
  catchsql { SELECT * FROM t1 } db2
} {0 {1 2 3 4}}

do_test jrnlmode2-1.7 {
  execsql { COMMIT }
  catchsql { SELECT * FROM t1 } db2
} {0 {1 2 3 4}}



do_test jrnlmode2-2.1 {
  db2 close
  execsql { PRAGMA journal_mode = truncate }
  execsql { INSERT INTO t1 VALUES(5, 6) }
} {}

do_test jrnlmode2-2.2 {
  file exists test.db-journal
} {1}

do_test jrnlmode2-2.3 {
  file size test.db-journal
} {0}

do_test jrnlmode2-2.4 {
  sqlite3 db2 test.db -readonly 1
  catchsql { SELECT * FROM t1 } db2
} {0 {1 2 3 4 5 6}}

do_test jrnlmode2-2.5 {
  file delete test.db-journal
} {}

do_test jrnlmode2-2.6 {
  sqlite3 db2 test.db -readonly 1
  catchsql { SELECT * FROM t1 } db2
} {0 {1 2 3 4 5 6}}

catch { db2 close }
finish_test

Changes to test/tester.tcl.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements some common TCL routines used for regression
# testing the SQLite library
#
# $Id: tester.tcl,v 1.140 2009/02/19 14:39:25 danielk1977 Exp $

#
# What for user input before continuing.  This gives an opportunity
# to connect profiling tools to the process.
#
for {set i 0} {$i<[llength $argv]} {incr i} {
  if {[regexp {^-+pause$} [lindex $argv $i] all value]} {













|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
# 2001 September 15
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements some common TCL routines used for regression
# testing the SQLite library
#
# $Id: tester.tcl,v 1.141 2009/03/26 17:13:06 danielk1977 Exp $

#
# What for user input before continuing.  This gives an opportunity
# to connect profiling tools to the process.
#
for {set i 0} {$i<[llength $argv]} {incr i} {
  if {[regexp {^-+pause$} [lindex $argv $i] all value]} {
286
287
288
289
290
291
292

293
294
295
296
297
298
299
  }
  sqlite3_soft_heap_limit 0
  incr nTest
  puts "$nErr errors out of $nTest tests"
  if {$nErr>0} {
    puts "Failures on these tests: $::failList"
  }

  if {[llength $omitList]>0} {
    puts "Omitted test cases:"
    set prec {}
    foreach {rec} [lsort $omitList] {
      if {$rec==$prec} continue
      set prec $rec
      puts [format {  %-12s %s} [lindex $rec 0] [lindex $rec 1]]







>







286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
  }
  sqlite3_soft_heap_limit 0
  incr nTest
  puts "$nErr errors out of $nTest tests"
  if {$nErr>0} {
    puts "Failures on these tests: $::failList"
  }
  run_thread_tests 1
  if {[llength $omitList]>0} {
    puts "Omitted test cases:"
    set prec {}
    foreach {rec} [lsort $omitList] {
      if {$rec==$prec} continue
      set prec $rec
      puts [format {  %-12s %s} [lindex $rec 0] [lindex $rec 1]]
955
956
957
958
959
960
961


    close $f
  }
}

# If the library is compiled with the SQLITE_DEFAULT_AUTOVACUUM macro set
# to non-zero, then set the global variable $AUTOVACUUM to 1.
set AUTOVACUUM $sqlite_options(default_autovacuum)









>
>
956
957
958
959
960
961
962
963
964
    close $f
  }
}

# If the library is compiled with the SQLITE_DEFAULT_AUTOVACUUM macro set
# to non-zero, then set the global variable $AUTOVACUUM to 1.
set AUTOVACUUM $sqlite_options(default_autovacuum)

source $testdir/thread_common.tcl