/ Check-in [eb3ac895]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add variable pager.dbHintSize, used to limit the number of calls made to the xFileControl(FCNTL_SIZE_HINT) method.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | experimental
Files: files | file ages | folders
SHA1:eb3ac895bd92c880ac6acca58de8cf9643c2b5e4
User & Date: dan 2010-08-10 09:58:57
Context
2010-08-10
15:46
Fix a bug in pager.c causing it to omit the xSync() call required following a hot-journal rollback. Closed-Leaf check-in: 42ba43ac user: dan tags: experimental
09:58
Add variable pager.dbHintSize, used to limit the number of calls made to the xFileControl(FCNTL_SIZE_HINT) method. check-in: eb3ac895 user: dan tags: experimental
2010-08-09
19:17
Modify some comments in pager.c. check-in: 5662da6d user: dan tags: experimental
Changes
Hide Diffs Side-by-Side Diffs Show Whitespace Changes Patch

Changes to src/pager.c.

   563    563   **   being modified.
   564    564   **
   565    565   **   Throughout a write-transaction, dbFileSize contains the size of
   566    566   **   the file on disk in pages. It is set to a copy of dbSize when the
   567    567   **   write-transaction is first opened, and updated when VFS calls are made
   568    568   **   to write or truncate the database file on disk.
   569    569   **
          570  +**   The only reason the dbFileSize variable is required is to suppress 
          571  +**   unnecessary calls to xTruncate() after committing a transaction. If, 
          572  +**   when a transaction is committed, the dbFileSize variable indicates 
          573  +**   that the database file is larger than the database image (Pager.dbSize), 
          574  +**   pager_truncate() is called. The pager_truncate() call uses xFilesize()
          575  +**   to measure the database file on disk, and then truncates it if required.
          576  +**   dbFileSize is not used when rolling back a transaction. In this case
          577  +**   pager_truncate() is called unconditionally (which means there may be
          578  +**   a call to xFilesize() that is not strictly required). In either case,
          579  +**   pager_truncate() may cause the file to become smaller or larger.
          580  +**
          581  +** dbHintSize
          582  +**
          583  +**   The dbHintSize variable is used to limit the number of calls made to
          584  +**   the VFS xFileControl(FCNTL_SIZE_HINT) method. 
          585  +**
          586  +**   dbHintSize is set to a copy of the dbSize variable when a
          587  +**   write-transaction is opened (at the same time as dbFileSize and
          588  +**   dbOrigSize). If the xFileControl(FCNTL_SIZE_HINT) method is called,
          589  +**   dbHintSize is increased to the number of pages that correspond to the
          590  +**   size-hint passed to the method call. See pager_write_pagelist() for 
          591  +**   details.
          592  +**
   570    593   ** errCode
   571    594   **
   572    595   **   The Pager.errCode variable is only ever used in PAGER_ERROR state. It
   573    596   **   is set to zero in all other states. In PAGER_ERROR state, Pager.errCode 
   574    597   **   is always set to SQLITE_FULL, SQLITE_IOERR or one of the SQLITE_IOERR_XXX 
   575    598   **   sub-codes.
   576    599   **
................................................................................
   615    638     u8 setMaster;               /* True if a m-j name has been written to jrnl */
   616    639     u8 doNotSpill;              /* Do not spill the cache when non-zero */
   617    640     u8 doNotSyncSpill;          /* Do not do a spill that requires jrnl sync */
   618    641     u8 subjInMemory;            /* True to use in-memory sub-journals */
   619    642     Pgno dbSize;                /* Number of pages in the database */
   620    643     Pgno dbOrigSize;            /* dbSize before the current transaction */
   621    644     Pgno dbFileSize;            /* Number of pages in the database file */
          645  +  Pgno dbHintSize;            /* Value passed to FCNTL_SIZE_HINT call */
   622    646     int errCode;                /* One of several kinds of errors */
   623    647     int nRec;                   /* Pages journalled since last j-header written */
   624    648     u32 cksumInit;              /* Quasi-random value added to every checksum */
   625    649     u32 nSubRec;                /* Number of records written to sub-journal */
   626    650     Bitvec *pInJournal;         /* One bit for each page in the database file */
   627    651     sqlite3_file *fd;           /* File descriptor for database */
   628    652     sqlite3_file *jfd;          /* File descriptor for main journal */
................................................................................
   827    851         assert( p->eLock!=UNKNOWN_LOCK );
   828    852         assert( pPager->errCode==SQLITE_OK );
   829    853         if( !pagerUseWal(pPager) ){
   830    854           assert( p->eLock>=RESERVED_LOCK );
   831    855         }
   832    856         assert( pPager->dbSize==pPager->dbOrigSize );
   833    857         assert( pPager->dbOrigSize==pPager->dbFileSize );
          858  +      assert( pPager->dbOrigSize==pPager->dbHintSize );
   834    859         assert( pPager->setMaster==0 );
   835    860         break;
   836    861   
   837    862       case PAGER_WRITER_CACHEMOD:
   838    863         assert( p->eLock!=UNKNOWN_LOCK );
   839    864         assert( pPager->errCode==SQLITE_OK );
   840    865         if( !pagerUseWal(pPager) ){
................................................................................
   846    871           assert( p->eLock>=RESERVED_LOCK );
   847    872           assert( isOpen(p->jfd) 
   848    873                || p->journalMode==PAGER_JOURNALMODE_OFF 
   849    874                || p->journalMode==PAGER_JOURNALMODE_WAL 
   850    875           );
   851    876         }
   852    877         assert( pPager->dbOrigSize==pPager->dbFileSize );
          878  +      assert( pPager->dbOrigSize==pPager->dbHintSize );
   853    879         break;
   854    880   
   855    881       case PAGER_WRITER_DBMOD:
   856    882         assert( p->eLock==EXCLUSIVE_LOCK );
   857    883         assert( pPager->errCode==SQLITE_OK );
   858    884         assert( !pagerUseWal(pPager) );
   859    885         assert( p->eLock>=EXCLUSIVE_LOCK );
   860    886         assert( isOpen(p->jfd) 
   861    887              || p->journalMode==PAGER_JOURNALMODE_OFF 
   862    888              || p->journalMode==PAGER_JOURNALMODE_WAL 
   863    889         );
          890  +      assert( pPager->dbOrigSize<=pPager->dbHintSize );
   864    891         break;
   865    892   
   866    893       case PAGER_WRITER_FINISHED:
   867    894         assert( p->eLock==EXCLUSIVE_LOCK );
   868    895         assert( pPager->errCode==SQLITE_OK );
   869    896         assert( !pagerUseWal(pPager) );
   870    897         assert( isOpen(p->jfd) 
................................................................................
  3825   3852   ** the database file.
  3826   3853   **
  3827   3854   ** If everything is successful, SQLITE_OK is returned. If an IO error 
  3828   3855   ** occurs, an IO error code is returned. Or, if the EXCLUSIVE lock cannot
  3829   3856   ** be obtained, SQLITE_BUSY is returned.
  3830   3857   */
  3831   3858   static int pager_write_pagelist(Pager *pPager, PgHdr *pList){
  3832         -  int rc;                              /* Return code */
         3859  +  int rc = SQLITE_OK;                  /* Return code */
  3833   3860   
  3834         -  /* Normally, this function is called in WRITER_DBMOD state.
  3835         -  **
  3836         -  ** However it may be called in WRITER_CACHEMOD state if the page being
  3837         -  ** written (and all other pages that reside on the same disk sector) was
  3838         -  ** a free-list leaf page at the start of the transaction. In that case
  3839         -  ** the database file is not really being modified, so it is Ok to write
  3840         -  ** to it in CACHEMOD state.
  3841         -  */
         3861  +  /* This function is only called for rollback pagers in WRITER_DBMOD state. */
  3842   3862     assert( !pagerUseWal(pPager) );
  3843         -  assert( pPager->eState==PAGER_WRITER_DBMOD
  3844         -       || pPager->eState==PAGER_WRITER_CACHEMOD 
  3845         -  );
  3846         -  assert( pPager->eState==PAGER_WRITER_DBMOD 
  3847         -       || (pList->pDirty==0 && pList->pgno<=pPager->dbFileSize)
  3848         -  );
  3849         -
  3850         -  /* At this point there may be either a RESERVED or EXCLUSIVE lock on the
  3851         -  ** database file. If there is already an EXCLUSIVE lock, the following
  3852         -  ** call is a no-op.
  3853         -  **
  3854         -  ** Moving the lock from RESERVED to EXCLUSIVE actually involves going
  3855         -  ** through an intermediate state PENDING.   A PENDING lock prevents new
  3856         -  ** readers from attaching to the database but is unsufficient for us to
  3857         -  ** write.  The idea of a PENDING lock is to prevent new readers from
  3858         -  ** coming in while we wait for existing readers to clear.
  3859         -  **
  3860         -  ** While the pager is in the RESERVED state, the original database file
  3861         -  ** is unchanged and we can rollback without having to playback the
  3862         -  ** journal into the original database file.  Once we transition to
  3863         -  ** EXCLUSIVE, it means the database file has been changed and any rollback
  3864         -  ** will require a journal playback.
  3865         -  */
  3866         -  rc = pager_wait_on_lock(pPager, EXCLUSIVE_LOCK);
         3863  +  assert( pPager->eState==PAGER_WRITER_DBMOD );
         3864  +  assert( pPager->eLock==EXCLUSIVE_LOCK );
  3867   3865   
  3868   3866     /* If the file is a temp-file has not yet been opened, open it now. It
  3869   3867     ** is not possible for rc to be other than SQLITE_OK if this branch
  3870   3868     ** is taken, as pager_wait_on_lock() is a no-op for temp-files.
  3871   3869     */
  3872   3870     if( !isOpen(pPager->fd) ){
  3873   3871       assert( pPager->tempFile && rc==SQLITE_OK );
................................................................................
  3874   3872       rc = pagerOpentemp(pPager, pPager->fd, pPager->vfsFlags);
  3875   3873     }
  3876   3874   
  3877   3875     /* Before the first write, give the VFS a hint of what the final
  3878   3876     ** file size will be.
  3879   3877     */
  3880   3878     assert( rc!=SQLITE_OK || isOpen(pPager->fd) );
  3881         -  if( rc==SQLITE_OK && pPager->dbSize>pPager->dbOrigSize ){
         3879  +  if( rc==SQLITE_OK && pPager->dbSize>pPager->dbHintSize ){
  3882   3880       sqlite3_int64 szFile = pPager->pageSize * (sqlite3_int64)pPager->dbSize;
  3883   3881       sqlite3OsFileControl(pPager->fd, SQLITE_FCNTL_SIZE_HINT, &szFile);
         3882  +    pPager->dbHintSize = pPager->dbSize;
  3884   3883     }
  3885   3884   
  3886   3885     while( rc==SQLITE_OK && pList ){
  3887   3886       Pgno pgno = pList->pgno;
  3888   3887   
  3889   3888       /* If there are dirty pages in the page cache with page numbers greater
  3890   3889       ** than Pager.dbSize, this means sqlite3PagerTruncateImage() was called to
................................................................................
  4058   4057       }
  4059   4058       if( rc==SQLITE_OK ){
  4060   4059         rc = pagerWalFrames(pPager, pPg, 0, 0, 0);
  4061   4060       }
  4062   4061     }else{
  4063   4062     
  4064   4063       /* Sync the journal file if required. */
  4065         -    if( pPg->flags&PGHDR_NEED_SYNC ){
         4064  +    if( pPg->flags&PGHDR_NEED_SYNC 
         4065  +     || pPager->eState==PAGER_WRITER_CACHEMOD
         4066  +    ){
  4066   4067         rc = syncJournal(pPager, 1);
  4067   4068       }
  4068   4069     
  4069   4070       /* If the page number of this page is larger than the current size of
  4070   4071       ** the database image, it may need to be written to the sub-journal.
  4071   4072       ** This is because the call to pager_write_pagelist() below will not
  4072   4073       ** actually write data to the file in this case.
................................................................................
  5120   5121         ** when it has an open transaction, but never to DBMOD or FINISHED.
  5121   5122         ** This is because in those states the code to roll back savepoint 
  5122   5123         ** transactions may copy data from the sub-journal into the database 
  5123   5124         ** file as well as into the page cache. Which would be incorrect in 
  5124   5125         ** WAL mode.
  5125   5126         */
  5126   5127         pPager->eState = PAGER_WRITER_LOCKED;
  5127         -      pPager->dbFileSize = pPager->dbOrigSize = pPager->dbSize;
         5128  +      pPager->dbHintSize = pPager->dbSize;
         5129  +      pPager->dbFileSize = pPager->dbSize;
         5130  +      pPager->dbOrigSize = pPager->dbSize;
  5128   5131         pPager->journalOff = 0;
  5129   5132       }
  5130   5133   
  5131   5134       assert( rc==SQLITE_OK || pPager->eState==PAGER_READER );
  5132   5135       assert( rc!=SQLITE_OK || pPager->eState==PAGER_WRITER_LOCKED );
  5133   5136       assert( assert_pager_state(pPager) );
  5134   5137     }