/ Check-in [ffce4aac]
Login
SQLite training in Houston TX on 2019-11-05 (details)
Part of the 2019 Tcl Conference

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Make the winTruncate() method of the windows VFS be a no-op if there are outstanding references to the memory-mapped pages. Otherwise, memory might be deleted out from under those references when the file is remapped during the truncate operation.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: ffce4aac18dacbf2a3112ae2ab56c7db20cb164f179683d90a66ef38f4a98f2b
User & Date: drh 2018-11-24 17:46:07
Context
2018-11-24
20:44
Fix a bug in the geopoly_ccw() function. Test cases in TH3. check-in: 773c5c1d user: drh tags: trunk
17:46
Make the winTruncate() method of the windows VFS be a no-op if there are outstanding references to the memory-mapped pages. Otherwise, memory might be deleted out from under those references when the file is remapped during the truncate operation. check-in: ffce4aac user: drh tags: trunk
16:07
Remove the unused mmapSizeActual field from the Windows sqlite3_file implementation. check-in: 0e7aa622 user: drh tags: trunk
2018-11-23
13:21
Make the winTruncate() method of the windows VFS be a no-op if there are outstanding references to the memory-mapped pages. Otherwise, memory might be deleted out from under those references when the file is remapped during the truncate operation. Leaf check-in: 8576ccb4 user: drh tags: branch-3.25
Changes
Hide Diffs Side-by-Side Diffs Ignore Whitespace Patch

Changes to src/os_win.c.

  2901   2901   */
  2902   2902   static int winTruncate(sqlite3_file *id, sqlite3_int64 nByte){
  2903   2903     winFile *pFile = (winFile*)id;  /* File handle object */
  2904   2904     int rc = SQLITE_OK;             /* Return code for this function */
  2905   2905     DWORD lastErrno;
  2906   2906   #if SQLITE_MAX_MMAP_SIZE>0
  2907   2907     sqlite3_int64 oldMmapSize;
         2908  +  if( pFile->nFetchOut>0 ){
         2909  +    /* File truncation is a no-op if there are outstanding memory mapped
         2910  +    ** pages.  This is because truncating the file means temporarily unmapping
         2911  +    ** the file, and that might delete memory out from under existing cursors.
         2912  +    **
         2913  +    ** This can result in incremental vacuum not truncating the file,
         2914  +    ** if there is an active read cursor when the incremental vacuum occurs.
         2915  +    ** No real harm comes of this - the database file is not corrupted,
         2916  +    ** though some folks might complain that the file is bigger than it
         2917  +    ** needs to be.
         2918  +    **
         2919  +    ** The only feasible work-around is to defer the truncation until after
         2920  +    ** all references to memory-mapped content are closed.  That is doable,
         2921  +    ** but involves adding a few branches in the common write code path which
         2922  +    ** could slow down normal operations slightly.  Hence, we have decided for
         2923  +    ** now to simply make trancations a no-op if there are pending reads.  We
         2924  +    ** can maybe revisit this decision in the future.
         2925  +    */
         2926  +    return SQLITE_OK;
         2927  +  }
  2908   2928   #endif
  2909   2929   
  2910   2930     assert( pFile );
  2911   2931     SimulateIOError(return SQLITE_IOERR_TRUNCATE);
  2912   2932     OSTRACE(("TRUNCATE pid=%lu, pFile=%p, file=%p, size=%lld, lock=%d\n",
  2913   2933              osGetCurrentProcessId(), pFile, pFile->h, nByte, pFile->locktype));
  2914   2934   

Changes to test/incrvacuum.test.

   778    778     catchsql {INSERT INTO t2 SELECT * FROM t1}
   779    779   
   780    780     execsql { 
   781    781       COMMIT;
   782    782       PRAGMA integrity_check;
   783    783     }
   784    784   } {ok}
          785  +
          786  +#-------------------------------------------------------------------------
          787  +# At one point it was unsafe to truncate a db file on windows while there
          788  +# were outstanding xFetch() references. This test case attempts to hit
          789  +# that case.
          790  +#
          791  +reset_db
          792  +do_execsql_test incrvacuum-16.0 {
          793  +  PRAGMA auto_vacuum = 2;
          794  +  CREATE TABLE t3(a);
          795  +  INSERT INTO t3 VALUES(1), (2), (3), (4);
          796  +
          797  +  CREATE TABLE t2(x);
          798  +  INSERT INTO t2 VALUES( randomblob(1000) );
          799  +  INSERT INTO t2 VALUES( randomblob(1000) );
          800  +  INSERT INTO t2 VALUES( randomblob(1000) );
          801  +  INSERT INTO t2 VALUES( randomblob(1000) );
          802  +  INSERT INTO t2 VALUES( randomblob(1000) );
          803  +  INSERT INTO t2 VALUES( randomblob(1000) );
          804  +} {}
          805  +
          806  +# Reopen db to ensure the page-cache is empty.
          807  +#
          808  +db close
          809  +sqlite3 db test.db
          810  +
          811  +# Open db in mmap-mode. Open a transaction, delete some data, then run
          812  +# incremental-vacuum. Do not commit the transaction. 
          813  +#
          814  +do_execsql_test incrvacuum-16.1 {
          815  +  PRAGMA mmap_size = 1000000;
          816  +  BEGIN;
          817  +  DELETE FROM t2;
          818  +  PRAGMA incremental_vacuum = 1000;
          819  +} {1000000}
          820  +
          821  +# Scan through table t3 (which is all clean pages - so mmap is used). Then,
          822  +# midway through, commit the transaction. This causes the db to be truncated
          823  +# while there are outstanding xFetch pages.
          824  +#
          825  +do_test incrvacuum-16.2 {
          826  +  set res [list]
          827  +  db eval { SELECT a FROM t3 } {
          828  +    if {$a==3} { db eval COMMIT }
          829  +    lappend res $a
          830  +  }
          831  +  set res
          832  +} {1 2 3 4}
   785    833   
   786    834   finish_test