SQLite

Check-in [8c402db7e0]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Modify the windows locking code so that it works correctly for a database being shared between Win95/98/ME and WinNT/2K/XP systems. Ticket #310. (CVS 988)
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 8c402db7e0745622d9950e5ca5d4d8e933da436c
User & Date: drh 2003-05-29 17:43:08.000
Context
2003-05-29
17:50
Change the row-size limit back to 1MB. It was temporarily raised to 16MB. We'll probably move it to 16MB eventually, but not just yet. (CVS 989) (check-in: b84c4035c6 user: drh tags: trunk)
17:43
Modify the windows locking code so that it works correctly for a database being shared between Win95/98/ME and WinNT/2K/XP systems. Ticket #310. (CVS 988) (check-in: 8c402db7e0 user: drh tags: trunk)
04:21
Added typeof() operator. Minor additions for ATTACH/DETACH. (CVS 987) (check-in: 8b8fa0fff2 user: jplyon tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to src/os.c.
988
989
990
991
992
993
994







995
996
997
998

999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
#endif
}

#if OS_WIN
/*
** Return true (non-zero) if we are running under WinNT, Win2K or WinXP.
** Return false (zero) for Win95, Win98, or WinME.







*/
int isNT(void){
  static osType = 0;   /* 0=unknown 1=win95 2=winNT */
  if( osType==0 ){

    OSVERSIONINFO sInfo;
    sInfo.dwOSVersionInfoSize = sizeof(sInfo);
    GetVersionEx(&sInfo);
    osType = sInfo.dwPlatformId==VER_PLATFORM_WIN32_NT ? 2 : 1;
  }
  return osType==2;
}
#endif

/*
** Windows file locking notes:  [the same/equivalent applies to MacOS]
**
** We cannot use LockFileEx() or UnlockFileEx() because those functions
** are not available under Win95/98/ME.  So we use only LockFile() and
** UnlockFile().
**
** LockFile() prevents not just writing but also reading by other processes.
** (This is a design error on the part of Windows, but there is nothing
** we can do about that.)  So the region used for locking is at the
** end of the file where it is unlikely to ever interfere with an
** actual read attempt.







>
>
>
>
>
>
>




>










|

|
|







988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
#endif
}

#if OS_WIN
/*
** Return true (non-zero) if we are running under WinNT, Win2K or WinXP.
** Return false (zero) for Win95, Win98, or WinME.
**
** Here is an interesting observation:  Win95, Win98, and WinME lack
** the LockFileEx() API.  But we can still statically link against that
** API as long as we don't call it win running Win95/98/ME.  A call to
** this routine is used to determine if the host is Win95/98/ME or
** WinNT/2K/XP so that we will know whether or not we can safely call
** the LockFileEx() API.
*/
int isNT(void){
  static osType = 0;   /* 0=unknown 1=win95 2=winNT */
  if( osType==0 ){
    int tmpOsType;
    OSVERSIONINFO sInfo;
    sInfo.dwOSVersionInfoSize = sizeof(sInfo);
    GetVersionEx(&sInfo);
    osType = sInfo.dwPlatformId==VER_PLATFORM_WIN32_NT ? 2 : 1;
  }
  return osType==2;
}
#endif

/*
** Windows file locking notes:  [similar issues apply to MacOS]
**
** We cannot use LockFileEx() or UnlockFileEx() on Win95/98/ME because
** those functions are not available.  So we use only LockFile() and
** UnlockFile().
**
** LockFile() prevents not just writing but also reading by other processes.
** (This is a design error on the part of Windows, but there is nothing
** we can do about that.)  So the region used for locking is at the
** end of the file where it is unlikely to ever interfere with an
** actual read attempt.
1029
1030
1031
1032
1033
1034
1035








1036
1037
1038
1039
1040
1041
1042
** either a read lock or a write lock.  This prevents two processes from
** attempting to get a lock at a same time.  The semantics of 
** sqliteOsReadLock() require that if there is already a write lock, that
** lock is converted into a read lock atomically.  The lock on the first
** byte allows us to drop the old write lock and get the read lock without
** another process jumping into the middle and messing us up.  The same
** argument applies to sqliteOsWriteLock().








**
** Note: On MacOS we use the resource fork for locking.
**
** The following #defines specify the range of bytes used for locking.
** N_LOCKBYTE is the number of bytes available for doing the locking.
** The first byte used to hold the lock while the lock is changing does
** not count toward this number.  FIRST_LOCKBYTE is the address of







>
>
>
>
>
>
>
>







1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
** either a read lock or a write lock.  This prevents two processes from
** attempting to get a lock at a same time.  The semantics of 
** sqliteOsReadLock() require that if there is already a write lock, that
** lock is converted into a read lock atomically.  The lock on the first
** byte allows us to drop the old write lock and get the read lock without
** another process jumping into the middle and messing us up.  The same
** argument applies to sqliteOsWriteLock().
**
** On WinNT/2K/XP systems, LockFileEx() and UnlockFileEx() are available,
** which means we can use reader/writer locks.  When reader writer locks
** are used, the lock is placed on the same range of bytes that is used
** for probabilistic locking in Win95/98/ME.  Hence, the locking scheme
** will support two or more Win95 readers or two or more WinNT readers.
** But a single Win95 reader will lock out all WinNT readers and a single
** WinNT reader will lock out all other Win95 readers.
**
** Note: On MacOS we use the resource fork for locking.
**
** The following #defines specify the range of bytes used for locking.
** N_LOCKBYTE is the number of bytes available for doing the locking.
** The first byte used to hold the lock while the lock is changing does
** not count toward this number.  FIRST_LOCKBYTE is the address of
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104








1105

1106
1107
1108
1109
1110
1111
1112
1113
  int rc;
  if( id->locked>0 ){
    rc = SQLITE_OK;
  }else{
    int lk = (sqliteRandomInteger() & 0x7ffffff)%N_LOCKBYTE+1;
    int res;
    int cnt = 100;
    int page = isNT() ? 0xffffffff : 0;
    while( cnt-->0 && (res = LockFile(id->h, FIRST_LOCKBYTE, page, 1, 0))==0 ){
      Sleep(1);
    }
    if( res ){
      UnlockFile(id->h, FIRST_LOCKBYTE+1, page, N_LOCKBYTE, 0);








      res = LockFile(id->h, FIRST_LOCKBYTE+lk, page, 1, 0);

      UnlockFile(id->h, FIRST_LOCKBYTE, page, 1, 0);
    }
    if( res ){
      id->locked = lk;
      rc = SQLITE_OK;
    }else{
      rc = SQLITE_BUSY;
    }







<
|



|
>
>
>
>
>
>
>
>
|
>
|







1108
1109
1110
1111
1112
1113
1114

1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
  int rc;
  if( id->locked>0 ){
    rc = SQLITE_OK;
  }else{
    int lk = (sqliteRandomInteger() & 0x7ffffff)%N_LOCKBYTE+1;
    int res;
    int cnt = 100;

    while( cnt-->0 && (res = LockFile(id->h, FIRST_LOCKBYTE, 0, 1, 0))==0 ){
      Sleep(1);
    }
    if( res ){
      UnlockFile(id->h, FIRST_LOCKBYTE+1, 0, N_LOCKBYTE, 0);
      if( isNT() ){
        OVERLAPPED ovlp;
        ovlp.Offset = FIRST_LOCKBYTE+1;
        ovlp.OffsetHigh = 0;
        ovlp.hEvent = 0;
        res = LockFileEx(id->h, LOCKFILE_FAIL_IMMEDIATELY, 
                          0, N_LOCKBYTE, 0, &ovlp);
      }else{
        res = LockFile(id->h, FIRST_LOCKBYTE+lk, 0, 1, 0);
      }
      UnlockFile(id->h, FIRST_LOCKBYTE, 0, 1, 0);
    }
    if( res ){
      id->locked = lk;
      rc = SQLITE_OK;
    }else{
      rc = SQLITE_BUSY;
    }
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199



1200



1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
#if OS_WIN
  int rc;
  if( id->locked<0 ){
    rc = SQLITE_OK;
  }else{
    int res;
    int cnt = 100;
    int page = isNT() ? 0xffffffff : 0;
    while( cnt-->0 && (res = LockFile(id->h, FIRST_LOCKBYTE, page, 1, 0))==0 ){
      Sleep(1);
    }
    if( res ){
      if( id->locked==0 



            || UnlockFile(id->h, FIRST_LOCKBYTE + id->locked, page, 1, 0) ){



        res = LockFile(id->h, FIRST_LOCKBYTE+1, page, N_LOCKBYTE, 0);
      }else{
        res = 0;
      }
      UnlockFile(id->h, FIRST_LOCKBYTE, page, 1, 0);
    }
    if( res ){
      id->locked = -1;
      rc = SQLITE_OK;
    }else{
      rc = SQLITE_BUSY;
    }







<
|



|
>
>
>
|
>
>
>
|



|







1211
1212
1213
1214
1215
1216
1217

1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
#if OS_WIN
  int rc;
  if( id->locked<0 ){
    rc = SQLITE_OK;
  }else{
    int res;
    int cnt = 100;

    while( cnt-->0 && (res = LockFile(id->h, FIRST_LOCKBYTE, 0, 1, 0))==0 ){
      Sleep(1);
    }
    if( res ){
      if( id->locked>0 ){
        if( isNT() ){
          UnlockFile(id->h, FIRST_LOCKBYTE+1, 0, N_LOCKBYTE, 0);
        }else{
          res = UnlockFile(id->h, FIRST_LOCKBYTE + id->locked, 0, 1, 0);
        }
      }
      if( res ){
        res = LockFile(id->h, FIRST_LOCKBYTE+1, 0, N_LOCKBYTE, 0);
      }else{
        res = 0;
      }
      UnlockFile(id->h, FIRST_LOCKBYTE, 0, 1, 0);
    }
    if( res ){
      id->locked = -1;
      rc = SQLITE_OK;
    }else{
      rc = SQLITE_BUSY;
    }
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
1308
1309
  }
  sqliteOsLeaveMutex();
  id->locked = 0;
  return rc;
#endif
#if OS_WIN
  int rc;
  int page = isNT() ? 0xffffffff : 0;
  if( id->locked==0 ){
    rc = SQLITE_OK;
  }else if( id->locked<0 ){
    UnlockFile(id->h, FIRST_LOCKBYTE+1, page, N_LOCKBYTE, 0);
    rc = SQLITE_OK;
    id->locked = 0;
  }else{
    UnlockFile(id->h, FIRST_LOCKBYTE+id->locked, page, 1, 0);
    rc = SQLITE_OK;
    id->locked = 0;
  }
  return rc;
#endif
#if OS_MAC
  int rc;







<


|
|



|







1316
1317
1318
1319
1320
1321
1322

1323
1324
1325
1326
1327
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
  }
  sqliteOsLeaveMutex();
  id->locked = 0;
  return rc;
#endif
#if OS_WIN
  int rc;

  if( id->locked==0 ){
    rc = SQLITE_OK;
  }else if( isNT() || id->locked<0 ){
    UnlockFile(id->h, FIRST_LOCKBYTE+1, 0, N_LOCKBYTE, 0);
    rc = SQLITE_OK;
    id->locked = 0;
  }else{
    UnlockFile(id->h, FIRST_LOCKBYTE+id->locked, 0, 1, 0);
    rc = SQLITE_OK;
    id->locked = 0;
  }
  return rc;
#endif
#if OS_MAC
  int rc;
Changes to src/sqliteInt.h.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
/*
** 2001 September 15
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** Internal interface definitions for SQLite.
**
** @(#) $Id: sqliteInt.h,v 1.186 2003/05/17 19:04:04 drh Exp $
*/
#include "config.h"
#include "sqlite.h"
#include "hash.h"
#include "vdbe.h"
#include "parse.h"
#include "btree.h"













|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
/*
** 2001 September 15
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** Internal interface definitions for SQLite.
**
** @(#) $Id: sqliteInt.h,v 1.187 2003/05/29 17:43:08 drh Exp $
*/
#include "config.h"
#include "sqlite.h"
#include "hash.h"
#include "vdbe.h"
#include "parse.h"
#include "btree.h"
128
129
130
131
132
133
134




135

136
137
138
139
140
141
142
143
** row of a single table.  The upper bound on this limit is 16777215
** bytes (or 16MB-1).  We have arbitrarily set the limit to just 1MB
** here because the overflow page chain is inefficient for really big
** records and we want to discourage people from thinking that 
** multi-megabyte records are OK.  If your needs are different, you can
** change this define and recompile to increase or decrease the record
** size.




*/

#define MAX_BYTES_PER_ROW  1048576

/*
** If memory allocation problems are found, recompile with
**
**      -DMEMORY_DEBUG=1
**
** to enable some sanity checking on malloc() and free().  To







>
>
>
>

>
|







128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
** row of a single table.  The upper bound on this limit is 16777215
** bytes (or 16MB-1).  We have arbitrarily set the limit to just 1MB
** here because the overflow page chain is inefficient for really big
** records and we want to discourage people from thinking that 
** multi-megabyte records are OK.  If your needs are different, you can
** change this define and recompile to increase or decrease the record
** size.
**
** The 16777198 is computed as follows:  238 bytes of payload on the
** original pages plus 16448 overflow pages each holding 1020 bytes of
** data.
*/
/* #define MAX_BYTES_PER_ROW  1048576 */
#define MAX_BYTES_PER_ROW 16777198

/*
** If memory allocation problems are found, recompile with
**
**      -DMEMORY_DEBUG=1
**
** to enable some sanity checking on malloc() and free().  To
Changes to www/faq.tcl.
1
2
3
4
5
6
7
8
9
10
11
#
# Run this script to generated a faq.html output file
#
set rcsid {$Id: faq.tcl,v 1.22 2003/05/03 19:04:04 drh Exp $}

puts {<html>
<head>
  <title>SQLite Frequently Asked Questions</title>
</head>
<body bgcolor="white">
<h1 align="center">Frequently Asked Questions</h1>



|







1
2
3
4
5
6
7
8
9
10
11
#
# Run this script to generated a faq.html output file
#
set rcsid {$Id: faq.tcl,v 1.23 2003/05/29 17:43:08 drh Exp $}

puts {<html>
<head>
  <title>SQLite Frequently Asked Questions</title>
</head>
<body bgcolor="white">
<h1 align="center">Frequently Asked Questions</h1>
195
196
197
198
199
200
201

202
203
204
205




206
207
208
209
210
211
212
  This problem was resolved in version 2.7.0 by implementing a user-space
  probabilistic reader/writer locking strategy in the windows interface
  code file.  Windows
  now works like Unix in allowing multiple simultaneous readers.</p>

  <p>The locking mechanism used to control simultaneous access might
  not work correctly if the database file is kept on an NFS filesystem.

  You should avoid putting SQLite database files on NFS if multiple
  processes might try to access the file at the same time.  On Windows,
  Microsoft's documentation says that locking may not work under FAT
  filesystems if you are not running the Share.exe daemon.</p>





  <p>Locking in SQLite is very course-grained.  SQLite locks the
  entire database.  Big database servers (PostgreSQL, Oracle, etc.)
  generally have finer grained locking, such as locking on a single
  table or a single row within a table.  If you have a massively
  parallel database application, you should consider using a big database
  server instead of SQLite.</p>







>



|
>
>
>
>







195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
  This problem was resolved in version 2.7.0 by implementing a user-space
  probabilistic reader/writer locking strategy in the windows interface
  code file.  Windows
  now works like Unix in allowing multiple simultaneous readers.</p>

  <p>The locking mechanism used to control simultaneous access might
  not work correctly if the database file is kept on an NFS filesystem.
  This is because file locking is broken on some NFS implementations.
  You should avoid putting SQLite database files on NFS if multiple
  processes might try to access the file at the same time.  On Windows,
  Microsoft's documentation says that locking may not work under FAT
  filesystems if you are not running the Share.exe daemon.  People who
  have a lot of experience with Windows tell me that file locking of
  network files is very buggy and is not dependable.  If what they
  say is true, sharing an SQLite database between two or more Windows
  machines might cause unexpected problems.</p>

  <p>Locking in SQLite is very course-grained.  SQLite locks the
  entire database.  Big database servers (PostgreSQL, Oracle, etc.)
  generally have finer grained locking, such as locking on a single
  table or a single row within a table.  If you have a massively
  parallel database application, you should consider using a big database
  server instead of SQLite.</p>