/ Changes On Branch begin-concurrent
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Changes In Branch begin-concurrent Excluding Merge-Ins

This is equivalent to a diff from 57c36a29 to b13dae1c

2020-02-10
19:37
Fix a typo in the in-tree begin-concurrent documentation (Leaf check-in: b13dae1c user: drh tags: begin-concurrent)
2020-02-05
18:28
Small size reduction and performance improvement in the sqlite3VdbeMemFromBtree() interface used to pull content out of the b-tree and into an sqlite3_value object. (check-in: ae6dd8d3 user: drh tags: trunk)
2020-02-04
20:11
Merge latest trunk changes into this branch. (Leaf check-in: 6ad4d6b7 user: dan tags: wal2)
20:09
Merge latest trunk changes, including the SQLITE_ENABLE_CORRUPT_PGNO patch, into this branch. (Leaf check-in: 23bc80cc user: dan tags: begin-concurrent-pnu)
20:08
Merge latest trunk changes, including the SQLITE_ENABLE_CORRUPT_PGNO patch, into this branch. (check-in: f253618a user: dan tags: begin-concurrent)
20:01
Enable more detailed log messages in SQLITE_ENABLE_CORRUPT_PGNO builds if database corruption is encountered. (check-in: 57c36a29 user: dan tags: trunk)
01:41
Extend the OP_Copy-coalesce optimization fix of check-in [b36126c1889e323c] so that it is also correctly disabled by the CASE operator. Ticket [9d3666754ac37d5a]. (check-in: 29a969d6 user: drh tags: trunk)

Added doc/begin_concurrent.md.





















































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106

Begin Concurrent
================

## Overview

Usually, SQLite allows at most one writer to proceed concurrently. The
BEGIN CONCURRENT enhancement allows multiple writers to process write
transactions simultanously if the database is in "wal" or "wal2" mode,
although the system still serializes COMMIT commands.

When a write-transaction is opened with "BEGIN CONCURRENT", actually 
locking the database is deferred until a COMMIT is executed. This means
that any number of transactions started with BEGIN CONCURRENT may proceed
concurrently. The system uses optimistic page-level-locking to prevent
conflicting concurrent transactions from being committed.

When a BEGIN CONCURRENT transaction is committed, the system checks whether 
or not any of the database pages that the transaction has read have been
modified since the BEGIN CONCURRENT was opened. In other words - it asks 
if the transaction being committed operates on a different set of data than
all other concurrently executing transactions. If the answer is "yes, this
transaction did not read or modify any data modified by any concurrent
transaction", then the transaction is committed as normal. Otherwise, if the
transaction does conflict, it cannot be committed and an SQLITE_BUSY_SNAPSHOT
error is returned. At this point, all the client can do is ROLLBACK the
transaction.

If SQLITE_BUSY_SNAPSHOT is returned, messages are output via the sqlite3_log
mechanism indicating the page and table or index on which the conflict
occurred. This can be useful when optimizing concurrency.

## Application Programming Notes

In order to serialize COMMIT processing, SQLite takes a lock on the database
as part of each COMMIT command and releases it before returning. At most one
writer may hold this lock at any one time. If a writer cannot obtain the lock,
it uses SQLite's busy-handler to pause and retry for a while:

  <a href=https://www.sqlite.org/c3ref/busy_handler.html>
      https://www.sqlite.org/c3ref/busy_handler.html
  </a>

If there is significant contention for the writer lock, this mechanism can be
inefficient. In this case it is better for the application to use a mutex or
some other mechanism that supports blocking to ensure that at most one writer
is attempting to COMMIT a BEGIN CONCURRENT transaction at a time. This is
usually easier if all writers are part of the same operating system process.

If all database clients (readers and writers) are located in the same OS
process, and if that OS is a Unix variant, then it can be more efficient to
the built-in VFS "unix-excl" instead of the default "unix". This is because it
uses more efficient locking primitives.

The key to maximizing concurrency using BEGIN CONCURRENT is to ensure that
there are a large number of non-conflicting transactions. In SQLite, each
table and each index is stored as a separate b-tree, each of which is
distributed over a discrete set of database pages. This means that:

  * Two transactions that write to different sets of tables never 
    conflict, and that

  * Two transactions that write to the same tables or indexes only 
    conflict if the values of the keys (either primary keys or indexed 
    rows) are fairly close together. For example, given a large 
    table with the schema:

      <pre>     CREATE TABLE t1(a INTEGER PRIMARY KEY, b BLOB);</pre>

    writing two rows with adjacent values for "a" probably will cause a
    conflict (as the two keys are stored on the same page), but writing two
    rows with vastly different values for "a" will not (as the keys will likly
    be stored on different pages).

Note that, in SQLite, if values are not explicitly supplied for an INTEGER
PRIMARY KEY, as for example in:

>
     INSERT INTO t1(b) VALUES(<blob-value>);

then monotonically increasing values are assigned automatically. This is
terrible for concurrency, as it all but ensures that all new rows are 
added to the same database page. In such situations, it is better to
explicitly assign random values to INTEGER PRIMARY KEY fields.

This problem also comes up for non-WITHOUT ROWID tables that do not have an
explicit INTEGER PRIMARY KEY column. In these cases each table has an implicit
INTEGER PRIMARY KEY column that is assigned increasing values, leading to the
same problem as omitting to assign a value to an explicit INTEGER PRIMARY KEY
column.

For both explicit and implicit INTEGER PRIMARY KEYs, it is possible to have
SQLite assign values at random (instead of the monotonically increasing
values) by writing a row with a rowid equal to the largest possible signed
64-bit integer to the table. For example:

     INSERT INTO t1(a) VALUES(9223372036854775807);

Applications should take care not to malfunction due to the presence of such
rows.

The nature of some types of indexes, for example indexes on timestamp fields,
can also cause problems (as concurrent transactions may assign similar
timestamps that will be stored on the same db page to new records). In these
cases the database schema may need to be rethought to increase the concurrency
provided by page-level-locking.

Added ext/session/changebatch1.test.





























































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
# 2016 August 23
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.
#

if {![info exists testdir]} {
  set testdir [file join [file dirname [info script]] .. .. test]
} 
source $testdir/tester.tcl
ifcapable !session {finish_test; return}

set testprefix changebatch1


proc sql_to_changeset {method sql} {
  sqlite3session S db main
  S attach *
  execsql $sql
  set ret [S $method]
  S delete
  return $ret
}

proc do_changebatch_test {tn method args} {
  set C [list]
  foreach a $args {
    lappend C [sql_to_changeset $method $a]
  }

  sqlite3changebatch cb db
  set i 1
  foreach ::cs [lrange $C 0 end-1] {
    set rc [cb add $::cs]
    if {$rc!="SQLITE_OK"} { error "expected SQLITE_OK, got $rc (i=$i)" }
    incr i
  }

  set ::cs [lindex $C end]
  do_test $tn { cb add [set ::cs] } SQLITE_CONSTRAINT
  cb delete
}

proc do_changebatch_test1 {tn args} {
  uplevel do_changebatch_test $tn changeset $args
}
proc do_changebatch_test2 {tn args} {
  uplevel do_changebatch_test $tn fullchangeset $args
}

#-------------------------------------------------------------------------
# The body of the following loop contains tests for database schemas
# that do not feature multi-column UNIQUE constraints. In this case
# it doesn't matter if the changesets are generated using
# sqlite3session_changeset() or sqlite3session_fullchangeset().
#
foreach {tn testfunction} {
  1 do_changebatch_test1
  2 do_changebatch_test2
} {
  reset_db

  #-------------------------------------------------------------------------
  #
  do_execsql_test $tn.1.0 {
    CREATE TABLE t1(a PRIMARY KEY, b);
  }
  
  $testfunction $tn.1.1 {
    INSERT INTO t1 VALUES(1, 1);
  } {
    DELETE FROM t1 WHERE a=1;
  }
  
  do_execsql_test $tn.1.2.0 {
    INSERT INTO t1 VALUES(1, 1);
    INSERT INTO t1 VALUES(2, 2);
    INSERT INTO t1 VALUES(3, 3);
  }
  $testfunction $tn.1.2.1 {
    DELETE FROM t1 WHERE a=2;
  } {
    INSERT INTO t1 VALUES(2, 2);
  }
  
  #-------------------------------------------------------------------------
  #
  do_execsql_test $tn.2.0 {
    CREATE TABLE x1(a, b PRIMARY KEY, c UNIQUE);
    CREATE TABLE x2(a PRIMARY KEY, b UNIQUE, c UNIQUE);
    CREATE INDEX x1a ON x1(a);
  
    INSERT INTO x1 VALUES(1, 1, 'a');
    INSERT INTO x1 VALUES(1, 2, 'b');
    INSERT INTO x1 VALUES(1, 3, 'c');
  }
  
  $testfunction $tn.2.1 {
    DELETE FROM x1 WHERE b=2;
  } {
    UPDATE x1 SET c='b' WHERE b=3;
  }
  
  $testfunction $tn.2.2 {
    DELETE FROM x1 WHERE b=1;
  } {
    INSERT INTO x1 VALUES(1, 5, 'a');
  }

  set L [list]
  for {set i 1000} {$i < 10000} {incr i} {
    lappend L "INSERT INTO x2 VALUES($i, $i, 'x' || $i)"
  }
  lappend L "DELETE FROM x2 WHERE b=1005"
  $testfunction $tn.2.3 {*}$L

  execsql { INSERT INTO x1 VALUES('f', 'f', 'f') }
  $testfunction $tn.2.4 {
    INSERT INTO x2 VALUES('f', 'f', 'f');
  } {
    INSERT INTO x1 VALUES('g', 'g', 'g');
  } {
    DELETE FROM x1 WHERE b='f';
  } {
    INSERT INTO x2 VALUES('g', 'g', 'g');
  } {
    INSERT INTO x1 VALUES('f', 'f', 'f');
  }

  execsql {
    DELETE FROM x1;
    INSERT INTO x1 VALUES(1.5, 1.5, 1.5);
  }
  $testfunction $tn.2.5 {
    DELETE FROM x1 WHERE b BETWEEN 1 AND 2;
  } {
    INSERT INTO x1 VALUES(2.5, 2.5, 2.5);
  } {
    INSERT INTO x1 VALUES(1.5, 1.5, 1.5);
  }

  execsql {
    DELETE FROM x2;
    INSERT INTO x2 VALUES(X'abcd', X'1234', X'7890');
    INSERT INTO x2 VALUES(X'0000', X'0000', X'0000');
  }
  breakpoint
  $testfunction $tn.2.6 {
    UPDATE x2 SET c = X'1234' WHERE a=X'abcd';
    INSERT INTO x2 VALUES(X'1234', X'abcd', X'7890');
  } {
    DELETE FROM x2 WHERE b=X'0000';
  } {
    INSERT INTO x2 VALUES(1, X'0000', NULL);
  }
}

#-------------------------------------------------------------------------
# Test some multi-column UNIQUE constraints. First Using _changeset() to
# demonstrate the problem, then using _fullchangeset() to show that it has
# been fixed.
#
reset_db
do_execsql_test 3.0 {
  CREATE TABLE y1(a PRIMARY KEY, b, c, UNIQUE(b, c));
  INSERT INTO y1 VALUES(1, 1, 1);
  INSERT INTO y1 VALUES(2, 2, 2);
  INSERT INTO y1 VALUES(3, 3, 3);
  INSERT INTO y1 VALUES(4, 3, 4);
  BEGIN;
}

do_test 3.1.1 {
  set c1 [sql_to_changeset changeset { DELETE FROM y1 WHERE a=4    }]
  set c2 [sql_to_changeset changeset { UPDATE y1 SET c=4 WHERE a=3 }]
  sqlite3changebatch cb db
  cb add $c1
  cb add $c2
} {SQLITE_OK}
do_test 3.1.2 {
  cb delete
  execsql ROLLBACK
} {}

do_test 3.1.1 {
  set c1 [sql_to_changeset fullchangeset { DELETE FROM y1 WHERE a=4    }]
  set c2 [sql_to_changeset fullchangeset { UPDATE y1 SET c=4 WHERE a=3 }]
  sqlite3changebatch cb db
  cb add $c1
  cb add $c2
} {SQLITE_CONSTRAINT}
do_test 3.1.2 {
  cb delete
} {}

#-------------------------------------------------------------------------
#
reset_db
do_execsql_test 4.0 {
  CREATE TABLE t1(x, y, z, PRIMARY KEY(x, y), UNIQUE(z));
}

do_test 4.1 {
  set c1 [sql_to_changeset fullchangeset { INSERT INTO t1 VALUES(1, 2, 3) }]
  execsql {
    DROP TABLE t1;
    CREATE TABLE t1(w, x, y, z, PRIMARY KEY(x, y), UNIQUE(z));
  }
  sqlite3changebatch cb db
  list [catch { cb add $c1 } msg] $msg
} {1 SQLITE_RANGE}



finish_test

Added ext/session/changebatchfault.test.





















































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
# 2011 Mar 21
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# The focus of this file is testing the session module.
#

if {![info exists testdir]} {
  set testdir [file join [file dirname [info script]] .. .. test]
} 
source [file join [file dirname [info script]] session_common.tcl]
source $testdir/tester.tcl
ifcapable !session {finish_test; return}
set testprefix changebatchfault

do_execsql_test 1.0 {
  CREATE TABLE t1(a, b, c PRIMARY KEY, UNIQUE(a, b));
  INSERT INTO t1 VALUES('a', 'a', 'a');
  INSERT INTO t1 VALUES('b', 'b', 'b');
}

set ::c1 [changeset_from_sql { delete from t1 where c='a' }]
set ::c2 [changeset_from_sql { insert into t1 values('c', 'c', 'c') }]

do_faultsim_test 1 -faults oom-* -body {
  sqlite3changebatch cb db
  cb add $::c1
  cb add $::c2
} -test {
  faultsim_test_result {0 SQLITE_OK} {1 SQLITE_NOMEM}
  catch { cb delete }
}


finish_test

Added ext/session/sqlite3changebatch.c.











































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485

#if !defined(SQLITE_TEST) || (defined(SQLITE_ENABLE_SESSION) && defined(SQLITE_ENABLE_PREUPDATE_HOOK))

#include "sqlite3session.h"
#include "sqlite3changebatch.h"

#include <assert.h>
#include <string.h>

typedef struct BatchTable BatchTable;
typedef struct BatchIndex BatchIndex;
typedef struct BatchIndexEntry BatchIndexEntry;
typedef struct BatchHash BatchHash;

struct sqlite3_changebatch {
  sqlite3 *db;                    /* Database handle used to read schema */
  BatchTable *pTab;               /* First in linked list of tables */
  int iChangesetId;               /* Current changeset id */
  int iNextIdxId;                 /* Next available index id */
  int nEntry;                     /* Number of entries in hash table */
  int nHash;                      /* Number of hash buckets */
  BatchIndexEntry **apHash;       /* Array of hash buckets */
};

struct BatchTable {
  BatchIndex *pIdx;               /* First in linked list of UNIQUE indexes */
  BatchTable *pNext;              /* Next table */
  char zTab[1];                   /* Table name */
};

struct BatchIndex {
  BatchIndex *pNext;              /* Next index on same table */
  int iId;                        /* Index id (assigned internally) */
  int bPk;                        /* True for PK index */
  int nCol;                       /* Size of aiCol[] array */
  int *aiCol;                     /* Array of columns that make up index */
};

struct BatchIndexEntry {
  BatchIndexEntry *pNext;         /* Next colliding hash table entry */
  int iChangesetId;               /* Id of associated changeset */
  int iIdxId;                     /* Id of index this key is from */
  int szRecord;
  char aRecord[1];
};

/*
** Allocate and zero a block of nByte bytes. Must be freed using cbFree().
*/
static void *cbMalloc(int *pRc, int nByte){
  void *pRet;

  if( *pRc ){
    pRet = 0;
  }else{
    pRet = sqlite3_malloc(nByte);
    if( pRet ){
      memset(pRet, 0, nByte);
    }else{
      *pRc = SQLITE_NOMEM;
    }
  }

  return pRet;
}

/*
** Free an allocation made by cbMalloc().
*/
static void cbFree(void *p){
  sqlite3_free(p);
}

/*
** Return the hash bucket that pEntry belongs in.
*/
static int cbHash(sqlite3_changebatch *p, BatchIndexEntry *pEntry){
  unsigned int iHash = (unsigned int)pEntry->iIdxId;
  unsigned char *pEnd = (unsigned char*)&pEntry->aRecord[pEntry->szRecord];
  unsigned char *pIter;

  for(pIter=(unsigned char*)pEntry->aRecord; pIter<pEnd; pIter++){
    iHash += (iHash << 7) + *pIter;
  }

  return (int)(iHash % p->nHash);
}

/*
** Resize the hash table.
*/
static int cbHashResize(sqlite3_changebatch *p){
  int rc = SQLITE_OK;
  BatchIndexEntry **apNew;
  int nNew = (p->nHash ? p->nHash*2 : 512);
  int i;

  apNew = cbMalloc(&rc, sizeof(BatchIndexEntry*) * nNew);
  if( rc==SQLITE_OK ){
    int nHash = p->nHash;
    p->nHash = nNew;
    for(i=0; i<nHash; i++){
      BatchIndexEntry *pEntry;
      while( (pEntry=p->apHash[i])!=0 ){
        int iHash = cbHash(p, pEntry);
        p->apHash[i] = pEntry->pNext;
        pEntry->pNext = apNew[iHash];
        apNew[iHash] = pEntry;
      }
    }

    cbFree(p->apHash);
    p->apHash = apNew;
  }

  return rc;
}


/*
** Allocate a new sqlite3_changebatch object.
*/
int sqlite3changebatch_new(sqlite3 *db, sqlite3_changebatch **pp){
  sqlite3_changebatch *pRet;
  int rc = SQLITE_OK;
  *pp = pRet = (sqlite3_changebatch*)cbMalloc(&rc, sizeof(sqlite3_changebatch));
  if( pRet ){
    pRet->db = db;
  }
  return rc;
}

/*
** Add a BatchIndex entry for index zIdx to table pTab.
*/
static int cbAddIndex(
  sqlite3_changebatch *p, 
  BatchTable *pTab, 
  const char *zIdx, 
  int bPk
){
  int nCol = 0;
  sqlite3_stmt *pIndexInfo = 0;
  BatchIndex *pNew = 0;
  int rc;
  char *zIndexInfo;

  zIndexInfo = (char*)sqlite3_mprintf("PRAGMA main.index_info = %Q", zIdx);
  if( zIndexInfo ){
    rc = sqlite3_prepare_v2(p->db, zIndexInfo, -1, &pIndexInfo, 0);
    sqlite3_free(zIndexInfo);
  }else{
    rc = SQLITE_NOMEM;
  }

  if( rc==SQLITE_OK ){
    while( SQLITE_ROW==sqlite3_step(pIndexInfo) ){ nCol++; }
    rc = sqlite3_reset(pIndexInfo);
  }

  pNew = (BatchIndex*)cbMalloc(&rc, sizeof(BatchIndex) + sizeof(int) * nCol);
  if( rc==SQLITE_OK ){
    pNew->nCol = nCol;
    pNew->bPk = bPk;
    pNew->aiCol = (int*)&pNew[1];
    pNew->iId = p->iNextIdxId++;
    while( SQLITE_ROW==sqlite3_step(pIndexInfo) ){ 
      int i = sqlite3_column_int(pIndexInfo, 0);
      int j = sqlite3_column_int(pIndexInfo, 1);
      pNew->aiCol[i] = j;
    }
    rc = sqlite3_reset(pIndexInfo);
  }

  if( rc==SQLITE_OK ){
    pNew->pNext = pTab->pIdx;
    pTab->pIdx = pNew;
  }else{
    cbFree(pNew);
  }
  sqlite3_finalize(pIndexInfo);

  return rc;
}

/*
** Free the object passed as the first argument.
*/
static void cbFreeTable(BatchTable *pTab){
  BatchIndex *pIdx;
  BatchIndex *pIdxNext;
  for(pIdx=pTab->pIdx; pIdx; pIdx=pIdxNext){
    pIdxNext = pIdx->pNext;
    cbFree(pIdx);
  }
  cbFree(pTab);
}

/*
** Find or create the BatchTable object named zTab.
*/
static int cbFindTable(
  sqlite3_changebatch *p, 
  const char *zTab, 
  BatchTable **ppTab
){
  BatchTable *pRet = 0;
  int rc = SQLITE_OK;

  for(pRet=p->pTab; pRet; pRet=pRet->pNext){
    if( 0==sqlite3_stricmp(zTab, pRet->zTab) ) break;
  }

  if( pRet==0 ){
    int nTab = strlen(zTab);
    pRet = (BatchTable*)cbMalloc(&rc, nTab + sizeof(BatchTable));
    if( pRet ){
      sqlite3_stmt *pIndexList = 0;
      char *zIndexList = 0;
      int rc2;
      memcpy(pRet->zTab, zTab, nTab);

      zIndexList = sqlite3_mprintf("PRAGMA main.index_list = %Q", zTab);
      if( zIndexList==0 ){
        rc = SQLITE_NOMEM;
      }else{
        rc = sqlite3_prepare_v2(p->db, zIndexList, -1, &pIndexList, 0);
        sqlite3_free(zIndexList);
      }

      while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pIndexList) ){
        if( sqlite3_column_int(pIndexList, 2) ){
          const char *zIdx = (const char*)sqlite3_column_text(pIndexList, 1);
          const char *zTyp = (const char*)sqlite3_column_text(pIndexList, 3);
          rc = cbAddIndex(p, pRet, zIdx, (zTyp[0]=='p'));
        }
      }
      rc2 = sqlite3_finalize(pIndexList);
      if( rc==SQLITE_OK ) rc = rc2;

      if( rc==SQLITE_OK ){
        pRet->pNext = p->pTab;
        p->pTab = pRet;
      }else{
        cbFreeTable(pRet);
        pRet = 0;
      }
    }
  }

  *ppTab = pRet;
  return rc;
}

/*
** Extract value iVal from the changeset iterator passed as the first
** argument. Set *ppVal to point to the value before returning.
**
** This function attempts to extract the value using function xVal
** (which is always either sqlite3changeset_new or sqlite3changeset_old).
** If the call returns SQLITE_OK but does not supply an sqlite3_value*
** pointer, an attempt to extract the value is made using the xFallback 
** function.
*/
static int cbGetChangesetValue(
  sqlite3_changeset_iter *pIter, 
  int (*xVal)(sqlite3_changeset_iter*,int,sqlite3_value**),
  int (*xFallback)(sqlite3_changeset_iter*,int,sqlite3_value**),
  int iVal,
  sqlite3_value **ppVal
){
  int rc = xVal(pIter, iVal, ppVal);
  if( rc==SQLITE_OK && *ppVal==0 && xFallback ){
    rc = xFallback(pIter, iVal, ppVal);
  }
  return rc;
}

static int cbAddToHash(
  sqlite3_changebatch *p, 
  sqlite3_changeset_iter *pIter, 
  BatchIndex *pIdx, 
  int (*xVal)(sqlite3_changeset_iter*,int,sqlite3_value**),
  int (*xFallback)(sqlite3_changeset_iter*,int,sqlite3_value**),
  int *pbConf
){
  BatchIndexEntry *pNew;
  int sz = pIdx->nCol;
  int i;
  int iOut = 0;
  int rc = SQLITE_OK;

  for(i=0; rc==SQLITE_OK && i<pIdx->nCol; i++){
    sqlite3_value *pVal;
    rc = cbGetChangesetValue(pIter, xVal, xFallback, pIdx->aiCol[i], &pVal);
    if( rc==SQLITE_OK ){
      int eType = 0;
      if( pVal ) eType = sqlite3_value_type(pVal);
      switch( eType ){
        case 0:
        case SQLITE_NULL:
          return SQLITE_OK;

        case SQLITE_INTEGER:
          sz += 8;
          break;
        case SQLITE_FLOAT:
          sz += 8;
          break;

        default:
          assert( eType==SQLITE_TEXT || eType==SQLITE_BLOB );
          sz += sqlite3_value_bytes(pVal);
          break;
      }
    }
  }

  pNew = cbMalloc(&rc, sizeof(BatchIndexEntry) + sz);
  if( pNew ){
    pNew->iChangesetId = p->iChangesetId;
    pNew->iIdxId = pIdx->iId;
    pNew->szRecord = sz;

    for(i=0; i<pIdx->nCol; i++){
      int eType;
      sqlite3_value *pVal;
      rc = cbGetChangesetValue(pIter, xVal, xFallback, pIdx->aiCol[i], &pVal);
      if( rc!=SQLITE_OK ) break;  /* coverage: condition is never true */
      eType = sqlite3_value_type(pVal);
      pNew->aRecord[iOut++] = eType;
      switch( eType ){
        case SQLITE_INTEGER: {
          sqlite3_int64 i64 = sqlite3_value_int64(pVal);
          memcpy(&pNew->aRecord[iOut], &i64, 8);
          iOut += 8;
          break;
        }
        case SQLITE_FLOAT: {
          double d64 = sqlite3_value_double(pVal);
          memcpy(&pNew->aRecord[iOut], &d64, sizeof(double));
          iOut += sizeof(double);
          break;
        }

        default: {
          int nByte = sqlite3_value_bytes(pVal);
          const char *z = (const char*)sqlite3_value_blob(pVal);
          memcpy(&pNew->aRecord[iOut], z, nByte);
          iOut += nByte;
          break;
        }
      }
    }
  }

  if( rc==SQLITE_OK && p->nEntry>=(p->nHash/2) ){
    rc = cbHashResize(p);
  }

  if( rc==SQLITE_OK ){
    BatchIndexEntry *pIter;
    int iHash = cbHash(p, pNew);

    assert( iHash>=0 && iHash<p->nHash );
    for(pIter=p->apHash[iHash]; pIter; pIter=pIter->pNext){
      if( pNew->szRecord==pIter->szRecord 
       && 0==memcmp(pNew->aRecord, pIter->aRecord, pNew->szRecord)
      ){
        if( pNew->iChangesetId!=pIter->iChangesetId ){
          *pbConf = 1;
        }
        cbFree(pNew);
        pNew = 0;
        break;
      }
    }

    if( pNew ){
      pNew->pNext = p->apHash[iHash];
      p->apHash[iHash] = pNew;
      p->nEntry++;
    }
  }else{
    cbFree(pNew);
  }

  return rc;
}


/*
** Add a changeset to the current batch.
*/
int sqlite3changebatch_add(sqlite3_changebatch *p, void *pBuf, int nBuf){
  sqlite3_changeset_iter *pIter;  /* Iterator opened on pBuf/nBuf */
  int rc;                         /* Return code */
  int bConf = 0;                  /* Conflict was detected */

  rc = sqlite3changeset_start(&pIter, nBuf, pBuf);
  if( rc==SQLITE_OK ){
    int rc2;
    for(rc2 = sqlite3changeset_next(pIter);
        rc2==SQLITE_ROW;
        rc2 = sqlite3changeset_next(pIter)
    ){
      BatchTable *pTab;
      BatchIndex *pIdx;
      const char *zTab;           /* Table this change applies to */
      int nCol;                   /* Number of columns in table */
      int op;                     /* UPDATE, INSERT or DELETE */

      sqlite3changeset_op(pIter, &zTab, &nCol, &op, 0);
      assert( op==SQLITE_INSERT || op==SQLITE_UPDATE || op==SQLITE_DELETE );

      rc = cbFindTable(p, zTab, &pTab);
      assert( pTab || rc!=SQLITE_OK );
      if( pTab ){
        for(pIdx=pTab->pIdx; pIdx && rc==SQLITE_OK; pIdx=pIdx->pNext){
          if( op==SQLITE_UPDATE && pIdx->bPk ) continue;
          if( op==SQLITE_UPDATE || op==SQLITE_DELETE ){
            rc = cbAddToHash(p, pIter, pIdx, sqlite3changeset_old, 0, &bConf);
          }
          if( op==SQLITE_UPDATE || op==SQLITE_INSERT ){
            rc = cbAddToHash(p, pIter, pIdx, 
                sqlite3changeset_new, sqlite3changeset_old, &bConf
            );
          }
        }
      }
      if( rc!=SQLITE_OK ) break;
    }

    rc2 = sqlite3changeset_finalize(pIter);
    if( rc==SQLITE_OK ) rc = rc2;
  }

  if( rc==SQLITE_OK && bConf ){
    rc = SQLITE_CONSTRAINT;
  }
  p->iChangesetId++;
  return rc;
}

/*
** Zero an existing changebatch object.
*/
void sqlite3changebatch_zero(sqlite3_changebatch *p){
  int i;
  for(i=0; i<p->nHash; i++){
    BatchIndexEntry *pEntry;
    BatchIndexEntry *pNext;
    for(pEntry=p->apHash[i]; pEntry; pEntry=pNext){
      pNext = pEntry->pNext;
      cbFree(pEntry);
    }
  }
  cbFree(p->apHash);
  p->nHash = 0;
  p->apHash = 0;
}

/*
** Delete a changebatch object.
*/
void sqlite3changebatch_delete(sqlite3_changebatch *p){
  BatchTable *pTab;
  BatchTable *pTabNext;

  sqlite3changebatch_zero(p);
  for(pTab=p->pTab; pTab; pTab=pTabNext){
    pTabNext = pTab->pNext;
    cbFreeTable(pTab);
  }
  cbFree(p);
}

/*
** Return the db handle.
*/
sqlite3 *sqlite3changebatch_db(sqlite3_changebatch *p){
  return p->db;
}

#endif /* SQLITE_ENABLE_SESSION && SQLITE_ENABLE_PREUPDATE_HOOK */

Added ext/session/sqlite3changebatch.h.





































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82

#if !defined(SQLITECHANGEBATCH_H_) 
#define SQLITECHANGEBATCH_H_ 1

typedef struct sqlite3_changebatch sqlite3_changebatch;

/*
** Create a new changebatch object for detecting conflicts between
** changesets associated with a schema equivalent to that of the "main"
** database of the open database handle db passed as the first
** parameter. It is the responsibility of the caller to ensure that
** the database handle is not closed until after the changebatch
** object has been deleted.
**
** A changebatch object is used to detect batches of non-conflicting
** changesets. Changesets that do not conflict may be applied to the 
** target database in any order without affecting the final state of 
** the database.
**
** The changebatch object only works reliably if PRIMARY KEY and UNIQUE
** constraints on tables affected by the changesets use collation
** sequences that are equivalent to built-in collation sequence 
** BINARY for the == operation.
**
** If successful, SQLITE_OK is returned and (*pp) set to point to
** the new changebatch object. If an error occurs, an SQLite error
** code is returned and the final value of (*pp) is undefined.
*/
int sqlite3changebatch_new(sqlite3 *db, sqlite3_changebatch **pp);

/*
** Argument p points to a buffer containing a changeset n bytes in
** size. Assuming no error occurs, this function returns SQLITE_OK
** if the changeset does not conflict with any changeset passed 
** to an sqlite3changebatch_add() call made on the same 
** sqlite3_changebatch* handle since the most recent call to
** sqlite3changebatch_zero(). If the changeset does conflict with 
** an earlier such changeset, SQLITE_CONSTRAINT is returned. Or, 
** if an error occurs, some other SQLite error code may be returned.
**
** One changeset is said to conflict with another if
** either:
**
**   * the two changesets contain operations (INSERT, UPDATE or 
**     DELETE) on the same row, identified by primary key, or
**
**   * the two changesets contain operations (INSERT, UPDATE or 
**     DELETE) on rows with identical values in any combination 
**     of fields constrained by a UNIQUE constraint.
**
** Even if this function returns SQLITE_CONFLICT, the current
** changeset is added to the internal data structures - so future
** calls to this function may conflict with it. If this function
** returns any result code other than SQLITE_OK or SQLITE_CONFLICT,
** the result of any future call to sqlite3changebatch_add() is
** undefined.
**
** Only changesets may be passed to this function. Passing a 
** patchset to this function results in an SQLITE_MISUSE error.
*/
int sqlite3changebatch_add(sqlite3_changebatch*, void *p, int n);

/*
** Zero a changebatch object. This causes the records of all earlier 
** calls to sqlite3changebatch_add() to be discarded.
*/
void sqlite3changebatch_zero(sqlite3_changebatch*);

/*
** Return a copy of the first argument passed to the sqlite3changebatch_new()
** call used to create the changebatch object passed as the only argument
** to this function.
*/
sqlite3 *sqlite3changebatch_db(sqlite3_changebatch*);

/*
** Delete a changebatch object.
*/
void sqlite3changebatch_delete(sqlite3_changebatch*);

#endif /* !defined(SQLITECHANGEBATCH_H_) */

Changes to ext/session/sqlite3session.c.

21
22
23
24
25
26
27







28
29
30
31
32
33
34
....
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059
2060
2061
2062
2063
2064
....
2113
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
....
2149
2150
2151
2152
2153
2154
2155
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
....
2332
2333
2334
2335
2336
2337
2338
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
....
2355
2356
2357
2358
2359
2360
2361
2362
2363
2364
2365
2366
2367
2368
2369
....
2400
2401
2402
2403
2404
2405
2406
2407
2408
2409
2410
2411
2412
2413
2414
....
2424
2425
2426
2427
2428
2429
2430
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
....
2484
2485
2486
2487
2488
2489
2490
2491

2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502

2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513

2514
2515
2516
2517
2518
2519
2520
....
2521
2522
2523
2524
2525
2526
2527
2528

2529










2530
2531
2532
2533
2534
2535
2536
....
4925
4926
4927
4928
4929
4930
4931

4932
4933
4934
4935
4936
4937
4938
4939
4940
4941
4942
# ifdef SQLITE_TEST
#   define SESSIONS_STRM_CHUNK_SIZE 64
# else
#   define SESSIONS_STRM_CHUNK_SIZE 1024
# endif
#endif








static int sessions_strm_chunk_size = SESSIONS_STRM_CHUNK_SIZE;

typedef struct SessionHook SessionHook;
struct SessionHook {
  void *pCtx;
  int (*xOld)(void*,int,sqlite3_value**);
  int (*xNew)(void*,int,sqlite3_value**);
................................................................................
**
** Otherwise, the old.* record contains all primary key values and the 
** original values of any fields that have been modified. The new.* record 
** contains the new values of only those fields that have been modified.
*/ 
static int sessionAppendUpdate(
  SessionBuffer *pBuf,            /* Buffer to append to */
  int bPatchset,                  /* True for "patchset", 0 for "changeset" */
  sqlite3_stmt *pStmt,            /* Statement handle pointing at new row */
  SessionChange *p,               /* Object containing old values */
  u8 *abPK                        /* Boolean array - true for PK columns */
){
  int rc = SQLITE_OK;
  SessionBuffer buf2 = {0,0,0}; /* Buffer to accumulate new.* record in */
  int bNoop = 1;                /* Set to zero if any values are modified */
................................................................................
    }

    /* If at least one field has been modified, this is not a no-op. */
    if( bChanged ) bNoop = 0;

    /* Add a field to the old.* record. This is omitted if this modules is
    ** currently generating a patchset. */
    if( bPatchset==0 ){
      if( bChanged || abPK[i] ){
        sessionAppendBlob(pBuf, pCsr, nAdvance, &rc);
      }else{
        sessionAppendByte(pBuf, 0, &rc);
      }
    }

    /* Add a field to the new.* record. Or the only record if currently
    ** generating a patchset.  */
    if( bChanged || (bPatchset && abPK[i]) ){
      sessionAppendCol(&buf2, pStmt, i, &rc);
    }else{
      sessionAppendByte(&buf2, 0, &rc);
    }

    pCsr += nAdvance;
  }
................................................................................
/*
** Append a DELETE change to the buffer passed as the first argument. Use
** the changeset format if argument bPatchset is zero, or the patchset
** format otherwise.
*/
static int sessionAppendDelete(
  SessionBuffer *pBuf,            /* Buffer to append to */
  int bPatchset,                  /* True for "patchset", 0 for "changeset" */
  SessionChange *p,               /* Object containing old values */
  int nCol,                       /* Number of columns in table */
  u8 *abPK                        /* Boolean array - true for PK columns */
){
  int rc = SQLITE_OK;

  sessionAppendByte(pBuf, SQLITE_DELETE, &rc);
  sessionAppendByte(pBuf, p->bIndirect, &rc);

  if( bPatchset==0 ){
    sessionAppendBlob(pBuf, p->aRecord, p->nRecord, &rc);
  }else{
    int i;
    u8 *a = p->aRecord;
    for(i=0; i<nCol; i++){
      u8 *pStart = a;
      int eType = *a++;
................................................................................
** This function is a no-op if *pRc is set to other than SQLITE_OK when it
** is called. Otherwise, append a serialized table header (part of the binary 
** changeset format) to buffer *pBuf. If an error occurs, set *pRc to an
** SQLite error code before returning.
*/
static void sessionAppendTableHdr(
  SessionBuffer *pBuf,            /* Append header to this buffer */
  int bPatchset,                  /* Use the patchset format if true */
  SessionTable *pTab,             /* Table object to append header for */
  int *pRc                        /* IN/OUT: Error code */
){
  /* Write a table header */
  sessionAppendByte(pBuf, (bPatchset ? 'P' : 'T'), pRc);
  sessionAppendVarint(pBuf, pTab->nCol, pRc);
  sessionAppendBlob(pBuf, pTab->abPK, pTab->nCol, pRc);
  sessionAppendBlob(pBuf, (u8 *)pTab->zName, (int)strlen(pTab->zName)+1, pRc);
}

/*
** Generate either a changeset (if argument bPatchset is zero) or a patchset
................................................................................
** If no error occurs, SQLITE_OK is returned and the new changeset/patchset
** stored in output variables *pnChangeset and *ppChangeset. Or, if an error
** occurs, an SQLite error code is returned and both output variables set 
** to 0.
*/
static int sessionGenerateChangeset(
  sqlite3_session *pSession,      /* Session object */
  int bPatchset,                  /* True for patchset, false for changeset */
  int (*xOutput)(void *pOut, const void *pData, int nData),
  void *pOut,                     /* First argument for xOutput */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
){
  sqlite3 *db = pSession->db;     /* Source database handle */
  SessionTable *pTab;             /* Used to iterate through attached tables */
................................................................................
      /* Check the table schema is still Ok. */
      rc = sessionTableInfo(db, pSession->zDb, zName, &nCol, 0, &azCol, &abPK);
      if( !rc && (pTab->nCol!=nCol || memcmp(abPK, pTab->abPK, nCol)) ){
        rc = SQLITE_SCHEMA;
      }

      /* Write a table header */
      sessionAppendTableHdr(&buf, bPatchset, pTab, &rc);

      /* Build and compile a statement to execute: */
      if( rc==SQLITE_OK ){
        rc = sessionSelectStmt(
            db, pSession->zDb, zName, nCol, azCol, abPK, &pSel);
      }

................................................................................
              int iCol;
              sessionAppendByte(&buf, SQLITE_INSERT, &rc);
              sessionAppendByte(&buf, p->bIndirect, &rc);
              for(iCol=0; iCol<nCol; iCol++){
                sessionAppendCol(&buf, pSel, iCol, &rc);
              }
            }else{
              rc = sessionAppendUpdate(&buf, bPatchset, pSel, p, abPK);
            }
          }else if( p->op!=SQLITE_INSERT ){
            rc = sessionAppendDelete(&buf, bPatchset, p, nCol, abPK);
          }
          if( rc==SQLITE_OK ){
            rc = sqlite3_reset(pSel);
          }

          /* If the buffer is now larger than sessions_strm_chunk_size, pass
          ** its contents to the xOutput() callback. */
................................................................................
** using sqlite3_free().
*/
int sqlite3session_changeset(
  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
){
  return sessionGenerateChangeset(pSession, 0, 0, 0, pnChangeset, ppChangeset);

}

/*
** Streaming version of sqlite3session_changeset().
*/
int sqlite3session_changeset_strm(
  sqlite3_session *pSession,
  int (*xOutput)(void *pOut, const void *pData, int nData),
  void *pOut
){
  return sessionGenerateChangeset(pSession, 0, xOutput, pOut, 0, 0);

}

/*
** Streaming version of sqlite3session_patchset().
*/
int sqlite3session_patchset_strm(
  sqlite3_session *pSession,
  int (*xOutput)(void *pOut, const void *pData, int nData),
  void *pOut
){
  return sessionGenerateChangeset(pSession, 1, xOutput, pOut, 0, 0);

}

/*
** Obtain a patchset object containing all changes recorded by the 
** session object passed as the first argument.
**
** It is the responsibility of the caller to eventually free the buffer 
................................................................................
** using sqlite3_free().
*/
int sqlite3session_patchset(
  sqlite3_session *pSession,      /* Session object */
  int *pnPatchset,                /* OUT: Size of buffer at *ppChangeset */
  void **ppPatchset               /* OUT: Buffer containing changeset */
){
  return sessionGenerateChangeset(pSession, 1, 0, 0, pnPatchset, ppPatchset);

}











/*
** Enable or disable the session object passed as the first argument.
*/
int sqlite3session_enable(sqlite3_session *pSession, int bEnable){
  int ret;
  sqlite3_mutex_enter(sqlite3_db_mutex(pSession->db));
................................................................................
  SessionTable *pTab;
  assert( xOutput==0 || (ppOut==0 && pnOut==0) );

  /* Create the serialized output changeset based on the contents of the
  ** hash tables attached to the SessionTable objects in list p->pList. 
  */
  for(pTab=pGrp->pList; rc==SQLITE_OK && pTab; pTab=pTab->pNext){

    int i;
    if( pTab->nEntry==0 ) continue;

    sessionAppendTableHdr(&buf, pGrp->bPatch, pTab, &rc);
    for(i=0; i<pTab->nChange; i++){
      SessionChange *p;
      for(p=pTab->apChange[i]; p; p=p->pNext){
        sessionAppendByte(&buf, p->op, &rc);
        sessionAppendByte(&buf, p->bIndirect, &rc);
        sessionAppendBlob(&buf, p->aRecord, p->nRecord, &rc);
        if( rc==SQLITE_OK && xOutput && buf.nBuf>=sessions_strm_chunk_size ){







>
>
>
>
>
>
>







 







|







 







|
|








|







 







|









|







 







|




|







 







|







 







|







 







|


|







 







|
>










|
>










|
>







 







|
>

>
>
>
>
>
>
>
>
>
>







 







>



|







21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
....
2057
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
....
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144
....
2156
2157
2158
2159
2160
2161
2162
2163
2164
2165
2166
2167
2168
2169
2170
2171
2172
2173
2174
2175
2176
2177
2178
2179
2180
....
2339
2340
2341
2342
2343
2344
2345
2346
2347
2348
2349
2350
2351
2352
2353
2354
2355
2356
2357
2358
....
2362
2363
2364
2365
2366
2367
2368
2369
2370
2371
2372
2373
2374
2375
2376
....
2407
2408
2409
2410
2411
2412
2413
2414
2415
2416
2417
2418
2419
2420
2421
....
2431
2432
2433
2434
2435
2436
2437
2438
2439
2440
2441
2442
2443
2444
2445
2446
2447
2448
....
2491
2492
2493
2494
2495
2496
2497
2498
2499
2500
2501
2502
2503
2504
2505
2506
2507
2508
2509
2510
2511
2512
2513
2514
2515
2516
2517
2518
2519
2520
2521
2522
2523
2524
2525
2526
2527
2528
2529
2530
....
2531
2532
2533
2534
2535
2536
2537
2538
2539
2540
2541
2542
2543
2544
2545
2546
2547
2548
2549
2550
2551
2552
2553
2554
2555
2556
2557
....
4946
4947
4948
4949
4950
4951
4952
4953
4954
4955
4956
4957
4958
4959
4960
4961
4962
4963
4964
# ifdef SQLITE_TEST
#   define SESSIONS_STRM_CHUNK_SIZE 64
# else
#   define SESSIONS_STRM_CHUNK_SIZE 1024
# endif
#endif

/*
** The three different types of changesets generated.
*/
#define SESSIONS_PATCHSET      0
#define SESSIONS_CHANGESET     1
#define SESSIONS_FULLCHANGESET 2

static int sessions_strm_chunk_size = SESSIONS_STRM_CHUNK_SIZE;

typedef struct SessionHook SessionHook;
struct SessionHook {
  void *pCtx;
  int (*xOld)(void*,int,sqlite3_value**);
  int (*xNew)(void*,int,sqlite3_value**);
................................................................................
**
** Otherwise, the old.* record contains all primary key values and the 
** original values of any fields that have been modified. The new.* record 
** contains the new values of only those fields that have been modified.
*/ 
static int sessionAppendUpdate(
  SessionBuffer *pBuf,            /* Buffer to append to */
  int ePatchset,                  /* True for "patchset", 0 for "changeset" */
  sqlite3_stmt *pStmt,            /* Statement handle pointing at new row */
  SessionChange *p,               /* Object containing old values */
  u8 *abPK                        /* Boolean array - true for PK columns */
){
  int rc = SQLITE_OK;
  SessionBuffer buf2 = {0,0,0}; /* Buffer to accumulate new.* record in */
  int bNoop = 1;                /* Set to zero if any values are modified */
................................................................................
    }

    /* If at least one field has been modified, this is not a no-op. */
    if( bChanged ) bNoop = 0;

    /* Add a field to the old.* record. This is omitted if this modules is
    ** currently generating a patchset. */
    if( ePatchset!=SESSIONS_PATCHSET ){
      if( ePatchset==SESSIONS_FULLCHANGESET || bChanged || abPK[i] ){
        sessionAppendBlob(pBuf, pCsr, nAdvance, &rc);
      }else{
        sessionAppendByte(pBuf, 0, &rc);
      }
    }

    /* Add a field to the new.* record. Or the only record if currently
    ** generating a patchset.  */
    if( bChanged || (ePatchset==SESSIONS_PATCHSET && abPK[i]) ){
      sessionAppendCol(&buf2, pStmt, i, &rc);
    }else{
      sessionAppendByte(&buf2, 0, &rc);
    }

    pCsr += nAdvance;
  }
................................................................................
/*
** Append a DELETE change to the buffer passed as the first argument. Use
** the changeset format if argument bPatchset is zero, or the patchset
** format otherwise.
*/
static int sessionAppendDelete(
  SessionBuffer *pBuf,            /* Buffer to append to */
  int eChangeset,                 /* One of SESSIONS_CHANGESET etc. */
  SessionChange *p,               /* Object containing old values */
  int nCol,                       /* Number of columns in table */
  u8 *abPK                        /* Boolean array - true for PK columns */
){
  int rc = SQLITE_OK;

  sessionAppendByte(pBuf, SQLITE_DELETE, &rc);
  sessionAppendByte(pBuf, p->bIndirect, &rc);

  if( eChangeset!=SESSIONS_PATCHSET ){
    sessionAppendBlob(pBuf, p->aRecord, p->nRecord, &rc);
  }else{
    int i;
    u8 *a = p->aRecord;
    for(i=0; i<nCol; i++){
      u8 *pStart = a;
      int eType = *a++;
................................................................................
** This function is a no-op if *pRc is set to other than SQLITE_OK when it
** is called. Otherwise, append a serialized table header (part of the binary 
** changeset format) to buffer *pBuf. If an error occurs, set *pRc to an
** SQLite error code before returning.
*/
static void sessionAppendTableHdr(
  SessionBuffer *pBuf,            /* Append header to this buffer */
  int ePatchset,                  /* Use the patchset format if true */
  SessionTable *pTab,             /* Table object to append header for */
  int *pRc                        /* IN/OUT: Error code */
){
  /* Write a table header */
  sessionAppendByte(pBuf, (ePatchset==SESSIONS_PATCHSET) ? 'P' : 'T', pRc);
  sessionAppendVarint(pBuf, pTab->nCol, pRc);
  sessionAppendBlob(pBuf, pTab->abPK, pTab->nCol, pRc);
  sessionAppendBlob(pBuf, (u8 *)pTab->zName, (int)strlen(pTab->zName)+1, pRc);
}

/*
** Generate either a changeset (if argument bPatchset is zero) or a patchset
................................................................................
** If no error occurs, SQLITE_OK is returned and the new changeset/patchset
** stored in output variables *pnChangeset and *ppChangeset. Or, if an error
** occurs, an SQLite error code is returned and both output variables set 
** to 0.
*/
static int sessionGenerateChangeset(
  sqlite3_session *pSession,      /* Session object */
  int ePatchset,                  /* One of SESSIONS_CHANGESET etc. */
  int (*xOutput)(void *pOut, const void *pData, int nData),
  void *pOut,                     /* First argument for xOutput */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
){
  sqlite3 *db = pSession->db;     /* Source database handle */
  SessionTable *pTab;             /* Used to iterate through attached tables */
................................................................................
      /* Check the table schema is still Ok. */
      rc = sessionTableInfo(db, pSession->zDb, zName, &nCol, 0, &azCol, &abPK);
      if( !rc && (pTab->nCol!=nCol || memcmp(abPK, pTab->abPK, nCol)) ){
        rc = SQLITE_SCHEMA;
      }

      /* Write a table header */
      sessionAppendTableHdr(&buf, ePatchset, pTab, &rc);

      /* Build and compile a statement to execute: */
      if( rc==SQLITE_OK ){
        rc = sessionSelectStmt(
            db, pSession->zDb, zName, nCol, azCol, abPK, &pSel);
      }

................................................................................
              int iCol;
              sessionAppendByte(&buf, SQLITE_INSERT, &rc);
              sessionAppendByte(&buf, p->bIndirect, &rc);
              for(iCol=0; iCol<nCol; iCol++){
                sessionAppendCol(&buf, pSel, iCol, &rc);
              }
            }else{
              rc = sessionAppendUpdate(&buf, ePatchset, pSel, p, abPK);
            }
          }else if( p->op!=SQLITE_INSERT ){
            rc = sessionAppendDelete(&buf, ePatchset, p, nCol, abPK);
          }
          if( rc==SQLITE_OK ){
            rc = sqlite3_reset(pSel);
          }

          /* If the buffer is now larger than sessions_strm_chunk_size, pass
          ** its contents to the xOutput() callback. */
................................................................................
** using sqlite3_free().
*/
int sqlite3session_changeset(
  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
){
  return sessionGenerateChangeset(
      pSession, SESSIONS_CHANGESET, 0, 0, pnChangeset, ppChangeset);
}

/*
** Streaming version of sqlite3session_changeset().
*/
int sqlite3session_changeset_strm(
  sqlite3_session *pSession,
  int (*xOutput)(void *pOut, const void *pData, int nData),
  void *pOut
){
  return sessionGenerateChangeset(
      pSession, SESSIONS_CHANGESET, xOutput, pOut, 0, 0);
}

/*
** Streaming version of sqlite3session_patchset().
*/
int sqlite3session_patchset_strm(
  sqlite3_session *pSession,
  int (*xOutput)(void *pOut, const void *pData, int nData),
  void *pOut
){
  return sessionGenerateChangeset(
      pSession, SESSIONS_PATCHSET, xOutput, pOut, 0, 0);
}

/*
** Obtain a patchset object containing all changes recorded by the 
** session object passed as the first argument.
**
** It is the responsibility of the caller to eventually free the buffer 
................................................................................
** using sqlite3_free().
*/
int sqlite3session_patchset(
  sqlite3_session *pSession,      /* Session object */
  int *pnPatchset,                /* OUT: Size of buffer at *ppChangeset */
  void **ppPatchset               /* OUT: Buffer containing changeset */
){
  return sessionGenerateChangeset(
      pSession, SESSIONS_PATCHSET, 0, 0, pnPatchset, ppPatchset);
}

int sqlite3session_fullchangeset(
  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
){
  return sessionGenerateChangeset(
      pSession, SESSIONS_FULLCHANGESET, 0, 0, pnChangeset, ppChangeset);
}


/*
** Enable or disable the session object passed as the first argument.
*/
int sqlite3session_enable(sqlite3_session *pSession, int bEnable){
  int ret;
  sqlite3_mutex_enter(sqlite3_db_mutex(pSession->db));
................................................................................
  SessionTable *pTab;
  assert( xOutput==0 || (ppOut==0 && pnOut==0) );

  /* Create the serialized output changeset based on the contents of the
  ** hash tables attached to the SessionTable objects in list p->pList. 
  */
  for(pTab=pGrp->pList; rc==SQLITE_OK && pTab; pTab=pTab->pNext){
    int eChangeset = pGrp->bPatch ? SESSIONS_PATCHSET : SESSIONS_CHANGESET;
    int i;
    if( pTab->nEntry==0 ) continue;

    sessionAppendTableHdr(&buf, eChangeset, pTab, &rc);
    for(i=0; i<pTab->nChange; i++){
      SessionChange *p;
      for(p=pTab->apChange[i]; p; p=p->pNext){
        sessionAppendByte(&buf, p->op, &rc);
        sessionAppendByte(&buf, p->bIndirect, &rc);
        sessionAppendBlob(&buf, p->aRecord, p->nRecord, &rc);
        if( rc==SQLITE_OK && xOutput && buf.nBuf>=sessions_strm_chunk_size ){

Changes to ext/session/sqlite3session.h.

314
315
316
317
318
319
320













321
322
323
324
325
326
327
** the same session object is disabled, no INSERT record will appear in the
** changeset, even though the delete took place while the session was disabled.
** Or, if one field of a row is updated while a session is disabled, and 
** another field of the same row is updated while the session is enabled, the
** resulting changeset will contain an UPDATE change that updates both fields.
*/
int sqlite3session_changeset(













  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
);

/*
** CAPI3REF: Load The Difference Between Tables Into A Session







>
>
>
>
>
>
>
>
>
>
>
>
>







314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
** the same session object is disabled, no INSERT record will appear in the
** changeset, even though the delete took place while the session was disabled.
** Or, if one field of a row is updated while a session is disabled, and 
** another field of the same row is updated while the session is enabled, the
** resulting changeset will contain an UPDATE change that updates both fields.
*/
int sqlite3session_changeset(
  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
);

/*
** CAPI3REF: Generate A Full Changeset From A Session Object
**
** This function is similar to sqlite3session_changeset(), except that for
** each row affected by an UPDATE statement, all old.* values are recorded
** as part of the changeset, not just those modified.
*/
int sqlite3session_fullchangeset(
  sqlite3_session *pSession,      /* Session object */
  int *pnChangeset,               /* OUT: Size of buffer at *ppChangeset */
  void **ppChangeset              /* OUT: Buffer containing changeset */
);

/*
** CAPI3REF: Load The Difference Between Tables Into A Session

Changes to ext/session/test_session.c.

214
215
216
217
218
219
220

221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244

245
246
247
248
249
250
251
...
267
268
269
270
271
272
273

274
275
276
277
278
279
280
281
282
283
284
285
286


287
288
289
290
291
292
293
294
295
296
297
298
299

300
301
302
303
304
305
306
...
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
....
1115
1116
1117
1118
1119
1120
1121

























































































































1122
1123
1124
1125
1126
1127
1128
....
1298
1299
1300
1301
1302
1303
1304




1305
1306
1307
1308
** Tclcmd:  $session attach TABLE
**          $session changeset
**          $session delete
**          $session enable BOOL
**          $session indirect INTEGER
**          $session patchset
**          $session table_filter SCRIPT

*/
static int SQLITE_TCLAPI test_session_cmd(
  void *clientData,
  Tcl_Interp *interp,
  int objc,
  Tcl_Obj *CONST objv[]
){
  TestSession *p = (TestSession*)clientData;
  sqlite3_session *pSession = p->pSession;
  struct SessionSubcmd {
    const char *zSub;
    int nArg;
    const char *zMsg;
    int iSub;
  } aSub[] = {
    { "attach",       1, "TABLE",      }, /* 0 */
    { "changeset",    0, "",           }, /* 1 */
    { "delete",       0, "",           }, /* 2 */
    { "enable",       1, "BOOL",       }, /* 3 */
    { "indirect",     1, "BOOL",       }, /* 4 */
    { "isempty",      0, "",           }, /* 5 */
    { "table_filter", 1, "SCRIPT",     }, /* 6 */
    { "patchset",     0, "",           }, /* 7 */
    { "diff",         2, "FROMDB TBL", }, /* 8 */

    { 0 }
  };
  int iSub;
  int rc;

  if( objc<2 ){
    Tcl_WrongNumArgs(interp, 1, objv, "SUBCOMMAND ...");
................................................................................
      rc = sqlite3session_attach(pSession, zArg);
      if( rc!=SQLITE_OK ){
        return test_session_error(interp, rc, 0);
      }
      break;
    }


    case 7:        /* patchset */
    case 1: {      /* changeset */
      TestSessionsBlob o = {0, 0};
      if( test_tcl_integer(interp, SESSION_STREAM_TCL_VAR) ){
        void *pCtx = (void*)&o;
        if( iSub==7 ){
          rc = sqlite3session_patchset_strm(pSession, testStreamOutput, pCtx);
        }else{
          rc = sqlite3session_changeset_strm(pSession, testStreamOutput, pCtx);
        }
      }else{
        if( iSub==7 ){
          rc = sqlite3session_patchset(pSession, &o.n, &o.p);


        }else{
          rc = sqlite3session_changeset(pSession, &o.n, &o.p);
        }
      }
      if( rc==SQLITE_OK ){
        Tcl_SetObjResult(interp, Tcl_NewByteArrayObj(o.p, o.n)); 
      }
      sqlite3_free(o.p);
      if( rc!=SQLITE_OK ){
        return test_session_error(interp, rc, 0);
      }
      break;
    }


    case 2:        /* delete */
      Tcl_DeleteCommand(interp, Tcl_GetString(objv[0]));
      break;

    case 3: {      /* enable */
      int val;
................................................................................
   || TCL_OK!=Tcl_GetIntFromObj(interp, Tcl_GetObjResult(interp), &res)
  ){
    Tcl_BackgroundError(interp);
  }

  Tcl_DecrRefCount(pEval);
  return res;
}  

static int test_conflict_handler(
  void *pCtx,                     /* Pointer to TestConflictHandler structure */
  int eConf,                      /* DATA, MISSING, CONFLICT, CONSTRAINT */
  sqlite3_changeset_iter *pIter   /* Handle describing change and conflict */
){
  TestConflictHandler *p = (TestConflictHandler *)pCtx;
................................................................................
  }
  if( rc!=SQLITE_OK ){
    return test_session_error(interp, rc, 0);
  }

  return TCL_OK;
}


























































































































/*
** tclcmd: CMD configure REBASE-BLOB
** tclcmd: CMD rebase CHANGESET
** tclcmd: CMD delete
*/
static int SQLITE_TCLAPI test_rebaser_cmd(
................................................................................
  int i;

  for(i=0; i<sizeof(aCmd)/sizeof(struct Cmd); i++){
    struct Cmd *p = &aCmd[i];
    Tcl_CreateObjCommand(interp, p->zCmd, p->xProc, 0, 0);
  }





  return TCL_OK;
}

#endif /* SQLITE_TEST && SQLITE_SESSION && SQLITE_PREUPDATE_HOOK */







>













<

|
|
|
|
|
|
|

|
>







 







>



|









>
>













>







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>




214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234

235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
...
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
...
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
....
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159
1160
1161
1162
1163
1164
1165
1166
1167
1168
1169
1170
1171
1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
....
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
** Tclcmd:  $session attach TABLE
**          $session changeset
**          $session delete
**          $session enable BOOL
**          $session indirect INTEGER
**          $session patchset
**          $session table_filter SCRIPT
**          $session fullchangeset
*/
static int SQLITE_TCLAPI test_session_cmd(
  void *clientData,
  Tcl_Interp *interp,
  int objc,
  Tcl_Obj *CONST objv[]
){
  TestSession *p = (TestSession*)clientData;
  sqlite3_session *pSession = p->pSession;
  struct SessionSubcmd {
    const char *zSub;
    int nArg;
    const char *zMsg;

  } aSub[] = {
    { "attach",       1, "TABLE"       }, /* 0 */
    { "changeset",    0, ""            }, /* 1 */
    { "delete",       0, ""            }, /* 2 */
    { "enable",       1, "BOOL"        }, /* 3 */
    { "indirect",     1, "BOOL"        }, /* 4 */
    { "isempty",      0, ""            }, /* 5 */
    { "table_filter", 1, "SCRIPT"      }, /* 6 */
    { "patchset",     0, "",           }, /* 7 */
    { "diff",         2, "FROMDB TBL"  }, /* 8 */
    { "fullchangeset",0, ""            }, /* 9 */
    { 0 }
  };
  int iSub;
  int rc;

  if( objc<2 ){
    Tcl_WrongNumArgs(interp, 1, objv, "SUBCOMMAND ...");
................................................................................
      rc = sqlite3session_attach(pSession, zArg);
      if( rc!=SQLITE_OK ){
        return test_session_error(interp, rc, 0);
      }
      break;
    }

    case 9:        /* fullchangeset */
    case 7:        /* patchset */
    case 1: {      /* changeset */
      TestSessionsBlob o = {0, 0};
      if( iSub!=9 && test_tcl_integer(interp, SESSION_STREAM_TCL_VAR) ){
        void *pCtx = (void*)&o;
        if( iSub==7 ){
          rc = sqlite3session_patchset_strm(pSession, testStreamOutput, pCtx);
        }else{
          rc = sqlite3session_changeset_strm(pSession, testStreamOutput, pCtx);
        }
      }else{
        if( iSub==7 ){
          rc = sqlite3session_patchset(pSession, &o.n, &o.p);
        }else if( iSub==9 ){
          rc = sqlite3session_fullchangeset(pSession, &o.n, &o.p);
        }else{
          rc = sqlite3session_changeset(pSession, &o.n, &o.p);
        }
      }
      if( rc==SQLITE_OK ){
        Tcl_SetObjResult(interp, Tcl_NewByteArrayObj(o.p, o.n)); 
      }
      sqlite3_free(o.p);
      if( rc!=SQLITE_OK ){
        return test_session_error(interp, rc, 0);
      }
      break;
    }


    case 2:        /* delete */
      Tcl_DeleteCommand(interp, Tcl_GetString(objv[0]));
      break;

    case 3: {      /* enable */
      int val;
................................................................................
   || TCL_OK!=Tcl_GetIntFromObj(interp, Tcl_GetObjResult(interp), &res)
  ){
    Tcl_BackgroundError(interp);
  }

  Tcl_DecrRefCount(pEval);
  return res;
}

static int test_conflict_handler(
  void *pCtx,                     /* Pointer to TestConflictHandler structure */
  int eConf,                      /* DATA, MISSING, CONFLICT, CONSTRAINT */
  sqlite3_changeset_iter *pIter   /* Handle describing change and conflict */
){
  TestConflictHandler *p = (TestConflictHandler *)pCtx;
................................................................................
  }
  if( rc!=SQLITE_OK ){
    return test_session_error(interp, rc, 0);
  }

  return TCL_OK;
}

#include "sqlite3changebatch.h"

typedef struct TestChangebatch TestChangebatch;
struct TestChangebatch {
  sqlite3_changebatch *pChangebatch;
};

/*
** Tclcmd:  $changebatch add BLOB
**          $changebatch zero
**          $changebatch delete
*/
static int SQLITE_TCLAPI test_changebatch_cmd(
  void *clientData,
  Tcl_Interp *interp,
  int objc,
  Tcl_Obj *CONST objv[]
){
  TestChangebatch *p = (TestChangebatch*)clientData;
  sqlite3_changebatch *pChangebatch = p->pChangebatch;
  struct SessionSubcmd {
    const char *zSub;
    int nArg;
    const char *zMsg;
    int iSub;
  } aSub[] = {
    { "add",          1, "CHANGESET",  }, /* 0 */
    { "zero",         0, "",           }, /* 1 */
    { "delete",       0, "",           }, /* 2 */
    { 0 }
  };
  int iSub;
  int rc;

  if( objc<2 ){
    Tcl_WrongNumArgs(interp, 1, objv, "SUBCOMMAND ...");
    return TCL_ERROR;
  }
  rc = Tcl_GetIndexFromObjStruct(interp, 
      objv[1], aSub, sizeof(aSub[0]), "sub-command", 0, &iSub
  );
  if( rc!=TCL_OK ) return rc;
  if( objc!=2+aSub[iSub].nArg ){
    Tcl_WrongNumArgs(interp, 2, objv, aSub[iSub].zMsg);
    return TCL_ERROR;
  }

  switch( iSub ){
    case 0: {      /* add */
      int nArg;
      unsigned char *pArg = Tcl_GetByteArrayFromObj(objv[2], &nArg);
      rc = sqlite3changebatch_add(pChangebatch, pArg, nArg);
      if( rc!=SQLITE_OK && rc!=SQLITE_CONSTRAINT ){
        return test_session_error(interp, rc, 0);
      }else{
        extern const char *sqlite3ErrName(int);
        Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
      }
      break;
    }

    case 1: {      /* zero */
      sqlite3changebatch_zero(pChangebatch);
      break;
    }

    case 2:        /* delete */
      Tcl_DeleteCommand(interp, Tcl_GetString(objv[0]));
      break;
  }

  return TCL_OK;
}

static void SQLITE_TCLAPI test_changebatch_del(void *clientData){
  TestChangebatch *p = (TestChangebatch*)clientData;
  sqlite3changebatch_delete(p->pChangebatch);
  ckfree((char*)p);
}

/*
** Tclcmd:  sqlite3changebatch CMD DB-HANDLE
*/
static int SQLITE_TCLAPI test_sqlite3changebatch(
  void * clientData,
  Tcl_Interp *interp,
  int objc,
  Tcl_Obj *CONST objv[]
){
  sqlite3 *db;
  Tcl_CmdInfo info;
  int rc;                         /* sqlite3session_create() return code */
  TestChangebatch *p;             /* New wrapper object */

  if( objc!=3 ){
    Tcl_WrongNumArgs(interp, 1, objv, "CMD DB-HANDLE");
    return TCL_ERROR;
  }

  if( 0==Tcl_GetCommandInfo(interp, Tcl_GetString(objv[2]), &info) ){
    Tcl_AppendResult(interp, "no such handle: ", Tcl_GetString(objv[2]), 0);
    return TCL_ERROR;
  }
  db = *(sqlite3 **)info.objClientData;

  p = (TestChangebatch*)ckalloc(sizeof(TestChangebatch));
  memset(p, 0, sizeof(TestChangebatch));
  rc = sqlite3changebatch_new(db, &p->pChangebatch);
  if( rc!=SQLITE_OK ){
    ckfree((char*)p);
    return test_session_error(interp, rc, 0);
  }

  Tcl_CreateObjCommand(
      interp, Tcl_GetString(objv[1]), test_changebatch_cmd, (ClientData)p,
      test_changebatch_del
  );
  Tcl_SetObjResult(interp, objv[1]);
  return TCL_OK;
}

/*
** tclcmd: CMD configure REBASE-BLOB
** tclcmd: CMD rebase CHANGESET
** tclcmd: CMD delete
*/
static int SQLITE_TCLAPI test_rebaser_cmd(
................................................................................
  int i;

  for(i=0; i<sizeof(aCmd)/sizeof(struct Cmd); i++){
    struct Cmd *p = &aCmd[i];
    Tcl_CreateObjCommand(interp, p->zCmd, p->xProc, 0, 0);
  }


  Tcl_CreateObjCommand(
      interp, "sqlite3changebatch", test_sqlite3changebatch, 0, 0
  );
  return TCL_OK;
}

#endif /* SQLITE_TEST && SQLITE_SESSION && SQLITE_PREUPDATE_HOOK */

Changes to main.mk.

429
430
431
432
433
434
435

436
437
438
439
440
441
442
443
...
976
977
978
979
980
981
982



983
984
985
986
987
988
989
  $(TOP)/ext/fts3/fts3_aux.c \
  $(TOP)/ext/fts3/fts3_expr.c \
  $(TOP)/ext/fts3/fts3_tokenizer.c \
  $(TOP)/ext/fts3/fts3_write.c \
  $(TOP)/ext/async/sqlite3async.c \
  $(TOP)/ext/misc/stmt.c \
  $(TOP)/ext/session/sqlite3session.c \

  $(TOP)/ext/session/test_session.c

# Header files used by all library source files.
#
HDR = \
   $(TOP)/src/btree.h \
   $(TOP)/src/btreeInt.h \
   $(TOP)/src/hash.h \
................................................................................
                  $(TOP)/test/tt3_index.c      \
                  $(TOP)/test/tt3_vacuum.c      \
                  $(TOP)/test/tt3_stress.c      \
                  $(TOP)/test/tt3_lookaside1.c

threadtest3$(EXE): sqlite3.o $(THREADTEST3_SRC) $(TOP)/src/test_multiplex.c
	$(TCCX) $(TOP)/test/threadtest3.c $(TOP)/src/test_multiplex.c sqlite3.o -o $@ $(THREADLIB)




threadtest: threadtest3$(EXE)
	./threadtest3$(EXE)

TEST_EXTENSION = $(SHPREFIX)testloadext.$(SO)
$(TEST_EXTENSION): $(TOP)/src/test_loadext.c
	$(MKSHLIB) $(TOP)/src/test_loadext.c -o $(TEST_EXTENSION)







>
|







 







>
>
>







429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
...
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
  $(TOP)/ext/fts3/fts3_aux.c \
  $(TOP)/ext/fts3/fts3_expr.c \
  $(TOP)/ext/fts3/fts3_tokenizer.c \
  $(TOP)/ext/fts3/fts3_write.c \
  $(TOP)/ext/async/sqlite3async.c \
  $(TOP)/ext/misc/stmt.c \
  $(TOP)/ext/session/sqlite3session.c \
  $(TOP)/ext/session/sqlite3changebatch.c \
  $(TOP)/ext/session/test_session.c 

# Header files used by all library source files.
#
HDR = \
   $(TOP)/src/btree.h \
   $(TOP)/src/btreeInt.h \
   $(TOP)/src/hash.h \
................................................................................
                  $(TOP)/test/tt3_index.c      \
                  $(TOP)/test/tt3_vacuum.c      \
                  $(TOP)/test/tt3_stress.c      \
                  $(TOP)/test/tt3_lookaside1.c

threadtest3$(EXE): sqlite3.o $(THREADTEST3_SRC) $(TOP)/src/test_multiplex.c
	$(TCCX) $(TOP)/test/threadtest3.c $(TOP)/src/test_multiplex.c sqlite3.o -o $@ $(THREADLIB)

bc_test1$(EXE): sqlite3.o $(TOP)/test/bc_test1.c $(TOP)/test/tt3_core.c
	$(TCCX) $(TOP)/test/bc_test1.c sqlite3.o -o $@ $(THREADLIB)

threadtest: threadtest3$(EXE)
	./threadtest3$(EXE)

TEST_EXTENSION = $(SHPREFIX)testloadext.$(SO)
$(TEST_EXTENSION): $(TOP)/src/test_loadext.c
	$(MKSHLIB) $(TOP)/src/test_loadext.c -o $(TEST_EXTENSION)

Changes to src/bitvec.c.

167
168
169
170
171
172
173






174
175
176
177
178
179
180
** Otherwise the behavior is undefined.
*/
int sqlite3BitvecSet(Bitvec *p, u32 i){
  u32 h;
  if( p==0 ) return SQLITE_OK;
  assert( i>0 );
  assert( i<=p->iSize );






  i--;
  while((p->iSize > BITVEC_NBIT) && p->iDivisor) {
    u32 bin = i/p->iDivisor;
    i = i%p->iDivisor;
    if( p->u.apSub[bin]==0 ){
      p->u.apSub[bin] = sqlite3BitvecCreate( p->iDivisor );
      if( p->u.apSub[bin]==0 ) return SQLITE_NOMEM_BKPT;







>
>
>
>
>
>







167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
** Otherwise the behavior is undefined.
*/
int sqlite3BitvecSet(Bitvec *p, u32 i){
  u32 h;
  if( p==0 ) return SQLITE_OK;
  assert( i>0 );
  assert( i<=p->iSize );
  if( i>p->iSize || i==0 ){
    sqlite3_log(SQLITE_ERROR, 
        "Bitvec: setting bit %d of bitvec size %d\n", (int)i, (int)p->iSize
    );
    abort();
  }
  i--;
  while((p->iSize > BITVEC_NBIT) && p->iDivisor) {
    u32 bin = i/p->iDivisor;
    i = i%p->iDivisor;
    if( p->u.apSub[bin]==0 ){
      p->u.apSub[bin] = sqlite3BitvecCreate( p->iDivisor );
      if( p->u.apSub[bin]==0 ) return SQLITE_NOMEM_BKPT;

Changes to src/btree.c.

463
464
465
466
467
468
469









































































































































































































































470
471
472
473
474
475
476
477
...
987
988
989
990
991
992
993
994
995







996
997
998
999
1000
1001
1002
....
2207
2208
2209
2210
2211
2212
2213











2214
2215
2216
2217
2218
2219
2220
....
3373
3374
3375
3376
3377
3378
3379

3380
3381
3382
3383
3384
3385
3386
....
3448
3449
3450
3451
3452
3453
3454

3455
3456
3457
3458
3459
3460
3461
3462
....
3508
3509
3510
3511
3512
3513
3514









3515
3516
3517
3518
3519
3520
3521
3522
3523

3524



3525
3526
3527
3528
3529
3530
3531
....
3950
3951
3952
3953
3954
3955
3956









































































































































































3957
3958
3959
3960
3961
3962
3963
....
3981
3982
3983
3984
3985
3986
3987

3988
3989

3990
3991
3992
3993
3994
3995
3996
3997
3998
3999




4000

4001
4002
4003
4004
4005
4006
4007
....
4036
4037
4038
4039
4040
4041
4042





4043
4044
4045
4046
4047
4048
4049
....
4265
4266
4267
4268
4269
4270
4271



4272
4273
4274
4275
4276
4277
4278
....
4288
4289
4290
4291
4292
4293
4294

4295
4296
4297
4298
4299
4300
4301
....
5075
5076
5077
5078
5079
5080
5081

5082
5083
5084
5085
5086
5087
5088
....
5089
5090
5091
5092
5093
5094
5095
5096





5097
5098
5099
5100
5101
5102
5103
....
5201
5202
5203
5204
5205
5206
5207

5208
5209
5210
5211
5212
5213
5214
....
5907
5908
5909
5910
5911
5912
5913
5914
5915
5916
5917
5918
5919
5920
5921
5922
5923









5924
5925
5926
5927
5928
5929
5930
5931
5932
5933
5934
5935


5936
5937
5938
5939
5940
5941
5942
5943
5944



5945
5946
5947
5948
5949
5950
5951
5952
5953
5954
5955
5956
5957
5958
5959
5960
5961
5962
....
6256
6257
6258
6259
6260
6261
6262
6263
6264
6265
6266
6267
6268
6269
6270
....
6572
6573
6574
6575
6576
6577
6578
6579
6580
6581
6582
6583
6584
6585
6586
6587
6588
6589
6590
6591
6592
6593
6594
6595
6596
6597
6598
6599
6600
6601
6602
6603
6604
....
6729
6730
6731
6732
6733
6734
6735

6736
6737
6738
6739
6740
6741
6742
....
6743
6744
6745
6746
6747
6748
6749
6750
6751
6752
6753
6754
6755
6756
6757
....
6763
6764
6765
6766
6767
6768
6769
6770
6771
6772
6773
6774
6775
6776
6777
6778
6779
6780
6781
6782
6783
6784
....
7308
7309
7310
7311
7312
7313
7314
7315
7316
7317
7318
7319
7320
7321
7322
....
7446
7447
7448
7449
7450
7451
7452
7453
7454
7455
7456
7457
7458
7459
7460
....
7497
7498
7499
7500
7501
7502
7503
7504

7505
7506
7507
7508
7509
7510
7511
....
7586
7587
7588
7589
7590
7591
7592

7593
7594
7595
7596
7597
7598
7599
....
7930
7931
7932
7933
7934
7935
7936
7937
7938
7939
7940
7941
7942
7943
7944
....
8026
8027
8028
8029
8030
8031
8032
8033
8034
8035
8036
8037
8038
8039
8040
....
8212
8213
8214
8215
8216
8217
8218
8219
8220
8221
8222
8223
8224
8225
8226
....
8295
8296
8297
8298
8299
8300
8301
8302
8303
8304
8305
8306
8307
8308
8309
....
8450
8451
8452
8453
8454
8455
8456
8457
8458
8459
8460
8461
8462
8463
8464
....
8784
8785
8786
8787
8788
8789
8790
8791
8792
8793
8794
8795
8796
8797
8798
....
9231
9232
9233
9234
9235
9236
9237
9238

9239
9240
9241
9242
9243
9244
9245
....
9246
9247
9248
9249
9250
9251
9252

9253
9254
9255
9256
9257
9258
9259
9260
9261
9262
9263
9264
9265
9266
9267
9268

9269

9270
9271
9272
9273
9274
9275
9276
....
9307
9308
9309
9310
9311
9312
9313
9314
9315
9316
9317
9318
9319
9320
9321
.....
10193
10194
10195
10196
10197
10198
10199
10200
10201


10202
10203
10204
10205
10206
10207
10208
10209
.....
10498
10499
10500
10501
10502
10503
10504





























































































10505
10506
10507
10508
10509
10510
10511
      pLock->eLock = READ_LOCK;
    }
  }
}

#endif /* SQLITE_OMIT_SHARED_CACHE */










































































































































































































































static void releasePage(MemPage *pPage);         /* Forward reference */
static void releasePageOne(MemPage *pPage);      /* Forward reference */
static void releasePageNotNull(MemPage *pPage);  /* Forward reference */

/*
***** This routine is used inside of assert() only ****
**
** Verify that the cursor holds the mutex on its BtShared
................................................................................
  Pgno iPtrmap;     /* The pointer map page number */
  int offset;       /* Offset in pointer map page */
  int rc;           /* Return code from subfunctions */

  if( *pRC ) return;

  assert( sqlite3_mutex_held(pBt->mutex) );
  /* The master-journal page number must never be used as a pointer map page */
  assert( 0==PTRMAP_ISPAGE(pBt, PENDING_BYTE_PAGE(pBt)) );








  assert( pBt->autoVacuum );
  if( key==0 ){
    *pRC = SQLITE_CORRUPT_BKPT;
    return;
  }
  iPtrmap = PTRMAP_PAGENO(pBt, key);
................................................................................
    pCur->pPage = pCur->apPage[pCur->iPage];
  }
  testcase( pgno==0 );
  assert( pgno!=0 || rc==SQLITE_CORRUPT );
  return rc;
}












/*
** Release a MemPage.  This should be called once for each prior
** call to btreeGetPage.
**
** Page1 is a special case and must be released using releasePageOne().
*/
static void releasePageNotNull(MemPage *pPage){
................................................................................
** no progress.  By returning SQLITE_BUSY and not invoking the busy callback
** when A already has a read lock, we encourage A to give up and let B
** proceed.
*/
int sqlite3BtreeBeginTrans(Btree *p, int wrflag, int *pSchemaVersion){
  BtShared *pBt = p->pBt;
  int rc = SQLITE_OK;


  sqlite3BtreeEnter(p);
  btreeIntegrity(p);

  /* If the btree is already in a write-transaction, or it
  ** is already in a read-transaction and a read-transaction
  ** is requested, this is a no-op.
................................................................................
    */
    while( pBt->pPage1==0 && SQLITE_OK==(rc = lockBtree(pBt)) );

    if( rc==SQLITE_OK && wrflag ){
      if( (pBt->btsFlags & BTS_READ_ONLY)!=0 ){
        rc = SQLITE_READONLY;
      }else{

        rc = sqlite3PagerBegin(pBt->pPager,wrflag>1,sqlite3TempInMemory(p->db));
        if( rc==SQLITE_OK ){
          rc = newDatabase(pBt);
        }else if( rc==SQLITE_BUSY_SNAPSHOT && pBt->inTransaction==TRANS_NONE ){
          /* if there was no transaction opened when this function was
          ** called and SQLITE_BUSY_SNAPSHOT is returned, change the error
          ** code to SQLITE_BUSY. */
          rc = SQLITE_BUSY;
................................................................................
          put4byte(&pPage1->aData[28], pBt->nPage);
        }
      }
    }
  }

trans_begun:









  if( rc==SQLITE_OK ){
    if( pSchemaVersion ){
      *pSchemaVersion = get4byte(&pBt->pPage1->aData[40]);
    }
    if( wrflag ){
      /* This call makes sure that the pager has the correct number of
      ** open savepoints. If the second parameter is greater than 0 and
      ** the sub-journal is not already open, then it will be opened here.
      */

      rc = sqlite3PagerOpenSavepoint(pBt->pPager, p->db->nSavepoint);



    }
  }

  btreeIntegrity(p);
  sqlite3BtreeLeave(p);
  return rc;
}
................................................................................
  return rc;
}

#else /* ifndef SQLITE_OMIT_AUTOVACUUM */
# define setChildPtrmaps(x) SQLITE_OK
#endif










































































































































































/*
** This routine does the first phase of a two-phase commit.  This routine
** causes a rollback journal to be created (if it does not already exist)
** and populated with enough information so that if a power loss occurs
** the database can be restored to its original state by playing back
** the journal.  Then the contents of the journal are flushed out to
** the disk.  After the journal is safely on oxide, the changes to the
................................................................................
** the write-transaction for this database file is to delete the journal.
*/
int sqlite3BtreeCommitPhaseOne(Btree *p, const char *zMaster){
  int rc = SQLITE_OK;
  if( p->inTrans==TRANS_WRITE ){
    BtShared *pBt = p->pBt;
    sqlite3BtreeEnter(p);

#ifndef SQLITE_OMIT_AUTOVACUUM
    if( pBt->autoVacuum ){

      rc = autoVacuumCommit(pBt);
      if( rc!=SQLITE_OK ){
        sqlite3BtreeLeave(p);
        return rc;
      }
    }
    if( pBt->bDoTruncate ){
      sqlite3PagerTruncateImage(pBt->pPager, pBt->nPage);
    }
#endif




    rc = sqlite3PagerCommitPhaseOne(pBt->pPager, zMaster, 0);

    sqlite3BtreeLeave(p);
  }
  return rc;
}

/*
** This function is called from both BtreeCommitPhaseTwo() and BtreeRollback()
................................................................................

    /* Set the current transaction state to TRANS_NONE and unlock the 
    ** pager if this call closed the only read or write transaction.  */
    p->inTrans = TRANS_NONE;
    unlockBtreeIfUnused(pBt);
  }






  btreeIntegrity(p);
}

/*
** Commit the transaction currently in progress.
**
** This routine implements the second phase of a 2-phase commit.  The
................................................................................
  assert( pBt->inTransaction==TRANS_WRITE );
  /* At the pager level, a statement transaction is a savepoint with
  ** an index greater than all savepoints created explicitly using
  ** SQL statements. It is illegal to open, release or rollback any
  ** such savepoints while the statement transaction savepoint is active.
  */
  rc = sqlite3PagerOpenSavepoint(pBt->pPager, iStatement);



  sqlite3BtreeLeave(p);
  return rc;
}

/*
** The second argument to this function, op, is always SAVEPOINT_ROLLBACK
** or SAVEPOINT_RELEASE. This function either releases or rolls back the
................................................................................
int sqlite3BtreeSavepoint(Btree *p, int op, int iSavepoint){
  int rc = SQLITE_OK;
  if( p && p->inTrans==TRANS_WRITE ){
    BtShared *pBt = p->pBt;
    assert( op==SAVEPOINT_RELEASE || op==SAVEPOINT_ROLLBACK );
    assert( iSavepoint>=0 || (iSavepoint==-1 && op==SAVEPOINT_ROLLBACK) );
    sqlite3BtreeEnter(p);

    if( op==SAVEPOINT_ROLLBACK ){
      rc = saveAllCursors(pBt, 0, 0);
    }
    if( rc==SQLITE_OK ){
      rc = sqlite3PagerSavepoint(pBt->pPager, op, iSavepoint);
    }
    if( rc==SQLITE_OK ){
................................................................................
** This function returns SQLITE_CORRUPT if the page-header flags field of
** the new child page does not match the flags field of the parent (i.e.
** if an intkey page appears to be the parent of a non-intkey page, or
** vice-versa).
*/
static int moveToChild(BtCursor *pCur, u32 newPgno){
  BtShared *pBt = pCur->pBt;


  assert( cursorOwnsBtShared(pCur) );
  assert( pCur->eState==CURSOR_VALID );
  assert( pCur->iPage<BTCURSOR_MAX_DEPTH );
  assert( pCur->iPage>=0 );
  if( pCur->iPage>=(BTCURSOR_MAX_DEPTH-1) ){
    return SQLITE_CORRUPT_BKPT;
................................................................................
  }
  pCur->info.nSize = 0;
  pCur->curFlags &= ~(BTCF_ValidNKey|BTCF_ValidOvfl);
  pCur->aiIdx[pCur->iPage] = pCur->ix;
  pCur->apPage[pCur->iPage] = pCur->pPage;
  pCur->ix = 0;
  pCur->iPage++;
  return getAndInitPage(pBt, newPgno, &pCur->pPage, pCur, pCur->curPagerFlags);





}

#ifdef SQLITE_DEBUG
/*
** Page pParent is an internal (non-leaf) tree page. This function 
** asserts that page number iChild is the left-child if the iIdx'th
** cell in page pParent. Or, if iIdx is equal to the total number of
................................................................................
    }
    rc = getAndInitPage(pCur->pBtree->pBt, pCur->pgnoRoot, &pCur->pPage,
                        0, pCur->curPagerFlags);
    if( rc!=SQLITE_OK ){
      pCur->eState = CURSOR_INVALID;
      return rc;
    }

    pCur->iPage = 0;
    pCur->curIntKey = pCur->pPage->intKey;
  }
  pRoot = pCur->pPage;
  assert( pRoot->pgno==pCur->pgnoRoot );

  /* If pCur->pKeyInfo is not NULL, then the caller that opened this cursor
................................................................................
  u32 n;     /* Number of pages on the freelist */
  u32 k;     /* Number of leaves on the trunk of the freelist */
  MemPage *pTrunk = 0;
  MemPage *pPrevTrunk = 0;
  Pgno mxPage;     /* Total size of the database file */

  assert( sqlite3_mutex_held(pBt->mutex) );
  assert( eMode==BTALLOC_ANY || (nearby>0 && IfNotOmitAV(pBt->autoVacuum)) );
  pPage1 = pBt->pPage1;
  mxPage = btreePagecount(pBt);
  /* EVIDENCE-OF: R-05119-02637 The 4-byte big-endian integer at offset 36
  ** stores stores the total number of pages on the freelist. */
  n = get4byte(&pPage1->aData[36]);
  testcase( n==mxPage-1 );
  if( n>=mxPage ){
    return SQLITE_CORRUPT_BKPT;
  }









  if( n>0 ){
    /* There are pages on the freelist.  Reuse one of those pages. */
    Pgno iTrunk;
    u8 searchList = 0; /* If the free-list must be searched for 'nearby' */
    u32 nSearch = 0;   /* Count of the number of search attempts */
    
    /* If eMode==BTALLOC_EXACT and a query of the pointer-map
    ** shows that the page 'nearby' is somewhere on the free-list, then
    ** the entire-list will be searched for that page.
    */
#ifndef SQLITE_OMIT_AUTOVACUUM
    if( eMode==BTALLOC_EXACT ){


      if( nearby<=mxPage ){
        u8 eType;
        assert( nearby>0 );
        assert( pBt->autoVacuum );
        rc = ptrmapGet(pBt, nearby, &eType, 0);
        if( rc ) return rc;
        if( eType==PTRMAP_FREEPAGE ){
          searchList = 1;
        }



      }
    }else if( eMode==BTALLOC_LE ){
      searchList = 1;
    }
#endif

    /* Decrement the free-list count by 1. Set iTrunk to the index of the
    ** first free-list trunk page. iPrevTrunk is initially 1.
    */
    rc = sqlite3PagerWrite(pPage1->pDbPage);
    if( rc ) return rc;
    put4byte(&pPage1->aData[36], n-1);

    /* The code within this loop is run only once if the 'searchList' variable
    ** is not true. Otherwise, it runs once for each trunk-page on the
    ** free-list until the page 'nearby' is located (eMode==BTALLOC_EXACT)
    ** or until a page less than 'nearby' is located (eMode==BTALLOC_LT)
    */
................................................................................
    }
    memset(pPage->aData, 0, pPage->pBt->pageSize);
  }

  /* If the database supports auto-vacuum, write an entry in the pointer-map
  ** to indicate that the page is free.
  */
  if( ISAUTOVACUUM ){
    ptrmapPut(pBt, iPage, PTRMAP_FREEPAGE, 0, &rc);
    if( rc ) goto freepage_out;
  }

  /* Now manipulate the actual database free-list structure. There are two
  ** possibilities. If the free-list is currently empty, or if the first
  ** trunk page in the free-list is full, then this page will become a
................................................................................
          pgnoOvfl++;
        } while( 
          PTRMAP_ISPAGE(pBt, pgnoOvfl) || pgnoOvfl==PENDING_BYTE_PAGE(pBt) 
        );
      }
#endif
      rc = allocateBtreePage(pBt, &pOvfl, &pgnoOvfl, pgnoOvfl, 0);
#ifndef SQLITE_OMIT_AUTOVACUUM
      /* If the database supports auto-vacuum, and the second or subsequent
      ** overflow page is being allocated, add an entry to the pointer-map
      ** for that page now. 
      **
      ** If this is the first overflow page, then write a partial entry 
      ** to the pointer-map. If we write nothing to this pointer-map slot,
      ** then the optimistic overflow chain processing in clearCell()
      ** may misinterpret the uninitialized values and delete the
      ** wrong pages from the database.
      */
      if( pBt->autoVacuum && rc==SQLITE_OK ){
        u8 eType = (pgnoPtrmap?PTRMAP_OVERFLOW2:PTRMAP_OVERFLOW1);
        ptrmapPut(pBt, pgnoOvfl, eType, pgnoPtrmap, &rc);
        if( rc ){
          releasePage(pOvfl);
        }
      }
#endif
      if( rc ){
        releasePage(pToRelease);
        return rc;
      }

      /* If pToRelease is not zero than pPrior points into the data area
      ** of pToRelease.  Make sure pToRelease is still writeable. */
................................................................................
    ** sorted order.  This invariants arise because multiple overflows can
    ** only occur when inserting divider cells into the parent page during
    ** balancing, and the dividers are adjacent and sorted.
    */
    assert( j==0 || pPage->aiOvfl[j-1]<(u16)i ); /* Overflows in sorted order */
    assert( j==0 || i==pPage->aiOvfl[j-1]+1 );   /* Overflows are sequential */
  }else{

    int rc = sqlite3PagerWrite(pPage->pDbPage);
    if( rc!=SQLITE_OK ){
      *pRC = rc;
      return;
    }
    assert( sqlite3PagerIswriteable(pPage->pDbPage) );
    data = pPage->aData;
................................................................................
    assert( &data[pPage->cellOffset]==pPage->aCellIdx );
    rc = allocateSpace(pPage, sz, &idx);
    if( rc ){ *pRC = rc; return; }
    /* The allocateSpace() routine guarantees the following properties
    ** if it returns successfully */
    assert( idx >= 0 );
    assert( idx >= pPage->cellOffset+2*pPage->nCell+2 || CORRUPT_DB );
    assert( idx+sz <= (int)pPage->pBt->usableSize );
    pPage->nFree -= (u16)(2 + sz);
    if( iChild ){
      /* In a corrupt database where an entry in the cell index section of
      ** a btree page has a value of 3 or less, the pCell value might point
      ** as many as 4 bytes in front of the start of the aData buffer for
      ** the source page.  Make sure this does not cause problems by not
      ** reading the first 4 bytes */
................................................................................
    pIns = pPage->aCellIdx + i*2;
    memmove(pIns+2, pIns, 2*(pPage->nCell - i));
    put2byte(pIns, idx);
    pPage->nCell++;
    /* increment the cell count */
    if( (++data[pPage->hdrOffset+4])==0 ) data[pPage->hdrOffset+3]++;
    assert( get2byte(&data[pPage->hdrOffset+3])==pPage->nCell || CORRUPT_DB );
#ifndef SQLITE_OMIT_AUTOVACUUM
    if( pPage->pBt->autoVacuum ){
      /* The cell may contain a pointer to an overflow page. If so, write
      ** the entry for the overflow page into the pointer map.
      */
      ptrmapPutOvflPtr(pPage, pPage, pCell, pRC);
    }
#endif
  }
}

/*
** The following parameters determine how many adjacent pages get involved
** in a balancing operation.  NN is the number of neighbors on either side
** of the page that participate in the balancing operation.  NB is the
................................................................................
    ** cell on the page to an overflow page. If either of these
    ** operations fails, the return code is set, but the contents
    ** of the parent page are still manipulated by thh code below.
    ** That is Ok, at this point the parent page is guaranteed to
    ** be marked as dirty. Returning an error code will cause a
    ** rollback, undoing any changes made to the parent page.
    */
    if( ISAUTOVACUUM ){
      ptrmapPut(pBt, pgnoNew, PTRMAP_BTREE, pParent->pgno, &rc);
      if( szCell>pNew->minLocal ){
        ptrmapPutOvflPtr(pNew, pNew, pCell, &rc);
      }
    }
  
    /* Create a divider cell to insert into pParent. The divider cell
................................................................................
      *pRC = rc;
      return;
    }
  
    /* If this is an auto-vacuum database, update the pointer-map entries
    ** for any b-tree or overflow pages that pTo now contains the pointers to.
    */
    if( ISAUTOVACUUM ){
      *pRC = setChildPtrmaps(pTo);
    }
  }
}

/*
** This routine redistributes cells on the iParentIdx'th child of pParent
................................................................................
** SQLITE_NOMEM.
*/
static int balance_nonroot(
  MemPage *pParent,               /* Parent page of siblings being balanced */
  int iParentIdx,                 /* Index of "the page" in pParent */
  u8 *aOvflSpace,                 /* page-size bytes of space for parent ovfl */
  int isRoot,                     /* True if pParent is a root-page */
  int bBulk                       /* True if this call is part of a bulk load */

){
  BtShared *pBt;               /* The whole database */
  int nMaxCells = 0;           /* Allocated size of apCell, szCell, aFrom. */
  int nNew = 0;                /* Number of pages in apNew[] */
  int nOld;                    /* Number of pages in apOld[] */
  int i, j, k;                 /* Loop counters */
  int nxDiv;                   /* Next divider slot in pParent->aCell[] */
................................................................................
  pgno = get4byte(pRight);
  while( 1 ){
    rc = getAndInitPage(pBt, pgno, &apOld[i], 0, 0);
    if( rc ){
      memset(apOld, 0, (i+1)*sizeof(MemPage*));
      goto balance_cleanup;
    }

    if( apOld[i]->nFree<0 ){
      rc = btreeComputeFreeSpace(apOld[i]);
      if( rc ){
        memset(apOld, 0, (i)*sizeof(MemPage*));
        goto balance_cleanup;
      }
    }
................................................................................
      if( rc ) goto balance_cleanup;
      zeroPage(pNew, pageFlags);
      apNew[i] = pNew;
      nNew++;
      cntOld[i] = b.nCell;

      /* Set the pointer-map entry for the new sibling page. */
      if( ISAUTOVACUUM ){
        ptrmapPut(pBt, pNew->pgno, PTRMAP_BTREE, pParent->pgno, &rc);
        if( rc!=SQLITE_OK ){
          goto balance_cleanup;
        }
      }
    }
  }
................................................................................
  **      with the cell.
  **
  ** If the sibling pages are not leaves, then the pointer map entry 
  ** associated with the right-child of each sibling may also need to be 
  ** updated. This happens below, after the sibling pages have been 
  ** populated, not here.
  */
  if( ISAUTOVACUUM ){
    MemPage *pOld;
    MemPage *pNew = pOld = apNew[0];
    int cntOldNext = pNew->nCell + pNew->nOverflow;
    int iNew = 0;
    int iOld = 0;

    for(i=0; i<b.nCell; i++){
................................................................................
    assert( apNew[0]->nFree == 
        (get2byteNotZero(&apNew[0]->aData[5]) - apNew[0]->cellOffset
          - apNew[0]->nCell*2)
      || rc!=SQLITE_OK
    );
    copyNodeContent(apNew[0], pParent, &rc);
    freePage(apNew[0], &rc);
  }else if( ISAUTOVACUUM && !leafCorrection ){
    /* Fix the pointer map entries associated with the right-child of each
    ** sibling page. All other pointer map entries have already been taken
    ** care of.  */
    for(i=0; i<nNew; i++){
      u32 key = get4byte(&apNew[i]->aData[8]);
      ptrmapPut(pBt, key, PTRMAP_BTREE, apNew[i]->pgno, &rc);
    }
................................................................................
  ** page that will become the new right-child of pPage. Copy the contents
  ** of the node stored on pRoot into the new child page.
  */
  rc = sqlite3PagerWrite(pRoot->pDbPage);
  if( rc==SQLITE_OK ){
    rc = allocateBtreePage(pBt,&pChild,&pgnoChild,pRoot->pgno,0);
    copyNodeContent(pRoot, pChild, &rc);
    if( ISAUTOVACUUM ){
      ptrmapPut(pBt, pgnoChild, PTRMAP_BTREE, pRoot->pgno, &rc);
    }
  }
  if( rc ){
    *ppChild = 0;
    releasePage(pChild);
    return rc;
................................................................................
          ** has completed, it is safe to release the pSpace buffer used by
          ** the previous call, as the overflow cell data will have been 
          ** copied either into the body of a database page or into the new
          ** pSpace buffer passed to the latter call to balance_nonroot().
          */
          u8 *pSpace = sqlite3PageMalloc(pCur->pBt->pageSize);
          rc = balance_nonroot(pParent, iIdx, pSpace, iPage==1,
                               pCur->hints&BTREE_BULKLOAD);
          if( pFree ){
            /* If pFree is not NULL, it points to the pSpace buffer used 
            ** by a previous call to balance_nonroot(). Its contents are
            ** now stored either on real database pages or within the 
            ** new pSpace buffer, so it may be safely freed here. */
            sqlite3PageFree(pFree);
          }
................................................................................
    if( !pPage->leaf ){
      memcpy(newCell, oldCell, 4);
    }
    rc = clearCell(pPage, oldCell, &info);
    testcase( pCur->curFlags & BTCF_ValidOvfl );
    invalidateOverflowCache(pCur);
    if( info.nSize==szNew && info.nLocal==info.nPayload 
     && (!ISAUTOVACUUM || szNew<pPage->minLocal)
    ){
      /* Overwrite the old cell with the new if they are the same size.
      ** We could also try to do this if the old cell is smaller, then add
      ** the leftover space to the free list.  But experiments show that
      ** doing that is no faster then skipping this optimization and just
      ** calling dropCell() and insertCell(). 
      **
................................................................................
** Erase the given database page and all its children.  Return
** the page to the freelist.
*/
static int clearDatabasePage(
  BtShared *pBt,           /* The BTree that contains the table */
  Pgno pgno,               /* Page number to clear */
  int freePageFlag,        /* Deallocate page if true */
  int *pnChange            /* Add number of Cells freed to this counter */

){
  MemPage *pPage;
  int rc;
  unsigned char *pCell;
  int i;
  int hdr;
  CellInfo info;
................................................................................

  assert( sqlite3_mutex_held(pBt->mutex) );
  if( pgno>btreePagecount(pBt) ){
    return SQLITE_CORRUPT_BKPT;
  }
  rc = getAndInitPage(pBt, pgno, &pPage, 0, 0);
  if( rc ) return rc;

  if( pPage->bBusy ){
    rc = SQLITE_CORRUPT_BKPT;
    goto cleardatabasepage_out;
  }
  pPage->bBusy = 1;
  hdr = pPage->hdrOffset;
  for(i=0; i<pPage->nCell; i++){
    pCell = findCell(pPage, i);
    if( !pPage->leaf ){
      rc = clearDatabasePage(pBt, get4byte(pCell), 1, pnChange);
      if( rc ) goto cleardatabasepage_out;
    }
    rc = clearCell(pPage, pCell, &info);
    if( rc ) goto cleardatabasepage_out;
  }
  if( !pPage->leaf ){

    rc = clearDatabasePage(pBt, get4byte(&pPage->aData[hdr+8]), 1, pnChange);

    if( rc ) goto cleardatabasepage_out;
  }else if( pnChange ){
    assert( pPage->intKey || CORRUPT_DB );
    testcase( !pPage->intKey );
    *pnChange += pPage->nCell;
  }
  if( freePageFlag ){
................................................................................
  rc = saveAllCursors(pBt, (Pgno)iTable, 0);

  if( SQLITE_OK==rc ){
    /* Invalidate all incrblob cursors open on table iTable (assuming iTable
    ** is the root of a table b-tree - if it is not, the following call is
    ** a no-op).  */
    invalidateIncrblobCursors(p, (Pgno)iTable, 0, 1);
    rc = clearDatabasePage(pBt, (Pgno)iTable, 0, pnChange);
  }
  sqlite3BtreeLeave(p);
  return rc;
}

/*
** Delete all information from the single table that pCur is open on.
................................................................................
      checkPtrmap(&sCheck, aRoot[i], PTRMAP_ROOTPAGE, 0);
    }
#endif
    checkTreePage(&sCheck, aRoot[i], &notUsed, LARGEST_INT64);
  }
  pBt->db->flags = savedDbFlags;

  /* Make sure every page in the file is referenced
  */


  for(i=1; i<=sCheck.nPage && sCheck.mxErr; i++){
#ifdef SQLITE_OMIT_AUTOVACUUM
    if( getPageReferenced(&sCheck, i)==0 ){
      checkAppendMsg(&sCheck, "Page %d is never used", i);
    }
#else
    /* If the database supports auto-vacuum, make sure no tables contain
    ** references to pointer-map pages.
................................................................................
  return (p->pBt->btsFlags & BTS_READ_ONLY)!=0;
}

/*
** Return the size of the header added to each page by this module.
*/
int sqlite3HeaderSizeBtree(void){ return ROUND8(sizeof(MemPage)); }






























































































#if !defined(SQLITE_OMIT_SHARED_CACHE)
/*
** Return true if the Btree passed as the only argument is sharable.
*/
int sqlite3BtreeSharable(Btree *p){
  return p->sharable;







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|







 







|

>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>







 







>







 







>
|







 







>
>
>
>
>
>
>
>
>









>
|
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>


>










>
>
>
>
|
>







 







>
>
>
>
>







 







>
>
>







 







>







 







>







 







|
>
>
>
>
>







 







>







 







|









>
>
>
>
>
>
>
>
>










<

>
>
|
|
|
|
|
|
|
|
|
>
>
>




<




<
<







 







|







 







|










|






<







 







>







 







|







 







|
<





<







 







|







 







|







 







|
>







 







>







 







|







 







|







 







|







 







|







 







|







 







|







 







|
>







 







>









|






>
|
>







 







|







 







|
|
>
>
|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
....
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
....
2447
2448
2449
2450
2451
2452
2453
2454
2455
2456
2457
2458
2459
2460
2461
2462
2463
2464
2465
2466
2467
2468
2469
2470
2471
....
3624
3625
3626
3627
3628
3629
3630
3631
3632
3633
3634
3635
3636
3637
3638
....
3700
3701
3702
3703
3704
3705
3706
3707
3708
3709
3710
3711
3712
3713
3714
3715
....
3761
3762
3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777
3778
3779
3780
3781
3782
3783
3784
3785
3786
3787
3788
3789
3790
3791
3792
3793
3794
3795
3796
3797
....
4216
4217
4218
4219
4220
4221
4222
4223
4224
4225
4226
4227
4228
4229
4230
4231
4232
4233
4234
4235
4236
4237
4238
4239
4240
4241
4242
4243
4244
4245
4246
4247
4248
4249
4250
4251
4252
4253
4254
4255
4256
4257
4258
4259
4260
4261
4262
4263
4264
4265
4266
4267
4268
4269
4270
4271
4272
4273
4274
4275
4276
4277
4278
4279
4280
4281
4282
4283
4284
4285
4286
4287
4288
4289
4290
4291
4292
4293
4294
4295
4296
4297
4298
4299
4300
4301
4302
4303
4304
4305
4306
4307
4308
4309
4310
4311
4312
4313
4314
4315
4316
4317
4318
4319
4320
4321
4322
4323
4324
4325
4326
4327
4328
4329
4330
4331
4332
4333
4334
4335
4336
4337
4338
4339
4340
4341
4342
4343
4344
4345
4346
4347
4348
4349
4350
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
4366
4367
4368
4369
4370
4371
4372
4373
4374
4375
4376
4377
4378
4379
4380
4381
4382
4383
4384
4385
4386
4387
4388
4389
4390
4391
4392
4393
4394
4395
4396
4397
4398
....
4416
4417
4418
4419
4420
4421
4422
4423
4424
4425
4426
4427
4428
4429
4430
4431
4432
4433
4434
4435
4436
4437
4438
4439
4440
4441
4442
4443
4444
4445
4446
4447
4448
4449
....
4478
4479
4480
4481
4482
4483
4484
4485
4486
4487
4488
4489
4490
4491
4492
4493
4494
4495
4496
....
4712
4713
4714
4715
4716
4717
4718
4719
4720
4721
4722
4723
4724
4725
4726
4727
4728
....
4738
4739
4740
4741
4742
4743
4744
4745
4746
4747
4748
4749
4750
4751
4752
....
5526
5527
5528
5529
5530
5531
5532
5533
5534
5535
5536
5537
5538
5539
5540
....
5541
5542
5543
5544
5545
5546
5547
5548
5549
5550
5551
5552
5553
5554
5555
5556
5557
5558
5559
5560
....
5658
5659
5660
5661
5662
5663
5664
5665
5666
5667
5668
5669
5670
5671
5672
....
6365
6366
6367
6368
6369
6370
6371
6372
6373
6374
6375
6376
6377
6378
6379
6380
6381
6382
6383
6384
6385
6386
6387
6388
6389
6390
6391
6392
6393
6394
6395
6396
6397
6398
6399
6400

6401
6402
6403
6404
6405
6406
6407
6408
6409
6410
6411
6412
6413
6414
6415
6416
6417
6418
6419

6420
6421
6422
6423


6424
6425
6426
6427
6428
6429
6430
....
6724
6725
6726
6727
6728
6729
6730
6731
6732
6733
6734
6735
6736
6737
6738
....
7040
7041
7042
7043
7044
7045
7046
7047
7048
7049
7050
7051
7052
7053
7054
7055
7056
7057
7058
7059
7060
7061
7062
7063
7064

7065
7066
7067
7068
7069
7070
7071
....
7196
7197
7198
7199
7200
7201
7202
7203
7204
7205
7206
7207
7208
7209
7210
....
7211
7212
7213
7214
7215
7216
7217
7218
7219
7220
7221
7222
7223
7224
7225
....
7231
7232
7233
7234
7235
7236
7237
7238

7239
7240
7241
7242
7243

7244
7245
7246
7247
7248
7249
7250
....
7774
7775
7776
7777
7778
7779
7780
7781
7782
7783
7784
7785
7786
7787
7788
....
7912
7913
7914
7915
7916
7917
7918
7919
7920
7921
7922
7923
7924
7925
7926
....
7963
7964
7965
7966
7967
7968
7969
7970
7971
7972
7973
7974
7975
7976
7977
7978
....
8053
8054
8055
8056
8057
8058
8059
8060
8061
8062
8063
8064
8065
8066
8067
....
8398
8399
8400
8401
8402
8403
8404
8405
8406
8407
8408
8409
8410
8411
8412
....
8494
8495
8496
8497
8498
8499
8500
8501
8502
8503
8504
8505
8506
8507
8508
....
8680
8681
8682
8683
8684
8685
8686
8687
8688
8689
8690
8691
8692
8693
8694
....
8763
8764
8765
8766
8767
8768
8769
8770
8771
8772
8773
8774
8775
8776
8777
....
8918
8919
8920
8921
8922
8923
8924
8925
8926
8927
8928
8929
8930
8931
8932
....
9252
9253
9254
9255
9256
9257
9258
9259
9260
9261
9262
9263
9264
9265
9266
....
9699
9700
9701
9702
9703
9704
9705
9706
9707
9708
9709
9710
9711
9712
9713
9714
....
9715
9716
9717
9718
9719
9720
9721
9722
9723
9724
9725
9726
9727
9728
9729
9730
9731
9732
9733
9734
9735
9736
9737
9738
9739
9740
9741
9742
9743
9744
9745
9746
9747
9748
....
9779
9780
9781
9782
9783
9784
9785
9786
9787
9788
9789
9790
9791
9792
9793
.....
10665
10666
10667
10668
10669
10670
10671
10672
10673
10674
10675
10676
10677
10678
10679
10680
10681
10682
10683
.....
10972
10973
10974
10975
10976
10977
10978
10979
10980
10981
10982
10983
10984
10985
10986
10987
10988
10989
10990
10991
10992
10993
10994
10995
10996
10997
10998
10999
11000
11001
11002
11003
11004
11005
11006
11007
11008
11009
11010
11011
11012
11013
11014
11015
11016
11017
11018
11019
11020
11021
11022
11023
11024
11025
11026
11027
11028
11029
11030
11031
11032
11033
11034
11035
11036
11037
11038
11039
11040
11041
11042
11043
11044
11045
11046
11047
11048
11049
11050
11051
11052
11053
11054
11055
11056
11057
11058
11059
11060
11061
11062
11063
11064
11065
11066
11067
11068
11069
11070
11071
11072
11073
11074
11075
11076
11077
11078
      pLock->eLock = READ_LOCK;
    }
  }
}

#endif /* SQLITE_OMIT_SHARED_CACHE */

#ifndef SQLITE_OMIT_CONCURRENT
/*
** The following structure - BtreePtrmap - stores the in-memory pointer map
** used for newly allocated pages in CONCURRENT transactions. Such pages are
** always allocated in a contiguous block (from the end of the file) starting
** with page BtreePtrmap.iFirst.
*/
typedef struct RollbackEntry RollbackEntry;
typedef struct PtrmapEntry PtrmapEntry;
struct PtrmapEntry {
  Pgno parent;
  u8 eType;
};
struct RollbackEntry {
  Pgno pgno;
  Pgno parent;
  u8 eType;
};
struct BtreePtrmap {
  Pgno iFirst;                    /* First new page number aPtr[0] */

  int nPtrAlloc;                  /* Allocated size of aPtr[] array */
  PtrmapEntry *aPtr;              /* Array of parent page numbers */

  int nSvpt;                      /* Used size of aSvpt[] array */
  int nSvptAlloc;                 /* Allocated size of aSvpt[] */
  int *aSvpt;                     /* First aRollback[] entry for savepoint i */

  int nRollback;                  /* Used size of aRollback[] array */
  int nRollbackAlloc;             /* Allocated size of aRollback[] array */
  RollbackEntry *aRollback;       /* Array of rollback entries */
};

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** If page number pgno is greater than or equal to BtreePtrmap.iFirst, 
** store an entry for it in the pointer-map structure.
*/
static int btreePtrmapStore(
  BtShared *pBt,
  Pgno pgno,
  u8 eType, 
  Pgno parent
){
  BtreePtrmap *pMap = pBt->pMap;
  if( pgno>=pMap->iFirst ){
    int iEntry = pgno - pMap->iFirst;

    /* Grow the aPtr[] array as required */
    while( iEntry>=pMap->nPtrAlloc ){
      int nNew = pMap->nPtrAlloc ? pMap->nPtrAlloc*2 : 16;
      PtrmapEntry *aNew = (PtrmapEntry*)sqlite3_realloc(
          pMap->aPtr, nNew*sizeof(PtrmapEntry)
      );
      if( aNew==0 ){
        return SQLITE_NOMEM;
      }else{
        int nByte = (nNew-pMap->nPtrAlloc)*sizeof(PtrmapEntry);
        memset(&aNew[pMap->nPtrAlloc], 0, nByte);
        pMap->aPtr = aNew;
        pMap->nPtrAlloc = nNew;
      }
    }

    /* Add an entry to the rollback log if required */
    if( pMap->nSvpt>0 && pMap->aPtr[iEntry].parent ){
      if( pMap->nRollback>=pMap->nRollbackAlloc ){
        int nNew = pMap->nRollback ? pMap->nRollback*2 : 16;
        RollbackEntry *aNew = (RollbackEntry*)sqlite3_realloc(
            pMap->aRollback, nNew*sizeof(RollbackEntry)
        );
        if( aNew==0 ){
          return SQLITE_NOMEM;
        }else{
          pMap->aRollback = aNew;
          pMap->nRollbackAlloc = nNew;
        }
      }

      pMap->aRollback[pMap->nRollback].pgno = pgno;
      pMap->aRollback[pMap->nRollback].parent = pMap->aPtr[iEntry].parent;
      pMap->aRollback[pMap->nRollback].eType = pMap->aPtr[iEntry].eType;
      pMap->nRollback++;
    }

    /* Update the aPtr[] array */
    pMap->aPtr[iEntry].parent = parent;
    pMap->aPtr[iEntry].eType = eType;
  }

  return SQLITE_OK;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** Open savepoint iSavepoint, if it is not already open.
*/
static int btreePtrmapBegin(BtShared *pBt, int nSvpt){
  BtreePtrmap *pMap = pBt->pMap;
  if( pMap && nSvpt>pMap->nSvpt ){
    int i;
    if( nSvpt>=pMap->nSvptAlloc ){
      int nNew = pMap->nSvptAlloc ? pMap->nSvptAlloc*2 : 16;
      int *aNew = sqlite3_realloc(pMap->aSvpt, sizeof(int) * nNew);
      if( aNew==0 ){
        return SQLITE_NOMEM;
      }else{
        pMap->aSvpt = aNew;
        pMap->nSvptAlloc = nNew;
      }
    }

    for(i=pMap->nSvpt; i<nSvpt; i++){
      pMap->aSvpt[i] = pMap->nRollback;
    }
    pMap->nSvpt = nSvpt;
  }

  return SQLITE_OK;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** Rollback (if op==SAVEPOINT_ROLLBACK) or release (if op==SAVEPOINT_RELEASE)
** savepoint iSvpt.
*/
static void btreePtrmapEnd(BtShared *pBt, int op, int iSvpt){
  BtreePtrmap *pMap = pBt->pMap;
  if( pMap ){
    assert( op==SAVEPOINT_ROLLBACK || op==SAVEPOINT_RELEASE );
    assert( iSvpt>=0 || (iSvpt==-1 && op==SAVEPOINT_ROLLBACK) );
    if( iSvpt<0 ){
      pMap->nSvpt = 0;
      pMap->nRollback = 0;
      memset(pMap->aPtr, 0, sizeof(Pgno) * pMap->nPtrAlloc);
    }else if( iSvpt<pMap->nSvpt ){
      if( op==SAVEPOINT_ROLLBACK ){
        int ii;
        for(ii=pMap->nRollback-1; ii>=pMap->aSvpt[iSvpt]; ii--){
          RollbackEntry *p = &pMap->aRollback[ii];
          PtrmapEntry *pEntry = &pMap->aPtr[p->pgno - pMap->iFirst];
          pEntry->parent = p->parent;
          pEntry->eType = p->eType;
        }
      }
      pMap->nSvpt = iSvpt + (op==SAVEPOINT_ROLLBACK);
      pMap->nRollback = pMap->aSvpt[iSvpt];
    }
  }
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** This function is called after an CONCURRENT transaction is opened on the
** database. It allocates the BtreePtrmap structure used to track pointers
** to allocated pages and zeroes the nFree/iTrunk fields in the database 
** header on page 1.
*/
static int btreePtrmapAllocate(BtShared *pBt){
  int rc = SQLITE_OK;
  if( pBt->pMap==0 ){
    BtreePtrmap *pMap = sqlite3_malloc(sizeof(BtreePtrmap));
    if( pMap==0 ){
      rc = SQLITE_NOMEM;
    }else{
      memset(&pBt->pPage1->aData[32], 0, sizeof(u32)*2);
      memset(pMap, 0, sizeof(BtreePtrmap));
      pMap->iFirst = pBt->nPage + 1;
      pBt->pMap = pMap;
    }
  }
  return rc;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** Free any BtreePtrmap structure allocated by an earlier call to
** btreePtrmapAllocate().
*/
static void btreePtrmapDelete(BtShared *pBt){
  BtreePtrmap *pMap = pBt->pMap;
  if( pMap ){
    sqlite3_free(pMap->aRollback);
    sqlite3_free(pMap->aPtr);
    sqlite3_free(pMap->aSvpt);
    sqlite3_free(pMap);
    pBt->pMap = 0;
  }
}

/*
** Check that the pointer-map does not contain any entries with a parent
** page of 0. Call sqlite3_log() multiple times to output the entire
** data structure if it does.
*/
static void btreePtrmapCheck(BtShared *pBt, Pgno nPage){
  Pgno i;
  int bProblem = 0;
  BtreePtrmap *p = pBt->pMap;

  for(i=p->iFirst; i<=nPage; i++){
    PtrmapEntry *pEntry = &p->aPtr[i-p->iFirst];
    if( pEntry->eType==PTRMAP_OVERFLOW1
     || pEntry->eType==PTRMAP_OVERFLOW2
     || pEntry->eType==PTRMAP_BTREE
    ){
      if( pEntry->parent==0 ){
        bProblem = 1;
        break;
      }
    }
  }

  if( bProblem ){
    for(i=p->iFirst; i<=nPage; i++){
      PtrmapEntry *pEntry = &p->aPtr[i-p->iFirst];
      sqlite3_log(SQLITE_CORRUPT, 
          "btreePtrmapCheck: pgno=%d eType=%d parent=%d", 
          (int)i, (int)pEntry->eType, (int)pEntry->parent
      );
    }
    abort();
  }
}

#else  /* SQLITE_OMIT_CONCURRENT */
# define btreePtrmapAllocate(x) SQLITE_OK
# define btreePtrmapDelete(x) 
# define btreePtrmapBegin(x,y)  SQLITE_OK
# define btreePtrmapEnd(x,y,z) 
# define btreePtrmapCheck(y,z) 
#endif /* SQLITE_OMIT_CONCURRENT */

static void releasePage(MemPage *pPage);  /* Forward reference */
static void releasePageOne(MemPage *pPage);      /* Forward reference */
static void releasePageNotNull(MemPage *pPage);  /* Forward reference */

/*
***** This routine is used inside of assert() only ****
**
** Verify that the cursor holds the mutex on its BtShared
................................................................................
  Pgno iPtrmap;     /* The pointer map page number */
  int offset;       /* Offset in pointer map page */
  int rc;           /* Return code from subfunctions */

  if( *pRC ) return;

  assert( sqlite3_mutex_held(pBt->mutex) );
  /* The master-journal page number is never added to a pointer-map page */
  assert( 0==PTRMAP_ISPAGE(pBt, PENDING_BYTE_PAGE(pBt)) );

#ifndef SQLITE_OMIT_CONCURRENT
  if( pBt->pMap ){
    *pRC = btreePtrmapStore(pBt, key, eType, parent);
    return;
  }
#endif

  assert( pBt->autoVacuum );
  if( key==0 ){
    *pRC = SQLITE_CORRUPT_BKPT;
    return;
  }
  iPtrmap = PTRMAP_PAGENO(pBt, key);
................................................................................
    pCur->pPage = pCur->apPage[pCur->iPage];
  }
  testcase( pgno==0 );
  assert( pgno!=0 || rc==SQLITE_CORRUPT );
  return rc;
}

#ifndef SQLITE_OMIT_CONCURRENT
/* 
** Set the value of the MemPage.pgnoRoot variable, if it exists.
*/
static void setMempageRoot(MemPage *pPg, u32 pgnoRoot){
  pPg->pgnoRoot = pgnoRoot;
}
#else
# define setMempageRoot(x,y)
#endif

/*
** Release a MemPage.  This should be called once for each prior
** call to btreeGetPage.
**
** Page1 is a special case and must be released using releasePageOne().
*/
static void releasePageNotNull(MemPage *pPage){
................................................................................
** no progress.  By returning SQLITE_BUSY and not invoking the busy callback
** when A already has a read lock, we encourage A to give up and let B
** proceed.
*/
int sqlite3BtreeBeginTrans(Btree *p, int wrflag, int *pSchemaVersion){
  BtShared *pBt = p->pBt;
  int rc = SQLITE_OK;
  int bConcurrent = (p->db->bConcurrent && !ISAUTOVACUUM);

  sqlite3BtreeEnter(p);
  btreeIntegrity(p);

  /* If the btree is already in a write-transaction, or it
  ** is already in a read-transaction and a read-transaction
  ** is requested, this is a no-op.
................................................................................
    */
    while( pBt->pPage1==0 && SQLITE_OK==(rc = lockBtree(pBt)) );

    if( rc==SQLITE_OK && wrflag ){
      if( (pBt->btsFlags & BTS_READ_ONLY)!=0 ){
        rc = SQLITE_READONLY;
      }else{
        int exFlag = bConcurrent ? -1 : (wrflag>1);
        rc = sqlite3PagerBegin(pBt->pPager, exFlag, sqlite3TempInMemory(p->db));
        if( rc==SQLITE_OK ){
          rc = newDatabase(pBt);
        }else if( rc==SQLITE_BUSY_SNAPSHOT && pBt->inTransaction==TRANS_NONE ){
          /* if there was no transaction opened when this function was
          ** called and SQLITE_BUSY_SNAPSHOT is returned, change the error
          ** code to SQLITE_BUSY. */
          rc = SQLITE_BUSY;
................................................................................
          put4byte(&pPage1->aData[28], pBt->nPage);
        }
      }
    }
  }

trans_begun:
#ifndef SQLITE_OMIT_CONCURRENT
  if( bConcurrent && rc==SQLITE_OK && sqlite3PagerIsWal(pBt->pPager) ){
    rc = sqlite3PagerBeginConcurrent(pBt->pPager);
    if( rc==SQLITE_OK && wrflag ){
      rc = btreePtrmapAllocate(pBt);
    }
  }
#endif

  if( rc==SQLITE_OK ){
    if( pSchemaVersion ){
      *pSchemaVersion = get4byte(&pBt->pPage1->aData[40]);
    }
    if( wrflag ){
      /* This call makes sure that the pager has the correct number of
      ** open savepoints. If the second parameter is greater than 0 and
      ** the sub-journal is not already open, then it will be opened here.
      */
      int nSavepoint = p->db->nSavepoint;
      rc = sqlite3PagerOpenSavepoint(pBt->pPager, nSavepoint);
      if( rc==SQLITE_OK && nSavepoint ){
        rc = btreePtrmapBegin(pBt, nSavepoint);
      }
    }
  }

  btreeIntegrity(p);
  sqlite3BtreeLeave(p);
  return rc;
}
................................................................................
  return rc;
}

#else /* ifndef SQLITE_OMIT_AUTOVACUUM */
# define setChildPtrmaps(x) SQLITE_OK
#endif

#ifndef SQLITE_OMIT_CONCURRENT
/*
** This function is called as part of merging an CONCURRENT transaction with
** the snapshot at the head of the wal file. It relocates all pages in the
** range iFirst..iLast, inclusive. It is assumed that the BtreePtrmap 
** structure at BtShared.pMap contains the location of the pointers to each
** page in the range.
**
** If pnCurrent is NULL, then all pages in the range are moved to currently
** free locations (i.e. free-list entries) within the database file before page
** iFirst.
**
** Or, if pnCurrent is not NULL, then it points to a value containing the
** current size of the database file in pages. In this case, all pages are
** relocated to the end of the database file - page iFirst is relocated to
** page (*pnCurrent+1), page iFirst+1 to page (*pnCurrent+2), and so on.
** Value *pnCurrent is set to the new size of the database before this 
** function returns.
**
** If no error occurs, SQLITE_OK is returned. Otherwise, an SQLite error code.
*/
static int btreeRelocateRange(
  BtShared *pBt,                  /* B-tree handle */
  Pgno iFirst,                    /* First page to relocate */
  Pgno iLast,                     /* Last page to relocate */
  Pgno *pnCurrent                 /* If not NULL, IN/OUT: Database size */
){
  int rc = SQLITE_OK;
  BtreePtrmap *pMap = pBt->pMap;
  Pgno iPg;

  for(iPg=iFirst; iPg<=iLast && rc==SQLITE_OK; iPg++){
    MemPage *pFree = 0;     /* Page allocated from free-list */
    MemPage *pPg = 0;
    Pgno iNew;              /* New page number for pPg */
    PtrmapEntry *pEntry;    /* Pointer map entry for page iPg */

    if( iPg==PENDING_BYTE_PAGE(pBt) ) continue;
    pEntry = &pMap->aPtr[iPg - pMap->iFirst];

    if( pEntry->eType==PTRMAP_FREEPAGE ){
      Pgno dummy;
      rc = allocateBtreePage(pBt, &pFree, &dummy, iPg, BTALLOC_EXACT);
      if( pFree ){
        assert( sqlite3PagerPageRefcount(pFree->pDbPage)==1 );
        sqlite3PcacheDrop(pFree->pDbPage);
      }
      assert( rc!=SQLITE_OK || dummy==iPg );
    }else if( pnCurrent ){
      btreeGetPage(pBt, iPg, &pPg, 0);
      assert( sqlite3PagerIswriteable(pPg->pDbPage) );
      assert( sqlite3PagerPageRefcount(pPg->pDbPage)==1 );
      iNew = ++(*pnCurrent);
      if( iNew==PENDING_BYTE_PAGE(pBt) ) iNew = ++(*pnCurrent);
      rc = relocatePage(pBt, pPg, pEntry->eType, pEntry->parent, iNew, 1);
      releasePageNotNull(pPg);
    }else{
      rc = allocateBtreePage(pBt, &pFree, &iNew, iFirst-1, BTALLOC_LE);
      assert( rc!=SQLITE_OK || iNew<iFirst );
      if( rc==SQLITE_OK ){
        releasePage(pFree);
        btreeGetPage(pBt, iPg, &pPg, 0);
        rc = relocatePage(pBt, pPg, pEntry->eType, pEntry->parent,iNew,1);
        releasePage(pPg);
      }
    }
  }
  return rc;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** The b-tree handle passed as the only argument is about to commit an
** CONCURRENT transaction. At this point it is guaranteed that this is 
** possible - the wal WRITER lock is held and it is known that there are 
** no conflicts with committed transactions.
*/
static int btreeFixUnlocked(Btree *p){
  BtShared *pBt = p->pBt;
  MemPage *pPage1 = pBt->pPage1;
  u8 *p1 = pPage1->aData;
  Pager *pPager = pBt->pPager;
  int rc = SQLITE_OK;

  /* If page 1 of the database is not writable, then no pages were allocated
  ** or freed by this transaction. In this case no special handling is 
  ** required. Otherwise, if page 1 is dirty, proceed.  */
  BtreePtrmap *pMap = pBt->pMap;
  Pgno iTrunk = get4byte(&p1[32]);
  Pgno nPage = btreePagecount(pBt);
  u32 nFree = get4byte(&p1[36]);

  assert( pBt->pMap );
  rc = sqlite3PagerUpgradeSnapshot(pPager, pPage1->pDbPage);
  assert( p1==pPage1->aData );

  if( rc==SQLITE_OK ){
    Pgno nHPage = get4byte(&p1[28]);
    Pgno nFin = nHPage;         /* Size of db after transaction merge */

    if( sqlite3PagerIswriteable(pPage1->pDbPage) ){
      Pgno iHTrunk = get4byte(&p1[32]);
      u32 nHFree = get4byte(&p1[36]);

      btreePtrmapCheck(pBt, nPage);

      /* Attach the head database free list to the end of the current
      ** transactions free-list (if any).  */
      if( iTrunk!=0 ){
        put4byte(&p1[36], nHFree + nFree);
        put4byte(&p1[32], iTrunk);
        while( iTrunk ){
          DbPage *pTrunk = sqlite3PagerLookup(pPager, iTrunk);
          iTrunk = get4byte((u8*)pTrunk->pData);
          if( iTrunk==0 ){
            put4byte((u8*)pTrunk->pData, iHTrunk);
          }
          sqlite3PagerUnref(pTrunk);
        };
      }

      if( nHPage<(pMap->iFirst-1) ){
        /* The database consisted of (pMap->iFirst-1) pages when the current
        ** concurrent transaction was opened. And an concurrent transaction may
        ** not be executed on an auto-vacuum database - so the db should 
        ** not have shrunk since the transaction was opened. Therefore nHPage
        ** should be set to (pMap->iFirst-1) or greater. */
        rc = SQLITE_CORRUPT_BKPT;
      }else{
        /* The current transaction allocated pages pMap->iFirst through
        ** nPage (inclusive) at the end of the database file. Meanwhile,
        ** other transactions have allocated (iFirst..nHPage). So move
        ** pages (iFirst..MIN(nPage,nHPage)) to (MAX(nPage,nHPage)+1). */
        Pgno iLast = MIN(nPage, nHPage);    /* Last page to move */
        Pgno nCurrent;                      /* Current size of db */

        nCurrent = MAX(nPage, nHPage);
        pBt->nPage = nCurrent;
        rc = btreeRelocateRange(pBt, pMap->iFirst, iLast, &nCurrent);

        /* There are now no collisions with the snapshot at the head of the
        ** database file. So at this point it would be possible to write
        ** the transaction out to disk. Before doing so though, attempt to
        ** relocate some of the new pages to free locations within the body
        ** of the database file (i.e. free-list entries). */
        if( rc==SQLITE_OK ){
          assert( nCurrent!=PENDING_BYTE_PAGE(pBt) );
          sqlite3PagerSetDbsize(pBt->pPager, nCurrent);
          nFree = get4byte(&p1[36]);
          nFin = nCurrent-nFree;
          if( nCurrent>PENDING_BYTE_PAGE(pBt) && nFin<=PENDING_BYTE_PAGE(pBt) ){
            nFin--;
          }
          nFin = MAX(nFin, nHPage);
          rc = btreeRelocateRange(pBt, nFin+1, nCurrent, 0);
        }

        put4byte(&p1[28], nFin);
      }
    }
    sqlite3PagerSetDbsize(pPager, nFin);
  }

  return rc;
}
#else
# define btreeFixUnlocked(X)  SQLITE_OK
#endif /* SQLITE_OMIT_CONCURRENT */

/*
** This routine does the first phase of a two-phase commit.  This routine
** causes a rollback journal to be created (if it does not already exist)
** and populated with enough information so that if a power loss occurs
** the database can be restored to its original state by playing back
** the journal.  Then the contents of the journal are flushed out to
** the disk.  After the journal is safely on oxide, the changes to the
................................................................................
** the write-transaction for this database file is to delete the journal.
*/
int sqlite3BtreeCommitPhaseOne(Btree *p, const char *zMaster){
  int rc = SQLITE_OK;
  if( p->inTrans==TRANS_WRITE ){
    BtShared *pBt = p->pBt;
    sqlite3BtreeEnter(p);

#ifndef SQLITE_OMIT_AUTOVACUUM
    if( pBt->autoVacuum ){
      assert( ISCONCURRENT==0 );
      rc = autoVacuumCommit(pBt);
      if( rc!=SQLITE_OK ){
        sqlite3BtreeLeave(p);
        return rc;
      }
    }
    if( pBt->bDoTruncate ){
      sqlite3PagerTruncateImage(pBt->pPager, pBt->nPage);
    }
#endif
    if( rc==SQLITE_OK && ISCONCURRENT ){
      rc = btreeFixUnlocked(p);
    }
    if( rc==SQLITE_OK ){
      rc = sqlite3PagerCommitPhaseOne(pBt->pPager, zMaster, 0);
    }
    sqlite3BtreeLeave(p);
  }
  return rc;
}

/*
** This function is called from both BtreeCommitPhaseTwo() and BtreeRollback()
................................................................................

    /* Set the current transaction state to TRANS_NONE and unlock the 
    ** pager if this call closed the only read or write transaction.  */
    p->inTrans = TRANS_NONE;
    unlockBtreeIfUnused(pBt);
  }

  /* If this was an CONCURRENT transaction, delete the pBt->pMap object.
  ** Also call PagerEndConcurrent() to ensure that the pager has discarded
  ** the record of all pages read within the transaction.  */
  btreePtrmapDelete(pBt);
  sqlite3PagerEndConcurrent(pBt->pPager);
  btreeIntegrity(p);
}

/*
** Commit the transaction currently in progress.
**
** This routine implements the second phase of a 2-phase commit.  The
................................................................................
  assert( pBt->inTransaction==TRANS_WRITE );
  /* At the pager level, a statement transaction is a savepoint with
  ** an index greater than all savepoints created explicitly using
  ** SQL statements. It is illegal to open, release or rollback any
  ** such savepoints while the statement transaction savepoint is active.
  */
  rc = sqlite3PagerOpenSavepoint(pBt->pPager, iStatement);
  if( rc==SQLITE_OK ){
    rc = btreePtrmapBegin(pBt, iStatement);
  }
  sqlite3BtreeLeave(p);
  return rc;
}

/*
** The second argument to this function, op, is always SAVEPOINT_ROLLBACK
** or SAVEPOINT_RELEASE. This function either releases or rolls back the
................................................................................
int sqlite3BtreeSavepoint(Btree *p, int op, int iSavepoint){
  int rc = SQLITE_OK;
  if( p && p->inTrans==TRANS_WRITE ){
    BtShared *pBt = p->pBt;
    assert( op==SAVEPOINT_RELEASE || op==SAVEPOINT_ROLLBACK );
    assert( iSavepoint>=0 || (iSavepoint==-1 && op==SAVEPOINT_ROLLBACK) );
    sqlite3BtreeEnter(p);
    btreePtrmapEnd(pBt, op, iSavepoint);
    if( op==SAVEPOINT_ROLLBACK ){
      rc = saveAllCursors(pBt, 0, 0);
    }
    if( rc==SQLITE_OK ){
      rc = sqlite3PagerSavepoint(pBt->pPager, op, iSavepoint);
    }
    if( rc==SQLITE_OK ){
................................................................................
** This function returns SQLITE_CORRUPT if the page-header flags field of
** the new child page does not match the flags field of the parent (i.e.
** if an intkey page appears to be the parent of a non-intkey page, or
** vice-versa).
*/
static int moveToChild(BtCursor *pCur, u32 newPgno){
  BtShared *pBt = pCur->pBt;
  int rc;

  assert( cursorOwnsBtShared(pCur) );
  assert( pCur->eState==CURSOR_VALID );
  assert( pCur->iPage<BTCURSOR_MAX_DEPTH );
  assert( pCur->iPage>=0 );
  if( pCur->iPage>=(BTCURSOR_MAX_DEPTH-1) ){
    return SQLITE_CORRUPT_BKPT;
................................................................................
  }
  pCur->info.nSize = 0;
  pCur->curFlags &= ~(BTCF_ValidNKey|BTCF_ValidOvfl);
  pCur->aiIdx[pCur->iPage] = pCur->ix;
  pCur->apPage[pCur->iPage] = pCur->pPage;
  pCur->ix = 0;
  pCur->iPage++;
  rc = getAndInitPage(pBt, newPgno, &pCur->pPage,
                        pCur, pCur->curPagerFlags);
  if( rc==SQLITE_OK ){
    setMempageRoot(pCur->pPage, pCur->pgnoRoot);
  }
  return rc;
}

#ifdef SQLITE_DEBUG
/*
** Page pParent is an internal (non-leaf) tree page. This function 
** asserts that page number iChild is the left-child if the iIdx'th
** cell in page pParent. Or, if iIdx is equal to the total number of
................................................................................
    }
    rc = getAndInitPage(pCur->pBtree->pBt, pCur->pgnoRoot, &pCur->pPage,
                        0, pCur->curPagerFlags);
    if( rc!=SQLITE_OK ){
      pCur->eState = CURSOR_INVALID;
      return rc;
    }
    setMempageRoot(pCur->pPage, pCur->pgnoRoot);
    pCur->iPage = 0;
    pCur->curIntKey = pCur->pPage->intKey;
  }
  pRoot = pCur->pPage;
  assert( pRoot->pgno==pCur->pgnoRoot );

  /* If pCur->pKeyInfo is not NULL, then the caller that opened this cursor
................................................................................
  u32 n;     /* Number of pages on the freelist */
  u32 k;     /* Number of leaves on the trunk of the freelist */
  MemPage *pTrunk = 0;
  MemPage *pPrevTrunk = 0;
  Pgno mxPage;     /* Total size of the database file */

  assert( sqlite3_mutex_held(pBt->mutex) );
  assert( eMode==BTALLOC_ANY || (nearby>0 && REQUIRE_PTRMAP ) );
  pPage1 = pBt->pPage1;
  mxPage = btreePagecount(pBt);
  /* EVIDENCE-OF: R-05119-02637 The 4-byte big-endian integer at offset 36
  ** stores stores the total number of pages on the freelist. */
  n = get4byte(&pPage1->aData[36]);
  testcase( n==mxPage-1 );
  if( n>=mxPage ){
    return SQLITE_CORRUPT_BKPT;
  }

  /* Ensure page 1 is writable. This function will either change the number
  ** of pages in the free-list or the size of the database file. Since both
  ** of these operations involve modifying page 1 header fields, page 1
  ** will definitely be written by this transaction. If this is an CONCURRENT
  ** transaction, ensure the BtreePtrmap structure has been allocated.  */
  rc = sqlite3PagerWrite(pPage1->pDbPage);
  if( rc ) return rc;

  if( n>0 ){
    /* There are pages on the freelist.  Reuse one of those pages. */
    Pgno iTrunk;
    u8 searchList = 0; /* If the free-list must be searched for 'nearby' */
    u32 nSearch = 0;   /* Count of the number of search attempts */
    
    /* If eMode==BTALLOC_EXACT and a query of the pointer-map
    ** shows that the page 'nearby' is somewhere on the free-list, then
    ** the entire-list will be searched for that page.
    */

    if( eMode==BTALLOC_EXACT ){
      assert( ISAUTOVACUUM!=ISCONCURRENT );
      if( ISAUTOVACUUM ){
        if( nearby<=mxPage ){
          u8 eType;
          assert( nearby>0 );
          assert( pBt->autoVacuum );
          rc = ptrmapGet(pBt, nearby, &eType, 0);
          if( rc ) return rc;
          if( eType==PTRMAP_FREEPAGE ){
            searchList = 1;
          }
        }
      }else{
        searchList = 1;
      }
    }else if( eMode==BTALLOC_LE ){
      searchList = 1;
    }


    /* Decrement the free-list count by 1. Set iTrunk to the index of the
    ** first free-list trunk page. iPrevTrunk is initially 1.
    */


    put4byte(&pPage1->aData[36], n-1);

    /* The code within this loop is run only once if the 'searchList' variable
    ** is not true. Otherwise, it runs once for each trunk-page on the
    ** free-list until the page 'nearby' is located (eMode==BTALLOC_EXACT)
    ** or until a page less than 'nearby' is located (eMode==BTALLOC_LT)
    */
................................................................................
    }
    memset(pPage->aData, 0, pPage->pBt->pageSize);
  }

  /* If the database supports auto-vacuum, write an entry in the pointer-map
  ** to indicate that the page is free.
  */
  if( REQUIRE_PTRMAP ){
    ptrmapPut(pBt, iPage, PTRMAP_FREEPAGE, 0, &rc);
    if( rc ) goto freepage_out;
  }

  /* Now manipulate the actual database free-list structure. There are two
  ** possibilities. If the free-list is currently empty, or if the first
  ** trunk page in the free-list is full, then this page will become a
................................................................................
          pgnoOvfl++;
        } while( 
          PTRMAP_ISPAGE(pBt, pgnoOvfl) || pgnoOvfl==PENDING_BYTE_PAGE(pBt) 
        );
      }
#endif
      rc = allocateBtreePage(pBt, &pOvfl, &pgnoOvfl, pgnoOvfl, 0);

      /* If the database supports auto-vacuum, and the second or subsequent
      ** overflow page is being allocated, add an entry to the pointer-map
      ** for that page now. 
      **
      ** If this is the first overflow page, then write a partial entry 
      ** to the pointer-map. If we write nothing to this pointer-map slot,
      ** then the optimistic overflow chain processing in clearCell()
      ** may misinterpret the uninitialized values and delete the
      ** wrong pages from the database.
      */
      if( REQUIRE_PTRMAP && rc==SQLITE_OK ){
        u8 eType = (pgnoPtrmap?PTRMAP_OVERFLOW2:PTRMAP_OVERFLOW1);
        ptrmapPut(pBt, pgnoOvfl, eType, pgnoPtrmap, &rc);
        if( rc ){
          releasePage(pOvfl);
        }
      }

      if( rc ){
        releasePage(pToRelease);
        return rc;
      }

      /* If pToRelease is not zero than pPrior points into the data area
      ** of pToRelease.  Make sure pToRelease is still writeable. */
................................................................................
    ** sorted order.  This invariants arise because multiple overflows can
    ** only occur when inserting divider cells into the parent page during
    ** balancing, and the dividers are adjacent and sorted.
    */
    assert( j==0 || pPage->aiOvfl[j-1]<(u16)i ); /* Overflows in sorted order */
    assert( j==0 || i==pPage->aiOvfl[j-1]+1 );   /* Overflows are sequential */
  }else{
    BtShared *pBt = pPage->pBt;
    int rc = sqlite3PagerWrite(pPage->pDbPage);
    if( rc!=SQLITE_OK ){
      *pRC = rc;
      return;
    }
    assert( sqlite3PagerIswriteable(pPage->pDbPage) );
    data = pPage->aData;
................................................................................
    assert( &data[pPage->cellOffset]==pPage->aCellIdx );
    rc = allocateSpace(pPage, sz, &idx);
    if( rc ){ *pRC = rc; return; }
    /* The allocateSpace() routine guarantees the following properties
    ** if it returns successfully */
    assert( idx >= 0 );
    assert( idx >= pPage->cellOffset+2*pPage->nCell+2 || CORRUPT_DB );
    assert( idx+sz <= (int)pBt->usableSize );
    pPage->nFree -= (u16)(2 + sz);
    if( iChild ){
      /* In a corrupt database where an entry in the cell index section of
      ** a btree page has a value of 3 or less, the pCell value might point
      ** as many as 4 bytes in front of the start of the aData buffer for
      ** the source page.  Make sure this does not cause problems by not
      ** reading the first 4 bytes */
................................................................................
    pIns = pPage->aCellIdx + i*2;
    memmove(pIns+2, pIns, 2*(pPage->nCell - i));
    put2byte(pIns, idx);
    pPage->nCell++;
    /* increment the cell count */
    if( (++data[pPage->hdrOffset+4])==0 ) data[pPage->hdrOffset+3]++;
    assert( get2byte(&data[pPage->hdrOffset+3])==pPage->nCell || CORRUPT_DB );
    if( REQUIRE_PTRMAP ){

      /* The cell may contain a pointer to an overflow page. If so, write
      ** the entry for the overflow page into the pointer map.
      */
      ptrmapPutOvflPtr(pPage, pPage, pCell, pRC);
    }

  }
}

/*
** The following parameters determine how many adjacent pages get involved
** in a balancing operation.  NN is the number of neighbors on either side
** of the page that participate in the balancing operation.  NB is the
................................................................................
    ** cell on the page to an overflow page. If either of these
    ** operations fails, the return code is set, but the contents
    ** of the parent page are still manipulated by thh code below.
    ** That is Ok, at this point the parent page is guaranteed to
    ** be marked as dirty. Returning an error code will cause a
    ** rollback, undoing any changes made to the parent page.
    */
    if( REQUIRE_PTRMAP ){
      ptrmapPut(pBt, pgnoNew, PTRMAP_BTREE, pParent->pgno, &rc);
      if( szCell>pNew->minLocal ){
        ptrmapPutOvflPtr(pNew, pNew, pCell, &rc);
      }
    }
  
    /* Create a divider cell to insert into pParent. The divider cell
................................................................................
      *pRC = rc;
      return;
    }
  
    /* If this is an auto-vacuum database, update the pointer-map entries
    ** for any b-tree or overflow pages that pTo now contains the pointers to.
    */
    if( REQUIRE_PTRMAP ){
      *pRC = setChildPtrmaps(pTo);
    }
  }
}

/*
** This routine redistributes cells on the iParentIdx'th child of pParent
................................................................................
** SQLITE_NOMEM.
*/
static int balance_nonroot(
  MemPage *pParent,               /* Parent page of siblings being balanced */
  int iParentIdx,                 /* Index of "the page" in pParent */
  u8 *aOvflSpace,                 /* page-size bytes of space for parent ovfl */
  int isRoot,                     /* True if pParent is a root-page */
  int bBulk,                      /* True if this call is part of a bulk load */
  Pgno pgnoRoot                   /* Root page of b-tree being balanced */
){
  BtShared *pBt;               /* The whole database */
  int nMaxCells = 0;           /* Allocated size of apCell, szCell, aFrom. */
  int nNew = 0;                /* Number of pages in apNew[] */
  int nOld;                    /* Number of pages in apOld[] */
  int i, j, k;                 /* Loop counters */
  int nxDiv;                   /* Next divider slot in pParent->aCell[] */
................................................................................
  pgno = get4byte(pRight);
  while( 1 ){
    rc = getAndInitPage(pBt, pgno, &apOld[i], 0, 0);
    if( rc ){
      memset(apOld, 0, (i+1)*sizeof(MemPage*));
      goto balance_cleanup;
    }
    setMempageRoot(apOld[i], pgnoRoot);
    if( apOld[i]->nFree<0 ){
      rc = btreeComputeFreeSpace(apOld[i]);
      if( rc ){
        memset(apOld, 0, (i)*sizeof(MemPage*));
        goto balance_cleanup;
      }
    }
................................................................................
      if( rc ) goto balance_cleanup;
      zeroPage(pNew, pageFlags);
      apNew[i] = pNew;
      nNew++;
      cntOld[i] = b.nCell;

      /* Set the pointer-map entry for the new sibling page. */
      if( REQUIRE_PTRMAP ){
        ptrmapPut(pBt, pNew->pgno, PTRMAP_BTREE, pParent->pgno, &rc);
        if( rc!=SQLITE_OK ){
          goto balance_cleanup;
        }
      }
    }
  }
................................................................................
  **      with the cell.
  **
  ** If the sibling pages are not leaves, then the pointer map entry 
  ** associated with the right-child of each sibling may also need to be 
  ** updated. This happens below, after the sibling pages have been 
  ** populated, not here.
  */
  if( REQUIRE_PTRMAP ){
    MemPage *pOld;
    MemPage *pNew = pOld = apNew[0];
    int cntOldNext = pNew->nCell + pNew->nOverflow;
    int iNew = 0;
    int iOld = 0;

    for(i=0; i<b.nCell; i++){
................................................................................
    assert( apNew[0]->nFree == 
        (get2byteNotZero(&apNew[0]->aData[5]) - apNew[0]->cellOffset
          - apNew[0]->nCell*2)
      || rc!=SQLITE_OK
    );
    copyNodeContent(apNew[0], pParent, &rc);
    freePage(apNew[0], &rc);
  }else if( REQUIRE_PTRMAP && !leafCorrection ){
    /* Fix the pointer map entries associated with the right-child of each
    ** sibling page. All other pointer map entries have already been taken
    ** care of.  */
    for(i=0; i<nNew; i++){
      u32 key = get4byte(&apNew[i]->aData[8]);
      ptrmapPut(pBt, key, PTRMAP_BTREE, apNew[i]->pgno, &rc);
    }
................................................................................
  ** page that will become the new right-child of pPage. Copy the contents
  ** of the node stored on pRoot into the new child page.
  */
  rc = sqlite3PagerWrite(pRoot->pDbPage);
  if( rc==SQLITE_OK ){
    rc = allocateBtreePage(pBt,&pChild,&pgnoChild,pRoot->pgno,0);
    copyNodeContent(pRoot, pChild, &rc);
    if( REQUIRE_PTRMAP ){
      ptrmapPut(pBt, pgnoChild, PTRMAP_BTREE, pRoot->pgno, &rc);
    }
  }
  if( rc ){
    *ppChild = 0;
    releasePage(pChild);
    return rc;
................................................................................
          ** has completed, it is safe to release the pSpace buffer used by
          ** the previous call, as the overflow cell data will have been 
          ** copied either into the body of a database page or into the new
          ** pSpace buffer passed to the latter call to balance_nonroot().
          */
          u8 *pSpace = sqlite3PageMalloc(pCur->pBt->pageSize);
          rc = balance_nonroot(pParent, iIdx, pSpace, iPage==1,
                               pCur->hints&BTREE_BULKLOAD, pCur->pgnoRoot);
          if( pFree ){
            /* If pFree is not NULL, it points to the pSpace buffer used 
            ** by a previous call to balance_nonroot(). Its contents are
            ** now stored either on real database pages or within the 
            ** new pSpace buffer, so it may be safely freed here. */
            sqlite3PageFree(pFree);
          }
................................................................................
    if( !pPage->leaf ){
      memcpy(newCell, oldCell, 4);
    }
    rc = clearCell(pPage, oldCell, &info);
    testcase( pCur->curFlags & BTCF_ValidOvfl );
    invalidateOverflowCache(pCur);
    if( info.nSize==szNew && info.nLocal==info.nPayload 
     && (!REQUIRE_PTRMAP || szNew<pPage->minLocal)
    ){
      /* Overwrite the old cell with the new if they are the same size.
      ** We could also try to do this if the old cell is smaller, then add
      ** the leftover space to the free list.  But experiments show that
      ** doing that is no faster then skipping this optimization and just
      ** calling dropCell() and insertCell(). 
      **
................................................................................
** Erase the given database page and all its children.  Return
** the page to the freelist.
*/
static int clearDatabasePage(
  BtShared *pBt,           /* The BTree that contains the table */
  Pgno pgno,               /* Page number to clear */
  int freePageFlag,        /* Deallocate page if true */
  int *pnChange,           /* Add number of Cells freed to this counter */
  Pgno pgnoRoot
){
  MemPage *pPage;
  int rc;
  unsigned char *pCell;
  int i;
  int hdr;
  CellInfo info;
................................................................................

  assert( sqlite3_mutex_held(pBt->mutex) );
  if( pgno>btreePagecount(pBt) ){
    return SQLITE_CORRUPT_BKPT;
  }
  rc = getAndInitPage(pBt, pgno, &pPage, 0, 0);
  if( rc ) return rc;
  setMempageRoot(pPage, pgnoRoot);
  if( pPage->bBusy ){
    rc = SQLITE_CORRUPT_BKPT;
    goto cleardatabasepage_out;
  }
  pPage->bBusy = 1;
  hdr = pPage->hdrOffset;
  for(i=0; i<pPage->nCell; i++){
    pCell = findCell(pPage, i);
    if( !pPage->leaf ){
      rc = clearDatabasePage(pBt, get4byte(pCell), 1, pnChange, pgnoRoot);
      if( rc ) goto cleardatabasepage_out;
    }
    rc = clearCell(pPage, pCell, &info);
    if( rc ) goto cleardatabasepage_out;
  }
  if( !pPage->leaf ){
    rc = clearDatabasePage(
        pBt, get4byte(&pPage->aData[hdr+8]), 1, pnChange, pgnoRoot
    );
    if( rc ) goto cleardatabasepage_out;
  }else if( pnChange ){
    assert( pPage->intKey || CORRUPT_DB );
    testcase( !pPage->intKey );
    *pnChange += pPage->nCell;
  }
  if( freePageFlag ){
................................................................................
  rc = saveAllCursors(pBt, (Pgno)iTable, 0);

  if( SQLITE_OK==rc ){
    /* Invalidate all incrblob cursors open on table iTable (assuming iTable
    ** is the root of a table b-tree - if it is not, the following call is
    ** a no-op).  */
    invalidateIncrblobCursors(p, (Pgno)iTable, 0, 1);
    rc = clearDatabasePage(pBt, (Pgno)iTable, 0, pnChange, (Pgno)iTable);
  }
  sqlite3BtreeLeave(p);
  return rc;
}

/*
** Delete all information from the single table that pCur is open on.
................................................................................
      checkPtrmap(&sCheck, aRoot[i], PTRMAP_ROOTPAGE, 0);
    }
#endif
    checkTreePage(&sCheck, aRoot[i], &notUsed, LARGEST_INT64);
  }
  pBt->db->flags = savedDbFlags;

  /* Make sure every page in the file is referenced. Skip this if the
  ** database is currently being written by a CONCURRENT transaction (it 
  ** may fail as pages that were part of the free-list when the transaction
  ** was opened cannot be counted).  */
  for(i=1; ISCONCURRENT==0 && i<=sCheck.nPage && sCheck.mxErr; i++){
#ifdef SQLITE_OMIT_AUTOVACUUM
    if( getPageReferenced(&sCheck, i)==0 ){
      checkAppendMsg(&sCheck, "Page %d is never used", i);
    }
#else
    /* If the database supports auto-vacuum, make sure no tables contain
    ** references to pointer-map pages.
................................................................................
  return (p->pBt->btsFlags & BTS_READ_ONLY)!=0;
}

/*
** Return the size of the header added to each page by this module.
*/
int sqlite3HeaderSizeBtree(void){ return ROUND8(sizeof(MemPage)); }

/*
** This function is called to ensure that all locks required to commit the
** current write-transaction to the database file are held. If the db is
** in rollback mode, this means the EXCLUSIVE lock on the database file.
**
** Or, if this is an CONCURRENT transaction on a wal-mode database, the WRITER
** lock on the wal file. In this case this function also checks that the
** CONCURRENT transaction can be safely committed (does not commit with any
** other transaction committed since it was opened).
**
** SQLITE_OK is returned if successful. SQLITE_BUSY if the required locks
** cannot be obtained due to a conflicting lock. If the locks cannot be
** obtained for an CONCURRENT transaction due to a conflict with an already
** committed transaction, SQLITE_BUSY_SNAPSHOT is returned. Otherwise, if
** some other error (OOM, IO, etc.) occurs, the relevant SQLite error code
** is returned.
*/
int sqlite3BtreeExclusiveLock(Btree *p){
  int rc;
  Pgno pgno = 0;
  BtShared *pBt = p->pBt;
  assert( p->inTrans==TRANS_WRITE && pBt->pPage1 );
  sqlite3BtreeEnter(p);
  rc = sqlite3PagerExclusiveLock(pBt->pPager, pBt->pPage1->pDbPage, &pgno);
#ifdef SQLITE_OMIT_CONCURRENT
  assert( pgno==0 );
#else
  if( rc==SQLITE_BUSY_SNAPSHOT && pgno ){
    PgHdr *pPg = 0;
    int rc2 = sqlite3PagerGet(pBt->pPager, pgno, &pPg, 0);
    if( rc2==SQLITE_OK ){
      int bWrite = -1;
      const char *zObj = 0;
      const char *zTab = 0;
      char zContent[17];

      if( pPg ){
        Pgno pgnoRoot = 0;
        HashElem *pE;
        Schema *pSchema;
        u8 *aData = (u8*)sqlite3PagerGetData(pPg);
        int i;
        for(i=0; i<8; i++){
          static const char hexdigits[] = {
            '0', '1', '2', '3', '4', '5', '6', '7',
            '8', '9', 'A', 'B', 'C', 'D', 'E', 'F' 
          };
          zContent[i*2] = hexdigits[(aData[i] >> 4)];
          zContent[i*2+1] = hexdigits[(aData[i] & 0xF)];
        }
        zContent[16] = '\0';

        pgnoRoot = ((MemPage*)sqlite3PagerGetExtra(pPg))->pgnoRoot;
        bWrite = sqlite3PagerIswriteable(pPg);
        sqlite3PagerUnref(pPg);

        pSchema = sqlite3SchemaGet(p->db, p);
        if( pSchema ){
          for(pE=sqliteHashFirst(&pSchema->tblHash); pE; pE=sqliteHashNext(pE)){
            Table *pTab = (Table *)sqliteHashData(pE);
            if( pTab->tnum==(int)pgnoRoot ){
              zObj = pTab->zName;
              zTab = 0;
            }else{
              Index *pIdx;
              for(pIdx=pTab->pIndex; pIdx; pIdx=pIdx->pNext){
                if( pIdx->tnum==(int)pgnoRoot ){
                  zObj = pIdx->zName;
                  zTab = pTab->zName;
                }
              }
            }
          }
        }
      }

      sqlite3_log(SQLITE_OK,
          "cannot commit CONCURRENT transaction "
          "- conflict at page %d "
          "(%s page; part of db %s %s%s%s; content=%s...)",
          (int)pgno,
          (bWrite==0?"read-only":(bWrite>0?"read/write":"unknown")),
          (zTab ? "index" : "table"),
          (zTab ? zTab : ""), (zTab ? "." : ""), (zObj ? zObj : "UNKNOWN"),
          zContent
      );
    }
  }
#endif
  sqlite3BtreeLeave(p);
  return rc;
}

#if !defined(SQLITE_OMIT_SHARED_CACHE)
/*
** Return true if the Btree passed as the only argument is sharable.
*/
int sqlite3BtreeSharable(Btree *p){
  return p->sharable;

Changes to src/btree.h.

327
328
329
330
331
332
333


334
335
336
337
338
339
340
...
384
385
386
387
388
389
390
391
392
393
#endif
void sqlite3BtreeClearCursor(BtCursor *);
int sqlite3BtreeSetVersion(Btree *pBt, int iVersion);
int sqlite3BtreeCursorHasHint(BtCursor*, unsigned int mask);
int sqlite3BtreeIsReadonly(Btree *pBt);
int sqlite3HeaderSizeBtree(void);



#ifndef NDEBUG
int sqlite3BtreeCursorIsValid(BtCursor*);
#endif
int sqlite3BtreeCursorIsValidNN(BtCursor*);

#ifndef SQLITE_OMIT_BTREECOUNT
int sqlite3BtreeCount(sqlite3*, BtCursor*, i64*);
................................................................................
# define sqlite3BtreeLeaveCursor(X)
# define sqlite3BtreeLeaveAll(X)

# define sqlite3BtreeHoldsMutex(X) 1
# define sqlite3BtreeHoldsAllMutexes(X) 1
# define sqlite3SchemaMutexHeld(X,Y,Z) 1
#endif


#endif /* SQLITE_BTREE_H */







>
>







 








<

327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
...
386
387
388
389
390
391
392
393

394
#endif
void sqlite3BtreeClearCursor(BtCursor *);
int sqlite3BtreeSetVersion(Btree *pBt, int iVersion);
int sqlite3BtreeCursorHasHint(BtCursor*, unsigned int mask);
int sqlite3BtreeIsReadonly(Btree *pBt);
int sqlite3HeaderSizeBtree(void);

int sqlite3BtreeExclusiveLock(Btree *pBt);

#ifndef NDEBUG
int sqlite3BtreeCursorIsValid(BtCursor*);
#endif
int sqlite3BtreeCursorIsValidNN(BtCursor*);

#ifndef SQLITE_OMIT_BTREECOUNT
int sqlite3BtreeCount(sqlite3*, BtCursor*, i64*);
................................................................................
# define sqlite3BtreeLeaveCursor(X)
# define sqlite3BtreeLeaveAll(X)

# define sqlite3BtreeHoldsMutex(X) 1
# define sqlite3BtreeHoldsAllMutexes(X) 1
# define sqlite3SchemaMutexHeld(X,Y,Z) 1
#endif


#endif /* SQLITE_BTREE_H */

Changes to src/btreeInt.h.

228
229
230
231
232
233
234

235
236
237
238
239
240
241
...
272
273
274
275
276
277
278



279
280
281
282
283
284
285
...
436
437
438
439
440
441
442



443
444
445
446
447
448
449
...
652
653
654
655
656
657
658
659
660
661
662
663
664







665
666
667
668
669
670
671
*/
#define MX_CELL(pBt) ((pBt->pageSize-8)/6)

/* Forward declarations */
typedef struct MemPage MemPage;
typedef struct BtLock BtLock;
typedef struct CellInfo CellInfo;


/*
** This is a magic string that appears at the beginning of every
** SQLite database in order to identify the file as a real database.
**
** You can change this value at compile-time by specifying a
** -DSQLITE_FILE_HEADER="..." on the compiler command-line.  The
................................................................................
*/
struct MemPage {
  u8 isInit;           /* True if previously initialized. MUST BE FIRST! */
  u8 bBusy;            /* Prevent endless loops on corrupt database files */
  u8 intKey;           /* True if table b-trees.  False for index b-trees */
  u8 intKeyLeaf;       /* True if the leaf of an intKey table */
  Pgno pgno;           /* Page number for this page */



  /* Only the first 8 bytes (above) are zeroed by pager.c when a new page
  ** is allocated. All fields that follow must be initialized before use */
  u8 leaf;             /* True if a leaf page */
  u8 hdrOffset;        /* 100 for page 1.  0 otherwise */
  u8 childPtrSize;     /* 0 if leaf==1.  4 if leaf==0 */
  u8 max1bytePayload;  /* min(maxLocal,127) */
  u8 nOverflow;        /* Number of overflow cell bodies in aCell[] */
................................................................................
#ifndef SQLITE_OMIT_SHARED_CACHE
  int nRef;             /* Number of references to this structure */
  BtShared *pNext;      /* Next on a list of sharable BtShared structs */
  BtLock *pLock;        /* List of locks held on this shared-btree struct */
  Btree *pWriter;       /* Btree with currently open write transaction */
#endif
  u8 *pTmpSpace;        /* Temp space sufficient to hold a single cell */



};

/*
** Allowed values for BtShared.btsFlags
*/
#define BTS_READ_ONLY        0x0001   /* Underlying file is readonly */
#define BTS_PAGESIZE_FIXED   0x0002   /* Page size can no longer be changed */
................................................................................
/*
** The ISAUTOVACUUM macro is used within balance_nonroot() to determine
** if the database supports auto-vacuum or not. Because it is used
** within an expression that is an argument to another macro 
** (sqliteMallocRaw), it is not possible to use conditional compilation.
** So, this macro is defined instead.
*/
#ifndef SQLITE_OMIT_AUTOVACUUM
#define ISAUTOVACUUM (pBt->autoVacuum)
#else
#define ISAUTOVACUUM 0
#endif









/*
** This structure is passed around through all the sanity checking routines
** in order to keep track of some global state information.
**
** The aRef[] array is allocated so that there is 1 bit for each page in
** the database. As the integrity-check proceeds, for each page used in







>







 







>
>
>







 







>
>
>







 







|
|

|


>
>
>
>
>
>
>







228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
...
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
...
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
...
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
*/
#define MX_CELL(pBt) ((pBt->pageSize-8)/6)

/* Forward declarations */
typedef struct MemPage MemPage;
typedef struct BtLock BtLock;
typedef struct CellInfo CellInfo;
typedef struct BtreePtrmap BtreePtrmap;

/*
** This is a magic string that appears at the beginning of every
** SQLite database in order to identify the file as a real database.
**
** You can change this value at compile-time by specifying a
** -DSQLITE_FILE_HEADER="..." on the compiler command-line.  The
................................................................................
*/
struct MemPage {
  u8 isInit;           /* True if previously initialized. MUST BE FIRST! */
  u8 bBusy;            /* Prevent endless loops on corrupt database files */
  u8 intKey;           /* True if table b-trees.  False for index b-trees */
  u8 intKeyLeaf;       /* True if the leaf of an intKey table */
  Pgno pgno;           /* Page number for this page */
#ifndef SQLITE_OMIT_CONCURRENT
  Pgno pgnoRoot;       /* Root page of b-tree that this page belongs to */
#endif
  /* Only the first 8 bytes (above) are zeroed by pager.c when a new page
  ** is allocated. All fields that follow must be initialized before use */
  u8 leaf;             /* True if a leaf page */
  u8 hdrOffset;        /* 100 for page 1.  0 otherwise */
  u8 childPtrSize;     /* 0 if leaf==1.  4 if leaf==0 */
  u8 max1bytePayload;  /* min(maxLocal,127) */
  u8 nOverflow;        /* Number of overflow cell bodies in aCell[] */
................................................................................
#ifndef SQLITE_OMIT_SHARED_CACHE
  int nRef;             /* Number of references to this structure */
  BtShared *pNext;      /* Next on a list of sharable BtShared structs */
  BtLock *pLock;        /* List of locks held on this shared-btree struct */
  Btree *pWriter;       /* Btree with currently open write transaction */
#endif
  u8 *pTmpSpace;        /* Temp space sufficient to hold a single cell */
#ifndef SQLITE_OMIT_CONCURRENT
  BtreePtrmap *pMap;
#endif
};

/*
** Allowed values for BtShared.btsFlags
*/
#define BTS_READ_ONLY        0x0001   /* Underlying file is readonly */
#define BTS_PAGESIZE_FIXED   0x0002   /* Page size can no longer be changed */
................................................................................
/*
** The ISAUTOVACUUM macro is used within balance_nonroot() to determine
** if the database supports auto-vacuum or not. Because it is used
** within an expression that is an argument to another macro 
** (sqliteMallocRaw), it is not possible to use conditional compilation.
** So, this macro is defined instead.
*/
#ifdef SQLITE_OMIT_AUTOVACUUM
#define ISAUTOVACUUM 0
#else
#define ISAUTOVACUUM (pBt->autoVacuum)
#endif

#ifdef SQLITE_OMIT_CONCURRENT
# define ISCONCURRENT 0
#else
# define ISCONCURRENT (pBt->pMap!=0)
#endif

#define REQUIRE_PTRMAP (ISAUTOVACUUM || ISCONCURRENT)

/*
** This structure is passed around through all the sanity checking routines
** in order to keep track of some global state information.
**
** The aRef[] array is allocated so that there is 1 bit for each page in
** the database. As the integrity-check proceeds, for each page used in

Changes to src/build.c.

55
56
57
58
59
60
61

62
63

64
65
66
67
68
69
70
....
4519
4520
4521
4522
4523
4524
4525
4526
4527
4528
4529
4530
4531
4532
4533
4534
4535
4536
4537
4538
4539
){
  Parse *pToplevel = sqlite3ParseToplevel(pParse);
  int i;
  int nBytes;
  TableLock *p;
  assert( iDb>=0 );


  if( iDb==1 ) return;
  if( !sqlite3BtreeSharable(pParse->db->aDb[iDb].pBt) ) return;

  for(i=0; i<pToplevel->nTableLock; i++){
    p = &pToplevel->aTableLock[i];
    if( p->iDb==iDb && p->iTab==iTab ){
      p->isWriteLock = (p->isWriteLock || isWriteLock);
      return;
    }
  }
................................................................................
  db = pParse->db;
  assert( db!=0 );
  if( sqlite3AuthCheck(pParse, SQLITE_TRANSACTION, "BEGIN", 0, 0) ){
    return;
  }
  v = sqlite3GetVdbe(pParse);
  if( !v ) return;
  if( type!=TK_DEFERRED ){
    for(i=0; i<db->nDb; i++){
      sqlite3VdbeAddOp2(v, OP_Transaction, i, (type==TK_EXCLUSIVE)+1);
      sqlite3VdbeUsesBtree(v, i);
    }
  }
  sqlite3VdbeAddOp0(v, OP_AutoCommit);
}

/*
** Generate VDBE code for a COMMIT or ROLLBACK statement.
** Code for ROLLBACK is generated if eType==TK_ROLLBACK.  Otherwise
** code is generated for a COMMIT.
*/







>


>







 







|





|







55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
....
4521
4522
4523
4524
4525
4526
4527
4528
4529
4530
4531
4532
4533
4534
4535
4536
4537
4538
4539
4540
4541
){
  Parse *pToplevel = sqlite3ParseToplevel(pParse);
  int i;
  int nBytes;
  TableLock *p;
  assert( iDb>=0 );

#ifdef SQLITE_OMIT_CONCURRENT
  if( iDb==1 ) return;
  if( !sqlite3BtreeSharable(pParse->db->aDb[iDb].pBt) ) return;
#endif
  for(i=0; i<pToplevel->nTableLock; i++){
    p = &pToplevel->aTableLock[i];
    if( p->iDb==iDb && p->iTab==iTab ){
      p->isWriteLock = (p->isWriteLock || isWriteLock);
      return;
    }
  }
................................................................................
  db = pParse->db;
  assert( db!=0 );
  if( sqlite3AuthCheck(pParse, SQLITE_TRANSACTION, "BEGIN", 0, 0) ){
    return;
  }
  v = sqlite3GetVdbe(pParse);
  if( !v ) return;
  if( type==TK_IMMEDIATE || type==TK_EXCLUSIVE ){
    for(i=0; i<db->nDb; i++){
      sqlite3VdbeAddOp2(v, OP_Transaction, i, (type==TK_EXCLUSIVE)+1);
      sqlite3VdbeUsesBtree(v, i);
    }
  }
  sqlite3VdbeAddOp3(v, OP_AutoCommit, 0, 0, (type==TK_CONCURRENT));
}

/*
** Generate VDBE code for a COMMIT or ROLLBACK statement.
** Code for ROLLBACK is generated if eType==TK_ROLLBACK.  Otherwise
** code is generated for a COMMIT.
*/

Changes to src/func.c.

507
508
509
510
511
512
513

514
515

516
517
518
519
520
521
522
...
534
535
536
537
538
539
540

541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
*/
static void randomFunc(
  sqlite3_context *context,
  int NotUsed,
  sqlite3_value **NotUsed2
){
  sqlite_int64 r;

  UNUSED_PARAMETER2(NotUsed, NotUsed2);
  sqlite3_randomness(sizeof(r), &r);

  if( r<0 ){
    /* We need to prevent a random number of 0x8000000000000000 
    ** (or -9223372036854775808) since when you do abs() of that
    ** number of you get the same value back again.  To do this
    ** in a way that is testable, mask the sign bit off of negative
    ** values, resulting in a positive value.  Then take the 
    ** 2s complement of that positive value.  The end result can
................................................................................
static void randomBlob(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
  sqlite3_int64 n;
  unsigned char *p;

  assert( argc==1 );
  UNUSED_PARAMETER(argc);
  n = sqlite3_value_int64(argv[0]);
  if( n<1 ){
    n = 1;
  }
  p = contextMalloc(context, n);
  if( p ){
    sqlite3_randomness(n, p);
    sqlite3_result_blob(context, (char*)p, n, sqlite3_free);
  }
}

/*
** Implementation of the last_insert_rowid() SQL function.  The return
** value is the same as the sqlite3_last_insert_rowid() API function.







>

<
>







 







>








|







507
508
509
510
511
512
513
514
515

516
517
518
519
520
521
522
523
...
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
*/
static void randomFunc(
  sqlite3_context *context,
  int NotUsed,
  sqlite3_value **NotUsed2
){
  sqlite_int64 r;
  sqlite3 *db = sqlite3_context_db_handle(context);
  UNUSED_PARAMETER2(NotUsed, NotUsed2);

  sqlite3FastRandomness(&db->sPrng, sizeof(r), &r);
  if( r<0 ){
    /* We need to prevent a random number of 0x8000000000000000 
    ** (or -9223372036854775808) since when you do abs() of that
    ** number of you get the same value back again.  To do this
    ** in a way that is testable, mask the sign bit off of negative
    ** values, resulting in a positive value.  Then take the 
    ** 2s complement of that positive value.  The end result can
................................................................................
static void randomBlob(
  sqlite3_context *context,
  int argc,
  sqlite3_value **argv
){
  sqlite3_int64 n;
  unsigned char *p;
  sqlite3 *db = sqlite3_context_db_handle(context);
  assert( argc==1 );
  UNUSED_PARAMETER(argc);
  n = sqlite3_value_int64(argv[0]);
  if( n<1 ){
    n = 1;
  }
  p = contextMalloc(context, n);
  if( p ){
    sqlite3FastRandomness(&db->sPrng, n, p);
    sqlite3_result_blob(context, (char*)p, n, sqlite3_free);
  }
}

/*
** Implementation of the last_insert_rowid() SQL function.  The return
** value is the same as the sqlite3_last_insert_rowid() API function.

Changes to src/main.c.

3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
....
4537
4538
4539
4540
4541
4542
4543





























4544
4545
4546
4547
4548
4549
4550
  sqlite3_mutex_enter(db->mutex);
  db->errMask = 0xff;
  db->nDb = 2;
  db->magic = SQLITE_MAGIC_BUSY;
  db->aDb = db->aDbStatic;
  db->lookaside.bDisable = 1;
  db->lookaside.sz = 0;

  assert( sizeof(db->aLimit)==sizeof(aHardLimit) );
  memcpy(db->aLimit, aHardLimit, sizeof(db->aLimit));
  db->aLimit[SQLITE_LIMIT_WORKER_THREADS] = SQLITE_DEFAULT_WORKER_THREADS;
  db->autoCommit = 1;
  db->nextAutovac = -1;
  db->szMmap = sqlite3GlobalConfig.szMmap;
  db->nextPagesize = 0;
................................................................................
/*
** Free a snapshot handle obtained from sqlite3_snapshot_get().
*/
void sqlite3_snapshot_free(sqlite3_snapshot *pSnapshot){
  sqlite3_free(pSnapshot);
}
#endif /* SQLITE_ENABLE_SNAPSHOT */






























#ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS
/*
** Given the name of a compile-time option, return true if that option
** was used and false if not.
**
** The name can optionally begin with "SQLITE_" but the "SQLITE_" prefix







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
....
4537
4538
4539
4540
4541
4542
4543
4544
4545
4546
4547
4548
4549
4550
4551
4552
4553
4554
4555
4556
4557
4558
4559
4560
4561
4562
4563
4564
4565
4566
4567
4568
4569
4570
4571
4572
4573
4574
4575
4576
4577
4578
4579
  sqlite3_mutex_enter(db->mutex);
  db->errMask = 0xff;
  db->nDb = 2;
  db->magic = SQLITE_MAGIC_BUSY;
  db->aDb = db->aDbStatic;
  db->lookaside.bDisable = 1;
  db->lookaside.sz = 0;
  sqlite3FastPrngInit(&db->sPrng);
  assert( sizeof(db->aLimit)==sizeof(aHardLimit) );
  memcpy(db->aLimit, aHardLimit, sizeof(db->aLimit));
  db->aLimit[SQLITE_LIMIT_WORKER_THREADS] = SQLITE_DEFAULT_WORKER_THREADS;
  db->autoCommit = 1;
  db->nextAutovac = -1;
  db->szMmap = sqlite3GlobalConfig.szMmap;
  db->nextPagesize = 0;
................................................................................
/*
** Free a snapshot handle obtained from sqlite3_snapshot_get().
*/
void sqlite3_snapshot_free(sqlite3_snapshot *pSnapshot){
  sqlite3_free(pSnapshot);
}
#endif /* SQLITE_ENABLE_SNAPSHOT */

SQLITE_EXPERIMENTAL int sqlite3_wal_info(
  sqlite3 *db, const char *zDb, 
  unsigned int *pnPrior, unsigned int *pnFrame
){
  int rc = SQLITE_OK;

#ifndef SQLITE_OMIT_WAL
  Btree *pBt;
  int iDb;

#ifdef SQLITE_ENABLE_API_ARMOR
  if( !sqlite3SafetyCheckOk(db) ){
    return SQLITE_MISUSE_BKPT;
  }
#endif

  sqlite3_mutex_enter(db->mutex);
  iDb = sqlite3FindDbName(db, zDb);
  if( iDb<0 ){
    return SQLITE_ERROR;
  }
  pBt = db->aDb[iDb].pBt;
  rc = sqlite3PagerWalInfo(sqlite3BtreePager(pBt), pnPrior, pnFrame);
  sqlite3_mutex_leave(db->mutex);
#endif   /* SQLITE_OMIT_WAL */

  return rc;
}

#ifndef SQLITE_OMIT_COMPILEOPTION_DIAGS
/*
** Given the name of a compile-time option, return true if that option
** was used and false if not.
**
** The name can optionally begin with "SQLITE_" but the "SQLITE_" prefix

Changes to src/os_unix.c.

1172
1173
1174
1175
1176
1177
1178




1179
1180
1181
1182
1183
1184
1185
....
1324
1325
1326
1327
1328
1329
1330







1331
1332
1333
1334
1335
1336
1337
....
2182
2183
2184
2185
2186
2187
2188








2189
2190
2191
2192
2193
2194
2195
....
4013
4014
4015
4016
4017
4018
4019



4020
4021
4022
4023
4024
4025
4026
....
4978
4979
4980
4981
4982
4983
4984



4985
4986
4987
4988
4989
4990
4991
....
5108
5109
5110
5111
5112
5113
5114






















5115
5116
5117
5118
5119
5120
5121
....
5547
5548
5549
5550
5551
5552
5553



5554
5555
5556
5557
5558
5559
5560
#if SQLITE_ENABLE_LOCKING_STYLE
  unsigned long long sharedByte;  /* for AFP simulated shared lock */
#endif
#if OS_VXWORKS
  sem_t *pSem;                    /* Named POSIX semaphore */
  char aSemName[MAX_PATHNAME+2];  /* Name of that semaphore */
#endif




};

/*
** A lists of all unixInodeInfo objects.
**
** Must hold unixBigLock in order to read or write this variable.
*/
................................................................................
  unixInodeInfo *pInode = pFile->pInode;
  assert( unixMutexHeld() );
  assert( unixFileMutexNotheld(pFile) );
  if( ALWAYS(pInode) ){
    pInode->nRef--;
    if( pInode->nRef==0 ){
      assert( pInode->pShmNode==0 );







      sqlite3_mutex_enter(pInode->pLockMutex);
      closePendingFds(pFile);
      sqlite3_mutex_leave(pInode->pLockMutex);
      if( pInode->pPrev ){
        assert( pInode->pPrev->pNext==pInode );
        pInode->pPrev->pNext = pInode->pNext;
      }else{
................................................................................
  return SQLITE_OK;
}

/*
** Close the file.
*/
static int nolockClose(sqlite3_file *id) {








  return closeUnixFile(id);
}

/******************* End of the no-op lock implementation *********************
******************************************************************************/

/******************************************************************************
................................................................................
      if( newLimit>0 && sizeof(size_t)<8 ){
        newLimit = (newLimit & 0x7FFFFFFF);
      }

      *(i64*)pArg = pFile->mmapSizeMax;
      if( newLimit>=0 && newLimit!=pFile->mmapSizeMax && pFile->nFetchOut==0 ){
        pFile->mmapSizeMax = newLimit;



        if( pFile->mmapSize>0 ){
          unixUnmapfile(pFile);
          rc = unixMapfile(pFile, -1);
        }
      }
      return rc;
    }
................................................................................

#if SQLITE_MAX_MMAP_SIZE>0
/*
** If it is currently memory mapped, unmap file pFd.
*/
static void unixUnmapfile(unixFile *pFd){
  assert( pFd->nFetchOut==0 );



  if( pFd->pMapRegion ){
    osMunmap(pFd->pMapRegion, pFd->mmapSizeActual);
    pFd->pMapRegion = 0;
    pFd->mmapSize = 0;
    pFd->mmapSizeActual = 0;
  }
}
................................................................................
      return SQLITE_IOERR_FSTAT;
    }
    nMap = statbuf.st_size;
  }
  if( nMap>pFd->mmapSizeMax ){
    nMap = pFd->mmapSizeMax;
  }























  assert( nMap>0 || (pFd->mmapSize==0 && pFd->pMapRegion==0) );
  if( nMap!=pFd->mmapSize ){
    unixRemapfile(pFd, nMap);
  }

  return SQLITE_OK;
................................................................................
#endif
  }

  if( pLockingStyle == &posixIoMethods
#if defined(__APPLE__) && SQLITE_ENABLE_LOCKING_STYLE
    || pLockingStyle == &nfsIoMethods
#endif



  ){
    unixEnterMutex();
    rc = findInodeInfo(pNew, &pNew->pInode);
    if( rc!=SQLITE_OK ){
      /* If an error occurred in findInodeInfo(), close the file descriptor
      ** immediately, before releasing the mutex. findInodeInfo() may fail
      ** in two scenarios:







>
>
>
>







 







>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>







 







>
>
>







 







>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>







1172
1173
1174
1175
1176
1177
1178
1179
1180
1181
1182
1183
1184
1185
1186
1187
1188
1189
....
1328
1329
1330
1331
1332
1333
1334
1335
1336
1337
1338
1339
1340
1341
1342
1343
1344
1345
1346
1347
1348
....
2193
2194
2195
2196
2197
2198
2199
2200
2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213
2214
....
4032
4033
4034
4035
4036
4037
4038
4039
4040
4041
4042
4043
4044
4045
4046
4047
4048
....
5000
5001
5002
5003
5004
5005
5006
5007
5008
5009
5010
5011
5012
5013
5014
5015
5016
....
5133
5134
5135
5136
5137
5138
5139
5140
5141
5142
5143
5144
5145
5146
5147
5148
5149
5150
5151
5152
5153
5154
5155
5156
5157
5158
5159
5160
5161
5162
5163
5164
5165
5166
5167
5168
....
5594
5595
5596
5597
5598
5599
5600
5601
5602
5603
5604
5605
5606
5607
5608
5609
5610
#if SQLITE_ENABLE_LOCKING_STYLE
  unsigned long long sharedByte;  /* for AFP simulated shared lock */
#endif
#if OS_VXWORKS
  sem_t *pSem;                    /* Named POSIX semaphore */
  char aSemName[MAX_PATHNAME+2];  /* Name of that semaphore */
#endif
#ifdef SQLITE_SHARED_MAPPING
  sqlite3_int64 nSharedMapping;   /* Size of mapped region in bytes */
  void *pSharedMapping;           /* Memory mapped region */
#endif
};

/*
** A lists of all unixInodeInfo objects.
**
** Must hold unixBigLock in order to read or write this variable.
*/
................................................................................
  unixInodeInfo *pInode = pFile->pInode;
  assert( unixMutexHeld() );
  assert( unixFileMutexNotheld(pFile) );
  if( ALWAYS(pInode) ){
    pInode->nRef--;
    if( pInode->nRef==0 ){
      assert( pInode->pShmNode==0 );
#ifdef SQLITE_SHARED_MAPPING
      if( pInode->pSharedMapping ){
        osMunmap(pInode->pSharedMapping, pInode->nSharedMapping);
        pInode->pSharedMapping = 0;
        pInode->nSharedMapping = 0;
      }
#endif
      sqlite3_mutex_enter(pInode->pLockMutex);
      closePendingFds(pFile);
      sqlite3_mutex_leave(pInode->pLockMutex);
      if( pInode->pPrev ){
        assert( pInode->pPrev->pNext==pInode );
        pInode->pPrev->pNext = pInode->pNext;
      }else{
................................................................................
  return SQLITE_OK;
}

/*
** Close the file.
*/
static int nolockClose(sqlite3_file *id) {
#ifdef SQLITE_SHARED_MAPPING
  unixFile *pFd = (unixFile*)id;
  if( pFd->pInode ){
    unixEnterMutex();
    releaseInodeInfo(pFd);
    unixLeaveMutex();
  }
#endif
  return closeUnixFile(id);
}

/******************* End of the no-op lock implementation *********************
******************************************************************************/

/******************************************************************************
................................................................................
      if( newLimit>0 && sizeof(size_t)<8 ){
        newLimit = (newLimit & 0x7FFFFFFF);
      }

      *(i64*)pArg = pFile->mmapSizeMax;
      if( newLimit>=0 && newLimit!=pFile->mmapSizeMax && pFile->nFetchOut==0 ){
        pFile->mmapSizeMax = newLimit;
#ifdef SQLITE_SHARED_MAPPING
        if( pFile->pInode==0 )
#endif
        if( pFile->mmapSize>0 ){
          unixUnmapfile(pFile);
          rc = unixMapfile(pFile, -1);
        }
      }
      return rc;
    }
................................................................................

#if SQLITE_MAX_MMAP_SIZE>0
/*
** If it is currently memory mapped, unmap file pFd.
*/
static void unixUnmapfile(unixFile *pFd){
  assert( pFd->nFetchOut==0 );
#ifdef SQLITE_SHARED_MAPPING
  if( pFd->pInode ) return;
#endif
  if( pFd->pMapRegion ){
    osMunmap(pFd->pMapRegion, pFd->mmapSizeActual);
    pFd->pMapRegion = 0;
    pFd->mmapSize = 0;
    pFd->mmapSizeActual = 0;
  }
}
................................................................................
      return SQLITE_IOERR_FSTAT;
    }
    nMap = statbuf.st_size;
  }
  if( nMap>pFd->mmapSizeMax ){
    nMap = pFd->mmapSizeMax;
  }

#ifdef SQLITE_SHARED_MAPPING
  if( pFd->pInode ){
    unixInodeInfo *pInode = pFd->pInode;
    if( pFd->pMapRegion ) return SQLITE_OK;
    unixEnterMutex();
    if( pInode->pSharedMapping==0 ){
      u8 *pNew = osMmap(0, nMap, PROT_READ, MAP_SHARED, pFd->h, 0);
      if( pNew==MAP_FAILED ){
        unixLogError(SQLITE_OK, "mmap", pFd->zPath);
        pFd->mmapSizeMax = 0;
      }else{
        pInode->pSharedMapping = pNew;
        pInode->nSharedMapping = nMap;
      }
    }
    pFd->pMapRegion = pInode->pSharedMapping;
    pFd->mmapSizeActual = pFd->mmapSize = pInode->nSharedMapping;
    unixLeaveMutex();
    return SQLITE_OK;
  }
#endif

  assert( nMap>0 || (pFd->mmapSize==0 && pFd->pMapRegion==0) );
  if( nMap!=pFd->mmapSize ){
    unixRemapfile(pFd, nMap);
  }

  return SQLITE_OK;
................................................................................
#endif
  }

  if( pLockingStyle == &posixIoMethods
#if defined(__APPLE__) && SQLITE_ENABLE_LOCKING_STYLE
    || pLockingStyle == &nfsIoMethods
#endif
#ifdef SQLITE_SHARED_MAPPING
    || pLockingStyle == &nolockIoMethods
#endif
  ){
    unixEnterMutex();
    rc = findInodeInfo(pNew, &pNew->pInode);
    if( rc!=SQLITE_OK ){
      /* If an error occurred in findInodeInfo(), close the file descriptor
      ** immediately, before releasing the mutex. findInodeInfo() may fail
      ** in two scenarios:

Changes to src/pager.c.

666
667
668
669
670
671
672



673
674
675
676
677
678
679
...
936
937
938
939
940
941
942

943

944
945
946
947
948
949
950
....
1821
1822
1823
1824
1825
1826
1827















































1828
1829
1830
1831
1832
1833
1834
....
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
....
2114
2115
2116
2117
2118
2119
2120
2121
2122
2123
2124
2125
2126
2127
2128
2129
....
3163
3164
3165
3166
3167
3168
3169
3170






3171










3172
3173
3174
3175
3176
3177
3178
....
3214
3215
3216
3217
3218
3219
3220


3221
3222
3223
3224
3225
3226
3227
....
4283
4284
4285
4286
4287
4288
4289
4290
4291
4292
4293
4294
4295
4296
4297
....
4641
4642
4643
4644
4645
4646
4647






4648
4649
4650
4651
4652
4653
4654
....
5537
5538
5539
5540
5541
5542
5543












5544
5545
5546
5547
5548
5549
5550
....
5879
5880
5881
5882
5883
5884
5885
5886
5887
5888
5889



5890
5891
5892
5893
5894
5895
5896
....
5901
5902
5903
5904
5905
5906
5907
5908
5909
5910
5911
5912
5913
5914
5915
....
5917
5918
5919
5920
5921
5922
5923
5924
5925

5926

5927
5928
5929
5930
5931
5932
5933
5934
5935
5936
5937
5938
5939
5940
5941
....
6227
6228
6229
6230
6231
6232
6233
6234
6235
6236
6237
6238
6239
6240
6241
....
6383
6384
6385
6386
6387
6388
6389
6390
6391
6392
6393
6394

6395
6396
6397
6398









6399
6400
6401
6402
6403
6404
6405
6406
6407
6408
6409
6410
6411














6412



6413
6414














































6415
6416
6417
6418
6419
6420
6421
....
7781
7782
7783
7784
7785
7786
7787





7788
7789
7790
7791
7792
7793
7794
  Pgno dbFileSize;            /* Number of pages in the database file */
  Pgno dbHintSize;            /* Value passed to FCNTL_SIZE_HINT call */
  int errCode;                /* One of several kinds of errors */
  int nRec;                   /* Pages journalled since last j-header written */
  u32 cksumInit;              /* Quasi-random value added to every checksum */
  u32 nSubRec;                /* Number of records written to sub-journal */
  Bitvec *pInJournal;         /* One bit for each page in the database file */



  sqlite3_file *fd;           /* File descriptor for database */
  sqlite3_file *jfd;          /* File descriptor for main journal */
  sqlite3_file *sjfd;         /* File descriptor for sub-journal */
  i64 journalOff;             /* Current write offset in the journal file */
  i64 journalHdr;             /* Byte offset to previous journal header */
  sqlite3_backup *pBackup;    /* Pointer to list of ongoing backup processes */
  PagerSavepoint *aSavepoint; /* Array of active savepoints */
................................................................................

    case PAGER_WRITER_LOCKED:
      assert( p->eLock!=UNKNOWN_LOCK );
      assert( pPager->errCode==SQLITE_OK );
      if( !pagerUseWal(pPager) ){
        assert( p->eLock>=RESERVED_LOCK );
      }

      assert( pPager->dbSize==pPager->dbOrigSize );

      assert( pPager->dbOrigSize==pPager->dbFileSize );
      assert( pPager->dbOrigSize==pPager->dbHintSize );
      assert( pPager->setMaster==0 );
      break;

    case PAGER_WRITER_CACHEMOD:
      assert( p->eLock!=UNKNOWN_LOCK );
................................................................................
      rc |= sqlite3BitvecSet(p->pInSavepoint, pgno);
      testcase( rc==SQLITE_NOMEM );
      assert( rc==SQLITE_OK || rc==SQLITE_NOMEM );
    }
  }
  return rc;
}
















































/*
** This function is a no-op if the pager is in exclusive mode and not
** in the ERROR state. Otherwise, it switches the pager to PAGER_OPEN
** state.
**
** If the pager is not in exclusive-access mode, the database file is
................................................................................
static void pager_unlock(Pager *pPager){

  assert( pPager->eState==PAGER_READER 
       || pPager->eState==PAGER_OPEN 
       || pPager->eState==PAGER_ERROR 
  );

  sqlite3BitvecDestroy(pPager->pInJournal);
  pPager->pInJournal = 0;
  releaseAllSavepoints(pPager);

  if( pagerUseWal(pPager) ){
    assert( !isOpen(pPager->jfd) );
    sqlite3WalEndReadTransaction(pPager->pWal);
    pPager->eState = PAGER_OPEN;
  }else if( !pPager->exclusiveMode ){
................................................................................
    if( p ){
      p->pageHash = 0;
      sqlite3PagerUnrefNotNull(p);
    }
  }
#endif

  sqlite3BitvecDestroy(pPager->pInJournal);
  pPager->pInJournal = 0;
  pPager->nRec = 0;
  if( rc==SQLITE_OK ){
    if( MEMDB || pagerFlushOnCommit(pPager, bCommit) ){
      sqlite3PcacheCleanAll(pPager->pPCache);
    }else{
      sqlite3PcacheClearWritable(pPager->pPCache);
    }
................................................................................
  ** been written (but not committed) to the log file, do one of the 
  ** following:
  **
  **   + Discard the cached page (if refcount==0), or
  **   + Reload page content from the database (if refcount>0).
  */
  pPager->dbSize = pPager->dbOrigSize;
  rc = sqlite3WalUndo(pPager->pWal, pagerUndoCallback, (void *)pPager);






  pList = sqlite3PcacheDirtyList(pPager->pPCache);










  while( pList && rc==SQLITE_OK ){
    PgHdr *pNext = pList->pDirty;
    rc = pagerUndoCallback((void *)pPager, pList->pgno);
    pList = pNext;
  }

  return rc;
................................................................................
    ** list here. */
    PgHdr **ppNext = &pList;
    nList = 0;
    for(p=pList; (*ppNext = p)!=0; p=p->pDirty){
      if( p->pgno<=nTruncate ){
        ppNext = &p->pDirty;
        nList++;


      }
    }
    assert( pList );
  }else{
    nList = 1;
  }
  pPager->aStat[PAGER_STAT_WRITE] += nList;
................................................................................

  assert( pPager->eState==PAGER_WRITER_CACHEMOD
       || pPager->eState==PAGER_WRITER_DBMOD
  );
  assert( assert_pager_state(pPager) );
  assert( !pagerUseWal(pPager) );

  rc = sqlite3PagerExclusiveLock(pPager);
  if( rc!=SQLITE_OK ) return rc;

  if( !pPager->noSync ){
    assert( !pPager->tempFile );
    if( isOpen(pPager->jfd) && pPager->journalMode!=PAGER_JOURNALMODE_MEMORY ){
      const int iDc = sqlite3OsDeviceCharacteristics(pPager->fd);
      assert( isOpen(pPager->jfd) );
................................................................................
  ){
    return SQLITE_OK;
  }

  pPager->aStat[PAGER_STAT_SPILL]++;
  pPg->pDirty = 0;
  if( pagerUseWal(pPager) ){






    /* Write a single frame for this page to the log. */
    rc = subjournalPageIfRequired(pPg); 
    if( rc==SQLITE_OK ){
      rc = pagerWalFrames(pPager, pPg, 0, 0);
    }
  }else{
    
................................................................................
  u8 noContent;                   /* True if PAGER_GET_NOCONTENT is set */
  sqlite3_pcache_page *pBase;

  assert( pPager->errCode==SQLITE_OK );
  assert( pPager->eState>=PAGER_READER );
  assert( assert_pager_state(pPager) );
  assert( pPager->hasHeldSharedLock==1 );













  if( pgno==0 ) return SQLITE_CORRUPT_BKPT;
  pBase = sqlite3PcacheFetch(pPager->pPCache, pgno, 3);
  if( pBase==0 ){
    pPg = 0;
    rc = sqlite3PcacheFetchStress(pPager->pPCache, pgno, &pBase);
    if( rc!=SQLITE_OK ) goto pager_acquire_err;
................................................................................
  return rc;
}

/*
** Begin a write-transaction on the specified pager object. If a 
** write-transaction has already been opened, this function is a no-op.
**
** If the exFlag argument is false, then acquire at least a RESERVED
** lock on the database file. If exFlag is true, then acquire at least
** an EXCLUSIVE lock. If such a lock is already held, no locking 
** functions need be called.



**
** If the subjInMemory argument is non-zero, then any sub-journal opened
** within this transaction will be opened as an in-memory file. This
** has no effect if the sub-journal is already opened (as it may be when
** running in exclusive mode) or if the transaction does not require a
** sub-journal. If the subjInMemory argument is zero, then any required
** sub-journal is implemented in-memory if pPager is an in-memory database, 
................................................................................

  if( pPager->errCode ) return pPager->errCode;
  assert( pPager->eState>=PAGER_READER && pPager->eState<PAGER_ERROR );
  pPager->subjInMemory = (u8)subjInMemory;

  if( ALWAYS(pPager->eState==PAGER_READER) ){
    assert( pPager->pInJournal==0 );

    if( pagerUseWal(pPager) ){
      /* If the pager is configured to use locking_mode=exclusive, and an
      ** exclusive lock on the database is not already held, obtain it now.
      */
      if( pPager->exclusiveMode && sqlite3WalExclusiveMode(pPager->pWal, -1) ){
        rc = pagerLockDb(pPager, EXCLUSIVE_LOCK);
        if( rc!=SQLITE_OK ){
................................................................................
        }
        (void)sqlite3WalExclusiveMode(pPager->pWal, 1);
      }

      /* Grab the write lock on the log file. If successful, upgrade to
      ** PAGER_RESERVED state. Otherwise, return an error code to the caller.
      ** The busy-handler is not invoked if another connection already
      ** holds the write-lock. If possible, the upper layer will call it.
      */

      rc = sqlite3WalBeginWriteTransaction(pPager->pWal);

    }else{
      /* Obtain a RESERVED lock on the database file. If the exFlag parameter
      ** is true, then immediately upgrade this to an EXCLUSIVE lock. The
      ** busy-handler callback can be used when upgrading to the EXCLUSIVE
      ** lock, but not when obtaining the RESERVED lock.
      */
      rc = pagerLockDb(pPager, RESERVED_LOCK);
      if( rc==SQLITE_OK && exFlag ){
        rc = pager_wait_on_lock(pPager, EXCLUSIVE_LOCK);
      }
    }

    if( rc==SQLITE_OK ){
      /* Change to WRITER_LOCKED state.
      **
................................................................................
}

/*
** Return TRUE if the page given in the argument was previously passed
** to sqlite3PagerWrite().  In other words, return TRUE if it is ok
** to change the content of the page.
*/
#ifndef NDEBUG
int sqlite3PagerIswriteable(DbPage *pPg){
  return pPg->flags & PGHDR_WRITEABLE;
}
#endif

/*
** A call to this routine tells the pager that it is not necessary to
................................................................................
    assert( !MEMDB );
    rc = sqlite3OsSync(pPager->fd, pPager->syncFlags);
  }
  return rc;
}

/*
** This function may only be called while a write-transaction is active in
** rollback. If the connection is in WAL mode, this call is a no-op. 
** Otherwise, if the connection does not already have an EXCLUSIVE lock on 
** the database file, an attempt is made to obtain one.
**

** If the EXCLUSIVE lock is already held or the attempt to obtain it is
** successful, or the connection is in WAL mode, SQLITE_OK is returned.
** Otherwise, either SQLITE_BUSY or an SQLITE_IOERR_XXX error code is 
** returned.









*/
int sqlite3PagerExclusiveLock(Pager *pPager){
  int rc = pPager->errCode;
  assert( assert_pager_state(pPager) );
  if( rc==SQLITE_OK ){
    assert( pPager->eState==PAGER_WRITER_CACHEMOD 
         || pPager->eState==PAGER_WRITER_DBMOD 
         || pPager->eState==PAGER_WRITER_LOCKED 
    );
    assert( assert_pager_state(pPager) );
    if( 0==pagerUseWal(pPager) ){
      rc = pager_wait_on_lock(pPager, EXCLUSIVE_LOCK);
    }














  }



  return rc;
}















































/*
** Sync the database file for the pager pPager. zMaster points to the name
** of a master journal file that should be written into the individual
** journal file. zMaster may be NULL, which is interpreted as no master
** journal (a single database transaction).
**
................................................................................
*/
void sqlite3PagerSnapshotUnlock(Pager *pPager){
  assert( pPager->pWal );
  sqlite3WalSnapshotUnlock(pPager->pWal);
}

#endif /* SQLITE_ENABLE_SNAPSHOT */





#endif /* !SQLITE_OMIT_WAL */

#ifdef SQLITE_ENABLE_ZIPVFS
/*
** A read-lock must be held on the pager when this function is called. If
** the pager is in WAL mode and the WAL file currently contains one or more
** frames, return the size in bytes of the page images stored within the







>
>
>







 







>
|
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
<







 







|
<







 







|
>
>
>
>
>
>

>
>
>
>
>
>
>
>
>
>







 







>
>







 







|







 







>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>







 







|
|


>
>
>







 







<







 







|
<
>
|
>







|







 







|







 







|
|
|
<

>
|
|
|
|
>
>
>
>
>
>
>
>
>

|











>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>







666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
...
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
....
1826
1827
1828
1829
1830
1831
1832
1833
1834
1835
1836
1837
1838
1839
1840
1841
1842
1843
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855
1856
1857
1858
1859
1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
....
1898
1899
1900
1901
1902
1903
1904
1905

1906
1907
1908
1909
1910
1911
1912
....
2165
2166
2167
2168
2169
2170
2171
2172

2173
2174
2175
2176
2177
2178
2179
....
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223
3224
3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237
3238
3239
3240
3241
3242
3243
3244
....
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
....
4351
4352
4353
4354
4355
4356
4357
4358
4359
4360
4361
4362
4363
4364
4365
....
4709
4710
4711
4712
4713
4714
4715
4716
4717
4718
4719
4720
4721
4722
4723
4724
4725
4726
4727
4728
....
5611
5612
5613
5614
5615
5616
5617
5618
5619
5620
5621
5622
5623
5624
5625
5626
5627
5628
5629
5630
5631
5632
5633
5634
5635
5636
....
5965
5966
5967
5968
5969
5970
5971
5972
5973
5974
5975
5976
5977
5978
5979
5980
5981
5982
5983
5984
5985
....
5990
5991
5992
5993
5994
5995
5996

5997
5998
5999
6000
6001
6002
6003
....
6005
6006
6007
6008
6009
6010
6011
6012

6013
6014
6015
6016
6017
6018
6019
6020
6021
6022
6023
6024
6025
6026
6027
6028
6029
6030
....
6316
6317
6318
6319
6320
6321
6322
6323
6324
6325
6326
6327
6328
6329
6330
....
6472
6473
6474
6475
6476
6477
6478
6479
6480
6481

6482
6483
6484
6485
6486
6487
6488
6489
6490
6491
6492
6493
6494
6495
6496
6497
6498
6499
6500
6501
6502
6503
6504
6505
6506
6507
6508
6509
6510
6511
6512
6513
6514
6515
6516
6517
6518
6519
6520
6521
6522
6523
6524
6525
6526
6527
6528
6529
6530
6531
6532
6533
6534
6535
6536
6537
6538
6539
6540
6541
6542
6543
6544
6545
6546
6547
6548
6549
6550
6551
6552
6553
6554
6555
6556
6557
6558
6559
6560
6561
6562
6563
6564
6565
6566
6567
6568
6569
6570
6571
6572
6573
6574
6575
6576
6577
6578
6579
6580
6581
6582
....
7942
7943
7944
7945
7946
7947
7948
7949
7950
7951
7952
7953
7954
7955
7956
7957
7958
7959
7960
  Pgno dbFileSize;            /* Number of pages in the database file */
  Pgno dbHintSize;            /* Value passed to FCNTL_SIZE_HINT call */
  int errCode;                /* One of several kinds of errors */
  int nRec;                   /* Pages journalled since last j-header written */
  u32 cksumInit;              /* Quasi-random value added to every checksum */
  u32 nSubRec;                /* Number of records written to sub-journal */
  Bitvec *pInJournal;         /* One bit for each page in the database file */
#ifndef SQLITE_OMIT_CONCURRENT
  Bitvec *pAllRead;           /* Pages read within current CONCURRENT trans. */
#endif
  sqlite3_file *fd;           /* File descriptor for database */
  sqlite3_file *jfd;          /* File descriptor for main journal */
  sqlite3_file *sjfd;         /* File descriptor for sub-journal */
  i64 journalOff;             /* Current write offset in the journal file */
  i64 journalHdr;             /* Byte offset to previous journal header */
  sqlite3_backup *pBackup;    /* Pointer to list of ongoing backup processes */
  PagerSavepoint *aSavepoint; /* Array of active savepoints */
................................................................................

    case PAGER_WRITER_LOCKED:
      assert( p->eLock!=UNKNOWN_LOCK );
      assert( pPager->errCode==SQLITE_OK );
      if( !pagerUseWal(pPager) ){
        assert( p->eLock>=RESERVED_LOCK );
      }
#ifndef SQLITE_OMIT_CONCURRENT
      assert( pPager->dbSize==pPager->dbOrigSize || pPager->pAllRead );
#endif
      assert( pPager->dbOrigSize==pPager->dbFileSize );
      assert( pPager->dbOrigSize==pPager->dbHintSize );
      assert( pPager->setMaster==0 );
      break;

    case PAGER_WRITER_CACHEMOD:
      assert( p->eLock!=UNKNOWN_LOCK );
................................................................................
      rc |= sqlite3BitvecSet(p->pInSavepoint, pgno);
      testcase( rc==SQLITE_NOMEM );
      assert( rc==SQLITE_OK || rc==SQLITE_NOMEM );
    }
  }
  return rc;
}

#ifndef SQLITE_OMIT_CONCURRENT
/*
** If they are not already, begin recording all pages read from the pager layer
** by the b-tree layer This is used by concurrent transactions. Return
** SQLITE_OK if successful, or an SQLite error code (SQLITE_NOMEM) if an error
** occurs.
*/
int sqlite3PagerBeginConcurrent(Pager *pPager){
  int rc = SQLITE_OK;
  if( pPager->pAllRead==0 ){
    pPager->pAllRead = sqlite3BitvecCreate(pPager->dbSize);
    pPager->dbOrigSize = pPager->dbSize;
    if( pPager->pAllRead==0 ){
      rc = SQLITE_NOMEM;
    }
  }
  return rc;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** Stop recording all pages read from the pager layer by the b-tree layer
** and discard any current records.
*/
void sqlite3PagerEndConcurrent(Pager *pPager){
  sqlite3BitvecDestroy(pPager->pAllRead);
  pPager->pAllRead = 0;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** Return true if the database is in wal mode. False otherwise.
*/
int sqlite3PagerIsWal(Pager *pPager){
  return pPager->pWal!=0;
}
#endif /* SQLITE_OMIT_CONCURRENT */

/*
** Free the Pager.pInJournal and Pager.pAllRead bitvec objects.
*/
static void pagerFreeBitvecs(Pager *pPager){
  sqlite3BitvecDestroy(pPager->pInJournal);
  pPager->pInJournal = 0;
  sqlite3PagerEndConcurrent(pPager);
}

/*
** This function is a no-op if the pager is in exclusive mode and not
** in the ERROR state. Otherwise, it switches the pager to PAGER_OPEN
** state.
**
** If the pager is not in exclusive-access mode, the database file is
................................................................................
static void pager_unlock(Pager *pPager){

  assert( pPager->eState==PAGER_READER 
       || pPager->eState==PAGER_OPEN 
       || pPager->eState==PAGER_ERROR 
  );

  pagerFreeBitvecs(pPager);

  releaseAllSavepoints(pPager);

  if( pagerUseWal(pPager) ){
    assert( !isOpen(pPager->jfd) );
    sqlite3WalEndReadTransaction(pPager->pWal);
    pPager->eState = PAGER_OPEN;
  }else if( !pPager->exclusiveMode ){
................................................................................
    if( p ){
      p->pageHash = 0;
      sqlite3PagerUnrefNotNull(p);
    }
  }
#endif

  pagerFreeBitvecs(pPager);

  pPager->nRec = 0;
  if( rc==SQLITE_OK ){
    if( MEMDB || pagerFlushOnCommit(pPager, bCommit) ){
      sqlite3PcacheCleanAll(pPager->pPCache);
    }else{
      sqlite3PcacheClearWritable(pPager->pPCache);
    }
................................................................................
  ** been written (but not committed) to the log file, do one of the 
  ** following:
  **
  **   + Discard the cached page (if refcount==0), or
  **   + Reload page content from the database (if refcount>0).
  */
  pPager->dbSize = pPager->dbOrigSize;
  rc = sqlite3WalUndo(pPager->pWal, pagerUndoCallback, (void *)pPager, 
#ifdef SQLITE_OMIT_CONCURRENT
      0
#else
      pPager->pAllRead!=0
#endif
  );
  pList = sqlite3PcacheDirtyList(pPager->pPCache);

#ifndef SQLITE_OMIT_CONCURRENT
  /* If this is an CONCURRENT transaction, then page 1 must be reread from 
  ** the db file, even if it is not dirty. This is because the b-tree layer 
  ** may have already zeroed the nFree and iTrunk header fields.  */
  if( rc==SQLITE_OK && (pList==0 || pList->pgno!=1) && pPager->pAllRead ){
    rc = pagerUndoCallback((void*)pPager, 1);
  }
#endif

  while( pList && rc==SQLITE_OK ){
    PgHdr *pNext = pList->pDirty;
    rc = pagerUndoCallback((void *)pPager, pList->pgno);
    pList = pNext;
  }

  return rc;
................................................................................
    ** list here. */
    PgHdr **ppNext = &pList;
    nList = 0;
    for(p=pList; (*ppNext = p)!=0; p=p->pDirty){
      if( p->pgno<=nTruncate ){
        ppNext = &p->pDirty;
        nList++;
        PAGERTRACE(("TO-WAL %d page %d hash(%08x)\n",
                     PAGERID(pPager), p->pgno, pager_pagehash(p)));
      }
    }
    assert( pList );
  }else{
    nList = 1;
  }
  pPager->aStat[PAGER_STAT_WRITE] += nList;
................................................................................

  assert( pPager->eState==PAGER_WRITER_CACHEMOD
       || pPager->eState==PAGER_WRITER_DBMOD
  );
  assert( assert_pager_state(pPager) );
  assert( !pagerUseWal(pPager) );

  rc = sqlite3PagerExclusiveLock(pPager, 0, 0);
  if( rc!=SQLITE_OK ) return rc;

  if( !pPager->noSync ){
    assert( !pPager->tempFile );
    if( isOpen(pPager->jfd) && pPager->journalMode!=PAGER_JOURNALMODE_MEMORY ){
      const int iDc = sqlite3OsDeviceCharacteristics(pPager->fd);
      assert( isOpen(pPager->jfd) );
................................................................................
  ){
    return SQLITE_OK;
  }

  pPager->aStat[PAGER_STAT_SPILL]++;
  pPg->pDirty = 0;
  if( pagerUseWal(pPager) ){
#ifndef SQLITE_OMIT_CONCURRENT
    /* If the transaction is a "BEGIN CONCURRENT" transaction, the page 
    ** cannot be flushed to disk. Return early in this case. */
    if( pPager->pAllRead ) return SQLITE_OK;
#endif

    /* Write a single frame for this page to the log. */
    rc = subjournalPageIfRequired(pPg); 
    if( rc==SQLITE_OK ){
      rc = pagerWalFrames(pPager, pPg, 0, 0);
    }
  }else{
    
................................................................................
  u8 noContent;                   /* True if PAGER_GET_NOCONTENT is set */
  sqlite3_pcache_page *pBase;

  assert( pPager->errCode==SQLITE_OK );
  assert( pPager->eState>=PAGER_READER );
  assert( assert_pager_state(pPager) );
  assert( pPager->hasHeldSharedLock==1 );

#ifndef SQLITE_OMIT_CONCURRENT
  /* If this is an CONCURRENT transaction and the page being read was
  ** present in the database file when the transaction was opened,
  ** mark it as read in the pAllRead vector.  */
  pPg = 0;
  if( pPager->pAllRead && pgno<=pPager->dbOrigSize ){
    PAGERTRACE(("USING page %d\n", pgno));
    rc = sqlite3BitvecSet(pPager->pAllRead, pgno);
    if( rc!=SQLITE_OK ) goto pager_acquire_err;
  }
#endif

  if( pgno==0 ) return SQLITE_CORRUPT_BKPT;
  pBase = sqlite3PcacheFetch(pPager->pPCache, pgno, 3);
  if( pBase==0 ){
    pPg = 0;
    rc = sqlite3PcacheFetchStress(pPager->pPCache, pgno, &pBase);
    if( rc!=SQLITE_OK ) goto pager_acquire_err;
................................................................................
  return rc;
}

/*
** Begin a write-transaction on the specified pager object. If a 
** write-transaction has already been opened, this function is a no-op.
**
** If the exFlag argument is 0, then acquire at least a RESERVED
** lock on the database file. If exFlag is >0, then acquire at least
** an EXCLUSIVE lock. If such a lock is already held, no locking 
** functions need be called.
**
** If (exFlag<0) and the database is in WAL mode, do not take any locks.
** The transaction will run in CONCURRENT mode instead.
**
** If the subjInMemory argument is non-zero, then any sub-journal opened
** within this transaction will be opened as an in-memory file. This
** has no effect if the sub-journal is already opened (as it may be when
** running in exclusive mode) or if the transaction does not require a
** sub-journal. If the subjInMemory argument is zero, then any required
** sub-journal is implemented in-memory if pPager is an in-memory database, 
................................................................................

  if( pPager->errCode ) return pPager->errCode;
  assert( pPager->eState>=PAGER_READER && pPager->eState<PAGER_ERROR );
  pPager->subjInMemory = (u8)subjInMemory;

  if( ALWAYS(pPager->eState==PAGER_READER) ){
    assert( pPager->pInJournal==0 );

    if( pagerUseWal(pPager) ){
      /* If the pager is configured to use locking_mode=exclusive, and an
      ** exclusive lock on the database is not already held, obtain it now.
      */
      if( pPager->exclusiveMode && sqlite3WalExclusiveMode(pPager->pWal, -1) ){
        rc = pagerLockDb(pPager, EXCLUSIVE_LOCK);
        if( rc!=SQLITE_OK ){
................................................................................
        }
        (void)sqlite3WalExclusiveMode(pPager->pWal, 1);
      }

      /* Grab the write lock on the log file. If successful, upgrade to
      ** PAGER_RESERVED state. Otherwise, return an error code to the caller.
      ** The busy-handler is not invoked if another connection already
      ** holds the write-lock. If possible, the upper layer will call it.  */

      if( exFlag>=0 ){
        rc = sqlite3WalBeginWriteTransaction(pPager->pWal);
      }
    }else{
      /* Obtain a RESERVED lock on the database file. If the exFlag parameter
      ** is true, then immediately upgrade this to an EXCLUSIVE lock. The
      ** busy-handler callback can be used when upgrading to the EXCLUSIVE
      ** lock, but not when obtaining the RESERVED lock.
      */
      rc = pagerLockDb(pPager, RESERVED_LOCK);
      if( rc==SQLITE_OK && exFlag>0 ){
        rc = pager_wait_on_lock(pPager, EXCLUSIVE_LOCK);
      }
    }

    if( rc==SQLITE_OK ){
      /* Change to WRITER_LOCKED state.
      **
................................................................................
}

/*
** Return TRUE if the page given in the argument was previously passed
** to sqlite3PagerWrite().  In other words, return TRUE if it is ok
** to change the content of the page.
*/
#if !defined(SQLITE_OMIT_CONCURRENT) || !defined(NDEBUG)
int sqlite3PagerIswriteable(DbPage *pPg){
  return pPg->flags & PGHDR_WRITEABLE;
}
#endif

/*
** A call to this routine tells the pager that it is not necessary to
................................................................................
    assert( !MEMDB );
    rc = sqlite3OsSync(pPager->fd, pPager->syncFlags);
  }
  return rc;
}

/*
** This function is called to ensure that all locks required to commit the
** current write-transaction to the database file are held. If the db is
** in rollback mode, this means the EXCLUSIVE lock on the database file.

**
** Or, if this is a non-CONCURRENT transaction on a wal-mode database, this
** function is a no-op.
**
** If this is an CONCURRENT transaction on a wal-mode database, this function
** attempts to obtain the WRITER lock on the wal file and also checks to
** see that the transaction can be safely committed (does not commit with 
** any other transaction committed since it was opened).
**
** If the required locks are already held or successfully obtained and
** the transaction can be committed, SQLITE_OK is returned. If a required lock
** cannot be obtained, SQLITE_BUSY is returned. Or, if the current transaction
** is CONCURRENT and cannot be committed due to a conflict, SQLITE_BUSY_SNAPSHOT
** is returned. Otherwise, if some other error occurs (IO error, OOM etc.),
** and SQLite error code is returned.
*/
int sqlite3PagerExclusiveLock(Pager *pPager, PgHdr *pPage1, Pgno *piConflict){
  int rc = pPager->errCode;
  assert( assert_pager_state(pPager) );
  if( rc==SQLITE_OK ){
    assert( pPager->eState==PAGER_WRITER_CACHEMOD 
         || pPager->eState==PAGER_WRITER_DBMOD 
         || pPager->eState==PAGER_WRITER_LOCKED 
    );
    assert( assert_pager_state(pPager) );
    if( 0==pagerUseWal(pPager) ){
      rc = pager_wait_on_lock(pPager, EXCLUSIVE_LOCK);
    }
#ifndef SQLITE_OMIT_CONCURRENT
    else{
      if( pPager->pAllRead ){
        /* This is an CONCURRENT transaction. Attempt to lock the wal database
        ** here. If SQLITE_BUSY (but not SQLITE_BUSY_SNAPSHOT) is returned,
        ** invoke the busy-handler and try again for as long as it returns
        ** non-zero.  */
        do {
          rc = sqlite3WalLockForCommit(
              pPager->pWal, pPage1, pPager->pAllRead, piConflict
          );
        }while( rc==SQLITE_BUSY 
             && pPager->xBusyHandler(pPager->pBusyHandlerArg) 
        );
      }
    }
#endif /* SQLITE_OMIT_CONCURRENT */
  }
  return rc;
}

#ifndef SQLITE_OMIT_CONCURRENT
/*
** This function is called as part of committing an CONCURRENT transaction.
** At this point the wal WRITER lock is held, and all pages in the cache 
** except for page 1 are compatible with the snapshot at the head of the
** wal file. 
**
** This function updates the in-memory data structures and reloads the
** contents of page 1 so that the client is operating on the snapshot 
** at the head of the wal file.
**
** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
*/
int sqlite3PagerUpgradeSnapshot(Pager *pPager, DbPage *pPage1){
  int rc;

  assert( pPager->pWal && pPager->pAllRead );
  rc = sqlite3WalUpgradeSnapshot(pPager->pWal);
  if( rc==SQLITE_OK ){
    rc = readDbPage(pPage1);
  }

  return rc;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** Set the in-memory cache of the database file size to nSz pages.
*/
void sqlite3PagerSetDbsize(Pager *pPager, Pgno nSz){
  pPager->dbSize = nSz;
}

/* !defined(SQLITE_OMIT_CONCURRENT)
**
** If this is a WAL mode connection and the WRITER lock is currently held,
** relinquish it.
*/
void sqlite3PagerDropExclusiveLock(Pager *pPager){
  if( pagerUseWal(pPager) ){
    sqlite3WalEndWriteTransaction(pPager->pWal);
  }
}
#endif  /* SQLITE_OMIT_CONCURRENT */


/*
** Sync the database file for the pager pPager. zMaster points to the name
** of a master journal file that should be written into the individual
** journal file. zMaster may be NULL, which is interpreted as no master
** journal (a single database transaction).
**
................................................................................
*/
void sqlite3PagerSnapshotUnlock(Pager *pPager){
  assert( pPager->pWal );
  sqlite3WalSnapshotUnlock(pPager->pWal);
}

#endif /* SQLITE_ENABLE_SNAPSHOT */

int sqlite3PagerWalInfo(Pager *pPager, u32 *pnPrior, u32 *pnFrame){
  return sqlite3WalInfo(pPager->pWal, pnPrior, pnFrame);
}

#endif /* !SQLITE_OMIT_WAL */

#ifdef SQLITE_ENABLE_ZIPVFS
/*
** A read-lock must be held on the pager when this function is called. If
** the pager is in WAL mode and the WAL file currently contains one or more
** frames, return the size in bytes of the page images stored within the

Changes to src/pager.h.

161
162
163
164
165
166
167
168
169
170
171
172
173
174

175
176
177
178
179
180
181
...
220
221
222
223
224
225
226

















227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
void *sqlite3PagerGetData(DbPage *); 
void *sqlite3PagerGetExtra(DbPage *); 

/* Functions used to manage pager transactions and savepoints. */
void sqlite3PagerPagecount(Pager*, int*);
int sqlite3PagerBegin(Pager*, int exFlag, int);
int sqlite3PagerCommitPhaseOne(Pager*,const char *zMaster, int);
int sqlite3PagerExclusiveLock(Pager*);
int sqlite3PagerSync(Pager *pPager, const char *zMaster);
int sqlite3PagerCommitPhaseTwo(Pager*);
int sqlite3PagerRollback(Pager*);
int sqlite3PagerOpenSavepoint(Pager *pPager, int n);
int sqlite3PagerSavepoint(Pager *pPager, int op, int iSavepoint);
int sqlite3PagerSharedLock(Pager *pPager);


#ifndef SQLITE_OMIT_WAL
  int sqlite3PagerCheckpoint(Pager *pPager, sqlite3*, int, int*, int*);
  int sqlite3PagerWalSupported(Pager *pPager);
  int sqlite3PagerWalCallback(Pager *pPager);
  int sqlite3PagerOpenWal(Pager *pPager, int *pisOpen);
  int sqlite3PagerCloseWal(Pager *pPager, sqlite3*);
................................................................................
#endif

/* Functions used to truncate the database file. */
void sqlite3PagerTruncateImage(Pager*,Pgno);

void sqlite3PagerRekey(DbPage*, Pgno, u16);


















#if defined(SQLITE_HAS_CODEC) && !defined(SQLITE_OMIT_WAL)
void *sqlite3PagerCodec(DbPage *);
#endif

/* Functions to support testing and debugging. */
#if !defined(NDEBUG) || defined(SQLITE_TEST)
  Pgno sqlite3PagerPagenumber(DbPage*);
  int sqlite3PagerIswriteable(DbPage*);
#endif
#ifdef SQLITE_TEST
  int *sqlite3PagerStats(Pager*);
  void sqlite3PagerRefdump(Pager*);
  void disable_simulated_io_errors(void);
  void enable_simulated_io_errors(void);
#else
# define disable_simulated_io_errors()
# define enable_simulated_io_errors()
#endif

#endif /* SQLITE_PAGER_H */







|






>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







<












161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
...
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251

252
253
254
255
256
257
258
259
260
261
262
263
void *sqlite3PagerGetData(DbPage *); 
void *sqlite3PagerGetExtra(DbPage *); 

/* Functions used to manage pager transactions and savepoints. */
void sqlite3PagerPagecount(Pager*, int*);
int sqlite3PagerBegin(Pager*, int exFlag, int);
int sqlite3PagerCommitPhaseOne(Pager*,const char *zMaster, int);
int sqlite3PagerExclusiveLock(Pager*, DbPage *pPage1, Pgno*);
int sqlite3PagerSync(Pager *pPager, const char *zMaster);
int sqlite3PagerCommitPhaseTwo(Pager*);
int sqlite3PagerRollback(Pager*);
int sqlite3PagerOpenSavepoint(Pager *pPager, int n);
int sqlite3PagerSavepoint(Pager *pPager, int op, int iSavepoint);
int sqlite3PagerSharedLock(Pager *pPager);


#ifndef SQLITE_OMIT_WAL
  int sqlite3PagerCheckpoint(Pager *pPager, sqlite3*, int, int*, int*);
  int sqlite3PagerWalSupported(Pager *pPager);
  int sqlite3PagerWalCallback(Pager *pPager);
  int sqlite3PagerOpenWal(Pager *pPager, int *pisOpen);
  int sqlite3PagerCloseWal(Pager *pPager, sqlite3*);
................................................................................
#endif

/* Functions used to truncate the database file. */
void sqlite3PagerTruncateImage(Pager*,Pgno);

void sqlite3PagerRekey(DbPage*, Pgno, u16);

#ifndef SQLITE_OMIT_CONCURRENT
void sqlite3PagerEndConcurrent(Pager*);
int sqlite3PagerBeginConcurrent(Pager*);
void sqlite3PagerDropExclusiveLock(Pager*);
int sqlite3PagerUpgradeSnapshot(Pager *pPager, DbPage*);
void sqlite3PagerSetDbsize(Pager *pPager, Pgno);
int sqlite3PagerIsWal(Pager*);
#else
# define sqlite3PagerEndConcurrent(x)
#endif

#if defined(SQLITE_DEBUG) || !defined(SQLITE_OMIT_CONCURRENT)
int sqlite3PagerIswriteable(DbPage*);
#endif

int sqlite3PagerWalInfo(Pager*, u32 *pnPrior, u32 *pnFrame);

#if defined(SQLITE_HAS_CODEC) && !defined(SQLITE_OMIT_WAL)
void *sqlite3PagerCodec(DbPage *);
#endif

/* Functions to support testing and debugging. */
#if !defined(NDEBUG) || defined(SQLITE_TEST)
  Pgno sqlite3PagerPagenumber(DbPage*);

#endif
#ifdef SQLITE_TEST
  int *sqlite3PagerStats(Pager*);
  void sqlite3PagerRefdump(Pager*);
  void disable_simulated_io_errors(void);
  void enable_simulated_io_errors(void);
#else
# define disable_simulated_io_errors()
# define enable_simulated_io_errors()
#endif

#endif /* SQLITE_PAGER_H */

Changes to src/parse.y.

94
95
96
97
98
99
100







101
102
103
104
105
106
107
...
133
134
135
136
137
138
139
140









141
142
143
144
145
146
147
...
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
....
1028
1029
1030
1031
1032
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
....
1769
1770
1771
1772
1773
1774
1775

1776
1777
1778
1779
1780
1781
1782
** TK_DELETE, or TK_INSTEAD.  If the event is of the form
**
**      UPDATE ON (a,b,c)
**
** Then the "b" IdList records the list "a,b,c".
*/
struct TrigEvent { int a; IdList * b; };








struct FrameBound     { int eType; Expr *pExpr; };

/*
** Disable lookaside memory allocation for objects that might be
** shared across database connections.
*/
................................................................................
trans_opt ::= .
trans_opt ::= TRANSACTION.
trans_opt ::= TRANSACTION nm.
%type transtype {int}
transtype(A) ::= .             {A = TK_DEFERRED;}
transtype(A) ::= DEFERRED(X).  {A = @X; /*A-overwrites-X*/}
transtype(A) ::= IMMEDIATE(X). {A = @X; /*A-overwrites-X*/}
transtype(A) ::= EXCLUSIVE(X). {A = @X; /*A-overwrites-X*/}









cmd ::= COMMIT|END(X) trans_opt.   {sqlite3EndTransaction(pParse,@X);}
cmd ::= ROLLBACK(X) trans_opt.     {sqlite3EndTransaction(pParse,@X);}

savepoint_opt ::= SAVEPOINT.
savepoint_opt ::= .
cmd ::= SAVEPOINT nm(X). {
  sqlite3Savepoint(pParse, SAVEPOINT_BEGIN, &X);
................................................................................
%right BITNOT.
%nonassoc ON.

// An IDENTIFIER can be a generic identifier, or one of several
// keywords.  Any non-standard keyword can also be an identifier.
//
%token_class id  ID|INDEXED.


// And "ids" is an identifer-or-string.
//
%token_class ids  ID|STRING.

// The name of a column or table can be any of the following:
//
................................................................................
  }else{
    /* When doing a nested parse, one can include terms in an expression
    ** that look like this:   #1 #2 ...  These terms refer to registers
    ** in the virtual machine.  #N is the N-th register. */
    Token t = X; /*A-overwrites-X*/
    assert( t.n>=2 );
    if( pParse->nested==0 ){
      sqlite3ErrorMsg(pParse, "near \"%T\": syntax error", &t);
      A = 0;
    }else{
      A = sqlite3PExpr(pParse, TK_REGISTER, 0, 0);
      if( A ) sqlite3GetInt32(&t.z[1], &A->iTable);
    }
  }
}
................................................................................
  TRUEFALSE       /* True or false keyword */
  ISNOT           /* Combination of IS and NOT */
  FUNCTION        /* A function invocation */
  UMINUS          /* Unary minus */
  UPLUS           /* Unary plus */
  TRUTH           /* IS TRUE or IS FALSE or IS NOT TRUE or IS NOT FALSE */
  REGISTER        /* Reference to a VDBE register */

  VECTOR          /* Vector */
  SELECT_COLUMN   /* Choose a single column from a multi-column SELECT */
  IF_NULL_ROW     /* the if-null-row operator */
  ASTERISK        /* The "*" in count(*) and similar */
  SPAN            /* The span operator */
.
/* There must be no more than 255 tokens defined above.  If this grammar







>
>
>
>
>
>
>







 







|
>
>
>
>
>
>
>
>
>







 







<







 







|







 







>







94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
...
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
...
268
269
270
271
272
273
274

275
276
277
278
279
280
281
....
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
....
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
** TK_DELETE, or TK_INSTEAD.  If the event is of the form
**
**      UPDATE ON (a,b,c)
**
** Then the "b" IdList records the list "a,b,c".
*/
struct TrigEvent { int a; IdList * b; };

/*
** Generate a syntax error
*/
static void parserSyntaxError(Parse *pParse, Token *p){
  sqlite3ErrorMsg(pParse, "near \"%T\": syntax error", p);
}

struct FrameBound     { int eType; Expr *pExpr; };

/*
** Disable lookaside memory allocation for objects that might be
** shared across database connections.
*/
................................................................................
trans_opt ::= .
trans_opt ::= TRANSACTION.
trans_opt ::= TRANSACTION nm.
%type transtype {int}
transtype(A) ::= .             {A = TK_DEFERRED;}
transtype(A) ::= DEFERRED(X).  {A = @X; /*A-overwrites-X*/}
transtype(A) ::= IMMEDIATE(X). {A = @X; /*A-overwrites-X*/}
transtype(A) ::= ID(X). {
   Token *p = &X;
   if( p->n==9 && sqlite3_strnicmp(p->z,"exclusive",9)==0 ){
     A = TK_EXCLUSIVE;
   }else if( p->n==10 && sqlite3_strnicmp(p->z,"concurrent",10)==0 ){
     A = TK_CONCURRENT;  /*A-overwrites-X*/
   }else{
     parserSyntaxError(pParse, p);
   }
}
cmd ::= COMMIT|END(X) trans_opt.   {sqlite3EndTransaction(pParse,@X);}
cmd ::= ROLLBACK(X) trans_opt.     {sqlite3EndTransaction(pParse,@X);}

savepoint_opt ::= SAVEPOINT.
savepoint_opt ::= .
cmd ::= SAVEPOINT nm(X). {
  sqlite3Savepoint(pParse, SAVEPOINT_BEGIN, &X);
................................................................................
%right BITNOT.
%nonassoc ON.

// An IDENTIFIER can be a generic identifier, or one of several
// keywords.  Any non-standard keyword can also be an identifier.
//
%token_class id  ID|INDEXED.


// And "ids" is an identifer-or-string.
//
%token_class ids  ID|STRING.

// The name of a column or table can be any of the following:
//
................................................................................
  }else{
    /* When doing a nested parse, one can include terms in an expression
    ** that look like this:   #1 #2 ...  These terms refer to registers
    ** in the virtual machine.  #N is the N-th register. */
    Token t = X; /*A-overwrites-X*/
    assert( t.n>=2 );
    if( pParse->nested==0 ){
      parserSyntaxError(pParse, &t);
      A = 0;
    }else{
      A = sqlite3PExpr(pParse, TK_REGISTER, 0, 0);
      if( A ) sqlite3GetInt32(&t.z[1], &A->iTable);
    }
  }
}
................................................................................
  TRUEFALSE       /* True or false keyword */
  ISNOT           /* Combination of IS and NOT */
  FUNCTION        /* A function invocation */
  UMINUS          /* Unary minus */
  UPLUS           /* Unary plus */
  TRUTH           /* IS TRUE or IS FALSE or IS NOT TRUE or IS NOT FALSE */
  REGISTER        /* Reference to a VDBE register */
  CONCURRENT      /* BEGIN CONCURRENT */
  VECTOR          /* Vector */
  SELECT_COLUMN   /* Choose a single column from a multi-column SELECT */
  IF_NULL_ROW     /* the if-null-row operator */
  ASTERISK        /* The "*" in count(*) and similar */
  SPAN            /* The span operator */
.
/* There must be no more than 255 tokens defined above.  If this grammar

Changes to src/random.c.

101
102
103
104
105
106
107






















108
109
110
111
112
113
114
    wsdPrng.s[wsdPrng.i] = wsdPrng.s[wsdPrng.j];
    wsdPrng.s[wsdPrng.j] = t;
    t += wsdPrng.s[wsdPrng.i];
    *(zBuf++) = wsdPrng.s[t];
  }while( --N );
  sqlite3_mutex_leave(mutex);
}























#ifndef SQLITE_UNTESTABLE
/*
** For testing purposes, we sometimes want to preserve the state of
** PRNG and restore the PRNG to its saved state at a later time, or
** to reset the PRNG to its initial state.  These routines accomplish
** those tasks.







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
    wsdPrng.s[wsdPrng.i] = wsdPrng.s[wsdPrng.j];
    wsdPrng.s[wsdPrng.j] = t;
    t += wsdPrng.s[wsdPrng.i];
    *(zBuf++) = wsdPrng.s[t];
  }while( --N );
  sqlite3_mutex_leave(mutex);
}

/*
** Initialize a fast PRNG.  A Fast PRNG is called "fast" because it does
** not need a mutex to operate, though it does use a mutex to initialize.
** The quality of the randomness is not as good as the global PRNG.
*/
void sqlite3FastPrngInit(FastPrng *pPrng){
  sqlite3_randomness(sizeof(*pPrng), pPrng);
  pPrng->x |= 1;
}

/*
** Generate N bytes of pseudo-randomness using a FastPrng
*/
void sqlite3FastRandomness(FastPrng *pPrng, int N, void *P){
  unsigned char *pOut = (unsigned char*)P;
  while( N-->0 ){
    pPrng->x = ((pPrng->x)>>1) ^ ((1+~((pPrng->x)&1)) & 0xd0000001);
    pPrng->y = (pPrng->y)*1103515245 + 12345;
    *(pOut++) = (pPrng->x ^ pPrng->y) & 0xff;
  }
}

#ifndef SQLITE_UNTESTABLE
/*
** For testing purposes, we sometimes want to preserve the state of
** PRNG and restore the PRNG to its saved state at a later time, or
** to reset the PRNG to its initial state.  These routines accomplish
** those tasks.

Changes to src/select.c.

2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
    while( zName && sqlite3HashFind(&ht, zName)!=0 ){
      nName = sqlite3Strlen30(zName);
      if( nName>0 ){
        for(j=nName-1; j>0 && sqlite3Isdigit(zName[j]); j--){}
        if( zName[j]==':' ) nName = j;
      }
      zName = sqlite3MPrintf(db, "%.*z:%u", nName, zName, ++cnt);
      if( cnt>3 ) sqlite3_randomness(sizeof(cnt), &cnt);
    }
    pCol->zName = zName;
    sqlite3ColumnPropertiesFromName(0, pCol);
    if( zName && sqlite3HashInsert(&ht, zName, pCol)==pCol ){
      sqlite3OomFault(db);
    }
  }







|







2017
2018
2019
2020
2021
2022
2023
2024
2025
2026
2027
2028
2029
2030
2031
    while( zName && sqlite3HashFind(&ht, zName)!=0 ){
      nName = sqlite3Strlen30(zName);
      if( nName>0 ){
        for(j=nName-1; j>0 && sqlite3Isdigit(zName[j]); j--){}
        if( zName[j]==':' ) nName = j;
      }
      zName = sqlite3MPrintf(db, "%.*z:%u", nName, zName, ++cnt);
      if( cnt>3 ) sqlite3FastRandomness(&db->sPrng, sizeof(cnt), &cnt);
    }
    pCol->zName = zName;
    sqlite3ColumnPropertiesFromName(0, pCol);
    if( zName && sqlite3HashInsert(&ht, zName, pCol)==pCol ){
      sqlite3OomFault(db);
    }
  }

Changes to src/sqlite.h.in.

9584
9585
9586
9587
9588
9589
9590

























9591
9592
9593
9594
9595
9596
9597
**
** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
**
** This interface is only available if SQLite is compiled with the
** [SQLITE_ENABLE_SNAPSHOT] option.
*/
SQLITE_EXPERIMENTAL int sqlite3_snapshot_recover(sqlite3 *db, const char *zDb);


























/*
** CAPI3REF: Serialize a database
**
** The sqlite3_serialize(D,S,P,F) interface returns a pointer to memory
** that is a serialization of the S database on [database connection] D.
** If P is not a NULL pointer, then the size of the database in bytes







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







9584
9585
9586
9587
9588
9589
9590
9591
9592
9593
9594
9595
9596
9597
9598
9599
9600
9601
9602
9603
9604
9605
9606
9607
9608
9609
9610
9611
9612
9613
9614
9615
9616
9617
9618
9619
9620
9621
9622
**
** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
**
** This interface is only available if SQLite is compiled with the
** [SQLITE_ENABLE_SNAPSHOT] option.
*/
SQLITE_EXPERIMENTAL int sqlite3_snapshot_recover(sqlite3 *db, const char *zDb);

/*
** CAPI3REF: Wal related information regarding the most recent COMMIT
** EXPERIMENTAL
**
** This function reports on the state of the wal file (if any) for database 
** zDb, which should be "main", "temp", or the name of the attached database.
** Its results - the values written to the output parameters - are only
** defined if the most recent SQL command on the connection was a successful 
** COMMIT that wrote data to wal-mode database zDb.
**
** Assuming the above conditions are met, output parameter (*pnFrame) is set
** to the total number of frames in the wal file. Parameter (*pnPrior) is
** set to the number of frames that were present in the wal file before the
** most recent transaction was committed. So that the number of frames written
** by the most recent transaction is (*pnFrame)-(*pnPrior).
**
** If successful, SQLITE_OK is returned. Otherwise, an SQLite error code. It
** is not an error if this function is called at a time when the results
** are undefined.
*/
SQLITE_EXPERIMENTAL int sqlite3_wal_info(
  sqlite3 *db, const char *zDb, 
  unsigned int *pnPrior, unsigned int *pnFrame
);

/*
** CAPI3REF: Serialize a database
**
** The sqlite3_serialize(D,S,P,F) interface returns a pointer to memory
** that is a serialization of the S database on [database connection] D.
** If P is not a NULL pointer, then the size of the database in bytes

Changes to src/sqliteInt.h.

1075
1076
1077
1078
1079
1080
1081

1082
1083
1084
1085
1086
1087
1088
....
1192
1193
1194
1195
1196
1197
1198








1199
1200
1201
1202
1203
1204
1205
....
1434
1435
1436
1437
1438
1439
1440

1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453

1454
1455
1456
1457
1458
1459
1460
....
4258
4259
4260
4261
4262
4263
4264


4265
4266
4267
4268
4269
4270
4271
typedef struct Bitvec Bitvec;
typedef struct CollSeq CollSeq;
typedef struct Column Column;
typedef struct Db Db;
typedef struct Schema Schema;
typedef struct Expr Expr;
typedef struct ExprList ExprList;

typedef struct FKey FKey;
typedef struct FuncDestructor FuncDestructor;
typedef struct FuncDef FuncDef;
typedef struct FuncDefHash FuncDefHash;
typedef struct IdList IdList;
typedef struct Index Index;
typedef struct IndexSample IndexSample;
................................................................................
*/
#ifndef SQLITE_DEFAULT_SYNCHRONOUS
# define SQLITE_DEFAULT_SYNCHRONOUS 2
#endif
#ifndef SQLITE_DEFAULT_WAL_SYNCHRONOUS
# define SQLITE_DEFAULT_WAL_SYNCHRONOUS SQLITE_DEFAULT_SYNCHRONOUS
#endif









/*
** Each database file to be accessed by the system is an instance
** of the following structure.  There are normally two of these structures
** in the sqlite.aDb[] array.  aDb[0] is the main database file and
** aDb[1] is the database file used to hold temporary tables.  Additional
** databases may be attached.
................................................................................
  unsigned int openFlags;       /* Flags passed to sqlite3_vfs.xOpen() */
  int errCode;                  /* Most recent error code (SQLITE_*) */
  int errMask;                  /* & result codes with this before returning */
  int iSysErrno;                /* Errno value from last system error */
  u16 dbOptFlags;               /* Flags to enable/disable optimizations */
  u8 enc;                       /* Text encoding */
  u8 autoCommit;                /* The auto-commit flag. */

  u8 temp_store;                /* 1: file 2: memory 0: default */
  u8 mallocFailed;              /* True if we have seen a malloc failure */
  u8 bBenignMalloc;             /* Do not require OOMs if true */
  u8 dfltLockMode;              /* Default locking-mode for attached dbs */
  signed char nextAutovac;      /* Autovac setting after VACUUM if >=0 */
  u8 suppressErr;               /* Do not issue error messages if true */
  u8 vtabOnConflict;            /* Value to return for s3_vtab_on_conflict() */
  u8 isTransactionSavepoint;    /* True if the outermost savepoint is a TS */
  u8 mTrace;                    /* zero or more SQLITE_TRACE flags */
  u8 noSharedCache;             /* True if no shared-cache backends */
  u8 nSqlExec;                  /* Number of pending OP_SqlExec opcodes */
  int nextPagesize;             /* Pagesize after VACUUM if >0 */
  u32 magic;                    /* Magic number for detect library misuse */

  int nChange;                  /* Value returned by sqlite3_changes() */
  int nTotalChange;             /* Value returned by sqlite3_total_changes() */
  int aLimit[SQLITE_N_LIMIT];   /* Limits */
  int nMaxSorterMmap;           /* Maximum size of regions mapped by sorter */
  struct sqlite3InitInfo {      /* Information used during initialization */
    int newTnum;                /* Rootpage of table being initialized */
    u8 iDb;                     /* Which db file is being initialized */
................................................................................
int sqlite3ExprCoveredByIndex(Expr*, int iCur, Index *pIdx);
int sqlite3FunctionUsesThisSrc(Expr*, SrcList*);
Vdbe *sqlite3GetVdbe(Parse*);
#ifndef SQLITE_UNTESTABLE
void sqlite3PrngSaveState(void);
void sqlite3PrngRestoreState(void);
#endif


void sqlite3RollbackAll(sqlite3*,int);
void sqlite3CodeVerifySchema(Parse*, int);
void sqlite3CodeVerifyNamedSchema(Parse*, const char *zDb);
void sqlite3BeginTransaction(Parse*, int);
void sqlite3EndTransaction(Parse*,int);
void sqlite3Savepoint(Parse*, int, Token*);
void sqlite3CloseSavepoints(sqlite3 *);







>







 







>
>
>
>
>
>
>
>







 







>













>







 







>
>







1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
....
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
....
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
....
4269
4270
4271
4272
4273
4274
4275
4276
4277
4278
4279
4280
4281
4282
4283
4284
typedef struct Bitvec Bitvec;
typedef struct CollSeq CollSeq;
typedef struct Column Column;
typedef struct Db Db;
typedef struct Schema Schema;
typedef struct Expr Expr;
typedef struct ExprList ExprList;
typedef struct FastPrng FastPrng;
typedef struct FKey FKey;
typedef struct FuncDestructor FuncDestructor;
typedef struct FuncDef FuncDef;
typedef struct FuncDefHash FuncDefHash;
typedef struct IdList IdList;
typedef struct Index Index;
typedef struct IndexSample IndexSample;
................................................................................
*/
#ifndef SQLITE_DEFAULT_SYNCHRONOUS
# define SQLITE_DEFAULT_SYNCHRONOUS 2
#endif
#ifndef SQLITE_DEFAULT_WAL_SYNCHRONOUS
# define SQLITE_DEFAULT_WAL_SYNCHRONOUS SQLITE_DEFAULT_SYNCHRONOUS
#endif

/*
** State of a simple PRNG used for the per-connection and per-pager
** pseudo-random number generators.
*/
struct FastPrng {
  unsigned int x, y;
};

/*
** Each database file to be accessed by the system is an instance
** of the following structure.  There are normally two of these structures
** in the sqlite.aDb[] array.  aDb[0] is the main database file and
** aDb[1] is the database file used to hold temporary tables.  Additional
** databases may be attached.
................................................................................
  unsigned int openFlags;       /* Flags passed to sqlite3_vfs.xOpen() */
  int errCode;                  /* Most recent error code (SQLITE_*) */
  int errMask;                  /* & result codes with this before returning */
  int iSysErrno;                /* Errno value from last system error */
  u16 dbOptFlags;               /* Flags to enable/disable optimizations */
  u8 enc;                       /* Text encoding */
  u8 autoCommit;                /* The auto-commit flag. */
  u8 bConcurrent;               /* Current transaction is "CONCURRENT" */
  u8 temp_store;                /* 1: file 2: memory 0: default */
  u8 mallocFailed;              /* True if we have seen a malloc failure */
  u8 bBenignMalloc;             /* Do not require OOMs if true */
  u8 dfltLockMode;              /* Default locking-mode for attached dbs */
  signed char nextAutovac;      /* Autovac setting after VACUUM if >=0 */
  u8 suppressErr;               /* Do not issue error messages if true */
  u8 vtabOnConflict;            /* Value to return for s3_vtab_on_conflict() */
  u8 isTransactionSavepoint;    /* True if the outermost savepoint is a TS */
  u8 mTrace;                    /* zero or more SQLITE_TRACE flags */
  u8 noSharedCache;             /* True if no shared-cache backends */
  u8 nSqlExec;                  /* Number of pending OP_SqlExec opcodes */
  int nextPagesize;             /* Pagesize after VACUUM if >0 */
  u32 magic;                    /* Magic number for detect library misuse */
  FastPrng sPrng;               /* State of the per-connection PRNG */
  int nChange;                  /* Value returned by sqlite3_changes() */
  int nTotalChange;             /* Value returned by sqlite3_total_changes() */
  int aLimit[SQLITE_N_LIMIT];   /* Limits */
  int nMaxSorterMmap;           /* Maximum size of regions mapped by sorter */
  struct sqlite3InitInfo {      /* Information used during initialization */
    int newTnum;                /* Rootpage of table being initialized */
    u8 iDb;                     /* Which db file is being initialized */
................................................................................
int sqlite3ExprCoveredByIndex(Expr*, int iCur, Index *pIdx);
int sqlite3FunctionUsesThisSrc(Expr*, SrcList*);
Vdbe *sqlite3GetVdbe(Parse*);
#ifndef SQLITE_UNTESTABLE
void sqlite3PrngSaveState(void);
void sqlite3PrngRestoreState(void);
#endif
void sqlite3FastPrngInit(FastPrng*);
void sqlite3FastRandomness(FastPrng*, int N, void *P);
void sqlite3RollbackAll(sqlite3*,int);
void sqlite3CodeVerifySchema(Parse*, int);
void sqlite3CodeVerifyNamedSchema(Parse*, const char *zDb);
void sqlite3BeginTransaction(Parse*, int);
void sqlite3EndTransaction(Parse*,int);
void sqlite3Savepoint(Parse*, int, Token*);
void sqlite3CloseSavepoints(sqlite3 *);

Changes to src/test1.c.

7770
7771
7772
7773
7774
7775
7776



































7777
7778
7779
7780
7781
7782
7783
....
8158
8159
8160
8161
8162
8163
8164
8165

8166
8167
8168
8169
8170
8171
8172
8173
  }else{
    if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
    rc = sqlite3_db_config(db, SQLITE_DBCONFIG_MAINDBNAME, "icecube");
    Tcl_SetObjResult(interp, Tcl_NewIntObj(rc));
    return TCL_OK;
  }
}




































/*
** Usage: sqlite3_mmap_warm DB DBNAME
*/
static int SQLITE_TCLAPI test_mmap_warm(
  void * clientData,
  Tcl_Interp *interp,
................................................................................
     { "sqlite3_snapshot_free", test_snapshot_free, 0 },
     { "sqlite3_snapshot_cmp", test_snapshot_cmp, 0 },
     { "sqlite3_snapshot_recover", test_snapshot_recover, 0 },
     { "sqlite3_snapshot_get_blob", test_snapshot_get_blob, 0 },
     { "sqlite3_snapshot_open_blob", test_snapshot_open_blob, 0 },
     { "sqlite3_snapshot_cmp_blob", test_snapshot_cmp_blob, 0 },
#endif
     { "sqlite3_delete_database", test_delete_database,    0 },

     { "atomic_batch_write",      test_atomic_batch_write, 0 },
     { "sqlite3_mmap_warm",       test_mmap_warm,          0 },
     { "sqlite3_config_sorterref", test_config_sorterref,   0 },
     { "decode_hexdb",             test_decode_hexdb,       0 },
  };
  static int bitmask_size = sizeof(Bitmask)*8;
  static int longdouble_size = sizeof(LONGDOUBLE_TYPE);
  int i;







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
>
|







7770
7771
7772
7773
7774
7775
7776
7777
7778
7779
7780
7781
7782
7783
7784
7785
7786
7787
7788
7789
7790
7791
7792
7793
7794
7795
7796
7797
7798
7799
7800
7801
7802
7803
7804
7805
7806
7807
7808
7809
7810
7811
7812
7813
7814
7815
7816
7817
7818
....
8193
8194
8195
8196
8197
8198
8199
8200
8201
8202
8203
8204
8205
8206
8207
8208
8209
  }else{
    if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
    rc = sqlite3_db_config(db, SQLITE_DBCONFIG_MAINDBNAME, "icecube");
    Tcl_SetObjResult(interp, Tcl_NewIntObj(rc));
    return TCL_OK;
  }
}

/*
** Usage: sqlite3_wal_info DB DBNAME
*/
static int SQLITE_TCLAPI test_wal_info(
  void * clientData,
  Tcl_Interp *interp,
  int objc,
  Tcl_Obj *CONST objv[]
){
  int rc;
  sqlite3 *db;
  char *zName;
  unsigned int nPrior;
  unsigned int nFrame;

  if( objc!=3 ){
    Tcl_WrongNumArgs(interp, 1, objv, "DB DBNAME");
    return TCL_ERROR;
  }
  if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
  zName = Tcl_GetString(objv[2]);

  rc = sqlite3_wal_info(db, zName, &nPrior, &nFrame);
  if( rc!=SQLITE_OK ){
    Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3ErrName(rc), -1));
    return TCL_ERROR;
  }else{
    Tcl_Obj *pNew = Tcl_NewObj();
    Tcl_ListObjAppendElement(interp, pNew, Tcl_NewWideIntObj((i64)nPrior));
    Tcl_ListObjAppendElement(interp, pNew, Tcl_NewWideIntObj((i64)nFrame));
    Tcl_SetObjResult(interp, pNew);
  }
  return TCL_OK;
}

/*
** Usage: sqlite3_mmap_warm DB DBNAME
*/
static int SQLITE_TCLAPI test_mmap_warm(
  void * clientData,
  Tcl_Interp *interp,
................................................................................
     { "sqlite3_snapshot_free", test_snapshot_free, 0 },
     { "sqlite3_snapshot_cmp", test_snapshot_cmp, 0 },
     { "sqlite3_snapshot_recover", test_snapshot_recover, 0 },
     { "sqlite3_snapshot_get_blob", test_snapshot_get_blob, 0 },
     { "sqlite3_snapshot_open_blob", test_snapshot_open_blob, 0 },
     { "sqlite3_snapshot_cmp_blob", test_snapshot_cmp_blob, 0 },
#endif
     { "sqlite3_delete_database", test_delete_database, 0 },
     { "sqlite3_wal_info", test_wal_info, 0 },
     { "atomic_batch_write",      test_atomic_batch_write,     0   },
     { "sqlite3_mmap_warm",       test_mmap_warm,          0 },
     { "sqlite3_config_sorterref", test_config_sorterref,   0 },
     { "decode_hexdb",             test_decode_hexdb,       0 },
  };
  static int bitmask_size = sizeof(Bitmask)*8;
  static int longdouble_size = sizeof(LONGDOUBLE_TYPE);
  int i;

Changes to src/test_config.c.

661
662
663
664
665
666
667






668
669
670
671
672
673
674
#endif

#ifdef SQLITE_OMIT_TRUNCATE_OPTIMIZATION
  Tcl_SetVar2(interp, "sqlite_options", "truncate_opt", "0", TCL_GLOBAL_ONLY);
#else
  Tcl_SetVar2(interp, "sqlite_options", "truncate_opt", "1", TCL_GLOBAL_ONLY);
#endif







#ifdef SQLITE_OMIT_UTF16
  Tcl_SetVar2(interp, "sqlite_options", "utf16", "0", TCL_GLOBAL_ONLY);
#else
  Tcl_SetVar2(interp, "sqlite_options", "utf16", "1", TCL_GLOBAL_ONLY);
#endif








>
>
>
>
>
>







661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
#endif

#ifdef SQLITE_OMIT_TRUNCATE_OPTIMIZATION
  Tcl_SetVar2(interp, "sqlite_options", "truncate_opt", "0", TCL_GLOBAL_ONLY);
#else
  Tcl_SetVar2(interp, "sqlite_options", "truncate_opt", "1", TCL_GLOBAL_ONLY);
#endif

#ifndef SQLITE_OMIT_CONCURRENT
  Tcl_SetVar2(interp, "sqlite_options", "concurrent", "1", TCL_GLOBAL_ONLY);
#else
  Tcl_SetVar2(interp, "sqlite_options", "concurrent", "0", TCL_GLOBAL_ONLY);
#endif

#ifdef SQLITE_OMIT_UTF16
  Tcl_SetVar2(interp, "sqlite_options", "utf16", "0", TCL_GLOBAL_ONLY);
#else
  Tcl_SetVar2(interp, "sqlite_options", "utf16", "1", TCL_GLOBAL_ONLY);
#endif

Changes to src/vacuum.c.

394
395
396
397
398
399
400

401
402
403
404
405
406
407
  ** database. No locks are held on any other files (since the main file
  ** was committed at the btree level). So it safe to end the transaction
  ** by manually setting the autoCommit flag to true and detaching the
  ** vacuum database. The vacuum_db journal file is deleted when the pager
  ** is closed by the DETACH.
  */
  db->autoCommit = 1;


  if( pDb ){
    sqlite3BtreeClose(pDb->pBt);
    pDb->pBt = 0;
    pDb->pSchema = 0;
  }








>







394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
  ** database. No locks are held on any other files (since the main file
  ** was committed at the btree level). So it safe to end the transaction
  ** by manually setting the autoCommit flag to true and detaching the
  ** vacuum database. The vacuum_db journal file is deleted when the pager
  ** is closed by the DETACH.
  */
  db->autoCommit = 1;
  assert( db->bConcurrent==0 );

  if( pDb ){
    sqlite3BtreeClose(pDb->pBt);
    pDb->pBt = 0;
    pDb->pSchema = 0;
  }

Changes to src/vdbe.c.

3309
3310
3311
3312
3313
3314
3315

3316
3317
3318
3319
3320
3321
3322
....
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404



3405
3406
3407
3408
3409
3410


3411
3412
3413

3414
3415


3416
3417
3418
3419
3420
3421
3422
3423

3424
3425
3426



3427




3428
3429
3430
3431
3432
3433
3434
3435
3436
3437

3438
3439

3440
3441
3442

3443
3444
3445
3446
3447
3448
3449
....
3626
3627
3628
3629
3630
3631
3632











3633
3634
3635
3636

3637
3638
3639
3640
3641
3642
3643
....
7118
7119
7120
7121
7122
7123
7124








7125
7126
7127
7128
7129
7130
7131
    }else{

      /* Determine whether or not this is a transaction savepoint. If so,
      ** and this is a RELEASE command, then the current transaction 
      ** is committed. 
      */
      int isTransaction = pSavepoint->pNext==0 && db->isTransactionSavepoint;

      if( isTransaction && p1==SAVEPOINT_RELEASE ){
        if( (rc = sqlite3VdbeCheckFk(p, 1))!=SQLITE_OK ){
          goto vdbe_return;
        }
        db->autoCommit = 1;
        if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){
          p->pc = (int)(pOp - aOp);
................................................................................
    }
  }
  if( rc ) goto abort_due_to_error;

  break;
}

/* Opcode: AutoCommit P1 P2 * * *
**
** Set the database auto-commit flag to P1 (1 or 0). If P2 is true, roll
** back any currently active btree transactions. If there are any active
** VMs (apart from this one), then a ROLLBACK fails.  A COMMIT fails if
** there are active writing VMs or active VMs that use shared cache.



**
** This instruction causes the VM to halt.
*/
case OP_AutoCommit: {
  int desiredAutoCommit;
  int iRollback;



  desiredAutoCommit = pOp->p1;
  iRollback = pOp->p2;

  assert( desiredAutoCommit==1 || desiredAutoCommit==0 );
  assert( desiredAutoCommit==1 || iRollback==0 );


  assert( db->nVdbeActive>0 );  /* At least this one VM is active */
  assert( p->bIsReader );

  if( desiredAutoCommit!=db->autoCommit ){
    if( iRollback ){
      assert( desiredAutoCommit==1 );
      sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
      db->autoCommit = 1;

    }else if( desiredAutoCommit && db->nVdbeWrite>0 ){
      /* If this instruction implements a COMMIT and other VMs are writing
      ** return an error indicating that the other VMs must complete first. 



      */




      sqlite3VdbeError(p, "cannot commit transaction - "
                          "SQL statements in progress");
      rc = SQLITE_BUSY;
      goto abort_due_to_error;
    }else if( (rc = sqlite3VdbeCheckFk(p, 1))!=SQLITE_OK ){
      goto vdbe_return;
    }else{
      db->autoCommit = (u8)desiredAutoCommit;
    }
    if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){

      p->pc = (int)(pOp - aOp);
      db->autoCommit = (u8)(1-desiredAutoCommit);

      p->rc = rc = SQLITE_BUSY;
      goto vdbe_return;
    }

    sqlite3CloseSavepoints(db);
    if( p->rc==SQLITE_OK ){
      rc = SQLITE_DONE;
    }else{
      rc = SQLITE_ERROR;
    }
    goto vdbe_return;
................................................................................
  assert( pOp->p2<SQLITE_N_BTREE_META );
  assert( pOp->p1>=0 && pOp->p1<db->nDb );
  assert( DbMaskTest(p->btreeMask, pOp->p1) );
  assert( p->readOnly==0 );
  pDb = &db->aDb[pOp->p1];
  assert( pDb->pBt!=0 );
  assert( sqlite3SchemaMutexHeld(db, pOp->p1, 0) );











  /* See note about index shifting on OP_ReadCookie */
  rc = sqlite3BtreeUpdateMeta(pDb->pBt, pOp->p2, pOp->p3);
  if( pOp->p2==BTREE_SCHEMA_VERSION ){
    /* When the schema cookie changes, record the new cookie internally */

    pDb->pSchema->schema_cookie = pOp->p3;
    db->mDbFlags |= DBFLAG_SchemaChange;
  }else if( pOp->p2==BTREE_FILE_FORMAT ){
    /* Record changes in the file format */
    pDb->pSchema->file_format = pOp->p3;
  }
  if( pOp->p1==1 ){
................................................................................
** P2 contains the root-page of the table to lock.
**
** P4 contains a pointer to the name of the table being locked. This is only
** used to generate an error message if the lock cannot be obtained.
*/
case OP_TableLock: {
  u8 isWriteLock = (u8)pOp->p3;








  if( isWriteLock || 0==(db->flags&SQLITE_ReadUncommit) ){
    int p1 = pOp->p1; 
    assert( p1>=0 && p1<db->nDb );
    assert( DbMaskTest(p->btreeMask, p1) );
    assert( isWriteLock==0 || isWriteLock==1 );
    rc = sqlite3BtreeLockTable(db->aDb[p1].pBt, pOp->p2, isWriteLock);
    if( rc ){







>







 







|





>
>
>






>
>



>


>
>








>
|
<
|
>
>
>
|
>
>
>
>









|
>


>
|


>







 







>
>
>
>
>
>
>
>
>
>
>




>







 







>
>
>
>
>
>
>
>







3309
3310
3311
3312
3313
3314
3315
3316
3317
3318
3319
3320
3321
3322
3323
....
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408
3409
3410
3411
3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434

3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462
3463
3464
3465
3466
3467
3468
....
3645
3646
3647
3648
3649
3650
3651
3652
3653
3654
3655
3656
3657
3658
3659
3660
3661
3662
3663
3664
3665
3666
3667
3668
3669
3670
3671
3672
3673
3674
....
7149
7150
7151
7152
7153
7154
7155
7156
7157
7158
7159
7160
7161
7162
7163
7164
7165
7166
7167
7168
7169
7170
    }else{

      /* Determine whether or not this is a transaction savepoint. If so,
      ** and this is a RELEASE command, then the current transaction 
      ** is committed. 
      */
      int isTransaction = pSavepoint->pNext==0 && db->isTransactionSavepoint;
      assert( db->bConcurrent==0 || db->isTransactionSavepoint==0 );
      if( isTransaction && p1==SAVEPOINT_RELEASE ){
        if( (rc = sqlite3VdbeCheckFk(p, 1))!=SQLITE_OK ){
          goto vdbe_return;
        }
        db->autoCommit = 1;
        if( sqlite3VdbeHalt(p)==SQLITE_BUSY ){
          p->pc = (int)(pOp - aOp);
................................................................................
    }
  }
  if( rc ) goto abort_due_to_error;

  break;
}

/* Opcode: AutoCommit P1 P2 P3 * *
**
** Set the database auto-commit flag to P1 (1 or 0). If P2 is true, roll
** back any currently active btree transactions. If there are any active
** VMs (apart from this one), then a ROLLBACK fails.  A COMMIT fails if
** there are active writing VMs or active VMs that use shared cache.
**
** If P3 is non-zero, then this instruction is being executed as part of
** a "BEGIN CONCURRENT" command.
**
** This instruction causes the VM to halt.
*/
case OP_AutoCommit: {
  int desiredAutoCommit;
  int iRollback;
  int bConcurrent;
  int hrc;

  desiredAutoCommit = pOp->p1;
  iRollback = pOp->p2;
  bConcurrent = pOp->p3;
  assert( desiredAutoCommit==1 || desiredAutoCommit==0 );
  assert( desiredAutoCommit==1 || iRollback==0 );
  assert( desiredAutoCommit==0 || bConcurrent==0 );
  assert( db->autoCommit==0 || db->bConcurrent==0 );
  assert( db->nVdbeActive>0 );  /* At least this one VM is active */
  assert( p->bIsReader );

  if( desiredAutoCommit!=db->autoCommit ){
    if( iRollback ){
      assert( desiredAutoCommit==1 );
      sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
      db->autoCommit = 1;
      db->bConcurrent = 0;
    }else if( desiredAutoCommit

            && (db->nVdbeWrite>0 || (db->bConcurrent && db->nVdbeActive>1)) ){
      /* A transaction may only be committed if there are no other active
      ** writer VMs. If the transaction is CONCURRENT, then it may only be
      ** committed if there are no active VMs at all (readers or writers).
      **
      ** If this instruction is a COMMIT and the transaction may not be
      ** committed due to one of the conditions above, return an error
      ** indicating that other VMs must complete before the COMMIT can 
      ** be processed.  */
      sqlite3VdbeError(p, "cannot commit transaction - "
                          "SQL statements in progress");
      rc = SQLITE_BUSY;
      goto abort_due_to_error;
    }else if( (rc = sqlite3VdbeCheckFk(p, 1))!=SQLITE_OK ){
      goto vdbe_return;
    }else{
      db->autoCommit = (u8)desiredAutoCommit;
    }
    hrc = sqlite3VdbeHalt(p);
    if( (hrc & 0xFF)==SQLITE_BUSY ){
      p->pc = (int)(pOp - aOp);
      db->autoCommit = (u8)(1-desiredAutoCommit);
      p->rc = hrc;
      rc = SQLITE_BUSY;
      goto vdbe_return;
    }
    db->bConcurrent = (u8)bConcurrent;
    sqlite3CloseSavepoints(db);
    if( p->rc==SQLITE_OK ){
      rc = SQLITE_DONE;
    }else{
      rc = SQLITE_ERROR;
    }
    goto vdbe_return;
................................................................................
  assert( pOp->p2<SQLITE_N_BTREE_META );
  assert( pOp->p1>=0 && pOp->p1<db->nDb );
  assert( DbMaskTest(p->btreeMask, pOp->p1) );
  assert( p->readOnly==0 );
  pDb = &db->aDb[pOp->p1];
  assert( pDb->pBt!=0 );
  assert( sqlite3SchemaMutexHeld(db, pOp->p1, 0) );
#ifndef SQLITE_OMIT_CONCURRENT
  if( db->bConcurrent 
   && (pOp->p2==BTREE_USER_VERSION || pOp->p2==BTREE_APPLICATION_ID)
  ){
    rc = SQLITE_ERROR;
    sqlite3VdbeError(p, "cannot modify %s within CONCURRENT transaction",
        pOp->p2==BTREE_USER_VERSION ? "user_version" : "application_id"
    );
    goto abort_due_to_error;
  }
#endif
  /* See note about index shifting on OP_ReadCookie */
  rc = sqlite3BtreeUpdateMeta(pDb->pBt, pOp->p2, pOp->p3);
  if( pOp->p2==BTREE_SCHEMA_VERSION ){
    /* When the schema cookie changes, record the new cookie internally */
    assert( pOp->p1==1 || db->bConcurrent==0 );
    pDb->pSchema->schema_cookie = pOp->p3;
    db->mDbFlags |= DBFLAG_SchemaChange;
  }else if( pOp->p2==BTREE_FILE_FORMAT ){
    /* Record changes in the file format */
    pDb->pSchema->file_format = pOp->p3;
  }
  if( pOp->p1==1 ){
................................................................................
** P2 contains the root-page of the table to lock.
**
** P4 contains a pointer to the name of the table being locked. This is only
** used to generate an error message if the lock cannot be obtained.
*/
case OP_TableLock: {
  u8 isWriteLock = (u8)pOp->p3;
#ifndef SQLITE_OMIT_CONCURRENT
  if( isWriteLock && db->bConcurrent && pOp->p2==1 && pOp->p1!=1 ){
    rc = SQLITE_ERROR;
    sqlite3VdbeError(p, 
        "cannot modify database schema within CONCURRENT transaction");
    goto abort_due_to_error;
  }
#endif
  if( isWriteLock || 0==(db->flags&SQLITE_ReadUncommit) ){
    int p1 = pOp->p1; 
    assert( p1>=0 && p1<db->nDb );
    assert( DbMaskTest(p->btreeMask, p1) );
    assert( isWriteLock==0 || isWriteLock==1 );
    rc = sqlite3BtreeLockTable(db->aDb[p1].pBt, pOp->p2, isWriteLock);
    if( rc ){

Changes to src/vdbeaux.c.

2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644

















2645
2646
2647
2648
2649
2650
2651
....
3033
3034
3035
3036
3037
3038
3039

3040
3041
3042
3043
3044
3045
3046
....
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
....
3095
3096
3097
3098
3099
3100
3101

3102
3103
3104
3105
3106
3107
3108
....
3116
3117
3118
3119
3120
3121
3122

3123
3124
3125
3126
3127
3128
3129
      if( db->aDb[i].safety_level!=PAGER_SYNCHRONOUS_OFF
       && aMJNeeded[sqlite3PagerGetJournalMode(pPager)]
       && sqlite3PagerIsMemdb(pPager)==0
      ){ 
        assert( i!=1 );
        nTrans++;
      }
      rc = sqlite3PagerExclusiveLock(pPager);
      sqlite3BtreeLeave(pBt);
    }
  }

















  if( rc!=SQLITE_OK ){
    return rc;
  }

  /* If there are any write-transactions at all, invoke the commit hook */
  if( needXcommit && db->xCommitCallback ){
    rc = db->xCommitCallback(db->pCommitArg);
................................................................................
        }else{
          /* We are forced to roll back the active transaction. Before doing
          ** so, abort any other statements this handle currently has active.
          */
          sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
          sqlite3CloseSavepoints(db);
          db->autoCommit = 1;

          p->nChange = 0;
        }
      }
    }

    /* Check for immediate foreign key violations. */
    if( p->rc==SQLITE_OK || (p->errorAction==OE_Fail && !isSpecialError) ){
................................................................................
        }else{ 
          /* The auto-commit flag is true, the vdbe program was successful 
          ** or hit an 'OR FAIL' constraint and there are no deferred foreign
          ** key constraints to hold up the transaction. This means a commit 
          ** is required. */
          rc = vdbeCommit(db, p);
        }
        if( rc==SQLITE_BUSY && p->readOnly ){
          sqlite3VdbeLeave(p);
          return SQLITE_BUSY;
        }else if( rc!=SQLITE_OK ){
          p->rc = rc;
          sqlite3RollbackAll(db, SQLITE_OK);
          p->nChange = 0;
        }else{
          db->nDeferredCons = 0;
          db->nDeferredImmCons = 0;
................................................................................
        eStatementOp = SAVEPOINT_RELEASE;
      }else if( p->errorAction==OE_Abort ){
        eStatementOp = SAVEPOINT_ROLLBACK;
      }else{
        sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
        sqlite3CloseSavepoints(db);
        db->autoCommit = 1;

        p->nChange = 0;
      }
    }
  
    /* If eStatementOp is non-zero, then a statement transaction needs to
    ** be committed or rolled back. Call sqlite3VdbeCloseStatement() to
    ** do so. If this operation returns an error, and the current statement
................................................................................
          p->rc = rc;
          sqlite3DbFree(db, p->zErrMsg);
          p->zErrMsg = 0;
        }
        sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
        sqlite3CloseSavepoints(db);
        db->autoCommit = 1;

        p->nChange = 0;
      }
    }
  
    /* If this was an INSERT, UPDATE or DELETE and no statement transaction
    ** has been rolled back, update the database connection change-counter. 
    */







|



>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>







 







|

|







 







>







 







>







2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649
2650
2651
2652
2653
2654
2655
2656
2657
2658
2659
2660
2661
2662
2663
2664
2665
2666
2667
2668
....
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
....
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095
3096
3097
3098
3099
3100
3101
3102
....
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
....
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
      if( db->aDb[i].safety_level!=PAGER_SYNCHRONOUS_OFF
       && aMJNeeded[sqlite3PagerGetJournalMode(pPager)]
       && sqlite3PagerIsMemdb(pPager)==0
      ){ 
        assert( i!=1 );
        nTrans++;
      }
      rc = sqlite3BtreeExclusiveLock(pBt);
      sqlite3BtreeLeave(pBt);
    }
  }

#ifndef SQLITE_OMIT_CONCURRENT
  if( db->bConcurrent && (rc & 0xFF)==SQLITE_BUSY ){
    /* An SQLITE_BUSY or SQLITE_BUSY_SNAPSHOT was encountered while 
    ** attempting to take the WRITER lock on a wal file. Release the
    ** WRITER locks on all wal files and return early.  */
    for(i=0; i<db->nDb; i++){
      Btree *pBt = db->aDb[i].pBt;
      if( sqlite3BtreeIsInTrans(pBt) ){
        sqlite3BtreeEnter(pBt);
        sqlite3PagerDropExclusiveLock(sqlite3BtreePager(pBt));
        sqlite3BtreeLeave(pBt);
      }
    }
  }
#endif

  if( rc!=SQLITE_OK ){
    return rc;
  }

  /* If there are any write-transactions at all, invoke the commit hook */
  if( needXcommit && db->xCommitCallback ){
    rc = db->xCommitCallback(db->pCommitArg);
................................................................................
        }else{
          /* We are forced to roll back the active transaction. Before doing
          ** so, abort any other statements this handle currently has active.
          */
          sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
          sqlite3CloseSavepoints(db);
          db->autoCommit = 1;
          db->bConcurrent = 0;
          p->nChange = 0;
        }
      }
    }

    /* Check for immediate foreign key violations. */
    if( p->rc==SQLITE_OK || (p->errorAction==OE_Fail && !isSpecialError) ){
................................................................................
        }else{ 
          /* The auto-commit flag is true, the vdbe program was successful 
          ** or hit an 'OR FAIL' constraint and there are no deferred foreign
          ** key constraints to hold up the transaction. This means a commit 
          ** is required. */
          rc = vdbeCommit(db, p);
        }
        if( (rc & 0xFF)==SQLITE_BUSY && p->readOnly ){
          sqlite3VdbeLeave(p);
          return rc;
        }else if( rc!=SQLITE_OK ){
          p->rc = rc;
          sqlite3RollbackAll(db, SQLITE_OK);
          p->nChange = 0;
        }else{
          db->nDeferredCons = 0;
          db->nDeferredImmCons = 0;
................................................................................
        eStatementOp = SAVEPOINT_RELEASE;
      }else if( p->errorAction==OE_Abort ){
        eStatementOp = SAVEPOINT_ROLLBACK;
      }else{
        sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
        sqlite3CloseSavepoints(db);
        db->autoCommit = 1;
        db->bConcurrent = 0;
        p->nChange = 0;
      }
    }
  
    /* If eStatementOp is non-zero, then a statement transaction needs to
    ** be committed or rolled back. Call sqlite3VdbeCloseStatement() to
    ** do so. If this operation returns an error, and the current statement
................................................................................
          p->rc = rc;
          sqlite3DbFree(db, p->zErrMsg);
          p->zErrMsg = 0;
        }
        sqlite3RollbackAll(db, SQLITE_ABORT_ROLLBACK);
        sqlite3CloseSavepoints(db);
        db->autoCommit = 1;
        db->bConcurrent = 0;
        p->nChange = 0;
      }
    }
  
    /* If this was an INSERT, UPDATE or DELETE and no statement transaction
    ** has been rolled back, update the database connection change-counter. 
    */

Changes to src/wal.c.

467
468
469
470
471
472
473

474
475

476
477
478
479
480
481
482
...
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
....
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
....
1378
1379
1380
1381
1382
1383
1384

1385
1386
1387
1388
1389
1390
1391
....
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
....
2055
2056
2057
2058
2059
2060
2061











































2062
2063
2064
2065
2066
2067
2068
....
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103

2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
2115
2116
2117
2118
....
2771
2772
2773
2774
2775
2776
2777

2778
2779
2780
2781
2782
2783
2784
....
2882
2883
2884
2885
2886
2887
2888
2889
2890
2891
2892
2893
2894
2895
2896
2897
....
2987
2988
2989
2990
2991
2992
2993





























2994
2995
2996
2997
2998
2999
3000
....
3002
3003
3004
3005
3006
3007
3008
























3009













3010
3011













































3012


















3013
3014

3015
3016
3017


3018






















3019
3020


3021







3022
3023
3024






















3025
3026
3027
3028
3029
3030
3031












3032
3033
3034


3035
3036

3037






3038
3039
3040

3041
3042
3043
3044
3045
3046
3047
....
3061
3062
3063
3064
3065
3066
3067


3068



3069
3070
3071
3072
3073
3074
3075
3076
3077







3078
3079
3080
3081
3082
3083
3084
....
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
....
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
....
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181

3182
3183
3184
3185
3186

3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
....
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387
3388
3389
3390
3391
....
3457
3458
3459
3460
3461
3462
3463

3464
3465
3466
3467
3468
3469
3470
....
3634
3635
3636
3637
3638
3639
3640
3641
3642
3643
3644
3645
3646
3647
3648
....
3855
3856
3857
3858
3859
3860
3861
3862












3863
  u8 truncateOnCommit;       /* True to truncate WAL file on commit */
  u8 syncHeader;             /* Fsync the WAL header if true */
  u8 padToSectorBoundary;    /* Pad transactions out to the next sector */
  u8 bShmUnreliable;         /* SHM content is read-only and unreliable */
  WalIndexHdr hdr;           /* Wal-index header for current transaction */
  u32 minFrame;              /* Ignore wal frames before this one */
  u32 iReCksum;              /* On commit, recalculate checksums from here */

  const char *zWalName;      /* Name of WAL file */
  u32 nCkpt;                 /* Checkpoint sequence counter in the wal-header */

#ifdef SQLITE_DEBUG
  u8 lockError;              /* True if a locking error has occurred */
#endif
#ifdef SQLITE_ENABLE_SNAPSHOT
  WalIndexHdr *pSnapshot;    /* Start transaction here if not NULL */
#endif
};
................................................................................
}
#endif /*defined(SQLITE_TEST) || defined(SQLITE_DEBUG) */
    

/*
** Set or release locks on the WAL.  Locks are either shared or exclusive.
** A lock cannot be moved directly between shared and exclusive - it must go
** through the unlocked state first.
**
** In locking_mode=EXCLUSIVE, all of these routines become no-ops.
*/
static int walLockShared(Wal *pWal, int lockIdx){
  int rc;
  if( pWal->exclusiveMode ) return SQLITE_OK;
  rc = sqlite3OsShmLock(pWal->pDbFd, lockIdx, 1,
................................................................................
  i64 nSize;                      /* Size of log file */
  u32 aFrameCksum[2] = {0, 0};
  int iLock;                      /* Lock offset to lock for checkpoint */

  /* Obtain an exclusive lock on all byte in the locking range not already
  ** locked by the caller. The caller is guaranteed to have locked the
  ** WAL_WRITE_LOCK byte, and may have also locked the WAL_CKPT_LOCK byte.
  ** If successful, the same bytes that are locked here are unlocked before
  ** this function returns.
  */
  assert( pWal->ckptLock==1 || pWal->ckptLock==0 );
  assert( WAL_ALL_BUT_WRITE==WAL_WRITE_LOCK+1 );
  assert( WAL_CKPT_LOCK==WAL_ALL_BUT_WRITE );
  assert( pWal->writeLock );
  iLock = WAL_ALL_BUT_WRITE + pWal->ckptLock;
................................................................................
  pRet->pDbFd = pDbFd;
  pRet->readLock = -1;
  pRet->mxWalSize = mxWalSize;
  pRet->zWalName = zWalName;
  pRet->syncHeader = 1;
  pRet->padToSectorBoundary = 1;
  pRet->exclusiveMode = (bNoShm ? WAL_HEAPMEMORY_MODE: WAL_NORMAL_MODE);


  /* Open file handle on the write-ahead log file. */
  flags = (SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|SQLITE_OPEN_WAL);
  rc = sqlite3OsOpen(pVfs, zWalName, pRet->pWalFd, flags, &flags);
  if( rc==SQLITE_OK && flags&SQLITE_OPEN_READONLY ){
    pRet->readOnly = WAL_RDONLY;
  }
................................................................................
  */
  if( rc==SQLITE_OK && eMode!=SQLITE_CHECKPOINT_PASSIVE ){
    assert( pWal->writeLock );
    if( pInfo->nBackfill<pWal->hdr.mxFrame ){
      rc = SQLITE_BUSY;
    }else if( eMode>=SQLITE_CHECKPOINT_RESTART ){
      u32 salt1;
      sqlite3_randomness(4, &salt1);
      assert( pInfo->nBackfill==pWal->hdr.mxFrame );
      rc = walBusyLock(pWal, xBusy, pBusyArg, WAL_READ_LOCK(1), WAL_NREADER-1);
      if( rc==SQLITE_OK ){
        if( eMode==SQLITE_CHECKPOINT_TRUNCATE ){
          /* IMPLEMENTATION-OF: R-44699-57140 This mode works the same way as
          ** SQLITE_CHECKPOINT_RESTART with the addition that it also
          ** truncates the log file to zero bytes just prior to a
................................................................................
    }
    WALTRACE(("WAL%p: closed\n", pWal));
    sqlite3_free((void *)pWal->apWiData);
    sqlite3_free(pWal);
  }
  return rc;
}












































/*
** Try to read the wal-index header.  Return 0 on success and 1 if
** there is a problem.
**
** The wal-index is in shared memory.  Another thread or process might
** be writing the header at the same time this procedure is trying to
................................................................................
** pWal->hdr, then pWal->hdr is updated to the content of the new header
** and *pChanged is set to 1.
**
** If the checksum cannot be verified return non-zero. If the header
** is read successfully and the checksum verified, return zero.
*/
static int walIndexTryHdr(Wal *pWal, int *pChanged){
  u32 aCksum[2];                  /* Checksum on the header content */
  WalIndexHdr h1, h2;             /* Two copies of the header content */
  WalIndexHdr volatile *aHdr;     /* Header in shared memory */

  /* The first page of the wal-index must be mapped at this point. */
  assert( pWal->nWiData>0 && pWal->apWiData[0] );

  /* Read the header. This might happen concurrently with a write to the
  ** same area of shared memory on a different CPU in a SMP,
  ** meaning it is possible that an inconsistent snapshot is read
  ** from the file. If this happens, return non-zero.
  **
  ** There are two copies of the header at the beginning of the wal-index.
  ** When reading, read [0] first then [1].  Writes are in the reverse order.
  ** Memory barriers are used to prevent the compiler or the hardware from
  ** reordering the reads and writes.
  */
  aHdr = walIndexHdr(pWal);
  memcpy(&h1, (void *)&aHdr[0], sizeof(h1));
  walShmBarrier(pWal);
  memcpy(&h2, (void *)&aHdr[1], sizeof(h2));

  if( memcmp(&h1, &h2, sizeof(h1))!=0 ){

    return 1;   /* Dirty read */
  }  
  if( h1.isInit==0 ){
    return 1;   /* Malformed header - probably all zeros */
  }
  walChecksumBytes(1, (u8*)&h1, sizeof(h1)-sizeof(h1.aCksum), 0, aCksum);
  if( aCksum[0]!=h1.aCksum[0] || aCksum[1]!=h1.aCksum[1] ){
    return 1;   /* Checksum does not match */
  }

  if( memcmp(&pWal->hdr, &h1, sizeof(WalIndexHdr)) ){
    *pChanged = 1;
    memcpy(&pWal->hdr, &h1, sizeof(WalIndexHdr));
    pWal->szPage = (pWal->hdr.szPage&0xfe00) + ((pWal->hdr.szPage&0x0001)<<16);
    testcase( pWal->szPage<=32768 );
................................................................................
    rc = walTryBeginRead(pWal, pChanged, 0, ++cnt);
  }while( rc==WAL_RETRY );
  testcase( (rc&0xff)==SQLITE_BUSY );
  testcase( (rc&0xff)==SQLITE_IOERR );
  testcase( rc==SQLITE_PROTOCOL );
  testcase( rc==SQLITE_OK );


#ifdef SQLITE_ENABLE_SNAPSHOT
  if( rc==SQLITE_OK ){
    if( pSnapshot && memcmp(pSnapshot, &pWal->hdr, sizeof(WalIndexHdr))!=0 ){
      /* At this point the client has a lock on an aReadMark[] slot holding
      ** a value equal to or smaller than pSnapshot->mxFrame, but pWal->hdr
      ** is populated with the wal-index header corresponding to the head
      ** of the wal file. Verify that pSnapshot is still valid before
................................................................................
  ** WAL were empty.
  */
  if( iLast==0 || (pWal->readLock==0 && pWal->bShmUnreliable==0) ){
    *piRead = 0;
    return SQLITE_OK;
  }

  /* Search the hash table or tables for an entry matching page number
  ** pgno. Each iteration of the following for() loop searches one
  ** hash table (each hash table indexes up to HASHTABLE_NPAGE frames).
  **
  ** This code might run concurrently to the code in walIndexAppend()
  ** that adds entries to the wal-index (and possibly to this hash 
  ** table). This means the value just read from the hash 
  ** slot (aHash[iKey]) may have been added before or after the 
  ** current read transaction was opened. Values added after the
................................................................................
Pgno sqlite3WalDbsize(Wal *pWal){
  if( pWal && ALWAYS(pWal->readLock>=0) ){
    return pWal->hdr.nPage;
  }
  return 0;
}































/* 
** This function starts a write transaction on the WAL.
**
** A read transaction must have already been started by a prior call
** to sqlite3WalBeginReadTransaction().
**
................................................................................
** the read transaction was started, then it is not possible for this
** thread to write as doing so would cause a fork.  So this routine
** returns SQLITE_BUSY in that case and no write transaction is started.
**
** There can only be a single writer active at a time.
*/
int sqlite3WalBeginWriteTransaction(Wal *pWal){
























  int rc;














  /* Cannot start a write transaction without first holding a read













































  ** transaction. */


















  assert( pWal->readLock>=0 );
  assert( pWal->writeLock==0 && pWal->iReCksum==0 );


  if( pWal->readOnly ){
    return SQLITE_READONLY;


  }























  /* Only one writer allowed at a time.  Get the write lock.  Return


  ** SQLITE_BUSY if unable.







  */
  rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1);
  if( rc ){






















    return rc;
  }
  pWal->writeLock = 1;

  /* If another connection has written to the database file since the
  ** time the read transaction on this connection was started, then
  ** the write is disallowed.












  */
  if( memcmp(&pWal->hdr, (void *)walIndexHdr(pWal), sizeof(WalIndexHdr))!=0 ){
    walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1);


    pWal->writeLock = 0;
    rc = SQLITE_BUSY_SNAPSHOT;

  }







  return rc;
}


/*
** End a write transaction.  The commit has already been done.  This
** routine merely releases the lock.
*/
int sqlite3WalEndWriteTransaction(Wal *pWal){
  if( pWal->writeLock ){
................................................................................
** to the WAL since the start of the transaction. If the callback returns
** other than SQLITE_OK, it is not invoked again and the error code is
** returned to the caller.
**
** Otherwise, if the callback function does not return an error, this
** function returns SQLITE_OK.
*/


int sqlite3WalUndo(Wal *pWal, int (*xUndo)(void *, Pgno), void *pUndoCtx){



  int rc = SQLITE_OK;
  if( ALWAYS(pWal->writeLock) ){
    Pgno iMax = pWal->hdr.mxFrame;
    Pgno iFrame;
  
    /* Restore the clients cache of the wal-index header to the state it
    ** was in before the client began writing to the database. 
    */
    memcpy(&pWal->hdr, (void *)walIndexHdr(pWal), sizeof(WalIndexHdr));








    for(iFrame=pWal->hdr.mxFrame+1; 
        ALWAYS(rc==SQLITE_OK) && iFrame<=iMax; 
        iFrame++
    ){
      /* This call cannot fail. Unless the page for which the page number
      ** is passed as the second argument is (a) in the cache and 
................................................................................
/* 
** Argument aWalData must point to an array of WAL_SAVEPOINT_NDATA u32 
** values. This function populates the array with values required to 
** "rollback" the write position of the WAL handle back to the current 
** point in the event of a savepoint rollback (via WalSavepointUndo()).
*/
void sqlite3WalSavepoint(Wal *pWal, u32 *aWalData){
  assert( pWal->writeLock );
  aWalData[0] = pWal->hdr.mxFrame;
  aWalData[1] = pWal->hdr.aFrameCksum[0];
  aWalData[2] = pWal->hdr.aFrameCksum[1];
  aWalData[3] = pWal->nCkpt;
}

/* 
................................................................................
** the values in the aWalData[] array. aWalData must point to an array
** of WAL_SAVEPOINT_NDATA u32 values that has been previously populated
** by a call to WalSavepoint().
*/
int sqlite3WalSavepointUndo(Wal *pWal, u32 *aWalData){
  int rc = SQLITE_OK;

  assert( pWal->writeLock );
  assert( aWalData[3]!=pWal->nCkpt || aWalData[0]<=pWal->hdr.mxFrame );

  if( aWalData[3]!=pWal->nCkpt ){
    /* This savepoint was opened immediately after the write-transaction
    ** was started. Right after that, the writer decided to wrap around
    ** to the start of the log. Update the savepoint values to match.
    */
................................................................................
**
** SQLITE_OK is returned if no error is encountered (regardless of whether
** or not pWal->hdr.mxFrame is modified). An SQLite error code is returned
** if an error occurs.
*/
static int walRestartLog(Wal *pWal){
  int rc = SQLITE_OK;
  int cnt;

  if( pWal->readLock==0 ){
    volatile WalCkptInfo *pInfo = walCkptInfo(pWal);
    assert( pInfo->nBackfill==pWal->hdr.mxFrame );
    if( pInfo->nBackfill>0 ){
      u32 salt1;
      sqlite3_randomness(4, &salt1);
      rc = walLockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);
      if( rc==SQLITE_OK ){
        /* If all readers are using WAL_READ_LOCK(0) (in other words if no
        ** readers are currently using the WAL), then the transactions
        ** frames will overwrite the start of the existing log. Update the
        ** wal-index header to reflect this.
        **
        ** In theory it would be Ok to update the cache of the header only
        ** at this point. But updating the actual wal-index header is also
        ** safe and means there is no special case for sqlite3WalUndo()
        ** to handle if this transaction is rolled back.  */
        walRestartHdr(pWal, salt1);
        walUnlockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);

      }else if( rc!=SQLITE_BUSY ){
        return rc;
      }
    }
    walUnlockShared(pWal, WAL_READ_LOCK(0));

    pWal->readLock = -1;
    cnt = 0;
    do{
      int notUsed;
      rc = walTryBeginRead(pWal, &notUsed, 1, ++cnt);
    }while( rc==WAL_RETRY );
    assert( (rc&0xff)!=SQLITE_BUSY ); /* BUSY not possible when useWal==1 */
    testcase( (rc&0xff)==SQLITE_IOERR );
    testcase( rc==SQLITE_PROTOCOL );
    testcase( rc==SQLITE_OK );
  }
  return rc;
}

/*
** Information about the current state of the WAL file and where
** the next fsync should occur - passed from sqlite3WalFrames() into
................................................................................
    u8 aWalHdr[WAL_HDRSIZE];      /* Buffer to assemble wal-header in */
    u32 aCksum[2];                /* Checksum for wal-header */

    sqlite3Put4byte(&aWalHdr[0], (WAL_MAGIC | SQLITE_BIGENDIAN));
    sqlite3Put4byte(&aWalHdr[4], WAL_MAX_VERSION);
    sqlite3Put4byte(&aWalHdr[8], szPage);
    sqlite3Put4byte(&aWalHdr[12], pWal->nCkpt);
    if( pWal->nCkpt==0 ) sqlite3_randomness(8, pWal->hdr.aSalt);
    memcpy(&aWalHdr[16], pWal->hdr.aSalt, 8);
    walChecksumBytes(1, aWalHdr, WAL_HDRSIZE-2*4, 0, aCksum);
    sqlite3Put4byte(&aWalHdr[24], aCksum[0]);
    sqlite3Put4byte(&aWalHdr[28], aCksum[1]);
    
    pWal->szPage = szPage;
    pWal->hdr.bigEndCksum = SQLITE_BIGENDIAN;
................................................................................
    nDbSize = (isCommit && p->pDirty==0) ? nTruncate : 0;
    rc = walWriteOneFrame(&w, p, nDbSize, iOffset);
    if( rc ) return rc;
    pLast = p;
    iOffset += szFrame;
    p->flags |= PGHDR_WAL_APPEND;
  }


  /* Recalculate checksums within the wal file if required. */
  if( isCommit && pWal->iReCksum ){
    rc = walRewriteChecksums(pWal, iFrame);
    if( rc ) return rc;
  }

................................................................................
    if( isChanged && pWal->pDbFd->pMethods->iVersion>=3 ){
      sqlite3OsUnfetch(pWal->pDbFd, 0, 0);
    }
  }

  /* Copy data from the log to the database file. */
  if( rc==SQLITE_OK ){

    if( pWal->hdr.mxFrame && walPagesize(pWal)!=nBuf ){
      rc = SQLITE_CORRUPT_BKPT;
    }else{
      rc = walCheckpoint(pWal, db, eMode2, xBusy2, pBusyArg, sync_flags, zBuf);
    }

    /* If no error occurred, set the output variables. */
................................................................................
#endif

/* Return the sqlite3_file object for the WAL file
*/
sqlite3_file *sqlite3WalFile(Wal *pWal){
  return pWal->pWalFd;
}













#endif /* #ifndef SQLITE_OMIT_WAL */







>


>







 







|







 







|







 







>







 







|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







|
<
<

<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
>
|
<
<
<
<
<
<
<







 







>







 







<
|







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

>
>
>
>
>
>
>
>
>
>
>
>
>
|
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
<
>
|
<
<
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
<
>
>
|
>
>
>
>
>
>
>
|
<
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
<

<
<
<
>
>
>
>
>
>
>
>
>
>
>
>
|
<
<
>
>
|
<
>
|
>
>
>
>
>
>
|


>







 







>
>
|
>
>
>

|







>
>
>
>
>
>
>







 







<







 







|







 







<






|













>




<
>
|
|
|
|
|
|
|
|
|
<







 







|







 







>







 







<







 








>
>
>
>
>
>
>
>
>
>
>
>

467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
...
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
....
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
....
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
....
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
1956
1957
....
2058
2059
2060
2061
2062
2063
2064
2065
2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084
2085
2086
2087
2088
2089
2090
2091
2092
2093
2094
2095
2096
2097
2098
2099
2100
2101
2102
2103
2104
2105
2106
2107
2108
2109
2110
2111
2112
2113
2114
....
2120
2121
2122
2123
2124
2125
2126
2127


2128



















2129
2130







2131
2132
2133
2134
2135
2136
2137
....
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801
2802
2803
2804
....
2902
2903
2904
2905
2906
2907
2908

2909
2910
2911
2912
2913
2914
2915
2916
....
3006
3007
3008
3009
3010
3011
3012
3013
3014
3015
3016
3017
3018
3019
3020
3021
3022
3023
3024
3025
3026
3027
3028
3029
3030
3031
3032
3033
3034
3035
3036
3037
3038
3039
3040
3041
3042
3043
3044
3045
3046
3047
3048
....
3050
3051
3052
3053
3054
3055
3056
3057
3058
3059
3060
3061
3062
3063
3064
3065
3066
3067
3068
3069
3070
3071
3072
3073
3074
3075
3076
3077
3078
3079
3080
3081
3082
3083
3084
3085
3086
3087
3088
3089
3090
3091
3092
3093
3094
3095

3096
3097
3098
3099
3100
3101
3102
3103
3104
3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160

3161
3162


3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188

3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199


3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212
3213
3214
3215
3216
3217
3218
3219
3220
3221
3222
3223

3224



3225
3226
3227
3228
3229
3230
3231
3232
3233
3234
3235
3236
3237


3238
3239
3240

3241
3242
3243
3244
3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
....
3273
3274
3275
3276
3277
3278
3279
3280
3281
3282
3283
3284
3285
3286
3287
3288
3289
3290
3291
3292
3293
3294
3295
3296
3297
3298
3299
3300
3301
3302
3303
3304
3305
3306
3307
3308
....
3326
3327
3328
3329
3330
3331
3332

3333
3334
3335
3336
3337
3338
3339
....
3341
3342
3343
3344
3345
3346
3347
3348
3349
3350
3351
3352
3353
3354
3355
....
3377
3378
3379
3380
3381
3382
3383

3384
3385
3386
3387
3388
3389
3390
3391
3392
3393
3394
3395
3396
3397
3398
3399
3400
3401
3402
3403
3404
3405
3406
3407
3408

3409
3410
3411
3412
3413
3414
3415
3416
3417
3418

3419
3420
3421
3422
3423
3424
3425
....
3599
3600
3601
3602
3603
3604
3605
3606
3607
3608
3609
3610
3611
3612
3613
....
3679
3680
3681
3682
3683
3684
3685
3686
3687
3688
3689
3690
3691
3692
3693
....
3857
3858
3859
3860
3861
3862
3863

3864
3865
3866
3867
3868
3869
3870
....
4077
4078
4079
4080
4081
4082
4083
4084
4085
4086
4087
4088
4089
4090
4091
4092
4093
4094
4095
4096
4097
  u8 truncateOnCommit;       /* True to truncate WAL file on commit */
  u8 syncHeader;             /* Fsync the WAL header if true */
  u8 padToSectorBoundary;    /* Pad transactions out to the next sector */
  u8 bShmUnreliable;         /* SHM content is read-only and unreliable */
  WalIndexHdr hdr;           /* Wal-index header for current transaction */
  u32 minFrame;              /* Ignore wal frames before this one */
  u32 iReCksum;              /* On commit, recalculate checksums from here */
  u32 nPriorFrame;           /* For sqlite3WalInfo() */
  const char *zWalName;      /* Name of WAL file */
  u32 nCkpt;                 /* Checkpoint sequence counter in the wal-header */
  FastPrng sPrng;            /* Random number generator */
#ifdef SQLITE_DEBUG
  u8 lockError;              /* True if a locking error has occurred */
#endif
#ifdef SQLITE_ENABLE_SNAPSHOT
  WalIndexHdr *pSnapshot;    /* Start transaction here if not NULL */
#endif
};
................................................................................
}
#endif /*defined(SQLITE_TEST) || defined(SQLITE_DEBUG) */
    

/*
** Set or release locks on the WAL.  Locks are either shared or exclusive.
** A lock cannot be moved directly between shared and exclusive - it must go
** through the concurrent state first.
**
** In locking_mode=EXCLUSIVE, all of these routines become no-ops.
*/
static int walLockShared(Wal *pWal, int lockIdx){
  int rc;
  if( pWal->exclusiveMode ) return SQLITE_OK;
  rc = sqlite3OsShmLock(pWal->pDbFd, lockIdx, 1,
................................................................................
  i64 nSize;                      /* Size of log file */
  u32 aFrameCksum[2] = {0, 0};
  int iLock;                      /* Lock offset to lock for checkpoint */

  /* Obtain an exclusive lock on all byte in the locking range not already
  ** locked by the caller. The caller is guaranteed to have locked the
  ** WAL_WRITE_LOCK byte, and may have also locked the WAL_CKPT_LOCK byte.
  ** If successful, the same bytes that are locked here are concurrent before
  ** this function returns.
  */
  assert( pWal->ckptLock==1 || pWal->ckptLock==0 );
  assert( WAL_ALL_BUT_WRITE==WAL_WRITE_LOCK+1 );
  assert( WAL_CKPT_LOCK==WAL_ALL_BUT_WRITE );
  assert( pWal->writeLock );
  iLock = WAL_ALL_BUT_WRITE + pWal->ckptLock;
................................................................................
  pRet->pDbFd = pDbFd;
  pRet->readLock = -1;
  pRet->mxWalSize = mxWalSize;
  pRet->zWalName = zWalName;
  pRet->syncHeader = 1;
  pRet->padToSectorBoundary = 1;
  pRet->exclusiveMode = (bNoShm ? WAL_HEAPMEMORY_MODE: WAL_NORMAL_MODE);
  sqlite3FastPrngInit(&pRet->sPrng);

  /* Open file handle on the write-ahead log file. */
  flags = (SQLITE_OPEN_READWRITE|SQLITE_OPEN_CREATE|SQLITE_OPEN_WAL);
  rc = sqlite3OsOpen(pVfs, zWalName, pRet->pWalFd, flags, &flags);
  if( rc==SQLITE_OK && flags&SQLITE_OPEN_READONLY ){
    pRet->readOnly = WAL_RDONLY;
  }
................................................................................
  */
  if( rc==SQLITE_OK && eMode!=SQLITE_CHECKPOINT_PASSIVE ){
    assert( pWal->writeLock );
    if( pInfo->nBackfill<pWal->hdr.mxFrame ){
      rc = SQLITE_BUSY;
    }else if( eMode>=SQLITE_CHECKPOINT_RESTART ){
      u32 salt1;
      sqlite3FastRandomness(&pWal->sPrng, 4, &salt1);
      assert( pInfo->nBackfill==pWal->hdr.mxFrame );
      rc = walBusyLock(pWal, xBusy, pBusyArg, WAL_READ_LOCK(1), WAL_NREADER-1);
      if( rc==SQLITE_OK ){
        if( eMode==SQLITE_CHECKPOINT_TRUNCATE ){
          /* IMPLEMENTATION-OF: R-44699-57140 This mode works the same way as
          ** SQLITE_CHECKPOINT_RESTART with the addition that it also
          ** truncates the log file to zero bytes just prior to a
................................................................................
    }
    WALTRACE(("WAL%p: closed\n", pWal));
    sqlite3_free((void *)pWal->apWiData);
    sqlite3_free(pWal);
  }
  return rc;
}

/*
** Try to copy the wal-index header from shared-memory into (*pHdr). Return
** zero if successful or non-zero otherwise. If the header is corrupted
** (either because the two copies are inconsistent or because the checksum 
** values are incorrect), the read fails and non-zero is returned.
*/
static int walIndexLoadHdr(Wal *pWal, WalIndexHdr *pHdr){
  u32 aCksum[2];                  /* Checksum on the header content */
  WalIndexHdr h2;                 /* Second copy of the header content */
  WalIndexHdr volatile *aHdr;     /* Header in shared memory */

  /* The first page of the wal-index must be mapped at this point. */
  assert( pWal->nWiData>0 && pWal->apWiData[0] );

  /* Read the header. This might happen concurrently with a write to the
  ** same area of shared memory on a different CPU in a SMP,
  ** meaning it is possible that an inconsistent snapshot is read
  ** from the file. If this happens, return non-zero.
  **
  ** There are two copies of the header at the beginning of the wal-index.
  ** When reading, read [0] first then [1].  Writes are in the reverse order.
  ** Memory barriers are used to prevent the compiler or the hardware from
  ** reordering the reads and writes.
  */
  aHdr = walIndexHdr(pWal);
  memcpy(pHdr, (void *)&aHdr[0], sizeof(h2));
  walShmBarrier(pWal);
  memcpy(&h2, (void *)&aHdr[1], sizeof(h2));

  if( memcmp(&h2, pHdr, sizeof(h2))!=0 ){
    return 1;   /* Dirty read */
  }  
  if( h2.isInit==0 ){
    return 1;   /* Malformed header - probably all zeros */
  }
  walChecksumBytes(1, (u8*)&h2, sizeof(h2)-sizeof(h2.aCksum), 0, aCksum);
  if( aCksum[0]!=h2.aCksum[0] || aCksum[1]!=h2.aCksum[1] ){
    return 1;   /* Checksum does not match */
  }

  return 0;
}

/*
** Try to read the wal-index header.  Return 0 on success and 1 if
** there is a problem.
**
** The wal-index is in shared memory.  Another thread or process might
** be writing the header at the same time this procedure is trying to
................................................................................
** pWal->hdr, then pWal->hdr is updated to the content of the new header
** and *pChanged is set to 1.
**
** If the checksum cannot be verified return non-zero. If the header
** is read successfully and the checksum verified, return zero.
*/
static int walIndexTryHdr(Wal *pWal, int *pChanged){
  WalIndexHdr h1;                 /* Copy of the header content */






















  if( walIndexLoadHdr(pWal, &h1) ){
    return 1;







  }

  if( memcmp(&pWal->hdr, &h1, sizeof(WalIndexHdr)) ){
    *pChanged = 1;
    memcpy(&pWal->hdr, &h1, sizeof(WalIndexHdr));
    pWal->szPage = (pWal->hdr.szPage&0xfe00) + ((pWal->hdr.szPage&0x0001)<<16);
    testcase( pWal->szPage<=32768 );
................................................................................
    rc = walTryBeginRead(pWal, pChanged, 0, ++cnt);
  }while( rc==WAL_RETRY );
  testcase( (rc&0xff)==SQLITE_BUSY );
  testcase( (rc&0xff)==SQLITE_IOERR );
  testcase( rc==SQLITE_PROTOCOL );
  testcase( rc==SQLITE_OK );

  pWal->nPriorFrame = pWal->hdr.mxFrame;
#ifdef SQLITE_ENABLE_SNAPSHOT
  if( rc==SQLITE_OK ){
    if( pSnapshot && memcmp(pSnapshot, &pWal->hdr, sizeof(WalIndexHdr))!=0 ){
      /* At this point the client has a lock on an aReadMark[] slot holding
      ** a value equal to or smaller than pSnapshot->mxFrame, but pWal->hdr
      ** is populated with the wal-index header corresponding to the head
      ** of the wal file. Verify that pSnapshot is still valid before
................................................................................
  ** WAL were empty.
  */
  if( iLast==0 || (pWal->readLock==0 && pWal->bShmUnreliable==0) ){
    *piRead = 0;
    return SQLITE_OK;
  }


  /* Each iteration of the following for() loop searches one
  ** hash table (each hash table indexes up to HASHTABLE_NPAGE frames).
  **
  ** This code might run concurrently to the code in walIndexAppend()
  ** that adds entries to the wal-index (and possibly to this hash 
  ** table). This means the value just read from the hash 
  ** slot (aHash[iKey]) may have been added before or after the 
  ** current read transaction was opened. Values added after the
................................................................................
Pgno sqlite3WalDbsize(Wal *pWal){
  if( pWal && ALWAYS(pWal->readLock>=0) ){
    return pWal->hdr.nPage;
  }
  return 0;
}

/*
** Take the WRITER lock on the WAL file. Return SQLITE_OK if successful,
** or an SQLite error code otherwise. This routine does not invoke any
** busy-handler callbacks, that is done at a higher level.
*/
static int walWriteLock(Wal *pWal){
  int rc;

  /* Cannot start a write transaction without first holding a read lock */
  assert( pWal->readLock>=0 );
  assert( pWal->writeLock==0 );
  assert( pWal->iReCksum==0 );

  /* If this is a read-only connection, obtaining a write-lock is not
  ** possible. In this case return SQLITE_READONLY. Otherwise, attempt
  ** to grab the WRITER lock. Set Wal.writeLock to true and return
  ** SQLITE_OK if successful, or leave Wal.writeLock clear and return 
  ** an SQLite error code (possibly SQLITE_BUSY) otherwise. */
  if( pWal->readOnly ){
    rc = SQLITE_READONLY;
  }else{
    rc = walLockExclusive(pWal, WAL_WRITE_LOCK, 1);
    if( rc==SQLITE_OK ){
      pWal->writeLock = 1;
    }
  }

  return rc;
}

/* 
** This function starts a write transaction on the WAL.
**
** A read transaction must have already been started by a prior call
** to sqlite3WalBeginReadTransaction().
**
................................................................................
** the read transaction was started, then it is not possible for this
** thread to write as doing so would cause a fork.  So this routine
** returns SQLITE_BUSY in that case and no write transaction is started.
**
** There can only be a single writer active at a time.
*/
int sqlite3WalBeginWriteTransaction(Wal *pWal){
  int rc = walWriteLock(pWal);
  if( rc==SQLITE_OK ){
    /* If another connection has written to the database file since the
    ** time the read transaction on this connection was started, then
    ** the write is disallowed. Release the WRITER lock and return
    ** SQLITE_BUSY_SNAPSHOT in this case.  */
    if( memcmp(&pWal->hdr, (void *)walIndexHdr(pWal), sizeof(WalIndexHdr))!=0 ){
      walUnlockExclusive(pWal, WAL_WRITE_LOCK, 1);
      pWal->writeLock = 0;
      rc = SQLITE_BUSY_SNAPSHOT;
    }
  }
  return rc;
}

/*
** This function is called by a writer that has a read-lock on aReadmark[0]
** (pWal->readLock==0). This function relinquishes that lock and takes a
** lock on a different aReadmark[] slot. 
**
** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
*/
static int walUpgradeReadlock(Wal *pWal){
  int cnt;
  int rc;
  assert( pWal->writeLock && pWal->readLock==0 );
  walUnlockShared(pWal, WAL_READ_LOCK(0));
  pWal->readLock = -1;
  cnt = 0;
  do{
    int notUsed;
    rc = walTryBeginRead(pWal, &notUsed, 1, ++cnt);
  }while( rc==WAL_RETRY );
  assert( (rc&0xff)!=SQLITE_BUSY ); /* BUSY not possible when useWal==1 */
  testcase( (rc&0xff)==SQLITE_IOERR );
  testcase( rc==SQLITE_PROTOCOL );
  testcase( rc==SQLITE_OK );
  return rc;
}



#ifndef SQLITE_OMIT_CONCURRENT
/* 
** This function is only ever called when committing a "BEGIN CONCURRENT"
** transaction. It may be assumed that no frames have been written to
** the wal file. The second parameter is a pointer to the in-memory 
** representation of page 1 of the database (which may or may not be
** dirty). The third is a bitvec with a bit set for each page in the
** database file that was read by the current concurrent transaction.
**
** This function performs three tasks:
**
**   1) It obtains the WRITER lock on the wal file,
**
**   2) It checks that there are no conflicts between the current
**      transaction and any transactions committed to the wal file since
**      it was opened, and
**
**   3) It ejects any non-dirty pages from the page-cache that have been
**      written by another client since the CONCURRENT transaction was started
**      (so as to avoid ending up with an inconsistent cache after the
**      current transaction is committed).
**
** If no error occurs and the caller may proceed with committing the 
** transaction, SQLITE_OK is returned. SQLITE_BUSY is returned if the WRITER
** lock cannot be obtained. Or, if the WRITER lock can be obtained but there
** are conflicts with a committed transaction, SQLITE_BUSY_SNAPSHOT. Finally,
** if an error (i.e. an OOM condition or IO error), an SQLite error code
** is returned.
*/
int sqlite3WalLockForCommit(
  Wal *pWal, 
  PgHdr *pPage1, 
  Bitvec *pAllRead, 
  Pgno *piConflict
){
  Pager *pPager = pPage1->pPager;
  int rc = walWriteLock(pWal);

  /* If the database has been modified since this transaction was started,
  ** check if it is still possible to commit. The transaction can be 
  ** committed if:
  **
  **   a) None of the pages in pList have been modified since the 
  **      transaction opened, and
  **
  **   b) The database schema cookie has not been modified since the
  **      transaction was started.
  */
  if( rc==SQLITE_OK ){
    WalIndexHdr head;

    if( walIndexLoadHdr(pWal, &head) ){
      /* This branch is taken if the wal-index header is corrupted. This 
      ** occurs if some other writer has crashed while committing a 
      ** transaction to this database since the current concurrent transaction
      ** was opened.  */
      rc = SQLITE_BUSY_SNAPSHOT;
    }else if( memcmp(&pWal->hdr, (void*)&head, sizeof(WalIndexHdr))!=0 ){
      int iHash;
      int iLastHash = walFramePage(head.mxFrame);
      u32 iFirst = pWal->hdr.mxFrame+1;     /* First wal frame to check */
      if( memcmp(pWal->hdr.aSalt, (u32*)head.aSalt, sizeof(u32)*2) ){
        assert( pWal->readLock==0 );

        iFirst = 1;
      }


      for(iHash=walFramePage(iFirst); iHash<=iLastHash; iHash++){
        WalHashLoc sLoc;

        rc = walHashGet(pWal, iHash, &sLoc);
        if( rc==SQLITE_OK ){
          u32 i, iMin, iMax;
          assert( head.mxFrame>=sLoc.iZero );
          iMin = (sLoc.iZero >= iFirst) ? 1 : (iFirst - sLoc.iZero);
          iMax = (iHash==0) ? HASHTABLE_NPAGE_ONE : HASHTABLE_NPAGE;
          if( iMax>(head.mxFrame-sLoc.iZero) ) iMax = (head.mxFrame-sLoc.iZero);
          for(i=iMin; rc==SQLITE_OK && i<=iMax; i++){
            PgHdr *pPg;
            if( sLoc.aPgno[i]==1 ){
              /* Check that the schema cookie has not been modified. If
              ** it has not, the commit can proceed. */
              u8 aNew[4];
              u8 *aOld = &((u8*)pPage1->pData)[40];
              int sz;
              i64 iOffset;
              sz = pWal->hdr.szPage;
              sz = (sz&0xfe00) + ((sz&0x0001)<<16);
              iOffset = walFrameOffset(i+sLoc.iZero, sz) + WAL_FRAME_HDRSIZE+40;
              rc = sqlite3OsRead(pWal->pWalFd, aNew, sizeof(aNew), iOffset);
              if( rc==SQLITE_OK && memcmp(aOld, aNew, sizeof(aNew)) ){
                rc = SQLITE_BUSY_SNAPSHOT;
              }

            }else if( sqlite3BitvecTestNotNull(pAllRead, sLoc.aPgno[i]) ){
              *piConflict = sLoc.aPgno[i];
              rc = SQLITE_BUSY_SNAPSHOT;
            }else if( (pPg = sqlite3PagerLookup(pPager, sLoc.aPgno[i])) ){
              /* Page aPgno[i], which is present in the pager cache, has been
              ** modified since the current CONCURRENT transaction was started.
              ** However it was not read by the current transaction, so is not
              ** a conflict. There are two possibilities: (a) the page was
              ** allocated at the of the file by the current transaction or 
              ** (b) was present in the cache at the start of the transaction.
              **


              ** For case (a), do nothing. This page will be moved within the
              ** database file by the commit code to avoid the conflict. The
              ** call to PagerUnref() is to release the reference grabbed by
              ** the sqlite3PagerLookup() above.  
              **
              ** In case (b), drop the page from the cache - otherwise
              ** following the snapshot upgrade the cache would be inconsistent
              ** with the database as stored on disk. */
              if( sqlite3PagerIswriteable(pPg) ){
                sqlite3PagerUnref(pPg);
              }else{
                sqlite3PcacheDrop(pPg);
              }
            }
          }
        }
        if( rc!=SQLITE_OK ) break;
      }
    }
  }

  pWal->nPriorFrame = pWal->hdr.mxFrame;
  return rc;
}





/* !defined(SQLITE_OMIT_CONCURRENT)
**
** This function is called as part of committing an CONCURRENT transaction.
** It is assumed that sqlite3WalLockForCommit() has already been successfully
** called and so (a) the WRITER lock is held and (b) it is known that the
** wal-index-header stored in shared memory is not corrupt.
**
** Before returning, this function upgrades the client so that it is 
** operating on the database snapshot currently at the head of the wal file
** (even if the CONCURRENT transaction ran against an older snapshot).
**
** SQLITE_OK is returned if successful, or an SQLite error code otherwise.
*/


int sqlite3WalUpgradeSnapshot(Wal *pWal){
  int rc = SQLITE_OK;
  assert( pWal->writeLock );

  memcpy(&pWal->hdr, (void*)walIndexHdr(pWal), sizeof(WalIndexHdr));

  /* If this client has its read-lock on slot aReadmark[0] and the entire
  ** wal has not been checkpointed, switch it to a different slot. Otherwise
  ** any reads performed between now and committing the transaction will
  ** read from the old snapshot - not the one just upgraded to.  */
  if( pWal->readLock==0 && pWal->hdr.mxFrame!=walCkptInfo(pWal)->nBackfill ){
    rc = walUpgradeReadlock(pWal);
  }
  return rc;
}
#endif   /* SQLITE_OMIT_CONCURRENT */

/*
** End a write transaction.  The commit has already been done.  This
** routine merely releases the lock.
*/
int sqlite3WalEndWriteTransaction(Wal *pWal){
  if( pWal->writeLock ){
................................................................................
** to the WAL since the start of the transaction. If the callback returns
** other than SQLITE_OK, it is not invoked again and the error code is
** returned to the caller.
**
** Otherwise, if the callback function does not return an error, this
** function returns SQLITE_OK.
*/
int sqlite3WalUndo(
  Wal *pWal, 
  int (*xUndo)(void *, Pgno), 
  void *pUndoCtx,
  int bConcurrent                 /* True if this is a CONCURRENT transaction */
){
  int rc = SQLITE_OK;
  if( pWal->writeLock ){
    Pgno iMax = pWal->hdr.mxFrame;
    Pgno iFrame;
  
    /* Restore the clients cache of the wal-index header to the state it
    ** was in before the client began writing to the database. 
    */
    memcpy(&pWal->hdr, (void *)walIndexHdr(pWal), sizeof(WalIndexHdr));
#ifndef SQLITE_OMIT_CONCURRENT
    if( bConcurrent ){
      pWal->hdr.aCksum[0]++;
    }
#else
    UNUSED_PARAMETER(bConcurrent);
#endif

    for(iFrame=pWal->hdr.mxFrame+1; 
        ALWAYS(rc==SQLITE_OK) && iFrame<=iMax; 
        iFrame++
    ){
      /* This call cannot fail. Unless the page for which the page number
      ** is passed as the second argument is (a) in the cache and 
................................................................................
/* 
** Argument aWalData must point to an array of WAL_SAVEPOINT_NDATA u32 
** values. This function populates the array with values required to 
** "rollback" the write position of the WAL handle back to the current 
** point in the event of a savepoint rollback (via WalSavepointUndo()).
*/
void sqlite3WalSavepoint(Wal *pWal, u32 *aWalData){

  aWalData[0] = pWal->hdr.mxFrame;
  aWalData[1] = pWal->hdr.aFrameCksum[0];
  aWalData[2] = pWal->hdr.aFrameCksum[1];
  aWalData[3] = pWal->nCkpt;
}

/* 
................................................................................
** the values in the aWalData[] array. aWalData must point to an array
** of WAL_SAVEPOINT_NDATA u32 values that has been previously populated
** by a call to WalSavepoint().
*/
int sqlite3WalSavepointUndo(Wal *pWal, u32 *aWalData){
  int rc = SQLITE_OK;

  assert( pWal->writeLock || aWalData[0]==pWal->hdr.mxFrame );
  assert( aWalData[3]!=pWal->nCkpt || aWalData[0]<=pWal->hdr.mxFrame );

  if( aWalData[3]!=pWal->nCkpt ){
    /* This savepoint was opened immediately after the write-transaction
    ** was started. Right after that, the writer decided to wrap around
    ** to the start of the log. Update the savepoint values to match.
    */
................................................................................
**
** SQLITE_OK is returned if no error is encountered (regardless of whether
** or not pWal->hdr.mxFrame is modified). An SQLite error code is returned
** if an error occurs.
*/
static int walRestartLog(Wal *pWal){
  int rc = SQLITE_OK;


  if( pWal->readLock==0 ){
    volatile WalCkptInfo *pInfo = walCkptInfo(pWal);
    assert( pInfo->nBackfill==pWal->hdr.mxFrame );
    if( pInfo->nBackfill>0 ){
      u32 salt1;
      sqlite3FastRandomness(&pWal->sPrng, 4, &salt1);
      rc = walLockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);
      if( rc==SQLITE_OK ){
        /* If all readers are using WAL_READ_LOCK(0) (in other words if no
        ** readers are currently using the WAL), then the transactions
        ** frames will overwrite the start of the existing log. Update the
        ** wal-index header to reflect this.
        **
        ** In theory it would be Ok to update the cache of the header only
        ** at this point. But updating the actual wal-index header is also
        ** safe and means there is no special case for sqlite3WalUndo()
        ** to handle if this transaction is rolled back.  */
        walRestartHdr(pWal, salt1);
        walUnlockExclusive(pWal, WAL_READ_LOCK(1), WAL_NREADER-1);
        pWal->nPriorFrame = 0;
      }else if( rc!=SQLITE_BUSY ){
        return rc;
      }
    }


    /* Regardless of whether or not the wal file was restarted, change the
    ** read-lock held by this client to a slot other than aReadmark[0]. 
    ** Clients with a lock on aReadmark[0] read from the database file 
    ** only - never from the wal file. This means that if a writer holding
    ** a lock on aReadmark[0] were to commit a transaction but not close the
    ** read-transaction, subsequent read operations would read directly from
    ** the database file - ignoring the new pages just appended
    ** to the wal file. */
    rc = walUpgradeReadlock(pWal);

  }
  return rc;
}

/*
** Information about the current state of the WAL file and where
** the next fsync should occur - passed from sqlite3WalFrames() into
................................................................................
    u8 aWalHdr[WAL_HDRSIZE];      /* Buffer to assemble wal-header in */
    u32 aCksum[2];                /* Checksum for wal-header */

    sqlite3Put4byte(&aWalHdr[0], (WAL_MAGIC | SQLITE_BIGENDIAN));
    sqlite3Put4byte(&aWalHdr[4], WAL_MAX_VERSION);
    sqlite3Put4byte(&aWalHdr[8], szPage);
    sqlite3Put4byte(&aWalHdr[12], pWal->nCkpt);
    if( pWal->nCkpt==0 ) sqlite3FastRandomness(&pWal->sPrng, 8, pWal->hdr.aSalt);
    memcpy(&aWalHdr[16], pWal->hdr.aSalt, 8);
    walChecksumBytes(1, aWalHdr, WAL_HDRSIZE-2*4, 0, aCksum);
    sqlite3Put4byte(&aWalHdr[24], aCksum[0]);
    sqlite3Put4byte(&aWalHdr[28], aCksum[1]);
    
    pWal->szPage = szPage;
    pWal->hdr.bigEndCksum = SQLITE_BIGENDIAN;
................................................................................
    nDbSize = (isCommit && p->pDirty==0) ? nTruncate : 0;
    rc = walWriteOneFrame(&w, p, nDbSize, iOffset);
    if( rc ) return rc;
    pLast = p;
    iOffset += szFrame;
    p->flags |= PGHDR_WAL_APPEND;
  }


  /* Recalculate checksums within the wal file if required. */
  if( isCommit && pWal->iReCksum ){
    rc = walRewriteChecksums(pWal, iFrame);
    if( rc ) return rc;
  }

................................................................................
    if( isChanged && pWal->pDbFd->pMethods->iVersion>=3 ){
      sqlite3OsUnfetch(pWal->pDbFd, 0, 0);
    }
  }

  /* Copy data from the log to the database file. */
  if( rc==SQLITE_OK ){

    if( pWal->hdr.mxFrame && walPagesize(pWal)!=nBuf ){
      rc = SQLITE_CORRUPT_BKPT;
    }else{
      rc = walCheckpoint(pWal, db, eMode2, xBusy2, pBusyArg, sync_flags, zBuf);
    }

    /* If no error occurred, set the output variables. */
................................................................................
#endif

/* Return the sqlite3_file object for the WAL file
*/
sqlite3_file *sqlite3WalFile(Wal *pWal){
  return pWal->pWalFd;
}

/* 
** Return the values required by sqlite3_wal_info().
*/
int sqlite3WalInfo(Wal *pWal, u32 *pnPrior, u32 *pnFrame){
  int rc = SQLITE_OK;
  if( pWal ){
    *pnFrame = pWal->hdr.mxFrame;
    *pnPrior = pWal->nPriorFrame;
  }
  return rc;
}

#endif /* #ifndef SQLITE_OMIT_WAL */

Changes to src/wal.h.

30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
..
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
...
132
133
134
135
136
137
138









139
140
141
142
143
144
145
146
147
148



149
150
# define sqlite3WalLimit(x,y)
# define sqlite3WalClose(v,w,x,y,z)              0
# define sqlite3WalBeginReadTransaction(y,z)     0
# define sqlite3WalEndReadTransaction(z)
# define sqlite3WalDbsize(y)                     0
# define sqlite3WalBeginWriteTransaction(y)      0
# define sqlite3WalEndWriteTransaction(x)        0
# define sqlite3WalUndo(x,y,z)                   0
# define sqlite3WalSavepoint(y,z)
# define sqlite3WalSavepointUndo(y,z)            0
# define sqlite3WalFrames(u,v,w,x,y,z)           0
# define sqlite3WalCheckpoint(q,r,s,t,u,v,w,x,y,z) 0
# define sqlite3WalCallback(z)                   0
# define sqlite3WalExclusiveMode(y,z)            0
# define sqlite3WalHeapMemory(z)                 0
................................................................................
Pgno sqlite3WalDbsize(Wal *pWal);

/* Obtain or release the WRITER lock. */
int sqlite3WalBeginWriteTransaction(Wal *pWal);
int sqlite3WalEndWriteTransaction(Wal *pWal);

/* Undo any frames written (but not committed) to the log */
int sqlite3WalUndo(Wal *pWal, int (*xUndo)(void *, Pgno), void *pUndoCtx);

/* Return an integer that records the current (uncommitted) write
** position in the WAL */
void sqlite3WalSavepoint(Wal *pWal, u32 *aWalData);

/* Move the write position of the WAL back to iFrame.  Called in
** response to a ROLLBACK TO command. */
................................................................................
int sqlite3WalSnapshotGet(Wal *pWal, sqlite3_snapshot **ppSnapshot);
void sqlite3WalSnapshotOpen(Wal *pWal, sqlite3_snapshot *pSnapshot);
int sqlite3WalSnapshotRecover(Wal *pWal);
int sqlite3WalSnapshotCheck(Wal *pWal, sqlite3_snapshot *pSnapshot);
void sqlite3WalSnapshotUnlock(Wal *pWal);
#endif










#ifdef SQLITE_ENABLE_ZIPVFS
/* If the WAL file is not empty, return the number of bytes of content
** stored in each frame (i.e. the db page-size when the WAL was created).
*/
int sqlite3WalFramesize(Wal *pWal);
#endif

/* Return the sqlite3_file object for the WAL file */
sqlite3_file *sqlite3WalFile(Wal *pWal);




#endif /* ifndef SQLITE_OMIT_WAL */
#endif /* SQLITE_WAL_H */







|







 







|







 







>
>
>
>
>
>
>
>
>










>
>
>


30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
..
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
...
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
# define sqlite3WalLimit(x,y)
# define sqlite3WalClose(v,w,x,y,z)              0
# define sqlite3WalBeginReadTransaction(y,z)     0
# define sqlite3WalEndReadTransaction(z)
# define sqlite3WalDbsize(y)                     0
# define sqlite3WalBeginWriteTransaction(y)      0
# define sqlite3WalEndWriteTransaction(x)        0
# define sqlite3WalUndo(w,x,y,z)                 0
# define sqlite3WalSavepoint(y,z)
# define sqlite3WalSavepointUndo(y,z)            0
# define sqlite3WalFrames(u,v,w,x,y,z)           0
# define sqlite3WalCheckpoint(q,r,s,t,u,v,w,x,y,z) 0
# define sqlite3WalCallback(z)                   0
# define sqlite3WalExclusiveMode(y,z)            0
# define sqlite3WalHeapMemory(z)                 0
................................................................................
Pgno sqlite3WalDbsize(Wal *pWal);

/* Obtain or release the WRITER lock. */
int sqlite3WalBeginWriteTransaction(Wal *pWal);
int sqlite3WalEndWriteTransaction(Wal *pWal);

/* Undo any frames written (but not committed) to the log */
int sqlite3WalUndo(Wal *pWal, int (*xUndo)(void *, Pgno), void *pUndoCtx, int);

/* Return an integer that records the current (uncommitted) write
** position in the WAL */
void sqlite3WalSavepoint(Wal *pWal, u32 *aWalData);

/* Move the write position of the WAL back to iFrame.  Called in
** response to a ROLLBACK TO command. */
................................................................................
int sqlite3WalSnapshotGet(Wal *pWal, sqlite3_snapshot **ppSnapshot);
void sqlite3WalSnapshotOpen(Wal *pWal, sqlite3_snapshot *pSnapshot);
int sqlite3WalSnapshotRecover(Wal *pWal);
int sqlite3WalSnapshotCheck(Wal *pWal, sqlite3_snapshot *pSnapshot);
void sqlite3WalSnapshotUnlock(Wal *pWal);
#endif

#ifndef SQLITE_OMIT_CONCURRENT
/* Tell the wal layer that we want to commit a concurrent transaction */
int sqlite3WalLockForCommit(Wal *pWal, PgHdr *pPg, Bitvec *pRead, Pgno*);

/* Upgrade the state of the client to take into account changes written
** by other connections */
int sqlite3WalUpgradeSnapshot(Wal *pWal);
#endif /* SQLITE_OMIT_CONCURRENT */

#ifdef SQLITE_ENABLE_ZIPVFS
/* If the WAL file is not empty, return the number of bytes of content
** stored in each frame (i.e. the db page-size when the WAL was created).
*/
int sqlite3WalFramesize(Wal *pWal);
#endif

/* Return the sqlite3_file object for the WAL file */
sqlite3_file *sqlite3WalFile(Wal *pWal);

/* sqlite3_wal_info() data */
int sqlite3WalInfo(Wal *pWal, u32 *pnPrior, u32 *pnFrame);

#endif /* ifndef SQLITE_OMIT_WAL */
#endif /* SQLITE_WAL_H */

Added test/bc_test1.c.

























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
/*
** 2016-05-07
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
*/


#include <sqlite3.h>
#include <stdlib.h>
#include <stddef.h>
#include "tt3_core.c"

#ifdef USE_OSINST
# include "../src/test_osinst.c"
#else
# define vfslog_time() 0
#endif

typedef struct Config Config;
typedef struct ThreadCtx ThreadCtx;

#define THREAD_TIME_INSERT   0
#define THREAD_TIME_COMMIT   1
#define THREAD_TIME_ROLLBACK 2
#define THREAD_TIME_WRITER   3
#define THREAD_TIME_CKPT     4

struct ThreadCtx {
  Config *pConfig;
  Sqlite *pDb;
  Error *pErr;
  sqlite3_int64 aTime[5];
};

struct Config {
  int nIPT;                       /* --inserts-per-transaction */
  int nThread;                    /* --threads */
  int nSecond;                    /* --seconds */
  int bMutex;                     /* --mutex */
  int nAutoCkpt;                  /* --autockpt */
  int bRm;                        /* --rm */
  int bClearCache;                /* --clear-cache */
  int nMmap;                      /* mmap limit in MB */
  char *zFile;
  int bOsinst;                    /* True to use osinst */

  ThreadCtx *aCtx;                /* Array of size nThread */

  pthread_cond_t cond;
  pthread_mutex_t mutex;
  int nCondWait;                  /* Number of threads waiting on hCond */
  sqlite3_vfs *pVfs;
};


typedef struct VfsWrapperFd VfsWrapperFd;
struct VfsWrapperFd {
  sqlite3_file base;              /* Base class */
  int bWriter;                    /* True if holding shm WRITER lock */
  int iTid;
  Config *pConfig;
  sqlite3_file *pFd;              /* Underlying file descriptor */
};

/* Methods of the wrapper VFS */
static int vfsWrapOpen(sqlite3_vfs*, const char*, sqlite3_file*, int, int*);
static int vfsWrapDelete(sqlite3_vfs*, const char*, int);
static int vfsWrapAccess(sqlite3_vfs*, const char*, int, int*);
static int vfsWrapFullPathname(sqlite3_vfs*, const char *, int, char*);
static void *vfsWrapDlOpen(sqlite3_vfs*, const char*);
static void vfsWrapDlError(sqlite3_vfs*, int, char*);
static void (*vfsWrapDlSym(sqlite3_vfs*,void*, const char*))(void);
static void vfsWrapDlClose(sqlite3_vfs*, void*);
static int vfsWrapRandomness(sqlite3_vfs*, int, char*);
static int vfsWrapSleep(sqlite3_vfs*, int);
static int vfsWrapCurrentTime(sqlite3_vfs*, double*);
static int vfsWrapGetLastError(sqlite3_vfs*, int, char*);
static int vfsWrapCurrentTimeInt64(sqlite3_vfs*, sqlite3_int64*);
static int vfsWrapSetSystemCall(sqlite3_vfs*, const char*, sqlite3_syscall_ptr);
static sqlite3_syscall_ptr vfsWrapGetSystemCall(sqlite3_vfs*, const char*);
static const char *vfsWrapNextSystemCall(sqlite3_vfs*, const char*);

/* Methods of wrapper sqlite3_io_methods object (see vfsWrapOpen()) */
static int vfsWrapClose(sqlite3_file*);
static int vfsWrapRead(sqlite3_file*, void*, int iAmt, sqlite3_int64 iOfst);
static int vfsWrapWrite(sqlite3_file*, const void*, int iAmt, sqlite3_int64);
static int vfsWrapTruncate(sqlite3_file*, sqlite3_int64 size);
static int vfsWrapSync(sqlite3_file*, int flags);
static int vfsWrapFileSize(sqlite3_file*, sqlite3_int64 *pSize);
static int vfsWrapLock(sqlite3_file*, int);
static int vfsWrapUnlock(sqlite3_file*, int);
static int vfsWrapCheckReservedLock(sqlite3_file*, int *pResOut);
static int vfsWrapFileControl(sqlite3_file*, int op, void *pArg);
static int vfsWrapSectorSize(sqlite3_file*);
static int vfsWrapDeviceCharacteristics(sqlite3_file*);
static int vfsWrapShmMap(sqlite3_file*, int iPg, int, int, void volatile**);
static int vfsWrapShmLock(sqlite3_file*, int offset, int n, int flags);
static void vfsWrapShmBarrier(sqlite3_file*);
static int vfsWrapShmUnmap(sqlite3_file*, int deleteFlag);
static int vfsWrapFetch(sqlite3_file*, sqlite3_int64 iOfst, int iAmt, void **);
static int vfsWrapUnfetch(sqlite3_file*, sqlite3_int64 iOfst, void *p);

static int vfsWrapOpen(
  sqlite3_vfs *pVfs, 
  const char *zName, 
  sqlite3_file *pFd, 
  int flags, 
  int *fout
){
  static sqlite3_io_methods methods = {
    3,
    vfsWrapClose, vfsWrapRead, vfsWrapWrite,
    vfsWrapTruncate, vfsWrapSync, vfsWrapFileSize,
    vfsWrapLock, vfsWrapUnlock, vfsWrapCheckReservedLock,
    vfsWrapFileControl, vfsWrapSectorSize, vfsWrapDeviceCharacteristics,
    vfsWrapShmMap, vfsWrapShmLock, vfsWrapShmBarrier,
    vfsWrapShmUnmap, vfsWrapFetch, vfsWrapUnfetch
  };

  Config *pConfig = (Config*)pVfs->pAppData;
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  int rc;

  memset(pWrapper, 0, sizeof(VfsWrapperFd));
  if( flags & SQLITE_OPEN_MAIN_DB ){
    pWrapper->iTid = (int)sqlite3_uri_int64(zName, "tid", 0);
  }

  pWrapper->pFd = (sqlite3_file*)&pWrapper[1];
  pWrapper->pConfig = pConfig;
  rc = pConfig->pVfs->xOpen(pConfig->pVfs, zName, pWrapper->pFd, flags, fout);
  if( rc==SQLITE_OK ){
    pWrapper->base.pMethods = &methods;
  }
  return rc;
}

static int vfsWrapDelete(sqlite3_vfs *pVfs, const char *a, int b){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xDelete(pConfig->pVfs, a, b);
}
static int vfsWrapAccess(sqlite3_vfs *pVfs, const char *a, int b, int *c){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xAccess(pConfig->pVfs, a, b, c);
}
static int vfsWrapFullPathname(sqlite3_vfs *pVfs, const char *a, int b, char*c){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xFullPathname(pConfig->pVfs, a, b, c);
}
static void *vfsWrapDlOpen(sqlite3_vfs *pVfs, const char *a){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xDlOpen(pConfig->pVfs, a);
}
static void vfsWrapDlError(sqlite3_vfs *pVfs, int a, char *b){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xDlError(pConfig->pVfs, a, b);
}
static void (*vfsWrapDlSym(sqlite3_vfs *pVfs, void *a, const char *b))(void){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xDlSym(pConfig->pVfs, a, b);
}
static void vfsWrapDlClose(sqlite3_vfs *pVfs, void *a){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xDlClose(pConfig->pVfs, a);
}
static int vfsWrapRandomness(sqlite3_vfs *pVfs, int a, char *b){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xRandomness(pConfig->pVfs, a, b);
}
static int vfsWrapSleep(sqlite3_vfs *pVfs, int a){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xSleep(pConfig->pVfs, a);
}
static int vfsWrapCurrentTime(sqlite3_vfs *pVfs, double *a){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xCurrentTime(pConfig->pVfs, a);
}
static int vfsWrapGetLastError(sqlite3_vfs *pVfs, int a, char *b){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xGetLastError(pConfig->pVfs, a, b);
}
static int vfsWrapCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *a){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xCurrentTimeInt64(pConfig->pVfs, a);
}
static int vfsWrapSetSystemCall(
  sqlite3_vfs *pVfs, 
  const char *a, 
  sqlite3_syscall_ptr b
){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xSetSystemCall(pConfig->pVfs, a, b);
}
static sqlite3_syscall_ptr vfsWrapGetSystemCall(
  sqlite3_vfs *pVfs, 
  const char *a
){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xGetSystemCall(pConfig->pVfs, a);
}
static const char *vfsWrapNextSystemCall(sqlite3_vfs *pVfs, const char *a){
  Config *pConfig = (Config*)pVfs->pAppData;
  return pConfig->pVfs->xNextSystemCall(pConfig->pVfs, a);
}

static int vfsWrapClose(sqlite3_file *pFd){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  pWrapper->pFd->pMethods->xClose(pWrapper->pFd);
  pWrapper->pFd = 0;
  return SQLITE_OK;
}
static int vfsWrapRead(sqlite3_file *pFd, void *a, int b, sqlite3_int64 c){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xRead(pWrapper->pFd, a, b, c);
}
static int vfsWrapWrite(
  sqlite3_file *pFd, 
  const void *a, int b, 
  sqlite3_int64 c
){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xWrite(pWrapper->pFd, a, b, c);
}
static int vfsWrapTruncate(sqlite3_file *pFd, sqlite3_int64 a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xTruncate(pWrapper->pFd, a);
}
static int vfsWrapSync(sqlite3_file *pFd, int a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xSync(pWrapper->pFd, a);
}
static int vfsWrapFileSize(sqlite3_file *pFd, sqlite3_int64 *a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xFileSize(pWrapper->pFd, a);
}
static int vfsWrapLock(sqlite3_file *pFd, int a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xLock(pWrapper->pFd, a);
}
static int vfsWrapUnlock(sqlite3_file *pFd, int a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xUnlock(pWrapper->pFd, a);
}
static int vfsWrapCheckReservedLock(sqlite3_file *pFd, int *a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xCheckReservedLock(pWrapper->pFd, a);
}
static int vfsWrapFileControl(sqlite3_file *pFd, int a, void *b){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xFileControl(pWrapper->pFd, a, b);
}
static int vfsWrapSectorSize(sqlite3_file *pFd){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xSectorSize(pWrapper->pFd);
}
static int vfsWrapDeviceCharacteristics(sqlite3_file *pFd){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xDeviceCharacteristics(pWrapper->pFd);
}
static int vfsWrapShmMap(
  sqlite3_file *pFd, 
  int a, int b, int c, 
  void volatile **d
){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xShmMap(pWrapper->pFd, a, b, c, d);
}
static int vfsWrapShmLock(sqlite3_file *pFd, int offset, int n, int flags){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  Config *pConfig = pWrapper->pConfig;
  int bMutex = 0;
  int rc;

  if(  (offset==0 && n==1)
    && (flags & SQLITE_SHM_LOCK) && (flags & SQLITE_SHM_EXCLUSIVE)
  ){
    pthread_mutex_lock(&pConfig->mutex);
    pWrapper->bWriter = 1;
    bMutex = 1;
    if( pWrapper->iTid ){
      sqlite3_int64 t = vfslog_time();
      pConfig->aCtx[pWrapper->iTid-1].aTime[THREAD_TIME_WRITER] -= t;
    }
  }

  rc = pWrapper->pFd->pMethods->xShmLock(pWrapper->pFd, offset, n, flags);

  if( (rc!=SQLITE_OK && bMutex)
   || (offset==0 && (flags & SQLITE_SHM_UNLOCK) && pWrapper->bWriter)
  ){
    assert( pWrapper->bWriter );
    pthread_mutex_unlock(&pConfig->mutex);
    pWrapper->bWriter = 0;
    if( pWrapper->iTid ){
      sqlite3_int64 t = vfslog_time();
      pConfig->aCtx[pWrapper->iTid-1].aTime[THREAD_TIME_WRITER] += t;
    }
  }

  return rc;
}
static void vfsWrapShmBarrier(sqlite3_file *pFd){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xShmBarrier(pWrapper->pFd);
}
static int vfsWrapShmUnmap(sqlite3_file *pFd, int a){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xShmUnmap(pWrapper->pFd, a);
}
static int vfsWrapFetch(sqlite3_file *pFd, sqlite3_int64 a, int b, void **c){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xFetch(pWrapper->pFd, a, b, c);
}
static int vfsWrapUnfetch(sqlite3_file *pFd, sqlite3_int64 a, void *b){
  VfsWrapperFd *pWrapper = (VfsWrapperFd*)pFd;
  return pWrapper->pFd->pMethods->xUnfetch(pWrapper->pFd, a, b);
}

static void create_vfs(Config *pConfig){
  static sqlite3_vfs vfs = {
    3, 0, 0, 0, "wrapper", 0,
    vfsWrapOpen, vfsWrapDelete, vfsWrapAccess,
    vfsWrapFullPathname, vfsWrapDlOpen, vfsWrapDlError,
    vfsWrapDlSym, vfsWrapDlClose, vfsWrapRandomness,
    vfsWrapSleep, vfsWrapCurrentTime, vfsWrapGetLastError,
    vfsWrapCurrentTimeInt64, vfsWrapSetSystemCall, vfsWrapGetSystemCall,
    vfsWrapNextSystemCall
  };
  sqlite3_vfs *pVfs;

  pVfs = sqlite3_vfs_find(0);
  vfs.mxPathname = pVfs->mxPathname;
  vfs.szOsFile = pVfs->szOsFile + sizeof(VfsWrapperFd);
  vfs.pAppData = (void*)pConfig;
  pConfig->pVfs = pVfs;

  sqlite3_vfs_register(&vfs, 1);
}


/*
** Wal hook used by connections in thread_main().
*/
static int thread_wal_hook(
  void *pArg,                     /* Pointer to ThreadCtx object */
  sqlite3 *db,
  const char *zDb, 
  int nFrame
){
  ThreadCtx *pCtx = (ThreadCtx*)pArg;
  Config *pConfig = pCtx->pConfig;

  if( pConfig->nAutoCkpt && nFrame>=pConfig->nAutoCkpt ){
    pCtx->aTime[THREAD_TIME_CKPT] -= vfslog_time();
    pthread_mutex_lock(&pConfig->mutex);
    if( pConfig->nCondWait>=0 ){
      pConfig->nCondWait++;
      if( pConfig->nCondWait==pConfig->nThread ){
        execsql(pCtx->pErr, pCtx->pDb, "PRAGMA wal_checkpoint");
        pthread_cond_broadcast(&pConfig->cond);
      }else{
        pthread_cond_wait(&pConfig->cond, &pConfig->mutex);
      }
      pConfig->nCondWait--;
    }
    pthread_mutex_unlock(&pConfig->mutex);
    pCtx->aTime[THREAD_TIME_CKPT] += vfslog_time();
  }

  return SQLITE_OK;
}


static char *thread_main(int iTid, void *pArg){
  Config *pConfig = (Config*)pArg;
  Error err = {0};                /* Error code and message */
  Sqlite db = {0};                /* SQLite database connection */
  int nAttempt = 0;               /* Attempted transactions */
  int nCommit = 0;                /* Successful transactions */
  int j;
  ThreadCtx *pCtx = &pConfig->aCtx[iTid-1];
  char *zUri = 0;

#ifdef USE_OSINST
  char *zOsinstName = 0;
  char *zLogName = 0;
  if( pConfig->bOsinst ){
    zOsinstName = sqlite3_mprintf("osinst%d", iTid);
    zLogName = sqlite3_mprintf("bc_test1.log.%d.%d", (int)getpid(), iTid);
    zUri = sqlite3_mprintf(
        "file:%s?vfs=%s&tid=%d", pConfig->zFile, zOsinstName, iTid
    );
    sqlite3_vfslog_new(zOsinstName, 0, zLogName);
    opendb(&err, &db, zUri, 0);
  }else
#endif
  {
    zUri = sqlite3_mprintf("file:%s?tid=%d", pConfig->zFile, iTid);
    opendb(&err, &db, zUri, 0);
  }

  sqlite3_busy_handler(db.db, 0, 0);
  sql_script_printf(&err, &db, 
      "PRAGMA wal_autocheckpoint = 0;"
      "PRAGMA synchronous = 0;"
      "PRAGMA mmap_size = %lld;",
      (i64)(pConfig->nMmap) * 1024 * 1024
  );

  pCtx->pConfig = pConfig;
  pCtx->pErr = &err;
  pCtx->pDb = &db;
  sqlite3_wal_hook(db.db, thread_wal_hook, (void*)pCtx);

  while( !timetostop(&err) ){
    execsql(&err, &db, "BEGIN CONCURRENT");

    pCtx->aTime[THREAD_TIME_INSERT] -= vfslog_time();
    for(j=0; j<pConfig->nIPT; j++){
      execsql(&err, &db, 
          "INSERT INTO t1 VALUES"
          "(randomblob(10), randomblob(20), randomblob(30), randomblob(200))"
      );
    }
    pCtx->aTime[THREAD_TIME_INSERT] += vfslog_time();

    pCtx->aTime[THREAD_TIME_COMMIT] -= vfslog_time();
    execsql(&err, &db, "COMMIT");
    pCtx->aTime[THREAD_TIME_COMMIT] += vfslog_time();

    pCtx->aTime[THREAD_TIME_ROLLBACK] -= vfslog_time();
    nAttempt++;
    if( err.rc==SQLITE_OK ){
      nCommit++;
    }else{
      clear_error(&err, SQLITE_BUSY);
      execsql(&err, &db, "ROLLBACK");
    }
    pCtx->aTime[THREAD_TIME_ROLLBACK] += vfslog_time();

    if( pConfig->bClearCache ){
      sqlite3_db_release_memory(db.db);
    }
  }

  closedb(&err, &db);

#ifdef USE_OSINST
  if( pConfig->bOsinst ){
    sqlite3_vfslog_finalize(zOsinstName);
    sqlite3_free(zOsinstName);
    sqlite3_free(zLogName);
  }
#endif
  sqlite3_free(zUri);

  pthread_mutex_lock(&pConfig->mutex);
  pConfig->nCondWait = -1;
  pthread_cond_broadcast(&pConfig->cond);
  pthread_mutex_unlock(&pConfig->mutex);

  return sqlite3_mprintf("commits: %d/%d insert: %dms"
      " commit: %dms"
      " rollback: %dms" 
      " writer: %dms"
      " checkpoint: %dms", 
      nCommit, nAttempt, 
      (int)(pCtx->aTime[THREAD_TIME_INSERT]/1000), 
      (int)(pCtx->aTime[THREAD_TIME_COMMIT]/1000), 
      (int)(pCtx->aTime[THREAD_TIME_ROLLBACK]/1000),
      (int)(pCtx->aTime[THREAD_TIME_WRITER]/1000),
      (int)(pCtx->aTime[THREAD_TIME_CKPT]/1000)
  );
}

int main(int argc, const char **argv){
  Error err = {0};                /* Error code and message */
  Sqlite db = {0};                /* SQLite database connection */
  Threadset threads = {0};        /* Test threads */
  Config conf = {5, 3, 5};
  int i;

  CmdlineArg apArg[] = {
    { "-seconds", CMDLINE_INT,  offsetof(Config, nSecond) },
    { "-inserts", CMDLINE_INT,  offsetof(Config, nIPT) },
    { "-threads", CMDLINE_INT,  offsetof(Config, nThread) },
    { "-mutex",   CMDLINE_BOOL, offsetof(Config, bMutex) },
    { "-rm",      CMDLINE_BOOL, offsetof(Config, bRm) },
    { "-autockpt",CMDLINE_INT,  offsetof(Config, nAutoCkpt) },
    { "-mmap",    CMDLINE_INT,  offsetof(Config, nMmap) },
    { "-clear-cache",    CMDLINE_BOOL,  offsetof(Config, bClearCache) },
    { "-file",    CMDLINE_STRING,  offsetof(Config, zFile) },
    { "-osinst",  CMDLINE_BOOL,  offsetof(Config, bOsinst) },
    { 0, 0, 0 }
  };

  conf.nAutoCkpt = 1000;
  cmdline_process(apArg, argc, argv, (void*)&conf);
  if( err.rc==SQLITE_OK ){
    char *z = cmdline_construct(apArg, (void*)&conf);
    printf("With: %s\n", z);
    sqlite3_free(z);
  }
  if( conf.zFile==0 ){
    conf.zFile = "xyz.db";
  }

  /* Create the special VFS - "wrapper". And the mutex and condition 
  ** variable. */
  create_vfs(&conf);
  pthread_mutex_init(&conf.mutex, 0);
  pthread_cond_init(&conf.cond, 0);

  conf.aCtx = sqlite3_malloc(sizeof(ThreadCtx) * conf.nThread);
  memset(conf.aCtx, 0, sizeof(ThreadCtx) * conf.nThread);

  /* Ensure the schema has been created */
  opendb(&err, &db, conf.zFile, conf.bRm);
  sql_script(&err, &db,
      "PRAGMA journal_mode = wal;"
      "CREATE TABLE IF NOT EXISTS t1(a PRIMARY KEY, b, c, d) WITHOUT ROWID;"
      "CREATE INDEX IF NOT EXISTS t1b ON t1(b);"
      "CREATE INDEX IF NOT EXISTS t1c ON t1(c);"
  );

  setstoptime(&err, conf.nSecond*1000);
  if( conf.nThread==1 ){
    char *z = thread_main(1, (void*)&conf);
    printf("Thread 0 says: %s\n", (z==0 ? "..." : z));
    fflush(stdout);
  }else{
    for(i=0; i<conf.nThread; i++){
      launch_thread(&err, &threads, thread_main, (void*)&conf);
    }
    join_all_threads(&err, &threads);
  }

  if( err.rc==SQLITE_OK ){
    printf("Database is %dK\n", (int)(filesize(&err, conf.zFile) / 1024));
  }
  if( err.rc==SQLITE_OK ){
    char *zWal = sqlite3_mprintf("%s-wal", conf.zFile);
    printf("Wal file is %dK\n", (int)(filesize(&err, zWal) / 1024));
  }

  closedb(&err, &db);
  print_and_free_err(&err);
  return 0;
}

Added test/concfault.test.































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
# 2015 Aug 25
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# This file contains fault injection tests designed to test the concurrent
# transactions feature.
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/malloc_common.tcl
set testprefix concfault

# This test will not work with an in-memory journal, as the database will
# become corrupt if an error is injected into a transaction after it starts
# writing data out to the db file.
ifcapable !concurrent {
  finish_test
  return
}

do_test 1-pre1 {
  execsql {
    PRAGMA journal_mode = wal;
    CREATE TABLE t1(a PRIMARY KEY, b);
    INSERT INTO t1 VALUES(randomblob(1000), randomblob(100));
    INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
    INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
    INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
    INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
    DELETE FROM t1 WHERE rowid%2;
  }
  faultsim_save_and_close
} {}

do_faultsim_test 1.1 -prep {
  faultsim_restore_and_reopen
} -body {
  execsql { 
    BEGIN CONCURRENT;
      INSERT INTO t1 VALUES(randomblob(1000), randomblob(100));
    COMMIT;
  }
} -test {
  faultsim_test_result {0 {}} 
  catchsql { ROLLBACK }
  faultsim_integrity_check
}

do_faultsim_test 1.2 -prep {
  faultsim_restore_and_reopen
} -body {
  execsql { 
    BEGIN CONCURRENT;
      INSERT INTO t1 VALUES(randomblob(1000), randomblob(100));
    ROLLBACK;
  }
} -test {
  faultsim_test_result {0 {}} 
  catchsql { ROLLBACK }
  faultsim_integrity_check
}

do_faultsim_test 1.3 -prep {
  faultsim_restore_and_reopen
} -body {
  execsql { 
    BEGIN CONCURRENT;
      DELETE FROM t1;
    COMMIT;
  }
} -test {
  faultsim_test_result {0 {}} 
  catchsql { ROLLBACK }
  faultsim_integrity_check
}


#-------------------------------------------------------------------------
reset_db

do_execsql_test 2.0 {
  PRAGMA auto_vacuum = 0;
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(a PRIMARY KEY, b);
  CREATE TABLE t2(a PRIMARY KEY, b);
  INSERT INTO t1 VALUES(randomblob(1000), randomblob(100));
  INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
  INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
  INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
  INSERT INTO t1 SELECT randomblob(1000), randomblob(1000) FROM t1;
  DELETE FROM t1 WHERE rowid%2;
} {wal}

faultsim_save_and_close
do_faultsim_test 1 -prep {
  faultsim_restore_and_reopen
  execsql {
    SELECT * FROM t1;
    BEGIN CONCURRENT;
      INSERT INTO t2 VALUES(1, 2);
  }
  sqlite3 db2 test.db
  execsql {
    PRAGMA journal_size_limit = 10000;
    INSERT INTO t1 VALUES(randomblob(1000), randomblob(1000));
  } db2
  db2 close
} -body {
  execsql { COMMIT }
} -test {
  faultsim_test_result {0 {}} 
  catchsql { ROLLBACK }
  set res [catchsql { SELECT count(*) FROM t1 }]
  if {$res!="0 9"} { error "expected {0 9} got {$res}" }
  faultsim_integrity_check
}

finish_test

Added test/concurrent.test.













































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
# 2015 July 26
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/lock_common.tcl
set ::testprefix concurrent

ifcapable !concurrent {
  finish_test
  return
}

do_execsql_test 1.0 {
  PRAGMA journal_mode = wal;
} {wal}

do_execsql_test 1.1 {
  CREATE TABLE t1(k INTEGER PRIMARY KEY, v);
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES(1, 'abcd');
  COMMIT;
}

do_execsql_test 1.2 {
  SELECT * FROM t1;
} {1 abcd}

do_execsql_test 1.3 {
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES(2, 'efgh');
  ROLLBACK;
}

do_execsql_test 1.4 {
  SELECT * FROM t1;
} {1 abcd}


#-------------------------------------------------------------------------
# CONCURRENT transactions cannot do cache spills.
#
foreach {tn trans spill} {
  1 {BEGIN CONCURRENT}  0
  2 {BEGIN}           1
} {
  do_test 1.5.$tn {
    sqlite3 db2 test.db
    set walsz [file size test.db-wal]

    execsql { PRAGMA cache_size = 10 } db2
    execsql $trans db2
    execsql {
      WITH cnt(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM cnt WHERE i<50)
        INSERT INTO t1(v) SELECT randomblob(900) FROM cnt;
    } db2

    expr {[file size test.db-wal]==$walsz}
  } [expr !$spill]

  execsql ROLLBACK db2
  db2 close
}

#-------------------------------------------------------------------------
# CONCURRENT transactions man not be committed while there are active
# readers.
do_execsql_test 1.6.setup {
  DROP TABLE t1;
  CREATE TABLE t1(a, b);
  INSERT INTO t1 VALUES(1, 2);
  INSERT INTO t1 VALUES(3, 4);
  INSERT INTO t1 VALUES(5, 6);
}
foreach {tn trans commit_ok} {
  1 {BEGIN CONCURRENT}  0
  2 {BEGIN}           1
} {
  do_test 1.6.$tn.1 {
    set stmt [sqlite3_prepare db "SELECT * FROM t1" -1 dummy]
    sqlite3_step $stmt
  } SQLITE_ROW
  do_test 1.6.$tn.2 {
    execsql $trans
    execsql { INSERT INTO t1 VALUES(7, 8) }
  } {}

  if { $commit_ok } {
    do_test 1.6.$tn.3 { catchsql COMMIT } {0 {}}
  } else {
    do_test 1.6.$tn.4 { catchsql COMMIT } {/1 {cannot commit transaction .*}/}
  }

  sqlite3_finalize $stmt
  catchsql ROLLBACK
}

#-------------------------------------------------------------------------
# CONCURRENT transactions may not modify the db schema.
#
foreach {tn sql} {
  1 { CREATE TABLE xx(a, b) }
  2 { DROP TABLE t1 }
  3 { CREATE INDEX i1 ON t1(a) }
  4 { CREATE VIEW v1 AS SELECT * FROM t1 }
} {
  do_catchsql_test 1.7.0.$tn.1 "
    BEGIN CONCURRENT;
    $sql
  " {1 {cannot modify database schema within CONCURRENT transaction}}

  do_execsql_test 1.7.0.$tn.2 {
    SELECT sql FROM sqlite_master;
    SELECT sql FROM sqlite_temp_master;
  } {{CREATE TABLE t1(a, b)}}

  do_execsql_test 1.7.0.$tn.3 COMMIT
}

# Except the temp db schema.
foreach {tn sql} {
  1 { CREATE TEMP TABLE xx(a, b) }
  2 { DROP TABLE xx }
  3 { CREATE TEMP TABLE yy(a, b) }
  4 { CREATE VIEW temp.v1 AS SELECT * FROM t1 }
  5 { CREATE INDEX yyi1 ON yy(a); }
  6 { CREATE TABLE temp.zz(a, b) }
} {
  do_catchsql_test 1.7.1.$tn.1 "
    BEGIN CONCURRENT;
    $sql
  " {0 {}}

  do_execsql_test 1.7.1.$tn.2 COMMIT
}


do_execsql_test 1.7.1.x {
  SELECT sql FROM sqlite_master;
  SELECT sql FROM sqlite_temp_master;
} {
  {CREATE TABLE t1(a, b)}
  {CREATE TABLE yy(a, b)} 
  {CREATE VIEW v1 AS SELECT * FROM t1} 
  {CREATE INDEX yyi1 ON yy(a)} 
  {CREATE TABLE zz(a, b)}
}

#-------------------------------------------------------------------------
# If an auto-vacuum database is written within an CONCURRENT transaction, it
# is handled in the same way as for a non-CONCURRENT transaction.
#
reset_db
do_execsql_test 1.8.1 {
  PRAGMA auto_vacuum = 1;
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x, y);
  INSERT INTO t1 VALUES('x', 'y');
} {wal}

do_execsql_test 1.8.2 {
  BEGIN CONCURRENT;
    SELECT * FROM t1;
  COMMIT;
} {x y}

do_catchsql_test 1.8.3 {
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES('a', 'b');
} {0 {}}

do_test 1.8.4 {
  sqlite3 db2 test.db
  catchsql {
    BEGIN CONCURRENT;
      INSERT INTO t1 VALUES('c', 'd');
  } db2
} {1 {database is locked}}

do_test 1.8.5 {
  db eval COMMIT
  db2 eval COMMIT
} {}
db close
db2 close

do_multiclient_test tn {

  #-----------------------------------------------------------------------
  # 1. Start an CONCURRENT transaction using [db1].
  #
  # 2. Start and then rollback a regular transaction using [db2]. This 
  #    can be done as the ongoing [db1] transaction is CONCURRENT.
  #
  # 3. The [db1] transaction can now be committed, as [db2] has relinquished
  #    the write lock.
  #
  do_test 2.$tn.1.1 {
    sql1 { 
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(k INTEGER PRIMARY KEY, v);
      INSERT INTO t1 VALUES(1, 'one');
    }
    sql1 { 
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(2, 'two');
    }
    code1 { sqlite3_get_autocommit db }
  } 0

  do_test 2.$tn.1.2 {
    sql2 {
      BEGIN;
        INSERT INTO t1 VALUES(3, 'three');
      ROLLBACK;
    }
  } {}

  do_test 2.$tn.1.3 {
    sql1 COMMIT
    sql2 { SELECT * FROM t1 }
  } {1 one 2 two}
  
  #-----------------------------------------------------------------------
  # 1. Start an CONCURRENT transaction using [db1].
  #
  # 2. Commit a transaction using [db2].
  #
  # 3. Try to commit with [db1]. Check that SQLITE_BUSY_SNAPSHOT is returned,
  #    and the transaction is not rolled back.
  #
  do_test 2.$tn.2.1 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(-1, 'hello world');
    }
  } {}

  do_test 2.$tn.2.2 {
    sql2 {
      INSERT INTO t1 VALUES(3, 'three');
    }
  } {}

  do_test 2.$tn.2.3.1 {
    set rc [catch { sql1 COMMIT } msg]
    list $rc $msg
  } {1 {database is locked}}

  do_test 2.$tn.2.3.2 {
    code1 { list [sqlite3_extended_errcode db] [sqlite3_get_autocommit db] }
  } {SQLITE_BUSY_SNAPSHOT 0}

  do_test 2.$tn.2.3.3 {
    sql1 {
      SELECT * FROM t1;
      ROLLBACK;
    }
  } {-1 {hello world} 1 one 2 two}
  
  #-----------------------------------------------------------------------
  # 1. Start an CONCURRENT transaction using [db1].
  #
  # 2. Open a transaction using [db2].
  #
  # 3. Try to commit with [db1]. Check that SQLITE_BUSY is returned,
  #    and the transaction is not rolled back.
  #
  # 4. Have [db2] roll its transaction back. Then check that [db1] can
  #    commit.
  #
  do_test 2.$tn.3.1 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(4, 'four');
    }
  } {}

  do_test 2.$tn.3.2 {
    sql2 {
      BEGIN;
        INSERT INTO t1 VALUES(-1, 'xyz');
    }
  } {}

  do_test 2.$tn.3.3.1 {
    set rc [catch { sql1 COMMIT } msg]
    list $rc $msg
  } {1 {database is locked}}

  do_test 2.$tn.3.3.2 {
    code1 { list [sqlite3_extended_errcode db] [sqlite3_get_autocommit db] }
  } {SQLITE_BUSY 0}

  do_test 2.$tn.3.3.3 {
    sql1 { SELECT * FROM t1; }
  } {1 one 2 two 3 three 4 four}

  do_test 2.$tn.3.4 {
    sql2 ROLLBACK
    sql1 COMMIT
    sql1 { SELECT * FROM t1; }
  } {1 one 2 two 3 three 4 four}

  #-----------------------------------------------------------------------
  # 1. Create a second table - t2.
  #
  # 2. Write to t1 with [db] and t2 with [db2].
  #
  # 3. See if it worked.
  #
  do_test 2.$tn.4.1 {
    sql1 { CREATE TABLE t2(a, b) }
  } {}
  do_test 2.$tn.4.2 {
    sql2 {
      BEGIN CONCURRENT;
        INSERT INTO t2 VALUES('i', 'n');
    }

    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(5, 'five');
      COMMIT;
    }

    sql2 COMMIT
  } {}

  do_test 2.$tn.4.3.1 {
    sql2 {SELECT * FROM t1}
  } {1 one 2 two 3 three 4 four 5 five}
  do_test 2.$tn.4.3.2 {
    sql1 {SELECT * FROM t1}
  } {1 one 2 two 3 three 4 four 5 five}

  do_test 2.$tn.4.3.3 { sql2 {SELECT * FROM t2} } {i n}
  do_test 2.$tn.4.3.4 { sql1 {SELECT * FROM t2} } {i n}

  #-----------------------------------------------------------------------
  # The "schema cookie" issue.
  #
  # 1. Begin and CONCURRENT write to "t1" using [db]
  #
  # 2. Create an index on t1 using [db2].
  #
  # 3. Attempt to commit the CONCURRENT write. This is an SQLITE_BUSY_SNAPSHOT,
  #    even though there is no page collision.
  #
  do_test 2.$tn.5.1 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(6, 'six');
    }
  } {}

  do_test 2.$tn.5.2 {
    sql2 { CREATE INDEX i1 ON t1(v); }
  } {}

  do_test 2.$tn.5.3 {
    list [catch { sql1 { COMMIT } } msg] $msg [sqlite3_errcode db]
  } {1 {database is locked} SQLITE_BUSY_SNAPSHOT}

  do_test 2.$tn.5.4 {
    sql2 { PRAGMA integrity_check }
  } {ok}
  catch { sql1 ROLLBACK }

  #-----------------------------------------------------------------------
  #
  # 1. Begin an CONCURRENT write to "t1" using [db]
  #
  # 2. Lots of inserts into t2. Enough to grow the db file and modify page 1.
  #
  # 3. Check that the CONCURRENT transaction can not be committed.
  #
  do_test 2.$tn.6.1 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(6, 'six');
    }
  } {}

  do_test 2.$tn.6.2 {
    sql2 { 
      WITH src(a,b) AS (
        VALUES(1,1) UNION ALL SELECT a+1,b+1 FROM src WHERE a<10000
      ) INSERT INTO t2 SELECT * FROM src;
    }
  } {}

  do_test 2.$tn.6.3 {
    sql1 { SELECT count(*) FROM t2 }
    list [catch { sql1 { COMMIT } } msg] $msg [sqlite3_errcode db]
  } {1 {database is locked} SQLITE_BUSY_SNAPSHOT}
  sql1 ROLLBACK

  do_test 2.$tn.6.4 {
    sql1 {
      SELECT count(*) FROM t1;
      SELECT count(*) FROM t2;
    }
  } {5 10001}

  #-----------------------------------------------------------------------
  # 
  # 1. Begin an big CONCURRENT write to "t1" using [db] - large enough to
  #    grow the db file.
  #
  # 2. Lots of inserts into t2. Also enough to grow the db file.
  #
  # 3. Check that the CONCURRENT transaction cannot be committed (due to a clash
  #    on page 1 - the db size field).
  #
  do_test 2.$tn.7.1 {
    sql1 {
      BEGIN CONCURRENT;
        WITH src(a,b) AS (
          VALUES(10000,10000) UNION ALL SELECT a+1,b+1 FROM src WHERE a<20000
        ) INSERT INTO t1 SELECT * FROM src;
    }
  } {}

  do_test 2.$tn.7.2 {
    sql2 { 
      WITH src(a,b) AS (
        VALUES(1,1) UNION ALL SELECT a+1,b+1 FROM src WHERE a<10000
      ) INSERT INTO t2 SELECT * FROM src;
    }
  } {}

  do_test 2.$tn.7.3 {
    list [catch { sql1 { COMMIT } } msg] $msg [sqlite3_errcode db]
  } {0 {} SQLITE_OK}

  do_test 2.$tn.7.4 { sql3 { PRAGMA integrity_check } } ok
}

#-------------------------------------------------------------------------
# Concurrent transactions may not modify the user_version or application_id.
#
reset_db
do_execsql_test 3.0 {
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x, y);
  INSERT INTO t1 VALUES('a', 'b');
  PRAGMA user_version = 10;
} {wal}
do_execsql_test 3.1 {
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES('c', 'd');
    SELECT * FROM t1;
} {a b c d}
do_catchsql_test 3.2 {
  PRAGMA user_version = 11;
} {1 {cannot modify user_version within CONCURRENT transaction}}
do_execsql_test 3.3 {
  PRAGMA user_version;
  SELECT * FROM t1;
} {10 a b c d}
do_catchsql_test 3.4 {
  PRAGMA application_id = 11;
} {1 {cannot modify application_id within CONCURRENT transaction}}
do_execsql_test 3.5 {
  COMMIT;
  PRAGMA user_version;
  PRAGMA application_id;
  SELECT * FROM t1;
} {10 0 a b c d}

#-------------------------------------------------------------------------
# However, another transaction modifying the user_version or application_id
# should not cause a conflict. And committing a concurrent transaction does not
# clobber the modification - even if the concurrent transaction allocates or
# frees database pages.
#
do_multiclient_test tn {
  do_test 4.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE ttt(y UNIQUE, z UNIQUE);
      PRAGMA user_version = 14;
      BEGIN CONCURRENT;
        INSERT INTO ttt VALUES('y', 'z');
    }
  } {wal}
  do_test 4.$tn.2 {
    sql2 { PRAGMA user_version = 16 }
    sql1 COMMIT
    sql1 { PRAGMA user_version }
  } {16}

  do_test 4.$tn.3 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO ttt VALUES(randomblob(10000), randomblob(4));
        PRAGMA user_version;
    }
  } {16}
  do_test 4.$tn.4 {
    sql2 { PRAGMA user_version = 1234 }
    sql1 {
        PRAGMA user_version;
      COMMIT;
      PRAGMA user_version;
      PRAGMA integrity_check;
    }
  } {16 1234 ok}

  do_test 4.$tn.5 {
    sql1 {
      BEGIN CONCURRENT;
        DELETE FROM ttt;
        PRAGMA user_version;
    }
  } {1234}
  do_test 4.$tn.4 {
    sql2 { PRAGMA user_version = 5678 }
    sql1 {
        PRAGMA user_version;
      COMMIT;
      PRAGMA user_version;
      PRAGMA integrity_check;
    }
  } {1234 5678 ok}
}

do_multiclient_test tn {
  do_test 5.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE tt(a INTEGER PRIMARY KEY, b);
      CREATE TABLE t2(a INTEGER PRIMARY KEY, b);
      INSERT INTO tt VALUES(1, randomblob(400));
      BEGIN CONCURRENT;
    }
  } {wal}

  do_test 5.$tn.2 {
    sql1 { UPDATE t2 SET b=5 WHERE a=3 }
    sql2 { INSERT INTO tt VALUES(2, randomblob(6000)) }
  } {}

  do_test 5.$tn.3 {
    sql1 { COMMIT }
  } {}
}

do_multiclient_test tn {
  do_test 6.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
      CREATE TABLE t2(a INTEGER PRIMARY KEY, b);
      INSERT INTO t1 VALUES(1, 'one');
      INSERT INTO t2 VALUES(2, 'two');
    }
  } {wal}

  do_test 6.$tn.2 {
    sql2 {
      BEGIN CONCURRENT;
        SELECT * FROM t2;
        INSERT INTO t1 VALUES(3, 'three');
    }
  } {2 two}

  do_test 6.$tn.3 {
    sql1 {
      INSERT INTO t2 VALUES(3, 'three');
    }
  } {}

  do_test 6.$tn.2 {
    list [catch { sql2 { COMMIT } } msg] $msg
  } {1 {database is locked}}
}

do_multiclient_test tn {
  do_test 7.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
      WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) 
      INSERT INTO t1 SELECT NULL, randomblob(400) FROM s;

      CREATE TABLE t2(a INTEGER PRIMARY KEY, b);
      WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<50000) 
      INSERT INTO t2 SELECT NULL, randomblob(400) FROM s;

      CREATE TABLE t3(a INTEGER PRIMARY KEY, b);
      WITH s(i) AS ( VALUES(1) UNION ALL SELECT i+1 FROM s WHERE i<100) 
      INSERT INTO t3 SELECT NULL, randomblob(400) FROM s;

      CREATE TABLE t4(a INTEGER PRIMARY KEY, b);

    }
    set {} {}
  } {}

  do_test 7.$tn.2 {
    sql2 {
      BEGIN CONCURRENT;
        SELECT * FROM t1;
        INSERT INTO t4 VALUES(1, 2);
    }
    set {} {}
  } {}

  do_test 7.$tn.3 {
    sql3 {
      BEGIN CONCURRENT;
        SELECT * FROM t3;
        INSERT INTO t4 VALUES(1, 2);
    }
    set {} {}
  } {}

  do_test 7.$tn.4 {
    sql1 {
      UPDATE t1 SET b=randomblob(400);
      UPDATE t2 SET b=randomblob(400);
      UPDATE t3 SET b=randomblob(400);
    }
  } {}

  do_test 7.$tn.5 {
    csql2 { COMMIT } 
  } {1 {database is locked}}

  do_test 7.$tn.6 {
    csql3 { COMMIT } 
  } {1 {database is locked}}


  csql2 ROLLBACK
  csql3 ROLLBACK

  # The following test works with $tn==1 (sql2 and sql3 use separate 
  # processes), but is quite slow. So only run it with $tn==2 (all
  # connections in the same process).
  #
  if {$tn==2} {
    do_test 7.$tn.7 {
      for {set i 1} {$i < 10000} {incr i} {
        sql3 { 
          PRAGMA wal_checkpoint;
          BEGIN CONCURRENT;
          SELECT * FROM t3;
          INSERT INTO t4 VALUES(1, 2);
        }

        sql1 {
          UPDATE t2 SET b = randomblob(400) WHERE rowid <= $i;
          UPDATE t3 SET b = randomblob(400) WHERE rowid = 1;
        }

        if {[csql3 COMMIT]!={1 {database is locked}}} {
          error "Failed at i=$i"
        }
        csql3 ROLLBACK
      }
    } {}
  }

}

finish_test

Added test/concurrent2.test.





































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
# 2015 July 26
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# Miscellaneous tests for transactions started with BEGIN CONCURRENT. 
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/lock_common.tcl
source $testdir/wal_common.tcl
set ::testprefix concurrent2

ifcapable !concurrent {
  finish_test
  return
}

do_multiclient_test tn {

  do_test 1.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(x);
      CREATE TABLE t2(y);
    }
  } {wal}
  do_test 1.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}

  # Test that an CONCURRENT transaction that allocates/frees no pages does
  # not conflict with a transaction that does allocate pages.
  do_test 1.$tn.2  {
    sql1 { 
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(4);
    }
    sql2 {
      INSERT INTO t2 VALUES(randomblob(1500));
    }
    sql1 {
      COMMIT;
    }
  } {}
  do_test 1.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}
  
  # But that an CONCURRENT transaction does conflict with a transaction
  # that modifies the db schema.
  do_test 1.$tn.3  {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(5);
    }
    sql2 {
      CREATE TABLE t3(z);
    }
    list [catch { sql1 COMMIT } msg] $msg
  } {1 {database is locked}}
  do_test 1.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}
  
  # Test that an CONCURRENT transaction that allocates at least one page 
  # does not conflict with a transaction that allocates no pages.
  do_test 1.$tn.4  {
    sql1 { 
      ROLLBACK;
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(randomblob(1500));
    }
    sql2 {
      INSERT INTO t2 VALUES(8);
    }
    sql1 {
      COMMIT;
    }
  } {}

  do_test 1.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}
}

do_multiclient_test tn {
  do_test 2.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(x UNIQUE);
      CREATE TABLE t2(y UNIQUE);
    }
  } {wal}

  do_test 2.$tn.2  {
    sql1 { 
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(randomblob(1500));
    }
    sql2 {
      INSERT INTO t2 VALUES(randomblob(1500));
    }
    sql1 COMMIT
  } {}

  do_test 2.$tn.3 { sql3 { PRAGMA integrity_check } } {ok}

  do_test 2.$tn.4  {
    sql1 { 
      BEGIN CONCURRENT;
        DELETE FROM t1;
    }
    sql2 {
      DELETE FROM t2;
    }
    sql1 COMMIT
  } {}

  do_test 2.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}

  do_test 2.$tn.6 {
    sql1 {
      INSERT INTO t1 VALUES(randomblob(1500));
      INSERT INTO t1 VALUES(randomblob(1500));
      INSERT INTO t2 VALUES(randomblob(1500));
      DELETE FROM t1 WHERE rowid=1;
    }

    sql1 {
      BEGIN CONCURRENT;
        DELETE FROM t1 WHERE rowid=2;
    }

    sql2 {
      DELETE FROM t2;
    }

    sql1 COMMIT
  } {}

  do_test 2.$tn.7 { sql3 { PRAGMA integrity_check } } {ok}
}

#-------------------------------------------------------------------------
# When an CONCURRENT transaction is opened on a database, the nFree and 
# iTrunk header fields of the cached version of page 1 are both set 
# to 0. This allows an CONCURRENT transaction to use its own private 
# free-page-list, which is merged with the main database free-list when
# the transaction is committed.
#
# The following tests check that nFree/iTrunk are correctly restored if
# an CONCURRENT transaction is rolled back, and that savepoint rollbacks
# that occur within CONCURRENT transactions do not incorrectly restore
# these fields to their on-disk values.
#
reset_db
do_execsql_test 3.0 {
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x, y);
  INSERT INTO t1 VALUES(randomblob(1500), randomblob(1500));
  DELETE FROM t1;
} {wal}

do_execsql_test 3.1 {
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES(1, 2);
  ROLLBACK;
}

do_execsql_test 3.2 { PRAGMA integrity_check } {ok}
do_execsql_test 3.3 { PRAGMA freelist_count } {2}

do_execsql_test 3.4.1 {
  BEGIN CONCURRENT;
    PRAGMA freelist_count;
} {2}
do_execsql_test 3.4.2 {
  SAVEPOINT xyz;
    INSERT INTO t1 VALUES(randomblob(1500), NULL);
    PRAGMA freelist_count;
} {0}
do_execsql_test 3.4.3 {
  ROLLBACK TO xyz;
} {}
do_execsql_test 3.4.4 { PRAGMA freelist_count } {0}
do_execsql_test 3.4.5 { COMMIT; PRAGMA freelist_count } {2}
do_execsql_test 3.4.6 { PRAGMA integrity_check } {ok}

do_execsql_test 3.5.1 {
  BEGIN CONCURRENT;
    UPDATE t1 SET x=randomblob(10) WHERE y=555;
    PRAGMA freelist_count;
} {0}
do_execsql_test 3.5.2 {
  ROLLBACK;
  PRAGMA freelist_count;
} {2}
do_execsql_test 3.5.3 { PRAGMA integrity_check } {ok}

#-------------------------------------------------------------------------
# Test that nothing goes wrong if an CONCURRENT transaction allocates a
# page at the end of the file, frees it within the same transaction, and
# then has to move the same page to avoid a conflict on COMMIT.
#
do_multiclient_test tn {
  do_test 4.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(x);
      CREATE TABLE t2(x);
    }
  } {wal}

  do_test 4.$tn.2 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(randomblob(1500));
        INSERT INTO t1 VALUES(randomblob(1500));
        DELETE FROM t1 WHERE rowid = 1;
    }

    sql2 {
      INSERT INTO t2 VALUES(randomblob(1500));
      INSERT INTO t2 VALUES(randomblob(1500));
      INSERT INTO t2 VALUES(randomblob(1500));
      INSERT INTO t2 VALUES(randomblob(1500));
      DELETE FROM t2 WHERE rowid IN (1, 2);
    }

    sql1 COMMIT
  } {}
}

#-------------------------------------------------------------------------
#
do_multiclient_test tn {
  do_test 5.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(x);
      CREATE TABLE t2(x);
      INSERT INTO t1 VALUES(randomblob(1500));
      PRAGMA page_count;
    }
  } {wal 4}

  do_test 5.$tn.2 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t2 VALUES(randomblob(1500));
        PRAGMA page_count;
    }
  } {5}

  do_test 5.$tn.3 {
    sql2 { 
      DELETE FROM t1;
      PRAGMA freelist_count;
      PRAGMA page_count;
    }
  } {1 4}

  do_test 5.$tn.4 { sql1 COMMIT } {}
  do_test 5.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}
}

#-------------------------------------------------------------------------
#
do_multiclient_test tn {
  do_test 6.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(x);
      INSERT INTO t1 VALUES(randomblob(1500));
      PRAGMA wal_checkpoint;
    }
  } {wal 0 5 5}

  do_test 6.$tn.2 {
    sql1 { 
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(randomblob(1500));
        INSERT INTO t1 VALUES(randomblob(1500));
    }
  } {}

  do_test 6.$tn.3 {
    sql2 {
      BEGIN;
        INSERT INTO t1 VALUES(randomblob(1500));
        INSERT INTO t1 VALUES(randomblob(1500));
      COMMIT;
    }
  } {}

  do_test 6.$tn.4 { 
    list [catch { sql1 COMMIT } msg] $msg
  } {1 {database is locked}}
  do_test 6.$tn.5 { sql3 { PRAGMA integrity_check } } {ok}
  do_test 6.$tn.5 { sql3 { SELECT count(*) from t1 } } {3}
}

#-------------------------------------------------------------------------
# Test that if a corrupt wal-index-header is encountered when attempting
# to commit a CONCURRENT transaction, the transaction is not committed
# (or rolled back) and that SQLITE_BUSY_SNAPSHOT is returned to the user.
#
catch { db close }
forcedelete test.db
testvfs tvfs
sqlite3 db test.db -vfs tvfs
do_execsql_test 7.1 {
  PRAGMA journal_mode = wal;
  BEGIN;
    CREATE TABLE t1(a, b, PRIMARY KEY(a));
    INSERT INTO t1 VALUES(1, 2);
    INSERT INTO t1 VALUES(3, 4);
  COMMIT;
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES(5, 6);
    INSERT INTO t1 VALUES(7, 8);
    SELECT * FROM t1;
} {wal 1 2 3 4 5 6 7 8}

# Corrupt the wal-index header
incr_tvfs_hdr test.db 11 1

do_catchsql_test 7.2.1 { COMMIT } {1 {database is locked}}
do_test 7.2.2 { sqlite3_extended_errcode db } SQLITE_BUSY_SNAPSHOT

do_execsql_test 7.3.1 {
  SELECT * FROM t1;
  ROLLBACK;
} {1 2 3 4 5 6 7 8}
do_execsql_test 7.3.2 {
  SELECT * FROM t1;
} {1 2 3 4}

#-------------------------------------------------------------------------
# Test that "PRAGMA integrity_check" works within a concurrent 
# transaction. Within a concurrent transaction, "PRAGMA integrity_check"
# is unable to detect unused database pages, but can detect other types
# of corruption.
#
reset_db
do_test 8.1 {
  execsql {
    PRAGMA journal_mode = wal;
    CREATE TABLE kv(k INTEGER PRIMARY KEY, v UNIQUE);
    INSERT INTO kv VALUES(NULL, randomblob(750));
    INSERT INTO kv SELECT NULL, randomblob(750) FROM kv;
    INSERT INTO kv SELECT NULL, randomblob(750) FROM kv;
    INSERT INTO kv SELECT NULL, randomblob(750) FROM kv;
    INSERT INTO kv SELECT NULL, randomblob(750) FROM kv;
    INSERT INTO kv SELECT NULL, randomblob(750) FROM kv;
    DELETE FROM kv WHERE rowid%2;
  }
  set v [db one {PRAGMA freelist_count}]
  expr $v==33 || $v==34
} {1}
do_execsql_test 8.2 { PRAGMA integrity_check } ok
do_execsql_test 8.3 { 
  BEGIN CONCURRENT;
    PRAGMA integrity_check;
} {ok}
do_execsql_test 8.4 { 
    INSERT INTO kv VALUES(1100, 1100);
    PRAGMA integrity_check;
} {ok}
do_execsql_test 8.5 { 
  COMMIT;
  PRAGMA integrity_check;
} {ok}

#-------------------------------------------------------------------------
# Test that concurrent transactions do not allow foreign-key constraints
# to be bypassed.
#
do_multiclient_test tn {
  do_test 9.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE pp(i INTEGER PRIMARY KEY, j);
      CREATE TABLE cc(a, b REFERENCES pp);

      WITH seq(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM seq WHERE i<100)
      INSERT INTO pp SELECT i, randomblob(1000) FROM seq;

      PRAGMA foreign_keys = 1;
    }
  } {wal}


  do_test 9.$tn.2.1 {
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO cc VALUES(42, 42);
    }
  } {}
  do_test 9.$tn.2.2 {
    sql2 { DELETE FROM pp WHERE i=42 }
    list [catch { sql1 COMMIT } msg] $msg
  } {1 {database is locked}}
  do_test 9.$tn.2.3 {
    sql1 ROLLBACK
  } {}

  do_test 9.$tn.3.1 {
    sql1 {
      PRAGMA foreign_keys = 0;
      BEGIN CONCURRENT;
        INSERT INTO cc VALUES(43, 43);
    }
  } {}
  do_test 9.$tn.3.2 {
    sql2 { DELETE FROM pp WHERE i=43 }
    list [catch { sql1 COMMIT } msg] $msg
  } {0 {}}

  do_test 9.$tn.4.1 {
    sql1 {
      PRAGMA foreign_keys = on;
      BEGIN CONCURRENT;
        INSERT INTO cc VALUES(44, 44);
    }
  } {}
  do_test 9.$tn.4.2 {
    sql2 { DELETE FROM pp WHERE i=1 }
    list [catch { sql1 COMMIT } msg] $msg
  } {0 {}}
}

#-------------------------------------------------------------------------
# Test that even if a SELECT statement appears before all writes within
# a CONCURRENT transaction, the pages it reads are still considered when
# considering whether or not the transaction may be committed.
#
do_multiclient_test tn {
  do_test 10.$tn.1.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(a);
      CREATE TABLE t2(b);
      CREATE TABLE t3(c);
      INSERT INTO t1 VALUES(1), (2), (3);
      INSERT INTO t2 VALUES(1), (2), (3);
      INSERT INTO t3 VALUES(1), (2), (3);
    }
  } {wal}

  do_test 10.$tn.1.2 {
    sql1 {
      BEGIN CONCURRENT;
        SELECT * FROM t1;
        INSERT INTO t2 VALUES(4);
    }
  } {1 2 3}

  do_test 10.$tn.1.3 {
    sql2 { INSERT INTO t1 VALUES(4) }
    list [catch {sql1 COMMIT} msg] $msg
  } {1 {database is locked}}
  sql1 ROLLBACK

  # In this case, because the "SELECT * FROM t1" is first stepped before
  # the "BEGIN CONCURRENT", the pages it reads are not recorded by the
  # pager object. And so the transaction can be committed. Technically
  # this behaviour (the effect of an ongoing SELECT on a BEGIN CONCURRENT
  # transacation) is undefined.
  #
  do_test 10.$tn.2.1 {
    code1 {
      set ::stmt [sqlite3_prepare db "SELECT * FROM t1" -1 dummy]
      sqlite3_step $::stmt
    }
  } {SQLITE_ROW}
  do_test 10.$tn.2.2 {
    sql1 {
      BEGIN CONCURRENT; 
        INSERT INTO t2 VALUES(4);
    }
    code1 {
      set res [list]
      lappend res [sqlite3_column_int $::stmt 0]
      while {[sqlite3_step $::stmt]=="SQLITE_ROW"} {
        lappend res [sqlite3_column_int $::stmt 0]
      }
      sqlite3_finalize $::stmt
      set res
    }
  } {1 2 3 4}
  do_test 10.$tn.2.3 {
    sql2 { INSERT INTO t1 VALUES(5) }
    sql1 COMMIT
  } {}

  # More tests surrounding long-lived prepared statements and concurrent
  # transactions.
  do_test 10.$tn.3.1 {
    sql1 {
      BEGIN CONCURRENT;
        SELECT * FROM t1;
      COMMIT;
    }
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t2 VALUES(5);
    }
    sql2 {
      INSERT INTO t1 VALUES(5);
    }
    sql1 COMMIT
    sql3 {
      SELECT * FROM t2;
    }
  } {1 2 3 4 5}
  do_test 10.$tn.3.2 {
    sql1 {
      BEGIN CONCURRENT;
        SELECT * FROM t1;
      ROLLBACK;
    }
    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t2 VALUES(6);
    }
    sql2 {
      INSERT INTO t1 VALUES(6);
    }
    sql1 COMMIT
    sql3 { SELECT * FROM t2 }
  } {1 2 3 4 5 6}
  do_test 10.$tn.3.3 {
    sql1 { BEGIN CONCURRENT }
    code1 {
      set ::stmt [sqlite3_prepare db "SELECT * FROM t1" -1 dummy]
      sqlite3_step $::stmt
    }
    sql1 {
      INSERT INTO t2 VALUES(7);
      SELECT * FROM t3;
      ROLLBACK;
      BEGIN CONCURRENT;
    }
    sql2 { INSERT INTO t3 VALUES(5) }
    code1 { sqlite3_finalize $::stmt }
    sql1 {
      INSERT INTO t2 VALUES(8);
      COMMIT;
    }
  } {}
}

do_multiclient_test tn {
  do_test 11.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(a);
    }
  } {wal}

  do_test 11.$tn.2 {
    code1 { sqlite3_wal_info db main }
  } {0 2}

  do_test 11.$tn.3 {
    sql1 { INSERT INTO t1 VALUES(1) }
    code1 { sqlite3_wal_info db main }
  } {2 3}

  do_test 11.$tn.4 {
    sql2 { INSERT INTO t1 VALUES(2) }
    code2 { sqlite3_wal_info db2 main }
  } {3 4}

  do_test 11.$tn.5 {
    sql1 { PRAGMA wal_checkpoint }
    sql2 { INSERT INTO t1 VALUES(3) }
    code2 { sqlite3_wal_info db2 main }
  } {0 1}
}

reset_db
do_execsql_test 12.0 {
  PRAGMA journal_mode = wal;
  CREATE TABLE tx(a INTEGER PRIMARY KEY, b);
} {wal}
do_test 12.1 {
  for {set i 0} {$i < 50} {incr i} {
    execsql {
      BEGIN CONCURRENT;
      INSERT INTO tx(b) VALUES( randomblob( 1200 ) );
      COMMIT;
    }
  }
  execsql { PRAGMA page_size }
} {1024}
do_execsql_test 12.2 {
  DELETE FROM tx;
}
do_test 12.3 {
  for {set i 0} {$i < 50} {incr i} {
    execsql {
      BEGIN CONCURRENT;
      INSERT INTO tx(b) VALUES( randomblob( 1200 ) );
      COMMIT;
    }
  }
  execsql { PRAGMA page_size }
} {1024}
do_execsql_test 12.4 {
  DELETE FROM tx;
}
do_test 12.5 {
  execsql { BEGIN CONCURRENT }
  for {set i 0} {$i < 5000} {incr i} {
    execsql {
      INSERT INTO tx(b) VALUES( randomblob( 1200 ) );
    }
  }
  execsql { COMMIT }
  execsql { PRAGMA page_size }
} {1024}


finish_test

Added test/concurrent3.test.















































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
# 2015 July 26
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# Tests for transactions started with BEGIN CONCURRENT. The tests in this
# file focus on testing that deferred page allocation works properly.
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/lock_common.tcl
set ::testprefix concurrent3

if {$AUTOVACUUM} { finish_test ; return }
ifcapable !concurrent {
  finish_test
  return
}

db close
sqlite3_shutdown
test_sqlite3_log xLog
proc xLog {error_code msg} {
  # puts "$error_code: $msg"
  # Enable the previous for debugging
}
reset_db

proc create_schema {} {
  db eval {
    PRAGMA journal_mode = wal;

    CREATE TABLE t1(x, y);
    CREATE TABLE t2(x, y);
    CREATE TABLE t3(x, y);
    CREATE TABLE t4(x, y);

    CREATE INDEX i1 ON t1(y, x);
    CREATE INDEX i2 ON t2(y, x);
    CREATE INDEX i3 ON t3(y, x);
    CREATE INDEX i4 ON t4(y, x);
  }
}

proc do_sql_op {iTbl iOp} {
  set db "db$iTbl"

  switch $iOp {
    "i" {
      set sql "
        WITH cnt(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM cnt WHERE i<10)
        INSERT INTO t$iTbl SELECT randomblob(800), randomblob(800) FROM cnt;
      "
    }

    "d" {
      set sql "
        DELETE FROM t$iTbl WHERE rowid IN (
          SELECT rowid FROM t$iTbl ORDER BY 1 ASC LIMIT 10
        )
      "
    }

    "D" {
      set sql "
        DELETE FROM t$iTbl WHERE rowid IN (
          SELECT rowid FROM t$iTbl o WHERE (
            SELECT count(*) FROM t$iTbl i WHERE i.rowid<o.rowid
          ) % 2
        )
      "
    }

    "I" {
      set sql "
        INSERT INTO t$iTbl SELECT randomblob(800), randomblob(800) FROM t$iTbl;
      "
    }

    default {
      error "bad iOp parameter: $iOp"
    }
  }

  $db eval $sql
}


set DBLIST {db1 db2 db3 db4} 

create_schema
foreach {tn oplist} {
  1  {1i   2i   3i   4i} 
  2  {1iii 2iii 3iii 4iii}
  3  {1d   2d   3d   4d} 
  .  -----------------------
  4  {1i}
  5  {1d 2i}
  .  -----------------------
  6  {1iii 2iii 3iii 4iii}
  7  {1di  2id  3iii 4ddd}
  8  {1iii 2iii 3iii 4iii}
  9  {1D  2II}
  10 {1I  2D  3I  4D}
  11 {1III 3dddddd 4III}
} {
  if {[string range $oplist 0 0]=="-"} {
    reset_db
    create_schema
    continue
  }
  foreach db $DBLIST { sqlite3 $db test.db }

  do_test 1.$tn {
    foreach db $DBLIST { $db eval "BEGIN CONCURRENT" } 

    foreach op $oplist {
      set iTbl [string range $op 0 0]
      foreach char [split [string range $op 1 end] {}] {
        do_sql_op $iTbl $char
      }
    }

    foreach db $DBLIST { $db eval "COMMIT" }
    db eval {PRAGMA integrity_check}
  } {ok}

  foreach db $DBLIST { 
    $db close 
  }
}

#-------------------------------------------------------------------------
#
proc create_schema2 {} {
  db eval {
    PRAGMA journal_mode = wal;
    CREATE TABLE t1(x INTEGER PRIMARY KEY, y);
    CREATE INDEX i1 ON t1(y);
  }
}

proc randint {nMax} {
  db eval {SELECT abs(random() % $nMax)}
}

proc do_sql_op2 {db iOp} {
  switch -- $iOp {
    i {
      # Insert 1 rows.
      set r [randint 1000000000]
      set ::rows($r) 1
      #puts "insert row $r"
      $db eval { INSERT OR IGNORE INTO t1 VALUES($r, randomblob(50)); }
    }

    d {
      # Insert 1 row
      set keys [array names ::rows]
      set r [randint [llength $keys]]
      set rowid [lindex $keys $r]
      $db eval { DELETE FROM t1 WHERE x=$rowid }
      unset ::rows($rowid)
    }
  }
}

foreach {tn nRepeat oplist} {
  - - ----------------------------
  1 100 { 1iiiiiiiiii }
  2 100 { 1i 2d }
  3 100 { 1d 2i }
  4  50 { 1d 2i 3d }
  5 500 { 1i 2i 3i 4i }
  6 500 { 1i 2d 3d 4d }
} {
  if {[string range $oplist 0 0]=="-"} {
    array unset rows
    reset_db
    create_schema2
    continue
  }

  foreach db $DBLIST { 
    sqlite3 $db test.db 
    set stats($db,0) 0
    set stats($db,1) 0
  }
  array unset used

  do_test 2.$tn {

    for {set i 0} {$i < $nRepeat} {incr i} {
      foreach db $DBLIST { $db eval "BEGIN CONCURRENT" } 

      foreach op $oplist {
        set iDb [string range $op 0 0]
        set used(db$iDb) 1
        foreach char [split [string range $op 1 end] {}] {
          do_sql_op2 "db$iDb" $char
        }
      }

      foreach db $DBLIST { 
        set rc [catch { $db eval COMMIT } msg]
        if {$rc} { $db eval ROLLBACK }
        incr stats($db,$rc)
      }
      set res [db eval {PRAGMA integrity_check}]
      if {$res != "ok"} { puts "after $db $rc: $res" ; after 1000000 }
    }
  } {}

  foreach db $DBLIST { 
    $db close 
  }
  # foreach k [lsort [array names used]] {
  #   puts "$k: $stats($k,0) committed, $stats($k,1) rolled back"
  # }
}



finish_test

Added test/concurrent4.test.





































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
# 2017 May 26
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# Miscellaneous tests for transactions started with BEGIN CONCURRENT. 
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/lock_common.tcl
source $testdir/wal_common.tcl
set ::testprefix concurrent4

ifcapable !concurrent {
  finish_test
  return
}

do_execsql_test 1.0 {
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x PRIMARY KEY, y UNIQUE);
  WITH s(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100)
  INSERT INTO t1 SELECT randomblob(400), randomblob(400) FROM s;
  DELETE FROM t1 WHERE rowid<2;
} {wal}

do_execsql_test 1.1 {
  BEGIN CONCURRENT;
    INSERT INTO t1(rowid, x, y) VALUES(1000, randomblob(3000), randomblob(3000));
    SAVEPOINT abc;
    DELETE FROM t1 WHERE rowid = 1000;
}

do_execsql_test 1.2 { ROLLBACK TO abc }
do_execsql_test 1.3 { COMMIT }
do_execsql_test 1.4 { PRAGMA integrity_check } {ok}

do_multiclient_test tn {
  do_test 2.$tn.1 {
    sql1 {
      PRAGMA journal_mode = wal;
      CREATE TABLE t1(a, b);
      WITH s(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100)
        INSERT INTO t1 SELECT randomblob(400), randomblob(400) FROM s;

      CREATE TABLE t2(a, b);
      WITH s(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100)
        INSERT INTO t2 SELECT randomblob(400), randomblob(400) FROM s;
    }

    sql1 {
      BEGIN CONCURRENT;
        INSERT INTO t1 VALUES(randomblob(3000), randomblob(3000));
    }
  } {}

  do_test 2.$tn.2 {
    sql2 {
      WITH s(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<10)
        INSERT INTO t2 SELECT randomblob(400), randomblob(400) FROM s;
    }
    sql2 {
      DELETE FROM t2 WHERE rowid<10;
    }
  } {}

  do_test 2.$tn.3 {
    sql1 {
      COMMIT;
      PRAGMA integrity_check;
    }
  } {ok}
  do_test 2.$tn.4 {
    sql2 {
      PRAGMA integrity_check;
    }
  } {ok}
}

reset_db
do_execsql_test 3.1 {
  PRAGMA page_size = 1024;
  PRAGMA journal_mode = wal;
  CREATE TABLE t2(x);
  INSERT INTO t2 VALUES(randomblob(5000));
  CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
  INSERT INTO t1 VALUES(25, randomblob(104));
  DELETE FROM t2;
} {wal}

do_execsql_test 3.2 {
  BEGIN CONCURRENT;
    REPLACE INTO t1 VALUES(25, randomblob(1117));
  COMMIT;
} {}

#-------------------------------------------------------------------------
# Test the effect of BEGIN CONCURRENT transactions that consist entirely
# of read-only statements.
#
reset_db
do_execsql_test 4.0 {
  PRAGMA page_size = 1024;
  PRAGMA journal_mode = wal;
  
  CREATE TABLE t4(a, b);
  INSERT INTO t4 VALUES(1, 2);
  INSERT INTO t4 VALUES(3, 4);
} {wal}
 
sqlite3 db2 test.db
do_test 4.1.1 {
  db eval {
    BEGIN CONCURRENT;
      INSERT INTO t4 VALUES(5, 6);
  }

  db2 eval {
    BEGIN CONCURRENT;
      SELECT * FROM t4;
    ROLLBACK;
  }
} {1 2 3 4}

do_test 4.1.2 {
  db eval { COMMIT }
  db2 eval { SELECT * FROM t4 }
} {1 2 3 4 5 6}

do_test 4.2.1 {
  db eval {
    BEGIN CONCURRENT;
      INSERT INTO t4 VALUES(7, 8);
  }

  db2 eval {
    BEGIN CONCURRENT;
      SELECT * FROM t4;
    COMMIT;
  }
} {1 2 3 4 5 6}

do_test 4.2.2 {
  db eval { COMMIT }
  db2 eval { SELECT * FROM t4 }
} {1 2 3 4 5 6 7 8}

do_test 4.3 {
  db2 eval {
    BEGIN CONCURRENT;
      SELECT * FROM t4;
  }

  db eval {
    BEGIN CONCURRENT;
      INSERT INTO t4 VALUES(9, 10);
    COMMIT;
  }
  db2 eval {
    SELECT * FROM t4;
    COMMIT;
  }
} {1 2 3 4 5 6 7 8}

set sz [file size test.db-wal]
do_test 4.4.1 {
  db eval {
    BEGIN CONCURRENT;
      SELECT * FROM t4;
      SELECT * FROM sqlite_master;
  }

  db eval COMMIT
  file size test.db-wal
} $sz
do_test 4.4.2 {
  db eval {
    BEGIN CONCURRENT;
      SELECT * FROM t4;
      SELECT * FROM sqlite_master;
    ROLLBACK;
  }
  file size test.db-wal
} $sz

finish_test

Added test/concurrent5.test.











































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
# 2017 May 26
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/lock_common.tcl
source $testdir/wal_common.tcl
set ::testprefix concurrent5

ifcapable !concurrent {
  finish_test
  return
}

db close
sqlite3_shutdown
test_sqlite3_log [list lappend ::log]
set ::log [list]

sqlite3 db test.db

proc do_test_conflict_msg {tn msg} {
  set msg "cannot commit CONCURRENT transaction - [string trim $msg]"
  uplevel [list do_test $tn {lindex $::log end} $msg]
}

do_execsql_test 1.0 {
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x, y);
  CREATE TABLE t2(c);
  WITH s(i) AS (SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100)
  INSERT INTO t1 SELECT randomblob(400), randomblob(400) FROM s;
} {wal}

sqlite3 db2 test.db

do_test 1.1.1 {
  set ::log [list]
  db2 eval {
    BEGIN CONCURRENT;
      SELECT count(*) FROM t1;
      INSERT INTO t2 VALUES(10);
  }

  db eval {
    INSERT INTO t1 VALUES(randomblob(200), randomblob(200));
  }

  catchsql COMMIT db2
} {1 {database is locked}}
do_test_conflict_msg 1.1.2 {
  conflict at page 2 (read-only page; part of db table t1; content=0500000063021100...)
}

do_test 1.2.1 {
  set ::log [list]
  db2 eval {
    ROLLBACK;
    BEGIN CONCURRENT;
      INSERT INTO t1 VALUES(11, 12);
  }

  db eval {
    INSERT INTO t1 VALUES(12, 11);
  }

  catchsql COMMIT db2
} {1 {database is locked}}

do_test_conflict_msg 1.2.2 {
  conflict at page 105 (read/write page; part of db table t1; content=0D00000002026100...)
}

do_test 1.3.1 {
  set ::log [list]
  db2 eval {
    ROLLBACK;
    BEGIN CONCURRENT;
      INSERT INTO t2 VALUES('x');
  }

  db eval {
    INSERT INTO t2 VALUES('y');
  }

  catchsql COMMIT db2
} {1 {database is locked}}

do_test_conflict_msg 1.3.2 {
  conflict at page 3 (read/write page; part of db table t2; content=0D0000000103FB00...)
}

do_test 1.4.1 {
  set ::log [list]

  execsql {
    ROLLBACK;
    CREATE TABLE t3(a INTEGER PRIMARY KEY, b INTEGER);
    CREATE INDEX i3 ON t3(b);

    WITH s(i) AS (
      SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<5000
    ) INSERT INTO t3 SELECT i, i FROM s;

    BEGIN CONCURRENT;
      INSERT INTO t3 VALUES(0, 5001);
  } db2

  execsql { INSERT INTO t3 VALUES(NULL, 5002) } db
  catchsql COMMIT db2
} {1 {database is locked}}

do_test_conflict_msg 1.3.2 {
  conflict at page 211 (read/write page; part of db index t3.i3; content=0A0310006300D800...)
}

db close
db2 close
sqlite3_shutdown
test_sqlite3_log 
sqlite3_initialize
finish_test

Added test/concurrent6.test.

























































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
# 2017 May 26
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
#

set testdir [file dirname $argv0]
source $testdir/tester.tcl
source $testdir/lock_common.tcl
source $testdir/wal_common.tcl
set ::testprefix concurrent6

ifcapable !concurrent {
  finish_test
  return
}

sqlite3 db2 test.db

do_execsql_test 1.0 {
  PRAGMA page_size = 1024;
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x);
  CREATE TABLE t2(x);
  CREATE TABLE t3(x);
  CREATE TABLE t4(x);

  INSERT INTO t1 VALUES(zeroblob(1500));
} {wal}

do_execsql_test -db db2 1.1 {
  BEGIN CONCURRENT;
    INSERT INTO t3 VALUES(zeroblob(4000));
    DELETE FROM t1;
}

do_execsql_test 1.2 {
  WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100) 
  INSERT INTO t2 SELECT zeroblob(1000) FROM s;

  WITH s(i) AS ( SELECT 1 UNION ALL SELECT i+1 FROM s WHERE i<100) 
  INSERT INTO t4  SELECT zeroblob(1000) FROM s;

  DELETE FROM t4;
}

do_execsql_test -db db2 1.3 {
  COMMIT;
}


finish_test

Added test/concurrent7.test.









































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
# 2018 Jan 5
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#


set testdir [file dirname $argv0]
source $testdir/tester.tcl
set ::testprefix concurrent7

sqlite3 db2 test.db

do_execsql_test 1 {
  PRAGMA journal_mode = wal;
  CREATE TABLE t1(x);
  CREATE TABLE t2(x);
} {wal}

do_execsql_test -db db2 2 {
  SELECT * FROM t1;
}

do_execsql_test 3 {
  BEGIN CONCURRENT;
    INSERT INTO t1 VALUES(randomblob(1500));
    INSERT INTO t1 VALUES(randomblob(1500));
    DELETE FROM t1 WHERE rowid = 1;
}

do_execsql_test -db db2 4 {
  INSERT INTO t2 VALUES(randomblob(1500));
  INSERT INTO t2 VALUES(randomblob(1500));
  INSERT INTO t2 VALUES(randomblob(1500));
  INSERT INTO t2 VALUES(randomblob(1500));
  DELETE FROM t2 WHERE rowid IN (1, 2);
}

do_execsql_test 5 {
  COMMIT;
  PRAGMA integrity_check;
} {ok}

finish_test


Deleted test/parser1.test.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
# 2014-08-24
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
# This file implements regression tests for SQLite library.
# The focus of this script is testing details of the SQL language parser.
# 

set testdir [file dirname $argv0]
source $testdir/tester.tcl

do_catchsql_test parser1-1.1 {
  CREATE TABLE t1(
    a TEXT PRIMARY KEY,
    b TEXT,
    FOREIGN KEY(b COLLATE nocase DESC) REFERENCES t1(a COLLATE binary ASC)
  );
} {1 {syntax error after column name "b"}}


# Verify that a legacy schema in the sqlite_master file is allowed to have
# COLLATE, ASC, and DESC keywords on the id list of a FK constraint, and that
# those keywords are silently ignored.
#
sqlite3_db_config db DEFENSIVE 0
do_execsql_test parser1-1.2 {
  CREATE TABLE t1(
    a TEXT PRIMARY KEY,
    b TEXT,
    FOREIGN KEY(b) REFERENCES t1(a)
  );
  INSERT INTO t1 VALUES('abc',NULL),('xyz','abc');
  PRAGMA writable_schema=on;
  UPDATE sqlite_master SET sql='CREATE TABLE t1(
    a TEXT PRIMARY KEY,
    b TEXT,
    FOREIGN KEY(b COLLATE nocase) REFERENCES t1(a)
  )' WHERE name='t1';
  SELECT name FROM sqlite_master WHERE sql LIKE '%collate%';
} {t1}
sqlite3 db2 test.db
do_test parser1-1.3 {
  sqlite3 db2 test.db
  db2 eval {SELECT * FROM t1 ORDER BY 1}
} {abc {} xyz abc}
db2 close

do_execsql_test parser1-1.4 {
  UPDATE sqlite_master SET sql='CREATE TABLE t1(
    a TEXT PRIMARY KEY,
    b TEXT,
    FOREIGN KEY(b ASC) REFERENCES t1(a)
  )' WHERE name='t1';
  SELECT name FROM sqlite_master WHERE sql LIKE '%ASC%';
} {t1}
sqlite3 db2 test.db
do_test parser1-1.5 {
  sqlite3 db2 test.db
  db2 eval {SELECT * FROM t1 ORDER BY 1}
} {abc {} xyz abc}
db2 close

do_catchsql_test parser1-2.1 {
  WITH RECURSIVE
    c(x COLLATE binary) AS (VALUES(1) UNION SELECT x+1 FROM c WHERE x<5)
  SELECT x FROM c;
} {1 {syntax error after column name "x"}}
do_catchsql_test parser1-2.2 {
  WITH RECURSIVE
    c(x ASC) AS (VALUES(1) UNION SELECT x+1 FROM c WHERE x<5)
  SELECT x FROM c;
} {1 {syntax error after column name "x"}}

# Verify that the comma between multiple table constraints is
# optional.
#
# The missing comma is technically a syntax error.  But we have to support
# it because there might be legacy databases that omit the commas in their
# sqlite_master tables.
#
do_execsql_test parser1-3.1 {
  CREATE TABLE t300(id INTEGER PRIMARY KEY);
  CREATE TABLE t301(
    id INTEGER PRIMARY KEY,
    c1 INTEGER NOT NULL,
    c2 INTEGER NOT NULL,
    c3 BOOLEAN NOT NULL DEFAULT 0,
    FOREIGN KEY(c1) REFERENCES t300(id) ON DELETE CASCADE ON UPDATE RESTRICT
        /* no comma */
    FOREIGN KEY(c2) REFERENCES t300(id) ON DELETE CASCADE ON UPDATE RESTRICT
        /* no comma */
    UNIQUE(c1, c2)
  );
  PRAGMA foreign_key_list(t301);
} {0 0 t300 c2 id RESTRICT CASCADE NONE 1 0 t300 c1 id RESTRICT CASCADE NONE}

finish_test
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<














































































































































































































Changes to test/threadtest3.c.

28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91

92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
**        ./a.out walthread3      -- Run the "walthread3" test
**        ./a.out 'wal*'          -- Run all of the wal* tests
**        ./a.out --help          -- List all available tests
**
** The exit status is non-zero if any test fails.
*/

/* 
** The "Set Error Line" macro.
*/
#define SEL(e) ((e)->iLine = ((e)->rc ? (e)->iLine : __LINE__))

/* Database functions */
#define opendb(w,x,y,z)         (SEL(w), opendb_x(w,x,y,z))
#define closedb(y,z)            (SEL(y), closedb_x(y,z))

/* Functions to execute SQL */
#define sql_script(x,y,z)       (SEL(x), sql_script_x(x,y,z))
#define integrity_check(x,y)    (SEL(x), integrity_check_x(x,y))
#define execsql_i64(x,y,...)    (SEL(x), execsql_i64_x(x,y,__VA_ARGS__))
#define execsql_text(x,y,z,...) (SEL(x), execsql_text_x(x,y,z,__VA_ARGS__))
#define execsql(x,y,...)        (SEL(x), (void)execsql_i64_x(x,y,__VA_ARGS__))
#define sql_script_printf(x,y,z,...) (                \
    SEL(x), sql_script_printf_x(x,y,z,__VA_ARGS__)    \
) 

/* Thread functions */
#define launch_thread(w,x,y,z)     (SEL(w), launch_thread_x(w,x,y,z))
#define join_all_threads(y,z)      (SEL(y), join_all_threads_x(y,z))

/* Timer functions */
#define setstoptime(y,z)        (SEL(y), setstoptime_x(y,z))
#define timetostop(z)           (SEL(z), timetostop_x(z))

/* Report/clear errors. */
#define test_error(z, ...)      test_error_x(z, sqlite3_mprintf(__VA_ARGS__))
#define clear_error(y,z)        clear_error_x(y, z)

/* File-system operations */
#define filesize(y,z)           (SEL(y), filesize_x(y,z))
#define filecopy(x,y,z)         (SEL(x), filecopy_x(x,y,z))

#define PTR2INT(x) ((int)((intptr_t)x))
#define INT2PTR(x) ((void*)((intptr_t)x))

/*
** End of test code/infrastructure interface macros.
*************************************************************************/




#include <sqlite3.h>
#include <unistd.h>
#include <stdio.h>
#include <pthread.h>
#include <assert.h>
#include <sys/types.h> 
#include <sys/stat.h> 
#include <string.h>
#include <fcntl.h>
#include <errno.h>

#include "test_multiplex.h"


/* Required to link test_multiplex.c */
#ifndef SQLITE_OMIT_WSD
int sqlite3PendingByte = 0x40000000;
#endif

/*
 * This code implements the MD5 message-digest algorithm.
 * The algorithm is due to Ron Rivest.  This code was
 * written by Colin Plumb in 1993, no copyright is claimed.
 * This code is in the public domain; do with it what you wish.
 *
 * Equivalent code is available from RSA Data Security, Inc.
 * This code has been tested against that, and is equivalent,
 * except that you don't need to include two pages of legalese
 * with every copy.
 *
 * To compute the message digest of a chunk of bytes, declare an
 * MD5Context structure, pass it to MD5Init, call MD5Update as
 * needed on buffers full of bytes, and then call MD5Final, which
 * will fill a supplied 16-byte array with the digest.
 */

/*
 * If compiled on a machine that doesn't have a 32-bit integer,
 * you just set "uint32" to the appropriate datatype for an
 * unsigned 32-bit integer.  For example:
 *
 *       cc -Duint32='unsigned long' md5.c
 *
 */
#ifndef uint32
#  define uint32 unsigned int
#endif

struct MD5Context {
  int isInit;
  uint32 buf[4];
  uint32 bits[2];
  union {
    unsigned char in[64];
    uint32 in32[16];
  } u;
};
typedef struct MD5Context MD5Context;

/*
 * Note: this code is harmless on little-endian machines.
 */
static void byteReverse (unsigned char *buf, unsigned longs){
  uint32 t;
  do {
    t = (uint32)((unsigned)buf[3]<<8 | buf[2]) << 16 |
          ((unsigned)buf[1]<<8 | buf[0]);
    *(uint32 *)buf = t;
    buf += 4;
  } while (--longs);
}
/* The four core functions - F1 is optimized somewhat */

/* #define F1(x, y, z) (x & y | ~x & z) */
#define F1(x, y, z) (z ^ (x & (y ^ z)))
#define F2(x, y, z) F1(z, x, y)
#define F3(x, y, z) (x ^ y ^ z)
#define F4(x, y, z) (y ^ (x | ~z))

/* This is the central step in the MD5 algorithm. */
#define MD5STEP(f, w, x, y, z, data, s) \
  ( w += f(x, y, z) + data,  w = w<<s | w>>(32-s),  w += x )

/*
 * The core of the MD5 algorithm, this alters an existing MD5 hash to
 * reflect the addition of 16 longwords of new data.  MD5Update blocks
 * the data and converts bytes into longwords for this routine.
 */
static void MD5Transform(uint32 buf[4], const uint32 in[16]){
  register uint32 a, b, c, d;

  a = buf[0];
  b = buf[1];
  c = buf[2];
  d = buf[3];

  MD5STEP(F1, a, b, c, d, in[ 0]+0xd76aa478,  7);
  MD5STEP(F1, d, a, b, c, in[ 1]+0xe8c7b756, 12);
  MD5STEP(F1, c, d, a, b, in[ 2]+0x242070db, 17);
  MD5STEP(F1, b, c, d, a, in[ 3]+0xc1bdceee, 22);
  MD5STEP(F1, a, b, c, d, in[ 4]+0xf57c0faf,  7);
  MD5STEP(F1, d, a, b, c, in[ 5]+0x4787c62a, 12);
  MD5STEP(F1, c, d, a, b, in[ 6]+0xa8304613, 17);
  MD5STEP(F1, b, c, d, a, in[ 7]+0xfd469501, 22);
  MD5STEP(F1, a, b, c, d, in[ 8]+0x698098d8,  7);
  MD5STEP(F1, d, a, b, c, in[ 9]+0x8b44f7af, 12);
  MD5STEP(F1, c, d, a, b, in[10]+0xffff5bb1, 17);
  MD5STEP(F1, b, c, d, a, in[11]+0x895cd7be, 22);
  MD5STEP(F1, a, b, c, d, in[12]+0x6b901122,  7);
  MD5STEP(F1, d, a, b, c, in[13]+0xfd987193, 12);
  MD5STEP(F1, c, d, a, b, in[14]+0xa679438e, 17);
  MD5STEP(F1, b, c, d, a, in[15]+0x49b40821, 22);

  MD5STEP(F2, a, b, c, d, in[ 1]+0xf61e2562,  5);
  MD5STEP(F2, d, a, b, c, in[ 6]+0xc040b340,  9);
  MD5STEP(F2, c, d, a, b, in[11]+0x265e5a51, 14);
  MD5STEP(F2, b, c, d, a, in[ 0]+0xe9b6c7aa, 20);
  MD5STEP(F2, a, b, c, d, in[ 5]+0xd62f105d,  5);
  MD5STEP(F2, d, a, b, c, in[10]+0x02441453,  9);
  MD5STEP(F2, c, d, a, b, in[15]+0xd8a1e681, 14);
  MD5STEP(F2, b, c, d, a, in[ 4]+0xe7d3fbc8, 20);
  MD5STEP(F2, a, b, c, d, in[ 9]+0x21e1cde6,  5);
  MD5STEP(F2, d, a, b, c, in[14]+0xc33707d6,  9);
  MD5STEP(F2, c, d, a, b, in[ 3]+0xf4d50d87, 14);
  MD5STEP(F2, b, c, d, a, in[ 8]+0x455a14ed, 20);
  MD5STEP(F2, a, b, c, d, in[13]+0xa9e3e905,  5);
  MD5STEP(F2, d, a, b, c, in[ 2]+0xfcefa3f8,  9);
  MD5STEP(F2, c, d, a, b, in[ 7]+0x676f02d9, 14);
  MD5STEP(F2, b, c, d, a, in[12]+0x8d2a4c8a, 20);

  MD5STEP(F3, a, b, c, d, in[ 5]+0xfffa3942,  4);
  MD5STEP(F3, d, a, b, c, in[ 8]+0x8771f681, 11);
  MD5STEP(F3, c, d, a, b, in[11]+0x6d9d6122, 16);
  MD5STEP(F3, b, c, d, a, in[14]+0xfde5380c, 23);
  MD5STEP(F3, a, b, c, d, in[ 1]+0xa4beea44,  4);
  MD5STEP(F3, d, a, b, c, in[ 4]+0x4bdecfa9, 11);
  MD5STEP(F3, c, d, a, b, in[ 7]+0xf6bb4b60, 16);
  MD5STEP(F3, b, c, d, a, in[10]+0xbebfbc70, 23);
  MD5STEP(F3, a, b, c, d, in[13]+0x289b7ec6,  4);
  MD5STEP(F3, d, a, b, c, in[ 0]+0xeaa127fa, 11);
  MD5STEP(F3, c, d, a, b, in[ 3]+0xd4ef3085, 16);
  MD5STEP(F3, b, c, d, a, in[ 6]+0x04881d05, 23);
  MD5STEP(F3, a, b, c, d, in[ 9]+0xd9d4d039,  4);
  MD5STEP(F3, d, a, b, c, in[12]+0xe6db99e5, 11);
  MD5STEP(F3, c, d, a, b, in[15]+0x1fa27cf8, 16);
  MD5STEP(F3, b, c, d, a, in[ 2]+0xc4ac5665, 23);

  MD5STEP(F4, a, b, c, d, in[ 0]+0xf4292244,  6);
  MD5STEP(F4, d, a, b, c, in[ 7]+0x432aff97, 10);
  MD5STEP(F4, c, d, a, b, in[14]+0xab9423a7, 15);
  MD5STEP(F4, b, c, d, a, in[ 5]+0xfc93a039, 21);
  MD5STEP(F4, a, b, c, d, in[12]+0x655b59c3,  6);
  MD5STEP(F4, d, a, b, c, in[ 3]+0x8f0ccc92, 10);
  MD5STEP(F4, c, d, a, b, in[10]+0xffeff47d, 15);
  MD5STEP(F4, b, c, d, a, in[ 1]+0x85845dd1, 21);
  MD5STEP(F4, a, b, c, d, in[ 8]+0x6fa87e4f,  6);
  MD5STEP(F4, d, a, b, c, in[15]+0xfe2ce6e0, 10);
  MD5STEP(F4, c, d, a, b, in[ 6]+0xa3014314, 15);
  MD5STEP(F4, b, c, d, a, in[13]+0x4e0811a1, 21);
  MD5STEP(F4, a, b, c, d, in[ 4]+0xf7537e82,  6);
  MD5STEP(F4, d, a, b, c, in[11]+0xbd3af235, 10);
  MD5STEP(F4, c, d, a, b, in[ 2]+0x2ad7d2bb, 15);
  MD5STEP(F4, b, c, d, a, in[ 9]+0xeb86d391, 21);

  buf[0] += a;
  buf[1] += b;
  buf[2] += c;
  buf[3] += d;
}

/*
 * Start MD5 accumulation.  Set bit count to 0 and buffer to mysterious
 * initialization constants.
 */
static void MD5Init(MD5Context *ctx){
  ctx->isInit = 1;
  ctx->buf[0] = 0x67452301;
  ctx->buf[1] = 0xefcdab89;
  ctx->buf[2] = 0x98badcfe;
  ctx->buf[3] = 0x10325476;
  ctx->bits[0] = 0;
  ctx->bits[1] = 0;
}

/*
 * Update context to reflect the concatenation of another buffer full
 * of bytes.
 */
static 
void MD5Update(MD5Context *ctx, const unsigned char *buf, unsigned int len){
  uint32 t;

  /* Update bitcount */

  t = ctx->bits[0];
  if ((ctx->bits[0] = t + ((uint32)len << 3)) < t)
    ctx->bits[1]++; /* Carry from low to high */
  ctx->bits[1] += len >> 29;

  t = (t >> 3) & 0x3f;    /* Bytes already in shsInfo->data */

  /* Handle any leading odd-sized chunks */

  if ( t ) {
    unsigned char *p = (unsigned char *)ctx->u.in + t;

    t = 64-t;
    if (len < t) {
      memcpy(p, buf, len);
      return;
    }
    memcpy(p, buf, t);
    byteReverse(ctx->u.in, 16);
    MD5Transform(ctx->buf, (uint32 *)ctx->u.in);
    buf += t;
    len -= t;
  }

  /* Process data in 64-byte chunks */

  while (len >= 64) {
    memcpy(ctx->u.in, buf, 64);
    byteReverse(ctx->u.in, 16);
    MD5Transform(ctx->buf, (uint32 *)ctx->u.in);
    buf += 64;
    len -= 64;
  }

  /* Handle any remaining bytes of data. */

  memcpy(ctx->u.in, buf, len);
}

/*
 * Final wrapup - pad to 64-byte boundary with the bit pattern 
 * 1 0* (64-bit count of bits processed, MSB-first)
 */
static void MD5Final(unsigned char digest[16], MD5Context *ctx){
  unsigned count;
  unsigned char *p;

  /* Compute number of bytes mod 64 */
  count = (ctx->bits[0] >> 3) & 0x3F;

  /* Set the first char of padding to 0x80.  This is safe since there is
     always at least one byte free */
  p = ctx->u.in + count;
  *p++ = 0x80;

  /* Bytes of padding needed to make 64 bytes */
  count = 64 - 1 - count;

  /* Pad out to 56 mod 64 */
  if (count < 8) {
    /* Two lots of padding:  Pad the first block to 64 bytes */
    memset(p, 0, count);
    byteReverse(ctx->u.in, 16);
    MD5Transform(ctx->buf, (uint32 *)ctx->u.in);

    /* Now fill the next block with 56 bytes */
    memset(ctx->u.in, 0, 56);
  } else {
    /* Pad block to 56 bytes */
    memset(p, 0, count-8);
  }
  byteReverse(ctx->u.in, 14);

  /* Append length in bits and transform */
  ctx->u.in32[14] = ctx->bits[0];
  ctx->u.in32[15] = ctx->bits[1];

  MD5Transform(ctx->buf, (uint32 *)ctx->u.in);
  byteReverse((unsigned char *)ctx->buf, 4);
  memcpy(digest, ctx->buf, 16);
  memset(ctx, 0, sizeof(*ctx));    /* In case it is sensitive */
}

/*
** Convert a 128-bit MD5 digest into a 32-digit base-16 number.
*/
static void MD5DigestToBase16(unsigned char *digest, char *zBuf){
  static char const zEncode[] = "0123456789abcdef";
  int i, j;

  for(j=i=0; i<16; i++){
    int a = digest[i];
    zBuf[j++] = zEncode[(a>>4)&0xf];
    zBuf[j++] = zEncode[a & 0xf];
  }
  zBuf[j] = 0;
}

/*
** During testing, the special md5sum() aggregate function is available.
** inside SQLite.  The following routines implement that function.
*/
static void md5step(sqlite3_context *context, int argc, sqlite3_value **argv){
  MD5Context *p;
  int i;
  if( argc<1 ) return;
  p = sqlite3_aggregate_context(context, sizeof(*p));
  if( p==0 ) return;
  if( !p->isInit ){
    MD5Init(p);
  }
  for(i=0; i<argc; i++){
    const char *zData = (char*)sqlite3_value_text(argv[i]);
    if( zData ){
      MD5Update(p, (unsigned char*)zData, strlen(zData));
    }
  }
}
static void md5finalize(sqlite3_context *context){
  MD5Context *p;
  unsigned char digest[16];
  char zBuf[33];
  p = sqlite3_aggregate_context(context, sizeof(*p));
  MD5Final(digest,p);
  MD5DigestToBase16(digest, zBuf);
  sqlite3_result_text(context, zBuf, -1, SQLITE_TRANSIENT);
}

/*
** End of copied md5sum() code.
**************************************************************************/

typedef sqlite3_int64 i64;

typedef struct Error Error;
typedef struct Sqlite Sqlite;
typedef struct Statement Statement;

typedef struct Threadset Threadset;
typedef struct Thread Thread;

/* Total number of errors in this process so far. */
static int nGlobalErr = 0;

struct Error {
  int rc;
  int iLine;
  char *zErr;
};

struct Sqlite {
  sqlite3 *db;                    /* Database handle */
  Statement *pCache;              /* Linked list of cached statements */
  int nText;                      /* Size of array at aText[] */
  char **aText;                   /* Stored text results */
};

struct Statement {
  sqlite3_stmt *pStmt;            /* Pre-compiled statement handle */
  Statement *pNext;               /* Next statement in linked-list */
};

struct Thread {
  int iTid;                       /* Thread number within test */
  void* pArg;                     /* Pointer argument passed by caller */

  pthread_t tid;                  /* Thread id */
  char *(*xProc)(int, void*);     /* Thread main proc */
  Thread *pNext;                  /* Next in this list of threads */
};

struct Threadset {
  int iMaxTid;                    /* Largest iTid value allocated so far */
  Thread *pThread;                /* Linked list of threads */
};

static void free_err(Error *p){
  sqlite3_free(p->zErr);
  p->zErr = 0;
  p->rc = 0;
}

static void print_err(Error *p){
  if( p->rc!=SQLITE_OK ){
    int isWarn = 0;
    if( p->rc==SQLITE_SCHEMA ) isWarn = 1;
    if( sqlite3_strglob("* - no such table: *",p->zErr)==0 ) isWarn = 1;
    printf("%s: (%d) \"%s\" at line %d\n", isWarn ? "Warning" : "Error",
            p->rc, p->zErr, p->iLine);
    if( !isWarn ) nGlobalErr++;
    fflush(stdout);
  }
}

static void print_and_free_err(Error *p){
  print_err(p);
  free_err(p);
}

static void system_error(Error *pErr, int iSys){
  pErr->rc = iSys;
  pErr->zErr = (char *)sqlite3_malloc(512);
  strerror_r(iSys, pErr->zErr, 512);
  pErr->zErr[511] = '\0';
}

static void sqlite_error(
  Error *pErr, 
  Sqlite *pDb, 
  const char *zFunc
){
  pErr->rc = sqlite3_errcode(pDb->db);
  pErr->zErr = sqlite3_mprintf(
      "sqlite3_%s() - %s (%d)", zFunc, sqlite3_errmsg(pDb->db),
      sqlite3_extended_errcode(pDb->db)
  );
}

static void test_error_x(
  Error *pErr,
  char *zErr
){
  if( pErr->rc==SQLITE_OK ){
    pErr->rc = 1;
    pErr->zErr = zErr;
  }else{
    sqlite3_free(zErr);
  }
}

static void clear_error_x(
  Error *pErr,
  int rc
){
  if( pErr->rc==rc ){
    pErr->rc = SQLITE_OK;
    sqlite3_free(pErr->zErr);
    pErr->zErr = 0;
  }
}

static int busyhandler(void *pArg, int n){
  usleep(10*1000);
  return 1;
}

static void opendb_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* OUT: Database handle */
  const char *zFile,              /* Database file name */
  int bDelete                     /* True to delete db file before opening */
){
  if( pErr->rc==SQLITE_OK ){
    int rc;
    int flags = SQLITE_OPEN_CREATE | SQLITE_OPEN_READWRITE | SQLITE_OPEN_URI;
    if( bDelete ) unlink(zFile);
    rc = sqlite3_open_v2(zFile, &pDb->db, flags, 0);
    if( rc ){
      sqlite_error(pErr, pDb, "open");
      sqlite3_close(pDb->db);
      pDb->db = 0;
    }else{
      sqlite3_create_function(
          pDb->db, "md5sum", -1, SQLITE_UTF8, 0, 0, md5step, md5finalize
      );
      sqlite3_busy_handler(pDb->db, busyhandler, 0);
      sqlite3_exec(pDb->db, "PRAGMA synchronous=OFF", 0, 0, 0);
    }
  }
}

static void closedb_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb                     /* OUT: Database handle */
){
  int rc;
  int i;
  Statement *pIter;
  Statement *pNext;
  for(pIter=pDb->pCache; pIter; pIter=pNext){
    pNext = pIter->pNext;
    sqlite3_finalize(pIter->pStmt);
    sqlite3_free(pIter);
  }
  for(i=0; i<pDb->nText; i++){
    sqlite3_free(pDb->aText[i]);
  }
  sqlite3_free(pDb->aText);
  rc = sqlite3_close(pDb->db);
  if( rc && pErr->rc==SQLITE_OK ){
    pErr->zErr = sqlite3_mprintf("%s", sqlite3_errmsg(pDb->db));
  }
  memset(pDb, 0, sizeof(Sqlite));
}

static void sql_script_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  const char *zSql                /* SQL script to execute */
){
  if( pErr->rc==SQLITE_OK ){
    pErr->rc = sqlite3_exec(pDb->db, zSql, 0, 0, &pErr->zErr);
  }
}

static void sql_script_printf_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  const char *zFormat,            /* SQL printf format string */
  ...                             /* Printf args */
){
  va_list ap;                     /* ... printf arguments */
  va_start(ap, zFormat);
  if( pErr->rc==SQLITE_OK ){
    char *zSql = sqlite3_vmprintf(zFormat, ap);
    pErr->rc = sqlite3_exec(pDb->db, zSql, 0, 0, &pErr->zErr);
    sqlite3_free(zSql);
  }
  va_end(ap);
}

static Statement *getSqlStatement(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  const char *zSql                /* SQL statement */
){
  Statement *pRet;
  int rc;

  for(pRet=pDb->pCache; pRet; pRet=pRet->pNext){
    if( 0==strcmp(sqlite3_sql(pRet->pStmt), zSql) ){
      return pRet;
    }
  }

  pRet = sqlite3_malloc(sizeof(Statement));
  rc = sqlite3_prepare_v2(pDb->db, zSql, -1, &pRet->pStmt, 0);
  if( rc!=SQLITE_OK ){
    sqlite_error(pErr, pDb, "prepare_v2");
    return 0;
  }
  assert( 0==strcmp(sqlite3_sql(pRet->pStmt), zSql) );

  pRet->pNext = pDb->pCache;
  pDb->pCache = pRet;
  return pRet;
}

static sqlite3_stmt *getAndBindSqlStatement(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  va_list ap                      /* SQL followed by parameters */
){
  Statement *pStatement;          /* The SQLite statement wrapper */
  sqlite3_stmt *pStmt;            /* The SQLite statement to return */
  int i;                          /* Used to iterate through parameters */

  pStatement = getSqlStatement(pErr, pDb, va_arg(ap, const char *));
  if( !pStatement ) return 0;
  pStmt = pStatement->pStmt;
  for(i=1; i<=sqlite3_bind_parameter_count(pStmt); i++){
    const char *zName = sqlite3_bind_parameter_name(pStmt, i);
    void * pArg = va_arg(ap, void*);

    switch( zName[1] ){
      case 'i':
        sqlite3_bind_int64(pStmt, i, *(i64 *)pArg);
        break;

      default:
        pErr->rc = 1;
        pErr->zErr = sqlite3_mprintf("Cannot discern type: \"%s\"", zName);
        pStmt = 0;
        break;
    }
  }

  return pStmt;
}

static i64 execsql_i64_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  ...                             /* SQL and pointers to parameter values */
){
  i64 iRet = 0;
  if( pErr->rc==SQLITE_OK ){
    sqlite3_stmt *pStmt;          /* SQL statement to execute */
    va_list ap;                   /* ... arguments */
    va_start(ap, pDb);
    pStmt = getAndBindSqlStatement(pErr, pDb, ap);
    if( pStmt ){
      int first = 1;
      while( SQLITE_ROW==sqlite3_step(pStmt) ){
        if( first && sqlite3_column_count(pStmt)>0 ){
          iRet = sqlite3_column_int64(pStmt, 0);
        }
        first = 0;
      }
      if( SQLITE_OK!=sqlite3_reset(pStmt) ){
        sqlite_error(pErr, pDb, "reset");
      }
    }
    va_end(ap);
  }
  return iRet;
}

static char * execsql_text_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  int iSlot,                      /* Db handle slot to store text in */
  ...                             /* SQL and pointers to parameter values */
){
  char *zRet = 0;

  if( iSlot>=pDb->nText ){
    int nByte = sizeof(char *)*(iSlot+1);
    pDb->aText = (char **)sqlite3_realloc(pDb->aText, nByte);
    memset(&pDb->aText[pDb->nText], 0, sizeof(char*)*(iSlot+1-pDb->nText));
    pDb->nText = iSlot+1;
  }

  if( pErr->rc==SQLITE_OK ){
    sqlite3_stmt *pStmt;          /* SQL statement to execute */
    va_list ap;                   /* ... arguments */
    va_start(ap, iSlot);
    pStmt = getAndBindSqlStatement(pErr, pDb, ap);
    if( pStmt ){
      int first = 1;
      while( SQLITE_ROW==sqlite3_step(pStmt) ){
        if( first && sqlite3_column_count(pStmt)>0 ){
          zRet = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 0));
          sqlite3_free(pDb->aText[iSlot]);
          pDb->aText[iSlot] = zRet;
        }
        first = 0;
      }
      if( SQLITE_OK!=sqlite3_reset(pStmt) ){
        sqlite_error(pErr, pDb, "reset");
      }
    }
    va_end(ap);
  }

  return zRet;
}

static void integrity_check_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb                     /* Database handle */
){
  if( pErr->rc==SQLITE_OK ){
    Statement *pStatement;        /* Statement to execute */
    char *zErr = 0;               /* Integrity check error */

    pStatement = getSqlStatement(pErr, pDb, "PRAGMA integrity_check");
    if( pStatement ){
      sqlite3_stmt *pStmt = pStatement->pStmt;
      while( SQLITE_ROW==sqlite3_step(pStmt) ){
        const char *z = (const char*)sqlite3_column_text(pStmt, 0);
        if( strcmp(z, "ok") ){
          if( zErr==0 ){
            zErr = sqlite3_mprintf("%s", z);
          }else{
            zErr = sqlite3_mprintf("%z\n%s", zErr, z);
          }
        }
      }
      sqlite3_reset(pStmt);

      if( zErr ){
        pErr->zErr = zErr;
        pErr->rc = 1;
      }
    }
  }
}

static void *launch_thread_main(void *pArg){
  Thread *p = (Thread *)pArg;
  return (void *)p->xProc(p->iTid, p->pArg);
}

static void launch_thread_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Threadset *pThreads,            /* Thread set */
  char *(*xProc)(int, void*),     /* Proc to run */
  void *pArg                      /* Argument passed to thread proc */
){
  if( pErr->rc==SQLITE_OK ){
    int iTid = ++pThreads->iMaxTid;
    Thread *p;
    int rc;

    p = (Thread *)sqlite3_malloc(sizeof(Thread));
    memset(p, 0, sizeof(Thread));
    p->iTid = iTid;
    p->pArg = pArg;
    p->xProc = xProc;

    rc = pthread_create(&p->tid, NULL, launch_thread_main, (void *)p);
    if( rc!=0 ){
      system_error(pErr, rc);
      sqlite3_free(p);
    }else{
      p->pNext = pThreads->pThread;
      pThreads->pThread = p;
    }
  }
}

static void join_all_threads_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Threadset *pThreads             /* Thread set */
){
  Thread *p;
  Thread *pNext;
  for(p=pThreads->pThread; p; p=pNext){
    void *ret;
    pNext = p->pNext;
    int rc;
    rc = pthread_join(p->tid, &ret);
    if( rc!=0 ){
      if( pErr->rc==SQLITE_OK ) system_error(pErr, rc);
    }else{
      printf("Thread %d says: %s\n", p->iTid, (ret==0 ? "..." : (char *)ret));
      fflush(stdout);
    }
    sqlite3_free(p);
  }
  pThreads->pThread = 0;
}

static i64 filesize_x(
  Error *pErr,
  const char *zFile
){
  i64 iRet = 0;
  if( pErr->rc==SQLITE_OK ){
    struct stat sStat;
    if( stat(zFile, &sStat) ){
      iRet = -1;
    }else{
      iRet = sStat.st_size;
    }
  }
  return iRet;
}

static void filecopy_x(
  Error *pErr,
  const char *zFrom,
  const char *zTo
){
  if( pErr->rc==SQLITE_OK ){
    i64 nByte = filesize_x(pErr, zFrom);
    if( nByte<0 ){
      test_error_x(pErr, sqlite3_mprintf("no such file: %s", zFrom));
    }else{
      i64 iOff;
      char aBuf[1024];
      int fd1;
      int fd2;
      unlink(zTo);

      fd1 = open(zFrom, O_RDONLY);
      if( fd1<0 ){
        system_error(pErr, errno);
        return;
      }
      fd2 = open(zTo, O_RDWR|O_CREAT|O_EXCL, 0644);
      if( fd2<0 ){
        system_error(pErr, errno);
        close(fd1);
        return;
      }

      iOff = 0;
      while( iOff<nByte ){
        int nCopy = sizeof(aBuf);
        if( nCopy+iOff>nByte ){
          nCopy = nByte - iOff;
        }
        if( nCopy!=read(fd1, aBuf, nCopy) ){
          system_error(pErr, errno);
          break;
        }
        if( nCopy!=write(fd2, aBuf, nCopy) ){
          system_error(pErr, errno);
          break;
        }
        iOff += nCopy;
      }

      close(fd1);
      close(fd2);
    }
  }
}

/* 
** Used by setstoptime() and timetostop().
*/
static double timelimit = 0.0;

static double currentTime(void){
  double t;
  static sqlite3_vfs *pTimelimitVfs = 0;
  if( pTimelimitVfs==0 ) pTimelimitVfs = sqlite3_vfs_find(0);
  if( pTimelimitVfs->iVersion>=2 && pTimelimitVfs->xCurrentTimeInt64!=0 ){
    sqlite3_int64 tm;
    pTimelimitVfs->xCurrentTimeInt64(pTimelimitVfs, &tm);
    t = tm/86400000.0;
  }else{
    pTimelimitVfs->xCurrentTime(pTimelimitVfs, &t);
  }
  return t;
}

static void setstoptime_x(
  Error *pErr,                    /* IN/OUT: Error code */
  int nMs                         /* Milliseconds until "stop time" */
){
  if( pErr->rc==SQLITE_OK ){
    double t = currentTime();
    timelimit = t + ((double)nMs)/(1000.0*60.0*60.0*24.0);
  }
}

static int timetostop_x(
  Error *pErr                     /* IN/OUT: Error code */
){
  int ret = 1;
  if( pErr->rc==SQLITE_OK ){
    double t = currentTime();
    ret = (t >= timelimit);
  }
  return ret;
}


/*************************************************************************
**************************************************************************
**************************************************************************
** End infrastructure. Begin tests.
*/








<
<
<
<

<
<
<

<
<
<
<
<
<
<
<
|

<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<

<
<
<
<
<
<
<
<
<


>






<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







28
29
30
31
32
33
34




35



36








37
38


























39









40
41
42
43
44
45
46
47
48
















































































































































































































































































































































































































































































































































































































































































































































































































































49
50
51
52
53
54
55
**        ./a.out walthread3      -- Run the "walthread3" test
**        ./a.out 'wal*'          -- Run all of the wal* tests
**        ./a.out --help          -- List all available tests
**
** The exit status is non-zero if any test fails.
*/














































#include <sqlite3.h>










#include "test_multiplex.h"
#include "tt3_core.c"

/* Required to link test_multiplex.c */
#ifndef SQLITE_OMIT_WSD
int sqlite3PendingByte = 0x40000000;
#endif


















































































































































































































































































































































































































































































































































































































































































































































































































































/*************************************************************************
**************************************************************************
**************************************************************************
** End infrastructure. Begin tests.
*/

Added test/tt3_core.c.























































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465
466
467
468
469
470
471
472
473
474
475
476
477
478
479
480
481
482
483
484
485
486
487
488
489
490
491
492
493
494
495
496
497
498
499
500
501
502
503
504
505
506
507
508
509
510
511
512
513
514
515
516
517
518
519
520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987
988
989
990
991
992
993
994
995
996
997
998
999
1000
1001
1002
1003
1004
1005
1006
1007
1008
1009
1010
1011
1012
1013
1014
1015
1016
1017
1018
1019
1020
1021
1022
1023
1024
1025
1026
1027
1028
1029
1030
1031
1032
1033
1034
1035
/*
** 2016-05-07
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
*/


#include <unistd.h>
#include <stdio.h>
#include <pthread.h>
#include <assert.h>
#include <sys/types.h> 
#include <sys/stat.h> 
#include <string.h>
#include <fcntl.h>
#include <errno.h>
#include <stdint.h>

/* 
** The "Set Error Line" macro.
*/
#define SEL(e) ((e)->iLine = ((e)->rc ? (e)->iLine : __LINE__))

/* Database functions */
#define opendb(w,x,y,z)         (SEL(w), opendb_x(w,x,y,z))
#define closedb(y,z)            (SEL(y), closedb_x(y,z))

/* Functions to execute SQL */
#define sql_script(x,y,z)       (SEL(x), sql_script_x(x,y,z))
#define integrity_check(x,y)    (SEL(x), integrity_check_x(x,y))
#define execsql_i64(x,y,...)    (SEL(x), execsql_i64_x(x,y,__VA_ARGS__))
#define execsql_text(x,y,z,...) (SEL(x), execsql_text_x(x,y,z,__VA_ARGS__))
#define execsql(x,y,...)        (SEL(x), (void)execsql_i64_x(x,y,__VA_ARGS__))
#define sql_script_printf(x,y,z,...) (                \
    SEL(x), sql_script_printf_x(x,y,z,__VA_ARGS__)    \
) 

/* Thread functions */
#define launch_thread(w,x,y,z)     (SEL(w), launch_thread_x(w,x,y,z))
#define join_all_threads(y,z)      (SEL(y), join_all_threads_x(y,z))

/* Timer functions */
#define setstoptime(y,z)        (SEL(y), setstoptime_x(y,z))
#define timetostop(z)           (SEL(z), timetostop_x(z))

/* Report/clear errors. */
#define test_error(z, ...)      test_error_x(z, sqlite3_mprintf(__VA_ARGS__))
#define clear_error(y,z)        clear_error_x(y, z)

/* File-system operations */
#define filesize(y,z)           (SEL(y), filesize_x(y,z))
#define filecopy(x,y,z)         (SEL(x), filecopy_x(x,y,z))

#define PTR2INT(x) ((int)((intptr_t)x))
#define INT2PTR(x) ((void*)((intptr_t)x))

/*
** End of test code/infrastructure interface macros.
*************************************************************************/


/************************************************************************
** Start of command line processing utilities.
*/
#define CMDLINE_INT     1
#define CMDLINE_BOOL    2
#define CMDLINE_STRING  3

typedef struct CmdlineArg CmdlineArg;
struct CmdlineArg {
  const char *zSwitch;
  int eType;
  int iOffset;
};

static void cmdline_error(const char *zFmt, ...){
  va_list ap;                   /* ... arguments */
  char *zMsg = 0;
  va_start(ap, zFmt);
  zMsg = sqlite3_vmprintf(zFmt, ap);
  fprintf(stderr, "%s\n", zMsg);
  sqlite3_free(zMsg);
  va_end(ap);
  exit(-1);
}

static void cmdline_usage(const char *zPrg, CmdlineArg *apArg){
  int i;
  fprintf(stderr, "Usage: %s SWITCHES\n", zPrg);
  fprintf(stderr, "\n");
  fprintf(stderr, "where switches are\n");
  for(i=0; apArg[i].zSwitch; i++){
    const char *zExtra = "";
    switch( apArg[i].eType ){
      case CMDLINE_STRING: zExtra = "STRING"; break;
      case CMDLINE_INT: zExtra = "N"; break;
      case CMDLINE_BOOL: zExtra = ""; break;
      default:
        zExtra = "???";
        break;
    }
    fprintf(stderr, "  %s %s\n", apArg[i].zSwitch, zExtra);
  }
  fprintf(stderr, "\n");
  exit(-2);
}

static char *cmdline_construct(CmdlineArg *apArg, void *pObj){
  unsigned char *p = (unsigned char*)pObj;
  char *zRet = 0;
  int iArg;

  for(iArg=0; apArg[iArg].zSwitch; iArg++){
    const char *zSpace = (zRet ? " " : "");
    CmdlineArg *pArg = &apArg[iArg];

    switch( pArg->eType ){
      case CMDLINE_STRING: {
        char *zVal = *(char**)(p + pArg->iOffset);
        if( zVal ){
          zRet = sqlite3_mprintf("%z%s%s %s", zRet, zSpace, pArg->zSwitch,zVal);
        }
        break;
      };

      case CMDLINE_INT: {
        zRet = sqlite3_mprintf("%z%s%s %d", zRet, zSpace, pArg->zSwitch, 
            *(int*)(p + pArg->iOffset)
        );
        break;
      };

      case CMDLINE_BOOL: 
        if( *(int*)(p + pArg->iOffset) ){
          zRet = sqlite3_mprintf("%z%s%s", zRet, zSpace, pArg->zSwitch);
        }
        break;
        
      default:
        zRet = sqlite3_mprintf("%z%s%s ???", zRet, zSpace, pArg->zSwitch);
    }
  }

  return zRet;
}

static void cmdline_process(
 CmdlineArg *apArg, 
 int argc,
 const char **argv,
 void *pObj
){
  int i;
  int iArg;
  unsigned char *p = (unsigned char*)pObj;

  for(i=1; i<argc; i++){
    const char *z = argv[i];
    int n = strlen(z);
    int iOpt = -1;

    if( z[0]=='-' && z[1]=='-' ){
      z++;
      n--;
    }

    for(iArg=0; apArg[iArg].zSwitch; iArg++){
      if( 0==sqlite3_strnicmp(apArg[iArg].zSwitch, z, n) ){
        if( iOpt>=0 ){
          cmdline_error("ambiguous switch: %s", z);
        }
        iOpt = iArg;
        switch( apArg[iArg].eType ){
          case CMDLINE_INT:
            i++;
            if( i==argc ){
              cmdline_error("option requires an argument: %s", z);
            }
            *(int*)(p + apArg[iArg].iOffset) = atoi(argv[i]);
            break;

          case CMDLINE_STRING:
            i++;
            if( i==argc ){
              cmdline_error("option requires an argument: %s", z);
            }
            *(char**)(p + apArg[iArg].iOffset) = sqlite3_mprintf("%s", argv[i]);
            break;

          case CMDLINE_BOOL:
            *(int*)(p + apArg[iArg].iOffset) = 1;
            break;

          default:
            assert( 0 );
            cmdline_error("internal error");
            return;
        }
      }
    }

    if( iOpt<0 ){
      cmdline_usage(argv[0], apArg);
    }
  }
}

/*
** End of command line processing utilities.
*************************************************************************/


/*
 * This code implements the MD5 message-digest algorithm.
 * The algorithm is due to Ron Rivest.  This code was
 * written by Colin Plumb in 1993, no copyright is claimed.
 * This code is in the public domain; do with it what you wish.
 *
 * Equivalent code is available from RSA Data Security, Inc.
 * This code has been tested against that, and is equivalent,
 * except that you don't need to include two pages of legalese
 * with every copy.
 *
 * To compute the message digest of a chunk of bytes, declare an
 * MD5Context structure, pass it to MD5Init, call MD5Update as
 * needed on buffers full of bytes, and then call MD5Final, which
 * will fill a supplied 16-byte array with the digest.
 */

/*
 * If compiled on a machine that doesn't have a 32-bit integer,
 * you just set "uint32" to the appropriate datatype for an
 * unsigned 32-bit integer.  For example:
 *
 *       cc -Duint32='unsigned long' md5.c
 *
 */
#ifndef uint32
#  define uint32 unsigned int
#endif

struct MD5Context {
  int isInit;
  uint32 buf[4];
  uint32 bits[2];
  union {
    unsigned char in[64];
    uint32 in32[16];
  } u;
};
typedef struct MD5Context MD5Context;

/*
 * Note: this code is harmless on little-endian machines.
 */
static void byteReverse (unsigned char *buf, unsigned longs){
  uint32 t;
  do {
    t = (uint32)((unsigned)buf[3]<<8 | buf[2]) << 16 |
          ((unsigned)buf[1]<<8 | buf[0]);
    *(uint32 *)buf = t;
    buf += 4;
  } while (--longs);
}
/* The four core functions - F1 is optimized somewhat */

/* #define F1(x, y, z) (x & y | ~x & z) */
#define F1(x, y, z) (z ^ (x & (y ^ z)))
#define F2(x, y, z) F1(z, x, y)
#define F3(x, y, z) (x ^ y ^ z)
#define F4(x, y, z) (y ^ (x | ~z))

/* This is the central step in the MD5 algorithm. */
#define MD5STEP(f, w, x, y, z, data, s) \
  ( w += f(x, y, z) + data,  w = w<<s | w>>(32-s),  w += x )

/*
 * The core of the MD5 algorithm, this alters an existing MD5 hash to
 * reflect the addition of 16 longwords of new data.  MD5Update blocks
 * the data and converts bytes into longwords for this routine.
 */
static void MD5Transform(uint32 buf[4], const uint32 in[16]){
  register uint32 a, b, c, d;

  a = buf[0];
  b = buf[1];
  c = buf[2];
  d = buf[3];

  MD5STEP(F1, a, b, c, d, in[ 0]+0xd76aa478,  7);
  MD5STEP(F1, d, a, b, c, in[ 1]+0xe8c7b756, 12);
  MD5STEP(F1, c, d, a, b, in[ 2]+0x242070db, 17);
  MD5STEP(F1, b, c, d, a, in[ 3]+0xc1bdceee, 22);
  MD5STEP(F1, a, b, c, d, in[ 4]+0xf57c0faf,  7);
  MD5STEP(F1, d, a, b, c, in[ 5]+0x4787c62a, 12);
  MD5STEP(F1, c, d, a, b, in[ 6]+0xa8304613, 17);
  MD5STEP(F1, b, c, d, a, in[ 7]+0xfd469501, 22);
  MD5STEP(F1, a, b, c, d, in[ 8]+0x698098d8,  7);
  MD5STEP(F1, d, a, b, c, in[ 9]+0x8b44f7af, 12);
  MD5STEP(F1, c, d, a, b, in[10]+0xffff5bb1, 17);
  MD5STEP(F1, b, c, d, a, in[11]+0x895cd7be, 22);
  MD5STEP(F1, a, b, c, d, in[12]+0x6b901122,  7);
  MD5STEP(F1, d, a, b, c, in[13]+0xfd987193, 12);
  MD5STEP(F1, c, d, a, b, in[14]+0xa679438e, 17);
  MD5STEP(F1, b, c, d, a, in[15]+0x49b40821, 22);

  MD5STEP(F2, a, b, c, d, in[ 1]+0xf61e2562,  5);
  MD5STEP(F2, d, a, b, c, in[ 6]+0xc040b340,  9);
  MD5STEP(F2, c, d, a, b, in[11]+0x265e5a51, 14);
  MD5STEP(F2, b, c, d, a, in[ 0]+0xe9b6c7aa, 20);
  MD5STEP(F2, a, b, c, d, in[ 5]+0xd62f105d,  5);
  MD5STEP(F2, d, a, b, c, in[10]+0x02441453,  9);
  MD5STEP(F2, c, d, a, b, in[15]+0xd8a1e681, 14);
  MD5STEP(F2, b, c, d, a, in[ 4]+0xe7d3fbc8, 20);
  MD5STEP(F2, a, b, c, d, in[ 9]+0x21e1cde6,  5);
  MD5STEP(F2, d, a, b, c, in[14]+0xc33707d6,  9);
  MD5STEP(F2, c, d, a, b, in[ 3]+0xf4d50d87, 14);
  MD5STEP(F2, b, c, d, a, in[ 8]+0x455a14ed, 20);
  MD5STEP(F2, a, b, c, d, in[13]+0xa9e3e905,  5);
  MD5STEP(F2, d, a, b, c, in[ 2]+0xfcefa3f8,  9);
  MD5STEP(F2, c, d, a, b, in[ 7]+0x676f02d9, 14);
  MD5STEP(F2, b, c, d, a, in[12]+0x8d2a4c8a, 20);

  MD5STEP(F3, a, b, c, d, in[ 5]+0xfffa3942,  4);
  MD5STEP(F3, d, a, b, c, in[ 8]+0x8771f681, 11);
  MD5STEP(F3, c, d, a, b, in[11]+0x6d9d6122, 16);
  MD5STEP(F3, b, c, d, a, in[14]+0xfde5380c, 23);
  MD5STEP(F3, a, b, c, d, in[ 1]+0xa4beea44,  4);
  MD5STEP(F3, d, a, b, c, in[ 4]+0x4bdecfa9, 11);
  MD5STEP(F3, c, d, a, b, in[ 7]+0xf6bb4b60, 16);
  MD5STEP(F3, b, c, d, a, in[10]+0xbebfbc70, 23);
  MD5STEP(F3, a, b, c, d, in[13]+0x289b7ec6,  4);
  MD5STEP(F3, d, a, b, c, in[ 0]+0xeaa127fa, 11);
  MD5STEP(F3, c, d, a, b, in[ 3]+0xd4ef3085, 16);
  MD5STEP(F3, b, c, d, a, in[ 6]+0x04881d05, 23);
  MD5STEP(F3, a, b, c, d, in[ 9]+0xd9d4d039,  4);
  MD5STEP(F3, d, a, b, c, in[12]+0xe6db99e5, 11);
  MD5STEP(F3, c, d, a, b, in[15]+0x1fa27cf8, 16);
  MD5STEP(F3, b, c, d, a, in[ 2]+0xc4ac5665, 23);

  MD5STEP(F4, a, b, c, d, in[ 0]+0xf4292244,  6);
  MD5STEP(F4, d, a, b, c, in[ 7]+0x432aff97, 10);
  MD5STEP(F4, c, d, a, b, in[14]+0xab9423a7, 15);
  MD5STEP(F4, b, c, d, a, in[ 5]+0xfc93a039, 21);
  MD5STEP(F4, a, b, c, d, in[12]+0x655b59c3,  6);
  MD5STEP(F4, d, a, b, c, in[ 3]+0x8f0ccc92, 10);
  MD5STEP(F4, c, d, a, b, in[10]+0xffeff47d, 15);
  MD5STEP(F4, b, c, d, a, in[ 1]+0x85845dd1, 21);
  MD5STEP(F4, a, b, c, d, in[ 8]+0x6fa87e4f,  6);
  MD5STEP(F4, d, a, b, c, in[15]+0xfe2ce6e0, 10);
  MD5STEP(F4, c, d, a, b, in[ 6]+0xa3014314, 15);
  MD5STEP(F4, b, c, d, a, in[13]+0x4e0811a1, 21);
  MD5STEP(F4, a, b, c, d, in[ 4]+0xf7537e82,  6);
  MD5STEP(F4, d, a, b, c, in[11]+0xbd3af235, 10);
  MD5STEP(F4, c, d, a, b, in[ 2]+0x2ad7d2bb, 15);
  MD5STEP(F4, b, c, d, a, in[ 9]+0xeb86d391, 21);

  buf[0] += a;
  buf[1] += b;
  buf[2] += c;
  buf[3] += d;
}

/*
 * Start MD5 accumulation.  Set bit count to 0 and buffer to mysterious
 * initialization constants.
 */
static void MD5Init(MD5Context *ctx){
  ctx->isInit = 1;
  ctx->buf[0] = 0x67452301;
  ctx->buf[1] = 0xefcdab89;
  ctx->buf[2] = 0x98badcfe;
  ctx->buf[3] = 0x10325476;
  ctx->bits[0] = 0;
  ctx->bits[1] = 0;
}

/*
 * Update context to reflect the concatenation of another buffer full
 * of bytes.
 */
static 
void MD5Update(MD5Context *ctx, const unsigned char *buf, unsigned int len){
  uint32 t;

  /* Update bitcount */

  t = ctx->bits[0];
  if ((ctx->bits[0] = t + ((uint32)len << 3)) < t)
    ctx->bits[1]++; /* Carry from low to high */
  ctx->bits[1] += len >> 29;

  t = (t >> 3) & 0x3f;    /* Bytes already in shsInfo->data */

  /* Handle any leading odd-sized chunks */

  if ( t ) {
    unsigned char *p = (unsigned char *)ctx->u.in + t;

    t = 64-t;
    if (len < t) {
      memcpy(p, buf, len);
      return;
    }
    memcpy(p, buf, t);
    byteReverse(ctx->u.in, 16);
    MD5Transform(ctx->buf, (uint32 *)ctx->u.in);
    buf += t;
    len -= t;
  }

  /* Process data in 64-byte chunks */

  while (len >= 64) {
    memcpy(ctx->u.in, buf, 64);
    byteReverse(ctx->u.in, 16);
    MD5Transform(ctx->buf, (uint32 *)ctx->u.in);
    buf += 64;
    len -= 64;
  }

  /* Handle any remaining bytes of data. */

  memcpy(ctx->u.in, buf, len);
}

/*
 * Final wrapup - pad to 64-byte boundary with the bit pattern 
 * 1 0* (64-bit count of bits processed, MSB-first)
 */
static void MD5Final(unsigned char digest[16], MD5Context *ctx){
  unsigned count;
  unsigned char *p;

  /* Compute number of bytes mod 64 */
  count = (ctx->bits[0] >> 3) & 0x3F;

  /* Set the first char of padding to 0x80.  This is safe since there is
     always at least one byte free */
  p = ctx->u.in + count;
  *p++ = 0x80;

  /* Bytes of padding needed to make 64 bytes */
  count = 64 - 1 - count;

  /* Pad out to 56 mod 64 */
  if (count < 8) {
    /* Two lots of padding:  Pad the first block to 64 bytes */
    memset(p, 0, count);
    byteReverse(ctx->u.in, 16);
    MD5Transform(ctx->buf, (uint32 *)ctx->u.in);

    /* Now fill the next block with 56 bytes */
    memset(ctx->u.in, 0, 56);
  } else {
    /* Pad block to 56 bytes */
    memset(p, 0, count-8);
  }
  byteReverse(ctx->u.in, 14);

  /* Append length in bits and transform */
  ctx->u.in32[14] = ctx->bits[0];
  ctx->u.in32[15] = ctx->bits[1];

  MD5Transform(ctx->buf, (uint32 *)ctx->u.in);
  byteReverse((unsigned char *)ctx->buf, 4);
  memcpy(digest, ctx->buf, 16);
  memset(ctx, 0, sizeof(*ctx));    /* In case it is sensitive */
}

/*
** Convert a 128-bit MD5 digest into a 32-digit base-16 number.
*/
static void MD5DigestToBase16(unsigned char *digest, char *zBuf){
  static char const zEncode[] = "0123456789abcdef";
  int i, j;

  for(j=i=0; i<16; i++){
    int a = digest[i];
    zBuf[j++] = zEncode[(a>>4)&0xf];
    zBuf[j++] = zEncode[a & 0xf];
  }
  zBuf[j] = 0;
}

/*
** During testing, the special md5sum() aggregate function is available.
** inside SQLite.  The following routines implement that function.
*/
static void md5step(sqlite3_context *context, int argc, sqlite3_value **argv){
  MD5Context *p;
  int i;
  if( argc<1 ) return;
  p = sqlite3_aggregate_context(context, sizeof(*p));
  if( p==0 ) return;
  if( !p->isInit ){
    MD5Init(p);
  }
  for(i=0; i<argc; i++){
    const char *zData = (char*)sqlite3_value_text(argv[i]);
    if( zData ){
      MD5Update(p, (unsigned char*)zData, strlen(zData));
    }
  }
}
static void md5finalize(sqlite3_context *context){
  MD5Context *p;
  unsigned char digest[16];
  char zBuf[33];
  p = sqlite3_aggregate_context(context, sizeof(*p));
  MD5Final(digest,p);
  MD5DigestToBase16(digest, zBuf);
  sqlite3_result_text(context, zBuf, -1, SQLITE_TRANSIENT);
}

/*
** End of copied md5sum() code.
**************************************************************************/

typedef sqlite3_int64 i64;

typedef struct Error Error;
typedef struct Sqlite Sqlite;
typedef struct Statement Statement;

typedef struct Threadset Threadset;
typedef struct Thread Thread;

/* Total number of errors in this process so far. */
static int nGlobalErr = 0;

struct Error {
  int rc;
  int iLine;
  char *zErr;
};

struct Sqlite {
  sqlite3 *db;                    /* Database handle */
  Statement *pCache;              /* Linked list of cached statements */
  int nText;                      /* Size of array at aText[] */
  char **aText;                   /* Stored text results */
};

struct Statement {
  sqlite3_stmt *pStmt;            /* Pre-compiled statement handle */
  Statement *pNext;               /* Next statement in linked-list */
};

struct Thread {
  int iTid;                       /* Thread number within test */
  void* pArg;                     /* Pointer argument passed by caller */

  pthread_t tid;                  /* Thread id */
  char *(*xProc)(int, void*);     /* Thread main proc */
  Thread *pNext;                  /* Next in this list of threads */
};

struct Threadset {
  int iMaxTid;                    /* Largest iTid value allocated so far */
  Thread *pThread;                /* Linked list of threads */
};

static void free_err(Error *p){
  sqlite3_free(p->zErr);
  p->zErr = 0;
  p->rc = 0;
}

static void print_err(Error *p){
  if( p->rc!=SQLITE_OK ){
    int isWarn = 0;
    if( p->rc==SQLITE_SCHEMA ) isWarn = 1;
    if( sqlite3_strglob("* - no such table: *",p->zErr)==0 ) isWarn = 1;
    printf("%s: (%d) \"%s\" at line %d\n", isWarn ? "Warning" : "Error",
            p->rc, p->zErr, p->iLine);
    if( !isWarn ) nGlobalErr++;
    fflush(stdout);
  }
}

static void print_and_free_err(Error *p){
  print_err(p);
  free_err(p);
}

static void system_error(Error *pErr, int iSys){
  pErr->rc = iSys;
  pErr->zErr = (char *)sqlite3_malloc(512);
  strerror_r(iSys, pErr->zErr, 512);
  pErr->zErr[511] = '\0';
}

static void sqlite_error(
  Error *pErr, 
  Sqlite *pDb, 
  const char *zFunc
){
  pErr->rc = sqlite3_errcode(pDb->db);
  pErr->zErr = sqlite3_mprintf(
      "sqlite3_%s() - %s (%d)", zFunc, sqlite3_errmsg(pDb->db),
      sqlite3_extended_errcode(pDb->db)
  );
}

static void test_error_x(
  Error *pErr,
  char *zErr
){
  if( pErr->rc==SQLITE_OK ){
    pErr->rc = 1;
    pErr->zErr = zErr;
  }else{
    sqlite3_free(zErr);
  }
}

static void clear_error_x(
  Error *pErr,
  int rc
){
  if( pErr->rc==rc ){
    pErr->rc = SQLITE_OK;
    sqlite3_free(pErr->zErr);
    pErr->zErr = 0;
  }
}

static int busyhandler(void *pArg, int n){
  usleep(10*1000);
  return 1;
}

static void opendb_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* OUT: Database handle */
  const char *zFile,              /* Database file name */
  int bDelete                     /* True to delete db file before opening */
){
  if( pErr->rc==SQLITE_OK ){
    int rc;
    int flags = SQLITE_OPEN_CREATE | SQLITE_OPEN_READWRITE | SQLITE_OPEN_URI;
    if( bDelete ) unlink(zFile);
    rc = sqlite3_open_v2(zFile, &pDb->db, flags, 0);
    if( rc ){
      sqlite_error(pErr, pDb, "open");
      sqlite3_close(pDb->db);
      pDb->db = 0;
    }else{
      sqlite3_create_function(
          pDb->db, "md5sum", -1, SQLITE_UTF8, 0, 0, md5step, md5finalize
      );
      sqlite3_busy_handler(pDb->db, busyhandler, 0);
      sqlite3_exec(pDb->db, "PRAGMA synchronous=OFF", 0, 0, 0);
    }
  }
}

static void closedb_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb                     /* OUT: Database handle */
){
  int rc;
  int i;
  Statement *pIter;
  Statement *pNext;
  for(pIter=pDb->pCache; pIter; pIter=pNext){
    pNext = pIter->pNext;
    sqlite3_finalize(pIter->pStmt);
    sqlite3_free(pIter);
  }
  for(i=0; i<pDb->nText; i++){
    sqlite3_free(pDb->aText[i]);
  }
  sqlite3_free(pDb->aText);
  rc = sqlite3_close(pDb->db);
  if( rc && pErr->rc==SQLITE_OK ){
    pErr->zErr = sqlite3_mprintf("%s", sqlite3_errmsg(pDb->db));
  }
  memset(pDb, 0, sizeof(Sqlite));
}

static void sql_script_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  const char *zSql                /* SQL script to execute */
){
  if( pErr->rc==SQLITE_OK ){
    pErr->rc = sqlite3_exec(pDb->db, zSql, 0, 0, &pErr->zErr);
  }
}

static void sql_script_printf_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  const char *zFormat,            /* SQL printf format string */
  ...                             /* Printf args */
){
  va_list ap;                     /* ... printf arguments */
  va_start(ap, zFormat);
  if( pErr->rc==SQLITE_OK ){
    char *zSql = sqlite3_vmprintf(zFormat, ap);
    pErr->rc = sqlite3_exec(pDb->db, zSql, 0, 0, &pErr->zErr);
    sqlite3_free(zSql);
  }
  va_end(ap);
}

static Statement *getSqlStatement(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  const char *zSql                /* SQL statement */
){
  Statement *pRet;
  int rc;

  for(pRet=pDb->pCache; pRet; pRet=pRet->pNext){
    if( 0==strcmp(sqlite3_sql(pRet->pStmt), zSql) ){
      return pRet;
    }
  }

  pRet = sqlite3_malloc(sizeof(Statement));
  rc = sqlite3_prepare_v2(pDb->db, zSql, -1, &pRet->pStmt, 0);
  if( rc!=SQLITE_OK ){
    sqlite_error(pErr, pDb, "prepare_v2");
    return 0;
  }
  assert( 0==strcmp(sqlite3_sql(pRet->pStmt), zSql) );

  pRet->pNext = pDb->pCache;
  pDb->pCache = pRet;
  return pRet;
}

static sqlite3_stmt *getAndBindSqlStatement(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  va_list ap                      /* SQL followed by parameters */
){
  Statement *pStatement;          /* The SQLite statement wrapper */
  sqlite3_stmt *pStmt;            /* The SQLite statement to return */
  int i;                          /* Used to iterate through parameters */

  pStatement = getSqlStatement(pErr, pDb, va_arg(ap, const char *));
  if( !pStatement ) return 0;
  pStmt = pStatement->pStmt;
  for(i=1; i<=sqlite3_bind_parameter_count(pStmt); i++){
    const char *zName = sqlite3_bind_parameter_name(pStmt, i);
    void * pArg = va_arg(ap, void*);

    switch( zName[1] ){
      case 'i':
        sqlite3_bind_int64(pStmt, i, *(i64 *)pArg);
        break;

      default:
        pErr->rc = 1;
        pErr->zErr = sqlite3_mprintf("Cannot discern type: \"%s\"", zName);
        pStmt = 0;
        break;
    }
  }

  return pStmt;
}

static i64 execsql_i64_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  ...                             /* SQL and pointers to parameter values */
){
  i64 iRet = 0;
  if( pErr->rc==SQLITE_OK ){
    sqlite3_stmt *pStmt;          /* SQL statement to execute */
    va_list ap;                   /* ... arguments */
    va_start(ap, pDb);
    pStmt = getAndBindSqlStatement(pErr, pDb, ap);
    if( pStmt ){
      int first = 1;
      while( SQLITE_ROW==sqlite3_step(pStmt) ){
        if( first && sqlite3_column_count(pStmt)>0 ){
          iRet = sqlite3_column_int64(pStmt, 0);
        }
        first = 0;
      }
      if( SQLITE_OK!=sqlite3_reset(pStmt) ){
        sqlite_error(pErr, pDb, "reset");
      }
    }
    va_end(ap);
  }
  return iRet;
}

static char * execsql_text_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb,                    /* Database handle */
  int iSlot,                      /* Db handle slot to store text in */
  ...                             /* SQL and pointers to parameter values */
){
  char *zRet = 0;

  if( iSlot>=pDb->nText ){
    int nByte = sizeof(char *)*(iSlot+1);
    pDb->aText = (char **)sqlite3_realloc(pDb->aText, nByte);
    memset(&pDb->aText[pDb->nText], 0, sizeof(char*)*(iSlot+1-pDb->nText));
    pDb->nText = iSlot+1;
  }

  if( pErr->rc==SQLITE_OK ){
    sqlite3_stmt *pStmt;          /* SQL statement to execute */
    va_list ap;                   /* ... arguments */
    va_start(ap, iSlot);
    pStmt = getAndBindSqlStatement(pErr, pDb, ap);
    if( pStmt ){
      int first = 1;
      while( SQLITE_ROW==sqlite3_step(pStmt) ){
        if( first && sqlite3_column_count(pStmt)>0 ){
          zRet = sqlite3_mprintf("%s", sqlite3_column_text(pStmt, 0));
          sqlite3_free(pDb->aText[iSlot]);
          pDb->aText[iSlot] = zRet;
        }
        first = 0;
      }
      if( SQLITE_OK!=sqlite3_reset(pStmt) ){
        sqlite_error(pErr, pDb, "reset");
      }
    }
    va_end(ap);
  }

  return zRet;
}

static void integrity_check_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Sqlite *pDb                     /* Database handle */
){
  if( pErr->rc==SQLITE_OK ){
    Statement *pStatement;        /* Statement to execute */
    char *zErr = 0;               /* Integrity check error */

    pStatement = getSqlStatement(pErr, pDb, "PRAGMA integrity_check");
    if( pStatement ){
      sqlite3_stmt *pStmt = pStatement->pStmt;
      while( SQLITE_ROW==sqlite3_step(pStmt) ){
        const char *z = (const char*)sqlite3_column_text(pStmt, 0);
        if( strcmp(z, "ok") ){
          if( zErr==0 ){
            zErr = sqlite3_mprintf("%s", z);
          }else{
            zErr = sqlite3_mprintf("%z\n%s", zErr, z);
          }
        }
      }
      sqlite3_reset(pStmt);

      if( zErr ){
        pErr->zErr = zErr;
        pErr->rc = 1;
      }
    }
  }
}

static void *launch_thread_main(void *pArg){
  Thread *p = (Thread *)pArg;
  return (void *)p->xProc(p->iTid, p->pArg);
}

static void launch_thread_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Threadset *pThreads,            /* Thread set */
  char *(*xProc)(int, void*),     /* Proc to run */
  void *pArg                      /* Argument passed to thread proc */
){
  if( pErr->rc==SQLITE_OK ){
    int iTid = ++pThreads->iMaxTid;
    Thread *p;
    int rc;

    p = (Thread *)sqlite3_malloc(sizeof(Thread));
    memset(p, 0, sizeof(Thread));
    p->iTid = iTid;
    p->pArg = pArg;
    p->xProc = xProc;

    rc = pthread_create(&p->tid, NULL, launch_thread_main, (void *)p);
    if( rc!=0 ){
      system_error(pErr, rc);
      sqlite3_free(p);
    }else{
      p->pNext = pThreads->pThread;
      pThreads->pThread = p;
    }
  }
}

static void join_all_threads_x(
  Error *pErr,                    /* IN/OUT: Error code */
  Threadset *pThreads             /* Thread set */
){
  Thread *p;
  Thread *pNext;
  for(p=pThreads->pThread; p; p=pNext){
    void *ret;
    pNext = p->pNext;
    int rc;
    rc = pthread_join(p->tid, &ret);
    if( rc!=0 ){
      if( pErr->rc==SQLITE_OK ) system_error(pErr, rc);
    }else{
      printf("Thread %d says: %s\n", p->iTid, (ret==0 ? "..." : (char *)ret));
      fflush(stdout);
    }
    sqlite3_free(p);
  }
  pThreads->pThread = 0;
}

static i64 filesize_x(
  Error *pErr,
  const char *zFile
){
  i64 iRet = 0;
  if( pErr->rc==SQLITE_OK ){
    struct stat sStat;
    if( stat(zFile, &sStat) ){
      iRet = -1;
    }else{
      iRet = sStat.st_size;
    }
  }
  return iRet;
}

static void filecopy_x(
  Error *pErr,
  const char *zFrom,
  const char *zTo
){
  if( pErr->rc==SQLITE_OK ){
    i64 nByte = filesize_x(pErr, zFrom);
    if( nByte<0 ){
      test_error_x(pErr, sqlite3_mprintf("no such file: %s", zFrom));
    }else{
      i64 iOff;
      char aBuf[1024];
      int fd1;
      int fd2;
      unlink(zTo);

      fd1 = open(zFrom, O_RDONLY);
      if( fd1<0 ){
        system_error(pErr, errno);
        return;
      }
      fd2 = open(zTo, O_RDWR|O_CREAT|O_EXCL, 0644);
      if( fd2<0 ){
        system_error(pErr, errno);
        close(fd1);
        return;
      }

      iOff = 0;
      while( iOff<nByte ){
        int nCopy = sizeof(aBuf);
        if( nCopy+iOff>nByte ){
          nCopy = nByte - iOff;
        }
        if( nCopy!=read(fd1, aBuf, nCopy) ){
          system_error(pErr, errno);
          break;
        }
        if( nCopy!=write(fd2, aBuf, nCopy) ){
          system_error(pErr, errno);
          break;
        }
        iOff += nCopy;
      }

      close(fd1);
      close(fd2);
    }
  }
}

/* 
** Used by setstoptime() and timetostop().
*/
static double timelimit = 0.0;

static double currentTime(void){
  double t;
  static sqlite3_vfs *pTimelimitVfs = 0;
  if( pTimelimitVfs==0 ) pTimelimitVfs = sqlite3_vfs_find(0);
  if( pTimelimitVfs->iVersion>=2 && pTimelimitVfs->xCurrentTimeInt64!=0 ){
    sqlite3_int64 tm;
    pTimelimitVfs->xCurrentTimeInt64(pTimelimitVfs, &tm);
    t = tm/86400000.0;
  }else{
    pTimelimitVfs->xCurrentTime(pTimelimitVfs, &t);
  }
  return t;
}

static void setstoptime_x(
  Error *pErr,                    /* IN/OUT: Error code */
  int nMs                         /* Milliseconds until "stop time" */
){
  if( pErr->rc==SQLITE_OK ){
    double t = currentTime();
    timelimit = t + ((double)nMs)/(1000.0*60.0*60.0*24.0);
  }
}

static int timetostop_x(
  Error *pErr                     /* IN/OUT: Error code */
){
  int ret = 1;
  if( pErr->rc==SQLITE_OK ){
    double t = currentTime();
    ret = (t >= timelimit);
  }
  return ret;
}

Changes to test/wal2.test.

30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
    incr sqlite_sync_count $adj
  } {
    ifcapable !dirsync {
      incr sqlite_sync_count $adj
    }
  }
}

proc set_tvfs_hdr {file args} {

  # Set $nHdr to the number of bytes in the wal-index header:
  set nHdr 48
  set nInt [expr {$nHdr/4}]

  if {[llength $args]>2} {
    error {wrong # args: should be "set_tvfs_hdr fileName ?val1? ?val2?"}
  }

  set blob [tvfs shm $file]
  if {$::tcl_platform(byteOrder)=="bigEndian"} {set fmt I} {set fmt i}

  if {[llength $args]} {
    set ia [lindex $args 0]
    set ib $ia
    if {[llength $args]==2} {
      set ib [lindex $args 1]
    }
    binary scan $blob a[expr $nHdr*2]a* dummy tail
    set blob [binary format ${fmt}${nInt}${fmt}${nInt}a* $ia $ib $tail]
    tvfs shm $file $blob
  }

  binary scan $blob ${fmt}${nInt} ints
  return $ints
}

proc incr_tvfs_hdr {file idx incrval} {
  set ints [set_tvfs_hdr $file]
  set v [lindex $ints $idx]
  incr v $incrval
  lset ints $idx $v
  set_tvfs_hdr $file $ints
}


#-------------------------------------------------------------------------
# Test case wal2-1.*:
#
# Set up a small database containing a single table. The database is not
# checkpointed during the test - all content resides in the log file.
#







<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







30
31
32
33
34
35
36





































37
38
39
40
41
42
43
    incr sqlite_sync_count $adj
  } {
    ifcapable !dirsync {
      incr sqlite_sync_count $adj
    }
  }
}






































#-------------------------------------------------------------------------
# Test case wal2-1.*:
#
# Set up a small database containing a single table. The database is not
# checkpointed during the test - all content resides in the log file.
#

Changes to test/wal_common.tcl.

85
86
87
88
89
90
91
92



93





































  upvar $hdrvar hdr
  set c1 0
  set c2 0
  wal_cksum_intlist c1 c2 [lrange $hdr 0 9]
  lset hdr 10 $c1
  lset hdr 11 $c2
}


















































>
>
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
  upvar $hdrvar hdr
  set c1 0
  set c2 0
  wal_cksum_intlist c1 c2 [lrange $hdr 0 9]
  lset hdr 10 $c1
  lset hdr 11 $c2
}

# This command assumes that $file is the name of a database file opened
# in wal mode using a [testvfs] VFS. It returns a list of the 12 32-bit
# integers that make up the wal-index-header for the named file.
#
proc set_tvfs_hdr {file args} {

  # Set $nHdr to the number of bytes in the wal-index header:
  set nHdr 48
  set nInt [expr {$nHdr/4}]

  if {[llength $args]>2} {
    error {wrong # args: should be "set_tvfs_hdr fileName ?val1? ?val2?"}
  }

  set blob [tvfs shm $file]
  if {$::tcl_platform(byteOrder)=="bigEndian"} {set fmt I} {set fmt i}

  if {[llength $args]} {
    set ia [lindex $args 0]
    set ib $ia
    if {[llength $args]==2} {
      set ib [lindex $args 1]
    }
    binary scan $blob a[expr $nHdr*2]a* dummy tail
    set blob [binary format ${fmt}${nInt}${fmt}${nInt}a* $ia $ib $tail]
    tvfs shm $file $blob
  }

  binary scan $blob ${fmt}${nInt} ints
  return $ints
}

proc incr_tvfs_hdr {file idx incrval} {
  set ints [set_tvfs_hdr $file]
  set v [lindex $ints $idx]
  incr v $incrval
  lset ints $idx $v
  set_tvfs_hdr $file $ints
}