SQLite

Check-in [063f1b3705]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Merge the sqlite3_system_errno() change from trunk.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | apple-osx
Files: files | file ages | folders
SHA1: 063f1b3705625b2f115af8e909d86705b892b529
User & Date: drh 2016-03-21 14:52:04.010
Context
2016-03-29
10:30
Merge 3.12.0 changes. (check-in: eef557449c user: drh tags: apple-osx)
2016-03-21
14:52
Merge the sqlite3_system_errno() change from trunk. (check-in: 063f1b3705 user: drh tags: apple-osx)
14:46
Add the sqlite3_system_errno() interface. (check-in: 4bd12b57ea user: drh tags: trunk)
2016-03-19
00:40
Fix a VDBE register allocation bug exposed by recent enhancements to PRAGMA integrity_check. (check-in: c73b5b9ccf user: drh tags: apple-osx)
Changes
Unified Diff Ignore Whitespace Patch
Changes to doc/lemon.html. 
1
2
3
4
5
6
7
8
9
10
11
12
13
14



15
16
17
18
19
20
21
22

<html>
<head>
<title>The Lemon Parser Generator</title>
</head>
<body bgcolor=white>
<h1 align=center>The Lemon Parser Generator</h1>

<p>Lemon is an LALR(1) parser generator for C or C++.  
It does the same job as ``bison'' and ``yacc''.
But lemon is not another bison or yacc clone.  It
uses a different grammar syntax which is designed to
reduce the number of coding errors.  Lemon also uses a more
sophisticated parsing engine that is faster than yacc and
bison and which is both reentrant and thread-safe.



Furthermore, Lemon implements features that can be used
to eliminate resource leaks, making is suitable for use
in long-running programs such as graphical user interfaces
or embedded controllers.</p>

<p>This document is an introduction to the Lemon
parser generator.</p>









|
|
|

|
|
|
>
>
>
|








1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25

<html>
<head>
<title>The Lemon Parser Generator</title>
</head>
<body bgcolor=white>
<h1 align=center>The Lemon Parser Generator</h1>

<p>Lemon is an LALR(1) parser generator for C.
It does the same job as "bison" and "yacc".
But lemon is not a bison or yacc clone.  Lemon
uses a different grammar syntax which is designed to
reduce the number of coding errors.  Lemon also uses a
parsing engine that is faster than yacc and
bison and which is both reentrant and threadsafe.
(Update: Since the previous sentence was written, bison
has also been updated so that it too can generate a
reentrant and threadsafe parser.)
Lemon also implements features that can be used
to eliminate resource leaks, making is suitable for use
in long-running programs such as graphical user interfaces
or embedded controllers.</p>

<p>This document is an introduction to the Lemon
parser generator.</p>
︙︙
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

80
81
82
83
84
85
86
87
88
89
90
91



92
93
94
95
96

97


98

99
100
101
102
103


104
105
106

107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

<ul>
<li>C code to implement the parser.
<li>A header file defining an integer ID for each terminal symbol.
<li>An information file that describes the states of the generated parser
    automaton.
</ul>
By default, all three of these output files are generated.
The header file is suppressed if the ``-m'' command-line option is
used and the report file is omitted when ``-q'' is selected.</p>

<p>The grammar specification file uses a ``.y'' suffix, by convention.
In the examples used in this document, we'll assume the name of the
grammar file is ``gram.y''.  A typical use of Lemon would be the
following command:
<pre>
   lemon gram.y
</pre>
This command will generate three output files named ``gram.c'',
``gram.h'' and ``gram.out''.
The first is C code to implement the parser.  The second
is the header file that defines numerical values for all
terminal symbols, and the last is the report that explains
the states used by the parser automaton.</p>

<h3>Command Line Options</h3>

<p>The behavior of Lemon can be modified using command-line options.
You can obtain a list of the available command-line options together
with a brief explanation of what each does by typing
<pre>
   lemon -?
</pre>
As of this writing, the following command-line options are supported:
<ul>
<li><tt>-b</tt>
<li><tt>-c</tt>
<li><tt>-g</tt>
<li><tt>-m</tt>
<li><tt>-q</tt>
<li><tt>-s</tt>

<li><tt>-x</tt>
</ul>
The ``-b'' option reduces the amount of text in the report file by
printing only the basis of each parser state, rather than the full
configuration.
The ``-c'' option suppresses action table compression.  Using -c
will make the parser a little larger and slower but it will detect
syntax errors sooner.
The ``-g'' option causes no output files to be generated at all.
Instead, the input grammar file is printed on standard output but
with all comments, actions and other extraneous text deleted.  This
is a useful way to get a quick summary of a grammar.



The ``-m'' option causes the output C source file to be compatible
with the ``makeheaders'' program.
Makeheaders is a program that automatically generates header files
from C source code.  When the ``-m'' option is used, the header
file is not output since the makeheaders program will take care

of generated all header files automatically.


The ``-q'' option suppresses the report file.

Using ``-s'' causes a brief summary of parser statistics to be
printed.  Like this:
<pre>
   Parser statistics: 74 terminals, 70 nonterminals, 179 rules
                      340 states, 2026 parser table entries, 0 conflicts


</pre>
Finally, the ``-x'' option causes Lemon to print its version number
and then stops without attempting to read the grammar or generate a parser.</p>


<h3>The Parser Interface</h3>

<p>Lemon doesn't generate a complete, working program.  It only generates
a few subroutines that implement a parser.  This section describes
the interface to those subroutines.  It is up to the programmer to
call these subroutines in an appropriate way in order to produce a
complete system.</p>

<p>Before a program begins using a Lemon-generated parser, the program
must first create the parser.
A new parser is created as follows:
<pre>
   void *pParser = ParseAlloc( malloc );
</pre>
The ParseAlloc() routine allocates and initializes a new parser and
returns a pointer to it.
The actual data structure used to represent a parser is opaque --
its internal structure is not visible or usable by the calling routine.
For this reason, the ParseAlloc() routine returns a pointer to void
rather than a pointer to some particular structure.
The sole argument to the ParseAlloc() routine is a pointer to the
subroutine used to allocate memory.  Typically this means ``malloc()''.</p>

<p>After a program is finished using a parser, it can reclaim all
memory allocated by that parser by calling
<pre>
   ParseFree(pParser, free);
</pre>
The first argument is the same pointer returned by ParseAlloc().  The








|
|

|

|




|
|















|
|
|
|
|
|
>
|
<
<
<
<
<
<
<
<
|
|
<
>
>
>
|
|
<
<
<
>
|
>
>
|
>
|
<
|
|
<
>
>
|
|
<
>

















|




|








43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84








85
86

87
88
89
90
91



92
93
94
95
96
97
98

99
100

101
102
103
104

105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135

<ul>
<li>C code to implement the parser.
<li>A header file defining an integer ID for each terminal symbol.
<li>An information file that describes the states of the generated parser
    automaton.
</ul>
By default, all three of these output files are generated.
The header file is suppressed if the "-m" command-line option is
used and the report file is omitted when "-q" is selected.</p>

<p>The grammar specification file uses a ".y" suffix, by convention.
In the examples used in this document, we'll assume the name of the
grammar file is "gram.y".  A typical use of Lemon would be the
following command:
<pre>
   lemon gram.y
</pre>
This command will generate three output files named "gram.c",
"gram.h" and "gram.out".
The first is C code to implement the parser.  The second
is the header file that defines numerical values for all
terminal symbols, and the last is the report that explains
the states used by the parser automaton.</p>

<h3>Command Line Options</h3>

<p>The behavior of Lemon can be modified using command-line options.
You can obtain a list of the available command-line options together
with a brief explanation of what each does by typing
<pre>
   lemon -?
</pre>
As of this writing, the following command-line options are supported:
<ul>
<li><b>-b</b>
Show only the basis for each parser state in the report file.
<li><b>-c</b>
Do not compress the generated action tables.
<li><b>-D<i>name</i></b>
Define C preprocessor macro <i>name</i>.  This macro is useable by
"%ifdef" lines in the grammar file.
<li><b>-g</b>








Do not generate a parser.  Instead write the input grammar to standard
output with all comments, actions, and other extraneous text removed.

<li><b>-l</b>
Omit "#line" directives int the generated parser C code.
<li><b>-m</b>
Cause the output C source code to be compatible with the "makeheaders"
program. 



<li><b>-p</b>
Display all conflicts that are resolved by 
<a href='#precrules'>precedence rules</a>.
<li><b>-q</b>
Suppress generation of the report file.
<li><b>-r</b>
Do not sort or renumber the parser states as part of optimization.

<li><b>-s</b>
Show parser statistics before existing.

<li><b>-T<i>file</i></b>
Use <i>file</i> as the template for the generated C-code parser implementation.
<li><b>-x</b>
Print the Lemon version number.

</ul>

<h3>The Parser Interface</h3>

<p>Lemon doesn't generate a complete, working program.  It only generates
a few subroutines that implement a parser.  This section describes
the interface to those subroutines.  It is up to the programmer to
call these subroutines in an appropriate way in order to produce a
complete system.</p>

<p>Before a program begins using a Lemon-generated parser, the program
must first create the parser.
A new parser is created as follows:
<pre>
   void *pParser = ParseAlloc( malloc );
</pre>
The ParseAlloc() routine allocates and initializes a new parser and
returns a pointer to it.
The actual data structure used to represent a parser is opaque &mdash;
its internal structure is not visible or usable by the calling routine.
For this reason, the ParseAlloc() routine returns a pointer to void
rather than a pointer to some particular structure.
The sole argument to the ParseAlloc() routine is a pointer to the
subroutine used to allocate memory.  Typically this means malloc().</p>

<p>After a program is finished using a parser, it can reclaim all
memory allocated by that parser by calling
<pre>
   ParseFree(pParser, free);
</pre>
The first argument is the same pointer returned by ParseAlloc().  The
︙︙
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167

The first argument to the Parse() routine is the pointer returned by
ParseAlloc().
The second argument is a small positive integer that tells the parse the
type of the next token in the data stream.
There is one token type for each terminal symbol in the grammar.
The gram.h file generated by Lemon contains #define statements that
map symbolic terminal symbol names into appropriate integer values.
(A value of 0 for the second argument is a special flag to the
parser to indicate that the end of input has been reached.)
The third argument is the value of the given token.  By default,
the type of the third argument is integer, but the grammar will
usually redefine this type to be some kind of structure.
Typically the second argument will be a broad category of tokens
such as ``identifier'' or ``number'' and the third argument will
be the name of the identifier or the value of the number.</p>

<p>The Parse() function may have either three or four arguments,
depending on the grammar.  If the grammar specification file requests
it (via the <a href='#extraarg'><tt>extra_argument</tt> directive</a>),
the Parse() function will have a fourth parameter that can be
of any type chosen by the programmer.  The parser doesn't do anything








|
|




|








146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166

The first argument to the Parse() routine is the pointer returned by
ParseAlloc().
The second argument is a small positive integer that tells the parse the
type of the next token in the data stream.
There is one token type for each terminal symbol in the grammar.
The gram.h file generated by Lemon contains #define statements that
map symbolic terminal symbol names into appropriate integer values.
A value of 0 for the second argument is a special flag to the
parser to indicate that the end of input has been reached.
The third argument is the value of the given token.  By default,
the type of the third argument is integer, but the grammar will
usually redefine this type to be some kind of structure.
Typically the second argument will be a broad category of tokens
such as "identifier" or "number" and the third argument will
be the name of the identifier or the value of the number.</p>

<p>The Parse() function may have either three or four arguments,
depending on the grammar.  If the grammar specification file requests
it (via the <a href='#extraarg'><tt>extra_argument</tt> directive</a>),
the Parse() function will have a fourth parameter that can be
of any type chosen by the programmer.  The parser doesn't do anything
︙︙
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203

   15    ParseFree(pParser, free );
   16    TokenizerFree(pTokenizer);
   17    return sState.treeRoot;
   18 }
</pre>
This example shows a user-written routine that parses a file of
text and returns a pointer to the parse tree.
(We've omitted all error-handling from this example to keep it
simple.)
We assume the existence of some kind of tokenizer which is created
using TokenizerCreate() on line 8 and deleted by TokenizerFree()
on line 16.  The GetNextToken() function on line 11 retrieves the
next token from the input file and puts its type in the 
integer variable hTokenId.  The sToken variable is assumed to be
some kind of structure that contains details about each token,








|








188
189
190
191
192
193
194
195
196
197
198
199
200
201
202

   15    ParseFree(pParser, free );
   16    TokenizerFree(pTokenizer);
   17    return sState.treeRoot;
   18 }
</pre>
This example shows a user-written routine that parses a file of
text and returns a pointer to the parse tree.
(All error-handling code is omitted from this example to keep it
simple.)
We assume the existence of some kind of tokenizer which is created
using TokenizerCreate() on line 8 and deleted by TokenizerFree()
on line 16.  The GetNextToken() function on line 11 retrieves the
next token from the input file and puts its type in the 
integer variable hTokenId.  The sToken variable is assumed to be
some kind of structure that contains details about each token,
︙︙
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297

declaration can occur at any point in the file.
Lemon ignores whitespace (except where it is needed to separate
tokens) and it honors the same commenting conventions as C and C++.</p>

<h3>Terminals and Nonterminals</h3>

<p>A terminal symbol (token) is any string of alphanumeric
and underscore characters
that begins with an upper case letter.
A terminal can contain lowercase letters after the first character,
but the usual convention is to make terminals all upper case.
A nonterminal, on the other hand, is any string of alphanumeric
and underscore characters than begins with a lower case letter.
Again, the usual convention is to make nonterminals use all lower
case letters.</p>








|








282
283
284
285
286
287
288
289
290
291
292
293
294
295
296

declaration can occur at any point in the file.
Lemon ignores whitespace (except where it is needed to separate
tokens) and it honors the same commenting conventions as C and C++.</p>

<h3>Terminals and Nonterminals</h3>

<p>A terminal symbol (token) is any string of alphanumeric
and/or underscore characters
that begins with an upper case letter.
A terminal can contain lowercase letters after the first character,
but the usual convention is to make terminals all upper case.
A nonterminal, on the other hand, is any string of alphanumeric
and underscore characters than begins with a lower case letter.
Again, the usual convention is to make nonterminals use all lower
case letters.</p>
︙︙
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366

must have alphanumeric names.</p>

<h3>Grammar Rules</h3>

<p>The main component of a Lemon grammar file is a sequence of grammar
rules.
Each grammar rule consists of a nonterminal symbol followed by
the special symbol ``::='' and then a list of terminals and/or nonterminals.
The rule is terminated by a period.
The list of terminals and nonterminals on the right-hand side of the
rule can be empty.
Rules can occur in any order, except that the left-hand side of the
first rule is assumed to be the start symbol for the grammar (unless
specified otherwise using the <tt>%start</tt> directive described below.)
A typical sequence of grammar rules might look something like this:
<pre>
  expr ::= expr PLUS expr.
  expr ::= expr TIMES expr.
  expr ::= LPAREN expr RPAREN.
  expr ::= VALUE.
</pre>
</p>

<p>There is one non-terminal in this example, ``expr'', and five
terminal symbols or tokens: ``PLUS'', ``TIMES'', ``LPAREN'',
``RPAREN'' and ``VALUE''.</p>

<p>Like yacc and bison, Lemon allows the grammar to specify a block
of C code that will be executed whenever a grammar rule is reduced
by the parser.
In Lemon, this action is specified by putting the C code (contained
within curly braces <tt>{...}</tt>) immediately after the
period that closes the rule.
For example:
<pre>
  expr ::= expr PLUS expr.   { printf("Doing an addition...\n"); }
</pre>
</p>

<p>In order to be useful, grammar actions must normally be linked to
their associated grammar rules.
In yacc and bison, this is accomplished by embedding a ``$$'' in the
action to stand for the value of the left-hand side of the rule and
symbols ``$1'', ``$2'', and so forth to stand for the value of
the terminal or nonterminal at position 1, 2 and so forth on the
right-hand side of the rule.
This idea is very powerful, but it is also very error-prone.  The
single most common source of errors in a yacc or bison grammar is
to miscount the number of symbols on the right-hand side of a grammar
rule and say ``$7'' when you really mean ``$8''.</p>

<p>Lemon avoids the need to count grammar symbols by assigning symbolic
names to each symbol in a grammar rule and then using those symbolic
names in the action.
In yacc or bison, one would write this:
<pre>
  expr -> expr PLUS expr  { $$ = $1 + $3; };








|















|
|
|















|

|





|








309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365

must have alphanumeric names.</p>

<h3>Grammar Rules</h3>

<p>The main component of a Lemon grammar file is a sequence of grammar
rules.
Each grammar rule consists of a nonterminal symbol followed by
the special symbol "::=" and then a list of terminals and/or nonterminals.
The rule is terminated by a period.
The list of terminals and nonterminals on the right-hand side of the
rule can be empty.
Rules can occur in any order, except that the left-hand side of the
first rule is assumed to be the start symbol for the grammar (unless
specified otherwise using the <tt>%start</tt> directive described below.)
A typical sequence of grammar rules might look something like this:
<pre>
  expr ::= expr PLUS expr.
  expr ::= expr TIMES expr.
  expr ::= LPAREN expr RPAREN.
  expr ::= VALUE.
</pre>
</p>

<p>There is one non-terminal in this example, "expr", and five
terminal symbols or tokens: "PLUS", "TIMES", "LPAREN",
"RPAREN" and "VALUE".</p>

<p>Like yacc and bison, Lemon allows the grammar to specify a block
of C code that will be executed whenever a grammar rule is reduced
by the parser.
In Lemon, this action is specified by putting the C code (contained
within curly braces <tt>{...}</tt>) immediately after the
period that closes the rule.
For example:
<pre>
  expr ::= expr PLUS expr.   { printf("Doing an addition...\n"); }
</pre>
</p>

<p>In order to be useful, grammar actions must normally be linked to
their associated grammar rules.
In yacc and bison, this is accomplished by embedding a "$$" in the
action to stand for the value of the left-hand side of the rule and
symbols "$1", "$2", and so forth to stand for the value of
the terminal or nonterminal at position 1, 2 and so forth on the
right-hand side of the rule.
This idea is very powerful, but it is also very error-prone.  The
single most common source of errors in a yacc or bison grammar is
to miscount the number of symbols on the right-hand side of a grammar
rule and say "$7" when you really mean "$8".</p>

<p>Lemon avoids the need to count grammar symbols by assigning symbolic
names to each symbol in a grammar rule and then using those symbolic
names in the action.
In yacc or bison, one would write this:
<pre>
  expr -> expr PLUS expr  { $$ = $1 + $3; };
︙︙
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396

397
398
399
400
401
402
403
404
405
406
407



408
409
410
411
412
413
414
415

includes a linking symbol in parentheses but that linking symbol
is not actually used in the reduce action, then an error message
is generated.
For example, the rule
<pre>
  expr(A) ::= expr(B) PLUS expr(C).  { A = B; }
</pre>
will generate an error because the linking symbol ``C'' is used
in the grammar rule but not in the reduce action.</p>

<p>The Lemon notation for linking grammar rules to reduce actions
also facilitates the use of destructors for reclaiming memory
allocated by the values of terminals and nonterminals on the
right-hand side of a rule.</p>


<h3>Precedence Rules</h3>

<p>Lemon resolves parsing ambiguities in exactly the same way as
yacc and bison.  A shift-reduce conflict is resolved in favor
of the shift, and a reduce-reduce conflict is resolved by reducing
whichever rule comes first in the grammar file.</p>

<p>Just like in
yacc and bison, Lemon allows a measure of control 
over the resolution of paring conflicts using precedence rules.
A precedence value can be assigned to any terminal symbol



using the %left, %right or %nonassoc directives.  Terminal symbols
mentioned in earlier directives have a lower precedence that
terminal symbols mentioned in later directives.  For example:</p>

<p><pre>
   %left AND.
   %left OR.
   %nonassoc EQ NE GT GE LT LE.








|







>











>
>
>
|








381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418

includes a linking symbol in parentheses but that linking symbol
is not actually used in the reduce action, then an error message
is generated.
For example, the rule
<pre>
  expr(A) ::= expr(B) PLUS expr(C).  { A = B; }
</pre>
will generate an error because the linking symbol "C" is used
in the grammar rule but not in the reduce action.</p>

<p>The Lemon notation for linking grammar rules to reduce actions
also facilitates the use of destructors for reclaiming memory
allocated by the values of terminals and nonterminals on the
right-hand side of a rule.</p>

<a name='precrules'></a>
<h3>Precedence Rules</h3>

<p>Lemon resolves parsing ambiguities in exactly the same way as
yacc and bison.  A shift-reduce conflict is resolved in favor
of the shift, and a reduce-reduce conflict is resolved by reducing
whichever rule comes first in the grammar file.</p>

<p>Just like in
yacc and bison, Lemon allows a measure of control 
over the resolution of paring conflicts using precedence rules.
A precedence value can be assigned to any terminal symbol
using the 
<a href='#pleft'>%left</a>,
<a href='#pright'>%right</a> or
<a href='#pnonassoc'>%nonassoc</a> directives.  Terminal symbols
mentioned in earlier directives have a lower precedence that
terminal symbols mentioned in later directives.  For example:</p>

<p><pre>
   %left AND.
   %left OR.
   %nonassoc EQ NE GT GE LT LE.
︙︙
521
522
523
524
525
526
527

528



529
530
531
532
533
534
535
536
537
538
539

540
541
542
543

544
545
546
547

548
549
550
551
552
553
554
555

556
557

558
559
560
561
562

563
564
565
566
567
568
569

570
571
572
573
574
575
576

577
578
579
580
581
582
583
584
585
586
587
588
589


<p>Lemon supports the following special directives:
<ul>
<li><tt>%code</tt>
<li><tt>%default_destructor</tt>
<li><tt>%default_type</tt>
<li><tt>%destructor</tt>

<li><tt>%extra_argument</tt>



<li><tt>%include</tt>
<li><tt>%left</tt>
<li><tt>%name</tt>
<li><tt>%nonassoc</tt>
<li><tt>%parse_accept</tt>
<li><tt>%parse_failure </tt>
<li><tt>%right</tt>
<li><tt>%stack_overflow</tt>
<li><tt>%stack_size</tt>
<li><tt>%start_symbol</tt>
<li><tt>%syntax_error</tt>

<li><tt>%token_destructor</tt>
<li><tt>%token_prefix</tt>
<li><tt>%token_type</tt>
<li><tt>%type</tt>

</ul>
Each of these directives will be described separately in the
following sections:</p>


<h4>The <tt>%code</tt> directive</h4>

<p>The %code directive is used to specify addition C/C++ code that
is added to the end of the main output file.  This is similar to
the %include directive except that %include is inserted at the
beginning of the main output file.</p>

<p>%code is typically used to include some action routines or perhaps

a tokenizer as part of the output file.</p>


<h4>The <tt>%default_destructor</tt> directive</h4>

<p>The %default_destructor directive specifies a destructor to 
use for non-terminals that do not have their own destructor
specified by a separate %destructor directive.  See the documentation

on the %destructor directive below for additional information.</p>

<p>In some grammers, many different non-terminal symbols have the
same datatype and hence the same destructor.  This directive is
a convenience way to specify the same destructor for all those
non-terminals using a single statement.</p>


<h4>The <tt>%default_type</tt> directive</h4>

<p>The %default_type directive specifies the datatype of non-terminal
symbols that do no have their own datatype defined using a separate
%type directive.  See the documentation on %type below for addition
information.</p>


<h4>The <tt>%destructor</tt> directive</h4>

<p>The %destructor directive is used to specify a destructor for
a non-terminal symbol.
(See also the %token_destructor directive which is used to
specify a destructor for terminal symbols.)</p>

<p>A non-terminal's destructor is called to dispose of the
non-terminal's value whenever the non-terminal is popped from
the stack.  This includes all of the following circumstances:
<ul>
<li> When a rule reduces and the value of a non-terminal on
     the right-hand side is not linked to C code.








>

>
>
>











>




>




>


|

|
|


>
|

>





>
|






>




|
|

>




|
|








524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
569
570
571
572
573
574
575
576
577
578
579
580
581
582
583
584
585
586
587
588
589
590
591
592
593
594
595
596
597
598
599
600
601
602
603
604


<p>Lemon supports the following special directives:
<ul>
<li><tt>%code</tt>
<li><tt>%default_destructor</tt>
<li><tt>%default_type</tt>
<li><tt>%destructor</tt>
<li><tt>%endif</tt>
<li><tt>%extra_argument</tt>
<li><tt>%fallback</tt>
<li><tt>%ifdef</tt>
<li><tt>%ifndef</tt>
<li><tt>%include</tt>
<li><tt>%left</tt>
<li><tt>%name</tt>
<li><tt>%nonassoc</tt>
<li><tt>%parse_accept</tt>
<li><tt>%parse_failure </tt>
<li><tt>%right</tt>
<li><tt>%stack_overflow</tt>
<li><tt>%stack_size</tt>
<li><tt>%start_symbol</tt>
<li><tt>%syntax_error</tt>
<li><tt>%token_class</tt>
<li><tt>%token_destructor</tt>
<li><tt>%token_prefix</tt>
<li><tt>%token_type</tt>
<li><tt>%type</tt>
<li><tt>%wildcard</tt>
</ul>
Each of these directives will be described separately in the
following sections:</p>

<a name='pcode'></a>
<h4>The <tt>%code</tt> directive</h4>

<p>The %code directive is used to specify addition C code that
is added to the end of the main output file.  This is similar to
the <a href='#pinclude'>%include</a> directive except that %include
is inserted at the beginning of the main output file.</p>

<p>%code is typically used to include some action routines or perhaps
a tokenizer or even the "main()" function 
as part of the output file.</p>

<a name='default_destructor'></a>
<h4>The <tt>%default_destructor</tt> directive</h4>

<p>The %default_destructor directive specifies a destructor to 
use for non-terminals that do not have their own destructor
specified by a separate %destructor directive.  See the documentation
on the <a name='#destructor'>%destructor</a> directive below for
additional information.</p>

<p>In some grammers, many different non-terminal symbols have the
same datatype and hence the same destructor.  This directive is
a convenience way to specify the same destructor for all those
non-terminals using a single statement.</p>

<a name='default_type'></a>
<h4>The <tt>%default_type</tt> directive</h4>

<p>The %default_type directive specifies the datatype of non-terminal
symbols that do no have their own datatype defined using a separate
<a href='#ptype'>%type</a> directive.  
</p>

<a name='destructor'></a>
<h4>The <tt>%destructor</tt> directive</h4>

<p>The %destructor directive is used to specify a destructor for
a non-terminal symbol.
(See also the <a href='#token_destructor'>%token_destructor</a>
directive which is used to specify a destructor for terminal symbols.)</p>

<p>A non-terminal's destructor is called to dispose of the
non-terminal's value whenever the non-terminal is popped from
the stack.  This includes all of the following circumstances:
<ul>
<li> When a rule reduces and the value of a non-terminal on
     the right-hand side is not linked to C code.
︙︙
598
599
600
601
602
603
604
605
606
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638
639
640
641
642
643
644
















































645
646
647
648
649
650
651

652
653
654
655
656
657
658
659
660
661
662
663

664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689

690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711

712
713
714
715

716
717

718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747

748
749
750
751

752
753

754
755
756
757
758
759
760

<pre>
   %type nt {void*}
   %destructor nt { free($$); }
   nt(A) ::= ID NUM.   { A = malloc( 100 ); }
</pre>
This example is a bit contrived but it serves to illustrate how
destructors work.  The example shows a non-terminal named
``nt'' that holds values of type ``void*''.  When the rule for
an ``nt'' reduces, it sets the value of the non-terminal to
space obtained from malloc().  Later, when the nt non-terminal
is popped from the stack, the destructor will fire and call
free() on this malloced space, thus avoiding a memory leak.
(Note that the symbol ``$$'' in the destructor code is replaced
by the value of the non-terminal.)</p>

<p>It is important to note that the value of a non-terminal is passed
to the destructor whenever the non-terminal is removed from the
stack, unless the non-terminal is used in a C-code action.  If
the non-terminal is used by C-code, then it is assumed that the
C-code will take care of destroying it if it should really
be destroyed.  More commonly, the value is used to build some
larger structure and we don't want to destroy it, which is why
the destructor is not called in this circumstance.</p>

<p>By appropriate use of destructors, it is possible to
build a parser using Lemon that can be used within a long-running
program, such as a GUI, that will not leak memory or other resources.
To do the same using yacc or bison is much more difficult.</p>

<a name="extraarg"></a>
<h4>The <tt>%extra_argument</tt> directive</h4>

The %extra_argument directive instructs Lemon to add a 4th parameter
to the parameter list of the Parse() function it generates.  Lemon
doesn't do anything itself with this extra argument, but it does
make the argument available to C-code action routines, destructors,
and so forth.  For example, if the grammar file contains:</p>

<p><pre>
    %extra_argument { MyStruct *pAbc }
</pre></p>

<p>Then the Parse() function generated will have an 4th parameter
of type ``MyStruct*'' and all action routines will have access to
a variable named ``pAbc'' that is the value of the 4th parameter
in the most recent call to Parse().</p>

















































<h4>The <tt>%include</tt> directive</h4>

<p>The %include directive specifies C code that is included at the
top of the generated parser.  You can include any text you want --
the Lemon parser generator copies it blindly.  If you have multiple
%include directives in your grammar file the value of the last
%include directive overwrites all the others.</p.


<p>The %include directive is very handy for getting some extra #include
preprocessor statements at the beginning of the generated parser.
For example:</p>

<p><pre>
   %include {#include &lt;unistd.h&gt;}
</pre></p>

<p>This might be needed, for example, if some of the C actions in the
grammar call functions that are prototyed in unistd.h.</p>


<h4>The <tt>%left</tt> directive</h4>

The %left directive is used (along with the %right and
%nonassoc directives) to declare precedences of terminal
symbols.  Every terminal symbol whose name appears after
a %left directive but before the next period (``.'') is
given the same left-associative precedence value.  Subsequent
%left directives have higher precedence.  For example:</p>

<p><pre>
   %left AND.
   %left OR.
   %nonassoc EQ NE GT GE LT LE.
   %left PLUS MINUS.
   %left TIMES DIVIDE MOD.
   %right EXP NOT.
</pre></p>

<p>Note the period that terminates each %left, %right or %nonassoc
directive.</p>

<p>LALR(1) grammars can get into a situation where they require
a large amount of stack space if you make heavy use or right-associative
operators.  For this reason, it is recommended that you use %left
rather than %right whenever possible.</p>


<h4>The <tt>%name</tt> directive</h4>

<p>By default, the functions generated by Lemon all begin with the
five-character string ``Parse''.  You can change this string to something
different using the %name directive.  For instance:</p>

<p><pre>
   %name Abcde
</pre></p>

<p>Putting this directive in the grammar file will cause Lemon to generate
functions named
<ul>
<li> AbcdeAlloc(),
<li> AbcdeFree(),
<li> AbcdeTrace(), and
<li> Abcde().
</ul>
The %name directive allows you to generator two or more different
parsers and link them all into the same executable.
</p>


<h4>The <tt>%nonassoc</tt> directive</h4>

<p>This directive is used to assign non-associative precedence to
one or more terminal symbols.  See the section on precedence rules

or on the %left directive for additional information.</p>


<h4>The <tt>%parse_accept</tt> directive</h4>

<p>The %parse_accept directive specifies a block of C code that is
executed whenever the parser accepts its input string.  To ``accept''
an input string means that the parser was able to process all tokens
without error.</p>

<p>For example:</p>

<p><pre>
   %parse_accept {
      printf("parsing complete!\n");
   }
</pre></p>


<h4>The <tt>%parse_failure</tt> directive</h4>

<p>The %parse_failure directive specifies a block of C code that
is executed whenever the parser fails complete.  This code is not
executed until the parser has tried and failed to resolve an input
error using is usual error recovery strategy.  The routine is
only invoked when parsing is unable to continue.</p>

<p><pre>
   %parse_failure {
     fprintf(stderr,"Giving up.  Parser is hopelessly lost...\n");
   }
</pre></p>


<h4>The <tt>%right</tt> directive</h4>

<p>This directive is used to assign right-associative precedence to
one or more terminal symbols.  See the section on precedence rules

or on the %left directive for additional information.</p>


<h4>The <tt>%stack_overflow</tt> directive</h4>

<p>The %stack_overflow directive specifies a block of C code that
is executed if the parser's internal stack ever overflows.  Typically
this just prints an error message.  After a stack overflow, the parser
will be unable to continue and must be reset.</p>









|
|



|






|
|



|
|
<
















|
|


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>





|
|
>












>


|
|
|
|




















>



|


















>



|
>
|

>



|











|














>



|
>
|

>








613
614
615
616
617
618
619
620
621
622
623
624
625
626
627
628
629
630
631
632
633
634
635
636
637
638

639
640
641
642
643
644
645
646
647
648
649
650
651
652
653
654
655
656
657
658
659
660
661
662
663
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
711
712
713
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
735
736
737
738
739
740
741
742
743
744
745
746
747
748
749
750
751
752
753
754
755
756
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771
772
773
774
775
776
777
778
779
780
781
782
783
784
785
786
787
788
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831

<pre>
   %type nt {void*}
   %destructor nt { free($$); }
   nt(A) ::= ID NUM.   { A = malloc( 100 ); }
</pre>
This example is a bit contrived but it serves to illustrate how
destructors work.  The example shows a non-terminal named
"nt" that holds values of type "void*".  When the rule for
an "nt" reduces, it sets the value of the non-terminal to
space obtained from malloc().  Later, when the nt non-terminal
is popped from the stack, the destructor will fire and call
free() on this malloced space, thus avoiding a memory leak.
(Note that the symbol "$$" in the destructor code is replaced
by the value of the non-terminal.)</p>

<p>It is important to note that the value of a non-terminal is passed
to the destructor whenever the non-terminal is removed from the
stack, unless the non-terminal is used in a C-code action.  If
the non-terminal is used by C-code, then it is assumed that the
C-code will take care of destroying it.
More commonly, the value is used to build some
larger structure and we don't want to destroy it, which is why
the destructor is not called in this circumstance.</p>

<p>Destructors help avoid memory leaks by automatically freeing
allocated objects when they go out of scope.

To do the same using yacc or bison is much more difficult.</p>

<a name="extraarg"></a>
<h4>The <tt>%extra_argument</tt> directive</h4>

The %extra_argument directive instructs Lemon to add a 4th parameter
to the parameter list of the Parse() function it generates.  Lemon
doesn't do anything itself with this extra argument, but it does
make the argument available to C-code action routines, destructors,
and so forth.  For example, if the grammar file contains:</p>

<p><pre>
    %extra_argument { MyStruct *pAbc }
</pre></p>

<p>Then the Parse() function generated will have an 4th parameter
of type "MyStruct*" and all action routines will have access to
a variable named "pAbc" that is the value of the 4th parameter
in the most recent call to Parse().</p>

<a name='pfallback'></a>
<h4>The <tt>%fallback</tt> directive</h4>

<p>The %fallback directive specifies an alternative meaning for one
or more tokens.  The alternative meaning is tried if the original token
would have generated a syntax error.

<p>The %fallback directive was added to support robust parsing of SQL
syntax in <a href="https://www.sqlite.org/">SQLite</a>.
The SQL language contains a large assortment of keywords, each of which
appears as a different token to the language parser.  SQL contains so
many keywords, that it can be difficult for programmers to keep up with
them all.  Programmers will, therefore, sometimes mistakenly use an
obscure language keyword for an identifier.  The %fallback directive
provides a mechanism to tell the parser:  "If you are unable to parse
this keyword, try treating it as an identifier instead."

<p>The syntax of %fallback is as follows:

<blockquote>
<tt>%fallback</tt>  <i>ID</i> <i>TOKEN...</i> <b>.</b>
</blockquote>

<p>In words, the %fallback directive is followed by a list of token names
terminated by a period.  The first token name is the fallback token - the
token to which all the other tokens fall back to.  The second and subsequent
arguments are tokens which fall back to the token identified by the first
argument.

<a name='pifdef'></a>
<h4>The <tt>%ifdef</tt>, <tt>%ifndef</tt>, and <tt>%endif</tt> directives.</h4>

<p>The %ifdef, %ifndef, and %endif directives are similar to
#ifdef, #ifndef, and #endif in the C-preprocessor, just not as general.
Each of these directives must begin at the left margin.  No whitespace
is allowed between the "%" and the directive name.

<p>Grammar text in between "%ifdef MACRO" and the next nested "%endif" is
ignored unless the "-DMACRO" command-line option is used.  Grammar text
betwen "%ifndef MACRO" and the next nested "%endif" is included except when
the "-DMACRO" command-line option is used.

<p>Note that the argument to %ifdef and %ifndef must be a single 
preprocessor symbol name, not a general expression.  There is no "%else"
directive.


<a name='pinclude'></a>
<h4>The <tt>%include</tt> directive</h4>

<p>The %include directive specifies C code that is included at the
top of the generated parser.  You can include any text you want --
the Lemon parser generator copies it blindly.  If you have multiple
%include directives in your grammar file, their values are concatenated
so that all %include code ultimately appears near the top of the
generated parser, in the same order as it appeared in the grammer.</p>

<p>The %include directive is very handy for getting some extra #include
preprocessor statements at the beginning of the generated parser.
For example:</p>

<p><pre>
   %include {#include &lt;unistd.h&gt;}
</pre></p>

<p>This might be needed, for example, if some of the C actions in the
grammar call functions that are prototyed in unistd.h.</p>

<a name='pleft'></a>
<h4>The <tt>%left</tt> directive</h4>

The %left directive is used (along with the <a href='#pright'>%right</a> and
<a href='#pnonassoc'>%nonassoc</a> directives) to declare precedences of 
terminal symbols.  Every terminal symbol whose name appears after
a %left directive but before the next period (".") is
given the same left-associative precedence value.  Subsequent
%left directives have higher precedence.  For example:</p>

<p><pre>
   %left AND.
   %left OR.
   %nonassoc EQ NE GT GE LT LE.
   %left PLUS MINUS.
   %left TIMES DIVIDE MOD.
   %right EXP NOT.
</pre></p>

<p>Note the period that terminates each %left, %right or %nonassoc
directive.</p>

<p>LALR(1) grammars can get into a situation where they require
a large amount of stack space if you make heavy use or right-associative
operators.  For this reason, it is recommended that you use %left
rather than %right whenever possible.</p>

<a name='pname'></a>
<h4>The <tt>%name</tt> directive</h4>

<p>By default, the functions generated by Lemon all begin with the
five-character string "Parse".  You can change this string to something
different using the %name directive.  For instance:</p>

<p><pre>
   %name Abcde
</pre></p>

<p>Putting this directive in the grammar file will cause Lemon to generate
functions named
<ul>
<li> AbcdeAlloc(),
<li> AbcdeFree(),
<li> AbcdeTrace(), and
<li> Abcde().
</ul>
The %name directive allows you to generator two or more different
parsers and link them all into the same executable.
</p>

<a name='pnonassoc'></a>
<h4>The <tt>%nonassoc</tt> directive</h4>

<p>This directive is used to assign non-associative precedence to
one or more terminal symbols.  See the section on 
<a href='#precrules'>precedence rules</a>
or on the <a href='#pleft'>%left</a> directive for additional information.</p>

<a name='parse_accept'></a>
<h4>The <tt>%parse_accept</tt> directive</h4>

<p>The %parse_accept directive specifies a block of C code that is
executed whenever the parser accepts its input string.  To "accept"
an input string means that the parser was able to process all tokens
without error.</p>

<p>For example:</p>

<p><pre>
   %parse_accept {
      printf("parsing complete!\n");
   }
</pre></p>

<a name='parse_failure'></a>
<h4>The <tt>%parse_failure</tt> directive</h4>

<p>The %parse_failure directive specifies a block of C code that
is executed whenever the parser fails complete.  This code is not
executed until the parser has tried and failed to resolve an input
error using is usual error recovery strategy.  The routine is
only invoked when parsing is unable to continue.</p>

<p><pre>
   %parse_failure {
     fprintf(stderr,"Giving up.  Parser is hopelessly lost...\n");
   }
</pre></p>

<a name='pright'></a>
<h4>The <tt>%right</tt> directive</h4>

<p>This directive is used to assign right-associative precedence to
one or more terminal symbols.  See the section on 
<a href='#precrules'>precedence rules</a>
or on the <a href='#pleft'>%left</a> directive for additional information.</p>

<a name='stack_overflow'></a>
<h4>The <tt>%stack_overflow</tt> directive</h4>

<p>The %stack_overflow directive specifies a block of C code that
is executed if the parser's internal stack ever overflows.  Typically
this just prints an error message.  After a stack overflow, the parser
will be unable to continue and must be reset.</p>
︙︙
775
776
777
778
779
780
781

782
783
784
785
786
787
788
789
790
791
792
793

794
795
796
797
798
799
800
801
802
803

804
805
806
807
808
809
810
811
812
813
814
815

816
817
818
819
820
821
822

</pre>
Not like this:
<pre>
   list ::= element list.      // right-recursion.  Bad!
   list ::= .
</pre>


<h4>The <tt>%stack_size</tt> directive</h4>

<p>If stack overflow is a problem and you can't resolve the trouble
by using left-recursion, then you might want to increase the size
of the parser's stack using this directive.  Put an positive integer
after the %stack_size directive and Lemon will generate a parse
with a stack of the requested size.  The default value is 100.</p>

<p><pre>
   %stack_size 2000
</pre></p>


<h4>The <tt>%start_symbol</tt> directive</h4>

<p>By default, the start-symbol for the grammar that Lemon generates
is the first non-terminal that appears in the grammar file.  But you
can choose a different start-symbol using the %start_symbol directive.</p>

<p><pre>
   %start_symbol  prog
</pre></p>


<h4>The <tt>%token_destructor</tt> directive</h4>

<p>The %destructor directive assigns a destructor to a non-terminal
symbol.  (See the description of the %destructor directive above.)
This directive does the same thing for all terminal symbols.</p>

<p>Unlike non-terminal symbols which may each have a different data type
for their values, terminals all use the same data type (defined by
the %token_type directive) and so they use a common destructor.  Other
than that, the token destructor works just like the non-terminal
destructors.</p>


<h4>The <tt>%token_prefix</tt> directive</h4>

<p>Lemon generates #defines that assign small integer constants
to each terminal symbol in the grammar.  If desired, Lemon will
add a prefix specified by this directive
to each of the #defines it generates.
So if the default output of Lemon looked like this:








>












>










>












>








846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897

</pre>
Not like this:
<pre>
   list ::= element list.      // right-recursion.  Bad!
   list ::= .
</pre>

<a name='stack_size'></a>
<h4>The <tt>%stack_size</tt> directive</h4>

<p>If stack overflow is a problem and you can't resolve the trouble
by using left-recursion, then you might want to increase the size
of the parser's stack using this directive.  Put an positive integer
after the %stack_size directive and Lemon will generate a parse
with a stack of the requested size.  The default value is 100.</p>

<p><pre>
   %stack_size 2000
</pre></p>

<a name='start_symbol'></a>
<h4>The <tt>%start_symbol</tt> directive</h4>

<p>By default, the start-symbol for the grammar that Lemon generates
is the first non-terminal that appears in the grammar file.  But you
can choose a different start-symbol using the %start_symbol directive.</p>

<p><pre>
   %start_symbol  prog
</pre></p>

<a name='token_destructor'></a>
<h4>The <tt>%token_destructor</tt> directive</h4>

<p>The %destructor directive assigns a destructor to a non-terminal
symbol.  (See the description of the %destructor directive above.)
This directive does the same thing for all terminal symbols.</p>

<p>Unlike non-terminal symbols which may each have a different data type
for their values, terminals all use the same data type (defined by
the %token_type directive) and so they use a common destructor.  Other
than that, the token destructor works just like the non-terminal
destructors.</p>

<a name='token_prefix'></a>
<h4>The <tt>%token_prefix</tt> directive</h4>

<p>Lemon generates #defines that assign small integer constants
to each terminal symbol in the grammar.  If desired, Lemon will
add a prefix specified by this directive
to each of the #defines it generates.
So if the default output of Lemon looked like this:
︙︙
834
835
836
837
838
839
840

841
842
843
844
845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
873
874
875
876











877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900

<pre>
    #define TOKEN_AND        1
    #define TOKEN_MINUS      2
    #define TOKEN_OR         3
    #define TOKEN_PLUS       4
</pre>


<h4>The <tt>%token_type</tt> and <tt>%type</tt> directives</h4>

<p>These directives are used to specify the data types for values
on the parser's stack associated with terminal and non-terminal
symbols.  The values of all terminal symbols must be of the same
type.  This turns out to be the same data type as the 3rd parameter
to the Parse() function generated by Lemon.  Typically, you will
make the value of a terminal symbol by a pointer to some kind of
token structure.  Like this:</p>

<p><pre>
   %token_type    {Token*}
</pre></p>

<p>If the data type of terminals is not specified, the default value
is ``int''.</p>

<p>Non-terminal symbols can each have their own data types.  Typically
the data type  of a non-terminal is a pointer to the root of a parse-tree
structure that contains all information about that non-terminal.
For example:</p>

<p><pre>
   %type   expr  {Expr*}
</pre></p>

<p>Each entry on the parser's stack is actually a union containing
instances of all data types for every non-terminal and terminal symbol.
Lemon will automatically use the correct element of this union depending
on what the corresponding non-terminal or terminal symbol is.  But
the grammar designer should keep in mind that the size of the union
will be the size of its largest element.  So if you have a single
non-terminal whose data type requires 1K of storage, then your 100
entry parser stack will require 100K of heap space.  If you are willing
and able to pay that price, fine.  You just need to know.</p>












<h3>Error Processing</h3>

<p>After extensive experimentation over several years, it has been
discovered that the error recovery strategy used by yacc is about
as good as it gets.  And so that is what Lemon uses.</p>

<p>When a Lemon-generated parser encounters a syntax error, it
first invokes the code specified by the %syntax_error directive, if
any.  It then enters its error recovery strategy.  The error recovery
strategy is to begin popping the parsers stack until it enters a
state where it is permitted to shift a special non-terminal symbol
named ``error''.  It then shifts this non-terminal and continues
parsing.  But the %syntax_error routine will not be called again
until at least three new tokens have been successfully shifted.</p>

<p>If the parser pops its stack until the stack is empty, and it still
is unable to shift the error symbol, then the %parse_failed routine
is invoked and the parser resets itself to its start state, ready
to begin parsing a new file.  This is what will happen at the very
first syntax error, of course, if there are no instances of the 
``error'' non-terminal in your grammar.</p>

</body>
</html>








>















|




















>
>
>
>
>
>
>
>
>
>
>











|








|




909
910
911
912
913
914
915
916
917
918
919
920
921
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
959
960
961
962
963
964
965
966
967
968
969
970
971
972
973
974
975
976
977
978
979
980
981
982
983
984
985
986
987

<pre>
    #define TOKEN_AND        1
    #define TOKEN_MINUS      2
    #define TOKEN_OR         3
    #define TOKEN_PLUS       4
</pre>

<a name='token_type'></a><a name='ptype'></a>
<h4>The <tt>%token_type</tt> and <tt>%type</tt> directives</h4>

<p>These directives are used to specify the data types for values
on the parser's stack associated with terminal and non-terminal
symbols.  The values of all terminal symbols must be of the same
type.  This turns out to be the same data type as the 3rd parameter
to the Parse() function generated by Lemon.  Typically, you will
make the value of a terminal symbol by a pointer to some kind of
token structure.  Like this:</p>

<p><pre>
   %token_type    {Token*}
</pre></p>

<p>If the data type of terminals is not specified, the default value
is "int".</p>

<p>Non-terminal symbols can each have their own data types.  Typically
the data type  of a non-terminal is a pointer to the root of a parse-tree
structure that contains all information about that non-terminal.
For example:</p>

<p><pre>
   %type   expr  {Expr*}
</pre></p>

<p>Each entry on the parser's stack is actually a union containing
instances of all data types for every non-terminal and terminal symbol.
Lemon will automatically use the correct element of this union depending
on what the corresponding non-terminal or terminal symbol is.  But
the grammar designer should keep in mind that the size of the union
will be the size of its largest element.  So if you have a single
non-terminal whose data type requires 1K of storage, then your 100
entry parser stack will require 100K of heap space.  If you are willing
and able to pay that price, fine.  You just need to know.</p>

<a name='pwildcard'></a>
<h4>The <tt>%wildcard</tt> directive</h4>

<p>The %wildcard directive is followed by a single token name and a
period.  This directive specifies that the identified token should 
match any input token.

<p>When the generated parser has the choice of matching an input against
the wildcard token and some other token, the other token is always used.
The wildcard token is only matched if there are no other alternatives.

<h3>Error Processing</h3>

<p>After extensive experimentation over several years, it has been
discovered that the error recovery strategy used by yacc is about
as good as it gets.  And so that is what Lemon uses.</p>

<p>When a Lemon-generated parser encounters a syntax error, it
first invokes the code specified by the %syntax_error directive, if
any.  It then enters its error recovery strategy.  The error recovery
strategy is to begin popping the parsers stack until it enters a
state where it is permitted to shift a special non-terminal symbol
named "error".  It then shifts this non-terminal and continues
parsing.  But the %syntax_error routine will not be called again
until at least three new tokens have been successfully shifted.</p>

<p>If the parser pops its stack until the stack is empty, and it still
is unable to shift the error symbol, then the %parse_failed routine
is invoked and the parser resets itself to its start state, ready
to begin parsing a new file.  This is what will happen at the very
first syntax error, of course, if there are no instances of the 
"error" non-terminal in your grammar.</p>

</body>
</html>
Changes to ext/fts5/fts5_index.c.
︙︙
3449
3450
3451
3452
3453
3454
3455
3456


3457


3458
3459
3460
3461
3462
3463


3464
3465
3466
3467













3468
3469
3470
3471
3472
3473
3474

static int fts5AllocateSegid(Fts5Index *p, Fts5Structure *pStruct){
  int iSegid = 0;

  if( p->rc==SQLITE_OK ){
    if( pStruct->nSegment>=FTS5_MAX_SEGMENT ){
      p->rc = SQLITE_FULL;
    }else{
      while( iSegid==0 ){


        int iLvl, iSeg;


        sqlite3_randomness(sizeof(u32), (void*)&iSegid);
        iSegid = iSegid & ((1 << FTS5_DATA_ID_B)-1);
        for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){
          for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){
            if( iSegid==pStruct->aLevel[iLvl].aSeg[iSeg].iSegid ){
              iSegid = 0;


            }
          }
        }
      }













    }
  }

  return iSegid;
}

/*








|
>
>
|
>
>
|
<
|
|
|
<
>
>
|
|
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>








3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461
3462

3463
3464
3465

3466
3467
3468
3469
3470
3471
3472
3473
3474
3475
3476
3477
3478
3479
3480
3481
3482
3483
3484
3485
3486
3487
3488
3489
3490
3491

static int fts5AllocateSegid(Fts5Index *p, Fts5Structure *pStruct){
  int iSegid = 0;

  if( p->rc==SQLITE_OK ){
    if( pStruct->nSegment>=FTS5_MAX_SEGMENT ){
      p->rc = SQLITE_FULL;
    }else{
      /* FTS5_MAX_SEGMENT is currently defined as 2000. So the following
      ** array is 63 elements, or 252 bytes, in size.  */
      u32 aUsed[(FTS5_MAX_SEGMENT+31) / 32];
      int iLvl, iSeg;
      int i;
      u32 mask;
      memset(aUsed, 0, sizeof(aUsed));

      for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){
        for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){
          int iId = pStruct->aLevel[iLvl].aSeg[iSeg].iSegid;

          if( iId<=FTS5_MAX_SEGMENT ){
            aUsed[(iId-1) / 32] |= 1 << ((iId-1) % 32);
          }
        }
      }

      for(i=0; aUsed[i]==0xFFFFFFFF; i++);
      mask = aUsed[i];
      for(iSegid=0; mask & (1 << iSegid); iSegid++);
      iSegid += 1 + i*32;

#ifdef SQLITE_DEBUG
      for(iLvl=0; iLvl<pStruct->nLevel; iLvl++){
        for(iSeg=0; iSeg<pStruct->aLevel[iLvl].nSeg; iSeg++){
          assert( iSegid!=pStruct->aLevel[iLvl].aSeg[iSeg].iSegid );
        }
      }
      assert( iSegid>0 && iSegid<=FTS5_MAX_SEGMENT );
#endif
    }
  }

  return iSegid;
}

/*
︙︙
3905
3906
3907
3908
3909
3910
3911

3912

3913
3914
3915
3916
3917
3918
3919

  Fts5PageWriter *pLeaf = &pWriter->writer;
  if( p->rc==SQLITE_OK ){
    assert( pLeaf->pgno>=1 );
    if( pLeaf->buf.n>4 ){
      fts5WriteFlushLeaf(p, pWriter);
    }
    *pnLeaf = pLeaf->pgno-1;

    fts5WriteFlushBtree(p, pWriter);

  }
  fts5BufferFree(&pLeaf->term);
  fts5BufferFree(&pLeaf->buf);
  fts5BufferFree(&pLeaf->pgidx);
  fts5BufferFree(&pWriter->btterm);

  for(i=0; i<pWriter->nDlidx; i++){








>
|
>








3922
3923
3924
3925
3926
3927
3928
3929
3930
3931
3932
3933
3934
3935
3936
3937
3938

  Fts5PageWriter *pLeaf = &pWriter->writer;
  if( p->rc==SQLITE_OK ){
    assert( pLeaf->pgno>=1 );
    if( pLeaf->buf.n>4 ){
      fts5WriteFlushLeaf(p, pWriter);
    }
    *pnLeaf = pLeaf->pgno-1;
    if( pLeaf->pgno>1 ){
      fts5WriteFlushBtree(p, pWriter);
    }
  }
  fts5BufferFree(&pLeaf->term);
  fts5BufferFree(&pLeaf->buf);
  fts5BufferFree(&pLeaf->pgidx);
  fts5BufferFree(&pWriter->btterm);

  for(i=0; i<pWriter->nDlidx; i++){
︙︙
Changes to ext/fts5/tool/fts5txt2db.tcl.
︙︙
13
14
15
16
17
18
19

20
21
22
23
24
25
26

    {fts5                 "use fts5 (this is the default)"}
    {fts4                 "use fts4"}
    {colsize   "10 10 10" "list of column sizes"}
    {tblname   "t1"       "table name to create"}
    {detail    "full"     "Fts5 detail mode to use"}
    {repeat    1          "Load each file this many times"}
    {prefix    ""         "Fts prefix= option"}

    database
    file...
  } {
  This script is designed to create fts4/5 tables with more than one column.
  The -colsize option should be set to a Tcl list of integer values, one for
  each column in the table. Each value is the number of tokens that will be
  inserted into the column value for each row. For example, setting the -colsize








>








13
14
15
16
17
18
19
20
21
22
23
24
25
26
27

    {fts5                 "use fts5 (this is the default)"}
    {fts4                 "use fts4"}
    {colsize   "10 10 10" "list of column sizes"}
    {tblname   "t1"       "table name to create"}
    {detail    "full"     "Fts5 detail mode to use"}
    {repeat    1          "Load each file this many times"}
    {prefix    ""         "Fts prefix= option"}
    {trans     1          "True to use a transaction"}
    database
    file...
  } {
  This script is designed to create fts4/5 tables with more than one column.
  The -colsize option should be set to a Tcl list of integer values, one for
  each column in the table. Each value is the number of tokens that will be
  inserted into the column value for each row. For example, setting the -colsize
︙︙
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228

set cols [create_table]
set sql "INSERT INTO $A(tblname) VALUES(\$R([lindex $cols 0])"
foreach c [lrange $cols 1 end] {
  append sql ", \$R($c)"
}
append sql ")"

db eval BEGIN
  while {$i < $N} {
    foreach c $cols s $A(colsize) {
      set R($c) [lrange $tokens $i [expr $i+$s-1]]
      incr i $s
    }
    db eval $sql
  }
db eval COMMIT











|







|




211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229

set cols [create_table]
set sql "INSERT INTO $A(tblname) VALUES(\$R([lindex $cols 0])"
foreach c [lrange $cols 1 end] {
  append sql ", \$R($c)"
}
append sql ")"

if {$A(trans)} { db eval BEGIN }
  while {$i < $N} {
    foreach c $cols s $A(colsize) {
      set R($c) [lrange $tokens $i [expr $i+$s-1]]
      incr i $s
    }
    db eval $sql
  }
if {$A(trans)} { db eval COMMIT }
Changes to ext/rbu/rbu1.test.
︙︙
607
608
609
610
611
612
613
614
615
616
617
618
619
620
621

         CREATE TABLE rbu.data_t1(a, b, rbu_control);
         INSERT INTO rbu.data_t1 VALUES(1, 2, 4);
       } {SQLITE_ERROR - invalid rbu_control value}

       9 {
         CREATE TABLE t1(a, b PRIMARY KEY) WITHOUT ROWID;
         CREATE TABLE rbu.data_t1(a, b, rbu_control);
         INSERT INTO rbu.data_t1 VALUES(1, 2, 2);
       } {SQLITE_ERROR - invalid rbu_control value}

       10 {
         CREATE TABLE t2(a, b);
         CREATE TABLE rbu.data_t1(a, b, rbu_control);
         INSERT INTO rbu.data_t1 VALUES(1, 2, 2);
       } {SQLITE_ERROR - no such table: t1}








|








607
608
609
610
611
612
613
614
615
616
617
618
619
620
621

         CREATE TABLE rbu.data_t1(a, b, rbu_control);
         INSERT INTO rbu.data_t1 VALUES(1, 2, 4);
       } {SQLITE_ERROR - invalid rbu_control value}

       9 {
         CREATE TABLE t1(a, b PRIMARY KEY) WITHOUT ROWID;
         CREATE TABLE rbu.data_t1(a, b, rbu_control);
         INSERT INTO rbu.data_t1 VALUES(1, 2, 3);
       } {SQLITE_ERROR - invalid rbu_control value}

       10 {
         CREATE TABLE t2(a, b);
         CREATE TABLE rbu.data_t1(a, b, rbu_control);
         INSERT INTO rbu.data_t1 VALUES(1, 2, 2);
       } {SQLITE_ERROR - no such table: t1}
︙︙
Changes to ext/rbu/rbudiff.test.
︙︙
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47

48
49
50
51
52
53





















54
55
56
57
58
59
60

#
if {![info exists testdir]} {
  set testdir [file join [file dirname [info script]] .. .. test]
}
source $testdir/tester.tcl
set testprefix rbudiff

if {$tcl_platform(platform)=="windows"} {
  set PROG "sqldiff.exe"
} else {
  set PROG "./sqldiff"
}
if {![file exe $PROG]} {
  puts "rbudiff.test cannot run because $PROG is not available"
  finish_test
  return
}
db close

proc get_rbudiff_sql {db1 db2} {
  exec $::PROG --rbu $db1 $db2
}

proc step_rbu {target rbu} {
  while 1 {
    sqlite3rbu rbu $target $rbu
    set rc [rbu step]
    rbu close
    if {$rc != "SQLITE_OK"} break
  }
  set rc
}

proc apply_rbudiff {sql target} {

  forcedelete rbu.db
  sqlite3 rbudb rbu.db
  rbudb eval $sql
  rbudb close
  step_rbu $target rbu.db
}






















proc rbudiff_cksum {db1} {
  set txt ""

  sqlite3 dbtmp $db1
  foreach tbl [dbtmp eval {SELECT name FROM sqlite_master WHERE type='table'}] {
    set cols [list]








<
<
<
|
<
<
<
<
<
<

















>






>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








14
15
16
17
18
19
20



21






22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73

#
if {![info exists testdir]} {
  set testdir [file join [file dirname [info script]] .. .. test]
}
source $testdir/tester.tcl
set testprefix rbudiff




set PROG [test_find_sqldiff]






db close

proc get_rbudiff_sql {db1 db2} {
  exec $::PROG --rbu $db1 $db2
}

proc step_rbu {target rbu} {
  while 1 {
    sqlite3rbu rbu $target $rbu
    set rc [rbu step]
    rbu close
    if {$rc != "SQLITE_OK"} break
  }
  set rc
}

proc apply_rbudiff {sql target} {
  test_rbucount $sql
  forcedelete rbu.db
  sqlite3 rbudb rbu.db
  rbudb eval $sql
  rbudb close
  step_rbu $target rbu.db
}

# The only argument is the output of an [sqldiff -rbu] run. This command
# tests that the contents of the rbu_count table is correct. An exception
# is thrown if it is not.
#
proc test_rbucount {sql} {
  sqlite3 tmpdb ""
  tmpdb eval $sql
  tmpdb eval {
    SELECT name FROM sqlite_master WHERE name LIKE 'data%' AND type='table'
  } {
    set a [tmpdb eval "SELECT count(*) FROM $name"]
    set b [tmpdb eval {SELECT cnt FROM rbu_count WHERE tbl = $name}]
    if {$a != $b} { 
      tmpdb close
      error "rbu_count error - tbl = $name" 
    }
  }
  tmpdb close
  return ""
}

proc rbudiff_cksum {db1} {
  set txt ""

  sqlite3 dbtmp $db1
  foreach tbl [dbtmp eval {SELECT name FROM sqlite_master WHERE type='table'}] {
    set cols [list]
︙︙
Added ext/rbu/rbuprogress.test. 
































































































































































































































































































































































































































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415

# 2016 March 18
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#

source [file join [file dirname [info script]] rbu_common.tcl]
set ::testprefix rbuprogress


proc create_db_file {filename sql} {
  forcedelete $filename
  sqlite3 tmpdb $filename  
  tmpdb eval $sql
  tmpdb close
}

# Create a simple RBU database. That expects to write to a table:
#
#   CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
#
proc create_rbu1 {filename} {
  create_db_file $filename {
    CREATE TABLE data_t1(a, b, c, rbu_control);
    INSERT INTO data_t1 VALUES(1, 2, 3, 0);
    INSERT INTO data_t1 VALUES(2, 'two', 'three', 0);
    INSERT INTO data_t1 VALUES(3, NULL, 8.2, 0);

    CREATE TABLE rbu_count(tbl, cnt);
    INSERT INTO rbu_count VALUES('data_t1', 3);
  }
  return $filename
}


do_execsql_test 1.0 {
  CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
}

do_test 1.1 {
  create_rbu1 rbu.db
  sqlite3rbu rbu test.db rbu.db
  rbu bp_progress
} {0 0}
do_test 1.2 { rbu step ; rbu bp_progress } {3333 0}
do_test 1.3 { rbu step ; rbu bp_progress } {6666 0}
do_test 1.4 { rbu step ; rbu bp_progress } {10000 0}
do_test 1.5 { rbu step ; rbu bp_progress } {10000 0}
do_test 1.6 { rbu step ; rbu bp_progress } {10000 0}
do_test 1.7 { rbu step ; rbu bp_progress } {10000 5000}
do_test 1.8 { rbu step ; rbu bp_progress } {10000 10000}
do_test 1.9 { rbu step ; rbu bp_progress } {10000 10000}

do_test 1.10 {
  rbu close
} {SQLITE_DONE}

#-------------------------------------------------------------------------
#
proc do_sp_test {tn bReopen target rbu reslist} {
  uplevel [list do_test $tn [subst -nocommands {
    if {$bReopen==0} { sqlite3rbu rbu $target $rbu }
    set res [list]
    while 1 {
      if {$bReopen} { sqlite3rbu rbu $target $rbu }
      set rc [rbu step]
      if {[set rc] != "SQLITE_OK"} { rbu close ; error "error 1" }
      lappend res [lindex [rbu bp_progress] 0]
      if {[lindex [set res] end]==10000} break
      if {$bReopen} { rbu close }
    }
    if {[set res] != [list $reslist]} {
      rbu close
      error "1. reslist incorrect (expect=$reslist got=[set res])"
    }

    # One step to clean up the temporary tables used to update the only
    # target table in the rbu database. And one more to move the *-oal 
    # file to *-wal. After each of these steps, the progress remains
    # at "10000 0".
    #
    if {[lindex [list $reslist] 0]!=-1} {
      rbu step
      set res [rbu bp_progress]
      if {[set res] != [list 10000 0]} {
        rbu close
        error "2. reslist incorrect (expect=10000 0 got=[set res])"
      }
    }

    rbu step
    set res [rbu bp_progress]
    if {[set res] != [list 10000 0]} {
      rbu close
      error "3. reslist incorrect (expect=10000 0 got=[set res])"
    }

    # Do the checkpoint.
    while {[rbu step]=="SQLITE_OK"} { 
      foreach {a b} [rbu bp_progress] {}
      if {[set a]!=10000 || [set b]<=0 || [set b]>10000} {
        rbu close
        error "4. reslist incorrect (expect=10000 1..10000 got=[set a] [set b])"
      }
    }

    set res [rbu bp_progress]
    if {[set res] != [list 10000 10000]} {
      rbu close
      error "5. reslist is incorrect (expect=10000 10000 got=[set res])"
    }

    rbu close
  }] {SQLITE_DONE}]
}

foreach {bReopen} { 0 1 } {
  reset_db
  do_test 2.$bReopen.1.0 {
    execsql {
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
    }
    create_db_file rbu.db {
      CREATE TABLE data_t1(a, b, c, rbu_control);
      INSERT INTO data_t1 VALUES(4, 4, 4, 0);
      INSERT INTO data_t1 VALUES(5, 5, 5, 0);
  
      CREATE TABLE rbu_count(tbl, cnt);
      INSERT INTO rbu_count VALUES('data_t1', 2);
    }
  } {}
  do_sp_test 2.$bReopen.1.1 $bReopen test.db rbu.db {5000 10000}
  
  reset_db
  do_test 2.$bReopen.2.0 {
    execsql { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c) }
    create_rbu1 rbu.db
  } {rbu.db}
  do_sp_test 2.$bReopen.2.1 $bReopen test.db rbu.db {3333 6666 10000}
  
  reset_db
  do_test 2.$bReopen.3.0 {
    execsql { 
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
      CREATE INDEX i1 ON t1(b);
      INSERT INTO t1 VALUES(1, 1, 1);
      INSERT INTO t1 VALUES(2, 2, 2);
      INSERT INTO t1 VALUES(3, 3, 3);
    }
    create_db_file rbu.db {
      CREATE TABLE data_t1(a, b, c, rbu_control);
      INSERT INTO data_t1 VALUES(4, 4, 4, 0);
      INSERT INTO data_t1 VALUES(2, NULL, NULL, 1);
      INSERT INTO data_t1 VALUES(5, NULL, NULL, 1);
  
      CREATE TABLE rbu_count(tbl, cnt);
      INSERT INTO rbu_count VALUES('data_t1', 3);
    }
  } {}
  do_sp_test 2.$bReopen.3.1 $bReopen test.db rbu.db {1666 3333 6000 8000 10000}
  
  reset_db
  do_test 2.$bReopen.4.0 {
    execsql { 
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
      CREATE INDEX i1 ON t1(b);
      INSERT INTO t1 VALUES(1, 1, 1);
      INSERT INTO t1 VALUES(2, 2, 2);
      INSERT INTO t1 VALUES(3, 3, 3);
    }
    create_db_file rbu.db {
      CREATE TABLE data_t1(a, b, c, rbu_control);
      INSERT INTO data_t1 VALUES(2, 4, 4, '.xx');
  
      CREATE TABLE rbu_count(tbl, cnt);
      INSERT INTO rbu_count VALUES('data_t1', 1);
    }
  } {}
  do_sp_test 2.$bReopen.4.1 $bReopen test.db rbu.db {3333 6666 10000}
  
  reset_db
  do_test 2.$bReopen.5.0 {
    execsql { 
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
      CREATE INDEX i1 ON t1(b);
      INSERT INTO t1 VALUES(1, 1, 1);
      INSERT INTO t1 VALUES(2, 2, 2);
      INSERT INTO t1 VALUES(3, 3, 3);
    }
    create_db_file rbu.db {
      CREATE TABLE data_t1(a, b, c, rbu_control);
      INSERT INTO data_t1 VALUES(4, NULL, 4, '.xx');
  
      CREATE TABLE rbu_count(tbl, cnt);
      INSERT INTO rbu_count VALUES('data_t1', 1);
    }
  } {}
  do_sp_test 2.$bReopen.5.1 $bReopen test.db rbu.db {10000}

  reset_db
  do_test 2.$bReopen.6.0 {
    execsql { 
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c);
      CREATE INDEX i1 ON t1(b);
      INSERT INTO t1 VALUES(1, 1, 1);
      INSERT INTO t1 VALUES(2, 2, 2);
      INSERT INTO t1 VALUES(3, 3, 3);
    }
    create_db_file rbu.db {
      CREATE TABLE data_t1(a, b, c, rbu_control);
      INSERT INTO data_t1 VALUES(4, 4, 4, 0);
      INSERT INTO data_t1 VALUES(2, NULL, NULL, 1);
      INSERT INTO data_t1 VALUES(5, NULL, NULL, 1);
    }
  } {}
  do_sp_test 2.$bReopen.6.1 $bReopen test.db rbu.db {-1 -1 -1 -1 -1 10000}
}

#-------------------------------------------------------------------------
# The following tests verify that the API works when resuming an update
# during the incremental checkpoint stage.
#
proc do_phase2_test {tn bReopen target rbu nStep} {
  uplevel [list do_test $tn [subst -nocommands {

    # Build the OAL/WAL file:
    sqlite3rbu rbu $target $rbu
    while {[lindex [rbu bp_progress] 0]<10000} { 
      set rc [rbu step]
      if {"SQLITE_OK" != [set rc]} { rbu close }
    }

    # Clean up the temp tables and move the *-oal file to *-wal.
    rbu step
    rbu step

    for {set i 0} {[set i] < $nStep} {incr i} {
      if {$bReopen} {
        rbu close
        sqlite3rbu rbu $target $rbu
      }
      rbu step
      set res [rbu bp_progress]
      set expect [expr (1 + [set i]) * 10000 / $nStep]
      if {[lindex [set res] 1] != [set expect]} {
        error "Have [set res], expected 10000 [set expect]"
      }
    }

    set rc [rbu step]
    if {[set rc] != "SQLITE_DONE"} {
      error "Have [set rc], expected SQLITE_DONE" 
    }

    rbu close
  }] {SQLITE_DONE}]
}

foreach bReopen {0 1} {
  do_test 3.$bReopen.1.0 {
    reset_db
    execsql {
      CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
      CREATE TABLE t2(a INTEGER PRIMARY KEY, b);
      CREATE TABLE t3(a INTEGER PRIMARY KEY, b);
      CREATE TABLE t4(a INTEGER PRIMARY KEY, b);
    }
    create_db_file rbu.db {
      CREATE TABLE data_t1(a, b, rbu_control);
      CREATE TABLE data_t2(a, b, rbu_control);
      CREATE TABLE data_t3(a, b, rbu_control);
      CREATE TABLE data_t4(a, b, rbu_control);
      INSERT INTO data_t1 VALUES(1, 2, 0);
      INSERT INTO data_t2 VALUES(1, 2, 0);
      INSERT INTO data_t3 VALUES(1, 2, 0);
      INSERT INTO data_t4 VALUES(1, 2, 0);
  
      CREATE TABLE rbu_count(tbl, cnt);
      INSERT INTO rbu_count VALUES('data_t1', 1);
      INSERT INTO rbu_count VALUES('data_t2', 1);
      INSERT INTO rbu_count VALUES('data_t3', 1);
      INSERT INTO rbu_count VALUES('data_t4', 1);
    }
  } {}
  do_phase2_test 3.$bReopen.1.1 $bReopen test.db rbu.db 5
}


foreach {bReopen} { 0 1 } {
  foreach {tn tbl} {
    ipk { CREATE TABLE t1(a INTEGER PRIMARY KEY, b, c) }
    wr  { CREATE TABLE t1(a INT PRIMARY KEY, b, c) WITHOUT ROWID }
    pk  { CREATE TABLE t1(a INT PRIMARY KEY, b, c) }
  } {

    foreach {tn2 rbusql r1 r3} {
      1 {
        CREATE TABLE data0_t1(a, b, c, rbu_control);
        INSERT INTO data0_t1 VALUES(15, 15, 15, 0);
        INSERT INTO data0_t1 VALUES(20, 20, 20, 0);
        CREATE TABLE rbu_count(tbl, cnt);
        INSERT INTO rbu_count VALUES('data0_t1', 2); 
      } 
      {2500 5000 7500 10000}
      {1666 3333 5000 6666 8333 10000}

      2 {
        CREATE TABLE data0_t1(a, b, c, rbu_control);
        INSERT INTO data0_t1 VALUES(10, 10, 10, 2);
        CREATE TABLE rbu_count(tbl, cnt);
        INSERT INTO rbu_count VALUES('data0_t1', 1); 
      } 
      {3333 6666 10000}
      {2000 4000 6000 8000 10000}

      3 {
        CREATE TABLE data0_t1(a, b, c, rbu_control);
        INSERT INTO data0_t1 VALUES(7, 7, 7, 2);
        INSERT INTO data0_t1 VALUES(10, 10, 10, 2);
        CREATE TABLE rbu_count(tbl, cnt);
        INSERT INTO rbu_count VALUES('data0_t1', 2); 
      } 
      {2500 4000 6000 8000 10000}
      {1666 2500 3750 5000 6250 7500 8750 10000}

    } {

      reset_db ; execsql $tbl
      do_test 4.$tn.$bReopen.$tn2.0 {
        execsql {
          CREATE INDEX t1c ON t1(c);
          INSERT INTO t1 VALUES(1, 1, 1);
          INSERT INTO t1 VALUES(5, 5, 5);
          INSERT INTO t1 VALUES(10, 10, 10);
        }
        create_db_file rbu.db $rbusql
      } {}

      set R(ipk) $r1
      set R(wr) $r1
      set R(pk) $r3
      do_sp_test 4.$tn.$bReopen.$tn2.1 $bReopen test.db rbu.db $R($tn)
    }
  }
}

foreach {bReopen} { 0 1 } {
  foreach {tn tbl} {
    nopk { 
      CREATE TABLE t1(a, b, c);
      CREATE INDEX t1c ON t1(c);
    }
    vtab { 
      CREATE VIRTUAL TABLE t1 USING fts5(a, b, c);
    }
  } {

    foreach {tn2 rbusql r1 r2} {
      1 {
        CREATE TABLE data0_t1(a, b, c, rbu_rowid, rbu_control);
        INSERT INTO data0_t1 VALUES(15, 15, 15, 4, 0);
        INSERT INTO data0_t1 VALUES(20, 20, 20, 5, 0);
        CREATE TABLE rbu_count(tbl, cnt);
        INSERT INTO rbu_count VALUES('data0_t1', 2); 
      } 
      {2500 5000 7500 10000}
      {5000 10000}

      2 {
        CREATE TABLE data0_t1(rbu_rowid, a, b, c, rbu_control);
        INSERT INTO data0_t1 VALUES(0, 7, 7, 7, 2);
        INSERT INTO data0_t1 VALUES(2, 10, 10, 10, 2);
        CREATE TABLE rbu_count(tbl, cnt);
        INSERT INTO rbu_count VALUES('data0_t1', 2); 
      } 
      {2500 4000 6000 8000 10000}
      {5000 10000}

      3 {
        CREATE TABLE data0_t1(rbu_rowid, a, b, c, rbu_control);
        INSERT INTO data0_t1 VALUES(1, NULL, NULL, NULL, 1);
        INSERT INTO data0_t1 VALUES(2, NULL, NULL, 7, '..x');
        CREATE TABLE rbu_count(tbl, cnt);
        INSERT INTO rbu_count VALUES('data0_t1', 2); 
      } 
      {2500 4000 6000 8000 10000}
      {5000 10000}
    } {

      reset_db ; execsql $tbl
      do_test 5.$tn.$bReopen.$tn2.0 {
        execsql {
          INSERT INTO t1 VALUES(1, 1, 1);
          INSERT INTO t1 VALUES(5, 5, 5);
          INSERT INTO t1 VALUES(10, 10, 10);
        }
        create_db_file rbu.db $rbusql
      } {}

      set R(nopk) $r1
      set R(vtab) $r2
      do_sp_test 5.$tn.$bReopen.$tn2.1 $bReopen test.db rbu.db $R($tn)
    }
  }
}


finish_test
Changes to ext/rbu/sqlite3rbu.c.
︙︙
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157

158
159
160
161
162
163
164

** RBU_STATE_COOKIE:
**   Valid if STAGE==1. The current change-counter cookie value in the 
**   target db file.
**
** RBU_STATE_OALSZ:
**   Valid if STAGE==1. The size in bytes of the *-oal file.
*/
#define RBU_STATE_STAGE       1
#define RBU_STATE_TBL         2
#define RBU_STATE_IDX         3
#define RBU_STATE_ROW         4
#define RBU_STATE_PROGRESS    5
#define RBU_STATE_CKPT        6
#define RBU_STATE_COOKIE      7
#define RBU_STATE_OALSZ       8


#define RBU_STAGE_OAL         1
#define RBU_STAGE_MOVE        2
#define RBU_STAGE_CAPTURE     3
#define RBU_STAGE_CKPT        4
#define RBU_STAGE_DONE        5









|
|
|
|
|
|
|
|
>








143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165

** RBU_STATE_COOKIE:
**   Valid if STAGE==1. The current change-counter cookie value in the 
**   target db file.
**
** RBU_STATE_OALSZ:
**   Valid if STAGE==1. The size in bytes of the *-oal file.
*/
#define RBU_STATE_STAGE        1
#define RBU_STATE_TBL          2
#define RBU_STATE_IDX          3
#define RBU_STATE_ROW          4
#define RBU_STATE_PROGRESS     5
#define RBU_STATE_CKPT         6
#define RBU_STATE_COOKIE       7
#define RBU_STATE_OALSZ        8
#define RBU_STATE_PHASEONESTEP 9

#define RBU_STAGE_OAL         1
#define RBU_STAGE_MOVE        2
#define RBU_STAGE_CAPTURE     3
#define RBU_STAGE_CKPT        4
#define RBU_STAGE_DONE        5
︙︙
196
197
198
199
200
201
202

203
204
205
206
207
208
209

  char *zTbl;
  char *zIdx;
  i64 iWalCksum;
  int nRow;
  i64 nProgress;
  u32 iCookie;
  i64 iOalSz;

};

struct RbuUpdateStmt {
  char *zMask;                    /* Copy of update mask used with pUpdate */
  sqlite3_stmt *pUpdate;          /* Last update statement (or NULL) */
  RbuUpdateStmt *pNext;
};








>








197
198
199
200
201
202
203
204
205
206
207
208
209
210
211

  char *zTbl;
  char *zIdx;
  i64 iWalCksum;
  int nRow;
  i64 nProgress;
  u32 iCookie;
  i64 iOalSz;
  i64 nPhaseOneStep;
};

struct RbuUpdateStmt {
  char *zMask;                    /* Copy of update mask used with pUpdate */
  sqlite3_stmt *pUpdate;          /* Last update statement (or NULL) */
  RbuUpdateStmt *pNext;
};
︙︙
240
241
242
243
244
245
246

247
248
249
250
251
252
253

  int bCleanup;                   /* True in "cleanup" state */
  const char *zTbl;               /* Name of target db table */
  const char *zDataTbl;           /* Name of rbu db table (or null) */
  const char *zIdx;               /* Name of target db index (or null) */
  int iTnum;                      /* Root page of current object */
  int iPkTnum;                    /* If eType==EXTERNAL, root of PK index */
  int bUnique;                    /* Current index is unique */


  /* Statements created by rbuObjIterPrepareAll() */
  int nCol;                       /* Number of columns in current object */
  sqlite3_stmt *pSelect;          /* Source data */
  sqlite3_stmt *pInsert;          /* Statement for INSERT operations */
  sqlite3_stmt *pDelete;          /* Statement for DELETE ops */
  sqlite3_stmt *pTmpInsert;       /* Insert into rbu_tmp_$zDataTbl */








>








242
243
244
245
246
247
248
249
250
251
252
253
254
255
256

  int bCleanup;                   /* True in "cleanup" state */
  const char *zTbl;               /* Name of target db table */
  const char *zDataTbl;           /* Name of rbu db table (or null) */
  const char *zIdx;               /* Name of target db index (or null) */
  int iTnum;                      /* Root page of current object */
  int iPkTnum;                    /* If eType==EXTERNAL, root of PK index */
  int bUnique;                    /* Current index is unique */
  int nIndex;                     /* Number of aux. indexes on table zTbl */

  /* Statements created by rbuObjIterPrepareAll() */
  int nCol;                       /* Number of columns in current object */
  sqlite3_stmt *pSelect;          /* Source data */
  sqlite3_stmt *pInsert;          /* Statement for INSERT operations */
  sqlite3_stmt *pDelete;          /* Statement for DELETE ops */
  sqlite3_stmt *pTmpInsert;       /* Insert into rbu_tmp_$zDataTbl */
︙︙
293
294
295
296
297
298
299





































300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316

317
318
319
320
321
322
323

struct RbuFrame {
  u32 iDbPage;
  u32 iWalFrame;
};

/*
** RBU handle.





































*/
struct sqlite3rbu {
  int eStage;                     /* Value of RBU_STATE_STAGE field */
  sqlite3 *dbMain;                /* target database handle */
  sqlite3 *dbRbu;                 /* rbu database handle */
  char *zTarget;                  /* Path to target db */
  char *zRbu;                     /* Path to rbu db */
  char *zState;                   /* Path to state db (or NULL if zRbu) */
  char zStateDb[5];               /* Db name for state ("stat" or "main") */
  int rc;                         /* Value returned by last rbu_step() call */
  char *zErrmsg;                  /* Error message if rc!=SQLITE_OK */
  int nStep;                      /* Rows processed for current object */
  int nProgress;                  /* Rows processed for all objects */
  RbuObjIter objiter;             /* Iterator for skipping through tbl/idx */
  const char *zVfsName;           /* Name of automatically created rbu vfs */
  rbu_file *pTargetFd;            /* File handle open on target db */
  i64 iOalSz;


  /* The following state variables are used as part of the incremental
  ** checkpoint stage (eStage==RBU_STAGE_CKPT). See comments surrounding
  ** function rbuSetupCheckpoint() for details.  */
  u32 iMaxFrame;                  /* Largest iWalFrame value in aFrame[] */
  u32 mLock;
  int nFrame;                     /* Entries in aFrame[] array */








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

















>








296
297
298
299
300
301
302
303
304
305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364

struct RbuFrame {
  u32 iDbPage;
  u32 iWalFrame;
};

/*
** RBU handle.
**
** nPhaseOneStep:
**   If the RBU database contains an rbu_count table, this value is set to
**   a running estimate of the number of b-tree operations required to 
**   finish populating the *-oal file. This allows the sqlite3_bp_progress()
**   API to calculate the permyriadage progress of populating the *-oal file
**   using the formula:
**
**     permyriadage = (10000 * nProgress) / nPhaseOneStep
**
**   nPhaseOneStep is initialized to the sum of:
**
**     nRow * (nIndex + 1)
**
**   for all source tables in the RBU database, where nRow is the number
**   of rows in the source table and nIndex the number of indexes on the
**   corresponding target database table.
**
**   This estimate is accurate if the RBU update consists entirely of
**   INSERT operations. However, it is inaccurate if:
**
**     * the RBU update contains any UPDATE operations. If the PK specified
**       for an UPDATE operation does not exist in the target table, then
**       no b-tree operations are required on index b-trees. Or if the 
**       specified PK does exist, then (nIndex*2) such operations are
**       required (one delete and one insert on each index b-tree).
**
**     * the RBU update contains any DELETE operations for which the specified
**       PK does not exist. In this case no operations are required on index
**       b-trees.
**
**     * the RBU update contains REPLACE operations. These are similar to
**       UPDATE operations.
**
**   nPhaseOneStep is updated to account for the conditions above during the
**   first pass of each source table. The updated nPhaseOneStep value is
**   stored in the rbu_state table if the RBU update is suspended.
*/
struct sqlite3rbu {
  int eStage;                     /* Value of RBU_STATE_STAGE field */
  sqlite3 *dbMain;                /* target database handle */
  sqlite3 *dbRbu;                 /* rbu database handle */
  char *zTarget;                  /* Path to target db */
  char *zRbu;                     /* Path to rbu db */
  char *zState;                   /* Path to state db (or NULL if zRbu) */
  char zStateDb[5];               /* Db name for state ("stat" or "main") */
  int rc;                         /* Value returned by last rbu_step() call */
  char *zErrmsg;                  /* Error message if rc!=SQLITE_OK */
  int nStep;                      /* Rows processed for current object */
  int nProgress;                  /* Rows processed for all objects */
  RbuObjIter objiter;             /* Iterator for skipping through tbl/idx */
  const char *zVfsName;           /* Name of automatically created rbu vfs */
  rbu_file *pTargetFd;            /* File handle open on target db */
  i64 iOalSz;
  i64 nPhaseOneStep;

  /* The following state variables are used as part of the incremental
  ** checkpoint stage (eStage==RBU_STAGE_CKPT). See comments surrounding
  ** function rbuSetupCheckpoint() for details.  */
  u32 iMaxFrame;                  /* Largest iWalFrame value in aFrame[] */
  u32 mLock;
  int nFrame;                     /* Entries in aFrame[] array */
︙︙
1140
1141
1142
1143
1144
1145
1146

1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159






1160
1161
1162
1163
1164
1165
1166

  if( p->rc==SQLITE_OK ){
    memcpy(pIter->abIndexed, pIter->abTblPk, sizeof(u8)*pIter->nTblCol);
    p->rc = prepareFreeAndCollectError(p->dbMain, &pList, &p->zErrmsg,
        sqlite3_mprintf("PRAGMA main.index_list = %Q", pIter->zTbl)
    );
  }


  while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pList) ){
    const char *zIdx = (const char*)sqlite3_column_text(pList, 1);
    sqlite3_stmt *pXInfo = 0;
    if( zIdx==0 ) break;
    p->rc = prepareFreeAndCollectError(p->dbMain, &pXInfo, &p->zErrmsg,
        sqlite3_mprintf("PRAGMA main.index_xinfo = %Q", zIdx)
    );
    while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){
      int iCid = sqlite3_column_int(pXInfo, 1);
      if( iCid>=0 ) pIter->abIndexed[iCid] = 1;
    }
    rbuFinalize(p, pXInfo);
    bIndex = 1;






  }

  rbuFinalize(p, pList);
  if( bIndex==0 ) pIter->abIndexed = 0;
}










>













>
>
>
>
>
>








1181
1182
1183
1184
1185
1186
1187
1188
1189
1190
1191
1192
1193
1194
1195
1196
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214

  if( p->rc==SQLITE_OK ){
    memcpy(pIter->abIndexed, pIter->abTblPk, sizeof(u8)*pIter->nTblCol);
    p->rc = prepareFreeAndCollectError(p->dbMain, &pList, &p->zErrmsg,
        sqlite3_mprintf("PRAGMA main.index_list = %Q", pIter->zTbl)
    );
  }

  pIter->nIndex = 0;
  while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pList) ){
    const char *zIdx = (const char*)sqlite3_column_text(pList, 1);
    sqlite3_stmt *pXInfo = 0;
    if( zIdx==0 ) break;
    p->rc = prepareFreeAndCollectError(p->dbMain, &pXInfo, &p->zErrmsg,
        sqlite3_mprintf("PRAGMA main.index_xinfo = %Q", zIdx)
    );
    while( p->rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pXInfo) ){
      int iCid = sqlite3_column_int(pXInfo, 1);
      if( iCid>=0 ) pIter->abIndexed[iCid] = 1;
    }
    rbuFinalize(p, pXInfo);
    bIndex = 1;
    pIter->nIndex++;
  }

  if( pIter->eType==RBU_PK_WITHOUT_ROWID ){
    /* "PRAGMA index_list" includes the main PK b-tree */
    pIter->nIndex--;
  }

  rbuFinalize(p, pList);
  if( bIndex==0 ) pIter->abIndexed = 0;
}
︙︙
1266
1267
1268
1269
1270
1271
1272

1273
1274
1275
1276
1277
1278
1279

        iOrder++;
      }
    }

    rbuFinalize(p, pStmt);
    rbuObjIterCacheIndexedCols(p, pIter);
    assert( pIter->eType!=RBU_PK_VTAB || pIter->abIndexed==0 );

  }

  return p->rc;
}

/*
** This function constructs and returns a pointer to a nul-terminated 








>








1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327
1328

        iOrder++;
      }
    }

    rbuFinalize(p, pStmt);
    rbuObjIterCacheIndexedCols(p, pIter);
    assert( pIter->eType!=RBU_PK_VTAB || pIter->abIndexed==0 );
    assert( pIter->eType!=RBU_PK_VTAB || pIter->nIndex==0 );
  }

  return p->rc;
}

/*
** This function constructs and returns a pointer to a nul-terminated
︙︙
1818
1819
1820
1821
1822
1823
1824








1825
1826
1827
1828
1829
1830
1831

  sqlite3_context *pCtx, 
  int nVal,
  sqlite3_value **apVal
){
  sqlite3rbu *p = sqlite3_user_data(pCtx);
  int rc = SQLITE_OK;
  int i;









  for(i=0; rc==SQLITE_OK && i<nVal; i++){
    rc = sqlite3_bind_value(p->objiter.pTmpInsert, i+1, apVal[i]);
  }
  if( rc==SQLITE_OK ){
    sqlite3_step(p->objiter.pTmpInsert);
    rc = sqlite3_reset(p->objiter.pTmpInsert);








>
>
>
>
>
>
>
>








1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888

  sqlite3_context *pCtx, 
  int nVal,
  sqlite3_value **apVal
){
  sqlite3rbu *p = sqlite3_user_data(pCtx);
  int rc = SQLITE_OK;
  int i;

  assert( sqlite3_value_int(apVal[0])!=0
      || p->objiter.eType==RBU_PK_EXTERNAL 
      || p->objiter.eType==RBU_PK_NONE 
  );
  if( sqlite3_value_int(apVal[0])!=0 ){
    p->nPhaseOneStep += p->objiter.nIndex;
  }

  for(i=0; rc==SQLITE_OK && i<nVal; i++){
    rc = sqlite3_bind_value(p->objiter.pTmpInsert, i+1, apVal[i]);
  }
  if( rc==SQLITE_OK ){
    sqlite3_step(p->objiter.pTmpInsert);
    rc = sqlite3_reset(p->objiter.pTmpInsert);
︙︙
2563
2564
2565
2566
2567
2568
2569











2570
2571
2572
2573
2574
2575
2576

  RbuObjIter *pIter = &p->objiter;
  sqlite3_value *pVal;
  sqlite3_stmt *pWriter;
  int i;

  assert( p->rc==SQLITE_OK );
  assert( eType!=RBU_DELETE || pIter->zIdx==0 );












  if( eType==RBU_IDX_DELETE || eType==RBU_DELETE ){
    pWriter = pIter->pDelete;
  }else{
    pWriter = pIter->pInsert;
  }









>
>
>
>
>
>
>
>
>
>
>








2620
2621
2622
2623
2624
2625
2626
2627
2628
2629
2630
2631
2632
2633
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644

  RbuObjIter *pIter = &p->objiter;
  sqlite3_value *pVal;
  sqlite3_stmt *pWriter;
  int i;

  assert( p->rc==SQLITE_OK );
  assert( eType!=RBU_DELETE || pIter->zIdx==0 );
  assert( eType==RBU_DELETE || eType==RBU_IDX_DELETE
       || eType==RBU_INSERT || eType==RBU_IDX_INSERT
  );

  /* If this is a delete, decrement nPhaseOneStep by nIndex. If the DELETE
  ** statement below does actually delete a row, nPhaseOneStep will be
  ** incremented by the same amount when SQL function rbu_tmp_insert()
  ** is invoked by the trigger.  */
  if( eType==RBU_DELETE ){
    p->nPhaseOneStep -= p->objiter.nIndex;
  }

  if( eType==RBU_IDX_DELETE || eType==RBU_DELETE ){
    pWriter = pIter->pDelete;
  }else{
    pWriter = pIter->pInsert;
  }
︙︙
2634
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645



2646
2647
2648
2649
2650
2651
2652
2653
2654

2655
2656
2657
2658
2659
2660
2661

  if( eType ){
    assert( eType==RBU_INSERT     || eType==RBU_DELETE
         || eType==RBU_REPLACE    || eType==RBU_IDX_DELETE
         || eType==RBU_IDX_INSERT || eType==RBU_UPDATE
    );
    assert( eType!=RBU_UPDATE || pIter->zIdx==0 );

    if( pIter->zIdx==0 && eType==RBU_IDX_DELETE ){
      rbuBadControlError(p);
    }
    else if( eType==RBU_REPLACE ){
      if( pIter->zIdx==0 ) rbuStepOneOp(p, RBU_DELETE);



      if( p->rc==SQLITE_OK ) rbuStepOneOp(p, RBU_INSERT);
    }
    else if( eType!=RBU_UPDATE ){
      rbuStepOneOp(p, eType);
    }
    else{
      sqlite3_value *pVal;
      sqlite3_stmt *pUpdate = 0;
      assert( eType==RBU_UPDATE );

      rbuGetUpdateStmt(p, pIter, zMask, &pUpdate);
      if( pUpdate ){
        int i;
        for(i=0; p->rc==SQLITE_OK && i<pIter->nCol; i++){
          char c = zMask[pIter->aiSrcOrder[i]];
          pVal = sqlite3_column_value(pIter->pSelect, i);
          if( pIter->abTblPk[i] || c!='.' ){








|



|
>
>
>









>








2702
2703
2704
2705
2706
2707
2708
2709
2710
2711
2712
2713
2714
2715
2716
2717
2718
2719
2720
2721
2722
2723
2724
2725
2726
2727
2728
2729
2730
2731
2732
2733

  if( eType ){
    assert( eType==RBU_INSERT     || eType==RBU_DELETE
         || eType==RBU_REPLACE    || eType==RBU_IDX_DELETE
         || eType==RBU_IDX_INSERT || eType==RBU_UPDATE
    );
    assert( eType!=RBU_UPDATE || pIter->zIdx==0 );

    if( pIter->zIdx==0 && (eType==RBU_IDX_DELETE || eType==RBU_IDX_INSERT) ){
      rbuBadControlError(p);
    }
    else if( eType==RBU_REPLACE ){
      if( pIter->zIdx==0 ){
        p->nPhaseOneStep += p->objiter.nIndex;
        rbuStepOneOp(p, RBU_DELETE);
      }
      if( p->rc==SQLITE_OK ) rbuStepOneOp(p, RBU_INSERT);
    }
    else if( eType!=RBU_UPDATE ){
      rbuStepOneOp(p, eType);
    }
    else{
      sqlite3_value *pVal;
      sqlite3_stmt *pUpdate = 0;
      assert( eType==RBU_UPDATE );
      p->nPhaseOneStep -= p->objiter.nIndex;
      rbuGetUpdateStmt(p, pIter, zMask, &pUpdate);
      if( pUpdate ){
        int i;
        for(i=0; p->rc==SQLITE_OK && i<pIter->nCol; i++){
          char c = zMask[pIter->aiSrcOrder[i]];
          pVal = sqlite3_column_value(pIter->pSelect, i);
          if( pIter->abTblPk[i] || c!='.' ){
︙︙
2725
2726
2727
2728
2729
2730
2731

2732
2733
2734
2735
2736
2737
2738
2739
2740
2741

2742
2743
2744
2745
2746
2747
2748

          "(%d, %d), "
          "(%d, %Q), "
          "(%d, %Q), "
          "(%d, %d), "
          "(%d, %d), "
          "(%d, %lld), "
          "(%d, %lld), "

          "(%d, %lld) ",
          p->zStateDb,
          RBU_STATE_STAGE, eStage,
          RBU_STATE_TBL, p->objiter.zTbl, 
          RBU_STATE_IDX, p->objiter.zIdx, 
          RBU_STATE_ROW, p->nStep, 
          RBU_STATE_PROGRESS, p->nProgress,
          RBU_STATE_CKPT, p->iWalCksum,
          RBU_STATE_COOKIE, (i64)p->pTargetFd->iCookie,
          RBU_STATE_OALSZ, p->iOalSz

      )
    );
    assert( pInsert==0 || rc==SQLITE_OK );

    if( rc==SQLITE_OK ){
      sqlite3_step(pInsert);
      rc = sqlite3_finalize(pInsert);








>









|
>








2797
2798
2799
2800
2801
2802
2803
2804
2805
2806
2807
2808
2809
2810
2811
2812
2813
2814
2815
2816
2817
2818
2819
2820
2821
2822

          "(%d, %d), "
          "(%d, %Q), "
          "(%d, %Q), "
          "(%d, %d), "
          "(%d, %d), "
          "(%d, %lld), "
          "(%d, %lld), "
          "(%d, %lld), "
          "(%d, %lld) ",
          p->zStateDb,
          RBU_STATE_STAGE, eStage,
          RBU_STATE_TBL, p->objiter.zTbl, 
          RBU_STATE_IDX, p->objiter.zIdx, 
          RBU_STATE_ROW, p->nStep, 
          RBU_STATE_PROGRESS, p->nProgress,
          RBU_STATE_CKPT, p->iWalCksum,
          RBU_STATE_COOKIE, (i64)p->pTargetFd->iCookie,
          RBU_STATE_OALSZ, p->iOalSz,
          RBU_STATE_PHASEONESTEP, p->nPhaseOneStep
      )
    );
    assert( pInsert==0 || rc==SQLITE_OK );

    if( rc==SQLITE_OK ){
      sqlite3_step(pInsert);
      rc = sqlite3_finalize(pInsert);
︙︙
2921
2922
2923
2924
2925
2926
2927




2928
2929
2930
2931
2932
2933
2934

        pRet->iCookie = (u32)sqlite3_column_int64(pStmt, 1);
        break;

      case RBU_STATE_OALSZ:
        pRet->iOalSz = (u32)sqlite3_column_int64(pStmt, 1);
        break;





      default:
        rc = SQLITE_CORRUPT;
        break;
    }
  }
  rc2 = sqlite3_finalize(pStmt);
  if( rc==SQLITE_OK ) rc = rc2;








>
>
>
>








2995
2996
2997
2998
2999
3000
3001
3002
3003
3004
3005
3006
3007
3008
3009
3010
3011
3012

        pRet->iCookie = (u32)sqlite3_column_int64(pStmt, 1);
        break;

      case RBU_STATE_OALSZ:
        pRet->iOalSz = (u32)sqlite3_column_int64(pStmt, 1);
        break;

      case RBU_STATE_PHASEONESTEP:
        pRet->nPhaseOneStep = sqlite3_column_int64(pStmt, 1);
        break;

      default:
        rc = SQLITE_CORRUPT;
        break;
    }
  }
  rc2 = sqlite3_finalize(pStmt);
  if( rc==SQLITE_OK ) rc = rc2;
︙︙
3027
3028
3029
3030
3031
3032
3033






























































































3034
3035
3036
3037
3038
3039
3040

*/
static void rbuDeleteVfs(sqlite3rbu *p){
  if( p->zVfsName ){
    sqlite3rbu_destroy_vfs(p->zVfsName);
    p->zVfsName = 0;
  }
}































































































/*
** Open and return a new RBU handle. 
*/
sqlite3rbu *sqlite3rbu_open(
  const char *zTarget, 
  const char *zRbu,








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








3105
3106
3107
3108
3109
3110
3111
3112
3113
3114
3115
3116
3117
3118
3119
3120
3121
3122
3123
3124
3125
3126
3127
3128
3129
3130
3131
3132
3133
3134
3135
3136
3137
3138
3139
3140
3141
3142
3143
3144
3145
3146
3147
3148
3149
3150
3151
3152
3153
3154
3155
3156
3157
3158
3159
3160
3161
3162
3163
3164
3165
3166
3167
3168
3169
3170
3171
3172
3173
3174
3175
3176
3177
3178
3179
3180
3181
3182
3183
3184
3185
3186
3187
3188
3189
3190
3191
3192
3193
3194
3195
3196
3197
3198
3199
3200
3201
3202
3203
3204
3205
3206
3207
3208
3209
3210
3211
3212

*/
static void rbuDeleteVfs(sqlite3rbu *p){
  if( p->zVfsName ){
    sqlite3rbu_destroy_vfs(p->zVfsName);
    p->zVfsName = 0;
  }
}

/*
** This user-defined SQL function is invoked with a single argument - the
** name of a table expected to appear in the target database. It returns
** the number of auxilliary indexes on the table.
*/
static void rbuIndexCntFunc(
  sqlite3_context *pCtx, 
  int nVal,
  sqlite3_value **apVal
){
  sqlite3rbu *p = (sqlite3rbu*)sqlite3_user_data(pCtx);
  sqlite3_stmt *pStmt = 0;
  char *zErrmsg = 0;
  int rc;

  assert( nVal==1 );
  
  rc = prepareFreeAndCollectError(p->dbMain, &pStmt, &zErrmsg, 
      sqlite3_mprintf("SELECT count(*) FROM sqlite_master "
        "WHERE type='index' AND tbl_name = %Q", sqlite3_value_text(apVal[0]))
  );
  if( rc!=SQLITE_OK ){
    sqlite3_result_error(pCtx, zErrmsg, -1);
  }else{
    int nIndex = 0;
    if( SQLITE_ROW==sqlite3_step(pStmt) ){
      nIndex = sqlite3_column_int(pStmt, 0);
    }
    rc = sqlite3_finalize(pStmt);
    if( rc==SQLITE_OK ){
      sqlite3_result_int(pCtx, nIndex);
    }else{
      sqlite3_result_error(pCtx, sqlite3_errmsg(p->dbMain), -1);
    }
  }

  sqlite3_free(zErrmsg);
}

/*
** If the RBU database contains the rbu_count table, use it to initialize
** the sqlite3rbu.nPhaseOneStep variable. The schema of the rbu_count table
** is assumed to contain the same columns as:
**
**   CREATE TABLE rbu_count(tbl TEXT PRIMARY KEY, cnt INTEGER) WITHOUT ROWID;
**
** There should be one row in the table for each data_xxx table in the
** database. The 'tbl' column should contain the name of a data_xxx table,
** and the cnt column the number of rows it contains.
**
** sqlite3rbu.nPhaseOneStep is initialized to the sum of (1 + nIndex) * cnt
** for all rows in the rbu_count table, where nIndex is the number of 
** indexes on the corresponding target database table.
*/
static void rbuInitPhaseOneSteps(sqlite3rbu *p){
  if( p->rc==SQLITE_OK ){
    sqlite3_stmt *pStmt = 0;
    int bExists = 0;                /* True if rbu_count exists */

    p->nPhaseOneStep = -1;

    p->rc = sqlite3_create_function(p->dbRbu, 
        "rbu_index_cnt", 1, SQLITE_UTF8, (void*)p, rbuIndexCntFunc, 0, 0
    );
  
    /* Check for the rbu_count table. If it does not exist, or if an error
    ** occurs, nPhaseOneStep will be left set to -1. */
    if( p->rc==SQLITE_OK ){
      p->rc = prepareAndCollectError(p->dbRbu, &pStmt, &p->zErrmsg,
          "SELECT 1 FROM sqlite_master WHERE tbl_name = 'rbu_count'"
      );
    }
    if( p->rc==SQLITE_OK ){
      if( SQLITE_ROW==sqlite3_step(pStmt) ){
        bExists = 1;
      }
      p->rc = sqlite3_finalize(pStmt);
    }
  
    if( p->rc==SQLITE_OK && bExists ){
      p->rc = prepareAndCollectError(p->dbRbu, &pStmt, &p->zErrmsg,
          "SELECT sum(cnt * (1 + rbu_index_cnt(rbu_target_name(tbl))))"
          "FROM rbu_count"
      );
      if( p->rc==SQLITE_OK ){
        if( SQLITE_ROW==sqlite3_step(pStmt) ){
          p->nPhaseOneStep = sqlite3_column_int64(pStmt, 0);
        }
        p->rc = sqlite3_finalize(pStmt);
      }
    }
  }
}

/*
** Open and return a new RBU handle. 
*/
sqlite3rbu *sqlite3rbu_open(
  const char *zTarget, 
  const char *zRbu,
︙︙
3073
3074
3075
3076
3077
3078
3079

3080
3081
3082

3083
3084
3085
3086
3087
3088
3089

    if( p->rc==SQLITE_OK ){
      pState = rbuLoadState(p);
      assert( pState || p->rc!=SQLITE_OK );
      if( p->rc==SQLITE_OK ){

        if( pState->eStage==0 ){ 
          rbuDeleteOalFile(p);

          p->eStage = RBU_STAGE_OAL;
        }else{
          p->eStage = pState->eStage;

        }
        p->nProgress = pState->nProgress;
        p->iOalSz = pState->iOalSz;
      }
    }
    assert( p->rc!=SQLITE_OK || p->eStage!=0 );









>



>








3245
3246
3247
3248
3249
3250
3251
3252
3253
3254
3255
3256
3257
3258
3259
3260
3261
3262
3263

    if( p->rc==SQLITE_OK ){
      pState = rbuLoadState(p);
      assert( pState || p->rc!=SQLITE_OK );
      if( p->rc==SQLITE_OK ){

        if( pState->eStage==0 ){ 
          rbuDeleteOalFile(p);
          rbuInitPhaseOneSteps(p);
          p->eStage = RBU_STAGE_OAL;
        }else{
          p->eStage = pState->eStage;
          p->nPhaseOneStep = pState->nPhaseOneStep;
        }
        p->nProgress = pState->nProgress;
        p->iOalSz = pState->iOalSz;
      }
    }
    assert( p->rc!=SQLITE_OK || p->eStage!=0 );
︙︙
3238
3239
3240
3241
3242
3243
3244




































3245
3246
3247
3248
3249
3250
3251

** Return the total number of key-value operations (inserts, deletes or 
** updates) that have been performed on the target database since the
** current RBU update was started.
*/
sqlite3_int64 sqlite3rbu_progress(sqlite3rbu *pRbu){
  return pRbu->nProgress;
}





































int sqlite3rbu_savestate(sqlite3rbu *p){
  int rc = p->rc;
  
  if( rc==SQLITE_DONE ) return SQLITE_OK;

  assert( p->eStage>=RBU_STAGE_OAL && p->eStage<=RBU_STAGE_DONE );








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








3412
3413
3414
3415
3416
3417
3418
3419
3420
3421
3422
3423
3424
3425
3426
3427
3428
3429
3430
3431
3432
3433
3434
3435
3436
3437
3438
3439
3440
3441
3442
3443
3444
3445
3446
3447
3448
3449
3450
3451
3452
3453
3454
3455
3456
3457
3458
3459
3460
3461

** Return the total number of key-value operations (inserts, deletes or 
** updates) that have been performed on the target database since the
** current RBU update was started.
*/
sqlite3_int64 sqlite3rbu_progress(sqlite3rbu *pRbu){
  return pRbu->nProgress;
}

/*
** Return permyriadage progress indications for the two main stages of
** an RBU update.
*/
void sqlite3rbu_bp_progress(sqlite3rbu *p, int *pnOne, int *pnTwo){
  const int MAX_PROGRESS = 10000;
  switch( p->eStage ){
    case RBU_STAGE_OAL:
      if( p->nPhaseOneStep>0 ){
        *pnOne = (int)(MAX_PROGRESS * (i64)p->nProgress/(i64)p->nPhaseOneStep);
      }else{
        *pnOne = -1;
      }
      *pnTwo = 0;
      break;

    case RBU_STAGE_MOVE:
      *pnOne = MAX_PROGRESS;
      *pnTwo = 0;
      break;

    case RBU_STAGE_CKPT:
      *pnOne = MAX_PROGRESS;
      *pnTwo = (int)(MAX_PROGRESS * (i64)p->nStep / (i64)p->nFrame);
      break;

    case RBU_STAGE_DONE:
      *pnOne = MAX_PROGRESS;
      *pnTwo = MAX_PROGRESS;
      break;

    default:
      assert( 0 );
  }
}

int sqlite3rbu_savestate(sqlite3rbu *p){
  int rc = p->rc;
  
  if( rc==SQLITE_DONE ) return SQLITE_OK;

  assert( p->eStage>=RBU_STAGE_OAL && p->eStage<=RBU_STAGE_DONE );
︙︙
Changes to ext/rbu/sqlite3rbu.h.
︙︙
396
397
398
399
400
401
402










































403
404
405
406
407
408
409

/*
** Return the total number of key-value operations (inserts, deletes or 
** updates) that have been performed on the target database since the
** current RBU update was started.
*/
sqlite3_int64 sqlite3rbu_progress(sqlite3rbu *pRbu);











































/*
** Create an RBU VFS named zName that accesses the underlying file-system
** via existing VFS zParent. Or, if the zParent parameter is passed NULL, 
** then the new RBU VFS uses the default system VFS to access the file-system.
** The new object is registered as a non-default VFS with SQLite before 
** returning.
**








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429
430
431
432
433
434
435
436
437
438
439
440
441
442
443
444
445
446
447
448
449
450
451

/*
** Return the total number of key-value operations (inserts, deletes or 
** updates) that have been performed on the target database since the
** current RBU update was started.
*/
sqlite3_int64 sqlite3rbu_progress(sqlite3rbu *pRbu);

/*
** Obtain permyriadage (permyriadage is to 10000 as percentage is to 100) 
** progress indications for the two stages of an RBU update. This API may
** be useful for driving GUI progress indicators and similar.
**
** An RBU update is divided into two stages:
**
**   * Stage 1, in which changes are accumulated in an oal/wal file, and
**   * Stage 2, in which the contents of the wal file are copied into the
**     main database.
**
** The update is visible to non-RBU clients during stage 2. During stage 1
** non-RBU reader clients may see the original database.
**
** If this API is called during stage 2 of the update, output variable 
** (*pnOne) is set to 10000 to indicate that stage 1 has finished and (*pnTwo)
** to a value between 0 and 10000 to indicate the permyriadage progress of
** stage 2. A value of 5000 indicates that stage 2 is half finished, 
** 9000 indicates that it is 90% finished, and so on.
**
** If this API is called during stage 1 of the update, output variable 
** (*pnTwo) is set to 0 to indicate that stage 2 has not yet started. The
** value to which (*pnOne) is set depends on whether or not the RBU 
** database contains an "rbu_count" table. The rbu_count table, if it 
** exists, must contain the same columns as the following:
**
**   CREATE TABLE rbu_count(tbl TEXT PRIMARY KEY, cnt INTEGER) WITHOUT ROWID;
**
** There must be one row in the table for each source (data_xxx) table within
** the RBU database. The 'tbl' column should contain the name of the source
** table. The 'cnt' column should contain the number of rows within the
** source table.
**
** If the rbu_count table is present and populated correctly and this
** API is called during stage 1, the *pnOne output variable is set to the
** permyriadage progress of the same stage. If the rbu_count table does
** not exist, then (*pnOne) is set to -1 during stage 1. If the rbu_count
** table exists but is not correctly populated, the value of the *pnOne
** output variable during stage 1 is undefined.
*/
void sqlite3rbu_bp_progress(sqlite3rbu *pRbu, int *pnOne, int *pnTwo);

/*
** Create an RBU VFS named zName that accesses the underlying file-system
** via existing VFS zParent. Or, if the zParent parameter is passed NULL, 
** then the new RBU VFS uses the default system VFS to access the file-system.
** The new object is registered as a non-default VFS with SQLite before 
** returning.
**
︙︙
Changes to ext/rbu/test_rbu.c.
︙︙
62
63
64
65
66
67
68

69
70
71
72
73
74
75

    const char *zUsage;
  } aCmd[] = {
    {"step", 2, ""},              /* 0 */
    {"close", 2, ""},             /* 1 */
    {"create_rbu_delta", 2, ""},  /* 2 */
    {"savestate", 2, ""},         /* 3 */
    {"dbMain_eval", 3, "SQL"},    /* 4 */

    {0,0,0}
  };
  int iCmd;

  if( objc<2 ){
    Tcl_WrongNumArgs(interp, 1, objv, "METHOD");
    return TCL_ERROR;








>








62
63
64
65
66
67
68
69
70
71
72
73
74
75
76

    const char *zUsage;
  } aCmd[] = {
    {"step", 2, ""},              /* 0 */
    {"close", 2, ""},             /* 1 */
    {"create_rbu_delta", 2, ""},  /* 2 */
    {"savestate", 2, ""},         /* 3 */
    {"dbMain_eval", 3, "SQL"},    /* 4 */
    {"bp_progress", 2, ""},    /* 5 */
    {0,0,0}
  };
  int iCmd;

  if( objc<2 ){
    Tcl_WrongNumArgs(interp, 1, objv, "METHOD");
    return TCL_ERROR;
︙︙
131
132
133
134
135
136
137












138
139
140
141
142
143
144

      int rc = sqlite3_exec(db, Tcl_GetString(objv[2]), 0, 0, 0);
      if( rc!=SQLITE_OK ){
        Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3_errmsg(db), -1));
        ret = TCL_ERROR;
      }
      break;
    }













    default: /* seems unlikely */
      assert( !"cannot happen" );
      break;
  }

  return ret;








>
>
>
>
>
>
>
>
>
>
>
>








132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157

      int rc = sqlite3_exec(db, Tcl_GetString(objv[2]), 0, 0, 0);
      if( rc!=SQLITE_OK ){
        Tcl_SetObjResult(interp, Tcl_NewStringObj(sqlite3_errmsg(db), -1));
        ret = TCL_ERROR;
      }
      break;
    }

    case 5: /* bp_progress */ {
      int one, two;
      Tcl_Obj *pObj;
      sqlite3rbu_bp_progress(pRbu, &one, &two);

      pObj = Tcl_NewObj();
      Tcl_ListObjAppendElement(interp, pObj, Tcl_NewIntObj(one));
      Tcl_ListObjAppendElement(interp, pObj, Tcl_NewIntObj(two));
      Tcl_SetObjResult(interp, pObj);
      break;
    }

    default: /* seems unlikely */
      assert( !"cannot happen" );
      break;
  }

  return ret;
︙︙
Changes to src/loadext.c.
︙︙
410
411
412
413
414
415
416
417


418
419
420
421
422
423
424

  sqlite3_bind_zeroblob64,
  /* Version 3.9.0 and later */
  sqlite3_value_subtype,
  sqlite3_result_subtype,
  /* Version 3.10.0 and later */
  sqlite3_status64,
  sqlite3_strlike,
  sqlite3_db_cacheflush


};

/*
** Attempt to load an SQLite extension library contained in the file
** zFile.  The entry point is zProc.  zProc may be 0 in which case a
** default entry point name (sqlite3_extension_init) is used.  Use
** of the default name is recommended.








|
>
>








410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426

  sqlite3_bind_zeroblob64,
  /* Version 3.9.0 and later */
  sqlite3_value_subtype,
  sqlite3_result_subtype,
  /* Version 3.10.0 and later */
  sqlite3_status64,
  sqlite3_strlike,
  sqlite3_db_cacheflush,
  /* Version 3.12.0 and later */
  sqlite3_system_errno
};

/*
** Attempt to load an SQLite extension library contained in the file
** zFile.  The entry point is zProc.  zProc may be 0 in which case a
** default entry point name (sqlite3_extension_init) is used.  Use
** of the default name is recommended.
︙︙
Changes to src/main.c.
︙︙
2243
2244
2245
2246
2247
2248
2249



2250
2251
2252
2253
2254
2255
2256

    return SQLITE_MISUSE_BKPT;
  }
  if( !db || db->mallocFailed ){
    return SQLITE_NOMEM_BKPT;
  }
  return db->errCode;
}




/*
** Return a string that describes the kind of error specified in the
** argument.  For now, this simply calls the internal sqlite3ErrStr()
** function.
*/
const char *sqlite3_errstr(int rc){








>
>
>








2243
2244
2245
2246
2247
2248
2249
2250
2251
2252
2253
2254
2255
2256
2257
2258
2259

    return SQLITE_MISUSE_BKPT;
  }
  if( !db || db->mallocFailed ){
    return SQLITE_NOMEM_BKPT;
  }
  return db->errCode;
}
int sqlite3_system_errno(sqlite3 *db){
  return db ? db->iSysErrno : 0;
}  

/*
** Return a string that describes the kind of error specified in the
** argument.  For now, this simply calls the internal sqlite3ErrStr()
** function.
*/
const char *sqlite3_errstr(int rc){
︙︙
Changes to src/os.c.
︙︙
264
265
266
267
268
269
270



271
272
273
274
275
276
277

#endif /* SQLITE_OMIT_LOAD_EXTENSION */
int sqlite3OsRandomness(sqlite3_vfs *pVfs, int nByte, char *zBufOut){
  return pVfs->xRandomness(pVfs, nByte, zBufOut);
}
int sqlite3OsSleep(sqlite3_vfs *pVfs, int nMicro){
  return pVfs->xSleep(pVfs, nMicro);
}



int sqlite3OsCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *pTimeOut){
  int rc;
  /* IMPLEMENTATION-OF: R-49045-42493 SQLite will use the xCurrentTimeInt64()
  ** method to get the current date and time if that method is available
  ** (if iVersion is 2 or greater and the function pointer is not NULL) and
  ** will fall back to xCurrentTime() if xCurrentTimeInt64() is
  ** unavailable.








>
>
>








264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280

#endif /* SQLITE_OMIT_LOAD_EXTENSION */
int sqlite3OsRandomness(sqlite3_vfs *pVfs, int nByte, char *zBufOut){
  return pVfs->xRandomness(pVfs, nByte, zBufOut);
}
int sqlite3OsSleep(sqlite3_vfs *pVfs, int nMicro){
  return pVfs->xSleep(pVfs, nMicro);
}
int sqlite3OsGetLastError(sqlite3_vfs *pVfs){
  return pVfs->xGetLastError ? pVfs->xGetLastError(pVfs, 0, 0) : 0;
}
int sqlite3OsCurrentTimeInt64(sqlite3_vfs *pVfs, sqlite3_int64 *pTimeOut){
  int rc;
  /* IMPLEMENTATION-OF: R-49045-42493 SQLite will use the xCurrentTimeInt64()
  ** method to get the current date and time if that method is available
  ** (if iVersion is 2 or greater and the function pointer is not NULL) and
  ** will fall back to xCurrentTime() if xCurrentTimeInt64() is
  ** unavailable.
︙︙
Changes to src/os.h.
︙︙
193
194
195
196
197
198
199

200
201
202
203
204
205
206

void *sqlite3OsDlOpen(sqlite3_vfs *, const char *);
void sqlite3OsDlError(sqlite3_vfs *, int, char *);
void (*sqlite3OsDlSym(sqlite3_vfs *, void *, const char *))(void);
void sqlite3OsDlClose(sqlite3_vfs *, void *);
#endif /* SQLITE_OMIT_LOAD_EXTENSION */
int sqlite3OsRandomness(sqlite3_vfs *, int, char *);
int sqlite3OsSleep(sqlite3_vfs *, int);

int sqlite3OsCurrentTimeInt64(sqlite3_vfs *, sqlite3_int64*);

/*
** Convenience functions for opening and closing files using 
** sqlite3_malloc() to obtain space for the file-handle structure.
*/
int sqlite3OsOpenMalloc(sqlite3_vfs *, const char *, sqlite3_file **, int,int*);








>








193
194
195
196
197
198
199
200
201
202
203
204
205
206
207

void *sqlite3OsDlOpen(sqlite3_vfs *, const char *);
void sqlite3OsDlError(sqlite3_vfs *, int, char *);
void (*sqlite3OsDlSym(sqlite3_vfs *, void *, const char *))(void);
void sqlite3OsDlClose(sqlite3_vfs *, void *);
#endif /* SQLITE_OMIT_LOAD_EXTENSION */
int sqlite3OsRandomness(sqlite3_vfs *, int, char *);
int sqlite3OsSleep(sqlite3_vfs *, int);
int sqlite3OsGetLastError(sqlite3_vfs*);
int sqlite3OsCurrentTimeInt64(sqlite3_vfs *, sqlite3_int64*);

/*
** Convenience functions for opening and closing files using 
** sqlite3_malloc() to obtain space for the file-handle structure.
*/
int sqlite3OsOpenMalloc(sqlite3_vfs *, const char *, sqlite3_file **, int,int*);
︙︙
Changes to src/os_unix.c.
︙︙
7467
7468
7469
7470
7471
7472
7473
7474
7475
7476
7477
7478
7479
7480
7481
7482
7483
7484
7485
7486
7487
7488
7489
7490
7491
7492
7493
7494
7495
7496
7497

  *prNow = i/86400000.0;
  return rc;
}
#else
# define unixCurrentTime 0
#endif

#ifndef SQLITE_OMIT_DEPRECATED
/*
** We added the xGetLastError() method with the intention of providing
** better low-level error messages when operating-system problems come up
** during SQLite operation.  But so far, none of that has been implemented
** in the core.  So this routine is never called.  For now, it is merely
** a place-holder.
*/
static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){
  UNUSED_PARAMETER(NotUsed);
  UNUSED_PARAMETER(NotUsed2);
  UNUSED_PARAMETER(NotUsed3);
  return 0;
}
#else
# define unixGetLastError 0
#endif


/*
************************ End of sqlite3_vfs methods ***************************
******************************************************************************/

/******************************************************************************








<

|
|
|
<
|





|

<
<
<








7467
7468
7469
7470
7471
7472
7473

7474
7475
7476
7477

7478
7479
7480
7481
7482
7483
7484
7485



7486
7487
7488
7489
7490
7491
7492

  *prNow = i/86400000.0;
  return rc;
}
#else
# define unixCurrentTime 0
#endif


/*
** The xGetLastError() method is designed to return a better
** low-level error message when operating-system problems come up
** during SQLite operation.  Only the integer return code is currently

** used.
*/
static int unixGetLastError(sqlite3_vfs *NotUsed, int NotUsed2, char *NotUsed3){
  UNUSED_PARAMETER(NotUsed);
  UNUSED_PARAMETER(NotUsed2);
  UNUSED_PARAMETER(NotUsed3);
  return errno;
}





/*
************************ End of sqlite3_vfs methods ***************************
******************************************************************************/

/******************************************************************************
︙︙
Changes to src/os_win.c.
︙︙
5589
5590
5591
5592
5593
5594
5595

5596
5597

5598
5599
5600
5601
5602
5603
5604

**   }
**
** However if an error message is supplied, it will be incorporated
** by sqlite into the error message available to the user using
** sqlite3_errmsg(), possibly making IO errors easier to debug.
*/
static int winGetLastError(sqlite3_vfs *pVfs, int nBuf, char *zBuf){

  UNUSED_PARAMETER(pVfs);
  return winGetLastErrorMsg(osGetLastError(), nBuf, zBuf);

}

/*
** Initialize and deinitialize the operating system interface.
*/
int sqlite3_os_init(void){
  static sqlite3_vfs winVfs = {








>

|
>








5589
5590
5591
5592
5593
5594
5595
5596
5597
5598
5599
5600
5601
5602
5603
5604
5605
5606

**   }
**
** However if an error message is supplied, it will be incorporated
** by sqlite into the error message available to the user using
** sqlite3_errmsg(), possibly making IO errors easier to debug.
*/
static int winGetLastError(sqlite3_vfs *pVfs, int nBuf, char *zBuf){
  DWORD e = osGetLastError();
  UNUSED_PARAMETER(pVfs);
  if( nBuf>0 ) winGetLastErrorMsg(e, nBuf, zBuf);
  return e;
}

/*
** Initialize and deinitialize the operating system interface.
*/
int sqlite3_os_init(void){
  static sqlite3_vfs winVfs = {
︙︙
Changes to src/sqlite.h.in.
︙︙
7912
7913
7914
7915
7916
7917
7918












7919
7920
7921
7922
7923
7924
7925

** ^Otherwise, if no error occurs, [sqlite3_db_cacheflush()] returns SQLITE_OK.
**
** ^This function does not set the database handle error code or message
** returned by the [sqlite3_errcode()] and [sqlite3_errmsg()] functions.
*/
int sqlite3_db_cacheflush(sqlite3*);













/*
** CAPI3REF: Database Snapshot
** KEYWORDS: {snapshot}
** EXPERIMENTAL
**
** An instance of the snapshot object records the state of a [WAL mode]
** database for some specific point in history.








>
>
>
>
>
>
>
>
>
>
>
>








7912
7913
7914
7915
7916
7917
7918
7919
7920
7921
7922
7923
7924
7925
7926
7927
7928
7929
7930
7931
7932
7933
7934
7935
7936
7937

** ^Otherwise, if no error occurs, [sqlite3_db_cacheflush()] returns SQLITE_OK.
**
** ^This function does not set the database handle error code or message
** returned by the [sqlite3_errcode()] and [sqlite3_errmsg()] functions.
*/
int sqlite3_db_cacheflush(sqlite3*);

/*
** CAPI3REF: Low-level system error code
**
** ^Attempt to return the underlying operating system error code or error
** number that caused the most reason I/O error or failure to open a file.
** The return value is OS-dependent.  For example, on unix systems, after
** [sqlite3_open_v2()] returns [SQLITE_CANTOPEN], this interface could be
** called to get back the underlying "errno" that caused the problem, such
** as ENOSPC, EAUTH, EISDIR, and so forth.  
*/
int sqlite3_system_errno(sqlite3*);

/*
** CAPI3REF: Database Snapshot
** KEYWORDS: {snapshot}
** EXPERIMENTAL
**
** An instance of the snapshot object records the state of a [WAL mode]
** database for some specific point in history.
︙︙
Changes to src/sqlite3ext.h.
︙︙
275
276
277
278
279
280
281


282
283
284
285
286
287
288

  /* Version 3.9.0 and later */
  unsigned int (*value_subtype)(sqlite3_value*);
  void (*result_subtype)(sqlite3_context*,unsigned int);
  /* Version 3.10.0 and later */
  int (*status64)(int,sqlite3_int64*,sqlite3_int64*,int);
  int (*strlike)(const char*,const char*,unsigned int);
  int (*db_cacheflush)(sqlite3*);


};

/*
** The following macros redefine the API routines so that they are
** redirected through the global sqlite3_api structure.
**
** This header file is also used by the loadext.c source file








>
>








275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290

  /* Version 3.9.0 and later */
  unsigned int (*value_subtype)(sqlite3_value*);
  void (*result_subtype)(sqlite3_context*,unsigned int);
  /* Version 3.10.0 and later */
  int (*status64)(int,sqlite3_int64*,sqlite3_int64*,int);
  int (*strlike)(const char*,const char*,unsigned int);
  int (*db_cacheflush)(sqlite3*);
  /* Version 3.12.0 and later */
  int (*system_errno)(sqlite3*);
};

/*
** The following macros redefine the API routines so that they are
** redirected through the global sqlite3_api structure.
**
** This header file is also used by the loadext.c source file
︙︙
518
519
520
521
522
523
524


525
526
527
528
529
530
531

/* Version 3.9.0 and later */
#define sqlite3_value_subtype          sqlite3_api->value_subtype
#define sqlite3_result_subtype         sqlite3_api->result_subtype
/* Version 3.10.0 and later */
#define sqlite3_status64               sqlite3_api->status64
#define sqlite3_strlike                sqlite3_api->strlike
#define sqlite3_db_cacheflush          sqlite3_api->db_cacheflush


#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */

#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
  /* This case when the file really is being compiled as a loadable 
  ** extension */
# define SQLITE_EXTENSION_INIT1     const sqlite3_api_routines *sqlite3_api=0;
# define SQLITE_EXTENSION_INIT2(v)  sqlite3_api=v;








>
>








520
521
522
523
524
525
526
527
528
529
530
531
532
533
534
535

/* Version 3.9.0 and later */
#define sqlite3_value_subtype          sqlite3_api->value_subtype
#define sqlite3_result_subtype         sqlite3_api->result_subtype
/* Version 3.10.0 and later */
#define sqlite3_status64               sqlite3_api->status64
#define sqlite3_strlike                sqlite3_api->strlike
#define sqlite3_db_cacheflush          sqlite3_api->db_cacheflush
/* Version 3.12.0 and later */
#define sqlite3_system_errno           sqlite3_api->system_errno
#endif /* !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION) */

#if !defined(SQLITE_CORE) && !defined(SQLITE_OMIT_LOAD_EXTENSION)
  /* This case when the file really is being compiled as a loadable 
  ** extension */
# define SQLITE_EXTENSION_INIT1     const sqlite3_api_routines *sqlite3_api=0;
# define SQLITE_EXTENSION_INIT2(v)  sqlite3_api=v;
︙︙
Changes to src/sqliteInt.h.
︙︙
1216
1217
1218
1219
1220
1221
1222

1223
1224
1225
1226
1227
1228
1229

  int nDb;                      /* Number of backends currently in use */
  int flags;                    /* Miscellaneous flags. See below */
  i64 lastRowid;                /* ROWID of most recent insert (see above) */
  i64 szMmap;                   /* Default mmap_size setting */
  unsigned int openFlags;       /* Flags passed to sqlite3_vfs.xOpen() */
  int errCode;                  /* Most recent error code (SQLITE_*) */
  int errMask;                  /* & result codes with this before returning */

  u16 dbOptFlags;               /* Flags to enable/disable optimizations */
  u8 enc;                       /* Text encoding */
  u8 autoCommit;                /* The auto-commit flag. */
  u8 temp_store;                /* 1: file 2: memory 0: default */
  u8 mallocFailed;              /* True if we have seen a malloc failure */
  u8 bBenignMalloc;             /* Do not require OOMs if true */
  u8 dfltLockMode;              /* Default locking-mode for attached dbs */








>








1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230

  int nDb;                      /* Number of backends currently in use */
  int flags;                    /* Miscellaneous flags. See below */
  i64 lastRowid;                /* ROWID of most recent insert (see above) */
  i64 szMmap;                   /* Default mmap_size setting */
  unsigned int openFlags;       /* Flags passed to sqlite3_vfs.xOpen() */
  int errCode;                  /* Most recent error code (SQLITE_*) */
  int errMask;                  /* & result codes with this before returning */
  int iSysErrno;                /* Errno value from last system error */
  u16 dbOptFlags;               /* Flags to enable/disable optimizations */
  u8 enc;                       /* Text encoding */
  u8 autoCommit;                /* The auto-commit flag. */
  u8 temp_store;                /* 1: file 2: memory 0: default */
  u8 mallocFailed;              /* True if we have seen a malloc failure */
  u8 bBenignMalloc;             /* Do not require OOMs if true */
  u8 dfltLockMode;              /* Default locking-mode for attached dbs */
︙︙
3762
3763
3764
3765
3766
3767
3768

3769
3770
3771
3772
3773
3774
3775

char sqlite3CompareAffinity(Expr *pExpr, char aff2);
int sqlite3IndexAffinityOk(Expr *pExpr, char idx_affinity);
char sqlite3ExprAffinity(Expr *pExpr);
int sqlite3Atoi64(const char*, i64*, int, u8);
int sqlite3DecOrHexToI64(const char*, i64*);
void sqlite3ErrorWithMsg(sqlite3*, int, const char*,...);
void sqlite3Error(sqlite3*,int);

void *sqlite3HexToBlob(sqlite3*, const char *z, int n);
u8 sqlite3HexToInt(int h);
int sqlite3TwoPartName(Parse *, Token *, Token *, Token **);

#if defined(SQLITE_NEED_ERR_NAME)
const char *sqlite3ErrName(int);
#endif








>








3763
3764
3765
3766
3767
3768
3769
3770
3771
3772
3773
3774
3775
3776
3777

char sqlite3CompareAffinity(Expr *pExpr, char aff2);
int sqlite3IndexAffinityOk(Expr *pExpr, char idx_affinity);
char sqlite3ExprAffinity(Expr *pExpr);
int sqlite3Atoi64(const char*, i64*, int, u8);
int sqlite3DecOrHexToI64(const char*, i64*);
void sqlite3ErrorWithMsg(sqlite3*, int, const char*,...);
void sqlite3Error(sqlite3*,int);
void sqlite3SystemError(sqlite3*,int);
void *sqlite3HexToBlob(sqlite3*, const char *z, int n);
u8 sqlite3HexToInt(int h);
int sqlite3TwoPartName(Parse *, Token *, Token *, Token **);

#if defined(SQLITE_NEED_ERR_NAME)
const char *sqlite3ErrName(int);
#endif
︙︙
Changes to src/test1.c.
︙︙
4862
4863
4864
4865
4866
4867
4868























4869
4870
4871
4872
4873
4874
4875

    Tcl_SetResult(interp, (char *)sqlite3ErrStr(rc), TCL_STATIC);
    return TCL_ERROR;
  }

  Tcl_ResetResult(interp);
  return TCL_OK;
}
























/*
** Usage:  sqlite3_db_filename DB DBNAME
**
** Return the name of a file associated with a database.
*/
static int test_db_filename(








>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>








4862
4863
4864
4865
4866
4867
4868
4869
4870
4871
4872
4873
4874
4875
4876
4877
4878
4879
4880
4881
4882
4883
4884
4885
4886
4887
4888
4889
4890
4891
4892
4893
4894
4895
4896
4897
4898

    Tcl_SetResult(interp, (char *)sqlite3ErrStr(rc), TCL_STATIC);
    return TCL_ERROR;
  }

  Tcl_ResetResult(interp);
  return TCL_OK;
}

/*
** Usage:  sqlite3_system_errno DB
**
** Return the low-level system errno value.
*/
static int test_system_errno(
  void * clientData,
  Tcl_Interp *interp,
  int objc,
  Tcl_Obj *CONST objv[]
){
  sqlite3 *db;
  int iErrno;
  if( objc!=2 ){
    Tcl_WrongNumArgs(interp, 1, objv, "DB");
    return TCL_ERROR;
  }
  if( getDbPointer(interp, Tcl_GetString(objv[1]), &db) ) return TCL_ERROR;
  iErrno = sqlite3_system_errno(db);
  Tcl_SetObjResult(interp, Tcl_NewIntObj(iErrno));
  return TCL_OK;
}

/*
** Usage:  sqlite3_db_filename DB DBNAME
**
** Return the name of a file associated with a database.
*/
static int test_db_filename(
︙︙
7274
7275
7276
7277
7278
7279
7280

7281
7282
7283
7284
7285
7286
7287

     { "sqlite3_stmt_readonly",         test_stmt_readonly ,0 },
     { "sqlite3_stmt_busy",             test_stmt_busy     ,0 },
     { "uses_stmt_journal",             uses_stmt_journal ,0 },

     { "sqlite3_release_memory",        test_release_memory,     0},
     { "sqlite3_db_release_memory",     test_db_release_memory,  0},
     { "sqlite3_db_cacheflush",         test_db_cacheflush,      0},

     { "sqlite3_db_filename",           test_db_filename,        0},
     { "sqlite3_db_readonly",           test_db_readonly,        0},
     { "sqlite3_soft_heap_limit",       test_soft_heap_limit,    0},
     { "sqlite3_thread_cleanup",        test_thread_cleanup,     0},
     { "sqlite3_pager_refcounts",       test_pager_refcounts,    0},

     { "sqlite3_load_extension",        test_load_extension,     0},








>








7297
7298
7299
7300
7301
7302
7303
7304
7305
7306
7307
7308
7309
7310
7311

     { "sqlite3_stmt_readonly",         test_stmt_readonly ,0 },
     { "sqlite3_stmt_busy",             test_stmt_busy     ,0 },
     { "uses_stmt_journal",             uses_stmt_journal ,0 },

     { "sqlite3_release_memory",        test_release_memory,     0},
     { "sqlite3_db_release_memory",     test_db_release_memory,  0},
     { "sqlite3_db_cacheflush",         test_db_cacheflush,      0},
     { "sqlite3_system_errno",          test_system_errno,       0},
     { "sqlite3_db_filename",           test_db_filename,        0},
     { "sqlite3_db_readonly",           test_db_readonly,        0},
     { "sqlite3_soft_heap_limit",       test_soft_heap_limit,    0},
     { "sqlite3_thread_cleanup",        test_thread_cleanup,     0},
     { "sqlite3_pager_refcounts",       test_pager_refcounts,    0},

     { "sqlite3_load_extension",        test_load_extension,     0},
︙︙
Changes to src/util.c.
︙︙
112
113
114
115
116
117
118










119
120
121


122
123
124
125
126












127
128
129
130
131
132
133

/*
** The string z[] is followed immediately by another string.  Return
** a poiner to that other string.
*/
const char *sqlite3StrNext(const char *z){
  return z + strlen(z) + 1;
}











/*
** Set the current error code to err_code and clear any prior error message.


*/
void sqlite3Error(sqlite3 *db, int err_code){
  assert( db!=0 );
  db->errCode = err_code;
  if( db->pErr ) sqlite3ValueSetNull(db->pErr);












}

/*
** Set the most recent error code and error string for the sqlite
** handle "db". The error code is set to "err_code".
**
** If it is not NULL, string zFormat specifies the format of the








>
>
>
>
>
>
>
>
>
>



>
>




|
>
>
>
>
>
>
>
>
>
>
>
>








112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157

/*
** The string z[] is followed immediately by another string.  Return
** a poiner to that other string.
*/
const char *sqlite3StrNext(const char *z){
  return z + strlen(z) + 1;
}

/*
** Helper function for sqlite3Error() - called rarely.  Broken out into
** a separate routine to avoid unnecessary register saves on entry to
** sqlite3Error().
*/
static SQLITE_NOINLINE void  sqlite3ErrorFinish(sqlite3 *db, int err_code){
  if( db->pErr ) sqlite3ValueSetNull(db->pErr);
  sqlite3SystemError(db, err_code);
}

/*
** Set the current error code to err_code and clear any prior error message.
** Also set iSysErrno (by calling sqlite3System) if the err_code indicates
** that would be appropriate.
*/
void sqlite3Error(sqlite3 *db, int err_code){
  assert( db!=0 );
  db->errCode = err_code;
  if( err_code || db->pErr ) sqlite3ErrorFinish(db, err_code);
}

/*
** Load the sqlite3.iSysErrno field if that is an appropriate thing
** to do based on the SQLite error code in rc.
*/
void sqlite3SystemError(sqlite3 *db, int rc){
  if( rc==SQLITE_IOERR_NOMEM ) return;
  rc &= 0xff;
  if( rc==SQLITE_CANTOPEN || rc==SQLITE_IOERR ){
    db->iSysErrno = sqlite3OsGetLastError(db->pVfs);
  }
}

/*
** Set the most recent error code and error string for the sqlite
** handle "db". The error code is set to "err_code".
**
** If it is not NULL, string zFormat specifies the format of the
︙︙
146
147
148
149
150
151
152

153
154
155
156
157
158
159

** To clear the most recent error for sqlite handle "db", sqlite3Error
** should be called with err_code set to SQLITE_OK and zFormat set
** to NULL.
*/
void sqlite3ErrorWithMsg(sqlite3 *db, int err_code, const char *zFormat, ...){
  assert( db!=0 );
  db->errCode = err_code;

  if( zFormat==0 ){
    sqlite3Error(db, err_code);
  }else if( db->pErr || (db->pErr = sqlite3ValueNew(db))!=0 ){
    char *z;
    va_list ap;
    va_start(ap, zFormat);
    z = sqlite3VMPrintf(db, zFormat, ap);








>








170
171
172
173
174
175
176
177
178
179
180
181
182
183
184

** To clear the most recent error for sqlite handle "db", sqlite3Error
** should be called with err_code set to SQLITE_OK and zFormat set
** to NULL.
*/
void sqlite3ErrorWithMsg(sqlite3 *db, int err_code, const char *zFormat, ...){
  assert( db!=0 );
  db->errCode = err_code;
  sqlite3SystemError(db, err_code);
  if( zFormat==0 ){
    sqlite3Error(db, err_code);
  }else if( db->pErr || (db->pErr = sqlite3ValueNew(db))!=0 ){
    char *z;
    va_list ap;
    va_start(ap, zFormat);
    z = sqlite3VMPrintf(db, zFormat, ap);
︙︙
Changes to src/vdbe.c.
︙︙
188
189
190
191
192
193
194

195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214

  **     different sized allocations. Memory cells provide growable
  **     allocations.
  **
  **   * When using ENABLE_MEMORY_MANAGEMENT, memory cell buffers can
  **     be freed lazily via the sqlite3_release_memory() API. This
  **     minimizes the number of malloc calls made by the system.
  **

  ** Memory cells for cursors are allocated at the top of the address
  ** space. Memory cell (p->nMem) corresponds to cursor 0. Space for
  ** cursor 1 is managed by memory cell (p->nMem-1), etc.
  */
  Mem *pMem = &p->aMem[p->nMem-iCur];

  int nByte;
  VdbeCursor *pCx = 0;
  nByte = 
      ROUND8(sizeof(VdbeCursor)) + 2*sizeof(u32)*nField + 
      (eCurType==CURTYPE_BTREE?sqlite3BtreeCursorSize():0);

  assert( iCur<p->nCursor );
  if( p->apCsr[iCur] ){
    sqlite3VdbeFreeCursor(p, p->apCsr[iCur]);
    p->apCsr[iCur] = 0;
  }
  if( SQLITE_OK==sqlite3VdbeMemClearAndResize(pMem, nByte) ){
    p->apCsr[iCur] = pCx = (VdbeCursor*)pMem->z;
    memset(pCx, 0, sizeof(VdbeCursor));








>
|
<
|

|







|








188
189
190
191
192
193
194
195
196

197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214

  **     different sized allocations. Memory cells provide growable
  **     allocations.
  **
  **   * When using ENABLE_MEMORY_MANAGEMENT, memory cell buffers can
  **     be freed lazily via the sqlite3_release_memory() API. This
  **     minimizes the number of malloc calls made by the system.
  **
  ** The memory cell for cursor 0 is aMem[0]. The rest are allocated from
  ** the top of the register space.  Cursor 1 is at Mem[p->nMem-1].

  ** Cursor 2 is at Mem[p->nMem-2]. And so forth.
  */
  Mem *pMem = iCur>0 ? &p->aMem[p->nMem-iCur] : p->aMem;

  int nByte;
  VdbeCursor *pCx = 0;
  nByte = 
      ROUND8(sizeof(VdbeCursor)) + 2*sizeof(u32)*nField + 
      (eCurType==CURTYPE_BTREE?sqlite3BtreeCursorSize():0);

  assert( iCur>=0 && iCur<p->nCursor );
  if( p->apCsr[iCur] ){
    sqlite3VdbeFreeCursor(p, p->apCsr[iCur]);
    p->apCsr[iCur] = 0;
  }
  if( SQLITE_OK==sqlite3VdbeMemClearAndResize(pMem, nByte) ){
    p->apCsr[iCur] = pCx = (VdbeCursor*)pMem->z;
    memset(pCx, 0, sizeof(VdbeCursor));
︙︙
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539

  sqlite3VdbeMemSetNull(pOut);
  pOut->flags = MEM_Int;
  return pOut;
}
static Mem *out2Prerelease(Vdbe *p, VdbeOp *pOp){
  Mem *pOut;
  assert( pOp->p2>0 );
  assert( pOp->p2<=(p->nMem-p->nCursor) );
  pOut = &p->aMem[pOp->p2];
  memAboutToChange(p, pOut);
  if( VdbeMemDynamic(pOut) ){
    return out2PrereleaseWithClear(pOut);
  }else{
    pOut->flags = MEM_Int;
    return pOut;








|








525
526
527
528
529
530
531
532
533
534
535
536
537
538
539

  sqlite3VdbeMemSetNull(pOut);
  pOut->flags = MEM_Int;
  return pOut;
}
static Mem *out2Prerelease(Vdbe *p, VdbeOp *pOp){
  Mem *pOut;
  assert( pOp->p2>0 );
  assert( pOp->p2<=(p->nMem+1 - p->nCursor) );
  pOut = &p->aMem[pOp->p2];
  memAboutToChange(p, pOut);
  if( VdbeMemDynamic(pOut) ){
    return out2PrereleaseWithClear(pOut);
  }else{
    pOut->flags = MEM_Int;
    return pOut;
︙︙
664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704

#endif

    /* Sanity checking on other operands */
#ifdef SQLITE_DEBUG
    assert( pOp->opflags==sqlite3OpcodeProperty[pOp->opcode] );
    if( (pOp->opflags & OPFLG_IN1)!=0 ){
      assert( pOp->p1>0 );
      assert( pOp->p1<=(p->nMem-p->nCursor) );
      assert( memIsValid(&aMem[pOp->p1]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p1]) );
      REGISTER_TRACE(pOp->p1, &aMem[pOp->p1]);
    }
    if( (pOp->opflags & OPFLG_IN2)!=0 ){
      assert( pOp->p2>0 );
      assert( pOp->p2<=(p->nMem-p->nCursor) );
      assert( memIsValid(&aMem[pOp->p2]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p2]) );
      REGISTER_TRACE(pOp->p2, &aMem[pOp->p2]);
    }
    if( (pOp->opflags & OPFLG_IN3)!=0 ){
      assert( pOp->p3>0 );
      assert( pOp->p3<=(p->nMem-p->nCursor) );
      assert( memIsValid(&aMem[pOp->p3]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p3]) );
      REGISTER_TRACE(pOp->p3, &aMem[pOp->p3]);
    }
    if( (pOp->opflags & OPFLG_OUT2)!=0 ){
      assert( pOp->p2>0 );
      assert( pOp->p2<=(p->nMem-p->nCursor) );
      memAboutToChange(p, &aMem[pOp->p2]);
    }
    if( (pOp->opflags & OPFLG_OUT3)!=0 ){
      assert( pOp->p3>0 );
      assert( pOp->p3<=(p->nMem-p->nCursor) );
      memAboutToChange(p, &aMem[pOp->p3]);
    }
#endif
#if defined(SQLITE_DEBUG) || defined(VDBE_PROFILE)
    pOrigOp = pOp;
#endif
  








|






|






|






|




|








664
665
666
667
668
669
670
671
672
673
674
675
676
677
678
679
680
681
682
683
684
685
686
687
688
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704

#endif

    /* Sanity checking on other operands */
#ifdef SQLITE_DEBUG
    assert( pOp->opflags==sqlite3OpcodeProperty[pOp->opcode] );
    if( (pOp->opflags & OPFLG_IN1)!=0 ){
      assert( pOp->p1>0 );
      assert( pOp->p1<=(p->nMem+1 - p->nCursor) );
      assert( memIsValid(&aMem[pOp->p1]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p1]) );
      REGISTER_TRACE(pOp->p1, &aMem[pOp->p1]);
    }
    if( (pOp->opflags & OPFLG_IN2)!=0 ){
      assert( pOp->p2>0 );
      assert( pOp->p2<=(p->nMem+1 - p->nCursor) );
      assert( memIsValid(&aMem[pOp->p2]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p2]) );
      REGISTER_TRACE(pOp->p2, &aMem[pOp->p2]);
    }
    if( (pOp->opflags & OPFLG_IN3)!=0 ){
      assert( pOp->p3>0 );
      assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
      assert( memIsValid(&aMem[pOp->p3]) );
      assert( sqlite3VdbeCheckMemInvariants(&aMem[pOp->p3]) );
      REGISTER_TRACE(pOp->p3, &aMem[pOp->p3]);
    }
    if( (pOp->opflags & OPFLG_OUT2)!=0 ){
      assert( pOp->p2>0 );
      assert( pOp->p2<=(p->nMem+1 - p->nCursor) );
      memAboutToChange(p, &aMem[pOp->p2]);
    }
    if( (pOp->opflags & OPFLG_OUT3)!=0 ){
      assert( pOp->p3>0 );
      assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
      memAboutToChange(p, &aMem[pOp->p3]);
    }
#endif
#if defined(SQLITE_DEBUG) || defined(VDBE_PROFILE)
    pOrigOp = pOp;
#endif
︙︙
789
790
791
792
793
794
795
796
797
798
799
800
801
802
803


/* Opcode:  Gosub P1 P2 * * *
**
** Write the current address onto register P1
** and then jump to address P2.
*/
case OP_Gosub: {            /* jump */
  assert( pOp->p1>0 && pOp->p1<=(p->nMem-p->nCursor) );
  pIn1 = &aMem[pOp->p1];
  assert( VdbeMemDynamic(pIn1)==0 );
  memAboutToChange(p, pIn1);
  pIn1->flags = MEM_Int;
  pIn1->u.i = (int)(pOp-aOp);
  REGISTER_TRACE(pOp->p1, pIn1);









|








789
790
791
792
793
794
795
796
797
798
799
800
801
802
803


/* Opcode:  Gosub P1 P2 * * *
**
** Write the current address onto register P1
** and then jump to address P2.
*/
case OP_Gosub: {            /* jump */
  assert( pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor) );
  pIn1 = &aMem[pOp->p1];
  assert( VdbeMemDynamic(pIn1)==0 );
  memAboutToChange(p, pIn1);
  pIn1->flags = MEM_Int;
  pIn1->u.i = (int)(pOp-aOp);
  REGISTER_TRACE(pOp->p1, pIn1);
︙︙
829
830
831
832
833
834
835
836
837
838
839
840
841
842
843

** If P2!=0 then the coroutine implementation immediately follows
** this opcode.  So jump over the coroutine implementation to
** address P2.
**
** See also: EndCoroutine
*/
case OP_InitCoroutine: {     /* jump */
  assert( pOp->p1>0 &&  pOp->p1<=(p->nMem-p->nCursor) );
  assert( pOp->p2>=0 && pOp->p2<p->nOp );
  assert( pOp->p3>=0 && pOp->p3<p->nOp );
  pOut = &aMem[pOp->p1];
  assert( !VdbeMemDynamic(pOut) );
  pOut->u.i = pOp->p3 - 1;
  pOut->flags = MEM_Int;
  if( pOp->p2 ) goto jump_to_p2;








|








829
830
831
832
833
834
835
836
837
838
839
840
841
842
843

** If P2!=0 then the coroutine implementation immediately follows
** this opcode.  So jump over the coroutine implementation to
** address P2.
**
** See also: EndCoroutine
*/
case OP_InitCoroutine: {     /* jump */
  assert( pOp->p1>0 &&  pOp->p1<=(p->nMem+1 - p->nCursor) );
  assert( pOp->p2>=0 && pOp->p2<p->nOp );
  assert( pOp->p3>=0 && pOp->p3<p->nOp );
  pOut = &aMem[pOp->p1];
  assert( !VdbeMemDynamic(pOut) );
  pOut->u.i = pOp->p3 - 1;
  pOut->flags = MEM_Int;
  if( pOp->p2 ) goto jump_to_p2;
︙︙
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112

  pOut->z = pOp->p4.z;
  pOut->n = pOp->p1;
  pOut->enc = encoding;
  UPDATE_MAX_BLOBSIZE(pOut);
#ifndef SQLITE_LIKE_DOESNT_MATCH_BLOBS
  if( pOp->p5 ){
    assert( pOp->p3>0 );
    assert( pOp->p3<=(p->nMem-p->nCursor) );
    pIn3 = &aMem[pOp->p3];
    assert( pIn3->flags & MEM_Int );
    if( pIn3->u.i ) pOut->flags = MEM_Blob|MEM_Static|MEM_Term;
  }
#endif
  break;
}








|








1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112

  pOut->z = pOp->p4.z;
  pOut->n = pOp->p1;
  pOut->enc = encoding;
  UPDATE_MAX_BLOBSIZE(pOut);
#ifndef SQLITE_LIKE_DOESNT_MATCH_BLOBS
  if( pOp->p5 ){
    assert( pOp->p3>0 );
    assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
    pIn3 = &aMem[pOp->p3];
    assert( pIn3->flags & MEM_Int );
    if( pIn3->u.i ) pOut->flags = MEM_Blob|MEM_Static|MEM_Term;
  }
#endif
  break;
}
︙︙
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159

** OP_Ne or OP_Eq.
*/
case OP_Null: {           /* out2 */
  int cnt;
  u16 nullFlag;
  pOut = out2Prerelease(p, pOp);
  cnt = pOp->p3-pOp->p2;
  assert( pOp->p3<=(p->nMem-p->nCursor) );
  pOut->flags = nullFlag = pOp->p1 ? (MEM_Null|MEM_Cleared) : MEM_Null;
  while( cnt>0 ){
    pOut++;
    memAboutToChange(p, pOut);
    sqlite3VdbeMemSetNull(pOut);
    pOut->flags = nullFlag;
    cnt--;
  }
  break;
}

/* Opcode: SoftNull P1 * * * *
** Synopsis:  r[P1]=NULL
**
** Set register P1 to have the value NULL as seen by the OP_MakeRecord
** instruction, but do not free any string or blob memory associated with
** the register, so that if the value was a string or blob that was
** previously copied using OP_SCopy, the copies will continue to be valid.
*/
case OP_SoftNull: {
  assert( pOp->p1>0 && pOp->p1<=(p->nMem-p->nCursor) );
  pOut = &aMem[pOp->p1];
  pOut->flags = (pOut->flags|MEM_Null)&~MEM_Undefined;
  break;
}

/* Opcode: Blob P1 P2 * P4 *
** Synopsis: r[P2]=P4 (len=P1)








|




















|








1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
1158
1159

** OP_Ne or OP_Eq.
*/
case OP_Null: {           /* out2 */
  int cnt;
  u16 nullFlag;
  pOut = out2Prerelease(p, pOp);
  cnt = pOp->p3-pOp->p2;
  assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
  pOut->flags = nullFlag = pOp->p1 ? (MEM_Null|MEM_Cleared) : MEM_Null;
  while( cnt>0 ){
    pOut++;
    memAboutToChange(p, pOut);
    sqlite3VdbeMemSetNull(pOut);
    pOut->flags = nullFlag;
    cnt--;
  }
  break;
}

/* Opcode: SoftNull P1 * * * *
** Synopsis:  r[P1]=NULL
**
** Set register P1 to have the value NULL as seen by the OP_MakeRecord
** instruction, but do not free any string or blob memory associated with
** the register, so that if the value was a string or blob that was
** previously copied using OP_SCopy, the copies will continue to be valid.
*/
case OP_SoftNull: {
  assert( pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor) );
  pOut = &aMem[pOp->p1];
  pOut->flags = (pOut->flags|MEM_Null)&~MEM_Undefined;
  break;
}

/* Opcode: Blob P1 P2 * P4 *
** Synopsis: r[P2]=P4 (len=P1)
︙︙
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227

  p2 = pOp->p2;
  assert( n>0 && p1>0 && p2>0 );
  assert( p1+n<=p2 || p2+n<=p1 );

  pIn1 = &aMem[p1];
  pOut = &aMem[p2];
  do{
    assert( pOut<=&aMem[(p->nMem-p->nCursor)] );
    assert( pIn1<=&aMem[(p->nMem-p->nCursor)] );
    assert( memIsValid(pIn1) );
    memAboutToChange(p, pOut);
    sqlite3VdbeMemMove(pOut, pIn1);
#ifdef SQLITE_DEBUG
    if( pOut->pScopyFrom>=&aMem[p1] && pOut->pScopyFrom<pOut ){
      pOut->pScopyFrom += pOp->p2 - p1;
    }








|
|








1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227

  p2 = pOp->p2;
  assert( n>0 && p1>0 && p2>0 );
  assert( p1+n<=p2 || p2+n<=p1 );

  pIn1 = &aMem[p1];
  pOut = &aMem[p2];
  do{
    assert( pOut<=&aMem[(p->nMem+1 - p->nCursor)] );
    assert( pIn1<=&aMem[(p->nMem+1 - p->nCursor)] );
    assert( memIsValid(pIn1) );
    memAboutToChange(p, pOut);
    sqlite3VdbeMemMove(pOut, pIn1);
#ifdef SQLITE_DEBUG
    if( pOut->pScopyFrom>=&aMem[p1] && pOut->pScopyFrom<pOut ){
      pOut->pScopyFrom += pOp->p2 - p1;
    }
︙︙
1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327

** the result row.
*/
case OP_ResultRow: {
  Mem *pMem;
  int i;
  assert( p->nResColumn==pOp->p2 );
  assert( pOp->p1>0 );
  assert( pOp->p1+pOp->p2<=(p->nMem-p->nCursor)+1 );

#ifndef SQLITE_OMIT_PROGRESS_CALLBACK
  /* Run the progress counter just before returning.
  */
  if( db->xProgress!=0
   && nVmStep>=nProgressLimit
   && db->xProgress(db->pProgressArg)!=0








|








1313
1314
1315
1316
1317
1318
1319
1320
1321
1322
1323
1324
1325
1326
1327

** the result row.
*/
case OP_ResultRow: {
  Mem *pMem;
  int i;
  assert( p->nResColumn==pOp->p2 );
  assert( pOp->p1>0 );
  assert( pOp->p1+pOp->p2<=(p->nMem+1 - p->nCursor)+1 );

#ifndef SQLITE_OMIT_PROGRESS_CALLBACK
  /* Run the progress counter just before returning.
  */
  if( db->xProgress!=0
   && nVmStep>=nProgressLimit
   && db->xProgress(db->pProgressArg)!=0
︙︙
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640

*/
case OP_Function0: {
  int n;
  sqlite3_context *pCtx;

  assert( pOp->p4type==P4_FUNCDEF );
  n = pOp->p5;
  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem-p->nCursor)+1) );
  assert( pOp->p3<pOp->p2 || pOp->p3>=pOp->p2+n );
  pCtx = sqlite3DbMallocRawNN(db, sizeof(*pCtx) + (n-1)*sizeof(sqlite3_value*));
  if( pCtx==0 ) goto no_mem;
  pCtx->pOut = 0;
  pCtx->pFunc = pOp->p4.pFunc;
  pCtx->iOp = (int)(pOp - aOp);
  pCtx->pVdbe = p;








|
|








1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640

*/
case OP_Function0: {
  int n;
  sqlite3_context *pCtx;

  assert( pOp->p4type==P4_FUNCDEF );
  n = pOp->p5;
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem+1 - p->nCursor)+1) );
  assert( pOp->p3<pOp->p2 || pOp->p3>=pOp->p2+n );
  pCtx = sqlite3DbMallocRawNN(db, sizeof(*pCtx) + (n-1)*sizeof(sqlite3_value*));
  if( pCtx==0 ) goto no_mem;
  pCtx->pOut = 0;
  pCtx->pFunc = pOp->p4.pFunc;
  pCtx->iOp = (int)(pOp - aOp);
  pCtx->pVdbe = p;
︙︙
2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144

  assert( pKeyInfo!=0 );
  p1 = pOp->p1;
  p2 = pOp->p2;
#if SQLITE_DEBUG
  if( aPermute ){
    int k, mx = 0;
    for(k=0; k<n; k++) if( aPermute[k]>mx ) mx = aPermute[k];
    assert( p1>0 && p1+mx<=(p->nMem-p->nCursor)+1 );
    assert( p2>0 && p2+mx<=(p->nMem-p->nCursor)+1 );
  }else{
    assert( p1>0 && p1+n<=(p->nMem-p->nCursor)+1 );
    assert( p2>0 && p2+n<=(p->nMem-p->nCursor)+1 );
  }
#endif /* SQLITE_DEBUG */
  for(i=0; i<n; i++){
    idx = aPermute ? aPermute[i] : i;
    assert( memIsValid(&aMem[p1+idx]) );
    assert( memIsValid(&aMem[p2+idx]) );
    REGISTER_TRACE(p1+idx, &aMem[p1+idx]);








|
|

|
|








2126
2127
2128
2129
2130
2131
2132
2133
2134
2135
2136
2137
2138
2139
2140
2141
2142
2143
2144

  assert( pKeyInfo!=0 );
  p1 = pOp->p1;
  p2 = pOp->p2;
#if SQLITE_DEBUG
  if( aPermute ){
    int k, mx = 0;
    for(k=0; k<n; k++) if( aPermute[k]>mx ) mx = aPermute[k];
    assert( p1>0 && p1+mx<=(p->nMem+1 - p->nCursor)+1 );
    assert( p2>0 && p2+mx<=(p->nMem+1 - p->nCursor)+1 );
  }else{
    assert( p1>0 && p1+n<=(p->nMem+1 - p->nCursor)+1 );
    assert( p2>0 && p2+n<=(p->nMem+1 - p->nCursor)+1 );
  }
#endif /* SQLITE_DEBUG */
  for(i=0; i<n; i++){
    idx = aPermute ? aPermute[i] : i;
    assert( memIsValid(&aMem[p1+idx]) );
    assert( memIsValid(&aMem[p2+idx]) );
    REGISTER_TRACE(p1+idx, &aMem[p1+idx]);
︙︙
2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406


  pC = p->apCsr[pOp->p1];
  p2 = pOp->p2;

  /* If the cursor cache is stale, bring it up-to-date */
  rc = sqlite3VdbeCursorMoveto(&pC, &p2);

  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  assert( pC!=0 );
  assert( p2<pC->nField );
  aOffset = pC->aOffset;
  assert( pC->eCurType!=CURTYPE_VTAB );








|








2392
2393
2394
2395
2396
2397
2398
2399
2400
2401
2402
2403
2404
2405
2406


  pC = p->apCsr[pOp->p1];
  p2 = pOp->p2;

  /* If the cursor cache is stale, bring it up-to-date */
  rc = sqlite3VdbeCursorMoveto(&pC, &p2);

  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  assert( pC!=0 );
  assert( p2<pC->nField );
  aOffset = pC->aOffset;
  assert( pC->eCurType!=CURTYPE_VTAB );
︙︙
2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649

  char cAff;               /* A single character of affinity */

  zAffinity = pOp->p4.z;
  assert( zAffinity!=0 );
  assert( zAffinity[pOp->p2]==0 );
  pIn1 = &aMem[pOp->p1];
  while( (cAff = *(zAffinity++))!=0 ){
    assert( pIn1 <= &p->aMem[(p->nMem-p->nCursor)] );
    assert( memIsValid(pIn1) );
    applyAffinity(pIn1, cAff, encoding);
    pIn1++;
  }
  break;
}









|








2635
2636
2637
2638
2639
2640
2641
2642
2643
2644
2645
2646
2647
2648
2649

  char cAff;               /* A single character of affinity */

  zAffinity = pOp->p4.z;
  assert( zAffinity!=0 );
  assert( zAffinity[pOp->p2]==0 );
  pIn1 = &aMem[pOp->p1];
  while( (cAff = *(zAffinity++))!=0 ){
    assert( pIn1 <= &p->aMem[(p->nMem+1 - p->nCursor)] );
    assert( memIsValid(pIn1) );
    applyAffinity(pIn1, cAff, encoding);
    pIn1++;
  }
  break;
}
︙︙
2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711

  ** of the record to data0.
  */
  nData = 0;         /* Number of bytes of data space */
  nHdr = 0;          /* Number of bytes of header space */
  nZero = 0;         /* Number of zero bytes at the end of the record */
  nField = pOp->p1;
  zAffinity = pOp->p4.z;
  assert( nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem-p->nCursor)+1 );
  pData0 = &aMem[nField];
  nField = pOp->p2;
  pLast = &pData0[nField-1];
  file_format = p->minWriteFileFormat;

  /* Identify the output register */
  assert( pOp->p3<pOp->p1 || pOp->p3>=pOp->p1+pOp->p2 );








|








2697
2698
2699
2700
2701
2702
2703
2704
2705
2706
2707
2708
2709
2710
2711

  ** of the record to data0.
  */
  nData = 0;         /* Number of bytes of data space */
  nHdr = 0;          /* Number of bytes of header space */
  nZero = 0;         /* Number of zero bytes at the end of the record */
  nField = pOp->p1;
  zAffinity = pOp->p4.z;
  assert( nField>0 && pOp->p2>0 && pOp->p2+nField<=(p->nMem+1 - p->nCursor)+1 );
  pData0 = &aMem[nField];
  nField = pOp->p2;
  pLast = &pData0[nField-1];
  file_format = p->minWriteFileFormat;

  /* Identify the output register */
  assert( pOp->p3<pOp->p1 || pOp->p3>=pOp->p1+pOp->p2 );
︙︙
2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801

    /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
    ** immediately follow the header. */
    j += sqlite3VdbeSerialPut(&zNewRecord[j], pRec, serial_type); /* content */
  }while( (++pRec)<=pLast );
  assert( i==nHdr );
  assert( j==nByte );

  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
  pOut->n = (int)nByte;
  pOut->flags = MEM_Blob;
  if( nZero ){
    pOut->u.nZero = nZero;
    pOut->flags |= MEM_Zero;
  }
  pOut->enc = SQLITE_UTF8;  /* In case the blob is ever converted to text */








|








2787
2788
2789
2790
2791
2792
2793
2794
2795
2796
2797
2798
2799
2800
2801

    /* EVIDENCE-OF: R-64536-51728 The values for each column in the record
    ** immediately follow the header. */
    j += sqlite3VdbeSerialPut(&zNewRecord[j], pRec, serial_type); /* content */
  }while( (++pRec)<=pLast );
  assert( i==nHdr );
  assert( j==nByte );

  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pOut->n = (int)nByte;
  pOut->flags = MEM_Blob;
  if( nZero ){
    pOut->u.nZero = nZero;
    pOut->flags |= MEM_Zero;
  }
  pOut->enc = SQLITE_UTF8;  /* In case the blob is ever converted to text */
︙︙
3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387

      p->minWriteFileFormat = pDb->pSchema->file_format;
    }
  }else{
    wrFlag = 0;
  }
  if( pOp->p5 & OPFLAG_P2ISREG ){
    assert( p2>0 );
    assert( p2<=(p->nMem-p->nCursor) );
    pIn2 = &aMem[p2];
    assert( memIsValid(pIn2) );
    assert( (pIn2->flags & MEM_Int)!=0 );
    sqlite3VdbeMemIntegerify(pIn2);
    p2 = (int)pIn2->u.i;
    /* The p2 value always comes from a prior OP_CreateTable opcode and
    ** that opcode will always set the p2 value to 2 or more or else fail.








|








3373
3374
3375
3376
3377
3378
3379
3380
3381
3382
3383
3384
3385
3386
3387

      p->minWriteFileFormat = pDb->pSchema->file_format;
    }
  }else{
    wrFlag = 0;
  }
  if( pOp->p5 & OPFLAG_P2ISREG ){
    assert( p2>0 );
    assert( p2<=(p->nMem+1 - p->nCursor) );
    pIn2 = &aMem[p2];
    assert( memIsValid(pIn2) );
    assert( (pIn2->flags & MEM_Int)!=0 );
    sqlite3VdbeMemIntegerify(pIn2);
    p2 = (int)pIn2->u.i;
    /* The p2 value always comes from a prior OP_CreateTable opcode and
    ** that opcode will always set the p2 value to 2 or more or else fail.
︙︙
4168
4169
4170
4171
4172
4173
4174
4175
4176
4177
4178
4179
4180
4181
4182

      if( p->pFrame ){
        for(pFrame=p->pFrame; pFrame->pParent; pFrame=pFrame->pParent);
        /* Assert that P3 is a valid memory cell. */
        assert( pOp->p3<=pFrame->nMem );
        pMem = &pFrame->aMem[pOp->p3];
      }else{
        /* Assert that P3 is a valid memory cell. */
        assert( pOp->p3<=(p->nMem-p->nCursor) );
        pMem = &aMem[pOp->p3];
        memAboutToChange(p, pMem);
      }
      assert( memIsValid(pMem) );

      REGISTER_TRACE(pOp->p3, pMem);
      sqlite3VdbeMemIntegerify(pMem);








|








4168
4169
4170
4171
4172
4173
4174
4175
4176
4177
4178
4179
4180
4181
4182

      if( p->pFrame ){
        for(pFrame=p->pFrame; pFrame->pParent; pFrame=pFrame->pParent);
        /* Assert that P3 is a valid memory cell. */
        assert( pOp->p3<=pFrame->nMem );
        pMem = &pFrame->aMem[pOp->p3];
      }else{
        /* Assert that P3 is a valid memory cell. */
        assert( pOp->p3<=(p->nMem+1 - p->nCursor) );
        pMem = &aMem[pOp->p3];
        memAboutToChange(p, pMem);
      }
      assert( memIsValid(pMem) );

      REGISTER_TRACE(pOp->p3, pMem);
      sqlite3VdbeMemIntegerify(pMem);
︙︙
4944
4945
4946
4947
4948
4949
4950
4951
4952
4953
4954
4955
4956
4957
4958

case OP_IdxDelete: {
  VdbeCursor *pC;
  BtCursor *pCrsr;
  int res;
  UnpackedRecord r;

  assert( pOp->p3>0 );
  assert( pOp->p2>0 && pOp->p2+pOp->p3<=(p->nMem-p->nCursor)+1 );
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  pC = p->apCsr[pOp->p1];
  assert( pC!=0 );
  assert( pC->eCurType==CURTYPE_BTREE );
  pCrsr = pC->uc.pCursor;
  assert( pCrsr!=0 );
  assert( pOp->p5==0 );








|








4944
4945
4946
4947
4948
4949
4950
4951
4952
4953
4954
4955
4956
4957
4958

case OP_IdxDelete: {
  VdbeCursor *pC;
  BtCursor *pCrsr;
  int res;
  UnpackedRecord r;

  assert( pOp->p3>0 );
  assert( pOp->p2>0 && pOp->p2+pOp->p3<=(p->nMem+1 - p->nCursor)+1 );
  assert( pOp->p1>=0 && pOp->p1<p->nCursor );
  pC = p->apCsr[pOp->p1];
  assert( pC!=0 );
  assert( pC->eCurType==CURTYPE_BTREE );
  pCrsr = pC->uc.pCursor;
  assert( pCrsr!=0 );
  assert( pOp->p5==0 );
︙︙
5450
5451
5452
5453
5454
5455
5456
5457
5458
5459
5460
5461
5462
5463
5464

  Mem *pnErr;     /* Register keeping track of errors remaining */

  assert( p->bIsReader );
  nRoot = pOp->p2;
  aRoot = pOp->p4.ai;
  assert( nRoot>0 );
  assert( aRoot[nRoot]==0 );
  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
  pnErr = &aMem[pOp->p3];
  assert( (pnErr->flags & MEM_Int)!=0 );
  assert( (pnErr->flags & (MEM_Str|MEM_Blob))==0 );
  pIn1 = &aMem[pOp->p1];
  assert( pOp->p5<db->nDb );
  assert( DbMaskTest(p->btreeMask, pOp->p5) );
  z = sqlite3BtreeIntegrityCheck(db->aDb[pOp->p5].pBt, aRoot, nRoot,








|








5450
5451
5452
5453
5454
5455
5456
5457
5458
5459
5460
5461
5462
5463
5464

  Mem *pnErr;     /* Register keeping track of errors remaining */

  assert( p->bIsReader );
  nRoot = pOp->p2;
  aRoot = pOp->p4.ai;
  assert( nRoot>0 );
  assert( aRoot[nRoot]==0 );
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pnErr = &aMem[pOp->p3];
  assert( (pnErr->flags & MEM_Int)!=0 );
  assert( (pnErr->flags & (MEM_Str|MEM_Blob))==0 );
  pIn1 = &aMem[pOp->p1];
  assert( pOp->p5<db->nDb );
  assert( DbMaskTest(p->btreeMask, pOp->p5) );
  z = sqlite3BtreeIntegrityCheck(db->aDb[pOp->p5].pBt, aRoot, nRoot,
︙︙
5640
5641
5642
5643
5644
5645
5646


5647
5648
5649
5650
5651
5652
5653

  if( (pRt->flags&MEM_Frame)==0 ){
    /* SubProgram.nMem is set to the number of memory cells used by the 
    ** program stored in SubProgram.aOp. As well as these, one memory
    ** cell is required for each cursor used by the program. Set local
    ** variable nMem (and later, VdbeFrame.nChildMem) to this value.
    */
    nMem = pProgram->nMem + pProgram->nCsr;


    nByte = ROUND8(sizeof(VdbeFrame))
              + nMem * sizeof(Mem)
              + pProgram->nCsr * sizeof(VdbeCursor *)
              + pProgram->nOnce * sizeof(u8);
    pFrame = sqlite3DbMallocZero(db, nByte);
    if( !pFrame ){
      goto no_mem;








>
>








5640
5641
5642
5643
5644
5645
5646
5647
5648
5649
5650
5651
5652
5653
5654
5655

  if( (pRt->flags&MEM_Frame)==0 ){
    /* SubProgram.nMem is set to the number of memory cells used by the 
    ** program stored in SubProgram.aOp. As well as these, one memory
    ** cell is required for each cursor used by the program. Set local
    ** variable nMem (and later, VdbeFrame.nChildMem) to this value.
    */
    nMem = pProgram->nMem + pProgram->nCsr;
    assert( nMem>0 );
    if( pProgram->nCsr==0 ) nMem++;
    nByte = ROUND8(sizeof(VdbeFrame))
              + nMem * sizeof(Mem)
              + pProgram->nCsr * sizeof(VdbeCursor *)
              + pProgram->nOnce * sizeof(u8);
    pFrame = sqlite3DbMallocZero(db, nByte);
    if( !pFrame ){
      goto no_mem;
︙︙
5676
5677
5678
5679
5680
5681
5682
5683

5684
5685
5686
5687
5688
5689
5690
5691
5692
5693
5694
5695
5696
5697
5698
5699
5700
5701
5702
5703
5704
5705
5706
5707
5708

    pEnd = &VdbeFrameMem(pFrame)[pFrame->nChildMem];
    for(pMem=VdbeFrameMem(pFrame); pMem!=pEnd; pMem++){
      pMem->flags = MEM_Undefined;
      pMem->db = db;
    }
  }else{
    pFrame = pRt->u.pFrame;
    assert( pProgram->nMem+pProgram->nCsr==pFrame->nChildMem );

    assert( pProgram->nCsr==pFrame->nChildCsr );
    assert( (int)(pOp - aOp)==pFrame->pc );
  }

  p->nFrame++;
  pFrame->pParent = p->pFrame;
  pFrame->lastRowid = lastRowid;
  pFrame->nChange = p->nChange;
  pFrame->nDbChange = p->db->nChange;
  assert( pFrame->pAuxData==0 );
  pFrame->pAuxData = p->pAuxData;
  p->pAuxData = 0;
  p->nChange = 0;
  p->pFrame = pFrame;
  p->aMem = aMem = &VdbeFrameMem(pFrame)[-1];
  p->nMem = pFrame->nChildMem;
  p->nCursor = (u16)pFrame->nChildCsr;
  p->apCsr = (VdbeCursor **)&aMem[p->nMem+1];
  p->aOp = aOp = pProgram->aOp;
  p->nOp = pProgram->nOp;
  p->aOnceFlag = (u8 *)&p->apCsr[p->nCursor];
  p->nOnceFlag = pProgram->nOnce;
#ifdef SQLITE_ENABLE_STMT_SCANSTATUS
  p->anExec = 0;
#endif








|
>














|


|








5678
5679
5680
5681
5682
5683
5684
5685
5686
5687
5688
5689
5690
5691
5692
5693
5694
5695
5696
5697
5698
5699
5700
5701
5702
5703
5704
5705
5706
5707
5708
5709
5710
5711

    pEnd = &VdbeFrameMem(pFrame)[pFrame->nChildMem];
    for(pMem=VdbeFrameMem(pFrame); pMem!=pEnd; pMem++){
      pMem->flags = MEM_Undefined;
      pMem->db = db;
    }
  }else{
    pFrame = pRt->u.pFrame;
    assert( pProgram->nMem+pProgram->nCsr==pFrame->nChildMem 
        || (pProgram->nCsr==0 && pProgram->nMem+1==pFrame->nChildMem) );
    assert( pProgram->nCsr==pFrame->nChildCsr );
    assert( (int)(pOp - aOp)==pFrame->pc );
  }

  p->nFrame++;
  pFrame->pParent = p->pFrame;
  pFrame->lastRowid = lastRowid;
  pFrame->nChange = p->nChange;
  pFrame->nDbChange = p->db->nChange;
  assert( pFrame->pAuxData==0 );
  pFrame->pAuxData = p->pAuxData;
  p->pAuxData = 0;
  p->nChange = 0;
  p->pFrame = pFrame;
  p->aMem = aMem = VdbeFrameMem(pFrame);
  p->nMem = pFrame->nChildMem;
  p->nCursor = (u16)pFrame->nChildCsr;
  p->apCsr = (VdbeCursor **)&aMem[p->nMem];
  p->aOp = aOp = pProgram->aOp;
  p->nOp = pProgram->nOp;
  p->aOnceFlag = (u8 *)&p->apCsr[p->nCursor];
  p->nOnceFlag = pProgram->nOnce;
#ifdef SQLITE_ENABLE_STMT_SCANSTATUS
  p->anExec = 0;
#endif
︙︙
5940
5941
5942
5943
5944
5945
5946
5947
5948
5949
5950
5951
5952
5953
5954
5955

*/
case OP_AggStep0: {
  int n;
  sqlite3_context *pCtx;

  assert( pOp->p4type==P4_FUNCDEF );
  n = pOp->p5;
  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem-p->nCursor)+1) );
  assert( pOp->p3<pOp->p2 || pOp->p3>=pOp->p2+n );
  pCtx = sqlite3DbMallocRawNN(db, sizeof(*pCtx) + (n-1)*sizeof(sqlite3_value*));
  if( pCtx==0 ) goto no_mem;
  pCtx->pMem = 0;
  pCtx->pFunc = pOp->p4.pFunc;
  pCtx->iOp = (int)(pOp - aOp);
  pCtx->pVdbe = p;








|
|








5943
5944
5945
5946
5947
5948
5949
5950
5951
5952
5953
5954
5955
5956
5957
5958

*/
case OP_AggStep0: {
  int n;
  sqlite3_context *pCtx;

  assert( pOp->p4type==P4_FUNCDEF );
  n = pOp->p5;
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  assert( n==0 || (pOp->p2>0 && pOp->p2+n<=(p->nMem+1 - p->nCursor)+1) );
  assert( pOp->p3<pOp->p2 || pOp->p3>=pOp->p2+n );
  pCtx = sqlite3DbMallocRawNN(db, sizeof(*pCtx) + (n-1)*sizeof(sqlite3_value*));
  if( pCtx==0 ) goto no_mem;
  pCtx->pMem = 0;
  pCtx->pFunc = pOp->p4.pFunc;
  pCtx->iOp = (int)(pOp - aOp);
  pCtx->pVdbe = p;
︙︙
6020
6021
6022
6023
6024
6025
6026
6027
6028
6029
6030
6031
6032
6033
6034

** argument is not used by this opcode.  It is only there to disambiguate
** functions that can take varying numbers of arguments.  The
** P4 argument is only needed for the degenerate case where
** the step function was not previously called.
*/
case OP_AggFinal: {
  Mem *pMem;
  assert( pOp->p1>0 && pOp->p1<=(p->nMem-p->nCursor) );
  pMem = &aMem[pOp->p1];
  assert( (pMem->flags & ~(MEM_Null|MEM_Agg))==0 );
  rc = sqlite3VdbeMemFinalize(pMem, pOp->p4.pFunc);
  if( rc ){
    sqlite3VdbeError(p, "%s", sqlite3_value_text(pMem));
    goto abort_due_to_error;
  }








|








6023
6024
6025
6026
6027
6028
6029
6030
6031
6032
6033
6034
6035
6036
6037

** argument is not used by this opcode.  It is only there to disambiguate
** functions that can take varying numbers of arguments.  The
** P4 argument is only needed for the degenerate case where
** the step function was not previously called.
*/
case OP_AggFinal: {
  Mem *pMem;
  assert( pOp->p1>0 && pOp->p1<=(p->nMem+1 - p->nCursor) );
  pMem = &aMem[pOp->p1];
  assert( (pMem->flags & ~(MEM_Null|MEM_Agg))==0 );
  rc = sqlite3VdbeMemFinalize(pMem, pOp->p4.pFunc);
  if( rc ){
    sqlite3VdbeError(p, "%s", sqlite3_value_text(pMem));
    goto abort_due_to_error;
  }
︙︙
6462
6463
6464
6465
6466
6467
6468
6469
6470
6471
6472
6473
6474
6475
6476

  sqlite3_vtab *pVtab;
  const sqlite3_module *pModule;
  Mem *pDest;
  sqlite3_context sContext;

  VdbeCursor *pCur = p->apCsr[pOp->p1];
  assert( pCur->eCurType==CURTYPE_VTAB );
  assert( pOp->p3>0 && pOp->p3<=(p->nMem-p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
  if( pCur->nullRow ){
    sqlite3VdbeMemSetNull(pDest);
    break;
  }
  pVtab = pCur->uc.pVCur->pVtab;








|








6465
6466
6467
6468
6469
6470
6471
6472
6473
6474
6475
6476
6477
6478
6479

  sqlite3_vtab *pVtab;
  const sqlite3_module *pModule;
  Mem *pDest;
  sqlite3_context sContext;

  VdbeCursor *pCur = p->apCsr[pOp->p1];
  assert( pCur->eCurType==CURTYPE_VTAB );
  assert( pOp->p3>0 && pOp->p3<=(p->nMem+1 - p->nCursor) );
  pDest = &aMem[pOp->p3];
  memAboutToChange(p, pDest);
  if( pCur->nullRow ){
    sqlite3VdbeMemSetNull(pDest);
    break;
  }
  pVtab = pCur->uc.pVCur->pVtab;
︙︙
6820
6821
6822
6823
6824
6825
6826

6827
6828
6829
6830
6831
6832
6833

abort_due_to_error:
  if( db->mallocFailed ) rc = SQLITE_NOMEM_BKPT;
  assert( rc );
  if( p->zErrMsg==0 && rc!=SQLITE_IOERR_NOMEM ){
    sqlite3VdbeError(p, "%s", sqlite3ErrStr(rc));
  }
  p->rc = rc;

  testcase( sqlite3GlobalConfig.xLog!=0 );
  sqlite3_log(rc, "statement aborts at %d: [%s] %s", 
                   (int)(pOp - aOp), p->zSql, p->zErrMsg);
  sqlite3VdbeHalt(p);
  if( rc==SQLITE_IOERR_NOMEM ) sqlite3OomFault(db);
  rc = SQLITE_ERROR;
  if( resetSchemaOnFault>0 ){








>








6823
6824
6825
6826
6827
6828
6829
6830
6831
6832
6833
6834
6835
6836
6837

abort_due_to_error:
  if( db->mallocFailed ) rc = SQLITE_NOMEM_BKPT;
  assert( rc );
  if( p->zErrMsg==0 && rc!=SQLITE_IOERR_NOMEM ){
    sqlite3VdbeError(p, "%s", sqlite3ErrStr(rc));
  }
  p->rc = rc;
  sqlite3SystemError(db, rc);
  testcase( sqlite3GlobalConfig.xLog!=0 );
  sqlite3_log(rc, "statement aborts at %d: [%s] %s", 
                   (int)(pOp - aOp), p->zSql, p->zErrMsg);
  sqlite3VdbeHalt(p);
  if( rc==SQLITE_IOERR_NOMEM ) sqlite3OomFault(db);
  rc = SQLITE_ERROR;
  if( resetSchemaOnFault>0 ){
︙︙
Changes to src/vdbeaux.c.
︙︙
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811

  */
  assert( p->nOp>0 );

  /* Set the magic to VDBE_MAGIC_RUN sooner rather than later. */
  p->magic = VDBE_MAGIC_RUN;

#ifdef SQLITE_DEBUG
  for(i=1; i<p->nMem; i++){
    assert( p->aMem[i].db==p->db );
  }
#endif
  p->pc = -1;
  p->rc = SQLITE_OK;
  p->errorAction = OE_Abort;
  p->nChange = 0;








|








1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811

  */
  assert( p->nOp>0 );

  /* Set the magic to VDBE_MAGIC_RUN sooner rather than later. */
  p->magic = VDBE_MAGIC_RUN;

#ifdef SQLITE_DEBUG
  for(i=0; i<p->nMem; i++){
    assert( p->aMem[i].db==p->db );
  }
#endif
  p->pc = -1;
  p->rc = SQLITE_OK;
  p->errorAction = OE_Abort;
  p->nChange = 0;
︙︙
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875

1876
1877
1878

1879
1880
1881
1882
1883
1884
1885

  nVar = pParse->nVar;
  nMem = pParse->nMem;
  nCursor = pParse->nTab;
  nArg = pParse->nMaxArg;
  nOnce = pParse->nOnce;
  if( nOnce==0 ) nOnce = 1; /* Ensure at least one byte in p->aOnceFlag[] */
  
  /* For each cursor required, also allocate a memory cell. Memory
  ** cells (nMem+1-nCursor)..nMem, inclusive, will never be used by
  ** the vdbe program. Instead they are used to allocate memory for
  ** VdbeCursor/BtCursor structures. The blob of memory associated with 
  ** cursor 0 is stored in memory cell nMem. Memory cell (nMem-1)
  ** stores the blob of memory associated with cursor 1, etc.
  **

  ** See also: allocateCursor().
  */
  nMem += nCursor;


  /* Figure out how much reusable memory is available at the end of the
  ** opcode array.  This extra memory will be reallocated for other elements
  ** of the prepared statement.
  */
  n = ROUND8(sizeof(Op)*p->nOp);              /* Bytes of opcode memory used */
  x.pSpace = &((u8*)p->aOp)[n];               /* Unused opcode memory */








|
<
|
<
<
<
<
>



>








1862
1863
1864
1865
1866
1867
1868
1869

1870




1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882

  nVar = pParse->nVar;
  nMem = pParse->nMem;
  nCursor = pParse->nTab;
  nArg = pParse->nMaxArg;
  nOnce = pParse->nOnce;
  if( nOnce==0 ) nOnce = 1; /* Ensure at least one byte in p->aOnceFlag[] */
  
  /* Each cursor uses a memory cell.  The first cursor (cursor 0) can

  ** use aMem[0] which is not otherwise used by the VDBE program.  Allocate




  ** space at the end of aMem[] for cursors 1 and greater.
  ** See also: allocateCursor().
  */
  nMem += nCursor;
  if( nCursor==0 && nMem>0 ) nMem++;  /* Space for aMem[0] even if not used */

  /* Figure out how much reusable memory is available at the end of the
  ** opcode array.  This extra memory will be reallocated for other elements
  ** of the prepared statement.
  */
  n = ROUND8(sizeof(Op)*p->nOp);              /* Bytes of opcode memory used */
  x.pSpace = &((u8*)p->aOp)[n];               /* Unused opcode memory */
︙︙
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949

    }
  }
  p->nzVar = pParse->nzVar;
  p->azVar = pParse->azVar;
  pParse->nzVar =  0;
  pParse->azVar = 0;
  if( p->aMem ){
    p->aMem--;                      /* aMem[] goes from 1..nMem */
    p->nMem = nMem;                 /*       not from 0..nMem-1 */
    for(n=1; n<=nMem; n++){
      p->aMem[n].flags = MEM_Undefined;
      p->aMem[n].db = db;
    }
  }
  p->explain = pParse->explain;
  sqlite3VdbeRewind(p);
}








<
|
|








1930
1931
1932
1933
1934
1935
1936

1937
1938
1939
1940
1941
1942
1943
1944
1945

    }
  }
  p->nzVar = pParse->nzVar;
  p->azVar = pParse->azVar;
  pParse->nzVar =  0;
  pParse->azVar = 0;
  if( p->aMem ){

    p->nMem = nMem;
    for(n=0; n<nMem; n++){
      p->aMem[n].flags = MEM_Undefined;
      p->aMem[n].db = db;
    }
  }
  p->explain = pParse->explain;
  sqlite3VdbeRewind(p);
}
︙︙
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055
2056
2057
2058
2059

    sqlite3VdbeFrameRestore(pFrame);
    p->pFrame = 0;
    p->nFrame = 0;
  }
  assert( p->nFrame==0 );
  closeCursorsInFrame(p);
  if( p->aMem ){
    releaseMemArray(&p->aMem[1], p->nMem);
  }
  while( p->pDelFrame ){
    VdbeFrame *pDel = p->pDelFrame;
    p->pDelFrame = pDel->pParent;
    sqlite3VdbeFrameDelete(pDel);
  }









|








2041
2042
2043
2044
2045
2046
2047
2048
2049
2050
2051
2052
2053
2054
2055

    sqlite3VdbeFrameRestore(pFrame);
    p->pFrame = 0;
    p->nFrame = 0;
  }
  assert( p->nFrame==0 );
  closeCursorsInFrame(p);
  if( p->aMem ){
    releaseMemArray(p->aMem, p->nMem);
  }
  while( p->pDelFrame ){
    VdbeFrame *pDel = p->pDelFrame;
    p->pDelFrame = pDel->pParent;
    sqlite3VdbeFrameDelete(pDel);
  }
︙︙
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080
2081
2082
2083
2084


#ifdef SQLITE_DEBUG
  /* Execute assert() statements to ensure that the Vdbe.apCsr[] and 
  ** Vdbe.aMem[] arrays have already been cleaned up.  */
  int i;
  if( p->apCsr ) for(i=0; i<p->nCursor; i++) assert( p->apCsr[i]==0 );
  if( p->aMem ){
    for(i=1; i<=p->nMem; i++) assert( p->aMem[i].flags==MEM_Undefined );
  }
#endif

  sqlite3DbFree(db, p->zErrMsg);
  p->zErrMsg = 0;
  p->pResultSet = 0;
}








|








2066
2067
2068
2069
2070
2071
2072
2073
2074
2075
2076
2077
2078
2079
2080


#ifdef SQLITE_DEBUG
  /* Execute assert() statements to ensure that the Vdbe.apCsr[] and 
  ** Vdbe.aMem[] arrays have already been cleaned up.  */
  int i;
  if( p->apCsr ) for(i=0; i<p->nCursor; i++) assert( p->apCsr[i]==0 );
  if( p->aMem ){
    for(i=0; i<p->nMem; i++) assert( p->aMem[i].flags==MEM_Undefined );
  }
#endif

  sqlite3DbFree(db, p->zErrMsg);
  p->zErrMsg = 0;
  p->pResultSet = 0;
}
︙︙
Changes to src/vdbemem.c.
︙︙
757
758
759
760
761
762
763
764
765
766
767
768
769
770
771

**
** This is used for testing and debugging only - to make sure shallow
** copies are not misused.
*/
void sqlite3VdbeMemAboutToChange(Vdbe *pVdbe, Mem *pMem){
  int i;
  Mem *pX;
  for(i=1, pX=&pVdbe->aMem[1]; i<=pVdbe->nMem; i++, pX++){
    if( pX->pScopyFrom==pMem ){
      pX->flags |= MEM_Undefined;
      pX->pScopyFrom = 0;
    }
  }
  pMem->pScopyFrom = 0;
}








|








757
758
759
760
761
762
763
764
765
766
767
768
769
770
771

**
** This is used for testing and debugging only - to make sure shallow
** copies are not misused.
*/
void sqlite3VdbeMemAboutToChange(Vdbe *pVdbe, Mem *pMem){
  int i;
  Mem *pX;
  for(i=0, pX=pVdbe->aMem; i<pVdbe->nMem; i++, pX++){
    if( pX->pScopyFrom==pMem ){
      pX->flags |= MEM_Undefined;
      pX->pScopyFrom = 0;
    }
  }
  pMem->pScopyFrom = 0;
}
︙︙
Changes to test/capi3.test.
︙︙
168
169
170
171
172
173
174

175
176
177
178
179
180
181
182
183
184
185
186
187
188
189

do_test capi3-3.2 {
  sqlite3_close $db2
} {SQLITE_OK}
do_test capi3-3.3 {
  catch {
    set db2 [sqlite3_open /bogus/path/test.db {}]
  }

  sqlite3_extended_errcode $db2
} {SQLITE_CANTOPEN}
do_test capi3-3.4 {
  sqlite3_errmsg $db2
} {unable to open database file}
do_test capi3-3.5 {
  sqlite3_close $db2
} {SQLITE_OK}
if {[clang_sanitize_address]==0} {
  do_test capi3-3.6.1-misuse {
    sqlite3_close $db2
  } {SQLITE_MISUSE}
  do_test capi3-3.6.2-misuse {
    sqlite3_errmsg $db2
  } {library routine called out of sequence}








>
|
|




|
|








168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190

do_test capi3-3.2 {
  sqlite3_close $db2
} {SQLITE_OK}
do_test capi3-3.3 {
  catch {
    set db2 [sqlite3_open /bogus/path/test.db {}]
  }
  set ::capi3_errno [sqlite3_system_errno $db2]
  list [sqlite3_extended_errcode $db2] [expr {$::capi3_errno!=0}]
} {SQLITE_CANTOPEN 1}
do_test capi3-3.4 {
  sqlite3_errmsg $db2
} {unable to open database file}
do_test capi3-3.5 {
  list [sqlite3_system_errno $db2] [sqlite3_close $db2]
} [list $::capi3_errno SQLITE_OK]
if {[clang_sanitize_address]==0} {
  do_test capi3-3.6.1-misuse {
    sqlite3_close $db2
  } {SQLITE_MISUSE}
  do_test capi3-3.6.2-misuse {
    sqlite3_errmsg $db2
  } {library routine called out of sequence}
︙︙
Changes to test/exclusive.test.
︙︙
415
416
417
418
419
420
421
422
423
424
425
426
427
428
429

  }
} {}
do_test exclusive-5.1 {
  # Three files are open: The db, journal and statement-journal.
  # (2016-03-04) The statement-journal is now opened lazily
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
} [expr 2 - ($TEMP_STORE>=2)]
do_test exclusive-5.2 {
  execsql {
    COMMIT;
  }
  # One file open: the db.
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds








|








415
416
417
418
419
420
421
422
423
424
425
426
427
428
429

  }
} {}
do_test exclusive-5.1 {
  # Three files are open: The db, journal and statement-journal.
  # (2016-03-04) The statement-journal is now opened lazily
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
} {2}
do_test exclusive-5.2 {
  execsql {
    COMMIT;
  }
  # One file open: the db.
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
︙︙
442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465

  execsql {
    INSERT INTO abc SELECT a+10, b+10, c+10 FROM abc;
  }
  # Three files are open: The db, journal and statement-journal.
  # 2016-03-04: The statement-journal open is deferred
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
} [expr 2 - ($TEMP_STORE>=2)]
do_test exclusive-5.5 {
  execsql {
    COMMIT;
  }
  # Three files are still open: The db, journal and statement-journal.
  # 2016-03-04: The statement-journal open is deferred
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
} [expr 2 - ($TEMP_STORE>=2)]
do_test exclusive-5.6 {
  execsql {
    PRAGMA locking_mode = normal;
    SELECT * FROM abc;
  }
} {normal 1 2 3 2 3 4 5 6 7 11 12 13 12 13 14 15 16 17}
do_test exclusive-5.7 {








|








|








442
443
444
445
446
447
448
449
450
451
452
453
454
455
456
457
458
459
460
461
462
463
464
465

  execsql {
    INSERT INTO abc SELECT a+10, b+10, c+10 FROM abc;
  }
  # Three files are open: The db, journal and statement-journal.
  # 2016-03-04: The statement-journal open is deferred
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
} {2}
do_test exclusive-5.5 {
  execsql {
    COMMIT;
  }
  # Three files are still open: The db, journal and statement-journal.
  # 2016-03-04: The statement-journal open is deferred
  set sqlite_open_file_count
  expr $sqlite_open_file_count-$extrafds
} {2}
do_test exclusive-5.6 {
  execsql {
    PRAGMA locking_mode = normal;
    SELECT * FROM abc;
  }
} {normal 1 2 3 2 3 4 5 6 7 11 12 13 12 13 14 15 16 17}
do_test exclusive-5.7 {
︙︙
Changes to test/sqldiff1.test.
︙︙
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33

#***********************************************************************
#
# Quick tests for the sqldiff tool
#
set testdir [file dirname $argv0]
source $testdir/tester.tcl

if {$tcl_platform(platform)=="windows"} {
  set PROG "sqldiff.exe"
} else {
  set PROG "./sqldiff"
}
if {![file exe $PROG]} {
  puts "sqldiff cannot run because $PROG is not available"
  finish_test
  return
}
db close
forcedelete test.db test2.db
sqlite3 db test.db

do_test sqldiff-1.0 {
  db eval {
    CREATE TABLE t1(a INTEGER PRIMARY KEY, b);








<
<
<
|
|
<
<
<
<
<








10
11
12
13
14
15
16



17
18





19
20
21
22
23
24
25

#***********************************************************************
#
# Quick tests for the sqldiff tool
#
set testdir [file dirname $argv0]
source $testdir/tester.tcl




set PROG [test_find_sqldiff]






db close
forcedelete test.db test2.db
sqlite3 db test.db

do_test sqldiff-1.0 {
  db eval {
    CREATE TABLE t1(a INTEGER PRIMARY KEY, b);
︙︙
Changes to test/tester.tcl.
︙︙
2173
2174
2175
2176
2177
2178
2179














2180
2181
2182
2183
2184
2185
2186
2187
2188

2189
2190
2191
2192

2193



2194

2195
2196
2197
2198
2199
2200
2201
2202
2203
2204

  sqlite3_shutdown
  eval sqlite3_config_pagecache $::old_pagecache_config
  unset ::old_pagecache_config 
  sqlite3_initialize
  autoinstall_test_functions
  sqlite3 db test.db
}















# Find the name of the 'shell' executable (e.g. "sqlite3.exe") to use for
# the tests in shell[1-5].test. If no such executable can be found, invoke
# [finish_test ; return] in the callers context.
#
proc test_find_cli {} {
  if {$::tcl_platform(platform)=="windows"} {
    set ret "sqlite3.exe"
  } else {

    set ret "sqlite3"
  }
  set ret [file normalize [file join $::cmdlinearg(TESTFIXTURE_HOME) $ret]]
  if {![file executable $ret]} {

    finish_test



    return -code return

  }
  return $ret
}

# If the library is compiled with the SQLITE_DEFAULT_AUTOVACUUM macro set
# to non-zero, then set the global variable $AUTOVACUUM to 1.
set AUTOVACUUM $sqlite_options(default_autovacuum)

# Make sure the FTS enhanced query syntax is disabled.
set sqlite_fts3_enable_parentheses 0








>
>
>
>
>
>
>
>
>
>
>
>
>
>






<
|
<
>
|
|
|
|
>
|
>
>
>
|
>
|
<
|








2173
2174
2175
2176
2177
2178
2179
2180
2181
2182
2183
2184
2185
2186
2187
2188
2189
2190
2191
2192
2193
2194
2195
2196
2197
2198
2199

2200

2201
2202
2203
2204
2205
2206
2207
2208
2209
2210
2211
2212
2213

2214
2215
2216
2217
2218
2219
2220
2221

  sqlite3_shutdown
  eval sqlite3_config_pagecache $::old_pagecache_config
  unset ::old_pagecache_config 
  sqlite3_initialize
  autoinstall_test_functions
  sqlite3 db test.db
}

proc test_find_binary {nm} {
  if {$::tcl_platform(platform)=="windows"} {
    set ret "$nm.exe"
  } else {
    set ret $nm
  }
  set ret [file normalize [file join $::cmdlinearg(TESTFIXTURE_HOME) $ret]]
  if {![file executable $ret]} {
    finish_test
    return ""
  }
  return $ret
}

# Find the name of the 'shell' executable (e.g. "sqlite3.exe") to use for
# the tests in shell[1-5].test. If no such executable can be found, invoke
# [finish_test ; return] in the callers context.
#
proc test_find_cli {} {

  set prog [test_find_binary sqlite3]

  if {$prog==""} { return -code return }
  return $prog
}

# Find the name of the 'sqldiff' executable (e.g. "sqlite3.exe") to use for
# the tests in sqldiff tests. If no such executable can be found, invoke
# [finish_test ; return] in the callers context.
#
proc test_find_sqldiff {} {
  set prog [test_find_binary sqldiff]
  if {$prog==""} { return -code return }
  return $prog
}



# If the library is compiled with the SQLITE_DEFAULT_AUTOVACUUM macro set
# to non-zero, then set the global variable $AUTOVACUUM to 1.
set AUTOVACUUM $sqlite_options(default_autovacuum)

# Make sure the FTS enhanced query syntax is disabled.
set sqlite_fts3_enable_parentheses 0
︙︙
Changes to tool/sqldiff.c.
︙︙
1240
1241
1242
1243
1244
1245
1246

1247
1248
1249
1250
1251
1252
1253

  char **azCol;                   /* NULL terminated array of col names */
  int i;
  int nCol;
  Str ct = {0, 0, 0};             /* The "CREATE TABLE data_xxx" statement */
  Str sql = {0, 0, 0};            /* Query to find differences */
  Str insert = {0, 0, 0};         /* First part of output INSERT statement */
  sqlite3_stmt *pStmt = 0;


  /* --rbu mode must use real primary keys. */
  g.bSchemaPK = 1;

  /* Check that the schemas of the two tables match. Exit early otherwise. */
  checkSchemasMatch(zTab);









>








1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254

  char **azCol;                   /* NULL terminated array of col names */
  int i;
  int nCol;
  Str ct = {0, 0, 0};             /* The "CREATE TABLE data_xxx" statement */
  Str sql = {0, 0, 0};            /* Query to find differences */
  Str insert = {0, 0, 0};         /* First part of output INSERT statement */
  sqlite3_stmt *pStmt = 0;
  int nRow = 0;                   /* Total rows in data_xxx table */

  /* --rbu mode must use real primary keys. */
  g.bSchemaPK = 1;

  /* Check that the schemas of the two tables match. Exit early otherwise. */
  checkSchemasMatch(zTab);
︙︙
1285
1286
1287
1288
1289
1290
1291

1292
1293
1294
1295
1296
1297
1298

    if( ct.z ){
      fprintf(out, "%s\n", ct.z);
      strFree(&ct);
    }

    /* Output the first part of the INSERT statement */
    fprintf(out, "%s", insert.z);


    if( sqlite3_column_type(pStmt, nCol)==SQLITE_INTEGER ){
      for(i=0; i<=nCol; i++){
        if( i>0 ) fprintf(out, ", ");
        printQuoted(out, sqlite3_column_value(pStmt, i));
      }
    }else{








>








1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300

    if( ct.z ){
      fprintf(out, "%s\n", ct.z);
      strFree(&ct);
    }

    /* Output the first part of the INSERT statement */
    fprintf(out, "%s", insert.z);
    nRow++;

    if( sqlite3_column_type(pStmt, nCol)==SQLITE_INTEGER ){
      for(i=0; i<=nCol; i++){
        if( i>0 ) fprintf(out, ", ");
        printQuoted(out, sqlite3_column_value(pStmt, i));
      }
    }else{
︙︙
1338
1339
1340
1341
1342
1343
1344






1345
1346
1347
1348
1349
1350
1351

    }

    /* And the closing bracket of the insert statement */
    fprintf(out, ");\n");
  }

  sqlite3_finalize(pStmt);







  strFree(&ct);
  strFree(&sql);
  strFree(&insert);
}

/*








>
>
>
>
>
>








1340
1341
1342
1343
1344
1345
1346
1347
1348
1349
1350
1351
1352
1353
1354
1355
1356
1357
1358
1359

    }

    /* And the closing bracket of the insert statement */
    fprintf(out, ");\n");
  }

  sqlite3_finalize(pStmt);
  if( nRow>0 ){
    Str cnt = {0, 0, 0};
    strPrintf(&cnt, "INSERT INTO rbu_count VALUES('data_%q', %d);", zTab, nRow);
    fprintf(out, "%s\n", cnt.z);
    strFree(&cnt);
  }

  strFree(&ct);
  strFree(&sql);
  strFree(&insert);
}

/*
︙︙
1852
1853
1854
1855
1856
1857
1858
1859






1860
1861
1862
1863
1864
1865
1866

  }
  rc = sqlite3_exec(g.db, "SELECT * FROM aux.sqlite_master", 0, 0, &zErrMsg);
  if( rc || zErrMsg ){
    cmdlineError("\"%s\" does not appear to be a valid SQLite database", zDb2);
  }

  if( neverUseTransaction ) useTransaction = 0;
  if( useTransaction ) printf("BEGIN TRANSACTION;\n");






  if( zTab ){
    xDiff(zTab, out);
  }else{
    /* Handle tables one by one */
    pStmt = db_prepare(
      "SELECT name FROM main.sqlite_master\n"
      " WHERE type='table' AND sql NOT LIKE 'CREATE VIRTUAL%%'\n"








|
>
>
>
>
>
>








1860
1861
1862
1863
1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880

  }
  rc = sqlite3_exec(g.db, "SELECT * FROM aux.sqlite_master", 0, 0, &zErrMsg);
  if( rc || zErrMsg ){
    cmdlineError("\"%s\" does not appear to be a valid SQLite database", zDb2);
  }

  if( neverUseTransaction ) useTransaction = 0;
  if( useTransaction ) fprintf(out, "BEGIN TRANSACTION;\n");
  if( xDiff==rbudiff_one_table ){
    fprintf(out, "CREATE TABLE IF NOT EXISTS rbu_count"
           "(tbl TEXT PRIMARY KEY COLLATE NOCASE, cnt INTEGER) "
           "WITHOUT ROWID;\n"
    );
  }
  if( zTab ){
    xDiff(zTab, out);
  }else{
    /* Handle tables one by one */
    pStmt = db_prepare(
      "SELECT name FROM main.sqlite_master\n"
      " WHERE type='table' AND sql NOT LIKE 'CREATE VIRTUAL%%'\n"
︙︙