SQLite

Check-in [062ecc1368]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Begin updating the architecture description to better describe how things are currently put together. (CVS 1247)
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 062ecc1368d6bb6d15da31e8d6f5953e8f5628b9
User & Date: drh 2004-02-18 16:56:32.000
Context
2004-02-18
16:57
Fix a memory leak that occurs if you call sqlite_interrupt() on a query using aggregate functions where the aggregate function returns a string longer than 32 characters. (CVS 1248) (check-in: 2c1e74e58a user: drh tags: trunk)
16:56
Begin updating the architecture description to better describe how things are currently put together. (CVS 1247) (check-in: 062ecc1368 user: drh tags: trunk)
01:31
Add more tests of the sqlite_interrupt() logic - looking for a reported memory leak. (Didn't find it.) (CVS 1246) (check-in: e4c8b1c3aa user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to www/arch.tcl. 
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34















35
36
37
38
39


40


41
42
43
44
45
46
47
48
49




50
51
52
53
54
55
56

#
# Run this Tcl script to generate the sqlite.html file.
#
set rcsid {$Id: arch.tcl,v 1.9 2003/03/19 03:14:03 drh Exp $}

puts {<html>
<head>
  <title>Architecture of SQLite</title>
</head>
<body bgcolor=white>
<h1 align=center>
The Architecture Of SQLite
</h1>}
puts "<p align=center>
(This page was last modified on [lrange $rcsid 3 4] UTC)
</p>"

puts {
<h2>Introduction</h2>

<table align="right" border="1" cellpadding="15" cellspacing="1">
<tr><th>Block Diagram Of SQLite</th></tr>
<tr><td><img src="arch.png"></td></tr>
</table>
<p>This document describes the architecture of the SQLite library.
The information here is useful to those who want to understand or
modify the inner workings of SQLite.
</p>

<p>
A block diagram showing the main components of SQLite
and how they interrelate is shown at the right.  The text that
follows will provide a quick overview of each of these components.
</p>
















<h2>Interface</h2>

<p>Most of the public interface to the SQLite library is implemented by
four functions found in the <b>main.c</b> source file.  The


<b>sqlite_get_table()</b> routine is implemented in <b>table.c</b>.


The Tcl interface is implemented by <b>tclsqlite.c</b>.  More
information on the C interface to SQLite is
<a href="c_interface.html">available separately</a>.<p>

<p>To avoid name collisions with other software, all external
symbols in the SQLite library begin with the prefix <b>sqlite</b>.
Those symbols that are intended for external use (in other words,
those symbols which form the API for SQLite) begin
with <b>sqlite_</b>.</p>





<h2>Tokenizer</h2>

<p>When a string containing SQL statements is to be executed, the
interface passes that string to the tokenizer.  The job of the tokenizer
is to break the original string up into tokens and pass those tokens
one by one to the parser.  The tokenizer is hand-coded in C.




|


















|











>
>
>
>
>
>
>
>
>
>
>
>
>
>
>



|
|
>
>

>
>









>
>
>
>








1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79

#
# Run this Tcl script to generate the sqlite.html file.
#
set rcsid {$Id: arch.tcl,v 1.10 2004/02/18 16:56:32 drh Exp $}

puts {<html>
<head>
  <title>Architecture of SQLite</title>
</head>
<body bgcolor=white>
<h1 align=center>
The Architecture Of SQLite
</h1>}
puts "<p align=center>
(This page was last modified on [lrange $rcsid 3 4] UTC)
</p>"

puts {
<h2>Introduction</h2>

<table align="right" border="1" cellpadding="15" cellspacing="1">
<tr><th>Block Diagram Of SQLite</th></tr>
<tr><td><img src="arch2.gif"></td></tr>
</table>
<p>This document describes the architecture of the SQLite library.
The information here is useful to those who want to understand or
modify the inner workings of SQLite.
</p>

<p>
A block diagram showing the main components of SQLite
and how they interrelate is shown at the right.  The text that
follows will provide a quick overview of each of these components.
</p>

<h2>History</h2>

<p>
There are two main C interfaces to the SQLite library:
<b>sqlite_exec()</b> and <b>sqlite_compile()</b>.  Prior to
version 2.8.0 (2003-Feb-16) only sqlite_exec() was supported.
For version 2.8.0, the sqlite_exec and sqlite_compile methods
existed as peers.  Beginning with version 2.8.13, the sqlite_compile
method is the primary interface, and sqlite_exec is implemented
using sqlite_compile.  Externally, there are API extensions but
not changes that break backwards compatibility.  But internally,
the plumbing is very different.  The diagram at the right shows
the structure of SQLite for version 2.8.13 and following.
</p>

<h2>Interface</h2>

<p>Much of the public interface to the SQLite library is implemented by
functions found in the <b>main.c</b> source file though some routines are
scattered about in other files where they can have access to data 
structures with file scope.  The
<b>sqlite_get_table()</b> routine is implemented in <b>table.c</b>.
<b>sqlite_step()</b> is found in <b>vdbe.c</b>.  
<b>sqlite_mprintf()</b> is found in <b>printf.c</b>.
The Tcl interface is implemented by <b>tclsqlite.c</b>.  More
information on the C interface to SQLite is
<a href="c_interface.html">available separately</a>.<p>

<p>To avoid name collisions with other software, all external
symbols in the SQLite library begin with the prefix <b>sqlite</b>.
Those symbols that are intended for external use (in other words,
those symbols which form the API for SQLite) begin
with <b>sqlite_</b>.</p>

<h2>SQL Command Process</h2>

<p>

<h2>Tokenizer</h2>

<p>When a string containing SQL statements is to be executed, the
interface passes that string to the tokenizer.  The job of the tokenizer
is to break the original string up into tokens and pass those tokens
one by one to the parser.  The tokenizer is hand-coded in C.
︙︙
113
114
115
116
117
118
119

120












121
122
123
124
125
126
127

up to three additional operands.</p>

<p>The virtual machine is entirely contained in a single
source file <b>vdbe.c</b>.  The virtual machine also has
its own header file <b>vdbe.h</b> that defines an interface
between the virtual machine and the rest of the SQLite library.</p>


<h2>B-tree Driver</h2>













<p>An SQLite database is maintained on disk using a B-tree implementation
found in the <b>btree.c</b> source file.  A separate B-tree is used for
each table and index in the database.  All B-trees are stored in the
same disk file.  Each page of a B-tree is 1024 bytes in size.  The key
and data for an entry are stored together in an area called "payload".
Up to 236 bytes of payload can be stored on the same page as the B-tree








>
|
>
>
>
>
>
>
>
>
>
>
>
>








136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163

up to three additional operands.</p>

<p>The virtual machine is entirely contained in a single
source file <b>vdbe.c</b>.  The virtual machine also has
its own header file <b>vdbe.h</b> that defines an interface
between the virtual machine and the rest of the SQLite library.</p>

<h2>Backend</h2>

<p>The backend is an abstraction layer that presents a uniform interface
to the virtual machine for either the B-Tree drivers for disk-based
databases or the Red/Black Tree driver for in-memory databases.
The <b>btree.h</b> source file contains the details.</p>

<h2>Red/Black Tree</h2>

<p>In-memory databases are stored in a red/black tree implementation
contain in the <b>btree_rb.c</b> source file.
</p>

<h2>B-Tree</h2>

<p>An SQLite database is maintained on disk using a B-tree implementation
found in the <b>btree.c</b> source file.  A separate B-tree is used for
each table and index in the database.  All B-trees are stored in the
same disk file.  Each page of a B-tree is 1024 bytes in size.  The key
and data for an entry are stored together in an area called "payload".
Up to 236 bytes of payload can be stored on the same page as the B-tree
︙︙
Added www/arch2.fig. 


















































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136

#FIG 3.2
Landscape
Center
Inches
Letter  
100.00
Single
-2
1200 2
0 32 #000000
0 33 #868686
0 34 #dfefd7
0 35 #d7efef
0 36 #efdbef
0 37 #efdbd7
0 38 #e7efcf
0 39 #9e9e9e
6 4950 4275 6525 4800
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 5025 4350 6525 4350 6525 4800 5025 4800 5025 4350
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 4950 4275 6450 4275 6450 4725 4950 4725 4950 4275
4 1 0 50 0 2 12 0.0000 4 135 1335 5700 4575 Red/Black Tree\001
-6
6 5400 5475 6825 6000
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 5400 5475 6750 5475 6750 5925 5400 5925 5400 5475
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 5475 5550 6825 5550 6825 6000 5475 6000 5475 5550
4 1 0 50 0 2 12 0.0000 4 135 630 6000 5775 Utilities\001
-6
6 5400 6300 6825 6825
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 5400 6300 6750 6300 6750 6750 5400 6750 5400 6300
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 5475 6375 6825 6375 6825 6825 5475 6825 5475 6375
4 1 0 50 0 2 12 0.0000 4 135 855 6000 6600 Test Code\001
-6
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 5475 2625 6825 2625 6825 3525 5475 3525 5475 2625
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 5400 2550 6750 2550 6750 3450 5400 3450 5400 2550
2 3 0 1 0 35 55 0 20 0.000 0 0 -1 0 0 9
	 2850 3675 4875 3675 4875 3975 6750 3975 6750 5025 4875 5025
	 4875 6975 2850 6975 2850 3675
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 6300 4575 6300 4575 6750 3225 6750 3225 6300
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 6375 4650 6375 4650 6825 3300 6825 3300 6375
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 5475 4575 5475 4575 5925 3225 5925 3225 5475
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 5550 4650 5550 4650 6000 3300 6000 3300 5550
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 4725 4575 4725 4575 5175 3225 5175 3225 4725
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 4800 4650 4800 4650 5250 3300 5250 3300 4800
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 3900 4575 3900 4575 4350 3225 4350 3225 3900
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 3975 4650 3975 4650 4425 3300 4425 3300 3975
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 5400 1800 6750 1800 6750 2250 5400 2250 5400 1800
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 5475 1875 6825 1875 6825 2325 5475 2325 5475 1875
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 5400 1050 6750 1050 6750 1500 5400 1500 5400 1050
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 5475 1125 6825 1125 6825 1575 5475 1575 5475 1125
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 1050 4575 1050 4575 1500 3225 1500 3225 1050
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 1125 4650 1125 4650 1575 3300 1575 3300 1125
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 1800 4575 1800 4575 2250 3225 2250 3225 1800
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 1875 4650 1875 4650 2325 3300 2325 3300 1875
2 2 0 1 0 7 51 0 20 0.000 0 0 -1 0 0 5
	 3225 2550 4575 2550 4575 3000 3225 3000 3225 2550
2 2 0 0 0 33 52 0 20 0.000 0 0 -1 0 0 5
	 3300 2625 4650 2625 4650 3075 3300 3075 3300 2625
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 3900 1500 3900 1800
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 3900 2250 3900 2550
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 3900 3000 3900 3900
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 4575 1950 5400 1350
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 5400 2925 4650 2175
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 3900 4350 3900 4725
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 4575 4200 4950 4500
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 3900 5175 3900 5475
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 3900 5925 3900 6300
2 2 0 1 0 37 55 0 20 0.000 0 0 -1 0 0 5
	 5175 750 7200 750 7200 3750 5175 3750 5175 750
2 2 0 1 0 34 55 0 20 0.000 0 0 -1 0 0 5
	 2850 750 4875 750 4875 3375 2850 3375 2850 750
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 6075 1500 6075 1800
2 1 0 1 0 38 50 0 -1 0.000 0 0 -1 1 0 2
	1 1 1.00 60.00 120.00
	 6075 2250 6075 2550
2 2 0 1 0 38 55 0 20 0.000 0 0 -1 0 0 5
	 5175 5250 7200 5250 7200 6975 5175 6975 5175 5250
4 1 0 50 0 2 12 0.0000 4 135 855 6075 1350 Tokenizer\001
4 1 0 50 0 2 12 0.0000 4 135 570 6075 2100 Parser\001
4 1 0 50 0 2 12 0.0000 4 135 420 6075 2925 Code\001
4 1 0 50 0 2 12 0.0000 4 135 855 6075 3150 Generator\001
4 1 0 50 0 2 12 0.0000 4 135 720 3900 4200 Backend\001
4 1 0 50 0 2 12 0.0000 4 135 1050 3900 6600 OS Interface\001
4 1 0 50 0 2 12 0.0000 4 135 615 3900 5025 B-Tree\001
4 1 0 50 0 2 12 0.0000 4 180 495 3900 5775 Pager\001
4 1 0 50 0 1 12 1.5708 4 180 1020 7125 2250 SQL Compiler\001
4 1 0 50 0 1 12 1.5708 4 135 885 7125 6150 Accessories\001
4 1 0 50 0 1 12 1.5708 4 135 645 3075 5250 Backend\001
4 1 0 50 0 1 12 1.5708 4 135 345 3075 2025 Core\001
4 1 0 50 0 2 12 0.0000 4 135 1290 3900 2850 Virtual Machine\001
4 1 0 50 0 2 12 0.0000 4 165 1185 3900 1995 SQL Command\001
4 1 0 50 0 2 12 0.0000 4 135 855 3900 2183 Processor\001
4 1 0 50 0 2 14 0.0000 4 150 870 3900 1350 Interface\001
Added www/arch2.gif.

cannot compute difference between binary files