Documentation Source Text

Check-in [c9667def97]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:More spelling errors fixed.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: c9667def97fe9e024c00b17ec8d0566756273f7231b1ee67f9a2f5261574c365
User & Date: drh 2018-03-30 18:41:47.106
Context
2018-04-02
11:10
Version 3.23.0 (check-in: 9f377d4453 user: drh tags: trunk, release, version-3.23.0)
2018-03-30
18:41
More spelling errors fixed. (check-in: c9667def97 user: drh tags: trunk)
18:33
Fix typos discovered by spell-check. (check-in: e436f9d464 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to misc/althttpd.md.
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
about 500,000 HTTP requests per day (about 5 or 6 per second)
delivering about 50GB of content per day (about 4.6 megabits/second) 
on a $40/month [Linode](https://www.linode.com/pricing).  The load 
average on this machine normally stays around 0.1 or 0.2.  About 19%
of the HTTP requests are CGI to various [Fossil](https://fossil-scm.org/)
source-code repositories.

Design Philosopy
----------------

Althttpd was originally designed to be launched from 
[xinetd](https://en.wikipedia.org/wiki/Xinetd) or
[stunnel4](https://www.stunnel.org/).  A separate process
is started for each incoming connection, and that process is
wholly focused on serving that one connection.  A single althttpd







|







9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
about 500,000 HTTP requests per day (about 5 or 6 per second)
delivering about 50GB of content per day (about 4.6 megabits/second) 
on a $40/month [Linode](https://www.linode.com/pricing).  The load 
average on this machine normally stays around 0.1 or 0.2.  About 19%
of the HTTP requests are CGI to various [Fossil](https://fossil-scm.org/)
source-code repositories.

Design Philosophy
----------------

Althttpd was originally designed to be launched from 
[xinetd](https://en.wikipedia.org/wiki/Xinetd) or
[stunnel4](https://www.stunnel.org/).  A separate process
is started for each incoming connection, and that process is
wholly focused on serving that one connection.  A single althttpd
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
extension using a table that is compiled into althttpd.

Security Features
-----------------

To defend against mischief, there are restrictions on names of files that
althttpd will serve.  Within the request URI, all characters other than
alphanumerics and ",-./:~" are converted into a single "_".  Futhermore,
if any path element of the request URI begins with "." or "-" then
althttpd always returns a 404 Not Found error.  Thus is it safe to put
auxiliary files (databases or other content used by CGI, for example)
in the document hierarchy as long as the filenames being with "." or "-".

An exception:  Though althttpd normally returns 404 Not Found for any
request with a path element beginning with ".", it does allow requests







|







166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
extension using a table that is compiled into althttpd.

Security Features
-----------------

To defend against mischief, there are restrictions on names of files that
althttpd will serve.  Within the request URI, all characters other than
alphanumerics and ",-./:~" are converted into a single "_".  Furthermore,
if any path element of the request URI begins with "." or "-" then
althttpd always returns a 404 Not Found error.  Thus is it safe to put
auxiliary files (databases or other content used by CGI, for example)
in the document hierarchy as long as the filenames being with "." or "-".

An exception:  Though althttpd normally returns 404 Not Found for any
request with a path element beginning with ".", it does allow requests
Changes to pages/assert.in.
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
as-delivered configuration, which is obviously impossible if assert(X)
statements are enabled.  For these reasons, assert(X) is a no-op for
release builds in SQLite.

<p>The ALWAYS(X) and NEVER(X) macros behave like assert(X) during
functionality testing, because the developers want to be immediately
alerted to the issue if the value of X is different from what is expected.
But for deliveriy, ALWAYS(X) and NEVER(X) are simple pass-through macros,
which provide defense-in-depth.  For coverage testing ALWAYS(X) and NEVER(X)
are hard-coded boolean values so that they do not cause unreachable
machine code to be generated.

<p>The testcase(X) macro is normally a no-op, but for a coverage test
build it does generate a small amount of extra code that includes at least
one branch, in order to verify that test cases exist for which X is both







|







128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
as-delivered configuration, which is obviously impossible if assert(X)
statements are enabled.  For these reasons, assert(X) is a no-op for
release builds in SQLite.

<p>The ALWAYS(X) and NEVER(X) macros behave like assert(X) during
functionality testing, because the developers want to be immediately
alerted to the issue if the value of X is different from what is expected.
But for delivery, ALWAYS(X) and NEVER(X) are simple pass-through macros,
which provide defense-in-depth.  For coverage testing ALWAYS(X) and NEVER(X)
are hard-coded boolean values so that they do not cause unreachable
machine code to be generated.

<p>The testcase(X) macro is normally a no-op, but for a coverage test
build it does generate a small amount of extra code that includes at least
one branch, in order to verify that test cases exist for which X is both
Changes to pages/qmplan.in.
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
nimble and low-ceremony, and to that end, much of the required
DO-178B documentation is omitted.  We retain only those parts that
genuinely improve quality for a open-source software project such
as SQLite.

<p>
The purpose of this document is to brief the reader on how
SQLite development team functions on a daily basis, as they continuely
enhance the SQLite software and work to improve its already high reliability.
The document achieves its purpose if a competent developer can be
assimilated into the development team quickly after perusing this
document.

<h2>About This Document</h2>

<p>
The quality management plan was originally composed by going through
the description of outputs in section 11 of DO-178B (pages 48 through 56)
and writing down those elements that seemed relevant to SQLite.
The text will be subsequent revised to track enhancements to the
SQLite quality process.

<h1>Software Development Plan</h1>

<p>
This section is a coalescense of the Plan For Software Aspects Of
Certification and the Software Development Plan sections of DO-178B.


<p>
See <a href='about.html'>About SQLite</a> for an overview of the
SQLite software and what it does and how it is different.








|

















|







27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
nimble and low-ceremony, and to that end, much of the required
DO-178B documentation is omitted.  We retain only those parts that
genuinely improve quality for a open-source software project such
as SQLite.

<p>
The purpose of this document is to brief the reader on how
SQLite development team functions on a daily basis, as they continuosly
enhance the SQLite software and work to improve its already high reliability.
The document achieves its purpose if a competent developer can be
assimilated into the development team quickly after perusing this
document.

<h2>About This Document</h2>

<p>
The quality management plan was originally composed by going through
the description of outputs in section 11 of DO-178B (pages 48 through 56)
and writing down those elements that seemed relevant to SQLite.
The text will be subsequent revised to track enhancements to the
SQLite quality process.

<h1>Software Development Plan</h1>

<p>
This section is a combination of the Plan For Software Aspects Of
Certification and the Software Development Plan sections of DO-178B.


<p>
See <a href='about.html'>About SQLite</a> for an overview of the
SQLite software and what it does and how it is different.

237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
<p>The SQLite code is broken up into multiple repositories, each described
in a separate section below.

<h3>SQLite Source Code</h3>

<p>The SQLite source code and the [TCL test suite] are stored together
in a single repository.  This one repository is all that is required to
build the SQLite.  The source respository is public and is
readable by anonymous passers by on the internet.

<ul>
<li> Primary location: [https://www.sqlite.org/src]
<li> Backup A: [https://www2.sqlite.org/src]
<li> Backup B: [https://www3.sqlite.org/src]
</ul>







|







237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
<p>The SQLite code is broken up into multiple repositories, each described
in a separate section below.

<h3>SQLite Source Code</h3>

<p>The SQLite source code and the [TCL test suite] are stored together
in a single repository.  This one repository is all that is required to
build the SQLite.  The source repository is public and is
readable by anonymous passers by on the internet.

<ul>
<li> Primary location: [https://www.sqlite.org/src]
<li> Backup A: [https://www2.sqlite.org/src]
<li> Backup B: [https://www3.sqlite.org/src]
</ul>
288
289
290
291
292
293
294
295
296
297
298
299
300
301
302

<h3>Test Harness #3</h3>

<p>
The [TH3|Test Harness #3] or [TH3] test suite is a private set of
test cases used to test SQLite to 100% MC/DC in an as-delivered
configuration.  TH3 sources are served on the same servers as the
other SQLite respositories, but differ from the others in being
proprietary.  The TH3 code is only accessible to SQLite developers.


<ul>
<li> Primary location: [https://www.sqlite.org/th3]
<li> Backup A: [https://www2.sqlite.org/th3]
<li> Backup B: [https://www3.sqlite.org/th3]







|







288
289
290
291
292
293
294
295
296
297
298
299
300
301
302

<h3>Test Harness #3</h3>

<p>
The [TH3|Test Harness #3] or [TH3] test suite is a private set of
test cases used to test SQLite to 100% MC/DC in an as-delivered
configuration.  TH3 sources are served on the same servers as the
other SQLite repositories, but differ from the others in being
proprietary.  The TH3 code is only accessible to SQLite developers.


<ul>
<li> Primary location: [https://www.sqlite.org/th3]
<li> Backup A: [https://www2.sqlite.org/th3]
<li> Backup B: [https://www3.sqlite.org/th3]
Changes to pages/sqlar.in.
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
know what a ZIP Archive or Tarball is, but if you say "SQLite Archive" you
are more likely to get a reply of "What?"  Tooling to process ZIP Archives
and Tarballs is more likely to be installed on stock computers.

<li><p>
Since an SQLite database is a more general format (it is designed to do
much more than simply store a bunch of files) it is not as compact as either
the ZIP Archive or Tarball formats.  An SQLite Achive is usually about 1%
larger than the equivalent ZIP Archive.  Tarballs are compressed as a single
unit rather than compressing each file separately as is done by both
SQLite and ZIP Archives.  For these reason, Tarballs tend to be smaller
than either ZIP or SQLite Archives.
<p>
As an example, the following table show the relative sizes for an
SQLite Archive, a ZIP Archive, and a Tarball of the 1,743 files







|







133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
know what a ZIP Archive or Tarball is, but if you say "SQLite Archive" you
are more likely to get a reply of "What?"  Tooling to process ZIP Archives
and Tarballs is more likely to be installed on stock computers.

<li><p>
Since an SQLite database is a more general format (it is designed to do
much more than simply store a bunch of files) it is not as compact as either
the ZIP Archive or Tarball formats.  An SQLite Archive is usually about 1%
larger than the equivalent ZIP Archive.  Tarballs are compressed as a single
unit rather than compressing each file separately as is done by both
SQLite and ZIP Archives.  For these reason, Tarballs tend to be smaller
than either ZIP or SQLite Archives.
<p>
As an example, the following table show the relative sizes for an
SQLite Archive, a ZIP Archive, and a Tarball of the 1,743 files
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
<p>
The recommended way of creating, updating, listing, and extracting
an SQLite Archive is to use the [sqlite3.exe command-line shell] 
for SQLite [version 3.23.0] ([dateof:3.23.0]) or later.  This CLI
supports the -A command-line option that allows easy management
of SQLite Archives.
The CLI for SQLite [version 3.22.0] ([dateof:3.22.0]) has the
[.archive command] for manageing SQLite Archives, but that requires
interacting with the shell.

<p>
To list all of the files in an SQLite Archive named "example.sqlar"
using one of these commands:

<codeblock>







|







165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
<p>
The recommended way of creating, updating, listing, and extracting
an SQLite Archive is to use the [sqlite3.exe command-line shell] 
for SQLite [version 3.23.0] ([dateof:3.23.0]) or later.  This CLI
supports the -A command-line option that allows easy management
of SQLite Archives.
The CLI for SQLite [version 3.22.0] ([dateof:3.22.0]) has the
[.archive command] for managing SQLite Archives, but that requires
interacting with the shell.

<p>
To list all of the files in an SQLite Archive named "example.sqlar"
using one of these commands:

<codeblock>
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
creates two new SQL functions.

<dl>
<dt><b>sqlar_compress(X)</b></dt>
<dd><p>
The sqlar_compress(X) function attempts to compress a copy of the
string or blob X using the [https://zlib.net/|Default] algorithm and
returns the result as a blob.  If the input X is uncompressible, then
a copy of X is returned.  This routine is used when inserting content
into an SQLite Archive.
<dt><b>sqlar_uncompress(Y,SZ)</b></dt>
<dd><p>
The sqlar_uncompress(Y,SZ) function will undo the compression accomplishd
by sqlar_compress(X).  The Y parameter is the compressed content (the output
from a prior call to sqlar_compress()) and SZ is the original uncompressed
size of the input X that generated Y.  If SZ is less than or equal to the
size of Y, that indicates that no compression occurred, and so
sqlar_uncompress(Y,SZ) returns a copy of Y.  Otherwise, sqlar_uncompress(Y,SZ)
runs the Inflate algorithm on Y to uncompress it and restore it to its
original form and returns the uncompressed content.







|




|







226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
creates two new SQL functions.

<dl>
<dt><b>sqlar_compress(X)</b></dt>
<dd><p>
The sqlar_compress(X) function attempts to compress a copy of the
string or blob X using the [https://zlib.net/|Default] algorithm and
returns the result as a blob.  If the input X is incompressible, then
a copy of X is returned.  This routine is used when inserting content
into an SQLite Archive.
<dt><b>sqlar_uncompress(Y,SZ)</b></dt>
<dd><p>
The sqlar_uncompress(Y,SZ) function will undo the compression accomplished
by sqlar_compress(X).  The Y parameter is the compressed content (the output
from a prior call to sqlar_compress()) and SZ is the original uncompressed
size of the input X that generated Y.  If SZ is less than or equal to the
size of Y, that indicates that no compression occurred, and so
sqlar_uncompress(Y,SZ) returns a copy of Y.  Otherwise, sqlar_uncompress(Y,SZ)
runs the Inflate algorithm on Y to uncompress it and restore it to its
original form and returns the uncompressed content.