Documentation Source Text

<title>Quality Management</title>
<table_of_contents>

<center>
<font color="red" size="5"><b>
This document is a work-in-progress.  It is incomplete and unverified.
</b></font>
</center>

<h1>Overview</h1>

<p>
This is the Quality Management Plan for SQLite.

<p>
Quality management documents tend to expand into
binders full of incomprehensible jargon that nobody
reads.  This document strives to break that pattern by
being concise and useful.

<p>
The inspiration for this document is
[https://en.wikipedia.org/wiki/DO-178B|DO-178B].
Among quality standards, DO-178B seems to have the highest usefulness
to paperwork ratio.  Even so, the amount of documentation needed
for a full-up DO-178B implementation is vast.  SQLite strives to be
nimble and low-ceremony, and to that end, much of the required
DO-178B documentation is omitted.  We retain only those parts that
genuinely improve quality for a open-source software project such
as SQLite.

<p>
The purpose of this document is to brief the reader on how
SQLite development team functions on a daily basis, as they continuely
enhance the SQLite software and work to improve its already high reliability.
The document achieves its purpose if a competent developer can be
assimilated into the development team quickly after perusing this
document.

<h2>About This Document</h2>

<p>
The quality management plan was originally composed by going through
the description of outputs in section 11 of DO-178B (pages 48 through 56)
and writing down those elements that seemed relevant to SQLite.
The text will be subsequent revised to track enhancements to the
SQLite quality process.

<h1>Software Development Plan</h1>

<p>
This section is a coalescense of the Plan For Software Aspects Of
Certification and the Software Development Plan sections of DO-178B.


<p>
See <a href='about.html'>About SQLite</a> for an overview of the
SQLite software and what it does and how it is different.

<h2>Software Life Cycle</h2>

<p>
SQLite uses a continuous integration process.  The software
is under constant enhancement and refinement.  The latest trunk
check-ins are frequently used internally for mission-critical
operations. 

<p>
There is no pre-defined release cycle.  Releases occur
when there is a critical mass of feature enhancements and/or
bug fixes.  Historically, releases have occurred about 5 or 6
times per year.
Users of SQLite pick up new releases from the website on an
as-needed basis.

<h3>Maintenance Releases</h3>

<p>
Routine maintenance releases of SQLite contain feature enhancements,
performance enhancements, and/or fixes for non-critical issues.
The version number for major releases are of the form "3.N.0"
for some integer N.  See the [version numbering conventions] document
for details.

<p>
Upcoming maintenance releases announced on the sqlite-users and
sqlite-dev [mailing lists] about two weeks prior to the anticipated
release.  Approximately one week prior to release, the lead developer
declares "pencils down" after which only bug-fix check-ins are
allowed on trunk.  A new [https://sqlite.org/checklists|release checklist]
is created and updated as needed.  As items of the checklist are 
verified, they are checked off and turn green.  The release occurs
when all elements of the checklist are green.  That process normally
takes about a week.

<h3>Patch Releases</h3>

<p>
Occasionally, a serious problem is found and a small "patch" release
must be made against a regular maintenance release.  Patch are distinct
from maintenance releases in that the number of lines of code changed
from the previous release is very small.  Every effort is made to avoid
patch releases by making sure that maintenance releases are bug free.

<p>
Patch releases may or may not have a release checklist, depending on the
issue.  This is a judgement call by the project leader.

<h2>Release History</h2>

<p>The documentation system automatically maintains a
[chronology] of past releases, as well as a
[complete list of SQLite releases] with change summaries.

<h2>Schedule</h2>

SQLite has a long-range vision.
Planning is done with the assumption that SQLite
will be used and supported through at least the year 2050.
All code is written with the idea that it will one day be read and
maintained by people not yet born.  The code is carefully commented
with an eye toward helping those future developers more easily 
understand the logic and the rationale behind the code.

<h1>Software Development Environment</h1>

<p>
SQLite is written in portable C code.
Development work occurs on a mix of Linux, Mac, and Windows workstations.
The developers use command-line tools and eschew integrated development
environments (IDEs) whenever possible.  All developers are expected to be
fluent with the unix command-line.

<p>
A minimum setup for compiling and testing SQLite from canonical
sources is as follows:

<ul>
<li> A host computer with a 32-bit or 64-bit address space.
     The OS can be Linux, Mac, Windows, *BSD, Solaris, or some other.
<li> A C99 compiler such as GCC (including MinGW variants for Windows),
     Clang, or MSVC
<li> A text editor of the user's choice supporting UTF-8 text.
<li> [https://core.tcl.tk/|Tcl] version 8.6 or later.
<li> The "make" utility, or optionally "nmake" on Windows.
</ul>

<p>
The Tcl script language is used to help translate canonical source code
into the [amalgamation] and to manage testing.  Tcl is not used directly
by SQLite itself (unless requested by a compile-time option).  End users
of the SQLite amalgamation sources do not need Tcl.

<p>
When building the [CLI], it is helpful, but not required, to have
the following third-party libraries on hand:

<ul>
<li> [https://zlib.net/|zLib]
<li> [http://git.savannah.gnu.org/cgit/readline.git?h=devel|readline]
     or [http://thrysoee.dk/editline/|editline]
     or [https://github.com/antirez/linenoise|linenoise] for
     command-line editing.
</ul>

<p>
A complete release-test of SQLite requires additional software,

<ul>
<li> [http://www.valgrind.org/|valgrind]
<li> [https://gcc.gnu.org/onlinedocs/gcc/Gcov.html|gcov]
</ul>

<p>
SQLite is expected to operate the same, and use exactly the same 
[file format|on-disk format],
on all modern operating systems, on all modern computer architectures,
and using all modern C compilers.  The developers are constantly testing
SQLite on as many diverse platforms as they can get their hands on.

<h1>Software Verification Plan</h1>

<p>The testing process for SQLite is described in the [testing] document.
Testing objectives include:

<ul>
<li> 100% MC/DC in an as-delivered configuration
<li> Testing of both source code and object code
<li> Testing on multiple platforms and with multiple compilers
<li> Code change inspection
<li> Dynamic and static analysis of the code
</ul>

<p>The testing process is controlled by the
[release testing checklists].  The checklists succinctly summary
all steps necessary to fully validate SQLite, and they record when
and by whom each validation step was performed.

<p>The set of checklist items for release checklist is potentially
updated for each release.  The content and complete
history of each release checklist are retained for the historical
record.

<h1>Software Configuration Management</h1>

<h2>Version Control</h2>

<p>
SQLite source code is managed using the [https://fossil-scm.org|Fossil]
version control system.  Fossil was written specifically to support
SQLite development.  Fossil provides both distributed version control
and issue tracking.

<h2>Survivability</h2>

<p>
All code is archived on three separate machines:
[https://www.sqlite.org], [https://www2.sqlite.org], [https://www3.sqlite.org].
These machines are located in different cities (Dallas, Newark, and
San Francisco, respectively) and managed by two different hosting
companies ([https://linode.com|Linode] for the first two and
[https://digitalocean.com|Digital Ocean] for the third).
This diversity is intended to avoid a single point of failure.

<p>
The main machine in Dallas [https://www.sqlite.org/] is the primary
server and the one that most people use.  The other two are considered
backups.

<p>
In addition to the official repositories, the developers typically
keep complete clones of all software on their personal machines.
And there are uncountable clones scattered about the internet.

<h2>Repositories</h2>

<p>The SQLite code is broken up into multiple repositories, each described
in a separate section below.

<h3>SQLite Source Code</h3>

<p>The SQLite source code and the [TCL test suite] are stored together
in a single repository.  This one repository is all that is required to
build the SQLite.  The source respository is public and is
readable by anonymous passers by on the internet.

<ul>
<li> Primary location: [https://www.sqlite.org/src]
<li> Backup A: [https://www2.sqlite.org/src]
<li> Backup B: [https://www3.sqlite.org/src]
</ul>

<p>There is an unofficial and unsanctioned Git clone of this repository
at [https://github.com/mackyle/sqlite].

<h3>SQLite Documentation Sources</h3>

<p>The documentation sources include documentation text and images with the
scripts and makefile needed to construct the SQLite website documentation.
This document is contained within the documentation sources.  The
document sources are kept in a separate repository distinct from the
source code.  The documentation sources repository is publicly readable.

<p>The makefiles and scripts used to generate the documentation gather
text from baseline documents in the documentation source repository.
Additional text is extracted from comments in the SQLite source code.
Requirements coverage information is extract from special comments in the
[TCL test suite] which is part of the source repository, and from
comments in the [TH3] test suite which is a separate private repository.

<ul>
<li> Primary location: [https://www.sqlite.org/docsrc]
<li> Backup A: [https://www2.sqlite.org/docsrc]
<li> Backup B: [https://www3.sqlite.org/docsrc]
</ul>

<h3>SQL Logic Test</h3>

<p>
The [SQL Logic Tests] are a set of test cases designed to show that
SQLite behaves the same as other SQL database engines.  These tests
are hosted in a separate code public repository.

<ul>
<li> Primary location: [https://www.sqlite.org/sqllogictest]
<li> Backups on private servers
</ul>

<h3>Test Harness #3</h3>

<p>
The [TH3|Test Harness #3] or [TH3] test suite is a private set of
test cases used to test SQLite to 100% MC/DC in an as-delivered
configuration.  TH3 sources are served on the same servers as the
other SQLite respositories, but differ from the others in being
proprietary.  The TH3 code is only accessible to SQLite developers.


<ul>
<li> Primary location: [https://www.sqlite.org/th3]
<li> Backup A: [https://www2.sqlite.org/th3]
<li> Backup B: [https://www3.sqlite.org/th3]
</ul>

<h3>Test Harness #3 Private Extensions</h3>

<p>
At one point, [TH3] was sometimes licensed to third-parties.
Such licensing no longer occurs.  However, back when it was
occurring, some of the TH3 test cases contained information that
was sensitive and could not be released even to licensees.  This
sensitive information is stored in yet another repository.

<ul>
<li> Primary location: [https://www.sqlite.org/th3private]
<li> Backups on private servers
</ul>

<h2>Software Verification Results</h2>

<p>
Release testing proceeds by [checklist].  The current status and
complete change history for each checklist is stored in a separate
SQLite database file.  These files are not version controlled, but
separate copies are maintained on private backup servers.

<p>The source code to the software that runs the checklists is stored
in its own Fossil repository at [https://www.sqlite.org/checklistapp].

<h1>Software Requirements Standards And Data</h1>

<p><i>TBD...</i>

<h1>Software Design And Coding Standands</h1>

<p><i>TBD...</i>

<h1>Problem Reports</h1>

<p><i>TBD...</i>

Those test cases will appear to fail when optimizations are disabled.
But the majority of test cases simply check that the correct answer
was obtained, and all of those cases can be run successfully with and
without the optimizations, in order to show that the optimizations do not
cause malfunctions.</p>


<tcl>hd_fragment cklist</tcl>
<h1>Checklists</h1>

<p>The SQLite developers use an on-line checklist to coordinate testing
activity and to verify that all tests pass prior each SQLite release.
<a href="http://www.sqlite.org/checklists/index.html">Past checklists</a>
are retained for historical reference.
(The checklists are read-only for anonymous internet viewers, but

Those test cases will appear to fail when optimizations are disabled.
But the majority of test cases simply check that the correct answer
was obtained, and all of those cases can be run successfully with and
without the optimizations, in order to show that the optimizations do not
cause malfunctions.</p>


<tcl>hd_fragment cklist {release testing checklists} {checklist}</tcl>
<h1>Checklists</h1>

<p>The SQLite developers use an on-line checklist to coordinate testing
activity and to verify that all tests pass prior each SQLite release.
<a href="http://www.sqlite.org/checklists/index.html">Past checklists</a>
are retained for historical reference.
(The checklists are read-only for anonymous internet viewers, but