<title>Appropriate Uses For SQLite</title>
<h2>Appropriate Uses For SQLite</h2>
<p>
SQLite is not directly comparable to other SQL database engines such
as Oracle, PostgreSQL, MySQL, or SQL Server since SQLite is trying to
solve a very different problem.
</p>
<p>
Other SQL database engines strive to implement a shared repository
of enterprise data. They emphasis scalability, concurrency, centralization,
and control.
</p>
<p>
SQLite, on the other hand, strives to provide local data storage for
individual applications and devices. SQLite emphasizes economy,
efficiency, reliability, independence, and simplicity.
</p>
<p>
SQLite is not designed to compete with
[http://www.oracle.com/database/index.html | Oracle].
SQLite is designed to compete with [http://man.he.net/man3/fopen | fopen()].
</p>
<h2>Situations Where SQLite Works Well</h2>
<ul>
<tcl>hd_fragment appfileformat</tcl>
<li><p><b>Application File Format</b></p>
<p>(Further details [application file-format | here].)</p>
<p>
SQLite has been used with great success as the on-disk file format
for desktop applications such as version control systems,
financial analysis tools, media cataloging and editing suites, CAD
packages, record keeping programs, and so forth. The traditional
File/Open operation calls sqlite3_open() to attach to the database
file. Updates happen automatically as application content is revised
so the File/Save menu option becomes superfluous. The File/Save_As
menu option can be implemented using the [backup API].
</p>
<p>
There are many advantages to using SQLite as an application file format,
including:
</p>
<ol type="1">
<li> There is no file parsing and generating code to write and debug.
<li> Content can be accessed and updated using powerful SQL queries,
greatly reducing the complexity of the application code.
<li> Extending the file format for new capabilities in later releases
is a simple as adding new tables or new columns to existing tables.
<li> Diverse content which might otherwise be stored as a "pile-of-files"
can be encapsulated into a single disk file.
<li> The content can be viewed using third-party tools.
<li> The application file is portable across all operating systems,
32-bit and 64-bit and big- and little-endian architectures.
<li> The application only has to load as much data as it needs, rather
than reading the entire application file and holding a complete parse
in memory. Startup time and memory consumption are reduced.
<li> Small edits only overwrite the parts of the file that change,
not the entire file, thus improving performance
and reducing wear on SSD drives.
<li> Content is updated continuously and atomically so
that there is no work lost in the event of a power failure or crash.
<li> Applications can leverage the
[full-text search] and [RTREE] capabilities that are built into SQLite.
<li> Performance problems can often be resolved using [CREATE INDEX]
rather than redesigning, rewriting, and retesting application code.
<li> A federation of programs, perhaps written in different programming
languages, can all access the same application file with no
compatibility concerns.
<li> Multiple processes can attach to the same application
file and can read and write without interfering with each another.
<li> Cross-session undo/redo can be implemented using triggers.
<li> In many common cases, loading content from an SQLite database
is faster than loading content out of individual files.
See [Internal Versus External BLOBs] for additional information.
<li> Content stored in an SQLite database is more likely to be
recoverable decades in the future, long after all traces of
the original application have been lost. Data lives longer than code.
</ol>
<p>
SQLite allows database files to have any desired filename extension, so
an application can choose a custom filename extension for its own use, if
desired. The [application_id pragma] can be used to set an "Application ID"
integer in the database file so that tools like
[http://www.darwinsys.com/file/ | file(1)] can determine that the file
is associated with your application and is not just a generic
SQL database.</p>
<li><p><b>Embedded devices and applications</b></p>
<p>Because an SQLite database requires little or no administration,
SQLite is a good choice for devices or services that must work
unattended and without human support. SQLite is a good fit for
use in cellphones, PDAs, set-top boxes, and/or appliances. It also
works well as an embedded database in downloadable consumer applications.
</p>
</li>
<li><p><b>Websites</b></p>
<p>SQLite usually will work great as the database engine for low to
medium traffic websites (which is to say, 99.9% of all websites).
The amount of web traffic that SQLite can handle depends, of course,
on how heavily the website uses its database. Generally
speaking, any site that gets fewer than 100K hits/day should work
fine with SQLite.
The 100K hits/day figure is a conservative estimate, not a
hard upper bound.
SQLite has been demonstrated to work with 10 times that amount
of traffic.</p>
</li>
<li><p><b>Replacement for <i>ad hoc</i> disk files</b></p>
<p>Many programs use
[http://man.he.net/man3/fopen | fopen()],
[http://man.he.net/man3/fread | fread()], and
[http://man.he.net/man3/fwrite | fwrite()] to create and
manage files of data in home-grown formats. SQLite works
particularly well as a
replacement for these <i>ad hoc</i> data files.</p>
</li>
<li><p><b>Internal or temporary databases</b></p>
<p>
For programs that have a lot of data that must be sifted and sorted
in diverse ways, it is often easier and quicker to load the data into
an in-memory SQLite database and use queries with joins and ORDER BY
clauses to extract the data in the form and order needed rather than
to try to code the same operations manually.
Using an SQL database internally in this way also gives the program
greater flexibility since new columns and indices can be added without
having to recode every query.
</p>
</li>
<li><p><b>Command-line dataset analysis tool</b></p>
<p>
Experienced SQL users can employ
the command-line <b>sqlite3</b> program to analyze miscellaneous
datasets. Raw data can be imported from CSV files, then that
data can be sliced and diced to generate a myriad of summary
reports. Possible uses include website log analysis, sports
statistics analysis, compilation of programming metrics, and
analysis of experimental results.
</p>
<p>
You can also do the same thing with an enterprise client/server
database, of course. The advantages to using SQLite in this situation
are that SQLite is much easier to set up and the resulting database
is a single file that you can store on a floppy disk or flash-memory stick
or email to a colleague.
</p>
</li>
<li><p><b>Stand-in for an enterprise database during demos or testing</b></p>
<p>
If you are writing a client application for an enterprise database engine,
it makes sense to use a generic database backend that allows you to connect
to many different kinds of SQL database engines. It makes even better
sense to
go ahead and include SQLite in the mix of supported databases and to statically
link the SQLite engine in with the client. That way the client program
can be used standalone with an SQLite data file for testing or for
demonstrations.
</p>
</li>
<li><p><b>Database Pedagogy</b></p>
<p>
Because it is simple to setup and use (installation is trivial: just
copy the <b>sqlite3</b> or <b>sqlite3.exe</b> executable to the target machine
and run it) SQLite makes a good database engine for use in teaching SQL.
Students can easily create as many databases as they like and can
email databases to the instructor for comments or grading. For more
advanced students who are interested in studying how an RDBMS is
implemented, the modular and well-commented and documented SQLite code
can serve as a good basis. This is not to say that SQLite is an accurate
model of how other database engines are implemented, but rather a student who
understands how SQLite works can more quickly comprehend the operational
principles of other systems.
</p>
</li>
<li><p><b>Experimental SQL language extensions</b></p>
<p>The simple, modular design of SQLite makes it a good platform for
prototyping new, experimental database language features or ideas.
</p>
</li>
</ul>
<h2>Situations Where Another RDBMS May Work Better</h2>
<ul>
<li><p><b>Client/Server Applications</b><p>
<p>If you have many client programs accessing a common database
over a network, you should consider using a client/server database
engine instead of SQLite. SQLite will work over a network filesystem,
but because of the latency associated with most network filesystems,
performance will not be great. Also, the file locking logic of
many network filesystems implementation contains bugs (on both Unix
and Windows). If file locking does not work like it should,
it might be possible for two or more client programs to modify the
same part of the same database at the same time, resulting in
database corruption. Because this problem results from bugs in
the underlying filesystem implementation, there is nothing SQLite
can do to prevent it.</p>
<p>A good rule of thumb is that you should avoid using SQLite
in situations where the same database will be accessed simultaneously
from many computers over a network filesystem.</p>
</li>
<li><p><b>High-volume Websites</b></p>
<p>SQLite will normally work fine as the database backend to a website.
But if you website is so busy that you are thinking of splitting the
database component off onto a separate machine, then you should
definitely consider using an enterprise-class client/server database
engine instead of SQLite.</p>
</li>
<li><p><b>Very large datasets</b></p>
<p>An SQLite database is limited in size to 140 terabytes
(2<sup><small>47</small></sup> bytes, 128 tibibytes).
And even if it could handle larger databases, SQLite stores the entire
database in a single disk file and many filesystems limit the maximum
size of files to something less than this. So if you are contemplating
databases of this magnitude, you would do well to consider using a
client/server database engine that spreads its content across multiple
disk files, and perhaps across multiple volumes.
</p>
</li>
<li><p><b>High Concurrency</b></p>
<p>
SQLite supports an unlimited number of simultaneous readers, but it
will only allow one writer at any instant in time.
For many situations, this is not a problem. Each application
does its database work quickly and moves on, and no lock lasts for more
than a few dozen milliseconds. But there are some applications that require
more concurrency, and those applications may need to seek a different
solution.
</p>
</li>
</ul>