Documentation Source Text

<title>Compilation Options For SQLite</title>
<tcl>hd_keywords {compile-time options}</tcl>

<h1>1.0 Compilation Options For SQLite</h1>

<p>
For most purposes, SQLite can be built just fine using the default
compilation options. However, if required, the compile-time options
documented below can be used to 

<title>Private Branches Of SQLite</title>
<tcl>hd_keywords {private branches}</tcl>

<h1 align="center">
Maintaining Private Branches Of SQLite
</h1>

<h2>1.0 Introduction</h2>

<p>SQLite is designed to meet most developer's needs without any 
changes or customization.  When changes are needed, they can normally
be accomplished using start-time [sqlite3_config | (1)]
or runtime
[sqlite3_db_config | (2)]
[sqlite3_limit | (3)]
[sqlite3_vfs_register | (4)] configuration methods
or via [compile-time options].  It is very rare that an application 
developer will need to edit the SQLite source code in order to 
incorporate SQLite into their product.<p>

<p>We call custom modifications to the SQLite source code that are held
for the use of a single application a "private branch".  When a private
branch becomes necessary, the application developer must take on the
task of keeping the private branch in synchronization with the public
SQLite sources.  This is tedious.  It can also be tricky, since while
the SQLite file format and published interfaces are very stable, the
internal implementation of SQLite changes quite rapidly.  Hundreds or
thousands of lines of code might change for any given SQLite point release.
</p>

<p>This article outlines one possible method for keeping a private branch
of SQLite in sync with the public SQLite source code.
There are many ways of maintaining a private branch, of course.
Nobody is compelled to use the method describe here.
This article is not trying to impose a particular procedure on 
maintainers of private branches.  The point of this article is to offer
an example of one process for maintaining a private branch which can
be used as a baseline for designing processes best suited for the
circumstances of each individual project.</p>

<img src="images/private_branch.gif" align="right">
<h1>2.0 The Basic Idea</h1>


<p>We propose to set up the
<a href="http://www.fossil-scm.org">fossil software configuration management</a>
system to set up two branches.  One branch (hereafter the "public branch")
contains the published SQLite sources and the other branch is the 
private branch which contains the code that is customized for the project.
Whenever a new public release of SQLite is made, that release is added to
the public branch and then the changes are merged into the private branch.</p>

<p>This document proposed to use
<a href="http://www.fossil-scm.org/">fossil</a>,
but any other distributed software configuration management system such as
<a href="http://www.monotone.ca/">monotone</a> or
<a href="http://www.selenic.com/mercurial/wiki/">mercurial</a> (a.k.a. "hg"), or
<a href="http://www.git-scm.org/">git</a> could do the problem just as well.
The concept will be the same, the specifics of the procedure will vary.</p>

<p>The diagram at the right illustrates the concept.
One begins with a standard SQLite release.  For the
sake of example, suppose that one intends to create a
private branch off of SQLite version 3.6.15.  In the
diagram this is version "1".  The
maintainer makes an exact copy of the the baseline
SQLite into the branch space, shown as version "2".
Note that "1" and "2" are exactly the same.  Then
the maintainer applies the private changes to 
version "2" resulting in version "3".  In other words,
version "3" is SQLite version 3.6.15 plus edits.</p>

<p>Later, SQLite version 3.6.15 is released, as shown 
by circle "4" in the diagram.  At the point, the private
branch maintainer does a merge which copies all of the
changes from "1" to "4" and applies those changes to
"3".  The result is version "5", which is SQLite 3.6.12
plus edits.</p>

<p>There might be merge conflicts.  In other words, it might
be that the changes from "2" to "3" are incompatible with the
changes from "1" to "4".  In that case, the maintainer will
have to manually resolve the conflicts.  Hopefully conflicts
will not come up that often.</p>

<p>The cycle above can be repeated many times.  The
diagram shows a third SQLite release, 3.6.17 in
circle "6".  The private branch maintain can do
another merge in order to incorporate the changes
moving from "4" to "6" into the private branch, resulting
in version "7".</p>

<h1>3.0 The Procedure</h1>

<p>The following steps explain how to maintain a private branch as
outlines aboive using <a href="http://www.fossil-scm.org/">fossil</a>.
We emphasize again that these steps are not intended to be the one
true and righteous method for maintaining private branch.  This approach
is one of many.  Use this document as a baseline for preparing
project-specific procedures.  Do not be afraid to experiment.</p>

<h2>3.1 Obtain The Software</h2>

<p><a href="http://www.fossil-scm.org/">Fossil</a> is a computer program
that must be installed on your machine before you use it.
Fortunately, installing fossil is very, very easy.  Fossil is a single
"*.exe" file that you simply download and run.  To uninstall fossil,
simply delete the exe file.  
<a href="http://www.fossil-scm.org/index.html/doc/tip/www/quickstart.wiki">
Detailed instructions</a> for installing and getting started with
fossil are available on the 
<a href="http://www.fossil-scm.org">fossil website</a>.</p>

<h2>3.2 Create A Project Repository</h2>

<p>Create a fossil repository to host the private branch using the
following command:</p>

<blockquote><pre>
fossil new private-project.fossil
</pre></blockquote>

<p>You can call your project anything you like.  The "<tt>.fossil</tt>"
suffix is optional.  For this document, we will continue to call the
project "<tt>private-project.fossil</tt>".  Note that
<tt>private-project.fossil</tt> is an ordinary disk file (actually an
SQLite database) that will contain your complete project history.
You can make a backup of the project simply by making a copy of that
one file.</p>

<p>If you want to configure the new project, type:</p>

<blockquote><pre>
fossil ui private-project.fossil
</pre></blockquote>

<p>The "ui" command will cause fossil running a miniature built-in webserver
(by default listening no port 8080) and to launch your web-browser pointing
at that webserver.  You can use your web-browser to configure your project
in various ways.  See the instructions on the fossil website for additional
information.</p>

<p>Once the project repository is created, create an open checkout of the
project by moving to the directory where you want to keep all of the
project source code and typing:</p>

<blockquote><pre>
fossil open private-project.fossil
</pre></blockquote>

<p>You can have multiple checkouts of the same project if you want. 
And you can "clone" the repository to different machines so that multiple
developers can use it.  See the fossil website for further information.</p>

<h2>3.3 Installing The SQLite Baseline In Fossil</h2>

<p>The repository created in the previous set is initially empty.  The
next step is to load the baseline SQLite release - circle "1" in the diagram
above.</p>

<p>Begin by obtaining a copy of SQLite in whatever form you use it.
The public SQLite you obtain should be as close to your private edited
copy as possible.  If your project uses the SQLite amalgamation, then
get a copy of the the amalgamation.  If you use the preprocessed separate
source files, get those instead.  Put all the source files in the
checkout directory created in the previous step.</p>

<p>The source code in public SQLite releases uses unix line endings
(ASCII code 10: "newline" only, NL) and spaces instead of tabs.  If you will
be changing the line ending to windows-style line endings
(ASCII codes 13, 10: "carriage-return" and "newline"; CR-NL) or if you will be
changing space indents into tab indents, <b>make that change now</b>
before you check in the baseline.  The merging process will only work
well if the differences between the public and the private branches are
minimal.  If every single line of the source file is changed in the
private branch because you changed from NL to CR-NL line endings, then
the merge steps will not work correctly.</p>

<p>Let us assume that you are using the amalgamation source code.
Add the baseline to your project as follows:</p>

<blockquote><pre>
fossil add sqlite3.c sqlite3.h
</pre></blockquote>

<p>If you are using separate source files, name all of the source files instead
of just the two amalgamation source files.  Once this is done, commit your
changes as follows:</p>

<blockquote><pre>
fossil commit
</pre></blockquote>

<p>You will be prompted for a check-in comment.  Say whatever you like.
After the commit completes, your baseline will be part of the repository.
The following command, if you like, to see this on the "timeline":
</p>

<blockquote><pre>
fossil ui
</pre></blockquote>

<p>That last command is the same "ui" command that we ran before.  It
starts a mini-webserver running and points your web browser at it.  But
this time we didn't have to specify the repository file because we are
located inside a checkout and so fossil can figure out the repository for
itself.  If you want to type in the repository filename as the second
argument, you can.  But it is optional.</p>

<p>If you do not want to use your web browser to view the new check-in,
you can get some information from the command-line using commands like
these:

<blockquote><pre>
fossil timeline
fossil info
fossil status
</pre></blockquote>

<h2>3.4 Creating The Private Branch</h2>

<p>The previous step created circle "1" in the diagram above.
This step will create circle "2".  Run the following command:</p>

<blockquote><pre> 
fossil branch new private trunk -bgcolor "#add8e8"
</pre></blockquote>

<p>This command will create a new branch named "private" (you can use
a different name if you like) and assign it a background color
of light blue ("#add8e8").  You can omit the background color if you want,
though having a distinct background does make it easier to tell the
branch from the "trunk" (the public branch) on timeline displays.  You
can change the background color of the private branch or of the public
branch (the "trunk") using the web interface if you like.</p>

<p>The command above created the new branch.  But your checkout is
still on the trunk - a fact you can see by running the command:</p>

<blockquote><pre>
fossil info
</pre></blockquote>

<p>To change your check-out to the private branch, type:</p>

<blockquote><pre>
fossil update private
</pre></blockquote>

<p>You can run the "info" command again to verify that you are on the
private branch.  To go back to the public branch, type:</p>

<blockquote><pre>
fossil update trunk
</pre></blockquote>

<p>Normally, fossil will modify all the files in your checkout when switching
between the private and the public branches.  But at this point, the files
are identical in both branches so not modifications need to be made.</p>

<h2>3.5 Adding Customizations To The Code In The Private Branch</h2>

<p>Now it is time to make the private, custom modifications to SQLite
which are the whole point of this exercise.  Switch to the private branch
(if you are not already there) using the "<tt>fossil update private</tt>"
command, then bring up the source files in your text editor and make
whatever changes you want to make.  Once you have finished making
changes, commit those changes using this command:</p>

<blockquote><pre>
fossil commit
</pre></blockquote>

<p>You will be prompted once again to enter a commit describing your
changes.  Then the commit will occur.  The commit creates a new checkin
in the repository that corresponds to circle "3" in the diagram above.</p>

<p>Now that the public and private branches are different, you can run
the "<tt>fossil update trunk</tt>" and "<tt>fossil update private</tt>"
commands and see that fossil really does change the files in the checkout
as you switch back and forth between branches.</p>

<p>Note that in the diagram above, we showed the private edits as a single
commit.  This was for clarity of presentation only.  There is nothing to stop
you from doing dozens or hundreds of separate tiny changes to and committing
each separately.  In fact, making many small changes is the preferred way
to work.  The only reason for doing all the changes in a single commit
is that it makes the diagram easier to draw.</p>

<h2>3.6 Incorporating New Public SQLite Releases</h2>

<p>Suppose that after a while (about a month, usually) a new version of
SQLite is released: 3.6.16.  You will want to incorporate this new
public version of SQLite into your repository in the public branch (the
trunk).  To do this, first change your repository over to the trunk:</p>

<blockquote><pre>
fossil update trunk
</pre></blockquote>

<p>Then download the new version of the SQLite sources and overwrite the
files that are in the checkout.<p>

<p>If you made NL to CR-NL line ending changes or space to tab
indentation changes in the original baseline, make the same changes
to the new source file.</p>

<p>Once everything is ready, run the "<tt>fossil commit</tt>" command to
check in the changes.  This creates circle "4" in the diagram above.</p>

<h2>3.7 Merging Public SQLite Updates Into The Private Branch</h2>

<p>The next step is to move the changes in the public branch over into
the private branch.  In other words, we want to create circle "5" in the
diagram above.  Begin by change to the private branch using
"<tt>fossil update private</tt>".  Then type this command:</p>

<blockquote><pre>
fossil merge trunk
</pre></blockquote>

<p>The "merge" command attempts to apply all the changes between
circles "1" and "4" to the files in the local checkout.  Note that
circle "5" has not been created yet.  You will need to run the
"commit" to create circle "5".</p>

<p>It might be that there are conflicts in the merge.  Conflicts
occur when the same line of code was changed in different ways between
circles "1" and "4 versus circles "2" and "3".  The merge command will
announce any conflicts and will include both versions of the conflicting
lines in the output.  You will need to bring up the files that contain
conflicts and manually resolve the conflicts.</p>

<p>After resolving conflicts, many users like to compile and test the
new version before committing it to the repository.  Or you can commit
first and test later.  Either way, run the "<tt>fossil commit</tt>"
command to check-in the circle "5" version.

<h2>3.8 Further Updates</h2>

<p>As new versions of SQLite are released, repeat steps 3.6 and 3.7 to
add them into the private branch.  Additional private changes can be
made on the private branch in between releases if desired.</p>