Documentation Source Text

<p><b>New Features And Enhancements:</b>
<li>The [CREATE VIEW] statement now accepts an optional list of
    column names following the view name.
<li>Added support for [indexes on expressions].
<li>Added support for [table-valued functions] in the FROM clause of a
    [SELECT] statement.
<li>Added support for [eponymous virtual tables].
<li>Added [the json1 extension] module in the source tree.
<li>A [VIEW] may now reference undefined tables and functions when
    initially created.  Missing tables and functions are reported when
    the VIEW is used in a query.
<li>The query planner is now able to use [partial indexes] that contain
    AND-connected terms in the WHERE clause.
<li>The sqlite3_analyzer.exe utility is updated to report the depth of
    each btree and to show the average fanout for indexes and 

<title>The JSON1 Extension</title>
<tcl>hd_keywords json1 {the json1 extension}</tcl>
<h2>The JSON1 Extension</h2>

<p>
The <b>json1</b> extension is a [loadable extension] that
implements ten [application-defined SQL functions] and
two [table-valued functions] that are useful for
managing JSON content stored in an SQLite database.
These are the SQL functions implemented by json1:

<blockquote>
<center><table border=0 cellpadding=5>
<tcl>
set tabcnt 0
proc tabentry {fx desc lnk} {
  global tabcnt
  incr tabcnt
  hd_puts "<tr><td width=30 valign='top'>$tabcnt.</td>"
  hd_puts "<td valign='top' width='30%'>\n"
  set fx [string trim $fx]
  set hlink "<a href='#$lnk'>"
  regsub -all {(json_[a-z_]+)} $fx "$hlink\\1</a>" fx
  regsub -all {(value?|path|label?)} $fx "<i>\\1</i>" fx
  regsub -all {\((json)} $fx "(<i>\\1</i>" fx
  hd_puts $fx\n
  hd_puts "</td><td valign='top'>\n"
  hd_puts [string trim $desc]\n
  hd_puts "</td></tr>\n\n"
}

tabentry {json_array(value1,value2,...)} {
  Return a JSON array holding the function arguments.
} jarray

tabentry {json_array_length(json)<br>json_array_length(json,path)} {
  Return a JSON array holding the function arguments.  The optional
  "path" argument specifies the location of the array in a larger
  JSON object.
} jarraylen

tabentry {json_extract(json,path)} {
  Extract a value from a JSON string.
} jex

tabentry {json_insert(json,path,value,...)} {
  Insert one or more values into a JSON string.
} jins

tabentry {json_object(label1,value1,...)} {
  Construct and return a new JSON object based on the arguments.
} jobj

tabentry {json_remove(json,path,...)} {
  Remove one or more values from a JSON string.
} jrm

tabentry {json_replace(json,path,value,...)} {
  Replace one or more in a JSON string with new values supplied.
} jrepl

tabentry {json_set(json,path,value,...)} {
  Insert or replace one or more values in a JSON string with new
  values supplied
} jset

tabentry {json_type(json)<br>json_type(json,path)} {
  Return the type of a JSON string or subcomponent.
} jtype

tabentry {json_valid(json)} {
  Return true (1) if the input text is a valid JSON string
} jvalid
</tcl>
</table></center></blockquote>

<p>The [table-valued functions] implemented by this routine are:

<blockquote><center><table border=0 cellpadding=5>
<tcl>
tabentry {json_each(json)<br>json_each(json,path)} {
  Return one row describing each element in an array or object
  at the top-level or at "path" within the input JSON.
} jeach

tabentry {json_tree(json)<br>json_tree(json,path)} {
  Walk the JSON recursively and return one row for each element.
} jtree
</tcl>
</table></center></blockquote>

<h2>1.0 Ground Rules</h2>

<p>
The json1 extension (currently) stores JSON as ordinary text.

<p>
Backwards compatibility constraints mean that SQLite is only able to
store values that are NULL, integers, floating-point numbers, text,
and BLOBs.  It is not possible add a sixth "JSON" type.  Future
versions of SQLite might be extended to support the concept of a
"sub-type" for text and BLOB values.  If and when that enhancement is
implemented, there will be a JSON "sub-type" available to
values of type "text".

<p>
The json1 extension does not (currently) support a binary encoding
of JSON.  Experiments have so far been unable to find a binary encoding
that is significantly smaller or faster than a plain text encoding.
(The present implementation is able to parse JSON text at a rate of
250 MB/s.)
The json1 extension might be enhanced in the future to support a 
JSONB sub-type that is stored as a BLOB.

<p>
The "1" at the end of the name for this extension is deliberate.
The designers anticipate that there will be future incompatible JSON
extensions building upon the lessons learned from json1.
Once sufficient experience is gained, some kind of
JSON extension might be folded into the SQLite core.  But for now,
JSON support remains an extension.

<h3>1.1 JSON arguments</h3>

<p>
For functions that accept JSON as their first argument, that argument
can be any JSON an object, array, number, string, or null.  SQLite numeric
values and NULL values are interpreted as JSON numbers and nulls, respectively.
SQLite text values can be understood as JSON objects, arrays, or strings.
If an SQLite text value that is not a well-formed JSON object, array, or
string is passed into json1 function, that function will usually throw
an error.  (The except to the previous sentence is the json_valid() function
which returns 1 if the argument is well-formed and 0 if it is not.)

<p>
For the purposes of determining validity, leading and trailing whitespace
on JSON inputs is ignored.  Interior whitespace is also ignored, in accordance
with the JSON spec.

<p>
Passing in a BLOB value as JSON always causes an error to be thrown
(except by the json_valid() function, which instead returns 0.) 
This extension may be enhanced in the future to interpret some BLOB
values as a binary encoding of JSON.


<h3>1.2 PATH arguments</h3>

<p>
For functions that accept PATH arguments, that PATH must be well-formed or
else the function will throw an error.
A well-formed PATH is a text value that begins with one or in some cases
two '$' characters.  The last '$' can be followed by zero or more instances
of ".<i>objectlabel</i>" or "&#91<i>arrayindex</i>&#93".

<h2>2.0 Function Details</h2>

<p>The following sections provide additional detail on the operation of
the various functions that are part of the json1 extension.

<tcl>hd_fragment jarray {json_array SQL function} {json_array}</tcl>
<h3>2.1 The json_array() function</h3>

<p>The json_array() SQL function accepts zero or more arguments and
returns a well-formed JSON array that is composed from those arguments.
If any argument to json_array() is a BLOB then an error is thrown.

<p>In the current implementation, if an argument to json_array() is
text that looks like JSON, it is quoted and interpreted as a single
JSON string value.  In future enhancements in which a SQLite text 
value can have a sub-type of "JSON", this routine will insert 
substructure instead of a single string value.  Please beware of this
future incompatibility and plan accordingly.


<tcl>hd_fragment jarraylen {json_array_length SQL function} \
         {json_array_length}</tcl>
<h3>2.2 The json_array_length() function</h3>

<p>The json_array_length(J) function returns the number of elements
in the JSON array J, or 0 if J is some kind of JSON value other
than an array.  The json_array_length(J,P) locates the array at path P
within J and returns the length of that array, or 0 if path P does not
locate an array within J.  Errors are thrown if either J is not 
well-formed JSON or if P is not a well-formed path.

<tcl>hd_fragment jex {json_extract SQL function} {json_extract}</tcl>
<h3>2.3 The json_extract() function</h3>

<p>The json_extract(J,P1,P2,...) extracts and returns one or more 
values from the
well-formed JSON at J.  If only a single path P1 is provided, then the
SQLite type of the value returned is NULL for a JSON null, INTEGER or REAL
for a JSON numeric value, INTEGER 0 for a JSON false value, INTEGER 1
for a JSON true value, the dequoted text for a JSON string value, and
a text representation for JSON object and array values.
If there are multiple path arguments (P1, P2, and so forth) then this
routine returns an SQLite text which is a well-formed JSON array holding
the various values.

<tcl>hd_fragment jins {json_insert SQL function} {json_insert}</tcl>
<tcl>hd_fragment jrepl {json_replace SQL function} {json_replace}</tcl>
<tcl>hd_fragment jset {json_set SQL function} {json_set}</tcl>
<h3>2.4 The json_insert(), json_replace, and json_set() functions</h3>

<p>The json_insert(), json_replace, and json_set() functions all take
a single JSON value as their first argument followed by zero or more
pairs of path and value arguments, and return a new JSON value formed
by updating the input argument by the paths and values.  The functions
differ only in how they deal with creating new values and overwriting
preexisting values.

<center>
<table border=1 cellpadding=3>
<tr>
<th>Function<th>Overwrite if already exists?<th>Create if does not exist?
<tr>
<td>json_insert()<td align='center'>No<td align='center'>Yes
<tr>
<td>json_replace()<td align='center'>Yes<td align='center'>No
<tr>
<td>json_insert()<td align='center'>Yes<td align='center'>Yes
</table></center>

<p>The json_insert(), json_replace(), and json_set() functions always
take an odd number of arguments.  The first argument is always the original
JSON to be edited.  Subsequent arguments occur in pairs with the first
element of each pair being a path and the second element being an value
to insert or replace or set on that path.

<p>Edits occurs sequentially from left to right.  Changes caused by
prior edits can affect the path search for subsequent edits.

<p>Path arguments in json_insert(), json_replace(), and json_set() are
allowed to have either one or two initial '$' characters.  If the path
has one initial '$' character, then SQLite text value are quoted and
inserted as if they are JSON string values.  If the path has two initial '$'
characters, then SQLite text values are inserted as JSON array or object
values.

<p>Future versions of this extension that support the JSON sub-type for
text will always insert as JSON objects or arrays any string that has
a sub-type of JSON and will only quote and insert values as JSON
strings if the input value is not of sub-type JSON.

<p>These routines throw an error if the first JSON argument is not
well-formed or if any PATH argument is not well-formed or if any
argument is of type BLOB.


<tcl>hd_fragment jobj {json_object SQL function} {json_object}</tcl>
<h3>2.5 The json_object() function</h3>

<p>The json_object() SQL function accepts zero or more pairs of arguments
and returns a well-formed JSON object that is composed from those arguments.
The first argument of each pair is the label and the second argument of
each pair is the value.
If any argument to json_array() is a BLOB then an error is thrown.

<p>The json_object() function does not currently object to having
duplicate labels, though this might change in a future enhancement.

<p>In the current implementation, if a value argument to json_object() is
text that looks like JSON, it is quoted and interpreted as a single
JSON string value.  In future enhancements in which a SQLite text 
value can have a sub-type of "JSON", this routine will insert 
substructure instead of a single string value.  Please beware of this
future incompatibility and plan accordingly.


<tcl>hd_fragment jrm {json_remove SQL function} {json_remove}</tcl>
<h3>2.6 The json_remove() function</h3>

<p>The json_remove(J,P,...) function takes a single JSON value as its
first argument followed by zero or more path arguments in the second
and subsequent arguments.  The json_remove(J,P,...) function returns
a new JSON value that is the input J value with all the elements 
identified by path arguments removed.  Paths that select elements
not found in J are silently ignored.

<p>Removals occurs sequentially from left to right.  Changes caused by
prior removals can affect the path search for subsequent arguments.

<p>If the json_remove(J) function is called with no path arguments,
then it returns the input J reformatted, with excess whitespace
removed.

<p>The json_remove() function throws an error if the first argument
is not well-formed JSON or if any later argument is not a well-formed
path, or if any argument is a BLOB.


<tcl>hd_fragment jtype {json_type SQL function} {json_type}</tcl>
<h3>2.7 The json_type() function</h3>

<p>The json_type(J) function returns the "type" of the outermost element
of J.  The json_type(J,P) function returns the "type" of the element
in J that is selected by path P.  The "type" returned by json_type() is
an SQL text value which is one of the following:
'null', 'true', 'false', 'integer', 'real', 'text', 'array', or 'object'.
If the path P in json_type(J,P) selects a element that does not exist
in J, then the function returns NULL.

<p>The json_type() function throws an error if any of its arguments are
not well-formed or are a BLOB.

<tcl>hd_fragment jvalid {json_valid SQL function} {json_valid}</tcl>
<h3>2.8 The json_valid() function</h3>

<p>The json_valid(J) function return 1 if the argument J is well-formed
JSON and return 0 if the argument J is not well-formed JSON.

<tcl>hd_fragment jeach {json_each table-valued function} {json_each}</tcl>
<tcl>hd_fragment jtree {json_tree table-valued function} {json_tree}</tcl>
<h3>2.9 The json_each() and json_tree() table-valued functions</h3>

<p>The json_each(J) and json_tree(J) [table-valued functions] walk the
JSON value provided as their first argument and return one row for each
element.  The json_each(J) function only walks the top-level
element (which is presumably an array or object) and does not descend into
substructure.  The json_tree(J) function recursively walks through the
JSON substructure.  Other than the fact that json_tree(J) recursively
descends through JSON substructure, the json_tree(J) and json_each(J)
functions are the same.

<p>The schema for the table returned by json_each() and json_tree() is
as follows:

<blockquote><pre>
CREATE TABLE json_tree(
    key ANY,             -- key for current element relative to its parent
    value ANY,           -- value for the current element
    type TEXT,           -- 'object','array','string','integer', etc.
    atom ANY,            -- value for primitive types, null for array & object
    id INTEGER           -- integer ID for this element
    parent INTEGER,      -- integer ID for the parent of this element
    fullkey TEXT,        -- full path describing the current element
    json JSON HIDDEN,    -- 1st input parameter: the raw JSON
    path TEXT HIDDEN     -- 2nd input parameter: the PATH at which to start
);
</pre></blockquote>

<p>
The "key" column is the integer array index for elements of a JSON array 
and the text label for elements of a JSON object.  The key column is
NULL in all other cases.

<p>
The "atom" column is the SQL value corresponding to primitive elements - 
elements other than JSON arrays and objects.  The "atom" column is NULL
for a JSON array or object.  The "value" column is the same as the
"atom" column for primitive JSON elements but takes on the text JSON value
for arrays and objects.

<p>
The "type" column is an SQL text value taken from ('null', 'true', 'false',
'integer', 'real', 'text', 'array', 'object') according to the type of
the current JSON element.

<p>
The "id" column is an integer that identifies a specific JSON element
within the complete JSON string.  The "id" integer is an internal housekeeping
number, the computation of which might change in future releases.  The
only guarantee is that the "id" column will be different for every row.

<p>
The "parent" column is the "id" integer for the parent of the current
element, or NULL for the top-level JSON element.

<p>
The "fullkey" column is a text path that uniquely identifies the current
row element within the original JSON string.


<h2>3.0 Examples</h2>