<title>Row Values</title>
<tcl>hd_keywords rowvalue {row value} {row values} {vectors}</tcl>

<table_of_contents>

<h1>Definitions</h1>

<p>A "value" is a single number, string, BLOB or NULL.
Sometimes the qualified name "scalar value" is used to emphasize that
only a single quantity is involved.

<p>A "row value" is an ordered list of two or more scalar values.
In other words, a "row value" is a vector.

<p>The "size" of a row value is the number of scalar values the row value contains.
The size of a row value is always at least 2.
A row value with a single column is just a scalar value.
A row value with no columns is a syntax error.

<h1>Syntax</h1>

<p>SQLite allows row values to be expressed in two ways:
<ol>
<li>A parenthesized, comma-separated list of scalar values.
<li>A subquery expression with two or more result columns.
</ol>

<p>SQLite can use row values in two contexts:
<ol>
<li>Two row values of the same size 
can be compared using operators &lt;, &lt;=, &gt;, &gt;=,
=, &lt;&gt;, IS, IS NOT, IN, NOT IN, BETWEEN, or CASE.
<li>In an [UPDATE] statement, a list of column names can be set to a row value of
the same size.
</ol>

<p>The syntax for row values and the circumstances in which row values
can be used are illustrated in examples below.

<h2>Row Value Comparisons</h2>

<p>Two row values are compared by looking at the constituent scalar
values from left to right.
A NULL means of "unknown".  
The overall result of comparison is NULL if it is possible to make the
result either true or false by substituting alternative values in place
of the constituent NULLs.
The following query demonstrates some row value comparisons:

<codeblock>
SELECT
  (1,2,3) = (1,2,3),          -- 1
  (1,2,3) = (1,NULL,3),       -- NULL
  (1,2,3) = (1,NULL,4),       -- 0
  (1,2,3) &lt; (2,3,4),          -- 1
  (1,2,3) &lt; (1,2,4),          -- 1
  (1,2,3) &lt; (1,3,NULL),       -- 1
  (1,2,3) &lt; (1,2,NULL),       -- NULL
  (1,3,5) &lt; (1,2,NULL),       -- 0
  (1,2,NULL) IS (1,2,NULL);   -- 1
</codeblock>

<p>The result of "(1,2,3)=(1,NULL,3)" is NULL because the result might be
true if we replaced NULL&rarr;2 or false if we replaced NULL&rarr;9.
The result of "(1,2,3)=(1,NULL,4)" is not NULL because there is no
substitutions of the constituent NULL that will make the expression true,
since 3 will never equal 4 in the third column.

<p>Any of the row values in the previous example could be replace by a
subquery that returns three columns and the same answer would result.
For example:

<codeblock>
CREATE TABLE t1(a,b,c);
INSERT INTO t1(a,b,c) VALUES(1,2,3);
SELECT (1,2,3)=(SELECT * FROM t1); -- 1
</codeblock>

<p>For a row-value [IN operator], the left-hand side (hereafter "LHS") can be either
a parenthesized list of values or a subquery with multiple columns.  But the
right-hand side (hereafter "RHS") must be a subquery expression.

<codeblock>
CREATE TABLE t2(x,y,z);
INSERT INTO t2(x,y,z) VALUES(1,2,3),(2,3,4),(1,NULL,5);
SELECT
   (1,2,3) IN (SELECT * FROM t2),  -- 1
   (7,8,9) IN (SELECT * FROM t2),  -- 0
   (1,3,5) IN (SELECT * FROM t2);  -- NULL
</codeblock>

<h2>Row Values In UPDATE Statements</h2>

<p>Row values can also be used in the SET clause of an [UPDATE] statement.
The LHS must be a list of column names.  The RHS can be any row value.
For example:

<codeblock>
UPDATE tab3 
   SET (a,b,c) = (SELECT x,y,z
                    FROM tab4
                   WHERE tab4.w=tab3.d)
 WHERE tab3.e BETWEEN 55 AND 66;
</codeblock>

<h1>Example Uses Of Row Values</h1>

<h2>Scrolling Window Queries</h2>

<p>Suppose an application wants to display a list of contacts
in alphabetical order by lastname, firstname, in a scrolling window
that can only show 7 contacts at a time.  Initialize the scrolling
window to the first 7 entries is easy:

<codeblock>
SELECT * FROM contacts
 ORDER BY lastname, firstname
 LIMIT 7;
</codeblock>

<p>When the user scrolls down, the application needs to find the
second set of 7 entries.  One way to do this is to use the OFFSET clause:

<codeblock>
SELECT * FROM contacts
 ORDER BY lastname, firstname
 LIMIT 7 OFFSET 7;
</codeblock>

<p>OFFSET gives the correct answer.  However, OFFSET requires time
proportional to the offset value.  What really happens
with "LIMIT x OFFSET y" is that SQLite computes the query as
"LIMIT x+y" and discards the first y values without returning them
to the application.  So as the window scrolls down toward
the bottom of a long list, and the y value becomes larger and larger,
successive offset computations take more and more time.

<p>A more efficient approach is to remember the last entry currently
displayed and then use a row value comparison in the WHERE
clause:

<codeblock>
SELECT * FROM contacts
 WHERE (lastname,firstname) &gt; (?1,?2)
 ORDER BY lastname, firstname
 LIMIT 7;
</codeblock>

<p>If the lastname and firstname on the bottom row of the previous
screen are bound to ?1 and ?2, then the query above computes the next
7 rows.  And, assuming there is an appropriate index, it does so
very efficiently &mdash; much more efficiently than OFFSET.

<h2>Comparison of dates stored as separate fields</h2>

<p>The usual way of storing a date in a database table is as a single
field, as either a unix timestamp, a julian day number, or an ISO-8601
dates string.  But some application store dates as three separate
fields for the year, month, and day.  

<codeblock>
CREATE TABLE info(
  year INT,          -- 4 digit year
  month INT,         -- 1 through 12
  day INT,           -- 1 through 31
  other_stuff BLOB   -- blah blah blah
);
</codeblock>

<p>When dates are stored this way, row value comparisons provide a
convenient way to compare dates:

<codeblock>
SELECT * FROM info
 WHERE (year,month,day) BETWEEN (2015,9,12) AND (2016,9,12);
</codeblock>

<h2>Search against multi-column keys</h2>

<p>Suppose we want to know the order number, product number, and quantity
for any item in which the product number and quantity match the product
number and quantity of any item in order number 365:

<codeblock>
SELECT ordid, prodid, qty
  FROM item
 WHERE (prodid, qty) IN (SELECT prodid, qty
                           FROM item
                          WHERE ordid = 365);
</codeblock>

<p>The query above could be rewritten as a join and without the use
of row values:

<codeblock>
SELECT t1.ordid, t1.prodid, t1.qty
  FROM item AS t1, item AS t2
 WHERE t1.prodid=t2.prodid
   AND t1.qty=t2.qty
   AND t2.ordid=365;
</codeblock>

<p>Because the same query could be written without the use of row values,
row values do not provide new capabilities.  However, many developers say
that the row value format is easier to read, write, and debug.

<p>Even in the JOIN form, the query can be made clearer through the use of
row values:

<codeblock>
SELECT t1.ordid, t1.prodid, t1.qty
  FROM item AS t1, item AS t2
 WHERE (t1.prodid,t1.qty) = (t2.prodid,t2.qty)
   AND t2.ordid=365;
</codeblock>

<p>This later query generates exactly the same [bytecode] as the previous
scalar formulation, but using syntax that it cleaner and
easier to read.

<h2>Update multiple columns of a table based on a query</h2>

<p>The row-value notation is useful for updating two or more columns
of a table from the result of a single query.
An example of this is in the full-text search feature of the
[https://www.fossil-scm.org/|Fossil version control system].

<p>In the Fossil full-text search system,
documents that participate in the full-text search (wiki pages, tickets,
check-ins, documentation files, etc) are tracked by a table called
"ftsdocs" (<u>f</u>ull <u>t</u>ext <u>s</u>earch <u>doc</u>ument<u>s</u>).
As new documents are added to the repository, they are not indexed right
away.  Indexing is deferred until there is a search request.  The
ftsdocs table contains an "idxed" field which is true if the document
has been indexed and false if not.

<p>When a search request occurs and pending documents are indexed for the
first time, the ftsdocs table must be updated by setting the idxed column
to true and also filling in several other columns with information pertinent
to the search.  That other information is obtained from a join.  The
query is this:

<codeblock>
UPDATE ftsdocs SET
  idxed=1,
  name=NULL,
  (label,url,mtime) = 
      (SELECT printf('Check-in &#91;%%.16s&#93; on %%s',blob.uuid,
                     datetime(event.mtime)),
              printf('/timeline?y=ci&amp;c=%%.20s',blob.uuid),
              event.mtime
         FROM event, blob
        WHERE event.objid=ftsdocs.rid
          AND blob.rid=ftsdocs.rid)
WHERE ftsdocs.type='c' AND NOT ftsdocs.idxed
</codeblock>

<p>(See the 
[https://www.fossil-scm.org/fossil/artifact/e5d6a82d?ln=1594-1605|source code]
for further detail.  Other examples
[https://www.fossil-scm.org/fossil/artifact/e5d6a82d?ln=1618-1628|here] and
[https://www.fossil-scm.org/fossil/artifact/e5d6a82d?ln=1641-1650|here].)

<p>Five out of nine columns in the ftsdocs table are updated.  Two of
the modified columns, "idxed" and "name", can be updated independently of
the query.  But the three columns "label", "url", and "mtime" all require
a join query against the "event" and "blob" tables.  Without row values,
the equivalent UPDATE would require that the join be repeated three times, 
once for each column to be updated.

<h2>Clarity of presentation</h2>

<p>Sometimes the use of row values just makes the SQL easier to read
and write.  Consider the following two UPDATE statements:

<codeblock>
UPDATE tab1 SET (a,b)=(b,a);
UPDATE tab1 SET a=b, b=a;
</codeblock>

<p>Both UPDATE statements do exactly the same thing.  (They generate
identical [bytecode].)  But the first form, the row value form, seems
to make it clearer that the intent of the statement is to swap the
values in columns A and B.

<p>Or consider these identical queries:

<codeblock>
SELECT * FROM tab1 WHERE a=?1 AND b=?2;
SELECT * FROM tab1 WHERE (a,b)=(?1,?2);
</codeblock>

<p>Once again, the SQL statements generate identical bytecode and thus
do exactly the same job in exactly the same way.  But the second form
is made easier for humans to read by grouping the query parameters together
into a single row value rather than scattering them across the WHERE
clause.

<h1>Backwards Compatibility</h1>

<p>Row values were added to SQLite
[version 3.15.0] ([dateof:3.15.0]).  Attempts to use row values in
prior versions of SQLite will generate syntax errors.
