Documentation Source Text

Check-in [783032878d]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update documentation to match the proposed window function changes.
Timelines: family | ancestors | descendants | both | window-functions
Files: files | file ages | folders
SHA3-256:783032878dd3070e89a919ec95f848166be65f4681ffcbc8cf48194aeb24a86b
User & Date: dan 2019-03-25 20:49:56
Context
2019-03-26
13:11
Update window functions documentation. check-in: bdc2a67078 user: dan tags: trunk
2019-03-25
20:49
Update documentation to match the proposed window function changes. Closed-Leaf check-in: 783032878d user: dan tags: window-functions
15:19
Update the CLI documentation to describe the new --insert option to ".archive" and the behavior change to the --update option. check-in: bf3c223723 user: drh tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to art/syntax/all-bnf.html.

cannot compute difference between binary files

Changes to art/syntax/all-text.html.

cannot compute difference between binary files

Changes to art/syntax/bubble-generator-data.tcl.

521
522
523
524
525
526
527
528
529
530
531
532

533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549








550
551
552
553
554
555
556
557
558
559
      {line /* {loop nil /anything-except-*/}
           {or */ /end-of-input}}
  }
  filter {
    line FILTER ( WHERE expr )
  }
  window-defn {
    stack {line ( {opt PARTITION BY {loop expr ,}}}
          {opt ORDER BY {loop ordering-term ,}}
          {line {optx frame-spec} )}
  }
  frame-spec {

    line {or RANGE ROWS} {or
       {line BETWEEN {or {line UNBOUNDED PRECEDING}
                         {line expr PRECEDING}
                         {line CURRENT ROW}
                         {line expr FOLLOWING}
                     }
             AND {or     {line expr PRECEDING}
                         {line CURRENT ROW}
                         {line expr FOLLOWING}
                         {line UNBOUNDED FOLLOWING}
                 }
       }
       {or   {line UNBOUNDED PRECEDING}
             {line expr PRECEDING}
             {line CURRENT ROW}
             {line expr FOLLOWING}
       }








    }
  }
  function-invocation {
     line /function-name ( {or {line {optx DISTINCT} {toploop expr ,}} {} *} )
  }
  window-function-invocation {
    line /window-func ( {or {line {toploop expr ,}} {} *} ) 
         {opt filter} OVER {or window-defn /window-name}
  }
}







|




>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
>
>
>
>
>
>
>
>










521
522
523
524
525
526
527
528
529
530
531
532
533
534
535
536
537
538
539
540
541
542
543
544
545
546
547
548
549
550
551
552
553
554
555
556
557
558
559
560
561
562
563
564
565
566
567
568
      {line /* {loop nil /anything-except-*/}
           {or */ /end-of-input}}
  }
  filter {
    line FILTER ( WHERE expr )
  }
  window-defn {
    stack {line ( {opt existing-window-name} {opt PARTITION BY {loop expr ,}}}
          {opt ORDER BY {loop ordering-term ,}}
          {line {optx frame-spec} )}
  }
  frame-spec {
    stack {
      line {or RANGE ROWS GROUPS} {or
         {line BETWEEN {or {line UNBOUNDED PRECEDING}
                           {line expr PRECEDING}
                           {line CURRENT ROW}
                           {line expr FOLLOWING}
                       }
               AND {or     {line expr PRECEDING}
                           {line CURRENT ROW}
                           {line expr FOLLOWING}
                           {line UNBOUNDED FOLLOWING}
                   }
         }
         {or   {line UNBOUNDED PRECEDING}
               {line expr PRECEDING}
               {line CURRENT ROW}
               {line expr FOLLOWING}
         }
      }
    } {
      line {opt {or 
         {line EXCLUDE NO OTHERS} 
         {line EXCLUDE CURRENT ROW} 
         {line EXCLUDE GROUP} 
         {line EXCLUDE TIES} 
      } }
    }
  }
  function-invocation {
     line /function-name ( {or {line {optx DISTINCT} {toploop expr ,}} {} *} )
  }
  window-function-invocation {
    line /window-func ( {or {line {toploop expr ,}} {} *} ) 
         {opt filter} OVER {or window-defn /window-name}
  }
}

Changes to art/syntax/frame-spec.gif.

cannot compute difference between binary files

Changes to art/syntax/window-defn.gif.

cannot compute difference between binary files

Changes to pages/windowfunctions.in.

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
...
159
160
161
162
163
164
165
166
167
168

169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186

187
188
189
190
191
192
193
194
195


196



197
198























199
200
201

202
203
204
205
206

207
208
209
210
211
212
213
...
229
230
231
232
233
234
235




236
237
238
239
240
241
242
...
289
290
291
292
293
294
295





























































































296
297
298
299
300
301
302


<tcl>
RecursiveBubbleDiagram window-function-invocation window-defn frame-spec filter
</tcl>

<p>Window functions are distinguished from ordinary SQL functions by the
presence of an OVER clause.  If a function invocation has an OVER clause
then it is a window function, and if lacks an OVER clause it is an ordinary
function.  Window functions might also have a FILTER
clause in between the function and the OVER clause.

<p>Unlike ordinary functions, window functions
cannot use the DISTINCT keyword.
Also, Window functions may only appear in the result set and in the
ORDER BY clause of a SELECT statement.

................................................................................
</tcl>

<p> The <yynonterm>frame-spec</yynonterm> determines which output rows are
read by an aggregate window function.  The
<yynonterm>frame-spec</yynonterm> consists of three parts:

<ul>
  <li> A frame type - either RANGE or ROWS.
  <li> A starting frame boundary, and
  <li> An ending frame boundary.

</ul>

<p>The ending frame boundary can be omitted, in which case it defaults
to CURRENT ROW.

<p>The default <yynonterm>frame-spec</yynonterm> is:

<codeblock>
    RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
</codeblock>

<p>The default means that aggregate window functions read all
rows from the beginning of the partition up to and including the
current row and its peers.

<p> If the frame type is RANGE, then rows with the same values for all ORDER BY
expressions are considered "peers". Or, if there are no ORDER BY terms,
all rows are peers. Rows that are peers always have the same window frames.


<p> There are five options for frame boundaries:

<table striped=1>
<tr><th>Frame Boundary <th>Description
<tr><td>UNBOUNDED&nbsp;PRECEDING <td> The start of the frame is the first
        row in the set.
<tr><td>&lt;expr&gt; PRECEDING <td> &lt;expr&gt; is a constant expression
        that evaluates to a non-negative integer value. The start


        or end of the frame is &lt;expr&gt; rows before the current row. "0



        PRECEDING" is the same as "CURRENT ROW". This frame boundary type may
        only be used with ROWS frames.























<tr><td>CURRENT ROW <td> The current row. For RANGE frame types, all peers of
        the current row are also included in the window frame, regardless of
        whether CURRENT ROW is used as the starting or ending frame boundary.

<tr><td>&lt;expr&gt; FOLLOWING <td> &lt;expr&gt; is a constant expression
        that must evaluate to a non-negative integer value. The start
        or end of the frame is &lt;expr&gt; rows after the current row. "0
        FOLLOWING" is the same as "CURRENT ROW". This frame boundary type may
        only be used with ROWS frames.

<tr><td>UNBOUNDED&nbsp;FOLLOWING <td> The end of the frame is the last
        row in the set.
</table>

<p> The ending frame boundary must not take a form that appears higher in
the above list than the starting frame boundary.

................................................................................
  <i>--   two   | 5 | E | E           </i>
  <i>-- </i>
  SELECT c, a, b, group_concat(b, '.') OVER (
    ORDER BY c, a ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
  ) AS group_concat
  FROM t1 ORDER BY c, a;
</codeblock>





<h2>The PARTITION BY Clause</h2>

<p> A <yynonterm>window-defn</yynonterm> may include a PARTITION BY clause.
If so, the rows returned by the SELECT statement are divided into groups -
partitions - with the same values for each PARTITION BY expression, and then
window-function processing is performed separately for each
................................................................................
  <i>--   one   | 7 | G | A.C.D.F.G   </i>
  <i>-- </i>
  SELECT c, a, b, group_concat(b, '.') FILTER (WHERE c!='two') OVER (
    ORDER BY a
  ) AS group_concat
  FROM t1 ORDER BY a;
</codeblock>






























































































<h1>Built-in Window Functions</h1>

<p> As well as aggregate window functions, SQLite features a set of built-in
window functions based on 
<a href=https://www.postgresql.org/docs/10/static/functions-window.html>
those supported by PostgreSQL</a>.







|
|
|







 







|
|
|
>








|






|
|
|
>








|
>
>
|
>
>
>
|
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
|
>

|
<
<
<
>







 







>
>
>
>







 







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
...
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204

205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233



234
235
236
237
238
239
240
241
...
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
...
321
322
323
324
325
326
327
328
329
330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
353
354
355
356
357
358
359
360
361
362
363
364
365
366
367
368
369
370
371
372
373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
423
424
425
426
427


<tcl>
RecursiveBubbleDiagram window-function-invocation window-defn frame-spec filter
</tcl>

<p>Window functions are distinguished from ordinary SQL functions by the
presence of an OVER clause. If a function invocation has an OVER clause,
then it is a window function. If it lacks an OVER clause, then it is an
ordinary aggregate or scalar function. Window functions may have a FILTER
clause in between the function and the OVER clause.

<p>Unlike ordinary functions, window functions
cannot use the DISTINCT keyword.
Also, Window functions may only appear in the result set and in the
ORDER BY clause of a SELECT statement.

................................................................................
</tcl>

<p> The <yynonterm>frame-spec</yynonterm> determines which output rows are
read by an aggregate window function.  The
<yynonterm>frame-spec</yynonterm> consists of three parts:

<ul>
  <li> A frame type - either ROWS, RANGE or GROUPS,
  <li> A starting frame boundary,
  <li> An ending frame boundary,
  <li> An EXCLUDE clause.
</ul>

<p>The ending frame boundary can be omitted, in which case it defaults
to CURRENT ROW.

<p>The default <yynonterm>frame-spec</yynonterm> is:

<codeblock>
    RANGE BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS
</codeblock>

<p>The default means that aggregate window functions read all
rows from the beginning of the partition up to and including the
current row and its peers.

<p> If the frame type is RANGE or GROUPS, then rows with the same values for
all ORDER BY expressions are considered "peers". Or, if there are no ORDER BY
terms, all rows are peers. Rows that are peers always have the same window
frames.

<p> There are five options for frame boundaries:

<table striped=1>
<tr><th>Frame Boundary <th>Description
<tr><td>UNBOUNDED&nbsp;PRECEDING <td> The start of the frame is the first
        row in the set.
<tr><td>&lt;expr&gt; PRECEDING <td> &lt;expr&gt; is a constant expression
        that evaluates to a non-negative numeric value. If the frame type
        is ROWS or GROUPS, the expression must evaluate to an integer
        value.<br><br>
        For a ROWS frame, the start or end of the frame is &lt;expr&gt; rows
        before the current row.<br><br>
        For a GROUPS frame, the frame begins or ends with the peers
        &lt;expr&gt; groups of peers before or after the current row's group.
        For both ROWS and GROUPS frames, "0 PRECEDING" is the same as "CURRENT

        ROW".
        <br><br> A RANGE frame that uses "&lt;expr&gt; PRECEDING" must be
        paired with an ORDER BY clause that contains a single expression. If
        the ORDER BY expression for the current row is not a numeric value, 
        then "&lt;expr&gt; PRECEDING" is equivalent to "CURRENT ROW".
        Otherwise, if the ORDER BY value is numeric for the current row, then
        the value of "&lt;expr&gt;" is used as a logical range for determining
        which groups are part of the current frame.
        <br><br>
        For example, if the ORDER BY clause is ASC (not DESC) and the
        "&lt;expr&gt; PRECEDING" is used as the start of the frame, then
        the first group in the frame is that with the smallest value for
        the ORDER BY expression that is greater than (current-row -
        &lt;expr&gt;). If the ORDER BY clause is DESC, then the first group
        is that with an ORDER BY expresion greater than (current-row +
        &lt;expr&gt;).
        <br><br>
        Or, if it is used as the end of the frame and the ORDER BY is ASC, then
        the last group in the grame is that with the largest value for the
        ORDER BY expression that is less than (current-row - &lt;expr&gt;).
        For DESC the last group is that with the smallest ORDER BY value
        that is greater than (current-row + &lt;expr&gt;).

<tr><td>CURRENT ROW <td> The current row. For RANGE and GROUPS frame types, all
        peers of the current row are also included in the window frame,
        regardless of whether CURRENT ROW is used as the starting or ending
        frame boundary.
<tr><td>&lt;expr&gt; FOLLOWING <td> &lt;expr&gt; is a constant expression
        that must evaluate to a non-negative numeric value. It is handled



        in similarly to "&lt;expr&gt; PRECEDING".
<tr><td>UNBOUNDED&nbsp;FOLLOWING <td> The end of the frame is the last
        row in the set.
</table>

<p> The ending frame boundary must not take a form that appears higher in
the above list than the starting frame boundary.

................................................................................
  <i>--   two   | 5 | E | E           </i>
  <i>-- </i>
  SELECT c, a, b, group_concat(b, '.') OVER (
    ORDER BY c, a ROWS BETWEEN CURRENT ROW AND UNBOUNDED FOLLOWING
  ) AS group_concat
  FROM t1 ORDER BY c, a;
</codeblock>

<p> Even though they are technically part of the frame-specification, 
[exclude clause|the EXCLUDE clause] and [window chaining] are described
separately below.

<h2>The PARTITION BY Clause</h2>

<p> A <yynonterm>window-defn</yynonterm> may include a PARTITION BY clause.
If so, the rows returned by the SELECT statement are divided into groups -
partitions - with the same values for each PARTITION BY expression, and then
window-function processing is performed separately for each
................................................................................
  <i>--   one   | 7 | G | A.C.D.F.G   </i>
  <i>-- </i>
  SELECT c, a, b, group_concat(b, '.') FILTER (WHERE c!='two') OVER (
    ORDER BY a
  ) AS group_concat
  FROM t1 ORDER BY a;
</codeblock>

<h2 tags="exclude clause">The EXCLUDE Clause</h2>

<p> The optional EXCLUDE clause may take any of the following four forms:

<ul>
  <li> <p><b> EXCLUDE NO OTHERS</b>: This is the default. In this case no
  rows are excluded from the window frame as defined by its starting and ending
  frame boundaries.

  <li> <p><b> EXCLUDE CURRENT ROW</b>: In this case the current row is 
  excluded from the window frame.

  <li> <p><b> EXCLUDE GROUP</b>: In this case the current row and all other
  rows that are peers of the current row are excluded from the group. When
  processing an EXCLUDE clause, all rows with the same ORDER BY values, or all
  rows in the partition if there is no ORDER BY clause, are considered peers,
  even if the frame type is ROWS.

  <li> <p><b> EXCLUDE TIES</b>: In this case the current row is part of the
  frame, but all other peers are excluded.
</ul>

<codeblock>
  <i>-- The following SELECT statement returns:</i>
  <i>-- </i>
  <i>--   c    | a | b | no_others     | current_row | grp       | ties</i>
  <i>--  one   | 1 | A | A.D.G         | D.G         |           | A</i>
  <i>--  one   | 4 | D | A.D.G         | A.G         |           | D</i>
  <i>--  one   | 7 | G | A.D.G         | A.D         |           | G</i>
  <i>--  three | 3 | C | A.D.G.C.F     | A.D.G.F     | A.D.G     | A.D.G.C</i>
  <i>--  three | 6 | F | A.D.G.C.F     | A.D.G.C     | A.D.G     | A.D.G.F</i>
  <i>--  two   | 2 | B | A.D.G.C.F.B.E | A.D.G.C.F.E | A.D.G.C.F | A.D.G.C.F.B</i>
  <i>--  two   | 5 | E | A.D.G.C.F.B.E | A.D.G.C.F.B | A.D.G.C.F | A.D.G.C.F.E</i>
  <i>-- </i>
  SELECT c, a, b,
    group_concat(b, '.') OVER (
      ORDER BY c GROUPS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE NO OTHERS
    ) AS no_others,
    group_concat(b, '.') OVER (
      ORDER BY c GROUPS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE CURRENT ROW
    ) AS current_row,
    group_concat(b, '.') OVER (
      ORDER BY c GROUPS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE GROUP
    ) AS grp,
    group_concat(b, '.') OVER (
      ORDER BY c GROUPS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW EXCLUDE TIES
    ) AS ties
  FROM t1 ORDER BY c, a;
</codeblock>

<h2 tags="window chaining">Window Chaining</h2>

<p>
Window chaining is a shorthand that allows one window to be defined in terms 
of another. Specifically, the shorthand allows the new window to implicitly
copy the PARTITION BY and optionally ORDER BY clauses of the base window. For
example, in the following:

<codeblock>
  SELECT group_concat(b, '.') OVER (
    win ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW
  )
  FROM t1
  WINDOW win AS (PARTITION BY a ORDER BY c)
</codeblock>

<p>
the window used by the group_concat() is equivalent to "PARTITION BY a ORDER 
BY c ROWS BETWEEN UNBOUNDED PRECEDING AND CURRENT ROW". In order to use window
chaining, all of the following must be true:

<ul>
  <li><p>The new window definition must not include a PARTITION BY clause. The
       PARTITION BY clause, or lack thereof, is always copied from the base
       window.

  <li><p>If the base window has an ORDER BY clause, it is copied into the new
       window. In this case the new window must not specify an ORDER BY clause.
       If the base window has no ORDER BY clause, one may be specified as part
       of the new window definition.
  
  <li><p>The base window may not specify a frame specification.
</ul>

<p>The two fragments of SQL below are similar, but not entirely equivalent, as
the latter will fail if the definition of window "win" contains a frame
specification.

<codeblock>
  SELECT group_concat(b, '.') OVER win ...
  SELECT group_concat(b, '.') OVER (win) ...
</codeblock>

<h1>Built-in Window Functions</h1>

<p> As well as aggregate window functions, SQLite features a set of built-in
window functions based on 
<a href=https://www.postgresql.org/docs/10/static/functions-window.html>
those supported by PostgreSQL</a>.