Documentation Source Text

Check-in [b4fa1baa79]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Add extra documentation for fts5 incremental optimize.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: b4fa1baa79852f0c3ea30a180ddfad51abd22673
User & Date: dan 2016-03-10 18:52:59.551
Context
2016-03-17
21:10
Fix the date (off by one) on the 3.9.2 release. (check-in: 08aa54dd95 user: drh tags: trunk)
2016-03-10
18:52
Add extra documentation for fts5 incremental optimize. (check-in: b4fa1baa79 user: dan tags: trunk)
2016-03-09
20:54
Update the fts5 documentation for the 'usermerge' option and 'merge' command. (check-in: 9ed173645f user: dan tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/fts5.in.
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222

1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
value of the parameter specified as part of the 'merge' command. The size of
each page is as configured by the [FTS5 pgsz option].

<p> If the parameter is a postive value, B-tree structures are only eligible
for merging if one of the following is true:

<ul>
  <li> There are [FTS5 usermerge option | usermerge] or more such b-trees on a
       single level (see the documentation for the [FTS5 automerge option]
       for an explanation of b-tree levels).

  <li> A merge has already been started (perhaps by a 'merge' command that
       specified a negative parameter).
</ul>

<p> It is possible to tell whether or not the 'merge' command found any 
b-trees to merge together by checking the value returned by the
[sqlite3_total_changes()] API before and after the command is executed. If
the difference between the two values is 2 or greater, then work was performed.
If the difference is less than 2, then the 'merge' command was a no-op. In this
case there is no reason to execute the same 'merge' command again, at least
until the FTS table is updated again.

<p> If the parameter is negative, and there are B-tree structures on more than
one level within the FTS index, all B-tree structures are assigned to the same
level before the merge operation is commenced. Additionally, if the parameter
is negative, the value of the usermerge configuration option is not 
respected - as few as two b-trees from the same level may be merged together.








|

|
>










|







1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
value of the parameter specified as part of the 'merge' command. The size of
each page is as configured by the [FTS5 pgsz option].

<p> If the parameter is a postive value, B-tree structures are only eligible
for merging if one of the following is true:

<ul>
  <li> There are U or more such b-trees on a
       single level (see the documentation for the [FTS5 automerge option]
       for an explanation of b-tree levels), where U is the value assigned
       to the [FTS5 usermerge option] option.
  <li> A merge has already been started (perhaps by a 'merge' command that
       specified a negative parameter).
</ul>

<p> It is possible to tell whether or not the 'merge' command found any 
b-trees to merge together by checking the value returned by the
[sqlite3_total_changes()] API before and after the command is executed. If
the difference between the two values is 2 or greater, then work was performed.
If the difference is less than 2, then the 'merge' command was a no-op. In this
case there is no reason to execute the same 'merge' command again, at least
until after the FTS table is next updated.

<p> If the parameter is negative, and there are B-tree structures on more than
one level within the FTS index, all B-tree structures are assigned to the same
level before the merge operation is commenced. Additionally, if the parameter
is negative, the value of the usermerge configuration option is not 
respected - as few as two b-trees from the same level may be merged together.

1259
1260
1261
1262
1263
1264
1265

















1266
1267
1268
1269
1270
1271
1272
<p>Refer to the documentation for the [FTS5 automerge option] for more details
regarding the relationship between the full-text index and its component
b-trees.

<codeblock>
  INSERT INTO ft(ft) VALUES('optimize');
</codeblock>


















<h2 tags="FTS5 pgsz option">The 'pgsz' Configuration Option</h2>

<p> This command is used to set the persistent "pgsz" option.

<p> The full-text index maintained by FTS5 is stored as a series of fixed-size
blobs in a database table. It is not strictly necessary for all blobs that make







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
<p>Refer to the documentation for the [FTS5 automerge option] for more details
regarding the relationship between the full-text index and its component
b-trees.

<codeblock>
  INSERT INTO ft(ft) VALUES('optimize');
</codeblock>

<p>Because it reorganizes the entire FTS index, the optimize command can 
take a long time to run. The [FTS5 merge command] can be used to divide
the work of optimizing the FTS index into multiple steps. To do this:

<ul>
  <li> Invoke the 'merge' command once with the parameter set to -N, then
  <li> Invoke the 'merge' command zero or more times with the parameter set to N.
</ul>

<p>where N is the number of pages of data to merge within each invocation of
the merge command. The application should stop invoking merge when the
difference in the value returned by the sqlite3_total_changes() function before
and after the merge command drops to below two. The merge commands may be
issued as part of the same or separate transactions, and by the same or
different database clients. Refer to the documentation for the [FTS5 merge
command | merge command] for further details.

<h2 tags="FTS5 pgsz option">The 'pgsz' Configuration Option</h2>

<p> This command is used to set the persistent "pgsz" option.

<p> The full-text index maintained by FTS5 is stored as a series of fixed-size
blobs in a database table. It is not strictly necessary for all blobs that make