Documentation Source Text

Check-in [393a3d19ae]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update the JSON parsing speed in the json1.html documentation.
Timelines: family | ancestors | descendants | both | branch-3.27
Files: files | file ages | folders
SHA3-256: 393a3d19ae2e7b56d1909d4225cc098c7825556473a5df9704a54c6925c1e42b
User & Date: drh 2019-03-01 13:27:27
Wiki:this checkin
About

JSON parsing performance was measured by this script:

.param init
INSERT INTO temp.[$Parameters](key,value) 
VALUES('$json',readfile('/home/drh/tmp/gsoc-2018.json'));
SELECT length(printf('{"a":%d,"b":%s}',50,$json));
.timer on
WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<1000)
SELECT DISTINCT json_valid(printf('{"a":%d,"b":%s}',x,$json)) FROM c;

WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<1000)
SELECT DISTINCT substr(printf('{"a":%d,"b":%s}',x,$json),1,5) FROM c;

It is necessary to feed slightly different JSON strings into the parser on each cycle in order to overcome the json cache. The first query (after starting the timer) measure the parser speed. The second query measure all of the extraneous non-parsing overhead of the first query. The idea is that the time used by the parser is the time of the first query minus the overhead time of the second query. Running the script above on an optimized version of SQLite on a 4-year-old Ubuntu desktop gives:

3327844
1
Run Time: real 3.218 user 3.176000 sys 0.040000
{"a":
Run Time: real 0.275 user 0.268000 sys 0.008000

So roughly 3 seconds were used to parse 3,327,844,000 bytes of JSON, which gives a parsing speed in excess of 1.1 GB/s. Round it down to an even 1 GB/s to be conservative.

Context
2019-03-04
23:49
New expensify logo. check-in: 81ea50fc28 user: drh tags: branch-3.27
2019-03-02
17:13
Merge fixes from the 3.27 branch. check-in: dcc4d55e47 user: drh tags: trunk
2019-03-01
13:27
Update the JSON parsing speed in the json1.html documentation. check-in: 393a3d19ae user: drh tags: branch-3.27
2019-02-27
23:13
Add a note to the printf() documentation to make clear that the built-in SQLite printf() does not implement positional parameters. check-in: e0c57dd3d0 user: drh tags: branch-3.27
Changes
Hide Diffs Unified Diffs Show Whitespace Changes Patch

Changes to pages/json1.in.

172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
store values that are NULL, integers, floating-point numbers, text,
and BLOBs.  It is not possible to add a sixth "JSON" type.

<p>
The json1 extension does not (currently) support a binary encoding
of JSON.  Experiments have been unable to find a binary encoding
that is significantly smaller or faster than a plain text encoding.
(The present implementation parses JSON text at over 300 MB/s.)
All json1 functions currently throw an error if any of their
arguments are BLOBs because BLOBs are reserved
for a future enhancement in which BLOBs will store the binary encoding
for JSON.

<p>
The "1" at the end of the name for the json1 extension is deliberate.







|







172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
store values that are NULL, integers, floating-point numbers, text,
and BLOBs.  It is not possible to add a sixth "JSON" type.

<p>
The json1 extension does not (currently) support a binary encoding
of JSON.  Experiments have been unable to find a binary encoding
that is significantly smaller or faster than a plain text encoding.
(The present implementation parses JSON text at over 1 GB/s.)
All json1 functions currently throw an error if any of their
arguments are BLOBs because BLOBs are reserved
for a future enhancement in which BLOBs will store the binary encoding
for JSON.

<p>
The "1" at the end of the name for the json1 extension is deliberate.