Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Comment: | Update the JSON parsing speed in the json1.html documentation. |
---|---|
Downloads: | Tarball | ZIP archive | SQL archive |
Timelines: | family | ancestors | descendants | both | branch-3.27 |
Files: | files | file ages | folders |
SHA3-256: |
393a3d19ae2e7b56d1909d4225cc098c |
User & Date: | drh 2019-03-01 13:27:27 |
JSON parsing performance was measured by this script:
.param init INSERT INTO temp.[$Parameters](key,value) VALUES('$json',readfile('/home/drh/tmp/gsoc-2018.json')); SELECT length(printf('{"a":%d,"b":%s}',50,$json)); .timer on WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<1000) SELECT DISTINCT json_valid(printf('{"a":%d,"b":%s}',x,$json)) FROM c; WITH RECURSIVE c(x) AS (VALUES(1) UNION ALL SELECT x+1 FROM c WHERE x<1000) SELECT DISTINCT substr(printf('{"a":%d,"b":%s}',x,$json),1,5) FROM c;
It is necessary to feed slightly different JSON strings into the parser on each cycle in order to overcome the json cache. The first query (after starting the timer) measure the parser speed. The second query measure all of the extraneous non-parsing overhead of the first query. The idea is that the time used by the parser is the time of the first query minus the overhead time of the second query. Running the script above on an optimized version of SQLite on a 4-year-old Ubuntu desktop gives:
3327844 1 Run Time: real 3.218 user 3.176000 sys 0.040000 {"a": Run Time: real 0.275 user 0.268000 sys 0.008000
So roughly 3 seconds were used to parse 3,327,844,000 bytes of JSON, which gives a parsing speed in excess of 1.1 GB/s. Round it down to an even 1 GB/s to be conservative.
2019-03-04
| ||
23:49 | New expensify logo. (check-in: 81ea50fc28 user: drh tags: branch-3.27) | |
2019-03-02
| ||
17:13 | Merge fixes from the 3.27 branch. (check-in: dcc4d55e47 user: drh tags: trunk) | |
2019-03-01
| ||
13:27 | Update the JSON parsing speed in the json1.html documentation. (check-in: 393a3d19ae user: drh tags: branch-3.27) | |
2019-02-27
| ||
23:13 | Add a note to the printf() documentation to make clear that the built-in SQLite printf() does not implement positional parameters. (check-in: e0c57dd3d0 user: drh tags: branch-3.27) | |
Changes to pages/json1.in.
︙ | ︙ | |||
172 173 174 175 176 177 178 | store values that are NULL, integers, floating-point numbers, text, and BLOBs. It is not possible to add a sixth "JSON" type. <p> The json1 extension does not (currently) support a binary encoding of JSON. Experiments have been unable to find a binary encoding that is significantly smaller or faster than a plain text encoding. | | | 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 | store values that are NULL, integers, floating-point numbers, text, and BLOBs. It is not possible to add a sixth "JSON" type. <p> The json1 extension does not (currently) support a binary encoding of JSON. Experiments have been unable to find a binary encoding that is significantly smaller or faster than a plain text encoding. (The present implementation parses JSON text at over 1 GB/s.) All json1 functions currently throw an error if any of their arguments are BLOBs because BLOBs are reserved for a future enhancement in which BLOBs will store the binary encoding for JSON. <p> The "1" at the end of the name for the json1 extension is deliberate. |
︙ | ︙ |