SQLite

Timeline
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

143 events for the month beginning 2023-07-01 by user drh

Following month ↑
2023-07-31
22:03
Restrict the new column cache to table-btree, which is the common case anyhow. That way, writes to indexes do not need to clear the column cache. (check-in: 659284ab user: drh tags: trunk)
20:02
The original column-cache implementation from check-in [ab1edcc7fedcf279] (merged to trunk at [771fe35074b50b8d]) is unsound. This check-in fixes the issue. Had to give back a little performance, the optimization is still a overall win. (check-in: ec95e970 user: drh tags: trunk)
17:39
Add bit to sqlite3_test_control(SQLITE_TESTCTRL_OPTIMIZATIONS) that will disable the one-pass optimizating in DELETE and UPDATE. (check-in: 22f6dd5c user: drh tags: trunk)
17:20
Improved debug tracking of the Parse.nQueryLoop value. (check-in: 4a8a1749 user: drh tags: trunk)
2023-07-29
22:58
Do not try to add MEM_Term to an MEM_Static string either. (check-in: 8920dcef user: drh tags: trunk)
21:22
Do not try to apply MEM_Term to a MEM_Ephem strings. (check-in: 2a378a66 user: drh tags: trunk)
17:05
Simplification to the sqlite3_stmt_explain() implementation. (check-in: dd5eb4c7 user: drh tags: trunk)
15:31
Enhancements to the new sqlite3_stmt_explain() interface implementation and documentation. (check-in: dc98ac6b user: drh tags: trunk)
2023-07-28
20:59
Do not allow the same JSON parse to be input to both arguments to the json_patch() routine. (check-in: 8e781c67 user: drh tags: trunk)
19:07
Merge the latest trunk enhancements into the wal-shm-exceptions branch. (check-in: c44fb4d0 user: drh tags: wal-shm-exceptions)
19:00
Merge the latest trunk enhancements into the reuse-schema branch. (check-in: b8731493 user: drh tags: reuse-schema)
18:52
Merge the latest trunk enhancements into the bedrock branch. (check-in: 1d6ded32 user: drh tags: bedrock)
18:47
Merge the latest trunk enhancements into the wal2 branch. (check-in: c1fa1cb5 user: drh tags: wal2)
18:43
Merge the latest trunk enhancements into the begin-concurrent branch. (check-in: 7407dda8 user: drh tags: begin-concurrent)
18:37
Add the sqlite3_stmt_explain() API. (check-in: 0443c0ef user: drh tags: trunk)
16:12
Minor tweaks to the JSON performance measurement documentation. (check-in: 0bed957e user: drh tags: trunk)
15:58
Fix minor typos in the JNI README.md file. (check-in: e21cf2e8 user: drh tags: jni)
14:20
Enhancements to the JSON performance testing scripts and instructions to cover the recent optimizations involving updates to large JSON strings that indexed. (check-in: f9213289 user: drh tags: trunk)
13:52
JSON performance improvements: (1) Add a cache for large string values on OP_Column to avoid having to extract the string multiple times. (2) Internal JSON caching improvements. (check-in: 771fe350 user: drh tags: trunk)
12:59
A failed attempt to add a new sqlite3_result_zeroterminated() interface that is a hint to SQLite that a TEXT result does have a zero terminator. The idea is to avoid unnecessary copying of TEXT values. This seems like an unnecessary complication at the moment, but maybe I'll return to this later. Note that this check-in does not work - it is a proof of concept only. (Closed-Leaf check-in: a0de0110 user: drh tags: sqlite3_result_zeroterminated)
11:30
Mark an unreachable branch as ALWAYS(). (Closed-Leaf check-in: 8fd06d11 user: drh tags: json-opt)
00:54
Remove some unnecessary and incorrect code that was mistakenly added in the previous check-in. (check-in: d3f458dc user: drh tags: json-opt)
2023-07-27
23:51
Switch to using jsonParseCached() for json_patch(). (check-in: 2ed9c59e user: drh tags: json-opt)
20:28
If the input JSON to a json function that uses cache comes from an RCStr value, then use that RCStr value in the parse rather than making a copy. (check-in: 509ae9c1 user: drh tags: json-opt)
19:39
The OP_Column opcode caches large column values coming from overflow pages. (check-in: ab1edcc7 user: drh tags: json-opt)
18:19
Merge trunk enhancements into the json-opt branch. (check-in: 5739a16a user: drh tags: json-opt)
00:23 Edit [837f2907e10b026f|837f2907]: Edit check-in comment. (artifact: 54d1082b user: drh)
00:21
Fix a performance regression in JSON associated with generating small snippets of JSON from a larger JSON string. See forum thread 15d6bd9cd37202a7. (check-in: 837f2907 user: drh tags: trunk)
2023-07-26
23:22
Reduce the number of memory allocations when parsing JSON. (check-in: 9edd6716 user: drh tags: trunk)
21:53
Minor changes to make coverage testing easier. (check-in: ec8b4338 user: drh tags: trunk)
19:22
Change a switch() case to default for coverage. (check-in: 04f49707 user: drh tags: trunk)
19:11
Make sure jsonReplaceNode() always leaves the JsonParse in a consistent state even if an error is encountered. (check-in: 01d52232 user: drh tags: trunk)
19:08 Edit [29c7b1cbc4349a0d|29c7b1cb]: Move to branch mistake. Mark "Closed". (artifact: 956f7b54 user: drh)
18:13
Make sure jsonReplaceNode() always leaves the JsonParse in a consistent state even if an error is encountered. (Closed-Leaf check-in: 29c7b1cb user: drh tags: mistake)
14:18
Improved caching and reuse of JSON parses. (check-in: df099ad7 user: drh tags: trunk)
13:17
Change the debugging "json_parse(X)" function so that it shows a more complete description of the JsonParse object on standard output and returns the mimified JSON. Former behavior was to return the text of a decode of the aNode array. (check-in: 1bf85d4e user: drh tags: json-opt)
11:53
More comment improvements in json.c. Do not run jsonLookup() following an OOM error. (check-in: cd5fda8c user: drh tags: json-opt)
11:43
Improved comments and other cleanup for the changes on this branch. (check-in: bac953a8 user: drh tags: json-opt)
11:00
Fix jsonForceRCStr() to also add the NULL terminator. (check-in: 134b01f3 user: drh tags: json-opt)
01:15
Merge the latest trunk fixes into the json-opt branch. (check-in: ef4e1664 user: drh tags: json-opt)
01:12
Add the concept of an RCStr string - a "Reference Counted String". Use these to keep track of big JSON strings that might need to be used in multiple places at the same time with different lifetimes. Add two different JSON string values to each parse - one with and one without changes. The net result is a large reduction in the number of parses that occur when doing an UPDATE of a indexed JSON field using json_replace() or similar. (check-in: a4c1af61 user: drh tags: json-opt)
01:05
Extend the enhancement to json_set() and json_replace(). Clean up cruft. (Closed-Leaf check-in: 2dbb22c7 user: drh tags: json-opt-rcstr)
00:48
Add the JsonParse.zAlt field to old revised JSON text after a change. Demonstrate that this elminates the need for reparsing after a change by using it in the json_remove() function. This is an incremental check-in containing lots of cruft. (check-in: f930b139 user: drh tags: json-opt-rcstr)
2023-07-25
20:26
Clarify ownership of the various objects involved in parsing JSON. (check-in: afe02a39 user: drh tags: json-opt-rcstr)
18:28
Incremental improvements to JSON parsing - trying to fold in the RCStr object. (check-in: 4cb15d93 user: drh tags: json-opt-rcstr)
15:43
Fix a minor problem with error reporting in JSON. (check-in: c456e4a8 user: drh tags: json-opt-rcstr)
15:08
Create the new RCStr class of strings and try to use them for JSON storage. (check-in: c1b87250 user: drh tags: json-opt-rcstr)
11:43 Edit [075eb09a4f15c023|075eb09a]: Move to branch json-opt-wcache1. (artifact: eb02784c user: drh)
00:13
First attempt to cache modified JSON parses. Does not work. This is an incremental check-in. (Leaf check-in: 075eb09a user: drh tags: json-opt-wcache1)
2023-07-24
23:27
It is an error to try to insert a BLOB value into JSON. (check-in: 28c6e964 user: drh tags: json-opt)
22:45
Clean up some #defines in json. (check-in: 327fff50 user: drh tags: json-opt)
22:34
Rework the JSON parse structure to facilitate better caching. Passes all tests. (check-in: ecdcb1de user: drh tags: json-opt)
17:59
Incremental progress toward improved caching of parsed JSON. (check-in: f2c06388 user: drh tags: json-opt)
13:30
Merge the latest trunk enhancements into the reuse-schema branch. (check-in: e0e7b01e user: drh tags: reuse-schema)
13:24
Merge the latest trunk enhancements into the bedrock branch. (check-in: a6f34439 user: drh tags: bedrock)
13:20
Merge the latest trunk enhancements into the wal2 branch. (check-in: b2e56f62 user: drh tags: wal2)
13:15
Merge the latest trunk enhancements into the begin-concurrent branch. (check-in: 967be041 user: drh tags: begin-concurrent)
12:59
Update the latest trunk enhancements into the wal-shm-exceptions branch. (check-in: 3187ee3f user: drh tags: wal-shm-exceptions)
12:37
Merge the latest trunk enhancements into the json-opt branch. (check-in: 00bfc491 user: drh tags: json-opt)
2023-07-22
16:37
Do not read past the end of a text buffer looking for a zero terminator, as that space might not be initialized. If the buffer is owned, just set the null terminator. This is a better fix for the OSSFuzz-detected use-of-initialized-value problem. (check-in: 931bccb0 user: drh tags: trunk)
15:21
Fix a harmless use-of-initialized-value warning from OSSFuzz that results from recent large string optmizations. (check-in: 1a6b3dd1 user: drh tags: trunk)
2023-07-21
22:28
Performance enhancements for queries using very large string values. (check-in: 3661b9cd user: drh tags: trunk)
22:22
Mark an unreachable branch as NEVER(). (Leaf check-in: 414a4d66 user: drh tags: big-function-text)
19:17
Fix harmless compiler warning introduced by the previous check-in. (check-in: 06f829e9 user: drh tags: big-function-text)
18:38
Avoid a potentially large strlen() in sqlite3DecOrHexToI64(). (check-in: 5413b02b user: drh tags: big-function-text)
18:09
Further improvements to large string handling in relation to JSON. (check-in: 1e5df0aa user: drh tags: big-function-text)
15:15 Edit [d0278cdedfa04fb0|d0278cde]: Move to branch big-function-text. (artifact: 5c6fafed user: drh)
15:07
Fix harmless compiler warning about signed/unsigned comparison. (check-in: 75cc3c89 user: drh tags: big-function-text)
15:01
Multiple optimizations that try to preserve or infer the zero-terminated property of TEXT values. Avoid unnecessary copying of text values destined to become function parameters. All changes help improve performance of doing UPDATEs on large JSON values that are indexed multiple ways. (check-in: d0278cde user: drh tags: big-function-text)
11:09
More infrastructure changes towards improving JSON cache performance. Incremental check-in. (check-in: 1955e66c user: drh tags: json-opt)
2023-07-20
17:45
Experimental framework upon which to build a better JSON parse structure that supports cached of modified JSON. All of these changes are tentative and subject to change or removal. Incremental check-in. (check-in: e384163a user: drh tags: json-opt)
2023-07-19
23:02
Performance enhancements to the JSON parser and code generator. (check-in: 0a745897 user: drh tags: trunk)
20:23
Revise the new JSON parser performance test to make the test run 25 times longer, and thus provide a more repeatable number. (check-in: bee9e403 user: drh tags: trunk)
19:23
Describe JSON parser performance testing procedure in test/json/README.md (check-in: c47056e1 user: drh tags: trunk)
17:24
Further improvement to JSON parser performance. (check-in: 144c8ccf user: drh tags: json-opt)
15:06
Further performance enhancements to JSON parsing and rendering. Total performance gain for large JSONs so far on this branch is about 11%. (check-in: adb4d6b0 user: drh tags: json-opt)
14:42 Edit [c9fbe0185cd5d649|c9fbe018]: Move to branch json-opt. (artifact: c3312332 user: drh)
13:50
Performance optimization for parsing large JSONs that contain lots of text. (check-in: c9fbe018 user: drh tags: json-opt)
12:52 Edit [2e6fbebc41f1a24f|2e6fbebc]: Mark "Closed". (artifact: 4cc487d4 user: drh)
12:52
Enhance the JSON parser cache such that it is able to extract lines from the cache and use them for writing, though they then have to be evicted from the cache. This was an experiment in trying to reduce the amount of parsing needed for a big UPDATE, but it does not seem to help any. Retained for reference only. (Closed-Leaf check-in: 2e6fbebc user: drh tags: json-write-cache)
10:17
Merge the latest trunk enhancements into the sqlite3_stmt_explain branch. (Closed-Leaf check-in: 06eae046 user: drh tags: sqlite3_stmt_explain)
09:52
Two minor improvements to sum(), one of which was inspired by forum post af5be98dbc. (check-in: a0d3e757 user: drh tags: trunk)
2023-07-18
21:06
Do not use the viewscan optimization on a query that has only a single loop, as the cost adjustments can cause problems for outer queries. Proposed fix for the performance regression reported by forum post 64d36440e473516c. (check-in: 76152ad2 user: drh tags: trunk)
20:41
Improved ".wheretrace" for the VIEWSCAN optimization. (check-in: 27057ea7 user: drh tags: trunk)
17:43
Merge the FTS5 memory leak fix from trunk. (check-in: fb65cb73 user: drh tags: fts5-contentless-delete)
17:29 Edit [c2fba6a6322bff10|c2fba6a6]: Mark "Closed". (artifact: 1faa440f user: drh)
17:29
Enhance the sqlite3_stmt_explain() interface so that avoids unnecessary reprepare operations. (check-in: 050f773a user: drh tags: sqlite3_stmt_explain)
16:06 Edit [609fbb94b8f01d67|609fbb94]: Edit check-in comment. (artifact: 31045735 user: drh)
15:06
Experimental (untested, non-working) changes that try reduce the number of reprepares generated by sqlite3_stmt_explain(). I think I see an easier way to do this now, so I'm parking this experiment on a branch to pursue the new idea. (Closed-Leaf check-in: c2fba6a6 user: drh tags: sqlite3-stmt-explain-opt1)
2023-07-17
16:20 Edit [ce6793e954f291b6|ce6793e9]: Edit check-in comment. (artifact: d82b1197 user: drh)
16:18 Edit [ce6793e954f291b6|ce6793e9]: Edit check-in comment. (artifact: ebd2d990 user: drh)
2023-07-15
16:48
Add the experimental sqlite3_stmt_explain(S,E) interface. (check-in: 5683743d user: drh tags: sqlite3_stmt_explain)
2023-07-13
14:49
Fix an off-by-one error in tests for cells overflowing the end of pages. The error is completely harmless for the default page cache, but might cause false-positive SQLITE_CORRUPT errors in systems that use an application-defined page caches that pack pages tightly together in memory. (check-in: ce6793e9 user: drh tags: trunk)
2023-07-12
18:38
Get access to SQL functions and virtual tables used for debugging and analysis of FTS5 using SQLITE_FTS5_DEBUG and without the need for SQLITE_TEST. (check-in: 383de8e2 user: drh tags: fts5-contentless-delete)
2023-07-11
19:54
Reduce divergence between branches: trunk and wal-shm-exceptions. (check-in: d38ed6b1 user: drh tags: trunk)
15:52
Get fault injection for testing working correctly. Other code clean-up. (check-in: ff492277 user: drh tags: wal-shm-exceptions)
2023-07-10
19:13
Compile for SEH if the USE_SEH=1 option is specified on the nmake command line. (check-in: 0e67a82f user: drh tags: wal-shm-exceptions)
18:16
Fix a warning. Unclear yet if the code is correct this way or not. (check-in: 2df4fd76 user: drh tags: wal-shm-exceptions)
18:05
Merge the latest trunk enhancements into the wal-shm-exceptions branch. (check-in: f655d08d user: drh tags: wal-shm-exceptions)
2023-07-08
19:28
Merge the latest trunk enhancements into the reuse-schema branch. (check-in: 7a9cb6f4 user: drh tags: reuse-schema)
19:16
Merge the latest trunk enhancements into the bedrock branch. (check-in: 35bfb5bf user: drh tags: bedrock)
19:09
Merge all the latest trunk enhancements into the wal2 branch. (check-in: f0793653 user: drh tags: wal2)
19:00
Merge the latest trunk enhancements into the begin-concurrent branch. (check-in: 3d0823c4 user: drh tags: begin-concurrent)
17:42
Always use the "LL" suffix on 64-bit integer literals. (check-in: 07d95ed6 user: drh tags: trunk)
14:27
Fix harmless scan-build warnings. (check-in: beab3c98 user: drh tags: trunk)
2023-07-07
18:49
Omit the use of memmove() in sqlite3FpDecode() in an attempt to avoid spurious warnings from valgrind. Also makes the code slightly smaller and faster. (check-in: b409943a user: drh tags: trunk)
16:58
Proof of concept for a new sqlite3_function_needed() interface. (Leaf check-in: cd67edc0 user: drh tags: need-function)
12:18
For the "Valgrind" test scenario, set LONGDOUBLE_TYPE=double as valgrind does not correctly emulated extended precision on x64. (check-in: d3532eae user: drh tags: trunk)
12:08
Add option 2 to SQLITE_TESTCTRL_USELONGDOUBLE to move the setting to its defaults. Enhance the CLI and testfixture to make use of this value. Base the default on sizeof(LONGDOUBLE_TYPE) rather than on sizeof(long double). (check-in: 64e6bd1c user: drh tags: trunk)
2023-07-06
20:34
Improved handling of integer inputs to the new sum() implementation. (check-in: 31561c63 user: drh tags: trunk)
18:29
A more precise implementation of sqlite3RealToI64(). (check-in: 8813317c user: drh tags: trunk)
17:55
Merge all recent trunk enhancements into the wal-shm-exceptions branch. (check-in: e59cea2d user: drh tags: wal-shm-exceptions)
17:14
Fix octet_length() test cases so that they work when the database encoding is utf16. (check-in: ad2e3962 user: drh tags: trunk)
15:44
Use the Kahan-Babushka-Neumaier algorithm to improve the accuracy of sum(). (check-in: c63e26e7 user: drh tags: trunk)
14:45
Fix minor inaccuracies in the new sum() algorithm. (Closed-Leaf check-in: e3f7a960 user: drh tags: kahan-babuska-neumaier-summation)
13:19
Experimental attempt to boost the accuracy of sum() using the Kahan-Babuska-Neumaier algorithm. (check-in: ebc5edd3 user: drh tags: kahan-babuska-neumaier-summation)
00:55
Use "volatile" isntead of "#pragma" to get floating point calculations working correctly when compiling with GCC for x86 machines. (check-in: 9427f426 user: drh tags: trunk)
2023-07-05
22:05
This is alternative work-around for the x86 float-point problem that uses "volatile" rather than compile-specific #pragmas. (Closed-Leaf check-in: 1d972a69 user: drh tags: fix-dekker-with-volatile)
19:56
Improved comments on the work-around to the GCC x86 floating point wonkiness. (check-in: 7b4c1673 user: drh tags: trunk)
18:59
Work around for an apparent GCC bug. See forum post ee7278611394034c for details. (check-in: 5d9e9364 user: drh tags: trunk)
15:34
Make the order of operations explicit for the error term of Dekker double-precision multiply. (check-in: 28f57b34 user: drh tags: trunk)
14:42
Simplification of the dekkerMul2() subroutine. (check-in: bfe6432a user: drh tags: trunk)
12:00
Add the ieee754_inc(r,N) function to the ieee754.c extension. (check-in: 9a36f269 user: drh tags: trunk)
2023-07-03
18:32
Enhance the sqlite3AtoF() function so that it can operate with or without long double and provides accurate results in both cases. (check-in: fd987508 user: drh tags: trunk)
18:18
Enhance sqlite3AtoF() so that it honors the USELONGDOUBLE test-control. Enable the test/atof1.test module on non-intel platforms. (Closed-Leaf check-in: b7850fd4 user: drh tags: fp-to-decimal-refactor)
15:30
Store the fp-speed-1.c test program used to measure relative performance of SQLite versus the standard library, in the test/ directory. A header comment explains how to use it. (check-in: eaff3762 user: drh tags: trunk)
15:11
Add the --nolongdouble option to speedtest1.c. (check-in: 6bf6018a user: drh tags: trunk)
15:03
Improved precision of binary-to-decimal conversion of floating-point values on all systems but especially non-x86 systems that lake extended precision hardward floating point. (check-in: 32befb22 user: drh tags: trunk)
14:56
Whether or not to use the long-double floating point conversion technique is now a test-control setting. (check-in: 02ae6caf user: drh tags: fp-to-decimal-refactor)
12:58
New test cases for floating-point conversions. (check-in: 9435ed76 user: drh tags: fp-to-decimal-refactor)
11:44
Use Dekker double-precision floating point to do the value normalization step in sqlite3FpDecode() for systems on which "long double" is 8 bytes or less. (check-in: 51f492b4 user: drh tags: fp-to-decimal-refactor)
11:39
Final clean-up of the idea of using Dekker double-precision floating computations to improve accuracy. (Closed-Leaf check-in: 40285d63 user: drh tags: fp-to-decimal-branch1)
10:35
Squeeze as many bits as we can out of normalization. (check-in: 25d1ad6d user: drh tags: fp-to-decimal-branch1)
10:18
Faster Dekker multiplication that removes the restriction on input magnitude. (check-in: 2994caf5 user: drh tags: fp-to-decimal-branch1)
10:00
Fix one constant in the normalization logic. Improved error output from atof1.test. (check-in: d3c48807 user: drh tags: fp-to-decimal-branch1)
09:07
Replace the dodgy error estimating logic in the previous check-in with full-up Dekker double-double multiplication, and this idea works much better. There are still a few inaccuracies, but it is much closer. (check-in: 4fa6938d user: drh tags: fp-to-decimal-branch1)
00:51
The fptest(X) routine used for exploring another idea for improving fp-to-decimal conversion without the use of long double. (Leaf check-in: d9a883d7 user: drh tags: fp-to-decimal-branch2)
00:40
An idea for improving accuracy of fp-to-decimal conversion for systems that do not have "long double". Does not quite work. Retained only for documentation purposes. (check-in: 70224333 user: drh tags: fp-to-decimal-branch1)
2023-07-01
20:21
This check-in adds the use of "long double" to sqlite3FpDecode() which when run on x86 using a compiler that translates "long double" into binary80, causes all legacy tests to pass. But on a system where "long double" is just an alias for "double", some of the tests are still failing. (check-in: ce06982f user: drh tags: fp-to-decimal-refactor)
18:33
Reinstate the fpdecode() SQL function for testing, but only when compiled with SQLITE_DEBUG. (check-in: 07eab52e user: drh tags: fp-to-decimal-refactor)
17:56
Improved rounding policy. (check-in: 6f1122e9 user: drh tags: fp-to-decimal-refactor)
15:23
Improved response to Infinity and NaN. Remove the termporary fpdecode() function. (check-in: 76ab8ae8 user: drh tags: fp-to-decimal-refactor)
14:03
Progress towards fixing corner-case floating-point formatting issues. (check-in: 1138b8d1 user: drh tags: fp-to-decimal-refactor)
Previous month ↓