/ Timeline
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

100 most recent check-ins

2022-09-24
19:54
Add the SQLITE_RECOVER_SLOWINDEXES option, for specifying that indexes should be created and populated along with tables, instead of separately at the end of the recovery operation. (Leaf check-in: ad9dba9d user: dan tags: recover-extension)
19:17
Add sqlite3_recover_step() to header file sqlite3recover.h. Update the API docs in this file. (check-in: 47f41615 user: dan tags: recover-extension)
18:05
Update things to use sqlite3_recover_step() internally. (check-in: f4b15aad user: dan tags: recover-extension)
11:32
Fiddle: move, rather than copy, data buffers between the threads. Fix the case of an uploaded db failing to install because its filename is the same as the opened db. (Leaf check-in: bcec4f96 user: stephan tags: fiddle-opfs)
10:15
Resolve "already configured" warnings from shell's main() when starting it up in fiddle mode. (check-in: 114ef355 user: stephan tags: fiddle-opfs)
10:12
Reworked out the OPFS async proxy metrics are fetched so that they play more nicely with the tight event-polling loop. (check-in: ef503ced user: stephan tags: fiddle-opfs)
07:36
Refactoring towards getting fiddle to support OPFS as a first-class citizen. Certain operations, e.g. import, export, and unlink, are not OPFS-aware. (check-in: 1b923ed6 user: stephan tags: fiddle-opfs)
2022-09-23
21:01
Internal changes to the recover extension towards a step() style interface. (check-in: 73926d5c user: dan tags: recover-extension)
11:40
Fix various compiler warnings in new code on this branch. (check-in: ae49e9ef user: dan tags: recover-extension)
11:30
Merge latest trunk changes into this branch. (check-in: e87fa70a user: dan tags: recover-extension)
2022-09-21
20:24
Reformulate some JS to work around a buggy/broken code transformation in one of the Emscripten-driven code optimizers. (check-in: e1249369 user: stephan tags: fiddle-opfs)
19:51
Put pieces in place for fiddle to support opfs, but more cleanup is required in the sqlite3.js/fiddle connection. bigIntEnabled now defaults to whether the Emscripten's module has bigint enabled. Add wasm-sensible defaults for several SQLITE_ENABLE/OMIT flags in sqlite3-wasm.c. (check-in: 7c7fb753 user: stephan tags: fiddle-opfs)
18:21
Reduced and improved comments on the constraintCompatibleWithOuterJoin() function. (Leaf check-in: 38aaf26e user: drh tags: trunk)
16:21
shell.c.in: when building in fiddle mode, define _POSIX_SOURCE (ifndef) so that emcc's string.h reveals strdup(). (check-in: fb85b269 user: stephan tags: fiddle-opfs)
14:02
Correct mistyped --shrink-memory flag in speedtest1-worker. Minor OPFS proxy cleanups. (check-in: 86e2b55e user: stephan tags: fiddle-opfs)
12:27
Doc cleanups and additions. Add a way for the OPFS async runner to propagate exception text to the calling thread. (check-in: 5c5e8065 user: stephan tags: fiddle-opfs)
12:25
Added some structure to the wasm demo link list. (check-in: 777077c4 user: stephan tags: fiddle-opfs)
09:54
The RIGHT JOIN logic added to version 3.39.0 added a new restriction on the query flattener which resulted in a performance regression for some queries, as documented by forum post 96b9e5709cf47cda. This change removes that restriction (thus restoring performance) and fixes the problem that RIGHT JOIN was having in a different way. (check-in: 501609ed user: drh tags: trunk)
08:41
Add a link to fiddle.html in the wasm test app index.html. (check-in: 96c734c0 user: stephan tags: fiddle-opfs)
08:39
Move fiddle build rules into the wasm-centric build files. Add rule to push wasm bits to the wasm test server. (check-in: 113f8204 user: stephan tags: fiddle-opfs)
00:16
Break out the decision of whether or not a constraint term is usable by an outer join into a subroutine: constraintCompatibleWithOuterJoin(). (Closed-Leaf check-in: c7fbc9b0 user: drh tags: flattener-omit-restriction-29)
2022-09-20
22:19
Do not allow EP_InnerON terms to be used on a outer join. (check-in: f47aa745 user: drh tags: flattener-omit-restriction-29)
21:07
Add new test cases that (current) get an incorrect answer. (check-in: fe5c3773 user: drh tags: flattener-omit-restriction-29)
19:45
New test cases for query flattening when there are ON clauses and outer joins. (check-in: db55387b user: drh tags: flattener-omit-restriction-29)
19:22
Modify the OP_IfNotOpen opcode so that the jump is taken if the cursor is open but was previously set to a NULL row using OP_NullRow. (check-in: 1292d68c user: drh tags: flattener-omit-restriction-29)
18:18
Remove code that has been previously commented out. (check-in: 852b385a user: drh tags: flattener-omit-restriction-29)
17:43
Merge the fuzzcheck fix from trunk into the flattener improvement branch. (check-in: 930a8c15 user: drh tags: flattener-omit-restriction-29)
17:21
Update the fuzz invariant checker to conform to the latest changes in dbsqlfuzz. (check-in: 8d8c1243 user: drh tags: trunk)
16:57
Is the query flattener restriction 29 (do not allow both EP_InnerON and EP_OuterON constraints on the same FROM clause term) really necessary? This branch explores what happens if we omit that restriction. This is an effort to address the performance regression reported by forum post 96b9e5709cf47cda that is caused by flattener restriction 29. (check-in: 19270577 user: drh tags: flattener-omit-restriction-29)
16:20
Correct a too-strict is-opfs-available check. (check-in: 1b5f1b4a user: stephan tags: fiddle-opfs)
16:10
Merge kv-vfs-magic-names branch into fiddle-opfs branch and make some kvvfs-relevant tweaks. (check-in: e3d36dcd user: stephan tags: fiddle-opfs)
14:52
Export the sqlite3_uri_...() family of functions to wasm. (check-in: 72bebc84 user: stephan tags: fiddle-opfs)
14:39
Merge opfs-proxy-atomics branch into the fiddle-opfs branch. (check-in: 25a36920 user: stephan tags: fiddle-opfs)
14:36
When compiled with SQLITE_OS_KV_OPTIONAL, the magic names ":localStorage:" and ":sessionStorage:" are recognized and converted to use the kv-vfs. (Leaf check-in: c5db9262 user: drh tags: kv-vfs-magic-names)
14:21
Remove an obsolete file. (Closed-Leaf check-in: 1e438554 user: stephan tags: opfs-proxy-atomics)
13:25
Speed up de/serialization of func args and return values in the OPFS VFS proxy. (check-in: 5bf235bb user: stephan tags: opfs-proxy-atomics)
10:47
OPFS proxy: remove one sanity-checking-only level of proxy function to shave off a few microseconds. (check-in: b534831f user: stephan tags: opfs-proxy-atomics)
10:11
Remove extraneous result-code SAB slots. Doc additions. (check-in: 88de2071 user: stephan tags: opfs-proxy-atomics)
08:27
An alternative messaging strategy for the OPFS VFS proxy which uses only SharedArrayBuffer and Atomics, instead of worker messages, for communication (only the initial one-time handshake during initialization uses worker messages). It runs speedtest1 approx. 15-20% faster but still 20-ish% slower than WASMFS Edit: further tests showed it to be very close to the same speed as the Worker-message variant, inconsistently faster or slower within a small margin of error. The former impl is easier to understand, though. (check-in: a83ee308 user: stephan tags: opfs-proxy-atomics)
03:31
OPFS VFS: further internal refactoring towards experimenting with a new comms model. (check-in: 5ca412ce user: stephan tags: fiddle-opfs)
01:28
OPFS VFS: moved i/o buffer from per-file to the VFS, and related refactoring, in prep for experimentation with a new inter-worker comms model. (check-in: d4d63e45 user: stephan tags: fiddle-opfs)
2022-09-19
19:25
Clarify comments that describe query flattener restrictions associated with outer joins. No code changes. (check-in: 2647d73b user: drh tags: trunk)
18:22
Further metrics and buffer-copy optimizations in the OPFS proxy, but with little effect. (check-in: d1f1fe6f user: stephan tags: fiddle-opfs)
17:41
Fix a minor comment typo. No changes to code or documentation. (check-in: 8032db4a user: drh tags: trunk)
17:09
Cut the speedtest1 runtime of the OPFS VFS proxy by approximately 3/4ths via xRead/xWrite buffer-copying optimizations. Still slower than the WASMFS impl by approx. 1/5th. (check-in: fb7f2873 user: stephan tags: fiddle-opfs)
14:56
Remove an obsolete file. Update wasmfs build for today's changes. Fix the case that OPFS.xTruncate() is passed a BigInt instead of a Number. (check-in: 97605392 user: stephan tags: fiddle-opfs)
13:44
Add sqlite3_web_vfs_list() to JS API. Corrected OPFS VFS's registering itself as the default VFS. speedtest1-worker now uses the xDelete() of both the default VFS and OPFS, to avoid that it starts up with a persistent OPFS test db (the native app calls unlink(), but that unlink call operates on a different virtual filesystem than the OPFS VFS). (check-in: 2ec7e091 user: stephan tags: fiddle-opfs)
13:16
Merge kv-vfs branch into fiddle-opfs. Adjust various JS APIs and apps to deal with the new method of handling kvvfs. Adjust speedtest1 JS build to include sqlite3-api.js so that it can use kvvfs and opfs VFSes. Permit passing of the vfs as a URL parameter to certain demo/test apps. Milestone: speedtest-worker.html?vfs=opfs runs with the standalone OPFS impl. (check-in: ec09f32f user: stephan tags: fiddle-opfs)
11:47
speedtest1: add --vfs NAME flag and eliminate code duplication in argc size verification for all flags which take a value. (check-in: 281d0986 user: stephan tags: fiddle-opfs)
10:13
Minor doc improvements in demo-123.js. (check-in: 9370ce94 user: stephan tags: fiddle-opfs)
09:58
Generic cleanups in the OPFS VFS proxies. (check-in: 7d5f6adc user: stephan tags: fiddle-opfs)
09:25
Correct OPFS VFS xRead() to copy the result buffer if the result code is SQLITE_IOERR_SHORT_READ. (check-in: 56668f99 user: stephan tags: fiddle-opfs)
05:19
Replace OPFS VFS xSleep() impl with a more efficient one (no Worker round-trip needed). (check-in: b9773f16 user: stephan tags: fiddle-opfs)
03:57
Rename demo-oo1.* to demo-123.* and add demo-123-worker.html, which runs the same demo via a Worker. Doc typo fixes. (check-in: 2e4a005b user: stephan tags: fiddle-opfs)
00:40
Disable build of wasmfs-using components on aarch64 (ARM), as wasmfs won't build there. (check-in: 9a9eeebc user: stephan tags: fiddle-opfs)
2022-09-18
17:59
Remove a NEVER macro in defragmentPage() that dbsqlfuzz discovered can be true. crash-32d9312f145cdce41613573f6431d9a3e439e3d7 (check-in: 90403759 user: drh tags: trunk)
17:32
Numerous cleanups in the JS bits. Removed some now-defunct wasm test files. Expose sqlite3.opfs object containing various OPFS-specific utilities. (check-in: 26e625d0 user: stephan tags: fiddle-opfs)
03:05
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. Rename x-sync-async.* to test-opfs-vfs.* Milestone: first successful test of OPFS without WASMFS. (check-in: b2abf60d user: stephan tags: fiddle-opfs)
02:35
Move the OPFS VFS bits back into api/sqlite3-api-opfs.js. Refactor the OPFS VFS init process to use a Promise-returning function which the client must call, as that eliminates any uncertainty about when the VFS (necessarily activated asynchronously) actually becomes available to the client. (check-in: 1c660970 user: stephan tags: fiddle-opfs)
00:16
Plug OPFS methods in to their sqlite3_vfs/io_methods counterparts. Add URL args to control debug output and running of sanity-checks in the OPFS init code. (check-in: a0e93ed2 user: stephan tags: fiddle-opfs)
2022-09-17
23:47
Correct OPFS xRead() and xWrite() impls and add a very basic sanity test for them. (check-in: cd06cc67 user: stephan tags: fiddle-opfs)
23:29
Implement OPFS xAccess(), albeit with more limited semantics than the VFS API calls for. Add a way for OPFS xDelete() to optionally recursively remove empty dirs left over after deleting a file. (check-in: c342b5d7 user: stephan tags: fiddle-opfs)
21:13
Generic cleanups in the OPFS sync/async proxy. (check-in: f36bddbe user: stephan tags: fiddle-opfs)
20:50
Add the remaining vfs/io_methods wrappers to the OPFS sync/async proxy, but most are not yet tested. (check-in: 44db9132 user: stephan tags: fiddle-opfs)
18:31
Fix an uninitialized variable in the decoder kv-vfs. (Leaf check-in: 354726aa user: drh tags: kv-vfs)
18:29
Include the kv-vfs as an optional VFS on unix builds if the SQLITE_OS_KV_OPTIONAL compile-time option is present. (check-in: 852812d1 user: drh tags: kv-vfs)
17:26
Merge the latest trunk enhancements into the kv-vfs branch. (check-in: 37c7ca25 user: drh tags: kv-vfs)
15:08
Add initial bits of an experimental async-impl-via-synchronous-interface proxy intended to marshal OPFS via sqlite3_vfs API. (check-in: 38da059b user: stephan tags: fiddle-opfs)
2022-09-16
20:16
Merge kv-vfs branch into fiddle-opfs branch. Adjust speedtest1 --size flags to account for new size limit. (check-in: afb79050 user: stephan tags: fiddle-opfs)
15:19
Corrections to the xRead method for databases in the os_kv.c VFS so that it correctly reads the header, and thus avoids unnecessary cache flushes. This changes also includes an optimization to the header read logic. (check-in: ef54961c user: drh tags: kv-vfs)
12:49
When calculating kvvfs sessionStorage/localStorage size, multiply it by 2 to account for JS using 16-bit characters. (check-in: 52d1b185 user: stephan tags: fiddle-opfs)
11:45
Merge kv-vfs branch into fiddle-opfs branch for [21915af560b1|synchronous=off fix]. Remove some duplicate debug output in OPFS test code. (check-in: 13899bb9 user: stephan tags: fiddle-opfs)
11:37
Fix os_kv.c so that it uses SQLITE_FCNTL_SYNC and hence works even with PRAGMA synchronous=OFF. (check-in: 21915af5 user: drh tags: kv-vfs)
02:30
Add batch-runner-kvvfs.html, a kvvfs-specific build of batch-runner.html. Reduce the speedtest1 --size X value for the batch list generation to 5 so that the kvvfs batch runner can handle it. (check-in: d8df2592 user: stephan tags: fiddle-opfs)
02:28
Remove --nosync flag from speedtest1-kvvfs.html, as that seems to be the trigger for the 'no such table: z1' error. That app now warns if that flag is passed to it via URL args. (check-in: ad0677e8 user: stephan tags: fiddle-opfs)
01:08
Pull the src/os_kv.c part of [13839759f8f4] into the kv-vfs branch. (check-in: e3344499 user: stephan tags: kv-vfs)
01:05
Add return value checks to kvstorageWrite(), necessary for recognizing storage-full errors in the JS binding. speedtest1-kvvfs.html runs successfully on localStorage with --size 1 to 6 and starts failing with storage shortage at --size 7. (check-in: 13839759 user: stephan tags: fiddle-opfs)
2022-09-15
06:42
More work on the synchronous OPFS experimentation. Numerous wasm/js build tweaks. Add speeedtest-wasmfs.html, the wasmfs/opfs counterpart of speedtest1.html. (check-in: 00ee49a3 user: stephan tags: fiddle-opfs)
03:54
Fix windows builds of recover extension. (check-in: abcbb6ab user: dan tags: recover-extension)
03:16
Remove some stale comments. Get scratchpad-opfs-worker2.js running again in prep for reuse in experimenting with the OPFS all-synchronous API. (check-in: 4cbbd370 user: stephan tags: fiddle-opfs)
03:09
Split wasmfs-enabled build of sqlite3.js/wasm into sqlite3-wasmfs.js/wasm, as enabling wasmfs breaks all tests/demos which run from a Worker thread. (check-in: 08476f3c user: stephan tags: fiddle-opfs)
02:27
Clean up some JS documentation. (check-in: 925fdbfc user: stephan tags: fiddle-opfs)
2022-09-14
20:45
Include recovery extension in configure/make builds of testfixture. (check-in: d2f46521 user: dan tags: recover-extension)
18:57
Extra tests and fixes. (check-in: deed5336 user: dan tags: recover-extension)
16:37
Fix recovery of utf-16 databases. (check-in: 5b05be08 user: dan tags: recover-extension)
2022-09-13
20:40
Add OOM tests for the recovery extension. (check-in: 9b6b4c71 user: dan tags: recover-extension)
19:47
Fix a broken error handling case in the recovery extension. (check-in: 60089547 user: dan tags: recover-extension)
19:27
Add/apply various kvvfs-specific utility APIs to the JS layer to assist in testing and analysis. Correct a backwards default arg check for sqlite3ApiBootstrap(). Add exports for sqlite3_db_handle(), sqlite3_file_control(), and the SQLITE_FCNTL_xxx enum. (check-in: 0d789618 user: stephan tags: fiddle-opfs)
18:08
Add tests. Deal issues surrounding aborting recovery from within the SQL callback, and avoiding the pending-byte page. (check-in: 4e97dd31 user: dan tags: recover-extension)
2022-09-12
22:27
Get testing1.js working with a kvvfs build. (check-in: 333a4572 user: stephan tags: fiddle-opfs)
20:21
Remove an extraneous key-copy op in the EM_JS impl of kvstorageDelete(). (check-in: 1c5aeee4 user: stephan tags: kv-vfs)
20:18
Merge kv-vfs branch into fiddle-opfs and add speedtest1-kvvfs.html test app (which currently fails early on). (check-in: 8b1608a9 user: stephan tags: fiddle-opfs)
20:02
Fix some problems with handling corrupt database in the recovery extension. (check-in: ed318be8 user: dan tags: recover-extension)
19:33
Fix the <sys/time.h> include in os_kv.c. (check-in: 878cc93e user: drh tags: kv-vfs)
19:29
Add a gettimeofday()-based implementation of xCurrentTimeInt64() to os_kv.c. (check-in: e393ed65 user: drh tags: kv-vfs)
19:23
More tests for the recover module. (check-in: 37fb093b user: dan tags: recover-extension)
18:10
Fix a debugging/testing edit in the previous check-in. (check-in: 6fc8d34c user: drh tags: kv-vfs)
17:59
Add sqlite3.oo1.DB.clearKvvfsStorage(). Add controls to kvvfs1.js demo to reset and query the db without requiring the dev console. (check-in: d845c6c2 user: stephan tags: fiddle-opfs)
17:44
Faster text decoder for kv_os.c. (check-in: 3354a2ed user: drh tags: kv-vfs)
16:09
Merge kv-vfs branch into fiddle-opfs branch to add kvvfs-based wasm build and demo. (check-in: a7d8b26a user: stephan tags: fiddle-opfs)