/ File History
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

History of src/vdbesort.c

2019-07-16
19:44
Add new assert() statements in an attempt to help static analyzers avoid false-positives. file: [3531ae3a] check-in: [9e664585] user: drh branch: trunk, size: 96267
2019-04-16
11:21
Avoid a sanitizer error (pointer arithmatic overflow) in vdbesort.c. file: [66592d47] check-in: [af61a2fc] user: dan branch: trunk, size: 96217
2019-04-13
04:01
Enforce the SQLITE_LIMIT_COLUMN limit on virtual tables. file: [31c7794a] check-in: [0b6ae032] user: drh branch: trunk, size: 96119
2018-08-21
12:46
Further attempts to reduce the number of false-positives genenerated by static analysis tools. file: [90aad5a9] check-in: [38f9ce65] user: drh branch: trunk, size: 96069
2017-08-28
17:26
Remove the rarely-used scratch memory allocator. This makes the code smaller, faster, and easier to maintain. In place of the scratch allocator, add the SQLITE_CONFIG_SMALL_MALLOC configuration option that provides a hint to SQLite that large memory allocations should be avoided. file: [789ad666] check-in: [cc440400] user: drh branch: begin-concurrent-branch-3.19, size: 95936
15:51
Remove the rarely-used scratch memory allocator. This makes the code smaller, faster, and easier to maintain. In place of the scratch allocator, add the SQLITE_CONFIG_SMALL_MALLOC configuration option that provides a hint to SQLite that large memory allocations should be avoided. file: [731a09e5] check-in: [54b00024] user: drh branch: trunk, size: 95881
2017-08-02
22:43
In the KeyInfo object, refactor the nField and nXField elements into nKeyField and nAllField, which are more useful and run a little faster. file: [fea2bea2] check-in: [aea5990e] user: drh branch: trunk, size: 95965
2017-05-30
18:34
Change sqlite3BtreeNext() and sqlite3BtreePrevious() so that they return SQLITE_DONE if they have already reached the end (or beginning) of the table. This gives a performance increase and size reduction. file: [f512c68d] check-in: [e972a386] user: drh branch: trunk, size: 96020
2017-05-27
22:42
Smaller and faster vdbeSorterCompareText(). file: [f26595df] check-in: [542dc4c5] user: drh branch: trunk, size: 95815
2017-04-03
12:04
Slightly smaller and faster implementation for vdbeSorterCompareInt(). file: [e72fe02a] check-in: [84fa069c] user: drh branch: trunk, size: 95846
2016-12-21
19:45
Simplifications to the way UnpackedRecord objects are allocated. Smaller and faster code that also fixes a subtle (currently unreachable) bug. file: [eda25cb2] check-in: [f7ab01f2] user: drh branch: trunk, size: 95912
2016-12-10
12:58
Reorder the fields in the VdbeCursor object so that those that need to be bulk zeroed on allocation are grouped at the beginning, and the memset() only runs over those fields that really need it. file: [bb34eed4] check-in: [8165f88b] user: drh branch: trunk, size: 96076
2016-05-20
14:54
Optimizations to link list merge sort code in vdbesort.c, pcache.c, and rowset.c. Resulting binaries are 10 bytes smaller and use 0.03% fewer CPU cycles. file: [91fda390] check-in: [9033afbb] user: drh branch: trunk, size: 96075
2016-04-14
15:44
Correctly interpret negative "PRAGMA cache_size" values when determining the cache-size used for sorting large amounts of data (i.e. the functionality in vdbesort.c). file: [0a8f9836] check-in: [79147dca] user: dan branch: trunk, size: 96013
2016-02-19
14:20
Fix a harmless uninitialized variable access. file: [307460bf] check-in: [398522e6] user: drh branch: trunk, size: 95795
2016-02-13
23:43
Enhance ability to debug out-of-memory errors. file: [74c22db6] check-in: [6a9c4a3e] user: mistachkin branch: noMemBkpt, size: 95791
2016-01-29
08:38
Avoid two more instances of pointer arithmetic on freed pointers. file: [ef3c6d1f] check-in: [2910ef64] user: dan branch: trunk, size: 95701
2016-01-25
22:08
Fix a compiler warning about doing pointer arithmetic involving a NULL pointer even though the result of computation is never used. file: [3bb1f1f0] check-in: [7c49a947] user: drh branch: trunk, size: 95692
2016-01-12
14:10
Fix a harmless use of an uninitialized variable following system errors in the multi-threaded sorter. file: [0971557e] check-in: [4bb2c1df] user: drh branch: trunk, size: 95653
2015-12-17
17:30
Reduce the size of the VdbeCursor object by a pointer (the pBt pointer used for ephemeral tables). file: [90a3545e] check-in: [98b710c3] user: drh branch: optimize-vdbecursor, size: 95658
2015-11-20
19:22
Refactor the VdbeCursor object. It is now slightly smaller and faster and is easier to understand. file: [a7ec02da] check-in: [9b1d174d] user: drh branch: trunk, size: 95652
2015-10-14
20:34
Fix harmless compiler warnings. file: [8b23930a] check-in: [1c46c194] user: mistachkin branch: trunk, size: 95237
2015-05-02
12:40
Reorganize some multi-threaded code in vdbesort.c so that full MC/DC test coverage does not depend on the outcome of a race condition. file: [f5009e7a] check-in: [78c7ec95] user: dan branch: trunk, size: 95229
2015-04-13
12:45
Remove an unreachable branch from the vdbeSorterCompareInt() routine. file: [2e7f6834] check-in: [f666ef06] user: drh branch: trunk, size: 94115
2015-04-11
20:44
Optimizations for VACUUM, CREATE INDEX and some cases of ORDER BY. file: [5a729a15] check-in: [79326d6e] user: dan branch: trunk, size: 94103
20:20
Fix a problem with sorting large amounts of partially ordered data. file: [a9d39d99] check-in: [acca97ef] user: dan branch: trunk, size: 88726
2015-04-02
09:06
Fix a problem in vdbesort.c to do with caching unpacked records. file: [74a41fcd] check-in: [80a00539] user: dan branch: sorter-opt, size: 94214
2015-03-30
12:06
Improve performance of multi-field sorts where the first field has a low cardinality. file: [7b368466] check-in: [601e7b6b] user: dan branch: sorter-opt, size: 94211
09:58
Remove some unnecessary code from vdbesort.c. file: [f283b28d] check-in: [b58191e9] user: dan branch: sorter-opt, size: 93458
2015-03-28
19:56
Further optimizations for sorting records that begin with integer or text values. file: [b3d16bbd] check-in: [24fe9f25] user: dan branch: sorter-opt, size: 93656
2015-03-26
11:55
Optimize cases where all the sorter is sorting a set of records that all begin with integer values, or that all begin with text values to be compared using BINARY. file: [a8d82ed4] check-in: [ce5ad17c] user: dan branch: sorter-opt, size: 90205
2015-03-18
16:00
Add another sqlite3FaultSim() to the multi-threaded sorter logic to improve testability. file: [919717d7] check-in: [49ea2cde] user: drh branch: trunk, size: 88837
2015-01-12
17:56
Add an assert() in order to calm a scan-build warning. file: [6d64c544] check-in: [11e81ac2] user: drh branch: trunk, size: 88780
2015-01-02
21:54
Fix a harmless compiler warning. file: [560bdfd9] check-in: [e0de5807] user: drh branch: trunk, size: 88743
15:55
Add the SQLITE_CONFIG_PMASZ start-time option. file: [fc1aa0d1] check-in: [acb0d1e8] user: drh branch: trunk, size: 88733
2014-12-30
14:40
If the sorter uses mmap'd temp files, ensure all pages of the temp file have been allocated before it is accessed. Otherwise, a disk-full condition might result in a SIGBUS exception. file: [80e40d88] check-in: [77664841] user: dan branch: trunk, size: 88904
2014-12-09
19:16
Lower the default SQLITE_SORTER_PMASZ value back to 10, where it has been for the past couple of releases. Applications that need a larger value can set one. file: [c150803a] check-in: [1ba8911c] user: drh branch: trunk, size: 88818
14:42
Increase the default minimum PMA size for multi-threaded sorting from 10x the page size to 250x the page size. Provide the SQLITE_SORTER_PMASZ compile-time option to change this default. file: [db015e20] check-in: [b05340fe] user: drh branch: trunk, size: 88819
2014-11-25
18:59
Fix an integer overflow bug in vdbesort.c. file: [42c166f7] check-in: [62382719] user: dan branch: trunk, size: 88778
2014-11-04
17:23
Add various requirements evidence marks for sqlite3_config() options. file: [87f39234] check-in: [d423349d] user: drh branch: trunk, size: 88525
2014-11-03
14:46
Use exponential buffer size growth in StrAccum, as long as the size does not grow to large, to avoid excess memory allocation resize operations. Also, document the fact that setting scratch memory causes SQLite to try to avoid large memory allocations. file: [daf87ea5] check-in: [a518bc33] user: drh branch: trunk, size: 88520
2014-10-16
21:39
Work around MSVC not being able to deduce that a local variable was initialized in a called function. file: [975aeffa] check-in: [06c576c1] user: mistachkin branch: trunk, size: 88635
2014-09-19
19:43
Do not attempt to extend the temp file if VFS version 3 is not supported and hence memory mapped I/O is unavailable. file: [5c1bacf9] check-in: [3ab20ba1] user: drh branch: trunk, size: 88631
00:43
Add the sqlite3VdbeMemClearAndResize() interface to be used in place of sqlite3VdbeMemGrow(). A C++ style comment was left in this check-in by mistake, and so it has been moved into a branch to avoid problems in any future bisects on windows. file: [75c66c2f] check-in: [5b9b8987] user: drh branch: Cplusplus-comment, size: 88631
2014-09-16
14:37
Reduce the number of arguments to RecordCompare functions from 4 to 3, resulting in a small performance increase. file: [09efa5e5] check-in: [8239c35a] user: drh branch: micro-optimizations, size: 88624
2014-09-15
16:50
Avoid attempting to call the xFetch() method of an sqlite3_io_methods object with a version number less than 3. file: [a7a40cec] check-in: [dedaa6fb] user: dan branch: trunk, size: 88630
15:34
Merge latest trunk changes with this branch. file: [36ac0900] check-in: [55b8011d] user: dan branch: ota-update, size: 88586
2014-09-06
16:39
Fix typos in comments. No code changes. file: [ab39574e] check-in: [e62aab5e] user: peter.d.reid branch: trunk, size: 88523
2014-09-04
11:03
Avoid calling sqlite3OsFetch() on a file-handle for which the xFetch method is NULL. file: [7c45bfcd] check-in: [071f7f2d] user: dan branch: ota-update, size: 88590
2014-09-01
17:36
Add support for using separate worker threads to speed large sorts. The SQLITE_MAX_WORKER_THREADS and SQLITE_DEFAULT_WORKER_THREADS compile-time options and the SQLITE_LIMIT_WORKER_THREADS argument to sqlite3_limit() and the "PRAGMA threads=N" pragma are added. file: [02646a9f] check-in: [b1c0f0bc] user: drh branch: trunk, size: 88527
2014-08-29
16:20
Add SQLITE_LIMIT_WORKER_THREADS for controlling the maximum number of worker threads. file: [f92628f3] check-in: [1b598c68] user: drh branch: threads, size: 88554
2014-08-25
23:44
Remove the SQLITE_CONFIG_WORKER_THREADS configuration parameter. The number of worker threads in the sorter is now determined only by the PRAGMA threads=N setting. file: [50fe3442] check-in: [e3305d4b] user: drh branch: threads, size: 88338
15:13
Query or change the maximum number of worker threads allowed on each database connection separately using the "PRAGMA threads" command. file: [3a76f51e] check-in: [29c5e8a7] user: drh branch: threads, size: 88161
2014-07-30
18:47
Add a new sqlite3FaultSim() call to vdbePmaReaderSeek() to facilitate tests of error handling in the sorter. file: [b9a83068] check-in: [655d8cfc] user: drh branch: threads, size: 88094
17:21
Mark some invariants in the vdbesort.c logic when SQLITE_MAX_WORKER_THREADS==0. file: [e37e4942] check-in: [721cd965] user: drh branch: threads, size: 88039
14:44
Merge recent trunk changes, and especially the fix for the CREATE UNIQUE INDEX problem of ticket [9a6daf340df99ba9]. file: [cab84b48] check-in: [5b50a838] user: drh branch: threads, size: 87416
13:56
Ensure that the correct number of columns in a UNIQUE index are checked for uniqueness, regardless of whether or not the original table has a ROWID or if the columns are NOT NULL, etc. Ticket [9a6daf340df99ba93c]. file: [f7f5563b] check-in: [6b785e92] user: drh branch: trunk, size: 36400
2014-07-29
18:46
Fix unreachable branches in the threads.c module. file: [2198e33d] check-in: [3175e366] user: drh branch: threads, size: 87462
17:22
Fix a harmless compiler warning. file: [48b99b15] check-in: [216d21d0] user: drh branch: threads, size: 87447
2014-07-28
20:16
Remove an unnecessary parameter from vdbeMergeEngineStep(). Rename a couple other routines to be more descriptive of what they do. file: [e78efeed] check-in: [f2407a40] user: drh branch: threads, size: 87457
19:58
Rename vdbeIncrMergerInit() to vdbeMergeEngineInit() - a much more accurate name. file: [143de4b9] check-in: [5b084a2d] user: drh branch: threads, size: 87545
18:57
In vdbesort.c, rename vdbeSorterDoCompare() to vdbeMergeEngineCompare() and move it closer to the one place where it is called. Other minor comment changes. file: [2ee08671] check-in: [09d50d9f] user: drh branch: threads, size: 87541
17:18
In vdbesort.c, rename all pointers to sqlite3_file objects "pFd" and use the name "pFile" only for pointers to SortFile objects. Other comment enhancements. file: [cbd9f27e] check-in: [518290a7] user: drh branch: threads, size: 87001
14:54
Improvements to comments in the multi-threaded sorter. Also include a function name change for clarity. And add a test to help show that the MergeEngine object is only used by a single thread. file: [ef998096] check-in: [9af50a87] user: drh branch: threads, size: 86450
2014-05-29
20:24
Merge changes from the trunk into the threads branch. file: [e2784e2e] check-in: [416cb091] user: drh branch: threads, size: 85339
2014-05-27
20:19
Remove an incorrect assert() in the sorter. file: [44441d73] check-in: [b8b66103] user: drh branch: trunk, size: 36456
2014-05-20
11:03
In vdbesort.c, change the names of PmaReader variables "pIter" to "pReadr". Other related comment changes. The generated object code should be the same. file: [a8b8798f] check-in: [99efb235] user: drh branch: threads, size: 85384
2014-05-19
19:26
Fix harmless compiler warnings in MSVC. file: [5b3e6963] check-in: [aa7e2041] user: drh branch: safer-malloc, size: 36506
15:16
Changes to help ensure that a multiplication does not overflow when computing the number of bytes needed for a memory allocation, and cause a malfunction. No problems existing problems were discovered. However, these changes should help to ensure that no problems arise in the future. file: [20246077] check-in: [17349a49] user: drh branch: safer-malloc, size: 36501
2014-05-16
20:24
Rearrange some conditionals and add #if statements to make the code more testable. file: [15d1405f] check-in: [17afd770] user: drh branch: threads, size: 85150
17:31
Add a pair of sqlite3FaultSim(100) calls to vdbesort.c to facilitate testing of obscure and hard to reach error conditions. file: [f126a837] check-in: [cceac14f] user: drh branch: threads, size: 84822
12:18
Use #ifdef logic to avoid an always-true branch when SQLITE_MAX_WORKER_THREADS=0 file: [7946115f] check-in: [88cfe6d7] user: drh branch: threads, size: 84758
2014-05-15
16:56
Use #ifdef to omit code that is not used when SQLITE_MAX_WORKER_THREADS is zero. file: [0cb40c33] check-in: [2e8d287d] user: drh branch: threads, size: 84748
2014-05-12
15:30
In the sorter, only use large memory allocations if scratch memory has not been configured. Add #ifdefs to disable unused code when SQLITE_MAX_WORKER_THREADS is zero. Other sorter changes in support of testability. file: [0daa0299] check-in: [d7e2b0d9] user: drh branch: threads, size: 84652
2014-05-05
15:58
Fix a race condition in the sorter code. file: [b3607043] check-in: [2d2edfe5] user: dan branch: threads, size: 84153
09:08
Add tests so that the "coverage-sorter" test permutation covers all branches in vdbesort.c. Fix a few minor problems in the same file. file: [af752fa4] check-in: [bde28b70] user: dan branch: threads, size: 83716
2014-05-03
20:43
Add an extra fault-injection test to sortfault.test. Remove an unreachable branch from vdbesort.c. file: [db67b5b5] check-in: [a33a366b] user: dan branch: threads, size: 83439
19:33
Fix a race condition in the sorter. file: [c443cdf0] check-in: [32ccf3ae] user: dan branch: threads, size: 83527
14:28
Fix a problem in the sorter causing it to return spurious SQLITE_NOMEM errors when configured to use memsys3 or memsys5. file: [3e8827bb] check-in: [3a66c4e1] user: dan branch: threads, size: 82807
11:22
Add the SQLITE_DEFAULT_WORKER_THREADS compile-time option. Fix a NULL-pointer dereference that can occur following OOM. file: [d205b56d] check-in: [e0dea89b] user: drh branch: threads, size: 82779
2014-05-02
16:22
Failure to extend a temp file for use with mmap() in vdbesort.c is benign. file: [6bcf73fb] check-in: [d4d39638] user: drh branch: orderby-planning, size: 45184
16:03
Remove a faulty assert() from vdbesort.c. file: [0095545a] check-in: [d95d68aa] user: dan branch: threads, size: 82737
15:25
Fix a faulty assert() statement. file: [d0fc5ecd] check-in: [9196ce40] user: drh branch: orderby-planning, size: 45218
2014-04-24
16:25
Improved header comment on the vdbesort.c module. No changes to code. file: [469ae9af] check-in: [bf09ce24] user: drh branch: orderby-planning, size: 45216
12:28
Improvements to comments. Store some extra information in SqliteThread that is useful for debugging. file: [2b13026e] check-in: [9fb5e212] user: drh branch: threads, size: 82799
2014-04-18
13:40
Fix harmless compiler warnings. file: [f93c8aaf] check-in: [f8f72ecb] user: drh branch: threads, size: 82615
2014-04-17
08:57
Fix build problems in vdbesort.c. Add further comments and changes to make things easier to understand. file: [a41721a8] check-in: [12b190db] user: dan branch: threads, size: 82457
2014-04-16
19:04
Clarify the purpose of the nField argument passed to sqlite3VdbeSorterInit(). file: [fcd15eb4] check-in: [c0c8cff1] user: dan branch: threads, size: 80124
17:41
Change the name of vdbeIncrInit2 to vdbePmaReaderIncrInit. Add a header comment to the same function. file: [70e42f93] check-in: [6622d876] user: dan branch: threads, size: 79253
16:43
Rework the way trees of MergeEngine objects are built in vdbesort.c to make it easier to follow. Fix memory leaks that could follow an OOM or IO error. Add various comments to explain functions in vdbesort.c. file: [30505a84] check-in: [69026ec7] user: dan branch: threads, size: 78879
2014-04-15
20:52
Fix some problems to do with OOM conditions in vdbesort.c. Some problems remain. file: [56c66377] check-in: [2f94f9ce] user: dan branch: threads, size: 75731
19:52
Fix further code and documentation issues in vdbesort.c. file: [ceb8e160] check-in: [d03f5b86] user: dan branch: threads, size: 75407
2014-04-14
18:41
Improve performance in single-threaded mode by having the final merge pass keys directly to the VDBE, instead of going via a final PMA. file: [15f59dc5] check-in: [02610cd9] user: dan branch: threads-experimental, size: 71102
08:45
Minor fixes so that builds with SQLITE_MAX_WORKER_THREADS=0 work. file: [364cb94c] check-in: [e400bbbf] user: dan branch: threads-experimental, size: 70325
07:30
Improve use of multiple threads in sqlite3VdbeSorterRewind(). file: [b047de6a] check-in: [e1bdc4b8] user: dan branch: threads-experimental, size: 70240
2014-04-12
19:34
Fix many issues with new code. file: [bc0d90e0] check-in: [62c406a0] user: dan branch: threads-experimental, size: 69093
2014-04-11
19:43
Avoid having the sorter merge too many PMAs at a time when incrementally merging data following a SorterRewind(). file: [2984e362] check-in: [98bf0307] user: dan branch: threads-experimental, size: 70966
2014-04-09
20:04
Experimental multi-threaded sorting changes to allow the sorter to begin returning items to the VDBE before all data is sorted. file: [26823b62] check-in: [f9d5e09a] user: dan branch: threads-experimental, size: 62972
2014-04-07
18:44
Experimental multi-threaded sorting changes to begin merging PMAs before SorterRewind() is called. file: [07c29a5a] check-in: [cbfc0f6d] user: dan branch: abandoned, size: 58722
2014-04-04
02:13
Change vdbeSorterExtendFile() so that it makes a best effort to create the PMA file of the desired size, but does not return an error if unable. file: [8da916fc] check-in: [217814bc] user: drh branch: threads, size: 56992
2014-04-03
16:25
Fix an integer overflow problem in the sorter. file: [252d7ab7] check-in: [9d3351b8] user: dan branch: threads, size: 57037
14:29
Fix minor errors causing compilation to fail with SQLITE_MAX_WORKER_THREADS set to a value greater than zero. file: [5e7ed44b] check-in: [0561272a] user: dan branch: threads, size: 56867
02:54
Refactor local object and method names in vdbesort.c so that their names more closely reflect their actual use. file: [a4e349ff] check-in: [d284e30e] user: drh branch: threads, size: 56849
2014-04-02
18:58
Add a big introductory comment to vdbesort.c explaining its operation at a high level. Also adjust some symbolic names and fix other comment issues in that file. file: [523283d7] check-in: [eef60f1b] user: drh branch: threads, size: 57109
15:15
Fix some problems with OOM handling in vdbesort.c. file: [e830ea4a] check-in: [47e702bd] user: dan branch: threads, size: 53303
14:38
Change the name of the SorterThread object to "SortSubtask" to avoid confusion with the SQLiteThread object. file: [0cb83fc3] check-in: [4ee2d910] user: drh branch: threads, size: 53270
14:16
Incorrect change to a comment. file: [ea221800] check-in: [abbdb925] user: drh branch: mistake, size: 53313
2014-04-01
18:41
When sorting data for a CREATE INDEX statement in single-threaded mode, assume that keys are delivered to the sorter in primary key order. Also fix various comments that had fallen out of date. file: [be494ad4] check-in: [821d1ac4] user: dan branch: threads, size: 53205
15:38
Even if compile time option SQLITE_MAX_WORKER_THREADS is set to one or greater, set the default number of worker threads to zero. Distribute data more evenly between threads in sqlite3VdbeSorterWrite() to improve performance when sorting large amounts of data. Add new test file sort2.test. file: [375919a7] check-in: [643c86a0] user: dan branch: threads, size: 53395
10:19
Fix a problem with OOM handling in the sorter code. file: [35c27063] check-in: [59cd5229] user: dan branch: threads, size: 53099
2014-03-31
19:57
Add the SQLITE_MAX_WORKER_THREADS compile time option. And the SQLITE_CONFIG_WORKER_THREADS sqlite3_config() switch. file: [b4d6133b] check-in: [2774710d] user: dan branch: threads, size: 52955
2014-03-29
19:48
Changes to make the multi-threaded sorter sort stably. file: [d42a32a4] check-in: [83a105c8] user: dan branch: threads-closed, size: 54149
10:01
Fix a broken assert() in vdbesort.c. file: [2881297f] check-in: [18d1b402] user: dan branch: threads, size: 52636
09:34
Fix a problem in vdbesort.c causing spurious SQLITE_NOMEM errors when using memsys3 or memsys5. file: [80812ceb] check-in: [a683c05f] user: dan branch: threads, size: 52636
06:27
Add the optimization to avoid some unnecessary calls to sqlite3VdbeRecordUnpack() added to the trunk by [707ea170b3]. file: [d7ef3c43] check-in: [fc4d04e6] user: dan branch: threads, size: 52552
2014-03-28
19:47
Fix a compiler warning and an after-OOM memory leak. file: [ad0f9f71] check-in: [58f7ca29] user: drh branch: orderby-planning, size: 42822
19:18
Merge latest changes from orderby-planning branch. file: [01068b89] check-in: [4c7fb542] user: dan branch: threads, size: 50418
2014-03-27
19:25
Instead of allocating a single large buffer at the beginning of each sort operation, start with a small buffer and extend it using realloc() as required. file: [08d5e1ee] check-in: [81987c8c] user: dan branch: orderby-planning, size: 42791
17:23
Use xFetch() to access temporary files in vdbesort.c. Use a single large allocation instead of many small allocations when accumulating records in vdbesort.c. This is an interim commit - it allocates a buffer the size of the page-cache every time data is sorted. file: [d46f384a] check-in: [f4ac1bf2] user: dan branch: orderby-planning, size: 40085
2014-03-25
17:07
Fix a problem in the code added by [707ea170b3] causing vdbesort.c to sort unstably. file: [691f2186] check-in: [d3e640af] user: dan branch: orderby-planning, size: 37272
14:12
Enable four sorting threads by default in the command-line shell. file: [c3e427de] check-in: [1cab8357] user: drh branch: threads, size: 43625
13:17
Merge all fixes and enhancements from trunk. file: [0fbaf5b3] check-in: [b415dfb6] user: drh branch: threads, size: 43625
2014-03-24
20:19
Omit the sequence value from sorter records used by GROUP BY queries that cannot use an index. file: [46e50c6b] check-in: [3f90abdd] user: dan branch: orderby-planning, size: 36556
2014-03-20
20:56
Merge trunk fixes for "x IN (?)" handling. file: [4abb7c0f] check-in: [eca35871] user: drh branch: orderby-planning, size: 36501
15:14
Fix harmless compiler warnings. file: [0fef5fa8] check-in: [b1435f26] user: drh branch: trunk, size: 36134
2014-03-19
23:42
Merge the vdbesort.c optimization from trunk. file: [17336685] check-in: [e4bfffb9] user: drh branch: orderby-planning, size: 36491
20:01
Avoid some unnecessary calls to sqlite3VdbeRecordUnpack() that were being made when merging data from two or more temp files together in vdbesort.c file: [0a9e5e63] check-in: [707ea170] user: dan branch: trunk, size: 36124
17:41
Make it possible for block-sort to use the OP_SorterOpen sorter in addition to a generic OP_OpenEphemeral. file: [553537f6] check-in: [7ce2daaf] user: drh branch: orderby-planning, size: 34646
2014-03-17
15:43
Add an experimental multi-threaded capability to vdbesorter.c. file: [1d973fcd] check-in: [ff0b5c85] user: dan branch: threads, size: 43206
2014-03-01
19:44
Remove the vdbeRecordCompareLargeHeader function. Fix some other details. file: [46801acb] check-in: [3861e853] user: dan branch: experimental, size: 34279
2014-02-27
20:44
Further changes to sqlite3VdbeRecordCompare(). file: [72290f12] check-in: [57089374] user: dan branch: experimental, size: 34498
2014-02-25
21:01
Attempt to speed up sqlite3VdbeRecordCompare() by various means. This code is in an interim state. file: [5144d84b] check-in: [85206e0b] user: dan branch: experimental, size: 34276
2013-11-01
22:02
Size KeyInfo objects so that IdxInserts always compare the correct number of fields. file: [9d83601f] check-in: [302a8139] user: drh branch: omit-rowid, size: 34283
2013-05-28
12:11
Many small harmless comment changes. Removal of obsolete comments and fixing misspelled words. No changes to code. file: [3937e06b] check-in: [a0d5cc93] user: drh branch: trunk, size: 34198
2013-03-24
22:56
Remove the SQLITE_OMIT_MERGE_SORT compile-time option and its related code. The merge sorter is now a required component. file: [4fad6407] check-in: [8b44d6fb] user: drh branch: trunk, size: 34198
2012-10-26
19:22
Fix a 32-bit integer overflow problem that could cause an error in a CREATE INDEX statement that writes more than 16GB of data. file: [c61ca318] check-in: [e24ba5be] user: dan branch: trunk, size: 34274
2012-08-22
15:16
Do not let the multi-core sorter use lookaside memory, which is not thread-safe. file: [acaefad5] check-in: [acdc7d12] user: drh branch: threads-sort-ex1, size: 37350
2012-08-16
20:05
Attempt to use two cores to do sorting. Unfortunately, instead of making sorts go faster as was hoped, this changes slows sorting down by about 10%. (Later:) The previous measurement was compiled using -pg. When compiled using -Os, this new code is roughly 10% faster than the original. file: [3945ae71] check-in: [11dd05e5] user: drh branch: threads-sort-ex1, size: 36805
2012-08-15
15:57
Experimental change to speed up ORDER BY clauses that sort based on a single expression. file: [e4aa1c16] check-in: [2bb8c492] user: dan branch: sorter-exp, size: 40282
2012-08-14
19:04
Silence three harmless compiler warnings in vdbesort.c. file: [0dc1b274] check-in: [a5431c86] user: drh branch: trunk, size: 34223
2012-08-07
22:53
Simplifications to the sorter to support full-coverage testing. file: [bd5ce83f] check-in: [de804f4c] user: drh branch: trunk, size: 34204
2012-08-06
19:28
Merge the sorter-coalesce-writes branch into the trunk. This improves CREATE INDEX performance on some platforms. file: [4897215f] check-in: [e1e9cb08] user: dan branch: trunk, size: 34459
19:12
Fix a crash that could follow an OOM condition. file: [1de867bf] check-in: [2e5741f7] user: dan branch: sorter-coalesce-writes, size: 34435
18:10
Update sorter-coalesce-writes branch with latest trunk changes. file: [ad307856] check-in: [214f8cda] user: dan branch: sorter-coalesce-writes, size: 34399
2012-07-26
09:21
Update some comments in vdbesort.c. file: [106796ce] check-in: [f4b3fded] user: dan branch: sorter-coalesce-writes, size: 34357
2012-07-24
19:46
Mark parameters to sorter interfaces as const where appropriate. file: [628b2bc0] check-in: [d8da26f1] user: drh branch: trunk, size: 29343
2012-07-23
20:10
Fix an edge case in vdbesort.c. file: [bd37dbd4] check-in: [4ba266fc] user: dan branch: sorter-coalesce-writes, size: 32886
19:25
Modify the code in vdbesort.c so that most reads and writes to temporary files are aligned page-sized blocks. file: [6822221a] check-in: [55e47ef3] user: dan branch: sorter-coalesce-writes, size: 32769
2012-02-02
21:02
More structure packing for smaller objects and less memory usage. file: [b25814d3] check-in: [f14e7f29] user: drh branch: trunk, size: 29301
2011-09-14
18:19
Remove 'const' from a variable that might actually be written when an OOM error occurs. Also, fix a couple MSVC/Win32 compiler warnings. file: [468d43c0] check-in: [3035dc1c] user: mistachkin branch: trunk, size: 29301
13:41
Remove unused parameters from internal routines in vdbesort.c. This is an additional simplification made possible by recent optimizations. file: [8dd9c1ac] check-in: [a1f3aeeb] user: drh branch: trunk, size: 29210
13:23
Remove code from vdbesort.c that was made unreachable by the recent sqlite3VdbeRecordUnpack() optimizations. file: [df1953a4] check-in: [607aba6c] user: drh branch: trunk, size: 29295
2011-09-05
20:16
Fix a minor performance regression in btreeMoveto(). file: [667bc652] check-in: [d0712dfb] user: dan branch: trunk, size: 30064
14:20
Refactor the sqlite3VdbeRecordUnpack() interface to better accommodate the vdbesort.c module. file: [4f326570] check-in: [f115b230] user: dan branch: trunk, size: 30016
2011-09-04
01:27
Fix a compiler warning about an unused parameter in the merge-sort code. file: [14d12756] check-in: [6b657ae7] user: drh branch: trunk, size: 30419
01:11
Improve the testability of the merge-sort logic. file: [4ceadaa2] check-in: [b5179baf] user: drh branch: trunk, size: 30591
2011-09-03
16:42
Simplification and performance tweaks in vdbeSorterMerge(). file: [e6d6f0c2] check-in: [99e34bdc] user: drh branch: merge-sort, size: 30200
14:36
Reduce the number of VdbeRecordUnpack() calls made in vdbesort.c. file: [5e38e7e1] check-in: [666c2c3c] user: dan branch: merge-sort, size: 30387
00:17
The build works again with -DSQLITE_OMIT_MERGE_SORT. The merge-sorter now avoids spilling to disk (letting the in-memory linked list grow without bound) if PRAGMA temp_store=3. file: [aa7ad2ef] check-in: [68e26c44] user: drh branch: merge-sort, size: 30049
2011-09-02
21:42
Remove some dead code. Fix a faulty assert(). Improve some variable names. file: [2ae618be] check-in: [a9a64592] user: drh branch: merge-sort, size: 29977
18:03
Combine two malloc calls in vdbesort.c. file: [b56437b8] check-in: [cf48ad83] user: dan branch: merge-sort, size: 30141
15:41
Reduce the number of malloc() calls made when creating an index on more than 2 columns. file: [a49d4449] check-in: [065b0c98] user: dan branch: merge-sort, size: 30283
11:45
If all data being sorted fits in memory, avoid writing any data out to temporary files in vdbesort.c. file: [1e9d4437] check-in: [71075673] user: dan branch: merge-sort, size: 29777
10:31
Instead of a temporary b-tree, use a linked-list and merge-sort to sort records in main memory in vdbesort.c. file: [9c2e8ca2] check-in: [7769fb98] user: dan branch: merge-sort, size: 28732
2011-08-31
23:57
Avoid using uninitialized variables after failures in the merge sort code. file: [f3d043a1] check-in: [2869ed28] user: drh branch: trunk, size: 25248
2011-08-26
00:34
Create a new pager type, PAGER_SORTER, for use in the external merge sort. Such pagers are always held in memory but do report when they are under memory pressure by calling pagerStress. file: [8a61a6d7] check-in: [c71d7320] user: drh branch: trunk, size: 25207
2011-08-24
15:12
Fix some harmless compiler warnings. file: [d2c87232] check-in: [46f5a68b] user: drh branch: trunk, size: 24892
2011-08-17
00:40
Use OP_Sort rather than OP_Rewind in order to sort the content of a new index on the CREATE INDEX statement. Add an ALWAYS() to the merge sort logic to document an unreachable branch. file: [a63bab05] check-in: [f01766f4] user: drh branch: trunk, size: 24880
2011-08-15
15:37
Fix an assert() in the merge-sort logic to account for I/O errors. file: [af82a8b0] check-in: [c1daa809] user: drh branch: trunk, size: 24873
14:41
When opening a temporary file to use in a merge-sort, pass SQLITE_OPEN_TEMP_JOURNAL instead of SQLITE_OPEN_TEMP_DB. This is a better fit, as merge-sort files are written and read sequentially like journal files. file: [18e27c40] check-in: [bd04756b] user: dan branch: trunk, size: 24856
2011-08-12
16:30
Merge latest trunk changes into experimental branch. file: [3e6b2268] check-in: [7e515055] user: dan branch: experimental, size: 24847
16:11
Remove an unused parameter from a function in vdbesort.c. Fix some comments and other details in the same file. file: [a756d20d] check-in: [1a8498d8] user: dan branch: experimental, size: 24847
15:02
Add the SQLITE_OMIT_MERGE_SORT pre-processor directive. To omit the code in vdbesort.c. file: [b475ceed] check-in: [4ced2394] user: dan branch: experimental, size: 23229
2011-08-08
19:26
Remove redundant parameter from vdbeSorterInitMerge() in vdbesort.c. file: [6498ab41] check-in: [eec8c0df] user: dan branch: experimental, size: 23153
16:44
Add fault-injection and other tests (and fixes) to improve coverage of vdbesort.c. file: [e9a7d969] check-in: [0e6defa6] user: dan branch: experimental, size: 22048
2011-08-06
15:09
Fix a problem with building large indexes introduced by the previous commit. file: [34523534] check-in: [038ec9ea] user: dan branch: experimental, size: 21545
12:01
In temp files used for merge sorting, store the size of each packed-memory-array at the start of the array itself. This is to avoid having to store the offsets of all arrays in the (potentially very large) file in main-memory. file: [d7739da9] check-in: [8051c176] user: dan branch: experimental, size: 21477
2011-08-05
11:49
Minor internal changes to vdbesort.c. Also, default to merging lists together 16 at a time. file: [f17fa625] check-in: [9ddc324a] user: dan branch: experimental, size: 19940
2011-08-04
18:43
Fix a comment in vdbesort.c. file: [87c3b292] check-in: [db8518ca] user: dan branch: experimental, size: 21085
12:14
Change to using packed-memory-arrays instead of b-trees when performing an offline merge-sort for CREATE INDEX. This makes it easier to control the number of disc seeks required when merging. file: [e7d1a86e] check-in: [a4770d07] user: dan branch: experimental, size: 21044
2011-08-02
10:56
Minor fixes to vdbesort.c code in preparation for a major rework. file: [40bb17d3] check-in: [7f339c0e] user: dan branch: experimental, size: 16990
2011-07-12
14:28
Experimental support for speeding up CREATE INDEX commands using an offline merge sort. file: [f07d526d] check-in: [30dbf0fe] user: dan branch: experimental, size: 16940 Added