Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Merge [ebd923dab6] and [491737c7cf]. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA1: |
8f18472bac03a46c83e48527a73daaf3 |
User & Date: | dan 2009-04-13 15:07:08.000 |
Context
2009-04-13
| ||
18:04 | Fix a requirement number conflict in fileformat.in. Enhanced and expanded vtab.in. (check-in: 9acad193dd user: drh tags: trunk) | |
15:07 | Merge [ebd923dab6] and [491737c7cf]. (check-in: 8f18472bac user: dan tags: trunk) | |
14:24 | Modifications to fileformat.html. (check-in: 491737c7cf user: dan tags: trunk) | |
12:58 | Initial check-in of documentation on the virtual table interface. (check-in: 0776bf4007 user: drh tags: trunk) | |
Changes
Changes to pages/changes.in.
︙ | ︙ | |||
37 38 39 40 41 42 43 44 45 46 47 48 49 50 | <a href="http://www.sqlite.org/cvstrac/timeline"> http://www.sqlite.org/cvstrac/timeline</a>.</p> } hd_close_aux hd_enable_main 1 } } chng {2009 March 31 (3.6.12)} { <li>Fixed a bug that caused database corruption when an [incremental_vacuum] is rolled back in an in-memory database. Ticket #3761. <li>Added the [sqlite3_unlock_notify()] interface. <li>Added the [reverse_unordered_selects pragma]. <li>The default page size on windows is automatically adjusted to match the | > > > > > > > > > > | 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 | <a href="http://www.sqlite.org/cvstrac/timeline"> http://www.sqlite.org/cvstrac/timeline</a>.</p> } hd_close_aux hd_enable_main 1 } } chng {2009 April 13 (3.6.13)} { <li>Fix a bug in [version 3.6.12] that causes a segfault when running a count(*) on the sqlite_master table of an empty database. Ticket #3774. <li>Fix a bug in [version 3.6.12] that causes a segfault that when inserting into a table using a DEFAULT value where there is a function as part of the DEFAULT value expression. Ticket #3791. <li>Fix data structure alignment issues on Sparc. Ticket #3777. <li>Other minor bug fixes. } chng {2009 March 31 (3.6.12)} { <li>Fixed a bug that caused database corruption when an [incremental_vacuum] is rolled back in an in-memory database. Ticket #3761. <li>Added the [sqlite3_unlock_notify()] interface. <li>Added the [reverse_unordered_selects pragma]. <li>The default page size on windows is automatically adjusted to match the |
︙ | ︙ |
Changes to pages/index.in.
︙ | ︙ | |||
76 77 78 79 80 81 82 | </td> <td width="20"></td><td bgcolor="#80a796" width="1"></td><td width="20"></td> <td valign="top"> <h3>Current Status</h3> <p><ul> | | | > | 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 | </td> <td width="20"></td><td bgcolor="#80a796" width="1"></td><td width="20"></td> <td valign="top"> <h3>Current Status</h3> <p><ul> <li><a href="releaselog/3_6_13.html">Version 3.6.13</a> of SQLite is recommended for all new development. Upgrading from version 3.6.12 is optional. Upgrading from versions 3.6.11 and earlier is recommended.</li> </ul></p> <h3>Common Links</h3> <p><ul> <li> <a href="features.html">Features</a> </li> <li> <a href="faq.html">Frequently Asked Questions</a> </li> |
︙ | ︙ |
Changes to pages/lang.in.
︙ | ︙ | |||
906 907 908 909 910 911 912 | in SQLite. However, in many cases you can use an [INSTEAD OF trigger] on the view to accomplish the same thing. Views are removed with the [DROP VIEW] command.</p> <tcl> ############################################################################## | | | | | | 906 907 908 909 910 911 912 913 914 915 916 917 918 919 920 921 922 923 924 925 926 927 928 929 930 931 932 933 934 935 936 937 938 939 | in SQLite. However, in many cases you can use an [INSTEAD OF trigger] on the view to accomplish the same thing. Views are removed with the [DROP VIEW] command.</p> <tcl> ############################################################################## Section {CREATE VIRTUAL TABLE} {createvtab} {{CREATE VIRTUAL TABLE}} BubbleDiagram create-virtual-table-stmt 1 </tcl> <p>A [virtual table] is an interface to an external storage or computation engine that appears to be a table but does not actually store information in the database file.</p> <p>In general, you can do anything with a [virtual table] that can be done with an ordinary table, except that you cannot create indices or triggers on a virtual table. Some virtual table implementations might impose additional restrictions. For example, many virtual tables are read-only. Virtual tables cannot be used in [sqlite3_enable_shared_cache | shared cache mode].</p> <p>The <module-name> is the name of an object that implements the virtual table. The <module-name> must be registered with the SQLite database connection using [sqlite3_create_module()] or [sqlite3_create_module_v2()] prior to issuing the CREATE VIRTUAL TABLE statement. The module takes zero or more comma-separated arguments. The arguments can be just about any text as long as it has balanced parentheses. The argument syntax is sufficiently general that the arguments can be made to appear as column definitions in a traditional [CREATE TABLE] statement. SQLite passes the module arguments directly |
︙ | ︙ |
Changes to pages/news.in.
︙ | ︙ | |||
14 15 16 17 18 19 20 21 22 23 24 25 26 27 | hd_puts "<h3>$date - $title</h3>" regsub -all "\n( *\n)+" $text "</p>\n\n<p>" txt regsub -all {[Tt]icket #(\d+)} $txt \ {<a href="http://www.sqlite.org/cvstrac/tktview?tn=\1">\0</a>} txt hd_resolve "<p>$txt</p>" hd_puts "<hr width=\"50%\">" } newsitem {2009-Mar-31} {Version 3.6.12} { SQLite [version 3.6.12] fixes a database corruption bug. If an [incremental_vacuum] is rolled back in an in-memory database, the database will often go corrupt. This only happens for in-memory databases. On-disk databases are unaffected. And the corruption only appears if an incremental vacuum is rolled back. Nevertheless, | > > > > > > > | 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 | hd_puts "<h3>$date - $title</h3>" regsub -all "\n( *\n)+" $text "</p>\n\n<p>" txt regsub -all {[Tt]icket #(\d+)} $txt \ {<a href="http://www.sqlite.org/cvstrac/tktview?tn=\1">\0</a>} txt hd_resolve "<p>$txt</p>" hd_puts "<hr width=\"50%\">" } newsitem {2009-Apr-14} {Version 3.6.13} { SQLite [version 3.6.13] fixes several minor issues that appeared in previous version, including Ticket #3774, #3791, and #3777. This is a bug-fix release only. There are no new features or enhancements. } newsitem {2009-Mar-31} {Version 3.6.12} { SQLite [version 3.6.12] fixes a database corruption bug. If an [incremental_vacuum] is rolled back in an in-memory database, the database will often go corrupt. This only happens for in-memory databases. On-disk databases are unaffected. And the corruption only appears if an incremental vacuum is rolled back. Nevertheless, |
︙ | ︙ |
Added pages/vtab.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 371 372 373 374 375 376 377 378 379 380 381 382 383 384 385 386 387 388 389 390 391 392 393 394 395 396 397 398 399 400 401 402 403 404 405 406 407 408 409 410 411 412 413 414 415 416 417 418 419 420 421 422 423 424 425 426 427 428 429 430 431 432 433 434 435 436 437 438 439 440 441 442 443 444 445 446 447 448 449 450 451 452 453 454 455 456 457 458 459 460 461 462 463 464 465 466 467 468 469 470 471 472 473 474 475 476 477 478 479 480 481 482 483 484 485 486 487 488 489 490 491 492 493 494 495 496 497 498 499 500 501 502 503 504 505 506 507 508 509 510 511 512 513 514 515 516 517 518 519 520 521 522 523 524 525 526 527 528 529 530 531 532 533 534 535 536 537 538 539 540 541 542 543 544 545 546 547 548 549 550 551 552 553 554 555 556 557 558 559 560 561 562 563 564 565 566 567 568 569 570 571 572 573 574 575 576 577 578 579 580 581 582 583 584 585 586 587 588 589 590 591 592 593 594 595 596 597 598 599 600 601 602 603 604 605 606 607 608 609 610 611 612 613 614 615 616 617 618 619 620 621 622 623 624 625 626 627 628 629 630 631 632 633 634 635 636 637 638 639 640 641 642 643 644 645 646 647 648 649 650 651 652 653 654 655 656 657 658 659 660 661 662 663 664 665 666 667 668 669 670 671 672 673 674 675 676 677 678 679 680 681 682 683 684 685 686 687 688 689 690 691 692 693 694 695 696 697 698 699 700 701 702 703 704 705 706 707 708 709 710 711 712 713 714 715 716 717 718 719 720 721 722 723 724 725 726 727 728 729 730 731 732 733 734 735 736 737 738 739 740 741 742 743 744 745 746 747 748 749 750 751 752 753 754 755 756 757 758 759 760 761 762 763 764 765 766 767 768 769 770 771 772 773 774 775 776 777 778 779 780 781 782 783 784 785 786 787 788 789 790 791 792 793 794 795 796 797 798 799 800 801 802 803 804 805 806 807 808 809 810 811 812 813 814 815 816 817 818 819 820 821 822 823 824 825 826 827 828 829 830 831 832 833 834 835 836 837 838 839 840 841 842 843 844 845 846 847 | <title>The Virtual Table Mechanism Of SQLite</title> <h1 align="center">The Virtual Table Mechanism Of SQLite</h1> <tcl>hd_keywords {virtual table} {virtual tables}</tcl> <h2>1.0 Introduction</h2> <p>A virtual table is an object that is registered with an open SQLite database connection. From the perspective of an SQL statement, the virtual table object looks like any other table or view. But behind the scenes, queries from and updates to a virtual table invoke callback methods on the virtual table object instead of reading and writing to the database file. <p>The virtual table mechanism allows an application to publish interfaces that are accessible from SQL statements as if they were tables. SQL statements can in general do anything to a virtual table that they can do to a real table, with the following exceptions: <p> <ul> <li> One cannot create a trigger on a virtual table. <li> One cannot create additional indices on a virtual table. (Virtual tables can have indices but that must be built into the virtual table implementation. Indices cannot be added separately using CREATE INDEX statements.) <li> One cannot run ALTER TABLE commands against a virtual table. <li> Virtual tables cannot be used in a database that makes use of the [shared cache] feature. </ul> <p>Particular virtual table implementations might impose additional constraints. For example, some virtual implementations might provide read-only tables. Or some virtual table implementations might allow INSERT or DELETE but not UPDATE. Or some virtual table implementations might limit the kinds of UPDATEs that can be made. <p>A virtual table might represent an in-memory data structures. Or it might represent a view of data on disk that is not in the SQLite format. Or the application might compute the content of the virtual table on demand. <p>Here are some postulated uses for virtual tables: <ul> <li> A full-text search interface <li> Spatial indices using R-Trees <li> Access to the filesystem of the host computer <li> Enabling SQL manipulation of data in statistics packages like R </ul> <h3>1.1 Usage</h3> <p>A virtual table is created using using a CREATE VIRTUAL TABLE statement. This statement creates a table with a particular name and associates the table with a "module". <blockquote><pre> CREATE VIRTUAL TABLE tablename USING modulename; </pre></blockquote> <p>One can also provide comma-separated arguments to the module following the module name: <blockquote><pre> CREATE VIRTUAL TABLE tablename USING modulename(arg1, arg2, ...); </pre></blockquote> <p>The format of the arguments to the module is very general. Each argument can consist of keywords, string literals, identifiers, numbers, and punctuation. The arguments are passed as written (as text) into the constructor metod of the virtual table implementation when the virtual table is created and that constructor is responsible for parsing and interpreting the arguments. The argument syntax is sufficiently general that a virtual table implementation can, if it wants to, interpret its arguments as column definitions in an ordinary CREATE TABLE statement. The implementation could also impose some other interpretation on the arguments. <p>Once a virtual table has been created, it can be used like any other table with the exceptions noted above and imposed by specific virtual table implementations. A virtual table is destroyed using the ordinary DROP TABLE syntax. <h2>Implementation</h2> <p>Several new C-level objects are used by the virtual table implementation: <blockquote><pre> typedef struct sqlite3_vtab sqlite3_vtab; typedef struct sqlite3_index_info sqlite3_index_info; typedef struct sqlite3_vtab_cursor sqlite3_vtab_cursor; typedef struct sqlite3_module sqlite3_module; </pre></blockquote> <p>The sqlite3_module structure defines a module object used to implement a virtual table. Think of a module as a class from which you can construct multiple virtual tables having similar properties. For example, one might have a module that provides read-only access to comma-separated-value (CSV) files on disk. That one module can then be used to create several virtual tables where each virtual table refers to a different CSV file. <p>The module structure contains methods that are invoked by SQLite to perform various actions on the virtual table such as creating new instances of a virtual table or destroying old ones, reading and writing data, searching for and deleting, updating, or inserting rows. The module structure is explained in more detail below. <p>Each virtual table instance is represented by an sqlite3_vtab structure. The sqlite3_vtab structure looks like this: <blockquote><pre> struct sqlite3_vtab { const sqlite3_module *pModule; int nRef; char *zErrMsg; }; </pre></blockquote> <p>Virtual table implementations will normally subclass this structure to add additional private and implementation-specific fields. The nRef field is used internally by the SQLite core and should not be altered by the virtual table implementation. The virtual table implementation can pass error message text to the core by putting an error message string obtained from [sqlite3_mprintf()] in zErrMsg. Prior to assigning a new value to zErrMsg, the virtual table implementation should free any preexisting content of zErrMsg using [sqlite3_free()]. Failure to do this might result in a memory leak. The SQLite core will free and zero the content of zErrMsg when it delivers the error message text to the client application or when it destroys the virtual table. The virtual table implementation only needs to worry about freeing the zErrMsg content when it overwrites the content with a new, different error message. <p>The sqlite3_vtab_cursor structure represents a pointer to a specific row of a virtual table. This is what an sqlite3_vtab_cursor looks like: <blockquote><pre> struct sqlite3_vtab_cursor { sqlite3_vtab *pVtab; }; </pre></blockquote> <p>Once again, practical implementations will likely subclass this structure to add additional private fields. <p>The sqlite3_index_info structure is used to pass information into and out of the xBestIndex method of the module that implements a virtual table. <p>Before a CREATE VIRTUAL TABLE statement can be run, the module specified in that statement must be registered with the database connection. This is accomplished using either of the [sqlite3_create_module()] or [sqlite3_create_module_v2()] interfaces: <blockquote><pre> int sqlite3_create_module( sqlite3 *db, /* SQLite connection to register module with */ const char *zName, /* Name of the module */ const sqlite3_module *, /* Methods for the module */ void * /* Client data for xCreate/xConnect */ ); int sqlite3_create_module_v2( sqlite3 *db, /* SQLite connection to register module with */ const char *zName, /* Name of the module */ const sqlite3_module *, /* Methods for the module */ void *, /* Client data for xCreate/xConnect */ void(*xDestroy)(void*) /* Client data destructor function */ ); </pre></blockquote> <p>The sqlite3_create_module() routines associates a module name with an sqlite3_module structure and a separate client data that is specific to each module. The only difference between the two create_module methods is that the _v2 method includes an extra parameter that specifies a destructor for client data pointer. The module structure is what defines the behavior of a virtual table. The module structure looks like this: <blockquote><pre> struct sqlite3_module { int iVersion; int (*xCreate)(sqlite3*, void *pAux, int argc, char **argv, sqlite3_vtab **ppVTab, char **pzErr); int (*xConnect)(sqlite3*, void *pAux, int argc, char **argv, sqlite3_vtab **ppVTab, char **pzErr); int (*xBestIndex)(sqlite3_vtab *pVTab, sqlite3_index_info*); int (*xDisconnect)(sqlite3_vtab *pVTab); int (*xDestroy)(sqlite3_vtab *pVTab); int (*xOpen)(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor); int (*xClose)(sqlite3_vtab_cursor*); int (*xFilter)(sqlite3_vtab_cursor*, int idxNum, const char *idxStr, int argc, sqlite3_value **argv); int (*xNext)(sqlite3_vtab_cursor*); int (*xEof)(sqlite3_vtab_cursor*); int (*xColumn)(sqlite3_vtab_cursor*, sqlite3_context*, int); int (*xRowid)(sqlite3_vtab_cursor*, sqlite_int64 *pRowid); int (*xUpdate)(sqlite3_vtab *, int, sqlite3_value **, sqlite_int64 *); int (*xBegin)(sqlite3_vtab *pVTab); int (*xSync)(sqlite3_vtab *pVTab); int (*xCommit)(sqlite3_vtab *pVTab); int (*xRollback)(sqlite3_vtab *pVTab); int (*xFindFunction)(sqlite3_vtab *pVtab, int nArg, const char *zName, void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), void **ppArg); int (*Rename)(sqlite3_vtab *pVtab, const char *zNew); }; </pre></blockquote> <p>The module structure defines all of the methods for each virtual table object. The module structure also contains the iVersion field which defines the particular edition of the module table structure. Currently, iVersion is always 1, but in future releases of SQLite the module structure definition might be extended with additional methods and in that case the iVersion value will be increased. <p>The rest of the module structure consists of methods used to implement various features of the virtual table. Details on what each of these methods do are provided in the sequel. <h3>1.2 Virtual Tables And Shared Cache</h3> <p>The virtual table mechanism assumes that each database connection keeps its own copy of the database schema. Hence, the virtual table mechanism cannot be used in a database that has [shared cache] enabled. The [sqlite3_create_module()] interface will return an error if the [shared cache] feature is enabled. <h3>1.3 Creating New Virtual Table Implementations</h3> <p>Follow these steps to create your own virtual table: <p> <ol> <li> Write all necessary methods. <li> Create an instance of the [sqlite3_module] structure containing pointers to all the methods from step 1. <li> Register your [sqlite3_module] structure using the [sqlite3_create_module()] API. <li> Run a [CREATE VIRTUAL TABLE] command that specifies your module in the USING clause. </ol> <p>The only really hard part is step 1. You might want to start with an existing virtual table implementation and modify it to suit your needs. There are several virtual table implementations in the SQLite source tree (for testing purposes). You might use one of those as a guide. Locate these test virtual table implementations by searching for "sqlite3_create_module". <p>You might also want to implement your new virtual table as a [sqlite3_load_extension | loadable extension]. <h2>2.0 Virtual Table Methods</h2> <h3>2.1 The xCreate Method</h3> <blockquote><pre> int (*xCreate)(sqlite3 *db, void *pAux, int argc, char **argv, sqlite3_vtab **ppVTab, char **pzErr); </pre></blockquote> <p>This method is called to create a new instance of a virtual table in response to a [CREATE VIRTUAL TABLE] statement. The db parameter is a pointer to the SQLite [database connection] that is executing the [CREATE VIRTUAL TABLE] statement. The pAux argument is the copy of the client data pointer that was the fourth argument to the [sqlite3_create_module()] or [sqlite3_create_module_v2()] call that registered the virtual table module. The argv parameter is an array of argc pointers to null terminated strings. The first string, argv[0], is the name of the module being invoked. The second, argv[1], is the name of the database in which the new virtual table is being created. The third element of the array, argv[2], is the name of the new virtual table. If present, the fourth and subsquent strings in the argv[] array report the arguments to the module name in the [CREATE VIRTUAL TABLE] statement. <p>The job of this method is to construct the new virtual table object (an [sqlite3_vtab] object) and return a pointer to it in *ppVTab. <p>As part of the task of creating a new [sqlite3_vtab] structure, this method must invoke [sqlite3_declare_vtab()] to tell the SQLite core about the columns and datatypes in the virtual table. The [sqlite3_declare_vtab()] API has the following prototype: <blockquote><pre> int sqlite3_declare_vtab(sqlite3 *db, const char *zCreateTable) </pre></blockquote> <p>The first argument to [sqlite3_declare_vtab()] is the pointer to the sqlite [database connection]. The first argument must be the same database connection pointer that was passed into the xCreate method. The second argument to [sqlite3_declare_vtab()] is a zero-terminated UTF-8 string that contains a well-formed [CREATE TABLE] statement that defines the columns in the virtual table and their data types. The name of the table in this [CREATE TABLE] statement is ignored, as are all constraints. Only the column names and datatypes matter. <p>If a column datatype contains the special keyword "HIDDEN" (in any combination of upper and lower case letters) then it is removed from the column datatype and the column marked as a hidden column internally. A hidden column differs from a normal column in three respects: <p> <ul> <li> Hidden columns are not listed in the dataset returned by "[PRAGMA table_info]", <li> Hidden columns are not included in the expansion of a "*" expression in the result set of a [SELECT], and <li> Hidden columns are not included in the implicit column-list used by an [INSERT] statement that lacks an explicit column-list. </ul> <p>For example, if the following SQL is passed to [sqlite3_declare_vtab()]: <blockquote><pre> CREATE TABLE x(a HIDDEN VARCHAR(12), b INTEGER, c INTEGER Hidden); </pre></blockquote> <p>Then the virtual table would be created with two hidden columns, declaration types "VARCHAR(12)" and "INTEGER". <p>The xCreate method should return [SQLITE_OK] if it is successful in creating the new virtual table, or [SQLITE_ERROR] if it is not successful. If not successful, no [sqlite3_vtab] structure should be allocated. An error message may optionally be returned in *pzErr if unsuccessful. The text of the error message should be obtained from [sqlite3_mprintf()]. <h3>2.2 The xConnect Method</h3> <blockquote><pre> int (*xConnect)(sqlite3*, void *pAux, int argc, char **argv, sqlite3_vtab **ppVTab, char **pzErr); </pre></blockquote> <p>The xConnect method is very similar to xCreate. It has the same parameters and constructs a new [sqlite3_vtab] structure just like xCreate. And it must also call [sqlite3_declare_vtab()] like xCreate. <p>The difference is that xConnect is called to establish a new connection to an existing virtual table whereas xCreate is called to create a new virtual table from scratch. <p>The xCreate and xConnect methods are only different when the virtual table has some kind of backing store that must be initialized the first time the virtual table is created. The xCreate method creates and initializes the backing store. The xConnect method just connects to an existing backing store. <p>As an example, consider a virtual table implementation that provides read-only access to existing comma-separated-value (CSV) files on disk. There is no backing store that needs to be created or initialized for such a virtual table (since the CSV files already exist on disk) so the xCreate and xConnect methods will be identical for that module. <p>Another example is a virtual table that implements a full-text index. The xCreate method must create and initialize data structures to hold the dictionary and posting lists for that index. The xConnect method, on the other hand, only has to locate and use an existing dictionary and posting lists that were created by a prior xCreate call. <p>The xConnect method should return [SQLITE_OK] if it is successful in creating the new virtual table, or [SQLITE_ERROR] if it is not successful. If not successful, no [sqlite3_vtab] structure should be allocated. An error message may optionally be returned in *pzErr if unsuccessful. The text of the error message should be obtained from [sqlite3_mprintf()]. <h3>2.3 The xBestIndex Method</h3> <p>The xBestIndex method has a prototype like this: <blockquote><pre> int (*xBestIndex)(sqlite3_vtab *pVTab, sqlite3_index_info*); </pre></blockquote> <p>The SQLite core communicates with the xBestIndex method by filling in certain fields of the [sqlite3_index_info] structure and passing a pointer to that structure into xBestIndex as the second parameter. The xBestIndex method fills out other fields of this structure which forms the reply. The [sqlite3_index_info] structure looks like this: <blockquote><pre> struct sqlite3_index_info { /* Inputs */ const int nConstraint; /* Number of entries in aConstraint */ const struct sqlite3_index_constraint { int iColumn; /* Column on left-hand side of constraint */ unsigned char op; /* Constraint operator */ unsigned char usable; /* True if this constraint is usable */ int iTermOffset; /* Used internally - xBestIndex should ignore */ } *const aConstraint; /* Table of WHERE clause constraints */ const int nOrderBy; /* Number of terms in the ORDER BY clause */ const struct sqlite3_index_orderby { int iColumn; /* Column number */ unsigned char desc; /* True for DESC. False for ASC. */ } *const aOrderBy; /* The ORDER BY clause */ /* Outputs */ struct sqlite3_index_constraint_usage { int argvIndex; /* if >0, constraint is part of argv to xFilter */ unsigned char omit; /* Do not code a test for this constraint */ } *const aConstraintUsage; int idxNum; /* Number used to identify the index */ char *idxStr; /* String, possibly obtained from sqlite3_malloc */ int needToFreeIdxStr; /* Free idxStr using sqlite3_free() if true */ int orderByConsumed; /* True if output is already ordered */ double estimatedCost; /* Estimated cost of using this index */ }; </pre></blockquote> <p>In addition, there are some defined constants: <blockquote><pre> #define SQLITE_INDEX_CONSTRAINT_EQ 2 #define SQLITE_INDEX_CONSTRAINT_GT 4 #define SQLITE_INDEX_CONSTRAINT_LE 8 #define SQLITE_INDEX_CONSTRAINT_LT 16 #define SQLITE_INDEX_CONSTRAINT_GE 32 #define SQLITE_INDEX_CONSTRAINT_MATCH 64 </pre></blockquote> <p>The SQLite core calls the xBestIndex method when it is compiling a query that involves a virtual table. In other words, SQLite calls this method when it is running [sqlite3_prepare()]. By calling this method, the SQLite core is saying to the virtual table that it needs to access some subset of the rows in the virtual table and it wants to know the most efficient way to do that access. The xBestIndex method replies with information that the SQLite core can then use to conduct an efficient search of the virtual table. <p>While compiling a single SQL query, the SQLite core might call xBestIndex multiple times with different settings in [sqlite3_index_info]. The SQLite core will then select the combination that appears to give the best performance. <h4>2.3.1 Inputs</h4> <p>Before calling this method, the SQLite core initializes an instance of the [sqlite3_index_info] structure with information about the query that it is currently trying to process. This information derives mainly from the WHERE clause and ORDER BY or GROUP BY clauses of the query, but also from any ON or USING clauses if the query is a join. The information that the SQLite core provides to the xBestIndex method is held in the part of the structure that is marked as "Inputs". The "Outputs" section is initialized to zero. <p>The main thing that the SQLite core is trying to communicate to the virtual table is the constraints that are available to limit the number of rows that need to be searched. The aConstraint[] array contains one entry for each constraint. There will be exactly nConstraint entries in that array. <p>Each constraint will correspond to a term in the WHERE clause or in a USING or ON clause that is of the form <blockquote> column OP EXPR </blockquote> <p>Where "column" is a column in the virtual table, OP is an operator like "=" or "<", and EXPR is an arbitrary expression. So, for example, if the WHERE clause contained a term like this: <blockquote><pre> a = 5 </pre></blockquote> <p>Then one of the constraints would be on the "a" column with operator "=" and an expression of "5". Constraints are not a literal representation of the WHERE clause. The query optimizer translates the WHERE clause in order to extract as many constraints as it can. So, for example, if the WHERE clause contained something like this: <blockquote><pre> x BETWEEN 10 AND 100 AND 999>y </pre></blockquote> <p>The query optimizer would translate this into three separate constraints: <blockquote><pre> x >= 10 x <= 100 y < 999 </pre></blockquote> <p>For each constraint, the aConstraint[].iColumn field indicates which column appears on the left-hand side of the constraint. The first column of the virtual table is column 0. The rowid of the virtual table is column -1. The aConstraint[].op field indicates which operator is used. The SQLITE_INDEX_CONSTRAINT_* constants map integer constants into operator values. <p>The aConstraint[] array contains information about all constraints that apply to the virtual table. But some of the constraints might not be usable because of the way tables are ordered in a join. The xBestIndex method should therefore only consider constraints that have a aConstraint[].usable flag which is true. <p>In addition to WHERE clause constraints, the SQLite core also tells the xBestIndex method about the ORDER BY clause. (In an aggregate query, the SQLite core might put in GROUP BY clause information in place of the ORDER BY clause information, but this fact should not make any difference to the xBestIndex method.) If all terms of the ORDER BY clause are columns in the virtual table, then nOrderBy will be the number of terms in the ORDER BY clause and the aOrderBy[] array will identify the column for each term in the order by clause and whether or not that column is ASC or DESC. <h4>2.3.2 Outputs</h4> <p>Given all of the information above, the job of the xBestIndex method it to figure out the best way to search the virtual table. <p>The xBestIndex method fills the idxNum and idxStr fields with information that communicates an indexing strategy to the xFilter method. The information in idxNum and idxStr is arbitrary as far as the SQLite core is concerned. The SQLite core just copies the information through to the xFilter method. Any desired meaning can be assigned to idxNum and idxStr as long as xBestIndex and xFilter agree on what that meaning is. <p>The idxStr value can be a string obtained from [sqlite3_mprintf()]. If this is the case, then the needToFreeIdxStr flag must be set to true so that the SQLite core will know to call [sqlite3_free()] on that string when it has finished with it, and thus avoid a memory leak. <p>If the virtual table will output rows in the order specified by the ORDER BY clause, then the orderByConsumed flag should be set to true. If the output is not automatically in the correct order then orderByConsumed should be left in its default false setting. This will indicate to the SQLite core that it will need to do a separate sorting pass over the data after it comes out of the virtual table. <p>The estimatedCost field should be set to the estimated number of disk access operations required to execute this query against the virtual table. The SQLite core will often call xBestIndex multiple times with different constraints, obtain multiple cost estimates, then choose the query plan that gives the lowest estimate. <p>The aConstraintUsage[] array contains one element for each of the nConstraint constraints in the inputs section of the [sqlite3_index_info] structure. The aConstraintUsage[] array is used by xBestIndex to tell the core how it is using the constraints. <p>The xBestIndex method may set aConstraintUsage[].argvIndex entries to values greater than one. Exactly one entry should be set to 1, another to 2, another to 3, and so forth up to as many or as few as the xBestIndex method wants. The EXPR of the corresponding constraints will then be passed in as the argv[] parameters to xFilter. <p>For example, if the aConstraint[3].argvIndex is set to 1, then when xFilter is called, the argv[0] passed to xFilter will have the EXPR value of the aConstraint[3] constraint. <p>By default, the SQLite core double checks all constraints on each row of the virtual table that it receives. If such a check is redundant, the xBestFilter method can suppress that check by setting aConstraintUsage[].omit. <h3>2.4 The xDisconnect Method</h3> <blockquote><pre> int (*xDisconnect)(sqlite3_vtab *pVTab); </pre></blockquote> <p>This method releases a connection to a virtual table. The virtual table is not destroyed and any backing store associated with the virtual table persists. This method is the opposite of xConnect. <h3>2.5 The xDestroy Method</h3> <blockquote><pre> int (*xDestroy)(sqlite3_vtab *pVTab); </pre></blockquote> <p>This method releases a connection to a virtual table, just like the xDisconnect method, and it also destroys the underlying table implementation. This method is the opposite of xCreate. <p>The xDisconnect method is called whenever a database connection that uses a virtual table is closed. The xDestroy method is only called when a [DROP TABLE] statement is executed against the virtual table. <h3>2.6 The xOpen Method</h3> <blockquote><pre> int (*xOpen)(sqlite3_vtab *pVTab, sqlite3_vtab_cursor **ppCursor); </pre></blockquote> <p>The xOpen method creates a new cursor used for reading information out of a virtual table. A successful invocation of this method will allocate the memory for the [sqlite3_vtab_cursor] (or a subclass), initialize the new object, and make *ppCursor point to the new object. The successful call then returns [SQLITE_OK]. <p>The cursor is not immediately usable. It must first be positioned using xFilter. Then elements of a row of the virtual table can be accessed using xColumn and xRowid. <p>The xClose method is used to close the cursor. <h3>2.7 The xClose Method</h3> <blockquote><pre> int (*xClose)(sqlite3_vtab_cursor*); </pre></blockquote> <p>The xClose method closes a cursor previously opened by xOpen. The SQLite core will always call xClose once for each cursor opened using xOpen. <p>This method must release all resources allocated by the corresponding xOpen call. The routine will not be called again if it returns an error. <h3>2.8 The xEof Method</h3> <blockquote><pre> int (*xEof)(sqlite3_vtab_cursor*); </pre></blockquote> <p>The xEof method must return false (zero) if the specified cursor currently points to a valid row of data, or true (non-zero) otherwise. This method is called by the SQL engine immediately after each xFilter and xNext invocation. <h3>2.9 The xFilter Method</h3> <blockquote><pre> int (*xFilter)(sqlite3_vtab_cursor*, int idxNum, const char *idxStr, int argc, sqlite3_value **argv); </pre></blockquote> <p>This method to begin a search of a virtual table. The first argument is a cursor opened by xOpen. The next two argument define a particular search index previously choosen by xBestIndex The specific meanings of idxNum and idxStr are unimportant as long as xFilter and xBestIndex agree on what that meaning is. <p>The xBestIndex function may have requested the values of certain expressions using the aConstraintUsage[].argvIndex values of the sqlite3_index_info structure. Those values are passed to xFilter using the argc and argv parameters. <p>If the filtering constraints configured by idxNum and idxStr do not match any rows of the virtual table (i.e. no rows of data will be returned), then a subsequent call to the xEof method of the same table should return non-zero. Otherwise, xEof should return zero and the cursor should be left pointing at a row of the virtual table. The xColumn and xRowid methods can be used to access that row. The xNext method can be used to advance to the next row. <p>This method should return SQLITE_OK if successful, or an sqlite error code if an error occurs. <h3>2.10 The xNext Method</h3> <blockquote><pre> int (*xNext)(sqlite3_vtab_cursor*); </pre></blockquote> <p>The xNext method advances a virtual table cursor to the next row of a result set initiated by xFilter. If the cursor is already pointing at the last row when this routine is called, then the cursor no longer points to valid data and a subsequent call to the xEof method should return non-zero. Otherwise, the xEof method returns zero. <p>This method should return SQLITE_OK if successful, or an sqlite error code if an error occurs. <h3>2.11 The xColumn Method</h3> <blockquote><pre> int (*xColumn)(sqlite3_vtab_cursor*, sqlite3_context*, int N); </pre></blockquote> <p>The SQLite core invokes this method in order to find the value for the N-th column of the current row. N is zero-based so the first column is numbered 0. The xColumn method uses one of the [sqlite3_result_blob | sqlite3_result_*() APIs] to return the result. This method can invoke [sqlite3_result_error()] to raise an exception, if desired. <h3>2.12 The xRowid Method</h3> <blockquote><pre> int (*xRowid)(sqlite3_vtab_cursor*, sqlite_int64 *pRowid); </pre></blockquote> <p>This method writes the 64-bit rowid of the current row into *pRowid. <h3>2.13 The xUpdate Method</h3> <blockquote><pre> int (*xUpdate)( sqlite3_vtab *pVTab, int argc, sqlite3_value **argv, sqlite_int64 *pRowid ); </pre></blockquote> <p>All changes to a virtual table are made using the xUpdate method. This one method can be used to insert, delete, or update. <p>The argc parameter specifies the number of entries in the argv array. Every argv entry will have a non-NULL value in C (but may contain the SQL value NULL). <p>The argv[0] parameter is the rowid of a row in the virtual table to be deleted. If argv[0] is NULL, then no deletion occurs. <p>The argv[1] parameter is the rowid of a new row to be inserted into the virtual table. If argv[1] is NULL, then the implementation must choose a rowid for the newly inserted row. Subsequent argv[] entries contain values of the columns of the virtual table, in the order that the columns were declared. The number of columns will match the table declaration that the xConnect or xCreate method made using the [sqlite3_declare_vtab()] call. <p>When doing an insert without a rowid (argc>1, argv[1]==NULL), the implementation must set *pRowid to the id of the newly inserted row; this will become the value returned by the [sqlite3_last_insert_rowid()] function. Setting this value in all the other cases has no effect, and cannot be used to change the rowid of an existing row or if you don't like what was in argv[1]. <p>Each call to xUpdate will fall into one of the following cases: <blockquote> <dl> <dt><b>argc == 1</b> <dd><p>The single row argv[0] is deleted; no insert occurs <dt><b>argc > 1 && argv[0] == NULL</b> <dd><p>A new row is inserted with rowid argv[1] <dt><b>argc > 1 && argv[0] != NULL && argv[0] == argv[1]</b> <dd><p>The row with rowid argv[0] is updated with new values in argv[2] and following parameters <dt><b>argc > 1 && argv[0] != NULL && argv[0] != argv[1]</b> <dd><p> The row with rowid argv[0] is updated with rowid argv[1] and new values in argv[2] and following parameters. This will occur when an SQL statement updates a rowid, as in the statement: <blockquote> [UPDATE] table SET rowid=rowid+1 WHERE ...; </blockquote> </dl> </blockquote> <p>It is currently undocumented what effect changes should have on existing cursors. <h3>2.14 The xFindFunction Method</h3> <blockquote><pre> int (*xFindFunction)( sqlite3_vtab *pVtab, int nArg, const char *zName, void (**pxFunc)(sqlite3_context*,int,sqlite3_value**), void **ppArg ); </pre></blockquote> <p>This method is called during sqlite3_prepare to give the virtual table implementation an opportunity to overload functions. This method may be set to NULL in which case no overloading occurs. <p>When a function uses a column from a virtual table as its first argument, this method is called to see if the virtual table would like to overload the function. The first three parameters are inputs: the virtual table, the number of arguments to the function, and the name of the function. If no overloading is desired, this method returns 0. To overload the function, this method writes the new function implementation into *pxFunc and writes user data into *ppArg and returns 1. <p>Note that infix functions (LIKE, GLOB, REGEXP, and MATCH) reverse the order of their arguments. So "like(A,B)" is equivalent to "B like A". For the form "B like A" the B term is considered the first argument to the function. But for "like(A,B)" the A term is considered the first argument. <p>It is currently undocumented how often this function is called or when you can free any resources allocated in order to respond - #2095 <h3>2.15 The xBegin Method</h3> <blockquote><pre> int (*xBegin)(sqlite3_vtab *pVTab); </pre></blockquote> <i>TBD...</i> <h3>2.16 The xSync Method</h3> <blockquote><pre> int (*xSync)(sqlite3_vtab *pVTab); </pre></blockquote> <i>TBD...</i> <h3>2.17 The xCommit Method</h3> <blockquote><pre> int (*xCommit)(sqlite3_vtab *pVTab); </pre></blockquote> <i>TBD...</i> <h3>2.18 The xRollback Method</h3> <blockquote><pre> int (*xRollback)(sqlite3_vtab *pVTab); </pre></blockquote> <i>TBD...</i> <h3>2.19 The xRename Method</h3> <blockquote><pre> int (*xRename)(sqlite3_vtab *pVtab, const char *zNew); </pre></blockquote> <p>This method provides sotification that the virtual table implementation that the virtual table will be given a new name. If this method returns SQLITE_OK then SQLite renames the table. If this method returns an error then the renaming is prevented. |