SQLite

Check-in [c444836e7b]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:First approximation of incremental blob IO API. (CVS 3892)
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: c444836e7b690c16dd6acff571c613a23beb42dc
User & Date: danielk1977 2007-05-01 17:49:49.000
Context
2007-05-01
18:25
Enable prefix-search in query-parsing and snippet generation. If the character immediately after the end of a term is '*', that term is marked for prefix matching. Modify term comparison in snippetOffsetsOfColumn() to respect isPrefix. fts2n.test runs prefix searching through some obvious test cases. (CVS 3893) (check-in: 7c4c659240 user: shess tags: trunk)
17:49
First approximation of incremental blob IO API. (CVS 3892) (check-in: c444836e7b user: danielk1977 tags: trunk)
17:14
Modify loadSegmentLeavesInt() to correctly handle prefix searching. The new function docListUnion() is used to accumulate a union of the hits for the matching terms, which will be merged across segments using docListMerge(). (CVS 3891) (check-in: 72c7963073 user: shess tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to main.mk.
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
LIBOBJ+= alter.o analyze.o attach.o auth.o btree.o build.o \
         callback.o complete.o date.o delete.o \
         expr.o func.o hash.o insert.o loadext.o \
         main.o opcodes.o os.o os_os2.o os_unix.o os_win.o \
         pager.o parse.o pragma.o prepare.o printf.o random.o \
         select.o table.o tclsqlite.o tokenize.o trigger.o \
         update.o util.o vacuum.o \
         vdbe.o vdbeapi.o vdbeaux.o vdbefifo.o vdbemem.o \
         where.o utf.o legacy.o vtab.o

# All of the source code files.
#
SRC = \
  $(TOP)/src/alter.c \
  $(TOP)/src/analyze.c \







|







58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
LIBOBJ+= alter.o analyze.o attach.o auth.o btree.o build.o \
         callback.o complete.o date.o delete.o \
         expr.o func.o hash.o insert.o loadext.o \
         main.o opcodes.o os.o os_os2.o os_unix.o os_win.o \
         pager.o parse.o pragma.o prepare.o printf.o random.o \
         select.o table.o tclsqlite.o tokenize.o trigger.o \
         update.o util.o vacuum.o \
         vdbe.o vdbeapi.o vdbeaux.o vdbeblob.o vdbefifo.o vdbemem.o \
         where.o utf.o legacy.o vtab.o

# All of the source code files.
#
SRC = \
  $(TOP)/src/alter.c \
  $(TOP)/src/analyze.c \
111
112
113
114
115
116
117

118
119
120
121
122
123
124
  $(TOP)/src/update.c \
  $(TOP)/src/util.c \
  $(TOP)/src/vacuum.c \
  $(TOP)/src/vdbe.c \
  $(TOP)/src/vdbe.h \
  $(TOP)/src/vdbeapi.c \
  $(TOP)/src/vdbeaux.c \

  $(TOP)/src/vdbefifo.c \
  $(TOP)/src/vdbemem.c \
  $(TOP)/src/vdbeInt.h \
  $(TOP)/src/vtab.c \
  $(TOP)/src/where.c

# Source code for extensions







>







111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
  $(TOP)/src/update.c \
  $(TOP)/src/util.c \
  $(TOP)/src/vacuum.c \
  $(TOP)/src/vdbe.c \
  $(TOP)/src/vdbe.h \
  $(TOP)/src/vdbeapi.c \
  $(TOP)/src/vdbeaux.c \
  $(TOP)/src/vdbeblob.c \
  $(TOP)/src/vdbefifo.c \
  $(TOP)/src/vdbemem.c \
  $(TOP)/src/vdbeInt.h \
  $(TOP)/src/vtab.c \
  $(TOP)/src/where.c

# Source code for extensions
405
406
407
408
409
410
411



412
413
414
415
416
417
418
	$(TCCX) -c $(TOP)/src/vdbe.c

vdbeapi.o:	$(TOP)/src/vdbeapi.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbeapi.c

vdbeaux.o:	$(TOP)/src/vdbeaux.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbeaux.c




vdbefifo.o:	$(TOP)/src/vdbefifo.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbefifo.c

vdbemem.o:	$(TOP)/src/vdbemem.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbemem.c








>
>
>







406
407
408
409
410
411
412
413
414
415
416
417
418
419
420
421
422
	$(TCCX) -c $(TOP)/src/vdbe.c

vdbeapi.o:	$(TOP)/src/vdbeapi.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbeapi.c

vdbeaux.o:	$(TOP)/src/vdbeaux.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbeaux.c

vdbeblob.o:	$(TOP)/src/vdbeblob.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbeblob.c

vdbefifo.o:	$(TOP)/src/vdbefifo.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbefifo.c

vdbemem.o:	$(TOP)/src/vdbemem.c $(VDBEHDR) $(HDR)
	$(TCCX) -c $(TOP)/src/vdbemem.c

Changes to src/btree.c.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/*
** 2004 April 6
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** $Id: btree.c,v 1.362 2007/04/30 16:55:01 danielk1977 Exp $
**
** This file implements a external (disk-based) database using BTrees.
** For a detailed discussion of BTrees, refer to
**
**     Donald E. Knuth, THE ART OF COMPUTER PROGRAMMING, Volume 3:
**     "Sorting And Searching", pages 473-480. Addison-Wesley
**     Publishing Company, Reading, Massachusetts.











|







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
/*
** 2004 April 6
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** $Id: btree.c,v 1.363 2007/05/01 17:49:49 danielk1977 Exp $
**
** This file implements a external (disk-based) database using BTrees.
** For a detailed discussion of BTrees, refer to
**
**     Donald E. Knuth, THE ART OF COMPUTER PROGRAMMING, Volume 3:
**     "Sorting And Searching", pages 473-480. Addison-Wesley
**     Publishing Company, Reading, Massachusetts.
6805
6806
6807
6808
6809
6810
6811























































6812
6813
6814
6815
6816
6817
6818
  rc = queryTableLock(p, iTab, lockType);
  if( rc==SQLITE_OK ){
    rc = lockTable(p, iTab, lockType);
  }
  return rc;
}
#endif
























































/*
** The following debugging interface has to be in this file (rather
** than in, for example, test1.c) so that it can get access to
** the definition of BtShared.
*/
#if defined(SQLITE_DEBUG) && defined(TCLSH)







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







6805
6806
6807
6808
6809
6810
6811
6812
6813
6814
6815
6816
6817
6818
6819
6820
6821
6822
6823
6824
6825
6826
6827
6828
6829
6830
6831
6832
6833
6834
6835
6836
6837
6838
6839
6840
6841
6842
6843
6844
6845
6846
6847
6848
6849
6850
6851
6852
6853
6854
6855
6856
6857
6858
6859
6860
6861
6862
6863
6864
6865
6866
6867
6868
6869
6870
6871
6872
6873
  rc = queryTableLock(p, iTab, lockType);
  if( rc==SQLITE_OK ){
    rc = lockTable(p, iTab, lockType);
  }
  return rc;
}
#endif

#ifndef SQLITE_OMIT_INCRBLOB
/*
** Argument pCsr must be a cursor opened for writing on an 
** INTKEY table currently pointing at a valid table entry. 
** This function modifies the data stored as part of that entry.
** Only the data content may only be modified, it is not possible
** to change the length of the data stored.
*/
int sqlite3BtreePutData(BtCursor *pCsr, u32 offset, u32 amt, const void *z){
  /* TODO: The following is only a stop-gap implementation. It needs
  ** to be made efficient using the optimistic overflow page trick. 
  ** Similar changes need to be made to sqlite3BtreeData().
  */
  i64 iKey;
  int rc;

  int nCopy;
  u32 nData;
  char *zData;

  rc = sqlite3BtreeKeySize(pCsr, &iKey);
  if( rc!=SQLITE_OK ){
    return rc;
  }

  rc = sqlite3BtreeDataSize(pCsr, &nData);
  if( rc!=SQLITE_OK ){
    return rc;
  }

  zData = sqliteMalloc(nData);
  if( !zData ){
    return SQLITE_NOMEM;
  }

  rc = sqlite3BtreeData(pCsr, 0, nData, (void *)zData);
  if( rc!=SQLITE_OK ){
    sqliteFree(zData);
    return rc;
  }

  nCopy = amt;
  if( nCopy>(nData-offset) ){
    nCopy = nData-offset;
  }
  if( nCopy>0 ){
    memcpy(&zData[offset], z, amt);
    rc = sqlite3BtreeInsert(pCsr, 0, iKey, zData, nData, 0);
  }

  sqliteFree(zData);
  return rc;
}
#endif

/*
** The following debugging interface has to be in this file (rather
** than in, for example, test1.c) so that it can get access to
** the definition of BtShared.
*/
#if defined(SQLITE_DEBUG) && defined(TCLSH)
Changes to src/btree.h.
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
**    May you share freely, never taking more than you give.
**
*************************************************************************
** This header file defines the interface that the sqlite B-Tree file
** subsystem.  See comments in the source code for a detailed description
** of what each interface routine does.
**
** @(#) $Id: btree.h,v 1.75 2007/04/26 14:42:36 danielk1977 Exp $
*/
#ifndef _BTREE_H_
#define _BTREE_H_

/* TODO: This definition is just included so other modules compile. It
** needs to be revisited.
*/







|







9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
**    May you share freely, never taking more than you give.
**
*************************************************************************
** This header file defines the interface that the sqlite B-Tree file
** subsystem.  See comments in the source code for a detailed description
** of what each interface routine does.
**
** @(#) $Id: btree.h,v 1.76 2007/05/01 17:49:49 danielk1977 Exp $
*/
#ifndef _BTREE_H_
#define _BTREE_H_

/* TODO: This definition is just included so other modules compile. It
** needs to be revisited.
*/
137
138
139
140
141
142
143

144
145
146
147
148
149
150
151
152
153
154
155
156
const void *sqlite3BtreeDataFetch(BtCursor*, int *pAmt);
int sqlite3BtreeDataSize(BtCursor*, u32 *pSize);
int sqlite3BtreeData(BtCursor*, u32 offset, u32 amt, void*);

char *sqlite3BtreeIntegrityCheck(Btree*, int *aRoot, int nRoot, int, int*);
struct Pager *sqlite3BtreePager(Btree*);



#ifdef SQLITE_TEST
int sqlite3BtreeCursorInfo(BtCursor*, int*, int);
void sqlite3BtreeCursorList(Btree*);
#endif

#ifdef SQLITE_DEBUG
int sqlite3BtreePageDump(Btree*, int, int recursive);
#else
#define sqlite3BtreePageDump(X,Y,Z) SQLITE_OK
#endif

#endif /* _BTREE_H_ */







>













137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
const void *sqlite3BtreeDataFetch(BtCursor*, int *pAmt);
int sqlite3BtreeDataSize(BtCursor*, u32 *pSize);
int sqlite3BtreeData(BtCursor*, u32 offset, u32 amt, void*);

char *sqlite3BtreeIntegrityCheck(Btree*, int *aRoot, int nRoot, int, int*);
struct Pager *sqlite3BtreePager(Btree*);

int sqlite3BtreePutData(BtCursor*, u32 offset, u32 amt, const void*);

#ifdef SQLITE_TEST
int sqlite3BtreeCursorInfo(BtCursor*, int*, int);
void sqlite3BtreeCursorList(Btree*);
#endif

#ifdef SQLITE_DEBUG
int sqlite3BtreePageDump(Btree*, int, int recursive);
#else
#define sqlite3BtreePageDump(X,Y,Z) SQLITE_OK
#endif

#endif /* _BTREE_H_ */
Changes to src/sqlite.h.in.
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** This header file defines the interface that the SQLite library
** presents to client programs.
**
** @(#) $Id: sqlite.h.in,v 1.201 2007/03/30 20:43:42 drh Exp $
*/
#ifndef _SQLITE3_H_
#define _SQLITE3_H_
#include <stdarg.h>     /* Needed for the definition of va_list */

/*
** Make sure we can call this stuff from C++.







|







8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** This header file defines the interface that the SQLite library
** presents to client programs.
**
** @(#) $Id: sqlite.h.in,v 1.202 2007/05/01 17:49:49 danielk1977 Exp $
*/
#ifndef _SQLITE3_H_
#define _SQLITE3_H_
#include <stdarg.h>     /* Needed for the definition of va_list */

/*
** Make sure we can call this stuff from C++.
1866
1867
1868
1869
1870
1871
1872


















1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
** If this is a problem for you, do not use the interface at this time.
**
** When the virtual-table mechanism stablizes, we will declare the
** interface fixed, support it indefinitely, and remove this comment.
**
****** EXPERIMENTAL - subject to change without notice **************
*/



















/*
** Undo the hack that converts floating point types to integer for
** builds on processors without floating point support.
*/
#ifdef SQLITE_OMIT_FLOATING_POINT
# undef double
#endif

#ifdef __cplusplus
}  /* End of the 'extern "C"' block */
#endif
#endif







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>













1866
1867
1868
1869
1870
1871
1872
1873
1874
1875
1876
1877
1878
1879
1880
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
** If this is a problem for you, do not use the interface at this time.
**
** When the virtual-table mechanism stablizes, we will declare the
** interface fixed, support it indefinitely, and remove this comment.
**
****** EXPERIMENTAL - subject to change without notice **************
*/

typedef struct sqlite3_blob sqlite3_blob;

int sqlite3_blob_open(
  sqlite3*,
  const char *zDb,
  const char *zTable,
  const char *zColumn,
  sqlite_int64 iRow,
  int flags,
  sqlite3_blob **ppBlob
);

int sqlite3_blob_close(sqlite3_blob *);

int sqlite3_blob_read(sqlite3_blob *, void *z, int n, int iOffset);
int sqlite3_blob_write(sqlite3_blob *, const void *z, int n, int iOffset);
int sqlite3_blob_bytes(sqlite3_blob *);

/*
** Undo the hack that converts floating point types to integer for
** builds on processors without floating point support.
*/
#ifdef SQLITE_OMIT_FLOATING_POINT
# undef double
#endif

#ifdef __cplusplus
}  /* End of the 'extern "C"' block */
#endif
#endif
Changes to src/tclsqlite.c.
8
9
10
11
12
13
14
15
16
17

18
19
20
21
22
23
24
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** A TCL Interface to SQLite.  Append this file to sqlite3.c and
** compile the whole thing to build a TCL-enabled version of SQLite.
**
** $Id: tclsqlite.c,v 1.179 2007/04/06 15:02:14 drh Exp $
*/
#include "tcl.h"


/*
** Some additional include files are needed if this file is not
** appended to the amalgamation.
*/
#ifndef SQLITE_AMALGAMATION
# include "sqliteInt.h"







|


>







8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
** A TCL Interface to SQLite.  Append this file to sqlite3.c and
** compile the whole thing to build a TCL-enabled version of SQLite.
**
** $Id: tclsqlite.c,v 1.180 2007/05/01 17:49:49 danielk1977 Exp $
*/
#include "tcl.h"
#include <errno.h>

/*
** Some additional include files are needed if this file is not
** appended to the amalgamation.
*/
#ifndef SQLITE_AMALGAMATION
# include "sqliteInt.h"
110
111
112
113
114
115
116












































































































































































117
118
119
120
121
122
123
  int rc;                    /* Return code of most recent sqlite3_exec() */
  Tcl_Obj *pCollateNeeded;   /* Collation needed script */
  SqlPreparedStmt *stmtList; /* List of prepared statements*/
  SqlPreparedStmt *stmtLast; /* Last statement in the list */
  int maxStmt;               /* The next maximum number of stmtList */
  int nStmt;                 /* Number of statements in stmtList */
};













































































































































































/*
** Look at the script prefix in pCmd.  We will be executing this script
** after first appending one or more arguments.  This routine analyzes
** the script to see if it is safe to use Tcl_EvalObjv() on the script
** rather than the more general Tcl_EvalEx().  Tcl_EvalObjv() is much
** faster.







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
261
262
263
264
265
266
267
268
269
270
271
272
273
274
275
276
277
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
294
295
296
  int rc;                    /* Return code of most recent sqlite3_exec() */
  Tcl_Obj *pCollateNeeded;   /* Collation needed script */
  SqlPreparedStmt *stmtList; /* List of prepared statements*/
  SqlPreparedStmt *stmtLast; /* Last statement in the list */
  int maxStmt;               /* The next maximum number of stmtList */
  int nStmt;                 /* Number of statements in stmtList */
};

typedef struct IncrblobChannel IncrblobChannel;
struct IncrblobChannel {
  sqlite3_blob *pBlob;
  int iSeek;              /* Current seek offset */
};

/*
** Close an incremental blob channel.
*/
static int incrblobClose(ClientData instanceData, Tcl_Interp *interp){
  IncrblobChannel *p = (IncrblobChannel *)instanceData;
  sqlite3_blob_close(p->pBlob);
  Tcl_Free((char *)p);
  return TCL_OK;
}

/*
** Read data from an incremental blob channel.
*/
static int incrblobInput(
  ClientData instanceData, 
  char *buf, 
  int bufSize,
  int *errorCodePtr
){
  IncrblobChannel *p = (IncrblobChannel *)instanceData;
  int nRead = bufSize;         /* Number of bytes to read */
  int nBlob;                   /* Total size of the blob */
  int rc;                      /* sqlite error code */

  nBlob = sqlite3_blob_bytes(p->pBlob);
  if( (p->iSeek+nRead)>nBlob ){
    nRead = nBlob-p->iSeek;
  }
  if( nRead<=0 ){
    return 0;
  }

  rc = sqlite3_blob_read(p->pBlob, (void *)buf, nRead, p->iSeek);
  if( rc!=SQLITE_OK ){
    *errorCodePtr = rc;
    return -1;
  }

  p->iSeek += nRead;
  return nRead;
}

static int incrblobOutput(
  ClientData instanceData, 
  CONST char *buf, 
  int toWrite,
  int *errorCodePtr
){
  IncrblobChannel *p = (IncrblobChannel *)instanceData;
  int nWrite = toWrite;        /* Number of bytes to write */
  int nBlob;                   /* Total size of the blob */
  int rc;                      /* sqlite error code */

  nBlob = sqlite3_blob_bytes(p->pBlob);
  if( (p->iSeek+nWrite)>nBlob ){
    *errorCodePtr = EINVAL;
    return -1;
  }
  if( nWrite<=0 ){
    return 0;
  }

  rc = sqlite3_blob_write(p->pBlob, (void *)buf, nWrite, p->iSeek);
  if( rc!=SQLITE_OK ){
    *errorCodePtr = EIO;
    return -1;
  }

  p->iSeek += nWrite;
  return nWrite;
}

/*
** Seek an incremental blob channel.
*/
static int incrblobSeek(
  ClientData instanceData, 
  long offset,
  int seekMode,
  int *errorCodePtr
){
  IncrblobChannel *p = (IncrblobChannel *)instanceData;

  switch( seekMode ){
    case SEEK_SET:
      p->iSeek = offset;
      break;
    case SEEK_CUR:
      p->iSeek += offset;
      break;
    case SEEK_END:
      p->iSeek = sqlite3_blob_bytes(p->pBlob) + offset;
      break;

    default: assert(!"Bad seekMode");
  }

  return p->iSeek;
}


static void incrblobWatch(ClientData instanceData, int mode){ 
  /* NO-OP */ 
}
static int incrblobHandle(ClientData instanceData, int dir, ClientData *hPtr){
  return TCL_ERROR;
}

static Tcl_ChannelType IncrblobChannelType = {
  "incrblob",                        /* typeName                             */
  TCL_CHANNEL_VERSION_2,             /* version                              */
  incrblobClose,                     /* closeProc                            */
  incrblobInput,                     /* inputProc                            */
  incrblobOutput,                    /* outputProc                           */
  incrblobSeek,                      /* seekProc                             */
  0,                                 /* setOptionProc                        */
  0,                                 /* getOptionProc                        */
  incrblobWatch,                     /* watchProc (this is a no-op)          */
  incrblobHandle,                    /* getHandleProc (always returns error) */
  0,                                 /* close2Proc                           */
  0,                                 /* blockModeProc                        */
  0,                                 /* flushProc                            */
  0,                                 /* handlerProc                          */
  0,                                 /* wideSeekProc                         */
  0,                                 /* threadActionProc                     */
};

/*
** Create a new incrblob channel.
*/
static int createIncrblobChannel(
  Tcl_Interp *interp, 
  SqliteDb *pDb, 
  const char *zDb,
  const char *zTable, 
  const char *zColumn, 
  sqlite_int64 iRow
){
  IncrblobChannel *p;
  sqlite3_blob *pBlob;
  int rc;
  Tcl_Channel channel;
  int flags = TCL_READABLE|TCL_WRITABLE;

  /* This variable is used to name the channels: "incrblob_[incr count]" */
  static int count = 0;
  char zChannel[64];

  rc = sqlite3_blob_open(pDb->db, zDb, zTable, zColumn, iRow, 1, &pBlob);
  if( rc!=SQLITE_OK ){
    Tcl_SetResult(interp, (char *)sqlite3_errmsg(pDb->db), TCL_VOLATILE);
    return TCL_ERROR;
  }

  p = (IncrblobChannel *)Tcl_Alloc(sizeof(IncrblobChannel));
  p->iSeek = 0;
  p->pBlob = pBlob;

  sprintf(zChannel, "incrblob_%d", ++count);
  channel = Tcl_CreateChannel(&IncrblobChannelType, zChannel, p, flags);
  Tcl_RegisterChannel(interp, channel);

  Tcl_SetResult(interp, (char *)Tcl_GetChannelName(channel), TCL_VOLATILE);
  return TCL_OK;
}

/*
** Look at the script prefix in pCmd.  We will be executing this script
** after first appending one or more arguments.  This routine analyzes
** the script to see if it is safe to use Tcl_EvalObjv() on the script
** rather than the more general Tcl_EvalEx().  Tcl_EvalObjv() is much
** faster.
672
673
674
675
676
677
678

679
680
681
682
683
684
685
686
687
688
689
690

691
692
693
694
695
696
697
  int rc = TCL_OK;
  static const char *DB_strs[] = {
    "authorizer",         "busy",              "cache",
    "changes",            "close",             "collate",
    "collation_needed",   "commit_hook",       "complete",
    "copy",               "enable_load_extension","errorcode",
    "eval",               "exists",            "function",

    "interrupt",          "last_insert_rowid", "nullvalue",
    "onecolumn",          "profile",           "progress",
    "rekey",              "rollback_hook",     "timeout",
    "total_changes",      "trace",             "transaction",
    "update_hook",        "version",           0
  };
  enum DB_enum {
    DB_AUTHORIZER,        DB_BUSY,             DB_CACHE,
    DB_CHANGES,           DB_CLOSE,            DB_COLLATE,
    DB_COLLATION_NEEDED,  DB_COMMIT_HOOK,      DB_COMPLETE,
    DB_COPY,              DB_ENABLE_LOAD_EXTENSION,DB_ERRORCODE,
    DB_EVAL,              DB_EXISTS,           DB_FUNCTION,

    DB_INTERRUPT,         DB_LAST_INSERT_ROWID,DB_NULLVALUE,
    DB_ONECOLUMN,         DB_PROFILE,          DB_PROGRESS,
    DB_REKEY,             DB_ROLLBACK_HOOK,    DB_TIMEOUT,
    DB_TOTAL_CHANGES,     DB_TRACE,            DB_TRANSACTION,
    DB_UPDATE_HOOK,       DB_VERSION,          
  };
  /* don't leave trailing commas on DB_enum, it confuses the AIX xlc compiler */







>












>







845
846
847
848
849
850
851
852
853
854
855
856
857
858
859
860
861
862
863
864
865
866
867
868
869
870
871
872
  int rc = TCL_OK;
  static const char *DB_strs[] = {
    "authorizer",         "busy",              "cache",
    "changes",            "close",             "collate",
    "collation_needed",   "commit_hook",       "complete",
    "copy",               "enable_load_extension","errorcode",
    "eval",               "exists",            "function",
    "incrblob",
    "interrupt",          "last_insert_rowid", "nullvalue",
    "onecolumn",          "profile",           "progress",
    "rekey",              "rollback_hook",     "timeout",
    "total_changes",      "trace",             "transaction",
    "update_hook",        "version",           0
  };
  enum DB_enum {
    DB_AUTHORIZER,        DB_BUSY,             DB_CACHE,
    DB_CHANGES,           DB_CLOSE,            DB_COLLATE,
    DB_COLLATION_NEEDED,  DB_COMMIT_HOOK,      DB_COMPLETE,
    DB_COPY,              DB_ENABLE_LOAD_EXTENSION,DB_ERRORCODE,
    DB_EVAL,              DB_EXISTS,           DB_FUNCTION,
    DB_INCRBLOB,
    DB_INTERRUPT,         DB_LAST_INSERT_ROWID,DB_NULLVALUE,
    DB_ONECOLUMN,         DB_PROFILE,          DB_PROGRESS,
    DB_REKEY,             DB_ROLLBACK_HOOK,    DB_TIMEOUT,
    DB_TOTAL_CHANGES,     DB_TRACE,            DB_TRANSACTION,
    DB_UPDATE_HOOK,       DB_VERSION,          
  };
  /* don't leave trailing commas on DB_enum, it confuses the AIX xlc compiler */
1618
1619
1620
1621
1622
1623
1624



























1625
1626
1627
1628
1629
1630
1631
      Tcl_SetResult(interp, (char *)sqlite3_errmsg(pDb->db), TCL_VOLATILE);
    }else{
      /* Must flush any cached statements */
      flushStmtCache( pDb );
    }
    break;
  }




























  /*
  **     $db interrupt
  **
  ** Interrupt the execution of the inner-most SQL interpreter.  This
  ** causes the SQL statement to return an error of SQLITE_INTERRUPT.
  */







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808
1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
1822
1823
1824
1825
1826
1827
1828
1829
1830
1831
1832
1833
      Tcl_SetResult(interp, (char *)sqlite3_errmsg(pDb->db), TCL_VOLATILE);
    }else{
      /* Must flush any cached statements */
      flushStmtCache( pDb );
    }
    break;
  }

  /*
  **     $db incrblob ?DB? TABLE COLUMN ROWID
  */
  case DB_INCRBLOB: {
    const char *zDb = "main";
    const char *zTable;
    const char *zColumn;
    sqlite_int64 iRow;

    if( objc!=5 && objc!=6 ){
      Tcl_WrongNumArgs(interp, 2, objv, "?DB? TABLE ROWID");
      return TCL_ERROR;
    }

    if( objc==6 ){
      zDb = Tcl_GetString(objv[2]);
    }
    zTable = Tcl_GetString(objv[objc-3]);
    zColumn = Tcl_GetString(objv[objc-2]);
    rc = Tcl_GetWideIntFromObj(interp, objv[objc-1], &iRow);

    if( rc==TCL_OK ){
      rc = createIncrblobChannel(interp, pDb, zDb, zTable, zColumn, iRow);
    }
    break;
  }

  /*
  **     $db interrupt
  **
  ** Interrupt the execution of the inner-most SQL interpreter.  This
  ** causes the SQL statement to return an error of SQLITE_INTERRUPT.
  */
Added src/vdbeblob.c.






















































































































































































































































































































































































































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
/*
** 2007 May 1
**
** The author disclaims copyright to this source code.  In place of
** a legal notice, here is a blessing:
**
**    May you do good and not evil.
**    May you find forgiveness for yourself and forgive others.
**    May you share freely, never taking more than you give.
**
*************************************************************************
**
** $Id: vdbeblob.c,v 1.1 2007/05/01 17:49:49 danielk1977 Exp $
*/

#include "sqliteInt.h"
#include "vdbeInt.h"

#ifndef SQLITE_OMIT_INCRBLOB

/*
** Valid sqlite3_blob* handles point to Incrblob structures.
*/
typedef struct Incrblob Incrblob;

struct Incrblob {
  int flags;              /* Copy of "flags" passed to sqlite3_blob_open() */
  int nByte;              /* Size of open blob, in bytes */
  int iOffset;            /* Byte offset of blob in cursor data */
  BtCursor *pCsr;         /* Cursor pointing at blob row */
  sqlite3_stmt *pStmt;    /* Statement holding cursor open */
};

/*
** Open a blob handle.
*/
int sqlite3_blob_open(
  sqlite3* db,
  const char *zDb,
  const char *zTable,
  const char *zColumn,
  sqlite_int64 iRow,
  int flags,              /* True -> read/write access, false -> read-only */
  sqlite3_blob **ppBlob
){
  int rc = SQLITE_OK;
  int nAttempt = 0;
  int iCol;               /* Index of zColumn in row-record */

  /* This VDBE program seeks a btree cursor to the identified 
  ** db/table/row entry. The reason for using a vdbe program instead
  ** of writing code to use the b-tree layer directly is that the
  ** vdbe program will take advantage of the various transaction,
  ** locking and error handling infrastructure built into the vdbe.
  **
  ** After seeking the cursor, the vdbe executes an OP_Callback.
  ** Code external to the Vdbe then "borrows" the b-tree cursor and
  ** uses it to implement the blob_read(), blob_write() and 
  ** blob_bytes() functions.
  **
  ** The sqlite3_blob_close() function finalizes the vdbe program,
  ** which closes the b-tree cursor and (possibly) commits the 
  ** transaction.
  */
  static const VdbeOpList openBlob[] = {
    {OP_Transaction, 0, 0, 0},     /* 0: Start a transaction */
    {OP_VerifyCookie, 0, 0, 0},    /* 1: Check the schema cookie */
    {OP_Integer, 0, 0, 0},         /* 2: Database number */

    /* One of the following two instructions is replaced by an
    ** OP_Noop before exection.
    */
    {OP_OpenRead, 0, 0, 0},        /* 3: Open cursor 0 for reading */
    {OP_OpenWrite, 0, 0, 0},       /* 4: Open cursor 0 for read/write */
    {OP_SetNumColumns, 0, 0, 0},   /* 5: Num cols for cursor */

    {OP_Variable, 1, 0, 0},        /* 6: Push the rowid to the stack */
    {OP_NotExists, 0, 10, 0},      /* 7: Seek the cursor */
    {OP_Column, 0, 0, 0},          /* 8  */
    {OP_Callback, 0, 0, 0},        /* 9  */
    {OP_Close, 0, 0, 0},           /* 10 */
    {OP_Halt, 0, 0, 0},            /* 11 */
  };

  Vdbe *v = 0;

  do {
    Parse sParse;
    Table *pTab;

    memset(&sParse, 0, sizeof(Parse));
    sParse.db = db;

    pTab = sqlite3LocateTable(&sParse, zTable, zDb);
    if( !pTab ){
      sqlite3Error(db, sParse.rc, "%s", sParse.zErrMsg);
      sqliteFree(sParse.zErrMsg);
      rc = sParse.rc;
      goto blob_open_out;
    }

    /* Now search pTab for the exact column. */
    for(iCol=0; iCol < pTab->nCol; iCol++) {
      if( sqlite3StrICmp(pTab->aCol[iCol].zName, zColumn)==0 ){
        break;
      }
    }
    if( iCol==pTab->nCol ){
      sqlite3Error(db, SQLITE_ERROR, "no such column: %s", zColumn);
      sqliteFree(sParse.zErrMsg);
      rc = SQLITE_ERROR;
      goto blob_open_out;
    }

    v = sqlite3VdbeCreate(db);
    if( v ){
      int iDb = sqlite3SchemaToIndex(db, pTab->pSchema);
      sqlite3VdbeAddOpList(v, sizeof(openBlob)/sizeof(VdbeOpList), openBlob);

      /* Configure the OP_Transaction */
      sqlite3VdbeChangeP1(v, 0, iDb);
      sqlite3VdbeChangeP2(v, 0, (flags ? 1 : 0));

      /* Configure the OP_VerifyCookie */
      sqlite3VdbeChangeP1(v, 1, iDb);
      sqlite3VdbeChangeP2(v, 1, pTab->pSchema->schema_cookie);

      /* Configure the db number pushed onto the stack */
      sqlite3VdbeChangeP1(v, 2, iDb);

      /* Remove either the OP_OpenWrite or OpenRead. Set the P2 parameter
      ** of the other to pTab->tnum. 
      */
      sqlite3VdbeChangeToNoop(v, (flags ? 3 : 4), 1);
      sqlite3VdbeChangeP2(v, (flags ? 4 : 3), pTab->tnum);

      /* Configure the OP_SetNumColumns. Configure the cursor to
      ** think that the table has one more column than it really
      ** does. An OP_Column to retrieve this imaginary column will
      ** always return an SQL NULL. This is useful because it means
      ** we can invoke OP_Column to fill in the vdbe cursors type 
      ** and offset cache without causing any IO.
      */
      sqlite3VdbeChangeP2(v, 5, pTab->nCol+1);
      sqlite3VdbeMakeReady(v, 1, 0, 1, 0);
    }

    sqlite3_bind_int64((sqlite3_stmt *)v, 1, iRow);
    rc = sqlite3_step((sqlite3_stmt *)v);
    if( rc!=SQLITE_ROW ){
      nAttempt++;
      rc = sqlite3_finalize((sqlite3_stmt *)v);
      v = 0;
    }
  } while( nAttempt<5 && rc==SQLITE_SCHEMA );

  if( rc==SQLITE_ROW ){
    /* The row-record has been opened successfully. Check that the
    ** column in question contains text or a blob. If it contains
    ** text, it is up to the caller to get the encoding right.
    */
    Incrblob *pBlob;
    u32 type = v->apCsr[0]->aType[iCol];

    if( type<12 ){
      rc = SQLITE_ERROR;
      goto blob_open_out;
    }
    pBlob = (Incrblob *)sqliteMalloc(sizeof(Incrblob));
    if( sqlite3MallocFailed() ){
      sqliteFree(pBlob);
      goto blob_open_out;
    }
    pBlob->flags = flags;
    pBlob->pCsr =  v->apCsr[0]->pCursor;
    pBlob->pStmt = (sqlite3_stmt *)v;
    pBlob->iOffset = v->apCsr[0]->aOffset[iCol];
    pBlob->nByte = sqlite3VdbeSerialTypeLen(type);
    *ppBlob = (sqlite3_blob *)pBlob;
    rc = SQLITE_OK;
  }else{
    if( rc==SQLITE_DONE ){
      rc = SQLITE_ERROR;
    }
  }

blob_open_out:
  if( rc!=SQLITE_OK || sqlite3MallocFailed() ){
    sqlite3_finalize((sqlite3_stmt *)v);
  }
  sqlite3Error(db, rc, "");
  return sqlite3ApiExit(db, rc);
}

/*
** Close a blob handle.
*/
int sqlite3_blob_close(sqlite3_blob *pBlob){
  Incrblob *p = (Incrblob *)pBlob;
  sqlite3_finalize(p->pStmt);
  sqliteFree(p);
  return SQLITE_OK;
}

/*
** Read data from a blob handle.
*/
int sqlite3_blob_read(sqlite3_blob *pBlob, void *z, int n, int iOffset){
  Incrblob *p = (Incrblob *)pBlob;
  if( (iOffset+n)>p->nByte ){
    return SQLITE_ERROR;
  }
  return sqlite3BtreeData(p->pCsr, iOffset+p->iOffset, n, z);
}

/*
** Write data to a blob handle.
*/
int sqlite3_blob_write(sqlite3_blob *pBlob, const void *z, int n, int iOffset){
  Incrblob *p = (Incrblob *)pBlob;
  if( (iOffset+n)>p->nByte ){
    return SQLITE_ERROR;
  }
  return sqlite3BtreePutData(p->pCsr, iOffset+p->iOffset, n, z);
}

/*
** Query a blob handle for the size of the data.
*/
int sqlite3_blob_bytes(sqlite3_blob *pBlob){
  Incrblob *p = (Incrblob *)pBlob;
  return p->nByte;
}

#endif /* #ifndef SQLITE_OMIT_INCRBLOB */
Added test/incrblob.test.










































































































>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
# 2007 May 1
#
# The author disclaims copyright to this source code.  In place of
# a legal notice, here is a blessing:
#
#    May you do good and not evil.
#    May you find forgiveness for yourself and forgive others.
#    May you share freely, never taking more than you give.
#
#***********************************************************************
#
# $Id: incrblob.test,v 1.1 2007/05/01 17:49:49 danielk1977 Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

do_test incrblob-1.1 {
  execsql {
    CREATE TABLE blobs(k PRIMARY KEY, v BLOB);
    INSERT INTO blobs VALUES('one', X'0102030405060708090A');
    INSERT INTO blobs VALUES('two', X'0A090807060504030201');
  }
} {}

do_test incrblob-1.2.1 {
  set ::blob [db incrblob blobs v 1]
} {incrblob_1}
do_test incrblob-1.2.2 {
  binary scan [read $::blob] c* data
  set data
} {1 2 3 4 5 6 7 8 9 10}
do_test incrblob-1.2.3 {
  seek $::blob 0
  puts -nonewline $::blob "1234567890"
  flush $::blob
} {}
do_test incrblob-1.2.4 {
  seek $::blob 0
  binary scan [read $::blob] c* data
  set data
} {49 50 51 52 53 54 55 56 57 48}
do_test incrblob-1.2.5 {
  close $::blob
} {}
do_test incrblob-1.2.6 {
  execsql {
    SELECT v FROM blobs WHERE rowid = 1;
  }
} {1234567890}

finish_test


Changes to test/tclsqlite.test.
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# This file implements regression tests for TCL interface to the
# SQLite library. 
#
# Actually, all tests are based on the TCL interface, so the main
# interface is pretty well tested.  This file contains some addition
# tests for fringe issues that the main test suite does not cover.
#
# $Id: tclsqlite.test,v 1.56 2006/09/01 15:49:06 drh Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

# Check the error messages generated by tclsqlite
#
if {[sqlite3 -has-codec]} {
  set r "sqlite_orig HANDLE FILENAME ?-key CODEC-KEY?"
} else {
  set r "sqlite3 HANDLE FILENAME ?MODE?"
}
do_test tcl-1.1 {
  set v [catch {sqlite3 bogus} msg]
  lappend v $msg
} [list 1 "wrong # args: should be \"$r\""]
do_test tcl-1.2 {
  set v [catch {db bogus} msg]
  lappend v $msg
} {1 {bad option "bogus": must be authorizer, busy, cache, changes, close, collate, collation_needed, commit_hook, complete, copy, enable_load_extension, errorcode, eval, exists, function, interrupt, last_insert_rowid, nullvalue, onecolumn, profile, progress, rekey, rollback_hook, timeout, total_changes, trace, transaction, update_hook, or version}}
do_test tcl-1.3 {
  execsql {CREATE TABLE t1(a int, b int)}
  execsql {INSERT INTO t1 VALUES(10,20)}
  set v [catch {
    db eval {SELECT * FROM t1} data {
      error "The error message"
    }







|


















|







11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
# This file implements regression tests for TCL interface to the
# SQLite library. 
#
# Actually, all tests are based on the TCL interface, so the main
# interface is pretty well tested.  This file contains some addition
# tests for fringe issues that the main test suite does not cover.
#
# $Id: tclsqlite.test,v 1.57 2007/05/01 17:49:49 danielk1977 Exp $

set testdir [file dirname $argv0]
source $testdir/tester.tcl

# Check the error messages generated by tclsqlite
#
if {[sqlite3 -has-codec]} {
  set r "sqlite_orig HANDLE FILENAME ?-key CODEC-KEY?"
} else {
  set r "sqlite3 HANDLE FILENAME ?MODE?"
}
do_test tcl-1.1 {
  set v [catch {sqlite3 bogus} msg]
  lappend v $msg
} [list 1 "wrong # args: should be \"$r\""]
do_test tcl-1.2 {
  set v [catch {db bogus} msg]
  lappend v $msg
} {1 {bad option "bogus": must be authorizer, busy, cache, changes, close, collate, collation_needed, commit_hook, complete, copy, enable_load_extension, errorcode, eval, exists, function, incrblob, interrupt, last_insert_rowid, nullvalue, onecolumn, profile, progress, rekey, rollback_hook, timeout, total_changes, trace, transaction, update_hook, or version}}
do_test tcl-1.3 {
  execsql {CREATE TABLE t1(a int, b int)}
  execsql {INSERT INTO t1 VALUES(10,20)}
  set v [catch {
    db eval {SELECT * FROM t1} data {
      error "The error message"
    }