Documentation Source Text

Check-in [8d02ad5577]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Typos and clarifications to the pointer-passing document.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 8d02ad5577e4530e0cfe2961753f03d1483c1c376c0a2324ab2c4f213b855ace
User & Date: drh 2017-07-24 11:34:02.203
Context
2017-07-24
12:00
Restore text mistakenly deleted from the change log. (check-in: f9a896f5c0 user: drh tags: trunk)
11:34
Typos and clarifications to the pointer-passing document. (check-in: 8d02ad5577 user: drh tags: trunk)
2017-07-23
20:57
Improved documentation on the pointer-passing interfaces. (check-in: 9bf45f99e1 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/bindptr.in.
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32

33
34
35
36
37
38
39
<title>Pointer Passing Interfaces</title>
<tcl>hd_keywords {pointer passing interfaces}
</tcl>

<fancy_format>

<h1>Overview</h1>

<p>
Three new "_pointer()" interfaces were added on to SQLite on
[https://sqlite.org/src/timeline?c=72de49f204277|2017-07-13]:
<ul>
<li> [sqlite3_bind_pointer()],
<li> [sqlite3_result_pointer()], and
<li> [sqlite3_value_pointer()].
</ul>

<p>Questions and confusion quickly arose
on the [mailing lists] about the purpose behind these new interfaces,
why they were introduced, and what problem they solve.  This essay attempts
to answer those questions and clear up the confusion.

<h1>A Brief History Of Pointer Passing In SQLite</h1>

<p>
It is sometimes convenient for SQLite extensions to
communicatin non-SQL values between subcomponents or between the extension
and the application.  Some examples:

<ul>
<li><p>
In the [FTS3] extension, the MATCH operator (that does the full-text search)

needs to communicate details of matching entrys to the [snippet()], [offsets()],
and [matchinfo()] functions so that those functions can convert the details
of the match into useful output.

<li><p>
In order for an application to 
[Extending FTS5|add new extensions to FTS5], such as new tokenizers, 









|
<




















|
>







1
2
3
4
5
6
7
8
9
10

11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
<title>Pointer Passing Interfaces</title>
<tcl>hd_keywords {pointer passing interfaces}
</tcl>

<fancy_format>

<h1>Overview</h1>

<p>
Three new "_pointer()" interfaces were added to SQLite 3.20.0 ([dateof:3.20.0]):

<ul>
<li> [sqlite3_bind_pointer()],
<li> [sqlite3_result_pointer()], and
<li> [sqlite3_value_pointer()].
</ul>

<p>Questions and confusion quickly arose
on the [mailing lists] about the purpose behind these new interfaces,
why they were introduced, and what problem they solve.  This essay attempts
to answer those questions and clear up the confusion.

<h1>A Brief History Of Pointer Passing In SQLite</h1>

<p>
It is sometimes convenient for SQLite extensions to
communicatin non-SQL values between subcomponents or between the extension
and the application.  Some examples:

<ul>
<li><p>
In the [FTS3] extension, the 
[FTS MATCH|MATCH operator] (which does the full-text search)
needs to communicate details of matching entrys to the [snippet()], [offsets()],
and [matchinfo()] functions so that those functions can convert the details
of the match into useful output.

<li><p>
In order for an application to 
[Extending FTS5|add new extensions to FTS5], such as new tokenizers, 
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
SELECT snippet(x'6092310100000000') FROM t1 WHERE cx MATCH $pattern;
</codeblock>

<p>
Historically, this was not considered a threat.  The argument was that if
a hostile agent is able to inject arbitrary SQL text into the application,
then that agent is already in full control of the application, so
letting the hostile agent forge a pointer is not giving the agent
any capability he does not already have.

<p>
For most cases, it is true that potential attackers have no way of injecting
arbitrary SQL, and so most uses of SQLite are immute to the attack above.
But there are some notable exceptions.  To wit:

<ul>







|
|







94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
SELECT snippet(x'6092310100000000') FROM t1 WHERE cx MATCH $pattern;
</codeblock>

<p>
Historically, this was not considered a threat.  The argument was that if
a hostile agent is able to inject arbitrary SQL text into the application,
then that agent is already in full control of the application, so
letting the hostile agent forge a pointer does not give the agent
any new capability.

<p>
For most cases, it is true that potential attackers have no way of injecting
arbitrary SQL, and so most uses of SQLite are immute to the attack above.
But there are some notable exceptions.  To wit:

<ul>
215
216
217
218
219
220
221
222
223
224
225
226
227
228
229
230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
252
253
SQL statement that tries to use the [hex()] function to read the
value of a pointer will get an SQL NULL answer.  The only way to
discover whether or not a value has an associated pointer is to
use the [sqlite3_value_pointer()] interface with the appropriate
type string T. 

<p>
Pointer values generated by [sqlite3_bind_pointer()] and
[sqlite3_result_pointer()] cannot be generated by pure SQL, only
by C-code extensions.  And so it is not possible for SQL to forge
a pointers.

<p>
Pointer values generated [sqlite3_bind_pointer()] and
[sqlite3_result_pointer()] cannot be read by pure SQL, only
by C-code extensions.  And so it is not possible for SQL to leak
the value of pointers.

<p>
In this way the new pointer-passing interface seems to solve all of the
security problems associated with passing pointer values from one
extension to another in SQLite.

<h2>Pointer Types</h2>

<p>
The "pointer type" which is the last parameter to [sqlite3_bind_pointer()],
[sqlite3_result_pointer()], and [sqlite3_value_pointer()] is used to prevent
pointers intended for one extension from being redirected to a different
extension.  For example, without the use of pointer types, an attacker 
could still get access to pointer information in a system that included 
by the [FTS3] and the [carray|CARRAY extension] using SQL like this:

<codeblock>
SELECT ca.value FROM t1, carray(t1,10) AS ca WHERE cx MATCH $pattern
</codeblock>

<p>
In the statement above, the FTS3 cursor pointer generated by the







|
<
|



|
|
|
<









|




|







215
216
217
218
219
220
221
222

223
224
225
226
227
228
229

230
231
232
233
234
235
236
237
238
239
240
241
242
243
244
245
246
247
248
249
250
251
SQL statement that tries to use the [hex()] function to read the
value of a pointer will get an SQL NULL answer.  The only way to
discover whether or not a value has an associated pointer is to
use the [sqlite3_value_pointer()] interface with the appropriate
type string T. 

<p>
Pointer values read by [sqlite3_value_pointer()]

cannot be generated by pure SQL.  Hence, it is not possible for SQL to forge
a pointers.

<p>
Pointer values generated by [sqlite3_bind_pointer()] and
[sqlite3_result_pointer()] cannot be read by pure SQL.
Hence, it is not possible for SQL to leak the value of pointers.


<p>
In this way the new pointer-passing interface seems to solve all of the
security problems associated with passing pointer values from one
extension to another in SQLite.

<h2>Pointer Types</h2>

<p>
The "pointer type" in the last parameter to [sqlite3_bind_pointer()],
[sqlite3_result_pointer()], and [sqlite3_value_pointer()] is used to prevent
pointers intended for one extension from being redirected to a different
extension.  For example, without the use of pointer types, an attacker 
could still get access to pointer information in a system that included 
both the [FTS3] and the [carray|CARRAY extension] using SQL like this:

<codeblock>
SELECT ca.value FROM t1, carray(t1,10) AS ca WHERE cx MATCH $pattern
</codeblock>

<p>
In the statement above, the FTS3 cursor pointer generated by the
278
279
280
281
282
283
284
285
286
287
288
289
290
291
292
293
<p>
Because pointer types must be static strings, and because string values
in SQLite are dynamic strings, that means that SQL values cannot be used
as a pointer type.  This prevents misguided developers from creating a
new SQL function that can manufacture pointer values directly from SQL.  
Such a function, if possible to write, would undermine the security of
the pointer-passing APIs.  Thus, the requirement that pointer types be 
static strings is an effective deterrent against writing such 
security-defeating extension functions.

<h1>Restrictions On The Use of Pointer Values</h1>

<p>
The pointers that piggy-back on SQL NULL values using the
[sqlite3_bind_pointer()], [sqlite3_result_pointer()], and
[sqlite3_value_pointer()] interface are transient and ephemeral.







|
<







276
277
278
279
280
281
282
283

284
285
286
287
288
289
290
<p>
Because pointer types must be static strings, and because string values
in SQLite are dynamic strings, that means that SQL values cannot be used
as a pointer type.  This prevents misguided developers from creating a
new SQL function that can manufacture pointer values directly from SQL.  
Such a function, if possible to write, would undermine the security of
the pointer-passing APIs.  Thus, the requirement that pointer types be 
static strings helps to prevent misuse of the pointer-passing interfaces.


<h1>Restrictions On The Use of Pointer Values</h1>

<p>
The pointers that piggy-back on SQL NULL values using the
[sqlite3_bind_pointer()], [sqlite3_result_pointer()], and
[sqlite3_value_pointer()] interface are transient and ephemeral.
308
309
310
311
312
313
314
315
316

317
318
319
320
321
322
323

<h1>Summary</h1>

<p>Key take-aways from this essay:

<ol>
<li><p>
The internet is an increasingly hostile place.  One should now assume that
attackers will find a way to execute arbitrary SQL in an application.

Applications should be designed to prevent the execution of arbitrary
SQL from escalating into a more severe exploit.

<li><p>
A few SQLite extensions benefit from passing pointers:
<ul>
<li>The [FTS3] MATCH operator passes pointers into [snippet()],







|
|
>







305
306
307
308
309
310
311
312
313
314
315
316
317
318
319
320
321

<h1>Summary</h1>

<p>Key take-aways from this essay:

<ol>
<li><p>
The internet is an increasingly hostile place.  These day, developers
should assume that attackers will find a way to execute arbitrary SQL 
in an application.
Applications should be designed to prevent the execution of arbitrary
SQL from escalating into a more severe exploit.

<li><p>
A few SQLite extensions benefit from passing pointers:
<ul>
<li>The [FTS3] MATCH operator passes pointers into [snippet()],
338
339
340
341
342
343
344
345
346
347
348
349
350
351
352
[sqlite3_bind_pointer()], [sqlite3_result_pointer()], and
[sqlite3_value_pointer()].

<li><p>
The use of pointer-passing is an advanced technique that should be used
infrequently and cautiously.  Pointer-passing should not be
used haphazardly or carelessly.  Pointer-passing is a sharp tool 
that leave deep scars if misused.

<li><p>
The "pointer type" string which is the last parameter to each of the
pointer-passing interfaces should be a distinct, application-specific
string literal that appears directly in the API call.  The pointer type
should not be a parameter passed in from a higher-level function.
</ol>







|







336
337
338
339
340
341
342
343
344
345
346
347
348
349
350
[sqlite3_bind_pointer()], [sqlite3_result_pointer()], and
[sqlite3_value_pointer()].

<li><p>
The use of pointer-passing is an advanced technique that should be used
infrequently and cautiously.  Pointer-passing should not be
used haphazardly or carelessly.  Pointer-passing is a sharp tool 
that can leave deep scars if misused.

<li><p>
The "pointer type" string which is the last parameter to each of the
pointer-passing interfaces should be a distinct, application-specific
string literal that appears directly in the API call.  The pointer type
should not be a parameter passed in from a higher-level function.
</ol>