/ View Ticket
Ticket UUID: 3211d778c0e445cc762a372dfc7d8acaf372cecf
Title: replace function works on blobs
Status: Closed Type: Feature_Request
Severity: Cosmetic Priority: Immediate
Subsystem: Unknown Resolution: Rejected
Last Modified: 2009-11-16 20:59:45
Version Found In: 3.4.2
http://www.sqlite.org/cvstrac/tktview?tn=2613 (plus other discussion)

The replace documentation states that it works on strings. However it also accepts blobs for any of the parameters and coerces the the result to a string.

It should really error when provided with blobs.

If blobs are going to be supported then it should be required that all parameters are either string or blobs (ie all 3 the same type) and the output type be the same as the parameters type. cast can be used on the parameters to clarify if this is a byte based or character based operation.

The code is fairly complicated already. Being able to mix and match bytes with characters is likely to lead to bugs related to encodings (eg different results depending on utf16 or or utf8) and the possibility of producing invalidly encoded result strings by replacing the wrong bytes.

drh added on 2009-11-16 12:36:30:
I think REPLACE() works fine just as it is.

rogerb added on 2009-11-16 19:18:34:
Are you absolutely sure there is no combination of parameter types and data that will not lead to invalid UTF8/16 and consequent buffer overruns?

It is also an application error to be mixing bytes/blobs with text.

drh added on 2009-11-16 20:59:45:
One is never 100% sure of anything with software of this complexity. However, we do make a point of testing invalid UTF8 and UTF16 byte sequences to make sure they do not overflow any buffers. You can view a few such tests at http://www.sqlite.org/src/artifact/8084dad398. There are many more in TH3. Running such tests with valgrind gives us confidence that the buffers never overflow.

At one point, we allowed read overruns of up to 2 bytes on invalid UTF, under the assumption that such overruns were harmless. But that assumption was proved wrong (see http://www.sqlite.org/src/info/3fe897352e) and such overruns have now been fixed.