View Ticket
Not logged in
Ticket UUID: c25aab7e7ea55c861313076cae9257fc8f3fbdc0
Title: The ".import" command does not parse CSV correctly.
Status: Open Type: Code_Defect
Severity: Important Priority: Low
Subsystem: Shell Resolution: Open
Last Modified: 2010-01-06 15:37:44
Detected By: Application_Fault
Version Found In: 3.6.17
Description & Comments:
when importing CSV data, in text fields separated by double quotes, commas are treated as field separator.

live example:

770,"011",2002,2001247,3,4,"6/SOCJ,",1,0,20020131,122602,20020131,1,"FBLI"

in 7th field, which is "6/SOCJ," the comma inside the quotes is mis-interpreted as field separator, and application submits error:

FGLEDX line 136090: expected 14 columns of data but found 15

In case you need more information, please contact me at rafal@pies.pl.

Best regards,

Rafal Stanilewicz


drh added on 2009-08-13 15:19:32:
The ".import" function is in the command-line shell, which is a separate program that is independent of the SQLite core. I have retagged this ticket to indicate as much.

There is no such thing as a "CSV" standard. Different programs do different things. There might be users that depend on the current behavior. If we fix this for the OP, it could cause problems for others. So it is unclear whether or not we should do anything for this. We will simply leave the ticket open for the time being.


rogerb added on 2009-10-06 18:10:35:
See also http://www.sqlite.org/cvstrac/tktview?tn=3812 where the shell isn't correctly importing the same data it exported.

It would be reasonable to expect that whatever SQLite exports would import as exactly the same data.


anonymous claiming to be PM added on 2009-10-14 11:15:47:
See also old ticket http://www.sqlite.org/cvstrac/tktview?tn=3276. We still suffer from the inconsistency between the .import and export of the csv files. Although csv is not a real standard, at least you can expect consistent behaviour in sqlite (shell) itself.


rogerb added on 2009-10-27 20:00:06:
A request to tweak some of the behaviour of .import such as continuing on errors

http://www.sqlite.org/cvstrac/tktview?tn=1506


rogerb added on 2009-10-27 21:17:53:
Header rows with .import http://www.sqlite.org/cvstrac/tktview?tn=1313


anonymous added on 2010-01-06 15:37:44:
There is a well-defined standard for CSV files of mime type text/csv. It can be found at:

http://tools.ietf.org/html/rfc4180