Documentation Source Text

Check-in [469ec4c63f]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update zipfile docs to match the implementation.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 469ec4c63f4d6aa94f22d5319fd146b7fb07f4132f7cb31bd56885df8fa8175e
User & Date: dan 2018-01-10 17:06:50.493
Context
2018-01-10
19:26
Improved documentation for recently added features such as SQL Archive support and the Zipfile extension. (check-in: 260f2fd7ee user: drh tags: trunk)
17:06
Update zipfile docs to match the implementation. (check-in: 469ec4c63f user: dan tags: trunk)
2018-01-09
21:05
Add mention of the FTS5-initial-token-syntax in the change log. (check-in: 668e39014a user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/zipfile.in.
1
2
3
4
5
6












7
8



















9
10




11

12
13
14
15
16
17
18
<title>The SQLite Zipfile Module</title>
<tcl>hd_keywords {Zipfile virtual table} {zipfile}</tcl>
<table_of_contents>

<h1>Overview</h1>













<h1>Obtaining and Compiling Zipfile</h1>




















<h1>Using Zipfile</h1>





<h2>Reading</h2>


<p>For reading existing zip archives, the Zipfile module provides a
[table-valued function] that accepts a single argument - the path to the zip
archive to be read. For example, to inspect the contents of zip archive
"test.zip" from the current directory:

<codeblock>






>
>
>
>
>
>
>
>
>
>
>
>


>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>


>
>
>
>
|
>







1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
<title>The SQLite Zipfile Module</title>
<tcl>hd_keywords {Zipfile virtual table} {zipfile}</tcl>
<table_of_contents>

<h1>Overview</h1>

<p> The zipfile module provides read/write access to simple zip archives.
At time of writing, simple zip archives are roughly those that:

<ul>
  <li> Do not use encryption, 
  <li> Do not span multiple files, and
  <li> Do not use zip64 extensions.
</ul>

<p> Some or all of these restrictions may be removed at some point
in the future.

<h1>Obtaining and Compiling Zipfile</h1>

<p>The code for the zipfile module is found in the ext/misc/zipfile.c file of
the main SQLite source tree. It may be compiled into an SQLite 
[loadable extension] using a command like:

<codeblock>
    gcc -g -fPIC -shared zipfile.c -o zipfile.so
</codeblock>

<p>Alternatively, the zipfile.c file may be compiled into the application. 
In this case, the following function should be invoked to register the
extension with each new database connection:

<codeblock>
  int sqlite3_zipfile_init(sqlite3 *db, void*, void*);
</codeblock>

<p> The first argument passed should be the database handle to register the
extension with. The second and third arguments should both be passed 0.

<h1>Using Zipfile</h1>

<p>The zipfile module provides two similar interfaces for accessing zip
archives. A table-valued function, which provides read-only access to
existing archives, and a virtual table interface, which provides both
read and write access.

<h2>Table-Valued Function (read-only access)</h2>

<p>For reading existing zip archives, the Zipfile module provides a
[table-valued function] that accepts a single argument - the path to the zip
archive to be read. For example, to inspect the contents of zip archive
"test.zip" from the current directory:

<codeblock>
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61

62
63



64
65
66
67

68
69



70
71
72
73

74












75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92

93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
109
110
111


112

113




114
115
116
117
118
119


120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138





























                  this column contains a NULL value.
<tr><td>method <td> The compression method used to compress the data (an
                    integer). The value 0 indicates that the data is stored
                    in the zip archive without compression. 8 means the
                    raw deflate algorithm. 
</table>

<h2>Writing</h2>

<p>In order to create or modify an existing zip file, a "zipfile" virtual 
table must be created in the database schema. The CREATE VIRTUAL TABLE
statement expects a path to the zip file as its only argument. For example, to
write to zip file "test.zip" in the current directory, a zipfile table may be
created using:

<codeblock>
  CREATE VIRTUAL TABLE temp.zip USING zipfile('test.zip');
</codeblock>

<p>Such a virtual table has the same columns as the table-valued function
described in the previous section. Records may be removed from an existing

zip archive by deleting the corresponding rows. For example, to remove file
"m.txt" from zip archive "test.zip" using the virtual table created above:




<codeblock>
  DELETE FROM temp.zip WHERE name = 'm.txt';
</codeblock>


<p>Similarly, entries may be added to a zip archive by inserting new rows.



For example, to add file "m.txt" containing the text "abcdefghi" to zip
archive "test.zip":

<codeblock>

  INSERT INTO temp.zip(name, data) VALUES('m.txt', 'abcdefghi');












</codeblock>

<p>The following rules and caveats apply to the values specified as part of
each INSERT statement:

<table striped=1>
<tr><th>Columns <th> Notes
<tr><td> name
  <td> A non-NULL text value must be specified for the name column. 
       Usually, it is an error if the specified name already exists in the
       archive. Or, if the insert statement is actually a "REPLACE" or "INSERT
       OR REPLACE" statement, then the existing archive member is removed
       before adding the new entry.

<tr><td> mode
  <td> If NULL is inserted into the mode column, then the mode of the
       new archive entry is automatically set to 33188 (-rw-r--r--). This
       is a regular non-executable file writable by its owner and readable

       by all others. <br><br>
       
       If the specified value is an integer (or text that looks like
       an integer), it is inserted verbatim. If the value is not a valid UNIX
       mode, some programs may behave unexpectedly when extracting files
       from the archive.<br><br>

       Finally, if the value specified for this column is not an integer
       or a NULL, then it is assumed to be a UNIX permissions string similar
       to those output by the "ls -l" command (e.g. "-rw-r--r--", "drwxr-xr-x"
       etc.). In this case, if the string cannot be parsed it is an error.

<tr><td> mtime
<td>   If NULL is inserted into the mtime column, then the timestamp
       of the new entry is set to the current time. Otherwise, the specified
       value is interpreted as an integer and used as is.

<tr><td> sz, rawdata, data, method
  <td> The data associated with the new entry is specified by providing 


       values for the final four columns of the zipfile virtual 

       table - sz, rawdata, data and method.  Only the following combinations




       are valid:<br><br>

       <b>sz=NULL, rawdata=NULL, data&ne;NULL, method=NULL</b>:
       In this case the value specified for the "data" column is stored as the
       associated data in the archive. Zipfile automatically compresses it
       using the deflate algorithm (method=8) if doing so would be advantageous.


       <br><br>

       <b>sz=NULL, rawdata=NULL, data&ne;NULL, method IN (0,8)</b>: 
       In this case the value supplied for the "method" column must
       be either 0 or 8. In either case the value specified for the "data"
       column is stored as the associated data, compressed if 8 was specified
       for the "method" column, or as is otherwise.
       <br><br>

       <b>sz&ne;NULL, rawdata&ne;NULL, data=NULL, method&ne;NULL</b>: 
       In this case the value supplied for column "rawdata" must be
       already compressed according to the integer method supplied for "method"
       (which can take any unsigned 16-bit value, not just 0 or 8). The size of
       the uncompressed data (an integer) must be specified for column "sz".
</table>

<p> UPDATE statements are not supported by zipfile virtual tables. Nor is
    specifying an explicit value for the rowid field as part of an INSERT
    statement.




































|












|
>
|
|
>
>
>

<
<
<
>

|
>
>
>
|



>
|
>
>
>
>
>
>
>
>
>
>
>
>









|
|
<
<



|
|
>
|

















|
>
>
|
>
|
>
>
>
>
|





>
>
















<
|
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104



105
106
107
108
109
110
111
112
113
114
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139


140
141
142
143
144
145
146
147
148
149
150
151
152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197

198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
213
214
215
216
217
218
219
220
221
222
223
224
225
226
227
228
                  this column contains a NULL value.
<tr><td>method <td> The compression method used to compress the data (an
                    integer). The value 0 indicates that the data is stored
                    in the zip archive without compression. 8 means the
                    raw deflate algorithm. 
</table>

<h2>Virtual Table Interface (read/write access)</h2>

<p>In order to create or modify an existing zip file, a "zipfile" virtual 
table must be created in the database schema. The CREATE VIRTUAL TABLE
statement expects a path to the zip file as its only argument. For example, to
write to zip file "test.zip" in the current directory, a zipfile table may be
created using:

<codeblock>
  CREATE VIRTUAL TABLE temp.zip USING zipfile('test.zip');
</codeblock>

<p>Such a virtual table has the same columns as the table-valued function
described in the previous section. It may be read from using a SELECT 
statement in the same way as the table-valued function can. 

<p>Using the virtual table interface, new entries may be added to a zip
archive by inserting new rows into the virtual table. Entries may be
removed by deleting rows. UPDATE statements are not supported by zipfile
virtual tables. 




<h3 tags="Adding to Zip">Adding Entries to a Zip Archive</h3>

<p>Entries may be added to a zip archive by inserting new rows. The easiest
way to do this is to specify values for the "name" and "data" columns only and
have zipfile fill in sensible defaults for other fields.  To insert a directory
into the archive, set the "data" column to NULL.  For example, to add the
directory "dir1" and the file "m.txt" containing the text "abcdefghi" to zip
archive "test.zip":

<codeblock>
  INSERT INTO temp.zip(name, data) VALUES('dir1', NULL);           <i>-- Add directory </i>
  INSERT INTO temp.zip(name, data) VALUES('m.txt', 'abcdefghi');   <i>-- Add regular file </i>
</codeblock>

<p>When a directory is inserted, if the "name" value does not end with
a '/' character, the zipfile module appends one. This is necessary for
compatibility with other programs (most notably "info-zip") that 
manipulate zip archives.

<p>To insert a symbolic link, the user must also supply a "mode" value.
For example, to add a symbolic link from "link.txt" to "m.txt":

<codeblock>
  INSERT INTO temp.zip(name, mode, data) VALUES('link.txt', 'lrwx-rw-rw', 'abcdefghi');
</codeblock>

<p>The following rules and caveats apply to the values specified as part of
each INSERT statement:

<table striped=1>
<tr><th>Columns <th> Notes
<tr><td> name
  <td> A non-NULL text value must be specified for the name column. 
       It is an error if the specified name already exists in the
       archive. 



<tr><td> mode
  <td> If NULL is inserted into the mode column, then the mode of the
       new archive entry is automatically set to either 33188 (-rw-r--r--)
       or 16877 (drwxr-xr-x), depending on whether or not the values 
       specified for columns "sz", "data" and "rawdata" indicate that
       the new entry is a directory.<br><br>
       
       If the specified value is an integer (or text that looks like
       an integer), it is inserted verbatim. If the value is not a valid UNIX
       mode, some programs may behave unexpectedly when extracting files
       from the archive.<br><br>

       Finally, if the value specified for this column is not an integer
       or a NULL, then it is assumed to be a UNIX permissions string similar
       to those output by the "ls -l" command (e.g. "-rw-r--r--", "drwxr-xr-x"
       etc.). In this case, if the string cannot be parsed it is an error.

<tr><td> mtime
<td>   If NULL is inserted into the mtime column, then the timestamp
       of the new entry is set to the current time. Otherwise, the specified
       value is interpreted as an integer and used as is.

<tr><td> sz, rawdata, data, method
<td>
       To insert a directory into the archive, all four of these fields
       must be set to NULL. In this case if a value was explicitly specified
       for the "mode" column, then it must be consistent with a directory 
       (i.e. it must be true that (mode & 0040000)=0040000). <br><br>

       Otherwise, these fields are used to specify the contents of a
       regular file or the target of a symbolic link. There are three
       different ways to do this, depending on whether compressed or
       uncompressed data is being inserted, and whether or not zipfile should
       automatically choose a compression algorithm and compress the data:<br><br>

       <b>sz=NULL, rawdata=NULL, data&ne;NULL, method=NULL</b>:
       In this case the value specified for the "data" column is stored as the
       associated data in the archive. Zipfile automatically compresses it
       using the deflate algorithm (method=8) if doing so would be advantageous.
       Otherwise, if the data is not compressible, it is stored verbatim in
       the archive (method=0).
       <br><br>

       <b>sz=NULL, rawdata=NULL, data&ne;NULL, method IN (0,8)</b>: 
       In this case the value supplied for the "method" column must
       be either 0 or 8. In either case the value specified for the "data"
       column is stored as the associated data, compressed if 8 was specified
       for the "method" column, or as is otherwise.
       <br><br>

       <b>sz&ne;NULL, rawdata&ne;NULL, data=NULL, method&ne;NULL</b>: 
       In this case the value supplied for column "rawdata" must be
       already compressed according to the integer method supplied for "method"
       (which can take any unsigned 16-bit value, not just 0 or 8). The size of
       the uncompressed data (an integer) must be specified for column "sz".
</table>


<p> Specifying an explicit value for the rowid field as part of an INSERT
statement is not supported. Any value supplied is ignored.

<h3> Deleting Zip Archive Entries </h3>

<p>Records may be removed from an existing zip archive by deleting the
corresponding rows. For example, to remove file "m.txt" from zip archive
"test.zip" using the virtual table created above:

<codeblock>
  DELETE FROM temp.zip WHERE name = 'm.txt';
</codeblock>

<p>Note that deleting records from a zip archive does not reclaim the 
space used within the archive - it merely removes an entry from the
archives "Central Directory Structure", making the entry inaccessible.
One way to work around this inefficiency is to create a new zip 
archive based on the contents of the edited archive. For example, after
editing the archive accessed via virtual table temp.zzz:

<codeblock>
  <i>-- Create a new, empty, archive: </i>
  CREATE VIRTUAL TABLE temp.newzip USING zipfile('new.zip');

  <i>-- Copy the contents of the existing archive into the new archive</i>
  INSERT INTO temp.newzip(name, mode, mtime, sz, rawdata, method) 
      SELECT name, mode, mtime, sz, rawdata, method FROM temp.zzz;
</codeblock>