Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | First cut at a separate document for the printf() string formatters. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
664ed2dce459beca21a5ce245f41a1bf |
User & Date: | drh 2018-02-20 02:53:36.176 |
Context
2018-02-20
| ||
13:46 | Further refinement of the new printf.html document. (check-in: 6c1f37df7f user: drh tags: trunk) | |
02:53 | First cut at a separate document for the printf() string formatters. (check-in: 664ed2dce4 user: drh tags: trunk) | |
2018-02-19
| ||
10:46 | Add a list of shadow tables to the fts5 documentation. (check-in: 1225bcc93d user: dan tags: trunk) | |
Changes
Changes to pages/crew.in.
1 2 3 4 5 6 7 8 | <title>SQLite Developers</title> <h2>The SQLite Development Team</h2> <img src="images/drh1.jpg" align="left" hspace="25" vspace="0"> <p> <b>D. Richard Hipp </b> began the SQLite project on 2000-05-29 | > > > | 1 2 3 4 5 6 7 8 9 10 11 | <title>SQLite Developers</title> <tcl>hd_keywords {crew} {Hipp} {Kennedy}</tcl> <fancy_format> <h2>The SQLite Development Team</h2> <img src="images/drh1.jpg" align="left" hspace="25" vspace="0"> <p> <b>D. Richard Hipp </b> began the SQLite project on 2000-05-29 |
︙ | ︙ |
Added pages/printf.in.
> > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > | 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 79 80 81 82 83 84 85 86 87 88 89 90 91 92 93 94 95 96 97 98 99 100 101 102 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 118 119 120 121 122 123 124 125 126 127 128 129 130 131 132 133 134 135 136 137 138 139 140 141 142 143 144 145 146 147 148 149 150 151 152 153 154 155 156 157 158 159 160 161 162 163 164 165 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 182 183 184 185 186 187 188 189 190 191 192 193 194 195 196 197 198 199 200 201 202 203 204 205 206 207 208 209 210 211 212 213 214 215 216 217 218 219 220 221 222 223 224 225 226 227 228 229 230 231 232 233 234 235 236 237 238 239 240 241 242 243 244 245 246 247 248 249 250 251 252 253 254 255 256 257 258 259 260 261 262 263 264 265 266 267 268 269 270 271 272 273 274 275 276 277 278 279 280 281 282 283 284 285 286 287 288 289 290 291 292 293 294 295 296 297 298 299 300 301 302 303 304 305 306 307 308 309 310 311 312 313 314 315 316 317 318 319 320 321 322 323 324 325 326 327 328 329 330 331 332 333 334 335 336 337 338 339 340 341 342 343 344 345 346 347 348 349 350 351 352 353 354 355 356 357 358 359 360 361 362 363 364 365 366 367 368 369 370 | <title>SQLite's Built-in printf()</title> <tcl>hd_keywords {string-formatter}</tcl> <table_of_contents> <h1>Overview</h1> <p>SQLite contains its own implementation of the string formatting routine "printf()", accessible via the following interfaces: <ul> <li> [printf()] → an SQL function returning the formatted string <li> [sqlite3_mprintf()] → Store the formatted string in memory obtained [sqlite3_malloc64()]. <li> [sqlite3_snprintf()] → Store the formatted string in a static buffer <li> [sqlite3_vmprintf()] → Varargs version of sqlite3_mprintf() <li> [sqlite3_vsnprintf()] → Varargs version of sqlite3_snprintf() </ul> <p>The same core string formatter is also used internally by SQLite. <h2>Advantages</h2> <p>Why does SQLite have its own private built-in printf() implementation? Why not use the printf() implementation from the standard C library? <p>Several reasons: <p> <ol> <li><p> By using its own built-in implementation, SQLite guarantees that the output will be the same on all platforms and in all LOCALEs. This is important for consistency and for testing. It would be problematic if one machine gave and answer of "5.25e+08" and another gave an answer of "5.250e+008". But answers are correct, but we prefer that SQLite also give the same answer. <li><p> We know of no way to use the standard library printf() C interface to implement the [printf() SQL function] feature of SQLite. The built-in printf() C implementation can be easily adapted, however. <li><p> The printf() built into SQLite supports new non-standard substitution types (%q, %Q, %w, and %z) that are useful both internally to SQLite and to applications using SQLite. Standard library printf()s cannot normally be extended in this way. <li><p> The built-in SQLite implementation supports the ability to render an arbitrary-length string into a memory buffer obtained from [sqlite3_malloc64()]. This is safer and less error prone than trying to precompute an upper size limit on the result string, allocate an appropriately sized buffer, and then calling snprintf(). <li><p> The SQLite-specific printf() supports a new flag (!) called the "alternate-form-2" flag. The alternate-form-2 flag changes the processing of floating-point conversions in subtle ways so that the output is always an SQL-compatible text representation of a floating-point number - something that is not possible to achieve with standard-library printf(). For string substitutions, the alternate-form-2 flag causes the width and precision to be measured in characters instead of bytes, which simplifies processing of strings containing multi-byte UTF8 characters. <li><p> The built-in SQLite has compile-time options such as SQLITE_PRINTF_PRECISION_LIMIT that provide defense against denial-of-service attacks for application that expose the printf() functionality to untrusted users. <li><p> Using a built-in printf() implementation means that SQLite has one fewer dependency on the host environment, making it more portable. </ol> <h2>Disadvantages</h2> <p> In fairness, having a built-in implementation of printf() also comes with some disadvantages. To wit: <ol> <li><p> The built-in printf() implementation uses extra code space (about 7800 bytes). <li><p> The floating-point to text conversion subfunction for the built-in printf() is limited in precision to 16 significant digits (or 26 significant digits if the "!" alternate-form-2 flag is used). Every IEEE-754 double can be represented exactly as a decimal floating-point value, but some doubles require more than 16 or 26 significant digits. <li><p> The order of the buffer pointer and buffer size parameters in the built-in snprintf() implementation is reversed from the order used in standard-library implementations. </ol> <p> In spite of the disadvantages, the developers believe that having a built-in printf() implementation inside of SQLite is a net positive. <h1>Formatting Details</h1> <p>The format string for printf() is a template for the generated string. Substitutions are made whenever a "%" character appears in the format string. The "%" is followed by one or more additional characters that describe the substitution. Each substitution has the following format: <blockquote> <b>%</b><i>[flags][width][</i><b>.</b><i>precision][length]type</i> </blockquote> <p>All substitutions begin with a single "%" and end with a single type character. The other elements of the substitution are optional. <p>To include a single "%" character in the output, put two consecutive "%" characters in the template. <h2>Substitution Types</h2> <p>The following chart shows the types supported by SQLite: <center> <table border=1 cellpadding="10" width="80%"> <tr> <th>Substitution Type<th>Meaning <tr> <td>% <td>Two "%" characters in a row are translated into a single "%" in the output, without substiting any values. <tr> <td>d, i <td>The argument is a signed integer which is displayed in decimal. <tr> <td>u <td>The argument is an unsigned integer which is displayed in decimal. <tr> <td>f <td>The argument is a double which is displayed in decimal. <tr> <td>e, E <td>The argument is a double which is displayed in exponential notation. The exponent character is 'e' or 'E' depending on the type. <tr> <td>g, G <td>The argument is a double which is displayed in either normal decimal notation or if the exponent is not close to zero, in exponential notation. <tr> <td>x, X <td>The argument is an integer which is displayed in hexadecimal. Lower-case hexadecimal is used for %x and and upper-case is used for %X <tr> <td>o <td>The argument is an integer which is displayed in octal. <tr> <td>s, z <td>The argument is a zero-terminated string that is displayed. For the %z type in C-language interfaces, [sqlite3_free()] is invoked on the string after it has be copied into the output. The %s and %z substitutions are identical for the SQL printf() function.<br><br> The %s substitution is universal, but the %z substitution is an SQLite enhancement, not found in other printf() implementations. <tr> <td>c <td>For the C-language interfaces, the argument is an integer which is interpreted as a character. For the [printf() SQL function] the argument is a string from which the first character is extracted and displayed. <tr> <td>p <td>The argument is a pointer which is displayed as a hexadecimal address. Since the SQL language has no concept of a pointer, the %p substitution for the [printf() SQL function] works like %x. <tr> <td>n <td>The argument is a pointer to an integer. Nothing is displayed for this substitution type. Instead, the integer to which the argument points is overwritten with the number of characters in the generated string that result from all format symbols to the left of the %n. <tr> <td>q, Q <td>The argument is a zero-terminated string. The string is printed with all single quote (') characters doubled so that the string can safely appear inside an SQL string literal. The %Q substitution type also puts single-quotes on both ends of the substituted string. <br><br>If the argument to %Q is a null pointer then the output is an unquoted "NULL". In other words, a null pointer generates an SQL NULL, and a non-null pointer generates a valid SQL string literal. If the argument to %q is a null pointer then no output is generated. Thus a null-pointer to %q is the same as an empty string. <br><br>For these substitutions, the precision is the number of bytes or characters taken from the argument, not the number of bytes or characters that are written into the output. <br><br> The %q and %Q substitutions are SQLite enhancements, not found in most other printf() implementations. <tr> <td>w <td>This substitution works like %q except that it doubles all double-quote characters (") instead of single-quotes, making the result suitable for using with a double-quoted identifier name in an SQL statement. <br><br> The %w substitution is an SQLite enhancements, not found in most other printf() implementations. </table> </center> <h2>The Optional Length Field</h2> <p>The length of the argument value can be specified by one or more letters that occur just prior to the substitution type letter. In SQLite, the length only matter for integer types. The length is ignored for the [printf() SQL function] which always uses 64-bit values. The following table shows the length specifiers allowed by SQLite: <center> <table border=1 cellpadding="10" width="80%"> <tr> <th>Length Specifier <th>Meaning <tr> <td><i>(default)</i> <td>An "int" or "unsigned int". 32-bits on all modern systems. <tr> <td>l <td>A "long int" or "long unsigned int". Also 32-bits on all modern systems. <tr> <td>ll <td>A "long long int" or "long long unsigned" or an "sqlite3_int64" or "sqlite3_uint64" value. These are 64-bit integers on all modern systems. </table> </center> <p>Only the "ll" length modifier ever makes a difference for SQLite. And it only makes a difference when using the C-language interfaces. <h2>The Optional Width Field</h2> <p>The width field specifies the minimum width of the substituted value in the output. If the string or number that is written into the output is shorter than the width, then the value is padded. Padding is on the left (the value is right-justified) by default. If the "-" flag is used, then the padding is on the right and the value is left-justified. <p>The width is measured in bytes by default. However, if the "!" flag is present then the width is in characters. This only makes a difference for multi-byte utf-8 characters, and those only occur on string substitutions. <p>If the width is a single "*" character instead of a number, then the actual width value is read as an integer from the argument list. If the value read is negative, then the absolute value is used for the width and the value is left-justified as if the "-" flag were present. <p>If the value being substituted is larger than the width, then full value is added to the output. In other words, the width is the minimum width of the value as it is rendered in the output. <h2>The Optional Precision Field</h2> <p>The precision field, if it is present, must follow the width separated by a single "." character. If there is no width, then the "." that introduces the precision immediately follows either the flags (if there are any) or the initial "%". <p>For string substitutions (%s, %z, %q, %Q, or %w) the precision is the number of byte or character used from the argument. The number is bytes by default but is characters if the "!" flag is present. If there is no precision, then the entire string is substituted. Examples: "%.3s" substitutes the first 3 bytes of the argument string. "%!.3s" substitutes the first three characters of the argument string. <p>For integer substitutions (%d, %i, %x, %X, %o, and %p) the precision specifies minimum number of digits to display. Leading zeros are added if necessary, to expand the output to the minimum number of digits. <p>For floating-point substitutions (%e, %E, %f, %g, %G) the precision specifies the number of digits to display to the right of the decimal point. <p>For the character substitution (%c) a precision N greater than 1 causes the character to be repeated N times. This is a non-standard extension found only in SQLite. <h2>The Options Flags Field</h2> <p>Flags consist of zero or more characters that immediately follow the "%" that introduces the substitution. The various flags and their meanings are as follows: <center> <table border=1 cellpadding="10" width="80%"> <tr> <th>Flag <th>Meaning <tr> <td><b>-</b> <td>Left-justify the value in the output. The default is to right-justify. If the width is zero or is otherwise less than the length of the value being substituted, then there is no padding and the "-" flag is a no-op. <tr> <td><b>+</b> <td>For signed numeric substitutions, include a "+" sign before positive numbers. A "-" sign always appears before negative numbers regardless of flag settings. <tr> <td><i>(space)</i> <td>For signed numeric substitutions, prepend a single space before positive numbers. <tr> <td><b>0</b> <td>Prepend as many "0" characters to numeric substitutions as necessary to expand the value out to the specified width. If the width field is omitted, then this flag is a no-op. <tr> <td><b>#</b> <td>This is the "alternate-form-1" flag. For %g and %G substitutions, this causes trailing zeros to be removed. This flag forces a decimal point to appear for all floating-point substitutions. For %o, %x, and %X substitutions, the alternate-form-1 flag cause the value to be prepended with "0", "0x", or "0X", respectively. <tr> <td><b>,</b> <td>This flag causes comma-separators to be added to the output of %d and %i substitutions, between every 3 digits from the left. This can help humans to more easily discern the magnitude of large integer values. For example, the value 2147483647 would be rendered as "2147483647" using "%d" but would appear as "2,147,483,647" with "%,d". This flag is a non-standard extension. <tr> <td><b>!</b> <td>This is the "alternate-form-2 flag. For string substitutions, this flag causes the width and precision to be understand in terms of characters rather than bytes. For floating point substitutions, the alternate-form-2 flag increases the maximum number of significant digits displayed from 16 to 26, forces the display of the decimal point and causes at least one digit to appear after the decimal point.<br><br> The alternate-form-2 flag is a non-standard extension that appears in no other printf() implementations, as far as we know. </table> </center> <h1>Implementation And History</h1> <p> The core string formatting routine is the sqlite3VXPrintf() function found in the [https://sqlite.org/src/file/src/printf.c|printf.c] source file. All the various interfaces invoke (sometimes indirectly) this one core function. The sqlite3VXPrintf() function began as code written by the first author of SQLite ([Hipp]) when he was a graduate student a Duke University in the late 1980s. Hipp kept this printf() implementation in his personal toolbox until he started working on SQLite in 2000. The code was incorporated into the SQLite source tree on [https://sqlite.org/src/timeline?c=f9372072a6|2000-10-08] for SQLite version 1.0.9. <p> The [https://www.fossil-scm.org/|Fossil Version Control System] uses its own printf() implementation that is derived from and early version of the SQLite printf() implementation, but those two implementations have since diverged. <p> The reason that the [sqlite3_snprintf()] has its buffer pointer and buffer size arguments reversed from what is found in the standard library snprintf() routine is because there was no snprintf() routine in the standard C library when Hipp was first implementing his version, and he chose a different order than the designers of the standard C library. |