Documentation Source Text

Check-in [b752742d9c]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update the database size limit to 281 terabytes.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: b752742d9c400667e44c79c11e22ded83b5b3801e9ba84ac73cc30cf29becee6
User & Date: drh 2020-07-30 17:46:34
Context
2020-08-01
00:14
Fix the max database size on the about.html page. (Leaf check-in: 4ad59ef382 user: drh tags: trunk)
2020-07-30
17:46
Update the database size limit to 281 terabytes. (check-in: b752742d9c user: drh tags: trunk)
2020-07-28
19:39
Update the size-and-speed spreadsheet. (check-in: 1c1f5f5f3b user: drh tags: trunk)
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/about.in.

7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<ul>
<li> [Full-featured SQL]
<li> <a href='mostdeployed.html'>Billions and billions of deployments</a>
<li> <a href='onefile.html'>Single-file database</a>
<li> <a href='copyright.html'>Public domain source code</a>
<li> All source code in one file (<a href='amalgamation.html'>sqlite3.c</a>)
<li> <a href='footprint.html'>Small footprint</a>
<li> Max DB size: <a href='limits.html'>140 terabytes</a>
     (2<sup><small>47</small></sup> bytes)
<li> Max row size: <a href='limits.html'>1 gigabyte</a>
<li> <a href='fasterthanfs.html'>Faster than direct file I/O</a>
<li> <a href='testing.html'>Aviation-grade quality and testing</a>
<li> <a href='zeroconf.html'>Zero-configuration</a>
<li> <a href='transactional.html'>ACID transactions, even after power loss</a>
<li> <a href='fileformat.html'>Stable, enduring file format</a>







|







7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
<ul>
<li> [Full-featured SQL]
<li> <a href='mostdeployed.html'>Billions and billions of deployments</a>
<li> <a href='onefile.html'>Single-file database</a>
<li> <a href='copyright.html'>Public domain source code</a>
<li> All source code in one file (<a href='amalgamation.html'>sqlite3.c</a>)
<li> <a href='footprint.html'>Small footprint</a>
<li> Max DB size: <a href='limits.html'>281 terabytes</a>
     (2<sup><small>47</small></sup> bytes)
<li> Max row size: <a href='limits.html'>1 gigabyte</a>
<li> <a href='fasterthanfs.html'>Faster than direct file I/O</a>
<li> <a href='testing.html'>Aviation-grade quality and testing</a>
<li> <a href='zeroconf.html'>Zero-configuration</a>
<li> <a href='transactional.html'>ACID transactions, even after power loss</a>
<li> <a href='fileformat.html'>Stable, enduring file format</a>

Changes to pages/changes.in.

22
23
24
25
26
27
28

29
30
31
32
33
34
35
    set xrefChng($vers) $nChng
  }
  incr nChng
}

chng {2020-08-22 (3.33.0)} {
<li> Support for [UPDATE FROM] following the PostgreSQL syntax.

<li> Extended the [PRAGMA integrity_check] statement so that it can optionally be
     limited to verifying just a single table and its indexes, rather than the
     entire database file.
<li> Added the [decimal extension] for doing arbitrary-precision decimal arithmetic.
<li> Enhancments to the [ieee754 extension] for working with IEEE 754 binary64 numbers.
<li> [CLI] enhancements:
    <ol type="a">







>







22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
    set xrefChng($vers) $nChng
  }
  incr nChng
}

chng {2020-08-22 (3.33.0)} {
<li> Support for [UPDATE FROM] following the PostgreSQL syntax.
<li> Increase the maximum size of database files to 281 TB.
<li> Extended the [PRAGMA integrity_check] statement so that it can optionally be
     limited to verifying just a single table and its indexes, rather than the
     entire database file.
<li> Added the [decimal extension] for doing arbitrary-precision decimal arithmetic.
<li> Enhancments to the [ieee754 extension] for working with IEEE 754 binary64 numbers.
<li> [CLI] enhancements:
    <ol type="a">

Changes to pages/fileformat2.in.

37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
<p>The main database file consists of one or more pages.  ^The size of a
page is a power of two between 512 and 65536 inclusive.  All pages within
the same database are the same size.  ^The page size for a database file
is determined by the 2-byte integer located at an offset of
16 bytes from the beginning of the database file.</p>

<p>Pages are numbered beginning with 1.  The maximum page number is
2147483646 (2<sup><small>31</small></sup> - 2).  The minimum size
SQLite database is a single 512-byte page.
The maximum size database would be 2147483646 pages at 65536 bytes per
page or 140,737,488,224,256 bytes (about 140 terabytes).  Usually SQLite will
hit the maximum file size limit of the underlying filesystem or disk
hardware long before it hits its own internal size limit.</p>

<p>In common use, SQLite databases tend to range in size from a few kilobytes
to a few gigabytes, though terabyte-size SQLite databases are known to exist
in production.</p>








|


|







37
38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
<p>The main database file consists of one or more pages.  ^The size of a
page is a power of two between 512 and 65536 inclusive.  All pages within
the same database are the same size.  ^The page size for a database file
is determined by the 2-byte integer located at an offset of
16 bytes from the beginning of the database file.</p>

<p>Pages are numbered beginning with 1.  The maximum page number is
4294967294 (2<sup><small>32</small></sup> - 2).  The minimum size
SQLite database is a single 512-byte page.
The maximum size database would be 2147483646 pages at 65536 bytes per
page or 281,474,976,579,584 bytes (about 281 terabytes).  Usually SQLite will
hit the maximum file size limit of the underlying filesystem or disk
hardware long before it hits its own internal size limit.</p>

<p>In common use, SQLite databases tend to range in size from a few kilobytes
to a few gigabytes, though terabyte-size SQLite databases are known to exist
in production.</p>

Changes to pages/limits.in.

376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
1073741823, is the maximum number of pages allowed in a single
database file.  An attempt to insert new data that would cause
the database file to grow larger than this will return
SQLITE_FULL.
</p>

<p>
The largest possible setting for SQLITE_MAX_PAGE_COUNT is 2147483646.
When used with the maximum page size of 65536, this gives a maximum
SQLite database size of about 140 terabytes.</p>

<p>
The <a href="pragma.html#pragma_max_page_count">
max_page_count PRAGMA</a> can be used to raise or lower this
limit at run-time.
</p>
}

limititem {Maximum Number Of Rows In A Table} {} {
<p>
The theoretical maximum number of rows in a table is
2<sup><small>64</small></sup> (18446744073709551616 or about 1.8e+19).
This limit is unreachable since the maximum database size of 140 terabytes
will be reached first.  A 140 terabytes database can hold no more than
approximately 1e+13 rows, and then only if there are no indices and if
each row contains very little data.
}

limititem {Maximum Database Size} {} {
<p>
Every database consists of one or more "pages".  Within a single database,
every page is the same size, but different database can have page sizes
that are powers of two between 512 and 65536, inclusive.  The maximum
size of a database file is 2147483646 pages.  At the maximum page size
of 65536 bytes, this translates into a maximum database size of 
approximately 1.4e+14 bytes (140 terabytes, or 128 tebibytes, or
140,000 gigabytes or 128,000 gibibytes).
<p>
This particular upper bound is untested since the developers do not 
have access to hardware capable of reaching this limit.  However, tests
do verify that SQLite behaves correctly and sanely when a database 
reaches the maximum file size of the underlying filesystem (which is
usually much less than the maximum theoretical database size) and when
a database is unable to grow due to disk space exhaustion.







|

|












|
|
|








|

|
|







376
377
378
379
380
381
382
383
384
385
386
387
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
411
412
413
414
415
416
417
418
419
1073741823, is the maximum number of pages allowed in a single
database file.  An attempt to insert new data that would cause
the database file to grow larger than this will return
SQLITE_FULL.
</p>

<p>
The largest possible setting for SQLITE_MAX_PAGE_COUNT is 4294967294.
When used with the maximum page size of 65536, this gives a maximum
SQLite database size of about 281 terabytes.</p>

<p>
The <a href="pragma.html#pragma_max_page_count">
max_page_count PRAGMA</a> can be used to raise or lower this
limit at run-time.
</p>
}

limititem {Maximum Number Of Rows In A Table} {} {
<p>
The theoretical maximum number of rows in a table is
2<sup><small>64</small></sup> (18446744073709551616 or about 1.8e+19).
This limit is unreachable since the maximum database size of 281 terabytes
will be reached first.  A 281 terabytes database can hold no more than
approximately 2e+13 rows, and then only if there are no indices and if
each row contains very little data.
}

limititem {Maximum Database Size} {} {
<p>
Every database consists of one or more "pages".  Within a single database,
every page is the same size, but different database can have page sizes
that are powers of two between 512 and 65536, inclusive.  The maximum
size of a database file is 4294967294 pages.  At the maximum page size
of 65536 bytes, this translates into a maximum database size of 
approximately 1.4e+14 bytes (281 terabytes, or 256 tebibytes, or
281474 gigabytes or 256,000 gibibytes).
<p>
This particular upper bound is untested since the developers do not 
have access to hardware capable of reaching this limit.  However, tests
do verify that SQLite behaves correctly and sanely when a database 
reaches the maximum file size of the underlying filesystem (which is
usually much less than the maximum theoretical database size) and when
a database is unable to grow due to disk space exhaustion.

Changes to pages/whentouse.in.

313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
...
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
But if the website is write-intensive or is so busy that it requires
multiple servers, then consider using an enterprise-class client/server 
database engine instead of SQLite.</p>
</li>

<li><p><b>Very large datasets</b></p>

<p>An SQLite database is limited in size to 140 terabytes 
(2<sup><small>47</small></sup> bytes, 128 tibibytes).
And even if it could handle larger databases, SQLite stores the entire
database in a single disk file and many filesystems limit the maximum
size of files to something less than this.  So if you are contemplating
databases of this magnitude, you would do well to consider using a
client/server database engine that spreads its content across multiple
disk files, and perhaps across multiple volumes.
................................................................................
</li>

<li><p><b>Big data? &rarr; choose client/server</b></p>

<p>If your data will grow to a size that you are uncomfortable
or unable to fit into a single disk file, then you should select
a solution other than SQLite.  SQLite supports databases up to
140 terabytes in size, assuming you can find a disk drive and filesystem
that will support 140-terabyte files.  Even so, when the size of the
content looks like it might creep into the terabyte range, it would
be good to consider a centralized client/server database.
</li>

<li><p><b>Otherwise &rarr; choose SQLite!</b></p>

<p>For device-local storage with low writer concurrency and less than a
terabyte of content, SQLite is almost always a better solution.  SQLite
is fast and reliable and it requires no configuration or maintenance.
It keeps things simple.  SQLite "just works".
</li>
</ol>







|







 







|
|












313
314
315
316
317
318
319
320
321
322
323
324
325
326
327
...
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410
But if the website is write-intensive or is so busy that it requires
multiple servers, then consider using an enterprise-class client/server 
database engine instead of SQLite.</p>
</li>

<li><p><b>Very large datasets</b></p>

<p>An SQLite database is limited in size to 281 terabytes 
(2<sup><small>47</small></sup> bytes, 128 tibibytes).
And even if it could handle larger databases, SQLite stores the entire
database in a single disk file and many filesystems limit the maximum
size of files to something less than this.  So if you are contemplating
databases of this magnitude, you would do well to consider using a
client/server database engine that spreads its content across multiple
disk files, and perhaps across multiple volumes.
................................................................................
</li>

<li><p><b>Big data? &rarr; choose client/server</b></p>

<p>If your data will grow to a size that you are uncomfortable
or unable to fit into a single disk file, then you should select
a solution other than SQLite.  SQLite supports databases up to
281 terabytes in size, assuming you can find a disk drive and filesystem
that will support 281-terabyte files.  Even so, when the size of the
content looks like it might creep into the terabyte range, it would
be good to consider a centralized client/server database.
</li>

<li><p><b>Otherwise &rarr; choose SQLite!</b></p>

<p>For device-local storage with low writer concurrency and less than a
terabyte of content, SQLite is almost always a better solution.  SQLite
is fast and reliable and it requires no configuration or maintenance.
It keeps things simple.  SQLite "just works".
</li>
</ol>