Documentation Source Text

Check-in [73745849d3]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Update description of AFL testing.
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 73745849d30e3ad84efd31f104bdcab10fb60e0e
User & Date: drh 2015-06-20 14:29:36
Context
2015-06-26
18:47
Fix the opcode documentation generator so that it can handle digits at the end of opcode names. check-in: 7a0fc411fe user: drh tags: trunk
2015-06-20
14:29
Update description of AFL testing. check-in: 73745849d3 user: drh tags: trunk
2015-06-19
18:57
Fix an error in fts5.in. check-in: 8f8e1e988f user: dan tags: trunk
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to pages/testing.in.

373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
...
388
389
390
391
392
393
394
395
396
397

398
399
400
401

402
403
404
405
406
407
408

<tcl>hd_fragment aflfuzz {American Fuzzy Lop fuzzer}</tcl>
<h4>4.1.1 SQL Fuzz Using The American Fuzzy Lop Fuzzer</h4>

<p>The <a href="http://lcamtuf.coredump.cx/afl/">American Fuzzy Lop</a>
or "AFL" fuzzer is a recent (circa 2014) innovation from Michal Zalewski.
Unlike most other fuzzers that blindly generate random inputs, the AFL
fuzzer instruments the program being tested (by editing the assembly-language
output from the C compiler) and uses that instrumentation to detect when
an input causes the program to do something different - to follow
a new control path or loop a different number of times.  Inputs that provoke
new behavior are retained and further mutated.  In this way, AFL is able
to "discover" new behaviors of the program under test, including behaviors
that were never envisioned by the designers.

................................................................................
<p>AFL has proven remarkably adept at finding arcane bugs in SQLite.
Most of the findings have been assert() statements where the conditional
was false under obscure circumstances.  But AFL has also found
a fair number of crash bugs in SQLite, and even a few cases where SQLite 
computed incorrect results.

<p>Because of its past success, AFL became a standard part of the testing
strategy for SQLite beginning with [version 3.8.10].  There is at least one
instance of AFL running against SQLite continuously, 24/7/365, trying new
randomly mutated inputs against SQLite at a rate of a few hundred to a few

thousand per second.  Billions of inputs have been tried, but AFL's 
instrumentation has narrowed them down to less than 20,000 test cases that
cover all distinct behaviors.  Newly discovered test cases are periodically
captured and added to the [TCL test suite].


<h3>4.2 Malformed Database Files</h3>

<p>There are numerous test cases that verify that SQLite is able to
deal with malformed database files.
These tests first build a well-formed database file, then add
corruption by changing one or more bytes in the file by some means







|







 







|


>
|
|

|
>







373
374
375
376
377
378
379
380
381
382
383
384
385
386
387
...
388
389
390
391
392
393
394
395
396
397
398
399
400
401
402
403
404
405
406
407
408
409
410

<tcl>hd_fragment aflfuzz {American Fuzzy Lop fuzzer}</tcl>
<h4>4.1.1 SQL Fuzz Using The American Fuzzy Lop Fuzzer</h4>

<p>The <a href="http://lcamtuf.coredump.cx/afl/">American Fuzzy Lop</a>
or "AFL" fuzzer is a recent (circa 2014) innovation from Michal Zalewski.
Unlike most other fuzzers that blindly generate random inputs, the AFL
fuzzer instruments the program being tested (by modifying the assembly-language
output from the C compiler) and uses that instrumentation to detect when
an input causes the program to do something different - to follow
a new control path or loop a different number of times.  Inputs that provoke
new behavior are retained and further mutated.  In this way, AFL is able
to "discover" new behaviors of the program under test, including behaviors
that were never envisioned by the designers.

................................................................................
<p>AFL has proven remarkably adept at finding arcane bugs in SQLite.
Most of the findings have been assert() statements where the conditional
was false under obscure circumstances.  But AFL has also found
a fair number of crash bugs in SQLite, and even a few cases where SQLite 
computed incorrect results.

<p>Because of its past success, AFL became a standard part of the testing
strategy for SQLite beginning with [version 3.8.10].  There are at least two
instance of AFL running against SQLite continuously, 24/7/365, trying new
randomly mutated inputs against SQLite at a rate of a few hundred to a few
thousand per second.  Both SQL statements and database files are fuzzed.
Billions and billions of mutations have been tried, but AFL's 
instrumentation has narrowed them down to less than 30,000 test cases that
cover all distinct behaviors.  Newly discovered test cases are periodically
captured and added to the [TCL test suite] where they can be rerun using
the "make fuzztest" command.

<h3>4.2 Malformed Database Files</h3>

<p>There are numerous test cases that verify that SQLite is able to
deal with malformed database files.
These tests first build a well-formed database file, then add
corruption by changing one or more bytes in the file by some means