Many hyperlinks are disabled.
Use anonymous login
to enable hyperlinks.
Overview
Comment: | Update test metrics on the testing.html page. |
---|---|
Downloads: | Tarball | ZIP archive |
Timelines: | family | ancestors | descendants | both | trunk |
Files: | files | file ages | folders |
SHA3-256: |
6fc727b5107887544e7d6692fd00e1aa |
User & Date: | drh 2017-07-26 15:06:51.827 |
Context
2017-07-26
| ||
17:25 | Add a news entry. Fix the release date. Fix typos. (check-in: 9759ba8053 user: drh tags: trunk) | |
15:06 | Update test metrics on the testing.html page. (check-in: 6fc727b510 user: drh tags: trunk) | |
2017-07-25
| ||
17:03 | Change log typo fixes. (check-in: 0e82b7e2dd user: drh tags: trunk) | |
Changes
Changes to pages/testing.in.
︙ | ︙ | |||
9 10 11 12 13 14 15 | # facilitate that, all the size values are defined by variables here # which are then used as needed through the document. # # NOTE: Also update the version number in the text!!! # # sloc sqlite3.c | | | | | | | | | | | | | | | | | > | | | | | 9 10 11 12 13 14 15 16 17 18 19 20 21 22 23 24 25 26 27 28 29 30 31 32 33 34 35 36 37 38 39 40 41 42 43 44 45 46 47 48 49 50 51 52 53 54 55 56 57 58 59 60 61 62 63 64 65 66 67 68 69 70 71 72 73 74 75 76 77 78 | # facilitate that, all the size values are defined by variables here # which are then used as needed through the document. # # NOTE: Also update the version number in the text!!! # # sloc sqlite3.c set stat(coreSLOC) 125375 ;# Non-comment lines of amalgamation code # sloc test*.c set stat(tclcSLOC) 25300 ;# Non-comment lines of test C code # ls test*.c tclsqlite.c | wc set stat(tclcNfile) 46 ;# Number of files of TCL C testcode + tclsqlite.c # ls -l test*.c tclsqlite.c | awk '{sum+=$5}END{print sum}' set stat(tclcNByte) 1193625 ;# Number of bytes of TCL C testcode + tclsqlite.c # sloc `find . -name '*.test' -print` test/*.tcl set stat(tclsSLOC) 317456 ;# Non-comment lines of TCL test script # ls `find . -name '*.test' -print` test/*.tcl | wc set stat(tclsNFile) 1135 ;# Number of files of TCL test script # wc `find . -name '*.test' -print` test/*.tcl set stat(tclsNByte) 13984314 ;# Number of bytes of TCL test script # cat `find . -name '*.test' -print` | egrep 'do[_a-z]*_test' | wc set stat(tclNTest) 39747 ;# Number of test cases in the TCL test suite set stat(tclNEval) 2504837 ;# Number of test case evaluations (fulltest) # grep fuzz device_one/test-out.txt | wc set stat(nSqlFuzz) 111268 ;# Number of SQL fuzz tests set stat(vqNEval) 142597 ;# Number of test evaluations for veryquick.test # set stat(vqStmtCov) 97.23 ;# veryquick statement coverage # set stat(vqBrCov) 92.57 ;# veryquick branch coverage # set stat(allStmtCov) 99.50 ;# all.test statement coverage # set stat(allBrCov) 97.41 ;# all.test condition/decision coverage # tclsh mkth3.tcl cfg/*.cfg */*.test >th3.c; sloc th3.c set stat(th3SLOC) 782314 ;# Non-comment lines in full th3.c # ls -l th3.c set stat(th3NByte) 57259210 ;# Number of bytes in full th3.c # grep th3testCheck */*.test |wc # grep '^--result' */*.test | wc # grep '^--glob' */*.test | wc # grep '^--notglob' */*.test | wc # grep '^--eqp' */*.test | wc set stat(th3NTest) 42213 ;# Number of test cases # from output of a min.rc test run. set stat(th3NECov) 1726070 ;# Number of test case evals for coverage #set stat(th3NETest) 7247055 ;# Number of test case evaluations #set stat(th3NEExt) 589175483 ;# Number of test case evals extended #set stat(th3NERel) 2500000000 ;# Number of test case evals release set stat(th3StmtCov) 100.00 ;# TH3 statement coverage set stat(th3BrCov) 100.00 ;# TH3 branch coverage # wc `find . -name '*.test'` | awk '{x+=$1}END{print x}' set stat(sltsSLOC) 90489494 ;# Non-comment lines of SLT test script # ls -l `find . -name '*.test'` | awk '{sum+=$5}END{print sum}' set stat(sltsNByte) 1116800308 ;# Bytes of SLT test script # find . -name '*.test' | wc set stat(sltsNFile) 622 ;# Files of SLT test script # sloc md5.c slt_*.c sqllogictest.c set stat(sltcSLOC) 1404 ;# Non-comment lines of SLT C code # grep '^query' `fossil ls | awk '/\.test$/{print $2}'` | wc set stat(sltNTest) 7195342 ;# Number of test cases in SLT # grep 'errors out of' all-out.txt | awk '{x+=$5}END{print x}' set stat(sltNRun) 11879758 ;# Number of tests run in SLT # grep 'assert(' sqlite3.c | wc set stat(nAssert) 5285 ;# Number of assert statements # grep 'testcase(' sqlite3.c | grep -v define | wc set stat(nTestcase) 846 ;# Number of testcase statements set stat(totalSLOC) [expr {$stat(tclcSLOC)+$stat(tclsSLOC)+ $stat(th3SLOC)+$stat(sltcSLOC)+$stat(sltsSLOC)}] proc GB {expr} { set n [uplevel #0 expr $expr] hd_puts [format %.2f [expr {$n/(1000.0*1000.0*1000.0)}]] |
︙ | ︙ | |||
102 103 104 105 106 107 108 | </tcl> <h1>Introduction</h1> <p>The reliability and robustness of SQLite is achieved in part by thorough and careful testing.</p> | | | 103 104 105 106 107 108 109 110 111 112 113 114 115 116 117 | </tcl> <h1>Introduction</h1> <p>The reliability and robustness of SQLite is achieved in part by thorough and careful testing.</p> <p>As of [version 3.20.0] ([dateof:3.20.0]), the SQLite library consists of approximately <tcl>KB {$stat(coreSLOC)}</tcl> KSLOC of C code. (KSLOC means thousands of "Source Lines Of Code" or, in other words, lines of code excluding blank lines and comments.) By comparison, the project has <tcl> hd_puts "[expr {int($stat(totalSLOC)/$stat(coreSLOC))}] times as much" |
︙ | ︙ | |||
165 166 167 168 169 170 171 | so that on a full test run millions of separate tests are performed. </p> </li> <li><p> The <b>[TH3]</b> test harness is a set of proprietary tests, written in | | > | 166 167 168 169 170 171 172 173 174 175 176 177 178 179 180 181 | so that on a full test run millions of separate tests are performed. </p> </li> <li><p> The <b>[TH3]</b> test harness is a set of proprietary tests, written in C that provide 100% branch test coverage (and [MC/DC|100% MC/DC test coverage]) to the core SQLite library. The TH3 tests are designed to run on embedded and specialized platforms that would not easily support TCL or other workstation services. TH3 tests use only the published SQLite interfaces. TH3 consists of about <tcl>MB {$stat(th3NByte)}</tcl> MB or <tcl>KB {$stat(th3SLOC)}</tcl> KSLOC of C code implementing <tcl>N {$stat(th3NTest)}</tcl> distinct test cases. TH3 tests are heavily parameterized, though, so a full-coverage test runs |
︙ | ︙ |