SQLite

Check-in [9570b6b43d]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Minor simplification to the tokenizer. Slightly smaller and faster.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA1: 9570b6b43df3bc5ce314cded20bca8be9e968efe
User & Date: drh 2016-02-16 13:04:19.100
References
2016-02-18
14:49
Avoid a potential buffer overrun if an SQL statement being parsed ends with an illegal "!" token. (This problem was detected by fuzzcheck running under valgrind. The problem was introduced by check-in [9570b6b43df3], two days ago and has not appeared in a release.) (check-in: 2a8d97e7c8 user: drh tags: trunk)
Context
2016-02-17
11:13
Documentation typo on sqlite3_wal_hook(). No changes to code. (check-in: fc2043ebaa user: drh tags: trunk)
2016-02-16
21:19
Experimental changes to Lemon for improved parser performance. (check-in: a65d583ce9 user: drh tags: parser-performance)
13:04
Minor simplification to the tokenizer. Slightly smaller and faster. (check-in: 9570b6b43d user: drh tags: trunk)
01:01
Improve the Lemon parser template (lempar.c) so that it avoids unnecessary work when the grammer defines YYNOERRORRECOVERY (as SQLite does). Slightly smaller and faster code results. (check-in: 9235b0cf6a user: drh tags: trunk)
Changes
Side-by-Side Diff Ignore Whitespace Patch
Changes to src/tokenize.c.
︙
519
520
521
522
523
524
525
526
527
528
529
530
531
532

533
534
535
536
537
538
539
540
541
542
543
544

519
520
521
522
523
524
525

526
527
528
529
530
531
532
533
534
535
536

537
538
539
540
541
542
543








-






+




-








      }
      if( tokenType==TK_ILLEGAL ){
        sqlite3ErrorMsg(pParse, "unrecognized token: \"%T\"",
                        &pParse->sLastToken);
        break;
      }
    }else{
      if( tokenType==TK_SEMI ) pParse->zTail = &zSql[i];
      sqlite3Parser(pEngine, tokenType, pParse->sLastToken, pParse);
      lastTokenParsed = tokenType;
      if( pParse->rc!=SQLITE_OK || db->mallocFailed ) break;
    }
  }
  assert( nErr==0 );
  pParse->zTail = &zSql[i];
  if( pParse->rc==SQLITE_OK && db->mallocFailed==0 ){
    assert( zSql[i]==0 );
    if( lastTokenParsed!=TK_SEMI ){
      sqlite3Parser(pEngine, TK_SEMI, pParse->sLastToken, pParse);
      pParse->zTail = &zSql[i];
    }
    if( pParse->rc==SQLITE_OK && db->mallocFailed==0 ){
      sqlite3Parser(pEngine, 0, pParse->sLastToken, pParse);
    }
  }
#ifdef YYTRACKMAXSTACKDEPTH
  sqlite3_mutex_enter(sqlite3MallocMutex());
︙