/ Check-in [f11beb16]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Rationalize some code in zonefile.c. Fix other minor issues in the same.
Downloads: Tarball | ZIP archive | SQL archive
Timelines: family | ancestors | descendants | both | zonefile
Files: files | file ages | folders
SHA3-256: f11beb16a87cc967e896cf67121b1e4045e427ebdc6a424e9f009ffced955d36
User & Date: dan 2018-02-27 19:50:28
Context
2018-02-27
20:09
Update zonefile README.md file to mention the frame cache. Leaf check-in: 84e9351b user: dan tags: zonefile
19:50
Rationalize some code in zonefile.c. Fix other minor issues in the same. check-in: f11beb16 user: dan tags: zonefile
15:47
Enhance test 'zonefile1-6.5' to account for platform differences. check-in: 8b617840 user: mistachkin tags: zonefile
Changes
Hide Diffs Unified Diffs Ignore Whitespace Patch

Changes to ext/zonefile/README.md.

51
52
53
54
55
56
57
58







59
60
61
62
63
64
65
..
85
86
87
88
89
90
91
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107
108
<td>The compression type used to compress the zonefile index structure.
All values that are valid for the <i>compressionTypeContent</i> parameter,
except for "zstd_global_dict", are also valid for this option.

<tr valign=top><td>encryptionType<td>"none"
<td>The encryption type to use. At present the only valid values are
"none" (no encryption) and "xor" (an insecure mock encryption method
useful for testing only).








<tr valign=top><td>encryptionKey<td>""
<td>The encryption key to use. The encryption key must be specified as an
even number of hexadecimal that will be converted to a binary key before
use. It is the responsibility of the caller to specify a key of the optimal
length for each encryption algorithm (e.g. 16 bytes (32 hex digits) for
a 128-bit encryption, or 32 bytes (64 digits) for a 256-bit method).
................................................................................

This creates two virtual tables in the database schema. One read-only table
named "z1", with a schema equivalent to:

>     CREATE TABLE z1(  -- this whole table is read-only
>       k INTEGER PRIMARY KEY,     -- key value
>       v BLOB,                    -- associated blob of data
>       file TEXT,                 -- file this key is read from 
>       sz INTEGER                 -- size of blob of data in bytes
>     );

And a read-write table named "z1_files" with a schema like:

>     CREATE TABLE z1_files(
>       filename TEXT PRIMARY KEY,
>       ekey BLOB,         -- encryption key
>       fileid INTEGER,    -- read-only
>       header JSON HIDDEN -- read-only
>     );

Both tables are initially empty. To add a zonefile to the index, insert a
row into the "z1_files" table:

>     INSERT INTO z1_files(filename) VALUES(<filename>);







|
>
>
>
>
>
>
>







 







|








<







51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
..
92
93
94
95
96
97
98
99
100
101
102
103
104
105
106
107

108
109
110
111
112
113
114
<td>The compression type used to compress the zonefile index structure.
All values that are valid for the <i>compressionTypeContent</i> parameter,
except for "zstd_global_dict", are also valid for this option.

<tr valign=top><td>encryptionType<td>"none"
<td>The encryption type to use. At present the only valid values are
"none" (no encryption) and "xor" (an insecure mock encryption method
useful for testing only). Enhanced implementations may support any or
all of the following encryption schemes:
<ul>
  <li> "AES_128_CTR"
  <li> "AES_128_CBC"
  <li> "AES_256_CTR"
  <li> "AES_256_CBC"
</ul>

<tr valign=top><td>encryptionKey<td>""
<td>The encryption key to use. The encryption key must be specified as an
even number of hexadecimal that will be converted to a binary key before
use. It is the responsibility of the caller to specify a key of the optimal
length for each encryption algorithm (e.g. 16 bytes (32 hex digits) for
a 128-bit encryption, or 32 bytes (64 digits) for a 256-bit method).
................................................................................

This creates two virtual tables in the database schema. One read-only table
named "z1", with a schema equivalent to:

>     CREATE TABLE z1(  -- this whole table is read-only
>       k INTEGER PRIMARY KEY,     -- key value
>       v BLOB,                    -- associated blob of data
>       fileid INTEGER,            -- file id (rowid value for z1_files)
>       sz INTEGER                 -- size of blob of data in bytes
>     );

And a read-write table named "z1_files" with a schema like:

>     CREATE TABLE z1_files(
>       filename TEXT PRIMARY KEY,
>       ekey BLOB,         -- encryption key

>       header JSON HIDDEN -- read-only
>     );

Both tables are initially empty. To add a zonefile to the index, insert a
row into the "z1_files" table:

>     INSERT INTO z1_files(filename) VALUES(<filename>);

Changes to ext/zonefile/zonefile.c.

330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
...
689
690
691
692
693
694
695








696
697
698
699
700
701
702
...
706
707
708
709
710
711
712







713
714
715
716
717
718
719
...
779
780
781
782
783
784
785




786
787
788
789
790
791













792
793
794
795
796
797
798
799
800
801
802
803
...
839
840
841
842
843
844
845




846
847
848
849
850
851
852
853
854
855
856
857










858
859
860
861
862
863
864
...
876
877
878
879
880
881
882








883
884
885
886
887
888
889
...
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
...
979
980
981
982
983
984
985



986
987
988
989
990




991
992
993
994
995
996
997
998
999
1000
1001




1002
1003
1004
1005
1006




1007
1008
1009
1010
1011




1012
1013
1014
1015
1016





1017
1018
1019
1020
1021





1022
1023
1024
1025
1026


1027
1028
1029
1030








1031
1032
1033
1034
1035
1036
1037
1038
1039
1040



1041
1042
1043
1044










1045
1046
1047
1048
1049
1050
1051
....
1091
1092
1093
1094
1095
1096
1097




1098
1099
1100
1101
1102
1103
1104














1105
1106
1107
1108
1109
1110
1111
....
1379
1380
1381
1382
1383
1384
1385



1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398



1399
1400
1401
1402
1403
1404
1405
....
1571
1572
1573
1574
1575
1576
1577




1578
1579
1580
1581
1582
1583
1584
....
1585
1586
1587
1588
1589
1590
1591
1592

1593
1594
1595
1596
1597










1598
1599
1600
1601
1602


1603
1604

1605
1606






















1607




1608
1609
1610
1611
1612
1613
1614
1615
1616
1617
1618
1619
1620
1621
1622
1623
1624
1625
1626
1627
1628
1629
1630
1631
1632
1633
1634
1635
1636
1637
1638
1639
1640
1641
1642
1643


1644
1645
1646
1647
1648
1649
1650
....
1673
1674
1675
1676
1677
1678
1679
1680
1681
1682
1683
1684
1685
1686
1687



1688
1689
1690
1691
1692
1693
1694
1695
1696
1697
1698
1699
1700
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715

1716
1717
1718
1719
1720
1721
1722
1723
1724
....
1732
1733
1734
1735
1736
1737
1738









1739
1740
1741
1742
1743
1744
1745
1746
1747
1748
1749
1750
1751
1752













1753
1754
1755

1756
1757

1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
....
2602
2603
2604
2605
2606
2607
2608
2609
2610
2611
2612
2613
2614
2615
2616

  return SQLITE_OK;
}

/*
** Search the key-store passed as the first argument for an encryption
** key to use with the file with file-id iFileid in zonefile table zTab
** in database zDb. If successful, set (*pzKey) to point to the key
** buffer and return the size of the key in bytes.
**
** If no key is found, return 0. The final value of (*pzKey) is undefined
** in this case.
*/
static int zonefileKeyFind(
  ZonefileGlobal *pGlobal,
  const char *zDb,                /* Database containing zonefile table */
  const char *zTab,               /* Name of zonefile table */
  i64 iFileid,                    /* File-id to configure key for */
................................................................................
    0,
    zfLz4CompressBound, zfLz4hcCompress, 
    zfGenericUncompressSize, zfLz4Uncompress
  },
#endif /* SQLITE_HAVE_LZ4 */
};









static int zonefileCompress(
  const char *zName,              /* Name of requested compression method */
  ZonefileCompress **pp,          /* OUT: Pointer to compression object */
  char **pzErr                    /* OUT: Error message */
){
  int i;
  assert( *pzErr==0 );
................................................................................
      return SQLITE_OK;
    }
  }
  *pzErr = sqlite3_mprintf("unknown compression scheme: \"%s\"", zName);
  return SQLITE_ERROR;
}








static ZonefileCompress *zonefileCompressByValue(int eType){
  int i;
  for(i=0; i<sizeof(aZonefileCompress)/sizeof(aZonefileCompress[0]); i++){
    if( aZonefileCompress[i].eType==eType ){
      return &aZonefileCompress[i];
    }
  }
................................................................................
  va_start(ap, zFmt);
  zMsg = sqlite3_vmprintf(zFmt, ap);
  sqlite3_result_error(ctx, zMsg, -1);
  sqlite3_free(zMsg);
  va_end(ap);
}





static void zonefileTransferError(sqlite3_context *pCtx){
  sqlite3 *db = sqlite3_context_db_handle(pCtx);
  const char *zErr = sqlite3_errmsg(db);
  sqlite3_result_error(pCtx, zErr, -1);
}














static int zonefilePrepare(
  sqlite3 *db,
  sqlite3_stmt **ppStmt,
  char **pzErr,
  const char *zFmt,
  ...
){
  int rc;
  va_list ap;
  char *zSql;
  va_start(ap, zFmt);
  zSql = sqlite3_vmprintf(zFmt, ap);
................................................................................
    default:
      assert( eType==SQLITE_NULL);
  }

  return 0;
}





int zonefileIsAutoFrame(sqlite3_value *pFrame){
  return (
      sqlite3_value_type(pFrame)==SQLITE_INTEGER 
   && sqlite3_value_int64(pFrame)==-1
  );
}

#define SQLITE_ZONEFILE_AES_128_CTR 1
#define SQLITE_ZONEFILE_AES_128_CBC 2
#define SQLITE_ZONEFILE_AES_256_CTR 3
#define SQLITE_ZONEFILE_AES_256_CBC 4











static int zonefileEncryption(const char *zName, int *peType, char **pzErr){
  struct Encryption {
    const char *zName;
    int eType;
  } a[] = {
    {"NONE", 0}, 
    {"AES_128_CTR", SQLITE_ZONEFILE_AES_128_CTR}, 
................................................................................
    }
  }

  *pzErr = sqlite3_mprintf("unknown encryption method: %s", zName);
  return SQLITE_ERROR;
}









static int zonefileGetParams(
  sqlite3_context *pCtx,          /* Leave any error message here */
  const char *zJson,              /* JSON configuration parameter (or NULL) */
  ZonefileParam *p                /* Populate this object before returning */
){
  sqlite3 *db = sqlite3_context_db_handle(pCtx);
  sqlite3_stmt *pStmt = 0;
................................................................................
  char *zErr = 0;
  int rc = SQLITE_OK;
  int rc2;

  memset(p, 0, sizeof(ZonefileParam));
  p->maxAutoFrameSize = ZONEFILE_DEFAULT_MAXAUTOFRAMESIZE;
  p->encryptionType = ZONEFILE_DEFAULT_ENCRYPTION;
  p->pCmpData = p->pCmpIdx = zonefileCompressByValue(0);

  rc = zonefilePrepare(db, &pStmt, &zErr,"SELECT key, value FROM json_each(?)");
  if( rc==SQLITE_OK ){
    sqlite3_bind_text(pStmt, 1, zJson, -1, SQLITE_STATIC);
  }
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
    const char *zKey = (const char*)sqlite3_column_text(pStmt, 0);
................................................................................
    }
    pBuf->a = aNew;
    pBuf->nAlloc = nNew;
  }
  return SQLITE_OK;
}




static void zonefileBufferFree(ZonefileBuffer *pBuf){
  sqlite3_free(pBuf->a);
  memset(pBuf, 0, sizeof(ZonefileBuffer));
}





static u64 zonefileGet64(u8 *aBuf){
  return (((u64)aBuf[0]) << 56)
       + (((u64)aBuf[1]) << 48)
       + (((u64)aBuf[2]) << 40)
       + (((u64)aBuf[3]) << 32) 
       + (((u64)aBuf[4]) << 24)
       + (((u64)aBuf[5]) << 16)
       + (((u64)aBuf[6]) <<  8)
       + (((u64)aBuf[7]) <<  0);
}





static void zonefileAppend32(ZonefileBuffer *pBuf, u32 v){
  zonefilePut32(&pBuf->a[pBuf->n], v);
  pBuf->n += 4;
}





static void zonefileAppend64(ZonefileBuffer *pBuf, u64 v){
  zonefileAppend32(pBuf, v>>32);
  zonefileAppend32(pBuf, v & 0xFFFFFFFF);
}





static void zonefileAppendBlob(ZonefileBuffer *pBuf, const u8 *p, int n){
  memcpy(&pBuf->a[pBuf->n], p, n);
  pBuf->n += n;
}






static int zonefileFileWrite(FILE *pFd, const u8 *aBuf, int nBuf){
  size_t res = fwrite(aBuf, 1, nBuf, pFd);
  return res!=(size_t)nBuf ? SQLITE_ERROR : SQLITE_OK;
}






static int zonefileFileRead(FILE *pFd, u8 *aBuf, int nBuf, i64 iOff){
  int rc = fseek(pFd, (long)iOff, SEEK_SET);
  if( rc==0 ){
    rc = fread(aBuf, 1, nBuf, pFd);
    rc = (rc==nBuf) ? SQLITE_OK : SQLITE_ERROR;


  }
  return rc;
}









static FILE *zonefileFileOpen(const char *zFile, int bWrite, char **pzErr){
  FILE *pFd = fopen(zFile, bWrite ? "wb" : "rb");
  if( pFd==0 ){
    *pzErr = sqlite3_mprintf("failed to open file \"%s\" for %s",
        zFile, bWrite ? "writing" : "reading"
    );
  }
  return pFd;
}




static void zonefileFileClose(FILE *pFd){
  if( pFd ) fclose(pFd);
}











static int zonefileAppendData(
  sqlite3_context *pCtx,          /* Leave any error message here */
  ZonefileCompress *pMethod,      /* Compression method object */
  void *pCmp,                     /* Compression handle */
  ZonefileCodec *pCodec,          /* Compression method, if any */
  ZonefileBuffer *pTo,            /* Append new data here */
  ZonefileBuffer *pFrom           /* Input buffer */
................................................................................
      if( zonefileFileWrite(pFd, buf, n) ) return SQLITE_ERROR;
      nRem -= n;
    }
  }
  return SQLITE_OK;
}





static int zonefileHexChar(char c){
  if( c>='0' && c<='9' ) return c-'0';
  c = c & ~0x20;
  if( c>='A' && c<='F' ) return c-('A'-10);
  return -1;
}















static int zonefileDecodeEncryptionKey(ZonefileParam *p, int *pn, char **pzErr){
  if( p->debugEncryptionKeyText==0 ){
    u8 *z = (u8*)p->encryptionKey;
    int n = *pn;
    int i;
    if( n&0x01 ) goto bad_format;
    for(i=0; i<n; i+=2){
................................................................................
  sqlite3_free(aSample);
  sqlite3_free(sParam.encryptionKey);
  if( rc==SQLITE_NOMEM ){
    sqlite3_result_error_nomem(pCtx);
  }
}




typedef struct ZonefileFilesTab ZonefileFilesTab;
struct ZonefileFilesTab {
  sqlite3_vtab base;              /* Base class - must be first */
  sqlite3 *db;
  char *zBase;                    /* Name of this table */
  char *zDb;                      /* Database containing this table */
  ZonefileGlobal *pGlobal;        /* Global object */
  sqlite3_stmt *pInsert;          /* Insert into the %_shadow_file table */
  sqlite3_stmt *pInsertIdx;       /* Insert into the %_shadow_idx table */
  sqlite3_stmt *pDeleteIdx;       /* Delete by fileid from %_shadow_idx table */
  sqlite3_stmt *pDelete;          /* Delete by rowid from %_shadow_file table */
};




typedef struct ZonefileFilesCsr ZonefileFilesCsr;
struct ZonefileFilesCsr {
  sqlite3_vtab_cursor base;       /* Base class - must be first */
  sqlite3_stmt *pSelect;
};

/*
................................................................................
** zonefile_files virtual table module xEof method.
*/
static int zffEof(sqlite3_vtab_cursor *cur){
  ZonefileFilesCsr *pCsr = (ZonefileFilesCsr*)cur;
  return pCsr->pSelect==0;
}





static void zonefileHeaderDeserialize(u8 *aBuf, ZonefileHeader *pHdr){
  pHdr->magicNumber = zonefileGet32(&aBuf[0]);
  pHdr->compressionTypeIndexData = aBuf[4];
  pHdr->compressionTypeContent = aBuf[5];
  pHdr->byteOffsetDictionary = zonefileGet32(&aBuf[6]);
  pHdr->byteOffsetFrames = zonefileGet32(&aBuf[10]);
  pHdr->numFrames = zonefileGet32(&aBuf[14]);
................................................................................
  pHdr->numKeys = zonefileGet32(&aBuf[18]);
  pHdr->encryptionType = aBuf[22];
  pHdr->encryptionKeyIdx = aBuf[23];
  pHdr->extendedHeaderVersion = aBuf[24];
  pHdr->extendedHeaderSize = aBuf[25];
}

static void zonefileJsonHeader(sqlite3_context *pCtx, const char *zFile){

  char *zErr = 0;
  FILE *pFd = zonefileFileOpen(zFile, 0, &zErr);
  if( pFd ){
    int rc;
    ZonefileHeader hdr = { 0 };










    u8 aBuf[ZONEFILE_SZ_HEADER];

    rc = zonefileFileRead(pFd, aBuf, ZONEFILE_SZ_HEADER, 0);
    if( rc==SQLITE_OK ){
      zonefileHeaderDeserialize(aBuf, &hdr);


    }


    if( rc!=SQLITE_OK ){
      zonefileCtxError(pCtx, "failed to read header from file: \"%s\"", zFile);






















    }else{




      char *zJson = sqlite3_mprintf("{"
          "\"magicNumber\":%u,"
          "\"compressionTypeIndexData\":%u,"
          "\"compressionTypeContent\":%u,"
          "\"byteOffsetDictionary\":%u,"
          "\"byteOffsetFrames\":%u,"
          "\"numFrames\":%u,"
          "\"numKeys\":%u,"
          "\"encryptionType\":%u,"
          "\"encryptionKeyIdx\":%u,"
          "\"extendedHeaderVersion\":%u,"
          "\"extendedHeaderSize\":%u}",
          (u32)hdr.magicNumber,
          (u32)hdr.compressionTypeIndexData,
          (u32)hdr.compressionTypeContent,
          (u32)hdr.byteOffsetDictionary,
          (u32)hdr.byteOffsetFrames,
          (u32)hdr.numFrames,
          (u32)hdr.numKeys,
          (u32)hdr.encryptionType,
          (u32)hdr.encryptionKeyIdx,
          (u32)hdr.extendedHeaderVersion,
          (u32)hdr.extendedHeaderSize
      );
      if( zJson ){
        sqlite3_result_text(pCtx, zJson, -1, SQLITE_TRANSIENT);
        sqlite3_free(zJson);
      }else{
        sqlite3_result_error_nomem(pCtx);
      }
    }
    fclose(pFd);
  }else{
    sqlite3_result_error(pCtx, zErr, -1);
    sqlite3_free(zErr);
  }


}

/* 
** zonefile_files virtual table module xColumn method.
*/
static int zffColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){
  ZonefileFilesCsr *pCsr = (ZonefileFilesCsr*)cur;
................................................................................
static int zffRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
  ZonefileFilesCsr *pCsr = (ZonefileFilesCsr*)cur;
  *pRowid = sqlite3_column_int64(pCsr->pSelect, 1);
  return SQLITE_OK;
}

/*
** Read and decode a Zonefile header from the start of the file opened
** by file-handle pFd. If successful, populate object (*pHdr) before
** returning SQLITE_OK. Otherwise, if an error occurs, set output
** parameter (*pzErr) to point to an English language error message and
** return an SQLite error code.
**
** It is the responsibility of the caller to eventually free any error
** message returned via (*pzErr) using sqlite3_free().



*/
static int zonefileReadHeader(
  FILE *pFd,                      /* File to read from */
  const char *zFile,              /* Name of file opened by pFd */
  ZonefileHeader *pHdr,           /* Populate this object before returning */
  char **pzErr                    /* OUT: Error message */
){
  u8 aBuf[ZONEFILE_SZ_HEADER];
  int rc = zonefileFileRead(pFd, aBuf, ZONEFILE_SZ_HEADER, 0);
  if( rc==SQLITE_OK ){
    zonefileHeaderDeserialize(aBuf, pHdr);
    if( pHdr->magicNumber!=ZONEFILE_MAGIC_NUMBER ){
      rc = SQLITE_ERROR;
    }
  }

  if( rc!=SQLITE_OK ){
    *pzErr = sqlite3_mprintf(
        "failed to read zonefile header from file \"%s\"", zFile
    );
  }

  return rc;
}

static int zonefileUncompress(
  ZonefileCompress *pMethod,
  void *pCmp,

  u8 *aIn, int nIn,
  u8 **paOut, int *pnOut
){
  int rc;
  int nOut = pMethod->xUncompressSize(pCmp, aIn, nIn);
  u8 *aOut = sqlite3_malloc(nOut);

  assert( pMethod->eType!=ZONEFILE_COMPRESSION_NONE );
  if( aOut==0 ){
................................................................................
  }

  *paOut = aOut;
  *pnOut = nOut;
  return rc;
}










static int zfFindCompress(int eType, ZonefileCompress **pp, char **pzErr){
  int rc = SQLITE_OK;
  ZonefileCompress *pCmp;
  pCmp = zonefileCompressByValue(eType);
  if( pCmp==0 ){
    *pzErr = sqlite3_mprintf("unsupported compression method: %d", eType);
    rc = SQLITE_ERROR;
  }else if( pCmp->eType==ZONEFILE_COMPRESSION_NONE ){
    pCmp = 0;
  }
  *pp = pCmp;
  return rc;
}














static int zonefileLoadIndex(
  ZonefileHeader *pHdr,
  FILE *pFd,

  u8 **paIdx, int *pnIdx,
  char **pzErr

){
  ZonefileCompress *pCmp = 0;
  int rc;
  u8 *aIdx = 0;
  int nIdx = 0;
    
  rc = zfFindCompress(pHdr->compressionTypeIndexData, &pCmp, pzErr);
  if( rc==SQLITE_OK ){
    if( pHdr->byteOffsetDictionary ){
      nIdx = pHdr->byteOffsetDictionary - ZONEFILE_SZ_HEADER;
    }else{
      nIdx = pHdr->byteOffsetFrames - ZONEFILE_SZ_HEADER;
    }
    nIdx -= pHdr->extendedHeaderSize;
................................................................................
    /* Read the zonefile header */
    if( rc==SQLITE_OK ){
      rc = zonefileReadHeader(pFd, zFile, &hdr, &zErr);
    }

    /* Find the compression method and open the compressor handle. */
    if( rc==SQLITE_OK ){
      rc = zfFindCompress(hdr.compressionTypeContent, &pCmpMethod, &zErr);
    }
    if( pCmpMethod ){
      int nDict = 0;
      u8 *aDict = 0;
      assert( rc==SQLITE_OK );
      if( hdr.byteOffsetDictionary ){
        nDict = hdr.byteOffsetFrames - hdr.byteOffsetDictionary;







|


|







 







>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>







 







>
>
>
>






>
>
>
>
>
>
>
>
>
>
>
>
>

|
|
|
|







 







>
>
>
>












>
>
>
>
>
>
>
>
>
>







 







>
>
>
>
>
>
>
>







 







|







 







>
>
>





>
>
>
>











>
>
>
>





>
>
>
>





>
>
>
>





>
>
>
>
>





>
>
>
>
>





>
>




>
>
>
>
>
>
>
>










>
>
>




>
>
>
>
>
>
>
>
>
>







 







>
>
>
>







>
>
>
>
>
>
>
>
>
>
>
>
>
>







 







>
>
>













>
>
>







 







>
>
>
>







 







<
>
|
|
|
|
|
>
>
>
>
>
>
>
>
>
>
|
<
|
|
|
>
>

|
>
|
<
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
>
>
>
>
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
<
<




>
>







 







|
|
|
|
|
<
|
|
>
>
>

<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<

|
<
>
|
|







 







>
>
>
>
>
>
>
>
>
|













>
>
>
>
>
>
>
>
>
>
>
>
>

|
<
>
|
<
>






|







 







|







330
331
332
333
334
335
336
337
338
339
340
341
342
343
344
345
346
347
...
689
690
691
692
693
694
695
696
697
698
699
700
701
702
703
704
705
706
707
708
709
710
...
714
715
716
717
718
719
720
721
722
723
724
725
726
727
728
729
730
731
732
733
734
...
794
795
796
797
798
799
800
801
802
803
804
805
806
807
808
809
810
811
812
813
814
815
816
817
818
819
820
821
822
823
824
825
826
827
828
829
830
831
832
833
834
835
...
871
872
873
874
875
876
877
878
879
880
881
882
883
884
885
886
887
888
889
890
891
892
893
894
895
896
897
898
899
900
901
902
903
904
905
906
907
908
909
910
...
922
923
924
925
926
927
928
929
930
931
932
933
934
935
936
937
938
939
940
941
942
943
...
944
945
946
947
948
949
950
951
952
953
954
955
956
957
958
....
1033
1034
1035
1036
1037
1038
1039
1040
1041
1042
1043
1044
1045
1046
1047
1048
1049
1050
1051
1052
1053
1054
1055
1056
1057
1058
1059
1060
1061
1062
1063
1064
1065
1066
1067
1068
1069
1070
1071
1072
1073
1074
1075
1076
1077
1078
1079
1080
1081
1082
1083
1084
1085
1086
1087
1088
1089
1090
1091
1092
1093
1094
1095
1096
1097
1098
1099
1100
1101
1102
1103
1104
1105
1106
1107
1108
1109
1110
1111
1112
1113
1114
1115
1116
1117
1118
1119
1120
1121
1122
1123
1124
1125
1126
1127
1128
1129
1130
1131
1132
1133
1134
1135
1136
1137
1138
1139
1140
1141
1142
1143
1144
1145
1146
1147
1148
1149
1150
1151
1152
1153
1154
1155
1156
1157
....
1197
1198
1199
1200
1201
1202
1203
1204
1205
1206
1207
1208
1209
1210
1211
1212
1213
1214
1215
1216
1217
1218
1219
1220
1221
1222
1223
1224
1225
1226
1227
1228
1229
1230
1231
1232
1233
1234
1235
....
1503
1504
1505
1506
1507
1508
1509
1510
1511
1512
1513
1514
1515
1516
1517
1518
1519
1520
1521
1522
1523
1524
1525
1526
1527
1528
1529
1530
1531
1532
1533
1534
1535
....
1701
1702
1703
1704
1705
1706
1707
1708
1709
1710
1711
1712
1713
1714
1715
1716
1717
1718
....
1719
1720
1721
1722
1723
1724
1725

1726
1727
1728
1729
1730
1731
1732
1733
1734
1735
1736
1737
1738
1739
1740
1741
1742

1743
1744
1745
1746
1747
1748
1749
1750
1751

1752
1753
1754
1755
1756
1757
1758
1759
1760
1761
1762
1763
1764
1765
1766
1767
1768
1769
1770
1771
1772
1773
1774
1775
1776
1777
1778
1779
1780
1781
1782
1783
1784
1785
1786
1787
1788
1789
1790
1791
1792
1793
1794
1795
1796
1797
1798
1799
1800
1801
1802
1803
1804
1805
1806
1807
1808


1809
1810
1811
1812
1813
1814
1815
1816
1817
1818
1819
1820
1821
....
1844
1845
1846
1847
1848
1849
1850
1851
1852
1853
1854
1855

1856
1857
1858
1859
1860
1861
























1862
1863

1864
1865
1866
1867
1868
1869
1870
1871
1872
1873
....
1881
1882
1883
1884
1885
1886
1887
1888
1889
1890
1891
1892
1893
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925

1926
1927

1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
....
2773
2774
2775
2776
2777
2778
2779
2780
2781
2782
2783
2784
2785
2786
2787

  return SQLITE_OK;
}

/*
** Search the key-store passed as the first argument for an encryption
** key to use with the file with file-id iFileid in zonefile table zTab
** in database zDb. If successful, set (*paKey) to point to the key
** buffer and return the size of the key in bytes.
**
** If no key is found, return 0. The final value of (*paKey) is undefined
** in this case.
*/
static int zonefileKeyFind(
  ZonefileGlobal *pGlobal,
  const char *zDb,                /* Database containing zonefile table */
  const char *zTab,               /* Name of zonefile table */
  i64 iFileid,                    /* File-id to configure key for */
................................................................................
    0,
    zfLz4CompressBound, zfLz4hcCompress, 
    zfGenericUncompressSize, zfLz4Uncompress
  },
#endif /* SQLITE_HAVE_LZ4 */
};

/*
** Find the ZonefileCompress object for the compression scheme named
** by zName. If successful, set output variable (*pp) to point to the
** object and return SQLITE_OK. Otherwise, return SQLITE_ERROR and
** leave output variable (*pzErr) pointing to an English language error
** message. It is the reponsibility of the caller to eventually free
** the error message buffer using sqlite3_free().
*/
static int zonefileCompress(
  const char *zName,              /* Name of requested compression method */
  ZonefileCompress **pp,          /* OUT: Pointer to compression object */
  char **pzErr                    /* OUT: Error message */
){
  int i;
  assert( *pzErr==0 );
................................................................................
      return SQLITE_OK;
    }
  }
  *pzErr = sqlite3_mprintf("unknown compression scheme: \"%s\"", zName);
  return SQLITE_ERROR;
}

/*
** Find a compression routine based on its associated integer constant
** (defined as part of the zonefile file format). If successful, return
** a pointer to the associated ZonefileCompression object. Or, if the
** nominated compression scheme is not supported in this build, return 
** NULL.
*/
static ZonefileCompress *zonefileCompressByValue(int eType){
  int i;
  for(i=0; i<sizeof(aZonefileCompress)/sizeof(aZonefileCompress[0]); i++){
    if( aZonefileCompress[i].eType==eType ){
      return &aZonefileCompress[i];
    }
  }
................................................................................
  va_start(ap, zFmt);
  zMsg = sqlite3_vmprintf(zFmt, ap);
  sqlite3_result_error(ctx, zMsg, -1);
  sqlite3_free(zMsg);
  va_end(ap);
}

/*
** Retrieve the error message from the database handle associated with
** context object pCtx and copy it into pCtx itself.
*/
static void zonefileTransferError(sqlite3_context *pCtx){
  sqlite3 *db = sqlite3_context_db_handle(pCtx);
  const char *zErr = sqlite3_errmsg(db);
  sqlite3_result_error(pCtx, zErr, -1);
}

/*
** Create an SQL statement from the printf() style format string passed
** as the 4th argument and any trailing arguments. Attempt to prepare
** it against database handle db. If successful, set output variable
** (*ppStmt) to point to the new statement handle and return SQLITE_OK.
** In this case it is the responsibility of the caller to eventually
** release the statement handle using sqlite3_finalize().
**
** Or, if an error occurs, return an SQLite error code and optionally
** set output parameter (*pzErr) to point to a buffer containing an
** English language error message. The caller must eventually free any
** such buffer using sqlite3_free().
*/
static int zonefilePrepare(
  sqlite3 *db,                    /* Database handle */
  sqlite3_stmt **ppStmt,          /* OUT: New statement handle */
  char **pzErr,                   /* OUT: Error message */
  const char *zFmt,               /* printf() style formatting string */
  ...
){
  int rc;
  va_list ap;
  char *zSql;
  va_start(ap, zFmt);
  zSql = sqlite3_vmprintf(zFmt, ap);
................................................................................
    default:
      assert( eType==SQLITE_NULL);
  }

  return 0;
}

/*
** Return true if the value object passed as the only argument contains
** integer value -1. False otherwise.
*/
int zonefileIsAutoFrame(sqlite3_value *pFrame){
  return (
      sqlite3_value_type(pFrame)==SQLITE_INTEGER 
   && sqlite3_value_int64(pFrame)==-1
  );
}

#define SQLITE_ZONEFILE_AES_128_CTR 1
#define SQLITE_ZONEFILE_AES_128_CBC 2
#define SQLITE_ZONEFILE_AES_256_CTR 3
#define SQLITE_ZONEFILE_AES_256_CBC 4

/*
** Argument zName is the name of an encryption scheme. If it is recognized
** output variable (*peType) is set to the corresponding integer constant
** (defined as part of the zonefile file format) and SQLITE_OK is returned.
** Or, if the named encryption method is not recognized, SQLITE_ERROR is
** returned and (*pzErr) set to point to a buffer containing an English
** language error message. In this case it is the responsibility of
** the caller to eventually free the error message buffer using a call
** to sqlite3_free().
*/
static int zonefileEncryption(const char *zName, int *peType, char **pzErr){
  struct Encryption {
    const char *zName;
    int eType;
  } a[] = {
    {"NONE", 0}, 
    {"AES_128_CTR", SQLITE_ZONEFILE_AES_128_CTR}, 
................................................................................
    }
  }

  *pzErr = sqlite3_mprintf("unknown encryption method: %s", zName);
  return SQLITE_ERROR;
}

/*
** Interpret the "json paramaters" argument passed to an invocation of
** the zonefile_write() SQL function. Populate the ZonefileParam indicated
** by the 3rd argument to this function with the results.
**
** If successful, return SQLITE_OK. Otherwise, return an SQLite error
** code and leave an error message in context object pCtx.
*/
static int zonefileGetParams(
  sqlite3_context *pCtx,          /* Leave any error message here */
  const char *zJson,              /* JSON configuration parameter (or NULL) */
  ZonefileParam *p                /* Populate this object before returning */
){
  sqlite3 *db = sqlite3_context_db_handle(pCtx);
  sqlite3_stmt *pStmt = 0;
................................................................................
  char *zErr = 0;
  int rc = SQLITE_OK;
  int rc2;

  memset(p, 0, sizeof(ZonefileParam));
  p->maxAutoFrameSize = ZONEFILE_DEFAULT_MAXAUTOFRAMESIZE;
  p->encryptionType = ZONEFILE_DEFAULT_ENCRYPTION;
  p->pCmpData = p->pCmpIdx = zonefileCompressByValue(ZONEFILE_COMPRESSION_NONE);

  rc = zonefilePrepare(db, &pStmt, &zErr,"SELECT key, value FROM json_each(?)");
  if( rc==SQLITE_OK ){
    sqlite3_bind_text(pStmt, 1, zJson, -1, SQLITE_STATIC);
  }
  while( rc==SQLITE_OK && SQLITE_ROW==sqlite3_step(pStmt) ){
    const char *zKey = (const char*)sqlite3_column_text(pStmt, 0);
................................................................................
    }
    pBuf->a = aNew;
    pBuf->nAlloc = nNew;
  }
  return SQLITE_OK;
}

/*
** Free the memory allocation associated with buffer pBuf.
*/
static void zonefileBufferFree(ZonefileBuffer *pBuf){
  sqlite3_free(pBuf->a);
  memset(pBuf, 0, sizeof(ZonefileBuffer));
}

/*
** Read and return a 64-bit unsigned integer in big-endian format from 
** buffer aBuf.
*/
static u64 zonefileGet64(u8 *aBuf){
  return (((u64)aBuf[0]) << 56)
       + (((u64)aBuf[1]) << 48)
       + (((u64)aBuf[2]) << 40)
       + (((u64)aBuf[3]) << 32) 
       + (((u64)aBuf[4]) << 24)
       + (((u64)aBuf[5]) << 16)
       + (((u64)aBuf[6]) <<  8)
       + (((u64)aBuf[7]) <<  0);
}

/*
** Append a 32-bit big-endian integer with value v to buffer pBuf. Space
** must have already been reserved in the buffer using zonefileBufferGrow().
*/
static void zonefileAppend32(ZonefileBuffer *pBuf, u32 v){
  zonefilePut32(&pBuf->a[pBuf->n], v);
  pBuf->n += 4;
}

/*
** Append a 64-bit big-endian integer with value v to buffer pBuf. Space
** must have already been reserved in the buffer using zonefileBufferGrow().
*/
static void zonefileAppend64(ZonefileBuffer *pBuf, u64 v){
  zonefileAppend32(pBuf, v>>32);
  zonefileAppend32(pBuf, v & 0xFFFFFFFF);
}

/*
** Append the n bytse of data in buffer p to buffer pBuf. Space must have
** already been reserved in the buffer using zonefileBufferGrow().
*/
static void zonefileAppendBlob(ZonefileBuffer *pBuf, const u8 *p, int n){
  memcpy(&pBuf->a[pBuf->n], p, n);
  pBuf->n += n;
}

/*
** Write nBuf bytes of data from buffer aBuf to the file opened by 
** file-handle pFd. Return SQLITE_OK if successful, or SQLITE_ERROR
** otherwise.
*/
static int zonefileFileWrite(FILE *pFd, const u8 *aBuf, int nBuf){
  size_t res = fwrite(aBuf, 1, nBuf, pFd);
  return res!=(size_t)nBuf ? SQLITE_ERROR : SQLITE_OK;
}

/*
** Read nBuf bytes of data from offset iOff of the file opened by 
** file-handle pFd into buffer aBuf. Return SQLITE_OK if successful, or
** SQLITE_ERROR otherwise.
*/
static int zonefileFileRead(FILE *pFd, u8 *aBuf, int nBuf, i64 iOff){
  int rc = fseek(pFd, (long)iOff, SEEK_SET);
  if( rc==0 ){
    rc = fread(aBuf, 1, nBuf, pFd);
    rc = (rc==nBuf) ? SQLITE_OK : SQLITE_ERROR;
  }else{
    rc = SQLITE_ERROR;
  }
  return rc;
}

/*
** Open the file identified by path zFile for writing (if bWrite==1) or
** reading (if bWrite==0). If successful, return a pointer to the new
** file-handle object. Otherwise, return NULL and set output variable
** (*pzErr) to point to a buffer containing an English language error
** message. It is the responsibility of the caller to eventually free
** any error message buffer using sqlite3_free().
*/
static FILE *zonefileFileOpen(const char *zFile, int bWrite, char **pzErr){
  FILE *pFd = fopen(zFile, bWrite ? "wb" : "rb");
  if( pFd==0 ){
    *pzErr = sqlite3_mprintf("failed to open file \"%s\" for %s",
        zFile, bWrite ? "writing" : "reading"
    );
  }
  return pFd;
}

/*
** Close the file handle passed as the only argument.
*/
static void zonefileFileClose(FILE *pFd){
  if( pFd ) fclose(pFd);
}

/*
** Append the contents of buffer pFrom to buffer pTo. If successful, return
** SQLITE_OK. Otherwise, return an SQLite error code and leave an error
** message in context object pCtx.
**
** If argument pMethod is not NULL, then it is used along with pCmp to 
** compress the data before appending it to pFrom. Similarly, if argument
** pCodec is not NULL, then it is used to encrypt the data before it is
** appended.
*/
static int zonefileAppendData(
  sqlite3_context *pCtx,          /* Leave any error message here */
  ZonefileCompress *pMethod,      /* Compression method object */
  void *pCmp,                     /* Compression handle */
  ZonefileCodec *pCodec,          /* Compression method, if any */
  ZonefileBuffer *pTo,            /* Append new data here */
  ZonefileBuffer *pFrom           /* Input buffer */
................................................................................
      if( zonefileFileWrite(pFd, buf, n) ) return SQLITE_ERROR;
      nRem -= n;
    }
  }
  return SQLITE_OK;
}

/*
** If character c is not a hexadecimal digit, return -1. Otherwise, return
** the value of the hex digit (a value between 0 and 15).
*/
static int zonefileHexChar(char c){
  if( c>='0' && c<='9' ) return c-'0';
  c = c & ~0x20;
  if( c>='A' && c<='F' ) return c-('A'-10);
  return -1;
}

/*
** String ZonefileParam.encryptionKey currently contains a string specified
** for the encryptionKey attribute of a JSON object passed to SQL function
** zonefile_write(). The string is (*pn) bytes in size.
**
** If the ZonefileParam.debugEncryptionKeyText flag is true this function
** is a no-op. Otherwise, an attempt is made to overwrite the hex string in
** ZonefileParam.encryptionKey with the corresponding binary data. If
** successful, SQLITE_OK is returned and (*pn) is set to the number of
** bytes in the binary key. Otherwise, if an error occurs, an SQLite error
** code is returned and (*pzErr) set to point to an English language error
** message. It is the responsibility of the caller to eventually free any
** error message buffer using sqlite3_free().
*/
static int zonefileDecodeEncryptionKey(ZonefileParam *p, int *pn, char **pzErr){
  if( p->debugEncryptionKeyText==0 ){
    u8 *z = (u8*)p->encryptionKey;
    int n = *pn;
    int i;
    if( n&0x01 ) goto bad_format;
    for(i=0; i<n; i+=2){
................................................................................
  sqlite3_free(aSample);
  sqlite3_free(sParam.encryptionKey);
  if( rc==SQLITE_NOMEM ){
    sqlite3_result_error_nomem(pCtx);
  }
}

/*
** Virtual table type for zonefile_files virtual tables.
*/
typedef struct ZonefileFilesTab ZonefileFilesTab;
struct ZonefileFilesTab {
  sqlite3_vtab base;              /* Base class - must be first */
  sqlite3 *db;
  char *zBase;                    /* Name of this table */
  char *zDb;                      /* Database containing this table */
  ZonefileGlobal *pGlobal;        /* Global object */
  sqlite3_stmt *pInsert;          /* Insert into the %_shadow_file table */
  sqlite3_stmt *pInsertIdx;       /* Insert into the %_shadow_idx table */
  sqlite3_stmt *pDeleteIdx;       /* Delete by fileid from %_shadow_idx table */
  sqlite3_stmt *pDelete;          /* Delete by rowid from %_shadow_file table */
};

/*
** Virtual table cursor type for zonefile_files virtual tables.
*/
typedef struct ZonefileFilesCsr ZonefileFilesCsr;
struct ZonefileFilesCsr {
  sqlite3_vtab_cursor base;       /* Base class - must be first */
  sqlite3_stmt *pSelect;
};

/*
................................................................................
** zonefile_files virtual table module xEof method.
*/
static int zffEof(sqlite3_vtab_cursor *cur){
  ZonefileFilesCsr *pCsr = (ZonefileFilesCsr*)cur;
  return pCsr->pSelect==0;
}

/*
** Deserialize the ZONEFILE_SZ_HEADER byte zonefile header in the
** buffer. Populate (*pHdr) with the results.
*/
static void zonefileHeaderDeserialize(u8 *aBuf, ZonefileHeader *pHdr){
  pHdr->magicNumber = zonefileGet32(&aBuf[0]);
  pHdr->compressionTypeIndexData = aBuf[4];
  pHdr->compressionTypeContent = aBuf[5];
  pHdr->byteOffsetDictionary = zonefileGet32(&aBuf[6]);
  pHdr->byteOffsetFrames = zonefileGet32(&aBuf[10]);
  pHdr->numFrames = zonefileGet32(&aBuf[14]);
................................................................................
  pHdr->numKeys = zonefileGet32(&aBuf[18]);
  pHdr->encryptionType = aBuf[22];
  pHdr->encryptionKeyIdx = aBuf[23];
  pHdr->extendedHeaderVersion = aBuf[24];
  pHdr->extendedHeaderSize = aBuf[25];
}


/*
** Read and decode a Zonefile header from the start of the file opened
** by file-handle pFd. If successful, populate object (*pHdr) before
** returning SQLITE_OK. Otherwise, if an error occurs, set output
** parameter (*pzErr) to point to an English language error message and
** return an SQLite error code.
**
** It is the responsibility of the caller to eventually free any error
** message returned via (*pzErr) using sqlite3_free().
*/
static int zonefileReadHeader(
  FILE *pFd,                      /* File to read from */
  const char *zFile,              /* Name of file opened by pFd */
  ZonefileHeader *pHdr,           /* Populate this object before returning */
  char **pzErr                    /* OUT: Error message */
){
  u8 aBuf[ZONEFILE_SZ_HEADER];

  int rc = zonefileFileRead(pFd, aBuf, ZONEFILE_SZ_HEADER, 0);
  if( rc==SQLITE_OK ){
    zonefileHeaderDeserialize(aBuf, pHdr);
    if( pHdr->magicNumber!=ZONEFILE_MAGIC_NUMBER ){
      rc = SQLITE_ERROR;
    }
  }

  if( rc!=SQLITE_OK ){

    *pzErr = sqlite3_mprintf(
        "failed to read zonefile header from file \"%s\"", zFile
    );
  }

  return rc;
}

/*
** Read the zonefile header from file zFile and set the result of pCtx
** to a JSON object that represents the contents. Or, if an error occurs,
** leave an error message in pCtx. This function is called whenever the
** "header" column of a zonefile_files virtual table is requested.
*/
static void zonefileJsonHeader(sqlite3_context *pCtx, const char *zFile){
  char *zErr = 0;
  int rc = SQLITE_OK;
  ZonefileHeader hdr = {0};

  FILE *pFd = zonefileFileOpen(zFile, 0, &zErr);
  if( pFd ){
    rc = zonefileReadHeader(pFd, zFile, &hdr, &zErr);
  }else{
    rc = SQLITE_ERROR;
  }

  if( rc==SQLITE_OK ){
    char *zJson = sqlite3_mprintf("{"
        "\"magicNumber\":%u,"
        "\"compressionTypeIndexData\":%u,"
        "\"compressionTypeContent\":%u,"
        "\"byteOffsetDictionary\":%u,"
        "\"byteOffsetFrames\":%u,"
        "\"numFrames\":%u,"
        "\"numKeys\":%u,"
        "\"encryptionType\":%u,"
        "\"encryptionKeyIdx\":%u,"
        "\"extendedHeaderVersion\":%u,"
        "\"extendedHeaderSize\":%u}",
        (u32)hdr.magicNumber,
        (u32)hdr.compressionTypeIndexData,
        (u32)hdr.compressionTypeContent,
        (u32)hdr.byteOffsetDictionary,
        (u32)hdr.byteOffsetFrames,
        (u32)hdr.numFrames,
        (u32)hdr.numKeys,
        (u32)hdr.encryptionType,
        (u32)hdr.encryptionKeyIdx,
        (u32)hdr.extendedHeaderVersion,
        (u32)hdr.extendedHeaderSize
          );
    if( zJson ){
      sqlite3_result_text(pCtx, zJson, -1, SQLITE_TRANSIENT);
      sqlite3_free(zJson);
    }else{
      sqlite3_result_error_nomem(pCtx);
    }


  }else{
    sqlite3_result_error(pCtx, zErr, -1);
    sqlite3_free(zErr);
  }

  zonefileFileClose(pFd);
}

/* 
** zonefile_files virtual table module xColumn method.
*/
static int zffColumn(sqlite3_vtab_cursor *cur, sqlite3_context *ctx, int i){
  ZonefileFilesCsr *pCsr = (ZonefileFilesCsr*)cur;
................................................................................
static int zffRowid(sqlite3_vtab_cursor *cur, sqlite_int64 *pRowid){
  ZonefileFilesCsr *pCsr = (ZonefileFilesCsr*)cur;
  *pRowid = sqlite3_column_int64(pCsr->pSelect, 1);
  return SQLITE_OK;
}

/*
** Uncompress buffer aIn/nIn using the compression routines pMethod
** and the compressor instance pCmp into space obtained from 
** sqlite3_malloc(). If successful, return SQLITE_OK and set output
** parameters (*paOut) and (*pnOut) to point to the allocated buffer
** and its size in bytes respectively. In this case it is the 

** responsibility of the caller to eventually free buffer (*paOut)
** using sqlite3_free().
**
** Or, if an error occurs, set both output parameters to zero and
** return an SQLite error code.
*/
























static int zonefileUncompress(
  ZonefileCompress *pMethod,      /* Compression routines */

  void *pCmp,                     /* Compressor instance */
  u8 *aIn, int nIn,               /* Input buffer */
  u8 **paOut, int *pnOut          /* Output buffer */
){
  int rc;
  int nOut = pMethod->xUncompressSize(pCmp, aIn, nIn);
  u8 *aOut = sqlite3_malloc(nOut);

  assert( pMethod->eType!=ZONEFILE_COMPRESSION_NONE );
  if( aOut==0 ){
................................................................................
  }

  *paOut = aOut;
  *pnOut = nOut;
  return rc;
}

/*
** Attempt to find the compression methods object for the compression method
** identified by integer constant eType (defined as part of the zonefile file
** format). If successful, set output parameter (*pp) to point to the object
** and return SQLITE_OK. Otherwise, return an SQLite error code and set
** (*pzErr) to point to a buffer containing an English language error
** message. It is the responsibility of the caller to eventually free any
** error message buffer using sqlite3_free().
*/
static int zonefileFindCompress(int eType, ZonefileCompress **pp, char **pzErr){
  int rc = SQLITE_OK;
  ZonefileCompress *pCmp;
  pCmp = zonefileCompressByValue(eType);
  if( pCmp==0 ){
    *pzErr = sqlite3_mprintf("unsupported compression method: %d", eType);
    rc = SQLITE_ERROR;
  }else if( pCmp->eType==ZONEFILE_COMPRESSION_NONE ){
    pCmp = 0;
  }
  *pp = pCmp;
  return rc;
}

/*
** Argument pHdr points to a deserialized zonefile header read from the
** zonefile opened by file handle pFd. This function attempts to read
** the entire zonefile-index structure into space obtained from 
** sqlite3_malloc(). If successful, it sets output parameters (*paIdx)
** and (*pnIdx) to point to the buffer and its size in bytes respectively
** before returning SQLITE_OK.
**
** Otherwise, if an error occurs, an SQLite error code is returned and
** output parameter (*pzErr) may be set to point to a buffer containing an
** English language error message. It is the responsibility of the caller to
** eventually free any error message buffer using sqlite3_free().
*/
static int zonefileLoadIndex(
  ZonefileHeader *pHdr,           /* Deserialized header read from file */

  FILE *pFd,                      /* File to read from */
  u8 **paIdx, int *pnIdx,         /* OUT: Buffer containing zonefile index */

  char **pzErr                    /* OUT: Error message */
){
  ZonefileCompress *pCmp = 0;
  int rc;
  u8 *aIdx = 0;
  int nIdx = 0;
    
  rc = zonefileFindCompress(pHdr->compressionTypeIndexData, &pCmp, pzErr);
  if( rc==SQLITE_OK ){
    if( pHdr->byteOffsetDictionary ){
      nIdx = pHdr->byteOffsetDictionary - ZONEFILE_SZ_HEADER;
    }else{
      nIdx = pHdr->byteOffsetFrames - ZONEFILE_SZ_HEADER;
    }
    nIdx -= pHdr->extendedHeaderSize;
................................................................................
    /* Read the zonefile header */
    if( rc==SQLITE_OK ){
      rc = zonefileReadHeader(pFd, zFile, &hdr, &zErr);
    }

    /* Find the compression method and open the compressor handle. */
    if( rc==SQLITE_OK ){
      rc = zonefileFindCompress(hdr.compressionTypeContent, &pCmpMethod, &zErr);
    }
    if( pCmpMethod ){
      int nDict = 0;
      u8 *aDict = 0;
      assert( rc==SQLITE_OK );
      if( hdr.byteOffsetDictionary ){
        nDict = hdr.byteOffsetFrames - hdr.byteOffsetDictionary;

Changes to ext/zonefile/zonefile1.test.

246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
  UPDATE cc_files SET ekey = '0123456789';
  SELECT quote(dd.v)==quote(cc.v) FROM cc JOIN dd USING (k)
} {1 1 1}

close [open test.zonefile w+]
do_catchsql_test 3.4 {
  SELECT header FROM cc_files
} {1 {failed to read header from file: "test.zonefile"}}

forcedelete test.zonefile
do_catchsql_test 3.5 {
  SELECT header FROM cc_files
} {1 {failed to open file "test.zonefile" for reading}}

do_execsql_test 3.6 {







|







246
247
248
249
250
251
252
253
254
255
256
257
258
259
260
  UPDATE cc_files SET ekey = '0123456789';
  SELECT quote(dd.v)==quote(cc.v) FROM cc JOIN dd USING (k)
} {1 1 1}

close [open test.zonefile w+]
do_catchsql_test 3.4 {
  SELECT header FROM cc_files
} {1 {failed to read zonefile header from file "test.zonefile"}}

forcedelete test.zonefile
do_catchsql_test 3.5 {
  SELECT header FROM cc_files
} {1 {failed to open file "test.zonefile" for reading}}

do_execsql_test 3.6 {