Documentation Source Text

Check-in [1c981267ee]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Enhancements to the althttpd.c SCGI mechanism: Added the "fallback:" and "relight:" lines to the *.scgi specification file format.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 1c981267ee08a50c452dd4eaefc4b393a06dc7cd4bce47c4ded7cc61c1374182
User & Date: drh 2019-02-16 15:41:35.554
Context
2019-02-16
16:42
Extra defensive coding in althttpd.c. (check-in: 2ae41f4427 user: drh tags: trunk)
15:41
Enhancements to the althttpd.c SCGI mechanism: Added the "fallback:" and "relight:" lines to the *.scgi specification file format. (check-in: 1c981267ee user: drh tags: trunk)
2019-02-15
21:21
Send the SCGI environment variable with a value of "1" on SCGI requests. (check-in: 216ba4ebe1 user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to misc/althttpd.c.
38
39
40
41
42
43
44
45

46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64


65
66
67
68
69
70
71
**        RFC-5785 to allow letsencrypt or certbot to generate a TSL cert
**        using webroot.
**
**    (4) Characters other than [0-9a-zA-Z,-./:_~] and any %HH characters
**        escapes in the filename are all translated into "_".  This is
**        a defense against cross-site scripting attacks and other mischief.
**
**    (5) Executable files are run as CGI.  Files whose name ends with ".scgi"****        trigger and SCGI request (see item 10 below).  All other files

**        are delivered as is.
**
**    (6) For SSL support use stunnel and add the -https 1 option on the
**        httpd command-line.
**
**    (7) If a file named "-auth" exists in the same directory as the file to
**        be run as CGI or to be delivered, then it contains information
**        for HTTP Basic authorization.  See file format details below.
**
**    (8) To run as a stand-alone server, simply add the "-port N" command-line
**        option to define which TCP port to listen on.
**
**    (9) For static content, the mimetype is determined by the file suffix
**        using a table built into the source code below.  If you have
**        unusual content files, you might need to extend this table.
**
**   (10) Content files that end with ".scgi" and that contain text of the
**        form "SCGI hostname port" will format an SCGI request and send it
**        to hostname:port, the relay back the reply.


**
** Command-line Options:
**
**  --root DIR       Defines the directory that contains the various
**                   $HOST.website subdirectories, each containing web content 
**                   for a single virtual host.  If launched as root and if
**                   "--user USER" also appears on the command-line and if







|
>


















|
>
>







38
39
40
41
42
43
44
45
46
47
48
49
50
51
52
53
54
55
56
57
58
59
60
61
62
63
64
65
66
67
68
69
70
71
72
73
74
**        RFC-5785 to allow letsencrypt or certbot to generate a TSL cert
**        using webroot.
**
**    (4) Characters other than [0-9a-zA-Z,-./:_~] and any %HH characters
**        escapes in the filename are all translated into "_".  This is
**        a defense against cross-site scripting attacks and other mischief.
**
**    (5) Executable files are run as CGI.  Files whose name ends with ".scgi"
**        trigger and SCGI request (see item 10 below).  All other files
**        are delivered as is.
**
**    (6) For SSL support use stunnel and add the -https 1 option on the
**        httpd command-line.
**
**    (7) If a file named "-auth" exists in the same directory as the file to
**        be run as CGI or to be delivered, then it contains information
**        for HTTP Basic authorization.  See file format details below.
**
**    (8) To run as a stand-alone server, simply add the "-port N" command-line
**        option to define which TCP port to listen on.
**
**    (9) For static content, the mimetype is determined by the file suffix
**        using a table built into the source code below.  If you have
**        unusual content files, you might need to extend this table.
**
**   (10) Content files that end with ".scgi" and that contain text of the
**        form "SCGI hostname port" will format an SCGI request and send it
**        to hostname:port, the relay back the reply.  Error behavior is
**        determined by subsequent lines of the .scgi file.  See SCGI below
**        for details.
**
** Command-line Options:
**
**  --root DIR       Defines the directory that contains the various
**                   $HOST.website subdirectories, each containing web content 
**                   for a single virtual host.  If launched as root and if
**                   "--user USER" also appears on the command-line and if
149
150
151
152
153
154
155















































156
157
158
159
160
161
162
**      character of the file or subdirectory name "-" or ".".
**
** (8)  The request URI must begin with "/" or else a 404 error is generated.
**
** (9)  This program never sets the value of an environment variable to a
**      string that begins with "() {".
**















































**
** Basic Authorization:
**
** If the file "-auth" exists in the same directory as the content file
** (for both static content and CGI) then it contains the information used
** for basic authorization.  The file format is as follows:
**







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







152
153
154
155
156
157
158
159
160
161
162
163
164
165
166
167
168
169
170
171
172
173
174
175
176
177
178
179
180
181
182
183
184
185
186
187
188
189
190
191
192
193
194
195
196
197
198
199
200
201
202
203
204
205
206
207
208
209
210
211
212
**      character of the file or subdirectory name "-" or ".".
**
** (8)  The request URI must begin with "/" or else a 404 error is generated.
**
** (9)  This program never sets the value of an environment variable to a
**      string that begins with "() {".
**
** Security Auditing:
**
** This webserver mostly only serves static content.  Any security risk will
** come from CGI and SCGI.  To check an installation for security, then, it
** makes sense to focus on the CGI and SCGI scripts.
**
** To local all CGI files:
**
**          find *.website -executable -type f -print
**     OR:  find *.website -perm +0111 -type f -print
**
** The first form of the "find" command is preferred, but is only supported
** by GNU find.  On a Mac, you'll have to use the second form.
**
** To find all SCGI files:
**
**          find *.website -name '*.scgi' -type f -print
**
** If any file is a security concern, it can be disabled on a live
** installation by turning off read permissions:
**
**          chmod 0000 file-of-concern
**
** SCGI Specification Files:
**
** Content files (files without the execute bit set) that end with ".scgi"
** specify a connection to an SCGI server.  The format of the .scgi file
** follows this template:
**
**      SCGI hostname port
**      fallback: fallback-filename
**      relight: relight-command
**
** The first line specifies the location and TCP/IP port of the SCGI server
** that will handle the request.  Subsequent lines determine what to do if
** the SCGI server cannot be contacted.  If the "relight:" line is present,
** then the relight-command is run using system() and the connection is
** retried after a 1-second delay.  Use "&" at the end of the relight-command
** to run it in the background.  Make sure the relight-command does not
** send generate output, or that output will become part of the SCGI reply.
** Add a ">/dev/null" suffix (before the "&") to the relight-command if
** necessary to suppress output.  If there is no relight-command, or if the
** relight is attempted but the SCGI server still cannot be contacted, then
** the content of the fallback-filename file is returned as a substitute for
** the SCGI request.  The mimetype is determined by the suffix on the
** fallback-filename.  The fallback-filename would typically be an error
** message indicating that the service is temporarily unavailable.
**
** Basic Authorization:
**
** If the file "-auth" exists in the same directory as the content file
** (for both static content and CGI) then it contains the information used
** for basic authorization.  The file format is as follows:
**
1179
1180
1181
1182
1183
1184
1185

































































1186
1187
1188
1189
1190
1191
1192
** Count the number of "/" characters in a string.
*/
static int countSlashes(const char *z){
  int n = 0;
  while( *z ) if( *(z++)=='/' ) n++;
  return n;
}


































































/*
** A CGI or SCGI script has run and is sending its reply back across
** the channel "in".  Process this reply into an appropriate HTTP reply.
** Close the "in" channel when done.
*/
static void CgiHandleReply(FILE *in){







>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>







1229
1230
1231
1232
1233
1234
1235
1236
1237
1238
1239
1240
1241
1242
1243
1244
1245
1246
1247
1248
1249
1250
1251
1252
1253
1254
1255
1256
1257
1258
1259
1260
1261
1262
1263
1264
1265
1266
1267
1268
1269
1270
1271
1272
1273
1274
1275
1276
1277
1278
1279
1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296
1297
1298
1299
1300
1301
1302
1303
1304
1305
1306
1307
** Count the number of "/" characters in a string.
*/
static int countSlashes(const char *z){
  int n = 0;
  while( *z ) if( *(z++)=='/' ) n++;
  return n;
}

/*
** Send the text of the file named by zFile as the reply.  Use the
** suffix on the end of the zFile name to determine the mimetype.
**
** Return 1 to omit making a log entry for the reply.
*/
static int SendFile(
  const char *zFile,      /* Name of the file to send */
  int lenFile,            /* Length of the zFile name in bytes */
  struct stat *pStat      /* Result of a stat() against zFile */
){
  const char *zContentType;
  int c;
  time_t t;
  FILE *in;
  char zETag[100];

  zContentType = GetMimeType(zFile, lenFile);
  if( zTmpNam ) unlink(zTmpNam);
  sprintf(zETag, "m%xs%x", (int)pStat->st_mtime, (int)pStat->st_size);
  if( CompareEtags(zIfNoneMatch,zETag)==0
   || (zIfModifiedSince!=0
        && (t = ParseRfc822Date(zIfModifiedSince))>0
        && t>=pStat->st_mtime)
  ){
    StartResponse("304 Not Modified");
    nOut += DateTag("Last-Modified", pStat->st_mtime);
    nOut += printf("Cache-Control: max-age=%d\r\n", mxAge);
    nOut += printf("ETag: \"%s\"\r\n", zETag);
    nOut += printf("\r\n");
    fflush(stdout);
    MakeLogEntry(0, 470);  /* LOG: ETag Cache Hit */
    return 1;
  }
  in = fopen(zFile,"rb");
  if( in==0 ) NotFound(480); /* LOG: fopen() failed for static content */
  StartResponse("200 OK");
  nOut += DateTag("Last-Modified", pStat->st_mtime);
  nOut += printf("Cache-Control: max-age=%d\r\n", mxAge);
  nOut += printf("ETag: \"%s\"\r\n", zETag);
  nOut += printf("Content-type: %s\r\n",zContentType);
  nOut += printf("Content-length: %d\r\n\r\n",(int)pStat->st_size);
  fflush(stdout);
  if( strcmp(zMethod,"HEAD")==0 ){
    MakeLogEntry(0, 2); /* LOG: Normal HEAD reply */
    fclose(in);
    fflush(stdout);
    return 1;
  }
  if( useTimeout ) alarm(30 + pStat->st_size/1000);
#ifdef linux
  {
    off_t offset = 0;
    nOut += sendfile(fileno(stdout), fileno(in), &offset, pStat->st_size);
  }
#else
  while( (c = getc(in))!=EOF ){
    putc(c,stdout);
    nOut++;
  }
#endif
  fclose(in);
  return 0;
}

/*
** A CGI or SCGI script has run and is sending its reply back across
** the channel "in".  Process this reply into an appropriate HTTP reply.
** Close the "in" channel when done.
*/
static void CgiHandleReply(FILE *in){
1263
1264
1265
1266
1267
1268
1269


1270
1271
1272
1273
1274
1275
1276
1277
1278
1279

1280
1281
1282
1283
1284
1285
1286
1287
1288
1289
1290
1291
1292
1293
1294
1295
1296



















1297
1298
1299
1300
1301
1302
1303
1304
1305

1306
1307
1308
1309
1310
1311

1312































1313
1314
1315
1316
1317
1318

1319
1320
1321
1322
1323
1324
1325
*/
static void SendScgiRequest(const char *zFile, const char *zScript){
  FILE *in;
  FILE *s;
  char *z;
  char *zHost;
  char *zPort = 0;


  int rc;
  int iSocket = -1;
  struct addrinfo hints;
  struct addrinfo *ai = 0;
  struct addrinfo *p;
  char *zHdr;
  size_t nHdr = 0;
  size_t nHdrAlloc;
  int i;
  char zLine[1000];

  in = fopen(zFile, "rb");
  if( in==0 ){
    Malfunction(700, "cannot open \"%s\"\n", zFile);
  }
  if( fgets(zLine, sizeof(zLine)-1, in)==0 ){
    Malfunction(701, "cannot read \"%s\"\n", zFile);
  }
  fclose(in);
  if( strncmp(zLine,"SCGI ",5)!=0 ){
    Malfunction(702, "misformatted SCGI spec \"%s\"\n", zFile);
  }
  z = zLine+5;
  zHost = GetFirstElement(z,&z);
  zPort = GetFirstElement(z,0);
  if( zHost==0 || zHost[0]==0 || zPort==0 || zPort[0]==0 ){
    Malfunction(703, "misformatted SCGI spec \"%s\"\n", zFile);
  }



















  memset(&hints, 0, sizeof(struct addrinfo));
  hints.ai_family = AF_UNSPEC;
  hints.ai_socktype = SOCK_STREAM;
  hints.ai_protocol = IPPROTO_TCP;
  rc = getaddrinfo(zHost,zPort,&hints,&ai);
  if( rc ){
    Malfunction(704, "cannot resolve SCGI server name %s:%s\n%s\n",
                zHost, zPort, gai_strerror(rc));
  }

  for(p=ai; p; p=p->ai_next){
    iSocket = socket(p->ai_family, p->ai_socktype, p->ai_protocol);
    if( iSocket<0 ) continue;
    if( connect(iSocket,p->ai_addr,p->ai_addrlen)>=0 ) break;
    close(iSocket);
  }

  if( iSocket<0 ){































    Malfunction(705, "cannot open socket to SCGI server %s\n",
                zScript);
  }
  s = fdopen(iSocket, "r+");
  if( s==0 ){
    Malfunction(706, "could not turn the socket into a FILE\n");

  }

  nHdrAlloc = 0;
  zHdr = 0;
  if( zContentLength==0 ) zContentLength = "0";
  zScgi = "1";
  for(i=0; i<(int)(sizeof(cgienv)/sizeof(cgienv[0])); i++){







>
>










>







<









>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>









>
|
|
|
|
|
|
>
|
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
>
|
|
|
<
<
<
>







1378
1379
1380
1381
1382
1383
1384
1385
1386
1387
1388
1389
1390
1391
1392
1393
1394
1395
1396
1397
1398
1399
1400
1401
1402
1403
1404

1405
1406
1407
1408
1409
1410
1411
1412
1413
1414
1415
1416
1417
1418
1419
1420
1421
1422
1423
1424
1425
1426
1427
1428
1429
1430
1431
1432
1433
1434
1435
1436
1437
1438
1439
1440
1441
1442
1443
1444
1445
1446
1447
1448
1449
1450
1451
1452
1453
1454
1455
1456
1457
1458
1459
1460
1461
1462
1463
1464
1465
1466
1467
1468
1469
1470
1471
1472
1473
1474
1475
1476
1477
1478
1479
1480
1481
1482
1483
1484



1485
1486
1487
1488
1489
1490
1491
1492
*/
static void SendScgiRequest(const char *zFile, const char *zScript){
  FILE *in;
  FILE *s;
  char *z;
  char *zHost;
  char *zPort = 0;
  char *zRelight = 0;
  char *zFallback = 0;
  int rc;
  int iSocket = -1;
  struct addrinfo hints;
  struct addrinfo *ai = 0;
  struct addrinfo *p;
  char *zHdr;
  size_t nHdr = 0;
  size_t nHdrAlloc;
  int i;
  char zLine[1000];
  char zExtra[1000];
  in = fopen(zFile, "rb");
  if( in==0 ){
    Malfunction(700, "cannot open \"%s\"\n", zFile);
  }
  if( fgets(zLine, sizeof(zLine)-1, in)==0 ){
    Malfunction(701, "cannot read \"%s\"\n", zFile);
  }

  if( strncmp(zLine,"SCGI ",5)!=0 ){
    Malfunction(702, "misformatted SCGI spec \"%s\"\n", zFile);
  }
  z = zLine+5;
  zHost = GetFirstElement(z,&z);
  zPort = GetFirstElement(z,0);
  if( zHost==0 || zHost[0]==0 || zPort==0 || zPort[0]==0 ){
    Malfunction(703, "misformatted SCGI spec \"%s\"\n", zFile);
  }
  while( fgets(zExtra, sizeof(zExtra)-1, in) ){
    char *zCmd = GetFirstElement(zExtra,&z);
    if( zCmd==0 ) continue;
    if( zCmd[0]=='#' ) continue;
    RemoveNewline(z);
    if( strcmp(zCmd, "relight:")==0 ){
      free(zRelight);
      zRelight = StrDup(z);
      continue;
    }
    if( strcmp(zCmd, "fallback:")==0 ){
      free(zFallback);
      zFallback = StrDup(z);
      continue;
    }
    Malfunction(704, "unrecognized line in SCGI spec: \"%s %s\"\n",
                zCmd, z ? z : "");
  }
  fclose(in);
  memset(&hints, 0, sizeof(struct addrinfo));
  hints.ai_family = AF_UNSPEC;
  hints.ai_socktype = SOCK_STREAM;
  hints.ai_protocol = IPPROTO_TCP;
  rc = getaddrinfo(zHost,zPort,&hints,&ai);
  if( rc ){
    Malfunction(704, "cannot resolve SCGI server name %s:%s\n%s\n",
                zHost, zPort, gai_strerror(rc));
  }
  while(1){  /* Exit via break */
    for(p=ai; p; p=p->ai_next){
      iSocket = socket(p->ai_family, p->ai_socktype, p->ai_protocol);
      if( iSocket<0 ) continue;
      if( connect(iSocket,p->ai_addr,p->ai_addrlen)>=0 ) break;
      close(iSocket);
    }
    if( iSocket<0 || (s = fdopen(iSocket,"r+"))==0 ){
      if( iSocket>=0 ) close(iSocket);
      if( zRelight ){
        rc = system(zRelight);
        if( rc ){
          Malfunction(721,"Relight failed with %d: \"%s\"\n",
                      rc, zRelight);
        }
        free(zRelight);
        zRelight = 0;
        sleep(1);
        continue;
      }
      if( zFallback ){
        struct stat statbuf;
        int rc;
        memset(&statbuf, 0, sizeof(statbuf));
        if( chdir(zDir) ){
          char zBuf[1000];
          Malfunction(720, /* LOG: chdir() failed */
               "cannot chdir to [%s] from [%s]", 
               zDir, getcwd(zBuf,999));
        }
        rc = stat(zFallback, &statbuf);
        if( rc==0 && S_ISREG(statbuf.st_mode) && access(zFallback,R_OK)==0 ){
          closeConnection = 1;
          rc = SendFile(zFallback, (int)strlen(zFallback), &statbuf);
          free(zFallback);
          exit(0);
        }else{
          Malfunction(706, "bad fallback file: \"%s\"\n", zFallback);
        }
      }
      Malfunction(707, "cannot open socket to SCGI server %s\n",
                  zScript);
    }



    break;
  }

  nHdrAlloc = 0;
  zHdr = 0;
  if( zContentLength==0 ) zContentLength = "0";
  zScgi = "1";
  for(i=0; i<(int)(sizeof(cgienv)/sizeof(cgienv[0])); i++){
1894
1895
1896
1897
1898
1899
1900
1901
1902
1903
1904
1905
1906
1907
1908
1909
1910
1911
1912
1913
1914
1915
1916
1917
1918
1919
1920
1921
1922
1923
1924
1925
1926
1927
1928
1929
1930
1931
1932
1933
1934
1935
1936
1937
1938
1939
1940
1941
1942
1943
1944
1945
1946
1947
1948
1949
1950
1951
1952
1953
1954
1955
    /* If the request URI for static content contains material past the
    ** actual content file name, report that as a 404 error. */
    NotFound(460); /* LOG: Excess URI content past static file name */
  }else{
    /* If it isn't executable then it
    ** must a simple file that needs to be copied to output.
    */
    const char *zContentType = GetMimeType(zFile, lenFile);
    time_t t;
    char zETag[100];

    if( zTmpNam ) unlink(zTmpNam);
    sprintf(zETag, "m%xs%x", (int)statbuf.st_mtime, (int)statbuf.st_size);
    if( CompareEtags(zIfNoneMatch,zETag)==0
     || (zIfModifiedSince!=0
          && (t = ParseRfc822Date(zIfModifiedSince))>0
          && t>=statbuf.st_mtime)
    ){
      StartResponse("304 Not Modified");
      nOut += DateTag("Last-Modified", statbuf.st_mtime);
      nOut += printf("Cache-Control: max-age=%d\r\n", mxAge);
      nOut += printf("ETag: \"%s\"\r\n", zETag);
      nOut += printf("\r\n");
      fflush(stdout);
      MakeLogEntry(0, 470);  /* LOG: ETag Cache Hit */
      return;
    }
    in = fopen(zFile,"rb");
    if( in==0 ) NotFound(480); /* LOG: fopen() failed for static content */
    StartResponse("200 OK");
    nOut += DateTag("Last-Modified", statbuf.st_mtime);
    nOut += printf("Cache-Control: max-age=%d\r\n", mxAge);
    nOut += printf("ETag: \"%s\"\r\n", zETag);
    nOut += printf("Content-type: %s\r\n",zContentType);
    nOut += printf("Content-length: %d\r\n\r\n",(int)statbuf.st_size);
    fflush(stdout);
    if( strcmp(zMethod,"HEAD")==0 ){
      MakeLogEntry(0, 2); /* LOG: Normal HEAD reply */
      fclose(in);
      fflush(stdout);
      return;
    }
    if( useTimeout ) alarm(30 + statbuf.st_size/1000);
#ifdef linux
    {
      off_t offset = 0;
      nOut += sendfile(fileno(stdout), fileno(in), &offset, statbuf.st_size);
    }
#else
    while( (c = getc(in))!=EOF ){
      putc(c,stdout);
      nOut++;
    }
#endif
    fclose(in);
  }
  fflush(stdout);
  MakeLogEntry(0, 0);  /* LOG: Normal reply */

  /* The next request must arrive within 30 seconds or we close the connection
  */
  omitLog = 1;







<
<
<
|
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<
<







2061
2062
2063
2064
2065
2066
2067



2068












































2069
2070
2071
2072
2073
2074
2075
    /* If the request URI for static content contains material past the
    ** actual content file name, report that as a 404 error. */
    NotFound(460); /* LOG: Excess URI content past static file name */
  }else{
    /* If it isn't executable then it
    ** must a simple file that needs to be copied to output.
    */



    if( SendFile(zFile, lenFile, &statbuf) ) return;












































  }
  fflush(stdout);
  MakeLogEntry(0, 0);  /* LOG: Normal reply */

  /* The next request must arrive within 30 seconds or we close the connection
  */
  omitLog = 1;