Documentation Source Text

Check-in [85bc3983a5]
Login

Many hyperlinks are disabled.
Use anonymous login to enable hyperlinks.

Overview
Comment:Fix typos in the lemon documentation.
Downloads: Tarball | ZIP archive
Timelines: family | ancestors | descendants | both | trunk
Files: files | file ages | folders
SHA3-256: 85bc3983a530a9b643f0cf20c384a878a6af7b51d23b0636f9ad0008a6f0f550
User & Date: drh 2019-10-16 18:07:04.283
Context
2019-10-18
02:16
Fix a typo in the "quirks.html" document. (check-in: 7dce3c2f8a user: drh tags: trunk)
2019-10-16
18:07
Fix typos in the lemon documentation. (check-in: 85bc3983a5 user: drh tags: trunk)
2019-10-14
00:26
Further clarification of automatic affinity conversion rules. (check-in: 3a3335a61e user: drh tags: trunk)
Changes
Unified Diff Ignore Whitespace Patch
Changes to pages/lemon.in.
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
     [https://sqlite.org/src/file/tool/lempar.c|lempar.c] → A template
     for the generated parser C-code.  The "lemon" utility program reads this
     template and inserts additional code in order to generate a parser.
</ul>

<h1>Advantages of Lemon</h1>

<p>Lemon generates an LALR(1) parser.  It's operation is similar to the
more familiar tools [https://en.wikipedia.org/wiki/Yacc|Yacc] and
[https://en.wikipedia.org/wiki/GNU_bison|Bison], but Lemon adds important
improvements, including:

<ul>
<li><p>
     The grammar syntax is less error prone - using symbolic names for







|







29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
     [https://sqlite.org/src/file/tool/lempar.c|lempar.c] &rarr; A template
     for the generated parser C-code.  The "lemon" utility program reads this
     template and inserts additional code in order to generate a parser.
</ul>

<h1>Advantages of Lemon</h1>

<p>Lemon generates an LALR(1) parser.  Its operation is similar to the
more familiar tools [https://en.wikipedia.org/wiki/Yacc|Yacc] and
[https://en.wikipedia.org/wiki/GNU_bison|Bison], but Lemon adds important
improvements, including:

<ul>
<li><p>
     The grammar syntax is less error prone - using symbolic names for
63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
<p>Lemon is used in two places in SQLite.

<p>The primary use of Lemon is to create the SQL language parser.
A grammar file ([https://sqlite.org/src/file/src/parse.y|parse.y]) is
compiled by Lemon into parse.c and parse.h.  The parse.c file is
incorporated into the [amalgamation] without further modification.

<p>Lemon is also used to generate parse for the query pattern
expressions in the [FTS5] extension.  In this case, the input grammar
file is [https://sqlite.org/src/file/ext/fts5/fts5parse.y|fts5parse.y].

<h2>Lemon Customizations Especially For SQLite</h2>

<p>One of the advantages of hosting code generator tools as part of
the project is that the tools can be optimized to serve specific needs of
the overall project.  Lemon has benefited from this effect. Over the years,
the Lemon parser generator has been extended and enhanced to provide
new capabilities and improved performance to SQLite.  A few of the
specific enhancements to Lemon that are specifically designed for use
by SQLite include:

<ul>
<li><p>
Lemon has the concept of a "fallback" tokens.
The SQL language contains a large number of keywords and these keywords
have the potential to collide with identifier names.
Lemon has the ability to designate some keywords has being able to
"fallback" to an identifier.  If the keyword appears in the input token
stream in a context that would otherwise be a syntax error, the token
is automatically transformed into its fallback before the syntax error
is raised.  This feature allows the parser to be very forgiving of







|















|







63
64
65
66
67
68
69
70
71
72
73
74
75
76
77
78
79
80
81
82
83
84
85
86
87
88
89
90
91
92
93
<p>Lemon is used in two places in SQLite.

<p>The primary use of Lemon is to create the SQL language parser.
A grammar file ([https://sqlite.org/src/file/src/parse.y|parse.y]) is
compiled by Lemon into parse.c and parse.h.  The parse.c file is
incorporated into the [amalgamation] without further modification.

<p>Lemon is also used to generate the parser for the query pattern
expressions in the [FTS5] extension.  In this case, the input grammar
file is [https://sqlite.org/src/file/ext/fts5/fts5parse.y|fts5parse.y].

<h2>Lemon Customizations Especially For SQLite</h2>

<p>One of the advantages of hosting code generator tools as part of
the project is that the tools can be optimized to serve specific needs of
the overall project.  Lemon has benefited from this effect. Over the years,
the Lemon parser generator has been extended and enhanced to provide
new capabilities and improved performance to SQLite.  A few of the
specific enhancements to Lemon that are specifically designed for use
by SQLite include:

<ul>
<li><p>
Lemon has the concept of a "fallback" token.
The SQL language contains a large number of keywords and these keywords
have the potential to collide with identifier names.
Lemon has the ability to designate some keywords has being able to
"fallback" to an identifier.  If the keyword appears in the input token
stream in a context that would otherwise be a syntax error, the token
is automatically transformed into its fallback before the syntax error
is raised.  This feature allows the parser to be very forgiving of
115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
</ul>

<p>The parsing of SQL statements is a significant consumer of CPU cycles 
in any SQL database engine.  On-going efforts to optimize SQLite have caused
the developers to spend a lot of time tweaking Lemon to generate faster
parsers.  These efforts have benefited all users of the Lemon parser generator,
not just SQLite.  But if Lemon had been a separately maintained tool, it
would have been more difficulty to make coordinated changes to both SQLite
and Lemon, and as a result not as much optimization would have been
accomplished.  Hence, the fact that the parser generator tool is included
in the source tree for SQLite has turned out to be a net benefit for both
the tool itself and for SQLite.

<h1>History Of Lemon</h1>

<p>Lemon was original written by D. Richard Hipp (also the creator of SQLite)
while he was in graduate school at Duke University between 1987 and 1992.
The original creation date of Lemon has been lost, but was probably sometime
around 1990.  Lemon generates an LALR(1) parser.  There was companion 
LL(1) parser generator tool named "Lime", but the source code for Lime
has been lost.

<p>The Lemon source code was originally written as separate source files,
and only later merged into a single "lemon.c" source file.

<p>The author of Lemon and SQLite (Hipp) reports that his C programming
skills were greatly enhanced by studying John Ousterhout's original
source code to Tcl.  Hipp discovered and studied Tcl in 1993.  Lemon
was written before then, and SQLite afterwards.  There is a clear
difference in the coding styles of these two products, with SQLite seeming
to be cleaner, more readable, and easier to maintain.







|







|


|












115
116
117
118
119
120
121
122
123
124
125
126
127
128
129
130
131
132
133
134
135
136
137
138
139
140
141
142
143
144
145
</ul>

<p>The parsing of SQL statements is a significant consumer of CPU cycles 
in any SQL database engine.  On-going efforts to optimize SQLite have caused
the developers to spend a lot of time tweaking Lemon to generate faster
parsers.  These efforts have benefited all users of the Lemon parser generator,
not just SQLite.  But if Lemon had been a separately maintained tool, it
would have been more difficult to make coordinated changes to both SQLite
and Lemon, and as a result not as much optimization would have been
accomplished.  Hence, the fact that the parser generator tool is included
in the source tree for SQLite has turned out to be a net benefit for both
the tool itself and for SQLite.

<h1>History Of Lemon</h1>

<p>Lemon was originally written by D. Richard Hipp (also the creator of SQLite)
while he was in graduate school at Duke University between 1987 and 1992.
The original creation date of Lemon has been lost, but was probably sometime
around 1990.  Lemon generates an LALR(1) parser.  There was a companion 
LL(1) parser generator tool named "Lime", but the source code for Lime
has been lost.

<p>The Lemon source code was originally written as separate source files,
and only later merged into a single "lemon.c" source file.

<p>The author of Lemon and SQLite (Hipp) reports that his C programming
skills were greatly enhanced by studying John Ousterhout's original
source code to Tcl.  Hipp discovered and studied Tcl in 1993.  Lemon
was written before then, and SQLite afterwards.  There is a clear
difference in the coding styles of these two products, with SQLite seeming
to be cleaner, more readable, and easier to maintain.