| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [View Raw Code] [Original HTTPS Page] |
meyer is a hand-written, zero-dependency Go parser for the SQLite SQL dialect. Read PLAN.md first — it defines the architecture, the AST shape, and what is deliberately out of scope. The grammar ground truth is SQLite's parse.y (Lemon) and tokenize.c; the dialect surface is https://sqlite.org/lang.html.
Each parser/testdata/<name>.test holds cases extracted from SQLite's test/<name>.test TCL script; there is one for every script in the pinned source tree that yields at least one literal-SQL block. For every case, the raw oracle results (one line per statement: prepared OK, or the exact error message, offset and parse tail) are stored; the harness derives the expectation from them:
Known looseness: messages produced by grammar actions (e.g. ORDER BY clause should come after UNION not before) are currently classified as semantic, so meyer is permitted to accept such statements. The pattern list lives in internal/testfile (syntaxFamily) and can be extended without regenerating the corpus, because the corpus stores raw oracle output.
Accept/reject conformance cannot see a dropped clause or a mis-associated operator, so two further checks stand in for the parse-tree goldens SQLite cannot produce:
The corpus is whatever SQLite's test suite happens to contain, which is overwhelmingly valid SQL: fewer than three hundred of its 20,971 cases are rejections, so error fidelity is barely exercised by it. cmd/difftest covers that gap by mutating corpus SQL — truncate, delete, duplicate, replace or insert one token — and checking that meyer and a live SQLite build still agree on the verdict, the message and the offset.
go run ./cmd/difftest # sweep the whole corpus
go run ./cmd/difftest -files select1,expr # a few files
go run ./cmd/difftest -per 60 -seed 7 # dig harder, reproduciblyIt needs the oracle, so it is not part of go test ./...; a separate workflow (.github/workflows/difftest.yml) runs it weekly and on demand. What it finds belongs in parser/errors_test.go, whose expectations are taken from the oracle rather than written by hand.
Two situations it deliberately skips, because the harness cannot compare them rather than because meyer might be wrong: a ; inside parentheses, where sqlite3_complete splits statements differently from a real parse, and a semantic failure on a statement's last token, which leaves pzTail at the end of the input with nothing to mark the parse as abandoned.
go run ./cmd/next-test # pick the next todo case to implement
# ... implement in lexer/, ast/, parser/ ...
go test ./parser -run TestParse -check-parse -v 2>&1 | grep "PARSE PASSES NOW"
go test ./... # everything else still green
# commit code + metadata.json changes together-check-parse re-runs todo cases and deletes metadata entries for cases that now pass. Never mark a case done by hand, and never "fix" a failing case by editing the corpus — if you believe a corpus expectation is wrong, the fix is in cmd/regenerate-parse (or the classification in internal/testfile), followed by a full regeneration and a reviewed diff.
go run ./cmd/regenerate-parse # every script in the pinned tree
go run ./cmd/regenerate-parse -files select1 # one fileThe tool downloads the pinned SQLite release (version + SHA-256 constants in cmd/regenerate-parse/main.go) into .sqlite/ (gitignored), compiles the oracle with the system C compiler, and rewrites corpus + metadata (new cases start as todo; existing case states are preserved). Advancing the pin means updating the constants and parser/testdata/README.md, regenerating everything, and reviewing the diff.
| Back | FazBrowse Home | New Git URL |