FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

ast: tagged, snake_case JSON output for parse and analyze by kyleconroy · Pull Request #4592 · sqlc-dev/sqlc · GitHub

/ sqlc Public

ast: tagged, snake_case JSON output for parse and analyze - #4592

Merged
kyleconroy merged 4 commits into
mainfrom
claude/ast-json-node-tag-yv4tsn
Aug 28, 2026
Merged

ast: tagged, snake_case JSON output for parse and analyze#4592
kyleconroy merged 4 commits into
mainfrom
claude/ast-json-node-tag-yv4tsn

Conversation

kyleconroy commented Aug 28, 2026
edited
Loading

Copy link
Copy Markdown
Collaborator

Reworks the JSON that sqlc parse and sqlc analyze --ast print, in three commits. Replaces #4591, which got the same tagging from a hand-written reflective encoder; this version gets it from stock encoding/json with no custom marshaller in the tree walk.

Tag every node with its type

ast.Node is an interface, so the JSON encoding carried no record of which node a given object was. Nodes with no fields of their own — A_Star, Null, TODO — all encoded as {}, and an empty List was indistinguishable from them. On INSERT ... RETURNING *, the star and four untranslated clauses printed as the same empty object.

Every node struct now declares a zero-sized marker as its first field:

type SelectStmt struct {
	Tag NodeTag[SelectStmt] `json:"tag"`
	...
}

encoding/json calls MarshalJSON on the field, and the generic type parameter carries the node's identity to it at compile time — no encoder walks the tree, plain json.Marshal of any node is tagged. The zero value is valid, so the engines' struct literals are untouched, and the field is zero-sized, so nodes cost the same memory.

The marker's UnmarshalJSON accepts only the containing node's name, so a document decoded into the wrong node type is an error (node tagged "RangeVar" decoded into SortBy) instead of a silently misfielded tree.

tag as the key avoids every collision: encoding/json matches field names case-insensitively on decode, so kind would capture A_Expr.Kind, type the eight nodes with a Type field, and node would silently capture SortBy.Node. Because the tag is a real field rather than a key injected beside the fields, nothing can shadow it.

Omit absent fields

25% of the keys in typical output were null and another 9% were empty containers. Pointer, slice, interface and map fields now carry ,omitempty; scalars deliberately don't, because zero is a value the parser can find: stmt_location is 0 for the first statement in a file, and LIMIT 0 parses to an ival of 0.

This also drops empty (not just nil) lists, a small normalization: the dolphin converter builds Items as an empty slice where the postgresql one leaves it nil, so the same SQL printed different JSON per engine.

snake_case field names

Every node field gets an explicit json tag with the snake_case form of its name: stmt_location, from_clause, relname. The keys read as JSON rather than Go leaking through, match the names libpg_query uses for the same fields in its own JSON output, and match analyze's existing columns/params keys. Field renames become a compile-time-visible decision about the JSON rather than a silent output change.

Sample, for INSERT ... RETURNING *:

"returning_list": {
  "tag": "List",
  "items": [
    {
      "tag": "ResTarget",
      "val": {
        "tag": "ColumnRef",
        "fields": { "tag": "List", "items": [ { "tag": "A_Star" } ] },
        "location": 93
      },
      "location": 93
    }
  ]
}

Notes

  • The codemod touched all 255 node structs (the set with pointer-receiver Pos()), correctly excluding the 76 non-struct Node implementers (enum types with Pos methods) and the fmt/comment structs. 702 reference-typed fields carry omitempty; 528 scalar fields carry a name-only tag; no two fields in any struct collapse to the same snake key.
  • docs/howto/parse.md already notes the JSON shape is beta; it gains sections on node tags, absent fields, and the naming convention. TODO tags are documented as "parsed but not represented in the AST", not "absent from the query".
  • Validation against ast: tag node types and omit absent fields in JSON output #4591: the first two commits were diffed golden-by-golden against the reflective encoder's output — byte-identical except the mysql empty-list normalization called out above.

Testing

Covered end to end: go test --tags=examples -timeout 20m ./... passes with PostgreSQL and MySQL running, plus go vet and gofmt. Eight goldens regenerated (seven parse_basic, analyze_ast/postgresql); the other 26 parse/analyze cases regenerate byte-identical, confirming nothing outside AST output moved.

claude added 4 commits August 28, 2026 03:42
Node is an interface, so the JSON that parse and analyze --ast print carried
no record of which node a given object was. Nodes with no fields of their own
all encoded as "{}": a star in a RETURNING clause was indistinguishable from
an untranslated clause, and an empty List was indistinguishable from both.

Every node struct now declares a zero-sized marker as its first field:

	Tag NodeTag[T] `json:"tag"`

with T the node's own type. encoding/json calls MarshalJSON on the field, and
the generic type parameter carries the node's identity to it at compile time,
so plain json.Marshal of any node emits its type name with no custom encoder
walking the tree. The zero value is valid, so the engines' struct literals
are unchanged, and the field is zero-sized, so nodes cost the same memory.

The field's UnmarshalJSON accepts only the containing node's name, making a
document decoded into the wrong node type an error instead of a silently
misfielded tree. A test checks that every node struct declares the field,
that its type parameter names the containing struct (the one mistake the
compiler cannot catch, since a copy-pasted NodeTag[OtherNode] compiles), and
that no node declares a colliding second field.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T9LCt1mwdY3mE14x3Zz5Vw
Most of what parse printed was absent fields. On a four-query file, 25% of
the keys were null and another 9% were empty containers, so reading an AST
meant scanning past clauses the statement never had.

Mark every pointer, slice, interface and map field of the node structs with
json:",omitempty". Scalars keep no such tag on purpose: StmtLocation is 0 for
the first statement in a file and LIMIT 0 parses to an Ival of 0, so omitting
zero-valued scalars would lose what the parser found rather than what it did
not. Field names are unchanged.

Beyond nil fields this also drops empty lists, which is a small normalization:
the dolphin converter builds Items as an empty slice where the postgresql one
leaves it nil, so the same SQL printed different JSON per engine. Both now
print as a bare tagged List.

This depends on the type tags. Omitting a nil Items turns an empty List into
"{}", which without a tag would be indistinguishable from A_Star and TODO.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T9LCt1mwdY3mE14x3Zz5Vw
Give every node field an explicit json tag with the snake_case form of its
name: StmtLocation prints as stmt_location, FromClause as from_clause. The
keys now read as JSON rather than as Go leaking through, and they match the
names libpg_query uses for the same fields in its own JSON output, so anyone
coming from pg_query's tree finds the fields where they expect them.

The tags also stop the output from being coupled to the Go field names: a
field rename is now a compile-time-visible decision about the JSON, not a
silent output change.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T9LCt1mwdY3mE14x3Zz5Vw
The source-level test checked that every node struct declares the Tag marker
with the right type parameter. Drop it: the end-to-end goldens pin the output
that matters, and the declarations are one convention, not behavior.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01T9LCt1mwdY3mE14x3Zz5Vw
kyleconroy merged commit dd1c96b into main Aug 28, 2026
12 checks passed
kyleconroy deleted the claude/ast-json-node-tag-yv4tsn branch August 28, 2026 04:49
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants


Back | FazBrowse Home | New Git URL