fix(parser): support PostgreSQL # binary operator (bitwise XOR / geometric intersection) - #2507
Merged
manticore-projects merged 1 commit intoAug 25, 2026
Conversation
Copy link
Copy Markdown
Contributor
Sorry, something went wrong.
…etric intersection) A lone '#' now lexes as a binary operator instead of a one-character identifier. In PostgreSQL '#' is the integer bitwise exclusive OR (docs Table 9.4) and the geometric intersection of lseg / line / box (Table 9.36), see JSQLParser#2499. Identifier lexing of '#' is untouched by longest match: SQL Server #temp / ##global (JSQLParser#1197) and the JSON operators #> / #>> (JSQLParser#1695) lex as before. A lone '#' can no longer be an identifier: bare column / table / alias names previously parsed silently and now fail loudly; quoted forms ("#", `#`) still parse. The AST node is the dedicated Intersects operator (sibling of GeometryDistance), wired into ExpressionVisitor, the adapter, the deparser, TablesNamesFinder and the validator. Signed-off-by: Fu Dian <fudianchn@gmail.com>
Copy link
Copy Markdown
Contributor
|
Nice and clean! Thank you for providing this, its definitely a win. |
Sorry, something went wrong.
manticore-projects
pushed a commit
that referenced
this pull request
Aug 26, 2026
…nts (#2508) * feat(parser): support MySQL # line comments behind allowHashLineComments The second step agreed in #2502: with Feature.allowHashLineComments (default off) a `#` runs to end of line as a comment, unconditional like MySQL itself (no blank needed, `42#24` is a comment too); with the flag off a lone `#` stays the binary operator introduced in #2507, so neither reading silently replaces the other. Mechanics: under the flag SimpleCharStream rewrites a token-start `#` in the buffer to a character no other lexical rule starts with, so the dedicated HASH_LINE_COMMENT production wins the match for every `#` form while identifier and JSON-operator lexing of the default mode stay untouched (rewriting the buffer keeps the matcher's backup / re-read arithmetic intact, and GetImage() restores the `#` in the token image). Unquoted identifiers (and @@variables) end at their first `#` via their token actions, which re-lex the remainder as the comment. Quoted forms ("#", `#`, "a#b") keep their `#` in both modes. Under the flag the statement semantics are MySQL's: `SELECT #temp FROM t` comments out the rest of the line and fails, quoted "#temp" still parses. Closes #2499, supersedes #2502. Signed-off-by: Fu Dian <fudianchn@gmail.com> * docs(parser): add regeneration warning to SimpleCharStream header The file is maintained by hand on top of the JavaCC template and carries the in-buffer rewrite of a leading # in BeginToken(), which Feature.allowHashLineComments depends on. Per review on #2508. --------- Signed-off-by: Fu Dian <fudianchn@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
AI disclosure: this change was prepared with AI coding agents, reviewed and revised line by line by me.
What
Support the PostgreSQL # binary operator, the first step agreed in #2502 (see #2499):
All currently fail with a ParseException.
Why / Root cause
# is a legal identifier start and part character, so a lone # lexes as a
one-character identifier and never reaches an operator production.
How
A dedicated S_HASH_OPERATOR token declared before S_IDENTIFIER (it wins
the length tie on a lone #) plus one case pair in prattArithRest
(precedence 5, with + - &, the existing mapping of PostgreSQL's
"any other operator" tier). Longest match keeps every other # lexing
untouched: #temp, ##global, #$tab1#, a#b stay identifiers
(#1197, #1695), #> / #>> JSON operators are unchanged.
The AST node is the dedicated Intersects operator, sibling of
GeometryDistance, wired into ExpressionVisitor, the adapter, the
deparser, TablesNamesFinder and the validator.
One behavior change: a lone # can no longer be an identifier. Bare
column / table / alias names written as a single # (SELECT # FROM t,
SELECT 1 FROM #, SELECT a AS # FROM t, SELECT 1 #, SELECT t.# FROM t,
INSERT INTO # VALUES (1)) previously parsed silently and now fail loudly
as an operator without operands; quoted forms ("#", `#`) still parse.
Follow-up to #2502: with this in place, the MySQL # line comment can be
attached behind a feature switch by routing the token into a special token
per lexer state (the SPECIAL_TOKEN vs TOKEN point).
Testing
PostgresTest: 2 new tests. The operator assertions were verified failing
on master; the guard test pins the identifier (incl. Oracle #$tab1#) and
JSON operator families and fails when # is dropped from the identifier
start set. Full suite green (4926 tests).
Performance
gradle jmh, JSQLParserBenchmark.parseSQLStatements on performance.sql,
version=latest, 10 forks × 10 iterations (100 samples) on a 32-core host,
interleaved master/branch × 2:
Δ within each window ≤ 0.6% with overlapping 99.9% CIs → no regression.