Skip to content

Preserve SQL comments adjacent to operators - #916

Open
MbappeWU wants to merge 1 commit into
andialbrecht:masterfrom
MbappeWU:fix/comments-after-operators
Open

MbappeWU wants to merge 1 commit into
andialbrecht:masterfrom
MbappeWU:fix/comments-after-operators

Conversation

@MbappeWU

@MbappeWU MbappeWU commented Oct 3, 2026 •

Copy link
Copy Markdown

Fixes #722.

An operator immediately followed by a comment can consume the start of that comment. Valid SQL such as SELECT 'a' ||--comment;\n'b'; SELECT 2; then splits into three statements instead of two, and strip_comments=True leaves the comment in place. The same boundary problem affects # and /* ... */, including semicolons inside block comments.

Stop operator runs before existing comment openers, and prevent the JSON #- pattern from taking the first dash of #--. Preserve the standalone unterminated /* fallback and established hint, quoted-string and normal operator behavior. Related historical PR #878 was closed unmerged by its author; this patch independently covers the block-comment and JSON-prefix cases as well.

Validation:

  • Full suite: 529 passed, 2 xfailed, 1 xpassed. The original suite has 506 passed with the same xfail/xpass counts. The final tests against the original module produce 19 failures and 510 passes, with the same 2 xfailed and 1 xpassed.
  • Full pinned Ruff check passes. Coverage is 97% using the original project settings.
  • Seven original test jobs pass on Linux Python 3.10–3.14 and macOS/Windows Python 3.11; each platform reports 529 passed, 2 xfailed, 1 xpassed. CodeQL also succeeds. Two normal fork pushes produced no runs, so the owned-fork snapshot adds only manual dispatch events to the two workflows. All original jobs, matrices, steps, permissions and triggers are identical. CI head a2cd648d0e0328cc795528c8d5897ebbf28cd086 differs from PR head only in those two event additions. The PR workflows are unchanged.
  • The optional Codecov upload failed because a token is required for this fork/manual event. Coverage generation succeeded; upload success is not claimed, and the workflow behavior was not weakened.
  • Fresh wheel/sdist contain the exact committed bytes of all 21 runtime modules. Source and installed wheel match on four comment cases, three hint cases and eight unchanged non-comment controls. Eight source/installed CLI runs match the public API. Dash and block scripts split correctly and execute in SQLite.

Developed with Codex, with separate local code and input-behavior reviews. This PR changes only the lexer rules and their regression tests.

@MbappeWU
MbappeWU marked this pull request as ready for review October 3, 2026 20:04
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

strip_comments does not strip when "|--" is found.

1 participant