Skip to content

feat(BREAKING): replace sqlformat with our own SQL formatter - #14

Merged
dsherret merged 6 commits into
mainfrom
feat/own-sql-formatter
Oct 18, 2026
Merged

dsherret merged 6 commits into
mainfrom
feat/own-sql-formatter

Conversation

@dsherret

Copy link
Copy Markdown
Member

Replaces the sqlformat-rs wrapper with our own SQL formatter: a dialect-aware lexer, a tolerant parser and dprint-core's IR, following what was done in dprint-plugin-toml and dprint-plugin-markdown.

Closes #3
Closes #12

What it does

  • Dialects: generic (the default), postgresql, mysql, sqlserver, sqlite, oracle, bigquery and snowflake, chosen with the dialect option. The dialect decides how comments, quotes, strings, variables and operators are read.
  • Procedural code and stored procedures:
    • BEGIN/END blocks and exception handlers
    • IF/ELSIF, T-SQL IF/WHILE, loops, CASE statements and labels
    • declaration sections, cursors, Oracle packages and type bodies
    • PL/pgSQL $$ bodies, which are formatted as SQL too
    • MySQL DELIMITER, T-SQL GO batches, SQL*Plus / and its commands, psql meta-commands
  • lineWidth: lists, conditions and brackets break only when they don't fit (Expose a config for maxInlineBlock / VALUES tuple wrapping #12).
  • Set operations: EXCEPT and MINUS go on their own line like UNION and INTERSECT (set operations inconsistently formatted #3).
  • Templates: dbt/Jinja tags ({{ }}, {% %}, {# #}) are kept as they are.
  • Range formatting.
  • Safety check: after formatting, the output is lexed again and compared with the input. The tokens must be the same, apart from keyword casing when casing changes it, and so must the line breaks that matter (between joined strings, and template tags in place). If anything else changed, the file is left as it is and an error is reported, so a formatter bug can't silently change code.

Breaking changes

  • The output differs from sqlformat's.
  • The global lineWidth (default 120) now affects SQL output. It was ignored before.
  • linesBetweenQueries is now a minimum: existing blank lines between statements are kept.
  • format_text returns a Result. Dialect and FormatError are #[non_exhaustive].
  • The daily workflow that published a release for each new sqlformat version is removed, since there's nothing to follow anymore.

All existing options are still accepted, including the deprecated uppercase.

Testing

  • Specs and tests: about 500 spec cases across every dialect and option, plus integration tests for errors, CRLF, ignore comments and performance.
  • Every dialect: every spec input is also formatted in all 8 dialects at widths 40 and 120. Each result is checked for stability, and for only whitespace changing.
  • Real-world SQL: about 3,100 files from the sqlfluff fixtures and the PostgreSQL, Oracle and SQL Server corpora, formatted in all 8 dialects at widths 30 and 120. No output changed on a second pass, nothing was refused because of a bug, and nothing crashed.
  • Fuzzing: a multi-byte character is inserted at every position of every spec input, in every dialect, about 960k checks. No crashes and no unstable output.
  • CI: now also runs clippy, cargo fmt --check and the tracing feature's tests.

Review

Four reviews (lexer, parser, generator, packaging) were done before opening this, and their findings are fixed in the follow-up commits:

  • Snowflake stage paths, BigQuery dashed project names, # in MySQL variables, unicode whitespace
  • statement splitting in T-SQL without semicolons
  • SQL*Plus commands
  • template tags around commas
  • PostgreSQL joined strings
  • exponential time on deeply nested CASEs and subqueries
  • instability with a comment after a clause keyword

Known follow-ups, none of which change code:

  • M1: when a single-action ALTER TABLE ... ADD COLUMN is too long, its type's brackets get broken (varchar(\n 255\n)). It should get the ALTER TABLE x\n ADD ... layout instead.
  • M2: some lines are left past the width: long || chains, a trigger's EXECUTE FUNCTION, and GRANT ... TO a, b, c.
  • M3: types are cased inconsistently with casing: upper: CAST(a AS text) but a::TEXT.
  • M4: a few procedural constructs are laid out poorly. They are still verified and stable:
    • PL/pgSQL #variable_conflict
    • Oracle compound triggers and RETURN SELF AS RESULT
    • END LOOP label, and T-SQL END CONVERSATION
    • a comment between a label and its loop
    • a MERGE inside an exception handler that has a comment before its WHEN
  • M5: in a T-SQL routine without BEGIN, linesBetweenQueries: 2 puts a blank line after AS.
  • L1: {% endif %} at the end of a list loses its indentation.
  • L2: /* dprint-ignore */ at the start of a line moves onto a line of its own.
  • L3: newlines inside strings follow newLineKind, like the rest of the file.

Formats SQL with a dialect-aware lexer, a tolerant parser and dprint-core's
IR instead of wrapping sqlformat-rs.

- Dialects: generic (default), postgresql, mysql, sqlserver, sqlite, oracle,
  bigquery and snowflake, selected with the new `dialect` option.
- Procedural code: BEGIN/END blocks, exception handlers, IF/ELSIF, T-SQL
  IF/WHILE, loops, CASE statements, labels, declaration sections, Oracle
  packages and type bodies, PL/pgSQL `$$` bodies, MySQL DELIMITER, T-SQL GO
  batches, SQL*Plus `/` and psql meta-commands.
- New `lineWidth` option. Lists, conditions and brackets break only when they
  don't fit.
- Output is checked to differ from the input only in whitespace (and keyword
  casing when it's changed). Otherwise the file is left as it is and an
  error is reported.
- Range formatting.

BREAKING CHANGE: `format_text` returns a `Result` and the formatted output
differs from sqlformat's. The update workflow that followed sqlformat
releases is removed.

Closes #3
Closes #12
- run clippy, rustfmt and the tracing tests in CI
- give linesBetweenQueries a minimum of 1 in the schema
- mark Dialect and FormatError as non-exhaustive
- drop the dprint-core-macros dependency
- mention the deprecated uppercase option in the readme
- end a variable's name at `#` where it starts a comment (MySQL, BigQuery)
- keep Snowflake stage paths and `file://` URLs as one token
- keep BigQuery project and table names holding dashes
- treat whitespace beyond ASCII as part of a name rather than as a space
- only let PostgreSQL operators end in `+` or `-`
- end BigQuery raw strings and backticked names after an escaped quote
- don't read a column named like a clause in a list as that clause
- keep the case of reserved words that name a table (ex. `FROM view`)
- look up the last token that isn't a comment without scanning back
…eep nesting quickly

- split an INSERT, UPDATE, DELETE or MERGE after a CREATE or ALTER that isn't a routine
- end an UPDATE or MERGE at a SET after its own, and a T-SQL RETURN at the end of its line
- don't read a view's or routine's WITH option as a common table expression
- tell a routine by the first word naming a kind of object (not `CREATE VIEW v AS SELECT event`)
- keep more SQL*Plus commands (SHOW, EXEC, COLUMN, DESC, ...) on their lines
- don't parse a CASE that failed to parse again, which took exponential time when nested
- always break deeply nested values onto lines, since fitting them took exponential time
- don't move a comma across a template tag, and check where template tags are in the code
- write a list with a template tag on its own line or against a comma as it flows
- keep the line break between strings that PostgreSQL joins
- indent a clause's body beneath a comment on a line of its own after its keyword, which was
  unstable
- only accept a word's case changing to the configured one
…ations stable

- only treat ASCII whitespace as space wherever the lexer trims or splits a line
- keep an UPDATE or DELETE after an ALTER in it (ex. ClickHouse's ALTER TABLE t UPDATE)
- continue a SQL*Plus command onto the next line after a trailing ` -`
@dsherret
dsherret merged commit 261c5f6 into main Oct 18, 2026
2 checks passed
@dsherret
dsherret deleted the feat/own-sql-formatter branch October 18, 2026 03:44
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Expose a config for maxInlineBlock / VALUES tuple wrapping set operations inconsistently formatted

1 participant