feat(format): skip the project load when paths are given - #6063
tripleaceme wants to merge 2 commits into
Conversation
`sqlmesh format` already accepted positional paths and never loaded state, but it still loaded every model in the project and then filtered them down with `Path.samefile`. That load is wasted work: formatting pretty-prints a file's own text, and needs nothing from the rest of the project graph. Formatting a file needs only three things — the file's text, the config of the project that owns its path, and the dialect and formatting flag. The first two come from `config_for_path`, which resolves a config from a path alone, and the last two are read off the file's own MODEL/AUDIT header. So when paths are given the project is no longer loaded, and `format` now joins `lint` in scoping its own load: with no paths it loads the project itself, which also leaves the web and LSP callers untouched since those contexts are already loaded. Whether a file is a model is now decided by parsing its header rather than by membership in the loaded project, which keeps macros and other non-model SQL a no-op and ignores Python model paths. One consequence worth calling out: a standalone audit is formatted when selected by path, while the project-wide pass still skips it, because it is loaded into `_standalone_audits` rather than `_audits`. That gap predates this change and is noted on the issue. Collecting targets through a small named tuple lets both the path-based and project-wide routes share one format-and-write loop, and drops the `samefile` call that ran once per model in the project. Signed-off-by: Adegbite Ayoade <tripleaceme@gmail.com>
|
@cmgoffena13 — closes #6025. With paths given, the project is no longer loaded at all: formatting a file only needs its own text, the config that owns its path, and the dialect and formatting flag off its own MODEL/AUDIT header. One thing I'd like your call on, also noted on the issue. The acceptance criteria say to format a path if it is "a SQL model or standalone audit", but standalone audits aren't formatted today at all — I've left the project-wide route exactly as it is and let the path route pick them up, with a test pinning both halves so it's a small change in either direction once you decide. Zero checks, pending workflow approval. |
Resolves the conflict with SQLMesh#6060, which added `--local` to `sqlmesh test`. The only conflict was tests/cli/test_cli.py, where each branch appended its own tests to the end of the file, so both blocks are kept. sqlmesh/cli/main.py merged cleanly and both changes survive: the `OPTIONAL_LOCAL_COMMANDS` gating for `test --local` from SQLMesh#6060, and the `("lint", "format")` scoped load from this branch. The two sit in different parts of the callback and don't interact. Verified afterwards that `format <path>` still skips `Context.load`, `format` with no paths still loads, and `test --local` still runs. Signed-off-by: Adegbite Ayoade <tripleaceme@gmail.com>
|
Heads up @cmgoffena13 — merging #6060 put this one into conflict too, so I've resolved it in c08310b rather than leave it for you to trip over. It's Only
The Still open on this one whenever you get to it: the standalone-audit question from the description. |
Description
Closes #6025.
sqlmesh formatalready accepted positional paths and never loaded state, but with paths it still loaded every model in the project and then filtered them down withPath.samefile. That load is wasted: formatting pretty-prints a file's own text and needs nothing from the rest of the project graph.Formatting a file needs three things — the file's text, the config of the project that owns its path, and the dialect and formatting flag. The first two come from
config_for_path, which resolves a config from a path alone, and the last two are read off the file's ownMODEL/AUDITheader. So with paths the project is no longer loaded at all.formatnow joinslintin scoping its own load: with no paths it loads the project itself, which also leaves the web and LSP callers untouched, since those contexts are already loaded and_loadedshort-circuits the call.Whether a file is a model is now decided by parsing its header rather than by membership in the loaded project. That keeps
macros/*.sqland other non-model SQL a no-op and ignores.pymodel paths, per the acceptance criteria.Both routes now collect
FormatTargets and share one format-and-write loop, which also drops thesamefilecall that ran once per model in the project.One behavioural note
A standalone audit is formatted when selected by path, while the project-wide pass still skips it. That is because
Context.formatiterateschain(self._models.values(), self._audits.values())and anAUDIT (..., standalone true)is loaded intoself._standalone_audits, a separate dict — so standalone audits aren't formatted today at all. I raised this on the issue: the acceptance criteria say to format "SQL models or standalone audits", which assumes behaviour that doesn't currently exist. The header-based check treats both kinds ofAUDITthe same, so the path route picks them up.I've left the project-wide route exactly as it was rather than quietly fixing the gap there.
test_format_paths_formats_standalone_auditspins both halves so the difference is explicit and easy to change in either direction once you've decided.Test Plan
Ten tests in
tests/core/test_format.py:test_format_paths_does_not_load_project— spies onContext.load, asserts it's never called and that the selected model is still formatted while an unselected one is untouchedtest_format_paths_output_matches_loaded_format— formats the same project twice, once through each route, and asserts the files come out byte-identicaltest_format_paths_ignores_non_model_sql/test_format_paths_ignores_python_modelstest_format_paths_honors_formatting_false,..._model_defaults_formatting_false,..._string_model_defaults_formattingtest_format_paths_check_reports_unformattedtest_format_without_paths_still_loadstest_format_paths_formats_standalone_auditsEach "file is untouched" assertion is paired with a file that must change, so none of them can pass merely because nothing was formatted.
Two in
tests/cli/test_cli.pycovering the CLI wiring in both directions.The three
test_dlt_*failures are pre-existing onmainin my environment (ModuleNotFoundError: dlt).Checklist
make styleand fixed any issuesmake fast-test)git commit -s) per the DCO