Skip to content

fix: non-breaking detection for redundant cast projections#5851

Open
albertosuman-1k5 wants to merge 5 commits into
SQLMesh:mainfrom
1K5-TECH:new-main
Open

fix: non-breaking detection for redundant cast projections#5851
albertosuman-1k5 wants to merge 5 commits into
SQLMesh:mainfrom
1K5-TECH:new-main

Conversation

@albertosuman-1k5

Copy link
Copy Markdown
Contributor

Description

Summary

  • Fixes a false-positive Breaking categorization when adding a mid-list projection with a redundant same-type cast, e.g. new_col::STRING above an existing ::STRING projection.
  • Adds a conservative projection-additivity fallback for SqlModel.is_breaking_change() when SQLGlot’s AST diff emits spurious Move / Update edits.
  • Preserves conservative behavior for real changes such as projection rewrites, removals, reorders, filter/group changes, and cardinality-changing UDTFs.

Context

SQLGlot’s tree diff can cross-match structurally similar cast nodes, especially identical DataType leaves such as STRING. This can make a purely additive projection change look like a non-Insert diff, causing SQLMesh to return None from is_breaking_change(). Under AutoCategorizationMode.FULL, that becomes BREAKING.

The new fallback verifies directly that:

  • both rendered queries are top-level SELECTs,
  • the query skeleton is unchanged except for the projection list,
  • previous projections are preserved in order,
  • and added projections do not include cardinality-changing UDTFs.

Test plan

  • python -m pytest tests/core/test_snapshot.py -k "categorize_change_sql or categorize_change_seed" -q
  • python -m pytest cast_breaking_change_demo/test_cast_breaking_change.py -q
  • python -m pytest tests/core/test_snapshot.py tests/core/test_model.py::test_is_breaking_change tests/core/test_plan.py -q
  • pre-commit run --files sqlmesh/core/model/definition.py tests/core/test_snapshot.py cast_breaking_change_demo/test_cast_breaking_change.py

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

@albertosuman-1k5

Copy link
Copy Markdown
Contributor Author

Can we trigger again the workflows? Pretty sure the first run failed because of the issue described in #5852

@mday-io

mday-io commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Can we trigger again the workflows? Pretty sure the first run failed because of the issue described in #5852

rerunning...

@mday-io mday-io self-requested a review June 23, 2026 13:50
Comment thread sqlmesh/core/model/definition.py
Comment thread sqlmesh/core/model/definition.py
@mday-io

mday-io commented Jun 23, 2026

Copy link
Copy Markdown
Collaborator

Could you add tests for queries that reference SELECT-list positions from other clauses, such as ordinal ORDER BY / GROUP BY, to ensure those stay undetermined instead of being auto-categorized as non-breaking?

It would also be good to add coverage for aliased UDTF projections, since those may not appear as a top-level UDTF expression in the projection list but should still remain conservative.

@albertosuman-1k5

Copy link
Copy Markdown
Contributor Author

@mday-io added logic to avoid report as non breaking when order/group by uses positional arguments whose reference is past the introduced expression and to correctly mark aliased UDTF

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants