FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

fix: Ensure that prod/dev interval and physical model logic consistently uses Deployability by davem-bis · Pull Request #5988 · SQLMesh/sqlmesh · GitHub

fix: Ensure that prod/dev interval and physical model logic consistently uses Deployability - #5988

Open
davem-bis wants to merge 5 commits into
SQLMesh:mainfrom
davem-bis:feature/DRM/resolve-indirect-non-breaking-deployable-representative-mismatch
Open

fix: Ensure that prod/dev interval and physical model logic consistently uses Deployability#5988
davem-bis wants to merge 5 commits into
SQLMesh:mainfrom
davem-bis:feature/DRM/resolve-indirect-non-breaking-deployable-representative-mismatch

Conversation

davem-bis commented Aug 26, 2026
edited
Loading

Copy link
Copy Markdown
Contributor

Description

Fixes #5793.

Changes the logic determining whether to use the prod or dev instance of intervals or physical model to rely solely on deployability. Previously this was inconsistently using either deployability and representativeness, leading to non-idempotent behaviour in non-prod environments.

Test Plan

  • Expansion of existing unit tests for changed logic.
  • Run through of INDIRECT_BREAKING_CHANGES snapshots to prove idempotence.
    • Run plan:
      1. sqlmesh init

        Option 1 throughout - basic DuckDB implementation with the demo models.

      2. sqlmesh plan

      3. Add 'ABC' AS extra_column to incremental_model.sql

      4. sqlmesh plan indirect_non_breaking

      5. sqlmesh run

      6. sqlmesh run indirect_non_breaking

      7. sqlmesh fetchdf "UPDATE sqlmesh._intervals SET end_ts = end_ts - 86400000, start_ts = start_ts - 86400000, last_altered_ts = last_altered_ts - 86400000"

        86400000 is the number of milliseconds in a day (246060*1000)

      8. sqlmesh run indirect_non_breaking

      9. sqlmesh run indirect_non_breaking

        Previously this would result in the intervals being re-run, it now correctly does nothing.

    • Comments:
      • Model Name Change Type Deployability Representativeness Version Used In indirect_non_breaking
        seed_model None True True Same as prod
        incremental_model NonBreaking True True Separate physical model
        full_model IndirectNonBreaking False True Separate physical model with dev extension
      • full_model now uses the __dev physical model, and derives missing intervals from there, and points at that physical model in the virtual layer.

Checklist

  • I have run make style and fixed any issues
  • I have added tests for my changes (if applicable)
  • All existing tests pass (make fast-test)
  • My commits are signed off (git commit -s) per the DCO

…deployability.

Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
…ndirect-non-breaking-deployable-representative-mismatch
…ty instead of representativeness for determining whether dev or prod instance of each should be used.

Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
Signed-off-by: davem-bis <68955845+davem-bis@users.noreply.github.com>
davem-bis marked this pull request as ready for review August 26, 2026 14:03

Copy link
Copy Markdown
Collaborator

@davem-bis -- I think you'll also want to change from representative to deployable in the to_table_mapping function in sqlmesh/core/snapshot/definition.py file as well. That should create a clear prod/dev split.

I'd also look into modifying the is_representative docstring in the DeployabilityIndex class. After this change, it will not be used for computing missing intervals. I think it will only be used for prod promotion and table creation stuff.

Something like this:

Unlike `is_deployable`, this variant also captures FORWARD_ONLY and INDIRECT_NON_BREAKING snapshots that
are not deployable by their nature but are currently promoted in production. This is used when constructing
a plan to determine prod promotion staging (e.g. which snapshots backfill before promote) and whether a
physical table needs to be created.

cmgoffena13 self-assigned this Aug 29, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters. Learn more about bidirectional Unicode characters
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Bug: INDIRECT_NON_BREAKING snapshots in dev environments write to a table nobody reads, causing repeated re-backfills

2 participants


Back | FazBrowse Home | New Git URL