| FazBrowse GitHub Viewer | Trending | | Home |
| Tools: [Download Repo ZIP] [Original HTTPS Page] |
Three cases deleted rows outside the cascade: - `(T & dj.Top(n)).delete()` and `delete_quick()` ignored the Top and deleted every row matching the other conditions. A Top seed is now replaced by the primary keys it selects. - A parent restriction was copied to a child whenever the restricted attributes were in the child's primary key. Through a nullable foreign key that shares a primary-key column, the copy also deleted child rows whose foreign key is NULL; in a diamond it missed rows. The copy now applies only when the restricted attributes are among the foreign key's attributes and the foreign key lies in the child's primary key. Otherwise the child is restricted by the projected parent. - With part_integrity="cascade", the part-to-master lift copied the part's condition to the master, which also deleted masters with no matching part rows. The master is now restricted by the restricted part, so a master joins the cascade only if one of its part rows did. The walk's restrictions on intermediate parts are restored once the master is materialized; the master cascades back to all of its parts. The Diagram copy constructor now copies each node's restriction list instead of deep-copying projected query expressions, which hold a live connection.
There was a problem hiding this comment.
Read through the change against 1589, 1590 and 1591. Each fix seems to close its issue, and the narrower copy rule looks like it leaves the common primary-key chains on the same path as before. A few notes and questions:
Sorry, something went wrong.
There was a problem hiding this comment.
Thanks for the thorough write-up @dimitri-yatsenko — the three root causes and the "restrict only what the cascade reaches" framing hold up when I read the diff. I haven't run the suite locally; these notes come from reading the code.
Requested change
✅ Add a regression test for an empty seed. _seed_restriction replaces a Top seed with AndList([table_expr.keys()]). When keys() returns [], this relies on AndList([[]]) being truthy and compiling to FALSE. From condition.py that looks right, but if it were ever treated as "no restriction" the cascade would delete every row, so it's worth pinning down. For example: (Item & dj.Top(2)).delete() on an empty table, with an assertion that the preview counts are 0 and nothing is deleted.
Merge ordering
#1525 (also in milestone v2.3.5) rewrites _propagate_part_to_master, and a trial merge of the two conflicts in diagram.py. Which should land first? Whichever goes second will need the saved/restored intermediate restrictions in this PR re-checked against the all-paths walk.
Questions
Everything else reads correct to me: the nullable-FK and diamond rule (parent_attrs <= set(attr_map) <= child_pk), the existential part→master lift, and the copy-constructor change. Let me know if anything needs adjusting!
Sorry, something went wrong.
…aced Top seeds The part-to-master walk saves the restrictions of the parts it walks through and restores them once the master is materialized. It saved each part's restricted-attribute set by reference, and the walk then extends that same set in place, so the restore changed nothing. The extra attributes only made later copies stricter, so the effect was an extra subquery, but the code did not do what its comment says. It now saves a copy of the set. Two tests pin down seeds this PR changes: a `Top` over a restriction that matches nothing previews no rows and deletes nothing, and `Diagram.trace` starts from the rows a `Top` seed selects.
|
Thanks, @lum-agilyti and @MilagrosMarin. Pushed f14d68b here; the same commit is on #1593. Done in this PR
Left for 2.4
Other questions
|
Sorry, something went wrong.
There was a problem hiding this comment.
Thanks @dimitri-yatsenko — re-read against f14d68b6. The three follow-ups land, and I traced the empty-seed mechanism this time rather than reasoning from the diff:
✅ keys() returning [] gives AndList([[]]), which is truthy (length 1), so the seed survives [restriction] if restriction else []; make_condition on the inner empty list falls to the OrList branch and returns False, so the seed compiles to a false condition rather than to "no restriction". test_top_seed_that_selects_no_rows_deletes_nothing pins that.
✅ test_top_seed_in_trace fails without the fix — the unpatched seed is AndList([]), which is falsy, so trace would count 5/5 instead of 2/2.
✅ saved now snapshots the attribute set rather than aliasing it.
✅ CI: 16 checks green here and on #1593. I'd noted the matrix names only MySQL images — it does cover PostgreSQL, since test-cov runs all of tests/ and the backend fixture is parameterized over both via testcontainers.
✅ #1593 carries byte-identical diagram.py, table.py and test_cascade_overdelete.py.
Three notes, none of them blocking, in descending order of how much they matter:
1. The early return inside the walk skips the restore. diagram.py:1099 returns between the saved snapshot (1082) and the restore loop (1145), so on that path the intermediate parts keep the walk's temporary restrictions — the ones the restore exists to remove. The branch is marked "shouldn't happen if shortest_path succeeded", so this is hygiene rather than a live bug, but a try: ... finally: around the walk would make the restore unconditional and cost nothing.
2. Is the attribute half of the restore observable? Before f14d68b6 it was a no-op and CI was green; after the fix it is a real restore and CI is still green, so no test distinguishes it. I traced the part-of-part shape in test_cascade_part_of_part_no_master_reference: because cascade restrictions OR-combine, the subsequent Master → PartA → PartB copy decisions come out the same either way. If there is a shape where it changes the result, a test would be worth having; if it is consistency only, saying so in the comment at 1083 is better than leaving it reading as load-bearing.
3. delete_quick() with a Top that selects nothing. The new empty-seed test goes through delete() and Diagram.cascade, not through the new table.py branch. There self.keys() is [], target & [] appends the Python bool False to the restriction, and where_clause renders WHERE (False) — correct on both backends, I checked. But it is the one place where a wrong answer deletes the whole table, and test_top_seed_deletes_only_the_top_rows already exercises delete_quick with a non-empty Top. One more line in the empty test would close it:
(Item & "item_id > 100" & dj.Top(2)).delete_quick() assert len(Item()) == 3
On Diagram.restrict(): understood that expand replaces it in 2.4. For 2.3.5, restrict still seeds from table_expr.restriction (diagram.py:778), and for T & dj.Top(2) that list is empty, so the Top is silently dropped and the diagram shows every matching row. It is a preview, not a delete, so the blast radius is small — but _seed_restriction now exists, and either reusing it there or noting the limitation in the restrict() docstring would keep it from reading as the same bug left unfixed.
Approving — everything else reads correct to me, and none of the three notes needs to hold up the v2.3.5 patch. Take them here or fold them into the 2.4 traversal work, whichever fits better.
Sorry, something went wrong.
There was a problem hiding this comment.
Thanks, f14d68b covers what I raised.
Sorry, something went wrong.
|
Thanks, @MilagrosMarin and @lum-agilyti. Merging for v2.3.5. The notes go to the 2.4 traversal work, where Diagram.expand replaces this code (datajoint/datajoint-docs#249):
|
Sorry, something went wrong.
| Back | FazBrowse Home | New Git URL |
Closes #1589, closes #1590, closes #1591.
A cascade delete reached rows outside the cascade in three cases:
Two follow-on changes:
Not changed: Diagram.trace keeps its upward copy for now. self.upstream traces from a row that make() has not inserted yet and relies on it. The traversal redesign for 2.4 defines that case explicitly.
Tests: tests/integration/test_cascade_overdelete.py adds four tests on MySQL and PostgreSQL. All fail on v2.3.4 and pass here. The full suite passes locally (1166 passed, 14 skipped).
The same commit is cherry-picked onto maint/2.3 for a v2.3.5 patch release.