Repository navigation
drop() leaves the jobs table behind, and a table re-declared under the same name inherits it #1601
Description
Summary
drop() removes an auto-populated table but not its jobs table, ~~<table_name>. drop_quick() (table.py:1224) drops only full_table_name. The jobs table is created lazily by AutoPopulate.jobs (autopopulate.py:182-188), and nothing else removes it, so it stays, with its records and its primary key.
When a table is later declared under the same name, its jobs property picks up the old jobs table. If the new table's primary key differs, populate(reserve_jobs=True) computes nothing and raises no error. refresh() adds key_source - target - jobs, and the old records, keyed on part of the new primary key, cover every new key.
Reproduction
DataJoint 2.3.5.dev8 (master, a00b89f), MySQL 8.0.46, Python 3.11:
import datajoint as dj
schema = dj.Schema("scratch_drop_jobs")
@schema
class Item(dj.Manual):
definition = """
item : int32
"""
@schema
class Param(dj.Lookup):
definition = """
param : int32
"""
contents = [(1,), (2,)]
@schema
class Result(dj.Computed):
definition = """
-> Item
---
value : int32
"""
def make(self, key):
raise RuntimeError("not ready")
Item.insert({"item": i} for i in range(3))
Result.populate(reserve_jobs=True, suppress_errors=True)
Result.drop(prompt=False)
print("tables after drop:", [r[0] for r in schema.connection.query("SHOW TABLES FROM `scratch_drop_jobs`")])
@schema
class Result(dj.Computed): # same name, primary key now includes Param
definition = """
-> Item
-> Param
---
value : int32
"""
def make(self, key):
self.insert1(dict(key, value=1))
print("jobs primary key:", Result.jobs.heading.primary_key, "records:", [j["status"] for j in Result.jobs.to_dicts()])
Result.populate(reserve_jobs=True)
print("rows after populate(reserve_jobs=True):", len(Result()), "of", len(Result.key_source))Output:
tables after drop: ['#param', 'item', '~lineage', '~~result'] jobs primary key: ['item'] records: ['error', 'error', 'error'] rows after populate(reserve_jobs=True): 0 of 6
Expected behavior
- drop() drops each table's jobs table along with the table, as it already removes the table's ~lineage entries.
- AutoPopulate.jobs checks that an existing jobs table's primary key matches the table's, and raises if it does not. That also catches jobs tables left behind by earlier releases.
Activity
dimitri-yatsenko commented on Oct 19, 2026
Two more cases, on master at a00b89f (2.3.5.dev8), MySQL 8.0.21:
For the fix: the jobs table is created lazily, by AutoPopulate.jobs in whichever process populates, so drop() cannot count on the jobs property of the class it is called on. A virtual module's classes, and the descendants the cascade reaches, never touch it. Dropping ~~<table_name> by name, when it exists, for each table the cascade drops, covers both cases.