FazBrowse GitHub Viewer | Trending |
URL:
| Home
Tools: [Download Repo ZIP]   [Original HTTPS Page]

Address PR #192 review (Round 4): fix balance_e NaN mismatch, reject … · UtilityTools/diff-diff@8c79b48 · GitHub

Commit 8c79b48

Browse files
andcommitted
Address PR igerber#192 review (Round 4): fix balance_e NaN mismatch, reject duplicate rows, strengthen docs
- Fix balance_e bootstrap NaN cohort mismatch: filter groups with NaN at anchor horizon in _prepare_es_agg_boot, matching analytical path - Add duplicate (unit, time) validation with clear ValueError; switch pivot_table to pivot as defense-in-depth - Strengthen REGISTRY bootstrap WIF note with explicit CS method reference - Add TestBalanceE class (3 tests) and duplicate validation test - Track deferred P2 items (small-cohort warnings, API docs) in TODO.md Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
1 parent 3102a7c commit 8c79b48

5 files changed

Lines changed: 85 additions & 3 deletions

File tree

‎TODO.md‎

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -45,6 +45,8 @@ Deferred items from PR reviews that were not addressed before merge.
4545
|-------|----------|----|----------|
4646
| ImputationDiD dense `(A0'A0).toarray()` scales O((U+T+K)^2), OOM risk on large panels | `imputation.py` | #141 | Medium (deferred — only triggers when sparse solver fails; fixing requires sparse least-squares alternatives) |
4747
| Bootstrap NaN-gating gap: manual SE/CI/p-value without non-finite filtering or SE<=0 guard | `imputation_bootstrap.py`, `two_stage_bootstrap.py` | #177 | Medium — migrate to `compute_effect_bootstrap_stats` from `bootstrap_utils.py` |
48+
| EfficientDiD: warn when cohort share is very small (< 2 units or < 1% of sample) — inverted in Omega*/EIF | `efficient_did_weights.py` | #192 | Low |
49+
| EfficientDiD: API docs / tutorial page for new public estimator | `docs/` | #192 | Medium |
4850

4951
#### Performance
5052

‎diff_diff/efficient_did.py‎

Lines changed: 10 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -217,6 +217,15 @@ def fit(
217217
"panel where every unit is observed in every time period."
218218
)
219219

220+
# Reject duplicate (unit, time) rows
221+
dup_mask = df.duplicated(subset=[unit, time], keep=False)
222+
if dup_mask.any():
223+
n_dups = int(dup_mask.sum())
224+
raise ValueError(
225+
f"Found {n_dups} duplicate ({unit}, {time}) rows. "
226+
"EfficientDiD requires exactly one observation per unit-period."
227+
)
228+
220229
# Validate absorbing treatment (vectorized)
221230
ft_nunique = df.groupby(unit)[first_treat].nunique()
222231
bad_units = ft_nunique[ft_nunique > 1]
@@ -259,7 +268,7 @@ def fit(
259268
period_1_col = period_to_col[period_1]
260269

261270
# Pivot outcome to wide matrix (n_units, n_periods)
262-
pivot = df.pivot_table(index=unit, columns=time, values=outcome, aggfunc="first")
271+
pivot = df.pivot(index=unit, columns=time, values=outcome)
263272
# Reindex to match all_units ordering and time_periods column order
264273
pivot = pivot.reindex(index=all_units, columns=time_periods)
265274
outcome_wide = pivot.values.astype(float)

‎diff_diff/efficient_did_bootstrap.py‎

Lines changed: 3 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -246,7 +246,9 @@ def _prepare_es_agg_boot(
246246

247247
if balance_e is not None:
248248
groups_at_e = {
249-
gt_pairs[j][0] for j, (g, t) in enumerate(gt_pairs) if t - g == balance_e
249+
gt_pairs[j][0]
250+
for j, (g, t) in enumerate(gt_pairs)
251+
if t - g == balance_e and np.isfinite(original_atts[j])
250252
}
251253
balanced: Dict[int, List[Tuple[int, float, float]]] = {}
252254
for j, (g, t) in enumerate(gt_pairs):

‎docs/methodology/REGISTRY.md‎

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -586,8 +586,9 @@ where `q_{g,e} = pi_g / sum_{g' in G_{trt,e}} pi_{g'}`.
586586
- **All units eventually treated**: Last cohort serves as "never-treated" by dropping last time period (Phase 1: raises ValueError; last-cohort-as-control fallback planned for Phase 2)
587587
- **Negative weights**: Explicitly stated as harmless for bias and beneficial for precision; arise from efficiency optimization under overidentification (Section 5.2)
588588
- **PT-Post regime (just-identified)**: Under PT-Post, EDiD automatically reduces to standard single-baseline estimator (Corollary 3.2). No downside to using EDiD -- it subsumes standard estimators
589+
- **Duplicate rows**: Duplicate `(unit, time)` entries are rejected with `ValueError`. The estimator requires exactly one observation per unit-period
589590
- **PT-All index set**: Under PT-All, valid (g', t_pre) pairs require only t_pre < g' (pre-treatment for the comparison group), not t_pre < g. Same-group pairs (g'=g) are valid and contribute overidentifying moments. This follows from Equation 3.9: the target group g appears only in the first term (Y_t - Y_1), which is independent of t_pre
590-
- **Bootstrap aggregation**: Multiplier bootstrap uses fixed cohort-size weights for overall/event-study aggregation, matching the CallawaySantAnna bootstrap pattern (staggered_bootstrap.py). The analytical path includes a WIF correction; the bootstrap implicitly accounts for all sources of sampling variability through EIF perturbation, subsuming the WIF correction. This is consistent with the R `did` package approach
591+
- **Bootstrap aggregation**: Multiplier bootstrap uses fixed cohort-size weights for overall/event-study aggregation, matching the CallawaySantAnna bootstrap pattern (CallawaySantAnnaBootstrapMixin._run_multiplier_bootstrap). The analytical path includes a WIF correction; the bootstrap captures sampling variability through per-cell EIF perturbation without re-estimating aggregation weights, consistent with both the library's CS implementation and the R `did` package approach
591592
- **Overall ATT convention**: The library's `overall_att` uses cohort-size-weighted averaging of post-treatment (g,t) cells, matching the CallawaySantAnna simple aggregation. This differs from the paper's ES_avg (Eq 2.3), which uniformly averages over event-time horizons. ES_avg can be computed from event study output as `mean(event_study_effects[e]["effect"] for e >= 0)`
592593

593594
*Algorithm (two-step semiparametric estimation, Section 4):*

‎tests/test_efficient_did.py‎

Lines changed: 68 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -321,6 +321,15 @@ def test_pt_post_no_never_treated_raises(self):
321321
with pytest.raises(ValueError, match="never-treated"):
322322
EfficientDiD(pt_assumption="post").fit(df, "y", "unit", "time", "first_treat")
323323

324+
def test_duplicate_unit_time_raises(self):
325+
"""Duplicate (unit, time) rows should be rejected."""
326+
df = _make_simple_panel()
327+
# Duplicate a row
328+
dup_row = df.iloc[[0]].copy()
329+
df = pd.concat([df, dup_row], ignore_index=True)
330+
with pytest.raises(ValueError, match="duplicate"):
331+
EfficientDiD().fit(df, "y", "unit", "time", "first_treat")
332+
324333

325334
class TestSklearnCompat:
326335
"""Test get_params / set_params."""
@@ -652,6 +661,65 @@ def test_anticipation_parameter(self):
652661
assert len(post_effects) > 0
653662

654663

664+
class TestBalanceE:
665+
"""Test balance_e event study balancing."""
666+
667+
def test_balance_e_basic(self):
668+
"""balance_e restricts event study to cohorts present at anchor horizon."""
669+
df = _make_staggered_panel(n_per_group=80, n_control=80, groups=(3, 5))
670+
result = EfficientDiD().fit(
671+
df,
672+
"y",
673+
"unit",
674+
"time",
675+
"first_treat",
676+
aggregate="event_study",
677+
balance_e=0,
678+
)
679+
assert result.event_study_effects is not None
680+
for e, d in result.event_study_effects.items():
681+
assert np.isfinite(d["effect"])
682+
683+
def test_balance_e_with_bootstrap(self, ci_params):
684+
"""Bootstrap balance_e should produce finite SEs."""
685+
n_boot = ci_params.bootstrap(99)
686+
df = _make_staggered_panel(n_per_group=80, n_control=80, groups=(3, 5))
687+
result = EfficientDiD(n_bootstrap=n_boot, seed=42).fit(
688+
df,
689+
"y",
690+
"unit",
691+
"time",
692+
"first_treat",
693+
aggregate="event_study",
694+
balance_e=0,
695+
)
696+
assert result.event_study_effects is not None
697+
for e, d in result.event_study_effects.items():
698+
if np.isfinite(d["effect"]):
699+
assert np.isfinite(d["se"])
700+
701+
def test_balance_e_nan_anchor_filters_group(self):
702+
"""When a group has NaN at the anchor horizon, bootstrap should
703+
exclude it from groups_at_e, matching the analytical path."""
704+
edid = EfficientDiD()
705+
edid.anticipation = 0
706+
707+
# Simulate: group 3 has finite effect at e=0, group 5 has NaN at e=0
708+
gt_pairs = [(3.0, 3), (3.0, 4), (5.0, 5), (5.0, 6)]
709+
original_atts = np.array([1.0, 1.5, np.nan, 0.8])
710+
cohort_fractions = {3.0: 0.4, 5.0: 0.3}
711+
712+
result = edid._prepare_es_agg_boot(gt_pairs, original_atts, cohort_fractions, balance_e=0)
713+
# Group 5 has NaN at e=0 (t=5, g=5), so it should be excluded
714+
# Only group 3 effects should appear in the balanced set
715+
for e, info in result.items():
716+
gt_indices = info["gt_indices"]
717+
groups_in_e = {gt_pairs[j][0] for j in gt_indices}
718+
assert 5.0 not in groups_in_e, (
719+
f"Group 5 (NaN at anchor) should be excluded at e={e}, " f"got groups {groups_in_e}"
720+
)
721+
722+
655723
# =============================================================================
656724
# Tier 3: Bootstrap
657725
# =============================================================================

0 commit comments

Comments
 (0)

Back | FazBrowse Home | New Git URL