Skip to content

marketing: bad-controls launch carousel (Junction palette) + claims-sync guard - #858

Merged
igerber merged 1 commit into
mainfrom
bad-controls-carousel
Oct 18, 2026
Merged

igerber merged 1 commit into
mainfrom
bad-controls-carousel

Conversation

@igerber

@igerber igerber commented Oct 18, 2026

Copy link
Copy Markdown
Owner

Summary

  • LinkedIn launch carousel for the v3.12 DMLDiD bad-control lane (Caetano, Callaway, Payne & Sant'Anna 2026): carousel/generate_bad_controls_carousel.py + committed carousel/diff-diff-bad-controls-carousel.pdf (11 slides, poster mode, "Junction" palette - ink/stone structure with a single amber accent reserved for the bad control and the fork motif's third road; the two wrong roads stay warm gray).
  • Practitioner-dilemma narrative spine: cover ("Include the covariate, or drop it? Both can be wrong." over the fork motif), the bad-control definition with the paper named before it is referenced, the include-at-t receipt (within-TWFE 0.4886 against a true 1.00 on the paper's first Monte Carlo design made staggered), the dark drop-it twist (no number, by design: the tutorial does not run that arm), the paper slide with preprint status on-slide, Route 1 (CallawaySantAnna base-period covariates, W left out vs W in), Route 2 (three-arm chart with both routes amber, plus what Route 2 buys: W explains the covariate's path only and never enters parallel trends, Remark 2), the bad_control_summary() pre-test table with both Remark 6 caveats on-slide, the dark code slide, the NLSY79 application told wrong-then-right (the authors' estimates, labeled as such, never reproduced) with the lane-scope / validation strip, and the CTA.
  • tests/test_bad_controls_carousel_claims.py (29 checks): every simulated estimate located on the committed tutorial-33 notebook surface (prose and rendered outputs, including the six ATT_X rows); the NLSY79 numbers and the paper's own "almost 10%" / "roughly 40% larger" wording located in the paper review; the cover's FEATURE_RELEASE = "3.12" pinned to the CHANGELOG section that ships the lane (the footer still derives from pyproject.toml); oracle tolerance and R package name pinned to the REGISTRY block; arithmetic behind the half-the-effect / SE-distance claims; slide-local qualifiers; a banned-absolutes list (no "unbiased", no "detects bad control", no Pedro); a valid-Python check on the code slide; tutorial referenced once on the CTA.

Methodology references (required if estimator / math changes)

  • Method name(s): N/A - no estimator/math changes (marketing collateral + claim tests only). Slide copy follows the REGISTRY DMLDiD "Bad-control extension (CCPS 2026)" block and the CallawaySantAnna Approach-1 Note.
  • Paper / source link(s): Caetano, Callaway, Payne & Sant'Anna (2026), arXiv:2608.03881 (preprint; docs/methodology/papers/caetano-2026-review.md).
  • Any intentional deviations from the source (and why): None. The deck states on-slide that the headline ATT keeps the CS simple weighting (not the paper's Remark 4 overall), the lane's panel-only / cluster-only scope, and that the application numbers are the authors' estimates.

Validation

  • Tests added/updated: tests/test_bad_controls_carousel_claims.py (new, 29 checks, green alongside tests/test_t33_bad_controls_drift.py).
  • Backtest / simulation / notebook evidence (if applicable): every simulated number on the deck is a seed-locked tutorial-33 value (the paper's DGP 1 made staggered, truth known by construction, said on-slide); all 11 pages visually inspected at zoom across three review passes; PDF regenerated from the committed generator in the same commit.

Security / privacy

  • Confirm no secrets/PII in this PR: Yes

Changelog

…ync guard

LinkedIn launch deck for the v3.12 DMLDiD bad-control lane (Caetano,
Callaway, Payne & Sant'Anna 2026): carousel/generate_bad_controls_carousel.py
+ committed carousel/diff-diff-bad-controls-carousel.pdf (11 slides, poster
mode, ink/stone structure with a single amber accent for the bad control and
the fork motif's third road).

tests/test_bad_controls_carousel_claims.py (29 checks): every simulated
estimate located on the committed tutorial-33 notebook surface, the NLSY79
application numbers and the paper's own wording located in the paper review,
the cover's release label pinned to the CHANGELOG section that shipped the
lane, validation anchors pinned to the REGISTRY block, arithmetic behind the
half-the-effect / SE claims, slide-local qualifiers, a banned-absolutes
list, and a valid-Python check on the code slide.
@github-actions

Copy link
Copy Markdown

Overall assessment: ✅ Looks good

No unmitigated P0/P1 findings in the three changed files.

Executive summary

  • Claims align with the Methodology Registry, committed paper review, and tutorial 33.
  • Simulated estimates, application attribution, and both pre-test caveats are consistent with those sources.
  • All 29 claim checks passed through direct invocation using a standard-library harness.
  • Full pytest execution and PDF regeneration were unavailable because dependencies are absent.

Methodology

P3 — Documented aggregation difference; informational.
Location: carousel/generate_bad_controls_carousel.py:L1220-L1222.

Impact: The deck correctly discloses that the headline ATT uses Callaway–Sant’Anna simple weighting rather than the paper’s Remark 4 weighting. This matches the Registry’s labeled Note.

Concrete fix: None required.

Reviewed methods: CallawaySantAnna’s Approach 1 and DMLDiD’s CCPS Approach 2. Their descriptions, distinction concerning W, and ATT_X interpretation match the Registry and committed review of arXiv v2/SA v1. No estimator or inference implementation changes are introduced.

Code Quality

No findings. Both Python files parse under Python 3.9 syntax. The generator cleans up temporary images through finally.

Performance

No findings. Generation runs outside library execution, and claim checks avoid importing rendering dependencies.

Maintainability

No findings. Statistical constants are centralized and checked against their intended slides and source artifacts.

Tech Debt

P3 — PDF synchronization guard is already tracked.
Locations: tests/test_bad_controls_carousel_claims.py:L64-L67; TODO.md:L101.

Impact: Source-based assertions cannot automatically detect a stale committed PDF. The extracted text and embedded comparison chart matched the reviewed source during this review.

Concrete fix: No change required for approval. The tracked follow-up is an automated PDF-to-generator comparison using fixed metadata or a rendered-content manifest.

Security

No findings. No secrets identified in the changed source, and the PDF contains no JavaScript, launch actions, or embedded files.

Documentation/Tests

No additional findings. All 29 claim checks passed when invoked directly with pytest fixture and parameter decorators emulated. This does not substitute for full pytest execution or tutorial rederivation.

The committed PDF contains 11 pages. Full-page visual rendering and regeneration were not verified. No changelog fragment is required because library behavior is unchanged.

@igerber igerber added the ready-for-ci Triggers CI test workflows label Oct 18, 2026
@igerber
igerber merged commit 14f225e into main Oct 18, 2026
29 of 30 checks passed
@igerber
igerber deleted the bad-controls-carousel branch October 18, 2026 18:54
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready-for-ci Triggers CI test workflows

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant