Repository navigation

fix(trainer-rank): account for recomputed mixer memory and head stages - #963

Draft

bradhilton wants to merge 52 commits into

mainfrom

dalinar/recompute-mixer-floor

Draft

fix(trainer-rank): account for recomputed mixer memory and head stages#963
bradhilton wants to merge 52 commits into
mainfrom
dalinar/recompute-mixer-floor

Commits on Sep 25, 2026
- Price the recomputed layer's attention or GDN activations in the checkpoint floor
  
  bradhilton
  and
  claude
  committed
  b253f67
- Count the recomputed mixer in generic admission without a MoE component
  
  bradhilton
  and
  claude
  committed
  7cf9aa0
- Ground the GDN recompute width in its measured sites and exchange widths
  
  bradhilton
  and
  claude
  committed
  a256cd3
- Price GDN l2norm outputs at value-head width
  
  bradhilton
  and
  claude
  committed
  91c98a8
- Price one dispatched H-wide input per routed row under HybridEP
  
  bradhilton
  and
  claude
  committed
  cf56c61
- Format the HybridEP converted-stage test
  
  bradhilton
  and
  claude
  committed
  d583cd2
- Name the EP1 all-to-all's second routed copy as its expert-sorted rows
  
  bradhilton
  and
  claude
  committed
  210acec
- Price what is live at the recompute peak; make the EP allowance EP-dependent
  
  bradhilton
  and
  claude
  committed
  c59a692
- Price HybridEP routed rows on the EP group's balanced share
  
  bradhilton
  and
  claude
  committed
  9d03f33
- Keep the per-boundary gradient unless MoE covers all recompute
  
  bradhilton
  and
  claude
  committed
  8bc0c98
- Require priced FC1 stages before trusting a slot's MoE coverage
  
  bradhilton
  and
  claude
  committed
  01f89b2
- Rewrap the EP allowance comment
  
  bradhilton
  and
  claude
  committed
  ae10b7e
Commits on Sep 26, 2026
- Merge main (0e0c31b32) into the recomputed-mixer floor
  
  bradhilton
  and
  claude
  committed
  fbe6943
- Keep boundary gradients above CP2; price TE beside the combine extent
  
  bradhilton
  and
  claude
  committed
  3d1d18e
Commits on Sep 27, 2026
- Price the adapter gradients a recompute backward holds at its peak
  
  bradhilton
  and
  claude
  committed
  f0670b9
- Count head-tied and custom checkpoint gradients as live throughout
  
  bradhilton
  and
  claude
  committed
  ecf2ce2
- Order slot names safely and leave base-model groups out of adapter slots
  
  bradhilton
  and
  claude
  committed
  067048e
- Merge the adapter-gradient floor (#1002 at ecf2ce21b) into the recomputed-mixer floor
  
  bradhilton
  and
  claude
  committed
  789db66
Commits on Sep 28, 2026
- Test that base-model gradient groups price and defer nothing
  
  bradhilton
  and
  claude
  committed
  c35b238
- Merge main (762c89dfa) into the adapter-gradient floor
  
  bradhilton
  and
  claude
  committed
  f1fe9e7
- Merge the adapter-gradient floor (#1002 at f1fe9e712, main 762c89dfa)
  
  bradhilton
  and
  claude
  committed
  ea0ae1e
- Walk gradient groups' backward one after another, in the worst order
  
  bradhilton
  and
  claude
  committed
  e54f4bc
- Merge #1002's sequential gradient groups (e54f4bc25)
  
  bradhilton
  and
  claude
  committed
  b2b66e4
- Price gradient groups' worst backward order in closed form
  
  bradhilton
  and
  claude
  committed
  0df629c
- Merge #1002's closed-form group order (0df629cf5)
  
  bradhilton
  and
  claude
  committed
  3700b37
- Test that a layer-count mismatch prices no adapter-gradient extra
  
  bradhilton
  and
  claude
  committed
  2a64195
- Merge the adapter-gradient floor's layer-mismatch test (#1002 2a6419528)
  
  bradhilton
  and
  claude
  committed
  fd5ef3a
- Price the head and decoder recompute backward stages apart
  
  bradhilton
  and
  claude
  committed
  23f4f95
- Price each row's RoPE and index state in the head stage
  
  bradhilton
  and
  claude
  committed
  06a3bb6
- Stage only traced head backwards; keep the unstaged price for others
  
  bradhilton
  and
  claude
  committed
  e198239
- Trust staging only after the fused head statistics have run
  
  bradhilton
  and
  claude
  committed
  6a249ec
- Learn memory profiles without the TE workspaces a wave first allocates
  
  bradhilton
  and
  claude
  committed
  cd7539f
- Revert TE-free profile readings
  
  bradhilton
  and
  claude
  committed
  e88b5f3
- Run a staged plan's fused head statistics strictly
  
  bradhilton
  and
  claude
  committed
  8aecea6
- Prove the target-only fused kernel itself before staging
  
  bradhilton
  and
  claude
  committed
  b0c9cfa
- Bind strict head statistics per thread, gradient group and staged admission
  
  bradhilton
  and
  claude
  committed
  35850f7
- Merge remote-tracking branch 'origin/main' into dalinar/adapter-grad-floor
  bradhilton
  committed
  f0e11e0
Commits on Sep 29, 2026
- Merge remote-tracking branch 'origin/main' into dalinar/adapter-grad-floor
  bradhilton
  committed
  d3093bf
- Replay the adapter-gradient floor from frozen slot facts
  
  bradhilton
  and
  claude
  committed
  be1f15d
- Merge remote-tracking branch 'origin/main' into dalinar/adapter-grad-floor
  bradhilton
  committed
  a9135c1
- Narrow the replay capture's slot name before sizing it
  
  bradhilton
  and
  claude
  committed
  b8a2fd1
- Bound replay's recorded layer count and tighten adapter facts
  
  bradhilton
  and
  claude
  committed
  6b1b918
- Patch the reader with monkeypatch in the custom-reader test
  
  bradhilton
  and
  claude
  committed
  eedef3d
- Merge remote-tracking branch 'origin/main' into dalinar/adapter-grad-floor
  bradhilton
  committed
  af503a0
- Validate adapter pending facts before scanning them
  
  bradhilton
  and
  claude
  committed
  a4f05f8
- Refuse mixed slot kinds in facts and a zero layer count at capture
  
  bradhilton
  and
  claude
  committed
  3fa5dbf
- Name the refusal each forged adapter fact must hit
  
  bradhilton
  and
  claude
  committed
  c41148e
- Price the recomputed mixer and staged head backward on the extracted layout
  
  bradhilton
  and
  claude
  committed
  e6edc93
- Record #963's pre-extraction history (35850f744) as ported
  
  bradhilton
  and
  claude
  committed
  73088e4
- Harden replay of the recomputed layer's readers
  
  bradhilton
  and
  claude
  committed
  e39b460
- Cover the staged CP2 head and recorded geometry in replay tests
  
  bradhilton
  and
  claude
  committed
  bab7be1
- Run the head-stage memory tests in the Megatron lane
  
  bradhilton
  and
  claude
  committed
  1763c67

Commits

Commits on Sep 25, 2026

Commits on Sep 26, 2026

Commits on Sep 27, 2026

Commits on Sep 28, 2026

Commits on Sep 29, 2026