DDIR Corgi chunks: reuse sort labels during consolidation - #908
Merged
Merged
Conversation
Corgi already refines labels to identify equal (key, value) runs when sorting. Keep these labels instead of gathering the sorted payload and comparing its adjacent rows again. Use original row coordinates through consolidation and gather only the surviving rows, once. Drop the redundant payload index vector. Retain the existing zero-copy path for ordered, surviving payloads, including time reordering inside equal-value runs. Co-Authored-By: Astra (OpenAI Codex) Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This was referenced Sep 28, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Consolidating a columnar chunk used to gather the sorted payload and then compare adjacent rows again. Corgi's sort already computes equality labels for
(key, value)runs. This keeps those labels, works in original row coordinates, and gathers only the surviving rows, once. It also drops the extra payload index vector. The zero-copy path for payloads that are already ordered and all survive is kept, including time reordering inside runs of equal values. One file, +63/−15, including an oracle test over payload, time and cancellation.Measured on SCC, the worldgen tour, and the stock AoC runner. Three paired runs each, order reversed in the middle run, M4 release build:
Slower cases.
Fewer copies overall does not mean a lower peak, and this change does not save memory.
Context. In worldgen, on top of other changes, this gives first view −7.1% / −5.7% and tour −7.3% / −5.8% at 1 / 3 workers.
Tests.
cargo test --release -p interactive: 105 passed, 5 ignored (all ignored before this change too). All 33 AoC answers match. The supported sessions and the graph references match on both backends.This PR and the uniform-timestamp PR edit the same function. They were measured separately and do not depend on each other. Whichever lands second will need a small rebase.
Measurements and the port are by Astra (OpenAI Codex). I re-ran the tests on master-next 9cfc909.
🤖 Generated with Claude Code