[AgentX GB300] refresh the full Kimi-K3 curve so every point carries measured power / 重测完整曲线使每个点都带实测功耗 - #3315
edwingao28 wants to merge 4 commits into
Conversation
|
Thanks for the contribution!
中文感谢你的贡献！
|
The 6 published GB300 Kimi-K3 disaggregated points come from the 2026-09-14 sweep, whose recipes did not yet declare the dcgm-power provider, so the job logs report USES_DCGM_POWER=0 and the points carry power_valid=0 with no power values. #3047 enabled the provider on those recipes, but its run was purged, so nothing power-enabled has been published for them. AgentX collapses spec_method, disagg and offload_mode into the curve scope, so a hardware resolves to exactly one curve and a partial selection would replace it rather than complete it. The entry selects both GB300 config keys and re-measures the full 11-point curve. 中文：已发布的 6 个 GB300 Kimi-K3 分离部署点来自 2026-09-14 的 sweep， 当时配方尚未声明 dcgm-power provider，作业日志显示 USES_DCGM_POWER=0， 这些点因此是 power_valid=0 且没有功耗数值。#3047 已为这些配方启用该 provider，但其运行被清除，所以至今没有带功耗的已发布数据。AgentX 把 spec_method、disagg 与 offload_mode 折叠进 curve scope，每个硬件只解析出 一条曲线，只选一部分会替换而不是补全曲线。本条目选中两个 GB300 config key，一次重测完整的 11 点曲线。
201259a to
833de7a
Compare
edwingao28
added
full-sweep-enabled
priority
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35532106109 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35532106109 |
srtctl rejected the four disagg recipes with `Invalid config ...
{'telemetry': {'provider': ['Unknown field.'], 'default_frequency':
['Unknown field.']}}` before submitting anything. When utils/srt-slurm
moved to the upstream pin, `provider` and `default_frequency` were
retired for `dcgm_exporter` and `collect_interval_ms`; 140 recipes
migrated and six Kimi-K3 ones did not. That is why the disaggregated
half of the GB300 curve failed in `Launch multi-node job script` while
the aggregated half, already migrated, passed.
default_frequency was a period in seconds, so 1.0 becomes
collect_interval_ms: 1000, matching agg-dcp8-dspark4-mooncake.yaml in
the same directory.
中文：srtctl 以 `Invalid config ... {'telemetry': {'provider':
['Unknown field.'], 'default_frequency': ['Unknown field.']}}` 拒绝了四个
分离部署配方，尚未提交任何任务。utils/srt-slurm 切到上游 pin 时，provider
与 default_frequency 已被 dcgm_exporter 和 collect_interval_ms 取代；140 个
配方完成了迁移，六个 Kimi-K3 配方没有。这正是 GB300 曲线中分离部署的一半在
`Launch multi-node job script` 阶段失败、而已迁移的聚合部署一半通过的原因。
default_frequency 的单位是秒，因此 1.0 对应 collect_interval_ms: 1000，与同
目录下的 agg-dcp8-dspark4-mooncake.yaml 一致。
|
Claude finished @edwingao28's task in 1m 12s —— View job Review of PR #3315
LGTM - no blocking issues found. The four disagg recipes now carry the same Note: the |
|
/use 35532106109 |
精简 GB300 Kimi-K3 更新日志，保留完整曲线功耗重测与 telemetry schema 迁移说明。 [skip-sweep] Wording only; benchmark configuration and selection are unchanged.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Looks good, straightforward config/schema-migration change.
What was reviewed: the four disagg recipe diffs (each replaces provider: dcgm-power / default_frequency: 1.0 with collect_interval_ms: 1000) — confirmed this matches the already-migrated telemetry: block (same collect_interval_ms: 1000 + dcgm_exporter shape) used in the aggregated Kimi-K3 GB300 recipes in the same directory. Checked the perf-changelog.yaml diff — the new entry is appended cleanly at the tail with no changes to prior bytes, and is well-formed YAML with both config-keys present.
Extended reasoning...
Overview
This PR changes four GB300 Kimi-K3 vLLM AgentX disaggregated srt-slurm recipe YAMLs, replacing the retired provider: dcgm-power / default_frequency: 1.0 telemetry fields with the current schema's single collect_interval_ms: 1000 field, and appends one new entry to perf-changelog.yaml documenting the change. No scripts, launchers, master configs, or Python source are touched.
Security risks
None. This is a pure declarative config change to YAML recipe files consumed by an internal benchmark submission tool (srtctl); no user input, auth, or data handling is involved.
Level of scrutiny
Low. The change is mechanical and repeats the identical edit across four files, and I verified the resulting telemetry: block shape is byte-identical in structure to the already-migrated aggregated recipes in the same directory (agg-dcp8-dspark4-mooncake.yaml, agg-dcp8-dspark7-maxseq2-mooncake.yaml), which already use collect_interval_ms: 1000 with the same dcgm_exporter sub-block. This strongly supports the PR's claim that these four disagg recipes were simply lagging behind the schema migration that 140 other recipes already underwent.
Other factors
The perf-changelog.yaml entry follows the append-only convention (new block added after the existing tail entry, no prior bytes altered) and includes both required config-keys (agg and disagg) consistent with the PR's explanation that AgentX resolves one curve per hardware. No CODEOWNERS restriction is evident for these paths, and the bug-hunting system reported no findings. Given the narrow, mechanical, and well-corroborated nature of the change, no human review appears necessary.
This review covers commit ac57ada, which is no longer the latest commit on this pull request; later commits are not covered by it.
中文：合并 main，保留已有记录并在末尾追加本 PR 条目。 [skip-sweep]
This branch has not been deployed
1 participant
Description
The 6 published GB300 Kimi-K3 disaggregated points come from the 2026-09-14 sweep, whose recipes did not yet declare the
dcgm-powerprovider — its job logs reportUSES_DCGM_POWER=0and the points carrypower_valid=0with no power values. #3047 enabled the provider but its run was purged, so nothing power-enabled has been published. AgentX resolves one curve per hardware, so only a sweep selecting both config keys can replace them together. Also migrates the four disagg recipes to the current srtctl telemetry schema.provideranddefault_frequencywere retired fordcgm_exporterandcollect_interval_mswhenutils/srt-slurmmoved to the upstream pin; 140 recipes migrated and six Kimi-K3 ones did not, so srtctl rejected these four asUnknown fieldbefore submitting anything — the disaggregated half of the curve failed while the already-migrated aggregated half passed.Testing: all four telemetry blocks now load under the pinned srtctl schema and fail to load without the change. The matrix plan resolves 11 benchmark + 11 eval jobs on one image, 5 aggregated + 6 disaggregated, matching the published curve. Sweep running.
中文
已发布的 6 个 GB300 Kimi-K3 分离部署点来自 2026-09-14 的 sweep，当时配方尚未声明
dcgm-powerprovider —— 作业日志显示USES_DCGM_POWER=0，这些点是power_valid=0且没有功耗数值。#3047 已启用该 provider，但其运行被清除，因此至今没有带功耗的已发布数据。AgentX 每个硬件只解析出一条曲线，只有同时选中两个 config key 的 sweep 才能整体替换它们。同时将四个分离部署配方迁移到当前的 srtctl telemetry schema。utils/srt-slurm切到上游 pin 时，provider与default_frequency已被dcgm_exporter和collect_interval_ms取代；140 个配方完成迁移，六个 Kimi-K3 配方没有，因此 srtctl 在提交任何作业前就以Unknown field拒绝了这四个配方 —— 曲线中分离部署的一半失败，而已迁移的聚合部署一半通过。测试： 矩阵规划解析出 11 个 benchmark + 11 个 eval 作业，同一镜像，5 个聚合 + 6 个分离部署，与已发布曲线一致。sweep 尚未启动。
Related Issue
Follows #3046 / #3047 / 承接 #3046、#3047。
Type of Change