[Klaud Cold] Update dsr1-fp8-mi325x-sglang SGLang image to v0.5.20-rocm720-mi30x / 将 dsr1-fp8-mi325x-sglang 的 SGLang 镜像更新至 v0.5.20-rocm720-mi30x

[adibarra]

Goal: Update SGLang image from lmsysorg/sglang:v0.5.19-rocm700-mi30x to lmsysorg/sglang:v0.5.20-rocm720-mi30x.
Baseline: 2026-09-10 · lmsysorg/sglang:v0.5.19-rocm700-mi30x
8k/1k · TP8/EP1 · Mean latency · Sources: API 1, API 2, API 3

Concurrency	Total tok/s/GPU ↑	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
4	223.94	24.91	1,025.49	18.53
8	385.84	43.38	1,026.66	21.56
16	626.19	69.39	1,163.49	26.75
32	854.62	95.57	1,363.26	39.45
64	1,035.63	114.89	1,917.85	66.55

Eval	Score ↑	Samples
gsm8k/em_strict · c32	95.6%	1,319
gsm8k/em_strict · c64	95.38%	1,319

Status: Klaud Cold could not publish the typed baseline or any typed report: the session credential is not the Klaud-Cold login, so the lifecycle helper rejects this PR as not owned. The baseline above was frozen locally by prepare-baseline from the public API (2026-09-10, producer run 34429203486, head 453890e61466) and rendered with the canonical renderer. The updated-image smoke run 35511687735 passed (c4 throughput and the c32 gsm8k eval) and the perf-changelog entry is appended; the final full sweep was not dispatched because the helper's check-final/finish gates cannot pass under this credential. See the Klaud comment for the source comparison, results and the blocker.

AI model disclosure

Prepared by Claude Fable 5.1 (claude-fable-5-1) running as the autonomous Klaud Cold candidate session; no other models or delegated agents were used.

中文

**目标：**将 SGLang 镜像从 lmsysorg/sglang:v0.5.19-rocm700-mi30x 更新为 lmsysorg/sglang:v0.5.20-rocm720-mi30x。
**基线：**2026-09-10 · lmsysorg/sglang:v0.5.19-rocm700-mi30x
8k/1k · TP8/EP1 · 平均延迟 · 来源： API 1, API 2, API 3；数值及异常说明见上表。

**状态：**Klaud Cold 无法发布类型化基线或任何类型化报告：会话凭据不是 Klaud-Cold 登录名，生命周期助手将本 PR 判定为非自有。上表基线由 prepare-baseline 从公开 API 本地冻结（2026-09-10，生产运行 34429203486，head 453890e61466）并用规范渲染器渲染。更新镜像的 smoke 运行 35511687735 已通过（c4 吞吐与 c32 gsm8k 评测），perf-changelog 条目已追加；由于助手的 check-final/finish 关口在此凭据下无法通过，未调度最终完整 sweep。源码比对、结果与阻塞原因见 Klaud 评论。

**AI 模型披露：**由 Claude Fable 5.1（claude-fable-5-1）作为自主 Klaud Cold 候选会话准备；未使用其他模型或委派代理。

🤖 Generated with Claude Code

[github-actions]

Thanks for the contribution!

• Review: If this PR changes files owned by someone other than a repository admin or @SemiAnalysisAI/core, ask one eligible CODEOWNER to complete the latest PR_REVIEW_CHECKLIST.md before contacting a core maintainer on Slack. Follow the template exactly, including As a PR reviewer and CODEOWNER, I have reviewed this and have, so sign-off verification triggers.
• PR verification: Sweeps only run on labeled PRs. Add full-sweep-fail-fast (strongly recommended); use full-sweep-enabled only when matrix jobs should continue after a failure.
• After merging: PR authors must ensure all GitHub Actions jobs pass. Transient failures often pass on rerun; see how to rerun failed jobs.

中文

感谢你的贡献！

• **审阅：**如果 PR 修改的文件归属于仓库管理员及 @SemiAnalysisAI/core 之外的 CODEOWNER，请先联系一位有资格的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，再通过 Slack 联系核心维护者。必须严格遵循模板，并保留 As a PR reviewer and CODEOWNER, I have reviewed this and have，才能触发签核验证。
• **PR 验证：**扫描仅在带有标签的 PR 上运行。强烈建议添加 full-sweep-fail-fast；仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。
• **合并后：**PR 作者必须确保所有 GitHub Actions 任务通过。临时性失败通常可以通过重新运行恢复；参见重新运行失败任务的说明。

[adibarra]

Initial attempt · Passed · Run 35511687735 / attempt 1 · 2026-09-20 13:21 UTC
lmsysorg/sglang:v0.5.20-rocm720-mi30x · 0762e65d6101 · 8k/1k · TP8/EP1 · Mean latency
Change: Update the master image to the SGLang v0.5.20 ROCm 7.2 MI30x release image (tag commit 94602c9c, Docker Hub digest sha256:4abecde156b531809c7f597e501930ea2023deb2c822b69f92efb66f3fdada86) and rename --cuda-graph-max-bs to --cuda-graph-max-bs-decode in the family's unshared recipe, because v0.5.20 removed the deprecated alias (#38375); see the source comparison below.

Concurrency	Output tok/s/GPU ↑	TTFT ms ↓	TPOT ms ↓
4	23.19 (-6.9%)	1,332.28 (+29.9%)	19.68 (+6.2%)

Note: All rows: request errors unavailable.

Eval	Score ↑	Samples
gsm8k/em_strict · c32	95.68% (+0.08 pp)	1,319 each

Next: Maintainer decision: restore the Klaud-Cold identity for the candidate credential, or adopt this PR (changelog entry appended at head 89cf7a165dc4, final matrix of 5 points and 2 gsm8k evals generated) and apply full-sweep-fail-fast; it stays draft with no sweep labels.

Smoke result: both jobs green on mi325x-amds_01 (c4 throughput, 40/40 requests, 0 errors) and mi325x-amds_00 (c32 gsm8k eval, infrastructure_success: true). Effective server args confirm cuda_graph_max_bs_decode: 128, attention_backend: aiter, kv_cache_dtype: fp8_e4m3 and mem_fraction_static: 0.68, i.e. the recipe's 0.8 scaled by the aiter context_len > 8192 rule that is identical at v0.5.19. No server errors or tracebacks; only the usual Ignore import error lines for unrelated model modules and the FP8 KV scaling factors of 1.0 notice. Power validity is reported as validation_artifact_unavailable for this targeted dispatch. The c4 throughput regressed against the 2026-09-10 baseline (different node than the baseline producer; single point, not a curve), and the eval delta is within one standard error (about 0.56 pp).

Source comparison (v0.5.19 → v0.5.20): SGLang tag commits 0bcd8223 → 94602c9c. Upstream retired the rocm700 MI30x image lane at v0.5.20 (#38632, #38767); the rocm720 MI30x variant is the closest remaining lane and already runs on this cluster (qwen3.5-fp8-mi325x-sglang on main, and the green MI325X jobs of #2870), so rocm10 was not chosen as an unproven driver pairing. docker/rocm.Dockerfile moves the AITER pin c16d44b9 → 4ad99832 (152 AITER commits, including MLA metadata/decode changes) and MORI 7c51d18f → 879983bd; the gfx942-rocm720 base stays rocm/pytorch:rocm7.2_ubuntu22.04_py3.10_pytorch_release_2.9.1; sglang-kernel 0.4.6.post1 → 0.4.7. Every flag in benchmarks/single_node/fixed_seq_len/dsr1_fp8_mi325x.sh remains defined in v0.5.20 (arg_groups/fields/{schedule,exec_,model,memory}.py: mem_fraction_static, chunked_prefill_size, num_continuous_decode_steps, max_prefill_tokens, kv_cache_dtype, attention_backend with the aiter choice, disable_radix_cache, context_length, trust_remote_code, --tensor-parallel-size), except --cuda-graph-max-bs: in v0.5.19 it was a DeprecatedAliasStoreAction writing cuda_graph_max_bs_decode, and v0.5.20 deleted it (#38375); with argparse prefix matching it would now be ambiguous against --cuda-graph-max-bs-{decode,prefill}, so the rename to --cuda-graph-max-bs-decode=128 preserves the exact prior semantics. SGLANG_USE_AITER (default off, set on by the recipe) and SGLANG_AITER_MLA_PERSIST (read in layers/attention/aiter_backend.py, default on) keep their definitions; the aiter mem_fraction_static 0.85 scaling for context_len > 8192 in arg_groups/attention_hook.py is identical at both tags. models/deepseek_v2.py changes on the HIP path are limited to the router-GEMM dispatch and a ROCm chunked-KV MHA prefill method; aiter_backend.py reworks the h12 Gluon decode selection and head padding. Provenance uncertainty: the image config exposes only the tag name, not the bundled SGLang commit. No engine patch exists on the selected launch path (runners/launch_mi325x-amds.sh → dsr1_fp8_mi325x.sh) and none is added. The image tag is not digest-pinned in the config because enroot import docker:// accepts only IMAGE[:TAG]; the digest is recorded here instead. Weights deepseek-ai/DeepSeek-R1-0528 come from the launcher's shared HF cache mount, as in the 2026-09-10 baseline run.

Blocker · session credential identity: the GitHub credential supplied to this candidate session is not the Klaud-Cold login, so this PR is authored by another login and the lifecycle helper rejects it as not owned (report, check-final, finish and the Stop-hook check-stop fail with Candidate ownership mismatch; runs dispatched here are also invisible to its ownership discovery). The frozen 2026-09-10 baseline (5 points and 2 published gsm8k evals from producer run 34429203486) is rendered in the PR body with the canonical renderer, but the typed record could not be embedded and no verified completion receipt can exist. Same failure as #3279 (2026-09-19) and #3311 (2026-09-20). Capacity for mi325x passed before edits, before branch/PR creation and before this dispatch. Because this candidate's roster is fully coverable and the change carries a real compatibility rename, the smoke run above was dispatched as startup evidence for the maintainer who adopts this PR; it is monitored in this session and reported here.

中文

初次尝试 · 已通过 · Run 35511687735 / attempt 1 · 2026-09-20 13:21 UTC
lmsysorg/sglang:v0.5.20-rocm720-mi30x · 0762e65d6101 · 8k/1k · TP8/EP1 · 平均延迟
**变更：**将主镜像更新为 SGLang v0.5.20 的 ROCm 7.2 MI30x 发布镜像（tag 提交 94602c9c，digest sha256:4abecde1…），并在该配方独有的脚本中把 --cuda-graph-max-bs 改名为 --cuda-graph-max-bs-decode，因为 v0.5.20 移除了该弃用别名（sgl-project/sglang#38375）；源码比对见下文。实测数值及异常说明见上表。
**下一步：**由维护者决定：为候选凭据恢复 Klaud-Cold 身份，或接管本 PR（changelog 条目已在 head 89cf7a165dc4 追加，已生成 5 个点与 2 个 gsm8k 评测的最终矩阵）并添加 full-sweep-fail-fast；本 PR 保持草稿且不带 sweep 标签。

**Smoke 结果：**两个任务均通过（c4 吞吐 40/40 请求、0 错误；c32 gsm8k 评测 infrastructure_success: true）。有效服务端参数确认 cuda_graph_max_bs_decode: 128、attention_backend: aiter、kv_cache_dtype: fp8_e4m3 与 mem_fraction_static: 0.68（配方的 0.8 经 aiter context_len > 8192 规则缩放，v0.5.19 中相同）。无服务端错误或 traceback。此次定向调度的功耗校验为 validation_artifact_unavailable。c4 吞吐相对 2026-09-10 基线有所回退（运行节点与基线生产节点不同；单点而非完整曲线），评测差值在一个标准误（约 0.56 pp）以内。

**源码比对（v0.5.19 → v0.5.20）：**SGLang tag 提交 0bcd8223 → 94602c9c。上游在 v0.5.20 停止发布 rocm700 MI30x 镜像（#38632、#38767），rocm720 MI30x 变体是最接近的现存镜像，且已在本集群上运行（main 上的 qwen3.5-fp8-mi325x-sglang 与 #2870 的 MI325X 绿色任务），因此未选用驱动配对尚未验证的 rocm10。docker/rocm.Dockerfile 将 AITER 固定提交从 c16d44b9 改为 4ad99832（152 个提交，含 MLA 元数据/解码改动），MORI 从 7c51d18f 改为 879983bd；gfx942-rocm720 基础镜像不变；sglang-kernel 0.4.6.post1 → 0.4.7。配方脚本使用的全部参数在 v0.5.20 中仍有定义，唯一例外是 --cuda-graph-max-bs：v0.5.19 中它是写入 cuda_graph_max_bs_decode 的弃用别名，v0.5.20 已删除（#38375），因此改用 --cuda-graph-max-bs-decode=128，语义完全一致。SGLANG_USE_AITER 与 SGLANG_AITER_MLA_PERSIST 的定义与默认值不变；aiter 后端对 context_len > 8192 的 mem_fraction_static 0.85 缩放在两个版本中相同。所选启动路径上不存在引擎补丁，也未新增。因 enroot import docker:// 仅接受 IMAGE[:TAG]，配置中未按 digest 固定镜像，digest 记录于上文。

**阻塞 · 会话凭据身份：**本候选会话使用的 GitHub 凭据并非 Klaud-Cold 登录名，生命周期助手将本 PR 判定为非自有（report、check-final、finish 与 Stop 钩子 check-stop 均报 Candidate ownership mismatch，此处调度的运行也不在其所有权发现范围内）。2026-09-10 的冻结基线（5 个点与 2 个已发布的 gsm8k 评测，生产运行 34429203486）已用规范渲染器渲染到 PR 正文，但类型化记录无法嵌入，也无法生成经验证的完成回执。与 #3279（2026-09-19）、#3311（2026-09-20）为同一故障。mi325x 容量检查在编辑前、创建分支/PR 前与本次调度前均通过。由于本候选的基线点集可完整覆盖且变更包含真实的兼容性改名，上述 smoke 运行作为启动证据已调度，并在本会话中监控与汇报。