[Klaud Cold] Add H200 Qwen3.8-27B-FP8 vLLM TP1 native MTP 1k1k / 新增 H200 Qwen3.8-27B-FP8 vLLM TP1 原生 MTP 1k1k 配方 - #3293
Open
functionstackx wants to merge 7 commits into
Open
functionstackx wants to merge 7 commits into
functionstackx wants to merge 7 commits into
Conversation
functionstackx
requested review from
Ankur-singh and
kedarpotdar-nv
as code owners
September 19, 2026 19:55
… 1k1k recipe on H200 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
functionstackx
added
full-sweep-fail-fast
qwen3.8-27b
functionstackx
force-pushed
the
feat/qwen38-27b-fp8-h200-vllm-mtp
branch
from
September 19, 2026 19:55
43f9940 to
c51e82a
Compare
Contributor
|
Thanks for the contribution!
中文感谢你的贡献！
|
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Contributor
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35497229484 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35497229484 |
Collaborator
Author
|
/reuse-sweep-run 35465800643 |
Sync with origin/main after the green sweep run 35465800643; the reuse gate authorizes that run on this head. 在绿色 sweep 运行 35465800643 之后与 origin/main 同步；reuse gate 在此 head 上授权该运行。 Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
应用 thinking-on 黄金 AL 2.52，保持原始 BF16 原生 MTP 头及草稿 KV cache，并在精度评测路径使用真实验证。
同步 main，逐字节保留 changelog 历史，并在末尾保留本 PR 的原始条目和黄金 AL 更新。
使用官方 --no-enable-flashinfer-autotune 选项规避原生 BF16 MTP 的 FA3 scheduler metadata 初始化错误。
记录已验证的 Qwen 原生 MTP autotune 初始化问题，并链接中英文处理说明及运行证据。
保留 main 新增的诊断记录与 Qwen MTP 的已验证处理说明。
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Goal: Add
qwen3.827b-fp8-h200-vllm-mtpon H200, using native MTP3 with the measured thinking-on synthetic AL 2.52 for throughput.Baseline: N/A — new model/SKU recipe; the acceptance reference is #3304 and SPEED-Bench run 35492787451.
vllm/vllm-openai:nightly-cd10ed6f9f6b37a8ace9cf380007e66fe12ec0c3· TP1 · 1k/1k · concurrency 1–128. The recipe readsthinking_on[3]fromgolden_al_distribution/qwen3.827b_fp8_mtp.yamland sendsrejection_sample_method=synthetic,synthetic_acceptance_length=2.52to vLLM. The pinned Qwen chat template enables thinking; the script requiresTHINKING_MODE=thinking_on.The FP8 target is pinned to
017b9c7af6b5689d5dd426a76e0bc077eb5ca20a. Its native MTP head comes from the original BF16Qwen/Qwen3.8-27Bat1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0, with BF16 draft KV cache and explicit MTP quantization exclusions. Eval-only and combined throughput+eval paths use real standard verification. The shared helper rejects unmeasured draft lengths outside 1–4 and missing or invalid AL values.Validation: 24 behavioral tests, Ruff, Bash syntax, ShellCheck, exact-key matrix generation, and changelog byte-prefix preservation passed. The CLI was exercised against the actual pinned checkpoint metadata for synthetic throughput and real eval configurations. Fresh full GPU sweep is queued. Fresh GPU validation: run 35497229484. The existing reuse authorization refers to the pre-AL recipe; select the new validated full sweep before merging. The CUDA recipe uses
--no-enable-flashinfer-autotuneto avoid the pinned image’s MTP dummy-prefill scheduler-metadata error. The startup fix passed H100 concurrency-1 validation; the fresh full sweep is linked above.AI model disclosure
AL integration and validation: GPT-6; the exact runtime model/version identifier could not be verified. The original PR's AI model/version was not recorded and could not be verified. No delegated agents contributed to this update.
中文
**目标：**在 H200 上新增
qwen3.827b-fp8-h200-vllm-mtp，使用原生 MTP3，并在吞吐测试中应用实测的 thinking-on 合成 AL 2.52。**基线：**N/A——新增模型/SKU 配方；接受长度依据 #3304 和 SPEED-Bench 运行 35492787451。
vllm/vllm-openai:nightly-cd10ed6f9f6b37a8ace9cf380007e66fe12ec0c3· TP1 · 1k/1k · 并发 1–128。配方从golden_al_distribution/qwen3.827b_fp8_mtp.yaml读取thinking_on[3]，向 vLLM 传入rejection_sample_method=synthetic、synthetic_acceptance_length=2.52。固定版本的 Qwen chat template 默认开启 thinking，脚本要求THINKING_MODE=thinking_on。FP8 目标固定为 revision
017b9c7af6b5689d5dd426a76e0bc077eb5ca20a。原生 MTP 头来自原始 BF16Qwen/Qwen3.8-27B，revision 为1d4bf0f2ff6012fd82039f2fa52739d0dd7c60c0；草稿 KV cache 保持 BF16，并显式排除 MTP 量化。仅精度评测以及吞吐后紧接精度评测的路径均使用真实的标准验证。共享辅助程序拒绝超出 1–4 范围的草稿长度，以及缺失或无效的 AL。验证：24 项行为测试、Ruff、Bash 语法、ShellCheck、精确 key 矩阵生成和 changelog 历史字节保留检查均通过。CLI 已使用实际固定版本的 checkpoint 元数据验证合成吞吐配置和真实精度评测配置。新的完整 GPU sweep 仍在排队。 本次更新的 GPU 验证：运行 35497229484。现有复用授权指向 AL 更新前的配方；合并前应选择新验证通过的完整 sweep。 CUDA 配方使用
--no-enable-flashinfer-autotune，避免固定镜像中 MTP dummy prefill 的 scheduler metadata 错误。初始化修复已通过 H100 并发 1 验证；新的完整 sweep 链接见上文。AI 模型披露：AL 集成和验证由 GPT-6 完成；无法核实运行时精确模型及版本标识。原始 PR 未记录 AI 模型及版本，无法核实。本次更新未使用委派代理。