[wip ai slop] Implement Phase 1 native H100 pilot / 实现阶段 1 原生 H100 试点 - #3299
functionstackx wants to merge 18 commits into
Conversation
…ation Land hosted source/publication issuance, exact immutable artifact transport, and independently tested Python client foundations without changing native pilot selection, matrix routing, or allocation. Document reader-first deployment and reviewed issuer/reader pins before a producer qualification change. 中文：在 native 验收前加入受信测量回执控制。提供托管源回执与发布记录 签发、准确的不可变产物传输，以及经过独立测试的 Python 客户端基础； 不改变 native pilot 选择、矩阵路由或资源分配。文档明确先部署读取端， 再部署已审查签发版本，最后通过独立变更开展产出端验收。 Validation: all 41 tests in the five requested suites pass, including installed wheel resources; Ruff checks and formatting pass for all 109 infx files. The four changed workflows pass the strict auditor-mode zizmor check locally (offline action-reference checks); no Slurm or production database execution. 验证：指定五组测试共 41 项全部通过，其中包含安装后的 wheel 资源验证； 全部 109 个 infx 文件通过 Ruff 检查和格式检查。四个变更工作流通过本地 严格 auditor 模式的 zizmor 检查（未进行在线 action 引用验证）；未执行 Slurm 任务，也未写入生产数据库。
中文：在独立的可信回执前置能力之上启用 H100 聚合 TP8 原生试点，保留八个吞吐点和独立 c28 真实 eval，加入显式节点需求、预备式任务身份、原生提交恢复及所有权清理、双语部署说明和跨仓库行为验证。正式硬件与发布验收仍待 GitHub sweep 完成。
|
Thanks for the contribution!
中文感谢你的贡献！
|
中文：仅在新增 changelog 尾部记录实际 PR #3299，保留全部历史字节。
中文：在分配前验证离线模型 refs/main、资源绑定及规范 snapshot 路径与服务端完全一致；保留 tokenizer 名称兼容性。新增 Linux CI 任务，核实运行时 pin、依赖锁与 NVIDIA 版本谱系，在独立非 editable 环境中运行原生单元测试及九点已安装运行时边界检查，并同步双语文档。
中文：同步 main 的 B200 DSpark6 更新，逐字节保留当前 main changelog，并仅在尾部追加阶段 1 自有记录，修复 PR sweep 的 append-only 比较。
|
View unofficial run (performance): https://inferencex.semianalysis.com/inference?unofficialRun=35487222760 View unofficial run (accuracy): https://inferencex.semianalysis.com/evaluation?unofficialRun=35487222760 |
使用 uv 管理的 Python 3.12 启动原生试点，修复 H100 runner 缺少 python 命令的问题；在准备和 Slurm 申请前明确报告缺失或不匹配的站点与部署变量，并增加行为回归测试。
Hash ZIP members in bounded chunks, retain CRC and measured-size validation, and parse evaluation samples one line at a time. Cover real large archives, bounded memory, corrupted CRCs, and false member sizes. 分块计算回执 ZIP 成员哈希，保留 CRC 与实际大小校验，并逐行解析评估样本。新增真实大归档、内存上限、CRC 损坏及成员大小不符的行为测试，并同步中英文文档。
合入受信任 collector 的流式回执校验，避免大体积 AgentX 归档和评估样本造成内存峰值过高。
记录真实 GitHub CI 通过结果及 H100 sweep 在 Slurm 提交前因站点和部署变量缺失而停止的证据，不将本地或 CPU 检查等同于硬件验收。
将 H100 DSV4.1 Flash 配方移入现有 model/engine/GPU/workload 目录，将运行时锁和客户端策略移入同一配方树的 configs 目录；同步主配置、CI、测试夹具与中英文架构文档。配方字节、生成矩阵和历史性能记录保持不变，仅更新路径引用。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding above, I checked two other candidate concerns raised during the hunt and ruled them out: identity_probe.py's per-file hashing covers every installed distribution by design — it's meant to freeze the full installed runtime identity, not just the requested distributions, so the scope is intentional rather than a bug. And cache.py's prepare/publish paths for the derived cache both take the same per-identity publication.lock (flock) before touching canonical, so two overlapping runs sharing a cache identity serialize or quarantine on mismatch rather than racing.
Extended reasoning...
This run has one CONFIRMED inline finding (benchmark-tmpl.yml:247, the self-hosted-runner git-repair step being skipped for srt-slurm jobs), which by itself signals a human should look at this PR, so approval is not appropriate. Rather than staying fully silent, I independently verified two of the five candidate issues listed as ruled out this run, reading the actual source (infx/benchmarks/identity_probe.py and infx/benchmarks/cache.py) rather than taking the prior ruled-out summaries at face value. The identity_probe hashing-everything behavior is consistent with the PR's stated goal of freezing "installed runtime/client identities" as evidence, so hashing the full installed set (not just requested distributions) looks like the intended design rather than an oversight. The cache.py MmapCache class uses a single publication.lock file per cache identity directory, taken via fcntl.flock in both prepare() and publish(), with quarantine-on-mismatch and atomic rename-based publication, which is a reasonable mitigation for the concurrent-overlapping-run race that was raised and ruled out. I did not re-verify the other three ruled-out items (stage-results.yml receipt hard-fail behavior, and the other cache/identity variants) or attempt to independently confirm the inline CONFIRMED finding's severity beyond what's already posted, so this note is narrowly scoped to what I personally checked.
绑定阶段 1 引擎实际并行拓扑，增加通过 CI 检查 H100 共享资源的显式入口，并同步中英文文档。
准备并验收原生 H100 试点运行环境；PR 诊断与发布分离，保留完整九点测量契约，修复中断清理并验证监听端口归属。 Validation: 1920 producer tests and 207 subtests passed; installed native boundary covers all nine points. Ruff, MCP compatibility, workflow actionlint and authenticated Zizmor passed. Real H100 qualification remains pending.
修复私有 Hugging Face 引用格式，按固定版本准备 AgentX 数据集缓存并逐行验证离线加载，使用正式兼容补丁校验 lm-eval 客户端行为。
使用符合原生约束的取消验证 intent，保留错误证据并清晰报告原生命令失败，覆盖实际固定运行时边界。
复用实际测量点的服务参数渲染逻辑，固定 c28 eval 的序列与图捕获上限，并记录真实失败及清理证据；不放宽生命周期验收要求。
固定已验证源码的原生运行时，保留全部历史 changelog 字节及完整九点负载，并同步中英文候选验证状态。
更新中英文函数与文件调用图，标明正式 sweep 和取消探针共用服务参数渲染函数。
同步中英文验证账本，保留已取消运行的真实证据及新环境准备和硬件验收的待完成状态。
![github-code-quality[bot]](https://avatars.githubusercontent.com/u/9919?s=60&v=4){kind=small}
| assert ( | ||
| subprocess.check_output( | ||
| ["git", "-C", str(fresh), "diff", "--cached", "--name-only"], text=True | ||
| ).strip() | ||
| == "candidate.txt" | ||
| ) |
| assert read_json(generation / "state.json") == { | ||
| "state": "failed", | ||
| "error_type": "RuntimeError", | ||
| "qualification_complete": False, | ||
| } |
| report["trigger"] = await_trigger( | ||
| native, receipt_path, receipt, directory, namespace, mode, observation_timeout_seconds | ||
| ) | ||
| except BaseException as caught: # noqa: BLE001 - signals must retain evidence and close owned jobs |
| if mode == "client": | ||
| report["writer_closure"] = verify_closed_writer(directory, receipt, namespace) | ||
| report.update(state="passed", lifecycle_qualified=True) | ||
| except BaseException as cleanup_error: # noqa: BLE001 - preserve both execution and cleanup failures |
| if state.exists() and (pid := json.loads(state.read_text()).get("writer_pid")): | ||
| try: | ||
| os.kill(pid, signal.SIGKILL) | ||
| except ProcessLookupError: |
This branch has not been deployed
1 participant
Phase 1 moves
dsv41flash-fp4-h100-vllm-agentic-dsparkonto prepared native srt-slurm execution: one exclusive H100 node, one direct vLLM TP8 worker, and Python-owned AgentX/eval clients. Same-repository PR sweeps can qualify the real workload without deploying an app reader or collector. Qualification artifacts are ineligible for publication.Implementation
require-poweris rejected.benchmarks/multi_node/srt-slurm-recipes/hierarchy.62beb5ec4f8c33abc26851ba0adaded29957ca5bfrom native PR #21. Native code owns allocation, durable submission intents, listener ownership, observation and bounded ownership-scoped cleanup. On the actual Step Manager cluster,squeue --stepsomitted a running worker; the fix queries job-specificscontrolstep records. Cancellation probes now share the real c28 eval serving-argument renderer.Qualification and publication
The complete native PR matrix uses
PreparedSiteandpurpose: pr-qualification. It emits one intent, nine point artifacts and a separately validated summary. Acceptance checks matrix/source identities, frozen digests, resources and terminal closure, raw/normalized AgentX results, and the full GSM8K corpus and score.publication_eligibleis always false.This route emits no normal benchmark/eval artifacts or
RESULT_FILENAME, skips normal collection/import, and is rejected by reuse and receipt/publication validators even in mixed inventories. Publication still requires independently deployed reader/collector identities. App reader PR #1179 was reverted by the separately user-authorized PR #1180. The additive schema remains; native ingestion/publication remain gated. This PR, native #21 and collector #3298 remain unmerged.Validation and remaining hardware work
Current head:
c3db6c7df5c0dbe5222eccff9a4403707c5c1bed. Its actual GitHub test-merge commit is19a0d648932ec5f668fecb26f0c377bdd229cf49; this is not a merged PR. Compared with candidate17423228f1f4ad225233364ba53b3f36f90a818b, only the English/Chinese guides changed.19a0d648...with the same counts and installed nine-point contract. Security 35485312486 and CodeQL 35485311143 passed. The unlabeled sweep skipped every GPU/publication job.50c3dacc..., including real offline client probes and full asset identities. Replacement 35484558552 passed all 38 stages on the true candidate branch commit17423228...with native62beb5ec.... All 168 native source/package files match the pinned source. The real installed-wrapper verifier passes against test-merge19a0d648...; runtime inputs are identical. The exact returnedPreparedSiteis configured and read back byte-for-byte. Its recorded provisioning source remains17423228....18324to terminal closure but missed the worker trigger. Client probe 35483590787 endedFAILEDduring vLLM initialization before its writer started. Neither lifecycle gate passed. New step discovery and c28 parity need actual replay; CUDA recovery is not assumed.19a0d648...and native62beb5ec...: live owned worker18326.5was observed, the allocation was explicitly cancelled, and native evidence confirms terminalCANCELLED,cleanup_complete:trueandrestoration_success:true. Client 35486685532 also passed: corrected c28 server reached readiness, live client step18327.8was observed, cancellation closed its 17-record writer on SIGTERM and the owned allocation reached terminalCANCELLEDwith native cleanup/restoration complete. No preceding teardown was found. This proves generic native-client cleanup, not in-flight HTTP cancellation or graceful model shutdown. The solefull-sweep-enabledlabel now started fresh full sweep 35487222760; all eight 3,600-second throughput points, full c28 GSM8K evaluation, strict summary and baseline comparison remain pending.Architecture, function/file call diagrams and qualification ledger: English guide / 中文指南.
AI model disclosure
This runtime describes Codex as based on GPT-6; the exact model/version identifier was not exposed. The primary agent integrated and reviewed the work. Delegated agents using the inherited runtime implemented or reviewed native execution/ownership, clients, provisioning, cancellation and publication boundaries.
中文
阶段 1 将 H100 DSV4.1 Flash AgentX 迁移到 prepared native srt-slurm：独占一个 H100 节点，一个直连 vLLM TP8 worker，Python 负责 AgentX/eval 客户端。同仓库 PR 可以在不部署 reader/collector 的情况下运行真实验收，但产物不能发布。
保留现有镜像、模型、五 token DSpark、golden AL 和完整工作负载：c1、2、4、8、16、20、24、28 各测量 3,600 秒，另行运行覆盖全部 1,319 文档及两个 filter 的真实 c28 GSM8K。功耗仍采用明确的临时一致性例外。Recipe 沿用现有目录；共享 gitlink、参考仓库和依赖拥有的 Bash 保留。范围是迁移路径不使用 InferenceX 自有 Bash。
原生 pin 为
62beb5ec4f8c33abc26851ba0adaded29957ca5b，负责分配、持久化 intent、监听归属、观察和有界的所属资源清理。实际 Step Manager 集群中squeue --steps漏报运行中的 worker；修正使用指定任务的scontrolstep 记录。取消探针现在与真实 c28 eval 共用服务参数渲染函数。验收路径使用
PreparedSite，输出 intent、九点产物和严格汇总，验证准确 source/矩阵、摘要、资源/终态、AgentX 原始结果及完整 GSM8K 文档/分数。始终记录publication_eligible: false，不生成常规导入产物。Reuse 与发布校验会拒绝这些诊断结果。App reader #1179 已由用户单独批准的 #1180 回退；新增 schema 保留，原生导入/发布仍受限制。本 PR、native #21 和 collector #3298 均未合并。当前 head 为
c3db6c7df5c0dbe5222eccff9a4403707c5c1bed，GitHub 测试用 merge commit 为19a0d648932ec5f668fecb26f0c377bdd229cf49，并非已合并 PR。相比候选17423228...，仅修改中英文指南。19a0d648...上以相同计数通过，九点安装契约通过；security 35485312486 与 CodeQL 35485311143 通过。无 sweep 标签的运行跳过全部 GPU 与发布任务。17423228...与 native62beb5ec...。168 个原生源码/包文件一致；实际 installed-wrapper 校验针对准确测试 merge19a0d648...通过，运行时输入一致。已配置并逐字节读回返回的 PreparedSite，记录的准备 source 仍为17423228...。18324取消并确认关闭，但未满足 worker 触发条件；客户端探针 35483590787 在 writer 启动前因 vLLM 初始化失败而终止。两项生命周期门禁均未通过。新 step 发现和 c28 参数一致性仍须实际重跑，不能预先假定 CUDA 错误已恢复。19a0d648...与原生62beb5ec...，观察到所属运行 worker18326.5，明确取消 allocation 后确认终态CANCELLED，且原生记录表明清理和恢复均完成。客户端检查 35486685532 也已通过：修正后的 c28 服务就绪、客户端 step18327.8运行，取消后 writer 以 SIGTERM 关闭且 17 条记录完好，唯一所属 allocation 达到终态CANCELLED，原生清理与恢复均完成；未发现此前已开始 teardown。这证明通用原生客户端清理，不代表进行中 HTTP 请求取消或模型优雅退出。已添加唯一full-sweep-enabled标签，启动新的 完整 sweep 35487222760；八个 3,600 秒吞吐点、完整 c28 GSM8K、严格汇总及基线比较仍待完成。中英文指南均保留架构与函数/文件调用图。运行时将 Codex 描述为基于 GPT-6，精确版本未暴露；主代理集成审查，继承 runtime 的子代理负责原生执行、客户端、准备、取消与发布边界的实现或审查。