Skip to content

feat: add Moore Threads multi-GPU Suite E and Suite B results - #62

Draft
Keke946 wants to merge 4 commits into
FreedomIntelligence:mainfrom
Keke946:feat/moorethreads-suite-e-tp
Draft

Keke946 wants to merge 4 commits into
FreedomIntelligence:mainfrom
Keke946:feat/moorethreads-suite-e-tp

Conversation

@Keke946

@Keke946 Keke946 commented Sep 8, 2026

Copy link
Copy Markdown

Summary

  • Add and validate multi-chip Suite E support for the Moore Threads vLLM-MUSA runner.
  • Submit MTT S4000 Suite E scaling results across 1, 2, 4, and 8 GPUs.
  • Add a four-GPU Suite B result for Meta-Llama-3-70B-Instruct.
  • Add successor runner moorethreads_vllm_musa_6607740b, fixing MUSA chip-count detection when pymtml is unavailable.

Suite E results

GPUs Best output throughput Speedup vs. 1 GPU Scaling efficiency
1 409.71 tok/s 1.00x 100.0%
2 498.36 tok/s 1.22x 60.8%
4 570.06 tok/s 1.39x 34.8%
8 530.78 tok/s 1.30x 16.2%

The four-GPU configuration gives the highest throughput; at eight GPUs, communication and synchronization overhead outweigh the added compute.

Suite B four-GPU result

  • Hardware: 4 x MTT S4000 48 GB
  • Model: Meta-Llama-3-70B-Instruct
  • Precision: FP16
  • Runner: moorethreads_vllm_musa_6607740b
  • Run ID: 91e1cc05
  • Accuracy: 0.33, valid
  • Offline output throughput: 148.00 / 147.71 / 147.51 tok/s at client concurrency 8 / 32 / 128
  • Online QPS 2, 5, 10, and 25 completed; all missed the 1 s TTFT p99 SLA, so max_valid_qps is 0

Validation

  • Suite E completed at 1x, 2x, 4x, and 8x tensor parallel configurations.
  • Suite B accuracy, offline, and online scenarios completed successfully.
  • The Suite B submission passes validate_submission.py.
  • Power telemetry is unavailable, so tokens/sec/watt is not reported.

@Keke946 Keke946 changed the title feat: add Moore Threads multi-GPU Suite E support feat: add Moore Threads multi-GPU Suite E and Suite B results Sep 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant