Issues

• **Issue with Dockerized DeepSpeed-MII Persistent Deployment (LLama 3 70B Model) – Tensor Parallelism Not Effective**

  deepspeedai/DeepSpeed-MII#568 · Filter by author InderjeetVishnoi opened on Jul 2, 2025

• ValueError: Unsupported model type qwen3

  deepspeedai/DeepSpeed-MII#566 · Filter by author dgzxx-2000 opened on May 14, 2025

• Question: Is Expert Parallelism Supported?

  deepspeedai/DeepSpeed-MII#565 · Filter by author paprika0741 opened on May 12, 2025

• [Bug] Excessive Memory Usage for Phi-3-Mini

  deepspeedai/DeepSpeed-MII#559 · Filter by author riedgar-ms opened on Mar 13, 2025
  1

• text-generation-bloom560m-example cannot run

  deepspeedai/DeepSpeed-MII#551 · Filter by author helloburke opened on Jan 13, 2025
  1

• ValueError: Unsupported model type mllama

  deepspeedai/DeepSpeed-MII#549 · Filter by author MaorSagi opened on Dec 1, 2024
  1

  

• splitting models across nodes

  deepspeedai/DeepSpeed-MII#547 · Filter by author shahsavari-mhd opened on Nov 26, 2024
  2

• How to use data parallelism in multi gpus inference

  deepspeedai/DeepSpeed-MII#546 · Filter by author hhf-hu opened on Nov 22, 2024

• Issue: Multi-node and Multi-GPU Inference Problems with DeepSpeed MII

  deepspeedai/DeepSpeed-MII#545 · Filter by author lcnmzz00 opened on Nov 20, 2024
  1

• Please clarify structured output support

  deepspeedai/DeepSpeed-MII#544 · Filter by author MRYingLEE opened on Nov 19, 2024

• Updating transformers issue with bloom models

  deepspeedai/DeepSpeed-MII#541 · Filter by author loadams opened on Nov 1, 2024
  

• Updating transformers issue with zero-shot-image-classification

  deepspeedai/DeepSpeed-MII#540 · Filter by author loadams opened on Nov 1, 2024
  

• Logits Processors

  deepspeedai/DeepSpeed-MII#532 · Filter by author psitronic opened on Oct 21, 2024

• need help understanding profiler in deespeed mio

  deepspeedai/DeepSpeed-MII#531 · Filter by author krishnanpooja opened on Oct 7, 2024

• DeepSpeed with Phi-3-mini-128K-instruct does not generate <|endoftext|> token

  deepspeedai/DeepSpeed-MII#529 · Filter by author shubhanshu786 opened on Sep 27, 2024
  1

  

• Repeated token generation with Phi-3-mini for longer context

  deepspeedai/DeepSpeed-MII#528 · Filter by author shubhanshu786 opened on Sep 27, 2024

• LoRA Support

  deepspeedai/DeepSpeed-MII#527 · Filter by author bagelbig opened on Sep 27, 2024

• deepspeed MoE all_to_all communication

  deepspeedai/DeepSpeed-MII#526 · Filter by author miaomiaoma0703 opened on Sep 19, 2024

• multi model deployment

  deepspeedai/DeepSpeed-MII#525 · Filter by author whcjb opened on Sep 13, 2024
  1

• Question About Offloading and Recomputation

  deepspeedai/DeepSpeed-MII#523 · Filter by author lxnlxnlxnlxnlxn opened on Sep 9, 2024

• Configuration setting to pass parameters to tokenizer while encoding and decoding

  deepspeedai/DeepSpeed-MII#522 · Filter by author krishnanpooja opened on Aug 29, 2024

• OpenAI server fails

  deepspeedai/DeepSpeed-MII#521 · Filter by author nivibilla opened on Aug 24, 2024
  3

• By default does deepspeed mii use bf16 dtype or fp16?

  deepspeedai/DeepSpeed-MII#518 · Filter by author krishnanpooja opened on Aug 21, 2024

• FileExistsError: [Errno 17] File exists: '/tmp/mii_cache' ` on generate function call

  deepspeedai/DeepSpeed-MII#516 · Filter by author krishnanpooja opened on Aug 6, 2024

• non-persistent example doesn't work on Mixtral-8*7B-v0.1

  deepspeedai/DeepSpeed-MII#513 · Filter by author tang-t21 opened on Jul 26, 2024
  1