Issues

• [RFC] Define accelerator-agnostic backend boundaries for core MoE operations

  databricks/megablocks#188 · Filter by author licy666 opened on Sep 7, 2026
  2

• Energy Efficiency: 10 Mathematical Techniques for 60-70% AI Energy Reduction (Phi6Simple, FFT-Mix, Phi MoE)

  databricks/megablocks#184 · Filter by author dancinlife opened on Mar 27, 2026

• Compatibility with CUDA 13.0

  databricks/megablocks#180 · Filter by author kiddyboots216 opened on Oct 23, 2025
  1

• difference between cuda kernel code and using torch.ops

  databricks/megablocks#179 · Filter by author abliao opened on Sep 22, 2025

• Issue with Sparse MLP

  databricks/megablocks#178 · Filter by author Venkatesh3132003 opened on Aug 14, 2025
  1

• MOE uses more memory than dense model and is slower

  databricks/megablocks#166 · Filter by author samuelwheeler opened on Mar 3, 2025
  1

• bump version for PyTorch 2.6

  databricks/megablocks#165 · Filter by author ad8e opened on Feb 26, 2025

• [rank5]: TypeError: SortOp.forward() takes from 2 to 3 positional arguments but 5 were given When running moe script

  databricks/megablocks#164 · Filter by author rtmadduri opened on Feb 8, 2025

• Grouped GEMM execution not possible with HW

  databricks/megablocks#163 · Filter by author cassanof opened on Jan 3, 2025
  2

• CUDA Error When Running Single GPU Experiment

  databricks/megablocks#161 · Filter by author kevin3567 opened on Nov 14, 2024

• Is current megablocks compatible with distributed optimizer in Megatron-LM?

  databricks/megablocks#160 · Filter by author Spico197 opened on Nov 11, 2024
  1

• amp_C undefined symbol after installing Megablocks

  databricks/megablocks#157 · Filter by author RachitBansal opened on Oct 11, 2024
  4

• Do you support the bias of mlp？

  databricks/megablocks#156 · Filter by author maobenz opened on Oct 9, 2024
  1

• what devices are supported?

  databricks/megablocks#155 · Filter by author Guodanding opened on Oct 8, 2024
  15

• Does it work with torch.compile?

  databricks/megablocks#153 · Filter by author Muennighoff opened on Sep 30, 2024
  2

• [Do not Merge] Add CI/CD Milestones

  databricks/megablocks#132 · Filter by author eitanturok opened on Aug 1, 2024

• Routing

  databricks/megablocks#118 · Filter by author alexliap opened on Jun 27, 2024
  1

• Illegal memory access on non-0 cuda devices from histogram

  databricks/megablocks#117 · Filter by author phillip-kravtsov opened on Jun 25, 2024

• Can we change self.blocking in dmoe.py from 128 to 64?

  databricks/megablocks#114 · Filter by author seanM29 opened on May 24, 2024
  2

• _LOAD_BALANCING_LOSS returns empty list sometimes

  databricks/megablocks#113 · Filter by author exnx opened on May 22, 2024
  1

• Bad throughput with GLU

  databricks/megablocks#110 · Filter by author Muennighoff opened on May 17, 2024
  1

• 1-expert worse than dense model

  databricks/megablocks#107 · Filter by author Muennighoff opened on May 8, 2024
  1

• OSError: Stale file handle with dMoE

  databricks/megablocks#106 · Filter by author Muennighoff opened on Apr 24, 2024
  3

• Add a fine-tune script for JetMoE

  databricks/megablocks#105 · Filter by author shamanez opened on Apr 17, 2024
  2

• ScatterMoE feature

  databricks/megablocks#104 · Filter by author ehartford opened on Apr 5, 2024
  5