mixture-of-experts

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

A 2.78-trillion-parameter Kimi K3 running inference on a single CPU in 8.24 GB of RAM. Portable C99: no BLAS, no framework, no GPU.

Optimizing inference proxy for LLMs

Decentralized deep learning in PyTorch. Built to train models on thousands of volunteers across the world.

Run Mixtral-8x7B models in Colab or consumer desktops

【TMM 2025🔥】 Mixture-of-Experts for Large Vision-Language Models

PyTorch Re-Implementation of "The Sparsely-Gated Mixture-of-Experts Layer" by Noam Shazeer et al. https://arxiv.org/abs/1701.06538

Codebase for Aria - an Open Multimodal Native MoE

Tutel MoE: Optimized Mixture-of-Experts Library, Support GptOss/DeepSeek/Kimi-K2/Qwen3 using FP8/NVFP4/MXFP4

⛷️ LLaMA-MoE: Building Mixture-of-Experts from LLaMA with Continual Pre-training (EMNLP 2024)

cuDNN Frontend is NVIDIA's modern, open-source entry point to the cuDNN library and a growing collection of high-performance open-source kernels.

SMT: The Surrogate Modeling Toolbox

A Pytorch implementation of Sparsely-Gated Mixture of Experts, for massively increasing the parameter count of language models

From scratch implementation of a sparse mixture of experts language model inspired by Andrej Karpathy's makemore :)

A TensorFlow Keras implementation of "Modeling Task Relationships in Multi-task Learning with Multi-gate Mixture-of-Experts" (KDD 2018)

Swiftlet is a Swift and Metal runtime that runs large Qwen Mixture-of-Experts models locally on Apple devices by streaming expert weights from storage, enabling 35B and 80B models to run with low RAM, including on iPhone.

中文Mixtral混合专家大模型（Chinese Mixtral MoE LLMs）

Run MoE models bigger than your RAM. Frontier-size MoE on a 12 GB phone, CPU only, lossless, on stock llama.cpp

Krasis is a Hybrid LLM runtime which focuses on efficient running of larger models on consumer grade VRAM limited hardware

A library for easily merging multiple LLM experts, and efficiently train the merged LLM.