🌸 Run LLMs at home, BitTorrent-style. Fine-tuning and inference up to 10x faster than offloading

InternEvo is an open-sourced lightweight training framework aims to support model pre-training without the need for extensive dependencies.

Slicing a PyTorch Tensor Into Parallel Shards

Decentralized LLMs fine-tuning and inference with offloading

Large scale 4D parallelism pre-training for 🤗 transformers in Mixture of Experts *(still work in progress)*

gLLM: Global Balanced Pipeline Parallelism System for Distributed LLM Serving with Token Throttling

JORA: JAX Tensor-Parallel LoRA Library (ACL 2024)

A distributed training framework for large language models powered by Lightning.

Fast and easy distributed model training examples.

Tensor Parallelism with JAX + Shard Map

This repository outlines a comprehensive guide for training a distributed deep learning model.

Communication-efficient Tensor Parallelism for GPT-2

vLLM - High-throughput, memory-efficient LLM inference engine with PagedAttention, continuous batching, CUDA/HIP optimization, quantization (GPTQ/AWQ/INT4/INT8/FP8), tensor/pipeline parallelism, OpenAI-compatible API, multi-GPU/TPU/Neuron support, prefix caching, and multi-LoRA capabilities

Repo for AMX + FAST

Training Qwen3 to solve Wordle using SFT and GRPO

GPU Memory Calculator for LLM Training - Calculate GPU memory requirements for training Large Language Models with support for multiple training engines including PyTorch DDP, DeepSpeed ZeRO, Megatron-LM, and FSDP.