开源框架整理

训练

• Megatron
• DeepSpeed

SFT

• hiyouga/LLaMA-Factory

RL

• volcengine/verl
• OpenRL-Lab/openrl
• alibaba/ROLL
• allenai/open-instruct
• NovaSky-AI/SkyRL
• PrimeIntellect-ai/prime-rl
• unslothai/unsloth
• OpenRLHF/OpenRLHF
• agentica-project/verl-pipeline
• RLinf/RLinf
• NVIDIA-NeMo/RL

调度

• LLM-d
• aibrix
• NVIDIA-dynamo
• Ray
• sgl-project/rbg

Ckpt

• Mooncake
• NIXL

KV 管理

• Mooncake
• LM-Cache

推理

• vLLM

• SGlang

• TensorRT-LLM

• LMdeploy

• ktransformers

• ggml-org/llama.cpp

• thu-pacman/chitu

• alibaba/rtp-llm

• jd-opensource/xllm

• Nano vLLM

• mini-sglang

accelerate

• flashinfer-ai/flashinfer
• Dao-AILab/flash-attention
• SageAttention

PD-disaggregation

• LLMServe/DistServe

Speculative-Decoding

• SafeAILab/EAGLE
• smart-lty/nano-PEARL

算子与通信

• deepseek-ai/DeepEP

• Triton

• PyTorch Distributed

• NIXL

• RDMA

• NVLimk

• PCIE

• Ascend HIXL

可观测

• Nsight System
• Nsight Compute
• Prometheus
• AIConfigurator
• AIprofiling

云原生

• Kubernetes
• slurm
• Docker
• Hami
• fluid-cloudnative/fluid

基础设施

• Hardware

  • CPU
  • GPU NVIDIA
  • NPU
  • Ascend

• Cloud

  • Aliyun
  • huoshan
  • tencent
  • AWS