DeepSpeed | LLMWay – The Way To LLM

Training Frameworks

DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

GitHub

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

torchtune 5,132

A Native-PyTorch Library for LLM Fine-tuning.

Colossal-AI 40,835

Making large AI models cheaper, faster, and more accessible.

Megatron-LM 12,211

Ongoing research training transformer models at scale.

Transformer Engine 2,384

A library for accelerating Transformer model training on NVIDIA GPUs.

Mesh Tensorflow 1,605

Mesh TensorFlow: Model Parallelism Made Easier.

BMTrain 584

Efficient Training for Big Models.