Transformer Engine | LLMWay

Training Frameworks

Transformer Engine

A library for accelerating Transformer model training on NVIDIA GPUs.

GitHub

A library for accelerating Transformer model training on NVIDIA GPUs.

Mesh Tensorflow 1,613

Mesh TensorFlow: Model Parallelism Made Easier.

torchtitan 4,158

A native PyTorch Library for large model training.

veRL 11,727

veRL is a flexible and efficient RL framework for LLMs.

torchtune 5,369

A Native-PyTorch Library for LLM Fine-tuning.

Colossal-AI 41,061

Making large AI models cheaper, faster, and more accessible.

Megatron-LM 13,042

Ongoing research training transformer models at scale.