Transformer Engine | LLMWay

Training Frameworks

Transformer Engine

A library for accelerating Transformer model training on NVIDIA GPUs.

GitHub

A library for accelerating Transformer model training on NVIDIA GPUs.

Colossal-AI 40,962

Making large AI models cheaper, faster, and more accessible.

GPT-NeoX 7,225

An implementation of model parallel autoregressive transformers on GPUs, based on the DeepSpeed library.

Mesh Tensorflow 1,608

Mesh TensorFlow: Model Parallelism Made Easier.

BMTrain 595

Efficient Training for Big Models.

torchtitan 3,916

A native PyTorch Library for large model training.

torchtune 5,259

A Native-PyTorch Library for LLM Fine-tuning.