Megatron-DeepSpeed
Training Frameworks
Megatron-DeepSpeed

DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.

GitHub
DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.

Relevant Sites

Colossal-AI 40,962

Making large AI models cheaper, faster, and more accessible.

Mesh Tensorflow 1,608

Mesh TensorFlow: Model Parallelism Made Easier.

maxtext 1,763

A simple, performant and scalable Jax LLM!

torchtune 5,257

A Native-PyTorch Library for LLM Fine-tuning.

veRL 9,415

veRL is a flexible and efficient RL framework for LLMs.

Megatron-LM 12,569

Ongoing research training transformer models at scale.

Leave a Reply

Your email address will not be published. Required fields are marked *