DeepSpeed
Training Frameworks
DeepSpeed

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

GitHub
DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Relevant Sites

BMTrain 613

Efficient Training for Big Models.

Mesh Tensorflow 1,620

Mesh TensorFlow: Model Parallelism Made Easier.

Megatron-DeepSpeed 2,188

DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.

maxtext 1,971

A simple, performant and scalable Jax LLM!

torchtitan 4,662

A native PyTorch Library for large model training.

Transformer Engine 2,892

A library for accelerating Transformer model training on NVIDIA GPUs.

Leave a Reply

Your email address will not be published. Required fields are marked *