A library for accelerating Transformer model training on NVIDIA GPUs.

Making large AI models cheaper, faster, and more accessible.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

Ongoing research training transformer models at scale.

Generative AI framework built for researchers and PyTorch developers working on Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV) domains.

DeepSpeed version of NVIDIA's Megatron-LM that adds additional support for several features such as MoE model training, Curriculum Learning, 3D Parallelism, and others.