A native PyTorch Library for large model training.

Mesh TensorFlow: Model Parallelism Made Easier.

DeepSpeed is a deep learning optimization library that makes distributed training and inference easy, efficient, and effective.

veRL is a flexible and efficient RL framework for LLMs.

Generative AI framework built for researchers and PyTorch developers working on Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV) domains.

A simple, performant and scalable Jax LLM!