Ongoing research training transformer models at scale.

Generative AI framework built for researchers and PyTorch developers working on Large Language Models (LLMs), Multimodal Models (MMs), Automatic Speech Recognition (ASR), Text to Speech (TTS), and Computer Vision (CV) domains.

A native PyTorch Library for large model training.

A simple, performant and scalable Jax LLM!

Mesh TensorFlow: Model Parallelism Made Easier.

A library for accelerating Transformer model training on NVIDIA GPUs.