Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer
Milestone Papers
Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer

(2019-10) T5 by Google

Paper
(2019-10) T5 by Google

Relevant Sites

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

An empirical analysis of compute-optimal large language model training

(2022-04) Chinchilla by DeepMind

On the Opportunities and Risks of Foundation Models

(2021-08) Foundation Models by Stanford

Unifying Language Learning Paradigms

(2022-05) UL2 by Google

The Llama 3 Herd of Models

(2024-05) Llama3 by Meta

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Leave a Reply

Your email address will not be published. Required fields are marked *