ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
Milestone Papers
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Paper
(2019-10) ZeRO by Microsoft

Relevant Sites

Visual Instruction Tuning

(2023-04) LLaVA by UW–Madison&Microsoft

Improving alignment of dialogue agents via targeted human judgements

(2022-09) Sparrow by DeepMind

Finetuned Language Models are Zero-Shot Learners

(2021-09) FLAN by Google

Language Models are Unsupervised Multitask Learners

(2019-02) GPT 2.0 by OpenAI

DeepSeek-R1 86,319

(2025-1) DeepSeek-R1 by DeepSeek

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

(2022-11) BLOOM by BigScience

Leave a Reply

Your email address will not be published. Required fields are marked *