ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
Milestone Papers
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Paper
(2019-10) ZeRO by Microsoft

Relevant Sites

Improving alignment of dialogue agents via targeted human judgements

(2022-09) Sparrow by DeepMind

GPT-4 Technical Report

(2023-03) GPT 4 by OpenAI

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

(2024-05) Mamba2 by CMU&Princeton

WebGPT: Browser-assisted question-answering with human feedback

(2021-12) WebGPT by OpenAI

DeepSeek-R1 91,457

(2025-1) DeepSeek-R1 by DeepSeek

Galactica: A Large Language Model for Science

(2022-11) Galactica by Meta

Leave a Reply

Your email address will not be published. Required fields are marked *