ZeRO: Memory Optimizations Toward Training Trillion Parameter Models
Milestone Papers
ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Paper
(2019-10) ZeRO by Microsoft

Relevant Sites

GPT-4 Technical Report

(2023-03) GPT 4 by OpenAI

Qwen2.5 Technical Report

(2024-12) Qwen2.5 by Alibaba

Language Is Not All You Need: Aligning Perception with Language Models

(2023-02) Kosmos-1 by Microsoft

Finetuned Language Models are Zero-Shot Learners

(2021-09) FLAN by Google

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

(2023-05) DPO by Stanford

Language models are few-shot learners

(2020-05) GPT 3.0 by OpenAI

Leave a Reply

Your email address will not be published. Required fields are marked *