Scaling Language Models: Methods, Analysis & Insights from Training Gopher
Milestone Papers
Scaling Language Models: Methods, Analysis & Insights from Training Gopher

(2021-12) Gopher by DeepMind

Paper
(2021-12) Gopher by DeepMind

Relevant Sites

Galactica: A Large Language Model for Science

(2022-11) Galactica by Meta

Evaluating Large Language Models Trained on Code

(2021-08) Codex by OpenAI

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

(2023-05) DPO by Stanford

Attention Is All You Need

(2017-06) Transformers by Google

PaLM: Scaling Language Modeling with Pathways

(2022-04) PaLM by Google

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Leave a Reply

Your email address will not be published. Required fields are marked *