An empirical analysis of compute-optimal large language model training
Milestone Papers
An empirical analysis of compute-optimal large language model training

(2022-04) Chinchilla by DeepMind

Paper
(2022-04) Chinchilla by DeepMind

Relevant Sites

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

(2022-06) BIG-bench by Google

Improving language models by retrieving from trillions of tokens

(2021-12) Retro by DeepMind

Language models are few-shot learners

(2020-05) GPT 3.0 by OpenAI

Holistic Evaluation of Language Models

(2022-11) HELM by Stanford

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

(2022-12) OPT-IML by Meta

Leave a Reply

Your email address will not be published. Required fields are marked *