Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models
Milestone Papers
Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

(2022-06) BIG-bench by Google

Paper
(2022-06) BIG-bench by Google

Relevant Sites

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

(2021-12) Gopher by DeepMind

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Multitask Prompted Training Enables Zero-Shot Task Generalization

(2021-10) T0 by HuggingFace et al.

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

(2024-05) Mamba2 by CMU&Princeton

Language models are few-shot learners

(2020-05) GPT 3.0 by OpenAI

OPT: Open Pre-trained Transformer Language Models

(2022-05) OPT by Meta

Leave a Reply

Your email address will not be published. Required fields are marked *