Language Models are Unsupervised Multitask Learners
Milestone Papers
Language Models are Unsupervised Multitask Learners

(2019-02) GPT 2.0 by OpenAI

Paper
(2019-02) GPT 2.0 by OpenAI

Relevant Sites

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Scaling Instruction-Finetuned Language Models

(2022-10) Flan-T5/PaLM by Google

PaLM: Scaling Language Modeling with Pathways

(2022-04) PaLM by Google

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

(2022-06) BIG-bench by Google

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

(2023-05) DPO by Stanford

Improving language models by retrieving from trillions of tokens

(2021-12) Retro by DeepMind

Leave a Reply

Your email address will not be published. Required fields are marked *