Improving language models by retrieving from trillions of tokens
Milestone Papers
Improving language models by retrieving from trillions of tokens

(2021-12) Retro by DeepMind

Paper
(2021-12) Retro by DeepMind

Relevant Sites

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

(2022-12) OPT-IML by Meta

Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

(2023-05) Dromedary by CMU et al.

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

(2021-12) GLaM by Google

Visual Instruction Tuning

(2023-04) LLaVA by UW–Madison&Microsoft

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Attention Is All You Need

(2017-06) Transformers by Google

Leave a Reply

Your email address will not be published. Required fields are marked *