Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Milestone Papers
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Paper
(2019-09) Megatron-LM by NVIDIA

Relevant Sites

Improving language models by retrieving from trillions of tokens

(2021-12) Retro by DeepMind

Improving alignment of dialogue agents via targeted human judgements

(2022-09) Sparrow by DeepMind

Improving Language Understanding by Generative Pre-Training

(2018-06) GPT 1.0 by OpenAI

LLaMA: Open and Efficient Foundation Language Models

(2023-02) LLaMA by Meta

Language Models are General-Purpose Interfaces

(2022-06) METALM by Microsoft

Finetuned Language Models are Zero-Shot Learners

(2021-09) FLAN by Google

Leave a Reply

Your email address will not be published. Required fields are marked *