Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Milestone Papers
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Paper
(2019-09) Megatron-LM by NVIDIA

Relevant Sites

Emergent Abilities of Large Language Models

(2022-06) Emergent Abilities by Google

The Llama 3 Herd of Models

(2024-05) Llama3 by Meta

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Improving Language Understanding by Generative Pre-Training

(2018-06) GPT 1.0 by OpenAI

GLM-130B: An Open Bilingual Pre-trained Model

(2022-10) GLM-130B by Tsinghua

Language models are few-shot learners

(2020-05) GPT 3.0 by OpenAI

Leave a Reply

Your email address will not be published. Required fields are marked *