Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Milestone Papers
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Paper
(2019-09) Megatron-LM by NVIDIA

Relevant Sites

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

(2023-04) Pythia by EleutherAI et al.

Language Is Not All You Need: Aligning Perception with Language Models

(2023-02) Kosmos-1 by Microsoft

Improving language models by retrieving from trillions of tokens

(2021-12) Retro by DeepMind

Improving alignment of dialogue agents via targeted human judgements

(2022-09) Sparrow by DeepMind

PaLM: Scaling Language Modeling with Pathways

(2022-04) PaLM by Google

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

(2024-01) DeepSeek-v2 by DeepSeek

Leave a Reply

Your email address will not be published. Required fields are marked *