Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Milestone Papers
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Paper
(2019-09) Megatron-LM by NVIDIA

Relevant Sites

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

(2018-10) BERT by Google

Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

(2023-05) Dromedary by CMU et al.

Visual Instruction Tuning

(2023-04) LLaVA by UW–Madison&Microsoft

OPT: Open Pre-trained Transformer Language Models

(2022-05) OPT by Meta

Scaling Language Models: Methods, Analysis & Insights from Training Gopher

(2021-12) Gopher by DeepMind

LaMDA: Language Models for Dialog Applications

(2022-01) LaMDA by Google

Leave a Reply

Your email address will not be published. Required fields are marked *