Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism
Milestone Papers
Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Paper
(2019-09) Megatron-LM by NVIDIA

Relevant Sites

GPT-4 Technical Report

(2023-03) GPT 4 by OpenAI

Tree of Thoughts: Deliberate Problem Solving with Large Language Models

(2023-05) ToT by Google&Princeton

Improving Language Understanding by Generative Pre-Training

(2018-06) GPT 1.0 by OpenAI

PaLM 2 Technical Report

(2023-05) PaLM 2 by Google

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

(2021-12) GLaM by Google

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Leave a Reply

Your email address will not be published. Required fields are marked *