GLaM: Efficient Scaling of Language Models with Mixture-of-Experts
Milestone Papers
GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

(2021-12) GLaM by Google

Paper
(2021-12) GLaM by Google

Relevant Sites

Language models are few-shot learners

(2020-05) GPT 3.0 by OpenAI

Using Deep and Megatron to Train Megatron-Turing NLG 530B, A Large-Scale Generative Language Model

(2022-01) Megatron-Turing NLG by Microsoft&NVIDIA

RWKV: Reinventing RNNs for the Transformer Era

(2023-05) RWKV by Bo Peng

On the Opportunities and Risks of Foundation Models

(2021-08) Foundation Models by Stanford

WebGPT: Browser-assisted question-answering with human feedback

(2021-12) WebGPT by OpenAI

Evaluating Large Language Models Trained on Code

(2021-08) Codex by OpenAI

Leave a Reply

Your email address will not be published. Required fields are marked *