Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity
Milestone Papers
Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Paper
(2021-01) Switch Transformers by Google

Relevant Sites

DeepSeek-R1 89,926

(2025-1) DeepSeek-R1 by DeepSeek

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

(2022-06) BIG-bench by Google

LaMDA: Language Models for Dialog Applications

(2022-01) LaMDA by Google

Improving alignment of dialogue agents via targeted human judgements

(2022-09) Sparrow by DeepMind

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

(2023-05) DPO by Stanford

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Leave a Reply

Your email address will not be published. Required fields are marked *