PaLM: Scaling Language Modeling with Pathways
Milestone Papers
PaLM: Scaling Language Modeling with Pathways

(2022-04) PaLM by Google

Paper
(2022-04) PaLM by Google

Relevant Sites

Pythia: A Suite for Analyzing Large Language Models Across Training and Scaling

(2023-04) Pythia by EleutherAI et al.

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Scaling Instruction-Finetuned Language Models

(2022-10) Flan-T5/PaLM by Google

OPT: Open Pre-trained Transformer Language Models

(2022-05) OPT by Meta

Visual Instruction Tuning

(2023-04) LLaVA by UW–Madison&Microsoft

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Leave a Reply

Your email address will not be published. Required fields are marked *