Scaling Instruction-Finetuned Language Models
Milestone Papers
Scaling Instruction-Finetuned Language Models

(2022-10) Flan-T5/PaLM by Google

Paper
(2022-10) Flan-T5/PaLM by Google

Relevant Sites

Language Models are Unsupervised Multitask Learners

(2019-02) GPT 2.0 by OpenAI

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

(2023-12) Mamba by CMU&Princeton

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Multitask Prompted Training Enables Zero-Shot Task Generalization

(2021-10) T0 by HuggingFace et al.

Scaling Laws for Neural Language Models

(2020-01) Scaling Law by OpenAI

DeepSeek-V2: A Strong, Economical, and Efficient Mixture-of-Experts Language Model

(2024-01) DeepSeek-v2 by DeepSeek

Leave a Reply

Your email address will not be published. Required fields are marked *