Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Milestone Papers
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

(2024-05) Mamba2 by CMU&Princeton

Paper

(2024-05) Mamba2

Relevant Sites

Holistic Evaluation of Language Models

(2022-11) HELM by Stanford

GPT-4 Technical Report

(2023-03) GPT 4 by OpenAI

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Mistral 7B

(2023-10) Mistral 7B by Mistral

Language Models are General-Purpose Interfaces

(2022-06) METALM by Microsoft

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

(2018-10) BERT by Google

Leave a Reply

Your email address will not be published. Required fields are marked *