Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality
Milestone Papers
Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

(2024-05) Mamba2 by CMU&Princeton

Paper

(2024-05) Mamba2

Relevant Sites

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Mistral 7B

(2023-10) Mistral 7B by Mistral

Evaluating Large Language Models Trained on Code

(2021-08) Codex by OpenAI

GLaM: Efficient Scaling of Language Models with Mixture-of-Experts

(2021-12) GLaM by Google

Direct Preference Optimization: Your Language Model is Secretly a Reward Model

(2023-05) DPO by Stanford

Principle-Driven Self-Alignment of Language Models from Scratch with Minimal Human Supervision

(2023-05) Dromedary by CMU et al.

Leave a Reply

Your email address will not be published. Required fields are marked *