RWKV: Reinventing RNNs for the Transformer Era
Milestone Papers
RWKV: Reinventing RNNs for the Transformer Era

(2023-05) RWKV by Bo Peng

Paper
(2023-05) RWKV by Bo Peng

Relevant Sites

Attention Is All You Need

(2017-06) Transformers by Google

DeepSeek-R1 90,681

(2025-1) DeepSeek-R1 by DeepSeek

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

(2024-05) Mamba2 by CMU&Princeton

Llama 2: Open Foundation and Fine-Tuned Chat Models

(2023-07) LLaMA2 by Meta

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

(2018-10) BERT by Google

Leave a Reply

Your email address will not be published. Required fields are marked *