RWKV: Reinventing RNNs for the Transformer Era
Milestone Papers
RWKV: Reinventing RNNs for the Transformer Era

(2023-05) RWKV by Bo Peng

Paper
(2023-05) RWKV by Bo Peng

Relevant Sites

Improving Language Understanding by Generative Pre-Training

(2018-06) GPT 1.0 by OpenAI

Finetuned Language Models are Zero-Shot Learners

(2021-09) FLAN by Google

Mistral 7B

(2023-10) Mistral 7B by Mistral

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

(2023-12) Mamba by CMU&Princeton

Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism

(2019-09) Megatron-LM by NVIDIA

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

(2022-12) OPT-IML by Meta

Leave a Reply

Your email address will not be published. Required fields are marked *