BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding
Milestone Papers
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

(2018-10) BERT by Google

Paper
(2018-10) BERT by Google

Relevant Sites

OPT-IML: Scaling Language Model Instruction Meta Learning through the Lens of Generalization

(2022-12) OPT-IML by Meta

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Transformers are SSMs: Generalized Models and Efficient Algorithms Through Structured State Space Duality

(2024-05) Mamba2 by CMU&Princeton

Unifying Language Learning Paradigms

(2022-05) UL2 by Google

Multitask Prompted Training Enables Zero-Shot Task Generalization

(2021-10) T0 by HuggingFace et al.

PaLM-E: An Embodied Multimodal Language Model

(2023-03) PaLM-E by Google

Leave a Reply

Your email address will not be published. Required fields are marked *