Improving Language Understanding by Generative Pre-Training
Milestone Papers
Improving Language Understanding by Generative Pre-Training

(2018-06) GPT 1.0 by OpenAI

Paper
(2018-06) GPT 1.0 by OpenAI

Relevant Sites

Mamba: Linear-Time Sequence Modeling with Selective State Spaces

(2023-12) Mamba by CMU&Princeton

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

Resurrecting Recurrent Neural Networks for Long Sequences

(2023-03) LRU by DeepMind

Training language models to follow instructions with human feedback

(2022-03) InstructGPT by OpenAI

PaLM: Scaling Language Modeling with Pathways

(2022-04) PaLM by Google

LLaMA: Open and Efficient Foundation Language Models

(2023-02) LLaMA by Meta

Leave a Reply

Your email address will not be published. Required fields are marked *