Scaling Laws for Neural Language Models
Milestone Papers
Scaling Laws for Neural Language Models

(2020-01) Scaling Law by OpenAI

Paper
(2020-01) Scaling Law by OpenAI

Relevant Sites

BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding

(2018-10) BERT by Google

Llama 2: Open Foundation and Fine-Tuned Chat Models

(2023-07) LLaMA2 by Meta

Language models are few-shot learners

(2020-05) GPT 3.0 by OpenAI

Mistral 7B

(2023-10) Mistral 7B by Mistral

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

(2023-01) Flan 2022 Collection by Google

Switch Transformers: Scaling to Trillion Parameter Models with Simple and Efficient Sparsity

(2021-01) Switch Transformers by Google

Leave a Reply

Your email address will not be published. Required fields are marked *