OPT: Open Pre-trained Transformer Language Models
Milestone Papers
OPT: Open Pre-trained Transformer Language Models

(2022-05) OPT by Meta

Paper
(2022-05) OPT by Meta

Relevant Sites

ZeRO: Memory Optimizations Toward Training Trillion Parameter Models

(2019-10) ZeRO by Microsoft

BLOOM: A 176B-Parameter Open-Access Multilingual Language Model

(2022-11) BLOOM by BigScience

GLM-130B: An Open Bilingual Pre-trained Model

(2022-10) GLM-130B by Tsinghua

Beyond the Imitation Game: Quantifying and extrapolating the capabilities of language models

(2022-06) BIG-bench by Google

The Flan Collection: Designing Data and Methods for Effective Instruction Tuning

(2023-01) Flan 2022 Collection by Google

DeepSeek-R1 89,926

(2025-1) DeepSeek-R1 by DeepSeek

Leave a Reply

Your email address will not be published. Required fields are marked *