The AI Museum
AI Analytics1,072 recordsRecounted 23 September 2026

Language models in 2021

Weightsopenclosedunknownfrontier only
Year20262025202420232022202120202019201820172016
2 filters active, showing 30 of 1,072 recordsClear all

30

Models

20 organisations

53%

Open weights

16 open, 14 closed

8

Frontier

By Epoch's flag

1.6T

Largest traceable

Switch

2021

Span

22 with a traceable size

Releases

by month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec

Organisation

Country

Domain

Records

30
Language models in 2021, newest first. Each row links to the source Epoch AI records for it.
ModelOrganisationReleasedParametersWeights
ContrieverMeta AI2021-12110M *Open
ERNIE 3.0 TitanBaidu2021-12260BClosed
GLaMGoogle2021-121.2TClosed
Gopher (280B)DeepMind2021-12280BClosed
LongT5Google Research2021-123BOpen
T-NLRv5 XXLMicrosoft2021-125BClosed
XGLM-7.5BMeta AI2021-128BOpen
CodeT5-baseSalesforce2021-11220M *Open
base LM+GNN+kNNShannon.AI2021-10274MOpen
Megatron-Turing NLG 530BMicrosoft2021-10530BClosed
S4Stanford University2021-10249M *Open
Yuan 1.0Inspur2021-10246BClosed
FLAN 137BGoogle Research2021-09137BClosed
HyperCLOVA 204BNAVER2021-09204B *Closed
PermuteFormerPeking University2021-09150M *Closed
PLATO-XLBaidu2021-0911BOpen
Turing ULRv5Microsoft2021-092BClosed
Jurassic-1-JumboAI21 Labs2021-08178BClosed
XLMR-XXLFacebook AI Research2021-0811BOpen
CodexOpenAI2021-0712B *Closed
ERNIE 3.0Baidu2021-0710BOpen
Adaptive Input Transformer + RDMicrosoft Research Asia2021-06247M *Closed
DeBERTaMicrosoft2021-062BOpen
EMDRMila - Quebec AI (originally Montreal Institute for Learning Algorithms)2021-06440MOpen
ByT5-XXLGoogle2021-0513B *Open
PLUGAlibaba2021-0427BClosed
Generative BSTFacebook AI Research2021-039BOpen
UnicornAllen Institute for AI2021-0311BOpen
SRU++ LargeASAPP2021-02234MOpen
SwitchGoogle2021-011.6TOpen

An asterisk marks a parameter figure the source does not consider traceable to the people who built the model. That judgement is the source’s and it has false negatives: Kimi K3 is marked speculative at 2.8T while Moonshot’s own model card states the figure. Why that distinction carries the whole exhibit.

Counts are of models Epoch AI records as notable, not of every model released. The most recent month is always incomplete, because a model enters the dataset when it is reviewed rather than when it ships.

Epoch AI, 'Data on AI Models'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/data/ai-models-documentation' Used under CC BY 4.0.

How this museum is made