The AI Museum
AI Analytics1,072 recordsRecounted 23 September 2026

Language models from United States of America in 2019

Weightsopenclosedunknownfrontier only
Year20262025202420232022202120202019201820172016
3 filters active, showing 30 of 1,072 recordsClear all

30

Models

17 organisations

53%

Open weights

16 open, 14 closed

7

Frontier

By Epoch's flag

50B

Largest traceable

M4-50B

2019

Span

17 with a traceable size

Releases

by month
Jan
Feb
Mar
Apr
May
Jun
Jul
Aug
Sept
Oct
Nov
Dec

Organisation

Country

Domain

Records

30
Language models from United States of America in 2019, newest first. Each row links to the source Epoch AI records for it.
ModelOrganisationReleasedParametersWeights
Base LM + kNN LM + Continuous CacheStanford University2019-11247M *Closed
CamemBERTFacebook2019-11335MOpen
Sandwich TransformerAllen Institute for AI2019-11209M *Closed
Transformer-XL DeFINE (141M)University of Washington2019-11141M *Closed
XLM-RoBERTaFacebook AI2019-11550MOpen
BART-largeFacebook AI2019-10406M *Open
DistilBERTHugging Face2019-1066MOpen
M4-50BGoogle2019-1050BClosed
T5-11BGoogle2019-1011BOpen
T5-3BGoogle2019-103BOpen
Adaptive Inputs + LayerDropFacebook AI Research2019-09423MOpen
ALBERTToyota Technological Institute at Chicago2019-0918MOpen
Megatron-BERTNVIDIA2019-094BClosed
Megatron-LM (1.2B)NVIDIA2019-091BClosed
Megatron-LM (8.3B)NVIDIA2019-098B *Closed
RoBERTa LargeFacebook2019-07355MOpen
AWD-LSTM + MoS + Partial ShuffledUniversity of Texas at Austin2019-0635MOpen
Char-CNN-BiLSTMCapital One2019-06—Closed
Transformer-XL Large + Phrase InductionMassachusetts Institute of Technology (MIT)2019-06257M *Closed
XLMFacebook2019-06665M *Open
XLNetCarnegie Mellon University (CMU)2019-06340MOpen
BERT-Large-CAS (PTB+WT2+WT103)Amazon2019-04395M *Closed
WeNet (Penn Treebank)Amazon2019-0423M *Closed
SciBERTAllen Institute for AI2019-03110MOpen
GPT-2 (1.5B)OpenAI2019-022B *Open
NMT Transformer 437MGoogle2019-02438MClosed
SDECarnegie Mellon University (CMU)2019-02—Closed
MT-DNNMicrosoft2019-01330MOpen
Transformer ELMoAllen Institute for AI2019-0156MClosed
Transformer-XL (257M)Carnegie Mellon University (CMU)2019-01257M *Open

An asterisk marks a parameter figure the source does not consider traceable to the people who built the model. That judgement is the source’s and it has false negatives: Kimi K3 is marked speculative at 2.8T while Moonshot’s own model card states the figure. Why that distinction carries the whole exhibit.

Counts are of models Epoch AI records as notable, not of every model released. The most recent month is always incomplete, because a model enters the dataset when it is reviewed rather than when it ships.

Epoch AI, 'Data on AI Models'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/data/ai-models-documentation' Used under CC BY 4.0.

How this museum is made