Open-weight language models in 2019
18
Models
13 organisations
100%
Open weights
18 open, 0 closed
5
Frontier
By Epoch's flag
11B
Largest traceable
T5-11B
2019
Span
12 with a traceable size
Releases
by monthOrganisation
Records
18| Model | Organisation | Released | Parameters | Weights |
|---|---|---|---|---|
| CamemBERT | 2019-11 | 335M | Open | |
| XLM-RoBERTa | Facebook AI | 2019-11 | 550M | Open |
| BART-large | Facebook AI | 2019-10 | 406M * | Open |
| DistilBERT | Hugging Face | 2019-10 | 66M | Open |
| T5-11B | 2019-10 | 11B | Open | |
| T5-3B | 2019-10 | 3B | Open | |
| Adaptive Inputs + LayerDrop | Facebook AI Research | 2019-09 | 423M | Open |
| ALBERT | Toyota Technological Institute at Chicago | 2019-09 | 18M | Open |
| RoBERTa Large | 2019-07 | 355M | Open | |
| AWD-LSTM + MoS + Partial Shuffled | University of Texas at Austin | 2019-06 | 35M | Open |
| XLM | 2019-06 | 665M * | Open | |
| XLNet | Carnegie Mellon University (CMU) | 2019-06 | 340M | Open |
| AWD-LSTM-DRILL + dynamic evaluation† (WT2) | IDIAP | 2019-05 | 34M * | Open |
| Cross-lingual alignment | Tel Aviv University | 2019-04 | — | Open |
| SciBERT | Allen Institute for AI | 2019-03 | 110M | Open |
| GPT-2 (1.5B) | OpenAI | 2019-02 | 2B * | Open |
| MT-DNN | Microsoft | 2019-01 | 330M | Open |
| Transformer-XL (257M) | Carnegie Mellon University (CMU) | 2019-01 | 257M * | Open |
An asterisk marks a parameter figure the source does not consider traceable to the people who built the model. That judgement is the source’s and it has false negatives: Kimi K3 is marked speculative at 2.8T while Moonshot’s own model card states the figure. Why that distinction carries the whole exhibit.
Counts are of models Epoch AI records as notable, not of every model released. The most recent month is always incomplete, because a model enters the dataset when it is reviewed rather than when it ships.
Epoch AI, 'Data on AI Models'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/data/ai-models-documentation' Used under CC BY 4.0.