Language models in 2016
8
Models
8 organisations
17%
Open weights
1 open, 5 closed
1
Frontier
By Epoch's flag
54M
Largest traceable
NAS with base 8 and shared embeddings
2016
Span
4 with a traceable size
Releases
by monthOrganisation
Country
Records
8| Model | Organisation | Released | Parameters | Weights |
|---|---|---|---|---|
| BIDAF | University of Washington | 2016-11 | 3M | Open |
| NAS with base 8 and shared embeddings | Google Brain | 2016-11 | 54M | Closed |
| VD-LSTM+REAL Large | Salesforce Research | 2016-11 | 51M | Closed |
| GNMT | 2016-09 | 278M * | Closed | |
| Pointer Sentinel-LSTM (medium) | MetaMind Inc | 2016-09 | 21M | Closed |
| Character-enriched word2vec | Facebook AI Research | 2016-07 | — | Not recorded |
| Gated HORNN (3rd order) | York University | 2016-04 | 9M * | Closed |
| Named Entity Recognition model | Carnegie Mellon University (CMU) | 2016-03 | — | Not recorded |
An asterisk marks a parameter figure the source does not consider traceable to the people who built the model. That judgement is the source’s and it has false negatives: Kimi K3 is marked speculative at 2.8T while Moonshot’s own model card states the figure. Why that distinction carries the whole exhibit.
Counts are of models Epoch AI records as notable, not of every model released. The most recent month is always incomplete, because a model enters the dataset when it is reviewed rather than when it ships.
Epoch AI, 'Data on AI Models'. Published online at epoch.ai. Retrieved from 'https://epoch.ai/data/ai-models-documentation' Used under CC BY 4.0.