Llama 3.1 (8B, 70B, 405B)
Llama 3.1 405B: first open-weights model Meta says is competitive with GPT-4-class closed models, 128K context.
Scaled to 405B dense (16K H100s, 15T+ tokens), 128K context, 8 languages, and changed the license to explicitly allow using outputs to train other models, legitimising distillation and synthetic data from an open model.
- Date
- Tuesday, 23 July 2024
- Lab
- Meta
- Kind
- open-weights
- Access
- open weights (restricted license)
Figures
| Measure | Value | Measured by |
|---|---|---|
| Flagship size / context | 405B dense, 128K tokens trained on 16,000 H100 GPUs | company |
Paper (558+ authors) posted 2024-07-31.
Sources
This record was checked against its sources on 6 October 2026. How we check