Grok-1 open weights
xAI releases the raw Grok-1 base checkpoint, 314B-parameter mixture-of-experts, under Apache 2.0.
Weights and architecture of a 314B MoE (about 25% of weights active per token) released as the pretraining checkpoint from October 2023, not fine-tuned for dialogue. Built on a custom JAX and Rust stack. One of the largest MoE weight drops available at the time.
- Date
- Sunday, 17 March 2024
- Lab
- xAI
- Kind
- open-weights
- Access
- open weights
Figures
| Measure | Value | Measured by |
|---|---|---|
| Parameters | 314B total, ~25% active per token MoE, base model only | company |
Release is the base model only, with no post-training data, and xAI's post gives no training-data details.
Sources
This record was checked against its sources on 6 October 2026. How we check