Phi-3 (mini 3.8B; small 7B and medium 14B to follow)
Phi-3-mini is a 3.8B model trained on 3.3T tokens that scores 69% on MMLU, fits on a phone and has a 128K-context variant.
Brought GPT-3.5-class results at 3.8B by training on heavily filtered web and synthetic data; first model in its class with 128K context. Phi-3-small (7B, 75% MMLU) and medium (14B, 78%) followed. Weak on factual recall like TriviaQA.
- Date
- Tuesday, 23 April 2024
- Lab
- Microsoft
- Kind
- open-weights
- Access
- open weights
Figures
| Measure | Value | Measured by |
|---|---|---|
| MMLU / MT-bench (phi-3-mini) | 69% / 8.38 3.3T training tokens; medium 78% / 8.9 | company |
Technical report posted 2024-04-22; blog and weights 2024-04-23.
Sources
- azure.microsoft.com/en-us/blog/introducing-phi-3-redefining-whats-possible-with-slms/
- arxiv.org/abs/2404.14219
This record was checked against its sources on 6 October 2026. How we check