Orca (13B)
13B model trained on GPT-4's step-by-step explanation traces matches ChatGPT on Big-Bench Hard and beats Vicuna-13B by over 100%.
Showed imitation learning from rich explanation traces, not just answers, closes much of the gap to a larger teacher; an early template for reasoning distillation and synthetic-data post-training.
- Date
- Monday, 5 June 2023
- Lab
- Microsoft
- Kind
- paper
- Access
- research preview
Figures
| Measure | Value | Measured by |
|---|---|---|
| Big-Bench Hard (zero-shot) | parity with ChatGPT over 100% better than Vicuna-13B; still behind GPT-4 | company |
arXiv v1 2023-06-05.
Sources
This record was checked against its sources on 6 October 2026. How we check