Byte Latent Transformer (BLT)
Tokenizer-free LLM that groups raw bytes into entropy-based patches; first FLOP-controlled byte-level scaling study to 8B params and 4T bytes.
Allocates more compute to hard-to-predict spans by making patch length depend on next-byte entropy. Matches token-based models on quality and improves inference efficiency and robustness, making a credible case for removing the tokenizer.
- Date
- Friday, 13 December 2024
- Lab
- Meta
- Kind
- paper
- Access
- research preview
Figures
| Measure | Value | Measured by |
|---|---|---|
| Scale | 8B params, 4T training bytes FLOP-controlled study | company |
arXiv v1 2024-12-13; Meta affiliation inferred from author names on the abstract page.
Sources
This record was checked against its sources on 6 October 2026. How we check