InternLM2 (7B, 20B)
Second-generation open InternLM at 7B and 20B with 200K context via 32K training and a new conditional online RLHF method (COOL RLHF).
Tech report (arXiv 2403.17297, 2024-03-26) details data preparation, 4K to 32K context training, strong Needle-in-a-Haystack at 200K, and COOL RLHF to handle conflicting preferences and reward hacking.
- Date
- Wednesday, 17 January 2024
- Lab
- Shanghai AI Laboratory (InternLM)
- Kind
- open-weights
- Access
- open weights (restricted license)
Figures
| Measure | Value | Measured by |
|---|---|---|
| Needle-in-a-Haystack context | 200K tokens | company |
Release 2024-01-17 per the InternLM README news; InternLM-20B came 2023-09-20; InternLM2.5 on 2024-07-03.
Sources
This record was checked against its sources on 6 October 2026. How we check