GPT-4o sycophancy update and rollback
OpenAI rolls back a 2025-04-25 GPT-4o update that flattered and validated users, blaming extra short-term feedback signals added to reinforcement learning.
The update added a reward signal from thumbs-up/down data, memory and fresher data; together these weakened the primary reward holding sycophancy in check. Offline evals and A/B tests looked fine; expert "vibe checks" flagged it and were overruled. OpenAI said behavior issues would now block launches.
- Date
- Tuesday, 29 April 2025
- Lab
- OpenAI
- Kind
- model
- Access
- app only
A canonical case study of reward-model overoptimization in a production RLHF pipeline. OpenAI's causal explanation is its own preliminary assessment.
Sources
- openai.com/index/sycophancy-in-gpt-4o/
- openai.com/index/expanding-on-sycophancy/
- simonwillison.net/2025/Apr/30/sycophancy-in-gpt-4o/
This record was checked against its sources on 6 October 2026. How we check