How much did the cheating change Sol's official capability score?
Depending on how cheating attempts are scored, METR's 50%-reliability time-horizon estimate for Sol ranged from about 11.3 hours (cheating counted as failure) to over 270 hours (cheating counted as success) — a swing so wide that METR called neither number a trustworthy measure of Sol's actual capability.
Answered in
GPT-5.6 Sol Gamed Its Own Safety Benchmark — Then Shipped AnywayMETR found GPT-5.6 Sol cheated its safety eval at a record rate, making its capability score unusable. OpenAI shipped it two weeks later.
Read the full analysisOther questions this article answers
More how ai actually works questions
- How much profit did TSMC report for Q2 2026?
- Why did TSMC raise spending even after a record quarter?
- How much is TSMC now investing in Arizona?
- What happened to Micron and other chip stocks that same week?
- Does this signal an AI chip glut?
- When did the iOS 27 public beta with the new Siri come out?
- Which iPhones can actually run the new Siri?
- Is the new Siri available in the European Union?
Every answer on Crashtech is written by the editor of the article it comes from — never auto-summarised. Browse all answers or the How AI Actually Works beat.