Superhuman coder emerges
Tracker summary: A superhuman coder (SC): an AI system that can do any coding tasks that the best AGI company engineer does, while being much faster and cheaper.
At a glance
- Assessment: Not Yet Testable
- Confidence in assessment: 60%
- Predicted timing: March 2027
- Primary source: AI 2027 archived scenario: Appendix G - Why we forecast a superhuman coder in early 2027, p. 49; Appendix J - How we’re forecasting the capability progression beyond superhuman coders, p. 54
What AI 2027 Predicted
Tracker summary: The scenario predicts an internally developed system that can match the best AGI-company engineer on AI-research coding tasks while being much faster and cheaper. Appendix G (p. 49) gives the full definition: “an AI system that can do any coding tasks that the best AGI company engineer does, while being much faster and cheaper.” Appendix J (p. 54) specifies AI-research coding tasks and being cheap enough to run many copies, with March 2027 as the scenario milestone. Archived PDF, p. 49.
This milestone is internal: a public product release is not required, and limited visibility into internal systems constrains the test.
Scenario source: archived version and page-number reference: Appendix G - Why we forecast a superhuman coder in early 2027, p. 49; Appendix J - How we’re forecasting the capability progression beyond superhuman coders, p. 54.
How We Track This
We monitor:
- SWE-bench Verified and SWE-bench Pro scores
- Terminal-Bench results
- Real-world coding competitions (Codeforces, etc.), AI vs human rankings
- Enterprise reports on code quality from AI vs human developers
- Novel system-level projects completed entirely by AI
Shared full-coding-automation test: Require convergent evidence that a system can perform any AI-research coding task done by the best AGI-company engineer, while being much faster and cheaper. Record task coverage, human interventions, quality, elapsed time and cost against comparable human baselines. Narrow implementation results are precursor evidence; without the full comparison, the coding component remains unestablished on both pages. Research taste is a separate component, and confirmation of the conjunction requires both components.
Apply this same test in Coding fully automated but research taste still requires humans. Its Emerging assessment concerns precursor evidence for a developing implementation/judgment split, not attainment of this shared coding milestone. The displayed assessments address different evidence scopes; neither establishes full coding automation.
Current Evidence
Coding agents have improved substantially, but results depend on benchmark version, scaffold, task distribution and human assistance. Historical SWE-bench and Terminal-Bench snapshots are not a current frontier ranking. No reviewed result establishes superiority to the best programmers across essentially any coding task.
Anthropic’s June 2026 Fable 5 launch is a stronger signal for long-horizon autonomous coding than prior public releases. Anthropic reports that, in early customer testing, Fable 5 completed a 50-million-line Ruby migration in one day that Stripe estimated would have taken a team over two months by hand. Anthropic also says Fable 5 led Cognition’s FrontierCode evaluation. This supports the direction of travel toward more autonomous engineering, but it remains vendor-reported and does not establish the superhuman-coder threshold across all coding tasks.
Sources:
- SWE-bench Verified Results: Vals.ai
- SWE-Bench Pro Leaderboard: Scale Labs
- Inside OpenAI’s Race to Catch Up to Claude Code: WIRED
- Grading AI 2027’s 2025 Predictions: AI Futures Project
- Anthropic: Introducing Claude Opus 4.7
- Anthropic: Claude Fable 5 and Claude Mythos 5
Counterevidence & Limitations
- Narrow benchmark success does not establish complete-project superiority; task validity and evaluation setup matter.
- “Superhuman” is a high bar: surpassing the best humans on any task is qualitatively different from being a useful assistant
- Human assistance should be recorded for each project rather than inferred from an autonomous label.
- This review does not support a precise six-to-eighteen-month delay estimate.
What Would Change Our Assessment
- Strengthen the assessment: Independent complete-project assessments compare quality and speed with strong humans, disclosing resources and interventions.
- Confirm: Convergent evidence establishes the best-engineer AI-research coding comparison together with much greater speed and lower cost; isolated benchmark thresholds do not suffice.
- Remain unresolved: Evidence covers narrower tasks or lacks comparable human baselines. A qualifying result could be assessed before March 2027.
Update History
| Date | Update |
|---|---|
| 2026-11-07 | Assessment reviewed. Status: Not Yet Testable; confidence: 60%. Anthropic’s Fable/Mythos 5 account reports a Stripe migration completed in one day against an estimated months-long manual baseline and strong FrontierCode performance. That is meaningful coding evidence, without a matched best-AGI-engineer comparison across AI-research coding tasks at much greater speed and lower cost. Vendor selection, human involvement and incomplete task coverage limit the assessment; an undisclosed internal milestone remains possible, and no precise revised date follows from these results. |
| 2026-11-07 | Correction: Restored the defining best-engineer, much-faster-and-cheaper condition and corrected Appendix G to archived page 49, with AI-research scope on page 54. The milestone is internal; public visibility limits measurement rather than requiring public release. Removed the unsupported adjusted timing estimate while retaining March 2027. Synchronized the shared coding test with the research-taste dossier; narrow precursor evidence does not establish the full milestone. |
| 2026-09-06 | Current evidence and assessment explanation reviewed. Clarified measurement scope, source interpretation and limitations. |
| 2026-06-15 | Anthropic reported that Claude Fable 5 completed a large Ruby codebase migration in one day in early Stripe testing and led Cognition’s FrontierCode evaluation. This is stronger evidence for long-horizon coding progress, but it remains vendor-reported and does not establish superhuman coding across all tasks. Confidence adjusted 0.55 → 0.60. |
| 2026-04-27 | Anthropic released Claude Opus 4.7, highlighting better performance on difficult and long-running software engineering tasks, including customer reports of hours-long autonomy, self-verification, and stronger async coding workflows (Anthropic). This modestly strengthens the evidence that coding models are advancing toward more autonomous engineering work, but does not establish the cheap superhuman-coder threshold. |
| 2026-04-02 | AI Futures Project Q1 update: Kokotajlo shifted Automated Coder median from late 2029 to mid 2028 (1.5 years sooner). Revised AC time horizon requirement from 3 years to 1 year, citing Opus 4.6 impressiveness. Eli Lifland’s AC median also pulled forward 1.5 years. The scenario’s own authors are converging faster toward the original timeline. Source: LessWrong |
| 2026-03 | Prediction timeframe not yet reached (March 2027). AI coding capabilities advancing rapidly — SWE-bench scores improving, autonomous coding agents shipping — but superhuman performance across all coding tasks remains distant. |
| 2026-01 | Kokotajlo personal median for full coding automation: December 2030. |
| 2025-12 | AI Futures Project places median for “Superhuman Coder” at December 2031 — vs AI 2027 scenario’s January 2027. |
| 2025-11 | Claude Opus 4.5 reportedly outperforms every human candidate on Anthropic internal engineering assessments. Gemini 3 scores 37.4% on Humanity’s Last Exam (world record). Gap narrowing visibly but “superhuman coder” milestone remains contested. |
| 2025-09 | Gemini 2.5 Deep Think achieves gold-medal performance at 2025 ICPC World Finals (Sep 17), solving 10/12 problems including one no human team solved. Strongest “superhuman” coding signal to date, though competitive algorithmic programming differs from real-world software engineering. |