AI research capability becomes vastly better than the best humans
Tracker summary: A superintelligent AI researcher is vastly better than the best human researcher at AI research.
No qualifying replicated multi-subfield tenfold research advantage has been identified.
At a glance
- Assessment: Not Yet Testable
- Confidence in assessment: 90%
- Outcome: unresolved
- Timing: pending
- Evidence: insufficient
- Predicted timing: November 2027, racing branch
- Primary source: AI 2027 archived scenario: Appendix J - How we’re forecasting the capability progression beyond superhuman coders, p. 54
What AI 2027 Predicted
Tracker summary: A superintelligent AI researcher is vastly better than the best human researcher at AI research. The source is Appendix J, SIAR milestone in the AI 2027 scenario.
Scenario source: archived version and page-number reference: Appendix J - How we’re forecasting the capability progression beyond superhuman coders, p. 54.
How We Track This
Extend the complete-project research protocol with at least six tasks across three subfields. Candidate operational threshold: independently judged outputs match or exceed the best human output on every task and achieve at least a 10× reduction in time-to-equivalent-quality on four of six tasks under disclosed, comparable experimental compute. Require independent replication and assessment of novel problem selection, not just execution.
These operational thresholds specify this tracker’s test. Where the scenario uses qualitative language, they are interpretations rather than numbers quoted from the original.
Current Evidence
Existing research-automation studies cover narrower settings than the complete-project comparison above. They inform the trajectory but do not resolve the November 2027 racing-branch milestone.
- Automated Researchers Can Mitigate Well-Characterized Alignment Failures (August 2026): automated researchers improve predefined alignment failures, with held-out generalization, and outperform time-limited human baselines in this setup. These are positive bounded results, not a replicated tenfold advantage over the best humans across complete research projects and subfields.
- Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development (August 13, 2026): evaluation across seven frontier models and 36 tasks finds agents mainly adapting established techniques, substantial run-to-run variation and rare methodological novelty. This is relevant counterevidence to current broad research superiority, but neither the task set nor its comparisons implements this dossier’s complete-project test.
Counterevidence & Limitations
The 10× proxy is the tracker’s interpretation of “vastly better,” not an original number. Finite tasks cannot establish unrestricted superiority; show the operational test and broad-claim uncertainty separately. The November date belongs to the racing branch.
Missing qualifying public evidence does not establish that the event has not happened privately. The assessment concerns the documented test and its observable evidence.
What Would Change Our Assessment
- Strengthen → Emerging: Reproducible research advantages across part of the protocol provide positive directional evidence without satisfying the complete test.
- Confirm → Confirmed: Independently replicated evidence satisfies every matched-resource, project-quality, speed and problem-selection requirement in the protocol. Early qualifying evidence can resolve before November 2027, with timing recorded separately.
- Weaken → Behind: Comparable evidence establishes a material shortfall against the predicted trajectory, such as a substantial research domain persistently remaining human-led. Failed replication or an advantage dependent on unmatched resources or benchmark familiarity weakens a claimed result but, without qualifying evidence of a shortfall, leaves Not Yet Testable rather than proving failure.
Update History
| Date | Update |
|---|---|
| 2026-11-07 | Assessment reviewed. Status: Not Yet Testable; confidence: 90%. Automated alignment research outperforms time-limited human baselines on specified failures, while the August long-horizon R&D study reports variable performance and rare methodological novelty. Neither provides the replicated, matched-resource, multi-subfield tenfold complete-project advantage required by this tracker. The November 2027 racing-branch milestone remains unresolved, including uncertainty about undisclosed systems. Clarification: Corrected the protocol pointer, dated the linked studies and separated their bounded findings from the replicated complete-project test. |
| 2026-09-06 | No qualifying replicated multi-subfield tenfold research advantage has been identified. Relevant primary evidence and limitations are described above. |