AI research capability becomes vastly better than the best humans

Last updated

Tracker summary: A superintelligent AI researcher is vastly better than the best human researcher at AI research.

No qualifying replicated multi-subfield tenfold research advantage has been identified.

At a glance

  • Assessment: Not Yet Testable
  • Confidence in assessment: 90%
  • Outcome: unresolved
  • Timing: pending
  • Evidence: insufficient
  • Predicted timing: November 2027, racing branch
  • Primary source: AI 2027 archived scenario: Appendix J - How we’re forecasting the capability progression beyond superhuman coders, p. 54

What AI 2027 Predicted

Tracker summary: A superintelligent AI researcher is vastly better than the best human researcher at AI research. The source is Appendix J, SIAR milestone in the AI 2027 scenario.

Scenario source: archived version and page-number reference: Appendix J - How we’re forecasting the capability progression beyond superhuman coders, p. 54.

How We Track This

Extend the complete-project research protocol with at least six tasks across three subfields. Candidate operational threshold: independently judged outputs match or exceed the best human output on every task and achieve at least a 10× reduction in time-to-equivalent-quality on four of six tasks under disclosed, comparable experimental compute. Require independent replication and assessment of novel problem selection, not just execution.

These operational thresholds specify this tracker’s test. Where the scenario uses qualitative language, they are interpretations rather than numbers quoted from the original.

Current Evidence

Existing research-automation studies cover narrower settings than the complete-project comparison above. They inform the trajectory but do not resolve the November 2027 racing-branch milestone.

Counterevidence & Limitations

The 10× proxy is the tracker’s interpretation of “vastly better,” not an original number. Finite tasks cannot establish unrestricted superiority; show the operational test and broad-claim uncertainty separately. The November date belongs to the racing branch.

Missing qualifying public evidence does not establish that the event has not happened privately. The assessment concerns the documented test and its observable evidence.

What Would Change Our Assessment

  • Strengthen → Emerging: Reproducible research advantages across part of the protocol provide positive directional evidence without satisfying the complete test.
  • Confirm → Confirmed: Independently replicated evidence satisfies every matched-resource, project-quality, speed and problem-selection requirement in the protocol. Early qualifying evidence can resolve before November 2027, with timing recorded separately.
  • Weaken → Behind: Comparable evidence establishes a material shortfall against the predicted trajectory, such as a substantial research domain persistently remaining human-led. Failed replication or an advantage dependent on unmatched resources or benchmark familiarity weakens a claimed result but, without qualifying evidence of a shortfall, leaves Not Yet Testable rather than proving failure.

Update History

DateUpdate
2026-11-07Assessment reviewed. Status: Not Yet Testable; confidence: 90%. Automated alignment research outperforms time-limited human baselines on specified failures, while the August long-horizon R&D study reports variable performance and rare methodological novelty. Neither provides the replicated, matched-resource, multi-subfield tenfold complete-project advantage required by this tracker. The November 2027 racing-branch milestone remains unresolved, including uncertainty about undisclosed systems. Clarification: Corrected the protocol pointer, dated the linked studies and separated their bounded findings from the replicated complete-project test.
2026-09-06No qualifying replicated multi-subfield tenfold research advantage has been identified. Relevant primary evidence and limitations are described above.