AI substantially outperforms the best humans across cognitive domains

Last updated

Tracker summary: Artificial superintelligence is much better than the best human at every cognitive task.

The broad cognitive-domain test remains unresolved; a collection of benchmark wins is insufficient.

At a glance

  • Assessment: Not Yet Testable
  • Confidence in assessment: 90%
  • Outcome: unresolved
  • Timing: pending
  • Evidence: insufficient
  • Predicted timing: December 2027, racing branch
  • Primary source: Appendix J - How we’re forecasting the capability progression beyond superhuman coders, archived PDF p. 54; December 2027: The Agent-5 Collective, archived PDF p. 26

What AI 2027 Predicted

Tracker summary: Artificial superintelligence is much better than the best human at every cognitive task. The source is Appendix J, ASI milestone; December 2027 racing branch in the AI 2027 scenario.

Archived scenario version and section-page references. The linked live scenario may change; these definitions use the archived version identified here.

Scenario source locators: Appendix J - How we’re forecasting the capability progression beyond superhuman coders: archived PDF, p. 54; December 2027: The Agent-5 Collective: archived PDF, p. 26.

How We Track This

Use a frozen domain matrix: AI research, software engineering, mathematics, scientific reasoning, strategic planning, communication and practical knowledge work. For each, require at least three unfamiliar complete tasks with leading-human comparators, blinded quality evaluation and disclosed resources. Candidate proxy: better quality on all domain aggregates and at least 2× faster time to equivalent quality in each domain, independently replicated. Record any material counterexample.

These operational thresholds specify this tracker’s test. Where the scenario uses qualitative language, they are interpretations rather than numbers quoted from the original.

Current Evidence

The available research evidence measures particular tasks and workflows, not the frozen, independently replicated domain matrix. This is an evidence limitation, not proof that a hidden system lacks the capability.

  • Beyond Final Scores (August 13, 2026) evaluates seven frontier models on 36 long-horizon R&D tasks; agents mainly adapt established techniques, with variable performance and rare methodological novelty. Its research-task scope does not cover all domains or supply the leading-human comparisons and independent replication required here.
  • Anthropic Risk Report (August 2026), executive summary and §3 assesses internal AI-assisted R&D and estimates acceleration below a factor of two, with difficult measurement. This developer assessment of one workflow is not an independent, multi-domain comparison against the best humans.

Counterevidence & Limitations

No finite matrix proves “every cognitive task.” Passing supports broad superhuman capability under the tested scope; it does not conclusively establish the universal claim. Physical embodiment, affordability and deployment are separate qualifiers. The date is branch-conditional.

Missing qualifying public evidence does not establish that the event has not happened privately. The assessment concerns the documented test and its observable evidence.

What Would Change Our Assessment

  • Strengthen → Emerging: Reproducible positive results across part of the domain matrix, with leading-human comparators, support the direction without satisfying the full test.
  • Confirm → Confirmed: Independently replicated evidence satisfies every domain, unfamiliar-task, quality and speed requirement in the frozen test; record early or within-window timing separately.
  • Weaken → Behind: Comparable evaluations establish persistent material shortfalls against the predicted trajectory, including a domain where leading humans reproducibly retain an advantage. Missing measurements alone retain Not Yet Testable.

Update History

DateUpdate
2026-11-07Assessment reviewed. Status: Not Yet Testable; confidence: 90%. InnovationEval finds limited independent ML innovation under its tested setup, while the long-horizon research evaluation finds rare methodological novelty. Neither supplies the broad, resource-matched comparison against the best humans required to test the full claim. Separated task-specific research evidence from the full multi-domain comparison.
2026-09-06The broad cognitive-domain test remains unresolved; a collection of benchmark wins is insufficient. Relevant primary evidence and limitations are described above.