AI teams conduct complete research projects with little human direction

Last updated

Tracker summary: AI collectives produce most research progress while most humans cease to contribute usefully; leading humans retain some research judgment.

Automated research is substantive in constrained settings; three qualifying complete projects have not been verified.

At a glance

  • Assessment: Emerging
  • Confidence in assessment: 80%
  • Outcome: unresolved
  • Timing: pending
  • Evidence: proxy
  • Predicted timing: June 2027
  • Primary source: June 2027: Self-improving AI, archived PDF p. 14; Appendix I - Managing a corporation of AIs, archived PDF p. 53

What AI 2027 Predicted

Tracker summary: AI collectives produce most research progress while most humans cease to contribute usefully; leading humans retain some research judgment. The source is June 2027: Self-improving AI in the AI 2027 scenario.

Archived scenario version and section-page references. The linked live scenario may change; these definitions use the archived version identified here.

Scenario source locators: June 2027: Self-improving AI: archived PDF, p. 14; Appendix I - Managing a corporation of AIs: archived PDF, p. 53.

How We Track This

Require three independently scoped machine-learning research projects lasting at least 14 calendar days each, covering question selection, experiment design, execution, interpretation and accepted output. Human interventions must account for no more than 10% of logged research decision time and must not supply the central hypothesis or conclusion. At least three of five blinded qualified reviewers judge each output technically valid and meaningfully novel relative to a frozen literature cutoff.

These operational thresholds specify this tracker’s test. Where the scenario uses qualitative language, they are interpretations rather than numbers quoted from the original.

Current Evidence

As a cross-domain precursor, Anthropic’s October 20 protein-design study reports a 64-hour campaign with four human follow-ups and approximately 10,000 GPU-hours. External laboratories tested 211 designs, of which 17 met the study’s binding criteria: every replicate had an SPR-measured KD no greater than 1 µM, with no CLEC12A signal. LqhIII was assessed only at Adaptyv, so not all binding results were reproduced at both laboratories. These are recombinant-toxin binding results, not demonstrated cell or animal neutralization or treatment efficacy. Humans supplied targets and tools, added designs and continued to direct the work and supervise reporting; no resource-matched human trial was established.

OpenAI’s separate biological-data analysis generated leads for experimental follow-up. Neither study establishes one of the complete autonomous machine-learning projects required here. The protein-design campaign does not establish at least 14 calendar days of qualifying autonomous research, logged human intervention at or below 10% of research decision time, absence of a human-supplied central hypothesis or conclusion, or acceptance by three of five blinded reviewers against a frozen literature cutoff. Later laboratory testing cannot simply extend the autonomous campaign’s duration, and individual designs are not independently scoped projects.

August 28, 2026: Anthropic study announcement: Automated alignment researchers improved well-characterized objectives over short research runs. The dated primary announcement links the full alignment study, whose page does not expose a publication date; August 28 dates the announcement, not an inferred first-publication timestamp for the study.

August 13, 2026: first arXiv submission: Beyond Final Scores: A Systematic Evaluation of Agents for Long-Horizon AI Research and Development reports variable research performance and limited conceptual novelty. The required project duration, intervention records and independent judgments remain unresolved.

OpenAI’s September 6 research account reports agents completing defined research tasks spanning several human workdays, with humans directing and judging the work. Anthropic’s September 4 formalization report describes an 11-day, largely autonomous Lean formalization of an existing proof of Fermat’s Last Theorem. It reports 13 million lines and 29,500 intermediate theorems used, with occasional high-level human instructions.

Google’s August 27 detailed Teamwork report, added here as earlier evidence, attributes seven mathematical results to Gemini 3.1 Pro, with three reproduced using Gemini 3.7 Flash. Google describes expert review and a Lean-verified result. These participant reports extend the evidence for coordinated research work. They do not establish three qualifying machine-learning projects with the required duration, intervention logs and blinded review. The mathematical artifacts have not been independently validated by this tracker.

OpenAI’s September 8 research release presents a 165-page proof and Lean formalization claiming finite-time blowup for forced three-dimensional Navier-Stokes. OpenAI says an internal system significantly more capable than Astra produced the result through a large parallel-agent effort. The Clay Mathematics Institute said on September 11 that the problem had “apparently been settled” and that its evaluation process would be unhurried. The Isaac Newton Institute described the result as an OpenAI claim. This is a substantial coordinated research artifact under external review, not an accepted result. It also falls outside this dossier’s test because it is not one of three machine-learning projects and the public intervention record does not establish the 10% human-decision ceiling.

Anthropic’s September 17 biomolecular-modeling report describes a general-purpose internal research model, supervised by two scientists, optimizing more than 30 open-source models in just under four weeks. Anthropic released the optimized code and reports output checks plus substantial inference-speed improvements. This is a multi-week research-engineering project with concrete artifacts. It does not satisfy this dossier’s complete-project test: humans supplied the objective and supervision, logged human decision time is not reported, and the required three independently scoped machine-learning projects and blinded novelty reviews are absent.

OpenAI’s October 26 scientific-research announcement reports further mathematics, computer-science and physics results and external collaborations to examine them. Mathematics and computer-science writeups and Lean artifacts are public; the physics group deferred release of its AI-produced artifacts while preparing its own account. OpenAI describes the outputs as raw research data. Participation by independent experts is not itself validation: these remain provisional claims here, not established resolutions of the underlying problems. The announcement does not establish the required three machine-learning projects, duration, human-decision records or blinded novelty review.

Counterevidence & Limitations

Epoch’s October InnovationEval found that two uncontaminated models did not reproduce a comparable method on one post-training target. Its separate 11-task organizational evaluation exposed weaknesses in experimental design and interpretation. Neither study tested three complete, independently scoped machine-learning projects with intervention logs and blinded novelty reviews. The existing supervised project artifacts remain affirmative but partial evidence.

The 14-day and 10% rules are tracker thresholds. Logs must include failed experiments, compute budgets and human assistance. Parallel coding, narrow benchmark optimization or nominal elapsed time do not suffice. Passing this test establishes autonomous projects, not the scenario’s entire country-scale research economy.

Missing qualifying public evidence does not establish that the event has not happened privately. The assessment concerns the documented test and its observable evidence.

What Would Change Our Assessment

  • Confirm: Three independently scoped projects meet the full test: at least 14 calendar days each; human intervention no more than 10% of logged research decision time, without supplying the central hypothesis or conclusion; and technical validity and meaningful novelty accepted by at least three of five blinded qualified reviewers against a frozen literature cutoff.
  • Weaken or move to Behind after June 2027: Humans supply decisive ideas or repairs, independent reviewers reject the work, or outputs cannot be reproduced, leaving the three-project test unmet. This is a prospective trigger, not a current status or confidence change.

Evidence satisfying the full test would change the assessment; early qualifying evidence can be considered before the scenario window. Partial proxies are recorded with their scope.

Update History

DateUpdate
2026-11-07Assessment reviewed. Status: Emerging; confidence: 80%. Anthropic’s September 17 biomolecular-modeling report describes a multi-week project optimizing more than 30 models with released code. Automated alignment research adds bounded research results. Human supervision and missing intervention logs and blinded novelty judgments leave the three-project autonomy test unresolved. Clarification: Dated the alignment work by its August 28, 2026 announcement and the long-horizon study by its August 13 submission. Confirmation and post-June-2027 weakening conditions remain tied to the complete-project test.
2026-11-02Added provisional science claims under external examination, distinguishing public mathematics/computer-science artifacts from deferred physics materials. The complete autonomous-project test remains unresolved. Status and confidence unchanged.
2026-10-26Added a cross-domain protein-design precursor with laboratory binding measurements and continuing human direction. The complete autonomous-ML-project test remains unresolved. Status and confidence unchanged.
2026-10-12Epoch’s method-discovery and workplace-task evaluations found bounded research-design and novel-method limitations. They do not resolve the three-project autonomy test or cancel the supervised project evidence. Status and confidence unchanged.
2026-09-21Anthropic reported a supervised, multi-week project in which Claude optimized more than 30 biomolecular models and released the code. The artifact is material but does not meet the three-project, human-intervention and blinded-review thresholds; status and confidence unchanged.
2026-09-14OpenAI released a proposed Navier-Stokes result and Lean formalization produced through a large parallel-agent effort. Clay said the problem had apparently been settled but its evaluation remains pending. The artifact is material and does not satisfy the three-project machine-learning research test. Status and confidence unchanged.
2026-09-07Added attributed lab reports of research agents, mathematical formalization and coordinated research. These extend project-level evidence without resolving the complete autonomous-research test. Status and confidence unchanged.
2026-09-06Automated research is substantive in constrained settings; three qualifying complete projects have not been verified. Relevant primary evidence and limitations are described above.