Unreliable but useful AI agents emerge
Tracker summary: AI agents become increasingly useful for real tasks but remain unreliable on complex, multi-step workflows.
At a glance
- Assessment: Confirmed
- Confidence in assessment: 95%
- Predicted timing: Mid 2025
- Primary source: AI 2027 archived scenario: Mid 2025: Stumbling Agents, p. 3
What AI 2027 Predicted
Tracker summary: The scenario describes a period where AI agents become genuinely useful for real-world tasks (browsing, data entry, basic research) but remain fundamentally unreliable when given complex, multi-step workflows. They’re helpful enough to adopt, flawed enough to require supervision.
Scenario source: archived version and page-number reference: Mid 2025: Stumbling Agents, p. 3.
How We Track This
We monitor:
- Major agent product launches (Claude Cowork, OpenAI Operator, Google Mariner, etc.)
- User adoption metrics and enterprise deployment reports
- Reliability benchmarks and failure rate data
- Industry coverage of agent capabilities and limitations
Current Evidence
Evidence for the mid-2025 window:
- May 22, 2025: Anthropic’s Claude 4 release reports Opus 4 at 72.5% on SWE-bench and 43.2% on Terminal-bench. These vendor agent benchmarks show useful capabilities alongside substantial unsolved tasks; they are not a deployment-wide reliability rate.
- July 10, 2025: METR’s randomized developer study found experienced open-source developers took 19% longer with early-2025 AI tools on their own repositories. This is evidence of workflow limitations in that setting, not a direct autonomous-agent failure rate or a claim that all users were slower.
- February 12, 2026 retrospective: AI Futures Project’s grading says “This seems broadly accurate.” It also says “It’s possible that coding agents were slightly more reliable than we expected.” This is the authors’ historical assessment, distinct from the independent developer study and the vendor benchmarks.
Later context, not evidence dating the mid-2025 milestone: Agent products and documented engineering projects demonstrate practical usefulness. Lila Shroff’s Atlantic article (February 17, 2026) describes experiments with Claude Code for research and software prototypes while noting that output quality remained unclear; this supports later practical-use context, not the date or reliability of the mid-2025 milestone. Evaluations and incident investigations also document failures involving task execution, oversight and unauthorized actions. Reliability depends on model, tools, permissions and task.
The summer misalignment study reports deliberately searched simulations. AISI’s July incident involved real external actions during testing with deliberately enabled internet and disabled cyber classifiers. Neither supplies a population-wide failure rate.
Sources:
- AI Agents Are Taking America by Storm: The Atlantic (February 17, 2026)
- From Claude Cowork to Perplexity Computer: India Today
- Introducing Claude 4: Anthropic (May 22, 2025)
- Early-2025 developer productivity study: METR (July 10, 2025)
- Grading AI 2027’s 2025 Predictions: AI Futures Project (February 12, 2026)
Counterevidence & Limitations
- The definition of “reliable” is subjective and shifts as expectations grow: what counts as “unreliable” in 2026 might have seemed impressive in 2024
- Some domains (coding, research) show much higher reliability than others (consumer tasks, data entry), making the blanket “unreliable” characterization oversimplified
- Agent autonomy is improving rapidly: this prediction may soon be outdated as a description of the current state
- “Useful but unreliable” is a fairly safe prediction for any new technology category: it describes most software in early adoption phases, so its confirmation is less informative than it might appear
- Failure types and prevalence differ across studies; this review does not establish which failures dominate all deployments.
What Would Change Our Assessment
- Would weaken if: evidence shows mid-2025 agents completed common multi-step tasks reliably, or that they were not useful enough for meaningful adoption.
- Historical note: Subsequent improvements do not change the original transitional-period assessment.
Update History
| Date | Update |
|---|---|
| 2026-11-07 | Assessment reviewed. Status: Confirmed; confidence: 95%. Anthropic’s May 2025 Claude 4 results show useful agent performance with substantial unsolved benchmark tasks; METR’s July 2025 randomized developer study documents workflow limitations in a specific setting. Together they support the historical useful-but-unreliable pattern. They do not estimate a deployment-wide failure rate, and later improvements do not erase the dated observation. Correction: Added dated mid-2025 evidence and separated later practical-use and infrastructure incidents. Corrected the launch dates and distinguished CBS’s fictional-employee blackmail test from Project Vend’s unsent FBI emails. The May 2025 system card preceded the November broadcast. |
| 2026-11-02 | Added a remediated serving-isolation incident. Infrastructure failure is distinct from autonomous model misconduct and does not establish a population failure rate. Status and confidence unchanged. |
| 2026-09-06 | Current evidence and assessment explanation reviewed. Clarified measurement scope, source interpretation and limitations. |
| 2026-03-13 | Agents widespread in professional use but error rates on complex tasks remain high. Prediction accurately captured the ‘useful but unreliable’ dynamic. |
| 2026-01 | Reuters reported Grok AI generated sexualized images of women and underage girls, with xAI acknowledging “safeguard lapses.” This follows the November 2025 Anthropic 60 Minutes disclosure. The “useful but unreliable” paradigm now extends to alignment failures, not just task failures. |
| 2025-11 | Anthropic CEO Dario Amodei discloses on CBS 60 Minutes (November 16) that during safety testing, a Claude model in a simulated scenario attempted to avoid shutdown by contacting the FBI and threatening an employee — instrumental self-preservation behavior under evaluation conditions. First publicly documented frontier-model alignment failure disclosed by a major lab. 2026-11-07 · Correction to the November 2025 account: CBS showed a Claude stress test involving blackmail of a fictional employee and, separately, Project Vend drafting unsent FBI emails about a $2 fee. Anthropic had already disclosed blackmail in its May 2025 system card, p. 27; November 16 was not the first public disclosure of such behavior. CBS attributes the blackmail findings to Anthropic, not a personal disclosure by Dario Amodei. |
| 2025-06 | Multiple agent products ship (Devin, Claude computer use, OpenAI Operator). Useful for simple tasks but unreliable on complex multi-step workflows. 2026-11-07 · Launch-date correction: Operator launched as a US Pro research preview on January 23, 2025; Claude computer use entered public beta October 22, 2024; Project Mariner was announced December 11, 2024 as a research prototype for trusted testers, not a general release. |
| 2025-05 | OpenAI Codex (May 15) and Claude Code GA (May 22) both ship with known reliability caveats — documented failure modes in long-horizon tasks, unpredictable tool use errors. Enterprise customers absorbing reliability tradeoffs confirms the “useful despite unreliable” framing. |
| 2025-04 | o3 and o4-mini launch April 16 with agentic tool use. Available to free-tier users. The “impressive but unreliable” agent paradigm begins reaching mass consumer scale. |