Foam
← All research

Root cause accuracy with Claude Code

We reran the Foam RCA benchmark with a newer agent. The agent got better in every setup, and telemetry access is still the largest gain.

In April we published a benchmark for root cause analysis: 22 real production incidents, each with a human-written answer key. An agent reads the codebase, investigates, and writes a root cause. A GPT-4o judge marks it correct only if it names the same incident, the same precise cause, and would lead an engineer to the same fix.

This run swaps one thing. Cursor on opus-4.6 is replaced by Claude Code on Fable 5.1. The 22 incidents, the answer key, the judge prompt, and the three tool setups are unchanged.

What changed between the April and September runs The agent is swapped from Cursor to Claude Code. The incidents, answer key, judge and three tool setups stay fixed. HELD FIXED 22 production incidents Human-written answer key GPT-4o judge, same rubric THE AGENT Cursor · opus-4.6 Claude Code · Fable 5.1 × THREE TOOL SETUPS Sentry MCP No extra tools Foam MCP (query-otel) Only the agent moves, so any change in score is the agent.

Results

86.4%
Claude Code with Foam MCP (19/22)
+23pp
From adding Foam MCP to Claude Code
+5
Incidents solved over Cursor with Foam MCP
RCA accuracy by tool setup, April versus September With Sentry MCP, accuracy rises from 40.9 to 54.5 percent. With no extra tools, from 54.5 to 63.6 percent. With Foam MCP, from 63.6 to 86.4 percent. April · Cursor on opus-4.6 September · Claude Code on Fable 5.1 Sentry MCP 40.9% (9/22) 54.5% (12/22) +3 No extra tools 54.5% (12/22) 63.6% (14/22) +2 Foam MCP 63.6% (14/22) 86.4% (19/22) +5 The stronger agent gains the most when it can query telemetry.

The ordering is the same as April: Sentry MCP scores below the bare agent, and Foam MCP scores highest. Claude Code beats Cursor in every setup by two to five incidents. The largest jump is with Foam MCP, where Claude Code reaches 19 of 22, the same score our own Foam agent reached in April.

A better model does not remove the need for telemetry. It makes telemetry more valuable: the gap between Claude Code alone and Claude Code with Foam MCP is 23 points, wider than it was for Cursor.
Accuracy gained from adding Foam MCP, April versus September Adding Foam MCP took Cursor from 54.5 to 63.6 percent and Claude Code from 63.6 to 86.4 percent. 54.5% 63.6% +9pp Cursor · April 63.6% 86.4% +23pp Claude Code · September agent alone agent + Foam MCP

For Cursor in April, adding Foam MCP was worth 9 points. For Claude Code it is worth 23. The same tool is worth more to the stronger agent.

Where the setups differ

Foam MCP gives the agent one tool, query-otel, for reading traces, logs, and metrics. Three incidents (4, 18, and 21) were solved only with it. In each, the deciding evidence is runtime data: the exact query a tool sent, the query that exhausted ClickHouse memory, and a run ID missing from the database.

How the 22 incidents split across tool setups Ten incidents are solved by all three setups, six by Foam MCP and one other setup, three only with Foam MCP, and three by no setup. 22 INCIDENTS, CLAUDE CODE ON FABLE 5.1 10 · all three setups 6 6 · Foam MCP + one other 3 · only Foam MCP 3 3 · no setup

Sentry MCP helped on two incidents (0 and 11) where the stack trace carries the whole story. It hurt on four (16, 19, 20, and 22) where the Sentry issue points at a symptom. There the agent anchored on the breadcrumbs instead of reading the code, and scored lower than it did with no extra tools at all.

Per-incident results for each tool setup A grid of 22 incidents by three setups. Foam MCP solves every incident except 8, 9 and 13, which no setup solves. 0 1 2 3 4 5 6 7 8 9 10 11 12 13 14 15 16 18 19 20 21 22 Sentry MCP No extra tools Foam MCP ○ solved × missed

Three incidents stayed unsolved in every setup, as they were for every Cursor setup in April. Incident 8 is a duplicate-key error that is expected lock behavior, not a bug. Incident 9 is an Anthropic rate limit exhausted by accumulated tool output. Incident 13 is a git pre-flight check that fails because production ECS containers ship without a .git directory.

Full results

Tool setup April · Cursor September · Claude Code
Sentry MCP9/22 (40.9%)12/22 (54.5%)
No extra tools12/22 (54.5%)14/22 (63.6%)
Foam MCP14/22 (63.6%)19/22 (86.4%)

Every agent output, score, and the scorer are in the benchmark repository.

Foam Research · September 2026