Study of 26,999 PRs: agent changes that touch both app code and deployment config wait 3.24x longer when reviewed, yet 43–56% merge with no human review
Researchers looked at 26,999 pull requests from the AIDev v5 and MOSAIC-3M datasets, covering six coding agents and a matched human baseline in repositories with 100+ stars. They focused on 'dual-scope' PRs that change both application code and deployment infrastructure (CI pipelines, containers, provisioning manifests). Only about 4.5% of agent PRs are dual-scope, but those PRs split two ways. When reviewed, they add a median of 13 hours and take 3.24 times as long to merge, mostly because of CI and provisioning changes. Otherwise, agent identity cuts the odds of human review by 63.7% compared with human contributors, so many merge silently.
The numbers (as reported)
| PRs studied | 26,999 (AIDev v5: 21,855; MOSAIC-3M: 5,144) |
|---|---|
| Dual-scope share | 4.48% to 4.55% of agent PRs |
| Review delay | Median +13.13 hours, 3.24x time-to-merge for reviewed dual-scope PRs |
| Unreviewed merges | 43.1% to 56.2% of dual-scope PRs merge without inspection, median under 31 minutes |
| Review odds | Agent identity lowers odds of human review by 63.7% vs. human peers |
Facts
- Company / subject
- Mohammad Wathiq Soualhi and Basem Suleiman (UNSW Sydney)
- Tools
- Claude Code, OpenAI Codex, GitHub Copilot, Cursor, Devin, Jules
- Codebase
- brownfield
- Work type
- Research / benchmark
- Stack
- —
- Published
- 2026-10-09
- Last checked
- 2026-10-26
What the source claims Independent
Reported by an independent source, such as a study or the press.
The source doesn't claim specific benefits.
Caveats
Preprint, not yet peer reviewed. One author is listed as an independent researcher. Open-source repositories only. Measures merge and review outcomes, not whether the merged changes broke anything. The six agents are taken from the paper's text; check the per-agent breakdown before publishing.
More stories by tool: Claude CodeOpenAI CodexCursorGitHub CopilotDevin
Related topics: Research and benchmarksBrownfield codebasesIndependent sources