✓ ShippedByAgents
Independent Research / benchmark

MSR 2026 study of 33,596 agent-written GitHub pull requests: 71% merged, and the rejects are mostly ignored, duplicated or failing CI

Researchers studied 33,596 pull requests written by five coding agents in GitHub projects with 100+ stars (AIDev-pop dataset). Overall, 71.48% were merged, with rates ranging from 82.59% for Codex down to 43.04% for Copilot. Docs, CI and build tasks merged most often, and performance and bug-fix work least. Unmerged PRs were larger and failed CI more often. In a hand-coded sample of rejected PRs, the most common reason was that no human ever meaningfully reviewed them (38%), followed by duplicates (23%) and CI/test failures (17%).

The numbers (as reported)

PRs studied33,596 across five agents
Overall merge rate71.48%
Merge rate by agentCodex 82.59%, Cursor 65.22%, Claude Code 59.04%, Devin 53.76%, Copilot 43.04%
Top rejection reasonAbandoned or never reviewed (228 of 600 sampled)
Effect of CI failuresEach failed CI check cuts merge odds by about 15%

Facts

Company / subject
Drexel University and Missouri University of Science and Technology (MSR 2026)
Tools
OpenAI Codex, GitHub Copilot, Devin, Cursor, Claude Code
Codebase
brownfield
Work type
Research / benchmark
Stack
—
Published
2026-01-21
Last checked
2026-10-20

What the source claims Independent

Reported by an independent source, such as a study or the press.

The source doesn't claim specific benefits.

Caveats

Short MSR Mining Challenge paper. Sample sizes per agent are very uneven (Codex 21,799 PRs vs. Claude Code 459). Measures merge outcomes, not code quality or productivity. Studies the same AIDev dataset as the existing MSR 2026 adoption story.

More stories by tool: Claude CodeOpenAI CodexCursorGitHub CopilotDevin

Related topics: Research and benchmarksBrownfield codebasesIndependent sources

Source

arXiv 2601.15195 (MSR 2026)

One email a week: what actually shipped

New sourced stories, the numbers behind them, and what didn't work. No hype.

We'll only use this to send the newsletter. See the privacy page.