MSR 2026 study of 33,596 agent-written GitHub pull requests: 71% merged, and the rejects are mostly ignored, duplicated or failing CI
Researchers studied 33,596 pull requests written by five coding agents in GitHub projects with 100+ stars (AIDev-pop dataset). Overall, 71.48% were merged, with rates ranging from 82.59% for Codex down to 43.04% for Copilot. Docs, CI and build tasks merged most often, and performance and bug-fix work least. Unmerged PRs were larger and failed CI more often. In a hand-coded sample of rejected PRs, the most common reason was that no human ever meaningfully reviewed them (38%), followed by duplicates (23%) and CI/test failures (17%).
The numbers (as reported)
| PRs studied | 33,596 across five agents |
|---|---|
| Overall merge rate | 71.48% |
| Merge rate by agent | Codex 82.59%, Cursor 65.22%, Claude Code 59.04%, Devin 53.76%, Copilot 43.04% |
| Top rejection reason | Abandoned or never reviewed (228 of 600 sampled) |
| Effect of CI failures | Each failed CI check cuts merge odds by about 15% |
Facts
- Company / subject
- Drexel University and Missouri University of Science and Technology (MSR 2026)
- Tools
- OpenAI Codex, GitHub Copilot, Devin, Cursor, Claude Code
- Codebase
- brownfield
- Work type
- Research / benchmark
- Stack
- —
- Published
- 2026-01-21
- Last checked
- 2026-10-20
What the source claims Independent
Reported by an independent source, such as a study or the press.
The source doesn't claim specific benefits.
Caveats
Short MSR Mining Challenge paper. Sample sizes per agent are very uneven (Codex 21,799 PRs vs. Claude Code 459). Measures merge outcomes, not code quality or productivity. Studies the same AIDev dataset as the existing MSR 2026 adoption story.
More stories by tool: Claude CodeOpenAI CodexCursorGitHub CopilotDevin
Related topics: Research and benchmarksBrownfield codebasesIndependent sources