Devin case studies: 6 sourced stories
Stories using Cognition's Devin agent. Of the 6 stories, 2 were published by a tool vendor, 2 by the people who did the work and 2 by independent researchers or press. 6 involve an existing codebase and 0 started from scratch. Every number links to where it was published.
Study of 26,999 PRs: agent changes that touch both app code and deployment config wait 3.24x longer when reviewed, yet 43–56% merge with no human review
Mohammad Wathiq Soualhi and Basem Suleiman (UNSW Sydney) · Claude Code, OpenAI Codex, GitHub Copilot, Cursor, Devin, Jules
CaveatsPreprint, not yet peer reviewed. One author is listed as an independent researcher. Open-source repositories only. Measures merge and review outcomes, not whether the merged changes broke anything. The six agents are taken from the paper's text; check the per-agent breakdown before publishing.
26,999 (AIDev v5: 21,855; MOSAIC-3M: 5,144)
PRs studied
Comarch compresses a five-year rebuild of its 10-million-line Clarion ERP to 18 months with Devin
Comarch · Devin
CaveatsPublished by Cognition as a customer story. The 18-month timeline is a plan in progress, not a finished migration. No cost or output code-size figures.
About 10 million lines of Clarion, 20 years old
Codebase
Assembled auto-approves low-risk PRs with Codex and Claude Code reviewers, lifting PRs merged to 2.4× its pre-agent baseline
Assembled · OpenAI Codex, Claude Code, Devin, 143.dev (Assembled)
CaveatsSelf-reported by Assembled's CTO. The author calls these observational rollout results, not a randomized trial, and says the auto-reviewer doesn't get credit for every point. The window is short (three weeks after full-team launch), and the bug figures are explicitly too early to be conclusive. The pre-agent baseline period is not precisely dated, and no absolute PR counts are given. Category multipliers differ between the intro (bugfixes 3.7×, refactors 7.5×, features 2.5×) and the results list (refactoring 7.5×, test work 3.4×). The $3.57 per attempt is an estimate, not total spend. Codex and Claude Code are the reviewers; Devin is mentioned only for non-engineer PRs, and the agents that wrote most of the PRs are not broken down.
2.4× pre-agent baseline
PRs merged
MSR 2026 study of 33,596 agent-written GitHub pull requests: 71% merged, and the rejects are mostly ignored, duplicated or failing CI
Drexel University and Missouri University of Science and Technology (MSR 2026) · OpenAI Codex, GitHub Copilot, Devin, Cursor, Claude Code
CaveatsShort MSR Mining Challenge paper. Sample sizes per agent are very uneven (Codex 21,799 PRs vs. Claude Code 459). Measures merge outcomes, not code quality or productivity. Studies the same AIDev dataset as the existing MSR 2026 adoption story.
33,596 across five agents
PRs studied
Vellum's 15 engineers double monthly PRs from 946 to 1,910 with background agents and Codex as first-pass reviewer
Vellum · Devin, OpenAI Codex, Cursor, Claude Code
CaveatsSelf-reported on Vellum's company blog, which also promotes a webinar with Cognition (maker of Devin). PR counts compare one month with another; PR size and quality are not measured. The year of the May baseline is 2024 per the post; the October figure's year is not stated explicitly. Humans review post-merge rather than before merge, so Codex review is the main pre-merge gate.
946 in May → 1,910 in October
PRs
Nubank uses fine-tuned Devin to split a 6M-line ETL monolith, claiming 8–12x engineering-hour efficiency
Nubank · Devin
CaveatsVendor-published. The page states 8x, 8–12x and 12x in different places. Efficiency compares typical manual hours with hours spent prompting and reviewing. Cost compares Devin's run cost with engineer hourly cost against an estimated manual baseline. The work was highly repetitive and suited to agents. Needed custom fine-tuning, and early weeks had frequent errors to fix.
6M+ lines, ~100,000 data-class implementations
Codebase