Atlassian's Rovo Dev code review agent cuts median PR cycle time 30.8% across 1,900+ internal repositories in a year-long evaluation
Atlassian built Rovo Dev Code Reviewer, an AI review agent integrated into Bitbucket, and evaluated it internally for over a year across more than 1,900 repositories, in a study accepted at ICSE 2026 with a Monash University co-author. It uses Claude 3.5 Sonnet with a structured zero-shot prompt (persona, chain of thought, review guidelines, PR and Jira context), a gpt-4o-mini LLM-as-judge to filter factually wrong comments, and a fine-tuned ModernBERT classifier to keep only actionable ones. Humans accept or decline every suggestion. Median PR cycle time fell 30.8%, human-written review comments fell 35.6%, and 38.7% of its comments led to a code change (against 44.45% for human comments).
The numbers (as reported)
| Scope | Over a year across 1,900+ repositories |
|---|---|
| Median PR cycle time | -30.8% |
| Human-written review comments | -35.6% |
| Comment resolution rate | 38.70% (human comments 44.45%) |
Facts
- Company / subject
- Atlassian
- Tools
- Rovo Dev
- Models
- Claude 3.5 Sonnet, gpt-4o-mini, ModernBERT
- Codebase
- Not applicable
- Work type
- Company rollout
- Stack
- Bitbucket, Jira
- Published
- 2026-04-07
- Last checked
- 2026-11-10
What the source claims Vendor-published
Claimed by the vendor that sells the tool. Not checked independently.
- 30.8% reduction in median PR cycle time -30.8%
“We observed a significant 30.8% reduction in median PR cycle time.”
- 35.6% fewer human-written review comments -35.6%
- 38.70% of Rovo Dev comments led directly to code changes 38.70% code resolution rate
Caveats
Atlassian sells Rovo Dev, so this is vendor-published, though the evaluation was accepted at ICSE 2026. It is a review agent, not one that writes the code. Cycle-time change is observational over adoption, not a controlled comparison.
Related topics: Company rollouts and costVendor case studies