✓ ShippedByAgents
Vendor-published Company rollout

Atlassian's Rovo Dev code review agent cuts median PR cycle time 30.8% across 1,900+ internal repositories in a year-long evaluation

Atlassian built Rovo Dev Code Reviewer, an AI review agent integrated into Bitbucket, and evaluated it internally for over a year across more than 1,900 repositories, in a study accepted at ICSE 2026 with a Monash University co-author. It uses Claude 3.5 Sonnet with a structured zero-shot prompt (persona, chain of thought, review guidelines, PR and Jira context), a gpt-4o-mini LLM-as-judge to filter factually wrong comments, and a fine-tuned ModernBERT classifier to keep only actionable ones. Humans accept or decline every suggestion. Median PR cycle time fell 30.8%, human-written review comments fell 35.6%, and 38.7% of its comments led to a code change (against 44.45% for human comments).

The numbers (as reported)

ScopeOver a year across 1,900+ repositories
Median PR cycle time-30.8%
Human-written review comments-35.6%
Comment resolution rate38.70% (human comments 44.45%)

Facts

Company / subject
Atlassian
Tools
Rovo Dev
Models
Claude 3.5 Sonnet, gpt-4o-mini, ModernBERT
Codebase
Not applicable
Work type
Company rollout
Stack
Bitbucket, Jira
Published
2026-04-07
Last checked
2026-11-10

What the source claims Vendor-published

Claimed by the vendor that sells the tool. Not checked independently.

  • 30.8% reduction in median PR cycle time -30.8%
    “We observed a significant 30.8% reduction in median PR cycle time.”
  • 35.6% fewer human-written review comments -35.6%
  • 38.70% of Rovo Dev comments led directly to code changes 38.70% code resolution rate

Caveats

Atlassian sells Rovo Dev, so this is vendor-published, though the evaluation was accepted at ICSE 2026. It is a review agent, not one that writes the code. Cycle-time change is observational over adoption, not a controlled comparison.

Related topics: Company rollouts and costVendor case studies

Source

Inside Atlassian (Andy Wong, Minwoo Jeong)

One email a week: what actually shipped

New sourced stories, the numbers behind them, and what didn't work. No hype.

We'll only use this to send the newsletter. See the privacy page.