CodeMeepo multi-agent migrations with Copilot CLI pass 18% more tests than ReCodeAgent across nine repos, including a full SONiC xcvrd Python-to-Rust port
Updated on Set published to 2026-10-25 from the arXiv submission date on abs/2610.34633.
CodeMeepo is a two-phase multi-agent repository migration workflow (translation then benchmark-guided optimization) orchestrated by a deterministic state machine over eight Copilot CLI custom-agent roles. Agents neither author nor edit a fixed external acceptance oracle; a parity agent turns coverage gaps into new milestones. On nine subjects in three language pairs (AlphaTrans Java→Python, CRUST-Bench C→Rust, SONiC xcvrd Python→Rust), all arms used claude-sonnet-5: CodeMeepo had the best pass rate on every subject and passed 1,334 vs ReCodeAgent’s 1,129 tests (+18%). On production xcvrd (15,554 LOC) it passed all 105 oracle tests vs ReCodeAgent’s 53; five optimization rounds then improved five of ten benchmarks by >5%, averaging 25% on those.
The numbers (as reported)
| Subjects | 9 repos, 3 language pairs |
|---|---|
| Tests passed vs ReCodeAgent | 1,334 vs 1,129 (+18%) |
| xcvrd (production daemon) | 105/105 oracle tests vs ReCodeAgent 53/105 |
| Optimization | 5 of 10 xcvrd benchmarks improved >5%, avg 25% on those |
| Model (all arms) | claude-sonnet-5 via Copilot CLI |
Facts
- Company / subject
- Microsoft / UC Irvine / OpenAI researchers (while at Microsoft)
- Tools
- GitHub Copilot
- Models
- claude-sonnet-5
- Codebase
- brownfield
- Work type
- Research / benchmark
- Stack
- Java, Python, C, Rust, SONiC, Apache Burr, GitHub Copilot CLI
- Published
- 2026-10-25
- Last checked
- 2026-11-16
- Updated
- 2026-11-16
What the source claims Independent
Reported by an independent source, such as a study or the press.
- Best pass rate on every subject; 18% more tests passed than ReCodeAgent on the same model +18% tests (1,334 vs 1,129)
“passes 18% more tests than a state-of-the-art multi-agent baseline using the same model”
- Full 105/105 pass on production SONiC xcvrd Python→Rust vs 53 for ReCodeAgent 105/105 vs 53/105
- Optimization phase improved half of xcvrd benchmarks by 25% on average 5/10 benchmarks, avg +25%
“its optimization phase then improves half of the benchmarks, by 25% on average”
Caveats
Research paper; authors affiliated with UC Irvine, Microsoft, and OpenAI, with contributions while at Microsoft. Benchmarks are controlled migrations with fixed oracles, not live product ship stories.
More stories by tool: GitHub Copilot
Related topics: Research and benchmarksBrownfield codebasesIndependent sources