Co-Design Bench: the best coding agent fully completes 26.4 percent of real user requests
Researchers from Alibaba and several universities built 273 tasks from real user coding sessions, where an agent must revise an existing app, sometimes from images, audio or video. The Decoder reports the best of 10 models, GPT-6 Astra, met about 70 percent of requirements on average but was fully correct on 26.4 percent of tasks. Every model did worse as a request combined more kinds of change.
Why it matters: It measures finished, working revisions of real requests, which is closer to everyday coding than single-file puzzles.
| Benchmark | Score | Type | Measured or published by |
|---|---|---|---|
| Co-Design Bench, fully correct tasks | 26.4% (GPT-6 Astra, best of 10) | Cited | Benchmark authors, as reported by The Decoder |
Alibaba makes some of the tested models. The Decoder covered it on Nov 8.