BriefsNewsAvailable now
Cognition launches FrontierCode 2.0 to test AI coding agent user experience
Cognition released FrontierCode 2.0, a new AI coding benchmark featuring broader repository coverage, stricter mergeability standards, and larger task scopes.
Summary
Cognition released FrontierCode 2.0 to evaluate how AI models handle user experience and unstated constraints. The updated benchmark covers more repositories, raises mergeability standards, and tests larger code modifications. Top models like Opus 5.5 score only 45 percent, showing plenty of room for improvement.
What changed
- Drew tasks from more repositories, covering security patches and performance optimizations.
- Raised mergeability standards with stricter correctness and design requirements.
- Increased task scope with reference solutions modifying more files and lines in larger codebases.
- Scored only 45 percent for the top-performing model, Opus 5.5.
Availability
Available now“We are excited to introduce FrontierCode 2.0 , a new generation of our coding benchmark series and our best attempt yet to evaluate user experience.”
Founder takeaway
Top AI coding models still struggle with complex tasks and unstated constraints in real codebases. Watch model outputs closely for errors until benchmarks improve.
Source
Researched and drafted with AI from the official sources above, then checked automatically before publishing. Spotted a mistake? Tell us.
Related
More briefs
Amazon Quick adds smart approvals for agentic AI actions
Amazon Quick introduces smart approvals to automate routine agentic tasks while keeping human oversight for high-impact actions.
AWS1 min read
ElevenLabs introduces Cue forced alignment AI model
ElevenLabs releases Cue, an AI forced alignment model that places every character in an audio file on a precise timeline and powers its Scribe model family.
ElevenLabs1 min read
GitHub makes Copilot Memory generally available on paid Copilot plans
GitHub introduces new AI intelligence capabilities including Copilot Memory, dreaming, and multi-repository code search.
GitHub1 min read
One email, every Monday
Briefs are published here through the day. Monday Heist is the email: one useful AI workflow, the files to use it, and the catch. The first issue is being prepared; subscribers get it first. Free, and you can leave in one click.
One email. Every Monday.