Briefs

BriefsToolAvailable now1 min read

Collinear AI releases CWE-bench v2 to test AI cyber capabilities

Collinear AI released CWE-bench v2, an AI benchmark featuring 100 tasks across 91 vulnerability classes to test cyber capabilities.

Collinear AI releases CWE-bench v2 to test AI cyber capabilities — image from Hugging Face
Image: Hugging Face

Summary

Collinear AI released CWE-bench v2, offering 100 public tasks covering 91 different weakness classes to test the cyber capabilities of AI models. The benchmark uses programmatic scoring and Dockerized environments via Harbor. Evaluated models include Gemini 4 Argon, Claude Opus 5.5, and DeepSeek V4.1 Flash.

What changed

  • Collinear AI released CWE-bench v2 with 100 public tasks covering 91 different CWEs.
  • Programmatic scoring allows deterministic evaluation of AI models.
  • Tasks can be run directly using Harbor from the Hugging Face repository.

Availability

Available now

“CWE-bench v2 opens up a public set of 100 tasks that anyone can run and evaluate their models on.”

Founder takeaway

If you are evaluating AI agent security capabilities, you can run CWE-bench v2 using Harbor and the public Hugging Face dataset.

Source

Hugging FaceCWE-bench v2: More coverage, quality, and transparency

Announced Checked

Researched and drafted with AI from the official sources above, then checked automatically before publishing. Spotted a mistake? Tell us.

SharePost on XLinkedIn

Related

More briefs

All briefs
  1. LangChain expands LangSmith BYOC to Azure and Google Cloud — image from LangChain

    ToolRolling out

    LangChain expands LangSmith BYOC to Azure and Google Cloud

    LangChain makes its LangSmith Bring Your Own Cloud deployment option available on AWS, Azure, and Google Cloud.

    LangChain1 min read

  2. Supabase adds Claude-issued identity assertions for MCP auth — image from Supabase

    ToolRolling out

    Supabase adds Claude-issued identity assertions for MCP auth

    Supabase adds support for Claude to issue signed identity assertions for Model Context Protocol authentication when identity providers lack extension support.

    Supabase1 min read

  3. Gen Digital built the Norton Revamp AI news engine using Amazon Bedrock AgentCore Web Search — image from AWS

    ToolAvailable now

    Gen Digital built the Norton Revamp AI news engine using Amazon Bedrock AgentCore Web Search

    Gen Digital details how its small engineering team built Norton Revamp, an AI-powered professional reputation platform using Amazon Bedrock AgentCore Web Search.

    AWS1 min read

One email, every Monday

Briefs are published here through the day. Monday Heist is the email: one useful AI workflow, the files to use it, and the catch. The first issue is being prepared; subscribers get it first. Free, and you can leave in one click.

One email. Every Monday.