VR-1.1 on LiveCVEBench: Remediating real vulnerabilities

Most security teams have more vulnerability findings than they can work through, and the list grows every time a scan runs or a new CVE is published. Each repair still waits on an engineer to understand the flaw, change the code, and check that nothing else broke. None of that gets faster as the list grows, so remediation ends up setting the pace for most vulnerability programs.

VR-1.1 is built for the repair. It is the latest in Cogent’s VR family of security agents and models, and it extends the vulnerability-remediation capabilities we introduced with VR-1 earlier this year. On LiveCVEBench, VR-1.1 repaired 57.9% of 190 recently disclosed CVEs (110 vulnerabilities), compared with 45.3% for both Claude Code and Codex (86 each).

Results on LiveCVEBench’s 190-task set, prepared for leaderboard submission. Inference budgets differ across systems.

LiveCVEBench turns newly disclosed CVEs into executable repair tasks. In the setup used here, each task placed the agent in a real development environment with a report and codebase, then assessed whether its patch addressed the vulnerability and preserved intended behavior.

Security benchmarks measure different parts of the work. CyberGym asks an agent to reproduce a reported vulnerability with a working proof of concept. CyberGym-E2E begins without a vulnerability report and asks the agent to discover a vulnerability, demonstrate it, and repair what it finds. Both draw largely from OSS-Fuzz, where the task begins with a sanitizer-detected crash. LiveCVEBench covers a broader mix of injection, access-control, configuration and memory-safety flaws. It also introduces newly disclosed CVEs regularly.

Below, we examine what separates a lasting repair from a patch that only looks right, then walk through how VR-1.1 handled one.

What makes a good repair?

A patch needs to stop the attack and keep the software useful. Disabling a feature may prevent an exploit, but it can remove functionality users need. Blocking one malicious input may leave another way to trigger the same bug.

The benchmark checks both security and functionality. After the repair, the vulnerability checks must pass, and the functional checks must continue to pass.

This makes remediation a demanding test for an AI agent. It has to understand the code well enough to change its behavior, then deal with what happens when that change runs. The tests can expose a mistake even when the explanation sounds convincing.

Why a patch can look right and still be wrong

A security patch can stop the reported exploit and still leave the system unsafe or less useful.

• Disable the capability. Broadly rejecting the affected operation removes the exploit along with legitimate use.

• Leave an incomplete repair. The reported example stops, but the same unsafe condition remains reachable through another input, route, component, or sequence of events.

• Cause a functional regression. Valid inputs, error-handling contracts, or surrounding workflows no longer behave as intended.

• Create new exposure. New validation, fallback, or cleanup logic introduces a different security weakness.

• Rely on narrow validation. A check exercises only one input, entry point, or sequence of events, allowing any of the problems above to go unnoticed.

A complete repair has three independent obligations: fully repair the vulnerability, preserve intended behavior, and avoid creating new exposure. Review challenges the repair against those obligations, new tests broaden the conditions exercised, and validation checks security and normal behavior together.

How VR-1.1 works through a repair

VR-1.1 can begin by inspecting the report and source code to build a proof of concept: a small test that triggers the reported flaw. A successful reproduction gives it a failure to trace and a test to rerun against the repair.

VR-1.1 traces a vulnerability through the affected code and looks for related paths that share the same flaw. It uses that analysis to make a focused repair, such as checking an index before use or enforcing permission at the point of access.

A separate review examines the patch in a fresh context, looking for ways around the fix and bugs introduced by the change.

VR-1.1 also writes regression tests for the flaw and related cases. These give the repair a repeatable check and help catch the bug if a later code change brings it back. Validation runs the available reproduction and tests, including checks for normal behavior.

If a repair needs another attempt, VR-1.1 carries the earlier analysis, patch, and failure details into the next attempt. It can revisit the diagnosis or revise the repair based on what went wrong.

A simplified view of the repair workflow. The PoC phase and available checks depend on the run configuration.

A repair in the Fiber web framework

In a recorded VR-1.1 run on CVE-2025-48075, a form parser accepted a nested field with an invalid index. The parser used that index in an operation that could crash request handling.

VR-1.1 reproduced the crash with its own proof of concept, then added a bounds check before the operation to reject negative or excessive indices. It reran the reproduction against the patched code to check the fix. LiveCVEBench’s tests showed the change: all nine vulnerability checks passed after the patch, up from zero before it. All twelve functional checks passed both before and after.

This example comes from a separate recorded 182-task run, rather than the final 190-task submission.

The fix was small, but its placement mattered. Checking the index before using it addressed the unsafe assumption in the parser while preserving the form handling covered by the tests.

How we evaluated it

During VR-1.1’s repair phase, the task environment blocked outbound network access. LiveCVEBench’s grading tests were withheld until the agent finished. We then ran those tests against its patch.

Scores reflect each system’s final patch after its internal repair attempts.

LiveCVEBench measures repairs to known vulnerabilities in isolated environments. Passing means a patch satisfied the supplied checks. A production rollout still needs review and testing in the environment where the software will run.

For security teams, the practical value is a patch they can inspect, with the diagnosis and test results alongside it.