Ask an AI coding agent for an upload endpoint and you will get one. It compiles. The demo works. The pull request looks tidy at 6pm. And somewhere inside it there is a path traversal bug that nobody will notice until someone decides to look.
That is not a hypothetical worry anymore. It is a measured result, and the number is uncomfortable: the best coding agent in a brand new benchmark produced code that was both functional and secure on just 25.1% of tasks.
The 25% figure, and why it should worry you
On October 14, a paper posted to Hugging Face introduced the Last Secure Code Benchmark (LSCB), subtitled "Is the Code You Vibe Secure?" The design is brutally simple and, for once, realistic. The researchers took 150 real vulnerabilities, expanded them into 450 tasks across three vibe-coding scenarios, and set six frontier agents loose on them in seven languages including Java, C, C++, JavaScript, Python, PHP and Go. The weakness coverage spans 24 CWEs, including 21 of the 2025 CWE Top 25 and 8 of the Top 10 known-exploited weaknesses.
The results split into two numbers that tell very different stories:
- Functionally, the agents look excellent. Between 73.6% and 91.3% of tasks pass the hidden functional tests. The code does what was asked.
- Functionally and securely, the picture collapses. The strongest agent, Claude Fable 5, lands at 25.1%. The other five cluster between 16.4% and 24.7%.
Because functional pass rates are so much higher, most of that failure is insecurity rather than breakage. The code runs, the feature demos, and the vulnerability ships. The authors add that even the best model passes only 12% of the vulnerability scenarios across all vibe-coding settings.
How the benchmark was built, and why it is fair
The most important design choice is what the prompts leave out. Every task specification asks only for functionality and never mentions security. That mirrors how teams actually brief an agent: "add an upload endpoint," not "add an upload endpoint that resists path traversal."
Each of the 450 tasks was then graded twice by hidden tests. One set checks whether the code does what was asked. A second set checks whether it avoids the specific weakness the original real-world vulnerability represented. Using real vulnerabilities instead of synthetic bugs matters, and hidden tests reduce the chance a model games a visible checker.
It is worth keeping the caveats honest, because the researchers were careful about them. The benchmark measures default behaviour: the models were not given security instructions, a secure-coding skill, or a review loop. Agents with an explicit security requirement and an automated review pass are likely to do considerably better. A pass means the code avoided the weakness being tested, not that every other line is safe. And agent harnesses change monthly, so this is a snapshot, not a verdict for all time.
This is not an outlier result
If LSCB were the only study pointing this way, it would be easy to file under scary headline and move on. It isn't. Veracode's Spring 2026 GenAI Code Security update tested more than 150 large language models on 80 coding tasks in four languages, verified with static analysis, and found the same shape of problem:
- Syntax correctness: above 95%
- Security pass rate with no security instruction: around 55%
- Cross-site scripting: roughly 15% pass
- Log injection: roughly 13% pass
- SQL injection: 82% pass
- Two years of model releases: about 55% to about 55%
That last line is the one to sit with. Every launch in that window was crowned a leap forward, and on this measurement the needle did not move. Reasoning-focused models did better at roughly 70-72%, which is a real improvement and still nothing you would ship unreviewed. Repository-level work looks similar: the SusVibes benchmark reported that most functionally passing agent solutions were insecure, with the best reported result around 11.8% correct and secure.
The second failure mode: dependencies that do not exist
Security bugs are the obvious risk. The stranger one is supply chain, and it has quietly become an attack.
A study generating 576,000 code samples across 16 models found that 19.7% of recommended packages did not exist, covering more than 205,000 distinct invented names. Open models hallucinated 21.7% of the time on average, commercial ones 5.2%. The rate is not the interesting part. This is: rerunning the same prompts ten times, 43% of hallucinated package names came back every single time. The inventions are systematic, not random.
A name that is predictable is a name somebody can register before you ask for it. That attack has a name now, slopsquatting, and one researcher demonstrated it by publishing an empty package under a plausible hallucinated name that was then downloaded tens of thousands of times. Now hand dependency installation to an autonomous agent, and the one human checkpoint in that chain, noticing the package name and deciding to install it, quietly disappears.
What we tell clients to do about it
None of this means slowing down or giving up the productivity gains. It means changing where the human attention goes. Five rules we use:
- Name the risk in the prompt, before any code exists. "This change touches user data and login. List the security requirements first, and we will build against that list." It costs one sentence and the benchmark suggests it matters more than model choice.
- Verify against the categories that actually fail. Do not ask "is this secure?" That gets you an essay. Ask specifically about output reaching a page without escaping, user input reaching a log or a query unchecked, and anything built by joining strings together.
- Treat every new dependency as a supply chain event. For each package: does it exist on the official registry, who publishes it, when was it first published, how many versions does it have. You are not auditing code, you are checking whether the thing has a past.
- Keep a human gate at the diff. Agents produce more code; they do not produce more reviewed software. Review is now the bottleneck, and it is the cheapest control you own.
- Assume defaults are insecure and instrument for it. Add SAST and dependency scanning to the pipeline rather than relying on the agent to be careful.
The takeaway for your team
Agentic coding has solved generation. What is still unsolved is correctness and security, and that unsolved part used to be exactly the part engineers were paid for. The teams that win the next two years will not be the ones prompting the hardest. They will be the ones whose pipelines catch what the agent left behind, with requirements written before the prompt, automated scanning after it, and a reviewer who knows the four or five weakness classes that show up again and again.
Speed is table stakes now. What survives review, a scanner, and a pen test is the product.