AI-Generated Code Security in 2026: The Numbers and a 12-Point Check

Akshit Ahuja
Co-Founder & Lead Engineer

The security pass rate of AI-generated code was 56% in [Veracode's 2026 GenAI Code Security Report](https://www.veracode.com/blog/2026-genai-code-security-report-ai-risk), against 55% in the 2025 edition. Roughly 44% of code generation tasks in their tests introduced a vulnerability. Models got better at almost everything in the same year; they did not get better at this.
This post is for a founder or a small team with an app that an AI tool wrote most of. It covers what the 2026 studies actually measured, the gap between lab tests and live apps, and twelve checks that cover the flaw categories the studies keep finding. It does not cover prompt injection into AI features inside your app; that is a separate problem with its own checklist.
What the studies measured
Three sources this year, with three different methods. They agree on the categories and differ on the numbers, which is what you would expect.
| Study | What was tested | Sample | Headline result |
|---|---|---|---|
| Veracode, 2026 GenAI Code Security Report | Code generated by many LLMs on security-relevant tasks, in a lab | Tasks across models (report describes the set) | ~44% of tasks introduced a vulnerability; 56% average pass rate (55% in 2025) |
| AppSec Santa, February 2026 | Six LLMs against OWASP Top 10 scenarios | 522 code samples | GPT-5.2 lowest at 19.5% vulnerable; Claude Opus 4.6, DeepSeek V3 and Llama 4 Maverick tied highest at 29.9% |
| Symbiotic Security, June 2026 | Live apps built with Lovable, v0, Bolt, Replit, Windsurf and others on Supabase | 1,072 sites scanned | 98% had at least one flaw, 16% a critical one, 5.9 findings per site on average |
| SupaExplorer (via Hacker News), early 2026 | Live launch URLs from five indie product directories | 20,052 URLs | 2,217 domains (11.04%) exposing Supabase credentials or unprotected tables |
Two things to read out of the table. The lab number (56% pass) and the live number (98% flawed) are measuring different things: one asks "did this generated function have a bug", the other asks "does this deployed app have at least one bug anywhere". Both are bad, and the live one is the one your users meet.
The second: the best model in AppSec Santa's test still produced insecure code one time in five. Choosing a better model is worth doing and is not a substitute for review.
A security pass rate, in Veracode's report, is the share of generated code samples that contained no vulnerability of the type the task was designed to elicit. A 56% pass rate means the model chose the insecure option in 44% of the tasks where an insecure option existed.
Why the numbers do not move
Veracode's headline finding is that functional output and secure output are separate things: models generate syntactically correct code close to 100% of the time, and secure code 56% of the time. Most security flaws are code that runs. A Supabase query with no row filter returns the right rows in the developer's test because the developer is the only user. A DELETE endpoint with no auth check deletes the record in the demo. Nothing in the feedback loop the model was trained on says "wrong".
The categories that fall out of this are stable across all three studies:
- Missing authorisation on data. The query trusts the caller. In Supabase, RLS is off or the policy is
using (true). - Secrets in the client. The service key, or a third-party API key, in the JavaScript bundle because the model needed it to make the feature work.
- Unvalidated input. File uploads with no type or size check, IDs taken from the URL and used directly, string concatenation into queries.
- Dependencies added without review. Evil Martians' write-up covers this one: small utility packages with few downloads, added because the model suggested them, occasionally malicious.
We have covered specific cases of the first category in Stripe webhooks and Supabase RLS tenant leaks and of what happens to new tables after October 30 in Supabase stops auto-exposing new tables.
The 12-point check
Most of these take a minute each. Numbers 1 through 4 are the ones that turn into incidents.
Authorisation
- Every table: RLS on, and at least one policy. In Supabase's SQL editor, list
pg_class.relrowsecurityfor every table inpublicand count policies per table. Any exposed table with RLS off is public. - No `using (true)` policies on user data. Grep your migrations for
using (true)andwith check (true). Each one is a table where any authenticated user can read or write every row. - Test as the anon role. Supabase's RLS Tester (dashboard feature preview since April 2026) runs a
SELECTas any role. Do it for every table. What comes back is what a visitor with your public key can fetch. - Every mutating endpoint checks the caller. For each route handler or server action that writes, find the line that reads the session and the line that compares the record's owner to it. If either is missing, the endpoint is open.
Secrets
- Grep the bundle. Build the app, then search the output for
service_role,sb_secret_,sk_live_,sk-and any key prefix your providers use. A hit means rotate first, fix second. - Grep the repo history.
git log -p | grep -E "sk_live_|service_role"finds keys that were committed and later removed. They are still in the history, and the repo may be public. - Server-only calls stay server-only. Any call to OpenAI, Stripe or Resend from a component file, rather than a route handler or server action, is sending the key to the browser.
Input
- Uploads have a type and size limit, enforced on the server, not only in the file picker.
- IDs from the URL are checked against the caller.
/api/invoices/[id]must verify the invoice belongs to the session's user before returning it. The studies call this broken object level authorisation; OWASP's API Security Top 10 has listed it as the number one API risk since 2019. - No string-built SQL. Grep for template literals that contain
select,insert,updateordelete. Prisma, Drizzle and supabase-js parameterise for you; raw query helpers do not.
Dependencies
- Read the lockfile diff. For every package the AI added, check weekly downloads and the publish date on npm. Evil Martians suggest a minimum release age before installing; a package published last week with 40 downloads does not go into production.
- Run `npm audit` and act on high and critical. Then set it to run in CI so the number stays at zero.
What we did not test
We did not run our own generation study; the pass rates above are Veracode's and AppSec Santa's, and the live-app numbers are Symbiotic Security's and SupaExplorer's, with their sample definitions on the linked pages. The twelve checks reflect the categories those studies report, not a measured detection rate; we have not counted how many of the flaws in a scanned app the manual pass would catch versus a scanner.
If the app handles payments or personal data and nobody on the team has done the twelve checks, the vibe-coded app audit starts with exactly this list and ends with the findings fixed rather than reported.
Frequently asked questions
- Is code from a newer model like GPT-5 safer than from older ones?
- Somewhat, on a per-sample basis, but not enough to skip review. AppSec Santa's February 2026 test of 522 samples found GPT-5.2 had the lowest vulnerability rate at 19.5%, and three other frontier models tied at 29.9%. One in five insecure samples from the best model is still one in five.
- What is the single most common security flaw in vibe-coded apps?
- Broken authorisation on the data layer. In Supabase apps it shows up as tables with Row Level Security disabled or a policy that allows everything. Symbiotic's scan found unauthenticated data deletion possible on 172 of 1,072 sites. The public anon key then reads or deletes whatever the policy allows.
- Do I need a security tool, or can I check this myself?
- The twelve checks below need a terminal, the Supabase dashboard and an hour. A scanner (Semgrep, Snyk, or the Supabase RLS Tester) catches more, but the categories in the studies are simple enough that a manual pass finds the expensive ones. Automate it once it has found something.
- Why does the pass rate stay flat while models improve?
- Veracode's headline finding is that functional output and secure output are separate: models now produce syntactically correct code close to 100% of the time while the security pass rate sits at 56%. A query that ignores the user's role still returns the right rows in a test, so nothing in the training signal marks it wrong.
Sources
- Veracode: 2026 GenAI Code Security Report
- Symbiotic Security: We scanned 1,072 vibe-coded apps, 98% had security flaws (June 2026)
- Hacker News: 11% of vibe-coded apps are leaking Supabase keys (SupaExplorer)
- AppSec Santa: AI code security study, 6 LLMs vs OWASP Top 10 (February 2026)
- Evil Martians: The 4 most common security risks when vibe coding your app
#AI code security #Veracode #vibe coding #OWASP #Supabase #secrets #dependencies #security checklist

Akshit Ahuja
Co-Founder & Lead Engineer
Backend systems specialist who thrives on building reliable, scalable infrastructure. Akshit handles everything from API design to third-party integrations, ensuring every product HeyDev ships is production-ready.


