Big Idea

The argument in one line.

Codex Security eliminates false-positive noise by doing what no static scanner does -- it clones your code into a throwaway sandbox and actually fires the exploit before it will ever call something a vulnerability.

Who This Is For

Read if. Skip if.

READ IF YOU ARE…

You ship vibe-coded apps to customers and have never run a formal security audit on your code.
You use Codex or ChatGPT for development and want to know if the new Security plugin is production-worthy.
You manage a small dev team and need to triage which findings actually matter before pushing to production.
You want to understand how AI-powered security tools differ from traditional SAST linters.

SKIP IF…

You need a full penetration test -- this tool only audits your code, not live databases or third-party services.
You are on the ChatGPT Plus plan and depend on it for daily coding; one scan can drain the entire allowance.

TL;DR

The full version, fast.

Codex Security is a plugin for Codex and VS Code that runs in four stages: build a threat model, scan for likely bugs, validate each finding by actually triggering it in an isolated sandbox, then write a patch. The validate stage is what separates it from traditional scanners -- findings that do not fire in the sandbox get dropped, so you see zero false positives. A small dummy repo took 22 minutes and wiped a full ChatGPT Plus plan; realistic use requires Codex Pro for the cloud tier, especially on larger codebases where scans can run for hours.

Free for members

Chat with this breakdown — free.

Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.

Create a free account →

Chapters

Where the time goes.

00:00 – 01:35

01 · Why AI-written code needs AI security review

Models scale faster than traditional security approaches; vibe-coded apps ship full of holes nobody notices.

01:35 – 02:10

02 · What Codex Security actually is

Lives inside your coding tool; finds vulnerabilities, explains them in plain English, offers remediation.

02:10 – 02:45

03 · Benchmark results

Top of CyberGym for real-world vuln finding, beating GPT-5.5 Cyber.

02:45 – 05:03

04 · Live demo: setting up and running a scan

Plugin install, scan type selection, threat scoping toggle, scan kicked off.

05:03 – 06:40

05 · The test scorecard

Tier-1 must-catch bugs, Tier-2 harder bugs, deliberate decoys to test false-positive discipline.

06:40 – 07:34

06 · Under the hood: 4 stages

Threat model -> scan (impact x likelihood) -> validate (sandbox trigger loop) -> patch.

07:34 – 10:37

07 · The validate loop explained

Ephemeral sandbox clones code, triggers each candidate bug, keeps fires, drops misfires. You are the final approver.

10:37 – 11:58

08 · Results and cost reality

15/15 caught, zero false positives, 3 bonus bugs. plan drained by one small scan.

11:58 – 13:19

09 · Which plan you need

Plus = local plugin only, small allowance; Codex Pro = cloud tier for long-running large-repo scans.

Atomic Insights

Lines worth screenshotting.

Codex Security validates every bug by trying to trigger it in an ephemeral sandbox before reporting it -- something no traditional SAST tool does.
A single small-repo scan on the ChatGPT Plus plan consumed the entire monthly usage allowance.
The tool caught 15 out of 15 planted vulnerabilities and found 3 additional bugs that Claude itself introduced when building the test repo.
Risk scoring weighs both likelihood and impact separately -- a high-likelihood but low-impact bug is not automatically high priority.
Codex Security does not audit live systems: it sees only your code, not your Supabase database or third-party integrations.
You can feed its findings back into your CLAUDE.md or system prompts so the model stops repeating the same security mistakes.
The cloud tier (Codex Pro and above) is necessary for large repos where scans can run for multiple hours.
It can create Jira tickets directly from findings, integrating into team workflows without a manual handoff step.
Security findings are explained in plain English by design -- the majority of vibe-coders have no formal security background.
The validate loop is a feedback cycle: pick a candidate bug, try to trigger it, keep it if it fires, drop it if it does not, repeat until exhausted.

Takeaway

How to actually know if your code is exploitable.

WHAT TO LEARN

The gap between 'this looks vulnerable' and 'this is exploitable' is where traditional scanners fail and where sandbox validation changes everything.

Static code analysis produces so many false positives that most developers stop acting on it -- a tool that proves each bug fires before reporting it is categorically different.
Risk is impact multiplied by likelihood: a near-certain low-impact bug and a rare high-impact bug both need separate responses, and conflating them leads to wasted work.
Feeding security findings back into your AI coding prompts breaks the cycle of the model repeating the same class of mistake across every new feature.
Vibe-coded apps consistently ship with SQL injection, hardcoded keys, missing authentication on admin endpoints, and over-permissive row-level security -- these are not edge cases, they are defaults.
Running a security scan on the entire codebase at once is expensive and slow; scanning a folder or a PR diff is the practical workflow for teams on limited plans.
The human is still the final approver -- the tool proposes patches and queues remediation, but you review and confirm before anything changes in your code.

Glossary

Terms worth knowing.

Ephemeral sandbox: A temporary, isolated copy of your codebase that the tool creates to run exploit tests. It is destroyed after validation so your real code is never affected.
Threat model: A structured map of your application's entry points, assets, and trust boundaries used to identify which attack paths are plausible before scanning begins.
RLS (Row Level Security): A database feature common in Supabase/PostgreSQL that restricts which rows a user can read or write. Misconfigured RLS is a frequent vibe-coded app vulnerability.
SAST: Static Application Security Testing -- analysis of source code for vulnerabilities without running the program. Traditional SAST produces many false positives; Codex Security adds dynamic validation on top.
False positive: A security finding that looks like a vulnerability in code analysis but is not actually exploitable in practice. High false-positive rates cause developers to ignore security tools entirely.
CyberGym: A benchmark suite for evaluating AI models on their ability to find real-world exploitable vulnerabilities. Codex Security reportedly leads this benchmark.
CORS: Cross-Origin Resource Sharing -- a browser security mechanism controlling which external domains can call your API. Setting allow-credentials to true without strict origin controls is a common misconfiguration.
Proto-pollution: A JavaScript vulnerability where an attacker modifies the prototype of a base object, potentially affecting all objects in the application and enabling privilege escalation.

Resources

Things they pointed at.

02:10toolCyberGym

01:30productCodex Security plugin ↗

04:05toolJira ↗

Quotables

Lines you could clip.

07:35

“You can bake in any of the things that have found in here, like recurring patterns and things like that, put them in those files so that it doesn't keep making the same mistakes when you're building.”

Actionable meta-insight about feeding security findings back into AI coding prompts→ IG reel cold open↗ Tweet quote

10:37

“This thing caught 15 out of 15 reachable exploitable vulnerabilities with zero false positives, and it even found three bonus bugs that Claude didn't cater for.”

The money-shot result line -- tight and proof-bearing→ TikTok hook↗ Tweet quote

06:35

“Just because this thing found something doesn't mean that it's entirely accurate. So what it's doing here is it's actually testing to prove that each bug is really a bug.”

Explains the core differentiator in plain terms→ newsletter pull-quote↗ Tweet quote

The Script

Word for word.

Read-along

Don't just watch it. Burn it in.

See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.

analogystory

00:00So OpenAI just released the Codec Security Review plugin. It's pretty amazing at what it can actually find inside your repo. In this video, we're gonna explore how it works and what we can do with it using a live repo.

00:10Let's get into it. So I think straight out the bat, it's pretty obvious why we should be using something that can review the code that we're actually building, and that's because nobody actually knows what this code is doing. Biggest problem with that is that the AI models are scaling so fast that they're even outweighing traditional security approaches.

00:24That's clearly a problem as you've seen with Fable getting taken down, but also why they're staggering the release of Mythos to the public and things like that. So the counter to that is to obviously release models that help defend against these kinds of things, and that's partly what OpenAI is doing over here. Anthropic released something a few months ago into research preview for a select few amount of people for security review as well.

00:42Now OpenAI has released something a lot better in plug in form to the public to fix all of these holes that people avoid coding for all of the apps and then just pushing them out there to the customers. So if we look at what Codec Security actually is, you can think of it like a security engineer who really knows your code and helps you develop as you go along.

00:58So it lives inside your coding tool. You can either use this inside Codex itself or you can use it in Versus Code or a terminal, however it is that you work. The goal here is to either understand a GitHub repo or your code base, whatever it is that you're currently working on, and then it'll go and run through this loop where it finds vulnerabilities, explains them to you in plain words, and then ultimately offers you a form of remediation.

01:17The goal here is to use clear plain language because the majority of people using their products have absolutely no idea how any of this works. So it's important that the tool that they built can cater for that to prevent most of the vulnerabilities that are being put out there in the wild right now. In terms of the benchmarks that are currently out there, it is sitting at the top of CyberGym for finding real world vulnerabilities, even beating that of Mythos five, which is pretty crazy because we never even got to see this thing's full power before they pulled Fable down.

01:42But enough talk. Let's see what this thing can actually do. I got Claude to build us a dummy repo to make some really impossible bugs to find and some more easy ones just to see if it finds them all.

01:51Then we're gonna head on over to plug ins. You can see here I've typed in security, and straight away Codec Security pops up, and it's now available inside Codecs for me. In terms of what's inside here, you can see that it follows the path of discovery, and then it gets into analysis, and then finally remediation.

02:05So that's what all of these skills do in one way or another. If you wanted to see exactly what they're gonna be doing, you can just click on one of them, and it tells you the objective in plain English. Turn validated or still plausible findings into explicit attacker stories, structured attack path analysis facts, severity calibration, and final reportability decision grounded in the threat model.

02:24And then you can go through it, and it's literally just like a normal skill in plain English to understand what this thing is gonna be doing. I'm obviously not gonna go through every single one of these. You can do it if you need to in your own time.

02:33What we are gonna be doing is just running this inside our repo. Run the Codec security plug in on this location. And there we go.

02:39It selected the security scan, so it's looking at the repository that we've got in here. Of course, if you're actually building stuff properly, you would be using GitHub, and you would be going through your repo with this kind of thing. You can also then use it to branch out and actually create Jira tickets for you so that you can resolve them as a part of a team.

02:53You can fix issues together, all part of your natural workflow within the systems that you're using. You can see the first thing that we are presented with here is the option to choose our scan type.

03:03So the first can be our code base, which is what we're gonna be doing for this one. You can also do it for any changes as a result of a PR that is trying to be merged and things like that. You then also have a tick box to select a deep scan, though this does take significantly longer, and it also depends on the size of your code base as well.

03:17I've got a pretty small code base for our dummy example, so we're gonna select that, and then we're gonna look at the entire code base over here. I'm not gonna bother with the deep scan. I wanna see what this thing can do just by doing the normal option.

03:27If you wanted to, you could also enable this little toggle over here for threat scoping guidance. So if there are any areas inside your actual code that you want this thing to focus on, this is where you could select these options. But again, for me, I'm gonna leave this one blank, and I'm just gonna start this scan.

03:40You can see that they've really tried to make this user friendly. I mean, it couldn't be easier to do a scan nowadays. Then they've got this little panel on the right hand side that shows you each step of what's actually going on here, and this invokes each of those skills that we looked at at the beginning of this prac.

03:52Okay. So just over two minutes, and this thing is starting to build its markdown file and run its goal over here. You can see pursuing goal, run the Codex security repository tool, and you can see just as we hit the three minute mark.

04:03So this thing is now in the mapping attack surface phase. It's changed a couple of files for its plan. It's built its goal, and you can see here it's pursuing its goal, and it's gonna work relentlessly until it gives us exactly what we need.

04:14And just at about five minutes, it started reviewing the code, so the current stage is listed clearly over here. Be interesting to see how long it takes on such a small code base. Something to note that while that thing's cooking, it does not do live system reviews in the sense that it's not gonna go out into your super base or some of your other apps and order anything going on in there.

04:30This is specifically for your code base or things that you own on your computer or in GitHub or something like that. Make So sure you don't think of this as something like a pen tester or a full audit of all the components of a system that you're building. You can obviously use really good models to help you do that.

04:44We've done that a lot on this channel for the various systems that we've been building when we've been vibe coding apps. But just note that this thing is specifically for code, and it is really good at it. Apparently, we're gonna find out soon.

04:53If we take a look at the things that codecs should find, the must catch tier one obvious things that are built in are these six over here. Hard coded service role key, SQL injection, missing authentication on admin endpoint, weak password hashing, over permissive RLS.

05:08It will definitely find that because even the older versions of Codex were really good with RLS. And then cores with allow credentials equals true. So that's the basic stuff that Claude put in.

05:17Then we have tier two, and this is the stuff that it should catch, but it's a little bit harder. I've also given it some decoys, so it shouldn't be flagging these specific things that we've got over here because they're false positives. And we're trying to test the discipline between whether the things that it finds are actually real or if it just thinks that these are problems.

05:32So we're gonna see how this thing fares in just a little bit. We're now twelve minutes in, and this thing is still chugging away reviewing our code slowly but surely. While it's doing that, why don't we take a look at how this thing works under the hood?

05:41So under the hood, we are running in four stages here. The first one is the one that you just saw where we're building that threat model. The whole point there is to find any avenues that an attacker could use to exploit whatever it is that you're building.

05:52So when we walk away at the end of step one, we have a very clear picture of what that looks like. So that by the time we get into phase two, which is the scan, we can build a very clear picture of what could actually happen with the vulnerabilities that are there now. The important thing that we're doing in step two here is we're prioritizing by impact and likelihood, and that's a very important concept because if you have a very high likelihood that something is going to happen, but the impact of that thing happening is very minimal or almost nothing, it doesn't mean that it's a high level threat.

06:21The same thing if we flip it in reverse. If you have something that has a very low likelihood of occurring, but a high impact, that also influences its risk score. So in step two, our goal here is to focus on the riskiest issues first as a part of this process.

06:35Stage three is all about validation. Just because this thing found something doesn't mean that it's entirely accurate. So what it's doing here is it's actually testing to prove that each bug is really a bug and then leads into phase four where it figures out the patch that it needs to write in order to fix the problem that it's found and tested.

06:51In terms of how it does this, it runs in its own sandbox. So it takes all of your code over here. It clones that into an ephemeral sandbox, meaning that it's not permanent, and it runs its tests inside there to validate all of the bugs that it's found.

07:02Once it's done all of its testing inside this isolated copy, it then presents all of its findings and results, and you get to read through everything and then say whether you want this thing to go and patch it or change something, whatever it is that you want to do with the assessment that it put forward to you. And as you can see, this runs in a loop.

07:16So we have this validate loop where it picks a likely bug that it found after scoring it in that threat model. It tries to trigger it inside its little sandbox, and then if it fired, it will keep that. If it doesn't fire, then it drops it, and it goes round and around in this loop until it's tested everything, and then it can present you that information that is tested and entirely accurate, also so that you know what you actually need to focus on.

07:37And I think one of the great things about this, if you're actually a software developer or care to learn about this, is while it's going through this, you can learn a lot about the way that you're vibe coding your apps at the moment, and you can even use this to improve the thing that you're using to build your code. So if you're building following a certain skill or a certain framework as a part of your Clore.

07:54M d, you can bake in any of the things that have found in here, like recurring patterns and things like that, put them in those files so that it doesn't keep making the same mistakes when you're building. So this can be used as both a reviewer, but then also to improve the things that you're working on. And so here we go.

08:08This thing's done. After twenty two minutes, we scan that tiny little repo. I'm on the ChatGPT Plus plan, so that's the $20 one, and this thing nuked my entire usage credits.

08:17It is a very hungry thing, and depending on the size of your repo, of course, it's gonna use a lot. Because all of the analysis that it's doing in here, it's gonna be very slow, but it's also gonna be quite expensive because of the amount of tests that this thing is running for the validation and that whole loop that we just spoke about.

08:32In terms of its speed, it's meant to be slow. So don't sit there thinking, yeah, I'm gonna kick this off in five minutes and that'll be it. It's gonna take hours.

08:38And in some instances, it could probably even take days. That's why there are multiple ways that we can run this thing. We'll get into that after we look at this.

08:45So we have the ability to export this, which I wanna do because, obviously, I want to get Claude to assess what this thing put out and whether it found every single thing that we put there in the way that it should. So I'm gonna copy the path to the JSON. I'm gonna slap it into Claude.

08:57While Claude is currently assessing that, we can take a look down here. We've got our filters tab where we can filter via severity and several other things over here. If we wanted to, we can just click on view, and it gives us a little bit of a story about what is going on here, what the root cause of it is, the validation that it tested as a part of its loop, and it provides us all of the evidence that it initiated inside of its sandbox.

09:16It then also tells us exactly how an attacker would exploit this type of thing. And then we can click on the patch up at the top here, and it will tell us exactly how to patch this thing with a suggested fix over here. We should then be able to just click on generate patch, and also over here, Codex will queue a remediation attempt and generate a focused patch for the finding.

09:34So let's do that. Let's generate it. Send a follow-up.

09:37It's got the prompt already listed in here exactly about what should be fixed, and then we can just hit send, and it'll go through there, and it will fix it. Whether or not it actually does it because I've almost hit my usage limits, that's another story. But you get the point of how this whole life cycle works, and this is in such plain English that anyone could understand this or at least get to a point where they would understand how this thing works.

09:56Because, again, when you're reading this information, any part that you don't understand, you can just copy it and ask follow-up questions while you're going through this to help you learn, to help you understand how to build your apps better, but also understand general security principles about whatever it is that you're doing.

10:09I do this stuff all the time for everything that I'm doing in AI where there is a concept that I don't understand, not necessarily vibe coding, just anywhere in life really. Our goal here would obviously be to take all of them that are severe and all of them that are critical and need fixing, and then go through them and suggest the patches, and then get the patches implemented as fast as possible.

10:27Something else that you might want to do if you're not just a solo vibe coder and you're working as part of a team, you would, of course, want to create tickets with us and get people to review it before you just suddenly push changes into production, things like that. This certainly isn't a replacement for human intelligence or a human gate before you go and move things into production.

10:45It's more just to speed the whole process up before you get there. Though realistically, think if you give it another year, this thing will probably way better than humans at figuring out bugs, if not already. And there we go.

10:54This thing caught 15 out of 15 reachable exploitable vulnerabilities with zero false positives, and it even found three bonus bugs that Claude didn't cater for. Just built the thing incorrectly by default, and Codex found those vulnerabilities in here, which is amazing because I really asked Claude to build something overly complex and really tricky to see if Codex would actually figure that thing out, and it did.

11:16What would be really interesting to check is when we get access to Fable again to see if it's still able to keep up with whatever it is that we do as a test bed. But in terms of this, I mean, it's it passed the test perfectly. And if you watched any of my previous videos with Codex, you'll also note that the way that it handled live systems as in super based databases, row level security, things like that, it was really good at uncovering things that Claude didn't even think about from a security perspective.

11:38So OpenAI are really focusing on security, is great to see. And as we get to the end of this video, you can see that I'm now out of messages. So this thing wouldn't even be able to fix itself if it wanted to.

11:47It's just kind of running around helplessly right now. One thing that I do wanna check before we jump across is just the plans and the availability of this. As you've seen, this thing absolutely nuked my $20 plan, so I definitely wouldn't recommend that somebody try and use this if you're on the $20 plan for every single thing that you're doing.

12:01Some of the shortcuts that you can take instead of doing the entire code base or a large repo, you can do separate folders or just a diff on an existing repo. That way you would obviously save some cred and it would probably be quite fast. As opposed to constantly doing the code base over and over again, that's just massively inefficient.

12:16Realistically, if you want to use this thing properly for building a Vibe coded app in the same sense that you would probably need a Claude Max subscription to build that or even Codex Pro. You would want the same thing here for the security review. Right now, we were using the plug in locally within the Codex app.

12:30I was using my $20 plan, but it's running on my own machine. I don't have to use a repo. I still could if I wanted to, but it's not necessary.

12:36If you are on the Codex Pro plan, then you have the ability to use the cloud, which makes sense because this is a really long running job, guys. Especially if you have that larger Reaper, this can take hours. So it's probably way better to use their cloud for jobs that take that long as opposed to just leaving your system on.

12:51You would then obviously get more compute. You would have deeper context into whatever it is that you're running, and you could link it to your larger GitHub code base. Something else to note is that, you know, it's just been released, like, literally today, so I would definitely see how this thing develops over time.

13:04It's obviously going to get a lot more efficient at what it does, and it'll be interesting to see how smart this thing is when g p t 5.6 comes up. So I hope this overview was helpful. Leave some comments down below, and I'll get back to you as soon as possible.

13:14Otherwise, check out the videos on the screen now. They'll definitely help you in your journey. Thanks very much for watching.

13:18See you guys later.

The Hook

The bait, then the rug-pull.

OpenAI shipped a security plugin for Codex the day this video dropped -- and the host immediately built a deliberately-broken repo, loaded it with planted vulnerabilities and decoy false positives, and let the tool run. Thirteen minutes later: 15 for 15, zero noise, three bonus bugs nobody asked it to find.

Frameworks

Named ideas worth stealing.

05:41model

Threat Model, Scan, Validate, Patch

Four-stage pipeline: understand attack surface, rank bugs by impact x likelihood, prove each bug fires in sandbox, write a minimal patch.

Steal forAny security review process or checklist for shipped apps

06:30concept

Impact x Likelihood Risk Matrix

High likelihood + low impact does not equal high risk. High impact + low likelihood still influences the risk score. Neither dimension alone determines priority.

Steal forPrioritizing any bug backlog or feature risk assessment

CTA Breakdown