Anthropic's Lydia Hallie walks through what happens to your Claude Code habits once the model, the harness and the environment take over the loop you used to sit in.
Posted
3 days ago
Duration
Format
Tutorial
educational
Views
589.4K
7K likes
57 · 43
Big Idea
The argument in one line.
As models get better, the old best practices of steering and approving don't disappear, they move into the model, the harness and the environment, and your job becomes starting the work and reviewing verified results.
Who This Is For
Read if. Skip if.
READ IF YOU ARE…
You use Claude Code daily and your CLAUDE.md is full of IMPORTANT and NEVER instructions you wrote for older models.
You keep getting prompted for approvals in auto mode and want to know what to configure instead of clicking allow.
You want to run several long Claude Code sessions in parallel and need to know how to keep them safe and correct without watching them.
You lead a team and need a review policy that scales when AI-written pull requests jump from a handful to dozens a day.
SKIP IF…
You are brand new to Claude Code and still need the basics of installing it and writing a first prompt.
You use a different coding agent, since the specific commands and settings here are Claude Code features.
TL;DR
The full version, fast.
Newer Claude models understand intent and code well enough that most steering and per-command approval is no longer your job, so sessions now run for hours and in parallel. What replaces you in the loop is three things. The model steers, so trim CLAUDE.md to what the code can't show. The harness and environment keep it safe, through auto mode, deny rules, sandboxes and cloud sessions. Verification keeps it correct, through tests, hooks that exit with code 2, and CI with autofix. Eventually an orchestrator session starts and checks the threads for you.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Opus 4.5 era sessions needed constant steering. Today Anthropic runs dozens of long sessions and orchestrates sessions from sessions.
01:43 – 02:49
02 · Who takes over when you leave the loop
30,000 changes a day, and the model, harness or environment must replace the human steering, approving, checking and starting.
02:50 – 04:10
03 · Upgrade the model and use auto mode
Switch to Opus 5.5 and auto mode, then prune CLAUDE.md down to what the code can't show, using the prompt audit command.
04:11 – 05:41
04 · Tuning auto mode
Auto mode only trusts the current repo by default. Configure trusted repos, buckets and domains in plain English in settings.
05:42 – 07:43
05 · Deny rules for what can't be undone
Auto mode misses about one in ten dangerous commands, so irreversible actions get deny rules that run before the classifier.
07:44 – 09:59
06 · Sandboxes and cloud sessions
Permissions can't limit side effects of npm test, so a sandbox and then a cloud session limit what scripts can reach.
10:00 – 11:59
07 · Verification points
With no one watching, set pass or fail checks along the way, since the same prompt can yield different auth migrations.
12:00 – 13:53
08 · Hooks and exit code 2
Hooks guarantee lint and test runs. Exit with code 2, not 1, so Claude is sent back to fix failures.
13:54 – 15:25
09 · CI, autofix and review by risk
CI runs the same checks on every PR, autofix and merge when ready reduce review load, and review depth scales with risk.
15:26 – 17:31
10 · Orchestrators and what to try next
One orchestrator session starts and verifies threads. The old practices moved to the model, harness and environment.
Atomic Insights
Lines worth screenshotting.
Anthropic went from about 2,500 changes a day in May to more than 30,000 a day in September, which forces developers to rethink what they focus on.
Every place you step out of the loop, something else has to take over: the model, the harness, or the environment it runs in.
Writing IMPORTANT and NEVER in CLAUDE.md can push a modern model off track, causing more tool calls and worse results.
A CLAUDE.md should only hold what Claude can't see from the code, like a legacy service that isn't referenced anywhere.
In Anthropic's August study, only about 14% of people caught a clearly dangerous command, while auto mode blocked 89% of them.
About one in ten dangerous commands still got through auto mode, so a guarantee can only come from the harness.
A matching deny rule is checked before the auto mode classifier, which never even sees the call.
A good test for what needs a deny rule: if Claude gets it wrong once, can you take it back? Wrong code can be reverted, a leaked key can't.
Permissions decide whether a tool call runs but can't limit what it reaches once running, which is why npm test can still open your .env.
A sandbox hands your deny rules to the operating system, so even child processes and dependencies can't open blocked files or unapproved hosts.
A sandbox only covers shell commands. Claude's own files, MCP servers and hooks still run on your machine.
A cloud session starts with a fresh copy of the repo, so an untracked .env and your SSH keys simply aren't there.
When a hook script exits with 1 the task is marked completed anyway. Exit with 2 to send Claude back to fix it.
Run the same prompt three times and you can get three different auth migrations, so verification points matter more than the prompt.
Reviewing short verification points is much cheaper than reviewing a whole change at the end.
Scale review to risk: docs and copy need no person, ordinary product changes one reviewer, risky changes two.
Takeaway
Trade steering for guardrails and checks
WHAT TO LEARN
When you stop steering every step, the model handles intent, the harness and environment enforce safety, and verification plus CI prove the work is right.
03Upgrade the model and use auto mode
Move to the newest model and auto mode, then strip CLAUDE.md of shouting rules and stack descriptions the model can already read from your code.
Configure auto mode in plain English with the repos, buckets and domains you trust, so it stops asking about routine pushes.
05Deny rules for what can't be undone
Use the question 'if this goes wrong once, can I take it back?' to decide which actions get a hard deny rule.
Remember deny rules run before the classifier, so they are the only real guarantee that Claude never touches a file like .env.
06Sandboxes and cloud sessions
Treat a sandbox as the answer to side effects, since permissions can't see what npm test or a compromised dependency opens.
Run long sessions in the cloud so secrets and SSH keys aren't on the machine, and add API keys as network secrets instead.
07Verification points
Define done as pass or fail checks the session must show output for, so you review short verification points instead of the whole diff.
08Hooks and exit code 2
Use hooks for checks that must always run, and make failing scripts exit with 2 so Claude goes back and fixes the problem.
09CI, autofix and review by risk
Let CI with autofix catch what checks can catch, and match human review depth to risk instead of reading everything.
10Orchestrators and what to try next
Start with something far bigger than you'd normally hand over, then find where sessions get stuck, since that is what to automate next.
Glossary
Terms worth knowing.
Auto mode
A Claude Code permission mode that runs a separate classifier on anything beyond reading or editing project files, allowing safe commands and blocking uncertain ones. It became the default in August.
Deny rule
A setting that blocks a specific tool call outright. It is checked before the auto mode classifier, so it is the way to guarantee Claude never does something.
Sandbox
An operating system level boundary around shell commands that limits which files they can read, which folders they can write and which hosts they can reach, including any child processes.
Cloud session
A Claude Code session that runs on its own remote machine with a fresh copy of the repo, so it keeps working when your laptop is closed and your local secrets aren't present.
Hook
A script Claude Code runs at a specific point, such as after a file edit or before a task is marked complete, used to run linters or tests that must always happen.
Orchestrator session
A main session that splits a job into tasks, starts other sessions for each one, and sends back any result that isn't actually done.
Claude Code Projects
A feature, still in beta, where one orchestrating session starts and verifies threads and manages context for you.
Resources
Things they pointed at.
03:49toolDoctor prompt audit command
04:35toolAuto mode settings (plain English rules)
15:30productClaude Code Projects (beta)
17:25linkClaude Code best practices docs
Quotables
Lines you could clip.
02:10
“Everywhere that we now step out of that loop, something else has to take over. And this is either the model, the harness, or the environment that it's running in.”
One sentence that frames the whole talk→ TikTok hook↗ Tweet quote
05:42
“Only about 14% of people caught a clearly dangerous command. Auto mode blocked 89% of them.”
Hard numbers that flip the 'humans are safer' assumption→ IG reel cold open↗ Tweet quote
06:30
“If Claude gets this wrong once, can I take it back?”
A portable decision test for any permission setup→ newsletter pull-quote↗ Tweet quote
12:56
“Make sure that your scripts exit with a two exit code, because that's what actually sends Claude back to fix it.”
A specific gotcha people hit and don't know→ TikTok hook↗ Tweet quote
16:50
“Almost everything we used to do ourselves is now either the model, the harness, or the environment.”
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
17px
I had some technical difficulties during my talk this morning, so I ran back home and I'm just going to record it from my computer instead. So hi everyone, I'm Lydia. I work on the Clock Code team at Anthropic.
And the models have gotten really good lately. So we have Opus 5 .5, which is my favorite daily driver. And earlier this week, we also launched Fable 5 .2.
And these models are extremely good at long -running tasks and understanding your code base. And especially Fable is also really good at verifying its own output. But as these models get better, the way that we work with them in Cloud Code also changes.
So earlier this year, I don't know if you remember, but sessions kind of used to look like this. So we'd use a model like Opus 4 -5, which honestly for its time was really good. I really enjoyed working with Opus 4 -5, but compared to today, it would just stop a lot earlier.
So it would just run for shorter amounts of time. And we'd also keep correcting it. We'd keep reminding it to do things, add things like important, exclamation mark, all these things to our CloudMD file.
And also features like auto mode didn't exist yet. So we were either approving everything manually or taking the riskier approach with dangerously skip permissions. So we were still very much in the loop as humans.
But as these models got better at taking on bigger tasks, our sessions also changed. So nowadays, at least at Anthropic, our sessions look more like this. So we're running dozens of sessions at the same time, running for hours, verifying their own results along the way, and we even have sessions orchestrate other sessions.
So whereas we used to be in the loop for almost every step and running one or just a couple sessions at a time, we've now essentially become managers for dozens of sessions. We also get a lot more done though. Now you may have seen our blog post in October about how Anthropic went from about 2 ,500 changes a day in May to more than 30 ,000 a day in September.
But this also means that we as developers just have to rethink on what we're focused on instead. So if you think about our best practices from earlier this year, it was mainly about steering sessions, approving commands, saying no to certain things or restricting what cloud could do. checking the output, and also starting the sessions.
So everywhere that we now step out of that loop, something else has to take over. And this is either the model, the harness, or the environment that it's running in. So if we are not in the loop to approve every command and we're running so many sessions at the same time, how can we make sure that the code it's writing is still high quality?
And most importantly, if these sessions are running for hours without our supervision, how can we make sure that they're running safely? So let's go through that by taking this old session and seeing what we have to change to get to that end result that I showed earlier. Well, first of all, let's use the actual defaults we start sessions in nowadays.
That's not really something you should think about. So let's update the model here from Opus45 to Opus55 and change the manual permission to auto mode. And Opus55 is a lot better at understanding your intent and understanding your code base.
So we really don't have to steer it as much anymore mid session. And you also no longer have to approve every command because we started making auto mode the default permission mode back in August. Now being better at understanding your intent and your code base also means that we have to rethink the instructions we have in our code base.
If you followed our best practices from earlier this year, you may still have things like important or never do this, exclamation mark, all these things in your CloudMD file. But with today's models, you really don't need that. And it can actually make the model go off track.
So you end up with more tool calls and overall worse results. So nowadays your CloudMD file should only include the things that Cloud cannot see from your code. So for example, maybe you have a service that's still using some legacy source somewhere, but it's not directly referenced in the code.
That's still good. to keep in there. But everything else, like what stack you're on or where to look, you really don't need that anymore.
And we actually have a command for this called doctor prompt audit, which makes it really easy to optimize your environment for a specific model. So if you're on Opus 5 .5 here and you run this command, it will automatically check your CloudMD and your skills to see what the new model no longer needs. Now, auto mode is a permission mode that runs a separate classifier on pretty much everything that isn't just reading or editing files in your project.
So it automatically allows commands that it knows are safe, like running tests or installing dependencies, but it still blocks commands where it's unsure whether they're safe. And normally that's pretty good. Normally that's kind of, you know, what we want.
But for example here, if... we ask it to ship something to staging we would have just you know approved the push to our deploy repo but by default auto mode only trusts this folder and this repo so it would block that command so we're back in the loop now the defaults should be good in most cases but if you still notice that you have to approve a lot of commands even in auto mode The first thing to look at is settings here, because you can configure auto mode in your own settings, just in plain English.
And this is what gets sent to the classifier, which then decides whether to reject or allow certain commands. So here we can just write which repos, buckets, domains we trust, and what it should always allow or never do. Honestly, this was pretty easy by just using today's defaults.
We've already gotten rid of most of our in the loop stuff because the model took over the steering and auto mode took over all the approving. So getting out of the loop is easy. But the harder question now is how do we make sure it's okay that we're not there anymore?
So first is keeping it safe and then also verification. So let's start with keeping it safe because everything so far, so upgrading the model and using auto mode, it's just working with a model. And in most cases it does what you tell it to do but that's not a guarantee.
So in a study we published back in August we tested auto mode versus real people in the loop and only about 14 % of people caught a clearly dangerous command. Auto mode blocked 89 % of them so it is a lot better at it than we were. But about one in 10 of those commands still went through.
So if you do want the guarantee, you can only enforce that in the harness. And this is where some of our older best practices still matters and that's permission. So for example, if you have a deny rule for a specific tool call in your project, Cloud Code looks at this rule before the automo classifier even looks at it.
In fact, if a tool call matches a deny rule, the classifier never even sees it. So having deny rules is still the best way to make sure Cloud actually never does something.
But then how do you know what needs to be a rule in your project versus what the model can just know because they've gotten so good at it? I think it kind of just comes down to if clot gets his wrong once, can I take it back? So if it writes the wrong code, you can still revert that.
So you can still leave that to the model. But if it throws away or uncommitted changes, or it reads a key that's now in your transcript and you don't want that, that should still be a rule. So for example, let's say that we just never want Cloud reading our .env file.
Now by default, Automode is fine with this. So it would have just approved it. So if you want to make sure that Cloud can never read this file, you should just add a deny rule.
And then commands like cat cannot open this file either. but it's not always that straightforward because sometimes certain commands can have side effects so for example let's say that npm test here a very normal command it's just a test so automotive runs it that's fine but when the script is running it runs code that you didn't write like your dependencies or maybe claude runs a script real quick that it wrote just to quickly check something so if one of those scripts is compromised it could just open that same end file or your ssh keys and send those to a server you've never heard of right and even if we were still there as humans approving everything honestly we would have just clicked allow because there's nothing to say no to it's just part of the script that in this example case npm test runs and Permissions only decide if a tool call runs, but they cannot limit what it can reach once it's running.
So we want to make sure that these kinds of scripts can still run because this is just part of how Cloud can check its own work running tests. So we cannot just block NPM tests, but we want to limit its side effects. So really, we just want to limit the environment that it's running in.
So if you're running it locally or on your own machine, you can run commands like that in a sandbox. So then you have control over which files in a command it can read, which folders it can write to, which hosts it can reach, and the operating system enforces that. Also, this also applies to everything that command then starts, so any child processes.
So on its own, that deny rule got checked by Claude before the command runs. But all it sees there is npm test, so it doesn't see which files that scripts ends up opening. But with the sandbox on, that same rule gets passed on to the operating system.
So now when that dependency tries to open or end file, the operating system just doesn't let it. And by default, it can still read most of your machine. So you'd want to add another rule for your SSH keys if you care about that.
And it also cannot reach a host that wasn't approved. But the sandbox only covers the shell commands that Cloud runs and whatever those start. But Cloud's own files, your MCP servers, and all of your hooks still run outside of it.
So the rest of your session still runs on your own machine and everything that's on there. So ideally, you want this whole thing to run somewhere where none of that is even there in the first place. And that is a cloud session.
Because there it gets its own machine with a fresh copy of a repo. And this runs even when we close our laptop, which is definitely something you want. for these longer running sessions that run for hours.
So in this case, this automatically gets rid of the end file issue because it's not in Git and neither are my SSH keys or our GitHub token. And if it still needs an API key, you can add that as a network secret or it will tell you like, hey, I'm missing an API key mid -session. So it can use it without actually ever sending it to the model.
And by default, also in a cloud session, network access is limited to things like package registries and GitHub. So you have to explicitly add those domains to your environment for cloud to reach other hosts. Okay, so with a cloud session, we can run these long -running sessions even with our laptop closed, and it keeps us safe because it runs in its own environment.
now it's still good to remember that this machine is disposable but the things that leave it aren't so a commit is pushed or a message is sent through a connector so we still run it in auto mode with our deny rules just commit it into the repo so they come along so it could still catch things like a force push or a secret and a commit all that kind of stuff and beyond just safety um we also need to make sure that cloud is actually working on the right thing so next is verification and the nice thing about being in a loop uh like we kind of knew the work was good because we'd already kind of seen every change we'd proved everything but now if cloud works for hours in the background we don't have that anymore and we're not really going to read every file at the end either so you know, say that we ask it to move our auth to a new API.
Well, Cloud is very good at understanding what we want, but there are still a lot of small decisions that it has to make along the way. So with the exact same prompt, you know, one run might move it over but leave the old API in there, or one might move it but only for new logins, and another one might move it but log everyone out.
And these all look pretty reasonable, but it really helps to set up these verification points along the way. So Cloud keeps verifying its own results. And those can be anything that comes back as a pass or fail.
So tests, a build, a screenshot, whatever you have. So now we just want to make sure that we review these verification points. They're a lot shorter to review than if we just had to, you know, review the whole change at the end.
So that's pretty reasonable. So if one of these runs would still log everyone out, it doesn't get past that first check. And then it just has to find another way.
And there's a couple ways to set this up. The easiest one is, of course, just in your prompt. So you say what done looks like and ask it to split that into tasks so they have their own tests.
And I also like to ask it to show the actual output because you weren't there to see it. But again, prompting is not a guarantee that a model will do it. So for the ones that have to run.
guaranteed run, I would use hooks, which are just, you know, scripts that cloud code runs at a specific point. And those give you a lot of control because you decide when they run and what they run and what they check. So for example, every time it edits a file, we can run our linter.
Every time it wants to mark a task as complete, we can run the tests for what it changed. So, you know, if it touched our auth code or something, it has to pass the auth tests. And when it wants to finish the whole session, we can just run the entire test suite.
And if one of those fails, it goes, oh, that's still broken. It goes back until it passes. So it fixes it.
Now, just one thing. When a test fails, by default, most tools that exit with a one exit code. But with a one, that task by Cloud Code just gets marked as completed anyway.
So make sure that your scripts exit with a two exit code because that's what actually sends Cloud back to fix it. Okay, now we have one session that I'm pretty comfortable with leaving alone. So now we can start with a few more of those because they've all got their own machine.
So they're, you know, not all working in the same folder and we don't really have to think about work trees too much ourselves anymore. But then we still start every session ourselves and we review every one of them. And honestly, this is already a pretty good place to be.
We are still fully in control over what gets started and when we call a task done. But even there, you know, we're still the ones kicking off every session and we're still in the loop for all of them. So I think the most important thing to have in place here now is your CI, because all of these sessions eventually open a PR and your CI runs the same checks on every one of them, no matter how that session was set up or, you know, what Cloud decided to do along the way.
And if you turn on autofix, we have like CI autofix. Cloud can also see when one of those checks fails and then go back to fix it. So you're not the one spending your review on things that, you know, a check could have already caught.
And for some of them, you can even turn on merge when ready. So then GitHub merges the PR once all of the checks have passed. So you're not in a loop to review.
But again, that's not the default. And that really depends on how risky the change is. So for example, on our team, every PR gets reviewed by cloud first.
So not by the session that wrote it. So it comes with a fresh context and also a separate group of agents to just double check every bug it finds. and then how much we review ourselves just depends on the risk so smaller changes like docs or copy they don't need a person so once the automated checks are green we're pretty good to go maybe ordinary product changes just need one reviewer and then risky changes might need two now ultimately kind of where we see things going, and we're already kind of doing more of that at Anthropic, is that you're not really starting all of those sessions yourself either.
Because if you're anything like me, I'm terrible at context switching, so having to manage so many sessions at the same time doesn't always go well. So instead, you just have one main session that's your orchestrator, and you can already see this in things like Cloud Code Projects, which is still in beta, but you just tell it what it needs to be done.
And the orchestrator splits that up and then starts other sessions for each task, and it keeps track of what comes back. So it automatically starts and verifies these threads, which are just new sessions. So instead of talking to eight sessions, you're just talking to one.
And the main session doesn't write the code itself. So there you want a model that's really good at deciding what to do and judging what comes back, a model like Fable. And if something comes back that isn't actually done, it just sends it back until it's actually done and passes everything.
And the threads need a model that's good at actually doing the work, like Opus 5 .5. And you can set those separately. So one model for the main session and then another one for the threads.
And another benefit is that with this setup, with Cloud Code Projects specifically, we don't manage context anymore. That's all done automatically for us. So we no longer have to think about, you know, when to compact or clear all of that stuff.
Okay, so if you look back at those old best practices, they're all still there, but they've just moved. So the model sears it now. Auto mode approves your deny rules and the environment restrict what's possible.
Your test, your hooks, and your CI checks it. And then ultimately, cloud can just start them as well. So almost everything we used to do ourselves is now either the model, the harness, or the environment.
And we're really just here to start that first one and to just review what comes out. So if you want to try this in your own setup, I would say just take one thing that's way bigger than you'd normally give Cloud, make sure your CI actually catches the things you care about, and then spend a bit of time just configuring your environment, like its network access, its secret, and its setup script.
And then start in the cloud and see where your sessions get stuck and where you're still in the loop, because that's the next thing to hand over to cloud. I also highly recommend checking out our best practices in the docs as well. Thank you so much.
The Hook
The bait, then the rug-pull.
The models got good enough that most of what we added to our Claude Code setups last year is now holding them back. Lydia Hallie shows which habits to drop, which to keep, and where each one moved.
Frameworks
Named ideas worth stealing.
02:10model
Who replaces you in the loop
The model: steers and understands intent
The harness: permissions, deny rules, hooks, auto mode
The environment: sandbox and cloud session limits
CI: checks every pull request
Each job a human used to do in the loop (steering, approving, checking output, starting sessions) is now handled by the model, the harness or the environment.
06:30concept
Can I take it back?
The test for whether something needs a deny rule. If Claude getting it wrong once is reversible, like bad code, leave it to the model. If not, like discarded uncommitted changes or a leaked key, make it a rule.
Steal fordeciding what to lock down in any agent setup
14:53list
Review depth by risk
Docs and copy: no person once checks are green
Ordinary product changes: one reviewer
Risky changes: two reviewers
How much humans review depends on the risk of the change, after Claude reviews every PR with a fresh context first.
CTA Breakdown
How they asked for the click.
VERBAL ASK
17:18link
“I also highly recommend checking out our best practices in the docs as well.”
A soft closing pointer to the docs after a concrete three-step homework suggestion.
Add Modern Creator as a preferred source and Google shows you more of our breakdowns in Search, Top Stories, and AI Overviews. It only changes what you see, and you can undo it in your Google settings anytime.
Add to Preferred SourcesOpens your Google source preferences with us pre-loaded. Tick the box and you're done.
A SpaceXSI engineer explains how her team ships 11,000 agent-written pull requests a month with a 0.5% revert rate, by hardening the codebase and verification instead of reviewing every diff.
A systems playbook for Claude Code and Codex: the five-step loop, the instruction files, and the guardrails that separate trustworthy AI output from expensive rework.
A 17-minute screen-recorded walkthrough of installing, configuring, and running the mattpocock/skills repo on a real codebase — from a vague idea to a reviewed, committed change.
00:19
00:32
00:45
00:59
01:12
01:38
01:51
02:17
02:31
02:44
02:57
03:23
03:36
03:49
04:03
04:16
04:42
04:55
05:08
05:21
05:35
06:01
06:14
06:27
06:40
06:53
07:06
07:20
07:33
07:46
07:59
08:12
08:25
08:52
09:05
09:18
09:31
09:44
09:57
10:10
10:24
10:37
11:03
11:16
11:29
11:42
11:55
12:09
12:48
13:01
13:14
13:27
13:41
13:54
14:07
14:33
14:46
14:59
15:13
15:26
15:39
16:05
16:18
16:31
16:45
17:11
17:24
17:31
1:36:30
16:02
52:32
29:19
17:17