19:12Theo - t3․gg · Reaction
Claude Code's creator has some really good advice
Theo reacts line-by-line to Boris Cherny's post arguing that automation — CLAUDE.md rules, lint checks, CI — matters more than ever in the agent era, not less.
July 21stThe argument in one line.
AI agents have functionally solved coding, the act of turning a well-specified plan into working code, but software engineering, the surrounding work of planning, verifying, and maintaining that code, remains a mostly human job.
Read if. Skip if.
READ IF YOU ARE…
- You use AI coding agents daily and want a clear framework for why they still ship dumb, easily-avoidable bugs.
- You lead a team deciding how much scrutiny an agent-generated pull request needs before it gets merged.
- You followed the Boris Cherny / Matt Pocock "coding is solved" argument on X and want the actual disagreement explained.
- You're trying to figure out what to build into your codebase so agents can verify their own changes instead of just producing plausible-looking code.
SKIP IF…
- You're looking for a hands-on tutorial on prompting or configuring Claude Code, Codex, or any specific agent.
- You don't use AI coding tools and have no stake in how the term "coding" gets defined.
The full version, fast.
Boris Cherny's tweet "Coding is solved, bugs are not yet solved" set off a fight with Matt Pocock that looks like a disagreement but is really a definitions problem. Narrowly, coding is typing in an editor; AI has solved that. Widely, coding is the whole idea-to-shipped-software loop; that's obviously not solved. Theo diagrams why: software quality depends on plan quality first, then on whether a verification loop (QA, staging, testing) catches what the plan and the code missed. Agents are excellent at tactical, well-specified work but show no evidence of strategic thinking, architecture, or checking their own results, which is why bugs like a truncated UI string ship even though the code looks completely fine on review.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 01:38
01 · Cold open: the tweet that started it
Theo plays Boris Cherny's tweet "Coding is solved, bugs are not yet solved" and Matt Pocock's viral pushback calling it "a raspberry ripple of bullshit," then states his contrarian take: both of them are right.
01:38 – 02:49
02 · Sponsor break: Blacksmith
Blacksmith CI runner ad: faster GitHub Actions hardware, faster cache downloads, and "sticky disks" that persist node_modules and git history across CI runs.
02:49 – 05:51
03 · The bug: Claude Code desktop's truncated update text
Peter's post shows a Claude Code desktop update notice with text cut off by the container it renders in, because an agent wrote the UI copy without ever checking it in the browser. Boris initially calls it a UX issue rather than a bug; Theo pushes back on that framing.
05:51 – 07:37
04 · Defining "coding" and "solved"
Theo argues the whole fight collapses once you fix the definition: narrowly, coding is typing in an editor, and AI has solved that; end-to-end, coding is idea-to-shipped-software, and that obviously isn't solved.
07:37 – 10:07
05 · The Password Game: bad code is still code
Theo uses Neal.fun's intentionally-terrible Password Game to separate two failure modes: code that's bad on purpose as satire, and code that's bad because the plan gave no or bad detail, not because the coder failed.
10:07 – 14:24
06 · The plan-quality framework
Theo draws a diagram: good, bad, and empty plans feed into engineers, producing software scored roughly 8-10, 1-4, and 1-10 respectively, the point being an empty plan makes the outcome close to random regardless of engineer skill.
14:24 – 18:51
07 · Swap in agent swarms: the verification gap
Theo replaces "engineers" in the diagram with agent swarms and adds a verification (QA) layer with a loop back to planning. He argues Claude Code desktop's bug happened because testing a full Electron app costs a dedicated VM per thread, versus a browser tab for the web app.
18:51 – 26:49
08 · Coding vs. software engineering
Theo argues coding is the narrow act of turning a plan into code, functionally solved by AI, while software engineering is the far wider discipline of architecture, verification, and checking results, which agents can't yet do without a human-built system around them.
26:49 – 33:08
09 · Matt Pocock's reply: tactical vs. strategic programming
Matt cites Stanford's John Ousterhout: tactical programming is day-to-day code, strategic programming is long-term architecture and codebase health. Matt says AI has solved tactical work but shows no evidence of thinking strategically; Theo largely agrees but sees agents as useful strategy consultants.
33:08 – 33:48
10 · Sign-off
Theo notes the video ran longer than the planned fifteen minutes, mentions he filmed it offline rather than on stream, and asks viewers where they land on coding vs. engineering.
Lines worth screenshotting.
- Coding, narrowly defined as typing code in an editor, is functionally solved by current AI agents; software engineering, the full idea-to-shipped-software loop, is not.
- A good plan run through good engineers or agents lands around 8-10 out of 10; a bad plan caps outcomes around 1-4 out of 10 regardless of execution skill.
- An empty or vague plan produces the widest possible range of software quality, from great to terrible, because the outcome becomes close to random.
- If a codebase is too hard for a human to spin up and test locally, it is effectively impossible for an agent to verify either.
- Claude Code desktop lags behind Claude Code web specifically because testing a full Electron app requires a dedicated VM per test thread, while testing a web app only needs a browser tab.
- The era of catching bugs by just reading the code has ended for the kinds of bugs agents can already spot themselves; the bugs that remain are the ones invisible until you actually run the software.
- Bad code from too little detail in a plan and bad code from bad details in a plan are different failures, both still count as coding, and both point back to the plan, not the coder.
- John Ousterhout's tactical vs. strategic programming split maps directly onto the coding vs. engineering split: AI has largely solved tactical, day-to-day coding but shows no evidence of strategic, architectural thinking.
- Agents are useful as strategy consultants who can evaluate an architecture or test a theory, even though they won't originate the strategy themselves.
- Boris Cherny says he personally stopped writing code by hand in November of the prior year and considers himself an average programmer already surpassed by Claude.
Coding is solved. Software engineering is a much bigger box.
WHAT TO LEARN
The "coding is solved" fight is really an argument about definitions: agents have functionally solved turning a well-specified plan into code, but the planning, verification, and architecture around that code are still mostly a human job.
01Cold open: the tweet that started it
- When two smart people seem to disagree about AI and coding, check whether they're using the word "coding" to mean different things before assuming one of them is wrong.
- A viral hot take often comes from real experience but gets misread because it's compressed into a sentence with no definitions attached.
03The bug: Claude Code desktop's truncated update text
- If nobody renders a feature before shipping it, an edge case like text overflow reaches production even though the code "looks fine" on review.
- Trusting an agent's pull request without checking the actual rendered output is the same mistake as trusting a coworker's PR without ever running it.
- Calling a visible bug a "UX issue" instead of a bug doesn't make it less broken for the user experiencing it.
04Defining "coding" and "solved"
- Narrow definition: coding is typing in an editor. Wide definition: coding is everything from idea to shipped software. Most arguments about "AI solved coding" collapse once you pick one.
- By the narrow definition, turning a well-specified plan into working code is functionally solved by current AI agents.
05The Password Game: bad code is still code
- Code that turns out badly because the plan gave no detail is a different failure than code that turns out badly because the plan gave bad detail, both are still "coding," just failing at different stages.
- Intentionally bad software and accidentally bad software both still count as the product of coding; quality and intent are separate questions from whether coding happened.
06The plan-quality framework
- A good plan run through good engineers or agents tends to land 8-10 out of 10; a bad plan caps out around 1-4 out of 10 no matter how good the execution is.
- An empty or vague plan produces the widest possible range of outcomes, from great to terrible, because success becomes close to random.
- Fixing "the AI got it wrong" usually starts with fixing the plan, not blaming the model.
07Swap in agent swarms: the verification gap
- A verification/QA layer with a feedback loop back to planning is what turns a "maybe it's fine" pull request into software you can actually trust.
- If a system is too hard for a human to spin up and test locally, it's effectively impossible for an agent to verify either, the fix is making the codebase easier to test, not smarter agents.
- Claude Code desktop lags Claude Code web specifically because testing a full Electron desktop app requires a real VM per test thread, while testing a web app just needs a browser tab, the verification cost is the bottleneck, not the model.
08Coding vs. software engineering
- Software engineering is the platform coding sits on: architecture, planning, and the systems that let the coding-and-verification loop repeat effectively hundreds of times a day.
- "Check the results, not just the code": a bug like a truncated UI string won't show up by reading the source, because the code looks completely correct; only running it reveals the problem.
- Bugs visible in the code itself, like architectural misuse or missing tests, can increasingly be caught by agents; bugs that only show up at runtime still need a human or an automated verification loop to catch.
09Matt Pocock's reply: tactical vs. strategic programming
- Tactical programming, day-to-day code, is largely automated by current AI agents; strategic programming, architecture and long-term codebase health, still shows no evidence of being something agents do proactively.
- Agents are useful as strategy consultants, you can ask them to evaluate your architecture or investigate a theory, even though they won't originate the strategy themselves.
- Pipelines like "bug report to repro to fix" or "RFC to refactor" are themselves a strategic decision a human has to design; agents only execute inside the pipeline you build.
Terms worth knowing.
- Tactical programming
- The on-the-ground, day-to-day work of writing code to make a specific change, as opposed to thinking about the codebase's long-term structure.
- Strategic programming
- The long-term work of codebase health and architectural decision-making, choosing the right structure so future changes stay easy.
- Agent swarm
- Multiple AI sub-agents spun up in parallel by a coding tool (like Claude Code or Codex) to execute different parts of a plan at once.
- Verification loop
- The QA, staging, and testing systems that check whether code actually does what the plan intended before it reaches real users.
Things they pointed at.
00:00toolClaude Code
14:31toolCodex
Lines you could clip.
00:00
“Coding is solved, bugs are not yet solved. Fix incoming.”
00:47
“It's like a raspberry ripple of bullshit, a vanilla ice cream of VC funding stirred with a little pinch of turd.”
18:30
“If it is too hard for you to spin up and test, it is way too hard for your agents to do the same.”
25:49
“The era of finding bugs by just reading the code has, let's be real, ended.”
29:09
“I consider myself an average programmer, so Claude has already surpassed me.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
metaphoranalogy
Coding is solved, bugs are not yet solved. Fix incoming. Oh, boy.
It's been a bit since a Boris tweet annoyed people as much as this one. I miss the days where he would just randomly post about how Claude Coat has replaced all of their engineers and everyone would get mad. But at the same time, a lot of the things he said before now kind of seem right.
They were obnoxious at the time and were just very far off, especially the timeline aspects of what he said. But the reality of engineering nowadays is that if you're writing your code by hand still, you are largely falling behind.
And I have to do a thing I am not excited to do. I have to defend what Boris said here. Because a lot of people are looking at this in the wrong light and pushing back in ways I don't necessarily find fair.
Matt Pocock's pushback in particular is beautiful. Been thinking about this quote for a solid twenty four hours, it's like a , a vanilla ice cream of VC funding stirred with a little pinch of turd. And the hot take you probably don't expect from me here, Matt's also right.
And I wanna do my best to break down what both people are saying here because they're using the same terms to say very, very different things. And if you understand where they come from and what they're actually trying to say underneath, I think you can learn a lot about where software is going and how people are applying these things in their work.
But first, I have to do my job quick, which means it's time for a sponsor break. I've been landing a lot of PRs recently, like a truly insane amount. Before filming today, I ended up landing like eight PRs on a day where I also had a bunch of meetings.
Sorry, it looks like it was actually more than that. It's crazy. And today's sponsor is one of the pieces that has helped the most here, is the CI Runner Blacksmith.
These guys have revolutionized what I'm willing to use GitHub Actions for. I was just so used to them being really slow and unreliable that I just stopped putting things on them I didn't have to. And then I started using Blacksmith, and all of our times got cut by like two x or more.
There's a lot of layers to what make Blacksmith so much better than traditional GitHub actions. From the hardware that's two times faster, to the cache downloads that are four times faster, to the absurd 40 times faster Docker builds. All of that by itself is enough to feel a real difference.
What's even cooler is that their documentation is so good that your agents can parse it really well. I asked my agents how I can speed up CI even more, and they found a bunch of ways to do more parallel work. But more importantly, they found sticky disks from Blacksmith, which lets you set up a chunk of your disk that is stored across runs, which make a bunch of things from downloading a ton of node modules to giant file transforms to your old huge git history all way, way faster.
I look at this chart and I cry a bit because I've seen GitHub action cache downloads take way over a minute. Some of our CI spent more time downloading than running. But if you switch over to sticky disks, there's literally no download speed because it's already there.
It takes three seconds instead of over a minute. If you want faster CI today, tell your agent to check out soydev.link/blacksmith. Okay.
Good to have you back. Let's start here with Boris' tweet. This is a response to Peter's post showing a bug in the Claude Co desktop app, which admittedly is a terrible showcase of what AI models are capable of because there is no excuse for the Cloud Code desktop app to be in as rough of a shape as it is.
The team is working hard on it. They've taken my feedback surprisingly well. They were a little beat up when I first called out how bad the Cloudco desktop app was, but they realized I'm doing it in good faith.
I just want it to be better. So now they're making real improvements. Those improvements involve shipping updates all the time.
And when you get an update, you can't actually read the update text because it gets truncated because a model wrote this field, and when the model wrote it, it was just reading the code, and it didn't realize that the window space available for this little callout thing wasn't long enough to actually fit the text being rendered.
This is such a common mistake that I see all the time with AI generated code that frankly didn't happen a whole lot before AI generated code took over. If you were building a feature like this, you would probably make sure it renders in the browser while you're working on it so that you could see how it looks. Hypothetically speaking, you could just write the code and put up the PR having never tested it, but in a world of humans writing code, why would you do that?
Why would you waste the time of your coworkers with code that you never even saw the output of? You'd probably make sure it shows up in the browser at the absolute least. And when you did that, you would notice this edge case and fix it before you bother your team.
By default, agents don't do that at all, and the result is things like this. And in a world where we've learned the agents write good code and it's usually safe to merge, we get really used to doing that where we just hit the merge button if we look through the code quick and everything seems fine. We've always trusted that original developer to test the thing they changed and make sure there are no bugs before we put it up for code review.
But now we live in a world where the code review going up does not imply any effort was put into checking the thing ahead, and you end up with stuff like this. This is why Peter's post quoting the coding is solved thing is actually hilarious, and rightfully so, because while agents can write the code, they're missing so many of the things that we would expect modern software to do, and we would expect more importantly for the humans writing the software to get right.
It is also absurd that we can't hot reload agents that are supposed to run twenty four seven. I wish it was easier to do that. I understand technically speaking why it's not.
I have been working on this a little bit with T three Code so I can empathize to the difficulty of this problem, but I think the bigger call out here is that you can't actually read the text in this. Boris' response here is less about the hot reload thing and more about the visible bug in this image, which is why his follow-up's saying that this isn't a bug, it's a UX issue that we have a fix for.
This is a bug. I gotta reply to this one. Boris, I'm doing my best to defend you here, but this reply's making it harder.
By what logic would you say the text being cut off in all reasonable display sizes isn't a bug? I'd be really curious what your definition of a bug is because this cannot be the intended experience. In order for us to have a conversation about this, we need definitions for the words coding and for solved.
I hope we all know what is means. Coding can mean a lot of different things. It can mean telling somebody about an idea and they go build the whole thing, turning the idea into an experience.
That whole end to end could be coding. Coding could be just the part where you open up your editor and you write code. Coding could be the process of turning a Jira ticket into a pull request.
Coding could also include the process of reviewing that code or pulling down someone else's changes. Coding can be a very narrow task or a very wide one depending on how you want to use the term. And let's be real, even before AI, how much of an engineer's time was actually spent writing code in their editor?
If we think of coding as the end to end aspect of what an engineer does, obviously coding is not solved. If coding is the thing I do from when I walk into the office as an engineer to when I leave, no, AI has not solved coding.
But if we think of coding as the thing that happens when I open up my text editor and I'm pressing keys on my keyboard, if we narrow that definition down to be that small, where coding is the act of typing code on my keyboard inside of an editor, yes, absolutely coding is solved.
I'd even widen a little bit and say if you include things like finding the right Jira ticket, going through the stuff that users are doing, turning the data from the experiences users have into the code that you want to write, that whole experience from like rough idea to code that works, that all can be done with the AI now.
And when it comes to actually generating code that functions, AI has solved that. But engineering isn't just coding.
Gonna do a silly example. Please hear me out. This is The Password Game by Neil Fun.
It's meant to mock how hard it is to sign up to most websites nowadays because their password requirements are absurd. So every time I fix what is missing, I have to make additional changes. Now it says the digits of my password have to add up to 25.
Cool. There we got that. Password must include a month of the year.
April must include one of our sponsors.
Hilarious. Okay. So why am I showing this?
This is obviously a terrible experience. If this was the password system in a real website, we would make fun of it endlessly because it's bad. We don't want to use this.
But was Neil coding when he built this? Obviously, yes. When Neil made this game, he coded it.
He had the idea. He opened an editor. He wrote code to make this thing exist, and now it exists, and we can experience it.
He did intend for it to be this rough, buggy, messy experience because that's the point, is making fun of these types of experiences. So he did it with intent, but he did code this. And I hope we all agree that the process of somebody like Neil having this idea and then putting it out there, that coding is at the very least involved heavily in this process.
So now I have to ask, if somebody built something this bad accidentally, where they were told to implement a password field in their website or web app, and when they did that, they made it really rough and miserable like this, were they coding? I would hope we can also agree the answer there is yes.
That just because the code didn't come out good or useful based on the actual goal that a user would have using it, doesn't mean they weren't coding. I hope we can also agree there's a difference if this type of horrible experience happens because there wasn't enough detail in the request made to the developer. Or if I told them, make a password system for our site, and then they went and whipped this up because they didn't know what a good or bad password like experience was, that is different from if I write a bunch of tickets that say we have all of these crazy rules, make sure that you apply all of them, and the developer creates something with all of that bad decision making applied to it.
These are very different. Bad code happening because of no detail given in the planning process versus bad code happening because of bad details in that same planning process. Missing details can lead to code that doesn't behave as expected, and bad details can also lead to code that doesn't behave as expected.
But that's the input side. Bad info going in, whether it's missing stuff or is just bad with the details it has, going into the code box results in bad code. But there's another thing that can often result in bad code, which is the other side.
Not that the code comes out bad, but because you don't have a system that allows you to verify the code on the other side. Let's look at it like this. You have these different qualities of plan.
You have the engineering cycle. It could be one person, it could be a team, it could be three engineers doing a sprint, it could be whatever. And when you put in a good plan, we hope that out of this comes good software.
There are plenty of ways that this can change, obviously, if the quality of the engineers isn't great, if the good plan gets lost along the way, if somebody rewrites the plan and goes off course. But generally speaking, if you have good engineers and a good plan, we hope that the result would be good software.
If you have a bad plan, the most that we can hope the engineers do is give feedback on that bad plan and say, hey, I'm not sure if this is the right idea. But if I've learned anything from my time as an engineer, it's that people outside of engineering often don't like when you come in and say that their plan for how to change the software isn't great.
And I understand. It's their job on the product team to figure out what we should be doing, not the engineer's job. It's our job to do it.
But I would then ask you, is a truly exceptional coder, a person whose job is to turn ideas into code, are they going to push back on a bad plan? Or would they just implement that bad plan because they're good at coding, not making good software? Because those are different things.
A bad plan going through great coders that aren't there to push back on plans, they're there to execute those plans. They're going to make bad software because the thing going in is bad, the thing going out is obviously also going to be bad. And then there is the empty or bare plan where there just isn't enough detail in the plan.
This has the widest range of potential output. Like if the plan is good and the engineers are good, we can say the range of success here would be like eight to 10, obviously, out of 10. Where with the bad plan, I would argue that the software coming out will also probably be bad as low as like a one to a four out of 10.
Regardless of how good the engineers are, this just kind of affects what range you're falling in there. And then with an empty or bare plan, the range is massive. I would say as far as one to 10.
You have no idea. Depends on a lot. It depends on how complex the thing is that is being requested.
Depends on how well the engineer understands all of this, as well as how well they understand the needs of their users, how complex the software is, how complex the request is. There's so many layers to this that the range, I would say, could be anything. If you give great engineers an empty plan that is a very, very vague idea of what you want, and they're excited to just write code and put up a pull request, the likelihood that this is good is random almost.
Probably not looking great if I'm being real. So what do we do to fix this? How do we handle this as an industry?
Obviously, step one is to write better plans, but if we can't control the quality of plans at all times, we need to make sure we have other solutions. We also need to make sure that the code that comes out is matching the needs of the plan because we don't know if the engineers in this box are good or not either.
So how can we improve our outcomes in the eng box here? I would argue we've had a solution for this for a long time now. It's the verification layer.
Things like QA, things like staging environments you can test in, things like slow rollouts that will make sure that bad code doesn't hit users, and if it does, that it minimizes the impact of that bad code. Good verification steps will allow for your engineers to verify the results against the plan, and perhaps in some cases, even notice that once you implement the plan, the experience isn't great.
If I have a plan that looks good, I can't tell it's a bad plan because it all seems fine, and then I build it and I put up the code, it's unlikely that I know the actual experience is bad until I try it. Trying it could be spinning it up on my machine, trying it could be testing the preview build when it happens. Trying it could be your agents going through the loop with it and showing you a video of it when it's done.
But you need something here to prevent the bad plans and even to an extent the good plans from having things sneak in that are not a good experience. Good plans and good engineers should result in good experiences, but I would argue they only do when the good engineer has a system to verify the changes are actually good.
But if you have a bad plan or bad engineers, then you need this even more in order to verify the quality of those outputs. Hopefully, you can already see what I'm about to do.
Let's replace this team of engineers with who is really doing the work today. Swarms of agents. You take your plan, you give it to Claude, you give it to Codex, it spins up a bunch of sub agents to execute all the parts, and then it puts up a pull request.
If it hasn't actually run the code to check, it might look fine, it might seem fine, but you don't know if it is fine until someone checks it. Usually, that'll be a human, and sadly enough, the human is often the one using the software in the end. It's not one of the humans who work on the product.
And that's exactly what happened here. Do you legitimately think after this code was written by an agent that any human or agent actually checked what it did in the app?
Obviously, no. Because this would be a very quick thing you see and you're like, oh, yeah, that is wrong, I should fix it.
But no one did because no one saw it. And by the time it has merged, the friction to unmerge it or to fix it is higher than the friction to build and merge it in the first place. To go back to our diagram here, let's say that after the verification loop, you've determined that the software is good, at the very least based on your understanding that the problem you're trying to solve has been solved and the thing that comes out in all obvious ways seems fine.
And I would expect that the result is good software, probably similar range, maybe we could even bump from that eight out of 10 to like nine out of 10 or 10 out of 10 range. What happens if the result isn't good? Well, ideally that bad result will send you back up to the planning stage so that you could adjust the plan based on what you've learned from the verification.
Maybe the bug is just something silly in the implementation and you can skip straight back to the agent swarm or the engineers working on it to have them fix whatever isn't right. The point I'm trying to make here is that if your verification system is good, then before the code ever goes to users, it should be able to identify the problems and fix them as they are working.
And right now, agents don't do this, at the very least by default. And this can be caused by a lot of things, many of which are actually human error. I cannot tell you how many code bases I've worked in where just spinning it up on my machine to see if my changes worked was more complex than making the change in the first place.
And it ends up being easier to just file the pull request and wait for the preview build to come up than it is to try and test it on my machine. If it is too hard for you to spin up and test, it is way too hard for your agents to do the same. You need to make it easier to actually verify your code bases.
And I find the companies struggling the most with this are the ones who are building in a way that is hard to verify. Funny enough, Anthropic is one of the most guilty of this. Since Claude Co.
Desktop is a full desktop backed electron app, in order to run QA and verify it, they need to give every thread its own graphical VM fully backed with a real instance of an operating system that is supported with Cloud Code Desktop so that it can run the build, spin up that Cloud Code Desktop instance in an environment where Cloud can control it and check it and prod at it.
And when you combine the fact that that is expensive and difficult and far from trivial to set up with the additional fact that ClaudeCode models are just so much worse at computer use than models from other places, you end up without that good quality verification loop that is necessary to prove this software works. It's silly to put it this way, but I think a significant portion of why the Claude Code desktop app isn't improving meaningfully in comparison to the Claude Code web app, which is improving constantly, simply comes down to how easy it is to set up Claude Code web for testing versus Claude Code desktop.
Since the Cloud website is built on, you know, the web, it's a lot easier to set up a loop where your agents can test it. You'd even have one given machine where your agents have eight tabs open for eight different dev builds checking different things, and it's totally fine. With Cloud Code desktop, good luck launching eight separate Cloud Code instances as a desktop electron app on the same computer that I can poke and prod through without getting confused along the way.
It's not gonna happen. But this would also be hard for a human. If I was trying to review two different PRs and test their changes at the same time on my computer, it wouldn't be pleasant.
I know because I've built Electron apps for a decade. If you have two things you want to check at the same time, you're kinda just screwed. And this means you have to build a system to make it easier.
Again, if it would be hard for the human to verify, it is probably impossible for the agent to do it. And Quadco Desktop is a golden example of what happens when you don't have a good enough loop for catching these types of things by actually testing the changes that you make. So going back to this diagram, my question for you would be what parts of this are coding?
Do you think coding is narrowly scoped to this part here where plans are transformed into code? The act of typing on the keyboard in the editor or the agent transforming these ideas into source code that you can actually run and use? Or do you think it's wider?
Do you think coding also includes the planning side, actually figuring out what you wanna do? Do you think it includes the verification side, making sure the changes are valid or not? Do you think it includes the full loop where that verification layer goes back into the coding part?
I think everyone defines this a bit differently, which is why the conversation has gotten as chaotic as it has. Because for some people, coding is just the section in the middle, the agent swarm part where you're writing the code. And for others, coding is everything you do going from idea to functioning software.
I can hear the argument either way. I personally don't care that much. But at the absolute least, I would say that the whole process here from start to end is software engineering.
And if anybody tries to say that software engineering is solved because agents write code okay in this middle section, then both Prime and I will be equally mad at them because software engineering isn't solved. Software engineering isn't just this little piece here where you make code. Software engineering isn't even just this square.
It is far wider and forces you to think about the architecture of the systems you're building so that this whole loop can happen more effectively hundreds of times a day. The engineering isn't just what you do inside of this box, it's the platform you build for the boxes to sit on and stack up and continue to grow. But Boris didn't say software engineering, he said coding.
And that's historically what he has said. If Boris had said software engineering is solved, I would be making a very different video right now. But he didn't.
Bugs are part of software engineering, the same way that building good verification loops is part of software engineering. Architecting systems in a way that you can make changes confidently is part of software engineering. These aren't just coding tasks, these are engineering work.
And that's why that word exists because it's not just the task of being a code monkey sitting at a computer typing letters on the keyboard. And if I'm being really frank here, agents need a lot of help with this part, the same way that an incredible engineer that you just hired would need help with these things in order to work in your code base.
If a new engineer showed up and they were assigned this change and they couldn't even get the test build running on their machine, much less three of them so they can test your changes too, they're just as screwed as the agents are. And I think a lot of us have code bases where a great engineer would be unhappy working in them and wouldn't have the resources they need to verify their stuff.
Now imagine that person has been blinded, that you literally . How good of software do you think they're going to make?
Probably still decent here and there if the changes don't require them to see what they're working on, but it's a lot better if you give them what they need to verify those changes. So when I'm hearing Boris say coding is solved, bugs are not yet solved, what I'm hearing him say is that this part here agents have gotten incredible at.
If you give them a well spec'd change, they will make that change. The change might not be what you actually want, but if you tell them what to do, they will do it 99% of the time, and if they get it wrong, you can tell them what they got wrong, and they'll fix it 95% of the time.
But the next part, which is finding and preventing bugs autonomously, ends up being significantly more difficult, not just because the agents are stupid and don't know how, but because our code bases are stupid and don't set them up for it too. We need improvements in our system and architecture in our apps in order to make them easier to test and verify.
We need better tools from the labs and people building AI development tech in order to make it easier for the agents to actually access those systems that we're building and to test things in verifiable, repeatable ways. We need better systems for our agents to show us that they actually checked their changes and to make that more normalized.
Hell, we need a way for GitHub to allow our agents to upload an image or a video proving their work without having to commit it to the repo, which has caused all sorts of disasters across a lot of projects. Fun fact, there's a new GitHub CLI update where they did actually finally add this, which I think is a huge stepping stone in getting all of this right.
But that's the key. We need to figure out both what we can do in our code basis to give agents more likelihood of verifying things well themselves, as well as expect the providers we rely on, whether that's Anthropic and OpenAI, or if it's GitHub and Google Chrome, whatever the layers are, we need them to be exposed in ways where our agents can verify their work and prove that they have done such too.
And even then, I've seen a few too many times where an agent shared a video of it proving the work it did is good just for the video to be obvious slop where the thing didn't work at all. I cannot tell you how many times I've had to say, wait, the video you shared shows this not working. What did you see?
And then it realizes it got wrong and it goes and fixes it. Another real hot take I have on this before we move to Matt's post, I think if you have the bug specked out properly, like a detailed enough bug report that describes what is wrong and what the fix should look like, and then add even the most minimal verification system there, the vast majority of frontier level agents and models can absolutely solve the bugs too.
They're not going to ship software without bugs. They're not going to preemptively find these bugs and fix them as often as we'd want. They're never going to ship bug free software.
That's just not really possible. There will always be edges that you didn't plan for in your systems and in your verification. But if you make it easier and easier to identify these bugs and the models get smarter and smarter at fixing them, then yes, to some extent, bugs will start to be solved as well.
Imagine a world where the agent starts working on the code, it makes the changes, it wants to test them, it realizes it can't because the repo isn't set up properly or its tooling isn't set up properly, so it then goes and builds all the pieces it's missing in order to verify its own changes. I can't tell you how much time I've spent with t three code setting up things so that the agent can actually spin up and test work as well as expose it to me.
When I'm working remotely and I have my box set up over Tailscale on another machine and I wanna see what changes it made in its dev server, I can't click a local host link when I'm on an entirely different network. I need it hosted through Tailscale. So I actually made a lot of changes, like thousands of lines of code changed in t three code to make it easier to run the dev server over a Tailscale environment while also pulling in a read only snapshot of your existing data to make it a more useful test.
These are the types of things I have done both to make it easier for me to verify the changes the agents made, as well as for the agents to verify their own work too. In the future, agents can notice that they need these things and unblock themselves and make suggestions or even full on create a new pull request just to set up your repo for them to catch bugs themselves, which will massively reduce the number of bugs that they are shipping.
But until then, it's our responsibility and we need to build the systems the agents need to verify the bugs and we need to let the agents know when the bugs happen. And most importantly, we need to check the results.
Not necessarily the code, because let's be real here. Nobody would have caught this bug in code review because the code looks fine. If it didn't, the agents would have solved it.
The era of finding bugs by just reading the code has, let's be real, ended. Any bug you notice from just reading code can almost certainly be noticed by the agents too and ideally noticed within their own development loops.
But you still can notice a lot from reading the code, like architectural failures, not using something that you should have been, using something you shouldn't have been, touching a place that shouldn't be touched for these changes, writing tests that aren't necessary, not writing tests that are necessary. There's lots of things you can see in the code, but those aren't bugs in the traditional sense.
And the bugs can absolutely be seen by an agent if the bugs are visible in the code itself. But sadly, a lot of bugs aren't, like the one we have here. What that means is we need to be actually using the code our agents write.
We need to pull it down on our machines and test it and check it and make sure it solves the problem it's intended to. At the absolute least, we need the agent to show us that it did that itself. And until the agents can do that whole loop themselves, bugs are definitely not solved.
So hopefully, you now understand my stance on this and what I think Boris was saying. So let's wrap up by going through what Matt said as well as Boris' responses. Matt's saying that this post reads like a vanilla ice cream of VC funding stirred with a little pinch of turd.
This is a totally reasonable read of what Boris said because he said it in a way meant to ruffle feathers. That's what the whole Coding is Solved thing does, especially if you have different definitions of coding.
If one of these people is using coding as a term for what they're doing on the keyboard and the other is using it for the entire software development life cycle, they can't agree because they're arguing the same term with different meanings. But Boris didn't offer any clarity in his post here, so it's a totally reasonable way to read it if you use coding as a way of describing the whole life cycle.
Boris replied, TBH, this is a good debate to have. Here's what the timeline of the near past and future looks like to him.
One, models are able to code better than he can. Two, models are able to do coding adjacent engineering work better than he can, like debugging, profiling, optimizing bug fixing, system design, abstraction design, UI design, idea generation, etcetera. And eventually, models are able to do most things that can be done on a computer better than most people.
For the types of coding work that Boris does, Claude has achieved the first, which is that models are coding better than he does, and some, but not all, of two, which is that they can do coding adjacent things like system design, bug fixing, optimizing profiling, debugging, etcetera. But it's only starting to show early signs of three, which is doing most things doable on computers better than people.
This is not the case for all of coding yet. If you're on Darius Helzberg and are a world class expert in compilers and type systems, Claude might not be at one yet. Boris is betting that it will get there quite soon, while at the same time pushing further into two to three territory for the rest of the world.
I would argue that we are actually surprisingly far into one for those types of god engineers. I know that for example, Ryan Carniato has been going more and more deep into Fable and realizing that he can use it not just to like make small changes to the framework he built, SolidJS, but to actually talk out complex ideas and test theories that would have been too time consuming for him to do.
So I am seeing more and more of like the top upper echelon best engineers in the world realizing that they can utilize these agents to do a lot of coding work, not like as incredible as them, but close enough to it with a lot more parallelism that lets them test things that they would not have been able to otherwise.
Back to what Boris said here. You can think of model capabilities as capturing the distribution of human ability.
I consider myself an average programmer, so Claude has already surpassed me. The same is not yet true for everyone. Back in November last year, Boris fully stopped writing code by hand, though he continues to code using agents every day.
That's the point where, for him and many others around him, it felt fair to say that coding is solved. But engineering is more than coding. Claude's code is not perfect.
It has bugs and inefficiencies. Claude can code, but it cannot yet do everything that goes into engineering. I swear I did not read this before deciding to make this video or filming the section before.
It's funny that he makes the same distinction, but again, I had a feeling that's what he meant. Opus four eight was the first model that felt like it could be better than he was.
Babel now routinely finds optimizations and debugs issues that would have not been able to be found by Boris himself. And with each generation, the code the model produces continues to improve and the model's able to do more of the non coding parts of engineering. Similar to how I had pushed back on Boris earlier with his definition of bug because he said that it wasn't a bug, it was a UX issue, Matt's asking for his definition of coding.
He cites John Osterhout's definition of tactical versus strategic programming. Tactical is the on the ground day to day aspects of coding, and strategic is the long term stuff like code based health, architectural design, making the right decisions.
Matt agrees that AI has largely solved tactical programming, but he has seen no evidence that it can think strategically. Asterhout talks about some employees as tactical tornadoes, able to churn out astonishing amounts of work with zero eye for the future, and that's how agents feel to him right now. I largely agree.
I don't think agents will preemptively, proactively do good with strategic planning of code based health and architecture over time, but I do think they can be useful resources during that workflow.
If you ask agents to look into these things and give you feedback on how your architecture looks or what changes you're planning on making, and you can even have it investigate theories to prove ideas you have out too, there's a lot of power to be had with agents helping you as consultants when you plan your strategy. And then once you've done all of the strategizing and you have a plan you're confident in, and you give your agents the things they need to verify their outputs, all of a sudden, they can go execute a crazy bold plan in time that makes no sense at all.
It's truly unbelievable how fast these agents can work if you set them up for success, which, spoiler, has almost always been the case with engineering.
If you set up your systems in a way where everyone at the company could easily pull down someone else's changes and verify that they fixed the issues they have, where it's easy to make those changes and find where in the code base is relevant to the stuff that you're doing, making it easy to actually put the code up for review and to get it approved and shipped and rolled back if it does break things.
All of these decisions you make around how the day to day works in your code base would have made engineers more effective when you brought them in. And you can bet your ass they're gonna make the agents more effective too. It's actually really funny reading all of this because I swear I had the whole video planned before I did and they were doing this back and forth right now, but it really does line up with what I was saying there.
Where Boris splits coding and programming up as the act of writing code and engineering as everything else in addition to that. Tactical and strategic's another reasonable split. You does see some parts of strategic coding being automated, like when they're starting to automatically maintain apps that are for Epic.
Recently, a number of their customers are too. We haven't bit this bullet quite yet at t three code. I'll still screenshot the tweet and go post it into t three code myself.
I won't have the agents propose fixes based on what people are tweeting yet. Matt calls out that he agrees partially on the strategic coding being automated thing, saying that people are building pipelines like bug report to repro to fix, feature request to prototype, or cron to code base architecture, RFC to refactor. Managing these pipelines is very much a strategic concern, one that a human needs to apply their judgment to.
Absolutely agree. In order to make your agents act more strategic and act more like engineers, you have to do the engineering upfront yourself. The agents can help you as you do it, but if you want the agents to be able to do these awesome things, you have to set them up for success.
Man, I thought this was gonna be a quick fifteen minute video and I'm looking at the timer and realizing it wasn't. Thankfully, I filmed this offline for once. I know usually my videos are done on stream, but cast, okay?
I didn't feel like going live today and just wanted to talk about the stuff that was interesting to me, and I hope you enjoyed it. This is quite a fun deep dive into all these things that I've already been thinking about, and I love the excuse that both Boris and Matt gave me to talk more about it. I'm curious how y'all feel.
Am I being crazy saying that engineering is still unsolved and coding is solved? Am I going too far by saying coding is solved, or am I not going far enough by saying engineering is still largely a human task? Let me know how you feel and where you guys are at in your real world work.
Until next time, peace nerds.
The Hook
The bait, then the rug-pull.
Boris Cherny's four-word tweet, "Coding is solved, bugs are not yet solved," set off a fight with Matt Pocock that looks like two smart people disagreeing. Theo argues they're actually both right, because they're using the word "coding" to mean two very different things.
Named ideas worth stealing.
10:25model
Plan Quality → Software Quality
- Good plan → 8-10/10 software
- Bad plan → 1-4/10 software
- Empty/bare plan → 1-10/10 software (random)
Theo's whiteboard model showing that the quality of the upfront plan bounds both the ceiling and floor of the resulting software quality, regardless of how good the engineers or agents executing it are.
Steal forframing why "the agent got it wrong" is usually a planning problem, not a model problem
14:31model
Verification Loop
- Plan → Agent swarms → Verification (QA) → Good result → Good software
- Verification → Bad result → loops back to Plan
Extends the plan-quality diagram by swapping engineers for agent swarms and adding an explicit verification/QA stage with a feedback loop back to planning when results come back bad.
Steal fordesigning CI/QA gates that agent-generated pull requests have to pass before merge
27:29list
Boris's Three-Stage Capability Timeline
- 1. Models code better than the human
- 2. Models do coding-adjacent engineering work better (debugging, profiling, system design, UI design)
- 3. Models do most computer-doable tasks better than most people
Boris Cherny's own framing, quoted from his reply to Matt, for how AI capability expands outward from raw coding into full engineering and eventually general computer work.
Steal forevaluating how far along a given AI coding tool actually is before trusting it further
30:11concept
Tactical vs. Strategic Programming
- Tactical: day-to-day, on-the-ground coding
- Strategic: long-term codebase health, architecture, making the right decisions
John Ousterhout's distinction, cited by Matt Pocock: AI has largely solved tactical programming but shows no evidence of strategic thinking, acting like a "tactical tornado."
Steal fordeciding which engineering decisions to delegate to agents versus keep human-owned
How they asked for the click.
VERBAL ASK
01:38product
“tell your agent to check out soydev.link/blacksmith”
Woven into his own workflow story (landed eight PRs that day) rather than bolted on, and tied directly into the video's verification/CI theme instead of feeling separate from it.
MENTIONED ON CAMERA
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
OTHER LINKSAlso linked in the description.
Visual structure at a glance.
