A two-hour sit-down where the skills guy walks a self-taught builder through his whole agent loop, live, and keeps landing on one word: care.
Posted
2 days ago
Duration
Format
Interview
educational
Views
276.7K
3.8K likes
57 · 43
Big Idea
The argument in one line.
AI now handles tactical programming, so the human's real job is strategic: shared language, deep modules and deterministic feedback loops that keep the agent's environment from rotting.
Who This Is For
Read if. Skip if.
READ IF YOU ARE…
You ship apps with Claude Code or Codex but have no software engineering background and can feel your codebase getting worse with every feature.
You already use the grill-me skill or similar prompts and want to see the full loop it sits inside, from spec to tickets to code review to retro.
You run agents on high or max effort by default and want a concrete argument for why medium effort and small context windows produce better work.
You want to merge agent pull requests without reading every line and need a framework for deciding which changes deserve human review.
You are a student or career-switcher wondering what to learn now that writing syntax is no longer the scarce skill.
SKIP IF…
You want a step-by-step build of a specific app; the demos are short and mostly exist to illustrate the philosophy.
You need security guidance; the guest says plainly he is the wrong person to ask and only suggests adding a security pass to code review.
You want a model or harness comparison; the guest deliberately sticks with one model and one subscription.
TL;DR
The full version, fast.
AI is excellent at tactical programming (solving the task in front of it) and poor at strategic programming (keeping the codebase easy to change over time), so the human's job is now the strategic layer. The working loop is: grill the agent until you share a vocabulary, write a spec when the work won't fit in roughly 150k tokens, split it into tickets, implement them with sub-agents in worktrees, run an adversarial review against the spec and your coding standards, then retro so the next run is better. Trust comes from deterministic feedback loops (strict types, fast tests, harsh lint rules) and from deep modules that can be tested from the outside. Use medium effort, keep context small, review by merge danger rather than line by line, and read the classic software design books.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Cold open montage: languages are tactical, engineers still out-ship non-engineers because they know what to ask for.
00:54 – 05:03
02 · Who Matt Pocock is
From XState and Total TypeScript to 'the skills guy'. The host frames himself as a self-taught builder with a magic wand who can feel its limits.
05:03 – 12:35
03 · Tactical vs strategic programming
AI is trained on tactical benchmarks, so it lets the house burn down around it. Vibe coding only fails because nobody keeps the environment in order. First sponsor break at 11:32.
12:35 – 25:18
04 · Live demo: Grill with Docs
A vague prompt about a cluttered teleprompter becomes a grilling session driven by a project glossary and 31 ADRs, closing the communication gap before any code is written.
25:18 – 34:35
05 · Smart zone, dumb zone and the spec
Small work goes straight to implement; anything over about 150k tokens becomes a spec and tickets. Why medium effort beats extra-high for almost everything.
34:35 – 44:52
06 · Sub-agents in worktrees and code review
Implement-spec fans tickets out to parallel sub-agents, then a two-axis review checks spec and coding standards. Human review is triaged by one-way vs two-way doors and blast radius.
44:52 – 52:31
07 · The trust ladder
Scaling past a handful of agents requires trusting them. Types, tests and lint rules let agents grade their own work, and retro ratchets those checks tighter every run.
52:31 – 1:01:01
08 · Starter stack and deep modules
GitHub, VS Code, Git and any skills-capable agent. Why most tests just repeat the implementation, and how deep modules let you turn the lights off inside parts of the codebase. Second sponsor break at 59:52.
1:01:01 – 1:08:15
09 · Improve codebase architecture, live
The skill runs against the video editor repo and returns an HTML before/after report, while a daily cron opens one improvement PR per repo. Detour into why phones are cigarettes.
1:08:15 – 1:14:11
10 · What you still need to learn
The teach skill turns an unreadable report into a personal course. Domain language is why engineers prompt better, and caring beats the bitter lesson for now.
1:14:11 – 1:26:38
11 · Hosting and deploying safely
Use managed Postgres and a push-to-deploy host; DevOps has been productized. Add a security pass to code review, and pick deterministic CI/CD over agents deploying by hand. Third sponsor break at 78:25.
1:26:38 – 1:34:41
12 · Open source in the age of slop PRs
Sandcastle and Everlight, why popular repos like Hono are closing PRs, and why good issues and repros are now the real contribution.
1:34:41 – 1:44:39
13 · Advice for new developers
Show strategic care through shipped projects, read the classic design books with the teach skill, and stop worrying about specific languages or degrees.
1:44:39 – 1:56:49
14 · Software factories and the AI video editor
A software factory removes humans from more steps of the loop. The open source course video manager edits as he records, and an audience wiki feeds course planning done on paper cards.
1:56:49 – 2:01:01
15 · Agents for your life
Daily stand-ups with agents and a watch-to-to-do pipeline, but knowledge work is harder than coding because it can't be verified.
2:01:01 – 2:02:36
16 · A prayer for you
The host's standing sign-off prayer for viewers anxious about AI and jobs.
Atomic Insights
Lines worth screenshotting.
AI is great at tactical programming and terrible at strategic programming, because benchmarks only reward finishing the task in front of it.
Strategic mistakes in code usually don't come back to bite you until about nine months later, which is exactly why models are never trained to avoid them.
Pure vibe coding fails over time because the code is the environment your agent works in, and every tactical shortcut makes that environment worse.
A shared glossary between you and your agent makes even a verbose model terse, because it stops inventing its own names for your domain.
Start getting nervous around 150k tokens of context; past that point the model makes measurably dumber decisions and every follow-up costs more.
Most people run their agents on too high an effort setting; codebase exploration doesn't get better with more effort, it just burns context.
Save high effort for the agent verifying its own work, such as clicking through the real app or running performance traces.
Before waiting for the next model to fix your problems, fix the environment your current model operates in; that is usually the bigger win.
Review AI pull requests by merge danger: a two-way door with a small blast radius can ship untouched, a one-way door with a big blast radius gets attacked.
Every human review comment should turn into a rule, ideally a deterministic lint rule, so you never have to write that comment again.
If you have a choice between a deterministic check and asking an agent to remember something, always pick the deterministic check.
Codebases aren't hard for agents because they're big; they're hard because they're poorly organized and hard to navigate.
A test that just repeats the implementation is churn, not safety; test deep modules through a narrow interface so you can change everything inside them.
Deploying with an agent logging into servers is non-determinism you don't need; CI/CD solved deployment years ago and costs zero tokens.
Open source is shifting from pull requests to high-quality issues and reproductions, because agents can now turn a good issue into a PR cheaply.
Coding is easier for AI than knowledge work because code is verifiable; there is no type checker for whether a course is better than the last one.
Removing addictive apps from your phone does not automatically refill your life with inspiration; nothing rushes in to fill the gap.
Takeaway
Your job is the strategy the agent won't do.
WHAT TO LEARN
Agents will write all the code you ask for, but only you will keep the codebase easy to change, so invest in shared language, small contexts, deep modules and deterministic checks.
03Tactical vs strategic programming
Models are trained to finish the task in front of them, so they optimize for today and quietly degrade the codebase unless someone keeps the long view.
Pure vibe coding breaks down because the code is the agent's working environment, and every shortcut makes the next task harder for it.
Most strategic programming is simply caring more than the person next to you about the quality of what ships.
04Live demo: Grill with Docs
Have the agent interview you before building, because a vague prompt leaves a communication gap no smarter model can close.
Keep a project glossary and short decision records so the agent uses your terms and doesn't repeat mistakes it can't see in the current code.
You don't need to write the glossary up front; let grilling sessions mint terms as you work on the hotspots you actually touch.
05Smart zone, dumb zone and the spec
Treat roughly 150k tokens as a soft ceiling; if the work won't fit, write a spec and split it into tickets sized for one session each.
Default to medium effort; exploration doesn't improve with more effort, and the extra tokens push you into the dumb zone sooner.
Fix the agent's environment before waiting on a better model, because the low-hanging fruit there is usually bigger than a model upgrade.
06Sub-agents in worktrees and code review
Run tickets with parallel sub-agents in worktrees, then have the agent review its own output against both the spec and your coding standards.
Decide human review by merge danger: one-way doors with a big blast radius get a human trying to break them, small two-way doors can ship.
Run a retro after sessions so every review comment becomes a rule, ideally a deterministic one, and you never write it twice.
07The trust ladder
Trust in agents scales with feedback loops, so give them strict types, fast tests and harsh lint rules that catch mistakes instantly.
Spend high effort where the agent verifies its own work, like clicking through the real app or running performance traces.
Ban escape hatches like TypeScript's any type, because each one weakens the checks your agent relies on downstream.
08Starter stack and deep modules
Tests that mirror single lines of code only create churn; test large deep modules through narrow interfaces so internals can change freely.
A codebase organized into deep modules lets you stop reviewing whole areas, as long as the tests around them are trustworthy.
09Improve codebase architecture, live
Run an architecture-improvement pass on a schedule so small, reversible refactors steadily pull the code back from slop.
10What you still need to learn
When an AI report uses terms you don't know, ask an agent to teach you, because the vocabulary is what lets you prompt precisely.
Engineers still out-ship non-engineers because naming the problem, like a hydration error or layout shift, gets a better result than describing symptoms.
11Hosting and deploying safely
Put data in managed Postgres and use a push-to-deploy host; DevOps has been productized enough that an agent can set it up safely.
Prefer CI/CD over agents deploying by hand, because deterministic pipelines behave the same every time and cost no tokens.
If security matters, add a third security axis to automated code review rather than trusting the default two.
12Open source in the age of slop PRs
Open source contribution now means high-quality issues and reproductions, since maintainers can turn good issues into PRs with agents.
Point your agent at well-run open source repos and ask it to teach you what they get right strategically.
13Advice for new developers
Prove you can program strategically by shipping a small project you clearly cared about, deployed where people can click through it.
Skip memorizing a language; read the classic design books and use an agent to explain the parts you don't follow yet.
14Software factories and the AI video editor
A software factory removes humans from more steps of the loop, but it needs strategic brakes or it becomes a slop cannon.
Do dense planning off-screen at human speed, then dictate the result back to the agent to structure.
15Agents for your life
Knowledge work is harder to automate than coding because there is no type checker for whether a course or script got better.
Glossary
Terms worth knowing.
Tactical programming
Writing code to solve the immediate problem or task, without much regard for how the change affects the codebase long term.
Strategic programming
Making changes with the long-term health of the codebase in mind, so future changes stay easy and safe. The term comes from John Ousterhout's A Philosophy of Software Design.
Smart zone / dumb zone
The idea that a model's decisions degrade as its context window fills up. The early part of the window is the smart zone; past roughly 150k tokens you drift into the dumb zone.
Domain-driven design
A software practice, from Eric Evans' book of the same name, that puts a shared vocabulary between developers and the business at the center of how code is named and structured.
ADR (architectural decision record)
A short markdown file recording an important, hard-to-reverse decision and why it was made, so future developers and agents don't repeat old mistakes.
Worktree
A Git feature that checks out a second copy of the repository on a separate branch, letting several agents work on different tickets at the same time without colliding.
One-way door / two-way door
A one-way door is a change you can't easily undo, like data loss or a mass email. A two-way door can be reverted. The distinction decides how hard a change gets reviewed.
Blast radius
How much damage a change does if it goes wrong. A small blast radius affects one minor feature; a large one can take down the whole product or its users' data.
Deep module
A large chunk of functionality hidden behind a small, simple interface. Tests hit the interface, so everything inside can change freely without breaking tests.
Locality
How likely it is that all the code related to a change lives in one place. High locality means a change touches one or two files instead of ten.
Linter
A program that scans code for bad patterns, wrong imports or banned constructs before it runs, giving fast deterministic feedback.
Semantic diffusion
When a term spreads so widely that it stops meaning its original thing. Vibe coding is the example given.
Software factory
A setup where triggers like new issues or slow-query alerts start agent work automatically, removing humans from as many steps as possible between idea and merge.
CI/CD
Continuous integration and continuous deployment: automated pipelines that test and deploy code every time it is merged, the same way every time.
BDFL
Benevolent dictator for life: the single person who has final say over what goes into an open source project.
Resources
Things they pointed at.
05:08bookA Philosophy of Software Design (John Ousterhout)
“So much of strategic programming is just caring a bit more than the person next to you.”
standalone, quotable, no setup needed→ newsletter pull-quote↗ Tweet quote
17:10
“The agent is just like a developer that you're hiring, and they've been dropped into this world like the guy from Memento, with no memory whatsoever.”
“How do you know that you've built a better course than you did two months ago? There's no way to verify that.”
sharp take on why knowledge work is harder for AI→ newsletter pull-quote↗ Tweet quote
Topic Map
Where the conversation goes.
00:00 – 05:03sparseWho the guest is and why this conversation
05:03 – 11:32denseTactical vs strategic programming
12:35 – 25:18denseGrilling, glossaries and ADRs
25:18 – 34:35denseContext windows and effort settings
34:35 – 44:52denseSpec, tickets, parallel agents and review
44:52 – 52:31denseFeedback loops and trust
52:31 – 1:08:15denseDeep modules and testing
1:08:15 – 1:14:11steadyLearning the strategic vocabulary
1:14:11 – 1:26:38steadyHosting, security and CI/CD
1:26:38 – 1:44:39steadyOpen source and careers
1:44:39 – 2:01:01steadySoftware factories, video tooling and life agents
The Script
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
17px
metaphoranalogy
Would you say that anyone needs to learn a specific programming language right now? Nah. I think that software engineering has always been easier than people think it is, right?
Coding has always been easier than people think. I would have relatives coming to me and say, you must be so smart, you're a coder, you know how all that stuff fits together, you know? But you just learn it.
I do not have a software development background, yet. I can now do it all. If I ask AI to code something, it can code it all day.
But I can feel the limitations, and I think everyone else does. Engineers tend to produce better products than non -engineers right now because they have the domain expertise. They know what to ask for.
They have the right language. Let's take a high school kid. They want to become a software developer.
And in this day and age, it's like, what does that mean? Should they? What I would say is...
First of all, thank you for making the time to do this. No worries. I know you've been busy.
You just launched your new AI crash course and your new skills. Was it 1 .3 you released? That's right.
Yep. That's awesome. Yep.
Congrats on it. I've got a baby coming in late November, and I'm trying to get a course out before then. It's unusual.
I have this hard of a deadline, you know, but the baby is coming. Let's go ahead and get started. So first...
For people who don't know who you are, which I'll kind of give you an intro and just kind of cold start here, you are Matt Pocock. And where I've heard about you was initially your skills, which I think is what most people, that's their gateway drug to Matt Pocock. And everyone uses them.
It's like the de facto thing to use for AI coding now. But everyone just kind of adopts your skills to their own workflow without exactly looking at everything you do, I think. That was kind of my approach.
But then I saw that you had a course and I'm like, I'm going to buy the course. So I bought your course and it's awesome, but it's also challenging, which is why I wanted to have this conversation with you. So real quick, if there's anything I missed about you, can you go ahead and tell the audience who you are and what you're about?
No, I mean, I right now I'm the AI guy, I suppose. I'm the skills guy. I've not always been this person.
Before I was the TypeScript person, actually. So I was a huge, well, I guess. I was an influencer in the TypeScript space.
I was teaching TypeScript courses and that came out from working in open source on very, very, very challenging TypeScript projects like Xstate. I don't know if you've heard of Xstate. It's a tiny little library.
Well, not actually that tiny. But it's about state machines and state charts in TypeScript. And I was trying to make that type safe.
And that was a bloomin' nightmare. Taught me a lot about TypeScript. And I released a course about that.
And went full -time teaching based on that course, which is called Total TypeScript. And that meant that when AI came around, I was looking for my next project, basically. I didn't know what I was going to do.
And AI came. And I realized, ah, this seems interesting. Took me a while to get there because I was pretty skeptical.
I have risen through my skepticism and now, yeah, I'm definitely one of the skills people pushing their skills out there and just trying to write down interesting processes I find and things that work and shipping them as Markdown files. So the reason I'm so excited to talk with you is I think I'm in a place of a lot of other people where I have now been given a magic wand.
I can do whatever I want, but. I need to go to Hogwarts to learn how to use this thing.
And I'm coding like crazy. I'm probably coding as we speak. I'm sure I have an agent doing something.
But I know I'm not doing it well. And I know there's a way to do it well. And where I'm really curious, and I think we're going to end up talking about this at some point, is...
where is coding sitting right now? How much can we give away to the agent and trust it versus how? And I think the term you use is putting agents in a straitjacket, which I love that because it's like they do go off the rails and they do get kind of crazy.
So that's why I'm excited. I think a lot of people in my audience are in the same boat. Like I do not have a software development background, yet I can now do it all.
I don't have to know how to write any code, but I can feel the limitations. And I think everyone else does. Like I made this app, but it's on my computer.
how do I get other people to use it? I don't know. What's the best way to do it?
Let me ask my AI agent. And then they spin up Vercel or they do something random and it's their opinion and it's not good. So what I would love to know, and I guess we can just go and jump into this.
Can you tell me what, actually you tell me, what is the best way to approach software development right now with AI coding? Because like you teach the course, you're the master. How can we step into this?
I don't like this framing. I don't like the framing of me being the guru and you guys being the students. You know what I mean?
Like, that's not what I'm going for at all. Like, you know, I've got a bald head. Right.
I live near Oxford. I've got glasses. That doesn't make me smart.
Doesn't make me smarter than you. Right. And I've got the British accent, too, which helps.
But it helps so much. You have no idea. So much.
Right. So, OK. John Osterhout in his book, A Philosophy of Software Design.
talks about the difference between tactical and strategic programming, right? Tactical versus strategic. Tactical is like the on -the -ground sergeant trying to win the battle.
And the strategic stuff is the general trying to win the war, right? The long view. And whenever, you know, pre -AI, whenever you were working, you would try to be working with a bit of tactical and a bit of strategic all the time, right?
You'd be thinking about, okay, how does this change make a difference in the long term? And also, how does this change, like, how does it accomplish my short -term goals? How does it help me win the battle of today?
Now, AI is really good at tactical programming and terrible at strategic programming. It's just garbage at it. Because of the way it's trained and because of the way, like, you think about an AI in a benchmark, right?
It has a task to fulfill and it's being judged on that specific task. In other words, its focus is 100 % tactical. Whereas you're not going to get better scores on the benchmark by thinking long term and thinking strategically.
And even how would you grade that? It's very difficult to grade that because your strategic mistakes don't usually come back at you until like nine months down the line. So in order to get good at working with AI, you need to get good at the strategic stuff because AI can take the tactical stuff, can eat it for you.
But unless you're careful, it'll just focus on the stuff for now and won't care about the stuff long term. OK, and I do feel that because you're right, like if I if I ask AI to code something, it can code it all day and it will even go in a very bad direction because it just wants to. be that agreeable assistant for me.
But strategy is where it's weak. Would you say that's the biggest weakness you see right now with people jumping into vibe coding? Totally.
I mean, with vibe coding, it's a People have like, there's this term called semantic diffusion where like words stop meaning the thing they originally meant, right? And so vibe coding is a stupid term that now means all sorts of different things.
I take the original definition, which is coding without looking at the code, without thinking about the code, with considering that the code is just like an artifact we don't care about. Now, vibe coding, I don't think has ever worked if it's the only thing you do. If you're only vibe coding, it means that your code will slowly get worse and worse and worse.
Because again, this tactical agent is just thinking about how to solve the current problem. It's not thinking about the long term. And it gets worse and worse and worse.
And because the code is the environment your agent operates in. The environment your agent operates in slowly gets worse. It's like the dog meme of the house burning around them, right?
And the dog is just sort of sat there going, oh, okay, right. The agent just watching the house fall down as they're trying to get their jobs done. So you need someone to keep the house in order.
And that person is probably at the moment you. I do think that agents might be getting better at this, but... they still need to be prompted and told to think about strategy, to care about strategy long -term.
And right now they just don't. So you don't call your course a vibe coding course. And I see why you would steer away from that because that vibe coding definition still has that negative connotation, even though everyone is kind of vibe coding.
If you call it AI -assisted vibe coding, vibe coding has a negative connotation. Do you think that developers right now, all of them should be using AI? Are they foolish to not use AI?
Yeah, it's ridiculous not to use AI because so much of our job before was tactical programming, right? It was just mostly tactical. And you would get these nice relaxing breaks where you could just focus on the tactical stuff, just solve the problem today.
Or I get to kind of just go into a nice flow state and just solve it. But now you can pay less than minimum wage for someone else to do it, right? It's unreasonable for you to be spending your time doing that.
And to be honest, the stuff that always interested me most as a developer, as someone who's interested in increasing velocity on our team, increasing the... Just the quality of code that we're putting out, the quality of features for people, the strategic stuff was always the most interesting stuff for me anyway. So, I mean, I love this new age.
I don't mind that tactical programming is dead. I'm fine with it. You know, just get me thinking about the long view.
Let me just sort of cook and create this beautiful environment for my agents to operate in. I find that stuff really exciting. Okay.
So that's where I want to live. And I'm hoping that... Throughout this conversation, you will give me and the audience a direction on where we can start actually becoming, I guess, becoming software engineers in a way.
Because like right now, I don't see a place where I'm going to actually go out and hire a software engineer. Like I know I can do it myself. I just know there's a few things I'm missing, a few gaps I need to fill in.
Would you say that's where people can be right now? Like can we just become software engineers with a few, like a course that you produce? I think that software engineering has always been Easier than people think it is right coding has always been easier than people think you know You know I would have relatives coming to music your you must be so smart You're a coder you know you know how all that stuff fits together You know but like it's just you just learn it right you know you just spend it like a couple of weeks learning it and you get better over time and you make mistakes and you know and you've got all these tools to help you out and now agents can help you too and it's always been easier than it looks and strategic programming is really just starting to think and care about different things than you're used to caring about and honestly so much of strategic programming is just caring a bit more than the person next to you right is caring about the quality of the thing you're producing of the code the environment or the agent operates in and so so much of like i could always tell which of the my team members
were strategic programmers that had that long -term view because they just cared more. And I think, honestly, if you're just interested in it, if you want to produce better code and you want your agent to do it, then that's the first place to start. It's just, you know, can I swear on this?
I'm not sure. Can I curse? Yeah, yeah, go ahead.
Yeah, okay. You have to give a f***, you know what I mean? You've just got to give a f***.
And so that's the first place to start. Hey time for a coffee break. And this coffee break is sponsored by threat locker and their learning event zero trust world.
Did you know about this? It's a thing and it's kind of sick. It's happening February 17th through 19th, 20 27.
Gosh, has it really already 2027 coming up at zero trust world. You're gonna get access to hands -on hacking labs where you get firsthand experience hacking stuff. And it's okay.
If you're new to hacking, they have every skill level. They're also gonna have main stage talks, breakout sessions, and technical deep dives. These are experts in the field, sharing their knowledge with you.
And speaking of experts, you're gonna be around a bunch of them at this event. Networking is one of the main reasons you wanna go to any type of learning event. You'll be spending time with people who are dealing with the same threats that you are.
And it's people who are practitioners and also executives. You're gonna be making some connections. And those sessions you take can be continuing education credits.
This event has so much and it grows every year. And the after party I heard is something you don't wanna miss. Why?
You gotta go and find out. So now I know you wanna go. I got a link below.
You'll get $200 off or use the code ZTW network. Chuck 27. So go to ZTW .com and use the code ZTW network.
Chuck 27. And thank you to threat locker for sponsoring this podcast. And thank you for taking a coffee break with me.
Okay. So I, I know I didn't. prep you for this?
And you can say no, but would you be able to walk me through an example of how you approach a coding problem? And with your skills. Yeah, totally.
So you want me to screen show? Yeah, that'd be great. I think that's the one thing most people are missing.
Anyone can jump into Codex now and go, make me this thing. And Codex will start doing lots of things using lots of words people have never seen before. They assume it knows what it's doing.
And then they spit out some code and you're like, that's cool. but it's not strategic. And I want people to kind of feel what that feels like.
Yeah, see what that feels like as you do this. Totally. So essentially my skills have evolved into a process, right?
And that process is different depending on the kind of task that I'm trying to do. And I think to answer your question is how am I thinking strategically is that... the strategic thinking is sort of blended into every stage of the process it's not just what am i solving today it's how how is the thing i'm solving going to be um you know going to benefit this project long term so one i'm i'm inside a repo right now which is i have a sort of course video manager repo which is essentially this huge, great big project, which is like a video editor.
It's an AI writer. It is essentially where I produce all of my videos, all of my content. The first place to start is by thinking very, very carefully about the language you use when you're working with agents.
Now, this comes from a software practice called domain -driven design. I've actually literally been filming this part of my course today, so I've got the book next to me. This is Eric Evans' Domain -Driven Design.
It's a bit of a tome, right? But it's the first couple of chapters that talk about the importance of language in development. It's so, so cool.
With a couple of my skills, I've got a skill called Grill with Docs, let's say. And Grill with Docs, what it does is it essentially quizzes you, grills you about the thing that you're about to build. You might have heard of my Grill Me skill, which is like my sort of most famous skill, which is a sort of simpler version of this.
So let's say I go in with Grill with Docs. I'm going to say I need to improve the teleprompter part of my video editor. I think it's just gotten a little bit cluttered, and I think there's some stuff I no longer use in there.
Can you help me out? So I'm just going to give it a vague prompt, basically. I'm going to send this off.
And it's going to do a couple of things. It's going to, first of all, explore the code base and figure out what we're even dealing with and what we're looking at. And it's going to look in this glossary here for stuff that I've defined in terms of what is actually in my application and what the entities are.
So inside here, you can see there's language, there's course structure, right? I have a course and then we have sections inside courses. We have course versions, right?
Because courses change over time and I keep track of them. And then lessons, a single learning unit within a section. We have draft versions, whether it has changes, pending versions.
And all of this stuff is language that I've developed with the agent over time. And that pays off because it means, you know, like how Opus 5 was just garbage to talk to, right? I didn't have that much of a problem with it because it was using the terms in this glossary, right?
It was using terms that I'd already agreed on. It wasn't that verbose because it was talking the language of my application already. And so...
And by the way, are you using Whisperflow right now? I'm using Whisperflow. Yeah.
Okay. Love Whisperflow. I use it all the time.
Yeah. I'm a huge dictation fan. Like if you're not dictating, you're just going way too slow, basically.
So... It's got a subagent sort of like looking in the background and sort of peeking at stuff. And then what it's doing is it's quizzing me about the thing we're about to build.
So this is essentially a grilling session. And it's going, what does the teleprompter read from? When you film today, what's on screen?
Now this stuff is actually, it's probably better if I just wait for the explore to finish because it's kind of asking me stuff that it can figure out in the code. When do I use it? What's on screen at once while you're reading?
What clutter have you noticed? So it's asking me sort of like vague questions that I can provide answers to. And we're sort of slowly getting aligned here.
So this is actually more tactical, right? about how do you get the agent to do what you want it to do? How do you align with the agent before you begin?
Because I think what most people do is they just prompt the agent and they underestimate that there's a communication gap between you and the agent. the agent is just like a developer that you're hiring in order to get the work done and they've been dropped into this world like the guy from memento right with no memory whatsoever just totally blank slate how the hell am i gonna like uh figure out what i need so it immediately dives into the code base and ask in in this case asks you good questions because usually what they'll do is they'll just kind of their way through and they'll just sort of say, oh, yeah, sure, that sounds good.
And I'll build something like what you want. Whereas if you force it to ask you questions, if you try to get over that communication barrier, it's much more effective and you're going to get the result that you want instead of the result that you sort of hope for. And I feel that often I won't be thorough enough in the initial ask and I'll get the result of it.
Actually, just before this meeting, I have I built a tool to help me. do interviews and keep track of my documents. And I noticed the feature.
I didn't ask it to build. It's just there. And I'm like, I kind of like that.
But it wasn't in the original design. It just made the choice for me, which I'm like, I'm not sure if I like that it did that. Exactly.
Yeah. And you have this sort of when you're communicating something, you have this like you kind of know some parts of what you want, but maybe you don't know everything you want. Right.
And maybe it feels clear to you, but it probably isn't that clear to. an agent, right? I've been a teacher for, since I was 21 years old, so that's 14 years I've been a teacher.
I'm quite good at knowing when someone understands what I'm saying or not, right? But even me, I will try to prompt something, I will get complacent and it will just build me something out of scope or like won't build the thing that I wanted. You know what I mean?
So there's always that communication gap. And that's not going to go away if the model gets smarter, right? That's just always there because it's the human that's in the wrong, essentially.
So what we've got now is we've got our grillings come back, and it's quizzing me on all of these sources. And it's saying, OK, it's using my language. So we've got beats and scripts and animatics.
Does the script still auto scroll? Is the status error at the top earning space, et cetera? So this is kind of like, to answer your original question, This is the tactical stuff coming through.
We are fixing the problem, and it's benefiting from this nice glossary that I've been building and working on as I've been working on this app over the last six months. So when you say grill with docs, because I'm very familiar with grill me. I use the grill me skill.
I'm not familiar with grill with docs. So you're saying the docs part is this document that you've created? Yes.
Or is it more docs that we're not seeing? I'll show you. So this is my skills repo.
And if we take a look at the grill me skill. So if we take a look at the grill me skill, it's essentially very short now. It's just call the skill tool with grilling.
So it's going to find you this grilling skill and call it. And this is the grilling part of the grill with docs and grill me. Now what grill with docs looks like is it looks like this.
It says use grilling, but also use this second skill called domain modeling. And domain modeling is this extra docs. It's this glossary and also this extra little line about architectural decision records as well, which is something I've added in here.
ADRs, what they do is they essentially document the stuff that you can't document in code. You know, code is like the current view of all of the decisions that went into building this product, right? But it's only the present.
It doesn't tell you about the past. And if you don't have, like, a small record about the past, then the agent will just continually make the same mistakes over and over. And so these ADRs, this little list of markdown files, essentially, they're quite small.
I don't know if I've got any in this repo, docs .adr. Yeah, here we go. So these ones here, yeah, I've got, what, 31 ADRs.
Just knocking about in the repo. And these essentially just track the important decisions I've made that are hard to reverse, expensive to take back. And it means that the agent just has a lot richer context when it goes to build features and grills me.
OK, so when you're grilling or when it's grilling you, it will go through those. It's instructed to go through those ADRs. It'll go through your glossary.
Is that something that you had to like within that skill? You have it written there. go to the ADRs, go to the glossary.
Is that built into the skill or is that something you added custom for you in this one repo? This is built into the skill and it's also... it's basically in every skill I have it says reference the glossary use the glossary when you're building specifications use it when you're building modules too so the code will actually be using the same language as when I speak about the code right that makes the repo incredibly easy to navigate because everything it just pulls up everything about a course version and the database is aligned the code base the front -end code base is aligned everything just reads properly and It's amazing how many repos you'll see and that I've worked with in my career where the front end will call, let's say we have a refund or something.
The front end will call it a reversal. The email that sends will call it a pending chargeback. And the code will say something completely different, right?
Code will call it a refund in the day space. This sort of language, it just makes the app so much harder to understand, so much harder to work on for you and for the agent. I think we underestimate, again, these communication barriers between you and the agent.
And if you use a shared language, it just feels like the agent is in your brain. You know what I mean? It's just a totally different experience.
So when you, let's say you initially started this project and you're starting from scratch, are you by hand typing out or are you dictating the glossary and telling it what to make? Or are you kind of just giving the agent a general idea and letting it create the glossary for you? So the cool thing about this is you really like if you're thinking okay I want to use this on my project But I don't I don't want to have to go through the whole rigmarole of having to settle this up, right?
That sounds like a pain in the a** What you can do is you can just start grilling and it will mint the terms and quiz you about the right terms as you go And that way you can just sort of slowly build it outwards, just focused on the hotspots that you're actually working on. And then it will say, okay, would you like to update the code so that it matches the glossary?
And slowly, slowly, it's like you're peeling like a little window of sanity in your vibe coded slot base, right? It's just beautiful. And like, as you do it, it just feels like magic because you just realize, God, I understand what's happening in my project now.
Everything reads as if it's like, as if I wrote it. It's hard to describe without actually walking through and doing it and seeing how this benefits you long term. But it's the first strategic thing that I'd recommend folks do.
So for anyone just like watching this now and just like, by the way, what you said, I'm sure someone's like, oh, that sounds awesome. That's what I need right now for this mess I'm coding. What's the quickest way to apply that?
Is it just to go out to your skill repo and just say install or use it, point your agent in that direction? What's the best way to do it? Yep, you just go to aihero .dev slash skills, and I've got essentially a full set of documentation on all of these skills.
And for your next feature, just use Grill with Docs instead of... Let me just get the screenshot so it's open. What was that?
I think that's tracking how much time I spend on Twitter. That's what that was. And guess what, guys?
The verdict is not good. So this is Grill with Docs. So essentially you just install the skill and you get started with the skill.
Okay. Okay, that's pretty simple. Okay.
That's awesome. And by the way, I had the same problem with Twitter. I had to uninstall it from my phone and I'm not coding a thing that will keep me off Twitter but keep me up to date.
So it's a thing. Anyways, it's a bad addiction. So once we've grilled, And we get to that point where the AI understands what we want and it understands the feature.
What's next? So you've really got two choices. It depends if the thing you're building is massive.
If the thing you're building is relatively small and you think it can fit inside the same context window as in the grilling, then you go to the implement skill. And the implement skill is essentially you turn the idea that you've just grilled about into code. When I talk about context window, I'm sort of talking about, I'm not really talking about the 1 million token context window that Opus has.
I consider there to be a smart zone and a dumb zone inside these context windows, which is as you add tokens to the context window, the model makes dumber and dumber decisions. sort of happens slowly as essentially you add tokens and the relationships between the tokens increase quadratically you get this sort of crazy attention degradation where it's having to hold all this information in its head but also listen to the most important stuff in a room that's getting full of people right it's very very difficult task and i don't models are getting better at it but they're still not great at it so i get start getting nervous around 150k tokens right Really?
That's early. Very early, right? Because I want the good stuff, right?
I want my models making smart decisions. And I don't want them... If I ask a follow -up question, I don't want to be paying again for the...
you know, 300 ,000 tokens that I've got in my context window, right? I really like optimizing for small context windows. It keeps things very nimble, keeps things very fast as well, because actually latency increases the more tokens you add, you know?
So for me, it just feels like a general principle of health, keeping my agent in the smart zone and keeping it very, very early. Now, the actual number I think doesn't matter. It's more about the instinct of context paranoia, of keeping things small, keeping the agent.
smart and so if you feel like your thing you're building is going to be bigger than 150 000 tokens then you create a spec so you take the conversation that you've just had in grill with docs and you turn it into a specification and this specification lives either in a markdown file or it lives in your github repo as an issue and then that spec sort of defines where you're going right and from there once you've figured out where you're going And you can kind of go, okay, I can check the final product against this specification.
Then you turn that spec into tickets. And those tickets are the thing that the agent is going to walk through and actually implement one by one. So each ticket is going to be a session.
So each ticket gets implement on it. And then you can use my code review to do a sort of adversarial review on the thing that was built to check that it matches up against that spec. So that is the whole flow, right?
So you've got two choices. Either you go down this spec and tickets route, or you just implement the thing that you grilled. Okay.
Now I have a few questions on context. How do you... What are your tactics to manage...
Sometimes when you're doing a grill me, I can imagine if you have a big code base, it can fill up that context window very quickly. So how... Do you have sub -agents doing it?
I mean, we saw a bit, the sub -agent was doing explorations. Are you using sub -agents to do a lot of that to manage the context? So, yes.
I think there's like four things there, right? The first thing is I think people equate big codebases as big codebases always use more context, right, to explore. I don't think that's true.
I think that hard -to -navigate, poorly organized codebases are harder to explore. And you can have a small code base that's actually hard to navigate, right? And you can have large code bases that are easy to navigate.
And so we can talk later about how to improve navigability of your code base because that's, again, something that we've got here. Well, you said there were four things that you do to manage. There are some tactics to manage context.
Yes. And I can't remember what they were. Oh, no, you compacted your context.
Oh, no, I've compacted at the wrong moment. Damn it. Damn it.
What is going on with me? I've been filming all day and my brain is mush. That's why.
I completely understand. By the way, thank you for taking the time to do this interview after filming all day. I know that you spend all your tokens in those moments.
I did have another question on that. So when you're doing Grill Me, which agent are you using? Are you using Fable?
Are you using Opus 5 .5? What do you trust with that? Yeah, I'm using Opus 5 .5.
And I think... I remember one of the points I wanted to make, which was, I think there's an epidemic of people using too high an effort, right? I think most folks are using way too high effort, either like high, X high or max, whatever.
And what that means is when they do an exploration, the agent, like, if you think about efforts, an effort like producing... let's say a painting, right, or a animation, you know, you get Opus 5 .5 and you maybe want it on like a higher effort because it's going to put more effort in and produce a better painting or produce a better animation.
But if you think about the task of exploration, like, is more effort going to get you a better exploration? You know what I mean? Sure, it might sort of look in more files, but does more files...
really just needs to read the right files, you know, like, and find the right information and give you that information back. So I think people are using way too high efforts to explore their code bases when mostly what I just use for pretty much everything is Opus 5 .5 medium, right? That's just one.
Yeah. So for me, I'm like, and I think there's still massive confusion on this because I think effort, well, I want my AI model to do his best every time. That's what I would expect from a human.
Like, do your best, always. Explore that with 110%. So I keep mine on extra high.
Yeah, and I think that is a big mistake because what you're doing there is you're just telling it to spend tokens on tasks where it doesn't need to spend tokens. And so you end up getting into the dumb zone faster because it's just, and it's going to perform worse because it spent all of its efforts on too early a stage.
Yeah, Opus 5 .5 Medium is what I'm on. And I'm even thinking of dropping that, right, to low, genuinely. Wow.
So are there any moments when you're speccing something out that you feel the need to go to a higher reasoning or a higher model? Not particularly. I think maybe research tasks are an interesting one, especially when it's the web, right?
Because the web is very hard to navigate. It's hard to find information in. And so...
I think probably high X high is a good example of that. When I'm doing performance testing, I think, or maybe debugging where it needs to try lots and lots of different things, I don't want it to stop early. That's when I might go to a higher effort level.
But in practice, usually I just default to medium because I'm thinking about other things and it's not something I'm constantly twiddling or tweaking. Gosh, that is crazy. And I'm probably going to switch to medium after this just to see how it feels.
And I imagine it's a lot faster and more token efficient. That would be fun. And Opus 5 .5 is rapid as well.
It's fast anyway. It's so good. It's a very good model, guys.
It's a really good model. Do you switch back and forth between Anthropic and OpenAI? I've got an Anthropic subscription, and I just haven't really picked up a Codex.
model or like I haven't had a fiddle around with it. I tend not to min max between models that much. I tend to think like I feel I think this is a little bit sort of I like holding contrary opinions, right?
It's fun to hold an opinion that most people don't hold. I tend to think that if you keep the model static and you stop the room around you from burning down, you're going to get much better. results in other words if you fix the environment your agent operates in there's often so much low hanging fruit there that people just underestimate basically and they think ah the new model will fix all my problems right or if i crank it up another effort level then i'll get better results and sometimes you will but you probably just burn a ton of tokens there i think instead of waiting for the next model to fix all your problems why not just fix all your problems that's my take interesting okay so and you're saying that Going through a method like you have, a methodology with your skills, and getting your code base organized, that will probably fix the problems that you're hoping a new agent that's smart enough to just do that for you would do.
Yeah, exactly. And we can touch on a skill that does specifically that, if you like. Yeah.
Or if you want to keep on context. Yeah, let's stay on context before we start a new session. We're just using a bunch of AI stuff.
I'm so AI -pilled, that's how I think now. I'm like, hold on, we need to start a new session because this context is already full. And it's so funny how humans are kind of like that.
So we've done, we can take a large idea. If it's a large idea, it'll put it into tickets, which I imagine at that point, you're having one orchestrator agent divvy that up between sub -agents and they're working that in like work trees. Is that what's happening?
Yeah, exactly. There is a skill here called implement spec, which basically you pass in the spec that you've just created in a new session. And it goes and implements it all with parallel subagents, doing it in work trees, merging it all back together until you get one mega PR ready for you to implement or ready for you to review.
Wow. Okay. I like that.
And I just can't get over the fact that you don't AI max. Do you only have one Anthropic subscription? Because that would surprise me.
Just one. That is crazy. One 200x, yeah.
My goodness. Well, I don't. I'm not token efficient, so I have three of those bad boys, and I also have probably a couple Codex ones too, but I'm just always testing stuff.
But the way you approach this, how simple, it's not exactly simple, but it's just very straightforward. It's refreshing. Yeah, I think that, again, it's this sort of slightly wanting to have a different view than everyone else.
I was just sort of surprised by how well this works. and by how token efficient it is, actually. I think a lot of people would say it's not token efficient, you know, like, sure, implementing sub -agent, like, people have different appetites for how many tokens they want to spend, and that's fine.
But... Yeah, I think just keeping it relatively simple, having a clear, coherent philosophy about the way you want to use AI is important. And I find the smart zone, dumb zone thing has been very helpful in clarifying my decision making and understand.
And this is not my idea, by the way. This is Dex Hawley came up with this. Who should have on?
Actually, he's very, very clever with this stuff. And yeah, just having that as like a guiding principle has been really nice just to clarify my thinking about how to use it. Now with creating those tickets, I'm sure the agent is smart enough to divvy up those tickets to where they're small enough that the sub -agent itself won't get into the dumb zone, correct?
Yes. They're not amazing at it, but yeah, they do a good enough job. And I think of it as having like, if you imagine each ticket should be around 150 ,000 tokens, then that's about right.
Okay. How do you keep track of the tickets it's created? I guess it creates it inside the repo, right?
Yeah, it creates it just as GitHub issues, basically. So you can just go on the GitHub issues. They're tracked as sub -issues underneath each spec.
And so then you can just see the spec being completed as it's being worked through. Okay. So then after it all completes and the work trees are finished and it's merging all the code, what's next?
Yeah. So I do two things. What we do is we go on a code review.
So the implement spec actually calls code review on the code that it's done. And here two things happen. So it does a standards review and a spec review.
So spec review is it compares the code against the original spec and it checks whether it did the right thing, right? Did it build the thing that I wanted or has it gone over scope or under scope? And then with standards, it reads the standards files that you have in your repo.
So let's say you have a codingstandards .md file, and I can pull one up for you if you like. Yeah. These codingstandards .md, this is full of information on how I want my applications to be built.
So I use effect, which is a very cool TypeScript library, and it has a ton of stuff in here for all the information I want to review. So every any in TypeScript is a leak. specific stuff about course versions.
You know, this is really dense stuff. I think a lot of people would have this in their agents .md file, right? I imagine maybe, is this something you do as well?
You know, what's funny is I don't even pay attention to that. I trust the agent with it. Yep, right.
You just think, okay, I'm not worrying about these steering files or not worrying about what the agent is reading or thinking about the agent's environment. yeah i'm more like i i definitely tell my agent i want to have a standard but i want you to create the standard for me and i don't want to have to look at it got you yeah i think that's um i would probably describe that as vibe coding i think i think that's vibe coding right because it's not really caring about you know i want you to be good but i don't care how you're good you know yeah it's it's like that that meme it was some kind of tick tock i saw a while back it's just like Think faster, deep research, do it, do just be smart, make it happen.
Like it's just making wishes and, and, um, yeah. Ultra, ultra think, you know, ultra think, ultra think, do it right until you're, um, you know, or I'll fire you or something, you know what I mean? Just like, so that is what we're, that's, that's the second axis that we're comparing on.
Is it built correct according to my coding standards? Because I sort of assume that when my agents implement something, they're going to get stuff wrong. And for me, that's fine because I've got a code review step that goes again and does this adversarial review and checks it against the standards.
And so once that's done, then I'll go and human review, right? And I don't do this for everything. I do it for some things for very important work.
And if I was being professionally paid to build an application, then I would probably do it for most things. Really? So describe to me what the human review looks like.
Is it you actually in the code looking at... I mean, is it line by line? Is it just like an overview or using an agent to help you look at it?
Yeah, so what I do is I have a PR skill, which is I think in reference skills here, which basically builds me a PR body that does a couple of things. This is a nice skill, by the way, because you can just install this and the agent will call it. It's unlike my other skills.
It's just sort of improved your PRs. And it does a few things. It gives me like a sort of visual summary of the thing that's being built.
it gives me evidence that it's working so it has to take like a screenshot or have like execution based evidence and then it tells me the merge danger what is the danger of me merging this whether it's a one -way door or a two -way door so if it's a one -way door then it's a door we can't walk back through right we merge it and then there's data loss or like you know an email gets sent out to 100 000 people where if it's a two -way door then we can merge it and come back And then also it's blast radius too.
So whether if it goes wrong, it's going to be a big bad thing or a small thing. And so I use this merge danger as basically like, do I need to review this? Because if it's a small blast radius and it's a two -way door, then I probably don't need to review it that hard.
You know what I mean? I can probably just test it in production. So if it's a one -way door with a big blast radius, you better believe I'm reviewing not necessarily line by line, but I'm looking at...
I'm trying to break it. So that's kind of what I do for human review. And I like human review because it's not only a chance to make sure the stuff I'm shipping is high quality, but I then take the stuff that I'm saying in human review and I pull that back into my coding standards, into my environments that my agent's in, right?
So I start thinking, how can I improve my environment so I never have to write this comment again? And that's what I'm thinking when I'm reviewing. No, is that like an automatic thing?
So when you're, so let's say you're reviewing it, human review, you're looking at the code. What is your method for giving? So I imagine you're not going to fix it yourself.
You'd probably just tell the agent to fix it. So it has that context. Is it automatic to take what you're saying and add that to your coding standards?
Or do you have to tell it that to actually initiate the skill? There is a skill for that. So we have a retro skill.
So this looks back on a session, looks back on a PR and suggests improvements based on what the agent did, right? So it's another look back on the agent session, on your review comments, on everything that just happened. Let's improve it next time.
Okay, okay. And I'm sure we'll get to this because it's more of an architectural and design question, but like your coding standards. Is that just something that you, as a software engineer with that background, just know what you want to have?
And how would someone who doesn't have that background even go about doing that? Yeah. This is where the strategic stuff comes in, right?
Because this is, I don't want this to happen next time. I'm thinking about the long view. And you need to be able to recognize what good code bases and bad code bases look like.
And that's really hard.
Do you want to dive into that now? Because we totally can.
Let's wait a moment. I'm like, yes, but also no. Because I know we probably want to keep going through the different skills we have as we're adding features.
Because I do have a question. Once we review it and it looks good, let's say it's a two -way door. Blast radius is small.
It's an easy thing to deploy. What's next? Do you deploy it?
Does your agent deploy it? What's your approach? I mean...
What's next is I press merge or I say, okay, if there's a bunch of them, I just say, okay, guys, merge all these five PRs or something and you figure out the right order, you merge them in together. And then we're kind of done, right? You know, we've done the loop and now we're back to wanting to ship some more features, either do another grilling session or just like plane ship something.
So that is the loop. We've covered the entirety of the main flow where you grill with something, you either then to spec and tickets, implement it, code review, and then retro so that it's better next time. So I think that that's probably the part where most people stop because anyone can go, Matt Pocock's a smart dude.
He's got skills that will make my agent smarter. I'm just going to copy paste and go. But the next part we're going to talk about is the actual thing, the thinking that needs us to understand architecture, design, and a little bit of coding.
So walk me through that. Okay. So before we think about.
we've got the tactical stuff kind of done, right? We now know how day -to -day we're going to be shipping code. But how do you then take the long view and think, how do I improve this over time and get my agents to a point where I can trust them?
And the point of this is that... Lauren Tan, who goes by Potato, who works at Grokbot, who's kind of famous now for shipping like 2 ,500 PRs a month or something. She talks about how do you essentially scale up your trust in your agents?
Because the more you can trust your agents, you sort of walk up this trust ladder. If you don't really trust your agents, then you're never going to be able to scale beyond like four agents at once, you know, because you're going to be watching them, everything they do, checking all of their work, human reviewing every part of it.
And even if you walk through this whole flow, you're still going to be doing that. Right. So her question is, if you can go up the trust ladder and you can trust maybe some PRs to be merged without human review, you know, the two way doors.
How are you going to get your agents to show you evidence that they've done their work? How are you going to constrain them so that they do better work every time? And if you've got essentially a really great environment for those agents to operate in, then you can start trusting them to levels where you can have hundreds of agents operating at the same time comfortably and trust that they're doing good work.
And so all of this is about trust. What you need is a code base that's a good code base. And if you read like the classics that I've got on my shelf, they tend to converge around one definition of a good code base, which is a code base that's easy to make changes in.
A code base where if you make a change, it's probably not going to screw anything up. And that's really hard to do, actually, because I don't know if it's necessarily hard to do, but you have to care about that in order to make your code base easy to change.
What that means for me is that you need a ton of feedback loops so that the agent can grade its own work and can verify that what it has done is good. And you asked me before about effort. I would say that...
The point where I really want a high effort level is when it's verifying its own work, when it's checking that what it's done actually functions and it's clicking around a real application where it's running like performance traces on its code to make sure it hasn't degraded anything. That's where I want it because if I can trust that the agent has done that work and has verified its own stuff and that the code base is easy to make changes in, then that is stuff I can just like merge without thinking about it.
I can just go, oh yeah, that looks good, merge it. I don't have to worry about the code. Are you there?
Do you trust your agents right now? I'm in an environment where I can, right? I can, absolutely, because I'm not working on a medical application.
I'm not working on a finance application. My domain is I'm building personal software for myself, right? So I have been in those situations before.
I've worked on banking finance applications. And in those situations, I would not, or I would be looking very, very carefully.
at certainly the one -way doors, right? I would probably be asking my agents to verify everything from a security point of view really heavily as well. I would be burning a lot of tokens on verification so that I could get to the point where I trusted them.
So what are some examples of feedback loops you have right now that build your trust in your agents? Great question. So there are a ton of them.
By the way, I have this really cool resource. It's called the AI Coding Dictionary. I've not really talked about it very much.
And it's essentially every term that I use is somewhere on this graph, right? So we have model providers, context windows, and all of them have a definition associated with them. We have somewhere in here will be automated checks.
And so automated checks are essentially what I'm about to talk about, which is strict typing. you know a fast test suite and a bunch of other stuff so the main automated this is this is free by the way you just you know access it there's a github repo if you want to point your agents at it very useful for just like this is kind of like the glossary that i showed you earlier but applied to ai coding right this is where i got obsessed with it so all of my resources use the same language now automated checks essentially you want strong types So you want to be using things like TypeScript and not JavaScript because types tell your agent very, very quickly that it's made a mistake.
Then you want tests. So you want actual runtime tests that verify that it's doing what it thinks it should be doing at runtime and that you don't break anything. You want things like linters as well that, for those who don't know, a linter is...
essentially a program that checks your code to make sure that it looks right, that you can do all sorts of things with it, like check that you're using the right functions, check that you're importing from the right modules, that you're not using bad patterns that the authors of the framework know are bad and are trying to protect you against.
So linters are very important too. And what you want is to get the most value out of those you can. So you want to use like types really effectively.
This is why I have, let's say, this little rule inside my coding standards here says don't use the any type in TypeScript because I know as a developer that if you use the any type, it's actually a cheat. So it's trying to cheat around the type system and it's going to reduce the quality of my type checking further down.
So I am trying to improve those automated checks so that they work better and catch more errors next time. Interesting, okay. All these automated checks, the linter, the CI, the testing, is this something that anyone can easily implement?
What's the process for that? You just have to care, right? You just tell your agent about them.
You say, I want to constrain my application more. I want you to find harsh lint rules. And this is something that retro does as well.
So what retro will do is it will say, okay, we found like a comment in the PR. And it tries to find ways that it can prevent it deterministically with an automated check next time. So every time it sort of ratchets up the power of these automated checks, every time you run retro, it will find something.
And so eventually you get to the point where the agent is so boxed in, they're in their straitjacket, right? They literally cannot make a mistake. What you worry about then is of course improving the speed of those checks to make sure that you can still ship work but agents are really good at that as well.
So all of these individual parts agents are good at. They're good at finding lint rules to constrain themselves. They're good at making those tests run really quickly.
It's just you have to care. You have to tell them to do that. And if they don't know to do it then they won't care about it either.
Is that a skill you have? I'm thinking about building a skill in this direction, actually. I do have a skill which we can talk about in a minute that sort of prompts the agent to improve its own environment a bit.
But essentially, this is retro. That's what retro does, is the more often you run retro, the more sessions you run it on, the better your code base is going to be. Okay.
Okay. So let's say we've got the average Joe. they have an idea for an application.
Let's say it's a website that, I don't know, makes ordering coffee and tea better and easier. You can find the cheapest tea and coffee in the world. It doesn't matter what the idea is.
How would they approach that idea? So obviously, cloud code is your favorite. I think it's very good too.
It's one of the best harnesses. So cloud code, would you say they need something like VS Code or can they just stay in the terminal? What's your preference on that?
I would recommend VS Code. It's nice to be able to easily access the code. I don't always use it to read code or think about code.
I would say that what my stack is, and it's very approachable, like it's free, is GitHub, is VS Code, Git, obviously, and Cloud Code or whatever agent you like. It doesn't really matter what the agent is, to be honest. As long as it can use skills and it can use my skills, then you're good to go.
harness out there is, you know, can use skills because they're a common standard. Okay. So it doesn't matter what the harness is really.
And I think most of them use sub -agents, like everything's kind of the same now. They've all copied each other. So once they have that set up, what are the things they have?
So when we get to the strategic thinking, when they're building their app, obviously they'll start talking with their AI. Do you have skills that ensure that they do this? Is that what the grill me is for?
Like, is that does that ensure they have that strategic thinking they need or is there something more they need to know? So it is baked into different parts of the process. So let's say when you call to spec, it's going to ask you sort of what shape of modules you want, which might be a little too technical for some people, but just sort of keeps you in the code a little bit.
I don't necessarily think you need like. like full awareness all the time of everything that's happening, right? You can, people talk about like software factories and they talk about a dark factory, right?
Just kind of like, you know, you have your factory, your agents are working in there and you never turn the lights on, right? You have no idea what's going on. I don't like the dark factory.
I do think you need to look in there and care, but you can have little dimmer switches, right? For different rooms and stuff. There are rooms in your code base that you never really need to walk into, right?
You can just sort of have a little peek in there. And I think one thing we should talk about before we go here is that there is another skill that I think really helps. I sort of talked earlier about wanting to get the most out of your automated checks.
And one thing that I think people really underestimate is that when you write tests for your application, so specifically the things that test your application at runtime, that actually run parts of your code and verify that they work, Tests are really hard. They're really hard to get right.
And this has been true before AI. The reason tests are hard is it's really easy to write a test that really doesn't verify very much. You look at a line of code and it just says the amount of characters you can have in a tweet is 240 or something.
I can't remember what the actual number is, but let's say 240. And then you'll have a test that just checks that number. and checks that that number is 240, right?
The test is just repeating the implementation. It's not actually like what you want from your test is you want a big chunk of code that I'm going to call a module, and you want that module to basically be hidden from the tests. The stuff inside the module is hidden from the tests because that means it can just test it via a very narrow little...
sort of entry points to that module. And it means that you can change everything inside that module without needing to alter the tests. Because if I like have a single line of code that's tied to a single test, then every time I change that line of code or I restructure it or do something with it, then that test will have to change as well.
And it's just sort of endless crappy churn. Whereas if I have these big, what I call deep, what John Osterhout calls deep modules, then you can actually sort of turn the lights off inside that module. You don't have to care about what's in there.
As long as the tests happen from outside and everything sort of seems to work, then it's probably going to be fine. And I have a skill that can basically turn your code base into this shape. It's called improved code base architecture.
So you can fire this off at any repo and it will give you a visual report. on the bad stuff that it finds, and it will suggest improvements for making it better. And then you can take the result of that and turn that into a spec and tickets and run it through the normal loop.
Ah, so using this skill, it obviously will show you the holes and the errors and all the bad things inside your code base, but it'll also help you build those tests. Are we talking about unit tests? Is that the same thing?
Yeah, unit tests, exactly. Okay, which I, as I was learning coding before AI, I never got to that point, but I heard it was always the worst part of coding, just creating really good tests. Totally.
It's really, really hard, but agents make it easy to type out tests, but they don't necessarily think about structuring the code base so that it's easy to change, right? So that those tests actually catch things that are important. It's very easy to write tiny, tiny little tests when what you need are tests that actually, like, take out huge chunks of the application and put it all under test at once so that's what this skill does okay okay and that i think that would be a massive help for a lot of people i mean i'm going to run this skill on a few of my code bases right after this interview just to see what happens um okay so we we've we can produce good tests and that's a good place for us to go to um to make sure our code base is is healthy and it's um And you said we could be able to dim the lights or turn lights off on a module as long as the tests are really good, right?
Yep, that's right. You have this feeling of, I don't know, when you work on a code base that's been vibe -coded, I always feel this terror, right? This, I don't know what's in there, and I think it's bad.
And what you start to feel is, right, How will I know if something goes wrong, right? Will I have to eventually look at this code and review this code?
How am I going to get to that? Whereas if you have this, if you have improved code base architecture and your code base is starting to be put into these big, beautiful modules, you can go, okay, I understand the shape of this and my app understands how to use these modules as well, how to leverage them and how to call their functions and things like this.
you have these lovely big chunks of stuff that you don't need to review that's all using your domain language or using your glossary terms and stuff so you know exactly what's going on in there and you realize okay i can start thinking of my apps code base in these big chunks right in like okay that's the bit where we do authentication that's the bit where we do um transcripts that's the bit where like the whole video editor interaction lives in one big chunk and it's all testable So once you have that, you start realizing, okay, cool.
I don't need to worry that much about this code base because it's structured in a way where it's more robust. Hey, real quick, let's take a coffee break. You and me.
This coffee break is sponsored by CoverOn. Big exact. We talk a lot about AI on this channel.
I mean, everyone's talking a lot about AI. And while AI has made you pretty good at your job, you know who else it has made pretty good at their jobs? Scammers.
They can clone a voice, write a perfect phishing email, spin up a fake bank website in minutes. actually maybe seconds. And they can do that at scale.
And they're not just coming after you. They're coming after your family, your parents, your kids, everybody. And that's where cover on comes in.
Their family plan watches the dark web for you because that's where all that stuff ends up. So they're watching the dark web and they will alert you when you or anyone in your family has like a new account set up. And if something does slip through, you got real people you can contact for help.
They'll help you clean it up. Plus up to $2 million for identity theft and up to a hundred grand for online fraud, cyber extortion and home title fraud. One scam can cost you.
and they're moving fast, protect yourself right now. Get 71 % off this Black Friday deal. Coveron .com forward slash chuckcoveron.
That's coveron .com forward slash chuckcoveron. Okay, so do you think we're at a place now where we can get to the, how do you structure your code base? How do you structure your project?
Yes, I think like there's no one like genius way to do it. Essentially, you use these deep modules. you if you have what i i really want to actually just run this and um because it's very very easy to run right because we can just literally go claude and then i'm just going to run improve code base architecture and what this does is it's going to pull in the code base design skill which sort of if you want to just read that that's got all of my uh like how i like to design code bases in there and it's finding the hotspots so it's going to find my recent commits and look inside the repo to explore that and explore what might need to be improved and then we're going to get a html report back of the things that it wants to change so once this is done i'll be able to like exactly tell you how to do it there is i think what people need to get is there's no one perfect code base right all you can do is just improve it over time and get it better over time.
And what I do is I have a cron job that runs on each of my repos where once a day it will run improve code -based architecture and it will give me a top suggestion back and then it will go and make a pull request based on improving that thing. And because those are two -way doors, then I don't really need to worry about it.
I just know that it's making decent improvements and slowly pulling the code into where it needs to be. And so... You're essentially just trying to fight back against the wave of slop, right?
Because agents will make tactical mistakes and strategic mistakes, and you just need to constantly be pushing it to this position where they use these deep modules. When you have these cron jobs running, do you trust it enough just to fix things without your approval? Yeah.
What I do is I do sampling, right? I will just sort of check in every five PRs or so. And this works for my application because it's, again, not finance, not medical.
So I will do sampling. I will just sort of check that the thing is running. I will occasionally retro it.
So just say, retro, look at the last five of these and see if they're actually reasonable, see if they make sense, see how we can improve them next time. So, yeah, basically I do. Interesting.
Now, do you? Do you use the Cloud Code app on your phone? Do you do things remotely?
Do you text it, use Telegram? What's your stack for that? So I have been in all sorts of...
I've tried all sorts of setups, basically. My recent one is I literally tore it down today, which I built a custom Discord bot for myself, kind of like my own Hermes, basically. I put it up in the cloud.
I was texting for my phone and stuff. I think phones are cigarettes, basically. Phones are poison.
So I tried to make my phone as boring as possible. So I'd sort of go away from that. I didn't want to do it anymore.
So right now, as of now, I'm just using Claude code and using GitHub actions for some scheduled jobs as well. So this is an open source repo. GitHub actions allow you to run Cron jobs.
And I just run Claude inside the GitHub action and it updates and creates pull requests for me. And agents know GitHub actions really well. It's actually free for open source repositories.
And so it's a really good option for me. Okay. I love what you said, that phones are poison.
And what's funny is I have it to where I can use iMessage for my agent right now. And I was just thinking this morning as I was on my walk, I'm like, this is so fun. Just being able to text my agent to do whatever I want it to do.
And it does feel freeing. So I find it interesting that you're like the opposite. You're like, no, phones are poison.
I want to make my phone as boring as possible. Why? Well, Because I had that too, right?
I would have Whisper Flow running on my phone. I would be chatting to it on Discord as I sort of went, you know, for my walk and stuff. But this is more, this is just a personal thing, really.
Like, I think we'll look back on this generation and think of it in the way that we thought about cigarettes in the 50s, you know? Like, you know, we're giving, you know, like, phones to 13 -year -old kids and stuff, you know? Like, it's miserable.
Like, these kids, like, it's just...
You know, life is there to be lived. You know, I just I don't having this constant access to information and especially the social side of it as well. And this constant hankering of people wanting to pull you in and scroll, basically just scroll, scroll, scroll, scroll a bit more.
I just hate it. I'm with you there. I like the idea of keeping my agents close to me, being able to text them.
But like I said, just this past weekend, I deleted Twitter. I deleted TikTok and YouTube and everything off my phone because it was sucking that time away. And it's so funny when you take that stuff away, you notice you kind of have that itch like, I'm bored for these three seconds.
I have to wait. Let me go check. No, I can't.
I don't have it. And you feel that urge start to like, and it's nasty. I've even gone further, which is I've removed the web browser and I've removed the email from my phone as well, which is apparently something you can do on Android.
And so it is as boring as possible. And what I noticed was I would hope that or hoped that when I did that, I would get all this extra inspiration. I'd feel so much more energy.
I would like become this whole new person. And then nothing happened. Right.
essentially nothing, nothing rushed in to fill that gap. You know what I mean? It turns out that, yeah, these things are really addictive and you, you know, I've been trying to read more books, I guess, but nothing is as good or feels as good as checking TikTok, right?
Or checking Twitter. So you just have to kind of live with that. It's like you're talking like a former addict or something like, I'm always a drug addict and there's nothing better than that, but it's ruining my life.
Absolutely. So what is this? Well, this is the report that we just got back from improved code -based architecture.
So what we are essentially seeing here is a before and after of each of the modules. what we have is a problem submit and duplicate course contain 150 near identical lines so there's some duplicated code inside our repo and what it's basically proposing is instead of this horrible sort of setup where we have a bunch of shallow thin modules on the left hand side we say okay all of these different parts of our application are all going to rely on a single deep module and we can then test that module and so this one feels like a like a total win for me So this is cool, but honestly, I think, and I speak for myself as well as probably my audience and many people watching.
I'm like, I see that I found stuff and it's very interesting, but also I probably have to ask AI about a lot of these things. Like, what does that mean? What does that mean?
Why did you show me that? Well, what is this? Absolutely.
Absolutely. So this kind of maybe goes into another section of our conversation, but I'll ask it now and maybe you can tell me where we should ask or answer it.
people who are wanting to get into this world and what it means to be a software developer, what is that knowledge that you still need to learn? What's the thing? Because obviously it's not, it's not tactical coding anymore, but I look at this and I'm like, I don't know this.
So I probably should learn this. What, what is it? So there's a skill for that.
So do you have a t -shirt with that? Like that should be a t -shirt. I have a teach skill, right?
I'm a teacher. I was bored on the tube one day and I thought, why don't I just write down everything that I know about teaching and turn that into a skill? And what I would say is that let's say you have improved code -based architecture.
You get a report and you look at the report and you think, I can't read that. I don't understand what's going on there. I don't understand the terms it's using.
I've thought very carefully about the terms that I've used in the skill so that they point out to books and they point out to well -known methodologies. For instance, I've got a big pile of books next to me. This is one of the books that I recommend called Working Effectively with Legacy Code by Michael Feathers.
Wonderful, wonderful book. Teaches a lot of this stuff. And what teach can do is you basically just say, You pass it the report you've just received.
You say, I don't understand this. Teach me about this. And it goes and finds a workspace on its own.
It keeps track of what you know. And then it quizzes you and builds these, again, HTML lessons so that you learn things slowly. So it's essentially a build your own course for whatever you want.
So it taught me how to do a Rubik's Cube, actually. So this was how I tested it. And I can now do a Rubik's Cube, which is wonderful.
But you can also, I've tried it out by teaching strategic programming as well. And it's really, really good at it. Because my, these reviews here, they're basically like they use a common language.
So they talk about locality. So you probably have no idea what locality is. A lot of software developers don't know what locality is.
It means how, if I make a change to a piece of code, how likely am I that all of the changes are going to be in the same location? In other words. are all of the related pieces of code together in one file or in a couple of files.
So locality is good. You want locality. And yeah, like you just basically give it this and you say, I want to understand this, teach me about it, and then it's going to do a great job.
So you're saying, by the way, that teach me skill is such a fun idea. I love that. So you're saying that it's important for us to know all this.
At what point is it not important for us to know this? Like, are the models going to get so good to where it doesn't matter? So it comes down to caring again, right?
And all you need to know is you need to know enough strategic programming to be dangerous and to know that it's effective and that improves things over time, right? Because... the models are still not going to care about it right i maybe there's a world in which they are prompted to do this and i've been thinking about this myself as well but for now it seems to me that people who are really good at something are going to be able to get models to be really good at that thing right so engineers tend to produce better products than non -engineers right now Because they have the domain expertise, they know what to ask for.
They have the right language, right? You know, you might say, what's a good example? Let's say that you have a problem in your application where you have some sort of flashing stuff coming in when you load the app, right?
It sort of shifts around in terms of layouts, right? And I know what would cause that. I would say, fix the hydration error on the homepage, right?
or fix the layout shift on the home page whereas someone else might need to say okay the when i load the layout page or the home page um it loads and then it shifts everything around it just allows me to communicate more effectively and prompt the agent in a better way because i know the domain language because i'm an expert in it and i've seen that kind of thing before and i can communicate what i want much much better that does make sense um And I'm sure you've seen this term, the bitter pill engineering idea.
Bitter lesson? Bitter lesson, yeah. I forget.
It's been used interchangeably. But the idea that all the architecture, all the straight jacketing we're doing will be replaced by better, smarter models. And we just need to kind of get out of their way because they might know better than us how to architecture a given thing.
But you're saying that it's the caring that matters the most. It's the caring, right? And I always say, like, I'm not a pundit, right?
I have no idea where things are going. You know, I'm just in, like, rural Oxfordshire. I'm not in Silicon Valley.
I don't know anything. My job is not to predict the future. My job is to tell you how to get these things working now, and they're very effective now.
But, yeah, I think there probably is some sort of harness, some way of doing this that... will make the agents care, right? Because you can get the agents to care.
It's just you need to care yourself. So that's something I've been thinking about and working on too, is how do you get agents to think strategically? And I think that probably is something you can do in the harness, just not, it doesn't have to be in the model.
But for now, this is what I consider the state -of -the -art way to do this. Okay. And then once you've I think what a lot of people don't understand how to do right now is they don't understand the architecture or rather the infrastructure, like where to actually put this app they're building.
Do they use a database? What type of database? Do they put it on AWS or Google Cloud?
Like all these things are big things that normally are done by DevOps engineers or software engineers who have that background. Now, how should someone approach that? especially to someone who only has done vibe coding.
Let's say they're watching your course and they're coming into it. Do they need to learn all that stuff or can they just ask the agent to do it? And do you have skills that help with that?
I don't have skills that help with that. And I would say we don't really need skills to help with that. What's a little bit of a history lesson, which is over the last 10 or 20 years, DevOps has moved from...
being something that you needed a degree in to figure out how to do, like self -hosting your own Linux box or something and doing really complicated stuff, stuff that could really go wrong and really bite you, into a set of products that have been built by Google Cloud, by AWS, especially serverless platforms, if you want to do some Googling, that really take a lot of that stress away from you.
And as resulted in products like Vercel, for instance, who I used to work for, Netlify and all this stuff that basically take all the pain away from you, right? And what you really need to do is just figure out where your data is going to live, whether it's going to be in a database.
You should probably just use like a Postgres database if you don't know what you're doing. And once you've got that, and there are many services that will host and manage your Postgres database for you, PlanetScale, Superbase. Once you've got your data, you can basically deploy your app anywhere, you know, just Netlify, Vercel, whatever you want, and it'll be fine.
This has gotten so much easier that an agent really can just do it for you and it will be good. So as long as you're running some security checks and you're doing relatively regular security reviews, which agents are also pretty good at, you'll be fine. It's a lot easier, just like coding.
It's a lot easier than most people think it is. Yeah, and I can speak to that. I'm very much a home labber.
I've got proxmox and all kinds of things in my environment. But as soon as I felt comfortable enough to give the keys over to my agent, I can simply text him and go, hey, I want to deploy this app, make a new VM, make it happen, just do all the things, give it an IP address, give it a domain name, and boom, it just does it.
And it's magic. It feels like magic. But I did need to have a bit of that domain expertise to even know how to tell it to do that and to give it those keys.
But you're saying that a lot of the DevOps things, so things like databases and Vercel hosting now, they've abstracted that away to where you don't really have to know it. Your agent can do it. Is that where you host your stuff?
Is it Vercel? Yeah, I'm on Vercel.
I would say just use the thing that's easiest to use, basically, especially when you're getting started. I wouldn't worry about, you know, like going to AWS manually or figuring out all that stuff. There's stuff you can, like there's foot guns there, right?
Which is why these companies like Vercel exist. So like they just sort of make that easier and you don't have to worry about, you know, manually deploying or anything. You just push to your GitHub repo and it deploys for you.
It really is very, very simple. You can even use something like GitHub pages or something. And all of this stuff, Even when it comes to domains and DNS records and things like that, all of that is really easy for agents to figure out information on.
And often these sites like Cloudflare as well.
Again, it's just something you have to care about. You just basically say to your agent, make this deploy as good as possible. Give me everything I might need in terms of security, distributed denial of service attacks, DDoS attacks.
Try to make it secure and just give me the basic level of what I need. And agents are really good at that. time for a quick coffee break i want to talk to you about something you've probably seen one of these before a video of a celebrity or a ceo giving away some kind of scammy thing actually i just saw that with the roadster announcement you know elon's flying car i was trying to find the live stream and i actually just found a scammer live stream that had a fake elon sitting there talking to me, it looked fake, but for many people it would not.
And there were like 30 ,000 people watching that stream and he was pumping some crypto scam, but it looks stinking real voice face, everything. Like I was showing it to my family. They're like, oh, are you sure?
It's not him. It's a deep fake scammers are literally using them all the time. And in a survey of 7 ,000 people, that's their biggest concern.
Deep fake scams are so bad now. And one in seven have claimed they've fallen for one. So here's some red flags you should look for when the lips are a little bit off from the words, if they have some weird blinking and an offer that's like too urgent or too true.
I mean, come on. Most of these announcements we see from founders are like pre -order in three months. They're not like buy this right now.
So when you see that pause, don't click, don't send any money, don't share it. And then. check it because Bitdefender has a free app called RealCheck.
You can drop in a link from TikTok, YouTube, Instagram, Facebook, X, upload the clip. And it tells you if the video looks altered and how likely it is to be a scam. And honestly, not every AI video is a scam.
RealCheck will help you discern which ones are. Now you may be thinking, well, Chuck, I don't get fooled by these scams. Well, what about your mom?
and your grandma and your dad and your brother and your sister. Do they know enough to spot these things? Give them a tool that will help them do it.
Run suspicious clips through Bitdefender RealCheck to see if it's a deep fake meant to scam you before you click or send money or share it. Links in the description. Bitdefend .me forward slash RealCheckNC.
That's Bitdefend .me forward slash RealCheckNC. And it's free. So like, what do you have to lose?
Just try it. What do you do for security? Because I know that that's one thing everyone...
Actually, most people don't think about what they should is all these AI agents are just making things very quickly. They want to make it work. And that's the primary objective.
But then security is a whole realm. Do you have skills for security? No, I'm the wrong person to ask because the projects I work on are personal software, essentially.
And I have a platform team that handles my site where we might have some security concerns. So, no, I don't. Don't think about it that much.
But I know it's a concern. What I would recommend is if you are interested in security is to add something to code review so that it does, instead of just two axis, instead of just thinking about standards and spec, you also do a security review. And many harnesses have a built -in security review skill.
So that's what I recommend. Have you hit any security issues with anything you've coded? Specifically with AI.
No, but I haven't had those cons. I've definitely screwed up in many, many ways, but I don't think I've ever, especially pre -AI, but I don't think I've ever caused a security issue. I don't think.
That's good. That's good. We can trust you now.
No, I'm just kidding. Now with your skills.
How do you make sure, what's your method for making sure the skills are good? Because, you know, anyone can make a skill, but how do you test them? How do you make sure they're effective?
Yeah, a lot of people ask me this, and especially how do you like eval your skills, right? So for those folks who never heard of evals, they're basically benchmarks that you run against some system by pushing a bunch of data through the system and then seeing what happens. So you might eval a coding agent.
by saying, okay, is it better with this model or this model by running a benchmark of something? You get it to do the same task every time and you see if it succeeds or fails. So that could be a 0 % score or 100 % score on the eval.
Now, skills are incredibly hard to eval. Incredibly hard and incredibly expensive to eval. How would I eval my grill with docks or something?
How would you eval... domain modeling? How do I know if I've chosen the right term?
A lot of these skills are just sort of augmenting what the model does anyway and what the harness does anyway, trying to kind of get out of the harness's way and just give it, basically trying to get it to care about different things. How do you eval that? And if essentially I have the fortune of having a ton of data going through my skills anyway, right?
Because I have, you know, you know millions of users all around the world who are using the skills and reporting issues back i've got very active discord where people talk about skills all the time so i'm sort of running evals except my evals are people you know they're just so what i do is i listen to them and i react and i try to improve my skills all the time um but i don't burn tokens trying to build evals that might or might not give me good data so that's the decision i've chosen If I was in a model lab and I was being paid to do this and I had unlimited tokens, I would build evals.
Okay. And which I think even those evals wouldn't be as helpful as what you're experiencing now with people. Exactly.
If I didn't, like, I'm very grateful to my users because they do report stuff really, really effectively. And that's the actual stuff I need, right? If I try to synthetically create that with evals, it would be worse.
So when they do give you feedback. How do you go about implementing that? How do you verify it?
How do you test it? I have an agent that looks at it and looks at the issue. It tries to recreate that issue.
So it runs it in like a dummy repo. And then it says, yes, this seems good. And then it creates a PR.
So I have that on a schedule running every day, I think, because I get quite high volume of issues. But I also get a ton of crap issues as well. So that agent also filters stuff out.
Did we ever talk about what happens after the PR? Do you deploy the changes yourself to your infrastructure or do you let the agent do it? I use a continuous integration platform.
So on, let's say, Vercel, for instance, it will just, whenever GitHub detects a change to the default branch, it will just ping off to Vercel and then Vercel will redeploy my application, will build it, will put it in the cloud. that's something that's been solved for yonks right before agents and so it's just not something i think about and it's and it's always deterministic so it always happens the same way every time and i don't burn any tokens to do it right it just does it by itself that's yeah and i didn't think about that like yeah cicd has already largely solved that problem i think so many people don't even think about cicd they just think maybe uh they think about just having their agents log in and deploy the code itself um so you would say that people should Think about CICD and implement that.
Oh, yeah. I mean, if you have a choice between determinism and non -determinism, why would you ever pick non -determinism, right? Why would you do it?
Like, why bother? Because determinism works the same way every time, right? Your agents can always go in and fix if things go wrong.
It costs you no tokens, and it just costs you CPU cycles or something. You know, it just costs you this resource, which is incredibly cheap. So always pick determinism.
And this is true, too, when you're doing, like, a retro or something. When you think... Should I change this into like a rule where my agent is going to code review against it?
Or should I change it into a lint rule that is deterministic and runs the same way every time? Always, always choose determinism when you have a choice. And most people don't realize they have that choice.
Yeah, I don't think they do. And even just me recently, I just started doing some CICD. I host, I self -host Git, Git -T or Gidia.
I'm not even sure how to say it. And I use some of their CI and action functions, which is great. Now, for you right now, I take it you mainly probably do mostly teaching, but you do also do some of your own development.
Do you have any other projects that you're constantly working on that you're maybe going to sell or make public, make it a SaaS? Nah, I'm not a big SaaS guy, basically. I'm in this teaching thing as a lifestyle choice, basically.
I have a young son, and I want to spend as much time with him as possible. Being at home here, getting to choose my own projects, choose what I care about, and choose my own time is incredibly important to me. As you can imagine, I've been approached by a fair few organizations saying, we would love to hire you and come work for us, but I'm basically unhirable at the moment.
Until my... my son and i've got another one coming uh go to school then like and i have a maybe a bit more time you know until that happens i'm going to stick exactly where i am so i'm not doing the sass thing but i am a bit of an open source boy so i have a project called sandcastle which is a kind of effort to um do agent orchestration in typescript using essentially allows you to drop in any agent and run them in sandboxes.
So like in Docker sandboxes or Podman or Vercel or actually whatever you want. And then I've got another one which is sort of falling a bit by the wayside at the moment, which is called Everlight. And Everlight is actually an evals framework, amazingly.
So I've worked on this like a year and a bit ago. I think it's really fallen by the wayside, to be honest. But essentially, it's like a visual evals framework that you can run locally.
And so, yeah, it's it's open source for me. And kind of that's that's where the bulk of my work is going. And also into.
this video editor that I've been building and that I use for everything. I'm fascinated by that because I also build lots of video editing workflows for myself, you know, because we make videos. We should chat about that in a separate discussion because I'm sure we can go get very nerdy very quickly and not many people are going to care about it.
Oh yeah. Now with your open source, I love that you called yourself an open source boy. And I think for many people, they would love to not only start their own open source projects, but contribute to open source.
but they don't know how. And they never could until really now. But how do you do that well?
Like, could they take your skills and start to contribute or even start their own open source project? And what would they need to like, what would they need to be aware of that isn't already like what we haven't already discussed? Yeah, I mean, what I'd say is that this was a different conversation about 18 months ago than it is now.
because now we are living in a world where it's incredibly easy for anyone to just ping up a slot PR to a repo, right? To just say, I want this change made, here's a slot PR. And very popular projects are now closing their PRs to new contributors.
So for instance, if we look at Hono .js, I think they just did this the other day, Hono is one of the most popular kind of like... server frameworks in JavaScript, and you can no longer create PRs against Hono unless you're a collaborator, right? Because of just the level.
And I've closed PRs on my skills, for instance, as well, because I want my skills to be very carefully curated. And it's a rather popular repo. It's nearly up to 300 ,000 stars now.
And that is, you know, people just pinging crap into it all the time. And I just didn't want to be notified about it. So open source now is all about providing issues and providing good evidence.
right because it's so cheap now to turn those issues into pull requests and to just get an agent to work through them and to figure it out so the higher quality repros that you can do is great and like that will teach you a lot about how to communicate to people working open source and how to communicate to other developers and how to communicate to ai but Of course, you can build your own thing, right?
And the cool thing about having all of this code available to you is that your agent can read it. Your agent can just take a look inside this code base. You can say, forward slash, teach me about the Hono code base.
What can I learn based on what I've told you about my experience already? What is cool about this code base? What are they getting right strategically?
How are they deploying their infrastructure? What infrastructure do they have? What do they need?
all of this stuff you should just look at these cool projects and just point your agent at them and say teach me about this and then like again open source is a lot um it's a lot simpler than people think it is you know it's really just like deploying something to uh npm or something you know so npm you can just use uh like a um a change set manager to just sort of do versioning and things like this.
Very, very easy to learn, very simple to get right. You can just do it within GitHub Actions, make it nice and secure, and you're good to go, and agents know how to do it. So if you want to build an open source piece of software, you can, and agents will enable you to do it.
It's interesting that you mentioned that people are closing their PRs to only verify contributors. Do you think that they'll go even further and just cut off contributors? Because you can just have AI do everything for you now.
Why even bother with the messiness of humans? I think some projects do need teams behind them. You still need teams of people making decisions and someone managing triage and contributing and that kind of stuff.
But I think, yeah, essentially what you're saying when you cut off PRs is only these five, 12, two people can contribute to this project using their agents. And I think that's sort of always been true. Like anyone who's done any long -term open source will have encountered the...
Everyone who's contributed to open source... has met a bdfl a benevolent dictator for life which is the person in charge of the open source project who is wants it to succeed but they also only want their point of view in the repo and they are the one who is the the gatekeeper for stuff entering it so Contributing to open source has always been weird, always been political, always been a hard thing to do well.
That's why I have a huge amount of respect for people that I know who are amazing at it and who contribute to all sorts of open source projects. But I think it's harder to get into than it's ever been.
So what do you think is the... And actually, I think we're going to get into some of the last things I wanted to ask you here. But I think before...
The benefit of contributing to open source was that it would be a free place to get experience and show off what you can do in order to help you land a job and actually make money from that. Because obviously open source, there's no money in that. Is that still valuable to be able to show off what you can do now that it's mostly AI doing that?
AI has eaten tactical programming, but it can't do strategic programming. So you need to show evidence that you can program strategically. You need to show evidence that you can manipulate infrastructure that you can build a beautiful repository right that you can improve your agent's environment that you can steer it properly and i think that strategic programming skill is actually more valuable than it's ever been you know i genuinely think that i think it's like because now you have a fleet of agents tactical agents that can just go off and do whatever you want them to do and so if you're able to raise their level and care about improving them then It's honestly so much better than when you were working with humans.
I've always cared about this, right? Even when I was working with people, I've always wanted to make the project better and make us all work faster. I've always cared about that, I think, more than maybe my colleagues had who just wanted to clock in and go home.
So not all of them, of course, but just being able to have those skills has just felt like a ridiculous uplift in the amount I'm able to trust my agents. I think that is an incredibly valuable skill because that can take you from a, you know, 5x agent user into a 100x, 200x agent user, you know. Okay.
So for someone who, let's take a high school kid. They want to become a software developer. And in this day and age, it's like, what does that mean?
What advice would you give them? How could they become a software developer? Should they?
And what do they have to learn? So I'm not, again, I'm not, I can't predict the future. I've not been on the job market for youngs, right, since 2020 or something maybe.
So I don't know what it's like out there. So your mileage may vary in this advice. But what I would say is demonstrate that you care and that you care about the strategic stuff.
you might not even need like great expertise in it. But if you're using language like deep modules, if you're thinking about, okay, how am I going to improve things next time? If you're constantly thinking about not only improving the code, but improving the next code that comes through, then I think that is enough to start, and you demonstrate that through projects, right?
That you're actually shipping stuff that may be assisting you in your own job hunts. You know, that's the classic one. you build a little CRM for yourself or something and you make it as beautiful as you can, you make it as performance as you can, you really care about it, and you deploy it somewhere that people can click through it and actually view it and fiddle around with it and go, wow, okay, this kid knows what he's doing and he's doing it all with agents.
That is what I'd say. And it is very possible to demonstrate that and demonstrate huge amounts of work. You just have to care about the strategic layer.
Okay. So you would say, Would you say that anyone needs to learn a specific programming language right now?
Nah. Programming languages, that's tactical stuff, right? Actually getting good at a programming language, it's important to be able to read it to understand what module boundaries are, but you can just get agents to just teach you about it and to teach you about it in pseudocode.
What I would recommend is start... start working through some of these classic books, you know, like a Michael Feathers book or something, actually buy the book and start, obviously it's going to be pretty dense, right? There's going to be words in there that you don't understand, but then use the teach skill or something.
So it pulls that down information for you. So you can start actually going, oh, I see how I would apply this in real life. You know, this book is from 2003 and I think it was wrapped in plastic and on my like bookshelf or something.
And I just thought, oh, I've been prompting agents to design good codebases, but this guy was doing that before there were even agents, right? He's been writing what a good codebase is.
And so I pulled it down, and I just couldn't believe what I was reading because it just felt so relevant to what I was looking at. And he was able to put into words the stuff that I sort of felt. And I just started plugging that stuff into agents, and that's how Improved Codebase Architecture came about.
So care about it. Read books about it. Use the teach skill to get your way there.
And that's how to do it. What other books should people read? Philosophy of Software Design by John Osterhout.
Pragmatic Programmer by David Thomas and Andrew Hunt. Michael Feathers is good. Refactoring by, what's his name, Martin Fowler.
Extreme Programming by Kent Beck. those are probably the big ones wow and so would you say that people should read these to become effective at ai programming yeah if you actually do the reading it puts you way ahead of everybody else in the field genuinely like um so yeah definitely i think i do think that experience is the only way to you know get there but it really helps When you have the experience and you have the words to explain that experience, because I think like just understanding the language and the categories of things and like, OK, this is more important than this is.
That's what strategic programming is all about. It's all about finding balance between lots of different things that are all competing. You know, like, okay, I want my agents to ship features really well, and so I'm going to write 4 ,000 tests for them.
Oh, but those 4 ,000 tests are actually slowing things down because they actually make the agent run really slowly. How do we winnow those down? How do we put those in a CI so that it can rely on them later?
How do we shard them so that they've got, you know, running in four different nodes instead of just one? you know all of that that might sound confusing and complicated but all of those are just words and you just learn the words and then you prompt the agent and then the agent can care about it so what you're doing is you're just giving the agent permission to care about the strategic layer and you're learning the words in order to do that okay and that makes sense and it makes me wonder and i know you don't you're not in the job market you can't really look into the future but how effective right now are computer science degrees I would ask how effective have they ever been, actually?
Because I didn't get a CS degree, right? I got a drama degree. You know, I was a voice coach for six years and I just taught myself this stuff in like 2015 and started moving over.
So, yeah, I don't know. I don't know if they've ever been that effective. The issue is people who care and people who don't.
That's interesting. And I'm sure many people will be watching this and they're in a computer science degree.
What would you tell them? Care. You know, just care about the stuff you're shipping, care about your users, care about the other devs on your team, even though they're agents, and care to learn more, care to be curious, and yeah, enjoy, because it is fun.
And this is so fascinating, because it's fun to talk with someone who, I didn't know you didn't have a comp sci degree, because that's even more interesting. You're self -taught. And I think that's been the best way to learn anything for a while now.
AI has made that even more obvious.
For you, if you were to hire a developer to join your team, should you need one? Actually, that's probably a two -part question. What would make you need one?
Why would you need one? And what would you look for? What skills would they need to have that would impress you?
I mean, I hate to harp on this again, but it is that strategic thinking, right? It is the environment. It is all of that and using the right language and caring about the right things.
For me, I do think there are bottlenecks all over the process still, right? This is not just, you can't just infinitely accelerate and just like. One person can ship 2 ,500 PRs a month, but if you have two people doing that, that's still 5 ,000 PRs a month.
That's still doubling the amount of work as long as both of them are enabled and both of them can work on cool stuff. And honestly, the... The thing that really differentiates good products these days is not the amount of tokens that have been spent on them, but the amount of attention that has been given to the product, the amount of use that it's had, and the smart things people observe about the product in order to make it better.
So it's the testing and the QA and the planning and figuring out strategically what needs to be done next to improve the product and noticing problems and essentially product management and product design. And that is something you need a human for, you know, or you need a human at least to tell an agent to care about it, you know.
So, yeah, that's what I'd say is that people now are all about directing agents and using their attention and their care to make the agents care about the right things. Okay. I said the word care about a thousand times and I wasn't planning to.
I think I've just found a new word that I like. Yeah, yeah. And I think that is important because as we've kind of noticed through this, you can't just go in and just write code.
You can't just go in and go, I'm going to make this thing work. You have to really care about what it's going to be. You have to think about...
It's like thinking about a child. What do I want you to be? What can I expose you to?
What are the standards I'm going to set for you? What's the rules for life I'm going to give you? That's interesting.
Now I'm curious, how much of this strategic programming, how to use AI agents, and I'm going to allow you to show your course a bit here because I love your course. How much of that is in the courses you create right now? Yeah, that's like I'm saying all this stuff matters.
As I'm saying it, I'm trying to find ways to teach people about it. I'm saying software fundamentals still matter, and so people's follow -up is, okay, what software fundamentals? How do I learn them?
How do I apply them? And so I've got a course coming out in November. I haven't released this information, but it's coming out November 9th.
It's like a two -week cohort course that's just timed perfectly so that I go on paternity leave right afterwards. And so it's stressing trying to get it done. And all of that is about using my skills, sure, but skills are essentially just pointers to a process.
And the process I'm trying to teach is that you need to understand how to improve your agent's environment. It's essentially the end point is where you create a software factory that does a lot of this stuff for you. So a lot of the caring is sort of automated in these cron jobs, in these routines that automatically try to make curations and its environment better.
As I was saying, it's sort of like, Ideally, the harness would improve the environment as you go, and that's what we're trying to get to by the end of the course. Interesting.
And I just heard the term software factory for the first time last week, and I know it's been thrown around for a minute. What exactly is that? Well, on the AI coding dictionary, we have a definition for a software factory.
There is a definition for that. And essentially, I won't pull it up, but... What it does is, if you imagine that producing work involves several points at which the human needs to interact with the work as it's being done.
So it might be that you need to tell the agent to do something. That's the first point. Then you might do a grilling session and you might approve the plan.
That's the next point. Then it might go all the way through until the work needs to be merged. Now, what a software factory does is it tries to take the human out of more points in that process.
So it might be that instead of you needing to initiate the work, it might be that there's some trigger that happens. Let's say a new issue is created on your repo that pings down to an agent and the agent decides whether to initiate some work based on that. Or it might be there's a slow database warning in your...
in your service that checks your database that again goes down to an agent the agent decides whether to proceed with that and then you might have ones where okay that might not even need a human in there all the way up to human review but maybe you don't do human review for things maybe you use that one -way two -way door thing and actually only one -way doors need a human review and the agent just goes and merges it itself so The software factory is basically just taking humans out of more points so that you can accelerate the development of software more.
And the issue with that is that the more you accelerate it, you need breaks as well. You need to be able to pull the software factory back so it doesn't just become a slob cannon, which again is where the strategic layer, the thinking about the environment really makes a big difference. Okay.
And that sounds great, but it also sounds really hard to do. Yeah. Yeah, I think so.
I think you're basically having to plug together a bunch of services in order to do it. I think as the tools and the harnesses improve and make those things easier, then... You know how people are obsessed with loop engineering a few months ago or whatever?
This is basically loop engineering. What you're doing is... A loop is one way to do it, where you just schedule the agent to just like pull the GitHub issues and just sort of initiate work on its own.
But you can have multiple different ways of doing it, essentially. So this is just loop engineering by another name, essentially. And it's very exciting.
And it means that you can churn out a ton of work and really improve the quality of what you're working on. And that's what Software Factory is. Interesting.
Okay. And then you, I'm curious about how you approach, so like for like right now, If I went to your course website, is that a platform or is that something you guys designed?
Yeah, it's something we built from scratch. I work with a team called badass .dev and they work with a few course creators and they build this course platform from scratch, basically. Oh, dang, that's pretty cool.
So you're not running the AI coding for that right now? No. Do you have any projects you're working on right now?
Apart from the open source stuff? No. Just Sandcastle, just trying to get this course built and trying to, yeah, ship this video editor that's always, always constantly improving.
Okay, okay, I can't resist. Tell me about this video editor. I've seen you post about it a few times and I'm like, that looks interesting.
Okay, he's having it create demos for him. before he records it. That's interesting.
So what is this thing? Oh, it's outrageous. So it is outrageous.
What it is, it came down to this idea that I hate editing video. And so what I wanted was I wanted to, as I recorded, I would essentially leave a little gap and then it would cut that and that would go be transcribed. automatically and just appear as a clip in the finished editor essentially so there's no editing so i edit as i record and from there that developed into okay i need like a i use a lot of diagrams in my setup i'm going to build like a custom diagram playground as well that links to the clip so I can see from the clip which diagram I was viewing so I can go back in time and just edit it and stuff and then maybe I oh yeah Elgato teleprompter that's quite nice maybe I need a teleprompter too um oh maybe I can customize the teleprompter based on what I'm seeing oh maybe I can get an agent to write scripts maybe I can get an agent to write animatics as well Opus 5 .5 looks really good with actually building out um custom videos for you so why don't I watch the video I'm about to record before I record the video that seems good and then from there I you know
and just grows and grows and grows and like and now like every edit i make it automatically sync to my course platform um i have you know version history i have linting i have ai writing like it's just a ridiculous wonderful project and it's it's open source too so it's in course video manager oh wow 2500 commits deep now um and it's just all in here you know so if you want to use this as Like I don't take pull requests and like I don't even take issues on this, actually.
But if you want to just have a look at it, it's there. Yeah, I'll take a look. And I do something not quite as streamlined as you, but I do have it to where when I finish recording, my AI has been trained enough back and forth with me to where it knows how to edit me.
It knows what kind of cuts I like. It knows that when I because I've been doing this for a long time. I used to edit my own stuff.
Now I have editors. So like when I say something. I'll freeze so I can remember what to say again, and we don't have to do a weird cut, and then I'll keep talking.
So it knows to watch for that. So when I finish recording, I'll just tell, hey, go to the place where we have our files recorded, offload it, ingest it. And what that means is it will transcribe it.
It'll use the Gemini Video API to watch it. So we know exactly what's happening. And then it will start to actually produce an edit and it will edit it to like a very clean draft.
We coded our own frame to IO. I'll watch it to make sure the story's right and tight. And then it will export an XML for Adobe Premiere where my editors just open up and there's the draft ready to go and they can make it look pretty.
It's very cool, but I want to get to more where you are. That sounds very interesting. It's just so great.
It's so great. Yeah, I mean, it's there if you want to have a play around with it. I just wanted a project that I knew that I would constantly use and constantly update.
So it's just always in the back of my head that, OK, I'm going to work on this, going to work on this. I've got agents right now playing around with it, improving the environments that the code operates in. This is my main playground, really, for trying out my ideas.
Now, you mentioned that you have AI scripting. How do you feel about that?
speak word for word from what the AI scripts for you? No, I mean, I did a drama degree, right? So like I'm good at improvising words and I just use the script as like a, it has like commands that I can copy and paste in order to set up the environments and stuff and like go to the right commit in the history for the project I'm working on.
So mostly it's just a guide essentially. So I'm always improvising what I'm saying. Interesting.
So, I mean, I create courses as well. So I'm very curious about your process. Is AI pretty much part of every process of your course creation from like ideation to episode or video management and all that?
So, well, one thing I do and one thing that I actually recommend to anyone who has an audience, who has people pinging them all the time, is I have a... a personal wiki which is for my audience and every three hours or so i have a cron job that looks at all my tweets people have sent me all my mentions all my discord messages all my slack messages my gmail basically every sort of input i can find and then writes that or sort of ingests it into a carpathy style llm wiki so that means that i can go into this it's just a repository it's private obviously And I can just ask it, hey, what is my audience interested in, right?
What questions do they have about these skills? And from there, I can create learning goals from that. So things that people are interested in learning.
From there, I have to actually get off AI. I can't use it for this next bit because it's too hard. I print off those learning goals onto these little bits of card, right?
Oh, I love it. I like rearrange them. physically in space until I realize, okay, this feels like a lesson.
This feels like a lesson. This feels like a lesson. That can take a couple of hours actually, just off computer, off AI, just for one section.
And my courses often have eight sections. So, and from there I go back and plug that back in. So it turns into lessons and turns into videos.
And then I start writing the scripts and getting agents to dry run it as well. I create these animatics and then I'm ready to record. So, you know, it can be months planning, of course.
It really can. It's very, very slow. That's fascinating.
So, I mean, two things that just jump out at me. First, the wiki idea is such a good idea. I'll probably steal that.
The second one is that you have to go offline to do this kind of thinking. I've got pen and paper, and I find the best thinking happens with this. And I do use AI quite a bit to bounce ideas off of and try to form an idea more.
It's those very human moments where I'm disconnected that the magic happens. So I'm curious, how often do you go back to that kind of thing where you're disconnected? It's mostly on tasks where there is a lot of information and I need to keep it all in my head at the same time.
Because if I'm like planning, if I am like talking to an agent and we're having a back and forth about a plan and maybe shaping the plan in certain ways, a ton of stuff right in response to me and i lose so much time by trying to get over that communication gap right constantly trying to go okay what do you mean and and in doing that i lose context right i lose information that i could if i just slowed down and i just kept everything in my head at a speed that works for me literally you know writing stuff on pen and paper it sort of slows you down so that you can consider things at the right speed it's It's me paced, right?
It's paced at a human speed. Whereas agents just work too fast for that. So for specifically planning, that's what I have to do.
And I've not found a better way to do that with agents. It's interesting you mentioned it that way. When you're talking to an agent, you're like trying to bounce an idea off of it or just like try to do anything.
It does feel too fast. And it'll say a bunch of stuff and it's so dense. You're like...
Or too slow as well, right? Or you're just waiting for a response for like two minutes and you think, oh, I'll just check Twitter. And then half an hour later, you go, oh, what did the agent say?
Right. Oh, yeah. Yeah.
Like seven o 'clock at night and you're like, oh, I did ask that question earlier tonight and I don't even care about that anymore. Exactly. You just got to get out of the conversation.
Just keep it in your own head for a while. And then I usually just brain dump it back to the agent. Just, you know, use whisper flow, dictate out a ton of stuff.
And then it just pulls it into. its own structure and its own data and then i just usually chuck the cards away because i don't don't need to go back to paper there because for small edits for little stuff once you've got the main structure ai is really good do you use ai in a way to where because i know we've been looking at coding agents and and getting the coding agents to know your repos and your code but do you use anything for your life do you have like a hermes agent or any kind of agent that like has the context of you um So yes, this audience wiki is sort of a bit of a blend of like me plus my audience, right?
Because those things are often closer than you think. And so, yeah, that repo sort of serves as like my home base for all of my agents. And so I have like a daily standup with my agents where I say, okay, what tasks need to be done today?
I have like a pixel watch that I dictate stuff into, and that goes on my to -do, like an agent triage to -do list. And the agent then pulls it into, It was like deciding every minute of the day for me, like scheduling out all that stuff.
And I have like a little thing in front of me. This is called a terminal. Oh, yeah.
I have one of those. Yeah. They're kind of – they're all right.
I don't know if they're any good. But I've got two of them actually, one downstairs and one upstairs. And I was just sort of ignoring it.
So I'm still trying to find the right interface for the agent to help me run my life. But it's a me problem, not an agent problem. The agents are really good at it.
I find the same thing. I'm trying to figure out the best way to help me schedule my day or help me do this and help me work with my team. I don't think anyone's quite solved it yet well.
It's really hard to figure out. No, and I think it's actually much harder than coding. Actually using it for knowledge work is much harder than coding because coding is so verifiable.
You can use these automated checks to actually make sure that the thing you're doing is correct, that you have high quality on the output. with knowledge work, with planning a course, how do you know that you've built a better course than you did two months ago? It's really hard.
There's no way to verify that. How do you know that this video script is better than your previous one? So yeah, I think knowledge work is really difficult, way harder than people think it is.
And I think we're going to be hitting that over the next few years as we try to force AI into all of these holes. And I love that this entire call, everything coding related, you're like, it's easier than you think. With knowledge work, you're like, it's actually harder than people realize.
Yeah, maybe I'm wrong there today. Maybe it's just because I know coding more than knowledge work. I think you're right, though.
It's that very, very distinct point you made is that coding is verifiable and that you can learn it and it's deterministic and you can test it. You know it's good. It works.
That's why agents are amazing at math problems and can solve math problems way better than humans because every part of it is verifiable. All the inputs are text and all the outputs are text. But it's also why they still can't draw hands, right?
like it's just different inputs than they're used to, right? They can't write a convincing tweet that we don't know is AI, right? Because that is just a much harder job because it's so much less verifiable.
And it's something I don't even really care that AI can do. I think there's human things that I want to keep human that just make us who we are. And communication is like one of those things that make us who we are.
Absolutely. Yeah. I play a ton of board games.
Keep agents out of my board games. I don't want to see any AI art. I don't want to hear any AI music.
I don't want to, I get a thousand AI tweets a day and I hate them. Yeah. Yeah.
I think we feel the same way. Yes, absolutely. Well, we are near the, actually, no, we are at the end of time.
Tell me where can people. find you and with your courses, where should they start? Go to AIhero .dev.
It's the place. Find my skills, install my skills, enjoy them. If you have any questions, go to the Discord, find me on Twitter.
You know, I'm pretty active in all those places. And I'm on YouTube as well. I recently crossed 400 ,000 subscribers, which I'm very pleased about.
And yeah, look me up. I'm there. Do you have like a wait list for your new course?
There's a newsletter. But yeah, sign up and I'll see you there. Cool.
Awesome. Well, Matt, this has been so fun and informative and you give me so many ideas. Thank you for taking the time to not only teach me, but my audience.
This has been very, very cool. Appreciate it, pal. Hey, made it to the end of the podcast.
At the end of my videos, I like to pray for you, my audience. Kind of weird. I know, but just go with it.
You're not here for, I mean, you are here for a reason. It's not an accident. And yeah, let's go with it.
I put it at the end so you have the option to just, you know, duck out. The information has already been given. But if you want to chill out and stay here with me right now, I'd like to pray for you and your life and your family.
Ready? One, two, three, pray. God, I thank you for the person on the other side of this screen, this camera.
Thank you for them, who they are. And I thank you that they stuck to the end of this and that they are learning about coding.
and just all this AI stuff, which can be stressful, but also is very life -giving if we approach it the right way. So God, I ask in your name that you shift any anxiety, any stress about AI or the future about jobs. I ask that you would just flip that switch in their brain to be joyful and excited and let them be at peace about it and just give them energy and give them ideas and a path forward.
If they're looking for a job right now, I pray you provide that job. If they're looking to start a company, give them that idea and the steps forward. Bless them right now.
Bless them and their family. Watch over their families.
Give them peace and joy in their lives right now and help them to take the knowledge from this video and apply it well. I ask this in your name, Jesus. Amen.
Thank you for letting me do that. I'll catch you guys next time.
The Hook
The bait, then the rug-pull.
The cold open puts the most provocative answer first: nobody needs to learn a specific programming language right now. The next two hours explain what you should learn instead, and it turns out to be the part of engineering the models are still worst at.
Add Modern Creator as a preferred source and Google shows you more of our breakdowns in Search, Top Stories, and AI Overviews. It only changes what you see, and you can undo it in your Google settings anytime.
Add to Preferred SourcesOpens your Google source preferences with us pre-loaded. Tick the box and you're done.
NetworkChuck tests Jev, a decision model from TypeSafe that only answers yes/no, multiple choice or a score, and finds it beats ChatGPT on speed and cost for background automation.
A 38-minute build-along that turns one free Checkmk server into a single pane of glass over Linux boxes, switches, Docker containers, Proxmox VMs, a Windows gaming PC, a NAS, Plex, Home Assistant, and a fleet of AI agents.
One host hires a Claude Code CEO, three Hermes agents, a Codex security lead, and two local-model workers into a single AI company, then hands them a real network mystery to solve.
NetworkChuck installs DHH's Omarchy, an opinionated Arch Linux build, in under two minutes, then hands an AI agent the keys to rice, theme, and troubleshoot the entire desktop while he watches.