34:08Nate Herk | AI Automation · Interview
How to Actually Choose the Right AI Agent
A 34-minute in-person conversation about why the thing wrapped around the model matters more than which model you picked.
September 11thThe argument in one line.
The limit on agent output is not model intelligence but the communication gap between you and the agent, so the most valuable skill is making what you care about explicit and fixing the environment the agent works in.
Read if. Skip if.
READ IF YOU ARE…
- A non-technical professional who is building real things with coding agents and keeps hitting results that are almost right but not what you pictured.
- A developer who writes less code by hand every month and wants to know which of your old fundamentals still pay off.
- A consultant or freelancer who can now ship a prototype in a weekend and is unsure how to prove expertise when everyone else can too.
- Someone running agents daily who is deciding between spec files, skills, and just fixing the codebase itself.
- An employee weighing whether to start an AI consulting business or push AI adoption inside the job you already have.
SKIP IF…
- You want model benchmarks or a head-to-head between Claude Code, Codex, and Copilot. The guest explicitly refuses to rank harnesses.
- You are looking for a step-by-step build tutorial. This is a conversation about judgment, not a walkthrough.
- You already run a mature agent setup with tests, browser access, and reusable skills. Most of this will be confirmation.
The full version, fast.
Agents got genuinely good around the Opus 4.5 release, and the guest stopped writing code by hand entirely. His claim is that no model will ever read your mind, so results come from closing the communication gap: tell the agent what you care about, and fix the codebase it operates in, because a neglected codebase produces neglected output. He failed for months at spec-driven development, watching the code get worse while the specs looked fine, and recovered by focusing back on the code itself. The practical skills are flushing out your own unknown unknowns, building precise shared language for the project, and climbing a trust ladder instead of handing agents everything at once.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Who's talking.
01:04hostNate Herk
01:04guestMatt Pocock
Where the time goes.
00:00 – 01:04
01 · Cold open
Three teaser answers stacked back to back: communicators win, tell agents what you care about, and watch for unknown unknowns.
01:04 – 04:04
02 · Ten years by hand, then the moment it clicked
A long-time skeptic explains the threshold where he stopped writing code himself, and why his interest in making colleagues better transferred straight onto agents.
04:04 – 06:52
03 · Tell agents what you care about
The communication barrier between human and agent never closes. The fix is caring about the environment the agent works in and raising the floor the way engineering teams have for twenty years.
06:52 – 08:30
04 · Don't trust what the agent teaches you
Hallucinations are rarer but not gone. Use a teach session to get the vocabulary for a topic, then read the primary source yourself.
08:30 – 09:48
05 · Sponsor break
Host-read sponsor segment for a VPS host, pitched around putting what you build somewhere other than your laptop.
09:48 – 15:07
06 · Where the opportunities are
Teaching and adoption are wide open, and the harder opening is turning knowledge work into processes agents can actually run, because knowledge work is far harder to verify than code.
15:07 – 20:27
07 · Unknown unknowns and the Grill Me skill
Nobody knows exactly what they want. A skill that makes the agent interview you relentlessly pulls the missing requirements out of your head before any code exists.
20:27 – 20:55
08 · Free kit break
Host-read mid-roll for a free AI operating system template built on his own four-part framework.
20:55 – 22:43
09 · How to stand out when anyone can ship in a day
Clients want someone who understands what they care about. A fast prototype that proves you understood their business beats a portfolio, and speed is a pricing question, not a credibility problem.
22:43 – 25:53
10 · Communication is the differentiator
The top performers have always been the clearest communicators. Domain-driven design and its ubiquitous language give a project one vocabulary shared by code, client, and agent.
25:53 – 29:27
11 · The spec-driven development failure
Months of trying to run a real project from specs produced steadily worse code. The recovery came from treating the codebase itself as the environment worth maintaining.
29:27 – 34:56
12 · Models, harnesses, and effort levels
Models will never read your mind, so a process layer stays. Running at medium or low effort buys speed and token efficiency, and giving an agent browser access beat any model upgrade.
34:56 – 39:30
13 · Climb the trust ladder
Pick a harness, start in an empty folder with an idea you actually want, watch the agent like a hawk, then add skills, more agents, and connections to your real systems.
39:30 – 41:36
14 · Give it your boring work
A direct rebuttal to the advice that you should hand new models your most ambitious task. Hand them the menial work you can measure and cheaply correct.
41:36 – 44:50
15 · Software factories and open models
What he is excited about: agents that start and improve work on their own, better ways to organize many agents, and open models reaching current frontier quality.
Lines worth screenshotting.
- No model will ever be good enough to read your mind, so some process layer over the communication gap is permanent, not temporary.
- An agent working in a neglected codebase produces neglected output, because the code is the environment it operates in.
- The right question is not how to prompt better but how you would raise the floor for the worst coder in a hundred-person team.
- Software engineering books written twenty years ago read like they were written last week, because raising the floor is exactly what agents need.
- Spec-driven development failed over months of real use: the specs stayed clean while the code got steadily worse.
- Good code means code that is easy to change without breaking something nobody thought about, which is also what makes agents effective in it.
- You should not trust what an agent teaches you. Use it to get the language for a topic, then go read the primary source.
- Nobody knows exactly what they want, so the job is flushing out unknown unknowns before they become silent assumptions in the build.
- If you do not mention security or deployment, the agent concludes they do not matter and quietly invents something.
- The best developers have always been the best communicators, and the clarity of their thinking shows up in the product they ship.
- Giving the project one shared vocabulary across code, database, interface, and client conversations removes a whole class of misalignment.
- Most people run their models at too high an effort level, losing token efficiency and speed for output their tests would have caught anyway.
- Giving an agent browser access to click through and screenshot its own work raised output quality more than upgrading the model would have.
- The advice to hand your most ambitious task to a new model is backwards. Hand it the boring work you can verify and correct.
- You learn the brake before the accelerator, which is why you watch an agent like a hawk before you run several at once.
- Shipping a fast prototype that proves you understood someone's business reads as trust, more than a portfolio of impressive builds does.
- Verification is why agents are strong at code and weak at knowledge work, and building verifiable processes for knowledge work is the open opportunity.
The gap between you and the agent never closes
WHAT TO LEARN
Better models do not remove the need to say what you care about, so the work is making your intent explicit and keeping the environment the agent works in worth working in.
02Ten years by hand, then the moment it clicked
- Agents crossed from novelty to dependable around a specific model release, which is why advice written before that point often no longer applies.
- Years of experience doing work by hand become leverage, not waste, because you can recognise when the output is subtly wrong.
- Being interested in how your colleagues work transfers directly to agents, since both come down to designing the environment someone else operates in.
03Tell agents what you care about
- No matter how capable the model, it cannot know what you want until you say it, and you have to say it every time.
- An agent inside a neglected codebase produces neglected work, so improving the codebase is a direct improvement to the output.
- Ask how you would raise the floor for the weakest member of a large team, then apply those same tests and checks to agents.
04Don't trust what the agent teaches you
- Hallucinations are less frequent but still real, so treat an agent's explanation as orientation rather than as the answer.
- Use an agent to acquire the vocabulary for an unfamiliar topic, then read the primary source that vocabulary unlocks.
06Where the opportunities are
- Teaching and adoption help are wide open, because most organisations are hitting internal resistance rather than technical limits.
- Agents excel at code because code is easy to verify, and struggle with knowledge work for exactly the opposite reason.
- Turning a specific organisation's knowledge work into processes an agent can run is genuinely hard, which is what makes it a real opportunity.
07Unknown unknowns and the Grill Me skill
- Nobody knows exactly what they want, so treat your own first description of a project as incomplete by default.
- Have the agent interview you hard before it builds, because the requirements you never thought to mention are the ones that break the result.
- Anything you leave unsaid, such as security or deployment, reads to the agent as unimportant and gets filled in with invention.
09How to stand out when anyone can ship in a day
- Clients buy the sense that you understand what they care about, which a quick prototype of their actual problem demonstrates faster than a portfolio.
- Speed that unsettles a buyer is a pricing and expectation problem, not a reason to pretend the work took longer.
10Communication is the differentiator
- The strongest performers are the clearest communicators, and that clarity shows up in the quality of what they ship.
- Fuzzy thinking produces fuzzy instructions, so you cannot describe the fix to an agent if you cannot describe the problem to a person.
- Give a project one vocabulary used identically in the code, the data, the interface, and the client conversation, and misalignment drops.
11The spec-driven development failure
- Writing specifications as the durable artifact can look productive while the underlying code quietly degrades over months.
- A popular idea backed by an impressive source is still worth abandoning when your own project keeps getting worse under it.
- Good code is code that is easy to change without breaking something unanticipated, which is also the condition agents need to work well.
12Models, harnesses, and effort levels
- Models will never read your mind, so expect a permanent process layer between your intent and the agent's output.
- Higher effort settings are not automatically better output, and lower effort buys speed and token efficiency when your checks are solid.
- Giving an agent the ability to run, click through, and screenshot the application improved results more than changing the model did.
13Climb the trust ladder
- Start in an empty folder with an idea you actually want to exist, because motivation is what keeps the feedback loop running.
- Learn the brake before the accelerator: watch an agent closely until you can predict it, then add skills, more agents, and connections.
14Give it your boring work
- Hand new models your boring, menial, easily verified work rather than your most ambitious task, because you can measure and correct the result.
- Automating the work you dislike is what frees attention for the strategic layer, which is where the actual value of the time sits.
15Software factories and open models
- Agents are currently poor at thinking long term about a codebase, so sustained quality still needs a human asking for the right way over the fast way.
Terms worth knowing.
- Harness
- The application wrapped around a language model that gives it tools, file access, and a loop to work in. Claude Code, Codex, and GitHub Copilot are harnesses.
- Vibe coding
- Building software by describing what you want and accepting whatever the agent produces, without reading or caring much about the resulting code.
- Spec-driven development
- Writing a detailed specification document as the durable artifact, then having an agent generate code from it and refining the spec rather than the code over time.
- Domain-driven design (DDD)
- A software design approach that organizes a system around the real concepts of the business it serves, rather than around technical layers.
- Ubiquitous language
- One agreed vocabulary for a project used identically in the code, the database, the interface, and conversations with clients and users, so a word means the same thing everywhere.
- Unknown unknowns
- The requirements and risks you are not aware you have missed. They never make it into your instructions, so an agent treats them as unimportant.
- Effort level
- A setting that controls how much reasoning a model spends before answering. Higher effort costs more tokens and time, and is not automatically better output.
- Trust ladder
- A staged approach to delegating work to agents, starting with tiny supervised tasks and adding scope only once you can predict how the agent behaves.
- Sub-agent
- A secondary agent spawned by a main agent to handle a scoped piece of work, letting one session coordinate several parallel tasks.
- Software factory
- A setup where agents start and improve work on a codebase on their own over time, rather than waiting for a human to hand them each task.
Things they pointed at.
02:12toolOpus 4.5
07:37toolThe teach skill
16:31bookThe Pragmatic Programmer
16:59toolThe Grill Me skill
23:54bookDomain-driven design
24:35toolThe Grill with Docs skill
26:23linkSean Grove's AI Engineer talk, specs are the new code
27:33toolDescript
31:07toolChief of staff skill for sub-agent management
31:56toolClaude Code
36:15toolCodex
36:15toolGitHub Copilot
36:15linkLauren Tan on climbing the trust ladder
37:12toolVS Code
Lines you could clip.
04:33
“People underestimate the communication barrier there is between you, the human, and the agent. And it doesn't matter how smart that model is. There's always going to be that gap between you.”
05:11
“If they're in a garbage vibe coded crappy code base that has never had any care put into it, then you're going to get garbage in garbage out. The agent is just going to produce rubbish because it's in a rubbish environment.”
05:57
“People have been thinking about this stuff for 20 years. There are books written about all of these concepts. And it turns out if you read those books, they feel like they were written last week.”
07:08
“You shouldn't, right? You should always be skeptical over anything that anyone tells you.”
16:31
“There's a great quote in The Pragmatic Programmer which says no one knows exactly what they want.”
21:54
“If it freaks people out that you're able to churn out work in a day, then just charge them for five days and just sit on your bum for four.”
23:05
“The clarity of their thinking ends up in the product they create.”
28:32
“Code is the environment that the agent operates in. If you have an agent that's basically that dog sitting there with a house on fire around it, it's not going to produce good code.”
30:18
“They're never going to be good enough to read your mind. Never going to be good enough to align with you perfectly every time.”
32:11
“I think most people are using way too high an effort level.”
36:43
“When you learn to drive, what's the first pedal you learn? You learn the brake first.”
39:57
“I think give it your most menial, boring work. Give it the stuff that you hate doing.”
43:16
“What I'm excited about is turning agents into that annoying developer who's always on the team who says, no, we should do it the right way, not the fast way.”
Where the conversation goes.
01:04 – 04:04steadyWhy agents finally clicked for a long-time skeptic
04:04 – 06:52denseCommunication gap and raising the floor
06:52 – 08:30steadyTrusting AI as a teacher
09:48 – 15:07denseCareer opportunities and agent processes for knowledge work
15:07 – 20:27denseUnknown unknowns, intent-driven prompting, interrogation skills
20:55 – 22:43steadyClient trust, positioning, and pricing speed
22:43 – 25:53denseClear language, DDD, and ubiquitous language
25:53 – 29:27denseSpec-driven development and why it failed
29:27 – 34:56denseModel choice, harnesses, effort levels, agent tooling
34:56 – 39:30denseGetting started and building trust in agents
39:30 – 41:36steadyAmbitious tasks versus boring tasks
41:36 – 44:50steadySoftware factories, agent organization, open models
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
metaphoranalogystory
How do you stand out in a world where everyone can build things so quick? I think the people who are really successful are the people who are the best communicators, right? So in my opinion, the best way to get the best work out of agents is to tell them what you care about.
So the skill there is being able to just get it all out of your head. The skill is being aware of unknown unknowns, right? Could be security, could be how you deploy it, could be just things you haven't thought of.
And if you're not careful about those, then the agent will just make stuff up. What would you see as the big opportunities to be looking at to help launch your career in the right spot? Taking agents, figuring out processes for knowledge work, and then bringing those for organizations.
Because every organization does things slightly differently. I think that's a I would love to hear a war story.
All right, Matt, I'm super excited to be sitting here with you today. You know, I am a frequent user of your skills, of course, and it's really interesting to me that you taught yourself to code. And when we first hopped on the call right before we started recording, you said that you think learning how to code is a lot easier than people make it out to be.
So you taught yourself how to code. You were doing it by hand for about 10 years. And then we entered this whole era of being able to use coding agents and things like that.
So I'm really curious to... hear your opinion on all this today, but I'd love to just kick it off real quick to you and understand what was that transition like for you when you had learned all these fundamentals of coding and you're doing it by hand and then you can sort of like, you know, the term vibe coding pops up and people are just building things in one day.
What did that transition look like for you? Well, thank you. Thank you for using my skills as well.
I appreciate it. That's very kind. I really started, I was a skeptic for a long time.
I had about 10 years of coding as a sort of amateur and then professionally for about eight years before vibe coding, before... stuff got really good. I consider like when Opus 4 .5 got released, when OpenClaw got built around December last year, 2025, I consider that to be the moment, the sort of threshold where I went, okay, I'm never going to write a line of code myself anymore.
So that's the moment when agents got really, really good. And it was the kind of year leading up to that where I was going, okay, yeah, I can see the benefits. I can see that everyone's very excited about this, but it hasn't quite...
clicked for me. And as soon as I... I don't know.
For me, I've always been very excited by the parts of my job that were like trying to make my colleagues better, right? Trying to build things that helped my colleagues ship stuff faster, trying to communicate to other devs and teach them how to do their job better, even if they were probably maybe senior to me. So I'm always interested in like...
the environment in which my colleagues operated and the environment in which I operated. And so when agents came around and they actually were really good, I started thinking, right, I can use these skills that I've developed over my career and my interest in this stuff to really help these agents get a lot, lot better, a lot faster.
And that hadn't really been working previously before Opus 4 .5. And then December last year, it started working. I realize, wow, okay, I can really put my skills to work here and actually pump out some pretty good stuff.
And it's been working so well so far. That's super cool. You know, one of the things that I've kind of taken away from the stuff that you're putting out there online is that you believe the software engineering fundamentals and principles are even more important now because AI is being slapped into all of these development processes.
But also on the same point, You don't necessarily believe that someone has to be super, super technical or have a huge coding background in order to take advantage of this stuff. So what does that look like for someone who's not technical?
Because I know that your audience is primarily pretty technical. What does it look like for someone who's not technical? How do they actually use software engineering foundations and principles to build in the right way?
Right. So in my opinion, the best way to get the best work out of agents is to tell them what you care about. Right.
Because agents will tend to like people underestimate how. People underestimate the communication barrier there is between you, the human and the agent. Right.
And it doesn't matter how smart that model is. There's always going to be that gap between you. And the thing about that gap is that the agent doesn't know precisely.
what you want it doesn't know what you care about and so you have to tell it again and again every time and so i think you're going to get better results out of agents if the environment in which they operate is useful for them right if they're in like a garbage vibe coded crappy code base that has never had any care put into it no one cares about the code base they only care about the outputs then you're going to get right the agent is just going to produce rubbish because it's in a rubbish environment and so you have to start caring about that environment and the way you start caring about it is you start thinking about how, let's say I was running a huge coding organization and I had a hundred humans, human coders, I know that sounds like a strange idea, to actually do my work for me, right?
Or push work out. How would I raise the floor so that the worst human coder that I had access to was going to perform better? What strategies would I use?
And it turns out people have been thinking about this stuff for 20 years, right? And there are books written about all of these concepts. And it turns out if you read those books, they feel like they were written last week.
They feel so relevant for right now because they're talking about how you make good tests, how you add checks to make sure that your code base gets better and better over time to raise the floor, to stop... classes of bugs to prevent mistakes. And that's what everyone is trying to do now.
And so I think the first step in that is to care about that and care about learning it and get interested in it because agents are pretty good at teaching you stuff as well. So if you're interested in it, if you're asking the right questions, then the agents will be able to understand that you care about that and start pushing you in that direction.
That's super interesting because I do, of course, I would say 80 % of the stuff that I've learned in the past year, I've had my agents teach it to me. And I know that you have a skill, the slash teach.
And, you know, one thing that I'd be curious about is how do you think about trusting what they're teaching you?
You shouldn't, right? You shouldn't. You should always be skeptical over anything that anyone tells you.
I think I'm like a born skeptic. I have always, like, it took me quite a while to get interested in AI because I was like, this thing is rubbish. This thing is like the new Web3 crypto bubble, you know, BS, whatever.
Am I allowed to swear on this? I'm not sure. I tend to swear.
Okay, good. I tend to swear a lot on podcasts. I don't know why it loosens me up.
So I was very skeptical for a long time. And I still remain skeptical unless I can read the primary source, right?
So when I design the teach skill, the teach skill is essentially all about going out, fetching a high trust, high quality primary sources, and then breaking those down for people with quotes, with like attribution, saying exactly where it links to it. And over time, the hallucinations of agents have gone way down. You know, they will tend to not make stuff up so much anymore.
They still do. Fable still does it. They still do it, but it's much less frequent.
And I found that what I mostly use those teach sessions for is I will just get it to introduce me to a topic and then I'll just go and read the primary source because that primary source has the actual information I need. It's just maybe I didn't quite have the language to interpret it before the teach session. All right, real quick, guys, a huge thanks to Hostinger for sponsoring this part of the video.
So I know that a lot of you guys are obviously learning to build with AI, and at some point, what you build has to live somewhere other than your local laptop, because once other people use it, you need a server that stays on and lets you change things without breaking what's actually working. So hosting your VPS is your own server with dedicated CPU and RAM and full root access, and it makes the setup the easy part so you can actually spend your time building instead of on the plumbing, because I know a lot of us aren't from like a DevOps background.
So I'd go with the KVM2 plan. For the price, you get two vCPUs, 8 gigs of RAM, and 100 gigs of storage, which is plenty for a first project. And you can always upgrade later if you need to.
Then you can pick the Cloud Code template, which comes with Cloud Code already installed, so you can start building as soon as you log in. And to keep what you build safe, you can snapshot your server before you change anything, and you can roll back if it breaks with weekly backups included. And if you get stuck, Hostinger Agent in the web console helps with server tasks, errors, and deployments just by using your natural plain English.
So you can grab KVM2 at hostinger .com slash natherkvps. The link is in the description. And you can use code natherk at checkout, which will take an extra 10 % off any yearly plan on top of the current pricing.
So thanks again, Hostinger, for sponsoring this part of the video. And now let's get back to it. Gotcha.
Okay. So what would you say is, you know, you're seeing a lot of different people. coming into your ecosystem and they're learning how to teach themselves things with AI, how to build products or automations, and they probably have a lot of different motivations in there.
What are you seeing as the different opportunities that are popping up for people that want to make sure, maybe they are a little bit skeptical like you were about this whole AI thing, but now we're seeing this, right? Like it's not going away. What would you see as the big opportunities to be looking at to help launch your career in the right spot?
Well, I mean, in any period of massive change, there are always going to be opportunities available, right? As everything shifts, then people are going to be adopting things at a slower pace. For me, I'm seeing a ton of like teaching opportunities, right?
Like, you know, people want to learn this stuff because it is so useful, it is so powerful. And every organization I talk to is having resistance with this stuff, trying to get it adopted. And let me make it clear.
parts about the AI world that are unethical or have dubious ethics. And I think there are attitudes out there that are absolutely correct. It is a bit weird the fact that Anthropic just nicked a bunch of novels and then just sort of paid them back later.
It is weird that it is going to strain the energy ecosystem to have all of this AI out there. But You're right.
It is here. It is not going away because there is an economic incentive to have it. So in terms of opportunities, I mean, they're just everywhere you look, really.
Like the ability to produce different types of applications that embed AI, to go into companies to say, I know how to use AI and I can help your systems improve and systems develop. It seems like everyone has this buzzword on their lips, and I think the opportunities are there if you look for them. Totally.
So let's take a practical example. Let's say someone right now works a full -time job and they're not technical. Let's say maybe they're a salesperson.
So not technical, they work a full -time job, but they want to figure out with AI, should I learn how to build this stuff and then start up a consulting business? Or should I try to use AI at work myself and just get promotions and try to work into a leadership position at my job? how would you yourself think about the the different paths and trying to figure out which one is is right for you so what i'd say is i think of this as kind of i take a sort of narrower view which is what are the skills that are going to be useful to me going forward right and if you have the skills that i'm about to describe then you will be able to like find your way to something useful, whether it's building your own thing or whether it's getting promoted within your own organization.
And the skills I think are useful are, like people think that agents, there's like no learning curve to them, right? They think you just talk to it and you just get stuff back. What I've found is that is absolutely not the case.
And as models improve, they don't necessarily lessen that learning curve. The learning curve is still there. It's just maybe a little bit hidden from most people.
So the concept of what can you trust from an agent? How do you organize agents? How do you get agents to behave in ways that follow what you want?
How do you control the information that agents receive? How do you design the information that you want the agents to receive? forms and data structures should those be in?
How do you then take those agents and turn them into like reusable processes? Or how do you take your processes and turn them into something that agents can follow? For instance, I've been using agents for code, obviously, and agents are really good at code because code is very easy to verify, you know, just like you see agents solving all these math problems because maths is very easy.
Sorry, math, math is very easy to... is very far right you can create these proofs that you just sort of like generate over time and you can solve these amazing math problems but real world work is much harder than coding for agents because it's much harder to verify and so I've had my biggest struggles trying to adopt agents for my day -to -day work.
And it's taken a long time and trying out innumerable processes until I finally found something I'm pretty happy with and I'm slowly improving it over time. So I think those are the big opportunities. It's actually taking agents, figuring out processes for knowledge work, and then bringing those for organizations.
Because every organization does things slightly differently and there's all of these opportunities out there. get that working and that's a skill that's really hard and like even me having fiddled about with agents for like a year year and a half now i'm still finding it very difficult and i think that's a massive opportunity so you said that the fundamentals are important and you've also said that explaining the context behind why you want something done so sort of like intent driven prompting and development.
Now, how do those things come together? Where does that break where a non -technical person that might not have knowledge of how to set up the right code base or how to create the right scoped API keys or whatever the issue might be, how does that person drive with intent while still making sure that things don't fall through?
And I'm not sure if you have any war stories of, and I'm sure this is where your whole vibe coding pivot away kind of, kind of comes into place, but I'd love to hear if you've seen specific examples of that falling short and how someone can think about, um, maybe it's, I guess the point I'm trying to make is what I've seen is you can, you can have like, you can have enough awareness to make your agent think about the problem and you might not know exactly how to solve the problem, but if you can identify that this is something that needs to be looked into a lot of times that can go.
a far a long way so i guess that's the point i'm trying to make is like how do you think about um a regular person being able to identify those those opportunities i mean you're talking about a regular person versus a coder right but everything you're saying there is like so familiar to me as a developer right it that there's no divide there there's a great quote in the pragmatic programmer which says no one knows exactly what they want right?
No one knows what they want. Everyone has these unknown unknowns in their head. Whenever they're specifying anything, whenever they're telling, like, let's say we're talking to a dev, whenever, like, I, as a normal person, talk to a developer, which is kind of like a normal person talking to an agent, there is that communication barrier, right?
And the dev is going to listen to everything and try to work out what I care about. And so one of my skills in particular, which is one of the most commonly used ones is called grill me and what that does is the agent essentially tells the agent to interview me relentlessly about the thing that we're trying to build and the reason that is so effective is because it then gets more context from you about what you care about and the thing you're actually trying to build and flushes out some of those unknown unknowns where sure some things you don't know that you don't know and some things that you do know that you don't know.
And that's very confusing. But essentially, it's stuff you're not aware of that you need to care about, right? Could be security, could be how you deploy it, could be just things you haven't thought of.
And if you're not careful about those, then the agent will just make stuff up or it will say, obviously, that's not important because they didn't mention it. And you'll end up with weird stuff happening or you'll just end up with something that's totally different from how you actually imagined it.
So the skill there is being able to just get it all out of your head. Yes. The skill is, I think, knowing that you're...
Hmm. What is the skill? The skill is being aware of unknown unknowns, right?
of knowing that you could be missing things in your design and not trusting that you've come up with a complete design. And whenever you create code based on some idea, it's really hard, right? That's a hard thing to do because by turning it into code, you realize the weaknesses in the design.
And sometimes those things just can't be... made clear until you've reached code because it's a kind of translation from the fuzzy human brain into executable statements, right? Into things that have to link to each other.
And there's always difficulty in translation there. Absolutely. I mean, I remember when I first started building automations, I remember doing the best I could for trying to predict all the edge cases and building guardrails to protect against those.
But then you put it in production and things happen, especially when the inputs to the system are human, because humans are just as unpredictable, if not more unpredictable than agents are. So I remember at least being aware of being able to communicate that to my client and saying, hey, I've built all these protections, but there are still some things that might happen in the real world.
And that's why I'm going to be here to monitor this and see what happens and make sure that all of these failures are still happening in a safe way. Now, a lot of my audience. is looking to build solutions for their clients or their stakeholders at work.
And one thing that I feel like I've been seeing a lot is a bit of a trust gap because it's so amazing how much you can build in one day and how expert you can appear if you lock yourself in a room for a weekend and just have a portfolio when you come out on Monday. How would you think about positioning yourself about your experience, your expertise?
your portfolio projects, how would you position yourself to, you know, a cold prospect to give them confidence that you are the guy or girl to get the job done, to be able to, you know, look at all the other maybe 20 vendors they're looking at or 20 people they're considering promoting? How do you stand out in a world where everyone can build things so quick?
Real quick, guys, I've got this completely free kit that I want to give to you all on helping you build your own AIOS. Whether you need to build it or whether you need to scale it up, this is the full kit. It's based on my actual framework.
I call the Herc Loop, context, connections, capabilities, and cadence. This thing comes with six skills. It helps you set up a brain.
It helps you set up your priorities and your business, and it interviews you. It is exactly how I built my own actual AI operating system, and it's how I'm able to move so fast. So if you guys want this, the link for this is down in the description.
But anyways, let's get back to the video. Honestly, I've never been in that position, right? I've never been a freelance consultant.
I've never been... I suppose I have been a freelance consultant. I just happened to have a huge audience while I was doing it, right?
So I had the kind of reputation that came before. I think what people really want in those situations is they want someone they can work with and they want someone who can understand what they care about, right? Who can try to cross the communication gap.
And so because agents allow you to ship prototypes to just...
I would say at this point, vibe coding is actually a good thing, right? Because you're just trying to get to an end point quick enough. If you can ship something that demonstrates to them that you understand their business, that you care about what they care about, and that you can demonstrate real acumen in building something real for them very quickly, I think they're going to like that.
So I feel like I can't offer concrete advice for that because I've never been in that competitive situation bidding for roles and stuff. Use AI to your advantage. And like, if it freaks people out that you're able to churn out work in a day, then just charge them for five days, you know, and just sit on your bum for four.
So like, I think there's a big, people don't realize how quickly you can churn out high quality stuff with agents yet, especially people who are not AI billed. And so I think you might just have to adjust. what you're doing for them.
What I think is interesting about what you said there is it sounds like it's the humanness that helps build that trust rather than look at all these cool things I've built or look at all these frameworks I understand. It's the humanness of, does this person care about what I care about? And are they thinking about protecting my business, not just getting a paycheck out of me?
What about with your students? I'm sure there's been some things that you've seen where it's like, this person is there's potential here, very talented.
And you might've also seen some traits where it's like, this person might need more work, a couple extra weeks. What are you seeing as far as like, what sticks out about your students to you? I think the people who are really successful are the people who are the best communicators, right?
That's always been true for developers and for non -developers too. Everyone I've seen who's been like the top, top tier of developer has always been incredibly good at communicating. That means that when they make a code change, they are writing a really detailed report about what changed.
It's easy to read, easy to understand, very clear from their thinking. In other words, the clarity of their thinking ends up in the product they create. And that is what I notice about...
folks who are not doing that is they get worse results because they can't communicate things to the agent and then they can't communicate what is wrong they're not thinking in clear terms and clear boundaries and so actually i think the quality of your thinking and the quality of your language is key right not only for when you're talking to the client but also when you're talking to the agent about what the client wants i have a few skills for this which is and there's Sort of a coding idea that comes into this as well, which is there's a concept called DDD which is domain driven design sounds very complicated but it has this idea at the core of it, which is that you have when you're building a project a What they call a ubiquitous language where the language?
in the code the terms you use in the code should be the same as the ones the developers use to talk about the product which is the same as the client asking you to build the thing in the product which is the same as the users using the product so if you have a concept of an order right what is an order that should look at the same in the code as it does in the database as it does in the UI as it does when you and the client are talking about it.
And actually iterating on that and creating that and the preciseness of that language, again, helps you align with what they actually want and what you're actually building. I can't recommend this technique enough. And actually, I have a modified version of Grill Me, which does this.
So helps you come up with these terms called Grill with Docs. That's what Grill with Docs does. It helps you build this domain language as you go.
I mean... I'm sort of shilling DDD there, but it's quite an advanced... You talk to a lot of coders, they won't have heard of DDD, but I've just found it extraordinary for building with agents.
Interesting. What do you think about JDD? Have you ever heard of that?
No, what's JDD? Jealousy -driven development.
Telling Claude, hey, Astra's doing this way better than you. What's your problem? What's going on here?
There's spectrum development. There's read me driven development is a fun one. Behavioral driven development.
I mean, it's everywhere. And I just saw that Anthropic announced something about there's going to be some sort of consequences or it will prohibit you from being cruel or mean to Claude. I don't know if you saw that.
I just saw that like last night. So we'll see what happens there. But you have talked a lot on YouTube here.
on stages about your skills. And there's a reason why I use them. There's a reason why so many people use them.
And what I would like to hear about from you is before all those, you know, I'm going to put you on the spot a little bit here. I would love to hear a war story. You know, I really like to learn from people by hearing about failures they've had or why they've learned a lesson.
And I'm sure there was a specific moment where you decided, OK, I need to build a skill around this so I don't run into that issue again. Or you had some sort of, you know. big problem inside of your projects or with your students?
I would love to just see if we could look inside your brain a little bit and understand one of these stories that you learned a big lesson from. Well, I was really intrigued by a talk that I saw given by a guy from OpenAI. It was on the AI engineer, Sean Grove, I think, saying that specs are the new code.
right specs are the new code and this theory was that you don't need to write code anymore because what you can do is you can write a specification just write a very detailed document and agents can help you do that on how you want the code to be and then what you do is you then get an agent to turn that specification into code And this is still a very, very popular idea.
People love this thing. And there are lots of frameworks built around it. I would say this is spec -driven development, basically.
That's a pretty muddy term, but that's how I define it, at least, which is you keep the specs around and you just sort of refine them as you go. And I tried doing this. I really tried.
This was before I come up with my skills and stuff. I have a repo that is my pride and joy, which is a custom video editor. It's like a sort of version of Descript.
It's got AI writing, design thumbnails on it. I do freaking everything on this thing. And I tried turning that into a big spec, and I tried doing spec -driven development, I suppose.
And what I found was I just kept wanting to look at the code, first of all, because I was curious. And I just saw the code getting worse and worse and worse. And the agent would produce worse and worse results.
And I tried this for months because I thought this guy was open AI, right? He knows what he's talking about. I'll be able to get this concept down.
And I just thought about it again and again and again. And it wasn't working. So I realized, no, I'm going to ditch these specs.
I'm actually going to focus more on the code. And the thing that triggered it for me was, Code is the environment that the agent operates in, right?
Like what I was talking about before. If you have an agent that's basically , it's not going to produce good code, right? The dog is in a bad environment.
What I wanted was good code and code that made sense. And if you think about what good code means, a good code base is a code base that's easy to make changes in. A change is not going to screw up something that the agent didn't think about.
A code base where everything is easy to modify. And that's what I started focusing on. And when I started focusing on that and started focusing back on the code, that's when I felt, okay, I'm actually starting to get somewhere and I can kick things into gear.
But the spec -driven development just did not work for me.
The models are getting better every time. Every time there's a new... number, iteration, they're getting better.
Why do you think that these models don't already understand a lot of these concepts or don't natively do them? And when do you think that'll happen, I guess? So it's about the communication barrier, right?
Models can't know what you care about until you tell them. And skills are one way of telling the model what you care about. Now, I think that You know, models are getting better and harnesses are getting better.
I think people underrate the impact of harnesses on models. You know, harness being Claude Code, Claude .ai. And Codex, whatever.
And I think then models are obviously going to get better, but they're never going to be good enough to read your mind, right? Never going to be good enough to align with you perfectly every time. And so...
You need some framework around that, some process to help this genius, genius developer understand what you want. Because it takes a really good product manager and a really good developer to make a great product. You have to care about things.
You have to look at the outputs and add your taste to it. So I always think there's going to be some layer of process over the top of that to just get over that communication barrier. That makes sense.
That makes sense. How much faster do you think that you can move now that you've learned these fundamentals from when you were doing it by hand? I mean, it's outrageous, the difference, right?
Like, it is outrageous, the amount of stuff I'm able to ship. Like, it's just not comparable. I've got four agents right now working on different projects.
I've given them all long -term tasks. I've said, okay, do this and do this and do this. Here are my long -term goals.
Go and figure it out. I've got a new couple of skills that I'm playing with, like a chief of staff skill, which does sub -agent management.
That's outrageous. That's like a team of eight developers or something that I've just got access to that are shipping for me. And I check in with them and I make sure their plans are good and I grill and all that stuff.
And I don't know. I mean, you could look at my GitHub and just look at the number of commits that I'm pushing in. And it's in 200, 300 a day or something.
Whereas before, 20 a day, I think would have been a good day. But again, you can look back at my stuff. So it's not comparable.
Not remotely. I have to ask the question about your favorite harness and model.
I've been using Claw Code and Opus 5 .5 currently. Opus 5 .5 is an excellent model. I use it on a much lower effort level than most people do.
And this is kind of one of my hot takes, which is I think most people are using way too high an effort level. I think people look at effort and they just think, yeah, I'll crank that up, right? Like I want it to put in effort, you know.
And I use it on Opus 5 .5 medium effort, basically. And I'm debating going to low effort, actually, because you get much more token efficiency. It replies much faster.
And because I've kind of raised the floor on my agents, like they're just, They don't need to work that hard to produce good code. You know, they just sort of, they just write stuff that passes the tests and the tests catch it.
So that's what I like at the moment, but I'm not that much of a polyglot. Like I don't go around like trying all of these models and harnesses because I think if they're like the model and the harness are good and obviously essential to creating good work. But I also think like, you know, just...
improve the environment that your agents operate in right that's where i find the most gains i recently moved a project from before i sort of hadn't figured out how to get this project to a place where i could get an agent to like look in the browser and click through it it was a complicated project it relied on lots of third -party dependencies like it had obs running locally this was my video editor like how do i make sure that's available and can do that while I'm also using it to record.
It was quite complicated. But I fixed that and now the agent can click through it, can create screenshots, can actually run the application properly. And just that difference has led to a massive increase in the quality of the stuff I'm able to produce because the agent has such better data to access.
And I think I could get just as good result like... with opus 4 .5 and that setup you know what i mean like it's it's amazing the difference that has made and so honestly i think working on the environment your agents operate in it just pays you back so much and i know it's kind of a cop -out from the question right because you want me to give a hot take of you know codex sucks and claude cody's great stuff but i just i didn't care that much about the harness really i really don't you know it's improving and it's great and opus 5 .5 especially is a much better model than five was much more efficient um but really I think of it as like just being employees in my company and I want my company to be as good as it can be so that the base level of employee is better.
Yeah. You know, and I actually really love that answer because I think if anything, that helps people realize you don't need to be stressing out so much about the newest model drop and making sure you fully understand what it's best at and all these benchmarks. Because, you know, if you think back to.
before we even had Opus and we had Sonnet and it was a really, really good model. Like you could automate tons of stuff. You can do tons of knowledge work and you could do a lot with that model.
So even Opus 5 .5 on low is better than that model was. And I think a lot of people don't realize that just because of how fast everything moves. And, you know, I've always believed that, you know, someone like you, Matt, with a very old model could still produce better output than someone with Opus 5 .5 that has no idea of how to communicate.
So that's what I think is really cool about that. And I do love that answer. Um, now one thing that I'm curious about for you as well is someone that might have been listening to this whole podcast so far, and maybe they're still feeling overwhelmed about a lot of the stuff that we're talking about, but you know, there's a lot of benefits and you've talked about how much faster you've been able to move.
And we talked about how the barrier to entry is so much lower. What would be kind of like your final, your final motivational message, your final push? to get someone to start taking action and just getting in there, getting their hands dirty and learning more.
So I think if you feel the urge to want to get better at this stuff, I have to say that my skills are a great place to start. If you're looking for just give me the basic idea of just do this and this and this. Then downloading my skills, downloading any harness, like just choose from the main ones, choose from Cloud Code, Codex, GitHub Copilot, whatever.
Don't spend too much time thinking about the harness, just try it out. And then from there, it's about, I interviewed Lauren Tan recently, Potato on Twitter, who works at Grokbot, and she talks about , right? Climbing the trust ladder.
Because you are not going to trust your agent. the first time you see it work. And that is good.
That's a good impulse. You shouldn't be trusting it blindly. You should, you know, when you learn to drive, what's the first pedal you learn?
Do you learn the accelerator or do you learn the brake? You learn the brake first, right? You learn how to stop because you don't want to just push on, push on, push on because you're going to burn through your rate limits.
You're going to produce stuff that you didn't want to see and you're not going to get a feel for the thing that you're using. So what I recommend is just to try opening up the model, giving it some prompts, try just open up an empty folder, use VS Code as well, because it is sometimes nice just to be able to look inside the code.
And from there, you can start working on some idea. It really helps to have an idea of the thing that you want to build and to see, have something cool in your mind that you can actually build towards. if you don't have like a reason or like a mission for doing this then it's a lot harder to get your motivation and your the feedback loop won't be as strong because if you're learning stuff and you're seeing a cool idea come to life that's just it's a multiplier and so from there you start to watch the agent like a hawk understand everything that's doing understand everything that a harness gives you and you can start to feel when you're trusting the agent and when you understand the agent.
And when you're at that point, then you can start maybe using two agents or you can start maybe writing a couple of skills to make the agent do custom things. You can start connecting it up to different data sources. You can plug it into your Slack, to your Twitter, to your, you know, whatever, to your Gmail.
And from there, you can start, you know, I've got an agent that handles my to -do list for me. So I've got, you know, I... dictate something to my watch and it goes and pings a system and it goes in my to -do list and then the agent triages it and whatever so you start integrating it in your life and integrating it into your work so that's it climb the truss ladder start small start with my skills if you want to or don't you know and from there you can just go nuts with it and it's amazing how if you just pay attention to it and try to think on its level it's amazing how quickly you can climb that truss ladder I love that answer.
I kind of call that in a lot of my videos or that theory. I refer to that sometimes as teaching a kid to ride a bike, like the bike method of how you have to earn that trust. And you don't start with trying to teach 10 kids at the same time.
You have to watch them closely. You adjust them. And you're still at the end of the day, you're not just going to have 20 kids go ride their bikes on the highway and then go inside and take a nap.
You're still very much in the process. So I love that answer. You know, I've seen Elon, Boris and Sam, I believe, and probably many more people basically say some version of this.
Give X model your most ambitious tasks or your most ambitious goals. And it's really interesting to me that they are coming out and saying this because, I mean, I totally believe in that. But I would love to hear from you.
Why do you think in their minds they had to make that a point to tell the world, hey. Try our model, but try your most ambitious tasks on this model. I don't know, because I think the opposite, right?
I think give it your most menial, boring work. Give it the stuff that you hate doing. Get it to draft replies to people who are asking you for things.
Give it simple achievable tasks where you can measure the output and measure how good it is, right? Now, of course, you can use it to brainstorm. You can use it to think about ideas and think about how you might turn those ideas into execution.
And it's pretty good at that. It's useful to use GrillMe for that, actually. But...
I would say that the real utility that I get from AI and the reason that I use it is because it helps automate the boring stuff. It helps me focus on the strategic layer and the stuff that I want to focus on. You know, all of the, like, I have video content that I produce and then written content that I produce.
And writing that written content has got so much easier since AI came around because I plug it into Sonic 5 .5 and it produces a first draft and I iterate on it. You know, that has been amazing for me because it's made my work more interesting to me because I didn't like writing those things before. It was a real pain.
And I even used to pay people to do it for me. So yeah, give it your boring stuff. Give it your low -level menial jobs, the stuff that you don't need to worry if it gets wrong or not because you can just go in and correct it.
So let it do that because that's what it's really good at. So I want to close off here with one question to... just kind of learn a little bit more about you and where you're headed.
So I'd love to hear when you think about the next, let's just say two to three years, which in the AI world is a decade. What are you really excited about? What are you building?
What are you working on? What's something in your life that's got you excited, especially as you think about how AI will evolve that? So I'm going to prefix this by saying I'm a teacher.
I'm not a pundit, right? I have no idea. I'm in Oxfordshire.
I'm not in SF. I don't have the ear of anyone important. No one is whispering the future into my ear.
I actually reject most offers to preview things because I just don't care, actually. I just want to make it work now, right? And that has been working well for me so far in terms of my sanity, too, because it's easy to get swept up in what's going to happen.
Your question was what is going to happen in three years or like what am I excited about? Yeah, what are you excited about? Yeah, I'm excited about finding the right way of Turning my ideas into work right and into code because I still don't think I've been trying to like, you know I suppose what I'm excited about right now are , ways that you can automate initiating work and improving code bases over time by themselves.
I think agents currently are not very good at thinking long term about a code base. And what I'm excited about right now is turning agents into that annoying developer who's always on the team who says, no, we should do it the right way, not the fast way. We should do it for the benefit of the people who come after us.
Agents are really bad at that right now. And I think there is a way to make them good at it. That's what I'm specifically excited about.
And I think that we haven't found the right way of organizing lots of agents yet. I still think there's huge potential in that. And nothing I've seen has made me go, yes, that is the right way yet.
And I'm also excited that open models are going to get better. That's another thing I'm really excited about. Because they are getting better.
And eventually we will have an open model that's as good as Opus 5 .5. And I'd be delighted with that. That's fantastic.
I don't need the models to be that good, right? Because I'm pretty good already. I can sort of make up for them.
Just having an Opus 5 .5 level of open model would be amazing. Even beyond that, just sort of getting more token efficiency, getting a nice plain speaking model who can ship work quickly. I don't know.
It's cool. It's an exciting time. It absolutely is.
It absolutely is. Well, Matt, thank you so much for coming on today. I really enjoyed this conversation and thank you for all of the golden nuggets you dropped on us today.
Where can people find you if they want to learn more and see what else you're up to? AIhero .dev is the place. I've got a newsletter there.
I'm on Twitter. I'm on YouTube. Those are the main three.
Awesome. Well, thank you so much again, Matt. And hopefully we'll see you again soon.
See you, man.
The Hook
The bait, then the rug-pull.
The cold open stacks three answers before the first question lands: the best communicators win, tell the agent what you care about, and the real skill is being aware of your own unknown unknowns. Everything that follows is one argument, made from ten years of writing code by hand and about a year of writing none.
How they asked for the click.
MENTIONED ON CAMERA
