1:36:30AI Engineer · Tutorial
Full Walkthrough: Workflow for AI Coding
A 96-minute live workshop on using software engineering fundamentals to get autonomous coding agents to ship real, non-slop features.
April 24thThe argument in one line.
You can trust agents to ship at massive volume without quality loss if you invest in the codebase and verification around them, turning every correction into a deterministic check instead of another written rule.
Read if. Skip if.
READ IF YOU ARE…
- An engineer or tech lead whose team is already using coding agents and drowning in pull requests that are slow to review.
- Someone who keeps adding lines to AGENTS.md or rules files every time an agent makes the same mistake and wants something more reliable.
- A developer curious how formal verification, TLA+ and Lean fit into everyday product work with agents.
SKIP IF…
- You are looking for a beginner intro to prompting a coding agent; this assumes a real production codebase and a team.
- You want vendor-neutral tooling advice; several examples lean on the speaker's own products (Grokbot, Dune, pstack).
The full version, fast.
Lauren Tan, who works on Grokbot at SpaceXSI, says the real job of an engineer now is keeping quality high while agents multiply output. Her team shipped 11,000 pull requests in a month with about a 0.5% revert rate. The method is an inverse pyramid: deterministic layers first (architecture, lint rules, types, tests), then verification (runtime, formal, property-based), and only then prose like rules and skills. Because agents copy whatever patterns exist, every correction becomes a lint rule or an API redesign, and review shrinks to verification artifacts like videos and screenshots.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 02:03
01 · Turning slop into gold
Opens with the claim of shipping 3,000 PRs in a month and the thesis that engineering now means keeping quality while multiplying output.
02:03 – 05:00
02 · The trust pyramid and codebase quality
Inverse pyramid of deterministic layers, verification, then prose rules; the codebase as a garden agents copy from.
05:00 – 06:56
03 · Dune and enforceable architecture
Banning useEffect in product code and using directory structure to keep main and renderer code apart.
06:56 – 09:40
04 · Replacing comments with lint rules
Agents hide behind comments and timers; Grokbot bans most comments and uses lint errors that demand root-cause fixes.
09:40 – 10:50
05 · Runtime verification and feature maps
Giving agents the ability to run the app, with a feature map and reusable custom CLIs.
10:50 – 11:51
06 · Formal verification and UI testing
TLA+ for user flows, Lean for a virtualization library, and Bombadil for property-based UI tests.
11:51 – 13:32
07 · Spending tokens on verification
Most tokens go to verification; agents make old, expensive testing techniques cheap; the state chart skill.
13:32 – 15:11
08 · 11,000 PRs and a 0.5% revert rate
Results: 11,000 PRs in a month, daily stable releases, 227 contributors, and performance that improved.
15:11 – 17:31
09 · Foundations before software factories
Fix the codebase and verification first, then automate; example of auto-fuzzing each release and auto-merging fixes.
Lines worth screenshotting.
- Agents learn from your codebase as their biggest input, so any workaround or deprecated pattern left in place will be copied by the next agent.
- Every time you correct an agent, turn the correction into a deterministic check, not another line in AGENTS.md.
- Anything that needs nuance to use well, like React's useEffect, produces slop from agents, so ban it in product code.
- Agents use comments and markdown to write down justifications for not doing the right thing, so Grokbot bans about 99% of comments.
- Directory structure can carry context: separate ElectronMain and ElectronRenderer folders with banned cross-imports stop agents mixing threads.
- Lint errors should tell the agent to fix the root cause, not patch around the symptom.
- Code review is shrinking into reviewing verification artifacts: videos, screenshots, diagrams and formal proofs.
- A feature map, a sitemap of features from the user's point of view, gives agents shared vocabulary instead of guessing what 'the thingy in the left sidebar' means.
- Verification techniques that were once too expensive to maintain are now cheap because agents write and maintain them.
- The team spends most of its tokens on verification, and one engineer reported about 70% of daily tokens going there.
- Automating a loop that produces slop just produces more slop, and burns more tokens doing it.
- At 11,000 PRs a month the revert rate stayed around half a percent and app performance improved on startup, typing and chat switching.
Make the environment trustworthy, not the review
WHAT TO LEARN
Agent output is only as good as the codebase and checks around it, so every correction should become a deterministic rule and most review effort should move to verification.
- Agents copy the patterns they find, so removing workarounds and anti-patterns from the codebase is the highest-leverage way to improve everything they write next.
- Prefer enforcement to instruction: a lint rule with a helpful error message is reproducible, while another line in AGENTS.md depends on the agent's judgment.
- Ban the APIs that need nuance to use well, such as useEffect, and the problem disappears from review for good.
- Use directory structure and dependency analysis to carry context, for example separating main-thread and renderer code and forbidding imports between them.
- Limit comments and prose rules, because agents use them to justify skipping the hard fix and to over-apply small corrections as permanent laws.
- Let agents run the real app with debugging tools, and give them a feature map and reusable custom CLIs so they share your vocabulary and spend fewer tokens.
- Layer in formal verification (TLA+, Lean) and property-based UI testing as extra tools, not silver bullets, since agents cut their maintenance cost.
- Expect most agent token spend to go to verification, and shift human review toward videos, screenshots and diagrams.
- Build the foundation before the factory, because automating a slop-producing loop only adds more slop and more cost.
Terms worth knowing.
- Software factory
- A setup where agents write, test and merge code at high volume with little human intervention, built on a trusted codebase and verification layers.
- Organic architecture
- A codebase structure with enforced constraints and guardrails so that new agent-written code grows in the right shape, like a building's foundations guiding growth.
- Lint rule
- An automated check that flags banned or risky code patterns, used here to make agents do the right thing by default.
- Feature map
- A sitemap of every feature in an application from the user's point of view, used so agents and humans share vocabulary.
- Runtime verification
- Letting an agent run the real application and inspect it with debugging tools such as the Chrome DevTools protocol to confirm behavior.
- Formal verification
- Using mathematical tools such as TLA+ or Lean to prove that a system or algorithm satisfies stated properties.
- Property-based testing
- Testing that generates many inputs against stated properties instead of fixed examples; Bombadil applies the idea to UIs.
Things they pointed at.
00:00productGrokbot
05:00toolDune
10:40toolTLA+
10:45toolLean
11:30toolBombadil
Lines you could clip.
02:40
“Whenever you're correcting your agent, think about how you can make that a deterministic check instead of adding another line to your rules.”
03:35
“Your code base is kind of like a garden.”
16:40
“If your inner coding loop produces slop, then automating that loop just produces more slop.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
So again, my name is Potato. I work on Grokbot at SpaceX SI. And can I ask you a quick show of hands?
How many of you have ever looked at code that somebody else wrote and you thought, that's all slop?
Well, I have too. But today, I'm going to convince you that you can turn that slop into gold.
And so I did something crazy last month, and I shipped 3 ,000 Ploycrests to production. Now, before Agents, we never really cared about volume, except for the outliers. And at my previous company, we called these people coding machines, people like Brett Taylor, who famously rewrote all of Google Maps in a weekend, just an insanely productive engineer.
And well, with Agents, anyone can now be a coding machine. And my thesis is that the job of the software engineer, the meta of being a software engineer, is figuring out how to maintain or even improve the quality of your code while massively increasing your productivity. To get there, some people like to call this a software factory.
But I like to keep it simple. And I think of it as all about trust. Not blind trust, but investing in your agent's environment so you can trust that your agents are going to do the right thing.
Now, it takes quite a lot of effort to get there. And it took me six months to get to a point where I could ship thousands of pull requests, a lot of them in my sleep. But it's certainly possible and it will take you much less time than I did.
So let's start by talking about how we move from a low trust environment to a high trust one.
So this inverse pyramid is how I think about where to spend time on your agent's environment. The first two layers are deterministic, so using your code itself. So things like good API design, architecture, refactoring, or using lit rules, type systems, compilers, or different kinds of testing.
And you can use both of these layers together to constrain what agents can do and force them to do the right thing by default. Verification, both runtime and formal, are great tools to layer on to find bugs and check properties and the invariants in your system that you want to hold true, and also help you check whether or not the agent understood your goals and your intent.
And then finally, you want to layer with a little bit of pros, so things like rules, skills, agents .md, things that require judgment. But let's start with your code base. agents generate their outputs based on their inputs.
And so the quality of the input really matters, and it just so happens that your code base is one of the largest inputs. So whenever you're correcting your agent, think about how you can make that a deterministic check instead of adding another line to your rules or your agents .md, because it's far more reproducible when it's in code, and it lets you trust it more.
And the analogy that I like to use is that your code base is kind of like a garden. You know, code used to arrive a few pull requests at a time. And I remember going into work in 2024 and reviewing the three pull requests that my colleagues wrote.
And today, that number is more like 300. So every engineer is now a team of hundreds or even thousands of agents, and code is arriving so rapidly that it almost feels like a continuous stream, and it almost feels organic, like a living thing. And in this analogy, your code base, the constraints that you impose, the guardrails that you put up, are the concrete foundations that the new code that your agents write grow around, just like the building in this picture.
And so in many ways, the job of being a software engineer is kind of becoming like gardening.
Because code is the biggest input for agents, it means that any workarounds, anti -patterns, deprecated code, tech -dep, it's now likely to be just copied by the next agent since it looks like the established pattern in your code base. And it's like you have some weeds or pests growing in your garden. and then they spread everywhere before you know it.
So you want to make your code base as pristine as possible so that agents are always copying the good stuff. All right, let's look at some real examples. So Dune is a client framework that we created for GraphBot to implement this idea for organic architecture.
It's not really that interesting technically. You can kind of think of Dune as kind of like Next .js for electron apps. But the interesting part of Dune is that it has a lot of constraints and lit rules that we've encoded a lot of the lessons that we've learned building desktop applications.
And one such lesson is that anything that requires nuance to use well often leads to slop. Your classic example is useEffect in React. It has its use cases, but it's often misused.
Some people joke that you need a PhD to use useEffect. And in other words, it really requires a lot of nuance to use well. And agents are not good at using it.
So our solution has been to just ban it in product code. And it's really only allowed in some places inside of Dune APIs. And effectively, this problem has now been categorically eliminated, and we never need to catch that during code review again.
Another interesting pattern that we use is context through directory structure. So one anti -pattern that we saw with Cursor, which is an electron app, is that agents would often import main thread code into the render thread, which would cause long tasks and jank. So in GrotBot and in Dune apps, we've made this very, very conventional.
We have these two top -level directories, literally called ElectronRenderer and ElectronMain, and imports between them are forbidden or banned by our dependency graph analyzer and some lint rules. And again, these are simple lint rules to set up, but it means one less thing that we need to review and think about during code review.
And then one more surprising thing that I've seen is agents using markdown. So things like documentation, code comments, rules or skills. And they use this as a place to write down their justifications for not doing the right thing.
Or, I don't know if you've ever seen this, but they'll over -rotate on some small correction that you made, and they treat it like a hard rule that can never be broken ever again. And here's another example from Cursor. So agents really loved using imperative timers.
So things like set timeout, request animation frame, request idle callback. And they were using these as quick fixes because they were really patching over an architecture clash between VS Code's imperative style and React's declarative style. So our choice of architecture actually made both...
the client and the agent performance much worse. And if you've ever done any performance work in JavaScript, you'll know that when you have tons of timers being rapidly created and destroyed, causes a lot of major garbage collection, which causes a lot of lag and jank. But the worst part is that agents wrote a lot of code comments, like you can see in red here, explaining why this quick fix was justified.
And before you know it, We had timers spreading everywhere and the performance of the app really suffered. Now in Grokbot, we learned our lesson there and we decided to just ban 99 % of comments because They go stale easily.
Humans just really don't bother reading the comment slop anyway. And again, it gives an agent an escape hatch to justify not doing the right thing, because it's hard. And the worst part about comments, though, is that they're not enforceable.
So what seems well -intentioned, like I'm going to tell the next agent not to do something, never ends up actually being anything that you can trust. In Grokbot and in Dune, we now have lint rules that give good and very helpful error messages telling the agent to fix the root cause, the real problem, rather than just patch around it.
And so our general principle is that every time we correct our agents, it turns into one of these deterministic checks, or we think about how to categorically eliminate that problem in the code through better API design, through better architecture. And if you do this enough times, you'll find that you have just less things to look out for in code review.
And at some point, you'll realize there's nothing left to review, and you can just let agents merge most of their own pull requests.
All right, moving on. Verification is also a really big part of how we make our agents more autonomous and also increase our confidence and trust in their work. And our code review process has now really shrunk into looking at verification artifacts instead.
So things like videos, screenshots, diagrams, formal verification. And the most basic example of this is runtime verification, which is just a fancy way of saying giving your agent a way to run the code for itself and use debugging tools that human developers use as well, like the Chrome DevTools protocol. In our runtime verification skills, we have this thing that we call a feature map.
And that's basically kind of like a site map of all of the features that are available in your application from the user's point of view. And so with this, agents don't need to guess when you say, you know, the thingy in the left sidebar. It gives you some shared vocabulary.
And we also make use of custom CLIs inside these skills so that agents can reuse the CLI and the driver every time, which cuts down on token usage and is also testable.
And then recently, we've also started looking at formal verification. So we now use TLA Plus to do some model checking of major user flows, such as notifications across multiple devices and platforms. We use a lean as well for proving certain algorithms.
For example, we wrote our own custom virtualization library and we use lean there to check that we don't have flickering or jittering problems. And we're still experimenting with this approach overall. It seems very promising.
We've already found quite a few bugs with formal verification. But it's certainly not a silver bullet. And we're treating it as another tool, another layer in our agent's environment.
And then this is a very fast video. But another thing that we just started looking at is property -based testing as well for UIs. So we use this library called Bombadil.
And this has also caught a few UI bugs for us already.
Okay, so zooming out, I saw this tweet from Cindy. And six months ago, she said that she was talking about not being able to trust her agents because of subtle bugs that were getting merged. And really what was missing for her in being able to trust her agents was the verification process.
And now, six months later, she says that 70 % of her daily tokens go into verification. And I'd say that that tracks with what I see as well, that most of my token usage goes to verification. And if you really look closely at our process, none of these things here are new or novel techniques that we've never had before.
These are just good software engineering tools, but ones that might have been prohibitively expensive to use before as they were hard to understand or write or maintain. But agents drastically cut this cost down. And really, we're entering a golden age of software testing and verification because agents let us test everything to a much greater degree than used to be economical.
And then formal verification can also help you better reason about how your product should work and whether your agents really understand your intent and goals. I made this new skill called state chart in my plug -in PSAC, potato stack, which helps you visualize state machines from... from the user's point of view.
And I find this much easier to review alongside the videos and screenshots that we provide. And so that's been great. Sorry for the flashbang.
Okay, now let's look at what the promised land looks like. What do you get in return for all of this investment? So on the Grofbot team, our velocity is taking off and rapidly increasing.
And last month, our team shipped a ridiculous 11 ,000 pull requests to production, which is a wild number. And our pace of releasing the client has also increased, and we cut a new stable version about once a day.
And then the core team is also shipping much more, and we also have a lot more casual or drive -by contributors who don't always have a lot of context, but they're also contributing. And in October, we had about 227 people who contributed at least one pull request.
So despite this velocity accelerating, you might be thinking to yourself, isn't this going to be chaos? Isn't this going to be a bunch of slop? But actually, the number of PRs that get reverted has actually gotten lower on a relative basis and is steady at about half a percent.
But I think the really interesting thing about this shipping pace and this volume of pull requests is that the performance of Grokbot has stayed not just stable, but has improved on multiple dimensions compared to when we launched. So startup, typing, opening new chats, switching chats, all of this has gotten faster.
And keep in mind, Grokbot is still an electron app for now. So there is a ceiling, right? But it's already much faster than many other agent UIs out there.
And for an electron app with this high velocity and this many people contributing, I'd say that's pretty good. So before you jump into building out a software factory and automating everything that you can, whenever you correct your agent, try and do it by improving your code base first. Start with a lint rule to prevent new occurrences of anti -patterns, and then clean up those old patterns as well.
And then make sure that your verification story is really strong. A lot of this work is a prerequisite to automation anyway. And if your inner coding loop produces slop, then automating that loop just produces more slop.
And you need to throw more and more and more tokens at the problem, which we all know is very expensive. And once you've made these investments into your code base, static analysis, runtime informal verification, you'll start to find that many things become possible with these foundations, and that you can really easily build your own factory on top of these foundations.
And when you're ready, GrotBot is a great way to build that outer loop. We have tons of connectors, routines that can kick off cursor, cloud agents, and projects. which let you orchestrate thousands of hundreds or even thousands of agents.
And as a quick example, whenever we cut a new release now, Grokbot will automatically spin up a new cursor project for us that will fuzz the entire application, the entire build, and it will find bugs by running the app and using it like a real user. And then it's going to open fixed PRs for each of those bugs that it finds, and it's going to auto -merge it, as crazy as that sounds.
And that's because our code base is good enough that pull requests tend to just be good by default. And most product code that we land no longer really requires deep human review.
So when you have solid foundations and good organic architecture, you can turn slop into gold, and you can ship with high velocity without compromising on quality. My name is Potato, also known as Lauren.
Thanks to you again for listening to my talk.
The Hook
The bait, then the rug-pull.
A show of hands about slop code opens a talk that ends with a team merging 11,000 agent-written pull requests a month. The through-line is trust: not blind trust, but an environment engineered so agents have little room to do the wrong thing.
Named ideas worth stealing.
02:09model
The inverse trust pyramid
- Deterministic code layer (API design, architecture, lint, types, tests)
- Verification (runtime and formal)
- Prose (rules, skills, AGENTS.md)
Spend effort on your agent's environment from the bottom up: constrain with code first, verify second, and use judgment-based prose last.
Steal forany team deciding where to invest in agent reliability
03:10concept
Correction to deterministic check
Whenever you correct an agent, encode the correction as a lint rule or better API design so it never needs catching in review again.
Steal forreplacing growing AGENTS.md files
How they asked for the click.
MENTIONED ON CAMERA
Visual structure at a glance.
