How Senior Engineers Actually Code with AI (Full Product Build)
A two-hour-forty build of a cited deep-research agent, used as a running demo of the plan, prompt, delegate and verify loop that separates shipped products from AI slop.
Posted
2 days ago
Duration
Format
Tutorial
educational
Views
10K
249 likes
57 · 43
Big Idea
The argument in one line.
Every engineer has the same models and agents now, so what separates a shipped product from AI slop is the environment you build around them: a written source of truth, small scopes, and agents that must prove their own work.
Who This Is For
Read if. Skip if.
READ IF YOU ARE…
A developer already using Claude Code, Cursor or Codex who keeps getting apps where half the buttons do nothing and wants a repeatable process instead.
An engineer who wants to see a real multi-agent workflow with lead, implementer and researcher model roles running on an actual product, not a toy.
Someone building an AI app that has to cite real sources and wants a concrete pattern for stopping the model from inventing quotes.
A builder who can get backend code out of agents easily but gets generic, cheap-looking UI and wants a method for pushing design quality up.
A team lead deciding how much AI-generated code to review by hand and looking for a defensible rule for where human review actually matters.
SKIP IF…
You want a short overview. This is a two hour forty minute screen-recorded build with long stretches of watching agents work.
You have never used a coding agent. The video assumes you know what a PRD, worktree, PR and sub-agent are.
You want a vendor-neutral scraping comparison. The web-data layer is the sponsor's API and roughly ten minutes are a sponsor segment.
TL;DR
The full version, fast.
Two engineers with the same model and agent get opposite results because one writes requirements and decides how to prove the app works before any code exists. The build starts with an idea definition (four core primitives, hard limits, a primary user flow), turns it into a PRD referenced from agents.md, and prompts with only a goal, constraints and definition of done. Work is split into foundation, backend and frontend sessions because quality drops sharply past roughly 500k tokens. The pstack framework routes each request to a playbook, delegates to cheaper implementer sub-agents and loops verification until checks pass. A project-specific verification skill, Bugbot plus Devin PR review, and screenshot-led design iteration close the loop. Review by hand only mission-critical code or anything a reviewer flags.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Cold open contrasting a one-shot prompt that yields broken buttons and fake sources with a requirements-first build. Introduces Research Marshal and the video's roadmap.
02:30 – 07:00
02 · Prerequisites: ADE, models, voice
Pick any major ADE with a built-in browser the agent can drive. Know each model's character to assign lead, implementer and researcher roles. Talk instead of type with voice dictation.
07:00 – 11:30
03 · Defining the idea and primary user flow
Research Marshal's four primitives (live web data, every claim has a quote, a hard budget, visible failures), what is explicitly out of scope, and the ask, watch, read, come-back user flow.
11:30 – 14:00
04 · Building shared understanding with a PRD
The agent knows nothing the human knows, so the idea diagram becomes a PRD: autonomy with guardrails, where the agent only stops to ask when the PRD has no answer.
14:00 – 20:00
05 · Prompting strategy: goal, constraints, done
Short prompts over engineered ones. The agent asks five questions, rewritten as A/B/C/D options with its recommendation. Terms are unified, agents.md points at the PRD, and a private repo is pushed.
20:00 – 34:00
06 · Phase 1: data foundation and scraping layer
Why scraping major sites is hard (IP blocks, bot detection, JS rendering, noisy HTML, per-site parsers). Sponsor segment on Decodo's Web Data API, then installing its CLI, agent skills and MCP server.
34:00 – 36:30
07 · Breaking the build into scopes
Instead of one prompt to do everything, the build splits into foundation, backend and frontend sessions.
36:30 – 42:00
08 · Token degradation
Performance falls roughly linearly then steeply past about 500k tokens, so scopes stay small. The tech stack is talked through and a tech-stack prompt is drafted without implementation.
42:00 – 53:00
09 · The pstack workflow and model roles
pstack's five primitives (playbooks, functional skills, principle skills, agent definitions, scripts) and how one router skill picks a playbook, delegates to sub-agents and loops verify and review.
53:00 – 58:00
10 · Catching an agent's mistake
The lead agent claims the sponsor's TypeScript SDK is abandoned; it was updated two weeks earlier. The human pushes back, picks the SDK for type safety, and swaps in Streamdown and Biome.
58:00 – 1:09:00
11 · Why agents are lazy, and boilerplate setup
Models prefer training data over searching to save tokens. Keys are stored, then a goal/constraints/done prompt installs and verifies the whole stack in 20 minutes via an Opus sub-agent.
1:09:00 – 1:16:00
12 · Three-step code verification
In-loop sub-agent verification, a PR bot (Cursor Bugbot) as backup, and human review reserved for mission-critical code or flagged issues. The chef-and-cooks analogy.
1:16:00 – 1:22:00
13 · The research pipeline explained
Plan 3-5 sub-questions, search, select by index capped at 8 pages, scrape to Markdown with SQLite caching, optional sitemap map, extract claims with exact quotes, verify quotes in code, gap check, up to three rounds, then write the report.
1:22:00 – 1:32:00
14 · Phase 2: backend implementation
A prompt that forces the agent to test the scraping API for real through MCP, CLI and SDK. Google search results come back as redirect links; the fix is a Google Search target template. About 5,000 lines land in PR #2.
1:32:00 – 1:38:00
15 · One full run, step by step
Recap of how the prompt flowed through router, feature playbook, architecture design by sub-agents, implementation, live proof, PR, and Devin Review finding five issues. How to actually review code.
1:38:00 – 1:50:00
16 · Project-specific verification skill
Almost the whole workflow generates code; verification is the weak link. A three-hour session fixes broken worktrees, adds doctor and check commands, CI and a feature map, and builds the verification skill using 22 sub-agents.
1:50:00 – 2:00:00
17 · Phase 3: frontend with reference material
'Beautiful' and 'not slop' are undefined, so the prompt carries screenshots and install code from a component gallery plus an ASCII-art library. The first two-hour pass adds 15,000 lines and looks merely fine.
2:00:00 – 2:22:00
18 · Iterating the UI with browser annotations
Annotate-in-browser prompts: copy the reference input exactly, rebuild the history sidebar, generate app-icon variants with an image model, test ASCII background variants, simplify the run header and add the data provider's logo.
2:22:00 – 2:30:00
19 · Building a custom design system
A palette derived from the San Francisco ASCII scene with fixed green/yellow/red status colors, a switch from blue to near-black backgrounds, a /design-system route with contrast checks, and a design-system skill for agents.
2:30:00 – 2:39:30
20 · Testing the final application
A live battery-life question runs three rounds with the Golden Gate ASCII art as a progress bar, hover citations and 45 verified versus 105 dropped claims. Parallel runs, the map endpoint in action, and a final word on review depth.
Atomic Insights
Lines worth screenshotting.
Coding agents with the same model produce slop or a real product depending almost entirely on whether requirements and proof-of-done were written before the first prompt.
A good prompt needs only three parts: the goal, the constraints, and the definition of done; heavily engineered prompts mostly box in smart models.
Ask the agent to restate its clarifying questions as A/B/C/D options with its own recommendation, then pick deliberately, because its recommendations are often wrong.
Put the PRD's path in agents.md, because agents.md is a standard every agent reads and a bare PRD file is not.
Model performance stays roughly linear up to a few hundred thousand tokens and then degrades sharply around 500k, so one-shot builds burn the worst tokens on the hardest parts.
Assign models by role: an expensive lead that plans and reviews, a mid-tier implementer that writes code, and a cheap fast model for research and reading.
LLMs prefer training data over searching, which is fine for history and biology but wrong for tools that change weekly; the agent here declared an actively maintained SDK abandoned.
A senior engineer's job with agents is no longer babysitting each step but checking that what the agent claims is actually true.
Let models pick search results by index number instead of URL, because indexes are cheap in tokens and can't be mangled.
Verify every extracted quote with plain string-matching code, not another LLM; in one real run 45 claims survived and 105 were dropped.
Sub-agent worktrees often lack node_modules and env files, so agents can write code there but can't verify it until the environment is fixed.
Better models and tools no longer move the needle much; a better environment for creating and verifying code is the remaining edge.
A project-specific verification skill (launch, doctor, drive, evidence, cleanup) took three hours to build and immediately found real bugs.
Telling an agent to use the browser continuously, not just at the end, catches visual mistakes while they are still cheap to fix.
Words alone can't define 'beautiful'; reference screenshots plus the actual component code give the agent something concrete to copy and improve.
Ask for four or five design variants at once and say there is no time limit, since agents otherwise rush to the first acceptable result.
Turn the design system into both a /design-system route for humans and a skill file for agents, since an HTML page is hard for agents to parse.
Hand-review mission-critical code like billing, auth and permissions, plus anything a sub-agent or PR bot flags; skimming the rest is the only scalable option at 50k lines a day.
Takeaway
Build the environment, not just the prompt.
WHAT TO LEARN
Agents produce real products when they have a written source of truth, small scopes, role-matched models and a way to prove their own work, with human review saved for what matters.
02Prerequisites: ADE, models, voice
Choose a coding environment with a browser the agent can drive, because an agent that cannot see its output cannot check it.
Assign models by role: an expensive lead to plan and review, a mid-tier implementer, and a cheap fast researcher.
03Defining the idea and primary user flow
Define the product's core rules, out-of-scope list and primary user flow before code, so the agent knows exactly where to stop.
04Building shared understanding with a PRD
Write a PRD and point agents.md at it so the agent answers its own questions and only interrupts you when the PRD is silent.
05Prompting strategy: goal, constraints, done
Write prompts as goal, constraints and definition of done, and leave room for the model to reason instead of scripting every step.
Have the agent rewrite its clarifying questions as lettered options with its own pick, then choose deliberately rather than accepting defaults.
06Phase 1: data foundation and scraping layer
Before building on any third-party API, make the agent test it through every access path, because docs and reality often differ.
08Token degradation
Split large builds into foundation, backend and frontend sessions to stay out of the steep quality drop past roughly 500k tokens.
09The pstack workflow and model roles
Use a fixed process per task type so verification happens every time, not only when you remember to ask for it.
Delegate implementation to sub-agents even when work is sequential, keeping the lead's context for planning and review.
10Catching an agent's mistake
Treat every factual claim from an agent about a tool as unverified until you check it; training data goes stale fast.
12Three-step code verification
Layer review: in-loop verification, a PR review bot, then personal review only for mission-critical or flagged code.
13The research pipeline explained
Let models choose sources by index, cap pages per round, and cache scraped pages so you never pay twice for the same URL.
Check extracted quotes with plain string matching against the page, and drop any claim whose quote isn't there.
14Phase 2: backend implementation
When a scrape fails on a specific site, try a site-specific template before rewriting your pipeline.
16Project-specific verification skill
Fix worktrees so sub-agents get env files and dependencies, otherwise they can write code but never verify it.
Build a project verification skill that launches the app, drives it along a feature map and returns evidence.
17Phase 3: frontend with reference material
Give UI agents reference screenshots and the actual component code, since words like 'beautiful' have no shared definition.
18Iterating the UI with browser annotations
Request several design variants at once, tell the agent there is no time limit, and pick from the options it shows you.
19Building a custom design system
Derive the palette from your strongest visual, keep status colors standard, and publish it as both a route and an agent skill.
20Testing the final application
Test the finished app with real parallel runs and inspect dropped claims to confirm the verification step is doing work.
Aim for a general understanding of generated code rather than line-by-line comprehension, which doesn't scale.
Glossary
Terms worth knowing.
ADE
Agentic development environment: an editor or app built around coding agents, such as Cursor, Claude Code, Codex or T3 Code, ideally with a built-in browser the agent can drive.
PRD
Product requirements document. A written description of what a product is, who it serves, its requirements and its limits, used here as the agent's single source of truth.
agents.md
A standardized instructions file that coding agents read at the start of a session, used to point them at project rules and reference documents.
Definition of done
The concrete, checkable condition that tells an agent a task is finished, such as all tests passing and one real run producing a cited report.
pstack
An open-source framework of playbooks, functional skills, principle skills, agent definitions and scripts that tells coding agents which step-by-step process to follow for features, bug fixes and shipping.
Playbook
A fixed, ordered list of steps for one type of task, so an agent follows the same process every time instead of improvising.
Principle skill
A judgment rule loaded into the agent, like 'prove it works' or 'fix the root cause', that guides decisions rather than describing how to do a task.
Sub-agent
A separate agent instance the lead agent delegates work to, often running a cheaper model in its own worktree so the lead's context stays clean.
Worktree
A git feature that checks out a second working copy of the same repository in another folder, letting parallel agents edit code without colliding.
Verification skill
A project-specific set of scripts and instructions that lets an agent launch the app, drive it like a user, and collect screenshots and outputs as evidence.
Target template
A scraping preset built for one site, such as Google Search or Amazon, that handles its layout and anti-bot defenses and returns structured results.
Map endpoint
A scraping API call that returns every URL on a site in one request, useful when the answer is known to live somewhere on a single domain.
Cache components
A Next.js rendering mode that caches parts of a page so navigation between routes only re-renders what changed.
Streamdown
A Markdown renderer from the Vercel team built to display text that streams in token by token from AI models.
“Instead of wishing for better models and better tools, sit down, improve your environment, make it as easy as possible to create code and verify code.”
contrarian take on the model race→ newsletter pull-quote↗ Tweet quote
2:07:48
“Say, hey, take your time. There is no time limit. Use your tokens. Because agents tend to try to get you the result as quickly as possible.”
practical tip anyone can try today→ TikTok hook↗ Tweet quote
2:37:48
“I myself generate about 50,000 lines of code every single day. Reviewing all of that line by line wouldn't work. It's not scalable.”
big number plus honest admission→ TikTok hook↗ Tweet quote
The Script
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
17px
metaphoranalogystory
Imagine two engineers. Same model, same coding agent, same task. Build a deep research agent.
The first engineer opens Cloud Code, types build me a research agent and hits enter. 10 minutes later the agent reports back. Done, everything works perfectly.
You open the application, click around and realize right away that half the buttons do nothing. Oh, and the report you generated, well, half the sources don't even exist. Now let's look at the second.
Second engineer, Chad. Chad doesn't start with the prompt. He defines the requirements, including the primary user flow, and decides how to prove it works before a single line of code exists.
Only then do the coding agents start building. And when they are done, every button works and every source is real. It looks slower, but in reality it isn't.
One engineer ends up with slop, while the other ships a real product. The only difference between the two is how they work. The workflow, the tools, and the environment.
And that's why in this video I'm going to show you how real senior engineers actually build with AI. We will build a real product together called Research Marshall. You ask it a research question and it scrapes the web for real -time data.
Behind the scenes a full research pipeline plans, searches, reads and verifies. The result is a clean report where every statement has a citation. And of course every run is saved so you can come back anytime.
We will do all of it from scratch. from A to Z, starting with my setup, the tools I use, the models I pick and how I use my voice to move faster. From there we get into my workflow, how I plan, how I prompt and how I split the work across multiple sub -agents.
Next comes the PRD, the single source of truth that enables my agents to work autonomously. Then we build the application. You will see how I make my agents verify their own changes before they tell me that they are done.
After that we will dive into the UI and honestly it's the hardest part of the whole build. Coding agents write backend code all day but ask them for a design and you get AI slop in return.
So we will give them something valuable to work with. A real design system. And since one shots don't work we will keep iterating until the result is perfect.
The whole way through I will show you exactly what's going on in my head, how I think, how I reason, how I break up a problem and most importantly how I review the code before it ships. So, are you ready? Well, grab a coffee or two, stay hydrated, and let's dive into this huge video.
Alright, let's get started with the prerequisites needed for this video. And the first one is an ADE, or in other words, a Gentic Development Environment. And I would suggest to pick one of the big players out on the market.
As an example, Cursor, Cloud Code, T3 Code, Codex, or maybe even Devon. It does not really matter, because most modern ADEs are good enough. they will create good enough results and like the differences are relatively marginal.
Now if you decide to pick an ADE which is not listed here which is fine by the way then please make sure that it has a built -in browser the agent can open to create screenshots and check out the result visually but at the same time also steer, click buttons, navigate between the different routes and all of the fancy stuff.
Now why is that needed? Well later on once we get started implementing code I will show you my AI coding workflow.
And one super essential step will be the verification step. We will give coding agents the ability to verify their own work. I know it sounds a bit funny, but trust me, it works super well.
And if an agent cannot see its own work, then it cannot check it. Essentially, it's flying blind, which you don't want. I personally will use T3 code in this video with the cloud code harness.
It's my preferred combo, my preferred ADE. I also created a super cool video on T3 code, which you can check out after this video.
As a second step, I want you to now think about all of the models you used in the past, let's say, four to six weeks and think about the individual characteristics. Now you might say, Jan, what do you mean by that? Well, every model is kind of different.
I will give you an example. Haiku 5 .5 is a super cheap model. It's super fast.
It's a great researcher, but it's not a good implementer because it's not super smart.
how should I say, vision. I hope that makes sense. Fable 5 .2, on the other hand, which got released about three days ago, is super smart, super capable.
It also understands what like good UI means, but at the same time, it's not very cheap, it's kind of expensive, and it's also not the fastest model out on the market. And the reason why I want you to think about the characteristics is because the coding workflow, which I will show you in a second, requires three different roles.
Elite agent, the smart, expensive, one which plans and reviews, the implementor, it's the workhorse which writes all of the code, and then finally also a researcher which is the cheap, fast one which reads and then also searches. In my case specifically, Fable 5 .2 is the lead agent, Opus 5 .5 is the implementor, and the researcher in most cases is Sonnet 5 .5, but for smaller things Haiku 5 .5 is also enough.
And I will also explain my model choice more thoroughly later on. we get started with the actual workflow. And finally, there's one more prerequisite and that's that I want you to use a voice dictation software tool.
It does not really matter. Essentially, I want you to talk instead of type. And yes, I know, keyboards are super cool.
They sound super good. Oh my god. But still, they create a lot of friction.
Using your voice is faster, it's easier, and it also allows you to express yourself more thoroughly, which is super important. You will also see that later on. You can use the built -in voice dictation feature in your ADE.
For example, T3 Code recently added one. Cursor has one. Cloud Code.
You can also use a dedicated software. That's what I will do. I use something called Willow.
Not sponsored, by the way. Or you can even use ChatGPT. It does not matter.
So now you might say, okay, Jan, we looked at all of the prerequisites needed for this video, which means let's get started with the project. Let's prompt away. Well, no, that's probably the biggest mistake I see people make every single day.
And let's come back to the general or to the original idea of this video. How do senior engineers do it? Do they sit down on a, what is it, a random Saturday and say, you know what, I recently read that there was Vercel CEO is a billionaire.
Let me recreate Vercel. Vercel 2 .0. What do I need to do?
Hosting. Okay. Serverless functions.
A bit of support for Python and JavaScript. What else? Load balancers.
Yeah, it's relatively simple. Well, no, that does not work. I wish it would work that way, but it doesn't.
The first step is to define your idea. And if you watched my last video on system design, then this should already be quite familiar to you. You want to define your idea, including functions.
functional requirements, non -functional requirements, what should your application offer, what should users be able to do. You should have answers for all of the important questions. You want to get a general or a very good idea of what you are trying to achieve.
Now, here's the thing. Of course, you don't have to have an answer for every little edge case. Nevertheless, you want to still be kind of specific.
This is what I want to do. This is what I don't want to do. Now, in our case, we want to create research marshal and the core idea.
is that you or a user asks a research question. What's an MCP server? How does an MCP server work?
What's inference? Stuff like that. And then our application generates a report where every claim is backed by a quote from a real web application.
The New York Post, Financial Times, but also like specific scientific websites. And this means the input from the user is one question and the output from our application is a cited markdown report. So now we...
defined our idea. Nevertheless, we didn't really look at the primitives, the core primitives. The first one is live web data.
Sources come from the web at the moment you ask. Nothing is hardcoded or made up. So we don't want to use the training data provided by our LMs simply because it has a certain cutoff date.
One year ago, two years ago, five years ago, who knows? We want to get the freshest and newest data possible. As a second step, every claim has to have a quote.
A claim only exists if its quote is found on the page it came from. The report can only cite claims that exist.
For that, we will create a verify quote function to make sure that all of the data we kinda cite is actually real. Thirdly, a hard budget. At most three rounds, eight pages per round, and one map.
Then it writes the report with what it has. Now you might say, Jan, what does that even mean? Well, essentially, we will start with round one.
Round one will allow the agent to scrape at most eight pages at once. If these eight pages don't have enough data, enough, you could say, information to answer the question, then we continue with round two.
Again, eight pages in total. If that's still not enough, we then continue with round three. If three magic rounds are not enough, well, then it is what it is.
We will just have to answer the question as is. Finally, failures are visible, blocked pages, empty searches, and dropped claims are shown on screen, never hidden. And for that, we will have three states, failed, skipped, and also dropped.
So now we defined our idea and we also looked at the four core primitives. Nevertheless, we are still not finished because there's one more thing missing, which is we want to also give the agent hard requirements. What do I mean by that?
Well, if you tell the agent, hey, this is in general what I want to create. These are all of the features. Get started.
Well, then the agent will just find ways to improve the application forever. Hey, let's implement authentication. Let's implement role.
and permissions, let's implement MFA, whatever it might be. And this necessarily isn't a bad thing, don't get me wrong, but in our case, we want to have a hard stop. Hey, this is what I want, this is what you should do, and here's where you should finish.
That means no accounts or login, no authentication, no authorization, no crawling of whole sites, and also runs locally for one user. And then there's one more thing I want you to always also think about, which is the primary user flow. user do from A to Z?
Or in other words, what should the user see? In our case, everything starts with the user asking a question. What's an MCP server?
What's inference? How does everything work? After that, our user should be able to watch all of the steps made by our agent.
So the live steps. Hey, we are scraping website one, two, three. Hey, we are now on round two.
This scrape failed because of reason XYZ. We want to be as open as possible. Step three is for the user to read the report.
report which our application and with that agent generates. And we want to also offer clickable citations. It should be as easy as possible to understand where the information came from.
Finally, the user should also be able to come back. If they do a hard refresh, let me show it to you, then the data should stay persisted. We want to have a sidebar which will show all of the past queries the user made.
Essentially, we need a database, correct? Let me now ask you the following. Do you understand what we are trying to build?
Now you might say, Jan, what a stupid question. Man, we just defined our idea. We looked at the requirements, the primary user flow, what we are not trying to build.
Of course, I understand what we are trying to build. Well, that's good. I'm just making sure.
But here's what I want you to think about. Are you the one who's going to write all of the code by using your keyboard? No, probably not.
I mean, if you're watching the video. then definitely not. Who's going to write the code?
It's our coding agents. And if I now go back to T3 code, for example, what do we have inside of here? Well, I created a research marshal project, which is an empty directory.
And if I would now say, hey, I want to create research marshal, implement all of the needed stuff, the agent will say, Jan. What do you mean? What's research martial?
What should I do? And that's the problem we have right now. We, as the humans, understand what we are trying to do.
But our agent is still kind of in the blind, right? It has no idea what research martial is, what we already talked about, all of the primitives. All of that is not known yet.
And that's exactly why I want to now take our idea definition, the diagram, and convert it into a PRD. Your coding agent should have as much context as possible. You want to create a rock -solid foundation which your agent will be able to use whenever it has questions?
Let me maybe break it down even further. What is a PRD on a basic level? It's a product requirements document.
It answers questions. It explains what the product is, what it should do, what the requirements are, what the features will be. And what we want to do is let our agent always look at the PRD whenever it has questions.
And this creates a rock -solid foundation and all. so creates autonomy with guardrails. Instead of the agent having to ask you, hey, do we want to implement authentication?
Should I already host the application? Should I now do XYZ? The agent will rather be able to look at the PRD and only if the source of truth does not have an answer to the specific question, the agent will stop, break the autonomy cycle and say, hey, I'm a bit confused, please help me.
So here's what I will do. I will create a super basic script. screenshot, then I will attach it inside of here in T3 code.
And one thing I want to also do right away is change my model. Instead of using Opus 5 .5, I will use Fable 5 .2 with high reasoning effort. And the next step is already to create a high quality prompt.
So let me quickly explain my prompting strategy. In the past, I was a very big fan of creating long and very detailed prompts. I essentially engineered prompts.
And that sounds I wouldn't really recommend anymore. Because what's the problem with a very detailed and long prompt?
First of all, it takes a lot of time to create it, but that's the smaller issue. The bigger issue is you constrain the agent. Models like Fable 5 .2, Sonnet 5 .5, Opus 5 .5, GPT -6 Astra, all of these models are smart.
Give them the ability to think, give them the ability to try things out. If you constrain the model right away, then it will never be able to use its full power to bring you to a new level, give it freedom.
So in a second, I will apply the same logic to my prompt. I will make my prompt simple. Some people might even say bad, but that's by design.
I want it to ask me questions. So I will now use my voice dictation tool and say the following. Look, I'm currently working on an application called Research Marshall, and the core idea is quite simple.
A user asks a question, a scientific question, an open question, a research question, whatever it might be, and then our application generates a report where every claim is backed by a quote from a real webpage. Essentially, the input is a question from the user, and the output is a cited markdown report. So here's what I have done.
I uploaded an image of a diagram I created, which explains what the product is, what the requirements are, how the primary user flow looks like, what I don't want to build, all of the necessary, you could say, like, data entry points. Now, still, the diagram is not perfect. It's not detailed enough, probably, for you.
So that's why I want you to ask me up to five questions. Like, look at the diagram, think about it, reason about it, and then ask me five questions. Also, here's the thing.
Let's right now not think about the tech stack and the individual technologies. That's something I want to dive into a bit later on, and I will also create a separate tech stack file for that. Ah, and here's the thing.
I did not even mention what the goal is. I want to create a PRD, a high -quality PRD, which I will of truth as a reference point.
Now, looking at this prompt, you might say, Jan, it's super basic, it looks fine, great, let's continue. But I want you to analyze the prompt a little bit, because I did something quite interesting. I added constraints.
What do I mean by that? Well, whenever creating prompts, I always do three things. First of all, I state my goal.
Hey, I want you to create a PRD, which I can use as a reference point. Now, in this case, the PRD or the goal was mentioned at the end, but that's fine. The order does not really matter.
As a second step, I also mentioned my constraints. Hey, I want you to look at the diagram. I want you to think.
I want you to reason. I want you to ask questions. And I also mentioned, or yeah, I included the definition of done.
Now, the definition of done in this case is kind of my goal, create a PRD, ask me questions. But if I would now integrate a feature, then the definition of done could be, for example, hey, the feature should work, check out the browser, see if the user flow works. stuff like that.
These are like the three subsections every prompt should include. And that's also how my prompting strategy works. I don't try to engineer the prompt.
I don't try to answer every edge case. All I'm trying to do is say, this is my goal. These are the constraints.
And this is the definition of done. That's all. Nothing more is needed.
And another thing you will see here is that I did not add any skills. I did not invoke any skills manually. And that's because it's not needed anymore.
And with that out of the way, I will now click on enter. Our agent is now finished.
It has a few questions for us. And here you can now do two things. You could either go through all of the questions individually and think about them and then answer them, or you can do the following.
I will now say, thank you for the questions. I want you to now rewrite the questions a bit by giving me like points or options A, B, C, and D, for example, and also give me your own recommendation. And now you might say, Jan, why do you want like these options A, B, C, D, and also the recommendation given by the agent?
Well, I want you just understand what options I have and I want you also know what the agent specifically recommends. And that's also what you will see here. We have five questions in total with four options each and always also a recommendation given by the agent.
Now, to answer these questions, there is no magic trick or anything like that. Like literally look at the questions, look at the options, look at the recommendation and then pick whatever works best for your application.
Don't blindly trust the agent. Don't just use all of the recommendations because they are also often incorrect. Yes, it's true.
Not every assumption made is correct. So please take the time, review everything and answer everything. Now to save some time, I already did that and I went quite into detail because that's my job.
I'm the senior engineer here and now we have a beautiful PRD. Let's quickly check it out. What do we have inside of here?
Research Marshall PRD, problem, product, user. success criteria, terms, that's very important because often you will see that an application will have multiple terms which all mean the same thing. Organization, workspace, tenant.
Don't do that. Every application should have one term which explains one feature or one primitive. If you would now work on multi -tenancy, then pick organization, tenant, or workspace.
Don't pick all three. What else do we have here? Requirements.
This all looks cool. Budget. Visible feature.
This is perfect. So, we can close this again and I will also say the following. Look, the PID is now finished.
Here's what I would like you to also do. Create an agents .md file. A super base.
one because we will update it later on and reference the PRD file in the agents .md file. I want my agents to always know, hey, this here is your reference point. This is the foundation.
If you have any questions, look at the PRD. And why do I want to create an agents .md file and reference the PRD in the agents .md file? Well, it's relatively simple.
A PRD file is not a standardized concept for agents, at least. We normal people know what a PRD is. The agent should know.
what a PRD is, but it's not standardized. An agents .md file is standardized. So let me quickly check it out.
Inside of here, we now have an agents .md file, research marshal, source of truth, prd .md. If you have any questions, then head over to the PRD. Perfect.
That's exactly what I wanted. So now I will also say, this is perfect. Here's what I want you to now do.
Create a GitHub repository, make it private, commit everything, and push it. And for the name, call the repository research marshal or something like that.
We now have a new repository as you see here and everything is now finished. So with what should we now continue? Well we want to now start implementing our application.
We want to first of all set up the foundation. Yes I mean we already set up the prd and agents .md file but we need to also set up our tech stack right. We want to for example I'm not sure yet install next .js typescript.
We need to set up all of the boilerplate. We need to then work on the back end.
And finally, we need to work on the front end. But before we do that, you might say, Jan, if we go back to your idea definition and to be specific, if we look at the requirements, then you mentioned here that we want to fetch live web data from big players, like for example, the New York Times, Financial Times, maybe even Yahoo.
Yes, that website still exists. And then you want to essentially take the scraped data and feed it into an LM to create or generate a report with all of the needs. citations, correct?
Yes. But how do you want to get that data? As an example, this here is the New York Times.
Do they provide an openly accessible API or do they allow you to just download the HTML of articles? Well, no, of course not. They actually want you to go to their website, look at all of their advertisements, and then read through the article.
But that does not work for us. We need the HTML or maybe even better LM Ready Markdown and we need to feed it in into our LM.
And that's exactly where scraping comes into play. So scraping normally is super, super complicated because as already mentioned, these big websites have, I will be honest with you, very good anti -bot protection setup. They will instantly block your IP.
You will never be able to access the website again. They have very good bot detection. algorithm setup.
A page can also come back empty because as you already know, there are two rendering types, server -side rendering and JavaScript rendering. If it's a JavaScript rendered website, well, then good luck. Also, raw HTML is pure noise for an LM because what do we have inside of here?
We have a nav bar, we have a lot of links, images, we have a footer at the bottom. You don't want to feed that into an LM. An LM has only a set amount of tokens and you don't want to use them for just pure garbage, right?
And at the same time, every site also normally needs its own parser because the HTML returned by the New York Times has a completely different structure than the one provided by Yahoo or the Financial Times. And finally, failed requests also cost you money every time and it gets expensive very quickly. Now you might say, Jan, What do you mean?
What should we now do? These websites don't provide any publicly accessible APIs. In contrary, they want to block us.
They want to disallow every request. So what should we now do? What about our application?
We need live web data. Well, that's exactly where Decodo comes into play. Decodo is today's video sponsor and they are the leading AI -ready proxy and scraping solutions provider.
The most efficient way to test, launch, and scale your web. data projects. Isn't that exactly what we want to do?
Yes. And in this video, we want to use their newly released WebData API. And the WebData API is there to scrape, crawl, map, and search the web.
The Kodos WebData API is a single API that scrapes pages, maps sites, crawls them, and also returns search results with proxies, JavaScript rendering, and also anti -bot handling built in. Oh my god, it's the solution to our problem. Isn't that cool?
So if we again go back to our nice little like, you know, list of problems, then you will see here that Decodo literally fixes every single one. Your IP gets blocked? Well, not with Decodo.
Decodo provides 125 million rotating IPs, residential IPs, data center IPs, and everything is rotated for you. Then we had a huge issue with bot detection. Guess what?
Kodo has anti -bot protection built in. Fingerprints are managed for you. What else did we have here?
The page comes empty with JavaScript rendering. Well, JavaScript rendering is built into the Kodo. Zero issues.
Raw HTML is nice for a model. Well, with the Kodo, you can get normal HTML, but you can also request nicely formatted Markdown or even JSON if you want to. We, for example, in our case, will use Markdown because it's the perfect format four LMs.
As you already know, in theory, every site needs its own parser. Well, with Decodo, first of all, you don't need any parser. You can request any website, but they also have ready -made templates for Google, Amazon, even ChatGPT, whatever you need.
And finally, you only pay for successful requests. A failed page costs nothing, which makes our whole pipeline fast, cheap, and secure. But now you might say, Jan, this here literally sounds and also looks perfect.
Nevertheless I'm still a little bit confused because inside of here you said scrape, crawl, map and search the web. What exactly do these you could say primitives do and how exactly does everything work? Great question.
So with the Codo you get four endpoints in one if that makes sense. So one URL, four endpoints. Let me zoom in a bit.
First of all they have this scrape endpoint and it essentially provides or returns clean data from literally any URL. Turn any web page into structured output. HTML, JSON or Markdown without building or maintaining your own parser.
Secondly the map endpoint. URL discovery for any site. Provide a URL.
and get back every page on the site, crawled links plus sitemap in one call. The third endpoint is the crawl endpoint and it allows you to do autonomous site crawling. Point it to a site and get up to 10 ,000 pages scraped automatically.
So instead of just scraping websites individually, you can scrape a batch of websites, 10, 50, 10 ,000, it does not matter. And finally, there's also the search endpoint and it's a super cool endpoint because it returns real search results, query the live web and get back organic results, yes, you heard right, not sponsored results, built for agents that need real -time context, not just static page data.
And in our case, we will use a combination of scrape, map, and search. But now you might say, Jan, this here kinda sounds too good to be true. Decodo is probably super overpriced, right?
No. Decodo has one of the most generous free tiers out on the market, and you even get $5 in free credits, which translates to about 10 ,000 requests.
And of course, no credit card is required. And with that out of the way, we can now finally get started. So please head over to my YouTube description underneath the like and subscribe buttons and click on the first link to get redirected to the Codo or scan the QR code you can see on the screen right now.
And you know what? I even negotiated a coupon code for you guys to get 10 % off. So using the code you can now see on the screen you will get 10 % off instantly.
So click on the first link, apply the coupon code and you will be all Gucci. But now it's already time to get started. Now if you don't have an account yet, then don't worry, just click on the start for free button and sign up.
It's completely free. But since I already have an account, I will now continue with Google and log in. I now got redirected back to the dashboard and as you see here I made quite a few requests.
requests, and the success rate is 97%. Damn. So here's what I want you to do.
Let's head over to the sidebar, and in the sidebar, you will find a WebData API section. Let's click on the WebData API button, and here we already have all of our endpoints. Scrape, search, map, and crawl.
And you know what? We can, for example, scrape my website, which even has antibody filtering set up. Inside of here, I want to request markdowns.
but we could now also update it to HTML or XHR. I can even set advanced settings for the location, proxy pool, language. Like as you already know, big websites kind of serve different content to different locales, different countries.
A user from Germany will get German copy, a user from the US will get English copy. So it's quite cool that you can set all of these, you could say filters or settings right inside of here. But what I will now do is click on scrape and the coder will go to my website, request all of the HTML, scrape all of it, and then convert the HTML into clean markdown.
And that's exactly what we have here. Stop procrastinating, ship your ideas. And what is this?
Is this normal HTML? No, it's clean markdown. Markdown which I can take and feed right into my LM.
We can also test out the search endpoint, which allows you to search for relevant pages using query keywords. We could now search for laptop. And here, let's return JSON.
I will click on search. And now we will get relevant results. And that's what you will see here.
Search parameters, query laptop, locale n, and then geolocation us. And these are all of the organic results. As already mentioned, you don't get sponsored results.
You only get organic results. And that's what you want. And one more endpoint, as you already know which we will use, is the map endpoint, which allows you to discover a site's structure with up to almost 150 ,000 URLs in one call.
So let's test out the Decodo website. How many subpages do they have? And here's the result.
Content, links, oh my god, they have quite a bit. So this looks all beautiful. And as a next step, we can connect our local environment to Decodo by installing the MCP server, the CLI.
and also the agent skills. So here's what I want you to do. Again, click on the first link.
Let's head over to the Decoder website. Inside of here, I will click on data for AI and let's start with the, you know what, CLI. Why not?
So I will scroll a bit to the bottom and here they already provide all of the needed commands. I will copy them and I will head over back to T3 code. Inside of here, I will open the built -in terminal and paste what I just copied.
As you see here, I need to now authenticate. myself, paste your webdata API key or auth token. So let's head over back to the decodo dashboard and on the top right you will find the API key.
I will copy it, bum, let's go back, and I will paste it inside of here. Everything was successful, setup complete, configuration saved under the following path. Let's now continue, step one is finished, the CLI is installed, the next step is to install what?
Let's do the agent skills, why not? And these agent skills are an open source set of skills that teach AI coding agents how to use the coder for web scraping.
And that's exactly what we need, right? Because our agents might be unfamiliar with the new web data API. We want to provide as much context as possible to our agents.
Don't let them assume things for you. So here's what I will do. I will open this link.
This is the GitHub repository for the agent skills. Let's scroll a bit to the bottom. And these are the needed installation commands.
So here's what I will do. I will copy the first command here, plugin marketplace add decodo agent skills. I also already opened a cloud code instance in my terminal and I will press on enter.
So we now added the plugin successfully and the next step is to also install the skills. I will copy the second command and paste it inside of here. Here I want to install for myself, meaning the user scope.
I will click on enter and everything is now successful. Let's close the terminal and what do we want to do? as a final step?
Well, we want to install the MCP server. For that, let's head over back to the Decoder website. I will click on Data for API, MCP Server, and here they already have a guide on how to do a local install from GitHub.
You can clone the repository or in other words, follow all of the necessary steps inside of here. But what you can also do is paste the following command into your Cloud Code instance, so cloud mcp add transport http, the endpoint for the MCP server, and for the scope, I selected user.
I will now click on enter and this will add the MCP server into Cloud Code, our harness of choice. The MCP server has now been installed, but we need to authenticate ourselves. So I will open up a new Cloud instance.
Yes, I trust the workspace, let's say slash MCP. And inside of here, we have the Decodo MCP server and it needs what? authentication.
So here's what I will do. I will click on enter. I want to authenticate myself.
I will now get this little pop -up. Allow the Codo MCP to use your scraping API key. Yes, that's absolutely fine.
We will only use it locally. So I will say allow access. And if I go back to Cloud Code, then you will see inside of here that authentication was successful.
Let's head over back to T3 Code. I will use my voice dictation software and just verify that everything was set up successfully. Hey there, could you please make sure that the Decodo MCP server is installed, that the skills are installed, and also that the CLI is installed and also authenticated.
And yes, everything is set up. I tested each piece and they all work. MCP server, skills, and the CLI.
So now you might say, Jan, since we are now finally finished with everything, we can probably continue by generating some beautiful code. But how should we get started? Should we go to T3 code and say, hey, agent, I'm kind of bored.
I want to scroll a bit on TikTok. So get started with the implementation. Do everything from A to Z and let me know once you are finished.
Or should we maybe break everything down into smaller scopes, into like a foundation block, a back -end block, and then a front -end block? And how should we like go ahead with the implementation process? Should we do things in parallel, sequentially, in one run?
What's the correct, how should I say, workflow, pipeline, whatever you want to call it? Well, here's the thing. Let's start with the basics.
How do we generate code? I mean, okay, that's a stupid question. But when you create code, like it has to build on top of something, right?
We want to create a web application. We need a rendering layer. We need a server.
We need an ORM. We need a database. We need some sort of like LM provider, an AISDK.
This means before we generate any code, it would probably make sense to first of all, like set up the boilerplate, right? Well, yes. So here's what I would suggest.
doing. The first step should always be the foundation. Now, we already did about 50%.
We created the PRD file, we created an agents .mt file, and at the same time, we now also set up the coder. We installed the MCP server, the skills, and also the CLI, and I guess we also already looked at the API key. We have access to it.
And the second step of this foundation layer is to now set up the boilerplate, your server, your database, your ORM, everything I already mentioned. Once that's finished, we will continue with the back end and to be specific, all of the business logic.
And once that's done, we will be able to continue with the front end. In total, we will create three sessions. Why is that?
Well, I could, of course, say, hey, do everything in one run, like essentially a one shot, and it probably will work. But here's the thing. Your agents have a certain amount of tokens and the fall off is quite steep.
What do I mean by that? Well, let's imagine the following. We have an Y axis.
We have an X axis. Inside of here on the Y -axis we have performance and on the X -axis we have tokens. So let me spell it correctly, tokens.
Now here's the thing, for example, Opus 5 .5 has 1 million tokens, right? Or you have access to a 1 million token context window. So if we now start off fresh, so zero tokens, then you have exactly 100 % of performance.
The model is super capable, super smart, super performant, everything you need it to be. At some point you will hit 100 ,000 tokens. 200 ,000 tokens, 300 ,000 tokens, maybe 400 ,000 tokens, you will quickly realize that there's like a somewhat drop -off in terms of performance.
And it's fine, like it's relatively linear as you see here. But at some point, let's say around 500 ,000 tokens, the performance degradation will be quite exponential. And that's why I never recommend to work with one -shots.
Sure, it's cool to test the model, but it's not what senior engineers do. Senior engineers work with scopes. So our scopes are...
foundation, back -end, front -end. If you would now work on an existing application, then one scope could be authentication or billing. Normally, that's still a bit too big.
Billing is quite a complicated topic. B2B, B2C, entitlements. Therefore, I would normally break the scope down even further.
MFA, OAuth, AuthZ, AuthN, whatever it might be. But yeah, that's what I want you to remember. Tokens are not linear.
The performance degradation is not linear. Therefore, keep the scope small so that you are able to use the most performant token. available.
So, let's quickly again think about all of the technologies. So, in terms of our idea, right? A user asks a question, we generate a report, and then we return the report.
Let's think about what a user has to do. A user has to ask a question. We need some sort of front -end interaction layer, right?
We need to style it. For example, next .js with React, Taewon CSS, shared CNUI, some sort of form library, sort, all of that. Then, we generate a report.
How do we do that? We need to fetch data. For that, we use the Codo.
We use the Codo to search, scrape, and map. Then at the same time, we need some sort of AI model. Because the scraped data is cool, don't get me wrong, but we need to also convert it.
We need to analyze it. And for that, we need an LM. So we will feed it into an LM.
For that, we need a provider, Anthropic, OpenAI, Gemini, or Google, whatever it might be. We probably need an AI SDK. For example, the one provided by Vercel.
the back and we could use Next .js. We need a database to store all of our data because inside of here we said the user should be able to come back and also view the history of past runs. And since we now have a general idea of the text stack, I will now continue by creating a prompt.
So let me use the voice dictation tool and in this case we can use Opus 5 .5. It's fine. Look, so I want to now continue with phase one of our build out, which is to finish off with the foundation.
Meaning I want to now already set up all of the boilerplate needed. I want you to think about the tech stack and the technologies I want to use. So a user should be able to ask a question.
This means we need some sort of front end interaction layer. I want to use React and Next .js for that. Maybe also Zord, then React hook form, shared CNUI, Tailwind CSS, stuff like that.
Then after that, the next step is to generate a report. So we want to scrape data. For that, I want to use the coder.
And with the coder, we will use the search endpoint, scrape endpoint, and also map endpoint. After that's done, once we have all of the data, we will have to feed it into an LM. For that, I want to use Anthropic, so probably Sonnet 5 .5, but let's debate that.
And since I want to make it as simple as possible, or since I want to make the workflow doable, I want to also use the AI SDK. Now, looking at the primary user flow, we need to also save everything. For that, let's use SQL.
Qlite. It should be good enough. Let's use an ORM.
I want to use Prisma instead of Drizzle. What else do we need? I already set Next .js for the server.
And here's the thing. Let's use the app router. Let's use cache components, TypeScript.
I don't need any source directory. Is there anything missing? I guess it's it.
I probably missed something. So let me know if there's something that you want to add or adjust. So essentially what we have here is the modern web developer stack.
Now this prompt isn't finished yet because I don't want the agent to implement anything. So let me say the following.
I don't want you to implement anything right now. Instead, I want you to essentially think about all of the like technology choices I made. Let me know what you think.
I want you also create a tech stack file and then reference it in the PRD file. Maybe we will even have to update the PRD file and the agents .md file. And maybe also before we do that, again, make sure that I or that we like have the complete set of the technologies we need for this approach.
Before I now click on the submit button, there's literally only one more thing I want to do, and that's invoke a skill. So inside of here, I will say slash psterk colon potato mold. And now you might say, Jan, hold up, wait a minute, something ain't right.
So instead of invoking a skill called, I don't know, architect, research, create PRD, something like that, something fancy, right? You decided to invoke a skill called, wait, let me look at it. Potato mode?
What do you mean? What does this skill, what do potatoes have to do with real engineering work? Well, to be honest with you, not much.
So therefore, let me explain myself, let me cook for a second. Potato mode isn't just some random skill I created or thought about yesterday. It's actually the main entry point into my AI coding workflow.
I use it every single day. It's the skill I use literally whenever I create a new session. I always started with potato mode.
And now you might say, okay, Jan, I'm kind of confused. So potato mode isn't a random skill. It's actually like the entry point into your AI coding workflow.
But what is your AI coding workflow? How does it look like? And how does it work?
Well, these are all great questions. So what you should know is that I use something called PSTARC. And PSTARC is an AI coding agent framework.
Now, if PSTARC sounds familiar to you, then that's because I already created a video on it. But if you don't know P -Stack from the video, then probably because it was created by Lauren Tern, a software engineer working at SpaceX AI, or in other words, SpaceX SI.
Anyways, she's like the author of this viral tweet. Here's how I shipped 2 ,500 PRs last month to production. And as you see here, it almost has 5 million views.
And spoiler alert, she used P -Stack. And now you might say, Jan. What is P -Stack?
How does it work? How do you use it? Great question.
So on a high level, P -Stack is a framework for AI coding agents. It provides a ready -made set of workflows and rules that tells the agent how to plan, build with you and verify its work. You describe what you want and it picks the right process for the job instead of letting the agent improvise.
And that's super important because here's what I want you to realize. What are we? We are software engineers.
AI engineers, however you want to call it. And our job is relatively simple. It's to implement features, do bug fixes, investigate, break code, stuff like that.
But the process is always the same. Let's say you want to fix a bug. What's the process?
Step one is to make sure that the bug even exists. It's essentially to verify the bug. Then it's to investigate the bug.
Then it's to implement the bug. Then it's to make sure that the fix works. And then it's fine.
to commit everything, wait for CI to become green, and I guess merge the PR. That's the general process on a super high level. And what most of us do is repeat this process every time.
We essentially babysit our agents. Hey agent, I have a bug, please make sure that it actually exists. Oh, you now found the issue?
Now please fix it. Oh, you fixed it? Now please make sure that the fix works.
Oh, you're finished with that? Now push the PR. And that's something we have to do every single day for literally any issue, any task.
Again, implementing features, fixing bugs, investigating code. So I hope that now makes a bit more sense. And now you might say, okay, Jan, so P -Stack is a coding framework.
As you already mentioned here, it gives us the right process for the job. It's essentially like a pipeline. Nevertheless, I'm still a bit confused.
What are the core primitives? What are the internals of P -Stack? What's potato mode?
Well, these are all great questions. So on a high level, P -Stack has or builds on top of five core primitives.
Playbooks, functional skills, principal skills, agent definitions, and also supporting scripts. So the first core primitive is playbooks. And playbooks store a step -by -step list per task type.
Feature, bug fix, investigation, shipping, committing code, evals, whatever it might be. There are 23 in total. Let me quickly show it to you.
So this here is the GitHub repository for P -Stack. And these are also all of the playbooks which currently exist. And let's open one, for example, the feature playbook, because it's one of the most used playbooks.
It's used whenever a feature gets implemented. And here we now have a pipeline with the needed steps in order. Step one, trace the affected subsystem.
Step two, use the architect skill. Step three, write the following. Then step four, step five, and so on.
And that's also what I meant by having a process. Instead of the agent improvising and just doing something depending on how it feels, right? Today I will verify the code and tomorrow I won't do that.
And instead of you also having to babysit the agent, there is a clear order. Do this, then this, then this. And once you are finished, only then, I don't know, hand over to me, create a PR, whatever the playbook mentions.
Then the second core primitive is functional skills. And these are needed for operations that do the work. A second ago, I showed you the feature playbook.
What does the feature playbook like tell the agent to do? It should invoke the architect skill. quickly show it to you.
So these are all of the functional skills, architect, arena, automate me, blast radius, bro, correct. Let's open the architect skill. And what does this file do?
Well, it is a file with instructions, sketch types, signatures, and module structure before code, then stay in the loop while implementation fills in. So it supplements the playbooks. The playbooks explain the order, the functional skills explain how to do something.
Then we also have principle skills. And principal skills work a bit differently because they don't really like tell the agent how to do something, but rather they explain like core, you could say, engineering practices.
So judgment rules for the agent. Prove it works. Fix the root cause.
Because as already mentioned, agents are lazy. They don't do everything. If you want to fix a bug, the agent will say, ah, I found a bug, let me now fix it.
Instead of diving into the root cause and fixing the root cause. Let me quickly show you. the principal skills.
If you go back to the GitHub repository, then all of the skills with the prefix principal are principal skills. Attack the premise, boundary discipline, build the lever, encode lessons in structure, exhaust the design space. And principal skills also apply throughout.
They are automatically invoked. Then there are also agent definitions, and they are roles for delegated work, potato, agent, common, sicko, and finally there are also supporting scripts, so local tools for PRs and plans.
Watch PR, ship PR, orchestrate, work tree audit. So instead of the agent spending its valuable tokens on creating random scripts, there are already pre -built scripts which can be used right away, which also work super reliably. And this means whatever ADE you use, whatever models you use, the workflow stays always durable, which is exactly what you want.
So now you might say, okay, Jan, so these are the five core primitives, but how exactly does now Potato mode work? Or in other words, how does the workflow look like?
Because you now defined in T3 code the skill or you invoked it, right? You invoked potato mode. But what does the skill now do?
Do we still have to say, hey, use the feature playbook, use the bug fix playbook? Should we also say, hey, use these functional skills and these principal skills? Well, no.
The only thing, the only skill you have to remember is potato mode. Now, sure, it's a good idea to understand what playbooks are available, what functional skills. are available, but it's not necessarily needed.
So let me quickly show you the workflow. You start with your task. We already did that.
We defined our goal, our constraints, and also the definition of done. The next step is to then use potato mode. I also did that.
And potato mode is that you take the request in, interpret it, understand it, and then route the request to a playbook. So it's essentially what? An internal router.
It takes the request, vroom, it then understands it, and then says, hey, this is maybe a feature implementation, let's use the feature playbook. Or this is a bug fix, let's use the bug fix playbook, etc.
So once the playbook is then selected, we have the steps and the order. So if it's the feature playbook, then first of all, understand what the user wants, use the how skill, use the why skill, use the orchestrate skill, use all of the needed functional skills, and then we do the work. As you see here, skills plus focused agents.
And what you have to also understand here is that P -Stack uses sub -agents very heavily for implementation because as already mentioned your tokens are very constrained or they are very valuable and instead of letting an agent one singular model agent do everything you want to distribute the load you want agents depending on their specifications implement the needed work and once the work is finished the next very important step is to verify and review and this here runs in the loop so the agent is finished with the work right it then hands everything over to a verify or multiple verify sub -agents and they pretty much like you could say interrogate the work.
Hey, does this work? Let me check this out. Let me run tests.
Let me use the built -in browser. And that's also why I mentioned you need a built -in browser. And if there's anything that fails, then we have a loop.
It goes back to step four and the agent says, hey, this does not work. Please fix it. And this could now run one time, two times, three times, four times, five times.
It does not really matter. The agents go as long as the code is not perfect.
And that's also how I was able last month to ship 1000 PRs. And once all of these steps are finished, so far in total, the agent then hands over to me the verified result with also all of the evidence collected. Screenshots, failures, logs, whatever it might be.
And that's also why I said right at the start, please think about all of the models you used in the past four weeks. Because with P -Stack you have roles. You have an implementer, you have a lead agent, you have a researcher.
So in our case now Fable 5 .1 is the lead. It's always the agent with which we start. It thinks, it reasons, and it delegates.
Then we have the implementor. For all of the implementors, I use Opus 5 .5. It could be one agent, it could be multiple ones.
Because as mentioned, PStack uses sub -agents. Finally, for the researchers, we want to use cheap models because sub -agents can become quite costly. If we have 10 sub -agents running, we don't want them to be Fable 5 .1.
5 .2 we want them to be something cheap something that works something that's fast and that's why you have to think yes about the model capabilities but also the cost you want to stay economic if that makes sense and in my opinion this right here is the perfect model combo the models are gigantic they are capable and depending on how you use them they can also be relatively cheap so since we are now finished with all of that i will switch the model to fable 5 .2 because i need a lead model model, which is smart, and I will then click on enter.
Look, our agent is finished, so let's quickly go through it. Next js -app -router -verdict -keep, that's good, cache -components -keep -narrow -use makes sense, tauren -shared -c and ui -zord -keep. Zord also passes the decoder responses and the model output.
And that's quite cool because what you have to realize is that the plan right here currently is to use the codos webdata API over HTTP using the native Fetch API. And the native Fetch API is not typesafe, both in terms of the input and also output.
But using Zort, we can validate the input, we can make sure that the user provides whatever we expect, but also we can validate the output and with that make the output typesafe. Then React Hook Form dropped the AirPass 1 form with one field, therefore installing a separate dependency is not really needed. It will just bloat our dependency size and yes, if the plan would be to add more forms, then we should probably install React Hookform, but it does not make sense for one tiny feature.
What else do we have here? The Kodo Search Scrape Map. Keep overplaying HTTP.
Hmm. So here's what you have to realize. We want to use the Codo in our application.
Therefore, we can't use the CLI or the MCP server. Both are there to test the Codo locally, not in production. But in production, we can also use the Codo in two ways.
We can use the WebData API over plain HTTP using the native fetch API, or we can use the Codo TypeScript SDK. And why did the agent decide to use the WebData API over plain HTTP? Let's see.
Kodo SDK has an empty readme and its last release is from August 2025. Huh, that's actually incorrect. And that's the biggest problem with agents.
And that's also what differentiates a junior engineer from a senior engineer. A junior engineer would now just look at all of this and say, okay, your recommendations make sense to me. Go ahead with the implementation.
A senior engineer's goal, on the other hand, is to make sure that whatever the agent proposes is... also true. And in this case our agent made a huge mistake.
It probably used its training data or something like that and with that decided that the SDK is probably unmaintained. But if we go back to the decoder website and if I click on get code then you will see here that we have four options to use the web data API. First of all the API using curl, using the native fetch API, whatever it might be.
Then secondly the CLI and inside of here yeah we are already installed it. It's quite simple.
You can use it locally. We also have access to the MCP server, which we already installed. And finally, we can also use the SDK.
And as already mentioned, there is a TypeScript SDK. And if we go back to our agent, it found some sort of normal SDK, which does not exist. We want to use the TypeScript SDK, which is just a wrapper around the normal WebData API, but it provides type safety, both in terms of the input, but also our As you see here, we instantly know what we have to add and we instantly also get like a type safe output.
And that's what I want you to realize here. Yes, we are using P stack, which already helps us a lot by making the result, let's say, more doable, right? The workflow more doable.
Nevertheless, it does not kind of fix every problem. Agents are still naive. They make mistakes.
They take shortcuts. So your goal as a senior engineer is not to babysit the agent. We are past that.
We are in 2026. almost 2027, our goal has now become to review and also keep the agent in check. So let me make the following comment right here.
Look, I'm not sure what you did, but it seems like for certain things, primitives, you used your training data. Because for example, in terms of the CODO, you mentioned that the SDK is not maintained. And that's incorrect.
You made a huge mistake. The CODO offers a TypeScript SDK, which is actively maintained. The last commit was created exactly two weeks ago.
So please update your research. Do that as a first step. And secondly, let's also maybe quick debate about should we use the native API over HTTP, plain HTTP, or should we use the SDK?
I personally would say that we should probably use the SDK because it automatically gives us type safety, which is a huge benefit. And we also get typed errors, minimal setup, everything that we want. Then what else do we have here?
The rest makes sense to me. AI SDK, SQLite with Prisma. For the model, yeah, I would agree.
Sonnet 5 .5 is the best model for our use. because Opus 5 .5 is too expensive and Haiku 5 .5 is too dumb, let's be honest. So, here our agent also has a list on what we are missing.
Run execution inside of Next .js, that all makes sense to me. Report rendering React Markdown with Remark GFM. Now, this is something I don't agree with because I don't want to use React Markdown, but rather I want to use StreamDown.
StreamDown is a dependency created by the Vercel team and it's a Markdown renderer designed for streaming content from AI models. Highly interactive, customizable and easy to use.
So let me copy the URL, paste it inside of here. What else do we have here? Tests.
The test. So let me again use my voice dictation tool. Look, in terms of run execution, that makes sense to me.
For report rendering, I don't want to use React Markdown with Remark GFM, but rather I want to use Streamdown. Then for lint and formatting, let's not use BEOM. Rather, I want to use ESLint.
I like it a bit more. Though, what did you say right here? Because the current TypeScript ESLint release accepts only TypeScript below 6 .1.
Ah, and we have TypeScript 7. Okay, you know what? Let's use BEOM.
Why not? Then runtime Node .js 26 and PNPM 12. That's all good.
What else do we have here? The laziness protocol, the little cuts, React hook form, and the Codo SDK. As you already know, you made a mistake here.
Not verified Prisma SQLite. So, the rest all looks... good to me.
So now, again, please check out the code, though. Also, make sure that the other assumptions you made are actually true and that you did not just use your training data. Also, let's take in Streamdown and everything else I mentioned.
Now, while our agent verifies everything by using the browser and all sorts of fancy stuff, I want you to realize something. And that's that agents, and with that, LMs, are inherently lazy. And why is that?
Well, a model wants to get your result as quickly as possible while using the least amount of tokens. An LM will always prefer to use its training data over doing a separate web search.
And that's fine for most normal things like if you ask a history question, you don't need some sort of like web search, right? You can just use the training data. If you ask a biology question, the training data is enough.
But for us developers, it's a bad thing because engineering practices, technologies, tools change every day, every hour, every minute, every second. So using or relying on training data for fundamentals is a good thing, but for specific tools, it's a huge mistake. And that's also why our agent right here said, oh, the SDK is outdated.
So let's just, I don't know, use the plain API, which wouldn't be a mistake because using the plain API is perfectly fine and maybe we will use it, we will see. But the agent did not make sure to actually give us all of the options available. And that's why I want you to always verify the output.
Don't just blindly... trust the agent. I know a lot of people have kind of become, how should I say it, without disrespecting anything.
Like, people have become too trusting. Because yes, Fable 5 .2 and all of these models are great, but they still make mistakes. And we will also still have mistakes with Fable 6 .7.
It is what it is. So make sure that you verify the claims, use your own browser, use your own hat. And with that, you will get perfect results.
So it took now six minutes to get a result. Let's see what the agent found. SDK or plain agent.
HTTP, use the SDK, I installed it, and read its type declarations. So it has all of the methods, search, map, scrape, and with that also crawl, typed errors, that's exactly what we wanted, dependencies, it only uses one, Zot4, and that's what I meant, it's just a wrapper. We could also use the API just with plain HTTP, the fetch API, and then we could wrap everything using Zot, we would then get the same result.
But why should we do that? If there's already an SDK which does everything, thing for us, and maybe even better, who knows, then why should we spend the time to create our own wrapper, which will at the end of the day do the same thing.
Then the one cost version 3 takes no abort signal, that's fine, and retries, it does not retry. So stream down, it's in, that's good, version 2 .7 .1, link safety, da -da -dum, Taywind 4, streaming, that's all cool, what I ran this time, Prisma 7, TypeScript 7, Next .js 16, that's all good, biome or biome, however you want to call it, 2 .6 .7.
So here's what I have now done. I said, so are we done? The agent said, yeah, we are pretty much done.
Nevertheless, we still need to update the PRD. So I said inside of here, update the PRD and our PRD has now been updated. So with that, we are now finally finished with our foundation.
One thing I would quickly also want to do is look at the code or what has been generated. So this is our tech stack file. do we have inside of here status one draft one i mean it's not really a draft anymore this document defines the technologies that's good what the prd fixes uh -huh stack node .js pnpm typescript framework okay ui library what else do we have here cli what else rendering stream down validation sort passes the question the kodo typescript sdk 3 .0 the three web actions that all good, next js -app -router, cache -components, so this looks good to me.
Let me now say the following, look this all looks great, so please commit and then push into main. Before I now let the agent implement everything and with that set up the boilerplate, there's one thing that we need to do, or in other words save, which is what? Environment variables, the decodo API key and also the entropic API key.
So let me say the following, before you now get started with the whole like setup of boilerplate, I want to first of all save the two needed environment variables, which is the Dekodo API key and also the Anthropic API key. If I remember correctly, there's a native T3 code tool, which you can call so that I'm able to add the needed environment variables.
So first of all, we need to add the Anthropic API key. Now I already saved it in my clipboard. By the way, you don't have to use mine because after this video, it will be gone forever.
Don't even try it. So let me save this securely. The Anthropic key is now saved.
Now we will get another pop -up and this time for Decodo. So let's head over to the Decodo dashboard. And on the top right, you will find a section with the API key or with your API key.
So I will copy it by clicking on the copy button let's go back i will paste it inside of here and also save the key securely and by the way if you want to rotate your api key you can just click on this magic rotate button so let's again go back to t3 code it will now save the two files in the dot env file and the dot env file will also automatically be good ignored so do the two keys work the entropic api answered with 200 and also the kodo answered with two hundred.
That's perfect. And the next step is with the boilerplate setup. So let's use our P stack skill to be specific.
Which one? Potato mold. Correct our internal router.
And let's again quickly think about my prompting strategy. What does every prompt have to include? Well, a goal.
Wow, who would have thought? Constraints and also what? I'm waiting.
Well, the definition of done. So, let me say the following. So, this is now all finished.
We saved our environment variables. You know exactly what I want, which means let's continue with the boilerplate setup. I want you to install all of the technologies and also set them up.
Now, here's the thing. What's the goal here? The goal is for everything to be installed and also to be working.
In terms of constraints, I want you to follow the tech stack file and the PRD. Please don't invent something new that we didn't discuss. Plus, if you want to deviate from the plan, then please let me know and be specific.
Ask me so that I can approve. And in terms of the definition of done, I want you to make sure that everything works. Now, you won't really be able to, I don't know, spin up the browser and click things.
Nevertheless, do code checks like, hey, does Anthropic work? Does the provider work? Are you able to make a call through the AISDK?
And is it also routed to Anthropic? Maybe also use the Decodo MCP server or the CLI. Again, I want you to verify.
that everything works as expected and that everything is also able to connect to each other. Now, I'm not asking you to create any back -end code. Again, we are just doing the boilerplate setup.
Nevertheless, make sure that everything is verified and is also working as expected. This looks beautiful to me. I stated the goal.
I also stated all of the constraints and the definition of done, meaning I included all of the three basic, could you say like core primitives of a good prompt? Yeah, I think so. So let me click on enter.
We will use Fable 5 .2. Did we define potato mode? Yeah, this all looks beautiful to me.
So I will click on enter. Our agent is finished. But before we check out the result and with that what the agent did, I want you first of all create and with that open a new PR.
Therefore, I will now say the following. I want you to create and with that open a PR for me. Don't turn on autofix or anything like that.
Just create, open and that's it. But with that out of the way, let's now check out what the agent did and with that created. So inside of here, it worked for 20 minutes and the boilerplate is installed and working in this work tree and nothing is committed yet.
So what were the checks? Install from the log file, passed and everything looks good. PNPM lint, no errors.
PNPM type check, passed. PNPM test, 4 out of 4 passed. So inside of here, we are currently using the test and the test is something I would always recommend to set up.
Before we had coding agents creating or setting up tests was super annoying. The test playwright. I hated everything about it and therefore my applications often were under tested if you want to call it like that.
But with coding agents testing has become super simple. You're not the one creating the tests it's your agent. So let your agent create tests.
It's super important and it will allow your application to become way more durable in the long run. Then what else do we have here?
pnpm build, past, pnpm verify stack. This is a command created by the agent itself, which is quite cool, and it also plays together with pstack, which I will also explain in a second. Then routing to Anthropic, everything works.
Oh, we even have rate limit headers, interesting. The Codo SDK, search, page, read, and map return data, and the bad key through authentication error. Inside Next .js, a temporary page, blah, blah, blah.
That's all fine. So let's quickly head over to the Dekodo dashboard. And on the top, you will find a section called usage statistics.
And today we made in total four requests. I mean, we did not do anything. Our agent created all of the needed requests.
So let's see for 100 % success rate. That's quite cool. And for example, we made a request to Pinecone, Scrape API, Rust, which is quite cool, Wikipedia, and even Spring.
And we even had some traffic, which is quite nice to see. And look at the response time. Do you see how fast everything was?
1 .5 seconds, 3 seconds, 1 second, 1 second. It was almost instant. Let's now go back to T3 code.
Is there anything else we have to look at? Scaffold default, biome ignores. That's all fine.
One decision for you. Okay. Next step, append the block to agents .md on every run.
That tells the agent you read the bundled next .js. Yeah, that's actually quite good. I recommend keeping it.
Yeah, I also recommend doing that. Then three behaviors you should know. Missing key, abort, decode or markdown.
That's all fine. And another thing I want you all to quickly mention is that P -Stack was very essential in this process. What do I mean by that?
Well, if I open this, how's it called, dropdown, then you will see here that we used a playbook. And the playbook actually wanted us to delegate the work to a sub -agent. So what did we do?
do? Well, we had a subagent running which implemented all of the code. Which model was used?
Opus 5 .5. Why is that? Well, Fable 5 .2 is our lead.
Opus 5 .5 is our implementer. And who's our researcher? It's Sonnet 5 .5.
One very core -like principle of P -Stack is parallelism. If work can be done in parallel, then we use subagents. Even if it can't be done in parallel, we still use subagents because we have model roles.
And even though Fable 5 .2 could have implemented everything. It did not do it.
It saved its tokens. It handed over the work to Opus 5 .5. It implemented everything.
It verified its own work. Then it again told the main agent, the lead agent, Fable 5 .2, I'm finished. And Fable 5 .2 then again reviewed the code.
And with that, we have this security in mind, the peace of mind that the code is clean. Let's also continue with my verification process. Because as you already know, a minute or two minutes ago, I created a PR, right?
And I mentioned that it's a very essential step in my verification process. What do I mean by that and how does that work? Well, let's head over to my GitHub repository.
Inside of here, I have an OpenPR and this PR actually has something running in the GitHub Actions, CursorBugBot. BugBot is a PR review tool which reviews the code in OpenPRs. And in this case, we did not get any error because the code is clean, but it's a very helpful indicator because we now have a multi -step review process.
What do I mean by that? Well, we first of all have the review process in our workflow itself because we have our coding agents which generate all of the code, use the needed skills, the needed agent definitions, supporting scripts, whatever it might be. And then we have the verify step where we have sub -agents which review their own work.
And this runs in a loop. As you already know, we go as long as the code is not perfect. And once the code is clean, what do we we have a handover with all of the evidence and then I create a PR or I tell the agent, hey, do it yourself, whatever it might be.
And then in the PR, we have another review tool reviewing all of the code. With that, I'm able to make sure that even if my local process fails, I always have a runner up which reviews the code again. And this makes everything way more doable and also easier for me.
And with that, you might now also ask me, Jan, what about code reviews? Will you now review the code that you reviewed? What will...
you do or what do you do most of the time? And that's a great question. Now, to answer that question, I will use an analogy I heard about relatively recently.
A kitchen. What do you have in the kitchen? You have a chef and you have cooks.
Let's think about the chef. What does the chef do? Does the chef, I don't know, boil the water, cook the noodles, add the salt?
Does the chef cook all of the meat? Does the chef do everything? No.
What does the chef do? Well, a chef is there to think about the menu or orchestrate or delegate all of the work do you of course like little probes try the meat is it finished maybe try the sauce is there enough salt enough pepper but the chef is not there to do every little step by himself for that we have what cooks.
And we have the same idea with coding agents. You are the chef. You are not there to create all of the code, create the authentication code, the authorization code, the web scraping code.
You are there to orchestrate, to delegate and also to try the code. Okay, you won't try the code, but you review the code. Do you want to review every line of code?
No, that's not scalable. A chef does not try every little dish. It does not work.
So, you want to review sections of code. And in our case, we now worked on a relatively simple task. What do I mean by that?
Well, what did we do? We scaffolded our application. We installed Next .js, Tailwind CSS, SharedC and UITypeScript, whatever it might be.
Is that a hard task? Well, in my opinion, no. It's relatively simple.
And that's also why, in our case, our application itself did not any issues but also bugbot did not find any issues. There's not a lot that can go wrong and therefore I also myself won't review the code here.
It's simply not needed. I'm trying to optimize for speed but at the same time also don't use my cognitive resources for stupid things and that's the same thing I would also recommend to you. You want to review code, don't get me wrong, it's still important but you don't want to review every line of code.
So let me maybe give you a few examples where I would review the code myself. Let's say we would work on some sort of super complicated feature, maybe billing for a B2B application, maybe roles and permissions, which are very mission critical.
What else? Authentication, some sort of enterprise feature. These are cases where I would sit down and review the code myself because it's mission critical code.
It has to work, especially if I'm working on an application with thousands of users. So now you might say, okay, So you essentially only review the code if it's mission -critical code, right?
Yes, but there's another time when I review code, and that's when either one of my sub -agents says, hey, something went wrong, or if BugBot gives me an error. Because this means, hey, we probably had some sort of problem, let me go over it myself. Let me make sure that we now actually fixed all of the root causes.
Because in this case, neither my sub -agents nor BugBot threw any errors. It does not make sense to review the code. code.
It's not mission critical, or I mean it is mission critical, but it's basic mission critical. If now Bugbot would throw an error here and say, oh my god, I don't know, cache components are broken, you forgot use cache, you forgot whatever, then yes, I would review the code because it means, hey, the agent probably missed something.
So with that out of the way, and since you now understand my review process, we can finally merge the pull request. Oh, and by the way, we now only talked about two pillars of code reviews, but there's a third one, which I will dive into once we implement all of our backend code.
So if we now go back to our phases, we are officially done with phase one. So let me mark it as done. And the next step is already to continue with the backend.
And the backend is interesting because it's not super simple. We now have to connect all of the moving pieces. We have to set up our AI SDK, get our entropic provider.
We need to set up the coder and then also create the pipeline. We want to scrape our data. We need to hand over the data to our LM.
We need to create the citations, generate a markdown report. There are a lot of moving pieces which need to be all connected to each other. And most importantly, everything has to work together nicely.
So here's what I will do. Since everything is committed, I will create a new thread to start off fresh. This looks beautiful.
And now let's think about it. What should we now do? Should we right away already create a new prompt?
Well, we could do that, but I wouldn't. suggested. Let's first of all get a better understanding of how everything should work.
So let's start with the research pipeline. So what happens between the user's question and the fully generated cited markdown report. Now to understand this diagram a bit better I broke it down into three different sections.
The orange section describes what the model thinks or does. The blue section describes what the coder does. And finally the gray section zone is there to explain what our code does and with that decides.
As you already know, everything starts with the input, the question of the user. It could be a super basic and open question, but it could be also a super specific and with that scientific question. Once we have the input from the user, we then hand the question over to our LM, Sonnet 5 .5.
And it takes the main question and breaks it down into three to five sub -questions, each with a search query. And this search query can then be used by who decodo and that's what you will see here.
We then take each search query again 3 to 5 and we use the decodo search endpoint. With that we get Google search results for every sub question. And what I want you to realize here is that we are not trying to scrape the data.
All we are trying to get here is the title or the titles in general and also the links. I mean let me again show you the structure so that you get a better understanding. So here we have the search endpoint.
I added a query paragraph. meter laptop and what's the result organic it's an array and here we have all of the search results with a rank the link the title and description and all we need is the link title and i guess we will also use the description i did not add it here but yeah we will also use the description the next step is then the selection process here we use our model sonnet 5 .5 and the model says which results are worth reading or in other words scraping so it answers with numbers results through 7, 12, 22.
They all look great. So let's scrape them. Let's read them.
And one thing you will also see here is that the model picks by index. And that's because numbers are doable. Links are relatively big in terms of size, but they are also fragile.
So with that, we are able to save tokens, but also stay more durable in terms of the workflow. And also our code caps it at the budget, because as you already know, we have a budget of eight pages per round. So we never read or scrape more than 8.
It's the limit. If we want more pages, we need what? a new round.
So after the selection process is done, once we have the eight pages, we then use the Kodo and to be specific, the scrape endpoint. So the Kodo opens each chosen page and returns it as clean Markdown. And we want to use Markdown because Markdown is LM ready data.
Yes, we could scrape the HTML, which is perfectly fine. But HTML is very, how should I say, it's very messy. And we, as you already know, have a limited amount of tokens.
And we want to just provide clean data and Markdown is the cleanest form of data we can provide. And at the same time, we also cache everything in our SQLite database so we save everything because we don't want to pay for the same page twice.
Once it's saved, once it's cached, it can't be re -scraped and that's very important. As you already know, we want to create a real business. Everything here should be scalable but at the same time also economical.
What's quite interesting here is that we also have an optional step and the option step is to use the map endpoint. If we need a specific page of a site, we can then use the map endpoint and Decodo lists then all of the site URLs and the model picks from the list.
So let me show you an example. Here I added or selected the map endpoint. I added the Decodo website and now we have all of the subpages.
So Decodo, WebData API, Video Downloader, Template, Scrape, whatever it might be. And we can do that for literally any website out on the The Codo, Walmart, the New York Times, the Financial Times.
It really does not matter. So what else do we have here? Once all of that is done, we then go over to the extraction step.
So the model, our LM, reads from all of the scraped data from each page and writes down the facts. I mean, it does not write anything down. It kind of saves everything.
And for every fact, it also must copy the exact sentence from the page that proves it. So which proves the solution, the answer to the question. Now, since we want to keep everything, how should I say, scientific and true to the fact, we also have a verify step.
And the verify step uses normal code, no AI, no LMs, no judgment, if that makes sense. And it searches the page for the copied sentence, for the answer. And if it's found, then the fact stays.
If not, the fact is thrown away. And why do we have such a step? Well, LMs love to hallucinate.
They love to make things up. And using normal code, we can make that even if the agent fails, we still stay as close to the truth as possible.
Then we also have a gap check. And why is it needed? Well, as you already know, we have rounds.
Now, the first round is that you scrape eight pages. What is if these eight pages don't have enough data for the solution? Well, then the model will say, hey, we don't have enough facts.
So let's spin up another round. And then everything goes in a loop. Again, we use the search endpoint.
We then do a selection. We select eight pages. We scrape.
them, extract everything, verify all of the quotes, and again go back to our gap check. If we have enough facts, enough data, well, then we are finished. If not, we go over and continue with round three.
But let's now imagine that everything looks good. We have all of the needed facts. What do we do then?
Well, we create a report. Our LM writes the report, it generates a markdown report, and it also cites all of the facts that we saved. And since you now understand that, we can also finally kick off the phase.
to implementation. So here's what I will do. I will create a screenshot of the diagram.
Let's go back to T3 code. I will create a new thread and also attach the image inside of here. Let's switch the model right away.
I want to use Fable 5 .2. I will also invoke a skill. Which one?
Well, you guessed correct. P stack or to be specific, potato mode. And I also already prepared a prompt.
So let's go through it and I want you to try to understand it. So we are starting with phase 2, the back end, no UI in this phase.
Before you write code, read the following files. Our PRD, the reference point, the agents .md file. What do we have in our agents .md file?
We first of all reference the PRD, but at the same time, we now also reference the next JS documentation. And then finally, techstack .md, the tech stack which we decided on. So the attached diagram, this one right here, shows the research pipeline.
will build. Orange boxes are the AI model. Blue boxes are the Codo.
Gray boxes are ordinary code. Here's each step in words. Essentially what we just talked about.
I formatted everything in bullet points so we don't have to go through it. And then what else do we have here? Don't guess how the Codo behaves.
Check it for real. Why did I add this line here? Well, the agent already made a mistake once when it assumed that there's no SDK or some sort of outdated SDK.
Therefore, this time I want to make sure that the agent actually takes the time and researches before committing to something. So use the Decodo MCP server to look up the docs and try each endpoint.
Use the Decodo CLI to run the same search and scrape yourself and compare the output. Use the Decodo SDK in the app code and make one call per endpoint to confirm the response matches what the code expects. Now, one thing you should already see right away is that this prompt is broken down into our three subsections.
How did I start this prompt? We are starting phase two, debug and no UI in this phase. What did I just do inside of here?
I stated my goal. So this is the goal. Then what did we talk about here?
These are all of the constraints. Look at these files. Then here, look at the diagram.
Here's like a small little explanation about how it works. What else did I say inside of here? Use the code though, don't guess.
These are all constraints. And now look at the definition of the You are done when all tests pass.
One real search run started from the terminal finishes with a cited report. You show me that report and the list of the code or calls it made. So there's like one fundamental point I want you to take away from this video.
Your prompts don't have to be super engineered, super specific, super almost high quality. It's not needed. All you need to do is state your goal, all of the constraints and the definition of done.
If you then also pair the prompt with psterk, you literally have like an engineering wonder machine. I guess we can click on enter, right?
So yeah, I will click on enter. I mean, we also merged our PR and the agent will now get started. And let me now ask you the following.
How will everything work right here? Because you should now already be familiar with the workflow. The agent will now first of all use a playbook.
The playbook will give the agent an order. Do this, this, this, and then this. The agent will invoke skills, functional skills, skills explain how to do something.
Once the agent is finished with generating code, and by the way, it will use sub -agents, it will verify the code. Once everything is done, once the code looks good, we will have a handover since I did not tell the agent to create a PR. So I will check out everything and then create a PR manually.
But of course, I could have now said, hey, if everything looks good, create a PR yourself. This fully depends on you and your workflow. So P -Stack is not a workflow that is set in stone, you can adapt it, you can change it, you can update it.
And that's also what I have normally. Normally, my agent always creates a PR by itself, but I wanted to show you the default naive workflow if that makes sense and look what the agent is also doing here it's using actively the mcp server we have installed for the kodo web search this is our result map website and look how cool it is we can use the kodo in production using the sdk but we can also use it locally using the cli or for example using the mcp server also here scrape as markdown we scraped or we added this url it's the input and here we have the output and it's nicely formatted what?
Markdown. So essentially what the agent is currently doing is just making sure that everything works. We are currently building the architecture around our core ideas.
And here the agent also, of course, found issues like, hey, the code maybe did not work as expected. Let me try to figure out how to use it the right way. It's again using the MCP server.
And that's also the reason why we installed the MCP server. All right, our agent is now finished. It opened a new PR, number two, and also 5 ,000.
lines of code have been added in total, which is quite a bit. So let's look at the done criteria. PNPM test passes, lint and type check are clean, tests first, one real call per endpoint, real run, and also independent check.
Here we now have the generated report, because as you already know, in the prompt we specifically said, hey, once you're finished with all of the code, once you created or implemented all of the business logic, I want you to generate a report to make that everything works as expected. Because yes, we don't have an interaction layer yet, but our backend is already built, meaning it should also work.
So here we have our super scientific question and then the generated report. Now, I want you to look at these array brackets because these are our citations with the IDs. So 7, 1, 3, 4, 5, 17.
And if I also scroll to the bottom, you will see that all of the sources are listed here. We have the page title, link, and then also the category, blog, blog, organization.
What else do we have here? Research, paper, news, and even forum, which is super cool to see. But now you might say, Jan, a moment ago when you talked about the MCP server, you mentioned that the agent had issues with the Codo because it did not work as expected.
Is that true? No, I actually was incorrect. The Codo worked as expected.
Everything was Gucci. The issue was with Google. Yes, you're heard right.
So let me give you a quick rundown of the issue and also how I fixed it. The normal Dekodo search endpoint worked as expected, but most of the links it returned could not be read. Redirect links instead of real page URLs.
In 14 real search calls, half of the three quarters of the search results came back as what? Google .com slash goto URL instead of the real page address. So instead of having our destination URL, financialtimes .com the New York Times, Wikipedia, what else, Rustlang as an example.
We did not get these pages, instead we got a redirect URL, a proxied URL. The field meant to hold the real address original link was always empty, and those links cannot be read. The code of scrape endpoint rejected a redirect link with a 400 validation error, so the read on other words scrape step had nothing to open for most of the results.
So the first fix was to pin a but that did not help at all. We still got redirect URLs and that meant that our result was too thin to create a good report with citations.
So how were we able to fix this problem? How were we able to fix Google? Great question.
So if we head over back to the Decodo dashboard, then you will already know that we have access to the playground, to the usage statistics, but also to target templates. And that's what I used. So what we did previously, is use the normal search or sorry scrape endpoint and it allows you to scrape normal websites.
The Kodo's website, my website, Amazon, whatever it might be, all of it works because we have the destination URL. But since we did not have them, we only had redirect URLs, we always got an error. And thankfully, the Kodo thought of that problem and created a template for us.
So here we have already a few categories, Amazon, YouTube, Walmart, Marketplace. We don't need these templates we want to use a google template and to be specific this one right here google search with ai overview which we've accurate search results rankings and also snippets this template is what allows us to scrape data reliably without having the actual destination url because again currently we only have the redirect url and that's what i love about the kodo you can use the normal scrape endpoint and it works perfectly if you have the destination url but you can also use templates which provide already nicely formatted results.
So you can use the Walmart template, the Amazon template. I, for example, created a video recently on P -Stack where I also showed you how I use the Amazon template because it returns nicely formatted data. But what I want you to realize here is that you can still use the normal scrape endpoint.
If you have a target URL, it will work with zero issues. The benefit of target templates is that they are purpose -built for its target, ensuring fast faster, more reliable extraction by adopting two dynamic layouts, unique HTML structures and anti -bot defenses so you spend less time troubleshooting and more time scaling.
And that's also what you will see here. Once we adopted the Google search template, everything worked right away, gave real page addresses for every result in 6 out of 6 test calls and again in the later research runs. And that's exactly what we wanted.
What I want you now also do is look at the general workflow. How were we able to go from a simple prompt to a finished and with that working result? Well, everything started with my prompt.
I added or I invoked the potato mode skill. I added all of the needed details, the phase two brief, the nine pipeline steps and words, the pipeline diagram. And also I used my prompting strategy by including my three done criteria, the goal, the constraints, and also the definition of done.
The potato mode skill. then took in the request, tried to digest it, and then merged my request with the needed playbook. In our case, the feature playbook because we were trying to implement a feature.
This playbook gave us a to -do list, eight steps in order. The agent then read the PRD file, the tech stack file, and also our agents .md file. Once all of that was done, once the agent understood what it should do, it started with the coder and it investigated how the coder works.
It used the MCP server, the CLI, and also the SDK. What I want you to realize here is that the MCP server and CLI, both of these primitives, are only usable locally.
The SDK is what's usable on or in the application once everything is deployed. And using these three core primitives, we are able to make sure that Decoder works as expected. And using, for example, the MCP server, the agent instantly realized, hey, we have a problem.
The search endpoint and with that the scrape endpoint. Both don't quite work as expected because Google does not give us what we are looking for. So let me look at the solutions.
Let's first of all try geolocking everything. Hey, that did not work. Let me try something else.
A template. And the template magically worked. So the agent stopped and asked me for questions.
First of all, should we now use the Google search target template? I said yes, let's go for it. Then the agent also asked me about rounds, meaning it wanted a clarification.
It also asked me about like the support check. And finally, what should happen if we have a failed run? And I said, hey, still write a report from what exists.
Nevertheless, mention, hey, we did not have enough data. So that's why this report might not be as good as it should be or as you would expect it to be. As a next step, we did not continue with the implementation, but rather our lead agent used subagents to design the architecture.
And that's why P -Stack is so important. here is not to get you the result as quickly as possible, no, but rather by creating the most clean, scalable, and secure code possible. And that's why we first of all designed the architecture and then implemented whatever we designed.
For that, our lead agent Fable 5 .2 delegated the work to Opus 5 .5, Fable 5 .2, and then also Sonnet 5 .5. The judge Opus and I picked the same base. Seven fixes from the review went into design before any code.
And these all of the skills which have been used. Architect, arena, model the domain, type system discipline, these are all functional skills.
After the design was set and with that the architecture, our lead agent took all of the input and then delegated the work to a sub -agent, an open sub -agent. Once our sub -agent built everything, then handed the work over to our lead agent, Fable 5 .2. The lead agent then reviewed the div and also proved it on the real thing.
What do I mean by real? thing? Well, it generated the report with all of the needed citations.
It scraped all of the data using the coder and also searched for it. It essentially just tested the whole pipeline, which we already talked about. Once everything was verified, it then created a PR.
And for the PR creation, again, a new playbook was used, opening a PR, which explains how to create a PR, how to, I don't know, listen for it, because there's actually a supporting script called watch pull requests. And once the PR was then created, we had my PR review tools running.
Bugbot and also Devon Review. I did not even know that Devon Review was turned on for this repository, but hey, it is what it is. And believe it or not, Devon actually found five issues, five potential issues, and four produced with a failing test, which then also got fixed, and one was not a real issue.
Once all of that was done, we created a handover. The agent created a handover for me, and it did not merge anything. So the current status is that our application already did a verification process with subagent.
but we also had another verification step in our PR. Everything is now green, everything looks good, which means the agent now said, hey, we are done, here's the handover, do whatever now needs to be done. Review the code, merge it, do whatever is needed.
Now, here's the thing, I already reviewed the code myself and it looks quite clean. And now you might say, Jan, how do I review code the right way? Is there some sort of magic trick?
Well, do you want to know the honest truth? No, there's literally no magic, no secret. All you have to do is sit down on your little chair and go through the code, look at it, try to understand it, ask questions, and if you find any issues, then let the agent fix it.
That's literally it. Now, is it boring? Yes.
Does it take time? Yes. Would I rather, I don't know, sit and scroll on Twitter or something like that?
Sure. But I'm a senior engineer, you are a senior engineer, and your role is to review the code and make sure that the mission -critical code works. and is also scalable, secure, clean, whatever it might be.
So again, I reviewed it. It looks good, meaning we can merge the PR. So here's what I will do.
I will close the sidebar and click on the merge button. So T3 code has like this native GitHub integration and the PR is now merged successfully. And with that, we can now already continue with the next step.
Now you might say, Jan, we are finished with phase one, phase two, which means we can finally continue with our front end interaction layer, right? Well, Well, no, not yet.
Almost. Because before we do that, we need to also create a project -specific verification skill. But now you might say, Jan, what even is a project -specific verification skill?
How does it work? Why do we need one? And how can we create one?
Well, these are all great questions. So let's look at our current workflow. Everything starts with our prompt, with our goal, constraints, and also definition of done.
Then we use potato mode. Potato mode, takes in the request and then routes it to the correct playbook.
What does the playbook give us? It gives us steps in order, skills to use everything needed. It's a pipeline in a workflow.
I hope that makes sense. Then we have subagents which do the implementation. In parallel, sequentially, it does not really matter.
And once that's done, we have again subagents which verify and review the code. They make sure that everything works as expected. But now, here's the thing.
Looking at the workflow, almost every step is there to create code. The task is there to generate code.
Potato mode is there to generate code. Playbooks are there to generate code. Sub -agents are there to generate the code.
And all of these or the whole code generation step also follows a specific order. Do this, this, this, and this. Use the following skills and then only verify the code.
But what is currently coming short? Well, it's the code verification itself. Yes, the agent is currently able to use the browser.
We have LIN setup, TypeScript, VITEST, which is all very powerful, don't get me wrong, but we need to give our agent more capabilities. We want to improve our environment so that it's as easy as possible to review the generated code. That means maybe, for example, creating scripts, maybe also making the browser easier to use.
How about we also improve the work tree setup so that the agent already right away gets all of the needed tools. There are a lot of things we can do to make the code generator easier, don't get me wrong, but at the same time also make the verification step easier, but also way more thorough.
Because with that, we will also automatically get way more autonomy. And that's also something that a lot of developers, even senior engineers, miss. They always think about the best tools.
Oh, we need better models. Fable 5 .2, 5 .5, 6 .7, whatever it might be. We need better tools, better ADEs.
We need more sandboxes. And yeah, that might be true. A better model is always cool.
Better tools are great. But we are at a point where it does not really matter anymore. Since Opus 4 .8 models are good enough, ADEs are good enough, we have all of the tools that are needed to create good applications.
The only thing that is now missing is for you to take these tools and create a good environment, a great environment for these tools. So instead of wishing for better models and with that better tools, rather sit down, improve your environment, make it as easy as possible to create code and verify code. And with that, you will literally be ahead.
of 99 % of developers. But now coming back to our project -specific verification skill, you might ask me, Jan, how exactly does it work? Well, it's a scripted way to drive the application like a real user would.
And for that, we have sub -primitives, launch, doctor, drive, evidence, and cleanup. So here's how the flow looks like. We make a change, our sub -agent makes a change, small scoped change, the goal stays fixed.
Then we run real check. using our verification skill, drives the actual application, follows the feature map, and then finally inspect the evidence, screenshots, real outputs, stored values, read back. And this then goes in a loop.
If something fails, is unclear or needs more work, we go back to step one, make a change, run the application, inspect the evidence. Oh, something fails again, let's go back to step one. And this makes the whole workflow even more doable.
We instantly have the ability to generate great code but also review code way more thoroughly. So what we want to now do is improve our environment and at the same time create a project -specific verification skill.
So here's what I will do. I will go back to T3 code. Let's create a new session inside of Research Marshall and I will switch the model to Fable 5 .2 and inside of here I will say the following.
We are now finished with phase one and also phase two. We implemented or we created the foundation and we also implemented our backend and with that business logic.
And the next step is now to improve our environment. I want to make it as easy as possible for agents to create code, generate code, but also more importantly, verify code. And this means we want to create scripts, maybe create tools, literally anything that can make it easier for the agents to do their work.
And to be specific, I want to create a project -specific verification skill. One issue I, for example, saw right away is that the agent had issues with work tree creation because it did not have access to all of the needed environment variables. That's one thing we have to fix.
But also, I want to make it easier for the agent to steer the application, make checks in terms of like performance. Hey, is the application still fast enough? Also, in terms of security, literally everything.
Look at the environment and also create a verification skill. Now, inside of here, what I will also do is invoke two skills. I will first of all use potato mode.
So let's say potato mode. And since we want to also create a verification skill, I will say slash P stack colon create verification skill. And with that, we now have everything that is needed.
Our model is updated. We have our prompt here. I did not really like think about my prompting strategy with the goal constraints and definition of done because it's not really needed right here.
It's a basic prompt. It's a basic task. So in my opinion, it's good enough.
Now, one thing I will probably do is tell the agent that it should use more sub -agents. P -Stack already uses sub -agents very heavily, but for this specific task, I want you to get as much information, as much context as possible. And by the way, I also don't want you to get started with the implementation yet, but rather I want you to know what the coding agent recommends.
Look, I want you to use sub -agents also very heavily here. I want you to have as much context as possible. And don't get started with the implementation yet.
I want you to first of all map out what issues we have and what we can do better in terms of the environment, but also the verification skill. One thing you should also know is that we use the Codo very heavily. This means you should also use it very heavily for testing.
You can do it using the SDK or maybe the CLI, MCP server. You can use all three. It does not really matter, but use the platform, use the tools you have access to.
Verification goes over. assumptions and speed. Be as thorough as possible and needed.
So that's now already it. Our agent will now get started. It will probably take around 10 or so minutes, but that's absolutely fine.
Let's wait for the result. And by the way, this is what I meant with work trees. I've confirmed a work tree is missing node modules, env .local, and the .cloud directory.
And that's not great. It's a very common issue with work trees. And that's why I specifically said improve the work tree creation step.
make it as easy as possible to create work trees. Because that's probably something you didn't know. When our lead agent delegates work to sub -agents, it actually also creates work trees for these sub -agents.
And currently these sub -agents are not really able to verify their own work because they don't have access to the needed environment variables to our node modules, which means our application essentially right now allows the agents to generate code, but not really verify the work, which is a... huge problem. Alright, we now have our updated environment and also we created our project -specific verification skill.
And how long do you think the whole process took to update our environment, to create a skill and also to fix bugs? Because the verification skill instantly found issues. Well, one hour?
No. Two hours? No.
Three hours. Damn. So let's see what our agent did.
What I found at the start, new work trees were broken, keys had no permanent... home, error messages didn't say how to fix anything, no single is my change okay command, no way to look at a past run without knowing the DB, every full test run cost money, etc. What I built for the environment, keys, one file, an env file, pnpm bootstrap, doctor, session start hook, pnpm check, we now have CI which runs on every PR run, which is super important, rule checks, cloud settings, what I built for verification, These are now all custom commands, pnpm runs, research offline, verify run, performance check, bundle size and pipeline parallelism gates.
Wow, this looks super fancy. Here we now also have a feature map and that's needed for the project specific verification skill. Now this feature map has to be updated like literally every week.
So what I would suggest doing is set up like an automation. If you use T3 code, you can right here just say like, hey, set up an automation every week. I want to update my feature map, use a few Sonnet 5 .5 sub -agents and figure out if anything changed.
Because it's very important that your agent or yeah, that your agent always knows what are the current features, what exists right now, what do I need to check out. If we are now, for example, building some sort of like billing feature, we need to know if multi -tenancy is implemented.
If so, they are directly connected to each other. And then right here, as mentioned, bugs were also found. And that's because of the project -specific verification skill.
And that's why I want you to also create it for literally any project you work on or want to work on in the future. And by the way, once thing I want to also quickly talk about is sub -agents because if you now look on the right you will see here that we used in total 22 sub -agents and now you might say Jan oh my god that's such a big mistake it's not economical it's too expensive blah blah blah and that's something I don't agree with anymore because yes in the past Fable 5 .0 or yeah Fable 5 was super expensive would be uneconomical and it would only make sense if you have millions in the bank but if you are like me in that does not work.
No, I'm joking. But now we are at a time where we have Opus 5 .5, Sonnet 5 .5, Haiku 5 .5, and these models are more than, or they are cheap enough. Like, you can use sub -agents.
Maybe not necessarily 22, it's maybe a bit too much, but having six sub -agents is totally fine, and I wouldn't discourage against it. Now, sure, Fable 5 .2 with max reasoning effort is still expensive, but that's also the whole idea behind model roles. Use the needle.
models for the needed tasks. You don't want to use Fable 5 .5 for implementation. You want to only use it as a lead agent.
If you want to implement something, then you need a model which is, yes, capable, but also cost efficient. And with that, Opus 5 .5 with medium reasoning effort is the perfect candidate in terms of just raw intelligence power, but also economic cost. And with that out of the way, did I merge everything?
Yep, everything is merged. And now we can already continue with Phase. 3, our front -end UI implementation.
So here's what I will do. I will create a new session. Let's right away switch the model.
And by the way, I can, in theory, just update the model in the settings. Sure, let me do that. So I will go to my where is it?
Projects right here. Research Marshall. And then for the default model, I want to use Fable 5 .2 with medium or high.
Let's use high reasoning effort. And that all looks good to me. So I will close this again.
Here we have our new session. Let me quickly switch the model, Fable 5 .2. Let me close the sidebar.
But now you might ask me, Jan, how should we now prompt our agent? Should we just say, hey, we are finished with the backend, now build on top of it, create a beautiful result, whatever that means. Don't create any AI slop and now get started.
Well, well, well, that sadly does not work. Because let's start with the basics. What does beautiful even mean?
It's not super objective. It's kind of subjective.
For you something might be beautiful while I might find it super ugly. Who knows? At the same time what is AI slop?
There is no unified definition. Sure there are telltale signs like this blue purplish gradient but there is no unified way of saying like this here is AI slop and this here is super unique and this is what you should create. And that's why it's so hard for LMs to create something beautiful in terms of visuals.
It's not easy. It's super hard. So now you might say, okay, Jan, so what's your strategy?
Well, it's relatively simple. I love to provide reference material. What I normally do is search for websites which might be in a similar niche, and then I just take screenshots and provide them as reference material.
Because what you have to understand is, yes, in the past, models were not capable at analyzing images. They were pure trash. They did not understand the dynamics of an image, how everything is composed, the composition of an image.
but that has finally changed. Models are able to analyze images, they are able to understand the composition, how everything is, you could almost say, architected. And that's why I have become a huge fan of essentially giving my agent eyes.
They are now able to easily see what I'm trying to do. Because in the past, all we could do is use our words, but the problem with words is that they are not super definitive. But pairing words plus images is a great combo, allowing you to be super specific about what you want, but at the same time also giving your agents, you can inspire them to create something better, something like visually more aesthetic.
So now you might say, okay, Jan, so let's now maybe gather some reference material. Sure, why not? So recently I found this website right here, devl .dev, and it provides free React components.
It's completely open source, free to use, and it's something I found quite recently. And I already looked through all of the blog, screens, the illustrations.
I mean, let's maybe quickly check out the screens. As you see here, these are beautiful dashboards, which definitely don't look like AI slop. And I already found a few, which I like quite a bit.
First of all, this, what is it? 500 error route. Now you might say, Jan, we don't want to create an error route.
We want to create like an input box. We want to have a sidebar, something like that. Well, the reason why I essentially saved this reference point is because I'm quite a big fan.
of this animation on the top. It's what you call ASCII. Then let's continue.
What else did I find? Ah, this page right here. What I love here is that we have a sidebar.
I want to also create a sidebar and I want to list out all of the conversations the human had, the user had. And then at the same time, I'm a huge fan of this input here because that's exactly what we want to create. We want to have an input where the user can ask a question.
As you already know, a scientific question, an open question, whatever. it might be. And looking at the screenshot, it might actually be also a great idea to allow the user to switch the model.
Opus 5 .5, Fable 5 .2, maybe GPT -6. Should we do that? Yeah, you know what?
Why not? So let's also save that in our mind here. And another example I found is this screen here.
So again, we have a sidebar with conversations, in this case tasks. And what I love here is the right sidebar because it's essentially like a progress panel. It shows all of the steps which have been made.
So web search, browser open, files saved. And we want to create something similar. So here's what I will do.
I will create screenshots. Let's start with the... route.
I will create a basic screenshot. Boom. Let me change the size a bit.
This looks beautiful to me. Let's go back. I will attach the image here.
Then let's create a screenshot of the split panel. Again, screenshot. I will copy it and then add it inside of T3 code.
And then finally, I will create a screenshot of the 4 -4 route or of the 500 error route. I will make this a bit smaller. Boom.
Let's capture it. And then I will also attach it. side of here.
So now we have reference material. Now the nice thing about DevL is that it builds on top of shared CNUI. Therefore, what I will also do is copy the commands for the installation script.
Let's go back first of all for this screen right here. Then let's copy the installation command for the split panel. I will also paste it down below.
And then finally, also for the 500 error, I will copy it and then paste it down below. So now we actually gave the agent even more context.
Like, yes, we have visual reference material, the agent is able to look at the needed screenshots, but we now even have the code here. So, it makes everything way more specific, which is super cool. Now, as already mentioned, I'm quite a big fan of this AirSki art right here, this pixelated art.
And recently, I found this component library here, add AirSki to anything in your application. Therefore, I will copy the URL. Let me also paste it down below.
And I will also create a screenshot so that the agent understands what I even mean by ASCII. So now we have all of the needed reference material and the next step is to create a high quality prompt.
Now let's quickly think about our prompting strategy. We want to define our goal first of all, we want to define our constraints and we want to also provide our definition of done. I probably made a spelling mistake but that's fine.
So let me say the following. We are now finished with a one and phase two we implemented all of the backend code all of the business logic and now i want to work on the front end ui interaction layer let's first of all look at what i already provided i attached four screenshots i want you to check out the first one shows a beautiful input and it's something i want you also implement it's where the user will be able to ask questions i'm also a big fan of this greeting on top so good afternoon good morning whatever it might be and there's actually one thing I want you all to copy here which is the model switcher.
So I think our PRD currently does not mention that but it's something we can add. I want you have a model switcher where the user should be able to switch the model. Fable 5 .2, Opus 5 .5, Sonnet 5 .5 and should we add Haiku 5 .5?
Sure, why not? It does not matter. So that's the first screenshot.
The second screenshot shows this split panel. We have a sidebar on the left which I want you all to create. I want you show the history so that the user is able to view all of the past conversations and then on the right I'm a huge fan of this progress panel which shows all of the websites or yeah all of the actions made by the agent because as you already know we want to do the same thing.
We want to be as open as possible. We want to show the user hey this website was scraped, this is what the agent did, this is what we are now doing. Essentially we want to have like a timetable almost with all of the actions.
The third screen screenshot is a 500 error route. We don't want to create an error route, but I'm a huge fan of this ASCII art on the top with the question mark.
And that's why for the fourth screenshot, I attached a component library, which makes it super easy to add ASCII to anything in your application. So we want to create a combo of all of these four screenshots. And I also added the needed shared CNUI install scripts, but they build on top of something called COS UI, which we don't use.
You don't need to install them. You can look at the code and create your own code on top of the foundation, if that makes sense. Now, let's talk about a few, you could say, constraints.
I want you to make sure that you also keep in mind that we want you to have the correct contrast ratio. I want you also have a professional design system. In terms of the primary color, I want you to use some sort of like bluish color.
I want you also make sure that we have a dark mode and light mode friendly website. We need a switcher. a theme switcher.
What else? Everything should work beautifully. It should look good.
And another thing that we have to look at is the user experience. Sure, the UI should look good. It should have good animations or smooth animations, but the user experience is super important.
Everything should be intuitive. And the definition of done here is that I want you to make sure that everything works with our backend. For that, use our verification skill, use the browser, and also use the browser continuously while implementing.
all of the work. I want you to have it as easy as possible to catch mistakes and fix them right away. So, I think that's already it.
Now, one thing you should realize here is that I told the agent to use the browser continuously. And that's because, yes, P -Stack already tells the agent to verify its own work, but it only does that at the end. And I want the agent to use the browser continuously the whole time.
Whenever it creates an H1, it should open the browser, create a screenshot, look at And for example, maybe change something if the result does not look as expected. So let me now do the following.
On the top, I will use our P stack skills. So potato mode to be specific. And honestly, I think that's all right yet.
There's not much more we can do. Sure, I could now maybe update the prompt even further. I could make it even more specific.
But I want to give our agent as much freedom as possible. That's something I have learned recently. If you constrain your agent too much, then yes, it will follow.
everything you want it to do, but it's often not able to outgrow itself. It's not able to experiment. And I want my agent to experiment.
I want it to try something new. I don't want it to be in a sandbox of my own mind, if that makes sense. So since this is now all finished, I will click on enter and our agent will get started.
All right, the implementation here now literally took over two hours and 15 ,000 lines of code have been added. Damn. And what we can now do is already instant.
look at the screenshots created by the agent. That's the power of P -Stack and our project specific verification skill. Because as you already know we have the implementation step, the verification step and finally we then have the handover with all of the evidence.
And evidence sure is the report of what happened but also screenshots. So let's check it out. And you know what?
It looks... Fine, don't get me wrong, but it's not as good as I want it to be. So here we have the sidebar on the left.
I can add or ask a new question. Here our report gets generated. And on the right, we have the sidebar with the progress.
The progress sidebar looks great. There's nothing I want to change here. But the landing page, the startup route could be way better, way more aesthetically pleasing.
So let's see what the agent said here. What you get starts. screen, run screen, progress panel, history, sidebar and then also theme and states, ok, definition of done, gates, pnpm check, build and check pairs, browser on fakes, cool, real, back end.
Then the models were also tested so as you already know the initial idea was to only use sonnet 5 .5 but then we decided later on to also offer multiple models with that haiku 5 .5, sonnet 5 .5 opus and also fable. And looking at it, it seems like Sonnet 5 .5 is the best model for this specific use case because it cited the most amount of sources.
Then what went wrong along the way? The first opus and fable runs failed because of a timeout. Cool, I don't care.
An independent bug review found three real defects. That's also cool. Principles that changed a decision.
So here the model used principle skills because as you already know, functional skills explain how to do something. Principle skills... guide the decision.
So model the domain, build the lever, prove it works. That's also all cool. So here's what I will do.
I will say the following. Please start the dev server with fake data to demo so that I can open it in a real browser. I will click on enter and also send now.
I don't need any compacting. All right our dev server is now running so let's check out our application. And I mean looking at it as already mentioned it's not bad, it's okay, but it's definitely not Perfect.
There are a lot of issues I see right away. Our logo is super basic. All we have here is essentially like a blue square with a few dots in the middle.
Then also the sidebar. I don't know. I'm not a huge fan of these green check marks.
The input for asking a question is also kind of too basic. I mean, let's look at the DevL example. Look at how sophisticated it looks.
The input is 10 times better. Also, the suggested questions kind of update, right? Do you see?
like this beautiful animation. It looks so much better. I feel like the agent just looked at the screenshots and said, hmm, the user asked me to build on top of the foundation, so let me create an even worse result.
No, that's not what I wanted. So let's do the following. I will head over back to T3 code.
I will open my sidebar and let's open the built -in or use the built -in browser. I will open the patron side of here. Let's make it full screen.
I will zoom in a bit. And what I want you to now use is the annotate feature.
It's a huge game changer. It's offered in Cursor, Cloud Code, Codex. Most of the ADEs out on the market have an annotate feature and trust me, you want to use it.
So I will click on this mouse button and let's create a few prompts. Now, here's the thing for the logo. I will use an image generation model.
As you already know, Anthropic does not have any image generation models, but thankfully Codex, or sorry, OpenAI has image generation. models. So I will right here just say hey use GPT -6 and create an image.
So let me select my image or app icon and I will say the following. Honestly the app icon which we currently have is way too basic and it's not good enough. So here's what I want you to do.
Spin up a GPT -6 Astra sub -agent and let it generate 45 different like app icon variants. In terms of like the normal text which we have then on the right so besides the icon research Marshall.
I want you also, how should I say, test a few different variants. I don't want you generate an image for that. You can just use like local fonts that we have installed, but check it out maybe with different colors where Research is white, Marshall is blue, maybe different font styles, different sizes, whatever it might be.
It can be way better, way more... Not necessarily sophisticated, but definitely more aesthetically pleasing. Here I will now click on Attach.
Let's continue. What else don't I like? The input, right?
It's not as good as I want it to be. The input you created is on a fundamental level fine, but it's not good enough. I told you that you should look at the DevL code example, right?
And almost copy it. Now, you didn't do that. You kind of just, I don't know, looked at the screenshot and created your own mess.
I don't want you to do that. There was a reason. why I gave you the shared CNUI install script.
Look at the code. In this case, I want you to copy the input because it looks beautiful. So in the input, there are suggested questions which switch every one to two seconds.
Also, the suggested questions at the bottom, which you can just click on, have like these cool icons, which I like way more. The model switcher is a bit more sophisticated. So honestly, what I want you to do is pretty much copy the code, maybe change it a little bit.
And you could say also make it even better. As already mentioned, use your browser continuously and improve everything continuously. So this is also attached.
Let's now continue with our sidebar. I will again select it right here and say the following. The sidebar is also fine, but it's not perfect.
You looked at the death L example and created something similar with the green check mark and that is stop icon or however you want to call it. I don't like that. I would rather like...
to orientate myself around like chat gpt or cloud something like that because what we have in the sidebar is the history of all of the chat sessions that the user created so i'm not sure if it makes sense to even have check marks and stuff like that i would want to rebuild it to be honest with you though please also keep the history meaning with today yesterday a week ago one month ago etc then as the next step i want you also improve the air ski art because it could be way better and here I want the agent to also experiment meaning I wanted to create different variants because as you already know for the logo I said create four or five different variants because with that I will be able to look at what's possible and I want to do the same thing here.
So I will say the following in terms of the air ski art I feel like we can create something more sophisticated something a bit more not necessarily experimental but something a bit more fancy because if I go back to the air shared CNUI component library, there are so many cool and fancy examples. One variant which I would love you to try out is this San Francisco background.
And in general, create four to five different variants, experiment, try something new, try different colors, try different compositions, and use your browser continuously. Use screenshots or take screenshots and try to improve it as much as you can. There is no time limit, take your time.
And by the way, this is something I would always recommend doing. So saying like, hey, take your time. There is no time limit.
Use your tokens. Use the browser. Because agents tend to try to get you the result as quickly as possible.
And again, that's fine for historical questions. But we want to create something. And therefore, we need to have a result which is as good as possible.
And good results take time. So let me attach this message. And honestly, I think that's enough for now.
So let me again make this. And then there's one more thing missing, our beautiful potato mode skill. So let me say slash P -Stack, call in potato mode.
And with that out of the way, I will now click on enter and the agent will get started. All right, our sub -agents are finished. So let's start with our app icons.
And man, oh man, oh man, I don't know, I don't know. It looks, well, let's say most of the icons are kind of ugly. So the first one is like this sheriff style badge, which is fine.
but it does not work for our application. The second one is like a verified quote, but it also, yeah, it looks ugly. The third one is a web lens.
It does not work. The fourth one is already closer to what I was looking for. It's an R with a few dots on the bottom right.
Yeah, it's better. And then the last one is also ugly. So I would say we should probably continue working on the fourth icon, the halftone R, but we will have to improve it even further.
This was our first task. The second task was to improve the sidebar and also the input.
So this here is our input. We have our model switcher, budget badges, so three rounds, five searches, eight pages per round, one sitemap, and here we have suggested questions. This already looks way better.
Let's look at the sidebar. Yeah, it's... Good.
I mean, it's simple, but that's what I wanted. And now finally, let's look at the final task, our ASCII art. For that, I will head over to Chrome.
Oh my god, it actually looks quite good. Damn, I'm a huge fan of the San Francisco background. It was my suggestion, as you already know.
Then let's check out the lighthouse. No, no, no, it does not work. Globe.
yeah it's basic it's simple but that isn't a bad thing it's in the contender list then ribbons what do we have here nope it's out right away and then star chat it's also out so the only options i like is b1 san francisco and then also b3 the globe let's quickly again head over to the ASCII homepage to the component library what options what components do they even have so examples let me zoom in a bit catalog miami tokyo new york say out of all of them probably san francisco looks the best then there are also like these landscapes damn canyon river looks quite cool this is also an interesting option here we have the globe is there anything else we could use portrait video nope nope nope a bit of text No, I would probably leave it as is.
So I would say we should leave the San Francisco iteration. We should also test one out with the canyon. And finally, I would like to also change like the globe variant with like different, with a different orientation.
Maybe it should be at the bottom. So the globe, maybe on the left, maybe on the top. So let's do the following.
I will create a new prompt inside of here. So here's the thing. Let's start with the air ski art.
So you did. quite a good job i'm quite a big fan if i'm honest nevertheless i would like to now try a few different variations so let's log in b1 san francisco it's perfect i like it there's nothing we need to change i would like to try another variant with the canyon river it looks quite interesting to me then i would also like to log in the globe but i would like to have a few different variants of the globe currently the globe is on the right side but maybe we should also try a few different position variance on the bottom, on the left, on the top, in the middle, whatever it might be.
You try yourself, experiment. And you know what? Also looking at the sidebar, the sidebar is good.
Ah, and now I see that if I make the screen smaller, the globe is then on the bottom. So it's positioned on the bottom. Interesting.
Let's talk about the sidebar. The sidebar is good. I like it quite a bit.
Though I would love to also have the option to collapse it. So to create a small 20, no, not even 20, probably like 10 pixel sidebar with just little icons.
It would be a cool idea but you try it out yourself and then we can judge it later on. Is there anything else I don't like? Yeah, let's talk about the app icons you created.
You generated five in total which all look somewhat okay but I only like the fourth one, the halftone R. Nevertheless, I would like to build on top of it and create four variations which build on top of the halftone R. Change the color, change the font and since you also know that we are big fans of air Maybe you can also take that in account.
Create a variation with like San Francisco embedded into it. One with a globe embedded into it. Try out different variants.
You have time. Use again GPT -6 Astra. Create a new sub -agent.
Don't use the one we already used. And I think that's already it for now. So yeah, get started with all of these points.
So our agent will now get started. Let's also check out the individual pages for the chat sessions. So I will zoom out a bit.
What do we have here? This here got cancelled. Is this running?
Yes. So we have a question and we have a response. Finished, zero sources, 22 minutes, searches, page reads, skipped, claims verified and dropped.
Here we have then the generated report, completed, sonnet 5 .5, rerun. What is if I click on the button? Aha, the right sidebar closes.
That's interesting, I like it. What do we have here? Budget, sitemap, activity.
Can I open this? Yeah, oh my god, that's quite cool. Hmm.
So I'm quite a big fan of the individual pages. And what I also like is that we follow best practices and use layout routes. So if I now go into a different chat session, the sidebar does not re -render.
The only thing that re -renders is the main page here and then also the right sidebar. That's exactly what is wanted. We want you to have a performant application.
And that's also the benefit of using, for example, Next .js because Next .js has the documentation embedded. in the node modules. So what I would do inside of here is probably head over to T3 code.
Let's open the sidebar. I will maximize this. Let's open a chat session and I want to improve it a little bit.
So what do I like, what do I not like here? Honestly, not that much. One thing I would probably do right here is render the Decodo logo for searches, maps, and also for scrapes.
So let me select the full right sidebar and I will just... Just say the following right here. What I would love to do here is also render the Dekodo logo and maybe even mention Dekodo whenever it's used.
So for searches, for scrapes and also for maps. So please head over to Google and search for a Dekodo app icon which we can render which is also dark mode and light mode friendly. What else can we do here?
I feel like this. How should we call it? The navbar here is a bit too, well, there's too much going on.
We have the title, completed, Sonnet 5 .5, rerun, then the buttons, you close the sidebar. So let me select it here and I will say the following. This navbar, that's how I will call it, is good, but there's too much going on.
We have the title, we have then this completed badge, the model used, the rerun button, then the sidebar to close the panel on the right. All of these options, all of these buttons, all of them make sense to me. But we want to keep it simple.
We don't want to overwhelm the user. So we have to make it or we have to simplify it. I'm not sure if this section needs to be so big.
It takes quite a bit of space. So I will say the following. This section takes quite a bit of space vertically and I'm not sure if it's needed because all we do right here say finished, sources, minutes, searches, page reads.
I mean, we have the right sidebar for that. It's cool info, don't get me wrong, but I don't think that it's actually needed. Is there anything else missing?
I don't think so. So let me again collapse this, make it a bit smaller. I want to use my potato mode skill.
I will say peace there, call in potato mode. And here, let's maybe try out the mobile page. How does it look on a mobile device?
I can open this. Yep, we have a nice sidebar. We also have it here.
This looks beautiful. So let me click on enter and our agent will get started. It's also currently working, right?
We have a few sub -agents. Let me make this smaller. Yeah, three running.
let's wait for everything to finish and then we will check it out together. Alright, our agent is finished. So, you know what?
Let's check out the results. Let's start with the app icons. Here we now have five different variants.
And looking at them, I would say the last one is the best out of the bunch. Simply because it's unique and also because it right away pops. It uses the serif font with this beautiful background.
So, yeah, let's use it. I already wrote it down right here. 4D is selected.
So, serif R. A white italic serif with the dot. trail.
It's beautiful. Let's continue. What else do we have here?
Is this our right sidebar? I think so. So progress, budget, aha, aha, search, searched with the Codo.
That's great. And web data by the Codo. Zero complaints.
What do we have here? Yeah, I guess like a little animation. We can leave it as is.
Let's continue. What does it say here? Header.
It now holds the question. Status block. Ah, the header.
What do we have here? Budget running. cancel.
We will have to check it out, but it looks already a bit better. It's a bit more simple. Let's continue.
What else do we have here? Our sidebar is now collapsed. What options do we have?
The logo itself, the collapse button, create a new thread button, and a search button. And here we can switch the theme. These are all of our ASCII art variations.
San Francisco, what's this? Canyon River, Canyon Dusk, Globe. Looking at I strongly prefer the San Francisco art, if that makes sense.
Let's continue. What else do we have here? I mean, this looks also good, but I don't love it.
So I will also write it down. San Francisco or SF it is. Then what else do we have here?
Globe Corner. Yeah, that's all fine. That's also all fine.
I guess we are also finished. So let me quickly open the actual application. Here we have our art.
And if I do a hard refresh, it also... animates in quite nicely. Now looking at it there's one thing I don't like and that's that our design system currently does not work with our background.
Because looking at the San Francisco ASCII art we have this orange color, we have this purple color, this yellow color, whatever it might be. It does not really work with our current design system, which uses a lot of blue and gray. So therefore, we will also have to update the design system.
So let me right here say update design system to work with SF. So what else do we have here? Let me open a thread, a session, however you want to call it.
This one, for example, we have a nice animation. That's nice. The navbar is now way cleaner, which I like a lot.
We also don't have this annoying. info here anymore with how many searches have been made, how many maps have been made. It's not needed.
We can still view the models. So in this case, Sonnet 5 .5 and the exact date and time. I also have my returned data.
Let's check out the right sidebar. I can make it bigger, which I like a lot. So round, sitemap, searches, and pages read.
Whole run. Here we now have the info with the searches, pages read, and so on. Activity, I can make it bigger.
And on the bottom, it's web data by the kodo and I should also have the option to view it here. So for example it failed, the quote is not on the page word for word, not on page, read with the kodo.
I love it. This already looks so much better. So here's what I will do.
Let's go back and I will now create a prompt using the notes we made here. So looking at all of the app icons you created, I would say that we should probably go with 4D. It looks the best.
So the one with the seven with font. Then looking at the background, I would say we should go with San Francisco.
It works the best. It's perfect. Now, the problem with San Francisco is that it currently does not work with our design system.
Therefore, what I want you to now do is update our design system so that it works with our San Francisco Air Ski Art. Also, then in the like individual routes where we can view the generated reports, I would probably also maybe integrate Air Ski Art in some way, shape or form.
I'm not sure how to do it, but it would be cool, at least like simple details. Don't create a background or something like that, but simple details always work quite good. Less is more, remember that.
Is there anything else missing? Nope, I think that's it. Ah, maybe in regards to the app icons or to the app icon, the individual one.
Since we will now update our design system to match with our San Francisco background, we will probably also have to change the background of the app icon. So the serif idea is great.
I don't want to change it. We will probably just have to change the background. Another thing I want to also update is the serif icon right away.
And is there anything else missing? I don't think so, right? It should be it.
Ah, now I remember. There was one thing and it's the logo on the top left, so the text. So let me say the following here.
I forgot there's one more thing and that's the logo. So the app icon is finished, right? We discussed it.
But the text Research Marshall, I'm not a big fan of it right now. Because Research uses the serif font and Marshall uses... I guess it's inter or something like that.
That does not work. Let's keep it just one singular style and it shouldn't be serif. It should be the normal style.
Let's maybe make the leading a bit bigger or smaller. I'm sorry. Let's make the font size a bit bigger and let's keep the research white.
And then the Marshall should have the primary color, whatever you decide on. Before you start creating anything or updating the code, let's maybe first of all set or let's first of all decide on a primary color. So look at our ASCII.
San Francisco background and propose whatever, in your opinion, works the best. And also, right away, think about all of the secondary colors, think about the contrast ratio, and then, yeah, let me know what you would suggest doing. Alright, so this here is now the proposed, how should you say, like, color system, color schema, whatever you want to call it.
The primary color will now be orange. Interesting. Okay.
Primary text, secondary, highlight, success, warning, and failed. Ah, no, no, no, no, no. I don't like that.
So having a failed color or whenever thinking about success and failed, what colors do you think about? Probably green and red, but definitely not pink. That does not work.
So I want it to be here green. Probably a darker yellow tone. And this here should also be red.
Let's also check out the light color theme. So failed pink again. It does not work.
Warning brown. It looks weird. It does not work.
So that's something we will have to change. So you know what? Let me create the prompt right away.
The idea is good. I'm a big fan of the primary color, primary text. All of that works.
The only thing I don't like is the success warning and failed colors. Because normally these colors are very static. standardized.
Success is green, warning is dark yellowish, and failed should always be red, definitely not pink. And the same thing also applies to our light theme. Success should be green, warning shouldn't be brown, but again, darkish yellow, and failed should also be red.
So please update that, and let me again first of all review the colors. Alright, let's check it out. So primary orange, success green, warning yellow, failed is then red.
This already looks 10 ,000 times better. And the light theme is also quite nice.
So primary success warning failed. It's orange here, orange here. So let's now do the following.
I will use PStack. So PStack potato mode. And let me create the following prompt.
So this is approved. I'm a huge fan of the proposed color system, color schema, however you want to call it. So get started with the implementation.
And you know what? Let's also implement all of the items we talked about previously, like the app icon. the font of the logo, stuff like that.
You know what? There's one more thing I want you to implement, and that's a separate design system route. So let me say the following.
In the screenshot, you already created a basic design system. You could say overview. What I want you to now do is create a separate route.
Let's call it design system as an example. So localhost 3000 slash design system. And inside of there, I want you to showcase all of the colors, our font sizes, maybe images, whatever it might be.
With that, I will be able to share my design system with other colleagues, but also point my agents to the route. And since you are already at it, maybe it would make sense to create a skill. You know what?
Let's do that. a design system skill which explains our colors, our font sizes, that will be even better for agents because a full -on HTML route is not super passable, for agents at least. Alright, we are now finished with phase 3, at least hopefully, and I need to warn you right away, because I stopped the agent mid -run and told it, hey, we need to change the plan.
So, what do I mean by that? Well, the agent was working, I was looking at all of the things it was doing and also the screenshots it was creating. And while looking at them, I wasn't a big fan of the blue background.
So, I said the following, I'm currently looking at the implementation and the agent shows a bluish background. The problem is, I feel like bluish backgrounds always look almost cheap if that makes sense and the agent agreed i think you are right and i would change it so i said hey give me then a quick side by side so that i can review everything and i mean look at the side by side be honest does a look better than b No, we look so much better.
It's so much cleaner, so much more sophisticated. Look at the blue background. No, I don't like it.
So let's close this again. So I told the agent, hey, looks great to me. Let's go with the dark background.
The agent did that and now we have a completely new website with a dark background and also this whole dark aesthetic. And you know what? Let's test it out right away.
Let's start with our design system route. So the design system of Research Marshall. idea we have our beautiful san francisco background the individual colors which make up the background here we then have our color schema color system so surfaces page side by end frame card etc what else do we have here text and you know what's quite cool we even have like this aspect ratio validator 6 .47 pass 8 .37 pass everything passes and that's exactly what we wanted so these are all of the colors Now, we don't have to look at all of them.
It's fine. So let me scroll to the bottom. Where do we have our typography?
It's right here. We have the font families, Geist and Geist Mono, the scale, 400 for running text, 500 for control, 600 for heading. So these are the font weights.
What else do we have here? Rules, shape, and depth. Man, it looks beautiful.
I love it. That's exactly what I wanted. So this is now something we can share with humans, with clients, with co -workers, whatever it might be.
But we can even point our agents to this file here and tell them, hey, please look at it. Make sure that you follow these requirements and get started. As an example, I can even change the theme here.
Man, man, man, we did such a great job. I'm so proud of us. And since we now know that everything works in terms of our design system, we can now continue with our application.
So let me go to localhost 3000. And currently we don't have anything here because it's a fresh database. Our product.
database. Let me zoom out a bit. I already asked my agent for a question.
So, for example, how long do lithium iron phosphate batteries last and what shortens their life? Let me copy the question. Let's go back.
I will paste it inside of here. What model should we use? Sonnet 5 .5, Opus 5 .5, maybe even Fable 5 .2.
Honestly, I would say Sonnet 5 .5 is perfect. So, what do we have here? Let me zoom in a bit.
We have our question. We have the model, the time and also with that the date.
And here look at this like animation. Searching the web we have a loading indicator and this is our Golden Gate Bridge. And you know what's quite cool?
I just realized that this Golden Gate Bridge actually functions as a loading progress indicator. The more pages we scrape and with that also read, the more dots turn orange which is super cool. Let's also open our right sidebar.
What do we have inside of here? Round two of three, sitemap zero, searches 4 out of 5, pages read 8 out of 8.
Now, what do we have inside of here? Let me start from the top and let me also zoom out a bit. So, the first step was the plan.
Five searches planned. We took the main question and broke it down into five sub -questions. Why is that?
Well, if we go back to our diagram, then you will see here, the input is our question. One prompt typed into the application. The second step is to break the prompt into three to five sub -questions, each with a search query.
That's exactly what we did here. So let's continue. What was the next step?
Search. We used the Dekodo search endpoint to search for the needed results. And here we now have Reddit.
What else? Facebook. Epochbatteries .com.
Smart Propel, which is quite cool. We then had another search round, another search round, another search round. And then we selected all of the needed results.
Meaning, if I go back to our diagram, we had our selection process. Model picks by index. Code caps it at the budget.
Let's again go back. So select is then finished. And now we already have our scrape step.
The degradation behavior of the following batteries. And here we even have all of our claims. But they, for example, failed.
The quote does not support the claim. The quote says SOC at both ends. Blah, blah, blah.
It's not important. I'm not some sort of battery researcher. I don't even understand what it means.
But it's cool. So let me again close this. So red, red, red.
Organization forum. all of the, how should I say, categories. We have the blog category, the forum category, the organization category.
Organization means it's some sort of company. We even have a few skipped pages, 28 pages found and not read. Gap check, not answered.
Aha, interesting. And then we have round two and let's see round three. And is everything finished?
Yes, report written by the model and completed. Guys, I'm so proud of us. It's so cool.
That's all. check out the sources. These are all of the sources which are used by our article, by the report, and we even can view all of the individual claims here.
The verified claims and the dropped claims. 45 are verified, 105 are dropped. And we also always have the reason for why the claim was dropped.
So, let me again close the right sidebar and here we now have our beautiful report. Now, I'm not sure if I'm the biggest fan of these orange dots. But I guess it's fine.
So here we have the report. We also always have our citations. And now look at the hover effect.
If I hover over one, it gives me this pop -up with the actual citation. And if I now click on it, it should automatically redirect me to the specific website. One, two, three.
Oh my god, it works. It's so cool. And at the bottom, I can again view all of the sources with also the citation IDs.
And that's exactly what I wanted. So, you know what, let's also see if everything works in parallel. I will now copy this rsodium ion batteries ready for grid storage question.
Let's go back. I will create a new thread, paste it inside of here. Bum.
Let's go back. I will then also copy this question. Bum.
Let's go back. I will create a new thread, paste it inside side of here.
Let's continue. I will copy this question and I think that will be enough. So I will add it inside of here.
Let's now switch the model, this time Fable 5 .2, and I will click on enter. And now we have three parallel sessions. And again, look at the loading indicator.
It looks so cool, man. I love it. Great job, Opus 5 .5.
Let's open the sidebar. What do we have here? So we see that everything is currently running.
Our agent, no, sorry, the Codo is currently scraping the needed data and it's loading and here we even have the decodo logo and if i open this you will also find all of the returned results it looks beautiful and the nice thing is everything loads in parallel and look how smooth everything is the response is instant essentially this is not cache now of course this application currently is not deployed but that's the power of cache components that's the power of using a project specific verification skill and also instructing the agent to optimize for performance and if i now go back to the Kodo dashboard, look at the total requests.
And also look at the success rate, 95%. That's super, super good. If you have ever worked with a scraping tool, then you will know that anything above 70 % is already good.
95 % is on a different level. Let's head over to the scraping section, web data API, and I want you to look at the usage statistics. What do we have here?
Our requests, response time, and also the traffic. Let's also look at our top targets, Google. That makes sense, we are using the search endpoint and it uses Google to search for results.
Epoch batteries, then also Wikipedia. Don't worry about the Amazon link here, it was for a different project I was working on. But yeah, in general, everything is working as expected and everything is also super reliable.
And look at this smooth animation. I will click on the collapse button in 3, 2, 1. Wow, it's so good.
But in general, everything works exactly as expected. If we look at our research pipeline diagram, everything starts with the question of the user. We then break the question into three to five sub -questions, each with a search query.
For that, we use the model already. Then we use the coder to search for these results. So Google results for every sub -question.
After that, we have a selection process. Model picks by index, code caps it at the button. it.
Once that's done, the Kodo is again used to scrape all of the needed data. Each PICS markdown, nicely formatted markdown which is LM ready and cached in our SQLite database. As an optional step, we also use the Kodo's map endpoint to get all of the subpages of a specific website.
Then we have our extraction step, so claims from each page, each with an exact quote. We take the nicely formatted markdown, we look at it, and we try to extract the needed citations.
We then have our verification step and if everything looks good, if there are no gaps, we then have the model which generates the report. If there is some sort of gap, if there's not enough data, not enough citations, we go over and again start with the next round. Round two, we can use our search endpoint, select, read, extract, verify, gap check and as you already know, it goes in a continuous cycle until we have the needed data.
And now you might say, Jan, what about the map endpoint? We haven't really seen it in action yet. Does it even work?
Of course it does. So here I now have a new question. What rate limits does the Kodo's own documentation give for its web scraping API?
And look at this. We are now using the sitemap. Why is that?
Well, essentially, we now have one website, the Kodo, and we want to get information from the Kodo's website. So instead of using the search API to find all of the needed subpages, it's easier to use the map endpoint because we have one origin, decodo .com, and we need the subpages of decodo.
So yes, we could use the search endpoint. It would give us all of the needed information, but it wouldn't be as reliable. And that's why we now have here our map endpoint.
Let's quickly look at the activity. So plan, search, search, search, select, and then map. 100 listed, 3 picked.
And these are the 3 picked endpoints. And while decodo now handles all of our data so scraping searching mapping whatever it might be we can quickly head over to t3 code and now also merge the pr because here's what has happened in the background we used our project specific verification skill internally with sub agents we then had our pr which got created and in the pr i had cursors bugbot and also devon run they reviewed the code and then the lead agent produced a handover with all of the evidence screenshots and also information about what it did, what failures or issues it had and fancy stuff like that.
I myself now also reviewed the code, 24 ,000 lines of code, though not everything was needed. For example, looking at this layout, you don't really have to sit there for 30 minutes and look at it. It's very simple.
You have to just fly through it. You know what? Let me give you like a final recommendation in terms of code reviews.
Now, I already said like sit down, look at the code. I know it's boring, blah, blah, blah. And all of that is true.
But you might ask me, Jan, do I need to understand every single line of code? Oh, what is this function? What does it do?
Where is it connected? Oh, what are these parameters? Where do we get them from?
Oh, the use ID again. What's this function? No, that's not needed.
And honestly, that's not scalable. I myself generate about 50 ,000 lines of code every single day. And now let's imagine I would review all of these 50 ,000 lines of code line by line.
That wouldn't work. It's not scalable. I would probably try to find a new profession if that would be the case.
It's not fun. So no, you don't have to review every single line of code, understand every single line of code. The goal is to have a good general understanding of the code and with that, what it does and how it works.
That's enough. And that's also what I do every single day, including a lot of senior engineers I know, which work at companies and get way too much money every single year. But yeah, with that out of the way, we are now finished with the video.
Thank you. the Kodo for making this video possible. I mean, this video is almost three hours long.
It took forever to create. So please show the Kodo some love, scan the QR code or head over to my YouTube description and click on the first link. And using the code you will now see on the screen, Marshall10, you will get 10 % off on any plan.
As already mentioned, the Kodo has a super generous free tier with up to 10 ,000 requests and $5 in free credit. Use it, try it out and trust me. you will love the Kodo as much as I do.
Please also don't forget to like and subscribe, it would mean a lot to me and my heart, so please do it. But yeah, enjoy your day and see you in the next video. Over and out, bye bye!
The Hook
The bait, then the rug-pull.
Two engineers get the same model, the same agent and the same assignment. One types 'build me a research agent' and gets dead buttons and invented sources; the other writes requirements first and ships. The rest of the video builds that second version live, start to finish, with every prompt on screen.
Frameworks
Named ideas worth stealing.
15:43list
Three-part prompt
Goal
Constraints
Definition of done
Every prompt states what to achieve, what limits apply, and the checkable condition for finished. Nothing else is engineered in; smart models get room to reason.
Steal forany agent task brief or delegation note
08:05list
Research Marshal core primitives
Live web data
Every claim has a quote
Hard budget (3 rounds, 8 pages, 1 map)
Failures are visible (failed, skipped, dropped)
Four non-negotiable rules that define the product before any code, plus an explicit not-in-this-build list (no accounts, no full-site crawls, single local user).
Steal forscoping any AI product so the agent knows where to stop
04:53model
Model roles
Lead: smart and expensive, plans and reviews
Implementer: workhorse that writes code
Researcher: cheap and fast, reads and searches
Match model cost and capability to the job. Lead was Fable 5.2, implementer Opus 5.5, researcher Sonnet 5.5 or Haiku 5.5.
Steal forkeeping multi-agent runs affordable
44:58list
pstack primitives
Playbooks (23)
Functional skills (33)
Principle skills (24)
Agent definitions (12)
Supporting scripts
Playbooks set step order per task type, functional skills say how to do each step, principle skills supply judgment rules, agent definitions define delegated roles, and scripts handle PR plumbing reliably.
Steal forstandardizing an agent's process across every task
48:52model
Agent loop
Your task
Router skill picks a playbook
Playbook sets the steps
Do the work with skills and sub-agents
Verify and review (loop back if checks fail)
Verified handover with evidence
The request is routed to a fixed process; verification sub-agents interrogate the work and send it back until it passes, then hand over screenshots and logs.
Steal forreplacing step-by-step babysitting of an agent
1:10:44list
Three pillars of code review
In-loop sub-agent verification
PR review bots (Bugbot, Devin Review)
Human review of mission-critical or flagged code
Layered review so a local miss is caught by the PR bot, and human attention goes only where stakes or warnings justify it.
Steal fordeciding how much AI code to read yourself
1:16:12model
Research pipeline
Plan 3-5 sub-questions
Search
Select by index, capped at 8
Scrape to Markdown and cache
Optional sitemap map
Extract claims with exact quotes
Verify quotes in code
Gap check, up to 3 rounds
Write cited report
Model steps, API steps and plain-code steps are separated so the non-AI verify step can throw out any claim whose quote isn't on the page.
Steal forany LLM feature that must cite real sources
1:40:48model
Verification skill loop
Launch
Doctor
Drive
Evidence
Cleanup
Make a small change, drive the real app along a feature map, inspect screenshots and stored values, and repeat until clean.
Steal forgiving agents a reliable way to prove UI and pipeline changes
36:12concept
Token degradation curve
Quality holds near 100% early, declines gently through a few hundred thousand tokens, then falls steeply around 500k. Small scoped sessions keep work in the high-quality zone.
Steal forplanning session boundaries on large builds
CTA Breakdown
How they asked for the click.
VERBAL ASK
2:38:46link
“Please show Decodo some love, scan the QR code or head over to my YouTube description and click on the first link. Using the code MARSHAL10 you will get 10% off on any plan.”
Sponsor is woven into the build rather than isolated: a ten-minute segment framed as the answer to a scraping problem list, then real use throughout (MCP testing, the Google redirect bug, usage dashboards). Closing ask repeats the code and free tier, then like and subscribe.
Add Modern Creator as a preferred source and Google shows you more of our breakdowns in Search, Top Stories, and AI Overviews. It only changes what you see, and you can undo it in your Google settings anytime.
Add to Preferred SourcesOpens your Google source preferences with us pre-loaded. Tick the box and you're done.
A six-hour live build of an Intercom-style support platform, from idea and PRD to a production deploy, with the workflow doing more work than the model.
A 42-minute live build of a link-in-bio app that walks through one simple greenfield workflow for coding agents: a PRD, a small AI layer of rules and commands, and repeatable plan, implement, validate loops.
A screen-recorded walkthrough of HyperFrames, HeyGen's free open-source video editor, built entirely from prompts inside a coding agent like Codex or Claude Code.