14:43Jack Roberts · Tutorial
Hermes Agent is the Greatest AI Tool Ever
A 14-minute walkthrough on wiring MiniMax M3 into the Hermes agent to get frontier-class AI at 4-8% of the usual cost.
June 23rdThe argument in one line.
ChatGPT Voice 2.0 replaced turn-taking voice chat with a persistent agent that keeps working while it talks, follows the user across every app window, and can be given vague, multi-part instructions instead of precise commands.
Read if. Skip if.
READ IF YOU ARE…
- You already work across ChatGPT, Codex, or Claude Code daily and want a voice layer that follows you between apps instead of a separate chat window.
- You run several concurrent tasks (client work, research, personal admin) and want to hand off multiple jobs with one spoken instruction.
- You run a local agent (Hermes, OpenClaw, or similar) and want a voice-triggered remote control for it.
- You're curious what actually changed under the hood in the newest OpenAI voice models versus the old back-and-forth voice mode.
SKIP IF…
- You need this to work on Windows, or need live screen-sharing on something other than a Mac desktop.
- You want a live camera feed you can point at a physical object and ask about — that's a separate mode this doesn't include.
The full version, fast.
OpenAI's ChatGPT voice mode changed from a turn-taking system into a persistent agent that keeps talking while a separate reasoning process works in the background, so the assistant no longer goes silent while it thinks. The video demonstrates five escalating levels: giving it three simultaneous tasks (desktop cleanup, live web research, building a landing page), controlling a desktop from a phone, handing it a vague idea and getting a finished brief back, triggering a separately-running local agent by voice to check its configuration and set reminders, and letting it click buttons and fill out forms hands-free using context it already knows. Two limitations are flagged: no live camera feed, and screen-reading only works on macOS.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 01:48
01 · The Jarvis pitch
Opens claiming the rebuilt voice mode is essentially Jarvis — it connects tools, works from the phone, and changes how you work if you know the right use cases.
01:48 – 02:24
02 · What actually changed under the hood
The app looks the same, but the two models running underneath it changed: it now keeps talking while a separate process thinks in the background, and handles real-time interruption instead of strict turn-taking.
02:24 – 02:30
03 · Five levels overview
Introduces the five-level difficulty ladder the rest of the video is structured around, from easiest to hardest.
02:30 – 06:01
04 · Level 1 — Multitask three jobs with one voice
Gives the agent three simultaneous jobs — organize the downloads folder, run deep web research with Firecrawl, and build a client landing page — while it checks the screen for context, catches a contradicted instruction, and keeps following across tabs.
06:01 – 06:46
05 · Level 2 — Steer the desktop from across the room
Signs the same account into a phone and shows commanding the desktop remotely, including asking it to check and report on what was last saved — turning the phone into a remote control.
06:46 – 09:01
06 · Level 3 — A vague idea out, a full brief back
Hands the agent loose, underspecified requests while browsing — comparing ElevenLabs pricing tiers, explaining a paused YouTube video, debugging console errors on a page — answered using whatever's currently on screen.
09:01 – 10:49
07 · Level 4 — Trigger a local agent by voice
Uses ChatGPT voice to talk to a separately-running local agent (Hermes) on the same machine, asking what model powers it and setting a reminder for the next morning.
10:49 – 12:20
08 · Level 5 — Fully hands-free browser control
Lets the agent click through a character-selection screen and a multi-question onboarding quiz using only spoken instructions, then fill in the quiz answers itself.
12:20 – 13:19
09 · Honest limitations and what's next
Flags that browser-driven voice control isn't fast, that there's no live camera mode, and that screen reading only works on macOS, before pointing to a follow-up video on the Hermes agent.
Lines worth screenshotting.
- ChatGPT's voice agent now keeps talking while a separate process reasons in the background, instead of going silent until it has a full answer.
- The same voice command can dispatch three unrelated jobs at once — desktop cleanup, live web research, and building a landing page — without waiting for one to finish before starting the next.
- Using a scraping tool like Firecrawl instead of raw browsing cuts the token cost of research tasks, which lowers both the bill and the wait time.
- The floating voice widget persists across every browser tab and app window, so a single running conversation follows the user instead of resetting per tab.
- The agent takes a screenshot to check context before acting on the desktop, which is what let it catch a contradicted instruction instead of executing it blindly.
- A ChatGPT account signed into both a phone and a desktop turns the phone into a remote control for anything running on that desktop.
- Giving the agent a vague verbal idea while it's already looking at an open page produces a usable answer faster than typing out a precise prompt.
- A locally-run agent can be triggered, queried, and given new reminders entirely by voice from the ChatGPT app, because it runs on the same machine the desktop app already controls.
- The hands-free browser mode can click buttons, select options, and complete a step-by-step flow using only spoken instructions.
- Given enough conversational context about the user, the agent will fill out an entire multi-question form on its own rather than asking the user to answer each field.
- Browser-driven voice control is noticeably slower than direct, API-based actions, since every click has to route through actual page rendering.
- The desktop app currently has no live camera mode, so pointing a phone at a physical object and asking about it isn't supported inside the same voice session.
- Screen reading is macOS-only right now, meaning the assistant can see what's on screen only on Mac desktops, not other operating systems.
The five levels of what ChatGPT's new voice mode can actually do for you.
WHAT TO LEARN
The real shift is a voice agent that keeps working while it talks and follows you across every window — useful well beyond dictation, with two clear limits worth knowing before you rely on it.
02What actually changed under the hood
- The interface didn't change; the two models running underneath it did, which is why the assistant now keeps talking while a separate process reasons in the background instead of going silent.
- Real-time interruption means you can talk over the assistant mid-response and it adjusts, rather than waiting for a fixed turn to end.
04Level 1 — Multitask three jobs with one voice
- One spoken command can dispatch three unrelated jobs at once — organizing files, running live web research, and building a webpage — without waiting for any one to finish before starting the next.
- The agent checks the current screen before acting, which is what let it catch a self-contradicted instruction instead of just executing the first version it heard.
- Swapping raw browsing for a scraping tool during research cuts token usage, which lowers both the cost and the time a research task takes to come back.
05Level 2 — Steer the desktop from across the room
- Signing the same account into a phone turns the phone into a remote control for anything already running on the desktop, useful for triggering or checking work remotely.
- The agent can both retrieve information about the desktop's current state and act on it remotely, not just report back on what it finds.
06Level 3 — A vague idea out, a full brief back
- Giving the agent a loose, underspecified question while it's already looking at an open page gets a usable answer faster than typing out a fully specified prompt.
- It can read complex, unfamiliar web pages and technical console errors directly and explain them in plain language, without switching to a separate chat window.
- Which underlying model powers a voice session is picked once at the start and applies for the whole session, so the choice is worth making deliberately.
07Level 4 — Trigger a local agent by voice
- A separately-running local agent on the same machine can be queried and controlled through the voice layer, because both are operating on the same computer.
- Asking the agent to check its own configuration gets a real, verified answer rather than a guess, because it reads the config file directly instead of assuming.
- Voice commands can set reminders handled by a completely separate local agent rather than by ChatGPT itself, using voice purely as the trigger.
08Level 5 — Fully hands-free browser control
- Hands-free browser control can click through a multi-step flow using only spoken instructions, with no manual clicking required.
- Real browser-based clicking through voice is noticeably slower than typing or using an API directly, even though it completes the steps correctly.
- Given enough context about the user, the agent will fill out an entire onboarding-style questionnaire on its own, choosing answers based on what it already knows.
09Honest limitations and what's next
- There is no live camera mode — you can't point a phone at a physical object and ask about it inside the same live voice session.
- Screen reading only works on the macOS desktop app right now, not on other platforms.
- Session memory is real but shallower than ChatGPT's full memory system, so referring to a specific file directly gets more reliable results than assuming it remembers everything.
Terms worth knowing.
- ChatGPT Voice 2.0
- The newest OpenAI voice mode, built on updated models, that lets the assistant keep working and talking at the same time instead of pausing after each turn.
- Firecrawl
- A web-scraping tool that converts pages into clean, AI-readable content, cutting the token cost of research tasks compared to raw browsing.
- Hermes agent
- A locally-run AI agent, built on Nous Research's Hermes model, that operates on the user's own computer and can be triggered remotely by voice.
- Codex app
- OpenAI's coding-focused assistant app, used here for building projects and web pages from loosely specified spoken instructions.
- Agentic voice mode
- A voice interface where the assistant doesn't just answer questions but can carry out actions — clicking, typing, running research — on the user's behalf.
Things they pointed at.
Lines you could clip.
00:07
“It is essentially Jarvis. It connects all of your tools, works from your phone, and it will change the way that you work.”
01:58
“The reason why it feels smart is simply because it will keep talking to you whilst it's thinking in the background, which means you're not kind of waiting fifteen working days to get an actual response back.”
11:51
“This is not the fastest thing on the planet if it's doing browser control. It's great if you want a hands free experience... but it isn't insanely fast.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
metaphoranalogy
OpenAI just dropped ChatGPT voice two point o, and it unlocks incredible capabilities.
It is essentially Jarvis. It connects all of your tools, works from your phone, and it will change the way that you work, but only if you know how to use it properly and the best use cases. And in this video, I'm gonna show you the five levels of ChatGPT voice step by step.
So you know exactly what to expect at each level, the key mistakes to avoid, and so you can get light years ahead of everybody else and save time. So if you haven't already, grab that beautiful coffee, and let's dive straight in. Now open up the ChatGPT app or the Codex app.
And by the way, these use cases are gonna blow your mind. They are incredible. Now I'm gonna come down and click on start voice.
Very, very cool, like so, and then we can begin the conversation. So it works across our desktop. We can explore, we can plan, we can learn, and it works across our projects with some really cool use cases that you have to understand.
So we go ahead, we pick our voice, and then we're ready to rock and roll. Hey, there. Could you do me a favor?
Could you do a quick Google search and find for me who is the tallest man that ever lived, and then save a file on my desktop called tallest man, and have that information inside, please. On it.
I'll check that and save the file to your desk And crucially, guys, we can basically mute this, move it around if we want to. We're gonna allow this all the time, so we don't have to basically keep on permitting it. So now we have a basically conversational interface with ChatGPT, and it can do things on our desktop.
As you can see, we have the information right there. Now before we go through level one, we first have to understand what's actually happened here. And by the way, if you're new, I'm Jack.
I built a Somaletech startup with a gazillion customers, and now I share here the stuff that actually works. If you find this helpful, feel free to comment and subscribe down below. Now here's the thing.
The app hasn't changed. The engine behind it actually has. We have two new models, already has a 150,000,000 weekly voice users.
And instead of having back and forth, you can basically talk to this agent in real time. So you can interrupt each other and it will absolutely get what's going on.
And the reason why it feels smart is simply because it will keep talking to you whilst it's thinking in the background, which means you're not kind of to get an actual response back. And as you can see, the performance upgrades here are incredible.
Now the cool thing here is we can do this on your current desktop, your laptop, and we can also do the exact same thing from a phone, which is incredible.
So we're gonna cover ChatGPT voice across five levels of difficulty, beginning with level one, which is its ability to multitask.
Check this out. Let's begin by pressing the toggle down below and let ChatGPT voice launch. Now I'm gonna give it three tasks simultaneously.
Hey there. Dude, do me a favor. First of all, want to go ahead and clear up my desktop.
It's looking a bit messy. Then I want to go ahead and open a brand new tab inside of the Codex app, and I want you to do some deep research using Firecrawl on the best strategies to grow your x account in 2026.
Make sure that you do some good research, find loads of different competitors, and make that fantastic. And whilst you're also at it, could you just build for me a really simple landing page for my client's website?
Make it interactive. Make it beautiful. Lots of different HTML.
I'll start on those together. To clean the desktop safely and sheet the landing page, I'm checking your screen in context first. So this is great actually, guys.
And I love the fact that I can move this around. It's fun to me that it yeah. It may not be convenient for me to have it down below.
So we can move this anywhere we go, and it will follow us around on any app that we're on. For example, now I'm over here on a different page. Again, any website I'm on, I can be followed along by this wonderful little icon that we can chat to at any time that we want to.
And there's some incredible capabilities that this unlocks in a second, which I'll touch on very briefly. Now again, it's doing all those different things by itself, and it's asking for clarifying question.
Before we move anything over, watch this day on your desktop versus go. Hey, dude. What I'd like to do is basically keep everything on my desktop.
I just want you to organize them into better folders so so I don't have any loose files. In fact, scratch the desktop. Just do this with my downloads.
Don't touch anything on my desktop, please. Okay. Cool.
I even gave it a slightly contrarian command now, and we're gonna see it do these three things simultaneously. Now the reason that I'm using Firecrawl here is simply because it's gonna save me a ton on the tokens that I'm using. And I've always found whether I'm building websites or doing any kind of activity, using the ability to scrape, interact, and pass websites saves you lots of tokens and actually cost, which makes it a faster experience and you get way better results because of using this kind of thing.
Now what's really cool here, the fact that we actually have tracked it between the website, one of the things I can do while I'm here is actually have a conversation with it. So I can even say, hey there, I'm on the Firecrawl website, so we're using this, right, to scrape things with our tools. What I want you to do is, can you just do me a favor and explain to me the things Checking I can do on this page,
that page now. You're on Firecrawl's search playground. You can search the live web and turn results into clean AI ready content.
There's also a scrape for one URL crawl Beautiful.
So you can see, basically, now, to Boutique can be on the website and explain some things to us, which is massively helpful. And again, we can bring it anywhere and continue on our conversation which is great. To do that, it does have to take a screenshot of the web page which is fine, but again, it just saves you that task.
And as you can see, Chattypeteer has even opened up a new tab as per my instructions that that we can click into and have a look if we want to, which is great. Beautiful. Then within the tabs, we've got everything.
We have the x report, don't report the AI news, tester, earn attention, make visible, convert intent. Really nice, crisp thing it's done via voice mode. Then we have actually, interestingly, a couple of different options here.
We have a website here, which is cool. We make what tomorrow needs. Really good.
It doesn't feel like a classic AI slab website. Gonna come down here, you can see proofing up promises. I think this is really good, guys.
I just did this on voice mode. Very cool. But this is the kind of thing you can do with it.
Give it multiple commands, give it permissions, and let it rock. So now we've given it three different jobs with one voice. We can then start looking at actually taking this a level further actually, is the ability to actually go ahead and use ChatGPT when we're not even at our desktop.
Now all you've got to do is connect your ChatGPT account to your desktop and you can download that by site basically, signing into the exact same account.
So let's say, for example, that you're heading off to the coffee shop. I can come down and ask it any question about anything on my desktop. I can even get it to build for me.
I could say, hey there, man. Could you do me a favor and just remind me what was the last thing I saved on my desktop? Let me check your desktop now.
Checking the most recently saved item. Not only can it retrieve information, it can actually do work for you on your desktop as well. So now you have the multiple tasks.
We have the ability at level two to effectively control this call, a kind of agentic voice mode anywhere we go. That takes us nicely onto the third level of capability, which is the ability to give it a very vague idea and get something significant back.
And what I mean by that is once you bring it up in any window, you can actually go ahead and use codecs for different tasks. For example, let's say that I'm on this page here. I might just have a chat with Codecs and have a conversation.
So you could say, for example, hey there, I'm just trying to buy one of these products I can see on the screen from Eleven Labs. You know a lot about my contacts, know about the build I'm trying to do. Which of these four would you recommend that I go for?
Free starter, Crotor, or what? Okay. I'll check those plans and match them to your use.
Got it. I'm looking at the 11 laps plans on screen, and given your build, I'm narrowing down which plan fits best. So as you can see, it already has the context of what we're doing.
And what I like most about it guys is that you can have this conversation, and then I could do other things. I could just go on to a separate web page.
I could be going to YouTube. I can meet it. I can do anything I want to.
And it follows you around the internet. Now, what's also really cool about this is, let's say that I click on a random video. Okay?
Like some. I can right click on something, and actually, on the web page itself, I can come down and do inspect. So if I'm looking at anything online, I'm doing work, you can see there's loads of errors on the right hand side that happen sometimes when you're building apps.
I could, for example, do this. Come over to Attract WPT and say, hey there, I just want you to explain to me the errors on this page. What's going wrong here?
Sure. I'll check the errors on this page. Beautiful.
And then you can carry about your day to day doing everything you want to, which helps you build better apps because you have this blue thing following you around everywhere that you go on all the different apps that you're using, which just enables basically just saves you time as you're going through everything. Now when you do actually load up the ChatGPT voice mode, you can pick the voice that you want.
Obviously, you go from like GPT 5.6 Sol, Terralight. I quite like GPT 5.6 Sol High when I do it. I come down, and then when I open it, that is the exact model that you're going to be talking to when it loads up, which is massively helpful.
And by the way, if all this sounds like I'm speaking Spanish and you do wanna go deeper, I'm gonna put a link down below for the full coding ClawCode Masterclass we built that takes you through foundations, building websites, memory systems, Hermes agent apps, and everything else. All the principles that work across ChatGit team codecs are exactly the same in Clued Code and you can go even deeper in some respects.
I'll put a link down below for that if you wanna go ahead and grab that. Now what's really interesting is that we talk a lot about Hermes agent right now. If I come over to Hermes agent, check this out.
I can actually trigger my Hermes agent by my ChatGPT, which is incredible because I run Hermes agent on my computer, which is awesome.
But I can trigger it both via voice here and on my desktop. And the reason why this is such a huge hack, whether you're using OpenClaw or Hermes Agent, it's incredibly beneficial. Now check this out.
I could say to it something on the lines like this. Hey there. You have access to my Hermes Agent.
I want to go ahead and tell me what is the model that is powering Hermes agent right now? Checking the Hermes configuration directly rather than guessing. Okay.
Awesome. So it's going ahead and checking the configuration. Why is this handy?
You know, anything you're doing on your computer or if you are just out and about, you can drop these voice messages. If you happen to be building the ChatGPT or you want to connect it to your other agentic systems, we can do that using the software. Hermes is currently configured to use GPT 5.6 Sol, the OpenAI Beautiful.
That's massively handy. Million. Do me do me one more favor, please.
Could you set a reminder ask Homi's agent to remind me at 11AM in the morning to get my two iced coffees for my flight to Tavat, please? Sure. Setting that up.
And so we can also trigger reminders to our agents remind you at 11AM. Let's let's do that tomorrow. Let's do that tomorrow.
Okay. Locking that in for tomorrow. Beautiful.
So that's exactly how we can communicate with our Homebase agent because my Homebase agent runs on my laptop, my computer, my desktop. So because it runs on the same computer, it can all run on the same thing because codecs is there too. And it just unlocks new capabilities.
What I think is really cool is you can have this just following you around on everything that you're doing, especially if you've got a great, you know, conversation and you wanna have like support with building something, you can just bring it in. And in many ways, it's a nice kind of like if you think about it, UI update.
It really makes it a little bit nicer to use. Then this takes us on to level five, which is a really exciting update, and I really like this. Where effectively, you can be completely hands free and actually have this beautiful floating agent on our screen to do things for us.
For example, let's take a look at this roadmap that I built. Okay. This is something I built my community to help people with their like goals and strategies.
I could come down here and check this out. Hey there, bro. Could you do me a favor?
Go ahead and select the ninja, and then I want you to click on to the next page. Alright. Doing that now.
Ninja is selected and the next button is enabled. That sounds cool. Go ahead and analyze my profile.
It. Analyzing your profile. The profile analysis page is open and ready to run.
Cool. Now just click that button for me to analyze my profile. Okay.
Running profile analysis now. So what's really interesting here, guys, isn't the fact that it can click buttons. It's the fact that it can look at the screen, do various different things, analyze what's going on, and go to the next step.
But where it really takes to a new level is the fact that it can actually take all of the context and information that it has about you, and then specifically make decisions. What's really cool and interesting here, if I just come down to say $10,000 as an example, what I can do is actually get ChatGPT to fill in this information based on what's happening.
So I could be like here and say, hey, you know a lot about me, and it's asking a of questions. I'd love you to go through this and just help me select what you think would be most relevant based on everything about me and my situation, please. Sure.
I'll match the choices to your profile and goals. What's really interesting here guys is it will go ahead and fill out information for us based on everything that we know, which is great and just make things a million times easier. Now, we talk about realistically a couple of the drawbacks that are great to know.
The first one to know is that this is not the fastest thing on the planet if it's doing browser control. It's great if you want a hands free experience and run through things, but it isn't insanely fast. They are releasing a new model soon, I believe, that is even faster, which is gonna be incredible.
So a couple of good caveats to know guys is that it doesn't have a live camera, so you can't put your phone at something and ask about it whilst GPT live. The screen is Mac desktop only, so there's no screen sharing into a live voice session, but the Mac OS desktop app does have app shots so it can read whatever's on the screen.
Now it has a memory, but not quite as comprehensive as using ChatGPT. So if you're asking it questions, it's good to refer to specific files sometimes to get that extra level of detail.
So now we know exactly how to use this ChatGPT voice agent. The next thing that we need to do is learn how to take this to a completely new level, which we're gonna do together using Hermes agent, and we can do that in this video right here.
The Hook
The bait, then the rug-pull.
The title trails off mid-sentence, and the opening line does the same trick out loud — something 'just dropped' that 'unlocks incredible capabilities' — bait that resolves a beat later into the video's actual comparison: this is essentially Jarvis.
Named ideas worth stealing.
02:24list
The Five Levels of ChatGPT Voice
- Multitask
- Steer the machine from across the room
- A vague idea out, a full brief back
- Trigger a local agent by voice
- It spots what you missed
A five-step difficulty ladder the video uses to structure every subsequent demo, from basic parallel tasking up to fully hands-free browser control.
Steal forstructuring any tutorial as an escalating-difficulty framework instead of a flat feature list
How they asked for the click.
VERBAL ASK
13:19next-video
“The next thing that we need to do is learn how to take this to a completely new level, which we're gonna do together using Hermes agent, and we can do that in this video right here.”
Soft, single next-video pointer at the very end rather than a mid-video ask — no product pitch inside the video itself despite affiliate links living in the description.
MENTIONED ON CAMERA
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
OTHER LINKSAlso linked in the description.
Visual structure at a glance.
