21:08Riley Brown · Tutorial
Building a Real AI Jarvis with Cursor and GPT Realtime 2
A step-by-step build of a talking, screen-controlling desktop AI companion — no coding experience required, just long prompts into Cursor.
July 1stThe argument in one line.
xAI's new Grok Voice Agent Builder gets a working, calendar-connected phone agent live in minutes, but the polish that separates a demo from a sellable client product still has to be built by hand.
Read if. Skip if.
READ IF YOU ARE…
- You build or resell AI voice agents for local service businesses and want to know how a new no-code builder compares to Vapi or Retell before switching.
- You are evaluating whether Grok Voice's telephony and calendar integrations are mature enough for a paying client project.
- You want to see the actual setup friction (permissions, phone number regions, calendar OAuth) rather than a marketing feature list.
SKIP IF…
- You are not building or selling voice/phone AI agents - this is a hands-on platform test, not a general AI news video.
- You need a finished, production-ready walkthrough - this is an unedited first-look reaction, not a polished tutorial.
The full version, fast.
Eric Michaud, who runs a voice-agent agency, tests xAI's brand new no-code Voice Agent Builder for Grok by building two phone agents live: a moving-company appointment scheduler and a wellness-spa consultation booker. Both connect to Google Calendar, use file-based knowledge bases, and complete real test calls with correct availability lookups, self-correction on a misheard word, and an actual calendar invite created under his own account. He rates the voice quality and latency as genuinely good and comparable to Vapi, but hits real friction: calendar write-permissions are off by default, and free phone numbers are not available in his region (Canada), requiring a bring-your-own-Twilio-number workaround. His conclusion is a qualified yes: strong out-of-the-box templates and voices, but still requires per-business tailoring before it's client-ready, and it lags Vapi on SMS support tied to the same number.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 00:45
01 · What is xAI's New Grok Voice Agent Builder?
Reads the announcement email live; free phone number, telephony, knowledge retrieval, guardrails, observability, and bring-your-own numbers/APIs/MCP.
00:45 – 01:03
02 · How to Build a Free Voice Agent in Under 2 Minutes
Starts the builder flow, picks the appointment scheduler template.
01:03 – 02:13
03 · How to Connect Google Calendar to Grok Voice
Names the agent after a friend's real moving company, connects Google Calendar (Outlook also offered), sets up call transfer.
02:13 – 02:47
04 · How to Setup AI System Prompts and Guardrails
Reviews the default system prompt, welcome message, and advanced settings including speech speed and silence handling.
02:47 – 04:31
05 · Can Grok Voice Handle Live Booking? (Moving Company Demo)
First live test call: checks Saturday availability, books a 7am slot; notes he doesn't want it to actually write to his friend's calendar.
04:31 – 05:25
06 · How to Build an AI Knowledge Base Using ChatGPT
Prompts ChatGPT to generate a company knowledge doc, uploads it as a file collection.
05:25 – 06:52
07 · How Realistic Are the New Grok Voices?
Switches to the 'Lumen' voice, runs a second test call with a Saturday/Sunday availability question and a location self-correction.
06:52 – 08:39
08 · Is Grok Voice Latency Actually Fast Enough?
Evaluates response speed and correction handling; flags that templates need business-specific playbook rules beyond the default.
08:39 – 10:07
09 · How to Customize AI Agents for Service Businesses
Switches to his own community's wellness-spa template, explains the variables-at-top approach for reusable templates, enables calendar write actions.
10:07 – 13:21
10 · Live Test: Booking a Spa Consultation with AI
Full live call booking a real consultation for himself (Eric Michaud, Sunday July 12, 10am) on his actual Google Calendar.
13:21 – 13:43
11 · Did the AI Actually Book the Calendar Invite?
Cuts to his real Google Calendar to confirm the invite landed.
13:43 – 14:47
12 · Grok Voice Builder vs. Vapi: Which is Better?
Direct comparison: praises voice quality and latency, flags SMS-on-same-number as a Vapi advantage.
14:47 – 15:17
13 · How to Connect Twilio Phone Numbers to Grok
Tries to provision a free number, hits a region block (Canada), falls back to bring-your-own-Twilio.
15:17 – 16:42
14 · How to Learn to Build and Sell Voice Agents
Closing thoughts and CTA to his free Skool community with the templates and resources used in the video.
Lines worth screenshotting.
- A brand-new no-code voice agent builder can get a phone-connected, calendar-aware AI agent live in under two minutes using a starter template.
- Default calendar permissions on the builder ship with create, update, and delete actions turned off, so a template that looks broken may just be unconfigured.
- The agent handled a live self-correction (a caller restating 'Main Island' instead of 'Mainland') without losing context or repeating the error.
- xAI's free agent phone numbers are region-limited and were not available in Canada during testing, forcing a bring-your-own-Twilio-number workaround.
- The gap between a working demo and a sellable client agent is business-specific detail: a scheduling agent needs explicit rules like 'the first available slot isn't necessarily the next open hour' or it will overbook.
- Templating an agent with variables at the top (business name, hours, appointment length) lets one system prompt be reused across multiple client businesses.
- A well-tuned voice agent can go from cold call to a real calendar invite in under three minutes end to end, including a name capture and confirmation step.
- Voice agents in this category consistently need a human-authored 'playbook' layer beyond the base template - the raw model is rarely enough for edge cases like same-day emergency bookings.
- SMS support tied to the same agent phone number is a differentiator Vapi still holds over this new builder, according to a builder who has used both extensively.
A working AI voice agent and a client-ready one are different products.
WHAT TO LEARN
New no-code voice agent builders can produce a genuinely functional, calendar-connected phone agent in minutes, but closing the gap to something you'd hand a paying client still takes deliberate, business-specific rule-writing.
- A no-code builder can connect a phone agent to a real calendar and complete a live booking within minutes of starting from a template.
- Default permissions on new platforms are often conservative - calendar write actions (create, update, delete) may need to be manually enabled before an agent can actually book anything.
- An agent handling a live self-correction gracefully (without repeating a misheard detail) is a meaningful quality signal distinguishing mature voice AI from clunky implementations.
- Templates that work in a demo still need explicit business rules added - such as how to handle first-available-slot logic for emergency or same-day requests - before they're safe to deploy for a real client.
- Structuring a reusable template with variables at the top (business name, hours, service length) lets the same underlying prompt scale across many client businesses with minimal rework.
- Regional limitations - like free phone numbers not being available in every country - are common friction points in fast-moving AI tooling and are worth testing before committing to a platform.
- When comparing competing tools in the same category, evaluate concrete dimensions (voice quality, latency, permission defaults, phone number support) rather than headline feature lists.
Terms worth knowing.
- Voice Agent Builder
- xAI's no-code tool for creating phone-based AI voice agents on top of Grok, bundling telephony, a phone number, knowledge retrieval, and guardrails in one interface.
- Knowledge base / file collection
- A set of uploaded documents a voice agent can search during a call to answer questions specific to a business, such as services offered or hours.
- Guardrails
- Configured rules and constraints that limit what an AI agent is allowed to say or do during a live call, used to keep responses on-topic and safe.
- Latency (in voice AI)
- The delay between when a caller finishes speaking and when the AI agent responds; lower latency makes a call feel more like talking to a human.
- Bring-your-own number (Twilio)
- Connecting a phone number purchased from a separate telephony provider like Twilio to a voice agent platform, used when the platform's own free numbers aren't available in a caller's region.
Things they pointed at.
00:00productxAI Voice Agent Builder
04:44toolChatGPT
13:53toolVapi
13:53toolRetell
15:27toolTwilio
Lines you could clip.
08:28
“It's very easy to get it off the ground, but it's harder to build out something that's good.”
17:37
“Every voice agent of all time ever, there is some tweaking you're gonna have to do.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
00:00XAI just released their new voice agent builder, and I thought you and I could check it out live together today. Okay? I just got back to the desk and got this email here.
00:07So if you haven't received the email, they're basically saying that it is a brand new no code platform for production voice agents with Grok Voice. And the cool part is every account includes a free phone number. They're saying you also get telephony, knowledge retrieval tools, guardrails, and observability.
00:21I'm assuming that's, like, costs and stuff like that all in one place. And you can also connect what you already have, which is phone numbers, APIs, or MCP servers. The idea is it's all in one place.
00:31Considering this is a service I actually offer, it is definitely worth checking out, and I thought I would bring you along for the ride. I haven't looked at it yet. Just again, we're we're gonna check this out, me and you, together.
00:41So let's try it for free. Create your own voice agent under two minutes. Go try for free.
00:46Okay. So I guess step one is we just go through this.
00:50They have a couple templates. Describe what you're doing.
00:53Let's just do the classic appointment scheduler. Yeah? What do you think?
00:57This is the first page you look at. Like, I haven't been here before. Great.
01:01Appointment scheduler. Is there a website? Yeah.
01:03Let's use my buddies. Brown Bear Moving.
01:07Moving company here in Victoria. Brownbearmoving.ca. If you're moving on the Lower Mainland Canada, check him out.
01:14He's a good guy. Perfect. Brown Bear Moving.
01:16Google Calendar. Let's use calendar. Let's just use mine.
01:19Let's you select Outlook too. That's awesome. 365 sometimes is, like, hard to connect with.
01:24Properly, you can connect with it, but it sucks. Would you like the agent to transfer caller? Yep.
01:29I'm not gonna put my real phone number. Your Brown Bear scheduler is almost ready. Once it is ready, tap view agent, log in to Google Calendar and connection section, and turn on booking so it can schedule appointments.
01:39Uh, k. View agent. Okay.
01:45Call tools. Transfer call with the number I gave it. Right?
01:49Web search for brown bear scheduler. What is that? Oh, search for hours.
01:55That's strange. Anyway, end call. Uh, we can log in to Google Calendar.
01:59Let's do it. There you go. Done.
02:01Google Calendar. Credit's missing. I don't have enough credits, ad collections, knowledge.
02:06Okay. Check out the voices. Have you heard the new Grock voice?
02:09Have you heard the new Grock voice? How can I help you today? Cool.
02:13Okay. So this is pretty straightforward. It's the system prompt here.
02:16Right? Guardrails, all that stuff. It's their basic appointment scheduler system prompt.
02:21And then the welcome message right here. Under advanced settings, this is where we get some different stuff. Auto detect language.
02:28Speech speed, I always found that if you speed this up a little bit, it's better. 1.2. Speak when silent.
02:35Keep the conversation after let's just go three seconds. Enter five to thirty.
02:41Okay. I guess it's three, or I guess it's five then. Google Calendar three, looks like we're connected.
02:46Let's give this a shot. Hi. Thanks for calling Brown Bear Moving.
02:49How can I help you today? Hi there. Yeah.
02:51I'm just wondering what your availability is like for, um, I'd like a last minute move this Saturday. Are you available?
03:01I'd be happy to check our availability for this Saturday. To get you the most accurate info, could you tell me where you're moving from and to and roughly how big the move is, like a studio, one bedroom, or larger? Any preferred time of day.
03:15Once I have those details, I can check our slots for you. Yeah. Absolutely.
03:18I'm moving from
03:20Vancouver Island, Victoria to Vancouver downtown, about a two bedroom apartment, like 1,200 square feet, and anytime, preferably morning so we can make the boats.
03:34One sec.
03:39I'll check our Google Calendar for this Saturday.
03:46We have some availability on Saturday, July 11. The morning looks open before 09:30AM, then we're booked until 11:30 and free again until 09:30PM.
03:55Does the start time around seven or 8AM work for you, or would you prefer something later in the day? If you pick a slot, I can go ahead and book it for you.
04:02No. No. 7AM works.
04:06Great. I'll book you in for a 7AM start on Saturday, July 11.
04:10Okay. I don't want them actually booking my calendar. Okay.
04:12This is great. Um, that's actually a really good start in terms of how easy that was.
04:18We'd obviously have to give it some details about, like, when you have a first available start, then, it's not like you stop at eleven if there's another appointment. We need to give it that information, which we can do where?
04:29It said knowledge bases. Do I have to have an account to add collections?
04:35File collections? Yeah. Okay.
04:37So you need an account to add file collections, so I'll do that. Payment successful.
04:42I don't really have anything for my friend's company here. Just gonna use ChatGPT. There you go.
04:47Boom. K. I just gave ChatGPT a blurb about can you create a knowledge doc for my friend's company?
04:52It's gonna dig into Brown Bear Moving and then create an actual file I can feed the Grok voice agent. Honestly, like, I'm I'm actually kind of impressed. I wanna do another one with my template that I had in my my community there because that's the one I gave people to, like, work with.
05:08Download Brown Bear Moving KB. Hecking yes. Alright.
05:12Thank you very much, ChatGPT. Alright. So let's put this in here.
05:17Choose files right there. Create collection.
05:21Alright. Sick. I wanna try a different voice, though.
05:23Lumen. Have you heard the new Grock voice? No.
05:27Thanks. Let me tell Have you heard the new Grock voice? Let me tell you a secret.
05:31I am the smartest and best AI. Great. Alright.
05:35My buddy is actually English, so this makes a lot of sense. Let's do it. Hi.
05:39Thanks for calling Brown Bear Moving. How can I help you today? Hi.
05:42Yeah. I know it's a little bit last minute, but I'm looking to schedule a move for this upcoming Saturday, potentially Sunday, if you have no availability.
05:52I'd be happy to help you get that booked.
05:56This Saturday is July 11 and Sunday the twelfth.
06:00Correct.
06:04Saturday the eleventh has some busy slots in the morning and evening, but plenty of availability in between. Sunday the twelfth is almost completely open with just a short busy period in the evening. To find the right slot for you, could you tell me where you're moving from and to and roughly how big the move is?
06:20Say, a studio, one bedroom, or larger. Also, do you have a preferred time of day? Yep.
06:25That's right. I'm moving from Victoria to
06:28Main Island. So it'd have to be early in the morning.
06:35Got it. Victoria to the Mainland, early morning start.
06:39No. Not the Mainland. Main Island.
06:44Oh, sorry about that. Main Island it is.
06:53Saturday the eleventh looks good for an early start. The first busy slot doesn't hit until 09:30AM, so anything before that should be clear.
07:00Sunday the twelfth is wide open in the morning too. What time were you thinking? Maybe a seven or 8AM start.
07:06I do like that I can get it on calendar.
07:09And that was actually, like, pretty good latency.
07:11The only thing is If you like, I can check specific times for you. Would you prefer to start at 7AM or 8AM?
07:17And does Saturday or Sunday work better for you? Let's start at seven, um, because the last boat off the island is 4PM.
07:28A July should work perfectly with that 4PM boat deadline.
07:38I'll go ahead and book that for you if it sounds good. Just let me know, and I'll get it on the calendar. Do you have any other details, like how long you think the move might take, or should I just reserve the morning slot?
07:48Okay. Honestly, that did okay. I liked how it, like, corrected itself, and the the the latency was not bad.
07:54I do wanna make sure it's got, like, the playbooks in terms of, like, what to do. Like, you can't just go and do a move, for example, sight unseen. So we really wanna push for, the first appointment would be a consultation or, like, a quoting sort of thing.
08:07Right? Like, let's see exactly what we're walking into. This is the friction a lot of people run into with voice agents.
08:12It's very easy to get it off the ground, but it's harder to build out something that's good. So you'd have to tailor your templates towards your actual business use case. I like what I'm seeing here though.
08:22Like the voice is is good. Latency is fine and it's calling the tools really quickly. It's also not dragging out.
08:29Like if I go and correct it with the the main island, not mainland, it It pick that up like really quickly as well. It it's actually seeming like it's doing a good job.
08:37Check my calendar, got my real availability. I'm gonna go in and actually use the template I gave folks in my community and just grab this guy because this has kind of more a playbook that we were after.
08:51So I'm gonna go and change this up. This template is for a wellness spa, like more traditional. Right?
08:58So let's go, hello. Thank you for calling into Grative Health.
09:06How may I help you today? Okay. So the difference being, with these templates, I like to do variables at the top so that I can actually go and use the same sort of template playbooks and then just change the company.
09:19So for service based businesses, like with the appointment being about sixty minutes, I keep this template and we just change the incoming. I'm gonna keep Leo. I like Leo.
09:29And let's just see how this sounds now with a little bit fatter of a system prompt. I'm just looking at this here. When I go edit, four tools enabled, search, get event list, availability, create and update.
09:41Create, update, and delete aren't clicked, so I need to actually save that. Good to know out of the box, these three, like, actual, uh, actions aren't enabled. So let's give this a shot again.
09:57Leo, how may I help you today? Let's do, like, my name is Leo, your automated assistant.
10:12How may I help you today? Alright. Let's give it a shot.
10:14See if it actually does the thing. I'd
10:19be happy to help you with that. No worries at all. Take your time.
10:23Sure. Sorry. It looks like I just you to started a conversation before you were ready.
10:29That's perfectly fine. I'm all set now. How can I assist you?
10:32Are you a new client looking to get started with us, or have you visited Integrative Health before? If you're looking to book, reschedule, or cancel an appointment, I can help with that right away. Uh, that sounds great.
10:42I've actually never been there before.
10:44Uh, I was looking to just come in and see what you guys do. Hopefully, you have availability this weekend.
10:55I'd be happy to help you get started. To begin, what service or concern are you looking for help with today? Once I know that, I can check our availability for this weekend and see what works for you.
11:05Yeah. Like I said, I've never been in. So probably just like a first time visit, whatever you guys do for that.
11:13I can help with that.
11:16Our typical starting point for new clients is a consultation where we can discuss your needs and what we offer. Have you used this type of service before? If you're open to it, I can check our availability for this weekend and help set something up.
11:29Yes, please.
11:35I'd be happy to help you book a consultation. Let's find a time that works for you. What day this weekend would you prefer to come in?
11:42Saturday or Sunday? Sunday. Please let me know.
11:44If you can. Great. Let me check our schedule for Sunday.
11:51Sometimes the latency is really, really good. Availability for Sunday. Sometimes it takes a minute, but that's, like, typical with most of these AI assistants and stuff.
11:59Otherwise I have availability on Sunday, July 10. Happy with this. Um, typically, it takes a little bit more iterations if you're on, like, Vabrio or whatever.
12:06Busy period in the evening around 09:30PM.
12:09A few options that would work are 10AM, 2PM, or 6PM. Would any of those work for you?
12:15Let's do Sunday at 10AM, please.
12:22Perfect. I'll hold that 10AM slot for you. To finalize this for you, could you tell me your full name?
12:27Once I have that, I'll confirm the details and lock in the appointment. Awesome. Yeah.
12:31My name is Eric Michaud. Thank you, Eric. I've got that.
12:38Just to confirm, I have you down for a consultation on Sunday, July 12 at 10AM.
12:43Is that all correct? Yep. That's exactly what we talked about.
12:48Excellent. Let me lock that in for you.
12:54Just a moment while I create the appointment. Your appointment is confirmed for Sunday, July 12 at 10AM. I'm just gonna check that out.
13:03Forward to seeing you then. If there's anything else I can assist you with before then, just let me know. Otherwise, have a wonderful day.
13:09Goodbye.
13:12Thank you.
13:16Does it hang up on its own? You're very welcome. Have a great day.
13:20Hey. And it hangs up on its own. Sometimes that was an issue with some of these other guys is they, uh, they don't like to hang up.
13:26Anyway, let's check it out on my actual calendar. Gonna have to look at next week.
13:33Saturday. Sunday, consultation, Eric Michaud. Hecking, yeah.
13:37Actually, put it on my real calendar. Love it. Yeah.
13:42All in all, like every voice agent of all time ever, there is some tweaking tweaking you're gonna have to do, like giving it more information obviously equals a better agent response. But otherwise, I mean, templates out of the box, like, really good.
13:57I'm actually more impressed with this than I am Vappy's new workflows and and that kind of thing. And I I still like Vappy.
14:05I'm not trashing Vappy. It's a great platform. But for the price and what you're getting and, like, how easy this is, I I think this is something to consider, man, if you're building out voice agents.
14:14Especially, like, if you can go in and, like, check out conversation logs and pull that in from your API, like, you can watch all of this stuff, that would be ideal. Yeah. You get the whole audio?
14:27Yep. That's exactly what we talked about. Yeah.
14:29You do. You get the entire transcript. So you can download transcripts and see what people are calling about.
14:34I'm hoping you can do that with the API. Um, let's find out. Discard draft, deployment, configuration.
14:42If I go publish, do I get a number? Changes published. Good.
14:46Oh, set up phone number to call your agent. Cool. Set up, uh, direct SIP.
14:52Bring this in. Okay. Okay.
14:53So this makes sense. Actually, um, maybe a bad example. I'm in Canada, and I can't use the free numbers.
14:58It's the same across the board whether it was, like, retail or VAPI. It was it was the same. I needed to bring in my own numbers from Twilio for this to work.
15:06I mean, that's not a big deal. I already have some numbers on Twilio. I can bring it in, but they do provide a free number that you can you can run on the Internet, which is great.
15:15I just can't show you that because I'm in the wrong spot. Also, though, man, this is actually pretty good. Like, it works as advertised.
15:20Your voice agent is up and running in less than two minutes. Yep. Check that box.
15:24And I'm actually a big fan of the voices. The voices sound good. The latency is okay.
15:29Right? It's definitely passable. And I use Grok for, like, transcriptions all the time.
15:34So it's nice to see them come back with, like, some usable voices on the other side. I might start using this inside of, like, my environment just to, like, you know, speak to because I like to talk to my Hermes agent from from Telegram. But I mean, yeah.
15:45Actually, I I do I think this is worth considering. The thing that Vapi gives you a little bit easier than this at the moment at least is SMS support. Like, Vapi ties that in, right, with the same number, and you can go back and forth.
15:56But, with these call tools, like, your add tool, you can do an API request to Twilio. Like, you can you can set that up. No sweat.
16:03It it's not hard. So I like that they made this easy and it's just the one window that you you have to look for. Anyway anyway, good things.
16:11If you guys want access to the voice agent resources I was using today, including how to set this up for yourself, how to sell these to clients and whatnot, you can click the top link in the description and join my school community. There are some paid products, but it is absolutely free to join. The biggest benefit is you're joining almost 3,000 other builders, other like minds out there building and selling these sorts of products for themselves every single day.
16:31So it's a great community to get in and learn and bounce ideas off of each other and whatnot. Anyway, thanks so much for hanging out with me. Was fun to check this stuff out with you.
16:38If you like this video, you'll love this one over here. We'll catch you in the next
The Hook
The bait, then the rug-pull.
An AI agency owner who sells voice agents for a living gets an email about xAI's brand-new Grok Voice Agent Builder and, with zero prep, builds two working phone agents live on screen to find out if it's worth switching to.
Named ideas worth stealing.
15:06concept
Variables-at-the-top templating
- business name
- hours
- appointment length
- incoming greeting
Keep one reusable system-prompt skeleton for a business category (e.g. service-based appointment businesses) and swap out only the variables at the top for each new client, rather than rewriting the prompt from scratch.
Steal forany agency reselling the same agent type to multiple clients
How they asked for the click.
VERBAL ASK
16:30newsletter
“click the top link in the description and join my school community... it is absolutely free to join”
Soft, single CTA placed at the very end after the comparison and honest take, not inserted mid-content.
MENTIONED ON CAMERA
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
OTHER LINKSAlso linked in the description.
Visual structure at a glance.
