The argument in one line.
ElevenLabs compresses the full AI receptionist build from system prompt to live phone line into a single platform, and the only real skill that determines quality is writing a well-structured system prompt.
Read if. Skip if.
READ IF YOU ARE…
- You want to build an AI receptionist or phone support agent for a client or your own business and have never used ElevenLabs before.
- You already know what voice AI can do but need a single guide covering prompt setup, voice tuning, knowledge base, and tool integrations end-to-end.
- You are running an AI automation agency and want to understand how ElevenLabs fits vs. alternatives like Vapi or Retell.
SKIP IF…
- You need speaker diarization, real-time transcription pipelines, or deeper API customization — this guide stays in the ElevenLabs UI.
- You are looking for a comparison of voice AI platforms; this focuses exclusively on ElevenLabs.
The full version, fast.
The ElevenLabs agents platform lets you build a functional AI voice receptionist in under an hour using its built-in prompt generator, voice library, and system tools. The key decisions are: how to structure your system prompt (use markdown, keep it lean, put rare info in the knowledge base instead), which voice stability and speed settings to tune per voice, which LLM to use (Gemini 2.5 Flash for speed), and whether to route external integrations through MCP servers or direct webhooks (webhooks win for anything that must not fail live on a call). Phone deployment runs through Twilio with a simple credential import.
Free for members
Chat with this breakdown — free.
Sign in and you get 23 free chat messages on us — ask for the hook, quote a framework, find the exact transcript moment, generate a markdown action plan. Bring your own key when you want unlimited.
Create a free account →Where the time goes.
00:00 – 01:15
01 · Intro + live demo call
States scope and plays a live ElevenLabs demo agent call to prove the quality upfront.
01:15 – 01:41
02 · Platform overview
Opens ElevenLabs dashboard, explains the ElevenCreative vs ElevenAgents split.
01:41 – 03:01
03 · Voice library and voice cloning
Tours the voice library (thousands of voices), mentions voice cloning capability.
03:01 – 04:01
04 · Creating a blank agent
Clicks through agent creation, names the agent demo-YouTube, notes chat and WhatsApp options.
04:01 – 05:08
05 · Agent settings and system prompt
Explains the agent tab, system prompt box, and right-side test panel.
05:08 – 07:11
06 · Procedures, Workflows, Branches tabs
Covers procedures (skip, alpha), workflows for multi-agent routing (rarely needed), branches for A/B testing first messages.
07:11 – 08:11
07 · Knowledge base setup
Explains uploading PDFs and docs; agent can surface info from documents at runtime.
08:11 – 09:17
08 · Tools tab overview
End call, transfer call, and custom tool calls explained at a high level.
09:17 – 11:00
09 · AI-generated system prompt
Uses ElevenLabs built-in prompt generator via voice input to build the accounting receptionist prompt in 10 seconds.
11:00 – 12:13
10 · Markdown formatting in prompts
Explains why # headings, ** bold, and bullets improve LLM comprehension of prompt structure.
12:13 – 13:46
11 · Choosing a conversational voice
Filters voice library to Conversational category, selects Emma, explains fit for phone systems.
13:46 – 14:52
12 · Voice settings: stability, speed, similarity
Explains each slider: stability (expressive vs. consistent), speed, similarity. Advises testing per-voice.
14:52 – 15:22
13 · Language and LLM selection
Language support (multilingual model), recommends Gemini 2.5 Flash for speed-first voice use case.
15:22 – 16:16
14 · Live test call with the built agent
Calls the agent live; demonstrates natural accounting receptionist conversation.
16:16 – 17:34
15 · Fine-tuning: response length and first message
Key advice: 1-2 sentence max responses, fight LLM verbosity in the prompt, customize first message.
17:34 – 18:34
16 · Knowledge base document upload
Walks through clicking Add Document; notes minimal configuration needed.
18:34 – 19:21
17 · System prompt vs knowledge base decision
Framework: critical/common info in prompt, rare/long-form in knowledge base. Bigger prompts cost more per minute.
19:21 – 20:57
18 · System tools: end call and transfer call
Enables End Conversation tool, Transfer to Number tool, configures descriptions and phone number.
20:57 – 22:03
19 · MCP server integrations
Shows MCP server setup with HubSpot example, notes current reliability limits for live call actions.
22:03 – 23:55
20 · Webhook and direct API tool calls
Demonstrates Add Webhook Tool; explains using n8n or make.com as middleware before hitting a CRM endpoint.
23:55 – 26:03
21 · Connecting a Twilio phone number + CTA
Phone Numbers tab, Import from Twilio flow, assign to agent. Closing subscribe CTA.
Lines worth screenshotting.
- The voice agent system prompt is the only thing that actually differentiates one agent from another — the platform UI is just a wrapper.
- LLMs are trained to produce long responses; you have to explicitly prompt a voice agent to keep answers to one or two sentences.
- Markdown formatting in system prompts improves LLM comprehension because most models were trained on markdown-heavy internet text.
- Put frequently needed info in the system prompt and rare reference material in the knowledge base — larger system prompts cost more per minute to run.
- MCP server integrations are convenient but not yet reliable enough for live call-critical actions like appointment booking; use direct webhook calls instead.
- Voice stability tuned low makes a voice more expressive but risks audio artifacts; test each specific voice before going to production.
- For voice agent LLM selection, speed beats reasoning depth — Gemini 2.5 Flash and OpenAI fast-tier models outperform slower frontier models in real-time conversation.
- The workflows tab (multi-agent routing canvas) is almost never needed in practice; modern LLMs handle large context windows well enough to avoid it.
- Connecting a real phone number requires a Twilio account — ElevenLabs imports the credentials and assigns the number directly to an agent.
- A knowledge base document upload requires almost no configuration — just drop the file and reference the knowledge base in the system prompt for relevant topics.
- Voice cloning is available natively in ElevenLabs — a client can clone their own voice for their phone system without a third-party tool.
- The same ElevenLabs agent can be deployed simultaneously as a voice agent, a website chat widget, and a WhatsApp channel without separate builds.
What actually determines an AI voice agent quality.
WHAT TO LEARN
The ElevenLabs platform handles the infrastructure — what separates a useful agent from a useless one is how well you write and structure its system prompt.
- Write system prompts using markdown headings, bold, and bullets — LLMs were trained on markdown and parse it better than plain prose.
- Limit voice agent responses to one or two sentences per turn; LLMs default to verbosity and you have to override that explicitly in the instructions.
- Keep the most frequently needed information in the system prompt where the LLM always sees it, and offload long reference documents to the knowledge base — larger prompts cost more per minute.
- For integrations that must not fail during a live call, use direct webhook tool calls rather than MCP servers, which are still too unreliable for synchronous actions.
- Choose a fast LLM such as Gemini 2.5 Flash for voice agents — response latency matters more than reasoning depth when someone is listening in real time.
- Voice stability settings behave differently per voice; lower stability sounds more human but risks audio artifacts — test each voice before going live.
- The ElevenLabs Workflows tab for multi-agent routing is an advanced feature for when prompt context windows genuinely overflow — most agents never need it.
Terms worth knowing.
- Conversational agent
- An AI system configured to hold real-time spoken or text conversations, as opposed to a one-shot completion or chatbot that waits for explicit user prompts.
- System prompt
- The master instruction block given to the LLM before any conversation starts, defining personality, scope, rules, and response style for the agent.
- Knowledge base
- A collection of uploaded documents the agent can search at runtime to answer questions not baked into the system prompt — useful for long PDFs and service catalogs.
- Tool call
- An instruction that causes the agent to call an external API or service mid-conversation, such as booking a calendar slot or creating a CRM lead.
- MCP (Model Context Protocol)
- A standard for AI agent-to-agent communication, letting the ElevenLabs agent hand off tasks to another AI service without raw API calls.
- Webhook tool
- A direct HTTP request from the agent to an external endpoint — the more reliable alternative to MCP for live call-critical actions like appointment confirmation.
- Voice stability
- A slider in ElevenLabs that controls how expressive vs. consistent a voice sounds; lower values are more emotional but risk audio artifacts.
- SIP trunk
- A standard VoIP protocol connection that lets a phone number route calls to a software service — ElevenLabs supports it as an alternative to Twilio for phone deployment.
- Branches
- An A/B testing feature in ElevenLabs that lets you run two versions of a first message to see which retains callers longer.
Things they pointed at.
20:57toolHubSpot MCP
Lines you could clip.
16:16
“I typically recommend getting the voice agent to not say too much, maybe one to two sentences max in its responses.”
18:34
“It is gonna cost you more money per minute to run an agent that does have a bigger system prompt, so you do need to be aware of that.”
09:30
“Ten seconds later, we have now got this entire prompt based on the request that I just gave it.”
Word for word.
Read-along
Don't just watch it. Burn it in.
See every word as it's spoken — crank it to 2× and still catch all of it. The same dual-channel trick behind Amazon's Kindle + Audible.
analogy
00:00This is a complete tutorial on how to build AI voice agents using the Eleven Labs conversational agents platform. If you're not familiar with Eleven Labs, if you've listened to any AI generated voices or built anything using AI voices before, chances are that Eleven Labs was powering that. And so because Eleven Labs is the best platform when it comes to these AI generated voices when building out AI reception systems or phone systems or website voice agents, we're gonna be getting the best quality voice out of this Eleven Labs platform.
00:26So by the end of this video, you're gonna be a complete expert as to building AI voice agents specifically through the Eleven Labs agents development platform. And just to give you a sneak peek as to exactly what one of these agents does sound like, I'm gonna go ahead and just call this demo agent on their website right now so you can take a look.
00:41Hi. I'm Eleven. Welcome to Eleven Labs agents.
00:44How can I help you? Hey. I was wondering if I'd be able build an AI receptionist through your platform.
00:50Yes. You absolutely can. Eleven Labs Agents is perfect for building an AI receptionist.
00:55You can have it handle tasks like answering frequently asked questions What about taking messages? Booking. Schedule yes.
01:03Appointment booking is definitely a popular use case. You can configure an agent to check availability, book appointments, and even send confirmations to your customers.
01:12Would you like to explore how this could work for your specific use case? Okay. So you get the point.
01:16We can have these natural conversations through just our own voice where we speak to the agent, speaks back to us, and we can have a natural conversation. This can happen through a website just like this, but we can also attach this to an actual phone number, which I'll go through in a minute as well. And then we can build an actual print system AI where we can build out AI receptionist systems and customer support so that we can tap into a real phone line where real calls are coming through and fully automate them using AI.
01:40Anyways, diving right into it. This right here is the Eleven Labs platform. If you're not yet signed up for Eleven Labs, I've got an affiliate link in the description if you wanna help support the channel.
01:47But otherwise, once you sign up for an account, you are gonna land on this page right here. So once again, the big advantage to Eleven Labs is that they are the biggest provider when it comes to AI generated voices. Essentially, Eleven Labs has sort of pioneered AI generated voices well before ChatGPT and well before voice agents, of course.
02:03And so their key platform is this sort of voice generation part of it. So And if I a look at the top left, you'll see if I click into eleven creative, this is the specific sort of panel for us to be able to create voices and choose voices that we like to use. So if I jump into their voice library, there are, I think, thousands or tens of thousands of voices here across all different accents and everything that we're gonna be able to access and we can use in our actual voice agent systems.
02:28Alongside this, we're gonna be also able to clone our own voice using Eleven Labs tech as well, which is one of the best when it comes to voice cloning. So if did wanna clone your voice or one of your clients wants to clone their voice for their own phone system, that is something that we can 100% do through this platform as well.
02:43If I go ahead and click in the top left and click on eleven creative, you'll see 11 agents, and this is where we can deploy and monitor agents. So this is the main platform, the main workspace that we're gonna be using in order to build out the voice agents.
02:56So once you're on this page, all we need to do is just hit the plus button in the top left right here in the agents section. I click into there. It's gonna take us into this panel where it's gonna prompt us if we wanna use some of their own templates or we can just start to use our own.
03:08You can browse the templates if you'd like to have a look at some of the other agents that are out there as an example to look at. But otherwise, I'm just gonna go ahead and click on blank agent as we're just gonna get started from the start. Let's do a random name.
03:19In this case, I'm gonna type demo YouTube. And you'll see under this, it also says chat only. So Eleven Labs, whilst it is really great for voice agents, they've also tailored this platform if you wanna build chat agents as well.
03:29If you do want a website support agent, the sort of bubble that sits in the bottom right corner of the website for somebody to be able to chat to and talk to, that's something we can build through. But we can also build WhatsApp agents as well. I'm not gonna dive too deep into that.
03:41We're focus mostly on the voice agent and phone system side of this. But just to let you know, we can actually connect these chat agents. The same agent that is the voice agent, we can connect that as sort of like a multichannel into WhatsApp as well.
03:51Anyways, I'm just gonna hit create agent. I'm not gonna touch this. And once I've done that, we're gonna land on this page right here.
03:56And this is gonna be all of the settings necessary in order to develop and build the voice agent. Most importantly, on the right hand side, this is where all of our testing is going to happen. So actually talking to the voice agent, sending messages, testing the voice agent to see that it does what it should.
04:09That is for the most part gonna happen all on this right hand side. And a lot of our deployment options as well are also gonna be on the right hand side. If we click into widget, this is gonna be the sort of website widget that we can actually add onto our website if we wanted to from the start a call button to talk with it through our voice or the chat icon if we wanted to do sending messages as well.
04:26Anyways, the most important part is gonna be on the left hand side of the top here in the agent settings. If I click on agent, this is really gonna be where the majority of the work in developing your voice agent is going to be done in. So on this page, get the settings of the system prompt.
04:38This system prompt is gonna be the most important part of actually developing your voice agent. This is where we're gonna be giving it all the context about who it is, if it's a receptionist, you know, which company it works for, what it needs to talk about, how to respond to questions, all of that gets done through our system prompt.
04:52And then beside this, we've got the voices and this is once again where we're gonna be able to access those thousands and thousands of voices, uh, specifically from obviously Elevenlabs voice library that we can choose from. I'll come back and dive deeper into these settings so you do know what to be aware of, but I'll just jump into some of these other panels on the left hand side.
05:07Just below the agents tab, we have these procedures right. I personally haven't played around with these procedures. I believe it is a new feature that's currently in alpha.
05:14So I haven't played around with it, so I'm not too familiar with exactly what this is for. So to be honest, I probably would wait for this feature to come out of alpha before you start playing around with it, but feel free to jump into the docs and learn about it if you'd like. But underneath the procedures tab, we have the workflows tab.
05:28Now this is something that you definitely do not have to use. What this workflows tab is is if we wanna make this agent a little bit more rigid and less reliant on the system prompts that we're using in the agent tab. So ultimately, the system prompt is just one big set of instructions to tell the voice agent how to act, how to respond to any kind of questions that come its way, and it is possible that these AI models hallucinate and they don't fully read that system prompt.
05:49And so the solution to that is to use a workflow tool like this where we're gonna be able to use of cards on this canvas right here where we can get it to read out very rigid responses to certain questions. So if I just click on the templates button and click into one of these templates, you'll see now we have this canvas and we have these multiple nodes on this canvas between different agents.
06:07So this workflows feature is really only for very big, very advanced agents where the amount of context in your prompt is just too much for that model to handle that it starts to hallucinate and forget certain things. And so this workflows tab is specifically built to essentially create multiple different agents. And then when a certain request comes through that might be for, you know, a technical question or a billing question or something very specific, we can then route that to a brand new agent with a whole new system prompts to specifically handle that problem.
06:35Now to be honest, we've built a lot of these voice agents for a lot of different companies in a lot of different industries, and we very rarely use a system like this. We've pretty much found that over time, these large language models have been getting better and better with bigger context windows where we've been able to shove more and more information into the prompt itself.
06:51And so it really has been quite rare for us to run into sort of an an issue where we have lost so much context because the prompt is so big that we've had to move to a system like that. I'm not saying that you won't run into that issue. There have been a couple of clients where we have had to move over to a system like that workflows tab, but I certainly wouldn't worry about it as a beginner if you're just getting started.
07:09It's not something to worry about right now. Beneath the workflows tab, we've got branches. So if you wanted to run an a b test on the first message of the voice agent against a slightly different version of that first message to see which converts better or which retains the most amount of people in the conversation.
07:22That is something that you can create a branch for and then just track that data through. Once again, this is not something that we would do too too often. If you're a complete beginner, I wouldn't worry too much about this.
07:31Underneath branches, we've got the knowledge base. This is something if you are a beginner, once again, is gonna be incredibly important. The knowledge base is essentially where we're gonna be uploading any documents or files related to context about their business or their services.
07:43So if we're gonna be building out a voice agent for an accounting firm, we would upload all of the information about their business, all the information about their services, and the pricing of their services. We would upload those PDF or Word documents into the knowledge base, and our voice agent is gonna be able to call upon that knowledge base when it needs to to find specific information out of our documents.
08:01I'll talk a little bit about when you should use knowledge base documents versus putting that information directly into the system prompt of the agent once I jump back to the actual agent development in just a few minutes. We've then got the tools tab. This is gonna be one of the most important parts of developing a voice agent.
08:15Essentially, a tool is the ability to communicate with any sort of external apps or data. So if you wanted to book an appointment, that would be a tool call off to Google Calendar. If you wanted to send any information into a CRM system, that would be a tool call to that specific CRM system or to some kind of a middleman automation platform.
08:32Eleventh Labs also has a lot of system tools, which you can see on the right hand side where we want to end the conversation or transfer the call to somebody else as well. So simply by enabling these tools, we can now prompt our agent to, you know, end the conversation at certain points.
08:45So we can tell our agent to, you know, intelligently wrap up the conversation when it thinks that the conversation is over. And then, obviously, with transferring the call, we can tell it that if somebody wants to speak to, you know, a manager, we could enable this function to specifically transfer to that manager upon that request.
09:00The majority of the other tabs are not critically important to getting a voice agent set up. Most of this is related to once we start receiving calls, we can track that through this call dashboard right here and actually have a look at what calls have been happening. We can evaluate whether or not they've been good or bad and make updates based on the calls that have come through.
09:17Alright. So jumping back into the agent tab. This is once again where the majority of the development is going to happen and more specifically in the system prompt box right here.
09:25Now back in the day, I used to do a lot of prompt work manually where I would jump into this system prompt and I would pretty much write the entire prompt myself without using AI at all. The other reason I ever did that was because the models honestly weren't good enough to write these prompts. But to be honest, in the last twelve months, I would say these models have gotten so incredibly good, especially at writing prompts that no doubt about it, the best way to actually get these prompts made is by just getting an AI to make it for you and then just getting the AI to make updates and changes based on your feedback.
09:53And luckily on Eleven Labs, what we can do is click this button right here, and so this is a built in tool to get Eleven Labs to generate that prompt for us. And so I can very simply tell it if I would like to build an accounting receptionist system. I'm just gonna tell it literally to build that.
10:06I can give it a name for the company, provide it with some details, and it should go ahead and just build the entire system prompt for us. Hey. I was looking for a voice agent receptionist system an accounting services company called Red Accounting Services.
10:18If you could build out that prompt for me, that would be great. You can make up some information about the actual business and the pricing and the services that they offer. You can just make that up.
10:26But for the most part, it's gonna be an accounting services company, this and is gonna be an AI receptionist system. Okay. I just used my speech to text tool in order to speak that out, but now I can send this off to the agent, and we should get back a nice prompt in which we can use.
10:39Alright. So literally ten seconds later, we have now got this entire prompt based on the request that I just gave it in order to generate, and we can obviously scroll through here and have a look at the specifics as to what it generated and how this agent is going to work and what it's gonna talk about. But ultimately, I can now literally jump into the test on the right hand side and start speaking to this agent already.
10:59Now just so you're aware as to some of the prompting techniques going on here, think the key thing to know, you can see right here on the top right, this hashtag is essentially markdown formatting. So the reason that this AI has used markdown formatting, when we manually write prompts, we use markdown formatting.
11:13Ultimately, this is just the best way to format a prompt for an AI to fully understand exactly what's going on. Pretty much majority of these AI models have been trained on markdown formatting themselves.
11:23The majority of the Internet is built on markdown formatting, and so it's able to read and interpret Markdown essentially very well. And what we're looking at with one hashtag and then the space and then the actual title word up here is essentially turning this into what's called a heading one. If you've ever used the Word document and you've got the really big heading, essentially, the AI is viewing this as a really big heading.
11:44So it knows that this is the personality section, and then it knows that this is the environment section and the tone section, and so it's all a bit neater and laid out for it. And as I scroll down a bit in the prompt as well, you'll see some of these asterisks which is making some of this content bold, and you'll see some of the dot points as well.
11:58So these are all markdown elements that's just making it a little bit easier for the AI to fully understand what's going on the prompt. So if you are gonna be making any manual edits, I would keep in mind the markdown formatting structure as being a really good approach to get your AI to fully understand what you're talking about as much as possible.
12:13Anyways, I'll jump into voices on the right hand side. If we click into this, it is gonna open up to the massive library of Loveland Labs voices, the thousands and thousands of voices. Realistically, you just wanna choose a voice that's good in the conversation aspect.
12:25So one thing I will note is that a lot of these voices are not necessarily the best voice for sort of a phone conversation or just a conversation in general, which aren't the most appropriate for, you know, let's say, our phone system. And so I would recommend just clicking into category and then just making sure you've got conversational selected so that you're actually using a voice that would be a bit more tailored for, an actual conversation that you're having, just a natural speech, um, or like a phone system.
12:49So I'll just go ahead and select a voice that I've used in the past. This voice right here, Emma, is a voice I've used in the past and it's pretty good for conversational. I'll just go ahead and click on that and now that voice is selected and we are now using that voice in our voice.
13:00Now when it comes to voice settings, there is quite a bit that we can change and it really does depend on the specific voice that you've chosen as to what you should change in these settings. The first setting is stability and this is gonna be making it more expressive or more consistent of a voice. If we move this slider down and make it more expressive, essentially, our voice is gonna be more emotional.
13:18It's gonna be a bit higher in some areas, a bit lower in some areas. And so that can definitely help to make it sound a bit more realistic, a bit more like an actual human. But in some cases, there can be issues with artifacts or screeching in the voice.
13:30You really do have to look out for that. And it really does differ on the voice that you've chosen, so you really do have to test this. And if we slide this all the way to the top and make it more consistent, it's just gonna be a bit more monotone.
13:40It's less risky, less chance of having those artifacts come up, but obviously, it might sound a little bit more robotic. Then we've got the speed, pretty self explanatory. We can slow it down or speed it up.
13:48And then we've got the similarity. So this is how consistent it is in its speech. It is gonna try its best to have that voice sound as consistent and as similar as possible throughout the conversation.
13:58It might sound a little bit more robotic, but if we were to drag this all the way to a low similarity, that random word, it might try and make up specifically for that voice. So you really do just have to play around with it and just see how it sounds. Jumping out of those voice settings on this tab right here, we'd actually click into the settings button on voice, which gives us some more voice settings as well.
14:15To be honest, I wouldn't worry too much about this, especially if you're just getting started. This is sort of more fine tuning later on, and we very rarely touch this even with production voice agents. So I really wouldn't worry too much about these settings.
14:26We've then got our language setting. Obviously, that's just gonna be the language in which this agent is speaking in. Eleven Labs has their own multilingual models.
14:33So if you are gonna be doing outside of English, it is capable of doing that. You will just wanna swap your model over to the multilingual agent. You can even add additional languages if you want it to be able to switch to a different language mid conversation.
14:43Underneath this, we've got the large language model. This is gonna be obviously a really important part of our voice agent. This is simply the large language model that is gonna be used to generate the responses based on the user's questions.
14:54One The that eleven Labs recommends using is Gemini's two and a half flash model. It's perfectly great model for this kind of use case, especially being a flash model. When it comes to voice agents, we do really want a faster model as we really don't need the most advanced reasoning model.
15:06We just need a model that's gonna be very quick and good enough. And this model for my testing is perfectly good enough, as well as some of the OpenAI models in their fast tier are perfectly good as well. Anyways, at this stage, our voice agent is good to go and has been good to go essentially once this prompt was created.
15:21So I'm gonna go ahead and give this agent a call, and let's see how it sounds. Hello. How can I help you today?
15:27Hey. I was wanting to see if I could get a quote on some bookkeeping services.
15:31Certainly. I can help with that. My name is Alex, and I'm the AI receptionist for Red Accounting Services.
15:37To give you an accurate quote, I'll need a bit more information about your specific bookkeeping needs. Could you please tell me your name and a little about your business?
15:47Yeah. My name is Brendan, and I run Inflate AI. We build AI receptionist services for other companies.
15:54Thank you, Brendan. It's a pleasure to speak with you. Inflate AI sounds like a very interesting company.
16:00Regarding our bookkeeping services, we offer monthly and quarterly packages. To provide you with a tailored quote, could you tell me a bit more about the size of your business, such as the number of transactions you typically have per month and if you require payroll services as well?
16:16Okay. So there you go. You can see it working in action.
16:19We're gonna be able to give a question. It's gonna be able to answer to us. It's using that voice that we've chosen, and it's responded back as well.
16:24And I can keep having that back and forth conversation. And so this is literally ready to deploy in an accounting services firm as a receptionist system for people to come to. Now I would say that there is a bit of work required on top of this AI generated prompt.
16:37Whilst we were able to generate quite a bit very quickly, there's still a lot of work that's required in terms of fine tuning. The conversation that I had wasn't terrible, but it definitely could be better to make it a bit more conversational. I typically recommend getting the voice agent to not say too much, maybe one to two sentences max in its responses.
16:53Typically in a conversation, you're not gonna be reading out an entire paragraph or giving somebody multiple questions at a time. You're typically gonna be reading out about one or two questions at the absolute maximum, waiting for a response, and then continuing that conversation. These AI models are typically trained to spit out as much as possible.
17:10That's how they're mostly trained. So you really do have to fight against that and get the model to just read out, you know, one or two sentences to make it a bit more conversational. We can also modify the first message as well that was read out at the start here, which said, hello.
17:22How can I help you today? Pretty standard for receptionist, but we can modify that by updating this all the way at the bottom here. So you can make that as to whatever you'd want it to be.
17:29Before the most part, the instruction set part of this voice agent is done and we're good to go. Now the next thing that you would do of course when building up this system is the tool calls and knowledge base. So as I mentioned earlier, what we can do is jump into the knowledge base section on the left hand side.
17:43It's pretty easy to jump into. We can then click add document and literally we can just upload if we have any service documents, any PDFs, Word docs. We can upload all of that into here and it will automatically give our agent the complete access to all the information in all those documents.
17:56So if we ever have a question about anything that's in those documents, this voice agent specifically is gonna be able to answer that. Now when it comes to the knowledge base, there's not a whole lot of work that we have to do. We pretty much just upload the documents right into here.
18:07We don't really have to touch the agent much more than just adding in those documents. It's gonna be able to surface anything we need. One thing that you can do to make it a little bit better is reference that knowledge base in the prompt itself.
18:17So if somebody has questions about pricing or about their services, you could potentially prompt in the system prompts right here to, you know, reference the knowledge base on, you know, more service information or more pricing information. It's gonna be able to then, you know, specifically target the knowledge base instead of the information in your prompt.
18:33Now one big thing to note is that we can put that service information in our system prompt as well as the knowledge base, of course. And I do get this question a lot as to should we put all our information in the system prompt, or should we separate that and pull it in the knowledge base? And the answer to that is we really do wanna put as much information in the system prompt as possible.
18:50The more information that we have in the system prompt means that our large language model is gonna have more context more of the time. But you really do need to find the balance of the information that is absolutely necessary versus the information that isn't necessary because it is gonna cost you more money per minute to run an agent that does have a bigger system prompt, so you do need to be aware of that.
19:07But really for any information that doesn't have to be read out a lot of the time, I would just add that as a knowledge based document. So if you've got a big PDF, just throw it in the knowledge base and get out all of the critical, most common information and put that into your system prompt. And then lastly, we've obviously got our tool calls.
19:21This is a really important part of this as well, of course. If we want our agent to have the ability to end the conversation, we'll just toggle this on right now. And now our agent has the active tool to end the conversation itself.
19:31And if we wanna prompt this tool with some specific set of instructions, we can click into the settings button. It's gonna give us a description. And in this, we can tell specifically that when it seems like the user is wrapping up the conversation, that's when we can end the call.
19:43Or once an appointment is booked, then we can end the call or something very specific, we can type that prompt and instruction set right here. The same goes for every single one of these tools if you wanna have a read through and see what they all do. But the ones that we use the most is gonna be the ending of the conversation and then transferring to a separate number.
19:58So if I go ahead and turn on transfer to number and toggle this on, it's gonna open this up right here. So in this configuration tab, we could specifically do a transfer to, you know, our manager. If our manager's name is John, we can do transfer to John.
20:10We can type in a description that if somebody wants to speak to the manager or if somebody wants to speak to John, it is specifically gonna do this transfer. And then beneath this in the add rule box, this is where we're specifically gonna be putting in their phone number. It's pretty simple to get set up.
20:23You just paste their phone number in the phone number box. Once again, you put the prompt instruction set for when to do that transfer. Once you've got the phone number in this box, it's pretty much set up and ready to go.
20:31And then once again, we can still jump back to our agent tab. We can jump into our system prompt, we can specifically say that if somebody wants to speak to John, run the transfer, you know, function to speak to John. So I would recommend putting it into the main system prompt and also adding that logic or that prompt into the function call as well.
20:48So that's it for setting up these system tools from Eleven Labs themselves, but if you do wanna do custom tools for booking appointments or sending information to other CRM systems, there are two different ways that we can do this. So we can click in the top left right here and click MC. An MCP server is the model context protocol.
21:04Ultimately, this is like an AI agent to another AI agent sort of communication line. We're gonna be able to plug our Eleven Labs agent into, let's say, HubSpot's MCP agent, and then our voice agent is gonna be able to talk to the HubSpot AI agent.
21:18And if it needs to send a new lead into HubSpot, it will simply tell HubSpot, we've got a new lead. Here are some of the details. And then HubSpot's agent's gonna look at that detail, and it's gonna be able to put it into their own CRM system without any sort of technical stuff happening.
21:33And so I can click add server. Once I do new custom MCP server, just go through this. It's just gonna provide me with some setup instructions in order to add my own server.
21:42So we can see here, it's now telling us to do a server configuration. I'm not gonna go through this now as it does take a little bit to set up, but ultimately, you would then go to HubSpot's MCP setup instructions. That would give you everything necessary in order to fill out all of these fields right here, and then you can be fully connected into your own HubSpot account through that MCP server.
21:59So that is one of the ways that we can quite easily integrate into other external applications. It is a bit of a battle as to when to use an MCP server versus a direct tool call. MCP servers are great, but they're not yet perfect.
22:11So if you're gonna be doing any appointment booking systems, I probably wouldn't yet use an MCP server. I would probably just use an MCP server for talking to maybe a CRM system or really anything that doesn't have to happen live right on the call right away because if it does fail, you could cause a bit of an issue. Anyways, I've got a lot of other content on my channel about MCP servers.
22:29So if you wanna learn more about them, you can check them out. But the more rigid and I would say secure way to talk to other applications is by doing a direct tool call, which we can do by clicking on the tools tab, clicking add a tool, and if we do this, we can do add a webhook tool, and this is gonna be your standard API request or your standard HTTP request where we're gonna be doing an API call to a certain endpoint.
22:50This could be a post request to send information to a CRM system. It could be a get request to get information from a CRM system. And so these are all of the settings necessary in order to get this set up.
23:00Once again, if you're not familiar with API setups or webhooks, I would recommend going ahead and watching some other videos, um, just to understand the basics around it as I'm not gonna go through that right now. But ultimately, you can essentially do a direct a call. Let's say the HubSpot once again, there might be an API endpoint for uploading leads into HubSpot.
23:18We can hit that endpoint with the data of the lead directly through this, or what we can do is use an intermediary like nnn or make.com to process and do a middleman automation on that data if we need to change anything before it hits HubSpot, for example. Once again, got a lot of content on my channel around building these automations that you can check out to learn a bit more about.
23:37But once you have added that tool in, it's just gonna pop up on this list right here where the tools are. You can then add that prompt in once again for when that tool should be triggered. Booking appointments, for example, could be a tool you might create.
23:47We can then come to our system prompt and tell it to specifically target that book appointment tool to call it at a specific point in the conversation. Okay. So hopefully, you now understand all the basics around building a voice agent here specifically on a Level Labs.
23:59The last thing that we need to do is now add this to a specific phone number. And we can very easily do this. We go to left hand side all the way at the bottom, click back to workspace, back out into this area.
24:10If we have a look on the left hand side, I think myself is covering it. I'll just move myself and you will see that there is a phone numbers tab. And once we're inside of this phone numbers tab, we're gonna be able to click import number.
24:21And then you'll see a couple different options to import a number from Twilio, a SIP trunk, or XOTEL. I would recommend just clicking on from Twilio. And this is ultimately where we're gonna be able to purchase our phone numbers directly from.
24:33So if we're gonna be building out our own dedicated phone line for a receptionist, for example, we can come to Twilio and we can purchase a phone number directly through them. Once again, I've got other videos out there on how to set up a number on Twilio so I would recommend watching them. I'm not gonna go through that entire process right now, but we ultimately just have to sign up for an account.
24:50Once we've done that, we can then purchase a a US or an Australian or whatever phone number through Twilio. We can then take our sort of login credentials or connection credentials back to our Level Labs, uh, with our Twilio account, um, authentication. We can connect our account, add the phone number, then it'll be connected, and then we can assign that number directly to our agent as well.
25:09If that does sound a little bit confusing to you, I would recommend joining my completely free school community which I've got linked down below in the description. We've got over 25,000 people building voice agents and automations.
25:19No doubt about it. Somebody in there could probably help you out. Alright.
25:22So at this point, we can get our phone number connected. We can then assign that to the agent specifically, and then pretty much we're gonna be able to literally over the phone call this agent that we can talk to right now, um, as exactly as we can on the web.
25:33And ultimately, that is it. You're now a complete expert when it comes to eleven Labs voice agent development platform. You should now be able to jump into this and build out these voice agents all by yourself and have them running on an actual phone number.
25:45Now just for my last note, I've recently discovered that 90% of you watching right now are not to the channel at all. So if you could check if you are subscribed, make sure to subscribe to the channel. That would be amazing.
25:55I've got a lot more voice AI content just like this coming out, so I don't want you to miss it. And if you are subscribed, you'll be the first person to see those videos.
The Hook
The bait, then the rug-pull.
The video opens with a promise and immediately cashes it — within 75 seconds the viewer hears a live AI receptionist handle a real question about booking an appointment. No theory first; the product speaks for itself before a single slide appears.
Named ideas worth stealing.
18:34concept
System prompt vs. knowledge base decision rule
Put frequently accessed, critical information in the system prompt (always in context). Put rare or long-form reference documents in the knowledge base (retrieved on demand). Bigger prompts cost more per minute.
Steal forany AI agent build where cost-per-conversation matters
22:03concept
MCP vs. webhook routing rule
Use MCP servers for CRM-style integrations where failure mid-call is tolerable. Use direct webhook tool calls for anything that must not fail live. MCP is convenient but not yet production-reliable for synchronous call actions.
Steal forAI agency clients evaluating integration reliability
16:16concept
1-2 sentence response rule
Voice agents should respond in 1-2 sentences maximum per turn. LLMs default to long outputs; override this explicitly in the system prompt to force conversational pacing.
Steal forany voice or chat agent system prompt
How they asked for the click.
VERBAL ASK
25:12subscribe
“90% of you watching right now are not subscribed to the channel at all. So if you could check if you are subscribed, make sure to subscribe.”
Mirrors the in-video stat call-out used earlier — same hook from description is delivered verbally at the end. Repetition across description + close is intentional.
MENTIONED ON CAMERA
FROM THE DESCRIPTION
PRIMARY CTAWhere the creator wants you to go next.
AFFILIATECommission earned if you click.
OTHER LINKSAlso linked in the description.
Visual structure at a glance.
