Welcome to the Deep Dive. We're here to pull
out the key bits of knowledge so you can get
up to speed fast. Today we're looking at something,
well, really transformative, I think. You know
that AI agent you used, the text -based one?
Super smart, super helpful. Well, what if he
can actually talk back to you? Yeah, not like,
you know, an old GPS voice. Exactly. Not clunky
or robotic, but like, with a natural, engaging
voice. A real personality. Something that sounds,
well, human. And the thing is, It's something
you can genuinely build today, like right now,
and without writing code. That's the amazing
part, the accessibility. Right. So our mission
for this deep dive is... So, OK, let's kick things
off. Method one. We're calling it the AI voicemail
system expert. This sounds well, it sounds clever,
like a smart way to handle tasks that aren't
instant. Right. This asynchronous idea. Exactly.
Yeah. Think of it like a super intelligent. Voice
mailbox. You, the user, leave a voice message,
maybe a complex question or instruction. And
the AI doesn't rush. It takes its time. It listens.
It thinks, processes everything, and then responds
with its own audio message back to you. Oh, OK.
So this asynchronous thing, it's perfect for
jobs that need, you know, deeper thought, summarizing
a really long document maybe or doing some complex
research. Right. Not that instant back and forth,
but more considered. Precisely. It's about getting
a thoughtful, rich response when you don't need
it that very second. That makes a lot of sense.
Like having a research assistant, you can brief
and they come back later with the findings. So,
OK, what's the actual journey from me hitting
record to the AI talking back? Walk us through
this voice message pipeline. Yeah, it's quite
an elegant flow, actually, like a little assembly
line. It starts when you send a voice message.
Let's say you send it to a telegram bot. Then
N8N, acting as the orchestrator, grabs that audio
file. downloads it, that raw audio, it goes straight
to 11 Labs, their models transcribe it, turn
it into text, and they do it brilliantly. That
text then gets fed to the AI brain. Could be
ChatGPT, could be Claude, whatever you're using.
The AI does its thing, generates the response
in text, and that text goes back to 11 Labs,
this time for text -to -speech, turning it into
that natural -sounding audio. And then finally,
N8N takes that new audio file and sends it back
to you in Telegram. It's a full circle. Wow.
Okay. Okay, that sounds incredibly capable. And
the bit that really grabs me is the no -code
part you mentioned. Like, for someone wanting
to build this, what are the actual pieces, the
Lego bricks in N8N? How do they, you know, snap
together? Right, yeah, Lego bricks is a great
way to put it. They really do feel like they're
made for each other. You're listening to Ghost.
In any in -the -ghosted area, like the Telegram
turn -out, you set it up to catch incoming audio
files, specifically the voice message format.
And crucially, it grabs the chatted... The return
address. Exactly. It's the unique return address,
so the AI knows where to send its response back.
Next trick. The universal translator. That's
an 11 labs node doing the speech -to -text...
It takes the audio file and just grabs and transcribes
it. The beauty is NAN handles passing that audio
data automatically. It's pretty seamless. Okay,
no complex mapping needed there. Nope. And it
totally changes the whole feel of the interaction.
So the system prompt is really the soul of the
agent. OK, then what? How does it get its voice
back? Right. So after the AI thinks and writes
its text response, you use another 11 Labs node.
This one's for text to speech. You connect the
AI's text output to this node. And this is where
you like. Cast your agent. You choose a voice
from Eleven Labs library. They have tons. Or
you can use a specific voice ID if you've cloned
one or have a favorite. Got it. Choosing the
voice actor. Pretty much. And the final step,
delivering the message. That's another telegram
node, a sender this time. You make sure it sends
the audio file generated by Eleven Labs back
to the original chat aid so the message goes
right back to the person who sent the request.
And then the test flight. The moment of truth,
as you called it. I bet it's satisfying seeing
it all light up. Oh, it really is. Watching each
node activate on the inning and canvas as your
message flows through. Very cool. But, you know,
things don't always work first time. What about
when, Houston, we have a problem? Any quick debugging
tips? Yeah, absolutely. The bugging and editing
is usually pretty straightforward. The visual
flow helps a lot. If a node doesn't light up
green, you know exactly where it's stopped. First
thing, check the trigger. Does the Telegram node
even receive the message? If not, maybe check
your WhatsApp and Telegram as well. Then follow
the noodles, those lines connecting the nodes.
Is everything connected? Did you link an output
to the right input? Easy mistake to make. But
honestly, the most common failure point nine
times out of ten is probably credentials. Ah,
yes, the classic. Triple check them. Your 11
labs key, your AI service key, one typo and the
whole thing falls over. And maybe most importantly,
read the error message. That's really solid advice.
OK, so this AI voicemail bot is fantastic for
those deeper asynchronous things. But what if
you want that instant back and forth, the conversational
feel? This is where it gets really interesting,
I think. We're talking about building a fluid,
real -time conversational AI, like talking to
a human assistant on the phone. This sounds like
the main event, the showstopper. Yeah, this is
where it feels truly interactive. So how do we
achieve that, the proper AI assistant experience?
Okay, so for this real -time system. you need
a totally different setup, a different kind of
partnership. I like to think of it using the
NASA mission control analogy. Ooh, okay. Tell
me more. Right. In this model, 11 Labs is mission
control. It's the sophisticated front end. It
handles the direct chat with the user, the voice,
the personality, managing the flow of the conversation.
It's talking to the astronaut, basically. Got
it. The voice on the comms. Exactly. Now, your
NA workflow, that's the specialist team back
in Houston. The engineers, the scientists, they
don't talk directly to the user. Instead, they
are a powerful tool that Mission Control 11 Labs
calls upon when the user asks something complex.
thing needing research or a specific action performed.
Ah, I see. So Eleven Labs handles the chat and
ANN does the heavy lifting in the background
when needed. Precisely. That's the key takeaway.
Eleven Labs manages the conversation and ANN
handles the actions or the deep information retrieval.
That analogy makes it crystal clear. So if 11
Labs is mission control, how do we actually build
it? How do we set up our agent in 11 Labs for
these live calls? You actually do it right inside
your 11 Labs account. They have a section called
Conversational AI. You go there, create a new
agent, and this is where you play casting director
again. Give it a name, something fitting. Choose
a voice from their library that matches the personality
you want. calm and professional, energetic and
friendly. And then you craft that initial greeting,
something simple to start, like, hello, how can
I help you today? Okay, straightforward enough.
But how do we connect mission control to the
specialist team, to NAN? That feels like the
critical link. It absolutely is. So back in your
11 Labs agent settings, you scroll down to tools.
Here, you add a custom webhook tool. This is
literally the direct phone line to your NAN workflow.
Okay. You need to give the AI a clear briefing
about this tool. So you give it a name, like
NAN Web Researcher, and a description, something
really clear like, call this tool to search the
web and find information about any topic. It
is an expert research assistant. So the AI knows
what it's for. Exactly. You set the method to
post and then you paste in the URL from an NAN
webhook node that connects them. Got it. So the
agent knows what the specialist team does, but
it also needs to know when to call them, right?
Yeah. That brings us to the prime directive,
the system prompt for the 11Labs agent itself.
How do we teach it to... Essentially, put the
user on hold and call the NAN tool at the right
moment. Yeah, this prompt is, it's like the agent's
constitution. It dictates everything. 11 Labs
has a nice generate with AI feature to get you
started with a base prompt. But you must manually
add a really crucial instruction. You have to
explicitly tell the agent. When the user asks
a question that requires up -to -date information
or web research, use the N8 Web Researcher tool.
Tell the user you are searching and then wait
for the tool's response before continuing. Ah,
okay, so it's a non -negotiable rule. Absolutely.
Without that specific instruction, the agent
might just try to answer from its general knowledge,
which could be outdated, or it might just get
confused about how to use the tool. This tells
it exactly how to handle research requests. That's
a really important detail. Okay, and what about
building the specialist team itself, the NANN
workflow that receives the call from 11 labs?
What are the key parts there? So the NANN research
backend is pretty focused. It usually has three
main parts. First, the researcher. The webhook
node receives the query from 11 labs. You pass
that query straight to, say, a perplexity node.
You'd probably use one of their sonar models.
They're designed for real -time web search, pulling
back sourced info. Okay, so it gets the raw data.
Right. But that raw data can be a lot. Maybe
too much for a quick conversational answer. So
next comes the editor. You pass Perplexity's
output to another AI agent node, its only job,
to be a ruthless editor. You give it a sign prompt
like, summarize the following information concisely,
no more than three sentences. Nice. Keep it brief.
Exactly. And finally, the report back. This is
just a respond to webhook node in ANN. It takes
that short, concise summary from the editor AI
and sends it straight back to the 11 labs agent
waiting on the line. Loop closed. This sounds
seriously powerful when put together. Let's make
it real. Can you walk us through a quick test
flight, a hypothetical conversation so we can
hear how it flows? Yeah, sure. Imagine you call
your agent. You could be using your browser,
your phone. You start. Hello, I'd like to do
some research on the company NVIDIA. The agent,
11 Labs, responds smoothly using the voice you
chose. I can certainly help with that. Is there
anything specific you'd like to know about NVIDIA?
Okay, nice and natural. Then you say. Yeah, let's
look at their Q4 2025 forecast. Now, the agent
recognizes this needs the specialist tool because
of that prompt rule. So it says, understood.
I'll search for NVIDIA's Q4 2025 forecast now.
Please give me just a moment. Ah, putting you
on hold politely. Exactly. And behind the scenes,
bam, the N8N workflow fires up. Perplexity searches,
the AI summarizes. The specialist team is working.
Right. Then maybe five, 10 seconds later, the
11 labs agent comes back. Okay. I have found
some information on NVIDIA's quarter four 2025
forecast. NVIDIA reported revenue of whatever
the summary is. Wow. It's a seamless conversation,
even with that complex lookup happening in the
background. That's genuinely impressive. It completely
changes the game for AI interaction. So thinking
bigger. What does this mean for expanding? You
mentioned a pro -level upgrade, the multi -specialist
idea. Yeah, this is where it gets really powerful.
You're not limited to just one NANA specialist
tool. You can create multiple NAN workflows,
each starting with a webhook, each designed for
a different task. Like what? Well, you could
have your web researcher, but also maybe an internal
database checker that looks at customer info.
or a calendar scheduler, or even one that sends
emails. Okay. You add each of these as separate
webhook tools in the 11 Labs agent settings,
each with a clear name and description. Then
your 11 Labs agent, Mission Control, becomes
much smarter. Based on your conversation, it
will figure out which specialist tool is the
right one to call for that specific task. So
it routes the request intelligently. Exactly.
It transforms your agent from just a researcher
into a truly versatile assistant. That's incredible,
giving the AI a whole team. Okay, so for people
wanting to actually deploy this, move beyond
just... Also, consider rate limiting on your
webhook to prevent accidental or intentional
overload. And keep a close eye on your API usage
11 labs, your AI model provider perplexity. Those
costs can add up if you're not monitoring them.
And technically, remember to switch your NAN
workflow from the test URL to the production
URL. Just remove the test part and make sure
the workflow toggle is set to active. Little
details, but crucial. Okay, beyond the text setup.
There's the experience itself, crafting the perfect
conversational experience. This sounds like an
art. Any tips on voice selection, keeping the
flow smooth? It absolutely is an art. For voice
selection, really think about the agent's role.
Is it a formal research assistant? Maybe a clear,
neutral voice? A creative brainstorming partner?
Perhaps something more expressive? And definitely
test voices with real content, not just, hello,
a voice that sounds great for one sentence might
get grading over a longer explanation. For conversation
flow, keep the agent's responses conversational
but also concise. Aim for maybe two or three
sentences for simple answers. Don't let it ramble.
And crucially, plan for errors. What happens
if the NAN workflow times out or fails? The agent
shouldn't just hang up. It needs a graceful exit,
like, I'm sorry, I seem to be having trouble
connecting to my research tool right now. Could
you try again in a moment? Handling failures
gracefully, yeah, that's key for a professional
field. And this brings us back to the prompt,
really. How do you ensure it behaves consistently?
That's advanced prompt engineering for voice.
The system prompt for your 11 Labs agent is its
constitution. It needs explicit rules. Define
the persona clearly. Set rules for conversation
management, like always ask clarifying questions
if a request is vague, keep information concise,
use natural language. And most importantly for
this setup, explicit tool usage guidelines. Define
exactly when to use each N8N tool, how to introduce
it, and what to tell the user if a tool fails
or takes too long. It sounds like the prompt
is doing a lot of heavy lifting in managing the
whole interaction. It really is. It's the brain
governing the conversation flow and tool orchestration.
So this blueprint, it really unlocks some serious
superpowers. These advanced voice agent patterns.
We're talking multi -tool agents doing research,
scheduling, emailing. Checking databases, analyzing
sales data. Yeah. And context -aware conversations.
Yeah. Connecting to memory systems like ZEP or
Supabase so it remembers past chats. Exactly.
So you can pick up where you left off or it can
build knowledge over time. Makes it feel much
more intelligent. And then my favorite, Star
Trek mode. Voice -activated workflows. Using
the voice agent as, like, a master controller
for other inane automations. Precisely. Imagine
just saying, computer, run the morning sales
report automation. Oh, man. For anyone who's
manually pulled reports every morning, that sounds
like pure magic. Have you actually seen teams
implement that kind of thing? Does this save
a lot of time? Oh, absolutely. E -efficiency
gains can be huge, especially for repetitive
tasks that can be triggered by a simple voice
command. It frees people up for more complex
work. And the agent talks too much, doesn't know
when to stop. That's almost always a prompt engineering
issue. You need to go back to that system prompt
and add stricter rules. Also, sales and lead
qualification. An AI voice agent can handle initial
outreach, answer basic questions, qualify leads,
and then pass the really warm ones over to a
human salesperson. Very efficient. And internally,
think voice -controlled tools for teams, allowing
people to query databases, run reports, or trigger
workflows completely hands -free while they're
doing other things. The possibilities really
do seem vast. Yeah. Okay, let's try and bring
this all together. The bottom line seems pretty
clear. The future is conversational. And think
about it. A few years back, the idea that a solo
creator, maybe even just you listening now, could
build an AI agent that holds a natural real -time
voice chat, hooks into the Internet, performs
tasks, that was pure sci -fi. Absolutely. Star
Trek stuff. Right. And today. It's just another
project you can build in AAN. You literally have
the blueprint now to create AI systems that don't
just crunch data, but actually engage, interact,
connect on a much more human level. It democratizes
some seriously advanced capabilities. Totally.
So here's your mission briefing, your takeaway
challenge, how you use this blueprint. Are you
going to cast your character? Maybe a witty,
sarcastic J -A -R -V -I -S like Iron Man's AI?
Or perhaps a calm, professional, endlessly patient
Star Trek computer voice. The personality is
half the fun. It really is. And think, what specific
nagging, recurring... problem in your life, your
work, your business could be totally transformed
by adding a voice. By batting one of these agents,
imagine connecting it to your smart home, your
office, literally saying, hey Jarvis, or whatever
you call it, turn on the lights in the studio
and start the coffee machine. The tech is there.
It's ready. It really is. It's just waiting for
your creativity. Time to get building. We really
hope this deep dive has sparked some ideas, maybe
given you some surprising insights. Keep exploring,
keep building, and we'll see you on the next
deep dive.