Imagine an AI that doesn't just, you know, pull
up facts, but one that truly understands how
those facts connect, how they influence each
other. An AI that can reason, much like a brilliant
research assistant, not just a super fast search
engine. We're really talking about a fundamental
shift in how AI interacts with information. It's
like giving AI not just a memory, but a mind,
you know, a way to connect things. This capability,
it genuinely changes what's possible. Welcome
to the Deep Dive. Today we're exploring a really
cutting edge development in AI. It's this fusion
of... agentic RG and knowledge graphs. Sounds
fancy. It does. But our mission here is to unpack
why the traditional ways AI retrieves information
often, well, they hit a wall. Right. And then
see how combining this sort of photographic memory
with the ability to connect the dots creates
something genuinely powerful. You'll get a feel
for what enables this, how it's actually built,
and what it could mean for how you interact with
your own data. Yeah, and I think a lot of us
have used AI to chat with documents, right? It
feels pretty magical sometimes. It does. You
ask a question, boom, it finds the answer right
there in your files. But what happens when the
questions get really complex, when you need to
understand relationships, maybe compare strategies
across different sources? That's where it gets
tricky. Right. So for anyone who's played with
traditional RRAG, that's retrieval augmented
generation, it's a good starting point. It's
meant to ground a language model in your specific
documents. But what are its limits? When do questions
need more than just a simple lookup? Well, think
of it like this. Traditional RRAG, it's great
for direct answers, finding that specific sentence.
But it's fundamentally inflexible. The process
usually involves breaking documents into these
little chunks. Then you create mathematical embeddings,
sort of numerical fingerprints for each chunk,
and store them in what's called a vector database.
This is for quick similarity searches. So when
you ask a question, the AI just finds the most
similar chunks and stuffs them into the prompt
for the language model. So it gets the context,
the raw text, but it can't really reason about
it. It doesn't understand the meaning behind
it. Precisely. The AI is constrained. It can't
say, look at the initial search results and think,
hmm, these aren't quite right. Let me try a different
angle. It's not built to explore complex relationships
or do multi -step reasoning across different
facts it finds. So essentially, it ends up being
a very sophisticated search engine. But still
just a search engine. Yeah, exactly. And one
that's kind of blind to the deeper connections.
It's like someone reading random paragraphs from
a dozen books, but never grasping the overall
story. Okay, so what specific kinds of questions
really break these traditional ARG agents? Where
do they really stumble? Questions about relationships,
comparisons, or maybe cause and effect. Those
often cause major trouble. Got it. Questions
needing more than just finding similar words.
Yeah, things that require synthesis. Right. Okay.
So here's where it gets really interesting then.
This new approach, agentic R, combined with knowledge
graphs, you said it gives AI not just that memory,
but also this ability to connect the dots. That
really does sound like a big leap. It truly is.
Think of it like this. If traditional R is a
really, really good filing cabinet, excellent
at pulling out specific files quickly, then this
new system... is more like your brilliant research
assistant. This assistant hasn't just read every
document you gave them. They've actually understood
how everything within those documents relates
to everything else. They've built this comprehensive
mental map. So the AI itself, the agent, it has
autonomy. It gets to decide the best strategy
to answer my question instead of just following
one rigid search then answer path. Exactly. That's
the core idea. For a simple fact, like what year
was this company founded? It might just do a
quick vector search. Fast and easy. If you ask
about relationships, like how does this new regulation
impact our main competitor, It can traverse the
knowledge graph, mapping out those connections.
And for really complex queries, it might intelligently
combine both methods. Two -sec silence, you know,
a real -world example. Instead of just finding
separate documents mentioning Microsoft and OpenAI,
this agent, because it has that knowledge graph,
it understands their multibillion -dollar partnership.
It knows how that connection impacts their strategies,
their products, everything. It's connected knowledge.
not just isolated facts. And that's what you
mean by agentic argue, the agent choosing the
strategy. Yep. An AI system where the agent itself
figures out the best way to find and use information
to answer your query. So how does this deeper
level of understanding fundamentally change what
AI can do for us? What kinds of problems can
it tackle now? Well, it allows the AI to actually
reason and provide much more nuanced answers
going way beyond simple fact retrieval. It starts
to sound like actual understanding. It's getting
closer. This capability, it sounds pretty advanced.
What are the actual tools making this possible?
Is this all super proprietary, complex stuff?
Surprisingly, no. A lot of it is built on open
source components. We're talking about things
like Pydantic AI. That's a Python library that
helps build structured, reliable outputs from
the AI agent. So it's not just rambling. It gives
you organized answers. Okay, structure is good.
Yeah. And for the knowledge graph itself, there's
a tool called Graffiti. to help define the structure,
the schema. And then Neo4j is pretty much the
leader for graph databases where all those intricate
connections actually live and get queried. Right,
Neo4j. And what about the vector database piece
for that initial similarity search? Ah, yeah,
that's a pretty clever choice in this setup.
It uses PostgreSQL, the familiar, robust SQL
database, but with an extension called pgVector.
So your standard relational database suddenly
gains high -performance vector search capabilities.
Whoa, wait. Postgresible with PGVector? So it's
like a database and a vector search engine rolled
into one, a two -in -one superpower? Yeah, kind
of like having a sports car that also happens
to be a submarine. Yeah. It's really powerful.
You get this solid, reliable SQL foundation,
but can also handle the vector stuff extremely
well for many use cases. That sounds incredibly
versatile. But I wonder, does putting vector
search inside PostgreSQL have any trade -offs
compared to using, say, a totally separate specialized
vector database? That's a fair question. You
definitely gain a lot of convenience and often
simplify your whole setup. You have one less
system to manage. But, you know, for scenarios
demanding absolutely massive scale, maybe billions
of vectors or ultra -low latency search. a dedicated
specialized vector database might still have
an edge. Okay. For most applications, though,
this Swiss army knife approach with Postgres
and PG vector is incredibly effective and flexible.
Plus, the whole thing uses fast API for the API
layer, which is modern and fast. Right. And critically
important, I think, you have amazing flexibility
with the large language models of brains, right?
You can plug in OpenAI's models or run open source
models locally using Alama or use Google Gemini
models. You're not locked into one vendor. No,
vendor lock -in is huge. So what's the key benefit,
the sort of magic, when you combine these specific
tools, Pydantic AI, Graffiti, Neo4j, Postgres
Vector, FastAPI? It's that seamless integration
that allows you to build these really powerful
yet customizable AI systems that can both search
and reason. Powerful and customizable. I like
the sound of that. Sponsoreed mid -roll. Okay,
so how does someone actually build this kind
of assistant? It sounds like a lot of moving
parts, but the article suggests it's pretty approachable
if you're willing to dive in. Yeah, the article
lays it out step by step. You basically get the
code, set up your Python environment. You'll
need PostgreSQL. And they suggest maybe using
a managed service like Neon, which handles the
vector stuff easily. Okay, Neon for Postgres.
And then your Neo4j instance for the graph database,
Beat. But the really crucial part, the magic
moment, is loading your knowledge base. Right,
your documents. Exactly. You just put your documents,
PDFs, text files, whatever, into a specific folder.
Then you run an ingestion script, and the AI
gets to work. What's it doing, though? It splits
the text into meaningful chunks, not just random
bits. It generates those vector embeddings for
the similarity search in Postgresql. And this
is key. It uses a large language model to read
through the text and identify the important entities
like people, companies, concepts, and the relationships
between them. Ah, so the LLM itself helps build
the knowledge graph. Precisely. It extracts that
structure, that understanding, and populates
the Neo4j graph database. That's where the real
understanding of your data gets built. Okay,
so once that's built and the graph is populated...
How do we test it? What kinds of questions really
show off its reasoning beyond just finding facts?
This is the fun part. You can actually see its
logic. Ask it something simple like, what are
Google's main AI initiatives? The agent will
probably just use a quick vector search. Finds
the relevant passages. Simple. Right. But then
ask something like, how are Amazon and Anthropic
connected? Now, it gets interesting. The agent
should realize this is about a relationship.
So it uses the graph search. It navigates the
connections in Neo4j and finds their big investment
relationship, maybe Anthropic's use of AWS infrastructure.
It pieces together the connection. That's cool.
And then for the really complex stuff, multi
-step reasoning. Ask it something like, compare
Microsoft's AI strategy with OpenAI's approach.
Okay, that's not a simple lookup. Not at all.
Beep, beep. Here, the agent has to be smart.
It needs to combine vector search to pull specific
facts about each company's strategy and graph
search to understand their deep partnership,
how one influences the other. The result should
be a really nuanced, comprehensive answer that
considers both the individual parts and their
relationship. It's genuinely impressive when
you see it work. Wow. Imagine scaling that kind
of analysis. A billion queries like that. The
depth of understanding you could unlock. It's
incredible. Really remarkable potential there.
So thinking about getting it running, beyond
just installing the software and setting environment
variables, what's often the trickiest part in
getting the quality right, making sure the understanding
is actually good. Yeah, that's key. Setting up
the databases and making sure all the connections
and like. API keys are right, can definitely
be tricky for first timers. But getting the quality
often comes down to the data feeding in and how
the agent reasons. Right, the inputs matter.
Hugely. Okay, so once a system is built, how
locked in are you? How customizable is it? Can
you really tailor it? Oh, absolutely. That's
the beauty of it being mostly open source. You
can totally customize the types of entities and
relationships it looks for. Maybe you need it
to understand legal clauses or specific scientific
concepts. You could integrate multimodal knowledge.
Imagine feeding it images or audio files alongside
text. You can set up real -time updates so as
new documents come in, the knowledge graph grows.
And you can definitely optimize performance for
really massive data sets. It's designed to be
flexible. That's really powerful. Now, what's
equally fascinating, you mentioned earlier, is
how the system itself was actually built. The
author used something called context engineering
with an advanced AI like Claude. So the AI wasn't
just like a co -pilot helping out. It was more
fundamental. Way more than a co -pilot, really.
The description makes it sound like... Imagine
the AI was the incredibly skilled camera operator,
the lighting expert, the sound engineer, and
the editor. Wow. While the human was primarily
the film director, providing the vision and the
overall plan. Apparently, over 90 % of the system's
code was generated by the AI in just 35 minutes.
35 minutes. Yeah. By feeding it a really detailed
project specification, the requirements, and
crucially, access to the documentation for all
those tools we mentioned. vulnerable admission
I mean I still wrestle with prompt drift myself
sometimes getting the AI to consistently do what
I want so seeing this level of complex system
generation through AI assistance is genuinely
mind -blowing and that's context engineering
giving the AI all that background material essentially
yeah it's about guiding the AI assistant by providing
comprehensive documentation clear examples detailed
plans basically giving it everything it needs
to understand the task deeply and generate the
right code or system components. So what's the
big takeaway then from seeing AI build such a
sophisticated system so rapidly? What does that
imply more broadly? It just shows how AI -assisted
development can drastically speed up the creation
of complex systems. It's a game changer for building
things quickly. A definite force multiplier.
Okay, so for listeners who are thinking, hey,
I want to try building this, are there any common
pitfalls, any gotchas they should watch out for
that might trip them up? Definitely a few things
to keep in mind. First, during your initial testing
phase, creating that knowledge graph from scratch
can sometimes take a really long time, especially
with lots of documents. Right. The good news
is you can actually skip that step temporarily
for faster iteration. There's usually a flag
you can use when running the ingestion script,
something like no knowledge graph, just to get
the vector search part working first. Okay. Handy
tip for testing. What else? Another common one.
Sometimes you find the agent gets stuck in a
rut. It always uses vector search or always tries
the graph, even when the question clearly suits
the other method. Ah, it's not making the right
decision. Exactly. If that happens, you need
to go back and refine the system prompt you give
the agent. You have to be super clear about the
criteria it should use to decide which tool vector
search or graph search is appropriate for different
kinds of questions. So tweaking the agent's instructions.
off irrelevant like it's not finding the right
chunks yeah that can happen first thing is maybe
experiment with different embedding models some
are better than others for certain types of text
okay but crucially pay attention to how you're
chunking the documents this idea of semantic
chunking is really important semantic chunking
yeah instead of just chopping documents into
rigid say 500 word blocks right You split them
based on logical breaks in the content, paragraph
sections, topic shifts, meaningful segments.
That usually gives the vector search much better
context to work with. Semantic chunking basically
is just dividing text into meaningful, logically
coherent parts, not just fixed size blocks. It
makes a big difference. Makes sense. Better chunks,
better search. And what about if you just can't
connect to Neo4j, the graph database? Oh yeah,
that's usually simpler. Double check your credentials,
username, password, database URL, and just make
sure the Neo4j service is actually running on
your machine or wherever it's hosted. It's often
just a configuration type or the service isn't
started. Standard troubleshooting then. So beyond
these specific fixes, what's a good general mindset
to have when you're troubleshooting these complex
AI systems? Patience. iteration it often involves
carefully tweaking the AI's internal prompts
trying different chunking strategies maybe adjusting
parameters and seeing how it affects the results
it's rarely a one -shot fix careful refinement
is key iteration and refinement got it so let's
pull back what does this all mean for us what's
the really big idea here it feels like more than
just a new technique I think it is. The big idea
is that the future of AI, especially when it
comes to understanding information, isn't just
about making search faster or training even bigger
language models. It's really about building systems
that can genuinely reason. We're moving away
from thinking about static databases that just
store isolated facts towards building dynamic,
interconnected knowledge networks where the relationships
are just as important as the facts themselves.
You know, traditional reg gives you a really
good search engine. Agetic reg, combined with
knowledge graphs, starts to give you something
closer to a brilliant research assistant. From
search engine to research assistant, that's a
powerful shift. It's a fundamental change in
how we can expect AI to help us make sense of
complex information. So this deep dive really
reveals a fundamental shift, doesn't it? Moving
from just simple search towards an intelligent
understanding that actually grasps context and
crucially relationships. Absolutely. And the
tools are becoming accessible enough to actually
build this now. So maybe some homework for our
listeners, if you're feeling ambitious, that
is. Yeah. Get your hands dirty. Try setting this
system up. The article provides the guide and
the code. Load it with your own documents. Maybe
it's your company's internal notes. Maybe research
papers you need to synthesize. Project documentation.
Whatever complex information you need to master.
And then test it. Exactly. Ask it different kinds
of questions, simple facts, relationship questions.
Those complex comparison queries push its boundaries.
See what it can do with your data. Because this
is really about more than just playing with code,
isn't it? It's about starting to grasp how AI
can navigate that incredibly complex web of relationships
within your information, uncovering insights
that were maybe hidden before. It's about starting
to build that future of AI for yourself, tailored
to what you need to understand. So the final
thought to leave... everyone with, what kind
of complex relationship -based questions could
you finally get answers to if your AI truly understood
your data? Out to your own music.