Okay, so today we're diving into like the foundation of AI,
training data sets, the huge amount of data
that all these amazing AI models we hear about.
Yeah. Learn from.
You said over this paper,
bridging the data provenance gap across text,
speech and video.
Yeah.
It's a massive study.
Like the biggest audit of its kind.
Right.
Covering like almost 4,000 data sets.
It is a really extensive study
and it's really the first of its kind
to take the sort of comprehensive look.
Across text, speech, A and D video.
Yeah.
So that's pretty crazy.
So that's basically saying,
where is AI getting its education from?
Yeah, exactly.
I mean, you think about like how we learn, you know,
as humans through books and experiences.
Yeah.
This is essentially AI's version of that.
So this is how it's learning about the world.
And it's really interesting to look
at where that data is coming from.
Yeah, totally.
So what were some of the things that kind of jumped out
at you when you first read this study?
Well, one of the first things that struck me
was the sheer volume of data that's coming from the internet.
We're talking about web crawled data,
you know, websites that are scraped for information,
social media posts,
even content that's been synthetically generated.
Wait, synthetically generated.
Yeah.
What do you mean?
So this is content that's actually created by AI.
Oh, wow.
So AI is learning from AI.
Yes.
That's interesting.
And that's a growing trend actually.
Really?
Yeah.
And it's fascinating, right?
It creates this sort of feedback loop
where AI is learning from itself.
But it also raises questions.
You know, if AI is learning from content created by other AI,
are we just perpetuating existing biases
or could we even be amplifying them?
Right, yeah.
That makes total sense.
So when we talk about speech and video data,
like what are the main sources for that?
Is it still, like is YouTube still the king?
YouTube is definitely a major player,
especially for video data.
The study found that almost a million hours of video data
used in AI training comes from YouTube.
That's way more than like movies or audio books,
for example.
Interesting.
So when it comes to speech, then, what are we looking at?
So speech data is a little bit more diverse.
You've got YouTube, of course.
You've got audio books.
You've got things like podcasts, lectures,
even things like voice recordings
from customer service interactions.
Wow.
OK, so we've got AI learning from all
of these different places, from web pages, social media,
even other AI.
YouTube is just like massive digital melting pot.
Yeah, it really is.
But can anyone just grab this data and use it?
Isn't it all publicly available?
That's where things get really interesting
and a little bit messy.
OK.
The research uncovered a huge issue
with what they call undocumented restrictions.
Undocumented restrictions.
What does that even mean?
So basically, over 80% of the content
used in these data sets.
Across all formats.
Across all formats has some kind of hidden limitation.
Duried in the fine print of the original source material.
OK, so even if a data set has an open license,
the actual data itself might not be free to use.
Exactly.
And this is a really big problem,
especially for AI developers who might,
assuming that they have the right to use certain data,
when in fact they don't, a lot of this content.
Especially if they're scraping it
from these different websites.
Right.
Especially stuff scraped from places like YouTube.
You know, might have restrictions against commercial use
even if the data set itself seems open.
Wow.
So that's a whole other layer of complexity.
So AI developers could be unknowingly training
their models on data that they're not even allowed to use.
That's right.
And we're already seeing lawsuits pop up
around AI training and copyright infringement.
So this is a really serious issue.
So it's not just about finding the data.
It's about understanding the legal complexities that
come with it.
So who is actually doing all of this data sourcing for AI?
Is it all like big tech companies just
gathering up all this information?
It's actually a lot more diverse than you might think,
especially when it comes to publicly available data sets.
Oh, really?
Yeah.
For speech and video data sets, academic institutions
are actually the biggest players.
Interesting.
I would have thought it was big tech.
Why are universities so involved in this?
Well, I think it's a combination of factors.
Academia has a long tradition of open research and data
sharing.
Yeah.
And they often have the expertise and the resources
to create high quality data sets,
even if those data sets are smaller in scale
than the ones created by large corporations.
They're responsible for over 70% of video data set creation.
Wow.
That's incredible.
Yeah.
So big tech might have a scale, but academia brings
the expertise and the focus on research.
What about text data sets?
Who's leading the charge there?
Text data sets are a lot more diverse.
You've got university researchers, of course.
But you also have industry labs, research groups,
even startups contributing to the creation of text data sets.
So it's not just like one monolithic entity?
No, not at all.
It's much more of a collaborative effort.
And probably contributing to those licensing issues.
You're absolutely right.
That's a key point.
Yeah.
The researchers highlighted, the more players
involved, the more complex that licensing landscape becomes.
Yeah, for sure.
Let's explore that in more detail in the next segment.
OK.
Yeah, it really is a double-edged sword, right?
Yeah.
It's great to have more people involved in creating
these data sets.
I mean, that's fantastic for innovation.
But it also makes it really hard to keep track
of all those different licensing agreements.
Yeah.
I can imagine.
Like herding cats.
Exactly.
And it gets even trickier when you
realize that a lot of this data is coming from sources
with strict restrictions on commercial use.
Right.
Think about social media platforms, websites
with very specific terms of service, even content that's
protected by copyright.
Right.
Yeah.
We talked about the undocumented restrictions
earlier that are lurking in that original source material.
This is where that really becomes a problem.
This is where it becomes a huge problem.
And think about all the data that's
being scraped from YouTube.
Yeah.
While the data sets themselves might seem
to have open licenses, the original content
is often covered by YouTube's terms of service, which often
prohibits things like data scraping and commercial use.
Wow.
So AI developers could be walking into a legal trap,
assuming they have the right to use data when they actually
don't.
Yeah.
And that's incredibly risky, especially now
that we're seeing all these lawsuits popping up.
Totally.
Yeah.
So how do you even begin to navigate that?
That's the million dollar question.
The researchers suggest that we need better tools
and practices for actually tracking data provenance
throughout the entire lifecycle of a data set.
You know, it's not enough to just kind of slap an open license
on a collection of data.
We need to know where each piece of data came from,
what the original licensing terms were,
and how those terms might have changed over time.
OK.
So let's say we've got to handle on all these licensing issues.
Are there any other challenges we
need to be thinking about when it comes to AI training data?
Oh, there definitely are.
OK.
And one of the biggest ones is the issue of representation.
You mean like whether the data actually reflects
the diversity of the real world?
Exactly.
If we want AI to be truly intelligent and useful,
it has to learn from a wide range of perspectives
and experiences.
Right.
It needs to get a well-rounded education.
That's a great analogy.
Otherwise it might have blind spots or even make decisions
that are biased or unfair.
Exactly.
And unfortunately, the study found
that things haven't really improved much when
it comes to geographic and linguistic representation.
Really?
Yeah.
That's kind of surprising.
I would have thought with all this talk
about building more inclusive AI,
we would be seeing some progress in this area.
Yeah.
It's a really complex issue.
And while we are seeing more data
from underrepresented languages and regions,
the overall picture is still very Western-centric.
Most of the data sets and the organizations
creating them are concentrated in North America and Europe.
So even though there's more diversity than before,
it's still not balanced.
Not even close.
For example, the study found that data sets
from organizations in Africa or South America
make up less than 0.2% of all the data.
Less than 0.2%.
That's incredibly low.
It's like those regions are barely even represented
in the data that's shaping the future of AI.
Yeah.
That's the reality right now.
And it has huge implications.
I mean, think about it.
AI systems trained on such skewed data
are likely to perpetuate existing biases.
And they're going to have trouble understanding
the nuances of different cultures and languages.
Yeah.
So AI could end up with this very narrow view of the world.
Exactly.
Missing out on all the richness and complexity
that comes from different cultures and perspectives.
Exactly.
And that's a problem.
If we want AI to be a force for good in the world,
if it's only exposed to a limited set of experiences,
it's only going to make decisions that are biased or unfair.
It's like teaching AI to see the world
through a very specific lens, potentially missing
the bigger picture.
So what can be done about this?
How do we address this?
Well, there are a few things we can do.
First of all, we need to be a lot more intentional
about collecting data from a wider range of sources.
That means working with organizations and communities
in underrepresented regions to build data sets that
reflect their lived experiences.
It also means investing in research and development
to create better tools and techniques.
That can help us identify and mitigate biases in AI systems.
So it's not just about collecting more data.
It's about collecting the right data.
Exactly.
In the right way.
And making sure that it's used responsibly.
Absolutely.
And it's not just the responsibility
of researchers and developers.
We need policymakers and industry leaders
to get involved, too.
We need guidelines.
We need incentives that promote the development
of more inclusive AI systems.
This makes me think about the long-term implications
of all this.
If we're not careful, we could end up
with AI that reflects and amplifies the inequalities that
already exist in the world.
It's a very real concern.
But I don't think it's too late to course correct.
We have this amazing opportunity to build AI that's
fair, equitable, and beneficial to everyone.
But it's going to take a collective effort from everyone,
from all the stakeholders to make that happen.
Well said.
Yeah, this conversation has really
highlighted the importance of thinking critically
about the data that's shaping the future of AI.
It's not just a technical issue.
It's a societal one.
Absolutely.
And it's a conversation that we need to keep having.
The choices we make today about AI training data
are going to have a huge impact on the world of tomorrow.
Yeah, it feels like we're really at this turning point
in AI development.
We have this amazing technology with so much potential.
But we're also starting to see what
happens when that technology is built on data that's
incomplete, biased, or even, like we said, illegally obtained.
That's a really good point.
And it brings us back to this fundamental issue
that this research paper is really highlighting,
the importance of data provenance.
Really understanding where our data comes from,
how it's been handled, what limitations it might have.
That's absolutely critical if we want to build
AI that's trustworthy and responsible.
It's like building a house.
You wouldn't just use any materials you found lying around.
Exactly.
You want to make sure they're strong, safe,
for the job, and you'd want to know where they came from.
You want to know where they came from, exactly.
And just like you would have a building inspector come in
and check the foundation of a house,
we need to find ways to audit the data that's
going into these AI systems.
We need to be able to trace the origins of the data,
verify the licensing, assess its potential for bias.
So it's not just about collecting more data.
It's about being much smarter about how we collect it,
how we document it, how we use it.
It's about quality over quantity in many ways.
Yeah, totally.
This makes me think about the role of transparency
in all this.
If developers are unknowingly using data
that they're not supposed to, or if AI systems are being
trained on biased data, how can we even
begin to address these issues?
Well, transparency is absolutely key.
We need much clearer documentation.
We need standardized licensing practices.
We need open communication between the people who
are creating these data sets and the developers who
are actually using them.
Right, and it also feels like there's a huge need
for more education and awareness within the AI community
itself.
So not every developer is going to be a legal expert
on data licensing or an ethicist who specializes
in bias detection.
Absolutely.
We need to give developers the knowledge and the tools.
They need to make really informed decisions about the data
that they're using.
And that means better education on things like data provenance,
licensing, ethical considerations.
And it also means developing new tools and resources
to help developers identify potential issues
with their data sets.
Yeah, it's almost like we need a whole new field
of expertise within AI.
Data provenance specialists who can help people navigate
these complexities.
That's a great idea.
I mean, it's something that we're already starting to see.
There's a growing demand for people
who understand the ethical and the technical aspects of data
sourcing for AI.
So it's not just about the algorithms anymore.
It's about the entire data ecosystem.
A whole ecosystem.
That supports those algorithms.
And that's why this research is so important.
It's really shining a light on this often overlooked,
but absolutely crucial aspect of AI development.
The data that we use to train these systems
is ultimately going to determine their capabilities
or limitations.
And you know how they impact the world.
Right.
Yeah, it's a powerful reminder that building responsible AI
is about so much more than just writing code.
It really is.
It's about making thoughtful choices
at every stage of the process, from data collection
to deployment.
Absolutely.
Well, this has been a really fascinating and eye-opening
conversation.
Yeah, it has.
Thanks for walking us through all this research
and helping us understand the complexities of data provenance
in AI.
My pleasure.
I'm really glad we had a chance to really dive deep
into this because it's something that needs a lot more attention.
Totally.
So listeners, I hope this deep dive has given you
a new appreciation for this unseen world of AI training data
and the incredibly important role it
plays in shaping the future of AI.
So until next time, keep those minds curious.
And those questions coming.