Hey there. Welcome to Data Democracy, a podcast
where we explore ways to make data and AI more
accessible to everybody. We do this by interviewing
experts across industries and asking them how
they think about data, what are some of the challenges
they face when it comes to data, and also if
they had a magic wand, to get whatever they needed.
what kind of intelligence and models they would
love to have. We've got a great guest today,
Mary Beth Mashkovis. Mary Beth is an entrepreneur,
a thought leader in data digital marketing. She
is the founder of InsightLine Analytics, which
is a digital consultancy agency focused on delivering
customer impact. She also runs the digital team
at Arch Digital, previously ran analytics marketing
teams at eCapacity, Fantastic ES, Tyak, and various
other companies in Europe and the U .S. Marybeth
is a thought leader who talks about leadership,
data coaching, customer -centric marketing, data
strategies. I just love reading through Marybeth's
blogs and articles because like me, she's focused
on customers' needs first and then kind of building
solutions to suit those needs versus the other
way around, which I feel very frustrated about.
But yeah, we have a ton of stuff to talk about,
so without further ado, welcome, Mary Beth. Thanks
so much for having me. Like I said before, it's
so nice to catch up again. It's been a while
since we really talked face -to -face. Awesome.
Thank you so much again for being on our podcast.
Yeah, absolutely. Awesome. Awesome. All right.
Let's get started and tell us something about,
tell us about your journey so far. How did you
get started and how do you get where you are?
Yeah, that's always a really important question
to ask. And I tell this story a few different
ways, but I think for this, How I ended up in
digital analytics, which as a field, when I got
started a little over 10 years ago, wasn't that
established yet, was that I actually have an
evolutionary biology background. And during that,
I took classes in advanced statistics and I learned
R. And just by some happy happenstance, I got
recruited at a custom software company that also
is a consulting firm as a marketing analyst because
they had this idea that marketing was going to
get a lot more data centric and that they were
going to need people with like a scientific background
to be able to evaluate results. They were right,
obviously, because things have blown up since
then. So that's really where I got started. really
fell in love with digital as a whole but was
really enticed by data and what it could do and
how you could use it to make a tangible difference
that you could see and measure in businesses.
So my career took me abroad and I spent almost
six years actually in Copenhagen working for
several companies including Kayak like you mentioned
and consulting which I've been back and forth
in -house and as a consultant and so when I came
back to the U .S. in 2020, trying to decide what
to do. I felt like it was the right time for
me to start my own consulting firm and consult
in like the way I wanted and about the things
that I wanted. That's awesome. That's awesome.
Yeah, I mean, it definitely, when I talk to you
and kind of read through your content, it definitely
comes across that you're passionate about both
data and marketing. So yeah, you've got the R
background and data background that really shows.
That's awesome. What does your day to day look
like at the Insights line? Yeah, so I kind of
wear a few different hats now because so the
firm has grown and now we're about six people.
So my day to day is kind of a combination of
serving our clients and then also running a business
now. So I often have one or two larger clients
that I'm serving either an interim role with.
So sometimes I'll sit as like a director of analytics
or something like that, or running a data strategy
project for a client. And then on the other side,
I'm creating content for our business and then
also coaching analysts and just running the team
in general. So, you know, it's a pretty right
now, a pretty balanced schedule between the two
and business development, of course. but still
get to do quite a lot of hands -on work as well.
That's awesome. That's awesome. Yeah, I mean,
that's a lot of different roles, different hats
that you wear. And also, it's always good to
kind of stay grounded with hands -on work. That
keeps me sane a lot of times. Yeah, definitely.
I have to say, you know, there's kind of two
approaches to leadership. There's like some people
who completely let go of whatever technical work
and don't get into the weeds ever. And then there's
others that try and keep that alive. And, you
know, whether it will drive my team crazy or
not, I still want to. stay at least a little
bit up to date. I'm letting go of some of the
details. I'm not going to be our, you know, in
-house GA4 expert for every single tag or anything
like that anymore. But, you know, I also make
sure that I'm still involved in projects and
understand, like, the trends of what's going
on in the industry. Yeah, that's exactly where,
you know, how I like to approach as well. I mean,
it's a balance. You don't want to get too hands
-on and, you know. frustrate the team. Right.
Yeah. I don't know what their opinion of it is,
but most of the time they tell me they're happy.
So I think I'm doing a pretty good job. And I
try and also like, if I feel like it's something
that will step on their toes, I'll like run a
project on my own or completely separate from
the company, just to make sure that I'm still
fresh with my knowledge. Okay. Yeah, that's a
great approach. That's awesome. And any tips
for aspiring entrepreneurs in the digital marketing
world? What have you learned and what would you
suggest? I think for someone who's starting out,
it does really help to have a strong background
in that field already. I know that there's a
lot of people who do try and start. companies
right out of school or whatnot with a lot of
experience. But for me personally, having that
really strong consulting background where I was
able to get a lot of things right and get some
things wrong on someone else's dime before I
started my own company was really beneficial.
Marketing and data are... you know, such broad
fields that I'd say that you don't really know
exactly where you want to niche down into. And
you really do need to do that to be successful
until you try on a few different things. And
it's much easier to do that while you're employed
and you have that safety net than it is to just
kind of throw yourself into something and be
not sure if that's like the direction that you
really want to go. Yeah, I mean, I totally agree
with that. I mean, I started my career as an
entrepreneur. we started building websites for
pocket money. But then we quickly realized that
we make so many mistakes that it costs us a lot
if we are running a company and also learning
at the same time. Like you said, if you can,
just go learn, make mistakes on somebody else's.
And if any of my past employers are listening,
I promise I didn't make that many mistakes. It
just gives you some more flexibility and you're
not worrying about, like you said, running a
business is a whole skill set in its own. And
I felt really happy that I really knew how to
consult and what I want to consult on and how
I was going to do my packages and everything.
And I could just focus on how do I do taxes and
how do I do all these other things that I've
never had to worry about before. Right. And if
any of my employers are listening, yes, I'm making
a lot of mistakes. I will make a lot of mistakes,
but don't make that rule. I mean, from my friends
that don't make the same mistake twice as long
as you keep that going, you know. Cool. That's
awesome. You talked about R and kind of data
analysis and everything. How is data after your
career so far, how has this data played a role
in your career? Ooh, yeah, that's a good one.
I mean, I think that having that kind of hard
quote unquote data skills to work with has really
defined how I've gone about roles, even when
I was in more marketing roles. So when I was
a marketing director, et cetera. I think what
it did was it gave me a completely different
perspective than someone who is coming from more
of a traditional marketing background or PR or
something like that, because I was looking, I
was really results driven and I was able to kind
of do a lot of the analysis I wanted to do myself,
where a lot of other marketers sometimes have
to wait for a data team to deliver on something
and then like hope that the person got their
business context question and everything like
that. So that's really defined my career. As
I got into the latter part of working for companies,
I realized that I really wanted to specialize
into data and be an analyst, like I started as
well, and evaluate both marketing and product
data as my focus. But yeah, starting with that,
now you would call it almost like a data science
background of having R. was really helpful in
that because I was able to do things super efficiently
to things that like with unwieldy spreadsheets
etc. I already kind of had some skill sets there
to get that done not that like I would say it's
like the most important skill but it's one of
the important skills. Yeah it definitely helps
and you know I always have learned that you know
marketing and data are joined at the hip because
you don't you can't work one without the other
and you know It's always good when, for me, it's
always good to partner with a person who has
had some data background on the marketing side,
so it makes my job easier. For sure. Yeah, you're
a thought leader, like I said, at the intersection
of digital marketing and data. And I know you
think a lot about how companies and clients should
approach data. Can you please share your thoughts
on it and give us some perspective? Yeah, I mean,
sorry, go ahead. Was there anything else? OK.
Yeah. So this is alluding a little bit to something
that's going to happen later this year. But I
do have a unique philosophy. And I think as time
has gone on, I've gotten a little more specific
about it. And what it is, is I think that I want
people to feel good about data. And so what that
ends up meaning is, as you look in our kind of
like data flooded world right now, even though
There's a ton of data being generated by pretty
much every company, just overwhelming amounts.
Not everyone feels super great about it. We don't
really use it to its fullest potential. And there's
a lot of waste or spillage of data where we spend
a lot of time collecting it. We're storing it
and everything, and we're not using it. And I
think what that really comes down to is that
there's like these other factors. Data is the
tool and what we're running into is there's a
lack of trust in data and organization and people
don't feel good about it when someone's sharing
something and they don't feel like it represents
the company and things like that. what my approach
is in digital marketing or marketing in general,
where that really can create a conflict because
a lot of times marketing is about feelings. Data
comes in and a traditional analyst can be so
exacting in how they want something, and those
two can sometimes cause some conflict. My approach
is really like, working to make the data quality
as good as possible, being really pragmatic about
what you're collecting and if it makes sense
to collect that. And then on the other hand,
focus a lot on using what you have and using
the most useful data. And what really you need
to do first to do that is to make sure you have
a culture that's ready to receive data. Yeah,
that's a great point. And you really have to,
there's a lot of... intricacies when it comes
to data. Well, what does having data mean? Especially
in the digital world, data is never accurate
and it will be the accuracy or you won't be able
to collect everything that you want. You go more
so going forward because of privacy laws and
everything and it should be that way. because
more power to the consumers, right? But being
on the data side, I'm like, no, I wish I had
all the data. But yeah, I think you passed on
a point. In terms of our goal, if our goal is
to get to intelligence, there are precursors
to it, which is you've got to have the right
infrastructure to collect the data and then...
you have to have adoption by the customers or
the users. And then you have to get the feedback
from them and then start developing intelligence.
So there are a few steps to getting to the intelligence
or, you know, be it testing and marketing, or
maybe in my world, be developing a machine learning
model that needs to be adopted and then provided
some feedback, only then we can tune it to get
to some... Although it sounds like some people
think of it like magic, it's just iterative process
of getting to a good phase. Yeah, you made some
great points there. Yeah, I followed you on LinkedIn
and I know... GA4 is big on your mind. And can
you please tell our audience what is Google Analytics
and how companies leverage it to go ahead? Yeah,
that's a great one. I think sometimes we skip
over explaining something like that and just
assume everybody knows. But Google Analytics
is the most popular. platform for tracking behavioral
data and marketing data for websites and apps.
It's one of the only free solutions. I'm putting
a little asterisk by that because there's some
things changing. What it does is you put a tag
on your website. And by tag, it's a little bit
of code that goes into the header. And what it
does is start to monitor what people do on your
site. So when they hit the site, where do they
go afterwards? Where did they come from? And
gets that general intelligence. And it's really
easy to implement, too. So almost every site
can implement Google Analytics and get something
out of that data. So that's it. my super simplified
version of what it does. Obviously, there's a
little bit more that goes into it for bigger
companies, but that's the general premise. That's
awesome. Yeah. It's super impressive the way
that those little tags can tell us so much about
our customers, especially on the behavioral side,
where they come in, the final metrics and all
that. But overarching, overarchingly in terms
of types of data. I think of it in terms of geographical
data when it comes to customers and then demographics
a little more nuanced and also psychography,
which is behavioral. So Google Analytics is super
helpful for us to understand the behavior of
our customers within our apps or websites and
stuff like that. So thanks for helping us understand
that. I know you use a couple more tools. Are
there any alternatives to Google Analytics that
you can tell us about? Yeah, there certainly
are. One of my personal favorites is Adobe Analytics.
That's on the enterprise level for sure, but
that's kind of, I would say, kind of the two
big. players in the market. And then of course
there's like a whole suite of more specialized
what I call best of breed tools. So this is something
that's once again changing a little bit right
now, but previously there's been tools that are
a little more focused towards apps. If you're
a company that's more focused on having an app
and you know maybe your website isn't your primary
focus, then there's tools like Mixpanel and Amplitude
that are also good options. And now with the
They've been around for a little longer than
this, but now with the switch of GA4, there's
definitely a lot of pain points around that.
There's also some other pain points about Google
within Europe. There's some other options like
Piwik Pro. I'm not sure if I'm pronouncing it
right. I think I've never heard it pronounced
out loud. And there's some other few free solutions
that are also starting to become a little bit
more prominent. So there are a lot of options.
And then there's also... some more custom solutions.
So if you were a really advanced company and
you wanted something that was completely your
own, you could be using something like Snowplow
and Snowflake together. So there's also some
solutions like that as well. So it's definitely
a complicated landscape, but I forget the exact
number, so I don't want to quote it exactly.
But there's a vast majority of sites, especially
small to medium businesses, are on Google Analytics.
That's awesome. Yeah, Google Analytics is definitely
the biggest player in the market. I know there's
a big shift that Google Analytics is going through.
Can you talk about that and what was it originally
called and what did that do and what's the next
phase? Yeah, so the current version of Google
Analytics that's been around for a few years
now is called Universal Analytics. was announced,
I forget exactly when it was, it feels like it
was a really long time ago, I've been slogging
through these migrations, is Google Analytics
4. And that transformation, and this is colored
by my opinion completely, but someone who's more
familiar with also some of those other analytics
tools, is that Google is catching on to moving
towards event -based tracking. So prior, they
were session -based, and really what the difference
is there is like, event based tracking is stronger
for combining multiple properties. So if you
have a website and an app, it was really difficult
with Google Analytics. And I thought that they
were really weak in it was measuring app analytics.
They just weren't really set up to be event focused,
which is what apps are. And so I think that's
one of their major moves. I think there's a lot
of other pressures of why Google Analytics is
changing. And one little like stepping back a
bit is that the way this new version is working
compared to some of the other version updates
that Google has done in the past, you have to
completely re -tag your implementation and switch
over. So it's really a tool migration, which
is brutal because any time that you do an analytics
tool migration or any other technical tool migration,
that's a big... a strategic undertaking and like
a big cost undertaking for companies. So Google
announced that they're going to do this and that
every company is expected to go on this journey.
And whether they were ready or not to go on to
do that sort of strategic project, they're going
to have to whether they choose to go to GA4 or
whether they choose to move on to another analytics
tool. And so We're still hearing bits and pieces
about what their final vision is. I honestly
couldn't speak to it. It would all be conjecture.
But I know it's partially because of some of
the privacy pressures that they're switching
the way their tool functions. Personally, like
I said, I think it's because they're trying to
move to a better interface. They've really operated
off of these static reports that are very marketer
friendly, but their competitors like Adobe Analytics
have had a lot more flexibility for some time
of making visualizations, catering things, having
it be a lot more in the flow analysis. So they're
definitely moving towards that and they're trying
to apply some more intelligence to that. That's
another thing that they've been a little slow
on the uptick. for compared to Adobe is anomaly
detection and things like that. So I know that
was a really broad explanation, but we're going
through a strategic technical migration, everybody
who was on Google Analytics, to a tool that's
going to function pretty differently. Yeah. I
mean, it's definitely a seismic shift for companies.
And a lot of companies are depending so heavily
on Google Analytics that they have to do it.
And I think if anybody is interested, July 1st
is the deadline? Yeah, July 1st is the deadline
for any non -Google Analytics 360, so the premium
version. And then I believe it's October 1st
for the Google Analytics 360 customers. So if
you're paying for the premium version, you have
a little bit more time. I don't know if it's
really a fair... a bit more time because anyone
who's on 360, their implementation is pretty
complex. So I feel like they could use a few
more months than just that, but yes. Yeah. And
also the cost difference is huge between free
and 360. We looked into it and then it was definitely
a lot more expensive. Yeah, just takeaways for
myself as well. GA4 is focused on more about
app analytics and also event -based analytics,
and it gives us a lot of fun metrics, and you
can customize the way that you could track different
buttons, scrolls, different objects within your
web pages or app pages. very marketing and data
geeks dreamland, in my opinion. Yeah, and definitely
a little bit, the more business users and not
as data -focused marketers are going to face
a reckoning, that's for sure. It's going to be
a very different way of getting information out
of the tool. Yeah, and one of the things that
I've been preparing my executives and my teams
for is that your historical data, you won't be
able to just continue on with it. There will
be some differences because the way that these
two versions are looking at the data is entirely
different. Like we said, session -based versus
event -based. So if you've had a thousand sessions
in the past, a month, you could see 1 ,500 or
800. It really depends. Yeah. I've been seeing
that already and hopefully I have been because,
by the way, if you're listening to this and you
haven't migrated yet, you're in trouble because
you only have a few months and you're probably
not going to finish the migration in time. I
still think you should start now. But yeah, there's
going to be discrepancies and also just differences
that aren't really like an issue, but I'm also
seeing that like When we start these implementations,
sometimes the tracking doesn't really do what
we expected. And there's a lot of pieces like
that. So have been going through that with clients
right now where we think that we got parity and
then we're looking, okay, is this a difference
in how they track it? Or is this that something
is not going right with the tagging? And that's
a lot of work to go in and make sure that those
are correct. And right now, ideally you started
your migration a year ago or something like that,
especially if you're a big company. And you've
been spending the last few months and will be
spending the next few months just continuing
to take inventory of that. Yeah, absolutely.
And I think running both these versions in parallel
and making sure that you're comfortable with
the Delta. And then you could probably, once
GAUA goes away, you could kind of extrapolate.
your data backwards and get year over year comparisons
just to keep your sanity. It's a lot of work
and yeah, if you've not started your migration,
definitely reach out to Mary Beth. She can help
you out. Yeah, we have a few slots left. I'll
say that that's the other warning that I have.
that all of the agencies that do this are going
to be really heavy hit. We do have some, as of
recording this on May 3rd, I don't know when
it's going to get published, we do have some
slots available and we're always happy to help
and we won't shame you too much. But it's definitely
something where like, if you don't have the resources
in house, it's like, we're almost getting to
the point where we won't bother dual tagging.
But what we do, that's I think it's an important
call out, is almost no one on Google Analytics
has like a data dictionary, right? Operating
a data dictionary is a list of all the tags that
you have, all the properties, what they do. And
so what we're finding we always have to do is
go in and create that data dictionary first.
So even if you don't dual tag, because otherwise
we don't really know what tags we need to make
in Google Analytics 4. So that's something that
is just something that you're gonna have to do.
And it's a great way to work once you have that
put together too. Yeah, that's a great point.
And also there's some differences like We don't
have segments anymore in GA4. I use segments
all the time to kind of just slap that in there.
And then I've got all the filters ready. So can
you explain to us what segments were and what
are we losing? Yeah. And I mean, I think there's
like some slight ways that there are some ways
that we're doing it, but it's not as. Nice in
some ways. So segments and segmentation is when
you create a subset of the data to do and primarily
the way I use it is to do comparisons. So in
Google Analytics Universal, you could make one
of my favorites is a purchaser versus someone
who abandoned cart or someone who's a non purchaser.
And what you were able to do is that Google Analytics
had been a lot of static reports where you could
then look at those two segments or those two
groups of behavioral you know the behaviors or
people etc and compare them and say okay well
with my people who didn't make a purchase I can
see that the bounce rate is a lot higher and
they came from these different sources so what
are we doing with those sources like what do
we need to improve and so Now, they're changing,
and it's not quite as easy to do that side -by
-side comparison. You can add and filter, et
cetera, but at least right now, we'll see how
they end up changing it, because every day I
log into Google Analytics 4, I swear that they've
changed something else. But it's definitely going
to be a slightly different way of looking at
it, and I'm curious what it's going to look like.
Like I said, I'm familiar with Adobe Analytics,
and they've... they have a similar interface
and they found a way to do that. But, you know,
it's going to be really interesting to see how
they address that because that's a core function
of Google Analytics. Yeah, for sure. And also
in our company, the way that we are doing kind
of overcoming some of those is we're actually
using their API through Google BigQuery to, you
know, export all the data into our warehouse,
Snowflake, or any other database, and then you
could build your own segmentation and reporting
outside of GA's interface. One of the advantages
to doing that is you can match your GA data with
your internal data or third -party data, so it
gives you a lot more flexibility, but then it's
a lot more investment. infrastructure and all
that. But depending on what phase the company
is in, there are multiple options. It can be
a lot, but it's super interesting for anybody
that likes data and marketing and everything.
Yeah, certainly. And I think that's another good
call out, really, about GA4, is that at least
the way that they're currently explaining it'll
work, you do kind of need to use BigQuery. And
what that means is, yes, there's a lot of infrastructure
and everything, and then you need somebody who
knows SQL, and BigQuery's weird version of SQL,
too. So that's a whole other thing, where some
companies may have been relying on analysts that
come from a less like And this is not a bad thing,
by the way, like a less technical background
where they're really good contextual analysts
and business analysts, but they didn't need to
use SQL before. So now we're going into this
place where to get the full usage out of your
data that you were before with Google Analytics,
you kind of need to consider whether you're going
to upscale somebody, bring new talent in, et
cetera. Yeah, it's definitely a lot to consider.
and it's thrown on companies and they never asked
for it, but here we are. Yeah, we'll see. I think
it's a pretty interesting play, but I can't really
comment on whatever the master plan is for Google,
so I'm always interested to see what their next
move will be. Yeah. I mean, it's a great tool
that they offer for free, so I can't complain
too much, but definitely something that Yeah,
July 1st deadline, so make sure you've got your
stuff in place. Awesome. So I know there are
a few things when it comes to metrics and optimization.
In the digital marketing space, A B testing and
optimizations are super important. One of the
things that We talked about initially, once you
have your infrastructure, you need to have adoption.
One of the things that help with adoption and
also with the feedback is how you measure your
tests or what you're doing. So what are some
of your thoughts on measuring when it comes to
measuring the impact of your A -B tests or optimizations
that you run? Oh, this is such a juicy topic.
So there's, and this is a really fun one too,
because there's been, I would say, a divergence
in the A -B testing world. And there's kind of,
there's growth marketers and then there's more
statistics focused people. And my philosophy
is that. We need to try and make tests as scientifically
sound as possible. Marketing is already a whole
mess, and our data is already a whole mess, and
it's not as accurate. And you have to accept
a little bit that we're never going to get as
accurate as a hard science. But we need to be
unrelenting in keeping those elements that make
things more sound. So what a few of those are
is I think that any test should be pre -tested
so that you can calculate how long it should
run for for you to be able to see an uplift and
whether you even have enough traffic to test
that particular thing. So sometimes, and this
is especially for like small to medium businesses,
you want to... test on your conversion rate,
for example, that's always, you know, get this
term conversion rate optimization. But if your
conversion rate is only like one or two percent
and you don't get that much traffic, it's literally
impossible for you to see a true statistical
uplift during that time. So I think being really
honest and strict with that is like element number
one. And element number two is to never let a
marketer go in and look at that test before it's
done running and tell you whether it's winning
or not, because you need to wait. So if you pre
-test and it says it'll take seven days for statistical
significance, then you wait seven days, then
you look at the results on day seven. And if
you look at it and it says there's no difference,
then you know that you have rejected your hypothesis.
it doesn't make a difference. A lot of times
what we do instead is say, oh, let's just let
it run a little longer. So that's one core area
of it. And then I also think being really cautious
about those thresholds, too, as much as possible
of what do you need to see as an uplift? Because
that's completely another thing. If you see a
5 % uplift in conversion rate, That can make
a really significant difference for your business.
And it makes it, in some ways, a little bit easier
because you're like, OK, well, I only need to
see a 5 % uplift. But for some other metrics,
a 5 % uplift might be negligible. So you might
be wanting to see something more to consider
it a winning test. So those are two elements.
I think the other part that gets a little bit
more sticky, and I've seen it done a ton of different
ways, is how you calculate out the diminishing
returns of operationalizing a test. I know that
sounded like a lot of really big complicated
words, but what that means is, you run the test
on your website, it wins, you maybe even do some
other confirmations and you're really clear.
Whatever feature we added is a value add to the
site. Then you have to go through development
and get that put into place. And then you need
to evaluate, well, how much impact did that really
make? And it's such a fun thing because it's
not a perfectly controlled test anymore. And
like I said, I've seen it done a lot of different
ways. I've seen people play around with modeling,
like a diminishing return so that they'll get
that perceived uplift over a few months and then
it starts to trickle down. So there's been some
kind of advanced models like that. Some people
just make a guess of like a flat number. So that's
where like very interested to hear like your
listeners, if they have any opinions of how they've
seen it done that they really like it. But I
think those are some of the core elements that
need to go into testing that make a big difference
in the quality of what you're running. That's
yeah. There's a lot to unpack in your answer.
There's a lot of super useful nuggets. If I made
some notes, as you were talking, be scientific
in your approach. Test needs to be pre -tested.
So you define what your tests need to be and
then get a gut check for it. And you test on
volume and having some statistically significant
volume. If you don't have decent enough volume,
there's no way you could run a scientific test
on it. One of the key points that you said was
this is more behavioral. You need to let the
S run its course and not meddle with the lever
as soon as you see something. Don't be jittery.
don't react to anything that you see on day one,
day two. You've got to run its course. And especially
with marketing, and especially if you're doing
paid media and everything, it's not just what
you're doing, it's also the algorithms of Google
and the social media platforms that they need,
those need to learn what you're trying to do.
and then adapt. And so it takes a course of a
couple of weeks to even some months depending
on how complex the models are. I've seen this
so many times that there's competing interests.
One is everybody understands intuitively that
we need to wait. but there's always a board meeting
coming up or there's a month -end reporting that,
you know, you start off with the same mind and
then all of a sudden, it's end of the month,
you've got to push on your paid media spend to
bump up the production. So it's a lot of... It
takes a lot of patience and... courage in a lot
of ways to keep that going. So yeah, that was
a great answer, a lot to unpack there. You also
talked about diminishing returns and ROIs. There
will be a certain point that a successful test
will no longer give you the same ROI or even
if you keep tuning it out, it'll stop at one
point. So you've got to be, you don't get attached
to. success too much. So yeah, that was a great
answer. Thank you. What are some of the key things
to measure when it comes to Google Analytics,
and so you talked about funnel metrics a little
bit. What are funnel metrics and what are some
of the key metrics that we want to measure when
it comes to websites? We'll just take the website
for now, because that's a loaded question. Yeah,
sure. So I'll start with what are funnel metrics,
especially on any sort of... It's any site that
you have a goal in mind that you want a visitor
to do, but we can operate with e -commerce as
an example because it's the core one we usually
think about. A funnel is any time that someone's
taking several different steps on a site and
you're watching what those drop -offs are between
your key steps. Most traditionally, we think
about a shopping funnel. landing on a certain
page, going into your category pages. So that's
like where you have the long list of items, clicking
into products and then adding to cart. And then
there's checkout funnels that get really detailed
into the different steps that you have. So once
someone's in the cart and then they start your
checkout process. Where are there any certain
spots where you are seeing a drop off that you
don't expect? There's always going to be leakage
in any sort of funnel. But those, you know, and
the funnels can be broader than that, too. Maybe
there's other actions that you want people to
take or you know that they take in a certain
order. And you want to evaluate how each of those
steps are going and whether you're expecting
the kind of leakage that you see or not. And
like, when it comes to what to measure, what
metrics to measure. There's both kind of some
core metrics that I think are important for any
sort of website based business to look at. And
then there's the whole conversation about KPIs.
And I think that a lot of people interchangeably
use those two terms, but to me they're very different.
And a KPI is like, There's only a few of them
that you should have as a business, and they
are the things that you are completely informing
your strategy on. Generally, I like to do them
in a pairing. I can't take credit for that. That's
an Avanash thing. But what I would say some key
metrics are is, and this one's changing, so right
now it's bounce rate, but it's going to turn
into engagement rate. But what you're trying
to understand from this metric is, It's kind
of like that you gross principle. So does someone
go to your site and go, this isn't for me and
immediately leave and you want to see whether
that number is where you expect it to be. There's
always going to be some people that go. hit a
site and then leave immediately. But that's one
that's like a great health gauge of pretty much
any type of website. If you have a really, really
high bounce rate, then you might be running into
some issues where your site's not loading fast
enough. Something about the content when first
people come isn't what they were expecting. So
that's one of the core ones. Some other ones
that are on the opposite side of the funnel are
obviously transactions. Conversion rate, we've
already talked a lot about. The universal analytics
definition is sessions by transactions. So you're
looking at the ratio of the number of sessions
that are happening, how many transactions are
there. And then before that is add to cart. Add
to cart is a really core one for e -commerce
websites because it's like an intent to buy.
but it's still pretty far away from actually
purchasing. E -commerce sites and consumers currently
can tend to use your ad to cart as a wish list,
but it's one of the first intents to purchase
that's a little more serious than just browsing
products. I think that those are some of the
broad ones, but what one you focus on improving
is really specific to what your goals are in
the business. That makes a lot of sense. Yeah,
there's so many things that you could focus on,
but yeah, you've got to define your pathway and
then start measuring from there on. And also
for some of the most advanced companies with
advanced infrastructure, they also combine some
of the marketing data within their own internal
data and other third party data. You can match
your data from Google Analytics with your paid
media, and also your customer care, your email
metrics, and a bunch of different things, which
is where I start geeking out. In my mind, there
are kind of three levels. You have your campaign
metrics for any measurement, and then you've
got your engagement metrics, which is Campaign
metrics are what is the campaign doing, how many
visits, how many add to cart and all that. Engagement
is after your customer purchases, what do they
do? Do they use your product? How many times
do they use your product? How well or how many
products do they use and all that. And then you
have performance metrics, which ties to the KPIs
that you talked about. performance metrics in
my mind are aligned with your company's goals.
It could be acquisitions goals, retention goals,
and stuff like that. What are some of your thoughts
on it? Are companies doing it this way or is
it just a few companies that are thinking in
these terms? It's a good question. And I think
it's really important to distinguish them. It
doesn't really matter how you're going to slice
it. I think there's definitely a few different
practices. I like that one. It is just getting
that distinguishing factor between this is something
that is measured for this reason, here's who
looks at it, and then all the way up to this
is what we're being judged on for our bonuses
and for the board of directors is looking at
these numbers. I've seen it done quite a few
different ways. I've also seen it done really
divided by And this is one of the ways that I
divide it in a governance thing. I wrote a governance
article recently about this, like who is looking
at the metrics and analyzing it. So like your
definition that like performance would be like.
board of directors, C -level directors are looking
at it. And then there's some really tactical
metrics that an analyst is going to look at.
I've also seen people talk about when they need
to be evaluated, like how often. There's some
of those metrics that even the evaluation of
them should be automated. And that's really an
opportunity for optimizing them using machine
learning or things like that. Those are ones
that you don't really want to waste an analyst's
time on a day -to -day basis. there's another
bucket that's things that need to be evaluated
weekly or monthly or really monitored, and then
so on and so forth. There might be something
that's really worthwhile for a deep dive analysis
once a year. So there's so many different ways
to slice it, which is not like I'm like, here's
an easy framework. And to answer that second
part about, do I see companies using this? The
answer is kind of yes and no. I see a lot of
companies misusing some of these and really The
biggest frustration or pain point I see is, especially
on the leadership level, misinterpreting a metric
to a KPI. And they say, we have this KPI dashboard,
you want to see it? And it has like... 300 different
metrics on it. I'm like, none of these are KPIs
because you don't know, you know, there's like
some little cartoon about it that has like a
dashboard like that. And they're like, what is
this one trending down? It says, it's our understanding
of all these metrics. So, you know, I think that
that's like the pain point there is that I think
that there's different differing levels of maturity
about understanding some of these things. And
all companies are in different places, even like
in the same industry. But that's the one where
I see the most frequent misunderstanding. I see
very few companies focusing down their KPIs properly
by company -wide, function -wide, et cetera,
to really focus people's efforts. Yeah, that's
a great point. What you said kind of reminded
me of another framework that one of my mentors
talked about. You have your stuff that your executives
look at, and then your managers look at, and
then your technical teams look at. They're different
metrics, and you have to treat them differently.
And then, like you said, the time horizons that
they need to be looked at is different from each
other. That's a good point to remember. And also,
with this podcast, we're trying to make it more
accessible. This is more advanced and we don't
really, not every company really needs all the
infrastructure and need to be measuring everything
in order to be successful. You can start with
marketing, you can start with the final metrics
you talked about and then slowly develop. and
grow into the other KPIs and performance metrics
that we touched on. Yeah, definitely. I think
that sometimes people think that they have to
build it all at once and they have to be flying
their rocket ship when they're still waddling
around in their diapers. And there's so much
that you can do with just, say, the... data that
you have from your advertising platforms. You
can do a lot with just that, or you can do a
lot with just customer feedback. And I think
that's a really good call out that it feels sometimes
like everybody has to have their own data link
or data mesh and all these fancy terms and do
all these things and have machine learning and
AI. And the answer is you don't. All you need
to do is to execute on your data. better than
your competitors. And if your competitors are
other small startups, then really what that means
is you're leveraging market intelligence a little
bit smarter. looking at your forecasting a little
bit smarter. Yes, even really small companies
can be doing really smart forecasting to make
it a little bit more sustainable, to have the
right amount of inventory, to predict where they're
going to be in a year. There's things like that
that really you don't need that much data to
do. You just need the right people thinking about
it the right way. Absolutely. You also passed
on. misinterpreting metrics. That's something
that I've got a lot of thoughts on. But there's
two types of extremes. People who already have
a story and then they just look at the data just
to confirm or look at data with a lens that they
already have. no matter what the data says, they
have the same story. Call them wisdom analytics
versus on the other side, you know, executives
could be too jittery and look at a data set and
a result and just change everything that they've
been doing, change their course of action. So
I think it's definitely a balance. And like you
said, you've got to have the right frame of mind.
to write people looking at their data with the
right frame of mind is super important. Oh, yeah.
I think it goes back to that feeling confident
good about data as part of it. But then also,
yeah, creating a culture, whatever you decide,
it doesn't have to be the same at every company.
That's like, we will look at this honestly. be
realistic, and then also having people who can
really temper something scary for executives.
Because I think that's something that's sometimes
missing is that we go in and we start to be like,
we're going to be data driven. We're going to
do so much with data. And then you start giving
a lot of information to executives that aren't
familiar with it. And it stresses them out because
no one is being that translator between an analyst
who's going to look at it super objectively or
like really clinically to the executive where
they're like, what does this mean? And they're
jumping to conclusions where if they have like
a data leader, they're going to be like, well,
you know, here's kind of the nuances to this.
Here's where you need to make a decision. Yeah,
that is so much important. And that's something
you talk about and I'm super passionate about
too. Like I've started to write in my blogs,
like the storytelling aspect to data is super
key. And you got to have a contextual understanding
of what the data is telling you because this
same result or same set of data, 10 people can
see it and interpret it 10 different ways. But
you really have to have somebody who understands
the product, the company, and then the data as
well to tie it all together and not just an analyst
looking at objective analysis of the data. Super
helpful. I think that kind of There's a good
segue into our topic on leadership. At InsightLine
Analytics, you cover all aspects of data, infrastructure,
marketing, and even results and measurements
and everything. How do you assess what stage
a client is at and what your work is going to
be? When you go in, how do you assess? That's
a good question. I think there's a few different
things. One that we're currently actually working
on to be even more measurable in that is creating
a data maturity model. There's a few digital
maturity models out there, but there's not as
many super focused on data. That's one thing
that's to come. soon. So I'm kind of like letting
the cat out of the bag about that. But as experienced
people in this field, it's something that like
myself and like the other senior analysts can
see right away. There's like a few key aspects
that indicate where someone's at. And so that
part of it's coming from experience now where
I kind of know, depending on even what a company
is approaching us with. where they're probably
going to be at in their maturity. And then we
start to get a little bit more into the details
as we get there. Of course, you don't want to
make any snap judgments of, wow, this company
is really immature or really mature. But there's
some indicators depending on, well, what are
they valuing right now? What's really important
to them? What are they super focused on? And
then There's also sometimes where we go into
this a lot deeper, which is we'll conduct an
entire data strategy project where we're going
in and mapping out their marketing and data tech
stack and comparing that to what we consider
industry standard. And we're evaluating like
their whole team structure and doing some needs
assessments of like, okay, well, they're here
and they need this, but they don't have it yet.
And so that's like the really comprehensive way
that we dig in. to it, especially with bigger
complex organizations. And that's where we get
a lot of information and are able to hold up
the mirror for an organization and say, well,
this is what we see as far as where you're at.
So we have some pretty detailed ways to do it.
And then there's also these initial, what is
someone asking us for? There's a very big difference
between a company that's asking for just like
conversion rate optimization for their site.
Hey, we really want to get more sales and we
want to lift our conversion rate and make way
more money. That's a very different level of
maturity than someone who's coming to us and
being like, We already have unified customer
records, and we really want to do some cool stuff
with that. We want to start doing machine learning,
and we want to apply that to our advertising.
Those are two very different conversations, and
there's nothing wrong with either of them, but
it is a big indicator of how mature they are.
Absolutely, and it's super important for us as
data experts and marketing experts to go in with
an open mind. You touched on that. You don't
want to shame anybody for where they are. And
everybody goes through the same process. People
are at different phases and different points
in time. So I think, yeah, definitely keeping
an open mind and understanding where they're
at will be super key. And also from a data perspective,
you did touch on this, like you have companies
running reports off of Excel sheets. Yeah, that's
one end. And then you next go into having a marketing
infrastructure in place, data infrastructure
in place, where you can get some of those funnel
metrics. And also next, you could probably look
at BI and automating reports with visualizations.
So at this day, you can talk about looking at
historical data and understanding patterns. Hey,
what's my seasonality of the sales or retention
or stuff like that? So pattern recognition, this
is where I think my opinion is the biggest ROI
because running reports off of Excel or just
looking at one or two metrics wouldn't give you
a larger perspective of to be able to get those
patterns, recognize those patterns. So BI and
visualization is super helpful at that point.
And then you go into predictive. So until now,
you looked at historical data. Now you can start
building models to predict what will happen in
the future based off of what has happened in
the past. And then you also have these other
phases where you have machine learning models,
you've got your data science and AI models. So
there's various phases to data maturity. So that's
super key and super important. Oh, certainly.
And I think for people who are listening, too,
it never happens just quite that linearly, too.
So that's a good call out. I've known companies
that I would think are fairly mature. They're
starting to do some predictive stuff, and they're
getting playful with that. And they still have
a few Excel sheets they're using to report to
executives. So there's one thing there. But also
would agree with you that that's where advanced
data science kind of stuff starts to have a good
ROI is always predict. I'm always going to say,
do predictive stuff. And do predictive stuff
that really influences the business if you're
not predicting. And you can do some lighter versions
before you even get there. Like, you should be
forecasting. Even if you're still in Excel, you
should be forecasting. But getting to the juiciness
of churn prediction or things like that is really
where you can highlight issues before they happen,
and take advantage of opportunities before they
happen. Yeah, that's a great point. And also,
yeah, you have to be understanding of where you
are. And not everybody has to do everything,
but you have to just know where you are. And
also, one of the things that you talk about a
lot is be customer centric, even if you're working
in a corporate environment, look at the business
needs and then develop a solution and not have
a solution in place just because you know how
to do it, then there's no use for it. If the
marketing team is not ready for it, then you
can't really, I mean, I can't build a CDP solution.
if the marketing team is still not ready to use
event -based data and event -based marketing
yet, if they are just sending out onboarding
and retention emails and you're talking about
engaging customers where they are and looking
at the different pages of the app that they're
looking at and changing the changing what you
show based off of that. So basically, event -based.
You've got to have your customer in mind before
developing solutions. Sorry, that was a long
-winded thing, but yeah. Definitely keep your
customers in mind before developing your solution.
We talked about infrastructure and data maturity.
How do you handle data adoption and the literacy
with your clients? With difficulty. No, just
kidding. It's a long game. And those two things
can be their own whole project strategic function
of your company. it does take some time. And
that's one of the reasons I really love when
we have these opportunities to stay in a company
for years and really help them develop this.
But there's the two aspects. One, like you said,
data literacy is something where I'll define
the term really quickly, too, because it sounds
kind of like, one, sometimes I feel like people
could be like, well, that's insulting. I know
how to. look at data or whatever. And then also
it sounds fancy at the same time of like, it's
another term like strategy or synergy or something
like that. But what it really means is like,
do people understand the data that your company
creates? Do they feel comfortable reading it
when it's presented to them in a deck? And do
they trust it? And do they feel comfortable using
what's coming out of their team to justify what
they're doing? And so that can have a lot to
unpack there, right? Because you need to evaluate
first, like, what level of data literacy does
everyone need? How will you train them? Because
it's something that does require some training.
And then the other parts of it, like I said,
trust in data and understanding of what you collect
and why, kind of bleeds into the second part,
which is about communication and adoption of
data. I think that communication of data quality
and governance. Everything like that is something
that should be operating at the highest level
of the company. If possible, owned by a whole
group like governance, for example, governance
and maybe communications tied together actually,
because it's where things can fall apart and
also where you can get the strongest message.
So this is something that If someone wanted to
take a script and do best practice, that's what
you would do. Because without it, you have a
really common scenario, which is that there's
often conflicts between departments, and data
is used more as a weapon than it is as a tool
to get to understand each other better and your
customer better. Just super unfortunate. One
of the ways that you get around that is by building
a culture where you and common agreement about
all the data. So making sure that you're really
evaluating what is our source of truth? How does
everybody use that? What are the agreed upon
terms we use as a company? And how do we validate
that and make sure that whatever we're communicating
all lines up to it? And that's so much easier
to do at a really high executive level and then
get buy -in and communicate it over time. All
of that I'm saying, it takes years to develop
properly. So I summarized it and said, that's
all you got to do. Ta -da. Now you have this
great company and everybody loves each other.
And there's no fighting between your IT team
and your analytics team. That's not necessarily
the truth. But it's something where it's like
an iterative process of evaluating, where do
we have gaps? And one trick would be wherever
you see conflict, like frustration. you hear
a repeated term like, I don't trust that data,
that's where you can dig in to really understand
what's going on. Yeah, that's super important.
Yeah, definitely top -down approach is a better
way to go. And also you kind of identify the
pain points and areas of pain points and then
you dig into it. to understand how you can get
better adoption, you know, solve some of these
data quality issues or data issues to get better
adoption. Awesome. So we talked a bunch of about
data. Your main role as a founder of your company
is to lead your team and You talked a lot about
working with the clients and everything. When
it comes to managing the team, I believe you
said you run a six -member team. You're a couple
of years old as a company, so that's a good size.
Is your team remote or do you have a hybrid model?
You're in Colorado, correct? Yeah. We like to
say we're a remote first company, but we actually
do have a headquarters in Durango, Colorado,
and we have four of us out of the office fairly
regularly, but it's entirely up to each employee.
We have some that are... We've actually had some
shifts recently, but we had a few in Texas, and
now pretty much all of it's moving to Colorado,
which is interesting. And that's not on purpose
or anything. It's just kind of how it's happened.
But that's the model is that there's some of
us that like to come into the office and live
here, but also it's not necessarily like a major
metro. So I don't have any expectations for people
to relocate or anything like that. get together
at least once a year, as long as everyone wants
to. If someone said, nah, I just don't feel like
doing that, we wouldn't hold it against them.
And then we do a little bit of travel to our
clients when it's really necessary and they really
want us to. But we started the company in 2020,
so really at the start of everything going remote
for work. And it seems like it's a model that
really works well for our team. Yeah, definitely.
startups and especially in data and marketing
and some of these areas. Remote work is definitely
feasible and also works really well. How do you
handle onboarding and building team culture and
all that when people are scattered around the
country and the globe? It's definitely a bit
more difficult than when you could just get someone
in the office and have them meet everyone in
person. There's a few different things we do.
We do actually try our best to have people's
first week start in the office. So we'll fly
them to Durango and so far we've done that with
everyone and that's been a really good kind of
start. That doesn't mean that everyone else is
going to also be at the office at the same time
now that we're getting a bit bigger, but it does
give them a little bit of time to meet with their
manager and really get to know the company. And
then the other parts of the culture is I think
Something that's really key when you're doing
a remote first company is to intentionally create
a culture. So we've done several iterations.
We work with a business coach that also helps
us with this, but we really look at like, well,
what are we now? What do we value? What are we
currently doing that we don't like and we want
to stop doing? And so we really have intentionally.
Decided that and I think that is always going
to be colored by who the founder is but I'd like
to think as it gets bigger it moves away much
less from just like who I am as a person and
it becomes a bit more of like who we all are
as a person. But, you know, I think that that's
that's part of it was like I had some core elements
that I wanted. I wanted, you know, people who
are interested in leadership and analytics. And
so that's like a core function of our culture.
We also wanted people who were really focused
on radical candor is like. the fancy term for
it, but like transparency, being really open
with each other. And people who really love what
they do and like are interested in it. And like
those are kind of the three, three of the core
pillars that we built the business on. And I
think we look for that when we're hiring and
we also look for people who are going to be a
culture add. And And then it's kind of like up
to everyone else, like how that's been created
since then. Like, there's a lot of things where
it's like, I realize now it's like a living breathing
thing. And I see things that are part of our
culture now that like I never intended that I
really like. And so, you know, but they're, I
think, starting with that intentionality really
helped. That's awesome. Yeah, me being having
some. goals in mind about the culture will definitely
help in one, expanding that culture and also
in hiring. You want to see that people coming
in are a good fit to the team that you already
have. That's super cool. Yep. I know you work
some in the corporate world and also you've got
your own, run your own business now. What are
some of the pros and cons of the corporate world?
I was hoping you would ask me this question.
And I talk about this a lot. And I'll also answer
it in the vein of like the difference between
consulting too and being in house. The pros of
being in corporate is that there is a lot of
upward mobility and possibility in the same company.
So if you really love a company, you have a lot
of different places you can go. And you can learn
a lot about politics. And I know that a lot of
people think about that word and they're like,
oh, I don't want to do company politics. But
you this is a hard truth. As you get higher up
in a company, if that's really what you want
to do, you have to play the game and you can
play it in a honest way and really be using it
correctly, but you have to be advocating for
yourself and your team, etc. You can't really
learn that as well when you're either a consultant
or working at a really small company. It's just
a different dynamic. Some other benefits is that
you get a lot more budget to play with and you
get to see things through to the end. If you're
a consultant, you pop in, you make your recommendations
and you just hope that they'll take some of them.
in -house you really get to see long -term following
through of projects, getting that satisfaction
of seeing the results and learning from things
that don't go well. Cons would be kind of the
almost the flip side of some of those things.
If you are someone who really wants to be entrepreneurial,
it's possible to be entrepreneurial in a company,
but it really depends on the company and how
they receive that. As someone who has always
been that way, it's been received well sometimes
and sometimes not so much. And so it can be something
where it causes friction if you're someone who's
really innovative in your ideas. And over time,
that can get stifled is what happens. If you
do choose to stay in corporate, you can start
to get a little bit comfortable. The other thing
with corporate is if you stay in the same company
for a long time, you don't get to see as many
different ways of doing things. If you're a consultant
or if you maybe change or switch between corporate
and smaller companies, you're going to see a
lot of different personalities and different
industries. completely different ways of operating.
And all of that can add color to your experience
moving forward. So I can probably go on, but
I think those are some of the really core elements
between the differences. And what I've found
is I really like working as a consultant for
corporate companies because I still get to enjoy
a lot of those pros that I really like. But then
also people are a little more accepting of whatever
wild ideas I'm coming in with, because they're
like, ah, she's just a consultant. So that's
at least where I found my sweet spot, but I think
it really depends on the person. Yeah, it really
depends on the person and the company that you're
in, like you said. You touched on a couple of
things that were interesting to me. I mean, the
politics can be either pro or a con, depending
on how you play it, or also how you don't like
it. The way that I think about it is just dealing
with people. And in the corporate world, you
deal with the same people for a long period of
time. So you've got to have that long -term mindset.
Hey, if you have a couple of projects, you lose
some, you win some, but... when you're in the
consulting world, it's more about the customer's
needs first, and you've got to make sure that
they're comfortable with everything. And also,
you touched on people who are entrepreneurial,
how they manage in the corporate world. One of
the things is, like you said, depends on the
company. One of the things is smaller companies
are usually open to give you more flexibility
versus big tech or really large companies. Most
of the times, they fit you in a box and then
your role is defined. You do it and unless you
move to a different role, you can't do too much
about it. Following up on our previous question,
what can startups and corporations learn from
each other? Another one that I think is really
fun to think about, I think that startups can
learn how to stick to a strategy set strategy
and stick to a strategy from corporate companies.
I think that that's a key element of startups
that are more successful is if they're able to
get at least a bit close to the level at which
some of these really big corporations are able
to choose a strategy and really structure everything
they do based on it. That's something that startups
can certainly learn from corporate. I think that
corporate can learn from startups that even though
they have this whole established way of doing
things. sometimes you can blow that up. And you
should, and you should listen to people who have
unique ideas and challenge the status quo. And
that's where you're going to be able to innovate
compared to competitors and be a lot more flexible.
And that's one thing that you see with corporate.
Like right now, I mean, we even see it in the
kind of closing and Bed Bath and Beyond struggling,
for example. Companies that fail to get excited
about whatever is happening in the world and
in the business world and like play around with
it and try new things even if they're still executing
within their strategy are the ones that succeed
and ones that really don't adapt fast enough
are going to fail and startups know that. Startups
always know that they need to be agile and that's
how they out compete. corporate companies sometimes
too, is that they're more willing to like go
bang into something new and they can pivot really
quickly. So I think that those are some of the
core areas that can really help. And I think
the other part that you kind of touched on a
bit that I'd like to pull into is even though
corporate wants to have their boxes for every
one of their employees, Allowing for some like,
and I forget the term, maybe we can post it in
your podcast notes if you remember it, but it's
like when you're an entrepreneur within a company.
And I think that you should encourage that. Intrapreneur,
you're right. Yes, thank you. Yeah, so you should
encourage some intrapreneurial activities, whether
you have innovation functional groups and things
like that. That's something that you don't want
to lose. the fun and creativity of that, even
if you need a lot of structure in your company.
Yeah, no, that's a really good answer. Yeah,
both sides can learn a lot from each other. And
it's depending on where you are as an individual,
you could choose one or the other or switch between
the two. Awesome. Let's dive into some of the
challenges in general. What are some of the challenges
you face in the digital marketing world? We know
we talked about GA4 and all that. So what are
some of the other challenges or if you want to
expand on GA4, that's cool too. Oh, well, it's
kind of more focused on digital marketing since
that's how you phrased it too. I think some other
challenges are How to properly show impact and
understand what's working and what's not is like
one of the biggest pain points of marketing ever.
It's not specific just to digital marketing,
but with digital marketing, it actually makes
it almost harder and more frustrating in some
ways because we think that we know. a lot more
about how the uplift is and what it's doing for
us, but it can really misguide us. So that's
a huge pain point is that with attribution, which
I still think is important and we need to figure
out how we do it, but the way that attribution
has been It's so hard to discern. We're rewarding
digital marketing over a lot of other things.
So digital marketing, we just take the whole
pie and we say that digital marketing has gotten
all these slices for where all of our traffic
came from. But if you think about the big picture,
that's not true at all. There's returning customers.
There's like your brand presence, there's all
the traditional marketing that you're doing,
you're not actually factoring for any of that
when you look at that pie chart in Google Analytics.
So that's one that is really difficult and it's
really difficult to properly explain to marketers.
One of them you touched on at the beginning of
this was about deprecation of cookies and additional
privacy. restrictions and laws that are going
into place. This is something that is difficult
because we have become accustomed in digital
marketing to operating in a certain way. And
the industry is really flexible, so I think that
they always will adjust. But now what it does
is forces companies to be more strategic about
how they collect data. They can't just collect
whatever they want and use it however they want
and store it however they want anymore. Now there's
laws holding them accountable. And then also
they have to be a lot more intentional about
gathering the data that they want. It's not just
going to fall into your lap. If you really want
to know about your customers, you're going to
have to ask them. And convincing companies to
do that is another one that's difficult where
it's like, hey, if you want zero party, first
party data, this is what has to happen and it's
a little more complicated than it has been in
the last five or so years. Yeah, it can definitely
get a little more infrastructure heavy. You want
to be able to allow the infrastructure to, say,
delete data upon request or build some more sophisticated
tagging mechanisms so that you understand where
what part of your funnel you're tracking and
all that. So it definitely is evolving as we
speak. So it's an interesting time to be in the
industry. What are some of the challenges in
the consulting world? Oh, there's a lot I think.
Finding clients is always going to be a difficult
thing. Finding the right clients that like what
you do is like when you first start out in consulting
and first establish yourself as an agency, it's
like, it can feel like an overwhelming task.
And, you know, I think, so that's like kind of
more of a tactical one, but I think another one
that's really relevant to what we're talking
about is like balancing recommendation. understanding
that you don't have the wealth of knowledge that
someone in -house does. There's pros and cons
to that. Part of it is that you can see things
they can't, and on the other side, you don't
know all of the reasons why they are the way
they are. Being a good consultant and really
supporting your clients is actually really fun
for me. I don't know if everyone would agree,
but it's really fun, but it's also very challenging
because you're always trying to be like, well,
I want to recommend things, but I want to make
sure that I have enough context to do so. And
then you also have to let go of any ownership
of what happens next because you might not be
there. And even if you're going to be there,
like sometimes we're running data strategy projects
or implementation, we can still only do so much.
We can create the project plan, we can highlight
any pitfalls, we can really try and drive things
forward, but in the end it's the company's kind
of responsibility and choice to like... how they're
going to do that. So I think that's something
that's always really challenging is to not get
overly invested or feel frustrated when someone
isn't taking your recommendation. But I also
have been told by someone else who's really experienced
with consulting that you definitely don't want
them to follow 100 % of your recommendations
because then you're 100 % responsible for what
happens. So it's just like hoping that they take
the pieces that are going to be most useful to
them and that'll make a difference for them.
Yeah, no you're just on a really some some really
important points there Consulting versus in -house
you definitely As a consultant you don't want
to be Arrogant going in saying hey, I'm a consultant.
I know what I'm talking about. Yes, you do know
your area of expertise, but you don't have a
lot of times you don't have the contextual knowledge
on why Things are the way they are And also,
we talked on this earlier in the podcast, but
a lot of times the data does not give you the
entire picture. There's a lot of history and
stories behind why operations are operating the
way they are. So it's really collaborating with
clients. to understand as much context as you
can is super helpful. And also letting go of
ownership. That's I love that because if you're
an in -house employee of a company, then you
can own a certain piece of the business for a
long time. But as a consultant, you build it
and you hand it over and you're more like a parent
needs to let go. Yeah, really cool. What are
some of the criticisms you have from the marketing
side about data teams and what can we do to improve?
We can take a jab or two, give it to us. Yeah,
I guess my first jab is that they're not big
enough often. And I guess maybe that's a jab
at leadership instead. So I'll have some specific
jobs at data teams as well. But like we spend
too much money on tools and not enough money
on smart people. I think another job is that
we sometimes focus too much on super technical
data scientist type people. And that's really
a trend right now is that data scientists sound
sexier. than an analyst, but they're both valuable.
And if you're a smaller company and you had to
pick one, I would pick an analyst any day because
you need people who understand both parts or
multiple parts of the business. You need someone
who can speak the language of IT or really understand
a marketer when they're talking about a brand
concept. Because like what you just said, there's
so much additional context that needs to be layered
with the data. And so I guess my next like punch
would be that oftentimes on larger organizations,
I don't think that I see enough leadership roles
with data. Like I think that a chief data officer
needs to become just kind of like a standard
part of the C -suite. And I don't see that. And
I also don't see that many, you know, director
level and above people advocating for it. I don't
see I don't see enough hunger in our current
data teams to do that. I think that we sometimes,
and here's my kick now, is that we act like order
-takers, where we should be the ones driving
the conversations instead. I think those are
some of the ones. I'm sure I could keep going,
but I think those are some of the most important
ones that are on my mind today. Awesome. Yeah,
definitely data teams need to be more proactive.
And because we see a lot of... data from a lot
of different angles. We have different perspectives
and somebody that's in operations and in the
business and who has developed certain blind
spots for those data teams can definitely, like
you said, data leadership needs to become more
obvious. chief data officers and a VP, senior
vice president, executives at the data level
who can tell a good story and be a bridge between
the business and the backend teams. It's really
important. You made a great point on companies
hiring data science folks because it's way more
sexier. And yeah, that's so true. Because we
talked about data maturity. So you have to get
to that level of data maturity to start hiring
data scientists. So you definitely need to have
analysts first. And then if you want to build
out your infrastructure, go hire data engineers
first to make sure that you build out the infrastructure.
Because 80 % per I wouldn't say 80%, but 60 to
70 % of the job is to get data from different
places, massage it in a way that they can work
well together, and then present it in a way for
analysis and model building. So that's all that
the data engineers do. And then you hire a data
scientist or machine learning engineers. So it's
really about data maturity, but you're spot on
when you say, you know, a new executive comes
in and he's like, hey, why don't we hire data
scientists, you know, PhDs and all that, but
they don't have what they need to, you know,
be good at their jobs. You're setting them for
failure. Yeah. And I think also, we misunderstand,
maybe this goes back to data literacy a bit,
the difference between a data scientist and an
analyst. And I think sometimes they hire a data
scientist and then they expect them to do an
analyst's job. And this PhD is like, look, that's
not what I do. And they're like, well, why won't
you just pull these metrics for me? And so yeah,
it's another one where. I don't know if it's
maybe like a branding issue of us, of our different
roles or something like that. But a lot of times
I've seen like a job description where they're
like, you know, the title says data scientist
and I read it. I'm like, you just want an analyst.
Or, you know, they also, or sometimes they even
have the description correct. And then that's
not what they ask the person to do at all. Exactly.
Exactly. Speaking of, I mean, kind of of moving
towards digital marketing again. Digital data
can never be perfect. We talk about regulations
coming in place and all that. Even without that,
there's always gaps. In a lot of people's minds,
digital marketing data is infinite and you have
everything you need, but it's not the case. There's
a lot of gaps. I could pop on the website and
then leave the session and go on my mobile phone
and log in. So it's really hard to tie those
two together and understand that I'm the same
person. And also it usually is directionally
correct. So how do you handle those situations?
How do you explain those to your clients and
executives? That's one where that like communication
and data literacy comes in again. It's one of
the first initiatives I almost always pick up
with a new company is saying, all right, well,
let's for real evaluate what our discrepancies
are. Let me show you what the industry averages
are too. And let's have some common language
about why that's actually OK. Because the average
discrepancy between, for example, Google Analytics
transaction revenue data and your back end is
usually like 20%. That's like a normal number.
sometimes people hear those numbers, or also
this is where these conversations or issues and
conflict comes up as someone goes, well, I can't
trust Google Analytics data because it's 20 %
different than the financial data. And the financial
analysts are like, how dare you use that data
to interpret anything? It's not accurate. But
the thing is, it's usually consistently 20 %
different. So you actually still have a directional
change. If your revenue goes up, it's going to
go up in both and it's going to be pretty proportional.
So it's like a long, longer term game. But the
first thing that I usually do is create like
a one sheet of like, here's where we're at. Here's
how we're going to keep being accountable about
this. We're going to check it every quarter or
something like that. And here's what it really
means. And then it continues to be like little
pieces that I build on as I present to a client.
I'll explain that often and keep going back to
the point like, this data is valuable for these
reasons. Here's why we should use it. And then
I'm also someone who really understands the other
side. I really love working with finance teams,
really love working with more advanced. data
engineer teams that are really starting to do
some more advanced things that get you a little
more closer to accuracy. So I totally understand
where they're coming from and why they're like
that. like to also honor when an analysis actually
does need to be done in finance data, for example,
because there are those differences. And so starting
to educate on that and make sure that we are
playing on the same team really helps because
establishing that agreement really helps. Sitting
down with finance and being like, look, I'm not
trying to step on your toes. Here's why I do
what I do. And here's where I'm going to focus
on. But if it goes into this field, financial
forecasting for the whole company or calculating
profitability or cogs or something like that.
I'm going to put that over the fence to you,
but we can both benefit from each other. Yeah,
absolutely. I think you touched on it. It's really
about setting up expectations. Finance and accounting
data is super accurate and that's what they are
paid to do, be accurate. in the digital world
and a lot of data, you know, customer care or
email marketing, all these data is mostly directional
and as long as you have those deviations and
expectations set, like you said, I think as long
as you're measuring the same set of data the
same way over time, you wouldn't know if there's
a directional change. or a direction of shift.
And that's when you can make decisions based
off of. And I think I heard somebody say, it's
all OK as long as you're consistently wrong.
Yeah. Yes. Yeah. I really like that. And it's
very true. And I think that sometimes discrepancies,
I even have a blog post about it, can be like,
a paralyzer for actually doing things and an
excuse for why you're not doing things. And I
would rather have data that I think is good enough.
And good enough is exactly that. It's good enough.
And be making a bunch of decisions on it and
looking at how that impacts things and driving
things forward and really having good conversations
too about, hey, what does this mean? How do we
want to approach this? versus choosing to do
nothing or something I've also seen companies
do is go so conservative that they put all the
power into finance or into an engineering team,
where then it takes them so long to do an analysis
that it's useless anyways. So it's really like
the business world moves fast and we have to
accept that we... know something, but we don't
know everything. And it's better to be imperfect
in what you're using than to just be guessing
completely without data. Yeah, it's really a
balance. Like we talked about earlier about being
scientific and knowing exactly what you're doing
in terms of your hypothesis and testing. But
also on the flip side, you don't want to be You
don't want to wait too long. You don't want to
wait six months to get the result of a test.
Like you said, the business world moves so fast
that it becomes stale at that point. That's so
true. I'm glad that you called that out because
some people might listen to this and be like,
oh my God, she just totally contradicted herself.
But yeah, it's really, and I think that the way
you do that is keep the business focus in mind.
It costs me practically nothing to pre -test
tests and be strict there. It's so easy. It doesn't
add operational complexity or anything like that.
But on the other hand, running all of our analysis
through a database that only finance has and
you have to ask for it instead of querying it
yourself or whatever the issue is, does add operational
complexity and makes it harder for you to do
your job. So always thinking about where's that
balance and what level of control needs to be
there because it also depends a bit on where
the data is going, what you're using it for.
and those two things do coexist because you're
always wanting to make it as accurate as possible,
as make sure your tracking is as clean as possible
and the data is clean as possible, but to a point.
There's a point of diminishing returns. You want
to get to that good enough point and then just
let it go and check it periodically, maintain
data quality, and then really push hard into
using that data. Absolutely. It always comes
down to balance. The business world is so nuanced
and there are a lot of times opposing interests.
So you just have to keep a balance so you don't
miss out on opportunities as well as not take
drastic decisions based off of wrong or small
set of data. in significant data. It's been a
long week, Mary Beth. It's only Wednesday. I
know. I agree. Cool. Cool. Cool. All right. Let's
talk AI and Chan GPT. We're in a revolution right
now. I am invested, but I have a hard time keeping
up with everything that's going on. I just hope
that there's a lot of value to AI in this thing.
I don't like all the hype around it. You've got
to temper that down and make sure you provide
value. Like you said, have customers in mind
and not just AI for AI. What are your thoughts
on current advancements in AI and chat GPD and
everything? Well, I have to say I laughed because
I'm in this group called Restaurant Technology
Network where we workshop some things and we
have this game in most of the working sessions
of like, how long can we go before someone says
something about chat GPT? So it took us quite
a while before I got there. But yeah, you're
going to love I just actually finished writing
another article that's going to go on our website
about this. But I agree. I think my number one
statement is don't believe the hype, but there's
some pretty interesting stuff going on. And another
thing that I say as someone who's been in the
data world for a while is like, well, like some
of the ways that people are using the term AI
is really just like kind of predictive analytics
and like, it's not that cool. Like, so yeah,
I think that there's some applications that are
going to totally change the world, I hope. in
a good way. But I also see in the business world
that it seems like there's a lot of new startups.
And I was just at a conference in Vegas a few
months ago and really saw that it was just like
people are really capitalizing on this change.
I mean, I saw it a bit when like like cookies
started to be deprecated and things like that
too. There's a ton of people saying how they're
gonna still be able to identify your customers
all these fancy ways. So like happens every time.
But I think for businesses, but you know, might
be listening to this. My advice would be, yeah,
think about what could really impact both your
customer. and the operations of your company
first. Like what will save time of smart people?
You don't want it to use it to replace smart
people or to replace specific jobs. What you
want to do is make it so that those smart people
can use even more of their time doing smart stuff.
Like data scientists are a perfect example or
like engineers. What if you could use AI to do
80 % of the data cleansing and data quality checks
that you're currently doing. Like imagine how
much more time would be freed up to actually
analyze data and actually make business decisions.
So, you know, I think, I think that's like where
I see is, is like, and I, and I have these, like
my new post is about like the truths of data,
like hard truths of AI and you know, that touches
a few of them. It's just like, I always see this.
We get all excited about stuff in the business
world and we get all worked up about it. Sometimes
it's a big deal and sometimes it's not, but we're
super adaptable. I think that there's some interesting
things, but they're not really like what I imagined
AI would be. If you had told me that AI was going
to become a hot topic in 2023, and you asked
me that a few years ago, I don't think I would
be describing something like chat GBT because
to me, it's like, It's just like the next generation
of Google search. Let's be honest. Yes, it has
a lot of things colored into that, and it helps
you do things, and it's really interesting, and
there's so many applications. But really, I'm
excited for what comes next, too. What is really
going to be pushing the envelope? Because I think
it's a long time coming, and there's been pieces
of it that we've been using for a long time.
Yeah, absolutely. Yeah, and so some of these
large language models are definitely a shoe to
leap in technical advancement, but for the business
user, it's more of a more sophisticated chatbot
in a lot of ways. Right. Yeah. Right. And there
are these other advancements like auto GPT, which
you don't have to prompt it every time. You just
give it a goal and you ask it to book a hotel
and go look at different hotels and look at time
slots and then book a hotel for you and stuff
like that. So there's some advancements there,
but totally get your point on. We definitely
have to allow it to mature and see where it goes.
And also you talked about the hype cycle. You
know, I think a lot of ways, I was at South by
Southwest and AI was a big topic. And last year
it was Web3 and crypto. So it was like replace
Web3 and crypto with AI. That was kind of the
theme this year. So I don't want that to happen
that, you know, a lot of companies trying to
capitalize on it, say AI for everything and then
just people lose interest in it because there's
a lot of ways that this is useful. Yeah, I guess.
I'm less impressed with crypto, but I agree.
It's something where I really do think it's the
start of something new that's really going to...
And I, at least in my hopes, and maybe I'm a
bit idealistic, you know, there's some people
who are all doom about it, but I don't really
think we're there yet for it to be really doomy.
But I have concerns about bias and models and
things like that. But I think there's things
like the healthcare system and global warming
and things like that, that we can really transform
the world with this. So I don't want people to
lose interest. And so I think it's for people
like us continuing the conversation in a really
pragmatic way and starting to show, here's the
real benefit. Here's some tangible ways that
make a lot of sense. It's the same thing with
predictive or prescriptive modeling. It's something
that's so useful. So I'm sure that we'll identify
pretty soon what those are in AI. And then making
sure they don't go away because we can see the
return on investment. So I'm super excited about
that. like you said, we're just still scratching
the surface there and there's a lot of maturity
that we'll need to develop. But I also think
what you're saying about companies, part of what
I'm saying to people is like, please don't hire
a see like a sea level like chief AI officer
right now like I think it's a good time to start
to understand that starts asking questions of
people in your company already and if you don't
have that chief data officer whatever you want
to call that get that person first because that's
more of like a holistic picture of everything
data and so I think that if people are excited
about AI and think that's the direction their
company wants to go and if you think about some
of the more advanced more data mature companies,
they do use either machine learning or just really
data -driven. I heard a talk from Stitch Fix
at Shop Talk this spring. Even though they're
an e -commerce business and things like that,
they just have data as a core way they function.
If you want to go that way, like assessing your
whole data maturity and taking an honest look
at yourself is a good first step because then
you're going to really know what is the right
decision when it comes to AI. No, that's a great
point and I've been thinking about this a lot
with a lot of talk about AI and data. What is
AI and how is it different from modeling or data
science aspects of this thing. I still have to
think more about it, but the way where I am right
now is data science is more about extracting
intelligence from your existing data, historical
data, and maybe predicting what will happen,
but based off of what you have. AI, at least
in its current iteration, is more of a productivity
tool. that you can come and use it as a plugin
to take your productivity to the next level.
I would really agree with that. That's kind of
how the way I would also think about it and had
chewed on that a bit myself. So yeah, I would
agree. I think it's really like there's ways
that we can apply it to make things easier for
ourselves. But it's not necessarily like I've
seen some startups do things like, look, you
can use this kind of AI chat bot thing and like
ask it questions and it'll answer like it was
your analyst. It'll pull data from like your
data sources. I'm like, oh, like I or like that
ones where they're like, oh, I'm going to replace
your copywriter with AI. I'm like, like, you
know, I think yes, if you really wanted to, but
like, isn't there like, why can't you Why don't
you make things 100 times more efficient for
those different people and make it so that they
can spend even more time answering your business
questions and writing good copy versus some of
the other more menial tasks that they need to
do? Yeah. And then this goes back to our previous
point on having context around what the data
is saying without those subject matter experts
in there to provide some context or understand
some context. these AI models can't really, it
can derive way different interpretations than
what it really should be or what the data is
actually telling you. Yeah, definitely. You need
to have that institutional knowledge for copywriting.
You can't just automate it. at least not in this
iteration without trading the model. Very cool.
And I'm not just saying that because I don't
want to be out of a job. No, like you said, it
makes copywriter's job a lot more easier, but
it cannot replace copywriter's. Yeah, it's like
the creative thing that like, you know, we are
using it to kind of generate some art and things
like that, but it's not necessarily like ever
going to have that same raw energy and like ideas
that like a person is going to have. Yeah, you
could, I mean, I'm using chat GPT to write some
of my blogs and stuff like that. The way that
I think about it is if I can't think of any ideas,
I'll use it to you know, give me a head start.
If I have an idea and I am like 80 % there and
use it to kind of fine tune it. So both sides,
but it cannot, it can give you the 20 % edge,
but you still have to do the 80 % of the work
that you need to do. Otherwise it's, I mean,
I could probably just ask, give a topic to ChatGPD
and ask it to write an article, but it'll be
so robotic. and they'll be lacking soul. I just
saw an article about it and maybe we can also
throw that in your podcast notes or whatever.
But I saw some other interesting ways where I
think what I'm considering using it for is I
don't every day get into R scripts and things
like that, but you can actually have it really
help you build some of that stuff for yourself.
And it's like, why not? And there's this one
article that I just shared to my team that was
about using a combination of R, ChatGBT, to help
you do data quality analysis. And I thought that
was a really interesting way to automate pieces
of that, where before we might spend a lot more
time tweaking the code and making sure everything's
right and everything's pulling in through. And
they can connect the dots a little bit for you.
I haven't read the... the whole article completely,
but I just like some interesting concepts about
how that can make our jobs easier. Yeah, there's
a lot of applications and different, like you
said, different pieces of the workflow and not
replacing the entire workflow. Speaking of which,
are you using ChadGBD in any way in your workflow?
And if so, how so? I mean, you know, I've played
around with it a bit. Like you said, I've also
kind of used it like before to be like, just
kind of see like, okay, how can it help me write
content? And, you know, and then I've looked
started just like look around at some of those
other ideas, like, well, how can we connect it
in with like what we're currently doing? And
like, maybe we can play with that. We haven't
like I haven't found any particular workflows
that are now part of my day -to -day. I've really
dipped my toes in it, really played with the
platform and spent a lot more time listening
to what everyone else is saying about it. I'm
super interested in all the discourse about the
privacy of it and the IP and everything like
that and what everyone's using it for. But I
have done some of those things too that just
make it really quick. I forget, I've done some
things like... make me a recipe or create a plan
for me. And like, I kind of use it like a little
bit of an assistant. And I'm super interested
in the idea of like, you know, some of the other
ones that can really like iteratively like go
figure something out for you. Because that's
something that like, you know, right now I'm
not once again, not replacing anyone's job, but
it can make things a lot easier. But haven't
found my like one thing that's like, okay, every
day I have it open. I'm really still in like
kind of play around mode. That makes sense. That
makes sense. Yeah. There's definitely a lot of
discourse about it and it's super interesting
time to just learn about what's going on. What
are some of the cool applications that you would
like to see of, say, these large language models?
I think the one I've mentioned in passing a bit
is things such as improving And this is something
I listened to another podcast recently about
creating simulations. So I think it could be
a way to make simulations a little easier. And
what's interesting about that or what is really
impactful would be thinking about some of the
critical things that can save lives. hospital
bed capacity was one of the examples in that
podcast. I really want to see us take a look
at some of those things or maybe traffic congestion,
some of these problems that are super hard to
solve for, and see if we can make simulations
and models that are a little bit easier for us
to conceptually see these big, big problems solved.
That's something that like simulations already
can do and they can train them really quickly.
But I wonder with like these bigger language
models and being able to just kind of dump a
ton of input into it, like it'd be so cool if
you could dump all that input, like almost like
a whole data lakes worth, tell it to. cleanse
it, normalize metrics, blend everything together,
consider all the pitfalls, and then tell it the
business question, and it's able to model things
out for you. I think we're super far, far away
from that right now, but that's where I think
it will be super helpful. Because another thing
from that... That podcast that was really interesting
to me was thinking about how that could explain
why we make some hard decisions in politics and
in the government and things like that. If you
really model it out and say, well, this is why,
it can be really impactful. But really, anything
that's going to positively impact people's lives
is where I really want to see the first one,
even though I sit here and I work for for -profit
companies all the time. No, that's a great outlook.
It definitely can help in a lot of areas. It
can help a lot of people. I think Satya Nadella,
Microsoft's CEO, talked about how language barriers
can be broken and you don't need to know language.
Google already does this with their Google Translate,
but this could take it to the next level. You
know, with ChatGBD's human -like speech abilities,
I can definitely take it to the next level. Yeah,
it'll be like Star Wars, right? Little translator
in your ear, I'm so excited. That's one I didn't
think of right now, but yes, that'll be really
interesting. Like, getting to the like, you know,
in the moment, being able to like translate directions
or whatever, super cool. Yeah, it's super helpful
and, you know. Like you said, you touched on
this, but right now you could use it more like
a personal assistant, free of cost right now.
So that's really cool. We will see how this goes.
No, appreciate your time. This was super fun.
I learned a lot about data literacy, wrote down
some notes, data literacy, customer focus, data
maturity, and data... Everything. Well, thank
you. I mean, I really love to hear someone else's
opinion about ChatGVT, who's really like being
an AI, like being more curious and pragmatic
about it. So it's good to hear that some of the
ideas that I have are validated. I'm not the
only one thinking of it that way. So thank you
so much for having me. It was so nice to catch
up. I feel like it wasn't that long ago that
we were having some meetings together internally.
So we should keep doing this. Absolutely, absolutely.
Thank you so much for your insights and we'll
definitely leave your company's websites and
LinkedIn and contact information with the podcast
notes. So yeah, thank you so much. Appreciate
you helping out. Thank you so much. I'll talk
to you later. Thank you. Bye.