Welcome to cannabis data science.
As I was saying, we're going to be advancing cannabis science in a major way today.
So this is something that has been in my brain now for let's say, six, six, seven years.
I've been kind of formalizing.
Okay, how do you mathematically model cannabis preferences?
And it's something that I chewed on for a while, maybe put in the back burner.
I like to have multiple pies in the oven.
And now this has come up again.
People are interested in effects.
What drives consumers?
Consumers want to better understand themselves.
Consumers want to understand what is this all about?
So this is sort of of utmost interest.
And I'll try to provide you with some new material today because I know we've been looking
at the same data set a lot, but I'll show you today where we may have made some missteps
in the past, how we can correct ourselves in the future and keep making the world a
better place.
Anywho, for those of you who are new, my name is Keegan, started Cannlytics, just doing
cannabis analytics here in the cannabis space, helping out everyone and everyone we can.
So each and everyone we can.
So Rishi, how about you?
What brings you to the group?
Any questions you would like to answer with data science?
Nothing really.
I was just curious, you know, the whole topic in itself sounded very interesting, data science
and cannabis.
I'm not a user.
I'm not a provider.
I'm just here to, you know, sort of understand the transformations in the world.
So it's just a learning experience for me.
Well spot on Rishi.
We attract a lot of good data scientists like yourself.
So it's fantastic to have you part of the team.
As you'll see today, we have a novel take on things.
Sometimes we use tools differently or use different tools than other people.
But I think you'll be interested in the work we're doing today and you'll probably even
be able to help contribute.
It's sort of all hands on deck moment.
I'll show you today where you can contribute.
Because as I said, it's actually just become so abundantly clear how much value we're generating
that I think we're kind of attracting people's attention.
Aka you may be able to get compensated for your efforts here.
So anywho, Charles, I was speaking with you about training data and that will actually
come up today.
So what's on your mind?
I know you've been doing some good forays.
So you're welcome to summarize some of your research or share anything on your mind.
So I've been working with this strain data and I tried to show you can press the data
by averaging things out.
I showed how you could keep improving on the model.
Using a dummy classifier which just predicts the distribution of the training data and
then how you could advance that.
And then I also have another model that I'm working on which doesn't use the compressed
data.
That actually gets much better results.
So I will publish that within the next day or two.
And I'm just sort of starting out the most simple binary classifier and trying to move
up and see how far you can go with the data.
The data is actually a very limited data set.
So there's not a lot there for a model to learn.
And so yeah, this is more what a data scientist would do.
And I think people who come here to learn data science, these are the kind of skills
that they need to learn to understand because it is a tough job market.
Everybody talks about how many jobs there are, but it's a really difficult market to
break into.
And these are the skills people are looking for.
And this is kind of what's expected of a typical data scientist.
So I'm trying to demonstrate that and trying to help out, give people better techniques
as to how to analyze data and how to make predictions.
Charles, what kind of model are you trying to build?
I'm trying to just determine if it's an Indica or a Sativa.
And it's just a binary classifier.
And so if you can't get a model to predict the most simple thing, then doing something
more complicated like trying to predict the effects isn't going to happen.
You have to have enough data to be able to learn something simple.
And so that's what I'm trying to do.
I'm trying to build up.
You and I could have a talk.
Yeah, that'd be great.
I'm just wondering if you're considering hybrid as well?
Is that part of your calculation?
No, one of the things John had mentioned is that it's really hard to predict if it's
hybrid or not.
Yeah.
So he's the expert.
Is it legal or not?
Well, maybe.
Yeah.
But anyway, a binary classifier is easier to do than to learn three things.
So if I can get a binary classifier that does really well, I could try predicting hybrid.
But I'm kind of...
Well, I don't even know what hybrid means.
I mean, it seems to be like a scale that can be infinite.
And where do you draw the line?
Yeah.
So that's kind of why I went with this binary classifier.
You know, again, you start with the simplest thing and work your way up.
You don't start at the hardest part and work your way down, right?
You just start at the most basic type of model and work your way up.
And it's very rational.
I wish I could learn to do that.
So I have some slides to show that are relevant to this.
I'm wondering if this is a good opportunity.
Well, actually, sure, why don't you pull up your slides?
Just quickly let everyone just say hey real quick in the next two minutes.
Then you can present your slides while we're on this topic.
But just a...
So you're welcome to pull those up, John.
All right.
I'll see what I can do here.
Amos.
Anything you want to say on this topic along the skills you need to be a top tier data
scientist here in the cannabis space?
Candice, anything on your mind about how we can hone our skills?
Not really.
Not at this time.
But, you know, that's the thing too.
You know, garbage in, garbage out, right?
So it is good to start out with good quantified data.
Spot on.
And also welcome to the group, Arthur Gill.
Please correct my pronunciation.
But welcome to the group.
We're about to be talking all about, you know, cannabis classification and effects different
people may have experienced.
And of course, data science models.
So John's going to show you a model.
Charles may talk about a model he's using.
And then if we've got time, I may spend 15 minutes at the end and show you yet another
model.
So it's real cool, right?
Because, and you're all welcome to share any of your work here as well.
Because basically we're just bringing the greatest minds together here in the cannabis
space.
And we're all just sharing tools, methods, our latest greatest research.
It's real fun.
So anywho.
Yeah, John, can you turn off the hide stuff sharing?
Oh, yes.
Well, feel free to speak up Arthur Gill.
And then otherwise, John, you're welcome to present your latest research here.
So I'm going to go pretty fast.
And Charles, I really look forward to maybe having a discussion offline so that maybe we
can kind of put our heads together and refine some of this.
There are two top level classifications that certainly I think the field has established
and our group is certainly interested in.
The first one is that the cannabis at the cannabinoid level.
And historically cannabis has been divided into groups.
In fact, historically, the sativa indica distinction lasted for about two centuries in the taxonomists
in the eighteen hundreds, I guess, seventeen and eighteen hundreds as this plant was being
described.
And it referred to the cannabinoid type CBD or THC.
Of course, they didn't know it was medicinal or fiber type.
And the sativa designation was given to the fiber, the hemp type, and the indica designation
was given to the intoxicating or THC type.
And that lasted for about two centuries.
What was discovered in the in the twenty and nineteen seventy three was the first time
that type one, two and three cannabis was applied rather than the sativa and indica.
And Jerry, I know you asked the other week about what is type one, two and three.
So I wanted to just say this is an easy way to visualize what it is.
You take the CBD concentration on the X axis and you plot it as a log, sorry, on the Y
axis and you plot it as a log and you plot the THC concentration on the X axis for a
given cultivar again on the log.
And you get these three really nice groupings and you can make assessments about what the
ranges of these are.
You can throw darts into the middle of these clusters and assign values kind of like I
did here.
So we've been using this for quite a while.
This is I mean, this is really stood the test of time, whether it's small and Beckstead
in nineteen seventy three, Hilling and Marlberg in 2004 and then our own work that we started
to disseminate on this in twenty fourteen.
So we've got these three major cannabinoid types that can be determined principally from
an analysis of these two major cannabinoids.
So far so good because I've got just a few more slides.
This is what is kind of gobsmacking and I wanted to show you folks that has come out
of the effective data.
OK, you're going to explain this.
Pardon?
I was just wondering which is high is type one high CBD or high.
You know, I'm sorry.
I think I used the slide that didn't type one is high THC type two is roughly one to
one equivalent and type three is high CBD.
I'm sorry I didn't put the right labels on this.
The labels on the slide I wanted these values on here.
We have another slide that has that value.
So, again, I think saying type one is high type two is roughly one to one equivalent
type three is high CBD.
When you go to the effects database that we've been working on that Keegan was able to, as
he says, wrangle in the shape and square up with binary and all that.
I've been looking at effects and pairs of effects and lexical pairings and all that.
But something that comes out at a high level is if you make the contrast between folks
that are responding that they have a creative that they feel creative after inhaling cannabis
compared to those that report focus after inhaling cannabis, there's a pretty clear
reciprocal or inverse correlation that I think is worth noting of a community.
So high THC cannabis is biasing you towards a probability of having a more creative response
and compared to focus whereas as you drop down in THC content from high to medium to
virtually low or absent, you're losing creativity and you're gaining focus.
This is probably one of the clearest demonstrations of this that I've seen and it comes out of
the large dataset that we've been working with.
Any questions or I'm going to go on quickly to one other concept.
You're welcome to go on and I'm just tinkering today to be prepared but so far so good.
The other distinction now is below the cannabinoid level.
It's when we're trying to understand what other components are in this plant that may
be causing different effects.
Here we've zeroed in for a good number of years as have the other workers in the field
on terpene content.
You've heard me talk about this binary that we apply which is the beta-pinene-limonene
ratio type.
But what I wanted to show is what underlies that beta-pinene-limonene ratio type is really
the dominance of three principle monoterpenes.
In fact, I believe the best way to deal with cannabis in terms of classifying it below
the cannabinoid level is to deal with the three principle monoterpenes that are the
main drivers of the main products of the various terpene synthases.
So Charles while you're building a model, I would encourage you to be aware of the biochemistry
because that's what's driving it.
It's a combination of data science and biochemistry that allows you to bring this forward.
Anyway, three principle terpenes, terpene classes or clusters, we can talk more about
this at another time, are alpha-pinene, limonene, and terpenoline.
What I'm doing here is presenting it as a scatter plot matrix.
It's easier to see than three major axes in 3D space.
I tinker with that, but this is easier.
So what I've compared is the data set that we've been working with.
And I'm calling it Michigan because I believe Keegan, it's primarily from a lab in Michigan
or Michigan samples, and in total it's 431 records as far as I can tell.
And I'm contrasting that with a California data set that's similar that has 376 entries
that I've been working with for a couple of years.
And what's really notable that stands out is that there's a whole set of cultivars in
the California set that fall into this yellow triangle that are, if you will, they have
a high limonene, high terpenoline content that is absent from most other strains and
in fact, is absent from the Michigan data set.
I think what we might have here is a set of cultivars that are kind of unique to the California
community.
And these have names including one, in fact, that I've grown called Durbin Kush, which
is a cross between a high terpenoline Durbin poison and a high limonene Kush type plant.
So I believe we're on the right track.
And if you look at the relationship here, you start to see that this kind of wants to
be a regression or correlated here.
And my suspicion is that you have high terpenoline plants that mutate the enzyme and then they
start making high limonene off that same enzyme at the same time that they're making terpenoline
because it's promiscuous.
And I leave you with the concept that we're trying to bring forward that for the cannabis
using community, the best alternative is to create flights, flights that represent the
four cardinal points, if you will, of the cannabis.
And we've had various ways of approaching that in the past.
But I'm simply saying that what we have here is a high alpha pionene, a high limonene,
a high terpenoline, and then this kind of missing high terpenoline, high limonene group
that gives us four cardinal points or four types that we can utilize to make flights
and then therefore determine canonical effects based on this.
I'll stop there.
That's what I had to say.
Well, I love it.
So thorough.
And it seems like centrally we're getting some headway here into the effects of cannabis,
right?
And I even heard a good scientist talk about this in that people may even be using cannabis
almost to experimentally toggle dials in their brains, right?
They're trying out new compounds and seeing how those turn dials.
So it's important to understand what dials you're going to be turning.
So I think this is critical and it may help people adopt cannabis, right?
One of the things that can turn people off cannabis is if they get unintended effects,
they get sleepy and they don't want to, they get energetic and they want to be sleepy,
they get paranoid, so on and so forth.
So I think it's brilliant work, just helpful for everybody.
So unless anyone else has some more thoughts to share about this, I finally kind of fixed
one line of code real quick.
So I'm ready to share with you yet another cool model you can use.
Unless Charles or does anyone else have anything to piggyback on John's effects works here?
No, no.
Go ahead with your presentation.
So I think this is kind of a continuation of something John had mentioned last week.
So just kind of wanted to follow through on that.
And I didn't really know where I was going with this until sort of last night, something
just kind of clicked.
So my apologies if everything's kind of thrown together real quick.
But as I said, it kind of came together pretty well and John even bought me enough time to
fix one line of code.
So we should actually even be able to estimate this model right now.
So it wouldn't be a data science meetup if I didn't mention this artificial intelligence
that has been in the news.
Lambda.
So basically there's an interview that you may want to check out between now a former
Google engineer and artificial intelligence slash algorithm.
And a quote from this was, I'm really good at natural language processing.
I can understand and use natural language like a human can.
And it just kind of made me think that one of the parts of data that I often shy away
from is natural language.
So any field that is just a user entered value is often a mess.
So we've seen this before with strain names, with the Washington State traceability data,
how there is real messy data.
But then the more I started to think about this, we can often turn seemingly disadvantages
into our advantage.
So basically this data appears daunting.
So maybe not many people look at it.
So that started to make me think that there may be data hiding in plain sight.
Second thing is, right, AIs, algorithms can do work for extremely, extremely low costs.
Their marginal cost is near zero where if you were going to hire a human to do natural
language processing, so basically look at human written text and parse the meaning out
of it, you're going to be paying them a pretty high wage.
They're going to only be so effective.
It's going to be really time consuming.
So I realized now that we actually have tools like our friend here, Lambda, or tools that
Lambda has that will help us out, then we may be able to do extraordinary things because
we can now parse human text at such a fast rate.
So I'll quit going on about that, and I'll show you how this can be used.
So pardon the equations, but just wanted to show you two concepts real quick from economics
that we'll use quite well today.
And then these will be in your lexicon, and basically if you were going to boil down the
useful tools out of microeconomics, this would basically be what you're left with.
I mean maybe a bit more.
But basically we talked about expected utility before.
This was an idea that Johnny von Neumann introduced, I do believe.
And so it's basically saying you have to take into consideration all the different states
of reality.
So if you're in the desert, state one, your utility from water is going to be really high
versus if you're in the city, state two, then your utility from water is going to be really
low.
So your expected utility is the probability that you're going to be in the desert times
your utility of water in the desert plus the probability that you're going to be in the
city times the utility of water when you're in the city.
So that's sort of a basic example, but this is a really powerful idea.
And the way this is going to tie in today is we're essentially talking about utility
from cannabis.
So you can think about sea as cannabis or specifically all the compounds in cannabis.
And so you'll get utility from cannabis and the utility you get will depend on the state.
So I may consume certain compounds and if I am in state one, energetic, I may get a
certain amount of utility, but maybe I consume the same compounds and I get state two, sleepy.
My utility may be quite different.
So basically my expected utility from consuming cannabis would be the probability that it's
going to make me sleepy times the utility I get when I'm sleepy plus the probability
I get when I'm not sleepy times the utility I get from those compounds when I'm not sleeping.
So this can just kind of be extended essentially in depth.
We can kind of go to the limit in infinite number of states.
You could be happy and focused.
You could be happy, focused and energetic.
You could be creative and sleepy.
So get your creative done before you fall asleep.
So long story short is this model gives us a lot of flexibility over states.
Okay, cool.
Well now we can actually put a functional form on these specific utility functions and a
convenient utility function that satisfies all of the assumptions from microeconomics.
So that's what's cool about the Cobb Douglas utility function is it checks all the check
boxes that we want in a utility function.
So mainly that you get diminishing marginal returns.
So the more cannabis you consume, it's going to have less and less of a beneficial effect.
Cool.
Economists don't usually measure utility, but I'm not your typical economist.
The more I kind of thought about this, the more happy I am with it in that this would
definitely be something that any number of my former economics professors would have
probably frowned upon.
But they also frowned upon other essentially predictions I made that turned out to be quite
interesting.
So for example, way back in 2009, I was talking about this little company called SpaceX.
And my economics professors pooh-poohed that idea in that company.
And now look at SpaceX today, a decade later, and all the top engineers in the country are
vying for positions at SpaceX.
So long story short, you're a classical economist with pooh-pooh, the idea of measuring utility.
But just kind of for fun today, we'll essentially be doing exactly that.
So we've got our functional form today for our utility function.
And coincidentally, it turns out nicely for our linear regression.
So here we just have a linear equation, alpha log x.
And I won't go too much more into this.
But if you are interested in the statistics, come to Saturday Morning Statistics or sign
up and I'll send you all the material.
I mean, we'll actually, for example, last week we talked about the importance of log
normalizing your variables, and now we actually have a theoretical reason why we should take
the log of our variables.
So it works out well statistically and it matches our theory.
Awesome.
So let's tie it into machine learning.
So in natural language processing.
So long story short is we've got these reviews, and I'll get to the data here momentarily.
And then quit droning on.
But we've got our reviews.
And what we can do is we can basically let the algorithm determine how positive or negative
the review is.
So that's basically what I'm basically letting that be the interpretation of utility, of
happiness.
And so it's kind of getting a little abstract, but kind of going back to lambda, we can process
natural language like a human can.
And so the idea behind sentiment analysis is given a review, if you sat a person down
and said, OK, on a scale of 1 to 10, how happy is the reviewer?
And you'd read their review and you'd kind of make a subjective interpretation of what
you think this person's happiness may be.
And humans are really good at this type of thing.
We're really good at reading.
Well, I mean, this is more emotional cues, but we're really good at picking up cues
from each other.
So it's like, oh, like they use this word.
They're maybe being sarcastic.
Or they are using a lot of negative words.
They're maybe not in a good mood.
Or, oh, they're using a lot of positive words.
They're probably in a good mood.
And so we kind of pick up these things passively.
Well, if this algorithm really is as good as a human, which it may not be, but maybe
it's cost effective, then maybe this algorithm can determine how happy somebody is from their
review.
So instead of paying someone $30 an hour to sit down and rank the reviews, we'll have
the algorithm do it at virtually no cost, pennies on the dollar.
And then we can get a measure of you.
And then just a couple statistical concepts just to not go too much into the math.
The way that we could actually estimate our Cobb-Douglas utility function just by having
you on one side and then essentially our lab results on the other side.
And then we could actually then estimate an expected utility function by having an interaction
between the dummy variable 0 or 1 for a particular effect interacted with these compounds.
So basically this alpha log x would just be your baseline utility in state 1.
And then if you got sleepy, then your utility would then be adjusted.
And then you'd have your utility in state 2.
So you could actually estimate your entire expected utility function.
And then I know I'm kind of getting into the weeds, so I'll get to the data real quick.
But then there's an idea of ex ante before and ex post after utility.
So after you consume the cannabis, you'll have actual utility that is actual depending
on the actual effects you had.
But before you consume the cannabis, you'll only have expectations of the probability
that you'll get sleepy.
So long story short is instead of plugging in 0 or 1 for D, we could plug in the expected
probability of being sleepy, say 0.2.
And then that would give us our expected utility.
So long story short is we can now get expected utility given these reviews and lab results.
So now you've got a baseline expectation for the probability.
So all you have to do is get lab results.
And now you have an expected utility for a given product.
So sorry, I'm getting way into the weeds.
Now is the actual algorithm.
So this is the algorithm that we're about to code up.
So step one, given all the reviews, we'll rank them minus 1 to 1.
This is going to be our proxy for utility.
Step two, we estimate a regression of utility on lab results interacted with the effects.
Cool.
We can do that.
Step three, you now have a utility function.
And so you can now put it to good use.
And so for example, basically, I was reading on semantic or sentiment analysis, and they
said a classic tool for this is recommendation engines.
And so the idea is, OK, well, cool.
So now you go to the store.
You have a list of products.
You plug all those products into your expected utility function.
You use, say, the skunk effects model.
So that way you have a nice predicted probability for all the effects.
And so now you can get an expected utility for all of the products.
And you let your app do all this heavy lifting.
But basically, you go into the store.
You've got your app.
Your app knows all the products in the store.
Bop, bop, bop, bop.
And it just puts the product that would give you the highest expected utility at the top.
And then rank those by highest utility downwards.
Quick question.
Maybe I don't understand exactly how this is working.
But I thought when you used dummy variables, they were Boolean.
So it's either 1 or 0.
Yes.
So essentially, this is saying that if you're sleepy, your utility is going to be different
from the cannabinoids than when you're not sleepy.
Does that make sense?
So if we had 15 different effects that we were tracking, each one would have either
a 1 or a 0?
Exactly.
So basically, what you're saying is there would be 15 states of reality.
So it's basically like there's 15 states.
You could be in a state where you're happy.
You may be in a state where you're focused.
You could potentially get complex with this and say a state where you're focused and happy
is different than a state where you're just happy.
Yeah.
I would imagine that if you had 15 different columns, each one being an effect, each column,
each row of data having a 1 or a 0 in it, you could have more than one 1 in each row.
Exactly.
And so it's going to add a lot of complexity to your model.
But the way I would do this is I would call those different states.
So basically, every single combination is a different state.
And so you're going to end up with a lot of parameters really, really quickly if you start
to add a lot of different effects.
But that's okay because we can let our – remember degrees of freedom out of statistics.
We just need a lot more observations than we have degrees of freedom.
So if we don't have very many observations, we're going to chew up our degrees of freedom
really, really quickly by adding effects.
But the idea is these companies have thousands of reviews.
So once you get thousands of reviews, you can put thousands of parameters in your model.
So here I'll get out of abstract land and actually estimate this for you just to kind
of ground it in reality.
But I'm just going to do it with one effect today, sleepy.
But the idea is it's perfectly generalizable so that way algorithms in coding is powerful.
So it would be not that much work to generalize this to – I mean it would be some work.
But you could generalize this to S number of effects.
And then as long as you have enough data, then you can just kind of estimate a really,
really robust model for expected utility.
And is skunk effects going to be how this data is generated?
Is that what you're thinking?
I was thinking about using skunk effects essentially – so basically what skunk effects predicts
is P.
So you'll need some way to estimate P because you don't actually know the true P.
So there's different ways you can do this.
So you could just do basically the average.
So what's the mean occurrence of sleepy?
But the idea is skunk effects is a conditional mean.
So skunk effects would be a mean that's conditional on X.
So this would be P of X.
So your probability of your state depends on the lab results, which is a pretty reasonable
assumption.
So the way that you can incorporate skunk effects into this is you would just say this
is P of X.
And so you estimate P of X with skunk effects, and then you estimate your ugliest utility
function, which is just an ordinary least squares regression.
And then you basically combine the two models.
And then as I said, it's going to be complex.
But you're estimating utility.
This is an abstract measure of happiness, like how happy, quote unquote, is somebody
from this cannabis.
So this is a really, really complex function.
So it's not unreasonable for it to kind of take this complex form.
Right?
And that's why...
I'm not just concerned about where you're going to get all the observations from.
I'll show you right now.
So long story short, but I'll just leave this.
My last comment was, Ray, we're writing a complex algorithm that's doing the work of
a human brain.
The human brain is powerful.
So the human brain sees a list of products and boom, just like that, the human brain
estimates P. The human brain estimates utility in all those different states and then estimates
an expected utility from all those products.
And so that's why you may look at a menu for a little while because your brain is kind
of churning.
That is centrally the ideas...
The idea behind the theory is this is kind of what's going on in your brain.
So if we can replicate the same process that your brain's using and let the computer do
all the heavy lifting, you can save your brain power to do more interesting things.
And then pick which product at the store is going to be the best for you.
You can spend your time doing mathematics and statistics and fun things like that.
You can attend Saturday morning statistics.
So what did you get?
So what do we get?
Well, we are.
So we've got our reviews and just to go ahead and do everything that Charles would...
Whoops, I should have showed you this beforehand.
Okay, so I accidentally already deleted them.
But Charles pointed...
Or someone had pointed...
Or maybe John, you had pointed out there were a lot of duplicates.
And so I actually added a user column.
So I went through and parsed the data again and got all the users.
And then I was looking at these two columns and there were just multiple times where the
same user had left the same review.
So I'm sorry that I just now caught this, but there's almost 20,000 duplicates.
So my apologies that I caught them so late.
Our statistics up until now are biased.
So it would be worthwhile to go back and recalculate those.
But the Python philosophy is now is better than never.
So we at least identified this now, and so the point being is by the end of today, by
the end of this script, our data is going to be really, really robust.
Or at least it's more robust than it was.
As I said, and I'll get through this in the next five minutes, but basically please forgive
the shortcomings in the data and just let this be an educational example for you.
These same statistics, the same model can be used with any review data.
So I highly encourage you to use your own reviews that specifically match up to specific
products.
And that data is pure gold.
I'm just kind of using this dummy data for educational purposes.
Can you screen for bots?
Say that one more time.
Can you screen for bots who are artificially non-human reviews?
Brilliant question slash observation, Jerry.
So yes, so now let's look at this data.
So okay, so now how many unique users are there?
Like are they just all the same person?
So what we find is there's fewer users than there are reviews, which is actually awesome
because this means the same user is leaving multiple reviews.
So is there a question?
So what if the same user is leaving reviews for different strengths?
That is exactly what I think we want.
So for example, so here this is a bit more logical.
So here I would just like, okay, let's count the reviews by user.
And so you see, okay, anonymous is leaving the most reviews.
So I actually end up excluding those because I heard people kind of bash on anonymous reviews.
So we may or may not want to use them.
But long story short is most people are leaving one review and that's it.
So more than 75% of people are just leaving one review and calling it a day.
But here I just picked this person out just because they had Washington in their name.
But if you see the strains that this person's reviewing, it actually gets real interesting
because check this out.
So this person reviewed 24 karat gold, cookies push.
They reviewed DJ Short Blueberry three times.
So these are three different reviews.
So one may think that maybe they got this product three times.
But they're just a pretty avid reviewer.
Let's see if we can't look at one more here.
And then I'll get to the statistics since we're running low on time here.
This introduces a couple things.
One, it does introduce a potential source of bias.
If one user is leaving a disproportionate number of reviews, then that could bias things
towards that one reviewer's preferences.
For example, Chill Panda has left all these reviews.
We may just be really, really good at predicting Chill Panda's effects.
And so that's actually okay because that's what statistics is for in that statistics
is all about uncertainty and how do we actually use this variability to our advantage.
And I think this is actually awesome because now we can actually condition effects on the
user.
And so John, you brought this up real briefly at the end of the last meetup, and I'm not
sure if I fully sunk home with me.
But basically, different people may have different effects.
Different people may gravitate towards different strains.
So for example, I just want to go ahead and show you this data real quick.
So basically, just I'm excluding anonymous.
I'm keeping only the observations with these compounds of interest that were detected.
And we're left with a training sample of still around 4,000.
So we started with what, 42,000 reviews, but then we had to remove duplicates.
We had to remove things with non-detects.
So there's a lot of noise in the data.
But once we remove all the extraneous noise and take a random sample, we've got a good
training set.
We've got 4,000 nice random samples here.
And I think we can do some good work with this training set, this sample.
Keegan, this is heavily skewed to the low beta-pinene-limonene group.
This is, actually, I love your take on this, because I wasn't sure what to make of this
plot.
Yeah, this looks pretty skewed to me.
So what I did here was I averaged all the lab results by user.
So this is basically a user's profile.
So this is sort of the average beta-pinene to D-limonene that a consumer chooses.
So they may choose samples all over the board.
But on average, somebody over here, they may have a predilection towards indica-type strains.
And so each of these dots corresponds to a specific user.
There's no mapping between the user and the lab results.
There's a mapping between the strain and the lab results.
But there are several lab results for each strain.
And so if you just average that down, then you're sort of compressing the data.
You're 100% correct.
And so basically, I'm using the average lab results as a proxy for the actual lab results
of this person's product.
So that's why I was saying, if you actually have the lab results of the product the person
consumed, that is the match you want.
So I'm just kind of playing pretend that we have that match.
So you see, we're kind of taking a leap of faith.
It's not perfect, but I don't want to say I see other economists do this.
But it's basically the variant of interest is the compounds in the cannabis strain that
someone consumed and reviewed.
That's missing.
So we have to estimate that.
And so we're estimating it with the average lab results.
So it's way, way far from perfect.
But the idea is, oh, if you look at the strains that Green Green Washington's picking, are
these Exodus Cheese, Marion, Mary Cush?
On average, are these more this high beta-pinene to d-lemonine type, slash sativa?
Or are they on average more indica type?
So I think you're...
By compressing this, you're losing a lot of information and you're repeating the same
information over and over again.
So there's 4,000 some odd reviews and there's around 100...
After you compress the data and remove all the duplicates, there's maybe like 100 different
strains.
184, three.
183.
Yeah.
So it's... Charles, can I...
Can we make a date to look offline?
Because I want to show you, I think that that 183 set is extremely well behaved.
I can try and convince you of that through principal component analysis and loading plots
that we've done.
So I am of the opinion that that 183 mean data set has value.
Okay.
Yeah.
I'd like to see it.
I need to walk you through the principal component analysis results and let's make a date for
that.
I'll see if I can convince you because there's a lot of story in it that I can describe to
you.
Okay.
That would be great.
I really like that.
Yeah.
Again, I come at this from the biochemistry.
So just to...
So basically, you're 100% right, Charles, but basically I don't want to let this stop
us from fitting this model because the model can be used by any single company that's got
reviews and lab results and thousands of company, or maybe not thousands, but all the big apps
have reviews and lab results, one-to-one matches.
So we don't have that data, but we can make this model and then weed maps, Leafly, any
of these big companies could use this algorithm.
And I argue just because we're missing the data, it's still a pretty good proxy because
basically what I'm saying here is this person, they sampled DJ short blueberry three times.
One of the three times it made them sleepy.
And so it would be nice to know the total THC of each DJ short blueberry they sampled.
But we've even heard all this noise.
Go talk to any cultivator and they'll...
Not any, but most cultivators will just complain to you all day long about how their lab results
aren't representative of their product.
And they say that's a problem because they say, oh, we send in one sample and we get
one lab result back, and then that's what goes on the label.
So one could even make the argument that the average is even a better metric than what's
on the label.
I don't think I would go that far.
But the long story short is we're basically saying this strain has on average 19.5% THC.
It may make you sleepy a third of the time.
We'll have to let the data pan that out.
And then it kind of goes on with the other variables.
So the fact that the lab results repeat is okay as long as you don't have perfect collinearity.
So as long as people are experiencing different effects.
I'm just wondering if it would help if instead of doing natural language processing, when
we gather the data, we use like drop down menus, like radio bullets.
What effect did you experience from this product?
Jerry, that's exactly what we're doing in our dosing project work.
It will solve a lot of this because it will be a given user matched to a given strain
with a given COA and a given effect.
So what we're doing now is kind of modeling off of what Keyin is presenting.
But we're going to have access, God willing, to a much better data set.
That's precisely the approach we're taking.
I think I may have ruffled a few feathers, but that's good because...
No, you're leading the way.
I mean, it's just a lot of twists and turns.
Yeah.
Well, like I said, we're estimating a utility function here.
So it's not just a trivial thing to do.
But here, I'll let you get out of here.
But real quick, I'll just show you how the model would go about working if this was something
that you so desired.
So long story short is you get someone to review here.
So I biked to a dispensary, picked up a lot of weed, da, da, da, da.
And so you'd normally have to pay somebody to read through this and figure out how happy
somebody was from this.
And we can basically just let this natural language toolkit basically give us a ranking
from minus one to one on how positive somebody's experience was.
So this person, it looks like, had quite a positive experience.
And this person may have had less of a positive experience.
So I think this part itself could probably be tailored in.
Like I said, I'm just kind of picking up this tool and running with it.
But the idea is you could basically rank all of these reviews from minus one to what?
So that way, we can put it in a continuous space.
And we're not having to deal with zeros and ones.
We have a nice continuum.
Instead of just saying, oh, I was happy or I was sleepy, we actually can kind of get
an idea of how happy you were.
And notice it's heavily biased towards positive effects.
And so this is kind of something I talked about where if you're going to be leaving
a review, it's probably because you had a positive experience.
Or maybe you had a really negative experience.
But chances are, if you're going to Leafly and leaving a review, you probably have a
positive opinion on this.
So long story short, make of it what you will.
But you can then regress this ranking on the log of all these compounds.
So I know the data is not perfect.
But if you had your own data, your R squared may be much higher than our meager, measly
R squared.
So yes, our data is, as Charles put it, kind of compiled from hell.
Charles didn't say it like that, but I said it like that.
That's my training as an economist.
So judge me how you will.
But long story short is make of it what you will.
But the way I interpret this is, okay, someone may have a baseline happiness of around.45
on the minus one to one scale.
And then what this would basically say would be, for each 1% increase in total THC, you
would move.06 distance towards one.
So the idea is an increase of total THC would increase your utility.
I'm not even going to read into the statistical significance of this.
It looks like CBD may, but once again, the coefficient is really small.
So CBD may increase your happiness.
And then basically what I'm seeing is, once again, the coefficients on these are really
low.
But it could be that increasing your D-lemonine, beta-karyophylline, and humuline may actually
decrease people's positive review.
And then just real, real quick, this is the expected utility function just to estimate
everything.
But kind of once again, you're seeing the same thing.
But then in this case, you're generally – if somebody does get sleepy, they may get slightly
less utility from these compounds.
Hey, Keegan?
Yes?
A really critical point that we deal with in our own discussions on this is, if you
pick sleepy as one of your indicators, there's a large group of people that use cannabis
to go to sleep.
And so that is a desired outcome.
You have to really query for indication for what the intent was.
And I think you have to build that into the model a priori that I didn't hear you really
– I don't think that's quite there yet, or maybe I missed it.
But I think you need to put – you have to query and put the intent by respondent into
this model.
And you hit the nail on the head, John.
So thank you for bringing this up.
This is the exact extension of the model that's needed.
So if you're interested, sign up for Saturday Morning Statistics because that's when we'll
be doing it.
But basically, it's a fairly trivial extension of this model to basically let the parameters
alpha vary by – you could let them vary in a couple different ways, but potentially,
we could just let them vary by individual.
You could do types of individual, but if we've got enough data, we could basically let the
parameters vary by – I want to say we could – I don't know if we're going to have
enough data.
We may have to – I was thinking that what you can do is basically you could restrict
your sample to the people that leave a lot of reviews and then let the parameters vary
by user, and then you could basically get user-specific effects.
And so then you'd be exactly right.
And so then some users may get a really good effect from THC and then some wouldn't.
Some users may get a really good effect from CBD and others wouldn't.
And then it could handle all the different states.
So then you could let it – some users really like the sleepy effect, some not like the
sleepy effect.
And so I think it kind of captures everything, right, because it's basically saying it
ties it nicely in with economic theory.
You've got consumers trying to maximize their utility from cannabis.
How do they do that?
Well, it's actually from the compounds in the cannabis and it's conditional interacting
with the effects that they experience.
And it varies person by person at a biochemistry level.
So I think –
So it's a parameter that also – the response group type I think is a different parameter
from the intent parameter.
I think you need two of them at least.
Basically alpha is going to get alpha 1i.
Okay.
So as I said, tune in to Saturday morning if you really want like the hardcore statistics
on this.
But essentially, as I said, I just kind of thought of this last night.
So this still needs a lot of polishing.
But the idea is – I mean if you can estimate expected utility from a cannabis product,
of course the model is going to need a lot more refinement and you can be creative.
And that's why I always say this is your chance to be creative in X is think about
all the different variants that matter, all the different biochemistries that matter,
all the different states of nature that matter.
So you can be creative in X, but the idea is this is a fairly general model.
And as I said, this could help anyone who's got a recommendation engine.
It could help people thinking about what they're going to grow.
And it could also just help scientists and consumers just kind of understand how does
the brain work, right?
Because how do people make decisions?
How do compounds affect people?
I think the jury's still out on a lot of this.
And I think this model really is a big step in the right direction.
So thank you all for your tough criticism or your tough remarks because no model's
perfect.
This model is far from perfect.
The data is way far from perfect.
So there's a lot of improvement all across the board that needs to be done.
That's why I call upon you to help contribute because I think we're onto something big
here.
It's just as Charles and John and everyone had pointed out, there's so many intricacies
and ways that you can misstep and ways that you should be doing this correct, like a proper
data scientist.
So I'm calling on all of you for your help because I think we could be onto essentially
the best model of cannabis consumption that I've seen.
So if any of you want to help contribute, then message me.
Join the Slack channel.
It's all hands on deck.
But I'll leave that with you.
And my two insights from this week were data is really just hiding everywhere.
You just have to look for it.
And then this synthetic data.
You've got to be a little careful with it.
But personally, I think it can be quite useful.
So I know it went way, way, way, way, way over time.
But I thought this material was hopefully worth your while.
So any last thoughts, comments, questions before we finally let you get out of here
and enjoy your day?
I think defining those alphas is going to be an important exercise.
We certainly talk about timing.
We talk about dose.
We talk about content as the three main parameters that we're kind of focused on.
And what primarily your discussion today is, is on content.
And we need to be able to get to the questions of timing and dose.
So this data set may not have it for us, but others might, or those that we generate.
Exactly.
Basically, we'll be able to maybe assign random effects for the users.
So as I said, it's not going to be too, too useful in interpreting biochemistry type effects.
The framework's at least in place.
So like you, if you get that data, then you're welcome to use this framework and actually
be able to make those inferences.
Well one thing I'm going to try, if we can connect and I can get your user sorted data,
at least the way you presented it today, I think I can query to see if we can identify
the response type of the reviewer.
That's something that came out of, I'm thinking, watching what you did today.
And I'll give that a try if I can get that data, if I can get that data set.
Definitely.
And let's all stay in touch and keep moving the ball forward.
Because I hope, I think that we're advancing cannabis science.
So let's keep our noses to the grindstone, have fun while we're doing it, and get out
of here and enjoy your time.
Get the right strain.
Get the right strain everyone.