Welcome to the Cannabis Data Science Meetup.
As always, the team of data scientists and brilliant minds from around the world coming
here to advance cannabis science.
Even if it's only one molecule at a time, bit by bit, we're moving forward and got some
cool statistics to share with you today.
We'd be kind of curious to just talk cannabis data with you real quick if you are interested.
So Ruth, I saw that you were working on the cannabis license data and that's kind of pertinent
to just the general discussion of cannabis data with it's a bit of a mysterious industry
after all and it's tough to come by quality data.
So we'd love to get some of your thoughts that you may want to put on the table before
I just carry on too much.
That way we can have a nice back and forth.
I had done an analysis a couple years ago looking at the license data and I was really
interested in understanding what the type of structures of ownership of the different
licenses were across states.
So in other words, what amount of horizontal and vertical integration is there?
Now certain states have restrictions so some require vertical integration and some actually
prohibit it.
But I was just interested in, I was thinking kind of from an economic standpoint.
If you think of technology can flow across state lines and know how can flow across state
lines and the economics of non-cannabis inputs should be relatively similar across state lines
and in all of those cases you would expect the organization of the industry, this is
kind of my forte, you know, do companies choose to remain independent or do they choose to
integrate?
Do you see people doing only cultivation, are they doing only dispensaries or are they
doing both cultivation and dispensaries and then perhaps manufacturing?
And again to the extent that most resources in cannabis can flow across state lines you
would expect to see the same types of structures across states.
And if we don't see the same types of structures across states then obviously the regulations
are playing a major role.
And I did this analysis and I looked at California, Oregon, Michigan and Nevada because that's
where I was able to get license data and as I said this is a couple years ago and there
turned out to be big differences across states which I thought was really interesting.
I didn't publish the results but now because Keegan's made this, you know, the license
data so available I've been going through and trying to understand kind of which licenses
are available.
Another interesting thing is some states for example enable distribution or wholesaling
while other states don't.
And so again you're seeing this huge regulatory shaping of the industries so that you're
going to have very different outcomes across states just by virtue of what the regulators
are requiring or restricting or what.
And so anyway long-winded I had done an early analysis on four different states because
that's what I had available but Keegan is making available licensing data from a lot
of different states.
I've been going through supplementing them where he didn't have all the information and
doing it on a lot more states and I'm kind of in the middle of that analysis and I'm
really excited to be able to do it.
So thank you Keegan.
I love it Ruth and hopefully we can put some more statistics in your tool belt today.
As always as I mentioned at the beginning and we'll welcome the newcomers here in a
second, we just go bit by bit, tiniest little statistic molecule that we can.
And please continue looking at the data because I'm sure you'll have more insights then.
That I could think of but one thing that I discovered today is a myriad of statistics
from the ecology literature and that's mainly what I wanted to share with you today and
basically I was going to share with you some statistics about diversity in particular you
know diversity of cannabis products.
But what you could potentially look at is so you're looking at market structure.
For next week I was going to start looking at similarity and that's kind of what you're
looking at and so you could kind of compare similarity across state and I'm sure there's
many ways you can do this but what comes to my mind since I've been studying all this
ecological literature is that you could look at which companies, which licenses are similar
across states.
So if you basically consider each licensee like a species, some will like spread out
and those will be your MSOs, your multi-state operators.
So that's one way you could kind of you know study the licensees but I think you're mostly
looking at you know regulatory effects.
So I'll chew on that for next week and then maybe if you want you're welcome to share
any progress you've had or definitely bring your thoughts.
So that's a good topic that we can chew on this week and cover at the next meetup.
I like that Ruth.
So please be chewing on that.
Real quick I'll let Candice say a quick word and then I'll get to some of the newcomers.
Well we've got some returning people but I want to welcome the newcomer here in a second.
So real quick Candice you have a word that you want to say, anything that you want to
put on the table for today basically on thinking product diversity, Ruth's thinking structural
similarities between states, license structures.
So what's on your mind?
What do you want to put on the table?
I don't have anything to put on the table.
I'm very interested in both your and Ruth's ideas and also too I want to say congratulations
to Ruth.
I saw that your article was published on LinkedIn.
That's fabulous girl and that's it.
That's all I have.
Thank you.
It's great to be here.
Thank you very much.
By all means feel free to put it in the chat and share.
That's exciting news.
Good, good.
Well our newcomer I would love to get your name.
I unfortunately can't read, it appears to be Chinese.
So your name just is in Chinese characters to me.
So if you want to introduce yourself you're welcome to.
Oh hi nice to meet you.
My name is Yuta and I'm a Japanese.
I'm sorry so I couldn't turn on the video because I just woke up right now and I just
out and so right now I'm in Edmonton in Canada and I'm looking for opportunity for the industries
and also right now I'm volunteering the production cannabis company.
So I just really want to know about the industries and people here.
So that's why I just attend here.
So thank you so much for the opportunity.
Happy to have you.
And I believe it was you too and so just correct me if that needs to be corrected.
So happy to have you here.
And Canada is definitely a frontier that we want to start studying.
We've kind of neglected it up to this point and we keep saying that oh yes let's study
Canada.
Canada has had legalized adult use for quite a while now.
I actually was seeing an article I want to say it's been five years or so.
So we want to take a look.
So I love that you put your thoughts on the table because now we can know that hey there's
some demand for cannabis data science in Canada.
Let's go fulfill that demand and maybe you're here to offer some supply too.
So good to have you here you too.
Thank you.
So, Larissa, you want to chime in anything you want to put on the table for today?
No pressure.
And then Caleb, coincidentally you attended on a pristine day because you've kept being
in my ear about the importance of strain cultivar diversity.
And that's exactly what we're going to study today.
Going to take a look at the diversity of strains in Washington and then going to piggyback
on that with something that another Meetup member, Lou, who was in my ear about product
diversity in Connecticut.
So we'll look at both of those today.
You've got some cool statistics to put on your plate.
But anything that you want to put on the table?
Happy to have you here, Caleb.
Yeah, excited to be here and just always interested to see the data and pull some pathways of understanding
out of it.
We're working on structuring the fundamentals for in-field data collection for open collaboration
framework that we've been working on.
And so it'll be really interesting to see what other people in the community really
value in terms of the diversity and different phenotypic traits and other things like that.
So I imagine we'll be looking more at the processed flower diversity.
But yeah, I'm just interested in capturing the diversity of the canvas plant in general,
first in my mind and then eventually into actual quantification systems.
Well, you're in for a treat today, a Dutch treat.
So we've got it.
I mean, just wait.
We've got it, like I said, probably a half a dozen or more really cool visualizations
and statistics to calculate.
And yes, you're going to see a bunch of top strains.
And also, we're going to actually, well, in my analysis with a lesson of why the work
you're doing is so important.
So stay tuned.
That's coming.
So big things coming up soon.
Yasha, you're always doing groundbreaking work.
We've definitely been moving everything forward in Massachusetts.
We'd love, and I mean nationwide for that matter.
So we'd love to hear about anything that you want to put on the table.
Can you hear me?
Yes.
I was having IT problems this morning and I missed your introduction to what you're
going to be talking about today.
So I'm like an excited child that walked into the middle of a conversation and I'm excited
for what you're going to share and the last few words that you said about the diversity
of strains.
Excited to see what you have there.
On my front, I have spoken to I think now more than half of the states that have legalized
cannabis have spoken to I think more than half of the regulators that represent those
states over the last two weeks presented what's happening nationwide.
Excellent discussions.
Nowhere are we getting nearly as deep as the work that you do, Keegan.
It's all very surface on data integrity.
That's pretty much it.
I feel like I'm only scraping the tip of the iceberg.
You're really hammering home some of the really tough to reach statistics in the laboratory
space and I love that you're also spreading the word because as a data scientist, you
have to wear so many hats and sure you may wear the researcher hat, the computer programmer
hat, but you also have to wear the communication expert hat.
You actually have to take these visualizations, statistics, put them in front of people and
help them understand them.
Candice has that old Edward Tufts books too, but one thing Edward Tufts says is basically
your visualization should be able to be read.
If somebody can't understand your visualization, it may be because it's not a good visualization.
It's not telling the story of the data correctly.
One, hopefully the visualization should be readily apparent to people, but two, they're
going to have follow-up questions and that's when things really get fun.
It's when people start to pick apart your research methods.
Just to add a few very quick points, one is I know that within the UX industry, UX profession,
in the 90s the approach was if you have a software and users are not able to use it,
it's the user's fault.
Then there was a change to, no, it's the designer's fault if users are not able to understand
it.
Same with presentation of data.
If you present way too much data thinking I want to display all the work that I've done,
that's not going to come across as well as identifying what's the presentation that would
be the easiest for everyone to understand.
That's important.
Good point.
That's why often I'll just resort to just a bar chart.
I've got a bunch of them for you today because there's a time and a place for different visualizations
and you don't want to just force some fancy chart on a situation because you want to show
off some technique.
It's basically a time and a place for all the various techniques and when in doubt,
go simple.
That's what I do.
But anywho, enough of that spilling.
Let's practice what we preach today and actually do this.
I'll share my screen with you and let's keep it pretty active today because it's a pretty
cool subject and so I don't want to lose your interest at any point.
I'm just going to keep things moving quick because there's a lot of cool ground to cover.
Long story short, got a good laugh out of this.
Saw this on Halloween and wanted to go ahead and share it with you because Yosh has probably
seen more than his fair share of paranormal distributions here in the cannabis industry.
So that's a good laugh.
And then here's a meme that I tried to create.
Hopefully I wanted like a Monty Python style foot like squashing the ghost because today
I discovered a rad distribution.
So just keep in mind the foot.
So hopefully that can be memorable.
Long story short, what's come up a couple times, right?
Ben Calves mentioned that he's concerned with helping people preserve the diversity of cannabis.
And then I heard a member, Lou, mentioned that in Connecticut, since the introduction
of adult use, there's been a decline in just the diversity of products on the shelf.
And basically just wanted to see if we could actually put some numbers to that because,
well, if something's meaningful to you and you also can measure it, well, then you can
kind of manage it and maybe do something about it.
So that's the idea for today.
So what metrics do we even have?
Well, just basically ask chat GPT or start searching around and you'll see that they've
been talking about this in the ecological literature for just a long time.
And basically the easy metrics, you basically hear they're talking about species, but we
can generalize that to anything.
So this could be strains, this could be products.
We were getting fancy and saying this could be licensees.
And actually we'll get super creative later on and we'll actually apply this to chemicals.
So that's when things get wild.
So anywho, the abundance, just the number of each species, then you can almost think
about these as just your typical statistics.
And then evenness, that's going to be kind of how evenly dispersed are the species or
do you have an equal number of each species?
So like Noah's Ark would be perfectly even, right?
He's got two of every species.
So that would be perfect evenness.
And then the other end would be when AI takes over and then you only have like one species.
But just kidding.
That's a future that I don't think is too likely.
But anywho, that would be dominance, right?
So that would be the degree that one species is dominant over the others.
So right now humans are pretty dominant.
And then richness, that's something that's of utmost concern by ecologists.
That's just the number of all the species.
So these two are the ones that you mostly hear people talking about.
So if something is endangered, that's because its abundance is low.
And then people are also talking about the number of species, right?
Like the number of different types of fish in the ocean, so on and so forth.
So let me just rush through this because I don't know, you can find this information
on Wikipedia.
So I don't want to be too repetitive, but it is a nice foundation for us to have.
But basically, let's try to get to math land.
And we can stand on the shoulders of giants.
And this is an equation that people have derived for diversity.
And so it's basically just the sum of all the species, so 1 to r.
So if there's only one species, r is 1.
And then p is the proportion of those species.
So if it's one species, then p is 1.
And then q, I think, is an optional parameter where you can basically weight either the
abundant or the rare species.
If you crank q up, that, I think, puts weight on the abundant species.
And then if you crank q down towards 0, then that puts weight on the rare species.
So long story short, that's sort of the abstract equation.
And then this is the one that we can actually estimate.
So here, we're just, they call this the Shannon Diversity Index.
And it's basically just the sum of all the species.
And we just multiply the proportion by the log of the proportion.
And so if we just do a count, so we just count all the strains, what proportion of them are
Gorilla Glue?
What proportion of them are Super Silver Haze?
What proportion of them are Runts?
So on and so forth.
So long story short, that's readily calculable.
And then I may skip over this, but this is actually the statistic that we've seen in
the past.
So I just wanted to give a quick mention to it, the Simpson Index.
And that's just the sum of all of the proportions squared.
And we actually have seen this before.
Ruth, this is pertinent to you, because this measure, Ruth, do you know what this is?
It goes by another name.
Well, it's a measure of industry concentration.
Exactly.
So economists call this the, what's it?
I'm going to get the H's backwards, but it's the...
Hirsch-Hirfendahl Index.
Exactly the Hirschman-Hirfendahl Index.
And it's basically the exact same statistic.
And basically this is where, you know, in science, there's like a rule that the metric
is never named after the person who first found it.
So I think, you know, somebody did the square root of this, I think back in 1945.
And then Simpson introduced this in 1949.
And then I think Hirfendahl came in at 1950 and maybe popularized it.
And basically, this is a go-to statistic in economics because it's readily calculable.
And you can basically determine the market sheet.
It's a, well, once you have the market share of all the firms, you can figure out how concentrated
the markets are.
And so this is a way that you could actually...
And actually, we've done this to a certain extent, but we only estimated it.
So we estimated the concentration of species, but of licenses, of market share in the various
states.
I mean, how do we do that?
We did that by total revenue divided by number of licenses.
Actually, I'm gonna actually need a refresher on that.
So I'll make sure that that old recording is uploaded.
That may have been in a Saturday morning statistic.
And long story short, Ruth, I wouldn't be surprised if you end up calculating some of
these statistics yourself.
So anywho, just wanted to make that quick connection because it's funny how sometimes
different fields will be in their own silo.
Ecosystems is in its own silo, ecologies in its own silo, but they're still using statistics
at the end of the day.
And one statistic that's useful in one field is often useful in another.
So I'll go ahead and move on because maybe I don't want to bore you too much with that.
But let's go ahead and get to the empirics, get out of theory land and get to the empirics
because I think this is where things get fun.
So here is a figure that's readily calculable with the, and that's the last time I'll say
that word, with the data that we have.
So as always, I basically like to show you a interesting chart or figure from the literature
that we can try to reproduce.
And this is the go-to chart in ecology that really helps us visualize all of these dimensions.
First off, you're plotting abundance.
So just subtract the relative for right now.
So you could just plot abundance and we're going to do that momentarily.
So that's plotted.
Or you could plot relative abundance, which is basically how abundant is everything to
the most abundant species or type.
And so that basically kind of encapsulates dominance, basically how, to what degree is
this first one dominant, so to speak.
And it also helps you visualize richness, which is just the number of species, so how
long is this tail.
So the longer the tail, the richer the community.
So this is a community of species.
And later on, what we can do is actually compare communities.
And so that's basically on the agenda for next week.
That's when we kind of get into Ruth's land, where we kind of think of all the different
states as communities or ecosystems.
That's cool.
That's coming up.
So we've got richness.
And then evenness is basically how steep or flat is this curve.
If we're on Noah's Ark, then this curve is perfectly flat.
Everything's evenly represented.
Conversely, if it's skewed, there's not much evenness.
OK, so you get the idea.
So hopefully, I haven't completely lost you yet, because let's go ahead and actually get
to data land and actually see if we can calculate these statistics.
But while I'm getting everything ready, are there any thoughts, comments, questions?
A quick one.
I believe Noah's Ark had most animals were in pairs, but clean animals, or those that
are kosher were in seven of each.
Sorry, I think that's true.
That's such a fun fact.
So that's wild.
So it will definitely change the abundance curve.
I think that's true, from memory.
That'd be a fun statistics lesson to actually kind of plot out the, oh, what if Noah's Ark
had two animals?
Oh, what if there was actually seven in a certain time?
And a second quick point.
I assume that the graph that you were just showing, if it was not in a logarithmic scale
on the y-axis, it would have been closer to Benford's law?
Oh, you know, I don't remember that one off the top of my head, but we did look at Benford's
law.
So does that follow this similar distribution?
I can't tell because it's logarithmic here, but there should be a, I think so.
I don't want to pause what you're doing.
Well, that's a good note though, because they actually do recommend, well, not recommend,
it's basically, I don't know what the right word is, customary in the literature or in
the field to represent things in log scale.
So we'll actually be doing that today.
So we'll double check if it resembles Benford's law.
So I think that's the occurrence of digits.
Exactly.
Not just digits, but anything that comes up, the most common words in a book or how often
a word comes up in the book, the most common is going to be X percent more often than the
next, the second and so on.
Letters apply same way.
It's not just digits.
I think you're a hundred percent right, Yasha.
Now you're kind of jogging my memory.
I think this person Simpson, I think he was originally studying characters.
So like he was, I think studying like, what's the probability, you know, the next character
is going to be a Z or an A so on and so forth.
And I don't know, it just, it seems like it's just such a small world because that's kind
of like almost exactly the talk of the town right now.
Right now everybody's like, right.
I've already mentioned chat GPT a couple of times, which is basically just the really,
really sophisticated language model.
And this is something that, you know, Simpson, right.
So chat GPT is predicting maybe word like word after word, but you know, Simpson, you
know, started and he's doing it character after character.
It's just, it's just wild how, you know, one giant stood on the shoulders of another giant
who stood on the shoulders of another giant.
And, you know, in the matter of 60 to 80 years, you know, statistics is, I mean, now it's
really just blowing people's minds.
So anywho, these statistics can be used in any field, right.
It's like they can be used in natural language processing.
They can be used in ecology and economics.
And as we'll show you, you're kind of in chemistry.
But let's go ahead and start plotting these because enough talk, let's do.
So what if basically we have the Washington data, I should maybe have told you a little
about it before we just jumped in here and started plotting, but you've seen it enough.
We have about 70,000 lab results for Washington.
We know the strain names.
Of course, the strain names could be cleaned up a good bit because they were entered by
humans.
So we could probably use some natural language processing.
I'll put that on kind of the side burner.
I'll post the code to GitHub where I'd like started to do some natural language processing.
It kind of detracts from the main purpose today, which is to start looking at these
abundance curves.
And so keep in mind, we're roughly trying to reproduce a chart that looks like this.
And this is kind of where I got the idea of the foot from earlier.
So maybe if you need an easy way to remember this kind of distribution, the relative abundance
distribution, the rad distribution, I think of a foot.
But anywho, here I just needed something to talk about while this was plotting.
But here's basically the relative, or actually this isn't relative, this is just the abundance
curve of strains in Washington.
And we can actually find the richness.
So the richness, there's 9,000 strains that have been grown in Washington between 2021
and the end of September of 2023.
So you can play with this time scale for your own interests.
But right now, I kind of weighed the pros and cons of excluding data and decided just
to include everything.
I don't know.
Maybe we should look at it year by year.
So I think I'll do that down below.
But anywho, this is something that everybody wants to know.
So we'll go ahead and find these statistics.
What are the top strains?
And we always mention this time and time again.
And keep in mind, I haven't applied any natural language processing.
We have shown in the past that if you start combining Gorilla Glue and Gorilla Glue number
4 and GG4, that actually does push Gorilla Glue way up the chart.
So I haven't done that here.
So just keep that in mind.
Once again, I'll show you some really, really cool abundance charts here in a second.
But I just wanted to go ahead and introduce you to the concept with just some strains,
right?
Here is just the top 20 in Washington.
And then we don't really need to do this.
But just to repeat, this is just the same charts but with relative abundance.
And here, I think it's relative to the total abundance instead of relative to the first
one.
But this just, I wanted to plot it because it's just a slightly different shaped curve.
So that way, you've seen the shape of actually maybe the same shape curve.
OK, so relative abundance, there you have it.
So just wanted to do it just to say we did it, right?
That was what we set out to do.
Relative abundance curve, relative abundance curve.
Cool.
We checked that checkbox.
But now I actually want to kind of show you where you can take this.
Because it's kind of cool to know, oh, what are the top 20 strains in Washington?
But yeah, what's in a strain name?
So I kind of want to move to the chemical side of things.
Because why do I want to do that?
Well, real quick, I'll show you one last one of these strain abundance charts that kind
of will let me show you why we want to start thinking about chemicals.
Because what if we look at abundance year by year?
And hopefully, one of you can think of a better way to visualize this because I'm not happy
with this visualization.
This is a case where you actually may want a more sophisticated chart than a bar chart.
So here you see the bar chart does a brilliant job of visualizing the relative abundance.
Ooh, thought comment question?
If you were to stack them year by year, so instead of having overlays, stack them.
Then you can see the relative sizes, and then you can also do one based on absolute numbers
and based on proportions.
OK.
I'm not certain I can pull that off right this second.
But chat GPT may.
So real quick, just for fun, see if old chat GPT can pull this off, and then we can maybe
look at chemicals.
Because I think you're absolutely right.
Can you turn this into a stacked bar chart, please?
So we'll just see if this works.
If it works, it works.
And if not, not.
So basically, while chat GPT is drafting that up, the way that I was interpreting my subpar
visualization is that you see some, this is basically the top 20 strains for the three
years.
And you see some strains drop off.
So these strains were top strains in 2021, but not in 2022 and 2023.
Same with the red ones.
The ones that are only red were top strains in 2022, but not in the other years.
And then these ones, there's all three colors, they're top strains each year.
And then you can kind of see if they've increased or decreased from year to year.
So if the yellow is higher, that means they've decreased in popularity from 2021.
And then if the blue is higher, it means like runts.
Runts is increased in popularity.
So let's see if this stacked bar chart just works.
And this is sort of, let's just kind of copy and paste this and just see what happens,
because we only have 15 minutes.
So let's just see how close chat GPT gets.
Looks like, let me just see if this error.
Okay.
So once again, we'll just let chat GPT think in the background and we'll come back at the
end and see if they're able to, and this is where large language models have come.
And this is like the kind of the cool thing about it, right?
Who I doubt Simpson would have ever, ever imagined that, you know, he's trying his
hardest to predict the next letter in a sentence.
And now like we're basically like having chat GPT like try to predict entire blocks of code.
So anywho, and you know, as you can see, I think the model is doing it letter by letter,
or at least word by word.
But anywho, let's go ahead and start looking at some of this chemical data and I'll try
this stacked chart one last time.
Let's just see if this works.
So it may just work.
Okay.
So I'm going to, just so I don't derail the meetup, maybe I'll let you all play around
with this or I'll take a stab at it after the meetup or Ruth, you're welcome to.
But basically that's, I think, as Ruth mentioned, I think that's a better way to visualize this
is a stacked bar chart.
But anywho, I'll thought common question and then I'll then move on to the chemistry side.
Sorry, Day.
Okay.
Well let's change gears a bit.
Programming something you really have to just kind of sit down and you know, you have to
like read it and do it gracefully.
Sometimes I'll try to take a stab at it like I just did there and try to do it on the spot,
but it's difficult.
So long story short, there's a lot of trial and error and I don't want to bore you with
all of my errors.
So let's change gears here.
But long story short, the important thing, the important chart over here is the abundance
of strains.
And essentially for next week, I don't know if we're going to be able to do it because
we'll actually have to use natural language processing, I think.
But you would like to basically compare relative abundance curves from one state to the other.
Which state has richer variety of strains?
You know, California, Oregon, Washington, Colorado, Massachusetts.
You know, it may go by population, but you may be surprised.
You know, Michigan may just have a really rich community of strains.
So that's not good.
Okay, so going to go ahead and change gears here and start looking at the Connecticut
data.
And was able to actually correct a mistake I made in a past week and actually get the
chemical sorted out.
So I hadn't correctly matched up the lab results yet, but I think I have everything
aligned.
So for example, if you look at one of these lab results, here's just this first one.
You see 1.1% THC.
If you go look at the lab result URL, then we get this COA and sure enough, you have
1.1% THC.
What's really kind of bizarre though is I think we're going to need to do some double
checking here.
Because like, so for example, like I don't know what to make of this, right?
Because it says, you know, one to one, it's a one to one drop.
But you know, here they have 1.1, 1.1, but then the COA has, you know, CBD 1.15.
Thought comment question?
Scrolling down just a little bit, no, sorry, on the COA, because within the code, it also
says the Canfor is at 1.1.
And I'm not seeing where that's pulled from.
This is something that identified is an error.
For these three compounds.
So let's do another random sample here.
Right.
Because basically the data integrity of these is something that I'm really, really not sure
about.
There's something else here.
And that's that when you have a brand that advertises, it's putting out a product that
they say, say one to one, then that's kind of what they're hoping for.
But there's going to be some small variation from plant to plant.
So generally, it's about one to one, but not necessarily exactly one to one.
Okay, here.
Exactly.
So that's why that may not have been the best sample.
So here, let's look at this one.
So this one is 73.56, and we do have a spot check there.
But it also says that the CBD is also 73.56 and Canfor is also.
Okay, so I wonder if it's not impossible that this was a coding error on my...
So basically, I haven't yet deduced if this is an error from me collecting the data or
from their actual API.
But here, I'll show you where I'm getting the data and then maybe you can help solve
this problem.
But another thought comment question.
So long story short, this is kind of why I'm calling on you for investigation here.
But I'm almost 100% certain that...
So here's their database.
And so you see in their database, for whatever reason, they're repeating these values.
So I guess just because they did it doesn't necessarily mean that we should ignore the
fault.
So long story short is, this is actually why we're gonna have to do it the hard way.
It's like life just can't...
Nothing's ever easy in this world.
So it's basically like, it would be so nice just to read the data from their API.
But we actually, I think, are going to have to go to the source of truth, which is the
COA.
So basically, at the end of the day, this certificate that has these signatures is the
source of truth.
This is the official document.
So we actually do have the technology to parse these COAs.
I was actually going to prepare that for you for today, but it was just getting too much.
So I think that maybe on the agenda for next week is maybe do some COA parsing, maybe do
some similarity.
We'll do some things like that.
But it was getting a little much for today.
So I may have to exclude CBD too, but I think we're just going to have to proceed for today
just knowing that some of the compounds like THC appear to be being read correctly.
But I think I'm gonna actually have to exclude CBD now too, because it looks like CBD is
incorrect.
So long story short, we're gonna have to get the data out of these COAs.
So that's on the docket for next week.
So let me actually do that real quick.
Where is this bad one?
CBD.
So here I've actually created a list of all the cannabinoids and terpenes, and it looks
like I'm actually gonna have to cut out CBD too.
So these are the first time that I've seen the statistics without this compound.
But good eyes, Yasha.
And this is why it's so important to double check the data.
I'm not even necessarily faulting anybody in Connecticut, because look, you're dealing
with including these three, 41 terpenes.
Whoops, that was misspelled.
That's a function.
So you're dealing with, and I want to say there's 18 cannabinoids, 10 cannabinoids.
So you're dealing with a lot of data.
Okay, so let's exclude those and power through this just to go ahead and do the best we can.
So we're gonna do the best we can with the data that we have, and just know that this
data is super, super suspect, because we've already identified four compounds that appear
to be incorrectly entered.
So long story short, if we think about chemicals as in the same terms as we were thinking about
strain abundance, well, this is chemical abundance.
So you see, of course, THC, THCA, those are the most abundant cannabinoids and cannabis.
But you're also seeing things like, and sorry that this graph is so bad.
If any of you have better ideas about how to visualize this or clean these labels up,
then by all means.
So I'm just gonna do my best, so sorry for this really, really bad visualization.
But you see CBGA, beta-curylophiline, you see betamersine and limonene are quite high.
And then compounds on this end are what you'd call rare compounds.
So Puelgal and Nairol are rare.
So if you're in the business of, they call them exotics.
So those are the people who are doing just the really rare and exotic type strains.
And you may want to look for terpenes that are really, really rare, whatever they may
be.
So terpenine is actually kind of rare.
So that's something you can think about.
And then these are our classic charts.
I'm actually gonna skip over those just because it kind of sidetracks from the main thing
we're doing here.
So actually, I'm not gonna skip over them.
Do you all want me to, it's at 9.30.
So do you want me to go ahead and show you all the visualizations here in like five more
minutes or?
I'd have to step away.
Keegan is always huge.
Thank you.
Thank you, everyone.
All right.
True cool, Yasha.
Thank you for coming.
And how about everybody else?
Do you want to see all the charts or just want me to wrap it up real quick?
I'll stick around.
Okay, let's just do it.
Yeah, I have a few more minutes to stick around and see the charts.
I'm interested so far.
Cool.
Well, we'll just do these last few charts real quick because they're interesting enough.
So long story short, we've looked at beta-pinene to d-limonene ratios in the past and just
said, oh, hey, you know, different cannabis strains of different ratios of these.
So one thing that I was kind of curious about is, you know, is this changing over time?
And, you know, is the market changing over time?
And basically what I was wondering is, you know, or kind of more like what we would think
of is like the indica leading, which I kind of do in purple.
And basically the way I think of this is, you know, if you do a quick search online,
you see, oh, sativa indica, like, oh, sativa plants may have like thinner leaves than,
you know, indica plants may be shorter.
And so there's all these sort of ad hoc characteristics.
And I just almost think of this as just a chemical characteristic.
So just as plants have different heights, plants have different chemical profiles.
And so just because something has, you know, a ratio that's high in d-limonene doesn't
necessarily mean it's an indica, but that's just a trait that's typical of what you would
think of as your typical indica plants.
This kind of what I think of this is this is more just a trait.
And so basically I was just curious, like, you know, are we seeing, you know, more or
less of one type of trait over the other over time?
And you can actually conclude with some degree of significance that, you know, it looks like
the market's kind of heading in one direction.
But then I don't know, I don't know how I wouldn't actually read too, too much into
this because it's kind of, it's kind of variable a little bit, right?
Because it's, you know, up here at 0.75, it dips down, then it goes way up.
Now it's dipping down.
So there may actually not be anything to this trend.
We could just, we may just need to see time play out on this one.
I have another information at least there that might be relevant for consideration.
Year to year climate changes and we know that the genetic triggers for these compounds can
be largely associated to, or at least can be largely affected through weather patterns
and even indoor grows, you know, respond to changes in outdoor climate to some degree.
So 2021-22 year was a notably strange year for cannabis farming.
That's a good point.
And I think some things like that, like the climate should be taken into consideration
because this is sort of almost like a different topic.
This is like, that's why I almost skipped this one.
This one's a bit of a tangent.
When you're talking about kind of like, oh, which types are more prevalent over the years,
but I don't know, it's kind of in the ecological realm.
And then I don't know, kind of more into the plant chemistry realm.
So I like that you brought that up, Caleb.
So long story short, something like this, it's not apparent that it's just because people
prefer the higher delimiting.
It could just be noise.
But here's another chart that may actually kind of jump out at us, right?
We want statistics to kind of jump out at you.
So here I just calculated the, remember we talked about earlier, the Shannon index.
At the end of the day, we want some index for diversity.
So there's the Simpson index.
We've looked at that in the past.
Today we're looking at the Shannon index.
And I want to say higher values are more diverse.
And I think if it goes to zero, I want to say it's completely undiverse.
But anyways, here it is.
Sorry, my brain's at about its capacity for today.
So I kind of encourage you all to look into this and make sure I'm doing it right.
But basically, here I just calculated the total diversity in chemicals for each product.
And then I took the average of that over time.
So it's basically like, given a given product.
So for example, this first product, I won't print out everything.
Just the product type and its diversity.
So for example, this first product, it just has a zero.
And that's because it only has THCA.
But if you look at another product, it's got a diversity score of 1.1.
And you see it has THCA.
It has beta-mercine.
It's got terpenyl.
It's got beta-karyophylline.
And so basically, the more diverse the chemical constituents of the product, the higher the
diversity score will be.
And so you can basically see here real quick.
And I'll be wrapping it up.
So you can basically see the distribution of diversity in the market, where you see
some products with just a really high degree of chemical diversity.
So these ones, they're just going to have a lot of different chemicals at different
concentrations.
And then products down here are just going to have fewer chemicals at more similar concentrations.
And then basically, what I did was I just grouped that by month and then took the average.
And it's kind of stark.
And then so for example, here with the beta-pinene to d-liminene, this is where I kind of like
to show you an example of a statistic that doesn't jump out at you too, too much versus
one that really jumps out at you.
This is like a time series that it does have a statistically negative trend.
But it may just be seasonality and we just haven't captured the next season yet versus
a plot like this.
I mean, there's just no other way to put it.
For whatever reason, and the reason may even just be COA data is bad.
It could just be that there was just a structural change in how the COA data was being recorded
at these various junctures in time.
But whatever the reason, it does appear that basically what I'm calling the chemical diversity
of cannabis products in Connecticut has decreased drastically over time.
And this is kind of in line with the anecdote that I had heard from Lou, which was whenever
he basically said, whenever Connecticut allowed adult use, you just saw a decrease in just
the various selection.
So it could be, we may want to put on a historian's hat and basically pinpoint the point in time
when Connecticut permitted adult use and see if we could do a difference in difference
and see, oh, did the allowance of adult use decrease the chemical diversity in cannabis
in Connecticut?
Or maybe it didn't.
Maybe it's just been decreasing over time for another reason.
Once again, it could be COA issues.
But then I just wanted to add one last conditional plot just for the fun of it.
And this is just, remember, there's not that many producers in Connecticut.
So I just was curious about what's the difference between the producers.
And basically you see they're all kind of decreasing their diversity of products.
It does look like these two companies, it looks like they were kind of holding out.
So maybe they were during this time period trying to keep some of the diverse products
on the shelf and then they may have just not been profitable.
And then it looks like for better or for worse, it looks like most people are just kind of
settling on pretty standard set of products that don't have a wide variety of chemical
profiles.
And so maybe someone like Lou can't find the product they're looking for.
But anywho, this is just a way that you can conceptualize diversity of products in cannabis.
And I just thought it was interesting because yes, we could look at strains, but what does
a strain name mean?
I think it's a bit more impactful to actually look at the diversity of chemicals.
But I think both are interesting, but hopefully you all found a nugget here.
But any thoughts, comments, questions?
I know I covered a ton of ground there, but hopefully you found something of interest.
On that note, I want to give you all a big thank you for coming.
I know it was a long meetup.
Hopefully you walked away with some good data, some good statistics and had a little fun
while you were there.
As always, thank you all for coming.
Go on, get on out of here.
Have a productive day and keep advancing cannabis science.