Welcome to the cannabis data science meetup group.
Couldn't be happier to have you here.
It's your ideas, your attention, you know, your thought, your ears, everything that you
bring to the table is what's moving the ball forward.
And we've covered a lot of ground and there's still more ground to cover.
So I pointed you in the direction of a few lab result data sets.
They need to be standardized a bit more before we can use them.
As we were mentioning last week, we're super interested in some of the terpene data, still
a bit of work ahead of us standardizing that.
So today, coincidentally, a different data set landed in my lap.
This giant data set from Massachusetts of historical THC and microbe tests.
So just wanted to do a, or do diligence and give it a look.
And then we can resume with this hunt for terpenes and land races and all of that fun
stuff next week.
So we'd love to hear about any of your adventures during the week before I start going off about
some of this cool new data.
It's a meetup after all.
So I'll start with Candice.
Would love to hear about, you know, what are you interested in or out of the Massachusetts
data, you know, whether your big interests, what questions would you like to see addressed?
Any thoughts that come to your mind?
Well, I'm excited that you have all this new data from Massachusetts.
And you know, I still, you know, search for the pesticide information, but this is great.
I'm just so excited that you were able to get this data Keegan.
Thanks.
As we always say, some data is better than no data.
So this is our first really big glance at the Massachusetts data.
It puts some of our data in perspective because as we've mentioned in the past, we have a
non-random sample of Massachusetts lab results, the public lab results we've collected from
MCR labs.
There's some rich terpene data there, but we didn't know, you know, what proportion
of lab tests did that comprise?
And it actually turns out not maybe the largest proportion.
There's tons and tons of lab tests.
So we'll actually get a metric of that today.
So love your interest and there will be more lab results in the future, hopefully to suit
your appetite, but today we'll get a taste.
And please remind me your name in case it's not coming to mind, but populate cultivars.
We'd love to hear about what you would like to see put on the table and you know, what
are you interested in?
What do you think the Cannabis Data Science Meetup Group could move forward on?
Great.
Well, my name is Caleb, Caleb DeLue, which I'm happy to remind as many times names are
silly like that.
This is my first time here at this meetup and again, thanks Keegan for inviting me.
I founded, co-founded, and I'm a data scientist at Copuleft Cultivars, a nonprofit where we're
working to protect and preserve vulnerable plants using copuleft and cannabis is kind
of our flagship one given the emerging market.
And so yeah, I just am also personally really passionate about cannabis and medicinal plants
in general, really agriculture in general, special focus towards cannabis and genetics
as well, super interested in that.
And so it's really great to be here and join in.
As to the data set, I'm fascinated by seeing the recent movement of uncovering biases and
this shifting benchmarking that's been happening with labs resulting in inflated numbers or
false negatives for biologics or, you know, I could list various other things that the
industry is honing in on right now.
But I think it'll be really interesting to view it through that lens.
And then also, I'm always thinking about how I can connect insights that I see in the cannabis
data to our work at Copuleft Cultivars where we're making galaxies that are copuleft protected
so can't be privatized or abused and which will hopefully allow people to compile trait
and gene data and other data growing data as well in this large public repository where
we can do like GWAS and other types of analysis and share that throughout the whole community
in an open source way.
So really keeping my mind open to the insights in that way as well, but really just excited
to get into the data.
And I love rolling through this kind of data and pulling out interesting insights.
Thanks for being here.
I absolutely love it.
You'll have to, if you want, share some more of your insights and knowledge with us, especially
as we start moving back into the realm of terpies and working around with, you know,
how people are classifying various varieties with strain names or perhaps chemo types.
That's something we've worked with in the past and definitely something that has a ton
more work to do.
So we'd love to have your input there.
And then as far as the talk on the town, the way we see it is people go back and forth,
but what can we bring to the table?
We can just bring data and statistics.
So that's really what we're going to do today is we're just going to supply you with just
a ton of data and statistics.
As always, make of it what you will.
Take the data at face value.
Take the statistics at face value.
But at least you'll have some data points in the conversation.
But it's relevant, right?
This is what people are talking about.
So now you'll actually have some statistics in your pocket and I'll even point out some
places where those come up in conversation and now you'll have a nice science back answer
or comment.
But anywho, cool things coming.
But Marianne, welcome back to the group.
Always love having you here.
You shed so much light on the actual ongoings at the, well, you were at the CCC and this
is really big.
This is the framework where all the data is being generated from.
So couldn't be happier to have you here.
Would love to hear your input.
Any thoughts, ideas, comments you would like to put on the table?
Sure.
For those who don't know me, this is my first, not my first time here, but it's been a long
time since I've attended one of these meetups.
I'm Marianne Sarkis.
I'm the director of data and analytics at the Cannabis Control Commission in the state
of Massachusetts.
So you are using one of our data sets.
This is kind of interesting to be in this position and just have everybody participating
in scrutinizing this data set.
And I'm actually really excited.
It's rare to have that kind of a situation and experience.
So I'm just looking forward to the meeting.
I love it.
You all have to grit your teeth at some parts and we're not going to be too critical, but
it's awesome, right?
This is kind of like seeing your child start walking on its own.
So it's really cool that the CCC released this data.
And as always, like you said, just the more eyes on it, the better because there's so
many ways to slice and dice the data.
And there's only so much time in the day.
So you know me, I just start with the very, very basic summary statistics and then just
start adding condition upon condition.
Just see how far we can take it.
It's just a tiny bit of data today, but we can go pretty far with that.
So I just wanted to share that with you today because it is real metric data and metric
likes to store their data in certain formats.
And I like to rearrange the data for analysis and I'll just kind of share with you some
of my techniques and some of the figures you can create.
So this is a starting point and then as always, I encourage you all to first, if you're interested,
repeat my statistics, double check my work, and then to take it further, go above and
beyond, add some conditions of your own, see where you can take it.
Excellent.
And then Stephanie, welcome to the group.
Happy to have you here.
If you have any thoughts, ideas, comments, questions, or anything at all that you'd like
to put on the table to make sure that we're addressing what you think is important here
in the cannabis space, we'd love to hear your input.
Hi, I'm Stephanie Thomas.
I am the data manager with the Cannabis Control Commission in the state of Massachusetts.
This was not intentional, just so you know, this was not intentional that we would have
both of us here just for this dataset.
This is complete synchronicity.
And unfortunately, my schedule has cleared up because when I originally signed up for
this group, I had a conflict.
And so the other meeting has ended.
So I'm very excited to be here and see what happens.
And it's awesome because you're working with our data.
It's such a coincidence.
Well, we may as well start sooner rather than later.
And basically, I'll just show you how we're going to look at the data.
So this is basically, okay, what are the data points that we think are pertinent?
How do we look at the data?
So this could it be a better group?
So let's go ahead and commence.
So basically, there's not a lot of rambling today, just all data.
I try to just focus more on the visualizations because I think that's what everybody can
relate with and draw insights with.
And I'll get the code posted to GitHub afterwards if you're interested.
But this was data from a public records request that was shared with me.
And if you just want to see the raw data here, it's just a big CSV.
And we don't have many data points.
So I'll have to refer you to the metric documentation.
But in metric, they basically split the lab test up into the lab result, which is what
you would think of is like, and once again, I could be butchering this, but from my understanding,
it's like the lab result is what you think of is like the certificate of analysis that
has the date tested and some of the meta details.
And then these are then the lab tests.
And you can think of this as every analyte that's being tested for.
So this is delta 9 THC, and you'll see down here there's THCA.
And just the way metric classifies this, this is just for raw plant material.
So this is just flour.
And then there's also microbes down here.
So total yeast and mold.
I just call them microbes.
There are other microbes, but it's a short word.
Okay, so we've got three different analytes that are being tested for.
We've got the test result.
There's an ID for the lab that tested it.
One of the most important data points as we've pointed out is just when it was tested.
So we've got, we have the time range starting at April of 2021.
And we just have data going through the end of 2021.
So as always, would be awesome to get a wider range of data.
And as I kind of stressed in the past, any data is better than no data.
You can't just be hypercritical.
You can't just say, oh, I just want data to 2023 or bust.
It's like, okay, we'll take it.
It's got data till the end of 21.
So there was better than nothing.
And once again, we'd love more and more analytes.
But of course, as we've noted time and time again, consumers are rather fixated on THC
numbers and THC is what's regulated by the federal government.
So out of all the data points, if we were going to only have one, and it's a pretty
good one to have also, welcome to the group, Dave, just looking at Massachusetts cannabis
test results.
I'm just going to start calculating a bunch of statistics here.
So just feel free to chime in at any time if you have any thoughts.
But okay, so the first things first, right, we noticed, okay, just have to standardize
the date, nothing fancy there.
Okay.
So first things first, and I don't know why I went in this order.
So I may actually go in a different order than the way I coded it.
But basically, first things first, the way I like to conceptualize this is more in like
the COA format, right, where you just have a package label.
One second, we may have another person joining us.
So the way I like to think about this is a package gets sent in for testing, and then
it gets tested for THC, THCA, microbes, so on and so forth.
And that happens on a specific date.
So basically, I wanted to just group everything by package.
And then basically ran into this one issue here.
So I think actually, actually, let's go asynchronously here.
Let's actually just start by looking at THC since that's what we've been looking at.
And then I'll kind of circle back for this one little note here.
So long story short, I'm just getting the very first result here.
So this is going to come up later.
So basically, there is the consideration that a package may get tested multiple times.
And in fact, I was...
And it's awesome that we have some people at the CCC here because I was digging through
the sampling protocol.
And there were some mentions in there about retesting.
So I'll kind of get to that here in a bit.
But just wanted to start off by saying that we'll have to take into consideration duplicate,
so to speak.
But I'm getting too bogged down with that.
Let's look at a graph here.
We need a histogram.
Okay, so a question has come up.
And for some reason, this isn't styled quite the way I was wanting it to be.
Okay, I'll try to just power through this.
Okay, there it is.
There it's a little bigger.
So long story short, a question came up at a really cool conversation yesterday.
So I think it's the Future Cannabis Project had a conversation the other day with Yasha
Khan from MCR Labs and some of the other people who have attended the Cannabis Data Science
Meetup Group in the past, Jamie Toth and Jeff Rawson.
And so that was an awesome conversation.
I'll have to point you in the direction of that.
But a question came up that was basically, what would be an unreasonable amount of THC
on a product label?
It was kind of a fun question because it basically put everybody on the spot to basically give
what we sometimes call the, if we're trying to sound sophisticated, the Bayesian prior,
which is just what statistical nerds, that's a fancy way of just saying that's just what
you personally believe a statistic would be.
So they're basically just asking, what do you think would be the tippity top, like the
99th percentile of THC?
Well, now that we actually have that data from Massachusetts, we can actually calculate
that.
And once again, it matters what you're actually looking at here.
And once again, consumers are often interested in just one number, so say total THC.
And so remember, total THC is a factor of THCA.
It's delta 9 THC plus 0.877 times THCA.
So long story short, then in 2021, the second two or the third two quarters of 2021, the
average THC was around 17 and a half percent.
Just exclude real quick anything that was zero, just in case the zeros are outliers.
So if you exclude everything that was zero, then you're looking at about 18 and a half
percent average total THC.
And then if you were wanting to know what would be like sort of a reasonable upper bound
to that?
Well, from testing, you would see that, OK, the reasonable upper bound is around 20, 28
and a half percent.
And then let's actually just real quick, let's just check out what the maps is.
So it looks like this must have just been some miscoded outlier.
So I'm wondering what the max is, maybe less than 40.
So you're seeing some that are really, really high, like to this one is at 38%.
But that's just a quite an outlier.
So long story short, the Yash and everybody had pretty good answers.
I think Jeff mentioned that he had tested a legitimate sample, testing at around 31%.
So you may see some outliers way up there.
But Dr. Anna said that, oh, really anything above 25% or so is starting to get a little
high.
But there is maybe a thought comment question from somebody.
I just hear a notification that somebody may have.
Sorry, that was me again.
I put in the link to the future cannabis project, not high enough reality versus expectations
YouTube link in the chat.
Phenomenal.
I meant to help you not interrupt you.
It's super good.
Super good.
And once again, I'm not even trying to get too fixated on THC numbers.
So I'm going to go ahead and start moving on here in a bit.
But I just have my ear open for statistics that people are curious about.
And that was a statistic.
What percentage THC is unreasonable?
And once again, if you're looking at THCA, you could measure 31% THCA.
But once again, it depends.
Are they talking about total THC?
Are they talking about THCA, delta 9 THC, so on and so forth?
Because as you notice, the delta 9 is actually really, really small in a flower.
Small concentrations.
OK, moving on.
Just slightly though.
So here's a interesting figure where once again, we wouldn't be doing our due diligence
if we didn't calculate these statistics.
Because this is just what everybody's talking about.
They're saying, oh, there's been THC inflation.
So the classic thing to look for there is just a trend in THC over time.
And once again, we'd love to get this to the current day.
And in 2021, we actually see the average, just on whole, drop from above 18% to around
the 17, 17 and 1.5%.
But once again, there could be no statistically different change to this.
And once again, with time series, you really, really want to see them play out a bit.
Because this may not be a long enough time range.
This could just all be, you know, there could be some sort of seasonality component in this.
It's not impossible.
But so there's that.
And then once again, I'm almost getting almost bored at pointing out variances by lab.
But it's something that comes up.
And it is data that we have.
But once again, it's pertinent just to see, OK, are there any variances in THC by lab?
Once again, as we've noticed in the past, there is sort of a wide variance in how labs
are measuring.
What you would like to see, and you kind of see it to a certain extent here, is you would
like to see maybe a wide variance in how people are measuring.
But over time, you would like to see them kind of hone in on what would be the true
mean, so to speak.
And so you kind of see that here to a certain degree.
But you see some labs testing on average pretty high, north of 20% for these two labs.
And then these ones are testing low.
This one, I think that's Lab D. And see, Lab D may have exited.
They may have been measuring too low.
They're measuring below 16%.
And once again, I'm having a little trouble following which lines which.
So my apologies to that.
So long story short, the labs may be kind of coming to a consensus around 18% or so.
But I would just like to point out this kind of interesting dynamic, which is right here
in October, you see Lab H enter the market.
And they're testing at pretty high percentages.
And curiously, Lab G looks like maybe their December number is not representative for
some reason.
So hard to say what happened with Lab G. And then I'm curious about sort of the future
trajectory here, because for whatever reason, Lab H and Lab X are now testing a little higher
than average.
So not too much more to say on that, other than we'd like to see where that one goes
over time, but I don't know.
Do any of you have any thoughts?
Basically my thought is there is basically repeated observations that there's just not
like the greatest variation in the world, but there is variation between labs.
So my comment is just if you're doing statistics as a cultivator or a retailer or something
like that, I would just highly recommend to include the laboratory as a variable, almost
like control for the lab.
If you're a cultivator and you're doing scientific studies, you're doing R&D, just one of your
variables should maybe just be lab, because there could be kind of something structurally
different between how Lab X is testing and Lab D.
So honestly, I should practice what I preach, because I just realized that I haven't been
maybe controlling through lab in some of my studies.
But anywho, I kind of want to move on to some other interesting aspects to the data, unless
am I talking too much about THC or what are people's thoughts so far?
Okay, on that note, I'm going to just do this real quick, because it was just I found this
a little bit interesting.
Basically I was just curious about like the retests by lab.
So I was just because it and once again, just tell me if I should skip this if it's getting
too tedious.
But basically, as I was calculating all these statistics, I realized that there were essentially
duplicates here in the data.
So I wanted to kind of figure out how to handle those.
So you see this sample, this package came in and was tested multiple times here.
It looks like it was tested twice.
So basically, was just, you know, going to look into that and see what's happening there.
But basically, what I've done is just what did I do?
I think this is yes.
So for each package, so here's this package, I basically just said, Okay, when was the
first time it was tested?
When was the last time it was tested?
What was the delta nine when it was first tested?
And then what was the delta nine when it was last tested?
And then same for the THC a, then went ahead and grabbed the lab to I did.
I tried to find out if the lab was changing between the tests, and it didn't appear to
be.
So it appears to be whichever lab tested it first is the same lab that tested it the second
time.
And just from pawing through the sampling protocol book, it seems to mention something
about Oh, when you're doing a retest, I want to say that you're supposed to use the initial
testing lab.
So this is pointing me to the direction of these are retests for one reason or the other.
And then basically, the natural thing to do is calculate the difference between these.
So I just, you know, calculated the difference, you know, between THC and THC a at these two
points in time.
And then let's see if we can print some of these things out for you.
So once again, I don't know what if I should be fixated on this or not.
So this, this is what one of the meetup members taught me that was just super valuable was
you've got to actually look at the data and just calculate some of these summary statistics.
There may be absolutely nothing to this.
But we can't be like making mistakes as we're calculating statistics.
So we have to at least acknowledge that there's some sort of duplicates in here, the retests
and kind of try to get to the bottom of them.
I'm starting to think there's not much to them.
I think they're just sort of routine retests for various reasons.
Maybe they failed microbes and they got retested for microbes, or maybe this is some sort of
routine testing.
So I'd kind of encourage you to investigate this further.
But long story short, actually, before I do it, I was just curious, you know, what does
the difference look like?
Right.
And it's actually mostly negative.
So for it, and this is great.
This was why you want to calculate statistics, right?
Because as I'm calculating this, I'm thinking, oh, I'm on to something, right?
Because I'm thinking that, right?
Whether you I'd like to admit it or not, but my null hypothesis was actually that the difference
was going to be positive that, oh, that these labs, they're retesting samples and they're
testing them at higher THC values.
But it doesn't actually the data doesn't actually support that.
It's you know, on average, the tests are about a half percent lower.
When they're retested, you see some that are testing at higher values for various reasons.
And then I just I just wanted to specifically look at Lab B, because the other labs just
don't really have enough retest to really draw my attention.
You know, if a lab just has one retest, I don't know.
But I should have looked at Lab E too.
And in fact, we can do that too.
But basically, you know, Lab B and Lab E are the only two that are really doing a lot
of like retests or duplicates.
So it would just be pertinent to just look at those two.
So this is the first time I've looked at Lab E, but may as well plot that one real quick.
And sorry that it's all in yellow.
So Lab E is a bit more all over the board.
And with these retests, so that so that's actually kind of curious in and of its own
self is like, why are the retests different between these two?
That for whatever for whatever reason, Lab B is is testing them at slightly lower value.
Wait, yes, they're testing it slightly.
Hold on.
Yeah, so yeah.
So Lab B is testing the same packages at slightly lower THC percentages later in time.
And remember, we talked in the past about stability.
And so this once again could be a natural study in the stability of cannabis.
This came up in the conversation the other day, too, is how do you distinguish between
degradation and say mislabeled cannabis products?
And it's basically it's difficult because you've got variance on variance.
But if you were going to attempt it, you have to at least have some metric of natural degradation
in cannabis products, especially it would be cool to have it experimental on the shelf
because it's one thing just to do a controlled experiment in your laboratory.
It's a whole nother for these products to be jostled about and bandied about the state.
They could experience all sorts of degradation.
So I don't want to read too much into this until I read more into this, the sampling
protocol.
I'll just open that just to show you just what it looks like.
So long.
It's a hefty document here.
But I want to say anyways, I won't look into it right now to bore you to death.
But we're going to have to look into it further or maybe get chat GPT to to summarize it for
us to figure out what's going on here with the retests.
Is this just noise to the data?
Is there anything meaningful here or at very, very best, it could be a natural study in
degradation.
That's what I'm personally hoping for.
I would just hope, hope, hope that there is just some sort of protocol where they're doing
stability testing or they once again, they may be retesting because of a microbe issue
or this or that.
But does anybody from the CCC want to chime in if I'm butchering this analysis too bad?
And then I'll kind of change gears one final time.
You're definitely not butchering things.
This has been a fascinating presentation so far.
The question, I have actually a question earlier that might have some implications on this
chart.
So when you were looking at the retest, what was the population for that?
It wasn't that high, right?
Yes, basically.
It was non negligible.
So there's 2,700 tests, but let's say they're just two packages each.
So there are about, just a quick back of the envelope estimate, there are about 1,300 packages
that were retested.
I want to say there were around 31,000 packages of 31,163.
So it's maybe around 4%, maybe between 4% and 5% of the samples of the packages are
getting tested multiple times.
So on a related note, how would you account for that for the earlier charts, right?
So for looking at the THC changes over time, wouldn't we need to exclude some of those?
That was actually a question that I put in the chat.
How do you handle that?
Good question.
So there's 31,000 here that we can double check this.
So let's just double check that this label is unique.
So actually, unfortunately, it looks like there are some duplicates in there.
So I thought I was...
So I wonder if...
Okay, so this is why I need you guys to double check my work.
Okay, so I may have made a mistake here because I was thinking that I was just getting the
first observed test.
So I was thinking that for each package here that I was just going to get the very first
time it was tested.
So let's see real quick if anything would change here if we get the last one.
Okay.
Okay, so as is good, your question has kind of shown light onto the imperfections of my
analysis.
So as always, it appears I've made some sort of mistake and take my statistics at face
value.
But what I was trying to do was just take the first time a lab result was tested.
And I guess what we could do here is just drop the duplicates just to exclude them.
Okay, so let's just try that real quick.
We're live.
So I think we can just...
I don't even have to do it.
That GBT will more or less handle this for us.
And so that's the interesting thing about coding these days.
So let's see what we have now.
So now we have removed the duplicates.
So why there are duplicates after this aggregation, I am not certain.
I can probably shed some light on that.
Some of the tests might have some subtests under them.
And we're only seeing the top test rather than the subtest.
That could be one of the reasons for the duplication.
So the package number stayed.
But the types of tests might have changed, if that makes sense.
Yes, and this is where you're talking about what's the difference between a negligible
and a non-negligible.
Exactly.
And so this is a borderline one, right, because the retests were about, like we said, around
four or five percent.
And now we've got these 500 or so oddballs.
And for today, I'm going to drop them.
But what I'd actually encourage you all to do is actually specifically look at these,
try to find out why the oddballs are odd.
Once again, there could be absolutely nothing to it.
It could just be some sort of weird coding, not coding, but just sort of some weird entry
detail.
But leave no stone unturned.
So today, unfortunately, I'm going to just toss this stone out.
But just keep in mind that that does go against my philosophy of never throw away any data.
So I'm kind of, I'm unfortunately going to throw that data away today.
But let's just redo these statistics.
So after we exclude those oddballs, I wonder if they're not the zeros.
But luckily, this is the power to the law of large numbers is our sample doesn't necessarily
have to be perfect.
There could be some flaws in there.
But as long as the number of observations we get goes up, we're going to start approximating
the mean sooner rather than later.
So long story short is we're still looking at around 28 and 1 1
And so long story short is we're still looking at around 28 and 1 1
And unfalible as has just been pointed out, and that's why the scientific process, one of the most important pieces is the repeat, right?
It's repeat, repeat, repeat, reproduce, reproduce, reproduce, reproduce, which is basically it's not really enough just for one person to do the analysis.
Really, just the more and more people that do it, the law of large numbers kicks in.
So basically, that's what I like to think about is as we're doing the scientific process, just like there's just a distribution of THC, there's sort of going to be a distribution of our own statistics.
And, you know, the average of everybody's studies may approximate the true value.
But just for fun here, once again, I'm not trying to go too far and down the yeast and mold rabbit hole, but it was it's the talk on the town.
So I just thought it would be pertinent to to look at, right?
We've got the data. It would not be.
It would not be wise to not even look at it.
So it costs barely anything just for us to look at it.
So why not? So long story short, this is wasn't supposed to be, but this is in scientific notation here.
Basically, the failure rate for yeast and mold is ten thousand colony forming units per gram.
That's the same as in Washington state.
And so in Massachusetts, basically, once again, looking at the sampling protocol, they basically say that if you fail for microbes once,
you can you can get it remediated.
And then I think if you fail a second time, then I think it has to be destroyed.
And you don't have to remediate it. You can destroy it after the first time.
And there may have been a third option, but don't quote me on that.
In fact, you know, check out the resources above.
But here's actually a better visualization of that.
So because we've we've mentioned this in the past, where, of course, you want to maintain a clean environment.
But as a cultivator, you have to measure your costs.
And if you don't, you know, you could run into the red.
And just the way I like to do this is if you're a cultivator, just kind of budget in the fact that you may fail a test now and again.
Right. You may be keeping running a clean ship.
But if you just look at the market on average, people are failing around six percent of the time.
So you just just budget that in, just say, OK, what would what would be our loss if we did have to destroy a whole batch?
Then multiply that by six percent.
And that's basically your cost of micro risk.
And you once again want to minimize your risk, keep a keep a clean environment and all that jazz.
So ideally, we mentioned this in the past.
You would like your failure rate to be lower than average.
And if you notice that you're failing above average, then.
It would be worthwhile looking into that because, you know, you could have higher costs than your your neighbor.
So I don't want to drum on that too, too much.
But once again, there's your good statistics to have in your back pocket.
And we'll have to actually double check what we calculated with the average was in Washington.
But I want to say that the average was was lower in Washington.
But it may not have been if it was lower, was it statistically different?
So that's actually a cool study that that you could all do.
And we actually have this data now.
You could actually compare yeast and mold detections in Massachusetts to Washington.
Who knows? And then the final thing here, which has been.
Been noted before, and once again, don't want to harsh too much on the labs,
but it would just be pertinent for us to show this one because.
This is a statistic that does kind of jump out at you.
We've mentioned in the past that you don't want to just draw too, too much from small differences in things like.
Like, so for for example, when we looked at the differences in lab testing of THC.
Yes, there was some variance, but.
You know, like we said, the line, a lot of the lines were around 18 percent.
You know, if there's one that's way different than take note.
Right. This is a difference that should jump out at you, right?
Obviously, there's some difference going on there.
And then this is another one that they kind of should jump out at you is.
It's just there. There's just some.
What appears to me like just some structural difference between,
you know, like how the the labs are measuring microbes.
And I put the the total number of tests that each lab is doing above each bar.
Because I think that's important, because so, for example, lab H, they are an outlier.
But they've only tested, you know, well, not only, I mean, they have tested 200 samples, 197.
But, you know, lab B's tested 14000 samples in this same time period.
Lab G looks comparable, but they only tested 70.
And then lab A and E each tested a big bulk here themselves.
So once again. This is actually I actually have to point it out, because I wouldn't be responsible if I didn't.
This is kind of how you can.
I don't want to say lie, but mislead with visualizations,
because this is technically kind of a misleading visualization,
because it does make you think like, oh, wow, there's this like sixfold difference in failure rate by lab.
But. Technically, lab H and lab X and even lab Z,
they're all kind of outliers in the sense that they're not testing very many samples.
Like they're like they really actually could be something structurally different about those labs.
They. They could be startup labs and they may have a specific clientele,
maybe for whatever reason, lab H is the remediation lab, right?
Maybe like for whatever reason. They just get tons of failures.
So to the ones so long story short is if you excluded the outliers.
And you're just looking at lab A to lab B, then, you know, it may not be the greatest variance in the world.
Right. You've got lab A that like two or three percent and then lab B,
maybe around eight or nine percent failure rate.
So it may not be like as drastic as this this chart kind of makes it seem.
But but I think we mentioned this in the past, where for whatever reason.
The microbe method just doesn't appear to be standardized in just quick anecdotal evidence,
you know, from working in a lab. It's it wasn't a super set in stone method,
because basically the standard protocol, just the I think the cheap once again, the cost is different.
But the standard cheapest is to test these microbes from plating,
which is sort of like the old school method.
And it's maybe not the most accurate, but that's just the in their microbes.
Right. So that is it's literally dirty. So it's quick.
Just a quick and dirty way of testing.
And then you can do other ways.
There's the PCR, which is a scientific instrument, and it has its own protocol, its own sample preparation.
And then you can maybe think of that as maybe the medium degree accuracy.
And then I think if you wanted like the highest degree of accuracy,
don't quote me on this, because I'm not a chemist.
You may or may may not be able to test for these on a mass spec.
But I think that's like that's like way overkill.
I think that's like if and you may not be able to.
I think you can test for mycotoxins on the mass spec.
I do not know for certain if you can test for yeast and mold on the mass spec.
But if you could, it would be a really, really expensive test to do something that you can normally test for cheaply with plating.
So long story short is it could just be just something as simple as lab B is using like a plating method,
which is maybe not the most accurate in the world.
And that's the rate they're getting.
And then it could be that maybe lab A and lab E, maybe they're using PCR testing.
And maybe it's more expensive, but they're able to exclude false positives.
And. Right, if they're excluding false positives, they'll have a lower failure rate.
And there they could potentially have happier clients.
So that's that's kind of where I want to leave this is.
I think, well, hopefully between the regulators and policy plus economic incentives,
I would hope to kind of see this over time, as we mentioned with the THC.
I would like to see them kind of come to a consensus.
So, you know, what's it going to be?
You know, are we going to test by by plating or are we going to test by PCR?
Just kind of come to a consensus of what instrument?
And they don't have to talk to each other because that could be seen as like collusion or
anti-competitive. Really, you don't necessarily want them talking to each other.
But what I'm just saying is market forces will kind of push them in that direction, where.
Maybe it is cost effective to invest in a PCR machine,
or maybe it is cost effective to use plating or who knows,
maybe somebody will discover a new instrument to test yeast and mold.
So that those are my main thoughts here is.
I wouldn't be too, too alarmed about the data unless you're working at the CCC and then,
you know, you're you're the regulator here, but just from a consumer
point of view, it's like, yes, you know, we've highlighted some some variances in THC in and microbes.
But I would just like to point out that this was in.
Twenty twenty one and would love to see this played through to twenty twenty three,
because my hypothesis is that slowly but surely, I think the industry is.
Slowly, but as I said, surely coming to a consensus of what they think
standard lab testing should look like.
So we may not be in the long term yet, but, you know,
hopefully this rocky journey will start settling down.
But any thoughts, comments, questions?
I know this was sort of a different session than normal.
Not as much back and forth today, mostly just me droning on about these charts.
So we'll be back, hopefully do a nice back and forth discussion next week and back
to the terpenes and strains and all of some of that fun consumer related stuff.
But just had to spend a little time and hammer out some of these dry statistics
about lab tests just to set the stage, just so we have some statistics in our in our tool belt.
But any thoughts, comments, questions after being bombarded by all those charts?
One of the things to keep in mind is that some of the labs did not come online until later.
So you're seeing kind of a snapshot of some of the older labs and some of the more recent labs.
So that I think you might need to get the dates for when the labs opened and maybe group them
in particular ways to make the comparison a little bit more even.
Because right now, as you showed in that last the last slides,
the difference among the labs is pretty stark visually, right?
But if you group them into two groups and then, you know, the ones who came online later
versus the earlier ones, there might be some other differences to observe as well.
You definitely gave a good bit of foreshadowing there because that's kind
of what I think we'll see when we hopefully eventually look at the data is I've got a feeling 2022
was a pretty turbulent year, especially since you mentioned that there could be some new labs coming on board.
And I mean, just anecdotally, that's what we've heard from a lot of people in the cannabis spaces.
You know, it's been tough times and there's been a lot of change.
So I've got a feeling.
Tons and tons of variation and dynamics in the market in 2022 and potentially.
Once again, are things coming to a consensus yet in 2023?
I don't know. I've got a feeling they're heading in that direction.
But as I mentioned, I don't think they're there yet.
So I'm that's my just hypothesis is I bet there's going to be tons of variability and change in 2022.
And maybe the variation slows down this year and into next.
Only time will tell and only the data will tell.
But that's my best guess.
Best hypothesis.
We will be releasing more data in the next few months.
I know I said that last year, but we're actually getting very close to reviewing more and more of the data
and we'll be able to release it and it will be available to everybody on our data catalog.
Yes, and as pointed out, this is super valuable because, as I said, we went all the way down this rabbit hole
and looking at all this lab by lab variants. But that wasn't even the important thing today.
The important thing today is that we were able to actually answer a real question that a lot of cannabis consumers have.
That was the top question that came up yesterday.
The conversation is at what point do your spidey senses start tingling?
Like at what point do you get a red flag?
Like when you see a THC number on a label.
And as we mentioned, it varies state by state.
But we've at least answered that question for Massachusetts.
And it's basically like, yes, like the very, very tippity top percentage, you know,
you may see for the cannabis flower is, you know, like 31.8% or so THCA.
So it is possible for like the tippity top shelf products to have north of 30% THCA.
But just keep in mind that not every product will, right?
That's only the 99 percentile.
So it's like, you know, if you get something that's north of 25 percent, that's as far as total THC goes, that's well above average.
So, you know, so I think as far as consumers go, I mean, surely it can it can only help to just have a better understanding of what is the distribution here?
You know, what is average, what's above average, what's the maximum that I should expect?
So there's simple summary statistics, but people out there have a lot of them have no idea, right?
At the conversation the other day was these were the best and brilliant minds, and they actually had super close answers.
As I mentioned, Jeff hit the 31 percent nail on the head and then Dr. Anna too also was was spot on in her answer in that she said that once it gets above 25 percent total THC, she gets a little skeptical.
So so simple, simple statistics, but they go a long way.
On that note, I didn't have too much time to review your comments, but I'll review them for next week.
And feel free to share any ideas that you have during the week and I'll make sure to touch on them next week.
Just want to give you all a huge thank you, a huge, huge thank you, especially Maryanne and Stephanie and also Quinton, too.
I didn't get a chance to thank you for joining the Vita.
And of course, Caleb and Candice couldn't do this without you, your eyes, your ears, your attention, your brilliant ideas.
And today was the cherry on the top was actually having insights from the Massachusetts Cannabis Control Commission, because as we've said, you're actually in the trenches making sure the data infrastructure is up to snuff.
We really, really, really, really appreciate your effort.
We really appreciate any glance at the data.
As always, you know, we can kind of be a little tough and it's just tough love, right?
It's just any good writer needs an editor that's willing to give them a ton of red ink.
The red ink doesn't mean we don't like you.
No, we love what you're doing.
So keep at it.
You know, and, you know, we're fallible to our charts and statistics may not be perfect, but it's just, you know, we're just doing our job at just being another point in the scientific process.
Very cool. Thank you for saying that.
Appreciate it. And this is this is a great, great forum, as I've said before, I'm going to try and come more often if I can.
Absolutely love it. Any time, anytime you're welcome.
All right, cool. Thank you.
Everybody. Thank you all for coming.
Now, go on, get out of here, be productive and keep advancing cannabis science.