Welcome to the Cannabis Data Science Meetup group in for a treat today as we finish 2022.
It's been a big year.
We've calculated cannabis statistics week after week after week.
We've wrangled data set after data set.
It's been a fruitful year.
I think we've reached some of the hard to reach statistics and I think there's still
some more to get a hold of and I'll share with you today exactly how we can do that.
Just for, well, I don't need to do an introduction, but I'll share with you today all of the interesting
statistics and data that I'll be chasing after in 2023.
But before I do that, just to give you all a chance to share something that you may be
interested in, Candice, what's on your mind and what would you hope to, well, I guess
we're already almost finished with the year, but what are your big goals for 2023 as far
as cannabis data goes?
Good to see you, Cindy.
I would, let me see, hold on.
I would like to probably pursue more Massachusetts and Florida data sets.
I would like to do that.
And then also too, I am working on the cannabis data science directory and some of the files
doing some housekeeping, getting them running and then put them into notebooks too for people
that might want to run the code too in a notebook.
Also, what was your final remark?
I'm sorry.
Is it hard to hear me?
Oh, I just cut you off.
I just started speaking before you finished.
Oh, sorry.
And yeah, that's, that's it.
Well, that's exciting because one of the things that's going to be needed, there's an interesting
new project, cannabis data.
As far as I know, it's the first of its kind, it's an open source project that's aimed at
curating cannabis data.
So Cantlydx has been building some data pike lines.
So Cantlydx will be being used in the project.
So that's exciting.
I'll share with you today some of the, it's really interesting how small projects can
start and then get adopted elsewhere.
So I'll share with you today some of the cooler tools that are being used.
It's always interesting when you develop software because as a software developer, it's fun
to write code.
So you're writing all this interesting code and it's not until it gets into the hands
of your users that you really figure out what's useful.
And it's interesting how Cantlydx is being used because some of the minor, what I considered
minor, some of the utility functions and the constants, those are actually some of the
most useful pieces of the code.
So that's awesome to discover because now we can expand upon those.
So I'll share with those with you today.
And then the other piece being cannabis data is based out of Washington state.
So that's definitely where we're focusing a lot of our energy right now.
And what we're thinking is there's almost a need for ambassadors or other contributors
throughout the, well, first the United States and then the world, because there's so much
ground to cover that a data scientist can't do it by themselves.
We need many awesome cannabis data scientists.
So while we're chewing on this data in Washington state, hopefully some of these same tools
can get picked up and then you can say Wrangled data in Massachusetts, Wrangled data in New
Jersey, Wrangled data in Florida, where have you.
And then we can each learn from each other, share interesting data.
So I think it's going to be a fruitful, promising project.
So I'll start sharing some of these links in the chat with you.
And while I share some of these links, Isaac, you're free to share some of the interesting
projects that you may hope to accomplish here in 2023.
Hello, can you hear me?
Yes, I can hear you.
We can hear you.
Perfect.
Yeah, yeah, yeah.
Perfect.
I'm sorry.
Apologies for my different background noises.
I'm at a public space.
But my hope for the year, I mean, I'm really thankful for this group to share the Washington
data set.
So I've been working on that.
I mean, maybe next meetup when I'm at a better place, I'm happy to show you some of my findings.
But my general hope for the next year is, well, it seems to me that kind of regulation
landscape is going to get some changes.
There's already some changes regarding pesticide use in Massachusetts, and people are noticeably
talking more about cannabis data.
So that's very good.
I'm just hoping if we can make some contributions to this more general trend.
And also for me personally, I think it's to learn more Python.
I have been analyzing data using R, and I thought most of your tools are in Python.
So I'm hoping I can get on board to learn how to use it and maybe contribute to the
project overall.
Phenomenal, Isaac.
I'd love to hear it.
Just to speak to some of these things.
I'd love that you're interested in pushing forward the regulatory framework in Massachusetts.
This is still a new industry, so there's a lot to iron out.
And in fact, that's the talk that's happening next week in Washington state.
People are talking about quality control testing and the consequences of Washington state mandated
pesticide and heavy metal screening in April of 2022.
And they've given licensees a long adoption period.
And people are going to be talking next week about how the industry experienced that change
in quality testing regulations.
So I think that's going to be an interesting talk.
I'll share with you some of that.
And then, as I said, it's always helpful when all of us come together and talk about some
of these regulations and the consequences, whether they be intended or unintended, because
simply talking about it helps move the ball forward.
And as I'll share with you today, as we document some of the things we do, these can get picked
up by others.
So for example, I was playing around with, of course, you have all heard of the chat
GBT, where you ask interesting questions and it will field a response.
And I was thinking that that's almost another reason why it's beneficial to keep studying
and putting out our material, because who knows, maybe some of the material that we
discuss about cannabis and cannabis related policies may get picked up by some model like
chat GPT and regurgitated.
So I don't know.
It's definitely an interesting thought.
The second part being what I always tell people is, you know, each programming language, it's
ultimately just a tool to get a job done.
I happened to learn Python back in college to build plots for various economic questions
we would look at.
I always found that making plots in Python was more reliable.
And actually, that's actually something that we'll touch on today.
It doesn't have to be Python, but any programming language, it can help make things like creating
plots in this case today, we'll talk about diagrams, reproducible.
And so that's why I picked up the programming language is I wanted good looking reproducible
plots.
And I ended up going with Python.
There were other students who used art, some used matplotlib.
At the end of the day, we could all build beautiful plots.
And for a long time, when I started out as a software developer, Python has kind of looked
down upon and people were using more sophisticated programming languages.
So I never really considered myself a cutting edge programmer.
But it's interesting that it's being used more and more.
So personally, I'm thankful, but I don't want to let other people miss out on the fun.
So I don't see any reason why you can't use R or your favorite programming language to
do data science.
But yes, a lot of our work, we'll get a little static here, a lot of our work is in Python.
And so if you can pick it up, then that could help.
But I'm droning on and droning on.
So we'll get back to cannabis and data science here momentarily.
But Sammy, welcome to the group.
Don't want to leave you out on the fun.
So feel free to ask for any introductions if you need them.
But we're basically talking about data science projects, cannabis related, that we would
like to try to accomplish in the coming year.
So you're welcome to share anything that you're interested in and any way that you may see
the group helping you out.
Okay, hi, thank you.
Yeah, so I'm kind of an unusual situation.
Can I say just like a word or two about who I am?
I just stumbled upon your event on Meetup.
By all means, it's a meetup after all.
So sorry if I didn't do a great introduction.
So my name is Keegan, founded this company, Canlytics, basically to help people with cannabis
and analytics out.
And basically this Meetup group, it's a roundtable for data scientists to come together and talk
about projects we're working on and see if we can't help each other move the ball forward.
Yeah, well, please share.
Yeah, so a couple things.
So I'm a mathematician who's been a lecturer in the university system for 10 years since
my PhD, and I've in the last few years gotten very interested in data science and been doing
a lot of it.
Playing with Python in particular, you just mentioned Python and a lot of the machine
learning tools there, scikit-learn and stuff.
And so I'm trying to make this a career.
I'm trying to get into this.
And yeah, and cannabis is such a relatively new kind of industry, right?
So you're in Washington State, right?
Not DC?
Exactly.
So I'm from the East Coast.
So when I hear Washington, I think Washington DC too.
So I always say, well, I guess I've gotten out of the habit, but exactly Washington State.
Well, I was asking because I'm in Oregon, we're sort of sister states.
We are also one of the early adopters of the cannabis decriminalization and all that.
But yeah, so I'm just, I think you had a query, you had a question when signing up for this
and said, what would you, what are you curious about?
And I mentioned something that was kind of very open, like aspirational thing, almost
like a research topic.
And it's from my experience with being in these cannabis shops where they have a thousand
flavors and I can't make heads or tails of them.
It's the same kind of paralyzing thing where I'm looking at the wines on the aisle in the
grocery store.
I'm just like, well, that label is cool.
And I was thinking, wait, this is, this maybe is a problem that has like a data science.
I don't want to call it a solution, but maybe like a, there's something there.
So I was, I was curious, like, I know some people have tried to catalog these terpenes
that show up in the different strains.
And basically somehow figure out a mapping from the space of on the one hand, the actual
strains, whether they're identified by their name or their producer or whatever, to some
other space that is more identifiable to people rather, because the names are just sort of
silly, right?
They're like, they don't really mean a lot.
So like, for example, and then I'll stop.
I've taken a lot of time here.
If I were to be like, well, okay, I bought this one strain one time and I really liked
it.
What's another one that's like that?
That's a question about like, there's some space where the, what are the other points
that are nearby in that space?
But like, what is that space?
Like that seems like a data science question.
Exactly.
So today I'll show you exactly how you can answer that question because that came up
last week and I was actually going to answer that question for our cannabis data science
crew member.
I haven't gotten to the answer yet because it's a big one, but once again, the work we're
doing is reproducible.
And once we get to the end goal, then it will be readily reproducible.
And so, exactly.
So basically the question is, okay, these cannabis flowers or products are getting sold
by strain name.
And next week we'll get a bit more into the history of that.
I might keep teasing it, but I think that's going to be a real cool way to start the new
year.
We're basically going to start with this grand strain hunt.
And I've got some things to share with you, so don't let me forget.
Okay.
So part of the interesting history, I think, is, well, cannabis used to be underground.
And I think I just kind of had this thought today.
I don't know how accurate it is, but people didn't used to want to be traced.
So I don't know if people ever thought, oh, maybe they could trace us back through the
genetics of the cannabis.
But these growers I've realized are super sophisticated, and maybe that was a concern
of theirs.
Because from basically the research I've been doing, people have been measuring, the cultivators
have been measuring the cannabinoid contents, from my understanding, since back in the 90s.
So this is something that people have been measuring, have been studying, but maybe it's
just prized information.
And so we're pretty open here, and so we're trying to free up any of this knowledge that
may be hidden.
We're not trying to step on anybody's toes, but we're just going for publicly available
data and seeing if there's any knowledge that can be gleaned there.
And long story short, we've got strain names.
In Washington State, we can at least find the THC and CBD for these strains.
And so for example, last time somebody was interested in what's considered a type 2 cannabis
testing.
I messed up my language last time, but I believe it's about equal proportion of THC and CBD.
To say the flower may have 7% THC, 7% CBD.
Well we could actually look at all the flowers in Washington State that were tested and try
to see which other ones were also near 7% to 7%.
And then we could say, okay, well which ones of those are in your area of Washington State?
And which ones were in stock last month?
So that way we could at least produce a list of similar products.
And that's a real bare minimal example.
And that's a lot of what we do here is just sort of a proof of concept.
But as you were saying, this could get substantially more sophisticated.
So instead of just using THC and CBD for product similarity, you would also use things like
terpenes.
And you could even look at, say, reviews.
And that was some of the work we were doing with one of our models, an effects prediction
model.
Looking at people's reviews of strains and seeing if we couldn't correlate those with
the chemical compositions.
So lots of interesting work to be done there.
And I'll share something with you real quick.
But first, Tammy, love to see you here at the meetup.
We're finishing the year off strong.
And I've got a surprise for you all.
Before I share this with you, we'd love to hear about any big projects you have planned
for 2023 or any way that we may be able to help you out in the coming year.
Well, I think my main project is to find a job.
That's what most of my time is spent on right now.
So I'll have to see if the ad is still valid.
I don't know if you have much of a chemistry background.
But the most in-demand jobs that I've seen are people looking for laboratory analysts
and chemists and directors.
The director would be, you know, they would of course want, of course they would love
someone with a PhD, but they may settle through someone with a master's.
But if that's your background, I think there is a laboratory in Oklahoma City that may
be hiring.
And then I've seen laboratories popping up in Louisiana that look like they're in desperate
need for scientists.
And I do believe laboratories in New York and perhaps New Jersey hiring as well.
So I'll have to keep an eye out for other jobs, but I do know people are in desperate
need for good scientists in the lab.
And in fact, I don't know if you're willing to relocate.
Well, my sister works at an environmental laboratory in North Carolina called EMSL.
And they're actually all on the East Coast.
And she said they're looking into getting into the cannabis testing.
Exactly.
And I've heard of a laboratory in North Carolina as well that tests for hemp.
And they weren't really on my radar, but I think exactly once.
So the way laboratory testing works is it's actually kind of unusual to have a bunch of
laboratories in all these different states, because normally you could just get a sample
and put it in the mail and send it anywhere in the country you want.
In cannabis, you can't do that.
So for example, there's probably already well established laboratories in certain places
in the country and they do tons of testing.
They're just haven't gotten into cannabis yet.
And so, for example, Louisiana, they need a testing laboratory.
Now they have to find a bunch of scientists and chemists and microbiologists in Louisiana.
And I'm sure they're there, but maybe they're already employed.
So that's the other thing.
So it's tricky.
And then the other laboratory that I know that's starting up, and this is a lovely place
in the country if you want to live there, is I do believe Confidence Analytics, a laboratory
that is out of Seattle, Washington.
I think they're starting a laboratory in Grover Beach, California.
And so that would be a lovely part of the country to live in.
I know they were looking for a scientific director, but they may look for other positions
as well.
Like I said, those were just jobs that came across my radar and I'll do a better job at
putting all this material in the slack in the coming year because there's opportunities
out there.
Like I said, I don't know how tied down you are to your location.
That's often a thing, but I'll keep posting these opportunities for people.
Sometimes cannabis-friendly positions are in need.
Yes.
I just wanted to share with you some other opportunities coming up.
So I had these made when I was in Las Vegas, but I never had them.
I don't know.
They didn't arrive in time.
So we were talking about this effects prediction model.
So here are two of these prediction model shirts.
And then I also have, we have the COA parser.
So we've got two COA doc shirts.
So that's cool.
So if anybody wants a shirt, if you just want to buy them outright, I was thinking just
to charge costs, which would be about $45.
So if you do want to buy one at cost, you can, but I'm going to be putting these shirts
up in the coming year for competitions.
And like I said, if you want, if you desperately want a shirt, I may list them on the website
or you can email me and we can get you a shirt.
But I think I was going to start thinking of interesting data science projects or competitions
that people can participate in and I'll send you a shirt.
And the idea is to get some proceeds and start sending people to conferences.
So I want to start sending people to all these, the conference that I've actually met the
person who organizes it is CannaCon and it's a family run business.
So it's, you know, a lot of the family members run various duties like marketing and arrangements.
So I'll have to get a list.
I had written them down here, but I can't find it on this short notice.
But anywho, yes.
So here they are.
So they're coming up in 2023.
There's Mississippi.
There's Oklahoma City CannaCon.
And you do not want to miss that one if you can get to Oklahoma.
So that's in, that's on March 31st.
And that's probably the, besides from the MJBizCon in Las Vegas, but you know, Las Vegas
is pretty pricey, but this is a real affordable conference and just it's real fun.
So you'll just, I think the past couple of years there's been 10,000 plus people.
So it's a real good way to meet people in the cannabis industry.
So if you are looking to get a job in Oklahoma, then bring your resume and a big smile and
some business cards and just talk with as many people as you can.
So that's a good opportunity there.
And then we've got New Mexico coming up in May, California, Long Beach in August, and
then Detroit, Michigan.
And there may be some other ones, Detroit's next October.
So a bunch of these cool CannaCons coming up.
And I'm going to try to think of ways that we can do competitions because ideally I'm
trying to fund you to get to these conferences and if you want, wear your Cantlitics shirt
while you're there.
So anywho, if you're in any of these areas, be in touch and we'll try to think of some
cool competitions to get you there.
So that's just kind of something fun to end the year with.
So shall we get into some of this data here?
And then I'll share with you, Sammy, how you can answer this question about how you can
find similar products in a nice logical manner.
So, okay.
Sounds good.
Yeah.
Okay.
Let's do this.
Okay.
So everybody's been talking about it.
So we wouldn't be doing our due diligence if we didn't check out chat GPT.
And so this morning I just said, oh, you know, can you please, you know, create a table of,
you know, 20 terpenes that are found in cannabis?
Because, you know, I was trying to think of terpenes off of the top of my head and the
only one I could think of super, super quick was Theta-karyophiline.
And so now say we had this tool with us, you know, now all I have to do is say, oh, you
know, here's a nice table of, you know, 20 terpenes that are found in cannabis.
Interestingly, they did list a cannabinoid.
Interesting fact, fun fact, cannabinoids are in fact terpenes.
And in fact, I've been reading a, right, I said last week, do your reading.
So I thought I wouldn't, I think I've been in the laboratory or tangential to the laboratory
space for too long without knowing chemistry.
So this is just my sister's old chemistry book, just organic chemistry by Joseph Hornback.
And you know, stand on the shoulders of giants and he's just an incredible chemist.
The book is, it's pretty dense, but there's some real interesting things in there.
And I was going to start sharing, sharing some of the fun ones with you next week, but
it's all chemistry at the end of the day.
And so Sammy, that's actually, I think an interesting thing that we may need to do is
sort of tie some of the chemistry to these strain names.
So for example, this chemist, that was another thing I was going to ask for.
Let's see if we can't get their, I think, they're finding cannabis with their chemical
structure.
So this is what we're used to seeing, right, peppery or woody or what have you.
This isn't the right thing.
I would actually, like I said, I'm not a chemist, so I'm not even using the right terminology.
Hold on one second, let me try one more thing.
Does anybody know what this is called?
The chemical, the bottle maybe?
Are you talking about the arrangement of, like given, like those chemical formulas just
list the number of each atom.
Oh yeah, the chemical formulas.
Yeah.
But there's something, an isomer, isomers are like the different ways that you can arrange
the same collection, but in different shapes, right?
Is that the word you're looking for?
Well, essentially I do believe chemists have like a shorthand way to just depict these
chemical formulas, like the chemical diagrams that you're used to seeing, like the pentagon
for carbon, I think.
Yeah.
But anywho, I won't spend too much time on this, but long story short is a good chemist
can see the chemical formula and tell you properties about the compound.
So I think we need to start incorporating chemistry more.
So also look at some of the properties of these molecules, like so for example, boiling
point, melting point.
And I think this is going to help us a lot more in our efforts.
But anywho, I'm getting a little sidetracked there.
Let me get back to the data at hand.
So I wanted to share with you an interesting tool.
So I mentioned at the beginning how it's useful to create diagrams.
And so this is a cool tool, Python diagrams, to create real beautiful diagrams.
And so once again, you can find the code on GitHub for those who are interested in the
code.
I'm going to focus more here on the actual diagram itself, since this is more what we're
interested in.
Okay.
So back in the day, people didn't want to be traced when they were cultivating cannabis.
And now it's the exact opposite.
People are getting traced down to the plant.
So this is interesting and it's interesting data.
I don't want to just say this over and over and over again, but really there's no other
industry like it.
We've just got this awesome data.
It's something that people are really interested in.
People love the cannabis industry.
It's high growth.
And then there's just all this phenomenal public data.
So just best of both worlds here.
So just want to share with you what cannabis data there is, at least in Washington state,
how we can go about connecting it and the various statistics that can be calculated.
And it's similar, but different in other states.
Every state's a little different.
And that's why we need ambassadors or cannabis data scientists all over the place in different
states, different countries.
For example, some probably need Canadian data scientists to help out with all of the numbers
that can be crunched in Canada.
But like I said, we can at least start somewhere.
So we can start here in Washington state.
So Washington commissions, licensees who are licensed to operate with cannabis so they
can either cultivate, process, test, or sell cannabis.
And we have various data points here for the licensees.
The main things that are of interest are, of course, their name.
If you wanted to look them up, do a search for them.
Knowing that their license was actually issued is fairly interesting.
And just knowing, really, the zip code's the most useful.
Just sort of knowing their geography.
That way you can do county by county statistics or even zip code by zip code statistics.
I haven't gotten that granular yet.
But I think there would be some real fruitful analysis along those veins.
So if anyone's ambitious and wants to do a zip code level analysis, then I haven't seen
one yet for Washington state.
But they may exist.
Just because I haven't seen it doesn't mean someone hasn't done it.
Okay.
So you've got licensees.
Well what are they doing?
First and foremost, the cultivators are growing plants.
So here are all the data points we have for plants.
And you see that it's tied to licensee.
Well it's not super clear, but it's tied to licensee by licensee ID.
That those connect.
And then, oh, it's your plants are in an area.
So they may be in flower room A or flower room B or what have you.
Each plant has a strain.
So strains are a whole other type of data field here.
And as we start seeing, the strains is almost an antiquated data set.
So basically my terminology here is each one of these is going to be a data set.
And these are all data sets.
So if you have better terminology, then please share.
In long story short, as we'll see, the strains is almost an antiquated data set.
And as Sammy was pointing out, other data sets or other data points, in particular lab
results, may be better.
There's still a lot in a name, but I'll share with you the strains is one of the smaller
data sets here.
All right, cool.
So we've got plants.
If you need to see when anything was destroyed, you can match those to the various plants.
If anybody wants to do plant level statistics, then some of the things that are on the agenda
that haven't yet been calculated yet are canopy.
This is something that Washington state itself, the regulatory body is concerned about.
So in 2015, there were 2.5 million square feet of cannabis plants.
So we would love to know what is the square feet of canopy in 2022.
And you may be able to parse this out of this data.
You may have to make some assumptions as to how many square feet a particular plant takes
up.
So I don't know if you can calculate square feet, but I think we've calculated number
of plants before.
So that's one thing, just to calculate the number of plants, because you can do it by
date.
So for example, you could say, oh, how many plants were harvested day by day?
So that's something that's of interest.
Then you can go granular.
So the way I always recommend people to approach statistics or start with the aggregate and
then keep adding conditions.
So the aggregate would just be how many plants were harvested in 2022.
Then the first condition would be by day.
So how many actually before that, the first condition is actually by licensee.
So how many plants were harvested, then by licensee, and then by day.
And then you can do lots of interesting things with that.
Then you could just say, oh, how many plants were harvested by day?
How many plants were harvested by month?
Who were the top cultivators?
Which licensee harvested the most plants?
Who harvested the most plants in any given month?
Who harvested the most on average?
What is the average number of plants harvested per month?
You can do all of these statistics.
And I think these are quite interesting to say a cultivator.
So for example, if you're a cultivator in Washington state, wouldn't it be interesting
to know what's the average number of plants being harvested on a month by month basis
by all the other cultivators?
That way you can know if you sit above or below average.
And maybe you already have an idea of if you're above or below average.
Well, you can find out what percentile you're in.
Are you in the 80th percentile?
Are you in the X percentile?
Go for it.
OK, so plants is an area that I've touched little.
So if you want to calculate statistics there, there's many, many, many cool plant statistics
that can be calculated.
Next, we've got those connected to strains.
Well, inventory is also connected to strains.
So this is really the only connection between plant and inventory, which may or may not
be a sufficient connection.
I think it requires more investigation.
Because the idea behind the traceability system was right from seed to sale.
So ideally, you would like to be able to trace a plant or plant all the way to sale.
You may not be able to go to a particular plant.
While this has a plant ID, this field's actually blank.
So I don't think you can trace a sale back to a particular plant.
But you may be able to trace.
One would hope you could at least trace a sale back to at least a batch of plants.
So that, I think, would be the goal is if you purchased runt flower, ideally, you would
like to be able to track this back and see at least which producer produced that runt.
And ideally, you would like to get some estimate of the harvest date.
So that's something that we'll work on.
I think that's going to require a fair amount of work.
And I'll share with you what I have gotten done.
So we'll come back to inventory here momentarily.
Let's jump to sales since we just mentioned it.
So on the far other side of the graph, we've got sales.
So this is what happens when, well, actually, when a consumer makes a purchase, this is
basically, you can think of the sale header as the receipt.
So this is the receipt.
And all you have there are the licensee to the retailer who sold it, when they sold it.
OK, so that's all you have.
Well, now you can match the items.
So these are all the sales items, the sales details.
You can match the items that were on that receipt.
So you actually have to make this connection to find the licensee.
So as you see, sales detail doesn't have the licensee.
Connected the sale headers.
Now we know the licensee.
And we know the sale date.
Awesome.
So you have to merge these.
Sorry if this is dry, but I'll get to some interesting statistics towards the end.
But long story short, you've got to merge the items with the receipts to find out who
sold it and when.
And once you've done that, well, now you can calculate sales by retailer by day.
And that's exactly what we'll be doing today.
And just to share with you some of this data.
Wait.
Here, I'll come back to that.
We'll keep talking about the data.
So you can calculate sales by licensee by day.
Cool.
Well, we want to know what strain was that.
And then we also want to know what THC and CBD was that.
So let's try to see if we can get those fields.
So how do you do that?
Well, we have the inventory ID.
So now you have to connect sales to inventory.
And that's another way to get the licensee ID.
So we made these connections and now we connect to inventory.
In inventory provides us with some interesting fields, but primarily we now need that to
get us to the product, to get the inventory type, its name, and a description.
Once you have the name, I don't think you need to get the name from the strain anymore,
but you could also connect this to the strain to get more information about the strain.
Then once you have the inventory, you can connect lab results to the inventory item.
So that is how you finally get all of the lab results for that sale.
So that is phenomenal.
So now we have items that were sold.
We know what type of items they were.
We know which strain they were.
And we even know the lab results.
So the only cannabinoids we have are THC and CBD, but that's okay.
It's better than nothing.
So we now have the lab results.
So we can actually do a lot now, a whole lot.
So let me share with you some of these statistics.
But before I go on, does anybody have any big picture questions here about the CCRS?
Sorry, I might have missed the big frame of this.
This is publicly available data, but what is the incentive behind it?
Is this a curiosity or who wants to know this, like what you're discovering here?
Or are you just demonstrating?
The company behind this, there's twofold.
I work for Cannlytics and we help people out with cannabis analytics.
And the company that's working on the cannabis data in Washington state is the company Cannabis
Data.
And they're creating an open source data platform for people to be able to access this data.
So the idea is you can do a Freedom of Information Act request to get all of this data.
But Washington announced, I'll show you momentarily, you basically just get a data dump of aggregate,
of all the data together, and it's really hard to use.
So I think cannabis data is for profit.
So everybody's for profit.
But long story short is, I think people want these data points enough that they're willing
to pay people to go pursue them.
So that's the main thing.
So definitely pursuing these for profit.
And in fact, that's what I tell people is, and in fact, this may almost be the most valuable
stage with some of these tools like GPT coming out.
I think the data curation is actually one of the highest value added stages, simply
turning this data from this web of, I guess it's clean, but I don't know, with unstructured,
that's the right word.
Moving unstructured data into structured data, I think is where the value is added.
It's becoming increasingly easy to build apps and websites, and all of a sudden, the actual
material is now astonishingly easy to create.
So I've been creating background images with some of the tools that OpenAI has put out.
DoubleBuff Fusion is the one that I use the most.
So I've already found a way to work that into my workflow to generate material.
And just today, I showed you a brief way that you could, that was a crude way, but you may
be able to start generating material for your website.
And so long story short, I think structuring the data is the most valuable part.
And like I said, analytics is in the business of helping people structure and analyze their
cannabis data.
There's this project out of Washington State.
I'm more just sharing this with you because this is a project I'm working on.
It's also open source.
So if you can think of a way to profit from creating this data, then by all means, join
in on the fun.
But also, if you need help, then be in touch with me because as I said, in the coming year,
I'm trying to get more and more people involved.
So send me a message on Slack or through email and I'll try to get you in on the projects.
So that's what I'm working on.
And then if you can take any of these ideas and use them, say, in other states or other
data sets, then maybe you can add value there.
So that's sort of the project at hand.
Just an open source for project, not for project, an open source for profit data curation project
that Candidates and Cannabis Data are working on.
Just wanted to share it with you to say, hey, if you want to help out, you're welcome.
If you want to take anything to use on your own, you're welcome.
And then we'd love to hear any ideas you may have because we're far from perfect.
And so if you have any ideas from how to improve, we're always open to hear those.
And then, you know, like as I said, you can take our ideas and run with them.
So that's sort of the idea behind it.
It's a win-win project.
Clearly, I still need to do a better job at formulating and explaining it.
But do you have any questions?
Should I do a half decent job or do you have more questions at hand?
Go on.
I might have questions later, but yeah.
Okay.
Well, here, why don't I just go ahead and get into some of this data and then that may
make it a bit more clear what we're after here.
So I just made this diagram just to kind of help in the data cleaning stage because it
helps to be able to visualize this because there's a lot going on here.
Okay.
So let's just...
Unfortunately, we may not have the data, which would be kind of...
Let's keep our fingers crossed.
There's actually one more thing I can try.
Okay.
So I'm going to unplug this and plug this back in.
Let me make sure I still have the camera on.
Okay.
We've got the data back.
Okay.
Phenomenal.
Let me just do one last double check.
Sorry for the rocky...
There we go.
Okay.
Everything's going well now.
Thanks for bearing with us.
Okay.
So we've been doing these periodic Freedom of Information Act requests.
We're going to start doing...
Well, we haven't.
Our good friends over at Cannabis Data have been doing them.
And I think they're going to start doing them on a nice month-by-month basis.
So what are you given?
So as I said, this is public data just sitting there.
And we're going to see, okay, can we take public data, get some knowledge out of it,
and do so in a way that we can't make a buck?
Okay.
So you can download this data.
How much do we have?
I think we can...
How many bytes?
Okay.
So that's almost 41 and a half.
So you have almost 42 gigabytes of data that was generated between, say, January and late
November of 2022 in Washington State.
So as we were pointing out, this adds up quick.
We should have been approaching this in piecemeal earlier in the year, but we get around to
it when we get around to it.
Now what are we given in this huge pile of data?
Well, check it out.
So there's lots and lots of inventory data.
There's a little bit of lab result data.
It's dense and important data.
So we'll explore that more.
There's a fair amount of plant data, product data, and check it out, the bulk are sales.
So this is to be expected.
In fact, consumer facing...
Oh, yes, that's what they call them.
They call them point of sales systems.
That type of software deals with enormous volumes of data.
So this is, I think, what Walmart's claim to fame.
So I think they were just able to handle enormous amounts of data, much, much better than everyone
else.
I mean, I think they also did logistics really, really well too.
And that just made them a leader.
And so as you can see, if you're a cultivator, you've got some data to deal with, but these
retailers just have an enormous amount of data to deal with.
So how much data is this?
So each of these files ranges from, say, 0.1 to 0.25 gigabytes.
And there's 60, so there's 120 of those.
Maybe I'm doing math off the top of my head, but I want to say that's around 30 gigabytes
of sales data.
And then look, all of that sales data, and there's just this one meager data file that's
not even 0.1 gigabytes of stream data.
So it's there, but I think it's definitely a minor part of the whole picture as a whole.
So I think, as you were kind of pointing out, Sammy, I think people do get kind of lost
in these strain names a lot when really the action that's going on are the sales, the
plants that are being grown, of course, the inventory, the products.
I think that's what I think really matters.
So let's dig into some of that data.
Sorry, I'll be kind of wrapping this up and getting to my main point here.
I'm trying to think about what my main point is.
I probably should have written this down.
Okay, well, I'll just share with you some of the rewards that have been reaped.
So once again, this script is on GitHub.
I'll let those of you who are interested in the code, paw through it.
But all I did was, okay, I read through all 60 of these sales details.
As you do that, say I read this one in, I now have to read in all, you don't actually
have to read in all 60, but you start reading in all 60 sales headers data files.
And then the way I have it programmed is, oh, it just stops once we've matched them
all.
So you read in the sales details, you start reading in the sales headers until they've
all been matched, and then you stop.
And then I haven't done this yet, but the next part is, okay, now you read in all 60
of all the inventory items until you've matched them all, and then you would stop.
And then you do the same thing with products.
And then lab results, I think you actually have to go in reverse because it's actually
the lab result gets matched to the inventory, the inventory doesn't get to the lab results.
So that one's a little confusing, but long story short, the script isn't hard to run,
but it just takes a long time.
So here, I'll just get it running and I'll show you.
So here it's running.
So basically we're reading in the first data file, hopefully we'll start getting some print
statements and then I'll basically just show you what the output would look like.
Okay, great.
So we just read in the sales details 61, read in the first sales headers file, we were able
to match 61% of the sales details.
Okay, cool.
So we now just read in the next batch of sales details, I mean sales headers, and we were
able to match 100% of them.
And now I think I forget what's commented and what's uncommented, but now I'm basically
saving all of these items licensed by, I'll go ahead and stop this here because that's
sufficient.
So now I'm basically just saving all these items after they've been augmented, licensed
by license.
Okay, why am I doing all of this?
Well, this is what I call data curation because now we have these nice data sets.
So that was the script, that was the hard part.
And now I think this is, so the script is the value added, the value adding, so the
script adds the value.
And then here is the result, a nice curated data set where this is called panel data,
where you've got license by date, and then this is the total sales on that date.
So now you can track this licensee selling over time.
And then if you scroll down, oh, here's another license that you can track over time.
And so this is called panel data, and you can do really cool analysis now.
So for example, you could now merge the licensee data, which would have their zip code, then
you could say, get statistics that you think are pertinent to that zip code.
So the one that we know is statistically significant is median income.
So you could find the median income for all these different zip codes, and you can see
how that correlates with price.
So that's maybe not the most interesting study ever, but you could do many, many different
analyses there.
You could see who's paying the most in taxes.
So let's just go ahead and end this strong.
I said, oh, I was going to share with you how you could solve this strain problem.
OK, well, how would you do it?
First, you have to augment all the data, which is the task at hand.
Then you say, OK, you basically, so the question we were given is, oh, can we find strains
that are, say, in the Seattle area that are chemically similar to ACDC?
Well, we can find all the lab results that anyone sent in that were tested for ACDC.
We could calculate the average THC and CBD of those ACDC strains.
We can use, I forget which one's the, I forget the name for it, but we had similarity models.
I forget what it's called.
I'll have to get the name for you.
It's when we did product recommendations.
I'll look it up for you later.
But you can find the products that are most similar chemically to ACDC.
And then you would basically find the inventory items in Seattle that are the most similar
to ACDC.
So you can answer that question.
So you can do really accurate product recommendations.
And then unfortunately, I don't think this script will run fast enough.
But I'll share this with you.
My apologies that I wasn't more prepared in demonstrating this code.
But some of the other questions you can answer are, oh, you know, who sold the most in November?
Who were the top 10 retailers?
How much cannabis was sold in 2022?
What was the average sales per retailer?
And I think these are interesting data points that I would argue a lot of people don't know.
And so long term, I would love to publish this knowledge, right?
Because I would love Cannlytics to be an authoritative source on cannabis statistics.
So for example, if we're publishing these statistics, oh, you know, who sold the most
in November of 2022?
We've got the data source.
We've got the statistic.
Well, maybe somebody may ask ChatGPT, you know, who sold the most cannabis in Washington State?
I wonder if they'll know.
Oh, that's interesting.
So it looks like they're only trained up to 2021.
So we still have time.
But anywho, GPT4 is coming out this coming year.
So we'll see.
Okay.
So it looks like they don't necessarily have that information.
So, but anywho, that's sort of the grander scheme is, oh, let's just be an authoritative
source of data.
Short term, how can you make a buck?
Well, in the couple of weeks ago, we went over a REMA forecasting.
So what you could do, and I often have these grand schemes, but I never really get around
to them.
So I always encourage people to capitalize on my, some of my business model ideas, because
as I said, I may not necessarily get around to them myself.
And as I said, there is more than one restaurant out there.
So there's no, there's no, any reason why Caneladix has to be the only provider of cannabis
statistics.
So for example, now that we have sales by licensee by retailer, well, you could just
reach out to the retailers and say, Hey, like, how well are you monitoring your sales?
Would you like to see how well you did compared to everybody else?
Well, guess what?
They're going to, they're going to want to know that.
And then you could also say, well, and once again, a lot of people deride forecasting
because in forecast should be taken with a grain of salt, but I still think they're,
they're at least useful to get my saying is any projections better than no projection,
you know, don't like put all your faith in it.
I mean, it would at least like help to like, say your sales are trending up.
What did it at least be like good to get an estimate of what they would be in the next
year?
Because they're probably not going to be the same if they're on a positive trend.
So you could at least get like an estimate of what they may be in the next year.
So you could at least offer all the retailers, Hey, you know, would you like to see your
statistics from 2022 and we can get you a forecast for 2023.
So right there, I think that's a, like a business proposition that's so rich that I won't even
be able to capitalize on all of that myself.
I mean, think about it.
There's a three to 400 retailers and we could potentially help them all out with rich data
and get them forecast for the next year.
And as I was even saying in my forecasting episode, a smart retailer would get as many
forecasts as they can.
So there's no reason why a smart retailer couldn't employ me and you to create a forecast
for them.
And as I said, let the best forecast win, right?
So you can measure the root mean squared error, the forecasting error.
And so that's, I think what some fun competitions would be.
I was thinking like, I never formally wrote out instructions, but that I talked about
it last year.
Oh, why don't we have a competition of who could forecast sales for the best in 2022?
So if any of you cooked up a forecasting model, you know, definitely share and we can compare
our results for fun.
But also thinking about things like that for the coming year.
But I've been talking way, way too long.
Unfortunately, as I said, I spent almost too much time writing code and not enough time
preparing a presentation.
So my apologies that things were kind of, I've been sort of flying by the seat of my
pants and just maybe not have had like as concrete of a plan as I should have.
But basically the idea behind today was to share with you, hey, there's all this CCRS
data.
We're finally starting to get our teeth into it and curate it.
Here are some interesting statistics.
Here are some that I may not have time to get around to.
See if we can't put our great minds together and make the at least the cannabis space a
little bit better molecule by molecule.
But that's my spiel for today.
Anyone have any thoughts, comments, questions before we get on to the new years?
Ooh, Isaac, what's on your mind?
Hey, yes, actually do have a question.
I'm just wondering how often do they update the database?
I believe the last time I accessed it, it says something like November data.
Do they just add in new data or there's like new Dropbox folders for each month?
They will create a new Dropbox folder.
And you don't actually have to wait around on us.
He's actually Jim McCray to give him proper credit.
And he'll share the language so I can share the language with you.
Periodically, he'll just send an email to the Washington State office and say, hey,
could I get the latest data?
And they'll send it to him and he'll share it with the cannabis data.
And he just does that about once a month.
And I think enough people request the data that they generally just wait and give everybody
the same link.
And it's around the 20th of each month.
So around the 20th, I think, is when you can expect the drop.
And so that's pretty good.
I'll find out if we can get it more frequent than that.
But Sammy, did you have a question?
Oh, well, I was just going to ask.
I'm also like actually looking for more work.
And so your company is based in Washington.
That's also like you're trying to only study Washington.
Is there an interest to extend this to Oregon or make sort of parallel things?
Absolutely.
The reason is just being it's manageable in a starting point.
So one thing we stress here is when you've got a big daunting task, just start somewhere.
And then that's at least a start.
And then you can think about where to proceed from there.
So we're just starting in Washington state really just for no better reason than that's
what's available.
Definitely would love to expand to more states, right?
Oregon, Massachusetts, everywhere under the sun.
Can you remind me what was the second part to your question?
Well, no, I'm just I'm like this looks interesting.
I'd like to work on this.
Or something inspired by this, like I said, like I'm here in Oregon.
So presumably there's similar data sets available.
But also like, yeah, no, I don't.
Like I said, this is my first date, first time with with your your meetup.
And I don't I'm not sure exactly what the goals are.
But I'm interested in what you're doing.
I'd love to talk to you basically about about this outside of here if you want.
Oh, yes.
Sorry, I don't think I fully answered your question.
But as I said, I'll think about it and pick pick up more next week.
And oh, yes, that it just jogged my memory.
So it's not exclusive to the cannabis space, but here's an example from the cannabis space.
How do you actually start an effective startup?
Well, a way is start locally and do it well.
So an example from the cannabis space is Ease.
They're a delivery company.
Guess what?
When they started, they were exclusive to the San Francisco area, from my understanding.
So I don't even think they touched other jurisdictions in California.
I think it was a very, very local thing.
They figured it out there.
They made sure they could do it there well and profitably.
And then you start looking to expand.
And there are other delivery companies.
Like I said, there's there's many restaurants, right?
I don't think there should just be one company doing something.
But they were definitely a successful transportation company.
And they just started small, started locally.
And as I said, they may be an exception to the rule.
So maybe that's it, right?
That may not necessarily be the way if everybody knew how to start a startup.
Yeah, no, I I know precious little about the business side of things.
Like I'm interested in the data science, like but I so yeah, I don't know.
Maybe we should talk outside of this.
I don't know.
Yes, let's all keep the conversation going.
We'll talk and then I'll get you on the slack.
So that's the idea is, you know, we come here to have a nice round table just to touch base.
And yes, let's keep the conversation going, because there is a need for awesome data scientists
everywhere.
So we'll start wrangling data in Oregon, wrangle some data in Massachusetts and Florida and
Oklahoma everywhere under the sun.
And I think we can learn from each other.
If I discover a new technique, maybe you can use it too.
And you know, if you have an idea for a way that we can improve, we'd always love to hear
that because we're always looking to improve.
So I think we can.
Right.
That's the idea behind kinetics.
And that's why we build open source projects and work on open data projects is we believe
in the win win.
So it's a long term game.
But we're here.
We're in it for the long haul, and hopefully we can have some fun while we're doing it.
Well, too cool.
Thank you all for coming and lending your brilliant eyes, your ears, your minds.
It's you that are really helping advance cannabis science.
I couldn't do it without you, right?
It's you putting your interest in here.
That's that's helping move the cannabis industry forward, even if it's only molecule by molecule.
I think it's helping.
So thank you all.
So sorry, I was just gonna say, how can we get in contact with you outside of here?
Okay.
If you all hold tight, and you can go there and then I'll put the link to the Slack channel
here momentarily.
I think.
But like, but yeah, I'll get the slack for you.
But as I said, there's a lot of work to be done with the cannabis data group.
So that one's one that's really picking up steam.
There's really concrete tasks that can be done with that one.
So this link will let you join the slack.
And hopefully we can make that a bit more active.
So currently, Candice and I do a lot of conversations there.
And as I said, I'm going to start doing a better job about posting jobs, interesting
articles, of course, data sets and things we're working on, trying to just get people
more engaged into the coming year.
They say you're going to start doing some competitions for t-shirts, start sending people
to conferences.
I'd love to get you all involved in data science projects.
A lot on the table for the coming year.
So definitely be in touch because want to share all the fun with all of you.
So Camlytics is you and it's a relatively small company right now?
Exactly.
So essentially, Camlytics is me plus all the cannabis data scientists that love to help
out.
So it's a small open source project at the moment, but as I was saying, it's kind of
like a flame, right?
So we just kind of need to nurse it.
In the past few months, I let the flame burn a little low.
I don't know.
As I was telling you all in the past few weeks, I just need to do a better job of focusing
on open source because that's what I believe in.
I think that's a nice long-term sustainable endeavor.
And I also just need to do a better job of prioritizing and getting people involved.
So I got behind on getting the recordings up.
But I've found new tools like OpenAI that can help me expedite that process.
And then I don't think I was doing a good enough job getting all of you involved because
there are ways that you can help out in the projects.
I don't know.
We all have things we're good at and things we're not good at.
And communication is not necessarily one of my strengths.
So that's why I encourage you all to don't feel bad about poking me because if I don't
reach out to you, that doesn't necessarily mean that I don't need your help.
I just didn't communicate that.
So feel free.
Just poke me and say, hey, is there anything I can help on or this or that?
And I'll try to get you involved.
All right.
Too cool.
But yeah, poke me.
And in fact, this is a valuable lesson that I learned and didn't take heed until way
too late in life.
The squeaky wheel gets the oil.
Be squeaky.
Squeak, squeak, squeak, squeak.
Don't feel bad about impeding on people.
Don't be shy because that's actually selfish because you're thinking about yourself.
Instead, think, oh, people actually do want to be engaged.
So be squeaky.
Engage people.
Ask.
And as I said, it doesn't hurt to ask, and if someone says no, don't take it too heavy.
Just move on.
So ask, be squeaky, get involved, and I think we've got an awesome year coming up.