Without further ado, welcome to the Cannabis Data Science Meetup Group.
Thank you for attending.
So we've got a jam packed day today.
So going to be doing, as always, data science.
We've got on the agenda forecasting.
So we'll be picking up with that.
So we started looking at forecasting last week.
We looked at a quite simple forecasting model.
We're going to extend upon that today.
So as we always do, we just build upon each week, provide some building blocks, and then
just keep adding, adding, adding, and getting more advanced, improving as we go.
So last week, we did forecasting with a vector autoregression, a VAR model.
And we looked at monthly data.
We saw some flaws.
So monthly data is maybe not the ideal frequency.
We were only able to look at sales data, and we had to use an a theoretical approach.
This week, we'll also use an a theoretical approach.
However, we can apply economics to our forecasts.
That will be interesting.
And we'll use a different forecasting methodology.
So we'll use what's called Box-Jenkins methodology.
And we touched on this in Saturday Morning Statistics this past Saturday.
And so you're all welcome to join us for Saturday Morning Statistics this coming week.
And we will dive into more statistics.
So we may introduce some brand new statistical models.
So we may continue working with time series data.
However, I've got some interesting statistical models to show you.
So we can do some quite interesting analysis.
And I'm going to save that for Saturday Morning Statistics.
So that's a bit of a teaser.
So definitely stay tuned, because I definitely try to give a little something extra for the
people that show up to Saturday Morning Statistics.
Since it costs $1 for you, I hope you can get more than $1 value out of it.
And your time to show up on Saturday Morning.
So I'll try to make that valuable.
So without further ado, we'll jump into today's forecasting and economics.
And sprinkle in some data science and cannabis talk where we can.
So I'm going to go ahead and start the presentation here.
Awesome.
So like we said, make sure to check out Saturday Morning Statistics, because we've got some
big plans coming with that.
So we're going to use what's called an ARIMA model today.
So we looked at a vector autoregressive model last week with vector meaning multiple variables
here.
This week, we're going to look at variable by variable.
However, we're going to extend upon the model.
So we're going to add in an integration component to make sure that we are not breaking any
assumptions when we're forecasting.
We're going to add in a moving average component to capture some cyclical trends.
And what's cool about this package is it's built ready to go with a bunch of features
that we would like to use, in particular, the auto ARIMA.
So this is where we, you know, one would almost call this an application of machine learning
if you continuously feed in the data as it arrives, right?
Because you're basically letting the computer select the best forecasting model given the
data and your parameters.
So you're just making sure we're still connected here, correct?
So pardon the interruption, but Cheyenne, we are still connected, correct?
Yeah, we are connected.
Yeah.
Awesome.
So anyways, moving forward with the ARIMA model.
So we shared this script in Saturday Morning Statistics, where this is a ARIMA model that
I wrote a few years back, back in 2017, and I've knocked the dust off of it.
So you know, it still runs fine, a nice maintainable code here.
And so this will, you can either forecast with a training period or with the minimum
BIC.
The algorithm I wrote only scans for optimal P's and Q's, and it doesn't have a lot of
interesting features built in, like exogenous variables.
So what we like to do is, you know, you can prove that you can do it by yourself.
So here, you know, we coded up our own auto ARIMA.
However, we can stand on the shoulder of giants and we can use the auto ARIMA package.
And this is published by Alkaline ML.
And you know, there's a handful of contributors, you know, 15 contributors.
It's used by a thousand plus people or organizations.
So you know, it's a little, it's arguably more vetted than the ARIMA model that I wrote
myself.
So instead of using this code, which we saw can be used to make forecasts, we'll go ahead
and use the PMD ARIMA package today.
All right.
So that's the programming.
We talked about the statistics.
Now time to get into the cannabis data.
So first things first, get the tools we're going to be using.
Next, we're going to get the data.
And so if you've tuned in in prior weeks, these are all of the data points that we can
get from the Massachusetts Open API data.
And so we're given a rich set of data here.
So we have, you know, so we have the sales by product types here.
You have prices.
It's an awesome data point.
You have a licensee data.
And so here you could look at a single licensee.
And so you have a plethora of data points for each licensee.
And we will actually capitalize on some of these data points today.
And a teaser for what's coming up is we're going to calculate an interesting statistic,
several of them, that I'm not certain that anyone's calculated before in Massachusetts
because there's maybe just not that many people looking at this data.
And there's many novel ways to look at this data.
And I believe we're going to look at the data in a novel way today.
So that's a teaser.
And so there's going to be one of these data points that we're going to capitalize on.
And so, you know, there's many interesting data points here, right?
You've got the geocoded location.
You have when the license was issued.
You of course have the name.
You have tons of interesting variables.
You have square footage, which I think would be an interesting analysis of its own, right?
So you could look at the square footage required for cultivation and see how that's trending
with the distribution.
I think there's a lot of interesting analysis that can be done there.
Of course, just seeing how the licensees are distributed geographically.
There may be some analysis that can be done with license fees.
So long story short, lots of data points here.
We'll be using one of them shortly.
Then of course, we calculate our sales data here.
And note to the newcomers that we're adjusting for outliers.
So just coding outliers is zero.
So we've gotten all our data here.
So for example, we can look at sales data.
And also for the newcomers, Massachusetts is interesting because unlike a lot of other
states where you saw a spike in cannabis sales right at the start of the pandemic, in Massachusetts,
they suspended cannabis sales for approximately a month.
So you have the exact opposite effect in Massachusetts as you see in other states.
In Massachusetts, sales dropped to zero.
In other states, they spiked.
So this actually opens the door to rich comparative analysis.
So you can see potentially how policies in Massachusetts may compare to other states.
What type of comparative analysis you would do, not 100% sure yet.
We are going to introduce some models.
So maybe teasing a little too much.
We're going to be introducing some models in Saturday morning statistics that specifically
let us do this type of comparative analysis where we can look at these breakpoints in
time and let these aid us.
So long story short, we'll tease that and keep moving forward for now.
But we'll get into the juicy bits now because there's a lot of ground to cover.
And I want to estimate a lot of statistics and create a lot of forecasts today.
So to do that, we're going to use our handy dandy PMD-ARIMA package.
So you'll need to pip install PMD-ARIMA.
If you're using another programming language, you may have to find another tool for the
job.
But it doesn't matter.
This is just statistics.
It can be done in any programming language.
So we're going to calculate some statistics here.
We're going to calculate weekly series.
So here is a tip slash trick that I shared with Saturday morning statistics.
I'll also share it here because it's such a useful tip and trick.
But they got it a week ahead of time in Saturday morning stats.
But long story short, a nifty trick is to use weekly series for forecasting the medium
term.
If you're going to be forecasting the very short term, like one week ahead, definitely
I would use daily data.
So if you're going to just, or even a month ahead, I would just use daily data.
So you could just use recent daily data.
I would try to add in day of the week effects because Fridays are probably different than
Mondays and Sundays and what have you.
So daily data is fine for short term forecasting and you can make do for long term forecasting.
As we saw, we can aggregate it into monthly series.
However, we lose a lot of the variability when we move to monthly series.
So I found that the sweet spot for forecasting the medium term, and when I say medium term,
I mean three months to two years.
Two years is sort of pushing it.
So maybe three months to one and a half years would be medium term.
If you're going to be doing two plus years, two to five years, that's going to be long
term.
And then, you know, all bets are off, I think, in 10 plus year forecasts.
But people make them.
But I think there's just so many structural changes, especially in this day and age that
will occur in 10 years that it's maybe not even worthwhile forecasting.
But you can still do it just for a mental exercise.
But you know, everyone's interested in the short to medium term.
So short term, really useful for general managers, right?
So if you need to know how many employee hours to employ the next week, so how long you need
to have your employees there, what days you need to have your employees there, what time
of day you need to have your employees there.
Short term forecasting works well.
Then for more of the executives, right?
For in the investors, they're looking at more of the short to medium term, right?
Because they want to see, okay, is the business going to potentially start to generate a return?
Or is the return going to be increasing or decreasing?
You know, and those players have slightly longer time horizons.
So today we're going to be doing a medium term for forecast.
So we're going to be forecasting the next year, essentially.
So the remainder of this year in 2022, that's sort of how I like to do things, the remainder
of the year and next year, just personal preference.
So we actually may need to crank up the number of weeks here.
So let's just crank this up to the next 60 weeks.
And we've here mean drone on and on.
Let's look at the data.
Early sales.
And we're also going to be looking at the number of plants grown.
This is the total number of tracked plants, vegetative and flowering.
So, and another interesting variable, we've got it.
So let's look at it.
We're given employees.
So we're going to aggregate this by the week, taking the average number of employees employed
at any given time.
You see quite a large growth rate.
So the number of employees has grown five times in the past year or so.
And so that's impressive.
So you've seen a large growth in this sector.
So a large number of employees are entering.
Awesome.
I'm going to jump ahead real quick.
So before we get to forecasting, which we're going to do shortly, I'm going to go ahead
and introduce to you a brand new statistic, which I just calculated this morning.
It's not perfect.
And I'll point out the flaws.
But this is a statistic that I believe you can calculate with the data given.
And I'm not certain if anyone's thought to look for it before.
So long story short, I wanted to know what is the total number of retailers in the market?
So we're given licensees.
So I thought we've got our licensees data here.
I thought, OK, why don't we just count the number of licensees that are retailers?
And we can do that.
Well, there was probably not 379 retailers on day one.
So I started thinking, man, that's unfortunate.
It would be nice to have a count of the retailers over time.
Well, we're given, right?
So if you look at one of these retailers, we're given when their application was created.
Oh, and I actually just thought about how we may be able to account for exits.
Because basically what I was going to say is, well, we can see when these licensees
entered.
So for example, this licensee, we're going to say entered on April 28 of 2021.
So the way we can potentially do account of the total retailers is we can go through time,
look at all the days, and simply add up all the retailers that had their app create date
before that given date.
So let's say this is a market of one.
Then you're going to have 00000000, and then for all the days prior to 2021, 428, then
all of a sudden on 2021, 428, our total licensees increases by one.
And then it's going to be one for perpetuity.
I just realized now that we were also given the activity date.
So in one cell swoop, we may also be able to control for licensees that have exited.
And I just had this idea.
So bear with me while I adjust the code and see if this still works.
So we'll listen to it before and after, right?
So basically, what I'm going to do here is I'm going to create a series, total retailers,
total cultivators, and total licensees.
And I would like to turn this into a time series where we can look at the total number
of these license types and total licensees over time.
So we can see how the total number of retailers may be growing or decreasing over time.
So let's just go ahead and code in the activity date part now just to go ahead and be precise
about this.
Actually, let's do it before and after.
So this is what I calculated this morning where I just say, OK, this is everybody.
For example, I'm going to count the total retailers as where the licensees, license
type is equal to, uh-oh, this should actually be retailer.
So it didn't do it 100% right this morning.
But here, I want to say, OK, where the license type is equal to the marijuana retailer and
the app create date is less than or equal to that time stamp.
And here, I'm just iterating over the production, right?
But here, I'm just iterating over the index.
And I'm saying, OK, the total retailers is where the date is less than or equal to 1015,
less than or equal to 1016.
So let's just do that, and then we'll account for exits.
OK.
We need some packages here for plotting.
Let's try this one more time.
Awesome.
So here, we actually, this is my first go at it.
We'll adjust it in a second.
But this is the first go at creating.
Well I kind of jumped the gun.
Let's go back to this.
So this is our attempt at calculating total retailers over time.
And so, well, here, I actually aggregated it into the week.
But we can look at it daily as well.
So you can see, going up, a real interesting curve here, right?
And so it looks like the market is reaching almost a natural level here of retailers
that the market can sustain.
Right?
So you have retailers entering.
You start off with, what do you start off with?
Start off with 118 retailers in the market.
And you end up with 379.
And there's the projected path.
Let's go ahead and correct this real quick.
And right, because just because a retailer entered,
they may have had a faulty business model.
Or they may have cut some corners and got dinged and got their license suspended.
You know, it's not impossible.
Like, these things happen.
Or they decided mining cryptocurrencies more and more profitable,
and they exited the industry entirely.
So who knows what's going on?
But there's always exits.
And this is something that's rarely, or not rarely,
but proportionally neglected in the field of economics,
is the way I like to phrase it.
Not many people like to talk about or research exit.
Right?
Like, of course, people are looking at market entry.
Right?
And so that gets a lot of focus.
And then, of course, production gets a lot of focus,
how people are actually operating and performing in the market, right?
How profitable people are.
And profitability has to do with your exit.
Right?
But there's such a thing as a strategic exit.
Knowing when do you exit?
Just because you're in the red doesn't necessarily mean you should exit.
Right?
So it depends a lot on your fixed costs, your variable costs,
the price in the market, the future trajectory.
So there's a lot of factors that go into exit.
So I think there's a lot of analysis that can be done simply
on predicting firm exit.
So if we're able to look at when these firms exit, why?
Are firms in particular geographic regions exiting at different rates?
You know, there's so, right?
So that may be something to pinpoint, right?
You may want to look at a map and see if there's
certain geographic areas where there's a high exit rate.
Right?
And people there may need a bit more business assistance or what have you.
So long story short, lots of fruitful analysis that can be done.
Without boring you too much on that, let's see if we can't parse out exits
by the activity date.
So first off, let's just look at this activity date
and see what's going on with it.
It looks like, unfortunately, we're not going
to be able to get exits out of this because it looks
like there's only one unique value.
So as much as I was just hammering on about, that's unfortunate.
So as much as I was just hammering on about,
as I was just hammering on about exits, I'm not sure if we can actually
get exits out of this data.
So we have when the app was created.
Ch-ch-ch-ch-ch-ch-ch-ch.
And we may want to check the application status to make sure no one's
got unimproved.
But I think, unfortunately, we're not going to be able to parse out exits.
So I am going to have to hedge all further analysis
about these specific statistics on the fact
that it's not going to account for exits, which is sort of a big deal.
Let's just see what's going on with the application status just
to see if there's any suspended licenses or anything of that sort
that we may need to be worried about.
So I'm going to go ahead and keep steamrolling forward here.
However, I think this is worth looking at for other interested parties
is I'm just going to go ahead and count all the retailers,
even if they potentially have exited or we're not accounting for exits.
So that may bias historic values.
Plus, I'm not going to be accounting for this approved license type right now.
So this may be worthwhile looking at seeing, OK,
is this a provisional consideration, provisional license,
final license, or in process?
Because if they're in process, they may not be actually selling yet.
So you may just want to look at the total number of final licenses over time.
So you can do that.
The numbers will change.
But I'm going to leave that for interested parties
here because I don't want to get too bogged down in this.
But what Canlytics is going to be doing is we're
going to be recording these licenses over time.
So that way, we actually can get an accurate count
of the total retailers, total cultivators,
and all the other businesses over time and an accurate count of entries
and exits with those accurate counts that will open the door to rich analysis
of market entry and exit, which we were just going on and on about earlier
about how understudied and potentially understudied.
And that's an opinion.
But in my opinion, it's understudied.
And there's fruitful insights that can be made.
So without getting you too bogged down in this,
why was this data interesting?
Well, like we said, we could look at just the total number estimate.
Remember, this is an estimate.
These numbers have bias because I wasn't able to effectively account for exits.
These are biased numbers.
They're inaccurate or they may be inaccurate.
But I always think a measure, even if it's imperfect,
is better than no measure because before, all we had was a single point number.
All we had was the total number of cultivators today, which was 285.
Well, with using the data we're given, we're able to estimate.
I want to make this super clear here that we're inferring here.
We're making an inference.
This is not a precise statistic.
That's why it's a statistic.
It's not a data point.
Well, it's not an observation.
It's a statistic.
So anyways, there's someone who knows more about statistics
than I do for the correct terminology.
But anywho, we're going from about 100 cultivators
at the very beginning to almost 300, 285.
So almost a 200% increase, so not bad.
Anyways, let's stay on focus here and keep calculating statistics.
We'll also look at the total number of licensees.
So these are interesting time series.
So like I said, these are imperfect.
But we've just created three brand new time series here.
And I would always encourage you to keep track of, find, record,
and utilize time series data.
So for those of you joining us, time series data,
it's basically right here.
It's a pair of observations.
You're given a time and the value at that time.
And so we're all used to this, daily weather.
We're all familiar with time series.
However, if you define it, there's
a lot of interesting statistics you can do, such as forecasting.
So all we're going to be doing with forecasting,
we're not going to be using any economic models or theory.
We're just going to be using a time series itself.
So we're going to be saying, given this time series,
can we extend this out a bit?
Because you say, like your manager said,
oh, will you predict this forward five months?
Well, a naive prediction is tomorrow.
And this is actually the definition.
The naive forecast, tomorrow's for the best naive forecast
is tomorrow's forecast is the same as today.
That's the best you can do.
And that's a pure autoregressive forecast.
So you're just going to say, OK, whatever today is,
that's what tomorrow's going to be.
And then that's what the day after is going to be.
It's better than nothing.
It's better than saying zero.
And so a naive forecast, you just draw a straight line out.
And that would be your forecast.
That's just sort of naively looking at past values
and playing it forward.
Well, we can get a bit more sophisticated than that.
So we may be able to parse out some cyclical behavior
or what have you.
These series don't have too much variability.
So they may not have rich, rich forecasts.
But that may be expected.
We may not really expect the total number of retailers
to waver that much in the coming months.
So if we look at the numbers, we're
going to see that we've had 379 retailers
since the beginning of October.
So it looks like this number here is quite steady.
So it looks like we've had about, in Massachusetts,
there's been about 379 retailers going on the past three months
now.
So things have maybe stabilized.
So that's stabilizing.
But what did we see was all over the board?
Sales, all over the board.
All over the board.
Like we saw, as you start aggravating,
it's going to get slightly smoother.
Weekly sales starting to smooth out.
Monthly sales would be even smoother.
But still variability.
Well, that means there's going to be
variability for the retailers.
So what would be the best way to do that?
Retailers.
So what would be awesome would be
to have the actual sales per retailer.
And in some states, like Washington state,
you can actually get the total sales per retailer.
So you can do a Freedom of Information Act request.
We've done this in previous meetups.
You can get the sales data in Washington state.
I actually calculated this statistic,
but still on my long-term to-do list.
Would love to-do list.
And that would be to calculate sales per retailer per day.
So then you could actually get a breakdown
of market concentration.
You could actually calculate the market portion
for each of the firms, each of the retailers at least.
So there's a lot of real interesting analysis
you can do in Washington state.
The problem is 100-plus gigabytes of data.
It's hard to work with.
So that's why we have to leverage powerful tools.
And I can let X and others provide us.
So for here, we're going to do an estimate,
not going to be exact.
So we can at least create a benchmark for the retailers.
So we can say, OK, retailers.
OK, what's the average number of retailers
that are present on any given week?
And then we can say, what's the average number of sales
on any given week?
And then a useful benchmark would just
be what's the sales per cultivator?
Awesome.
Awesome, awesome, awesome.
And this is a number, a statistic, an estimate.
An estimated statistic that business managers,
investment firms or individuals would be interested in.
So this is the average amount of sales per retailer.
And even your general manager is going
to be interested in this.
Because you want to, and this is by week.
And you could aggregate it and do it by month as well.
So this is the average sales per week per retailer.
So if you're a retailer, are your sales above or below average?
So if you're at this date only doing about 20,000 a week in sales,
then you're underperforming potentially the average.
And so if you're a general manager,
you may want to be looking for ways you can get lead and get improved.
Get your profits up.
Maybe you need to do some advertising
or find a better location or you name it.
Conversely, if you're significantly above the average,
say you're doing 100,000 a week in sales,
well, you may be feeling a little good about yourself.
You may want to make sure you're not slipping.
You're staying above average.
You may want to do a self-analysis and say, OK,
what are we doing that makes us perform above average?
And maybe we can capitalize on that.
So if you can figure out why you're doing better than others,
maybe you can do more of that.
So maybe you can find your niche or your comparative advantage.
And you can just keep performing better and better.
Or maybe you're looking for investment.
You can go sell yourself to investors and say, hey,
look, the average sales in Massachusetts is X, around 80,000.
We're performing 25% above average.
Invest in us.
We're one of the outliers.
So that's something interesting you can do.
Well, hey, the cultivators say, yeah, you're leaving us out of all the fun.
Well, don't worry.
We've got something for you, too.
So for the cultivators, well, we've calculated the weekly number of plants
going up and up and up.
Well, the total number of cultivators also going up and up and up.
Interestingly, hitting a plateau.
Quite interesting.
Well, how can we gauge the performance of a retailer?
I mean, of a cultivator.
It would be awesome to have wholesale sales.
Short of wholesale sales, we can use plants.
And so this will proxy the size of the cultivators.
You know, it's not perfect.
Some cultivators have different growing styles.
So maybe your growing style favors more plants or less plants.
So you may grow these big bushy plants, these ginormous plants.
Or you may do what's called like a sea of green approach, where you just have tons of smaller, shorter plants.
So you may.
I've heard of people stacking plants.
I've heard that may not be super successful.
But what do I know?
You know, I'm not a cultivator.
So all these cultivators, I'm sure, have different growing styles.
So one cultivators plant is probably not equal to another cultivators plant.
So it would be awesome to maybe look at yields or what have you.
Like I said, wholesale sales, it's hard to just beat sales.
But but anywho, there's other measures.
We're going to use the data we're given.
And we'll just calculate plants per cultivator.
And I think this is interesting.
Right. So what you see is.
From what data is showing, there's a there's been a recent dip.
So like we said, we can't read in to to too much into these statistics, but it's like, you know, what's why, why the dip?
Why the dip?
You know, there's still the same number of cultivators, but there's just not as many plants.
So, you know, this could be harvest season.
But I just don't know if that's harvest season, right, because.
You know that there's been prior harvest seasons and we haven't seen that this extraordinary dip.
So, you know, it may have been economies of scale.
Right. These growers, they may have reached their minimum long run average cost.
And I'm not going to get into.
Cost functions today, maybe another day.
This is actually something that I know pretty well.
So.
So long story short, these cultivators may have been exploiting economies of scale.
Right. So they're going to keep growing more and more plants at a lower and lower cost.
Lower and lower average cost per plant.
However, the average cost per plant can only get so low.
And then at a certain point, if you want to get produce plants above that, your average cost is going to be increasing again.
So.
You know, right. So it's like, you know, if you really want to push the gas, you know, that much further, you know, like you want to you really, really, you really have to add that a thousand in first plant.
You're going to need a whole other facility costs are going to go through the roof, you know, so there's, you know.
There are like limits to these economies of scale. And so I'm not, I'm not by any means saying that these economies of scale have been reached.
But I just find it interesting that you see this dip. It may be entirely.
Explained by other factors, right. There could be a lull in the business cycle. There may be something going on in the capital markets. So people may have a hard time getting the investment they need.
There could not there could be there is something going on in the labor markets, right. And so these are.
With production inputs, right. So if you have something going on in the labor market that could affect your capital supply, like, you know how much capital you need to use.
So there's a lot of factors going in here.
But once again, we're given a rough measure. So if you're a cultivator and you know that you're only growing 100 plants.
At any given time, then you're, you know, significantly below average.
But maybe you're a cultivator and you've got 10,000 plants.
I'm not 100% sure what the regulations are in Massachusetts on plant counts. But there's undoubtedly people that have above average plant counts and people that are below average.
Just because you're at a below average plant count doesn't necessarily mean you're.
Running your business poorly, right. Maybe you can yield more with 600 plants than somebody can yield with 1000 plants.
Who knows. So, but it's a measure, right. And that's where we're here to say we're here to say that a measure is better than no measure.
And so, you know, at this date, we can say that, you know, the average number of plants per cultivator is about 800.
800 plants. So that's a sizeable amount. That's more than a home cultivation for sure.
So there you have it. Some brand new statistics. So
just to keep adding statistics on statistics, it's like how many statistics can we do in five minutes? Well, about to show you we can do a
lot of statistics in five minutes. So let's do another one. Employees per licensee, right.
So we saw that, oh, we can calculate the total number of licensees.
Estimate the total number of licensees. Check that box.
Well, we can see how many weekly employees there are.
Right. And so this is interesting, right. Employees going up and up and up and up. Right. So maybe it's not the labor markets that's explaining what's going on with plants, unless there's something funny going on with wages, which
easily may be, right, because we were looking at prices and inflation in prior weeks. We were having a hard time estimating the competitive wage in Massachusetts.
If you'll look below, that's on our to do list to revisit that. So perhaps next week we're going to revisit competitive wages and interest rates in Massachusetts, because I still want to estimate those and forecast them.
And I believe we were saying that wages were rising. Don't quote me on that. They may have been falling. Wages may have been falling. But for some reason, I want to say they were rising.
So whatever's going on with the price of labor wage and the price of capital, the interest rate, that's going to affect your production inputs.
So one of those is labor. Once again, you're going to have big firms and small firms, but we can see, okay, about how many employees are there per license.
And we found this interesting. So looks like at the very beginning, they're short staffed. You know, you haven't, like you're starting with really few employees here.
So you're starting to see less, you know, so these first days may not really be very representative. But, you know, let's just say you're out in January or so, you have around two employees per license.
And so you see the firm size is increasing over time, and it hasn't hit a plateau yet. So there could still be economies of scale in labor, right?
So this would be specialization, right? So the more employees you bring on board, the more specialized they can get, right?
So when you only have two employees, they're going to be doing almost everything. And you see this in businesses, right?
You see maybe the founder and the partner or the first hire, just doing an extraordinary amount of tasks.
And then they keep bringing on more and more people and you can get specialized. So I work a lot with laboratories.
And so you'll see this at the laboratory. So you can bring on new chemists and new analysts. And these chemists may at first do many tasks and then they can get specialized.
So you may eventually have a chemist whose specialty is testing for heavy metals or another chemist whose specialty is testing for aromatics.
So that would be terpenes in your residual solvents. And you would have another person who excels in microbiology.
They're in the micro lab. So you can start specializing and the same is true with the cultivation.
I'm sure the same is true in retail to a certain extent. So you can get people to specialize.
Awesome. Well, we promised forecasts. So let's forecast.
And I'm going to just do it briefly and then we can go back into it next time.
So long story short, we're using these historic time periods to forecast.
And I'm going to do this more in depth next time. I'm just going to show it to you today just so you can see it.
And then we'll go into it next week. And then for those of you in Saturday Statistics,
we can do some some real interesting comparative analysis and we go a little bit more in depth in Saturday Statistics.
So make sure you make sure you attend if you can. So without further ado, we've got our weekly sales.
Define our forecast horizon the next 60 weeks.
I'm going to go into this more in depth next time, but I'm going to add month fixed effects.
So we're basically going to add an effect for which month it is.
So we're going to say control for the fact that January's may be different than April and April may be different than August.
We need a baseline. So I'm going to exclude January. So we basically going to compare every single month to January.
So April is X percent different than January.
August is X percent different than January, so on and so forth.
So we're going to look at the month fixed effects and then we're just going to use past historic values using the auto arema model.
Which you could argue is a form of machine learning if you repeatedly feed it new data.
And we're basically going to let the computer and statistics fit the best forecasting model for us.
So we're going to let it try a bunch of different models, whichever model fits best in sample.
We're going to use for predicting and then it's just playing it forward using past values to predict future values.
So without further ado, also note I'm restricting the time frame from essentially August.
I mean, June of 2020 onwards because this avoids the gap where business was closed in Massachusetts.
There's a couple ways you could do this. One, include the gap. Two, exclude the gap, like cut it out.
But then that may mess up your frequency. I thought for simplicity's sake, since there may have been a structural change,
foreshadowing that occurred, that may have occurred during the pandemic,
then I think the forecasting model, in my personal opinion, is going to be different post pandemic than prior pandemic.
I still want to include a long time range and I don't like missing data.
So I arbitrarily picked 2020-06-01 to begin training for forecasting. This is my judgment.
Please use your own judgment when you're forecasting, right? It's not a perfect science.
There are judgment calls to be made and you should be upfront with your judgment calls to whoever you present your forecast to.
It's simple to say, hey, I made these assumptions. I started my training, my data on this time frame because of this reason.
Please take that into consideration. This is not the end all be all.
I don't know what the word is, but we can't foresee the future. We can just use statistics.
So long story short, let's just go ahead and play this series out. We're looking at weekly sales. Awesome.
So here I fit the model. We're using 85 observations. I'm estimating in a remote.
100. I don't actually think this is the best model, so we may revisit this next week because I think we need to integrate this to forecast correctly without breaking our assumptions.
And I'll explain more next week.
So long story short, short, you see 11 fixed effects. And so these are comparing all the months to January.
It actually looks because all the coefficients are positive. It actually looks like all the months are higher than January on average for sales.
And then here's our autoregressive component.
And this I believe are constant.
Oh, I wonder if I need to include a constant.
So this is something that we'll revisit next week because I may be leaving out the constant, which would force our model through zero, which is not what we want.
So once again, I'm just going to present this, but we're going to revisit this next week. Do it correctly.
And I'll explain it a bit more in depth.
But long story short, just to show you some plots before concluding today, thanks for staying a handful of extra minutes.
But we're basically, right, we made our forecast.
And I'm going to go ahead and beautify this real quick. And so if you tune into Saturday Morning Statistics, you'll see how we go about making these charts.
I've sort of boiled everything down right here, and I'm not going to go into it in depth. So this is why you should tune in to Saturday Morning Statistics, because I'll make sure that you can walk away at the end of the day knowing how to make a beautiful visualization.
Beautiful visualizations is almost what data science is all about.
He who controls the beautiful visualization controls the decision.
May sound not intuitive, counterintuitive, but it's true.
So arguably true. So long story short, try it yourself. Create some visual beautiful visualizations and
see what that brings you.
There's always some
ogling to be done. Right, so maybe where do you want to slap on this legend? But there's still some work that can be done with this visualization, but here you see our rough estimate of sales in Massachusetts for the coming year.
And we're adding the fixed effects for the months.
And you see our predictions are sales dipped to zero in January. Will they actually dip to zero? Probably not. Will they dip? Maybe.
So, and then this is where you know the models,
they lose their effective power in the future, right, because our model may kind of get the direction right, that sales may be dipping in January.
But the model, because it lacks a structural component, it may not really capture the effect that sales, you know, may rise in 2022.
Right, this is still our best estimate. So, you know, it's quite possible that sales will be within this blue region here.
If we were Bayesians, we could add a probability distribution to see how probable it is that sales are up here versus down here. So that's sort of the difference between the Bayesian forecast
and a frequentist forecast, which is what we just did. So there's way, there's many ways that you can extend upon these forecasts.
However, we were able to extend upon the monthly forecast we made in the prior week, and now we have
weekly forecasts with month fixed effects. So we have
a much more dynamic path for our forecasts here.
And so we're going to save this figure, and then in the coming months, we're going to plot the actual. So we're going to see how the actual trends
along with our forecast. And that's what's so cool about the cannabis data science meetup group is, you know, we're working with real data in real time.
So we can make forecasts, and then we can check them. So, you know, in the coming months,
in November and in December and then into 2022, please take this code and and make your own forecasts and check your forecasts and
keep at it, right, because maybe we can write this forecast in blue, and maybe next month, we can look at the data and we can make a new forecast and plot the new forecast in purple.
And we can see if our new forecast predicts better than our old forecast. So you keep iterating and you keep making forecasts.
What's awesome is we can also forecast all of these other series.
So we can forecast
and I'll be concluding here momentarily, but we can also forecast the number of plants
where we dip and we go up.
We can also forecast the number of employees.
So we can forecast how many employees there are going to be in the market.
Or in the end, yeah, working in the Massachusetts market.
Interestingly, we predict that, oh, wow, the total number of employees is going to fall off a cliff, and then maybe, you know, go up and stabilize around 8000 before falling off another cliff.
As we saw, doesn't really work like that with employees, right, because there's a cost to hiring and firing employees so you know your total number of employees doesn't quite vary like this so this may be a suboptimal slash poor forecast.
Well, I don't want to spend too too too much time here
and bore you to death and just drag on forever. So I'm going to go ahead and tease what's coming up in the next week, and we'll go even further than this, but this is at least what we're covered.
So I want to go back over the forecasting models just to explain, you know, how we are making these forecasts.
Then I want to show you how we can use this same forecasting methodology, right, we're just using time series, a single series to predict itself moving forward.
So we can also predict, right, the total retailers, so we can predict the total number of retailers that are going to be in the market.
We can predict the total number of cultivators that are going to be in the market.
And then we can predict the sales per retailer.
We can predict the plants per cultivator going into 2022.
So, you know, we saw that we could look at those historic values. Well now we can predict them forward. So now managers, executives, investors can make good decisions about the coming year, right, so we saw that oh you know the plants may dip to
800 plants per cultivator.
Well, how many plants per cultivator. Do we expect there are going to be in 2022.
Well, it will just take.
We could do it in about five minutes, but I'm going to save it for next time. But next time, or if you're adventurous you're welcome to take this code, which I am going to commit momentarily to GitHub.
You can take this code and calculate these yourselves.
So tune in next Wednesday, and we will predict these statistics. So we're going to predict sales per retailer in 2022 and plants per cultivator in 2022.
That way, if you're a manager, an executive or an investor of a retailer or a cultivator, cultivation, then you can have an expectation for what your sales goals may need to be how many plants, you need to get into the ground.
Right, or into the water if you're doing hydroponics.
So,
we're going to try to create some values and valuable statistics that people can actually use. So, we could do this in the next five minutes, but
we've already stayed an extra 15 minutes or so. So I'm just going to tease this, and this is what we'll do next time. And then, if we can get to it. I would like to also estimate the
competitive wage and interest rate. Right, so we can look at the historic competitive wage and interest rate. Well, guess what, we can also predict the competitive wage and interest rate into the coming year.
So we will not only have done a market analysis, but we'll have created a forecast of what the market, of how the market will perform in the coming years, in the coming year, where we can say how many players there may be in the market,
how many sales there may be, how many products are going to be sold, right, we could do how many plants are going to be grown in 2022.
And then we can estimate the competitive wage rates so we can do all the prices.
So, incredibly rich analysis.
And I think we've only just begun. So, it's, it's, it's exciting.
So, I'm going to go ahead and end the presentation there, but I hope you've gotten something out of it, Shyam, and everyone else.
So, until next week, definitely feel free to email me or message me your questions, comments, concerns, any ask, or even if you've got topics that you want delved in upon next week.
And I will get this code posted for you to utilize.
And I'm excited to share with you some brand new statistics next week, so we can look at even more novel.
First of the kind, first of the whole statistics next week.
There may have been some other entrepreneurial, adventurous people out there that may have calculated these statistics before us.
And we will hear from them. So it's always, it's always fun.
Fun time, data science plus cannabis, plus statistics, the cannabis data science meetup group.
So, as we always say, until next time, stay productive, keep your nose to the grindstone, and have fun.
All right, Shyam, I'm going to go ahead and end it here.
But thanks a million for attending, and I'll look forward to seeing you in Saturday Statistics.
And I've got some interesting models to share with you, so stay tuned and we'll have a good time on Saturday.