1 00:00:00,000 --> 00:00:01,000 Right. 2 00:00:01,000 --> 00:00:23,000 Right. 3 00:00:23,000 --> 00:00:43,000 Welcome to the meetup for the 31st. 4 00:00:43,000 --> 00:00:54,000 Got a jam packed day for today. 5 00:00:54,000 --> 00:01:00,000 So I was going to introduce to you forecasting. 6 00:01:00,000 --> 00:01:15,000 A book that I'll want to share with you is economic and business forecasting by john Sylvia and. 7 00:01:15,000 --> 00:01:27,000 Professor equal as Eric as Eric. 8 00:01:27,000 --> 00:01:40,000 Last week we talked about Colorado data and so there's been a lot of talk about what may happen in 2021. 9 00:01:40,000 --> 00:01:46,000 So we're going to build a forecasting model. 10 00:01:46,000 --> 00:01:58,000 First. 11 00:01:58,000 --> 00:02:09,000 We need to take into consideration that, you know, we need to, you know, maybe have a set in stone methodology for forecasting. 12 00:02:09,000 --> 00:02:17,000 So we're going to follow these 10 steps. 13 00:02:17,000 --> 00:02:21,000 Number one, we need to know what we're forecasting. 14 00:02:21,000 --> 00:02:28,000 Number two, we need to understand the purpose of forecasting. 15 00:02:28,000 --> 00:02:32,000 We need to acknowledge the cost of forecasting. 16 00:02:32,000 --> 00:02:35,000 We need to rationalize the horizon. 17 00:02:35,000 --> 00:02:40,000 We need to know what variables we're using. 18 00:02:40,000 --> 00:02:42,000 We need to know what. 19 00:02:42,000 --> 00:02:45,000 Forecasting model we're using. 20 00:02:45,000 --> 00:02:49,000 We then need to know how to present the results. 21 00:02:49,000 --> 00:02:54,000 We need to know how to how to decipher the results. 22 00:02:54,000 --> 00:02:58,000 We need to iterate over time. 23 00:02:58,000 --> 00:03:05,000 Using recursive methods, so we need to not just leave this. We need to keep working with it. 24 00:03:05,000 --> 00:03:10,000 And then we need to understand that the model may evolve over time. 25 00:03:10,000 --> 00:03:19,000 So we may understand we may figure out new and better ways to forecast. 26 00:03:19,000 --> 00:03:21,000 Just. 27 00:03:21,000 --> 00:03:27,000 Well, well, actually, I come back today to the statistics here in a bit. 28 00:03:27,000 --> 00:03:29,000 So. 29 00:03:29,000 --> 00:03:34,000 Sorry for the rocky introduction. 30 00:03:34,000 --> 00:03:36,000 But. 31 00:03:36,000 --> 00:03:42,000 We'll just go ahead and start here. So knowing what we're going to forecast. 32 00:03:42,000 --> 00:03:46,000 So we've been working with the data in Colorado. 33 00:03:46,000 --> 00:03:48,000 So. 34 00:03:48,000 --> 00:03:56,000 Last week, we showed how given sales. 35 00:03:56,000 --> 00:04:06,000 So given sales. 36 00:04:06,000 --> 00:04:08,000 Implants. 37 00:04:08,000 --> 00:04:11,000 We were able to. 38 00:04:11,000 --> 00:04:20,000 Calculate a production function for Colorado, and we were able to estimate. 39 00:04:20,000 --> 00:04:27,000 A competitive wage and the real rate of capital in Colorado. 40 00:04:27,000 --> 00:04:42,000 So those were historic. So if you wanted to set wages or invest in 2021, it would be useful to know what you could expect. 41 00:04:42,000 --> 00:04:56,000 The minimum wage or the competitive wage and the rate of return of capital to be so. 42 00:04:56,000 --> 00:05:02,000 That will bring us to step two, understanding the purpose of forecasting. 43 00:05:02,000 --> 00:05:09,000 To get better expectations for the size of the Colorado cannabis industry in 2021. 44 00:05:09,000 --> 00:05:12,000 Let me pause. 45 00:05:12,000 --> 00:05:37,000 Real quick and get my coffee and then we'll move on to step three. 46 00:05:37,000 --> 00:05:52,000 All right. 47 00:05:52,000 --> 00:05:55,000 So. 48 00:05:55,000 --> 00:05:57,000 After all. 49 00:05:57,000 --> 00:05:59,000 We're doing. 50 00:05:59,000 --> 00:06:09,000 So. 51 00:06:09,000 --> 00:06:10,000 All right. 52 00:06:10,000 --> 00:06:14,000 So bearing with me. 53 00:06:14,000 --> 00:06:19,000 We're now going to understand the forecasting error. 54 00:06:19,000 --> 00:06:33,000 So after all, we're using this to predict the number of plants in the number of sales. So if we overstate. 55 00:06:33,000 --> 00:06:37,000 We may lead to full hardy decisions. 56 00:06:37,000 --> 00:06:40,000 But if we understate. 57 00:06:40,000 --> 00:06:47,000 We may leave money on the table. 58 00:06:47,000 --> 00:06:50,000 We're just going to rationalize the forecasting horizon. 59 00:06:50,000 --> 00:06:55,000 So we're just going to estimate 2021. 60 00:06:55,000 --> 00:06:58,000 We think 2021 should be informative. 61 00:06:58,000 --> 00:07:02,000 Any time frame less than that. 62 00:07:02,000 --> 00:07:04,000 It may not be worthwhile. 63 00:07:04,000 --> 00:07:07,000 And then any longer time frame. 64 00:07:07,000 --> 00:07:10,000 We start to lose credibility in our forecasts. 65 00:07:10,000 --> 00:07:14,000 It may be worthwhile to forecast 2022. 66 00:07:14,000 --> 00:07:17,000 Anything beyond 2022. 67 00:07:17,000 --> 00:07:20,000 You start to have a good bit of skeptic. 68 00:07:20,000 --> 00:07:22,000 You start to approach with a good. 69 00:07:22,000 --> 00:07:26,000 Lens of skeptic habit. 70 00:07:26,000 --> 00:07:30,000 Next, we want to understand our choice of variables. 71 00:07:30,000 --> 00:07:36,000 We're simply going to be. 72 00:07:36,000 --> 00:07:41,000 Using sales. 73 00:07:41,000 --> 00:07:45,000 And plants. 74 00:07:45,000 --> 00:07:49,000 And under the principle of never throwing away data. 75 00:07:49,000 --> 00:07:54,000 We're going to be using the time frame of January 2014. 76 00:07:54,000 --> 00:08:05,000 June of 2020. 77 00:08:05,000 --> 00:08:11,000 To introduce the forecasting model. 78 00:08:11,000 --> 00:08:13,000 We've been. 79 00:08:13,000 --> 00:08:16,000 Working with regressions. 80 00:08:16,000 --> 00:08:19,000 So this is actually an. 81 00:08:19,000 --> 00:08:22,000 A theoretical model. 82 00:08:22,000 --> 00:08:26,000 That utilizes your standard regression model. 83 00:08:26,000 --> 00:08:30,000 So we're essentially saying. 84 00:08:30,000 --> 00:08:34,000 That sales. 85 00:08:34,000 --> 00:08:44,000 Or plants. 86 00:08:44,000 --> 00:08:48,000 In time period T. 87 00:08:48,000 --> 00:08:57,000 Depend on sales in T minus one. 88 00:08:57,000 --> 00:09:07,000 And they may depend on sales in T minus two. 89 00:09:07,000 --> 00:09:12,000 All the way up. 90 00:09:12,000 --> 00:09:16,000 To a certain period. 91 00:09:16,000 --> 00:09:18,000 So. 92 00:09:18,000 --> 00:09:21,000 This is. 93 00:09:21,000 --> 00:09:27,000 Not theoretical, but it's a useful way of modeling data. 94 00:09:27,000 --> 00:09:35,000 And it actually can be quite useful for forecasting. 95 00:09:35,000 --> 00:09:40,000 And then the moving average process is essentially saying that the error. 96 00:09:40,000 --> 00:09:45,000 Also depends on errors from prior periods. 97 00:09:45,000 --> 00:09:54,000 And we'll approach that with our statistical model. 98 00:09:54,000 --> 00:09:58,000 As always, we'll want to show the data. 99 00:09:58,000 --> 00:10:00,000 This best way to do that. 100 00:10:00,000 --> 00:10:02,000 If you've. 101 00:10:02,000 --> 00:10:10,000 Know of Edward Tuft is a figure. 102 00:10:10,000 --> 00:10:15,000 And we'll want to look at the forecasting results. 103 00:10:15,000 --> 00:10:17,000 We'll want to look for seasonality. 104 00:10:17,000 --> 00:10:20,000 We may want to look at the totals. 105 00:10:20,000 --> 00:10:24,000 So anything of interest. 106 00:10:24,000 --> 00:10:26,000 Of course. 107 00:10:26,000 --> 00:10:28,000 You want to use recursive methods. 108 00:10:28,000 --> 00:10:29,000 So. 109 00:10:29,000 --> 00:10:32,000 After we make our forecast. 110 00:10:32,000 --> 00:10:34,000 In several months time. 111 00:10:34,000 --> 00:10:36,000 We'll want to come back and. 112 00:10:36,000 --> 00:10:38,000 Double check the forecast. 113 00:10:38,000 --> 00:10:40,000 See how far they were off. 114 00:10:40,000 --> 00:10:44,000 And perhaps make new forecasts for the coming year. 115 00:10:44,000 --> 00:10:45,000 So. 116 00:10:45,000 --> 00:10:47,000 Perhaps in six months time. 117 00:10:47,000 --> 00:10:49,000 We can double check the forecast. 118 00:10:49,000 --> 00:10:52,000 See how far we were off. 119 00:10:52,000 --> 00:10:55,000 And try to do better next time. 120 00:10:55,000 --> 00:10:57,000 And. 121 00:10:57,000 --> 00:10:58,000 That's. 122 00:10:58,000 --> 00:11:00,000 Takes in that leads us into principle 10. 123 00:11:00,000 --> 00:11:05,000 Understanding that the forecasting models will evolve over time. 124 00:11:05,000 --> 00:11:06,000 For example. 125 00:11:06,000 --> 00:11:09,000 We may see that there's seasonality. 126 00:11:09,000 --> 00:11:13,000 And think of ways to control for seasonality. 127 00:11:13,000 --> 00:11:16,000 Or introduce. 128 00:11:16,000 --> 00:11:19,000 Seasonal effects. 129 00:11:19,000 --> 00:11:20,000 So. 130 00:11:20,000 --> 00:11:24,000 There's a quick dirty presentation. 131 00:11:24,000 --> 00:11:37,000 And so now let's go ahead and. 132 00:11:37,000 --> 00:12:02,000 Go ahead and. 133 00:12:02,000 --> 00:12:07,000 Go ahead and. 134 00:12:07,000 --> 00:12:11,000 This data that we've. 135 00:12:11,000 --> 00:12:15,000 Collected from the Colorado. 136 00:12:15,000 --> 00:12:18,000 Reports. 137 00:12:18,000 --> 00:12:21,000 These are quarterly reports. 138 00:12:21,000 --> 00:12:24,000 So currently we only have data through. 139 00:12:24,000 --> 00:12:26,000 June of 2020. 140 00:12:26,000 --> 00:12:29,000 But we have data on. 141 00:12:29,000 --> 00:12:32,000 People working. 142 00:12:32,000 --> 00:12:34,000 We have the number of plants. 143 00:12:34,000 --> 00:12:36,000 We have total revenue. 144 00:12:36,000 --> 00:12:37,000 We have some. 145 00:12:37,000 --> 00:12:38,000 We have some. 146 00:12:38,000 --> 00:12:43,000 Interesting metrics here. 147 00:12:43,000 --> 00:12:44,000 So. 148 00:12:44,000 --> 00:12:45,000 First things first. 149 00:12:45,000 --> 00:12:49,000 We're going to go ahead and use our. 150 00:12:49,000 --> 00:12:53,000 Good friends pandas. 151 00:12:53,000 --> 00:12:58,000 And we're going to read in the data. 152 00:12:58,000 --> 00:13:02,000 Always look at the data. 153 00:13:02,000 --> 00:13:05,000 Look sound. 154 00:13:05,000 --> 00:13:08,000 And so now I will introduce you to a. 155 00:13:08,000 --> 00:13:13,000 Forecasting model that I wrote several years ago. 156 00:13:13,000 --> 00:13:14,000 So. 157 00:13:14,000 --> 00:13:16,000 We are essentially. 158 00:13:16,000 --> 00:13:17,000 Using. 159 00:13:17,000 --> 00:13:20,000 Arima. 160 00:13:20,000 --> 00:13:22,000 From. 161 00:13:22,000 --> 00:13:25,000 Stats model. 162 00:13:25,000 --> 00:13:34,000 And Arima will calculate. 163 00:13:34,000 --> 00:13:36,000 This model for us. 164 00:13:36,000 --> 00:13:40,000 So we're essentially estimating an ARP. 165 00:13:40,000 --> 00:13:45,000 Plus. 166 00:13:45,000 --> 00:13:48,000 Right so we're estimating. 167 00:13:48,000 --> 00:13:53,000 Essentially an ARMA. 168 00:13:53,000 --> 00:13:56,000 Plus. 169 00:13:56,000 --> 00:13:59,000 B. 170 00:13:59,000 --> 00:14:02,000 Q. 171 00:14:02,000 --> 00:14:09,000 So we're essentially combining these two processes. 172 00:14:09,000 --> 00:14:11,000 Sorry for that dirty. 173 00:14:11,000 --> 00:14:16,000 Writing. 174 00:14:16,000 --> 00:14:20,000 How are we going to pick the best model? 175 00:14:20,000 --> 00:14:24,000 And so I wanted to introduce to you the concept of. 176 00:14:24,000 --> 00:14:27,000 Root mean squared error. 177 00:14:27,000 --> 00:14:33,000 So here. 178 00:14:33,000 --> 00:14:43,000 We take our actual. 179 00:14:43,000 --> 00:14:50,000 Predictions. 180 00:14:50,000 --> 00:15:00,000 And what's the predicted. 181 00:15:00,000 --> 00:15:11,000 And you would sum that over all of the periods where you made predictions and you have actual events. 182 00:15:11,000 --> 00:15:13,000 Actuals. 183 00:15:13,000 --> 00:15:15,000 I mean how do we cut. 184 00:15:15,000 --> 00:15:18,000 How do we calculate a root mean squared prediction. 185 00:15:18,000 --> 00:15:22,000 If we don't have predictions. 186 00:15:22,000 --> 00:15:27,000 So essentially we're going to utilize a holdout period. 187 00:15:27,000 --> 00:15:32,000 So here we're going to have a holdout period of. 188 00:15:32,000 --> 00:15:35,000 A certain amount of months. 189 00:15:35,000 --> 00:15:38,000 So looking at this data. 190 00:15:38,000 --> 00:15:41,000 It seems relatively arbitrary. 191 00:15:41,000 --> 00:15:44,000 But I think. 192 00:15:44,000 --> 00:15:47,000 A holdout period of six months. 193 00:15:47,000 --> 00:15:50,000 Maybe. 194 00:15:50,000 --> 00:15:53,000 Maybe a interesting span of time. 195 00:15:53,000 --> 00:15:57,000 So you could essentially use all of the data. 196 00:15:57,000 --> 00:16:01,000 Up to. 197 00:16:01,000 --> 00:16:06,000 December of 2019. 198 00:16:06,000 --> 00:16:15,000 And then forecast. 199 00:16:15,000 --> 00:16:20,000 The first six months of 2020. 200 00:16:20,000 --> 00:16:23,000 And then whichever model. 201 00:16:23,000 --> 00:16:25,000 Forecasts. 202 00:16:25,000 --> 00:16:28,000 The first six months the best. 203 00:16:28,000 --> 00:16:32,000 You'll then use that model to then go ahead and predict. 204 00:16:32,000 --> 00:16:35,000 All the way. 205 00:16:35,000 --> 00:16:42,000 Through. 206 00:16:42,000 --> 00:16:44,000 You know. 207 00:16:44,000 --> 00:16:47,000 Through December of 2021. 208 00:16:47,000 --> 00:16:50,000 So we're going to make forecasts. 209 00:16:50,000 --> 00:16:56,000 For the next 18 months. 210 00:16:56,000 --> 00:17:00,000 So just to kind of give you a brief of this code. 211 00:17:00,000 --> 00:17:03,000 Just to kind of show you what's going on under the hood. 212 00:17:03,000 --> 00:17:06,000 So. 213 00:17:06,000 --> 00:17:09,000 Nothing fancy. 214 00:17:09,000 --> 00:17:12,000 Essentially we're just taking a time series. 215 00:17:12,000 --> 00:17:15,000 We're separating it into a training period. 216 00:17:15,000 --> 00:17:18,000 And a testing period. 217 00:17:18,000 --> 00:17:28,000 And then that's just going to be by the holdout period. 218 00:17:28,000 --> 00:17:31,000 Then we're essentially going to scan. 219 00:17:31,000 --> 00:17:33,000 So. 220 00:17:33,000 --> 00:17:38,000 If you go back to the presentation. 221 00:17:38,000 --> 00:17:41,000 You'll see that. 222 00:17:41,000 --> 00:17:44,000 He and Q are arbitrary. 223 00:17:44,000 --> 00:17:52,000 So you can have an arbitrary number of. 224 00:17:52,000 --> 00:17:54,000 Regressors. 225 00:17:54,000 --> 00:17:56,000 Auto. 226 00:17:56,000 --> 00:17:59,000 Auto regressive regressors. 227 00:17:59,000 --> 00:18:02,000 And you can have been however many. 228 00:18:02,000 --> 00:18:07,000 Moving average regressors. 229 00:18:07,000 --> 00:18:09,000 So. 230 00:18:09,000 --> 00:18:11,000 We can't. 231 00:18:11,000 --> 00:18:14,000 Calculate an infinite number of models. 232 00:18:14,000 --> 00:18:18,000 So we're going to limit our scope of models. 233 00:18:18,000 --> 00:18:23,000 So we're essentially going to limit the lag order. 234 00:18:23,000 --> 00:18:26,000 To perhaps six. 235 00:18:26,000 --> 00:18:29,000 So we're essentially you can do more. 236 00:18:29,000 --> 00:18:32,000 This morning I tried it with 12 and it just. 237 00:18:32,000 --> 00:18:34,000 Wouldn't estimate. 238 00:18:34,000 --> 00:18:38,000 So. 239 00:18:38,000 --> 00:18:40,000 The more data you have. 240 00:18:40,000 --> 00:18:44,000 You can typically use longer processes. 241 00:18:44,000 --> 00:18:47,000 But you can also use some rationality. 242 00:18:47,000 --> 00:18:49,000 And. 243 00:18:49,000 --> 00:18:50,000 You know. 244 00:18:50,000 --> 00:18:54,000 You may not want observations from that long ago. 245 00:18:54,000 --> 00:18:56,000 But you can actually. 246 00:18:56,000 --> 00:19:00,000 You know using to predict what's happening in period T. 247 00:19:00,000 --> 00:19:05,000 You can use a mix of common sense plus what actually forecast the best. 248 00:19:05,000 --> 00:19:08,000 We're going to limit our. 249 00:19:08,000 --> 00:19:10,000 Scan. 250 00:19:10,000 --> 00:19:13,000 Of Arma models to six. 251 00:19:13,000 --> 00:19:15,000 So now we're basically just going to loop. 252 00:19:15,000 --> 00:19:17,000 And we're just going to say okay. 253 00:19:17,000 --> 00:19:19,000 So now we're going to show for. 254 00:19:19,000 --> 00:19:27,000 Q. 255 00:19:27,000 --> 00:19:39,000 We essentially just create an Arima model. 256 00:19:39,000 --> 00:19:44,000 And this is actually does a rolling forecast where. 257 00:19:44,000 --> 00:19:52,000 You you you enter you. 258 00:19:52,000 --> 00:20:08,000 You you'll use the your your forecast to go ahead and predict a new model in each iteration. 259 00:20:08,000 --> 00:20:15,000 Once you've estimated. 260 00:20:15,000 --> 00:20:19,000 A Rema model. 261 00:20:19,000 --> 00:20:24,000 For each P Q combination. 262 00:20:24,000 --> 00:20:29,000 So. 263 00:20:29,000 --> 00:20:35,000 Keeping in mind that if. 264 00:20:35,000 --> 00:20:40,000 So if even if you're only doing it to six. 265 00:20:40,000 --> 00:20:42,000 Lags. 266 00:20:42,000 --> 00:20:45,000 If it's still is an unstable model. 267 00:20:45,000 --> 00:20:48,000 Then we're essentially ignoring it. 268 00:20:48,000 --> 00:20:54,000 So we're just ignoring unstable models. 269 00:20:54,000 --> 00:20:56,000 And keeping the forecast. 270 00:20:56,000 --> 00:21:00,000 And the root mean squared errors of. 271 00:21:00,000 --> 00:21:09,000 Successfully fit models. 272 00:21:09,000 --> 00:21:11,000 Using. 273 00:21:11,000 --> 00:21:13,000 The. 274 00:21:13,000 --> 00:21:16,000 Root mean squared error of. 275 00:21:16,000 --> 00:21:18,000 The. 276 00:21:18,000 --> 00:21:31,000 Handful of a Rema P Q's we estimated right so we're basically estimating. 277 00:21:31,000 --> 00:21:37,000 Right so we're essentially estimating great like a Rema. 278 00:21:37,000 --> 00:21:56,000 Like zero zero one. 279 00:21:56,000 --> 00:22:01,000 You know are my one zero. 280 00:22:01,000 --> 00:22:08,000 Or my one one. 281 00:22:08,000 --> 00:22:12,000 So on and so forth. 282 00:22:12,000 --> 00:22:17,000 You're then going to see. 283 00:22:17,000 --> 00:22:20,000 Which of these models. 284 00:22:20,000 --> 00:22:24,000 Predicts. 285 00:22:24,000 --> 00:22:27,000 These variables the best. 286 00:22:27,000 --> 00:22:32,000 So does the arm. 287 00:22:32,000 --> 00:22:36,000 Zero one is on zero two does the arm of one zero. 288 00:22:36,000 --> 00:22:40,000 Which one of these predicts total revenue the best. 289 00:22:40,000 --> 00:22:43,000 Or our holdout period. 290 00:22:43,000 --> 00:22:50,000 And then we'll select that model to go ahead and predict the next 18 months. 291 00:22:50,000 --> 00:22:52,000 So. 292 00:22:52,000 --> 00:22:57,000 We've identified the minimum. 293 00:22:57,000 --> 00:23:00,000 Mean squared error. 294 00:23:00,000 --> 00:23:04,000 And then we're going to use that model. 295 00:23:04,000 --> 00:23:08,000 To predict the. 296 00:23:08,000 --> 00:23:12,000 The forecast period. 297 00:23:12,000 --> 00:23:17,000 So is this like a grid search. 298 00:23:17,000 --> 00:23:20,000 It is. 299 00:23:20,000 --> 00:23:23,000 It. 300 00:23:23,000 --> 00:23:26,000 It's essential it is a grid search. 301 00:23:26,000 --> 00:23:29,000 It's and. 302 00:23:29,000 --> 00:23:32,000 You could make it more complex. 303 00:23:32,000 --> 00:23:39,000 You know today. 304 00:23:39,000 --> 00:23:42,000 For you know this is. 305 00:23:42,000 --> 00:23:46,000 Just scratching the surface of what you can do. 306 00:23:46,000 --> 00:23:51,000 So this is. 307 00:23:51,000 --> 00:23:54,000 What you would call an auto. 308 00:23:54,000 --> 00:23:57,000 Or an auto well this isn't even a. 309 00:23:57,000 --> 00:23:59,000 This is an arm. 310 00:23:59,000 --> 00:24:01,000 So this is an auto. 311 00:24:01,000 --> 00:24:04,000 So you're essentially scanning. 312 00:24:04,000 --> 00:24:07,000 You're like you're doing a grid search. 313 00:24:07,000 --> 00:24:10,000 Of a handful of models. 314 00:24:10,000 --> 00:24:13,000 And then picking the best one. 315 00:24:13,000 --> 00:24:18,000 Just to kind of so that's the logic. 316 00:24:18,000 --> 00:24:23,000 That abstracted away and actually use it. 317 00:24:23,000 --> 00:24:28,000 So we're going to be using a lag order of six. 318 00:24:28,000 --> 00:24:33,000 We're going to be using the six holdout periods. 319 00:24:33,000 --> 00:24:37,000 And then we're going to forecast 18 periods. 320 00:24:37,000 --> 00:24:43,000 The head. 321 00:24:43,000 --> 00:24:58,000 I wonder if we could print out. 322 00:24:58,000 --> 00:25:03,000 Oh just let it run. 323 00:25:03,000 --> 00:25:04,000 Right. 324 00:25:04,000 --> 00:25:07,000 So. 325 00:25:07,000 --> 00:25:12,000 Here we go. 326 00:25:12,000 --> 00:25:16,000 To hear we're fitting a bunch of different models. 327 00:25:16,000 --> 00:25:18,000 You'll see that. 328 00:25:18,000 --> 00:25:20,000 This maximum likelihood. 329 00:25:20,000 --> 00:25:22,000 So the map the model. 330 00:25:22,000 --> 00:25:25,000 The models being. 331 00:25:25,000 --> 00:25:28,000 It by maximum likelihood. 332 00:25:28,000 --> 00:25:32,000 So. 333 00:25:32,000 --> 00:25:35,000 Seeing some error statements being printed out. 334 00:25:35,000 --> 00:25:39,000 When the model fails to converge. 335 00:25:39,000 --> 00:25:42,000 And that and that will happen if you've essentially. 336 00:25:42,000 --> 00:25:45,000 Overfit the model. 337 00:25:45,000 --> 00:25:49,000 So if you try to pack in too many regressors. 338 00:25:49,000 --> 00:25:52,000 So for what they call high order. 339 00:25:52,000 --> 00:25:53,000 Armors. 340 00:25:53,000 --> 00:25:56,000 So like an arm of six six. 341 00:25:56,000 --> 00:25:59,000 With very little data. 342 00:25:59,000 --> 00:26:10,000 Would likely fail to convert. 343 00:26:10,000 --> 00:26:13,000 And so see here we're getting into these higher order. 344 00:26:13,000 --> 00:26:15,000 Arima's. 345 00:26:15,000 --> 00:26:18,000 Like a rema four six. 346 00:26:18,000 --> 00:26:30,000 So. 347 00:26:30,000 --> 00:26:33,000 So it looks like these higher order. 348 00:26:33,000 --> 00:26:35,000 Arima's are mostly sailing. 349 00:26:35,000 --> 00:26:48,000 So. 350 00:26:48,000 --> 00:26:57,000 Well. 351 00:26:57,000 --> 00:26:59,000 And here's our final. 352 00:26:59,000 --> 00:27:04,000 Arima six six. 353 00:27:04,000 --> 00:27:07,000 Arima's are mostly sailing. 354 00:27:07,000 --> 00:27:22,000 Looks like it's having a little difficulties. 355 00:27:22,000 --> 00:27:24,000 All right. And so. 356 00:27:24,000 --> 00:27:30,000 After all that rigor moral. 357 00:27:30,000 --> 00:27:32,000 We got our best model. 358 00:27:32,000 --> 00:27:34,000 Arima four six. 359 00:27:34,000 --> 00:27:37,000 So. 360 00:27:37,000 --> 00:27:41,000 We're basically saying that we think. 361 00:27:41,000 --> 00:27:46,000 That there are four. 362 00:27:46,000 --> 00:27:49,000 Auto regressive terms. 363 00:27:49,000 --> 00:27:53,000 And six. 364 00:27:53,000 --> 00:27:55,000 Moving average terms. 365 00:27:55,000 --> 00:28:00,000 Or regressors. 366 00:28:00,000 --> 00:28:04,000 And then we're going to go to the data. 367 00:28:04,000 --> 00:28:06,000 And rule number one. 368 00:28:06,000 --> 00:28:19,000 Go with the data. 369 00:28:19,000 --> 00:28:30,000 And then we're going to go to the data. 370 00:28:30,000 --> 00:28:43,000 Are forecast for the next 18 months. 371 00:28:43,000 --> 00:29:11,000 And. 372 00:29:11,000 --> 00:29:27,000 Well, that's maybe not the best way to do it. 373 00:29:27,000 --> 00:29:42,000 I would create a quick. 374 00:29:42,000 --> 00:29:45,000 Yeah, there is a, you know, a really slick pandas way to do this, but I can't tell you off the top of my head. 375 00:29:45,000 --> 00:29:54,000 I'm not that familiar either. 376 00:29:54,000 --> 00:29:57,000 But that's essentially. 377 00:29:57,000 --> 00:30:01,000 Brings us to. 378 00:30:01,000 --> 00:30:03,000 Knowing how to present the results. 379 00:30:03,000 --> 00:30:07,000 And now we need to get this into a decent looking figure. 380 00:30:07,000 --> 00:30:08,000 Okay. 381 00:30:08,000 --> 00:30:10,000 So is this. 382 00:30:10,000 --> 00:30:12,000 So this is like an ensemble. 383 00:30:12,000 --> 00:30:15,000 You have a series of regressors. 384 00:30:15,000 --> 00:30:16,000 Right. 385 00:30:16,000 --> 00:30:18,000 And so you're. 386 00:30:18,000 --> 00:30:19,000 You're taking. 387 00:30:19,000 --> 00:30:20,000 You're coming up with a final result. 388 00:30:20,000 --> 00:30:24,000 And then you're going to put in a weighted average of them. 389 00:30:24,000 --> 00:30:25,000 So it's kind of like a. 390 00:30:25,000 --> 00:30:26,000 Yeah. 391 00:30:26,000 --> 00:30:27,000 Yeah. 392 00:30:27,000 --> 00:30:28,000 We're waiting them. 393 00:30:28,000 --> 00:30:38,000 So this is like an ensemble of regressors. 394 00:30:38,000 --> 00:30:39,000 So here. 395 00:30:39,000 --> 00:30:42,000 Just to show you a bit more detail of what's going on here. 396 00:30:42,000 --> 00:30:50,000 Now that we've we've we've decided, okay, this Arima four six is the best model. 397 00:30:50,000 --> 00:30:53,000 Let me just show you what an Arima is. 398 00:30:53,000 --> 00:30:55,000 I think. 399 00:30:55,000 --> 00:30:57,000 I think that would be a light. 400 00:30:57,000 --> 00:31:26,000 Lightening. 401 00:31:26,000 --> 00:31:31,000 This is a four. 402 00:31:31,000 --> 00:31:44,000 Let's see if we can't fit this model real quick. 403 00:31:44,000 --> 00:32:04,000 So then you can print model summary. 404 00:32:04,000 --> 00:32:05,000 Okay. 405 00:32:05,000 --> 00:32:06,000 So this is. 406 00:32:06,000 --> 00:32:09,000 This is the regression that we ran. 407 00:32:09,000 --> 00:32:11,000 So. 408 00:32:11,000 --> 00:32:18,000 We have 78 observations. 409 00:32:18,000 --> 00:32:22,000 We have a constant. 410 00:32:22,000 --> 00:32:24,000 We then have. 411 00:32:24,000 --> 00:32:27,000 Four. 412 00:32:27,000 --> 00:32:29,000 Of these moving average. 413 00:32:29,000 --> 00:32:34,000 I mean of these autoregressive terms. 414 00:32:34,000 --> 00:32:38,000 And these are those their coefficients. 415 00:32:38,000 --> 00:32:41,000 So. 416 00:32:41,000 --> 00:32:46,000 You see a positive coefficient on beta one. 417 00:32:46,000 --> 00:32:58,000 That would indicate higher sales in the previous period would lead to higher sales in period T. 418 00:32:58,000 --> 00:33:01,000 However, you can't really. 419 00:33:01,000 --> 00:33:03,000 Remember. 420 00:33:03,000 --> 00:33:08,000 The pox Jenkins methodologies a theoretical. 421 00:33:08,000 --> 00:33:15,000 So you really can't put too much interpretation on these coefficients. 422 00:33:15,000 --> 00:33:21,000 But essentially what you have here are your your four. 423 00:33:21,000 --> 00:33:23,000 Autoregressive terms. 424 00:33:23,000 --> 00:33:28,000 And you have your six. 425 00:33:28,000 --> 00:33:30,000 Moving average terms. 426 00:33:30,000 --> 00:33:39,000 I can maybe send you some material on how to calculate the moving averages. 427 00:33:39,000 --> 00:33:43,000 It's. 428 00:33:43,000 --> 00:34:00,000 I'm still a bit more involved, but it's. 429 00:34:00,000 --> 00:34:02,000 So I'll want to. 430 00:34:02,000 --> 00:34:07,000 I'll want to get some material on moving averages and add those to the repository. 431 00:34:07,000 --> 00:34:13,000 Do you still have. 432 00:34:13,000 --> 00:34:19,000 Does that answer your question or or you still have the question. 433 00:34:19,000 --> 00:34:22,000 Yeah, no, I see what's going on. 434 00:34:22,000 --> 00:34:24,000 I mean, it's sort of like. 435 00:34:24,000 --> 00:34:28,000 It's actually more like a neural network where the output is. 436 00:34:28,000 --> 00:34:31,000 It's a weighted sum. 437 00:34:31,000 --> 00:34:35,000 Of these outputs, right? 438 00:34:35,000 --> 00:34:38,000 These autoregressors. 439 00:34:38,000 --> 00:34:41,000 You could think of it. 440 00:34:41,000 --> 00:34:46,000 You could think of it that way. 441 00:34:46,000 --> 00:34:49,000 If you want, if you essentially if you want to. 442 00:34:49,000 --> 00:34:53,000 Think of beta as as your weight. 443 00:34:53,000 --> 00:34:56,000 But keep in mind. 444 00:34:56,000 --> 00:35:00,000 It's almost more of like a magnitude. 445 00:35:00,000 --> 00:35:02,000 Versus a weight, right? 446 00:35:02,000 --> 00:35:14,000 Because some of these may have negative coefficients. 447 00:35:14,000 --> 00:35:18,000 Right, because if it was just weighted, then they would all have a positive effect. 448 00:35:18,000 --> 00:35:21,000 But, you know, some of these may have a negative effect. 449 00:35:21,000 --> 00:35:24,000 Some may have a positive effect. 450 00:35:24,000 --> 00:35:30,000 You know, some may have a small, some may have a large effect. 451 00:35:30,000 --> 00:35:35,000 Okay, yeah, I get the gist of what's going on. 452 00:35:35,000 --> 00:35:37,000 Luke, that's awesome. 453 00:35:37,000 --> 00:35:41,000 And so. 454 00:35:41,000 --> 00:35:47,000 This is, you know, it's it's a little bit of an ad hoc way to forecast. 455 00:35:47,000 --> 00:35:52,000 But. 456 00:35:52,000 --> 00:35:56,000 You know, as you saw. 457 00:35:56,000 --> 00:36:21,000 I'm just fitting this whole thing in. 458 00:36:21,000 --> 00:36:31,000 Well, here, let's let's just do the same thing, but maybe with with plants to do a smaller. 459 00:36:31,000 --> 00:36:43,000 Here. Yeah, in the remaining time, why don't we just do it with plants and then add the add the time series so we can actually interpret our results. 460 00:36:43,000 --> 00:36:46,000 Because we still have a good 15 minutes. 461 00:36:46,000 --> 00:36:52,000 Okay, now. 462 00:36:52,000 --> 00:37:12,000 Let's reduce the lag order to three, just to make things more manageable. 463 00:37:12,000 --> 00:37:23,000 And then while we're doing that, I'm going to do a quick Google search for how we can add a. 464 00:37:23,000 --> 00:37:31,000 Two things when we need to add time index to a panda series. 465 00:37:31,000 --> 00:37:48,000 And then to we need to create a time series between the two points. 466 00:37:48,000 --> 00:38:03,000 Let's see if there's not. 467 00:38:18,000 --> 00:38:34,000 Okay, so that's what we want. 468 00:38:34,000 --> 00:38:52,000 Bear with me, but this is this is how you get. 469 00:38:52,000 --> 00:39:07,000 This is how the sausage gets made sometimes. 470 00:39:22,000 --> 00:39:27,000 So we'll do total revenue for the time being. 471 00:39:27,000 --> 00:39:31,000 And I'll refactor this. 472 00:39:31,000 --> 00:39:38,000 The leader point. 473 00:39:38,000 --> 00:39:44,000 Yeah, you have to make like a list of the dates that you want as the index. 474 00:39:44,000 --> 00:39:57,000 And then when you create the series, use that list, tell it to use that list as the index, but I can't tell you like the exact syntax off the top of my head. 475 00:39:57,000 --> 00:40:02,000 I would have something I would have to look up to. 476 00:40:02,000 --> 00:40:07,000 It's one of those things where I think I've coded it up before. 477 00:40:07,000 --> 00:40:15,000 So instead of digging for an old snippet, I think I'm going to see if we can't just code this up real quick. 478 00:40:15,000 --> 00:40:19,000 So apologize for making you bear through this. 479 00:40:19,000 --> 00:40:28,000 I know that's fine. I do this all the time. 480 00:40:28,000 --> 00:40:37,000 I can't because we're basically we're just trying to go until 2022. 481 00:40:37,000 --> 00:40:43,000 Essentially, let's see if. 482 00:40:43,000 --> 00:41:00,000 That looks like it actually looks like the index we want. 483 00:41:00,000 --> 00:41:27,000 That's it. 484 00:41:27,000 --> 00:41:32,000 Not so painful after all. See, this is why. 485 00:41:32,000 --> 00:41:38,000 And this is just a beauty to work with. 486 00:41:38,000 --> 00:41:45,000 So now we've just in a matter of five minutes, we added the time index on there. 487 00:41:45,000 --> 00:41:49,000 So now we can actually. 488 00:41:49,000 --> 00:41:51,000 Move on to. 489 00:41:51,000 --> 00:41:58,000 Now we actually have presented the data in a barely acceptable manner. 490 00:41:58,000 --> 00:42:04,000 So now we can actually decipher the forecasting results. 491 00:42:04,000 --> 00:42:14,000 So we're essentially forecasting. It looks like monthly sales. 492 00:42:14,000 --> 00:42:18,000 I want to say that. 493 00:42:18,000 --> 00:42:24,000 I guess we could maybe even plot this. 494 00:42:24,000 --> 00:42:33,000 Wonder if we could plot this in millions. 495 00:42:33,000 --> 00:42:48,000 OK, awesome. So now I've essentially plotted this in millions of Canada sales. So you'll see that we forecasted sales will dip. 496 00:42:48,000 --> 00:42:52,000 Down in October and January. 497 00:42:52,000 --> 00:43:03,000 So that's about 165 million a month. 498 00:43:03,000 --> 00:43:14,000 Increase during the summer and peak at around 195 million dollars a month in Colorado. 499 00:43:14,000 --> 00:43:22,000 So that is. 500 00:43:22,000 --> 00:43:25,000 You know, quite. 501 00:43:25,000 --> 00:43:49,000 A lot of revenue. And so just to remember, so if we hold out the first six months, because that's 2020. 502 00:43:49,000 --> 00:44:02,000 You know, you're looking at about two billion dollars worth of sales forecasted in Colorado in 2021. 503 00:44:02,000 --> 00:44:04,000 So. 504 00:44:04,000 --> 00:44:07,000 So that's. 505 00:44:07,000 --> 00:44:11,000 You know, so that's how we essentially. 506 00:44:11,000 --> 00:44:26,000 You know, we'll want to now use recursive methods. So now hold me to that. So now, you know, so now mark my words on March 31st. 507 00:44:26,000 --> 00:44:28,000 2021. 508 00:44:28,000 --> 00:44:32,000 We predicted that in 2021. 509 00:44:32,000 --> 00:44:36,000 We're looking into the future. 510 00:44:36,000 --> 00:44:41,000 We're predicting that there's going to be two billion. 511 00:44:41,000 --> 00:44:47,000 2.1 billion dollars worth of sales in Colorado in 2021. 512 00:44:47,000 --> 00:44:49,000 So now. 513 00:44:49,000 --> 00:44:53,000 But it's a we play the waiting game. 514 00:44:53,000 --> 00:44:55,000 So now we'll wait. 515 00:44:55,000 --> 00:45:00,000 And as this data comes out. 516 00:45:00,000 --> 00:45:05,000 We can double check our forecasts. 517 00:45:05,000 --> 00:45:08,000 And we can wait till the year's over. 518 00:45:08,000 --> 00:45:11,000 You know, plus time for them to get the data out. 519 00:45:11,000 --> 00:45:14,000 And we can see if. 520 00:45:14,000 --> 00:45:20,000 You know, this 2.1 billion dollars was that far off. 521 00:45:20,000 --> 00:45:26,000 And so how do we do that? 522 00:45:26,000 --> 00:45:29,000 We'll use our root mean squared error. 523 00:45:29,000 --> 00:45:35,000 So it's nicely defined. We'll simply take the actual. 524 00:45:35,000 --> 00:45:38,000 And 2021. 525 00:45:38,000 --> 00:45:40,000 Minus our predicted. 526 00:45:40,000 --> 00:45:47,000 So we'll actually have a measure of how well this forecasting model. 527 00:45:47,000 --> 00:45:50,000 Predicted. 528 00:45:50,000 --> 00:45:55,000 And that brings us to the final step. 529 00:45:55,000 --> 00:45:57,000 You know. 530 00:45:57,000 --> 00:46:00,000 Understand that this can evolve over time. 531 00:46:00,000 --> 00:46:03,000 So remember, we. 532 00:46:03,000 --> 00:46:05,000 We estimated the. 533 00:46:05,000 --> 00:46:07,000 You know, the Arima. 534 00:46:07,000 --> 00:46:10,000 Four six. 535 00:46:10,000 --> 00:46:13,000 Maybe next time we won't even use Arima. 536 00:46:13,000 --> 00:46:18,000 Maybe next time we'll just use a theoretical regression model. 537 00:46:18,000 --> 00:46:25,000 Or maybe you could compare it to your manager's rule of thumb forecast. 538 00:46:25,000 --> 00:46:29,000 You know, there's a bunch of different mechanisms of forecasting. 539 00:46:29,000 --> 00:46:32,000 So you could use an entirely different model. 540 00:46:32,000 --> 00:46:36,000 Or you could just use a slightly different Arima model. 541 00:46:36,000 --> 00:46:40,000 Or you could use the exact same Arima model, but with more data. 542 00:46:40,000 --> 00:46:44,000 Less data. A longer training period. 543 00:46:44,000 --> 00:46:49,000 A shorter training period. 544 00:46:49,000 --> 00:46:55,000 Less memory. So you may not want to include data going all the way back to 2014. 545 00:46:55,000 --> 00:46:58,000 If you think there's been a structural change. 546 00:46:58,000 --> 00:47:01,000 So. 547 00:47:01,000 --> 00:47:05,000 There are a lot of, you know, there are a lot of toggles. 548 00:47:05,000 --> 00:47:09,000 That you can adjust on your forecast. 549 00:47:09,000 --> 00:47:11,000 So. 550 00:47:11,000 --> 00:47:17,000 That's why it's an iterative process. 551 00:47:17,000 --> 00:47:21,000 Because this is our initial forecast. 552 00:47:21,000 --> 00:47:27,000 But there's always room for improvement. 553 00:47:27,000 --> 00:47:33,000 So that brings us nearer to the end. 554 00:47:33,000 --> 00:47:40,000 And so I'm going to go ahead and I guess turn it to questions. 555 00:47:40,000 --> 00:47:50,000 Do you have any questions about this quick and dirty forecasting that we've done, Charles? 556 00:47:50,000 --> 00:47:54,000 No, no. This was really good. 557 00:47:54,000 --> 00:48:00,000 I'd be interested to see, you know, like some other kind of models. 558 00:48:00,000 --> 00:48:04,000 How they would predict versus this. 559 00:48:04,000 --> 00:48:11,000 But yeah, this was interesting. 560 00:48:11,000 --> 00:48:16,000 You know, that's kind of my whole thing right now is I understand the programming end of it. 561 00:48:16,000 --> 00:48:20,000 And I can wrangle the data all I want. But like what you do with it. 562 00:48:20,000 --> 00:48:27,000 Is what I'm having. It's kind of what I'm struggling with or trying to learn. 563 00:48:27,000 --> 00:48:31,000 You know, and so these are good. These are interesting techniques. 564 00:48:31,000 --> 00:48:37,000 And these are the things that I don't know. 565 00:48:37,000 --> 00:48:44,000 Well, I'm awesome too. 566 00:48:44,000 --> 00:48:46,000 It's awesome to hear that, Charles. 567 00:48:46,000 --> 00:48:51,000 And, you know, I just sort of want to share those things with you. 568 00:48:51,000 --> 00:48:58,000 Because a lot of these are, you know, they're just. 569 00:48:58,000 --> 00:49:02,000 Tools see a lot of mileage. 570 00:49:02,000 --> 00:49:10,000 So, you know, like the arm of forecasting, like people in a lot of different industries use that. 571 00:49:10,000 --> 00:49:15,000 And, you know, these are just sort of tools of economists. 572 00:49:15,000 --> 00:49:20,000 So that's maybe why I know them. But yes, it's awesome to share them with you. 573 00:49:20,000 --> 00:49:24,000 So if you. So let me get that formally. 574 00:49:24,000 --> 00:49:30,000 So you essentially are looking for new forecasting models or. 575 00:49:30,000 --> 00:49:35,000 New data science techniques or. 576 00:49:35,000 --> 00:49:41,000 I mean, I know most of them. I know most of the models and how to use them and stuff. 577 00:49:41,000 --> 00:49:48,000 But like, I don't like, you know, like these. 578 00:49:48,000 --> 00:50:00,000 You know, the way that you apply them to economics, like I don't really have a good grasp of like how to apply them to real world things. 579 00:50:00,000 --> 00:50:02,000 I see. 580 00:50:02,000 --> 00:50:06,000 Well, that's exactly what we're doing here in. 581 00:50:06,000 --> 00:50:12,000 The state of science group is we're essentially we're taking the economic theory. 582 00:50:12,000 --> 00:50:26,000 We're using some statistics and we're applying that to real world cannabis data because we're empiricists and we want to take. 583 00:50:26,000 --> 00:50:30,000 What we're given. So remember. 584 00:50:30,000 --> 00:50:34,000 Remember, this all comes from. 585 00:50:34,000 --> 00:50:40,000 Well, you remember, so this all comes from the Colorado. 586 00:50:40,000 --> 00:50:44,000 All these Colorado data dumps. 587 00:50:44,000 --> 00:50:47,000 So we're. 588 00:50:47,000 --> 00:50:59,000 You know, we're just taking this raw data like this is as raw as it gets. It's not even in Excel. It's just raw from PDFs. 589 00:50:59,000 --> 00:51:01,000 Just completely raw. 590 00:51:01,000 --> 00:51:05,000 We're getting it into an acceptable. 591 00:51:05,000 --> 00:51:10,000 Format is an Excel spreadsheet that like is the very least we can do. 592 00:51:10,000 --> 00:51:21,000 And then we're reading it in and then we're using, you know, modern statistics, economic theory, and we're. 593 00:51:21,000 --> 00:51:30,000 We're applying it to try to make useful insights. You know, we're trying to have some some useful takeaways here. 594 00:51:30,000 --> 00:51:41,000 And so what's what's exciting about this. 595 00:51:41,000 --> 00:51:52,000 Is remember we were calculating the competitive wage over time in the historic rate of return over time. 596 00:51:52,000 --> 00:52:01,000 Well, now you can even forecast that forward and then apply these data points to your economic theory. 597 00:52:01,000 --> 00:52:04,000 So that way you can. 598 00:52:04,000 --> 00:52:13,000 Make predictions about will wages rise or fall in 2021, which is. 599 00:52:13,000 --> 00:52:25,000 Which is incredibly helpful insight to people working in the cannabis industry, because knowing what the actual wage maybe isn't that useful. 600 00:52:25,000 --> 00:52:37,000 But knowing if the wage may rise or fall, you know, that could that could be vitally important to someone trying to staff their business. 601 00:52:37,000 --> 00:52:46,000 Yeah, no, this is really this is all really cool. This is, you know, this is the kind of stuff they don't teach you in these in classes. 602 00:52:46,000 --> 00:52:57,000 And just like dealing with these large pandas data frames. I mean, you never encountered that in a class. You never learn like these tricks. 603 00:52:57,000 --> 00:53:07,000 Exactly. Then pandas is a big, big tools toolbox. So it's tough to get all the pieces right. 604 00:53:07,000 --> 00:53:20,000 And they always give you these sort of toy problems, but these are actually like real problems using like real, you know, real, real theory, you know, coming out with like sort of real, you know, real estimates. 605 00:53:20,000 --> 00:53:26,000 So this is good. This is kind of the stuff I, you know, this is this has been really helpful. 606 00:53:26,000 --> 00:53:29,000 Well, that's awesome to hear, Charles. 607 00:53:29,000 --> 00:53:47,000 And then here in this last minute or two, I'm just going to add a time index to the data, just so we can't see if we can't just plot everything. 608 00:53:47,000 --> 00:53:56,000 Is it total revenue? 609 00:53:56,000 --> 00:54:19,000 So now this is sort of sort of our takeaway from today, right? So we have the historic sales going way up. And, you know, you not you naively may think, oh, this is just going to go on forever. 610 00:54:19,000 --> 00:54:28,000 But our model is smart enough that it's it's taken into account a little bit of this business cycle seasonality. 611 00:54:28,000 --> 00:54:51,000 So you have, you know, so you are actually, you know, as you can see, it's smooth. So it's not, you know, a perfect forecast, but it can at least kind of give you, you know, an idea of the path that sales may take in the coming year. 612 00:54:51,000 --> 00:55:05,000 So I think I'm going to go ahead and wrap it up for today. 613 00:55:05,000 --> 00:55:23,000 But it was a little bit of a, you know, a quick and dirty exercise, so it could have could have been a bit cleaner. And so for next week, may clean it up a bit and extend upon it. 614 00:55:23,000 --> 00:55:43,000 So that way we can do forecasts and actually apply economic theory to the forecast. So just want to thank you for coming. And I'll be including it here. So see you all next week. 615 00:55:43,000 --> 00:55:44,000 Okay, see you. Thanks. 616 00:55:44,000 --> 00:55:58,000 All right. See you Charles. Have an awesome day.