1 00:00:00,000 --> 00:00:05,000 Welcome to the Improving Development Evaluation Podcast. 2 00:00:05,900 --> 00:00:09,600 I'm your host David Wand and welcome to episode eight, 3 00:00:09,600 --> 00:00:13,000 part one, where we in this part one introduce you 4 00:00:13,000 --> 00:00:17,080 to a new international development organization 5 00:00:17,080 --> 00:00:19,960 and we give you a brief description of the project 6 00:00:19,960 --> 00:00:23,280 delivered by that international development organization 7 00:00:23,280 --> 00:00:27,280 and a little summary of my review of the outcomes 8 00:00:27,280 --> 00:00:29,840 and the outcome indicators for this project. 9 00:00:29,840 --> 00:00:33,160 So I'll begin by saying we're doing a little bit 10 00:00:33,160 --> 00:00:35,800 of a different one on this episode eight. 11 00:00:35,800 --> 00:00:38,200 If you've been following the podcast, 12 00:00:38,200 --> 00:00:41,680 all of the episodes focus on delivering services 13 00:00:41,680 --> 00:00:46,140 to actual human beings, which means you can measure 14 00:00:46,140 --> 00:00:50,500 those actual human beings and that's the only way 15 00:00:50,500 --> 00:00:54,280 you're gonna get proper measurement and evaluation. 16 00:00:54,280 --> 00:00:56,680 And as you've learned from this podcast, 17 00:00:56,680 --> 00:01:00,260 a lot of these international development organizations 18 00:01:00,260 --> 00:01:03,920 are avoiding direct measurement 19 00:01:03,920 --> 00:01:06,580 and only they know the reasons why. 20 00:01:06,580 --> 00:01:09,200 But anyway, in this episode eight, 21 00:01:09,200 --> 00:01:12,040 instead of delivering a bunch of services 22 00:01:12,040 --> 00:01:14,040 to a bunch of human beings, 23 00:01:14,040 --> 00:01:17,640 we are going to look at a product that's delivered 24 00:01:17,640 --> 00:01:21,040 to a bunch of human beings and then what are the expected 25 00:01:21,040 --> 00:01:23,520 outcomes from delivering that product. 26 00:01:23,520 --> 00:01:27,020 The new international development organization 27 00:01:27,020 --> 00:01:28,800 for episode eight, part one is 28 00:01:28,800 --> 00:01:32,020 the United Nations Development Program. 29 00:01:32,020 --> 00:01:35,500 You can learn more about the UNDP at their website, 30 00:01:35,500 --> 00:01:40,500 UNDP.org and the particular project here 31 00:01:41,340 --> 00:01:46,340 is in Bangladesh at Cox's Bazaar District 32 00:01:46,580 --> 00:01:50,940 and in particular, the Cox's Bazaar Refugee Camp 33 00:01:50,940 --> 00:01:55,100 for the Rohingyas who've escaped from, 34 00:01:55,100 --> 00:01:58,340 depends who you ask, Myanmar or Burma, 35 00:01:58,340 --> 00:02:02,320 but it also includes the host Bangladesh communities 36 00:02:02,320 --> 00:02:05,780 around the refugee camp at Cox's Bazaar. 37 00:02:06,740 --> 00:02:08,140 That's the location. 38 00:02:08,140 --> 00:02:11,100 The title of the project is called 39 00:02:11,100 --> 00:02:13,380 Fighting Environmental Degradation 40 00:02:13,380 --> 00:02:15,840 and Promoting Greener Energy. 41 00:02:17,440 --> 00:02:19,940 The cost to the Canadian taxpayer for this project 42 00:02:19,940 --> 00:02:24,300 is $30 million and the magical product 43 00:02:24,300 --> 00:02:27,260 that they're delivering is LPG, 44 00:02:27,260 --> 00:02:32,260 liquefied petroleum gas to the refugees 45 00:02:32,620 --> 00:02:35,860 and the purpose of that is to replace 46 00:02:35,860 --> 00:02:39,140 the refugees having to go out and collect firewood 47 00:02:39,140 --> 00:02:42,160 to then burn to cook their meals. 48 00:02:42,160 --> 00:02:44,060 Now the interesting thing is in the project, 49 00:02:44,060 --> 00:02:48,740 they indicate that it also is delivering this LPG 50 00:02:48,740 --> 00:02:53,620 to the Bangladesh communities outside of the refugee camp 51 00:02:53,620 --> 00:02:56,100 but in the performance measurement framework 52 00:02:56,100 --> 00:02:59,280 that I've acquired through an access to information request, 53 00:02:59,280 --> 00:03:02,220 there are no monitoring indicators at all 54 00:03:03,220 --> 00:03:06,440 on the LPG that's been delivered 55 00:03:06,440 --> 00:03:08,580 to the host Bangladesh communities 56 00:03:08,580 --> 00:03:10,740 outside of the refugee camp. 57 00:03:10,740 --> 00:03:13,540 So that's something that I would love to discuss 58 00:03:13,540 --> 00:03:17,300 with the UNDP representative for part two 59 00:03:17,300 --> 00:03:19,780 if they show up for the podcast. 60 00:03:19,780 --> 00:03:24,780 So let me get into the outputs for this project. 61 00:03:24,780 --> 00:03:28,940 The first one obviously is liquefied petroleum gas 62 00:03:28,940 --> 00:03:30,500 to refugee households. 63 00:03:30,500 --> 00:03:35,500 They have a target of 827,611 refills per year 64 00:03:37,940 --> 00:03:40,980 of the LPG in the tanks. 65 00:03:40,980 --> 00:03:43,500 They also have an output called energy studies 66 00:03:43,500 --> 00:03:48,500 to assess alternatives to LPG, liquefied petroleum gas 67 00:03:48,860 --> 00:03:51,340 because you're hoping in the long term, 68 00:03:51,340 --> 00:03:54,380 even the firewood that's replaced by the LPG, 69 00:03:54,380 --> 00:03:56,940 then the LPG you hope will be replaced 70 00:03:56,940 --> 00:03:59,000 by something more renewable. 71 00:03:59,000 --> 00:04:01,480 If you're familiar with refugee camps, 72 00:04:01,480 --> 00:04:05,740 there are solar cookers that use the sun to cook meals 73 00:04:05,740 --> 00:04:07,700 and there's also biogas. 74 00:04:07,700 --> 00:04:12,320 That's at the refugee camp if I remember in South Sudan. 75 00:04:12,320 --> 00:04:15,940 I've also been to the Cox's Bazaar refugee camp. 76 00:04:15,940 --> 00:04:19,060 So this is why I picked this one just out of interest. 77 00:04:19,060 --> 00:04:21,780 But forget about me, let's get back to the project. 78 00:04:21,780 --> 00:04:24,940 So one point I'd like to make here 79 00:04:24,940 --> 00:04:26,780 is that on these studies, 80 00:04:26,780 --> 00:04:31,780 if those studies don't look at solar cookers or biogas, 81 00:04:32,620 --> 00:04:36,100 then it's a waste of money out of the 30 million. 82 00:04:37,020 --> 00:04:39,380 So let's hope that the studies, 83 00:04:39,380 --> 00:04:42,300 it would be nice if they were on the Global Affairs Canada 84 00:04:42,300 --> 00:04:44,780 project browser website, but they're not. 85 00:04:44,780 --> 00:04:46,320 But that would be nice if they were there. 86 00:04:46,320 --> 00:04:49,260 So we could see what plan they have 87 00:04:49,260 --> 00:04:53,580 and if it's feasible to then phase out the LPG 88 00:04:53,580 --> 00:04:58,580 and bring in solar cookers or biogas systems 89 00:04:59,100 --> 00:05:01,820 to replace the LPG. 90 00:05:01,820 --> 00:05:05,100 And one other final point is since the Canadian taxpayers 91 00:05:05,100 --> 00:05:10,100 popping out 30 million bucks to pay for these LPG tanks, 92 00:05:10,100 --> 00:05:15,100 827,611 is the target per year. 93 00:05:16,460 --> 00:05:19,060 Another curious question I would ask UNDP 94 00:05:19,060 --> 00:05:21,300 if they show up for part two is, 95 00:05:21,300 --> 00:05:23,420 did you hire a Canadian oil company 96 00:05:25,060 --> 00:05:30,060 to supply with this 30 million dollars the LPG tanks? 97 00:05:32,040 --> 00:05:33,780 So that's number one output. 98 00:05:33,780 --> 00:05:37,580 And the idea is obviously to replace the firewood 99 00:05:37,580 --> 00:05:39,320 that they have to go out and collect 100 00:05:39,320 --> 00:05:41,780 because there's a whole bunch of negative consequences 101 00:05:41,780 --> 00:05:43,260 from going out and collecting firewood. 102 00:05:43,260 --> 00:05:46,500 One is it's not safe for the women to do that 103 00:05:46,500 --> 00:05:50,200 because they're susceptible to sexual assault and rape. 104 00:05:50,200 --> 00:05:53,360 Number two, the children when they burn the firewood, 105 00:05:53,360 --> 00:05:55,660 they inhale the smoke, it's not good. 106 00:05:56,820 --> 00:05:59,100 And obviously it's not environmentally sound. 107 00:05:59,100 --> 00:06:01,780 And another reason, if you're gonna notice 108 00:06:01,780 --> 00:06:04,220 from the outputs that I'm gonna describe later, 109 00:06:04,220 --> 00:06:07,660 is it degrades the surrounding forest 110 00:06:07,660 --> 00:06:10,980 because they have to cut down the trees deforestation 111 00:06:10,980 --> 00:06:13,220 and which then leads to mudslides, 112 00:06:13,220 --> 00:06:15,660 landslides, flooding, et cetera, 113 00:06:15,660 --> 00:06:18,420 because the forest isn't there to absorb the water. 114 00:06:18,420 --> 00:06:21,860 So those are the reasons for wanting to replace 115 00:06:21,860 --> 00:06:26,380 the firewood with liquidified petroleum gas. 116 00:06:26,380 --> 00:06:30,300 Another output outside of just delivering the LPG 117 00:06:30,300 --> 00:06:33,940 is the monitoring and the management of the forest 118 00:06:33,940 --> 00:06:36,660 where refugees go to collect firewood 119 00:06:36,660 --> 00:06:40,180 before the LPG arrives and to make sure 120 00:06:40,180 --> 00:06:44,900 that the forest is maintained, protected, and restored. 121 00:06:44,900 --> 00:06:49,540 And related to that, the output number three is the land, 122 00:06:49,540 --> 00:06:52,380 including the forest, is quote, rehabilitated 123 00:06:52,380 --> 00:06:55,420 and revegetated, but also they indicate 124 00:06:55,420 --> 00:06:57,380 in the performance measurement framework 125 00:06:57,380 --> 00:07:00,020 some of the outputs are the stream banks 126 00:07:00,020 --> 00:07:02,380 are rehabilitated and maintained. 127 00:07:03,260 --> 00:07:06,260 And some other pieces of the $30 million 128 00:07:06,260 --> 00:07:08,460 the Canadian taxpayer is paying for 129 00:07:08,460 --> 00:07:12,220 is cash that's given to the refugees 130 00:07:12,220 --> 00:07:16,580 to pay them for the labor for rehabilitating the land 131 00:07:16,580 --> 00:07:18,580 and revegetating it, 132 00:07:18,580 --> 00:07:21,620 because there's a lot of labor involved in doing that, 133 00:07:21,620 --> 00:07:23,780 probably clearing the brush 134 00:07:23,780 --> 00:07:27,460 and also the planting of seedlings, et cetera. 135 00:07:28,380 --> 00:07:30,780 And then finally, there's another payment 136 00:07:30,780 --> 00:07:35,420 from the $30 million, which is cash for quote, 137 00:07:35,420 --> 00:07:37,820 workers provided with equipment. 138 00:07:37,820 --> 00:07:41,020 So I'm assuming that means they're paying workers 139 00:07:41,020 --> 00:07:44,540 for the rental of the equipment that they provide. 140 00:07:44,540 --> 00:07:47,980 Maybe it's the Bangladesh local businesses 141 00:07:47,980 --> 00:07:50,140 that have the equipment available 142 00:07:50,140 --> 00:07:51,740 for them to clear the land. 143 00:07:51,740 --> 00:07:54,380 It's not clear, but I think that's what's going on here. 144 00:07:54,380 --> 00:07:56,700 It's payment for the rental of the equipment 145 00:07:56,700 --> 00:08:01,700 to quote, rehabilitate, revegetate the land. 146 00:08:01,740 --> 00:08:04,980 And another fourth output is the training. 147 00:08:04,980 --> 00:08:08,820 There is some services to human beings in this project, 148 00:08:08,820 --> 00:08:11,660 not just the delivery of LPG. 149 00:08:11,660 --> 00:08:13,340 And that is the training to the refugees 150 00:08:13,340 --> 00:08:18,340 on resolving conflicts over natural resources. 151 00:08:18,500 --> 00:08:21,460 And that would be the fighting that goes on 152 00:08:21,460 --> 00:08:24,340 over the firewood, access to firewood, 153 00:08:24,340 --> 00:08:27,740 but you would hope as the LPG comes in, 154 00:08:27,740 --> 00:08:31,900 less people will be going out and getting firewood. 155 00:08:31,900 --> 00:08:34,980 And also the growing of the crops, 156 00:08:36,180 --> 00:08:40,180 because part of the training is also to do gardening, 157 00:08:40,180 --> 00:08:43,940 so that they can grow vegetables and alternatives 158 00:08:43,940 --> 00:08:47,540 to what they're currently depending on. 159 00:08:47,540 --> 00:08:50,540 Also in this output, they're providing households 160 00:08:50,540 --> 00:08:53,940 with renewable energy technology, 161 00:08:53,940 --> 00:08:55,580 but they don't indicate in the performance 162 00:08:55,580 --> 00:08:59,420 measurement framework, what types for this project. 163 00:08:59,420 --> 00:09:04,420 I'm assuming and hoping it's related to the project, 164 00:09:04,420 --> 00:09:08,420 which is to get them to cook with alternatives 165 00:09:08,420 --> 00:09:13,420 other than firewood and other than liquidified petroleum gas. 166 00:09:15,420 --> 00:09:19,940 So you would hope that they're introducing solar cookers 167 00:09:19,940 --> 00:09:23,620 or biogas as alternatives, but we don't know, 168 00:09:23,620 --> 00:09:25,260 because the current focus of the project 169 00:09:25,260 --> 00:09:27,740 is simply to bring in the liquefied petroleum gas 170 00:09:27,740 --> 00:09:32,740 to replace their need to go out and collect firewood. 171 00:09:33,820 --> 00:09:37,380 And some little issues in the outputs. 172 00:09:37,380 --> 00:09:40,900 One of the output indicators is the percent of conflicts 173 00:09:40,900 --> 00:09:43,780 over natural resources successfully managed. 174 00:09:43,780 --> 00:09:46,300 That's an outcome, so that should be moved 175 00:09:46,300 --> 00:09:49,980 into the outcome area, which we'll talk about in part two. 176 00:09:51,900 --> 00:09:54,020 They also have training on home gardening 177 00:09:54,020 --> 00:09:57,020 and fresh food waste management, 178 00:09:57,020 --> 00:09:59,140 hint, hint, composting. 179 00:10:00,340 --> 00:10:02,660 So that's good, that's another output. 180 00:10:02,660 --> 00:10:05,380 And they've got another outcome indicator 181 00:10:05,380 --> 00:10:07,620 that they call incorrectly an output indicator, 182 00:10:07,620 --> 00:10:10,780 which is percent of targeted households 183 00:10:10,780 --> 00:10:14,540 practicing sustainable skills development. 184 00:10:14,540 --> 00:10:16,780 So that's an outcome, after you train them, 185 00:10:16,780 --> 00:10:18,420 you expect them to be doing it. 186 00:10:19,580 --> 00:10:24,580 No targets provided in the PMF for that indicator, 187 00:10:24,580 --> 00:10:28,100 which is another issue, but the major point is 188 00:10:28,100 --> 00:10:29,900 that is an outcome indicator 189 00:10:29,900 --> 00:10:31,900 that should be moved into the outcomes. 190 00:10:33,020 --> 00:10:34,740 And what's interesting, as I mentioned earlier, 191 00:10:34,740 --> 00:10:38,660 is these outputs only focus on the refugees, 192 00:10:38,660 --> 00:10:41,340 but if you read in the project browser 193 00:10:41,340 --> 00:10:44,700 on Global Affairs Canada, they mention the host communities, 194 00:10:44,700 --> 00:10:46,180 the Bangladesh communities. 195 00:10:46,180 --> 00:10:48,820 So there may be liquefied petroleum gas 196 00:10:48,820 --> 00:10:50,900 going into the host communities 197 00:10:50,900 --> 00:10:53,340 outside of the refugee project. 198 00:10:53,340 --> 00:10:55,940 But we don't know, from the PMF, 199 00:10:55,940 --> 00:10:58,660 there's no separate target group 200 00:10:58,660 --> 00:11:01,020 of host Bangladesh communities. 201 00:11:02,140 --> 00:11:04,500 So there's no output indicators there, 202 00:11:04,500 --> 00:11:07,420 so that would be another question to ask UNDP. 203 00:11:07,420 --> 00:11:10,260 Are you only serving the refugees inside the camp, 204 00:11:10,260 --> 00:11:14,660 or are also you delivering liquefied petroleum gas 205 00:11:14,660 --> 00:11:16,820 to the host Bangladesh communities? 206 00:11:18,620 --> 00:11:20,420 So that's basically the project. 207 00:11:20,420 --> 00:11:24,580 Now, the interesting thing is, as I alluded to earlier, 208 00:11:25,300 --> 00:11:28,580 the logic of replacing the firewood with the LPG 209 00:11:28,580 --> 00:11:31,700 is so that a variety of expected outcomes 210 00:11:31,700 --> 00:11:33,020 should be achieved, right? 211 00:11:33,020 --> 00:11:35,620 So I'm logically just telling you what they are. 212 00:11:35,620 --> 00:11:37,500 One is reduce the number of times 213 00:11:37,500 --> 00:11:39,140 they have to go out and collect firewood 214 00:11:39,140 --> 00:11:40,820 because they now have LPG. 215 00:11:41,780 --> 00:11:45,820 Also, they have quicker access to being able to cook 216 00:11:45,820 --> 00:11:47,980 with the LPG, they just fire up the smoke 217 00:11:47,980 --> 00:11:51,700 with the LPG, they just fire up the stove. 218 00:11:51,700 --> 00:11:55,820 And that means that they probably can prepare 219 00:11:55,820 --> 00:11:58,700 more frequently nutritious meals 220 00:11:58,700 --> 00:12:00,860 rather than with firewood. 221 00:12:00,860 --> 00:12:04,380 Which means you would expect, particularly the children, 222 00:12:04,380 --> 00:12:08,340 their nutritional status, that is, their weights to go up. 223 00:12:09,300 --> 00:12:12,260 Also, the reduction in burning firewood 224 00:12:12,260 --> 00:12:17,260 also reduces smoke inhalation, which is a related illness. 225 00:12:17,260 --> 00:12:21,060 So you would expect reported illnesses related 226 00:12:21,060 --> 00:12:24,580 to smoke inhalation from the firewood burning 227 00:12:24,580 --> 00:12:26,500 would also be going down. 228 00:12:26,500 --> 00:12:29,420 So those are some expected outcomes from it. 229 00:12:29,420 --> 00:12:33,060 And in part two, we'll talk about some of the issues 230 00:12:33,060 --> 00:12:35,780 with the outcome indicators and even the outcomes 231 00:12:35,780 --> 00:12:39,060 that they've defined based on what I've just told you there 232 00:12:39,060 --> 00:12:43,020 of what you would expect to achieve as an outcome 233 00:12:43,020 --> 00:12:46,300 when you bring in this LPG to replace the firewood. 234 00:12:46,300 --> 00:12:48,260 They have only three outcomes 235 00:12:48,260 --> 00:12:50,740 in the performance measurement framework 236 00:12:50,740 --> 00:12:54,060 and a total of nine outcome indicators 237 00:12:54,060 --> 00:12:55,660 measuring those three outcomes 238 00:12:55,660 --> 00:12:57,300 as to whether they've been achieved. 239 00:12:57,300 --> 00:13:00,380 I've looked at all nine outcome indicators. 240 00:13:00,380 --> 00:13:03,420 Two of them are really good out of the nine. 241 00:13:03,420 --> 00:13:05,580 The other seven are not good, 242 00:13:05,580 --> 00:13:08,740 but I'm not going to jump to the conclusion 243 00:13:08,740 --> 00:13:13,220 that they can't claim the project is achieving its outcomes. 244 00:13:13,220 --> 00:13:17,380 It's not that severe because it is a product 245 00:13:17,380 --> 00:13:19,140 you're just bringing in LPG. 246 00:13:19,140 --> 00:13:23,540 It's not a lot of measurement going on with human beings, 247 00:13:23,540 --> 00:13:26,540 but there are still some issues that we'll talk about 248 00:13:26,540 --> 00:13:31,220 on the remaining seven outcome indicators in part two. 249 00:13:31,220 --> 00:13:35,780 So what I'm gonna do now is I'm gonna send this part one 250 00:13:35,780 --> 00:13:38,660 along with my summary of the nine outcome indicators 251 00:13:38,660 --> 00:13:42,100 indicating that only two of them are really properly done. 252 00:13:42,100 --> 00:13:46,540 And I'm gonna send it to UNDP in New York, 253 00:13:46,540 --> 00:13:51,540 but I'm also gonna send it to the UNDP office email address 254 00:13:52,020 --> 00:13:55,340 in Bangladesh, because that's where they're delivering it. 255 00:13:55,340 --> 00:13:59,580 And also the regional Asia office 256 00:13:59,580 --> 00:14:04,100 that oversees Bangladesh at UNDP. 257 00:14:04,100 --> 00:14:07,540 And I also had the opportunity to, 258 00:14:07,540 --> 00:14:09,860 when I was at Cox's Bazaar, 259 00:14:09,860 --> 00:14:14,860 work with the NGO, I think I have it right, called SAFE. 260 00:14:14,980 --> 00:14:18,140 I think they're the local NGO responsible 261 00:14:18,140 --> 00:14:22,020 for delivering the LPG and those other outputs, 262 00:14:22,020 --> 00:14:23,180 but I may be wrong there. 263 00:14:23,180 --> 00:14:25,780 It could be UNDP is doing it directly, 264 00:14:25,780 --> 00:14:29,220 but often they'll contract out a local partner, 265 00:14:29,220 --> 00:14:32,900 international development, non-governmental organization 266 00:14:32,900 --> 00:14:36,780 to deliver the outputs into the camp. 267 00:14:36,780 --> 00:14:41,780 And we'll see if they agree to show up and or 268 00:14:43,300 --> 00:14:45,980 give a written response if they don't show up 269 00:14:45,980 --> 00:14:48,460 to part two of the podcast. 270 00:14:48,460 --> 00:14:53,460 And I'll hopefully have that done by the end of June, 271 00:14:54,500 --> 00:14:55,500 part two. 272 00:14:55,500 --> 00:15:00,500 So stay tuned for part two of this podcast, 273 00:15:01,300 --> 00:15:03,980 episode eight, United Nations Development Program 274 00:15:03,980 --> 00:15:06,620 for LPG and Cox's Bazaar. 275 00:15:06,620 --> 00:15:09,820 And again, if you're interested in receiving 276 00:15:09,820 --> 00:15:12,540 any of the performance measurement frameworks 277 00:15:12,540 --> 00:15:16,780 and my critique of those performance measurement frameworks, 278 00:15:16,780 --> 00:15:21,780 feel free to send me an email at evaluatecanadaaidatgmail.com 279 00:15:23,140 --> 00:15:25,020 and I'd be happy to send them to you. 280 00:15:26,500 --> 00:15:29,300 Stay tuned for episode eight, part two. 281 00:15:29,300 --> 00:15:36,300 Bye for now.