1 00:00:00,000 --> 00:00:05,000 Welcome to the Improving Development Evaluation Podcast. 2 00:00:05,680 --> 00:00:08,100 I'm your host David Wand, 3 00:00:08,100 --> 00:00:10,860 and welcome to episode eight, part two, 4 00:00:10,860 --> 00:00:14,360 where we continue our discussion 5 00:00:14,360 --> 00:00:17,140 of the international development organization 6 00:00:17,140 --> 00:00:21,600 called United Nations Development Program, or UNDP. 7 00:00:21,600 --> 00:00:23,940 You can learn more about the United Nations 8 00:00:23,940 --> 00:00:27,180 Development Program at UNDP.org, 9 00:00:27,180 --> 00:00:31,960 and we're looking at government of Canada funded project 10 00:00:31,960 --> 00:00:36,960 cost to the taxpayer of $30 million in Bangladesh, 11 00:00:37,000 --> 00:00:40,400 entitled Fighting Environmental Degradation 12 00:00:40,400 --> 00:00:43,000 and Promoting Greener Energy. 13 00:00:43,000 --> 00:00:47,080 And in summary, basically going to the Cox's Bazaar 14 00:00:47,080 --> 00:00:52,080 refugee camp, but also communities outside of the camp, 15 00:00:52,240 --> 00:00:55,960 giving them liquefied petroleum gas 16 00:00:55,960 --> 00:01:00,240 to replace their need to go into the sacred bush 17 00:01:00,240 --> 00:01:02,160 and get firewood. 18 00:01:03,160 --> 00:01:06,980 So basically, that's the idea behind the project. 19 00:01:06,980 --> 00:01:10,080 And as I will elaborate further, 20 00:01:10,080 --> 00:01:12,320 there are lots of expected outcomes 21 00:01:12,320 --> 00:01:16,460 from replacing firewood with LPG. 22 00:01:16,460 --> 00:01:19,080 One of them is improved nutritional status 23 00:01:19,080 --> 00:01:22,200 for the children because you can cook more frequently 24 00:01:22,200 --> 00:01:23,640 nutritious meals. 25 00:01:23,640 --> 00:01:27,720 And also, it's a safety issue for women. 26 00:01:27,720 --> 00:01:30,400 They would not have to go out into the forest 27 00:01:30,400 --> 00:01:33,400 and collect firewood, which reduces the risk 28 00:01:33,400 --> 00:01:36,280 or eliminates it if they don't have to go out at all 29 00:01:36,280 --> 00:01:38,680 because they have LPG at home, 30 00:01:38,680 --> 00:01:41,480 reduces the risk of sexual assault and rape. 31 00:01:41,480 --> 00:01:45,800 And the other thing it does is it reduces the harmful smoke 32 00:01:45,800 --> 00:01:48,220 from burning firewood. 33 00:01:48,220 --> 00:01:50,400 So those are some of the expected outcomes 34 00:01:50,400 --> 00:01:53,760 of replacing the need to go out and get firewood 35 00:01:53,760 --> 00:01:56,640 when you have it replaced by LPG. 36 00:01:56,640 --> 00:01:59,160 But another expected outcome I'll talk about 37 00:01:59,160 --> 00:02:01,680 in more detail is you hope, 38 00:02:01,680 --> 00:02:03,420 since we're worried about the environment, 39 00:02:03,420 --> 00:02:07,200 that in the longer term, as I mentioned in part one, 40 00:02:07,200 --> 00:02:12,200 they'll replace the LPG with solar cookers or biogas. 41 00:02:14,720 --> 00:02:19,280 And although that conflicts with perhaps 42 00:02:19,280 --> 00:02:21,920 Canadian LPG companies, 43 00:02:21,920 --> 00:02:24,800 because they're gonna lose business if that happens, 44 00:02:24,800 --> 00:02:27,060 but ideally that's what you'd like to see, 45 00:02:27,060 --> 00:02:29,700 even though the Canadian LPG companies, 46 00:02:29,700 --> 00:02:32,880 assuming they're using Canadian LPG companies 47 00:02:32,880 --> 00:02:37,700 to supply this camp, they're gonna lose a customer. 48 00:02:37,700 --> 00:02:41,440 But in the long run, that's gonna benefit the environment. 49 00:02:41,440 --> 00:02:42,840 So let's get with it. 50 00:02:42,840 --> 00:02:46,480 So some background is on June 10th, 51 00:02:46,480 --> 00:02:49,480 I sent the performance measurement framework 52 00:02:49,480 --> 00:02:53,220 and the critique of the performance measurement framework, 53 00:02:53,220 --> 00:02:56,800 which basically said that of the nine outcome indicators, 54 00:02:56,800 --> 00:02:58,440 measuring the three outcomes 55 00:02:58,440 --> 00:03:00,640 in the performance measurement framework, 56 00:03:00,640 --> 00:03:03,560 I only found two out of the nine 57 00:03:03,560 --> 00:03:08,440 that were properly measuring the outcomes. 58 00:03:08,440 --> 00:03:09,920 And on June 10th, as I said, 59 00:03:09,920 --> 00:03:12,920 I sent it to three departments of the UNDP. 60 00:03:12,920 --> 00:03:16,460 I sent it to the Independent Evaluation Office in New York, 61 00:03:16,460 --> 00:03:19,160 on June 10th, also sent it to 62 00:03:19,160 --> 00:03:22,280 the United Nations Development Program in Bangladesh, 63 00:03:22,280 --> 00:03:23,280 their country office, 64 00:03:23,280 --> 00:03:26,960 because that's where the project is being implemented. 65 00:03:26,960 --> 00:03:30,360 And I also sent it to the United Nations Development Program, 66 00:03:30,360 --> 00:03:35,360 Asia and Pacific Regional Communications Advisor. 67 00:03:35,700 --> 00:03:38,900 On June 18th, I got one email reply 68 00:03:38,900 --> 00:03:43,840 from the Asia and Pacific Regional Communications Advisor, 69 00:03:43,840 --> 00:03:45,920 and he advised me, he was on vacation, 70 00:03:45,920 --> 00:03:47,200 so he couldn't respond. 71 00:03:47,200 --> 00:03:50,040 He told me to email the Bureau 72 00:03:50,040 --> 00:03:54,540 for External Relations and Advocacy, UNDP in New York, 73 00:03:54,540 --> 00:03:59,420 which I did, and he confirmed the right email address. 74 00:03:59,420 --> 00:04:01,880 I sent that also on June 18th, 75 00:04:01,880 --> 00:04:04,200 but as of today, June 28th, 76 00:04:04,200 --> 00:04:07,740 I've received no reply from any of those 77 00:04:07,740 --> 00:04:10,120 three UNDP departments. 78 00:04:10,120 --> 00:04:13,520 And I have also not received any written response 79 00:04:13,520 --> 00:04:17,480 to my critique, even though in the email on June 10th, 80 00:04:17,480 --> 00:04:19,640 I said that was another option. 81 00:04:19,640 --> 00:04:22,200 If they didn't want to be on the podcast, 82 00:04:22,200 --> 00:04:25,280 this part two, to discuss my critique, 83 00:04:26,240 --> 00:04:28,020 they could send a written response, 84 00:04:28,020 --> 00:04:30,460 but I haven't received that. 85 00:04:30,460 --> 00:04:33,200 So on June 28th, which is today, 86 00:04:33,200 --> 00:04:36,960 I'm now going to do part two, unfortunately, without them, 87 00:04:36,960 --> 00:04:38,760 and without a written response. 88 00:04:38,760 --> 00:04:42,340 And then at the end of that, I will, as I usually do, 89 00:04:42,340 --> 00:04:46,600 send it to the Minister for International Development, 90 00:04:46,600 --> 00:04:49,560 as well as the shadow critics from the Conservative Party, 91 00:04:49,560 --> 00:04:53,100 the NDP, and the Bloc Québécois, 92 00:04:53,100 --> 00:04:55,360 and also the Green Party, so everybody knows, 93 00:04:55,360 --> 00:04:58,960 because this is not a political particular affiliation 94 00:04:58,960 --> 00:05:00,280 that you need to have. 95 00:05:00,280 --> 00:05:03,380 Everyone of every political persuasion 96 00:05:03,380 --> 00:05:07,080 should be concerned about the fact 97 00:05:07,080 --> 00:05:09,700 that Canadian government funds 98 00:05:09,700 --> 00:05:12,280 are not being properly evaluated. 99 00:05:13,900 --> 00:05:16,640 So let's get on with it. 100 00:05:16,640 --> 00:05:19,640 So there are nine outcome indicators 101 00:05:19,640 --> 00:05:21,800 measuring the three outcomes. 102 00:05:21,800 --> 00:05:24,240 So we'll start with the first outcome, number one, 103 00:05:24,240 --> 00:05:27,720 and I quote from the performance measurement framework. 104 00:05:27,720 --> 00:05:32,220 Targeted households have safe and sufficient access 105 00:05:32,220 --> 00:05:36,120 to cleaner cooking energy to meet their basic needs. 106 00:05:36,120 --> 00:05:38,780 Problem there is, that's an output. 107 00:05:38,780 --> 00:05:41,040 You're just delivering the LPG. 108 00:05:41,040 --> 00:05:42,080 It's not an outcome. 109 00:05:42,080 --> 00:05:44,440 It's after you deliver the gas, 110 00:05:44,440 --> 00:05:46,680 what are the expected outcomes? 111 00:05:46,680 --> 00:05:47,700 And I've already mentioned 112 00:05:47,700 --> 00:05:49,980 some possible expected outcomes. 113 00:05:49,980 --> 00:05:52,440 So that needs to be changed and reworded. 114 00:05:53,900 --> 00:05:57,640 And it could, to some effect, be, like I've described, 115 00:05:58,940 --> 00:06:03,640 forest areas where refugees harvest firewood is reduced 116 00:06:03,640 --> 00:06:06,320 and replaced by seedlings. 117 00:06:06,320 --> 00:06:08,720 That could be an example of an expected outcome. 118 00:06:08,720 --> 00:06:11,520 But there's other expected outcomes I mentioned earlier. 119 00:06:11,520 --> 00:06:14,080 But let's get to the outcome indicator 120 00:06:14,080 --> 00:06:17,240 for this flawed outcome, number one. 121 00:06:17,240 --> 00:06:19,020 And that outcome indicator, number one, 122 00:06:19,020 --> 00:06:22,580 is percent of households who report 123 00:06:22,580 --> 00:06:25,640 buying or collecting firewood. 124 00:06:25,640 --> 00:06:30,020 The problem there is, it's all full of self-reporting bias, 125 00:06:30,020 --> 00:06:32,680 but even more importantly, it is forget about it. 126 00:06:33,540 --> 00:06:35,420 Don't even measure the indicator. 127 00:06:35,420 --> 00:06:39,400 Don't waste your time asking households 128 00:06:39,400 --> 00:06:42,220 how much firewood they bought or collected. 129 00:06:43,180 --> 00:06:45,880 Go right to the chase and look at some 130 00:06:45,880 --> 00:06:47,860 of the expected outcomes instead. 131 00:06:47,860 --> 00:06:51,320 Weigh the children to see if their nutritional status 132 00:06:51,320 --> 00:06:52,400 has improved. 133 00:06:53,600 --> 00:06:57,920 Ask the women how many times they've had to go out 134 00:06:57,920 --> 00:06:58,880 and collect firewood, 135 00:06:58,880 --> 00:07:01,440 because you expect that to go down, 136 00:07:01,440 --> 00:07:05,640 if they've got the LPG in their house, 137 00:07:05,640 --> 00:07:08,580 in their location in the refugee camp, right? 138 00:07:08,580 --> 00:07:12,880 So those are better indicators rather than just asking them 139 00:07:12,880 --> 00:07:15,500 if they've been collecting firewood. 140 00:07:15,500 --> 00:07:20,500 And the other one is, look at the children or the adults 141 00:07:20,660 --> 00:07:24,080 to see if they have the LPG, 142 00:07:24,080 --> 00:07:27,260 if smoke inhalation-related illnesses 143 00:07:27,260 --> 00:07:30,040 related to burning firewood have gone down. 144 00:07:30,040 --> 00:07:32,640 So those would be the basic things you would look at 145 00:07:32,640 --> 00:07:35,760 instead of running around trying to figure out 146 00:07:35,760 --> 00:07:38,800 how much firewood they've got in their location 147 00:07:38,800 --> 00:07:40,120 or how much they're collecting. 148 00:07:40,120 --> 00:07:42,700 So it's not a very good indicator, 149 00:07:42,700 --> 00:07:45,780 and it creates a lot of waste in terms of measurement. 150 00:07:45,780 --> 00:07:48,320 So you could just do what everybody else is doing, 151 00:07:48,320 --> 00:07:51,120 which is weighing the children anyways, right? 152 00:07:51,120 --> 00:07:55,260 So that's where I recommend instead of that indicator, 153 00:07:55,260 --> 00:08:00,060 focusing on those as alternate indicators. 154 00:08:00,060 --> 00:08:01,700 The second indicator, 155 00:08:01,700 --> 00:08:04,020 this is one of the two that's really good, 156 00:08:04,020 --> 00:08:05,840 and they've mentioned it, 157 00:08:05,840 --> 00:08:08,500 the number of equivalent hectares 158 00:08:08,500 --> 00:08:11,180 of natural forest protected. 159 00:08:11,180 --> 00:08:13,000 That's an excellent indicator. 160 00:08:13,000 --> 00:08:15,760 So I've noted that in the critique, 161 00:08:15,760 --> 00:08:20,720 and they actually have a target of 10,700 hectares 162 00:08:20,720 --> 00:08:24,000 of natural forest protected. 163 00:08:24,000 --> 00:08:27,160 And that's actually in the performance measurement framework. 164 00:08:27,160 --> 00:08:28,220 So that's excellent. 165 00:08:29,400 --> 00:08:32,640 They even indicate in the performance measurement framework 166 00:08:32,640 --> 00:08:34,820 that they will be planting seedlings. 167 00:08:34,820 --> 00:08:37,400 So the key there is to make sure 168 00:08:37,400 --> 00:08:40,320 that that target is reached, 169 00:08:40,320 --> 00:08:41,820 and I'll talk about that a little bit later. 170 00:08:41,820 --> 00:08:43,560 So that's an excellent indicator. 171 00:08:43,560 --> 00:08:44,960 Forget about the first one. 172 00:08:44,960 --> 00:08:46,420 I mean, that's just a waste of money. 173 00:08:46,420 --> 00:08:49,200 And leading on to that, the third indicator 174 00:08:49,200 --> 00:08:54,200 is the number of tons of carbon dioxide emissions abated. 175 00:08:56,040 --> 00:08:57,880 So good luck with that. 176 00:08:57,880 --> 00:08:59,680 I would again say forget about it. 177 00:08:59,680 --> 00:09:03,300 Don't even waste Canadian taxpayer money measuring that. 178 00:09:03,300 --> 00:09:07,240 Focus on the forest that they're going to 179 00:09:07,240 --> 00:09:10,380 to collect firewood and monitor it 180 00:09:10,380 --> 00:09:13,720 by making sure that seedlings are replacing 181 00:09:13,720 --> 00:09:16,920 where they in the past at the baseline 182 00:09:16,920 --> 00:09:20,520 ripped out the wood, chopping down the tree. 183 00:09:20,520 --> 00:09:23,640 The other point is how are you gonna claim 184 00:09:23,640 --> 00:09:26,920 that any reduction in carbon dioxide, 185 00:09:26,920 --> 00:09:28,120 because it's floating in the air, 186 00:09:28,120 --> 00:09:29,760 I don't know how they do this, 187 00:09:29,760 --> 00:09:33,240 how do they claim that it's been abated? 188 00:09:33,240 --> 00:09:35,600 The only way to show that, 189 00:09:35,600 --> 00:09:37,740 and maybe that's what they're implying here, 190 00:09:37,740 --> 00:09:42,160 is by the forest where they used to harvest the firewood 191 00:09:42,160 --> 00:09:44,000 is now being reduced, 192 00:09:44,000 --> 00:09:49,000 and they can monitor that area around the camp to show that. 193 00:09:49,040 --> 00:09:51,680 So that's what they should be doing. 194 00:09:51,680 --> 00:09:53,560 So take out that indicator. 195 00:09:53,560 --> 00:09:57,460 And in fact, in outcome number two, this is what they do, 196 00:09:57,460 --> 00:09:59,580 and I'll read the outcome. 197 00:09:59,580 --> 00:10:04,220 Number two, environment and ecosystems rehabilitated. 198 00:10:04,220 --> 00:10:05,700 Now that's the good part. 199 00:10:06,840 --> 00:10:09,440 The next part is a bit flaky. 200 00:10:09,440 --> 00:10:14,240 To strengthen community resilience to mitigate conflict 201 00:10:14,240 --> 00:10:18,920 over natural resources, disasters, and climate shocks. 202 00:10:20,800 --> 00:10:23,560 The first indicator for that outcome is, 203 00:10:23,560 --> 00:10:25,440 and this is indicator number four, 204 00:10:25,440 --> 00:10:30,000 is the percent of land ecosystems rehabilitated 205 00:10:30,000 --> 00:10:34,860 in Cox's Bazaar through SAFE Plus Two program interventions. 206 00:10:34,860 --> 00:10:38,580 SAFE is the name of the NGO locally 207 00:10:38,580 --> 00:10:41,120 that has been hired by UNDP 208 00:10:41,120 --> 00:10:45,120 to bring in the liquefied petroleum gas. 209 00:10:45,120 --> 00:10:47,720 So that's an excellent indicator. 210 00:10:47,720 --> 00:10:48,600 And in fact, if you look 211 00:10:48,600 --> 00:10:50,740 at the performance measurement framework, 212 00:10:50,740 --> 00:10:52,740 they actually have, I think it's called 213 00:10:52,740 --> 00:10:56,240 geographic information system monitoring. 214 00:10:56,240 --> 00:10:59,840 They use satellite imagery of that specific area 215 00:10:59,840 --> 00:11:02,660 of the forest where the refugees used to go 216 00:11:02,660 --> 00:11:04,040 to get the firewood, 217 00:11:04,040 --> 00:11:06,480 and they're planting seedlings in there 218 00:11:06,480 --> 00:11:10,880 to replace the firewood that was ripped out. 219 00:11:10,880 --> 00:11:12,560 So they just need to track it 220 00:11:12,560 --> 00:11:16,900 and to show that that target of 10,700 hectares 221 00:11:16,900 --> 00:11:20,560 is being protected and sustained. 222 00:11:20,560 --> 00:11:21,580 And that's excellent. 223 00:11:21,580 --> 00:11:22,640 So that's a great indicator. 224 00:11:22,640 --> 00:11:26,520 That's the second indicator, which is pretty good. 225 00:11:26,520 --> 00:11:30,880 And the first one, which is equivalent basically to it, 226 00:11:30,880 --> 00:11:33,080 is the number of equivalent hectares 227 00:11:33,080 --> 00:11:34,520 natural forest protected. 228 00:11:34,520 --> 00:11:36,440 It's basically the same thing. 229 00:11:36,440 --> 00:11:39,600 So it's great that they're doing that. 230 00:11:39,600 --> 00:11:42,600 So that's an excellent indicator. 231 00:11:42,600 --> 00:11:45,840 Indicator number five, again, is a problem. 232 00:11:45,840 --> 00:11:49,520 It's tons of carbon dioxide sequestered 233 00:11:49,520 --> 00:11:50,920 through program interventions. 234 00:11:50,920 --> 00:11:53,280 Again, how are you gonna claim 235 00:11:54,240 --> 00:11:58,700 that the LPG coming in has actually sequestered the carbon? 236 00:11:58,700 --> 00:12:01,000 Now their response could be, duh, 237 00:12:01,000 --> 00:12:03,600 it is the seedlings that we're planting. 238 00:12:03,600 --> 00:12:04,800 Okay, I get it. 239 00:12:04,800 --> 00:12:08,520 But actually measuring the carbon dioxide sequestered, 240 00:12:08,520 --> 00:12:10,520 waste of time and waste of money. 241 00:12:10,520 --> 00:12:15,520 Just focus on the forest around the refugee camp, 242 00:12:15,560 --> 00:12:19,980 which they have done, and show that it's been sustained. 243 00:12:19,980 --> 00:12:22,240 But also, as I mentioned earlier, 244 00:12:22,240 --> 00:12:26,920 look at the other expected outcomes from bringing in the LPG. 245 00:12:28,720 --> 00:12:32,000 And then this is where it gets really a lot of waste 246 00:12:32,000 --> 00:12:33,880 and fun and time and money. 247 00:12:33,880 --> 00:12:37,040 The next indicator they've got 248 00:12:37,040 --> 00:12:40,400 on the environment and ecosystems rehabilitated to, quote, 249 00:12:40,400 --> 00:12:44,040 strengthen community resilience to mitigate conflict 250 00:12:44,040 --> 00:12:47,740 over natural resources, disasters, and climate shocks. 251 00:12:47,740 --> 00:12:50,400 The next indicator they have is, quote, 252 00:12:50,400 --> 00:12:55,520 resilience capacity score increased compared to baseline. 253 00:12:55,520 --> 00:12:57,960 The resilience capacity score, 254 00:12:57,960 --> 00:13:00,960 it measures households' perception 255 00:13:00,960 --> 00:13:04,720 of their resilience capacities to generic 256 00:13:04,720 --> 00:13:08,400 or country-specific shocks and stressors. 257 00:13:09,760 --> 00:13:12,920 And this again is where I say forget about it. 258 00:13:12,920 --> 00:13:15,400 Don't ask about their perception 259 00:13:15,400 --> 00:13:19,180 on fighting over firewood. 260 00:13:20,620 --> 00:13:21,700 Don't do it. 261 00:13:21,700 --> 00:13:26,440 Just look at, instead of asking for their perception 262 00:13:26,440 --> 00:13:29,240 to become resilient, which is a waste of time 263 00:13:29,240 --> 00:13:30,520 and money in measuring it, 264 00:13:30,520 --> 00:13:33,200 it's also got self-reporting bias. 265 00:13:33,200 --> 00:13:35,640 Weigh the children, because you're expecting 266 00:13:35,640 --> 00:13:38,160 their nutritional status to go up. 267 00:13:38,160 --> 00:13:40,440 Inspect the children, for lack of a better word, 268 00:13:40,440 --> 00:13:45,440 to see if the smoke inhalation-related illnesses 269 00:13:45,680 --> 00:13:48,480 related to burning firewood has gone down. 270 00:13:48,480 --> 00:13:51,280 So all the symptoms of that have been reduced. 271 00:13:51,280 --> 00:13:54,880 Also ask the women if they've had to go out 272 00:13:54,880 --> 00:13:57,480 and get firewood, because if they, 273 00:13:57,480 --> 00:13:59,240 of course they could lie, 274 00:13:59,240 --> 00:14:01,800 but what you wanna look at is the frequency 275 00:14:01,800 --> 00:14:04,480 of reporting going out and collecting firewood 276 00:14:04,480 --> 00:14:05,860 has gone down. 277 00:14:05,860 --> 00:14:10,860 Those are better measures, direct objective measures 278 00:14:11,900 --> 00:14:16,880 to see that the LPG coming in is actually having 279 00:14:16,880 --> 00:14:18,960 the desired outcome, right? 280 00:14:18,960 --> 00:14:22,000 Forget about this resilience capacity score. 281 00:14:22,940 --> 00:14:23,840 You don't need it. 282 00:14:23,840 --> 00:14:27,920 So there's another outcome number three, 283 00:14:27,920 --> 00:14:31,400 which is the resilience of Rohingya refugees, 284 00:14:31,400 --> 00:14:33,380 especially women, is sustained. 285 00:14:34,360 --> 00:14:37,480 And there they've got an indicator, 286 00:14:38,320 --> 00:14:41,000 which is the resilience capacity score 287 00:14:41,000 --> 00:14:43,080 is increased compared to the baseline. 288 00:14:43,080 --> 00:14:48,000 Again, you could ask the women what their perception is 289 00:14:48,000 --> 00:14:51,500 of fear of going out and collecting firewood, 290 00:14:51,500 --> 00:14:54,240 but even better, just get to the frequency 291 00:14:54,240 --> 00:14:56,560 that they're going out and collecting firewood, 292 00:14:56,560 --> 00:14:58,200 because hopefully it's gone down. 293 00:14:58,200 --> 00:15:00,040 You could even go a little step further, 294 00:15:00,040 --> 00:15:01,700 but I wouldn't even try it, 295 00:15:01,700 --> 00:15:05,480 which is looking at the number of reported, 296 00:15:05,480 --> 00:15:06,600 because that's tricky, 297 00:15:06,600 --> 00:15:10,080 reported incidents of sexual assault and rape 298 00:15:10,080 --> 00:15:14,640 that have occurred in the area where the firewood 299 00:15:14,640 --> 00:15:16,400 is being collected, right? 300 00:15:16,400 --> 00:15:20,820 It's better to focus on the outcomes I described earlier. 301 00:15:20,820 --> 00:15:23,360 And another indicator they've got, 302 00:15:23,360 --> 00:15:26,200 this is indicator number eight, 303 00:15:27,960 --> 00:15:32,960 is the average reduced coping strategy index. 304 00:15:33,040 --> 00:15:36,320 The coping strategies index is a tool 305 00:15:36,320 --> 00:15:38,120 that measures what people do 306 00:15:38,120 --> 00:15:41,080 when they cannot access enough food. 307 00:15:41,080 --> 00:15:44,200 It is a series of questions about how households 308 00:15:44,200 --> 00:15:48,680 manage to cope with a shortfall in food for consumption 309 00:15:48,680 --> 00:15:53,560 and results in a simple numeric score. 310 00:15:53,560 --> 00:15:58,560 Now the problem here again is you have to look at the output. 311 00:15:58,720 --> 00:16:03,720 Now in addition to bringing in liquefied petroleum gas, 312 00:16:03,860 --> 00:16:06,640 taxpayers from Canada, $30 million, 313 00:16:06,640 --> 00:16:11,640 also are spending money in this project on cash. 314 00:16:11,680 --> 00:16:16,000 That is giving cash to buy food in the refugee camp 315 00:16:16,000 --> 00:16:20,720 and also cash, I mean from the $30 million, 316 00:16:20,720 --> 00:16:24,560 to train the refugees in growing their own food 317 00:16:24,560 --> 00:16:25,920 in a community garden. 318 00:16:26,920 --> 00:16:30,600 And they only measure this average reduced 319 00:16:30,600 --> 00:16:33,960 coping strategy index once a year. 320 00:16:33,960 --> 00:16:36,520 That's not good enough, that's the first problem. 321 00:16:36,520 --> 00:16:39,580 The second problem is, hello, 322 00:16:39,580 --> 00:16:42,680 if you're giving them cash to buy food, 323 00:16:42,680 --> 00:16:47,680 don't worry about this coping strategy index, 324 00:16:47,840 --> 00:16:50,560 figure out if they're buying the food with it 325 00:16:50,560 --> 00:16:52,160 and you can do that at the camp. 326 00:16:52,160 --> 00:16:53,840 I happen to be at the camp. 327 00:16:53,840 --> 00:16:56,620 I've seen the World Food Program in action. 328 00:16:56,620 --> 00:16:58,440 The refugees have a digital card 329 00:16:58,440 --> 00:17:03,440 where they go into stores inside the camp and buy food 330 00:17:03,520 --> 00:17:05,680 and it's registered on these cards. 331 00:17:05,680 --> 00:17:10,680 So the World Food Program has detailed data 332 00:17:10,680 --> 00:17:15,120 on the amount of food that's actually been bought. 333 00:17:15,120 --> 00:17:17,080 And again, like I said earlier, 334 00:17:17,080 --> 00:17:19,800 they can also weigh the children in these families 335 00:17:19,800 --> 00:17:23,240 to see if their nutritional status is going up. 336 00:17:23,240 --> 00:17:25,560 There's no need to have this average reduced 337 00:17:25,560 --> 00:17:28,400 coping strategy index measured, 338 00:17:28,400 --> 00:17:29,880 especially just once a year. 339 00:17:29,880 --> 00:17:32,880 It's a waste of time and a waste of money 340 00:17:32,880 --> 00:17:34,520 because the idea here is to increase 341 00:17:34,520 --> 00:17:35,940 their nutritional status. 342 00:17:35,940 --> 00:17:38,400 So you do it every six months, I would recommend. 343 00:17:38,400 --> 00:17:41,040 Forget about this strategy index 344 00:17:41,040 --> 00:17:43,840 and measure the amount of food they've actually bought 345 00:17:44,920 --> 00:17:47,680 and or weigh the children. 346 00:17:47,680 --> 00:17:49,480 And you can even weigh the adults to show 347 00:17:49,480 --> 00:17:52,000 this is standard nutritional status measures. 348 00:17:52,000 --> 00:17:53,960 You don't have to have this funky, 349 00:17:53,960 --> 00:17:56,900 flaky coping strategy index. 350 00:17:57,800 --> 00:18:01,660 Also measure the garden crop yields if you want 351 00:18:01,660 --> 00:18:03,920 on the training on how to grow crops. 352 00:18:03,920 --> 00:18:06,720 But I would even argue who cares about that. 353 00:18:06,720 --> 00:18:09,200 Go to the end expected outcome, 354 00:18:09,200 --> 00:18:11,000 which is increased weight gain, 355 00:18:11,000 --> 00:18:12,800 nutritional status of the children. 356 00:18:14,680 --> 00:18:17,600 And also the cash purchases of the food items. 357 00:18:17,600 --> 00:18:19,920 I'm repeating myself here, but that's the idea. 358 00:18:19,920 --> 00:18:23,760 So no need for this coping strategies index, 359 00:18:23,760 --> 00:18:27,760 especially when the key outputs in this project are LPG 360 00:18:28,620 --> 00:18:32,460 and cash for food and training to grow food. 361 00:18:32,460 --> 00:18:35,840 So just measure the crop yields, food consumption, 362 00:18:35,840 --> 00:18:37,680 and the weights of the children. 363 00:18:37,680 --> 00:18:39,880 And that would be good enough. 364 00:18:39,880 --> 00:18:44,880 The final indicator on this outcome 365 00:18:45,080 --> 00:18:47,580 of the resilience of Rohingya refugees, 366 00:18:47,580 --> 00:18:50,580 especially women is sustained. 367 00:18:50,580 --> 00:18:54,960 Final outcome indicator is percent of households 368 00:18:54,960 --> 00:18:59,160 with acceptable food consumption score. 369 00:18:59,160 --> 00:19:03,200 Now again, it's only measured once per year. 370 00:19:03,200 --> 00:19:07,480 And I would say you need to measure it more frequently. 371 00:19:09,000 --> 00:19:11,000 And it's a consumption score. 372 00:19:11,000 --> 00:19:15,640 Maybe they use the data from the World Food Program 373 00:19:15,640 --> 00:19:17,320 to be fair to them, but we don't know 374 00:19:17,320 --> 00:19:19,360 because they haven't given me a written response 375 00:19:19,360 --> 00:19:22,640 or showed up for this part too. 376 00:19:22,640 --> 00:19:26,840 So maybe they measure acceptable food consumption 377 00:19:26,840 --> 00:19:30,440 with the data that comes from the World Food Program. 378 00:19:30,440 --> 00:19:32,280 I'm not, I just don't know. 379 00:19:32,280 --> 00:19:33,880 It's only measured once per year. 380 00:19:33,880 --> 00:19:36,920 I would rather go in and measure every six months 381 00:19:36,920 --> 00:19:38,820 the weights of the children. 382 00:19:38,820 --> 00:19:42,200 And that would show their nutritional status going up. 383 00:19:42,200 --> 00:19:46,000 So that's the end of episode eight, part two. 384 00:19:46,000 --> 00:19:50,460 What I'm gonna do now is send this to the minister 385 00:19:50,460 --> 00:19:54,400 and the shadow critics as well as UNDP. 386 00:19:54,400 --> 00:19:57,700 I will copy them on that email so they're aware. 387 00:19:57,700 --> 00:20:00,800 And if you are interested in receiving 388 00:20:00,800 --> 00:20:03,640 the performance measurement framework 389 00:20:03,640 --> 00:20:07,760 and the critique that I've verbally just described here, 390 00:20:07,760 --> 00:20:12,760 you can send me an email at evaluatecanadaaidatgmail.com. 391 00:20:13,680 --> 00:20:15,540 And I'd be happy to send it to you. 392 00:20:15,540 --> 00:20:17,780 I can also send you any of the other 393 00:20:17,780 --> 00:20:20,760 performance measurement frameworks for season one. 394 00:20:20,760 --> 00:20:25,600 This ends season one for the podcast. 395 00:20:25,600 --> 00:20:30,600 I will be returning in September for season two. 396 00:20:31,520 --> 00:20:36,520 And before I do that, I will be doing one more episode, 397 00:20:37,240 --> 00:20:40,020 sort of a prologue of season one, 398 00:20:40,020 --> 00:20:42,600 which gives you an idea of how season two 399 00:20:42,600 --> 00:20:46,440 is going to be slightly different than season one 400 00:20:46,440 --> 00:20:51,440 and give you some hints as to what I'm expanding on 401 00:20:51,600 --> 00:20:54,660 and maybe what I would be refocusing on. 402 00:20:54,660 --> 00:20:58,320 And I'll probably launch that in a few weeks. 403 00:20:58,320 --> 00:21:01,540 Thank you for listening and thank you for the feedback. 404 00:21:01,540 --> 00:21:04,840 I've received some very positive emails 405 00:21:04,840 --> 00:21:09,240 regarding potential learning opportunities on the podcast, 406 00:21:09,240 --> 00:21:12,820 which I'm thinking about, but I'll elaborate more on that 407 00:21:12,820 --> 00:21:15,200 in the prologue to season one. 408 00:21:15,200 --> 00:21:16,680 So that's it. 409 00:21:16,680 --> 00:21:19,320 Thank you for listening and stay tuned 410 00:21:19,320 --> 00:21:23,160 for the prologue for season one and also season two, 411 00:21:23,160 --> 00:21:26,440 which will start up again in September. 412 00:21:26,440 --> 00:21:53,440 Bye for now.