WEBVTT 00:00:00.000 --> 00:00:02.759 Welcome to the Improving Development Evaluation 00:00:02.759 --> 00:00:06.179 Podcast. I'm your host, David Wand, and in this 00:00:06.179 --> 00:00:08.199 episode, we're going to feature the international 00:00:08.199 --> 00:00:11.080 development organization CUSO International, 00:00:11.660 --> 00:00:15.080 that's C -U -S -O International. You can learn 00:00:15.080 --> 00:00:18.100 more about CUSO International at their website 00:00:18.100 --> 00:00:24.980 at www .cusointernational .org. We used to affectionately 00:00:24.980 --> 00:00:28.859 refer to CUSO back in the 1980s when I was in 00:00:28.859 --> 00:00:33.070 Seattle. leone with cuso as the canadian unemployed 00:00:33.070 --> 00:00:38.250 socialists overseas and the david haslett wonderful 00:00:38.250 --> 00:00:41.990 man who was the field staff officer for sierra 00:00:41.990 --> 00:00:44.250 leone back then he was the one who introduced 00:00:44.250 --> 00:00:47.130 me to that acronym but of course we've made some 00:00:47.130 --> 00:00:49.729 other ones that are quite funny but we'll stop 00:00:49.729 --> 00:00:53.549 right there and get on with the episode we're 00:00:53.549 --> 00:00:56.170 going to be looking at two cuso projects one 00:00:56.170 --> 00:00:59.759 called blueprints at a cost to the taxpayer of 00:00:59.759 --> 00:01:05.359 $2 .5 million. And another project called Building 00:01:05.359 --> 00:01:10.239 Bricks, at a cost of $5 million to the Canadian 00:01:10.239 --> 00:01:15.959 taxpayer, for a total of $7 .5 million. And Blueprints 00:01:15.959 --> 00:01:18.079 has nine outcomes, and we're going to be looking 00:01:18.079 --> 00:01:20.219 at the performance measurement framework and 00:01:20.219 --> 00:01:24.459 seeing how well the 13 outcome indicators for 00:01:24.459 --> 00:01:28.519 those nine outcomes properly validly measure 00:01:28.519 --> 00:01:31.840 the achievement of those outcomes and for building 00:01:31.840 --> 00:01:34.939 bricks we're looking at eight outcomes for that 00:01:34.939 --> 00:01:38.140 project and a total of 15 outcome indicators 00:01:38.140 --> 00:01:41.280 designed to measure whether or not those eight 00:01:41.280 --> 00:01:45.019 outcomes have indeed been achieved. Before we 00:01:45.019 --> 00:01:46.920 get into it I just wanted to give you some background. 00:01:47.739 --> 00:01:50.319 There's been some significant delays in producing 00:01:50.319 --> 00:01:53.299 these performance measurement frameworks. Rebel 00:01:53.299 --> 00:01:55.920 Media, they're not very well respected in Canada 00:01:55.920 --> 00:02:00.180 but they've reported in their opinion, that some 00:02:00.180 --> 00:02:02.640 of these access to information requests like 00:02:02.640 --> 00:02:05.920 mine for PMFs are intentionally being delayed 00:02:05.920 --> 00:02:08.439 because, of course, that information that they 00:02:08.439 --> 00:02:12.919 release makes the government look bad. Now, I 00:02:12.919 --> 00:02:14.719 can only speculate as to why there have been 00:02:14.719 --> 00:02:18.039 delays in me receiving my performance measurement 00:02:18.039 --> 00:02:20.979 frameworks. These performance measurement frameworks 00:02:20.979 --> 00:02:25.000 need to be produced before the government of 00:02:25.000 --> 00:02:28.400 Canada can disperse funds to CUSO International, 00:02:28.719 --> 00:02:31.979 for example. I requested these two PMFs for these 00:02:31.979 --> 00:02:36.780 two projects back in October of 2024. It's already 00:02:36.780 --> 00:02:39.419 been over a year since I received them. So I 00:02:39.419 --> 00:02:42.439 suspect there's a bit of delay going on. And 00:02:42.439 --> 00:02:46.560 I've also requested two other international development 00:02:46.560 --> 00:02:50.280 organizations and their PMFs. And one of those 00:02:50.280 --> 00:02:54.550 organizations, the PMFs I'm asking for... The 00:02:54.550 --> 00:02:58.889 projects cost over $50 million, and I'm very 00:02:58.889 --> 00:03:01.590 upset. It's been over a year, and they keep delaying 00:03:01.590 --> 00:03:04.330 the release of these PMFs. There's no excuse 00:03:04.330 --> 00:03:06.210 for it because they've already been produced. 00:03:06.330 --> 00:03:09.990 They're only 10 to 15 pages long, and they have 00:03:09.990 --> 00:03:12.729 to be produced by the International Development 00:03:12.729 --> 00:03:15.650 Organization before they disperse the funds. 00:03:15.810 --> 00:03:18.849 And as we speak, funds are already being dispersed 00:03:18.849 --> 00:03:22.439 for these projects. because I can see it on the 00:03:22.439 --> 00:03:26.139 Government of Canada Project Browser website, 00:03:26.419 --> 00:03:30.439 and I'm still not getting the PMFs. So that's 00:03:30.439 --> 00:03:33.659 a bit disturbing. If any of you want the PMFs 00:03:33.659 --> 00:03:37.180 and my critique of the PMFs for each of these 00:03:37.180 --> 00:03:41.020 episodes, you can email me at evaluatecanadaid 00:03:41.020 --> 00:03:45.939 at gmail .com. And as you know... The link for 00:03:45.939 --> 00:03:48.240 this episode is sent to the Secretary of State 00:03:48.240 --> 00:03:50.460 for International Development, along with the 00:03:50.460 --> 00:03:55.000 shadow critics at the Parliament in Canada. So 00:03:55.000 --> 00:03:58.460 everybody knows, regardless of political affiliation, 00:03:58.599 --> 00:04:01.599 that your tax dollars are not being properly 00:04:01.599 --> 00:04:03.879 evaluated when it comes to foreign aid projects. 00:04:06.740 --> 00:04:12.120 So you can also email in the episode. I'll provide 00:04:12.120 --> 00:04:15.669 the emails to recommend. To the Secretary of 00:04:15.669 --> 00:04:18.269 State, feel free to email them yourself with 00:04:18.269 --> 00:04:21.470 the link for this episode telling them to defund 00:04:21.470 --> 00:04:23.629 these international development organizations 00:04:23.629 --> 00:04:27.550 until they start producing proper outcome indicators 00:04:27.550 --> 00:04:30.910 for the outcomes that they keep claiming falsely 00:04:30.910 --> 00:04:33.449 on their websites that they're achieving when 00:04:33.449 --> 00:04:35.430 we don't even know that they're being achieved. 00:04:35.730 --> 00:04:40.329 So that's important. So I'll also be producing 00:04:40.329 --> 00:04:45.490 a petition to... at change .org that I'll be 00:04:45.490 --> 00:04:48.649 putting in each of the episodes notes. So if 00:04:48.649 --> 00:04:51.850 you want to sign a petition urging the Secretary 00:04:51.850 --> 00:04:55.350 of State for International Development to indeed 00:04:55.350 --> 00:05:00.370 urge them to defund these organizations until 00:05:00.370 --> 00:05:05.490 they produce proper outcome indicators and also 00:05:05.490 --> 00:05:08.149 ideally we want these performance measurement 00:05:08.149 --> 00:05:11.310 frameworks as well as the data that goes with 00:05:11.310 --> 00:05:14.970 them. to be produced on the Government of Canada 00:05:14.970 --> 00:05:17.670 Project Browser website, as well as the International 00:05:17.670 --> 00:05:20.709 Development Organization's websites. Because 00:05:20.709 --> 00:05:23.509 currently, they produce a lot of data, but it's 00:05:23.509 --> 00:05:26.949 useless. Garbage in, garbage out. And that's 00:05:26.949 --> 00:05:29.810 the problem. We've discovered that the design 00:05:29.810 --> 00:05:32.769 of these indicators in the first place is flawed. 00:05:32.970 --> 00:05:36.769 And in some cases, as I've noted in other episodes, 00:05:37.110 --> 00:05:40.290 they don't have any outcome indicators at all. 00:05:40.839 --> 00:05:43.019 And they keep claiming that they're achieving 00:05:43.019 --> 00:05:45.040 outcomes. But if you look at the reports they 00:05:45.040 --> 00:05:48.060 send me, there's no outcome indicators at all. 00:05:48.980 --> 00:05:51.839 Instead, they just talk about outputs, how much 00:05:51.839 --> 00:05:54.899 services they've delivered, but no evaluation 00:05:54.899 --> 00:05:58.800 of what happens after those services are delivered. 00:05:59.379 --> 00:06:02.879 And like I said, if you believe in foreign aid 00:06:02.879 --> 00:06:07.839 and you want it to continue to be supported without 00:06:07.839 --> 00:06:10.889 somebody coming along and saying, that there's 00:06:10.889 --> 00:06:12.930 no evaluation and therefore we're going to scrap 00:06:12.930 --> 00:06:16.910 them, these foreign aid projects. It's better 00:06:16.910 --> 00:06:19.629 if you believe in foreign aid to show that the 00:06:19.629 --> 00:06:22.329 foreign aid is actually working. It's effective 00:06:22.329 --> 00:06:26.649 and it has impact, which is what the development 00:06:26.649 --> 00:06:29.930 assistance criteria is all about. But as you've 00:06:29.930 --> 00:06:33.350 discovered, probably through my podcast and other 00:06:33.350 --> 00:06:37.129 information, this is not happening. So it's a 00:06:37.129 --> 00:06:41.420 direct threat to funding, for foreign aid projects. 00:06:41.560 --> 00:06:44.819 So in many cases, it's quite simple to show that 00:06:44.819 --> 00:06:49.060 foreign aid is working. But in some cases, it's 00:06:49.060 --> 00:06:50.560 a little bit more difficult, but there should 00:06:50.560 --> 00:06:53.779 at least be an attempt to measure and then evaluate. 00:06:54.199 --> 00:06:56.819 But that's not going on. So I'll leave it at 00:06:56.819 --> 00:07:00.060 that and let's get into it. So blueprints, nine 00:07:00.060 --> 00:07:06.060 outcomes, 13 outcome indicators. Give you a brief 00:07:06.060 --> 00:07:09.189 description. of what the project does. It's focused 00:07:09.189 --> 00:07:12.029 on the following countries in the Caribbean, 00:07:12.209 --> 00:07:15.610 Dominica, Grenada, St. Lucia, and St. Vincent. 00:07:15.769 --> 00:07:18.610 And they have two target groups, public institutions 00:07:18.610 --> 00:07:22.689 in those countries, as well as women's rights 00:07:22.689 --> 00:07:26.769 organizations. And they focus on training these 00:07:26.769 --> 00:07:29.709 public institutions and women's rights organizations 00:07:29.709 --> 00:07:34.689 on gender equality. By accessing materials on 00:07:34.689 --> 00:07:37.730 online platforms to address issues of gender 00:07:37.730 --> 00:07:42.290 equality. And CUSO helps them organize meetings 00:07:42.290 --> 00:07:45.050 to share best practices on addressing gender 00:07:45.050 --> 00:07:48.850 equality issues amongst public institution representatives 00:07:48.850 --> 00:07:52.370 and these women rights organization representatives 00:07:52.370 --> 00:07:55.910 to increase awareness among these two target 00:07:55.910 --> 00:07:58.550 groups on how to address gender equality issues. 00:08:00.199 --> 00:08:02.800 So they also spend money, CUSO, this funding, 00:08:03.040 --> 00:08:06.839 $100 ,000 to increase engagement between these 00:08:06.839 --> 00:08:09.500 public institutions and the women's rights organization 00:08:09.500 --> 00:08:13.980 that represent girls and women, which leads to 00:08:13.980 --> 00:08:15.980 these public institutions being more effective 00:08:15.980 --> 00:08:19.519 in addressing these gender equality issues. So, 00:08:19.519 --> 00:08:21.959 for example, women and girls will show up to 00:08:21.959 --> 00:08:26.819 attend these dialogue spaces between the girls 00:08:26.819 --> 00:08:29.139 and women and the government, public institution. 00:08:29.709 --> 00:08:32.629 This includes communication campaigns and communication 00:08:32.629 --> 00:08:36.149 engagement sessions to increase access to public 00:08:36.149 --> 00:08:39.970 services. So these girls and women will show 00:08:39.970 --> 00:08:43.169 up, provide feedback through scorecards to government 00:08:43.169 --> 00:08:46.129 reps. The government feedback committees will 00:08:46.129 --> 00:08:50.330 be set up to receive these scorecards. And the 00:08:50.330 --> 00:08:52.450 government staff in these public institutions 00:08:52.450 --> 00:08:55.110 are trained in, quote, accountability practices. 00:08:56.039 --> 00:08:58.740 They deliver information fairs on gender services 00:08:58.740 --> 00:09:01.659 to women and girls. They're trained in these 00:09:01.659 --> 00:09:03.919 public institution representatives on gender 00:09:03.919 --> 00:09:07.620 data and how to incorporate these gender development 00:09:07.620 --> 00:09:11.059 indicators into their departments. They're trained 00:09:11.059 --> 00:09:13.740 on gender responsive operational plans, including 00:09:13.740 --> 00:09:17.259 how to reduce barriers to access services for 00:09:17.259 --> 00:09:20.559 girls and women. They're also trained on gender 00:09:20.559 --> 00:09:23.360 responsive M &E, monitoring and evaluation and 00:09:23.360 --> 00:09:26.639 service delivery. They're also trained the public 00:09:26.639 --> 00:09:31.340 institutions on gender budgeting and how to adapt 00:09:31.340 --> 00:09:34.019 gender budgeting tools. Workshops are delivered. 00:09:34.440 --> 00:09:37.940 Gender equality CUSO volunteers are also deployed 00:09:37.940 --> 00:09:40.179 to the Caribbean. They're also trained these 00:09:40.179 --> 00:09:42.240 government departments on how to develop gender 00:09:42.240 --> 00:09:45.320 responsive capacity development plans. They're 00:09:45.320 --> 00:09:48.019 also trained on how to complete needs assessments 00:09:48.019 --> 00:09:51.899 to identify technical assistance needs for gender 00:09:51.899 --> 00:09:55.539 equality. So government staff are reporting increased 00:09:55.539 --> 00:09:58.340 knowledge and skill to carry out these needs 00:09:58.340 --> 00:10:01.220 assessments. So that gives you basically an idea 00:10:01.220 --> 00:10:07.399 of the Blueprints project. Now moving on to the 00:10:07.399 --> 00:10:12.580 Building Bricks CUSO project. It's going on in 00:10:12.580 --> 00:10:15.720 also the Caribbean, Belize, Dominica, Grenada, 00:10:15.879 --> 00:10:20.059 Guyana, Jamaica, St. Lucia, St. Vincent, Grenadines, 00:10:20.059 --> 00:10:23.269 and Suriname. And their goal is to influence 00:10:23.269 --> 00:10:27.090 legislative reform and increase access to services 00:10:27.090 --> 00:10:33.529 for those from the LGBTQIA plus lesbian, gay, 00:10:33.789 --> 00:10:39.809 bisexual, trans, gendered, queer community through 00:10:39.809 --> 00:10:42.830 organizations that represent those communities. 00:10:45.419 --> 00:10:48.320 What they often focus on is advocacy, training 00:10:48.320 --> 00:10:51.299 on how to design and deliver communication strategies. 00:10:53.159 --> 00:10:57.600 Also training on how to develop awareness campaigns, 00:10:57.799 --> 00:11:01.460 advocacy agendas, and how to sensitize and train 00:11:01.460 --> 00:11:07.580 service providers on the rights of the LGBTQIA 00:11:07.580 --> 00:11:12.080 plus community. And also training these organizations 00:11:12.080 --> 00:11:18.379 that represent the LGBTQIA plus communities in 00:11:18.379 --> 00:11:21.500 resource mobilization and monitoring and evaluation 00:11:21.500 --> 00:11:25.019 system design. Also training them on governance, 00:11:25.279 --> 00:11:28.659 leadership, ability to complete needs assessments, 00:11:28.899 --> 00:11:34.019 how to respond to the needs of these particular 00:11:34.019 --> 00:11:36.899 communities for both the organizations that represent 00:11:36.899 --> 00:11:40.100 them and the service providers. So the key target 00:11:40.100 --> 00:11:44.039 groups there are the lesbian, gay, bisexual, 00:11:44.379 --> 00:11:47.460 transgender, queer, IA plus organizations, as 00:11:47.460 --> 00:11:50.779 well as the service providers themselves. And 00:11:50.779 --> 00:11:54.519 the cost of that is building bricks is $5 million. 00:11:55.419 --> 00:11:59.639 And we have eight outcomes for building bricks 00:11:59.639 --> 00:12:03.399 and 15 outcome indicators we're going to look 00:12:03.399 --> 00:12:05.399 at from the performance measurement framework 00:12:05.399 --> 00:12:08.440 for those eight outcomes. So there you have a 00:12:08.440 --> 00:12:12.090 broad description of the two projects. And before 00:12:12.090 --> 00:12:15.409 we get into the PMF for both of them, to see 00:12:15.409 --> 00:12:20.529 if they actually did properly measure and evaluate 00:12:20.529 --> 00:12:23.450 the achievement of those outcomes for those projects, 00:12:24.309 --> 00:12:27.490 we again will talk about whether you are living 00:12:27.490 --> 00:12:29.830 in the Global South, where it's usually, but 00:12:29.830 --> 00:12:33.769 not always, hot and humid, or you work for CUSO 00:12:33.769 --> 00:12:36.570 International or for another international development 00:12:36.570 --> 00:12:40.409 organization. And lo and behold, you may have 00:12:40.409 --> 00:12:42.610 to go to the global self to actually monitor 00:12:42.610 --> 00:12:46.269 and evaluate one of your projects. And so it's 00:12:46.269 --> 00:12:48.429 hot and humid there. So you probably want to 00:12:48.429 --> 00:12:52.389 check out some sun hats, sunglasses, and some 00:12:52.389 --> 00:12:56.529 cool clothing. And if you do that, you'll see 00:12:56.529 --> 00:13:01.009 in the episode, I have a link called https colon 00:13:01.009 --> 00:13:07.009 backslash backslash mydeals .page backslash one 00:13:07.009 --> 00:13:12.960 h. And if you click on that link, you will get 00:13:12.960 --> 00:13:16.860 access to these three companies where you can 00:13:16.860 --> 00:13:21.299 get sun hats at ericjavits .com, where you can 00:13:21.299 --> 00:13:24.000 travel friendly and protectively from the sun 00:13:24.000 --> 00:13:27.419 with these hats. You get a 20 % discount off 00:13:27.419 --> 00:13:32.340 any product if you use my link, mydeals .page. 00:13:33.840 --> 00:13:35.940 For some reason, if you can't, you always use 00:13:35.940 --> 00:13:39.980 wand cool t -shirts, W -A -N -D -K -O -O -L t 00:13:39.980 --> 00:13:44.639 -shirts. But the link should work. And you'll 00:13:44.639 --> 00:13:47.700 get a 20 % discount. And I get a 20 % commission 00:13:47.700 --> 00:13:51.240 from the total purchase you make. So you might 00:13:51.240 --> 00:13:54.259 be wondering these sun hats at ericjavits .com. 00:13:54.360 --> 00:13:57.159 How protective are they from the sun? You get 00:13:57.159 --> 00:13:59.899 ultraviolet protection factor with a 50 plus 00:13:59.899 --> 00:14:05.080 ultra protection. factor rating, blocking 98 00:14:05.080 --> 00:14:10.279 % of UVA and UVB radiation. They're travel -friendly, 00:14:10.379 --> 00:14:12.980 easy to pack, but retains its shape when you 00:14:12.980 --> 00:14:15.860 visit your international development project 00:14:15.860 --> 00:14:18.820 located in the Global South. You can also pick 00:14:18.820 --> 00:14:23.879 up some sunglasses at akila .la, A -K -I -L -A 00:14:23.879 --> 00:14:27.679 dot L -A. Includes prescription lenses, all made 00:14:27.679 --> 00:14:32.870 from 50 % recycled material. You get a 10 % discount 00:14:32.870 --> 00:14:40.110 if you use my link, mydeals .page backslash 1HJL 00:14:40.110 --> 00:14:42.850 that you'll see in the episode notes. And I get 00:14:42.850 --> 00:14:45.350 a 15 % commission when you purchase any of the 00:14:45.350 --> 00:14:48.730 products using my promo code or that link at 00:14:48.730 --> 00:14:52.409 mydeals. So you have your hat and your sunglasses 00:14:52.409 --> 00:14:54.889 to protect you from the sun and the global south, 00:14:55.049 --> 00:14:58.730 but you need the right clothes to be comfortable 00:14:58.730 --> 00:15:01.899 in that hot. and humid climate. How about some 00:15:01.899 --> 00:15:06.740 men's polo t -shirts or button -ups at hypernaturalstyle 00:15:06.740 --> 00:15:11.720 .com. You get 20 % off any product when you use 00:15:11.720 --> 00:15:16.720 my promo code JuanCoolT -shirts or the link. 00:15:17.659 --> 00:15:20.860 And I get a 20 % commission of the total purchase 00:15:20.860 --> 00:15:25.580 you make when you use my link at my deals. So 00:15:25.580 --> 00:15:28.700 what is cool about these hypernatural t -shirts? 00:15:29.519 --> 00:15:32.419 They have a patent -pending hyper -cooled jade 00:15:32.419 --> 00:15:35.700 technology that, when it's hot, lowers your body 00:15:35.700 --> 00:15:41.200 temperature by 3 to 5 degrees. They also are 00:15:41.200 --> 00:15:44.919 breathable, thanks to Supina cotton. And they 00:15:44.919 --> 00:15:47.519 are sold in premium retailers like Nordstrom. 00:15:48.139 --> 00:15:50.659 And they were voted the best men's polo shirts 00:15:50.659 --> 00:15:56.320 in 2024 in Men's Journal, Forbes, and Esquire. 00:15:56.419 --> 00:16:06.450 So, that's basically it there. So let's start 00:16:06.450 --> 00:16:09.610 with blueprints for gender equality at a cost 00:16:09.610 --> 00:16:14.610 of $2 .5 million to the Canadian taxpayer. And 00:16:14.610 --> 00:16:17.230 we'll take a look at some of the outcomes. Nine 00:16:17.230 --> 00:16:21.049 outcomes, 13 outcome indicators. The first outcome 00:16:21.049 --> 00:16:24.850 is improved capacity of public institutions. 00:16:26.509 --> 00:16:29.110 in targeted Caribbean countries to share good 00:16:29.110 --> 00:16:32.669 practices and lessons learned on addressing the 00:16:32.669 --> 00:16:36.049 most pressing gender equality issues faced by 00:16:36.049 --> 00:16:38.690 the poorest, most vulnerable women and girls 00:16:38.690 --> 00:16:42.210 in all their diversity, including women's rights 00:16:42.210 --> 00:16:46.009 organizations. The first outcome indicator for 00:16:46.009 --> 00:16:49.570 that outcome is percent of ministries, departments, 00:16:49.830 --> 00:16:52.490 and agencies that have improved their capacity 00:16:52.490 --> 00:16:55.370 to coordinate with public institutions across 00:16:55.370 --> 00:16:58.669 the Caribbean. to share good practices and lessons 00:16:58.669 --> 00:17:02.090 learned, to address most pressing gender equality 00:17:02.090 --> 00:17:05.210 issues faced by poorest, most vulnerable women 00:17:05.210 --> 00:17:08.529 and girls, disaggregated by institution, sector, 00:17:08.750 --> 00:17:14.150 and geographic location. But if you look at the 00:17:14.150 --> 00:17:16.309 performance measurement framework, you'll find 00:17:16.309 --> 00:17:19.950 there's a conflict of interest, as CUSO and the 00:17:19.950 --> 00:17:22.849 government departments do a, quote, partnership 00:17:22.849 --> 00:17:26.880 review. So there's a self -reporting bias. Instead, 00:17:27.019 --> 00:17:29.940 what they should be doing is using an external 00:17:29.940 --> 00:17:32.880 evaluator should be measuring the technical capacity 00:17:32.880 --> 00:17:36.859 of the individuals inside of these public institutions 00:17:36.859 --> 00:17:41.779 to see if their capacity to share best practices 00:17:41.779 --> 00:17:46.079 on gender equality have actually been done. So 00:17:46.079 --> 00:17:48.299 that's not a very good indicator. They can't 00:17:48.299 --> 00:17:51.200 claim, once again, they've improved the capacity 00:17:51.200 --> 00:17:53.579 of these public institutions. because they have 00:17:53.579 --> 00:17:55.339 a conflict of interest. They need an external 00:17:55.339 --> 00:17:58.380 consultant to come in and use an objective measure 00:17:58.380 --> 00:18:03.380 instead. The next outcome they have is enhanced 00:18:03.380 --> 00:18:08.119 capacity of public institutions to engage with 00:18:08.119 --> 00:18:10.940 the poorest, most vulnerable women and girls 00:18:10.940 --> 00:18:14.160 in all their diversity, including women's rights 00:18:14.160 --> 00:18:17.400 organizations in targeted Caribbean countries. 00:18:18.000 --> 00:18:20.920 The outcome indicator for that outcome is... 00:18:21.160 --> 00:18:23.240 percent of ministries, departments, and agencies 00:18:23.240 --> 00:18:26.359 that have increased their knowledge about barriers 00:18:26.359 --> 00:18:29.839 faced by poorest, most vulnerable women and girls 00:18:29.839 --> 00:18:33.700 to increase engagement with poorest, most vulnerable 00:18:33.700 --> 00:18:36.460 women and girls, disaggregated by institution, 00:18:36.740 --> 00:18:40.880 sector, and geographic location. Again, if you 00:18:40.880 --> 00:18:42.339 look at the performance measurement framework, 00:18:42.660 --> 00:18:45.880 who's doing this? It's a conflict of interest 00:18:45.880 --> 00:18:48.319 as CUSO and the government departments do a quote 00:18:48.319 --> 00:18:52.000 partnership review. Instead, an external evaluator 00:18:52.000 --> 00:18:55.019 should be measuring the knowledge levels of the 00:18:55.019 --> 00:18:57.279 individuals inside of these public institutions 00:18:57.279 --> 00:19:00.099 to see if their knowledge levels of recalling 00:19:00.099 --> 00:19:03.849 and identifying to increasing engagement with 00:19:03.849 --> 00:19:07.089 vulnerable women has actually occurred. So again, 00:19:07.190 --> 00:19:10.170 this is not a good outcome indicator to claim 00:19:10.170 --> 00:19:12.589 that they're achieving, quote, enhanced capacity 00:19:12.589 --> 00:19:16.990 of public institutions. Doesn't work. The next 00:19:16.990 --> 00:19:19.789 outcome they've got here is increased knowledge 00:19:19.789 --> 00:19:22.910 and skills of public institutions to effectively 00:19:22.910 --> 00:19:26.650 deliver. gender responsive public programs and 00:19:26.650 --> 00:19:29.890 services that address the most pressing gender 00:19:29.890 --> 00:19:33.410 equality issues faced by the poorest, most vulnerable 00:19:33.410 --> 00:19:36.430 women and girls in all their diversity in targeted 00:19:36.430 --> 00:19:40.589 Caribbean countries. The indicator for that is 00:19:40.589 --> 00:19:43.190 percent of ministries, departments and agencies 00:19:43.190 --> 00:19:46.450 who have improved their performance in delivery 00:19:46.990 --> 00:19:50.210 of open inclusive and gender responsive public 00:19:50.210 --> 00:19:53.490 programs and services for poorest most vulnerable 00:19:53.490 --> 00:19:56.049 women and girls disaggregated by institution 00:19:56.049 --> 00:20:00.710 sector and geographic location yay in this one 00:20:00.710 --> 00:20:04.869 they get a green light they actually use and 00:20:04.869 --> 00:20:09.650 mention external consultant which is good And 00:20:09.650 --> 00:20:13.190 there are data sources in the PMF, document review, 00:20:13.390 --> 00:20:16.690 case studies, that suggest this can be done objectively 00:20:16.690 --> 00:20:21.190 and free from the public institutions self -reporting 00:20:21.190 --> 00:20:24.170 on their skill levels in delivering gender responsive 00:20:24.170 --> 00:20:28.289 services, which is biased and not a proper valid 00:20:28.289 --> 00:20:31.130 measure of this outcome. But in this case, they 00:20:31.130 --> 00:20:34.009 use an external consultant. So it begs the question, 00:20:34.130 --> 00:20:36.890 are they going to use an external consultant 00:20:36.890 --> 00:20:40.180 again? for some of the other outcome indicators 00:20:40.180 --> 00:20:44.200 that are coming up. Let's see. So, that's good. 00:20:44.579 --> 00:20:49.359 The next indicator, same outcome, increased knowledge 00:20:49.359 --> 00:20:51.460 and skills of public institutions, blah, blah, 00:20:51.480 --> 00:20:54.180 blah. The next indicator they've used for this 00:20:54.180 --> 00:20:57.140 same outcome is percent of ministries, departments, 00:20:57.440 --> 00:21:00.660 and agencies that have adopted results -based 00:21:00.660 --> 00:21:04.019 gender -responsive planning, monitoring and evaluation 00:21:04.019 --> 00:21:08.279 procedures and practices. disaggregated by institution, 00:21:08.779 --> 00:21:12.500 sector, and geographic location. Did they use 00:21:12.500 --> 00:21:15.099 an external consultant again for this? Sounds 00:21:15.099 --> 00:21:17.940 like they should. No, they didn't. What did they 00:21:17.940 --> 00:21:20.380 do? They dropped the external consultant here 00:21:20.380 --> 00:21:25.000 and instead are using CUSO, which is a conflict 00:21:25.000 --> 00:21:27.160 of interest. They should use the same external 00:21:27.160 --> 00:21:30.519 consultant to go in and check to see, maybe using 00:21:30.519 --> 00:21:33.940 a checklist. If the public institutions have 00:21:33.940 --> 00:21:36.660 indeed adopted monitoring and evaluation procedures. 00:21:37.079 --> 00:21:42.079 This in turn can then be used to see if they're 00:21:42.079 --> 00:21:44.799 able to achieve the outcome of skills to evaluate 00:21:44.799 --> 00:21:47.640 whether they have effectively delivered gender 00:21:47.640 --> 00:21:51.319 responsive services. Clear self -reporting bias 00:21:51.319 --> 00:21:54.980 and strange that they will not use the same external 00:21:54.980 --> 00:21:58.319 consultant. So that's not good. So far, only 00:21:58.319 --> 00:22:04.109 one good indicator. Now, again, New outcome here, 00:22:04.210 --> 00:22:07.890 enhanced gender responsive budgeting processes 00:22:07.890 --> 00:22:11.789 of public institutions to address the most pressing 00:22:11.789 --> 00:22:14.690 gender equality issues faced by the poorest, 00:22:14.769 --> 00:22:17.049 most vulnerable women and girls in all their 00:22:17.049 --> 00:22:20.269 diversity in targeted Caribbean countries. The 00:22:20.269 --> 00:22:22.569 outcome indicator they use for this outcome is 00:22:22.569 --> 00:22:25.190 percent of ministries, departments and agencies 00:22:25.190 --> 00:22:27.930 with increased knowledge and skills on gender 00:22:27.930 --> 00:22:32.000 and participatory. budgeting processes disaggregated 00:22:32.000 --> 00:22:35.720 by geographic location and sector of ministries, 00:22:35.720 --> 00:22:38.920 departments, and agencies. Great indicator, but 00:22:38.920 --> 00:22:43.059 no external consultant. Instead, again, CUSU 00:22:43.059 --> 00:22:45.720 is doing it, and even worse, they are asking 00:22:45.720 --> 00:22:48.299 the individuals in these public institutions 00:22:48.299 --> 00:22:52.640 to self -report on their knowledge levels in 00:22:52.640 --> 00:22:55.480 the target column of the performance measurement 00:22:55.480 --> 00:22:59.160 framework, and even ask them how quote satisfied 00:22:59.160 --> 00:23:03.000 they are with those knowledge and skill levels 00:23:03.000 --> 00:23:06.200 as noted in the data source of the performance 00:23:06.200 --> 00:23:09.279 measurement framework where it shows a satisfaction 00:23:09.279 --> 00:23:16.339 survey a clear again self -reporting bias instead 00:23:16.339 --> 00:23:19.480 they need to get that external consultant to 00:23:19.480 --> 00:23:22.960 develop some objective criteria of what one needs 00:23:22.960 --> 00:23:26.089 to see to conclude that the public institution 00:23:26.089 --> 00:23:29.130 does indeed have, quote, a gender responsive 00:23:29.130 --> 00:23:34.730 budgeting process in place. So again, they've 00:23:34.730 --> 00:23:36.769 done it once, an external consultant, but somehow 00:23:36.769 --> 00:23:39.130 they don't want to do it for the other indicators. 00:23:39.369 --> 00:23:43.369 Not good. The next outcome is increased knowledge 00:23:43.369 --> 00:23:45.950 and technical skills of public institutions to 00:23:45.950 --> 00:23:48.490 develop and implement gender responsive public 00:23:48.490 --> 00:23:52.309 policies, programs, and services that address 00:23:52.309 --> 00:23:55.150 the most pressing, Gender equality issues faced 00:23:55.150 --> 00:23:57.769 by the poorest, most vulnerable women and girls 00:23:57.769 --> 00:24:00.130 in all their diversity in targeted Caribbean 00:24:00.130 --> 00:24:02.670 countries. Now for this, they have three indicators 00:24:02.670 --> 00:24:06.750 for this one outcome. First indicator is average, 00:24:06.849 --> 00:24:10.029 open, inclusive, gender responsive and accountable 00:24:10.029 --> 00:24:14.809 score, public policies, programs and services. 00:24:16.589 --> 00:24:19.789 Design implementation for poorest, most vulnerable 00:24:19.789 --> 00:24:23.019 women and girls component. The other indicator 00:24:23.019 --> 00:24:26.940 they have for the same outcome is average open, 00:24:27.039 --> 00:24:29.019 inclusive, gender responsive and accountable 00:24:29.019 --> 00:24:34.220 participatory needs assessment score, gender 00:24:34.220 --> 00:24:37.579 equality component. And the third indicator for 00:24:37.579 --> 00:24:42.480 this is an average open, inclusive, gender responsive 00:24:42.480 --> 00:24:45.400 and accountable score for monitoring and evaluation 00:24:45.400 --> 00:24:49.339 and learning component to collect gender disaggregated 00:24:49.339 --> 00:24:56.819 data. So the three indicators for this results 00:24:56.819 --> 00:25:00.819 in a, quote, needs assessment report completed 00:25:00.819 --> 00:25:04.099 by, and quote, if you look at the PMF, joint 00:25:04.099 --> 00:25:07.480 sessions between CUSO and the individuals within 00:25:07.480 --> 00:25:11.299 these public institutions. Once again, they need 00:25:11.299 --> 00:25:13.640 to get an external consultant to measure their 00:25:13.640 --> 00:25:16.440 technical skill on the three areas that would 00:25:16.440 --> 00:25:20.369 lead one to conclude. The skill on, quote, developing 00:25:20.369 --> 00:25:22.670 and implementing gender responsive policies, 00:25:22.930 --> 00:25:26.309 programs and services has indeed increased. Right 00:25:26.309 --> 00:25:30.250 now, we have a conflict of interest where there's 00:25:30.250 --> 00:25:33.349 no external consultant. And CUSO is doing this 00:25:33.349 --> 00:25:36.430 internally with the public institutions that 00:25:36.430 --> 00:25:38.789 they're trying to measure and evaluate. Not good. 00:25:38.910 --> 00:25:43.279 And also a needs assessment report. doesn't seem 00:25:43.279 --> 00:25:45.779 to be a valid measure of increased skill on, 00:25:45.799 --> 00:25:48.940 quote, developing and implementing gender responsive 00:25:48.940 --> 00:25:53.519 policies, programs, and services. So that's the 00:25:53.519 --> 00:25:55.779 problem there. Moving along to intermediate outcomes, 00:25:56.019 --> 00:26:00.440 we have enhanced gender responsive public policies, 00:26:00.640 --> 00:26:03.960 programs, and services to address the most pressing 00:26:03.960 --> 00:26:07.339 gender equality issues faced by the poorest, 00:26:07.420 --> 00:26:09.940 most vulnerable women and girls in all their 00:26:09.940 --> 00:26:13.339 diversity. in targeted Caribbean countries and 00:26:13.339 --> 00:26:18.259 the outcome indicator for that is average ministries 00:26:18.259 --> 00:26:21.400 departments and agencies open inclusive and gender 00:26:21.400 --> 00:26:25.519 responsive participatory needs assessment score 00:26:25.519 --> 00:26:30.339 so in the performance measurement framework the 00:26:30.339 --> 00:26:33.279 data collection methods are between CUSO and 00:26:33.279 --> 00:26:36.440 the public institutions producing a needs assessment 00:26:36.440 --> 00:26:39.819 report this has nothing to do with measuring 00:26:40.410 --> 00:26:43.309 whether the services they delivered were gender 00:26:43.309 --> 00:26:46.869 responsive. You need to ask the women and girls 00:26:46.869 --> 00:26:51.029 who received the services from these public institutions 00:26:51.029 --> 00:26:55.390 if the services were gender responsive. So again, 00:26:55.549 --> 00:27:00.150 a lousy indicator that doesn't measure at all 00:27:00.150 --> 00:27:03.930 enhanced gender responsive services. The next 00:27:03.930 --> 00:27:07.549 outcome is increased effective delivery of gender 00:27:07.549 --> 00:27:11.269 responsive services. that address the most pressing 00:27:11.269 --> 00:27:13.549 gender equality issues faced by the poorest, 00:27:13.670 --> 00:27:15.970 most vulnerable women and girls in all their 00:27:15.970 --> 00:27:19.269 diversity in targeted Caribbean countries. The 00:27:19.269 --> 00:27:22.869 outcome indicator from the PMF is percent of 00:27:22.869 --> 00:27:25.130 poorest, most vulnerable women and girls with 00:27:25.130 --> 00:27:28.730 effective access to open, inclusive, gender responsive 00:27:28.730 --> 00:27:31.809 and accountable public programs and services 00:27:31.809 --> 00:27:35.630 in four countries. Disaggregated by socioeconomic 00:27:35.630 --> 00:27:39.240 status, gender. age, ability, ethnicity, language, 00:27:39.460 --> 00:27:42.619 geographic location, and type of service. Sounds 00:27:42.619 --> 00:27:46.039 great. Problem is when you look at the PMF, the 00:27:46.039 --> 00:27:52.940 data sources are CUSO blueprint reports. Doesn't 00:27:52.940 --> 00:27:55.339 the access come from the locations where these 00:27:55.339 --> 00:27:57.799 women and girls show up to receive the services? 00:27:58.119 --> 00:28:01.240 So I guess we are relying on these blueprint 00:28:01.240 --> 00:28:05.009 reports that contain the percent of women and 00:28:05.009 --> 00:28:07.910 girls that receive the gender responsive services. 00:28:08.190 --> 00:28:10.549 To show that this access has increased, a better 00:28:10.549 --> 00:28:13.049 word would be maybe effective, we need to see 00:28:13.049 --> 00:28:16.250 the measuring instrument as well as the criterion 00:28:16.250 --> 00:28:19.289 used to conclude that the services delivered 00:28:19.289 --> 00:28:22.970 indeed meet the standard of being described as 00:28:22.970 --> 00:28:27.170 being gender responsive. Again, they should use 00:28:27.170 --> 00:28:29.470 an external consultant reviewing the records 00:28:29.470 --> 00:28:32.970 where the service was delivered rather than using 00:28:32.970 --> 00:28:36.930 a CUSO blueprint report because that would have 00:28:36.930 --> 00:28:40.250 no self -reporting bias so that's the problem 00:28:40.250 --> 00:28:44.250 there is again they're relying on internal QSO 00:28:44.250 --> 00:28:48.490 reports when if you really want to see if these 00:28:48.490 --> 00:28:51.390 girls and women have had effective access to 00:28:51.390 --> 00:28:54.789 gender responsive services you would want to 00:28:54.789 --> 00:28:59.029 have the external consultant go in and review 00:28:59.029 --> 00:29:02.509 the reports That would be a better approach. 00:29:02.769 --> 00:29:05.549 The next outcome is actually for the same indicator, 00:29:05.809 --> 00:29:08.589 excuse me, for the same outcome, increased effective 00:29:08.589 --> 00:29:13.269 delivery of gender responsive services, was level 00:29:13.269 --> 00:29:17.670 of satisfaction of poorest, most vulnerable women 00:29:17.670 --> 00:29:20.309 and girls in all their diversity with open, inclusive, 00:29:20.529 --> 00:29:23.430 gender responsive and accountable public programs 00:29:23.430 --> 00:29:25.450 and services in four countries. This is great. 00:29:25.630 --> 00:29:27.710 Great indicator. They're actually going to go 00:29:27.710 --> 00:29:31.710 to the women and girls and ask them. Are you 00:29:31.710 --> 00:29:34.509 satisfied with the services? And are they indeed, 00:29:34.730 --> 00:29:37.609 in your opinion, gender responsive? We'll give 00:29:37.609 --> 00:29:39.950 you a definition of what we mean by that. Fantastic. 00:29:40.529 --> 00:29:44.869 Problem again is that CUSO and the public institutions, 00:29:45.130 --> 00:29:48.529 not an external consultant, are responsible for 00:29:48.529 --> 00:29:51.529 asking these women and girls how satisfied are 00:29:51.529 --> 00:29:54.130 they with the services they receive from these 00:29:54.130 --> 00:29:57.809 public institutions. Now, it's funny. They used 00:29:57.809 --> 00:30:00.250 an external consultant before, as I mentioned. 00:30:00.670 --> 00:30:03.710 So why not for this very good indicator? A good 00:30:03.710 --> 00:30:07.910 indicator ruined by CUSO public institution conflict 00:30:07.910 --> 00:30:11.509 of interest. Get an external consultant to complete 00:30:11.509 --> 00:30:14.410 the satisfaction survey and produce the report 00:30:14.410 --> 00:30:19.609 that claims, and I quote, increased effective 00:30:19.609 --> 00:30:23.289 delivery of gender responsive services. The current 00:30:23.289 --> 00:30:27.829 blueprint reports as the data source in the PMF 00:30:27.829 --> 00:30:32.240 are potentially biased. The next outcome statement 00:30:32.240 --> 00:30:36.119 improved collaboration and dialogue across public 00:30:36.119 --> 00:30:38.960 institutions in targeted Caribbean countries 00:30:38.960 --> 00:30:43.079 on the most pressing gender equality issues faced 00:30:43.079 --> 00:30:45.200 by the poorest, most vulnerable women and girls 00:30:45.200 --> 00:30:48.039 in all their diversity. The outcome indicator 00:30:48.039 --> 00:30:52.279 for that outcome? Percent of staff who recognize 00:30:52.279 --> 00:30:55.880 public institutions as key actors in sharing 00:30:55.880 --> 00:30:58.720 knowledge to address most pressing gender equality 00:30:58.720 --> 00:31:02.099 issues. faced by poorest, most vulnerable women 00:31:02.099 --> 00:31:06.640 and girls. So they've got increased recognition. 00:31:08.720 --> 00:31:12.319 Recognize. So increased recognition, knowledge. 00:31:12.539 --> 00:31:15.660 It does not equate with, quote, the outcome, 00:31:15.839 --> 00:31:18.900 improved collaboration and dialogue, which is 00:31:18.900 --> 00:31:22.680 a behavior, right? They need a better indicator 00:31:22.680 --> 00:31:25.440 for increased collaboration and dialogue. such 00:31:25.440 --> 00:31:28.119 as either frequency of meetings and communications 00:31:28.119 --> 00:31:31.880 specific to actions to address gender equality. 00:31:33.539 --> 00:31:36.200 Note, this is still process rather than results, 00:31:36.500 --> 00:31:40.279 where process is not measuring any changes in 00:31:40.279 --> 00:31:44.039 increased gender equality. So that's an example 00:31:44.039 --> 00:31:46.380 where they've got an indicator, but it's not 00:31:46.380 --> 00:31:48.400 related to measuring the outcome of improved 00:31:48.400 --> 00:31:53.359 collaboration and dialogue. Then finally, the 00:31:53.359 --> 00:31:57.779 ultimate outcome in this project is enhanced 00:31:57.779 --> 00:32:03.940 gender responsive governance in Dominica, Grenada, 00:32:04.039 --> 00:32:06.039 St. Lucian and St. Vincent and the Grenadines 00:32:06.039 --> 00:32:08.500 for the poorest, most vulnerable women and girls 00:32:08.500 --> 00:32:11.680 in all their diversity. The outcome indicator 00:32:11.680 --> 00:32:17.400 for this ultimate outcome is number of poorest, 00:32:17.400 --> 00:32:20.670 most vulnerable women and girls reached. by open, 00:32:20.710 --> 00:32:22.890 inclusive, and gender -responsive public programs 00:32:22.890 --> 00:32:25.930 and services in four countries. Now, this is 00:32:25.930 --> 00:32:28.529 a bit of a problem because how is this any different 00:32:28.529 --> 00:32:31.250 than the indicator for 1200, which was percent 00:32:31.250 --> 00:32:33.869 of women and girls with effective access to gender 00:32:33.869 --> 00:32:36.170 -responsive services? It looks like they're going 00:32:36.170 --> 00:32:40.190 countrywide here. It would be better to actually 00:32:40.190 --> 00:32:43.710 use a measure of gender equality that is an expected 00:32:43.710 --> 00:32:47.430 outcome if there is an increase in gender -responsive 00:32:47.430 --> 00:32:51.720 services delivered. Where more women and girls 00:32:51.720 --> 00:32:54.299 get those gender responsive services because 00:32:54.299 --> 00:32:56.579 they've already measured that in some of the 00:32:56.579 --> 00:32:58.539 other outcome indicators that I just mentioned. 00:32:58.599 --> 00:33:01.599 So the next level would be at the country level 00:33:01.599 --> 00:33:05.859 is governance regarding gender responsiveness 00:33:05.859 --> 00:33:09.279 and gender equality being achieved for women 00:33:09.279 --> 00:33:13.160 outside of the project, right? So an example 00:33:13.160 --> 00:33:16.910 could be the UNDP. Gender inequality index, where 00:33:16.910 --> 00:33:19.529 they look at maternal mortality, adolescent birth 00:33:19.529 --> 00:33:23.049 rate, females that have secondary education, 00:33:23.349 --> 00:33:25.910 the female share of parliamentary seats, the 00:33:25.910 --> 00:33:30.109 female labor force participation rate. Now, although 00:33:30.109 --> 00:33:33.829 attributing the project to those changes in the 00:33:33.829 --> 00:33:37.789 gender inequality index would be difficult, at 00:33:37.789 --> 00:33:41.250 least it should be compared with the project 00:33:41.250 --> 00:33:45.460 measure. of increased access to gender responsive 00:33:45.460 --> 00:33:49.220 services. Because I think in the outcome, the 00:33:49.220 --> 00:33:51.259 ultimate outcome, they're implying we're going 00:33:51.259 --> 00:33:54.339 to go to the country level outside of the project. 00:33:54.500 --> 00:34:00.640 In summary, for this project, you've really got 00:34:00.640 --> 00:34:03.299 13 outcome indicators, but all of them are bad 00:34:03.299 --> 00:34:06.180 except one. There's only one really good indicator. 00:34:06.440 --> 00:34:09.519 There's just too much self -reporting bias going 00:34:09.519 --> 00:34:13.139 on. And in some cases, the indicator is not even 00:34:13.139 --> 00:34:16.480 related to the achievement of the outcome. So 00:34:16.480 --> 00:34:19.239 that's the big problem with this performance 00:34:19.239 --> 00:34:21.980 measurement framework. So I could, again, would 00:34:21.980 --> 00:34:25.280 conclude that they're not achieving any of their 00:34:25.280 --> 00:34:27.320 outcomes because again, they're not properly 00:34:27.320 --> 00:34:29.440 measuring them. So they can't make the claim 00:34:29.440 --> 00:34:33.340 that their outcomes are being achieved. So let's 00:34:33.340 --> 00:34:35.920 look at Building Brooks for $5 million cost to 00:34:35.920 --> 00:34:39.260 the Canadian taxpayer. They've got eight outcomes 00:34:39.260 --> 00:34:45.280 and For those outcomes, they've got 15 outcome 00:34:45.280 --> 00:34:50.340 indicators. So if we start from the bottom with 00:34:50.340 --> 00:34:54.300 the immediate outcomes that they expect to achieve, 00:34:54.579 --> 00:34:58.920 we've got enhanced coordination and collaboration 00:34:58.920 --> 00:35:03.320 action on rights -based agendas among local and 00:35:03.320 --> 00:35:08.539 regional LGBTQIA plus organizations and networks 00:35:08.539 --> 00:35:12.230 in the Caribbean. The outcome indicator for that 00:35:12.230 --> 00:35:15.909 was number of organizations and networks coordinating 00:35:15.909 --> 00:35:21.130 human rights and gender equality agendas. Now, 00:35:21.150 --> 00:35:24.429 you're probably suspecting already, it's not 00:35:24.429 --> 00:35:28.469 directly related to the outcome. How does increase 00:35:28.469 --> 00:35:31.690 in the number of organizations equate with, quote, 00:35:31.849 --> 00:35:34.769 enhanced coordination? Even if the increase in 00:35:34.769 --> 00:35:37.449 number of organization happens, was it due to 00:35:37.449 --> 00:35:39.840 the project output? Or could they increase that 00:35:39.840 --> 00:35:42.539 number by themselves without you spending $5 00:35:42.539 --> 00:35:47.219 million? The project creates, quote, dialogue 00:35:47.219 --> 00:35:49.679 spaces that are then used to do this coordination. 00:35:50.219 --> 00:35:52.760 But could this happen without the creation of 00:35:52.760 --> 00:35:55.920 these dialogue spaces? And if so, would this 00:35:55.920 --> 00:35:59.019 increase in the number of organizations doing 00:35:59.019 --> 00:36:02.500 this coordination, would it be much greater than 00:36:02.500 --> 00:36:05.659 without these project dialogue spaces? So this 00:36:05.659 --> 00:36:08.710 is a case where... Yes, they could maybe claim 00:36:08.710 --> 00:36:11.849 increased coordination and enhanced coordination, 00:36:12.170 --> 00:36:14.650 but maybe it would happen anyways without the 00:36:14.650 --> 00:36:16.510 project. So this is where they're not really 00:36:16.510 --> 00:36:20.409 looking at impact. So it's a little questionable, 00:36:20.530 --> 00:36:26.469 right? So the next outcome is, again, enhanced 00:36:26.469 --> 00:36:29.170 coordination. And they have another indicator 00:36:29.170 --> 00:36:32.630 for it. And they have number of organizations 00:36:32.630 --> 00:36:36.679 and networks reporting increased ability. to 00:36:36.679 --> 00:36:39.940 participate in legislative reform in the Caribbean. 00:36:40.159 --> 00:36:44.480 Again, self -reporting bias, right? If you want 00:36:44.480 --> 00:36:46.920 to see if this project has actually increased 00:36:46.920 --> 00:36:49.420 their ability, their skills and knowledge to 00:36:49.420 --> 00:36:53.659 engage in trying to get legislative reform, then 00:36:53.659 --> 00:36:56.679 don't ask them themselves, hey, how you feel? 00:36:56.840 --> 00:36:59.460 Don't. That's self -reporting bias. You need 00:36:59.460 --> 00:37:01.980 to get an external consultant to objectively 00:37:01.980 --> 00:37:05.099 measure. whether the individuals within these 00:37:05.099 --> 00:37:08.420 organizations have indeed increased their ability 00:37:08.420 --> 00:37:12.039 to participate in legislative reform. So you've 00:37:12.039 --> 00:37:16.039 got self -reporting bias. The next outcome statement, 00:37:16.420 --> 00:37:19.639 increased capacity of local and regional organizations 00:37:19.639 --> 00:37:25.559 and networks to advocate and campaign for LGBTQIA 00:37:25.559 --> 00:37:29.780 plus rights and gender equality using an intersectional 00:37:29.780 --> 00:37:33.380 approach. The outcome indicator here is number 00:37:33.380 --> 00:37:36.559 of individuals reached through national and regional 00:37:36.559 --> 00:37:40.659 campaigns. The survey that they use here in the 00:37:40.659 --> 00:37:44.920 PMF on reach is excellent. So it's very good. 00:37:45.340 --> 00:37:48.239 That indicator is good. The problem is the other 00:37:48.239 --> 00:37:52.239 indicators that they use for the same outcome. 00:37:52.940 --> 00:37:55.980 Number of large -scale national regional campaigns 00:37:55.980 --> 00:38:00.179 designed and launched. So that's good too. That's 00:38:00.179 --> 00:38:02.519 the second indicator that's good because as the 00:38:02.519 --> 00:38:05.440 number of launches goes up, that shows they've 00:38:05.440 --> 00:38:09.840 increased their capacity to advocate and campaign 00:38:09.840 --> 00:38:16.619 for LGBTQIA plus rights. So these are two indicators 00:38:16.619 --> 00:38:19.480 that are actually quite good. So that's well 00:38:19.480 --> 00:38:23.579 noted. But the third indicator, once again, is 00:38:23.579 --> 00:38:27.940 biased. And that is... Number of these organizations 00:38:27.940 --> 00:38:30.699 that report increased ability to advocate. I 00:38:30.699 --> 00:38:32.940 would just take it out. This is self -reporting 00:38:32.940 --> 00:38:36.800 bias. Getting the number launched is good. Getting 00:38:36.800 --> 00:38:40.139 the number that have been reached is good. And 00:38:40.139 --> 00:38:42.099 that by themselves, those two indicators are 00:38:42.099 --> 00:38:44.440 good to show that they've actually increased 00:38:44.440 --> 00:38:48.780 their capacity to advocate and campaign for those 00:38:48.780 --> 00:38:53.539 rights. That's a good example. So not bad there. 00:38:54.760 --> 00:39:01.940 Next outcome, increased access by LGBTQIA plus 00:39:01.940 --> 00:39:05.059 people to safe, inclusive, environmentally friendly, 00:39:05.360 --> 00:39:08.400 and gender responsive socioeconomic services 00:39:08.400 --> 00:39:10.760 in targeted countries. The problem here again 00:39:10.760 --> 00:39:14.440 is the outcome indicator is the number of organizations 00:39:14.440 --> 00:39:17.500 and networks that report increased ability to 00:39:17.500 --> 00:39:21.079 coordinate with and strengthen the ability of 00:39:21.079 --> 00:39:23.739 service providers to deliver quality services. 00:39:25.039 --> 00:39:27.760 Self -reporting, again, it is biased. So this 00:39:27.760 --> 00:39:31.139 should be taken out. But good news, if we look 00:39:31.139 --> 00:39:37.059 at the next indicator for increased access, they 00:39:37.059 --> 00:39:40.280 actually go to the people in those communities, 00:39:40.380 --> 00:39:46.760 LGBTQIA +, and ask them directly the number that 00:39:46.760 --> 00:39:50.579 have reported increased access to services. Very 00:39:50.579 --> 00:39:54.769 good indicator. of the outcome if we mean improved 00:39:54.769 --> 00:39:58.269 or adequate quality in achieving gender responsive 00:39:58.269 --> 00:40:02.369 services is equivalent to increased access we 00:40:02.369 --> 00:40:06.449 have to be careful here just as the number goes 00:40:06.449 --> 00:40:08.690 up that are reporting they got the right services 00:40:08.690 --> 00:40:13.869 we should also make sure that the quality is 00:40:13.869 --> 00:40:16.650 adequate too but we're going to assume that for 00:40:16.650 --> 00:40:20.280 now and it is in the pmf The surveys seem to 00:40:20.280 --> 00:40:23.159 be doing that. So that's another measure. That's 00:40:23.159 --> 00:40:26.119 three good indicators now. So that's good. But 00:40:26.119 --> 00:40:29.380 again, we get back to problem of self -reporting 00:40:29.380 --> 00:40:31.860 with this next outcome. Increase knowledge and 00:40:31.860 --> 00:40:35.980 skills of local and regional organization and 00:40:35.980 --> 00:40:38.980 networks in governance, leadership, and management 00:40:38.980 --> 00:40:44.590 to represent LGBTQ. plus rights in targeted countries. 00:40:44.829 --> 00:40:47.329 So as you remember from this project, there's 00:40:47.329 --> 00:40:50.590 lots of training going on for these individuals. 00:40:51.130 --> 00:40:53.889 So what they have to do is stop asking them how 00:40:53.889 --> 00:40:57.989 they feel. Stop asking them to self -report on 00:40:57.989 --> 00:41:00.050 whether or not their skills have been increased. 00:41:00.909 --> 00:41:03.670 It's getting ridiculous. It just doesn't look 00:41:03.670 --> 00:41:06.170 good. What they need to do is get an external 00:41:06.170 --> 00:41:09.469 measure of those skills in leadership and management. 00:41:09.550 --> 00:41:12.769 Get a consultant to come in. And say, sorry, 00:41:12.909 --> 00:41:15.309 I'm going to measure whether or not you've learned 00:41:15.309 --> 00:41:17.809 what you're supposed to learn. I'm going to test 00:41:17.809 --> 00:41:22.929 you. Very easy to do. Self -reporting, bias, 00:41:23.070 --> 00:41:29.530 not good. And they also do it again for another 00:41:29.530 --> 00:41:33.190 indicator on human resources governance and monitoring 00:41:33.190 --> 00:41:36.989 and evaluation. Self -reporting, where the indicator 00:41:36.989 --> 00:41:40.199 is percent of staff of organizations. Reporting 00:41:40.199 --> 00:41:42.280 increased knowledge and abilities in human resources 00:41:42.280 --> 00:41:44.920 governance and monitoring and evaluation. Again, 00:41:45.079 --> 00:41:47.920 they're using self -reporting. They need to use 00:41:47.920 --> 00:41:50.840 an external measure of increased knowledge of 00:41:50.840 --> 00:41:52.699 human resources governance and monitoring and 00:41:52.699 --> 00:41:56.219 evaluation. Next indicator after these immediate 00:41:56.219 --> 00:41:59.159 incomes is the intermediate outcomes. Here's 00:41:59.159 --> 00:42:00.940 the outcome statement. Increased effectiveness 00:42:00.940 --> 00:42:04.280 of local and regional organizations and networks 00:42:04.280 --> 00:42:10.059 to coordinate the LGBTQ. IA plus rights movement 00:42:10.059 --> 00:42:12.900 in the Caribbean. The outcome indicator is number 00:42:12.900 --> 00:42:16.260 of organizations and networks, again, reporting 00:42:16.260 --> 00:42:18.920 increased effectiveness to implement mechanisms 00:42:18.920 --> 00:42:22.719 that positively influence gender sensitive policies. 00:42:22.960 --> 00:42:26.780 Again, a bias. Don't ask them. Self -reporting. 00:42:26.800 --> 00:42:29.559 Get an external measure of increased effectiveness 00:42:29.559 --> 00:42:33.659 to implement mechanisms that influence gender 00:42:33.659 --> 00:42:36.989 sensitive policies instead. This could complement 00:42:36.989 --> 00:42:39.909 the next measure on effectiveness to coordinate. 00:42:41.130 --> 00:42:43.929 They should also add what exactly would a gender 00:42:43.929 --> 00:42:47.190 -sensitive policy look like in terms of legislation. 00:42:47.570 --> 00:42:50.349 So that could be aimed for. Maybe that comes 00:42:50.349 --> 00:42:55.510 later. So another indicator for increased effectiveness 00:42:55.510 --> 00:42:59.349 is number of policies, laws, and frameworks presented 00:42:59.349 --> 00:43:03.860 for legislative reform by... organizations, and 00:43:03.860 --> 00:43:06.380 networks. This is a great measure of increased 00:43:06.380 --> 00:43:10.860 effectiveness to coordinate. But be aware, just 00:43:10.860 --> 00:43:13.400 because they've increased the number of policies 00:43:13.400 --> 00:43:16.219 presented, they still could be garbage, right? 00:43:16.539 --> 00:43:19.579 You need someone to look at the quality of these 00:43:19.579 --> 00:43:22.539 policies, laws, and frameworks. Is it sufficient 00:43:22.539 --> 00:43:26.739 as reviewed by external experts or even by the 00:43:26.739 --> 00:43:31.900 government agencies that have received this advocacy 00:43:31.900 --> 00:43:35.679 document? or recommended policy. It doesn't matter. 00:43:35.719 --> 00:43:37.360 It could be an external expert. It could be the 00:43:37.360 --> 00:43:39.219 civil servant that's received the document and 00:43:39.219 --> 00:43:42.280 says, this is crap. It doesn't even include any 00:43:42.280 --> 00:43:46.559 data to support the change in legislation and 00:43:46.559 --> 00:43:50.139 the need for this change. So not just the number 00:43:50.139 --> 00:43:54.659 presented, but at least they can claim that they're 00:43:54.659 --> 00:43:59.219 doing it. But increased effectiveness, you need 00:43:59.219 --> 00:44:01.880 to look at the quality and even better. to see 00:44:01.880 --> 00:44:05.639 if it made a difference indeed in any changes 00:44:05.639 --> 00:44:10.420 in legislation. The next indicator is enhanced 00:44:10.420 --> 00:44:13.539 performance of local and regional organizations 00:44:13.539 --> 00:44:16.480 and networks, gender responsive programming and 00:44:16.480 --> 00:44:22.980 advocacy for rights of LGBTQIA plus people and 00:44:22.980 --> 00:44:25.460 gender equality in targeted countries in the 00:44:25.460 --> 00:44:28.699 Caribbean. The outcome indicator there, again, 00:44:30.380 --> 00:44:33.119 Self -reporting bias, number of organizations 00:44:33.119 --> 00:44:36.480 and networks that report increased ability to 00:44:36.480 --> 00:44:41.800 advocate for the rights of LGBTQIA plus people. 00:44:42.420 --> 00:44:44.840 So again, a problem with self -reporting. Instead, 00:44:45.019 --> 00:44:47.659 they need an external measure, maybe an external 00:44:47.659 --> 00:44:52.400 consultant of enhanced performance instead. If 00:44:52.400 --> 00:44:54.440 their performance was convincing and enhanced, 00:44:54.719 --> 00:44:58.000 you would expect a press release or even better. 00:44:58.539 --> 00:45:02.280 legislation indicating a change, reflecting the 00:45:02.280 --> 00:45:05.599 changes that were advocated for in that, quote, 00:45:05.780 --> 00:45:10.360 enhanced performance. Another indicator for enhanced 00:45:10.360 --> 00:45:14.420 performance is number of organizations and networks 00:45:14.420 --> 00:45:17.679 with increased delivery of programming. So they're 00:45:17.679 --> 00:45:20.139 asking for increases in programming in this advocacy. 00:45:20.400 --> 00:45:26.119 That's what they want, right? Increased delivery 00:45:26.119 --> 00:45:29.579 of programming. So why not just look at that? 00:45:29.659 --> 00:45:32.619 Forget about the number of organizations that 00:45:32.619 --> 00:45:36.179 are doing it. Under the target column, they have 00:45:36.179 --> 00:45:38.159 the number of organizations that have improved 00:45:38.159 --> 00:45:42.840 their delivery of services. But they want more 00:45:42.840 --> 00:45:45.760 services also, not just improved services. Instead, 00:45:46.019 --> 00:45:50.099 list the new programs and what they were delivering 00:45:50.099 --> 00:45:52.559 and the number of people receiving the program 00:45:52.559 --> 00:45:56.199 by type of gender identity to see if it has increased. 00:45:56.860 --> 00:46:00.000 due to this project's training on advocacy and 00:46:00.000 --> 00:46:04.219 lobbying for increased access to services. Have 00:46:04.219 --> 00:46:07.159 the number of people receiving services increased 00:46:07.159 --> 00:46:11.380 is what should be measured here as the expected 00:46:11.380 --> 00:46:17.079 outcome. So here the indicator is not relevant 00:46:17.079 --> 00:46:19.719 to measuring the outcome of enhanced performance 00:46:19.719 --> 00:46:23.380 to advocate for rights, right? You want to go 00:46:23.380 --> 00:46:25.969 to the end and see. If there's been an increase 00:46:25.969 --> 00:46:28.690 in the number of services delivered and also 00:46:28.690 --> 00:46:33.389 perhaps also legislative reform, because usually 00:46:33.389 --> 00:46:36.690 they go hand in hand. Another outcome here is 00:46:36.690 --> 00:46:39.829 improved institutional capacity and sustainability 00:46:39.829 --> 00:46:42.989 of local and regional organizations and networks 00:46:42.989 --> 00:46:46.170 in targeted countries in the Caribbean. The outcome 00:46:46.170 --> 00:46:49.369 indicator for that outcome, number of organizations 00:46:49.369 --> 00:46:53.699 using sustainability, best practices. Now, when 00:46:53.699 --> 00:46:55.639 you read that, that's pretty good. But again, 00:46:55.860 --> 00:46:57.280 if you don't have the performance measurement 00:46:57.280 --> 00:46:59.380 framework in front of you, there's a little trick 00:46:59.380 --> 00:47:02.559 they're playing here. The outcome and its indicator 00:47:02.559 --> 00:47:05.739 that I just stated are very good, very clear. 00:47:05.980 --> 00:47:09.019 But if you look at the baseline data column in 00:47:09.019 --> 00:47:11.280 the performance measurement framework, it's not 00:47:11.280 --> 00:47:14.360 related to the indicator. Instead of going into 00:47:14.360 --> 00:47:17.320 an organization to see if they're actually, quote, 00:47:17.480 --> 00:47:21.559 using sustainability best practices. They are 00:47:21.559 --> 00:47:23.760 just checking to see if the organization has 00:47:23.760 --> 00:47:28.360 a, quote, sustainability plan. That is not actually 00:47:28.360 --> 00:47:32.139 a practice or a behavior. It's just a piece of 00:47:32.139 --> 00:47:36.579 paper, a plan. Are they actually, you know, maybe 00:47:36.579 --> 00:47:39.519 in the budget? And for example, I'll describe. 00:47:39.659 --> 00:47:43.199 The target column only counts how many organizations 00:47:43.199 --> 00:47:47.059 were trained in sustainability. That is not relevant 00:47:47.059 --> 00:47:51.829 to the outcome statement. The target column does 00:47:51.829 --> 00:47:55.409 also have a measure of the number of organizations, 00:47:55.630 --> 00:48:00.250 quote, implementing best practices. Let's hope 00:48:00.250 --> 00:48:03.769 there is a manual on the criteria on how to decide 00:48:03.769 --> 00:48:07.710 that an organization is indeed implementing sustainability. 00:48:08.170 --> 00:48:12.550 What is that sustainability? Well, one example, 00:48:12.769 --> 00:48:16.329 a reduction in foreign funds and an increase 00:48:16.329 --> 00:48:19.469 in developing country funds to become less dependent. 00:48:20.030 --> 00:48:24.170 on foreign aid, i .e. more sustainable. So what 00:48:24.170 --> 00:48:26.889 you could do is actually show as a better indicator 00:48:26.889 --> 00:48:31.710 the budget of the organization and seeing a reduction 00:48:31.710 --> 00:48:35.750 in the foreign funding from CUSO or anybody else 00:48:35.750 --> 00:48:38.710 and an increase in the local fundraising where 00:48:38.710 --> 00:48:41.389 they've got money either from the private sector 00:48:41.389 --> 00:48:45.789 or and or from the government, their local government, 00:48:45.929 --> 00:48:48.889 right? So that's an example where the indicator 00:48:49.710 --> 00:48:53.789 Sounds good, but when you look in the PMF, the 00:48:53.789 --> 00:48:56.329 data that they're actually collecting is not 00:48:56.329 --> 00:48:58.710 related to measuring the achievement of the outcome 00:48:58.710 --> 00:49:05.269 or even practicing the outcome indicator. The 00:49:05.269 --> 00:49:08.449 next outcome is increased realization of human 00:49:08.449 --> 00:49:14.849 rights by LGBTQIA plus people and advancement 00:49:14.849 --> 00:49:18.929 of gender equality in. Belize, Dominica, Grenada, 00:49:19.050 --> 00:49:21.409 Guyana, Jamaica, St. Louis, St. Vincent, and 00:49:21.409 --> 00:49:23.570 the Grenadines, and Suriname in the Caribbean. 00:49:24.010 --> 00:49:29.309 The outcome indicator is number of LGBTQIA people 00:49:29.309 --> 00:49:33.769 who have realized their human rights. Now, if 00:49:33.769 --> 00:49:35.769 you look at the data source in the performance 00:49:35.769 --> 00:49:39.409 measurement framework, it indicates, quote, project 00:49:39.409 --> 00:49:46.960 reports along with a baseline total of 847. Now, 00:49:46.980 --> 00:49:49.780 this suggests only those in the CUSO project 00:49:49.780 --> 00:49:54.239 are being surveyed. So there is a bias. This 00:49:54.239 --> 00:49:57.639 is the ultimate outcome. And you would expect 00:49:57.639 --> 00:50:00.599 it to increase the realization of human rights 00:50:00.599 --> 00:50:04.760 to go beyond the CUSO project people. So the 00:50:04.760 --> 00:50:08.179 survey needs to be countrywide to claim that 00:50:08.179 --> 00:50:10.739 their ultimate outcome of increased realization 00:50:10.739 --> 00:50:16.139 by LGB. LGBTQIA plus people has been reported 00:50:16.139 --> 00:50:19.400 at the country level. So here's an example where 00:50:19.400 --> 00:50:24.300 I suspect, but it's not clear, but I'm assuming 00:50:24.300 --> 00:50:27.199 it is because they talk about ultimate outcome 00:50:27.199 --> 00:50:29.420 being at the country level because they list 00:50:29.420 --> 00:50:32.739 the countries so that they would need to go beyond 00:50:32.739 --> 00:50:37.440 just surveying the CUSO project people that received 00:50:37.440 --> 00:50:41.989 this advocacy or... the benefits of the advocacy 00:50:41.989 --> 00:50:46.389 project. So thank you for listening and we will 00:50:46.389 --> 00:50:51.250 send this off and invite CUSO to respond to my 00:50:51.250 --> 00:50:54.789 critique for these two projects and feel free 00:50:54.789 --> 00:50:58.929 to email the minister, the Secretary of State 00:50:58.929 --> 00:51:02.030 responsible for him to stop funding CUSO immediately 00:51:02.030 --> 00:51:06.309 until they start developing better outcomes and, 00:51:06.329 --> 00:51:08.329 for example, getting rid of this self -reporting 00:51:08.329 --> 00:51:11.320 bias. Thank you for listening. And stay tuned 00:51:11.320 --> 00:51:14.019 when I get another PMF or collection of PMS. 00:51:14.280 --> 00:51:16.780 We will do another episode. Bye for now.