Welcome to the Improving Development Evaluation
Podcast. I'm your host, David Wand, and in this
episode, we're going to feature the international
development organization CUSO International,
that's C -U -S -O International. You can learn
more about CUSO International at their website
at www .cusointernational .org. We used to affectionately
refer to CUSO back in the 1980s when I was in
Seattle. leone with cuso as the canadian unemployed
socialists overseas and the david haslett wonderful
man who was the field staff officer for sierra
leone back then he was the one who introduced
me to that acronym but of course we've made some
other ones that are quite funny but we'll stop
right there and get on with the episode we're
going to be looking at two cuso projects one
called blueprints at a cost to the taxpayer of
$2 .5 million. And another project called Building
Bricks, at a cost of $5 million to the Canadian
taxpayer, for a total of $7 .5 million. And Blueprints
has nine outcomes, and we're going to be looking
at the performance measurement framework and
seeing how well the 13 outcome indicators for
those nine outcomes properly validly measure
the achievement of those outcomes and for building
bricks we're looking at eight outcomes for that
project and a total of 15 outcome indicators
designed to measure whether or not those eight
outcomes have indeed been achieved. Before we
get into it I just wanted to give you some background.
There's been some significant delays in producing
these performance measurement frameworks. Rebel
Media, they're not very well respected in Canada
but they've reported in their opinion, that some
of these access to information requests like
mine for PMFs are intentionally being delayed
because, of course, that information that they
release makes the government look bad. Now, I
can only speculate as to why there have been
delays in me receiving my performance measurement
frameworks. These performance measurement frameworks
need to be produced before the government of
Canada can disperse funds to CUSO International,
for example. I requested these two PMFs for these
two projects back in October of 2024. It's already
been over a year since I received them. So I
suspect there's a bit of delay going on. And
I've also requested two other international development
organizations and their PMFs. And one of those
organizations, the PMFs I'm asking for... The
projects cost over $50 million, and I'm very
upset. It's been over a year, and they keep delaying
the release of these PMFs. There's no excuse
for it because they've already been produced.
They're only 10 to 15 pages long, and they have
to be produced by the International Development
Organization before they disperse the funds.
And as we speak, funds are already being dispersed
for these projects. because I can see it on the
Government of Canada Project Browser website,
and I'm still not getting the PMFs. So that's
a bit disturbing. If any of you want the PMFs
and my critique of the PMFs for each of these
episodes, you can email me at evaluatecanadaid
at gmail .com. And as you know... The link for
this episode is sent to the Secretary of State
for International Development, along with the
shadow critics at the Parliament in Canada. So
everybody knows, regardless of political affiliation,
that your tax dollars are not being properly
evaluated when it comes to foreign aid projects.
So you can also email in the episode. I'll provide
the emails to recommend. To the Secretary of
State, feel free to email them yourself with
the link for this episode telling them to defund
these international development organizations
until they start producing proper outcome indicators
for the outcomes that they keep claiming falsely
on their websites that they're achieving when
we don't even know that they're being achieved.
So that's important. So I'll also be producing
a petition to... at change .org that I'll be
putting in each of the episodes notes. So if
you want to sign a petition urging the Secretary
of State for International Development to indeed
urge them to defund these organizations until
they produce proper outcome indicators and also
ideally we want these performance measurement
frameworks as well as the data that goes with
them. to be produced on the Government of Canada
Project Browser website, as well as the International
Development Organization's websites. Because
currently, they produce a lot of data, but it's
useless. Garbage in, garbage out. And that's
the problem. We've discovered that the design
of these indicators in the first place is flawed.
And in some cases, as I've noted in other episodes,
they don't have any outcome indicators at all.
And they keep claiming that they're achieving
outcomes. But if you look at the reports they
send me, there's no outcome indicators at all.
Instead, they just talk about outputs, how much
services they've delivered, but no evaluation
of what happens after those services are delivered.
And like I said, if you believe in foreign aid
and you want it to continue to be supported without
somebody coming along and saying, that there's
no evaluation and therefore we're going to scrap
them, these foreign aid projects. It's better
if you believe in foreign aid to show that the
foreign aid is actually working. It's effective
and it has impact, which is what the development
assistance criteria is all about. But as you've
discovered, probably through my podcast and other
information, this is not happening. So it's a
direct threat to funding, for foreign aid projects.
So in many cases, it's quite simple to show that
foreign aid is working. But in some cases, it's
a little bit more difficult, but there should
at least be an attempt to measure and then evaluate.
But that's not going on. So I'll leave it at
that and let's get into it. So blueprints, nine
outcomes, 13 outcome indicators. Give you a brief
description. of what the project does. It's focused
on the following countries in the Caribbean,
Dominica, Grenada, St. Lucia, and St. Vincent.
And they have two target groups, public institutions
in those countries, as well as women's rights
organizations. And they focus on training these
public institutions and women's rights organizations
on gender equality. By accessing materials on
online platforms to address issues of gender
equality. And CUSO helps them organize meetings
to share best practices on addressing gender
equality issues amongst public institution representatives
and these women rights organization representatives
to increase awareness among these two target
groups on how to address gender equality issues.
So they also spend money, CUSO, this funding,
$100 ,000 to increase engagement between these
public institutions and the women's rights organization
that represent girls and women, which leads to
these public institutions being more effective
in addressing these gender equality issues. So,
for example, women and girls will show up to
attend these dialogue spaces between the girls
and women and the government, public institution.
This includes communication campaigns and communication
engagement sessions to increase access to public
services. So these girls and women will show
up, provide feedback through scorecards to government
reps. The government feedback committees will
be set up to receive these scorecards. And the
government staff in these public institutions
are trained in, quote, accountability practices.
They deliver information fairs on gender services
to women and girls. They're trained in these
public institution representatives on gender
data and how to incorporate these gender development
indicators into their departments. They're trained
on gender responsive operational plans, including
how to reduce barriers to access services for
girls and women. They're also trained on gender
responsive M &E, monitoring and evaluation and
service delivery. They're also trained the public
institutions on gender budgeting and how to adapt
gender budgeting tools. Workshops are delivered.
Gender equality CUSO volunteers are also deployed
to the Caribbean. They're also trained these
government departments on how to develop gender
responsive capacity development plans. They're
also trained on how to complete needs assessments
to identify technical assistance needs for gender
equality. So government staff are reporting increased
knowledge and skill to carry out these needs
assessments. So that gives you basically an idea
of the Blueprints project. Now moving on to the
Building Bricks CUSO project. It's going on in
also the Caribbean, Belize, Dominica, Grenada,
Guyana, Jamaica, St. Lucia, St. Vincent, Grenadines,
and Suriname. And their goal is to influence
legislative reform and increase access to services
for those from the LGBTQIA plus lesbian, gay,
bisexual, trans, gendered, queer community through
organizations that represent those communities.
What they often focus on is advocacy, training
on how to design and deliver communication strategies.
Also training on how to develop awareness campaigns,
advocacy agendas, and how to sensitize and train
service providers on the rights of the LGBTQIA
plus community. And also training these organizations
that represent the LGBTQIA plus communities in
resource mobilization and monitoring and evaluation
system design. Also training them on governance,
leadership, ability to complete needs assessments,
how to respond to the needs of these particular
communities for both the organizations that represent
them and the service providers. So the key target
groups there are the lesbian, gay, bisexual,
transgender, queer, IA plus organizations, as
well as the service providers themselves. And
the cost of that is building bricks is $5 million.
And we have eight outcomes for building bricks
and 15 outcome indicators we're going to look
at from the performance measurement framework
for those eight outcomes. So there you have a
broad description of the two projects. And before
we get into the PMF for both of them, to see
if they actually did properly measure and evaluate
the achievement of those outcomes for those projects,
we again will talk about whether you are living
in the Global South, where it's usually, but
not always, hot and humid, or you work for CUSO
International or for another international development
organization. And lo and behold, you may have
to go to the global self to actually monitor
and evaluate one of your projects. And so it's
hot and humid there. So you probably want to
check out some sun hats, sunglasses, and some
cool clothing. And if you do that, you'll see
in the episode, I have a link called https colon
backslash backslash mydeals .page backslash one
h. And if you click on that link, you will get
access to these three companies where you can
get sun hats at ericjavits .com, where you can
travel friendly and protectively from the sun
with these hats. You get a 20 % discount off
any product if you use my link, mydeals .page.
For some reason, if you can't, you always use
wand cool t -shirts, W -A -N -D -K -O -O -L t
-shirts. But the link should work. And you'll
get a 20 % discount. And I get a 20 % commission
from the total purchase you make. So you might
be wondering these sun hats at ericjavits .com.
How protective are they from the sun? You get
ultraviolet protection factor with a 50 plus
ultra protection. factor rating, blocking 98
% of UVA and UVB radiation. They're travel -friendly,
easy to pack, but retains its shape when you
visit your international development project
located in the Global South. You can also pick
up some sunglasses at akila .la, A -K -I -L -A
dot L -A. Includes prescription lenses, all made
from 50 % recycled material. You get a 10 % discount
if you use my link, mydeals .page backslash 1HJL
that you'll see in the episode notes. And I get
a 15 % commission when you purchase any of the
products using my promo code or that link at
mydeals. So you have your hat and your sunglasses
to protect you from the sun and the global south,
but you need the right clothes to be comfortable
in that hot. and humid climate. How about some
men's polo t -shirts or button -ups at hypernaturalstyle
.com. You get 20 % off any product when you use
my promo code JuanCoolT -shirts or the link.
And I get a 20 % commission of the total purchase
you make when you use my link at my deals. So
what is cool about these hypernatural t -shirts?
They have a patent -pending hyper -cooled jade
technology that, when it's hot, lowers your body
temperature by 3 to 5 degrees. They also are
breathable, thanks to Supina cotton. And they
are sold in premium retailers like Nordstrom.
And they were voted the best men's polo shirts
in 2024 in Men's Journal, Forbes, and Esquire.
So, that's basically it there. So let's start
with blueprints for gender equality at a cost
of $2 .5 million to the Canadian taxpayer. And
we'll take a look at some of the outcomes. Nine
outcomes, 13 outcome indicators. The first outcome
is improved capacity of public institutions.
in targeted Caribbean countries to share good
practices and lessons learned on addressing the
most pressing gender equality issues faced by
the poorest, most vulnerable women and girls
in all their diversity, including women's rights
organizations. The first outcome indicator for
that outcome is percent of ministries, departments,
and agencies that have improved their capacity
to coordinate with public institutions across
the Caribbean. to share good practices and lessons
learned, to address most pressing gender equality
issues faced by poorest, most vulnerable women
and girls, disaggregated by institution, sector,
and geographic location. But if you look at the
performance measurement framework, you'll find
there's a conflict of interest, as CUSO and the
government departments do a, quote, partnership
review. So there's a self -reporting bias. Instead,
what they should be doing is using an external
evaluator should be measuring the technical capacity
of the individuals inside of these public institutions
to see if their capacity to share best practices
on gender equality have actually been done. So
that's not a very good indicator. They can't
claim, once again, they've improved the capacity
of these public institutions. because they have
a conflict of interest. They need an external
consultant to come in and use an objective measure
instead. The next outcome they have is enhanced
capacity of public institutions to engage with
the poorest, most vulnerable women and girls
in all their diversity, including women's rights
organizations in targeted Caribbean countries.
The outcome indicator for that outcome is...
percent of ministries, departments, and agencies
that have increased their knowledge about barriers
faced by poorest, most vulnerable women and girls
to increase engagement with poorest, most vulnerable
women and girls, disaggregated by institution,
sector, and geographic location. Again, if you
look at the performance measurement framework,
who's doing this? It's a conflict of interest
as CUSO and the government departments do a quote
partnership review. Instead, an external evaluator
should be measuring the knowledge levels of the
individuals inside of these public institutions
to see if their knowledge levels of recalling
and identifying to increasing engagement with
vulnerable women has actually occurred. So again,
this is not a good outcome indicator to claim
that they're achieving, quote, enhanced capacity
of public institutions. Doesn't work. The next
outcome they've got here is increased knowledge
and skills of public institutions to effectively
deliver. gender responsive public programs and
services that address the most pressing gender
equality issues faced by the poorest, most vulnerable
women and girls in all their diversity in targeted
Caribbean countries. The indicator for that is
percent of ministries, departments and agencies
who have improved their performance in delivery
of open inclusive and gender responsive public
programs and services for poorest most vulnerable
women and girls disaggregated by institution
sector and geographic location yay in this one
they get a green light they actually use and
mention external consultant which is good And
there are data sources in the PMF, document review,
case studies, that suggest this can be done objectively
and free from the public institutions self -reporting
on their skill levels in delivering gender responsive
services, which is biased and not a proper valid
measure of this outcome. But in this case, they
use an external consultant. So it begs the question,
are they going to use an external consultant
again? for some of the other outcome indicators
that are coming up. Let's see. So, that's good.
The next indicator, same outcome, increased knowledge
and skills of public institutions, blah, blah,
blah. The next indicator they've used for this
same outcome is percent of ministries, departments,
and agencies that have adopted results -based
gender -responsive planning, monitoring and evaluation
procedures and practices. disaggregated by institution,
sector, and geographic location. Did they use
an external consultant again for this? Sounds
like they should. No, they didn't. What did they
do? They dropped the external consultant here
and instead are using CUSO, which is a conflict
of interest. They should use the same external
consultant to go in and check to see, maybe using
a checklist. If the public institutions have
indeed adopted monitoring and evaluation procedures.
This in turn can then be used to see if they're
able to achieve the outcome of skills to evaluate
whether they have effectively delivered gender
responsive services. Clear self -reporting bias
and strange that they will not use the same external
consultant. So that's not good. So far, only
one good indicator. Now, again, New outcome here,
enhanced gender responsive budgeting processes
of public institutions to address the most pressing
gender equality issues faced by the poorest,
most vulnerable women and girls in all their
diversity in targeted Caribbean countries. The
outcome indicator they use for this outcome is
percent of ministries, departments and agencies
with increased knowledge and skills on gender
and participatory. budgeting processes disaggregated
by geographic location and sector of ministries,
departments, and agencies. Great indicator, but
no external consultant. Instead, again, CUSU
is doing it, and even worse, they are asking
the individuals in these public institutions
to self -report on their knowledge levels in
the target column of the performance measurement
framework, and even ask them how quote satisfied
they are with those knowledge and skill levels
as noted in the data source of the performance
measurement framework where it shows a satisfaction
survey a clear again self -reporting bias instead
they need to get that external consultant to
develop some objective criteria of what one needs
to see to conclude that the public institution
does indeed have, quote, a gender responsive
budgeting process in place. So again, they've
done it once, an external consultant, but somehow
they don't want to do it for the other indicators.
Not good. The next outcome is increased knowledge
and technical skills of public institutions to
develop and implement gender responsive public
policies, programs, and services that address
the most pressing, Gender equality issues faced
by the poorest, most vulnerable women and girls
in all their diversity in targeted Caribbean
countries. Now for this, they have three indicators
for this one outcome. First indicator is average,
open, inclusive, gender responsive and accountable
score, public policies, programs and services.
Design implementation for poorest, most vulnerable
women and girls component. The other indicator
they have for the same outcome is average open,
inclusive, gender responsive and accountable
participatory needs assessment score, gender
equality component. And the third indicator for
this is an average open, inclusive, gender responsive
and accountable score for monitoring and evaluation
and learning component to collect gender disaggregated
data. So the three indicators for this results
in a, quote, needs assessment report completed
by, and quote, if you look at the PMF, joint
sessions between CUSO and the individuals within
these public institutions. Once again, they need
to get an external consultant to measure their
technical skill on the three areas that would
lead one to conclude. The skill on, quote, developing
and implementing gender responsive policies,
programs and services has indeed increased. Right
now, we have a conflict of interest where there's
no external consultant. And CUSO is doing this
internally with the public institutions that
they're trying to measure and evaluate. Not good.
And also a needs assessment report. doesn't seem
to be a valid measure of increased skill on,
quote, developing and implementing gender responsive
policies, programs, and services. So that's the
problem there. Moving along to intermediate outcomes,
we have enhanced gender responsive public policies,
programs, and services to address the most pressing
gender equality issues faced by the poorest,
most vulnerable women and girls in all their
diversity. in targeted Caribbean countries and
the outcome indicator for that is average ministries
departments and agencies open inclusive and gender
responsive participatory needs assessment score
so in the performance measurement framework the
data collection methods are between CUSO and
the public institutions producing a needs assessment
report this has nothing to do with measuring
whether the services they delivered were gender
responsive. You need to ask the women and girls
who received the services from these public institutions
if the services were gender responsive. So again,
a lousy indicator that doesn't measure at all
enhanced gender responsive services. The next
outcome is increased effective delivery of gender
responsive services. that address the most pressing
gender equality issues faced by the poorest,
most vulnerable women and girls in all their
diversity in targeted Caribbean countries. The
outcome indicator from the PMF is percent of
poorest, most vulnerable women and girls with
effective access to open, inclusive, gender responsive
and accountable public programs and services
in four countries. Disaggregated by socioeconomic
status, gender. age, ability, ethnicity, language,
geographic location, and type of service. Sounds
great. Problem is when you look at the PMF, the
data sources are CUSO blueprint reports. Doesn't
the access come from the locations where these
women and girls show up to receive the services?
So I guess we are relying on these blueprint
reports that contain the percent of women and
girls that receive the gender responsive services.
To show that this access has increased, a better
word would be maybe effective, we need to see
the measuring instrument as well as the criterion
used to conclude that the services delivered
indeed meet the standard of being described as
being gender responsive. Again, they should use
an external consultant reviewing the records
where the service was delivered rather than using
a CUSO blueprint report because that would have
no self -reporting bias so that's the problem
there is again they're relying on internal QSO
reports when if you really want to see if these
girls and women have had effective access to
gender responsive services you would want to
have the external consultant go in and review
the reports That would be a better approach.
The next outcome is actually for the same indicator,
excuse me, for the same outcome, increased effective
delivery of gender responsive services, was level
of satisfaction of poorest, most vulnerable women
and girls in all their diversity with open, inclusive,
gender responsive and accountable public programs
and services in four countries. This is great.
Great indicator. They're actually going to go
to the women and girls and ask them. Are you
satisfied with the services? And are they indeed,
in your opinion, gender responsive? We'll give
you a definition of what we mean by that. Fantastic.
Problem again is that CUSO and the public institutions,
not an external consultant, are responsible for
asking these women and girls how satisfied are
they with the services they receive from these
public institutions. Now, it's funny. They used
an external consultant before, as I mentioned.
So why not for this very good indicator? A good
indicator ruined by CUSO public institution conflict
of interest. Get an external consultant to complete
the satisfaction survey and produce the report
that claims, and I quote, increased effective
delivery of gender responsive services. The current
blueprint reports as the data source in the PMF
are potentially biased. The next outcome statement
improved collaboration and dialogue across public
institutions in targeted Caribbean countries
on the most pressing gender equality issues faced
by the poorest, most vulnerable women and girls
in all their diversity. The outcome indicator
for that outcome? Percent of staff who recognize
public institutions as key actors in sharing
knowledge to address most pressing gender equality
issues. faced by poorest, most vulnerable women
and girls. So they've got increased recognition.
Recognize. So increased recognition, knowledge.
It does not equate with, quote, the outcome,
improved collaboration and dialogue, which is
a behavior, right? They need a better indicator
for increased collaboration and dialogue. such
as either frequency of meetings and communications
specific to actions to address gender equality.
Note, this is still process rather than results,
where process is not measuring any changes in
increased gender equality. So that's an example
where they've got an indicator, but it's not
related to measuring the outcome of improved
collaboration and dialogue. Then finally, the
ultimate outcome in this project is enhanced
gender responsive governance in Dominica, Grenada,
St. Lucian and St. Vincent and the Grenadines
for the poorest, most vulnerable women and girls
in all their diversity. The outcome indicator
for this ultimate outcome is number of poorest,
most vulnerable women and girls reached. by open,
inclusive, and gender -responsive public programs
and services in four countries. Now, this is
a bit of a problem because how is this any different
than the indicator for 1200, which was percent
of women and girls with effective access to gender
-responsive services? It looks like they're going
countrywide here. It would be better to actually
use a measure of gender equality that is an expected
outcome if there is an increase in gender -responsive
services delivered. Where more women and girls
get those gender responsive services because
they've already measured that in some of the
other outcome indicators that I just mentioned.
So the next level would be at the country level
is governance regarding gender responsiveness
and gender equality being achieved for women
outside of the project, right? So an example
could be the UNDP. Gender inequality index, where
they look at maternal mortality, adolescent birth
rate, females that have secondary education,
the female share of parliamentary seats, the
female labor force participation rate. Now, although
attributing the project to those changes in the
gender inequality index would be difficult, at
least it should be compared with the project
measure. of increased access to gender responsive
services. Because I think in the outcome, the
ultimate outcome, they're implying we're going
to go to the country level outside of the project.
In summary, for this project, you've really got
13 outcome indicators, but all of them are bad
except one. There's only one really good indicator.
There's just too much self -reporting bias going
on. And in some cases, the indicator is not even
related to the achievement of the outcome. So
that's the big problem with this performance
measurement framework. So I could, again, would
conclude that they're not achieving any of their
outcomes because again, they're not properly
measuring them. So they can't make the claim
that their outcomes are being achieved. So let's
look at Building Brooks for $5 million cost to
the Canadian taxpayer. They've got eight outcomes
and For those outcomes, they've got 15 outcome
indicators. So if we start from the bottom with
the immediate outcomes that they expect to achieve,
we've got enhanced coordination and collaboration
action on rights -based agendas among local and
regional LGBTQIA plus organizations and networks
in the Caribbean. The outcome indicator for that
was number of organizations and networks coordinating
human rights and gender equality agendas. Now,
you're probably suspecting already, it's not
directly related to the outcome. How does increase
in the number of organizations equate with, quote,
enhanced coordination? Even if the increase in
number of organization happens, was it due to
the project output? Or could they increase that
number by themselves without you spending $5
million? The project creates, quote, dialogue
spaces that are then used to do this coordination.
But could this happen without the creation of
these dialogue spaces? And if so, would this
increase in the number of organizations doing
this coordination, would it be much greater than
without these project dialogue spaces? So this
is a case where... Yes, they could maybe claim
increased coordination and enhanced coordination,
but maybe it would happen anyways without the
project. So this is where they're not really
looking at impact. So it's a little questionable,
right? So the next outcome is, again, enhanced
coordination. And they have another indicator
for it. And they have number of organizations
and networks reporting increased ability. to
participate in legislative reform in the Caribbean.
Again, self -reporting bias, right? If you want
to see if this project has actually increased
their ability, their skills and knowledge to
engage in trying to get legislative reform, then
don't ask them themselves, hey, how you feel?
Don't. That's self -reporting bias. You need
to get an external consultant to objectively
measure. whether the individuals within these
organizations have indeed increased their ability
to participate in legislative reform. So you've
got self -reporting bias. The next outcome statement,
increased capacity of local and regional organizations
and networks to advocate and campaign for LGBTQIA
plus rights and gender equality using an intersectional
approach. The outcome indicator here is number
of individuals reached through national and regional
campaigns. The survey that they use here in the
PMF on reach is excellent. So it's very good.
That indicator is good. The problem is the other
indicators that they use for the same outcome.
Number of large -scale national regional campaigns
designed and launched. So that's good too. That's
the second indicator that's good because as the
number of launches goes up, that shows they've
increased their capacity to advocate and campaign
for LGBTQIA plus rights. So these are two indicators
that are actually quite good. So that's well
noted. But the third indicator, once again, is
biased. And that is... Number of these organizations
that report increased ability to advocate. I
would just take it out. This is self -reporting
bias. Getting the number launched is good. Getting
the number that have been reached is good. And
that by themselves, those two indicators are
good to show that they've actually increased
their capacity to advocate and campaign for those
rights. That's a good example. So not bad there.
Next outcome, increased access by LGBTQIA plus
people to safe, inclusive, environmentally friendly,
and gender responsive socioeconomic services
in targeted countries. The problem here again
is the outcome indicator is the number of organizations
and networks that report increased ability to
coordinate with and strengthen the ability of
service providers to deliver quality services.
Self -reporting, again, it is biased. So this
should be taken out. But good news, if we look
at the next indicator for increased access, they
actually go to the people in those communities,
LGBTQIA +, and ask them directly the number that
have reported increased access to services. Very
good indicator. of the outcome if we mean improved
or adequate quality in achieving gender responsive
services is equivalent to increased access we
have to be careful here just as the number goes
up that are reporting they got the right services
we should also make sure that the quality is
adequate too but we're going to assume that for
now and it is in the pmf The surveys seem to
be doing that. So that's another measure. That's
three good indicators now. So that's good. But
again, we get back to problem of self -reporting
with this next outcome. Increase knowledge and
skills of local and regional organization and
networks in governance, leadership, and management
to represent LGBTQ. plus rights in targeted countries.
So as you remember from this project, there's
lots of training going on for these individuals.
So what they have to do is stop asking them how
they feel. Stop asking them to self -report on
whether or not their skills have been increased.
It's getting ridiculous. It just doesn't look
good. What they need to do is get an external
measure of those skills in leadership and management.
Get a consultant to come in. And say, sorry,
I'm going to measure whether or not you've learned
what you're supposed to learn. I'm going to test
you. Very easy to do. Self -reporting, bias,
not good. And they also do it again for another
indicator on human resources governance and monitoring
and evaluation. Self -reporting, where the indicator
is percent of staff of organizations. Reporting
increased knowledge and abilities in human resources
governance and monitoring and evaluation. Again,
they're using self -reporting. They need to use
an external measure of increased knowledge of
human resources governance and monitoring and
evaluation. Next indicator after these immediate
incomes is the intermediate outcomes. Here's
the outcome statement. Increased effectiveness
of local and regional organizations and networks
to coordinate the LGBTQ. IA plus rights movement
in the Caribbean. The outcome indicator is number
of organizations and networks, again, reporting
increased effectiveness to implement mechanisms
that positively influence gender sensitive policies.
Again, a bias. Don't ask them. Self -reporting.
Get an external measure of increased effectiveness
to implement mechanisms that influence gender
sensitive policies instead. This could complement
the next measure on effectiveness to coordinate.
They should also add what exactly would a gender
-sensitive policy look like in terms of legislation.
So that could be aimed for. Maybe that comes
later. So another indicator for increased effectiveness
is number of policies, laws, and frameworks presented
for legislative reform by... organizations, and
networks. This is a great measure of increased
effectiveness to coordinate. But be aware, just
because they've increased the number of policies
presented, they still could be garbage, right?
You need someone to look at the quality of these
policies, laws, and frameworks. Is it sufficient
as reviewed by external experts or even by the
government agencies that have received this advocacy
document? or recommended policy. It doesn't matter.
It could be an external expert. It could be the
civil servant that's received the document and
says, this is crap. It doesn't even include any
data to support the change in legislation and
the need for this change. So not just the number
presented, but at least they can claim that they're
doing it. But increased effectiveness, you need
to look at the quality and even better. to see
if it made a difference indeed in any changes
in legislation. The next indicator is enhanced
performance of local and regional organizations
and networks, gender responsive programming and
advocacy for rights of LGBTQIA plus people and
gender equality in targeted countries in the
Caribbean. The outcome indicator there, again,
Self -reporting bias, number of organizations
and networks that report increased ability to
advocate for the rights of LGBTQIA plus people.
So again, a problem with self -reporting. Instead,
they need an external measure, maybe an external
consultant of enhanced performance instead. If
their performance was convincing and enhanced,
you would expect a press release or even better.
legislation indicating a change, reflecting the
changes that were advocated for in that, quote,
enhanced performance. Another indicator for enhanced
performance is number of organizations and networks
with increased delivery of programming. So they're
asking for increases in programming in this advocacy.
That's what they want, right? Increased delivery
of programming. So why not just look at that?
Forget about the number of organizations that
are doing it. Under the target column, they have
the number of organizations that have improved
their delivery of services. But they want more
services also, not just improved services. Instead,
list the new programs and what they were delivering
and the number of people receiving the program
by type of gender identity to see if it has increased.
due to this project's training on advocacy and
lobbying for increased access to services. Have
the number of people receiving services increased
is what should be measured here as the expected
outcome. So here the indicator is not relevant
to measuring the outcome of enhanced performance
to advocate for rights, right? You want to go
to the end and see. If there's been an increase
in the number of services delivered and also
perhaps also legislative reform, because usually
they go hand in hand. Another outcome here is
improved institutional capacity and sustainability
of local and regional organizations and networks
in targeted countries in the Caribbean. The outcome
indicator for that outcome, number of organizations
using sustainability, best practices. Now, when
you read that, that's pretty good. But again,
if you don't have the performance measurement
framework in front of you, there's a little trick
they're playing here. The outcome and its indicator
that I just stated are very good, very clear.
But if you look at the baseline data column in
the performance measurement framework, it's not
related to the indicator. Instead of going into
an organization to see if they're actually, quote,
using sustainability best practices. They are
just checking to see if the organization has
a, quote, sustainability plan. That is not actually
a practice or a behavior. It's just a piece of
paper, a plan. Are they actually, you know, maybe
in the budget? And for example, I'll describe.
The target column only counts how many organizations
were trained in sustainability. That is not relevant
to the outcome statement. The target column does
also have a measure of the number of organizations,
quote, implementing best practices. Let's hope
there is a manual on the criteria on how to decide
that an organization is indeed implementing sustainability.
What is that sustainability? Well, one example,
a reduction in foreign funds and an increase
in developing country funds to become less dependent.
on foreign aid, i .e. more sustainable. So what
you could do is actually show as a better indicator
the budget of the organization and seeing a reduction
in the foreign funding from CUSO or anybody else
and an increase in the local fundraising where
they've got money either from the private sector
or and or from the government, their local government,
right? So that's an example where the indicator
Sounds good, but when you look in the PMF, the
data that they're actually collecting is not
related to measuring the achievement of the outcome
or even practicing the outcome indicator. The
next outcome is increased realization of human
rights by LGBTQIA plus people and advancement
of gender equality in. Belize, Dominica, Grenada,
Guyana, Jamaica, St. Louis, St. Vincent, and
the Grenadines, and Suriname in the Caribbean.
The outcome indicator is number of LGBTQIA people
who have realized their human rights. Now, if
you look at the data source in the performance
measurement framework, it indicates, quote, project
reports along with a baseline total of 847. Now,
this suggests only those in the CUSO project
are being surveyed. So there is a bias. This
is the ultimate outcome. And you would expect
it to increase the realization of human rights
to go beyond the CUSO project people. So the
survey needs to be countrywide to claim that
their ultimate outcome of increased realization
by LGB. LGBTQIA plus people has been reported
at the country level. So here's an example where
I suspect, but it's not clear, but I'm assuming
it is because they talk about ultimate outcome
being at the country level because they list
the countries so that they would need to go beyond
just surveying the CUSO project people that received
this advocacy or... the benefits of the advocacy
project. So thank you for listening and we will
send this off and invite CUSO to respond to my
critique for these two projects and feel free
to email the minister, the Secretary of State
responsible for him to stop funding CUSO immediately
until they start developing better outcomes and,
for example, getting rid of this self -reporting
bias. Thank you for listening. And stay tuned
when I get another PMF or collection of PMS.
We will do another episode. Bye for now.