1
00:00:00,000 --> 00:00:06,380
Welcome to the Improving Development Evaluation Podcast. I'm your host David

2
00:00:06,380 --> 00:00:12,440
Wand and welcome to season two, episode three. It's been quite a bit of time

3
00:00:12,440 --> 00:00:17,720
since my previous episode and this is because I've been waiting several months

4
00:00:17,720 --> 00:00:23,000
from the United States Agency for International Development as well as the

5
00:00:23,000 --> 00:00:29,400
United States State Department to both confirm in writing that an activity

6
00:00:29,400 --> 00:00:36,800
monitoring evaluation learning plan or its equivalent has not been found for

7
00:00:36,800 --> 00:00:41,200
the organization I'm going to talk about today that received five million dollars

8
00:00:41,200 --> 00:00:46,000
from the United States Agency for International Development. So I'm going

9
00:00:46,000 --> 00:00:51,480
to give you some background on that and what I'm doing next. So we're going to

10
00:00:51,480 --> 00:01:00,160
talk about the Global Community Engagement Resilience Fund, GCERF, G-SERF

11
00:01:00,160 --> 00:01:05,840
and you can learn more about this organization at their website gsERF.org.

12
00:01:05,840 --> 00:01:11,960
They're an organization that claims to be preventing violent extremism as they

13
00:01:11,960 --> 00:01:17,760
get funds from various donor countries. They then turn around as a fund and then

14
00:01:17,760 --> 00:01:23,480
give those funds in terms of creating grants they call them or projects to

15
00:01:23,480 --> 00:01:28,640
their implementing partners that are located in developing countries. If you

16
00:01:28,640 --> 00:01:34,680
go to their website at gsERF.org there's some very good information that they

17
00:01:34,680 --> 00:01:39,000
provide. The first thing they do is according to their strategy documents

18
00:01:39,000 --> 00:01:46,360
they very clearly state that if you give them money they will achieve one, two,

19
00:01:46,360 --> 00:01:52,280
three or four of the following expected outcomes because they argue that if

20
00:01:52,280 --> 00:01:58,680
these outcomes are achieved this in turn will reduce or prevent violent extremism.

21
00:01:58,680 --> 00:02:05,800
Those four expected outcomes are one increased social cohesion, two increased

22
00:02:05,800 --> 00:02:11,160
ability to mobilize, organize, represent their own interests which they call

23
00:02:11,160 --> 00:02:17,840
agency, expected outcome number three increased knowledge skills as well as

24
00:02:17,840 --> 00:02:22,320
subsequent increases in income or access to livelihoods and expected

25
00:02:22,320 --> 00:02:29,560
outcome number four increased confidence, critical thinking, life skills, self-worth,

26
00:02:29,560 --> 00:02:36,280
resilience which they call sense of purpose. So it's very clear what they

27
00:02:36,280 --> 00:02:41,840
expect to achieve if they get these funds which they then turn around and

28
00:02:41,840 --> 00:02:45,560
give to implementing partners in developing countries in the form of

29
00:02:45,560 --> 00:02:50,560
grants or projects. The other thing that's important to note on their website

30
00:02:50,560 --> 00:02:59,400
at gsERF.org is they make the claim that they will be responsible for properly

31
00:02:59,400 --> 00:03:05,320
evaluating whether or not these expected outcomes have been achieved which is

32
00:03:05,320 --> 00:03:10,920
good and I quote, at the grant level each project is guided by a theory of change

33
00:03:10,920 --> 00:03:17,640
and a results framework that would be the equivalent of an activity MEL plan

34
00:03:17,640 --> 00:03:24,520
which is what USAID uses with outcome and output indicators requiring grantees to

35
00:03:24,520 --> 00:03:31,160
collect baseline data, report quarterly and conduct final assessments. So given

36
00:03:31,160 --> 00:03:37,640
this information I sent a request, a freedom of information request to USAID

37
00:03:37,640 --> 00:03:47,640
in April of 2024 requesting all results frameworks from gsERF and this is more

38
00:03:47,640 --> 00:03:53,320
importantly since they're a fund the results frameworks from the implementing

39
00:03:53,320 --> 00:03:57,920
partners that they work with because I don't want to see the results framework

40
00:03:57,920 --> 00:04:03,160
of gsERF for the five million dollars that I found from their annual report

41
00:04:03,160 --> 00:04:08,200
that they received in 2023 because it could be just project management that

42
00:04:08,200 --> 00:04:14,040
they sent funds or grants out to ABCD and E international development

43
00:04:14,040 --> 00:04:18,280
organizations located in developing countries. No, I need to see the results

44
00:04:18,280 --> 00:04:22,720
frameworks like they claim on their website for each grant that's been

45
00:04:22,720 --> 00:04:30,800
created from that five million dollars. So I asked in April of 2024 USAID to

46
00:04:30,800 --> 00:04:35,600
provide all of the results frameworks for all of the grants that were created

47
00:04:35,600 --> 00:04:40,720
out of this five million dollars that gsERF received and in September I

48
00:04:40,720 --> 00:04:47,360
received a letter from USAID saying they could not find any record of an activity

49
00:04:47,360 --> 00:04:55,360
mail plan or its equivalent for 2022 and 2023. Then luckily on November 11th I

50
00:04:55,360 --> 00:05:00,560
emailed USAID with the project details of the five million dollars because I

51
00:05:00,560 --> 00:05:07,200
managed to locate the exact description of that five million dollars that USAID

52
00:05:07,200 --> 00:05:14,840
sent to gsERF and it is the following activity ID number 77266 project number

53
00:05:14,840 --> 00:05:25,480
SCTOOP21VC004. The funding agency is clearly stated as USAID. The managing

54
00:05:25,480 --> 00:05:29,320
agency is clearly stated as the Department of State. The dollar amount

55
00:05:29,320 --> 00:05:35,800
was five million dollars USD. The activity end date was September 15th 2023.

56
00:05:35,800 --> 00:05:40,600
The international sector was government and civil society. The United States

57
00:05:40,600 --> 00:05:45,000
government sector was counter-terrorism. So there was no confusion about this

58
00:05:45,000 --> 00:05:50,920
money and I gave it to USAID and asked them again by email given that

59
00:05:50,920 --> 00:05:55,960
information do you have any activity mail plans that come out of this five

60
00:05:55,960 --> 00:06:03,480
million dollars and they replied sorry we don't but they suggested that I send

61
00:06:03,480 --> 00:06:08,360
another freedom of information request to the US Department of State because

62
00:06:08,360 --> 00:06:15,640
quote they are the managing agency for that award. Fine no problem. So I did on

63
00:06:15,640 --> 00:06:20,720
December 4th 2024 I sent a freedom of information request to the United States

64
00:06:20,720 --> 00:06:25,480
State Department with the same detailed information for the five million dollars

65
00:06:25,480 --> 00:06:33,880
project number activity ID amount etc and then on January 23rd 2025 I received

66
00:06:33,880 --> 00:06:38,120
another letter from the United States Department of State that states it has

67
00:06:38,120 --> 00:06:43,000
no records of an activity mail plan or its equivalent for the five million

68
00:06:43,000 --> 00:06:50,840
dollars USD. And they recommended in that letter that I contact the US

69
00:06:50,840 --> 00:06:55,400
Agency for International Development but as I mentioned I'd already done that and

70
00:06:55,400 --> 00:07:02,200
actually by email had indicated in detail the project number the activity ID and

71
00:07:02,200 --> 00:07:09,320
everything. So with that result I then decided on February 6th to send a hard

72
00:07:09,320 --> 00:07:16,240
copy letter to the chairman Brian Mast of the Foreign Affairs Committee along

73
00:07:16,240 --> 00:07:21,400
with the copies of those two letters from the USAID and US State Department at

74
00:07:21,400 --> 00:07:28,360
raising concerns that there were no outcome indicators for these four

75
00:07:28,360 --> 00:07:33,440
expected outcomes that GSERF keeps claiming that they are achieving. There's

76
00:07:33,440 --> 00:07:36,840
nothing wrong with claiming you're going to achieve these outcomes but as you

77
00:07:36,840 --> 00:07:41,000
know from my podcast you either put up or shut up so if you're going to make

78
00:07:41,000 --> 00:07:46,840
these claims you need to provide the measurement on those outcomes. Increased

79
00:07:46,840 --> 00:07:54,160
social cohesion for example for 40 bucks you can buy a software called University

80
00:07:54,160 --> 00:08:02,520
of California Irvine Net, UC net and for 40 bucks you can show increased

81
00:08:02,520 --> 00:08:07,720
social cohesion between two groups. That's just one idea but we have no

82
00:08:07,720 --> 00:08:12,240
idea what measurement on social cohesion they're doing at all and they I do

83
00:08:12,240 --> 00:08:20,400
recall them having a project an event sporting event between madrasa students

84
00:08:20,400 --> 00:08:26,640
having a soccer match against non madrasa non-religious students to create

85
00:08:26,640 --> 00:08:31,240
increased social cohesion and trust between these two groups. That's just a

86
00:08:31,240 --> 00:08:35,880
one example where you could measure using this software before the match

87
00:08:35,880 --> 00:08:41,920
starts although it's not very scientific you could show that levels of density

88
00:08:41,920 --> 00:08:46,200
and reciprocity between the two groups is low they hold the soccer match and

89
00:08:46,200 --> 00:08:51,480
lo and behold the levels of density and reciprocity between the two groups that's

90
00:08:51,480 --> 00:08:57,040
the madrasa religious students on one team and the non madrasa non-religious

91
00:08:57,040 --> 00:09:00,880
public school students on the other soccer team playing that it's gone up

92
00:09:00,880 --> 00:09:07,240
after the match. That's just one example of please attempt to measure the

93
00:09:07,240 --> 00:09:11,960
expected outcomes using outcome indicators that you're claiming you're

94
00:09:11,960 --> 00:09:16,880
achieving these outcomes. That's all we're asking for. So what I did was I sent a

95
00:09:16,880 --> 00:09:22,880
letter to the chairman Brian Mast indicating that these results frameworks

96
00:09:22,880 --> 00:09:27,680
that G-SERF is claiming that they are developing in collaboration with their

97
00:09:27,680 --> 00:09:33,280
implementing partners in developing countries that Brian Mast demand that

98
00:09:33,280 --> 00:09:40,360
they put these results frameworks for this USAID five million dollars that G-SERF

99
00:09:40,360 --> 00:09:45,520
received on their website so we can actually see if they achieve their

100
00:09:45,520 --> 00:09:49,840
expected outcomes. That's all we're asking about. We're not talking about

101
00:09:49,840 --> 00:09:53,280
disputing whether or not they should get the money in the first place whether

102
00:09:53,280 --> 00:09:59,280
it's charity whether it's meets American interests all we want to do is have them

103
00:09:59,280 --> 00:10:07,000
put up or shut up. So that's the latest stage we are at right now and it I had

104
00:10:07,000 --> 00:10:11,080
to send them a hard copy because most of these US government departments don't

105
00:10:11,080 --> 00:10:15,320
have emails where you can send attachments. So I've sent a hard copy of

106
00:10:15,320 --> 00:10:19,320
the letter to Brian Mast who's the chair of the Foreign Relations Committee

107
00:10:19,320 --> 00:10:26,040
because they actually have a mailing address with the letter and the two

108
00:10:26,040 --> 00:10:30,920
copies of the letters from USAID and US State Department saying they can't find

109
00:10:30,920 --> 00:10:37,200
any results frameworks with this reflecting this five million dollars and

110
00:10:37,200 --> 00:10:43,480
I've also sent electronic version of this letter to the US State Department

111
00:10:43,480 --> 00:10:48,600
their contact form where you can't include attachments and we'll see what

112
00:10:48,600 --> 00:10:56,240
happens but I very much doubt given the volume of projects they're reviewing

113
00:10:56,240 --> 00:11:00,560
right now that I'll get a response but at least it's on file and it's been

114
00:11:00,560 --> 00:11:07,400
documented. I will also send a copy of the letter to Brian Mast along with the

115
00:11:07,400 --> 00:11:14,080
USAID letter and the US State Department letter to GSERF but I don't expect them

116
00:11:14,080 --> 00:11:22,360
to reply or even respond but maybe maybe they will begin to put results

117
00:11:22,360 --> 00:11:26,800
frameworks and the data that actually shows the outcome indicators and the

118
00:11:26,800 --> 00:11:31,880
fact that they're achieving their indicators their targets such as

119
00:11:31,880 --> 00:11:40,040
increased social cohesion has gone up from the project or increased skill

120
00:11:40,040 --> 00:11:46,120
levels for women that they teach how to sew and subsequently increased incomes

121
00:11:46,120 --> 00:11:51,880
have gone up that's you know the the second outcome or maybe hopefully

122
00:11:51,880 --> 00:11:57,440
they've shown increased ability to mobilize or organize because they train

123
00:11:57,440 --> 00:12:02,560
these individuals how to lobby their own local governments so maybe they have a

124
00:12:02,560 --> 00:12:09,880
measure of increased knowledge and skill and finally expected outcome number four

125
00:12:09,880 --> 00:12:13,840
maybe they have some measures of increased confidence critical thinking

126
00:12:13,840 --> 00:12:21,760
life skills self-worth resilience these are all old very familiar constructs and

127
00:12:21,760 --> 00:12:27,080
social psychology that they can easily measure at least attempt to measure and

128
00:12:27,080 --> 00:12:34,960
off they go but we don't see anything so far so let's hope that they move in that

129
00:12:34,960 --> 00:12:39,600
direction like all of the other organizations on my podcast where they

130
00:12:39,600 --> 00:12:45,120
have not moved in that direction and we'll see if I get any response I will

131
00:12:45,120 --> 00:12:51,160
update you in an additional episode but right now the American people have no

132
00:12:51,160 --> 00:12:56,680
idea if G surf is actually achieving these expected outcomes that they claim

133
00:12:56,680 --> 00:13:03,360
they are achieving on their website so without these results frameworks they

134
00:13:03,360 --> 00:13:08,200
cannot make this claim that they're achieving these expected outcomes until

135
00:13:08,200 --> 00:13:12,460
we get a look at those results frameworks and the outcome indicators to

136
00:13:12,460 --> 00:13:17,200
see actually if they're properly validly measuring the expected outcome shouldn't

137
00:13:17,200 --> 00:13:22,080
be difficult they can hire a social psychologist to design a psychometric

138
00:13:22,080 --> 00:13:29,120
tool very easily to be very precise for the location and the target group that

139
00:13:29,120 --> 00:13:32,520
they are targeting depending on the developing country so there's really no

140
00:13:32,520 --> 00:13:37,560
excuse so they either have to put up and show they're achieving those outcomes or

141
00:13:37,560 --> 00:13:43,480
just go down the other pathway and say look we can't prevent violent extremism

142
00:13:43,480 --> 00:13:47,200
because we don't have the evidence that shows we're achieving any of these four

143
00:13:47,200 --> 00:13:52,520
expected outcomes we're just a charity and we're delivering these services

144
00:13:52,520 --> 00:13:56,800
because the people need them that's fine too but this middle ground is

145
00:13:56,800 --> 00:14:01,280
unacceptable where they're claiming they're achieving these expected

146
00:14:01,280 --> 00:14:06,000
outcomes and then going further and making the stretch that they're even

147
00:14:06,000 --> 00:14:11,240
preventing violent extremism when we don't even know if the expected outcomes

148
00:14:11,240 --> 00:14:15,400
that they claim you first have to achieve to then reduce or prevent

149
00:14:15,400 --> 00:14:20,080
violent extremism we don't even know if they've been achieved so it's a bit of a

150
00:14:20,080 --> 00:14:25,120
stretch as they say so thanks for listening and I'll give you an update if

151
00:14:25,120 --> 00:14:30,200
there is one if there's a response from the Foreign Relations Committee and or

152
00:14:30,200 --> 00:14:35,320
G surf on whether or not we're going to see these results frameworks otherwise

153
00:14:35,320 --> 00:14:41,560
the American people are getting ripped off