1
00:00:00,000 --> 00:00:04,960
In this episode we're going to introduce you to an international development organization.

2
00:00:04,960 --> 00:00:11,220
That international development organization is the United Nations Population Fund.

3
00:00:11,220 --> 00:00:15,440
You can learn more about the United Nations Population Fund if you go to their website

4
00:00:15,440 --> 00:00:20,960
at www.unfpa.org.

5
00:00:20,960 --> 00:00:24,580
Their headquarters is located in New York, USA.

6
00:00:24,580 --> 00:00:29,420
The project we're going to be talking about, that the UNFPA has been funded by the Government

7
00:00:29,420 --> 00:00:36,720
of Canada to deliver, is called Human Resources for Sexual Reproductive Health Rights and

8
00:00:36,720 --> 00:00:38,720
Gender-Based Violence.

9
00:00:38,720 --> 00:00:44,100
This project is being delivered in South Sudan and the cost of this project to the Canadian

10
00:00:44,100 --> 00:00:48,880
taxpayer is $37 million.

11
00:00:48,880 --> 00:00:53,040
And in this part one I'm going to give you a brief description of the services delivered

12
00:00:53,040 --> 00:00:59,860
in this project and a critique of the United Nations Population Fund, South Sudan's Performance

13
00:00:59,860 --> 00:01:01,600
Measurement Framework.

14
00:01:01,600 --> 00:01:07,760
The UNFPA have developed this performance measurement framework to measure the project's

15
00:01:07,760 --> 00:01:13,120
performance using the outcome indicators in that performance measurement framework.

16
00:01:13,120 --> 00:01:19,160
I obtained this performance measurement framework through an access to information request.

17
00:01:19,160 --> 00:01:26,400
I also obtained the United Nations Population Fund Evaluation Plan for this project.

18
00:01:26,400 --> 00:01:33,360
However, this evaluation plan provides no additional detail as to how the outcome indicators

19
00:01:33,360 --> 00:01:38,000
will be used to measure the project outcomes for this project.

20
00:01:38,000 --> 00:01:43,920
Therefore I will not be discussing the evaluation plan as part of this episode.

21
00:01:43,920 --> 00:01:49,000
But before I describe the project services that are being delivered for this project

22
00:01:49,000 --> 00:01:55,260
by the UNFPA in South Sudan as well as the expected outcomes to be achieved from the

23
00:01:55,260 --> 00:02:01,400
delivery of those services, I'm going to go to my summary and conclude that in this

24
00:02:01,400 --> 00:02:07,840
performance measurement framework for this project there are 16 outcome indicators that

25
00:02:07,840 --> 00:02:12,520
are measuring 7 outcomes for this project.

26
00:02:12,520 --> 00:02:18,920
My review of all 16 outcome indicators has led me to conclude that all of these indicators,

27
00:02:18,920 --> 00:02:23,800
except one, are not properly measuring the outcomes.

28
00:02:23,800 --> 00:02:32,080
So, the organization, the United Nations Population Fund, South Sudan, country office, cannot make

29
00:02:32,080 --> 00:02:37,200
the claim that it's achieving its project outcomes with the services they're delivering

30
00:02:37,200 --> 00:02:42,800
in the project simply because their outcome indicators are not properly measuring the

31
00:02:42,800 --> 00:02:45,640
outcomes for this project.

32
00:02:45,640 --> 00:02:50,880
As you probably know from the trailer for this podcast, those 16 outcome indicators

33
00:02:50,880 --> 00:02:54,860
fall into one of five problem areas.

34
00:02:54,860 --> 00:03:01,800
In this project, eight of those 16 outcome indicators fall into the no comparison group

35
00:03:01,800 --> 00:03:08,320
problem, which is where there's no comparison group of people outside of the project that

36
00:03:08,320 --> 00:03:11,000
are also measured on the same outcome.

37
00:03:11,000 --> 00:03:16,080
Which means even if the people in the project are achieving the outcome, we don't know if

38
00:03:16,080 --> 00:03:21,200
they're achieving them any better than another group of people that could also achieve the

39
00:03:21,200 --> 00:03:26,360
outcome outside of the project without receiving the project services.

40
00:03:26,360 --> 00:03:32,720
Another seven of the outcome indicators out of the 16 fall into the problem area where

41
00:03:32,720 --> 00:03:38,720
the outcome indicator is not measuring the outcome at all and is measuring something

42
00:03:38,720 --> 00:03:40,880
not related to the outcome.

43
00:03:40,880 --> 00:03:46,800
So, and finally, we have one outcome indicator that is properly measuring the outcome.

44
00:03:46,800 --> 00:03:52,040
So, before I give you three examples where the outcome indicator fails to properly measure

45
00:03:52,040 --> 00:03:57,160
the project outcome, I'm going to briefly describe the services the project delivers

46
00:03:57,160 --> 00:04:00,760
and the target groups that receive those services.

47
00:04:00,760 --> 00:04:06,860
And then I'm going to list the seven outcomes that the project expects to achieve from delivering

48
00:04:06,860 --> 00:04:10,260
those services to those target groups.

49
00:04:10,260 --> 00:04:13,200
There are 11 target groups in this project.

50
00:04:13,200 --> 00:04:19,240
The first target group is staff at targeted health facilities and the services being delivered

51
00:04:19,240 --> 00:04:25,960
to them are training on a maternal and perinatal death surveillance and response system so

52
00:04:25,960 --> 00:04:32,160
that these health facilities have a functional system to report on maternal and perinatal

53
00:04:32,160 --> 00:04:33,800
deaths.

54
00:04:33,800 --> 00:04:38,700
Staff at these health facilities are also provided training on how to repair and maintain

55
00:04:38,700 --> 00:04:45,560
solar and water supply systems since these health facilities are provided essential medicines,

56
00:04:45,560 --> 00:04:48,160
solar panels and water supply.

57
00:04:48,160 --> 00:04:52,980
These health facilities are also provided with health care workers, especially health

58
00:04:52,980 --> 00:04:59,880
care workers trained in delivering gender responsive sexual reproductive health services

59
00:04:59,880 --> 00:05:06,680
and being able to respect the rights of those receiving these services.

60
00:05:06,680 --> 00:05:09,880
A second target group are health care personnel.

61
00:05:09,880 --> 00:05:15,640
This group receives training in how to mentor and supervise students completing their practicums.

62
00:05:15,640 --> 00:05:24,840
A third target group are staff at one stop center family protection centers who are provided

63
00:05:24,840 --> 00:05:31,920
training on how to deliver information and services for victims of gender based violence,

64
00:05:31,920 --> 00:05:35,280
including how to take cases to court.

65
00:05:35,280 --> 00:05:40,400
A fourth target group are women and girls and men and boys who are delivered messages

66
00:05:40,400 --> 00:05:45,880
on the importance and availability of using sexual reproductive health and gender based

67
00:05:45,880 --> 00:05:47,800
violence services.

68
00:05:47,800 --> 00:05:54,760
A fifth target group are community action groups and religious and civic leaders that

69
00:05:54,760 --> 00:06:00,880
receive messages to promote women's use of sexual reproductive health and gender based

70
00:06:00,880 --> 00:06:02,720
violence services.

71
00:06:02,720 --> 00:06:07,880
It is not clear from the performance measurement framework, but we can assume that either training

72
00:06:07,880 --> 00:06:14,960
or messages are also delivered to these community action groups on how to prevent and respond

73
00:06:14,960 --> 00:06:18,760
to gender based violence and child marriage.

74
00:06:18,760 --> 00:06:24,960
Because the outcome in the PMF is and I quote, community structures are mobilized to take

75
00:06:24,960 --> 00:06:30,080
actions to prevent and respond to gender based violence and harmful practices.

76
00:06:30,080 --> 00:06:36,680
A sixth target group are men and boys who are oriented on their role as men and boys

77
00:06:36,680 --> 00:06:43,560
to prevent and respond to gender based violence and other harmful practices, as well as promote

78
00:06:43,560 --> 00:06:52,200
women and girls rights to access sexual reproductive health and gender based violence services.

79
00:06:52,200 --> 00:06:58,160
A seventh target group are service providers and community based organizations, state and

80
00:06:58,160 --> 00:07:04,640
national coordination bodies who receive training on sexual reproductive health and gender based

81
00:07:04,640 --> 00:07:07,440
violence prevention and response.

82
00:07:07,440 --> 00:07:14,000
An eighth target group are health training institutions, including tutors within these

83
00:07:14,000 --> 00:07:21,080
institutions where these tutors are trained on how to train health professionals in gender

84
00:07:21,080 --> 00:07:26,400
sensitive sexual reproductive health rights and gender based violence services.

85
00:07:26,400 --> 00:07:32,040
These institutions are also provided with educational materials and solar power.

86
00:07:32,040 --> 00:07:39,080
These health training institutions include health sciences institutes that receive quality

87
00:07:39,080 --> 00:07:42,800
assurance regular field supervision visits.

88
00:07:42,800 --> 00:07:50,320
A ninth target group are health professional associations that are provided with technical

89
00:07:50,320 --> 00:07:57,040
support on how to advocate for and monitor sexual reproductive health and gender based

90
00:07:57,040 --> 00:07:58,480
violence services.

91
00:07:58,480 --> 00:08:04,320
A tenth target group are Ministry of Health staff at national and state level who are

92
00:08:04,320 --> 00:08:09,960
provided with technical support on how to coordinate and manage sexual reproductive

93
00:08:09,960 --> 00:08:13,080
health and gender based violence services.

94
00:08:13,080 --> 00:08:20,720
Finally, the Performance Measurement Framework, PMF states an expected outcome is of laws

95
00:08:20,720 --> 00:08:26,960
and policies being developed, revised and enacted with the Minister of Health and Parliament

96
00:08:26,960 --> 00:08:29,080
passing those acts.

97
00:08:29,080 --> 00:08:34,200
Those acts are the Midwife Act and the Anti-Gender Based Violence Act.

98
00:08:34,200 --> 00:08:38,560
Therefore the 11th target group would be the Minister of Health and Parliament.

99
00:08:38,560 --> 00:08:43,840
However, the PMF does not indicate if the Minister of Health and Members of Parliament

100
00:08:43,840 --> 00:08:49,840
need training to achieve this expected outcome of passing these acts.

101
00:08:49,840 --> 00:08:54,440
They probably are assuming in this project that the Ministry of Health staff and the

102
00:08:54,440 --> 00:09:00,680
Health Professional Association staff, not the politicians, through their training received

103
00:09:00,680 --> 00:09:07,720
from this project is sufficient to convince these politicians to pass these acts.

104
00:09:07,720 --> 00:09:13,460
So that gives you an idea of the services this $37 million project delivers in South

105
00:09:13,460 --> 00:09:17,600
Sudan by the United Nations Population Fund.

106
00:09:17,600 --> 00:09:23,100
Now after those services are delivered they expect to achieve for these target groups

107
00:09:23,100 --> 00:09:25,280
seven outcomes.

108
00:09:25,280 --> 00:09:31,520
And those outcomes are, outcome number one, improved national and state government capacities

109
00:09:31,520 --> 00:09:37,600
to develop and implement sexual reproductive health, gender based violence, HIV policies,

110
00:09:37,600 --> 00:09:41,120
legislation and coordination mechanisms.

111
00:09:41,120 --> 00:09:47,080
Outcome number two, strengthened capacities of health training institutions to produce

112
00:09:47,080 --> 00:09:54,200
quality human resources for sexual reproductive health rights, gender based violence and HIV

113
00:09:54,200 --> 00:09:55,200
services.

114
00:09:55,200 --> 00:10:01,520
Outcome number three, increased awareness among women and girls to utilize gender responsive

115
00:10:01,520 --> 00:10:06,260
sexual reproductive health, gender based violence and HIV services.

116
00:10:06,260 --> 00:10:12,800
Outcome number four, strengthened capacity of targeted national and state level health

117
00:10:12,800 --> 00:10:19,760
facilities to provide quality integrated sexual reproductive health rights, gender based violence

118
00:10:19,760 --> 00:10:23,120
and HIV services.

119
00:10:23,120 --> 00:10:28,600
Outcome number five, strengthened enabling environment at the national state and local

120
00:10:28,600 --> 00:10:35,240
level for women and girls access to sexual reproductive health rights and gender based

121
00:10:35,240 --> 00:10:37,380
violence services.

122
00:10:37,380 --> 00:10:43,080
Outcome number six, increased use of gender responsive sexual reproductive health rights

123
00:10:43,080 --> 00:10:49,100
and sexual gender based violence services by women and girls, especially those living

124
00:10:49,100 --> 00:10:51,280
in vulnerable situations.

125
00:10:51,280 --> 00:10:57,880
And finally, outcome number seven, increased realization of sexual and reproductive health

126
00:10:57,880 --> 00:11:02,820
and rights of women and girls in South Sudan.

127
00:11:02,820 --> 00:11:08,440
Now we're going to look at three outcomes and three outcome indicators for each of those

128
00:11:08,440 --> 00:11:13,840
outcomes and explain how they do not properly measure the outcome.

129
00:11:13,840 --> 00:11:20,000
So let's start with outcome number five, strengthened enabling environment at the national state

130
00:11:20,000 --> 00:11:26,040
and local level for women and girls access to sexual reproductive health rights and gender

131
00:11:26,040 --> 00:11:28,080
based violence services.

132
00:11:28,080 --> 00:11:34,480
One of the two outcome indicators in the PMF that is used to measure strengthened enabling

133
00:11:34,480 --> 00:11:42,840
environment is proportion of targeted health science institutions that meet the minimum

134
00:11:42,840 --> 00:11:47,400
educational standard for training health professionals.

135
00:11:47,400 --> 00:11:53,240
We're going to assume that if these health science institutions are training health professionals

136
00:11:53,240 --> 00:12:00,220
up to a minimum educational standard, that this is good for women and girls who receive

137
00:12:00,220 --> 00:12:05,140
sexual reproductive health and gender based violence services from these students who

138
00:12:05,140 --> 00:12:10,800
graduate from these institutions and start delivering these services to these women and

139
00:12:10,800 --> 00:12:12,740
girls.

140
00:12:12,740 --> 00:12:18,080
So this is the same as saying we have a strengthened enabling environment for women and girls when

141
00:12:18,080 --> 00:12:20,440
they access these services.

142
00:12:20,440 --> 00:12:26,760
That seems to be a good match between the outcome indicator and the outcome statement.

143
00:12:26,760 --> 00:12:32,560
But when we look at the performance measurement framework for this outcome indicator, we see

144
00:12:32,560 --> 00:12:40,800
that an assessment is done every three months at the institution along with a progress report.

145
00:12:40,800 --> 00:12:45,840
Upon reviewing the performance measurement framework as well as the evaluation plan,

146
00:12:45,840 --> 00:12:53,720
I cannot find a measurement tool on how this proportion of health science institutions

147
00:12:53,720 --> 00:13:01,560
that meet this minimum educational standard for training health professionals is calculated.

148
00:13:01,560 --> 00:13:09,180
The PMF shows that the people inside of this institution who are responsible for training

149
00:13:09,180 --> 00:13:15,640
health professionals will be assessed as to whether they meet this minimum educational

150
00:13:15,640 --> 00:13:16,640
standard.

151
00:13:16,640 --> 00:13:21,440
The outcome indicator is excellent, but there is no explanation in the performance measurement

152
00:13:21,440 --> 00:13:25,920
framework as to how they calculate this percentage.

153
00:13:25,920 --> 00:13:31,300
We need to ask the United Nations Population Fund, South Sudan representative, where is

154
00:13:31,300 --> 00:13:36,240
the method and calculation that comes up with this percentage?

155
00:13:36,240 --> 00:13:43,360
Also in the PMF, the staff at the health science institution itself, the staff at the Ministry

156
00:13:43,360 --> 00:13:49,040
of Health and the United Nations population staff are responsible for coming up with this

157
00:13:49,040 --> 00:13:50,720
percentage.

158
00:13:50,720 --> 00:13:57,640
It would be better if an external third party would be responsible for assessing these staff

159
00:13:57,640 --> 00:14:04,320
in these health science institutions to see what percentage of them have reached this

160
00:14:04,320 --> 00:14:10,280
minimum educational standard for training health professionals.

161
00:14:10,280 --> 00:14:15,240
Then this could be repeated across all health science institutions to get the proportion

162
00:14:15,240 --> 00:14:22,120
of all health science institutions that achieve this minimum educational standard for the

163
00:14:22,120 --> 00:14:23,380
indicator.

164
00:14:23,380 --> 00:14:29,360
That would remove any bias since right now the United Nations Population Fund staff and

165
00:14:29,360 --> 00:14:35,660
these health science institution staff are assessing themselves on how well they train

166
00:14:35,660 --> 00:14:41,040
the health professionals to reach this minimum educational standard.

167
00:14:41,040 --> 00:14:46,720
You may recall from the project services I described earlier that tutors at these health

168
00:14:46,720 --> 00:14:52,920
science institutions are trained by the project on how to train health professionals to be

169
00:14:52,920 --> 00:14:59,720
quote gender sensitive in their delivery of sexual reproductive health rights and gender

170
00:14:59,720 --> 00:15:03,900
based violence services.

171
00:15:03,900 --> 00:15:10,760
The project also delivers quality assurance regular field supervision visits at these

172
00:15:10,760 --> 00:15:18,240
health science institutions which should or could include supervision of these tutors

173
00:15:18,240 --> 00:15:24,340
in how they train midwives in delivering gender sensitive sexual reproductive health rights

174
00:15:24,340 --> 00:15:28,040
and gender based violence services.

175
00:15:28,040 --> 00:15:34,520
So what is the minimum percentage of tutors at these health science institutions that

176
00:15:34,520 --> 00:15:42,120
first need to reach this minimum educational standard based on this assessment that's listed

177
00:15:42,120 --> 00:15:50,180
in the PMF before that health science institution as a whole is declared to have reached this

178
00:15:50,180 --> 00:15:55,260
minimum educational standard for training health professionals.

179
00:15:55,260 --> 00:15:59,600
And then we have to figure that out and then we can move on to the next health science

180
00:15:59,600 --> 00:16:05,120
institution and repeat it so that for all health institutions we figure out what the

181
00:16:05,120 --> 00:16:10,320
percentages that meet this minimum educational standard.

182
00:16:10,320 --> 00:16:14,640
In the performance measurement framework there is a target where they're trying to achieve

183
00:16:14,640 --> 00:16:20,960
80% of health science institutions reaching this minimum educational standard.

184
00:16:20,960 --> 00:16:25,120
But if only one tutor let's say out of a hundred tutors in one of these institutions shows

185
00:16:25,120 --> 00:16:29,600
up for the assessment it passes it.

186
00:16:29,600 --> 00:16:35,320
Is that enough to say that there's a sufficient number of tutors that have passed the minimum

187
00:16:35,320 --> 00:16:42,320
requirements to train the midwives before we move on to the next health science institution?

188
00:16:42,320 --> 00:16:48,120
So the percentage could be misleading we just don't know what's going on inside the institution

189
00:16:48,120 --> 00:16:54,800
to come up with this magical percent of health science institutions that have reached this

190
00:16:54,800 --> 00:16:57,360
minimum educational standard.

191
00:16:57,360 --> 00:17:02,320
So again we need to ask the United Nations population fund what percent of tutors do

192
00:17:02,320 --> 00:17:07,560
you require to assess before you conclude that the whole health science institution

193
00:17:07,560 --> 00:17:13,380
of all the tutors is good enough and it's reached the minimum educational standard.

194
00:17:13,380 --> 00:17:19,000
And then what percent of those tutors do you have as a minimum requirement to say ah we

195
00:17:19,000 --> 00:17:25,280
got 50% of them met the assessment minimum educational standard therefore we conclude

196
00:17:25,280 --> 00:17:30,240
that the whole health science institution is good enough we'll go on to the next health

197
00:17:30,240 --> 00:17:35,240
science institution before we get to this target of 80%.

198
00:17:35,240 --> 00:17:40,560
So we need to ask United Nations population fund about that and what's the assessment

199
00:17:40,560 --> 00:17:48,020
tool that they actually use to say oh that tutor you're up you're ready you have reached

200
00:17:48,020 --> 00:17:52,920
the minimum educational standard when you're training midwives on delivering gender-based

201
00:17:52,920 --> 00:17:57,920
violence and sexual reproductive health services you're okay and then they move on to the next

202
00:17:57,920 --> 00:17:59,880
tutor and assess.

203
00:17:59,880 --> 00:18:04,040
So we see there's progress reports we see that they do an assessment but there's no

204
00:18:04,040 --> 00:18:09,120
mention of how they come up with this percentage and that should be included in the performance

205
00:18:09,120 --> 00:18:15,160
measurement framework and even in the evaluation plan which is there's nothing in there it's

206
00:18:15,160 --> 00:18:21,520
just a generic evaluation plan for any project it's not specific to this project so that

207
00:18:21,520 --> 00:18:23,440
is a problem.

208
00:18:23,440 --> 00:18:28,960
Let's look at outcome number six increased use of gender responsive sexual reproductive

209
00:18:28,960 --> 00:18:34,520
health rights and sexual gender based violence services by women and girls especially those

210
00:18:34,520 --> 00:18:37,160
living in vulnerable situations.

211
00:18:37,160 --> 00:18:42,240
One of the outcome indicators measuring this increased use of gender responsive sexual

212
00:18:42,240 --> 00:18:47,840
reproductive health rights and gender based violence services is proportion of women aged

213
00:18:47,840 --> 00:18:57,000
15 to 49 who receive four or more antenatal visits as per targeted locations.

214
00:18:57,000 --> 00:19:01,800
This indicator has two problems first there is no mention of the proportion of women who

215
00:19:01,800 --> 00:19:09,040
receive four or more antenatal visits where those antenatal visits were specific to gender

216
00:19:09,040 --> 00:19:15,760
responsive sexual reproductive health rights as reported by the women and girls who receive

217
00:19:15,760 --> 00:19:17,960
these antenatal services.

218
00:19:17,960 --> 00:19:23,620
Remember this project trained tutors at health institutions to quote train health professionals

219
00:19:23,620 --> 00:19:30,400
in quote gender sensitive sexual reproductive health rights and gender based violence services.

220
00:19:30,400 --> 00:19:35,760
The indicator should include the same statement in the outcome on gender responsive sexual

221
00:19:35,760 --> 00:19:43,040
health reproductive rights to read proportion of women aged 15 to 49 who receive four or

222
00:19:43,040 --> 00:19:50,360
more antenatal visits as per targeted locations and where those visits were reported by these

223
00:19:50,360 --> 00:19:56,640
women as being quote gender responsive and respecting their sexual reproductive health

224
00:19:56,640 --> 00:19:57,960
rights.

225
00:19:57,960 --> 00:20:04,440
So we need to ask United Nations population fund why did they drop the increased use of

226
00:20:04,440 --> 00:20:10,480
gender responsive sexual reproductive health rights from the outcome indicator when it

227
00:20:10,480 --> 00:20:16,040
is expected to be achieved as stated in the outcome.

228
00:20:16,040 --> 00:20:22,160
The second problem is that the outcome indicator is only measured in quote targeted locations.

229
00:20:22,160 --> 00:20:29,060
So even if the proportion of women who receive four or more antenatal visits and who report

230
00:20:29,060 --> 00:20:35,560
those antenatal rights as being gender responsive and respecting their sexual reproductive health

231
00:20:35,560 --> 00:20:40,820
rights is going up thanks to the projects men and boys promoting women and girls right

232
00:20:40,820 --> 00:20:47,120
to access those services and the projects training of tutors to train health professionals

233
00:20:47,120 --> 00:20:52,640
on being gender responsive and respecting sexual reproductive health rights.

234
00:20:52,640 --> 00:20:57,520
It could be going up even more in another part of the country where the project is not

235
00:20:57,520 --> 00:21:00,860
delivering those training services right.

236
00:21:00,860 --> 00:21:06,520
So we need a comparison group of women not located in the project area that are also

237
00:21:06,520 --> 00:21:13,820
receiving antenatal services to also ask them if the antenatal services they received also

238
00:21:13,820 --> 00:21:20,840
respected their sexual reproductive health rights and were gender responsive.

239
00:21:20,840 --> 00:21:26,600
Then we compare their percentage against the percentage of women in the project who report

240
00:21:26,600 --> 00:21:28,160
those antenatal services.

241
00:21:28,160 --> 00:21:31,720
We would expect the project percent to be higher.

242
00:21:31,720 --> 00:21:39,120
Finally, outcome number seven increased realization of sexual and reproductive health and rights

243
00:21:39,120 --> 00:21:42,720
of women and girls in South Sudan.

244
00:21:42,720 --> 00:21:48,880
One of the outcome indicators for this outcome is the maternal mortality ratio.

245
00:21:48,880 --> 00:21:54,200
This indicator is a universally agreed upon indicator and could be seen as a proper measure

246
00:21:54,200 --> 00:22:00,340
of increased realization of sexual reproductive health for women and girls.

247
00:22:00,340 --> 00:22:06,500
If these women and girls are experiencing lower rates of death due to pregnancy after

248
00:22:06,500 --> 00:22:11,560
receiving sexual reproductive health services in this project.

249
00:22:11,560 --> 00:22:15,360
But there are two points I need to make clear here.

250
00:22:15,360 --> 00:22:20,900
First, the project delivers these necessary sexual reproductive health services to women

251
00:22:20,900 --> 00:22:24,800
and girls in quote targeted locations.

252
00:22:24,800 --> 00:22:30,400
For example, from the performance measurement framework, health facilities in these targeted

253
00:22:30,400 --> 00:22:34,080
locations are provided with health care workers.

254
00:22:34,080 --> 00:22:39,680
This suggests there are health facilities outside of these targeted locations that do

255
00:22:39,680 --> 00:22:44,880
not receive these health care workers along with the sexual reproductive health services

256
00:22:44,880 --> 00:22:46,880
they deliver.

257
00:22:46,880 --> 00:22:53,520
Secondly, this project also delivers other services other than sexual reproductive health

258
00:22:53,520 --> 00:23:00,120
services to women and girls such as training for tutors and mentors of midwife students

259
00:23:00,120 --> 00:23:05,840
on how to deliver gender responsive sexual reproductive health services and how to respect

260
00:23:05,840 --> 00:23:11,360
the sexual reproductive health rights of women and girls when delivering these services.

261
00:23:11,360 --> 00:23:16,640
There's also training to men and boys to promote sexual reproductive health rights for

262
00:23:16,640 --> 00:23:17,960
women and girls.

263
00:23:17,960 --> 00:23:23,300
There's also training to Ministry of Health staff on how to manage the delivery of sexual

264
00:23:23,300 --> 00:23:26,760
reproductive health services to these women and girls.

265
00:23:26,760 --> 00:23:33,320
There's also training to health profession associations on how to advocate for and monitor

266
00:23:33,320 --> 00:23:37,700
sexual reproductive health services to these women and girls.

267
00:23:37,700 --> 00:23:45,200
So even if the percent of deaths due to pregnancy goes down, we need to show that this $37 million

268
00:23:45,200 --> 00:23:51,440
project did it better with greater reductions in maternal deaths than in locations where

269
00:23:51,440 --> 00:23:53,840
this project was not operating.

270
00:23:53,840 --> 00:23:59,040
And we also need to show that all these trainings delivered to all these other target groups

271
00:23:59,040 --> 00:24:05,640
worked better than these same target groups in locations where the project was not delivering

272
00:24:05,640 --> 00:24:07,040
these trainings.

273
00:24:07,040 --> 00:24:12,700
Once again, we need comparison groups for each of these target groups.

274
00:24:12,700 --> 00:24:17,720
So now what I'm going to do is email this episode, part one, to the United Nations Population

275
00:24:17,720 --> 00:24:24,200
Fund representatives, inviting them to attend part two of this podcast where they can respond

276
00:24:24,200 --> 00:24:29,120
to this critique of this performance measurement framework.

277
00:24:29,120 --> 00:24:34,680
I will also send to the UNFPA representatives this performance measurement framework as

278
00:24:34,680 --> 00:24:41,000
well as my summary critique of all of the 16 outcome indicators for the project.

279
00:24:41,000 --> 00:24:45,500
And I'm also going to give the United Nations Population Fund representatives the option

280
00:24:45,500 --> 00:24:51,760
of sending me a written response to my critique if they are unable to attend the podcast.

281
00:24:51,760 --> 00:24:55,660
An evaluation expert will be attending part two of this podcast.

282
00:24:55,660 --> 00:25:00,200
And in part two of this podcast, we will be covering all of the seven outcomes for the

283
00:25:00,200 --> 00:25:05,260
project with at least one outcome indicator for each of the seven outcomes.

284
00:25:05,260 --> 00:25:09,560
If you have any comments or suggestions for this episode or wish to receive a copy of

285
00:25:09,560 --> 00:25:15,260
the performance measurement framework and my summary critique of all 16 outcome indicators

286
00:25:15,260 --> 00:25:22,480
for this UNFPA project, you can email me at evaluatecanadaaid at gmail.com.

287
00:25:22,480 --> 00:25:28,040
And if there are any evaluation experts who'd like to be on the podcast to look at other

288
00:25:28,040 --> 00:25:32,240
performance measurement frameworks, you can email me and I'd be happy to send you the

289
00:25:32,240 --> 00:25:38,360
remaining performance measurement frameworks that we have lined up for season one.

290
00:25:38,360 --> 00:25:43,440
Thank you for listening and stay tuned for episode three, part two, featuring the United

291
00:25:43,440 --> 00:25:45,240
Nations Population Fund.

292
00:25:45,240 --> 00:25:52,640
And there's $37 million project in South Sudan called the Human Resources for Sexual Reproductive

293
00:25:52,640 --> 00:26:09,000
Health Rights and Gender-Based Violence.