Welcome to Light Bites, an occasional podcast
from Leeds Institute for Teaching Excellence
at the University of Leeds. Episodes will be
hosted by members of the LITE team. And we'll
be showcasing the scholarship of teaching and
learning from across the university. Hello and
welcome to LITE Bites. I'm Robert Averies and
I'm joined today by Alba del Pozo. Hello.
To talk about her LITE Fellowship on formative
assessment. I'm also joined by LITE's Izzy Harvey,
who's developing her own long-term research
in learner-focused feedback. How are you both
today? We're good. Thank you, Robert. Very well,
thank you. Excited to be on LITE Bites? Always.
This is your first time. I know, but there might
be more. Okay. We'll hold you to that. So, Alba,
could you tell listeners a bit more about what
you do at the University of Leeds? Yes. So, I
mean, I have many hats, starting with I'm a lecturer
of Spanish language at the School of Language,
Cultures and Societies, but I'm also the Incubator
Lead for Feedback at LITE. I just finished my
LITE fellowship. on formative assessment, as
you said. I'm also the director of scholarship
in my school. So in a way, all these roles, they
are linked to teaching, to scholarship or pedagogical
research and to the activities you do around
teaching. Fantastic. And Izzy, would you like
to just introduce what you do at LITE? Yeah,
I only have one hat and I'm a research assistant
at the Leeds Institute for Teaching Excellence.
Fantastic. And you'll be bringing in some thoughts
from your own research and the relationship between
that and Alba's own fellowship, which I'm really
looking forward to. So Alba, what motivated you
to start your LITE Fellowship and what issue
did you decide to address? That's a good question
because... It was really a combination of chance,
luck and my own pedagogical interest around formative
assessment. I'll explain that a bit more. So
basically, my fellowship was a sponsored fellowship,
which means that unlike other fellowship projects.
There was this title, which was, as you were
saying before, about inclusive formative assessment
design and the impact on student outcomes. And
you applied for that as you would apply for a
job position more than developing a project and
sending a project. And I had a previous interest.
On peer assessment, I had, as part of my own
teaching practice, I had implemented a small
project on peer oral assessment in the Spanish
language teaching class. I had published a small
piece as a book chapter on that. And I thought,
hmm, this is something that I actually could
explore further. So I sent the application with
actually very little hope it could be successful.
But it was successful and this is how I started
my LITE Fellowship project. And then I started
to explore a bit more how students perceive formative
assessment. Great to hear that the LITE Fellowship
enabled you to look into the underlying factors
of an issue you'd identified and not just an
issue that's exclusive to your own disciplinary
context. Could you provide listeners with a brief
overview of how you went about your project?
Yeah, so because it was a sponsor fellowship,
I was given a lot of freedom to design the project.
I think my main question at the beginning was,
well, a lot of colleagues, a lot of academic
staff, they tend to always complain about students
not really engaging with formative activities,
why they're not doing these formative essays
or these submissions. So initially I started
with that research question. We did a series
of focus groups. In fact, I have here the numbers.
We did nine student focus groups with a total
number of 27 participants, which was quite a
success. And I also interviewed nine members
of staff to try to compare how students perceive
their barriers to engage with formative assessments,
but also... what were the barriers perceived
by the staff. As you know yourself, Robert, and
Izzy here, you helped running some focus groups
because I did the interviews with the staff myself,
but then the focus groups were run by colleagues
from LITE or even by some students as well for
ethical reasons because maybe the students were
feeling kind of, you know, a bit that they were
pressured to say what I was expecting them to
say. And in fact, it would be good to hear from
you what was your experience during the focus
groups. Robert, do you want to go first? Sure.
It's testing my memory a bit. This was a little
while ago and not long after I arrived at LITE,
so things were a bit of a blur. But I can remember
the focus group discussion. I was a part of and
facilitating to be really dynamic. And there
was a strong theme of feedback rising to the
top of the discussion at various stages. Peer
and participatory feedback opportunities were
seen as normatively good things, but varied quite
a lot in practice. Don't want to go in any more
detail than that. A, because it's, as I say,
stretching my memory. B, because I know you're
going to talk about your findings shortly and
I don't want to provide any spoilers. How about
you, Izzy? My reflections might provide a few
spoilers. So first of all. Provide spoilers.
Is that OK? Yeah. OK. So first of all, again,
I was only a few months into my new role as a
research assistant. And this was probably one
of my first experience of focus groups from the
research aside. It was very useful for me how
organised Alba was with everything admin related.
And again, I would agree, Robert, I think that
there was a really strong discussions arose around
feedback. And two things that stick out for me
to this day are one, the full spectrum of feedback
experiences described by the students. So from
formative opportunities with every assignment
to very little to no formative feedback opportunities
and nothing really beyond a mark or a grade for
some courses. And I guess being quite new to.
Being exposed to higher education and this kind
of landscape, that was something that really
surprised me because I just didn't realise how
vast the differences were. And that's something
that stuck with me. And then my second reflection
is actually in relation to the formative feedback
opportunities that were discussed by one of the
students. And she said that they actually got
the chance with every assignment that they did
to hand in an essay or a draft plan. Really interestingly
to me, the student reported how few of her peers
took up this opportunity. And that was really
surprising because it seems so valuable and apparently
quite rare across the university course, but
still wasn't taken up by that many people. And
that is kind of where I know we're going to talk
about this a bit later, but that was kind of
one of the first early sparks of interest for
me to explore feedback in higher education further,
starting with student engagement and formative
feedback opportunities. And yeah, I would say
those focus groups were one of the earliest sparks
for me that's led to many more things that we'll
be discussing soon. Fantastic. I love how I came
out of that focus group thinking, wow, feedback
is really important and then did nothing about
it. And you came out of it and started a PhD.
And the rest of history, as they say. Lovely.
You should be the host. Wrong reversal. So was
that interesting to listen to? Yeah, yeah, it
was because it reminded me of conversations that
I had with you both, actually, about we wanted
the students in the focus group to talk about
formative assessment and they kept talking about
feedback. And in a way, I'm giving spoilers of
what I'm going to talk about in a minute, but
it actually came up that the students care deeply
about feedback. They didn't care that much about
the assessment being formative or summative.
Fantastic. So that does lead us nicely into my
next question, which what did you learn? What
were the main findings of the research? One word,
feedback. And we can finish the podcast here.
No, really. As I was saying, I think the main
finding, which is not new, is that students were
actually the main. There were many. reasons for
students not to engage in formative activities,
if you remember. But because there were deadlines,
like everything, all those submissions happening
at the same time, so they would prioritise like
summative assessments. It was workload as well,
which had to do with deadlines. But then it came
up that it was about the expectations that they
had about the feedback they would get. If students
believe that the feedback they were going to
get from that formative activity wasn't useful
for future formative activities, they wouldn't
engage with it. And I think that was the main
finding. But also there were kind of previous
expectations about what feedback was. There was
a lot of this idea. regarding having mocks or
having drills and having a lot of repetition,
which it made me ask as well, is this what higher
education is for or is it about previous expectations
from students doing their levels and being focused
so many years of their formative life, their
formative years and on doing an exam and preparing
for that exam as well. I have here a list of
findings that I'm going to check. Something that
came up, well, it didn't came up, but it made
me reflect as well, was about feedback culture
and feedback literacy. Students had this tendency
of considering feedback as something you get
in written form. tailored and personalised. And
that was the feedback they valued the most. But
quite often, I would have liked to see in the
data that students were aware that feedback can
take many forms, that it can be more informal,
that it can be given by peers, that sometimes
academic staff might give feedback to the group
as a whole. And there wasn't a lot of mention
to that. So hopefully the university and... possibly
we'll talk about that later, is going to deliver
a strong message on what is feedback and all
the feedback opportunities students might have.
And something I remember very well was the completely
mixed opinions about peer feedback. There's a
massive body of literature saying how good peer
feedback is and why we should all be doing peer
feedback. I've done it myself. It's more difficult
than it seems and it's to be massively scaffolded
for it to work and sometimes it doesn't. But,
for example, there were students that they were
saying that the best feedback they had received
in their lives was from peers. And those were
students' healthcare from the area of healthcare
and medicine. And I wonder if in that area, giving
feedback is actually one of the skills that is
expected from graduates and they work on that.
But on the humanities and social sciences side,
literally, I have a quote that said that they
thought it was cheap to give peer feedback and
they felt it was cheap. So completely mixed opinions
that... Gives you an idea of how complex is feedback
and how messy education is, which it makes it
joyful as well. So there's, although I mentioned
earlier, obviously this is an issue that takes
place across higher education in different spaces,
there are also contextual differences, including
disciplinary assumptions, for example. Is that
about right? Yeah, because... When we did the
coding, one of the actual codes I have in NVivo
is in my disciplinary identity, in my studies,
in humanities or in healthcare or in STEM, we
do this. And there was a lot of interventions
from students saying, oh, we do this in my modules
or in my program. Oh, but in our module, we do
this. We talk about feedback in general quite
often and sometimes we don't take into account
that we might be talking about different processes
and different things depending on the context
and the discipline. So feel free to shoot me
down if this is jumping ahead. I know Izzy's
going to come in shortly, but does this call
for students to be more explicitly informed about
the reasons for doing formative assessment and
through that feedback? And if so, what are some
of the ways of highlighting this? I think there
needs to be a... cultural shift. And I would
like to see feedback detached from assessment.
I've had this conversation many times with colleagues
that they think, well, but feedback comes with
assessment. How can you have feedback without
assessment? I actually think we should have feedback
not without assessment, but at least before assessment.
There is this institutional obsession about providing
feedback with a grade and explaining that grade
at the end of the course. And quite often this
is not working for students or is not working
the way we expect for students. And at the same
time, I'm thinking about the interviews with
staff and any colleague belonging to academic
staff, they will tell you that they're most...
dreaded period of the year is marking season
or assessment season as something that we tend
not to like. Because in a way you put all that
effort knowing that it might not be read and
you cannot blame the students for that because
the students, it's the end of the course, they
won't be able to change anything on what they've
submitted. They won't agree. That feedback should
have taken place before, but the institution
is telling us, no, you need to use your, I don't
know how many points of workload for assessment
at this point, not before. You still need to
write 200 words of feedback or 300 words or I
don't know how many. So I think I would like
to see a cultural shift, if that makes sense.
Yeah, I'd agree, Albert. I think that, yeah,
there's way too much emphasis placed on summative
feedback. And many of the times it's actually
got no use and no place to land for students.
They have nowhere to take it and use it in the
future because that module's over. And with the
system at university, often these don't link.
So next year they're going to be doing something
completely different. So, yeah, I totally agree
that there should be more those efforts and that
workload model or whatever. Their time they're
given would be much more valuable in formative
ways, providing that feedback throughout the
course in the build up to the summative and not
spending the staff time providing those when
it has very little relevance for anyone at that
point. I'm right in saying that you were also
involved at a later stage. Yes. And again, I
was fascinated to see that what I'd heard from
just the one focus group I did had carried on
and how much feedback was mentioned and discussed.
in the results by staff and by students it seemed
that no one could talk about formative assessment
without being swayed away towards talking about
formative feedback and yeah I agree with all
the things Alba was saying about there needing
to be a cultural shift and as Robert has said
so the early interest sparked by those focus
groups and then helping out with coding the reason
I needed a bit of coding for Alba was because
by then I was developing my own uh research proposal
which has now turned into my PhD when Alba is
my supervisor so we we had to part ways in the
fellowship support and yeah so that's and I am
now looking at some of those um some of those
understandings and what I mean things in the
first year of the PhD are messy and they shift
all the time but we're kind of exploring perceptions
and understandings of um staff at the teaching
level and at the leadership level and how how
they align or differ and what if there's some
some things we can unpick there to help with
this cultural shift. Thanks Izzy so bringing
this all together then what sorts of changes
do you think you might like to see as a result
of this research? I think we've already hinted
that with the word cultural shift which I'm aware
is it can be a very vague word as well but As
I was saying, feedback tends to be linked with
final assessments and it shouldn't be linked
to that. Final assessments should be something
that gives you a grade and explain that grade
because those are usually as well looked at by
external examiners. But there should be feedback
and formative. I don't want to talk about formative
assessment. I'd rather use the word formative
activities or formative opportunities before
those summative assessments that have a tendency
to take place at the end of a course or at the
end of a module. And I would like to see that
need specifically recognised in student education
policies beyond... oh, that's a nice thing to
have, or we should all be doing that. And all
the staff I interviewed, they were aware that
we should all be doing that, but it's just not
recognised, and there isn't any incentive to
do that. And that's a wider problem that goes
beyond feedback. It's about recognising teaching
and giving value to teaching. But it cannot depend
on individual willingness to be implemented.
It needs to be part of policy in the same way
that everybody assumes that when you're doing
a final summative assessment, you're going to
follow some steps. There should be some, not
only guidance, but some policy. It needs to be
recognised on workloads. And I know, I mean,
when I ask staff, what were the main barriers
to implement formative assessments or activities
or opportunities or formative feedback workloads
was the main. And I know there are constraints
and I know that the higher education landscape
right now, there's a lot of constraints and financial
constraints around that. But we need to recognise
this and this time fairly. And it cannot be part
of, for example, in my school, any formative
feedback or formative opportunity that you give
is part of the class preparation time you have
in your workload. Every workload is a different
world in different schools. But no, it should
be specifically recognised in some way. Thinking
about students as well, I would like to hear
a stronger message about engaging. with feedback
and first understanding all the different forms
in which feedback take place. Feedback is not
only a written thing that you'll get on Minerva,
for example. Feedback can be collective. Feedback
can be given by peers. to kind of more actively
engage with that. And I'd like a stronger message
towards students about expectations on feedback.
Feedback is not only mock exams or drills, but
it's something way more complex. I'm actually
working on something that will be shared with
the whole student community about, for example,
how can you have all your feedback in one place?
How can you check the feedback you've received
before you submit an assessment? Sometimes the
systems do not help as well because, for example,
in the current systems that we have, students
will see the grade before seeing the feedback.
So quite often, once they see the grade, they
might not want to look at the feedback, maybe
because they're happy with the grade, maybe because...
They're scared, if they're not happy, they're
scared of what they might read about it. So that
bit, the word feedback literacy, it seems to
be the flavour of the month. But helping students
to engage with feedback better than they do.
And additionally, I would like to see the NSS
question about feedback. Quite often when you
read papers about feedback, they start saying,
well, we need to improve on that because that's
the lowest score in the whole sector. And I'm
going to read out the questions for the podcast.
There's question 13 that says, how often have
you received assessment feedback on time? And
question 14, how often does feedback help you?
improve your work? Those questions create a lot
of confusion, especially because they're asked
at final year students. The last piece of feedback
was potentially the last piece of feedback they're
going to receive at the university. And usually
the last piece of feedback is aimed at explaining
a grade and justifying a grade, not at improving
a piece of work. So it's not clearly... explain
what feedback those questions are referring to.
And when they ask about assessment feedback on
time, in which time frame, on time in regards
to what. So I don't think they're really well
rephrased. I mean, additionally, I would like
to see the NSS abolished in general, but possibly
that's something to discuss in another episode.
A lot to reflect on there. Lots of really juicy
stuff to get into and at different levels of
scale. Something that I don't know about you,
Izzy, but something that I feel free to come
in, but something that I kept coming back to
was a lot of the questions around feedback contribute
to wider questions of educational purpose and
kind of the idea that all educational policy
and practice is inherently teleological. So it
contributes to things that happen outside of
the university. Is there a conversation that
needs to be had? And part of that might be disentangling
feedback from assessment about the broader purposes
of feedback and the ends to which feedback contributes.
Yeah, I mean, I think there's a conversation
to be had about what happens in a classroom,
which resources do you allocate to teaching and
how learning takes place. And obviously feedback
has. has a role to play in that because Izzy
has read a lot of papers talking about how feedback
is essential to learning, more than me at this
point with your PhD possibly. But there is a
conversation to be had about what is learning,
what is higher education for. I tend to go always
into the big questions, into the bigger questions.
And how this is valued as well by institutionally.
I think that is the bottom line is feedback is
undervalued. And it's not given its value in
the classroom. And then it's not given its value
as what it can contribute, like you said, to
the wider student success, students going forwards
into later life. It is just undervalued. I think
that's the bottom line. Yeah, especially in relation
to what we experienced in those focus groups
in terms of how students valued feedback and
in many cases were quite critical of their experiences.
I think the engagement thing is always put on
students when actually a lot of it is, I think
Alba might have mentioned this early on, is about
the timing that it's offered or about its perceived
usefulness or how it's given it come across to
them. I think there's a lot of work to be done
there as well. This reminds me of, we had a student
doing a project this summer that she was piloting.
a questioner for his PhD. And when we had the
last meeting with the student and we asked her,
like, what have you learned during this process?
She said something that resonates, that she said,
well, I've just realised that students and lecturers,
we have the same problems and we're all kind
of the same boat and we face very similar challenges.
In a way, if you think about feedback and you
think about some of the student responses regarding
deadlines, workload, engagement, and some of
the responses from academic staff colleagues,
they're more similar than one would think because
it's about value and engagement, as Izzy was
saying. Yeah, and that 100 % resonates in the
literature. One of my top coded phrases in my
reading is misaligned understanding. Yeah. Well,
fantastic. Not fantastic, but you know what I
mean. So for listeners that would like to know
more, because it would be great to go in more
depth, but obviously we can't right now. Are
there any further research outputs or activities
that can be engaged with? The incubator. This
is the advertising bit. So I said at the beginning
of this podcast that in a way, this fellowship
led me to apply. for the role of the feedback
Incubator Lead with LITE. Right now, we're trying
to build a community of colleagues. And when
I say colleagues, I say that including professional
services, even including students, including
the whole university community interested in
feedback and in pedagogical research about feedback.
good practice sharing about feedback or just
an interest in the topic. If you don't have an
idea but you have an interest, please join us.
And we are right now in the process of building
this community. There will be a reading list
to start with for anybody interested in reading
about feedback. It might not be an engaging topic
for everybody, but maybe you want to do something
small. with your practice, with your teaching,
and you don't know where to start, you can start
with us. So anybody interested in feedback, please
send me an email or contact LITE, any colleagues
from LITE, and they will signpost you to the
incubator. But hopefully I want to put in writing
and publish some of those findings and some of
those data, because otherwise... Pedagogical
research is not research or scholarship if you
don't share it with the world. And please listen
to the rest of the podcast. We'll also be providing
a link to any other research -related outputs.
For example, your light snapshot with a summary
of what you've spoken about today. That's a work
in progress. It's coming. No worries. Just holding
you to account by this recording. Great. Thanks
for that, Alba. And curious to ask, what next?
Developing the incubator and all the things you've
already mentioned. Well, building a community,
as I was saying, trying to write a paper on the
stuff I've been talking about, but also as part
of the incubator activity. I was thinking to
try to get like an open collective book at the
University of Leeds, the library. They have this
amazing opportunity that you can... publish a
book in an open format. And it's not just like
a PDF that you upload somewhere. It's more like,
you know, it's quite more interactive. They use
WordPress and they give you a lot of support.
So I'm interested in putting a call out there,
maybe optimistically by spring to see if there's
an interest in other activities, projects, reflections,
resources. to put everything together around
feedback and what's going on at the university
in just one place. I mean, what's next for me
is a year on maternity leave. But no, I am, well,
I'm currently looking again at my PhD questions
for the third time this year, having read a lot
more literature, attended conferences, had great
conversations with people. So I'm hoping to...
get those a bit more finalised before I go off.
And then I'm excited when I come back in 2027
to join Alba's thriving feedback incubator again
and pick up my PhD and I'm excited to see where
that research will take me. Fantastic, yeah.
As an outside observer, I'm really looking forward
to seeing how that community develops and the
areas of research that get developed as a result
of that. Actually, something that came up in
many conversations I have with colleagues and
with the LITE team, it is that feedback seems
like an under -research area, but at the same
time is a priority for the university and a priority
for Light. But as EZ is realising developing
her PhD project, it is quite difficult to do
research on feedback. I mean, there are many,
many questions that are really difficult to answer
that we've had many conversations about, like,
what is effective feedback? Even what is feedback?
What are we talking about when we talk feedback?
How do you measure that if that can be measured?
So it's actually quite a complex area, but also
quite under -researched. So we really welcome
people with an interest in the area. So many
of those things mean different things to different
people. And then also we're battling now with
who's involved and at what levels and who can
make the changes and where should we be tackling
this as well. I'm sure many areas of research
in higher education have many levels of complexity,
but this just feels like a huge web whenever
we delve into these conversations. Come and join
the web. It sounds like that complexity only
increases the benefits of joining the community
and having those collective conversations and
sharing expertise. Yeah, don't be alone. Last
but not least, my favourite question, perhaps.
Do you have any advice for anyone else who might
be considering embarking in a pedagogical research
project and is there anything you might have
done differently? I thought for this question,
I would list all my sins. So all this advice
is based on everything I've done myself wrong.
First, if you're doing data collection, always
pilot your questions. You know, interviews, questions,
always pilot that because possibly you're going
to have to tweak that. I didn't do it. This might
seem really obvious, but reading is important.
Literature reviews are important. Have a general
sense of what's out there, what has been written
about, what has been done before and where you
can, in a way, plant your flag and how can you
contribute to that body of knowledge. Even if
it's something quite small, and that's absolutely
fine, you don't need to do massive projects.
PedRes or scholarship in my role, in my school,
has to do with doing something small. And sometimes
that can have an impact in your own teaching.
And that's absolutely fine. And finally, avoid
reinventing the wheel because possibly that great
idea for a project that you have, it has already
been done. That has happened to me a lot of times.
Then don't be alone, as I was saying before,
kind of try to build a community, find your community,
find your tribe in a way, talk to colleagues,
come to the incubator again and ask questions
to your community and try to go on this journey
accompanied by colleagues, because otherwise
it can feel a bit lonely. Thank you very much.
And I would, I mean, Alba's mentioned this, but
this is one that I have learned in my first year
of my PhD is the importance of the literature.
And you'll probably almost never be finished
with reading. And it's so important to stay up
to date with the field. And the more you read,
the more informed you become. And I think also
it's important. to not be afraid to change and
develop things and let their literature guide
and aid you even if that wasn't what you originally
planned like Alba said you need to plant your
flag somewhere and you might need to take some
twists and turns because things are published
all the time we actually funnily enough one of
my earlier ideas quite close to my original proposal
was published by some of the higher mighties
in the feedback field about two weeks ago so
we were very relieved that we'd we'd been guided
recently to um take a new tangent great thanks
both yes my favourite section because I received
such great advice for myself selfishly um that's
it that's the end of the episode thanks Robert
thank you both for your time today thanks again
for listening and tune in next time