Welcome to LITE Bites, an occasional podcast
from Leeds Institute for Teaching Excellence
at the University of Leeds. Episodes will be
hosted by members of the LITE team. And we'll
be showcasing the scholarship of teaching and
learning from across the university. Hello and
welcome to LITE Bites. I'm Robert Averys. You
thought we'd forgotten, but no. You're about
to listen to part two of Manoj Ravi and Mohsen
Besharat's LITE Bites episode. on their LITE
Fellowship, Assessment Flexibility to Embed Real
Professional Practice. Not to put you to any
pressure, but the first part, which we released
in October 2024 and saw you present the findings
from the first phase of your fellowship, was
our most listened to episode to date. What do
you put that down to? I think, I don't know,
probably we just distributed that on different
platforms and people were interested, so to listen
to that. Yeah, we were quite lucky. We really
appreciate that. Clickbait title, probably. You're
being modest. So for the benefit of those who
haven't listened to part one, Manoj and Mohsen
are now going to introduce themselves and the
purpose of their LITE Fellowship. Very good.
Thank you so much for having us again, Robert.
So I'm now Associate Professor at the School
of Civil Engineering and I'm still Programme
Leader for three undergraduate programmes. I'm
involved in other university -wide initiatives
like sustainable curriculum and yeah with Manage
we started to do this LITE fellowship and the
aim for the fellowship was actually to understand
how we can support our learners in order to get
that sort of like skills and the knowledge they
need for professional practices we found that
sometimes students are a bit deviated from that
path and they probably focus on Mark and also
the process of the assessment is not actually
well reflecting what we are doing in industry.
So, yeah. Brilliant. Thanks, Mohsen. Thank you,
Robert, for having us back on The LITE Podcast.
Great to be back. Yeah, Manoj Ravi over from
the School of Chemical and Process Engineering,
where I work as a lecturer, joined through the
Curriculum Redefined Initiative at the university.
I'm also the program lead for the undergraduate
chemical engineering program in our school. And
yes, Mohsen's kindly introduced the remit and
the broad scope and aims of our fellowship. We
were looking at authentic assessments and flexible
assessments, kind of reinforcing. each other.
We're here for part two because in part one we
spoke primarily around the findings from the
student survey that we'd done largely in the
engineering and physical sciences faculty with
our students and it was a pretty large sample
size and there were findings which we kind of
expected when we put together the project proposal
but there were A couple of other interesting
things as well. So in terms of like a quick recap,
of course, do go listen to our first podcast
if not. But sort of quick recap would be we're
looking at a wide array of questions picked based
on the Authentic Assessment Framework and other
literature which has been published in that direction,
specifically to our students at EPS, Engineering
and Physical Sciences. A vast, vast majority
of students said that... They would incentivize
early submission if given an opportunity to get
formative feedback before they hand in their
summative piece. But it was not something necessarily
they were used to at scale and as often as possible
in their current curriculum across any of the
programs offered in the faculty. Yeah, so only
7 % have got my numbers printed out. 7 % said
that they experienced any kind of flexibility
in assessments in that survey. And closely linked
to flexibility was our... desire to couple creativity
to this and how do we kind of foster creativity
in students when they're handing in these assessments.
And again, only around 14 % said that they can
express creativity in coursework assessments.
So those were some of the things that we potentially
wanted to address in our pilot in our second
year, which is what we're going to talk about
on this podcast. So yeah, that's a brief recap
from the findings from the student survey in
terms of what scored high, what scored relatively
low. and the interventions that we potentially
wanted to make. And yeah, if you stay tuned to
this episode, you'll find out whether we failed
or we've had some reasonable success. That's
the hook. So you've now completed the second
and final year of your LITE Fellowship, and
I think we're all eager to hear how it's gone.
How and where was the pilot implemented? So for
pilot, we decided to implement the ideas we got
from the survey in a couple of modules. For myself,
I'm speaking from... My school then manager actually
can explain how he piloted the idea. So in School
of Civil Engineering, I piloted these ideas in
two MSC modules, which are in our two very successful
kind of MSC programs. One of them is called Wash,
Water, and Sanitation, and Health. And the other
one, EAPM, Environmental Engineering and Project
Management. And the name of the modules, just
for listeners, if they want, they can go and
check the modules on our module catalog, are
called Water Supply and the other one is Advanced
Wastewater Management. So from what we discussed
and what I actually managed now mentioned, we
tried to understand how we can implement ideas
of fostering the creativity and... providing
students with flexibility and more importantly
to give them this kind of feedback loop that
they wanted because 86 percent of students they
said that they incentivize actually they like
to have feedback early in order to work based
on the feedback not actually looking at the feedback
when they have got the mark that looking back
kind of approach but forward feedback so we improved
the authentic delivery because another finding
from the first phase of the project was that
authentic assessment is not actually one term
is actually the design of the assessment plus
the delivery of that so we try to focus on authentic
delivery and like I said we increase the flexibility
we focus on feedback quality and also we encourage
reflection in the in the process How we did that,
I introduced a group project in the assessment.
So students actually started to work on a design
task, which was like an open -end, very authentic
kind of problem. And through that, they work
in the groups through tutorial sessions that
were designed in order to provide them with the
feedback. And then they were given this opportunity
to submit their work from the group work at some
point. get feedback on that from from lectures
of the module and plus peer review from from
people in the group and eventually they could
take this feedback into the final phase which
is the submission of the of the report in terms
of the timetable I mean time frame of that it
was like the submission of the group activity
was at the end of there for example November
because this was like a semester one module then
at the beginning of the December they got feedback
from the group submission which was in the format
of a poster and we saw a lot of like creative
design of the poster then we had an oral presentation
of the poster which they received the feedback
from module contributors and then at the end
of the December they were due to submit their
final report so And during this journey, we did
journaling. So it was like six -week journaling
of the student to understand how they are developing
over the course of this experience. In terms
of some indices that we picked from the first
phase of the research, like stress and perception
of the workload and creativity, we saw a big
jump from week one to week six. For example,
a huge amount of students, they said that they
were really stressed by the... assessment at
the beginning at week one but then at week six
we had about 100 percent actually of students
they said they found the workload manageable
and they feel less stressed by the work so I
think the process overall supported student to
develop this sort of like developing different
skills and learning different things designed
for the project during the six weeks. And an
interesting part was that towards the end of
the journey, towards the week six, they were
provided with more tasks to do. But despite that,
they found the experience quite kind of manageable
and less stressed. The quality of the work eventually
was really good. We had very low failure rate,
only one student actually in the cohort failed,
and we had a lot of marks above 70%. Brilliant.
Thanks for that, Mohsen. Very extensive. work
over at the School of Civil Engineering. In terms
of the motivations, yeah, it was very similar
to me, where I implemented the pilot at the School
of Chemical and Process Engineering for a first
-year undergraduate module that's taken by chemical
engineering students. And this was around a lab
-based assessment, so a practical work, and then
a piece of coursework, which is usually linked
to the lab practical. Just to probably put it
in context, the way it's usually worked is we've
designed, this is largely again to first year
students, so we've designed the experiment to
a fairly nuanced degree. Students come in in
small groups, they perform the experiment in
the lab, they take away the data from it and
the coursework usually involves data analysis
and interpretation, usually working to a two
week. turnaround time from the day to the practical
to when the submission is due. So in terms of
the pilot, this assessment was already in place,
but we went about making three specific changes
in terms of looking back at what students told
us from that all faculty survey in terms of flexibility.
So we kind of wanted to embed this. three ways.
We started off by giving them, firstly, flexibility
in the operating conditions. So we did not necessarily
specify the exact experiments that they're going
to run in the lab. We gave them a kind of window
of operation of things like temperature and flow
rates. Don't want to go into the nitty gritties
of the experiment. But yeah, there were choices
in the operating conditions that they could collectively
decide as a group. So that was the first element
of flexibility. The second thing which we also
started embedding based on the fact that, as
Mohsen very rightly said, that students greatly
value formative feedback. So we wanted to move
away from the approach of they do the experiment
in the lab, go away, work in silos for two weeks,
trying to get the data interpreted and then hand
it in for a mark. But we kind of embedded. this
opportunity to get formative feedback from the
instructor. So for me on that module, one week
into their assessment period. So after a week
that they'd done the lab, they could come and
show any kind of work. This need not be, it could
have been a data table. It could have been a
paragraph on the introduction. It could have
been an interpretation of the results. It was
up to the students in terms of what they wanted
to get feedback on and how extensive they wanted
it to be. But it was verbal feedback. So there
was no pressure on them in terms of. You have
to give me a PowerPoint slide or you have to
write up a specific A4 paper. It was very much
up to them. And the third and final element of
flexibility, which we tried to link this more
to the creativity really, was in terms of the
format of their submission. So we said you could
either hand this in as a report or it could also
be a video recording. But the rubric and the
assessment brief was designed in a way where
we explicitly showed how the learning outcomes
were the same in terms of what they were meeting
it. irrespective of whether they decided to hand
it in as a report or if they decided to do it
as a video recording. So those were the three
elements of flexibility. In terms of how we analyzed
the data from the pilot, very similar to what
Mohsen did, but because we were working to a
two -week turnaround period, it was effectively
a... sort of end of first week journaling, and
then very similar set of questions for the end
of assessment survey. So that's what worked there.
In terms of the main positives, and certainly
formative feedback, again, was extremely valued.
We saw 100%. return rate in terms of did you
find the feedback useful going into your eventual
submission is a 100 % positive response on that.
And then what we did was make a comparison across
the other questions in terms of we had the averages
from the faculty survey and wanted to test if
our interventions in the pilot was successful.
So 82 % like the flexibility offered on this
particular lab practicals that compares to a
low 7 % in the faculty pool who said they'd experienced
any kind of flexibility. So that was a certain
win on that count. The other thing which... we
also found worked was just in terms of challenging
assumptions when it comes to analyzing and interpreting
data, because that is pretty key when, again,
it might not be just for engineering students,
but you tend to work in very idealized environments
in academia. But when you graduate, you should
be in a position to challenge assumptions and
theories. And there, again, it was close to 50
% on our pilot study, which was down around about
15, 16 % in the faculty poll. But the third thing
that that I will highlight, and we'll speak a
little bit about this as we go along when we
go to staff perspectives, is about Having these
assessment guidelines in place, which should
be clear enough to set out the expectations in
terms of what we want from the students, but
they should not potentially be hindering the
expression of creativity. So that is what we
asked students if they felt that the guidelines
restricted expression of creativity. 33 % said
yes in the faculty poll, and it was down to 0
% on our pilot. So no one felt that the guidelines
was restricting them from expressing creativity.
Having said that, all of that, I don't want to
say that whatever we did, whatever we touched
turned to gold. There were things where potentially
did not work as envisioned or as planned. One
of the things that we are keen on as pedagogic
researchers, but also as educators, is to make
the assessment experience enjoyable for students
during the process. So one of these key questions
that we ask them is... Do you experience sort
of a sense of passion and purpose when working
on the assessment? So that's over the two week
window. And then there's the overriding sense
of accomplishment and achievement once they hand
the submission in after a couple of weeks. And
usually we see there's a lot of students say,
yes, we feel as if we've accomplished something
once they've handed it in. But the experience
during the course of that two week window is
probably not as enjoyable. Now, we still continue
to observe, at least in the pilot that I led
in the School of Chemical Process Engineering,
there was still that divide. There were a lot
more students who felt that, you know, there's
a sense of achievement after handing in the report,
the video recording, as opposed to how much actually
felt that sense of passion, purpose during the
journey. So I think that's potentially going
forward is something that we have to continue
to look at and see how we could improve that
experience for our students. Thank you both.
That was a really excellent, comprehensive kind
of overview of how you drew upon the findings
from year one and embedded it into your authentic
assessment delivery in your own contexts. I wonder
what you'd say are the main sort of implications
for people who might be listening to this and
might be interested, hopefully, in embedding
this as an approach into their own work. Very
good question, Robert. So some... Probably reflections
from what we did we can actually share here.
I think the first point is focusing on assessment
delivery and the feasibility of that in their
assessment. So I think it would be really good
for teaching staff to understand how they can
incorporate this concept of delivery and provide
students with the opportunity to have feedback
through the journey, not just at the end. in
order to enhance the quality of the assessment
delivery. I think that was something that we
really enjoyed ourselves and also students positively
reacted to that because the journey, and as Manner
said, if they find the learning process pleasant
and if they learn something through the journey,
then the shift will be on the learning rather
than on the mark. will be much more engaged and
also the quality of the work as we experience
in our pilot will be increased. Another thing
which I can mention here is focus on creativity,
particularly for disciplines like engineering.
Creativity is very important because we are not
only dealing with problems that the final answer
is clear, is defined. We are working with open
-ended problems in engineering that particularly
in this environment in this world everything
is getting more complex and multidisciplinary
so the creativity is something that definitely
we need we need to improve that we need to help
our learners to have that creativity and so again
Manoj mentioned different kind of submission
formats I had the same thing in my pilot so I
actually helped students to walk through different
formats like poster the report also oral presentation
so I think these sort of opportunities give students
again room for creativity. There's a lot of literature
in this area. A couple of places people might
want to do some reading is around super complexity,
Barnett's concept, you're familiar. Also wicked
problems, I'm not sure who introduced that. Go
away and look it up. But that's a super interesting
concept as well. There are a lot of pedagogies.
Cradle. It's a Deakin University. Definitely.
There are a lot of pedagogies on that. Yeah,
so I mentioned the mark focus thing and probably
we'll speak about that a little bit more throughout
this podcast. But focus on mark is something
that sometimes completely hinders students from
developing that sort of creativity or the learning
because they are just... kind of targeting the
bare minimum and trying to get the mark and pass
some element of the whole module or element of
the module. And another thing which we face,
and I think it's very important, we need probably
to elaborate on that a little bit more, is the
institutional support through this journey. Because
it's not possible to do these things because
of the workload implications and time limitations
to do that. any sort of like an institutional
support any sort of like tools or I don't know
mechanism that can support educators in order
to go through this journey and provide more and
more kind of assessment delivery and flexible
assessment ideas would be really appreciated
I think fantastic we could spend a lot of time
talking about some of these wider implications
and I think you'd probably encourage people to
reach out to you to engage with your work What
sorts of research outputs have you produced?
And is there anything else that's in the works
that people can look out for? Yeah, thanks, Robert.
It's very much in the pipeline. We did take some
of these results to the CEFI European Engineering
Education Conference recently, where we ran a
workshop kind of using our assessment briefs
in a way, but challenging the delegates to think
of how would you embed authenticity if you had,
you know. full reins over how you're going to
design the assessment. So yeah, that conference
workshop paper should be online pretty soon.
But we've also working on a manuscript with specific
findings from these pilot studies, which hopefully
should see the light of public domain soon. I'm
sure it will. And we'll also include a link to
the show notes, a link in the show notes to Manoj
and Mohsen's LITE Fellowship profile, which
will have the snapshot with a summary of the
findings that we're talking about in this episode.
Should we move on to your findings from the staff
perspectives, which helped inform the pilot study?
Yes. So just to do a short recap, we did the
staff interview in order to also bring staff
perspective into the assessment, the authentic
assessment and flexible assessment ideas. In
terms of how we did that, we interviewed, we
had like a semi... structure interview interviewed
10 academic staff mostly in faculty of engineering
in different schools and different disciplines
and also we had stuff from different kind of
career level we had high level like management
level stuff like a director of student education
and also lecturers professors so that was really
good kind of diverse mix we had from different
perspective of staff at different levels How
we brought what we found from that was trying
to understand, are they reinforcing some of our
findings or not? One thing that was really highlighted
during the conversation was the focus on authentic
delivery again. And most of the stuff they mentioned
that they find sometimes the assessment process
is not actually reflecting the professional practices
completely. Something else which was really interesting
was because we are in the era of AI, artificial
intelligence, now they are everywhere. It's a
great era, I haven't heard anything about it.
So some of the staff, they mentioned that students
nowadays are very reliant actually on AI. There
are a lot of like work academy integrity issues
because of using AI as like a... black box just
producing some sort of mostly text and they found
that probably the assessment delivery would be
really good tool in order to reduce that over
reliance on AI because we are giving them this
opportunity to work on the assessment bit by
bit through the feedback that we provide and
I think something that I found myself in my modules
was that students will rely on AI when they are
not actually completely sure what they want to
do, how to do the work. Because in that case,
they see that they have limited time to deliver
the work and then they start to rush to do something
in order to get something from AI and submit
it. So if we give them the feedback, if we help
them to learn how to do the work step by step
and how to deliver the work, get feedback, improve
the work based on that, I think that would be...
something that enhances or reduces actually the
over -reliance on AI. Highlighting the implementation
barrier occurred a lot actually during the interviews.
A lot of stuff mentioned that it is really hard
because of the workload implications to bring
flexible assessment into their modules because
they need to sometimes design different rubrics
for that. They need to change the So it is not
actually an easy work. It requires a lot of time
and effort. And despite being really kind of
interested to do that, but time and workload
implication is something that they mentioned.
So that's probably linked to what I mentioned
before in terms of like institutional support.
This is something that we can probably we need
to focus on. Yeah, I completely agree. And just
a couple of points too. pretty much reinforce
what Mohsen has just said. So when we interviewed
staff, and these were semi -structured interviews,
the overarching themes for me personally on reflection
kind of centered around two main things. One
was to go from, we design assessments with specific
intended learning outcomes is what we're assessing
for, which is very much the correct approach.
But also to think of in, as Mohsen has just alluded
to, in the AI -based era, to kind of make that
shift potentially towards more a process -based
assessment and not just looking at the final
deliverable and the final outcome was very much
an emerging theme when we had these conversations
with staff. And it was about how do we make sure
that we, A, create the right environment for
students to work on tasks authentically, express
creativity, and that would mean setting up the
right environment. And then you kind of monitor
the process as they... work towards the deliverable.
So that is the first point to emphasize. And
the second one that staff also picked up a lot
on in the interviews that we did was sort of
this approach potentially to go more towards
holistic marking as opposed to reductionist marking
of saying, okay, you've got... that element of
the assessment done so you get a mark for that
kind of approach, which, again, a lot of us continue
to use that approach. And that is very much clear,
kind of that kind of approach when you explain
to students and when you set up the rubrics.
But that's probably where that element of training,
both from a staff training, but also a student
training perspective of how does holistic marking
work? And because it might not be necessarily
as clear as the conventional rubric. and marking
grading criteria, which is what staff and students
are more used to. So those are the kind of changes
where potentially we have to kind of envision
going forward. Great. And something I'd really
like listeners to learn more about, because it
wasn't just you interpreting the findings, was
people you worked with. Could you tell us a bit
more about that? Absolutely. That was one of
the most exciting things, the fellowship, wasn't
it? So, yeah, spoken about this a little bit
at some of the LITE events, but for people who
weren't there, the idea was we were sitting on
this data, the staff interview data that we've
just spoken about. And we as two academics pretty
much said, OK, so we're going to go through this,
apply effects of thematic analysis. But we wondered
if the richness of the data would not be fully
captured if we were just two members of academic
staff looking at staff interview data. So one
of the things that we wanted. to do was involve
student research assistants as core researchers
on the staff interviews. So we could tap into
that valuable student positionality when doing
the reflexive thematic analysis. So it's not
just a staff perspective of looking at things.
And we ended up getting two students who were
certainly far more competent than the two of
us and ended up doing a wonderful job of lending
that student positionality to looking at the
data. was interesting because we ended up analyzing
our data independently, of course, because we
didn't want to have any kind of bias or influencing
them and vice versa. Then when we got together
through a series of meetings, eventually sharing
our notes and seeing how we'd interpreted things,
it was really refreshing to see how those two
different perspectives on technically the same
data set, but I think that is... Something really,
truly unique to qualitative research and as people
who come from a very quantitative background
where it doesn't matter who's looking at a quantitative
data set, you can interpret it only one way,
but that's where qualitative data is very different.
It's been a very rich learning experience for
us as well. So yeah, a great experience and I
recommend anyone who's interested in pedagogic
research where you can embed that student perspective,
go for it. It's a richly rewarding experience.
I think whether you're a staff member considering
doing pedagogical research, possibly for a LITE
Fellowship, Look into possibilities for working
co -creatively with students. We have a team
of student research assistants. Equally, if you're
a student that happens to be listening to this
podcast, look out for opportunities to get involved
in this kind of research. I'm sure there are
some students who are really keen on our podcast
series. Maybe then you could share with any particular
elements of the findings that I think it was
Winnie and Asa were struck by. played a particular
role in you developing your findings? So something
that came out from that work was that authenticity
actually is about like underlying the skills.
So the development of the skills, which it's
really good because at University of Leeds, we
have very nice kind of approach towards skill
development. And we have this idea of skill matrix.
So I think if listeners probably can search about
skill metrics at the University of Leeds, that's
a really nice idea to learn about that. So I
think they believe that the focus should be on
underlying skills like critical thinking, problem
solving and reflection and this sort of stuff,
because these skills. will prepare learners for
the professional environment. When they are in
that environment, that sort of skill that they
learn through the journey of education will help
them to walk towards different problems. Also,
the flexibility is suggested as backbone of authentic
assessment because not only it supports students
through the creativity and working towards different
kind of submission form and this sort of stuff.
But also it's very inclusive approach. It is
very nice. It can actually cater for the requirement
of students from different backgrounds and different
needs. The crucial role of feedback, I think
we discussed this a lot and that is something
that again and again, even through the... analysis
carried out by Winnie and Asa was highlighted
and also they from their own feedback they found
that feedback from their own experience they
found that feedback would help them to improve
the work a lot. Something else which I can share
about that is some implications of the assessment
deadlines and assessment kind of arrangements
that probably needs to be looked at. from the
program level, not only module level, because
sometime in the program level, we have series
of modules and by adding more assessments in
order to make the process more flexible, we might
actually have a couple of like deadlines happening
at the same time or clashes between deadlines
or sometime the assessment fatigue might be part
of that. So I think the program level view of
the assessment delivery design is something that
we need to focus. when we are speaking about
that. So, obviously, training, again, as Manoj
mentioned and also as highlighted before, even
for stuff like for CPD sessions and this sort
of stuff, for stuff to learn how to implement
these ideas and effectively actually do the work
without putting a lot of time and effort on that
would be really good. And eventually the co -creation
part of that. I think if we can sit down with
students and listen to them in terms of their
perception about the assessment. Because something
that, again, was mentioned by Manoj and we found
during the first part of the research was that
the guidance needs to be clear for students.
But at the same time, that shouldn't be very
perspective to limit them and to put them in
a specific channel. So something like that, I
think it would be really nice. And our student
researchers, they mentioned that it would be
really useful to sit down with students to understand
how to actually design a specific guidance that
provides that sort of freedom, but also provides
a clear understanding about the expectation of
the work. Fantastic. Now, the next question I've
got, what sorts of changes would you like to
see going forwards? I feel as though listeners
might already have a good sense of what some
of those might be, which I think suggests that
you embedded that sort of impact thinking into
your research really effectively. What would
you say to that? I think I'd probably go back
to one of the top points that we've tried to
make through this podcast, but also in part one,
was the delivery aspect of authenticity. So it's
really creating that environment and thinking
about, not as an educator, how is the student
going to prove that? you know, they've met a
specific learning outcome, which is certainly
something that should be front and centre. But
then how will we be enabling students over that
journey of evidencing meeting the learning outcome?
And I think that requires a lot of conscious
thought and design, and it requires quite a lot
of effort as well, making sure that that scaffolding
is in place. And as Mohsen said, that delicate
balance of how do you foster creativity to the
sense that... Yeah, the guidelines shouldn't
be stifling them from potentially expressing
creativity. But at the same time, there should
be a reasonably clear set of expectations of
what the assessment is asking them. um to do
which was what our student research assistants
picked up as well when looking through that data
where you know it's easy for us to say you know
we want to pioneer flexible assessments and let's
give them a lot of choice but choice can be overwhelming
at times and uh and that's not where we want
to take our students uh necessarily right so
i think it is those things that we need to it's
a delicate trade -off on a number of those elements
um which requires as i said i think conscious
thought and design and something we touched on
earlier as well as Possible areas for further
research? Yeah, personally for me, reflecting
on the pilot and some of the things that I said
there, Robert, in terms of, yeah, things that
we potentially wanted to see further up, not
quite there, was this element of creativity.
I think one thing which has certainly worked
is the formative feedback. And in terms of what
next on that count, I would go. How do we scale
this up, particularly if you're dealing with
significantly large student cohorts, well in
excess of 100 students? How do you embed that
formative feedback loop multiple number of times
if you're the solo instructor? So that that is
very much the I think the obvious research question
on that basis. But the other one that I'd be
interested in to kind of tap into a lot more
is around creativity, because I think we. Sort
of undid the first element, which is we did not
want our guidelines to stifle creativity, as
I said, and our students certainly agreed with
that over the course of the pilot. They did say
that the guidelines did not restrict them from
expressing creativity. But just saying the guidelines
did not restrict them does not necessarily translate
into, yes, I could express creativity through
this assessment, because those two questions
did not necessarily receive the same sort of
commensurate. proportionate percentage responses.
So I think that's the sort of the next step in
the ladder, if you will, of, okay, yes, the guidelines
are not restricting them, but what more can we
do to kind of, you know, encourage them to actively
be more creative? So those are sort of personally
for me, what's next in life. Very good points
mentioned by Manoj. But on top of that, I'm personally
interested in... some initiatives like automatic
kind of feedback formative feedback systems if
we can implement that in high quality level in
order to provide constructive and feedback the
student can work based on that in order to improve
their work so in this case instead of like reliance
on AI they might be much more interested to submit
the piece of work get feedback and at the same
time the implications of additional workload
for big cohorts particularly can be somehow resolved
there are a lot of like nice works that we saw
actually when we were in the conference in the
CEFI conference in Finland people are doing an
automatic kind of feedback methods which can
be looked at I think as a solution for these
so I believe there are a lot of things that can
be done in order to enhance the assessment experience
for students and make this very rich assessment
journey. And probably we mentioned a few of them.
People might end up with much richer ideas in
that sense. So last but not least, perhaps my
favourite question. Do you have any further advice
in addition to the advice you gave this time
last year? I don't know if you can remember what
advice you gave. do you have any additional advice
for others conducting or thinking of conducting
pedagogical research scholarship of teaching
and learning that's a difficult question to finish
the podcast isn't it yeah I'd probably go just
being one of the things in general with research
is yeah is that You have to trust your ability.
It is uncharted territory and you'd have to be
exposed to potentially new research methodologies.
It's very much been the case with us for this
fellowship project. But it is just embracing
that uncertainty. And, well, I think we have
to be thankful that here at the University of
Leeds we have a good support infrastructure for
pedagogic research. I think it's leveraging all
of those different support opportunities and,
yeah, continuing to... to trust in your abilities
and being driven by that passion of all of us
shared this vision of wanting to improve the
student learning experience. And I think if we
continue to draw inspiration from that, we should
be very successful pedagogic researchers as well.
I remember during the first episode, actually,
I said I would not give any advice. I give suggestions,
but this time I will give one single advice.
I think my single advice would be for people
thinking about pedagogical research. please if
you are interested do that this is really nice
journey particularly for my engineering fellows
because in engineering field people usually look
at that as something very strange something very
new they are not interested but through this
journey we found that really nice and very providing
us with very deep kind of insight in different
things and also it was really nice to have network
with other kind of engineering fields in other
settings around the globe that they are working
on the same things they are actually facing the
same challenges so that's my single advice get
into this pedagogical research kind of field
particularly engineering field because we need
that we need actually to improve the assessment
particularly in this kind of over complex world
to provide students with the skills that they
need for professional practices I wonder if there's
anything particular about sort of your experience
working with students in the research process
that hopefully this is something that listeners
will be considering doing. What was that process
like? I know you spoke about that at a LITE
event recently. Yeah. I think you did. Yeah,
I certainly did. Yeah, I was sort of scratching
my head as to what I said there. No, I think
one of the things with working with the student
research assistants, I think is it's a really
rewarding experience because. A, you do not know
what the outcome is, what the output is necessarily
going to be, but it is ensuring that you provide
reasonably good structure to the research process,
which in a way, if I had to draw a parallel,
it works reasonably well to co -creation of assessments,
for instance, which is not a research exercise
on its own. But you're very much there saying
there's a fair degree of scaffolding in terms
of where you potentially... where, you know,
just direction of travel. But the eventual destination
is a little bit flexible there in terms of the
student co -creation element. So, yeah, I would
say it was something that we'd not envisioned
at the start of our LITE Fellowship that we
would. We certainly wanted to do the staff interviews,
but in terms of the analysis, we hadn't thought
that far in terms of we would be lacking the
student positionality. It was something that
as we sort of gained exposure to reflexive thematic
analysis that... that kind of entered our, yeah,
our sort of thought space and we took it from
there. So I would certainly, I think the advice
there, well, again, don't think I'm necessarily
qualified enough to be giving advice for people
to be doing pedagogic research. But yeah, it
would just be being open to ideas and exploring
those possibilities. And I think students do
provide a great perspective that we easily miss.
Yeah. Thumbs up from us then. And you're not
giving any more advice, you've said that. The
theme of sort of like there's uncertainty in
pedagogical research. You're not necessarily
using methods or theories that you are familiar
with. You might be working with students in a
new way. But that is also part of the learning
and part of the research. And I think that's
a really interesting, it's something that comes
up a lot in these episodes. Well, all that remains
to say is thanks again for joining us. Thank
you very much for hosting us, Robert. Happy year
two. The fellowship went well. And don't worry,
you won't miss us too much. I'm sure we'll be
inviting you back to future events to talk about
your fellowship and your experiences. Great,
I'll write that down. Thanks for listening, everybody.
Do check out the show notes for further links
to the research. And we'll see you next time.
Thank you so much for having us. Thank you very
much.