Welcome to Light Bites, an occasional podcast from Leeds Institute for Teaching Excellence
at the University of Leeds.
Episodes will be hosted by members of the LITE team.
And we'll be showcasing the scholarship of teaching and learning from across the university.
Hello and welcome to another episode of the LITE Bites podcast.
It's Emma Peasland here.
I'm research and impact officer in LITE at the University of Leeds.
And today I'm delighted to be joined by Jianni Liu, who is a lecturer in English for academic
purposes at the university.
But Jianni's going to share with us some insights from her LITE Fellowship, which was called
Assessment Literacy and Student Success and looked at how embedded academic skills provision
influence students assessment literacy.
But you had a different job then, Jianni.
So when I was doing my LITE Fellowship, I was based in the amazing library learning
development team.
The team itself basically has the two main strengths of the first one is supporting or
developing students academic literacies in the curriculum.
And that's what my project was focusing on.
And the other strength, I suppose, is more well known to some people at Leeds, which
delivers the amazing skills at library service, which is like an a co-curricular academic
skills service.
Great.
Well, it's great to have you along today.
So thanks very much for coming along.
Thank you.
Yeah, I'm very glad to be here.
Hello, it's Emma here.
I just wanted to pause before I continue my conversation with Jianni to just clarify and
explain a couple of terms that come up in the conversation for anyone who's listening
who doesn't work at the University of Leeds.
So in the conversation, Jianni refers to Minerva and Minerva is our VLE or virtual learning
environment at the university.
We also discuss or mention the curriculum redefined program.
And that's a big 10 year project that's happening at the university right now, which is working
to review and refresh all of the undergraduate and taught postgraduate programs with the
aim of ensuring that the curriculum prepares students to be global citizens who are ready
for their kind of future employment.
And then finally, we refer to discovery modules in our conversation.
And discovery modules are those modules which students can choose to take perhaps outside
of their discipline to broaden their academic experience and enhance their employability.
Back to the conversation.
Well, let's get started talking about your fellowship a little bit then.
So I wondered if you wanted to start off by telling us a little bit about your fellowship
and what it was you wanted to investigate and why.
Yeah.
So when I was learning the why it's based in the library, I spent I think up until I
was doing my lab project, I was like there for at least seven years and I spent the majority
of my time designing, embedding and then delivering academic skills, workshops, lectures, seminars
and webinars during the pandemic.
Supporting students development on topics such as like essay writings or planning for
essay structuring, critical thinking, presentation skills, time management, all sorts of like
academic literacy skills that would in my head benefit students preparation for assignments.
However, when I was looking into literature and exploring the link between our work and
assessment or impact on students preparedness for assessment, there wasn't much in the field
directly looking at academic skills and there was loads of discussions around assessment
literacy but embedded into the context of their study.
So I was quite, I guess in my head curious to find out whether our work would have any
impact on students assessment literacy development or not and if so, how and how would students
perceive that development be similar or different from staff's perception?
Interesting.
So would you tell us a little bit about how you went about investigating this question?
As in the methods, the approaches I've taken.
I think at first I was just like spending some time reading around the topic of academic
literacy, there's plenty of research on that but also assessment literacy and looking at
how these two marry or the crossover, the shared approaches, models and theories and
then I, because my work at that time was predominantly based in the school of languages, cultures
and societies and I work with lecturers and module leads and programme leads in LCS undergraduate
and postgraduate programmes to deliver academic skills trainings basically.
So I've selected two undergraduate modules to investigate in more details, one from the
Department of Spanish and Latin American Studies, so it was like first year Spanish research
skills module where I and the module lead have designed like a series of scaffolding
or like a series of workshops and online resources and formative assessment to develop student
assessment literacy in a more structured way and I've also selected like another higher
level, level three module where I deliver, I suppose co-deliver because the module lead
has always been there and sharing his perspective as well as the marker, so I co-deliver and
assess my literacy workshop where students work in groups and work with their peers to
assess exemplars, so I've picked up those two examples from my work and then I conducted
surveys and focus groups with students.
I've also interviewed staff, so there's like a mixed method approach to compare the perspectives,
the stories, the experience of students with staff, have perceived their experience being
through the journey of their modules, so it's a mixed method approach I suppose.
And so how many kind of participants did you have taking part in that from those two modules
that you chose to work with?
So I had four student focus groups with nine students participated and I had a small survey
where like 18 students fill in the questionnaire, but I also had like four staff interviews.
Great, so you got that kind of oversight from several students in the module in the survey
and then a little bit more of a kind of a richer and deeper insight into their perceptions
and experiences from those focus groups and the interviews from both the staff and student
side.
Yeah and I think the good thing about having the survey and some student focus groups was
that when I look at the data and I realised I had quite a good even spread of different
student levels, so I had like pretty much equal amount of students from level one and
level three participating in both the survey and the focus groups and from both modules
as well because those modules are one I think I said from the Department of Spanish Studies
and the other one I should have said it's from the Department of East Asian Studies,
so it's a level three Buddhism module.
Great and so you want to share a little bit about what it was that you found out from
the data you collected?
Yeah, so students say that the workshops they were part of, you know where they work with
their peers and with me facilitating their discussions around analysing the student examples
from previous years of the same module where they then attempt to mark those examples against
the marking criteria but also with the module lead to give them some feedback.
So they say that there's those type of workshops they found most useful to develop their understanding
and awareness of how they're going to be assessed by the module, so basically you know the standards,
the process, the criteria of their particular assignment and I should have said the assignment
I was investigating was the literature review for both modules actually.
So the students found the workshops most useful to develop their assessment literacy basically.
They've also found examples that we uploaded in Minerva with staff's annotation useful
to look at to a certain degree because they also said that without hearing staff's feedback
of how these different criteria being met on the examples it's sometimes quite difficult
to get their head around.
They said they would use the marking criteria after the workshop as a result to self-evaluate
their own submission or their own summative assessment, however the marking criteria on
their own in Minerva are not very useful for them to get their head around the standards
and expectations of assessment which was in a way different from staff's perception because
there were staff saying that they felt like the marking criteria being available in Minerva
would be good enough for students to check on the standards and expectations of this
particular assignment.
Another key finding of this project was that students were asking for more contact time
on assessment support and feedback opportunities.
They were asking for more academic skills workshops and in-class discussions around
assessment around that skills development, however staff were reporting a real tension
between having to balance, you know, there's just so much content to get through in a,
well you know even though we have 11 weeks for modules at the standard for example in
semester one but then you always have the introductory lecture and then you know so
10 hours on average is not enough for them to cover all the content so they have, so
staff were reporting a real tension between having to give up their content time to skills
time.
So it's interesting that staff think that when you give the students the marking criteria
that's a kind of a clear message about what needs to be included and what a great assignment
should look like and that students didn't necessarily find that particularly kind of
insightful or useful but they did when they were, staff's annotations were added.
So how would you recommend that people perhaps change to marking criteria or added to marking
criteria to support students with their assessment literacy in that space?
Yeah, interestingly I was at an assessment literacy workshop that Pam Britto delivered
at the ESSEC at student education conference and we were having a similar discussion and
some ideas we came up with was and again you know it's not just my ideas, right, literature
is already suggest many many different approaches and people at least been doing some great
work so some ideas were like we could have a more consistent approach with the language
that we're using in the marking criteria and I know that it's not easy job you know
it's very difficult to articulate the difference between the first critical analysis or exceptional
critical analysis to a good amount of critical analysis but students would find some
examples be really beneficial so there were some suggestions of breaking some examples
down and then actually insert some where the marking criteria would be.
This in practice may now be practical for certain type of assessment and especially
potentially making it hard to actually be of standard practice across the board if you
have loads of different types of assessment.
I think students were generally saying they would find contact time the most beneficial
so I think as a starting point instead of looking at the marking criteria and thinking
about breaking some examples and inserting them into that which could actually be more
amount of work thinking about your staff concern on workload in general I would suggest finding
at least like a 50 minute slot you know like a standard seminar slot to have a go at delivering
some peer assessment workshops where students could just have a look at some extracts from
the same assignment preferably you know previous student summations to the same module and
to actually have a go at marking.
It's a fairly safe activities been out there again you know in the world in the assessment
literacy world for 15 years and people's been done doing it in various different ways at
least as well I think I would if he was not thinking about exploring this option they
could always speak to either their learning advisor from the learning development team
or they may have some colleagues that are in a department have already done it or they
could work together with EAP lectures from the Longree Centre so I think there's various
collaborative opportunities as well available so yeah just have a go.
So there's lots of support out there.
So we've kind of moved into starting to think about what are the kind of practical insights
from your findings then so I'm hearing perhaps from what you've just said is that there are
things that could be done to help make the assessment criteria more understandable for
students but actually time spent working on a peer marking exercise or actually thinking
about the assessment criteria in contact time is more valuable than that from what the students
in your study said is that right?
Yeah yeah there were real desire for more contact time on skills and literacy development.
Great and so we've talked about then how you could change marking criteria that more contact
time would be great were there any other kind of practical insights or implications for
practice that you might be recommending?
I think with curriculum redefined people have been talking about like program level assessment
and learning outcomes etc etc and I think one thing I would recommend and I think a
lot of colleagues have already been saying that they've been doing that in their curriculum
redefined can I just call it CR from now?
Yeah in their CR spring workshops it's mapping out the different types of assessment at a
different points where students encounter them and actually have an overview of what
students are experiencing their trajectory of assessment literacy development for the
program and then actually mapping out the points of where peer assessment workshops
or some assessment like and feedback sessions delivered by the program team will be useful
to students overall or where a more module specific approach can be taken you know if
there's like for example I think some modules are now using podcasting as the type of assessment
and they may not be like in a common genre of assessment out there so then some module
specific support sessions on how to develop your podcasting but also what are the skills
outcomes that you're assessing students and how to achieve that would be quite useful
I think otherwise just for staff to take more of an overview and awareness of where students
are encounter different types of assessment and how do you actually scaffold development
would be quite beneficial because especially the third year students were saying they found
those discussions on literature review so valuable because they would be you know expecting
of doing one for their bigger final year project and if they didn't take this module because
the module is actually discovery if they didn't select this module they wouldn't have had
it like a goal at doing a literature review until the very high stake final year project
and they were saying they felt really nervous about that before but because of this module
they had a goal they felt like oh actually I know kind of now how I do it but then you
kind of think all right you know so the last two years at least didn't really prepare you
for this type of assessment until your final year project that's that is quite you know
I think in students mind that's what they felt was they felt really overwhelmed and
they also felt really they felt at risk.
Yeah yeah gosh yeah because that really does it is high stakes isn't it yeah final year
dissertation yeah kind of waiting in your in demand for that yeah.
I think one student used the word terrifying she was saying like it was honestly so terrifying
and yeah I was looking at the some I was trying to find some course the other day in my Envivo
library and I found that it was terrifying but then she found a discussion around the
example so valuable and then she actually achieved a first and then she felt it must
be working that her way of writing must be working because he got us first and now she
felt like she was better you know she was like more ready against her final year project.
And so this might be difficult to do but if you had to pick out a kind of one key finding
or recommendation to share with people what would you choose?
I think my key finding or piece of advice would be let's don't take student experience
of assessment for granted there's a lot of like hidden curriculum assumptions out there
that we need to really unpack and prepare students for as staff and some I'm really
quite guilty of that myself so just speaking from my own experience having no work at least
for now into the 10th academic years there are a lot of assumptions that we would or
we could make of our students of their final years they would know what critical analysis
look like in a literature review right but if they haven't they've never done it before
and some of the students who actually from STEM disciplines and they were taking a discovery
module in us they never experienced this way of being assessed of course they wouldn't
have developed that awareness that understanding of what we are expecting of them and I know
that again you know students different pathways and portfolios and discovery modules is another
thing to worry about but at least I think within a department where people have control
over the trajectory of their students academic literacy or skills development a more crafted
approach a more scaffolded approach would be really beneficial rather than leaving students
to explore and somehow figure it out on their own to muddle through kind of yeah great so
slightly different kind of change in tact now I'm wondering is there anything that you
might do differently if you're going to start your fellowship over I think now having done
it right I felt like I knew what I was doing however I spent so much time obviously not
knowing what I was doing or doubting myself of what I was doing so I think I spent too
much time being in the low comfort low confident zone of just being very guilty of imposter
syndrome which I think perhaps coming from a more from a practitioner background and
then also obviously when I did a fellowship when I was based in a professional service
setting would be quite natural or could be quite common for colleagues who share similar
experience and backgrounds so you know I've been recently talking to again colleagues
at SAC and then I was trying to bring this part of experience to them I think you know
just go for it this fantastic support available from LITE from Emma from Dave when he was
here and from everyone and just always appear there to support your ideas and to actually
help you navigate the fellowship experience so don't struggle on your own yeah
that's what we're here for to support fellows along the way so yeah well and I'm glad that
we could do that with you it was a pleasure working with you on this fellowship and then
I'm also curious to understand how the fellowship was for you so did you experience any personal
benefits from doing your fellowship I think doing this fellowship and also it is my first
time of doing a proper I would consider a big piece of research project on my own you
know makes me realize how much I actually enjoy scholarship and I wanted to go a bit
further with it and you know there was just an opportunity that came up at least that
would allow this and also making my scholarship and would have done what I have done in education
being recognized in the teaching and scholarship contract or role anyway so I moved on to become
a lecturer now and scholarship is now part of my portfolio and I have decided that I
will keep going with assessment literacy that's what I've said to people in my interview
and that's what I promised what I would do in a new role so I'm currently exploring again
trying to find a different angle to this topic because I'm in the field of like language
language assessment and language teaching but also with the rise of gen AI the existence
of gen AI we were just thinking about how we would develop and best support students
AI literacy but also with unfortunately certain type of assessment you can now you can't
actually students are not allowed to use gen AI so it's how we actually navigate this together
with students.
Yeah oh well we'll look forward to hearing about the findings of your next project then
when you get to that space of being able to share it.
So Jan it's been a pleasure to have this conversation with you today it's been great working with
you over the fellowship and seeing how it's developed and how you've been able to share
your research findings at various conferences and things like that so thanks for coming
along today and we'll look forward to seeing what comes next.
And thanks for having me and just thank you so so much to Emma and also to everyone at
LITE for nurturing me through my fellowship and actually beyond I always I still feel
like I'm part of LITE even though I'm kind of like done and gone.
Always a LITE fellow.
Excellent that's very kind of you to say thank you very much.
Mileage of