Hello and welcome to a special episode to conclude
season one of our Methods mini -series at LITE.
Politics, lifestyle, entertainment, history,
you name it. There's a podcast Q &A for it, so
we thought, why not us too? I'm Robert Averies
and I've recruited the help of some members of
the LITEResearch team to answer your questions.
Thank you for submitting your questions. Let's
start with the first one, which is for Izzy.
Thank you, Robert. So when each participant is
a co -researcher and co -author, it's important
to begin to develop ethical protocols together
with all participants from the outset. This includes
things like informed consent, confidentiality
and data privacy. It's important that you maintain
an ongoing dialogue with the participants you
have as co -researchers. To include them as co
-researchers, their voices need to be considered
of equal value and they must be involved in decision
making at all stages of the project. Setting
time aside for formal meetings, regular check
-ins can help facilitate this. Other things you
may want to consider are the different views,
backgrounds, knowledge of each participant in
the group and what this may bring and how this
could influence the research and the group members.
For example, there may be power dynamics that
you need to navigate and overcome. It may also
be important to consider epistemologies and positionality
of each of the researchers. In relation to the
epistemologies that collaboration involves, it's
useful to remember the reasons why we're collaborating
and inviting people into these projects. There
are good epistemic reasons to invite collaborators
into our projects. They bring new perspectives
and fresh eyes. challenge myths or preconceptions
we might have. So ethically considering these
epistemic consequences of collaboration should
be as much about actively pursuing these epistemic
reasons that drive us to collaborate in the first
place as it is about minimizing harms. Yeah,
I agree there, Matthew. I think that's really
important. And I think it speaks to the wider
point that, and this has been occurring throughout
this series, methods that you use and ethical
considerations. are so closely connected and
woven together. And it's not just about emphasising
them and making sure they fit, but it's about
making sure one informs and improves the other.
Fantastic. Brilliant. We'll move on to the second
question, which I believe you're going to be
answering, Lucy, which is how do I create a safe
space for participants? Yeah, I think this is
quite a broad question. So I think it's worth
breaking down. But it is quite relevant at the
minute because just this week, some institutional
priorities for inclusive inclusivity and accessibility
have been published and one of them is create
psychologically safe spaces so it's it's quite
important at the university at the minute This
is defined as creating learning spaces where
students feel safe to be themselves during classroom
conversations. But this question, I think, is
referring to participatory research, which can
be defined as the practice of establishing ground
rules and guidelines for conversations and behaviours,
which it's useful to think about that in relation
to broader inclusive by design arguments and
inclusive environments more generally. I think
it's just good to flag that there are tensions
with this term, safe spaces. I think that we
might argue that space is never truly safe because
it's complex. It's not a neutral kind of container.
There's a lot of negotiable, contested relations
that build up space. And naturally, that is going
to lead to conflict, right? Conflict is okay
in situations like participatory research if
managed with care. It actually means that new
shared meanings can be developed and new ways
of being can emerge. I think one of our fellows,
Barbara Kempf, has kind of... acknowledge this
in her work she's got a padlet on her fellowship
about group work where she highlighted resources
on managing conflict and advice to reframe conflict
and normalise it as something in research I think
that in this case it's okay to say safe space
I would maybe advise aiming for safer space or
braver space because actually the binary of safe
and unsafe is probably unhelpful um we can't
guarantee it as researchers uh we we can aim
towards it but in promising safety that there's
a bit of an ethical consideration to be made
there however it's still important to maybe aim
towards safer spaces so i think A good place
to start is maybe building trust. That's kind
of the big thing to aim towards. It's kind of
looking at the vulnerability that comes from
situations where participants are sharing parts
of themselves for the research and harnessing
that. So without overstepping. So I'd probably
take story circles as a good example of this.
participatory method where a small group creates
space for sharing and learning from different
people's perspectives and one of our previous
fellows Jenna Isherwood she has a toolkit about
this and she goes into principles that create
a safer space in my opinion a lot of them align
with the things that Izzy was saying before that's
just good ethical practice but one of the main
things that she did and I'd probably recommend
as a starting point is to maybe co -create guidelines
with participants at the start of the research
whatever that method may be so this might look
like speaking from your own experience participating
at your own comfort level listening for understanding
and not judging and maybe making that or allowing
that to be recorded in a way that participants
can access it throughout the exercise, whatever
method it may be. And importantly, it's up to
the participant to hold themselves accountable,
but also everybody else in the room. So I would
say that. Thank you. A couple of quick reflections
from me, Lucy. I really like that you... You
tried to normalise that it doesn't have to be
considered, it can't be considered fully safe.
It's about being safer and addressing those tensions
and being transparent with yourself and with
participants. I also like that you mentioned
story circles because that's a nice synergy with
the first episode from this series. What you
were saying about creating trust with participants
or with students really resonates with what Rachel
was talking about in her... episode but also
in her on reflective blogs but also her wider
fellowship on building trust with students in
reverse mentoring and more generally in staff
student partnerships and there are loads of ways
you can do that I'd really recommend listeners
check out Rachel's work for ways of building
that rapport with students whether that's in
a piece of pedagogical research as we're discussing
today or more generally when working in partnership
yeah I mean I think it's It's important to also
acknowledge that there is an intersectional element
as well. And for some people, achieving safety
is going to be a lot different depending on where
you're from, background, that kind of thing.
But actually, the facilitator has quite an important
role there in managing the guidelines and making
sure that people feel comfortable. Okay, thank
you very much. Let's move on to the next question.
And this is a really, really interesting and
relevant question that we've been talking about
a lot recently. The question is, I'm about to
conduct some pedagogical research and plan to
gather data by an in -person workshop. But I'd
like to expand my data set with a broader survey.
What generalisation factors do I need to consider?
And Matthew, I believe you're going to pick up
this answer. Thanks, Robert. So I think this
draws together some of what we've already been
talking about in these questions. Generalisability
is how well our current research equips us to
talk about new situations where some part of
the experiment or circumstances have changed.
We can think about this in terms of how representative
different aspects of the experiment are of some
broader group and what might change when the
situation changes. One approach to this is to
break it down into questions about the setting,
the treatment, the sample and the measure. The
setting is the context. context the experiment
is happening in and all the extraneous factors
that might be affecting our results. In this
case they're both happening in the same year
at the same university perhaps at different times
of the year and one online and one in person.
So we might expect very broad external factors
are probably relatively similar but timing in
the academic year and online in person might
affect the students responses. The treatment
is what we actually do in the method. In this
case What does the workshop or the survey offer
in terms of the data? Will the workshop make
people feel more comfortable? Or will the lack
of anonymity affect the responses I get, which
the survey might preserve? So for some topics,
anonymity is going to matter a lot, while for
others, the timescale to build trust will matter
much more. So the sample asks how we might understand
the population of respondents and the ways these
may or may not matter for the questions we're
interested in. Are these populations representative
of other... broader populations, all their responses
represent something a bit more unique or idiosyncratic.
While numbers of responses can offer some security
here, it's important to still be careful about
asking these questions because not all effects
are going to be distributed in a way that's easily
addressed by simply increasing the sample size.
Measurement generalizability tends to look at
the type of data you've gathered and the ways
that data gets packaged or coded. Here we should
be asking if you've got good reasons to generalize
from like a high Likert score in our survey to
a positive discussion in a workshop. How neatly
do these different types of measure capture the
same underlying phenomenon? So in terms of what
this looks like in practice, we look at our research
through each of these different lenses and then
do good transparent research as we would whenever
any other questions come up. What can we find
in the literature about the relevant effects
here? In what ways is this research tapping into
something being representative of broader effects?
And then give evidence to justify our levels
of confidence in the generalisations we draw
when we arrive at conclusions from our data.
So we find data about the effects of timing in
the academic year, the importance of trust building
in workshops, the effect that having, say, mostly
first years respond as participants, and the
reliability of, say, Likert scores relative to
a long -form discussion. and we justify some
of the generalisation moves we make. In all these
cases, it should be emphasised that what we're
doing is making these moves explicit so that
we can see our research as part of an ongoing,
broader conversation. Thinking about what will
or might or should matter in other contexts will
help us respond to the fact that our classrooms
are not universal, they're not homogenous, they're
not always predictable, and they're part of a
global learning experience. Thanks very much
for that answer, Matthew. And I think those four
lenses are a really helpful way to take a step
back and reflect on why you're expanding the
data set. My second reflection is towards the
end when you were talking about the wider context
of your research and that it's not about seeing
your research in isolation, it's part of a wider
conversation. a particular place, a particular
moment in time. That made me think of, and this
is going back a little bit, but to Paul Ashwin's
keynote in 2020, Student Education Conference
2024, when he spoke about these collective bodies
of knowledge and trying to see yourself as someone
playing a really important educational role,
but an educational role that is embedded within
wider communities of practice. So that, for me,
is a really helpful reminder. to place your research
in that wider context, which we obviously do
when we think about literature, we think about
what the issue is, but on an ongoing basis, as
you continue to carry out your research, to keep
that at the forefront of your mind? Absolutely.
It also ties together some of the community considerations
we have when we're conducting research and the
good epistemology. really considering those questions
seriously and like they matter. They're bigger
than just what will happen in my classroom this
year. It's asking something about how this is
tying into something larger that's going to affect
the students for a lot longer, that's going to
affect the life of the university for much longer
as well. Great. And further to that, Matthew,
we recently held a workshop with New Light Fellows
where we explored both generalisability and reflexivity.
Could you tell listeners a little bit about how
you consider the two to work in tandem? Is that
the right way of putting it? Sort of, yeah. I
think they're closely related concepts or approaches
in research. I think they're sometimes seen as
being in tension because the more reflexive you
are in your practice, the more you're considering
yourself a participant in that research, the
more involved you are in it, the less someone
else is able to come in and sort of go, OK, I
can pick up your piece of research and stretch
it to this new circumstance or understand it
in this new context. I think that's really a
benefit of thinking about generalizability in
terms of these lenses. It means we can then start
asking, OK, but are we really getting at an underlying
phenomenon here, which might only be accessible
if I'm being properly reflexive in my practice?
If that's the case, then what we might say is
that actually it's going to be more generalizable
by being reflexive. And what we then encourage
is that if someone else is trying to do something
that's going to generalise this piece of research
to a new context, that we might encourage them
to take up that reflexive practice in their own
approach, because that's where the method is
going to link into the underlying phenomenon
that we're trying to get at. Brilliant. That's
a really nice way of getting us to think about...
these two things reflexivity and generalizability
that are often seen as at odds with one another
but actually to take a step back and consider
talking about tensions earlier Lucy and how they
might interact in a more positive sense and probably
should probably give a little bit of background
briefly to reflexivity this is something that
came up in a couple of our previous episodes
and Lucy you've created a resource on it Yeah,
I created a resource just to help researchers
really understand more about what reflexivity
is and kind of just to think about themselves
and what they bring to the research more centrally
in the resource I define. reflexivity is the
continuous collaborative multifaceted processes
through which the researcher self -consciously
evaluates how their subjectivity and their context
influence the research so it takes a position
of subjectivity being inherently involved and
I can see how that's a big consideration when
we're thinking about generalizability as well
but yeah my research it's kind of a brief overview
of what reflexivity is it goes into how it differs
from reflection and there's also like thinking
prompts and different types of reflexivity such
as methodological reflexivity and ethical reflexivity
for people to consider and then there's also
some practical steps to actually use resources
to help you embed it within your day -to -day
because I think that's really important rather
than just sort of thinking about it after the
research project has happened starting from the
beginning and making it kind of a thorough and
pervasive feature of all the research. Fantastic
thank you and we'll provide links to that resource
and any other resources we draw upon today in
the show notes. Fantastic. Let's move on to our
final question then, and we're circling back
round to Izzy. Are there any methods or advice
around interviewing pre -university students?
Are there any resources created by others who
have done this? Thank you, Robert. So the first
place to start will be to navigate the ethical
issues surrounding this age group. I know we've
talked about ethics a bit already in this podcast,
but here the students are much more likely to
be under the age of 18, and therefore you'll
need to... have a lot more considerations they
will be considered potentially a more vulnerable
group you'll need to gain consent not only from
the participant themselves but from a parent
or guardian and there may be some safeguarding
protocols that you may need to need to adhere
to as well as all the usual ethical considerations
anonymity the power dynamics data use etc so
it's really important that you spend lots of
time going through that ethics gaining that ethical
approval as the first step In terms of considerations
around methods, certainly more creative methods
can work well with this age group, such as interviewing
using photo elicitation, drawings, crafting,
constructing with Lego. There's resources on
the internet around Lego serious play that you
could incorporate to make the interview more
engaging and less intimidating. You may also
wish to consider how you can make the students
feel more comfortable. Obviously, the creative
methods is one way. You might want to interview
in pairs or small focus groups. This can be particularly
helpful in some circumstances if students already
know each other. When planning the interview
protocol and the questions, you'll consider the
language materials carefully. You don't want
to have anything overly academic or abstract.
Really try to speak in their terms. It could
be helpful if you could speak to maybe their
teachers, run some ideas by them, gain ideas
and advice that may help facilitate these things.
A discussion with the teacher or someone that
knows them better could give you an idea of what
would and wouldn't be suitable in certain situations.
In terms of resources, there isn't specifically
anything that's been created by others from light,
but there's a couple of websites we'll link in
the show notes. that you may be interested in
browsing. So we've got the ERIC project, which
is ethical research involving children. Further
links for the university's website at Leeds,
which has got a lot of different links around
research integrity. And then in terms of methods,
we'll link up the National Centre for Research
Methods, which has got lots of videos and advice
on creative participatory methods with young
people specifically. Thank you very much for
that really comprehensive overview, Izzy. Really,
really helpful, I'm sure, to listeners. To add
to the first section of your answer, talking
about informed consent, and this is an additional
link I can share in the show notes, some Gov
.uk guidance on firstly informed consent more
generally, but then also in particular relation
to working with under -18s. And what struck me
was that how we think about informed consent
on a basic level is the same whether we're talking
about... under 18s or adults, but then there
are specific considerations, ways of adapting
the core tenets of informed consent to under
18s, for example. You would always ask in a consent
form whether your recruitment materials and information
documents make the reason for the project and
the reason for the participant to be invited
to the research clear and digestible, and there
are particular considerations you want to...
you'll want to bear in mind when you're aiming
that towards under 18s. So using potentially
pictures or digestible language. And again, in
any informed consent process, you'd want to ensure
that participants know they can withdraw. Whereas
with under 18s, you might want to check in more
regularly than you would with an adult to ensure
that they know they can withdraw during the research
process, where those power dynamics may prevent
that from occurring. So there are those core
tenets, as I say, that... need to be applied
and adapted to under 18s but then there are also
additional safeguards like you say with gatekeepers
guardians and also ensuring the consent form
is written not for the person who's potentially
signing the document. Yeah and I think it's really
important that although you have to gain some
consent from a guardian or a parent or someone
it's like you said it's always important to consider
that the actual participant is the student or
the young person and they need to have all those
rights as well. Brilliant. Thank you very much.
And thanks, Lucy, Izzy, Matthew. We've reached
the end of our methods Q &A. And thank you, listeners,
for following our methods miniseries. We hope
you enjoyed it and learned something new about
pedagogical research methods. We're back with
season two at some point. I said it. In the meantime,
hit subscribe to ensure you don't miss out on
upcoming Light Bites or Enlightening Dialogues
episodes. Please send any additional pun -related
ideas to the Light inbox. Take care and see you
next time.