WEBVTT 00:00:08.490 --> 00:00:11.169 Welcome to Light Bites, an occasional podcast 00:00:11.169 --> 00:00:13.269 from Leeds Institute for Teaching Excellence 00:00:13.269 --> 00:00:16.269 at the University of Leeds. Episodes will be 00:00:16.269 --> 00:00:18.550 hosted by members of the LITE team. And we'll 00:00:18.550 --> 00:00:20.370 be showcasing the scholarship of teaching and 00:00:20.370 --> 00:00:28.870 learning from across the university. Hello and 00:00:28.870 --> 00:00:31.390 welcome to LITE Bites. I'm Robert Averies and 00:00:31.390 --> 00:00:33.810 I'm joined today by Alba del Pozo. Hello. 00:00:34.329 --> 00:00:37.070 To talk about her LITE Fellowship on formative 00:00:37.070 --> 00:00:40.630 assessment. I'm also joined by LITE's Izzy Harvey, 00:00:40.829 --> 00:00:42.590 who's developing her own long-term research 00:00:42.590 --> 00:00:45.649 in learner-focused feedback. How are you both 00:00:45.649 --> 00:00:48.590 today? We're good. Thank you, Robert. Very well, 00:00:48.590 --> 00:00:50.630 thank you. Excited to be on LITE Bites? Always. 00:00:51.469 --> 00:00:54.210 This is your first time. I know, but there might 00:00:54.210 --> 00:00:59.170 be more. Okay. We'll hold you to that. So, Alba, 00:00:59.189 --> 00:01:00.829 could you tell listeners a bit more about what 00:01:00.829 --> 00:01:04.150 you do at the University of Leeds? Yes. So, I 00:01:04.150 --> 00:01:08.109 mean, I have many hats, starting with I'm a lecturer 00:01:08.109 --> 00:01:11.730 of Spanish language at the School of Language, 00:01:11.829 --> 00:01:17.129 Cultures and Societies, but I'm also the Incubator 00:01:17.129 --> 00:01:21.750 Lead for Feedback at LITE. I just finished my 00:01:21.750 --> 00:01:24.670 LITE fellowship. on formative assessment, as 00:01:24.670 --> 00:01:28.010 you said. I'm also the director of scholarship 00:01:28.010 --> 00:01:31.670 in my school. So in a way, all these roles, they 00:01:31.670 --> 00:01:35.430 are linked to teaching, to scholarship or pedagogical 00:01:35.430 --> 00:01:40.250 research and to the activities you do around 00:01:40.250 --> 00:01:45.180 teaching. Fantastic. And Izzy, would you like 00:01:45.180 --> 00:01:46.840 to just introduce what you do at LITE? Yeah, 00:01:46.859 --> 00:01:49.079 I only have one hat and I'm a research assistant 00:01:49.079 --> 00:01:52.359 at the Leeds Institute for Teaching Excellence. 00:01:54.140 --> 00:01:56.480 Fantastic. And you'll be bringing in some thoughts 00:01:56.480 --> 00:01:58.599 from your own research and the relationship between 00:01:58.599 --> 00:02:01.840 that and Alba's own fellowship, which I'm really 00:02:01.840 --> 00:02:05.239 looking forward to. So Alba, what motivated you 00:02:05.239 --> 00:02:07.079 to start your LITE Fellowship and what issue 00:02:07.079 --> 00:02:10.139 did you decide to address? That's a good question 00:02:10.139 --> 00:02:15.240 because... It was really a combination of chance, 00:02:15.520 --> 00:02:22.240 luck and my own pedagogical interest around formative 00:02:22.240 --> 00:02:25.259 assessment. I'll explain that a bit more. So 00:02:25.259 --> 00:02:28.759 basically, my fellowship was a sponsored fellowship, 00:02:29.000 --> 00:02:31.659 which means that unlike other fellowship projects. 00:02:33.610 --> 00:02:36.430 There was this title, which was, as you were 00:02:36.430 --> 00:02:40.189 saying before, about inclusive formative assessment 00:02:40.189 --> 00:02:43.629 design and the impact on student outcomes. And 00:02:43.629 --> 00:02:47.370 you applied for that as you would apply for a 00:02:47.370 --> 00:02:50.810 job position more than developing a project and 00:02:50.810 --> 00:02:54.710 sending a project. And I had a previous interest. 00:02:55.659 --> 00:02:59.560 On peer assessment, I had, as part of my own 00:02:59.560 --> 00:03:03.099 teaching practice, I had implemented a small 00:03:03.099 --> 00:03:06.180 project on peer oral assessment in the Spanish 00:03:06.180 --> 00:03:10.879 language teaching class. I had published a small 00:03:10.879 --> 00:03:13.580 piece as a book chapter on that. And I thought, 00:03:13.680 --> 00:03:16.120 hmm, this is something that I actually could 00:03:16.120 --> 00:03:20.099 explore further. So I sent the application with 00:03:20.099 --> 00:03:23.039 actually very little hope it could be successful. 00:03:23.840 --> 00:03:27.080 But it was successful and this is how I started 00:03:27.080 --> 00:03:32.460 my LITE Fellowship project. And then I started 00:03:32.460 --> 00:03:36.280 to explore a bit more how students perceive formative 00:03:36.280 --> 00:03:39.259 assessment. Great to hear that the LITE Fellowship 00:03:39.259 --> 00:03:41.139 enabled you to look into the underlying factors 00:03:41.139 --> 00:03:44.539 of an issue you'd identified and not just an 00:03:44.539 --> 00:03:46.479 issue that's exclusive to your own disciplinary 00:03:46.479 --> 00:03:49.479 context. Could you provide listeners with a brief 00:03:49.479 --> 00:03:51.340 overview of how you went about your project? 00:03:52.259 --> 00:03:56.759 Yeah, so because it was a sponsor fellowship, 00:03:57.159 --> 00:04:02.340 I was given a lot of freedom to design the project. 00:04:04.360 --> 00:04:08.740 I think my main question at the beginning was, 00:04:08.939 --> 00:04:11.719 well, a lot of colleagues, a lot of academic 00:04:11.719 --> 00:04:15.280 staff, they tend to always complain about students 00:04:15.280 --> 00:04:18.139 not really engaging with formative activities, 00:04:18.319 --> 00:04:20.839 why they're not doing these formative essays 00:04:20.839 --> 00:04:24.519 or these submissions. So initially I started 00:04:24.519 --> 00:04:30.360 with that research question. We did a series 00:04:30.360 --> 00:04:34.939 of focus groups. In fact, I have here the numbers. 00:04:35.040 --> 00:04:38.579 We did nine student focus groups with a total 00:04:38.579 --> 00:04:43.199 number of 27 participants, which was quite a 00:04:43.199 --> 00:04:46.939 success. And I also interviewed nine members 00:04:46.939 --> 00:04:49.699 of staff to try to compare how students perceive 00:04:49.699 --> 00:04:53.620 their barriers to engage with formative assessments, 00:04:53.819 --> 00:04:57.019 but also... what were the barriers perceived 00:04:57.019 --> 00:05:05.139 by the staff. As you know yourself, Robert, and 00:05:05.139 --> 00:05:08.500 Izzy here, you helped running some focus groups 00:05:08.500 --> 00:05:11.259 because I did the interviews with the staff myself, 00:05:11.339 --> 00:05:14.300 but then the focus groups were run by colleagues 00:05:14.300 --> 00:05:19.139 from LITE or even by some students as well for 00:05:19.139 --> 00:05:22.259 ethical reasons because maybe the students were 00:05:22.259 --> 00:05:26.120 feeling kind of, you know, a bit that they were 00:05:26.120 --> 00:05:30.300 pressured to say what I was expecting them to 00:05:30.300 --> 00:05:33.399 say. And in fact, it would be good to hear from 00:05:33.399 --> 00:05:35.399 you what was your experience during the focus 00:05:35.399 --> 00:05:38.899 groups. Robert, do you want to go first? Sure. 00:05:39.120 --> 00:05:42.620 It's testing my memory a bit. This was a little 00:05:42.620 --> 00:05:44.899 while ago and not long after I arrived at LITE, 00:05:45.000 --> 00:05:48.379 so things were a bit of a blur. But I can remember 00:05:48.379 --> 00:05:51.509 the focus group discussion. I was a part of and 00:05:51.509 --> 00:05:55.290 facilitating to be really dynamic. And there 00:05:55.290 --> 00:05:58.730 was a strong theme of feedback rising to the 00:05:58.730 --> 00:06:02.930 top of the discussion at various stages. Peer 00:06:02.930 --> 00:06:05.649 and participatory feedback opportunities were 00:06:05.649 --> 00:06:09.029 seen as normatively good things, but varied quite 00:06:09.029 --> 00:06:11.329 a lot in practice. Don't want to go in any more 00:06:11.329 --> 00:06:13.790 detail than that. A, because it's, as I say, 00:06:13.810 --> 00:06:15.370 stretching my memory. B, because I know you're 00:06:15.370 --> 00:06:17.110 going to talk about your findings shortly and 00:06:17.110 --> 00:06:19.290 I don't want to provide any spoilers. How about 00:06:19.290 --> 00:06:22.079 you, Izzy? My reflections might provide a few 00:06:22.079 --> 00:06:25.839 spoilers. So first of all. Provide spoilers. 00:06:26.220 --> 00:06:30.300 Is that OK? Yeah. OK. So first of all, again, 00:06:30.459 --> 00:06:33.459 I was only a few months into my new role as a 00:06:33.459 --> 00:06:35.759 research assistant. And this was probably one 00:06:35.759 --> 00:06:37.600 of my first experience of focus groups from the 00:06:37.600 --> 00:06:41.420 research aside. It was very useful for me how 00:06:41.420 --> 00:06:44.800 organised Alba was with everything admin related. 00:06:45.769 --> 00:06:48.149 And again, I would agree, Robert, I think that 00:06:48.149 --> 00:06:50.730 there was a really strong discussions arose around 00:06:50.730 --> 00:06:53.610 feedback. And two things that stick out for me 00:06:53.610 --> 00:06:57.250 to this day are one, the full spectrum of feedback 00:06:57.250 --> 00:06:59.949 experiences described by the students. So from 00:06:59.949 --> 00:07:01.990 formative opportunities with every assignment 00:07:01.990 --> 00:07:05.269 to very little to no formative feedback opportunities 00:07:05.269 --> 00:07:10.389 and nothing really beyond a mark or a grade for 00:07:10.389 --> 00:07:13.810 some courses. And I guess being quite new to. 00:07:14.319 --> 00:07:17.379 Being exposed to higher education and this kind 00:07:17.379 --> 00:07:18.980 of landscape, that was something that really 00:07:18.980 --> 00:07:21.339 surprised me because I just didn't realise how 00:07:21.339 --> 00:07:23.959 vast the differences were. And that's something 00:07:23.959 --> 00:07:26.459 that stuck with me. And then my second reflection 00:07:26.459 --> 00:07:31.060 is actually in relation to the formative feedback 00:07:31.060 --> 00:07:32.899 opportunities that were discussed by one of the 00:07:32.899 --> 00:07:35.779 students. And she said that they actually got 00:07:35.779 --> 00:07:37.699 the chance with every assignment that they did 00:07:37.699 --> 00:07:43.120 to hand in an essay or a draft plan. Really interestingly 00:07:43.120 --> 00:07:46.319 to me, the student reported how few of her peers 00:07:46.319 --> 00:07:48.839 took up this opportunity. And that was really 00:07:48.839 --> 00:07:51.360 surprising because it seems so valuable and apparently 00:07:51.360 --> 00:07:53.300 quite rare across the university course, but 00:07:53.300 --> 00:07:57.060 still wasn't taken up by that many people. And 00:07:57.060 --> 00:07:59.259 that is kind of where I know we're going to talk 00:07:59.259 --> 00:08:01.139 about this a bit later, but that was kind of 00:08:01.139 --> 00:08:03.199 one of the first early sparks of interest for 00:08:03.199 --> 00:08:06.100 me to explore feedback in higher education further, 00:08:06.199 --> 00:08:09.600 starting with student engagement and formative 00:08:09.600 --> 00:08:11.850 feedback opportunities. And yeah, I would say 00:08:11.850 --> 00:08:14.689 those focus groups were one of the earliest sparks 00:08:14.689 --> 00:08:17.269 for me that's led to many more things that we'll 00:08:17.269 --> 00:08:19.649 be discussing soon. Fantastic. I love how I came 00:08:19.649 --> 00:08:22.029 out of that focus group thinking, wow, feedback 00:08:22.029 --> 00:08:24.410 is really important and then did nothing about 00:08:24.410 --> 00:08:27.529 it. And you came out of it and started a PhD. 00:08:28.730 --> 00:08:30.910 And the rest of history, as they say. Lovely. 00:08:31.129 --> 00:08:36.730 You should be the host. Wrong reversal. So was 00:08:36.730 --> 00:08:39.120 that interesting to listen to? Yeah, yeah, it 00:08:39.120 --> 00:08:42.179 was because it reminded me of conversations that 00:08:42.179 --> 00:08:45.940 I had with you both, actually, about we wanted 00:08:45.940 --> 00:08:48.120 the students in the focus group to talk about 00:08:48.120 --> 00:08:51.200 formative assessment and they kept talking about 00:08:51.200 --> 00:08:55.000 feedback. And in a way, I'm giving spoilers of 00:08:55.000 --> 00:08:57.360 what I'm going to talk about in a minute, but 00:08:57.360 --> 00:09:01.600 it actually came up that the students care deeply 00:09:01.600 --> 00:09:03.980 about feedback. They didn't care that much about 00:09:03.980 --> 00:09:06.659 the assessment being formative or summative. 00:09:08.080 --> 00:09:10.620 Fantastic. So that does lead us nicely into my 00:09:10.620 --> 00:09:12.700 next question, which what did you learn? What 00:09:12.700 --> 00:09:16.080 were the main findings of the research? One word, 00:09:16.279 --> 00:09:19.279 feedback. And we can finish the podcast here. 00:09:20.679 --> 00:09:25.179 No, really. As I was saying, I think the main 00:09:25.179 --> 00:09:30.700 finding, which is not new, is that students were 00:09:30.700 --> 00:09:35.980 actually the main. There were many. reasons for 00:09:35.980 --> 00:09:38.139 students not to engage in formative activities, 00:09:38.379 --> 00:09:40.919 if you remember. But because there were deadlines, 00:09:41.419 --> 00:09:44.700 like everything, all those submissions happening 00:09:44.700 --> 00:09:47.539 at the same time, so they would prioritise like 00:09:47.539 --> 00:09:50.519 summative assessments. It was workload as well, 00:09:50.679 --> 00:09:54.120 which had to do with deadlines. But then it came 00:09:54.120 --> 00:09:58.399 up that it was about the expectations that they 00:09:58.399 --> 00:10:03.379 had about the feedback they would get. If students 00:10:03.379 --> 00:10:05.240 believe that the feedback they were going to 00:10:05.240 --> 00:10:08.860 get from that formative activity wasn't useful 00:10:08.860 --> 00:10:12.740 for future formative activities, they wouldn't 00:10:12.740 --> 00:10:16.519 engage with it. And I think that was the main 00:10:16.519 --> 00:10:22.480 finding. But also there were kind of previous 00:10:22.480 --> 00:10:28.480 expectations about what feedback was. There was 00:10:28.480 --> 00:10:33.870 a lot of this idea. regarding having mocks or 00:10:33.870 --> 00:10:38.370 having drills and having a lot of repetition, 00:10:38.669 --> 00:10:43.309 which it made me ask as well, is this what higher 00:10:43.309 --> 00:10:46.669 education is for or is it about previous expectations 00:10:46.669 --> 00:10:50.730 from students doing their levels and being focused 00:10:50.730 --> 00:10:53.570 so many years of their formative life, their 00:10:53.570 --> 00:10:57.570 formative years and on doing an exam and preparing 00:10:57.570 --> 00:11:02.320 for that exam as well. I have here a list of 00:11:02.320 --> 00:11:05.419 findings that I'm going to check. Something that 00:11:05.419 --> 00:11:08.279 came up, well, it didn't came up, but it made 00:11:08.279 --> 00:11:11.480 me reflect as well, was about feedback culture 00:11:11.480 --> 00:11:19.419 and feedback literacy. Students had this tendency 00:11:19.419 --> 00:11:22.460 of considering feedback as something you get 00:11:22.460 --> 00:11:27.259 in written form. tailored and personalised. And 00:11:27.259 --> 00:11:30.879 that was the feedback they valued the most. But 00:11:30.879 --> 00:11:33.639 quite often, I would have liked to see in the 00:11:33.639 --> 00:11:35.840 data that students were aware that feedback can 00:11:35.840 --> 00:11:38.159 take many forms, that it can be more informal, 00:11:38.460 --> 00:11:41.940 that it can be given by peers, that sometimes 00:11:41.940 --> 00:11:45.480 academic staff might give feedback to the group 00:11:45.480 --> 00:11:49.139 as a whole. And there wasn't a lot of mention 00:11:49.139 --> 00:11:53.620 to that. So hopefully the university and... possibly 00:11:53.620 --> 00:11:57.580 we'll talk about that later, is going to deliver 00:11:57.580 --> 00:12:00.220 a strong message on what is feedback and all 00:12:00.220 --> 00:12:04.379 the feedback opportunities students might have. 00:12:05.399 --> 00:12:09.080 And something I remember very well was the completely 00:12:09.080 --> 00:12:13.889 mixed opinions about peer feedback. There's a 00:12:13.889 --> 00:12:17.929 massive body of literature saying how good peer 00:12:17.929 --> 00:12:20.610 feedback is and why we should all be doing peer 00:12:20.610 --> 00:12:24.250 feedback. I've done it myself. It's more difficult 00:12:24.250 --> 00:12:26.769 than it seems and it's to be massively scaffolded 00:12:26.769 --> 00:12:30.590 for it to work and sometimes it doesn't. But, 00:12:30.750 --> 00:12:34.330 for example, there were students that they were 00:12:34.330 --> 00:12:36.429 saying that the best feedback they had received 00:12:36.429 --> 00:12:41.169 in their lives was from peers. And those were 00:12:41.169 --> 00:12:45.309 students' healthcare from the area of healthcare 00:12:45.309 --> 00:12:48.789 and medicine. And I wonder if in that area, giving 00:12:48.789 --> 00:12:51.470 feedback is actually one of the skills that is 00:12:51.470 --> 00:12:54.009 expected from graduates and they work on that. 00:12:54.169 --> 00:12:57.350 But on the humanities and social sciences side, 00:12:57.590 --> 00:13:01.610 literally, I have a quote that said that they 00:13:01.610 --> 00:13:04.909 thought it was cheap to give peer feedback and 00:13:04.909 --> 00:13:08.450 they felt it was cheap. So completely mixed opinions 00:13:08.450 --> 00:13:12.929 that... Gives you an idea of how complex is feedback 00:13:12.929 --> 00:13:18.629 and how messy education is, which it makes it 00:13:18.629 --> 00:13:22.190 joyful as well. So there's, although I mentioned 00:13:22.190 --> 00:13:23.850 earlier, obviously this is an issue that takes 00:13:23.850 --> 00:13:26.710 place across higher education in different spaces, 00:13:26.870 --> 00:13:28.950 there are also contextual differences, including 00:13:28.950 --> 00:13:31.909 disciplinary assumptions, for example. Is that 00:13:31.909 --> 00:13:37.080 about right? Yeah, because... When we did the 00:13:37.080 --> 00:13:42.960 coding, one of the actual codes I have in NVivo 00:13:42.960 --> 00:13:49.679 is in my disciplinary identity, in my studies, 00:13:49.960 --> 00:13:53.940 in humanities or in healthcare or in STEM, we 00:13:53.940 --> 00:13:57.200 do this. And there was a lot of interventions 00:13:57.200 --> 00:14:00.100 from students saying, oh, we do this in my modules 00:14:00.100 --> 00:14:03.840 or in my program. Oh, but in our module, we do 00:14:03.840 --> 00:14:07.110 this. We talk about feedback in general quite 00:14:07.110 --> 00:14:09.970 often and sometimes we don't take into account 00:14:09.970 --> 00:14:13.389 that we might be talking about different processes 00:14:13.389 --> 00:14:15.809 and different things depending on the context 00:14:15.809 --> 00:14:18.289 and the discipline. So feel free to shoot me 00:14:18.289 --> 00:14:20.490 down if this is jumping ahead. I know Izzy's 00:14:20.490 --> 00:14:22.309 going to come in shortly, but does this call 00:14:22.309 --> 00:14:24.269 for students to be more explicitly informed about 00:14:24.269 --> 00:14:26.370 the reasons for doing formative assessment and 00:14:26.370 --> 00:14:28.629 through that feedback? And if so, what are some 00:14:28.629 --> 00:14:31.990 of the ways of highlighting this? I think there 00:14:31.990 --> 00:14:39.740 needs to be a... cultural shift. And I would 00:14:39.740 --> 00:14:42.899 like to see feedback detached from assessment. 00:14:43.340 --> 00:14:47.000 I've had this conversation many times with colleagues 00:14:47.000 --> 00:14:50.299 that they think, well, but feedback comes with 00:14:50.299 --> 00:14:53.700 assessment. How can you have feedback without 00:14:53.700 --> 00:14:57.580 assessment? I actually think we should have feedback 00:14:57.580 --> 00:15:02.019 not without assessment, but at least before assessment. 00:15:02.960 --> 00:15:07.539 There is this institutional obsession about providing 00:15:07.539 --> 00:15:11.120 feedback with a grade and explaining that grade 00:15:11.120 --> 00:15:15.659 at the end of the course. And quite often this 00:15:15.659 --> 00:15:19.100 is not working for students or is not working 00:15:19.100 --> 00:15:21.539 the way we expect for students. And at the same 00:15:21.539 --> 00:15:25.419 time, I'm thinking about the interviews with 00:15:25.419 --> 00:15:28.419 staff and any colleague belonging to academic 00:15:28.419 --> 00:15:31.220 staff, they will tell you that they're most... 00:15:31.720 --> 00:15:35.519 dreaded period of the year is marking season 00:15:35.519 --> 00:15:39.919 or assessment season as something that we tend 00:15:39.919 --> 00:15:43.360 not to like. Because in a way you put all that 00:15:43.360 --> 00:15:48.620 effort knowing that it might not be read and 00:15:48.620 --> 00:15:51.240 you cannot blame the students for that because 00:15:51.240 --> 00:15:54.279 the students, it's the end of the course, they 00:15:54.279 --> 00:15:57.480 won't be able to change anything on what they've 00:15:57.480 --> 00:16:01.440 submitted. They won't agree. That feedback should 00:16:01.440 --> 00:16:05.639 have taken place before, but the institution 00:16:05.639 --> 00:16:10.259 is telling us, no, you need to use your, I don't 00:16:10.259 --> 00:16:12.279 know how many points of workload for assessment 00:16:12.279 --> 00:16:16.080 at this point, not before. You still need to 00:16:16.080 --> 00:16:19.840 write 200 words of feedback or 300 words or I 00:16:19.840 --> 00:16:23.960 don't know how many. So I think I would like 00:16:23.960 --> 00:16:27.720 to see a cultural shift, if that makes sense. 00:16:28.889 --> 00:16:30.970 Yeah, I'd agree, Albert. I think that, yeah, 00:16:31.029 --> 00:16:34.110 there's way too much emphasis placed on summative 00:16:34.110 --> 00:16:36.389 feedback. And many of the times it's actually 00:16:36.389 --> 00:16:40.250 got no use and no place to land for students. 00:16:40.289 --> 00:16:43.590 They have nowhere to take it and use it in the 00:16:43.590 --> 00:16:46.970 future because that module's over. And with the 00:16:46.970 --> 00:16:49.389 system at university, often these don't link. 00:16:49.490 --> 00:16:50.929 So next year they're going to be doing something 00:16:50.929 --> 00:16:53.190 completely different. So, yeah, I totally agree 00:16:53.190 --> 00:16:55.929 that there should be more those efforts and that 00:16:55.929 --> 00:16:58.450 workload model or whatever. Their time they're 00:16:58.450 --> 00:17:01.789 given would be much more valuable in formative 00:17:01.789 --> 00:17:04.309 ways, providing that feedback throughout the 00:17:04.309 --> 00:17:06.769 course in the build up to the summative and not 00:17:06.769 --> 00:17:11.049 spending the staff time providing those when 00:17:11.049 --> 00:17:13.069 it has very little relevance for anyone at that 00:17:13.069 --> 00:17:15.289 point. I'm right in saying that you were also 00:17:15.289 --> 00:17:17.769 involved at a later stage. Yes. And again, I 00:17:17.769 --> 00:17:20.690 was fascinated to see that what I'd heard from 00:17:20.690 --> 00:17:23.589 just the one focus group I did had carried on 00:17:23.589 --> 00:17:26.630 and how much feedback was mentioned and discussed. 00:17:27.420 --> 00:17:30.819 in the results by staff and by students it seemed 00:17:30.819 --> 00:17:33.839 that no one could talk about formative assessment 00:17:33.839 --> 00:17:36.380 without being swayed away towards talking about 00:17:36.380 --> 00:17:39.299 formative feedback and yeah I agree with all 00:17:39.299 --> 00:17:40.900 the things Alba was saying about there needing 00:17:40.900 --> 00:17:45.220 to be a cultural shift and as Robert has said 00:17:45.220 --> 00:17:48.059 so the early interest sparked by those focus 00:17:48.059 --> 00:17:51.359 groups and then helping out with coding the reason 00:17:51.359 --> 00:17:53.140 I needed a bit of coding for Alba was because 00:17:53.140 --> 00:17:56.740 by then I was developing my own uh research proposal 00:17:56.740 --> 00:18:00.599 which has now turned into my PhD when Alba is 00:18:00.599 --> 00:18:03.920 my supervisor so we we had to part ways in the 00:18:03.920 --> 00:18:07.400 fellowship support and yeah so that's and I am 00:18:07.400 --> 00:18:10.440 now looking at some of those um some of those 00:18:10.440 --> 00:18:13.579 understandings and what I mean things in the 00:18:13.579 --> 00:18:15.299 first year of the PhD are messy and they shift 00:18:15.299 --> 00:18:18.339 all the time but we're kind of exploring perceptions 00:18:18.339 --> 00:18:22.859 and understandings of um staff at the teaching 00:18:22.859 --> 00:18:26.900 level and at the leadership level and how how 00:18:26.900 --> 00:18:29.500 they align or differ and what if there's some 00:18:29.500 --> 00:18:32.000 some things we can unpick there to help with 00:18:32.000 --> 00:18:35.319 this cultural shift. Thanks Izzy so bringing 00:18:35.319 --> 00:18:37.380 this all together then what sorts of changes 00:18:37.380 --> 00:18:39.119 do you think you might like to see as a result 00:18:39.119 --> 00:18:42.660 of this research? I think we've already hinted 00:18:42.660 --> 00:18:46.279 that with the word cultural shift which I'm aware 00:18:46.279 --> 00:18:51.579 is it can be a very vague word as well but As 00:18:51.579 --> 00:18:55.480 I was saying, feedback tends to be linked with 00:18:55.480 --> 00:18:59.180 final assessments and it shouldn't be linked 00:18:59.180 --> 00:19:02.579 to that. Final assessments should be something 00:19:02.579 --> 00:19:07.599 that gives you a grade and explain that grade 00:19:07.599 --> 00:19:12.079 because those are usually as well looked at by 00:19:12.079 --> 00:19:15.430 external examiners. But there should be feedback 00:19:15.430 --> 00:19:18.470 and formative. I don't want to talk about formative 00:19:18.470 --> 00:19:20.230 assessment. I'd rather use the word formative 00:19:20.230 --> 00:19:23.349 activities or formative opportunities before 00:19:23.349 --> 00:19:27.630 those summative assessments that have a tendency 00:19:27.630 --> 00:19:32.150 to take place at the end of a course or at the 00:19:32.150 --> 00:19:36.029 end of a module. And I would like to see that 00:19:36.029 --> 00:19:40.289 need specifically recognised in student education 00:19:40.289 --> 00:19:44.380 policies beyond... oh, that's a nice thing to 00:19:44.380 --> 00:19:49.880 have, or we should all be doing that. And all 00:19:49.880 --> 00:19:52.359 the staff I interviewed, they were aware that 00:19:52.359 --> 00:19:55.000 we should all be doing that, but it's just not 00:19:55.000 --> 00:19:58.960 recognised, and there isn't any incentive to 00:19:58.960 --> 00:20:01.400 do that. And that's a wider problem that goes 00:20:01.400 --> 00:20:03.920 beyond feedback. It's about recognising teaching 00:20:03.920 --> 00:20:11.849 and giving value to teaching. But it cannot depend 00:20:11.849 --> 00:20:15.410 on individual willingness to be implemented. 00:20:15.529 --> 00:20:19.309 It needs to be part of policy in the same way 00:20:19.309 --> 00:20:23.309 that everybody assumes that when you're doing 00:20:23.309 --> 00:20:25.150 a final summative assessment, you're going to 00:20:25.150 --> 00:20:29.349 follow some steps. There should be some, not 00:20:29.349 --> 00:20:34.809 only guidance, but some policy. It needs to be 00:20:34.809 --> 00:20:37.390 recognised on workloads. And I know, I mean, 00:20:37.410 --> 00:20:41.509 when I ask staff, what were the main barriers 00:20:41.509 --> 00:20:43.890 to implement formative assessments or activities 00:20:43.890 --> 00:20:47.369 or opportunities or formative feedback workloads 00:20:47.369 --> 00:20:50.309 was the main. And I know there are constraints 00:20:50.309 --> 00:20:55.009 and I know that the higher education landscape 00:20:55.009 --> 00:20:58.769 right now, there's a lot of constraints and financial 00:20:58.769 --> 00:21:03.829 constraints around that. But we need to recognise 00:21:03.829 --> 00:21:10.150 this and this time fairly. And it cannot be part 00:21:10.150 --> 00:21:12.289 of, for example, in my school, any formative 00:21:12.289 --> 00:21:14.450 feedback or formative opportunity that you give 00:21:14.450 --> 00:21:18.150 is part of the class preparation time you have 00:21:18.150 --> 00:21:19.990 in your workload. Every workload is a different 00:21:19.990 --> 00:21:23.049 world in different schools. But no, it should 00:21:23.049 --> 00:21:30.210 be specifically recognised in some way. Thinking 00:21:30.210 --> 00:21:34.589 about students as well, I would like to hear 00:21:34.589 --> 00:21:38.559 a stronger message about engaging. with feedback 00:21:38.559 --> 00:21:42.680 and first understanding all the different forms 00:21:42.680 --> 00:21:46.079 in which feedback take place. Feedback is not 00:21:46.079 --> 00:21:50.519 only a written thing that you'll get on Minerva, 00:21:50.539 --> 00:21:53.680 for example. Feedback can be collective. Feedback 00:21:53.680 --> 00:21:59.380 can be given by peers. to kind of more actively 00:21:59.380 --> 00:22:02.900 engage with that. And I'd like a stronger message 00:22:02.900 --> 00:22:07.420 towards students about expectations on feedback. 00:22:07.559 --> 00:22:11.920 Feedback is not only mock exams or drills, but 00:22:11.920 --> 00:22:14.640 it's something way more complex. I'm actually 00:22:14.640 --> 00:22:18.059 working on something that will be shared with 00:22:18.059 --> 00:22:20.680 the whole student community about, for example, 00:22:20.759 --> 00:22:25.440 how can you have all your feedback in one place? 00:22:26.109 --> 00:22:28.190 How can you check the feedback you've received 00:22:28.190 --> 00:22:31.569 before you submit an assessment? Sometimes the 00:22:31.569 --> 00:22:35.369 systems do not help as well because, for example, 00:22:35.369 --> 00:22:37.650 in the current systems that we have, students 00:22:37.650 --> 00:22:40.190 will see the grade before seeing the feedback. 00:22:40.630 --> 00:22:45.710 So quite often, once they see the grade, they 00:22:45.710 --> 00:22:48.170 might not want to look at the feedback, maybe 00:22:48.170 --> 00:22:51.289 because they're happy with the grade, maybe because... 00:22:51.819 --> 00:22:55.559 They're scared, if they're not happy, they're 00:22:55.559 --> 00:23:01.119 scared of what they might read about it. So that 00:23:01.119 --> 00:23:04.420 bit, the word feedback literacy, it seems to 00:23:04.420 --> 00:23:09.500 be the flavour of the month. But helping students 00:23:09.500 --> 00:23:14.279 to engage with feedback better than they do. 00:23:14.799 --> 00:23:19.619 And additionally, I would like to see the NSS 00:23:19.619 --> 00:23:26.119 question about feedback. Quite often when you 00:23:26.119 --> 00:23:29.359 read papers about feedback, they start saying, 00:23:29.420 --> 00:23:32.579 well, we need to improve on that because that's 00:23:32.579 --> 00:23:36.140 the lowest score in the whole sector. And I'm 00:23:36.140 --> 00:23:38.319 going to read out the questions for the podcast. 00:23:38.779 --> 00:23:42.500 There's question 13 that says, how often have 00:23:42.500 --> 00:23:45.200 you received assessment feedback on time? And 00:23:45.200 --> 00:23:48.359 question 14, how often does feedback help you? 00:23:49.370 --> 00:23:54.769 improve your work? Those questions create a lot 00:23:54.769 --> 00:23:57.269 of confusion, especially because they're asked 00:23:57.269 --> 00:24:01.430 at final year students. The last piece of feedback 00:24:01.430 --> 00:24:05.049 was potentially the last piece of feedback they're 00:24:05.049 --> 00:24:06.990 going to receive at the university. And usually 00:24:06.990 --> 00:24:09.029 the last piece of feedback is aimed at explaining 00:24:09.029 --> 00:24:12.390 a grade and justifying a grade, not at improving 00:24:12.390 --> 00:24:17.720 a piece of work. So it's not clearly... explain 00:24:17.720 --> 00:24:22.140 what feedback those questions are referring to. 00:24:22.380 --> 00:24:26.460 And when they ask about assessment feedback on 00:24:26.460 --> 00:24:30.440 time, in which time frame, on time in regards 00:24:30.440 --> 00:24:34.039 to what. So I don't think they're really well 00:24:34.039 --> 00:24:38.599 rephrased. I mean, additionally, I would like 00:24:38.599 --> 00:24:42.220 to see the NSS abolished in general, but possibly 00:24:42.220 --> 00:24:44.700 that's something to discuss in another episode. 00:24:46.799 --> 00:24:51.640 A lot to reflect on there. Lots of really juicy 00:24:51.640 --> 00:24:53.420 stuff to get into and at different levels of 00:24:53.420 --> 00:24:55.759 scale. Something that I don't know about you, 00:24:55.779 --> 00:24:57.519 Izzy, but something that I feel free to come 00:24:57.519 --> 00:24:58.839 in, but something that I kept coming back to 00:24:58.839 --> 00:25:01.700 was a lot of the questions around feedback contribute 00:25:01.700 --> 00:25:04.480 to wider questions of educational purpose and 00:25:04.480 --> 00:25:07.000 kind of the idea that all educational policy 00:25:07.000 --> 00:25:09.720 and practice is inherently teleological. So it 00:25:09.720 --> 00:25:12.660 contributes to things that happen outside of 00:25:12.660 --> 00:25:16.839 the university. Is there a conversation that 00:25:16.839 --> 00:25:19.880 needs to be had? And part of that might be disentangling 00:25:19.880 --> 00:25:22.160 feedback from assessment about the broader purposes 00:25:22.160 --> 00:25:26.319 of feedback and the ends to which feedback contributes. 00:25:26.900 --> 00:25:29.980 Yeah, I mean, I think there's a conversation 00:25:29.980 --> 00:25:34.319 to be had about what happens in a classroom, 00:25:34.539 --> 00:25:38.720 which resources do you allocate to teaching and 00:25:38.720 --> 00:25:43.180 how learning takes place. And obviously feedback 00:25:43.180 --> 00:25:47.170 has. has a role to play in that because Izzy 00:25:47.170 --> 00:25:50.369 has read a lot of papers talking about how feedback 00:25:50.369 --> 00:25:54.329 is essential to learning, more than me at this 00:25:54.329 --> 00:25:57.349 point with your PhD possibly. But there is a 00:25:57.349 --> 00:26:00.369 conversation to be had about what is learning, 00:26:00.569 --> 00:26:03.650 what is higher education for. I tend to go always 00:26:03.650 --> 00:26:05.569 into the big questions, into the bigger questions. 00:26:06.029 --> 00:26:10.170 And how this is valued as well by institutionally. 00:26:10.349 --> 00:26:12.569 I think that is the bottom line is feedback is 00:26:12.569 --> 00:26:16.400 undervalued. And it's not given its value in 00:26:16.400 --> 00:26:18.700 the classroom. And then it's not given its value 00:26:18.700 --> 00:26:22.140 as what it can contribute, like you said, to 00:26:22.140 --> 00:26:26.700 the wider student success, students going forwards 00:26:26.700 --> 00:26:30.799 into later life. It is just undervalued. I think 00:26:30.799 --> 00:26:33.559 that's the bottom line. Yeah, especially in relation 00:26:33.559 --> 00:26:35.759 to what we experienced in those focus groups 00:26:35.759 --> 00:26:38.339 in terms of how students valued feedback and 00:26:38.339 --> 00:26:42.740 in many cases were quite critical of their experiences. 00:26:43.640 --> 00:26:45.440 I think the engagement thing is always put on 00:26:45.440 --> 00:26:47.559 students when actually a lot of it is, I think 00:26:47.559 --> 00:26:49.720 Alba might have mentioned this early on, is about 00:26:49.720 --> 00:26:53.180 the timing that it's offered or about its perceived 00:26:53.180 --> 00:26:57.079 usefulness or how it's given it come across to 00:26:57.079 --> 00:26:59.180 them. I think there's a lot of work to be done 00:26:59.180 --> 00:27:05.119 there as well. This reminds me of, we had a student 00:27:05.119 --> 00:27:09.579 doing a project this summer that she was piloting. 00:27:10.190 --> 00:27:14.589 a questioner for his PhD. And when we had the 00:27:14.589 --> 00:27:17.069 last meeting with the student and we asked her, 00:27:17.109 --> 00:27:20.609 like, what have you learned during this process? 00:27:21.089 --> 00:27:24.430 She said something that resonates, that she said, 00:27:24.470 --> 00:27:28.210 well, I've just realised that students and lecturers, 00:27:28.390 --> 00:27:32.089 we have the same problems and we're all kind 00:27:32.089 --> 00:27:34.910 of the same boat and we face very similar challenges. 00:27:37.230 --> 00:27:39.769 In a way, if you think about feedback and you 00:27:39.769 --> 00:27:42.890 think about some of the student responses regarding 00:27:42.890 --> 00:27:46.329 deadlines, workload, engagement, and some of 00:27:46.329 --> 00:27:49.670 the responses from academic staff colleagues, 00:27:50.029 --> 00:27:54.069 they're more similar than one would think because 00:27:54.069 --> 00:27:56.690 it's about value and engagement, as Izzy was 00:27:56.690 --> 00:27:58.930 saying. Yeah, and that 100 % resonates in the 00:27:58.930 --> 00:28:03.210 literature. One of my top coded phrases in my 00:28:03.210 --> 00:28:07.789 reading is misaligned understanding. Yeah. Well, 00:28:07.869 --> 00:28:10.369 fantastic. Not fantastic, but you know what I 00:28:10.369 --> 00:28:13.009 mean. So for listeners that would like to know 00:28:13.009 --> 00:28:15.289 more, because it would be great to go in more 00:28:15.289 --> 00:28:18.769 depth, but obviously we can't right now. Are 00:28:18.769 --> 00:28:20.710 there any further research outputs or activities 00:28:20.710 --> 00:28:24.269 that can be engaged with? The incubator. This 00:28:24.269 --> 00:28:27.890 is the advertising bit. So I said at the beginning 00:28:27.890 --> 00:28:31.329 of this podcast that in a way, this fellowship 00:28:31.329 --> 00:28:36.880 led me to apply. for the role of the feedback 00:28:36.880 --> 00:28:41.420 Incubator Lead with LITE. Right now, we're trying 00:28:41.420 --> 00:28:46.599 to build a community of colleagues. And when 00:28:46.599 --> 00:28:49.299 I say colleagues, I say that including professional 00:28:49.299 --> 00:28:52.140 services, even including students, including 00:28:52.140 --> 00:28:55.299 the whole university community interested in 00:28:55.299 --> 00:29:00.200 feedback and in pedagogical research about feedback. 00:29:00.759 --> 00:29:03.420 good practice sharing about feedback or just 00:29:03.420 --> 00:29:07.700 an interest in the topic. If you don't have an 00:29:07.700 --> 00:29:11.839 idea but you have an interest, please join us. 00:29:12.039 --> 00:29:15.339 And we are right now in the process of building 00:29:15.339 --> 00:29:18.099 this community. There will be a reading list 00:29:18.099 --> 00:29:21.420 to start with for anybody interested in reading 00:29:21.420 --> 00:29:24.140 about feedback. It might not be an engaging topic 00:29:24.140 --> 00:29:27.259 for everybody, but maybe you want to do something 00:29:27.259 --> 00:29:29.930 small. with your practice, with your teaching, 00:29:30.190 --> 00:29:32.890 and you don't know where to start, you can start 00:29:32.890 --> 00:29:38.549 with us. So anybody interested in feedback, please 00:29:38.549 --> 00:29:42.009 send me an email or contact LITE, any colleagues 00:29:42.009 --> 00:29:44.609 from LITE, and they will signpost you to the 00:29:44.609 --> 00:29:49.329 incubator. But hopefully I want to put in writing 00:29:49.329 --> 00:29:54.009 and publish some of those findings and some of 00:29:54.009 --> 00:29:57.210 those data, because otherwise... Pedagogical 00:29:57.210 --> 00:29:59.970 research is not research or scholarship if you 00:29:59.970 --> 00:30:02.890 don't share it with the world. And please listen 00:30:02.890 --> 00:30:09.009 to the rest of the podcast. We'll also be providing 00:30:09.009 --> 00:30:12.369 a link to any other research -related outputs. 00:30:12.369 --> 00:30:15.089 For example, your light snapshot with a summary 00:30:15.089 --> 00:30:18.089 of what you've spoken about today. That's a work 00:30:18.089 --> 00:30:21.849 in progress. It's coming. No worries. Just holding 00:30:21.849 --> 00:30:27.140 you to account by this recording. Great. Thanks 00:30:27.140 --> 00:30:30.539 for that, Alba. And curious to ask, what next? 00:30:32.539 --> 00:30:34.960 Developing the incubator and all the things you've 00:30:34.960 --> 00:30:39.799 already mentioned. Well, building a community, 00:30:39.839 --> 00:30:44.539 as I was saying, trying to write a paper on the 00:30:44.539 --> 00:30:49.000 stuff I've been talking about, but also as part 00:30:49.000 --> 00:30:52.769 of the incubator activity. I was thinking to 00:30:52.769 --> 00:30:56.910 try to get like an open collective book at the 00:30:56.910 --> 00:31:00.750 University of Leeds, the library. They have this 00:31:00.750 --> 00:31:04.559 amazing opportunity that you can... publish a 00:31:04.559 --> 00:31:07.880 book in an open format. And it's not just like 00:31:07.880 --> 00:31:10.880 a PDF that you upload somewhere. It's more like, 00:31:10.920 --> 00:31:14.380 you know, it's quite more interactive. They use 00:31:14.380 --> 00:31:17.440 WordPress and they give you a lot of support. 00:31:17.740 --> 00:31:21.180 So I'm interested in putting a call out there, 00:31:21.299 --> 00:31:26.680 maybe optimistically by spring to see if there's 00:31:26.680 --> 00:31:31.839 an interest in other activities, projects, reflections, 00:31:32.099 --> 00:31:35.309 resources. to put everything together around 00:31:35.309 --> 00:31:38.150 feedback and what's going on at the university 00:31:38.150 --> 00:31:44.410 in just one place. I mean, what's next for me 00:31:44.410 --> 00:31:49.630 is a year on maternity leave. But no, I am, well, 00:31:49.829 --> 00:31:52.930 I'm currently looking again at my PhD questions 00:31:52.930 --> 00:31:55.730 for the third time this year, having read a lot 00:31:55.730 --> 00:31:58.289 more literature, attended conferences, had great 00:31:58.289 --> 00:32:01.269 conversations with people. So I'm hoping to... 00:32:01.640 --> 00:32:04.319 get those a bit more finalised before I go off. 00:32:04.539 --> 00:32:08.059 And then I'm excited when I come back in 2027 00:32:08.059 --> 00:32:12.359 to join Alba's thriving feedback incubator again 00:32:12.359 --> 00:32:16.599 and pick up my PhD and I'm excited to see where 00:32:16.599 --> 00:32:20.200 that research will take me. Fantastic, yeah. 00:32:20.339 --> 00:32:22.640 As an outside observer, I'm really looking forward 00:32:22.640 --> 00:32:25.359 to seeing how that community develops and the 00:32:25.359 --> 00:32:28.420 areas of research that get developed as a result 00:32:28.420 --> 00:32:32.500 of that. Actually, something that came up in 00:32:32.500 --> 00:32:34.839 many conversations I have with colleagues and 00:32:34.839 --> 00:32:38.920 with the LITE team, it is that feedback seems 00:32:38.920 --> 00:32:42.579 like an under -research area, but at the same 00:32:42.579 --> 00:32:47.740 time is a priority for the university and a priority 00:32:47.740 --> 00:32:53.380 for Light. But as EZ is realising developing 00:32:53.380 --> 00:32:56.980 her PhD project, it is quite difficult to do 00:32:56.980 --> 00:32:59.849 research on feedback. I mean, there are many, 00:32:59.930 --> 00:33:01.970 many questions that are really difficult to answer 00:33:01.970 --> 00:33:04.769 that we've had many conversations about, like, 00:33:04.890 --> 00:33:08.990 what is effective feedback? Even what is feedback? 00:33:09.109 --> 00:33:11.190 What are we talking about when we talk feedback? 00:33:11.609 --> 00:33:16.589 How do you measure that if that can be measured? 00:33:16.809 --> 00:33:21.470 So it's actually quite a complex area, but also 00:33:21.470 --> 00:33:23.609 quite under -researched. So we really welcome 00:33:23.609 --> 00:33:28.390 people with an interest in the area. So many 00:33:28.390 --> 00:33:30.369 of those things mean different things to different 00:33:30.369 --> 00:33:33.730 people. And then also we're battling now with 00:33:33.730 --> 00:33:37.170 who's involved and at what levels and who can 00:33:37.170 --> 00:33:40.049 make the changes and where should we be tackling 00:33:40.049 --> 00:33:45.369 this as well. I'm sure many areas of research 00:33:45.369 --> 00:33:47.630 in higher education have many levels of complexity, 00:33:47.970 --> 00:33:50.670 but this just feels like a huge web whenever 00:33:50.670 --> 00:33:53.829 we delve into these conversations. Come and join 00:33:53.829 --> 00:33:56.549 the web. It sounds like that complexity only 00:33:56.549 --> 00:33:58.990 increases the benefits of joining the community 00:33:58.990 --> 00:34:00.769 and having those collective conversations and 00:34:00.769 --> 00:34:04.009 sharing expertise. Yeah, don't be alone. Last 00:34:04.009 --> 00:34:06.750 but not least, my favourite question, perhaps. 00:34:07.430 --> 00:34:09.610 Do you have any advice for anyone else who might 00:34:09.610 --> 00:34:11.630 be considering embarking in a pedagogical research 00:34:11.630 --> 00:34:13.630 project and is there anything you might have 00:34:13.630 --> 00:34:16.449 done differently? I thought for this question, 00:34:16.510 --> 00:34:23.530 I would list all my sins. So all this advice 00:34:23.530 --> 00:34:29.590 is based on everything I've done myself wrong. 00:34:30.329 --> 00:34:35.250 First, if you're doing data collection, always 00:34:35.250 --> 00:34:39.289 pilot your questions. You know, interviews, questions, 00:34:39.349 --> 00:34:42.349 always pilot that because possibly you're going 00:34:42.349 --> 00:34:47.449 to have to tweak that. I didn't do it. This might 00:34:47.449 --> 00:34:49.989 seem really obvious, but reading is important. 00:34:50.930 --> 00:34:54.510 Literature reviews are important. Have a general 00:34:54.510 --> 00:34:58.590 sense of what's out there, what has been written 00:34:58.590 --> 00:35:02.889 about, what has been done before and where you 00:35:02.889 --> 00:35:05.530 can, in a way, plant your flag and how can you 00:35:05.530 --> 00:35:08.989 contribute to that body of knowledge. Even if 00:35:08.989 --> 00:35:11.389 it's something quite small, and that's absolutely 00:35:11.389 --> 00:35:14.090 fine, you don't need to do massive projects. 00:35:16.059 --> 00:35:19.739 PedRes or scholarship in my role, in my school, 00:35:19.900 --> 00:35:24.019 has to do with doing something small. And sometimes 00:35:24.019 --> 00:35:26.280 that can have an impact in your own teaching. 00:35:26.400 --> 00:35:31.300 And that's absolutely fine. And finally, avoid 00:35:31.300 --> 00:35:34.059 reinventing the wheel because possibly that great 00:35:34.059 --> 00:35:36.920 idea for a project that you have, it has already 00:35:36.920 --> 00:35:39.880 been done. That has happened to me a lot of times. 00:35:40.380 --> 00:35:42.920 Then don't be alone, as I was saying before, 00:35:43.039 --> 00:35:46.639 kind of try to build a community, find your community, 00:35:46.860 --> 00:35:50.800 find your tribe in a way, talk to colleagues, 00:35:51.079 --> 00:35:56.340 come to the incubator again and ask questions 00:35:56.340 --> 00:36:02.460 to your community and try to go on this journey 00:36:02.460 --> 00:36:05.960 accompanied by colleagues, because otherwise 00:36:05.960 --> 00:36:09.849 it can feel a bit lonely. Thank you very much. 00:36:10.610 --> 00:36:13.329 And I would, I mean, Alba's mentioned this, but 00:36:13.329 --> 00:36:16.590 this is one that I have learned in my first year 00:36:16.590 --> 00:36:19.969 of my PhD is the importance of the literature. 00:36:20.349 --> 00:36:23.070 And you'll probably almost never be finished 00:36:23.070 --> 00:36:26.630 with reading. And it's so important to stay up 00:36:26.630 --> 00:36:28.929 to date with the field. And the more you read, 00:36:28.949 --> 00:36:30.829 the more informed you become. And I think also 00:36:30.829 --> 00:36:35.789 it's important. to not be afraid to change and 00:36:35.789 --> 00:36:38.489 develop things and let their literature guide 00:36:38.489 --> 00:36:41.750 and aid you even if that wasn't what you originally 00:36:41.750 --> 00:36:43.710 planned like Alba said you need to plant your 00:36:43.710 --> 00:36:46.110 flag somewhere and you might need to take some 00:36:46.110 --> 00:36:47.869 twists and turns because things are published 00:36:47.869 --> 00:36:51.989 all the time we actually funnily enough one of 00:36:51.989 --> 00:36:55.329 my earlier ideas quite close to my original proposal 00:36:55.329 --> 00:36:58.619 was published by some of the higher mighties 00:36:58.619 --> 00:37:01.179 in the feedback field about two weeks ago so 00:37:01.179 --> 00:37:03.900 we were very relieved that we'd we'd been guided 00:37:03.900 --> 00:37:08.579 recently to um take a new tangent great thanks 00:37:08.579 --> 00:37:10.639 both yes my favourite section because I received 00:37:10.639 --> 00:37:15.380 such great advice for myself selfishly um that's 00:37:15.380 --> 00:37:17.659 it that's the end of the episode thanks Robert 00:37:17.659 --> 00:37:20.380 thank you both for your time today thanks again 00:37:20.380 --> 00:37:22.139 for listening and tune in next time