1 00:00:00,000 --> 00:00:15,280 Welcome to Light Bites, an occasional podcast from Leeds Institute for Teaching Excellence 2 00:00:15,280 --> 00:00:20,000 at the University of Leeds. Episodes will be hosted by members of the Light team. And 3 00:00:20,000 --> 00:00:29,240 we'll be showcasing the scholarship of teaching and learning from across the University. 4 00:00:29,240 --> 00:00:34,800 Hello you're listening to Light Bites. I'm Robert Avery's Research and Impact Officer 5 00:00:34,800 --> 00:00:41,200 at the Leeds Institute for Teaching Excellence. Two for the price of one today, because we're 6 00:00:41,200 --> 00:00:47,760 joined by not one but two Light Fellows who are halfway through their two-year Light Fellowship, 7 00:00:47,760 --> 00:00:54,680 which is called Assessment Flexibility to Embed Real Professional Practice. Good afternoon, 8 00:00:54,680 --> 00:01:01,320 Mr. Ravie, Mohsen Besharad. How are you both? Very good, thank you. Very good, thanks. 9 00:01:01,320 --> 00:01:05,720 Very good, thanks. Excited to get started? Yeah, absolutely. Excellent. So why don't 10 00:01:05,720 --> 00:01:12,120 we begin by you telling us a little bit about what you do at the University of Leeds. Good 11 00:01:12,120 --> 00:01:17,640 afternoon, Robert. Thank you for having us in this podcast session. So my name is Mohsen 12 00:01:17,640 --> 00:01:22,400 Besharad. I'm a Curriculum Redefined Lecturer at the School of Civil Engineering, joined 13 00:01:22,400 --> 00:01:30,400 the University of Leeds in 2022. I'm also Program Leader for Undergraduate, three undergraduate 14 00:01:30,400 --> 00:01:36,840 programs, Civil and Structural Engineering, Civil and Environmental Engineering and Civil 15 00:01:36,840 --> 00:01:42,880 Engineering. And yeah, thank you so much for having us. Thank you, Robert. Delighted to 16 00:01:42,880 --> 00:01:48,180 be on the podcast here with you. Look forward to discussing what we've really gathered in 17 00:01:48,180 --> 00:01:52,360 terms of data and findings over the course of the last year. So I'm Manoj Ravie over 18 00:01:52,360 --> 00:01:57,960 from the School of Chemical and Process Engineering. So I work as a lecturer there and like Mohsen, 19 00:01:57,960 --> 00:02:04,480 joined the University in 2022 as a Curriculum Redefined Lecturer. Fantastic. Let's get on 20 00:02:04,480 --> 00:02:12,080 to talking about your fellowship then. What motivated you to undertake this piece of research? 21 00:02:12,080 --> 00:02:18,720 Great. So there are a lot of motivations, but for me, actually, I have background in 22 00:02:18,720 --> 00:02:24,240 industry and also in academia. And when I was working in industry, I observed a lot 23 00:02:24,240 --> 00:02:29,080 of like graduates that they come to industry, they are struggling to use their skills developed 24 00:02:29,080 --> 00:02:35,880 during education in terms of their communication, developing the reports, doing the work itself, 25 00:02:35,880 --> 00:02:42,400 because in academia mostly we teach students to focus on a perfect piece of the work to 26 00:02:42,400 --> 00:02:51,120 get the final mark. And yeah, so that motivated me to look at the gap between these kind of 27 00:02:51,120 --> 00:02:57,600 approach we have in academia and the requirement of industry to see how we can improve students 28 00:02:57,600 --> 00:03:04,240 to be more work ready. Yeah, just building on what Mohsen said there, I think my interests 29 00:03:04,240 --> 00:03:08,960 were very similar as well. So in terms of when students are going out there, there is 30 00:03:08,960 --> 00:03:15,360 this general narrative which is painted outside of higher education that students are unnecessarily 31 00:03:15,360 --> 00:03:20,800 ready to take on the challenges of the professional world. But having said that, when we think 32 00:03:20,800 --> 00:03:27,440 of it from within academia and within higher education circles, the whole concept of authentic 33 00:03:27,440 --> 00:03:32,840 assessments has been around for a really long time. So it was that dichotomy which I wanted 34 00:03:32,840 --> 00:03:37,480 to address saying we as educators are saying that we're making assessments increasingly 35 00:03:37,480 --> 00:03:43,920 authentic across all disciplines. But despite our claims of doing that, it's not being met 36 00:03:43,920 --> 00:03:49,760 by the same opinion on the outside. So what is the reason or potentially multiple reasons, 37 00:03:49,760 --> 00:03:55,360 but what are these and how could they be addressed going forward? That was the real thing that 38 00:03:55,360 --> 00:04:01,000 I want to hammer in on over the course of this fellowship. Fantastic. Could you explain 39 00:04:01,000 --> 00:04:09,160 to listeners a little bit about how authentic assessment delivery differs from authentic 40 00:04:09,160 --> 00:04:16,440 assessment design? So the authentic assessment design is mostly focusing on the design of 41 00:04:16,440 --> 00:04:24,960 the assessment brief or work itself that mimics the work in people doing the professional 42 00:04:24,960 --> 00:04:30,840 world. But that's the brief which we deliver to students in order to start to work on that. 43 00:04:30,840 --> 00:04:37,760 The delivery side of that starts right after that when students start to work on their 44 00:04:37,760 --> 00:04:46,120 assessment. So the whole process from reading the brief to develop the work itself, to communicate 45 00:04:46,120 --> 00:04:52,640 with other peers, to communicate with instructors, to go through different resources to understand 46 00:04:52,640 --> 00:05:01,200 how to solve that problem and also to the final delivery point is the delivery of an 47 00:05:01,200 --> 00:05:06,360 assessment that we have in mind. Would you say then that the relevance of your fellowship 48 00:05:06,360 --> 00:05:13,560 and its findings can transcend your discipline and impact authentic assessment delivery in 49 00:05:13,560 --> 00:05:19,260 other disciplines? Oh yes, very much so. Absolutely. Because if you look at a lot of the recently 50 00:05:19,260 --> 00:05:26,000 published pedagogic research and scholarship articles, potentially critiquing some of the 51 00:05:26,000 --> 00:05:30,960 use of the term authentic assessment because it has become a buzzword but it's potentially 52 00:05:30,960 --> 00:05:38,280 not being used in the real depth and richness that the term encompasses. And that's where 53 00:05:38,280 --> 00:05:42,800 we were looking at it in terms of this design and delivery component as you referred to 54 00:05:42,800 --> 00:05:48,640 Robert. So where we're not just looking at the question or the task which has been given 55 00:05:48,640 --> 00:05:56,080 to the student which might or might not resemble to some extent the challenge of a professional 56 00:05:56,080 --> 00:06:01,320 but it's about how does the student actually go about addressing that challenge. So if 57 00:06:01,320 --> 00:06:05,400 they have a couple of weeks to come up with a report, poster, whatever the assessment 58 00:06:05,400 --> 00:06:11,760 is, how much of that working practice is similar to how a professional would work on a similar 59 00:06:11,760 --> 00:06:17,300 challenge over two weeks time. And that when I say professional pretty much sweeps across 60 00:06:17,300 --> 00:06:22,600 all disciplines. It's not just engineers or scientists but any discipline which is why 61 00:06:22,600 --> 00:06:29,560 I think the project has relevance for everyone. Fantastic. You're one year into your fellowship 62 00:06:29,560 --> 00:06:34,080 and you have preliminary findings. Looking forward to hearing about what they are and 63 00:06:34,080 --> 00:06:39,000 how that's going to inform the second year of your fellowship. Before we go into findings, 64 00:06:39,000 --> 00:06:44,760 I'm sure that listeners are really keen to hear about how you've got to the point that 65 00:06:44,760 --> 00:06:52,440 you're at now. How then did you go about research design and conducting research up until this 66 00:06:52,440 --> 00:06:56,280 point? Well Robert, this is a two-year fellowship 67 00:06:56,280 --> 00:07:01,880 which the first year is the planning phase and the second year would be the pilot phase. 68 00:07:01,880 --> 00:07:07,360 So we are at the end of the first year. For the planning phase, we went after all the 69 00:07:07,360 --> 00:07:13,000 ethical approval and this sort of stuff, we started to do a survey in the Faculty of Engineering 70 00:07:13,000 --> 00:07:19,320 and Physical Science and the survey was designed based on a couple of criterias. For example, 71 00:07:19,320 --> 00:07:25,120 we have something which is called employer indicator framework in university developed 72 00:07:25,120 --> 00:07:32,080 based on a project with industrial and external partners which gave us a good insight at what 73 00:07:32,080 --> 00:07:39,000 are the main criterion that employers are looking when they are searching for a graduate 74 00:07:39,000 --> 00:07:45,560 to do employ. So we took something like peer review, the commercial awareness, reflective 75 00:07:45,560 --> 00:07:53,640 practices, engaging with multiple conflicting requirements like main takeaways from that 76 00:07:53,640 --> 00:08:01,520 project. Also, we look at the broader assessment practice that can replicate the professional 77 00:08:01,520 --> 00:08:09,760 practices in order to bring something also from that like time and project management, 78 00:08:09,760 --> 00:08:14,440 controlling the stress, anxiety, the passion and accomplishment that comes after doing 79 00:08:14,440 --> 00:08:22,440 the project, the creativity and also we looked at the potential solution for that which is 80 00:08:22,440 --> 00:08:31,200 assessment flexibility. We embedded these points into the survey and also we, so most 81 00:08:31,200 --> 00:08:39,000 of these things were embedded through a Likert scale questionnaire and then we added one 82 00:08:39,000 --> 00:08:45,960 open end text based question also in order to have more in depth insight into the perception 83 00:08:45,960 --> 00:08:54,380 of students regarding the assessment and also these criteria. So that phase as I mentioned 84 00:08:54,380 --> 00:08:59,360 has been done, we have responses from students, we have analyzed them which we will speak 85 00:08:59,360 --> 00:09:06,080 about them more in depth today and the second phase, the pilot phase starts from the next 86 00:09:06,080 --> 00:09:15,760 academy year 2024-25 when we, for now we have verified three modules in undergraduate and 87 00:09:15,760 --> 00:09:22,280 also postgraduate levels in order to apply this concept in the assessment and we are 88 00:09:22,280 --> 00:09:30,080 assessing two additional modules to potentially get that to five modules to pilot this and 89 00:09:30,080 --> 00:09:36,080 yeah we are here to discuss findings from the student survey mostly from the planning 90 00:09:36,080 --> 00:09:37,080 phase. 91 00:09:37,080 --> 00:09:45,720 Great, so could you give us a sense then of the kinds of questions students have asked? 92 00:09:45,720 --> 00:09:52,520 Yeah, indeed so we ask questions like I said in terms of the criterion about how much the 93 00:09:52,520 --> 00:09:59,040 submission deadline stress you, we mentioned questions like to what extent do you feel 94 00:09:59,040 --> 00:10:04,660 you can express creativity in your coursework assessment, how much flexibility do you currently 95 00:10:04,660 --> 00:10:10,580 have in your assessment, also we try to understand how, what is the student's perception in 96 00:10:10,580 --> 00:10:17,920 terms of the incentivizing the flexibility for like something like early submission as 97 00:10:17,920 --> 00:10:23,760 something that can be a flexibility, do they prefer that through bonus marks or do they 98 00:10:23,760 --> 00:10:30,800 prefer through the feedback, also we had some because you know we mentioned the authenticity 99 00:10:30,800 --> 00:10:35,620 can be through the design and also delivery so we had some questions in order to verify 100 00:10:35,620 --> 00:10:42,300 the design side of the authenticity in terms of to what extent students are using different 101 00:10:42,300 --> 00:10:50,180 resources for their coursework, how they feel in terms of learning on the go and these kind 102 00:10:50,180 --> 00:10:56,000 of questions and also another part of that like I mentioned was an open end question 103 00:10:56,000 --> 00:11:01,640 which was reflecting on a recent coursework submission comment on what changes could be 104 00:11:01,640 --> 00:11:08,420 made to make the assessment less stressful for you and that would have helped you come 105 00:11:08,420 --> 00:11:13,960 up with a more creative and high quality submission, yeah those were some sample questions. 106 00:11:13,960 --> 00:11:18,560 Brilliant that's a really nice segue because I was just about to ask what did you find 107 00:11:18,560 --> 00:11:21,760 out in this phase of data collection? 108 00:11:21,760 --> 00:11:25,600 Thank you Robert, yeah lots of interesting findings that's come through and I'll probably 109 00:11:25,600 --> 00:11:30,400 go back again to that to the two branches that we were looking at initially which was 110 00:11:30,400 --> 00:11:35,100 authenticity in design and authenticity in delivery. 111 00:11:35,100 --> 00:11:40,800 Students survey across all engineering schools and the sciences within the Faculty of Engineering 112 00:11:40,800 --> 00:11:46,060 and Physical Sciences pretty much unanimously say that students are indeed using a diverse 113 00:11:46,060 --> 00:11:51,640 range of resources to prepare their assessments for instance for coursework so it's not that 114 00:11:51,640 --> 00:11:58,300 they're relying on just lecture notes to prepare things be it posters, reports, whatever form 115 00:11:58,300 --> 00:12:03,600 of assessment so we're really doing a good job in terms of embedding authenticity in 116 00:12:03,600 --> 00:12:05,440 design on that front. 117 00:12:05,440 --> 00:12:10,840 There's still some scope for improvement in terms of more systems thinking competencies 118 00:12:10,840 --> 00:12:16,880 and by systems thinking I mean bringing together various skills it's not just considering the 119 00:12:16,880 --> 00:12:23,620 technical aspect of a question or a task but it's potentially colliding those or the technical 120 00:12:23,620 --> 00:12:29,280 factor alongside economic considerations, business considerations, social considerations 121 00:12:29,280 --> 00:12:31,040 and so on. 122 00:12:31,040 --> 00:12:35,400 So on that front there's still some scope for improvement but by and large authenticity 123 00:12:35,400 --> 00:12:41,920 in design as it has been referred to in literature as well we're doing a very good job. 124 00:12:41,920 --> 00:12:50,920 However as we'd expected authenticity in delivery has a lot to do or a lot to improve in that 125 00:12:50,920 --> 00:12:55,600 sense because for instance when we ask students how often do you engage in peer review and 126 00:12:55,600 --> 00:13:02,800 peer feedback it was a very low percentage of students who said that they often or you 127 00:13:02,800 --> 00:13:08,360 know pretty much always engage in peer review practice and so consequently what this gets 128 00:13:08,360 --> 00:13:13,840 to is the fact that yes as educators we are designing assessment questions that are more 129 00:13:13,840 --> 00:13:19,880 or less resembling challenges in the professional world because students within the same survey 130 00:13:19,880 --> 00:13:24,720 also agree to the fact that they're challenged to learn on the go by these assessments so 131 00:13:24,720 --> 00:13:30,280 it's not that assessments are inauthentic or unauthentic whichever way you call it by 132 00:13:30,280 --> 00:13:36,160 design but it's more that the environment that we're setting up for them to go address 133 00:13:36,160 --> 00:13:42,680 this particular challenge is quite rigid you know it does not give them the freedom to 134 00:13:42,680 --> 00:13:48,080 be creative and it's potentially there where we need to concentrate more of our efforts 135 00:13:48,080 --> 00:13:54,320 to say well how do we kind of remove that rigid structure embrace flexibility and allow 136 00:13:54,320 --> 00:13:59,720 students to be more creative engage in peer review and peer feedback because that's what 137 00:13:59,720 --> 00:14:04,640 people will do graduates would do once they take up any job again irrespective of which 138 00:14:04,640 --> 00:14:09,280 discipline they're going to go into because they're not going to work in silos teamwork 139 00:14:09,280 --> 00:14:15,680 is important so it's how do we you know create that environment which is where kind of all 140 00:14:15,680 --> 00:14:17,900 the findings came together. 141 00:14:17,900 --> 00:14:23,900 Just wanted to build on this in terms of my experience working in industry it's very normal 142 00:14:23,900 --> 00:14:29,440 kind of approach and working routine industry for people to interact with each other on 143 00:14:29,440 --> 00:14:34,360 a daily basis to give each other feedback so that shows in industry people have more 144 00:14:34,360 --> 00:14:43,960 freedom and that freedom gives more passion to kind of show their creativity show their 145 00:14:43,960 --> 00:14:48,840 self in working in different kind of problems you know because the problems in industry 146 00:14:48,840 --> 00:14:54,720 is not like specific problems that they have all similar answers you face with different 147 00:14:54,720 --> 00:15:00,560 kind of challenges and you need to be creative you need to be like a good problem solver 148 00:15:00,560 --> 00:15:05,480 in order to understand how to answer those questions how to kind of find a solution for 149 00:15:05,480 --> 00:15:11,640 those problems and this is what we want to to improve in terms of the education for students 150 00:15:11,640 --> 00:15:17,800 to interact with each other as as managed mention that to work in isolation but to get 151 00:15:17,800 --> 00:15:25,440 out to get feedback from from instructor from peers and use all this interactivity to be 152 00:15:25,440 --> 00:15:31,740 more authentic and to to find solution for like very challenging problems that makes 153 00:15:31,740 --> 00:15:37,280 them completely ready when they get into the into the professional career. 154 00:15:37,280 --> 00:15:42,640 Great and I'll say this because I've because I've read your journal submission which I'd 155 00:15:42,640 --> 00:15:48,960 point listeners towards reading if you'd like to do that please get in touch with Roussein 156 00:15:48,960 --> 00:15:54,400 or Manage and they'll point you towards where to find it and when it's going to be published 157 00:15:54,400 --> 00:16:02,920 the three I suppose teleological aims of assessment that you talk about quite a lot creativity 158 00:16:02,920 --> 00:16:08,960 innovation and passion is it the case then that these aims which link explicitly to the 159 00:16:08,960 --> 00:16:15,440 realities of the workplace should be considered throughout the assessment design and delivery 160 00:16:15,440 --> 00:16:20,680 process not only in your discipline but has relevance to others too. 161 00:16:20,680 --> 00:16:25,160 Absolutely Robert I think you've hit the nail on the head there and it links in very well 162 00:16:25,160 --> 00:16:31,640 with what Mohsen was just talking about so you know broad brush if I were to stereotype 163 00:16:31,640 --> 00:16:36,660 based on the responses that we've got from students again this is very broad by a broad 164 00:16:36,660 --> 00:16:43,200 brush kind of picture making based on the responses that we've got is that as an engineering 165 00:16:43,200 --> 00:16:48,280 or a physical science student or even more broadly we're asking them in terms of course 166 00:16:48,280 --> 00:16:53,120 work here's a question which more or less resembles a professional challenge so they're 167 00:16:53,120 --> 00:16:57,980 going about saying okay we've got a professional challenge to address but I've got two weeks 168 00:16:57,980 --> 00:17:02,840 to produce a report where I cannot share it with my peers because we'd run into academic 169 00:17:02,840 --> 00:17:07,960 integrity issues and this is an individual piece of assessment and I have two weeks as 170 00:17:07,960 --> 00:17:12,720 a solid deadline I need to get as much work as possible done over the course of say the 171 00:17:12,720 --> 00:17:17,720 first week week and a half so I give myself enough time towards the end of the deadline 172 00:17:17,720 --> 00:17:23,320 to perfect that document you know dot the i's cross the t's look at the formatting issues 173 00:17:23,320 --> 00:17:28,440 and then hand it in that is pretty much the majority of student responses if I had to 174 00:17:28,440 --> 00:17:32,880 stereotype it into what kind of assessment experience they're having in terms of that 175 00:17:32,880 --> 00:17:39,680 entire assessment cycle that is the experience that they're having on the whole which is 176 00:17:39,680 --> 00:17:44,400 chalk and cheese when you compare to what Mohsen was just talking about with what happens 177 00:17:44,400 --> 00:17:49,160 in the professional world so if you were working out there and your boss calls you over and 178 00:17:49,160 --> 00:17:54,480 says you have two weeks to address this you're not going to be doing it in exactly the same 179 00:17:54,480 --> 00:17:59,380 way that the student has done it you'd probably take a day or two to curate some data look 180 00:17:59,380 --> 00:18:04,160 at it individually but then you'd go about sharing the draft with your peers potentially 181 00:18:04,160 --> 00:18:09,960 with your seniors try and get some feedback and almost value multiple creative outputs 182 00:18:09,960 --> 00:18:15,040 as opposed to just perfecting one piece of document without even knowing how much value 183 00:18:15,040 --> 00:18:21,680 it has so it's kind of addressing that through the three words that you brought in there 184 00:18:21,680 --> 00:18:27,480 which is about yes being innovative feeling more passionate about it and embedding that 185 00:18:27,480 --> 00:18:33,680 creativity so it's valuing creativity ahead of perfection which unfortunately our rigid 186 00:18:33,680 --> 00:18:39,420 structures rigid assessment structures tend to promote I think it relates to what I was 187 00:18:39,420 --> 00:18:47,160 reading in your manuscript around in the survey many more students referred to a sense of 188 00:18:47,160 --> 00:18:55,360 accomplishment when submitting assessments or assignments than did students who had a 189 00:18:55,360 --> 00:19:02,680 sense of passion or a sense of purpose interestingly you spoke about sense of accomplishment more 190 00:19:02,680 --> 00:19:08,920 as sense of relief could you tell us a bit about why you did that and some insight into 191 00:19:08,920 --> 00:19:15,040 that interpretation sure so I mean when we were designing the survey it was again because 192 00:19:15,040 --> 00:19:20,360 we were thinking of authenticity in delivery so we were thinking about what is the lived 193 00:19:20,360 --> 00:19:25,520 experience of the student when they're preparing the assessment deliverable alongside what 194 00:19:25,520 --> 00:19:31,560 is that overriding emotion let's say once they hand the deliverable in and kind of building 195 00:19:31,560 --> 00:19:36,640 those two questions in which is well how do you feel during the assessments deliverable 196 00:19:36,640 --> 00:19:41,800 period and then how do you feel after submitting the assessment piece kind of lends itself 197 00:19:41,800 --> 00:19:48,120 a little bit from Daniel Kahneman the Nobel Prize winners concept of well experienced 198 00:19:48,120 --> 00:19:54,160 happiness but there's also the self remembering happiness of what do you remember about a 199 00:19:54,160 --> 00:19:59,440 period of time in terms of when you were happy and what we found from the survey results 200 00:19:59,440 --> 00:20:04,720 as you just alluded there Robert is that a high percentage I think close to 70% up in 201 00:20:04,720 --> 00:20:10,400 my head say well they felt a great sense of accomplishment when they handed in the deliverable 202 00:20:10,400 --> 00:20:15,320 and then when we asked them did you feel a sense of passion or purpose when working on 203 00:20:15,320 --> 00:20:22,480 the assessment deliverable that number jumps way down from about 70% around about 40% which 204 00:20:22,480 --> 00:20:27,520 is why we thought well so there is a high chance that a number of these students who 205 00:20:27,520 --> 00:20:32,440 are not feeling that sense of passion or purpose when working on the assessment deliverable 206 00:20:32,440 --> 00:20:38,840 but have this overriding positive emotion when they submit the piece is largely because 207 00:20:38,840 --> 00:20:44,520 they could be engaging with assessment for assessment sake they're here to obtain a 208 00:20:44,520 --> 00:20:51,600 degree and passing an assessment is an important component of it and they're happy that they've 209 00:20:51,600 --> 00:20:56,760 managed to get their submission in on time so it's again trying to see how to be addressed 210 00:20:56,760 --> 00:21:03,800 that divide if 70% are feeling that sense of hopefully authentic accomplishment again 211 00:21:03,800 --> 00:21:10,240 that the same 70% also feel that sense of passion and purpose when working on it. 212 00:21:10,240 --> 00:21:17,320 Yeah and you know this links very well to something that we touched a little bit for 213 00:21:17,320 --> 00:21:22,400 in terms of the guidelines that actually we provide to students so the guidelines we provide 214 00:21:22,400 --> 00:21:28,080 it seems to be mostly restrictive because we ask students to focus on something like 215 00:21:28,080 --> 00:21:31,720 an authentic task. 216 00:21:31,720 --> 00:21:39,640 Having said that I think the guidelines should be clear enough to pave the way for students 217 00:21:39,640 --> 00:21:48,360 in order to provide a work that they can express themselves but at the same time should it 218 00:21:48,360 --> 00:21:55,360 be very descriptive to completely limit them on a specific framework and in this way that 219 00:21:55,360 --> 00:22:04,240 improves the passion for students they will be able to show some creativity and go through 220 00:22:04,240 --> 00:22:11,080 some solutions that might be even beyond the requirement of the task more creative more 221 00:22:11,080 --> 00:22:15,320 challenging when we give them freedom. 222 00:22:15,320 --> 00:22:21,600 Okay and that reminds me of your project title and the importance of flexibility right? 223 00:22:21,600 --> 00:22:26,040 Exactly Robert so making that transition from rigid structures which is what I've been talking 224 00:22:26,040 --> 00:22:32,760 about to embracing more flexibility and there are multiple means that we as educators can 225 00:22:32,760 --> 00:22:38,400 embed flexibility so one as I was just commenting on would be through flexible deadlines and 226 00:22:38,400 --> 00:22:44,320 how we could potentially incentivise student submissions according to these flexible deadlines 227 00:22:44,320 --> 00:22:49,760 working on to the same time scales I was commenting on earlier if it's a two week submission can 228 00:22:49,760 --> 00:22:55,320 we build in an opportunity where a student is able to hand in a draft one week in and 229 00:22:55,320 --> 00:23:00,960 we as instructors can give them feedback that they can effectively feed forward into their 230 00:23:00,960 --> 00:23:06,400 summative submissions so incentivising in terms of flexible deadlines is one option 231 00:23:06,400 --> 00:23:12,440 but that's not the only option the other one which we've also kind of wondered a little 232 00:23:12,440 --> 00:23:17,720 bit about is about assessment format so yes the challenge is set in terms of the assessment 233 00:23:17,720 --> 00:23:23,440 task but it need not always be done through a report it need not always be done through 234 00:23:23,440 --> 00:23:28,360 a presentation it could be done for instance through a pitch deck which might be more commonplace 235 00:23:28,360 --> 00:23:34,360 in industry or start ups so again offering that flexibility to students saying this is 236 00:23:34,360 --> 00:23:39,280 the learning outcome that we want you to meet but how you're going to demonstrate it is 237 00:23:39,280 --> 00:23:46,040 up to you the format is up to you as for the student to choose and likewise with what skills 238 00:23:46,040 --> 00:23:50,920 they want to emphasise in terms of linking in well with the surfacing skills project 239 00:23:50,920 --> 00:23:56,360 here at Leeds as well saying this is the technical skill that we do want you to address because 240 00:23:56,360 --> 00:24:01,880 that is a hardcore learning outcome but what are the other skills that go along with it 241 00:24:01,880 --> 00:24:05,960 offer the flexibility to students to choose that. 242 00:24:05,960 --> 00:24:12,120 Did anything in particular shine through in the open text question of the survey? 243 00:24:12,120 --> 00:24:18,040 Yeah a couple of things that stood out Robert one was the importance that students associate 244 00:24:18,040 --> 00:24:23,800 with timely feedback I think it's something that we speak a lot about as educators but 245 00:24:23,800 --> 00:24:29,640 tend to easily underestimate how much importance students really give to feedback and particularly 246 00:24:29,640 --> 00:24:34,320 formative feedbacks and not this decoupling between formative assessment and summative 247 00:24:34,320 --> 00:24:39,720 assessment but if students can get feedback on a draft version that they're going to hand 248 00:24:39,720 --> 00:24:45,320 in for a summative piece eventually they do tend to really appreciate that. 249 00:24:45,320 --> 00:24:49,520 That was a reoccurring theme on the open text question. 250 00:24:49,520 --> 00:24:57,920 Yeah on that context I was listening to one episode of Light Bites by Gianni Liu that the 251 00:24:57,920 --> 00:25:04,320 title was assessment literacy and student success in the context of language education 252 00:25:04,320 --> 00:25:09,760 so it was a different discipline rather than engineering and interestingly the finding 253 00:25:09,760 --> 00:25:14,760 was the students are asking for more contact and also feedback related to the assessment 254 00:25:14,760 --> 00:25:21,040 literacy and so just wanted to mention is that that fellowship somehow has the same 255 00:25:21,040 --> 00:25:26,040 funding in terms of the feedback similar to what I found from our survey. 256 00:25:26,040 --> 00:25:31,600 And the second point I wanted to touch on yes we need to be more flexible in terms of 257 00:25:31,600 --> 00:25:37,920 can be marking schemes rubrics the assessment briefs but it's also that delicate trade 258 00:25:37,920 --> 00:25:42,720 off because when we were looking at student responses across all year groups starting 259 00:25:42,720 --> 00:25:49,200 from first year of undergraduate study through to postgraduate level we see this delicate 260 00:25:49,200 --> 00:25:55,100 balance between the fact that an assessment brief must have reasonably enough information 261 00:25:55,100 --> 00:26:02,520 in there so that it's clear what the expectations of the assessment are to the students however 262 00:26:02,520 --> 00:26:06,880 it should not be overly prescriptive to the extent that it stifles creativity so it's 263 00:26:06,880 --> 00:26:12,160 about getting that delicate balance right and when we ran reflexive thematic analysis 264 00:26:12,160 --> 00:26:18,600 over the responses that we received for that open text question that theme was very obvious 265 00:26:18,600 --> 00:26:24,200 but that's where again students say that please engage in partnerships with us in terms of 266 00:26:24,200 --> 00:26:28,960 whether it be assessment design or even more high level curriculum design it's kind of 267 00:26:28,960 --> 00:26:35,680 taking students on board looking at what those mutual expectations are and how do we give 268 00:26:35,680 --> 00:26:40,720 them as I said the best possible environment for them to thrive the question the authenticity 269 00:26:40,720 --> 00:26:46,520 in the question is there but how do we create the environment for them to thrive be more 270 00:26:46,520 --> 00:26:50,800 creative and come up with a high quality submission. 271 00:26:50,800 --> 00:26:54,720 So I think it's great to hear you referring to your use of reflexive thematic analysis 272 00:26:54,720 --> 00:26:59,480 there that certainly came through in reading your manuscript especially when you're talking 273 00:26:59,480 --> 00:27:05,400 about what I referred to previously about how you interpret some of the things that 274 00:27:05,400 --> 00:27:10,520 people said whether in the light cut survey or in the open text question for example around 275 00:27:10,520 --> 00:27:19,000 accomplishment and actually relief which I suppose brings us on now to what I'm sure 276 00:27:19,000 --> 00:27:27,640 listeners at home or work interested in knowing how is this going to inform your work going 277 00:27:27,640 --> 00:27:32,540 forwards and your pilot phase in year two of your fellowship. 278 00:27:32,540 --> 00:27:39,680 As I mentioned before the pilot phase start from the October this year so academic year 279 00:27:39,680 --> 00:27:48,560 twenty twenty four twenty five we have a couple of modules in both undergraduate and postgraduate 280 00:27:48,560 --> 00:27:55,360 which means that taught MSc programs we have selected for these pilot phase we are trying 281 00:27:55,360 --> 00:28:03,480 to introduce some ideas from these flexible assessment and also the perception of the 282 00:28:03,480 --> 00:28:11,800 assessment we got from student to provide more feedback more peer review opportunities 283 00:28:11,800 --> 00:28:20,880 and also flexible assessment hand in dates with the incentive of providing feedback. 284 00:28:20,880 --> 00:28:28,720 These are like initial thoughts that we have in terms of getting from the survey to the 285 00:28:28,720 --> 00:28:33,920 modules and to apply in the pilot phase. 286 00:28:33,920 --> 00:28:38,200 Sure to just back up on what Mohsen said so we've identified these modules where we're 287 00:28:38,200 --> 00:28:42,880 going to pilot flexible assessments and we'd be collecting some more data in terms of journey 288 00:28:42,880 --> 00:28:48,580 experiences to go back and compare with the larger student survey that we've of course 289 00:28:48,580 --> 00:28:54,880 just run so that would be the primary source of data and trying to yeah hopefully spread 290 00:28:54,880 --> 00:29:01,000 the word and get more educators across the university take on such practice and in that 291 00:29:01,000 --> 00:29:06,600 sense we've been having conversations with assessment strategy leads both at within the 292 00:29:06,600 --> 00:29:12,000 faculty of EPS but also more broadly at the university level to see how our findings can 293 00:29:12,000 --> 00:29:16,200 feed into upcoming policy guidance. 294 00:29:16,200 --> 00:29:21,880 Excellent and just a note to listeners Manoj and Mohsen will be coming back in a year's 295 00:29:21,880 --> 00:29:30,120 time to share these findings from implementing the pilot and personally we're looking forward 296 00:29:30,120 --> 00:29:34,280 to hearing about how that turns out for you both. 297 00:29:34,280 --> 00:29:41,000 You mentioned Mohsen your plans for disseminating in addition to this podcast and achieving 298 00:29:41,000 --> 00:29:42,000 impact. 299 00:29:42,000 --> 00:29:45,200 Could you tell us a bit more about your plans going forwards? 300 00:29:45,200 --> 00:29:52,560 Yeah for dissemination as you previously mentioned we already wrote a manuscript submitted which 301 00:29:52,560 --> 00:30:00,240 is under review currently and we hope to get a result for that soon to share with listeners. 302 00:30:00,240 --> 00:30:07,120 Also we have plan in order to publish kind of not publish but participate in couple of 303 00:30:07,120 --> 00:30:16,280 conferences in Europe and also abroad in probably in America and Asia to be engaged in different 304 00:30:16,280 --> 00:30:21,880 conversation not only in scale of the Europe but in other scales to understand how these 305 00:30:21,880 --> 00:30:28,220 concept of flexible assessment have been explored and have conversation with other colleagues 306 00:30:28,220 --> 00:30:35,760 in different educational settings to get their feedback to see how we can improve this and 307 00:30:35,760 --> 00:30:43,640 all these to a level as Manoj mentioned to have more educators more lecturers and instructors 308 00:30:43,640 --> 00:30:45,680 on board with their idea. 309 00:30:45,680 --> 00:30:50,680 Great and are there any particular strategic frameworks at the university that you'd be 310 00:30:50,680 --> 00:30:53,000 looking to inform? 311 00:30:53,000 --> 00:30:57,840 Absolutely Robert the one we've identified is the access and student success strategy 312 00:30:57,840 --> 00:31:02,280 links in very well with some of the points that we've just mentioned here on the podcast 313 00:31:02,280 --> 00:31:07,500 as well to particularly the attainment and the progression pillars because we've collected 314 00:31:07,500 --> 00:31:13,880 data as I stated across all year groups we really can look into that granular level detail 315 00:31:13,880 --> 00:31:20,620 of what we need to be doing across different year groups to help them progress and attain. 316 00:31:20,620 --> 00:31:24,120 So those are the things that we are looking at at the moment. 317 00:31:24,120 --> 00:31:30,360 Okay and just taking a step back then how's the overall experience been for you both so 318 00:31:30,360 --> 00:31:33,720 far in your light fellowship? 319 00:31:33,720 --> 00:31:38,240 Brilliant Robert if I had to put it in one word I think just the experience of you know 320 00:31:38,240 --> 00:31:44,320 systematically investigating a pedagogic research question was very interesting certainly learnt 321 00:31:44,320 --> 00:31:49,600 a lot because yeah I come from an engineering background so I do not quite have that hat 322 00:31:49,600 --> 00:31:56,760 as a pedagogic researcher or as a scholarship person but still kind of going through the 323 00:31:56,760 --> 00:32:00,680 data going through the entire experience has been a rich learning curve. 324 00:32:00,680 --> 00:32:05,520 I've had networking opportunities a lot of them through light so I have to thank light 325 00:32:05,520 --> 00:32:12,480 for that through talking to engineers also some staff interviews which we did not mention 326 00:32:12,480 --> 00:32:17,120 over the course this podcast but it's data that we'll be looking at going forward. 327 00:32:17,120 --> 00:32:21,360 So yeah just the fact that light has given me the time and space to think about the project 328 00:32:21,360 --> 00:32:25,760 and potentially influence the practice of others it's just absolutely brilliant. 329 00:32:25,760 --> 00:32:31,800 Yeah same here it was an excellent experience for me too and a big learning curve in terms 330 00:32:31,800 --> 00:32:38,160 of learning how to work with qualitative data because we are engineers literally so working 331 00:32:38,160 --> 00:32:44,600 with digits and quantitative data is something normal for us but qualitative data is a big 332 00:32:44,600 --> 00:32:47,100 learning curve for us. 333 00:32:47,100 --> 00:32:54,600 How to go to different thematic analysis methods the reflexive thematic analysis how to analyze 334 00:32:54,600 --> 00:33:02,760 the data that is not in digits it was very interesting learning curve. 335 00:33:02,760 --> 00:33:08,360 So the challenge of time was a bit evident for myself because I have other commitments 336 00:33:08,360 --> 00:33:15,760 to do in conjunction of this fellowship but overall it was brilliant experience and the 337 00:33:15,760 --> 00:33:22,860 support I had from the light also other as managed mentioned opportunities for networking 338 00:33:22,860 --> 00:33:28,160 and speak with people from other disciplines was amazing. 339 00:33:28,160 --> 00:33:33,560 I'm really glad that for both of you stepping into the world of social science research 340 00:33:33,560 --> 00:33:41,520 has been so far an enriching experience it's not unusual for it to be a difficult process 341 00:33:41,520 --> 00:33:47,040 in terms of adapting to in many ways very different way of seeing data and interpreting 342 00:33:47,040 --> 00:33:52,640 data and generating findings from that. 343 00:33:52,640 --> 00:33:57,280 It's great to hear that learning how to apply reflexive thematic analysis in particular has 344 00:33:57,280 --> 00:34:03,760 been enriching to you I know that my colleague Lucy has been in touch with you supporting 345 00:34:03,760 --> 00:34:08,240 you with that and is currently developing a resource on reflexivity for light fellows 346 00:34:08,240 --> 00:34:12,840 to use and anyone doing pedagogic research. 347 00:34:12,840 --> 00:34:17,960 Great what advice then would you have for other people that are starting out in their 348 00:34:17,960 --> 00:34:22,280 scholarship of teaching and learning or pedagogic research projects? 349 00:34:22,280 --> 00:34:29,360 Probably some suggestion rather than advice you know like I said it makes sense for people 350 00:34:29,360 --> 00:34:34,880 to prepare for that experience in terms of the learning and reading some resources before 351 00:34:34,880 --> 00:34:41,600 going into the work itself and also be aware of some challenges because you know when we 352 00:34:41,600 --> 00:34:49,120 are dealing with this social science approach we mostly go through the route of survey and 353 00:34:49,120 --> 00:34:55,480 questionnaire and this sort of stuff and in higher education I think we have the survey 354 00:34:55,480 --> 00:35:02,720 fatigue somehow for students we do a lot of survey and evaluation so I would say for someone 355 00:35:02,720 --> 00:35:09,840 that tries to enter actually this realm be aware of that survey fatigue not to push students 356 00:35:09,840 --> 00:35:17,920 with a lot of like different surveys because that reduces the number of respondents diminishes 357 00:35:17,920 --> 00:35:19,560 the value of the survey. 358 00:35:19,560 --> 00:35:25,720 Yeah it can definitely be a challenge another shameless plug from another colleague of mine 359 00:35:25,720 --> 00:35:32,160 is he's been developing a resource hosted on our website which has a range of qualitative 360 00:35:32,160 --> 00:35:37,040 data collection methods so for people that want to consider something else and have a 361 00:35:37,040 --> 00:35:43,280 way of learning about an issue that's definitely worth looking into and considering. 362 00:35:43,280 --> 00:35:49,040 Thank you I pretty much second what Mohsen has just said there in terms of more suggestions 363 00:35:49,040 --> 00:35:54,520 than advice really and since he's touched on more specific learnings in terms of the 364 00:35:54,520 --> 00:36:00,800 methodology and the findings for our project I'd probably go more high level and say again 365 00:36:00,800 --> 00:36:07,720 more suggestions than advice would be to have bias towards action especially if you are 366 00:36:07,720 --> 00:36:12,580 not from a very social sciences discipline so in our case for instance as engineers stepping 367 00:36:12,580 --> 00:36:18,720 into a pedagogic research project you can have that imposter syndrome which comes up 368 00:36:18,720 --> 00:36:25,560 more than once throughout the entire project so it's about having that bias towards action 369 00:36:25,560 --> 00:36:31,200 and continuing to look at data feeling confident and you know one tiny step forward and will 370 00:36:31,200 --> 00:36:36,040 eventually given us the results that we have in a year's time and we look forward to seeing 371 00:36:36,040 --> 00:36:38,040 what the next year holds. 372 00:36:38,040 --> 00:36:45,240 Thanks so much that is incredibly valuable feedback and suggestions for people that are 373 00:36:45,240 --> 00:36:50,920 considering undertaking a light fellowship or pedagogic research and thanks to both of 374 00:36:50,920 --> 00:36:56,840 you for being such great light fellows so far and I know I'll speak for everybody at 375 00:36:56,840 --> 00:37:01,120 light in saying that we're really excited to see what's in store in a second year of 376 00:37:01,120 --> 00:37:07,520 your fellowship and again I'll reiterate to everyone listening that Manoj and Mohsen 377 00:37:07,520 --> 00:37:13,040 are very happy to be contacted if you have any questions or would like access to the 378 00:37:13,040 --> 00:37:19,720 journal article or would like to apply the findings from the fellowship into their own 379 00:37:19,720 --> 00:37:24,840 work so thanks again to you both. 380 00:37:24,840 --> 00:37:29,320 Thank you very much Robert absolute pleasure doing this yes if you want to get in touch 381 00:37:29,320 --> 00:37:35,240 with us by all means too because as we said we all here are passionate about student education 382 00:37:35,240 --> 00:37:40,400 so if whatever work we've done can feed into your practice we'd be more than happy to see 383 00:37:40,400 --> 00:37:43,000 that happen thank you very much. 384 00:37:43,000 --> 00:37:48,840 Yes same for me I would be very glad to connect with people if they have any question would 385 00:37:48,840 --> 00:37:56,100 be really interested to respond and to join forces in the mission that we have in order 386 00:37:56,100 --> 00:38:02,520 to educate a generation that could be useful for our world and also to have great contributions 387 00:38:02,520 --> 00:38:05,160 yeah glad to be here thank you so much. 388 00:38:05,160 --> 00:38:09,760 Brilliant thank you and thanks to everybody listening to this episode of Light Bites just 389 00:38:09,760 --> 00:38:16,760 a reminder that we'll be revisiting the final findings of this fellowship this time next 390 00:38:16,760 --> 00:38:39,720 year thanks for listening and see you next time.