1 00:00:30,000 --> 00:00:39,860 Well, hello again, everybody, and welcome back to Episode 8 of Season 3 of the Polka 2 00:00:39,860 --> 00:00:40,860 Podcast. 3 00:00:40,860 --> 00:00:44,320 It's hard to believe that we're already on Episode 8. 4 00:00:44,320 --> 00:00:49,440 Seems like we started yesterday, but we still have a few more to go, so keep on listening. 5 00:00:49,440 --> 00:00:53,240 So our interviewer today is Siobhan Julian. 6 00:00:53,240 --> 00:00:54,240 Hello, Siobhan. 7 00:00:54,240 --> 00:00:55,240 Hello, Wayne. 8 00:00:55,240 --> 00:00:58,460 I'm really excited about this episode today. 9 00:00:58,460 --> 00:01:04,020 We are talking about metacognition, and with us today to talk about metacognition are Mary 10 00:01:04,020 --> 00:01:06,480 Van Obstel and Marty Perry. 11 00:01:06,480 --> 00:01:12,380 Mary has a PhD in chemistry education with a dissertation on students' metacognitive 12 00:01:12,380 --> 00:01:14,080 strategy use in chem lab. 13 00:01:14,080 --> 00:01:16,720 So what a perfect guest for us today. 14 00:01:16,720 --> 00:01:22,320 She teaches introductory chemistry at two community colleges, Harbor College and McHenry 15 00:01:22,320 --> 00:01:28,920 County College in Northern Illinois, and she's taught several PD courses on evidence-based 16 00:01:28,920 --> 00:01:31,080 strategies to other teachers. 17 00:01:31,080 --> 00:01:35,440 So she has a background in working with other teachers on this. 18 00:01:35,440 --> 00:01:41,740 She's been involved in the Poggle projects since 2015, been using Poggle since 2012, 19 00:01:41,740 --> 00:01:47,020 and her goal as a teacher is to make students aware of their own learning, perfect for metacognition 20 00:01:47,020 --> 00:01:51,560 for our conversation today, and to help them become science literate. 21 00:01:51,560 --> 00:01:56,080 Marty Perry is the associate professor and vice chair for professional education at the 22 00:01:56,080 --> 00:02:02,040 College of Pharmacy at the University of Arkansas for Medical Sciences in Little Rock, Arkansas. 23 00:02:02,040 --> 00:02:04,560 And Marty is an OG Poggler. 24 00:02:04,560 --> 00:02:10,740 He was introduced to Poggle in 1997, which is the year that I graduated high school. 25 00:02:10,740 --> 00:02:16,000 So you know, yeah, Marty, you should feel a little old there. 26 00:02:16,000 --> 00:02:21,400 His career spans facilitating learning in both high school, undergraduate, and professional 27 00:02:21,400 --> 00:02:22,400 programs. 28 00:02:22,400 --> 00:02:26,200 He has really taught at all levels and he's used Poggle at all levels. 29 00:02:26,200 --> 00:02:32,000 So he is one of the absolute pillars of the Poggle project, and he definitely, I can't 30 00:02:32,000 --> 00:02:35,500 wait to hear what he has to say today about metacognition. 31 00:02:35,500 --> 00:02:39,800 So yeah, metacognition, let's get into this. 32 00:02:39,800 --> 00:02:43,600 When we talk about the process skills, I feel like there's always one process skill that 33 00:02:43,600 --> 00:02:48,560 gets sort of shunted to the side when we're having students work in teams, and that's 34 00:02:48,560 --> 00:02:49,560 metacognition. 35 00:02:49,560 --> 00:02:56,160 Personally, for me, out of all the process skills, metacognition is the one that I struggle 36 00:02:56,160 --> 00:02:57,360 with the most. 37 00:02:57,360 --> 00:03:02,440 It's the one that I tend to do the least with in my classroom, which is why it's so awesome 38 00:03:02,440 --> 00:03:08,640 that we have Mary and Marty, who are experts in dealing with this, who have strategies, 39 00:03:08,640 --> 00:03:12,080 who know what they're talking about when it comes to metacognition here today to share 40 00:03:12,080 --> 00:03:15,400 that so we can really dive deeper into this. 41 00:03:15,400 --> 00:03:22,160 So first question, we know that metacognition is essentially thinking about thinking, but 42 00:03:22,160 --> 00:03:24,660 I'd love to go a little bit deeper into the definition. 43 00:03:24,660 --> 00:03:26,920 How do you define metacognition? 44 00:03:26,920 --> 00:03:29,600 And Marty, I'll go to you first. 45 00:03:29,600 --> 00:03:30,600 Well thank you, Siobhan. 46 00:03:30,600 --> 00:03:38,540 It's great to be here with you and Mary to have some conversation today about this topic. 47 00:03:38,540 --> 00:03:39,640 You did get one thing right. 48 00:03:39,640 --> 00:03:40,640 I am old. 49 00:03:40,640 --> 00:03:46,240 I've been around the project for a long time. 50 00:03:46,240 --> 00:03:51,560 And it's really through that involvement with the project and all the great people that 51 00:03:51,560 --> 00:03:57,160 I've had the chance to work with over my career that have influenced what I know and try to 52 00:03:57,160 --> 00:04:02,960 adopt and adapt into my own curriculum, including metacognition. 53 00:04:02,960 --> 00:04:08,440 But simply, I think I would define it as you're self-aware. 54 00:04:08,440 --> 00:04:11,400 You really know what you're thinking about. 55 00:04:11,400 --> 00:04:16,680 And for a student or a learner, you're employing effective strategies to help you study and 56 00:04:16,680 --> 00:04:17,680 learn. 57 00:04:17,680 --> 00:04:22,200 You're not doing things that are ineffective, or if you are, you at least recognize they're 58 00:04:22,200 --> 00:04:26,480 ineffective and you try to adapt and do things better the next time. 59 00:04:26,480 --> 00:04:31,400 You recognize that something's not working and you go and visit with another peer or 60 00:04:31,400 --> 00:04:37,200 an instructor, somebody that's going to help you do better the next time. 61 00:04:37,200 --> 00:04:44,120 So a learner should be metacognitive, should be self-aware, and recognize when you know 62 00:04:44,120 --> 00:04:46,400 something and when you don't know something. 63 00:04:46,400 --> 00:04:54,120 And it's hard for novice learners to be good at that, probably even hard for more experienced 64 00:04:54,120 --> 00:05:00,240 learners to continue to adapt our strategies, especially as the situation may dictate, having 65 00:05:00,240 --> 00:05:04,200 a different strategy than what you might be accustomed to employing. 66 00:05:04,200 --> 00:05:06,040 Yeah, that's great. 67 00:05:06,040 --> 00:05:07,040 Thank you so much for that. 68 00:05:07,040 --> 00:05:08,640 Mary, what do you think? 69 00:05:08,640 --> 00:05:11,600 How do you define metacognition? 70 00:05:11,600 --> 00:05:15,200 So when I think about it, it's absolutely being self-aware. 71 00:05:15,200 --> 00:05:18,920 And you really, Marty, you really hit on a lot of pieces and I want to put some words. 72 00:05:18,920 --> 00:05:23,380 So planning, monitoring, and evaluating strategies. 73 00:05:23,380 --> 00:05:29,120 So those are really what you're doing as a teacher, what students are doing when they're 74 00:05:29,120 --> 00:05:30,120 studying. 75 00:05:30,120 --> 00:05:31,160 Are they planning ahead? 76 00:05:31,160 --> 00:05:34,000 Are they monitoring that they get something or don't get something? 77 00:05:34,000 --> 00:05:39,480 Or chemists are running an experiment and something's not working. 78 00:05:39,480 --> 00:05:43,520 Are they monitoring and recognizing that it's actually, you know, that they need to do that 79 00:05:43,520 --> 00:05:46,680 and then evaluate at the end about that. 80 00:05:46,680 --> 00:05:50,920 So I'm always thinking when it all comes down to it, I'm always thinking about planning, 81 00:05:50,920 --> 00:05:54,520 monitoring, and evaluating for any of those things that you mentioned, Marty. 82 00:05:54,520 --> 00:05:59,320 And in relation to what the students are doing, and then I also, we have to be metacognitive 83 00:05:59,320 --> 00:06:04,400 as well to make sure our class is matching what we say it's matching and what the students 84 00:06:04,400 --> 00:06:05,400 are expecting. 85 00:06:05,400 --> 00:06:10,040 So it's a lot of transparency, I think that helps with that. 86 00:06:10,040 --> 00:06:13,800 I'm already getting good ideas from the two of you, so thank you. 87 00:06:13,800 --> 00:06:15,920 Yes, that's what we want. 88 00:06:15,920 --> 00:06:20,760 Yeah, because I had a student come to me and say that they weren't sure how to study. 89 00:06:20,760 --> 00:06:24,560 And you know, given everything that's happened with COVID, that makes a lot of sense. 90 00:06:24,560 --> 00:06:27,960 And I was trying to come up with some things, but I realized I wasn't being terribly helpful. 91 00:06:27,960 --> 00:06:32,000 But I already feel like I can now go back to that student and say, okay, well, we got 92 00:06:32,000 --> 00:06:33,000 some ideas. 93 00:06:33,000 --> 00:06:35,560 So thank you already. 94 00:06:35,560 --> 00:06:39,800 So with all that, with that great, I love how you put it, being self-aware, knowing 95 00:06:39,800 --> 00:06:43,360 that you know something or when you don't know something. 96 00:06:43,360 --> 00:06:49,440 When you have a group of new students, either a full year course or a semester long college 97 00:06:49,440 --> 00:06:52,320 course, when do you introduce metacognition to your students? 98 00:06:52,320 --> 00:06:55,500 And what does that introduction look like? 99 00:06:55,500 --> 00:06:57,040 So Mary, I'll start with you. 100 00:06:57,040 --> 00:07:01,100 Okay, so when I first started, I was just incorporating in the lab, because that's where 101 00:07:01,100 --> 00:07:03,000 my dissertation was. 102 00:07:03,000 --> 00:07:08,080 And so I really, we really focused on, we talked about it just briefly kind of as you 103 00:07:08,080 --> 00:07:13,040 know, some slides, but we kind of just looked at what students did. 104 00:07:13,040 --> 00:07:16,840 But now when I think about it, I usually talk about it right at the beginning, because it's 105 00:07:16,840 --> 00:07:21,880 literally part of my convinced students that Poggle is, you know, is great. 106 00:07:21,880 --> 00:07:23,480 And it will help you learn. 107 00:07:23,480 --> 00:07:27,760 So I have a whole first day slide of the things that I have them do. 108 00:07:27,760 --> 00:07:29,680 So I usually talk about it. 109 00:07:29,680 --> 00:07:36,360 And it shifted from being some slides to part of an exam reflection after their first exam, 110 00:07:36,360 --> 00:07:39,520 to now it's a part of all your formative assessments. 111 00:07:39,520 --> 00:07:44,000 So whether it's a quiz or taking in class, whether it's a Poggle activity, like, did 112 00:07:44,000 --> 00:07:46,080 you get what happened? 113 00:07:46,080 --> 00:07:50,840 And so I think it really meshes nicely with the idea of formative assessments, which we 114 00:07:50,840 --> 00:07:55,880 as teachers are doing all, you know, really all the time, you know, all during class, 115 00:07:55,880 --> 00:07:58,880 we're checking in, are they, are they getting it? 116 00:07:58,880 --> 00:08:01,080 So you have to ask them, are they getting it? 117 00:08:01,080 --> 00:08:08,720 You know, so I think of it that way about how, you know, when we're introducing metacognition, 118 00:08:08,720 --> 00:08:14,200 and I do use that term, but I've incorporated it to include reflection and kind of planning 119 00:08:14,200 --> 00:08:15,720 and things like that. 120 00:08:15,720 --> 00:08:21,840 So the word comes up, but I generally think about it in terms of, you know, are they, 121 00:08:21,840 --> 00:08:23,800 you know, formative, are you assessing yourself? 122 00:08:23,800 --> 00:08:24,800 What do you know? 123 00:08:24,800 --> 00:08:28,660 What evidence do you have that you know this thing? 124 00:08:28,660 --> 00:08:31,040 And what do some of those formative assessments look like? 125 00:08:31,040 --> 00:08:35,840 So you said that you have them, like with a regular quiz or regular test. 126 00:08:35,840 --> 00:08:37,880 So what might that look like? 127 00:08:37,880 --> 00:08:41,320 So one, one that I do every day is retrieval practice. 128 00:08:41,320 --> 00:08:43,960 It's one of the best evidence-based teaching strategies out there. 129 00:08:43,960 --> 00:08:48,120 And when they come in, students have to sit down and they have to answer this question 130 00:08:48,120 --> 00:08:53,760 without notes with, you know, and then either they can answer it and I say, okay, great, 131 00:08:53,760 --> 00:08:55,720 check your notes, make sure that it's right. 132 00:08:55,720 --> 00:08:58,460 Or if they don't know, then they need to go back to their notes and find it. 133 00:08:58,460 --> 00:09:03,560 So it's giving them different levels of, do I know this at all? 134 00:09:03,560 --> 00:09:07,640 Like we did this last class, you should, you know, you should have learned it in some capacity. 135 00:09:07,640 --> 00:09:11,380 And if that, so it's kind of that's, that's one. 136 00:09:11,380 --> 00:09:16,000 And then I have these bigger quizzes that are more over a unit that I treat as kind 137 00:09:16,000 --> 00:09:17,560 of self-checks. 138 00:09:17,560 --> 00:09:21,760 So in terms of I don't grade them and that was, it hits on the alternative grading podcast 139 00:09:21,760 --> 00:09:28,160 last week is that they're more for revision and checking what you know, rather than this 140 00:09:28,160 --> 00:09:32,180 is a summative assessment because it's not like they're still learning the material. 141 00:09:32,180 --> 00:09:34,720 So I treat that and then we do a lot of revision in class. 142 00:09:34,720 --> 00:09:40,880 So I have them revise their quizzes and go back and they have to tell me how do I know, 143 00:09:40,880 --> 00:09:42,920 you know, or I'm like, how do you know it's the right answer now? 144 00:09:42,920 --> 00:09:46,800 And they could say, well, I checked my notes or I went back and did this. 145 00:09:46,800 --> 00:09:51,240 And so it's trying to get them to go back and see why they didn't get it the first time. 146 00:09:51,240 --> 00:09:56,880 And it's a lot of positive, I think consistent language that I use to really engage them 147 00:09:56,880 --> 00:09:57,880 in that. 148 00:09:57,880 --> 00:10:00,680 Yeah, that's great. 149 00:10:00,680 --> 00:10:06,420 Marty, having taught at both high school and secondary institutions, what about you? 150 00:10:06,420 --> 00:10:10,800 When do you introduce metacognition and what does that look like? 151 00:10:10,800 --> 00:10:17,480 Well, if I could just say ditto to Mary's answers, we could probably move right on because 152 00:10:17,480 --> 00:10:19,560 I absolutely agree. 153 00:10:19,560 --> 00:10:21,400 It has to be day one. 154 00:10:21,400 --> 00:10:26,960 It's part of setting the stage and helping students understand what it's going to take 155 00:10:26,960 --> 00:10:33,600 to be successful learning in an environment for which a lot of students are not necessarily 156 00:10:33,600 --> 00:10:34,600 accustomed. 157 00:10:34,600 --> 00:10:35,600 Right. 158 00:10:35,600 --> 00:10:39,320 They're used to the traditional sorts of formats for classrooms. 159 00:10:39,320 --> 00:10:42,120 And of course, Pogel does not look like that. 160 00:10:42,120 --> 00:10:47,740 So I want them to understand from the very beginning that things are going to work differently 161 00:10:47,740 --> 00:10:51,080 and that's going to require them to think more than what they've been accustomed to 162 00:10:51,080 --> 00:10:54,400 thinking at least during class time. 163 00:10:54,400 --> 00:10:58,680 And so the more I can get them attuned to that and to think about their own thinking 164 00:10:58,680 --> 00:11:07,120 from the very beginning, I'll just list metacognition or being metacognitive as a key to success 165 00:11:07,120 --> 00:11:09,600 in this learning environment. 166 00:11:09,600 --> 00:11:14,400 It's just one of a few things that I really point out from the beginning. 167 00:11:14,400 --> 00:11:19,280 And a lot of the things that Mary has described as assessment strategies to get them to do 168 00:11:19,280 --> 00:11:26,440 that are things that I also do throughout the course, but I can really want them from 169 00:11:26,440 --> 00:11:29,400 the beginning to understand the importance. 170 00:11:29,400 --> 00:11:31,680 So let's say, and I'll direct this towards Marty. 171 00:11:31,680 --> 00:11:36,840 So let's say you've got students coming in who have most likely never had a system like 172 00:11:36,840 --> 00:11:38,840 this in place in their classes. 173 00:11:38,840 --> 00:11:45,080 How are you going to model to them what you mean when you say, how do I know if I know 174 00:11:45,080 --> 00:11:46,080 it? 175 00:11:46,080 --> 00:11:47,080 How do I know if I don't know it? 176 00:11:47,080 --> 00:11:51,040 How do you, what are things that you as the educator will do to help the students sort 177 00:11:51,040 --> 00:11:52,640 of get started on that path? 178 00:11:52,640 --> 00:12:01,120 Well, again, I think it's good for them to see us as individuals and to identify examples 179 00:12:01,120 --> 00:12:07,440 in our own day-to-day practice where we have to use metacognition, right? 180 00:12:07,440 --> 00:12:12,320 So we're thinking about what we're doing and what's working well and what's not working 181 00:12:12,320 --> 00:12:16,840 well or what I know about this topic or what I have to learn that's come out new in my 182 00:12:16,840 --> 00:12:24,520 field that I now have to find a way to change my approach and what I'm doing in my classroom 183 00:12:24,520 --> 00:12:26,200 as a result of that. 184 00:12:26,200 --> 00:12:34,880 So I think modeling for them how those of us who are in front of them are being metacognitive 185 00:12:34,880 --> 00:12:36,560 is important. 186 00:12:36,560 --> 00:12:44,280 But I think getting them to do these retrieval practice exercise that Mary described and 187 00:12:44,280 --> 00:12:45,800 to do that routinely, right? 188 00:12:45,800 --> 00:12:50,640 This is not once every three or four weeks I need to be metacognitive and see how I've 189 00:12:50,640 --> 00:12:51,640 done. 190 00:12:51,640 --> 00:12:57,800 So I think it's a daily study habit that they need to get accustomed to early on and practice 191 00:12:57,800 --> 00:12:59,920 over and over and over again. 192 00:12:59,920 --> 00:13:03,120 Just like any other skill, if you want to get good at it, you've got to practice it 193 00:13:03,120 --> 00:13:04,280 frequently. 194 00:13:04,280 --> 00:13:08,760 So if they're not practicing metacognitive behaviors on a regular basis, they're not 195 00:13:08,760 --> 00:13:13,360 going to be any better at it than any other skill, content or process skill that you would 196 00:13:13,360 --> 00:13:15,280 want them to master. 197 00:13:15,280 --> 00:13:17,680 All right. 198 00:13:17,680 --> 00:13:18,680 So next question. 199 00:13:18,680 --> 00:13:21,560 Well, we're here on the POGLE podcast after all. 200 00:13:21,560 --> 00:13:24,880 So let's talk about POGLE and metacognition. 201 00:13:24,880 --> 00:13:30,040 How can POGLE activities facilitate the development of metacognitive skills? 202 00:13:30,040 --> 00:13:32,080 And I'll go to Marty first on this one. 203 00:13:32,080 --> 00:13:34,280 That's a great question. 204 00:13:34,280 --> 00:13:42,080 And again, I think it starts with the facilitation of the activities and what you do as a facilitator 205 00:13:42,080 --> 00:13:43,160 of those activities. 206 00:13:43,160 --> 00:13:48,880 So I mean, there might be some built in questions to the activities that encourage metacognition, 207 00:13:48,880 --> 00:13:52,120 but I think your role as the facilitator. 208 00:13:52,120 --> 00:13:55,880 And again, are you doing some of the things that Mary described just a few moments ago 209 00:13:55,880 --> 00:13:59,800 on a routine basis to encourage that? 210 00:13:59,800 --> 00:14:05,240 I also think the reflector role, which we model in a lot of our workshops and meetings 211 00:14:05,240 --> 00:14:11,320 that we have in POGLE, or maybe you call it strategy analyst or something like that, that 212 00:14:11,320 --> 00:14:16,320 is a role that is critical, I think, to metacognitive advancement. 213 00:14:16,320 --> 00:14:21,320 If we want our students to practice that role, that's going to help. 214 00:14:21,320 --> 00:14:27,360 So as a reflector in my classroom, they have a sheet that they are required to fill out 215 00:14:27,360 --> 00:14:30,240 and submit usually after class. 216 00:14:30,240 --> 00:14:35,360 So I don't require an immediate, you've got to get this filled out before you leave class. 217 00:14:35,360 --> 00:14:41,240 But before they come back to the next class, I want them to address three major points. 218 00:14:41,240 --> 00:14:47,240 Number one, can you summarize the important concepts that were learned in class that day? 219 00:14:47,240 --> 00:14:52,240 They don't have to be long and drawn out on any of these explanations, but something more 220 00:14:52,240 --> 00:14:58,480 than just the bolded headings of the activity, you know, something that indicates they've 221 00:14:58,480 --> 00:15:03,680 actually learned something about whatever the topic was and the content of that activity. 222 00:15:03,680 --> 00:15:10,440 Secondly, I ask them to reflect on whatever the key process skill was for that activity. 223 00:15:10,440 --> 00:15:15,640 So if it was problem solving, I'm asking them to think about what did their team do well 224 00:15:15,640 --> 00:15:21,040 that would illustrate problem solving and also think about what they might do better 225 00:15:21,040 --> 00:15:25,480 next time, what's a way that they can improve their problem solving skills. 226 00:15:25,480 --> 00:15:32,440 And again, that particular skill will be different every class period typically, so that each 227 00:15:32,440 --> 00:15:37,160 time they reflect, it's not always on the same process skill. 228 00:15:37,160 --> 00:15:41,280 And then lastly, I'm asking them for where they're still having difficulty. 229 00:15:41,280 --> 00:15:43,520 What's the muddiest point? 230 00:15:43,520 --> 00:15:44,640 Where are you still uncertain? 231 00:15:44,640 --> 00:15:48,160 So again, thinking about what they don't know well. 232 00:15:48,160 --> 00:15:53,860 That also is great information for me because then it allows when I come back the next time 233 00:15:53,860 --> 00:16:00,520 to address areas of concern that I see for lots and lots of teams in my classroom. 234 00:16:00,520 --> 00:16:04,880 So that's something that I do on a daily basis with activities. 235 00:16:04,880 --> 00:16:10,960 Yeah, I like that taking that analyst role and really drilling down some of the metacognitive 236 00:16:10,960 --> 00:16:12,680 natures there. 237 00:16:12,680 --> 00:16:15,120 Mary, what about you? 238 00:16:15,120 --> 00:16:19,520 How do Poggle activities in your classroom help to facilitate metacognition? 239 00:16:19,520 --> 00:16:23,840 So I go back to the idea of planning, monitoring, and evaluating. 240 00:16:23,840 --> 00:16:28,920 And so Marty talked a lot about the evaluating process and I think planning because they're 241 00:16:28,920 --> 00:16:33,360 in a group and they all have to answer the same question, they might consider like, are 242 00:16:33,360 --> 00:16:34,800 we getting the correct answer? 243 00:16:34,800 --> 00:16:36,400 You know, are we doing this right? 244 00:16:36,400 --> 00:16:40,480 If there's some confusion or if you mean as a facilitator, you might notice that one group 245 00:16:40,480 --> 00:16:45,000 is taking forever on a question and you realize like they just are stuck. 246 00:16:45,000 --> 00:16:49,560 They've kind of they've stopped, they've monitored themselves, but at some point they just haven't 247 00:16:49,560 --> 00:16:52,560 gotten to that we need to do something about this and move on. 248 00:16:52,560 --> 00:16:54,200 You know, they kind of are wrapped up. 249 00:16:54,200 --> 00:17:02,480 And so I focus a lot on, you know, kind of the times where you, you know, the check-ins, 250 00:17:02,480 --> 00:17:06,320 you know, are the keys on some of the Poggle activities where it says, did everyone get 251 00:17:06,320 --> 00:17:07,320 the right answer? 252 00:17:07,320 --> 00:17:11,600 And so sometimes I use those as, you know, we write it up on the board and everybody, 253 00:17:11,600 --> 00:17:15,780 we know everybody's in the same place, but sometimes I do individual kind of check-ins 254 00:17:15,780 --> 00:17:21,040 to say like, how comfortable are you as an individual with this answer? 255 00:17:21,040 --> 00:17:24,780 Because obviously in a group, not everybody tracks at the same, you know, there's one 256 00:17:24,780 --> 00:17:30,200 person who's like, I know everything and somebody else who very much feels like I am slowly 257 00:17:30,200 --> 00:17:31,200 getting this. 258 00:17:31,200 --> 00:17:36,360 And I think it's important as a teacher to recognize where your students are, because 259 00:17:36,360 --> 00:17:42,060 you know, they're not all have the exact same flow, you know, and it's great when you do 260 00:17:42,060 --> 00:17:45,480 have a group that just flows together, but a lot of times they don't. 261 00:17:45,480 --> 00:17:49,760 So that's kind of where I focus a lot on the monitoring. 262 00:17:49,760 --> 00:17:57,280 And you know, on and off, I've done evaluating reflectors reports in that way, but it fluctuates. 263 00:17:57,280 --> 00:17:59,720 I've been mostly online for the last two years. 264 00:17:59,720 --> 00:18:05,920 I'm finally back in person, so I've had to like relearn how to do parts of Pogle again. 265 00:18:05,920 --> 00:18:08,760 So that's kind of where I land on the Pogle activities. 266 00:18:08,760 --> 00:18:14,480 They're super great for monitoring their learning, at least in the beginning. 267 00:18:14,480 --> 00:18:17,920 My favorite thing of what Mary just said, other than it all being awesome is that idea 268 00:18:17,920 --> 00:18:23,120 of with the key points being like rate yourself individually, not as a team. 269 00:18:23,120 --> 00:18:27,360 Like we know we all got there as a team, just where are you right now? 270 00:18:27,360 --> 00:18:28,360 Like do you get it? 271 00:18:28,360 --> 00:18:29,360 Do you kind of get it? 272 00:18:29,360 --> 00:18:32,200 Have you gotten there on your own or did you need the team to help you out? 273 00:18:32,200 --> 00:18:34,160 I like that a lot. 274 00:18:34,160 --> 00:18:39,880 And I just want to say related to that, I put little confidence emojis. 275 00:18:39,880 --> 00:18:44,800 So I also ask them, those are more on quizzes, but it gets them to say, are they confident 276 00:18:44,800 --> 00:18:45,800 or not confident? 277 00:18:45,800 --> 00:18:51,320 And so that gives them a little themselves a bit of an idea about, oh wow, this question 278 00:18:51,320 --> 00:18:55,080 looks super confusing and maybe they get it right and they rated themselves low. 279 00:18:55,080 --> 00:19:02,960 So trying to work on that idea of confidence and related because it is related to metacognitive 280 00:19:02,960 --> 00:19:03,960 strategies. 281 00:19:03,960 --> 00:19:10,880 Well, also just the idea of putting emojis on papers for high school students automatically 282 00:19:10,880 --> 00:19:13,280 would make everything 10 times cooler. 283 00:19:13,280 --> 00:19:14,720 So that is great. 284 00:19:14,720 --> 00:19:16,040 I love that idea. 285 00:19:16,040 --> 00:19:17,040 I love that data. 286 00:19:17,040 --> 00:19:18,040 Right. 287 00:19:18,040 --> 00:19:19,440 How confident are you that this is correct? 288 00:19:19,440 --> 00:19:24,520 Do you know this or are you still having questions about that? 289 00:19:24,520 --> 00:19:29,480 So if your students are practicing strong metacognition skills, if you feel like, yes, 290 00:19:29,480 --> 00:19:38,480 they are very good at planning, monitoring and evaluating, what would that look like 291 00:19:38,480 --> 00:19:39,480 in your classrooms? 292 00:19:39,480 --> 00:19:40,840 What would they be doing? 293 00:19:40,840 --> 00:19:44,320 What kinds of conversations are they having? 294 00:19:44,320 --> 00:19:46,360 What is that going to really be looking like? 295 00:19:46,360 --> 00:19:48,360 So Mary, I'll start with you. 296 00:19:48,360 --> 00:19:49,360 Okay. 297 00:19:49,360 --> 00:19:52,560 So I have a little list of things I think about. 298 00:19:52,560 --> 00:19:53,560 Go for it. 299 00:19:53,560 --> 00:19:58,680 Attempting to answer any question provided, whether or not, you know, and maybe it's not 300 00:19:58,680 --> 00:20:02,640 raising their hand, but, you know, writing it in their notes, looking back at their notes 301 00:20:02,640 --> 00:20:05,480 to check their answers, making changes. 302 00:20:05,480 --> 00:20:10,000 So in the lab, making changes to a lab procedure that's not working or thinking about stopping 303 00:20:10,000 --> 00:20:12,360 and being like, this is not going well. 304 00:20:12,360 --> 00:20:16,640 I need to do something instead of keep going at setting up. 305 00:20:16,640 --> 00:20:19,840 A big thing is like having them set up their own procedures for some things. 306 00:20:19,840 --> 00:20:25,720 If you have the ability in your class to use some very guided inquiry labs, use some very 307 00:20:25,720 --> 00:20:28,560 open-ended labs like that. 308 00:20:28,560 --> 00:20:33,160 And so students would be experiencing, a lot of times just letting them figure stuff out, 309 00:20:33,160 --> 00:20:36,360 like giving them the time to think in class. 310 00:20:36,360 --> 00:20:38,840 And it's hard because we want to cover so much. 311 00:20:38,840 --> 00:20:42,920 Sometimes we just have to let them hang and let them figure out for themselves what's 312 00:20:42,920 --> 00:20:43,920 going on. 313 00:20:43,920 --> 00:20:47,600 So I always think the conversations that go between like, we're figuring this thing out 314 00:20:47,600 --> 00:20:51,120 and then talking a little bit, you know, and having, just seeing them interact. 315 00:20:51,120 --> 00:20:55,680 And, you know, just like you're trying to solve a problem anywhere, even if it was at 316 00:20:55,680 --> 00:20:58,720 home, but it's in the lab. 317 00:20:58,720 --> 00:21:02,960 And then like outside the class, I kind of think it's like testing themselves on practice 318 00:21:02,960 --> 00:21:06,840 problems, explaining to another student, you know, kind of letting that peer interaction 319 00:21:06,840 --> 00:21:09,200 happen in revising assignments. 320 00:21:09,200 --> 00:21:13,800 If given the opportunity, you know, if they have a quiz to revise or a lab report to revise 321 00:21:13,800 --> 00:21:17,960 that they take that up and attempt to make changes to it. 322 00:21:17,960 --> 00:21:24,080 Cause they recognize there was an issue and that they could fix it. 323 00:21:24,080 --> 00:21:25,320 That is a fantastic list. 324 00:21:25,320 --> 00:21:28,760 I love that you had shared that list with us. 325 00:21:28,760 --> 00:21:32,840 Marty, your kids are doing great at metacognition. 326 00:21:32,840 --> 00:21:34,640 What is that going to look like? 327 00:21:34,640 --> 00:21:35,640 Yeah. 328 00:21:35,640 --> 00:21:36,640 Mary's list is awesome. 329 00:21:36,640 --> 00:21:42,880 The only things, the only things I would emphasize or maybe add to it, they're questioning each 330 00:21:42,880 --> 00:21:44,320 other. 331 00:21:44,320 --> 00:21:49,160 So they're not just saying, oh, well, he knows the answer and I'm going to write that down. 332 00:21:49,160 --> 00:21:54,240 That they really do reflect on whether or not that makes sense to them. 333 00:21:54,240 --> 00:21:55,240 Right? 334 00:21:55,240 --> 00:22:00,240 So they're continually thinking about what somebody else has offered as an idea and trying 335 00:22:00,240 --> 00:22:05,520 to assess whether or not it's correct in terms of their own way of thinking about it. 336 00:22:05,520 --> 00:22:10,560 And they're self correcting, whether it's individually or as a team, when they get to 337 00:22:10,560 --> 00:22:15,040 question eight and they realize that question five is wrong, they're going back and they're 338 00:22:15,040 --> 00:22:17,680 fixing it. 339 00:22:17,680 --> 00:22:21,320 You as the facilitator are not having to fix it for them. 340 00:22:21,320 --> 00:22:24,680 They recognize when they've made a mistake. 341 00:22:24,680 --> 00:22:30,520 So they're definitely practicing metacognition and that's scenario, I believe. 342 00:22:30,520 --> 00:22:35,560 And then the outside of the classroom environment, I think is just as critical as the inside 343 00:22:35,560 --> 00:22:36,680 piece. 344 00:22:36,680 --> 00:22:40,520 So all those things Mary just talked about, there's those study habits and their plans 345 00:22:40,520 --> 00:22:46,680 and the planning, the monitoring, what they're doing outside of the classroom and preparation 346 00:22:46,680 --> 00:22:52,040 for the next class, the next quiz, the next exam, whatever that may be. 347 00:22:52,040 --> 00:22:57,120 Again, that's where a lot of us in the old days when Poggle didn't exist, that's where 348 00:22:57,120 --> 00:23:03,520 we practiced metacognition in our own learning environments because our classrooms weren't 349 00:23:03,520 --> 00:23:07,820 necessarily encouraging us to be metacognitive during class time. 350 00:23:07,820 --> 00:23:11,760 So we had to think about what we learned from class outside of the classroom. 351 00:23:11,760 --> 00:23:15,360 And certainly we want to continue to encourage our students to do that. 352 00:23:15,360 --> 00:23:21,880 So again, when you meet with a student who has performed poorly on some major assessment, 353 00:23:21,880 --> 00:23:30,200 you're probably going to ask them, what were the things that you did to prepare yourself 354 00:23:30,200 --> 00:23:33,360 for this particular assessment in the course? 355 00:23:33,360 --> 00:23:34,440 What did you do? 356 00:23:34,440 --> 00:23:36,320 How did you study? 357 00:23:36,320 --> 00:23:38,000 Did you study with others? 358 00:23:38,000 --> 00:23:41,400 Did you make study materials, study aids? 359 00:23:41,400 --> 00:23:46,440 Did you simply just reread the activities and not work any problems? 360 00:23:46,440 --> 00:23:52,280 What is it that you did in preparation for this particular assessment? 361 00:23:52,280 --> 00:23:59,320 Fun fact, I just had a student complete a set of test reflections and one of my questions 362 00:23:59,320 --> 00:24:02,040 was what did you do to prepare for this test? 363 00:24:02,040 --> 00:24:04,240 And they said nothing. 364 00:24:04,240 --> 00:24:06,520 I was like, okay. 365 00:24:06,520 --> 00:24:11,720 And then the next question was, well, what could I do differently to prepare for the 366 00:24:11,720 --> 00:24:12,720 next test? 367 00:24:12,720 --> 00:24:13,720 And he said, actually study. 368 00:24:13,720 --> 00:24:16,560 And I was like, okay, well, you're honest. 369 00:24:16,560 --> 00:24:18,720 There's a start. 370 00:24:18,720 --> 00:24:21,120 There you go. 371 00:24:21,120 --> 00:24:24,400 There's the start. 372 00:24:24,400 --> 00:24:29,160 So one of the things that you both mentioned in your response about what that would look 373 00:24:29,160 --> 00:24:36,840 like is questions that they're asking questions either of other students or of you. 374 00:24:36,840 --> 00:24:40,720 And we know that students ask questions in class regardless. 375 00:24:40,720 --> 00:24:46,720 So what would be the difference between the types of questions they'd be asking if they're 376 00:24:46,720 --> 00:24:51,560 doing a really nice job in reflecting on their learning versus just the types of questions 377 00:24:51,560 --> 00:24:56,000 that they're asking if there are more, I'll put it this way, a typical high school or 378 00:24:56,000 --> 00:24:58,480 possibly middle school students? 379 00:24:58,480 --> 00:25:00,080 And Marty, what are your thoughts? 380 00:25:00,080 --> 00:25:04,720 Well, I mean, I think the typical question we get from every student who's probably not 381 00:25:04,720 --> 00:25:10,120 practicing at the level we want is, is this right? 382 00:25:10,120 --> 00:25:12,120 Ah, yes. 383 00:25:12,120 --> 00:25:13,120 Right. 384 00:25:13,120 --> 00:25:16,680 They've answered number eight and they want to know, is it right? 385 00:25:16,680 --> 00:25:17,680 Is this correct? 386 00:25:17,680 --> 00:25:18,680 Yeah. 387 00:25:18,680 --> 00:25:23,720 And again, we want to avoid saying yes or no. 388 00:25:23,720 --> 00:25:26,880 So what do we normally say? 389 00:25:26,880 --> 00:25:31,040 What do you think or what's led you to this point? 390 00:25:31,040 --> 00:25:35,480 What were the previous leading questions that got you here and help them again develop the 391 00:25:35,480 --> 00:25:41,660 confidence that they're thinking about the process that led to the answer to that question 392 00:25:41,660 --> 00:25:44,660 is correct and get them to affirm that. 393 00:25:44,660 --> 00:25:48,920 Or we send them to talk to another team in the room that's nearby to see what they got 394 00:25:48,920 --> 00:25:50,920 and to explain what they were thinking. 395 00:25:50,920 --> 00:25:56,300 So avoid just the confirmation of yes, you got it right or no, it's not. 396 00:25:56,300 --> 00:26:01,760 So we're trying to encourage them when they get there to be metacognitive, even though 397 00:26:01,760 --> 00:26:03,760 we're not using that terminology necessarily. 398 00:26:03,760 --> 00:26:10,360 And I'm sure a lot of Pogal facilitators are already doing these things in their own environment. 399 00:26:10,360 --> 00:26:12,440 So this is not novel. 400 00:26:12,440 --> 00:26:16,840 This is something everybody's probably doing on a regular basis. 401 00:26:16,840 --> 00:26:22,200 I have noticed in students like post COVID, there's definitely an increase in the anxiety 402 00:26:22,200 --> 00:26:24,920 about, did I get this question correct? 403 00:26:24,920 --> 00:26:27,400 And this need for affirmation. 404 00:26:27,400 --> 00:26:32,240 It's a pretty simple question. 405 00:26:32,240 --> 00:26:33,240 You look okay there. 406 00:26:33,240 --> 00:26:34,240 You look pretty confident. 407 00:26:34,240 --> 00:26:36,240 You know how to do it. 408 00:26:36,240 --> 00:26:38,280 And they're trying to read your facial expression. 409 00:26:38,280 --> 00:26:40,600 So the better the poker face you have. 410 00:26:40,600 --> 00:26:41,960 Ooh, that's interesting. 411 00:26:41,960 --> 00:26:42,960 I like that. 412 00:26:42,960 --> 00:26:43,960 Yeah. 413 00:26:43,960 --> 00:26:47,880 So because again, depending on the face you're giving them, they may think, oh, I've done 414 00:26:47,880 --> 00:26:53,000 it wrong because the way you're looking at me. 415 00:26:53,000 --> 00:26:57,640 So I think that's why it's also good as a facilitator to walk around the room and ask 416 00:26:57,640 --> 00:27:02,800 students why they got the answer they did to question eight, even when it's right. 417 00:27:02,800 --> 00:27:03,800 Yeah. 418 00:27:03,800 --> 00:27:09,180 Mary, what sorts of questions do you think students would be asking each other if they're 419 00:27:09,180 --> 00:27:13,400 really displaying a high level of reflection? 420 00:27:13,400 --> 00:27:16,480 I fall back on the idea of argumentation. 421 00:27:16,480 --> 00:27:20,800 If they can set up and they say, I think the answer is this for these reasons. 422 00:27:20,800 --> 00:27:23,960 They might say, why did you get that answer? 423 00:27:23,960 --> 00:27:27,800 And I always think a really good conversation has both sides. 424 00:27:27,800 --> 00:27:29,640 One person saying, what do you think of this? 425 00:27:29,640 --> 00:27:33,440 And the other person might be saying, well, I think this because of this. 426 00:27:33,440 --> 00:27:37,200 And so I spend a lot of time on argumentation in my class as well. 427 00:27:37,200 --> 00:27:40,240 They've got to write every time I say, I'm going to use the word evidence. 428 00:27:40,240 --> 00:27:41,720 I'm going to ask you questions all the time. 429 00:27:41,720 --> 00:27:44,960 I'm never going to give you a straight answer. 430 00:27:44,960 --> 00:27:48,620 And they start to recognize the shift that question. 431 00:27:48,620 --> 00:27:53,120 But I realize that within a semester, it can be difficult to have those changes. 432 00:27:53,120 --> 00:27:57,480 But I see it's much more of a conversation between two students rather than one being 433 00:27:57,480 --> 00:27:58,480 like, oh, yeah, I agree. 434 00:27:58,480 --> 00:28:00,040 I agree and moving on. 435 00:28:00,040 --> 00:28:02,160 That's why there's so many of those. 436 00:28:02,160 --> 00:28:06,120 I think there's a recent post asking, well, what happens if you have these really simple 437 00:28:06,120 --> 00:28:07,480 answers to questions? 438 00:28:07,480 --> 00:28:11,920 Students aren't going to have very different answers because they all just say, it's like 439 00:28:11,920 --> 00:28:14,400 when kids in elementary school are like, what's your favorite food? 440 00:28:14,400 --> 00:28:16,840 And everybody says ice cream because the first kid did. 441 00:28:16,840 --> 00:28:21,560 And so you want to make sure that your questions that you're asking are setting up your classroom 442 00:28:21,560 --> 00:28:27,640 that they feel like they can say other things or in different ways so it's not so much focused 443 00:28:27,640 --> 00:28:30,240 on I just want a right answer. 444 00:28:30,240 --> 00:28:35,240 And I mean, there is for many vocal activities, obviously, there's a more correct definition 445 00:28:35,240 --> 00:28:37,240 than other correct definitions. 446 00:28:37,240 --> 00:28:42,500 But you want to get at that idea of they can say it in multiple ways. 447 00:28:42,500 --> 00:28:43,840 They can add pieces on. 448 00:28:43,840 --> 00:28:46,040 And especially for different levels of students. 449 00:28:46,040 --> 00:28:50,000 Maybe one is just we just need a basic sentence about the definition, but somebody else might 450 00:28:50,000 --> 00:28:53,560 say, well, look it in here, it also you can add this on. 451 00:28:53,560 --> 00:28:59,640 And so trying to provide those different levels of interest to students and kind of ability. 452 00:28:59,640 --> 00:29:01,920 Yeah, definitely. 453 00:29:01,920 --> 00:29:03,740 Thank you so much for that. 454 00:29:03,740 --> 00:29:08,360 As you go throughout your course, and hopefully you're having your students develop these 455 00:29:08,360 --> 00:29:12,160 skills, how do you point out to them their progress? 456 00:29:12,160 --> 00:29:20,080 How do you let them know how they're improving or like that they have improved or that they 457 00:29:20,080 --> 00:29:21,800 still have some room to improve? 458 00:29:21,800 --> 00:29:28,440 So how do you literally communicate with your students about their reflective capabilities? 459 00:29:28,440 --> 00:29:31,240 And Mary, what do you think? 460 00:29:31,240 --> 00:29:34,800 It's a lot of I think feedback, very consistent. 461 00:29:34,800 --> 00:29:38,680 In short, I mean, I know we can get into the habit of writing so much, but I always have 462 00:29:38,680 --> 00:29:47,280 a rule of came from Pogal, like you have like a strength and improvement and an insight. 463 00:29:47,280 --> 00:29:51,520 And so when I give feedback on any of their lab reports or any of their quizzes, it's 464 00:29:51,520 --> 00:29:54,440 always like, you've done really well at this. 465 00:29:54,440 --> 00:29:57,740 You need to improve on this and it might, and then I, you know, you can show what's 466 00:29:57,740 --> 00:30:03,040 wrong and right, but that I really try to focus on because they do need to see positive 467 00:30:03,040 --> 00:30:06,520 encouragement that they are doing something right in class, even if they don't get the 468 00:30:06,520 --> 00:30:11,120 content right all the time, but they've gotten better at this over the semester. 469 00:30:11,120 --> 00:30:13,320 So I do mostly feedback. 470 00:30:13,320 --> 00:30:19,440 I've taken a step back from numbers for grading for a while, so I'm focused on giving feedback 471 00:30:19,440 --> 00:30:23,520 and verbally, you know, and I have small classes. 472 00:30:23,520 --> 00:30:28,520 So granted, I have 24 students or fewer in a class for someone who has over a hundred. 473 00:30:28,520 --> 00:30:30,920 There's going to have to, you're going to have to think of different ways. 474 00:30:30,920 --> 00:30:35,020 Maybe you have the same type of feedback you would give like a kudos kind of like good 475 00:30:35,020 --> 00:30:40,400 job on I've noticed you've been doing this in class, but a lot of just the verbal and 476 00:30:40,400 --> 00:30:45,520 the written feedback and that's kind of where I land and I incorporate kind of this idea 477 00:30:45,520 --> 00:30:49,080 of assessment into into their overall grade. 478 00:30:49,080 --> 00:30:54,520 Like if you've been revising and reflecting, that shows me that you're trying harder to 479 00:30:54,520 --> 00:30:59,200 get, you know, to get a grade than if you're just going to do the bare minimum, which is 480 00:30:59,200 --> 00:31:00,560 fine for some students. 481 00:31:00,560 --> 00:31:05,000 That's what the goal is, but you want to encourage them to say, do this, revise this, see where 482 00:31:05,000 --> 00:31:07,240 you went wrong. 483 00:31:07,240 --> 00:31:10,680 So written and verbal feedback was kind of the SII. 484 00:31:10,680 --> 00:31:14,000 That's a big thing for me. 485 00:31:14,000 --> 00:31:19,760 Siobhan just mentally put a post-it on that and it's like, yeah, like that idea going 486 00:31:19,760 --> 00:31:20,760 forward. 487 00:31:20,760 --> 00:31:21,760 Thank you, Mary. 488 00:31:21,760 --> 00:31:24,400 Marty, what about you as you go throughout the course? 489 00:31:24,400 --> 00:31:29,480 Like how are you providing, how are you letting the students sort of know or communicate to 490 00:31:29,480 --> 00:31:32,160 them about the development of this skill? 491 00:31:32,160 --> 00:31:37,520 Sure, again, the reflection sheets that I use on a daily basis are probably the primary 492 00:31:37,520 --> 00:31:38,520 way. 493 00:31:38,520 --> 00:31:39,760 Do you respond to those? 494 00:31:39,760 --> 00:31:46,920 I do respond to each one of them and sometimes the responses are more in-depth than others. 495 00:31:46,920 --> 00:31:56,760 And I try to find a student very early on who does an exceptional job at writing a reflection 496 00:31:56,760 --> 00:32:03,000 and share that anonymously with that student's permission with the rest of the class so that 497 00:32:03,000 --> 00:32:11,680 they get a good exemplar of what I'm looking for or what I would hope to see from others 498 00:32:11,680 --> 00:32:18,520 who might be writing minimal, a few words at best and it's not even a complete sentence 499 00:32:18,520 --> 00:32:25,420 to try to encourage a little more depth of processing of the content, the process skill 500 00:32:25,420 --> 00:32:27,800 and where they are struggling. 501 00:32:27,800 --> 00:32:34,800 Predominantly, again, I'm trying not to get into evaluating them for grade as much as 502 00:32:34,800 --> 00:32:36,600 I am completion of the exercise. 503 00:32:36,600 --> 00:32:42,640 And again, I think the feedback, as Mary has mentioned, is more important in these cases 504 00:32:42,640 --> 00:32:48,880 than a numerical score would be on something that, again, is a lot more subjective to grade. 505 00:32:48,880 --> 00:32:51,640 Again, we're both scientists and all of us are here. 506 00:32:51,640 --> 00:32:55,640 And so we think of things very objectively in a lot of the things we teach, but this 507 00:32:55,640 --> 00:33:02,040 seems to be something that it's harder without a glorified rubric to go through and give 508 00:33:02,040 --> 00:33:03,040 points for. 509 00:33:03,040 --> 00:33:09,640 So I'm really wanting to encourage their reflections and reward them from having completed them 510 00:33:09,640 --> 00:33:11,880 as fully as I would expect them to. 511 00:33:11,880 --> 00:33:18,760 Again, if they get in the habit of only giving me a few words every time or not answering 512 00:33:18,760 --> 00:33:24,520 the answer, the strength of the team, but not the way the team could improve, then I'll 513 00:33:24,520 --> 00:33:28,920 start digging them a little bit just to say, hey, I'm expecting you to do a little bit 514 00:33:28,920 --> 00:33:33,560 more than what you're putting forward here. 515 00:33:33,560 --> 00:33:35,040 Mary mentioned exam wrappers earlier. 516 00:33:35,040 --> 00:33:41,280 I do exam wrappers usually after each exam for all students to, again, help them assess 517 00:33:41,280 --> 00:33:45,800 where they are and what they're doing and what's effective. 518 00:33:45,800 --> 00:33:52,880 But part of that exam wrapper often includes having them predict a score on their exam 519 00:33:52,880 --> 00:33:55,760 before they know what the score is. 520 00:33:55,760 --> 00:33:56,920 So they've just completed the exam. 521 00:33:56,920 --> 00:34:01,940 I tell them to complete the exam wrapper soon thereafter and to make a prediction about 522 00:34:01,940 --> 00:34:04,600 how they have performed on that exam. 523 00:34:04,600 --> 00:34:07,080 And of course, you have some students who are very good at that. 524 00:34:07,080 --> 00:34:09,480 They can get within a few points. 525 00:34:09,480 --> 00:34:14,560 And as you might expect, there are some students whose predictions are way off, right? 526 00:34:14,560 --> 00:34:20,640 So 15, 20, 30 points off from where they actually ended up performing. 527 00:34:20,640 --> 00:34:22,400 Probably in both directions. 528 00:34:22,400 --> 00:34:23,960 In both directions, right? 529 00:34:23,960 --> 00:34:30,000 So you do have that subset of students who Dunning and Krueger refer to as being unskilled 530 00:34:30,000 --> 00:34:31,080 and unaware. 531 00:34:31,080 --> 00:34:35,000 So they think they've made an A, but they made a D, right? 532 00:34:35,000 --> 00:34:40,680 So they really thought they knew the material, but yet their score shows the complete opposite. 533 00:34:40,680 --> 00:34:45,520 And I think, again, that can be instructive for students to recognize, hey, I thought 534 00:34:45,520 --> 00:34:47,760 I knew this stuff, but I didn't. 535 00:34:47,760 --> 00:34:53,200 And you can have that conversation with that student and say, what are you doing? 536 00:34:53,200 --> 00:34:55,120 What was your preparation like? 537 00:34:55,120 --> 00:35:00,560 And how can we change that so that you will have a better performance in the future? 538 00:35:00,560 --> 00:35:02,760 Yeah, definitely. 539 00:35:02,760 --> 00:35:03,760 Uninformed. 540 00:35:03,760 --> 00:35:05,960 What was it? 541 00:35:05,960 --> 00:35:06,960 Unskilled. 542 00:35:06,960 --> 00:35:09,960 Unskilled and unaware. 543 00:35:09,960 --> 00:35:13,440 Yes. 544 00:35:13,440 --> 00:35:21,720 So with all this stuff about reflection and metacognition, we know that the way the brain 545 00:35:21,720 --> 00:35:28,960 develops throughout adolescence, there are certainly times where that's just a bit of 546 00:35:28,960 --> 00:35:30,680 a tall ask. 547 00:35:30,680 --> 00:35:36,800 So does the age of the student make a difference when you're talking about introducing this 548 00:35:36,800 --> 00:35:41,040 sort of level of reflection and self-examination? 549 00:35:41,040 --> 00:35:45,880 Do you think there are any adjustments that might need to be made for, let's say, middle 550 00:35:45,880 --> 00:35:50,400 school classes, if you're doing this in a middle school class versus a high school class 551 00:35:50,400 --> 00:35:58,200 versus community college versus a four-year or even further? 552 00:35:58,200 --> 00:36:00,000 Mary, thoughts? 553 00:36:00,000 --> 00:36:01,680 Well, absolutely. 554 00:36:01,680 --> 00:36:07,600 So metacognition is knowledge and skill tend to grow as a child grows. 555 00:36:07,600 --> 00:36:12,200 And they really don't have as much when they're in middle school and high school and even 556 00:36:12,200 --> 00:36:15,400 in right entering college. 557 00:36:15,400 --> 00:36:17,480 It's not until your 20s. 558 00:36:17,480 --> 00:36:22,160 And if we think, if you continued on in school for a long time, things got easier. 559 00:36:22,160 --> 00:36:26,880 You knew that you weren't doing well in a class or things like that. 560 00:36:26,880 --> 00:36:33,280 I'm a big fan of a book called Powerful Teaching, and so they talk about by Pooja Agarwal and 561 00:36:33,280 --> 00:36:34,280 Patrice Bain. 562 00:36:34,280 --> 00:36:36,720 And they really talk about different things that you can do. 563 00:36:36,720 --> 00:36:42,320 And one of them's in higher ed, one of them's in, I think, middle school or high school. 564 00:36:42,320 --> 00:36:47,280 And they really give this kind of toolbox of things that you can use at all levels. 565 00:36:47,280 --> 00:36:51,640 I think where the importance comes in is sometimes when we say, students, please reflect. 566 00:36:51,640 --> 00:36:54,200 And students are like, what are you talking about? 567 00:36:54,200 --> 00:36:56,600 What does reflect mean? 568 00:36:56,600 --> 00:37:02,800 And I think I've started to recognize that a student speaking another language, having 569 00:37:02,800 --> 00:37:06,480 different levels of fluency of English. 570 00:37:06,480 --> 00:37:09,400 Sometimes you realize people don't know what that word means. 571 00:37:09,400 --> 00:37:10,480 What does reflect mean? 572 00:37:10,480 --> 00:37:15,040 We always think about it in meditation. 573 00:37:15,040 --> 00:37:16,760 Just reflect on your life. 574 00:37:16,760 --> 00:37:18,760 But you want very specific things. 575 00:37:18,760 --> 00:37:24,280 So the thing that I would say for basically any level is that you need consistency and 576 00:37:24,280 --> 00:37:27,160 you need to explain what it means. 577 00:37:27,160 --> 00:37:31,640 So when I ask you to reflect on this lab experiment, I'm thinking, what are the two errors that 578 00:37:31,640 --> 00:37:33,600 you can figure out that happened? 579 00:37:33,600 --> 00:37:36,960 I want you to think back through the process that you took. 580 00:37:36,960 --> 00:37:37,960 And that's going to take time. 581 00:37:37,960 --> 00:37:41,800 And so you can't just assume the first month of school, they will have it. 582 00:37:41,800 --> 00:37:45,080 This is going to be, especially for middle and high school, you've got a whole year, 583 00:37:45,080 --> 00:37:49,280 oftentimes, with students to be able to see them grow in that. 584 00:37:49,280 --> 00:37:52,440 In college, some teachers do it, some don't. 585 00:37:52,440 --> 00:37:55,160 And so they might get thrown in and out of it. 586 00:37:55,160 --> 00:37:59,500 But to really say, look, I know that you know these things. 587 00:37:59,500 --> 00:38:01,600 You know that you study well or don't do this. 588 00:38:01,600 --> 00:38:03,440 I mean, you're aware. 589 00:38:03,440 --> 00:38:04,800 So it's not as trying to step it up. 590 00:38:04,800 --> 00:38:09,920 But again, I think the language you use is really going to be helpful in explaining what 591 00:38:09,920 --> 00:38:12,960 does this mean and what I expect to see out of it. 592 00:38:12,960 --> 00:38:18,980 And that can, I think, help them by even taking it home and adding it to their study thing 593 00:38:18,980 --> 00:38:19,980 for any class. 594 00:38:19,980 --> 00:38:24,120 And I always say, what I'm teaching you in all of these strategies, this can be used 595 00:38:24,120 --> 00:38:25,120 anywhere. 596 00:38:25,120 --> 00:38:26,720 This is not just for chemistry class. 597 00:38:26,720 --> 00:38:32,160 And that's a really important thing to me as you are a new student in college, just 598 00:38:32,160 --> 00:38:33,160 in college. 599 00:38:33,160 --> 00:38:35,360 You know, what do you not know how to do? 600 00:38:35,360 --> 00:38:39,440 And how is my class going to assist you in that as an introductory course? 601 00:38:39,440 --> 00:38:41,640 You know, junior year, that's a whole different thing. 602 00:38:41,640 --> 00:38:45,080 There's other things you'd be adding on to those, very minimal. 603 00:38:45,080 --> 00:38:50,760 But at my level, it's important, just as important to teach them how to be a student as for them 604 00:38:50,760 --> 00:38:52,120 to learn the content. 605 00:38:52,120 --> 00:38:57,440 Because the content is, it's a lot less than an upper level course. 606 00:38:57,440 --> 00:38:58,440 Thank you. 607 00:38:58,440 --> 00:39:03,640 Marty, age of the student, and especially as someone who has taught at lots of different 608 00:39:03,640 --> 00:39:06,360 levels. 609 00:39:06,360 --> 00:39:10,720 What do you think about introducing this, especially in the light of what Mary said? 610 00:39:10,720 --> 00:39:13,440 Yeah, Mary made some great points. 611 00:39:13,440 --> 00:39:19,000 And what I would like to emphasize is the maturity of the learner should not necessarily 612 00:39:19,000 --> 00:39:21,440 be tied to their age. 613 00:39:21,440 --> 00:39:23,480 Yes. 614 00:39:23,480 --> 00:39:29,880 I had some very talented high school students who I thought their maturity of learning and 615 00:39:29,880 --> 00:39:35,560 ability to be metacognitive would be equal to or better than some of the professional 616 00:39:35,560 --> 00:39:39,200 students that I'm encountering right now. 617 00:39:39,200 --> 00:39:45,960 So I think it is appropriate to target, again, what might be the norm for your classroom 618 00:39:45,960 --> 00:39:54,400 environment and to not have expectations that would exceed what a competent student in your 619 00:39:54,400 --> 00:39:58,840 area of teaching should be expected to do. 620 00:39:58,840 --> 00:40:06,360 But again, you're going to have some talented students who are capable of going deeper, 621 00:40:06,360 --> 00:40:09,600 being a lot more reflective and metacognitive than others. 622 00:40:09,600 --> 00:40:16,480 But I absolutely believe that it can be practiced, again, not necessarily at the same depth and 623 00:40:16,480 --> 00:40:19,320 expectation, probably at all levels. 624 00:40:19,320 --> 00:40:26,480 I mean, we're still wanting students even at middle school or below levels to learn 625 00:40:26,480 --> 00:40:31,320 and to question what they're doing and question what somebody else is doing and thinking and 626 00:40:31,320 --> 00:40:32,760 trying to understand things. 627 00:40:32,760 --> 00:40:37,080 So I don't think that we don't want them to be metacognitive. 628 00:40:37,080 --> 00:40:38,080 Absolutely. 629 00:40:38,080 --> 00:40:44,200 It's just not going to be at the same depth that we would expect of a college or graduate 630 00:40:44,200 --> 00:40:45,640 or professional student to be able to do. 631 00:40:45,640 --> 00:40:46,960 Yeah, it's going to just look different. 632 00:40:46,960 --> 00:40:49,160 It'll look different, yeah. 633 00:40:49,160 --> 00:40:57,440 And just what Mary said beforehand about it being clear language and very clear expectations. 634 00:40:57,440 --> 00:41:01,200 I always think that's just good teaching, right? 635 00:41:01,200 --> 00:41:05,800 That's just good teaching that you are absolutely clear with what you're looking for and you 636 00:41:05,800 --> 00:41:08,080 don't just say, okay, reflect on the lesson. 637 00:41:08,080 --> 00:41:10,520 It's like, no, no, no, no. 638 00:41:10,520 --> 00:41:11,600 What does that mean? 639 00:41:11,600 --> 00:41:12,680 What does that look like? 640 00:41:12,680 --> 00:41:14,240 What do I expect to see from you? 641 00:41:14,240 --> 00:41:16,520 And like Marty said too, showing the exemplar. 642 00:41:16,520 --> 00:41:19,160 Oh, I'm a big fan of exemplars. 643 00:41:19,160 --> 00:41:21,480 I think it really helps. 644 00:41:21,480 --> 00:41:26,200 You can explain until you turn blue in the face, but if you show your students, you know, 645 00:41:26,200 --> 00:41:28,240 this is what I'm looking for. 646 00:41:28,240 --> 00:41:32,400 I think it tends to click. 647 00:41:32,400 --> 00:41:33,980 And then the final question. 648 00:41:33,980 --> 00:41:40,040 So I started off this conversation thinking that I wasn't really doing that much with 649 00:41:40,040 --> 00:41:46,640 metacognition, but Marty and Mary have actually convinced me I'm doing more than I realize. 650 00:41:46,640 --> 00:41:48,600 I can still definitely step it up. 651 00:41:48,600 --> 00:41:51,680 I mean, I'm not saying that there's no room for improvement, but I thought I was like 652 00:41:51,680 --> 00:41:56,280 at like a 0.5 on a scale of zero to 10. 653 00:41:56,280 --> 00:41:59,360 Maybe I'm now more at like a two or three. 654 00:41:59,360 --> 00:42:05,040 But what advice would you give to a practitioner who isn't really highlighting this in their 655 00:42:05,040 --> 00:42:07,800 classroom on where to start? 656 00:42:07,800 --> 00:42:10,160 Like, where do you start? 657 00:42:10,160 --> 00:42:15,680 What strategies would you suggest they start using to introduce it in a way that allows 658 00:42:15,680 --> 00:42:18,200 their students to start becoming better reflectors? 659 00:42:18,200 --> 00:42:21,280 Marty, I'll start with you. 660 00:42:21,280 --> 00:42:27,000 While at the risk of overstating the obvious, I would encourage people to be metacognitive. 661 00:42:27,000 --> 00:42:30,640 No, that's not overstating the obvious at all. 662 00:42:30,640 --> 00:42:33,380 Like, so I'm serious. 663 00:42:33,380 --> 00:42:39,800 You know your students, you know your environment better than anybody else does. 664 00:42:39,800 --> 00:42:44,880 So think about what's going to work best for your population that's going to encourage 665 00:42:44,880 --> 00:42:51,400 them to study and learn and to think more effectively than what they've been doing. 666 00:42:51,400 --> 00:42:54,840 So again, we've highlighted some examples today. 667 00:42:54,840 --> 00:43:03,040 You know, whether it's retrieval practice, reflections, exam wrappers, you know, revising 668 00:43:03,040 --> 00:43:07,000 laboratory reports, you know, all of these things are metacognitive behaviors in your 669 00:43:07,000 --> 00:43:08,200 writing mind. 670 00:43:08,200 --> 00:43:11,440 You're probably doing a lot more than you give yourself credit for. 671 00:43:11,440 --> 00:43:13,480 I'm sure a lot of people are. 672 00:43:13,480 --> 00:43:21,200 They just didn't realize it necessarily fit under this umbrella of metacognition in their 673 00:43:21,200 --> 00:43:22,200 classroom. 674 00:43:22,200 --> 00:43:24,600 Or it could be done more purposefully. 675 00:43:24,600 --> 00:43:25,600 Correct. 676 00:43:25,600 --> 00:43:30,880 So in addition to the resource that Mary mentioned, powerful teaching, I think there's a couple 677 00:43:30,880 --> 00:43:36,840 of other resources many in our Poggle community are probably familiar with that I think again 678 00:43:36,840 --> 00:43:41,520 are great for encouraging some of the things we've been talking about today. 679 00:43:41,520 --> 00:43:48,440 So the first I'd mention is the book co-authored by Mark McDaniel called Make It Stick. 680 00:43:48,440 --> 00:43:51,280 Many of you may have seen Mark at a Poggle event before. 681 00:43:51,280 --> 00:43:56,080 He was one of our plenary speakers at the first NCAP that we did in 2017. 682 00:43:56,080 --> 00:43:57,840 He's also been at other Poggle events. 683 00:43:57,840 --> 00:44:00,400 He's a professor at WashU. 684 00:44:00,400 --> 00:44:06,940 So again, I think this book is very readable even by a general lay audience. 685 00:44:06,940 --> 00:44:12,680 We have students in our own program read the book in preparation for their study in our 686 00:44:12,680 --> 00:44:16,520 professional program here in the College of Pharmacy and we do some follow-up exercises 687 00:44:16,520 --> 00:44:17,520 with them. 688 00:44:17,520 --> 00:44:21,740 We've actually had Mark come and speak with them and even his daughter this past year. 689 00:44:21,740 --> 00:44:28,900 So again, it hits on lots of themes, but metacognition and thinking about how students learn effectively 690 00:44:28,900 --> 00:44:30,540 is one of them. 691 00:44:30,540 --> 00:44:36,360 The second resource that I would like to call to bear is the Ellipse Project. 692 00:44:36,360 --> 00:44:43,400 Again, this was started within the Poggle project itself by some very experienced practitioners 693 00:44:43,400 --> 00:44:46,960 and lots of people inside the Poggle project have contributed to it. 694 00:44:46,960 --> 00:44:52,600 So Ellipse stands for Enhancing Learning by Improving Process Skills in STEM and they 695 00:44:52,600 --> 00:44:58,140 have lots of wonderful tools that you can use in your classroom that are rubrics that 696 00:44:58,140 --> 00:45:00,920 can help students develop their process skills. 697 00:45:00,920 --> 00:45:04,240 One of which of course is metacognition and self-assessment. 698 00:45:04,240 --> 00:45:10,840 So again, I encourage you to check out those rubrics for use in your classroom if you haven't 699 00:45:10,840 --> 00:45:12,880 thought about using them before. 700 00:45:12,880 --> 00:45:16,880 And again, there's a wonderful website that's going to be part of the show notes for the 701 00:45:16,880 --> 00:45:24,280 Ellipse Project as well as some website information for both of the textbooks that we've mentioned 702 00:45:24,280 --> 00:45:27,920 make it stick and powerful teaching. 703 00:45:27,920 --> 00:45:30,120 Mary. 704 00:45:30,120 --> 00:45:33,440 My big thing is just make one or two small changes. 705 00:45:33,440 --> 00:45:37,840 If you are going to say, I'm going to do retrieval practice, just do that. 706 00:45:37,840 --> 00:45:42,120 I know that making changes is exhausting at all levels. 707 00:45:42,120 --> 00:45:44,080 Teachers are doing so much. 708 00:45:44,080 --> 00:45:46,320 They're teaching the classroom and then they have to prep. 709 00:45:46,320 --> 00:45:51,080 So being asked to change your classroom is difficult and you can't expect to change your 710 00:45:51,080 --> 00:45:56,160 entire lab sequence to make it more, to engage students more metacognitively. 711 00:45:56,160 --> 00:46:02,000 But you could change a small part of some labs to have them to start seeing how it looks. 712 00:46:02,000 --> 00:46:05,040 And it can be surprising that you're like, wow, it's not as much. 713 00:46:05,040 --> 00:46:09,280 And my biggest thing is I was trying to think back is when I started some of this stuff 714 00:46:09,280 --> 00:46:13,480 and yes, I've been doing it for 10 years, but I've been doing a lot of this stuff for 715 00:46:13,480 --> 00:46:21,560 a long time, but it wasn't until recently that I was like, I'm actually doing it consistently. 716 00:46:21,560 --> 00:46:25,000 And so change takes a while, but make one or two small changes. 717 00:46:25,000 --> 00:46:31,880 And again, looking up different strategies to use, I think those small kind of, I've 718 00:46:31,880 --> 00:46:36,280 talked about classroom assessment techniques or I think they're called CATS, but you can 719 00:46:36,280 --> 00:46:42,120 use powerful teaching or these different ideas that you're just trying to get students to 720 00:46:42,120 --> 00:46:45,200 do things to make themselves more aware. 721 00:46:45,200 --> 00:46:46,680 And a big thing is model it. 722 00:46:46,680 --> 00:46:50,440 So just one thing that you could simply do is when you're doing a problem on the board 723 00:46:50,440 --> 00:46:51,440 is do it wrong. 724 00:46:51,440 --> 00:46:52,440 And then- 725 00:46:52,440 --> 00:46:55,600 Oh, I like that so much. 726 00:46:55,600 --> 00:46:56,600 Go back. 727 00:46:56,600 --> 00:47:01,840 So when I draw the structure of nitrogen gas, I go through all the possibilities and say, 728 00:47:01,840 --> 00:47:04,480 look, man, I keep messing up. 729 00:47:04,480 --> 00:47:06,760 I use this rule and then I use this one. 730 00:47:06,760 --> 00:47:11,000 And then you finally get through and you realize, and then there might be times where you try 731 00:47:11,000 --> 00:47:14,920 to give yourself something harder so that they actually see you go through this process 732 00:47:14,920 --> 00:47:17,720 of I know something's wrong. 733 00:47:17,720 --> 00:47:19,640 How am I going to fix it? 734 00:47:19,640 --> 00:47:24,240 Or on a test, I've noticed that sometimes students will do really poorly on a question 735 00:47:24,240 --> 00:47:27,940 and I'll come out and say, hey, I thought we did this in class. 736 00:47:27,940 --> 00:47:29,640 I thought we did this. 737 00:47:29,640 --> 00:47:34,000 Everything's amiss and I need to go back and change this question. 738 00:47:34,000 --> 00:47:36,600 You guys need to, we need to talk about it and kind of figure it out. 739 00:47:36,600 --> 00:47:42,240 And so making it a conversation in modeling it, being consistent and being intentional 740 00:47:42,240 --> 00:47:43,240 and transparent. 741 00:47:43,240 --> 00:47:47,200 Like I'm trying to get you guys to learn this stuff because it's going to help you in your 742 00:47:47,200 --> 00:47:48,200 lifeline goals. 743 00:47:48,200 --> 00:47:54,520 Yes, it's not chemistry, but it's something you're going to need. 744 00:47:54,520 --> 00:47:55,720 Absolutely. 745 00:47:55,720 --> 00:47:58,320 Thank you so, so much, Marty and Mary. 746 00:47:58,320 --> 00:48:02,840 We really appreciate your time today that you could come in and have this conversation. 747 00:48:02,840 --> 00:48:07,720 And again, I have made so many mental post-it notes during this discussion that I'm going 748 00:48:07,720 --> 00:48:12,840 to go back to the classroom tomorrow and there is some stuff that I want to start implementing. 749 00:48:12,840 --> 00:48:17,120 So I hope that our listeners also got some great ideas. 750 00:48:17,120 --> 00:48:23,560 Again, Marty, Mary, thank you so much and I'll send it back to you, Wayne. 751 00:48:23,560 --> 00:48:29,120 Well thanks to Siobhan, Mary and Marty for such a great discussion of this very important 752 00:48:29,120 --> 00:48:30,760 process skill. 753 00:48:30,760 --> 00:48:34,560 How do you incorporate metacognition into your facilitation? 754 00:48:34,560 --> 00:48:39,320 Let us know by joining the conversation at hashtag the Poggle podcast on the Facebook 755 00:48:39,320 --> 00:48:41,120 practitioners page. 756 00:48:41,120 --> 00:48:46,160 Earlier this month, we held the first zoom happy hour with the networks group and we 757 00:48:46,160 --> 00:48:48,060 had a great time. 758 00:48:48,060 --> 00:48:50,600 We are planning our next meeting on March 2nd. 759 00:48:50,600 --> 00:48:55,440 Be sure to RSVP to the next invite and join us as we get together with our fellow Poggle 760 00:48:55,440 --> 00:48:57,200 practitioners. 761 00:48:57,200 --> 00:49:02,600 Until next time for everyone at the Poggle podcast, this is Wayne Pearson. 762 00:49:02,600 --> 00:49:21,600 So long everybody.