1 00:00:00,000 --> 00:00:02,880 This is the Reading Instruction Show. 2 00:00:02,880 --> 00:00:05,720 I'm your host, as always, Dr. Andy Johnson. 3 00:00:05,720 --> 00:00:12,520 The topic of today's podcast is understanding science and reading science. 4 00:00:12,520 --> 00:00:18,160 And I want to just cover some of the basics of what science and reading science is. 5 00:00:18,160 --> 00:00:21,040 So let's define our terms. 6 00:00:21,040 --> 00:00:26,560 Much of the confusion around the science of reading is related to a lack of understanding 7 00:00:26,560 --> 00:00:33,480 about what science is and research and educational research and literacy research. 8 00:00:33,480 --> 00:00:36,600 I call it the great ununderstanding. 9 00:00:36,600 --> 00:00:43,320 I think we generally assume there's a common understanding, but there is not. 10 00:00:43,320 --> 00:00:49,640 So let's try to get on the same page starting with science. 11 00:00:49,640 --> 00:00:51,840 Science is a process. 12 00:00:51,840 --> 00:00:54,200 It's a verb to science. 13 00:00:54,200 --> 00:00:56,200 It's something we do. 14 00:00:56,200 --> 00:00:58,960 The basic essence of science is this. 15 00:00:58,960 --> 00:01:04,680 You ask a question and you use data to answer the question. 16 00:01:04,680 --> 00:01:12,920 And what makes science different from simply collecting data is that science uses a systematic 17 00:01:12,920 --> 00:01:16,160 process to collect data. 18 00:01:16,160 --> 00:01:21,960 Systematic means that there's a fixed plan in place based on a set of principles that 19 00:01:21,960 --> 00:01:25,440 are generally agreed upon by the field. 20 00:01:25,440 --> 00:01:36,480 Now a research method is the specific systematic process used to gather and analyze the data. 21 00:01:36,480 --> 00:01:40,840 And here's the first understanding related to the science of reading. 22 00:01:40,840 --> 00:01:44,600 There is no scientific method. 23 00:01:44,600 --> 00:01:47,520 What you say I can hear you thinking. 24 00:01:47,520 --> 00:01:48,520 Repeat. 25 00:01:48,520 --> 00:01:51,720 There is no singular method of science. 26 00:01:51,720 --> 00:01:55,840 Rather, there are methods of science. 27 00:01:55,840 --> 00:02:03,200 While it's commonly thought that experimental research with control and experimental groups 28 00:02:03,200 --> 00:02:10,920 pre and post test is the scientific method, this is only one type of scientific method. 29 00:02:10,920 --> 00:02:16,840 There are a variety of scientific methods, including observational research, qualitative 30 00:02:16,840 --> 00:02:21,120 research, descriptive research, ethnographic research. 31 00:02:21,120 --> 00:02:25,000 And I don't want to get into a lecture on all the different types of research methods 32 00:02:25,000 --> 00:02:33,760 used, but the big idea here is that research starts with a question. 33 00:02:33,760 --> 00:02:40,640 The method used, the method of science used is dependent on the question. 34 00:02:40,640 --> 00:02:45,320 Different questions call for different methods. 35 00:02:45,320 --> 00:02:50,920 Now according to the science of reading, only strategies that have been shown to be effective 36 00:02:50,920 --> 00:02:58,560 by research conducted in actual learning environments using controlled experimental research can 37 00:02:58,560 --> 00:03:02,360 be said to be research based. 38 00:03:02,360 --> 00:03:04,880 These are the evidence based strategies. 39 00:03:04,880 --> 00:03:09,400 These are the ones that should be used to teach reading. 40 00:03:09,400 --> 00:03:14,120 And the typical controlled experimental research study goes something like this. 41 00:03:14,120 --> 00:03:17,360 You start with two relatively similar groups. 42 00:03:17,360 --> 00:03:22,200 You do something to one group and nothing to the other group to see if that something 43 00:03:22,200 --> 00:03:24,520 caused the other thing to happen. 44 00:03:24,520 --> 00:03:29,600 If everything is the same in the two groups except for that one thing and there's a difference 45 00:03:29,600 --> 00:03:35,080 at the end, we can say with a great deal of confidence that there is a causal relationship, 46 00:03:35,080 --> 00:03:39,640 that the one thing caused the other thing to happen. 47 00:03:39,640 --> 00:03:46,240 There ever after, if we want one thing to occur, we just have to present the other thing. 48 00:03:46,240 --> 00:03:52,920 If A causes B and we want B, just do a lot of A and B will occur. 49 00:03:52,920 --> 00:03:55,760 What could be simpler than that? 50 00:03:55,760 --> 00:03:57,360 Yes? 51 00:03:57,360 --> 00:04:03,000 Controlled experimental research works well in the physical sciences where things are 52 00:04:03,000 --> 00:04:10,440 relatively similar, meaning that a strain of bacteria in Bolivia is similar to the same 53 00:04:10,440 --> 00:04:13,880 type of bacteria in Alaska. 54 00:04:13,880 --> 00:04:19,320 However, humans are not standardized entities. 55 00:04:19,320 --> 00:04:23,400 We're all wonderfully gloriously unique. 56 00:04:23,400 --> 00:04:28,960 We all have a variety of different forces impacting us. 57 00:04:28,960 --> 00:04:35,640 A third grade student from a poor rural school is not the same as a third grade student from 58 00:04:35,640 --> 00:04:38,560 a wealthy suburban school. 59 00:04:38,560 --> 00:04:47,120 As well, teachers are unique, environments, values, cultures, situations are all unique. 60 00:04:47,120 --> 00:04:49,400 Humans do not exist in a vacuum. 61 00:04:49,400 --> 00:04:52,760 They do not exist in a Petri dish. 62 00:04:52,760 --> 00:04:59,040 This is why research in the physical sciences looks different from research in the social 63 00:04:59,040 --> 00:05:02,440 sciences. 64 00:05:02,440 --> 00:05:09,560 The methods used to study bacteria in a Petri dish really are limited when trying to examine 65 00:05:09,560 --> 00:05:13,120 children in a classroom. 66 00:05:13,120 --> 00:05:18,840 Human beings do not react similarly to similar stimuli. 67 00:05:18,840 --> 00:05:25,480 There are far too many variables at play in any classroom and with human beings to try 68 00:05:25,480 --> 00:05:28,520 to control and account for them all. 69 00:05:28,520 --> 00:05:35,560 And the very act of controlling an environment for research makes that environment a non-real 70 00:05:35,560 --> 00:05:38,720 environment. 71 00:05:38,720 --> 00:05:44,920 The science of reading contends that reading science involves only that singular type of 72 00:05:44,920 --> 00:05:47,360 research methodology. 73 00:05:47,360 --> 00:05:53,560 Only controlled experimental research conducted in an actual learning environment can be used 74 00:05:53,560 --> 00:06:01,640 to ask and answer questions related to reading or to determine causal relationships. 75 00:06:01,640 --> 00:06:08,200 I do think this understanding of research is a little bit limiting. 76 00:06:08,200 --> 00:06:13,320 By limiting the type of research that can be used and the type of data that counts as 77 00:06:13,320 --> 00:06:19,560 knowledge, you're creating a narrow people through which to view reality. 78 00:06:19,560 --> 00:06:29,160 However, I could live with this limited understanding of research if it were consistently used. 79 00:06:29,160 --> 00:06:36,440 If the research used with the science of reading was accurately interpreted and correctly applied 80 00:06:36,440 --> 00:06:43,400 and if a wide body of research were used to come to conclusions, but it's not. 81 00:06:43,400 --> 00:06:48,280 And it's often misapplied or misinterpreted. 82 00:06:48,280 --> 00:06:54,080 Now, contrary to popular thought, and here's a popular thought, you can make research say 83 00:06:54,080 --> 00:06:56,320 anything you want to say. 84 00:06:56,320 --> 00:06:58,200 Well, you can't. 85 00:06:58,200 --> 00:07:03,160 You can't make research say anything you want to say. 86 00:07:03,160 --> 00:07:07,720 You can collect data to say anything you want to say. 87 00:07:07,720 --> 00:07:14,320 You can use anecdotal evidence and personal stories to say anything you want to say. 88 00:07:14,320 --> 00:07:21,440 You can misuse research or overinterpret research results to say anything you want to say, but 89 00:07:21,440 --> 00:07:27,960 you can't make research say anything you want to say. 90 00:07:27,960 --> 00:07:33,000 Remember, research is different from collecting data. 91 00:07:33,000 --> 00:07:36,400 It's different from doing a study. 92 00:07:36,400 --> 00:07:41,960 Research is not research unless and until it has been subjected to blind peer review 93 00:07:41,960 --> 00:07:46,600 and published in a journal or some other academic source. 94 00:07:46,600 --> 00:07:51,200 And we never use a singular study to come to conclusions about anything. 95 00:07:51,200 --> 00:07:56,280 We use a wide body of research to get a general sense of the pattern of things. 96 00:07:56,280 --> 00:08:04,040 Now, as an example of the misuse of research, a white paper was put out on a website by 97 00:08:04,040 --> 00:08:09,960 Louisa Mott's in support of her letter's professional development program. 98 00:08:09,960 --> 00:08:16,120 And I tracked down the citations and I analyzed the research that she used to support her 99 00:08:16,120 --> 00:08:18,120 declarative statement. 100 00:08:18,120 --> 00:08:21,000 And I found two things. 101 00:08:21,000 --> 00:08:26,720 Some of the research cited to support her declarative statements was not research at 102 00:08:26,720 --> 00:08:28,200 all. 103 00:08:28,200 --> 00:08:36,480 It was data collected by think tanks or other organizations made to look like it was research. 104 00:08:36,480 --> 00:08:41,280 Research is not research unless and until it has been subjected to blind peer review 105 00:08:41,280 --> 00:08:43,840 and published in an academic journal. 106 00:08:43,840 --> 00:08:50,360 And the second thing, much of the research she cited to support her statements had little 107 00:08:50,360 --> 00:08:55,480 or nothing to do with the declarative statement she was making. 108 00:08:55,480 --> 00:09:01,520 She would make a statement, statement A, and cite it using research study B, but when you 109 00:09:01,520 --> 00:09:09,160 actually looked at research study B, you find that it's unrelated, misinterpreted, or over-interpreted 110 00:09:09,160 --> 00:09:12,840 and didn't support statement A at all. 111 00:09:12,840 --> 00:09:15,200 And that's not right. 112 00:09:15,200 --> 00:09:19,840 Did she do that on purpose or did she really think that that's how research should be 113 00:09:19,840 --> 00:09:20,840 used? 114 00:09:20,840 --> 00:09:22,340 Both. 115 00:09:22,340 --> 00:09:26,940 Both thoughts are rather disturbing. 116 00:09:26,940 --> 00:09:32,920 And let's look at generalization, a common misuse of research is to over-interpret the 117 00:09:32,920 --> 00:09:39,560 results and to try to generalize to a larger population where it is unwarranted. 118 00:09:39,560 --> 00:09:45,480 For example, if you look at the studies analyzed in the National Reading Panel Report in Chapter 119 00:09:45,480 --> 00:09:50,960 2 on phonics, and I encourage everybody to read this, it's online, you can get it for 120 00:09:50,960 --> 00:09:52,120 free. 121 00:09:52,120 --> 00:09:58,640 You'll find that systematic phonics instruction was significantly better than unsystematic 122 00:09:58,640 --> 00:10:02,960 phonics instruction or no phonics instruction. 123 00:10:02,960 --> 00:10:06,920 Of course, something is always better than nothing. 124 00:10:06,920 --> 00:10:12,040 Now some read this and said, see systematic phonics. 125 00:10:12,040 --> 00:10:13,040 There's the research. 126 00:10:13,040 --> 00:10:14,040 It proves it. 127 00:10:14,040 --> 00:10:19,720 We need to make sure we are teaching systematic phonics in all our classrooms. 128 00:10:19,720 --> 00:10:23,240 But that would be silly. 129 00:10:23,240 --> 00:10:28,400 To be good consumers of educational research, we must learn to critically examine. 130 00:10:28,400 --> 00:10:35,600 Yes, it said systematic phonics instruction was significantly better than unsystematic 131 00:10:35,600 --> 00:10:43,720 phonics instruction or no phonics instruction and better at what you say. 132 00:10:43,720 --> 00:10:50,960 It turns out that using phonics to identify single words or pseudo words out of context, 133 00:10:50,960 --> 00:10:54,440 that's what the measure was. 134 00:10:54,440 --> 00:10:56,840 And that's what phonics is. 135 00:10:56,840 --> 00:11:03,120 Meaning that teaching phonics is better than not teaching phonics on measures of phonics 136 00:11:03,120 --> 00:11:05,480 and who would have thought? 137 00:11:05,480 --> 00:11:11,920 But do these phonics blips transfer to authentic reading situations? 138 00:11:11,920 --> 00:11:17,040 Does it enhance students ability to create meaning with print? 139 00:11:17,040 --> 00:11:22,960 Secondly, the majority of studies related to phonics instruction in the National Reading 140 00:11:22,960 --> 00:11:27,600 Panel report looked at subjects in kindergarten and first grade. 141 00:11:27,600 --> 00:11:31,600 These are beginning readers. 142 00:11:31,600 --> 00:11:39,880 You can't generalize to older populations based on findings of phonics with K1 students. 143 00:11:39,880 --> 00:11:47,280 The report then went on to clarify and this is why you have to read the total report. 144 00:11:47,280 --> 00:11:51,560 You can't pick pieces from a report or a study. 145 00:11:51,560 --> 00:11:54,040 That's not scientific. 146 00:11:54,040 --> 00:11:56,000 The report went on to clarify. 147 00:11:56,000 --> 00:12:02,120 Phonics instruction failed to produce a significant impact on the reading performance of low achieved 148 00:12:02,120 --> 00:12:05,800 being readers in second through sixth grade. 149 00:12:05,800 --> 00:12:07,440 What often happens? 150 00:12:07,440 --> 00:12:11,280 Low achieving readers, more phonics. 151 00:12:11,280 --> 00:12:14,160 This does not say that phonics should not be used. 152 00:12:14,160 --> 00:12:16,960 It says we have to look at the results. 153 00:12:16,960 --> 00:12:18,280 What does it say? 154 00:12:18,280 --> 00:12:22,400 The impact of phonics on comprehension is limited. 155 00:12:22,400 --> 00:12:27,920 The majority of studies focused on first grade students asking them to read words or a single 156 00:12:27,920 --> 00:12:30,760 sentence. 157 00:12:30,760 --> 00:12:37,000 Phonics instruction contributed only weekly, if at all, in helping poor readers apply these 158 00:12:37,000 --> 00:12:42,800 skills to reading text. 159 00:12:42,800 --> 00:12:47,720 There was insufficient data to draw any conclusions about the effects of phonics instruction with 160 00:12:47,720 --> 00:12:52,880 normally developing readers above grade one. 161 00:12:52,880 --> 00:12:55,080 What is the science of reading? 162 00:12:55,080 --> 00:12:57,680 What are they advocating? 163 00:12:57,680 --> 00:13:01,680 More systematic phonics instruction. 164 00:13:01,680 --> 00:13:06,560 They thought there was a reading crisis, which I have yet to see some data to support. 165 00:13:06,560 --> 00:13:13,600 They thought that phonics, which they came up with, was the cause of it. 166 00:13:13,600 --> 00:13:17,280 If you're going to be a science of reading, you can't be a partial science of reading. 167 00:13:17,280 --> 00:13:19,680 You can't be a limited science of reading. 168 00:13:19,680 --> 00:13:25,400 If you're concluding that there's a reading crisis, let's get some valid data to show 169 00:13:25,400 --> 00:13:26,600 that. 170 00:13:26,600 --> 00:13:35,400 If you think phonics is the cause of the reading crisis, let's see some valid data, some scientific 171 00:13:35,400 --> 00:13:37,920 data to support that. 172 00:13:37,920 --> 00:13:44,440 If you think phonics is the answer for the mythical reading crisis, let's see some valid 173 00:13:44,440 --> 00:13:46,360 data to support that. 174 00:13:46,360 --> 00:13:52,200 You can't be just a little bit of science of reading or partial science of reading or 175 00:13:52,200 --> 00:13:55,640 semi quasi science of reading. 176 00:13:55,640 --> 00:13:59,120 You either are or you aren't. 177 00:13:59,120 --> 00:14:05,160 To be good consumers of educational research, you must learn how to critically examine research. 178 00:14:05,160 --> 00:14:13,440 Don't accept by blind face when someone says research shows that or research says or research 179 00:14:13,440 --> 00:14:16,720 has proven or it's been debunked. 180 00:14:16,720 --> 00:14:19,120 Scientists have debunked. 181 00:14:19,120 --> 00:14:20,520 Always ask for the citation. 182 00:14:20,520 --> 00:14:25,240 It takes a minute or two to find an abstract and then get a sense of the study or read 183 00:14:25,240 --> 00:14:28,000 the actual study. 184 00:14:28,000 --> 00:14:32,240 Let's look at experimental and quasi experimental research. 185 00:14:32,240 --> 00:14:38,040 To be a true scientific experiment, there must be random assignment to groups. 186 00:14:38,040 --> 00:14:44,520 That is, subjects are randomly put into either the control or the experimental group. 187 00:14:44,520 --> 00:14:50,240 But this is often not possible in education because students are already in groups called 188 00:14:50,240 --> 00:14:52,640 classrooms. 189 00:14:52,640 --> 00:14:58,160 So in education, you see quasi experimental research. 190 00:14:58,160 --> 00:15:04,920 This is research in experimental research in which students are not randomly assigned 191 00:15:04,920 --> 00:15:06,080 to groups. 192 00:15:06,080 --> 00:15:11,720 Quasi experimental research, the subjects are not randomly assigned to groups. 193 00:15:11,720 --> 00:15:15,000 But you can't compare things that are not comparable. 194 00:15:15,000 --> 00:15:21,240 You can't make comparison of control and experimental groups if they aren't the same at the beginning. 195 00:15:21,240 --> 00:15:26,720 So there's always some sort of measure done up front to show that the groups are relatively 196 00:15:26,720 --> 00:15:27,720 the same. 197 00:15:27,720 --> 00:15:32,240 And you must look for that in quasi experimental groups. 198 00:15:32,240 --> 00:15:38,280 Have they made the case that these two groups are the same or relatively equal? 199 00:15:38,280 --> 00:15:41,200 And let's take a look at ethics. 200 00:15:41,200 --> 00:15:45,560 Educational research isn't always black and white because you can't withhold treatment 201 00:15:45,560 --> 00:15:48,440 that you know is essential. 202 00:15:48,440 --> 00:15:55,760 For example, we know that reading volume is highly correlated with gains in word identification, 203 00:15:55,760 --> 00:15:59,520 comprehension, fluency and conceptual knowledge. 204 00:15:59,520 --> 00:16:04,280 But we can't test this out in the classroom using control and experimental groups. 205 00:16:04,280 --> 00:16:09,840 We can't deny or limit the amount of reading students do because we know that reading is 206 00:16:09,840 --> 00:16:10,840 good. 207 00:16:10,840 --> 00:16:13,340 It would be unethical. 208 00:16:13,340 --> 00:16:19,800 So the best we can do here is to find existing groups in which some students read a lot and 209 00:16:19,800 --> 00:16:22,120 some students didn't read. 210 00:16:22,120 --> 00:16:28,360 And this becomes complex because the teacher, the environmental conditions, ability levels, 211 00:16:28,360 --> 00:16:32,280 age, these are all different. 212 00:16:32,280 --> 00:16:38,280 Now I don't mean to imply that controlled experimental research should not be used. 213 00:16:38,280 --> 00:16:39,280 That's not true. 214 00:16:39,280 --> 00:16:42,440 It is a valuable tool. 215 00:16:42,440 --> 00:16:45,680 It should not be used exclusively. 216 00:16:45,680 --> 00:16:51,880 It should not be the only tool that you use to determine causal relationships, to find 217 00:16:51,880 --> 00:16:54,040 what works in a classroom. 218 00:16:54,040 --> 00:17:00,400 And with any research-based claim, you must always critically examine the research and 219 00:17:00,400 --> 00:17:01,880 evaluate it. 220 00:17:01,880 --> 00:17:07,520 Now, I want to end with the idea of irrefutable proof. 221 00:17:07,520 --> 00:17:14,840 Another great understanding related to science and reading science is that irrefutable proof 222 00:17:14,840 --> 00:17:19,040 has been found that certain strategies work best. 223 00:17:19,040 --> 00:17:20,760 They've been proven. 224 00:17:20,760 --> 00:17:24,040 It's been proven that one thing works and another thing hasn't. 225 00:17:24,040 --> 00:17:30,960 And this is a naive, simplistic understanding of science and reading science and educational 226 00:17:30,960 --> 00:17:33,800 science. 227 00:17:33,800 --> 00:17:38,440 If you teach phonics to group A and no phonics to group B, group A is going to score higher 228 00:17:38,440 --> 00:17:40,400 on measures of phonics. 229 00:17:40,400 --> 00:17:41,720 Absolutely. 230 00:17:41,720 --> 00:17:46,640 But this does not mean that phonics should be used for everyone for all time. 231 00:17:46,640 --> 00:17:48,440 It's not irrefutable proof. 232 00:17:48,440 --> 00:17:52,960 You can't make a universal application based on one study. 233 00:17:52,960 --> 00:17:57,320 You always have to look at context. 234 00:17:57,320 --> 00:18:03,160 And we should never use a single source, a single study, to come to conclusions on anything. 235 00:18:03,160 --> 00:18:12,360 Now, there are many areas in reading in which we need to make improvements. 236 00:18:12,360 --> 00:18:17,400 But the hysteria of a national reading crisis is not borne out by the data. 237 00:18:17,400 --> 00:18:22,760 There's no irrefutable evidence that says we're experiencing a national literacy crisis. 238 00:18:22,760 --> 00:18:28,680 And even if there were, there is no irrefutable evidence that it's caused by a lack of phonics 239 00:18:28,680 --> 00:18:32,120 and that more phonics instruction will help. 240 00:18:32,120 --> 00:18:37,600 If you're going to adhere to the science of reading, you have to adhere to all of the 241 00:18:37,600 --> 00:18:39,080 science of reading. 242 00:18:39,080 --> 00:18:45,640 You can't cherry pick data and bits of science that you want to attend to. 243 00:18:45,640 --> 00:18:47,480 This has been the Reading Instruction Show. 244 00:18:47,480 --> 00:19:15,560 I'm your host, Dr. Andy Johnson.