WEBVTT 00:00:00.680 --> 00:00:03.319 This is the Reading Instruction Show. I'm your 00:00:03.319 --> 00:00:06.860 host, as always, Dr. Andy Johnson. Topic of today's 00:00:06.860 --> 00:00:12.839 podcast, the latest NCTQ technical report does 00:00:12.839 --> 00:00:17.379 not meet basic academic standards. And it's an 00:00:17.379 --> 00:00:19.739 excerpt from my book, Making Sense of the Science 00:00:19.739 --> 00:00:23.120 of Reading, Teachers Matter, that will be published 00:00:23.120 --> 00:00:27.160 by Guilford sometime in 2028. Now the National 00:00:27.160 --> 00:00:31.589 Council on Teacher Quality, NCTQ, recently put 00:00:31.589 --> 00:00:36.170 out another technical report. This technical 00:00:36.170 --> 00:00:40.229 report graded 700 teacher preparation programs 00:00:40.229 --> 00:00:45.450 across the country. However, a critical evaluation 00:00:45.450 --> 00:00:49.789 shows why this report can't be taken seriously. 00:00:50.890 --> 00:00:55.270 It contains an abundance of misinformation, disinformation, 00:00:55.609 --> 00:01:00.009 and convenient inaccuracies. And it's not known 00:01:00.009 --> 00:01:02.950 if these were intentional on the part of the 00:01:02.950 --> 00:01:07.010 writers or a result of a lack of knowledge. And 00:01:07.010 --> 00:01:12.489 this chapter contains my initial analysis, but 00:01:12.489 --> 00:01:16.230 first a little context. Part one, the necessity 00:01:16.230 --> 00:01:21.390 of analyses and critical evaluations. Academia, 00:01:21.469 --> 00:01:25.230 or the academy, refers to the community of scholars 00:01:25.230 --> 00:01:31.319 and researchers who seek new knowledge. And within 00:01:31.319 --> 00:01:35.340 the academy, members seek to maintain standards 00:01:35.340 --> 00:01:40.879 within their respective academic fields. This 00:01:40.879 --> 00:01:45.019 means we critically evaluate each other's work. 00:01:45.700 --> 00:01:49.780 And this is an important part of the academic 00:01:49.780 --> 00:01:55.359 process. This type of critical evaluation along 00:01:55.359 --> 00:01:59.620 with accompanying academic discourse keeps our 00:01:59.620 --> 00:02:04.780 various academic fields evolving. Now academic 00:02:04.780 --> 00:02:07.540 discourse has traditionally occurred through 00:02:07.540 --> 00:02:11.520 our books, journals, and academic conferences, 00:02:12.240 --> 00:02:17.860 but today increasingly these conversations also 00:02:17.860 --> 00:02:23.000 occur on various social media platforms. This 00:02:23.000 --> 00:02:27.860 is a healthy development as it opens up the conversation 00:02:27.860 --> 00:02:33.860 to a wider audience and more participants. However, 00:02:34.139 --> 00:02:37.439 the one thing missing on social media platforms, 00:02:37.960 --> 00:02:41.520 as you know, is any form of peer review or checks 00:02:41.520 --> 00:02:45.780 and balances. Anybody can write things or say 00:02:45.780 --> 00:02:50.409 things. And in this sense, The conversations 00:02:50.409 --> 00:02:53.629 taking place on social media are no different 00:02:53.629 --> 00:03:00.550 from this NCTQ technical report. Literacy instruction 00:03:00.550 --> 00:03:04.949 is a field within the Academy, and I am a scholar 00:03:04.949 --> 00:03:09.689 in the field of literacy instruction. As such, 00:03:09.789 --> 00:03:13.449 I will be analyzing and critically evaluating 00:03:13.449 --> 00:03:18.539 this technical report on its merit. Nobody gets 00:03:18.539 --> 00:03:22.659 a free ride in the academy. In the academy we 00:03:22.659 --> 00:03:27.419 must all be held accountable for our words. And 00:03:27.419 --> 00:03:30.800 I would expect that someone will do the same 00:03:30.800 --> 00:03:35.280 type of evaluation with this chapter. And that 00:03:35.280 --> 00:03:40.659 is a wonderful thing. Chapter 2, there is no 00:03:40.659 --> 00:03:45.560 reading crisis. The first sentence of the second 00:03:45.560 --> 00:03:49.300 paragraph of this report says that, quote, 40 00:03:49.300 --> 00:03:53.159 % of fourth graders in the United States cannot 00:03:53.159 --> 00:03:57.460 read at a basic level. The worst results in over 00:03:57.460 --> 00:04:03.340 two decades, unquote. Now this sentence would 00:04:03.340 --> 00:04:07.259 have you believe there's a reading crisis, but 00:04:07.259 --> 00:04:12.069 this would be a convenient inaccuracy. There 00:04:12.069 --> 00:04:17.069 is no irrefutable evidence that we are experiencing 00:04:17.069 --> 00:04:24.009 a national literacy crisis. NAEP reading scores 00:04:24.009 --> 00:04:28.949 for fourth and eighth graders from 1971 to 2020 00:04:28.949 --> 00:04:32.709 shows that there have been natural fluctuations 00:04:32.709 --> 00:04:37.889 as we would expect, but there have been no massive 00:04:37.889 --> 00:04:43.209 declines in reading test scores. In both fourth 00:04:43.209 --> 00:04:47.370 grade and eighth grade, average scores are slightly 00:04:47.370 --> 00:04:51.730 higher than they were in 1971, up to the COVID 00:04:51.730 --> 00:04:57.889 dip in 2020. Also, pre -COVID scores are basically 00:04:57.889 --> 00:05:04.329 the same as they were 20 years ago. Because the 00:05:04.329 --> 00:05:08.329 same norm reference test versus criterion reference 00:05:08.329 --> 00:05:13.550 tests are given over time, NAP data enables comparisons 00:05:13.550 --> 00:05:17.269 to be made between states and various subgroups 00:05:17.269 --> 00:05:22.329 and over time. Also, the populations tested are 00:05:22.329 --> 00:05:25.389 representative and large enough for the results 00:05:25.389 --> 00:05:29.230 to be generalized with a fair amount of confidence. 00:05:29.610 --> 00:05:34.810 That's why we use NAEP data and not state criterion 00:05:34.810 --> 00:05:39.949 reference tests. In both fourth and eighth grade, 00:05:40.189 --> 00:05:44.149 the 2024 scores were lower than they were before 00:05:44.149 --> 00:05:48.449 the COVID crisis in 2019, as we would expect, 00:05:48.490 --> 00:05:51.769 and this difference is statistically significant. 00:05:52.519 --> 00:05:56.139 Statistical significance means that the differences 00:05:56.139 --> 00:06:00.339 are large enough to eliminate chance or error 00:06:00.339 --> 00:06:07.959 as the cause of the difference. By 2022, 37 states 00:06:07.959 --> 00:06:12.779 and territories had passed reading laws. We might 00:06:12.779 --> 00:06:17.300 have expected to see some impact on the national 00:06:17.300 --> 00:06:20.480 trends of fourth grade students by 2024, but 00:06:20.480 --> 00:06:27.680 this has not been the case. As of 2026, 44 states 00:06:27.680 --> 00:06:32.939 and Washington DC have passed legislation mandating 00:06:32.939 --> 00:06:36.680 evidence -based reading instruction aligned with 00:06:36.680 --> 00:06:42.300 the science of reading. And if this news scientifically 00:06:42.300 --> 00:06:46.290 based Reading instruction works as promised. 00:06:47.170 --> 00:06:52.990 In 2028 and later, we should see NAP scores of 00:06:52.990 --> 00:06:56.129 fourth grade students to be at least where they 00:06:56.129 --> 00:07:00.050 were in the pre -COVID era. That means 220 or 00:07:00.050 --> 00:07:06.709 higher. Check back in 2028. Part three, a basic 00:07:06.709 --> 00:07:10.170 understanding of basic level. So let's get to 00:07:10.170 --> 00:07:14.740 the crux of the matter. As stated above, the 00:07:14.740 --> 00:07:19.720 NCTQ report claims that 40 % of fourth grade 00:07:19.720 --> 00:07:24.019 students in the United States cannot read at 00:07:24.019 --> 00:07:29.000 a basic level. And if this were even a little 00:07:29.000 --> 00:07:33.560 bit true, it would be a horrible thing. But the 00:07:33.560 --> 00:07:36.639 problem with this technical report is that it's 00:07:36.639 --> 00:07:41.430 not even a little bit true. So that means that 00:07:41.430 --> 00:07:44.350 everything that follows is based on a false premise. 00:07:45.250 --> 00:07:51.290 Let me explain. Basic level. NAEP test data shows 00:07:51.290 --> 00:07:53.250 that in general, teachers have been doing a pretty 00:07:53.250 --> 00:07:58.949 good job. So to find some failure in 1992, the 00:07:58.949 --> 00:08:03.569 NAEP decided to add three levels of proficiency. 00:08:04.230 --> 00:08:10.389 Basic, proficient, and advanced. However, These 00:08:10.389 --> 00:08:14.990 proficiency levels were arbitrarily defined. 00:08:17.029 --> 00:08:21.170 In addressing NAEP levels, Rinking Ruby and Risco 00:08:21.170 --> 00:08:41.820 said, and I quote, who had been contracted to 00:08:41.820 --> 00:08:45.080 advise the department of this approach, but who 00:08:45.080 --> 00:08:47.899 were overruled when they were argued against 00:08:47.899 --> 00:08:55.980 doing it at all." These NAEP proficiency levels 00:08:55.980 --> 00:09:01.919 should be given very little credence. The National 00:09:01.919 --> 00:09:06.120 Center for Educational Statistics website provides 00:09:06.120 --> 00:09:13.700 three reasons for this. They said, this is NCEES, 00:09:14.340 --> 00:09:16.740 proficiency levels are based on expectations, 00:09:17.080 --> 00:09:21.159 meaning that these levels are aspirational, not 00:09:21.159 --> 00:09:26.100 mathematical. Two, these proficiency levels are 00:09:26.100 --> 00:09:29.519 currently being used on a trial basis, since 00:09:29.519 --> 00:09:33.700 1992 apparently, but that's what it says on their 00:09:33.700 --> 00:09:37.460 website. And three, Further evidence is needed 00:09:37.460 --> 00:09:40.379 to determine if these proficiency levels are 00:09:40.379 --> 00:09:47.000 reasonable, valid, and informative. So let me 00:09:47.000 --> 00:09:51.620 put it plainly. A group of people got together 00:09:51.620 --> 00:09:56.659 in 1992. They saw that the average score of fourth 00:09:56.659 --> 00:10:03.379 grade students was 217. So if that was average, 00:10:03.639 --> 00:10:07.830 they decided That basic had to be something a 00:10:07.830 --> 00:10:12.690 little below that. And a score of 208 was a little 00:10:12.690 --> 00:10:18.450 below 217. So 208 was declared to be the basic 00:10:18.450 --> 00:10:23.450 level. It had nothing to do with basic skills. 00:10:23.990 --> 00:10:27.669 It had nothing to do with what was or wasn't 00:10:27.669 --> 00:10:32.970 learned. The sole determination of basic level 00:10:33.230 --> 00:10:40.909 was that it wasn't quite 217. Imagine that. If 00:10:40.909 --> 00:10:44.169 you arbitrarily decide that the basic level will 00:10:44.169 --> 00:10:47.929 be slightly below average, you are guaranteeing 00:10:47.929 --> 00:10:52.529 that at least 40 % of a normal population will 00:10:52.529 --> 00:10:57.029 be reading below the basic level. In other words, 00:10:57.230 --> 00:11:00.769 if the basic level is at the 40th percentile, 00:11:00.889 --> 00:11:05.009 40 % of the population will be below a basic 00:11:05.009 --> 00:11:09.470 level. That's how percentiles work, you see? 00:11:11.529 --> 00:11:15.330 Let me show you from another perspective. A normal 00:11:15.330 --> 00:11:19.590 population is represented by the familiar bell 00:11:19.590 --> 00:11:23.809 -shaped curve. And in any normal population, 00:11:24.269 --> 00:11:28.429 50 % are going to be above the mean and 50 % 00:11:28.429 --> 00:11:33.860 below. This means that there always be 50 % who 00:11:33.860 --> 00:11:39.700 score below average. This is called normal. And 00:11:39.700 --> 00:11:44.600 if you arbitrarily define basic level as slightly 00:11:44.600 --> 00:11:48.659 below the mean, you'll have at least 40 % of 00:11:48.659 --> 00:11:52.419 the population reading below the basic level. 00:11:53.259 --> 00:12:01.149 Imagine that. So, final word. In this chapter, 00:12:01.210 --> 00:12:03.850 I've addressed only the first three paragraphs 00:12:03.850 --> 00:12:09.330 of this 107 -page technical report. But based 00:12:09.330 --> 00:12:13.649 on this, one should carefully consider whether 00:12:13.649 --> 00:12:18.570 to spend time reading the next 106 pages. However, 00:12:18.789 --> 00:12:23.549 six big ideas here. Number one, literacy instruction 00:12:23.549 --> 00:12:28.000 is a field within the greater academy. Critical 00:12:28.000 --> 00:12:32.100 analyses and evaluations are an essential part 00:12:32.100 --> 00:12:37.639 of the academy. It keeps academic fields evolving, 00:12:38.279 --> 00:12:40.679 maintaining a set of academic standards, and 00:12:40.679 --> 00:12:45.259 provides a set of checks and balances. Two, there 00:12:45.259 --> 00:12:50.600 never was a literacy crisis. Three, the basic 00:12:50.600 --> 00:12:55.059 level is an arbitrarily defined level, guaranteeing 00:12:55.100 --> 00:12:58.840 that approximately 40 % of the population will 00:12:58.840 --> 00:13:04.379 always be below the basic level. Four, the basic 00:13:04.379 --> 00:13:07.779 level is not based on a set of skills mastered, 00:13:08.080 --> 00:13:11.960 rather it's based on a score being a little below 00:13:11.960 --> 00:13:17.379 average. Five, we should all be held accountable 00:13:17.379 --> 00:13:21.720 for our words. Nobody in the academy should get 00:13:21.720 --> 00:13:30.110 a free ride. And if the NCTQ enters into an academic 00:13:30.110 --> 00:13:33.490 field, such as literacy instruction, if they 00:13:33.490 --> 00:13:38.049 enter that discussion, they should be held to 00:13:38.049 --> 00:13:41.690 the same academic standards as the rest of us. 00:13:43.509 --> 00:13:49.009 And number six, the misuse of endnotes in citations 00:13:49.009 --> 00:13:53.009 has not been addressed. However, it's abundantly 00:13:53.009 --> 00:13:56.470 clear after reading the first three paragraphs 00:13:56.470 --> 00:14:02.950 that this NCTQ technical report does not meet 00:14:02.950 --> 00:14:07.490 basic academic standards. In the next podcast 00:14:07.490 --> 00:14:12.710 or chapter, we'll examine this NCTQ report as 00:14:12.710 --> 00:14:15.830 it relates to teacher preparation programs. This 00:14:15.830 --> 00:14:18.289 has been The Reading Instruction Show. I am your 00:14:18.289 --> 00:14:19.769 host, Dr. Andy Johnson.