1
00:00:00,000 --> 00:00:06,160
This is the Reading Instruction Show. I'm your host as always, Dr. Andy Johnson.

2
00:00:06,160 --> 00:00:13,520
The topic of today's podcast is evidence-based instruction and evidence-based is in quotation

3
00:00:13,520 --> 00:00:22,920
marks. Another name for today's topic is evidence outside of context is Faldarol.

4
00:00:22,920 --> 00:00:29,840
The hoopla in the education world around the term evidence-based. That term is often thrown

5
00:00:29,840 --> 00:00:36,120
around carelessly like a dead fish at a rummage sale and used to end discussion.

6
00:00:36,120 --> 00:00:45,200
It's evidence-based, they declare. We use evidence-based instruction. Their t-shirts

7
00:00:45,200 --> 00:00:53,680
proudly exclaim and on the sides of their shiny packages they print its evidence-based and you

8
00:00:53,680 --> 00:01:00,520
say, of course it is. And of course you do. And of course it is. And pardon me all to hell for

9
00:01:00,520 --> 00:01:07,400
asking the question. I'll try to do better next time. However, there are words and there is

10
00:01:07,400 --> 00:01:15,920
reality. Reality is comprised of words, but words by themselves are not reality. Words,

11
00:01:15,920 --> 00:01:24,760
words, words. Evidence-based is by a hyphenated word or term that has little meaning by itself,

12
00:01:24,760 --> 00:01:31,320
other than to indicate that one has heard this term before and one thinks it appropriate to use

13
00:01:31,320 --> 00:01:40,360
to explain or defend practices. But I would question and I would pause it and pardon me for

14
00:01:40,360 --> 00:01:47,000
being so inquisitive that when you say a practice or strategy is evidence-based,

15
00:01:47,000 --> 00:01:54,520
have you actually looked at the evidence of the evidence-based? Or are you just using the term

16
00:01:54,520 --> 00:02:01,240
because you heard somebody use it? Now let us consider for the sake of argument some evidence-based

17
00:02:01,240 --> 00:02:09,280
practices. Giving electric shocks to mice in a skinner box is evidence-based, but that doesn't

18
00:02:09,280 --> 00:02:17,120
mean we're going to advocate this practice in a classroom. Yes? You could cut out all recesses

19
00:02:17,120 --> 00:02:24,480
and make kids stay inside and fill out worksheets. And I'm sure you could collect evidence that would

20
00:02:24,480 --> 00:02:31,000
indicate that these kids got better at filling out worksheets. It would be an evidence-based

21
00:02:31,000 --> 00:02:40,560
practice, but that doesn't mean we advocate it. You could assign 10 to 15 words for students to

22
00:02:40,560 --> 00:02:47,560
study every week and give them a spelling test on Friday. You could collect evidence that showed

23
00:02:47,560 --> 00:02:54,560
that students got more correct on Friday than they did on Monday, but this doesn't mean they're

24
00:02:54,560 --> 00:03:01,160
better able to spell under real life writing conditions. And it doesn't mean that traditional

25
00:03:01,160 --> 00:03:07,600
spelling instruction is better than other methods or better than no method at all or better than

26
00:03:07,600 --> 00:03:15,920
simply writing a lot with no spelling instruction. Yet, since there is evidence, it must be evidence.

27
00:03:15,920 --> 00:03:26,840
Yes? No? You could give reading instruction to 25 students in Arkansas using a for-profit

28
00:03:26,840 --> 00:03:34,200
reading program that was implemented with fidelity and compare it to reading instruction to 25

29
00:03:34,200 --> 00:03:41,600
students in West Virginia using a meaning-based program where students read a lot of books. If

30
00:03:41,600 --> 00:03:50,080
the average post-treatment scores of the Arkansas students was say 75% and West Virginia students

31
00:03:50,080 --> 00:03:57,880
say 73%, you could indeed say that there was evidence showing that the for-profit program

32
00:03:57,880 --> 00:04:07,320
students scored higher. But is it valid evidence? Is it reliable evidence? And if the evidence

33
00:04:07,320 --> 00:04:15,880
was collected by the for-profit company, would it be creditable evidence? Yet, one could say

34
00:04:15,880 --> 00:04:23,640
that the for-profit program was implemented with fidelity and it was evidence-based. Imagine

35
00:04:23,640 --> 00:04:32,080
that. When someone trots out the tireless refrain of evidence-based, one must always

36
00:04:32,080 --> 00:04:41,880
ask what kind of evidence? For whom? For what? For what purpose? Evidence outside of any

37
00:04:41,880 --> 00:04:50,000
context is falderol. Falderol, by the way, is trivial or nonsensical fuss and that's

38
00:04:50,000 --> 00:05:00,520
exactly what decontextualized evidence is. It's trivial or nonsensical fuss. In any research

39
00:05:00,520 --> 00:05:08,880
study, the type of evidence collected depends upon the question asked and the method of

40
00:05:08,880 --> 00:05:16,280
collection. For example, I could ask the question, will method X improve students' ability

41
00:05:16,280 --> 00:05:25,000
to sound out words? I would give sounding out word pre-treatment measures to a group. Method

42
00:05:25,000 --> 00:05:31,880
X would include sounding out word instruction. The post-treatment sounding out word measure

43
00:05:31,880 --> 00:05:38,600
would most likely show that there was an increase in students' ability to sound out words. One

44
00:05:38,600 --> 00:05:46,800
could thus say that sounding out word instruction is evidence-based. Okay, fair enough. There's

45
00:05:46,800 --> 00:05:52,800
evidence to support its efficacy of sounding out words in students' ability to sound out

46
00:05:52,800 --> 00:05:59,840
words. Yes, indeed. That's like saying teaching students to play tennis enhances their ability

47
00:05:59,840 --> 00:06:07,560
to play tennis. But this does not mean that method X is better than other measures in

48
00:06:07,560 --> 00:06:13,320
recognizing words or identifying words or comprehension text or comprehending text or

49
00:06:13,320 --> 00:06:23,800
helping students learn to read. But yet, method X is said to be evidence-based. Now, let's

50
00:06:23,800 --> 00:06:29,680
take a look at Special Ed World and the Science of Reading Comedy Club. This is the last

51
00:06:29,680 --> 00:06:36,480
thing I'll say about evidence-based. Can it really be evidence-based if it only looks

52
00:06:36,480 --> 00:06:44,840
at a certain type of evidence? And that's the whole problem with the liturgical chance

53
00:06:44,840 --> 00:06:50,800
of evidence and evidence-based made by the Science of Reading Comedy Club to support

54
00:06:50,800 --> 00:06:58,800
a litany of practices taken from their direct instruction holy book. That's the problem

55
00:06:58,800 --> 00:07:05,840
in Special Ed World with their declaration of faith stating that practices used to manipulate

56
00:07:05,840 --> 00:07:16,240
and control behavior are evidence-based. These represent a simplistic understanding of research

57
00:07:16,240 --> 00:07:26,280
in the social sciences. Within Special Ed World and the Science of Reading Comedy Club,

58
00:07:26,280 --> 00:07:32,640
controlled experimental studies are thought to be the only way of establishing causal

59
00:07:32,640 --> 00:07:40,840
relationships that one thing makes the other thing happen. They have determined that only

60
00:07:40,840 --> 00:07:49,400
a single type of research methodology can be used to ask and answer questions in the field

61
00:07:49,400 --> 00:08:01,080
of education. This is ironically labeled as scientifically-based research. However, in

62
00:08:01,080 --> 00:08:10,480
true scientific inquiry, the question determines the method. The method does not determine

63
00:08:10,480 --> 00:08:19,560
the question. And how can anything be said to be truly scientific if it ignores or invalidates

64
00:08:19,560 --> 00:08:27,600
an abundance of data? These groups, Special Ed World and the Science of Reading Comedy

65
00:08:27,600 --> 00:08:36,160
Club, mandate that the same research methods used to study bacteria in a Petri dish be

66
00:08:36,160 --> 00:08:44,480
used to study children in a classroom. But humans are not standardized products and there

67
00:08:44,480 --> 00:08:52,360
are far too many variables in the real world to control or isolate. As well, the results

68
00:08:52,360 --> 00:08:59,280
of large controlled experimental research studies do not always generalize to smaller

69
00:08:59,280 --> 00:09:09,480
populations. In identifying scientifically-based research and evidence-based practices in education,

70
00:09:09,480 --> 00:09:14,960
it's not the type of research methodology that's the problem. I mean, there's nothing

71
00:09:14,960 --> 00:09:21,840
wrong with controlled experimental studies. I'm not advocating that they be eliminated.

72
00:09:21,840 --> 00:09:32,920
It's the exclusive use of one type of research methodology to the exclusion of all others.

73
00:09:32,920 --> 00:09:41,000
This creates a methodological people that results in a very narrow and unrealistic view

74
00:09:41,000 --> 00:09:49,840
of the very educational reality it seeks to examine. The results is that terms like research

75
00:09:49,840 --> 00:09:58,760
and research-based and evidence-based are often used inappropriately to justify a variety

76
00:09:58,760 --> 00:10:08,680
of bad decisions. Saying something that is evidence-based does not end the conversation.

77
00:10:08,680 --> 00:10:16,360
What kind of evidence? For what? For whom? For what purpose? We always must look at the

78
00:10:16,360 --> 00:10:19,800
evidence behind the evidence-based claims.