This is the Reading Instruction Show.
I'm your host, as always, Dr. Andy Johnson.
The topic of today's podcast is understanding science and reading science.
And I want to just cover some of the basics of what science and reading science is.
So let's define our terms.
Much of the confusion around the science of reading is related to a lack of understanding
about what science is and research and educational research and literacy research.
I call it the great ununderstanding.
I think we generally assume there's a common understanding, but there is not.
So let's try to get on the same page starting with science.
Science is a process.
It's a verb to science.
It's something we do.
The basic essence of science is this.
You ask a question and you use data to answer the question.
And what makes science different from simply collecting data is that science uses a systematic
process to collect data.
Systematic means that there's a fixed plan in place based on a set of principles that
are generally agreed upon by the field.
Now a research method is the specific systematic process used to gather and analyze the data.
And here's the first understanding related to the science of reading.
There is no scientific method.
What you say I can hear you thinking.
Repeat.
There is no singular method of science.
Rather, there are methods of science.
While it's commonly thought that experimental research with control and experimental groups
pre and post test is the scientific method, this is only one type of scientific method.
There are a variety of scientific methods, including observational research, qualitative
research, descriptive research, ethnographic research.
And I don't want to get into a lecture on all the different types of research methods
used, but the big idea here is that research starts with a question.
The method used, the method of science used is dependent on the question.
Different questions call for different methods.
Now according to the science of reading, only strategies that have been shown to be effective
by research conducted in actual learning environments using controlled experimental research can
be said to be research based.
These are the evidence based strategies.
These are the ones that should be used to teach reading.
And the typical controlled experimental research study goes something like this.
You start with two relatively similar groups.
You do something to one group and nothing to the other group to see if that something
caused the other thing to happen.
If everything is the same in the two groups except for that one thing and there's a difference
at the end, we can say with a great deal of confidence that there is a causal relationship,
that the one thing caused the other thing to happen.
There ever after, if we want one thing to occur, we just have to present the other thing.
If A causes B and we want B, just do a lot of A and B will occur.
What could be simpler than that?
Yes?
Controlled experimental research works well in the physical sciences where things are
relatively similar, meaning that a strain of bacteria in Bolivia is similar to the same
type of bacteria in Alaska.
However, humans are not standardized entities.
We're all wonderfully gloriously unique.
We all have a variety of different forces impacting us.
A third grade student from a poor rural school is not the same as a third grade student from
a wealthy suburban school.
As well, teachers are unique, environments, values, cultures, situations are all unique.
Humans do not exist in a vacuum.
They do not exist in a Petri dish.
This is why research in the physical sciences looks different from research in the social
sciences.
The methods used to study bacteria in a Petri dish really are limited when trying to examine
children in a classroom.
Human beings do not react similarly to similar stimuli.
There are far too many variables at play in any classroom and with human beings to try
to control and account for them all.
And the very act of controlling an environment for research makes that environment a non-real
environment.
The science of reading contends that reading science involves only that singular type of
research methodology.
Only controlled experimental research conducted in an actual learning environment can be used
to ask and answer questions related to reading or to determine causal relationships.
I do think this understanding of research is a little bit limiting.
By limiting the type of research that can be used and the type of data that counts as
knowledge, you're creating a narrow people through which to view reality.
However, I could live with this limited understanding of research if it were consistently used.
If the research used with the science of reading was accurately interpreted and correctly applied
and if a wide body of research were used to come to conclusions, but it's not.
And it's often misapplied or misinterpreted.
Now, contrary to popular thought, and here's a popular thought, you can make research say
anything you want to say.
Well, you can't.
You can't make research say anything you want to say.
You can collect data to say anything you want to say.
You can use anecdotal evidence and personal stories to say anything you want to say.
You can misuse research or overinterpret research results to say anything you want to say, but
you can't make research say anything you want to say.
Remember, research is different from collecting data.
It's different from doing a study.
Research is not research unless and until it has been subjected to blind peer review
and published in a journal or some other academic source.
And we never use a singular study to come to conclusions about anything.
We use a wide body of research to get a general sense of the pattern of things.
Now, as an example of the misuse of research, a white paper was put out on a website by
Louisa Mott's in support of her letter's professional development program.
And I tracked down the citations and I analyzed the research that she used to support her
declarative statement.
And I found two things.
Some of the research cited to support her declarative statements was not research at
all.
It was data collected by think tanks or other organizations made to look like it was research.
Research is not research unless and until it has been subjected to blind peer review
and published in an academic journal.
And the second thing, much of the research she cited to support her statements had little
or nothing to do with the declarative statement she was making.
She would make a statement, statement A, and cite it using research study B, but when you
actually looked at research study B, you find that it's unrelated, misinterpreted, or over-interpreted
and didn't support statement A at all.
And that's not right.
Did she do that on purpose or did she really think that that's how research should be
used?
Both.
Both thoughts are rather disturbing.
And let's look at generalization, a common misuse of research is to over-interpret the
results and to try to generalize to a larger population where it is unwarranted.
For example, if you look at the studies analyzed in the National Reading Panel Report in Chapter
2 on phonics, and I encourage everybody to read this, it's online, you can get it for
free.
You'll find that systematic phonics instruction was significantly better than unsystematic
phonics instruction or no phonics instruction.
Of course, something is always better than nothing.
Now some read this and said, see systematic phonics.
There's the research.
It proves it.
We need to make sure we are teaching systematic phonics in all our classrooms.
But that would be silly.
To be good consumers of educational research, we must learn to critically examine.
Yes, it said systematic phonics instruction was significantly better than unsystematic
phonics instruction or no phonics instruction and better at what you say.
It turns out that using phonics to identify single words or pseudo words out of context,
that's what the measure was.
And that's what phonics is.
Meaning that teaching phonics is better than not teaching phonics on measures of phonics
and who would have thought?
But do these phonics blips transfer to authentic reading situations?
Does it enhance students ability to create meaning with print?
Secondly, the majority of studies related to phonics instruction in the National Reading
Panel report looked at subjects in kindergarten and first grade.
These are beginning readers.
You can't generalize to older populations based on findings of phonics with K1 students.
The report then went on to clarify and this is why you have to read the total report.
You can't pick pieces from a report or a study.
That's not scientific.
The report went on to clarify.
Phonics instruction failed to produce a significant impact on the reading performance of low achieved
being readers in second through sixth grade.
What often happens?
Low achieving readers, more phonics.
This does not say that phonics should not be used.
It says we have to look at the results.
What does it say?
The impact of phonics on comprehension is limited.
The majority of studies focused on first grade students asking them to read words or a single
sentence.
Phonics instruction contributed only weekly, if at all, in helping poor readers apply these
skills to reading text.
There was insufficient data to draw any conclusions about the effects of phonics instruction with
normally developing readers above grade one.
What is the science of reading?
What are they advocating?
More systematic phonics instruction.
They thought there was a reading crisis, which I have yet to see some data to support.
They thought that phonics, which they came up with, was the cause of it.
If you're going to be a science of reading, you can't be a partial science of reading.
You can't be a limited science of reading.
If you're concluding that there's a reading crisis, let's get some valid data to show
that.
If you think phonics is the cause of the reading crisis, let's see some valid data, some scientific
data to support that.
If you think phonics is the answer for the mythical reading crisis, let's see some valid
data to support that.
You can't be just a little bit of science of reading or partial science of reading or
semi quasi science of reading.
You either are or you aren't.
To be good consumers of educational research, you must learn how to critically examine research.
Don't accept by blind face when someone says research shows that or research says or research
has proven or it's been debunked.
Scientists have debunked.
Always ask for the citation.
It takes a minute or two to find an abstract and then get a sense of the study or read
the actual study.
Let's look at experimental and quasi experimental research.
To be a true scientific experiment, there must be random assignment to groups.
That is, subjects are randomly put into either the control or the experimental group.
But this is often not possible in education because students are already in groups called
classrooms.
So in education, you see quasi experimental research.
This is research in experimental research in which students are not randomly assigned
to groups.
Quasi experimental research, the subjects are not randomly assigned to groups.
But you can't compare things that are not comparable.
You can't make comparison of control and experimental groups if they aren't the same at the beginning.
So there's always some sort of measure done up front to show that the groups are relatively
the same.
And you must look for that in quasi experimental groups.
Have they made the case that these two groups are the same or relatively equal?
And let's take a look at ethics.
Educational research isn't always black and white because you can't withhold treatment
that you know is essential.
For example, we know that reading volume is highly correlated with gains in word identification,
comprehension, fluency and conceptual knowledge.
But we can't test this out in the classroom using control and experimental groups.
We can't deny or limit the amount of reading students do because we know that reading is
good.
It would be unethical.
So the best we can do here is to find existing groups in which some students read a lot and
some students didn't read.
And this becomes complex because the teacher, the environmental conditions, ability levels,
age, these are all different.
Now I don't mean to imply that controlled experimental research should not be used.
That's not true.
It is a valuable tool.
It should not be used exclusively.
It should not be the only tool that you use to determine causal relationships, to find
what works in a classroom.
And with any research-based claim, you must always critically examine the research and
evaluate it.
Now, I want to end with the idea of irrefutable proof.
Another great understanding related to science and reading science is that irrefutable proof
has been found that certain strategies work best.
They've been proven.
It's been proven that one thing works and another thing hasn't.
And this is a naive, simplistic understanding of science and reading science and educational
science.
If you teach phonics to group A and no phonics to group B, group A is going to score higher
on measures of phonics.
Absolutely.
But this does not mean that phonics should be used for everyone for all time.
It's not irrefutable proof.
You can't make a universal application based on one study.
You always have to look at context.
And we should never use a single source, a single study, to come to conclusions on anything.
Now, there are many areas in reading in which we need to make improvements.
But the hysteria of a national reading crisis is not borne out by the data.
There's no irrefutable evidence that says we're experiencing a national literacy crisis.
And even if there were, there is no irrefutable evidence that it's caused by a lack of phonics
and that more phonics instruction will help.
If you're going to adhere to the science of reading, you have to adhere to all of the
science of reading.
You can't cherry pick data and bits of science that you want to attend to.
This has been the Reading Instruction Show.
I'm your host, Dr. Andy Johnson.