This is the Reading Instruction Show.
I'm your host, Dr. Andy Johnson.
Today is the first in a two-part series of podcasts looking at documenting growth and
monitoring progress for writing or assessing writing.
Now, I want to start out with this idea.
A pig doesn't get heavier by weighing it.
readers don't become better readers and writers by assessing them.
There's this idea that we can assess our way to reading and writing proficiency.
And that's a kooky, wacky, nutty, zany idea that we must divest ourselves out.
So let's start with the why of assessment.
The why questions don't get asked often enough.
Why do we assess?
For what purpose?
Now in terms of writing, I hope the answer is this.
I hope the answer is we assess so that we can help students become better writers.
If not, we're wasting our time.
We assess to determine strengths and to build upon weaknesses.
We assess to document growth, but ultimately we assess to find out what we can do to help
students become better writers.
Now, to offman, assessment is used to, quote, hold teachers accountable, unquote, ug.
Now why is it that some groups continue this war against teachers and insist on the deep
professionalization of education?
Hold teachers accountable.
We want intelligent creative teachers to enter the field, but then we treat them like young
wayward adolescents.
We don't trust them.
We think we need to hold them accountable.
We tell them what they must teach and how they must teach it.
We assign standards and measures that must be used.
Maybe we should give them a curfew as well.
That's it.
Let's give all teachers a curfew and a mandatory bedtime and restrict their TV watching and
phone access if test scores fall below a certain percentage.
Yeah, that's the ticket.
Hold them accountable.
However, what happens when groups continually try to hold teachers accountable?
Teachers teach to the accountability measure.
In other words, they teach to the test.
I mean, who wouldn't?
The result of this accountability nonsense is that instead of testing what is taught,
teachers teach what is tested.
And this is called ask backwards in some circles, other clownery in other circles.
With such ask backward clownery, the curriculum becomes narrow, focusing only on that which
can be measured and quantified by an external entity.
So what doesn't work?
What about standardized tests for writing?
That's the topic of this two-part series assessing writing.
Standardized tests are relatively easy to administer and score.
Takes an hour at most, maybe two.
Whole bunch of students can sit down all at the same time.
And within a short amount of time, the test is complete.
And a norm reference test enables schools to compare each student to a national norm.
This is a wonderful idea.
Yes.
No.
Standardized tests can measure spelling, punctuation, and grammar using multiple choice
or true false questions, but that's not writing.
So let's look at writing tests.
These writing tests have been around for a while.
Here generally students are given a prompt and a piece of paper to do some pre-writing
thinking before writing.
And then they have a time limit to write anywhere from 15 to 30 minutes usually.
And then these are either computer scored or sent off to be rated by scored, trained,
rated and scored by trained raters.
Now Richie Rootman said this about trained raters.
She said, in general, scores are poorly paid, have minimal training, are given very little
time to read each sample, and are expected to read student papers for many hours at a
stretch.
And these people are determining your success as teachers of writing.
Now these are better than standardized tests, the writing tests, but still not good because
it's an artificial writing situation.
It's not what real writers do when they write, hopefully.
Thus, with the writing test, it's a validity question.
Now it would be very easy to teach students to perform like trained seals at a circus
and make writing scores go up on these artificial writing measures.
Absolutely, it would be easy.
You simply break this artificial writing down into steps and teach each step with guided
and independent practice.
Then you'd create a lot of artificial writing experiences with artificial writing prompts
and artificial time limits, and you'd have artificial writing practice, and you'd spend
time talking about your artificial writing and how to get better at artificial writing.
And maybe we could teach students to balance the ball on their nose at the same time.
But the question is this.
Do you really want students to perform like trained seals on some cockamamie writing assessment?
Or do you want students to be able to write and think?
Now the way some schools approach assessment of writing, I sometimes wonder.
Seems like they want the trained seals.
Now let's look at traits of writing.
What if we define the traits of good writing and then taught and assessed each of the traits?
Now that's been done.
In her book Six Plus One, Traits of Writing, Ruth Cullum recommends teaching writing by
focusing on the six plus one or seven traits.
Her emphasis is on teaching these traits and using rubrics to assess.
Now here are the traits.
Number one, ideas, meeting and development of the message.
Number two, organization, the internal structure of the piece.
Number three, voice, the tone of the piece and the personal stamp.
Word choice, the specific vocabulary the writer uses.
Sentence fluency, the way the words and phrases flow.
Conventions, that's the mechanical correctness and presentation, the overall appearance.
These are the seven traits that Cullum says are so very important.
And she creates rubrics.
This approach is based on rubrics.
Each rubric has seven traits, the seven traits I just said and descriptions of five levels
of each trait.
You simply look at students writing and assign them a number for each trace based on the
rubric description.
Better writing through rubrics, yes?
No.
The problem with the six plus one traits is that the focus is on the product or the traits
and not on the process.
The traits then become the writing program.
You're teaching traits and not process.
But who was it that decided that these were the traits that were important and why?
Do they reflect the real world writing that real world writers write?
Now, I'm a real writer.
I really am.
I'm real and I'm a writer.
Thus, I'm a real writer.
And as I'm writing this podcast, I'm looking back at Cullum's six plus one traits.
And you know what?
In my professional life, writing letters and memos and emails and articles and reports
and books, they just don't fit into this other worldly list of traits.
Voice and tone.
Now, in most professional writing, the goal is objective, academic or professional voice,
word choice.
What does that mean?
They simply were looking for things they could measure.
And they said, aha, we can measure these traits.
Let's get them up on a rubric and we can teach and we can numberize our kids.
We've been over rubricized.
The basis of a rubric is this.
You look at the product or performance you're trying to teach.
Then you define three to six traits of a good product or performance.
And then you describe three to five levels of each trait.
For example, given this trait, this is what a rating four would look like, a rating three,
a rating two, a rating one and zero.
And then you compare the product or performance to each trait and find the number level that
seems to match.
Simple as that.
Now, there's nothing wrong with using a rubric here and there, but we've become over rubricized.
We've got ourselves a bad case of rubritis and the stinking rubrics do little to move
us forward in our teaching and little to move students forward in their writing.
But since the stinking rubrics generate stinking numbers, the stinking rubrics continue to exist.
Now I want you to know you youngsters out there, there was educational life before rubrics
and it was a wonderful life.
We taught stuff and children learn stuff.
But then the number monkeys began to creep into education.
Nothing exists, they said, if it can't be quantifying.
So the number monkeys threw down their bananas and invented the rubric.
Rubrics enable them to put numbers to things and enumerate our students and quantify their
learning.
They could point their little monkey fingers and say, that's an 18 and that's a 16 and
18 is better than 16, so that one is better than that one.
And then they'd screech and shake their monkey arms above their head.
And if the average writing rubric scores in class A was 21.3 and the average writing rubric
score in class B was 18.7, the number monkeys would conclude that teacher A is doing a better
job of teaching than teacher B.
We've got the whole teacher B accountable, see?
We've got to get those test scores, those 18s up to 21s, everyone has to be a 21.
So what's wrong with the stinkin' rubrics?
Every form of assessment has strengths and limitations.
And here are some of the limitations with using rubrics.
Number one, they are too complex and cumbersome to be of any use.
It takes time to figure out what exactly constitutes each level of performance with each trait.
Then evaluating and deciding if it's one level or another, a two or three, a three or four,
takes additional time.
They take time to create and time to use and the cost reward here seems high in terms of
use and time.
Number two, they give the illusion of objectivity.
We think because it's described, we can just say, there it is, it's definitely a two or
definitely a three.
Just match the paper to the trait, right?
But often a paper will meet some of level three and some of level two and then do you
give it a 2.5?
Who defines the traits and the levels of each trait?
The rubric subjectivity is wrapped in an illusion of objectivity.
The subjectivity of the rubric is wrapped in the illusion of objectivity.
Number three, the focus is on the traits or writing product and not on the writing process.
Number four, a rubric often creates a distorted version of the performance they're trying
to capture.
In writing, rubrics never capture the full range of what writers do.
And number five, rubrics get students focusing on the micro instead of the macro.
It's assumed that by putting all the little micro rubric traits together that they'll
have an excellent macro product or performance.
This is called part to whole learning, but this is not how we learn complex things most
effectively.
Whole to part learning has been shown to be much more effective for learning complex skills
such as writing than part to whole.
All right, this has been the reading instruction show.
We've been looking at some assessment, some ways to document students growth and progress
as writers.
In the next podcast, we'll look at some better ways, some alternatives that make sense that
are more valid and reliable and help students growth as writers.
Imagine that.