State Has a Strange Way with Words
Tribune staff reporter
June 15, 2004
At 14, Ulises Gonzales is the kind of writer who makes his English teacher sigh with appreciation. His imagery is vivid, his style fluid and imaginative, his mechanics flawless.
Ulises' Burr Ridge teacher believes he will be a published writer someday. She also suspects he failed his 8th grade standardized writing test.
The person who grades that test will be a $10-an-hour temporary worker in a conference room in Tampa who spends about three minutes on each essay. Ulises is likely to lose points because, among other things, his ended without a summary and makes no explicit reference to the test question--a criteria on the checklist given to each grader.
All children in Illinois public schools, and many elsewhere in the nation, write an essay for a standardized test at some point in their education. Next year, similar writing samples will become part of the ACT and SAT college entrance exams.
That, in turn, is reshaping the way schools teach this essential skill--for the worse, critics say.
But Ulises' essay illuminates the difficulty of trying to evaluate the infinitely variable craft of writing in an objective and mechanical way.
Standardized writing tests measure certain benchmarks of basic competence--complete sentences, well-organized paragraphs, supporting details, correct pronouns.
The tests do not measure the grace and innovation found in the best writing.
They penalize pupils who struggle to finish in the prescribed 40 minutes, as Ulises did, without necessarily crediting his unconventional uses of dialogue and descriptive passages that have characters "yelling with a surprising ferocity" and "detention slips clenched in tight fists."
In the end, what these tests evaluate is so formulaic that in Indiana, a machine does the grading. In May, some 50,000 high school juniors there took an online essay test that was evaluated by computers using a form of artificial intelligence designed to mimic human readers.
Indiana launched the experiment to see if automated scoring could save time and money without sacrificing accuracy. If it works, other states could adopt the technology.
"We didn't build this system to evaluate the Hemingways and Shakespeares," said Richard Swartz, an executive director at Educational Testing Services, which designed Indiana's system and also uses computer programs to grade essays for the GMAT, the business graduate school entrance exam.
"The [artificial intelligence] is not going to be able to separate creative approaches from mundane approaches, but I would argue that doesn't happen with human readers either," Swartz said. "We're evaluating the kind of writing students are asked to produce, and 90 percent of that writing is pretty mundane."
The Illinois writing exam, administered since 1993, has had a profound impact on the way pupils here learn to write.
Children write more and at younger ages than ever before, educators say. But too often, their creativity is squelched by instruction that pushes formulaic writing because formulaic essays are enough to pass the state test.
"Writing is problematic because of the time it takes to teach and evaluate," said Ken Hunter, a Chicago high school principal. Hunter sits on a committee of educators that creates the state writing standards each year, scoring a sample of essays that are used to guide the temporary graders.
"The test is supposed to enhance instruction, but it really scares people," he said. "I think what happens sometimes is the only writing that gets done [in the classroom] is done for the test."
Schools making change
That is starting to change, with an increasing number of schools looking to improve their writing instruction by retraining teachers and beefing up assignments in classrooms across all grades.
College-educated readers at four sites around the nation, employed by a North Carolina company called Measurement Inc., are in the final stages of grading 1.2 million samples of Illinois writing for 2004. The process will cost the state $6.5 million.
Unlike other subjects, the writing test proved easy to pass, even for low-performing schools, but difficult to master, even at schools packed with the brightest pupils.
Only about 3 percent of elementary school pupils statewide exceed standards on the writing, compared to about 20 percent who exceed standards in reading. More than 900 Illinois schools didn't have a single 5th grader exceeding standards in writing, including schools in Hoffman Estates, Downers Grove and Bartlett.
By contrast, at Drake Elementary in Chicago--where nearly all the kids come from low-income homes--63 percent of 5th graders passed writing, though only 38 percent passed reading.
Yet in a handful of districts, student writers have been able to reach the highest level.
Among 3rd graders in River Forest District 90, 91 percent passed, with 30 percent exceeding standards. The district invested money to train teachers in writing instruction, and principals urge teachers to get their kids writing every day, across all subjects, from math to art.
Hunter said he's seen "extraordinary improvement" in the last five years, when the state changed its writing exam and sought to make its standards less formulaic.
Yet he acknowledged that many teachers still teach "the formula:" the five-paragraph, three-topic essay with lots of repetition and tired paragraph transitions that begin with "first," "second" and "in conclusion." Hunter said this kind of staid essay is enough to pass the test, but not enough to exceed standards, which is why so few pupils rise to the top level.
Writing isn't difficult for Ulises, but he cares far more about the beauty of his prose than passing the state test.
Ulises is a soft-spoken boy who sits in the middle of Elizabeth Wheeler's classroom and listens with a shy smile. Before he writes, he thinks--sometimes for 20 minutes before he will commit a single syllable to paper.
But when he starts, the words flow with a graceful ease.
"I can't write something just to get it down on paper. I try to make my writing fluid, to vary the structure and the words I choose," said Ulises, who wants to be a scientist. "I do care if I pass, but it's not a priority."
Scores reflect on teacher
Wheeler admires Ulises' refusal to compromise his writing quality, but she also knows she will be judged on the passing rate of her pupils. Her school is a diverse one in a neighborhood of million-dollar homes, where parents expect top-notch scores on state exams. She agonizes over how hard to push pupils like Ulises when she's teaching her class how to write for the ISAT, the state exam.
"The test works for the student who doesn't have any understanding about writing. It gives them a starting point," said Wheeler. "But really great writers don't write that way. They break the rules."
Burr Ridge Principal Debra LeBlanc said testing exacts a toll in all classrooms, but the writing exam troubles her most because she believes it is too arbitrary for a subject where mastery can take many forms.
She sees so much sameness in her pupils' writing that "even my thank-you notes read like little ISAT tests. `I really liked having lunch with you. Here are three reasons why.'"
She also finds it incomprehensible that the workers scoring her kids' essays can do a good job in two or three minutes--Burr Ridge teachers spent hours evaluating the essays of just one classroom of 5th graders.
Sometimes a bad prompt can doom even the best writers.
One year the test asked 11-year-olds to explain driving a car to someone from a foreign country. Another year it was about manatees, a sea mammal most Illinois children know nothing about.
Testing experts say the problem is not with Illinois' writing assessment or its scoring. If the kids are getting bogged down in formula, it's because teachers are not pushing them to write beyond the limits of the tests.
The test wasn't designed to prepare kids to write beautiful prose, said Samuel Krug, president of MetriTech, the company that develops the writing test for Illinois. Rather, it is intended to measure their ability to do the kind of straightforward writing that will be expected of them in the work world.
"Some might say the writing is pedestrian," Krug said. "But one thing the writing test does, even though it's only 40 minutes, is determine whether kids can put together ideas coherently, if not elegantly."
- - -
The vagaries of scoring
The two essays at this link were completed by 5th graders at Burr Ridge Middle School in 2003. The exam gave pupils 40 minutes to write about a teacher whom they admire. The typical writing test was scored in about three minutes by a temporary worker using a checklist and a sample of previously graded tests.
Reduced size Full size
Copyright © 2004, Chicago Tribune