Education Week Issue 25 - Technology Counts 2014 - 13

By Caralee J.
Adams

Automating
Writing Evaluations
Teachers are
turning to essaygrading software
to critique student
writing, but critics
point to serious
flaws in the
technology

eff Pence knows the best way for his 7th grade English students to improve their writing is to do more
of it. But with 140 students, it would take him at
least two weeks to grade a batch of their essays.
So the Canton, Ga., middle school teacher uses
an online, automated essay-scoring program
that allows students to get feedback on their writing before
handing in their work.
"It doesn't tell them what to do, but it points out where issues may exist," said Mr. Pence, who says the Pearson WriteToLearn program engages the students almost like a game.
With the technology, he has been able to assign an essay a
week and individualize instruction efficiently. "I feel it's pretty
accurate," Mr. Pence said. "Is it perfect? No. But when I reach
that 67th essay, I'm not real accurate, either. As a team, we are
pretty good."
With the push for students to become better writers and
meet the new Common Core State Standards, teachers are
eager for new tools to help out. Pearson, which is based in
London and New York City, is one of several companies upgrading its technology in this space, also known as artificial
intelligence, ai, or machine-reading. New assessments to test
deeper learning and move beyond multiple-choice answers
are also fueling the demand for software to help automate the

scoring of open-ended questions.
Critics contend the software doesn't do much more
than count words and therefore can't replace human readers, so researchers are working hard to improve the software
algorithms and counter the naysayers.
While the technology has been developed primarily by companies in proprietary settings, there has been a new focus on
improving it through open-source platforms. New players in
the market, such as the startup venture LightSide and edX,
the nonprofit enterprise started by Harvard University and the
Massachusetts Institute of Technology, are openly sharing their
research. Last year, the William and Flora Hewlett Foundation sponsored an open-source competition to spur innovation
in automated writing assessments that attracted commercial
vendors and teams of scientists from around the world. (The
Hewlett Foundation supports coverage of "deeper learning" issues in Education Week.)
"We are seeing a lot of collaboration among competitors and individuals," said Michelle Barrett, the director of research systems
and analysis for ctb/McGraw-Hill, which produces the Writing
Roadmap for use in grades 3-12. "This unprecedented collaboration is encouraging a lot of discussion and transparency."
Mark D. Shermis, an education professor at the University of
Akron, in Ohio, who supervised the Hewlett contest, said the
meeting of top public and commercial researchers, along with
input from a variety of fields, could help boost performance of
the technology. The recommendation from the Hewlett trials
is that the automated software be used as a "second reader" to
monitor the human readers' performance or provide additional
information about writing, Mr. Shermis said.
"The technology can't do everything, and nobody is claiming it
can," he said. "But it is a technology that has a promising future."

Courtesy of CTB/McGraw Hill

'Hot Topic'
The first automated essay-scoring systems go back to the
early 1970s, but there wasn't much progress made until the
1990s with the advent of the Internet and the ability to store
data on hard-disk drives, Mr. Shermis said. More recently,
improvements have been made in the technology's ability to
PAGE 15 >

Writing Roadmap,
an automated essay
grader produced by
CTB/McGraw-Hill,
provides a numerical
score and qualitative
feedback on a piece
of writing from a 4th
grader. Experts say
there is much more
sharing of research
now about what
works best to build
automated essaygrading tools.
TECHNOLOGY COUNTS 2014

>> www.edweek.org/go/tc14

http://www.edweek.org/go/tc14

Education Week Issue 25 - Technology Counts 2014

Table of Contents for the Digital Edition of Education Week Issue 25 - Technology Counts 2014