Education Week Issue 25 - Technology Counts 2014 - 15
CONTINUED FROM PAGE 13
evaluate language, grammar, mechanics, and style; detect plagiarism; and provide quantitative and
The computer programs assign
grades to writing samples, sometimes on a scale of 1 to 6, in a variety of areas, from word choice to
organization. The products give
feedback to help students improve
their writing. Others can grade
short answers for content. To save
time and money, the technology can
be used in various ways on formative exercises or summative tests.
The Educational Testing Service
first used its e-rater automatedscoring engine for a high-stakes
exam in 1999 for the Graduate
Management Admission Test,
or gmat, according to David Williamson, a senior research director
for assessment innovation for the
Princeton, N.J.-based company. It
also uses the technology in its Criterion Online Writing Evaluation
Service for grades 4-12.
Over the years, the capabilities
changed substantially, evolving
from simple rule-based coding to
more sophisticated software systems. And statistical techniques
from computational linguists, natural language processing, and machine learning have helped develop
better ways of identifying certain
patterns in writing.
But challenges remain in coming up with a universal definition
of good writing, and in training a
computer to understand nuances
such as "voice."
In time, with larger sets of data,
more experts can identify nuanced
aspects of writing and improve the
technology, said Mr. Williamson,
who is encouraged by the new era
of openness about the research.
"It's a hot topic," he said. "There
are a lot of researchers and academia and industry looking into
this, and that's a good thing."
In addition to using the technology to improve writing in the
classroom, West Virginia employs
automated software for its statewide annual reading language arts
assessments for grades 3-11. The
state has worked with CTB/McGraw-Hill to customize its product
and train the engine, using thousands of papers it has collected, to
score the students' writing based on
a specific prompt.
"We are confident the scoring is
very accurate," said Sandra Foster,
the lead coordinator of assessment
and accountability in the West
Virginia education office, who
acknowledged facing skepticism
initially from teachers. But many
were won over, she said, after a
comparability study showed that
the accuracy of a trained teacher
and the scoring engine performed
better than two trained teachers.
Training involved a few hours in
how to assess the writing rubric.
Plus, writing scores have gone up
since implementing the technology.
Automated essay scoring is also
used on the act Compass exams
for community college placement,
the new Pearson General Educational Development tests for a
high school equivalency diploma,
and other summative tests. But it
has not yet been embraced by the
College Board for the sat or the
rival act college-entrance exams.
The two consortia delivering the
new assessments under the Common Core State Standards are
reviewing machine-grading but
have not committed to it.
Jeffrey Nellhaus, the director
of policy, research, and design for
the Partnership for Assessment of
Readiness for College and Careers,
or parcc, wants to know if the technology will be a good fit with its assessment, and the consortium will
be conducting a study based on
writing from its first field test to see
how the scoring engine performs.
Likewise, Tony Alpert, the chief
operating officer for the Smarter
Balanced Assessment Consortium,
said his consortium will evaluate
the technology carefully.
Seton Hall University graduates enjoy successful
careers as Superintendents, Central Office and
Building Leaders in 30 states and 7 countries.
Our Executive Ed.D. degree offers a Proven Value, plus:
* Extensive networking and professional development opportunities
for aspiring, new and established superintendents
* Intensive 2-year program; weekends and summer sessions
* Engaging full-time faculty who mentor as well as teach
* National Reputation
* Cohort community supporting advancement during and after
* Dissertation research that forwards leadership vision and practice
Michael I. Cohen, Ed.D.
Manager in Assessment, Research & Evaluation
Denver Public Schools
DISCOVER AN INNOVATIVE ED.D. PROGRAM
THAT PUTS YOU ON THE PATH TO SUCCESS.
To learn more, visit us at
or call 1-800-313-9833.
400 South Orange Avenue * South Orange, NJ 07079
With his new company LightSide, in Pittsburgh, owner Elijah
Mayfield said his data-driven approach to automated writing assessment sets itself apart from
other products on the market.
"What we are trying to do is
build a system that instead of
correcting errors, finds the strongest and weakest sections of the
writing and where to improve," he
said. "It is acting more as a revisionist than a textbook."
The new software, which is available on an open-source platform, is
being piloted this spring in districts
in Pennsylvania and New York.
In higher education, edX has
just introduced automated software to grade open-response
questions for use by teachers and
professors through its free online
courses. "One of the challenges in
the past was that the code and
algorithms were not public. They
were seen as black magic," said
company President Anant Argawal, noting the technology is in
an experimental stage. "With edX,
we put the code into open source
where you can see how it is done
to help us improve it."
Still, critics of essay-grading
software, such as Les Perelman,
want academic researchers to
have broader access to vendors'
products to evaluate their merit.
Now retired, the former director
of the mit Writing Across the Curriculum program has studied some
of the devices and was able to get a
high score from one with an essay
"My main concern is that it
doesn't work," he said. While the
technology has some limited use
with grading short answers for
content, it relies too much on
counting words and reading an
essay requires a deeper level of
analysis best done by a human,
contended Mr. Perelman.
"The real danger of this is that it
can really dumb down education,"
he said. "It will make teachers teach
students to write long, meaningless
sentences and not care that much
about actual content." n
The Leadership and Learning Center®
TRANSFORM LEARNING WITH TECHNOLOGY
ARE YOU READY?
Our new TIPS: Technology Integration Planning Study™ provides
a high-level check-up of your district's "technology health," from
infrastructure, curriculum, instruction, and resources to progress
monitoring and beyond. TIPS provides key information to help you
effectively implement technology-based pedagogies, gain success
in a 1:1 environment, and enhance teaching and learning.
This powerful study will:
* Gather data to support system-wide, data-driven decisions
* Assess staff capabilities for integrating technology
* Identify infrastructure and device needs to
support student learning
Put TIPS to work in your district.
Call 866.399.6019 or visit
Houghton Mifflin Harcourt™ and TIPS: Technology Integration Planning
Study™ are trademarks of Houghton Mifflin Harcourt Publishing Company.
The Leadership and Learning Center® is a registered trademark of Advanced
Learning Centers, Inc. © Houghton Mifflin Harcourt Publishing Company.
All rights reserved. 02/14 MS96904
TECHNOLOGY COUNTS 2014
Education Week Issue 25 - Technology Counts 2014
Table of Contents for the Digital Edition of Education Week Issue 25 - Technology Counts 2014
Education Week Issue 25 - Technology Counts 2014 - 1
Education Week Issue 25 - Technology Counts 2014 - 2
Education Week Issue 25 - Technology Counts 2014 - Contents
Education Week Issue 25 - Technology Counts 2014 - 4
Education Week Issue 25 - Technology Counts 2014 - 5
Education Week Issue 25 - Technology Counts 2014 - 6
Education Week Issue 25 - Technology Counts 2014 - 7
Education Week Issue 25 - Technology Counts 2014 - 8
Education Week Issue 25 - Technology Counts 2014 - 9
Education Week Issue 25 - Technology Counts 2014 - 10
Education Week Issue 25 - Technology Counts 2014 - 11
Education Week Issue 25 - Technology Counts 2014 - 12
Education Week Issue 25 - Technology Counts 2014 - 13
Education Week Issue 25 - Technology Counts 2014 - 14
Education Week Issue 25 - Technology Counts 2014 - 15
Education Week Issue 25 - Technology Counts 2014 - 16
Education Week Issue 25 - Technology Counts 2014 - 17
Education Week Issue 25 - Technology Counts 2014 - 18
Education Week Issue 25 - Technology Counts 2014 - 19
Education Week Issue 25 - Technology Counts 2014 - 20
Education Week Issue 25 - Technology Counts 2014 - 21
Education Week Issue 25 - Technology Counts 2014 - 22
Education Week Issue 25 - Technology Counts 2014 - 23
Education Week Issue 25 - Technology Counts 2014 - 24
Education Week Issue 25 - Technology Counts 2014 - 25
Education Week Issue 25 - Technology Counts 2014 - 26
Education Week Issue 25 - Technology Counts 2014 - 27
Education Week Issue 25 - Technology Counts 2014 - 28
Education Week Issue 25 - Technology Counts 2014 - 29
Education Week Issue 25 - Technology Counts 2014 - 30
Education Week Issue 25 - Technology Counts 2014 - 31
Education Week Issue 25 - Technology Counts 2014 - 32
Education Week Issue 25 - Technology Counts 2014 - 33
Education Week Issue 25 - Technology Counts 2014 - 34
Education Week Issue 25 - Technology Counts 2014 - 35
Education Week Issue 25 - Technology Counts 2014 - 36
Education Week Issue 25 - Technology Counts 2014 - 37
Education Week Issue 25 - Technology Counts 2014 - 38
Education Week Issue 25 - Technology Counts 2014 - 39
Education Week Issue 25 - Technology Counts 2014 - 40