Education Week Issue 25 - Technology Counts 2014 - 22

Testing Students
In Simulated Worlds
New digital simulations are designed
to assess how students solve problems,
communicate, and work with others

ultiple-choice exams measuring the
breadth of students' content knowledge
have hardly disappeared.
But the creators of a leading international exam, the U.S. government
agency behind "the nation's report
card," and the makers of new classroom learning tools are all
turning their attention to something new: digital simulations
aiming to measure how students solve problems, communicate, and work with others.
Examples include simulations that ask students to collaborate with a computer avatar to keep a tankful of tropical
fish alive, help a virtual village fix its broken water pump,
develop a model of an atom, and help an online city reduce
pollution while growing its economy.
"In the real world, there is often more than one route to the
right answer, or more than one right answer," said Michael
Barber, the chief education strategist for education publishing giant Pearson and a strong backer of the new assessment
trend. "With simulations, you can have complex problems and
allow different students to find different ways through."
The use of simulations for assessment isn't new: For decades,
for example, airline companies and the military have used
simulators to train and screen prospective pilots. Many educators have also had a long-standing interest in more traditional
forms of low-tech simulated performance assessments, in which

By Benjamin
Herold

students are required to complete a sophisticated task, such as
designing and conducting a science experiment.
But questions of cost, reliability, validity, and scalability
have been stumbling blocks in the past to widespread adoption of both simulations and performance assessments, said
Eva L. Baker, a research professor in the graduate school of
education and information studies at the University of California, Los Angeles. She said those same challenges remain
with today's emerging technologies.
"We all want to avoid the gee-whiz factor," said Ms. Baker,
the director of the National Center for Research on Evaluation, Standards, & Student Testing, or cresst. "We have to
make sure there is adequate evidence these things work and
are not simply the next big thing."

Real-Time Feedback
Proponents maintain that digital simulations, which experts in the field say generally involve less narrative, design,
and purely fun elements than digital games, can increase
student engagement and provide real-time feedback to educators. They also believe the torrents of data that simulations
generate can be mined for evidence of deeper learning-not
just whether a student selects the correct canned response,
but whether he or she asks the right questions, uncovers
all the necessary information to make a smart choice, and
shares that information effectively with a partner.
"Knowing the right answer is just half of what we should be
looking for," said Peggy G. Carr, the associate commissioner
PAGE 24 >

National Center for Education Statistics
/// TESTING STUDENTS IN STEM

Digital simulations
are making their way
into a wide variety of
assessments, used for
an equally wide variety
of purposes. Some of
the prominent players
in this emerging field
include:

EDUCATION WEEK

How do you measure students' ability to understand, use, and improve technology?
With "task-based scenarios" online, believe officials of the nces,
which administers the National Assessment for Educational Progress.
In one such task on the agency's new technology, engineering, and
literacy exam, students are asked to assume the role of an engineer who must figure
out why a village's water well has stopped pumping water. The item gauges test-takers'
troubleshooting skills: Do they access and use the most relevant information? Can they
quickly diagnose the problem?
"It's intended to provide an assessment that's more authentic for students," said
Peggy G. Carr, the associate commissioner of the assessments division at the nces, a
branch of the U.S. Department of Education.
The exam is currently going through its second large field test, although no live administration has yet been scheduled. The nces is investing in the new assessment, Ms. Carr
said, in part because science, technology, engineering, and mathematics skills often cut
across academic content areas and are untouched by subject-specific tests.
The reliability and validity of digital simulations remains a concern, but she said an earlier
field test of the new exam helped: The nces feels confident, for example, that it has identified the most efficient way for test-takers to troubleshoot the broken pump. Test-takers who
pursue that path will get the highest score. "There has to be a way to score these tasks such
that you can put them on a scale with higher or lower levels of proficiency," Ms. Carr said. n

>> MARCH 13, 2014

Test-takers must determine
the best way to fix a broken
well in a simulation for the new
technology, engineering, and
literacy exam by the NCES.

Education Week Issue 25 - Technology Counts 2014

Table of Contents for the Digital Edition of Education Week Issue 25 - Technology Counts 2014