Diplomas Count - June 5, 2014 - 23

EDUCATION WEEK JUNE 5, 2014
I
Diplomas Count > www.edweek.org/go/dc14
| 23
used measures of student engagement,
one place where they have
failed to take hold is in the accountability
systems proposed by states
under waivers from the federal No
Child Left Behind law. Nearly every
state has received a waiver, but only
a handful, including New Mexico
and South Dakota, have used the opportunity
for additional flexibility to
incorporate student surveys into accountability
measures.
During the American Youth Policy
Forum session, Elaine Allensworth,
the director of the University of Chicago
Consortium on Chicago School
Research, shed light on some of the
difficulties in publicizing and attaching
stakes to surveys. She drew on her
experiences with the Chicago 5 Essentials
Survey, a survey of students and
teachers developed by the consortium
to gauge school effectiveness. Although
used since 1997, the results were not
publicly released until 2009. Consortium
researchers worried the publicity
would lead principals to game the
system by encouraging higher ratings,
thus compromising the survey's validity.
However, Ms. Allensworth concluded
that the publicity instead led to
improvements in the schools. I
Students Rated on 'Grit'
With New Report Cards
KIPP'S CHARACTER
REPORT CARD
By Sarah D. Sparks
Most school report cards contain a component for behavior or character, but the Character
Lab at the University of Pennsylvania is looking for a more comprehensive approach
to measuring and providing feedback on student motivation.
In partnership with the Knowledge Is Power Program charter school network and
four other schools, the lab is completing its second year of field-testing a 24-item "character
growth" report card, measuring eight character domains: zest, grit, interpersonal
and academic self-control, optimism, gratitude, social intelligence, and curiosity.
"They're an end in themselves-it's great to be curious just to be curious, great
to be hopeful just to have hope-but they are also a means to getting these kids
to succeed," said David Levin, a KIPP co-founder.
Led by associate professor of psychology Angela L. Duckworth, the researchers
have found that grit and self-control can predict students' likelihood
of performing well academically, graduating from high school, and
going on to college.
To devise the report card, the researchers surveyed hundreds of secondary
school students about the behaviors associated with the eight domains,
then winnowed the list down to those clearly associated with school. Students
are rated by their teachers on a 1-5 scale.
The Character Lab researchers are using the report cards to track
both how closely the behaviors predict positive school outcomes and
whether students can change their behaviors over time. KIPP and the
other participating schools also see an opportunity to get students
and staff members talking more about boosting internal motivation.
"If schools talk about this stuff at all, very often they'll have a conduct
grade," Mr. Levin said. "When you talk about conduct, you're
really talking about compliance ... When people start to use character
and strengths from the growth card, they start to have a
broader definition of what this means."
It's too early to tell whether the ratings will increase high
school graduation rates, but schools have started to see improvements
in several areas associated with lower dropout
rates, including academic achievement and students' satiswoven
into Newton's Playground were
valid when measured against traditional
indicators.
Gauging Tenacity
Another video game example is
Tenacity, which was developed by the
Games+Learning+Society center at
the University of Wisconsin-Madison.
That game aims to help students regulate
their attention and awareness
by monitoring their own breathing.
Researchers say that skill is a key
building block of learning.
There are barriers to using games
to assess social and emotional skills:
For one, the games themselves are
expensive to build, at least if they're
meant to be engaging to the typical
teenager. There are also questions
about traditional psychometric
concerns: How do educators know if
developers' claims about what their
games can assess are in fact true?
And perhaps most significantly, said
Kurt Squire, an associate professor
at UW-Madison and the director of
Games+Learning+Society, questions
remain about the extent to which the
type of persistence shown in a video
game is relevant in the real world.
"Grit seems to me to be an interaction
between a person, a context, and
a particular task," Mr. Squire wrote
in an email. "There may be limits
in how well a game can measure or
foster a 'generalized grit'-although
there may also be a very clever or
brilliant game designer who can do
just that." I
Data Mining Provides
Clues on Engagement
By Holly Yettick
Ask students if they always try their hardest
in class, and they might say "yes" because
that's what their mother told them to do. Or
maybe if the students are a little older, they
say "never," because they are trying on a
carefree attitude to impress their friends.
Those are just some of the problems that
researchers encounter when they try to
measure engagement by surveying students
about themselves.
Enter educational data mining. Long used
in business settings, the practice is newer to
education. It involves using strings of data
collected during computerized instruction to
identify patterns. Once identified, the patterns
can then be used to improve the instruction
by, for instance, eliminating a scenario in
which students tend to lose interest and stop
playing an educational game. The strategy
can be used to track when and why students
might be off task and to draw broader conclusions
about different aspects of instruction,
including learner engagement.
For instance, for a study that appeared last
year in the peer-reviewed Journal of Educational
Data Mining, Jennifer Sabourin, then
a doctoral student at North Carolina State
University, and her colleagues analyzed data
from "Crystal Island," a computer game that
teaches middle school students about microbiology.
Based on that data, the researchers
were able to tell when students were playing
the game as intended and when they seemed
to be disengaged because they were indulging
in off-task behavior like climbing trees to reach
virtual rooftops. As expected, the researchers
found that students who were frequently off
task learned less.
But the game had another feature that led
to a more unexpected conclusion: Every seven
minutes, a box popped up onscreen asking students
to classify their emotional states. That
led researchers to discover that off-task behavior
seemed to help some students collect themselves
when they got frustrated. After playing
around a bit, their frustration lessened and
they returned to the game.
Tool for Program Evaluation
Besides its use in studying or assessing the
engagement of individual students, data mining
can also be a tool for ongoing program or
course evaluation. In a 2012 article published
in the peer-reviewed journal, Educational
Technology & Society, a research team led by
Jui-Long
Hung, an associate
professor of educational technology
at Boise State University in Idaho,
demonstrated how this might work. The team
combined student learning logs, demographic
data, and end-of-course evaluation surveys to
assess a supplemental, online learning program
for K-12 students. The data included
information from 7,539 students taking 883
courses in the program. The researchers created
an engagement index that assessed how
students interacted with each course that
included the frequency of logins and clicks
and the average number of discussion-board
entries. After combining these results with
other information, such as students' grades,
the researchers found that more-engaged students
got better grades. But in the entry-level
courses, engaged and disengaged students
alike had lower performance, leading researchers
to suggest that these courses might have
"structure, design, and/or support issues."
In their report, Mr. Hung and his co-authors
tout the benefits of combining data mining
with other information to evaluate engagement.
"The result is a much richer and deeper
analysis of student performance and teaching,
as well as of effective course design, than could
ever be accomplished with survey data or behavior
mining alone," they write. I
faction with the school.
"By helping kids see a broader definition of who they
are as people," Mr. Levin said, "we are hoping it helps
them become increasingly able to find their own sources
of motivation." I
At KIPP and other schools, teachers
are measuring student progress in
developing zest, grit, self-control,
and other skills and qualities.
ZEST
* Actively participates
* Shows enthusiasm
* Invigorates others
GRIT
* Finishes whatever he or she begins
* Tries very hard even after experiencing failure
* Works independently with focus
SELF-CONTROL-SCHOOL WORK
* Comes to class prepared
* Pays attention and resists distractions
* Remembers and follows directions
* Gets to work right away rather than procrastinating
SELF-CONTROL-INTERPERSONAL
* Remains calm even when criticized or otherwise provoked
* Allows others to speak without interruption
* Is polite to adults and peers
* Keeps temper in check
SOURCE: Knowledge Is Power Program

Diplomas Count - June 5, 2014

Table of Contents for the Digital Edition of Diplomas Count - June 5, 2014