In the 5 years I have used TBL in my graduate
nurse practitioner gyn course, I have experienced gaming the system and
rebellion against forced ranking so extensively that I decided to omit
peer evaluation entirely this semester. With the exception of one
team 3 years ago (with one individual being the primary cause of the dissonance),
the teamwork has been very high functioning especially as the refinement
of the course has occurred. I may consider incorporating qualitative
peer evaluation back into the course in the future.
Robin L. Hills, MS, RN, WHNP-BC, CNE
Clinical Assistant Professor
VCU School of Nursing
Department of Family and Community Health Nursing
Box 980567, Richmond, VA 23298
(804) 828-5578
Hi Maria:
In my experience with medical
students, when I forced them to discriminate in their peer evaluations
they were very unhappy. In consultation with many other schools I have
heard similar stories, and many people actually abandoned their discriminative
peer evaluation system because of rebellion on the part of the students.
The more “high functioning” the team, the more likely they were
to dislike the requirement to score one teammate higher than another, and
the more likely to “game” the system so that each teammate could have
a similar score.
I spent considerable time
discussing this dilemma with Larry Michaelsen, who created the “forced
discrimination” peer evaluation system, and he told me that “gaming the
system” was not necessarily a bad thing, since it was an indication of
cohesion on the part of the team. He and I have debated this point—I believe
that in health science students “gaming” can create an unprofessional
environment and set up adversarial relationship between the instructor
and the class.
Larry and I agreed to disagree
about the whole point of the merits of forced discrimination. I have chosen
(and recommend) to instructors who I counsel about peer evaluation to use
a system like the one devised by Dee Fink in which students can choose
to give everyone the same score or not (in his system, students divide
100 points among their teammates—in a team of 6, a student can give all
his peers 20 points or give some more or some less, but not everyone can
get high scores so there is no problem with grade inflation). I use the
sum of the scores and multiply them by the group grade for an “adjusted”
group grade and give the qualitative feedback back to the students.
The advantage of the Fink
method (in which there is both quantitative and qualitative peer evaluation)
is that students can CHOOSE to discriminate or not in their allocation
of points. In my experience students do so about 20% of the time. The meaningful
evaluation comes mostly from the qualititative feedback.. This evaluation
is much more popular with the students but also provides evaluation which
can be used for both grading and useful formative and summative feedback.
Ruth Levine
University of Texas Medical
Branch, Galveston
From: Team-Based Learning [mailto:[log in to unmask]]
On Behalf Of Summa, Maria
Sent: Wednesday, March 23, 2011 10:19 AM
To:[log in to unmask]
Subject: Forced ranking in the peer evaluation process?
Dear TBL Community,
Our School of Pharmacy
is launching a 3-calendar-year (33-month) post-baccalaureate Doctor of
Pharmacy program this coming Fall. The curriculum follows a modified-block
model with a heavy emphasis on active-learning instructional strategies.
Beginning with our
inaugural class, we will pilot a process for peer/learner evaluation at
a curricular level (rather than at the individual course level).
Teams of 6 students will complete peer evaluations multiple times
each academic term. A “capstone” course at the end of each calendar
year will “house” the grade for peer evaluations, although learners will
receive timely feedback on their progress after each evaluation submission.
The peer evaluations will represent a significant portion of the
capstone course final grade.
The peer evaluation
tool we have developed is based on a forced-ranking system, whereby
learners would use a scale to rank each member of their learning team on
a set of abilities. The abilities were developed from our School’s
ability-based outcomes and literature on effective teams. In addition
to ranking their peers, learners will be required to justify each ranking
through brief written comments that clearly identify areas of high performance
and those requiring further growth/development. Comments will be
evaluated by faculty and will constitute a portion of the overall peer
evaluation grade.
While our faculty has
approved the peer evaluation process and the associated tool, there is
some degree of hesitation with the use of a forced ranking system, despite
our requirement for peer score justification and written comments. I
would appreciate hearing all opinions on the use of forced ranking in peer
evaluation from this group, particularly from those of you who employ this
model.
Thank you,
Maria Summa
Maria A. Summa, PharmD,
BCPS
Associate Professor,
Pharmacy Practice and Administration
Saint Joseph College
School of Pharmacy
229 Trumbull Street,
Hartford, CT 06103