We've struggled with some of the anxiety about peer evaluation (although sometimes our problems were more related to giving everyone high marks even though they had concerns/problems with team mates, thus not dealing with it). Although, we did have one situation where the "team" had agreed they would equalize the score, and then one team member reneged on the agreement - which really angered the team against that student for a while. They even had a blow up in class during an application, and the whole team stormed out. But they wanted to work it out - and they did. It turned into a positive impact for their work together later on.
So, what we really thought was at the heart of the problem for our peer evaluation system was not clear understanding of why it is important and how to give constructive feedback - it is not just about a number. So we actually give them a session on why this is critical to them now and in the future, and give them some tools on how to give constructive feedback. Comments are not useful if they cannot offer suggestions on how there can be improvements/changes to behavior that will benefit the group. Thus, if the feedback is behavior specific, identify consequences, offer suggestions, and explain how that will improve the group dynamics.
We also give a trial run - so that the first time around people get informed, but grades are not impacted.
Lastly, which we think is a beneficial addition, we realized that we need to give them skills in processing feedback. They are going to spend their lives getting feedback (good and bad, deserved and not). So we ask the students to reflect on the feedback they have gotten. What is it telling them and what might they do about it.
It is tough for all to have peer eval when it is not stellar - but what can be learned from it is an important lesson.
********************************************************
WE HAVE CHANGED OUR EMAIL ADDRESS - PLEASE CHANGE TO [log in to unmask]
Sandy COOK, PhD | Associate Dean, Curriculum Development | Medical Education, Research, and Evaluation (MERE) | Duke-NUS Graduate Medical School Singapore | Khoo Teck Puat Building | 8 College Road
Singapore |169857 | W: (65) 6516 8722| F: (65) 6227 2698 | email: [log in to unmask] | web: http://www.duke-nus.edu.sg;
Administrative Executive: Belinda Yeo | [log in to unmask] | 6516-8511
Important: This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Thank you.
-----Original Message-----
From: Team-Based Learning [mailto:[log in to unmask]] On Behalf Of Sweet, Michael S
Sent: Saturday, October 24, 2009 12:11 AM
To: [log in to unmask]
Subject: Re: problem with peer review
(1) I agree with Ruth's suggestions about making the first iteration of peer review a formative round for practice and without points attached.
(2) The addition of IF-ATs makes the re-evaluation of one's team-mates "real time" (not when everything is over and done with for that RAP) and therefore more immediate, incremental and less dramatic. Definitely give those a try.
I have found peer review processes to be the most context-sensitive aspect of TBL. One teacher I worked with taught at a virtually open-enrollment institution and students *hated* it but when he taught the same class at a smaller, more competitive institution, students *loved* it.
In my mind, peer review is the aspect of TBL that gets most directly at the heart of student-student relationships, and in our culture we are not good at talking honestly about relationships. We tend to try to manage things so they'll look good, rather than address things so we all learn about ourselves. I'd bet there's at least a few dozen dissertations to be written on the psychodynamics of peer review.
I applaud the rigor with which you have implemented TBL and encourage you to keep communicating with us about this issue. You are not alone!
-M
-----Original Message-----
From: Team-Based Learning [mailto:[log in to unmask]] On Behalf Of Levine, Ruth
Sent: Friday, October 23, 2009 10:43 AM
To: [log in to unmask]
Subject: Re: problem with peer review
Lisa:
Your situation really raises two issues-
1)How to deal with the immediate problem of the one student inhibiting the other student from talking, and
2)How to prevent a problem like this in the future.
I find the immediate problem a little more vexing, because one hates to interfere with team process by talking with either the critical student or the student with hurt feelings. Getting the whole team together to talk about team process is a thought, but again it is outside of normal class processes--unless you institute it now for the whole class- and that is an option. It is not a guarantee. But it might do something.
For the future:
The reason I believe that Larry mentions using the if/at is because the immediate feedback will prompt the team to pull the student with hurt feelings to participate--they will want him to share his knowledge and will not allow him to be quiet. Larry- is that what you are thinking?
I would suggest for future iterations to cut back on graded peer feedback early in the semester and just have formative evaluation--followed by small group time to process the feedback. I find that my students will be very open and even more critical with their feedback (I use the fink method) with the formative feedback but since its not graded it doesn't tend to inhibit group cohesion.
Ruth
Ruth Levine
Professor of Psychiatry
Department of Psychiatry and Behavioral Sciences
The University of Texas Medical Branch, Galveston
-----Original Message-----
From: Team-Based Learning [mailto:[log in to unmask]] On Behalf Of Lisa Hager
Sent: Friday, October 23, 2009 10:15 AM
To: [log in to unmask]
Subject: Re: problem with peer review
In response to people's questions--
I am not using the IF-AT answer sheets but I do use Scantrons and grade them
in class. I am also using activities that follow the 4 Ss. I have 5 teams in
my class (2 teams of 5 and 3 teams of 6). I developed the teams based on
Larry's suggestions. I'm using the Team-Based Learning book, plus the
website, plus materials I originally got from the Case Studies Workshop in
Buffalo. For the peer review eval I am using Kole's form. Students set the
grading scale and Team Maintenance is 15% of the grade. Quantitative scores
are 60% and Qualitative scores are 40%. The students submit the quantitative
evals via Excel with no identifying info and they submit the qual evals via
Word and I compile them all into individual files for each student. I can't
do much more to control for anonymity. They do the peer evals outside of
class.
Overall the few points the student is losing won't make much of a dent in
his grade (or the other students who are also being graded harshly by this
person) but I'm more concerned about his unwillingness to participate. He is
one of the strongest students in the class. He loves the TBL and says he is
learning more in this class than he ever has in other classes. I want him to
maintain that enthusiasm.
I haven't had them discuss the feedback with each other. That might be a
good idea.
Lisa
----- Original Message -----
From: "Larry Michaelsen" <[log in to unmask]>
To: <[log in to unmask]>; <[log in to unmask]>
Sent: Friday, October 23, 2009 8:19 AM
Subject: Re: problem with peer review
Lisa,
I don't know that I've ever had the problem you describe. I suspect that
you may have missed an important practice that has the effect of preventing
this type of situation. I would add three other questions to those posed by
Ruth.
1) Are you using the IF-AT answer sheets for your tRATs?
2) What kinds of team application activities have you had them do (i.e., Do
they follow the 4 S's?
3) What kind of peer evaluation instrument are you using... specifically
does it have any sort of "forced-choice" (vs. a rating scale on which peers
have the option of weighting everyone high--or low)?
Larry
--
Larry K. Michaelsen
Professor of Management
University of Central Missouri
Dockery 400G
Warrensburg, MO 64093
[log in to unmask] <---PLEASE NOTE NEW E-MAIL ADDRESS !!!
660/543-4124 voice
660/543-8465 fax
>>> Lisa Hager <[log in to unmask]> 10/22/09 6:49 PM >>>
I am new to using TBL and I have a problem I need some help with. I have one
team that has an individual who is assigning very low numbers to 4 out of 5
of the team members he/she is evaluating. One member of the team has not
participated during the last two classes b/c he was rated as not being open
to others' ideas and got a qualatitative comment that he is too vocal and
doesn't give others a chance. This student was also rated as not being
prepared. I know that the student is prepared and that he is vocal but I
know that he does work well with the team. The rater has also rated 3 other
members of the team with low scores. Do I step in and talk to the one
student? Do I let the students work it out? Do I advise the student in how
to approach his team about the problem? I initially decided to let it work
itself out but then the one student came to ask my advice and to tell me his
is uncomfortable with participating in his group now. They are ready to
evalute each other for the 3rd time and are half way through the semester.
Any advice would be greatly appreciated.
Lisa
Lisa D. Hager, Ph.D., Chair
Social Sciences Division
Department of Psychology
Spring Hill College
4000 Dauphin St.
Mobile, AL 36608
(251) 380-3055
|
