The in class peer rating for my classes is base on 9 assessments and the opportunity to give a bonus.  They rate their teammates from 0-3 on each item.  Students are instructed not to grade themselves and not to give every student the same rating. I show them data that never in my history of TBL have all students on the same team done the same on the iRAT (one of the the items is "How well did the team member prepare for the iRAT?"  This comes out in the T-RAT discussions.  If a student says, "I just guessed" or "honestly, I didn't read that part", or "I remember in the text, it said..." you know how well they prepared).  So I challenge them that by abstraction it is doubtful that all the team members did exactly the same on the rest of the measures.  They also can give one team member a bonus (but they are not required to give anyone a bonus). 

The qualitative responses ("what is the team member doing well" and "what's one area where the team member might improve") should compare with the score ratings.  If the team member was rate low on their contribution to the T-RAT discussion, then the improvement should say something about speaking up more ore studying better for the RATs. 

So far this has been the most efficient way for me to gather the peer ratings and to get the students to do them in a reasonable amount of time as well as get meaningful feedback that I use to help the teams to improve.  I've only had one team that seemed to game the system by colluding on peer ratings.  I over heard them say one time "ok, who gets the bonus this time?"  Sure enough each time one member got the bonus the bonus from each of the other members and it rotated.  It was hard for me to get upset with them because they were a high performing team and made sure that everyone was participating and contributing even if a member was absent (I over heard that one member even went to get another when they had car trouble).


On Fri, May 2, 2014 at 4:39 AM, Bridges, Kristie <[log in to unmask]> wrote:

I think this would be possible if it were a single course but we've not reached consensus on how to incorporate peer evaluation grades longitudinally across multiple courses.  Even if we counted their evaluation scores toward a grade, I'm not sure it would address issues such as providing identical feedback to all team members.  We would need to grade the quality of their feedback as well as the scores they receive from peers.  Is that what you do?

From: Dee Fink <[log in to unmask]>
Sent: Thursday, May 1, 2014 5:36 PM
To: Bridges, Kristie
Cc: [log in to unmask]
Subject: Re: Poor quality peer evaluation
 
Kristie,

Why don't you count the peer evaluations toward the course grade?  Is there a reason for that?
     It seems like if you did, students would take them seriously.

My view is/was that one of my learning goals was for students to learn how to work effectively as part of a group.  
  • If that was a course learning goal, then I had to support that and assess that.  
  • Peer evaluation is the primary way of assessing that.  
  • And if I want students to take it seriously, I have to make it matter.  And making it part of the course grade is my way of "making it matter."
Dee Fink


On Thu, May 1, 2014 at 4:02 PM, Bridges, Kristie <[log in to unmask]> wrote:

Hello,

 

We were wondering how others handle the issue of students who do not take peer evaluation seriously.    We currently use iPeer and the Koles method but the results don’t count toward a grade.  With 2 classes of 200 students each and limited resources, it’s difficult to provide individual feedback to students regarding the quality of their comments.  Students who make inappropriate comments are reported for lack of professionalism but this happens very rarely.   Much more common issues include students copying and pasting identical feedback for multiple team members, using brief one-word responses or stating in the self-evaluation that their own performance is perfect.  They’ve been addressed as a class regarding this and are given examples of helpful/unhelpful responses but there currently are no individual or team consequences.  We’ve considered several options including grading the responses (probably not viable for us at this time), reporting all students whose responses don’t meet a minimum standard for lack of professionalism or making the entire group re-do the evaluation if any team member fails to take it seriously.  We’d love to hear how the rest of you handle this type of behavior.  Many thanks!

 

Regards,

Kristie Bridges

 

Kristie Grove Bridges, PhD

Associate Professor, Biomedical Sciences

WVSOM

400 N Lee St

Lewisburg, WV 24901

304-647-6223

 




--
***********************
L. Dee Fink         
234 Foreman Ave.
Norman, OK  73069
Phone/FAX:  405-364-6464
Email:  [log in to unmask]
Websites:      
        www.designlearning.org   [multiple resources on course design]
        www.deefinkandassociates.com   [offer workshops & online courses]
        www.finkconsulting.info  [Fink's consulting activities & publications]

**Former President of the POD Network in Higher Education (2004-2005)
**Author of: Creating Significant Learning Experiences (2003, Jossey-Bass)
**Senior Associate, Dee Fink & Associates Consulting Services





--

Herb Coleman, Ph.D
Dir. Instructional Computing and Technology
Adjunct Professor of Psychology
Austin Community College
Highland Business Center
5930 Middle Fiskville Rd.
Austin, TX 78752
[log in to unmask]
512-223-7746
********************************************************************************
Saruman believes it is only great power that can hold evil in check, but that is not what I have found. I found it is the small everyday deeds of ordinary folk that keep the darkness at bay. Small acts of kindness and love.
— Gandalf the Grey from the 2012 motion picture "The Hobbit"

*********************************************************************************