Hi all Interesting conversation....maybe someone should do a panel on the use of peer evaluation in TBL at the upcoming conference? http://2007.teambased.org We use an online tool to do peer evaluations. It is a tool that we have developed in house (see attach screenshot).....students really can't game the system since giving everyone 100% only makes them average in the group....student may be able collude in sub-teams to game the system... We give a formative version after 4 weeks of class and then a summative at the end of the course....we release the formative evals back to individuals with names automatically stripped off, randomized and written comments moderated by instructor I do like Michaels "appreciations" and "requests" ....I might change our commenting to include this More info of the tool at http://ipeer.apsc.ubc.ca/ipeer_site/ open source and free to download Jim _________________________________________________ Jim Sibley Manager Centre for Instructional Support Faculty of Applied Science University of British Columbia 2208-6250 Applied Science Lane Vancouver, BC Canada V6T 1Z4 Phone 604.822.9241 Fax 604.822.7006 Email [log in to unmask] Web www.learning.apsc.ubc.ca Blog: Adventures in Instructional Support - http://ipeer.apsc.ubc.ca/wordpress/ Portfolio: Jim's Wiki Portfolio ________________________________________________ (c) Copyright 2006, Jim Sibley, All rights reserved The information contained in this e-mail message and any attachments (collectively "message") is intended only for the personal and confidential use of the recipient (or recipients) named above. If the reader of this message is not the intended recipient, you are hereby notified that you have received this message in error and that any review, use, distribution, or copying of this message is strictly prohibited. If you have received this message in error, please notify the sender immediately by e-mail, and delete the original message. ___________________________________________________ -----Original Message----- From: Team Learning Discussion List [mailto:[log in to unmask]] On Behalf Of Michael Gholam Sent: Wednesday, January 24, 2007 3:43 PM To: [log in to unmask] Subject: FW: Peer Evaluations ("Hold the Peevishness!") Hi all, This is my response to Michael Sweet. Stay well Michael Gholam Lebanon -----Original Message----- From: Michael Gholam [mailto:[log in to unmask]] Sent: Thursday, January 25, 2007 1:33 AM To: 'Sweet, Michael S' Subject: RE: Peer Evaluations ("Hold the Peevishness!") Hi Michael, I like your way of dodging the "Peevishness and gaming the system", but the fact remains that all will decide to give each other "positive" "appreciations and requests" knowing that would lead to your bump. If students want to game the system, they will. It is a matter of personal integrity. Besides, I would forfeit upgrading or downgrading the group performance grade if I informally find that the points given to one member do not "correlate" with the progress as seen by their individual RAT's. If I see equal grades allocated to all and collective improvement in individual scores, I would consider giving the exact group grade to all. I am not saying that correlating iRat's with points allocated to each individual is the perfect solution, but it can be an indicator. I also would observe from time to time how groups are wprking at application tasks or a culminating group task, and also conduct an informal corroborative investigation. Michael Gholam Lebanon -----Original Message----- From: Team Learning Discussion List [mailto:[log in to unmask]] On Behalf Of Sweet, Michael S Sent: Thursday, January 25, 2007 12:54 AM To: [log in to unmask] Subject: Peer Evaluations ("Hold the Peevishness!") We don't have students give each other points anymore, just written feedback. BUT we have figured out a way to keep peer evaluations at the forefront of their minds while avoiding the peevishness and "gaming the system" that can arise when you have them distribute points to each other. Instead of points, we have them provide anonymous "appreciations and requests" twice during the semester. (Each student provides at least one "appreciation" and one "request" for each of their team-mates.) These are then fed-back to the students anonymously, as most people do it. BUT we introduce the peer evaluations at the beginning of the semester by saying: "At the end of the semester, I often have students come to my office and say 'I am so close to the next grade up--is there anything I can do for extra credit to bump me up?' I tell them 'No, but I will look at your peer evaluation feedback. If the appreciations and requests are all positive or started out rough and improved, then I will give you that bump. If not, then, nope--sorry, nothing to be done." The beauty of this is that none of the students know whether they will be candidates for a "bump" until the very end of the term, so they are on good behavior for the entirety of the course while still getting the important group process information in the form of "appreciations" and "requests." -M ________________________________ From: Team Learning Discussion List on behalf of Kubitz, Karla Sent: Wed 1/24/2007 1:48 PM To: [log in to unmask] Subject: Re: Forced Ranking in Peer Evaluation For what it's worth, I've found it helpful to put the following statement on the back of the peer evaluation form (Michaelsen's version) and ask them to sign it. I hereby certify that I have provided an honest assessment of the contribution of my teammates to our team's productivity. My team maintenance scores are not based on any 'in or out-of-class' agreements among my teammates and myself to distribute points in a particular way (i.e., a way that does not consider the quality or quantity of individual efforts). Karla Kubitz ________________________________ From: Team Learning Discussion List [mailto:[log in to unmask]] On Behalf Of Maureen Jonason Sent: Wednesday, January 24, 2007 12:41 PM To: [log in to unmask] Subject: Re: Forced Ranking in Peer Evaluation I usually do not tell students about this ahead of time. They read it to themselves on the last day of class and figure it out and quietly do what I ask. In the past, I made the mistake of giving the team evals out to fill out outside of class. One team simply ignored me and gave everyone equal scores. Another team cleverly figured out a way to give equal points by each agreeing to make one of the others the low-pointer and, so they all ended up with equal scores anyway! I had to give them credit on that one. I am not bothered by rule-breaking, so I accepted their decisions/choices. AS others have pointed out, if the teams bond and everyone does truly contribute equally in their view, then the lesson has been learned. usually, when there is clearly someone who does less work, they are more than happy to give points accordingly. ________________________________ From: Team Learning Discussion List [mailto:[log in to unmask]] On Behalf Of Don McCormick Sent: Wednesday, January 24, 2007 11:32 AM To: [log in to unmask] Subject: Forced Ranking in Peer Evaluation Hi TBLers I teach management and I used the peer evaluation form that requires students to rate their peers and give at least one a "9" (which is one below average) and one "11" (which is one above average). When I announced it last night, the class exploded in a revolt, objecting that it wasn't fair because "in my group everyone did an equally good job of contributing," they couldn't figure out a basis for rating others one way or another, etc. I know the form says "If you give everyone pretty much the same score you will be hurting those who did the most and helping those who did the least, " but I also am sympathetic to the students' point of view. I understand the reason given above for forcing some minimal ranking and I also realize that students are often terrified of giving negative feedback to other students. I want to help them learn to overcome this fear because they need to learn how to give negative feedback in the workplace. If they don't learn to do this, they will truly suck as managers. But it isn't clear that in the cases where they genuinely feel each person in their group has contributed equally how forced ranking will help them learn this. Is there more to the requirement of forced ranking that I am missing? From your point of view, what is the learning objective that this helps students to meet? - Don --- Don McCormick Department of Management College of Business and Economics California State University Northridge https://www.csun.edu/~dmccormick/Don%20McCormick/Home.html