The way I use the "forced ranking" is to weight the overall team scores. If a team does well, it is still possible for everyone on the team to do well, even with forced rankings. Similarly, it is possible for the highest rated teammate to earn less than an A if the team does poorly enough. If students are trying to sabotage their own teammates, they are sabotaging themselves as well. I don't find students competing to be the "best" teammate but rather competing to make the team the best. Some students are more natural leaders, some are more quiet, but with variety in the types of activities and ways students interact within their teams, everyone has the opportunity to find how they can best help their team and the students seem to recognize the different ways they each contribute. The focus in well functioning teams is to try to get the best team product, and evaluations tend to balance out (even when they aren't trying to game the system). Molly Espey, Professor Applied Economics and Statistics Clemson University Clemson, SC 296344 ________________________________ From: Team-Based Learning [[log in to unmask]] On Behalf Of Kaufman, Eric [[log in to unmask]] Sent: Thursday, March 24, 2011 11:33 AM To: [log in to unmask] Subject: Re: Forced ranking in the peer evaluation process? I teach a course on team leadership, and one of the concepts I highlight in the course is the detrimental effects of within-group competition. When group members are forced to compete for rewards (i.e. top spot within a peer evaluation ranking), it creates goal confusion. Rather than sharing all information and focusing on what is best for the team and the project(s), they have incentive to act in a way that makes them look the best (in comparison to their teammates). The competition with one another can result in sabotaging or downplaying the work of others. As a result, team performance is likely to decline. Accordingly, I cannot in good conscience implement a peer evaluation system that requires ranking of teammates. I have set a requirement that they cannot give everyone a 100%, but I do allow them to rate everyone in the “A” range. My peer evaluation system includes a combination of a matrix, a percentage rating, and qualitative feedback. Of those, I believe the qualitative feedback is most useful. For your reference, I have attached a sample of the report students receive. Eric K. Kaufman Assistant Professor in Leadership Agricultural and Extension Education Virginia Tech 266 Litton Reaves (0343) Blacksburg VA 24061 Phone: 540-231-6258 Website: www.aee.vt.edu<https://xmail.clemson.edu/owa/UrlBlockedError.aspx> From: Robin L Hills/FS/VCU [mailto:[log in to unmask]] Sent: Wednesday, March 23, 2011 2:24 PM Subject: Re: Forced ranking in the peer evaluation process? In the 5 years I have used TBL in my graduate nurse practitioner gyn course, I have experienced gaming the system and rebellion against forced ranking so extensively that I decided to omit peer evaluation entirely this semester. With the exception of one team 3 years ago (with one individual being the primary cause of the dissonance), the teamwork has been very high functioning especially as the refinement of the course has occurred. I may consider incorporating qualitative peer evaluation back into the course in the future. Robin L. Hills, MS, RN, WHNP-BC, CNE Clinical Assistant Professor VCU School of Nursing Department of Family and Community Health Nursing Box 980567, Richmond, VA 23298 (804) 828-5578 From: "Levine, Ruth" <[log in to unmask]> To: [log in to unmask] Date: 03/23/2011 12:30 PM Subject: Re: Forced ranking in the peer evaluation process? Sent by: Team-Based Learning <[log in to unmask]> ________________________________ Hi Maria: In my experience with medical students, when I forced them to discriminate in their peer evaluations they were very unhappy. In consultation with many other schools I have heard similar stories, and many people actually abandoned their discriminative peer evaluation system because of rebellion on the part of the students. The more “high functioning” the team, the more likely they were to dislike the requirement to score one teammate higher than another, and the more likely to “game” the system so that each teammate could have a similar score. I spent considerable time discussing this dilemma with Larry Michaelsen, who created the “forced discrimination” peer evaluation system, and he told me that “gaming the system” was not necessarily a bad thing, since it was an indication of cohesion on the part of the team. He and I have debated this point—I believe that in health science students “gaming” can create an unprofessional environment and set up adversarial relationship between the instructor and the class. Larry and I agreed to disagree about the whole point of the merits of forced discrimination. I have chosen (and recommend) to instructors who I counsel about peer evaluation to use a system like the one devised by Dee Fink in which students can choose to give everyone the same score or not (in his system, students divide 100 points among their teammates—in a team of 6, a student can give all his peers 20 points or give some more or some less, but not everyone can get high scores so there is no problem with grade inflation). I use the sum of the scores and multiply them by the group grade for an “adjusted” group grade and give the qualitative feedback back to the students. The advantage of the Fink method (in which there is both quantitative and qualitative peer evaluation) is that students can CHOOSE to discriminate or not in their allocation of points. In my experience students do so about 20% of the time. The meaningful evaluation comes mostly from the qualititative feedback.. This evaluation is much more popular with the students but also provides evaluation which can be used for both grading and useful formative and summative feedback. Ruth Levine University of Texas Medical Branch, Galveston From: Team-Based Learning [mailto:[log in to unmask]] On Behalf Of Summa, Maria Sent: Wednesday, March 23, 2011 10:19 AM To: [log in to unmask] Subject: Forced ranking in the peer evaluation process? Dear TBL Community, Our School of Pharmacy is launching a 3-calendar-year (33-month) post-baccalaureate Doctor of Pharmacy program this coming Fall. The curriculum follows a modified-block model with a heavy emphasis on active-learning instructional strategies. Beginning with our inaugural class, we will pilot a process for peer/learner evaluation at a curricular level (rather than at the individual course level). Teams of 6 students will complete peer evaluations multiple times each academic term. A “capstone” course at the end of each calendar year will “house” the grade for peer evaluations, although learners will receive timely feedback on their progress after each evaluation submission. The peer evaluations will represent a significant portion of the capstone course final grade. The peer evaluation tool we have developed is based on a forced-ranking system, whereby learners would use a scale to rank each member of their learning team on a set of abilities. The abilities were developed from our School’s ability-based outcomes and literature on effective teams. In addition to ranking their peers, learners will be required to justify each ranking through brief written comments that clearly identify areas of high performance and those requiring further growth/development. Comments will be evaluated by faculty and will constitute a portion of the overall peer evaluation grade. While our faculty has approved the peer evaluation process and the associated tool, there is some degree of hesitation with the use of a forced ranking system, despite our requirement for peer score justification and written comments. I would appreciate hearing all opinions on the use of forced ranking in peer evaluation from this group, particularly from those of you who employ this model. Thank you, Maria Summa Maria A. Summa, PharmD, BCPS Associate Professor, Pharmacy Practice and Administration Saint Joseph College School of Pharmacy 229 Trumbull Street, Hartford, CT 06103 Phone: 860.231.5885 Fax: 860.231.5759 Email: [log in to unmask]<mailto:[log in to unmask]> Click here<http://www.sjc.edu/about/social_media/parisi.html> to read our new Blog!