Jorden,
Jim Sibley has provided excellent advice about using a multiplier to prevent students from "gaming" the peer evaluation process. Let me add another suggestion or two. I apologize in advance for the length of my post. .
One strategy (of which I am not particularly fond) is to treat a composite mean helping behavior score of 10 as a "C" letter grade (75% of the helping behavior points).
C is average. Only those individuals scoring above 13 receive an "A" (100% of the helping behavior points) for their helping behavior. Individuals with a composite mean helping behavior score between 11-12 will receive a "B" (85% of the helping behavior points) Individuals scoring between 7 and 8 will receive a D (60% of the helping behavior points). This strategy prevents grade inflation and discourages student "gaming." Unfortunately, it may also serve to weaken positive student attitudes about working in teams.
Many faculty who use TBL for the first time implement a summative peer evaluation process (what I will refer to as team "helping behavior") but fail to provide opportunities for the team members to provide corrective formative peer evaluations to allow for behavior modification over the course of the semester that will improve individual and team learning. As a result, many students are not completely honest in their responses (they attempt to game the process). Therefore, I have added a paragraph to my TBL syllabus which serves to reduce this tendency, and to eliminate the possibility of grade inflation.
FROM MY SYLLABUS
HELPING BEHAVIOR
In addition to Peer Evaluations done periodically throughout the semester, each individual will rate the helpfulness of all of the other team members prior to the final exam. Individual helping behavior scores will be calculated using the mean of two scores: a) the average of the points you receive from the members of your group; and b) the participation score you receive from me. Assuming arbitrarily that: 1) helping behavior is worth 10 points, and 2) that there are five members in a team, an example of this grading procedure is as follows: Each individual must assign a total of 40 points to the other four members in your group without giving the same score to every member. Not all group members contribute equally. Some members are more motivated or more communicative than others. For this reason raters must differentiate their ratings. This means that each rater would have to give at least one score of 11 or higher (with a maximum of 15) and at least one score of 9 or lower. The instructor will assign a participation grade for each of the members and this score will be added to the average score from the group members to derive a composite helping behavior score. Helping behavior scores produce differences in grades only within a team. As a result, group members cannot help everyone in their group to earn an A by giving them a high peer score. The only way for everyone in a group to earn an A is by doing an outstanding job on the individual and team exams and projects. Although rare, should a problem arise with anyone's group participation, we will address the problem discreetly, but directly.
In include another paragraph on the Helping Behavior Form (when students provide their final "summative" helping behavior grade to each of their team members):
Teams work most effectively when every person on the team is accountable for their own behavior. Throughout the semester you have frequently given and received feedback as a member of your team using a formative feedback process created by your team. Now it is time to assign a summative helping behavior score to each group member to indicate the extent to which individuals contributed to your team success.
As you assign your helping behavior scores below they should be representative of how HELPFUL each member of your group has been throughout the entire year. Helping behavior scores do not come from one isolated instance but rather as a cumulative measure of individual accountability to the group. Please be HONEST in your evaluation of your other group members. I understand that some students may be hesitant to be truthful about the performance of others, but individual accountability is paramount to successful teams.
If you do not submit BOTH written and numerical peer feedback for EACH member of your team you will receive a zero for your helping behavior score. YOUR FEEDBACK is CONFIDENTIAL. Team members will receive feedback but NOT WHO provided it.
Helping behavior (summative evaluation) provides a means to assess individual accountability across a semester and insures that the students take the formative process feedback seriously. It is very important that the final helping behavior score account for no less than 5-10% (up to approximately one letter grade) of the final course grade. Unlike the formative peer evaluations, the helping behavior scores need to be confidential and shared only with the professor. The summative helping behavior process should include parameters that allow for complete and honest disclosure.
Feedback-both process feedback and summative helping behavior feedback should be direct, clear, and ensure that group members are accountable to the rest of the team. The value of the process feedback is that all of the members of the team have the opportunity to discuss the feedback and make performance improvements over the course of the semester.
Based on interviews with over 6,000 team members and leaders, LaFasto and Larson (2001) identified eight characteristics of high performance teams: a clear elevating goal, a results-driven structure, competent team members, unified commitment, collaborative climate, standards of excellence, external support and recognition, and principled leadership. The eight characteristics are similar to Katzenbach and Smith's (1999) six team basics that define the discipline required for team performance: small number, complementary skills, common purpose, common set of specific performance goals, commonly agreed upon working approach, and mutual accountability. Peer evaluation procedures could potentially provide a results-driven structure while the specific criteria would provide the standards of excellence and help to ensure mutual accountability.
The importance of process feedback can be illustrated with a simple example. If a person boards an airplane in San Francisco and wants to fly to Washington D.C. they can expect to see a mountain range within the first 15 or 20 minutes of their flight. If instead all they see is ocean, then they are obviously going in the wrong direction because there is no ocean between San Francisco and Washington D.C. For students experiencing Team-Based Learning, this is what process feedback in the form of peer evaluation ought to be. It should establish benchmarks for them to figure out, given their assessment, whether or not their strategy is taking them to their goal. Evaluation is something every group member should do in order to make sure that the group is successful.
Students will support that which they help to create. I have students design their own rubric and process for measuring individual accountability. Before students begin to develop a rubric for measuring each of the team criteria, they are required to read a short four-page article written by Michaelsen and Schultheiss (1988) that clearly outlines seven characteristics of helpful feedback:
1.descriptive, not evaluative, and is "owned" by the sender
2.specific, not general
3.honest and sincere
4.expressed in terms relevant to the self-perceived needs of the receiver
5.timely and in context
6.desired by the receiver, not imposed on him or her
7.usable; concerned with behavior over which the receiver has control.
Michaelsen, L. K., & Schultheiss, E. E. (1988). Making feedback helpful. The Organizational Behavior Teaching Review, 13, 109-112.
When student TBL teams implement effective peer evaluation strategies for gathering feedback about how well their team is functioning they can make the necessary corrections that will ultimately enhance their overall productivity. The optimal condition occurs when team members share equally in all tasks, employ emergent leadership that shifts as a function of specific tasks and expertise alignment, and work together to produce learning that is greater than the sum of what could have been learned working independently. When TBL is working well, individual and team learning scores are high. As a result, helping behavior scores should also be high.
I've provided detailed information about the peer evaluation process (formative feedback) here: http://www.uky.edu/~drlane/groups/peer.html
There will be a special issue of "New Directions for Teaching and Learning" dedicated to Team-Based Learning published in January 2009 that should serve to clarify additional issues related to peer evaluation procedures and criteria. In the interim, I trust several of you will continue to find this TBL listserv useful.
Have a great week!
-Derek
Derek R. Lane, Ph.D.
Associate Dean for Graduate Programs in Communication<http://www.uky.edu/CommInfoStudies/GRAD/welcome.html>
College of Communications & Information Studies<http://www.uky.edu/CommInfoStudies/>
133 Grehan Building
University of Kentucky<http://www.uky.edu/>
Lexington, KY 40506-0042
Tel: (859) 257-7805
Fax: (859) 323-9879
Email: [log in to unmask]<mailto:[log in to unmask]>
Faculty website: http://www.uky.edu/~drlane
[cid:image001.jpg@01C88365.4B78E990]
IMPORTANT DISCLAIMER REGARDING THIS TRANSMISSION:
The contents of this email message and any attachments are confidential and intended solely for the addressee(s). The information may also be confidential and legally privileged. This transmission is sent in trust, for the sole purpose of delivery to the intended recipient(s). If you have received this transmission in error, any use, reproduction, or dissemination of this transmission is strictly prohibited. Neither the transmission of this email message and any attachments nor any error in transmission or misdelivery shall constitute waiver of any applicable legal privilege. If you are not the intended recipient, please immediately notify the sender by reply email and delete this message and any attachments.
From: Team Learning Discussion List [mailto:[log in to unmask]] On Behalf Of Jorden C. Sahl
Sent: Tuesday, March 11, 2008 10:32 AM
To: [log in to unmask]
Subject: the math of peer evals
Hi Everyone,
I recently finished my second Introductory Psychology course using TBL. I have a question that is not that complicated but for some reason I can't get myself around the math and I want to double check with other TBL users before I teach it again.
I used the Peer Evaluation form from the TBL book that assigns an average of 10 points, with one 9 and at least one 11 (max 15). A student asked me - so what does my average have to be to get "perfect" on the peer evaluations? And I was stumped (I had just never thought of it that way)
The max anyone can get would be 15, so I would assume in the rare case scenario of one person getting all 15s from their peers that would be "perfect". But then, if 10 is average, is 10 "perfect" (I.e., if someone gets an average of 10, do they get 100% on peer evaluations, with the possibility for students to get higher than 100%)? (My class picked 30% for the peer evaluations grade weight). Calculating it this way (10 is 100%) increases everyone's grade quite a bit (about half a letter grade), which I don't have a problem with, as long as it is correct!
I am second guessing myself with the math and want some input before I use this method again.
Thanks so much.
Jorden Cummings Sahl, M.A.
Graduate Student, Clinical Psychology
Depression & Wellness Research Lab (302.831.2215)
University of Delaware
Newark, DE 19716
[log in to unmask]<mailto:[log in to unmask]>
|
