Like Molly, I have not had any issues with peer evaluations (not sure if this says more about economists than TBL :-)) but I did wonder what kind of instructions you all are giving your students for thinking about what the quantitative score represents? That is, what is your rubric? I give my students a fairly detailed rubric that lists the kinds of things that they should be using to make their evaluation (e.g., what good and less good participation looks like) and expect the qualitative explanation they give for their score to reflect what is in the rubric. That also gives students language to use when they do give lower scores and I think that may help.I have some ongoing quantitative research on peer evaluations, hoping to send out for review in early summer.
While I don’t have enough racial diversity to include in the analysis, I have found females get higher peer evaluations from both males and females in my introductory level microeconomics courses (with over 4000 peer-to-peer evaluations) but this difference diminishes as the students advance in economics courses. That is, females get higher evaluations than males in my intermediate microeconomics but not by as much, and there is no gender difference in evaluations in my environmental economics course (that requires the intermediate as a prerequisite).
I give midsemester evaluations that do not count toward the students’ final grade, as well as end-of-semester evaluations that do. I use the midsemester evaluations to give a warning/feedback to students who do not get good ratings, or for whom the peer evaluation weighted team portion of their grade is lower than their individual performance. Sometimes good students don’t get “bad” peer evaluations but could still step up more to help a team do better.
I think a problem some encounter is with asking for peer evaluations too often. Students should be reminded that their interactions can affect their grade but they shouldn’t be asked for evaluations after every interaction (in my humble opinion). Mid-semester provides the opportunity to give feedback (anonymous), allowing students to adjust their behavior for the better.
I have not found a problem with students not differentiating evaluations in a way that does not reflect contributions fairly accurately, for the most part. I tend to see pretty consistent evaluations across students for a given teammate, that is, if one is rated lower than the others by one teammate, that student tends to be rated lower by the rest as well. I also rely on qualitative comments and my own subjective observations, which I know instructors of very large classes cannot do.
I remain a proponent of both quantitative and qualitative evaluations.
Molly Espey, Professor
John E. Walker Dept. of Economics
247 Sirrine
Clemson University
Clemson, SC 29634
(864) 656-6401
From: Team-Based Learning <[log in to unmask]> on behalf of "Amy E. Hughes" <[log in to unmask]>
Reply-To: "Amy E. Hughes" <[log in to unmask]>
Date: Friday, March 20, 2020 at 12:50 PM
To: "[log in to unmask]" <[log in to unmask]>
Subject: Re: Questioning the value of quantitative peer ratings
Thanks very much for bringing this up, Herbert—
I, too, have these concerns, after roughly 7 years of using TBL in a Humanities classroom. I’ve struggled with peer evals again and again, particularly when students object to giving differentiated scores to their teammates. But I kept doing them because I was taught it is a crucial part of the pedagogy. But Herbert's post — especially the observations about African American students getting consistently lower evals — makes me want to abandon the quantitative peer eval as well.
During this semester in particular -- when I've had to move my class online five weeks before the semester is over (I'm SO grateful to everyone on this list who provided help and guidance about that! Thank you!) -- I don't know if it is a good idea to ask students to evaluate each other, when so many of them are grappling with unusual problems that cannot be "fixed" (e.g., uneven internet access, missing class in order to travel home, mental health challenges, etc).
I know reflection and constructive feedback is valuable. But in light of the challenges and drawbacks that many of us experience while using peer evaluations, not to mention the potential for someone’s bias to impact a student’s grade, are the cons and risks outweighing the pros and benefits?
How are TBL teachers accounting for and/or offsetting the potential for bias in peer evals? Has any research has been done about how race, gender, disability, etc. affects students’ peer evaluations in TBL classrooms? As we know, there is ample research that faculty who identify as POC, women, etc. tend to get lower ratings and comments from students, sometimes laced with biased language. Therefore, it makes sense (unfortunately) that this might happen when students evaluate each other.
I'd welcome any references, guidance, and strategies you can share.
Thank you,
Amy Hughes
Dr. Amy E. Hughes (she/her)
Associate Professor of Theatre & DramaHead, Bachelor of Theatre Arts (BTA)
University of Michigan, Ann Arbor
[log in to unmask]Bio: http://bit.ly/AEHughesUM
Coeditor, A Player and a Gentleman: The Diary of Harry Watkins, Nineteenth-Century US American Actor: http://bit.ly/PlayerGentleman and http://harrywatkinsdiary.org
Author, Spectacles of Reform: Theater and Activism in Nineteenth-Century America: http://bit.ly/HughesSpectacles
On Thu, Mar 19, 2020 at 1:30 PM Herbert Coleman <[log in to unmask]> wrote:
Over the years my students and I have struggled with peer ratings. Originally, they had difficulty dividing points between members so I changed the format. Then some teams kept giving all top scores to every member. When I put in a penalty for that a few teams collaborated and decided to gage so that they each varied their ratings and each gave a different person a bonus.
Two years ago, I noticed that African-American students were consistently getting lower ratings. This was the case even when I observed one who was clearly the team leader and the qualitative feedback says they doing great. Now for the second time, I have students giving ratings to a student who had dopped the class or was absent on the date of the TRQ or even the whole unit. This last case enraged me because the team members who completed the ratings gave the person who was absent a higher rating than someone who was actually there!
If students are resiting completing these faithfully, then what value are they, really? I use a qualitative rating but that doesn't figure into the score for the one being rated just the one doing the rating.
I've had the discussions and explained the value but still, this is the kind responses I'm getting.
I guess I'm asking how do we get students to faithfully hold each other accountable? I should mention that they also do a self-evaluation for each unit.
--
Herb Coleman, Ph.D
Adjunct Professor of Psychology and Student Development
RETIRED--Dir. Campus Technology Services
Austin Community College
[log in to unmask]
(512) 223-1790 ext. 22162
************************************************************************************************************
“Keep working, keep striving, never give up. Fall down 7 times get up 8.Without commitment, you’ll never start. But more importantly, without consistency, you’ll never finish.
Ease is a greater threat to progress than hardship. So, keep moving, keep growing, keep learning.
See you at work.”
― Denzel Washington,
************************************************************************************************************
To unsubscribe from the TEAMLEARNING-L list, please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
To unsubscribe from the TEAMLEARNING-L list, please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
To unsubscribe from the TEAMLEARNING-L list, please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
To unsubscribe from the TEAMLEARNING-L list, please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.