I hope I’ve understood the question correctly – I think your problem is to do with outliers?
I reduce these by normalising the peer evaluation scores using a hopefully self explanatory spreadsheet, see below.
It also eliminates the potential attempt at ‘gaming the system’ by team member ‘C’.
|
Raw scores (3 point scale) |
AV |
Normalised scores (%) |
||||||
Team member |
A |
B |
C |
D |
|
A |
B |
C |
D |
A |
|
3 |
2 |
3 |
2.67 |
|
113% |
75% |
113% |
B |
3 |
|
2 |
3 |
2.67 |
113% |
|
75% |
113% |
C |
1 |
1 |
|
1 |
1.00 |
100% |
100% |
|
100% |
D |
2 |
3 |
2 |
|
2.33 |
86% |
129% |
86% |
|
Normalised PE score --- >
|
99% |
114% |
79% |
108% |
So far I’ve not had anyone with an uplift of more than about 25% (ie 125%) using this method. The minimum last year was 71% (ie 29% reduced mark)
This is applied to a combination of tRAT (max around 90; weighted 10%) and graded application exercise (max around 75: weighted 20%), with a resultant maximum score around 80%
I’ve got close to 100% weighted score this way (97% last year) but so far not exceeded it.
(even if I did exceed it slightly, I wouldn’t be over concerned, as the team mark is worth 30% of the assignment with the rest as iRAT (20%) and individual assignment (50%)).
You have to run this against each team, but it’s quite quick to do this.
Having a formative peer evaluation is of course vital but that’s a whole different topic.
I hope this is helpful
Steve
From: Team-Based Learning <[log in to unmask]>
On Behalf Of Carter, Neal
Sent: 07 April 2023 17:16
To: [log in to unmask]
Subject: Re: Peer evaluation guidance
CAUTION:
This email came from outside of the University. To keep your account safe, only click on links and open attachments if you know the person who sent the email, or you expected to receive
this communication. |
Hi John,
I have a few of questions for your consideration.
Hope this helps!
Neal Carter
Professor of Political Science
Brigham Young University—Idaho
From: Team-Based Learning <[log in to unmask]>
On Behalf Of John Gotwals
Sent: Thursday, April 06, 2023 1:59 PM
To: [log in to unmask]
Subject: Peer evaluation guidance
Hi all,
I’m looking for some peer evaluation guidance.
I use peer evaluations to adjust students’ averages across the team quizzes. I use the Fink Method to conduct this adjustment. This is where the ratio of the student’s average peer evaluation score and the team’s average peer evaluation score is multiplied
against their average across the team quizzes.
This works great, except in cases where there is one team member that gets a peer evaluation score that’s much, much lower than their teammates. This pulls down the team average and results in the person with the low evaluation taking a big hit on their team
quiz score while the other team members get a big boost (sometimes adjusting their team quiz scores well above 100%).
To deal with this I’ve thought of treating the team member with the really low evaluation as an outlier, removing their score from the adjustment calculations, and just applying a set deduction to their team quiz average. The challenges with this approach are:
Has anyone else dealt with this issue? Any suggestions or thoughts? I’m all ears!
Thanks,
John
--
*********************************
John K. Gotwals, Ph.D. (he/him)
Associate Professor & Graduate Coordinator
School of Kinesiology
Lakehead University
955 Oliver Rd.
Thunder Bay, Ontario P7B 5E1
Canada
phone: 807 343-8010 ext 7952
Lakehead University resides on the traditional territory of the Fort William First Nation and the Ojibwe, Odawa, and Pottawatomi nations, collectively known as the Three Fires Confederacy. I am
grateful for the opportunity to base my work out of that territory.
Are you ok? Our
How to ask for Help at Lakehead guide was made for you.
Need to talk to someone right now? Good 2 Talk is a free, confidential 24/7 post-secondary
student helpline. Call 1-866-925-5454 or text GOOD2TALKON to 686868.
To unsubscribe from the TEAMLEARNING-L list,
please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
To unsubscribe from the TEAMLEARNING-L list,
please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
To unsubscribe from the TEAMLEARNING-L list, please click here.
Further information about the UBC Mailing Lists service can be found on the UBC IT website.