I fear this leads to the K-type questions that National Board of Medical Examiners abandoned – see attached “graveyard” of questions for rational. http://www.nbme.org/PDF/ItemWriting_2003/2003IWGappendix.pdf (full document can be found at: http://www.nbme.org/PDF/ItemWriting_2003/2003IWGwhole.pdf Sandy ******************************************************** Sandy COOK, PhD | Senior Associate Dean, Curriculum Development | Medical Education, Research, and Evaluation (MERE) | W: (65) 6516 8722| F: (65) 6227 2698 | Assistant Manager: Belinda Yeo | [log in to unmask]<mailto:[log in to unmask]> | 6516-8511 Important: This email is confidential and may be privileged. If you are not the intended recipient, please delete it and notify us immediately; you should not copy or use it for any purpose, nor disclose its contents to any other person. Thank you. From: Team-Based Learning [mailto:[log in to unmask]] On Behalf Of Herb Coleman Sent: Saturday, December 28, 2013 8:59 AM To: [log in to unmask] Subject: Re: Assessing and marking student confidence on MCQs I'm still considering using the combined multiple choice method. It would work like this: Each item would have 8 choices... A). B). C). D). All of the above E). None of the above F). A and B but not C G). B and C but not A H). A and C but not B The options are to award partial credit if they choose A, B or C when that's only part of the answer or to only give credit if they get it exactly correct. Either way, the idea is to determine if they fully understand all of the components of a particular concept. It also reduces the likelihood of a successful guess. I do realize that it can also lead to mor second guessing but it should differentiate between those who truly know the answer and those who don't. Herb Coleman, Ph.D Adj. Prof.of Psychology Austin Community College On Dec 27, 2013, at 12:31 PM, Neil Haave <[log in to unmask]<mailto:[log in to unmask]>> wrote: This is interesting, though I must confess that I did not spend much time on the mathematical proof showing that the logarithm of students' confidence in the answer choice is better than the simple arithmetic sum. Larry discussed using this at the Atlanta TBL workshop at the end of Sept, and I must say I find the idea intriguing. I plan on using this next time I teach a TBL course by simply assigining each MCQ the weight of the number of answer choices and allowing students to distribute those points across the different answer choices according to their confidence in the answers. So if a MCQ contains 4 answer choices, students may assign all four points to one answer if they are entirely sure that is the correct answer. If they have no clue which answer is correct, I assume that they will assign one point to each of the four answer choices. The circulated paper seems to suggest that the logarithm of the student assigned points to their answers better illustrates their degree of certainty. Not sure I completely understand how that is so. Anyone on the listserv willing to give a layman's explanation? I hope everyone had, or is having, a great holiday break. Neil On Thursday, December 26, 2013, Kreinovich, Vladik wrote: My colleague Laura Madson from New Mexico State encouraged me to share this paper with the mailing list Shahnaz Shahbazova and Olga Kosheleva, " 'Fuzzy' Multiple-Choice Quizzes and How to Grade Them", Journal of Uncertain Systems, 2014, Vol. 8, to appear. http://www.cs.utep.edu/vladik/2013/olg13-04.pdf From: Laura Madson [mailto:[log in to unmask]<javascript:_e(%7b%7d,%20'cvml',[log in to unmask]);>] … paper presents a very neat idea that is well worth sharing with the rest of the team-based learning community if you'd like. There is a TBL list-serv connecting TBL practitioners internationally (more info here: http://www.teambasedlearning.org/listserv). I encourage you to share your article on the list-serv! Laura Madson, Ph.D. Associate Professor Department of Psychology New Mexico State University Las Cruces, NM 88003 [log in to unmask]<javascript:_e(%7b%7d,%20'cvml',[log in to unmask]);>