All,

Thanks for the great discussion around the topic of formative vs summative assessments and weighting of various TBL components. I wanted to share some related work that we've recently done in this area, in case it's of interest to the group... 

In short, we recently conducted a 2x2 crossover study in which students were randomized to either a graded or ungraded iRAT at the start of the semester. At the midpoint in the course, students crossed over to the other grading condition. We found a modest reduction in iRAT scores in the ungraded group (72% vs 68%' p = 0.011), no difference in examination performance, no difference in tRAT scores, and no difference in students' achievement goals. We've submitted our results for publication but are waiting on peer review -- I've attached a poster of our findings as well as the drafted manuscript in case you're interested in reading more. 

FWIW, we did explore students' self-reported study time and found no difference when the iRAT was graded versus ungraded. We did, however, find that students in the ungraded condition spent ~20% less time taking the iRAT -- whether that is because they were less prepared and just selected an answer or simply didn't have enough "stake" in the game, I don't know. 

I'll also state that these findings should be interpreted cautiously within other educational contexts. For example, ungraded iRATs in an undergraduate course are likely to produce very different results. I even argue that if we were to perform the same study in first year pharmacy student (as opposed to second or third year) that the results would likely be different. It's a very nuanced discussion!