Dear Derek,

I strongly recommend against using open-book RATs for a number of reasons. The primary purposes of the RATs are:

1. Holding students accountable for thorough pre-class preparation so that you have class time to focus on concept applications.
2. Ensuring that students understand and internalize the foundational concepts related to the upcoming applications. Otherwise, their ability to tackle difficult applications is limited. (Note: The RATs don’t need to ask about details because the applications are open-book which allows students to develop an understanding of the details within a context.)
3. Ensuring that students learn through experience that, when faced with difficult decisions, even the best-informed member is far less likely to make correct decisions than relying on decisions made by teams discussing the alternatives and reaching a team consensus. This promotes the development of the both the skills and attitudes that are needed for effective teamwork during the applications and subsequent professional practice.

Based on both experience and empirical evidence, allowing open-book RATs work against all three of the above objectives:

• Knowing that they will have to remember the key ideas for a closed-book iRAT motivates students to develop a deeper understanding that they would need to be able to “quickly look up” ideas about which they were unsure. In fact, it is a good idea to remind them to be prepared to “explain the reasons for their answers” because that is what they will need to do during the closed-book tRATs.
• If you allow students to access their readings during the tRAT, doing so will limit their opportunities to learn from each other. This happens because, instead of actually discussing the concepts you want them to understand, much of the time will be spent with members individually reviewing the pre-class preparation materials. In fact, it is entirely rational for them to divide up the questions so that each member is responsible for looking up the answers to their assigned questions. When teams use this strategy, their scores are high but pretty much the only thing members talk about is who looks up what instead of actually discussing the concepts themselves. (Note: For several years, the Duke/NUS Graduate Medical School allowed students to use their pre-class study materials to answer a subset of the tRAT questions to see how doing so would affect their retention of the material. The result was that, even though they always correctly answered the questions they looked up, they were fare more likely to incorrectlyanswer final exam questions over the same concepts as compared to students who were not allowed to use their study materials and had to struggle with the identical questions on the tRAT.) 
• Another really harmful outcome of allowing open-book tRATs is that the reduction in time spent in actually discussing the concepts almost totally eliminates the opportunity for learning two critical lessons. One is that students will seldom experience the terrific power of group-consensus decisions for both learning and solving difficult problems. The other is that because students spend so little time discussing concept-related discussions, very few will develop the team decision-making skills that are critical to their success in both the concept applications and later as they enter professional practice.

On Thu, Aug 18, 2022 at 1:38 PM Derek Murray <[log in to unmask]> wrote: