John, In general, I like to have 1/3 to 1/2 of the questions fairly objective in the sense that they ask definitions of the key concepts, major conclusions, etc. (and you'll find that very few of the questions will be answered correctly by everyone) and the other 1/2 to 2/3 that would be more application, analysis or synthesis in the Bloom Taxonomy. I put the definitions in the first part of the test, then ask later questions that require students to think through the relationships between concepts. For example, in the first unit of an organizational behavior cours I have students read a couple of old studies, a chapter that contains definitions of different ways of studying organizations, and an article that outlines an organizational effectiveness model. Then on the RAT (see attached), I test their understanding at several levels. For example: - Questions 19-30 are [note odd numbering to allow for split answers] very straightforward. I use them to check to see if they recognize the definitions in the effectiveness model. Some of the questions do much more. For example - Questions 1-3 and 13-15 test their ability apply definitions from the text to classify the variables used in the two studies. - Question 31-33 tests their ability to apply the definitions in the model to a completely different situation. - Question 34-36 tests their ability to apply the definitions of centralized and decentralized from the book based on the information contained in the methods section of the study. - Questions 43-45, 46-48 and 49-51 test their ability to draw conclusions about one study and project outcomes in a completely different situation. The latter type of questions are still objective in the sense that I can idntify (and defend) a correct answer, but they still require a lot more than being able to recognize definitions. Thus, they are much more difficult and will create a lot of discussion without requiring students to memorize detail. They have to be able to understand the key concepts and think about what they mean. Thus, part of your "expert" role is to decide what are the really key ideas and then figure out how to ask questions that test more than students?ability to recognize them. A final note--questions that test students?higher-level thinking skills are difficult to write. You'll find that you'll get (and grant) a lot of appeals the first time or two you use questions of this type. >>> "Ludlum, John" <[log in to unmask]> 01/31/05 09:56 AM >>> I am a member of a book group which is reading and discussing Team-Based Learning. We're excited about the process and I have been fortunate enough to attend workshops led by Drs. Fink and Michaelson. But. two questions seem to pop up a lot: "What are the guidelines for constructing good RATs?" and "What do they look like?" I'm wondering if any of you would be willing to forward samples to me that I could share with the (faculty) members of my group. Obviously, we would keep these confidential. The members of our group each courses in education, business (accounting and finance), biochemistry, music history, and communication. John Ludlum Department of Communication Otterbein College 614-823-3381 e-mail: [log in to unmask]