John,
In general, I like to have 1/3 to 1/2 of the questions fairly objective
in the sense that they ask definitions of the key concepts, major
conclusions, etc. (and you'll find that very few of the questions will
be answered correctly by everyone) and the other 1/2 to 2/3 that would
be more application, analysis or synthesis in the Bloom Taxonomy. I put
the definitions in the first part of the test, then ask later questions
that require students to think through the relationships between
concepts. For example, in the first unit of an organizational behavior
cours I have students read a couple of old studies, a chapter that
contains definitions of different ways of studying organizations, and an
article that outlines an organizational effectiveness model. Then on
the RAT (see attached), I test their understanding at several levels.
For example:
- Questions 19-30 are [note odd numbering to allow for split answers]
very straightforward. I use them to check to see if they recognize the
definitions in the effectiveness model.
Some of the questions do much more. For example
- Questions 1-3 and 13-15 test their ability apply definitions from the
text to classify the variables used in the two studies.
- Question 31-33 tests their ability to apply the definitions in the
model to a completely different situation.
- Question 34-36 tests their ability to apply the definitions of
centralized and decentralized from the book based on the information
contained in the methods section of the study.
- Questions 43-45, 46-48 and 49-51 test their ability to draw
conclusions about one study and project outcomes in a completely
different situation.
The latter type of questions are still objective in the sense that I can
idntify (and defend) a correct answer, but they still require a lot more
than being able to recognize definitions. Thus, they are much more
difficult and will create a lot of discussion without requiring students
to memorize detail. They have to be able to understand the key concepts
and think about what they mean. Thus, part of your "expert" role is to
decide what are the really key ideas and then figure out how to ask
questions that test more than students?ability to recognize them.
A final note--questions that test students?higher-level thinking skills
are difficult to write. You'll find that you'll get (and grant) a lot
of appeals the first time or two you use questions of this type.
>>> "Ludlum, John" <[log in to unmask]> 01/31/05 09:56 AM >>>
I am a member of a book group which is reading and discussing Team-Based
Learning. We're excited about the process and I have been fortunate
enough to attend workshops led by Drs. Fink and Michaelson. But. two
questions seem to pop up a lot: "What are the guidelines for
constructing good RATs?" and "What do they look like?"
I'm wondering if any of you would be willing to forward samples to me
that I could share with the (faculty) members of my group. Obviously,
we would keep these confidential.
The members of our group each courses in education, business (accounting
and finance), biochemistry, music history, and communication.
John Ludlum
Department of Communication
Otterbein College
614-823-3381
e-mail: [log in to unmask]
|
