Dear David, I have a number of observations about the design of the class that I think might help. I'll List them below. Larry K. Michaelsen Professor of Management Dockery 400G Central Missouri State University Warrensburg, MO 64093 O: 660 543-4124 F: 660 543-8465 >>> "Smith, David W" <[log in to unmask]> - 1/17/05 9:46 AM >>> Dear Larry, I have been using TBL for about 10 years. I was at your presentation for new faculty at OU, though I was at the Health Science Center. I had 26 students in 4 teams in Intro Stat for (graduate) students in public health. There is a wide range of training and work experience in the group, eg, about 6 MDs, one DVM, two DDSs, and the rest with BAs. About 5 were American military, two Canadian military (MDs and DVM). The Air Force Preventive Medicine Residency Program is just across town and the Canadians use it as well as the Army to some extent. There was a very wide range of work experience. I don't think the heterogeniety is a problem. It normally helps rather than hurts. I use RATs. They are short answer, I grade them by hand. I don't know what an IF-AT answer sheet is. I have a RAT each week early in the course and about every two weeks in the second half, except for exam weeks, about 5, 10, 15. Each RAT corresponds to a lesson, usually one chapter from my text, sometimes two chapters. In my judgment, this one is the key. The IF-AT answer sheets (see the attached file), which we only discovered about 3 years ago, would likely have solved the problem all by itself. Using IF-AT's for the team RATs gives immediate feedback on a QUESTION-BY-QUESTION basis. This allows team members quickly to correct their misconceptions of the subject matter and, even more importantly, learn how to work together effectively. The IF-AT virtually eliminates the problem of one or two members dominating team discussions and is the single most powerful team-building tool we know of. "Pushy" members are only one scratch away from having to "eat crow" and quiet members are one scratch away from being validated and 2 scratches away from being told that they need to speak up. The impact of the IF-AT on student learning is dramatic, as rooms often fill with cheers and high fives when correct answers are identified--and moans when answers are missed. I didn't really know what was going on with my group. They weren't obviously dysfunctional, but three of the six members only spoke in class to present the required minimum of problems. I never saw one of them in my office and the other two only showed up when they did badly on the first exam. (To some extent, the dysfunction was the luck of the draw in group assignment.) This sounds like the group never really became a team. I think the IF-AT would have helped a bunch. The team activities and assignments are all done in one week. I have two sessions a week of two hours each, start them on problems the first session, and have a presentation by one group member the second session. The group chooses the person to present, but they must rotate. Each team does 1-2 different problems. This way most of the problems in each chapter are presented in class. I sit with the class, ask for comments, and make comments. Obviously these are not the Same Problem, but I do have Simultaneous Report, as allowed by time and equipment. I don't recall Specific Choice. This sounds like different groups doing different problems and sequential rather than simultaneous reports and probably doesn't require them to make specific choices (see chapter 3 of the TBL book for an in-depth explanation). Using different problems for each team takes away key factors affecting their willingness to challenge other teams (thus, there isn't much energy in whole-class discussions): 1) Motivation--if everyone is working with the same problem and teams come up with different answers then it's evident that someone didn't get it right and everyone is motivated to re-examine both their own and other teams' work. If different teams work with different problems, it is pretty easy for a team to make a mistake that other teams won't catch. 2) Information--even if someone thinks the other team might have made a mistake, they are usually hesitant to call them on it. This is because, the challenger is at a clear informational DISadvantage because they only have a fraction of the time the presenters had to do an analysis. I have used weekly peer evaluations in the past, but students objected in Fall 2003, so I eliminated them and did not introduce them in Fall, 2005. This may have been a mistake but I thought my students were somewhat more mature than in past years. Some of them have had explicit training in group processes. Your experience highlights the "two-edged sword" characteristics of the peer evaluation. On one hand, they give members the opportunity to improve. On the other hand, requiring members to evaluate each other can significantly hamper the team-development process (see appendix B in the TBL book). In this situation, I think you made the right decision. I have team exams as well as team RATs. These have actually gone stunningly well. The exams are time-limited, but this is not a big problem. First, individual exam, then team exam (about 30 minutes), then I give them an answer sheet and have them grade the team exam themselves (about 15 minutes). They leave the room knowing quite well how they did. Exams are about making decisions using statistics. No calculation is done, so no calculators are allowed, nor books, nor notes. I'm not surprised that this works so well. You are clearly meeting two key requirements of good assignment design (see chapter 3): 1) Everyone is working with the same problem(s). 2) At both the individual and team stages, they are making specific choices. If you really wanted to have some fun, I'd take it to the next level and have them simultaneously report to the class as a whole. That would give them even more feedback. If all the teams have made the same choice, then they can celebrate together. If they have different answers, then you have a created a teaching/learnin moment for both you and the students. One other observation: I usually have my application-focused exams open-book. That way they learn how to use the material as references and also allows you ask questions that require knowing "details" without placing an unnecessary emphasis on memorizing them. It may very well be that being a true professional requires that kind of memorization. If so, I'd continue to do as you now are. The key question is, "How do true professionals in the field do their work?" Some of my methods come from the recent work in Calculus Reform, which is widely available on the web. Math teaching, which has always been about doing, has explicitly shifted away from lectures, and into active learning theory, eg, seeing, doing, presenting, writing, with regard to problems and solutions. Statistics is about decisions and judgements using data. Many students do not know this and some even resent it. What they really want is to know how their answer matches up with the correct one in the back of the book. This is mechanical and I have to move them past this into understanding that our decisions can be wrong in formal ways, ie, Type I and Type II errors, and in informal ways, eg, good experiments on subjects or drugs that have nothing to do with the people being treated, or only having observational studies for inferences such as smoking causes lung cancer and heart disease. Really good assignments always involve making judgements (choices). The fact that you are approaching your course from that perspective is probably why TBL is working well for you. I hope this helps. Keep in touch. Larry -----Original Message----- From: Larry Michaelsen [mailto:[log in to unmask]] Sent: Friday, January 14, 2005 4:46 PM To: [log in to unmask]; Smith, David W Subject: Re: Advice to groups on working well Hi David, I might have something that could help. However, before I send anything, I'd like to know more about the extent of your experience with TBL and what aspects of the process you are using. Also, I don't think the problem is likely to have been students' maturity. Based on my experience, when something like this happens the problem is in the way I set things up. My questions: 1) How long (years/courses) have you been using TBL? 2) Are you using RATs and, if so, are using the IF-AT answer sheets for the team test? How many RATs are you using? 3) Was this particular team dysfunctional on everything--or just some of the time? If some of the time, on what kinds of activities/assignments did they struggle? 4) Are the majority of your application activities designed around the three "S's" (i.e., Same Problem, Specific Choice, Simultaneous Report)? Larry Larry K. Michaelsen Professor of Management Dockery 400G Central Missouri State University Warrensburg, MO 64093 O: 660 543-4124 F: 660 543-8465 >>> "Smith, David W" <[log in to unmask]> - 1/14/05 2:48 PM >>> Hi all, This past fall I had one team that did not work well. They seemed to have trouble getting organized and keeping on task in class. Some teams do quite well and I attribute much of it to being older, more experienced, and even having some training in team building and organizing. I would like to give my students so they can keep on track, substantively. Does anyone have a brief, ie, one page, handout for the students on how to get organized and keep organized in your team? I would appreciate any help, direct or direction to another source, on this matter. Regards, David Smith David W. Smith, Ph.D., M.P.H. Associate Professor, Biostatistics Fellow, Institute for Health Policy The University of Texas School of Public Health San Antonio Branch Campus voice: (210) 562-5512 e-mail: [log in to unmask] or [log in to unmask]