Dear Lark: your questions about accountability for the applications are ones I have wrestled with over the last 10 years. Teaching in a medical school, my students are very conscious about grades, extremely focused on getting the best grade possible. Ours is a standard-based assessment system, with a 70% grade required to pass each course. TBL typically contributed 10-30% of each course's final grade. Knowing this, I assumed that grading all applications would be the best way to maximize students' engagement as they worked through significant medical problems. I graded all applications for 7 years (2002-2008). A conundrum developed: as I created applications that were intentionally challenging for the best teams in the class, the inter-team discussions would occasionally be derailed by disgruntled teams who wrangled over small details in order to "get a point" (even when that single question was worth an exceedingly small portion of their team grade). I maintained an appeals policy that encouraged all teams to submit appeals during the 24 hours post-session if they believed an alternate answer was defensible. As you might surmise, the number of appeals increased in proportion to the difficulty of the application questions, requiring hours of my time to analyze the appeals. A second unhealthy consequence was the atmosphere that developed in the room during applications--anxiety about performance, tension, or downright hostility when large numbers of teams did not choose the best answer. In its worst expression, this resulted in students venting against faculty publicly ("unfair question", "why is this important?", "how could we possibly figure this out", etc.).
In 2009, after conferring with other TBL practitioners, I experimented with ungraded applications. To my surprise and relief, about 90% of all students remained highly engaged during applications, working hard to arrive at consensus and defending their team's choices vigorously. The whole atmosphere of tension and hostility disappeared, wrangling for points disappeared, and the discussions became more focused on principles, reasoning, and evidence for decision-making. A study was completed by a team of MS2 students (submitted for publication and under review), documenting reasons why students preferred the ungraded application format. The major negative outcome with ungraded apps was that about 5-10% of students became more passive and less concerned about their team's decisions--disengaged or weakly engaged. Interestingly, the other 90-95% perceived this behavior as unprofessional and inappropriate. The slackers were called out for irresponsibility on the peer evaluations, which negatively affected their personal score). In summary, since 2009, I have not graded applications while progressively increasing their difficulty level to enhance learning--clearly a good thing. I feel you will probably experience similar results IF the learning culture established among you and your students promotes accountability on a professional rationale, as opposed to an assessment-driven motivation. You might remind your students that their behavior toward teammates resembles closely their behavior toward patients who will depend on their skills and whom they will serve. best, Paul
On Jun 13, 2013, at 5:50 PM, Jim Sibley wrote:
> Hi
>
> In 11 years and probably a couple of thousand teams we have never assessed application activities....and never had any issues with engagement or motivation.
>
> I know that Paul Koles found that when he stopped assessing application Activities...he could really crank up the difficult without setting off student discontent.
>
> jim
>
>
> On Thu, Jun 13, 2013 at 10:51 AM, Lark Claassen <[log in to unmask]> wrote:
> I am putting together my first TBL course, which will be a hybrid course. It will serve 50-80 students. The course is a sophomore-level nutrition course designed for allied health majors, with the biggest population at our campus being pre-nursing majors.
>
> I have the content broken down into five 1-3 week modules to be delivered over 15 weeks with 13 actual F2F class meetings.
>
> I plan to give an online iRAT followed by an in-class tRAT. Then for the remainder of the 75 minute in-class session do application exercises. Students will be held accountable for pre-class preparation by their iRAT's and by team peer reviews (using the CATME program) done at mid-semester and at the end. So students have the opportunity to get feedback they can use to improve, I was thinking of weighting their two peer evaluations as 25% for their first peer review factor and 75% for their final peer review. These scores will be used to come up with a multiplication factor I apply to their tRAT scores to come up with a number that represents 15% of their final grade.
>
> Students will have 3 exams and a final plus some individual projects.
>
> In this scheme I've got a lot going on, but no room for team accountability for the application exercises. I'm hesitant to award points for their performance on the activities because I don't want to penalize them for what should be a formative process. And I don't want to grade on participation because that is what their peer evaluations are for. So how do I get them to care about the quality of their work after the tRAT is over? My worst fears? They will all get up and leave after it's over!
>
> One thought I had was to use the weekly grid I had planned to put in their team folders to record their team members attendance and their tRAT scores. What I would do is give them a sheet of gold stars- about 1.5 times more stars than we have class meetings- and have them at the end of each session consider who should get the gold star for the day for their pre-class preparation and in-class contributions. I will tell students two things: 1) the gold stars will serve as a record of how their team members are doing which will be helpful when they sit down to do their peer evaluations, 2) the gold stars will be considered when awarding letter grades if a student is "on the border".
>
> Can anyone see any problems this might create in the social dynamics of the team? Do I need to worry about them caring about the quality of their work once the tRAT's end or can I count on the four S's to make this a non-issue?
>
> --
> Lark A. Claassen, Ph.D.
> Lecturer
> Department of Biological Sciences
> University of Maryland Baltimore County
> 1000 Hilltop Circle
> Baltimore, MD 21250
>
>
>
> --
> Jim Sibley and Amanda Bradley
> 106-2575 West 4th Ave.
> Vancouver, BC
> Canada
>
> h 604-564-1043
> w 604-822-9241