Great thoughts - just to add to this
I use an online platform (OK, intedashboard!) and it works really well. I always have a paper backup but most weeks I get 100% of students completing with their mobile phone or laptop. This gives me real time feedback (invaluable for planning the mini lecture) and makes post analysis much easier.
Students are required to close all books and stay silent for the iRAT, I also ‘invigilate’ (wander round the class). I make it clear that if I see anyone accessing the course notes I will award them zero for the iRAT. In practice, the iRAT is pacey enough (10 questions in 15 minutes) that students don’t really have time to access anything else. Plus, I have tried to make the MCQs more about ‘application’ of the knowledge rather than straight memory. That prepares them well for the application activity.
And I randomise the answers.
I’ve only done this for 1 module, 10 weeks with a class of 80, and it took a few weeks for the rhythm to settle in but once established it worked really well.
By the way, I am doing TBL for a Project Management module, if anyone else is doing this please feel free to get in touch directly.
Bw
Steve
-----Original Message-----
From: Team-Based Learning [mailto:[log in to unmask]] On Behalf Of Brian O'Dwyer
Sent: 09 January 2019 03:54
To: [log in to unmask]
Subject: Re: i-RAT online
Hi Marina, great question
Jim and Ernesto, great ideas
I teach with airline management with TBL and built an online TBL platform (www.intedashboard.com) after getting frustrated with the paper-based process for IRAT, TRAT, Applications and Peer Evaluation. I have worked with 80 instructions to implement an online TBL platform and do about 3 demos with TBL educators every week. This issue comes up every week. Here’s what I have heard:
TYPES OF INTEGRITY CONCERNS:
1. Students copy from each other during the IRAT
2. Students look up answers on the web
3. Students copy the questions and pass them along to the next batch of students
TYPES OF MITIGATIONS:
1. Integrity question: The first question on my RATs are about aviation safety and integrity. I like to think this primes students to be more honest but I have no data to support this
2. Low/no points: Some make the RAT a very low portion of total grade. Some medical schools have switched to pass/fail.
3. Question mechanics: Randomize distractors and question order (as Ernesto pointed out); also prevent students from changing answers or going backwards or skipping forward
4. Collect data: Track the time it takes to answer each question by student. One school had us do this as they felt it would help them find and prosecute cheaters.
5. Restrict access to certain websites/IP address: Some LMS have this option. However, I have heard of students using cellular data on their phone to get around this.
6. Lockdown / Kiosk mode iPads: One institution used school owned ipads and set them in Kiosk mode to prevent browsing.
7. Browser blocking: There are some browser blockers out there (Respondus). However, each time our CTO has investigated this, he has have found just as many ways to get around the browser block than to implement it.
8. More complicated questions: Make the time per question short (e.g. 45 seconds each) and the question stem long and complex enough to that there isn’t enough time to switch tabs and google search
9. Open book RATs or don’t care: Some have taken the view that low order RATs are the basics. They don’t care if students look up the answer and are more concerned that the understand the basic theory even if they do it during the test. They make the RATs open book or don’t seem to worry about potential cheating. We found this for most corporate trainers using TBL. If learners have access to their work manuals on the job they then can access them during the RATs as well.
10. Free response RATs: This is one I haven’t seen done yet in TBL, but one that I am interested in developing the technology to do. Presumably it would be harder to cheat of the RATs were free response questions instead of multiple choice.
OUR FINDINGS:
1. In 2017, we surveyed 10 institutions using our web platform for TBL and integrity was the largest concern (36%). Regardless, 75% of faculty would recommend a web TBL platform because of the real-time data (81%) and time savings (54%).
2. In 2018, we surveyed 23 institutions using our web platform and integrity didn’t make the list of the top three faculty concerns. 93% of faculty and 70% of students recommended a web TBL platform.
3. My conclusion: There are benefits of going to a web platform particularly in the real-time data and time savings. A big concern is typically integrity. It depends how relevant the integrity issue is in your context (culture, institution, faculty and students) as to whether or not you it is something you want to try.
OTHER RESOURCES:
1. Posters on using technology enabled TBL are available in my poster bank:
https://www.intedashboard.com/wp-content/uploads/2017/10/face-to-face-TBL.png
https://www.intedashboard.com/wp-content/uploads/2018/03/Enhanced-technology-for-Team-based-learning-2018-03-02-FINAL.pdf
2. Online TBL Workshops: We are launching a monthly series of workshops about TBL online using a web TBL platform. This might be of interest to see what the student experience is like. The one for the European time zone is next week (Wednesday the 16th)
https://www.blog.intedashboard.com/copy-of-learn-more-developingtblmod
3. TBLC March 2019 Conference: There are two related sessions at the TBLC conference this year. One about academic integrity in online TBL and one about a medical school that switched to ungraded RATs.
Kind regards,
Brian
Brian O'Dwyer
CognaLearn Founder and Executive Chairman
InteDashboard: Empowering Team-Based Learning with Technology www.intedashboard.com
########################################################################
To unsubscribe from the TEAMLEARNING-L list, please click the following link:
https://lists.ubc.ca/scripts/wa.exe?SUBED1=TEAMLEARNING-L&A=1
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
########################################################################
To unsubscribe from the TEAMLEARNING-L list, please click the following link:
https://lists.ubc.ca/scripts/wa.exe?SUBED1=TEAMLEARNING-L&A=1
Further information about the UBC Mailing Lists service can be found on the UBC IT website.
|
