The "Live Exam" Format

Spring was a nasty surprise for everyone. “Everything must go online!” Stripped of traditional classroom teaching midway through a semester was quite the challenge with maybe two or so weeks to make the transition. Policies had to be amended to accommodate the new environment. Lecture videos had to be created, edited and uploaded. Assignments, quizzes, and exams had to be completely digital. New programs, interfaces, and equipment had to be learned and mastered to survive in the virtual world. Luckily, I am apart of a University that was able to support their faculty in some of these areas. Innovation and creativity was still up to the individual lecturers, professors, and departments.

Academic Integrity

As if the long list of responsibilities above was hard enough, one of the toughest challenges was figuring out a way to maintain academic integrity.

I chose the route of a typical Canvas/Blackboard type exam given the time constraints at that point in the spring semester. Policies were in place to accommodate students by offering a large “window of opportunity” to take an exam somewhere along the lines of 72 hours. This allowed for plenty of time for exam material to be shared by the early exam takers. To mitigate this type of cheating, we adopted a brute force approach by writing exams with absurdly wide question pools, minimizing the value of any “shared information”.

Enter Chegg.

The next obvious solution to gain an unfair academic advantage was to resort to resources like Chegg. It did not take long to find plenty of exam questions copy/pasted onto these types of websites. The temptation to participate in this type of activity was too great to ignore given the lack of oversight and accountability that an online exam comes with. Responsibly following up on each post required days of painstaking data collection that translated into a 500+ page report of detailed evidence against each person accused of cheating. This level of enforcement is simply unsustainable, and an alternative way to administer exams virtually had to be identified.

Live Exams

What if there was a way to remove the temptation to post an exam question online and wait for a solution? Chegg tries to get your question an answer in 2 hours, though I have seen “Expert Answers” posted in about 10-15 minutes, sometimes less. Was there a way to use this flaw to our advantage?

Prof. Edwin Webster approached me about with an idea to somehow give an exam online, synchronously, where an element of that exam would be under our direct control. This is when we developed the concept of a “Live Exam” which I implemented in the Fall semester to around 400 students across two classes.

The Live Exam is just that, an exam that is synchronous and live (like “live TV”). This concept involves the merging of two online platforms, one being Canvas and the other being Twitch.tv, to deliver an exam that was accessible online. Canvas would act as the exam interface while Twitch was used to livestream exam content, keeping things synchronous. Without both resources, an exam could not properly be taken.

For example, imagine opening up an exam on Canvas and being faced with the following question:

There would be no way to know how to answer this without being connected to the livestream portion of the exam.

Here is what you would see on the livestream:

Okay, so two resources are needed to take an exam. So what?

The livestream portion of the exam is really what allowed me direct control over the pacing of an exam. It gave me the ability to put a timer on each, individual exam question. I could now control how long a question was accessible for.

If you take a look at the top right corner of the image above, you might notice a time limit. That is how long a student had to solve that question before the livestream moved on to the next question. A timer would be visible next to “Time Elapsed”, showing how much time had passed since the question was shown. A tutorial on how to create these types of exams can be found here.

We now have a foundation for removing the temptation to cheat simply because there was no time to get a response. The exam would not wait. It would continue on until it was over, and when it was over, the exam had to be submitted (late exams would earn a zero).

Practice, practice, practice

The obvious flaw in this approach would be a mountain of student complaints about this novel form of examination. A timer on each question would be viewed as unfair and stressful. I decided to tackle this potential problem by implementing the concept of “drilling”.

Often times students will practice a chemistry problem until they finally arrive at the solution and then move on. Time is precious and studying is usually low-priority. As faculty, we know that this is not the most effective way to learn as it does not lead to mastery. Mastery requires time and effort and lots of practice. As someone who has preached relentlessly about the merits of “putting a clock in front of you when taking a practice exam”, it never seemed to get get any buy-in from the majority of my students.

Enter the Timed Assessment Series.

To prepare for the Live Exam concept, it was necessary to provide an environment to practice in. I created a series of Timed Assessment videos that were crafted with the following design principles in mind:

• Each video targeted one concept.
• Each concept came with three similar questions.
• Each question presented had a built-in timer.
• The time limit for each question was consistently reduced as the assessment progressed.

For example, my Heating Curve TAS video contains a series of three questions, each relating to solving a Heating Curve problem. Each question is very similar in nature and requires the same mathematical approach to solving each of them. The first question is allotted 5 minutes, the second question gets 4 minutes, and the third question has a goal of 3 minutes. When done sequentially, the mathematical process of solving that question is “drilled” through repetition and exposes the student to the different ways the “same” question could be asked “differently”. It also gives the student an idea for how long it should take to solve this type of problem.

I managed to get 31 TAS videos created (among other things) before the end of the semester. This equates to 93 practice problems with a timer built-in to each question.

To further encourage practice, I made absolutely sure to model many of my exam questions after the ones I had my students practicing. I think it worked well.

Flexible exam options

The Live Exam format is not that rigid and allows for flexible exam options.

Multiple versions

If you have access to multiple computers, you can deliver multiple versions of a live exam. This is incredibly useful, particularly for large classes.

Extended time accomodations

I group students who are granted extended times on exams to one version of the exam. I simply change the time limit on the questions (x1.5) and deploy an exam for those students.

Question formats

All the question formats (multiple choice, multiple answer, numerical response, etc.) in Canvas/Blackboard are able to be used with the Live Exam format.

Additional proctoring layers

One can add additional layers of proctoring on top of a live exam using things like HonorLock or Lockdown browser. The thing to keep in mind is bandwidth. It can be difficult for someone with average bandwidth to watch a livestream while simultaneously maintaining a webcam uplink through a proctoring service.

So, did it work?

In short, yes.

I think the timed-question approach encouraged actual practice with the material with supported the retention of information (my comprehensive final exam scores indicated as much). Furthermore, after seeing none of my Exam 1 questions posted online, I was able to go home at the end of every exam night and not worry about the mountain of paperwork that I would have to face for identifying those that did not uphold the Honor Code of our fine institution. This “Live Exam” method simply made it impossible to do.