In-class Blackboard Exam Support at NAU

students testing

Take a panoramic photo of the class during the exam with your phone as an attendance check

Why are we doing this?

In high enrollment classes, multiple choice exams are hard to avoid. Photocopying hundreds of exams is both time consuming and expensive. Creating multiple versions of exams, and multiple scantron keys is tedious. Scantron forms are expensive. Scantrons are subject to read errors from erasure marks and incomplete bubbling. Scantrons must be processed after the exam, and then the scores need to be entered manually, or uploaded from a digital file that might take days to get back from the people who process the scantrons. Losing even a single copy of the exam compromises the integrity of the test. When students take an exam in Blackboard, both question and answer order can be randomized, creating unlimited versions, and the grades go directly to the gradebook, so students can immediately (or with a delay you specify) know how they did, and even (if you allow it) what they got wrong. Providing students with rapid feedback is good for learning and helps with retention. Blackboard even has sophisticated Item Analysis tools to help instructors identify questions that might be poorly worded, too easy, or too hard. Eliminating menial tasks like photocopying and data entry frees up valuable faculty time for more meaningful work. So, reduced use of consumables, significant cost savings, greater efficiency, and gains in teaching, learning, and retention are some of the important reasons that we've tried this new digital approach to in-class testing. It's working, and faculty and students both seem to like it. Yes, it introduces a few new challenges,* but in our minds they are worth it!

1. Before the Exam


Scheduling: Request support from e-Learning (elc-help@nau.edu) well in advance of each exam so we can get it on our calendars. Let us know the date, start time, room number, and approximate number of students in your class.

Authoring: You can write your questions directly in Blackboard, or there are several ways to write questions offline. You can use Respondus if you have a Windows computer. Ask ELC for help getting Respondus downloaded and installed. You can get a copy from the ITS Software Downloads page. You can also try one of the web-based authoring tools available online, but understand that these are unsupported, and you should use at your own risk, as we do not control them.

NAU Software Downloads page: http://nau.edu/software (for Respondus)

Online Test Generators: https://www.cod.edu/it/blackboard/testgenerators.htm

Exam Password: Protect your exam with a password, and it’s a good idea to change the password a few minutes into the exam, once everyone is settled, just to try to prevent anyone not in the room from gaining access to the test. Set a password from the pull-down menu, by selecting Edit Test Options, and scroll down to the last item in #2, Test Availability

Randomizing Question and Answer order: In most cases, you’ll want to randomize both question order and answer order within question. This will make it harder for adjacent students to compare exams. However it means you have to be careful with “all of the above”, “none of the above” and “more than one of the above” kinds of questions, because “all of the above” might be answer choice “a” and that will confuse students. Instead, write your questions like this, because the answer choices are clear in any order they are presented:

Which of the following fruits are usually orange?

a) Apple
b) Banana
c) Orange
d) Banana and Orange
e) Apple, Banana, and Orange

To randomize questions, Edit the Test Options from the pull-down menu, scroll down to #7, check the box for Randomize Question Order.

To randomize answers, select Edit the Test from the pull-down menu, click the Question Settings button, scroll down to #5, and check the box for Randomize Answer Order.

Question Pools: It is possible to write large banks of questions and to deliver a random subset of questions from the pool to students. You can also create multiple pools for different content topics. Textbooks often come with sets of questions, but beware! These sets sometimes end up online as “study guides” posted by instructors at other institutions, or fall into the hands of student resource websites.

More about question pools: https://en-us.help.blackboard.com/Learn/Instructor/Tests_Pools_Surveys/070_Reuse_Questions/030_Question_Pools

Images: We have seen some problems with the “hotspot” question type, particularly if a student attempts to zoom (magnify) the image to some value other than 100%. There are also cases where the Save and Next button may obscure the part of the image you need them to see. One option, which also makes cheating from outside the room even more difficult, would be to put the images, labelled Figure 1, Figure 2, etc. only on the main screen in the room (rather than in the exam itself), and refer to these figures in the questions.

Other Bb Learn Features: We recommend avoiding the “one question at a time” option in a room with poor wireless, as it causes frequent page loads. We strongly recommend avoiding the “force completion” option, since we find this feature is buggy, and locks students out of the test if anything goes wrong, as it often does, with their wifi connection. This requires a clear of all student responses and manual reset of the attempt.

Exam Access Date and Time: Set the date and time that the exam will be available to students.

Disability Resources: If you have students taking the exam at DR, be sure the proctors have the first and second passwords. If the date and time are different from the standard exam time, create a window of access for these students.

Release of Answers: Decide what information you want to release to students about right and wrong answers, and when you want to release it. We recommend you don’t release correct answers immediately after the exam, because then a student could share these with someone still taking the exam in a nearby seat.

Blackboard URL: A few days in advance of the exam, remind students that the exam will be administered in Blackboard. Encourage them to charge up their machines, run any updates that might cause problems, and log into Blackboard (and to your course) so that they know where to go. The direct URL for Bb Learn is https://bblearn.nau.edu and you’d be surprised how many people get there in odd and convoluted ways.

Loaners: If students don’t have a machine, Cline Library checks out Mac and Windows laptops. e-Learning will also bring some Chromebooks to your class, but these should be reserved for “last resort” kind of situations rather than student convenience. We bring machines for troubleshooting situations such as machines crashing, failure to connect to wireless, battery runs out, restart into forced updates, etc. We may ask for student ID cards from students who borrow our equipment just to keep track of how many are out, and who has them. We still believe that a secure, proctored, testing lab is a better solution but, in the meantime, this is the best option.

Compatibility: Students can take the exam with a Mac or Windows laptop, a Windows Surface Pro or Surface Book, a Chromebook, or an iOS or Android tablet. We do not recommend using a cell phone to take the exam, because the screen size is too small, and Blackboard’s mobile app isn’t very good.

Lockdown Browser: We experimented with a product called Lockdown Browser, but found it was too aggressive at locking out students if their wi-fi connection hiccuped through no fault of their own. This forced a reset of the exam and they lost all work, and we deemed that unacceptable. On wired machines, in a testing lab, this tool would provide a good additional layer of security, but in our most common situation, we find that live proctors at the back of the room are a better solution.

2. At the Beginning of the Exam

Seating: It helps, when possible, to leave the back row of seats in the classroom empty, so that proctors can watch all screens from behind.

Cell Phones: Encourage students to put cell phones and other non-testing devices in Airplane Mode or shut them down to reduce strain on the wireless, and to prevent them from texting the exam password to someone outside the room.

Rules: It is important to tell students what’s allowed during the exam and what’s not, and preferably what the consequences are, in advance. For example, if they need a calculator, or scratch paper, or if they must stay only in the exam window and not use another window or tab for Google, chat, etc, it’s good to let them know. If cell phones are banned, they should not be visible. Once the rules have been stated, give out the password to start the exam. Make sure cell phones are all put away before giving out the password.

Suggested Rules for Students (put these up in PowerPoint as students enter)

Once it’s quiet and exam rules are in effect, then release the EXAM PASSWORD (change it a few minutes later; don't release the new password)

WiFi Connectivity: Have students use the NAU wireless, and reserve the NAU-Guest network for problems. If students use NAU-Guest, they will need to open a browser window and Agree to the terms of use. In classes with connectivity issues, have students turn the wifi off and back on again after they are seated. Ensure that all extra devices are off or in Airplane mode. If you want to make the point with some humor, tell them that if their phone rings during the exam, you get to answer it!

3. During the Exam

Proctoring: ELC staff will be there for troubleshooting but will not be able to answer content questions, so it’s not a bad idea to tell your students to point to us if they need tech support, and to a content person (TA, GA, proctor, instructor) for content questions.

Cheating: What to look for? Students may try to text the exam password to someone not in the classroom. Watch for suspicious cell phone usage at the beginning. Change the password after everyone has entered the exam. From behind, it’s very easy to see students switching from the testing screen to other windows because the screen flicker is very noticeable. Have proctors watch for this. Also, some students may dim their screens or angle their screens to reduce detection, so you may want to ensure that their screens are visible to proctors. In some cases, instructors ask students to sign a sign-in sheet with their name and student ID number. Early in the exam, before anyone leaves, you can also use your cell phone to take a high quality panoramic photo of the classroom so that you could, if necessary, ask a student to prove they were present. If you are worried that someone is taking the test from outside the classroom, you can do a headcount of students in the room and compare it to the Bb Learn Grade Center column for the test. (We are working on an IP Address Masking technique that will only allow the test to be taken from specified locations; we can easily check the address of a suspected individual, but checking the whole class or looking for an unknown cheater is harder.) An in-progress icon will show for students taking the exam. Don’t forget about students you have allowed to take the exam at Disability Resources. You will need to refresh the screen to update the head count.

Saving Answers: Encourage students to verify that answers are saving as they go. There is a test progress tool at the top of the exam screen that shows them which answers they have saved. Click on the triangle to open it. If answers are not saving, have them “reload” the page in the web browser and ensure that they are still connected to the wireless. One caution though: unsaved answers will be lost when they reload the page. Sometimes, turning the wireless off and then back on fixes the connection problem.

question completion status

Question Completion Status: Gray questions (1-3) are saved

Rejoin Test or Clear Attempt: If a student’s machine has technical problems (usually wifi disconnection), they can try to re-load the page and/or re-connect to the wifi. They can switch to another machine, open the exam with the password, and resume (as long as Force Completion is not selected). Any saved work should still be there. Students can see which questions are saved by opening the Test Progress bar at the top of their exam. If the question number box is gray, it’s saved. If the question number box is white, it’s unsaved. If a student accidentally submits the exam before they are done, you can reset the student’s attempt. This will, unfortunately, clear any work done but gives them a chance to re-take the exam. You can reset the attempt by going to the Grade Center, selecting Tests, and finding the student who submitted accidentally. Click the Clear Attempt button after opening the Test Information area.

4. At the End of the Exam

Exam Submission: If students have any problem with final submission of the exam, you can manually submit it for them by viewing their attempt in the grade center and opening the exam information details. Many instructors ask students to show the Exam Completion screen before leaving the room. This prevents a student from being able to resume the exam after they have left the room. A good way to do this is to have them show their laptop screen as they leave the room, stationing a proctor at each exit.

Exam Completion: What are students allowed to do when they are done with the exam? Can they leave? Do they need to stay until a certain time? What can they do while they wait? Many instructors require students to show them the test completion screen, since a student could otherwise, potentially, close their notebook, leave the room, and continue the exam outside. Some instructors have the students sign a sign-out sheet with the time they leave. These measures help to ensure that nobody leaves with the exam still open, allowing them to continue to take it outside the room. We can look at the log for an individual student if you know their name, but it’s harder to look at the whole class in search of suspicious activity.

5. After the Exam

Survey: We are gathering data on how this service is working for students. Please encourage students to take our survey. You can share the link with them by e-mail, or embed it in your Blackboard course. We can help if you’re not sure how to do that.

Survey link: https://goo.gl/forms/EiqMN0EaAte5YIR63

Item Analysis: You can run a question item analysis on your completed exam to learn which questions were easy and hard, and which questions good students had trouble with (a possible indicator of a poorly worded question)

Concerns: Discuss with ELC staff any issues that came up, so we can try to resolve them or come up with workarounds for next time.

*New Challenges: Our system depends on a BYOD (bring your own device) model. Therefore students with fast, reliable computers may be at an advantage. Because we are dealing with hundreds of students taking the exam in a traditional classroom, we put a heavy load on the wireless. When a student's wifi connection hiccups, they can lose their work unless we can re-establish a connection. Unfortunately, tools that lock down the browser misinterpret these wifi hiccups as attempts to cheat, so we haven't been able to add that layer of security. Students have also figured out that, if they can get the password for the test, they can take it from outside the room, so we have had to become more vigilant about that, and have devised some counter-measures.

New Solutions: e-Learning comes to the exam to provide technical support, and brings a supply of loaner laptops to students who don't have one or who have technical problems with their own device. In-class proctors watch from the back, where all screens are visible and screen flicker is very noticeable, to ensure that students don't leave the test to "Google" an answer. Ultimately, we'd like to see the development of testing labs where students can make an appointment to take a test on a standard, wired computer. This will help to level the playing field, and will allow us to more reliably lock down the browser to prevent cheating.