Tom Holme, Iowa State University
We’ve been emphasizing our traditional testing products lately in the DivCHED Newsletter, so it seems appropriate to return to the new products that are ready for users. In particular, our on-line testing system, called Metior, developed by the Center for Educational Software Development at UMass-Amherst, has been in pilot testing for several semesters and is ready to be used as a regular Exams Institute product. At the time this article is being written, we anticipate the processes for ordering exams will be in place by the time this Newsletter is out, or soon thereafter.
Some commonly asked questions about on-line testing that we can address include:
Will we stop selling tests printed on paper?
There is no plan to do so at this time, and really for the foreseeable future either.
What are the requirements for using the on-line tests?
It’s important to emphasize that the on-line testing system will still require exam security, so a secure and proctored room is needed. The activation codes for exams are connected to a specific range of IP addresses for the computers at the school where students will take the test, as part of the security precaution as well. Aside from the security, the major requirement is a computer with an appropriate browser. At present, four browsers are being supported – Chrome, Safari, Internet Explorer, Firefox.
How can I look at the items on the test?
The majority of the exams currently available are derived from exams that have been released as paper exams. Therefore, the process to look at the content of the items remains the same as it has for years. Specifically, a “professional copy” is obtained using our customary order process with it’s associated security protocols.
The only exam, to date, that has been developed directly for computer administration is the new Laboratory Exam for General Chemistry. We’re working on the process by which potential users will be able to preview items for this exam and encourage users interested in this exam to contact our Assistant Director for On-line Testing, Julie Adams, at email@example.com
Will the exam norms differ for on-line testing?
Ultimately yes. Because the on-line platform will allow the capture of each student performance, norms will be continuously updated for the on-line tests. Reports that are available to instructors will show their students performances relative to the test takers who have completed the test prior to them. To begin, the norming will be “seeded” with data from the original paper versions of the exam, and migrated to on-line only performances as soon as data set sizes allow it.
How will the ordering process for on-line tests differ from the paper tests?
The biggest difference is that in order to manage the technology set-up required, orders for on-line testing must be initiated at least several weeks before the exam is to be administered. The initial obtaining of an order form from the Exams Institute will be similar (for security reasons, order form requests always include an interaction with staff from the Exams Institute.) On-line exams are priced on a per-instance bases. In other words, unlike paper exams, the cost is associated with each student who takes an exam. (The cost is set at roughly 25% the cost of the concomitant paper exam.) Thus, when an exam is ordered there is a preliminary invoice generated that can be adjusted to reflect the actual number of students who take the test. This is followed up with a formal invoice after the usage is confirmed.
What steps are taken to set-up for the use of on-line testing aside from the ordering process itself?
The school ordering the exam designates an individual to lead their process – a lead examiner, if you will. This person is authorized to set-up additional personnel at the school to serve as proctors for the exam (and extensive proctoring tools are available to manage the exam administration itself.) The actual timing of the exam is also established at this point. Activation codes are designed to be used within specific time windows.
In addition to designating the proctors, the lead examiner provides answers about technology such as the range of IP addresses that the computers being used for testing will have. This step enhances the security of the exam, because if a student somehow obtains an extra copy of an activation code and tries to use a computer from another place, the code would not work.
An additional component of the security system handled by the lead examiner is a security check. Importantly, this process serves as a test of the system as well, as the instructor accesses Metior from an IP-allowed computer during the security check. Thus, when the active exam administration occurs, the system will have already worked – providing confidence in the set up parameters.
We anticipate constant improvements of the on-line testing system as we move forward. Those who have used the exams during the development process have found them to be very reliable. If you have questions about this system or about any of the things we are working on at the Exams Institute, don’t hesitate to contact me at firstname.lastname@example.org