News from the Exams Institute, Winter 2012

Tom Holme

The Exams Institute has been a busy place in 2012 with a large number of active exam committees. Some exams were released in the fall of 2012 and others will be coming in January and February of 2013. Between the Biennial Conference this summer and the ACS National Meeting in Philadelphia 10 exam committees were engaged in exam development this year.

The exams that will be released for use in early 2013 are as follows: Analytical Chemistry, whose exam development committee was chaired by Doug Beussman of St. Olaf College. The new Biochemistry exam will be released. In this case, the committee (chaired by Chrisy Miller of Adams State University) has constructed the exam to have a “core” section of 40 items that would be appropriate for a one-semester survey course, and an additional 20 items that are on more advanced topics. Thus students in a two-semester biochemistry sequence would take a 60-item exam.  The Institute will calculate norms for both the 40-item and 60-item versions of the exam.

The new Diagnostic of Undergraduate Chemistry Knowledge (DUCK) has also been completed and is ready to release in early 2013. As in the case of the previous exam, all items are interdisciplinary within chemistry and associated with scenarios for this exam, created by a committee chaired by Miles Koppang of the University of South Dakota.  The new version of the General-Organic-Biochemistry exam will be released by February 2013. The committee for this exam was chaired by Jeanie Collins of the University of Southern Indiana. Also in the introductory college chemistry category, the 2013 General Chemistry exam will be released in February as well. This exam was chaired by Bill Donovan of the University of Akron. We also have a new High School Chemistry exam that will be released in later February or early March. The committee that built this exam was chaired by Mark Dibben of the Air Force Academy Preparatory School.

The new on-line Laboratory Assessment for general chemistry is also available for use this spring semester. This exam will be an on-line only exam and uses different types of exam constructs, such as drag and drop interfaces, which are possible because it is on-line. It also includes full-motion video often to test aspects of appropriate techniques for each of the lab scenarios. The Lab Assessment has six such scenarios, three designed for use during the first semester and three for the second semester.  Costs for users of this system will be set based on the number of students who need to take the exam. Drop me an email at taholme@iastate.edu if you are interested in using this exam and need information on purchasing. During this first semester of active sales, we will be using a discounted pricing system for users who agree to provide additional feedback about how the exams were used, logistics of the exam administration, and other factors related to on-line testing environments.

In terms of our ongoing research and development efforts, we have kept up work on the undergraduate content map, which we call the Anchoring Concepts Content Map (ACCM). At the ACS meeting in Philadelphia groups met to look at the Analytical Chemistry component of the map and the Physical Chemistry component. The Organic Chemistry portion of the map will be out the door for publication before this newsletter hits the web site.

I’ll take a moment to note one big project we are about to initiate. We are engaging in a large-scale project to look at ACS Exams longitudinally in terms of their content coverage (aligned to the ACCM) and the levels of complexity of the items. We anticipate that this project will allow us to build a database for research purposes that will include an array of information about items that have been on ACS Exams. At this point, we anticipate including fields in the database for at least the following characteristics of exam items: (1) content as mapped to the ACCM; (2) cognitive complexity of items; (3) classical test theory parameters for student performance on items; (4) Item-response theory fit parameters for student performance on items. All of these variables will be characterized in terms of reliability of the ratings. We will begin this process looking at general chemistry exams going back roughly 25 years, but will seek to expand to other areas as time allows. If you can imagine data you would be particularly interested in seeing in such a database, feel free to contact me (at my email noted earlier.)

Finally, I will note that we continue to address issues related to exam security. It is vitally important that users of ACS Exams realize that there is no circumstance, regardless of the age of a particular exam, where an ACS Exam can be posted to the internet. We understand the desire to provide study materials to students and are working on developing improved resources in this arena. Nonetheless, there is never any implication of permission to publish any ACS Exam product on the internet. All such postings are violations of copyright and just as importantly represent a blow to the efforts of a large community that seeks to protect the integrity of these exams. Moreover, professors are also not allowed to use any items from an ACS Exam on a test they create themselves. ACS Exams are not storehouses of items for individuals to use. Rather they are whole instruments, and the only means by which they represent a valid measure of student knowledge is when they are used as an exam as constructed. We will continue to try to put the word out to the chemistry education community that ACS Exams remain copyright protected even when they are not the most recent version of the exam. Please help keep ACS Exams as useful to the community as possible and help prevent misuse of items or reproduction of any ACS Exam, either a complete exam or its items in any way. Thanks to all who have been careful over the years and who have helped us become aware of problems when they have arisen.

Theme by Danetsoft and Danang Probo Sayekti inspired by Maksimer