Rapid Evaluation using Multiple Choice Tests and Petr Sojka Masaryk University in Brno, Faculty of Informatics Botanicka 68a, 602 00 Brno, Czech Republic sojka@fi.muni.cz ABSTRACT This paper describes a framework for effective design, typesetting, use and evaluation of students that uses multiple-choice tests. With this approach, based on a Tr-X engine, macros and a small program, several hundred customized tests can be typeset, printed and evaluated within several hours, allowing significant savings of educator's time and rapid electronic dissemination of test results. Categories and Subject Descriptors H.1.2 [User/Machine Systems]: Human information processing; J.I [Administrative Data Processing]: Education; J.7 [Computers in Other Systems]: Publishing General Terms Management, Performance, Human Factors, Design, Algorithms Keywords multiple-choice tests, TgX, evaluation, information system 1. INTRODUCTION There are many purposes of testing students: measuring their achievements (end of course), placement (uni entrance) or just diagnostic (feedback). With ever increasing numbers of students1, there is a natural demand for the automation of this costly evaluation proces. An evaluation based on multiple choice tests becomes one option, with its pros and cons. 2. A SYSTEM FOR THE GENERATION AND EVALUATION OF CUSTOMIZED MULTIPLE-CHOICE TESTS We have developed and used a system based on TpX macros by Jordi Saludes [2]. Our evaluation procedure consists of following steps: 'For example, six hundred new computer science students enroll every year at the Faculty of Informatics, MU Brno. 1. Teacher writes test questions in TpX (or randomly chooses from a pool of them) and indicates the correct answers in the source. Another option is to generate source file, e.g. from MySQL database of prepared questions. 2. Teacher saves the list of students who signed up for a test exported from the university web-based information system (ISMU, h t t p : / / i s . m u n i . c z ) . 3. For every student, a unique customized version of the multiple-choice test is created and a DVI file is produced. This is achieved by running TgX on the test source file with the student identity as a parameter. The random generator inside Tr-X is used to permute the possible answers to prevent peeking in a crowded examination room. Every answer on every page is labeled with a number in such way that from the sum of numbers in labels of answers chosen by a student, all answered questions can be effectively computed [2]. 4. Sorted DVI files are concatenated with a d v i c o n c a t program and then PostScript file generated with d v i p s with exam sheets is sent to the printer as a single print job. 5. Printed sheets are used for the test. Students themselves are asked to sum up all numbers corresponding to answers chosen, for every page, at the end of exam. There is simple divisibility test to prevent errors in summation. 6. Now only unique test number and sums for every page in the test are entered into a special program and grading is computed based only on those few numbers and on the log file created by TjX during the test generation phase. 7. Results/grades are imported into IS, and thus immediately seen by students via Internet2. 3. CONCLUSION We have six years experience with this approach by now. It is saving enormous amounts of time when regularly evaluating hundreds of students. The time for grading has been squeezed from days to hours or even minutes by this system: grades usually reach the students via IS on the same day. It is even quicker than scanning special answer sheets [1] and evaluating them. Examples of tests (in Czech) can be downloaded from h t t p : //www. f i . muni . cz/usr/sojka/mctests/. 4.

/lp/association-for-computing-machinery/rapid-evaluation-using-multiple-choice-tests-and-tex-HnkhmMjmQ1