Opción Múltiple

Decision Sciences Journal of Innovative Education Volume 3 Number 1 Spring 2005 Printed in the U.S.A.

Multiple-Choice Tests and Student Understanding: What Is the Connection?
Mark G. Simkin† and William L. Kuechler
College of Business Administration, University of Nevada, Reno, Nevada 89557, e-mail: simkin@equinox.unr.edu

ABSTRACT
Instructors can use both “multiple-choice” (MC) and“constructed response” (CR) questions (such as short answer, essay, or problem-solving questions) to evaluate student understanding of course materials and principles. This article begins by discussing the advantages and concerns of using these alternate test formats and reviews the studies conducted to test the hypothesis (or perhaps better described as the hope) that MC tests, by themselves, perform anadequate job of evaluating student understanding of course materials. Despite research from educational psychology demonstrating the potential for MC tests to measure the same levels of student mastery as CR tests, recent studies in specific educational domains find imperfect relationships between these two performance measures. We suggest that a significant confound in prior experiments has beenthe treatment of MC questions as homogeneous entities when in fact MC questions may test widely varying levels of student understanding. The primary contribution of the article is a modified research model for CR/MC research based on knowledge-level analyses of MC test banks and CR question sets from basic computer language programming. The analyses are based on an operationalization of Bloom’sTaxonomy of Learning Goals for the domain, which is used to develop a skills-focused taxonomy of MC questions. However, we propose that their analyses readily generalize to similar teaching domains of interest to decision sciences educators such as modeling and simulation programming.

Subject Areas: Constructed-Response Tests, Student Assessment, and Multiple-Choice Tests. INTRODUCTION
Collegeinstructors can use a wide variety of test formats for evaluating student understanding of key course topics, including multiple-choice (MC) questions, true-false, fill-in-the-blank, short answer, problem-solving exercises, and essay questions. Most of the alternates to MC and true-false questions are described in the literature as “constructed-response” (CR) questions, meaning that they requirestudents to create their own answers rather than select the correct one from a list of prewritten alternatives. Despite the wide diversity of test formats, there are many reasons why most students and many instructors prefer MC tests over the other types of examination questions. However, the literature contains a considerable
† Corresponding

author.

73

74

Multiple-Choice Tests and StudentUnderstanding

body of analysis and debate over this issue. Even after years of research on the trait-equivalence of MC versus CR examinations in different domains “. . . the evidence is inconclusive” (Martinez, 1999). The effectiveness of MC questions alone in measuring student understanding of class materials is of particular interest to those decision sciences instructors teachingquantitative courses (such as Production courses) or procedural language programming classes (such as Visual Basic or simulation languages). For example, many decision science instructors have heavy teaching responsibilities, or must balance their preferences for penetrating but labor-intensive computational tests against their research and publication responsibilities. MC tests provide an easy alternativeto such constructed-response tests, but nagging questions remain regarding the ability of such examinations to accurately and fairly measure student understanding of course concepts. The next section of this article examines this debate in greater detail and highlights some of the issues involved in test formats in general and MC testing in particular. It discusses both the research in...