In the Classroom
Using Oral Examination as a Technique W To Assess Student Understanding and Teaching Effectiveness Lee Roecker Department of Chemistry, Berea College, Berea, KY 40404;
[email protected] In this Journal and elsewhere, attention has been given to modifying teaching practices in order to facilitate student learning (1–9). These papers offer strategies to increase the conceptual thinking abilities of students (2–6) and offer suggestions for student assessment (7, 8). Brooks, Schraw, and Crippen recently described the interactive compensatory model of learning (ICML) and how this model can be used to improve student learning (5). The five components of the model that contribute to learning are defined in their paper (and summarized here): • Cognitive ability • Knowledge base • Learning strategy • Metacognition • Motivation
To improve each component, Brooks, Schraw, and Crippen suggest classroom approaches. They advocate, for example, strategic instruction and frequent opportunities for students to practice and receive feedback in order to improve their problem-solving abilities and metacognitive skills. To increase student motivation they encourage a mastery orientation of material, as opposed to a performance orientation. In a subsequent article, the same authors suggest that performance-related feedback is the key to effective instruction; they describe ways to provide feedback through techniques such as frequent testing or practice, in-class pair discussion, cooperative learning, and computer-assisted instruction. This paper describes the use of oral examinations to assess student understanding of chemical concepts. The oral examination format emphasizes many components of the ICML and provides immediate, performance-related feedback to students in a testing environment. In addition to examining the benefits of oral examinations in the context of the ICML, examination design and administration are also discussed. The ideas reported here are based on experiences using the oral examination format over the last 15 years in advanced inorganic chemistry, quantitative analysis, instrumental analysis, and general chemistry. Sample examinations used in these courses are available as Supplemental Material.W This article relates specifically to the use of oral examinations in advanced inorganic and in general chemistry. The Nuts and Bolts of Oral Examinations
Oral Examinations Need Appropriate Design Types and numbers of questions asked on oral examinations vary depending on the course, class size, and purpose of the examination. Any concept can be tested orally. However, in advanced inorganic chemistry, the topics of crystal
www.JCE.DivCHED.org
•
packing along with symmetry and group theory are particularly well suited to the oral format. Framing questions as “Show me…” statements encourages students to engage models in a manner that demonstrates an appropriate grasp of the material. For example, “The rock salt structure is described as a 6PO structure. Using the model of rock salt, show me what the term 6PO means.” Questions requiring an initial response of yes or no are always followed up by a request to show why the answer is yes or no. Two oral examinations are offered each term. Each student is allotted 15 minutes for the first exam, which covers crystal packing; 30 minutes per student are allotted for the second exam, which covers symmetry and group theory. The two oral exams account for 10% of the course grade. In the introductory chemistry course, the large number of students (32 in a section) prevents extensive use of the oral examination format. The oral format is particularly useful, however, for remedial testing early in the semester for the smaller number of students experiencing difficulties. Students earning lower than a grade of C on the first quiz are offered the opportunity to retake a quiz over similar material in the oral format. These quizzes consist of 3–5 questions and take about ten minutes to administer. The same offer is made to students who earn less than a grade of C on the first exam. The retest is usually about half the length of the written exam and takes 30–45 minutes to complete. An effort is made to include visual aids such as chemicals, flasks, graphs, or models when quizzing students orally in the introductory course. For example, students having difficulty understanding the concept of molarity are asked to weigh ~1/2 g of sodium chloride and dissolve it in water in a 250 mL volumetric flask. They are then asked to (i) calculate the number of moles of sodium chloride that they added; (ii) calculate the volume of the flask in liters; (iii) define molarity; and (iv) calculate the molarity of the solution that they prepared. After completion of these questions, the original quiz question that asked them to calculate the molarity of a solution is repeated. Few students are unable to provide the correct answer.
Oral Examinations Need Appropriate Administration With the possible exception of oral examinations in language courses, most students have never taken an oral examination before. This lack of experience causes their anxiety level to be high before the first examination. To help alleviate stress, instructions immediately before the examination are important to familiarize students with the process. Each student is given the same set of oral instructions. Students are told that they will be asked a series of questions that are designed to evaluate their understanding of the material. Students are instructed to “think aloud” in order
Vol. 84 No. 10 October 2007
•
Journal of Chemical Education
1663
In the Classroom
that their thought process can be observed (9). The grading system is explained and they are shown a scoring sheet. Four scores are possible for each answer: 3, 2, 1, or 0. Students answering a question correctly and without prompting earn 3 points. Each prompt a student receives results in a deduction of 1 point. Mistakes made along the way have no consequence on the grade if they are self-corrected. Following the questions in the order they appear on the scoring sheet allows for consistency in the administration of the examination. Students are told that comments will be made frequently on the scoring sheet, although students should not view them as negative comments on their performance because positive aspects are also recorded. Along with listing any prompts that were required, persistent difficulties a student or a class encounters are noted in addition to interesting problem-solving strategies. These notes are useful for making modifications to lectures or assignments as well as in assisting individual students. Students are encouraged to ignore the scoring sheet while they are taking the test. In addition, students are told that just because a followup question is asked does not mean that they have lost points or have not provided a correct answer. Follow-up questions are asked after nearly every response in order to better understand the students’ logic or further gauge their comprehension of the core concept being tested. Finally, each student is reminded that an oral examination is not a race—that ample time is available for its completion. The Benefits and Costs of Oral Examinations
Oral Examinations Allow for Assessment of Student Understanding Observing students and listening to them as they think aloud while they set up problems, use their calculators, read graphs, explain concepts, or twirl models in their hands allows the instructor to assess student understanding. The oral format helps illuminate problems and successes that students have transferring knowledge and their approaches to problem solving (9). In addition, the oral format often allows students the freedom to approach problems from a variety of acceptable perspectives and to provide a variety of acceptable answers. During one term in advanced inorganic chemistry, for example, students were asked to describe what they thought was the most obvious difference between the structures of fluorite and zinc blende. Two students, in a class of eight, responded that the empirical formulas were different in the two structures. The empirical formula was 1:1 in zinc blende and 2:1 in fluorite. The remaining six students stated that both were cubic close packed structures, but that zinc blende wasn’t as “full” as fluorite. At this point, three students added that zinc blende had an unoccupied tetrahedral layer while that same layer was occupied in fluorite. The remaining three students noted that zinc blende was a 6PT structure while fluorite was a 9PTT structure. All of these answers are acceptable and are not different from answers that might be provided on a written examination—each could be provided by recall from memory. What makes the oral examination
1664
Journal of Chemical Education
•
format more valuable than the typical written examination, however, is that student understanding of concepts can be assessed. Following each of the responses above, students were requested to use the models to illustrate the meaning of the answer that they provided. Students who only stated that the empirical formulas differed in each structure were asked how the formulas differed. Once the difference was stated, students were asked to point out the atoms involved in the 2:1 coordination of fluorite. Students responding that zinc blende had an unoccupied tetrahedral layer were asked to point out the location of the missing layer. Students stating that fluorite was a 9PTT structure were required to use the model to illustrate the meaning of the term 9PTT. Follow-up questions prevent students from being fully rewarded for a correct answer if they do not grasp the concept behind their answer (1). While concept clarification is important to students in the introductory course, the one-on-one interaction with faculty provided by the oral format is particularly valuable to students having difficulty. While the knowledge base of many weaker students is often adequate, they lack the sophistication to use that knowledge in a strategic manner to enhance their learning (6). Through the opportunity to observe students individually via the oral quiz, faculty can offer strategy instruction tailored to each learner.
Oral Examinations Result in Student Success Scores on the oral examinations in advanced inorganic chemistry are usually about 15–20% higher compared to scores on written examinations over similar material. All students who performed at an unsatisfactory level on the first quiz in the introductory course earned a satisfactory mark after taking the oral quiz. Four probable reasons explain the higher scores: 1. The most significant contributor to higher grades is the self-correcting nature of the oral format—students always arrive at a correct response before moving on to the next question. This correct response, even though they might have been assisted to reach it, sets the stage for them to answer subsequent questions correctly. On traditional written examinations, missing the first part of a multipart question often results in answering all parts of the question incorrectly. 2. Requiring students to think aloud during the oral examination makes them think more carefully. This extra measure of care is often evident as a student will start a response, and then, even before they have completed their initial thought, will see a better way to look at the problem and logically work their way to a correct answer from a new starting point. 3. The oral examination tests a relatively small body of material and students are able to focus their study efforts. This focus is surely intensified by the knowledge that the testing will be done one-on-one. They do not want to do poorly in such a personal situation. 4. When testing some concepts, such as crystal packing or molarity, the questions are concrete in that students have objects to manipulate.
Vol. 84 No. 10 October 2007
•
www.JCE.DivCHED.org
In the Classroom
Higher scores are not surprising within the framework of the ICML (5, 6). Providing feedback to the examinee during the examination—dynamic assessment—leads to improved performance (6, 10). Through performance-related feedback, the knowledge base of students is refined during the examination and their inherent cognitive abilities allow them to apply this refined knowledge to subsequent problems. Student motivation is maintained throughout the examination by responding to students in ways that encourage thinking. The “played back” strategy suggested by Kovacs-Boerger, where the teacher reiterates the student’s answer using the student’s own words, is particularly helpful during oral examinations (3). Upon hearing their responses, many students instantly see where they might have erred (11). Discovering errors on their own enhances their motivation to learn and increases knowledge about themselves as learners (8). Struggling students, in particular, appear to benefit from the oral examination format. The success of these students seems largely to derive from the increase in motivation as a result of personalized strategy instruction, an important component of the ICML. Personalized strategy instruction leads to improved performance and the satisfaction of doing well increases their desire to continue doing well. Many of these weaker students fear college-level chemistry before entering the course. Doing poorly on the first quiz confirms the view they hold of themselves as learners of scientific material. The personal, early intervention that oral quizzes provide enables them to perform better the rest of the semester.
Oral Examinations Allow for Instructor Growth Instructors grow as they find ways to help students learn. It is difficult to help students learn, however, if the barriers that interfere with their learning are not clearly understood. Observing students solving problems helps to uncover those barriers and provides a few precious minutes of mentoring that cannot be provided in any other way. Clearly seeing the weaknesses that students possess allows for the development of appropriate strategies by the instructor to improve student learning. Oral examinations can also uncover weaknesses in instruction. The first year that I used an oral examination, in my second year of teaching, I discovered that many students in advanced inorganic chemistry had no appreciation of why it is important to identify the highest-order rotational axis in a molecule. They had no idea, for example, to look for C2 axes perpendicular to the highest-order axis and had difficulty in distinguishing vertical from horizontal mirror planes. As a result of the insight provided by the oral examination, exercises similar to those on the examination are now used as hands-on activities during lecture to highlight the importance of identifying the highest-order rotational axis of a molecule. The basis for student failure would not have been realized without the observations made during the oral examinations (7). Oral Examinations Are a Positive Experience for Most Students Student comments about the oral examinations obtained in anonymous course evaluations and personal exit interviews
www.JCE.DivCHED.org
•
at the end of the term have always been consistent and enlightening. Most students believe that the oral examination provided a fair reflection of their knowledge. They were satisfied with their performance and would welcome oral examinations in other classes. Most students reported studying more for the oral examinations. Surprisingly, about half of the students interviewed volunteered that the oral examination provided a better reflection of their knowledge compared to written examinations because on written exams they could write something that was “fairly close” to being correct and get by with it. These students thought that the oral exam format made them demonstrate their understanding of the material. The anxiety that many students experience during the first oral examination is greatly decreased during subsequent examinations because they have a better idea of what to expect. For a few students, however, the oral examination is an ordeal. For these students, the pressure of being on the spot is intense and they often require additional time to complete the examination. These students, however, given their obvious anxiety about oral communication, probably benefit the most from the experience. Given the importance of communication skills in the workplace and the anxiety that oral examinations can arouse in graduate students (12–14), repeated experiences of orally explaining scientific material should be an essential component of an undergraduate curriculum.
Oral Examinations Require a Different Use of Instructor Time When using the oral examination in lieu of a regular examination in a class of ten students (such as in advanced inorganic chemistry), the amount of time spent at work does not change, but the use of that time changes dramatically. Instead of spending 5 hours grading written examinations, that time is spent directly observing and assessing students. Hambrecht ascertained that oral examinations, while requiring more time to administer compared to written examinations, do not require more instructor time overall because grading is finished as soon as the last student has left the office (8). Using oral quizzes as described in the introductory course can be an additional consumer of faculty time. In a typical class of 32 students in introductory chemistry at Berea College, one-third of the class will be invited to take the first oral quiz and about half of those students usually accept. Quizzes are often administered during office hours. The greatest use of time arises when one or two of the students who take the quiz subsequently stop by regularly for help—some make standing appointments. Working with a single student 90 minutes a week consumes more time than oral quizzes require for the entire semester. Given the benefits of oral examinations to student learning, being unable to extend the opportunity to all students in large classes can be frustrating. Larger classes could be evaluated by involving upper-level teaching assistants or students majoring in science education in exam administration. Oral examinations, however, are only one way to provide performance-related feedback to students. Many of the suggestions offered by Brooks, Schraw, and Crippen that provide performance-related feedback such as in-class pair discussion,
Vol. 84 No. 10 October 2007
•
Journal of Chemical Education
1665
In the Classroom
cooperative learning, computer-assisted instruction, and lecturing are useful in classes that contain large numbers of students (6). Relying on these other strategies with the entire class reduces the numbers of students who need the personalized attention of a one-on-one interaction. Summary Oral examinations are useful tools to assess the conceptual understanding of students and result in increased student learning. Within the framework provided by the ICML, these results are understandable given the opportunity for performance-related feedback provided during the examination and the resulting boost to student knowledge and motivation. As they administer the examination, teachers can discover gaps in student knowledge or common misconceptions that might be corrected by changes in lecture material, style, or assignments. Also of importance, oral examinations serve as one more way in which the communication skills of students can be improved. Students gain experience, under pressure, at providing oral explanations of technical material.
The author thanks Berea College for financial support through the College-sponsored professional development program, Communication across the College. Discussions with colleagues in that program and their encouragement are gratefully acknowledged.
Journal of Chemical Education
Supplemental Material
Sample oral quizzes and examinations used in advanced inorganic chemistry, instrumental analysis, quantitative analysis, and general chemistry are available in this issue of JCE Online. Literature Cited 1. 2. 3. 4. 5. 6. 7. 8. 9. 10.
Acknowledgments
1666
W
•
11. 12. 13. 14.
Moore, J. W. J. Chem. Educ. 2004, 81, 7. Phelps, A. J. J. Chem. Educ. 1996, 73, 301–304. Kovacs-Boerger, A. E. J. Chem. Educ. 1994, 71, 302–303. Teixeira-Dias, J. J. C.; Pedrosa de Jesus, H.; Neri de Souza, F.; Watts, M. Int. J. Sci. Educ. 2005, 27, 1123–1137. Schraw, G.; Brooks, D. W.; Crippen, K. J. J. Chem. Educ. 2005, 82, 637–640. Brooks, D. W.; Schraw, G.; Crippen, K. J. J. Chem. Educ. 2005, 82, 641–644. Hartman, I. S. J. Coll. Sci. Teach. 1995, 24, 401–403. Hambrecht, G. Delta Kappa Gamma Bulletin 2003, 69, 31– 32. Bowen, C. W. J. Chem. Educ. 1994, 71, 184–190. Swanson, H. L.; Lussier, C. M. Rev. Educ. Res. 2001, 71, 321– 363. Garrison, C. The Teaching Professor 2002, 4, 3–4. Brown, J. J. Coll. Teach. 1992, 40, 101. Arnkoff, D. B.; Glass, C. R.; Robinson, A. S. J. Counseling Psych. 1992, 39, 382–388. Kuhlenschmidt, S. L. Teach. Psych. 1992, 19, 86–90.
Vol. 84 No. 10 October 2007
•
www.JCE.DivCHED.org
