In the Classroom
Using Poster Sessions as an Alternative to Written Examinations—The Poster Exam Pamela A. Mills* and William V. Sweeney Department of Chemistry, Hunter College, CUNY, New York, NY 10021; *
[email protected] Stephen DeMeo Department of Chemistry, York College, CUNY, Jamaica, NY 11433 Robert Marino Department of Physics, Hunter College, CUNY, New York, NY 10021 Sandra Clarkson Department of Mathematics and Statistics, Hunter College, CUNY, New York, NY 10021
Science curriculum reform aimed at retaining students in science and encouraging students traditionally underrepresented in science often includes alternative examination or assessment practices (1–3). The use of posters has been a minor part of this reform. In chemistry, posters have been used as part of laboratory course work (4, 5), as a supplement to the lecture material (6 ), and as part of an AP chemistry course in high school (7). Posters have been incorporated into the course work of other science disciplines as well (8–11). Several advantages have been claimed for posters, including familiarizing students with this common format for scientific meetings (4 ), promoting collaborative learning, and developing communication skills (11). We present here a description of the use of posters in place of a traditional written exam in our integrated, four-semester chemistry/physics/ mathematics course. We cannot expect students to perform adequately on a poster exam without practice. Students are introduced to posters in the first semester in the laboratory. In the second semester the poster exam is introduced as an alternative to a midterm exam. The Poster Session in the Laboratory Following each laboratory exercise, a group of students is selected to present the results of the previous week’s work using the poster format. The presentation takes place in the laboratory, lasts for approximately 20 minutes of the threehour period, and serves as a forum to discuss the previous week’s work. Our laboratories range in duration from two to four weeks. Thus many of the poster presentations are “preliminary” and do not reflect the completion of the laboratory exercise. The posters are not graded, but the students and the laboratory instructor provide feedback on both the presentation and the content of the discussion. The laboratory poster sessions enable students to develop their oral presentation skills. At the end of the semester each student lab group selects a previously delivered poster and presents a revised version of the poster at a group session. Faculty from chemistry, biology, physics, and mathematics are invited to attend. In contrast to the lab presentations, the final poster session has the feel of a scientific meeting with multiple posters set up throughout the room. For the first time, the students present 1158
their work to an unfamiliar audience. This session differs from a poster exam because questions are limited to the laboratory experiment and because the grade counts for only a small portion of the overall laboratory grade. As a result of the semester-long experience with posters, students have acquired familiarity with all aspects of a poster presentation: the content, the visual presentation, and the oral communication, argumentation, and defense of the poster. These aspects are refined and adapted to the poster exam administered in the next semester. The Poster Exam Beginning with the second semester of our four-semester course and continuing through subsequent semesters, we use the poster exam as an alternative assessment process that replaces one written exam each semester. The poster exam covers one module or topic in the semester, and typically counts as 15–20% of the overall grade (equivalent to a onehour exam). For example, in the second semester, the poster exam serves as the assessment tool for the module on gases (12). Currently, in the third semester students present a poster related to entropy and free energy, and in the fourth semester the poster exam is based on an optics module, one of the topics in physics. We have also used this format for assessment in mathematics, with a poster exam on rational functions, logarithms, and exponentials. The grading of the poster exam occurs in two parts. First students are graded on the content, organization, and design of the poster itself, the oral presentation of the poster, and their ability to answer questions on the poster. The students receive a group grade, which we chose to score as 60% of the exam. The detailed breakdown of the poster evaluation is shown in Figure 1. The second part of the grade is a series of brief individual oral exams. This individually graded portion makes up the remaining 40% of the exam grade. Students are instructed to prepare a 5- to 8-minute presentation of their poster. Although students are permitted to “tag-team” the presentation, we suggest that each group choose one person to be responsible for the presentation. We remind students that they would all receive the same grade for the poster presentation and suggest they work as a group on the presentation. In addition, the poster evaluator can ask any of the students questions. While we share the concern
Journal of Chemical Education • Vol. 77 No. 9 September 2000 • JChemEd.chem.wisc.edu
In the Classroom
of many faculty members that this format would enable one or two students to carry the primary responsibility for the work and the consequent grade, we find that most groups spend considerable time practicing with the presenter. However, we did find on several occasions that one student in a group was clearly not contributing substantially to the group effort. This lack of participation becomes evident in the individual oral exam portion of the poster exam. Individual knowledge of the broad subject matter is probed in the second half of the test with an individual oral examination administered to each student in the poster group. Questions are drawn from a database constructed by the course designers and instructors, and labeled according to degree of difficulty. After each poster exam the database is refined and enlarged. The examiners are asked to draw questions randomly from the database and score the answers on a three-point scale of outstanding, acceptable, or unacceptable. Each student’s response is scored and entered on an answer sheet, which the students do not see, and returned to a folder that remains with the group. The next examiner then has access to the folder and does not repeat the prior questions asked. In this way each student receives at least two different oral exams, and most are given three. Examiners tend to conduct this aspect of the poster exam in one of two ways: some ask each student a different question, while others give the same question to all students.
Grader:
The 60/40 weighting of the group (poster) grade to the individual (oral) grade results in an overall class grade distribution different from that for a traditional written exam. In all of our poster exams, all the students who participated in the poster presentation “passed” the examination. Our grade distributions ranged from A to D, in contrast to a written exam, which some students typically fail. We hypothesize that the public nature of the poster exam motivates students to put together at least a minimally acceptable poster. The difference between grades given on the poster exam and grades given on the written exam diminishes by the fourth semester of the course. Another difference between the poster and written exams is which students excel. On almost every exam (we have administered seven poster exams to date), the best posters and the best presentations were not by the students who received the best overall grades in the course. Some students thrive on the opportunity to carefully construct a scientific argument on a limited topic. The students receiving the highest overall grades in the course never did poorly on a poster exam but the poster exam affords an opportunity to test additional skills and for other students to excel. We believe that creating opportunities for many students to excel is an extremely positive feature of alternative assessment measures such as the poster exam. Billington has stressed the need to use a diversity of assessment strategies (10).
Student Group: Team 4, Afternoon Lab: Student Names
I. GROUP GRADE--60%
Points
Visual Presentation of the Poster--10%
(Score range 1-10)
A. Visually Attractive Unacceptable Acceptable Outstanding Compelling, Makes you want to read it No Handwritten panels ......................................................................... U A O B. Graphs Legible: Title Included, Axes labeled Data points Readable .............. U A O Content and Organization of the Poster--20% A. Flow of the Panels is Sensible Introduction, Results, Data Analysis, Discussion, Conclusion .... B. Important Features of the Module are Selected and Highlighted ................ C. Use of mathematics and mathematical expressions clearly understood ....... D. Physical concepts are accurately identified and used properly .........................
U U U U
A A A A
O O O O
A. Content: Succinct, main features quickly summarized .................................... B. Delivery: Confident manner, Clearly audible..................................................... C. Time limit observed ...................................................................................................
U U U
A A A
O O O
Question/Answer Session--15% ..............................................................................................
U
A
O
Oral Presentation (5 minutes maximum)--15%
II. INDIVIDUAL GRADE--40% Student #1 Question Number
Student #2 Difficulty Level
Accuracy of Response (U/A/O)
Question Number
Difficulty Level
Accuracy of Response (U/A/O)
Figure 1. A representative first page from the grade sheet given to faculty evaluators for use during a poster exam.
JChemEd.chem.wisc.edu • Vol. 77 No. 9 September 2000 • Journal of Chemical Education
1159
In the Classroom
The administration of the poster exam requires participation by faculty not involved in teaching the course, in addition to the course instructors. In a class of approximately 80 and with an optimum group size of four students (three to five is reasonable), there are approximately 20 groups. We have found that each exam takes 20 to 40 minutes, so that a single examiner can be expected to complete no more than 4 exams in the two-hour exam period. Thus at least 10 faculty members are required. Sisak reports a requirement for substantially fewer faculty, with the expectation that a single faculty member can evaluate 9 or 10 posters in a single hour (6 ), compared to the 2-per-hour rate in our experience. Part, but not all, of this discrepancy is due to the length of the oral exam component of our poster exam. Examiners need to be instructed not only in the basics of what the poster exam is, but also in some aspects of how to conduct the oral exam. They need to be told that this component of the exam is not a teaching opportunity for the examiners, and they are instructed to provide no feedback to the students regarding their answers. Of course, after the examiner leaves, the students are free to discuss the questions asked and the answers they have given. Although the “no feedback” rule may seem obvious, it is often difficult to get teachers to refrain from correcting or commenting on students’ answers. The exam the students encounter varies depending upon the individual examiner they receive. Even though we ask faculty to draw randomly from the database, they often select questions closely related to their discipline. In the gas poster exam, the chemistry, physics, and biology faculty were extremely knowledgeable about the gas laws and the experimental methodology involved in the students’ work. The mathematics faculty was not knowledgeable in these areas, however, and tended to ask more about the functions the students were graphing, the families the functions belonged to, and the identification of asymptotes. This lack of uniformity among the actual oral examinations presented to the students is a significant feature of the poster exam. Nonuniform examinations contradict a precious tenet of fair examinations—that all students receive the same exam and therefore students are judged by the same criteria. We had anticipated, given how important a “fair” exam is to students, that this could cause some problems. Surprisingly, students perceived the poster exam and written exam formats to be equally fair: Questions
Less
Same More
1. How motivated did you find yourself to study for a poster exam compared to a traditional written exam?
7
16
11
2. How would you rate the effectiveness of poster exams as a learning tool compared to traditional written exams?
2
9
23
3. How much work did you do to prepare for the poster exam compared to traditional written exams?
4
15
15
4. How enjoyable did you find poster exams compared to the traditional written exams?
6
8
20
5. How accurately do you feel the grade on a poster exam reflected your knowledge, relative to a traditional written exam?
9
18
7
6. How well correlated do you feel the work you put in was to the grade you received, compared to a traditional written exam?
9
19
5
7. How fair do you feel this poster exam was relative to a traditional written exam?
4
27
2
1160
This is consistent with the results of Baird (11). Occasionally a group felt that the grade unfairly reflected their knowledge and preparation. In these cases the course instructors consented to further examine the group. In all of these cases, the grade the student had received from the original exam was identical to the grade from the extra exam. The oral format provides an excellent opportunity for students to see the relationship between the grade they receive and their knowledge—a connection students can fail to make with an objective, written exam. In many respects, the oral format is less forgiving than the written format— students are given an opportunity to show what they know. If their knowledge is less thorough than they believe, they are immediately confronted with their ignorance. We receive far fewer complaints (virtually none) about the grades on the poster exam than about those on a written exam. We surmise that students feel they have more control over the material— they select the aspect of the module they want to emphasize and they must become “experts” on that aspect. At least 60% of the exam is on the aspects they stressed in their poster. We postulate that student comfort with the poster exam is critically dependent upon the practice poster session, which is administered a week before the graded exam. The practice exam is attended only by the instructors of the course, who critique the posters and the oral presentations. At these sessions students see the posters of other groups and roleplay the examiner. The two-step process (practice followed by exam) more accurately mimics the typical scientific process where posters are reviewed and revised before the actual presentation. The Effectiveness of the Poster Exam as a Learning Tool To assess the effectiveness of the poster exam as a learning tool, we administered two types of measures: a questionnaire on student perceptions and a concept test on ideal gases published in this Journal (13). While the results are not definitive, they do suggest that the poster exam may be a more effective learning tool than conventional written exams. Student perceptions, as reported in the questionnaire (question 2), suggest that the poster exam is a more effective learning tool than traditional written exams. These sentiments were echoed in focus groups in which students said that they feel more comfortable with the concepts. They report that they “study differently” for the poster exam and, in particular, spend more time preparing posters, practicing their presentation, and quizzing each other. The single feature students say the poster exam fails to reinforce as well as written exams is problem solving: students say they spend less time practicing problems in preparation for the exam. As another measure of effectiveness, a multiple-choice concept test, was administered to our students immediately after the gas poster exam and to students in traditional classes immediately after their written exam on gases. The cross-class comparison of the performances on the multiple-choice exam (reported in ref 12) demonstrated that the students in the integrated course performed better than the corresponding cohort in the traditional classes. This measure does not directly test the effectiveness as a learning tool of the poster exam alone, since the integrated course differs from the
Journal of Chemical Education • Vol. 77 No. 9 September 2000 • JChemEd.chem.wisc.edu
In the Classroom
traditional course in the entire presentation of gases. However, it does address the concern that students may feel less pressure because they are not taking individual written exams, and thus learn less. It is our impression, echoed by the student responses, that students actually feel more pressured by the poster exam because of the oral component.
Acknowledgments
Conclusions
1. Lloyd, B. W.; Spencer, J. N. J. Chem. Educ. 1994, 71, 206. 2. Eg¯e, S. N.; Coppola, B. P.; Lawton, R. G. J. Chem. Educ. 1997, 74, 84. 3. Nakhleh, M. B. J. Chem. Educ. 1993, 70, 52. 4. Kennedy, J. J. Chem Educ. 1985, 62, 1104. 5. Dunstan, M.; Bassinger, P. J. Chem. Educ. 1997, 74, 1067. 6. Sisak, M. E. J. Chem. Educ. 1997, 74, 1065–1067. 7. Rigeman, S. J. Chem. Educ. 1998, 75, 727–730. 8. Romey, W. D.; Elberty, W. T. Jr. J. Geol. Educ. 1983, 31, 179–183. 9. Gore, P. A. Teach. Psychol. 1987, 14, 243–244. 10. Billington, H. L. J. Biol. Educ. 1997, 31, 218–220. 11. Baird, B. N. Teach. Psychol. 1991, 18, 27–29. 12. Mills, P.; Sweeney, W. V.; Marino, R.; Clarkson, S. J. Chem. Educ. 2000, 77, 1161–1165. 13. Cornely-Moss, K. J. Chem. Educ. 1995, 72, 715–716.
The poster exam provides a valuable and viable alternative assessment measure to the traditional, in-class written exam. Students acquire oral skills required of a practicing scientist. In addition, they learn to organize, summarize, and represent data. The poster exam is well received by the students and it may be a more effective learning tool than the standard written exam. It offers an alternative opportunity to the written exam for students to demonstrate excellence in their study of science. This may be particularly important in a nonresidential urban setting with a diverse student population, such as Hunter College. That different students excel on a poster exam from those who excel on a written exam creates a more inclusive environment and, we hypothesize, will contribute to the retention of students in science.
We would like to gratefully acknowledge support from the National Science Foundation (DUE-9555202) and the Department of Education (P116B50062). Literature Cited
JChemEd.chem.wisc.edu • Vol. 77 No. 9 September 2000 • Journal of Chemical Education
1161
