Constructing the Components of a Lab Report Using Peer Review

In the Classroom

Constructing the Components of a Lab Report Using Peer Review David E. Berry* Department of Chemistry, University of Victoria, Victoria, British Columbia, Canada V8W 3V6 *[email protected] Kelli L. Fawkes Department of Chemistry, Vancouver Island University, Nanaimo, British Columbia, Canada V9R 5S5

Writing in chemistry has been used by many instructors to enhance student engagement with concepts and to raise critical thinking to a higher level (1-5). It has long been recognized that the quality of technical writing is often poor and that students lack the motivation to improve (6). An excellent annotated bibliography for references dated to approximately the turn of the century is given in the book by Kovac and Sherwood (1). Over the past decade, the Science Writing Heuristic was constructed as a teaching style to support inquiry-based lab work (7-11) in a learner-centered environment. In addition, several other writing strategies have found their niche in a lab course. Designating courses as “writing intensive” sets the tone for an analytical lab to begin by examining a standard critique of a report (12) or for introducing peer evaluation of writing in a course on organic synthesis (13). It is clear that in a lab program even the reticent student can understand the need for writing skills (1, 6). Peer review is a technique that has a rich history in the literature for writing chemistry assignments (5, 14-16) and in lab reporting (1, 3, 13, 17-19). Most recently, a system called Calibrated Peer Review (CPR) was developed that solves some of the writing assignment difficulties encountered in courses with large numbers of students (20, 21). Some initiatives have used writing consultants of different expertise, for example, a recent graduate of the course (12), a graduate student in communications (22), or an undergraduate writing expert (18). Advice to writers inevitably emphasizes the need for drafts and multiple revisions (16, 18, 22-24). As the lab course progresses through the term, multiple reworking of the same material can be cumbersome. This may lose the advantage of engagement that is created by using the experimental work as the basis for the writing exercise. An alternative method is to focus on fragments of the report in succession (25). Although second-year students could be expected to be fairly well versed in writing journal-style lab reports,1 we have found the quality and depth of the details to vary widely. Typically, students lack a wide range of experience. They rarely have seen examples of written lab reports and rely on their own perception of what is required, molded by the feedback given by past instructors. Their reports are typically penalized because of omissions rather than incorrect statements. A significant elevation of cognitive level is required in their writing (3) to prepare for the requirements of advanced courses where the consultation of literature is used more extensively. We have sought to improve our students' writing by simultaneously using a piecemeal

_

approach, peer review, and a limited draft option within our regular lab programs. These techniques have independently been identified in the literature as ways to improve writing. We implemented the changes entirely within the lab portion of the lecture-lab combination and tried not to significantly increase the course work burden of the student. This article describes our protocol to emphasize chemical writing using two different styles of lab courses. We demonstrate that a greater emphasis can be given to quality writing in a lab course without reducing experimental content. Method The context of our teaching has been in the lab components of an analytical-physical course (12-16 students) (K.L.F.) and an inorganic synthesis course (D.E.B.). The inorganic course has about 100 students in the spring term with lab sections containing a maximum of 16 students and one or two lab sections in the summer term. This course is typically taken at the end of the second year for chemistry majors or later for biochemistry majors, although it is an elective for many other students at the second year and beyond. Each of these second-year lab courses represents less than 10% of the total academic credit for one term of a full-time student. The premise for these two lab courses is that each experiment requires a written report. All reports contain a core section where data are presented and processed and brief conclusions are drawn. This ensures that the practical aspects of the lab work are not treated superficially and that some after-class reflection is required for every experiment. Since the students carry out experiments independently but share data within working groups (3-4 students), they are in a position to make meaningful comments on their own contributions to the accumulated data. In addition to this core section, each report emphasizes at least one section of the typical journal-style format, for example, an abstract, an introduction, a procedure, or a discussion. Since there is more than one experiment run in the class each week, the style of report required is dependent on the week, rather than on the experiment. The style alternates between writing an original version and critiquing a classmate's submission (excluding the core section). By the end of the term, each student will have completed all experiments and written in all of the styles, but not necessarily with the same coupling of experimental content to report style.

_

r 2009 American Chemical Society and Journal of Chemical Education, Inc. pubs.acs.org/jchemeduc Vol. 87 No. 1 January 2010 10.1021/ed8000107 Published on Web 12/18/2009

_

Journal of Chemical Education

57

In the Classroom Table 1. Rotation of Experiments and Correlation with the Report Styles in the Analytical Course Order of Experiments

1st Round Report Due Week 4 (introduction and core)

1

B, C, D

Expt B

Expt C

Expt D

2

C, D, B

Expt C

Expt D

Expt B

3

D, B, C

Expt D

Expt B

Expt C

Student Group

2nd Round Report Due Week 6 (discussion and core)

3rd Round Report Due Week 7 (critique of introduction and discussion, and core)

Table 2. Schedule for the Inorganic Synthesis Course Experiment for Student Groupsa Week

Due Date for Student Groupsa Reportb

A and B

C and D

1

1

1

Short form written during the second class

2

2

2

Abstract and introduction

3

3a

4a

Abstract and introduction (two copies of each)

Expt 3 due at start of class in week 5


4

3b

4b

Abstract and introduction (two copies of each)



5

4a

3a

Critique of one abstract and critique of one introduction



6

4b

3b

Critique of one abstract and critique of one introduction



7

5

6a

Discussion (two copies)



8

6a

6b

Discussion (two copies)



9

6b

5

Critique of one discussion



7

7

Short form written in class with procedure written in detail

10

A and B

C and D End of the class in week 2

Option of submitting a draft for instructor's feedback before due date; final version due at start of class in week 3

End of the class in week 10

a Student groups A-D are in the same lab section. b All lab reports require a core section, which includes raw data, calculated results, examples of calculations, and conclusions.

Analytical-Physical Course The analytical-physical course follows the rotational model for the analytical portion in the first half of the course (Table 1; a table of the full course schedule is given in the supporting information). In round 1, the first of the series of experiments B-D is encountered by a particular student. The student is required to write an introduction to the theory including equations (both chemical and mathematical) and a core section, which includes raw data, calculated results, examples of calculations, and conclusions. The anonymous, ungraded introductions are distributed as photocopies and are viewed by the rest of the class for use in their full analytical report in round 3. The original report (including the introduction) is graded by the instructor and returned to the author within a week of submission. In round 2, the student writes a core section and a full discussion of the experimental results. The discussion includes how the experimental results relate to the theory, possible errors and their effects, and suggestions for improvements to the method. These anonymous, ungraded discussions are again viewed by the rest of the class for use in their full report in round 3. The graded discussion is returned to the original author within one week of submission. In round 3, the construction of a full report is required. In addition to the usual data sections, the student must include one 58


_

Vol. 87 No. 1 January 2010

_

version of an introduction and one version of a discussion selected from the photocopies distributed earlier. These sections were written by classmates in rounds 1 and 2, respectively. The choice of the anonymous introduction and discussion must be explained in a short summary, with a constructive analysis of the positive aspects, the correction of any errors, and suggestions for improvement. All choices are valid; it is not meant to be their idea of the best example of each. Inorganic Synthesis Course A lockstep style is used in the inorganic synthesis course. There is a slight variation during weeks 3-9 as half of the students in a lab section (8 of 16) follow one sequence and the other half follow a complementary sequence. By the end of term, all students have completed all experiments. The schedule used in the spring and summer terms of 2008 is outlined in Table 2.2 The core section of each lab report covers the same data manipulation described for the analytical-physical course experiments, that is, to tabulate the raw data and calculate results, to provide examples of the calculations, and to draw suitable conclusions to tie the various parts of the experiment together in brief comments. The style of report for experiment 2 was created by request of the students in the first cohort, and they were given

pubs.acs.org/jchemeduc

_

r 2009 American Chemical Society and Journal of Chemical Education, Inc.

In the Classroom

the option of submitting a draft early for the instructor's comments. The students also asked for the first of these new-style piecemeal reports to be read only by the instructor. Their reason was to confirm the format before their peers read their work, albeit anonymously. The literature suggests students are likely to write with more clarity when their audience is their “knowledgeable peers” rather than the instructor (3). It is thought that students writing to their instructor tend to omit logical steps in their arguments as they assume that the instructor is already in agreement with the outcome (3). For weeks 3-6 of the term, two experiments are run simultaneously, each taking two weeks and done by half of the lab section at one time. After week 4, in addition to the core section of the report for experiments 3 and 4, there is also a requirement of (a) a one-paragraph abstract (about 6 sentences) and (b) an introduction that describes in detail the objective of the experiment and what the data are likely to prove or disprove (about 1 page). Parts (a) and (b) are submitted along with an extra copy without the student names. The latter are scanned and are made available on the protected course Web site for all members of that lab section to view. After week 6, the reports include one abstract and one introduction written by other members of the lab section in addition to the core section. A detailed review of the abstract and introduction is required, commenting on what is good about their selections, what is factually wrong or missing, and how these sections might be improved. Similarly, with two more experiments (expts 5 and 6), the requirements are modified to focus on first writing and then critiquing a discussion section instead of the abstract and introduction. The final experiment (expt 7) of the lab course is written in class as a short report. The focus is to write the procedure in detail, in contrast to the experimental reports where a reference to the written procedure is all that is required. Experiment 7 requires the student to devise the procedure using the experience gained throughout the term and some hints from specific physical properties of the reagents (26). The typical difficulty for the students is that they often write what they intended to happen rather than what actually did. Results and Discussion Prior to the revisions of these courses, students had been guided in their writing with instructions included in the lab manual (both courses) and an annotated mock report of the introductory exercise (inorganic course). This was insufficient to prepare them for the upper-level courses that require extensive use of the published literature to augment the discussion of their data. It is likely that this shortcoming stems from their previous conditioning of the form-style reporting associated with verification laboratories coupled to grading guides that reward the “correct” answer. A radical change in writing technique is not likely to be achieved in 10 weeks, but by introducing a methodical approach and integrating it with a lab syllabus, significant improvements in writing style, efficiency, and engagement can be gained. A range of techniques has been adopted so that the wide variety of learning styles can be accommodated without making the workload burdensome. The need to describe the experiment and draw appropriate conclusions acts as a motivator for the student who might be reluctant to otherwise complete a writing assignment. As Kovac and Sherwood state (1):


_

The laboratory report is a logical place to develop writing skills, but in many chemistry departments this opportunity is wasted.

The demands of the syllabus dictate the pace of new material throughout the term. This can be an advantage in that new content is always available, but it makes reworking a draft difficult to schedule. Even with a reduced number of “full” reports, Bailey and Geisler comment on the problematic timing when describing their use of writing consultants for a lab report (22), although there are clear benefits to rewriting the same piece of work (23). We offer an opportunity for one draft submission early in the term (for expt 2 of the inorganic course), but typically less than half the students use this voluntary option. Often those students who do partake seem to be getting a feel for the instructor's grading preferences rather than looking for basic guidance in writing skills. Instituting multiple rewrites is not feasible in the current structure of our courses. When constructing this writing protocol, we were mindful of both schedule and workload. The piecemeal approach has been described by Olmsted for an upper-level course (25), and we have used this concept to allow focus without increasing the workload of the student. In the analytical course, each student has one attempt at writing an introduction and a discussion on different experiments before they are required to construct a complete report. Although they do not write all of the material used in the full report, the third submission includes their critiques of others' writing. The critiques are intended to enhance reflection upon the report-writing process, while the peer reading rapidly broadens the experience. Comments in an anonymous survey were appreciative of the chance to focus on small sections at a time, especially given that some students had no experience with writing a report. There was a general agreement that a full report should be written completely by the student on one occasion3: Drawbacks were that we never did hand in a full formal lab report, so we don't have a rough estimate of how long it will take to properly do one or how much emphasis to put on each part of the lab report...

The inorganic course adopted a similarly fragmented approach, but used it in a slightly different way to match the lockstep model of delivery. The use of two-week experiments made it easier to distribute (or post to the Web site) the material for peer review in a timely fashion. The order of the emphasized section was not selected for any significance other than it was logistically convenient to have the procedure written in class in the final week. Interestingly, one respondent would have liked to have more opportunities to write each section4: ...Because we had never really done a proper intro/abstract/discussion etc., when we got our marks back most of the class got horrible marks ( and rightly so). The problem I had with this was that we never got to do another intro/abstract/discussion so that we could use the feedback we had gotten from [the instructor] to make up some of the marks we had lost on the first attempt.

It is unfortunate that time did not permit the course to culminate with a full report completely written by the student. However, we have deliberately used a second-year course for this exercise so that the compulsory third-year lab courses will afford that opportunity. The literature cites several articles where peer review has been used to aid lab report writing. Although we have had positive feedback from the students on this technique when used anonymously (17) it is not without its critics. We polled the


_


_


59

In the Classroom

students (in monthly course committee meetings and end-ofterm questionnaires) as to whether they benefit more from reading other students' work or from receiving feedback from those peers. Opinions vary but generally the reading is considered more valuable (19). Nilson cites emotional reasons why peer feedback may be compromised and recommends charging the reader with specific tasks to address (15). There has been at least one example in the literature where anonymity is not preserved (an idea that we have not explored) and deliberate interaction between author and reviewer has been part of the protocol (5). Several educators have used peer reading in the evaluative sense, providing a grading rubric to the students (13, 14, 21). We have deliberately avoided creating an atmosphere of assessment by the peer reader as we urge our students to use the reading to improve their own expression of the concepts rather than to be correcting someone else's (19). After several years of returning students' feedback to the original author, we no longer do so. In talking with our students, it quickly became obvious that there was a broad range of tactics used in the reading of peers' work. We expected that before writing a critique, some students would read all of the articles made available, while others would read considerably less4: I found [reading the discussions] to be particularly difficult. Most of [them] had made the wrong conclusions about the reaction order - in fact, only two people made appropriate conclusions. Also in general, the way in which they presented the information was confusing. I suppose I learned to make sure to examine the data carefully and draw appropriate conclusions. The importance of explaining carefully the reasons for the deductions and including appropriate factual information from literature to back up the findings was also an important point which seemed to be missed.

In the analytical course, the introductions were distributed for students to read before they did that experiment (in round 3). Some students found this was a valuable way to prepare for the lab work. The timing was not right for this to happen in the inorganic course, as the due date was the same as the day the experiment was to be performed again in the next round. Somewhat unexpectedly, some students also read the pieces written by their peers who had performed the experiment at the same time as themselves. I learned more about the experiment that we had performed ( better understanding of purpose) and I realized what I had left out.3...I found

this to be even more useful than reading abstracts and introductions written by the students in the other group.4 In this protocol there are some similarities to the Calibrated Peer Review (20), but we have chosen not to adopt that system. There is a significant investment of time initiating the Calibrated Peer Review method, and students have reported a high “hassle factor” (21). We prefer to use the material created by our students within that particular lab section for the review process as it inhibits the recycling of reviews from other sections or years, although in principle the students could review old reports since the critiques are not returned to the original authors. We believe that by critiquing current work the students are required to reflect more than they might when providing just short answers to the questions posed in CPR. The students can choose how many versions of each introduction or discussion that they read. Obviously, the minimum number is one but the maximum is 1/3 of the class size (∼5) for the analytical course and 1/2 of the class size (∼8) for the inorganic course. The students do not have access to material posted to the Web site for other sections. There 60


_


_

is some work for the course coordinator associated with checking the original writings (a duplicate copy is submitted) for lingering identifiers, and digitally scanning the documents to provide photocopies or for Web site posting. The section instructor marks the original report while the students are reviewing it, so the instructors are encouraged to keep good notes for themselves. This significantly shortens the time spent on marking in the subsequent critiquing rounds, but it does necessarily oblige them to read the results of the reviewing in detail. This constitutes important feedback to the instructor that might be lost if CPR methods were adopted (21). Anonymous feedback has been sought from the students after each of these courses was completed. Response numbers are low, which we find is typical for this kind of questionnaire. We have combined the data from various courses.3-6 All but one respondent found value in the critiquing process. Several respondents commented on how reading others' work broadened their horizons or temporarily placed them in the role of the instructor. Just over half the respondents found the critiquing process harder than writing an original version, and in equal proportions, the rest found it easier or were inconclusive4: ...It was definitely harder than writing the section yourself. Not only did you have to critique and make comments on their work but you had to basically rewrite the report as so much explanation/conclusions about the data were missing etc. particularly for the discussions. This was not so much of a problem for the abstracts and introductions.

An interesting tactic was suggested by the following comment6: ...[I]t was easier to critique the abstract and intro without having written one. [I]t was more difficult to critique a discussion without hav[ing] done one myself first. I think if I did this again I would write [a discussion] myself and then go through and find the differences and `stuff' missed by either them or myself.

The combination of the piecemeal approach and the peer review seems to work well with the logistics of both the rotational model and the modified lockstep. With appropriate attention to the workload and schedule, the writing burden need not increase from a traditional format of writing full reports for every experiment. We have fine-tuned our demands made of the students and will continue to monitor this in future iterations of this writing protocol. Students (65%) ensure us that they learned more by the described method than the traditional one and are much happier not writing full reports regularly, even though there is not any saving in time or effort. One student6 mused: I think both systems have a measure of merit. I personally would have preferred full reports but I'm sure my classmates would crucify me for saying that.

Conclusions Our students report that their writing skills have improved using this combination of piecemeal and peer-review techniques. We have shown that the logistics can be modified to suit two styles of lab courses in the rotational and modified lockstep models. We foresee a challenge in time management for the student if a lockstep model using exclusively one-week experiments were to be employed. This would drastically reduce the time for reading and reviewing. We recognize that good writing often involves multiple drafts and revisions and have opted to


_


In the Classroom

sacrifice those for the sake of moving at a regular pace through the syllabus of the course. We believe that our students draw the same depth of conclusion from processing the data of each experiment, even though they only write a fragment of the report compared to previous cohorts writing a traditional report. This reassures us that the core section requires students to explain their results of the data analysis (through tables, graphs, and calculations), whereas the other sections serve to improve their written communication skills. For the sake of prompt feedback, we have continued to assess work within one week of submission. Graded reports are shown to the authors but not released, as critiques are still being written by other members of the class at this time. We have not encountered any problems of cheating associated with this procedure. Acknowledgment The authors wish to thank the reviewers for their detailed comments that guided the revision of this manuscript. Notes 1. Typical first year students write about 12 full reports at University of Victoria and 2 full reports at Vancouver Island University. 2. Because the experiment selection changes each year at University of Victoria, the schedule will vary in different years of the course, depending on the number of weeks each experiment takes to complete. 3. Anonymous student comment (of 5 replies), Chem 221, Malaspina University-College (renamed Vancouver Island University in Sept 2008), fall 2006. 4. Anonymous student comment (of 5 replies), Chem 222, University of Victoria, summer 2007. 5. Anonymous student comment (of 14 replies), Chem 222, University of Victoria, spring 2008. 6. Anonymous student comment (of 6 replies), Chem 222, University of Victoria, summer 2008.

Literature Cited 1. Kovac, J.; Sherwood, D. W. Writing Across the Chemistry Curriculum: An Instructor's Handbook; Prentice Hall: Upper Saddle River, NJ, 2001. 2. Kovac, J.; Sherwood, D. W. J. Chem. Educ. 1999, 76, 1399– 1403.


_

3. Rosenthal, L. C. J. Chem. Educ. 1987, 64, 996–998. 4. Oliver-Hoyo, M. T. J. Chem. Educ. 2003, 80, 889–903. 5. Schepmann, H. G.; Hughes, L. A. J. Chem. Educ. 2006, 83, 1024– 1028. 6. Doody, T. C.; Gibbens, V. E. J. Chem. Educ. 1954, 31, 8–12. 7. Greenbowe, T. J.; Hand, B. M. In Chemists' Guide to Effective Teaching; Pienta, N. J., Cooper, M. M., Greenbowe, T. J., Eds.; Prentice Hall: Upper Saddle River, NJ, 2005. 8. Burke, K. A.; Greenbowe, T. J.; Hand, B. M. J. Chem. Educ. 2006, 83, 1032–1038. 9. Rudd, J. A., II.; Greenbowe, T. J.; Hand, B. M.; Legg, M. J. J. Chem. Educ. 2001, 78, 1680–1686. 10. Poock, J. R.; Burke, K. A.; Greenbowe, T. J.; Hand, B. M. J. Chem. Educ. 2007, 84, 1371–1379. 11. Rudd, J. A.; Greenbowe, T. J.; Hand, B. M. J. Chem. Educ. 2007, 84, 2007–2011. 12. Whelan, R. J.; Zare, R. N. J. Chem. Educ. 2003, 80, 904–906. 13. Hollenbeck, J. J.; Wixson, E. N.; Geske, G. D.; Dodge, M. W.; Tseng, T. A.; Clauss, A. D.; Blackwell, H. E. J. Chem. Educ. 2006, 83, 1835–1843. 14. Shibley, I. A., Jr.; Milakofsky, L. M.; Nicotera, C. L. J. Chem. Educ. 2001, 78, 50–53. 15. Nilson, L. B. College Teaching 2003, 51, 34–38. 16. Guilford, W. H. Adv. Physiol. Educ. 2001, 25, 167–175. 17. Fawkes, K. L.; Berry, D. E. Positive Pedagogy 2001, 1 (2), ISSN: 1496-8126. 18. Widstrand, C. G.; Nordell, K. J.; Ellis, A. B. J. Chem. Educ. 2001, 78, 1044–1046. 19. Koprowski, J. L. J. Coll. Sci. Teach. 1997, 27, 133–135. 20. Chapman, O. L.; Fiore, M. A. Calibrated Peer Review Home Page. http://cpr.molsci.ucla.edu/ (accessed September 2009). 21. Margerum, L. D.; Gulsrud, M.; Manlapez, R.; Rebong, R.; Love, A. J. Chem. Educ. 2007, 84, 292–295. 22. Bailey, R. A.; Geisler, C. J. Chem. Educ. 1991, 68, 150–152. 23. Dusseault, C. Secret to Good Writing? Rewriting! University Affairs; Nov 1, 2006, www.universityaffaris.ca (accessed December, 2008). 24. Zimmerman, S. S. J. Chem. Educ. 1978, 55, 727. 25. Olmsted, J. J. Chem. Educ. 1984, 61, 798–800. 26. Berry, D. E.; Fawkes, K. L; Chivers, T. Chem. Educ. 2001, 6, 109–111.

Supporting Information Available Detailed lab syllabuses for the analytical-physical and inorganic courses and grading considerations for the inorganic course. This material is available via the Internet at http://pubs.acs.org.


_


_


61

Constructing the Components of a Lab Report Using Peer Review

Recommend Documents