Evaluation Tools To Guide Students' Peer-Assessment and Self

Research: Science and Education

Evaluation Tools To Guide Students’ Peer-Assessment and Self-Assessment in Group Activities for the Lab and Classroom Thomas J. Wenzel Department of Chemistry, Bates College, Lewiston, ME 04240; [email protected]

It is increasingly common for chemistry instructors to incorporate group learning activities into their courses (1). Examples of the use of cooperative or collaborative group learning in general (2–12), analytical (13–27), biological (28– 29), inorganic (30), organic (31–36), and physical (37–39) chemistry courses have been described, although this is by no means an exhaustive list of all of the published examples on the use of group learning methods in the laboratory or classroom components of chemistry courses. In the aggregate, these—coupled with other assessment studies (40–42)—show that cooperative and collaborative group learning often leads to enhancement in student achievement and a reduction in attrition rates from courses. Furthermore, group activities have the ability to promote other learning outcomes such as teamwork, leadership, communication, problem solving, and

Textbox 1. Sample Group Processing and Individual Performance Questionsa

Group Process • Did everyone in your group contribute to the activity today? If so, explain how; if not, identify what individuals need to do to assure participation by all. • Did everyone in your group understand the material covered in the activity today? If so, explain how your group assured that everyone understood; if not, identify what your group needs to do to assure that everyone in the group understands the material. • Identify three things your group could do to work more effectively and efficiently. • Identify one contribution made by each member of your group to today’s activity. • What constructive feedback can you give another group member on application of a skill? • Develop a short-term plan and a long-term plan to strengthen your team’s performance.

Individual Performance • Identify three ways you and the other group members have modified or could modify study habits and strategies in order to improve performance on examinations. • List two strengths and two improvements in reference to yourself in today’s workshop. • Cite two examples of how you carried out your team responsibility or role. a

182

Adapted from ref 8.

Journal of Chemical Education

•

critical thinking skills. These other skills are highly valued by industrial employers of chemists (43). Formative and Summative Assessment of Students Engaged in Group Activities A question that arises with group activities is how to assess student performance. This not only involves assessment of academic achievement, and whether to incorporate group and individual components into the grade, but the assessment of group performance and the other skills that can be developed through group activities. Formative assessment of the workings of a group is useful because it will allow the instructor to intervene with groups that are exhibiting dysfunctional behavior. The importance of group processing is stressed in the literature on cooperative learning (42). Simply expecting all students to function well when working in groups is overly optimistic. Most undergraduates do not have much experience working as part of a group or team, and there is always the potential for personalities or styles to clash. Group learning is more effective when the groups examine their performance to identify what is working well and what needs improvement. Summative assessment is important because fully appreciating the contribution of each individual to the success of a group project can be difficult, especially if students undertake activities outside of the class or laboratory (e.g., literature searches, project design and development, data analysis, report writing, problem solving) or have access to the laboratory at hours when an instructor may not be present. Submission of a group report creates uncertainty about whether everyone contributed equally to the written document. One strategy to address some of these concerns can involve having each student submit an analysis of the laboratory data on a weekly basis as a way of keeping everyone on task (37). Another is to ask groups to provide a list of tasks that need to be done to complete the project with each task specifically assigned to a group member (10). In role-playing laboratory experiences, where one individual is assigned as a group leader or manager, a management interview at the completion of the project can be used to assess how well the group functioned (25). When group activities or assignments are used, a component of the grade ought to reflect each student’s contribution to the functioning and outcomes of the group. A recent article highlighted the importance of providing performancerelated feedback to students (44). Since group work ought to promote development of teamwork and leadership skills, students should be provided feedback on their performance. Coupling peer- and self-assessment with the instructor’s observations can be an effective way to determine an individual’s

Vol. 84 No. 1 January 2007

•

www.JCE.DivCHED.org


contribution to group work. The information can be used to assign credit and provide feedback. Using peer- and self-assessment for feedback is easier to implement than peer-grading, although incorporating the outcomes of peer- and self-assessment into the grading helps ensure that students take the process more seriously (45). Research on peer- and self-assessment supports its use as a tool that improves learning (45). Peer- and self-assessment has the potential to promote critical thinking, increase students’ commitment to the course, and help students develop their own self-assessment skills. In addition, peer- and selfassessment can increase self-confidence, responsibility, and an awareness of group dynamics. Studies conclude that peer- and self-assessment is useful, reliable, and valid, provided the instructor ensures that no cartels have formed within a group (45). Many of the published reports that describe the use of group learning activities in chemistry courses note the utilization and importance of peer- and self-evaluation, although fewer of these reports provide specific information about the actual instruments that are used to conduct the peer- and self-evaluation. What follows is a compilation of a number of these methods as well as a discussion of my own use and experience with peer- and self-evaluation. Selecting those components of the assessment instruments that others have devised should enable instructors who use group activities to construct a suitable peer- and self-evaluation instrument for their own courses. Procedures for Incorporating Peer-Assessment and Self-Assessment in the Lab and Classroom One general approach to group processing and peerevaluation is to provide open-ended questions as guidance for the students to respond to. For example, students could be asked to provide a self-assessed grade for their own participation in a group session and to provide the rationale for that grade (6). Textbox 1 outlines a series of questions that a group could use for collective and individual assessment of performance (8). Other instructors report the use of similar questions that ask the students to identify strengths, areas that need improvement, behaviors that need to be stopped, and a plan for effecting change within the group (6, 39, 46). Usually each student is expected to answer these questions as they relate to the student’s own contributions, and the entire group is expected to collectively discuss and address the questions. Textbox 2 provides a set of questions that could be used to evaluate contributions to a laboratory project (36). In this example, one student was designated as the coordinator for the experiment. In the peer-evaluation, the coordinator evaluates the contributions of each group member to the experiment and each group member evaluates the effectiveness of the coordinator. An alternative to offering open-ended questions is to give students a set of specific criteria that they then numerically evaluate. List 1 provides criteria that could be used for peerand self-assessment of a laboratory project (45). A numerical scale (i.e., 5 = outstanding; 4 = very good, 3 = good, 2 =

www.JCE.DivCHED.org

•

Textbox 2. Questions for Peer Evaluation of a Group Laboratory Projecta

Evaluation of Group Members by the Coordinator • Was the student prepared to do the experiment? • Was he or she able to independently set up and conduct the experiment or was assistance required? If so, what level of assistance was necessary? • Did the student contribute to the group in completing the worksheet and experiment assessment form? As a percentage of the total group, how much did this student contribute to completing the laboratory exercise? • What grade (i.e., A, B, C, etc.) has this student earned for the work he or she did on this lab exercise?

Evaluation of the Coordinator by the Group Members • Was the coordinator prepared? • Were the experimental procedures explained clearly and understandable? Did you understand how to do the experiment based on the coordinator's explanation? Was the coordinator able to help you with the actual setup and execution of the laboratory experiment? • Were theoretical aspects of the experiment explained well? Were the coordinator’s explanations accurate? Were there any factual errors in his or her explanation? • Did the coordinator address questions from the group? If so, were they answered effectively? Did the coordinator rely on other group members to present the experiment or answer questions posed by the group? • Did the coordinator speak clearly and in an organized fashion? Did the coordinator use charts, graphs, or diagrams to help explain the experiment? If so, were they useful? • What grade (i.e., A, B, C, etc.) has this student earned for the work he or she did on this lab exercise? a

Adapted from ref 36.

List 1. Criteria for Peer- and Self-Assessment of a Group Laboratory Project a. Gathering preliminary background literature b. Helping to develop and write the project plan c. Attendance d. Undertaking a fair share of the work e. Ability to generate good ideas and solve problems f. Ability to arrive at consensus and overcome difficulties g. Ability to facilitate the group’s efforts h. Contribution to the final written report i. Contribution to the final oral presentation


•


183


Textbox 3. Survey for Peer Assessment of Group Worka Most of our meetings were confused

1 2 3 4 5

Most of our meetings were well organized

We often got side-tracked

1 2 3 4 5

We stuck to the task most of the time

We didn’t listen to each other

1 2 3 4 5

We did listen to each other

Some talked too much and some did not talk enough

1 2 3 4 5

We all contributed to the discussion

We did not think through our ideas sufficiently

1 2 3 4 5

We thought through our ideas well

Some got aggressive and some got upset

1 2 3 4 5

We were able to argue and discuss without rancor

Most of us seemed to be bored by the discussion

1 2 3 4 5

Most of us seemed to enjoy the discussion

Most of us did not improve our discussion skills

1 2 3 4 5

Most of us did improve our discussion skills

Most of us did not learn much

1 2 3 4 5

Most of us did learn through our group work

How could the group have worked better? aReprinted

with permission from ref 45.

List 2. Criteria for Peer- and Self-Assessment of Group Classroom Activities a. Ability to communicate with the other members of the group b. Ability to remain on task c. Willingness to help the group achieve its goals d. Willingness to cooperate and follow through on group expectations and responsibilities e. Ability to listen to others f. Willingness to share resources, materials, ideas, and tasks

satisfactory, 1 = unsatisfactory) could be included and space for comments on the individual items should be provided. Textbox 3 provides an instrument for assessing students’ involvement in group exercises in class (45). Using this or a similar instrument, students could complete a peer- and selfassessment of the members of a classroom group (47). List 2 suggests criteria that could be used to evaluate students participating in group classroom activities. Students could determine a numerical rating for each item for each group member (and themselves) and provide clarifying comments. Textbox 4 outlines criteria with corresponding questions to guide peer evaluation of students for in-class activities, including those with assigned roles (e.g., chair, secretary) (35). Each group must reach consensus on an overall group ranking

Textbox 4. Criteria for Peer Evaluation of Group Classroom Activities a

Group Instructions As a group, rank the following seven criteria for peer evaluation by assigning points to each. This ranking must be the consensus of the group. Each of these criteria is to be addressed.

Assessment Criteria

Questions for Consideration

a. Degree of preparation

Did this student make a serious effort?

b. Contribution to group discussions

Did this student speak up or never talk?

c. Attitude

Was this student willing to teach and to learn in the group? Did this student want the group as a whole to make progress?

d. Attendance

Did this student come to “official” group meetings or partial-group meetings?

e. General knowledge of chemistry

Did this student demonstrate adequate chemistry knowledge relative to your group?

f. Service as secretary

Did this student keep notes accurately and fairly? Specify the approximate number of times served.

g. Service as chair

Did this student set the agenda for the meeting? Did this student effectively direct the meeting? Specify the approximate number of times served.

a

184

Adapted from the supplemental materials of ref 35.


•


•

www.JCE.DivCHED.org


of the assessment criteria and assign points to each criterion. Each group member individually submits a confidential evaluation of other group members to the instructor. At the University of Kansas, students complete a peerand self-assessment of the group laboratory projects in the instrumental analysis course (22). The set of criteria on which each student is evaluated (which was not provided in the cited publication) is included in List 3. An essential component of any peer- and self-evaluation is feedback from and processing with someone in an instructional capacity. The instructor must help the students interpret the peer- and self-evaluation and provide guidance aimed at improving the functioning of individual and group performance. Some instructors who use peer-evaluation have noted that students are initially reticent to express criticism of their peers and are overly generous in their evaluation (16, 36). Participation of the instructor in a discussion of the evaluations and repetitive use of peer-evaluation as the term progresses improves the quality of the student responses and feedback. After group members complete a survey such as those provided above, someone in an instructional capacity should examine the responses and discuss them with the group. This exercise helps students develop an awareness of group skills and improves group effectiveness. In the laboratory program at the University of Kansas, the group leader, who is either a teaching assistant or faculty member, couples the information from the peer evaluation (List 3) with her or his own observations and meets with each student in the group to provide a performance evaluation (22). In the analytical courses at Emory University, the students keep self-evaluation logs (16). These are then read aloud and discussed in the group with a facilitator present. I now use peer- and self-assessment instruments comparable to those in Lists 1 and 2, and Textbox 3, in my introductory general chemistry course (12) and upper-level analytical chemistry course (13, 14) to assess group laboratory projects and cooperative learning activities in the classroom. Groups are assigned by me on the second day of class after I have gathered information about each student’s major, year of study, and course background. Groups are made as heterogeneous as possible with regards to gender, race, and academic background. Group size is 4–5 students in my introductory course and 3–4 in my upper-level courses, and the groups work together over the entire term. Laboratory groups involve four students in the introductory course and two or three in my analytical chemistry course. Laboratory groups work together for the entire semester on their project. I spend a substantial portion of the first class describing the group-learning process and my expectations for students working within a group. I also explain to the students why it is important for us to undertake peer- and self-assessment, and how I will couple the student evaluations with my own observations to look for consistency. I first incorporated peer- and self-assessment in a summative manner for the laboratory projects as a way of determining the contributions of individual students to the project. Even though I had my personal observations in the laboratory and a sense of each student’s contributions, there

www.JCE.DivCHED.org

•

List 3. Evaluation Areas for Staff Assessment of Studentsa

Technical • Level of technical knowledge demonstrated • Level of activity to enhance technical knowledge • Quality of laboratory work and data • Quality of data interpretation

Professional • Input into project direction • Ability to work in a team framework • Quality of written reports • Quality of oral reports

Personal • Evidence of leadership • Contribution to project team • Attendance • Attitude toward work

General Comments a Adapted from the University of Kansas course, Chemistr y 636: Problem-Based Learning in Instrumental Analysis. See the Performance Evaluation Page at http://www.chem.ku.edu/GWilsonGroup/ Chem_636/Packet/performance_evaluation.htm (accessed Oct 2006).

are a considerable number of activities related to the completion of the project that occur outside of laboratory. Furthermore, in my upper-level course, the laboratory is open to the students at off-hours, including evenings and weekends, so my observations of the student’s activities are incomplete. While these summative evaluations indicated that most of the groups functioned effectively with a fairly equitable contribution from all members, there were some groups where the peer- and self-evaluations indicated that an individual or two did not contribute equitably to the project. Using appropriate questions from this instrument as a formative assessment halfway through the course has allowed me to better identify groups with problems. If the situation is one in which an individual is not fulfilling his or her responsibilities to the group, I meet with this person to describe my expectations for group work and the improvements I hope to see over the remainder of the semester. This has almost always had the desired outcome, especially since these individuals have usually been honest in their self-evaluations, indicating that they have not contributed to the extent that is expected. I also make a point of informing the diligent students in the group that their efforts are appreciated. If the evaluations indicate that the group as a whole is experiencing difficulty working together, I meet with the entire group to discuss the situation and explore ways to overcome the difficulties. The discussion, coupled with attention from the


•


185


instructor to the situation, has always led to improvement in performance. Similarly, conducting a formative self- and peerevaluation with the in-class groups has enabled me to better identify groups that are not functioning well and to intervene as appropriate. Conclusions I have found through course surveys that the students appreciate the chance to provide such evaluations, especially in the situation in which they have a laboratory partner who is not fulfilling his or her responsibilities on the project. I also find that the students are honest in assessing their own contributions, as well as their peers’, to group work, as evidenced by the self- and peer-criticism that comes across in the ratings and response to the questions. The use of peerand self-evaluations allows me to provide better feedback to the students and to feel more confident in assigning each individual a grade for their contribution to the group laboratory project. Literature Cited 1. Pienta, N, J.; Cooper, M. M.; Greenbowe, T. J. Chemist’s Guide to Effective Teaching; Pearson Prentice Hall: Upper Saddle River, NJ, 2005; pp 117–128, 155–171. 2. Selco, J. I.; Roberts, J. L., Jr.; Wacks, D. B. J. Chem. Educ. 2003, 80, 54–57. 3. Shibley, I. A., Jr.; Zimmaro, D. M. J. Chem. Educ. 2002, 79, 745–748. 4. Oliver–Hoyo, M. T.; Allen, D.; Hunt, W. F.; Hutson, J.; Pitts, A. J. Chem. Educ. 2004, 81, 441–448. 5. Wimpfheimer, T. J. Chem. Educ. 2004, 81, 1775–1776. 6. Kovac, J. J. Chem. Educ. 1999, 76, 120–124. 7. Farrell, J. J.; Moog, R. S.; Spencer, J. N. J. Chem. Educ. 1999, 76, 570–574. 8. Hanson, D.; Wolfskill, T. J. Chem. Educ. 2000, 77, 120–130. 9. Wright, J. C.; Millar, S. B.; Kosciuk, S. A.; Penberthy, D. L.; Williams, P. H.; Wampold, B. E. J. Chem. Educ. 1998, 75, 986–992. 10. Juhl, L. J. Chem. Educ. 1996, 73, 72–77. 11. Mills, P.; Sweeney, W. V.; Marino, R.; Clarkson, S. J. Chem. Educ. 2000, 77, 1161–1165. 12. Wenzel, T. J. In Gender, Science and the Undergraduate Curriculum: Building Two Way Streets; Association of American Colleges and Universities: Washington, DC, 2001; pp 29–46. 13. Wenzel, T. J. Anal. Chem. 1998, 70, 790A–795A. 14. Wenzel, T. J. Anal. Chem. 1995, 67, 470A–475A. 15. Williamsson, V. M.; Rowe, M. W. J. Chem. Educ. 2002, 79, 1131–1134. 16. Ram, P. J. Chem. Educ. 1999, 76, 1122–1126. 17. Allison, J. J. Chem. Educ. 2001, 78, 965–969. 18. Houghton, T. P.; Kalivas, J. H. J. Chem. Educ. 2000, 77, 1314– 1318.

186


•

19. Jackson, P. T.; Walters, J. P. J. Chem. Educ. 2000, 77, 1019– 1025. 20. Wright, J. C. J. Chem. Educ. 1996, 73, 827–832. 21. Fitch, A.; Wang, Y.; Mellican, S.; Macha, S. Anal. Chem. 1996, 68, 727A–731A. 22. Wilson, G. S.; Anderson, M. R.; Lunte, C. E. Anal. Chem. 1999, 71, 677A–681A. 23. Hope, W. W.; Johnson, L. P. Anal. Chem. 2000, 72, 460A– 467A. 24. Thompson, R. Q.; Edmiston, P. L. Anal. Chem. 2001, 73, 678A–684A. 25. Walters, J. P. Anal. Chem. 1991, 63, 977A–985A; Anal. Chem. 1991, 63, 1077A–1087A; Anal. Chem. 1991, 63, 1179A– 1191A. 26. Juhl, L.; Yearsley, K.; Silva, A. J. J. Chem. Educ. 1997, 74, 1431–1433. 27. Ross, M. R.; Fulton, R. B. J. Chem. Educ. 1994, 71, 141– 143. 28. Peters, A. W. J. Chem. Educ. 2005, 82, 571–574. 29. Craig, P. A. J. Chem. Educ. 1999, 76, 1130–1135. 30. Dunn, J. G.; Phillips, D. N. J. Chem. Educ. 1998, 75, 866– 869. 31. Bradley, A. Z.; Ulrich, S. M.; Jones, M., Jr.; Jones, S. M. J. Chem. Educ. 2002, 79, 514–519. 32. Browne, L. M.; Blackburn, E. V. J. Chem. Educ. 1999, 76, 1104–1107. 33. Davis, D. S.; Hargrove, R. J.; Hugdahl, J. D. J. Chem. Educ. 1999, 76, 1127–1130. 34. Paulson, D. R. J. Chem. Educ. 1999, 76, 1136–1140. 35. Hagen, J. P. J. Chem. Educ. 2000, 77, 1441–1444. 36. Hass, M. A. J. Chem. Educ. 2000, 77, 1035–1038. 37. Deckert, A. A.; Nestor, L. P.; DiLullo, D. J. Chem. Educ. 1998, 75, 860–863. 38. Towns, M. H.; Kreke, K.; Fields, A. J. Chem. Educ. 2000, 77, 111–115. 39. Hinde, R. J.; Kovac, J. J. Chem. Educ. 2001, 78, 93–99. 40. Spencer, J. N. J. Chem. Educ. 1999, 76, 566–569. 41. Bowen, C. W. J. Chem. Educ. 2000, 77, 116–119. 42. Johnson, D. W.; Johnson, R. T.; Smith, K. A. Cooperative Learning: Increasing College Faculty Instructional Productivity, ASHE–ERIC Higher Education Report, Vol. 20, No. 4.; The George Washington University, Graduate School of Education and Human Development: Washington, DC, 1991; ED347871 43. Kerr, S.; Runquist, O. J. Chem. Educ. 2005, 82, 231–233. 44. Brooks, D. W.; Schraw, G.; Crippen, K. J. J. Chem. Educ. 2005, 82, 641–644. 45. Brown, G.; Bull, J.; Pendlebury, M. Assessing Student Learning in Higher Education; Routledge: New York, 1997; pp 170– 184. 46. Towns, M. H. J. Chem. Educ. 1998, 75, 67–69. 47. Weber, E. Student Assessment That Works: A Practical Approach; Allyn and Bacon: Boston, 1999; pp 140, 228–229.


•

www.JCE.DivCHED.org

Evaluation Tools To Guide Students' Peer-Assessment and Self

Recommend Documents