Development and Application of a Scoring Rubric for Evaluating

Aug 23, 2013 - ABSTRACT: The purpose of this study was to develop a scoring rubric to assess students, manipulation skills and identify students, lear...
0 downloads 7 Views 1MB Size
Download PDF
Article pubs.acs.org/jchemeduc

Development and Application of a Scoring Rubric for Evaluating Students’ Experimental Skills in Organic Chemistry: An Instructional Guide for Teaching Assistants Hui-Jung Chen,†,‡ Jui-Lin She,*,§ Chin-Cheng Chou,∥ Yeun-Min Tsai,§ and Mei-Hung Chiu*,† †

Graduate Institute of Science Education, National Taiwan Normal University, Taipei City 11677, Taiwan Office of Research and Development, Higher Education Evaluation and Accreditation Council of Taiwan, Taipei City 10644, Taiwan § Department of Chemistry, National Taiwan University, Taipei City 10617, Taiwan ∥ Department of Science Education, National Taipei University of Education, Taipei 106, Taiwan ‡

S Supporting Information *

ABSTRACT: The purpose of this study was to develop a scoring rubric to assess students’ manipulation skills and identify students’ learning difficulties in conducting organic chemistry experiments. In constructing the scoring rubric, we first analyzed the skills needed in the experiment, then divided the skills into subskills, and finally constructed a rubric for each subskill. We applied this developed instrument to on-site evaluation of 55 second-year students by eight teaching assistants (TAs). The data were analyzed using one-way analysis of variance, which compared the students’ mean score on each skill at the 0.05 significance level. Results showed that the skill of gravity filtration and suction filtration needed to be improved, as the students had the lowest mean scores in these two skills. On-site observations further indicated that only 5% of the students correctly performed the rinsing technique for gravity filtration. We also found that TAs became more aware of students’ experimental skills after using the rubric for assessment. By using a scoring rubric as an assessment tool, students’ learning difficulties could be easily identified and TAs could improve their teaching effectiveness by checking student performance and modifying their instructional strategies accordingly. This study provides a valuable and easy-to-apply quantitative assessment for use in the organic chemistry laboratory. KEYWORDS: Second-Year Undergraduate, Organic Chemistry, Laboratory Instruction, Testing/Assessment, Synthesis, TA Training/Orientation, Chemical Education Research FEATURE: Chemical Education Research

H

promote scientific competence by providing opportunities to observe scientific phenomena, conduct hands-on activities, and develop inquiry and problem-solving skills. As such, laboratory work becomes an important extension of the regular classroom. By developing various assessments, the learning outcomes and the alignments of laboratory goals, instructional strategies, and student performance can be studied.2,3,8 As a result, accurate assessment of student laboratory performance is critical and is a major focus of current research. Assessment of laboratory work can be used to (i) align student performance with laboratory goals; (ii) align instructional strategies with laboratory goals; and (iii) adjust instructional strategies based on student performance (see Figure 1).

ands-on experiences in laboratories have long been recognized for their importance in science education.1,2 However, accurate and informative assessment of students’ laboratory skills is still a challenge.2−4 Adding to the difficulty is the fact that most laboratory courses in large research universities are instructed and assessed by teaching assistants (TAs) who often have insufficient training and many students to assess from large class enrollments.5,6 This study is designed to improve this situation by developing an assessment instrument, the scoring rubric, to precisely identify what students have properly learned and not learned in their chemistry laboratory. Each rubric clearly describes the requirements for experiment skills and subskills and is used for evaluating specific lab skills as well as general lab performance.

Align Student Performance with Laboratory Goals

■

It is important to evaluate student performance according to curriculum goals. Such assessment could check the degree to which educational goals are being met, and help with

THEORETICAL FRAMEWORK Laboratory investigations offer important learning opportunities for students to connect science concepts and theories.7 One of the purposes of laboratory work is that these experiences © 2013 American Chemical Society and Division of Chemical Education, Inc.

Published: August 23, 2013 1296

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302

Journal of Chemical Education

Article

identified which specific skills were developed and which skills remained undeveloped in our student sample. In this study, the scoring rubric was developed and validated to assess the laboratory skills of students while conducting chemistry experiments. This process is described, as are the results from implementing a rubric with 55 second-year students studying organic chemistry.

■

METHOD In this section, we describe the characteristics of the laboratory course and content of the experiments, as well as the stages of development and implementation of the rubric for assessing the students. Laboratory Course and Content of the Experiment

A laboratory reform for students majoring in chemistry was carried out recently in the university investigated in this study. The basic idea of this reform was to integrate the classical stand-alone general, organic, and analytical chemistry lab courses into one series of laboratory courses. Basic analytical and organic chemistry techniques were integrated into the first two courses in the first year. In the second year, more advanced experiments were designed. These experiments integrated instrumental analysis and multistep organic synthesis. By breaking the boundaries of the classical lab courses, the reform efforts intended to better acquaint students with current chemical technologies.24,25 This study investigated students’ laboratory performance in the second-year course. The aniline synthesis experiment was chosen for evaluation. The experiment is a multistep organic synthesis experiment.26 Several important organic laboratory skills are required in the synthesis. This experiment was usually executed at the end of the second semester of the laboratory session. On-site evaluation was conducted during the experiment to see how students performed the experimental skills they had been taught during the prior and current semesters. Each student conducted the experiment individually. The experiment started with the reduction of nitrobenzene, followed by acylation, nitration, and hydrolysis to obtain pnitroaniline, which cannot be obtained from nitrobenzene directly. The experiment was separated into two parts (see Figure 2), and each part took four hours to complete.

Figure 1. The relationship among goals, instructional strategies, student performance, and assessments.

curriculum-related decision making. This is especially relevant given the current climate of curriculum-based reforms.9 For example, important laboratory goals include helping students to observe phenomena, enhancing students’ hands-on abilities, and developing students’ problem-solving and inquiry skills.10,11 Assessments should be designed to align with these objectives in order to provide evidence related to how well each student has achieved these objectives.12−17 The rubric presented here could be one of the assessments available to instructors wishing to evaluate technical skills in the laboratory. Align Instructional Strategies with Laboratory Goals

Most university-level introductory laboratory courses are instructed by TAs. They play a predominant role in undergraduate laboratory education.18−20 Most TAs make decisions pertaining to instruction and assessment with little experience, support, or guidance from faculty and staff.21 The scoring rubric presented here is an assessment instrument that could help TAs adjust their instruction based on laboratory goals. In curriculum reform, it is especially important for teachers to realize the new goals of the reformed curriculum in order to implement the curriculum successfully. In addition, TAs are unlikely to be as well informed about reform-based changes to curriculum goals as university faculty. As such, scoring rubrics could help TAs examine the effectiveness of their instruction with regard to reform efforts and new objectives.

Design, Validation, and Implementation of the Scoring Rubric

Three stages were involved (see Figure 3) in the development of the scoring rubric for organic chemistry laboratory work.

Adjust Instructional Strategies Based on Student Performance

Most TAs receive limited feedback from faculty or administration on their instruction and performance in the laboratory.22,23 The major and sometimes sole source of feedback about TA teaching effectiveness is from student evaluations at the end of each semester. The use of scoring rubrics allows TAs to observe specific skills that are required for conducting laboratory work. Administering rubrics allows TAs to quickly learn whether students have mastered specific lessons or particular content, and the administration of multiple rubrics demonstrates student learning over time. In addition, with specific descriptions of goals and levels of performance, scoring rubrics can guide TA instruction and help TAs effectively modify their own teaching strategies. Based on the research issues discussed above, two goals were identified in this study. First, we introduced a systematic method to assess students’ laboratory skills. Second, we

Figure 2. The preparation of aniline and its derivatives. 1297

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302

Journal of Chemical Education

Article

Taiwan. The evaluators included an experienced chemistry lecturer (the second author, J.-L.S.) and two full-time TAs each with a Master’s degree in chemistry. After the pilot study, the rubric was discussed and revised by the authors. (See the final version in the Supporting Information.) The Cronbach α value for inter-rater reliability was 0.892.29 Rubric Implementation. The on-site evaluation was conducted in the chemistry department of a research university, which was ranked in the top 1% of its kind in Taiwan. The group of participants consisted of 55 chemistry majors in their second year. The students were separated into four sections with approximately 14 students in each section. All the students worked individually. One group performed the experiment at a time, but all experiments were conducted in the same lab. Enough TAs were available so that no TA had to observe more than two students. The same group of eight TAs was used to observe all four groups of students. Students were scored based on strict definitions of each subskill as shown in Table 1. For example, in Table 1, S5‑1, the appropriate funnel size means the upper rim of the folded filter paper should not extend over the rim of the funnel. The participant students in the main evaluation were assessed by our evaluators, who were full-time TAs with at least one year of chemistry laboratory teaching experience and who were familiar with the process and skills of this experiment. Training sessions were held to make sure all evaluators understood the descriptions in the rubric and to reach consensus for eliminating any inconsistency. Any irregular operation not listed in the rubric that appeared during on-site evaluation was required to be recorded for later discussions. The data were analyzed with the Statistical Package Social Sciences software (SPSS version 18.0); p values < 0.05 were considered to be statistically significant differences.

Figure 3. The three stages of rubric development and application.

Rubric Design. The original scoring rubric was designed by a chemistry professor (one of the authors, Y.-M.T.), who has been teaching organic chemistry for over 20 years. The rubric draft was further revised by all the authors. Eight basic manipulative skills were identified to be learned in the aniline synthesis experiment, including ref lux; direct steam-distillation; extraction; drying the extract; gravity f iltration; vacuum concentration; suction f iltration; and recrystallization. By referencing the handbooks for chemical technicians,27,28 each manipulative skill was further divided into several subskills. Then each subskill was described by detailed operational requirements in order to build up a checklist for evaluation. For example, the skill of gravity f iltration was divided into six subskills, including choose appropriate f unnel; use of f ilter paper; set up apparatus; f iltration; rinse; and disposal of f ilter paper (Table 1). Each subskill was scored on a 0−2 scale, in which 2 indicated that requirements were all met; 1 indicated that requirements were partially met; and 0 indicated that requirements were neglected. Rubric Validation and Reliability. The scoring rubric includes the skills, subskills, and operational requirements for the laboratory work verified by two experienced organic chemistry professors from research universities in Taiwan. After the completion of the rubric, a pilot study of the scoring rubric was carried out with 12 randomly selected second-year students from the department of pharmacy in a university in

■

RESULTS AND DISCUSSION In the following sections, we present the major findings on students’ performance in the laboratory, more specifically. We discuss which skills were well developed as opposed to what skills were underdeveloped in order to shed some light on our understanding of students’ lab skills. Identifying Student Laboratory Performance

Results of student performance are shown in Figure 4 and Table 2. The mean score for each manipulative skill was calculated from the scores for each subskill listed for that particular manipulative skill, and the score for the overall

Table 1. Example of the Scoring Rubric for Skill S5, Gravity Filtration Score Assignment Conditions Subskills

2: Requirements Are All Met

1: Requirements Are Partially Met

S5‑1. Choose appropriate funnel S5‑2. Use of filter paper S5‑3. Set up apparatus S5‑4. Filtration

Choose a conical funnel; funnel is the appropriate size

Use a funnel with inappropriate size (the upper rim of the filter paper extends over the rim of the funnel)

Fold the filter paper correctly; position the filter paper properly Support the funnel by a ring clamp

Filter paper folded improperly; filter paper not seated in the proper position

Pour the solution at a proper rate (no overflowing); no liquid is dripped outside the funnel in the process Rinse the content in the filter cone with solvent twice Use tweezers to pick up the contaminated filter paper

Liquid is dripped outside the funnel in the process

S5‑5. Rinse S5‑6. Disposal of filter paper

Rinse the content in the filter cone only once

1298

0: Requirements Are Neglected Both requirements neglected; Hirsch funnel used Both requirements neglected Requirement neglected Funnel filled with so much solution that it overflows Requirement neglected Bare hands used to grasp the contaminated filter paper

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302

Journal of Chemical Education

Article

Table 2. Student Performance Results for the Aniline Synthesis Experiment Skills S1. Reflux S1‑1. Use clean and dry apparatus S1‑2. Use of extension clamp S1‑3. Use of condenser S1‑4. Set up apparatus S2. Direct Steam Distillation S2‑1. Use of extension clamp S2‑2. Set up apparatus S2‑3. Use of thermometer S2‑4. Use of condenser S2‑5. Boiling chips S2‑6. Use of heating mantle S3. Extraction S3‑1. Transfer solution S3‑2. Release pressure S3‑3. Extraction S3‑4. Separate the layers S3‑5. Handle of solvents S3‑6. Fume hood S4. Drying the Extract S4‑1. Choose appropriate container S4‑2. Add drying agent S5. Gravity Filtration S5‑1. Choose appropriate funnel S5‑2. Use of filter paper S5‑3. Set up apparatus S5‑4. Filtration S5‑5. Rinse S5‑6. Disposal of filter paper S6. Vacuum Concentration S6‑1. Manipulate the rotary evaporator S6‑2. Release pressure S7. Suction Filtration S7‑1. Set up apparatus S7‑2. Use of filter paper S7‑3. Use of water aspirator S7‑4. Suction filtration S7‑5. Rinse S7‑6. Vent the system S8. Recrystallization S8‑1. Choose apparatus S8‑2. Use of solvent S8‑3. Collect crystals S8‑4. Rinse Mean

Figure 4. Means and standard deviations of student performance on the eight skills (N = 55).

assessment was calculated from the scores of the eight skills, with mean (M) = 1.57 and standard deviation (SD) = 0.40. We identified the weaknesses and strengths in the students’ performance by comparing the means with the score of each skill. This showed that four skills were below average: S5, gravity filtration (M = 1.14, SD = 0.30); S7, suction filtration (M = 1.31, SD = 0.31); S3, extraction (M = 1.45, SD = 0.29); and S8, recrystallization (M = 1.54, SD = 0.33). Students received good scores and performed well on the other four skills: S1, reflux (M = 1.84, SD = 0.24); S4, drying the extract (M = 1.83, SD = 0.34), S2, direct steam-distillation (M = 1.74, SD = 0.25); and S6, vacuum concentration (M = 1.74, SD = 0.50). Multiple comparisons with one-way analysis of variance (ANOVA) and post hoc LSD were carried out to compare the eight lab skills. The ANOVA resultsF(7, 432) = 4.38; p < 0.01revealed significant differences between the eight skills among the students. The post hoc LSD test showed that the mean scores of two skills, namely, S5 (gravity filtration) and S7 (suction filtration), were significantly lower than the mean scores for the other skills. The mean scores of the other six skills (S1, S2, S3, S4, S6, and S8) were not significantly different from each other. The results of student performance revealed three main findings. First, students showed good practices on most of the skills. Table 2 shows that the scores for six of the eight main skills were above 1.40 (out of a possible score of 2). Further examination of the subskills shows that 26 out of the 36 subskills were above 1.40. This indicates that most of the time students demonstrated appropriate skills in the experiment. In particular, students received good scores on S1, reflux (M = 1.84, SD = 0.24), and S4, drying the extract (M = 1.83, SD = 0.34). Both scores are well above average. Second, students need improvement on skills S5 and S7. Although students showed proficiency in most of the skills, the results from the rubric showed that students needed improvement on skills S5 and S7. Results revealed that three out of the six subskills in S5 had lower than average scores. For example, 55% of the students did not support funnels by using ring clamps in S5‑3 (see Figure 5),30 89% of the students did not rinse the product with solvent at all in S5‑5, and 73% of the students used their bare hands to grasp the filter paper contaminated with residual chemicals in S5‑6. For S7, 49% of the students did not empty the water in the safety trap (S7‑4), and 55% of the students improperly rinsed the flasks in subskill S7‑5. Although S5 and S7 include commonly performed laboratory tasks that chemistry students are expected to be familiar with, the lower scores on these skills indicate that students have to practice these two important skills more.

Mean Scorea

SD

1.84b 1.84 1.92 1.85 1.76 1.74b 1.91 1.78 1.27 1.89 1.84 1.73 1.45b 1.13 1.49 1.73 1.65 1.89 0.84 1.83b 1.69 1.96 1.14b 1.85 1.53 0.87 1.85 0.16 0.55 1.74b 1.78 1.69 1.31b 1.34 1.49 1.49 1.29 0.58 1.64 1.54b 1.82 1.60 1.89 0.84 1.57c

0.24 0.42 0.30 0.45 0.47 0.25 0.29 0.46 0.97 0.42 0.54 0.56 0.28 0.47 0.57 0.53 0.55 0.46 0.57 0.34 0.63 0.27 0.30 0.40 0.63 1.00 0.40 0.50 0.90 0.50 0.46 0.69 0.31 0.82 0.88 0.79 0.57 0.71 0.78 0.33 0.47 0.49 0.42 0.83 0.40

a

Assessment scale ranged from 0 to 2: 0 = requirements are neglected; 1 = requirements are partially met; 2 = requirements are all met. N = 55. bMean scores based on subskills’ scores of the same main skill. c Mean was calculated from the scores of main skills.

Third, strengths and weakness of subskills could be easily identified. The scoring rubrics could help to study the strengths and weaknesses of students’ manipulation skills. We analyzed the scoring rubrics of subskills with mean scores below average. Students had good performance on use of extension clamp (S1‑2, M = 1.92; S2‑1, M = 1.91). In contrast, results showed that students need more instruction and practice on the following three types of skills: (i) skills related to laboratory safety; (ii) skills affecting the experimental results; and (iii) rinsing 1299

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302

Journal of Chemical Education

Article

Figure 5. A: Improper operation of filtration with one hand holding the flask and funnel instead of using an extension clamp to fix the flask. B: The correct operation showing a sturdy setup having the flask fixed by a clamp.

Figure 6. A: Improper operation of suction filtration with the hand holding the filtration assembly instead of clamping the suction tube in position. B: The correct setup with the suction tube fixed by a clamp.

subskills (see Table 3). Several volatile and hazardous chemicals were used in the experiment described in this study. It is necessary to properly use the fume hood and to avoid touching certain chemicals directly with bare hands. Over half of the students grasped the potentially contaminated filter paper with their bare hands and without using tweezers (S5‑6), and 65% of the students operated extraction in the fume hood but did not lower the sash, thus decreasing venting efficiency (S3‑6). The results showed that 27% of students set the position of the thermometer bulb too high, resulting in lower readings at the boiling point (S2‑3); 25% of students folded the filter paper improperly, affecting the efficiency of gravity filtering (S5‑2); and 22% of the students used a conical funnel instead of a Hirsch funnel to operate the suction filtration. Several rinsing steps are included in this experiment, including S3‑1, S5‑5, S7‑5, and S8‑4. However, results showed that many students (49, 89, 77, and 68% of students, respectively) improperly conducted these subskills. In addition, during the on-site evaluation, we also observed several improper operations that were not specifically listed on the scoring rubric, such as not clamping the suction tube in position; and operating the filtration over a heating mantle, which may pose a danger in the lab (see Figures 6 and 7). From the above results, we found the scoring rubric helped identify which laboratory skills the students had mastered and which specific skills required further remediation. Using this approach, educators and teachers can clearly identify students’

Figure 7. Using oversized filter paper and operating filtration over a heating mantle.

strengths and weaknesses in manipulating skills via direct observation of the students during each step of an experiment.

■

CONCLUSION

This study presents a three-stage strategy to develop and implement a chemistry laboratory scoring rubric for the on-site assessment of student performance. The rubric is a quantitative and easy-to-understand assessment tool for TAs, designed with detailed descriptions of operation requirements and levels of proficiency. Applying this tool, TAs can better understand the weaknesses and strengths of student performance in the laboratory.

Table 3. Mean Scores for the Subskills and Percentage of Students Using Improper Operations Skills

Subskills

Mean 1.27

Position of thermometer bulb was too high

27

1.13 1.49 0.84 1.53 0.87 0.16 0.55

Not rinsing the flask Not releasing the pressure before shaking Not lowering the sash down Not folding the filter paper properly Not supporting the funnel by a ring clamp Not rinsing the filter cake with solvent at all Using bare hands to grasp the contaminated filter paper

49 29 65 25 55 89 73

S7. Suction filtration

S2‑3. Use of thermometer S3‑1. Transfer solution S3‑2. Release pressure S3‑6. Fume hood S5‑2. Use of filter paper S5‑3. Set up apparatus S5‑5. Rinse S5‑6. Disposal of filter paper S7‑1. Set up apparatus

1.34

42 (22, 20)

1.49 1.49

S8. Recrystallization

S7‑2. Use of filter paper S7‑3. Use of water aspirator S7‑4. Suction filtration S7‑5. Rinse S8‑4. Rinse

Using a conical funnel instead of Hirsch funnel; size of the neoprene filter adaptor is improper Size of the filter paper is improper Misconnecting the water hoses of the water reservoir; flow rate of water is too high and flooding occurs Not emptying the water in safety-trap bottle Not rinsing the flask; rinsing once or more than three times Not rinsing the flask; rinsing only once

S2. Direct steamdistillation S3. Extraction

S5. Gravity filtration

1.29 0.58 0.84

Common Improper Operations

1300

Students, %

25 35 (20, 15) 49 77 (55, 22) 66 (44, 24)

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302

Journal of Chemical Education

■

Article

FUTURE STUDIES AND EDUCATIONAL IMPLICATIONS OF THE SCORING RUBRIC

improve the TAs understanding of students’ performance, this will also encourage students to develop precise and appropriate techniques necessary for advanced study.

Developing Scoring Rubrics for Additional Laboratory Courses

Cooperation between Science Educators and Chemistry Experts

The method for developing the scoring rubric is described in this study. This method can be applied for developing other onsite assessments, such as scoring rubrics for general, analytical, and physical chemistry experiments. The rubric’s explicit statements about what skills are expected can serve as vehicles for designing and selecting appropriate teaching materials/ activities as well as serve as the basis for designing assessment tools in the school laboratory.

The collaborative model used in this study brought science educators and chemists together. The science educators constructed the evaluation framework, rationale, and structure for the rubric used in this study, while the chemists adopted the rationale and structure of the rubric for the skills to be developed in the organic synthetic laboratory.

■

Applying the Scoring Rubric in Longitudinal Studies

ASSOCIATED CONTENT

S Supporting Information *

The scoring rubric can be used for monitoring students’ performance of laboratory skills in one laboratory session. In addition, students could build a portfolio of their scores and monitor their learning progress from a longitudinal perspective. At present, little research focuses on how one laboratory course influences other courses in the long term. This rubric, being a quantitative measurement, could be considered an avenue to conducting longitudinal studies about the impact of chemistry education.

Scoring rubric for on-site evaluation of organic manipulative skills. This material is available via the Internet at http://pubs. acs.org.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected] (J.-L.S.); [email protected] (M.-H.C.).

Improving TAs’ Teaching Effectiveness

Notes

The scoring rubric, with descriptions of standard operations and different levels of performance, provides a convenient, efficient, and effective way to help TAs conceptualize the skills students are expected to develop, empowers TAs’ knowledge and monitoring skills of laboratory work, and improves their perception of their roles in the science laboratory (both conceptual and procedural). Also, rubrics can be used as formative assessment to provide feedback for TAs to modify their instruction based on student work. As TAs play important roles in university laboratory courses, the use of structural and systematic evaluation methods can improve their teaching effectiveness and also their confidence and competence in leading meaningful and authentic laboratory work. In this research, TAs observed two students at a time, yet in a typical setting, TAs would likely have to observe 30 students in a lab period. However, we believe rubrics still offer tremendous advantages to TAs and ultimately to students. Ways of overcoming the limitations of rubrics could include videotaping the experiment and using tapes to rate the students’ lab skills, or allowing peers to conduct ratings, which would help all students learn the skills expected of them.

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS Funding for this two-year research was provided by the National Science Council in Taiwan (NSC 91-2511-S-002-004 and NSC 92-2511-S-002-019). We express our thankfulness for the students and TAs who participated in this study.

■

REFERENCES

(1) Tobin, K. G. Sch. Sci. Math. 1990, 90, 403−415. (2) Hofstein, A.; Lunetta, V. N. Sci. Educ. 2004, 88, 28−54. (3) Hofstein, A.; Lunetta, V. N. Rev. Educ. Res. 1982, 52 (2), 201− 217. (4) Hofstein, A. Chem. Educ. Res. Pract 2004, 5, 247−264. (5) Travers, P. L. Coll. Teach. 1989, 37, 147−149. (6) Herrington, D. G.; Nakhleh, M. B. J. Chem. Educ. 2003, 80, 1197−1205. (7) Tyler, R. W. Basic Principles of Curriculum and Instruction; University of Chicago Press: Chicago, 1949. (8) Kelly, A. V. The Curriculum: Theory and Practice, 6th ed.; SAGE: Thousand Oaks, CA, 2009. (9) Holme, T.; Bretz, S. L.; Cooper, M.; Lewis, J.; Paek, P.; Pienta, N.; Stacy, A.; Stevens, R.; Towns, M. H. Chem. Educ. Res. Pract. 2010, 11, 92−97. (10) Fay, M. E.; Grove, N. P.; Towns, M. H.; Lowery, S. Chem. Educ. Res. Pract. 2007, 8 (2), 212−219. (11) Bruck, L. B.; Towns, M. H. J. Chem. Educ. 2009, 86, 820−822. (12) Krajcik, J.; Mamlok, R.; Hug, B. In Education across a Century: The Centennial Volume, Corno, L., Ed.; University of Chicago Press: Chicago, 2001. (13) National Research Council. Inquiry and the National Science Education Standards; National Academy Press: Washington, DC, 2000. (14) Jalid, P. A. J. Chem. Educ. 2006, 83, 159−163. (15) Green, W. J.; Elliott, C. J. Chem. Educ. 2004, 81, 239−241. (16) Mohrig, J. R.; Hammond, C. N.; Colby, D. A. J. Chem. Educ. 2007, 84, 992−998. (17) Gaddis, B. A.; Schoffstall, A. M. J. Chem. Educ. 2007, 84, 848− 851. (18) Travers, P. L. Coll. Teach. 1989, 37, 147−149. (19) Volkmann, M. J.; Zgagacz, M. J. Res. Sci. Teach. 2004, 41 (1), 584−602.

Linking Assessments with Laboratory Goals, Instruction, and Learning

As Stufflebeam has pointed out, “the purpose of evaluation is not to prove but to improve”.31 Evaluation should play a more constructive role in chemistry education. The scoring rubric designed in this study could facilitate connections among assessment, curriculum goals, and instruction, and ultimately improve educational practice. More specifically, the scoring rubric encourages untrained or undertrained TAs to pay attention to the errors that students commonly make and also actively participate in the instruction and assessment procedures (including understanding students’ difficulties in conducting laboratory work), aligning curriculum goals, choosing effective instructional strategies to promote students’ awareness of hands-on and minds-on activity, and monitoring students’ performance. Helping TAs via direct and close observation of students’ hands-on laboratory skills not only will 1301

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302

Journal of Chemical Education

Article

(20) Demir, A.; Abell, S. K. J. Res. Sci. Teach. 2010, 47 (6), 716−741. (21) Luft, J. A.; Kurdziel, J. P.; Roehrig, G. H.; Turner, J. J. Res. Sci. Teach. 2004, 41 (3), 211−233. (22) Baiocco, S. A.; DeWaters, J. N. Successful College Teaching: Problem-Solving Strategies of Distinguished Professors; Allyn and Bacon: Needham Heights, MA, 1998. (23) Seymour, E.; Hewitt, N. M. Talking about Leaving: Why Undergraduates Leave the Sciences; Westview Press: Boulder, CO, 1997. (24) Chen, H. J.; Chiu, M. H.; Tsai, Y. M.; She, J. L. Presented at the 2006 Annual Conference of National Association of Research in Science Teaching, San Francisco, CA, April 3−6, 2006. (25) Chiu, M. H.; Chen, H. J. Alignment of students’ learning outcomes with assessment, curriculum standards, and scientific literacy. In Learning Outcomes in Science Education; Bernholt, S., Neumann, K., Nentwig, P., Eds.; Waxmann Verlag GmbH: Kiel, Germany, 2012; pp 303−340. (26) Department of Chemistry of the National Taiwan University. Experiments in Organic Chemistry; National Taiwan University Press: Taipei, 2002. (27) Shugar, G. J.; Shugar, R. A.; Bauman, L.; Bauman, R. S. Chemical Technicians’ Ready Reference Handbook, 2nd ed.; McGraw-Hill: New York, 1981. (28) Pavia, D. L; Lampman, G. M.; Kriz, G. S. Introduction to Organic Laboratory Techniques: A Contemporary Approach; Saunders College Publishing: New York, 1976. (29) The inter-rater reliability was calculated using the completed rubrics of three randomly selected students’ in the organic chemistry laboratory. Each student was scored by two TAs in 2013 to reconfirm the reliability of the rubric. We found the rubric was quite stable and reliable for use in the laboratory. (30) All the photographs used in this paper were taken during the students’ on-site experiment periods. (31) Stufflebeam, D. L.; Nevo, D. In The International Encyclopedia of Education, 2nd ed.; Husen, T., Postletwaite, T. N., Walberg, H. J., Eds.; Pergamon: Oxford, 1994.

1302

dx.doi.org/10.1021/ed101111g | J. Chem. Educ. 2013, 90, 1296−1302