Transforming the Organic Chemistry Lab ... - ACS Publications

Oct 10, 2017 - taught the value in keeping a professional lab notebook but, in the ..... (9) (a) Haight, G. P. Bringing Undergraduates to the Chemical...
2 downloads 0 Views 1MB Size
Download PDF
Article Cite This: J. Chem. Educ. XXXX, XXX, XXX-XXX

pubs.acs.org/jchemeduc

Transforming the Organic Chemistry Lab Experience: Design, Implementation, and Evaluation of Reformed Experimental ActivitiesREActivities Christina G. Collison,*,† Thomas Kim,† Jeremy Cody,† Jason Anderson,‡ Brian Edelbach,‡ William Marmor,† Rodgers Kipsang,† Charles Ayotte,† Daniel Saviola,† and Justin Niziol† †

School of Chemistry and Materials Science, Rochester Institute of Technology, Rochester, New York 14623, United States Department of Chemistry and Geosciences, Monroe Community College, Rochester, New York 14623, United States

‡

S Supporting Information *

ABSTRACT: Reformed experimental activities (REActivities) are an innovative approach to the delivery of the traditional material in an undergraduate organic chemistry laboratory. A description of the design and implementation of REActivities at both a four- and two-year institution is discussed. The results obtained using a reformed teaching observational protocol are described and correlated to the transferability of REActivities between different instructors and their transportability between different institutions. KEYWORDS: Second-Year Undergraduate, Organic Chemistry, Laboratory Instruction, Inquiry-Based/Discovery Learning

■

INTRODUCTION

Despite these valid limitations, reinventing the way in which the organic lab curriculum is traditionally delivered is a vast undertaking few institutions have the time and energy to pursue; it requires more than creating a lab curriculum from an amalgam of independently constructed experiments. As such, we endeavored to address the challenges faced by traditional organic lab delivery by designing and implementing innovative and easy-to-adopt reformed experimental activities (REActivities) for the organic laboratory curriculum. Such REActivities are composed of familiar organic lab exercises that infuse some aspects of both a guided-inquiry model and a studio approach. Our REActivities teaching method transforms the commonplace lab periods into powerful lab experiences dedicated to enhancing student engagement, troubleshooting problems, and

The traditional (expository) delivery of an organic lab perpetuates a culture among students whereby the experience is devalued and the students fail to correlate the material in the lab with what is being taught in the lecture.1,2 Expository organic laboratories have a common order of operations: a prelab assignment and prelab lecture either at the start of the lab or as a separate lab lecture followed by experimental execution and a postlab writeup. Efforts to reform an organic chemistry laboratory experience using proven inquiry-based methods seem successful “only if teachers are skilled in inquiry teaching methods and students are given the time and guidance required to become comfortable with the new methods and expectations”.3,2 Given that many laboratories are taught by an ever changing cadre of graduate teaching assistants, reformed teaching methods requiring intensive training prove challenging to successfully incorporate and deliver. © XXXX American Chemical Society and Division of Chemical Education, Inc.

Received: March 29, 2017 Revised: October 10, 2017

A

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Scheme 1. Overview of REActivity Reactions Run During Second-Semester Organic Chemistry Lab

developing and honing lab skills. The REActivities are amenable to a typical 4 h lab time frame, or a flexible 3 h lab period, where final analyses may be performed the following week. To determine the efficacy of our approach, assessments were conducted at the principal institution and a local community college to evaluate the reformed nature of our pedagogy.

Despite the data published supporting the benefits of reformed methods, most organic chemistry laboratories continue instruction using traditional methods due largely to scheduling, time, cost, and novice instructional teams. For example, to adopt a studio model, the typical lecture and lab times are merged into one block connecting the material covered. However, this is a vast infrastructure change to both the faculty’s time and the institute’s resources and cannot be adopted easily unless the department and university are committed to the change.11 Should a strictly inquiry-based method be implemented, instructors should attend inquirybased training workshops in order to effectively deliver the reformed lab. The learning curve to deliver inquiry-based models is high for most instructors and often best suited for seasoned faculty.6,10,12 Recognizing the barriers to adoption of current reformed practices, REActivities incorporate certain features of both the studio model and guided-inquiry and avoid the trappings attributed to each method. In particular, REActivities do not require a large amount of formal lab instructor training and reduce the indoctrination period of the students. Additionally, REActivities are implemented during the traditional weekly 3 or 4 h time frame already in place at most institutions. Additionally, the REActivities approach minimizes the amount of formal training required by the lab instructor.

■

OBJECTIVES FOR REACTIVITIES The objective of the organic lab delivery overhaul was to strengthen the foundational learning of the technique laboratories and create a more seamless and contextual link between these techniques and the reactions run later in the semester, all the while addressing the following student learning outcomes: • Improve student engagement in the lab leading to enhanced troubleshooting skills • Improve material connectivity between the lecture and the lab (notably the technique laboratories) • Improve student confidence in the lab It was critical that the REActivities’ instructional materials were designed robustly in order to ensure facile transferability among different instructors and smooth transportability across a variety of adopting institutions to ensure that the learning outcomes can be met when REActivities are employed. As such, REActivities were designed through the construction of an organized lab workbook, are delivered during a traditional 3 or 4 h period separate from the corequisite lecture, and have both a macroscale and a microscale version depending upon the accommodations of the institution. REActivities prompt students to derive hypotheses, practice a skill without fear of failure, engage in discussion with their lab partner, and execute experiments similar to a studio format.4 Given the organization of the worksheets, prelab lecture is unnecessary allowing for more time in the lab for active learning and experimental work.5 The guided-inquiry concepts infused into REActivities structure the lab to allow students to explore the foundational underpinnings of the science governing their observations.6 Supporters of inquiry-based methods highlight the motivational value and excitement of discovering the principles for one’s self.7 As such, each REActivity technique lab has a practice time whereby the students are guided through short exercises that allow them to fail and retry a technique. The practice time develops the student’s confidence prior to beginning the actual experiment that is recorded in their lab notebook. The use of guidedinquiry in an organic chemistry laboratory has been reported.8 The literature suggests that both the students’ learning gains and interest are greater when the experiments are conducted using guided-inquiry.8−10

■

REACTIVITIES ADDRESS CURRENT CHALLENGES REActivities are administered via robustly structured worksheets that guide the student through their learning process. The material covered by our REActivities method is the same as traditional organic chemistry lab manuals. However, the REActivities delivery of the content is vastly different from traditional organic laboratories given that the collective grouping of traditional experiments often suffers from the following: • Silo experiments, no molecular continuity (each week a new set of substrates is employed) • A devalued laboratory experience • Disposal of newly synthesized compounds (students are congratulated for achieving the synthesis of a pristine white solid and then promptly instructed to throw it into solid waste) • Lack of revisiting newly acquired lab techniques (students are taught an “important” technique and rarely use the technique in a subsequent lab) • Lack of revisiting previous notebook entries (students are taught the value in keeping a professional lab notebook but, in the teaching lab, are never required to rely on their previous entries) • Lack of meaningful/rich peer-to-peer engagement (students who work independently often do not engage B

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

experience. Additionally, institutions looking to reduce lab chemical costs, and streamline organic lab chemical preps, will find great value in the adoption of this approach. (See the Supporting Information.) REActivities strategically employ the communal substrates and lab techniques used in the early REActivity laboratories. As such, the absence of repetitive details and instructions ensures that previous notebook entries and learned techniques are revisited (Table 2). For example, upon conducting the saponification REActivity, students must use their notebook entry from the Extraction Lab to fill in the missing experimental details for how to separate any remaining starting ester 3 from carboxylic acid product 4. Separation of ester 3 and carboxylic acid 4 was the exact system used when students were learning the extraction technique earlier in the first semester. REActivities are also structured to incorporate partner work that requires both peer-to-peer communication and verbiage noticeably related to the lecture material. Specifically, a postlab question is posed for each technique lab requiring the student to correlate at least two concepts from the lecture course relevant to the foundational principles of the technique. Peerto-peer communication is also ensured in the methodical design of REActivities. The successful delivery of a REActivity guides students to learn the concepts and the techniques in lab through a learning cycle that mirrors the scientific method: observe, reason, and test.13 To accomplish this, the REActivity time is designed to prompt three types of interactions: partner work, lab practice/experimental work, and group discussions as shown in the first published REActivity, SN1/SN2.2 The first type of interaction includes partner work, whereby two students discuss the foundational concepts and the setup involved for a particular lab technique or experiment. This is followed by either lab practice time or the execution of experimental wet chemistry. The third type of interaction requires the student pair to engage with a nearby group to compare their data and initiate a discussion. An overview of the delivery timetable of an expository lab compared with organic lab REActivities is mapped in Figure 1. Coverage of the material by any instructor should be thorough and balanced since the delivery of a REActivity excludes any prelab lecture by the instructor and incorporates contextually timed partner work and reflection prompts in the workbook. Stop sign icons within the workbook require the student to seek out the instructor for validation or clarification, thus removing the obligation to prepare instructors to explain certain things in certain ways and at particular points in time. In this way, the workbook serves to aid lab coordinators when overseeing instructors that are preparing to teach the laboratories. Simply, instructor training involves a review of

with nearby classmates in the lab, and those who work in pairs often work in parallel and not as a team) • A perceived disconnect between lecture and lab material (technique laboratories are the most susceptible to being labeled by the students as unrelated to the course material) • Lost opportunities to evaluate data and troubleshoot (students will often leave the lab early and analyze their data independently instead of sticking around to compare and contrast results obtained) REActivities address these limitations by taking a holistic approach to the delivery of the lab. One part of this multifaceted design is the use of a communal pool of substrates along a general pathway (Scheme 1). As such, the compounds bear a common structural motif allowing the students to more easily track changes in the structure as techniques are learned and chemistry is performed. The communal pool of substrates removes one variable from the students’ learning process when learning new techniques such as thin layer chromatography, extraction, or melting point measurement, etc. For example, a student will learn how thin layer chromatography works by using compounds 1, 2, 3, and 4. The following week, the technique of column chromatography is introduced, and the utilized molecules will be familiar since the students must perform a separation of compounds 3 and 4. Table 1 shows the Table 1. Frequency Map of REActivity Labs and the Communal Substrates Employed Substrate Reused REActivity MP Recrystallization TLC Column Extraction NMR/IR Bromination Elimination Oxidation Wittig Saponification Aldol

1

2

√ √

√ √

√

√

√

√ √ √

3

4

√

√ √ √

√ √

√ √

√ √ √

5

6

√ √

√

√ √

occurrence of each communal substrate in its relevant REActivity. Thus, molecular continuity creates a rationale for the molecule used and/or synthesized, eliminates silo experiments, and gives purpose to a material used or synthesized in subsequent laboratories, adding value to the student’s lab

Table 2. Techniques and Previous Notebook Entries Revisited by Subsequent REActivities Revisited Notebook Entry for These Techniques Current REActivity Column Extraction Bromination Elimination Oxidation Wittig Saponification Aldol

MP

Recrystallization

Column

Extraction

NMR/IR

Distillation

√ √ √ √ √

√

√ √ √

√ √ √ √ √ √

TLC

√ √

√ √ √ √

√ √

C

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

■

Article

REACTIVITIES ASSESSMENT Numerous studies have described the difficulties inherent in adopting new, reformed practices in the chemistry classroom.4,5,11,15 For those instructors who do attempt to institute reformed teaching methods, there also appears to be additional issues sustaining these reformed practices.16 One of the factors contributing to this lack of sustainable pedagogy reform is that instructors often try to adapt the pedagogy to their local conditions but end up dispensing with certain elements of the pedagogy that are essential to its success. In an attempt to ensure fidelity of implementation,17 the REActivities described herein were designed to be incorporated in a variety of organic teaching laboratories, and the student-centered nature of the activities requires little in the way of instructor intervention. Thus, the evaluation of implementing REActivities was conducted to investigate three goals: 1. REActivities are more student-centered when compared with expository laboratories. 2. The integrity of delivering REActivities is upheld among different instructors. 3. The integrity of delivering REActivities is upheld across different institutions. To investigate the student-centeredness of the REActivities in comparison to expository delivery, we used the reformed teaching observation protocol (RTOP). RTOP is an observation instrument focused on inquiry and student-centeredness.18 It is a 25-item observation protocol in which each item is scored on a Likert scale from 0 (never occurred) to 4 (very descriptive) and contains subscores that fall into categories of Lesson Design and Implementation (LDI), Content, and Classroom Culture (CC). Using RTOP scores, we expected to observe if and when instructors intervene in a way that attenuates student inquiry. To establish reliability for RTOP measurements, the same two trained student observers simultaneously scored each laboratory, both expository and REActivities, and discussed their scoring iteratively until overall scoring agreement reached 85%. Once this level of reliability was established, each observer scored all subsequent laboratory observations independently of the other as dictated by RTOP protocol.18 RTOP was used by these same two student scorers at both the principal institution (Rochester Institute of Technology, RIT) and a local community college, Monroe Community College (MCC). As shown in Figure 2, there is a measurable difference in the overall reformed levels observed between the REActivities as compared to related expository laboratories. While the ordinal scale of RTOP data is not amenable to statistical comparison of means, the general trends in the data verify a marked difference in the levels of reformed practices expected in these contrasting laboratory settings. RTOP scores greater than 50 are considered to indicate “considerable presence of ‘reformed teaching’”,19 and this threshold appears to distinguish the REActivities from the expository laboratory experiments. Furthermore, the range of REActivities RTOP scores (54− 72) indicates that the RTOP instrument has sufficient resolution to detect where changes can be made to improve a specific REActivity with regard to its student-centeredness and degree of inquiry. For example, the average RTOP scores for double solvent recrystallization lab (R3) are lower than those for the single solvent recrystallization lab (R2). It is understandable that the Content category is lower given that it is an extension of the prior week’s lab; however, Classroom

Figure 1. Delivery comparison between expository laboratories and REActivities.

the workbook questions and experimental procedures as the basis for discussion among lab instructors. As a result of the well-structured REActivities worksheets and holistic curriculum, institutions looking to simplify graduate TA/instructor training will find great value in the adoption of this novel lab curriculum. Lastly, our curriculum includes one previously published REActivity which makes use of the time during the first day of lab.14 The second is a novel REActivity that formalizes troubleshooting training and the proper considerations when analyzing data and writing conclusions. With the materials developed and implemented at both the home institution and a local community college, it was now important to ensure the transferability and transportability of the REActivities and evaluate student interactions while conducting such laboratories. D

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 2. Comparison of RTOP scores for REActivities (R1−R9) and comparable expository lab scores (E1−E11). The REActivities scores are averages of all data obtained from MCC and RIT, and taught by various instructors. The expository scores are averages of all data obtained from RIT, and taught by multiple instructors. Laboratories 1−4 of each set cover matching topics (e.g., R1 and E1: Extraction). E5−E9 were not available lab formats for observation.

Figure 3. RTOP scores for five different instructors (I−V) using REActivities.

five instructors, the primary differences correspond to variation in the instructors’ Classroom Culture category. Since Classroom Culture reflects the degree of interactivity, both among students and between students and instructors, it is not surprising that this is where individual variations would occur. Despite the variations found within the Classroom Culture scores, the overall scores indicate that REActivities are generally resistant to instructor revision and that implementation of REActivities affords instructors high potential for achieving

Culture scores show room for improvement potentially achieved through additional partner work and group discussion. We also intended to use the RTOP data to measure any potential variation of implementation across instructors. Figure 3 shows a comparison of observations across five different instructors using REActivities. While the scores for all five instructors fall within the reformed teaching range, there are distinct differences. While the categories of Lesson Design & Implementation and Content are typically consistent across the E

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

the data indicate that the amount of student-centeredness was increased and that the implementation of REActivities affords instructors high potential for achieving authentic student inquiry without requiring considerable levels of individual professional development. Third, the lack of significant variation in RTOP scores supports the expectation that the REActivities method ensures fidelity of implementation across institutions. In addition to the major findings is the notion that using the REActivities method addresses the common limitations in current organic chemistry lab delivery by adding value to the lab experience. The benefits of adopting REActivities is seen by the removal of silo experiments, the deliberate questions that link lecture and lab material, and the value added to the lab experience through the use of a communal pool of compounds, repeated techniques, and the requirement to use previous notebook entries. We hope that instructors teaching undergraduate organic chemistry laboratories will find REActivities to be a useful teaching resource and will be encouraged to convert other creative lab experiments into the REActivities format.

authentic student inquiry without requiring considerable levels of individual professional development. The ability of the RTOP instrument to detect these variations among instructors not only provides a means for establishing the transferability of REActivities among instructors, but also provides a formative assessment tool that can be used to inform instructors regarding their implementation of REActivities. To examine their transportability across institutions, REActivities were implemented at a second institution (MCC) and were measured by RTOP. The data for this institution are compared with scores at the principal institution (RIT), shown in Figure 4. The aggregate RTOP scores for each institution include five common REActivities in addition to 1− 3 unique REActivities at each individual institution.

■

FUTURE WORK Future RTOP studies are planned at additional institutions to measure the effectiveness of REActivities across a broader span of instructors and institutions. Additionally, data collected from the Meaningful Learning in the Laboratory Instrument20 for each institution adopting REActivities will be used to assess both the cognitive and affective domains of student learning. These data will hopefully lend themselves to the evaluation of student confidence when using REActivities. Lastly, we plan to establish collaborations with new adopters of the REActivities method in order to further convert laboratory experiments. It is also our goal that this approach to delivering these laboratories contributes to the development of future classroom materials in other disciplines, including the K−12 curriculum.

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.7b00234. Example of three representative REActivities (PDF) Cost analysis between semester I expository lab and semester I REActivities lab (PDF)

Figure 4. Comparison of averaged RTOP scores for all REActivities delivered at MCC and RIT, respectively. The MCC results are an aggregate of six REActivities, whereas the RIT results are an aggregate of eight REActivities. Five of the REActivities were identical across both sets of results.

■

Both aggregate scores fall well above the threshold score of 50 indicating an observation of reformed teaching in both contexts. The lack of significant variation in RTOP scores supports the expectation that REActivities can be implemented across institutions without significant erosion of reformed practices. Future RTOP studies are planned at additional institutions to measure the effectiveness of REActivities across a broader span of instructors and institutions.

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. ORCID

Christina G. Collison: 0000-0002-1361-447X Notes

The authors declare no competing financial interest.

■ ■

■

ACKNOWLEDGMENTS We would like to acknowledge our funding from the National Science Foundation (TUES-1245160).

CONCLUSIONS On the basis of the RTOP scores, we assert that the implementation of REActivities has met the intended goals to improve student engagement in the lab given that both the LDI and Classroom Culture scores were appreciably higher. First, RTOP data support that the delivery of REActivities is significantly different than an expository lab delivery. Second,

REFERENCES

(1) Reid, N.; Shah, I. The Role of Laboratory Work in University Chemistry. Chem. Educ. Res. Pract. 2007, 8 (2), 172−185.

F

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(2) Collison, C. G.; Cody, J.; Stanford, C. An SN1−SN2 Lesson in an Organic Chemistry Lab Using a Studio-Based Approach. J. Chem. Educ. 2012, 89 (6), 750−754. (3) Herron, J. D.; Nurrenbern, S. C. Chemical Education Research: Improving Chemistry Learning. J. Chem. Educ. 1999, 76 (10), 1353− 1361. (4) (a) Gottfried, A. C.; Sweeder, R. D.; Bartolin, J. M.; Hessler, J. A.; Reynolds, B. P.; Stewart, I. C.; Coppola, B. P.; Holl, M. M. B. Design and Implementation of a Studio-Based General Chemistry Course. J. Chem. Educ. 2007, 84 (2), 265−270. (b) Gibson, L. Student-Directed Learning: An Exercise in Student Engagement. College Teaching 2011, 59 (3), 95−101. (5) Oliver-Hoyo, M. T.; Allen, D.; Hunt, W. F.; Hutson, J.; Pitts, A. Effects of an Active Learning Environment: Teaching Innovations at a Research I Institution. J. Chem. Educ. 2004, 81 (3), 441−448. (6) Domin, D. S. A Review of Laboratory Instruction Styles. J. Chem. Educ. 1999, 76 (4), 543−547. (7) Hodson, D. Laboratory work as scientific method: three decades of confusion and distorion. J. Curr. Stud. 1996, 28, 115−135. (8) (a) Jalil, P. A. A Procedural Problem in Laboratory Teaching: Experiment and Explain, or Vice-Versa? J. Chem. Educ. 2006, 83 (1), 159−163. (b) Landis, C. R.; Peace, G. E.; Scharberg, M. A.; Branz, S.; Spencer, J. N.; Ricci, R. W.; Zumdahl, S. A.; Shaw, D. The New Traditions Consortium: Shifting from a Faculty-Centered Paradigm to a Student-Centered Paradigm. J. Chem. Educ. 1998, 75 (6), 741−744. (9) (a) Haight, G. P. Bringing Undergraduates to the Chemical Frontier. J. Chem. Educ. 1967, 44 (12), 766−767. (b) Ricci, R. W.; Ditzler, M. A. Discovery Chemistry: A Laboratory-Centered Approach to Teaching General Chemistry. J. Chem. Educ. 1991, 68 (3), 228− 231. (c) Ricci, R. W.; Dizler, M. A.; Jarret, R.; McMaster, P.; Herrick, R. The Holy Cross Discovery Chemistry Program. J. Chem. Educ. 1994, 71 (5), 404−405. (10) Gaddis, B. A.; Schoffstall, A. M. Incorporating Guided-Inquiry Learning into the Organic Chemistry Laboratory. J. Chem. Educ. 2007, 84 (5), 848−851. (11) (a) Altmiller, H. Another Approach to Freshman Chemistry. J. Chem. Educ. 1973, 50 (4), 249. (b) DiBiase, W. J.; Wagner, E. P. Aligning General Chemistry Laboratory with Lecture at a Large University. Sch. Sci. Math. 2002, 102 (4), 158−171. (c) Bailey, C.; Kingsbury, K.; Kulinowski, K.; Paradis, J.; Schoonover, R. An Integrated Lecture-Laboratory Environment for General Chemistry. J. Chem. Educ. 2000, 77 (2), 195−199. (12) Bodner, G. M.; Hunter, W. J. F.; Lamba, R. S. What happens when discovery laboratories are integrated into the curriculum at a large research university? Chem. Educ. 1998, 3, 1−21. (13) Schroeder, J. D.; Greenbowe, T. J. Implementing POGIL in the Lecture and the Science Writing Heuristic in the LaboratoryStudent Perceptions and Performance in Undergraduate Organic Chemistry. Chem. Educ. Res. Pract. 2008, 9 (2), 149−156. (14) Collison, C. G.; Cody, J.; Smith, D.; Swartzenberg, J. Formalizing the First Day in an Organic Chemistry Laboratory Using a Studio-Based Approach. J. Chem. Educ. 2015, 92 (9), 1510− 1513. (15) Apple, T.; Cutler, A. The Rensselaer Studio General Chemistry Course. J. Chem. Educ. 1999, 76 (4), 462−463. (16) (a) Henderson, C. The Challenges of Instructional Change under the Best of Circumstances: A Case Study of One College Physics Instructor. Am. J. Phys. 2005, 73 (8), 778. (b) Rogers, E. M. Diffusion of Innovations; Simon & Schuster: New York, 1995. (17) Stains, M.; Vickrey, T. Fidelity of Implementation: An Overlooked Yet Critical Construct To Establish Effectiveness of Evidence-Based Instructional Practices. CBE Life Sciences Education 2017, 16 (1), rm1−11. (18) (a) Sawada, D.; Piburn, M. D.; Judson, E.; Turley, J.; Falconer, K.; Benford, R.; Bloom, I. Measuring Reform Practices in Science and Mathematics Classrooms: The Reformed Teaching Observation Protocol. Sch. Sci. Math. 2002, 102 (6), 245−253. (b) Piburn, M.; Sawada, D. Reformed Teaching Observation Protocol (RTOP): Reference Manual. http://www.public.asu.edu/~anton1/AssessArticles/

Assessments/Chemistry%20Assessments/ RTOP%20Reference%20Manual.pdf (accessed Jul 2017). (19) MacIsaac, D.; Falconer, K. Reforming Physics Instruction Via RTOP. Phys. Teach. 2002, 40 (8), 479−485. (20) Galloway, K. R.; Bretz, S. L. Development of an Assessment Tool To Measure Students’ Meaningful Learning in the Undergraduate Chemistry Laboratory. J. Chem. Educ. 2015, 92, 1149−1158.

G

DOI: 10.1021/acs.jchemed.7b00234 J. Chem. Educ. XXXX, XXX, XXX−XXX