Integrating Primary Research into the Teaching ... - ACS Publications

Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

pubs.acs.org/jchemeduc

Integrating Primary Research into the Teaching Lab: Benefits and Impacts of a One-Semester CURE for Physical Chemistry Leah C. Williams* and Michael J. Reddish† Department of Chemistry, Emory University, Atlanta, Georgia 30322, United States S Supporting Information *

ABSTRACT: Many chemistry laboratory exercises follow a given protocol with known results. Such traditional laboratories rarely give students an accurate representation of how research is conducted, the scientific practices involved in research, and the ownership that accompanies developing and carrying out an independent project. Several laboratory reforms have sought to overcome these limitations, including the creation of course-based undergraduate research experiences (CUREs). The CURE design is meant to emulate authentic research in the teaching laboratory by having students perform novel experiments with unknown results. In this article, we describe our implementation of a CURE for an upper-level physical chemistry laboratory course. Our students carried out novel research using molecular dynamics simulations, isothermal titration calorimetry, and stopped-flow kinetics to study ligand binding to the protein human serum albumin. We studied the effects of the CURE laboratory redesign via a mixed-methods approach. We use the CURE Survey by Lopatto and colleagues to record students’ perceived gains in course elements and benefits. We also conducted student interviews to gain an in-depth view of their experience with the CURE laboratory. Our findings suggest that implementing a CURE in an upper-level chemistry laboratory results in similar outcomes to other CURE experiences (which most often occur at the introductory level), can standardize undergraduate research training, and can increase student ownership of laboratory work. We conclude that developing CURE courses for upper-level chemistry courses is an effective way of enhancing undergraduate laboratory training and increasing student experience with research. KEYWORDS: Chemical Education Research, Inquiry-Based/Discovery Learning, Upper-Division Undergraduate, Laboratory Instruction, Biochemistry, Physical Chemistry, Biophysical Chemistry, Undergraduate Research FEATURE: Chemical Education Research

■

INTRODUCTION

Well-designed laboratories can help students to develop competence with scientific practices such as experimental design, argumentation, formulation of scientific questions, and the use of discipline-specific equipment... All but the last of these are included in the scientific practices defined in the Next Generation Science Standards (NGSS).6,7 These practices can serve as a concrete outline for developing learning goals in the laboratory. If we choose to highlight these scientific practices in the laboratory, then what is the best way to help students realize and achieve these goals? That is, how can we design an engaging laboratory that requires students to carry out scientific practices and allows them to experience chemistry research? There have been several successful laboratory reform efforts in the past decade. Notable examples include process-oriented guided-inquiry learning (POGIL)8 and cooperative, problembased laboratories (PBLs).9 Recent years, however, have seen an uptick in course-based undergraduate research experiences

While chemists and educators agree that the laboratory is an integral part of postsecondary chemistry education, there is little consensus as to what the goals of a laboratory experience should be.1−3 Bruck, Towns, and Bretz interviewed faculty members about their laboratory goals, and responses from those who taught upper-division laboratories showed little overlap. Faculty identified as successful grant writers expressed an emphasis on “experimental design and understanding uncertainty in measurement” while other faculty preferred to highlight “specific laboratory techniques and skills”. One physical chemistry lab instructor stressed their concern that their “[students] are still somewhat immature in their ability to think things through”. This would seem to indicate that, despite our best efforts, faculty worry that even upper-level chemistry students are ill-prepared to function as practicing chemists and researchers.4 The Discipline-Based Education Research report states (ref 5, p 130): © XXXX American Chemical Society and Division of Chemical Education, Inc.

Received: November 9, 2017 Revised: April 1, 2018

A

DOI: 10.1021/acs.jchemed.7b00855 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

upper-level CURE lab compare to those reported by others engaged in CURE laboratories?

(CUREs), sometimes called authentic research experiences, in response to a call for reform from the Presidents’ Council of Advisors on Science and Technology.10 CUREs are meant to emulate the success of undergraduate research experiences (UREs) and the summer equivalent (SUREs).11−13 In general, CUREs should be accessible to most, if not all, students and require them to engage in novel research.14,15 Weaver and colleagues state that implementing this kind of lab structure “creates an environment in which students are participants in the development of new knowledge, and where the instructors are facilitating this process as research mentors”.16 While the number of papers referencing CUREs has risen significantly in recent years,15,17−26 there has been less consensus as to what defines a CURE. Our physical chemistry lab was redesigned to meet the criteria outlined by Auchincloss and colleagues who state that a CURE should engage students in the use of scientific practices, discovery, broadly relevant work, collaboration, and iteration.15 In the design of our CURE, we emphasized many of the scientific practices outlined by the NGSS6 while including experiments that could result in publishable data. In a CURE, neither the instructor nor the students should know the outcome of the experiments conducted; otherwise, the work completed is not particularly novel. This is a key difference between CUREs and other lab reform efforts like inquiry-based laboratories. Weaver and colleagues describe the level of student responsibility in lab as a continuum ranging from traditional laboratories to research apprenticeships. Research-based laboratories fall on the higher end of the scale, requiring more student responsibility than guided-inquiry or open-inquiry laboratories.16 Additionally, the National Academy of Sciences report on incorporating research into the undergraduate curriculum states that CUREs should provide “a collaborative atmosphere but [require] students to make decisions” and that they should include “a schedule that [allows] for failure and reiteration”.27 Our design included many opportunities for collaboration between and within student groups and gave students the opportunity to revise and repeat individual experiments (particularly when experiments inevitably failed). By implementing a research-embedded experience into an upper-level chemistry course, our goals differed from those traditionally associated with CURE laboratories in introductory courses. In these laboratories, goals often include promoting persistence in science and developing science identity.14−16,22,28 Most upper-level students, however, have already given serious thought to their career goals and ambitions. These students have remained in the sciences up until this point and are unlikely to change majors. Because of this, we did not include persistence in science as one of our goals for the course. Our course design goals included providing students with opportunities to make critical decisions in the research process and requiring them to engage in authentic scientific practices to show them how chemists engage in research from the development of a research question all the way to publication. Our specific research questions for this study were the following: RQ1. What effects does implementing a CURE in an upperlevel physical chemistry have on students’ understanding of the process of research and on their confidence in their ability to carry out a research project? RQ2. How do the experiences with research elements and the resulting benefits gained by students participating in an

■

LAB DEVELOPMENT Our main goal for this research project was to develop a CURE model of lab exercises for the upper-level physical chemistry course required for chemistry majors at Emory University. In the Emory curriculum, physical chemistry is a two-course sequence with each semester requiring a separate but concurrent lab course. The physical chemistry lab emphasizes writing for the chemical sciences and meets a university-wide writing requirement. Students typically take these courses in their junior or senior year and are required to have completed four introductory courses (two general and two organic) along with analytical chemistry as a pre- or corequisite. It is common for students enrolled in physical chemistry to have also taken other courses like inorganic chemistry or biochemistry. This research project focuses on the second-semester physical chemistry lab course with content areas including phases of matter, statistical mechanics, kinetics, and thermodynamics. All of the students who participated in this research project previously completed the first-semester physical chemistry course and lab; all but one student took the first course in the sequence in the previous semester. The first-semester lab followed a traditional style, starting a new lab experiment every 2 weeks using known procedures and outcomes. The lab assessments emphasized writing by requiring students to submit research reports in the style of a traditional research article for each lab exercise. We redesigned the second-semester lab to move away from a range of individual experiments and to instead focus on three projects directly related to a single research topic: the interaction of uremic toxins with the protein human serum albumin (HSA). Uremic syndrome is related to the gradual increase in concentration of substances that are normally removed from the body with normal kidney function. Some uremic toxins, a subset of these substances, are dangerous because they interact with serum proteins, like HSA, and become difficult to remove using conventional hemodialysis.29,30 To improve dialysis methods, the students were tasked with studying the interactions of specific uremic toxins with HSA using three approaches: binding kinetics via stopped-flow mixing, equilibrium thermodynamics using isothermal titration calorimetry (ITC), and hydrogen-bonding interactions by molecular dynamics simulations. The CURE structure used in this study required students to work in groups throughout the semester to collect data and write individual reports based on their findings. The students were sorted into small groups of three or four, which were then combined into ligand teams, shown in Table 1. Similar to the strategy recommendations in the NAS report to create “parallel Table 1. Ligand Teams and Rotation Assignments Ligand Teams (Student Groups) Rotation

Indoxyl Sulfate (A, B, C)

1

Molecular dynamics

2

Isothermal titration calorimetry Stopped-flow kinetics

3

B

p-Cresol (D, E, F) Stopped-flow kinetics Molecular dynamics Isothermal titration calorimetry

Hippuric Acid (G, H, I) Isothermal titration calorimetry Stopped-flow kinetics Molecular dynamics



Article

Table 2. Summary of CURE Course Structure Week

Course Element

Course Assessment

3a 4 5−10

Introduction to research problem Class discussion to determine techniques to use 2 week rotations with an experimental technique to plan and implement experiments Class discussion comparing data obtained from each large experimental team Class discussion to determine conclusions

Students investigate and list techniques that will be useful for research problem. Students write Introduction section of article to describe research problem and approach. Students complete prelab worksheet covering a new technique. Students write methods, results, and discussion sections of an article relevant to the 2-week rotation. No assessment this week.

11 12

Students revise previous written reports to include data from all teammates into a full journal article-style report.

a Due to scheduling issues, the CURE course structure started during the third week of class. The first 2 weeks of class used traditional lab exercises and assessment.

population of students, many driven by a desire to go to medical school or other professional programs after graduation. Demographic data for the class is provided in Table 3.

projects” and use student collaboration to collect large bodies of data,27 we designed one overarching research problem that the entire class contributed to, but each team studied a different uremic toxin: indoxyl sulfate, hippuric acid, or p-cresol. This allowed students to share experiences, methods, and data within their ligand team and allowed the class as a whole to collect a large data set. The kinetics and molecular dynamics experiments have not been reported previously in the literature, and the published calorimetry data could be enhanced. Therefore, the students’ work has potential for publication in peer-reviewed journals. Over the first 2 weeks of the CURE curriculum, students were introduced to the research problem through lecture, homework assignments, and class discussion. The instructor provided a general overview of the current research and discussed its relevance to improving treatment for uremic syndrome. For the following 6 week period, students collected data in three 2 week rotations where each rotation used a different experimental method. The students designed the experimental parameters themselves and were expected to report their findings for each rotation in the manner of a Journal of the American Chemical Society article’s Methods, Results, and Discussion sections. Examples of student data for each of the three rotations can be found in Supporting Information, Figures S1−S3. The course design necessitated completing work as a small group. Collaborations within the larger team often took place to troubleshoot experiments, but such collaborations were not explicitly required. Teaching assistants and the instructor were available to help plan experiments and train students to use equipment. They were also required to supervise data collection for ITC and stoppedflow experiments. The final 2 weeks of the course were allotted for student teams to aggregate and analyze their corporate data. Each student submitted a full journal article-style report using data from all three experimental approaches as a final assessment. A general summary of the CURE course structure can be found in Table 2.

Table 3. CURE Lab Student Demographic Data Parameter

Number of Students (Total N = 33) Sex

Female Male

16 17 Ethnicity

Asian White Black Other

13 10 2 8 College Major

Chemistry Biology Other

24 4 5

a

Includes noncitizens of the United States and students who did not provide this information. bIncludes mathematics and undecided majors.

The effects of the course on students’ understanding of how research is carried out, their confidence in their ability to conduct research, and how those effects compare to others reported for CURE courses were investigated using a mixedmethods approach. We administered the CURE survey, designed by Schaffer and colleagues, to students before the start of the research lab experience and then again at the end of the semester after lab work had been completed.20,31−34 In the findings reported below, we have only included data from students that completed both the pre- and postsurvey in its entirety (N = 22). Some students were absent or tardy on the day that either the pre- or postsurvey was administered, and one student chose not to participate in the study, resulting in a smaller population. The CURE survey asks students to self-report gains with course elements and course benefits after engaging in a CURE lab using Likert-scale items. Course element items include features common to laboratory practice such as working in small groups, writing a research proposal, and presenting findings. Course benefits items include potential outcomes like readiness for research and learning to work independently. The course benefits items in the CURE survey mirror those included in the SURE survey developed by Lopatto and colleagues.31,32,34 By keeping these items the same, Schaffer and colleagues have used both the CURE and SURE surveys to explore similarities in course benefits between UREs and CUREs.34 High internal consistency and concurrent validity of

Study Methodology

There were 33 students who participated in the research-based physical chemistry laboratory course in the spring semester of 2016. The same instructor taught both the fall and spring laboratories for the 2015−2016 academic year. There were 24 enrolled students who listed chemistry as their primary major with other majors including biology and math. There were 11 students who indicated that they were working toward a double major, of which 3 were pursing both majors in the sciences. Of the students, 28 had a cumulative GPA of 3.0 or higher, and the average cumulative GPA for students enrolled at the start of the course was 3.55. In general, this was a high-performing C



Article

Figure 1. Emory student response frequencies for course elements after engaging in the CURE lab (postsurvey). Course elements included above are items where over 70% of Emory students reported a large or very large gain. Starred elements (*) indicate statistically significant differences when comparing pre- and postsurvey responses.

the course benefits items from the SURE survey have been reported previously by Lopatto31 as well as high ecological correlation from replication studies.32 Lopatto has acknowledged the lack of other reliability and validity measures for the SURE survey, such as the use of a control group, and to our knowledge, no reliability and validity studies have been carried out for course benefits and course elements items in the context of the CURE survey.32 Given that only course elements are included in both the preand postcourse survey, we compared Emory student gains pre/ post via Wilcoxon signed-rank analysis. We chose to use nonparametric analyses due to the ordinal nature of the data. Pearson’s coefficient (r) is reported to indicate the effect size for any statistically significant differences. We also compared our students’ responses on the survey to a larger body of responses collected by Lopatto and colleagues between June 1, 2015, and May 24, 2016. We refer to this set of responses as “all students”. Data for this larger group comes from the CURE survey responses from a wide range of students involved in CURE laboratories collected by Lopatto and colleagues.33 They provide mean and standard deviation data for the “all students” group to researchers who give the CURE survey to their students. Using the data provided, any comparisons between Emory students and the larger “all students” group require parametric analyses. We compared Emory students to the larger population for both course elements and course benefits using an independent t-test. We report Cohen’s d to indicate effect size for any statistically significant differences.35 While the CURE survey provides insight into students’ selfreported gains, we conducted interviews with 18 students at the end of the semester to expand on their experience in the course. Specifically, we wanted to ask students about scientific practices that reflect how traditional research is conducted (which align with specific course elements in the CURE survey), how they engaged (or did not engage) in those practices, and interesting or difficult aspects of the laboratory experience as a whole. Interviews were conducted with individuals or with groups depending on availability and lasted from 30 to 60 min. Audio was recorded using a digital voice recorder and transcribed verbatim by a transcription service. Transcripts were reviewed by the lead author for accuracy. The transcripts were analyzed in two phases. Initially, the lead author used an inductive approach to individually code each student interview, looking for emergent features related to their understanding of the nature of research and their

confidence in carrying out a research project. For the second phase, coded interview segments were compared across all interviews to identify common effects of the CURE course on student understanding and confidence.36 Groups of related codes were created to address three emergent themes: “encountering obstacles”, “collaboration” (both of which relate to an understanding of how research is conducted), and “project ownership” (related to student confidence). These three themes align with possible benefits identified by Corwin and colleagues in their summary of reported CURE benefits in the literature.37 A constant comparison approach was used to confirm that codes and themes were being applied consistently across all transcripts and to refine a formal coding scheme.36 To address reliability of the coding scheme, a second coder with a background in chemical education independently coded all interview transcripts for each of the three main themes after discussion of the coding scheme. While the lead author and second coder were in complete agreement on two themes (“encountering obstacles” and “collaboration”), the remaining theme of “project ownership” resulted in a percent agreement of 83.3% and a Cohen’s kappa of 0.429. After further discussion and careful clarification of the coding scheme, final codes for “project ownership” were applied on the basis of mutual agreement. A copy of the interview protocol and coding scheme with example quotes are provided in Supporting Information. All students in this study agreed to participate and signed informed consent forms. Pseudonyms have been used below in place of student names to protect student privacy. IRB approval was given for the data included in this study.

■

RESULTS: STUDENT RESEARCH FINDINGS The three student ligand teams had variable success in acquiring data for each of the three experimental methods. The p-cresol and indoxyl sulfate ligand teams were able to measure equilibrium constants and thermodynamic parameters for their ligands with HSA by ITC; however, more repetitions are required before the data can be published. Although the hippuric acid team was unable to successfully make ITC measurements, all three teams were able to critically evaluate currently published ITC work based on their experiences. None of the ligand groups were able to collect the intended data from stopped-flow experiments. The experimental design relied on a fluorescent dye, 8-anilino-1-napthalenesulfonic acid (ANS), as a competitive ligand for the same HSA binding site as the uremic toxins. The students discovered, however, that ANS either does D



Article

Figure 2. Emory student response frequencies for course benefits after engaging in the CURE lab (postsurvey). Course benefits included above are items where over 70% of Emory students reported a large or very large gain.

Figure 3. Comparing self-reported mean gains for Emory and “all students” groups for course elements in the post-CURE survey. Starred course elements (*) indicate statistically significant differences between groups.

including gains with a lab where no one knows the outcome and a project in which students have input into the process or topic. These two course elements are associated with CUREstyle laboratories and the process of research. Frequencies of presurvey responses for all 25 course elements with descriptive statistics (Figure S4 and Table S1) can be found in Supporting Information. Similarly, descriptive statistics for postsurvey responses for all 25 course elements can be found in Table S2 along with frequencies of responses (Figure S5).

not compete for the same binding site or binds differently in the presence of each of their toxins. This led to the conclusion that future studies using competitive kinetic experiments will likely need a different approach. Finally, the molecular dynamics simulations were not intended to produce publishable research, but rather to serve as a way for students to explore structural relationships between biological macromolecules and their ligands. All three ligand groups identified potential hydrogen-bonding interactions for their ligand. For publication, these studies would need crystallographic evidence, which was beyond the scope of this course.

Course Elements: Comparing Emory Student Pre-/Postsurvey Responses

■

We compared pre- and postsurvey responses for all course elements using Wilcoxon signed-rank analyses. With a Bonferroni correction to adjust the significance level (p-value < 0.002),38 we found two course elements to have statistically significant differences with large effect sizes:35 a project where students have input into the process or topic (Z = 3.47, p = 0.001, r = 0.52) and computer modeling (Z = 3.73, p < 0.001, r = 0.58). The first of these aligns with our design of the CURE lab where students were encouraged to work collaboratively to design and carry out experiments that could lead to publishable data. While the students did not choose the research question, this significant difference highlights their involvement in the design, data collection, and analysis portions of the research project. As for the gains in computer modeling, this lab was the first time that most of the students were introduced to computational methods. Students were required to devote 2 weeks to computational studies using molecular dynamics. Results of pre/postsurvey comparisons using Wilcoxon signed-

RESULTS: CURE SURVEY In order to address RQ1, we analyzed student reported gains in course elements and course benefits from the CURE survey. Gains in course elements should show if our students felt that they had engaged in specific practices associated with the research process whereas gains in course benefits can help us identify what effects students report from participating in the CURE course. Course Elements: Response Frequencies

For the pre- and postsurvey, students were asked to self-report their perceived gains for 25 course elements. In the presurvey, students are asked to rate their experience on a 5-point scale from inexperienced to extensive experience, and in the postsurvey, they are asked to rate their perceived gain on a 5point scale from very small gain to very large gain. For the postsurvey, over 70% of Emory students reported a large or very large gain for the elements shown below in Figure 1, E



Article

Figure 4. Comparing self-reported mean gains for Emory and “all students” groups for course benefits in the post-CURE survey. Starred course benefits (*) indicate statistically significant differences between groups.

p < 0.001, d = 1.11 large effect size). For all other course elements, the Emory students reported similar mean gains to the larger “all students” group. A comparison of means for Emory students and the “all students” group for course benefits is shown in Figure 4. We found three course benefits to show statistically significant differences between the Emory group and the “all students” group after applying a Bonferonni correction (p-level < 0.0024): “tolerance for obstacles” (t = 3.65, p < 0.001, d = 0.56 medium effect size), “readiness for research” (t = 3.58, p < 0.001, d = 0.61 medium effect size), and “skill in science writing” (t = 6.38, p < 0.001, d = 0.89 large effect size). The Emory and “all students” group reported similar mean gains for all other course benefits. Comparisons for all course elements and benefits can be found in Table S5 and S6 in Supporting Information. The comparisons of course elements and benefits both indicate that, for all but a select few items, Emory students report similar gains to those reported by the larger group of students enrolled in CURE courses. Significant differences in course elements “no one knows the outcome” and “computer modeling” highlight key, research-like aspects of our course. As is true of novel research, one of the defining features of a CURE is that neither the instructor nor the students should know the outcome.15 Additionally, not all chemistry research is conducted in a wet lab. Our CURE course included computer modeling as part of the research design, and we see a significant gain reported by students as a result. As for course benefits, the significance difference in science writing emphasizes the writing requirement nature of the course. Writing is an integral component of the research process, and every student submitted a journal-style article with their findings by the end of the course. The significant differences in tolerance for obstacles and readiness for research highlight students’ understanding of the research process and their ability to engage in it. The first of the two has been reported by others in the literature, including Hunter and colleagues, as a “reality of research”,13,39 and is discussed in further detail below.

rank (Table S3) for all course elements can be found in Supporting Information. Course Benefits: Response Frequencies

In the post-CURE survey, students were asked to report perceived gains for 21 course benefits. They were asked to rate these benefits on a 5-point scale from no gain to very large gain. For course benefits, over 70% of Emory students reported a large or very large gain with the three benefits shown in Figure 2, including tolerance for obstacles faced in the research process and skill in scientific writing. Tolerance for obstacles is an integral part of the research process and relates to some of our interview findings discussed below. The gains in science writing can be attributed to the fact that this particular lab course (and its predecessor) serves as a writing requirement course for the department. Students were asked to submit sections of a journal article writeup every 2 weeks for grading and feedback. These sections culminated into a full report of their research findings in the format of an ACS journal article by the end of the semester. Descriptive statistics for postsurvey gains in course benefits and frequencies of student reported gains can be found in Supporting Information (Table S4 and Figure S6, respectively). Course Elements and Course Benefits: Comparing Emory Students to “All Students”

In order to address RQ2, we compared Emory student responses on course elements and benefits to a larger group, entitled “all students”, who participated in CUREs during the 2015−2016 academic year. We were able to compare gains using mean and standard deviation data provided to us by Lopatto and colleagues. A comparison of Emory students and the “all students” group for course elements is shown in Figure 3. Using a Bonferroni correction to adjust the significance level (p-value < 0.002), we found two course elements to have statistically significant differences with large effect sizes: “a lab where no one knows the outcome” (t = 4.41, p < 0.001, d = 0.63 medium effect size) and “computer modeling” (t = 7.618, F



■

Article

time-consuming and riddled with failure.13 Experiencing these “realities of research” is key to helping students understand what it means to engage in real scientific research.

RESULTS: INTERVIEWS In addition to collecting survey data, we interviewed 18 students at the end of the spring semester to gain a more detailed picture of their experience in the CURE course. To better address RQ1, we asked about specific practices that students engaged in as well as interesting or difficult moments throughout the course. We found three common themes throughout our interviews: encountering obstacles, project ownership, and collaboration.

Project Ownership

Perhaps as a result of their encounters with obstacles in the lab, all but one student discussed at least one instance of project ownership. Glenda described how her group took charge of their experiments in lab, stating We were kind of given the option to be like “Oh, do we want to test temperature, do we want to test concentration, do we want to test this, do we want to test that?” So, after cohesively talking as a group, we figured out the question we wanted to answer. For Glenda, this led to an increased sense of responsibility; her group had the ability to control what happened in lab. Lucy described a specific instance with the molecular dynamics lab: There is a huge learning curve with figuring out how to do molecular dynamics and learning how to code... we got to be part of that sort of vetting process. I think it let us take more ownership of it... we got to be able to sort of say like, “What if we did it this way, what if we used this code?” Both students described an increased responsibility and confidence in making decisions about the research design and analysis. Even if the experiment did not work out as planned, they had an appreciation for their increased control over the direction of the experiments. Lucy described it best, stating “[Our TAs] were there to really be experts in how the experiment worked. We were the experts in our own ligand.” Our own findings of students’ increased project ownership align with other published studies that report it as a benefit for students engaging in UREs.13,39,41,43 The same “becoming a scientist” category identified by Hunter and colleagues described above also includes students’ descriptions of “figuring things out for themselves (and with their research peer group) rather than relying on faculty”. Their students described an increased sense of independence akin to what we have described as project ownership.13 Hanauer and Dolan identified eight positive indicators of project ownership from interviews with students who participated in UREs. Several of these indicators align with our own students’ descriptions of project ownership including “agency combined with mentorship”, “expressions of a sense of personal scientific achievement”, and “expressions of understanding the unexpected aspects of science”.41 The project ownership described here highlights our students’ increased confidence in their ability to make critical decisions during the research process.

Encountering Obstacles

One defining aspect of a CURE is that neither the instructor nor the students know the outcome of the research they are going to engage in throughout the semester.15 This novelty means that a research experience requires significant time spent managing problems as they arise. All students who were interviewed discussed at least one instance of encountering obstacles during the course of the CURE lab. For instance, Trevor describes his group’s struggle with a kinetics experiment: It can be really frustrating when you are not getting anything... 2 h, 45 min to prepare the solution, prepare the dye, prepare the protein and then just to have it [fail]. It just shows you that research is tough. It is hard... you really have to keep at it because it is hard work. Most of the teaching laboratories that students had participated in up until this point were carefully scripted. That is not to say that students do not encounter obstacles in these traditional laboratories, but that there is often little room for students to grapple with failure. For students like Agnes, this was a welcomed opportunity, “It was kind of exciting because nobody else has ever done this before. Even with all of the failures that we’ve had, it was exciting to find a new piece of the puzzle.” While the majority of our students viewed the unpredictable nature of research as an opportunity to tackle a challenging obstacle, three individuals did not find the experience as enriching. Jason is an example of one such student who stated, “I would like to know exactly what I’m doing before I do it. That’s just me I guess... I know it’s frustrating when you’re doing it and then I’m not exactly sure what I’m doing.” For Jason, the frustration associated with failed experiments was not an opportunity for improvement, and this is not entirely surprising. Given the students’ previous experiences with traditional laboratories, transitioning to a CURE-style lab is a large leap; one that some students might not be comfortable making. Our students’ descriptions of these encounters indicate their understanding of the novel nature of conducting research and the consequences of that novelty: frequent unforeseen obstacles. Their descriptions are mirrored in other published studies on the benefits of research experiences.13,39−42 For instance, Richmond and Kurth identified “the role of uncertainty” as one of four dimensions of scientific practice that emerged from interviews with high school students involved in research apprenticeships. Their students experienced uncertainty, and some detailed their frustration when things did not go as planned.40 Similarly, Hunter and colleagues interviewed students and faculty members about possible benefits for students participating in a URE. A subset of students described gains in an interview category called “becoming a scientist”, which includes students’ experiences with “the realities of research”, more specifically that research is

Collaboration

The ability to work collaboratively was key in helping students to address frustrations and find solutions to problems. Scientific research is inherently collaborative, and the work carried out by students in a CURE lab should reflect that. By assigning the same ligand to several groups, our students were able to coordinate work that, individually, would have taken them far more time to complete. Lucy described how her team used the group design for molecular dynamics simulations: G



Article

which was significantly higher than the gain reported by “all students” engaging in a CURE course. Students that we interviewed discussed obstacles encountered during the lab course and described how those obstacles were beneficial to the overall experience. These obstacles are a standard component of the research process and are often left out of the traditional undergraduate laboratory. While three of the students we interviewed failed to see these obstacles as a learning experience, this may be a consequence of their inexperience with CURE- or inquiry-style laboratories. Thiry and colleagues noted that one novice student they interviewed “described her project as ‘finicky’... she demonstrated that she was not yet able to locate her own experiences within the broader context of the scientific research process.” They found that students who had participated in multiyear research experiences were better able to tolerate these setbacks.44 Increasing students’ understanding and tolerance of these obstacles in a CURE contributes to their overall understanding of what it means to conduct research. In addition to tolerance for obstacles, we found statistically significant differences comparing our students to the larger “all students” group for gains in computational methods (as well as in pre/postsurvey comparisons for Emory students). Molecular dynamics simulations played a large role in the design of the course. While interview themes relating to computational methods did not emerge from the data, many of our students discussed how this was their first interaction with computational methods. Most undergraduate teaching laboratories are wet laboratories, yet there are many chemistry researchers who never work with chemicals. By incorporating molecular dynamics simulations, we were able to introduce our students to another aspect of chemistry research that many of them originally reported little to no experience with (Supporting Information, Figure S5). The significant gains with this course element indicate our students’ increased understanding that computational methods can serve as one more way to conduct scientific research. Our survey and interview findings highlight our students’ increased confidence in their ability to carry out a research project (RQ1). From the CURE survey, we saw significant differences in students’ experience with “a project where students have input into the process” (when comparing pre/ postsurvey results) and “readiness for research” (when comparing Emory students to “all students”). The first of these relates to one of Hanauer and colleagues’ five elements that foster project ownership in a URE, facilitating personal agency. They recommend that UREs allow students to “make decisions concerning research questions and methods”.41 Our own student interviews show increased project ownership where students described making decisions regarding design, data collection, and analysis. These decisions, as well as the reported increase in readiness for research, show an increase in confidence as a result of participation in the CURE lab.

We would get to a point and we would troubleshoot along the way with the TAs... and then the next group would go after we had solved our problems, they would get the next step, troubleshoot that step. So, it was like a staggered troubleshooting process... whenever we had troubleshooting, very little of it was actually communicated through [the instructor] or the TAs. It was really communicated among the 11 of us. Agnes described a similar scenario as her group carried out stopped-flow experiments. Her team collaboratively worked through issues with data collection and produced a body of information that could be used by all three groups. I think just because we were all divided into subgroups, when my stuff did not work out and then the next group finds out, “Oh my God, it works now” and then we all try it again. It is kind of exciting to, you know... That is what I like about research even though it fails a lot of the time. It is important to note that student groups were not required to work with other groups on the same ligand team. The instructor set aside 1 h each week for teams to meet and discuss upcoming experiments. Groups were asked to share data with their ligand team, but it was left up to the students to decide how much they wanted to collaborate. This resulted in two of the three teams using a system like the one described by Lucy. Responses from all four students interviewed from the third ligand team, however, indicated that their team chose to conduct their experiments as independent groups with little collaboration. Allie described the lack of collaboration during the ITC lab, “We picked the conditions beforehand, we kind of compared it with the other subgroups... it was to ensure that we were doing different conditions, I don’t know but it wasn’t a super successful kind of like discussion period.” Similarly, Stanley described the overall experience of the CURE lab, stating “you have to just collaboratively think and work as a team, which didn’t happen enough.” Both of these students and Jason, who previously discussed his frustration with the uncertainty of lab and his desire for a more scripted experience, were part of the same ligand team. It is entirely possible that some of Jason’s frustration could have been mediated by more interaction and cooperation with the other groups on his ligand team. Auchincloss and colleagues specifically identify collaboration as one of five defining features of a CURE. They discuss how collaboration accurately reflects how research is conducted and can serve as a pedagogical tool to show students the power of teamwork to tackle a problem.15 Hanauer and colleagues list social interaction as a critical element of UREs that “may reduce feelings of frustration that may accompany novel research questions and procedures”.41 For the ligand team that did not collaborate, none of the four students interviewed discussed working systematically to address problems in lab. Considering the benefits of collaboration reported in the literature as well as those described by the other two ligand groups, we intend to require students to work together as a larger ligand group in the future to enrich the research experience.

■

■

CONCLUSIONS AND IMPLICATIONS

Benefits to Incorporating Research into an Upper-Level Lab

DISCUSSION: COMPARING SURVEY AND INTERVIEW DATA Both our survey and interview data show an increased understanding of the process of research for our students (RQ1). In the CURE survey, over 70% of Emory students reported a large gain in working on a lab where no one knows the outcome as well as tolerance for obstacles, the latter of

Most CUREs reported in the chemistry education literature occur at the introductory level.17,22−25,42,45,46 As such, many of the intended outcomes for CUREs in these settings are meant to encourage students to explore research opportunities and science careers. We did not expect to find an increased interest in STEM or desire to pursue more scientific research as a result H



Article

Transitioning into CURE-Style Laboratories

of our CURE. We were curious, however, what other effects we might see from engaging juniors and seniors in a CURE. Our findings using the CURE survey show gains similar to those reported by other CURE programs administered in the 2015−2016 academic year, with significant differences in elements and benefits that relate directly to the research-like nature of the course. Some of our interview themes align with our survey findings, and all themes are supported by reports in the literature regarding students’ increased tolerance for obstacles,13,39,40,42,44 collaboration,13,37,39,40 and project ownership.13,37,39,41,43 All of these findings highlight the benefit of administering a CURE course at the upper-level and indicate that the design and implementation of such a course at this level is worth the time and effort for instructors.

The move from traditional laboratories to CURE-style laboratories can be a drastic change for students. If their previous experience was centered on traditional laboratories with explicit instructions and known outcomes, then we cannot be surprised when they do not immediately embrace a research experience. Perhaps the most obvious solution is to introduce students early and often to laboratories that require them to engage in scientific practices like designing and carrying out experiments or asking questions. The Next Generation Science Standards have identified these scientific practices for K−12 education, and lecture and laboratory instructors at the postsecondary level have already begun to embrace them.7 Requiring collaboration can help alleviate some of the frustration and failure that often accompany letting students engage in experiments beyond the well-curated laboratories vetted by instructors. Implementing inquiry laboratories and project-based laboratories early in introductory courses can provide a foundation for students in these scientific practices. By carefully structuring early laboratories to allow for more student input and decision-making, students should be more prepared to embrace a research experience in an upper-level course. In our research, we found several benefits to incorporating a research experience into an upper-level lab. The gains we found via the CURE survey indicate that our students experienced similar benefits compared to other students engaged in CURE laboratories. The interview findings also suggest that, as a result of providing the opportunity to make decisions in the research process and work collaboratively toward addressing a larger research question, our students reported an increased sense of project ownership and willingness to tackle research obstacles. We would encourage others to explore incorporating research into their own courses. Instructors might be hesitant to devote the considerable time needed to develop and implement a CURE-style lab.47 We would posit that instructors might find it less daunting to transform an upper-level course with smaller enrollment numbers. Many of these students are already proficient with basic laboratory skills. As such, less time would need to be devoted to teaching these skills, and instead more time could be spent creating an experience that allows students to conduct novel research and provides them with increased responsibility and independence.

Benefits for Students with Previous Research Experience

Many of the students enrolled in our CURE lab had already engaged in some form of research. Four students had previously participated in an introductory biology lab where individual modules included CURE elements such as proposing a research question and designing experiments. There were 11 students who had enrolled in at least one credit hour of research within the chemistry department (more often three or four credits in a single semester). While we do not know exactly how many students had previous research experience, 16 of the 18 students interviewed described a past or current research experience. After further discussion, it became apparent that these experiences varied widely in content, duration, and impact. For example, Harry described his biochemistry research within the department and pointedly drew comparisons to his experience in the CURE lab: [the CURE lab is] realistic to the actual research that I’ve done... it takes a lot more time to troubleshoot a lot of methods. It also makes you think in different ways, not just what’s happening to the machine but how are you going to fix it? Other students like Stanley, however, described the stark contrast between their previous research experiences and their time in the CURE lab. I had my boss over the summer who would analyze stuff, my data, and then tell me what to do next. But here, it actually felt like I was kind of trying to figure out what I would have done next and then show future directions. For our students, it would seem that not all previous research experiences are created equal. Hanauer and colleagues have shown that, in general, students who participated in UREs described a higher level of project ownership than those enrolled in a traditional laboratory course. However, they also found “differences in the levels of project ownership expressed in the interview data concerning different types of research experiences”.41 Implementing a CURE lab for upper-level students can provide a consistent and standardized research experience for all students, even for those that have already joined a research group or internship. It also allows students to experience research in a subdiscipline that may differ from their current area of research. Many of our students were engaged in biochemical research, organic synthesis for drug discovery, or medical research. By incorporating research into the physical chemistry lab, students were exposed to a different area of research and new methods (e.g., computation).

■

LIMITATIONS OF THE STUDY

Student Self-Report Instruments

The CURE survey in this study uses students’ self-reported gains and is therefore only a measure of students’ perceived gains. Without supplementary data from other sources, we cannot say, for example, if students actually made gains in the skill of scientific writing. By including interview data where students describe various aspects of the CURE experience, we strove to provide support for some of the gains reported by students on the CURE survey. Additionally, other authors have noted that the CURE survey is missing reliability and validity studies to give added weight to survey findings. While the CURE survey is widely used, further studies are needed to move beyond documenting the benefits of CURE courses and to begin to understand the underlying elements that result in specific benefits.37,43 I



■

Time Restrictions in Lab

ACKNOWLEDGMENTS This research was supported in part by a grant to Emory from the Howard Hughes Medical Institute through the science education program (Grant 52008096) as well as a grant from the National Science Foundation (Grant CHE-0342877). David Lopatto of Grinnell College designed the CURE survey administered in this study. We would like to thank him and Leslie Jaworski for help in collecting and compiling the data. We would also like to thank Drew Kohlhorst for his assistance with data collection and for arranging for interview transcription and Doug Mulford for his help with the interview data. Finally, we would like to give a special thank you to the students who participated in this study and the graduate teaching assistants who agreed to try something new and innovative with us.

Research typically requires more than 3−6 h a week; that is, it takes more than the amount of time allotted to most laboratory courses. In designing our course, we had to decide what we would realistically expect a student conducting research for lab credit to accomplish and how we could modify lab to emulate research while addressing the time constraints of a single course. As a result, we chose to focus on three different experiments that, collectively, should have provided enough new data on the three specified ligands to allow for publication. Students were instructed to work on collecting new data roughly 3 h per week during the research rotation phase of the course. For both the ITC and stopped-flow experiments, students were required to have either the instructor or a TA present to supervise. In hindsight, the very obstacles that helped create a more realistic experience for students also consumed a significant portion of the allotted 3 h. Since our students were unable to collect ITC or stopped-flow data outside of the supervised periods above, the class did not gather as much data as we anticipated. Wink, in the NAS report on integrating research into the undergraduate curriculum, described a similar challenge with their CASPiE project for first- and second-year students. He notes the difficulties in generating enough data in the span of a single semester to result in a publication.17,27 Given our experience, we would expect several iterations of this lab would be needed to ultimately provide enough data to submit for publication. Conducting multiple iterations of the lab is not a limitation, but conforming to course time restrictions means that some students might not see the outcomes of their work published for several years and may question the significance of their contribution.

■

■

REFERENCES

(1) Hofstein, A.; Lunetta, V. N. The Role of the Laboratory in Science Teaching: Neglected Aspects of Research. Review of Educational Research 1982, 52 (2), 201−217. (2) Hofstein, A.; Lunetta, V. N. The Laboratory in Science Education: Foundations for the Twenty-First Century. Sci. Educ. 2004, 88, 28−54. (3) Reid, N.; Shah, I. The Role of Laboratory Work in University Chemistry. Chem. Educ. Res. Pract. 2007, 8 (2), 172−185. (4) Bruck, L. B.; Towns, M.; Bretz, S. L. Faculty Perspectives of Undergraduate Chemistry Laboratory: Goals and Obstacles to Success. J. Chem. Educ. 2010, 87 (12), 1416−1424. (5) National Research Council. Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering; Singer, S. R, Nielson, N. R., Schweingruber, H. A., Eds.; National Academies Press: Washington, DC, 2012. (6) National Research Council. Next Generation Science Standards: For States, By States; The National Academies Press: Washington, DC, 2013. (7) Cooper, M. M. Chemistry and the Next Generation Science Standards. J. Chem. Educ. 2013, 90, 679−680. (8) Moog, R. S.; Creegan, F. J.; Hanson, D. M.; Spencer, J. N.; Straumanis, A.; Bunce, D. M.; Wolfskill, T. POGIL: Process-Oriented Guided-Inquiry Learning. In Chemists’ Guide to Effective Teaching; Pienta, N. J., Cooper, M. M., Greenbowe, T. J., Eds.; Prentice Hall: Upper Saddle River, NJ, 2009; Vol. II, pp 90−107. (9) Sandi-Urena, S.; Cooper, M. M.; Stevens, R. Effect of Cooperative Problem-Based Lab Instruction on Metacognition and Problem-Solving Skills. J. Chem. Educ. 2012, 89, 700−706. (10) President’s Council of Advisors on Science and Technology. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics; Executive Office of the President: Washington, DC, 2012. https://files. eric.ed.gov/fulltext/ED541511.pdf (accessed Mar 2018). (11) Sadler, T.; Burgin, S.; McKinney, L.; Ponjuan, L. Learning Science through Research Apprenticeships: A Critical Review of the Literature. J. Res. Sci. Teach. 2010, 47 (3), 235−256. (12) Robnett, R.; Chemers, M.; Zurbriggen, E. Longitudinal Associations Among Undergraduates’ Research Experience, SelfEfficacy, and Identity. J. Res. Sci. Teach. 2015, 52 (6), 847−867. (13) Hunter, A.-B.; Laursen, S.; Seymour, E. Becoming a Scientist: The Role of Undergraduate Research in Students’ Cognitive, Personal, and Professional Development. Sci. Educ. 2007, 91 (1), 36−74. (14) Linn, M. C.; et al. Undergraduate Research Experiences: Impacts and Opportunities. Science 2015, 347 (6222), 1261757. (15) Auchincloss, L. C.; Laursen, S. L.; Branchaw, J. L.; Eagan, K.; Graham, M.; Hanauer, D. I.; Lawrie, G.; McLinn, C. M.; Pelaez, N.; Rowland, S.; Towns, M.; Trautmann, N. M.; Varma-Nelson, P.; Weston, T. J.; Dolan, E. L. Assessment of Course-Based Undergraduate Research Experiences: A Meeting Report. Cell Biology Education 2014, 13 (1), 29−40.

ASSOCIATED CONTENT

* Supporting Information S

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.7b00855. Samples of student data, the interview protocol, the coding scheme and example quotes, additional tables and figures showing all student responses to CURE survey, and additional tables showing statistical analysis (t-test, Wilcoxon signed-rank, and relevant effect sizes) for comparisons on all “course elements” and “course benefits” survey items (PDF, DOCX)

■

Article

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. ORCID

Leah C. Williams: 0000-0001-7401-5262 Michael J. Reddish: 0000-0002-0237-786X Present Address †

Department of Biochemistry, Vanderbilt University School of Medicine, Nashville, Tennessee 37232, United States. Notes

Any opinions, findings, or recommendations written here are solely those of the authors and do not necessarily reflect the views of the Howard Hughes Medical Institute or the National Science Foundation. The authors declare no competing financial interest. J



Article

(16) Weaver, G. C.; Russell, C. B.; Wink, D. J. Inquiry-Based and Research-Based Laboratory Pedagogies in Undergraduate Science. Nat. Chem. Biol. 2008, 4 (10), 577−580. (17) Weaver, G. C.; Wink, D.; Varma-Nelson, P.; Lytle, F.; Morris, R.; Fornes, W.; Russell, C. B.; Boone, W. J. Developing a New Model to Provide First and Second-Year Undergraduates with Chemistry Research Experience: Early Findings of the Center for Authentic Science Practice in Education (CASPiE). Chem. Educ. 2006, 11 (2), 125−129. (18) Russell, C. B.; Bentley, A. K.; Wink, D. G.; Weaver, G. C. The Center for Authentic Science Practice in Education: Integrating Science Research into the Undergraduate Laboratory Curriculum. In Making Chemistry Relevant; Wiley: Hoboken, NJ, 2010; pp 193−206. (19) Jordan, T. C.; Burnett, S. H.; Carson, S.; Caruso, S. M.; Clase, K.; DeJong, R. J.; Dennehy, J. J.; Denver, D. R.; Dunbar, D.; Elgin, S. C.; Findley, A. M. A Broadly Implementable Research Course in Phage Discovery and Genomics for First-Year Undergraduate Students. mBio 2014, 5 (1), e01051−13. (20) Lopatto, D.; Alvarez, C.; Barnard, D.; Chandrasekaran, C.; Chung, H.-M.; Du, C.; Eckdahl, T.; Goodman, A. L.; Hauser, C.; Jones, C. J.; Kopp, O. R.; et al. Genomics Education Partnership. Science 2008, 322 (5902), 684−685. (21) Fay, M. E.; Grove, N. P.; Towns, M. H.; Bretz, S. L. A Rubric to Characterize Inquiry in the Undergraduate Chemistry Laboratory. Chem. Educ. Res. Pract. 2007, 8 (2), 212−219. (22) Tomasik, J. H.; Cottone, K. E.; Heethuis, M. T.; Mueller, A. Development and Preliminary Impacts of the Implementation of an Authentic Research-Based Experiment in General Chemistry. J. Chem. Educ. 2013, 90 (9), 1155−1161. (23) Tomasik, J. H.; LeCaptain, D.; Murphy, S.; Martin, M.; Knight, R. M.; Harke, M. A.; Burke, R.; Beck, K.; Acevedo-Polakovich, I. D. Island Explorations: Discovering Effects of Environmental ResearchBased Lab Activities on Analytical Chemistry Students. J. Chem. Educ. 2014, 91 (11), 1887−1894. (24) Winkelmann, K.; Baloga, M.; Marcinkowski, T.; Giannoulis, C.; Anquandah, G.; Cohen, P. Improving Students’ Inquiry Skills and SelfEfficacy through Research-Inspired Modules in the General Chemistry Laboratory. J. Chem. Educ. 2015, 92 (2), 247−255. (25) Ford, J. R.; Prudente, C.; Newton, T. A. A Model for Incorporating Research into the First-Year Chemistry Curriculum. J. Chem. Educ. 2008, 85 (7), 929−933. (26) Anunson, P.; Winkler, G. R.; Winkler, J. R.; Parkinson, B. A.; Schuttlefield Christus, J. D. Involving Students in a Collaborative Project to Help Discover Inexpensive, Stable Materials for Solar Photoelectrolysis. J. Chem. Educ. 2013, 90 (10), 1333−1340. (27) National Academies of Sciences, Engineering, and Medicine. Integrating Discovery-Based Research into the Undergraduate Curriculum: Report of a Convocation; The National Academies Press: Washington, DC, 2015. (28) Bangera, G.; Brownell, S. Course-Based Undergraduate Research Experiences Make Scientific Research More Inclusive. CBE Life Sci. Educ 2014, 13 (4), 602−606. (29) Duranton, F.; Cohen, G.; De Smet, R.; Rodriguez, M.; Jankowski, J.; Vanholder, R.; Argilés, A. Normal and Pathologic Concentrations of Uremic Toxins. J. Am. Soc. Nephrol. 2012, 23, 1258−1270. (30) Lisowska-Myjak, B. Uremic Toxins and Their Effects on Multiple Organ Systems. Nephron Clinical Practice 2015, 128 (3−4), 303−311. (31) Lopatto, D. Survey of Undergraduate Research Experiences (SURE): First Findings. Cell Biol. Educ. 2004, 3 (4), 270−277. (32) Lopatto, D. Undergraduate Research Experiences Support Science Career Decisions and Active Learning. CBE Life Sci. Educ 2007, 6 (4), 297−306. (33) Lopatto, D. CURE Survey. http://www.grinnell.edu/ academics/areas/psychology/assessments/cure-survey (accessed Mar 29, 2018). (34) Shaffer, C. D.; Alvarez, C.; Bednarski, A. E.; Dunbar, D.; Goodman, A. L.; Reinke, C.; Rosenwald, A. G.; Wolyniak, M. J.; Bailey,

C.; Barnard, D.; Bazinet, C.; et al. A Course-Based Research Experience: How Benefits Change with Increased Investment in Instructional Time. CBE Life Sci. Educ 2014, 13 (1), 111−130. (35) Cohen, J. Statistical Power Analysis for the Behavioral Sciences, 2nd ed..; Routledge: Hillsdale, NJ, 1988. (36) Corbin, J.; Strauss, A. Basics of Qualitative Research: Techniques and Procedures for Developing Grounded Theory, 3rd ed.; Sage Publications: Thousand Oaks, CA, 2008. (37) Corwin, L.; Graham, M.; Dolan, E. Modeling Course-Based Undergraduate Research Experiences: An Agenda for Future Research and Evaluation. CBE Life Sci. Educ 2015, 14 (1), 1−13. (38) Cohen, B. Explaining Psychological Statistics, 4th ed.; Wiley: Hoboken, NJ, 2013. (39) Seymour, E.; Hunter, A.-B.; Laursen, S.; Deantoni, T. Establishing the Benefits of Research Experiences for Undergraduates in the Sciences: First Findings from a Three-Year Study. Sci. Educ. 2004, 88 (4), 493−534. (40) Richmond, G.; Kurth, L. A. Moving from Outside to Inside: High School Students’ Use of Apprenticeships as Vehicles for Entering the Culture and Practice of Science. J. Res. Sci. Teach. 1999, 36 (6), 677−697. (41) Hanauer, D.; et al. Linguistic Analysis of Project Ownership for Undergraduate Research Experiences. CBE Life Sci. Educ 2012, 11 (4), 378−385. (42) Russell, C. B.; Weaver, G. C. A Comparative Study of Traditional, Inquiry-Based, and Research-Based Laboratory Curricula: Impacts on Understanding of the Nature of Science. Chem. Educ. Res. Pract. 2011, 12 (1), 57−67. (43) Hanauer, D. I.; Dolan, E. L. The Project Ownership Survey: Measuring Differences in Scientific Inquiry Experiences. Cell Biology Education 2014, 13 (1), 149−158. (44) Thiry, H.; Weston, T. J.; Laursen, S. L.; Hunter, A.-B. The Benefits of Multi-Year Research Experiences: Difference in Novice and Experienced Students’ Reported Gains from Undergraduate Research. CBE Life Sci. Educ 2012, 11 (3), 260−272. (45) Szteinberg, G. A.; Weaver, G. C. Participants’ Reflections Two and Three Years after an Introductory Chemistry Course-Embedded Research Experience. Chem. Educ. Res. Pract. 2013, 14 (1), 23−35. (46) Shaner, S. E.; Hooker, P. D.; Nickel, A. M.; Leichtfuss, A. R.; Adams, C. S.; de la Cerda, D.; She, Y.; Gerken, J. B.; Pokhrel, R.; Ambrose, N. J.; Khaliqi, D.; et al. Discovering Inexpensive, Effective Catalysts for Solar Energy Conversion: An Authentic Research Laboratory Experience. J. Chem. Educ. 2016, 93 (4), 650−657. (47) Shortlidge, E.; Bangera, G.; Brownell, S. Faculty Perspectives on Developing and Teaching Course-Based Undergraduate Research Experiences. BioScience 2016, 66 (1), 54−62.

K


Integrating Primary Research into the Teaching ... - ACS Publications

Recommend Documents