Reforming a Large Foundational Course: Successes and Challenges

Sep 24, 2017 - ... subject to a 3 year review to be completed in the summer of 2017. ..... Support Services in the CBC Department is greatly appreciat...
0 downloads 0 Views 665KB Size
Download PDF
This is an open access article published under an ACS AuthorChoice License, which permits copying and redistribution of the article or any adaptations for non-commercial purposes.

Article pubs.acs.org/jchemeduc

Reforming a Large Foundational Course: Successes and Challenges Vicente Talanquer* and John Pollard Department of Chemistry and Biochemistry, University of Arizona, Tucson, Arizona 85721, United States ABSTRACT: Calls for educational reform in undergraduate STEM education have become more prominent in recent years, particularly in introductory/foundational courses. Such reform efforts were initiated 10 years ago in the general chemistry program at the University of Arizona. In this contribution, we describe the major successes and challenges encountered during the full implementation of a new chemical thinking curriculum across all sections of a large course serving thousands of science and engineering majors every year. Besides describing the goals and structure of the alternative curriculum, as well as its impact on student learning, our work seeks to provide insights into institutional conditions, resources, and constraints that foster or hinder the success of major educational reforms. KEYWORDS: First-Year Undergraduate/General, Second-Year Undergraduate, Curriculum, Collaborative/Cooperative Learning, Inquiry-Based/Discovery Learning

■

INTRODUCTION Over the past three decades a variety of educators and educational researchers have expressed concerns about the limited educational impact of conventional curricula and teaching approaches in college science courses across the US.1−4 Calls for educational reform in colleges and universities have become louder and more frequent in recent years.5,6 Evidence from discipline-based educational research7 shows that traditional educational approaches, often focused on the didactic transmission of information, are less effective at fostering meaningful learning than strategies that more actively engage students with disciplinary content.8 Concerned chemistry educators have proposed and engaged in the implementation of diverse projects to improve student learning outcomes, particularly in foundational chemistry courses (i.e., general chemistry, organic chemistry). A majority of these projects have focused on developing, implementing, and evaluating the effects of more student-centered teaching strategies and instructional models.9 Well-known examples include the process-oriented guided inquiry learning (POGIL) educational model,10 the peer-led team learning (PLTL) pedagogy,11 and the flipped-classroom approach.12 These different “pedagogies of engagement” are content-neutral in the sense that they can be used independently of the nature of the curriculum. Fewer reform projects have proposed alternative curricular frameworks in which course content is reorganized to emphasize core ideas in a domain or facilitate the integration of concepts.13−15 These projects tend to be pedagogy-neutral as they do not necessarily require changes in teaching practices. Recently, some chemistry educators have advanced ideas for curricular reform in foundational general chemistry courses that involve a reconceptualization of how content is structured and how students engage with the course material. These reform proposals are based on results from research in chemistry and science education, and the attainment of their curricular goals demands the implementation of evidence-based teaching © XXXX American Chemical Society and Division of Chemical Education, Inc.

practices. One of such projects is the Chemistry, Life, the Universe and Everything (CLUE) curriculum16 structured around four crosscutting concepts: structure and properties, bonding and interactions, energy, and change and stability. Students following this curriculum actively engage in causal mechanistic reasoning to build explanations about how and why chemical phenomena occur. A second example is the chemical thinking (CT) curriculum,17,18 which is structured around eight essential questions that drive the chemical enterprise (see Table 1). In this curriculum, students actively engage in the Table 1. Essential Questions That Guide the Presentation, Discussion, and Integration of Content in the Chemical Thinking Curriculum Unit 1 2 3 4 5 6 7 8

Questions the Curriculum Addresses How How How How How How How How

do do do do do do do do

we we we we we we we we

distinguish substances? determine structure? predict properties? characterize chemical processes? predict chemical change? control chemical processes? synthesize substances? harness chemical energy?

development, analysis, and application of chemical ways of thinking that can be used to answer questions and solve problems in diverse contexts. This paper focuses on the implementation of this latter project. The main goal of this contribution is to describe the major successes and challenges encountered in the full implementation of the chemical thinking curriculum across all sections of general chemistry in a large, research-intensive public university. Published studies on educational reform at the Received: June 8, 2017 Revised: August 17, 2017

A

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

one-semester course, while Units 5−8 are addressed in a second semester. Course work in each unit and module of the CT curriculum is guided by a set of central ideas and performance expectations that are available to everyone online.23 These performance expectations include examples of the types of questions used to evaluate whether students have met the learning goals. Performance expectations are used to design in-class activities, homework tasks, and formative and summative assessments in the course. Given the strong emphasis on students’ developing productive ways of thinking, teaching practices seek to create learning environments in which students (a) actively work in small groups exploring data, building models, and constructing explanations; (b) have opportunities to share ideas in wholeclass discussions; and (c) apply their knowledge and reasoning in the resolution of questions and problems in diverse contexts. Work in the laboratory is organized around guided inquiry projects lasting from 2 to 4 weeks.

college level often present the results of interventions carried out in one or two sections of a course taught by one or two different instructors and composed of a relatively low numbers of students. The main goals of such publications are to describe the changes made and characterize their educational benefits. Although our paper shares some of these goals, we also seek to provide insights into institutional conditions, resources, and constraints that foster or hinder the success of major educational reforms involving thousands of students, hundreds of teaching assistants, and diverse faculty with varying teaching assignments. These insights are derived from 10 years of sustained efforts to transform a general chemistry program. Although the particular history, structure, and culture of colleges and universities are known to affect the outcomes of educational reforms,19 analysis of specific cases often helps increase our understanding of factors that affect educational change.

■

CHEMICAL THINKING

Implementation

Goals and Structure

The development of the CT curriculum began in 2007 with the support of a grant from the National Science Foundation (DUE-CCLI). These resources were used to create major components of the curriculum that were pilot-tested, evaluated, and revised at the University of Arizona (UA) and Pima Community College (PCC) in 2008 and 2009. From 2010 through 2014, the CT curriculum was used in two sections (∼240 students each) of the general chemistry course for science and engineering majors at the UA taught by the curriculum developers. During this period, the curriculum was refined and data were collected to compare performance of students enrolled in conventional and reformed sections. These data included results in a full-year ACS standardized final exam24 and students’ grades in the subsequent organic chemistry course. Additionally, different educational resources were developed to support the full implementation of the curriculum including a textbook, a set of baseline class notes, laboratory guides, web-based interactive simulations, educational videos, and assessment tasks (many of these resources are available online23). In spring of 2014, the Faculty in the Department of Chemistry and Biochemistry (CBC) at the UA voted to adopt the CT curriculum in all sections of the General Chemistry course for science and engineering majors starting fall of 2014. The adoption was approved subject to a 3 year review to be completed in the summer of 2017. The full implementation has involved an average of 2,400 students each semester distributed in 10 different lecture sections and 100 different laboratories. Lectures have been taught by 11 different instructors in the past three years, and work in the laboratories has been coordinated and supervised by over 50 different teaching assistants (TAs) each semester. Traditionally, general chemistry lecture classes are taught in large lecture halls with a capacity for 240−290 students. Starting in 2015, some class sections have been offered in a large Collaborative Learning Space (CLS) with a capacity for 264 students working on flat tables for six students each. The CLS has created better opportunities for implementing group activities that are an integral part of the chemical thinking curriculum, but also introduced new teaching challenges. Close to one-third of the students enrolled in General Chemistry every semester now take the class in the CLS.

Different authors have identified and described major issues that limit the educational impact of the conventional general chemistry curriculum for science and engineering majors.17,20−22 These include the following: segmented presentation of concepts that hinders integration of knowledge, strong emphasis on content coverage versus in-depth analysis of core ideas, dominance of memorization and algorithmic problem solving over conceptual understanding, and disconnect between course goals and students’ personal and professional interests and motivations. The chemical thinking (CT) curriculum was designed to address these issues by17,18 • Representing chemistry as a powerful way of thinking rather than as a static body of knowledge • Using essential questions in the discipline to guide, integrate, and give purpose to ideas discussed in the course • Emphasizing conceptual understanding of core concepts and ideas in chemistry • Building student understanding through carefully designed learning progressions based on existing research in chemistry education • Offering multiple opportunities for students to actively engage with core concepts and ideas through instructional tasks that ask them to analyze data, model chemical systems, and generate evidence-based arguments and explanations • Engaging students in thinking about important issues in four critical areas of interest for the science and technology of the 21st century: energy sources, environmental issues, life and medicine, and materials by design The CT curriculum is organized in eight units, each of them addressing an essential question that chemical thinking allows us to answer (see Table 1). Any given unit is subdivided into modules where core concepts and ideas are explored, built, and applied. Units 1−3 focus on the development and analysis of submicroscopic models of matter and structure−property relationships to explain, predict, and control physical properties of substances. Units 4−6 focus on the application of such models and relationships to explain, predict, and control chemical behavior. In Units 7 and 8 core ideas and ways of thinking are integrated and applied to the production of materials and energy. Units 1−4 are designed to span a first B

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

■

Article

PROMOTING AND PREPARING FOR CHANGE

analyzed each of the modules of the CT curriculum and reflected on potential implementation issues. • Development of resources: During the pilot-testing period, a variety of educational resources (e.g., textbook, class notes) were developed not only to support student learning but also to facilitate the work of other instructors who might want to implement the CT curriculum. • Dissemination and collaboration outside the department: Through different avenues (e.g., students’ comments, informal talks, formal presentations), people outside the CBC Department became aware of the CT curriculum. This opened the door for interactions with faculty and administrators who were interested in promoting change in foundational STEM courses. The CT project became a flagship for educational change at the UA, helping secure external resources to support reform efforts. This broader recognition further influenced the views of CBC faculty about the benefits of the project. To different degrees, our particular context and the factors listed above created the necessary environment for change with the support of a majority (∼80%) of the CBC faculty.

Context

The UA is a large, research-intensive public university with over 34,000 undergraduate students, 41% of them from underrepresented groups. The UA’s central mission is to provide broad access and educational opportunity to a diverse population of students, and its acceptance rate is close to 76% with an average SAT score of 1108. Around 2600 of the incoming freshmen declare a STEM major upon entering our university. Most of these majors are required to take one or two semesters of general chemistry, which the CBC Department offers as two 4 credit lecture-lab courses (150 min of lecture + 170 min of lab a week). The CBC Department is composed of 40 tenure-eligible and 10 non-tenure-eligible faculty. We will refer to them as research faculty and teaching faculty, respectively, based on their major professional interests and focus. Over 80% of all general chemistry sections have been taught by teaching faculty in the past 10 years. On average, close to 30 students are accepted into the CBC graduate program every year, and most of them work as teaching assistants in the general chemistry laboratories. The rest of the lab sections are taught by secondand third-year graduate students, as well as senior undergraduate students. All major curricular changes in the CBC Department are discussed and approved by the entire faculty. In general, most research faculty are not engaged in educational reform, but they are not necessarily resistant to it. Only a minority of these faculty are openly skeptical about the benefits of changing curriculum or teaching practices.

■

MAKING IT WORK

Creating a Framework

The full implementation of changes in our large general chemistry program was facilitated by the development of a framework within which all instructors agreed to work. This framework has enhanced collaboration in achieving common goals. The key components of this framework are • Common baseline instruction: Instructors use a common set of baseline class notes that define basic content and in-class group activities that all students are expected to complete. There is ample room for instructors to shape their course, but there is agreement on a set of core ideas, experiences, and ways of thinking to develop. • Common summative assessments: Instructors use common midterm and final exams to evaluate student learning. Midterm exams are developed collectively by the set of instructors, and ACS conceptual standardized exams are used as final evaluations. • Weekly coordination meetings: Course instructors meet weekly for at least 1 h to share experiences in their classrooms, discuss the implementation of upcoming course modules, and construct common summative assessments. • Peer observations: Instructors are encouraged to regularly visit each other’s classes, observe instruction, engage with students working in small groups, and provide feedback for each other. This is particularly useful when instructors join the general chemistry teaching team or when an instructor introduces a new teaching strategy or tool (e.g., using whiteboards, trying an alternative classroom response system). • Support for professional development: Instructors in general chemistry are strongly encouraged to participate in faculty learning communities on evidence-based teaching practices created at the UA. Economic resources obtained through the sales of the chemical thinking textbook are dedicated to support instructors’ participation in different chemistry education conferences and events every year.

Setting the Ground

Although accurately identifying the factors and conditions that allowed and led to the full transformation of the general chemistry program at our institution may be difficult, we highlight the major elements that facilitated the change: • Sponsored pilot-project: Initial support from the NSF to develop and pilot-test the CT curriculum was critical not only for bringing core ideas to fruition, but also for creating an official space to implement and evaluate new ideas in the classroom. • Collaboration between research and teaching faculty: The development of the CT curriculum was led by a research faculty with specialty in chemistry education and a teaching faculty recognized as one of the best instructors in the department. Their close collaboration over several years resulted into successful pilot-testing and garnered momentum for change. • Continuous collection of credible evaluation data: The collection of student performance data on ACS standardized final exams across all sections of the course, even when these assessments were not fully aligned with the CT curriculum, generated evidence of positive impacts on student learning using an assessment tool valued and accepted by all faculty. • Gradual involvement of other instructors: The CT curriculum was pilot-tested and refined over a six year period. During the first four years, instructors teaching conventional and reformed sections of general chemistry had informal interactions about the curriculum. Those interactions increased and were formalized through a biweekly meeting almost two years prior to full implementation. During those meetings, instructors C

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

• Continuous TA training: Incoming teaching assistants are involved in a two week training during the summer where they are introduced to basic ideas on how to support students working in guided inquiry projects. This training continues during the semester in 2 h weekly meetings where TAs carry out the lab projects and engage in discussions and reflections about teaching.

that the full-year ACS conceptual exam was used by our department. Data for S14 (N = 793) correspond to results from the ACS conceptual exam for students who completed the two semesters of general chemistry in conventional sections, and data for S15 (N = 1065), S16 (N = 1060), and S17 (N = 1049) correspond to results for students who completed the two semesters working with the reformed chemical thinking curriculum. The data in Figure 1 show that the introduction of the CT curriculum has led to a significant (p ≪ 0.001) improvement over the S14 results, although the effect size as determined by Cramers’ V = 0.21 is small. This value, however, is only slightly lower than those reported in meta-analyses of the effects of active learning interventions on student performance in courses with very large numbers of students.8 Based on national norms for ACS exams, UA student performance in general chemistry has moved from the 68th percentile in S14 to the 74th in the last three years. Average failure rates in the ACS conceptual exam (i.e., percentage of students with a grade lower than 60%) between S14 and S15−S17 have changed from 38.5% to 29.2%, corresponding to a 32% decrease in the number of students failing this test. Additionally, item by item analysis of questions in the conceptual ACS exam (see Figure 2) shows positive improvements in the comprehension of a majority of concepts for the CT versus the conventional approach. As summarized in Table 2, our analysis shows that educational benefits extend to different student populations. Average grades in the ACS conceptual exam in S14 (before CT implementation) for females and males were 62.3% and 69.1%, respectively. Average failure (i.e., percent of students with a grade lower than 60%) in this exam was at 46.7% for females and 27.2% for males. With the CT curriculum, the average grade in the ACS exam has moved up to close to 66% for females and 71% for males with no significant differences in these grades in the past three years. In the same period, the failure rate has decreased to 33.9% for females and 21.9% for males. The performance gap between females and males in the ACS conceptual exam disappears in the final grades awarded for the course (the final exam accounts for only 11% of the final grade). Females and males perform at a similar level in midterm exams, and females tend to outperform males in other assignments (e.g., lab work, homework). Gender differences in student performance in multiple choice assessments, despite equivalence in measures of academic preparation such as academic index, GPAs, and ACT/SAT scores, have been reported by several authors.25 The performance of students from under-represented (UR) groups (African American, Hispanic) has also improved under the CT curriculum. Average grades in the ACS conceptual exam in S14 (before CT implementation) for UR and white students were 60.6% and 67.7%, respectively. Average failure in this exam was at 52.9% for UR students and 28.6% for white students. With the CT curriculum, the average grade has moved above 63% for under-represented students and close to 70% for white students with no significant difference in this grades in the past three years. The failure rate has decreased to 41.3% for UR students and 24.7% for white students. Although smaller, the gap between these two populations of students persists in final course grades. Academic index, GPA, and ACT/SAT data indicate that these two populations of students are not equivalent, and we have begun to implement different interventions (e.g., supplemental instruction, offering a

Evaluation and Adjustment

One of the major challenges that we have faced in the 10 years since the beginning of the CT project has been the gathering of data that could be used to evaluate its educational impact. When introducing educational changes in large scale programs, people carrying out the intervention often do not have full control over the system being affected. For example, given departmental and institutional constraints, during the pilottesting we were not allowed to control enrollment to ensure that groups of students in conventional versus chemical thinking course sections were equivalent. Collecting comparative student performance data from conventional course sections was difficult for several other reasons: content sequence in conventional versus CT sections was different across the academic year, assessment instruments were not equivalent, and systematic collection of student data from conventional sections was not in place or was restricted. Over time, the CBC Department agreed to use ACS exams23 as final evaluations across all sections in both semesters of the general chemistry course. This allowed us to collect data in conventional sections which could be used as reference (historical data) to compare with results from the CT implementation starting in fall 2014. In the following paragraphs, we present a comparative analysis using the historical data that is available. The one year CT curriculum addresses most of the same core concepts and ideas as the conventional one-year general chemistry curriculum, but using a different framework and sequence. Comparisons in student learning outcomes are thus more appropriate at the end of the academic year. The graph in Figure 1 depicts the grade averages in the final full-year ACS conceptual exam across different semesters, from spring 2014 (S14) to spring 2017 (S17). Spring of 2014 is the first semester

Figure 1. Average percent grades in final ACS exams across different years in the general chemistry course at the UA. The red bar corresponds to results in the full-year ACS conceptual exam for students enrolled in conventional sections of general chemistry (S14; N = 793), and the blue bars correspond to results in the same ACS exam for students who completed one year of general chemistry using the CT curriculum (S15; N = 1065; S16, N = 1060; S17, N = 1049). D

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 2. Percent difference in the average number of students who answered correctly a given question in the ACS conceptual exam between CT sections in spring 2015 through spring 2017 and conventional sections in spring 2014 (prior to CT implementation). The questions have been arranged by topic (using different color bars per topic) from left to right in the following order: (a) Gases, Liquids, Solids, and IMFs (7 questions); (b) Chemical Quantities and Reactions (8 questions); (c) Atomic Structure and Periodicity (3 questions); (d) Chemical Bonding (5 questions); (e) Kinetics (4 questions); f) Thermodynamics (9 questions); (g) Equilibrium (5 questions); (h) Acid−Base (4 questions); (i) Electrochemistry (4 questions). Statistically significant differences in the responses to a question have values of |Δ| > 3%.

Table 2. Comparative Average Grades in the ACS Conceptual Exam for Diverse Students Based on Gender and Ethnicity Average Grade, % before (CONV) and after Chemical Thinking (CT) Implementation Student Demographics Female Male Under-represented White

Spring 2014 (CONV) 62.3 69.1 60.6 67.7

(N (N (N (N

= = = =

Spring 2015 (CT)

459) 333) 154) 397)

66.4 71.8 63.7 70.6

(N (N (N (N

= = = =

608) 406) 172) 541)

Spring 2016 (CT) 66.5 71.1 63.9 69.9

(N (N (N (N

= = = =

645) 414) 155) 513)

Spring 2017 (CT) 65.7 71.8 63.4 69.6

(N (N (N (N

= = = =

653) 387) 167) 537)

Figure 3. Distribution of letter grades in the first semester of organic chemistry at the UA from (a) fall 2012 to spring 2015 (for students who took the conventional general chemistry course), and from (b) fall 2015 to spring 2017 (for students who completed the CT curriculum).

this course has not changed significantly in years prior to the implementation of the CT curriculum. Figure 3a shows the average distribution of letter grades in this organic chemistry course from fall of 2012 to spring of 2015 (N = 4145), corresponding to students who completed the conventional general chemistry sequence (prior to full CT implementation). Figure 3b shows the course grade distribution from fall 2015 to spring 2017 (N = 2504) for students who took the CT curriculum. These distributions are significantly different with

preparatory chemistry course) to better support the work of our weaker students. As part of the evaluation, we have also monitored the performance of students in the first organic chemistry course that many science and engineering majors take after completing the general chemistry program. Letter grades in organic chemistry are assigned by each instructor using their own criteria and are not typically curved. However, analysis of existing data shows that the distribution of final letter grades in E

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

χ2 (4, N = 6649) = 54.9, p < 0.0001, but with a small effect size as measured by Cramers’ V = 0.09. Analysis of residuals reveals a significant increase in the percentage of students getting an “A” grade in the organic chemistry course and a significant decrease in the percent of grades “D” and “E” (failing) awarded in the past two years. Although other factors may have also contributed to the grade differences (e.g., instructors with different grading policies teaching the courses), the change in grading trend has only occurred in the last two years which suggests a positive effect of the CT curriculum on student preparation for the organic chemistry course. Decisions about data collection in the CT project have been strongly influenced by the major recipients of such data. Project evaluation has focused on gathering evidence of impact using data that faculty in the CBC Department would accept as valid and reliable. Results in standardized tests and distribution of final grades in different courses are measures that our faculty have found convincing. We have collected other types of data, such as students’ responses to an “intuitive chemistry” questionnaire built on the basis of results from chemistry ̈ alternative frameworks commonly education research on naive held by novice chemistry students.26,27 Comparative results with historical data using this questionnaire show significant (p ≪ 0.001) improvements in student performance of over half a standard deviation ( Cramers’ V= 0.53) while failure rates decreased by 37%. We have also used assessment tools developed by other chemistry education researchers, such as the “Implicit Information from Lewis Structures Instrument”,28 to explore the effects of the CT curriculum on students’ recognition of basic structure−property relationships. Our results show positive impacts that are similar to those reported for students using a different research-based curriculum (CLUE).29

■

CHALLENGES

Although the evaluation data collected in the past three years shows encouraging signs of improvement in general chemistry student preparation at our institution, some of these effects are small, and we are aware of several major challenges that we still have to face. These include • Fidelity of implementation: Eleven different instructors have participated in the full implementation of the CT curriculum in the past three years. Only four of them have taught the courses every semester, while others have joined or left the teaching team because of varying teaching assignments (on average, two different instructors have switched in/out the general chemistry program every year). Despite the existence of an educational framework that facilitates the integration of instructors into the program, systematic classroom observations conducted as part of an associated research project have revealed major differences in the way instructors implement the same classroom activities and guide students through whole-class discussions. These observations suggest that the nature and quality of the opportunities to learn created by instructors may be vastly different, and preliminary factor analyses indicate that such differences can significantly affect student performance in the course (mainly on midterm exams). Observations of different teaching assistants also reveal major differences in the teaching practices being used to guide student work in the laboratory. Ensuring fidelity of

■

implementation has become a priority in our reform efforts, but the large scale of the program and the limited human resources available to support instruction impose serious constraints on potential actions. • Student accountability in large class settings: Group work in the classroom is a critical component of the CT curriculum. This work is most effective when all students have invested time preparing for class tasks. In classrooms with 250 students, however, ensuring such preparation and making students accountable for their productive participation in collaborative activities is a major challenge. Instructors rely on a variety of tools that make individual student work more visible and facilitate formative assessment in the classroom (e.g., learning assistants, classroom response systems, whiteboards, short in-class quizzing). However, classroom observations suggest that a significant number of our students fail to prepare and engage meaningfully with the course content. To address this challenge, we are currently discussing and exploring the potential benefits of introducing elements of hybrid or blended instructional approaches into the program. • Aligned assessments: So far, evaluation of the impact of the CT curriculum on student learning has been mostly based on student performance on the ACS exam. This assessment tool is not fully aligned with the goals of the curriculum, but has helped ensure faculty support for the project. Data collection and analysis in large programs demands time and resources that need to be carefully allocated, and adding additional program evaluation measures is always a challenge. We are currently developing and testing alternative assessment tools (e.g., structure−property reasoning questionnaire) that better target the ways of thinking that the CT curriculum seeks to foster. • Program influence: Although existing data show the benefits of the CT curriculum, few CBC faculty outside from those involved in teaching general chemistry courses have shown interest in extending the approach to other areas. Work in the CT curriculum has motivated some CBC faculty to introduce evidence-based practices into their teaching, but so far there are no other curriculum reform efforts aligned with the CT framework within the CBC Department. Curiously, work on the CT project has had more influence on current reform initiatives in foundational courses in physics, biology, and chemical engineering in our university than within the boundaries of CBC. In each of these cases there has been a coalition of motivated individuals willing to pilottest ideas and push them forward. Using this strategy, we are currently working with a small group of organic chemistry instructors who want to begin building bridges between the two foundational chemistry courses by extending the project-based approach into the organic chemistry laboratories.

FINAL COMMENTS Our work and experiences in the development, implementation, and evaluation of the CT curriculum for general chemistry at the UA in the past 10 years suggest that educational reform in foundational courses in large, research-intensive public universities is possible, but it is a major challenge. Change F

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Undergraduate STEM Education Initiative. We want to thank all of the general chemistry instructors who have been involved in the CT project, as well as the undergraduate preceptors and graduate teaching assistants who have collaborated in its implementation over the years. The continuous support from the staff in the Teaching Support Services in the CBC Department is greatly appreciated. We give special thanks to Gail Burd, Senior Vice Provost for Academic Affairs at the UA, who has been a relentless supporter of the CT project.

actors may need not only to commit to a long, incremental, and sustained project, but also be ready to adapt to their particular context, compromise, strategically build alliances with faculty and administrators inside and outside their academic department, search for evidence of positive impacts that are convincing for their target audience, and recognize that achieving large size effects on student learning may require changes that go beyond the initial scope of the project. Looking at what has been achieved after several years of continuous work, one may have doubts about the worthiness of the effort. Although we have seen statistically significant improvements in student performance using different measures, most of these effects are not large. Whether this is due to intrinsic limitations of the CT curriculum or to some of the major challenges that we have identified (i.e., ensuring highfidelity and high-quality implementation in all course sections, increasing student readiness and accountability for learning) will have to be discerned in the future. There are, however, other types of impacts that, although less tangible and quantifiable, need to highlighted. For example, the implementation of the CT project has exposed thousands of undergraduate students and hundreds of graduate students to evidence-based teaching practices both in lecture and the laboratory. Through collaboration with other STEM departments, the use of these practices has spread out to other courses to the point that engagement in collaborative group work has become common in large science foundational courses at our institution. With the help of the central administration, instructors in the CT curriculum sparked the creation of Collaborative Learning Spaces (CLS) that could better support group work in large classrooms. Using a space in the science library adapted to accommodate ∼260 students, general chemistry instructors successfully pilot-tested the idea in the spring of 2015 and provided support to other instructors who wanted to teach in this classroom. By fall 2017, the UA will have 20 different CLSs to be used by over 250 faculty from different areas (beyond STEM). The CLS initiative has become one of the most successful agents of change at our institution and was born from the motivation and drive of faculty already involved in educational reform through the CT project. Our story suggests that implementing major educational change in a large service course in a public university may not only have diverse benefits for the many undergraduate students, graduate teaching assistants, and faculty involved in the project, but also have major impacts on other academic departments. Embarking in change could be daunting, but the extent of its influence is likely to make it worth it.

■

■

REFERENCES

(1) Tobias, S. Revitalizing Undergraduate Science: Why Some Things Work and Most Don’t; Research Corporation: Tucson, AZ, 1992. (2) Seymour, E.; Hewitt, N. Talking About Leaving: Why Undergraduates Leave the Sciences; Westview Press: Boulder, CO, 2002. (3) Atkinson, R. D.; Mayo, M. Refueling the U.S. Innovation Economy: Fresh Approaches to Science, Technology, Engineering and Mathematics (STEM) Education. Information Technology & Innovation Foundation; 2010. Available at http://www.itif.org/files/2010refueling-innovation-economy.pdf (accessed Aug 2017). (4) President’s Council of Advisors on Science and Technology (PCAST), White House. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics; National Academy Press: Washington, DC, 2012. (5) Association of American Universities (AAU). Framework for Systemic Change in Undergraduate STEM Teaching and Learning. AAU Undergraduate STEM Education Initiative; AAU, Washington, DC, 2013. (6) National Research Council (NRC). What Research Says About Effective Instruction in Undergraduate Science and Engineering; The National Academies Press: Washington, DC, 2015. (7) National Research Council (NRC). Discipline-Based Education Research; National Academy Press: Washington, DC, 2012. (8) Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active Learning Increases Student Performance in Science, Engineering, and Mathematics. Proc. Natl. Acad. Sci. U. S. A. 2014, 111, 8410−8415. (9) Eberlein, T.; Kampmeier, J.; Minderhout, V.; Moog, R. S.; Platt, T.; Varma-Nelson, P.; White, H. B. Pedagogies of Engagement in Science. Biochem. Mol. Biol. Educ. 2008, 36 (4), 262−273. (10) Moog, R. S.; Spencer, J. N. Process Oriented Guided Inquiry Learning (POGIL); ACS Symposium Series; American Chemical Society, Washington, DC, 2008. (11) Gosser, D.; Kampmeier, J.; Varma-Nelson, P. Peer-Led Team Learning: 2008 James Flack Norris Award Address. J. Chem. Educ. 2010, 87, 374−380. (12) Seery, M. K. Flipped Learning in Higher Education Chemistry: Emerging Trends and Potential Directions. Chem. Educ. Res. Pract. 2015, 16, 758−768. (13) Esterling, K. M.; Bartels, L. Atoms-First Curriculum: A Comparison of Student Success in General Chemistry. J. Chem. Educ. 2013, 90 (11), 1433−1436. (14) Lafarge, D. L.; Morge, L. M.; Meheut, M. M. A New Higher Education Curriculum in Organic Chemistry: What Questions Should Be Asked? J. Chem. Educ. 2014, 91, 173−178. (15) Schaller, C. P.; Graham, K. J.; Johnson, B. J.; Fazal, M. A.; Jones, T. N.; McIntee, E. J.; Jakubowski, H. V. Developing and Implementing a Reorganized Undergraduate Chemistry Curriculum Based on the Foundational Chemistry Topics of Structure, Reactivity, and Quantitation. J. Chem. Educ. 2014, 91, 321−328. (16) Cooper, M.; Klymkowsky, M. Chemistry, Life, the Universe, and Everything: A New Approach to General Chemistry, and a Model for Curriculum Reform. J. Chem. Educ. 2013, 90, 1116−1122. (17) Talanquer, V.; Pollard, J. Let’s Teach How We Think Instead of What We Know. Chem. Educ. Res. Pract. 2010, 11, 74−83.

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. ORCID

Vicente Talanquer: 0000-0002-5737-3313 Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS We acknowledge the support of the National Science Foundation (NSF) through Grant DUE-0736844 and from the Association of American Universities (AAU) through their G

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(18) Sevian, H.; Talanquer, V. Rethinking Chemistry: A Learning Progression in Chemical Thinking. Chem. Educ. Res. Pract. 2014, 15, 10−23. (19) Henderson, C.; Beach, A.; Finkelstein, N. Facilitating Change in Undergraduate STEM Instructional Practices: An Analytic Review of the Literature. J. Res. Sci. Teach. 2011, 48, 952−984. (20) Gillespie, R. J. What is Wrong with the General Chemistry Course? J. Chem. Educ. 1991, 68, 192−194. (21) Hawkes, S. J. (2005), Introductory Chemistry Needs a Revolution. J. Chem. Educ. 2005, 82, 1615−1616. (22) Cooper, M. The Case for Reform of the Undergraduate General Chemistry Curriculum. J. Chem. Educ. 2010, 87 (2), 231−23. (23) Chemical Thinking Website; https://sites.google.com/site/ chemicalthinking/structure (accessed Aug 2107). (24) For more information about ACS exams visit https://uwm.edu/ acs-exams/ (accessed Aug 2017). (25) Ben-Shakhar, G.; Sinai, Y. Gender Differences in MultipleChoice Tests: The Role of Dfferential Guessing Tendencies. J. Educ. Meas. 1991, 28, 23−25. (26) Talanquer, V. Common Sense Chemistry: A Model for Understanding Students’ Alternative Conceptions. J. Chem. Educ. 2006, 83, 811−816. (27) Talanquer, V. (2013). Threshold Concepts in Chemistry: The Critical Role of Implicit Schemas. J. Chem. Educ. 2015, 92, 3−9. (28) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z. Development and Validation of the Implicit Information from Lewis Structures Instrument (IILSI): Do Students Connect Structures with Properties? Chem. Chem. Educ. Res. Pract. 2012, 13, 195−200. (29) Cooper, M. M.; Underwood, S. M.; Hilley, C. Z.; Klymkowsky, M. W. Development and Assessment of a Molecular Structure and Properties Learning Progression. J. Chem. Educ. 2012, 89, 1351−1357.

H

DOI: 10.1021/acs.jchemed.7b00397 J. Chem. Educ. XXXX, XXX, XXX−XXX