A Parallel Controlled Study of the Effectiveness of a Partially Flipped

Sep 22, 2016 - All surveys were administered during class time on paper, with the option of a link to a SurveyMonkey form for the three university fli...
1 downloads 0 Views 864KB Size
Download PDF
Article pubs.acs.org/jchemeduc

A Parallel Controlled Study of the Effectiveness of a Partially Flipped Organic Chemistry Course on Student Performance, Perceptions, and Course Completion James C. Shattuck* Department of Chemistry, University of Hartford, West Hartford, Connecticut 06117, United States S Supporting Information *

ABSTRACT: Organic chemistry is very challenging to many students pursuing science careers. Flipping the classroom presents an opportunity to significantly improve student success by increasing active learning, which research shows is highly beneficial to student learning. However, flipping an entire course may seem too daunting or an instructor may simply choose to use this approach selectively. This exploratory, mixed-methods study examines the effectiveness of a partially flipped course in the first semester organic chemistry course. Two sections were taught by the author in Fall 2015: a control section (n = 28 students) using a lecture-based format, and a flipped section (n = 26 students), where 8, 75 min classes (a third of the course) were taught with flipped pedagogy. Significant improvements in test questions on flipped topics were observed, as well as a significant reduction in the course withdrawal rate. While the average overall course grade was similar in the two sections, the flipped section had 25% more A’s and B’s. Survey and focus group data show that by the end of the semester students in the flipped section felt significantly more confident with the course material than the control section. As measured by student surveys over the course of the semester, students in the flipped section showed a significant change in their preferred type of instruction from lecture to a more collaborative approach, and also showed a significant increase in their comfort level with working in groups and using active learning strategies. KEYWORDS: Chemical Education Research, Organic Chemistry, Second-Year Undergraduate, Internet/Web-Based Learning, Collaborative/Cooperative Learning, Computer-Based Learning, Student-Centered Learning FEATURE: Chemical Education Research

■

solving, a form of active learning, flipping the classroom can enhance teaching effectiveness in the chemistry classroom. It is important to place flipping pedagogy into the context of educational theories to better understand its merits. Seery6,7 suggests that cognitive load theory helps explain the value of flipping. According to this theory, new learners have a limited processing capacity. A lecture containing too much new and difficult material in a short time period will quickly overwhelm a learner’s capacity, resulting in limited learning. Seery cites the recent work of Abeysekera and Dawson,8 suggesting that the pacing in the flipped approach, with introductory videos before class, may benefit students by reducing cognitive load and therefore aid learning. This notion is further supported in studies,9 including Schroeder, McGivney-Burelle, and Xue’s work in flipping the calculus course.10 Students in the flipped math course were shown to work more attentively for longer periods of time, effectively spreading out their learning.

INTRODUCTION

Organic chemistry instructors realize that practice in solving problems, coupled with instructor feedback, is the most effective way for students to learn the most challenging concepts. But in-class problem solving is time-consuming and can be at odds with the competing good of content coverage.1 One avenue for addressing this dilemma was introduced to the chemistry community through the work of two high school teachers, Bergmann and Sams, with the flipped classroom.2 By introducing students to the course material through videos prior to class, class time is freed up for more robust active learning activities. Compared to traditional lecture-based teaching, active learning strategies such as collaborative learning enhance student success in attaining learning outcomes more effectively.3 In their large scale meta-study, Freeman and coworkers4 and Weiman5 showed that student test performance, on average, improved by 6% for students in courses with active learning and students in pure lecture courses were 1.5 times more likely to fail than students in courses involving active learning. By creating more class time for group problem © XXXX American Chemical Society and Division of Chemical Education, Inc.

Received: May 27, 2016 Revised: August 26, 2016

A

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Increased student time on task is a prime benefit of flipping pedagogy. Flynn11 cites the work of Cooper et al.12 and Bodner13 in applying a constructivist framework to flipping. Students need to build new knowledge upon existing knowledge and make meaningful connections all within a social context.12 In flipping pedagogy, preclass work uses prior knowledge to create a foundation, upon which in-class work then builds, creating meaningful connections within the content.11 In addition, group problem solving provides a social dimension to aid learning.13 Studies of the flipped classroom are reported in a variety of university chemistry courses. A review of these studies was published by Seery recently.6 Flipping the classroom was reported in chemical thermodynamics,14 inorganic and organic chemistry laboratory,15 and analytical chemistry.16 Ryan and Reid conducted one of the few reported parallel controlled studies of the flipped chemistry classroom where the same instructor taught both the flipped and control courses. Students selected the flipped or traditional section upon enrollment in a nonmajors general chemistry course. The bottom third of the students in the flipped section of the course saw statistically significant improvement on all five course exams.17 In addition, there was a 56% decrease in the DFW rate (percentage of grades of D, F, or withdrawal). In the first year general chemistry course, several studies on the flipped classroom recently appeared in the literature. Smith studied student attitudes toward the flipped classroom in general chemistry.18 Students reported that the flipped classroom was more engaging (65% agreeing) and more effective (60% agreeing) than a traditional lecture course, but 48% reported that the flipped class required significantly more time than a traditional class. In a different study, Butzler analyzed the underlying factors that caused a difference in grades between a flipped and lecture section of general chemistry. She determined that differences in math background and class rank accounted for 18 and 21%, respectively, of the grade variance.19 In a five year study of the majors sections of general chemistry, Hibbard, Sung, and Wells reported statistically significant differences in ACS exam scores between flipped sections and traditional sections from prior semesters with a small to moderate effect size.9 Weaver and Sturtevant’s three year study of the majors sections of general chemistry showed significant improvement on ACS exam scores in the flipped section compared to a more traditional format taught by a different professor.20 After subdividing the ACS exam scores into scores for algorithmic and conceptual questions, they generally saw greater gains in the conceptual questions. Weaver and Sturtevant also noted that students most benefit from flipped pedagogy when taught how to structure their study time and how to best utilize the out of class resources.20 Eichler and Peeples report on a parallel controlled study where 29% of the classroom sessions were flipped in a large enrollment general chemistry course.21 While significant differences between sections in final exam scores or the DFW rate were not observed, the flipped section showed significantly higher overall course grades as well as a decrease in grades of C and an increase of grades of A and B. The use of flipping pedagogy in organic chemistry was evaluated in a handful of studies. Ealy reported on a hybrid approach and offered suggestions for flipping implementation including checking students’ notes from the videos to confirm preclass work completion.22 Rossi saw significant improvement on three exams using the flipped approach at a two-year

college.23 There was high satisfaction (80%) with the flipped format, and 54% of the students felt that Organic I was understood greater or much greater than had it been delivered in the traditional format. Christiansen conducted student surveys at weeks 5 and 15 in a flipped organic chemistry course and saw that students had reservations about the flipped classroom in week 5 but by week 15 reported greater preference for flipped teaching over traditional lecture.24 He concluded that students require an adjustment period to learn how to study in a flipped class. Flynn selected learning outcomes in her organic chemistry and spectroscopy courses that were best suited for flipped pedagogy.11 She observed a statistically significant improvement in overall course grades and a reduction in the DFW rate in comparing two large flipped organic chemistry sections (364 and 409 students) to data from the previous four years. Fautch suggested that a primary advantage of flipping pedagogy is to assist students who otherwise would have withdrawn from the course.25 During class time during student group work, she would interact with each student individually, ask questions, and be sure each student was prepared for the class. By providing individualized support and accountability, the weaker students in particular were better poised to succeed. Fautch saw the DFW rate decrease compared to historical data, but final ACS exam scores were unchanged. Like Christiansen, Fautch noted the importance of encouragement from the instructor early in the semester as students adapted to the flipped approach. Trogden reported on a parallel controlled study involving a partially flipped course, where one of three class periods per week was flipped.26 The DFW rate decreased by 11% in the flipped section, and the average grade also showed slight improvement, with a pronounced increase in grades of B. In all of these studies, it is clear that there is little published research on the effectiveness of flipping a subset of classes, rather than every class, in a university level chemistry course. In addition, the literature primarily focuses on data collection using overall course grades and final exam scores. While one study20 reported a comparison of algorithmic vs conceptual final exam questions, an analysis of the effectiveness of flipping on specific topics in organic chemistry has not been reported, to the best of the author’s knowledge. The most challenging topics in the first semester organic chemistry course, requiring higher-order thinking skills in Bloom’s taxonomy, can most benefit from instructor guided practice during class. For this experiment, four topics were chosen for flipping: 1. Arrow notation for acid/base, alkene and aromatic reaction mechanisms 2. Predicting products, reactants, or reagents for acid/base, alkene, and aromatic reactions 3. Using Hückel’s rules to determine aromaticity 4. Multistep aromatic synthesis Eight classes were taught with flipped pedagogy to address these topics. Test performance data, student surveys, and a focus group analysis were conducted. This study received approval by the university’s Institutional Review Board (IRB).

■

RESEARCH QUESTIONS The following are the five main research questions in this study: • How does a partially flipped organic chemistry course affect student test performance on flipped topics, withdrawal rate, and student perceptions of their learning? B

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

■

Article

• How does flipping pedagogy affect student perception of both the difficulty level and their interest level in organic chemistry? • Do student perceptions of time spent outside of class studying organic chemistry differ between a traditional and a partially flipped section? • How does the experience of flipping the classroom affect student perceptions of collaborative learning? • How does the experience of flipping the classroom affect student perceptions of their preferred type of instruction?

Table 2. Major and Race for Flipped and Control Sections

METHODS AND FRAMEWORK

Population Studied

a

Flipped

Control

26 9 17 538 (104)a 526 (86) 1064 61.5% 3.16 (0.73) 3.17 (0.53)

28 10 18 518 (54) 526 (69) 1044 78.5% 3.00 (0.86) 3.03 (0.66)

Control

1 5 10 4 3

2 2 10 5 6 1 1

1 1 1 Flipped (%)

Control (%)

35.7 10.7 28.6 10.7 7.1 7.1

50 21.4 14.2 0 0 14.2

colleges. In the department of chemistry, within the College of Arts and Sciences, the ACS certified B.S. degree in chemistry is offered, as well as a B.S. degree in the chemistry−biology joint major, which is our biochemistry major. Two sections of CH 230, Organic Chemistry I, both taught by the author, were offered in the Fall semester and met for 75 min twice a week with one 3 h laboratory period. All exam questions on flipped topics were the same in both sections. Both sections had the same syllabus, same assigned homework questions from the textbook (McMurry’s Organic Chemistry with Biological Applications, 2nd ed., Brooks/Cole27) and access to the three 1 h review sessions scheduled throughout the week. All review sessions and all class sessions were recorded using Echo360 lecture capture software.28 These recordings were made available to both the flipped and control sections through the course management system, Blackboard Learn. The assigned textbook homework problems (four to 13 problems per class period, with an average of eight problems per class) were collected weekly and graded for completion. A 3% bonus was given for each of the three class exams for each student who submitted the completed homework on time. The control section of the course was composed of 85% lecture with 15% lower level active learning activities, described as follows. For about 5% of course time, interspersed throughout a lecture, the class was given short problems to solve, either working alone or with a neighbor, and then a student was randomly called on to present their solution, followed by large group discussion. In addition, students were randomly called upon to answer questions and contribute to lecture presentation, for about 10% of the course. This was equivalent in the flipped section of the course for the two-thirds of the classes that were not flipped. For each of the eight flipped classes in the flipped section, students were required to watch 3−6 instructor created videos prior to each flipped class. While students in the control section did not have access to these videos, these videos were essentially identical to the lectures delivered in class on those topics. Each video averaged 10.3 min in length with a range of 1 to 15 min. The total video time per class ranged from 29 to 48 min with an average of 37.5 min. Videos consisted of annotated PowerPoint slides with voice-over instructor narration. A partial set of notes on the PowerPoint slides was distributed at least 1

Table 1. Comparison Data on Flipped and Control Sections Parameters

Flipped

Caucasian African American Hispanic Asian Two or more races No reply

In Fall 2016, two sections taught by the author were offered of CH 230, Organic Chemistry I, the first semester of a twosemester course sequence required of biology, chemistry, and health science majors. After students had enrolled, one of these sections was arbitrarily selected to serve as a flipped section, where eight lectures were flipped, and the other section (control) was taught using traditional (predominantly lecture) format. Table 1 summarizes the demographics of the two sections.

Number of students Gender: M Gender: F SAT: V SAT: M Total SAT % reporting SAT score General Chemistry grade GPA

Major Chemistry Chemistry/biology Biology Health science Post-baccalaureate Biomedical engineering Spanish/psychology Respiratory therapy Undecided High school Race

Standard deviations are reported in parentheses.

None of these differences were determined to be statistically significant using a one-tailed t test of two independent means to the p = 0.05 significance level. In addition, all student transcripts were checked for any evidence of prior organic chemistry coursework, which was noted for two students in each section. An instructor generated 12 point pretest of basic organic chemistry knowledge (included in Supporting Information) was administered to both sections, and no student in either section had a passing grade. Therefore, both sections appeared very similar in terms of academic ability and prior organic chemistry knowledge. Comparison data on race and major is included in Table 2 showing an even distribution of majors and good diversity in both sections, with greater diversity than the overall Hartford student body (56% Caucasian). While the flipped section did have three more chemistry/biology majors, the control section had three more post-baccalaureate students. One may predict that majors would perform better in the course, but the same can be said of post-baccalaureate students, a group that tends to be highly motivated.

■

COURSE DESIGN The University of Hartford is a comprehensive, private, and predominantly undergraduate university composed of seven C

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 3. Quiz and Exam Performance Data and Analysis Flipped

Control

Topic

n

Score

Std. Dev

n

Score

Std. Dev

p Value

Cohen’s d Value

Resonance quiz ques Resonance Test 1 ques Acid/base Test 1 ques Addition to conjugated diene mech. Test 2 ques Final exam alkene mechanism ques Hückel’s rules Test 3 ques Aromatic reaction products Test 3 ques Multistep synthesis Test 3 ques

27 26 26 25 22 22 22 22

2.74/4 8.98/12 9.46/13 5.64/8 10.02/14 8.02/10 12.36/19 11.2/15

1.1 2.49 3.02 2.9 2.78 1.81 3.51 2.93

26 28 28 24 21 21 21 21

1.58/4 7.30/12 7.18/13 4.24/8 8.62/14 6.60/10 9.38/19 9.5/15

1.31 3.24 4.01 2.64 2.44 1.74 3.64 3.4

0.000422 0.0193 0.0113 0.0387 0.0433 0.00595 0.00489 0.0514

0.96 0.58 0.64 0.51 0.54 0.8 0.83 0.54

86% on the final. The surveys were face validated, and reliability was assessed using Cronbach’s alpha (0.84, 0.80, and 0.80 for the three surveys, respectively). In addition, two instructorgenerated student perception surveys, with different versions for the flipped and control sections, were administered midway through the flipped content and on the last day of the semester. The questions were initially generated by the author, who has 20 years of experience in college teaching. Following the generation of the items, they were distributed to three expert faculty members to determine if the items adequately assessed the desired information. The panel of professors agreed that the items captured the information and hence had face validity. The surveys for the flipped section had 4 descriptive items and 7 Likert scale items, with the final survey including two free response questions. The surveys for the control section had 6 items to obtain descriptive information. Reliability was assessed using Cronbach’s alpha (0.81 for the midsemester survey and 0.85 for the final). The response rate for the midsemester survey was 96% for the flipped section and 88% for the control section. For the final survey, the response rate was 91% for the flipped section and 86% for the control section. All surveys were administered during class time on paper, with the option of a link to a SurveyMonkey form for the three university flipping surveys. There was 100% participation in the focus group during class time at the end of the semester. This study, all surveys, and the focus group were approved by the University’s Human Subjects Committee and our IRB, and all student participants completed informed consent forms. Copies of all surveys and the student focus group questions are included in the Supporting Information.

week prior to each flipped class. The videos were created using the free web-based recording software, Screencast-O-Matic.29 Video files were then uploaded to Ensemble Video,30 the university’s online video platform, which students then accessed through Blackboard Learn. Students were required to take notes during the videos and fill in the missing content on the PowerPoint slides. As a formative assessment, several practice problems were embedded in the videos to be completed prior to class. Preclass notes and problems were checked in class and counted as two extra-credit quiz grades in addition to the ten regular quizzes. Students viewed this as a generous incentive and readily did the work, but in reality it resulted in only a 0.5% grade addition to the students in the flipped section. During a flipped class period, students were allowed to select their own groups of three to four students. Students were told to include at least one new group member for each subsequent flipped class. The student groups discussed the problems from the videos and came to a consensus on the correct answers during the first five to ten min of class. During this time, the instructor checked each student’s notes and preclass problems. The next five to ten min was spent in large group instructor-led discussion of the correct answers for the problems. Minilectures were also given during this time, as needed. Then, a set of four to six instructor-created problems was given to each group to complete within the next 40 min. Each problem set was scaffolded, building to more challenging problems. The instructor visited with every student to ask and answer questions. Since the classroom consisted of fixed tables, it was challenging for the instructor to access student groups in the middle of a row. Ideally, a classroom with round tables would ease instructor movement and promote student discussion. Toward the end of the period, one student from each group was selected by the instructor to present a problem on the board. The discussion of these answers took the remaining 15 min of the period.

■

RESULTS AND DISCUSSION

Test Performance and Final Grades

■

Table 3 summarizes the results from quiz and test questions on flipped topics where the flipped section scored significantly better than the control section. All differences in this table are statistically significant, except the last entry, as measured with a one-tailed t test for independent samples (p < 0.05), with medium to large effect sizes, as measured by Cohen’s d parameter. Two classes that occur early in the semester were flipped. These classes covered the topics of drawing resonance structures and using arrow notation in acid/base reactions to show electron movement and drawing reaction products based on those arrows. Exposing students to flipping pedagogy within the first weeks of the semester allowed students to become acclimated to flipping before tackling more challenging flipped material. This follows Fautch’s and Christiansen’s recommendation to allow students time to adapt to the flipped

ASSESSMENTS Students in both sections were assessed with ten weekly quizzes (10%), three exams (45%), and a comprehensive final exam (20%). The laboratory portion accounts for the remaining 25% of the course grade. The final exam and test items relating to any flipped topics were identical for the flipped and control sections. Three surveys and a student focus group were administered to the flipped section as part of a university-wide study on the efficacy of flipping pedagogy. These Likert scale surveys were administered early in the semester, midway through the flipped content, and on the last day of the semester. The first survey was composed of 18 items, and the second and third surveys had 29 items. Response rates on the three surveys were 81% on the first, 72% on the second, and D

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

method.24,25 Quiz and test questions showed that the flipped section performed significantly better than the control section on these topics. Interestingly, on the three sections of the first exam that dealt with topics that were not flipped, there were no significant grade differences and the control section average was higher in two of these three sections. Several weeks later, the bulk of the flipped content was taught on alkene and aromatic chemistry (3 classes on each topic). For alkene reactions and mechanisms, statistically significant improvement in the flipped class was noted on the reaction mechanism question on Test 2 (addition of HX to a conjugated diene) and on the final exam (addition of X2 to an alkene). In comparing nonflipped topics on the second exam, there were no significant differences between the sections except on an alkane reaction mechanism question (free radical halogenation), where the flipped class’s higher score was statistically significant. It is reasonable that the flipped coverage of alkene mechanisms was helpful in learning this mechanism as well. While the flipped section scored higher on Test 2 questions on predicting the products of alkene reactions, the difference was not statistically significant. However, the control class scored slightly higher on alkene reaction questions that involved predicting the missing reagent or the missing reactant, although the differences were not statistically significant. In the flipped class, emphasis was placed on applying reactions to solve new problems; however, the requirement to memorize the reagents was not emphasized as heavily as in the control section. Failure to memorize fundamentals would obscure any gains made in application, which was emphasized in class. As explained by Hartman and co-workers in their recent publication,31 the cognitive science model of reasoning holds that working memory can only process three to five concepts that are not thoroughly memorized. Therefore, in chemistry problem solving, it is important to have students drill fundamentals into long-term memory to be optimally successful. In response to the performance of the flipped section on alkene reaction Test 2 questions, students were required to create reaction flash cards as part of their preclass work on the next flipped reaction topic, aromatic chemistry. During the class period, students were asked to compare cards, find similarities and differences among reactions, note limitations, and quiz each other. On the third exam, the flipped section outperformed the control in all topics of aromatic chemistry with statistically significant improvements on aromatic reactions, as well as in using Hückel’s rules to predict aromaticity. On the most challenging topic, multistep aromatic synthesis, the flipped class scored better but the difference was not significant. However, the flipped section had the highest score observed in the past eight years on the aromatic multistep synthesis question on the third test. In addition, the flipped class scored significantly higher on the nonflipped section of the third test on stereochemistry. This may suggest that the increased engagement provided by the flipped classes spilled over to other content areas taught in a more traditional way. For overall assessment of the course, the flipped class scored higher than the control class on each of the three tests by 3.6 to 11%. Yet the flipped class scored just 0.6% higher on the final exam. The final course average for all students earning a letter grade in the course was 77.2% in the control section and 80.8% in the flipped section, which is not statistically significant. Final grades for both sections for the course are shown in Figure 1.

Figure 1. Overall course grade in CH 230 (Organic Chemistry I) for flipped (n = 26) and control sections (n = 28).

To clarify the grades of “pass” or “no pass”, at University of Hartford, a student may elect to take one course per semester with this grade option. On the transcript, the grade signifies course completion, but does not factor into a student’s GPA. Figure 1 shows that, in spite of the similar overall course averages, there were 25% more A’s and B’s in the flipped section than in the control. The lack of grades of F in both sections is not unusual since students on a trajectory to fail are strongly advised to withdraw, but it is striking that the withdrawal rate in the flipped section is 54% lower than that in the control section. This significant difference is convincing evidence that even a partially flipped course has a dramatic impact on student success. This difference in withdrawal rates indicates that more struggling students completed the course in the flipped section. As noted by Freeman and co-workers,4 one would expect this to lower the final average of the flipped section, potentially masking some positive benefit of active learning through flipping pedagogy. Student Focus Group and Survey Data

Survey data were used to compare student perceptions in both sections at two intervals, midway through the flipped content and at the end of the term. There was no significant difference in student attendance between the flipped and control sections. Also, students in both sections equally agreed that organic chemistry is an interesting topic, with 89% of the control section and 95% of the flipped section agreeing or strongly agreeing in the final survey. In comparing individual student survey results in each section between midsemester and end of the term, 5% of the flipped class showed an increase in interest in organic chemistry, while 12.5% of the control section reported a decrease in their level of interest. This suggests that flipping the classroom helped sustain student interest in the course throughout the term. Student perception of the level of challenge, a measure of their confidence in the course, is shown in Figure 2. In the midst of the flipped content, the flipped section reported a slightly higher level of challenge than the control section (2.29 vs 2.45 where 1 = too hard and 4 = easy). This midsemester data is consistent with an initial period of adjustment to the flipping approach that has been reported by others, including Christiansen.24 As the course progressed, students in the flipped section felt better able to meet the demands of the course at the same time that students in the control section felt a loss of confidence. In the final survey, the control section felt more challenged in the course than the flipped section and the E

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 2. Comparison of level of challenge in flipped and control sections.

Figure 3. Comparison of total hours spent outside of class working on organic chemistry in flipped and control sections.

correlate this tracking data with the date of accessing the video file to draw the following conclusions. All students accessed all of the required preclass videos. By the end of the semester, 52% of the class was watching the videos over 2 days. Students rewatched the videos at least once in completing homework (22% of the class), in studying for an exam (26%), and in preparing for the final exam (26%). Student survey responses and focus group comments often stated that one of the best benefits of flipping was the ability to rewatch the videos in studying, to pause them as needed, and to watch them at a time of day when they were mentally alert. Many students felt more prepared for the flipped class than for their other classes, and they felt that the ability to watch videos multiple times helped them to learn much more effectively. A few students did comment that watching videos was time-consuming, some videos were too long, and one did not have the ability to ask the professor a question immediately during a video as one can do in a lecture. In the final survey, 90% of the class agreed or strongly agreed that the videos were effective in helping to learn the course topics. Students in both the flipped and control sections were asked to report total hours per week spent out of class on organic chemistry, with results shown in Figure 3. While none of the differences between sections are statistically significant, the flipped class on average reported spending more time (midsemester survey = 6.7 h, end of semester survey = 6.5 h) outside of class on organic chemistry than the control class (both midsemester and final surveys = 6.1 h). Based on this data, the flipped section reported spending only about 30 min on average more per week on the course than the control

difference is statistically significant (flipped = 2.55, control = 2.06, p = 0.0031). In comparing matched surveys from midsemester to end of semester, 37% of the control class had an increased perception of course challenge vs 11% of the flipped section. For the flipped section, 21% reported a decreased perception of challenge while no students in the control section reported a decrease in the challenge level of the course. Some of the increased confidence of students in the flipped section is likely due to students’ in-class experiences. In-class problem solving allowed for immediate and individualized instructor feedback. In survey responses and focus group comments, students appreciated the in-class problem sets, writing answers on the board, and felt that they had more time to ask questions in class. One student commented, “In past classes, listening to the material for the first time during lecture was sometimes difficult. The feature of flipping made it easier to hear after I had an idea of the material already. It also assured me to bring questions if I had any.” Others added that flipping “made the topic a bit easier to understand and remember”, and others liked that the professor was constantly walking around the room to check students’ work and to answer questions, which greatly aided their understanding. In the flipped section surveys, students were asked to comment on the videos and the time spent in watching them. In the midsemester survey, students reported spending an average of 1.85 h (standard deviation = 0.60) per week in watching videos outside of class. This is slightly more than the average of 1.25 h of video content for a week. Since BlackBoard Learn tracks each time a student opens a video file, one can F

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 4. Comparison of student perception of preferred type of instruction in flipped and control sections.

section. The partially flipped course does not appear to require significantly more student effort outside of class versus the traditional lecture model. While more time is spent before class in watching the videos, less time may be required to complete the homework problems since significant class time was devoted to problem solving. A future study will look into this further. In comparing matched student surveys from the midpoint to the end of the semester, a net 11% of students in the flipped section reported an increase in their study time, while a net 6.5% of students in the control section reported a decrease in their study time. This suggests greater sustained student effort in the flipped section. In both sections of the course, students were asked to report on their preferred type of instruction, as seen in Figure 4. The most commonly reported instructional style in the flipped class in the midflip survey was, “I work with others and the teacher lectures”, while the most common response in the control section was, “The teacher lectures and tells us what to learn.” By the end of the semester, 80% of the flipped class reported that at least some peer-to-peer learning was a component in their preferred type of instruction while the control class was largely split between, “I learn by myself” and lecture. Only 17% of the control class included peer-to-peer learning as a component of their preferred type of instruction. In addition, when comparing matched student survey responses from the midsemester to final surveys, a net 26% of the flipped class changed their response to a more collaborative type of instruction while there was no net change in the control class. Therefore, the students’ experiences in the flipped class changed their perception of their preferred instructional style while there was no change in either direction for the control class. This suggests that any initial student reluctance to the flipped method can change as students adjust to the approach and have success with the method. In addition, the change in preferred instructional style for students in the flipped section may lead to more receptivity to active learning in future courses. Student focus group comments and free responses to survey questions indicate that many students valued peer interaction during the flipped class periods. One student commented, “I liked working with my peers because it allowed me to really consolidate my understanding of material and build friendships at the same time.” Others expressed that explaining concepts to others made it easier for them to remember the information. Students also appreciated the ability to ask many questions during a flipped class period to each other, as well as to the

instructor, whereas in a traditional class each student might be able to just ask a question or two only directed at the instructor. Indeed, this distinction is one of the most significant differences between the flipped and traditional classrooms. In the final of the three university surveys for flipped courses, students “agreed” or “strongly agreed” that they devoted more time (100%) and prepared more (95%) for this course than for their other courses. Students felt that the flipped class improved their critical thinking ability (95%), their problem solving ability (95%), and their ability to connect course concepts with one another (84%). It should be noted that while students were asked to answer the survey questions based on their experience on flipping, it may be that the strong response to these questions would be true for any organic course and not necessarily be solely the effect of flipping. Students also felt that flipping will lead to better transfer of knowledge to future courses (95%). Team work and collaboration skills (79%) as well as comfort with technology as a learning tool (84%) were increased through student experience in the flipped course. Many of these learning gains are broader than course-specific learning outcomes and are transferable to future courses. In total, 74% of the students felt that they learned better in the flipped course while 58% agreed that the flipped class approach would lead to a better grade in the course. A majority (68%) agreed that they would take another flipped class, while 80% would use instructional videos in another course. This suggests that instructors will find a receptive audience if they create video content to supplement the course, regardless if the course is flipped or not. In several questions where students were asked to rank their level of agreement with each statement, as seen in Table 4, students report almost always taking notes on the videos. This reflects a high level of student preparation for each flipped class. Students equally valued both out-of-class and in-class activities in assisting them with course concepts and report very high involvement with in-class activities. Students’ comfort level with accessing videos, participating in small group work, and using active learning strategies showed statistically significant improvement from the beginning to the end of the semester (p < 0.01) with a very large effect size (Cohen’s d = 1.04 to 1.18). These students may be more comfortable in using technology, peer learning, and other active learning strategies even in future courses. G

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

■

Table 4. Flipped Section Survey Dataa (4 = Always, 3 = Most of the Time, 2 = Sometimes, 1 = Never)

3.84 (0.50) 3.42 (0.84)

Accessing course material out of class (i.e., viewing videos on BlackBoard) Participating in small group activities during class Using active learning strategies in class (i.e., being an active participant vs a passive and silent notetaker) a

3.53 (0.77) 3.42 (0.84)

Final

2.36 (1.26)

3.17 (0.99)

3.47 (0.84)

2.32 (1.13)

3.17 (0.71)

3.37 (0.76)

2.27 (1.08)

2.89 (0.96)

3.37 (0.76)

LIMITATIONS

This study is based on a one-semester controlled study with 54 students with average SAT scores of 1053 (M + V) at a private university. Institutions with different class sizes and different average SAT scores may experience different results. Students enrolled into two sections of the course not knowing that one would be flipped and one would be a traditional lecture section. The instructor chose the section to be flipped on an arbitrary basis after course enrollment had closed. The flipped and control sections were compared by SAT score, GPA, general chemistry grades, prior organic chemistry coursework, and an organic chemistry pretest. No statistically significant differences were noted. However, the two groups are not identical and other factors may have influenced the differences observed in class performance. Another limitation is that student survey data is perceptual and can be limited by a student’s ability to accurately recollect their course experience. For example, in survey items such as “time spent outside of class in studying organic chemistry”, students were asked to recollect and make a judgment call rather than carefully record their actual hours spent on the course throughout the semester. However, since the purpose of the survey question was to look for a difference in student perception on time spent on the course between the flipped and control sections, the survey results are useful. Test and quiz questions on flipped topics were the same in both the flipped and control sections. Since the two classes were not held at the same time, it is possible that students in one section could share test information with their peers in the later section. Graded papers were never returned until all students in both sections had completed the test. As the schedule worked out, students in the flipped section were always the first to be tested on the material. Since the scores of the flipped class on the flipped topics were almost always higher than the control, there was little benefit, if any, to the control class in taking their tests a few days later.

Final

I took notes on the assigned out-of-class materials The out-of-class materials helped me better understand the course concepts I actively participated during in-class activities The in-class activities helped me better understand the course concepts (4 = Very Comfortable, 3 = Comfortable, 2 = Uncomfortable, 1 = Very Uncomfortable) Start Mid

Article

Standard deviations are in parentheses.

■

CONCLUSIONS In summary, the partially flipped course section showed improved student achievement in all flipped content areas with significant improvements in drawing resonance structures, drawing reaction mechanisms, and predicting products for aromatic reactions. While emphasizing problem solving skills in flipped teaching of these topics, it is important to equally highlight the critical need to memorize facts, a lower-order skill. Although final exam grades and overall course averages were very similar, the flipped section earned 25% more course grades of A and B than the control. In addition, the withdrawal rate was 54% lower in the flipped section. Students in the flipped section felt more confident of their abilities as seen in a significantly different perception of the level of challenge in the course. Both sections of the class did not significantly differ on their self-reported time spent on the course outside of class time. Through the ability to watch, pause, and review videos, as well as having extensive in-class collaborative problem solving time, students in the flipped section increased their confidence, as well as their understanding, of the flipped topics. As the semester progressed, a net 26% of students in the flipped section changed their perception of their preferred instructional style to a more collaborative one while there was no net change in the control section. Over the semester, students in the flipped section reported an increase in their average study time, while the control section, on average, had a slight decrease in their study time. The increased persistence of students in the flipped section is also evident in the dramatic difference in the withdrawal rates between sections. Students reported that their experience in flipping resulted in many transferable skills. Throughout the semester, students reported improvement in their comfort level in using technology outside of class, comfort in working in groups, and comfort in participating in active learning activities. Students felt that the flipped class improved their critical thinking and problem solving ability, their ability to connect course concepts with one another, and their transfer of knowledge to future courses.

■

IMPLICATIONS This study suggests that flipping the classroom for approximately a third of the organic chemistry course leads to significant improvements in student content mastery and improved student perceptions about their learning experience. Several topics that require higher-order thinking in Bloom’s taxonomy (“apply” and “analyze” vs “remember”) were selected for flipped teaching, and students benefited from the flipped approach. However, care must be taken when emphasizing higher-order problem solving to show that lower-order skills, including memorization of reagents and reaction limitations, are equally critical in mastering organic chemistry. This study supports previous study findings11,17,25,26 where a dramatic reduction in student course withdrawal is noted in a flipped course. It also shows the importance of active learning in improving student learning. The use of videos for viewing and reviewing instructor explanations of challenging topics is reported by students to be very helpful to their learning. While no significant improvement was seen in overall course averages and cumulative final exam score averages in this case, future work will look for more robust improvements when a greater percentage of the course content is flipped. Lastly, in implementing flipped pedagogy, it is wise to keep the amount of face-to-face instructional time the same as for a traditional course, since freeing up this valuable time for active peer-toH

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(12) Cooper, M.; Grove, N. P.; Underwood, S. M.; Klymkowsky, M. W. Lost in Lewis Structures: An Investigation of Student Difficulties in Developing Representational Competence. J. Chem. Educ. 2010, 87, 869−874. (13) Bodner, G. M. Theoretical Frameworks for Research in Chemistry/ Science Education; Pearson/Prentice Hall: Upper Saddle River, NJ, 2006. (14) Seery, M. K. ConfChem Conference on Flipped Classroom: Student Engagement with Flipped Chemistry Lectures. J. Chem. Educ. 2015, 92, 1566−1567. (15) Teo, T. W.; Tan, K. C. D.; Yan, Y. K.; Teo, Y. C.; Yeo, L. W. How Flip Teaching Supports Undergraduate Chemistry Laboratory Learning. Chem. Educ. Res. Pract. 2014, 15, 550−567. (16) Fitzgerald, N.; Li, L. Using Presentation Software to Flip and Undergraduate Analytical Chemistry Course. J. Chem. Educ. 2015, 92, 1559−1563. (17) Ryan, M. D.; Reid, S. A. Impact of the Flipped Classroom on Student Performance and Retention: A Parallel Controlled Study in General Chemistry. J. Chem. Educ. 2016, 93, 13−23. (18) Smith, J. D. Student Attitudes Toward Flipping the General Chemistry Classroom. Chem. Educ. Res. Pract. 2013, 14, 607. (19) Butzler, K. B. ConfChem Conference on Flipped Classroom: Flipping at an Open-Enrollment College. J. Chem. Educ. 2015, 92, 1574−1576. (20) Weaver, G. B.; Sturtevant, H. G. Design, Implementation, and Evaluation of a Flipped Format General Chemistry Course. J. Chem. Educ. 2015, 92, 1437−1448. (21) Eichler, J. F.; Peeples, J. Flipped Classroom Modules for Large Enrollment General Chemistry Courses: a Low Barrier Approach to Increase Active Learning and Improve Student Grades. Chem. Educ. Res. Pract. 2016, 17, 197−208. (22) Ealy, J. B. Development and Implementation of a First-Semester Hybrid Organic Chemistry Course: Yielding Advantages for Educators and Students. J. Chem. Educ. 2013, 90, 303−307. (23) Rossi, R. D. ConfChem Conference on Flipped Classroom: Improving Student Engagement in Organic Chemistry Using the Inverted Classroom Model. J. Chem. Educ. 2015, 92, 1577−1579. (24) Christiansen, M. A. Inverted Teaching: Applying a New Pedagogy to a University Organic Chemistry Class. J. Chem. Educ. 2014, 91, 1845−1850. (25) Fautch, J. M. The Flipped Classroom for Teaching Organic Chemistry in Small Classes: is it Effective? Chem. Educ. Res. Pract. 2015, 16, 179−186. (26) Trogden, B. G. ConfChem Conference on Flipped Classroom: Reclaiming Face Time − How an Organic Chemistry Flipped Classroom Provided Access to Increased Guided Engagement. J. Chem. Educ. 2015, 92, 1570−1571. (27) McMurry, J. Organic Chemistry: With Biological Applications; Brooks/Cole: Belmont, CA, 2010. (28) Echo360 website. http://echo360.com (accessed August 2016). (29) Screencast-O-Matic website. https://screencast-o-matic.com/ home (accessed August,2016). (30) Ensemble Video website. https://ensemblevideo.com (accessed August, 2016). (31) Hartman, J. R.; Dahm, D. J.; Nelson, E. A. ConfChem Conference on Flipped Classroom: Time-Saving Resources Aligned with Cognitive Science to Help Instructors. J. Chem. Educ. 2015, 92, 1568−1569.

peer learning and other active learning strategies is a great benefit of flipping the classroom.

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.6b00393. CH 230 flipping pretest, university surveys, midpoint and final surveys for flipped course and control, and student focus group questions (PDF)

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Notes

The author declares no competing financial interest.

■

ACKNOWLEDGMENTS This research was supported by a grant to the University of Hartford from the Davis Educational Foundation for “Flipping the Classroom: Increasing Depth, Engagement and Transfer of Learning in Undergraduate Education”. Thanks to all involved with the cCWCS Summer 2015 Workshop on “Active Learning in Organic Chemistry” for many helpful ideas. Also, thanks to Drs. Jean McGivney-Burelle, Robert Duran, and Jessica Nicklin for their guidance, and the members of the Hartford Davis grant leadership team, Drs. Mark Blackwell, Demaris Hansen, Frederick Sweitzer, and Lisa Zawilinski.

■

REFERENCES

(1) Luker, C.; Muzyka, J.; Belford, R. Introduction to the Spring 2014 ConfChem on the Flipped Classroom. J. Chem. Educ. 2015, 92, 1564− 1565. (2) Bergmann, J.; Sams, A. Flip Your Classroom: Reach Every Student in Every Class Every Day; International Society for Technology in Education: Alexandria, VA, 2012. (3) Kuh, G.; Kinzie, J.; Schuh, J.; Witt, E. Student Success in College: Creating Conditions that Matter; Association for the Study of Higher Education: Washington, DC, 2005. (4) Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active Learning Increases Student Performance in Science, Engineering, and Mathematics. Proc. Natl. Acad. Sci. U. S. A. 2014, 111, 8410−8415. (5) Wieman, C. E. Large-Scale Comparison of Science Teaching Methods Sends Clear Message. Proc. Natl. Acad. Sci. U. S. A. 2014, 111, 8319−8320. (6) Seery, M. K. Flipped Learning in Higher Education Chemistry: Emerging Trends and Potential Directions. Chem. Educ. Res. Pract. 2015, 16, 758−768. (7) Seery, M. K.; McDonnell, C. The Application of Technology to Enhance Chemistry Education. Chem. Educ. Res. Pract. 2013, 14, 227− 228. (8) Abeysekera, L.; Dawson, P. Motivation and Cognitive Load in the Flipped Classroom: Definition, Rationale, and a Call for Research. High. Educ. Res. Dev. 2015, 34, 1−14. (9) Hibbard, L.; Sung, S.; Wells, B. Examining the Effectiveness of a Semi-Self-Paced Flipped Learning Format in a College General Chemistry Sequence. J. Chem. Educ. 2016, 93, 24−30. (10) Schroeder, L.; McGivney-Burelle, J.; Xue, F. To Flip or Not to Flip? An Exploratory Study Comparing Student Performance in Calculus I. PRIMUS. 2015, 25, 876−885. (11) Flynn, A. B. Structure and Evaluation of Flipped Chemistry Courses: Organic & Spectroscopy, Large and Small, First to Third Year, English and French. Chem. Educ. Res. Pract. 2015, 16, 198−211. I

DOI: 10.1021/acs.jchemed.6b00393 J. Chem. Educ. XXXX, XXX, XXX−XXX