The Effectiveness of Course Flipping in General Chemistry â Does It

Chapter 2

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch002

The Effectiveness of Course Flipping in General Chemistry – Does It Work? Dominick Casadonte* Department of Chemistry and Biochemistry, Texas Tech University, 1 Memorial Circle, Lubbock, Texas 79409 *E-mail: [email protected]

The author has been involved in flipping classes in both on-line and face-to-face formats since 2008. In this study, I have flipped the Honors General Chemistry course sequence at Texas Tech University from the fall of 2010 through the fall of 2015. All of the pre-class lectures were recorded using the Mediasite platform and placed on Blackboard for students to watch in advance of class time. Online web learning homework assignments were used to determine if students had watched the lecture. Class time was used 1) in a discussion format to summarize lectures and clear up muddy conceptual points, and 2) to work advanced problems using a variety of active learning modalities. The efficacy of the method was determined by giving exams that had been given to other honors classes 5 years previously as a baseline and comparing exam results, as well as through standardized ACS content exams. I was especially interested in the pre-post differential percentile rankings as an indication of improvement in student learning outcomes over time. A 40-question Likert assessment and a 40-question free-response assessment were also given to the students in a pre-post format. Results of the various assessments, as well as the effectiveness of the method for different student cohorts, are discussed.

© 2016 American Chemical Society Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Introduction


The Concept of Course Flipping The beginning of the terminology associated with “course flipping” can be traced to Lage’s discussion of the “inverted classroom” in the early 2000’s (1). In its most general concept, the inverted classroom allows for the movement of learning activities which have historically occurred inside the classroom and during class time to be recontextualized as activities to be conducted in an extramural or asynchronous setting, freeing up classroom time for alternatives to the traditional lecture. The inverted classroom was designed to provide flexibility with regard to students’ differing learning modalities as well as to accommodate the variety of teaching styles used by faculty. The idea of using technology to move lecture content outside of the physical classroom in order to allow for greater discussion and more engaged and active learning strategies in the classroom environment has been dubbed, in different settings, “time-shifted (a term that has been co-opted from the video industry during the court battles between Universal and Sony in 1984) instruction” (2), “reverse instruction” (3), and “naked teaching” (4), and has been particularly effective in terms of increasing the amount of time in the classroom that can be devoted to discussion in the arts, humanities, and professional schools (5–10). Although the concept of moving the traditional lecture outside of the classroom in order to provide learning space during class time for active learning strategies has been around for more than a decade, the application and coining of the term “course flipping” to a chemistry classroom environment is credited to the pioneering work of two high school chemistry teachers, Jonathan Bergmann and Aaron Sams, in 2012 (11). They began by asking the question, “how can we use our in-class time more effectively to teach our students chemistry?”. This query led them to the use of technology to flip the traditional lecture-homework-test paradigm so common in the traditional chemistry course at the high school level. Shortly thereafter the term “flipped classroom” was applied at the university level. Since the advent of course flipping in particular in the university chemistry course environment, the number of cases where flipping has been examined as an alternative and perhaps superior pedagogy has substantially increased. The flipped chemistry classroom has been tried and evaluated in general chemistry classes (12–18), in organic chemistry classes (19–23), in physical chemistry lecture courses (24, 25), in analytical chemistry classes (26), in biochemistry classrooms (27), and in chemistry laboratory classes (28–30). The flipping of chemistry classes both at the K-12 and university levels has been the subject of reviews (31, 32) and at least two ACS symposia (33, 34).

Advantages and Disadvantages of Course Flipping Many potential advantages, both with regard to student learning outcomes as well as for the instructor, exist in the flipped pedagogy compared to the traditional classroom. Some of these include: 20 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


For the Student: 1. Students are more engaged in their own learning 2. There is the real potential for more engaged time on task 3. Students can review the lecture as often as they would like or need 4. Students can watch the lecture when most convenient between real-time interaction with the instructor 5. Better scheduling of class; students know what to study 6. It allows the students to learn at their own pace 7. Flexibility of the platform: Students can use computers or portable devices (smart phones, etc) to watch the videos, and are hence not tied to the classroom or technology class setting 8. Potentially less time required for exam preparation

For the Instructor: 1. The instructor can produce specific, targeted lecture topics and materials 2. It allows the instructor greater design and control of the classroom setting 3. It provides more flexibility in designing classroom interaction 4. It provides for increased interaction with students in the classroom 5. The instructor has time to use guided inquiry in the classroom if desired 6. A better understanding of student’s thinking often emerges 7. It puts the responsibility for learning significantly into the hands of the students 8. It can (depending upon the design of the course) allow more time on task for the student 9. It allows for the identification of groups of students during the classroom session in need of remediation and the consequent development of peer or teacher-led mini-tutorials. 10. It potentially saves time by not having to repeat lectures or topics from year to year 11. Since the assignments are placed on the internet, various educational platforms (e.g., Blackboard) allow the instructor to determine whether or not students have watched the videos 12. Assessments can be built in prior to class attendance 13. Chat rooms can be set up and out-of-class discussion with the instructor or teaching assistant can occur to facilitate understanding during the preclass experience 14. It can potentially improve exam and class performance 15. Asymmetric learning situations are possible 16. Can be done in a purely online format, if there is real-time instructor interaction online 17. Flipping has value added relative to online classes, in that there is faceto-face in-class active learning to complement video presentations. The instructor is a necessary component to the success of student learning and student learning outcomes (SLOs). 21 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


There are, to be fair, some disadvantages that have been identified to flipping a course, especially the first time that the course is taught (31, 32, 35). These include: 1. 2. 3. 4. 5. 6. 7. 8.

Time-intensive to set up Can lead to poorer class attendance Can cause students to disengage Takes more time for the student Potentially more difficult for ESL students? Requires responsible student Works well for majors and honors students, but may not be as effective for lower-performing students May not be as effective for lower socioeconomic groups who do may not have access to technology

Items (4)–(8) require some additional commentary. The existing literature concerning ESL students tend to indicate that, contrary to anecdotal belief, the use of a flipped classroom setting actually increases the verbal fluency and use of language of ESL students (5). With regard to the level of engagement, prior knowledge, or performance level of the students within a class, the data are mixed. Some studies have indicated that there is statistically little difference in learning outcomes between upper and lower performing students (12, 13), while others have observed that the learning improvements occurred for the higher-performing students. With regard to weaker students in the class, some studies have shown little or no improvement (15), while others have shown substantive learning gains by previously bottom performing students (36). It has been suggested by a number of practitioners that the improvements that occur in the flipped setting depend, to a certain extent, on the nature of the active learning strategies involved (12). In studies at both open-enrollment colleges (37) as well as for primarily HBCUs (38), significant learning gains have been observed by the use of course flipping techniques.

The Flipping Components As Bergmann and Sams have pointed out (11), there is no specific way to flip a class, and, in fact, the ability to flip a classroom in a variety of ways is one of the strengths of the pedagogy. It is useful, however, to consider the flipping process in terms of five possible components that may be blended to provide the actual flipped environment: • Pre-Class Instruction: This can take the form of instructor-prepared online videos , textual, or multimedia presentations as well as those that can be found online (such as those produced by the Khan Academy) (39). Depending upon the context, students can either access the online information separately or work together in groups using various collaboration software. • Pre-Class Assessment: This is often done using a web-based assessment package from either professional content managers or through learning 22 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


management systems. It could also include active learning assessments such as Just-In-Time Teaching (40), POGIL, or hands-on activities. • In-Class Discussion: This methodology is often used to help the students recap the material from the pre-class instruction. It is also useful to clear up muddy points or misconceptions. Case studies have also been used as a vehicle for in-class discussion of topics (41). • In-Class Active Learning Activities: This, again, can take many forms, including POGIL, Think-Pair-Share, Clickers, Active Response Systems, etc. One of the hallmarks of the flipped classroom is that the active learning strategies implemented are dependent upon the needs of the students and the ability and imagination of the instructor. • In-Class Assessment: Many possibilities exist here, including short quizzes, authentic activity assessements, online quizzing, etc. These components within the pre-class and in-class activities may, to a certain extent, be mixed and matched according the needs of the students and instructor and the effectiveness of the pedagogies that one employs. As new technologies and strategies for active learning unfold, the opportunities for even richer flipped environments will appear. Definition of Course Flipping For the purposes of the rest of this chapter, I will define course flipping as the process by which the typical lecture-homework-lecture-homework-test-lecturehomework paradigm is altered so that the lecture content is delivered outside of class, typically in an online or multimedia setting. The “in-class” time (whether face-to-face or in real-time online interaction, usually through video) can be spent having in-depth discussion for mastery or by engaging the students in any number of active learning strategies. This is the working definition employed in this study of the efficacy of the flipped class.

The Study Concurrent with many of the studies cited, I have recently completed a five-and-a-half year longitudinal study concerning the effectiveness of course flipping in a moderately-sized honors general chemistry class. My fundamental research question was whether or not course flipping would provide significant improvements in learning outcomes in a general chemistry classroom setting. The following will discuss the manner of the study as well as the outcomes. This study was approved by the Texas Tech University Institutional Review Board. The Course of Instruction The author began course flipping in the spring semester of 2009 in an on-line graduate conceptual chemistry class taught at Texas Tech University as part of a multidisciplinary master’s science degree. The methodology was borne from much the same motivation that Bergmann and Sams had for flipping at the high 23 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


school level, i.e., to be able to maximize teaching effectiveness during the limited time available with the students. By 2010 it was clear to the author that “course flipping” (as the term had been coined) could potentially provide a more effective means of improving SLOs in the general chemistry classroom than through the normal lecture-homework-exam paradigm. A study was begun to evaluate the efficacy of the pedagogy. The first classes involved were the Honors General Chemistry courses for F 2011 - S 2012 at Texas Tech University (CHEM 1307, Principles of Chemistry I (Fall) and CHEM 1308, Principles of Chemistry II (Spring)). The courses contained 75 and 70 students, respectively, and met T Th 9:30 AM – 10:50 AM. In 2015 the class size was increased to 96 students to cover an increasing population of honors students. All of the lectures for each course were pre-recorded in the summer of 2011 (for CHEM 1307) and the fall of 2011 (for CHEM 1308) using a Mediasite® recorder, a document camera, and a video camera. The Mediasite® recorder had picture-in-picture capabilities. All of the lectures were composed of class notes with strategic blanks for examples to be worked. The notes were provided to the students, who could then fill in the blanks while watching the lectures. This allowed for a tactile component to the learning process as well as the visual and audio representations in the recordings. The lectures were subsequently re-recorded in the summer of 2015 in high definition. Table 1 shows the characteristics of the lectures. Although the average lecture time was over 30 minutes, in a free-response survey given at the end of the first year of flipped instruction in CHEM 1308, 67% of the students thought that the videos were not too long. The author has polled the students in the CHEM 1308 classes every year from 2012-2015 concerning the length of the videos and has found a similar response. The main comment was that the students could pause the videos if they wanted to parse the time spent in viewing. There have been limited studies in the STEM disciplines concerning video length as it relates to improved SLOs (42). The author is currently performing a study of video length as it relates to course flipping in general chemistry.

Table 1. Lecture Characteristics Lecture Times No. of Videos

Shortest

Longest

Average

CHEM 1307

26

3:40

64:35

33:20

CHEM 1308

27

19:27

47:17

31:25

The syllabus carefully listed the lecture number and the topic for each class. Each lecture was correlated to a folder on the Learning Management System (LMS; here, Blackboard) that contained the videotaped lecture, a set of notes to be filled in, and a link to the Cengage online learning platform OWL for post-video quizzing. After watching each lecture (homework), the students then worked 6-10 homework questions using the OWL format (Mastery question bank). The 24 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

scores were then recorded, and counted for 150 points out of a total of 800 points allotted for the course. This allowed for a determination of who had watched the videos each week. Class time was divided into two parts:


-

-

First half: Review of the lecture material. During this time, the instructor checked (in a discussion format) for main ideas and was able to clear up any misconceptions. In this way the instructor could determine what the class had learned by watching the videos, and could provide additional information and insight, as well as prevent any misconceptions or muddiness from propagating through the curriculum. This review often involved a variety of techniques, including having the students “act out” molecular-level processes. Second half: This involved problem solving, using problems from the textbook (Oxtoby, Gillis, and Campion/Butler, Principles of Modern Chemistry. 7th and 8th Eds., Cengage Learning, 2012 and 2015) (43, 44) which had been previously indicated in the syllabus, so that the students could try the problems before coming to class, if so desired. Problems were worked in a variety of formats, depending upon the material and the class including group work, going to the board, modeling the answers, think-pair-share, etc.)

In addition to class time, the class was roughly divided in half and attended one of two zero-credit hour 1.5-hour discussion sections. The discussion section had additional interaction with the course material as well as preparation for a quiz given during each of the sections (one quiz per week per student). Three exams were given during the semester, as well as a final exam (cumulative). An ACS End-of-Term exam was administered as a pre- and post- test. As a way on incentivizing the exam, students were told that if they scored at or above the 90th percentile in the post-test, they did not have to take the class-based final exam. An additional item that is often discussed is the number of contact hours in the flipped model compared to course credit hours, so that the students are not engaged for longer than the number of credit hours mandate. This is not really an issue, as in a traditional lecture-homework format, the out of class homework can take a variable number of hours, depending on the number and level of difficulty of questions asked. Care is often taken in the development of flipped classes so that if there is out-of-class assessment, the number of questions is relatively small (here 6-10 low to moderate-level questions, to assess initial understanding of the lecture material only). Given the in-class time constraints, the number of advanced problems worked is usually small (in this study, typically 3-5, after an initial discussion). Evaluation of the Model: Methods of Assessment Four methods of assessment were involved to determine whether or not course flipping in the method described above would be able to improve learning outcomes. These include: 25 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

• • • •

Method I: Class-based Exam Score Comparisons Method II: ACS End-of-Term Exam Score Comparisons Method III: 40 –Question Likert Scale Questionnaire Method IV: Free-Response Questionnaire (Spring, 2012)


Evaluation of the Model: Results and Outcomes The author has been the only teacher of the Honors sections of CHEM 1307 and 1308 since 1998, and, as such, has access to data for relatively homogeneous student populations over time (the average SAT scores (verbal + math; pre-2012 scale) over the period of study was 1350 ± 50). Consequently, a historical approach has been used for comparison. The demographics of the study group are shown in Table 2.

Table 2. Demographics of Study Group Fall 2006

Fall 2011

Fall 2012

Fall 2013

Fall 2014

Fall 2015

Spring 2007

Spring 2012

Spring 2014

Spring 2015

Male %

44

58

45

58

49

43

35

53

43

38

Female %

56

42

55

42

51

57

65

47

57

62

White %

93

84

88

80

78

79

88

77

69

72

Hispanic %

4

5

7

7

7

12

7

7

19

7

Asian/ Other %

2

11

5

12

14

9

2

14

11

20

Black %

0

0

0

1

1

0

2

1

1

1

Table 3 provides the average scores for the three exams and final exam that were given in the fall of 2006 in CHEM 1307 (pre-flipped) as well as the fall exam periods from 2011-2013, the years in which the study was conducted. For each comparative data set, a 1-tailed heteroscedastic t test was performed to determine the statistical significance.

26 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Table 3. Exam Score Comparisons, CHEM 1307, Fall, 2011 - 2015


Exam I (Std Dev)

Exam II

Exam III

Final

Significance

Average

F 2006 (n= 45)

77.8(14.8) 77.9(12.8) 79.9(11.2) 78.8(12.0) 78.6(12.7)

F 2011 (n = 73)

87.5(9.2)

85.4(10.3) 85.8(10.0) 92.4(14.9) 87.8(11.1)

(p = 0.0011)

F 2012 (n = 75)

92.6(6.3)

85.8(7.7)

87.6(9.1)

(p = 0.0026)

F 2013 (n = 76)

90.2(7.1)

87.4(12.2) 83.2(12.3) 89.8(13.4) 87.7(11.3)

(p = 0.0022)

F 2014 (n = 74)

90.6(5.8)

90.4(7.2)

87.2(8.8)

88.1(10.1)

89.1(8.0)

(p = 0.0023)

F 2015 (n = 89)

90.9(10.1) 86.7(11.8)

88.8(7.8)

87.8(12.9) 88.6(10.6)

(p = 0.0024)

Average Score

90.4(8.9)

87.1(10.3) 85.9(10.3) 89.7(12.3) 88.3(10.5)

(p = 0.00008)

Average Δ

12.6

9.2

84.5(11.9) 90.6(10.3)

6.0

10.9

-

9.7

It is important to note that the exams given in the flipped classes were the exact exams given in 2006. Consequently, in this part of the study, the same instructor, same content, and same exams were used. As Table 3 demonstrates, the average increase in the exam scores as a result of using the course flipping pedagogy is more than nine percent. Each of the increases for each set of exams in the flipped class years relative to the pre-flipped year for CHEM 1307 are statistically significant at the p < 0.01 level. This is especially notable, given that the number of students taking the course increased by nearly 70% from 2006 to 2011. The more effective use of classroom time seems to be one reason for the increase in test scores. A similar effect was seen during the second semester of general chemistry, as noted in Table 4. The author did not teach this course in 2013. As in the case with CHEM 1307, the 2012 and 2007 exams were the same. Again, each of the increases for each set of exams in the flipped class years relative to the pre-flipped year for CHEM 1308 are statistically significant at the p < 0.01 level. The largest effect during the three years of the study was observed in the final exam statistics for CHEM 1308. There are several possible reasons why this might be the case. The ability to review the material due to the recorded nature of the lectures has been cited by the students (vide infra). Also, the continual preparation afforded by additional active learning during class time provides for a stronger ability for synthesis (the final exams were, in all cases, cumulative).


Table 4. Exam Score Comparisons, CHEM 1308, Spring, 2012-2015


Exam I (Std Dev)

Significance

Exam II

Exam III

S 2007 (n = 43)

84.6(12.2) 81.4(9.5)

83.6(8.0)

S 2012 (n = 70)

91.2(6.5)

84.7(7.9) 86.2(10.5)

87.4(14.1)

S 2014 (n = 75)

89.6(8.3)

84.5(8.3) 86.4(12.5)

87.9(17.5) 87.1(11.7)

(p = 0.0020)

S 2015 (n = 82)

90(5.6)

86.7(6.7)

88.8(7.2)

87.9(10.5)

(p = 0.0016)

Average Score

90.3(8.2)

85.3(8.1)

87.1(9.6)

87.7(14.0) 87.6(10.0)

5.7

3.9

2.5

Average Δ

Final

Average

72.8(13.8) 78.6(10.9)

14.9

87.3(9.8)

88.4(7.5)

(p = 0.0015)

(p = 0.0017)

9.0

In an attempt to mitigate any instructor bias in the preparation of the assessments, the 2005-2006 American Chemical Society First and Second Term General (EOT I and II) algorithmic exams were administered to the students (except in 2007, when a shorter, conceptual test was given; these results are not reported, due the differing nature of the test). The results of the exam scores for students scoring over the 95th and 80th percentiles on the 2005 ACS First Term General Chemistry exam over a ten-year period are shown in Table 5. It is striking that in the pre-flipped 2009 class only two students scored at or above the 80th percentile. Once flipping began in earnest in 2011, the number of students scoring above this benchmark began to significantly increase (an average of 16.0% ± 6.2 scored above the 80th percentile pre-flipping, while 23.2% ± 4.2 scored above the 80th percentile post-flipping (p = 0.09)). Given that the exam was administered to a relatively homogeneous student population with the same instructor teaching the class each year, it is strongly suggestive that the enhanced student attention and time on task provided in the flipped environment is a likely cause of the improvement. Similar results were observed for the two most recent years in which CHEM 1308 was taught by the author, using the 2006 ACS Second Term General Chemistry Exam as a comparison with the spring semester of 2010 (Table 6). Spring of 2010 was the last semester that CHEM 1308 was taught in a non-flipped format.


Table 5. 2005 ACS First Term General Chemistry Exam Comparisons for CHEM 1307 F 2005

F 2006

F 2008a

F 2009

F 2010

Above 95th %ile

3

3

0

0

2

80-94th %ile

5

9

6

2

6

8 (18.6)

12 (26.1)

6 (13.6)

2 (4.2)

8 (17.4)

F 2011

F 2012

F 2013

F 2014

F 2015

5

4

1

4

9

10 15 (20.0)

9 13 (18.3)

7 8 (23.5)

11 15 (20.3)

22 31 (33.7)


Total >80 %ile (%)

Above 95th %ile 80-94th %ile Total >80 %ile (%) a

In 2007 a short form of the ACS End-of-Term Exam was used

Table 6. 2006 ACS End-of-Term II Exam Comparisons for CHEM 1308 Sp 2010

Sp 2014

Sp 2015

Above 95th %ile

3

3

10

80-94th %ile

2

9

13

5 (10.4)

12 (16.4)

23 (27.7)

Total >80%ile(%)

Several other studies have used the ACS-end-of term exam average percentile rankings as indicators of overall improvements in SLOs using the flipped course format compared to a traditional lecture format (12, 14). One criticism that could be raised about using percentile averages is that it does not provide an indication of overall learning gains throughout the course of the semester, merely an indication of the relative content knowledge of the students at the end of the course of instruction only. If the students entered the course with significant prior knowledge, then a marginal increase in percentile score might be expected using either a traditional or a flipped course approach. This would still be recorded as a significantly high score. In an attempt to determine whether the flipped course of instruction resulted in significant increases in content knowledge (at least in an algorithmic sense, as the conceptual ACS exam was not routinely administered), we performed a prepost differential analysis for CHEM 1307 comparing F 2008 with F 2015. These were the two years with the largest differentials between pre- and post-flipping percentiles within the data set. The results are shown in Figure 1. The histograms are presented as the percent of students within each class that achieved the pre-post differential within the given bin. Percentages are used to normalize to class sizes. The normal distribution is superimposed for each year.



Figure 1. Percentile Differentials for 2008 and 2015 ACS EOT I Exam

The average normal differentials are 35 and 42 for 2008 and 2015 respectively. The results are significant at the p < 0.01 level. A similar and more compelling result is seen for the spring 2010 and spring 2015 pre-post percentile differentials (Figure 2). The average normal differentials are 27 and 34 for 2008 and 2015 respectively. The results are significant at the p < 0.001 level. These data tend to indicate that the use of the flipped class pedagogy significantly increases student learning outcomes compared to the traditional lecture format throughout the course of an entire semester of study for honors students. Since the students in this study were honors students who were pre-selected by the Honors College to be placed in the class, we can add little to the question of whether or not a flipped class environment will be of benefit to lower-level students.

Figure 2. Percentile Differentials for the 2010 and 2015 ACS EOT II Exam 30 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

It should be noted that only small variations in the average pre-test percentile scores were noted between the years shown in Figures 1 and 2 (EOT I Exam, 2008, 21.4 ± 8.9; 2015, 23.2 ± 11.5; EOT II Exam, 2010, 16.4 ± 9.2; 2015, 16.8 ± 8.75), further indicating the significance of the pre-post differential.


Likert Questionnaire A questionnaire was administered to the students in each class over the course of the 2011-2012 academic year. The number of students responding in the fall of 2011 (CHEM 1307) was 63, while 43 provided responses in the spring of 2012 in CHEM 1308. The following are some of the significant responses (score > 3.0) using a 1 (disagree; negative) to 5 (agree; positive) Likert scale for the fall, 2011 CHEM 1307 class. The numbers in parentheses refer to the positive response percentage in the spring 2012 CHEM 1308 class, and the number in brackets refers to the average score for the spring class. • 52 (70) % [3.50 ± 0.55] of students thought that they spent more time in the flipped course • 78 (86) % [4.05 ± 0.93] of the class felt that time shifting put more of the responsibility for learning the material on the student • 57 (77) % [3.65 ± 0.61] of the class agreed that there was increased interaction between the professor and class in the time-shifted format compared to other classes • 75 (91) % [4.03 ± 0.88] of the class felt that the instructor worked an adequate number of examples in class. • 78 (90) % [4.34 ± 1.16] of students believed that the instructor was a partner in their learning of chemistry. • 69 (67) % [4.03 ± 0.91] of students liked the use of OWL to test their understanding after watching the lecture • 77 (93) % [3.26 ± 0.20] of students, knowing what they know now, would NOT have taken a different section. • 37 (72) % [3.90 ± 0.92] of students would take another time-shifted course again while 23% felt that this was n/a. • 55 (84) % [3.57 ± 0.66] of the students felt that the time-shifted lecture/ discussion section format was useful, while 22 (7) % of the students did not. It should be noted that the increase in positive responses during the spring semester is most likely due to the fact that the majority of the students who took CHEM 1308 took CHEM 1307 the previous semester in a flipped format.

Free-Response Questionnaire In addition to the Likert-scale questionnaire, a free response section on the questionnaire given after the CHEM 1308 course was complete provides some useful information with regard to some of the mechanical aspects of the course. Some of the relevant data include: 31 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


• How many lectures per week did you actually watch? 79% of the students watched all of the lectures. Commentary: This tends to indicate that the students did not pick and choose which lectures to watch. • How many classes per week did you actually attend? 86% of the students attended every class. Commentary: This allayed some of the fear that students would believe that watching the out-of-class videos was sufficient to provide all of the relevant content for the course. • When watching the lectures did you actually watch it as if you were actually attending class? 74% of the students watched the lectures as if actually attending class. Commentary: The students tended to treat the videos in an analogous fashion to actually attending the lecture. The use of a notebook with blanks that could be filled in while watching the video probably enhanced this effect. • Did you watch the video lectures as review for quizzes and exams? 60% of the students did NOT watch the video lectures as review for quizzes and exams. Commentary: Although the students did not watch the entire video lecture set as a review for exams or quizzes, they reported that they spot-watched the video as a review for topics that were somewhat unclear for them. • Did you think this method of teaching left more time for procrastination between tests compared to a traditional lecture style? 53% of the students said that this method did not leave more time for procrastination. Commentary: The pace of the course was designed to relatively closely match the number of contact hours relative to the traditional lecture course. How many hours per week on average did you study for this class? 76% of the students studied between 2-4 hours per week. 17% of the students studied more than 4 hours. 7% of the students studied less than 2 hours. Commentary: The reduction in the number of hours on average that the students “studied” for the course is most likely due to the increased repetition (albeit in different formats) with which they worked with the various topics in the course. They studied less because their study time was used more efficiently. • How many hours do you prepare for each test? 36% of the students prepared 1-2 hours; 20% of the students prepared 3-4 hours; 27% of the students prepared 5-6 hours; 17% prepare 7 or more hours. Commentary: The reduction in the number of hours on average that the students “studied” for exams is again most likely due to more efficient time on task. • When you attended class, on a scale of 1-5 did you actually work the problems with the professor (5) or just simply write the answers down (1)? Those answering 3-5: 81%. Those answering 0-2: 19%. Commentary: This is almost of necessity, as the problems were worked using active and engaged learning strategies. • On a scale of 1 to 5 (1 being no, 5 being very important) did you think you actually needed to watch the lectures to earn a good grade in this class? 79% of the students answered 4-5. 32 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


Commentary: This partially speaks to the effectiveness of the video presentations as well as its initial use in terms of providing content. • Did you think this was an effective way to teach? 88% of the students thought this was an effective way to teach. Commentary: This is a compelling response. • Did you think discussion sessions were useful in this teaching style? 98% of the students thought the discussion sessions were useful. Commentary: It is not surprising that in an Honors class more time on task would be welcome. • What do you think about the length of the videos? Too long, too short, just right? 67% of the students said the length was just right. Commentary: The students also indicated that they appreciated having control of how to parse the videos. One aspect of other studies related to flipping (11) indicate that seeing one’s instructor on a video or multimedia presentation helps to provide buy-in for the students. Flipping is less effective when there is a disconnect between the instructor and the content. • What do you think about the quality of the video? 64% of the students said the quality was fine. The other responses mentioned that the handwriting was sometimes hard to read. Commentary: This was remedied in 2015 when the videos were re-recorded in HD format with superior recording equipment. Another important aspect of the flipping experience is the quality of both the out-of-class instruction as well as the engagement mechanism during class time. • Would videos of misconceptions be useful or can they be covered in class? 50% of the students said that misconceptions could be covered in class. 40% of the students did not comment. 10% of the students said that videos of misconceptions would be useful. Commentary: It is important that the instruction not rely solely on videos or video capture. There is research data that suggest that internet videos alone can actually reinforce scientific misconceptions, as, even though the students feel comfortable that they have learned the material, there is not the depth of interaction or reflection in the one-way transmission of information for the student to understand that what is presented may differ from the prior knowledge (misconception) that they may have. However, research also suggests that discussion of misconceptions in the flipped classroom setting is effective both in terms of removing the misconceptions as well as significantly improving learning outcomes (45). One of the value-added components of the time-shifted classroom is enhanced interaction between the faculty member and students in real time during what was formerly the lecture period. • On a scale of 1 to 5 (1 being no, 5 being very important) did you think watching the lectures was a major factor in how you performed on tests and quizzes? 69% of the students answered 4-5: watching the videos was a major factor in performance Commentary: Both aspects of the flipping paradigm (out-of-class content, in-class active learning) are important.


Advantages and Disadvantages of Flipping a Course: Student Responses


As part of the evaluation of course flipping, the students in the CHEM 1308 spring 2012 class were asked to assess various perceived advantages and disadvantages of the flipping paradigm. The results are given below: Advantages Identified 1. Work at own pace: 39% of those responding 2. Material availability: 23% of those responding 3. Self-directed learning: 23% of those responding 4. More ways to learn: 16% of those responding The major advantage reported by a group of exceptionally bright students taking general chemistry at a public university was the ability to work asynchronously. This is not surprising, given the abilities of the class. The access to on-line homework and being able to have more independence with regard to their own learning outcomes was also reported as strengths of the pedagogy. Somewhat more surprising is the notion that less than 20% of the students identified the variety and diversity of learning types (e.g, think-pair-share, group work, work at the board, etc) available in the flipped environment as seminal to and a real advantage of the method of instruction. Disadvantages Identified: 1. Procrastination: 46% of those responding 2. More time and effort required of the student: 42% of those responding 3. Felt disengaged: 12% of those responding Although 53% of the students in the free-response section indicated that they believed that the flipped classroom did not allow for procrastination, when asked about it as a potential disadvantage to the methodology, almost half of the respondents indicated that being able to watch videos on their own time rather than in a more conventional, fixed-time classroom setting could lead to inappropriate time management. A significant number of students indicated that a flipped class requires more time and effort than a traditional lecture-homework classroom. It is interesting to note, however, that 36% of the respondents indicated that they spent only 1-2 hours preparing for each test. The “additional time and effort” involved in watching the lectures, doing the out-of-class homework, and participating in class translated into fewer hours actually studying for the exams and, on average, significantly higher exam scores.

Conclusions The results of this study indicate that course flipping in the honors general chemistry classroom can lead to significant improvements in learning outcomes. Some general conclusions from this study include: 34 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

• • •


• •

Course flipping can be an effective modality for teaching general chemistry Course flipping provides enhanced learning outcomes Course flipping generates greater learning gains over the course of a semester of study Course flipping provides for more teacher – student interaction Course flipping is more work (initially) for everyone

Course flipping provides the freedom that students often want in terms of learning content with the value-added component of a face-to-face active learning classroom experience. In concept, it appears to be superior to either the instructor-centric traditional lecture-homework paradigm or the student-centric online experience. In one sense, it provides optimum flexibility for allowing the teacher to do their best teaching and the student to have the richest learning environment possible through multiple modalities. It is liberating for student and teacher alike, and empowering for both. Through refinements such as flipped-mastery teaching (11), the future of the flipped classroom is bright indeed.

References 1.

2.

3.

4.

5.

6. 7. 8.

9.

Lage, M. J.; Platt, G. J.; Treglia, M. Inverting the Classroom: A Gateway to Creating an Inclusive Learning Environment. J. Econ. Educ. 2000, 31, 30–43. United States Supreme Court, SONY CORP. v. UNIVERSAL CITY STUDIOS, INC. (1984, January 17). http://caselaw.findlaw.com/ussupreme-court/464/417.html (accessed March 8, 2016) Martin, J. Reverse Instruction: Dan Pink and Karl’s “Fisch Flip”; November 7, 2010; http://www.connectedprincipals.com/archives/1534 (accessed March 8, 2016). Bowen, J. A. Teaching naked: How moving technology out of your college classroom will improve student learning; John Wiley & Sons: New York, 2012. Leis, A.; Cooke, S.; Tohei, A. The Effects of Flipped Classrooms on English Composition Writing in an EFL Environment. Int. J. Comput.-Assisted Lang. Learn. Teach. (IJCALLT) 2015, 5, 37–51. Berrett, D. How ‘flipping’the classroom can improve the traditional lecture. The Chronicle of Higher Education 2012, 12, 1–14. Barkley, A. Flipping the College Classroom for Enhanced Student Learning. NACTA J. 2015, 59, 240. Roehl, A.; Reddy, S.; Shannon, G. The flipped classroom: An opportunity to engage millennial students through active learning strategies. J. Family Consumer Sci. 2103, 105, 44–49. Critz, C.; Knight, D. Using the flipped classroom in graduate nursing education. Nurse Educator 2013, 38, 210–213. 35 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


10. McDonald, K.; Smith, C. The flipped classroom for professional development: part I. Benefits and strategies. J. Contin. Educ. Nurs. 2013, 44, 437–438. 11. Bergmann, J.; Sams, A. Flip your classroom: Reach every student in every class every day; International Society for Technology in Education: Alexandria, VA, 2012. 12. Weaver, G.; Sturtevant, H. Design, implementation, and evaluation of a flipped format general chemistry course. J. Chem. Educ. 2015, 92, 1437–1448. 13. Ryan, M.; Reid, S. Impact of the Flipped Classroom on Student Performance and Retention: A Parallel Controlled Study in General Chemistry. J. Chem. Educ. 2016, 93, 13–23. 14. Hibbard, L.; Sunf, S.; Wells, B. Examining the Effectiveness of a Semi-SelfPaced Learning Format in a College General Chemistry Sequence. J. Chem. Educ. 2016, 93, 24–30. 15. Yestrebsky, C. Flipping the classroom in a large chemistry class-research university environment. Procedia Soc. Behav. Sci. 2015, 191, 1113–1118. 16. Smith, J. Student attitudes toward flipping the general chemistry classroom. Chem. Educ. Res. Pract. 2013, 14, 607–614. 17. Eichler, J.; Peeples, J. Flipped classroom modules for large enrollment general chemistry courses: a low barrier approach to increase active learning and improve student grades. Chem. Educ. Res. Pract. 2016, 17, 197–208. 18. Belford, R.; Stoltzfus, M.; Houseknecht, J. Confchem Conference on Flipped Classroom: Spring 2014, ConfChem Virtual Poster Session. J. Chem. Educ. 2015, 92, 1582–1583. 19. Trogden, B. ConfChem Conference on Flipped Classroom: Reclaiming Face Time: How an Organic Chemistry Flipped Classroom Provided Access to Increased Guided Engagement. J. Chem. Educ. 2015, 92, 1570–1571. 20. Rossi, R. D. ConfChem Conference on Flipped Classroom: Improving Student Engagement in Organic Chemistry Using the Inverted Classroom Model. J. Chem. Educ. 2015, 92, 1577–1579. 21. Fautch, J. The flipped classroom for teaching organic chemistry in small classes: Is it effective? Chem. Educ. Res. Pract. 2015, 16, 179–186. 22. Rein, K.; Brookes, D. Student Response to a Partial Inversion of an Organic Chemistry Course for Non-Chemistry Majors. J. Chem. Educ. 2015, 92, 797–802. 23. Cavazos, Jr. R.; Petros, A.; Petros, R. Successfully Flipping the Classroom for Organic Chemistry. In Innovations in College Science Teaching; Society for College Science Teachers: Washington, DC, 2015; pp 79−91. 24. Seery, M. ConfChem Conference on Flipped Classroom: Student Engagement with Flipped Classroom Lectures. J. Chem. Educ. 2015, 92, 1566–1567. 25. Goss, L. Flipping the Physical Chemistry Lecture Course: Making More Time for Understanding. Abstr. Pap. Am. Chem. Soc. 2014, 248−249. 26. Fitzgerald, D.; Li, L. Using Presentation Software to Flip an Undergraduate Analytical Chemistry Course. J. Chem. Educ. 2015, 92, 1559–1563. 36 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


27. Ojennus, D. Assessment of learning gains in a flipped biochemistry classroom. Biochem. Mol. Biol. Educ. 2016, 44, 20–27. 28. Devine, T.; Gormley, C.; Doyle, P. Lights, Camera, Action: Using Wearable Camera and Interactive Video Technologies for the Teaching & Assessment of Lab Experiments. Int. J. Innov. Sci. Math. Educ. 2015, 23, 22–33. 29. Fung, F. Using First-Person Perspective Filming Techniques for a Chemistry Laboratory Demonstration To Facilitate a Flipped Pre-Lab. J. Chem. Educ. 2015, 92, 1518–1521. 30. Teo, T.; Kim Tan, K.; Yan, Y.; Teo, Y.; Yeo, L. How flip teaching supports undergraduate chemistry laboratory teaching. Chem. Educ. Res. Pract. 2014, 15, 758–768. 31. Seery, M. Flipped learning in higher education chemistry: emerging trends and potential directions. Chem. Educ. Res. Pract. 2015, 16, 758–768. 32. Boesdorfer, S. B. Review of Chemistry Education: Best Practices, Opportunities and Trends. J. Chem. Educ. 2015, 92, 1975–1976. 33. Casadonte, D. Flipping the Classroom Symposium. Abstr. Pap. Am. Chem. Soc. Dallas, TX, 2014, 34. Luker, C.; Muzyka, J.; Belford, R. Introduction to the Spring 2014 ConfChem on the Flipped Classroom. J. Chem. Educ. 2015, 92, 1564–1565. 35. Kim, M. The experience of three flipped classrooms in an urban university: an exploration of design principles. Internet Higher Educ. 2014, 22, 37–50. 36. Ryan, M. D.; Reid, S. A. J. Chem. Educ. 2016, 93, 13–23. 37. Butzler, K. ConfChem Conference on Flipped Classroom: Flipping at an Open-Enrollment College. J. Chem. Educ. 2015, 92, 1574–1576. 38. Talley, C.; Scherer, S. The Enhanced Flipped Classroom: Increasing Academic Performance with Student-recorded Lectures and Practice Testing in a “Flipped” STEM Course. J. Negro Educ. 2013, 82, 339–347. 39. Khan, S. Let’s Use Videos to Reinvent Education. Presented at TED2011, Long Beach, CA, March 2011. https://www.ted.com/talks/ salman_khan_let_s_use_video_to_reinvent_education (accessed December 15, 2015). 40. Muzyka, J. ConfChem Conference on Flipped Classroom: Just-in-Time Teaching in Chemistry Courses with Moodle. J. Chem. Educ. 2015, 92, 1580–1581. 41. Herreid, C.; Schiller, N. Case studies and the flipped classroom. J. Coll. Sci. Teach. 2013, 42, 62–66. 42. Zappe, S. “Flipping” the Classroom to Explore Active Learning in a Large Undergraduate Course. Proceedings, American Society for Engineering Education Annual Conference & Exposition; Atlanta, GA, 2009; 14.1385.1–14.1385.21. 43. Oxtoby, D. W.; Gillis, H. P.; Campion, D. Principles of Modern Chemistry, 7th ed.; Cengage Learning: Boston, MA, 2012; ISBN: 0-840-04931-5. 44. Oxtoby, D. W.; Gillis, H. P.; Butler, L. J. Principles of Modern Chemistry, 8th ed., Cengage Learning: Boston, MA, 2015; ISBN: 978-1-305-07911-3. 45. Muller, D. A. Designing Effective Multimedia for Physics Education. Ph.D. Dissertation, University of Sydney, Australia, 2008. 37 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

The Effectiveness of Course Flipping in General Chemistry â Does It

Recommend Documents

The Effectiveness of Course Flipping in General Chemistry â Does It