Biochemistry and the Liberal Arts: Content and Communication in a

has been raised in a device-centric (phone, iPad etc.) social .... on their own before class, bringing to class questions that might have come up duri...
0 downloads 0 Views 590KB Size
Chapter 8

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

Biochemistry and the Liberal Arts: Content and Communication in a Flipped Classroom Gregory W. Muth* Department of Chemistry, St. Olaf College, 1520 St. Olaf Ave., Northfield, Minnesota 55057 *E-mail: [email protected]

This chapter reflects on seven semesters of teaching a one-semester majors biochemistry course in a liberal arts setting before flipping the classroom and six semesters of teaching the same course after flipping the classroom. The median exam score prior to flipping was 82% and after flipping was 79% (p = 0.009). Analysis of the level of difficulty of the exam questions revealed that after flipping, the exams contained 23% more points at higher cognitive levels as assessed by Bloom’s taxonomy. This indicated that even though the students performed the same, the exams required higher order thinking skills for success. Students also reported gains in proficiency working in groups, communication skills and problem solving abilities.

Introduction Teaching pedagogy has evolved with changing technology. In years gone by we taught with chalk on slate, acetate sheets on overhead projectors, white boards, using Powerpoint and now with video supplements. In each instance there was a period of adjustment and/or pushback from the faculty and a period of adjustment and pushback from the students. Despite the struggles during the transition, the change was good and ended with a better, more progressive learning environment. The age of video instruction has complemented the active learning classroom (1–4). While both video instruction and active learning each have merit, the combination of the two can provide an excellent balance between delivering content and promoting meaningful reflection and learning (3, 5–7). The style © 2016 American Chemical Society Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

and content of videos for teaching are as wide and varied as the methods for making them (8). Background, special topics, guest lectures, problem solutions and responses to questions all have been the subjects of videos to enhance student learning and create time in-class for high-impact, active learning strategies. With class time liberated for active learning strategies, faculty are employing a vast array of proven techniques in the classroom (9, 10). Personal response systems (clickers), paper and pencil problems, skits, student presentations, workshops, demonstrations and a class-lab hybrid all are more accessible with the lecture portion being moved outside of the class. But beyond content, the flipped classroom provides students an opportunity to improve interpersonal skills (11, 12). The current generation of college students has been raised in a device-centric (phone, iPad etc.) social structure while many of the older generations and even current students from less privileged backgrounds have not had this opportunity. This disparity can create a challenge for quality and effective interpersonal communication. To this end, students need coaching and practice in the art of both speaking and listening. Even for those who are proficient, there is always room to refine communication skills in a technical area. In this chapter the journey of creating and implementing a flipped classroom using video lectures for a one-semester 300-level biochemistry course will be presented. Comparisons of exams and exam scores are made between the same course taught in traditional lecture format and in the flipped format. Additional self-reported student data are given to support the claim that they are gaining skills valued by the liberal arts community.

Methodology The process for creating a video lecture supplement for viewing outside of class can begin with PowerPoint slides and lecture notes that already exist for a given course. There is no requirement to rewrite lectures or modify slides that have proven effective for student learning and previous experiences. In conversations with faculty who are interested in trying to create video lectures they often become overwhelmed with the number of choices in software available to capture video voice and screen simultaneously. While there are dozens to choose from they all do about the same thing. In this respect the choices therefore become very personal. This author recommends several features that have proven effective for the past several years. First is convenience. It is very convenient to be able to record, edit and publish a lecture video in the privacy and comfort of your own office. Some may argue for the need for a professional quality recording studio and yes, that will give the highest quality production product in the end but there is a trade-off for the convenience of simply being able to close your door record a lecture and publish it to the web. To this end this author has used Telescreen’s ScreenFlow software for all of his video lecture production (13). ScreenFlow allows the simultaneous capture of voice, a headshot (optional) and the events that are happening on the screen whether it is an animation or simply the mouse pointer being used to illustrate different features on the slide (Figure 1). 128 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

Figure 1. Representative screen shot of a typical video lecture containing the PowerPoint slide, talking head of the professor and the mouse pointer to highlight text.

The second feature that is critical in screen capture software is the ability to conveniently edit the file. Editing the file can be extremely complex and timeconsuming, relatively straightforward, or not employed at all. We know that in our classrooms not every lecture is perfect. Often times we pause, we may stumble on words, shuffle our notes or even lose our train of thought. The students are accepting of these small idiosyncrasies therefore we should feel no different when producing a video lecture. A slight pause to check where you are in your notes, a moment to collect your thoughts or an invitation for students to pause the video to work a problem does not distract from the content of the video lecture itself. This “good enough” philosophy can lower a barrier and facilitate creation of a valuable teaching tool. If one does desire to edit out large gaffes or create a video masterpiece, the software should be easy and convenient to use. With ScreenFlow, large and/or small sections of audio and video can be cut and spliced back together to create a relatively seamless flow of information from start to finish. It is also possible to add voice overs or additional material later on if desired. Finally, all software packages must have the ability to export the file from an editable screen capture file or similar to a .MOV, .mp4 or .m4v file (or similar) that can be viewed on nearly any device whether it is phone, tablet PC or Mac. Note that the editable files created in ScreenFlow are rather large and potentially cumbersome, so having an external hard drive is a nice way to store them safely without having to burden a device with these large files. A sixteen-minute video as an editable Screenflow file is 8.3GB. Once converted to .MOV or .m4v the final file is only 170MB. Once exported, storing the files on Google Drive or on an internal server allows for easy access via links on course websites or your institution’s learning management system (LMS). To complement the online lectures, students can be given access to all the PowerPoint slides used in the lectures. They can choose between viewing them electronically, downloading and printing them themselves, or purchasing the printed version from the bookstore/campus copy center in a bound volume. 129 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

Anecdotal evidence suggests that students listen more actively (as if they were in class) and are more successful when they have the printed slides in front of them while watching the video lectures. The in-class portion of the course can be based on a variety of proven highimpact practice strategies. Whether it is unique applications of the basic concepts presented in literature-centered case studies, POGIL activities or simply working problems from the back of the chapter in groups, the idea is to engage students in activities that reinforce key concepts and allow them to practice problem solving in an atmosphere of “social constructivism (14–16).” The activities should follow the central dogma of the high impact classroom where activities are effortful, build relationships, allow the instructor to provide immediate feedback/coaching and apply/reinforce the knowledge gained from outside of class efforts (17). Small groups of 3 to 4 established at random and shuffled after each exam has been the methodology to date based on previous reports (18). This author uses a 50-50 split between end of chapter or exam-style questions to engage students with the material and case study assignments either self-authored or published (19–21). Regardless of the activity the students are given the material ahead of time and encouraged to work through the material on their own before class, bringing to class questions that might have come up during the process of working on the problems or cases. The ideal scenario is for a student to truly engage in the material outside of class and bring specific roadblock questions to class to discuss with classmates and the instructor. This scenario rarely plays out, yet class-time can be fruitful without much pre-class preparation. Usually students spend more time in-class focusing on background information and orienting themselves to the problems rather than delving deeply into the nuances of the biochemical concepts. The key to successful in-class activities is to make sure the students have a product to be graded to turn in at the end of class even if it is just the answer to one problem completed by the group. This ensures that the groups remain focused on the material rather than on side conversations or current events. The “hand-in” at the end of class also provides the instructor a vehicle to provide feedback on their written work prior to an exam. Because of the structure of the flipped classroom there is no limit on the creativity of the instructor for in-class activities. Games, skits and even serious discussions about current research articles and ethical dilemmas can be brought before the class without having to worry about sacrificing precious lecture time needed to cover content.

130 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

Results Observations and data collected for this chapter are from a typical semester at St. Olaf College with 12 weeks of instruction covering the content for a 300 level biochemistry class targeted at junior and senior undergraduates who have completed two semesters of 200-level organic chemistry. In the 12 weeks the students were assigned 51 video lectures to be viewed outside of class to support their reading of the textbook. The lectures ranged in length from five minutes to 30 minutes with an average video length of 16 minutes. The tone of the video lectures could be described more as a one-on-on tutorial during office-hours rather than a lecture-hall style presentation. During a given week the average time students were assigned to spend outside of class listening to lectures was a little over 60 minutes provided they did a single viewing. By flipping the classroom all students were guaranteed at least 165 minutes per week of active, guided engagement in material related to problem solving and critical thinking. With an average class size of 32, dividing the students into eight groups of 4 allowed for personal interactions with each group during the scheduled meeting time and a manageable grading load for the in-class assignments. In the seven semesters prior to flipping the classroom, 34 exams were administered to 325 students. The typical exam format consisted of 7-8 short answer questions where the students had to complete a calculation, draw a figure, interpret data or write a response in their own words within the 1-hour time limit. The median score on these exams was 82%. In the six semesters following flipping the classroom 24 exams were administered to 196 students. Exam format and time restriction was comparable to those administered before flipping the classroom. The median score on these exams was 79%. A two-tail distribution analysis gave a P level of 0.009 that suggests that there was no difference in the median score between the two data sets. While there was no significant difference between the median scores before and after flipping the classroom, a detailed analysis of the exams themselves showed a difference in the difficulty of the exams. Each question on each exam was evaluated and ranked according to Bloom’s taxonomy (22). Table 1 below shows the criteria for ranking each test question and Table 2 shows the average percentage of exam points assigned at each level before-and-after flipping the class.

131 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Table 1. Exam question levels and corresponding Bloom’s taxonomy

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

Level

Type

Rationale

Key Words

1

Knowledge

Requires the student to recognize or recall information

list, label, define, describe

2

Comprehension When students can reproduce and communicate ideas and information without verbatim repetition

arrange, identify, explain, sort, classify

3

Application

The ability to use this information in particular, concrete, situations

choose, solve, draw demonstrate, prepare

4

Analysis

Breaking down ideas into constituent parts in order to make the organization clear

analyze, contrast, examine, test, compare

5

Synthesis

The ability to integrate ideas into a unified whole

create, design, propose, modify

6

Evaluation

The ability to judge the value of an idea, model, procedure etc. using appropriate criteria

judge, predict, defend, support, assess

Table 2. Aggregate percentages of exam points at each level before and after flipping the classroom. Level

Before

After

1-2

49%

26%

3-4

42%

51%

5-6

9%

23%

Each semester following flipping the classroom, students were given an assessment worksheet and asked to respond to a series of prompts on a five point Likert scale where the highest response was “strongly agree” and the lowest response was “strongly disagree”. Questions could be divided into two categories one having to do with the structure of the class (Table 3) and one having to do with personal growth during class (Table 4).

132 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Table 3. End of course survey questions used to measure student opinion of the structure of the course Structure I feel the book was valuable to learning biochemistry I feel working problems in groups during class time was valuable to learning biochemistry I feel the On Line Lectures (OLL) were a suitable replacement for in-class lectures Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

I feel the instructor created a supportive, encouraging environment for learning

Student responses over five semesters are summarized in aggregate form in figure 2.

Figure 2. Results summarized from Table 3 questions. The solid point is the average percentage of students who responded, “agree” or “strongly agree” to the corresponding question. The vertical bar spans the range of responses over five semesters.

133 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Table 4. End of course survey questions used to measure student opinion of personal gains during the course Growth I feel I improved my ability to think beyond the basics I feel I improved my ability to work with others I feel I improved my ability to solve problems

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

I feel I improved in my ability to be a more independent learner

Student responses over five semesters are summarized in aggregate form in figure 3.

Figure 3. Results summarized from Table 4 questions. The solid point is the average percentage of students who responded, “agree” or “strongly agree” to the corresponding question. The vertical bar spans the range of responses over five semesters.

Discussion Despite the growing evidence that the flipped classroom is an excellent method for delivering both content and building lifelong learning skills, student feedback can be negative as they adapt to this new environment (3, 5, 6, 15, 23–25). This is likely true in many situations where students are asked to switch from the low energy-passive classroom to an active high-energy learning environment. An important aspect in course design for the flipped classroom is to provide students with the opportunity to understand the methodology and pedagogy being employed in the flipped classroom. This initial buy-in allows them to understand the rationale and embrace that the changes are being done 134 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

in their best interest. One simple technique is to always refer to the videos as “lectures” whether it is “online lectures” or “video lectures” or other vernacular. Keeping the term “lecture” in the title insures students that they are not missing out on an all too familiar teaching strategy. Another technique that alleviates some of the pushback from students is to share with them some examples from the growing body of scientific evidence that their grades will be better and that their learning will be deeper and last longer in the active learning, flipped classroom model (26). These small things can help prevent misunderstandings and provide motivation down the road. Additionally simply stating, “we learn by doing” helps launch activities on a positive note. With the growing number of responsibilities that students take on during the course of a semester, it is important to respect the amount of time students work on each of their courses. The flipped classroom model with online lectures ensures that the students will be exposed to 165 minutes every week (3 x 55 minute class periods) of quality, active, efficient learning that once was used to deliver lecture. In this model, questions that do arise can be addressed immediately either by asking a classmate or the instructor. This immediacy helps to maintain focus during a problem solving session and prevents wasted time in frustration (27). In addition to this efficiency it also creates an environment for informal discussions, allowing students to conjecture and imagine and bring things together from other classes or research that might be important to their studies as a whole. One of things that struggling students fail to do is to regularly attend office hours. The flipped classroom model creates office hours within the classroom, allowing a vehicle for communication between student and instructor. Additionally the more informal style allows the instructor to get to know students more personally and lowers the potential intimidation factor for dropping in during office hours. This personal interaction also allows for deeper, more meaningful content in letters of recommendation requested by the students. As faculty, we know that exam writing is both an art and a science. In good faith, we write exams that are at the appropriate level for our students; our goal is to be challenging and creative, but fair. It was under this assumption that the initial analysis of the aggregate exam-score analysis before and after flipping the classroom was so disappointing. The students should have done better according to the research (24, 26, 28–33). It was honestly unintentional that the exams became more challenging after flipping the class. Each exam was crafted knowing the student’s skills and abilities with the perception that the students were prepared and had practiced (with coaching) answering the more difficult questions. Only after the fact were the exams analyzed and determined to contain a higher percentage of the Bloom’s level 3 questions. This serendipitous result speaks to addressing two of the essential learning outcomes as outlined by the Association of American Colleges and Universities in their LEAP campaign (34). The presence of questions at all three levels allows a student to demonstrate their knowledge of the physical and natural world. This content assessment is an essential portion of the course and is not sacrificed by having an active learning environment. What perhaps is more significant and speaks to the measurement of the second essential learning outcome where the demonstrated success on the more difficult exams shows how the students are 135 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

able to leverage their gains in an active learning environment giving them the confidence and opportunity to be successful. What we see are students developing and working towards mastery of their intellectual skills honing and refining them, particularly with respect to inquiry and analysis and critical and creative thinking. Despite the growing use of technology in society, the most valued interactions for learning as perceived by the students were due to direct human interactions. The percentage of students agreeing to statements about working in groups and the instructor’s interventions during class were greater than those agreeing to the efficacy of video lectures or even reading the textbook to their learning. This is continued evidence that the flipped classroom strategy optimizes interactions the students find most favorable for their learning. Finally, a large part of the active learning classroom centers on working in teams and teamwork falls under the AAC&U’s practical skills essential learning outcomes. The key feature for a successful team is quality and effective oral communication. Regardless of a student’s vocation, speaking clearly and listening to understand are skills that are needed and need to be practiced. This tenet of the liberal arts curriculum is accomplished in the flipped classroom without sacrificing the course content. The self-reported data speak to the efficacy of the course design. While many of the St. Olaf students need little coaching on how to be a team player or effective communicator, an overwhelming majority admitted to improving critical thinking, teamwork and problem solving as part of their experience in the flipped classroom.

Conclusion This chapter has illustrated the efficacy of the flipped classroom in the context of an upper-level liberal arts biochemistry course. Following the “good enough” philosophy allowed video lectures to be created and disseminated easily and efficiently with a low activation energy. Pairing video lectures with high impact practices in the classroom preserved content and allowed for students to thrive in an environment where the expectation was to work at a higher cognitive level as measured by exam data. Finally, the flipped classroom promoted outcomes that are in-line with the essential learning outcomes of a liberal arts education specifically, inquiry, critical thinking, oral communication and teamwork supported by student self-report data.

References 1. 2.

3.

Prud’homme-Généreux, A. Student-Produced Videos for the Flipped Classroom. J. Coll. Sci. Teach. 2016, 45, 58–62. Morgan, H.; McLean, K.; Chapman, C.; Fitzgerald, J.; Yousuf, A.; Hammoud, M. The flipped classroom for medical students. Clin. Teach. 2015, 12, 155–160. Mortensen, C. J.; Nicholson, A. M. The flipped classroom stimulates greater learning and is a modern 21st century approach to teaching today’s undergraduates. J. Anim. Sci. 2015, 93, 3722–3731. 136 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

4.

5.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

6.

7. 8. 9. 10.

11. 12. 13. 14. 15.

16.

17. 18.

19. 20. 21. 22.

Kay, R. H. Exploring the use of video podcasts in education: A comprehensive review of the literature. Comput. Hum. Behav. 2012, 28, 820–831. Love, B.; Hodge, A.; Grandgenett, N.; Swift, A. W. Student learning and perceptions in a flipped linear algebra course. Int. J. Math. Educ. Sci. Technol. 2014, 45, 317–324. McLean, S.; Attardi, S. M.; Faden, L.; Goldszmidt, M. Flipped classrooms and student learning: not just surface gains. Adv. Physiol. Educ. 2016, 40, 47–55. Westermann, E. B. A Half-Flipped Classroom or an Alternative Approach?: Primary Sources and Blended Learning. Educ. Res. Q. 2014, 38, 43–57. Drouin, M.; Hile, R. E.; Vartanian, L. R.; Webb, J. Student Preferences For Online Lecture Formats. Q. Rev. Distance Educ. 2013, 14, 151–162. Lau, W. M.; Chong, D. W. Combining Team-Based Learning with a Flipped Classroom Approach: Is This Possible. Med. Educ. 2014, 48, 8–8. Waltz, C. F.; Jenkins, L. S.; Han, N. The Use and Effectiveness of Active Learning Methods in Nursing and Health Professions Education: A Literature Review. Nurs. Educ. Perspect. 2014, 35, 392–400. Ferreri, S. P.; O’Connor, S. K. Redesign of a Large Lecture Course Into a Small-Group Learning Course. Am. J. Pharm. Educ. 2013, 77, 1–9. Schwartzstein, R. M. Getting the Right Medical Students -- Nature versus Nurture. New Engl. J. Med. 2015, 372, 1586–1587. Screencasting and Video Editing Software http://www.telestream.net/ screenflow (accessed Mar 1, 2016) Herreid, C. F.; Schiller, N. A. Case Studies and the Flipped Classroom. J. Coll. Sci. Teach. 2013, 42, 62–66. Pierce, R.; Fox, J. Vodcasts and Active-Learning Exercises in a "Flipped Classroom" Model of a Renal Pharmacotherapy Module. Am. J. Pharm. Educ. 2012, 76, 1–5. Powell, K. C.; Kalina, C. J. Cognitive and Social Constructivism: Developing Tools for an Effective Classroom. Education 2009, 130, 241–250. Kuh, G. D. High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter. Peer Review 2012, 14, 29–29. Eberlein, T.; Kampmeier, J.; Minderhout, V.; Moog, R. S.; Platt, T.; VarmaNelson, P.; White, H. B. Pedagogies of engagement in science. Biochem. Mol. Biol. Educ. 2008, 36, 262–273. National Center for Case Study Teaching in Science. http:// sciencecases.lib.buffalo.edu/cs/ (accessed Mar 1, 2016). Cornely, K., Cases in Biochemistry. John Wiley & Sons: New York, 1999. Jenny Loertscher, V. M., Foundations of Biochemistry. 3rd ed.; Pacific Crest: Lisle, Il, 2011. Bloom, B., Englehart, M. Furst, E., Hill, W., Krathwohl, D., Taxonomy of educational objectives: The classification of educational goals. Handbook I: Cognitive domain. Longmans, Green: New York, Toronto, 1956.

137 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Downloaded by UNIV OF CALIFORNIA SAN DIEGO on December 12, 2016 | http://pubs.acs.org Publication Date (Web): December 1, 2016 | doi: 10.1021/bk-2016-1228.ch008

23. Khanova, J.; McLaughlin, J. E.; Rhoney, D. H.; Roth, M. T.; Harris, S. Student Perceptions of a Flipped Pharmacotherapy Course. Am. J. Pharm. Educ. 2015, 79, 1–8. 24. Peterson, D. J. The Flipped Classroom Improves Student Achievement and Course Satisfaction in a Statistics Course. Teach. Psychol. 2016, 43, 10–15. 25. Rossi, R. D. ConfChem Conference on Flipped Classroom: Improving Student Engagement in Organic Chemistry Using the Inverted Classroom Model. J. Chem. Educ. 2015, 92, 1577–1579. 26. Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. U.S.A 2014, 111, 8410–8415. 27. Baepler, P.; Walker, J. D.; Driessen, M. It’s not about seat time: Blending, flipping, and efficiency in active learning classrooms. Comput. Educ. 2014, 78, 227–236. 28. Haughton, J.; Kelly, A. Student Performance in an Introductory Business Statistics Course: Does Delivery Mode Matter. J. Educ. Bus. 2015, 90, 31–43. 29. Mason, G. S.; Shuman, T. R.; Cook, K. E. Comparing the Effectiveness of an Inverted Classroom to a Traditional Classroom in an Upper-Division Engineering Course. IEEE Trans. Educ. 2013, 56, 430–435. 30. Ryan, M. D.; Reid, S. A. Impact of the Flipped Classroom on Student Performance and Retention: A Parallel Controlled Study in General Chemistry. J. Chem. Educ. 2016, 93, 13–23. 31. Talley, C. P.; Scherer, S. The Enhanced Flipped Classroom: Increasing Academic Performance with Student-recorded Lectures and Practice Testing in a "Flipped" STEM Course. J. Negro Educ. 2013, 82, 339–347. 32. Tune, J. D.; Sturek, M.; Basile, D. P. Flipped classroom model improves graduate student performance in cardiovascular, respiratory, and renal physiology. Adv. Physiol. Educ. 2013, 37, 316–320. 33. Weaver, G. C.; Sturtevant, H. G. Design, Implementation, and Evaluation of a Flipped Format General Chemistry Course. J. Chem. Educ. 2015, 92, 1437–1448. 34. The LEAP Challenge: Education for a World of Unscripted Problems; American Association of Colleges and Universites: 2015.

138 Muzyka and Luker; The Flipped Classroom Volume 2: Results from Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2016.