Using a Partially Flipped Learning Model To Teach First Year

Mar 18, 2019 - Sample of a lecture worksheet showing examples of theory based and interactive lecture ... Sample course evaluation questionnaire (PDF)...
0 downloads 0 Views 2MB Size
Download PDF
Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

pubs.acs.org/jchemeduc

Using a Partially Flipped Learning Model To Teach First Year Undergraduate Chemistry Rena Bokosmaty,*,† Adam Bridgeman,† and Meloni Muir‡ †

School of Chemistry and ‡School of Medical Sciences, The University of Sydney, Sydney, NSW 2006, Australia

Downloaded via DREXEL UNIV on March 19, 2019 at 04:39:27 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.

S Supporting Information *

ABSTRACT: As part of curriculum renewal in three introductory chemistry courses at the University of Sydney, a partially flipped classroom model was implemented. Content that is conventionally delivered in the in-class sessions was moved online through the use of web-based tutorials, videos and quizzes. During the in-class sessions, active learning opportunities were created through the use of guided-inquiry worksheets and clicker responses to promote peer-to-peer and instructor-led discussion. Student evaluations indicated high satisfaction with the quality of teaching and learning resources. In comparison to before these changes, there was a statistically significant increase in the number of students achieving the higher grades across all three courses. Failure rates decreased or remained unchanged. The course with the historically highest attrition showed a marked improvement. These results suggest that a partially flipped learning model can be useful to enhance student engagement, support learning, and positively impact on retention and academic performance. KEYWORDS: First Year Undergraduate/General, Curriculum, Inquiry-Based/Discovery Learning, Student-Centered Learning

■

INTRODUCTION AND CONTEXT Introductory chemistry courses are often perceived to be conceptually difficult for undergraduate students due to specialized language, symbols, and the abstract nature underlying many chemical concepts.1 Traditionally, these courses are taught by didactic lecture-based approaches.2 In this teacher-centered approach, students passively engage with the learning process and often attain knowledge without understanding the foundational principles.3 Students may struggle with retaining, transferring or applying these concepts to real-life applications or future courses.4 To address these issues, the higher education sector has recognized the value of adopting student-centered approaches such as the flipped classroom.5−10 This paper describes a partially flipped classroom model introduced into three large introductory chemistry courses as part of a curriculum renewal to improve student engagement. The flipped classroom is a pedagogical approach that is becoming increasingly popular in higher education chemistry. In a flipped classroom, the instructor “flips” or “inverts” the learning design, so that students are initially exposed to course content outside of class, through videos and quizzes delivered online.11−13 The in-class sessions are designed to promote students’ engagement with interactive learning activities.12 This model provides students with the opportunity to collaboratively apply their problem-solving skills and develop their conceptual and procedural understanding of concepts.7,8 The design and implementation of the model can vary widely across different educational contexts.5−12 In a partially flipped learning model, instructors flip some aspects of the course by identifying where learning outcomes are most effectively delivered in-class and in the online learning environment.7 The design principles of the flipped approach are grounded in two theoretical frameworks: social constructivism14,15 and cognitive load theory.16 Bodner17 emphasizes the importance © XXXX American Chemical Society and Division of Chemical Education, Inc.

of students actively engaging in the learning process and constructing their own knowledge and understanding by building on existing prior knowledge. The theory of constructivism recognizes that learning is mediated through social interactions.15 In a flipped learning model, the preclass work builds on students’ prior knowledge and forms the foundational basis upon which students can develop a deeper understanding of the material addressed in class.18,19 According to cognitive load theory,16 the working memory consists of a limited space in which information is being used, processed, and stored.20 If the process of learning new material overwhelms the working memory, a student’s ability to process new information can be restricted.19,21 The online learning resources used in the flipped learning model can reduce student cognitive overload during the in-class session.8,22 The use of videos before class allows students to initially process knowledge at a pace that suits their learning needs.19,23 This could reduce demands on the working memory during the inclass session and potentially increase the students’ ability to process new knowledge.10,22 A flipped learning model can enhance student satisfaction and engagement with learning content.7−10,19 The majority of studies based on student evaluations have revealed that they appreciate the structure of the model and the ability to access learning material at their own pace. However, some students perceive the approach to involve more preparation time compared to learning in a traditional format.24 Christiansen25 revealed that students’ preference to learn in a flipped model improves over time. This suggests that students may require an adjustment period to become familiar with learning how to study in a flipped classroom compared to a traditional format. Received: June 3, 2018 Revised: February 15, 2019

A

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Other studies have evaluated the impact of a flipped or partially flipped learning model on student academic performance. Generally, improvements in course grades and reduction in failure and withdrawal rates have been observed.7−10,18,19,22,25−27 Variations in performance, however, have been noted. A two-year longitudinal study in first year general chemistry showed significant improvements in American Chemical Society (ACS) exam scores in the first year of the flipped course implementation but not in the second year.26 Weaver and Sturtevant’s three-year longitudinal study demonstrated a significant increase in ACS exam scores in a flipped format compared to a traditional lecture-based format with students performing better in conceptual problem solving than algorithmic problems.9 Yesterbsky’s parallel study reported that there was no significant difference in the ACS exam scores between the flipped and traditional course formats.27 However, students in the flipped format showed significantly higher overall course grades but no changes in D and F grades. Ryan and Reid’s year-long parallel controlled study noted only the bottom third of students in the flipped course showed significant improvement across all five exams compared to students in the nonflipped course.10 In the flipped course, there was a significant decrease in D, F, and withdrawal (DFW) rates compared to the control. In Flynn’s study of four flipped chemistry courses, significant improvements in grades and decreased DFW rates in the flipped courses were observed.7 Shattuck’s controlled parallel study18 in a partially flipped organic chemistry course noted a significant improvement in exam question answers for topics delivered in the partially flipped format. When comparing the two formats overall, course grades were similar. In the flipped format, however, there was an increase in the A and B grades and a decrease in withdrawal rates. The partially flipped format consisted of flipping a subset of classes (a third of the course) that focused on challenging organic chemistry topics. Trogden’s parallel controlled study using a partially flipped format observed slight improvements in average exam scores and an 11% decrease in DFW rates in the partially flipped segments.28

(CHEM1101) and the Advanced Chemistry course (CHEM1901). The three courses have broadly similar learning outcomes but cater to students with different chemistry backgrounds (Table 1). Table 1. Comparative Student Demographics in Three Introductory Chemistry Courses Parameters Average enrollment (2013−2016), N High school chemistry background knowledge Sex: female (%) Sex: male (%) Average agea English language speakers (%) Chinese language speakers (%) Other language speakers (%) a

CHEM1001

CHEM1101

CHEM1901

603

867

208

No prior

Sound

Exceptional

64 35 19.7 63 15 22

56 44 18.8 56 23 21

54 46 18 64 21 15

Median age was 18 for each of these classes.

Course Design and Structure

The partially flipped learning model implemented consists of the same weekly face-to-face teaching mode: three interactive in-class sessions (50 min each) and one tutorial (50 min) over 13 weeks plus 3 h laboratory over 9 weeks. Assessment for each course includes an end of semester examination (60%), 3 tutorial quizzes (15%), weekly online quizzes (10%), and laboratory work (15%). In this model, content is not delivered entirely didactically; instead, there is a combination of approximately 50% instructor-led and 50% integrated active learning activities supported by the online learning resources (Figure 1). On the

■

AIM The aim of this paper is to describe a partially flipped learning model introduced into three large introductory university chemistry courses as part of a curriculum renewal and contribute to the understanding of the following: 1. Student perceptions toward learning in a partially flipped model, 2. Student academic performance in a partially flipped learning model, 3. Student retention in a partially flipped learning model.

■

Study Context

Figure 1. Overview of the partially flipped design implemented in the chemistry courses. The preclass work is delivered in the online learning platform. In-class learning activities are delivered in segments of varying length; suggested timing in minutes is provided.

This study was carried out in three first year undergraduate chemistry courses at a large comprehensive, research-focused institution, the University of Sydney. Students complete one of these three courses to either major in chemistry or to meet the requirement of other degree programs. The partially flipped learning approach was initiated in 2013 for the Fundamentals of Chemistry course (CHEM1001). In 2015, the model was implemented in the mainstream Chemistry course

basis of the coordinator’s experience with the course content and desired learning outcomes, key concepts (e.g., moles, nomenclature, orbitals, etc.) were selected for the online component for developing foundational knowledge and understanding of introductory chemistry. These concepts require a developmental learning approach whereby students

METHODOLOGY

B

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 2. Example of the tool for contributing links and resources.

the students and instructors perceive the resource. Instructors’ votes, however, receive a higher weighting than those of students.

engage with them a number of times prior to the in-class session to build their understanding. In-class sessions are delivered by one instructor in a fixed row-by-row, tiered, theater-style auditorium seating approximately 300 students. Each course has 2 or 3 live repeat streams of the weekly in-class sessions as there is no one venue on campus large enough to accommodate all enrolled students at one time. The same content and slides are used to ensure consistency across streams. A team of six academics have continuously taught these courses from 2009 to 2016. One of the authors (Bridgeman) was the first year coordinator from 2007 to 2015 and designer of the learning resources (online quizzes and videos, and in-class worksheets) described in this article. Students are expected to view one online video, complete one online quiz, and attend 3 in-class sessions each week. The tutorial and laboratory classes, taught by the lecturers and graduate students, were not part of the curriculum renewal and will not be discussed.

Web-Based Tutorials

Web-based tutorials were developed to provide targeted explanations of particular chemistry concepts.30 The interface used also incorporates interactive visual representations, animations, and manipulations to transform the abstract nature of chemistry and foster an interactive learning environment. For example, if students need to develop an understanding of molecular vibrations of common molecules, the platform allows them to visualize differences between a bend, a symmetric, and an asymmetric stretch.31 Videos

Weekly preclass videos using screencasts capturing audio narration over slides were developed in-house. These costeffective resources were then published and delivered through a YouTube channel and are embedded in the LMS. The screencast materials are chunked into short segments (5−10 min) to maintain student engagement and minimize the preclass workload. The videos are typically used to introduce difficult concepts (e.g., thermodynamics), to review foundational concepts (e.g., atomic structures), or work through mathematical calculations (e.g., those involving moles).32 The use of videos to review traditionally difficult concepts has been shown to assist students to self-direct their learning due to their flexible access allowing students to progress at a pace that suits their learning needs.6,11,33

■

DESIGN OF THE ONLINE LEARNING SPACE/PREWORK The University’s learning management system (LMS; Blackboard Learn) is the primary platform by which students access course materials: lecture notes, podcasts, tutorial sheets, videos, discussion forum, etc. In the renewed curriculum, an active e-learning site29 was developed on a separate server and embedded in the LMS. This site was created by one of the authors (Bridgeman) specifically for these first year courses. Course resources are extended by a “crowd sourcing” approach similar to that used on news aggregation sites. For each topic, members of the teaching team and students are invited to add resources under specific topics (e.g., links to online videos and open textbooks). The user completes a simple form providing the web address of the resource, a brief description and optionally, their name (Figure 2). Each resource is automatically tagged to indicate the topic covered and who, student or instructor, added the item. The list of contributions is displayed to students in their weekly resources, ordered according to votes based on how “useful” or “useless”

Online Quizzes

Online quizzes provide an opportunity for students to answer concept-building questions addressed in the prework. They are designed as low-stake, formative assessments; an incentive mark (1%) is given for each contributing to a total of 10%. The mastery format of these quizzes allows students multiple attempts to improve their score within a two-week period. This period of availability helps ensure that all students across the different streams have the chance to complete them. For each attempt, quiz questions are randomly drawn from a large, purpose-built pool of questions, thus ensuring that a different C

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

familiarize students with the use of terminology necessary to engage in chemical conversations that take place during class and laboratory components of the course.34 They also encourage students to work through procedural steps and answer calculation-based questions. Drag and drop questions help students build a mental model or representation of theoretical models or chemical structures.

set of questions at the same level is generated for each attempt. At the end of the two-week period, the score of the best attempt is recorded in the LMS gradebook. Once graded, the quiz remains open for review only. The nature and design of these quizzes support students gaining mastery of concepts in a time-efficient manner. Quizzes vary in length and format, consisting of 5−10 single best answer questions (SBAs), short answer questions (SAQs) or interactive drag and drop questions (Figure 3). Quiz

Discussion Platform

An online discussion forum (Piazza) is embedded in the LMS. This free, web-based platform promotes social interaction, encouraging students to voluntarily participate in peer or instructor-led online discussions. The platform is highly interactive and is organized by week with student-posed questions categorized under relevant subheadings. Instructors are able to acknowledge a student’s contribution, endorse questions and answers, monitor and moderate responses. The Supporting Information contains an example of a studentinstructor discussion.

■

ACTIVE IN-CLASS LEARNING COMPONENT To facilitate a student-centered learning environment, active learning pedagogies are embedded in the in-class component. The structure of the in-class sessions was influenced by the Process Oriented Guided Inquiry Learning (POGIL) instructional initiative.35 In a POGIL learning environment, students work in small self-managed groups on specifically designed inquiry-based activities facilitated by the instructors. They are given the opportunity to explore and construct their own understanding and develop skills to apply their knowledge in various contexts.35,36 A recent meta-analysis revealed that POGIL improves students’ process skills and leads to measurable success in academic performance and retention rates.37 In our study, the design of the learning activities were developed and evolved through an initial national project involving collaboration with the POGIL team.38 The materials presented during the in-class sessions are delivered in a series of segments.39−41 The structure and the length of each segment (Figure 1) are designed to optimally engage students. Each in-class session begins with a brief instructor-led lecture that focuses on introducing the concepts to be addressed. Then students work in groups completing inquiry-based worksheets. Students discuss their answers within the group while the instructor walks about engaging with the groups. Compared to the traditional POGIL learning environment, the learning space available, a tiered lecture theater with seats in fixed rows, is challenging and typically does not suit formal group work. Additionally, only a single instructor is available for each class of approximately 300 students. As such, the process skill development is less formalized and necessarily rather more teacher-centered than in the POGIL approach. To address this and promote increased student-instructor interaction, after each section of the worksheet, instructors encourage students to participate in clicker questions to gauge their understanding of the material before delivering the next lecture segment. The worksheets contain application-based activities similar to those implemented in a POGIL learning environment. The worksheets are a two-sided, single sheet with a very short description of the theory and a set of questions broken down into two or three self-contained blocks. Hard copy worksheets are provided to students at the beginning of each in-class

Figure 3. Examples of weekly online quiz formats and autogenerated feedback for (a) single best answer choice questions, (b) short answer questions, and (c) drag and drop questions.

content encompasses terminology, chemical diagrams and structures, rules, and practice algorithmic and procedural-based concepts. Autogenerated answers and feedback are provided for all questions using software developed by one of the authors (Bridgeman). SBAs are designed to reflect misunderstandings associated with the underlying principle of the particular concept. SAQs D

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 4. Student satisfaction regarding the quality of teaching across first year science courses based on a 5-point Likert scale ranging from strongly agree (SA) to strongly disagree (SD). Chemistry course results are reported before and after curriculum renewal. Student response rate before implementation (CHEM1001: 29% (n = 104); CHEM1101: 27% (n = 134); CHEM1901: 10% (n = 24)). Student response rate after implementation (CHEM1001: 20% (n = 120 average for 2013−2016) and 29% (n = 162 average for 2015−2016); CHEM1101: 35% (n = 317); CHEM1901: 37% (n = 75)). Other first year science course results are provided for comparison (2015−2016). Student response rate (Biology: 29% (n = 3124); Physics: 26% (n = 937); Mathematics: 29% (n = 6416); Overall Science: 29% (n = 15836); Overall Advanced Science: 40% (n = 688)).

Figure 5. Student satisfaction regarding the quality of learning resources across first year science courses based on a 5-point Likert scale ranging from strongly agree (SA) to strongly disagree (SD). Chemistry course results are reported before and after curriculum renewal. Student response rate before implementation (CHEM1001: 29% (n = 104); CHEM1101: 27% (n = 134); CHEM1901: 10% (n = 24)). Student response rate after implementation (CHEM1001: 20% (n = 120 average for 2013−2016) and 29% (n = 162 average for 2015−2016); CHEM1101: 35% (n = 317); CHEM1901: 37% (n = 75)). Other first year science course results are provided for comparison (2015−2016). Student response rate (Biology: 29% (n = 3124); Physics: 26% (n = 937); Mathematics: 29% (n = 6416); Overall Science: 29% (n = 15836); Overall Advanced Science: 40% (n = 688)).

Formative clicker questions are embedded throughout the class time providing the instructor and students with real time feedback on their progress. Groups of students submit agreed answers. These real time responses allow instructors to identify misconceptions, as well as tailor topic discussions to student interests and promote a social learning environment.15,35,43 Formative clicker questions are used to extend sections of the worksheet to stretch students who have finished the assigned work ahead of the majority of the class. For example, in transition metal chemistry, students are presented with additional problems on d-electron configurations. In addition, formative clicker questions are used to poll students for opinions on certain topics. For example, in 2015, before

session as well as online postclass through the LMS. Two types of worksheets are used; either a worksheet that is predominately theory based or one that focuses on integrating interactive lecture demonstrations (ILDs).42 Representative worksheets are available in the Supporting Information. The ILD-integrated worksheets provide students with an opportunity to develop a better understanding of theoretical concepts.35,36 For each demonstration, students are required initially to predict what will occur as well as note their observations. Concept building questions embedded in the ILD worksheet guide students and encourage peer-to-peer collaboration through sharing of answers. E

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

implementation (2013−2016). Student satisfaction was higher than that of first year Biology (75%), Physics (77%), and Mathematics (79%) courses and all other first year science courses (79%) during 2015−2016. Data for 2013 and 2014 were not available. CHEM1101 student satisfaction before curriculum renewal (2009−2014) was 64% and improved to 88% postmodel implementation (2015−2016). Student satisfaction was again higher than that in all other first year science courses. CHEM1901 satisfaction before the implementation (2009−2014) was 57% and improved to 64% with the implementation of the model (2015−2016). CHEM1901 satisfaction levels toward the learning resources, however, were lower than all other Advanced first year science courses (76%) during 2015−2016. The open-ended responses provide insights into the course evaluation data. The majority of students commented positively (239 out of 243) on the structure of the course and the usefulness of the various learning resources in developing their understanding of chemistry. Other comments suggested that the model used in chemistry should be implemented in other science courses. There were very few negative comments (6 out of 243). CHEM1901 students in particular suggested that instructors should be more aware of their background knowledge and use resources to extend their understanding. The other negative comments related to technical factors associated with the online learning space.

discussion of an industrial chemistry topic, students were asked for their views on the most important inventions in chemistry and the most important discoveries that needed to be made. Contrary to lecturers’ expectations, students voted in the two questions for the development and improvement of batteries. The same poll in 2016 gave the same results. Class discussion revealed that this interest was, of course, due to poor battery life of their mobile devices. As a result of the poll, the section on electrochemistry was changed to highlight current related research in the School. What had been one of the less popular parts of the curriculum became one of the most engaging.

■

TRACKING AND PERSONALIZED SUPPORT An “Early Warning System” has been implemented in the courses whereby student online course engagement can be tracked (e.g., the last login to the LMS or their online quiz completions). If the system detects limited student engagement, the instructor can send a personalized e-mail to the student. A sample system-generated e-mail is available in the Supporting Information.

■

RESULTS

Student Evaluation of the Partially Flipped Learning Model

At the end of the semester, students are invited to voluntarily complete an anonymous, online course evaluation carried out centrally by the University. As such it is not possible to identify if respondents were representative of the courses as no information regarding student demographics is collected. Student response rates for the course evaluation in Figure 4 and 5 are provided in the Supporting Information. The evaluation consists of ten 5-point Likert scale questions on effectiveness of resources, quality of teaching, engagement, feedback, and overall course satisfaction. In addition, two open-ended questions are included asking what students perceived to be most useful to their learning and what needed improvement. A copy of the course evaluation questionnaire is provided in the Supporting Information. An improvement in student satisfaction with teaching quality was observed in CHEM1001 and CHEM1101 since implementation of the model (Figure 4). CHEM1001 student satisfaction improved from 78% before curriculum renewal (2009−2012) to 85% postrenewal (2013−2016). Student satisfaction was higher than that of first year Biology (73%), Physics (73%), and Mathematics (73%) courses and all other first year science courses (75%) during 2015−2016. Data for 2013 and 2014 were not available. CHEM1101 student satisfaction before model implementation (2009−2014) was 81% and improved to 88% postmodel (2015−2016). Student satisfaction was again higher than that in all other first year science courses. CHEM1901 student satisfaction with teaching quality before model implementation (2009−2014) was 86% and decreased to 64% postimplementation (2015−2016). CHEM1901 satisfaction levels toward the quality of teaching were lower than all other Advanced first year science courses (85%) during 2015−2016. An improvement in student satisfaction with quality of learning resources was observed across the three chemistry courses since the implementation of the model (Figure 5). CHEM1001 student satisfaction before curriculum renewal (2009−2012) was 62% and improved to 85% postmodel

Academic Performance

For each course, historical data were used as the control and compared with data after model implementation to measure students’ academic performance. Table 2 details how the Table 2. Australian University Grade Distribution Grade Name

Grade Code

Mark Range (%)

High distinction Distinction Credit Pass Fail

HD DI CR PS FA

85−100 75−84 65−74 50−64 0−49

University grade descriptors relate to marks. A chi-squared test was performed to compare student grade distributions and failure rates from 2009 through 2016 (Figure 6). Each year, a different exam with the same format, assessing the same learning outcomes, was administrated. Student demographics remained very consistent across the years with no significant changes to the university admission criteria or course requirements. An explanation and summary of the entry requirements are provided in the Supporting Information. CHEM1001 grade distributions since model implementation (2013) were compared to the traditional course format (2009−2012) distributions. Prior to implementation, the HD/DI combined grade distribution ranged between 8 and 10% (Figure 6a). Since model implementation, a statistically significant increase in the percentage of HD/DI grades was found in 2014, 2015, and 2016 but not in 2013 (Table 3). Prior to the model, the failure rate (FA) ranged from 15 to 20% (Figure 6a). Since implementation, a consistent decrease in FA rate was observed. This decrease, however, was not statistically significant when compared to the traditional course format in previous years (Table 3). CHEM1101 combined grade distributions since model implementation (2015−2016) were compared to the tradiF

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 6. Grade distributions for the three chemistry courses (a) CHEM1001, (b) CHEM1101, and (c) CHEM1901 (Advanced). Values shown are the percentage of students’ grades ranging from HD = High distinction (100−85%), DI = Distinction (84−75%), CR = Credit (74−65%), PS = Pass (64−50%), FA = Failure (Below 50%). Vertical black lines indicate when the partially flipped model was implemented. Refer to Table 3 for sample sizes.

Table 3. HD/DI and FA Grade Distribution for the Chemistry Courses before and after Implementation of the Partially Flipped Learning Model Courses

Years

N

Grade of HD/D (%)

p-value

Grade of FA (%)

p-value

CHEM1001

2009−2012 2012 2013a 2014 2015 2016 2009−2014 2014 2015a 2016 2014 vs 2015 2014 vs 2016 2009−2014 2014 2015 2016 2014 vs 2015 2014 vs 2016

460 487 551 616 553 558 642 816 889 889

10 11 13 19 24 24 16 21 40 40

16 16 11 13 11 13 13 10 6 8

250 301 279 230

30 41 61 58

− − 0.081895 0.000018b 0.00001b 0.00001b − − 0.00001b 0.00001b 0.0000001c 0.0000001c − − 0.000001b 0.000106b 0.000004c 0.000245c

− − 0.019547 0.153833 0.018386 0.178406 − − 0.000004b 0.002062b 0.001794c 0.136605 − − 0.0177 0.0210 0.894294 0.88491

CHEM1101

CHEM1901

3 6 6 5

Partially flipped model was implemented in 2013 for CHEM1001, and 2015 for CHEM1101 and CHEM1901. bSignificance at p < 0.01. Significance at p < 0.01 when compared to 2014 data for CHEM1101 and CHEM1901.

a c

G

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

increase in federal funding nationally, which coincided with an increase in enrolment in CHEM1001 (Figure 7). There were, however, no changes to the University entry requirements so there were no significant changes in the student cohort other than an increase in number. Before model implementation (2008−2012), the average retention rate was 90%. From 2013, student retention increased to 95%. Increases were statistically significant in 2013 (6%; χ2 = 6.2176, p = 0.014157), 2014 (5%; χ2 = 13.6866, p = 0.000216) and 2015 (5%; χ2 = 11.607, p = 0.000657). Data for 2016 are not available.

tional course format (2009−2014) distributions. Prior to implementation, the combined HD/DI grade distribution ranged between 12 and 21% (Figure 6b). Since model implementation, there has been a statistically significant increase in the percentage of HD/DI grades (Table 3). There were statistically significant and sustained improvements in the HD/DI grades postimplementation in each year individually (2015 and 2016) as well, even when compared to the quite strong results in 2014. Since the model implementation in CHEM1101, the shift of students from the PS (23%) to CR (28%) is approximately matched by the shift from CR (28%) to HD/DI (40%). Prior to the model (2009−2014), the combined FA rate ranged from 8 to 14% (Figure 6b). Since model implementation (2015−2016), a statistically significant decrease in the combined FA rate was observed (Table 3). The FA rate postimplementation by individual year, however, significantly decreased only in 2015. CHEM1901 combined grade distributions since model implementation (2015−2016) were compared to the traditional course format (2009−2014 distributions; Figure 6). Prior to implementation, the combined HD/DI grade distribution ranged between 25 and 41% (Figure 6c). Since model implementation, there has been a significant increase in the percentage of HD/DI grades. There were statistically significant and sustained improvements in the HD/DI grades postimplementation in each year individually (2015 and 2016) as well, even when compared to the quite strong results in 2014. Since the model implementation (2015) for CHEM1901, the shift of students from the PS (12%) to CR (27%) is approximately matched by the shift from CR (27%) to HD/DI (30%). Prior to the model (2009−2014), the FA rate ranged from 2 to 4% (Figure 6c). No significant change in the FA rate has been observed with curriculum renewal in this course (Table 3). The 2014 HD/DI grade distributions and FA rate prior to the implementation of the model (2009− 2013) were examined for CHEM1101 and CHEM1901. There was a significant increase in the HD/DI grade distribution for both courses in 2014. The FA rate for both of these courses significantly decreased during the same time period (Figure 6b,c).

■

DISCUSSION Overall, student satisfaction regarding the quality of teaching improved in CHEM1001 and CHEM1101 (Figure 4). Students were also satisfied with the learning resources of the partially flipped learning model (Figure 5). The cohorts reported positive aspects of the model and suggested improvements in navigation and technical software. They valued the flexibility of accessing the online learning resources at a pace that suited their learning needs. The interactive inclass learning activities were also appreciated, and students suggested they be adopted in other first year courses. On the basis of student comments, it is evident that the various components of the model impacted positively on their learning experience, in keeping with other published studies.7−10,19 CHEM1901 satisfaction levels regarding the quality of teaching, however, decreased with implementation of the partially flipped model (Figure 4) with students commenting that resources were not sufficiently challenging. Despite this, CHEM1901 students showed significant improvements in their academic performance. A recent study also reported that student perceptions toward the model do not influence their final grades.44 To improve student satisfaction, resources could be modified or revised for CHEM1901 to better align with the chemistry background of students. Our results showed a shift in student grade distributions over time, with significant increases in higher grades. In CHEM1001 in the first year of the model, no significant change in student academic performance was observed, but in subsequent years significant improvement was noted (Figure 6). This could be related to initial challenges experienced by instructors teaching with a new instructional approach. Initially, they may have been challenged by the new model to effectively deliver the in-class material, such as leading class discussions and facilitating an active learning environment with 300 students at a time. In CHEM1101 and CHEM1901, courses in which the model was introduced after two years of use in CHEM1001, student performance consistently improved beginning in the first year of model implementation (Figure 6). Instructors had been teaching with this model for two years and would be familiar with the approach. Our findings align with other studies reporting significant improvements in student grades in flipped and partially flipped chemistry courses.7−10,19,22,23 It has been suggested that the organized structure of this approach scaffolds and better supports student learning and engagement with course material.9,22 In addition, the structure of the model reduces cognitive load as well as allows students to pace their learning to suit their style and needs. Enhanced student engagement with the material and interactions with peers and instructors may also have contributed to improved academic performance. Ryan and

Retention and Withdrawal Rates

Trends in retention and withdrawal for CHEM1001, but not CHEM1101 and CHEM1901, were collected from 2008 to 2015 (Figure 7). In Australian public universities, student places are limited by federal funding. In 2010, there was an

Figure 7. Retention and withdrawal of CHEM1001 students. Blue represents the number of students completing the course, while green represents the number of students withdrawing before the end of the semester. The vertical black line indicates when the partially flipped model was implemented. H

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

between our partially flipped model, or any particular component of it, and the observed improved academic performance and reduced withdrawal rate. Although some demographic features of the student cohort that may have contributed to the observed changes in academic performance were known to be consistent across the semesters, there are others (e.g., those intending to major in chemistry vs nonchemistry majors, first language spoken, etc.) for which information was not available regarding their consistency, and they may also have contributed to the results. Another shortcoming of model implementation was that students were not informed about the teaching model they would be experiencing in their chemistry courses. Students can benefit from explicit guidance on how to study or how they will be taught.9,53 As teaching with a flipped or partially flipped model was not occurring in Science courses at our university at the time we implemented the curriculum change, students would not have been accustomed to learning in this model and, in some cases, their ability to transition and adapt their learning approaches might have adversely impacted their engagement with the course material. A brief introduction outlining the model, expected student responsibilities, and potential benefits is recommended with any pedagogical change.53 The author responsible for developing and preparing the course materials notes that initially the workload for staff was increased, but on the basis of the outcomes in terms of student performance and satisfaction, this investment was worthwhile. After the initial time investment, managing or adjusting any material was minimal. Although the perceptions of the other academics were not formally evaluated, they reported that they enjoyed teaching more in this partially flipped learning format.

Reid10 observed a positive relationship between student attendance and exam performance. In our study, attendance was not formally recorded; however, based on our observations, increased attendance was noticeable compared to premodel years. We also observed higher levels of interactions and discussions between students and instructors despite the staffing and physical constraints. Another potential factor contributing to the improved academic performance may be the feedback mechanisms embedded in our model. The online quizzes, with autogenerated feedback, helped students identify potential gaps and misconceptions in their knowledge (Figure 3) and are essential pacing mechanisms for student learning 45 as well as encouraging engagement with other resources. The use of clickers in class provided students and instructors with instantaneous feedback and an opportunity to address misunderstandings in a timely manner. Clicker responses were also used to encourage discussion among students, which provided the opportunity for peer-to-peer learning, thus building on their understanding of chemistry. Improvements in student performance with clicker use has been noted by others46 and their use could be built upon to strengthen peer teaching.47 For CHEM1001, increased student enrolment in 2010 was followed by an increase in withdrawal and failure rates in 2010−2012. Since the introduction of the partially flipped model, withdrawal rates have significantly decreased in this course (Figure 7). Factors of our model potentially contributing to this reduction may be the weekly reinforcement of concepts online and in-class facilitated by the partially flipped model, as well as the use of the early warning system. Students may have felt better supported and hence remained enrolled. This finding supports work by Shattuck18 that found a significant decrease in withdrawal rates when partially flipping a third of their organic chemistry course. Similar reductions in withdrawal rates were observed in Flynn’s study7 when comparing student withdrawal rates in the flipped format to the traditional course format. A recent meta-analysis noted the use of active learning strategies in STEM courses significantly improved student retention rates.48 Failure rates decreased but only significantly in CHEM1101 (Figure 6, Table 3). It is difficult to identify why lower achieving students do not benefit in the same way as their higher achieving colleagues. Motivation has been identified as an important factor in students’ learning and academic performance with low achieving students showing the most pronounced decline in motivation during a semester.49−52 Academically successful students may be better able to adapt to different learning and teaching models leading to improvements in their academic success, while academically less successful students are less able to successfully adapt their learning to a new situation,49 resulting in little change in FA rates among this group. The impact of a flipped or partially flipped model on FA rates reported in the literature is inconsistent with some studies identifying no change while others show a reduction.10,18,27,28 Silverthorn53 notes that there will always be some students that struggle academically in interactive learning environments and that there is no one reason for their difficulty in adapting and succeeding in a new model. Due to the immediate need to improve student outcomes in first year chemistry, the intervention was not designed as an experiment. Therefore, it is not possible to draw a causal link

■

CONCLUSION

This paper outlines a partially flipped learning model used in three large first-year introductory chemistry courses, and investigates student perceptions of learning in such a model and the impact on student performance and withdrawal rates associated with this model. Face-to-face classes were transformed from didactic lectures into partially interactive learning environments that embedded a mixture of short lectures with a range of activities to actively guide student learning. The online learning space provided further activities to support student content understanding prior to attending class as well as being useful in consolidating their knowledge afterward. The model provided flexibility for students to access learning material at their own pace and in a variety of formats to suit their personal learning needs, which is greatly appreciated by students. The initial implementation of this model in 2013 demonstrated improved student performance and withdrawal rates for CHEM1001. With this success, in 2015 it was extended to the other first year chemistry courses, CHEM1101 and CHEM1901. Overall, our results show a shift in the student grade distributions, with significant increases in higher grades and decreased failure rates compared to the traditional course format. A causal link to the partially flipped model, however, cannot be concluded given the way in which the model was implemented. Nevertheless, we feel that the significant improved academic performance of students and their positive evaluation of the course after implementation of the model points to its value and usefulness in improving student learning and class engagement. I

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(10) Ryan, M. D.; Reid, S. A. Impact of the flipped classroom on student performance and retention: a parallel controlled study in general chemistry. J. Chem. Educ. 2016, 93 (1), 13−23. (11) Bates, S.; Galloway, R. The Inverted Classroom in a Large Enrolment Introductory Physics Course: A Case Study. In Proceedings of the HEA STEM Learning and Teaching Conference; London, England, April 12−13, 2012. (12) Bergmann, J.; Sams, A. Flip Your Classroom: Reach Every Student in Every Class Every Day; ISTE: Alexandria, VA, 2012. (13) Lage, M. J.; Platt, G. J.; Treglia, M. Inverting the classroom: A getaway to creating an inclusive learning environment. J. Econ. Educ. 2000, 31 (1), 30−43. (14) Piaget, J. The Origins of Intelligence in Children; International Universities Press: New York, NY, 1952. (15) Vygotsky, L. S. Mind in Society: The Development of Higher Psychological Processes; Harvard University Press: Cambridge, MA, 1978. (16) Sweller, J. Cognitive load during problem solving: Effects on learning. Cognitive Science 1988, 12 (2), 257−285. (17) Bodner, G. M. Theoretical Frameworks for Research in Chemistry/Science Education; Pearson/Prentice Hall: Upper Saddle, NJ, 2006. (18) Shattuck, J. S. A parallel controlled study of the effectiveness of a partially flipped organic chemistry course on student performance, perceptions, and course completion. J. Chem. Educ. 2016, 93 (12), 1984−1992. (19) Mooring, S. R.; Mitchell, C. E.; Burrows, N. L. Evaluation of a flipped, large-enrollment organic chemistry course on student attitude and achievement. J. Chem. Educ. 2016, 93 (12), 1972−1983. (20) Sweller, J. Cognitive load theory, learning difficulty, and instructional design. Learn. Instr. 1994, 4 (4), 295−312. (21) Kirschnere, P. A. Cognitive load theory: Implications of cognitive load theory on the design of learning. Learn. Instr. 2002, 12 (1), 1−10. (22) Seery, M. K.; Donnelly, R. The implementation of pre-lecture resources to reduce in-class cognitive load: A case study for higher education chemistry. Brit J. Educ. Technol. 2012, 43, 667−677. (23) Abeysekera, L.; Dawson, P. Motivation and Cognitive load in the Flipped classroom: Definition rationale, and a call for research. High. Educ. Res. Dev. 2015, 34 (1), 1−14. (24) Smith, J. D. Student attitudes toward flipping the general chemistry classroom. Chem. Educ. Res. Pract. 2013, 14 (4), 607−614. (25) Christiansen, M. A. Inverted teaching: applying a new pedagogy to a university organic chemistry class. J. Chem. Educ. 2014, 91 (11), 1845−1850. (26) Baepler, P.; Walker, J.; Driessen, M. It’s not about seat time: Blending, flipping and efficiency in active learning classrooms. Comput. Educ. 2014, 78, 227−236. (27) Yestrebsky, C. Flipping the classroom in large chemistry classresearch university environment. Procedia Soc. Behav. Sci. 2015, 191, 1113−1118. (28) Trogden, B. D. ConfChem Conference on Flipped Classroom Reclaiming Face Time − How an organic Chemistry Flipped Classroom Provided Access to Increased Guided Engagement. J. Chem. Educ. 2015, 92 (9), 1570−1571. (29) Bridgeman, A. J. Collaborative and active eLearning: contributing, ranking and tagging web resources in first year chemistry. In Proceedings of the Australian Conference on Science and Mathematics Education; Melbourne, Australia, 2011; pp 54−61. (30) Bridgeman, A. J. The University of Sydney − First Year Chemistry Home Page. https://scilearn.sydney.edu.au/fychemistry/ iChem/ (accessed February 15, 2019). (31) Bridgeman, A. J. The University of Sydney − iChem Home Page. https://scilearn.sydney.edu.au/fychemistry/iChem/vibrations. cfm?molecule=h2o&type=bend (accessed February 15, 2019). (32) Bridgeman, A. J. YouTube Channel Home Page. https://www. youtube.com/watch?v=vWy3rThoDiU (accessed February 15, 2019).

Student engagement with the online learning resources using learning analytics and interviews is currently being investigated. In future work, why higher achieving students appear to benefit more from a partially flipped learning model will be examined.

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.8b00414. Summary of documents (PDF, DOCX) Sample of the discussion platform features with an example of student-instructor led discussions (PDF) Sample of a lecture worksheet showing examples of theory based and interactive lecture demonstration questions (PDF) Sample e-mail generated from the early warning system (PDF) Sample student reflective comments (PDF) Sample course evaluation questionnaire (PDF) Sample university entry requirements (PDF)

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. ORCID

Rena Bokosmaty: 0000-0002-5406-8524 Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS The authors would like to acknowledge the first year undergraduate students enrolled in the three chemistry courses for their participation in this research.

■

REFERENCES

(1) Woldeamanuel, M. M.; Atagana, H.; Engida, T. What makes chemistry difficult? Afr. J. Chem. Educ. 2014, 4 (2), 31−43. (2) Lund, T. J.; Stains, M. The importance of context: an exploration of factors influencing the adoption of student-centered teaching among chemistry, biology and physics faculty. Int. J. STEM. Educ. 2015, 2 (1), 1−21. (3) Bodner, G. M. Why changing the curriculum may not be enough. J. Chem. Educ. 1992, 69 (3), 186−190. (4) Danker, B. Using flipped classroom approach to explore deep learning in large classrooms. IAFOR Journal of Education 2015, 3 (1), 171−186. (5) Ferreri, S. P.; O’Connor, S. K. Instructional design and assessment: Redesign of a large lecture course into a small-group learning course. Am. J. Pharm. Educ. 2013, 77 (1), 1−9. (6) O’Flaherty, J.; Phillips, C. The use of flipped classrooms in higher education: a scoping review. Internet High. Educ. 2015, 25, 85− 95. (7) Flynn, A. B. Structure and evaluation of flipped chemistry courses: Organic and Spectroscopy, large and small, first to third year, English and French. Chem. Educ. Res. Pract. 2015, 16 (2), 198−211. (8) Seery, M. K. Flipped learning in higher education chemistry: emerging trends and potential directions. Chem. Educ. Res. Pract. 2015, 16 (4), 758−768. (9) Weaver, G. B.; Sturtevant, H. G. Design, Implementation, and Evaluation of a flipped format general chemistry course. J. Chem. Educ. 2015, 92 (9), 1437−1448. J

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(33) Flipped Learning Network. The four pillars of F-L-I-P, 2014. Flipped learning network Web site. http://www.flippedlearning.org (accessed February 15, 2019). (34) Bridgeman, A. J. Language Support Strategies in First Year Science Students. In Sustainable Language Support Practices in Science Education: Technologies and Solutions; Zhang, F., Lidbury, B. A., Richardson, A. M., Yates, B. F., Gardiner, M. G., Bridgeman, A. J., Schulte, J., Rodger, J. C., Mate, K. E., Eds.; IGI: Singapore, 2012; pp 18−57. (35) Moog, R. S.; Spencer, J. N. POGIL: Process Oriented Guided Inquiry Learning; Oxford University Press: New York, NY, 2008. (36) Williamson, N. M.; Huang, D. M.; Bell, S. G.; Metha, G. F. Guided Inquiry learning in an introductory chemistry course. Int. J. Innov. Sci. Math. 2015, 23 (6), 34−51. (37) Walker, L.; Warfa, A. M. Process oriented guided inquiry learning (POGIL®) marginally effects student achievement measures but substantially increases the odds of passing a course. PLoS One 2017, 12 (10), 1−17. (38) Bedgood, D. R., Jr.; Bridgeman, A.; Buntine, M.; Mocerino, M.; Southam, D.; Lim, K. F.; Gardiner, M.; Yates, B.; Morris, G.; Pyke, S. M.; Zadnik, M. The development of teaching skills to support active learning in university Science (ALIUS). J. Learn. Des. 2015, 3 (3), 10−19. (39) Bridgeman, A. J.; George, A. V. Personalised learning, support and feedback in a large first year chemistry class, Darwin, Australia, 2014. http://fyhe.com.au/past_papers/papers14/07A.pdf. (accessed February 15, 2019). (40) Bunce, D. M.; Flens, E. A.; Neiles, K. Y. How long can students pay attention in class? A study of student attention decline using clickers. J. Chem. Educ. 2010, 87 (12), 1438−1443. (41) Bligh, D. A. What’s the Use of Lectures?; Jossey-Bass: San Francisco, CA, 2000. (42) Badiola, K. A.; Bartimote-Aufflick, K.; Bridgeman, A. J.; George, A. V.; Hudson, T. S.; Neto, C.; Schmid, S. A. Using Interactive Lecture Demonstrations to Invigorate Chemistry Lectures. In Proceedings of the Australian Conference on Science and Mathematics Education; Sydney, Australia, 2013; pp 97−107. (43) Prince, M. Does active learning work? A review of the research. J. Eng. Educ. 2004, 93 (3), 223−231. (44) McNally, B.; Chipperfield, J.; Dorsett, P.; Del Fabbro, L.; Frommolt, V. Flipped classroom experiences: student preferences and flip strategy in a higher education context. Int. J. Higher Educ. Res. 2017, 73, 281−298. (45) Cormier, C.; Voisard, B. Flipped classroom in organic chemistry has significant effect on students’ grades. Front. ICT. 2018, 4 (30), 1−15. (46) Keough, S. Clickers in the classroom: a review and a replication. J. Manage. Educ. 2012, 36 (6), 822−847. (47) MacArthur, J.; Jones, L. A review of literature reports of clickers applicable to college chemistry classrooms. Chem. Educ. Res. Pract. 2008, 9 (3), 187−195. (48) Freeman, S.; Eddy, S. L.; McDonough, M.; Smith, M. K.; Okoroafor, N.; Jordt, H.; Wenderoth, M. P. Active learning increases student performance in science, engineering, and mathematics. Proc. Natl. Acad. Sci. U. S. A. 2011, 111 (23), 8410−8415. (49) Lynch, D.; Trujillo, H. Motivational beliefs and learning strategies in organic chemistry. Int. J. Sci. Math. Educ. 2011, 9, 1351− 1365. (50) Pintrich, P.; Schunk, D. Motivation in Education; Theory, Research, and Applications, 2nd ed.; Prentice-Hall, Inc: Englewood Cliffs, NJ, 2002; pp 410−444. (51) Zusho, A.; Pintrich, P.; Coppola, B. Skill and will: The role of motivation and cognition in the learning of college chemistry. Int. J. Sci. Educ. 2003, 25 (9), 1081−1094. (52) Eccles, J.; Wigfield, A.; Schiefele, U. Motivation to succeed. In Handbook of Child Psychology: Social, Emotional and Personality Development; Damon, W., Eisenberg, N., Eds.; John Wiley & Sons Inc: Hoboken, NJ, 1998; pp 1017−1095.

(53) Silverthorn, D. A. Teaching and learning in the interactive classroom. Adv. Physiol. Educ. 2008, 30 (4), 135−140.

K

DOI: 10.1021/acs.jchemed.8b00414 J. Chem. Educ. XXXX, XXX, XXX−XXX