Online Course Development and the Effect on the ... - ACS Publications

Chapter 8

Downloaded by PURDUE UNIV on August 24, 2016 | http://pubs.acs.org Publication Date (Web): August 16, 2016 | doi: 10.1021/bk-2016-1217.ch008

A Dialogue between Online and On-Campus Versions of the Same Course: Lessons from Harvard’s Science and Cooking Course Pia M. Sörensen* and Michael P. Brenner Harvard John A. Paulson School of Engineering and Applied Sciences, Harvard University, 29 Oxford Street, Cambridge, Massachsetts 02138, United States *E-mail: [email protected]

We describe here the give-and-take that occurred between online and on-campus versions of Science and Cooking, a general education course at Harvard University that teaches chemistry and physics through food and cooking. We discuss how engaging in successive rounds of online and on-campus teaching resulted in a fruitful exchange between the two that ended up being beneficial to both course formats. We present data indicating what was more and less successful in this process, and we propose some lessons learned, which we hope can be of interest to other educators working along the interface of on-campus and online teaching.

1. Introduction When creating an online course, instructors often draw on their execution of established on-campus courses. Online courses, in turn, can inspire changes to on-campus teaching. As further iterations of a course are developed, this creates an ongoing process of give and take, in which online and on-campus courses can contribute to each other in interesting and instructive ways. This chapter discusses the ongoing give and take that occurred in the general education course Science and Cooking: From Haute Cuisine to Soft Matter Science, which is taught at the undergraduate level at Harvard University. We describe in detail the different iterations of the course development process, including practical examples, successes, challenges, and research © 2016 American Chemical Society Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


findings from both the on-campus and online courses. Our goal is to give an account of what we did, with the hope that this will provide resources and ideas to others working along the interface of online and on-campus learning. We begin this chapter with (2.1) a brief account of the content and structure of the original on-campus course. This provides background and context for the key decisions involved in the design of subsequent online and on-campus versions of the course. After this, the sections are arranged according to the give-and-take of four subsequent versions, two of which were offered online and two on-campus. In (2.2.1) we describe the first online class, followed by a discussion of (2.2.2) four noteworthy advantages of the online format. In (2.3.1) we describe three limitations of the on-campus course which we were hoping to overcome, followed by (2.3.2) how incorporating material from the online course into the on-campus course helped us overcome these. We conclude this discussion in (2.3.3) with empirical data, collected from student evaluations, showing which aspects of the on-campus course were deemed more or less helpful to their learning experience. In (2.4.1) we describe a second version of the on-campus course which was constructed to respond to the evaluations of the on-campus course of the previous year. Also, in (2.4.2) we present further empirical data from student evaluations of this second on-campus version. In (2.5) we describe how these changes, in turn, were integrated into the second iteration of the online class.

2. Results and Discussion 2.1. A Brief Account of the On-Campus Version of Science and Cooking Science and Cooking is a general education course at Harvard that explores how everyday cooking and haute cuisine can illuminate basic principles in chemistry and physics (1). The course fulfills the physical sciences requirement for non-science majors at Harvard College and has typically enrolled between 200-300 students every fall since its first offering in 2010. The course covers science topics ranging from phase transitions and diffusion, to viscosity, emulsions, and fermentation reactions (Figure 1a). Each of these topics is introduced with a lecture by a visiting chef who showcases signature recipes from his or her kitchen. The week covering the scientific topic of diffusion may, for example, feature a chef’s discussion of various recipes of spherification, a culinary technique invented by Ferran Adria and one of the hallmarks of molecular gastronomy. Spherification involves the diffusion of calcium ions into a liquid food containing small amounts of alginate. The calcium ions cross-link the alginate polymers, and the result is a very, very thin gel, thick enough to hold the liquid food in a sphere, but thin enough to break in your mouth. Inspired by this dish, the science of the week, as presented by the course instructors, explores polymers and gels, random walks, and diffusion of small molecules (Figure 1b). The same science concepts are also explored in the context of other foods. For example, the Latin American dish, ceviche, made by immersing raw fish in lemon juice, is used in a discussion of proton diffusion and acid-induced denaturation and coagulation of proteins. 90 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


Figure 1. (A) Structure of the semester showing science topics for weeks 1-12. (B) Example of a typical week as illustrated by lectures, lab, and “equation of the week” for the week covering the scientific topic of diffusion. The core scientific concept of each week is captured by the “equation of the week”. In the case of diffusion, the “equation of the week” is the diffusion equation, , which predicts the distance of diffusion, L, over a time, t, for a molecule with diffusion coefficient, D. Students also engage with the material in a weekly two-hour lab, where hands-on exercises illustrate key scientific concepts. The diffusion week lab prompts students to deduce their own physical constants — the diffusion coefficients for calcium ions and protons in water — by cooking recipes for spherification and ceviche. As a final component of the course, students are asked to apply the acquired practical and theoretical knowledge by working on a final project of their choosing. Students spend four weeks on this, either investigating a culinary topic in scientific terms, or designing a solution to a culinary problem and explaining in scientific terms why it works. The course ends with a class-wide science fair where students present their project findings. 2.2. First Offering of the Online Class 2.2.1. Development of the First Online Class The on-campus course was still relatively new when we decided to develop a parallel online version in Spring 2013. It had only been offered three times, each involving additional course design and optimization of content and structure. 91 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


Despite this optimization, certain challenges were difficult to address within the current course format. It was partly with the hope to address these challenges that we decided to embark on the online project. We will discuss the details of the challenges in section 2.3.1, while this section summarizes some of the key features of the online course, and emphasizes in more detail some of the more successful elements. The overall structure of the online class was designed to be as follows: The science concepts were delivered in five to ten minute video segments as is the common format on the Edx platform. These were grouped together with videos by chefs, creating an immediate integration of science and cooking concepts that can be difficult to achieve in the on-campus classroom, which required two separate lectures on different days of the week. Both science and chef videos were interspersed with short questions, which either prompted students to think about a topic before being fully exposed to it in a video, or tested and elaborated on concepts after the videos. The overall scope, pace, and difficulty level were in many ways similar to the on-campus course: the topics were comparable in content and order, the materials were released at the same pace, and weekly requirements included a homework and a lab exercise of comparable difficulty level. In order to accommodate the limitations of online grading, the assessments, which consisted of homeworks, labs, and a final project, were redesigned to include multiple choice, short numerical answers, or self-grading. As a way to provide the interaction between peers and faculty that is common in an on-campus course, students interacted with each other and the course staff on an online discussion forum. The science fair was also adapted to an online format by having students record and share videos of their final projects on the course wiki. The number of students attracted by this first online course indicates that the general public has a thirst for science outreach education of this kind. When the course went live in October 2013 about 65,000 people had registered for the course. This number increased to more than 90,000 people from 176 countries before the course closed six months later (2). The course demographics were in some cases similar, and in other cases different, to other HarvardX courses. Of the registrants, about 78% had at least a bachelors degree, which is just slightly higher than the average HarvardX class (70%), and 51% were female, compared to only 37% for the average HarvardX class (3, 4). Based on a survey conducted at the beginning of the course, which was completed by almost 27,000 registrants, 49% intended to finish the course, 30% planned to audit, and 17% were not sure (5). Similar to other MOOCs, though, attrition was high, in part due to the ease of registering, compared to the considerably higher time commitment required to finish. About 1,841 students, corresponding to 2.0%, completed Science and Cooking with a passing grade (6). “Passing” in this case was equivalent to successfully completing 60% of the course assignments, a not insubstantial feat given both the rigor of the course, and the required time commitment.

92 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

2.2.2. Four Advantages of the Online Format As part of the online development process we hoped to preserve many of the successful features of the on-campus class, while also utilizing the unique advantages of the online environment. We identified four specific ways that we were able to use the online environment to our advantage. We summarize these here, in the hopes that other teachers working along the on-campus and online interface can make use of them.


(i). The Home as a Unique Resource for Online Classes One of the difficulties in converting an on-campus class to an online format are the imposed restrictions that come with the lack of resources commonly found on traditional campuses. Depending on the course, this may include access to specific equipment, classrooms, libraries, etc, but also such non-material assets as in-person interaction, stemming from students and instructors being in the same place. In the case of Science and Cooking, one of these valuable resources was the access to a lab space. The on-campus class uses a “cooking lab” for the weekly labs. Over the first few years of the course, this space has gradually been outfitted to house a variety of instruments and ingredients, both culinary and scientific, making a wide range of experiments possible. How could we find an equivalent of this essential resource for chemistry in an online course? As it turns out, everyone has access to a lab in the home — the kitchen. This realization not only solved the lab space problem, it also accomplished an original goal of the Science and Cooking class: the mission to teach basic scientific principles through everyday phenomena. What better way to show how chemistry and physics is everywhere, than to help people conceive of their everyday living spaces as places where they can study and learn science? This makes science familiar and accessible, in stark contrast to the common perception that chemistry and physics are done mainly away from the home, in a lab. With this in mind we set out redesigning the lab exercises with the goal of making them feasible to do in a typical kitchen. We eliminated, or provided alternatives, to labs needing certain appliances or ingredients. For example, we provided instructions on how to build your own scale from a clothes hanger, and designed alternative labs for those requiring hard-to-come-by ingredients such as alginate, xanthan gum, and calcium chloride. Thus, for the diffusion lab, students could, in addition to spherification, chose to study either diffusion of lemon juice in ceviche, or diffusion of soy sauce or food coloring in egg white. We also eliminated the need for appliances that are less common in non-western style kitchens, such as ovens, in order to accommodate course participants from all over the world. For example, the molten chocolate cake lab, which illustrates the scientific concept of heat transfer, was modified to include options of cooking the cake on a stove top or hot plate instead of in an oven. For each lab exercise, students were offered the choice between experiments ranging from simple to complex, with the hope of providing accessible and 93 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


appropriately challenging options to students with varying levels of interest and background in science or cooking. For example, the viscosity lab featured a less involved experiment studying the viscosity of sugar water at increasing sugar concentrations, as well as a more involved experiment determining how the volume fractions of the various ingredients in mac&cheese contribute to the sauce’s viscosity. Though the content of Science and Cooking may lend itself especially well to the resources available in a typical home, making a place in the home a natural environment for study and experiment could be a goal, and a tool, of many different types on online classes. After all, online education distinguishes itself by reaching people in their homes in a way that traditional universities do not. This reveals some unique opportunities for developers of online science courses.

(ii). Massive Enrollment as a Potential Resource The massive aspect of Massive Open Online Courses, or MOOCs, can certainly have some disadvantages. But the large number of enrollees can also be an asset. In our case, the thousands of students performing experiments around the world provided an opportunity to illustrate several key concepts in the course, mainly by collecting a large amount of data. In the very first lab exercise of the course, students were asked to measure out a cup of flour, weigh it, and submit the weight on a google form. We then compiled the data and, referring to the large variation, used it in a discussion on scientific error and precision in cooking (Figure 2a). We also returned to this data several weeks later with a discussion on packing, which, in addition to being fundamental for understanding how densely flour packs, also governs the underlying sciene of emulsions and foams. The first lab also required students who owned an oven to calibrate it. This could be done with a thermometer, or by using the melting point of sugar to find the discrepancy between true and reported oven temperatures at sugars melting point of 186°C (7). Similar to the flour experiment, the temperature of ovens around the world varies widely and we used this for further discussion of precision and error (Figure 2b). Several subsequent lab exercises continued in the same spirit: students estimated values for the diffusion coefficients of calcium ions and protons in water by doing measurements on spherification and ceviche. They also found the heat diffusion constant in water by taking measurements on the crumb thickness in molten chocolate cake. By compiling the data, comparing it to values in the literature, and reporting back to the students, we were able effectively to show not only that physical constants reported in the literature can be deduced from simple measurements in the kitchen, but also that these very constants are fundamental in determining basic steps in recipes, such as how long to marinate ceviche or how long to cook a molten chocolate cake.



Other online courses may similarly benefit from using the large enrollment as a teaching tool, and the collective contribution to class data may also help create a sense of community among students in diverse online classes.

Figure 2. (A) Weight of one cup of flour as measured and reported by course participants. (B) Reported dial settings at oven temperatures of 186°C, i.e. when a small amount of sugar (mp=186°C) placed in the oven was observed to start melting. 95 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


(iii). The Advantage of an Expandable, Movable, and Zoomed in Classroom The common practice of delivering course content in video format in online courses has several advantages. These advantages may seem self-evident and are easily taken for granted, but emphasizing them can bring the online course format to its full potential. In the case of Science and Cooking, taking full advantage of the video format allowed us to improve on the course in two main ways. First, as remarkable as it is for students to view live presentations from visiting chefs, there are certain aspects to the chefs’ craft that get lost with this approach. In a large lecture hall with 200-300 students, most of whom are far from the podium, the extraordinary skill and detail with which the chefs demonstrate their creations can be easily overlooked. The online video classroom, on the other hand, allows every detail of the demonstration to be zoomed in on and captured, and watched repeatedly at will – a very helpful learning resource. Second, many chefs think of their approach to food and cooking as being deeply rooted in their local environments. Several of the visiting Catalan chefs, for example, have created modernist variations of traditional Catalan dishes using ingredients sourced from local markets or gardens. In addition, chefs’ restaurants are impressive workshops, and certainly some of the chefs’ magic gets lost in the awkward setting at the front of a lecture hall. By being able to travel to the chefs’ restaurants and film segments of the lectures there, we were able to showcase both of these aspects of the chefs’ art — the local environment and the workshop itself. These field trips would be highly impractical even for a small class, and certainly impossible for a large class like ours. Thus, unlike the on-campus classroom, which is fixed in space, the video format allowed us to move and expand the online classroom to cover a much larger area in geographical space. These affordances could be utilized in a wide range of courses where the ability to zoom and having different filming places would substantially add to the course content.

(iv). A Culturally and Geographically Diverse Setting Most MOOCs have participants from all over the world, but this often has little bearing on the course itself. In Science and Cooking, however, students were able to make good use of this cultural diversity by applying scientific principles to foods and recipes from an impressive range of cooking traditions. This offered a unique cultural education all of its own, alongside the scientific education. We encouraged students to connect with each other in a way that highlights this diversity. For example, at the beginning of the course we asked students to introduce themselves to their classmates on the online forum with their name and favorite dish. The range of dishes was inspiring. There were dishes as familiar to Americans as pumpkin pie, right along side Burik, a potato dish submitted by an English teacher in Yemen. There were unusual entries such as water kefir. One entry that intrigued many people was bulgogi, a Korean meat dish submitted by a 10 year old boy taking the class with the help of his parents. The final projects also showed this cultural diversity, ranging from scientific studies of the resting time of Yorkshire pudding and heat transfer in Japanese 96 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.

Takoyaki. Thus the science fair at the end of the course was not only a forum of impressive science projects, but a cultural food fair in its own right. There are a myriad ways in which other online courses can utilize a highly diverse student body in similar ways. 2.3. First Implementation of Online Materials in the On-Campus Class


2.3.1. Challenges of the On-Campus Course Which Could Be Met by the Development of Online Materials Our original motivation for creating an online course was to use it as a way to address some of the challenges in the on-campus course. These challenges can be summarized in the following three points:

(i). Lack of Available Teaching Resources Science and Cooking is a new type of course: soft matter physics is typically not taught at all at an introductory college level, and though some of the chemistry and physics concepts are indeed taught in other introductory courses, they are rarely addressed through examples of food and cooking. Because of this there are limited resources for students to access material outside of lecture. For example, unlike typical introductory science courses, there was no textbook that adequately described, as a unity, the science concepts we wanted to cover. This is why we felt compelled to write a textbook to accompany the course (described in section 2.3.2 and 2.4.1).

(ii). Need for Further Development of Course Content At the time of the creation of the online course there was still much potential for further development of on-campus course content that would significantly strengthen the course. The primary goal of the course is to teach science through concepts in cooking, but a constant challenge is what topics to choose. What you do in the kitchen is strongly rooted in science, and cutting edge chefs constantly exploit basic scientific principles to create remarkable new foods, so there are innumerable possible concepts to choose from. The development of the online course gave us the opportunity to test new material, and alternative ways of presenting it, for example in different labs, homework assessments, and in-class demonstrations.

(iii). Limitations of On-Campus Course Structure In addition to course content there was also the challenge of structure. The format of the on-campus course, where one lecture focused primarily on science and the other primarily on cooking, resulted in a low ratio of science to cooking. 97 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


The chef demonstrations in the cooking lecture typically involved such a multitude of science concepts that they could not be satisfactorily explained in depth during the science lecture. Because of this, many opportunities to delve deeper into the science were lost. The limitations in course structure also became apparent by the fact that the course enrolls a wide range of students. Though the course is designed for nonscience majors, in reality the science backgrounds of students vary widely, making it challenging to calibrate the level of difficulty. Ultimately we hoped to make the same class challenging and interesting to this diverse set of students.

2.3.2. How We Responded to the Challenges by Implementing Material from the Online Course into the On-Campus Course The development of the online class gave us new resources to address the challenges in (2.3.1) in a number of ways. (i) First, through the online materials, the on-campus students now had access to what was essentially an interactive textbook that offered an alternative resource for accessing course materials outside of class. On-campus students could watch videos, or work through questions between videos, before coming to class or as review. The videos had the added advantage of providing course material as searchable text — the subtitles, provided by a semi-automated captioning service, could be made accessible as downloadable text files and also as searchable text on the side of each video. By developing the online class, we thus ended up creating a resource that could be channeled back to the original residential course in a way that would not otherwise have been possible. This helped address the first of the challenges above. (ii) By being forced to re-conceive of material for a different audience, and thus approaching it from a different angle, we generated numerous ideas for redeveloping the labs and homeworks with more intuitive experiments and problems. This helped address the second of the challenges above. For example, several of the labs that had been designed for the at-home kitchen seemed more suitable for illustrating the connection between everyday cooking and science than the original on-campus experiments. The mac & cheese lab developed for the online course was, for example, incorporated into the on-campus curriculum on viscosity. Also, the experiment of weighing a cup of flour to illustrate scientific error and packing, also developed for the online course, became one of the first in-class demonstrations of the on-campus course. (iii) Both content and structure, i.e. the second and third challenges, were primarily addressed by having access to the pre-recorded video materials. This allowed us substantially to rework how material was covered each week, summarized in Figure 3. By making the video material from the online class an assignment before lecture, we were able to use class time for a learning experience which we would not otherwise have had time 98 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


or preparation for. Rather than spending three hours a week lecturing, we now required students to watch the science videos ahead of lecture. This amounted to a total of two hours a week. We then used class time in two primary ways: to review material in a question-answer format using clickers, and as a way to delve into the science more deeply. Review of material was emphasized during the first 15-20 minutes of each lecture. For the chef lecture, this allowed students to have a better grasp of the science prior to hearing the chef presentation, thus enabling a more complete learning experience. For the science lecture, the fact that students already had some idea of the basic science concepts prior to coming to class, not only made it possible for instructors to cover more science, but also to design a more interactive lecture.

Figure 3. Weekly structure for the first implementation of online materials in the on-campus course. The science lecture was thus re-conceived of as a scientific “cooking show” which was run by the instructors. Each week, the “show” centered around one of the students’ favorite recipes, which were submitted at the beginning of the semester, and chosen each week by the instructors. The student who had submitted the recipe was invited to cook it with a small number of classmates at the front of the lecture hall, and the recipe was then carefully deconstructed to illustrate key scientific points. For example, the week covering the concept of elasticity featured the cooking and dissection of three different pancake recipes. The class cooked pancakes, took measurements, calculated the elasticity and cross-link distance using the equation of the week, and discussed the underlying science of the different elasticities such as the effect of gluten, starch, and denaturing egg proteins. As a way to make up for the time commitment of watching two hours of videos each week, the homework was designed to follow closely the key points in the cooking show, i.e. attending lecture was designed to be highly beneficial in completing the homework. Hence, once students had prepared for and attended lecture, only limited time was required to complete the remaining course requirements each week. Our new approach to class time had a profound effect on content in that it allowed instructors multiple opportunities to illustrate the link between science and 99 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


cooking in a way that would otherwise not have been possible. It also addressed the challenge of structure in fundamental ways: It allowed us to balance out the previously-mentioned imbalance in the ratio of science to cooking by making more lecture time available for additional science concepts. It also enabled instructors the option of assigning some of the science videos as homework outside of lecture. The new structure also helped adapt the course for students with diverse science backgrounds: Students with less science background could choose to watch more review materials, while students with stronger science background could be directed to the more advanced science videos and problems. Access to the videos also allowed us to share the local environment and the chefs’ workshops with on-campus students i.e. the expandable, movable, and zoomed in classroom from the online class could be transported into the fixed classroom for the benefit of the on-campus students as well.

2.3.3. Empirical Data on How the New Approaches Were Received by On-Campus Students How did this educational experiment turn out? It is hard to imagine that a student who completed all course requirements in this new version of the course did not learn at least as much, if not more and at a deeper level, than students in previous versions of the course. Because the exams and other assessments each year were different, our perspective is subjective, but we do believe that the students showed a deeper command of more topics in this version of the course. There are some interesting aspects of this work on the interface between online and on-campus versions of a course that can be evaluated. One such aspect is students’ perception of this classroom experiment, and this was studied extensively with surveys and focus groups. The following provides an overview of this data, including what we found was successful and not successful. Overall, course evaluations dropped considerably from 3.7 (an average over the previous three years) to 3.2 out of 5.0 (N=310) (8). Teasing apart the specifics, we found that the science lecture with the cooking show was overall well-received: 60% of students found that it added to their scientific understanding. However, 60% also found the science lectures to be disorganized, thus offering some insight into the less enthusiastic reception of the new teaching approach (9). This highlights one of the key challenges in integrating online material in the classroom: doing so effectively takes some trial-and-error to mold online materials to a specific on-campus constituency. It is especially challenging in high enrollment classes where various types of exercises are much harder to implement than in smaller classes, and where the format of the lecture hall is not usually suited to team work or easy roaming of the course staff. According to our data, students were also skeptical to the use of clickers as a way to review material and enforce watching of online videos before coming to lecture (10). As much as 45% did not think that clickers reinforced the science concepts (9). The online videos, on the other hand, were appreciated, with 60% of students finding them “interesting and engaging”. Despite this, the task of video watching was not necessarily taken so seriously: 30% reported multi-tasking while 100 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


watching and 45% skipped or fast-forwarded (9). This behavior is not necessarily problematic. In fact, by holding students accountable to video watching by way of the video data log, it is not unexpected that students would pay less attention to videos containing material they already felt they had grasped in lecture but still wanted to get credit for watching. The labs were highly popular: 87% of students thought they added to their understanding of scientific concepts, and 93% found them engaging (9, 10). Overall though, we think it would be fair to say that reception of the new format was lukewarm. When asked if they would enroll in a course of this format again 40% of students said yes, 40% said no, and the remaining 20% indicated that they did not know (9). A few factors other than the new course format may have influenced the outcome of these surveys. For example, enrollment, was in the upper limit of what is typically the case, and this may have added to students’ sense of disorganization. Our model also required a couple hours more work per week than in previous years. Perhaps most importantly, the format was completely different from that of traditional courses and from what students have learned to expect. It is possible that as courses with this format become more common, there will be less resistance in student populations. 2.4. Second Implementation of Online Materials in the On-Campus Class 2.4.1. How We Improved the Implementation of Online Materials in the On-Campus Class Based on Student Feedback For the next offering of the on-campus course we made considerable changes based on the outcome of the first implementation described in 2.3.1-3. Specifically, we wanted to change the aspects of the course that received a negative or ‘lukewarm’ reception in course evaluations. Most importantly, instead of repeating the flipped classroom format, we returned to a primarily lecture-style class with a weekly structure similar to that of the original course (Figure 4). However, we also kept some of the key successful features from the previous year. One of the most appreciated aspects of the first implementation of the oncampus course was the additional resources that the online materials provided. Our main effort for the second implementation thus went into developing these appreciated materials further. From the work done for the online class we had graphics and short animations of the course material. Most importantly we also had transcripts from the videos which could be heavily edited to provide the basis of a textbook for the course. As mentioned in section 2.3.1(i), we felt this would fill an important gap in materials available to support the interface of learning science through cooking. By incorporating the graphics, short video animations, and practice examples with the text we were able to produce an interactive textbook in a mixed media format (11). As a result, the production of the online class significantly lowered the barrier of producing this needed resource on our own. With this textbook in hand, we abandoned the idea of requiring students to watch videos before coming to class, as well as the use of clickers. These had 101 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


been the least successful components of the first on-campus class. Instead we focused on making the course material available to students in three main ways: in lecture with accompanying lecture slides, as supplementary video materials, and in textbook format. We began each lecture by pointing out the exact videos and textbook chapters that corresponded to the material we were covering in class, thus clearly indicating the available resources.

Figure 4. Weekly structure for the second implementation of online materials in the on-campus course. With this new format we were still able to incorporate more science in the class by occasionally assigning video watching before lecture or as part of homework assignments. We also continued to use review or advanced videos as a way to address the diverse science backgrounds among students. In addition, much of the content, such as labs and in-class demonstration, that had successfully been transferred from the online class were also kept. Thus this second implementation was in many ways a middle-ground between the original on-campus class containing no online materials at all, and the completely re-worked class from the year before.

2.4.2 Empirical Data on How the Changes Were Received by On-Campus Students Compared to the previous year, the course jumped from a ‘lukewarm’ reception to an enthusiastic reception (8, 12, 13). The overall assessment of the course increased from 3.2 to 4.0 out of 5.0, the best in its five-year history (N=152) (8). Of the survey respondents, 98% rated the class as either “good”, “very good”, or “excellent”, comparing to only 73% the previous year. Of the three ways students could access course material, the textbook turned out to be the most popular with 60% saying they used it frequently, compared to only 17% for the supplementary videos (Figure 5a). The textbook was also perceived as the most useful resource: 93% found it useful, compared to 87% for science lectures and 83% for the supplementary science videos (Figure 5b). The perceived organization, one of the main complaints the previous year, improved significantly from 23% to 74%. Similarly, the integration of science with cooking, one of the ongoing challenges in the course, increased from 52% to 90% (Figure 5c). 102 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


Figure 5. (A) Self-reported frequency of use of supplementary materials. (B) Perceived usefulness of the different course components in facilitating understanding of course concepts. (C) Perceived structure, integration and understanding of science concepts as reported by students in the first and second implementations of the on-campus class. Figures adapted with permission from reference (12).



Finally, we wanted to find out if there was any desire among students to bring back the interactive lecture format from the previous year. Again though, students were divided, with the majority preferring to keep things as they were: 31% said yes to a more interactive format, 44% said no, and the remaining 24% did not care (12). Other than the changes in course format, there are some other possible reasons for the more positive reception of the class. For one, the class was smaller the second time around (180 students instead of 320), and this alone made the class more manageable. The evaluations from the previous year may also have lead to a certain amount of self-selection, i.e. students with a genuine interest in the class decided to take it despite the low evaluations, and, not surprisingly, found that they liked it.

2.5. Second Offering of the Online Class The second version of the online class was offered from June to October, 2015 (14). For this version, the science and chef videos were edited to better integrate science and cooking concepts. In addition, a small number of visiting chefs were added, and the format for presenting and testing the “equation of the week” was structured and emphasized. Finally, the interactive textbook was available for purchase on Amazon and the iBook Store, making the course material available in multiple formats to online students as well. We offered the textbook at a low price (USD 9.99) in order to make it available to a wide range of income levels. By the time the course launched in June 2015, 22,000 people had registered for the course. Within a week and a half, this number had increased to 60,000, and by the time the course closed in October there were 68,700 registrants from 195 countries (Figure 6a) (2). Of the registrants ~500 participants finished the course with a passing grade, where “passing” corresponded to successful completion of 60% of the course assignments, the same criteria as in the first version of the online class (6). Who were the people who signed up and how did they interact with the course? Most registrants were based in the US (41%), comparable to the 2013 class (38%). The country with the second most registrants was Canada in both years (5% in 2015 and 3.3% in 2013) (2). Also, similar to the first course were some of the other demographics: a comparable number of registrants, 74%, had a bachelor’s degree or higher (78% in 2013) and 55% were female (51% in 2013) (3, 4). At most there were 28,000 people active in the course in some way in the same week; this occurred in the first week (Figure 6b). Of these, 10,500 watched a video, and 5,200 tried a problem. By the end of the course these numbers decreased significantly. On average between 6-26% of the problems on the homeworks were answered incorrectly. The number was much lower for labs, 4-15%, partly because lab assessments were primarily self-assessment questions or had a very large error margin in order to accommodate a broad range of experimental data (15).



Figure 6. (A) Comparison of student enrollment in two online offerings of Science and Cooking. Data extracted from HarvardX World Maps (2). (B) Weekly student engagement in the course as illustrated by the number of students attempting to solve a problem, watching a video, and engaging in any activity, i.e. any student who logged into the course. Data extracted from edX Insights (15).

3. Summary Through our experience adapting online materials for the on-campus classroom, we learned that simply engaging in multiple rounds of online and on-campus teaching, resulted in a dialogue between the two that ended up being beneficial to both course formats. This is true even in the case of things we tried which were not well-received on course evaluations, such as with some aspects of blended learning. In our case, one of the most beneficial results in this dialogue was the development of a textbook from the online materials. This ended up being a much-appreciated resource to students in both courses, and it implemented teaching tools of both formats. Other courses adapting on-campus material for an online audience might also find that materials created for online courses can successfully be re-purposed into similarly innovative learning materials that provide a great service to students in both on-campus and online formats. 105 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


Acknowledgments We thank Melissa Barnett, Nancy Kahlil, and the Program in General Education at Harvard for assistance with student focus groups. We also thank Jenny Bergeron and the Derek Bok Center for Teaching and Learning at Harvard for assistance with course evaluations. We are grateful to David Weitz and Stephanie Kenen for helpful discussions, the teams at HarvardX and EdX, the visiting Chefs and Teaching Fellows who have helped with the development and execution of both online and on-campus courses. We gratefully acknowledge funding from the Program of General Education at Harvard, the Harvard John A. Paulson School of Engineering and Applied Sciences, the Kavli Foundation, and NSF grants from the Harvard Materials Research and Science and Engineering Center DMR1420570 and DMS-1411694.

References 1.

Rowat, A. C.; Sinha, N.N.; Sörensen, P.M.; Campas, O.; Castells, P.; Rosenberg, D.; Brenner, M.P.; Weitz, D.A. The Kitchen as a Physics Classroom. Physics Education 2014, 49, 512–522. 2. Nesterko, S. O., Seaton, D. T., Kashin, K., Han, Q., Reich, J., Waldo, J., Chuang I., Ho, A. D. World Map of Enrollment, HarvardX Insights. http://harvardx.harvard.edu/harvardx-insights/world-map-enrollment (accessed October 8, 2013; March 30, 2014; June 17, 2015; October 15, 2015). 3. Nesterko, S. O., Seaton, D. T., Kashin, K., Han, Q., Reich, J., Waldo, J., Chuang I., Ho, A. D. World Map of Education Composition, HarvardX Insights. http://harvardx.harvard.edu/harvardx-insights/world-mapeducation-composition (accessed October 8, 2013; June 17, 2015). 4. Nesterko, S. O., Seaton, D. T., Kashin, K., Han, Q., Reich, J., Waldo, J., Chuang I., Ho, A. D. World Map of Gender Composition, HarvardX insights. http://harvardx.harvard.edu/world-map-gender-composition (accessed October 8, 2013; June 17, 2015). 5. Reich, J. MOOC Completion and Retention in the Context of Student Intent, Educause. http://er.educause.edu/articles/2014/12/mooc-completion-andretention-in-the-context-of-student-intent (accessed January 23, 2016). 6. Nesterko, S. O., Seaton, D. T., Kashin, K., Han, Q., Reich, J., Waldo, J., Chuang I., Ho, A. D. World Map of Certificate Attainment, HarvardX Insights. http://harvardx.harvard.edu/harvardx-insights/world-mapcertificate-attainment (accessed March 30, 2014; October 15, 2015). 7. Potter, J., Cooking for Geeks: Real Science, Great Hacks, and Good Food; O-Reilly Media, Inc.: Sebastopol, CA, 2010. 8. Course Evaluations, Harvard Q Guide. http://www.fas.harvard.edu/~evals/ index.htm (accessed September 15, 2015). 9. Derek Bok Center for Teaching and Learning, Harvard University, Course Report AY2013 Science of Physical Universe 27: Science and Cooking. 10. SPU27 Student Focus Group Summary Report. Program in General Education, Harvard University, 2013. 106 Sörensen; Online Course Development and the Effect on the On-Campus Classroom ACS Symposium Series; American Chemical Society: Washington, DC, 2016.


11. Brenner, M. P., Sörensen, P. M., Weitz, D. A. Science and Cooking: A Companion to the Harvard Course. Interactive e-book, self-published, 2015. 12. Derek Bok Center for Teaching and Learning, Harvard University, Course Report AY2014 Science of Physical Universe 27: Science and Cooking. 13. SPU27 Student Focus Group Summary Report. Program in General Education, Harvard University: 2014. 14. Science and Cooking: From Haute Cuisine to Soft Matter Science, edX. https://www.edx.org/course/science-cooking-haute-cuisine-soft-harvardxspu27x-0 (accessed September 15, 2015). 15. Sörensen, P. M. Science and Cooking: From Haute Cuisine to Soft Matter Science, edX Insights. https://insights.edx.org/courses/HarvardX/SPU27x/ 2013_Oct/ (accessed November 15, 2015)


Online Course Development and the Effect on the ... - ACS Publications

Recommend Documents