Affecting Student Engagement in an Online Course through Virtual

Chapter 4

Affecting Student Engagement in an Online Course through Virtual Laboratory Exercises Downloaded by UNIV OF FLORIDA on October 29, 2017 | http://pubs.acs.org Publication Date (Web): October 26, 2017 | doi: 10.1021/bk-2017-1261.ch004

Erland P. Stevens* Department of Chemistry, Davidson College, Box 7120, Davidson, North Carolina 28035, United States *E-mail: [email protected].

Comments within a discussion board for a massive open online course on the topic of medicinal chemistry have been analyzed across four separate course iterations. Data indicate that virtual laboratory group exercises, introduced after the first offering of the course, boosted student engagement based upon increases in the volume of discussion board comments. As more virtual laboratory exercises were incorporated into the class materials, the exercises accounted for 13, 38, and 46% of all student discussion board posts in the second, third, and fourth runs of the course, respectively. Increases in engagement were mostly limited to students who passed the course rather than the broader population of enrolled students.

Introduction The number of massive open online courses (MOOCs) being offered has risen dramatically in recent years with many online platforms hosting classes from universities and even non-traditional institutions, such as Amnesty International and The Smithsonian Institution (1). Although MOOCs represent a potential means of making education more accessible, MOOCs face several challenges. A common issue in MOOCs is low student engagement, which manifests itself in low MOOC completion and passing rates. A notable study of MOOCs was generated by Harvard and MIT in 2014 (2). The report followed the outcomes of students from 17 different MOOCs, and the overall passing rate was 5.1%. Advocates of MOOCs point out that many students who enroll in MOOCs only

© 2017 American Chemical Society Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Downloaded by UNIV OF FLORIDA on October 29, 2017 | http://pubs.acs.org Publication Date (Web): October 26, 2017 | doi: 10.1021/bk-2017-1261.ch004

want to sample the content and therefore never have any intention of passing the online course (2). Most MOOC instructors strive to increase student engagement as measured by passing rate or other data, such as mouse click events, video lectures views, or discussion board posts. Methods that can increase engagement in MOOCs resemble techniques used in residential classrooms but with the challenges posed by an online environment. Examples of specific methods include group projects, peer grading, incorporating quizzes into lectures, allowing students to work at their own pace, and class discussions (3). Over the past three years we have been teaching a MOOC on medicinal chemistry (4, 5). The MOOC has been taught four times with each subsequent offering being modified from earlier versions. Some of the changes were intended to drive more student engagement. Between each run of the course, different patterns in student behaviors on the discussion board have been noted. We have started to study patterns in student posts to the discussion board to determine the impact of our attempts to increase student engagement.

Results and Discussion The medicinal chemistry MOOC was offered on the edX platform with launch dates of March 2014, October 2014, October 2015, and March 2016. Enrollments ranged between roughly 5,000 and 15,000 students (Table 1). Reporting meaningful enrollments in MOOCs is challenging as many students join the course too late to have a realistic chance to pass the course. Because this study focuses upon student outcomes, registered students were included in Table 1 only if they were registered by a cutoff date. The cutoff date was determined by the latest date by which any student could enroll in the course and still earn a passing grade.

Table 1. Medicinal Chemistry Mooc Enrollments By Course Instance registered students

discussion posts

student posters (%)

posts per posting student

March 2014

13,004

3,879

1,082 (8.3)

3.46

October 2014

6,930

1,250

309 (4.5)

4.05

October 2015

5,695

1,265

360 (6.3)

3.52

March 2016

4,989

1,432

358 (7.2)

4.00

course run

The data in Table 1 reveal several engagement trends. First, the course run with the highest posting rate in the discussion board was the March 2014 offering at 8.3%. This figure is above the typical forum engagement in MOOCs at 35% (6) and very close to the 7% rate observed in another chemistry MOOC (7). 48 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


The posting activity in the October 2014 edition fell below 5%. The drop-off in student activity between the courses was starkly clear to the course staff during the October 2014 run. Lower total enrollment and rate of student participation yielded a discussion board that was relatively quiet and even dormant. The dip in activity can be attributed to three factors, two of which may be related to course design. First, the rate at which students made hello-type posts was high in the March 2014 course. A full 22.7% (879 posts) of all the posts for the March 2014 run were of this type, while the hello posts amounted to just 11.9% (149 posts) of the October 2014 posts. The higher level of welcome posts in March 2014 both increased the number of total posts and the percentage of active student posters (Table 1). The high engagement for March 2014 could reflect enthusiasm surrounding early MOOCs. The novelty was short-lived as, by fall 2014, Wired’s blog ran a post entitled in part, “MOOCs Are Dead” (8). The second reason for decreased posting activity in the second run is improvement in the course itself. Technical issues, unclear content, and even outright errors were prevalent in the March 2014 course materials. These issues drove students to the discussion board to seek help and clarification. The problems were fixed for the October 2014 run, and the volume of discussion posts dropped. The third suspected reason for falling discussion engagement is the addition of “common questions” sections throughout the October 2014 version of the course. The common questions sections likely addressed potential student questions so that students did not need to seek additional help. The October 2014 version of the course had one pleasant surprise in the discussion board. In the seventh week of the course, a new activity was added. The assignment was for the students to modify the structure of atenolol, a known G-protein-coupled receptor antagonist used to manage hypertension, into a new molecule with likely protease inhibitor activity. To accomplish this task, students were directed to use freely available tools on the website of Molinspiration (9). The Molinspiration tools allow a user to draw molecules and then predict the biological activity of the molecules against a range of common drug targets, including proteases. Biological activity is given a numerical score ranging approximately from –5.0 (low) to +2.5 (high). For reference, approved drugs typically have Molinspiration scores in the +0.75 to +1.00 range. In the exercise, students modified a molecule’s structure and posted their new structure with its predicted activity to the discussion board. Students could start with the structure of atenolol or use a molecule that had already been posted by another student. The purpose of the exercise was for students to experience the connection of a molecule’s structure with its biological activity. Understanding such structure-activity relationships is the foundation of the field of medicinal chemistry. Student response to the exercise was positive. By the second day of the exercise, students had built off the ideas of one another and morphed atenolol (1), the recommended starting point, into several completely new structures (Figure 1). The most potent structure (2) had an impressive activity score of +2.13. Because of the class’ rapid progress, additional criteria were added to the exercise to keep the students engaged. One extra criterion was molecular weight, which is a molecular property that affects a drug’s oral absorption. Another factor was selectivity, as 49 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


students were encouraged to design molecules with high potency against proteases but weak activity against other drug targets, such as ion channels and nuclear receptors. Students continued working with the Molinspiration website and by day 7 came up with structure 3 as a promising molecule with a reasonable activity of +0.80 and good selectivity relative to other targets. For the final stage of the project, two additional websites were added, admetSAR (10) and PROTOX (11). These sites allowed students to check for potential drug-drug interactions and toxicity. By day 10, students had designed structure 4, which satisfied all the different design criteria.

Figure 1. Protease inhibitors designed from atenolol (1) by students in the October 2014 MOOC iteration.

Although student posts were not required, the exercise drew students to the discussion board to post molecules and share ideas. Board traffic spiked noticeably during the exercise in week 7 of the class (Figure 2). The number of posts attributed to the class project was 161 and represents 84% of all the week 7 posts and 13% of the posts in the entire course. This impact was surprising because discussion board activity in MOOCs tapers off as students are lost through attrition. An exception is at the close of the course when many students post to say thank you and goodbye to their classmates (Figure 2, farewell). The columns in Figure 2 reflect the structure of the October 2014 MOOC. The first week includes two sections, both welcome and the week 1 instructional materials. The eighth and final week also includes two sections, the week 8 instructional content and the farewell material. Comments in the first week can be distinguished as originating from either the welcome or week 1 instructional content. The same is true for comments in the eighth week. 50 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


Figure 2. Discussion board post counts by course section in the October 2014 MOOC iteration.

Based on the effect of the class project on discussion board activity and student engagement, the third iteration of the course was designed to include weekly class projects, entitled “virtual labs”. Seven labs were created (Table 2). The labs required students to design molecules with qualities that may be predicted through different tools available on the web. Two of the labs were validation exercises in which students used online tools to predict selected activity values for known drugs and then compared those predicted values to experimental values from respected databases. All labs were built upon the content presented during the corresponding week of class. Early virtual labs were relatively simple, and later labs combined more complex ideas as the students’ mastery of medical chemistry increased. The medicinal chemistry MOOC launched again in October 2015, and despite a lower enrollment by over 1,000 students, the number of posts and students making posts were higher than in the October 2014 edition (Table 1, above). These increases are likely attributable to the newly introduced virtual labs. In total, 477 posts, or 37.7% of the 1,265 posts in the course, originated from the virtual labs. The number of posts for each virtual lab is shown in Figure 3. Weeks 3 and 5 showed the lowest posting activity. In these two weeks, the students simply transferred values from the resources to populate a table of activity data. These weeks did not encourage peer-to-peer interaction. Despite some weeks with low activity in the discussion board, the virtual lab exercises appeared to be successful in increasing engagement based on student posting behavior. 51 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


Table 2. Virtual Lab Exercises for the October 2015 MOOC Offering wk

description

web tools

1

Design compounds with maximized activity against a drug target

2

Design compounds with selective activity against a drug target

Molinspiration

3

Validate predicted activities by comparing to experimental values

Molinspiration Guide to Pharmacology (12)

4

Design compounds with selective activity and bioavailability

Molinspiration admetSAR (10)

5

Validate predicted metabolic activity by comparing to experimental values

admetSAR DrugBank (13)

6

Redesign a drug to lessen off-target side effects while retaining potency

Molinspiration admetSAR

7

Redesign a peptide inhibitor to retain potency and elevate bioavailability

Molinspiration admetSAR

Molinspiration (9)

Figure 3. Discussion board posts on the virtual labs by week (wk) in the October 2015 MOOC iteration.

Compared to the third MOOC offering in October 2015, the latest iteration in March 2016 had relatively minor changes. The virtual labs for weeks 3 and 5 were replaced with exercises that involved more peer-to-peer interaction through posting molecules to the discussion board. As a result, in the March 2016 run of the course, the number of total discussion board posts continued to rise beyond the 52 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


previous two versions despite the total enrollment dipping to below 5,000 students (Table 1, above). The total number of posts arising from the virtual labs was 658 (45.9%) of all the discussion posts. The week-by-week virtual lab posting totals are shown in Figure 4. The decrease in student posting activity closely mirrors the loss of students to attrition.

Figure 4. Discussion board posts on the virtual labs by week (wk) in the March 2016 MOOC iteration.

The virtual labs have dramatically increased student engagement based on trends in discussion board activity, but the information in Table 1 (above) raises some issues. First, despite increased student engagement based on the number of posts, the number of students posting to the discussion board, while rising over the last three course runs, still represents considerably less than 10% of the registered students. Second, the number of posts per student on the discussion board is essentially constant regardless of changes to the course. With respect to student outcomes, the passing rate for students is also nearly constant at approximately 5% for each course run (Table 3). This figure is similar to reported passing rates in the Harvard/MIT data report (2). While the passing rate is steady, the course data do show that students who post on the discussion board have a much higher passing rate than students who do not post. Others have noted a connection between student posting behavior and class performance in a MOOC. In a very recent study of a small MOOC on the topic of programming in C#, 68% of students who made a discussion post completed the course while only 11% of non-posters finished the course (14). Completing and passing a course are not identical, but the data from the programming and medicinal chemistry courses indicate the same idea—students who post to a discussion board in a MOOC enjoy a greater level of success. 53 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Table 3. Passing Rates of Different Student Groups Across Four Iterations of the Medicinal Chemistry MOOC


course run

all registered students (%)

students who posted (%)

students who did not post (%)

March 2014

6.3

35.5

3.7

October 2014

5.3

46.8

3.3

October 2015

5.2

31.1

3.5

March 2016

5.2

45.8

4.2

Figure 5 shows the passing rates of different groups of students relative to advancement through the course material. As with Figure 2 (above), posting activity is based on course section. The first and last weeks contain extra sections, start here and farewell, respectively. The white bars reveal the performance of all the students. The overall passing rate for all registered students is 5.2% (white bar, overall). Not all students who register for a MOOC view course content. The passing rate for students who both registered and viewed the welcome materials (white bar, welcome) is almost double the overall passing rate. Students who advanced to the first week of instructional content (white bar, wk 1) had an 11.4% chance of eventually passing the class. The gray bars show the passing rate for students who posted during a specific week of content. For example, if a student made a post to the discussion board during week 1 (grey bar, wk 1), then that student had a 47% chance of ultimately passing the course. The black bars show the passing rate for students who posted specifically about the virtual labs during a given week. For example, students who made a post on the wk 2 virtual lab had a passing rate of just over 70% (black bar, wk 2). The passing rates for the virtual labs posters are higher but similar to the broader discussion board posters (i.e., black bars vs. gray bars). Figure 5 reveals that early posting activity, even posting in the first week of the course, seems to be predictive of more favorable student outcomes. For example, students who reach the welcome section of the course have a passing rate of 9.2% (white bar, wk 1), but for students who make a post in the welcome section, the passing rate jumps almost four-fold to 35% (gray bar, wk 1). These early posts in the welcome section are generally superficial, hello-type posts or questions about course deadlines. Almost none of these posts cover course content. Regardless, students engaging in this simple posting behavior have a much higher chance of passing the course. By the third week of the course, almost all posts to the discussion board are from students who will continue with the course and earn a passing grade, but the typical student reaching week 3 has less than a 40% chance of passing. While the act of posting to the discussion board is highly unlikely to cause a student to pass a MOOC, posting activity is a behavior found in many students who do excel in the course. As the medicinal chemistry MOOC moves toward a fifth iteration in spring 2017, three changes will be implemented to further encourage students to engage with their peers through the discussion board. 54 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


Figure 5. Passing rates of different student groups by course section in the March 2016 MOOC (white = all registered students, gray = students who posted to the discussion board; black = students who posted to the discussion board about a virtual lab at any point in the course).

First, course data showing connections between posting activity and passing the course will be shared with students at the start of the course. Sharing this information may increase engagement, student outcomes, and awareness of the discussion board as a resource. A second change for the next iteration is a reduction in some of the repetitive assignments in the course. MOOCs are often compared to textbooks, and, like textbooks, MOOCs tend to have material added to them and can easily become bloated. The medicinal chemistry MOOC has gained new material over its previous editions. As less essential material is culled from the course for the fifth edition, the schedules of students, particularly the less prepared students, may be opened to allow more exploration of activities like the virtual labs and student interaction on the discussion board. The final change to the next iteration will be a revision of the syllabus to put more grade weight upon activities that encourage peer-to-peer interaction, namely the virtual labs. Currently, the labs are bundled in the class participation grade, and the threshold for being able to check the “yes” box to receive credit is low. Students are merely asked to read the posts of other students in order to receive participation credit. This bar will be raised with stronger encouragement for each student to share their own molecules and results with the other students through a discussion board post. Collectively, the proposed changes will almost certainly lead to more posting activity on the discussion board. While the quality of the discussion board posts may be slightly diluted, some loss in quality is acceptable if the increased engagement results in better outcomes for the students and a rise in the passing rate for the course as a whole. 55 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Acknowledgments We thank the members of the DavidsonX team, the Novartis Institutes for BioMedical Research for financial support, and Dr. Daniel Seaton of the Massachusetts Institute of Technology for extracting MOOC student activity information from the edX data logs.

References


1. 2.

3. 4.

5.

6.

7.

8.

9. 10. 11. 12. 13. 14.

edX. Schools and Partners; https://www.edx.org/schools-partners (accessed March 23, 2017). HarvardX and MITx: The First Year of Open Online Courses; https:// papers.ssrn.com/sol3/papers.cfm?abstract_id=2381263 (accessed March 23, 2017). Wyatt, L. G. Nontraditional Student Engagement: Increasing Adult Student Success and Retention. J. Contin. Higher Educ. 2011, 59, 10–20. edX. Medicinal Chemistry: The Molecular Basis of Drug Discovery; https://www.edx.org/course/medicinal-chemistry-molecular-basis-drugdavidsonx-d001x-1 (accessed March 23, 2017). Stevens, E. P. Creation of a Medicinal Chemistry MOOC as a Teaching Tool for Both Online and Residential Students. In Online Course Development and the Effect on the On-Campus Classroom; ACS Symposium Series 1217; American Chemical Society: Washington, DC, 2016; pp 75−88. Baek, J.; Shore, J. Promoting student engagement in MOOCs. In Proceedings of the Third ACM Conference on Learning @ Scale; April 25-26, 2016, Edinburgh, Scotland. Comer, D. K.; Clark, C. R.; Canelas, D. A. Writing to learn and learning to write across the disciplines: Peer-to-peer writing in introductory-level MOOCs. Int. Rev. Res. Open Distr. Learn. [Online] 2014, 15; http://www.irrodl.org/index.php/irrodl/article/view/1850/3066 (accessed March 23, 2017). Borden, J. MOOCs are dead–Long live the MOOC. https://www.wired.com/ insights/2014/08/moocs-are-dead-long-live-the-mooc/ (accessed March 23, 2017). Molinspiration Cheminformatics. Cheminformatics on the Web; http://www.molinspiration.com/ (accessed March 23, 2017). admetsSAR @ LMMD. http://lmmd.ecust.edu.cn:8000/ (accessed March 23, 2017). PROTOX – Prediction of Rat Oral TOXicity; http://tox.charite.de/tox/ (accessed March 23, 2017). Home IUPHAR/BPS Guide to Pharmacology. http://www.guideto pharmacology.org/ (accessed March 23, 2017). DrugBank; http://www.drugbank.ca/ (accessed March 23, 2017). Tseng, S.-F.; Tsao, Y.-W.; Yu, L.-C.; Chan, C.-L.; Lai, K. R. Who will pass? Analyzing learner behaviors in MOOCs. Res. Pract. Technol. Enhanc. Learn. 2016, 8, 1–11. 56 Sörensen and Canelas; Online Approaches to Chemical Education ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Affecting Student Engagement in an Online Course through Virtual

Recommend Documents