Mixed-Methods Study of Online and Written Organic Chemistry

Article pubs.acs.org/jchemeduc

Mixed-Methods Study of Online and Written Organic Chemistry Homework Kinza Malik, Nylvia Martinez, Juan Romero, Skyler Schubel, and Philip A. Janowicz* Department of Chemistry and Biochemistry, California State University, Fullerton, California 92831, United States ABSTRACT: Connect for organic chemistry is an online learning tool that gives students the opportunity to learn about all aspects of organic chemistry through the ease of the digital world. This research project consisted of two fundamental questions. The first was to discover whether there was a difference in undergraduate organic chemistry content knowledge between students at a public institution who complete homework assignments online and those who complete homework assignments on paper. The second was to determine what kind of effect online homework had on student learning. Courses were designated at random to perform either online homework or traditional homework. At the end of the semester, students were asked to fill out a survey regarding their thoughts on the homework method used for their course. Students’ exam grades were then analyzed. Students who utilized the Connect homework program did significantly better than those who completed traditional homework (p < 0.006, n = 112). The ACS average exam percentile score for sections using online homework was higher, 59.5, than those who used paper homework, 45.9. From the surveys, students mentioned that feedback and ease of use were the primary advantages to the online homework system. KEYWORDS: Chemical Education Research, Second-Year Undergraduate, Organic Chemistry, Internet/Web-Based Learning, Constructivism, Student-Centered Learning FEATURE: Chemical Education Research

■

objective of constructivism is “understanding and reconstruction”.10 Construction of knowledge occurs when an attempt is incorrect and new information is immediately given to alleviate cognitive dissonance. Based on the theoretical framework of constructivism, interactions or feedback an individual receives from their environment are crucial in the advancement of the individual’s knowledge. Feedback, in general, is the process in which information about the present or past impacts the student’s understanding of the concept in the present or future. Providing consistent, instant, and detailed feedback to students has been a major challenge in Web-based homework, given the complexity of the homework assignments and concepts being taught, and the security and reliability requirements that the program must provide.11 Web-based homework with instant feedback has had a great influence on student achievement.12−14 Not only does instant feedback influence student learning, but research indicates that it also improves student attitudes toward a course and encourages them to spend more time doing homework.12 Web-based homework programs with instant feedback can also offer an advantage in terms of the timing of feedback to students. This is a great advantage when compared to written homework where a large time gap between when the assignment is completed and when it is returned exists, causing the student to lose interest in reviewing the material.

INTRODUCTION Numerous studies have been conducted on student learning and the challenges often faced by students. When devising the appropriate learning environment for students, certain considerations must be taken into place such as age, maturity level, interests, and learning styles.1 There are several principles of teaching presently being used in the classroom such as experiential learning theory, which emphasizes balancing experience and conceptual learning contexts to achieve the greatest adaptive flexibility in learning.2 Interteaching, an alternative to behavioral teaching, offers flexibility for methods based on behavioral principles.3 This is a principle that is widely used in the classroom and permits students to construct information on feedback they receive from their peers for effective student learning.3 The introduction of “blended learning” fuses face-to-face course structure with an online course structure.4 These different forms of instruction are all examples of effective methods of teaching, and these methods are deemed successful because students are constructing knowledge based on feedback. The use of feedback is grounded in the theory of constructivism. Constructivism is a theory of learning that is based on how individuals incorporate new knowledge into existing knowledge.5−8 There are various forms of constructivism, and although they differ in concepts and ideas, they do share some commonalties, such as how individuals, groups, communities, and even a culture build up knowledge.9 The © XXXX American Chemical Society and Division of Chemical Education, Inc.

A

dx.doi.org/10.1021/ed400798t | J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 1. Examples of problems given to students in online and written homework sections.

public, primarily undergraduate, institution who complete homework assignments online and those who complete homework assignments on paper? (2) How does the medium of homework delivery affect student learning attitudes in organic chemistry?

Incorporating instant feedback in chemistry-related courses has been achieved in various ways in order to promote a more positive environment for student learning. For instance, clickers have been used in high-school chemistry courses.15 The use of clickers gave students the opportunity to participate and be engaged in the material, and students using clickers displayed great improvements in their performance in conceptual assessments.15 Also, clickers provided instant feedback and a means to assess student knowledge without penalty. Feedback has been taken to the next level with the use of online Webbased homework programs for general and organic chemistry courses taught at the college level, such as OWL, WileyPlus, Mastering Chemistry, Sapling Learning, and ACEOrganic.16−20 Feedback is given about how the problem should be worked, regardless of whether the student had answered the question right or wrong. The opportunity for multiple attempts in order to get the correct answer was also another form of feedback received. Previous literature has shown that instant feedback within online homework, in chemistry, has had a positive impact on both course grades and student attitudes.21,22 For this experiment, Connect, a Web-based organic chemistry homework program, was utilized, and its influence on student learning, whether positive or negative, was compared to written homework. The main difference between the two media of homework was that the online program provided instant feedback and grading at any time or location, while the written homework needed to be graded by hand and then returned. Although some previous literature has shown that Web-based homework does not have any effectual advantage over paperbased homework, the experimental designs were significantly different from this study. Some variables were not able to be controlled such as whether the homework was graded for correctness or completion,23 whether the instructor was the same between sections,24 the amount of homework between the sections,24 the style of homework questions between the sections,25 and the actual content of the homework questions between the sections.25 Each of these previous studies gives valuable insight into how students learn and interact with online homework, and the methods in this study aim to eliminate as many potential variables as possible from the experimental design so that specific items related to online homework and its effects on student learning can be elucidated. This research project consisted of two fundamental research questions: (1) Is there a difference in undergraduate organic chemistry content knowledge between students at a large

■

METHODS The study was completed at California State University Fullerton, a large public primarily undergraduate institution in Orange County, California. The University is a Hispanic Serving Institution, an Asian and Pacific Islander Serving Institution, and the largest destination for community college transfers in the state of California. The two sections contained 52.8% men and 47.2% women. The two sections contained 73% Biology majors, 18% Biochemistry majors, 5% Chemistry majors, and 4% other. The two sections contained 61% senior status (students having completed 90+ units), 35% junior status (students having completed 60−89 units), 0% sophomore status (students having completed 30−59 units), 0% freshman status (students having completed 0−29 units), and 4% nondegree students. In the two sections, 35% of the students had taken either organic 1 or organic 2 more than once. Students registered for one of two sections of organic chemistry 2, both of which were taught by the same instructor. The two sections were taught back-to-back in the same room with the same teaching methods and content, including the same exams. In the fall of 2011, the online homework and written sections contained 26 and 36 students, respectively, and in the spring of 2012, the online homework and written homework sections contained 20 and 26 students, respectively. Both sections had access to the same course Web site with the same materials and course announcements, office hours, Supplemental Instruction (SI) sessions, and instructor e-mail. Both sections received the same homework assignments containing the same questions in the same layout. The average number of questions per assignment was 17, some consisting of multiple parts. The written homework section received screenshots of the online homework; Figure 1 displays a screenshot of one of the homework assignments. Students were given the same amount of time to complete each assignment, which was due at the beginning of the next class period and contained problems related to what was discussed in the previous class. The homework assignments, for both sections, were given twice a week, when the students met B



Article

questions covering all topics discussed in the full-year organic chemistry curriculum. This metric was chosen because it is an externally validated and reliable measurement of student content knowledge in organic chemistry.27 Anonymous surveys were distributed at the end of the semester to collect qualitative data. This research study was examined by our institutional review board (IRB), and it was approved and exempt for the research described herein.

for class. The only difference was that the written homework had to be handed in person to the instructor, while the online homework had to be submitted online before the start of class. The written homework was graded exactly the same way as the online homework, and the average scores between the two groups were not different (online group = 73.5%, written group = 73.4%). Hence, the performance on the homework was not a potential variable affecting our data and concluding results. Although there were many similarities, there were differences between the two sections: (1) class meeting time (the two sections were 90 min apart); (2) the students’ prior chemistry knowledge; and (3) the medium of homework delivery. To control the effect of the different class times between the two classes, the online homework section was designated the earlier class in the fall of 2011, and the written homework section was designated the later class. In the spring of 2012, these designations were switched. In order to gauge students’ prior knowledge entering the course, students’ previous chemistry grades were calculated as well as grades in all previous math and science courses (Table 1). The general chemistry grades and grades in organic

■

RESULTS The ACS Organic Chemistry full-year examination was graded on a 0 to 100 percentile range, for both the written and online sections. Figure 2 illustrates the results of the ACS exam in both

Table 1. Average Chemistry and Math GPAs Calculated from Both Experimental Sections Chem GPA Math/Sci GPA

Online (n = 46)

Written (n = 62)

2.35 ± 0.065 2.47 ± 0.062

2.35 ± 0.059 2.41 ± 0.063

Figure 2. ACS exam percentile scores comparison between sections with online homework versus those with written homework.

chemistry 1 were significant to the study since several concepts learned in organic chemistry 2 are based on the fundamentals obtained from general chemistry and especially organic chemistry 1.26 The average chemistry GPA for students in the online sections was identical to the GPA for students in the written sections. Statistical analysis of students’ GPA was calculated using a one-way ANOVA to determine whether there was a significant difference between average math and science GPA for students in both sections. All the assumptions for running this ANOVA were met; the samples, which were the GPAs for each student, were independent of one another; the distribution of these samples was normal (homogeneity of variances = 0.931); and there was no skew on either end of the distribution for either group (skewness for online group = 0.310, kurtosis for online group = −0.628, skewness for written group = −0.265, kurtosis for written group = 0.081). The combination of data between semesters was permissible because the distributions of each semester were normal and not significantly different from one another. There was no selfselection, and the variables are independent of one another. The average math and science GPA for students in the online sections was not significantly different from the average math and science GPA for students in the written sections (n = 112, F = 0.094, p = 0.760). These data showed that the prior chemistry knowledge of students entering the course was not a significant factor in predicting their performance on the final exam. The only significant difference remaining between the two sections was the medium of homework delivery, and those results are depicted below. Content knowledge was measured by comparing student scores on the 2008 American Chemical Society (ACS) FullYear Organic Exam, which consists of 70 multiple-choice

sections. The averages were calculated for both sections; the online homework section had an average of 59.5 ± 3.51, and the average for the written homework section was 45.9 ± 3.72. A one-tailed t test, assuming equal variances, was run, and the p value of less than 0.006 shows that there was a significant difference in the ACS exam scores between the online homework and written homework sections (Cohen’s d effect size = 0.503). Once again, all the assumptions for running this t test were met; the samples, which were the ACS scores for each student, were independent of one another; the distribution of these samples was normal (homogeneity of variances = 0.553); and there was no skewness shown on either end of the distribution (skewness of online group = −0.232, kurtosis of online group = −0.197, skewness of written group = 0.409, kurtosis of written group = −0.602). There was no selfselection, and combination of the semesters was permissible. Surveys were collected from both students in both semesters. The surveys were arranged into 3 categories: positive, negative, and neutral. The positive category included students who favored their assigned medium of homework over the other. The negative category included students who favored the other medium of homework over their own. The neutral category included students who did not state an opinion, had no preference in homework medium delivery, or gave contradictory or irrelevant information. Figures 3 and 4 display the results of this collected surveyed data. In the written homework section, 41% of students expressed a preference for written homework, 26% of students preferred the online homework that they had; and 33% gave a neutral response. In the online homework section, 62% of the students preferred online homework, 0% preferred written homework, and 38% gave a neutral response. C



Article

mediums of homework, as measured by the ACS exam. Because the p value obtained was less than 0.006, there was a greater than 99.4% certainty that the results were not random, and, therefore, the null hypothesis was rejected. The p value was a strong indication that the results observed were acceptable, and online homework did have a greater impact on how the students tested on the ACS exam. These results of high test scores for the online homework section are very similar to the results in other experiments conducted before, such as in the research conducted on the online program, WE_LEARN, for organic chemistry. In this experiment as well, a positive change occurred in the classroom upon introduction of this system, and higher test scores and a boost in students’ confidence toward the material was accomplished.22 Knowledge prior to entering the course did not seem to have a great influence on which section would perform better on the ACS exam, because, as observed from Table 1, the average chemistry GPA, for both sections, was the same. The written section had an average GPA of 2.35 ± 0.065, while the online section had an average GPA of 2.35 ± 0.059. The difference in overall math and science GPA, for both sections, was not significantly different either. The online section had an average GPA of 2.47 ± 0.062, and the written section had an average math and science GPA of 2.41 ± 0.063. The qualitative data gave interesting results, as shown in Figures 3 and 4. Students from both sections were given a survey on the last day of class on which they were asked the following: “comment on the homework, mentioning its style of delivery, difficulty, and scope of questions.” The results obtained were remarkably fascinating. It was surprising that there was no negative feedback indicating that written homework was better than online homework. Also, approximately 62% of the students were satisfied with online homework, mentioning that online homework provided instant feedback, allowed multiple attempts, and saved trees and ink. One of the main advantages of instant feedback that a majority of the student comments mentioned was that a student could easily receive feedback at early hours in the morning compared to a student with written homework who did not have that option. For the “written homework” survey data, as observed in Figure 3, only 41% of the students were satisfied with written homework and preferred it over online homework. A few of the main reasons for these positive results were that many of the students were used to this medium of homework and many preferred drawing the structures out on paper. There was a lot of negative feedback received as well; 26% of the students stated that they preferred online homework over written homework. The major complaints received about written

Figure 3. Qualitative results pertaining to overall written homework satisfaction.

Figure 4. Qualitative results pertaining to overall online homework satisfaction.

Table 2 is a summation of the most frequent positive responses that students from the online homework section expressed of the medium in which homework was delivered to them. There were no negative comments regarding the online homework. Table 3 is a summation of the most frequent positive and negative comments that students from the written homework section expressed. The most frequent responses stated by students in the online homework sections were the instant feedback received when a problem was submitted, the opportunity of having multiple attempts, and the ease in navigating the Connect program. The most frequent positive response in the written homework section was the enjoyment of using the traditional pencil-and-paper form. The most frequent negative responses were the cost of printing and the absence of immediate feedback.

■

DISCUSSION Overall, it was observed that online homework was, in fact, a beneficial and enhanced tool for learning when compared to written organic homework. The ACS average exam score for sections utilizing online homework was considerably higher, 59.5, than that for those that used paper as the medium for homework, 45.9. The null hypothesis is that there is no difference between the two

Table 2. Qualitative Results of Student Views on the Online Homework Formata Most Common Response Easy to use (6) Instant feedback (5) Multiple attempts (3) a

Example “Good online homework assignments. Easy program to use. 1st time I felt like I could do the homework without stressing out on how to use the online program. Would recommend continuing online homework.” “The online homework was extremely helpful for me to figure out what I was doing wrong and to correct the mistake before the test.” “I really like the online homework. It gives up multiple chances to attempt the question and it gives me the opportunity to try different ways.”

The frequency of responses is shown in parentheses, and a representative response is shown on the right. D



Article

Table 3. Qualitative Results of Student Views on the Written Homework Formata Most Common Response Cost of printing (10) No immediate feedback (5) Like drawing on paper (4) a

Example “I think I have burned through 2 reams of paper and 2 ink cartridges from the homework. Everyone should get to do it online.” “It would have been more helpful to have online homework so I could have gotten online instant feedback to help me understand what I was doing wrong.” “I like the “print” homework because it is easier for me to do it on paper than on a computer.”

The frequency of responses is shown in parentheses, and a representative response is shown on the right. (2) Mainemelis, C.; Boyatzis, R.; Kolb, D. Learning styles and adaptive flexibility: Testing experiential learning theory. Manage. Learn. 2002, 33 (1), 5−33. (3) Saville, B.; Lambert, T.; Robertson, S. Interteaching: Bringing behavioral education into the 21st century. Psychol. Rec. 2011, 61 (1), 153−165. (4) Gedik, N.; Kiraz, E.; Ozden, M. Y. The optimum blend: Affordances and challenges of blended learning for students. Turkish Online J. Qual. Inquiry 2012, 3 (3), 102−117. (5) Nussbaum, J. (1989). Classroom conceptual change: Philosophical perspectives. The history and philosophy of science in science teaching; Herget, D. E., Ed.; Florida State University: Tallahassee, FL, 1989; pp 278−291. (6) Tobin, K. Social constructivist perspectives on the reform of science education. Aust. Sci. Teach. J. 1990, 36, 29−35. (7) Von Glasersfeld, E. Introduction: Aspects of constructivism. Constructivism: Theory, perspectives, and practice; Fosnot, C. T., Ed.; Teachers College Press: New York, 2005; pp 3−7. (8) Ausubel, D. P. Educational Psychology: A Cognitive View; Holt: New York, 1968. (9) Stavridou, H.; Solomonidou, C. Conceptual reorganization and the construction of the chemical reaction concept during secondary education. Int. J. Sci. Educ. 1998, 20, 205−221. (10) Lincoln, Y. S.; Guba, E. G. (2000). Paradigmatic controversies, contradictions, and emerging confluences. In Handbook of qualitative research; Denzine, N. K., Lincoln, Y. S., Eds.; Sage: Thousand Oaks, CA, 2000; pp 163−185. (11) Palocsay, S. W.; Stevens, S. P. A Study of the Effectiveness of Web-Based Homework in Teaching Undergraduate Business Statistics. Decis. Sci. J. Innovative Educ. 2008, 6, 213−232. (12) Allain, R.; Williams, T. The Effectiveness of Online Homework in an Introductory Science Class. J. Coll. Sci. Teach. 2006, 35. (13) Freasier, B.; Collins, G.; Newitt, P. A web-based interactive homework quiz and tutorial package to motivate undergraduate chemistry students and improve learning. J. Chem. Educ. 2003, 80 (11), 1344. (14) Cole, R.; Todd, J. Effects of web-based multimedia homework with immediate rich feedback on student learning in general chemistry. J. Chem. Educ. 2003, 80 (11), 1338. (15) Vital, F. Creating a Positive Learning Environment with the Use of Clickers in a High School Chemistry Classroom. J. Chem. Educ. 2012, 4, 470−473. (16) WileyPlus. https://www.wileyplus.com/WileyCDA/ (accessed May 2013). (17) MasteringChemistry. http://masteringchemistry.com/ (accessed May 2013). (18) ACEOrganic. http://www.pearsonhighered.com/aceorganic/ (accessed May 2013). (19) Evans, J. OWL (Online Web-Based Learning). J. Chem. Educ. 2009, 86 (6), 695. (20) Sapling Learning. http://www2.saplinglearning.com/ (accessed May 2013). (21) Loudon, G.; Parker, L. Case Study Using Online Homework in Undergraduate Organic Chemistry: Results and Student Attitudes. J. Chem. Educ. 2013, 90, 37−44. (22) Drelick, J.; Henry, Z.; Richards-Babb, M.; Robertson-Honecker, J. Online Homework, Help or Hindrance? What Students Think and How They Perform. J. Coll. Sci. Teach. 2011, 40 (4), 81−93.

homework were that there were too many pages to print out and that there was no immediate feedback. Students in each section were able to communicate with students in the other section, but they were not able to change their medium of homework delivery throughout the semester. It is possible that this communication between groups led to the larger negative responses in the written group and the lack of negative responses in the online group.

■

CONCLUSIONS AND FUTURE WORK This study does give evidence to the constructivist view that the interactions or feedback an individual receives from the environment plays a crucial role in the advancement of the individual’s knowledge. The results illustrate how the medium of homework can have a significant impact on student learning and satisfaction with the course. It is unclear whether the feedback itself caused the higher ACS exam scores, even though it was explicitly mentioned by the students, in the surveys. Therefore, further studies need to be conducted along the lines of work completed by Richards-Babb in order to figure out how the students exactly utilize the homework program and learn more about their experiences using the online homework.28 Time on task can also be one alternative hypothesis to the observed differences in student performance, but it is impossible to accurately control and measure the actual time taken to complete each assignment for both media of homework. Opportunities for engagement on material outside of class are paramount. Previous research shows that there is a strong correlation with a student’s time on tasks, such as homework, and their level of success in a specific course.29−34 The more time that students spend on a specific task, the more actively engaged they are in effectively learning the material. It is possible that the students spent more time on the online homework than the written homework, engaging in such activities as taking time to draw the structures online and submit the answers, unlike on the written homework, and it can be suggested that these students were more actively engaged in learning. A more thorough examination of how students use the online homework is needed in a future study.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Notes

The authors declare the following competing financial interest(s): Philip Janowicz is currently a digital author for McGraw-Hill Education.

■

REFERENCES

(1) Yilmaz-Soylu, M.; Akkoyunlu, B. The effect of learning styles on achievement in different learning environments. Turkish Online J. Educ. Technol. 2009, 8 (4), 43−50. E



Article

(23) Fyenwever, H. The Comparison of the Effectiveness of Webbased and Paper-based Homework for General Chemistry. Chem. Educ. 2008, 13, 264−269. (24) Bonham, S.; Beichner, R.; Deardorff, D. Online Homework: Does it Make a Difference? Phys. Teach. 2001, 39. (25) Nedeff, V.; Penn, J. Organic Chemistry and the Internet: A Web-Based Approach to Homework and Testing Using the WE_LEARN System. J. Chem. Educ. 2000, 77, 227−231. (26) Anaya, G. College impact on student learning: Comparing the use of self-reported gains, standardized test scores, and college grades. Res. Higher Educ. 1999, 40 (5), 499−526. (27) ACS Organic Chemistry Full-Year Examination (2008) Reliability and Validity Data. http://chemexams.chem.iastate.edu/ national-norms/or08.html (accessed May 2014). (28) Richards-Babb, M.; Jackson, J. K. Chem. Educ. Res. Pract. 2011, 12, 409−419. (29) Brock, S. E. Time on Task. In Encyclopedia of School Psychology; Lee, S., Ed.; Sage Publications: Thousand Oaks, CA, 2005; pp 567− 568. (30) Greenwood, C.; Horton, B.; Utley, C. Academic Engagement: Current Perspectives on Research and Practice. School Psychol. Rev. 2001, 31, 328−349. (31) VanDeWeghe, R. Research matters: What is engaged learning? Engl. J. 2006, 95 (3), 88−91. (32) Slavin, R. Educational Psychology: Theory and Practice; Pearson Education: Boston, 2003. (33) Marks, H. Student engagement in instructional activity: Patterns in elementary, middle and high school years. Am. Educ. Res. J. 2013, 37, 153−184. (34) Romero, M.; Barbera, E. Quality of E-Learners’ Time and Learning Performance Beyond Quantitative Time-on-Task. Int. Rev. Res. Open Distance Learn. 2011, 12.

F


Mixed-Methods Study of Online and Written Organic Chemistry

Recommend Documents