Form versus Function: A Comparison of Lewis Structure Drawing

Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

pubs.acs.org/jchemeduc

Form versus Function: A Comparison of Lewis Structure Drawing Tools and the Extraneous Cognitive Load They Induce Patrick L. Duffy,† Kory M. Enneking,‡ Tyler W. Gampp,§ Khatijah Amir Hakim,† Amelia F. Coleman,† Krista V. Laforest,‡ Dylan M. Paulson,‡ Erik T. Paulson,‡ Justin D. Shepard,‡ Jessica M. Tiettmeyer,† Kristina M. Mazzarone,∥ and Nathaniel P. Grove*,†

J. Chem. Educ. Downloaded from pubs.acs.org by YORK UNIV on 12/08/18. For personal use only.

†

Department of Chemistry and Biochemistry, University of North Carolina Wilmington, Wilmington, North Carolina 28403, United States ‡ Department of Biology and Marine Biology, University of North Carolina Wilmington, Wilmington, North Carolina 28403, United States § Eugene Ashley High School, 555 Halyburton Memorial Parkway, Wilmington, North Carolina 28412, United States ∥ Science Department, Cape Fear Community College, Wilmington, North Carolina 28401, United States ABSTRACT: In recent decades, technology has become a constant presence in the chemistry classroom with online homework extensively used in many programs. Although research has pointed to both increased course performance and better study habits in classes that utilize online homework, little work has explored the extraneous load that these platforms place on the learner’s working memory. Cognitive load theory suggests that optimal learning occurs when sufficient capacity is available in working memory to process new information, and as such, it is important that extraneous load be minimized. This research study compared the extraneous load placed on working memory as students constructed Lewis structures under three conditions: using pen-and-paper, using beSocratic, or using a traditional, button-driven online homework system. The results showed that the traditional online homework system induced a statistically significantly larger extraneous load than the other two drawing formats. KEYWORDS: First-Year Undergraduate/General, Chemical Education Research, Lewis Structures, Second-Year Undergraduate, Computer-Based Learning FEATURE: Chemical Education Research

■

INTRODUCTION From computer-based laboratory instruction,1−13 to animations and visualizations,14−20 to the use of personal-response systems,21−24 technology has quickly become a ubiquitous presence in the classroom that has forever changed how chemistry is taught and experienced by students of all ages. In an age of ever-increasing class sizes, many faculty have turned to online homework as a replacement for the pencil-and-paper equivalent. Not only is the process of assigning and grading homework automated, but many systems also provide detailed feedback and guided tutorials when students need assistance. Indeed, some systems are even adaptive and change to meet the students’ needs.25 Recent research has suggested that the use of online homework systems has significantly improved student performance.25−28 This is often attributed to the immediate feedback that many online homework systems offer. In terms of student perception, students have generally expressed positive attitudes toward their use.26,27,29,30 A study by Richards-Babb et al., for example, reported that students saw the assignments as valuable and that they led to more consistent study habits.27 In the past, the design of online homework systems typically focused on student and instructor feedback; current software programs are being developed to push past this traditional approach to an adaptive-response feedback design.25 Recent research suggests that this adaptive approach may be beneficial © XXXX American Chemical Society and Division of Chemical Education, Inc.

for students as those who utilized such a system scored higher on their final examination than those who used a more traditional feedback system.25

■

ONLINE HOMEWORK AND THE CREATION OF LEWIS STRUCTURES In addition to the submission of multiple choice and alphanumeric strings, many online homework systems also provide students with the ability to construct representations like Lewis structures. Although there are differences among systems, most are remarkably consistent: students create their representations by selecting options from a series of menus or on-screen buttons; some systems require students to place atoms, bonds, and electron pairs in specific positions defined by gridlines, while others offer a more free-form palette in which to construct representations.31 Regardless, most systems do not employ naturalistic interfaces, and students often face steep learning curves necessary to master their use.31 Unfortunately, this may contribute a sizable amount of extraneous cognitive load to the already complex construction process. Indeed, when an incorrect structure is created, it is Received: July 18, 2018 Revised: November 23, 2018

A

DOI: 10.1021/acs.jchemed.8b00574 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

begin to use the system to complete assigned tasks. A memorable system is one that is easy for users to remember, even after an extended break from using it, and efficiency speaks to the level of effort that must be expended to complete the task. Systems should be designed to minimize errors related to incorrect and unwanted user actions, and just as importantly, when those errors do occur, to provide a mechanism by which the user can quickly rectify the problem. Finally, the system should be enjoyable to use, which contributes to the overall satisfaction.42 In recent years, researchers have used cognitive load theory to inform more effective approaches toward HCI.38,40,43 Harrison, Flood, and Duce, for example, introduced the PACMAD (people at the centre of mobile application development) model that weaves cognitive load into the more traditional usability guidelines proposed by Nielsen.43 Suboptimal onscreen information presentation can increase extraneous load as it hinders the ability to differentiate relevant and irrelevant information; in turn, this may lead to increased error rates and lower user satisfaction. Interfaces with confusing or ambiguous icons or menu options, poor color selection, or poor spatial layout can negatively impact learnability and efficiency and deplete users’ already-limited cognitive reserves that would be better devoted toward the actual task.38,44−46 Such poor design choices encourage a manual and, therefore, less efficient processing approach by the user.45 At the same time, interfaces can be intelligently designed not only to minimize extraneous cognitive load but also to encourage more positive forms of cognitive load associated with schema creation (germane cognitive load).40 Unfortunately, a recent literature review found that only about a quarter of researchers systematically studied their interface’s impact on cognitive load, despite the importance that it has on application usability,43 and in the case of education technology, how effective it is at supporting student learning. This is an area ripe for additional study, with implications for the development of educational technology not only in chemistry, but beyond. As such, we sought to answer the following research question with the current study: What effect does the format (pen-and-paper, beSocratic, or a traditional, button-driven homework system) that general and organic chemistry students use to draw Lewis structures have on the extraneous cognitive load of the activity?

often difficult to distinguish whether the problem stems from a conceptual misunderstanding of how to construct Lewis structures or merely an inability to appropriately navigate the interface.31 Exceptions to the typical menu-driven navigational approach to the construction of Lewis structures include OrganicPad31 and its spiritual successor, beSocratic.32 In the case of both programs, students draw Lewis structures using either a stylus or their finger (if a touchscreen is available) or with a mouse (in the absence of a touchscreen). Handwriting recognition is used to convert the students’ hand-drawn letters and symbols into formatted text which can be subsequently analyzed by the system. In this case, the user interface has been specifically designed to mimic the pen-and-paper approach students are familiar with, and therefore, it is not unreasonable to hypothesize that the use of such an interface may induce less extraneous cognitive load on the learner’s working memory than the more menu-heavy approaches employed by other commonly used systems.

■

COGNITIVE LOAD THEORY AND HUMAN−COMPUTER INTERACTION Extraneous load is just one of three types of cognitive load identified in the research literature, with the others being germane and intrinsic, and their sum represents the total cognitive load associated with a particular exercise.33−37 Extraneous load is most commonly generated when students are tasked with engaging with poorly designed instructional materials. For example, extraneous load is produced when a textbook graphic and its explanation are placed on separate pages, a situation that requires the reader to maintain additional information in working memory as s/he flips back and forth between the pages. Cognitive load theory suggests that optimal learning occurs when sufficient capacity is available in working memory to process new information.37 In instances where insufficient capacity exists, the learner may become cognitively overloaded and subsequent learning will be hampered. As such, it is essential that instructional experiences be designed in such a way as to minimize extraneous cognitive load.37 Given the prevalent use of technology in the classroom, and in particular, the increasing use of online homework systems, understanding the impact that the use of such systems has on students’ cognitive load is essential. For a system to be a viable replacement for traditional “pen-and-paper” assignments, it must support the learning process by disappearing seamlessly into the background. Since at least the 1970s, researchers have been interested in studying this challenge, and from this interest, the field of human−computer interaction (HCI) was born.38 HCI is concerned with “the design, evaluation and implementation of interactive computing systems for human use and with the study of major phenomena surrounding them”,39 and one of its central tenets is that of usability, the extent to which a learner can complete a task effectively, efficiently, and with satisfaction.40,41 Any number of usability guidelines have been proposed, but some of the more commonly cited are those developed by Nielsen.42 This model outlines five broad goals that any interactive system should achieve with its design: (1) learnability, (2) memorability, (3) efficiency, (4) low error rates, and (5) high user satisfaction.42 Learnability relates to how quickly a user can learn to proficiently work a system. In other words, the greater the learnability, the quicker a user can

■

EXPERIMENTAL METHODS

Research Population

Data were gathered during the 2015−2016 academic year from two groups of students: those enrolled in a second-semester general chemistry course (N = 36) and those enrolled in a firstsemester organic chemistry course (N = 59). A replication study was conducted during the 2017−2018 academic year with a second group of first-semester organic chemistry students (N = 35). In all cases, students were asked to participate approximately halfway through the semester and were interviewed over the course of about a week in order to avoid any impact subsequent learning may have had on the results. The students received a small amount of extra credit for their participation, and all students that volunteered were interviewed as part of the research. The population of general chemistry students was mainly freshmen (approximately 60% freshmen, 25% sophomores, and 15% juniors and seniors), while the organic chemistry students B



Article

Figure 1. Picture of a participant drawing a Lewis structure for water using the beSocratic interface on an iPad.

Students also did not interact with any form of online homework as part of the course; however, it is important to note that the student volunteers had started to use ChemDraw to create reaction and mechanistic schemes for their organic chemistry laboratory reports, and many of the students had used the same online homework system in general chemistry. As such, the organic chemistry students had prior exposure to the Lewis structure drawing interface.

were primarily sophomores (65% sophomores and 35% juniors and seniors). The general chemistry students were mostly intending to pursue majors in biology, chemistry, clinical research, computer science, environmental science, engineering, exercise science, geology, marine biology, nursing, and public health, but there were also a small percentage of nonSTEM students taking the course to satisfy the university’s general education science requirements. Students enrolled in organic chemistry were almost exclusively biology, chemistry, and marine biology majors. In the case of all three groups, the volunteers closely mirrored the population of the entire course in terms of gender, ethnicity, intended major, year in school, and final performance in the course.

Data Collection

Our previous work led to the development of a valid and reliable technique for measuring cognitive load in students working chemistry problems by monitoring changes in heart rate.48 In instances where demand on working memory increases, the learner stresses, thus causing a corresponding increase in heart rate. This technique is adaptable to a wide range of populations and research settings and makes use of modern, wireless technology.48,49 Research participants were outfitted with a Scosche myTREK wireless heart rate monitor that was attached to the participant’s nonwriting arm using the included Velcro strap. Heart rate data were recorded using an Apple iPod touch that was running the Wahoo Fitness app. This particular app was selected because of its ability to export the collected heart rate data as a comma-separated values (csv) file that can be easily read by Excel or SPSS. Heart rate readings were transmitted wirelessly and continuously every second from the monitor to the app via Bluetooth. Respondents were asked not to drink caffeinated beverages such as coffee or energy drinks or to exercise 2−3 h before their appointment because of the artificially elevated heart rate that these activities can cause. In order to investigate the impact of the extraneous load associated with multiple Lewis structure construction formats, Sweller50 suggested that any experimental approach must be structured so that the intrinsic and germane loads remain constant. In such instances, then, any differences measured in

Research Environment

All sections of general chemistry at UNCW make use of an online homework system (which will not be identified in this report) in which students are typically required to complete assignments at the conclusion of each chapter. These assignments were created collaboratively by all instructors of the course and were designed to take approximately 2 h for the students to complete. The assignments were worth 10% of the students’ final course grade. Some instructors also used the system to assign quizzes for students to complete outside of class. The online homework system utilized a typical buttondriven interface that incorporated elements from ChemDraw, a popular structure drawing program heavily utilized by chemists.47 Molecular structure and, in particular, Lewis structures are topics typically covered during the last month of the firstsemester of general chemistry, and their construction is a component of a single online homework assignment. Students enrolled in organic chemistry 1 did not formally cover the construction of Lewis structures during class; it was material that the organic faculty assumed had been learned and mastered by students in their previous chemistry courses. C



Article

Figure 2. Average percent change in heart rate for each Lewis structure for general chemistry students in each of the three treatment groups.

with a traditional desktop and mouse to construct their Lewis structures. All students had access to a periodic table. In the case of all three groups, the start and end times in which students constructed their structures were recorded to assist with subsequent analysis. Before being instructed to move on to the next formula, a brief “cool down” period was employed so that the participant’s heart rate could return to resting levels. This was determined by monitoring the participant’s heart rate in real time. Informed consent was secured from all participants before data collection began, and all phases of this research were approved by UNCW’s IRB.

the total cognitive load can be attributed to the extraneous load induced by the system. Participants from both general chemistry 2 (N = 36) and organic chemistry 1 (N = 59 and N = 35) were asked to construct a series of 10 Lewis structures: H2O, NH3, N2F5+, P2S3, SI6, HSO4−, CBr4, SiF4, CO, and C2H3O2−. These formulas were used in our previous research studies49 and were selected because they present students with a variety of difficulty levels, and their structures require the integration of several different structural characteristics including multiple bonds and central atoms, expanded octets, and positive and negative charges. Given that all students were asked to construct the same 10 Lewis structures and in the same order (the order listed above), the intrinsic load of the activity was held constant. Further, the participants within each group had roughly the same prior knowledge related to Lewis structure construction based on the courses they had taken previously either in high school or college. Participants were randomly assigned to one of three treatment groups: (1) the pen-and-paper (PP) group, (2) the beSocratic group (bS), and (3) the online homework (OHW) group. As suggested by the names of the treatment groups, the difference among them was the manner in which students were asked to construct their Lewis structures. In the case of PP, students constructed their Lewis structures using a piece of paper and a pencil or pen. Each of the 10 formulas was printed on the top of a separate piece of paper and provided to students one at a time. The bS students utilized an iPad running beSocratic, using their fingers to draw their structures as depicted in Figure 1. Finally, the OHW group used the online homework system described previously in this report

■

RESULTS AND DISCUSSION Data were analyzed according to procedures described in detail in our previous work.48,49 In short, a maximum heart rate change for each Lewis structure was determined by subtracting the participant’s resting heart rate from the highest recorded heart rate observed during the time in which s/he was constructing the Lewis structure. These so-called peak loads were then converted into percentages by dividing the change by the participant’s resting heart rate and multiplying by 100%: % change =

(peak − resting) × 100% resting

The percent changes for each Lewis structure were averaged within each of the three treatment groups and are reported for the general chemistry students in Figure 2; those for the organic chemistry students are in Figure 3 (2015−2016 academic year) and Figure 4 (2017−2018 academic year). Note that the results for H2O were not included in the D



Article

Figure 3. Average percent change in heart rate for each Lewis structure for organic chemistry students (2015−2016 academic year) in each of the three treatment groups.

and their inferential reasoning abilities.53−56 More naturalistic interfaces like those utilized by beSocratic have been shown to promote increased expression of nonlinguistic (diagrams, symbols, and numbers) representations among learners compared to those that rely upon a keyboard and mouse.54−57 It is interesting to note how quickly students adapted to the use of the beSocratic system. None of the students, whether enrolled in general chemistry or organic chemistry, had ever used the program before participating in this research, and yet within the construction of a few Lewis structures, the total cognitive load induced by the construction process for those students appeared to be comparable to those who drew their structures using a piece of paper and a pencil. Indeed, there are many instances in all three groups where the overall load for the bS group was qualitatively lower than that for the PP group. This speaks well of the learnability of the beSocratic system.42 The results were explored for statistical significance using a series of Kruskal−Wallis analyses, the nonparametric equivalent of the one-way analysis of variance (ANOVA). Additional posthoc analyses (Mann−Whitney U-test) were performed in instances where significant differences were documented by the Kruskal−Wallace tests to determine which groups were significantly different from each within the results collected for each Lewis structure. The results of the Kruskal−Wallis analyses performed on the data collected from the general chemistry students revealed that only the results for NH3 were significantly different (p < 0.05), and the posthoc analyses showed that the change in heart rate for PP was significantly lower (p < 0.05) than the changes observed for bS and OHW.

analysis; this structure served as an introduction to the research activity for all three treatment groups and, for students assigned to the bS and OHW groups, as a tutorial on how to use the program to draw Lewis structures. Qualitatively, the results collected from the general chemistry and organic chemistry students tell a similar story: in general, the students assigned to OHW experienced the most cognitive load while constructing their Lewis structures. This is not unexpected given that the button-driven nature of the online homework system these students were tasked with using to create their structures greatly departed from the students’ existing work practices, i.e., using pencil-and-paper. The additional complexity of the interface appeared to contribute an additional extraneous load to the activity not observed within the other two groups. Initially, the general chemistry students using bS experienced an intermediate amount of cognitive load; however, that moderated as students progressed with their Lewis structure construction and apparently became more familiar and proficient with using the beSocratic system. In comparison, the cognitive load experienced by both groups of organic chemistry students assigned to the bS treatment was more similar to PP from the beginning of the process. Similar trends to those documented in this study were observed by researchers exploring interface effects in mathematics,51,52 language composition,53 and biology.54 Further, this research has shown that not only does the choice of interface have a dramatic impact on the overall cognitive load of the activity, but also it can directly influence students’ generation of new ideas, their approach to problem-solving, E



Article

Figure 4. Average percent change in heart rate for each Lewis structure for organic chemistry students (2017−2018 academic year) in each of the three treatment groups.

system was for the organic chemistry students involved in these studies. Individual p-values and their corresponding effect sizes are provided in Tables 1 and 2. One of the more dramatic differences observed between the groups of students in this research is just how much more load inducing the use of the online homework system was for the organic chemistry students versus their general chemistry counterparts. These results are counterintuitive as theory would predict that the overall cognitive load for the organic chemistry students should be less given their greater chemical content knowledge.60 What, then, explains this aberration? It may be that the time that elapsed since the organic chemistry students had taken general chemistry and used the online homework system “reset” their familiarity and comfort with its interface, and as such, it induced more load than anticipated. This hypothesis, however, is unsatisfying given that the organic chemistry students had started to use ChemDraw (the interface upon which the Lewis structure construction portion of the online homework system was modeled) to create reaction and mechanistic schemes for their laboratory reports. Alternatively, it may be that the organic chemistry students’ familiarity and comfort with the construction process itself had degraded over time given that it had been some time since they were required to create such representations. Regardless, this is certainly a trend worthy of additional research.

The analyses conducted on both groups of organic chemistry students indicated that the distribution within each Lewis structure and the overall average was significantly different (p < 0.05) among the three treatments. Posthoc analyses revealed that the average change in heart rate, and by extension, the average change in cognitive load, for students assigned to either PP and bS was significantly lower (p < 0.05) than for those students assigned to OHW. There were no significant differences between the PP or bS students. Effect sizes, r, were calculated for the significant differences for both groups of organic chemistry students. Given the nonparametric nature of the analyses performed, r was calculated as follows: r=

z N

where z is the z-score provided as part of the Mann−Whitney U-test, and N is the total number of subjects.58 The effect sizes calculated for the significant differences observed in the 2015− 2016 organic chemistry data ranged between 0.463 and 0.857, while those for the 2017−2018 organic chemistry data were between 0.439 and 0.744. Similar to the interpretation guidelines proposed by Cohen,59 small effect sizes start at 0.1, medium at 0.3, and large at 0.5, and as such, the calculated values represent medium to large effect sizes. These results add weight to the magnitude of the differences recorded between the PP and bS groups and the OHW group and highlight just how much more load inducing the use of the online homework F


r

p 0.184

Form versus Function: A Comparison of Lewis Structure Drawing

Recommend Documents