Communication Cite This: J. Chem. Educ. XXXX, XXX, XXX-XXX
pubs.acs.org/jchemeduc
ConfChem Conference on Select 2016 BCCE Presentations: Tracking Student Use of Web-Based Resources for Chemical Education Robert Bodily* and Steven Wood Instructional Psychology and Technology, Chemistry, Brigham Young University, Provo, 84602 Utah, United States S Supporting Information *
ABSTRACT: This paper presents the technical infrastructure required to track student use of web-based resources in an introductory chemistry course, the design of a student dashboard, and the results from analyzing student web-based resource use. Students were tracked as they interacted with online homework problems and high quality course content videos. These videos were developed using funds from the National Science Foundation. We found that students did not use the online video resources as much as expected. If students do not use web-based resources, practitioners and researchers need to (i) track and analyze student use, (ii) implement strategies to increase student use, and (iii) only invest in resources students will use. This Communication summarizes one of the invited papers to the Select 2016 BCCE Presentations ACS CHED Committee on Computers in Chemical Education online ConfChem held from October 30 to November 22, 2016. KEYWORDS: Distance Learning/Self Instruction, Internet/Web-Based Learning, First-Year Undergraduate/General, Undergraduate Research, Student-Centered Learning
■
INTRODUCTION This paper presents the technical infrastructure required to track student use of web-based resources in an introductory chemistry course, the design of a student dashboard, and the results from analyzing student web-based resource use. The paper was discussed in the Fall 2016 ConfChem: Select 2016 BCCE Presentations,1 organized by Jennifer Muzyka, Tanya Gupta, and Bob Belford, which dealt with selected presentations from CCCE Sponsored Symposia during the 2016 BCCE. This Communication includes highlights from the full paper that was discussed online.2 (See the Supporting Information.)
and videos (play, pause, skip forward/backward, increase speed, change volume). A screenshot from our dashboard system can be seen below (Figure 1).
■
METHODS During the fall semester of 2015, we implemented our webresource tracking system in a general level blended chemistry course at a large western US university. There were 200 students that consented to participate in the study. The chemistry course content that was available to these students via the system we had developed consisted of (1) high production quality videos, including animations and audio, developed with an NSF grant and publically available at chempath.byu.edu, (2) short formative quizzes related to the videos in the course, and (3) a dashboard that provided students with information on their knowledge gaps and metacognitive skill abilities based upon the use and performance data generated on their performance and use.
■
BACKGROUND As online and blended learning in chemistry education continues to grow,3 it is increasingly important to understand how students interact with resources online. Once we begin to understand how students use web-based resources, that data can be used to improve the design, access, and content, which closes the feedback loop and improves the learning potential of these resources.4 This is the focus and objective of learning analytics.5 The learning analytics process includes selecting data, capturing data, using data, and acting on data.6 We developed a learning analytics system, in the form of a class dashboard, to capture student user data as they interact in realtime with course content such as videos and quizzes.7
■
RESULTS There were a number of results and issues discussed in the online paper and discussion. These included the following: • Student use of videos and dashboard was low. • The median student accessed 20% of the videos. • Class structure changes affect student resource use. • Only 40% of students accessed the online dashboard system. • Predictors of student success included actively watching videos, being confident in your answers, not working late at night, and consistently working on the class every day.
■
ANALYTICS SYSTEM We implemented a quiz system and a video system from open source software and then used interoperability standards (Learning Tools Interoperability and Experience API) to track student data across applications. This data was visualized for students in real-time in the class dashboard. We collected all click events that occurred in quizzes (answer a question, go to another question, view hint, view answer, confidence in answer) © XXXX American Chemical Society and Division of Chemical Education, Inc.
Received: December 17, 2016 Revised: May 8, 2017
A
DOI: 10.1021/acs.jchemed.6b00976 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Communication
Figure 1. Class dashboard displaying student mastery scores.
■
• There are many systems that provide video and quiz functionality, but to better investigate how students engage with online feedback, we created our own system because (1) we have access to data for real-time reporting and analysis, and (2) we can change the class dashboard tool whenever we want. • Additional research should be conducted with additional variables such as student opinions, student demographic information, and learner characteristic data.
user behavior. Both qualitative data and quantitative data need to be collected to strengthen findings from learning analytics.
■
CONCLUSION We have developed a web-based resource tracking system (quiz, video, and dashboard) in order to track student use of resources. We found that students do not always use resources, even if they are high quality. We also found that pairing assessments to videos increased the use of videos. Learning analytics has the potential to enable practitioners to redesign their courses to increase student use of resources, increase student engagement, and improve student mastery of the content material.
DISCUSSION SUMMARY Because this research is exploratory in nature, most of the comments were suggestions and ideas to further investigate student use of web-based resources. Suggestions included technology products that provide similar functionality, additional data points that would be useful to track, features to add that may affect student use of these resources, and learning assumptions that have implications in online teaching.
■
ASSOCIATED CONTENT
S Supporting Information *
The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.6b00976. Full text of the original paper and associated discussions from the ConfChem Conference (PDF)
■
IMPLICATIONS FOR PRACTICE Tracking the use of various resources by students can provide rich information to practitioners that can shape instructor decisions regarding the use of such resources. For example, if instructors have access to information about student use of videos, they can add a graded quiz after or within the video ensuring that students use the video resources. The student resource use data generated from learning analytics could also lead to redesign of course materials. If some resources are used more than others, the course instructor could evaluate the resources to better understand why this is the case. In the future, data generated from student use of resources can be coupled with student perception and other data on learner characteristics or demographics. This would provide strength to learning analytics as a measure of understanding
■
AUTHOR INFORMATION
Corresponding Author
*E-mail:
[email protected]. ORCID
Robert Bodily: 0000-0002-6985-6224 Notes
The authors declare no competing financial interest.
■
REFERENCES
(1) American Chemical Society, Division of Chemical Education, Committee on Computers in Chemical Education. Fall 2016
B
DOI: 10.1021/acs.jchemed.6b00976 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Communication
ConfChem: Select 2016 BCCE Presentations. https://confchem.ccce. divched.org/2016fallconfchem (accessed Apr 2017). (2) Bodily, R.; Wood, S. Tracking Student Use of Web-Based Resources for Chemical Education. http://confchem.ccce.divched.org/content/ tracking-student-use-web-based-resources-chemical-education (accessed Apr 2017). (3) Allen, I. E.; Seaman, J. Tracking Online Education in the United States; Babson Survey Research Group and Quahog Research Group, L L C :: W e l l e s l e y , M A , 2 0 1 4 ; p p 1 − 45 . h t t p : / / w w w . onlinelearningsurvey.com/reports/gradechange.pdf (accessed Apr 2017). (4) Strader, R.; Thille, C. The Open Learning Initiative: Enacting Instruction Online. In Game Changers: Education and Information Technologies; Oblinger, D. G., Ed.; Educause: Louisville, CO, 2012; pp 201−213. https://www.educause.edu/research-and-publications/ books/game-changers-education-and-information-technologies (accessed Apr 2017). (5) Siemens, G. In 1st International Conference on Learning Analytics and Knowledge 2011, Banff Alberta, Feb 27−Mar 1 2011. https://tekri. athabascau.ca/analytics (accessed Apr 2017). (6) Elias, T. Learning Analytics: Definitions, Processes and Potential, 2011. http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.456. 7092 (accessed Apr 2017). (7) Verbert, K.; Duval, E.; Klerkx, J.; Govaerts, S.; Santos, J. L. Learning Analytics Dashboard Applications. American Behavioral Scientist 2013, 57 (10), 1500−1509.
C
DOI: 10.1021/acs.jchemed.6b00976 J. Chem. Educ. XXXX, XXX, XXX−XXX
