Learning Management System: Education Research in the Era of

Chapter 2

Learning Management System: Education Research in the Era of Technology Downloaded by DUKE UNIV on November 28, 2017 | http://pubs.acs.org Publication Date (Web): November 20, 2017 | doi: 10.1021/bk-2017-1260.ch002

Akash Mehta*,2 and Maria Kalyvaki1 1High

Performance Computing Domain Specialist, South Dakota State University, Brookings, South Dakota 57007, United States 2Independent Researcher and Consultant, Brookings, South Dakota 57007, United States *E-mail: [email protected].

Reports show that 99% of colleges and universities use Learning Management Systems (LMS). In this new online teaching environment, instructors have the ability to track their students’ interactions with each other and with the educational resources. The progress is recorded, and instructors can access those measured variables and later extract data to evaluate the effectiveness of their students’ learning experience. Performance analytics can lead to a deeper understanding of the learning process and propose improvements. This chapter aims to navigate students and young researchers from a variety of instruments to be used to assess data correlated with their student’s academic achievements.

Introduction It was in the 1990s when Higher Education started to emerge the use of Learning Management Systems (LMS) on campuses, and now almost 99% of colleges and universities are currently using an LMS. The use of the LMS has become part of the students learning experience either they select a face to face course, an online or hybrid (1). Modern learning management systems like BlackBoard, BrightSpace by D2L, Moodle, Canvas, etc. offer the opportunity to the instructors to create rich digital courses. The use of animations, videos, and other multimedia provides © 2017 American Chemical Society Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Downloaded by DUKE UNIV on November 28, 2017 | http://pubs.acs.org Publication Date (Web): November 20, 2017 | doi: 10.1021/bk-2017-1260.ch002

an engaging environment for the students. Nowadays, in contrast to traditional learning environments, students have use of asynchronistic and synchronistic interaction and communication within a virtual environment (2, 3). Web-based courses provide flexibility and accessibility for those students that are located off campus and those that either their schedule or their physical conditions do not allow them to participate in a traditional class (3). LMS systems are multidisciplinary by nature. Many different sciences are involved in the success of a digital course. Instructors that plan to teach by using an LMS are expected to have some experience with computer science, information systems, psychology, education and instructional technology (4). The LMS used acts like the bridge between the instructors and learners (5). Apart from being a learning port, the LMS serves as a teaching platform for the student’s progress and success (5). Part of the e-learning course planning is the selection of the appropriate techniques and tools (6). Studies have shown that learning outcomes have been the same as in traditional courses and students with prior experience using computers were more satisfied with online learning environments (7). LMS provide a broad range of tools to enhance their students learning experience, but relatively few are those that use those systems in their full capacity (1). The progress of technology evolves and every interaction and resource accessed can be captured and stored inside the LMS (8).

The Struggles of a Beginner Researcher For a beginner researcher (9–11) in the area of chemical education, often the production of research may seem overwhelming. Nearly every researcher has to create a research plan and go through a particular set of steps like obtaining institutional approvals, and a well thought out and planned strategy to collect data. The thought of data collection also brings the question of what data to gather and disregard, how to collect it, how to securely store it and where to store it? Almost every educational research project requires institutional review board (IRB) (12) approval, and the approval process requires a detailed description of these and related questions about data privacy, confidentiality, access and safety which often contain information about the research subjects (students and/or instructors). While going through this process and while actually collecting data, a beginner researcher is always looking for ways to store all the papers generated through the collection of original or copies of students’ work whether it is homework, quiz, lab reports, exams, etc. A one-semester research investigation involving such data collection creates a huge stack of papers that requires proper cataloging and enough physical space to securely store the data that will frequently be accessed by the researcher to further code, organize and analyze. The collected data helps the researcher gain insight of his proposed research questions. Such data will contribute to answer the research questions and conclude the investigation with valid evidence and reliability. It soon becomes overwhelming for a researcher when they have to collect data over several semesters. This is 10 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


often the case when a research investigation is being carried out by establishing a case for or against a teaching intervention, new curriculum material, course restructure, etc. This often leads to an office full of documents. Also, under such situation, it may also become difficult to ensure the data security requirements as required under IRB protocol for researchers with limited physical space. Almost every education institute, be it school district or college/university, uses Learning Management Systems (LMS) for conducting a variety of courses that they offer to their students pursuing academics in any grade/ major. Those LMS encapture helpful information that has the potential to help students, teachers and institutions make better choices that will lead to improved learning outcomes (13). While using LMS for course delivery to students, often researchers neglect the analytics tools that could help them develop their research. The underutilization of LMS despite its robust infrastructure could be attributed to poorly structured training provided to use LMS, which is often mainly focused on training instructors about how to use them for course content delivery in face-to-face or hybrid/online structure and creating and maintaining the grade books!

Why Using Learning Management Systems in Chemical Education Research? Learning management systems are very well integrated into the current academics. Some of the concerns that a researcher and IRB may have is the data authenticity confidentiality and privacy. Inside the LMS those concerns are adequately addressed at the highest priority as the access to information is strictly regulated within an academic institution. For example, a course access to an instructor or a teaching assistant (TA) is granted only upon approval of the appropriate departmental procedure and can be very well regulated for the duration (typically one semester) and type of information (student name, campus identification only) available to conduct the course successfully. Such access may be authorized for LMS data analysis. Data authenticity is very well maintained as all the information collected remain on a secure server that is only accessible to authorized individuals (instructor, TA) that they can only access through their own individually credentials. Moreover, as stated above, the access to data may be limited by adding only institute’s personnel in different roles that have pre-determined access allowances/ restrictions based on role, e.g., course administrator, course instructor, course teaching assistant, etc. Most academic boards (board of regents, school district boards) require that academic data, such as generated through LMS, be securely stored for an extended period of time (7-10 years) and after that, a determination may be made if further storage is required. This provides a real assurance for any researcher who wishes to either look at the data from the past for comparison purpose or for an ongoing sequential research investigation. In all these, the data is securely stored with access available through proper documentation. Also, this ensures that data is very well organized as each academic semester progresses. 11 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


Exploring Tools within the Learning Management System Most LMS offer an opportunity to create digitally rich courses with potential to incorporate interactive simulations, videos, and multimedia that provides an engaging learning environment (3) to the students. Instructors have the ability to track their students’ interactions with each other and with the educational resources. The progress is recorded, and instructors can access those measured variables and later extract data to evaluate the effectiveness of their students’ learning experience (2). Performance analytics can lead to a deeper understanding of the learning process and propose improvements. LMS serves as a tracking platform of the students’ progress and success (5). LMS provide a broad range of tools to enhance their students learning experience, but relatively few are those that use those systems in their full capacity (1). Various tools/functions are available in most LMSs where their utilization is mostly restricted to content delivery/collection (assignment submission) only while conducting a course (face-to-face or hybrid/online). Despite the merits of these tools, their use in conducting data collection for research investigation is seldom heard! A few of these tools are described below with the focus to collect factual data for research study while remaining a part of a course and without distorting the course structure:

Discussion Board The discussion board (DB) is a tool in the LMS that enables students to share their thoughts, opinions, and understanding of a topic with the subject matter. A discussions board includes discussion forums and threads (Figure 2.). When is utilized by the instructor can either promote a topic based discussion among student-student, student-instructor or any other possible combinations to promote critical thinking and social interaction. Often this tool is used as an aid to ask questions when an instructor is not available immediately and allows for anyone to respond. A discussion board is a fine example for adopting the constructivism learning theory (14) (Figure 1). The discussion board analytics could provide an important insight because it confirms what has been learned in the literature and provides interval-level data (14).

Cloud Storage Cloud storage (15, 16) systems such as Dropbox, Box, Microsoft OneDrive, Google Drive, etc. are often provided within LMS to allow for submission of assignments in a secure manner and without having to access them through a separate login. This tool is often used for submitting assignments that require students’ report/ opinion on content based topics. Based on the requirement of the course, a cloud submission may be individual or group submission. Assignments submitted through cloud could be graded online without being printed. Figure 12 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


3 depicts a typical course with multiple assignments within a course that may be activated at an appropriate time in the semester for students to submit their assignment.

Figure 1. Knowledge creation in discussion board.

Figure 2. D2L Discussion Board. 13 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


Figure 3. D2L Dropbox Folder System.

Quiz A familiar and a valuable tool in LMS for conducting either a face-to-face or hybrid/online course. Few of its features allow conducting quizzes in various modes, such as timed, conditional, randomized, sectional/individual release, etc. Often these modes are rarely used with most commonly used being randomized and timed. However, when employed in all such modes, it has the potential to generate most authentic research data without compromising on the course structure. Survey The survey tool (Figure 4) is almost never utilized tool in conducting a chemistry course thus; assuming that instructor/ researcher could use it for their research might be a long shot in utilizing this very useful tool whose potential is very crucial to most of the education research including chemical education research.

Discussion The world we live in today is highly data intensive (17). Almost everything that we do today generates a variety of data in digital format thus, leaving a trail of predictable behaviors about anyone in any setup. This insight into user’s pattern of living is of immense benefit to private companies that use such data for 14 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


their product design, development, and marketing to generate increased revenues. This also conveys that the job market will always be looking for candidates who are adept at such big data generation, management, and analysis using a variety of computation tools that are available today and grow further on them. All these are excellent news for a student education researcher who is undertaking such data-intensive studies during their academics, however, the caveat is that they use current day methods of data collection, management, and analysis. As mentioned earlier, most of the research in education setup is a mix of current day computer based as well as not so current manual data collection methods. While the computer based part is mostly used when it comes to data arrangement, coding and analysis and manual part are where data is stored in hard copy format, which provides no additional benefit for such storage that requires physical space to store and does not always guarantee the security needed for the regulatory boards.

Figure 4. D2L Survey Tool.

A well-planned research study has a well-planned data collection strategy that can be efficiently executed using LMS. The time requirements for such data collection during and before/after classroom meetings, e.g., lecture, lab, recitation, etc., would be almost same and may be less in some instance based on the nature of research study. A discussion board (DB) may be effectively used to forge in class discussion among students; such platform can serve the purpose of in-class observations. However, it has the potential to provide more in-depth data on students’ thinking process in their own writing in a classroom setting. During such discussions, an instructor may also interact posing probing questions that may prompt the same or other students to join in the discussion. Such DB also offers an excellent platform to students who would be too shy to speak out but may find it easy to communicate through posting in such DB. The merits of DB are often utilized by creating DBs, such as, ‘ask the professor ’or ‘student discussion board’ that allows only for out of class communication on non-urgent matters. Authors (17, 18) have used DB for in-class discussions and have first-hand observed an intense discussion among students-students and students-instructor on a topic under discussion and a participation rate higher than the one found during in-class oral discussions. A similar observation was made when authors created Dropbox folders for submission of lab-reports in a multi-section lab-course. The very first appreciate that student showed was about how much paper it saved for a one-semester 15 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


course that requires 11 lab-reports per student with at least 4-5 pages per report. A simple calculation for a course enrolling about 300 students puts this number to around 13,200 – 16,500 sheets of paper in original student reports, and when collecting data for research investigation, almost all of these reports will have to be copied again that will generate an equal number of pages. Collecting lab-report on Dropbox reduced this number zero pages being used for lab-report, provided very organized data for each student, moreover, cloud-based submission allowed students to resubmit their reports that allowed for researchers to directly compare their evolving thinking pattern for the same concept as well as over the period of the semester. The data collected from cloud storage could be submitted in any file format required by the course instructor, e.g., MS-Word, MS-Excel, PDF, etc., and most of this files are digitized thus, allowing for easy data integration into quantitative/ qualitative analysis software packages, viz., ATLAS.ti, IBM SPSS, NVivo, etc. On the contrary, a scanned document often requires a tedious task of manually entering each piece of information into such analytical software that takes additional time while leaving room for errors. The cloud-based data collection is very efficient and less error prone which offers for faster data analysis compared to the manual collection. The quiz function in LMS is a widely used feature and mostly associated with ‘end of chapter/lecture’ quiz or sometimes with homework assignments wherever, the course doesn’t have publisher provided homework system, such as OWLv2 (19), Mastering Chemistry (20), etc. Most quizzes are offered as timed activities and sometimes with randomization enabled. However, in this format, it is an underutilized function that even doesn’t serve a course instructor as well and not at all to the researcher trying to derive more information from such virtual tests. Most LMS offer quiz functions with a variety of possibilities of conducting them as well as generating data that may be useful to the course instructor and most definitely to a researcher. Such possibilities are often not known to either of them. In addition to the quiz being timed, conditional, randomized, sectional/individual release, a quiz can be set to generate reports on individual students’ time log, responses, and group/class log. These are readily available to any instructor/ researcher that provide detailed information about how students are approaching each question. It is certainly helpful to an instructor if they wish to make changes in their teaching strategy based on this information moreover; it is very useful to a researcher (21) who wishes to find a correlation between the intervention method and learning strategy adopted by students or any such variable that may be crucial to an investigator’s research. During their teaching, authors utilized quiz in this variety of functionality and generated data that helped them two folds’ way: 1) improvising their teaching strategy in an ongoing course while addressing the pressing concerns of the majority of students, 2) gain better insight into the intervention strategy adopted (3). These tools are very efficient compared to manual data collection method where, any such insight will only come when data is organized, which in such cases is almost always at the end of the semester. Thus, the insight gained from manual data collection cannot lead to addressing learning concerns of students of the current semester, however; they may be only incorporated in subsequent semester only. This doesn’t help students who face the problem at first in a semester for which such data is collected nor does it 16 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


give instructor/researcher any information about the improvised intervention’s suitability as it is first time tried in the next semester and not with the student who actually faced the challenge. Surveys are widely used in education research, a variety of surveys based on standardized and widely accepted instruments are regularly used. LMS offers to develop such surveys with almost the same amount of time it takes to create them first time after that, only requiring them to be copied into each course where such surveys need to be conducted. This is immensely time-saving and learning from the example of cloud-based submission lab-report, LMS based surveys also have no use of papers. Like LMS based quiz, surveys can also be conducted in a variety of question-answer format, viz., multiple choice, multiple select, Likert, short/ long answer, fill in the blank, etc. Report generated through such surveys can be readily imported into the analytical software. A conditional release is a tool that is rarely used in LMS. However, authors have utilized this function in the following order: At the first student would participate in ‘during and after’ class discussions this will let them submit the assignments on cloud storage. And upon submission, the quiz associated with the assignment will be open for them to take. Once they complete the quiz, they will be able to take the survey. Thus, it is not possible for a student to take any component without following a systematic order. And at every step, an instructor/ researcher is getting a better insightful data on individual student, and in many instances, such information is very helpful in providing instant help to students struggling in subject matter and may be considering to drop the course.

Conclusion With this paper, our effort has been to highlight the utility of Learning Management System in conducting an efficient, authentic, and secure research in chemical education. When fully utilized, LMS has the potential to provide substantial insight into a course on an ongoing basis that may be helpful to the instructor for making in-course interventions to keep their students interested, engaged and enrolled in course as well as a reliable tool to collect data for pursuing research investigation. The author will be publishing their own research studies conducted using LMS for all of their data collection in a separate manuscript. Academic institutions should incorporate into their training on LMS about how the various features may be utilized to collect data and conduct education research. Data-intensive research is already proving its potential in all walks of our lives, and it is important that such tools be completely utilized in conducting chemical education research. LMS based research meet the merits of data authenticity, security, and efficiency. A well-planned research study using LMS will certainly help collect better quality data that will offer better insight into current teaching practices and new potentials in research-based teaching interventions. The increased integration of LMS in conducting chemical education research will happen with more tools and functionalities being added, and as researchers evolve in their use of LMS for conducting their research study. 17 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Acknowledgments Authors wish to thank Dr. Tanya Gupta for the constant support, encouragement, and critical feedback, which were instrumental in writing this manuscript.

References 1.


2. 3. 4. 5. 6. 7. 8. 9. 10. 11.

12.

13.

14. 15.

16.

17. 18.

Dahlstrom, E.; Brooks, D. C.; Bichsel, J.; Brooks, D. C.; Bichsel, J. Educ. Res. Rep. 2014. Ozkan, S.; Koseler, R. Comp. Educ. 2009, 1285–1296 DOI:10.1016/ j.compedu.2009.06.011. Broadbent, J.; Poon, W. L. Internet High. Educ. 2015, 1–13 DOI:10.1016/ j.iheduc.2015.04.007. Ozkan, S.; Koseler, R. Comp. Educ. 2009, 1285–1296 DOI:10.1016/ j.compedu.2009.06.011. Cavus, N.; Uzunboylu, H.; Ibrahim, D. J. Educ. Comp. Res. 2007, 301–321. Ćukušić, M.; Garača, Ž.; Jadrić, M. Comp. Educ. 2014, 100–109 DOI:10.1016/j.compedu.2013.10.018. Gikandi, J. W. W.; Morrow, D.; Davis, N. E. E. Comput. Educ. 2011, 2333–2351 DOI:10.1016/j.compedu.2011.06.004. Pardo, A.; Siemens, G. Brit. J. Educ. Technol. 2014, 438–450 DOI:10.1111/ bjet.12152. Kluge, M. A. J. Aging Phys. Act. 2001, 329–335 DOI:10.1123/japa.9.3.329. Schwarz Mccotter, S. Qual. Rep. 2001, 1–24. Horvat, E. M.; Heron, M. L. The Beginner’s Guide to Doing Qualitative Research : How to Get into the Field, Collect Data, and Write up Your Project; Teacher’s College Press, 2013. Commissioner, O. of the. Search for FDA Guidance Documents Institutional Review Boards Frequently Asked Questions - Information Sheet; U.S. Food and Drug Administration, 2016. Analytics: getting helpful information out of an LMS; Salvetore, 2012; https://salvetore.wordpress.com/2012/07/26/analytics-getting-helpfulinformation-out-of-an-lms/ (accessed May 17, 2017). Harman, K.; Koohang, A. Interdiscip. J. Knowl. Learn. Objects 2005, 1, 67–77. Karabayeva, K. Z. Creation of a Unified Educational Space Within a SLA University Classroom Using Cloud Storage and on-line Applications. Int. J. Environ. Sci. Educ. 2016, 11, 8669–8678. Karamete, A. Computer Education and Instructional Technology Teacher Trainees’ Opinions about Cloud Computing Technology; 2015; pp 2043–2050 DOI: 10.5897/ERR2015.2297. Mehta, A. Use of LMS in Conducting Pretequisite Chemistry Course and Education Research. Manuscript in preparation. Mehta, A.; Tummala, H. Using LMS to Assess Students’ Understanding of Concepts, Critical Thinking, and Problem-Solving in the Formulation and Compounding in Pharmacy Course. Manuscript in preparation. 18

Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.


19. OWLv2 - Cengage Learning; http://www.cengage.com/owlv2/ (accessed May 17, 2017). 20. Mastering Chemistry; Pearson, 2017 http://www.pearsonmylaband mastering.com/northamerica/masteringchemistry/ (accessed May 17, 2017). 21. Gámiz Sánchez, V.; Montes Soldado, R.; Pérez López, M. C. SelfAssessment via a Blended-Learning Strategy to Improve Performance in an Accounting Subject. RUSC. Univ. Knowl. Soc. J. 2014 DOI:10.7238/rusc.v11i2.2055.

19 Gupta; Computer-Aided Data Analysis in Chemical Education Research (CADACER): Advances and Avenues ACS Symposium Series; American Chemical Society: Washington, DC, 2017.

Learning Management System: Education Research in the Era of

Recommend Documents