Qualitative and Quantitative Evaluation of Three Types of Student

Jan 6, 2017 - Instructional Innovation Services, Distance Education and Learning Technology ... Interdisciplinary Physiology Graduate Program, College...
3 downloads 0 Views 494KB Size
Download PDF
Article pubs.acs.org/jchemeduc

Qualitative and Quantitative Evaluation of Three Types of StudentGenerated Videos as Instructional Support in Organic Chemistry Laboratories Melinda C. Box,† Cathi L. Dunnagan,‡ Lauren A. S. Hirsh,‡ Clinton R. Cherry,§ Kayla A. Christianson,† Radiance J. Gibson,† Michael I. Wolfe,† and Maria T. Gallardo-Williams*,† †

Department of Chemistry, North Carolina State University, Raleigh, North Carolina 27695, United States Instructional Innovation Services, Distance Education and Learning Technology Applications (DELTA), North Carolina State University, Raleigh, North Carolina 27695, United States § Interdisciplinary Physiology Graduate Program, College of Agricultural and Life Sciences, North Carolina State University, Raleigh, North Carolina 27695, United States ‡

S Supporting Information *

ABSTRACT: This study was designed to evaluate the effectiveness of student-generated videos as a supplement to teaching assistant (TA) instruction in an undergraduate organic chemistry laboratory. Three videos covering different aspects of lab instruction (experimental technique, use of instrumentation, and calculations) were produced using student-generated scripts. A laboratory classroom was outfitted with video cameras and sound recording equipment that allowed the research team to monitor all TA−student and student−student interactions. Six course sections led by three randomly assigned TAs were selected. Two sections from each TA were observed (control and treatment), each at the same time of day, 1 week apart. Students in the control group had their TA conduct the lab briefing and supervise the lab, but were given no access to the instructional videos. The treatment group had videos available to supplement the TA’s lab briefing but was otherwise identical to the control group. Both groups were given a questionnaire that contained two comprehension questions per category to be completed during the lab before performing the experiment. Statistical analysis of the responses to this pre-experimental questionnaire showed that students who watched the videos had a better understanding of the methods than the students in sections that only received the TA lab briefing. Effect size calculations using Cohen’s d indicate that the Instrumentation video had a large positive effect on the number of correct responses in the treatment groups, while small effects were found for the Technique and Calculation videos. Content analysis of the lab transcripts supports these findings. In addition to these effects, treatment groups invariably completed the lab in less time than the control groups. Results from a follow-up survey e-mailed to students the week after their lab session show that most students found the videos to be valuable when completing the lab, with the Technique video being generally ranked as most helpful. KEYWORDS: Second-Year Undergraduate, Organic Chemistry, Collaborative/Cooperative Learning, Internet/Web-Based Learning, Multimedia-Based Learning, Instrumental Methods, Laboratory Equipment/Apparatus, Student-Centered Learning, TA Training/Orientation, Computer-Based Learning

L

However, to achieve these benefits of lab work, students must actively prepare. Without orientation to techniques, equipment, and method of analysis, students tend to focus on mechanical skills at the expense of deeper learning, whereas, with cognitive preparation, they apply both conceptual and procedural knowledge and better integrate their experience

aboratory work plays an important role in science education. Through it, students develop a deeper understanding of theoretical concepts, use of technology, and methods for investigation by hands-on manipulation of related materials. They also improve their critical thinking skills and clarify important ideas by discussing and debating with their lab mates as they learn in a cooperative setting.1 Finally, through social interaction necessary to accomplish lab experiments, they build their self-esteems and develop positive attitudes toward the subject matter, all of which are indispensable for student success and course retention.2,3 © XXXX American Chemical Society and Division of Chemical Education, Inc.

Received: June 21, 2016 Revised: November 28, 2016

A

DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

static video cameras and 360° spherical video with transcription of the TA microphone’s recorded audio, as well as via pre- and postexposure questionnaires and student surveys. The emergence of affordable, high-quality 360° spherical video recording and production technologies has made this approach possible. While in previous studies on the use of studentgenerated videos our team quantitatively established improvements in student outcomes,8 the current study was designed to determine how student focus and higher-order thinking behaviors were affected by integration of videos in lab instruction, using a mixed-methods approach. Three different types of video (technique, instrumentation, and calculation), all used during one single laboratory period, were included in this evaluation. All of these aspects were expected to contribute to a more beneficial laboratory experience. In previewing and pacing of information, students avoid cognitive overload due to exposure to excess demands on their attention and comprehension that may lead to reduced understanding. By allowing for development of both manipulative and process skills, students may absorb concepts and information in a more transferable way. With such powerful preparation, obligations toward safety, finishing on time, and writing up results can be less stressful and thus make more room for higher-order thinking. It is important to note that on this project the video content was sourced from students who recently completed the course. Although a professional crew did the filming, editing, and production, student veterans of the target course wrote the scripts, provided the visual demonstrations, and did the voiceover narrations. In this peer-to-peer transfer of knowledge, these contributors filled the gap between their recently acquired expertise and that of incoming novices. Because they had more immediate familiarity with the perspective of their fellow students, who come from many different majors and a wide range of experience, they were in a unique position to generate targeted instructional content that would remedy deficiencies in knowledge on an ongoing, as-needed basis. The format of the videos was chosen to maximize student use and benefit. In particular, the videos were made brief and focused to increase the likelihood of student engagement; they were made with the visual reinforcement of on-screen callouts to increase student understanding, and they were made with active demonstration to provide a useful model rather than strictly static images.8 These qualities are believed to be essential in obtaining and maintaining students’ attention, maximizing their comprehension, and maintaining their belief in the efficacy of the content.

with meaning. They are also less likely to miss important observations because their established theoretical framework improves their ability to interpret what they see.4,5 Whether by reading assignment, written exercise, instructor presentation, or video viewing, it has been shown that some form of related prelab activity improves lab outcomes, with video being associated with the greatest impacts.6,7 Meanwhile, barriers to use and distribution of videos are coming down, and as a result, videos are playing an increasing role in secondary education.6,8 The simultaneous ubiquity of cell phone cameras and video hosting sites such as YouTube and Vimeo have contributed to comfort and familiarity with streaming video for the current generation of students, which has made instructional incorporation an easy fit.9 However, videos are not just an easy way to distribute content now; they are also a more beneficial way to educate than traditional lecture with textbook. Because they provide multimodal input, videos help students to integrate new material with their existing understanding. By stimulating both the visual/pictorial and the auditory/verbal channels, videos can maximize working memory, so that concurrent thinking and processing occurs, leading to greater retention and accessibility of knowledge.10 Prior to this dramatic expansion in availability and use, videos already played an important role in science education because they provide methods of presentation that are not otherwise possible in a lecture. By animating things not visible to the naked eye or that are too big or too slow to see at one time, videos present concepts in a scope that is manageable. By capturing sudden, dangerous, or unusual phenomena, they provide access to extraordinary and essential exemplars. And by demonstrating complex methods such as laboratory techniques and mathematical problem-solving, they allow students selfpaced integration of difficult skills. In fact, video use for skill and technique instruction has benefits beyond those found for other applications. It improves users’ feelings of self-efficacy,3,11 as well as their meaningful interpretation and adaptable application of process steps.6 Indeed, it has been shown that, compared with just written or in-person instruction, video demonstration of practical skills improved student implementation over many subjects, including such disparate curricula as anatomy dissection,12 electric toothbrush usage,13 pharmacy compounding,14 and chemistry laboratories.4,8,15 This illustrative power combined with favorable technology conditions is creating an opportunity for significant impact. Because cell phone video viewing is standard and portable, viewers are able to watch technique demonstrations when they are ready for the information. They can also pace their exposure to information by pausing, starting, and replaying desired portions of the video. In addition, because images can be optimized through perspective via close-ups and visual angle and through modifications such as arrows, callouts, and highlighting, attention of viewers can be drawn to those things that will maximize success. Finally, through implicit consistency of content, students can have more confidence in implementing what they see.16

■

METHODS

Course Description

CH222 (Organic Chemistry Laboratory I) is a one-credit, semester long course with one lab meeting every other week (2 h 45 min long). It is part one of a two-part sequence that is completed by taking CH224 (Organic Chemistry Laboratory II). The course is open to all students, but contains only a small percentage of chemistry, biochemistry, and chemical engineering majors, who are encouraged to sign up for a more rigorous sequence (CH226/CH228). Examples of typical students who take CH222 include those majoring in animal sciences; agricultural sciences; biology; engineering; environmental sciences; food science; marine, earth, and atmospheric sciences; and textile sciences. Typical enrollment is approximately 1500 students per course per semester, divided into sections of about

■

PROJECT DESCRIPTION This study was designed to gather information about the differences in learning behaviors and outcomes associated with the use of student-generated instructional videos as a supplement to traditional instruction in an organic chemistry laboratory, as compared to a control group that did not have access to the video instruction. Information was collected via B

DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Google Cardboard. This allowed the researcher to look in any direction and observe what was happening in any particular moment. Due to the length of time required for recording, this set of six Go-Pro video cameras had a high fail rate due to overheating. However, the lessons learned provided valuable information for further investigations of 360° spherical video for higher education. All observation videos were stored on North Carolina State University servers with restricted access, according to the conditions required by the institutional review board. All identifying student information was maintained in strict confidentiality, and both questionnaires and surveys were collected under conditions of anonymity.

20 students, led by teaching assistants under centralized supervision by a faculty member. Control and Treatment Groups

For the purpose of this study, one laboratory room and its adjacent instrument room were outfitted with video cameras and a sound recording device that allowed the research team to observe all teaching assistant (TA)/student and student/ student interactions. Control groups (3 sections for a total of 60 students) had their TAs conduct the lab briefing and supervise the lab. Treatment groups (3 sections for a total of 59 students) had 3 videos available to supplement the TA lab briefing but were otherwise identical to the control groups. Students watched the videos on lab computers or on their cell phones. The links to the lab videos were provided to the students via e-mail, and bar codes that linked to the lab videos were posted in the lab and instrument room. The total time to watch all 3 videos was 9 min. Students were given a questionnaire that contained two questions per video category to be completed before performing the lab, but after completion of the TA lab briefing or TA lab briefing plus video viewing, depending on whether they were assigned to a control group or a treatment group. Treatment groups also received a follow-up survey the week after their lab was completed. Six course sections led by three randomly assigned teaching assistants (from a pool of experienced second-year TAs) were selected. Two sections from each TA were observed, each at the same time of day, 1 week apart. The first section from each TA conducted the lab without watching the instructional videos. The second section (the following week) conducted the lab after watching the instructional videos (compliance was ensured via review of the class video coupled with tracking of video views on YouTube). Participation in the study was voluntary, and all enrolled students chose to participate in the study.

Questionnaires and Surveys

There were six questions on the pre-experimental questionnaire, two related to each of the three key portions of the experiment, which are categorized below as Technique, Instrumentation, and Calculations. Each video is about 2 min long and covers one of these three components. All videos are available on YouTube; links are provided for each video. The Technique video was titled “Microscale Distillation Using a Hickman Still Head”17 (3:00 min); the questionnaire items related to it were the following: • How many pieces of glassware will you need to set up a microscale distillation apparatus for this experiment? • Is there any safety hazard(s) that you need to be mindful of when you perform today’s experiment? The Instrumentation video was titled “Gas Chromatography”18 (2:45 min); the questionnaire items related to it were the following: • How much sample will you inject to run the gas chromatogram? • How many peaks do you expect to obtain as a result of the GC separation? The Calculations video was titled “Peak Areas in a Gas Chromatogram”19 (3:20 min); the questionnaire items related to it were the following: • How will you calculate the areas of the peaks? • What will the areas of the peaks tell you about the products of today’s reaction? Students’ answers were evaluated in terms of correctness and totaled by category for each section. Because the questions are open-ended and related to experimental steps, the results are a reflection of students’ mental preparedness. Treatment groups were surveyed using a Qualtrics online instrument a week after the lab had taken place. The survey questions are listed below: • Did watching the prelab videos help you complete the lab? • If you said, yes, the prelab videos helped you complete the lab, please explain how the videos helped. • If you said, no, the prelab videos did not help you complete the lab, please provide us with information on changes you would make to the videos. • Please rank the prelab videos in order of helpfulness from 1 (most helpful) to 3 (least helpful). • If you ranked the Microscale Distillation Using a Hickman Still Head prelab video as most helpful please explain why.

Video Recordings and Transcription

Lab observations were conducted by recording video and audio. A Go-Pro Hero3+ camera was placed in the laboratory, positioned to capture the TA’s prelab presentation, students’ movements, and completion time for the lab. A second Go-Pro Hero3+ camera was placed in the instrument room, stationed to observe student interaction with specific instruments. Both cameras were equipped with 32 GB micro-SD cards, encased in skeleton cases, and powered through connections to the lab’s electrical outlets. The camera tripods were secured with gaffer’s tape to minimize movement through accidental contact. The teaching assistant wore a microphone to capture questions asked by students. The TA’s name, lab section, date, time, and class were noted. To ensure successful media collection, both video cameras and the TA’s audio were monitored for the entire duration of the lab period by a team located outside the lab. After the lab ended, the video editor combined the two videos (laboratory and instrument room) into a single video file, placing them side-by-side to be viewed simultaneously, and added the audio from the TA’s microphone. Audio files were submitted to AutomaticSync, an independent transcription service, for transcriptions that were then used for content analysis. In addition to the standard video recordings, the research team set up a Go-Pro 360° video rig in the center of the laboratory. The resulting spherical video recording could be viewed on screen or with a virtual reality headset, such as C

DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 1. Content Analysis Coding Scheme Summary Category Predetermined

Emergent

Code Technique Instrumentation Calculation Mechanism of the reaction Safety Miscellaneous Unrelated to class

Description Questions pertain to content covered in video 1, “Microscale Distillation Using a Hickman Still Head” Questions pertain to content covered in video 2, “Gas Chromatography” Questions pertain to content covered in video 3, “Peak Areas in a Gas Chromatogram” Questions pertain to the mechanism underlying the chemical reaction being studied; in this case, the phosphoric-acidcatalyzed dehydration of 2-methylcyclohexanol Questions pertain to lab safety issues directly related to the completion of the experiment Questions pertain to the class or the subject of chemistry but may be unrelated to the current lab experiment Questions are not related to the class or the subject of chemistry

• If you ranked the Gas Chromatography prelab video as most helpful please explain why. • If you ranked the Peak Areas in a Gas Chromatogram prelab video as most helpful please explain why. • Is there anything that you think we should change about the prelab videos?

no access to instructional videos. After each TA’s prelab presentation, a six-question pre-experimental questionnaire was distributed to the students, who completed and returned it to the TA before beginning the lab experiment. The second week, the same questionnaire was given to three new sections taught by the same TAs, but it was administered after both the prelab presentation by the TA and instructional video viewing. Answers on the questionnaires were marked right or wrong and summed by category corresponding to content of the instructional videos. The purpose of the pre-experimental questionnaire was to determine the degree to which the students were prepared for lab, and from this perspective, results were favorable. Comparison of each TA’s sections, without and with video, showed a higher or roughly equivalent percent of questions answered correctly in each category by those sections that watched the instructional videos. From this we gathered that the instructional videos prepared students to a degree equivalent to that of a thorough TA. Therefore, for the less thorough presentations, the videos filled in the gaps. This, in effect, maintained a uniformity of instruction, a desirable outcome considering the variability in a group of TAs. In addition to a comparison of the questionnaire results for each TA’s pair of sections, one without and one with video, a comparison was also made of combined results, i.e., for all those that did not watch the videos versus all those that did for each category. This was done in order to judge the effect of the videos on all students and to compensate for variations among recipients. It was accomplished using Cohen’s d, which is a form of descriptive statistics that, unlike p-values, identifies the size of an effect, not just whether or not one occurs. The outcome was that all categories showed a measurable effect, small for Technique and Calculations and large for Instrumentation. Composite results for all six lab sections are shown in Figure 1; effect sizes are shown in Table 2. These effect sizes may be explained by the nature of students’ needs in relation to video content. For example, the large Instrumentation effect may be the result of the novelty to viewers of working with a gas chromatograph. Without access to dynamic illustration as provided by the video, those viewers were less likely to retain procedural information about it. It is also likely due to the inclusion of modeling a hands-on process, since doing so has a proven impact on learning transfer.4,8,15 The Technique video, on the other hand, was less devoted to process demonstration and more to apparatus setup, so perhaps its impact was not as significant due to overlap with the content in the lab manual, which had a procedural description and an illustration of the setup as well. Lastly, the effect size for Calculations might be small also because similar content is available in the lab manual, which contains a worked example. It may be because students do not benefit as much from a

Content Analysis

The transcripts for each of the six laboratories were used for content analysis and coding using MS Word. Alternatively, transcripts were imported into ATLAS.ti (version 7.5, build 10). The data analysis stage began, as recommended by Creswell,20 by a combination of thematic coding and summarizing content analysis. Both manifest and latent content (i.e., words as uttered by participants as well as their underlying meaning or significance) were examined and interpreted within the coding frame. To ensure qualitative reliability, the research team employed reliability procedures recommended by Gibbs.21 Categories were defined as exhaustive and mutually exclusive. The coding scheme is summarized in Table 1. A complete coding scheme, including examples of each category, is included in the Supporting Information. Where areas of overlap were found, the coding scheme was reconsidered and refined using constant-comparative method.22 Familiarity with the experiment and teaching it influenced the ability of the evaluator to categorize student questions. Access to the accompanying video made it possible to identify whether a question on the transcript came from a student or a TA, or whether an exchange was initiated by the student or the TA. Statistical Analysis

After collection and organization of the data, Stata (StataCorp LP), a statistical analysis software package, was used to calculate the binomial mean and standard error. These binomial statistics were calculated for two data sets: the responses for students who received video instruction and the responses for students who received TA instruction, to compare the effectiveness of each type of instruction. Cohen’s d values were calculated in order to determine the effect size for each video treatment.23

■

RESULTS The lab experiment chosen for this study was the acid-catalyzed dehydration of 2-methylcyclohexanol (experiment 4). Three different videos were created by students familiar with the material: a video demonstrating the experimental technique, a video showing how to use the instrumentation required for the analysis of the reaction mixture (gas chromatograph), and a video with step-by-step instructions on how to perform the calculations required to determine the relative amounts of products obtained. During the first week of observations, three TAs taught one lab section each, and students were provided D

DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

comparisons (from three pairs of sections discussing three topics each) all TAs took less time to explain a topic with the group that had seen the videos. In addition, the Cohen’s d comparison across all those who hadn’t seen the videos versus all those who had also showed a small to medium effect size for all three topics. By contrast, time devoted by the TAs to explaining Mechanism, a topic not covered by any of the videos, was not affected when analyzed in this same fashion. Thus, incorporating videos in the students’ preparation may have reduced the time taken in person to prepare students, but did not necessarily shift the TAs’ focus to explaining concepts involving higher-order thinking such as the underlying Mechanism of the reaction. Content of the 360° videos was analyzed using transcripts of the recordings from the TAs’ microphones. Results are shown in Table 4. Students’ questions were identified and categorized Figure 1. Composite pre-experimental questionnaire results. Control group N = 60. Treatment group N = 59.

Table 4. Content Analysis of Transcripts Mean Number of Student Questions

Table 2. Composite Pre-Experimental Questionnaire Results Mean of Correct Responses, %

No Video, N = 60

Video, N = 59

Cohen’s d

Technique

25 ± 4

19 ± 5

1.25

Instrumentation

4.3 ± 0.6

5.6 ± 5.0

0.37

Calculation

7.0 ± 2.7

10 ± 6

0.64

Mechanism

6.7 ± 4.2

6.7 ± 9.1

0.00

Question Topic

Question Category

No Video, N = 60

Video, N = 59

Cohen’s d

Effect Size

Technique Instrumentation Calculation

72 ± 19 47 ± 5 49 ± 23

84 ± 10 73 ± 15 60 ± 1

0.56 2.58 0.63

0.27 (Small) 0.79 (Large) 0.30 (Small)

dynamic demonstration of thought processes as they do of hands-on processes. Instead, they must practice to improve understanding, and thus retention, in things like calculations and problem solving. All six sections were monitored using 360° spherical video, which was then analyzed for content and time on tasks, as shown in Table 3. This review of the lab footage showed that for paired sections (those that had the same TA) treatment groups invariably completed the lab in less time than the control groups. The students in the treatment sections had to watch 9 min of video in addition to the TA’s briefing. The fact that the treatment laboratories were completed in less time even with the addition of 9 min of video indicates to us that those students were more efficient and less dependent on their TAs for clarifications. In addition, verification using Cohen’s d on the collective (average with standard deviation of the three sections that did not use the videos compared to the average with standard deviation of the three that did) showed a medium effect size of video use on total time required to complete the experiment. Next, the time the TA spent explaining each video-related topic was measured, and with only one exception out of the nine

Effect Size 0.53 (Medium) 0.18 (No effect) 0.305 [?] (Small) 0.00 (No effect)

by video-related topic, as well as the additional topic of “Mechanism”. The totals for each category were determined, and comparisons were made without and with video for each TA and for the conglomerate. While there was apparently no consistency in direction of results (at most two out of three paired sections changed in the same direction for any given topic), the conglomerate showed a medium effect size for video use in Technique and a small effect size for Calculations. This may be due to the fact that each section consisted of only 18− 22 students and, by virtue of small numbers, could readily differ from one another in collective qualities such as initiative, confidence, and academic aptitude. Therefore, comparing all students who watched the video to all those who did not compensated for this normal variety. More significant than the size of the effects was their directions; they were opposite. While the average number of questions about Technique decreased, the average number about Calculations increased, and one may be related to the other. The decrease in students’ questions about Technique may reflect a reduction in cognitive load by the practicalities of the experiment, and a shift toward greater understanding of the

Table 3. Comparative Time Spent on Task Mean Time on Task, min Task Completion of Experiment including prelab or prelab + video Technique Instrumentation Calculation

No Video, N = 60 109 ± 15 TA Explanation of Topic 8.8 ± 5.3 8.1 ± 3.1 7.3 ± 1.6 E

Video, N = 59

Cohen’s d

Effect Size

90.3 ± 10.3

1.49

0.60 (Medium)

4.8 ± 1.3 4.7 ± 1.0 4.0 ± 3.6

1.03 1.47 1.19

0.46 (Small) 0.59 (Medium) 0.51 (Medium) DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

analysis. In other words, because students needed less verification and guidance on the method, they could shift their energy to the higher-order thinking required in doing calculations. By contrast, there was no effect by video use on student questions about Instrumentation or Mechanism. For Instrumentation, this may be because of two overlapping influences. One is that without the video students may not have known enough to ask questions due to the novelty of the instrument and process. The other is that, with the video, their confidence not only increased enough to do the procedure, but it may have increased enough to ask other instrument-related questions. For Mechanism, the absence of effect could be due to factors such as significant variations in TAs’ prelab presentations and variations in students’ individual abilities. Finally, in addition to the pre-experimental questionnaire and the content analysis of observational videos, a follow-up survey was given to solicit feedback from those who watched the videos. The response rate was roughly 25% of those who attended the lab session the week before (15 out of the 59). This is admittedly a low response rate, and we only offer the comments from students as a reflection on student satisfaction. From those respondents, 12 out of 15 students affirmed that they found the videos helpful in completing the experiment. One respondent said, “Actually seeing parts of the lab before I had to do them helped to clear away any confusion”. This clearing away helped students not only shift their focus toward more challenging things such as the reaction mechanism, but it helped them complete the experiment in less time. Another result from the follow-up survey was that most students ranked the Technique video the most helpful (9 out of 11 of those who answered the question). This supports the complementary result from the content analysis of the observational video that fewer questions were asked on this topic by those using the videos. One respondent said, “The lab was a little complex using many chemicals, so it helped keep everything in order”, which is another good description of the way that cognitive load was reduced, by the Technique video in particular. This reduced demand on students’ concentration by practicalities not only allowed them to complete the experiment in less time, but it freed them up to shift their focus toward the analysis of calculations, as reflected in the increase in the number of questions students using the videos asked on that topic.

Small effects were found for the Technique and Calculation videos. Review of the lab footage showed that treatment groups invariably completed the lab in less time than the control groups and were, in most cases, less dependent on explanations from their TAs in the subjects covered by the videos. Follow-up survey results show that most students found the videos to be valuable when completing the lab, with the Technique video being generally ranked as most helpful.

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.6b00451. Complete coding scheme, including examples of each category (PDF, DOCX)

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. ORCID

Maria T. Gallardo-Williams: 0000-0002-0056-264X Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS This project was possible due to the support of a 2-year NC State University DELTA large course redesign grant. The authors wish to acknowledge the invaluable contributions of Traci Temple and the DELTA team: John Gordon, Michael Castro, Arthur Earnest, Mike Cuales, and Chrissie Van Hoever. The authors also thank Mohamed Shakur and Victoria SaraldiGallardo for their assistance with script development.

■

REFERENCES

(1) Hofstein, A. The Laboratory in Chemistry Education: Thirty Years of Experience with Developments, Implementation, and Research. Chem. Educ. Res. Pract. 2004, 5, 247−264. (2) Buntine, M. A.; Read, J. R.; Barrie, S. C.; Bucat, R. B.; Crisp, G. T.; George, A. V.; Jamie, I. M.; Kable, S. H. Advancing Chemistry by Enhancing Learning in the Laboratory (ACELL): A Model for Providing Professional and Personal Development and Facilitating Improved Student Laboratory Learning Outcomes. Chem. Educ. Res. Pract. 2007, 8 (2), 232−254. (3) Cheung, D. The Combined Effects of Classroom Teaching and Learning Strategy Use on Students’ Chemistry Self-Efficacy. Res. Sci. Educ. 2015, 45, 101−116. (4) Powell, C. B.; Mason, D. S. Effectiveness of Podcasts Delivered on Mobile Devices as a Support for Student Learning during General Chemistry. J. Sci. Educ. Technol. 2013, 22, 148−170. (5) Mayer, R. E. Learning and Instruction; Pearson Education, Inc: Upper Saddle River, NJ, 2003; pp 287−288. (6) Burewicz, A.; Miranowicz, N. Effectiveness of Multimedia Laboratory Instruction. Chem. Educ. Res. Pract. 2006, 7 (1), 1−12. (7) Gregory, S.-J.; Di Trapani, G. A Blended Learning Approach to Laboratory Preparation. International Journal of Innovation in Science and Mathematics Education 2012, 20 (1), 56−70. (8) Jordan, J. T.; Box, M. C.; Eguren, K. E.; Parker, T. A.; SaraldiGallardo, V. M.; Wolfe, M. I.; Gallardo-Williams, M. T. Effectiveness of Student-Generated Video as a Teaching Tool for an Instrumental Technique in the Organic Chemistry Laboratory. J. Chem. Educ. 2016, 93 (1), 141−145.

■

CONCLUSIONS We found evidence that the use of student-generated instructional videos in the organic chemistry laboratories observed for this study had a positive impact on lab preparedness and learning outcomes. In particular, prelab learning increased as reflected by the higher percentage of correct answers in each of the three topics on the preexperimental questionnaire, as compared to a control group who did not have access to the video instruction. And this increase in prelab learning allowed students to shift their in-lab efforts toward the higher-order thinking involved in evaluating their results, which was evident in the apparent shift from asking Technique questions toward asking Calculation questions. Statistical analysis of the questionnaire results and effect size calculation using Cohen’s d showed that the Instrumentation video had the largest positive effect on the treatment groups. F

DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(9) Drahl, C. Lights, Camera, Chemistry: Learning How To Make Lab Demonstration Videos Can Pay Off for Instructors And Students. Chem. Eng. News 2010, 88 (6), 56−58. (10) Mayer, R. E.; Moreno, R. Nine Ways To Reduce Cognitive Load in Multimedia Learning. Educ. Psychol. 2003, 38 (1), 43−52. (11) Blonder, R.; Jonatan, M.; Bar-Dov, Z.; Benny, N.; Rap, S.; Sakhnini, S. Can You Tube It? Providing Chemistry Teachers with Technological Tools and Enhancing Their Self-Efficacy Beliefs. Chem. Educ. Res. Pract. 2013, 14, 269−285. (12) Doubleday, A. F. We Are What We Do: Examining LearnerGenerated Content in the Anatomy Laboratory through the Lens of Activity Theory. Anat. Sci. Educ. 2014, 7 (5), 361−369. (13) Renton-Harper, P.; Addy, M.; Warren, P.; Newcombe, R. G. Comparison of Video and Written Instructions for Plaque Removal by an Oscillating/Rotating/Reciprocating Electric Toothbrush. J. Clin. Periodontol. 1999, 26, 752−756. (14) Cretton-Scott, E.; Cruthirds, D.; Coward, L. Incorporation of Hands-On Sterile Technique Instruction in an Introductory Pharmacy Practice Experience. Am. J. Pharm. Educ. 2015, 79 (2), 28. (15) Nadelson, L. S.; Scaggs, J.; Sheffield, C.; McDougal, O. M. Integration of Video-Based Demonstrations To Prepare Students for the Organic Chemistry Laboratory. J. Sci. Educ. Technol. 2015, 24, 476−483. (16) Dusenbury, L. A.; Hansen, W. B.; Giles, S. M. Teacher Training in Norm Setting Approaches to Drug Education: A Pilot Study Comparing Standard and Video-Enhanced Methods. J. Drug Educ. 2003, 33 (3), 325−336. (17) NC State Undergraduate Organic Chemistry Teaching LaboratoriesS.M.A.R.T. Lab Videos: Microscale Distillation Using a Hickman Still Head. https://www.youtube.com/watch?v= 46gqI90ELuU (accessed Sep 2016). (18) NC State Undergraduate Organic Chemistry Teaching LaboratoriesS.M.A.R.T. Lab Videos: Gas Chromatography. https://www.youtube.com/watch?v=6Z61ezJFfyA (accessed Sep 2016). (19) NC State Undergraduate Organic Chemistry Teaching LaboratoriesS.M.A.R.T. Lab Videos: Peak Areas in a Gas Chromatogram. https://www.youtube.com/watch?v=03uDmX9ycIg (accessed Sep 2016). (20) Creswell, J. W. Research Design: Qualitative, Quantitative, and Mixed Methods Approaches, 4th ed.; Sage: Thousand Oaks, CA, 2014. (21) Gibbs, G. R. Analyzing Qualitative Data. In The Sage Qualitative Research Kit; Flick, U., Ed.; Sage: Thousand Oaks, CA, 2007. (22) Glaser, B. G.; Strauss, A. L. The Discovery of Grounded Theory: Strategies for Qualitative Research; Transaction Publishers Aldine Pub. Co.: Chicago, 1967. (23) Cohen, J. A. Power Primer. Psychol. Bull. 1992, 112, 155−159.

G

DOI: 10.1021/acs.jchemed.6b00451 J. Chem. Educ. XXXX, XXX, XXX−XXX