Analysis of a Distance-Education Program in Organic Chemistry

Research: Science and Education

Chemical Education Research

Analysis of a Distance-Education Program in Organic Chemistry Martha J. Kurtz* Department of Chemistry, Central Washington University, Ellensburg, WA 98926; [email protected] Brandt E. Holden Zillah Senior High School, Zillah, WA 98953

Distance education has become a very popular mode for providing education to students who would not otherwise be able to take classes (1–4 ). Historically, distance education has taken many forms, most of which involve a work-at-yourown-pace philosophy. Modern fiber-optic communication technology permits much more sophisticated distance education formats, including ones that mirror very closely a traditional classroom situation. Several states have begun to implement and test various distance education curricula using this technology (5, 6 ). Many studies in a wide variety of subject areas have analyzed many modes of distance education (e.g., 7–11). In general the results of these studies show that distance education works as well as traditional methods of instruction (12). Unfortunately, studies comparing distance education methods are inherently confounded by the inability to randomly assign learners to groups (12). In most cases the students taught by distance education and those who take on-campus courses are from different populations. Although distance education formats have been used for lecture in chemistry courses, little is known about the relative effectiveness of the method compared to traditional methods with regard to student performance in this area, especially in relation to the newer interactive video methods available today. It is important to begin to evaluate the effectiveness of this mode of education before spending lots of money on technology that is less than effective for the modern scientific learner. The use of distance education to provide access to organic chemistry is appealing. Because of lack of necessary lab equipment, instrumentation, instructor time and expertise, and students, organic chemistry is not usually offered at small rural community colleges. Postponing enrollment in an organic chemistry course can put science and preprofessional students at a serious disadvantage when they transfer to a four-year institution. Offering organic chemistry from a four-year institution in a distance-learning setting enables students to fulfill their organic chemistry requirements conveniently at their community college. Concern exists, however, about the effectiveness of distance learning. The purpose of this study was to assess the performance and attitudes of students learning organic chemistry from an off-campus site in comparison to the performance and attitudes of students learning organic chemistry in a traditional classroom. Method For this study, two rural community colleges were electronically connected to a small regional university. The distanceeducation organic course used real-time interactive video to teach the lecture portions of the course to students off campus. 1122

Cameras located both on site and off site provided live video and audio feeds between the on-campus classroom and the offsite classrooms. In this way, the organic chemistry instructors were able to communicate directly with students at all three locations. A course Web page was developed to facilitate the learning of all students. The course syllabus, homework assignments, answers to homework assignments, and quizzes were posted to the Web page. Instructors were available to all students via email. The nature of the communities from which students came allows a reasonable assumption that students are from the same population. Because the four-year college is a small regional university, its typical student is very similar to the typical student attending the two community colleges. All three communities have an economy driven by agriculture. Two measures of group similarity, spatial ability and general chemistry background, were used to validate the assumption that the students were drawn from the same larger population. Both spatial ability (13, 14) and general chemistry (15) have been shown to correlate with success in organic chemistry. If students from the different groups perform similarly on these measures, they would be expected to perform similarly in the organic courses under similar teaching and learning circumstances. However, since some students learned chemistry from an off-site location, their performance in the organic courses would be different if the method of teaching affected their ability to learn. The participants included the students in two sections of organic chemistry lecture. One section was taught in the traditional way; the other was taught via interactive video. The interactive video section included some on-site students in the university’s distance-education classroom and some students at the off-site classrooms. Interactive video allows students off site to ask questions of the instructor if they are seated in the receiving room. Therefore, distance education in this case means students are attending class at the same time at another location. Only students at the off-site locations were considered to be learning by distance education because the on-site students were being instructed on campus in the same way and with the same access to instructors as students in the traditional section; they were just being taught at the same time as the off-site students. Using this definition of distance education (only off-campus students), in the first quarter 47 students completed the traditional instruction (of the 49 who started) and 7 students completed the course via distance education (of 7 starting). The following quarter, the second course of a three-quarter organic sequence, 47 students completed the course traditionally (of 54 starting) and 4 completed the course via distance education (of 5 starting). An interview with the instructors indicated that the distance-education

Journal of Chemical Education • Vol. 78 No. 8 August 2001 • JChemEd.chem.wisc.edu


student in the second quarter who dropped probably did so because of multiple obligations including work and family. Two veteran faculty whose expertise is in organic chemistry team-taught both the traditional and distance education sections. The course was structured so that one faculty member lectured to both sections for approximately one week and the other attended as often as possible. The roles would then be switched for the next week. This method eliminates instructor effects on the results of the study. The two lecture instructors taught the laboratory to on-campus students with sections of 18 and 11 students. Two other instructors taught laboratory sections at the community colleges for the off-campus students with sections of 4 and 13 students. Students were allowed to

Box 1. Survey I Name: ______________________ SSN: ______________________ 1. Complete the following table for previous chemistry courses taken: Course

Name of School

Instructor

Year

2. Check the box if you have used the following items prior to taking this course:

register for any lab so the lab sections contained mixes of distance and non-distance lecture participants. On average two lectures were given each quarter at each of the distance sites, one at each site by each faculty member. The lectures given at distance sites were scheduled for convenience—for example, on a day when exams were being handed back or when instructors felt off-site students needed reassurance. During the first quarter lectures were given at one site but only visits, no lectures, were given at the other site because of technical difficulty. A spatial ability test score and a general chemistry score were used to detect any initial differences between the sections of organic chemistry students. Both measures are associated with success in organic chemistry (13–15). The Purdue Spatial Visualization Test was administered to all students at the beginning of the first course. It consists of 20 questions measuring the ability to rotate a three-dimensional object mentally (16 ). A representation of the general chemistry background of both the on-campus and off-campus students was obtained from scores on the first organic chemistry exam. Questions were categorized as general chemistry or organic chemistry and two separate scores were tabulated. Three exams were given during each course. The same version of each exam was given to the on-campus and offcampus students. The lecture instructors assigned specific

Mac

Box 2. Survey II

IBM PC or clone Word Processing

Name: ______________________ SSN: ______________________

Spreadsheets E-mail

1. (a) Check all that apply:

Internet

3. Where do you access computers? Circle appropriate answer(s): home

school

other

4. Have you had any problems accessing school computers? Explain. 5. Do you feel there are any ways the off-site facilities can help make distance learning more effective? Give your suggestions. 6. Why did/didn’t you decide to take organic chemistry through the distance learning program? 7. What do you feel are some of the strengths/weaknesses of distance learning? 8. Do you have any suggestions for getting quizzes and lab reports back in a more efficient manner? 9. If you had a choice between either three 50-min lectures/week or two 80-min lectures/week for organic chemistry (all else being equal), which would you prefer?

Fall 1997 Distance ed section Fall 1997 Non-distance ed section Winter 1997 Distance ed section Winter 1997 Non-distance ed section

______________ ______________ ______________ ______________

(b) Give reasons for your registration in these sections. (c) What would you recommend to a student planning to take organic chemistry next year who has the option of a distance ed section? 2. What do you feel are some of the strengths/weakness of distance learning? 3. (a) When did you take the lab section last? Fall 1997 __________

Winter 1998 __________

(b) Which would you recommend to another student and why? 4. Was the laboratory schedule convenient for you? (Day, time, trips) 5. Have you attempted to contact your instructor by E-mail? If so, how many times? 6. Complete the following table:

10. Do you find team teaching to be an effective means of learning in this course? Why or why not?

Total number of hours per week spent on computer

11. Are the off-site visitations by the faculty helpful to you? Why or why not?

Number of times using E-mail per week for this course

12. Have you attempted to contact your instructor by E-mail? If so, how many times? 13. Complete the following table: Total number of hours per week spent on computer Total number of hours per week spent on computer for this course Number of times using E-mail per week for this course Number of times accessing the course Web page per week

Total number of hours per week spent on computer for this course Number of times accessing the course Web page per week

7. Have you had any problems accessing school computers? Explain. 8. What type of software would you be willing to purchase to help in this course? And in what price range? 9. What software would you use for this course if it was provided on campus computers? 10. What have you found most useful for understanding and analyzing the stereochemistry of structures?

JChemEd.chem.wisc.edu • Vol. 78 No. 8 August 2001 • Journal of Chemical Education

1123


Results Because of the low numbers of students in the off-campus group and the resulting numerical inequality between groups, there was not enough power to perform independent sample t-tests. The Mann–Whitney U test, a nonparametric statistical test to determine whether two independent samples are from the same population (17 ), was used to compare the groups. There was no significant difference between groups in mean spatial ability scores or in scores on the general chemistry portion of the first exam (see Table 1). Mean scores for each course exam were calculated for the first and second courses (see Fig. 1). Table 2 shows that none of the scores differ significantly between the two groups of students. The total points accumulated in the course (including homework, quizzes, and exams) were also analyzed. No significant differences were found between on-campus and off-campus students, although off-campus students did outscore on-campus students slightly in both courses. Analysis of the survey responses showed students had some common opinions about learning by interactive video, team teaching, and the course Web page. When asked why they enrolled in the interactive video section, off-site students said they enrolled because of geographical convenience and on-site students enrolled primarily because of scheduling convenience. Because the course was presented by interactive video to the off-campus students, it was convenient to make videotapes that were made available at all sites for checkout by the students. The students expressed their appreciation at having access to previous lessons on tape. Students in this section also liked the smaller teacher–student ratio. They expressed no fear of the interactive video format, but did note that technical difficulties were a periodic problem. Students 1124

Table 1. Initial Measures of Group Equivalence for On-Campus and Off-Campus Students Test

Section

N

Mean

U

Spatial ability

off-campus on-campus

26 29

14.2 12.3

59.5

General chemistry


27 44

15.9 15.7

p .227

128

.440

Table 2. Scores for On-Campus and Off-Campus Students Compared by the Mann–Whitney U Test Exam

Section

N

Mean

U

p

First Quarter 1


7 45

87.7 78.7

99.5

.120

2


7 46

67.1 72.2

132.0

.446

3


7 45

56.0 60.2

143.5

.707

Total pointsa


7 47

296.3 289.5

158.0

.867

4


4 47

66.2 56.0

55.0

.172

5


4 47

91.0 85.2

63.5

.285

6


4 46

49.5 50.0

89.0

.914

Total pointsa


4 47

206.7 190.2

64.0

.293

Second Quarter

a Includes

homework, quizzes, and exams.

100

On-Campus

90

Off-Campus

80

Percent Correct

questions for each instructor to grade for all of the exams. Thus consistency in grading between the two different sections was assumed. Exam scores from the first- and second-quarter lectures were analyzed to detect any statistically significant differences between the distance and non-distance sections. The researchers constructed surveys that were administered to all organic chemistry students to assess attitudes toward distance education, team teaching, the course Web page, and the laboratory portion of the course. Survey I (see Box 1) was given midway through the first quarter of organic. Questions were designed to assess students’ familiarity with technology, give the instructors feedback on specific aspects of the course particularly related to distance education, and give researchers a tool for measuring attitudes about distance education from both distance and non-distance students. A second survey was designed for the second course (see Box 2) to elicit attitudes about distance education and to give instructors specific feedback about the second course. Attitudes were inferred from the explanations for registration in different sections of the course. Some students completed one quarter via distance education and the next traditionally and vice versa. Some questions were repeated on Survey II to see if any changes in attitude had occurred by the second quarter. The researchers collected the surveys directly from the students. Students were promised that their instructors would not see their responses to the surveys. With this assurance, they were expected to answer the surveys candidly.

70 60 50 40 30 20 10 0 Exam 1

Exam 2

Exam 3

First Quarter Exams

Exam 4

Exam 5

Exam 6

Second Quarter Exams

Figure 1. Exam scores for on-campus and off-campus students.

felt that “time was wasted when technical difficulties occur” but these did not occur very often. One off-site location did experience technical difficulty with approximately 20% of the lecture transmissions (as reported by instructors) during the first quarter. The difficulties included poor sound quality and a disruption in video or sound. Most of the difficulties were eliminated by the second quarter, since they were due mainly to a new facility and inexperienced operators. The off-site students overwhelmingly supported distance education because “[I] wouldn’t have been able to take the course otherwise.”

Journal of Chemical Education • Vol. 78 No. 8 August 2001 • JChemEd.chem.wisc.edu


Since both sections of organic chemistry involved team teaching, students were asked to comment on their feeling about the effectiveness of the team-teaching method. The majority felt the team-teaching method and the differing styles of the two professors made the teaching and learning more effective. For example, several students responded that the two different teaching styles were refreshing. They also stated that “if one teacher does not explain something well, the other one usually does.” A few students, however, did not like the team teaching saying, “it was hard to switch back and forth between style of teaching and note taking.” The traditional section and the interactive video section used the same Web page. The survey showed students liked having detailed quiz and test answers available on the course Web page. Twenty-five students declared the Web page to be helpful while only five stated it was not. Off-site students were often required to email assignments to the instructors; this did not present a problem for any of the students. Overall, the students were positive about their experiences using all of the technology. One off-site student stated “it’s more efficient, optical and data technology over mechanical (driving).”

from small rural settings. The initial measures of differences support the notion that these two groups are similar. The results of this study are encouraging. Interactive video allowed students who did not have the opportunity for daily contact with their instructor to learn. Continuing analysis of this method will add to the validity of the conclusions by increasing the numbers of students involved in the statistical analyses. Future work will incorporate a more reliable measure of group similarity, such as the ACS General Chemistry standardized exam. Acknowledgments We wish to thank JoAnn DeLuca and John Gerdes for their cooperation in allowing us to gather the data used in this study. John DiBari and Richard Logan were instrumental in the success of this study by facilitating at the distance site. Marte Fallshore provided statistical consultation. Funding for the distance-education organic course was provided by CWU College of the Sciences. Literature Cited

Conclusions The spatial ability test and the general chemistry questions from the first exam show group equivalence between the off-campus and on-campus students. The general chemistry questions were, however, inherently influenced by the general chemistry topics discussed in the organic course. Still, both groups appeared to be equivalent at the beginning of the course. Interestingly, the two exams with the largest difference in the means (exams 1 and 4) favored the offcampus students. The groups were not significantly different at the end of either the first course or the second course, suggesting that for this small group, learning organic chemistry by a distance-education mode is as effective as traditional modes. Responses from surveys indicated an overall positive attitude toward distance education. Students also responded favorably to the other technological methods that they were asked to employ, such as using email and a Web site to gain and transmit information. The few technological problems that occurred seemed to have little impact on the students’ attitude toward the distance-education model tested here. Admittedly, this study contains too few off-campus students to permit solid conclusions. The particular students involved in distance education in this study could have been different from the students in the traditional classroom in that they may have been more highly motivated. Community colleges tend to attract a different clientele than four-year universities. However, the community colleges in this study attract a similar population, since most students at all the institutions come

1. Quigley, A. Res. Dist. Educ. 1989, 1, 3–5. 2. Mikandawire, D. S. J.; Jere, D. R. Stud. Educ. Eval. 1988, 14, 139–146. 3. Barker, B. O. J. Rural Small Schools 1986, 1, 4–7. 4. DeChenne, J. A. Educ. Technol. 1980, 20, 55. 5. Hausman, R. M.; Housman, K. K. In Rural Special Education for the New Millennium; Proceedings of the 19th Conference of the American Council on Rural Special Education (ACRES), Albuquerque, NM, Mar 25–27, 1999; pp 347–354. 6. Burke, K. A.; Greenbowe, T. J. J. Chem. Educ. 1998, 75, 1308– 1312. 7. Seitz, V. J. Home Econ. 1988, 80, 42–44. 8. DeLoughty, T. J. Chron. Higher Educ. 1988, 34, 15. 9. Glisan, E. W.; Dudt, K. P.; Howe, M. S. Foreign Lang. Ann. 1998, 31, 49–66. 10. Fulmer, J.; Hazzard, M.; Jones, S.; Keene, K. J. Prof. Nurs. 1992, 8, 289–294. 11. Sonner, B. S. J. Educ. Bus. 1999, 74, 243–247. 12. What’s the Difference? A Review of Contemporary Research on the Effectiveness of Distance Learning in Higher Education; Institute for Higher Education Policy: Washington, DC, April 1999. 13. Pribyl, J. R.; Bodner, G. M. J. Res. Sci. Teach. 1987, 24, 229– 240. 14. Small, M. Y.; Morton, M. E. J. Coll. Sci. Teach. 1983, 13, 41–43. 15. Rixse, J. S.; Pickering, M. J. Chem. Educ. 1985, 62, 313–315. 16. Guay, R. Purdue Spatial Visualization Test; Purdue Research Foundation: West Lafayette, IN, 1976. 17. SPSS, Release 9.0.0; SPSS, Inc.: Chicago, Il, 1998.

JChemEd.chem.wisc.edu • Vol. 78 No. 8 August 2001 • Journal of Chemical Education

1125

Analysis of a Distance-Education Program in Organic Chemistry

Recommend Documents