Field Test Evaluation Report on Introduction to Polymer Chemistry Kenneth Chapman and Janice Fleming Project CEDS, Department of Educational Activities, American Chemical Society, 1155 Sixteenth Street, N.W., Washington, DC 20036 Project CEDS (Continuing Education Delivery System) is a research and development effort of the American Chemical Society with funding from the National Science Foundation. The project's focus is two types of continuing education courses: one using audiovisual materials and the other using interactive computer programs. Investigating the distribution of these courses is another important feature of the project. Manv factors contribute to limitine adult oarticioation in traditional continuing education programs. They include demands on the time and the diversity of backgrounds and experiences of the clientele. Methods that are being used by Project CEDS for minimizing these inherent problems of continuing education include augmentation of texts and self-study printed matter via computer andlor audiovisual technolog;so that users can participate more actively in the courses and can benefit from visual presentations not feasible otherwise. Instruction using these methods stresses individual control over course content, depth of study, and rate ofstudy. Also, the choice of convenientstudy conditions for the individual is permitted. Evaluation of the course, "Introduction to Polymer Chemistry," by Dr. Harry Gibson and Dr. John Pochan with comouter orogram develooment hv Dr. Stanlev Smith. is the suhjkct oith; paper. his course was field-tested via the PLATO" comouter svsteml in the sorine of 1980. Extensive . course evaluation techniques based on previous Project CEDS evaluation exoerience were emoloved durine the field trial. "~ntroduct~on to Polymer chemistry" u h e d user-controlled interactive computer lessons and computer tests supplemented with self-study textbook readings to teach the fundamentals of polymer chemistry. The text used was "Polymers: Chemistry and Physics of Modern Materials" by J. M. G. Cowie (1973). The computer lessons included diagnostic pre- and post-tests, interactive readings, rrview questions, and simulat~dexperiments. A srudv~uideprovidpd the basics of usinr the PLAT0 s w e m and interfaced the romputer lessons with textbook readings. The user could choose hislher own path through the computer's lessons, accessing as many of the sections as desired; however, a recommended order of lesson selection was provided through the course menu. Provisions were made so the user could go forward or backward in the course material as desired. and could never become "stuck" without help in any of thesections. Test sites for "Introduction to Polymer Chemistry" consisted of two ACS local sections and five industrial sites. Persons participating in the course evaluation included chemical technicians, chemists, supervisors, managers, and some non-chemistry oriented workers, with all levels of academic degrees. All aspects of the course were reviewed by the evaluation process. The sequence of evaluation for the course was
-
1. Completion of a workleducational background form and an
rxperted benefits form. 2. Pre-tet i m poiymrr chemistry (laken on the computer). 3. Coml~lttionOI lesson r e a d s (indicating pn,l,iemr encountered with the course),a course evaluation fom, and a perceived benefits form. 4. Post-test on polymer chemistry (taken on the computer). 904
Journal of Chemical Education
In addition to the above information the computer system recorded the time soent on the comnuter lessons. - ~ ~ lessons ~- ~~ ~ ~ started or completed,'and messages wriiten by the course users to CEDS staff. At the conclusion of the course. on-site interviews with many of the course users were cond&ed by cEI% staff. Edited typescripts of these interviews are available. ~~
~
~~
Evaluation Methods and Materials "Introduction to Polymer Chemistry" was offered for evaluation purposes from 28 Februarv to 14 Mav 1980 to individuals a t the test sites. Some of the.users had access to the course lessons via PLATO terminals that were "in-house" at industrial locations, while others used Control Data Learning Centers where PLATO terminals are made available by appointment. All course registrants were required to complete and return two evaluation forms prior to accessing any of the computerized course materials: 1. A User-Eoaluator Background form, which indicated the individual's educational training and work experience. 2. An Expected Benefits form, which permitted the individual to indicate his expectations with regard to the course and the relative importance of those expectations. Next, a pre-test on polymer chemistry was administered by the computer to each person who began computer interactions. Course material could not he accessed until the pre-test was completed. The computer graded the completed test at the user's request and immediately presented the score and a diagnostic statement of weaknesses. The individual was not allowed to see the specific items missed, nor the correct answers to each item. The computer stored results by item and by total test results and the time (in min) that the user spent taking the pre-test. Of the 125 individuals who registered for the course, 109 (87 percent) began computer interactions, as shown by their having pre-test scores. During the comouter lesson use period, a number of evaluation activities w&e ongoing simulianeously.Lesson use data on every individual accessing the cornouter materials, and the amount of time spent on the comp&r were stored by the computer. In addition, the users had the special option of sending notes over the computer system to CEDS staff and the course producer. With the touch of a few keys, the course user could freeze hisher place in the programmed lesson, send a note, and then continue the lesnm from that point. The (.'El% staffchecked the comouwr-usestatistia and usernotes on a daily basis during the fikld test period. By doing so, some of the more severe oroblems with course roe ram mine were corrected before tl;ey posed a prohlem io i l l course'kers. Heturn messages - were sent from CEDS staff to mers who had written notes. The second evaluation activity during the computer use
Resented at ttm National Meeting of ttm American Chemical Sdmty. March 1981. ' A network based upon a Cyber 70 Mainframe computer and a terminal designed specificallyfor educational purposes. PLAT@ is a trademark of the Control Data Corporation.
,~
~
period required lesson records that the user completed each time the computer was accessed. These forms helped to identify what, if any, environmental problems or distractions the user exoerienced durine hislher comouter interactions. There was also space on thelesson record for the user to indicate how much outside reading or other preparation helshe had done prior to accessing the computer lessons that day. Of the 109 individuals who began the computer interactions, 95 (87 percent) returned a t least one lesson record. After completing the lesson materials to hisher satisfaction, the user wasahle to access the post-test on polymer chemistry. In this field test, the post-test was identical to the pre-test with the exception that the questions and their respective multiple choice answers were presented in a new random order. After completing the test, the user's score was presented by the computer, along with a diagnostic statement of sections in the course to review, depending on the item responses. At this time the user was allowed to see what items were answered correctly or incorrectly and to see the correct responses to each of the questions. The individual could take the post-test only once. As with the ore-test, the comouter stored the scores, item responses, andtime spent takingthe post-test. Of the 109 individuals beeinning the comouter interactions.. 96 (88 . Der& cent) took thebost-tist. After completing the course materials, the course users were asked to complete and return two final evaluation forms. These forms were 1. A Post-course Benefits form, which allowed the individual to judge whether the pre-course benefits had been achieved. 2. A Course Eualuation form, in which the individual answered questions on the usefulness of the course, the course conditions, the level of effort required for the course, and the evaluation activities.
The final evaluation activity was that of interviewing some of the course users a t their work sites. CEDS staff used focus erouD technioues that allowed discussion and interaction Let;een the members of each group. An attempt was made to select the erouo oarticioants according to their ioh levels andlor functrons 8; that [he interactions-within each group would be among peers. D a t a Analysls
UP to 200 data items were collected for each individual c
