A. C. Vandenbroucke, Jr. Wortburo* Colleoe W o ~ e y, r lono 50677 "
I I
Evaluation of a Personalized instruction System in General Chemistry
Warthurg College is on a 4-4-1 calendar. This means that a student normally takes four courses each of the first two terms and one course during what is called the May Term. The General Chemistry program is sequential and covers all three terms. This discussion deals only with the Fall and Winter Terms, which are each 14 weeks long. The course has been divided into two sections, mornim. (A) and afternoon (B), for a number of years. This is the third w a r of our exploration of alternativesto the standard ~eEtureapproach to general chemistry. Two years ago we used a programmed text "Chemical Principles" by Runquist, Creswell, and Head in one section (B) while using the second edition of "Principles of Chemistry" by Masterton and Slowinski in the other section (A). The section using the programmed text had two lectures and two help-sessions per week. Encouraged by this experiment and the works of Keller ( I ) , Koen ('4, and Green (3, 4) we initiated the Personalized System of Instruction (PSI) last year in the B section. Both sections used the fourth edition of "College Chemistry" by Nehergall, Schmidt, and Holtzclaw. The students were informed before registration that the B section would use PSI. As usual the A section had four 50-minute lectures per week and each student was evaluated on the basis of his or her best four out of five hour tests, a two-hour final examination a t the end of the term, and the lahoratory. The B section used the PSI approach with three 50-minute testing periods and one lecture per week. Paid upperclass chemistry majors were used as tutor-graders to aid the instructor in the course. The one lecture per week initially was an attempt to expand and enrich the text hut because of low attendance and student questions it was converted to a help-session the second term. This year it has become in effect a fourth testing period. Evaluation of the students in the B section was based on their progress through the 20 units of material, a two-hour final examination when the student had completed the 20 units, and the lahoratory. Ninety percent mastery was necessary for the passage of a unit. (For a more detailed description of PSI see Lewis and Wolf (5) and references therein.) Each section covered the same material hoth terms, chapters 1-17 and 19-39 (omitting 27) in the Fall and Winter terms, respectively. The students in both sections took the same final examination a t the end of the first term and a t the end of the second term they took the ASC-Cooperative Examination in General Chemistry. In the laboratory no distinction between the students in the two sections was made. The laboratory consisted of a weekly three-hour session: 12 separate experiments the first term and qualitative analysis, six unknowns, and five tests, the second term. ~
~
being juniors. Medical technology and physical therapy are the two areas that a majority of students intend to pursue when they enroll in General Chemistry. The remaining students have interests in biology, pharmacy, medicine, and chemistry. Tahle 1 compares the American College Testing (ACT) scores for the eight sections of General Chemistry taught during the last two years. From this it can he seen that when comparing the lecture (A) and the PSI (B) sections for a given term or a section with the mean of the whole student hody only two areas stand out. First, the mean ACT scores in mathematics, social studies, natural science, and the composite of the PSI sections (111B and 112B) are markedly higher than those of the lecture(A) sections in 1972-73. Second, there are the natural science scores of the PSI sections in 1973-74 which are markedly higher than those of the lecture sections (111A and 112A) that year. It should he noted that Tahle 1 also shows that for all four lecture (A) sections hut for only one of the PSI sections (112B, 73-74) the mean ACT scoresfor students receiving No Credit or withdrawing are below the college student hody mean scores. This difference is mainly due to the problem of procrastination among students in the PSI sections. Tahle 2 shows that the students that received No Credit in the lecture (A) sections had completed all the work in the course or all but the final examination Of the students withdrawing from the lecture sections the majority completed 50% or more of the course work. (This is possible because Warthurg has a policy that allows withdrawal up to the last week of the term.) In the PSI (B) sections only one of the students receiving NC or W completed 50% or more of the course work. Of the 16 students receiving No Credit or withdrawing from General Chemistry in 1972-73, 10 have retaken the course and all 10 have passed. Table 1. Comparison of the Mean ACT Scores of Students Receiving CreditandThose Receiving Nocreditorwithdrawing from General Chemistry
-
1972-73
No. Stud. English
Math. Soc.Std. Nat.Sei. Comp.
l l l A Credit NC or W l l l B credit NC or W 112A Credit NC or W 112B Credit
'Description of Students
Freshmen account for between 40 and 50% of the enrollment in hoth sections of General Chemistry. Thirty to 45 percent of the students are sophomores with the balance Pzesented in part at the 1st National Conference on Personalized Instruction in Higher Education, Washington, D.C., April, 1974. 516 / Journal of Chemical Education
Six deleted because t h e i ~ACT seoresarenot available
Table 2. Work Completion Record of Students Receiving No Credit, NC, in or Withdrawing, W, from General Chemistry 1972-73 111A' 1. N C ABFL 2. N C AUG 3. N C All* 4. w ABFE 5. W ABF' 6. W Nonea 7. w NO"@ o ABF =
111Bb N C 1. N C 13d W '6
w W W
1973-74 112A'
11286
IllAa
lllBb
112A.
112Bh
W ABF'
N C Sc N C 0'
N C AU N C All N C ALL NC AU W 50% W AU W 50%
W1
W 50% W 15%
NC 4 W S W 2
4r 4" 40
AU but the Final.
Testing and Performance
Table 3 gives the scores on the two-hour final examination a t the end of the first term. The maximum score on these tests was 200. The scores for 1971-72 are included for sake of comparison. The B section that year was taught using a programmed text as described above. Because of the different texts used that year only 17 of the 20 questions on the final were identical. There is not a significant difference in the performance of the lecture (A) and the PSI (B) sections on the final examinations a t the end of the first term. Tahle 4 sets out the oerformance of the A and B sections on the ACS-Cooperative Examination in General Chemistrv. In addition to 1971-72 the scores from 1970-71 are inclided. In 1970-71 General Chemistry was not taught by the author and both sections had the lecture approach from the same instructor and with the same text. The number of correct answers is included because the Raw Score is determined by using a factor to correct for guessing (Raw Score = Right - Wrong13). There is no direct penalty for not answering. Unlike the values given in Tahle 3 the PSI (B) section in 1972-73 scored significantly higher on this test. Going hack to the discussion of the differences in ACT scores between the sections (Tahle 1) it is possible to propose an explanation for these test results. As stated above, the PSI (B) sections in 1972-73 did have definitely higher scores on most of the ACT tests. Why, therefore, is the mean score of the PSI (B) section on the 111 final (Table 3) about the same as that of the lecture (A) section, while the mean score on the ACS-Cooperative Examination is higher for the PSI (B) section (Tahle 4)? This can he attributed mainly to mathematical ability as measured by the ACT test. The difference in the mean ACT mathematics scores is 1.3 for l l l A and l l l B while it is 2.6 for 112A and 112B. In 1973-74 the differences in the mean ACT mathematics scores between the lecture and the PSI sections are 1.0 and 0.2. To support this explanation the Pearson product-moment correlation coefficients (r) between the five ACT parameters and three variables were obtained. For .the eight sections the correlation coefficients between the five ACT parameters and final examination scores and total points earned in the course were determined. Using data from the four 111 sections the correlation coefficients between the ACT scores and cumulative grade point averages were calculated. In all 20, correlation coefiicients we& determined for each ACT parameter (See Tahle 5). The ACT mathematics scores correlate more frequently and with a higher level of confidence with the three variables than do the other ACT scores. The scores on the final examinations in 1973-74 (Tables 3 and 4) are consistent with this explanation. While the ACT natural science scores are markedly higher for the PSI (B) sections the differences in the ACT mathemathics scores for the A and B sections are small. Because the cor-
Table 3. Mean Score on the Final Examination after the First Term of General Chemistrv lllA (Lecture)
Year
No. Stud.
lllB(PS1)
No. Stud.
M e a n acme on 17of 20 questionson the test.
Table4. Number of Correct Answers, Raw Score and Percentile on the ACS-Cooperative Examination in General Chemistry (Form 1970) 112B (PSI)
112A (Lecture)
year
NO. cor.
NO. %tile Stud.
RS'
NO. Cor.
RSD
NO. %tile Stud.
'In addition to General Chemistry, Wartburg Couege offers another first course in chemistry, but at an s d v a n d level. These atudents a h take the ACS-Cooperation Examination a t the end of the m n d term. If these scores are combined with those of the General chemistry group the mean raw more compares favorsbly with the national normsfor this test. Table 5. Number of Times the Pearson Product-Moment Correlation Coefficients ( r ) between the ACT Parameters and Final Examination Scores. Total Points Earned in the Course. and Cumulative Grade pdint Averages Reach the0.01 and the 0.001 Confidence Limitss ~
~-
ACT Test
~
~~
~
0.01
~
~
Confidence Limits 0001
Totar
a Fisher, R. A,, and Yates, F., "Statistical Tables for Biologiea1,~Agrieulturd and Medical Research," 5th Ed.. Hsfner Publishing Co. Inc., New York, 1957. p. 59. q m n t y mrrektions determined for each ACT parameter
relation of the final examination scores with the ACT mathematics scores is higher than the correlation with the ACT natural science scores, we would expect there to he little difference between the final examination scores of the PSI (B) and lecture (A) sections in 1973-74. There is no significant difference in these final examination scores. We conclude, therefore, that the higher score on the ACS-Cooperative Examination by the PSI section in 1972-73 is attributable not to the method of instruction hut to the ability of the students in this section. There is some indication that retention improves with the PSI approach (fi). Let us consider the performance of both the lecture and the PSI groups on the material from the first term covered by the ACS-Cooperative Examination. Of the total of 60 questions on this test 23 of them dealt with material that was covered exclusively in the Volume 52, Number 8, August 1975 / 517
first term. In 1972-73 the PSI group had a mean number correct of 10.0 (Raw Score = 7.1) while that of the lecture group was 8.1 (Raw Score = 6.0) on these 23 questions. In 1973-74 the PSI group had a mean number correct of 9.0 (Raw Score = 6.8) while that of the lecture group was 8.6 (Raw Score = 6.6) on the same 23 questions. These differences do not approach a significant level. Therefore, it seems that retention of the material from the first term is about the same for hoth groups. Since, however, the students have the option of not answering on this ACS test without a direct penalty the case concerning retention is not clear from this study. Because of the small differences in these scores the differences in the total scores on the ACS-Cooperative Examination (Table 4) are due to the students' performance on material primarily from the second term. As in the case of other experiments with PSI the percentage of A's and B's given in such courses is much higher than in the lecture approach. Table 6 shows this to also be the case here. The number of withdrawals was high only in the first t e h that PSI wasused.
Table 6.
Number (and Percentage) of Students Receiving Each Letter Grade in General Chemistry
lllA 72-73 73-74
Conclusions
The test results show that the students in the PSI sections are learning a t about the same level as those students of comparable ability in the lecture sections. The difference is that the students in PSI are actively involved in the learning process. If they are not, then there is no progress. Students seem to like and appreciate this. They tell you that they feel that they have learned more in PSI. While the test results in this study seem to contradict this, possibly we are not testing for the kinds of things that the students are learning. They leave the course with a better feeling about chemistry than do students in the lecture sections. More important, the students have a het-
518 /
Journal of ChemicalEdu~~tion
112A 72-73 73-74
112B 72-73 73-74
'One Pabs/No Credit option. *Four Pass/iYo Credit options.
Table 7. Transfer Patterns and Enrollment in the Second Term of General Chemistry 1972-73 No. rseiving credit in l l l A No. transferring from l l l A to 112B No. transferring from l l l B to 112B NO. not continuing with ehemktry No. who did not teke 111
in 1l1B
63 -3 +4 - 10 0
-
No. enrolled in 112A
14 +3 -4 -2 +2
-
54 13 No. enrolled in 112B
1 9 1 1 1.. 4 --.-
Student Attitude
There has not been a significant amount of transferring from the PSI section to the lecture section after the first term. Actually, as is shown in Tahle 7 the numbers of students transferring into and out of the PSI section are about equal. The enrollment in the PSI sections dropped about 7% between the first and second terms hoth years while that of the lecture sections dropped about 14% hoth years. Student response to the course has been very favorable. Even among students that have had trouble with procrastination, most have felt that they were a t fault, not the course. In March of this year a questionnaire was sent out to the 41 students who had enrolled in the PSI sections for either one or two terms in the 1973-74 academic year. Thirty-three students (80%)responded. Table 8 is a partial list of the questions and the student responses. In addition, 67% felt that they would prefer that additional chemistry courses they would take be taught by the PSI approach. Sixty-four percent responded that there was no great need for lectures in this course while 30% felt that occasional lectures or one lecture per week would he helpful. Seventy-six percent of the students responding rated this course ahove or well ahove average, while only 3% rated it below average.
lllB 72-73 73-74
No. receiving credit in l l l A No. transferring from l l l A to 112B No. transferring from l l l B to l l 2 A No. not continving with chemistry NO. who did not take 111
70 -7 +5 -9 +1
in l l l B
-
No. enrolled in 112A
33 +7 -5 -5 +1
-
31 NO. enrolled in l l 2 B
60
Table 8. Responses to Selected Questions from a Questionnaire Sent to Students" Enrolled in PSI Sections During 1973-74 More
Same
Lese No Ans. (%I (%I
Q"esti0"b
(%I
Amount of time spent on PSI course Level of difficvlty of PSI course Interaction with instruetor in PSI
39 13 53
42 39 27
15 39 12
3 3 3
39 70
45 30
12 0
4
(%)
cou-
Level of learning in PSI course Ability to learn on your own after
3
Queationnairessentto 41 atudenta, 33atudent. ~ p o n d e d . In questions 1-4 the students were asked to mmpare the PSI course with another science course taught by the lecture method. For most shldenta this meant a eompariaon with a biology course. a
ter attitude about learning, and learning is the goal of teaching. Acknowledgment
I wish to thank Dr. John L. Craft for his helpful discussions and suggestions during all phases of the work on this course. Literature Cited (1) Ksllcr. F. S.. J. Appl. BshoulorolAnolyxls I. 79i19681. 121 Kwn.B.V., and Koller. F. S.. Lny L d u c . GI. 504119711.
(31 Green,B.A..Amw J. Phw, 39.7M 119711.
14) Green,B.A.. J. Colle#eSc:ci l'eochtng, 1. 50(L971). 16) Lewis,D.K.. and Wolf, W. A . J. CHEM. EDUC.. j0.511L9731
161 Loo,Micah W. M.. J.CHEM.EDUC..50.4911973).
