Mark Albanese, David W. Brooks, Victor W. Day, Roger A. Koehler, J. D. Lewis, Robert S. Marianelli, E. P. Rack, and Carol Tornlinson-Keasey University of Nebraska-Lincoln Lincoln, 68588
II
Piagetian Criteria as Predictors of Success in First Year Courses
I
J. Dudley Herron is to be complimented upon his carefully written article. "Piaaet for Chemists" (I ). Although Piagetian ideas have existed f& many years, only noware they becoming anureciated (2.3). Herron makes a point in his article (I) that , iu of great illtrrest t a g those wncrrned with prohlerns of studcwt rrlacrment. Hr claim; a uorrrlution of 0.8 hc.twern prrformince on a battery of Piagetian tasks and total points earned in one of his chemistry courses a t Purdue. This correlation is of a magnitude to be extremely useful if widely and conveniently applicable to student placement. It is important for the reader to comprehend what is involved when a student performs on a Piagetian task. One such task .- involves eeneration of a color bv mixine - chemical solutions. The student is provided with five colorless, odorless solutions: 1)dilute sulfuric acid, 2) water, 3) dilute hydrogen peroxide, 4) aqueous sodium thiosulfate, 5) aqueous potassium iodide. The person administering the test takes two vessels, one previously filled with water, and the other filled with a mixture of dilute acid and peroxide, and adds a few drops of iodide solution to each. only the second mixture gives a yellow color. After the demonstration, students are asked to mix the chemicals using solution 5 plus any combination of the other liauids. The obiective is to determine which combinations lead toLcolorformation. Chemists immediately recognize that the color forming reaction requires acid a i d peroxide, that water will have no effect, and that thiosulfate will prevent the formation of a vellow , - - ~color ~- ~ Typical results of the experiment are described by Inhelder and Piaget (4). Also, a demonstration of the application of this task is shown in a film by Robert Karplus (5). Piagetian tasks are lengthy to administer. One is concerned not only with the student's performance, but also with any iustification offered on behalf of the prohlem solving strategy $elected. It is almost impossible to conceive of praitical ckcumstances where Piagetian tasks, which always involve
.
1 A comment by Haffner in reference (7) implies that the information developed from SAT scores alone is effective in generating placement recommendations,and that the data gleaned from chemistry and math validation tests offer refinement. Quoting from reference (7), p. 160: "lt (sic,the Academic Composite, a sum of four College Board test scores-math aptitude, math achievement,verbal aptitude, and English eomposition-plus the prior academic achievement score) is, initself, agood predictor of overall grade point average and predicts most course grades fairly well." A recent contribution by Pickering (9)describes use of SAT scores done to predict "maximum" course grades. John Renner of the University of Oklahoma and Glenn Croshy of Washington State Univemity have developed and thoroughly tested instruments which are useful in student placement. Both of these congenial individuals seem most pleased to share their experiences with the academic community at large. T h e analyses of the placement test data were performed by Mark Albanese and constituted partial fulfillment of his requirements for the degree of Master ofArts in the Department of Educational Psychology and Measurements. Detailed discussions of the instrument, subjects, course structures, techniques of analysis, reliability of the instrument, and validity of the instrument may beobtained by writing to David W. Brooks at the Department of Chemistry, University of Nebraska-Lincoln, Lincoln, Nebraska 68588.
Student Results on the Placement TePr
Subscale
12
Characteristic Measured
Number of Items F(dfb, dfel
Task1 Identification of CornllOUndl (Piagetian Task1
7
2.95
Percent of Additional Variance in Course Grade
0.27 21.4
nT~iedo Placement Test, Partr 1-6.
manipulation of some concrete objects such as chemical solutions, can be administered to groups of thousands of students. Therefore.. exnerimenters have tried to devise nencil . and paper tests which "get at" the same measurablevariables ( 6 ) .It is the feeline of manv that students uerform less well when confronted vn'th the and paper Gsts than with the manipulative tasks themselves. Although pencil and paper tests can be evaluated more raoidlv than manioulative tasks.. thev - still reouire that answers b e read carefully, that problem so~ving'strate~ies be evaluated. and that scorine -trv.to reflect the same assessment of the stage of inr14ectunl develupment as would olnain from obwnation during- rnanir,ulativr tasks. Such scoring - is tedious and expensive. Because several of us had some experience in preparing and evaluating written Piagetian tasks, we decided to include such tasks as Dart of a placement test and invest the time and money td determide whether or not we could significantly improve our course placement procedures. (The cost of scoring each student's written responses was ahout $0.50 per stul dent).
Placement Tests Registration of students within multi-track general chemistry courses is traditionallv accom~lishedon the basis of ren~mmendationsof ofarlle~eadmissions officers (using student reu,rds. class rank. SAT \ACT1 srores. etc.) and or ~ e r f o r mance on placement tests: The Division of Chemical Education has long supported the notion of chemistry placement tests, and the Toledo Chemistry Placement Examination was reissued in 1974. Articles in this Journal (7)' and other journals (8)have described the use of placement tests. Many teachers such as John Renner and Glenn Crosby have devoted a great deal of effort to the problem of p l a ~ e m e n t . ~ Volume 53. Number 9. September 1976 / 571
A t the beginning of t h e Fall Semester, 1974, a placement test in zeneral chemistw was administered t o classes a t t h e lin1\.ersity of h'ehraskall.incoln. This rest ronsisred of the 1974 "Toledo" test together with other subsections inrluding two pencil-and-pape; Piagetian tasks. T h e results for 885 students a r e summarized in t h e table. Discusdons T h e purpose of t h e placement test we developed was t o improve upon the olacement advice e v e n t o enterine students h y b u r adkssionsofficers, a n d t o test t h e notion t h a t scores o n written Piaeetian tasks would b e ~ a r t i c u l a r l veffective as course grade predictors. W e have ciearly failed in t h e first reaard, since we are unahle to account for more than 21% of t h e criterion variable (predicted gradeh4 T h e impact of the Buckley amendment on this research is t h a t we a r e unahle t o conveniently obtain a n d correlate t h e data available for each individual student with other data such as A C T score availahle in nersonal files. However. i t is exceedingly unlikely t h a t most of t h e variance we account for is not elsewhere accounted for i n t h e admissions data. T h a t is, i t is unlikely t h a t we are assessing cognitive skill levels t h a t were n o t previously assessed using d a t a available t o our admissions officers.
Seven course sections were studied and, when examined separately, the percent of variance accounted for ranged from 28.1-42.2%. For the two faculty members teaching two sections each, the percentages of variance in grade accounted for were 29.91422% and 28.11422%. The instrument is a better medidor for studentsonce they know which class they are in-and, one could advise students ahout where their grade opportunity would he maximized. This is because each professor had different exams, different grade weighting schemes, and different grade distributions. When all of the courses are lumped tugether, the dnraotrhe rat,lerc~ult. Itefrrenre Id) dc.&br.; rhe usr of the ACS-NSTA Cwprrarivr Exwninnr~on-High School Chrmisrry as parr of a wade predicrion formula. In that study, the test alone &e&nted fo;29% of variance in criterion variable (course made). and the overall remession eauation, including h~phsrhonl ;average, S A T nlnrh, and whool of r e w . rrarron, accwnted for 114,civarmnre. The numbrr uf lreruring inrulty or the number of course sections was not indicated. When indiuidual lecture sections of this study are examined, the best ease is 42% of variance accounted for by the 12 suhscales of the placement test, with 3996 of the variance accounted for in the Toledo Placement Examination. In the worst case, 28% of varianee was accounted for with 21.5% of that variance accounted far by the Toledo Placement Examination. Factor analysis indicated that the Piagetian suhscales factored well within themselves, and did not factor with items on the Toledo Examination. Whatever the Piagetian subscales are measuring does not seem to be highly predictive of performance in any of the seven chemistry classes studied. When the suhscales are rearranged and the Piagetian scales placed first, they account for no more than 5.71% of variance. In this ease, the variance is confounded with that of the other suhscales of the instrument. Hemon (reference (1 1, p. 148) reports a correlation of 0.7 between scores on a "chemistry placement test" and scores on the battery of Piagetian tasks. This is consistent with the notion that whatever placement test he uses gets at some of the same skills as does the hatter" of Piaretian tasks. " V ssvnereism , ~" hetween manioulative tasks and coenitive learnine has bren diicu3urd 1 i f I dnd nnny wrll play an mpwranr nde in arcountin:: Iur the uttcn rxtrrnmr differ~nrrsol,sereed herueen pencil and paper tasks and manipulative Piagetian tasks. These differences are currently under investigation. When the 19 women in the chemistry major's course of this study were considered separately, fully 88% of the varianee in criterion can he accounted for, including 43% from the general knowledge suhscale of the Toledo Chemistry Placement Examination and 13.4%from the Piagetian subseales, 11 and 12. Only the variance accounted for by the general knowledge subseale was significant. 4
~~
It is also noteworthy t h a t our Piagetian tasks account for verv little variance i n course performance. T h a t is. t h e speciail" designed I'iagetian task; did not appear tu a r c o ~ ~ n t . f ~ , r additional variability in criterion score over and atx)ve thnt which axs already accounted for in the suhscdes of the Tultdo ('hemistry Placement Examination.' Rv wnv of explanation we might &gue t h a t the measures used, while not attempting t o specificallv t a p loeical reasoning, d o t a p these Drocesses. In f k , the n~;th~r;s~rtthe ACI'tesr daim rhat rnans;d the wst nvms nre inttm~lrdto mensure skills which we ww~lrlinrerpret n i formal opernt~imnlskills (101.Iirnce, muvh of tht, variimce arcounted for by any Piagetian task should have nt least uartinllv hren ncrollnred for in the ACT !or 5.4'1'1 smres. W e &ill have n o t accounted for t h e hisparity between Herron's results and those reported herein. Several possible explanations may he offered 1) Pencil and paper Piagetian tasks are not as effective as manipulative Piagetian tasks in assessing the formal operational skills of s t ~ d e n t sOn . ~ the other hand, one might expect penciland paDer tests to better reflect the circumstan
