and
M i l e s Pickering' Princeton University Princeton, NJ 08540 R o b e r t H. Crabtree Yale University New Haven, CT 06520
I I
How Students Cope with a Procedureless Lab
M a n y chemical educators conceive t h e ultimate s t u d e n t experiment t o be one in which t h e student is asked to measure x , devising his own procedure, a n d t h e n is graded o n how good his answer is. T h e tradition of grading rm results is an old a n d almost unexamined one. The few previous statistical studies (1, 2) have concentrated o n practical e x a m s with complete directions available to thestudent. Nostudies have been m a d e of s t u d e n t strategies in experiments without specified laboratory procedures (3,4 , s ) . Since real research requires t h e ability t o extend o r modify a n old method t o fit a new problem, t h e r e is a n i m p o r t a n t pap in o u r assessment of scientific competence. his paper i e p o r t s a detailed analysis of a n open-ended practical examination. T h i s s t u d y has led t o some surprising conclusions a b o u t how s t u d e n t s cope with procedureless situations a n d how t h e y interact with laboratory "recipes" in general The Test To carry out this type of "research ability" exercise, one must choose a measurement that is simple enough to do quickly with elementary apparatus and yet requires some flexibility of method on the part of the student. The measurement has to he quick enough to permit the student t o refine his methods and get a "hest"value in a single period. The general nature of the problem should be known in advance, su that the student can develop his strategy before coming to the lab and also so that suhsequent sections taking the test do not have an unfair advantage over the first section. Finally, any apparatus has to be simple and foolpruof. For these reasons, the problem chosen was the measurement of the heat' of solution of an unknown salt. In December 1977 the test was run in Yale's freshman physical chemistry lab course by a group of about 30 students, a particularly elite group, since advanced placement in chemistry is required. Few uf these students had math SAT'S below 700. Most take a lecture course in physical chemistry simultaneously with the lab. The exam was not completely procedureless. Students were given a method to determine the heat capacity of the solution resulting from the dissolution of the salt, a table of the ranges of molecular weight solubility, and enthalpies of solution to he expected, and a sample calculation. Hut there was considerable room left for student ingenuity in terms of container insulation, choice of sample size, temperature extrapolation, and choice of conditions. Students were not permitted to discuss their procedures with their TA's or with the professor in charge (RHC). During the exam the usual practical exam formalities (silence, no collahoratkm) were observed. At the end of the period, all the procedures that students had prepared and the test books that served as substitute notebooks were turned in. Mathematical errors were corrected before the results were graded. About a third of the class reported results within two percent of the true value. A postmortem analysis of the written material handed in by the students was performed hy MP. Thisauthor has never met any of the students involved, so the conclusions are not tainted by the "halo effect." Discussion a n d Analysis In order to determine if there is a correlation between how well students performed and how thoroughly they had prepared (a difficult variable to measure), a word count was made on each student's prelab write-up. There was s wide variability of ward count, as shown in Table 1, but no significant correlation emerged. In fact, two of the "owest oerfurmers had the kmeest orelab write-uos. 'The absence of
amined, then twogroupscan he distinguished broadly. One group of students i"emoirieists") built feedback kroos into their orocedures.
make the differences clearer for the reader, quotes from student procedures are assembled in Tahle 2. These groups are not totally distinct. There is also a borderline gmup ( 3 students) who used procedures with vague or irrelevant feedback. Some feedback adjustments prepared in advance do not aooear . . in the notebook transcriot recorded durine" the exam: the student foreot his mod intentions under oressure. Some checks mav h.wr hcen d < m ~ n . that ~ o d.. ntnplrrnr in the wrllren materials At'ter rll, even chc.u..mr: "1x.t" v.dur. wlhtr rhnn d m d wrmamg. i s feedhack of a sort. The scores of the two groups are compared in Tahle 3. The "empiricists" did much better un the practical exam than the "dead reckoners," with the borderline cases in between. Because of the skewness of the grade distribution on the practical exam, no significance tests were performed un these scores. However, the pattern on an ooen-hook written exam in the same course showed theemoirieists
.
~
~~
~~
beyond the practical exam itself. Even in this elite class, the dead reckoners were in the majority. Their pmcedures gave some cluesas to why they chase thisapproach. These students seemed to he concerned primarily with not forgetting some crucial measurement. Characteristically, they made checklists, underlined steps where data was to berecarded, or made blank tables to be filled in. Usually "empiricists" calculated results as they went along. The "deed reckoner" would do it a t the end of the period. The "dead reckoning approach" is, of course, close to the "cookhook" idea. These students, when gi&n a totally unstructured lab situation, responded by developing a "recipe." It may be that this is the only way many students can cope with lab. Just doing the meosuremrnts requires all the auailable mental energy and attention. The student cannot simultaneously think about how to improve his Length-on Test Grade Table 1. Effect of Prelab Write-up Word Count
Average Grade
0-99 100-150 150-199 120- m
45 4 44 3
Table 2.
N --
5
48 6
9 5
41 2
7
Ouoles from Typical Student Papers of Various Types-
Empiricist:
Borarr ne
Dead Reckoning:
"If temperature jump is very small (
