Paper-and-pencil Tests for the Laboratory' B. CLIFFORD HENDRICKS University of Nebraska, Lincoln, Nebraska
T
measures of achievement in chemistry fail to show any consistent or marked advantage of individual laboratory instruction over demonstration or lecture methods." In his review*of the Cooperative College Chemistry Testing Program for 1938-39 Dr. Martin presented a chart of the relation of success in that test to the number of hours per week of laboratory work in the instructional program of the students tested. Judged from that graph four hours of laboratory work per week produced less successful students in college chemistry than three, and six hours per week of laboratory work had less effect upon the students taking it than five hours per week. The inconsistency of these results does not inspire confidence in this test as a measure of laboratory outcomes. Thus it might appear that the refusal of both teachers and deans to accord laboratory instruction its proper recognition is due to the failure of the testing program. If practical and valid tests were available which would isolate and evaluate the distinctive contributions which the individual laboratory part of our teaching program makes to the total achievement in the course, the situation described in the first two paragraphs of this paper could be corrected. Horton's investigation6with his high-school students in 1928and that of Smith and colleagues' in 1936 agreed that laboratory performance tests do revcal outcomes from individual laboratorv work whicl~are distinctive and objectively ascertainable. Such tests might, on first thought, appear to solve the problem. However, laboratory performance tests are time consuming. The best that either study could attain was five or more minutes of teacher time per student examined. The disruption of the time schedules of teacher and students, and of stockroom service, makes such tests impractical. Three qualities are desirable in tests: objectivity, validity, and practicality. The pencil-and-paper tests are acce~tableso far as obiectivitv is concerned; the laboratory performance tests are. valid measures of Prepared for the 1945 meeting-in-print of the Division of laboratory outcomes but are not practical for routine Chm&l Eduration. American Chemiril Society. ' AnAws, C. S.,"The imponancc of laboratory work in g~neral school use. It is necessary to improve either the collwe chemistrv." I. CIIEI. L'DLT.. 19. 269 (1942,. a HENDRICKS,-B. ?LIPFORD, ',HOW we temh fr&hman college practicality of the laboratory performance tests or the validity of the pencil-and-paper tests for laboratory chemistry." ibid., 12, 1191 (1925). ' ADAMS,C. S., 1 0 ~~. 3 p.. .268. purposes. The purpose of this paper is to follow the KNOX, W. W.. ''The demonstration method of teaching second of these two alternatives. Even if the resulting 13, 166 (1935). chemistry," J. CHEM.EDUC., HORTON, R. E.."Measurable Outcomes of Individual Labo- written test has a lower validity than that of the perratory Work in High-School Chemistry," Bureau of Publication formance test it may have other assets that make such Teachers College, Columbia University, New York, 1928. ' BORROWS, J. AUSTIN. "Methods of Laboratory Instruction." a reduced validity tolerable. EACHERS of chemistry must have a low estimate of the laboratory as a teaching adjunct of their courses. Otherwise why do they assign the job to their most inexperienced teachers? They have little confidence in their estimate of its contribution to the course's achievement, and so allow its grade to count but 26 to 30 per cent on the total course mark,2 this in the face of the general practicea of scheduling 50 per cent of the instructional time for laboratory. Deans and their administrative committees are critical 6f laboratory schedules, generally favoring the demonstration plan of instruction. This preference is quite understandable when the hour-for-hour cost4 of laboratory instruction and equipment is about double that of other courses' in the cumculum. The administrators seek to minimize this inflated cost of laboratory subjects by appraising laboratory teaching time as worth no more than half weight in computing the teacher's instructional load. This discount of the laboratory as a part of our instructional program is doubtless traceable to the failure of its proponents to produce objective evidence that the laboratory actually makes a distinctive contribution to the achievement of the courses to which it is appended. The studies that have been made in an effort to get such evidence have, almost without exception, been with high-school students.6 The evidence, so secured, has generally been favorable to demonstration teaching rather than individual laboratory work. One of these studies6 called attention to the failure of previous investigations by reason of the inadequacy of the tests used. One of its conclusions is: "Written tests are not valid for detecting differences in method$ of laboratory work." Dr. 0. M. Smith and colleagues a t the Oklahoma A. & M. College made a study of three methods of teaching chemistry: individual laboratory, demonstration, and the lecture method divorced from any laboratory or demonstration helps.' After a twoyear investigation, they also stated: "Paper-and-pencil
Unpublished monograph, Oklahoma A. & M. College, Stillwater. 1936.
8
Martin, F. D., J. CHEM. EDUC., 17, 6 (1940)
A more detailed examination of Horton's report0 reveals a test form which he calls a "test of chemical judgment." It is a pencil-and-paper test and its use, as shown by his tables, detects a real difference between routine individual laboratory achievement and that from what he calls the "problem method of chemistry instruction." Similarly, an inspection of the "laboratory written test" scores from the Oklahoma study shows, for students with scaled intelligence scores of 600 to 960, that there is a differentiation between the individual laboratory and the demonstration methods of instruction, as is also shown by the laboratory performance tests. Dr. Theodore Ashfordlo made an analysis of the answers of students to the 1942 Cooperative College Chemistry Test. As a part of that analysis he computed the validity indexes for the total score of the items for section H, lahoratory technique and procedure, by using groupings obtained by scores upon other sections of the test. The results are given in Table 1. TABLE 1
Volidih Indc. Bosirfar Grouping
for Scclia" H
Seetion. A and B (Information) Sections C and D (Applicntion of principles) Sections P and G (Scientific method) section E (Equations and problems)
0.29
section
G
(Theory)
Section H (1,aboratory) Test as a whole
0.20
0.18 0.23 0.20 0.47 0.32
Test technicians tell us that the higher the validity index the more valid that test is for obtaining a measure of that which is being measured by the base used for grouping. From this table it appears reasonable to conclude that section H is ranking students with reference to something measurably different from that of other parts of the test. Wbatever this "something" may be, there is encouragement for further trial of the paper-and-pencil tests of lahoratory achievement. Knox" bas stated the problem to be solved very well: "A measuring instrument must be devised for determining the educational product unobtainable without the laboratory method." In the light of his statement laboratory objectives need to be screened for those whose attainment is impossible or definitely limited "without the laboratory method." Horton concluded his study by saying he could determine "differences in ability (1) to manipulate apparatus, ( 2 ) to make experiments involving the use of apparatus, (3) to solve perplexities or projects involving use of chemical facts in lahoratory situations." If his lead is followed the test desired will probe for those abilities which have to do with apparatus and chemicals and the solution of difficulties bv a llaboratorv situations. Would it , a ~ ~ e to be an oversimplification to impose upon each objective the requirement that i t either express chemistry in terms of chemicals and equipment or that chemical substances and equipment be required to have chemical
-
HOKTON. R. E., loc. cit., p. 83. '"ersonal communication to the author. KNOX.W. W., loc. cit., p. 171.
meanings? One experienced teacher of chemistry says, "The lahoratory trains one to understand the language of test tubes." The statement of objectives in a form to be used readily in making test items requires that they should be phrased in terms of student behavi~r.'~. l3 For lahoratory objectives the statements should be in words of the behavior of students in the laboratory, either in person or by self-projection. In consequence the outcomes to be rated will be in t m s of chemicals and apparatus and stated in a manner to involve a lahoratory setting or situation. From an unpublished list of "objectives and outcomes" prepared1&the summer of 1941 the following rephrased objectives are selected. They are stated in the language of behavior and they try to illustrate the use of apparatus or the "laboratory situation" in a paper-and-pencil test. With each, an' illustrative test item is given with its validity index.'= Laboratory objective: Can the student read a measuring device with required precision? Test item : The volume of the liquid as read upon the buret diagrammed below is: 1. 30.0 ml. 2. 29.0 ml. 3. 25.4 ml. 4. 29.5 ml. Validity index of this item was found to he 0.34."
Laboratory objective: Has the student skill in estimating amounts by weight, volume, or othenuise? Test item: Recalling that your graduate used in the laboratory has a volume of 25 ml.. estimate the volume of an ordinary teaspoon: 1. 1.0 ml. 2. 2.0 ml. 3. 10.0 ml. 4. 4.0 ml. Validity index: 0.29.
I' ADAMS.C.S., ZOC. cit., p. 267. '5These validity indexes were computed by use of 47 associated items from a one-hour laboratory test. '8 Items with a validity index of 0.20 or above are rated as useful contributors to thhtest as a whole
L a b o r a t o r y objective: Can the student bring the proper pieces of apparatus together for correctly doing an assigned laboratory task? Test item:
2. Glycerin, CzH6(OH)-b. p. 290°C. 3. Water, H.O--b. p. 10O0C. 4. Carbon tetrachloride, CCII-b. p. 190'C. Validity index: 0.25.
Which of the following operations will require the greatest number of pieces of apparatus for its performance? (Count a l y the pieces of equipment that are absolutely essential for the procedure suggested.) 1. Crystallization 2. Distillation 3. Sublimation 4. Measuring a liquid's density Validity index: 0.35. L a b o r a t o r y objective: Does the student know how to care for apparatus and keep it i n usable shape? Test item: Sugar was heated in a pyrur test tube until only carbon remained. The deposit of carbon can be most readily removed by: 1. R u b b i g with steel wool 2. Treating with aqua regia 3. Boiling in nitric acid 4. Washing with alcohol Validity index: 0.35.
Does the student recognize and anticipate the hazards of the experiments which he performs? Test i t e m : L a b o r a t o r y objective:
Which of the following experiments is most likely t o result in a serious accident? 1. Passing an electric spark through pure hydrogen in a tube 2. Pouring concentrated sulfuric acid slowly into water 3. Heating a mixture of charcoal with potassium chlorate 4. Heatine " solid carbon dioxide Ydrv . - ice") over an ooen flame Validity index: 0.38.
Laboratory objective: Can the student correctly choose the apparatus and the proper drying agent for gases? Test i t e m : When ammonium hydroxide, NH40H, is heated it is decomposed into water and ammonia. NHI. If ammonia is desired from the reaction the water will have t o be absorbed. T o accomplish this absorption which of the following pieces of appa. ratus will he best for holdine the absorbent? 1. Pneumatic trough 2. Test tube 3. Liter flask 4. A calcium chloride tube or a U tube Validity index: 0.50. L a b o r a t o r y objective: Can the student correctly select apparatus for processing substances i n terms of their diffwing heat constants? Test i t e m : Some IIOs and 1, (both solids) are mined together. I t is desired t o separate the free iodine from the oxide. It is known that IIOs decomposes a t 300DC.and that I*boils a t 1 8 3 T . I t is decided to nse a ? ? ? bath t o accomplish most effectively the separation without loss or decomposition of the 120s. Which of the followingshould be put into B, the beaker of the bath, t o do the separation? (Note: The boiling paints are given after the substances.) 1. Stannous chloride, S n C l p b . p. 623'C.
Laboratory objective: Does the duden1 correclly anticibute the effect of the ckanee i n conrlih'onc uPon the outcome of his exfieriment? Test i t e m :
-
When the pressure in the apparatus, diagrammed below has became great enough t o make the carbon dioxide gas bubble through the water in the test tube it will have pressed the acid back up into the thistle tube t o the level marked by: 1. A,2, B,3.
c,-
4. D,-
Validity index: 0.34.
tb
SOLUTION
L a b o r a t o r y objective: Does the student recognize the effect of the liquid of his pneumatic trough upon the purity of the gas collected over it? Test i t e m :
Some concentrated hydrochloric acid was poured into a test tube until the tube was partly filled and then thoroughly shaken. After the shaking it was completely filled with the acid and inserted in the beaker. as shown in the diagram on page 546. The upper end of the tube was then heated until the liquid had dropped as shown in the diagram. After the heating, the space S above the liquid contains: 1. A mixture of hydrogen, oxygen, and water vapor 2. A vacuum 3. A mixture of water vapor and air 4. A mixture of air, water vapor, and hydrogen chloride gas Validity index: 0.47.
It is assumed that correct answers to such items as these require ability gained from the laboratory rather than from other activities. There is no very objective evidence for this assumption. It is recognized that a validity based upon the scores of a laboratory perfomance test covering the same objectives would be much more satisfying and is definitely the next step in the development of a pencil-and-paper test for laboratory achievement. However, the formulation of items that have creditable validity ascertained by groupingsfrom scores upon the test as a whole will need to precede the more difficult validation through the assistance of the
laboratory performance test with its ensuing advantages. If a pencil-and-paper test can be developed that will have only a tolerable validity, it will help to determine what our chemistry teaching program is doing. To be specific, if we knew with some certainty just what our laboratory is doing for our students we could revise its procedures with more confidence and eliminate useless parts. A valid laboratory test would aid the teacher in convincing the student, who considers that a fat notebook is evidence of laboratory achievement, that an understanding mind is more important than a full notebook in success. And what a boon for that transfer student, whose previous laboratory course is an uncertainty, if the teacher could require that he demonstrate his laboratory achievement by taking a standard laboratory achievement test whose validity is accepted! A similar service for repeaters in college chemistry courses would probably reduce the number of those who are excused from laboratory during the "second exposure." It is also conceivable that a test might be prepared of such dependability that its score would reveal parts of a laboratory course that a given student could delete from his laboratory program.
