ARE WE TEACHING OUR STUDENTS to DISTINGUISH BETWEEN FACT and THEORY? THEODORE A. ASHFORD
AND
WILLIAM M. SHANNER
The University of Chicago, Chicago, Illinois
..
T
HERE is wide agreement among science teachers that one of the most important objectives of science instruction is the teaching of scientific thinking (I,2, 3, 4 , 5 ) . However, there is no uniformity in teaching methods for the attainment of this important objective. In fact, in most courses, n6 special provision is made to teach the scientific method as such (6, 7, 8, 9, 10, 11). The average instructor probably expects that the student by dealing with science material will somehow develop the ability to think scientifically. The extent to which students develop ability in scientific thinking is a t present a matter of opinion. Thus far, but few attempts have been made to evaluate by means of reliable tests the extent to which this objective is being attained (12,13, 14). The absence of uniformity in teaching scientific thinking, the paucity of studies on the outcome of such teaching, and the lack of conclusiveness of the results of the available studies may he explained, in part, by the fact that there is no general agreement as to the meaning of terms. "Scientific thinking" or "the scientific method" has not been clearly defined. Likewise there is no agreement on the meaning of terms such as fact, evidence, proof, theory, hypothesis, assumption, deduction, and so forth. Whether it is possible to give
.
precise meaning to these elusive terms, is a moot question. It might be that there is no one scientificmethod -that for each field of investigation special methods must be devisedand that the terms mentioned have diierent meanings in different situations. Whatever the ultimate nature of "the scientific method" may be, it is clear that a t present any study in this field is likely to suffer from misunderstandings and ambiguities. During the past year, under the direction of Dr. Ralph W. Tyler, Chief Examiner for the University of Chicago, several attempts hive been made to evaluate student achievement in scientific thinking. Many items have been constructed and have been included in several examinations. The present paper is a report of one such item, designed to test two aspects of the evaluation of a theoretical principle, namely, whether a given statement supports or has little or no bearing on the principle, and whether the statement is experimental or theoretical. The test item is given in full. Principle: In reading, atoms of both melals and non-metals tend to assume an electron confLguration similar to that of theraregases. In the blank before each of the following statements, write l-if 2-if
the statement is lrue and rupgortr the foregoing principle; the statement is lruc, but has little or no bearing on the principle;
JULY, 3-if
307
1940 the statement is false.
- (1) The rare gases have "complete" shells. - (2) Complete shellsareeatraordinarily stable. - (3) Therare gases form no stable chemical compounds.
- (4) Elements of main group I in solution readily form ions wlth a single positive charge. - (5) Elements of main group I1 in solution readily form ions with a double positive charge. - (6) E!ements of main group I1 m solution readily fonn ions wzth a negative charge of s k . - (7) Elements of maingroup VII form in solution ions with a smgle negative charge. ions - (8) E!ements main group VI readily form in w ~ t hapositivecharge of six. - (9) Metallic elements are good conductors of electricity. -(lo) Non-metallic elements are poor conductors of electricity. -(11) Ions of elements of groups I and I1 are chemically unreactive. -(12) Ions of elements of group VII are mare reactive than the free elements. -(13) Most metallic elements are solids a t ordinary temperatures. -(14) Some metallic elements show more than one valence. -(15) Non-metaUicelementsfarm compounds withother nonelements. -(16) metallic Metallic elements form compounds with other metallic elements. -(17) Some non-metallic elements are gases. -(IS) Many elements consist of mixtures of isotopes. ~ ( 1 9 Molten ) salts are good conductors of electricity. ~ ( 2 0 Salts ) are highly ionited in solution. Reconsider the statements which you marked 1 or 2 and mark further 4--if the statement is cxperimcntal eYidence; &if the statement is a theoretical assumption or deduction.
This item was included in three comprehensive examinations on three distinct academic levels. No special instrnction on the scientific method was given on any of these levels. The courses on each level and the examinations for these courses may be described briefly as follows. ( A ) The High-School Physical Science General Course. -This is a two-year course for grades 11 and 12. It is an integration of physics, chemistry, astronomy, and geology. The chemistry content is+about thirty per cent. of the course, most of which is taught during the twelfth year. The examination for the twelfth year was four hours in length and was taken by sixty students. (B) The College Physical Science General Course.This is a one-year course for university freshmen or sophomores. I t is an integration of physics, chemistry, astronomy, geology, and mathematics. The chemistry content is somewhat over twenty-five per cent. The examination was six hours in length and was taken by five hundred thirty students. ( C ) The College Chemistry Sequence.-This is a series of three courses, consisting of two courses in inorganic chemistry and one course of either elementary organic chemistry or elementary qualitative analysis. The examination was six hours in length and was taken by one hundred twenty students. Table 1 gives the percentage of each group making the various responses. It may be noted that the sum of columns 1 , 2 , and 3 is equal to 100; and that the sum of columns 4 and 5 is also equal to 100. For example, in
the college general course, seventy-four per cent. of the five hundred thirty students considered the first statement, namely, "The rare gases have 'complete' shells," as true and supporting the stated principle. Twentyfour per cent. considered the statement true but thought it to have little or no bearing on the principle. Two per cent. were completely wrong and considered the statement false. Of those indicating that the statement is true ( i . e., either response 1 or 2) ten per Cent. considered the statement experimental, while ninety per cent. considered it theoretical. TABLE 1
P s a c ~ ~ r ~oro Sa~ v o s a r sIN E ~ c xOP THE THRBBCOURSES MAKING BIce R~SPONI&
Collcgc General Courrc (530 Sl%dcdr)
Stolem e n 1 1 2 3 4 5 1 1 2 3 4 5 6 7 8 9 10 11
:
14 15 16 17 18 18
74 77 22 71 71 6 70 6 15 11
24 20 51 24 24 6 21 7 84 83 11 4
2: 23 19 9 7 8 22
2 3 27 5 5 8 8 9 8 7 1 6 85
10 47 96 55 54
90 78 53 83 4 20 45 91 46 91 0 55 45 91 9 99 1 13 99 1 7 79 21 51 12 98 61 39 13 98 2 30 98 2 11 9 7 3 2 47 53 1 99 1 19
:: 4i
67 10 63 18 43 48 9 0 2 79 13 58 20
'
nigh-school Ccnrrol Caurrc (60 S ~ d c n l s )
chcmiriry Scgurncc (120 Studcnlr)
z
3
4
5
1
2
3
22 15 62 7 9 2 8 3 87 91 11 11
0 2 18 2 0 98 1 88 0 2
9 47 93 73 80
91 53 7 21 20
75
25
99 98
1 2 6
75 80 31 87 85 0 82 10 13 10 13 32
22 18 61 13 15 2 16 3 87 88 2 20
3 2 8 0 0 98 2 87 0 2 85 48
loo
38 94 77
'
97
98
82 5 66 33 20 75 5 68 2 98 2 38 37 25 52 37 97 3 22 31 47 93 5 9 7 3 5 9 3 2 86 13 51 49 2 66 32 1 1 3 80 7 74 7 99
4
5
8 58 98 52 48
91 42 2 48 52
50 50 I00 0 I00 0 89 11
98 68 32 95 2 100 0 98 2 44 56 48 2
Z0 40 59 57 43 43 56 62 38 25 61 l 4 50 he number. in italics repreant the answers which the staff have agreed on aE the best answer in each care.
..
.
.
I n this tabie the italicized responses are those which ten experts associated with the courses either as instrucTABLE 2
FYRT~H ANALYSIS X OP RBSPONS~S BY STOD~NTS RBCBIYINC EACHLBTTBR GRAD= IN COLCBOB GBNBF+L COURSB
AII S~a~ernc=l Res~onsc 1 1 1 5 2 I 2 5 1 3 4 5 5 5 6 3 11 5 12 3 14 4 18 4 20 1 20 5
.
9udmt*
74 90 77 53 22 45 46 88 21 40 61 47 40 43
A* 82 100 87 85 42 43 43 100 21 71 8s 62 60 63
B*
C*
D*
P*
79 84 84 61 28 54 54 9s 29 50 62 50 48 48
72 87 78 43 16 47 49 88 7 29 56 42 33 40
69 87 58 35 14 38 42 73 1'1 28 49 39 26 29
58 82 46 46 6 25 19 54 67 17 53 39 29 28
-8
AII figures given @rein perecnta~es.
tors or as examiners have deemed as the best answers. Where no agreement among the experts could be ohtained, as in items 14 and 19, more than one alternative answer has been allowed. For each academic level, each response has been further analyzed to determine how each group of students receiving a given letter grade has answered the
question. Table 2 gives a detailed analysis on representative responses, made by students of the college general course. Thus while statement 1 was considered theoretical (response 5) by ninety per cent. of the total group of five hundred thirty students, it was so considered by one hundred per cent. of the A's, by ninetyfour per cent. of the B's, by eighty-seven per cent. of the C's, by eighty-seven per cent. of the D's, and by only eighty-two per cent. of the F's. INTERPRETATIONS
Distinction 6etween True and False Statements.-It is evident from Table 1 that students do not experience great difficulty in distinguishing between true (response 1or 2) and false statements. Statements 12 and 16 are notable exceptions. In all cases, however, the agreement with the answer key is better, the higher the scholastic ability of the students. Thus in question 12, while only forty per cent. of the students in the college general course recognized the statement as false, seventy-seven per cent. of the students in the college sequence recognized it as such. Furthermore, in the college general course, seventy-one per cent. of the A's, hut only seventeen per cent. of the F's recognized it as false. Distinction between Whether a Statement Supports or Has Little or No Bearing on the Principle.-There is wide disagreement among students on this distinction. In nearly all cases, however, the majority of the answers agree with the answer key. ~urthermore,the higher the scholastic ability, the greater the agreement with the answer key. Thus in statement 2, while seventy-seven per cent. of the entire group agreed that the statement supported the principle, eighty-seven per cent. of the A's and only forty-six per cent. of the F's agreed. Likewise, the percentage of answers agreeing with the answer key is greater for the college chemistry sequence than for the college general course. Distinction between Fad a n d Theory.-In regard to this distinction, the statements fall i n t o one of three categories. The first category consists of statements in which essentially all students agree. Statements 3, 9, 10, 13, 15, 16, 17, and 19 are clearly experimental, while statement 1 is clearly theoretical. These statements are also those upon which the experts agreed without any difficulty. We may conclude, therefore, that the students have no difficulty in making the distinction in cases in which the statement is obviously either theoretical or experimental.
The second category consists of statements in which there is wide disagreement among the students, but the agreement with the answer key is better, the higher the scholastic ability. Statements 2, 18, and 20 are of this category. Statement 20, for example, has been considered theoretical by but forty-three per cent. of the students of the college general course. Detailed analysis shows, however, that sixty-three per cent. of the A's and only twenty-eight per cent. of the F's have answered it as theoretical. Similar relations obtain in the other statements and among students of the other two courses. It is significant to point out that in this respect the increased training of three college chemistry courses did not enable the students of the college chemistry sequence to do better than the students of the general courses. It seems as though, the greater the training in chemistry, the greater the tendency to regard as experimental, statements such as "Complete shells are extraordinarily stable." The third category consists of statements 4, 5, 7, and 11. m i l e in these statements there is a trend for better agreement with the answer key, the better the scholastic ability, nevertheless there is a great deal of confusion. It seems that the word "ion" is widely considered as referring directly to experiment. Finally, the question may be raised to what extent these responses are significant and to what extent they are due to chance. No definite answer can be given to this question. However, in test items in which one of two alternatives must be answered, even if chance operates fully, the wrong response is rarely made by more than fifty per cent. of the population. The fact that in this test many "wrong" responses were made by more than fifty per cent. of the students indicates a definite bias in the "wrong" direction. In conclusion, the findings may be summarized as follows. Students have little difficultyin distinguishing between true and false statements. They have some difficulty in deciding whether a given statement supports or has little or no bearing upon a given principle. In distinguishing between experimental and theoretical statements they have little difficnlty if the statement is obvious, but experience considerable difficulty if the statement is not obvious. Increased training in chemistry does not help in this distinction. In all other distinctions, students do better, the longer their training in chemistry, and the higher their scholastic standing. More research is needed before these findings can be considered as established.
LITERATURE CITED
"What (1) S m ~ o s ~ u ~ : are our objectives in teaching chemistry?' J. C ~ MEDUC., . 2,971-97 (1925). (2) "Science in general education," Report of the committee on the functionof science in general education. Commission of Secondary School Curriculum, Progressive Education Association. D. Appleton-Century, New York City, 1938, pp. 32642.
(3) Pomns, Chairman, "A program for teaching science," 316 Yearbook of the National Society for the Study of Edycation, Part I. Public School Publishing Co., Bloommgton, 1932. (4) COMM~TTEE ON THE STATE TEACHERS' ASSOCIATION.''A Wisconsin philosophy of science teaching," Sch. Sci. Math., 32, 7 6 W (1932).
JULY,
1940
(5) DEWEY."The supreme intellectual obligation," Sci. Educ., 16,l-4 (1934). (6) TYLER,"Ability to use scientificmethod," Educ. Rcs. Bull., 11, 1-9 (1932). (7) BEAUCHAMP, "Teaching scientific method," Sch.Sci.Math., 34, 508-11 (1934). (8) &RTI~, "Teaching scientific methods," {bid.,34, 816-20 (1934). (9) DAVIS,"Is this the scientific method?" ibd.,34, 83-6 (1934).
309 (10) DOWNING, "Does science teach scientific thinking?" Sci. Educ..87-9 (Apr., 1933). (11) BELL,"The scientificmethod in practice," J. C~sna.Eouc.. 8, 1817-23 (1931). (12) STEAUSS,"Some results for the test of scientific thinking," Sci. Educ.,16, 89-93 (1931). (13) Ana~s, "The development of thinking in science," High Poinls, 1 4 , 3 7 4 0 (Jan.. 1932). (14) HERRIND,"Measurements in scientific thinking," I.Educ. Psych., 9, 535-58 (1919).
