Research: Science and Education edited by
Chemical Education Research
Diane M. Bunce The Catholic University of America Washington, DC 20064
Designing, Testing, and Validating an Attitudinal Survey on an Environmental Topic
Michael J. Sanger Middle Tennessee State University Murfreesboro, TN 37132
A Groundwater Pollution Survey Instrument for Secondary School Students Idoya Lacosta-Gabari,* Rosario Fernández-Manzanal, and Dolores Sánchez-González Department Didáctica de las Ciencias Experimentales, University of Zaragoza, 50009, Zaragoza, Spain;
[email protected] Since Rensis Likert introduced his method for measuring attitudes in 1932, survey instruments have been developed to obtain information about a wide range of subjects (1–10). Measurement of student attitudes regarding the environment is one important example of the application of Likert-based survey instruments (11–15). Noteworthy examples include scales for measuring environmental attitudes to a specific problem, such as the Children’s Attitudes Toward the Environment Scale (CATES) to assess the attitudes of school children towards water consumption (16), or attitudes of a specific group, such as the Environmental Attitudes of the University Scale (EAU) suggested for university students (17). Water pollution is one of the most urgent environmental problems (18). However, the measurement of attitudes toward groundwater pollution has not been much explored, despite the fact that pollution is a basic subject for determining environmental behavior (19). As Stern suggests (20), environmental attitudes should be measured in relation to specific subjects that optimize the prediction of particular behavior. Water is considered to be contaminated when its quality has changed to such an extent that it is unsuitable for a specific use. In this respect, water contamination relates with the environmental attitudes of the population and the problem can thus be successfully addressed if these attitudes change (21). It is therefore important to educate the population as a whole with knowledge of and respect for groundwater resources in local aquifers. In Spain, however, the subjects taught in secondary education (12–16 years) have paid little attention to changing environmental attitudes. Furthermore, according to the most recent study carried out in our country (22), the 12–16 year age group is the least-studied sector regarding environmental attitudes. The curriculum for secondary education is currently under review in Spain. The changes will include incorporation of objectives in all subjects, including chemistry, aimed at promoting sustainable relations with the environment. This is the context in which this work has been carried out. The study has the modest intention of providing a new, valid, and reliable measurement scale for evaluating the attitudes of secondary school students in the chemistry class towards groundwater pollution. Research Problem and Objectives The main problem in this research is to develop a Likerttype scale to measure the attitudes of 14–16-year-old chemistry students toward groundwater pollution that allows application of the necessary steps to ensure the validity and reliability of the
scale. We also sought to define the components of the scale so that it can be used to identify possible areas of intervention and remedial action regarding students’ attitudes. Development of the Survey Instrument The Groundwater Pollution Test (GPT) described in this paper is a Likert-type attitude scale developed for secondary school students, following the specifications described within the specialized literature (23–26). As Spector points out, “this type of scale has a high degree of reliability and validity, is cheap and easy to apply, and does not provoke complaints from respondents” (25). There are several steps in the development of such a scale—which we describe below—and checks that minimize its defects. To begin with, the attitude target was defined. Four basic strands were selected, taking into account the thematic content of the subject, the habits of the population contributing to water pollution, the consequences of water pollution for health and the environment, and the need for purification treatment (27–28). The four basic strands that define “attitude” in the GPT include views on:
1. Water quality and health, defined as the importance students attach to the relationship between water quality and health
2. Sources of water pollution, defined as the importance students attach to their daily activities as sources of water pollution
3. Water-contaminating agents, referring to the importance students attach to substances used daily that are water pollutants
4. Treatment of contaminated water, referring to the connection students make between water pollution caused by their daily activities and the subsequent purification treatment required
In Bennett’s words, “the term attitude was used to describe the overall picture gained from responses in all strands” (2). To construct an initial pool of items corresponding to these strands, interviews were conducted with a sample of secondary school students (N = 32). Their opinions were also collected during three working sessions in which they were invited to debate the problem of water pollution. The information gathered was used to draw up the items. The second step was to design the format of the scale. The GPT follows the usual format of Likert-type scales. This consists of asking the students to mark each item with an X in one of
© Division of Chemical Education • www.JCE.DivCHED.org • Vol. 86 No. 9 September 2009 • Journal of Chemical Education
1099
Research: Science and Education List 1. Response Categories for Item 14 Item 14 Statement, Groundwater Pollution Survey I would go to an extracurricular activity that showed me how to reduce water pollution in my hometown. Possible Reasons Corresponding to Item 14 A. I strongly agree because I would learn more about substances that pollute water. B. I strongly agree because I would know more about what I consume. C. I strongly agree because I would learn more about how better to protect nature and the environment. D. I strongly agree because it is good to learn. E. I agree but I might not turn up. F. I neither agree nor disagree because it won’t change anything. G. I disagree because I have little free time and I can’t go in the afternoons. H. I strongly disagree because I already spend a lot of time at school.
List 2. Groundwater Pollution Survey Items Statements for Response 1. 2.
I usually throw away used batteries in the trash. I would be in favor of supporting a business that does not pollute water.
3.
I drink tap water even if the local authorities say it is not fit for drinking.
4.
I try not to dispose of waste in the water so that it will be cheaper to clean.
5.
I try to limit the amount of soap I use in the shower. I’m in favor of asking the local authorities to impose fines on those who pollute water.
6. 7.
Water belongs to all of us and we can use it in any way we like.
8.
It doesn’t matter if groundwater is polluted because it isn’t used for anything.
9.
We should support farmers who use few chemical products.
10. I don’t think it matters if water is polluted because it can be cleaned at the purification plant. 11. I don’t think waste (chewing gum, cigarette ends, oil, etc.) should be thrown in the toilet. 12. I take drinking water with me when I go on an outing. 13. I don’t think agriculture causes water pollution. 14. I would go to an extracurricular activity that showed me how to reduce water pollution in my hometown. 15. I know when I’m wasting water, but I keep doing it. 16. I drop food crumbs, sauces, and oils down the drain. 17. I try to save water in my daily activities. 18. I would like to know how the water I drink can affect my health. 19. The pollution we cause is naturally cleaned up by nature. Note: Items numbers in bold type (1, 3, 7, 8, 10, 13, 15, 16, and 19) are negative statements, so a “strongly agree” would be given a score of 1 and a “strongly disagree” would be given a score of 5. For all other items (2, 4, 5, 6, 9, 11, 12, 14, 17, and 18), a “strongly agree” would be given a score of 5 and a “strongly disagree” would be given a score of 1.
1100
the five boxes corresponding to the possible answers: “strongly agree”; “agree”; “neither agree nor disagree”; “disagree”; and “strongly disagree”. This format enables the neutral position to be recorded and thus minimizes acquiescence bias (25). At this point the initial pool of items was revised by three chemistry teachers from the Science Education Department. This expert revision assisted in confirming the proposed strands for the purposes of assessing attitudes toward groundwater pollution. As De Vellis has pointed out, “having experts review your item pool can confirm or invalidate your definition of the phenomenon. You can ask your panel of experts to rate how relevant they think each item is to what you intend to measure”(24). Colloquialisms were avoided and the wording adapted to the reading level of the respondents. The items were phrased in the first person to reflect the declarative nature of the questionnaire. Furthermore, both positively and negatively worded items were included, as bias can be reduced by using items phrased in opposite terms. The initial list of items was almost double that of the final version (24). In the third step, the provisional version of the GPT was subjected to a pilot test in which each answer was categorized on a standard answer sheet. The students in the sample (N = 32) were asked to perform two tasks: to indicate their level of agreement with the item statement in the corresponding box, and to write a paragraph justifying their choice (2). These answers were categorized for each item by one teacher only, and this provided information about the ambiguity of some items and agreement among student responses. The categorization of the answers identifies the different positions adopted by the students and includes different reasons given in relation to the same statement. List 1 shows an example of categorization for item 14. The student answers justify the decision to include values for the position of indifference, the degree of agreement, and the degree of disagreement. Finally, a survey of 30 items was given to a sample of 267 students of the same age and characteristics as those to whom the questionnaire was to be directed. The questionnaire was given to the students by the authors of the study and all were given the same directions for its completion. The data were collated to establish the reliability and validity of the scale. A definitive scale of 19 items was obtained, which was re-tested with a sample of students (N = 24) with characteristics similar to the previous sample. Sample Demographics The total sample involved in the study comprised 323 students aged 14–16 from secondary schools in Navarre, Spain. They were distributed in the following manner. At the interview stage 32 students took part in two different groups with equivalent numbers of boys and girls (50% each). A sample of 267 students was then used to validate the instrument. The sample size satisfies Nunnally’s criterion that there should be five subjects per item in the initial version of the scale (29). Also, Spector notes that “the item analysis requires a sample size of about 100 to 200 respondents” (25). Subsequently, the scale was re-tested with a sample of 24 students who were the same age and had the same demographic characteristics as the previous students. In this case the sample is equivalent to that proposed by Spector in the validation of his Work Locus of Control Scale (25).
Journal of Chemical Education • Vol. 86 No. 9 September 2009 • www.JCE.DivCHED.org • © Division of Chemical Education
Research: Science and Education
Data Analysis The students’ responses to the questionnaire were manually transcribed and allotted a numerical value in the range of 1–5. Five points were assigned to the positively worded items marked “strongly agree” and to the negatively worded items marked “strongly disagree”. Other points correspond to the intermediate positions between these two extremes. Statistical tests were performed using a combination of tools, in particular SPSS 13.0. The data were subject to an item analysis and the results were used to choose the items that form an internally consistent scale. List 2 shows the final numbered definitive items. Results and Discussion Analysis of the Items and Reliability of the Scale Two types of reliability were evaluated: the internal consistency of the scale, and the test–retest replication. The internal consistency measurement includes the item–total correlation
for each item and Cronbach’s α coefficient. Internal consistency among a set of items suggests that they share common variance or that they are indicators of the same underlying construct. Therefore the items with low correlations compared to other items were eliminated from the instrument. At this stage, we retained items for the final version that had item–total correlation values considered acceptable, between 0.23 and 0.50 (25, 26). We operated with the assumption that a set of items with moderate item–total correlation values can also represent a homogeneous set with regard to the feature to be measured (25). The Cronbach’s α value obtained was 0.74. According to De Vellis, “ranges for research scales between 0.65 and 0.70 are minimally acceptable; between 0.70 and 0.80 respectable”(24). The test–retest replication represented a second reliability test. The α value obtained in the test and retest was 0.68 and the test–retest correlation value was 0.64. This result reflects scale measurement consistency over time. This sample provides an approximate idea of the test–retest reliability. A further test with a larger sample should be carried out in the future.
Table 1. Comparison of the Groundwater Pollution Survey Inventory Factors and Loading Profiles Strand
Item
Inventory Statements
Number
Factors F1
F2
F3
1: Views on water quality and health 18
I would like to know how the water I drink can affect my health.
0.498
03
I drink tap water even if the local authorities say it is not fit for drinking.
0.445
12
I take drinking water with me when I go on an outing.
0.312
2: Views on the sources of water pollution 13
I don’t think agriculture causes water pollution.
0.427
08
It doesn’t matter if groundwater is polluted because it isn’t used for anything.
0.667
17
I try to save water in my daily activities.
02
I would be in favor of supporting a business that does not pollute water.
0.542
09
We should support farmers who use few chemical products.
0.672
0.639
3: Views on water-contaminating agents 19
The pollution we cause is naturally cleaned up by nature.
0.531
11
I don’t think waste (chewing gum, cigarette ends, oil, etc.) should be thrown in the toilet.
0.313
05
I try to limit the amount of soap I use in the shower.
0.718
01
I usually throw away used batteries in the trash.
0.381
16
I drop food crumbs, sauces, and oils down the drain.
0.570
4: Views on treatment of contaminated water 07
Water belongs to all of us and we can use it in any way we like.
0.526
10
I don’t think it matters if water is polluted because it can be cleaned in the purification plant.
0.638
04
I try not to dispose of waste in the water so that it will be cheaper to clean.
0.579
15
I know when I’m wasting water, but I keep doing it.
0.458
06
I’m in favor of asking the local authorities to impose fines on those who pollute water.
0.480
14
I would go to an extracurricular activity that showed me how to reduce water pollution in my hometown.
0.544
Note: The rotation method used was varimax with Kaiser normalization.
© Division of Chemical Education • www.JCE.DivCHED.org • Vol. 86 No. 9 September 2009 • Journal of Chemical Education
1101
Research: Science and Education
Validation of the Scale and Exploratory Factor Analysis The scale was validated using exploratory factor analysis. This analysis helps to identify scale items that appear to be related owing to the fact that they produce similar answer patterns. There is detailed discussion in the literature about the procedures to be followed and the decisions to be taken to identify scale items (30). In order to assess whether the data would factor well, the Kaiser–Meyer–Olkin measure (KMO) was applied together with Bartlett’s test. The KMO measure of sampling adequacy is an index for comparing the magnitudes of the observed correlation coefficients to the magnitudes of the partial correlation coefficients. Large values for the KMO measure indicate that a factor analysis is warranted. In fact, 0.80 ≥ KMO ≥ 0.70 are considered acceptable by Kaiser (30). The value obtained for the GPT, 0.76, was sufficient for the exploratory factor analysis to be continued. Bartlett’s test of sphericity is used to test the null hypothesis that the variables in the population correlation matrix are uncorrelated. The scale has shown a χ2 value of 700.933 and a significance level of 0.000. This test concluded that the strength of the relationship among items is strong enough to proceed to a factor analysis for the data (31). A common technique is the varimax rotation (orthogonal process), which attempts to minimize the complexity of the factors by making the large loadings larger and the small loadings smaller within each factor. The factor analysis provided three principal factors with six eigenvalues greater than 1. The structure was explored by extracting three to six factors and examining the pattern and magnitude of the load of each item in each factor or principal component. A minimum value of about 0.30–0.35 is required to consider that an item loads on a factor (25). The factors extracted were identified with sources and effects of pollution; personal actions and pollution; and awareness of pollution (this factor includes different declarations of the personal involvement or awareness of the students expressed by “I would…”, “I am in favor of …”). The items organized into each factor are shown in Table 1. Conclusions Education is, of course, considered to be vitally important for forming attitudes in general. In the words of Reid (1): [A]ttitudes are very important in that they can influence subsequent behavior. Thus, attitudes related to the sciences developed at school may well be retained into adulthood and play a major role in all kinds of patterns of behavior. Negative attitudes may well have potentially very harmful effects at personal, social or national levels.
As regards this specific study, the following conclusions can be drawn. About the Validity and Reliability of the GPT The validity and reliability values of the GPT suggest that this survey is a good instrument for researching attitudes of secondary school students toward groundwater pollution. However, it must be recognized that the scale is based on the results of a specific sample. Future research into the extent to which the GPT survey could be adapted to other samples and populations would be of considerable interest. As De Vellis has pointed out, future researchers should take into account that 1102
the value of Cronbach’s α coefficient can decrease when the scale is administered to samples distinct from those used in its development (24). Factors for Analysis The study involves the identification of three factors or principal components reflecting the attitudes under consideration, into which the items are arranged. These are: F1, sources and effects of pollution; F2, personal actions and pollution; and F3, awareness of pollution. Their extraction through factor analysis reinforces the evidence that an attitudinal construct is measured by the scale. Applicability of the GPT The GPT can be applied to evaluate the attitude of a group of students toward groundwater pollution, to compare the attitudes of different groups, and to measure changing attitudes over time or after a specific teaching activity. Teachers concerned about environmental matters can design specific tasks to improve student sensibilities toward groundwater pollution. Other Applications and Transferability The use of this scale by other researchers would be of interest to determine whether the factors obtained are maintained in subsequent applications. This instrument was written, administered, and validated in the students’ native language of Spanish. Before this instrument is used in English, it should be validated in that language as well. Acknowledgments We would like to thank the students who participated in the sample and the schools that took part in the survey. Literature Cited 1. Reid, N. Res. Sci. Tech. Edu. 2006, 24, 3–27. 2. Bennett, J.; Rollnick, M.; Green, G.; White, M. Int. J. Sci. Educ. 2001, 23, 833–845. 3. Bauer, C. F. J. Chem. Educ. 2005, 82, 1864–1870. 4. Thompson, J.; Soyibo, K. Res. Sci. Tech. Educ. 2002, 20, 25–37. 5. Kelter, P.; Hughes, K.; Murphy, A.; Condon, K.; Heil, P.; Lehman, M.; Netz, D.; Wager, T. J. Chem. Educ. 1994, 71, 864–866. 6. Voegel, P. D.; Quashnock, K. A.; Heil, K. M. J. Chem. Educ. 2004, 81, 681–684. 7. Voegel, P. D.; Quashnock, K. A.; Heil, K. M. J. Chem. Educ. 2005, 82, 634–636. 8. Oliver-Hoyo, M. T.; Allen, D. J. Chem. Educ. 2005, 82, 944– 949. 9. Pell, T.; Jarvis, T. Int. J. Sci. Educ. 2003, 25, 1273–1295. 10. Reid, N.; Skryabina, E. Res. Sci. Tech. Edu. 2002, 20, 67–81. 11. Wiegel, R. H.; Wiegel, J. Environ. Behav. 1978, 10, 3–15. 12. Thompson, S. C. G.; Barton, M. A. J. Environ. Psychol. 1994, 14, 149–157. 13. Leeming, F. C.; Dwyer, W. O.; Bracken, B. A. J. Environ. Educ. 1995, 26, 22–31. 14. Dunlap, R. E.; Van Liere, K. D.; Mertig, A. D.; Jones, R. E. J. Soc. Issues. 2000, 56, 425–442. 15. Misiti, F. L.; Shrigley, R. L.; Hanson, L. Sci. Educ. 1991, 75, 525–540. 16. Musser, L. M.; Malkus, A. J. J. Environ. Educ. 1994, 25, 22–26.
Journal of Chemical Education • Vol. 86 No. 9 September 2009 • www.JCE.DivCHED.org • © Division of Chemical Education
Research: Science and Education 17. Fernández-Manzanal, R.; Rodríguez-Barreiro, L.; Carrasquer, J. Sci. Educ. 2007, 91, 988–1009. 18. Local Actions for a Global Challenge, 4th World Water Forum; World Water Council and the Secretariat of the 4th World Water Forum: México City, March 16–22, 2006. http://www.worldwatercouncil.org/fileadmin/wwc/Library/Publications_and_reports/ Final_Report_4th_Forum.pdf (accessed Jul 2009). 19. Van Liere, K. D.; Dunlap, R. E.; Environ. Behav. 1981, 13, 651–676. 20. Stern, P. C. J. Soc. Issues. 2000, 56, 407–424. 21. United States Environmental Protection Agency Home Page for Groundwater and Drinking Water. http://www.epa.gov/safewater/ (accessed Jul 2009). 22. Amérigo, M. Medio Ambiente y Comportamiento Humano 2006, 7, 45–71. 23. Oppenheim, A. N. Questionnaire Design and Attitude Measurement; Basic Books: New York, 1996. 24. De Vellis, R. F. Scale Development. Theory and Applications; Applied Social Research Methods Series: Newbury Park, CA, 1991.
25. Spector, P. E. Summated Rating Scale Construction. An Introduction; Sage Publications: Newbury Park, CA, 1992. 26. Abdel-Gaid, S.; Trueblood, C. R.; Shrigley, R. L. J. Res. Sci. Teach. 1986, 23, 823–839. 27. Yen, T. F. Environmental Chemistry: Essentials of Chemistry for Engineering Practice; Prentice Hall: Upper Saddle River, NJ, 1999. 28. Crosby, D. G. Environmental Toxicology and Chemistry; Oxford University Press: New York, 1998. 29. Nunally, J. C. Educational Measurement and Evaluation; McGrawHill: New York, 1972. 30. Kaiser, H. F. Psychometrika 1974, 39, 31–36. 31. Dyer, D. D.; Keating, J. P. J. Am .Stat. Assoc. 1980, 75, 313–319.
Supporting JCE Online Material
http://www.jce.divched.org/Journal/Issues/2009/Sep/abs1099.html Abstract and keywords Full text (PDF) Links to cited URLs and JCE articles
© Division of Chemical Education • www.JCE.DivCHED.org • Vol. 86 No. 9 September 2009 • Journal of Chemical Education
1103
