In the Classroom
Using News Assignments To Develop Skills for Learning about Science from Public Information Sources
W
Mary M. Walczak Department of Chemistry, St. Olaf College, Northfield, MN 55057;
[email protected] The need for increased scientific and technological literacy among the general population is well documented (1, 2). Concerns have been raised about the waning scientific workforce and calls have been made for changes that make science accessible to all students (1–6). The urgency of training the next generation of scientists is recognized in tandem with the need for increasing scientific and technological literacy among all citizens. Although “scientific literacy” has been interpreted to mean different things over the last 50 years (7), for purposes of this study scientific literacy will be taken to mean functional scientific literacy—having command of scientific vocabulary and being able to converse, read, and write coherently in a nontechnical context (7, 8). These characteristics are essential components of conscientious citizenship (9). Courses that focus on issues of science, technology, and society have been developed in an effort to bring the relevancy of science to all and to increase functional scientific literacy among all citizens. Many of these efforts have been in general education (nonmajor) courses, although some chemistry programs have introduced similar components within the major. Several chemistry textbooks are available specifically for general education students (10), most of which include features illustrating the importance of chemistry in everyday life. Various approaches to emphasize the relevancy of chemistry with nonmajors have been taken, including involving the public in data collection (11), doing hands-on activities (12) or demonstrations (13) with elementary school children, or using examples of general interest—sports, for example (14). Anecdotal reports of faculty using public information sources in courses for general education students are fairly commonplace (15), however no reports appear in the literature of studies in which such efforts have been evaluated. Courses for chemistry majors that specifically address issues of relevancy often include projects about chemical applications to everyday life that emphasize oral and written communication skills. Parill had organic chemistry students write about a chemical reaction that they encounter on a regular basis (16). Part of the assignment involves describing why this reaction is of importance to the student. Based on student grades on the papers, Parill found that giving incremental feedback to students at several points during the projects resulted in better papers. White et al. (17) described a capstone course taken by senior chemistry majors in which peer presentations on societal issues are an integral part. Students made presentations in agreement or disagreement with an issue (e.g., drinking water should be treated with hydrogen peroxide rather than chlorine). Surveys at the beginning and end of the course indicated that student opinions on issues changed only for issues presented by their peers during the course. No changes in student opinions were observed for issues not addressed in class.
www.JCE.DivCHED.org
•
General education science courses are generally designed to help students develop functional scientific literacy skills that are needed by conscientious citizens in an increasingly technology-based society. Citizens look to the popular media to learn about all kinds of issues, including those related to science and technology (2, 18). Unfortunately, these sources sometimes contain errors or biased representations and not all citizens have the necessary background knowledge to critically evaluate the reliability of the information. Toby’s article about chemical misinformation (19) illustrates but a few examples. Consequently, this paper reports an approach for incorporating public information sources into a general education course. In particular, the goals of the project (in part) were to: 1. Foster independent research and analysis of topics by incorporating multiple assignments that involve a decreasing level of in-class discussion and analysis. 2. Develop an understanding of and a means of assessing source reliability by asking students to evaluate the Web-based sources they use.
In this paper the news assignments and observations suggesting gains related to both goals, above, are described. Information is also presented to aid instructors who wish to incorporate similar exercises into courses. Methods This project was conducted during the spring and fall semesters in 2004. In the spring semester (Semester 1) 44 students enrolled in Chemistry and Your World, a general education chemistry course taken by nursing majors and other nonscience majors. Approximately half of the students enrolled in this semester were nursing majors; 90% of the students were in the first two years of college (65% first-year, 25% secondyear). In the fall semester (Semester 2) 10 students1 enrolled in A Matter of the Environment, an environmental chemistry course taken as an elective by environmental studies majors and taken to fulfill the college general education requirements by nonscience majors. Two of the students enrolled were environmental studies majors; 78% of the students were in the first two years of college (22% first-year, 56% second-year). Neither course had prerequisites. The study was conducted with approval by the St. Olaf College Institutional Review Board. In both courses, students signed an informed consent formW agreeing to be part of this study. Both courses utilized textbooks designed for general education chemistry courses (10, 20). These books focus on teaching chemistry through topics of relevance to nonscience students. To increase the course’s focus on contemporary issues, a series of news assignments were developed to specifically investigate current topics. Three (Semester 1) or five
Vol. 84 No. 6 June 2007
•
Journal of Chemical Education
961
In the Classroom
(Semester 2) such assignments were incorporated into the 13week semester. The topics considered are given below in List 1. More details on each topic are provided in the Supplemental Material.W Student grades for the course were determined by a combination of exams, homework assignments, laboratory reports, class participation, and news assignments. The assignments were intended to help students develop skills necessary to investigate science topics through the popular media. For the earliest assignments, an entire (85-minute) class period was devoted to discussing the topic, but less class time was reserved for the topics as the semester went on in order to foster independent analysis skills. At the beginning of the Semester 1 course, the instructor demonstrated that information available on the Internet is not always reliable and gave a short lecture about the culture of peer-review in science and the lack of peer-review on much of the Internet. Some students nevertheless used questionable information sources for their assignments. Therefore, the Semester 2 course used a more formal means of assessing Web source reliability. In addition to an introductory lecture similar to that for Semester 1, an evaluation form was required for each Web site referenced in their assignments.W At the end of each semester students were asked to complete a course evaluation in which they were asked what they found most interesting or worthwhile about the assignments and what suggestions they had for improving the news assignments. At the end of the Semester 2 course, students were also asked several questions about how the news assignments contributed to their learning gains. The Student Assessment of Learning Gains Web site was used for this evaluation (21).
Analysis of each assigned topic began with each student reading a set of two–four articles found in the newspaper or on the Internet. Typically, readings came from the New York Times, federal agency Web sites (e.g., EPA), U.S. News and
List 1. News Assignment Topics by Semester
Spring 2004: Semester 1 • Risk Assessment of the Influenza Vaccine— Should You Be Immunized? • Nuclear Energy Storage—Impact of Expansion at the Prairie Island Site • The Low Carb Diet Craze—A Sensible Diet?
Fall 2004: Semester 2 • Methane Emissions and Global Warming • Ozone Pollution and National Parks • Nuclear Energy Storage—Impact of Expansion at the Prairie Island Site • Is Buying a Hybrid Vehicle Worth It? • The Hydrogen Economy—Future or Folly?
Journal of Chemical Education
• Group 1: What is influenza (“the flu”) and who should be immunized? • Group 2: What is “avian flu”, and should we worry? • Group 3: Does this year’s vaccine really protect against the flu? • Group 4: Is this year’s flu season really worse than usual?
Students read up to four articles related to one of these questions and answered four different focus questions. Again using the flu vaccine assignment as an example, the students in Group 2 read an information page from the U.S. Centers for Disease Control and Prevention (CDC) Web site and three New York Times articles (22) and considered these focus questions: • Why is the avian influenza A H5N1 virus outbreak occurring in Southeast Asia a concern to the world’s health authorities? • What is different about this strain in comparison with those included in this year’s influenza vaccine? • What means are available to prevent the spread of this influenza strain among birds? Among humans?
Description of Assignments
962
World Report, and other similar sources. In preparation for class, each student read articles related to one aspect of the issue and was asked to answer four questions based on their reading. The news assignment topics and focus questions are included in the Supplemental Materials.W Up to five different sets of information were distributed to the class, with individual students being responsible for one set of information. For example, the influenza vaccine assignment included four sets of readings related to the following questions:
•
• Dead migratory birds have tested positive for this influenza. Why is this finding significant?
Although most of the focus questions are expository questions—knowledge and comprehension cognitive domains (23)—subsequent work on the assignments involves some higher-order thinking skills. A complete description of the entire assignment and news assignment synopsis annotated with cognitive domains is included in the Supplemental Material.W The expository questions primarily ask students to identify, state, discuss, or summarize aspects of the issue found in their reading. The more advanced questions ask students to express, speculate, formulate, or support opinions related to the issue. At the beginning of the class period, students who read the same background articles met in small groups (∼3–5 students). Students in these groups discussed what they had read and their answers to the assigned focus questions. The groups summarized their thinking about the issue in light of the information read and discussed. For instance, groups composed of Group 1 students arrived at an answer to their topic question “What is influenza and who should be immunized?” After about 15–20 minutes of discussion, new “heterogeneous” groups formed with one representative from each of the initial groups. These “jigsaw” groups (24) shared the main points from each of the initial groups. Again, in the jigsaw groups, students are expected to formulate opinions based on the array of resources provided to the class. In this
Vol. 84 No. 6 June 2007
•
www.JCE.DivCHED.org
In the Classroom
manner, all students heard reports on the main points from all the reading material. These groups met for about 15–20 minutes of discussion. All references were distributed to all of the students (or they were posted on the class Web site). The remainder of the class period was used for a whole-class discussion of the topic, during which various opinions were expressed and discussed. The instructor spent some time clarifying the science behind the issue, but most of the time was devoted to student exploration of the issues raised in the assignment. The amount of time devoted to small group discussion in the initial groups was decreased as the semester went on. For the final assignment, this step was skipped entirely, and the discussion began in the jigsaw groups because it was assumed that students would have developed some critical analysis skills by that point in the course. To complete the project, students were asked to write a one-page position paper answering the topic question and support their position using evidence. Students were free to use the materials assigned to the class, but were expected to find at least two other information sources on their own. Three to five news assignments were completed over the course of the semester. As reported above, evaluating the reliability of the information was emphasized only in the Semester 2 class. Students evaluated each Web-based resource referenced in their position papers based on authority, accuracy, currency, and objectivity or bias. They were asked to express their level of confidence in an information source based on their assessment of these attributes. Although using less credible sources was not prohibited, the assessment exercises gave students an opportunity to consider the reliability of the Web sites they found. The form students used, Evaluating Web Sites,W is based on similar forms developed by Thurston Miller and Laura Wenk (25). Results and Discussion As described in the Methods section above, the Web source evaluation was only required in the Semester 2 course, which limits the analysis of the results because of the small sample size (10 students). In addition, a few students chose to either not include any Web references, or to not evaluate the Web sources they used. In the latter case, assignment grades reflected this missing component. Nevertheless, piloting the Web evaluation with this size class has generated some observations regarding students’ ability to evaluate Web source reliability. A Web source was identified as reliable by Semester 2 students for the following reasons: it contained information consistent with other sources; it contained “just the facts”; it was endorsed by professional organizations (e.g., environmental activists); it listed staff that had strong credentials; it was well-organized; or it was from an established organization (e.g., public radio or the U.S. government). Interestingly, government sources were deemed reliable by some students and biased by others. Some of the student-identified criteria for reliable Web sites mentioned above (e.g., containing information consistent with other sources) suggest success at objectively evaluating Web sites, which was goal two of this
www.JCE.DivCHED.org
•
project (develop an understanding of and a means of assessing source reliability). On the other hand, students using the general appearance of a site (e.g., being well-organized or professionally presented) suggests a need for additional instruction in this regard. A scaffolding approach, in which students are first given examples of how an information source might be evaluated, then asked to conduct such a review themselves, and finally instructed to include this type of review as part of a news assignment, is recommended. On the end-of-course evaluations for both Semester 1 and Semester 2, students were asked to respond to open-ended questions regarding the news assignments. First, they were asked what aspects of the news assignments were interesting or worthwhile. The most frequent responses were positive statements concerning learning about and better understanding relevant, current topics (25 of 54 comments). Other, less frequent responses included researching the topics (6/54), formulating an opinion (4/54), considering different viewpoints (4/54), focusing on the science behind the issues (4/54), and comments about specific topics (e.g., flu vaccines) (8/54). One student response from the Semester 2 course commented on the benefit of considering the reliability of Web sources. Two students responded that they found nothing about the news assignments interesting or worthwhile. Students were also given the opportunity to make suggestions on how the news assignments could be improved. In the Semester 1 class, several students criticized the grading rubric used that semester. In response, the rubric (news assignment grading rubricW), was revised for the Semester 2 class. No comments about the grading rubric were received at the end of the Semester 2 course, suggesting that students were generally satisfied with the grading rubric. Since so few students were graded using the rubric, it is not possible to draw any conclusions about its effectiveness. The average student score on the assignments increased modestly during Semester 2 (from 88% to 96%).2 As expected, student opinions on some issues were opposing. Some students suggested that more time than one week be given to complete the position papers, while others would prefer to have the papers due sooner. Some students preferred more class time spent in discussion or more of a certain type of class discussion (e.g., whole-class discussion), while others found the time spent in class discussing the topic of little value. One student suggested that the research be emphasized rather than the paper and another student suggested requiring less research. Such conflicting responses suggest that individual preferences play a significant role in the student reaction to these assignments and that instructors might choose to provide a wider variety of options to students. Other ideas mentioned in the evaluation include letting students choose their own topic(s), making the papers less formal, increasing the number of assignments, and having students express their views through a presentation rather than a paper. The end-of-course evaluation for Semester 2 also asked students questions about the news assignments and their learning in the course. The possible responses ranged from “no help” to “very much help” or from “not at all” to “a great deal” on a five-point, Likert-type scale. Relevant questions and responses from the Semester 2 course evaluation are
Vol. 84 No. 6 June 2007
•
Journal of Chemical Education
963
In the Classroom
included in Table 1.3 Briefly, students reported that their work in the course was “a lot” of help in finding information on current issues and being able to formulate and communicate an opinion on those issues (Q4) and evaluating the reliability of public information sources (Q5). Students think they will carry aspects of the course forward “somewhat” or “a lot”, including the ability to use resources to obtain information on a scientific topic (Q9), the ability to evaluate the reliability of resources (Q10), and the ability to make good decisions about scientific topics (Q11).
These scores, and the others presented in Table 1, are generally quite positive, suggesting that students believe they are more competent in these areas as a consequence of taking this course. Furthermore, the student responses to Q5 and Q10 about evaluating the reliability of information sources suggest increased ability in this area, thereby supporting gains related to goal two of this project. Similarly, student self-reporting on the development of their independent research and analysis skills during this course suggest gains related to project goal one. Specifically,
Table 1. Distribution of Responses to Student Assessment of Learning Gains Questions Related to News Assignmentsa Questions and Statements for Response
1
2
3
4
5
Avg.
St. Dev.
How much did each of the following aspects of the class help your learning?
No Help
A Little Help
Moderate Help
Much Help
Very Much Help
(N ⫽ 9)
(N ⫽ 9)
01. The number and spacing of news NNNassignments
0
4
3
1
1
2.89
0.99
02. News assignment resources
0
1
5
2
1
3.33
0.82
Not at All
A Little
Somewhat
A Lot
A Great Deal
(N ⫽ 9)
(N ⫽ 9)
03. Some applications of chemistry in the NNNworld around us, especially as pertains NNNto the environment
0
0
1
6
2
4.11
0.57
04. How to find information on current issues NNNrelated to science and society and to NNNformulate and communicate an opinion NNNon the issue
0
0
2
7
0
3.78
0.42
05. How to evaluate the reliability of NNNinformation found in resources such as NNNthe Web and news articles
0
0
2
6
1
3.89
0.57
How much has this class added to your skills in each of the following?
Not at All
A Little
Somewhat
A Lot
A Great Deal
(N ⫽ 9)
(N ⫽ 9)
06. Finding information on scientific or NNNtechnological topics
0
0
7
2
0
3.22
0.42
07. Considering a variety of perspectives on NNNan issue
0
2
2
5
0
3.33
0.82
How much of the following do you think you will remember and carry with you into other classes or aspects of your life?
Not at All
A Little
Somewhat
A Lot
A Great Deal
(N ⫽ 9)
(N ⫽ 9)
08. Understanding the role that chemistry NNNplays in everyday phenomena
0
0
5
3
1
3.56
0.68
09. Ability to use resources (texts, Web) NNNto obtain information about a scientific NNNtopic
0
1
3
5
0
3.44
0.68
10. Ability to determine whether resources NNN(texts, Web) on scientific topics are NNNreliable
0
0
4
5
0
3.56
0.50
11. Ability to make good decisions about NNNscientific topics
0
0
3
6
0
3.67
0.47
As a result of your work in this class, how well do you think that you now understand each of the following?
aThis
964
survey was administered at the end of the course for the Semester 2 class.
Journal of Chemical Education
•
Vol. 84 No. 6 June 2007
•
www.JCE.DivCHED.org
In the Classroom
scores above the midpoint of the Likert scale were reported for Q4 and Q9, questions directly related to developing skills for independently finding information about and analyzing a scientific topic (goal one). The questions are asked about their entire course experience, so it is not possible to ascribe these gains specifically to the news assignments. However, the news assignments were the primary focus for research and analysis in the course. Therefore, student opinions reported in the end-of-course evaluations suggest that the structure of the assignments with the decreasing level of class involvement as the semester progressed was a good approach for this group of students. Strategies for Success Instructors interested in including similar assignments in their courses are cautioned to think through their approach to avoid having the assignment too open-ended. Some consideration of the topics selected and the mechanics of the assignment are given below as examples.
Topic Selection Choose topics from current news stories that are interesting and relevant to students. Even “old news” can be used; students may be unfamiliar with some of these stories and events. The Prairie Island nuclear energy assignment was based on articles from a year earlier when the Minnesota State Legislature considered expansion of on-site, dry-cask storage at the plant. Most students were unaware of the issue, even though the nuclear power plant is within 50 miles of campus. A topic should be sufficiently narrow and yet have more than one defensible perspective (i.e., reasonable people can disagree about the topic). A broad topic, such as “energy” could lead in many different directions. Narrowing the topic to a more defined issue, such as “Is hydrogen a viable source of energy for the future?” allows the students to focus on a particular aspect of the issue. The science behind the topic needs to be accessible to the course. Topics suitable for upperlevel courses should likely require more background science knowledge than would be appropriate for an introductory course. Mechanics A class discussion among prepared students is far more effective than one with unprepared students. Therefore, students were asked to both read articles and answer focus questions ahead of time. The questions served to promote a more thorough reading of the articles. Answers to the questions were checked to ensure that students were prepared for discussion, and the position papers were graded. Alternatively, students could be asked to write and answer their own questions based on the reading as the introductory assignment. Clear deadlines were provided at the beginning of the assignment. Students were well informed about when each stage of the assignment would take place. Grading the student papers can present some problems, as noted in Semester 1. Providing a clear grading rubric is important because students enrolled in their first college science course may not know what kind of writing is required in a science class. A one-page paper requires students to write
www.JCE.DivCHED.org
•
concisely and can be graded in a short period of time, allowing students to receive feedback from the instructor quickly. The students’ ability to evaluate the reliability of information found on the Web continues to be a problem, but the evaluation form used here helped by giving students some evaluation criteria. More examples and opportunities are probably needed to help students become better at discerning the reliability of a site. An early assignment focused only on the Web site evaluation, as suggested above, will be implemented in future offerings. Conclusions News assignments based on contemporary issues with a scientific or technological basis were used in two general education chemistry courses for nonmajors. Student responses to the assignments were generally favorable. Student selfreported learning gains suggest that gains were achieved for the two project goals—fostering independent research and analysis and assessing the reliability of Web sources. Student comfort with independent work varied by individual: some were uncomfortable with the increased independence as the semester went on and others wished less class time was spent in discussion. The evidence students used in evaluating Web source reliability included consistency with other sources, endorsement by professional organizations, and being wellorganized and professionally prepared. Future course offerings will continue to include news assignments based on contemporary issues and will strengthen the Web site evaluation component by including a project early in the semester focused on developing that skill. Student input on topics and presentation formats will also be solicited. Ongoing work will include asking students to pose their own questions on contemporary issues, research the proposed topics, and present their own supported opinion. Acknowledgments Thurston Miller (Notre Dame) and Laura Wenk (Hampshire College) were generous in allowing adaptation of their Web source evaluation materials. Juliette Lantz’s careful reading of a preliminary form of this paper is sincerely appreciated. The constructive comments of the reviewers are also appreciated. W
Supplemental Materials
Several items described in this paper are available in this issue of JCE Online: the informed consent letter and form; the Web site evaluation form; the guiding questions for the news assignments; the complete assignment annotated with cognitive domains; and the grading rubric using in the Semester 2 course. Notes 1. Nine students responded to the survey. 2. During Semester 1, the average student scores on the three news assignments also increased from 86% to 94%. 3. Although ten students were enrolled in the course and completed news assignments, only nine students took the end of the semester survey because one student was absent on the last day of class.
Vol. 84 No. 6 June 2007
•
Journal of Chemical Education
965
In the Classroom
Literature Cited 1. American Association for the Advancement of Science. Science for All Americans: Project 2061; Oxford University Press: New York, 1990. 2. National Research Council. From Analysis to Action: Undergraduate Education in Science, Mathematics, Engineering and Technology, Report of a Convocation; National Academies Press: Washington, DC, 1996. 3. Advisory Committee to the National Science Foundation Directorate for Education and Human Resources. Shaping the Future: New Expectations for Undergraduate Education in Science, Mathematics, Engineering and Technology; National Science Foundation: Washington, DC, 1996. 4. Rothman, F. G.; Narum, J. L. Then, Now, and In the Next Decade: A Commentary on Strengthening Undergraduate Science, Mathematics, Engineering and Technology Education; Project Kaleidoscope: Washington, DC, 1999. 5. Tobias, S. They’re Not Dumb, They’re Different: Stalking the Second Tier; Research Corporation: Tucson, AZ, 1990. 6. National Research Council. BIO2010: Transforming Undergraduate Education for Future Research Biologists; National Academies Press: Washington, DC, 2002. 7. Laugksch, R. C. Science Education 2000, 84, 71–94. 8. Shamos, M. H. The Myth of Scientific Literacy; Rutgers University Press: New Brunswick, NJ, 1995. 9. Tro, N. J. J. Chem. Educ. 2004, 81, 54–57. 10. See, for example, Joesten, M. D.; Wood, J. L.; Castellion, M. E. World of Chemistry Essentials, 3rd ed.; Thomson–Brooks/ Cole Publishers: Belmont, CA, 2004. 11. Trumbull, D. J.; Bonney, R.; Bascom, D.; Cabral, A. Science Education 2000, 84, 265–275. 12. Cracolice, M. S.; Ward, K. Chem. Educator 1998, 3, S14304171(98)03213-2. 13. Ophardt, C. E.; Applebee, M. S.; Losey, E. N. J. Chem. Educ. 2005, 82, 1174–1177.
966
Journal of Chemical Education
•
14. Giffin, G. A.; Boone, S. R.; Cole, R. S.; McKay, S. E. J. Chem. Educ. 2002, 79, 813–819. 15. For example, the Case Ideas Web site of the Case Studies in Science Project, http://ublib.buffalo.edu/libraries/projects/cases/ ideas.htm (accessed Mar 2007), lists many public information sources as resources for generating ideas for cases. 16. Parill, A. L. J. Chem. Educ. 2000, 77, 1303–1305. 17. White, H. B., III; Brown, S. D.; Johnston, M. V. J. Chem Educ. 2000, 77, 1590–1596. 18. Fourez, G. Social Studies of Science 1997, 27, 903–936. 19. Toby, S. J. Chem. Educ. 1997, 74, 1285–1287. 20. Stanitski, C.; Eubanks, L.; Middlecamp, C. H.; Pienta, N. J. Chemistry in Context, 4th ed.; McGraw Hill: New York, 2003. 21. The Student Assessment of Learning Gains (SALG) Web site is available at http://www.wcer.wisc.edu/salgains/instructor/ (accessed Mar 2007). 22. The CDC Web site, http://www.cdc.gov/flu/avian/index.htm (accessed Mar 2007), has been modified since the Semester 1 course. The three New York Times articles were: (a) Bradsher, Keith; Altman, Lawrence K. Thais Infected With Bird Flu; Virus Spreads. The New York Times, Jan 24, 2004, p 1. (b) Bradsher, Keith. Spread of Flu Across Asia Laid to Birds That Migrate. The New York Times, Jan 27, 2004, p 11. (c) Bradsher, Keith. China Finds Birds with Virulent Strain of Flu in 3 Provinces. The New York Times, Jan 28, 2004, p 9. 23. Bloom, B. S., Engelhart, M. D., Furst, E. J., Walker, H. H., Krathwohl, D. R., Eds.; Taxonomy of Educational Objectives: The Classification of Educational Goals. Handbook 1: Cognitive Domain; David McKay Company, Inc.: New York, 1956. 24. (a) Eilks, Ingo. J. Chem. Educ. 2005, 82, 313–319; (b) Seetharaman, Mahadevan; Musier-Forsyth, Karin. J. Chem. Educ. 2003, 80, 1404–1407. 25. Thurston Miller (University Libraries, University of Notre Dame) and Laura Wenk (School of Cognitive Science, Hampshire College), personal communications.
Vol. 84 No. 6 June 2007
•
www.JCE.DivCHED.org
