Information
•
Textbooks
•
Media
•
Resources
Developing Web-Based Pedagogical Content Coursework for High School Chemistry Teachers David W. Brooks* Department of Teaching, Learning, and Teacher Education, University of Nebraska–Lincoln, Lincoln, NE 68588; *
[email protected] John P. Markwell Department of Biochemistry, University of Nebraska–Lincoln, Lincoln, NE 68588 Marjorie A. Langell Department of Chemistry, University of Nebraska–Lincoln, Lincoln, NE 68588 Randy Emry Lincoln Southeast High School, Lincoln, NE 68506 Kent J. Crippen University of Nevada–Las Vegas, Las Vegas, NV 89154 Helen B. Brooks Synaps Chem Tools, Lincoln, NE 68506 Amjad Abuloum Faculty of Educational Sciences, Department of Curriculum and Instruction, The Hashemite University, Zarqa, Jordan Karen C. Cohen Karen C. Cohen and Associates, Weston, MA 02493
We are reporting on the development and delivery of a series of Web-based content courses for in-service high school chemistry teachers. A decade ago when this project was conceived, Web courses were new. We opted for a Web format in the hopes that a wide array of participants could be served. The courses were successful, although the development team has learned several unanticipated lessons. This paper informs others of our process, describes the results, and provides suggestions to others considering similar Web-based content delivery ventures. Project Rationale In our experience, high school chemistry teachers enjoy courses rich in chemistry content and express difficulty in finding appropriate opportunities to receive such instruction. While Web-based courses have been used successfully in college science teaching (1), Web-based content courses for teachers have not generally been made available. Use of the Web to deliver science content to a geographically diverse teaching force seemed practical, as these courses address access issues and thus far have led to similar learning outcomes when compared with conventional face-to-face instruction (2). Our project intended to create a series of Web-based chemistry content courses for high school teachers. Pedagogical content knowledge (PCK) was the guiding principle behind the development of courses in this project and is supported by findings that suggest that the more formal education teachers undertake, the better they are as teachers (3). PCK is a notion that combines a sense of knowing the content and knowing ways to transfer that content in a teachwww.JCE.DivCHED.org
•
ing situation (4). Our courses attempted to strengthen a teacher’s knowledge of chemistry, including specifics about teaching. Design Rationale Mastery learning is a cognitive framework for providing instruction based on how students learn. The effectiveness of mastery learning has been demonstrated in numerous situations (5–9). Implementing a mastery-learning program with nontrivial learning goals is a complex task. Without technology, instructor time needs to be significant for such programs to be effective and well received. The advent of the Web makes it possible to accomplish much of what is needed for an effective mastery program (e.g., Keller Plan or PSI— Personalized System of Instruction) without a heavy instructor burden (2). Effective Web-based mastery learning involves repeatable testing in which the power of the Web server is used to individualize the creation of test items and feedback based on performance. Computers are excellent at grading traditional test items. Web servers can rapidly grade and provide feedback for a wide range of item types. Student responses can be followed quickly by performance-related feedback, a strategy known to increase the rate of learning (10). Technology’s paramount role in learning may be in providing feedback with time for reflection and revision (11). Several reports indicate favorable learning outcomes when online systems are used as part of college science courses (12, 13). Web-based assessment systems that emphasize performance-related feedback have been shown to increase exam
Vol. 84 No. 11 November 2007
•
Journal of Chemical Education
1861
Information
•
Textbooks
•
Media
•
Resources
scores and increase student self-confidence (14–16). Students participating in Web-based practice testing score higher on exams given for credit than others (17–19). We employed Web-based testing with performancerelated feedback in a mastery framework. A Web server graded most items providing individualized feedback. Essay item reList 1. Standardized Topics Incorporated in the Subject Courses Content Topics
Pedagogy Topics
Principal Topics
Inquiry
Practical Applications
Descriptive
Science Integration
Safety Culture
Mathematics Integration
Misconceptions
Home Labs
Environmental
Graphing Calculator Lesson
Historical
Probes Lesson
Demonstrations
Industrial
Experiments
Simulation
Content Integration
Project Implementation
Mathematics Integration Graphing Calculators Probe Experiments Simulations Special Emphasis Resources
List 2. List of Course Topics a Structure and Properties of Matter: Water and Solutions Structure and Properties of Matter: Periodicity Structure and Properties of Matter: Bonding and Structure Structure and Properties of Matter: Carbon Chemistry and Polymers Structure and Properties of Matter: Gases and the Atmosphere Structure of Atoms: Nuclear Chemistry Chemistry of Life Processes: Biomolecules Chemistry of Life Processes: DNA Chemistry of Life Processes: Energy and Metabolism Chemical Reactions: Acids and Bases Chemical Reactions: Oxidation, Reduction, and Electrochemistry Equilibrium: Unifying Theme Interactions of Matter and Energy Conservation of Energy and the Increase in Disorder: Thermodynamics Inquiry and the Nature of Science: Analysis and Instrumentation a
All course titles begin with “Chemistry for Secondary School Classrooms.”
1862
Journal of Chemical Education
•
sponses were stored and subsequently graded by an expert. All items were graded as “pass” or “no pass” where students had the option to retake any particular item until a passing mark was earned. Demonstrating mastery of the course material was based upon passing a large percentage of the quiz items. Students earned a grade of “A” by completing greater than 93% of the quiz items or a grade of “B” by completing greater than 87% of the quiz items. Since grades below “B” are not typically counted for graduate coursework, other letter grades were not offered. Any particular course was considered complete and a final grade was posted only when a student indicated that they were done taking quizzes.
Fifteen PCK chemistry courses were developed and made available at the graduate level to anyone with Web access. (20). One semester-hour of graduate credit in either Chemistry or Teaching, Learning, and Teacher Education per course is available. The courses were designed to be flexible components for inclusion in existing graduate degree programs. In fact, the University of Nebraska–Lincoln (UNL) counts up to one-third of a masters degree in education for these courses. Course content was representative of undergraduate or early graduate work and included the integration of chemistry with other disciplines (especially mathematics) coupled with practical teaching examples. List 1 lists standardized topics; List 2 lists courses and specific content. Early versions of courseware applications such as the now merged BlackBoard and WebCT were deemed unsuitable for this project as neither supported much online testing, and the testing that was supported was not mastery oriented (i.e., repeatable). While some authoring was needed, much high quality Web material was freely available at the project’s outset. Our model embedded existing Web materials within an assessment system. Our plan was for mentors to use these courses without the burden of development and maintenance; over 30 mentors were recruited from across the U.S. Technical aspects of the software system are provided elsewhere (21, 22). Courses were divided by “topics” that represented major concepts of the chemistry content, cross-disciplinary science themes, and related pedagogy. Participants selected topic titles from a coded, indented list on a selector page. Clicking a title brings up a content page that may or may not have links to display quiz items. There also are links to in-depth content, often content that is external to the course and project. The pages created at log-in are labeled with the participant’s name, date, and percentage completion in the course. Quiz links change color as quiz items are “passed”. While a quiz is always accessible, once passed the quiz results are no longer recorded. Participants log on using an e-mail address and a password that is assigned by the course administrator at the time of registration. Upon initial registration, participants are enrolled for one or more courses. Our system tracks as course test items move from the “not done” to the “done” category. All responses are time stamped and recorded. Ten quiz item formats were developed. Nine of these are processed automatically; the participant responds and immediately receives pro-reflective feedback with an indication of
Vol. 84 No. 11 November 2007
•
www.JCE.DivCHED.org
Information
•
Textbooks
•
Media
•
Resources
Figure 1. Map of the continental United States of America and Hawaii (inset) showing the distribution of participants by state and the credits earned in each state.
Figure 2. Frequencies of participants earning credit as a function of units of credit.
whether the item is passed. The item formats include fill in the blank, multiple choice, rank, matching, check (sort several foils into one of two lists), true or false, select the correct representation, and multiple labeling (sort several foils into several lists). Each short-answer item is programmed uniquely; all other items are managed by templates. Upon receiving responses—especially for short-answer items—detailed, worked-out solutions using problem specifics are calculated and returned. Often millions of variations are possible. Upon submitting an essay, the participant immediately receives a model response. Course mentors access essay responses via the Web, evaluate them, and provide feedback along with a pass–no pass assessment. Though not immediate, responses to essays are typically provided in one–two days. An essay reader has the option of storing a response such that the response can be entered for subsequent students by that reader through clicking. We recruited project participants through notices posted on chemistry teacher discussion lists, presentations at regional and national meetings, letters sent to state departments of education, and an announcement in an NSTA publication.
indicate PCK-focused coursework (rather than just pedagogyfocused coursework). The extensive evaluation conducted by Cohen and associates and numerous e-mail communications from participants suggest very positive teacher satisfaction (23). The number of courses completed per participant is shown in Figure 2. Participant work is seasonal, with high completion rates during spring break periods and summer, but standstills over the winter break. The impending removal of the tuition incentive caused work to be completed at a feverish pace, likely enhanced by a fortuitous overlap with various spring recess schedules. Once the courses went into a tuition-based format, use slowed to about 50 credits per year. The most serious problem we encountered centered on the stability of Web resources. At the time we proposed this project, quality core content (such as how a glass pH electrode works or a description of the third law of thermodynamics) had been consistently available on the Web. As courseware such as BlackBoard and WebCT improved, faculty Web use increased and valuable learning materials were hidden behind firewalls. The loss of hyperlinked resources that we did not control was significant and unanticipated. We use the term link-rot to describe hyperlink failure caused by changing or disappearing Web-based resources and have studied the phenomenon quantitatively through this project (24, 25). As a result of link-rot, our courses needed significant maintenance beyond simply updating, correcting, and responding to participant performance.
Project Outcomes Participants come from throughout the United States. During the grant-supported initial phase of this project, 106 participants from 27 states earned a total of 508 credits (Figure 1). The first course, Water and Solutions, came online in April 2000. The final course, Thermodynamics, was released in late February 2003. Table 1 shows the course completions by total enrollment, course, and credit awarded. Initially, 508 courses were completed with 504 students receiving A’s and 4 students receiving B’s. Many participants completed 100% of the items. Patterns emerged when the items left undone by participants were examined. For example, essay items tended to be undone and requiring completion of these items may have improved the quality of the learning experience. Participants chose science over education credit (9:2) suggesting an interest in having transcripts www.JCE.DivCHED.org
•
Issues Related to Participation
Recruiting Participants No forms of recruiting have proven especially effective. High school teachers were very involved at all stages of our project and their suggestions weighed heavily in our choices for venues to use in recruiting potential students. Even when no tuition was involved, delay times between initial participant contact and enrollment were large; often prospects would e-mail indicating that they had been at a presentation two years earlier. Brochures produced low yields. Postings of
Vol. 84 No. 11 November 2007
•
Journal of Chemical Education
1863
Information
•
Textbooks
•
Media
•
Resources
available courses on listservs also were low yielding. Currently the courses are listed and described in University Extended Education and Outreach catalogs (20). The best recruiting sources were self-selection resulting from Web searches conducted by the participants and word-of-mouth from successful participants. It is reasonable for projects like this to include a full-time recruiter.
Course Completion Factors During the period of 03Ⲑ01Ⲑ–04Ⲑ30Ⲑ2002, participants completed 30 courses. On 02Ⲑ27Ⲑ2003, participants with active enrollments were informed that the free-tuition program would end on 4Ⲑ30Ⲑ2003. During the period 03Ⲑ01Ⲑ– 4Ⲑ30Ⲑ2003, participants completed 139 courses. We conclude that tuition is an enormous factor in participant decisions to complete our courses. A survey of participants enrolled between 11Ⲑ01Ⲑ2002 and 03Ⲑ01Ⲑ2003 focused on tuition. Of the 28 participants in this group, 20 responded. Of those, 80% (16 of 20) indicated that tuition was of major positive importance in their decision to take a course. Of 247 participants (participants signed up after an exchange of e-mail), only 106 successfully completed a course. Although the completion rate has been higher now that tuition is involved, it still is surprisingly low. Apparently this is not unusual in such self-paced distance courses, however. Pro-
spective participants were asked to review a course to gain a feel for the system, and then to select a starting course based on personal comfort. When participants enrolled, they received an extended e-mail response with deadline dates and other information. After enrollment, many were never heard from again; participants had no financial or academic risk, however. While we have speculated about reasons for this, it remains a mystery. The completion rate for courses starting in January 2002 exceeded that for those starting in May 2001, yet many later course registrations came from previously successful participants. The cost per unsuccessful participant is very low, probably averaging 15 minutes per person per course: basically, the time it takes to respond to a few e-mail messages. It is not at all like conventional instruction where a participant signs up and takes a place that might be used by someone else. Few of the participants felt connected to a local faculty person for mentoring. Those identifying mentors frequently did so by looking at a map we provided that listed mentors volunteering at the project outset. The 22 mentors came from California (4), Nebraska (3), Texas (3), Ohio (2), and one each from Hawaii, Indiana, Kansas, Kentucky, Massachusetts, Missouri, Mississippi, New York, South Carolina, and Tennessee. Most participants seemed happy to be able to use a UNL faculty mentor. One of the original partners continues to offer the courses and recruit students. Because the
Table 1. Distribution of Course Completion Data by Major Field of Study Number of Participants Earning Credits in These Fields Course Titles (Number of Participants)
Chemistry
Biochemistry
Biology
Curriculum and Instruction
Total Credits Earned
Total Completed (%)
Water and Solutions0 (118)
054
11
065
055
Nuclear Chemistry0 1(91)
043
13
056
062
Periodicity0 1(62)
048
09
057
092
Bonding0 1(27)
022
05
027
100
Organic and Polymer Chemistry0 1(33)
020
02
022
067
Energy and Matter0 1(57)
039
08
047
082
Molecular Biology0 1(21)
010
02
014
067
Gases and Atmoshpere0 1(52)
036
09
045
087
Biomolecules0 1(57)
016
3
3
02
024
042
Energy and Metabolism0 1(17)
008
1
3
02
014
082
Acids and Bases0 (122)
051
13
064
052
Chemical Equilibrium0 1(32)
022
05
027
084
Redox0 1(22)
012
03
015
068
Thermodynamics0 1(13)
006
03
009
069
Instrumentation0 1(33)
017
05
022
067
Totals (757)
404
92
508
67
1864
Journal of Chemical Education
•
1
5
1
7
Vol. 84 No. 11 November 2007
•
www.JCE.DivCHED.org
Information
mentoring was an add-on to base load, most mentors are not interested in continuing.
Academic Honesty Academic integrity is a legitimate issue. The likelihood of persons other than participants responding is small. This is a “low-stakes” testing situation. The amount of time required to potentially engage in such a strategy is quite large since so much testing is involved. It is unlikely that one can find an expert willing to spend time learning some of the specific content such as the details of a particular demonstration or experiment. A person sufficiently knowledgeable on course content would likely be another chemistry professional or peer and would not be approached for such dishonest activity. Two participants have logged onto our public site, sought answers, and then logged onto tests and regurgitated answers, a process unlikely to produce any kind of meaningful learning. We devised a software-based tracking system to detect this activity and alert the course administrator. Thus far, when such inappropriate activities have been called to a participant’s attention by e-mail, the inappropriate behavior has stopped. Moreover, most participants send e-mail asking for additional help or explanation when an answer is scored as incorrect. There is much more evidence of desire to learn than of lack of integrity.
3.
4.
5. 6. 7. 8. 9. 10. 11.
12. 13.
Recommendations
14.
Automated practice in a mastery-learning framework has the potential for improving most, if not all, science courses by affording a vehicle for setting and achieving minimum learning goals. Coupled with other course elements, automated practice can be an extremely powerful teaching and learning tool. We make four recommendations for developing Webbased courses to benefit or influence in-service teachers. 1. Reduce or eliminate tuition costs if at all possible. 2. Allow substantial time for course development and early testing (formative evaluation). 3. Address the loss of Web-based resources (link-rot) by making all of the materials required for the courses available on your servers; do not depend upon the stability of external Web resources as part of your courses. 4. Finally, include one person whose full-time responsibility is recruiting participants and keeping records.
15. 16. 17.
18. 19. 20.
21.
Acknowledgments This material is based on work supported by the National Science Foundation (NSF) under Grant No. ESI-9819377. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of NSF. Literature Cited
23.
24.
1. King, P.; Hildreth, D. J. Coll. Sci. Teach. 2001, 31, 112–115. 2. Brooks, D. W.; Nolan, D.; Gallagher, S. Web-Teaching: A Guide to Designing Interactive Teaching for the World Wide Web,
www.JCE.DivCHED.org
22.
•
25.
•
Textbooks
•
Media
•
Resources
2nd ed.; Kluwer AcademicⲐPlenum Publishers: New York, 2001. Darling-Hammond, L.; Ball, D. L. Teaching For High Standards: What Policymakers Need To Know and Be Able To Do. http://www.eric.ed.gov/ERICDocs/data/ericdocs2sql/content_ storage_01/0000019b/80/17/35/9f.pdf (accessed Aug 2007). Examining Pedagogical Content Knowledge: The Construct and Its Implications for Science Education, Gess-Newsome, J., Lederman, N. G., Eds.; Kluwer: Dordrecht, The Netherlands, 2001. Bloom, B. S. Human Characterisitics and School Learning; McGraw-Hill: New York, NY, 1976. Kulik, C. C.; Kulik, J. A. J. Educ. Tech. Syst. 1987, 15, 325– 345. Kulik, J. A.; Kulik, C. C.; Carmichael, K. Science 1974, 183, 379–383. Kulik, J. A.; Kulik, C. C.; Bangert-Drowns, R. L. Rev. Educ. Rsch. 1990, 60, 265–299. Kulik, J. A.; Kulik, C. C.; Cohen, P. A. Am. Psychol. 1979, 34, 307–318. Lhyle, K. C.; Kulhavy, R. W. J. Educ. Psychol. 1987, 79, 320– 322. How People Learn: Brain, Mind, Experience, and School, Bransford, J. D., Brown, A. L., Cocking, R. R., Eds.; National Academy Press: Washington, DC, 1999. Loegering, J. P.; Edge, W. D. J. Coll. Sci. Teach. 2001, 31, 252–257. Riffell, S. K.; Sibley, D. H. J. Coll. Sci. Teach. 2003, 32, 394– 399. Mestre, J.; Hart, D. M.; Rath, K. A.; Dufresne, R. J. Comput. Math. Sci. Teach. 2002, 21, 229–251. Penn, J. H.; Nedeff, V. M.; Gozdzik, G. J. Chem. Educ. 2000, 77, 227–231. Brooks, D. W.; Schraw, G.; Crippen, K. J. J. Chem. Educ. 2005, 82, 641–644. Pilant, M. S.; Hall, R. J.; Epstein, J.; Hester, Y.; Strader, R. A. In Proceedings of International Conference on Mathematics and Science Education and Technology; San Diego, CA, 2000; p 334–339. Crippen, K. J.; Earl, B. L. J. Comput. Math. Sci. Teach. 2004, 23, 151–167. Crippen, K. J.; Earl, B. L. Comput. Educ. 2007, 49 (3), 809– 821. Pedagogical Content Courses for High School Chemistry Teachers (16 are now available). http://dwb4.unl.edu/Chem/ ChemPed.html (accessed Aug 2007). Brooks, D. W.; Brooks, H. B.; Abuloum, A.; Crippen, K. J.; Markwell, J. A System for Delivering Web-based Courses Emphasizing Automatic Assessment. http://www.eclipse.net/ ~pankuch/Newsletter/Pages_NewsF03/F2003_News.html (accessed Aug 2007). Brooks, D. W. The Evolution of an Automated Practice System. http://www.eclipse.net/~pankuch/Newsletter/Pages_News F05/F2005_News.html (accessed Aug 2007). Cohen, K. C.; Schmeeckle, J. WWW-based Graduate Postcertification Teacher-Training in Chemistry External Evaluation Final Report. http://dwb.unl.edu/Chem/NSF/ FinalRepKC.pdf (accessed Aug 2007). Markwell, J.; Brooks, D. W. J. Sci. Educ. Tech. 2002, 11, 105– 108. Markwell, J.; Brooks, D. W. Biochem. Mol. Biol. Educ. 2003, 31, 69–72.
Vol. 84 No. 11 November 2007
•
Journal of Chemical Education
1865
