Chemistry Outreach Project to High Schools Using a Mobile Chemistry

Article pubs.acs.org/jchemeduc

Chemistry Outreach Project to High Schools Using a Mobile Chemistry Laboratory, ChemKits, and Teacher Workshops Gary L. Long,*,† Carol A. Bailey,‡ Barbara B. Bunn,† Carla Slebodnick,† Michael R. Johnson,† Shad Derozier,† Susanne M. Dana,§ and Julie R. Grady⊥ †

Department of Chemistry, ‡Department of Sociology, Virginia Tech, Blacksburg, Virginia 24061, United States Blacksburg High School, Blacksburg, Virginia 24060, United States ⊥ Rural STEM Education Center, Arkansas State University, State University, Arkansas 72467, United States §

S Supporting Information *

ABSTRACT: The Chemistry Outreach Program (ChOP) of Virginia Tech was a university-based outreach program that addressed the needs of high school chemistry classes in underfunded rural and inner-city school districts. The primary features of ChOP were a mobile chemistry laboratory (MCL), a shipping-based outreach program (ChemKits), and teacher workshops. ChOP targeted schools that lacked basic chemistry equipment and resources, as well as schools where science teachers had limited training in chemistry. The MCL, a fully equipped chemistry laboratory in a 78-foot-long tractor−trailer, visited high school chemistry classes on a regularly scheduled basis. From 2000 to 2004, the MCL served 38 high schools in which 9100 students performed 36,200 experiments. ChemKits, working in concert with the MCL, resulted in an additional 23,450 experiments conducted by students in 33 high schools from 2002 to 2004 and 17 high schools in 2004 to 2005. Teachers attended ChOP workshops in the summer prior to using the MCL or ChemKits in their classrooms. As a result of the collaboration between ChOP and high schools, participating chemistry classes posted an average 37-point gain in their pass rate on standardized state exams in chemistry over a three-year period. KEYWORDS: High School/Introductory Chemistry, Chemical Education Research, Curriculum, Public Understanding/Outreach, Testing/Assessment FEATURE: Chemical Education Research

I

n 2000, the Chemistry Outreach Program (ChOP) of Virginia Tech was created to address the needs of rural high schools in southwestern Virginia (Appalachia), southern Virginia, and inner-city Richmond, Virginia. ChOP was developed by the Department of Chemistry and the College of Arts and Sciences of Virginia Tech in response to the Standards of Learning (SOL) exams of the Virginia Department of Education.1 The state guidelines emphasize the “investigation of the interactions of matter and energy through the use of laboratory techniques, manipulation of chemical quantities, and problem solving applications”1 and “technology, including graphing calculators, probeware, and computers [being] employed where feasible.”1 The schools served by ChOP could not address these requirements, in large part because of their inadequate laboratory facilities. Before the introduction of ChOP in 2001, the 18 schools scored 16% points lower on the pass rate of the SOL exams than the state average. In the following years, they were statistically the same as the state average. The Chemistry Outreach Program was designed to increase the chances of students and teachers meeting the requirements of the chemistry SOL exams. ChOP consisted of four major components:

2. The ChemKits program 3. Teacher workshops 4. Curriculum developed by college faculty members and high school teachers that was aligned with the state standards The mobile chemistry laboratory (MCL) and ChemKits provided a modern chemical laboratory setting, instrumentation, and resources that were otherwise unavailable, thereby allowing students to conduct “hands-on” scientific inquiry. During summer workshops, the teachers were trained in chemistry theory, chemistry pedagogy, and performed ChOP laboratory experiments. The ultimate goal was to translate experience with ChOP activities into improvements in the students’ understanding of chemistry, as measured by standardized state exams. The program evaluation found that teachers perceived their knowledge of chemistry and chemistry pedagogy increased, and students’ scores on state-mandated, standardized chemistry exams improved considerably during the time the schools participated in ChOP.

1. The mobile chemistry laboratory © 2012 American Chemical Society and Division of Chemical Education, Inc.

Published: July 19, 2012 1249

dx.doi.org/10.1021/ed200185e | J. Chem. Educ. 2012, 89, 1249−1258

Journal of Chemical Education

Article

Table 1. Experiments and Related Standards in the Chemistry Outreach Program

a b

Topic Category

Experimenta

Chemistry SOL Standardsb

Acid−base chemistry Acid−base chemistry Acid−base chemistry Acid−base chemistry Acid−base chemistry Acid−base chemistry Acid−base chemistry Electrochemistry Electrochemistry Electrochemistry Environmental chemistry Environmental chemistry Formulas and moles Formulas and moles Formulas and moles Formulas and moles Formulas and moles Kinetics Kinetics Physical measurements Physical measurements Physical measurements Physical measurements Physical measurements Physical measurements Physical separation Physical separation Physical separation Physical separation Spectroscopy Spectroscopy Spectroscopy Spectroscopy Spectroscopy Thermodynamics

Acids, bases, indicators Analysis of drain cleaners End point determination by conductivity How effective is your antacid? Ka of a weak acid Red cabbage leaves What makes a solution buffered? Electrochemical cells Electrolysis of water How much vitamin C is in your...? Acid rain Water analysis using Hach kitsa Equilibrium: LeChâtelier’s principle Limiting reagent Solubility of ionic solids I and IIa Stoichiometrya Synthesis of esters Graham’s law of effusion The clock reaction Conductivitya Density Freezing point depression Gas laws I and II Like dissolves like Nuclear chemistry Distillation Gas chromatography How much fat is in your chips? Thin layer chromatographya Determination of an equilibrium constant Infrared spectra of esters UV−vis determination of caffeine in sodas UV−vis spectra of sunscreens Visible spectra of commercial dyes Heat of reaction

1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1, 1,

4fg 4efg, 5g 4fg 4efg, 5g 4fg 4efg, 5g 4fg 2h, 3e 2h, 3e 2h, 4e 4g, 6c 2h, 4g, 6c 3a−cf, 4b 3a−c, 4b 3a−c, 4b 3a−c, 4b 3a−c, 4b, 6a 2af, 4d 3fg, 4e 3d, 5g 5 5h 4d, 5a−d 4e, 5g 2a−c, 6b 5adg, 6a 2ah, 5g, 6a 2h, 6a 2 h, 3cd, 5g, 6a 2h, 4f 2h, 6a 2h, 4e 2h, 6a 2h 3a−ce, 5f

Special Equipmentc Computer, pH probe Computer, pH probe Computer, conductivity probe Computer, pH probe Computer, pH probe  Computer, pH probe  Computer, voltage probe Computer, voltage probe  Hach water kits    Balances  Effusion setup, pump  Conductivity probes   Computer, P and T probes  Computer, radiation monitor Distillation setup Gas chromatographs  TLC plates Computer, spectrometer IR spectrometer Computer, spectrometer Computer, spectrometer Computer, spectrometer Computer, T probe

These were ChemKit experiments. Virginia Chemistry Standards of Learning (SOL):

1. Experimentation: (a) laboratory techniques; (b) safety; (c) emergency response; (d) multiple variables; (e) data recording and analysis; (f) error analysis; (g) mathematical manipulations.

2. Periodic Table: (a) mass/atomic number; (b) isotope/half-life/nuclear particles; (c) particle/mass charge; (d) families/groups; (e) series/periods; (f) trends/patterns (radii, etc.); (g) electron configuration, oxidation numbers; (h) chemical/physical properties; (i) historical/quantum models.

3. Chemical Formulas and Equations: (a) nomenclature; (b) balancing equations; (c) writing chemical formulas; (d) bonding types (ionic, covalent); (e) reaction types/thermodynamics; (f) physical and chemical equilibria; (g) kinetics.

4. Four Molar relationships: (a) Avogadro’s principle, molar volume; (b) stoichiometric relationships; (c) partial pressure; (d) gas laws; (e) solution concentration; (f) chemical equilibrium; (g) acid/base/pH, electrolyte, titration.

5. Phases of Matter: (a) pressure, temperature, volume; (b) vapor pressure; (c) partial pressure; (d) phase changes; (e) molar heats of fusion/ vaporization; (f) specific heat capacity; (g) solutions; (h) colligative properties.

6. Fields of Chemistry: (a) organic and biochemistry; (b) nuclear chemistry; (c) environmental chemistry. c

Special equipment means any equipment required for the experiment that is not found in the typical classroom.

■

project, led by Auburn University.4 During the year prior to the implementation of the MCL, meetings were held with high school teachers in the regions to evaluate and design labs. Experiments were adapted from the Science in Motion Project, papers in the Journal of Chemical Education, and published laboratory manuals. The faculty and staff from the Juniata and Alabama projects were generous with sharing ideas, experiments, and logistical information on outreach to ChOP. The design of the ChOP curriculum took into account the constraints of a high school chemistry laboratory period and the SOL standards.1 The time allowed for setup, student

THE CHEMISTRY OUTREACH PROGRAM The initial development of ChOP began a year before the MCL was placed into service. Primary tasks included creating the curriculum and designing the first teacher workshops. Curriculum

Mobile labs (van-based) and chemistry kit programs have been effectively used in secondary school outreach and are documented in the literature.2 The primary models used during the development of the MCL were the successful Science in Motion Project of Juniata College,3 and the Alabama statewide 1250



Article

experiments to be delivered to the high schools. The ChemKit program consisted of experiments not requiring advanced instrumentation. The template for the ChemKit program was the successful “Biotech in a Box” program of the Fralin Center at Virginia Tech.5 A breakdown of the MCL and ChemKit experiments is also shown in Table 1. Information as to which SOL topic is addressed and what equipment is required is presented in this table. More specific information is also available on the ChOP Web site.6 ChemKits provided 23,450 experiments for students to conduct in 33 high schools from 2002 to 2005 and 17 high schools in 2004 to 2005. A summary of these data is shown in Table 2. In the 2003−2004 academic year, 38 high schools were visited by the MCL four times, and 24 of these schools elected to participate in the ChemKit program, receiving four to six ChemKits during the school year. An additional nine schools participated in the ChemKits program but did not receive MCL support; these schools were outside of the MCL service area (more than 200 miles from the university campus). Whether using the MCL or ChemKits, teachers requested specific experiments from the listing of available labs at the beginning of the academic year. The travel schedule for the MCL and the shipping schedule for the ChemKits was based on the teachers’ selections.

instruction, and completion of a high school chemistry laboratory at the schools initially served by ChOP was one class period (∼55 min). The ChOP team of faculty members and educators that designed the experiments for the MCL and ChemKit programs were cognizant of this limitation and focused on fundamental experiments that could be completed in a timely fashion. A collection of 35 experiments was assembled, each of which supported key SOL topics, as the close alignment with the SOL exams was an important factor in generating teacher participation. In the following two years, this collection was trimmed to 29 MCL experiments, with the remaining six experiments transferred to the ChemKits program. A listing of the experiments is found in Table 1. In addition to those who helped develop the curriculum, a team of six individuals became associated with the project. The team members included five university staff members (two teachers, a driver, a lab technician, and an administrative assistant) and one faculty member. Some team members worked on the curriculum during the academic year, and some served as instructors in the summer workshop. The Mobile Chemistry Lab

The mainstay of this outreach effort was the MCL, a 78-footlong tractor−trailer that was a self-contained chemistry laboratory. (See Figure 1 for a picture of the MCL. A schematic is

Teacher Workshops

Considerable evidence indicates that professional development workshops for teachers enhance science knowledge, pedagogical skills, teaching self-efficacy, effectiveness, and outcome expectations.7−11 Thus, an important feature of ChOP was the teacher workshops. Teachers wishing to use the MCL or ChemKits were required to attend a weeklong summer workshop, hosted at the university. Over a three-year period, seven workshops were offered. These included basic workshops, where teachers were introduced to the experiments and laboratory equipment, advanced workshops (for those teachers who had completed a basic workshop) in which teachers would design or adapt experiments for their classrooms, and a ChemKit workshop. Teachers had the option of taking the workshop for graduate credit or for receiving hours toward their reaccreditation. The ChOP workshops were designed to be particularly useful for teachers that were teaching out-of-field, had minimal knowledge of chemistry, had training in chemistry but wanted to refresh and build upon their previous education, or lacked experience with the state-of-the-art equipment on the MCL. Goals of the workshops also included increasing teachers’ selfefficacy related to teaching chemistry, and creating networks among the teachers and university personnel that would extend beyond the summer workshops. The workshops focused on the theory and operation of the chemical experiments that could be conducted on the MCL, chemistry fundamentals, experiments suitable for preparing students for the SOL exam, and pedagogical strategies for teaching chemistry. Hands-on experience with the computers and probeware in the MCL was provided. A total of 44 teachers were trained in seven summer workshops from 2000 to 2003, with 13 teachers receiving training on the ChemKits. The advanced workshop was attended by 18 returning teachers of the 44 total teachers; this is a possible indicator of the teachers’ positive assessment of the first workshop. Follow-up training such as that provided by ChOP has been shown to be important to maintain and enhance the

Figure 1. Photograph of the MCL tractor and transporter.

contained in the Supporting Information.) The MCL began service to 18 high schools in the fall of 2000. The following year, another seven high schools added the MCL to their curriculum. In the third year of operation, an additional 13 schools adopted the use of the MCL. Between the years 2000 and 2004, the MCL provided the central laboratory experience for 11th grade chemistry students in 38 high schools, reaching the capacity of the program. Nearly 9100 high school students used the mobile laboratory, with 36,200 experiments being performed over three academic years. The data on experiments performed are presented in Table 2. Table 2. Numbers of Student-Conducted Experiments and Schools Served from 2000 to 2005a Academic Year

MCL SCE

Schools

CK SCE

Schools

2000−2001 2001−2002 2002−2003 2003−2004 2004−2005 Total

8492 9510 9482 8742  36,226

18 25 38 38 

  7948 8786 6716 23,450

  33 33 17

a

MCL is Mobile Chemistry Lab; SCE is Student-Conducted Experiments; CK is ChemKit Programs.

ChemKits

Beginning in Fall 2002, ChemKits were added to the outreach program. This shipping-based addition allowed six “low-tech” 1251



Article

benefits of initial workshops.12 ChOP advanced workshops covered the testing and creation of new experiments, as well as the revision of existing experiments. Teachers were introduced to advanced experiments that could be conducted by students in advanced placement chemistry classes and chemistry II classes.

of students from low-income families, minority households, or both. Hence, the regions served by ChOP were economically depressed, and the participating schools reflected the gross inequalities that exist in school funding and educational opportunities in Virginia. A clear manifestation of the lack of funding is the dismal state of the chemistry laboratories in these schools. On an evaluation questionnaire, teachers indicated that their schools had inadequate resources for effective chemistry education, such as no laboratory space, laboratories with no safety hoods and water, chemicals that were unsafe because of their age, no money for supplies, and outdated equipment. One teacher described his lab as “one beaker”. Many of the teachers had laboratory budgets of $50 to $200 for the school year. Their students were severely hindered or prevented from meeting some SOL requirements, such as using and understanding appropriate “technology, including computers, graphing calculators, and probeware for gathering data and communicating results”1 because the schools did not have this equipment. Past evidence indicates that the lack of resources for chemistry education would have continued without ChOP.

■

PROGRAM EVALUATION The program evaluation of the ChOP was based on data from the teacher workshops and from the SOL exam scores of the participating high schools. The evaluation of the workshops was conducted by the second author, who was not associated with ChOP, other than her role as outside evaluator. Data for the evaluation of the workshops were collected through nonparticipant observation, semistructured and unstructured interviews, focus groups, and self-administered surveys. Data were obtained from the high school teachers and ChOP personnel. Another source of data used in the evaluation was scores on the state administered SOL exam in chemistry.1 Students were not the focus of the evaluation, per se; for example, their perceptions of the program and the experiments were not included in the analysis. Rather, the unit of analysis for much of the evaluation was the pass rate for groups of schools or individual schools; although obviously, it was the students’ performances on the SOL exams that were aggregated to obtain the published pass rate for the schools. The methodology employed in the evaluation prevents the assertion that ChOP solely caused the increases in SOL exam scores; no doubt a series of intervening, antecedent, and other variables remain unspecified. For example, we cannot speak to the effect of ChOP on teacher preparation prior to laboratory experiments. Further, we have no direct data on possible changes in students’ attitudes toward science or effort. As seen in Figure 1, the MCL was physically impressive: it may well be the case that the overwhelming presence of the 78-foot-long tractor−trailer contributed to the increased attendance on MCL days, which could then translate into higher scores as well as the reported greater enrollments of students in chemistry the following year. Nonetheless, independent of other possible causes, we suggest that ChOP was a primary factor in the significant increases in the SOL pass rate for participating schools and that ChOP was successful in meetings its primary goal of improving science education.

Context for Teacher Workshops

Perennial concerns in education include the low retention rates of teachers, lack of teacher certifications and qualifications, and insufficient numbers of teachers in some locations and in some subject areas. These factors are closely related. For example, there is a shortage of qualified science and mathematics teachers in lowincome areas, particularly in urban centers, where the attrition rate for teachers is higher than in other locations.15,16 Out-of-field teaching is also a problem that has gained considerable attention during the past decade.17 Teachers in schools in areas with high poverty rates are less likely to have majored or minored or had minimal academic training in the subject they are assigned to teach than teachers in other areas.15,17,18 Particularly relevant for ChOP are the data from high-poverty areas in Virginia. In 1999−2000, 38% of teachers in high-poverty schools did not have a minor in the courses they were teaching.17,19 The vast majority of the teachers involved with the MCL were teaching out-of-field. Only four had a degree in chemistry. Most of the teachers had degrees in biology, with earth science as the next most common degree. The remaining teachers had degrees in physical education, except for one who majored in English. Out-of-field teaching has been shown to partly account for lower levels of achievement in sciences in low-income areas.11 Teachers who either lack training in the areas they are assigned to teach or lack confidence in their ability to teach a particular subject have lower expectations for student learning, and low expectations have consistently been shown to be associated with low achievement.20,21 It is overstated to assert that the lack of certification and training and out-of-field teaching necessarily means ineffective teaching. Still, it is arguably the case that otherwise highly qualified teachers may be less successful when not adequately trained in the content and pedagogy of the subjects they have been assigned to teach.15,17,18,22 Further, even teachers who majored in sciences can fall behind in content knowledge, loose motivation and enthusiasm, lack self-efficacy, and be unaware of pedagogical advancement in science education, all which further disadvantage their students. Many of the concerns discussed above apply to the schools and teachers involved with ChOP.

Context of the Standards of Learning Exam in Chemistry

As defined by the Commonwealth of Virginia, the standards of learning are a series of “content rich, grade by grade academic standards that define what teachers have to teach and what students were expected to learn”.13 These standards were developed using guidelines from Harcourt Brace Educational Measurement and the input from members of the Content Review Committees. The SOL exams were phased into Commonwealth’s curriculum in 1995. Students must pass a certain number of SOL exams to receive an accredited high school diploma. The SOL exams are similar to Tennessee’s Gateway Examinations in chemistry; in 2010, nine other states had similar end-of-class exams in either or both biology and science. By 2012, 28 states will have end-of class exams or comprehensive assessments.14 Context of the Service Area

■

The service area of the MCL was southwestern Virginia (Appalachia), southern Virginia, and inner-city Richmond, Virginia. Although southwestern Virginia schools serve a predominately rural population, schools served in southern Virginia and inner-city Richmond were populated with a large number

EVALUATION OF TEACHER WORKSHOPS The overall conclusions were that ChOP provided excellent workshops: they were considered to be well planned, well 1252



Article

groups were formed for the 18 schools using ChOP for a threeyear period, and for the 9 schools that only used the ChemKit program. The number of peer schools equaled the number of MCL or ChemKit schools in each statistical study. Statistical analyses of these data were accomplished with a one-tailed t-test. In that we wish to show trends in the data, α = 0.10 was selected for the statistical tests. The calculations were performed using “form 2” of the t-test when comparing the ChOP groups versus the peer groups. Form 1 of the t-test was used for comparing the groups versus the state average and for determining whether the difference from the first year of usage to the last year was significant (d = 0). A complete table of the statistical analyses is contained in the Supporting Information. The data were analyzed in groups, based on the years of service by ChOP. The three-year group was composed of the 18 schools that used the program for three years and was further divided into subgroups A and B. The four schools with the lowest initial pass rate on the SOL chemistry exam comprised Group A. These four schools were predominately minority populated schools. Group B consisted of the remaining 14 schools that used the program for three years. The second group of schools, the two-year group, was made up of the 7 schools that used ChOP for a two-year period. The third group, the one-year group, was composed of the 13 schools that used ChOP for a single year. Lastly, the ChemKit Group was made up of 9 schools that had access only to the ChemKit mailing program. The data from this study are presented in Tables 3−8, and visualized in Figures 2−5. The tabulated data show the schools’ performance on the SOL chemistry exam for the year prior to the introduction of ChOP. Schools are referred to as MCL schools or CKO schools for schools receiving ChemKits only. The schools that used both the MCL and the ChemKits are still referred to as MCL schools in this analysis. Figures 2−5 also include data from the participating schools, from peer schools in the same geographic area, and socioeconomic strata at the participating schools, and the state SOL exam average in chemistry.

executed, and consistent with the research on the characteristics of effective professional development workshops.13,16 The selection of labs was appropriate, clearly connected to the SOL, and applicable to the teachers’ classes. Teachers were asked to rate the basic workshop on a scale of 1 to 7, with 7 being the most positive response. The average response was 6.9. Teachers attending the advanced workshop in 2002 also rated the workshop as 6.9 on a scale of 1 to 7. Almost all teachers strongly agreed with the statement that their knowledge of chemistry improved during the workshop. Teachers also mentioned learning in the open-ended questions on the evaluation survey. The following quotes exemplify their responses: I learn more chemistry doing these labs than I do any formal course! [Teacher 16] A great opportunity to learn new things and refresh prior knowledge. [Teacher 9] Greatly increased knowledge and decreased apprehension about more sophisticated equipment (gas chromatography, UV−vis). [Teacher 25] In response to another open-ended question, some teachers wrote: It has been a challenging workshop but one that really can be used in the classroom with the presence of the MCL. [Teacher 4] The information is the most directly applicable to my classroom activities and lesson plans. [Teacher 20] It is very informative for the preparation of my curriculum guide for this year. [Teacher 2] It allowed the chance of some excellent ideas to implement in class. [Teacher 32] This opportunity has been so important to me in helping me improve instruction. [Teacher 5] In summary, teachers uniformly agreed that the information and labs would be useful in their classes. Further, teachers indicated that their knowledge of chemistry increased and that they gained information on how to teach chemistry, both requirements for effective science education.23 In informal conversations, teachers reported they felt respected and treated as colleagues by the university personnel.

Three-Year Group

The three-year group schools consisted of 18 schools that used the ChOP program for three academic years. As shown in Figure 2 and described in Table 3, the average passing rate

■

EVALUATION OF STANDARDS OF LEARNING EXAM SCORES The second component of the evaluation was an analysis of SOL exam scores. Data consisted of the percentage of students that passed the SOL exams before involvement with ChOP and afterward for each year the schools used ChOP. These data are available from the State Board of Education for each school using ChOP for the year before involvement with ChOP and the years afterward. Statewide and peer school SOL data were also obtained. A student must pass the SOL exam (a value set by the state, nominally 70%) in order to have the course count toward an accredited diploma. Additionally, the pass rate of a school must be in alignment with the metrics of No Child Left Behind24 for the school to remain accredited in the subject. To use the pass rate as a metric in this study, constraints had to be placed upon the available data. For a comparison to be made, it is necessary that the same teacher had to be teaching chemistry at the high school over the evaluation period. Additionally, the curriculum could not change. Peer schools were selected based on geography, student body size, and the percentage of students receiving free lunch. Peer

Figure 2. Plot of the pass rate percentages for the SOL chemistry exam for the years from 2000 to 2004. The average pass rate percentages of the 18 schools that used ChOP (designated MCL) for a three-year period are shown, as well as averages from peer schools that did not use ChOP. Also shown are data for the state average pass rate percentages. The year 2000 data provide a baseline, as ChOP was not operational then. 1253



Article

SOL exam scores for the three-year group A were compared with other schools with a similar percentage of students that received federal assistance for lunch. The three-year group A average for federal lunch assistance was 44%, while the average for the peer schools was 51%. The three-year group A SOL exam pass rate was statistically the same as the peer rate in 2000; both were below the state average of 63.8%. The threeyear group A average rose significantly to 45.6% after one year of use, and to 85.9% in the third year of use. The peer group average was not statistically different from the state average in 2001 to 2003. While the three-year group A schools were 12.0% points below the peer schools in the year prior to the introduction of ChOP, they experienced a statistically significant rise of 26.3% points above the peer schools in 2003. The three-year group A schools were 44.2% points below the state average in 2000, but in 2003 they bested the state average, with all four schools achieving a pass rate of >70%. While all three-year group A schools experienced considerable improvements in their SOL exam scores, a closer look at two schools, MCL-1 and MCL-2, is warranted. These schools were inner-city schools with minority populations of 95% and higher. The 2000 scores for MCL-1 and MCL-2 were extremely poor (2.9% and 12.8%). By 2003, the pass rate for MCL-1 increased to 75.0% and 88.9% for MCL-2. The same teachers, same texts, and same syllabi were used during this period. As an interesting aside, the truancy rate for chemistry classes in these two schools fell to near zero on days when the MCL was at the two schools.25 Three-Year Group B. The individual school data for threeyear group B is shown in Table 5. These 14 schools (MCL-5 to

Table 3. SOL Chemistry Exam Average Pass Rates for Three-Year Group, Peer Group, and State

a

Averages

2000, %a

2001, %

2002, %

2003, %

3-Year MCL 3-Year Peer State

48.2 56.1 63.8

67.7 61.0 73.8

72.1 68.8 78.3

86.9 76.4 84.2

ChOP was not in service.

on the chemistry SOL exam for the three-year group in 2000 was only 48.2%; it was significantly below the state average of 63.8%, and below the peer average of 56.1% (but not statistically significantly different). In the first year of MCL use (2001), the mean pass rate rose nearly 20 points to 67.7% and then to 72.1% in the second year (2002) of MCL use. It rose to 86.9% in 2003. In this three-year period, these schools went from being statically below the state average to being the same as the state average. Peer schools experienced the opposite trend. In the first year, they were statistically the same as the state average, but in years 2001−2003 fell statistically below the state average. In year 2003, the MCL schools average was statistically higher than the peer school average. This rise in the MCL group was statistically higher that both the peer group and the state average. Three-Year Group A. The impact of the program on the three-year group requires further elaboration. The scores from the four schools of Group A are shown individually in Table 4 and displayed as a group in Figure 3. The statewide averages, as well as the results of the peer schools, are included in the figure. Table 4. SOL Chemistry Exam Pass Rates for Three-Year Group A, Peer Group, and State

a

Sites

2000, %a

2001, %

2002, %

2003, %

MCL-1 MCL-2 MCL-3 MCL-4 3-Year A MCL 3-Year A Peer State Average

2.9 12.8 30.5 32.3 19.6 31.6 63.8

34.4 60.2 57.7 30.1 45.6 43.7 73.8

53.7 45.9 45.5 92.0 59.2 45.5 78.3

75.0 88.9 82.6 97.0 85.9 59.6 84.2

Table 5. SOL Chemistry Exam Pass Rates for Three-Year Group B, Peer Group, and State


a

Sites

2000a

2001

2002

2003

MCL-5 MCL-6 MCL-7 MCL-8 MCL-9 MCL-10 MCL-11 MCL-12 MCL-13 MCL-14 MCL-15 MCL-16 MCL-17 MCL-18 3-Year B MCL Average 3-Year B Peer Average State Average

42.3 46.7 48.7 48.8 48.9 50.0 50.0 56.7 57.7 58.8 60.0 66.6 68.3 86.2 56.4 63.1 63.8

80.0 66.0 25.0 73.1 92.5 100.0 41.2 71.5 79.3 74.5 83.3 72.9 85.6 90.6 74.0 65.9 73.8

45.5  65.4 72.7 87.2  83.3 69.3 88.5 87.3 60.0 89.6 81.7 86.7 76.4 75.4 78.3

46.0  89.4  88.8  100.0 84.2 90.1 94.9 81.0 93.1 92.6 100.0 87.3 81.2 84.2


MCL-18) experienced increases in SOL exam scores over the three-year period. With one exception (MCL-18), none of the three-year group B schools achieved a pass rate of 70% prior to the introduction of ChOP. The averages for three-year group B schools are contrasted to the pass rate percentages of the peer schools and the statewide average in Figure 4. Prior to the introduction of ChOP, the participating schools had a statistically significant lower average pass rate than the peer schools. While both ChOP schools and the peer averages rose over the period, ChOP schools experienced a 30.9% point

Figure 3. Plot of the pass rate percentages for the SOL chemistry exam for the years from 2000 to 2004. The average pass rate percentages of the four schools (either inner city, predominantly minority students, or both) that used ChOP (designated MCL) for a three-year period are shown, as well as averages from four peer schools that did not use ChOP. Also shown are data for the state average pass rate percentages for the SOL chemistry exam. The year 2000 data provide a baseline, as ChOP was not operational then. 1254



Article

Table 6. SOL Chemistry Exam Pass Rates for Two-Year Group and State

a

Figure 4. Plot of the pass rate percentages for the SOL chemistry exam for the years from 2000 to 2004. The average pass rate percentages of the thirteen schools that used ChOP (designated MCL) for a threeyear period are shown, as well as averages from four peer schools that did not use ChOP. Also shown are data for the state average pass rate percentages for the SOL chemistry exam. The year 2000 data provide a baseline, as ChOP was not operational then.

Sites

2001a

2002

2003

MCL-19 MCL-20 MCL-21 MCL-22 MCL-23 MCL-24 MCL-25 2-Year Average State Average

58.5 60.2 63.0 64.7 79.3 90.6 97.2 73.4 73.8

51.3 81.1 65.9 87.5 82.5 86.7 100.0 79.3 78.3

84.6 85.1 81.3 84.0 90.8 100.0 100.0 89.4 84.2

ChOP was not used.

Table 7. SOL Chemistry Exam Pass Rates for One-Year Group and State

gain, as contrasted to an 18.1% point gain of the peer schools. This difference in these four years, between three-year group B and the peer group, is statistically significant. A comparison of these data to the state average shows a similar trend: ChOP schools of three-year group B were 6.7% points below the state average in 2000. By 2003, ChOP schools were 3.1% points above. After one year, the three-year group B average (74.0%) was the same as the state average, while the peer group had a statistically lower average than the state average. Two schools, MCL-6 and MCL-10, were eliminated from the comparison of these data after the first year of use. School MCL-6 had a change of teacher in 2002. School MCL-10 lost access to the MCL because of restrictions from the Virginia Department of Transportation. This school was located on the side of a 5000-foot mountain and was only accessible by a long, winding road. The road had to be closed to traffic for the MCL tractor−trailer to climb the mountain, using both lanes of the road, to the school. Only MCL-5 had a score of less than 70% by 2003. Prior to the program, the SOL exam score for MCL-5 was 42.3%, which was not much lower than the score of 45.5% in the third year of MCL use. Surprisingly, the pass rate increased to 80.0% in the first year of MCL use and then dropped to close to the original rate in the remaining two years for MCL-5. There was no change in teacher, text, or syllabi during this period. However, there was an increase in the number of students enrolled in the course.

a

Sites

2002a

2003

MCL-26 MCL-27 MCL-28 MCL-29 MCL-30 MCL-31 MCL-32 MCL-33 MCL-34 MCL-35 MCL-36 MCL-37 MCL-38 1-Year Average State Average

15.6 74.5 75.0 75.7 82.7 83.5 85.9 86.4 87.6 88.4 90.0 91.1 90.2 79.1 78.3

40.9 96.2 78.7 80.7 81.8 87.2 88.8 89.1 91.0 87.0 90.9 85.5 89.0 83.6 84.2

ChOP was not used.

School MCL-26 is worthy of additional comment because it had a pass rate of 15.6% prior to ChOP, and then 40.9% after one year of service by ChOP. This school is located in an economically depressed, urban area in southern Virginia with a 95% minority population. By way of contrast, MCL-27 also has a high minority population but is in a rural area. The pass rate for MCL-27 increased from the initial 74.5% to 96.2% after involvement with the program. ChemKit Schools

The last set of data is the SOL exam mean percentage for nine schools that only received ChemKits from 2003 to 2005. These schools were outside of the MCL service area because of their distance from the university. The data for these schools are listed in Table 8 and displayed in Figure 5. Scores from peer schools for the two-year period also are presented with these data. Prior to receiving the ChemKits, six of the nine schools did not achieve a 70% pass rate on the SOL exam. The values ranged from a low of 21.9% to a high of 88.9% with an average of 61.8%. Prior to receiving ChemKits, the nine schools were an average of 16.5% below the state average, a statistically significant difference. The peer group had an average that did not statistically differ from the state average in 2002. After one year, the schools using ChemKits posted an average that was statistically the same as the state average. This trend continued in 2004. Prior to the introduction of the ChemKits, the CKO schools performed 11.3% below their peers. After the introduction of

Two-Year Group

Seven schools participated in the program in 2002 and 2003. Their individual scores are shown in Table 6. Four schools in 2001 had pass rates below 70%. After one year in the program, the percentage pass rate improved for five of the seven schools, with only two below 70%. The two-year group pass rate in 2002 was 79.3%, one percentage point higher than the state average. In 2003, all of the seven schools achieved a pass rate that was five percentage points above the state average. One-Year Group

The individual pass rate percentages for the schools that used ChOP for just one year are listed in Table 7; the 13 schools are labeled MCL-26 to MCL-38. With the exception of MCL-26, all achieved a pass rate of 70% before their use of ChOP. The rise in this two-year group’s pass rate paralleled the state average. 1255



Article

Corporate donations did comprise a small part of the program funding. Funding from Volvo of North America was instrumental in acquiring the tractor. Other corporate funding was garnered for salaries and operations, but this level was less than 10% of the operating budget. Direct solicitation of corporate donors was prohibited, per university policy. The MCL was taken out of service in May 2004 because of a lack of funding. The ChemKit program suffered the same fate in 2005. Despite university lobbying efforts, the state legislature struck funding for the programs for FY 2004−2006. The MCL was sold in 2008 to an outreach program operated by the University of Pittsburgh and is now known as the Pitt Mobile Science Lab.26 The equipment of the MCL and ChemKit programs has been integrated into the undergraduate chemistry laboratories at Virginia Tech. Clearly, the operational expense of the mobile laboratory led to the demise of the program. It was widely agreed that the program was effective and innovative. Had the schools in the service area been equipped with minimal lab space, a van delivery program similar to Juniata College3 or Alabama4 would have been the more cost-effective means of bringing chemistry experiments to students. However, many schools in Appalachia did not have such space, and even though the impact of ChOP was positive, it was simply too expensive for these underfunded school districts. In retrospect, the business plan for ChOP should have been developed before the MCL was initiated. While ChOP did receive federal funding through reviewed proposals, from corporate partners in the start up of the program, and from the university toward the outfitting of the mobile lab, the cost of running the program outstripped the available support. Had these business limitations been fully realized by the scientists and educators of ChOP, more effort would have been placed on the development of a business plan (involving the university foundation) at the genesis of the project. It is the recommendation of these authors that a large outreach program must have a business plan before the onset of any outreach work. Given the results of ChOP and other outreach-based projects, it should be no longer necessary to again demonstrate the potential impact of any similar outreach project. The funding makeup must primarily come from state and corporate sources, then federal sources, and last university sources. It is the opinion of the authors that university contributions should comprise approximately 10% of the operating budget. While the physical program is no more, the Web site for the program is still active.6 Teachers continue to access the pages for reference materials, MCL-based experiments, and ChemKit experiments.

Table 8. SOL Chemistry Exam Pass Rates for the ChemKit Only Group and State Sitesa

2002b

2003

2004

CKO-1 CKO-2 and CKO-3c CKO-4 CKO-5 CKO-6 CKO-7 CKO-8 CKO-9 CKO Average CKO Peer Average State Average

21.9 51.1 58.1 59.2 61.3 75.0 78.7 88.9 61.8 73.1 78.3

48.8 59.0 68.1 95.7 87.7 96.4 87.4 91.1 79.2 82.4 84.2

75.5 85.4 85.0 95.8 97.6  84.2 96.5 88.5 83.4 86.3

a

CKO designates schools that had ChemKits only. bChemKits were not in service. cCKO-2 and CKO-3 are different schools but have the same teacher.

Figure 5. Plot of the pass rate percentages for the SOL chemistry exam for the years from 2000 to 2004. The average pass rate percentages of the nine schools that used only ChemKits for a twoyear period are shown, as well as average pass rate percentages from nine peer schools that did not use ChemKits. Also shown are data for the state average pass rate percentages for the SOL chemistry exam. The year 2002 data provide a baseline, as the ChemKit program was not in service then.

ChemKits, the difference in the CKO and the peer group schools from 2002 to 2004 is statistically different. The CKO schools bested the peer schools by five points in the last year of use.

■

SUSTAINABILITY OF THE PROGRAM The MCL was funded by federal, corporate, private, state, and university sources. A tenet of the program was that schools not be charged for any MCL or ChemKit experiments. Given the economic conditions of rural Virginia, no contributions could be expected from teachers or school districts to support the outreach program. The ChOP suffered the fate of many other outreach programs; although successful in its educational mission, it was not successful in garnering state funds to ensure the continuance of the program. The funding from federal sources was essential in establishing the program from 2000 to 2004, but the funding for the operational expenses of ChOP in later years was the responsibility of the state. The Commonwealth of Virginia did not rise to the occasion. While the impact of the program was noted, it was deemed too costly to continue. The yearly cost of ChOP was nearly $332,000 for providing 17,000 experiments to the ChOP schools.

■

SUMMARY In four academic years of operation, ChOP had impacted nearly 9100 high school chemistry students and 44 high school chemistry teachers from 38 schools. Although students from the traditionally underserved areas of Virginia scored far below the state average on the SOL chemistry exam before the inception of this program in 2000, students from these schools demonstrated knowledge gains on the test in the chemical sciences after they used the MCL and ChemKits. After using the program, their SOL exam scores were similar to the schools from more affluent areas of the state. 1256



Article

(2) Mitchell, M.; Hermans, C.; Shubert, D. ChemKits: A TeacherTraining and Instrument-Sharing Project. J. Chem. Educ. 1999, 76, 1409−1411. (3) Mitchell, D. J. Juniata College’s Science Outreach Program. Council on Undergraduate Research Quarterly 1998, 19 (1), 15−20. (4) Alabama Science in Motion. https://fp.auburn.edu/asim/ (accessed Jun 2012). (5) Fralin Biotech Outreach Program at Virginia Tech. http://www. fralin.vt.edu/content/outreach (accessed Jun 2012). (6) Chemistry Outreach Program. http://www.chem.vt.edu/mcl (accessed Jun 2012). (7) Sarquis, A. Recommendations for Offering Successful Professional Development Programs for Teachers. J. Chem. Educ. 2001, 78, 820−823. (8) Khourey-Bowers, C.; Simonis, D. Longitudinal Study of Middle Grades Chemistry Professional Development: Enhancement of Personal Science Teaching Self-Efficacy and Outcome Expectancy. J. Sci. Teach. Educ. 2004, 15, 175−195. (9) Burns, T. Maximizing the Workshop Experience: An Example from the PRA Rural Initiatives Program. Phys. Teach. 2003, 41, 500− 501. (10) Vandergrift, V.; Crafton, A. The Influence of Two Recent NSF Summer Workshops on High School Chemistry and Physical Science Teachers. J. Chem. Educ. 1990, 67, 1047−1051. (11) Sanders, W.; Rivers, J. Cumulative and Residual Effects of Teachers on Future Students Academic Achievement. 1996, University of Tennessee Value-Added Research and Assessment Center, State Department of Education. http://www.cgp.upenn.edu/pdf/Sanders_ Rivers-TVASS_teacher%20effects.pdf (accessed Jun 2012). (12) Watson, G. Technology Professional Development: Long-Term Effects on Teacher Self-Efficacy. J.Technol. Teach. Educ. 2006, 14, 151− 165. (13) Study on the Effectiveness of the Virginia Standards of Learning (SOL) Reforms; StandardWorks, Inc.: Washington, DC, 2003. (14) Dietz, S.; Jennings, J.; Rentner, D. 2010 State High School Tests: Exit Exams and Other Assessments; Center on Education Policy: Washington, DC, 2010. (15) Cavallo, A.; Ferreira, M.; Roberts, S. Increasing Student Access to Qualified Science and Mathematics Teachers through an Urban School-University Partnership. Sch. Sci. Math. 2005, 105, 363− 372. (16) Huang, S.; Yun, Y.; Haycock, K. Interpret with Caution: The First State Title II Reports on the Qualify of Teacher Preparation; The Education Trust: Washington, DC, 2002. http://www.edtrust.org/ sites/edtrust.org/files/publications/files/titleII.pdf (accessed Jun 2012). (17) Jerald, C. All Talk, No Action: Putting an End to Out-of-Field Teaching; The Education Trust: Washington, DC, 2002. http://www. edtrust.org/sites/edtrust.org/files/publications/files/AllTalk.pdf (accessed Jun 2012). (18) Ingersoll, D. The Realities of Out-of-Field Teaching. Educ. Leadership 2001, 58, 42−45. (19) Peske, H.; Kaycock K. Teaching Inequality: How Poor and Minority Students Are Shortchanged on Teacher Quality; The Education Trust: Washington, DC, 2006. http://www.edtrust.org/sites/edtrust. org/files/publications/files/TQReportJune2006.pdf (accessed Jun 2012). (20) Khourey-Bowers, C.; Simonis, D. Longitudinal Study of Middle Grades Chemistry Professional Development: Enhancement of Personal Science Teaching Self-Efficacy and Outcome Expectancy. J. Sci. Teach. Educ. 2004, 15, 175−195. (21) Angelo, T. A Teacher’s Dozen”: Fourteen General, ResearchBased Principles for Improving Higher Learning in our Classrooms. AAHE Bull. 1993, 3, 3−13. (22) Ingersoll, R. The Problem of Under-Qualified Teachers: A Sociological Perspective. Sociol. Educ. 2005, 78, 175−178. (23) Carpenter, S.; Hizer, T. A Chemistry Workshop for Secondary School Science Teachers. J. Chem. Educ. 1999, 76, 387−389.

The methodology for the evaluation precludes stating conclusively why the SOL exam scores improved. Nonetheless, we argue that ChOP played an important role in the increase in student performance. One of the most consistent findings in the research on student learning is that active learning is better than passive learning.21 Obviously, students cannot have active, hands-on laboratory experiences if their schools have no chemistry labs. Without the infrastructure for appropriate pedagogy, it is not surprising that students from the schools without even minimal resources do not score well on achievement tests. ChOP provided laboratory space, equipment, and supplies so that the students could reap the well-known benefits of hands-on learning. It is argued here that the combination of the different components of ChOP directly or indirectly led to more effective chemistry education and gave disadvantaged students a chance to excel. In spite of the fact that ChOP is no longer in operation, it is hoped that the benefits of ChOP will continue through the teachers who participated and by other using the wealth of information available on the ChOP Web site.6

■

ASSOCIATED CONTENT

* Supporting Information S

Program details; description of the MCL, with schematics; map of service area; data analyses for Tables 3−8. This material is available via the Internet at http://pubs.acs.org.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS Many individuals and agencies must be acknowledged when discussing the success of this outreach program. The following individuals are acknowledged for their contributions. From Virginia Tech: Patricia Amateis and Larry Taylor. From regional high schools: Julie Grady, Suzanne Dana, and Mary Lou Hearn. The faculty and staff of the Juniata College project and the Alabama project are gratefully acknowledged for their support sharing their insight and experiments. Agencies that supported this work are: the National Science Foundation (NSF-CHE-0079119 and NSF-CHE-011150), the Dwight D. Eisenhower Professional Development Foundation (2000, 2001, and 2002), and the Camille and Henry Dreyfus Foundation (2000 and 2002). Corporations include: E. I. DuPont de Nemours Company (2001, 2002, 2003), Volvo Trucks North America (2000) for the gift of the Volvo tractor, Featherlite Company (2000) for assistance and support in the fabrication of the transporter. Other support includes the Division of Outreach and International Affairs of Virginia Tech, the College of Science of Virginia Tech, and the Edward Via School of Osteopathic Medicine.

■

REFERENCES

(1) Science Standards of Learning for Virginia Public Schools January 2010: Chemistry. http://www.doe.virginia.gov/testing/sol/ standards_docs/science/2010/courses/stds_chemistry.pdf (accessed Jun 2012). 1257



Article

(24) U.S. Department of Education, Elementary and Secondary Education Act. http://www.ed.gov/esea (accessed Jun 2012). (25) Personal communication, Nelson Colbert, Superintendent of Richmond City Schools, 2002. (26) Pitt Mobile Science Lab. http://www.mobilelab.pitt.edu/ (accessed Jun 2012).

■

NOTE ADDED AFTER ISSUE PUBLICATION Two authors were left off of the author list in the version published on July 19, 2012. The revised version was published to the Web on December 7, 2012.

1258


Chemistry Outreach Project to High Schools Using a Mobile Chemistry

Recommend Documents