Applying the Next Generation Science Standards to Current Chemistry

Article Cite This: J. Chem. Educ. 2019, 96, 1308−1317

pubs.acs.org/jchemeduc

Applying the Next Generation Science Standards to Current Chemistry Classrooms: How Lessons Measure Up and How to Respond Natalia M. Kellamis and Ellen J. Yezierski* Department of Chemistry & Biochemistry, Miami University, Oxford, Ohio 45056, United States

Downloaded via IDAHO STATE UNIV on July 17, 2019 at 16:29:22 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.

S Supporting Information *

ABSTRACT: With the introduction of the Next Generation Science Standards (NGSS), curriculum and professional development have had to change rapidly to fit the new standards. To aid with those changes, the EQuIP rubric was released as a guide for NGSS alignment. This study aims to evaluate lesson plans developed through the Target Inquiry project to determine if the lessons developed under the National Science Education Standards are aligned with the new standards. The utility of the EQuIP rubric was also evaluated. Target Inquiry at Miami University (TIMU) teachers authored 19 lessons, which were evaluated using version 2 of the EQuIP rubric, which was modified to better fit the aims of the study. Each lesson evaluation included a numerical score in each of the EQuIP lesson quality criteria and a list of the grade band for each occurrence of an NGSS dimension (crosscutting concepts, disciplinary core ideas, and science and engineering practices). Overall, the lessons scored well on criteria concerning use of scientifically accurate information and direct evidence of student learning. The lessons did not score well on criteria that related to specific characteristics of the lesson plan format, which was unsurprising because the lessons were not written with the criteria of EQuIP in mind. It was also discovered that the lessons contained many occurrences of the dimensions that were below the 9−12 grade level. This could indicate that the level to which the NGSS expects teachers to be teaching is higher than what is currently ongoing in secondary classrooms. This study also has implications for NGSS implementation stemming from the intensive study required of the researchers to detect which specific NGSS standards were being applied in lessons with integrity. Better rubrics and a large amount of preparation is likely needed for teachers to learn to apply NGSS with fidelity. KEYWORDS: High School/Introductory Chemistry, Chemical Education Research, Curriculum, Inquiry-Based/Discovery Learning, Professional Development, Standards National/State, Student-Centered Learning FEATURE: Chemical Education Research

■

INTRODUCTION Since 1996 and until quite recently, the National Science Education Standards (NSES)1 guided the development of curriculum, instruction, and assessments in K−12 science. The chemistry education community responded with a useful guide,2 while the chemistry education research community responded with numerous studies addressing curriculum, instruction, and assessments. Because teachers are at the center of the reform process, studies in chemistry also examined teacher professional development and teacher change. One well studied initiative was that of Target Inquiry (TI),3−9 an inquiry-based, long-term, rigorous professional development program for secondary teachers launched first in chemistry and recently expanded to other science disciplines. TI lasts for 2.5 years and includes a research experience for teachers, materials adaptation, and a teacher-led evaluation of the new instructional materials through action research. Since the inception of TI in 2005 and the first teacher cohort completing the program in 2008, over 69 laboratory and classroom activities have been developed as part © 2019 American Chemical Society and Division of Chemical Education, Inc.

of the materials adaptation component of the TI program, and the materials have been published on the TI project websites. A sample of the activities have been published in peer-reviewed journals.10−19 There have been over 4300 teachers around the world who have downloaded TI activities. TI integrated teacher learning, materials development, and research on teacher change with a significant dissemination model; however, the foundation of the program and its products was the NSES, which has now been replaced by the Next Generation Science Standards (NGSS).20 The Next Generation Science Standards20 were released in 2013 and were based upon the 2011 release of A Framework for K−12 Science Education: Practices, Crosscutting Concepts, and Core Ideas.21 As in the time immediately following the publication of the NSES, the chemistry education community Received: October 12, 2018 Revised: June 2, 2019 Published: June 24, 2019 1308

DOI: 10.1021/acs.jchemed.8b00840 J. Chem. Educ. 2019, 96, 1308−1317

Journal of Chemical Education

Article

has responded to the NGSS. In particular, Cooper22 and Talanquer and Sevian23 have examined the chemistry content and science practices in the NGSS, noting similarities and differences with existing curricula (topics and methods). Resources beyond the Framework21 and NGSS20 have been developed by Achieve to support the development of curriculum, instruction, and assessments aligned with NGSS; however, they are emerging slowly. We would expect significant commonalities between the NGSS and NSES because both are based on fundamental research on how students learn science. However, it is important to understand the NSES−NGSS alignments on a number of practical fronts, including the extent to which chemistry teachers who developed expertise framed by NSES are equipped to employ the NGSS and what models of professional development are appropriate to support teachers in transitioning methods and materials from NSES to NGSS. Our aim is not to propose an entire research agenda but rather to situate the current study and findings in an emerging area of research that is critical for secondary chemistry education in the time of NGSS. With a focus on materials, we sought to understand how the TI chemistry materials recently developed by a TI cohort of chemistry teachers at Miami University framed around NSES aligned with NGSS. Carrying out this work required a rubric to frame the evaluation. Tools are emerging in tertiary science education, particularly the Three-Dimensional Learning Assessment Protocol (3-D LAP) designed by Laverly, Underwood, Posey, Carmel, Caballero, and Cooper.24 The 3-D LAP characterizes and supports the development of assessment tasks across multiple science disciplines with respect to their alignment to NGSS dimensions. However, the 3-D LAP was not suitable for our study because the TI chemistry materials are coherent lesson plans including objectives, student activities, and assessments. Fortunately, an NGSS-focused resource aimed at the evaluation of curricula was available through Achieve. The resource named Educators Evaluating the Quality of Instructional Products (EQuIP, version 225) provided the basis for the lesson analysis. We set out to understand the utility of EQuIP for assisting with this important transition to chemistry classrooms built upon NGSS ideas. When necessary, we modified the rubric to meet the aim of the study: to draw conclusions about a set of TI (NSES-focused) chemistry lessons. With these purposes in mind, we evaluated the lessons from Target Inquiry at Miami University (TIMU) with respect to their alignment with NGSS guided by the following research questions: 1. Which criteria on the EQuIP rubric are strongly aligned with lessons designed using the NSES framework? 2. Which crosscutting concepts (CCCs), disciplinary core ideas (DCIs), and science and engineering practices (SEPs) are most prevalent in the chemistry lessons developed from the NSES framework? 3. What is the NGSS grade band for lessons based on current high school chemistry lessons framed around NSES?

■

meant to be used collaboratively by a group of publishers and curriculum developers, who fill out the form with evidence for each criterion and suggestions on how to improve the lesson. The rubric is formatted with three categories: (1) Alignment to the NGSS, (2) Instructional Supports, and (3) Monitoring Student Progress. Because this study aimed to characterize the lessons, rather than to provide developmental feedback, the rubric had to be modified. We focused on description rather than evaluation for the application of Category 1 of EQuIP (NGSS Alignment). In our early aims to make Category 1 evaluative, we found it too difficult to generate scoring criteria for each of the dimensions that would be consistent for each lesson given the holistic nature of the first category of the rubric. Such scoring would have required criteria that would simultaneously identify the dimension, its grade band, if the dimension was explicit in the lesson, and its integration with every other dimension appearing in the lesson. Instead, to capture the essential elements of Category 1, the practices, core ideas, and crosscutting concepts from each lesson were identified in the lesson along with the grade band at which they fell as determined by the grade level progressions in NGSS Appendices E−G.27 The interconnectedness of the dimensions was largely ignored. Other NGSS-related work, such as that with the 3-D LAP,24 has yet to qualify how interconnectedness among the three dimensions is evaluated. There are sections of the EQuIP rubric following each category labeled “A unit or longer lesson.” These criteria were discarded, as the lesson set under study contained mostly shorter, 1−2 day lessons that would not meet the definition of a longer lesson or unit. For Categories 2 (Instructional Supports) and 3 (Monitoring Student Progress), we took a more evaluative rather than descriptive approach. A numerical, ordinal scale from 0 to 3 was added to each of the criteria on the rubric for Categories 2 and 3. Interestingly, the addition of a scale was a modification that Achieve noted as forthcoming in later revisions of the rubric.28 The scale we used indicated the level of quality of the criterion (0 = absent, 1 = low, 2 = medium, 3 = high). Early work used a binary scale (present or absent); however, we were able to make it slightly more precise and still yield a high level of agreement between two raters. Applying a scale to each item to create the modified rubric enabled the research team to evaluate the lessons according to the criteria in the rubric and report the evaluation data in a condensed form. Once the scales were inserted, the issues with the multipart nature of some of the rubric criteria as well as some redundancies in the criteria became apparent. Splitting criteria and carefully defining terms improved inter-rater agreement. Examples of criteria that contained multiple elements and hence needed separation are found in Category 2 (Instructional Supports). Box 1 shows Category 2 (Instructional Supports) Criteria C and D. As shown in Box 1, both of these criteria (2C and 2D) present multiple, unique ideas that can and should be evaluated separately. For example, a lesson could present information that was scientifically accurate but not grade appropriate. The converse phenomenon was not observed but could easily occur if a common misconception was presented as fact. Because of the need to make these distinctions, the criteria were expanded as shown in Box 2. We refer to our modified rubric as QA-EQuIP (QA for quantitative analysis); it can be found in the Supporting Information.

METHODS

Rubric: Modifying the Educators Evaluating the Quality of Instructional Products (EQuIP) Rubric

The study began with EQuIP version 2.25 EQuIP was developed by Achieve, the publisher of the Next Generation Science Standards, to provide publishers and curriculum developers with a feedback guide to determine alignment with NGSS.26 EQuIP is 1309



Article

Lesson Selection

Box 1. Criteria C and D from the EQuIP Rubric Section Titled, “Instructional Supports”, Showing How Single Criteria Actually Attend to Multiple Features of a Lesson

To develop the scoring criteria for each piece of the rubric, three lessons were chosen from the 19 total to be scored separately by two investigators. These lessons were chosen for their variability in three areas: extent of student-designed procedures; concept development versus concept verification; and the types of phenomena, such as macroscopic representations, simulations, or symbolic models. After each lesson was independently scored by each author, the scores were compared and any disagreements were discussed and resolved by the author team. In cases where EQuIP itself was not specific or clear enough to yield agreeable scores, the NGSS20 and the Framework21 were used to clarify the language in the rubric. After the first three lessons were scored, three lessons from the same teacher authors were chosen and scored by one investigator. By choosing the same teacher authors, we aimed to control for writing style differences that might obscure differences among the rubric components. The first author scored the balance of the lessons in a random order.

2C: Uses scientifically accurate and grade-appropriate scientific information, phenomena, and representations to support students’ three-dimensional learning. 2D: Provides opportunities for students to express, clarify, justify, interpret, and represent their ideas and respond to peer and teacher feedback orally and/or in written form as appropriate to support students’ three-dimensional learning. Box 2. Criteria C and D from QA-EQuIP Showing the Expansion of the Original Criteria 2Ci: Uses scientifically accurate information, phenomena, and representations to support students’ three-dimensional learning (3 = no content errors). 2Cii: Uses grade-appropriate information, phenomena, and representations to support students’ three-dimensional learning. 2Di: Provides opportunities for students to express, clarify, justify, interpret, and represent their ideas orally and/or in written form as appropriate to support students’ threedimensional learning (Bloom’s Taxonomy). 2Dii: Provides opportunities for students to respond to peer feedback orally and/or in written form as appropriate to support students’ three-dimensional learning (1 = groups). 2Diii: Provides opportunities for students to respond to teacher feedback orally and/or in written form as appropriate to support students’ three-dimensional learning.

Scoring

While scoring the lessons with QA-EQuIP, an evidence and reasoning memo29 was written for each lesson. Memos contained lists of all criteria on the rubric as well as each occurrence of the three dimensions with their grade bands. The reasoning supporting each score was elaborated on to help researchers document rationales for particular scores. This made comparisons among scores easier, especially when such comparisons occurred well after the original scoring was done.

■

RESULTS AND DISCUSSION The results draw from the evaluation and modification of EQuIP as well as the analysis of the 19 chemistry lessons using QAEQuIP and descriptions of the standards in NGSS Appendices E−G.27 The results are organized in response to each of the research questions that guided this study.

Criterion C was expanded into two parts, one for scientific accuracy and one for grade appropriateness. Criterion D was expanded into three parts to separate the feedback components from the representation of student ideas. Increasing the total number of items by splitting these two items into multiple parts does not impact the use of the rubric because the total score of each lesson is not used to draw conclusions about the lessons. Instead, we attended to the individual QA-EQuIP criteria scores across the lesson set. In other words, our conclusions were to be derived not from the total score but rather from the highest and lowest scoring criteria in QA-EQuIP. A few other modifications to EQuIP were useful to the research team. First we bolded key terms for emphasis to focus researchers on the key points within particular criteria. Second, Criterion A from Category 2 (Box 3) was found to be redundant when examining Category 1. Therefore, this specific criterion was omitted from the modified rubric.

Research Question 1

Research Question 1 was “Which criteria on QA-EQuIP are strongly aligned with lessons designed using the NSES framework?” Table 1 shows the distributions of lesson scores for each criterion in Categories 2 and 3 sorted from the highest scoring criterion to the lowest for the lesson set. Scores on each criterion for each individual lesson in the set can be found in the Supporting Information. As shown in Table 1, all of the lessons scored high (mostly 3 and some 2) on the criteria concerning use of “scientifically accurate information” and “direct, observable evidence” of the dimensions. Because the lessons underwent a peer and faculty review as part of the development process in TIMU, it was not surprising that they were scientifically accurate. The lessons as a whole scored low on the criteria concerning use of “context” “grade appropriate information,” “teacher feedback,” “extra support for struggling students,” “extension activities for higher achieving students,” and “formative assessments.” The lack of extension and support activities is due to the intended broad dissemination of the TIMU lesson plans. As these lessons were to be distributed internationally on the Internet, the lessons did not contain differentiation for student exceptionalities; teachers using the lessons must tailor them to their individual students. The six highest scoring lessons by total score were also examined. The total score was only used to sort the lessons for this analysis. As stated previously, it is much more meaningful to

Box 3. Category 2 Criterion A Subcriterion iii from the EQuIP Section Titled, “ Instructional Supports”, Showing the Redundancy of Various Rubric Criteria 2Aiii: Engages students in multiple practices that work together with disciplinary core ideas and crosscutting concepts to support students in making sense of phenomena and/or designing solutions to problems. 1310



Article

Table 1. Distribution of Lesson Scores (0−3) in Categories 2 and 3 of QA-EQuIP Number of Lessonsa QA-EQuIP Criterion

0

1

2

3

2Ci. Uses scientifically accurate information, phenomena, and representations to support students’ three-dimensional learning. 3A. Elicits direct, observable evidence of three-dimensional learning by students using practices with core idea and crosscutting concepts to make sense of phenomena and/or to design solutions. 2Aii. Provides students with relevant phenomena (either firsthand experiences or through representations) to make sense of and/or provides relevant problems to solve. 2Aiv. Provides opportunities for students to connect their explanation of a phenomenon and/or their design solution to their own experience. 3C. Includes aligned rubrics and scoring guidelines that provide guidance for interpreting student performance along the three-dimensions to support teachers in (a) planning instruction and (b) providing ongoing instruction to students. 2B. Develops deeper understanding of the practices, disciplinary core ideas, and crosscutting concepts by identifying and building on students’ prior knowledge. 2Dii. Provides opportunities for students to respond to peer feedback orally and/or in written form as appropriate to support students’ threedimensional learning. 2Di. Provides opportunities for students to express, clarify, justify, interpret, and represent their ideas orally and/or in written form as appropriate to support students’ three-dimensional learning. 2Cii. Uses grade-appropriate scientific information, phenomena, and representations to support students’ three-dimensional learning. 3D. Assessing student proficiency using methods, vocabulary, representations, and examples that are accessible and unbiased for all students. 2Ai. The context, including phenomena, questions, or problems, motivates students to engage in three-dimensional learning. 2Diii. Provides opportunities for students to respond to teacher feedback orally and/or in written form as appropriate to support students’ three-dimensional learning. 3B. Formative assessments of three-dimensional learning are embedded throughout the instruction. 2Eiv. Provides extensions for students with high interest or who have already met the performance expectations to develop deeper understanding of the practices, disciplinary core ideas, and crosscutting concepts. 2Eiii. Provides suggested extra support (e.g., phenomena, representations, tasks) for students who are struggling to meet the performance expectations.

0 0

0 0

0 7

19 12

0

2

4

13

0 0

2 4

9 8

8 7

3

4

5

7

1

7

6

5

0

9

8

2

4 3 6 5

6 8 5 6

6 5 3 4

3 3 5 4

5 6

6 5

4 5

4 3

15

3

1

0

a

N = 19.

Figure 1. Occurrences of each crosscutting concept20 in the lesson set.

size comparison of a few relevant objects from their everyday lives that then feed into a larger comparison as prior knowledge. This larger scale is created physically using paper, allowing the direct observation of student knowledge. At the very beginning and end of the activity, students are asked to calculate the number of atoms in aluminum foil using their prior knowledge and their experience in the lesson. The teacher guide in this lesson is specific in its answers as well as in its prompts for teacher feedback to students. All of these qualities led to a high total score on the rubric.

examine the score distribution across the lesson set rather than by total lesson scores because we were interested in the particular lesson features in the set that met the QA-EQuIP criteria. In addition to the high scoring features of all of the lessons, the six highest scoring lessons in the set were distinguished by four QA-EQuIP criteria: “relevant phenomena,” “connections to student experience,” “identify and build on prior student knowledge,” and “includes aligned rubrics and scoring guidelines.” Itsy Bitsy Atoms was one of the lessons that scored high on all of these criteria. In the lesson, students are asked to place objects on a size scale to help them visualize the size of an atom. Some of those objects include a tree, aluminum foil, a leaf, and the Earth. Students asked to complete a smaller 1311



Article

Figure 2. Occurrences of each disciplinary core idea20 in the lesson set.

Figure 3. Occurrences of each science and engineering practice20 in the lesson set.

crosscutting concepts in the lessons were “Systems and System Models,” “Energy and Matter,” “Patterns.” and “Scale, Proportion, and Quantity.” The concept “Systems and System Models” was present in the lessons to a great degree because the focuses of the lessons were on using multiple representations of the processes being studied, with each representation applied for its predictive or explanatory power and thus considered to be a model here. The crosscutting concept “Cause and Effect” was the least represented because the focus of the CCC is quite narrow and solely on causal relationships with empirical evidence backing up claims. There was about one disciplinary core idea per lesson. As shown in Figure 2, the most frequently occurring DCIs were

Research Question 2

Research Question 2 was “Which CCCs, DCIs, and SEPs are most prevalent in the chemistry lessons developed from the NSES framework?” The following section describes the results of tabulating the number of occurrences of each concept, core idea, or practice within each of the three dimensions in the lesson set. As stated in the Methods section, the interconnectedness of the dimensions was largely ignored despite the language of Category 1 of the rubric stating, “The three dimensions work together to support students to make sense of phenomena and/ or to design solutions to problems.” There were approximately two crosscutting concepts per lesson. As shown in Figure 1, the most frequently occurring 1312



Article

Figure 4. Occurrences of each NGSS dimension split by grade band.

“Chemical Reactions,” “Structure and Properties of Matter,” and “Conservation of Energy and Energy Transfer.” It should be noted that one of these commonly occurring DCIs is from “PS3: Energy,” as not all chemistry content is in the “PS1: Matter and Its Interactions” section of the NGSS. Talanquer and Sevian23 pointed out missing chemistry topic areas when they examined PS1 by itself, without including PS3, the CCCs, or the SEPs, as was done in Cooper et al.22 Like Cooper et al.,22 we employed a more holistic view of the NGSS when comparing the standards to the TIMU lesson set. None of the lessons in the lesson set touched upon ideas in “PS2: Motion and Stability” or “PS4: Waves and Their Applications in Technologies for Information Transfer,” as those ideas were tied much more closely to physics curricula. In addition, none of the TIMU lessons contained DCIs from Life Science or Earth Science. Only one lesson in the set focused on “Nuclear Processes.” This reflects a larger trend in chemistry classrooms where nuclear chemistry is either present to a minimal degree or not addressed at all. There is little overlap between chemical reactions and bonding (the vast majority of the chemistry curriculum) and nuclear reactions. Outside of the equation balancing mechanism that is used across multiple chemistry concepts, there is very little in common between nuclear decay and chemical reactions. Although Atwood and Sheline30 make a case for more nuclear chemistry in the curriculum, it turns out not to be emphasized in the NSES or NGSS. There were about two science and engineering practices (SEP) per lesson. As shown in Figure 3, the most frequently occurring practices were “Constructing Explanations,” “Analyzing and Interpreting Data,” “Developing and Using Models,” and “Using Mathematics and Computational Thinking.” Practice 1, “Asking Questions and Defining Problems,” did not occur at all. At face value, this seems unexpected because the lessons were designed using the NSES, which was focused heavily upon inquiry. The action of asking questions does not mean that a lesson contains this particular SEP. At the 9−12 grade band, Appendix F of the NGSS stipulates that students should be “generating high quality, testable questions about phenomena they experience for the purpose of hypothesis creation.”27 Because the TIMU lessons were created to employ guided inquiry, students do not generate these types of questions because they are not creating hypotheses to test. This SEP reflects a broader phenomenon in the NGSS, where the dimensions are much more narrow and specific than their titles suggest. For example, students doing calculations and sharing

their answers with the class does not indicate the students were using the two practices of “Using Mathematics and Computational Thinking” and “Obtaining, Evaluating, and Communicating Information.”27 Research Question 3

Research Question 3 was “What is the NGSS grade band for lessons based on current high school chemistry concepts framed around NSES?” When categorizing the NGSS dimensions in the lessons, it quickly became clear that the occurrences of the dimensions were not all at the 9−12 grade band as would be expected. The results in Table 1 show that only 3 out of the 19 lessons scored 3/3 for grade level appropriateness. Figure 4 summarizes the frequency with which each of the three dimensions occurred in the lesson set as well as the grade band at which they were addressed. Any occurrences below the 6−8 grade band were not counted, because they were considered too far below grade level for a secondary chemistry course. As can be seen in Figure 4, the SEP and CCC occurrences are almost evenly split between the 6−8 and 9−12 grade bands, and DCIs are split evenly. This is notable, because all of the dimensions should occur entirely in the 9−12 grade band in high school lessons written by in-service chemistry teachers. A more fine-grained examination of the particular CCCs, DCIs, and SEPs provides some explanation of how and why lessons are at or below the 9−12 grade band. The following examples provide evidence to support the grade band categorizations of CCCs shown in Figure 4. The evidence that follows is in the form of quotations taken directly from memos written during the evaluation of the lessons following the application of the modified EQuIP rubric. Crosscutting Concept 4, “Systems and System Models,” particularly the treatment of model limitations, is a good example to show how the appendices were used to distinguish between the grade bands. In the 6−8 grade band, students are asked to demonstrate understanding “that models are limited in that they only represent certain aspects of the system under study” (Appendix G).27 The lesson More is Less fits into this grade band, as “Students are using the models to represent the atom. They are also having a discussion of the limitations of the model, but not about its precision as a model” (emphasis added, from a memo from More is Less analysis). The 9−12 grade band takes the same idea one step further, asking students to “use models and simulations to predict the behavior of a system and recognize 1313



Article

Figure 5. Occurrences of each crosscutting concept20 split by grade band.

Figure 6. Occurrences of each disciplinary core idea20 split by grade band.

A closer examination of each crosscutting concept by grade band within the lesson set reveals more detail about how such concepts are represented in the TIMU chemistry lessons (Figure 5). “Systems and System Models” and “Energy and Matter” were the most frequently occurring CCCs in the lesson set at the 9− 12 grade level. “Systems and System Models” and “Scale, Proportion, and Quantity” were the most frequently occurring CCCs at the 6−8 grade band. Similarly to the CCCs, visualizing the individual DCIs by grade band illuminates the areas in which the lessons are most and least aligned with the NGSS core ideas. Figure 6, Figure 4. Two of the core ideas only occurred at the 9−12 grade band: “Nuclear Processes” and “Conservation of Energy and Energy

that these predictions have limited precision and reliability due to the assumptions and approximations inherent in the models.”27 The lesson Gas Properties and Collisions fits into this grade band as it “[d]iscusses the inherent limitations of the ideal gas law (mathematical model) when comparing it to the real world. [The lesson] uses the model of the ideal gas law to simulate interactions between the variables” (emphasis added, from a memo from Gas Properties and Collisions analysis). The distinguishing feature between Gas Properties and Collisions and More is Less is the treatment of the limitations of the models, as More is Less only points out that there are limitations, whereas Gas Properties and Collisions points out how the limitations of the ideal gas law change the precision of the model in the real world. 1314



Article

Figure 7. Occurrences of each science and engineering practice20 split by grade band.

criteria such as “teacher feedback” and “formative assessments.” Although strong in a few lessons, “context” and “grade appropriate information” were not prominently high scoring criteria in the TIMU lesson set.

Transfer.” It should be noted, however, that the core idea PS1.C, “Nuclear Processes,” is only addressed at the 9−12 grade band according to the grade progression in Appendix E.27 Finally, the SEPs by grade band, as shown in Figure 4, show that more of the SEPs addressed in the lessons are in the 6−8 grade band rather than in the 9−12 grade band. More specifically, as shown in Figure 7, the prevalence of “Constructing Explanations and Designing Solutions” and “Analyzing and Interpreting Data” is not surprising given the inquiry-centered pedagogy at the heart of the NSES. Three of the practices only occurred in one of the two grade bands, “Planning and Carrying Out Investigations,” “Engaging in Argument from Evidence,” and “Obtaining, Evaluating, and Communicating Information,” and two of these three SEPs occurred at the 6−12 band. The prevalence of “Developing and Using Models” is likely because the large number of articles read and cited by the TIMU teacher authors referencing the particulate nature of matter and Johnstone’s levels of chemistry.31

■

NGSS Dimensions in TIMU Lessons

The crosscutting concepts (about two per lesson) and disciplinary core ideas (about one per lesson) were strongly associated with chemistry content. The predominant CCCs were “Systems and System Models,” “Energy and Matter,” “Patterns,” and “Scale, Proportion, and Quantity,” and the DCIs were “Chemical Reactions,” “Structure and Properties of Matter,” and “Conservation of Energy and Energy Transfer.” The science and engineering practices (about two per lesson) were associated with a guided inquiry pedagogy and predominantly were “Constructing Explanations,” “Analyzing and Interpreting Data,” “Developing and Using Models,” and “Using Mathematics and Computational Thinking.” It was not surprising that particular SEPs related to inquiry and modeling were common because the lessons were written with the NSES (framed around inquiry) in mind, as well as a conscious focus on levels of representation. However, there were several underrepresented SEPs in the lesson set that reflect differences between NSES and NGSS emphases, such as “Argumentation.” Overall, there was a reasonably wide and chemistry-focused distribution across the CCCs and DCIs in the lesson set; however, missing SEPs highlight that the lesson set does not attend to the variety of types of procedures NGSS calls for students to engage in during science courses.

CONCLUSIONS

TIMU Lessons’ Performance on QA-EQuIP

Because only a few EQuIP criteria strongly aligned with the TIMU lessons designed using the NSES framework, we conclude that key lesson features were missing from the set in light of the EQuIP rubric. The focus on inquiry in the TIMU lessons and peer and faculty review yielded a couple of strong alignments with the QA-EQuIP criteria in the lessons (“scientifically accurate information” and “direct, observable evidence” of the dimensions). The top lessons in the set scored high in “relevant phenomena,” “connections to student experience,” “identify and build on prior student knowledge,” and “includes aligned rubrics and scoring guidelines.” However, more current reform emphasis on differentiation seems to point to missing features in the TIMU lessons, namely, “extra support for struggling students” and “extension activities for higher achieving students.” Assessment is more explicitly emphasized in NGSS than in NSES, as shown by lower scores for the lessons on

Grade-Level Alignment between TIMU Lessons and NGSS

Characterizing the lesson set by grade level of the three dimensions revealed a large gap between the current level of what is being taught in chemistry (TIMU lessons framed around the NSES) as compared with what the NGSS asserts as grade 9− 12 expectations. In fact, two of the three dimensions were found to be below grade level in over half of their occurrences. Only DCIs were aligned with the 9−12 grade band for most of the lessons. If the goal for the activities students do is to be at grade level for secondary chemistry, the findings from this study point 1315



Article

lesson plans. One concern with the old and new EQuIP rubrics and even QA-EQuIP is that they depend on explicit details about every teacher and student action before, during, and after the lesson. This points to a broader issue concerning teacher action in the classroom versus what they actually deem necessary to write down. Achieve has made it clear that the EQuIP rubric is to be used on professionally developed lessons, but teachers frequently create their own lessons. These lessons tend to lack documentation, as the developing teacher may not perceive a need to explicitly write every detail of a lesson they are going to implement themselves. The TIMU lessons were created with other teachers in mind. As such, the lessons are far more detailed than typical teacher-developed lesson plans. However, the TIMU lessons still were not detailed enough to provide all of the information needed to apply EQuIP. As such, PD focused on detailed documentation of lesson plans is a precursor to the application of any NGSS-centered rubric, even emerging tools that employ classroom video of lesson implementation.32 The lesson distribution over the middle and high school grade bands brings an important but unexpected consideration to the application of the findings to NGSS implementation. Can alignment to NGSS in K−12 happen concurrently at all grade levels? Because the lessons studied herein represent the level of current high school chemistry, the grade band results suggest that NGSS and current instruction are not aligned. Although the study only examined high school chemistry, the results have implications for science curriculum alignment in other secondary subjects. The dramatic differences between the current 9−12 chemistry approaches and the requirements of three-dimensional learning in chemistry have important K−8 implications requiring science leaders to work more closely between grade bands. Students and teachers will be facing gaps between existing practices and standards that will grow as students progress through K−12. The NGSS, setting a new standard for all science classrooms, is more rigorous than any before, meaning that students in high school now, without 10 years of an NGSS science education, would be behind students that have developed an understanding of the practices, concepts, and core ideas throughout their schooling.

to a larger issue in schools related to the preparedness of secondary schools to adopt NGSS. The gap between the standards and current school curricula cannot be closed unless students enter secondary chemistry having already met the grades 6−8 expectations. The results of the grade band analysis have important implications for the need for systemic NGSS implementation before and during grades 9−12.

■

IMPLICATIONS Even after modification to QA-EQuIP, the rubric remains difficult to use. Categories 2, “Instructional Supports” and 3, “Monitoring Student Progress,” pose challenges because of the required prerequisite knowledge of educational theory underlying the items and the lack of specificity of the terms used. Before using either of these sections, terms like “context,” “relevant,” “formative assessment,” and “evidence” need to be defined and specified to increase inter-rater agreement. Category 1, “Alignment to the NGSS,” requires extensive knowledge of the NGSS20 as well as the Framework21 to understand the criteria. Additionally, knowledge of the theory behind modeling and the nature of science in the educational context is necessary. Synthesizing the theory and components of the NGSS is required to determine where in the lesson a dimension occurs. Achieve released the third version of the EQuIP science rubric30 as this study concluded. In examining the new rubric, the issue of the multipart criteria in Category 2, “Instructional Supports,” is still problematic in the new version. Although the new Achieve rubric has a numerical scale, it is used to give an overall score for each section of the rubric and then one for the rubric as a whole rather than the finer-grained score for each criterion in QA-EQuIP. The scores generated in this study were expressly not used to provide an overall lesson score but rather to compare lesson features (strengths and weaknesses) within the lesson set. Admittedly, EQuIP (all versions) were developed as tools for giving constructive feedback to the curriculum designer, whereas QA-EQuIP was designed to compare lessons in a set using NGSS criteria. However, we assert that the modifications made in generating QA-EQuIP can improve its utility for curriculum designers and a key audience that should be targeted by Achieve: classroom teachers. The QA-EQuIP has the potential to inform future studies to evaluate curricular alignment with NGSS. Future studies investigating the robustness of findings using QA-EQuIP with different types of raters, such as classroom teachers, can simultaneously educate teachers on NGSS criteria and curriculum change while improving existing curricular materials. Other curricula could be analyzed similarly to this study. By examining the distributions of DCIs, CCCs, and SEPs by grade band in the lesson, other existing chemistry curricula and activities could be evaluated for their NGSS alignment. Data visualizations as were used herein could help to characterize complex curricular units and collections to enable teachers and curriculum designers to make meaningful selection and design decisions about curricula. The evaluation of lessons using QA-EQuIP, including intense work with the Framework21 and NGSS appendices,27 could be the central activity of teacher professional development (PD). Achieve cites forthcoming sample lessons with explanations for each score. PD providers could use this study to develop tools for PD with explanations akin to the memos written with supporting evidence for selecting the individual dimensions and their grade bands. Carrying out this PD requires detailed, explicit

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.8b00840. Individual QA-EQuIP lesson scores (PDF, DOCX) EQuIP blank worksheet (XLSX) Sample research memos (PDF, DOCX) Researcher reflection providing additional professional development insights from this study (PDF, DOCX)

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS We sincerely thank the TIMU teachers for their participation in the program and development of the lessons. We also thank TI codeveloper Deborah Herrington for her role in inspiring this 1316



Article

(22) Cooper, M. M. Chemistry and the Next Generation Science Standards. J. Chem. Educ. 2013, 90 (6), 679−680. (23) Talanquer, V.; Sevian, H. Chemistry in Past and New Science Frameworks and Standards: Gains, Losses, and Missed Opportunities. J. Chem. Educ. 2014, 91 (1), 24−29. (24) Laverty, J. T.; Underwood, S. M.; Matz, R. L.; Posey, L. A.; Carmel, J. H.; Caballero, M. D.; Fata-Hartley, C. L.; Ebert-May, D.; Jardeleza, S. E.; Cooper, M. M. Characterizing college science assessments: the three-dimensional learning assessment protocol. PLoS One 2016, 11 (9), No. e0162333. (25) EQuIP Rubric for Lessons & Units: Science, Version 2; Next Generation Science Standards, 2014. http://www.nextgenscience.org/ sites/default/files/EQuIP%20Rubric%20for%20Science%20v2.pdf (accessed June 2019). (26) EQuIP Rubric for Science Released, 2014. Next Generation Science Standards. http://www.nextgenscience.org/news/equip-rubricscience-released (accessed June 2019). (27) NGSS Lead States. Next Generation Science Standards: For States, By States; The National Academies Press: Washington, DC, 2013; Vol. 2. (28) EQuIP Rubric for Lessons & Units: Science, Version 3; Next Generation Science Standards, 2016. http://www.nextgenscience.org/ sites/default/files/EQuIP%20Rubric%20for%20Science%20v3.pdf (accessed June 2019). (29) Corbin, J.; Strauss, A.; Strauss, A. L. Chapter 6: Memos and Diagrams. Basics of qualitative research; Sage: Thousand Oaks, CA, 2014; pp 117−124. (30) Atwood, C. H.; Sheline, R. K. Nuclear Chemistry: Include it in Your Curriculum. J. Chem. Educ. 1989, 66 (5), 389. (31) Johnstone, A. H. Why is Science Difficult to Learn? Things are seldom what they seem. Journal of Computer Assisted Learning 1991, 7, 75−83. (32) Bancroft, S. F.; Herrington, D. G.; Dumitrache, R. Semiquantitative characterization of secondary science teachers’ use of three-dimensional instruction. J. Sci. Teach. Ed. 2019, 30, 379−408.

study. Lastly, we are grateful for the support and constructive feedback from reviewers. This material is based upon work supported by the National Science Foundation under grant no. DRL-1118749.

■

REFERENCES

(1) National Research Council. National science education standards. National Academies Press: Washington, DC, 1996. (2) Chemistry in the National Science Education Standards, 2nd ed.; Lowery Bretz, S., Ed.; American Chemical Society: Washington, DC, 2008. (3) Yezierski, E. J.; Herrington, D. G. The road to scientific literacy: How Target Inquiry is improving instruction and student achievement in high school chemistry. In Exemplary Science Practices in Professional Development; Wojnowski, B., Koba, S., Eds.; National Science Teachers Association: Washington, DC, 2013. (4) Kennedy, L. M.; Yezierski, E. J.; Herrington, D. G. Whose science is it anyway? Models of science according to chemistry students, faculty, and teachers. Sci. Educ. 2008, 17 (1), 1−9. (5) Herrington, D.; Luxford, K.; Yezierski, E. Target Inquiry: Helping teachers use a research experience to transform their teaching practices. J. Chem. Educ. 2012, 89 (3), 442−448. (6) Yezierski, E. J.; Herrington, D. G. Chem. Educ. Res. Pract. 2011, 12 (3), 344−354. (7) Herrington, D. G.; Yezierski, E. J.; Luxford, K. M.; Luxford, C. J. Target inquiry: Changing chemistry high school teachers’ beliefs about inquiry instruction and their classroom practice. Chem. Educ. Res. Pract. 2011, 12 (1), 74−84. (8) Herrington, D.; Yezierski, E. Professional development aligned with AP Chemistry curriculum: Promoting science practices and facilitating enduringconceptual understanding. J. Chem. Educ. 2014, 91 (9), 1368−1374. (9) Herrington, D. G.; Yezierski, E. J.; Bancroft, S. F. Tool trouble: Challenges with using self-report data to evaluate long-term chemistry teacher professional development. J. Res. Sci. Teach. 2016, 53 (7), 1055−1081. (10) Ryan, S.; Herrington, D. G. Sticky ions: A student-centered activity using magnetic models to explore the dissolving of ionic compounds. J. Chem. Educ. 2014, 91 (6), 860−863. (11) Putti, A. All things being equal. Sci. Teach. 2012, 79 (7), 58−63. (12) Bridle, C. A.; Yezierski, E. J. Evidence for the effectiveness of inquiry-based, particulate-level instruction on conceptions of the particulate nature of matter. J. Chem. Educ. 2012, 89 (2), 192−198. (13) Cullen, D. M.; Pentecost, T. C. A model approach to the electrochemical cell: An inquiry activity. J. Chem. Educ. 2011, 88 (11), 1562−1564. (14) Putti, A. JCE Classroom Activity #109: My Acid Can Beat Up Your Acid! J. Chem. Educ. 2011, 88 (9), 1278−1280. (15) Herrington, D.; Scott, P. Get in the Game with Team Density. Sci. Teach. 2011, 78 (4), 58−61. (16) Putti, A. Student attitudes and beliefs on using the Science Writing Heuristic in an AP chemistry class. J. Chem. Educ. 2011, 88 (4), 516−521. (17) Kimberlin, S.; Yezierski, E. Effectiveness of inquiry-based lessons using particulate level models to develop high school students’ understanding of conceptual stoichiometry. J. Chem. Educ. 2016, 93 (6), 1002−1009. (18) Husting, C.; Harshman, J.; Yezierski, E. Using teacher action research in high school chemistry to develop novel assessment tools. J. Teach. Action Res. 2017, 3 (2), 14−29. (19) Vilardo, A.; MacKenzie, A. E.; Yezierski, E. J. Using students’ conceptions of air to evaluate a guided-inquiry activity classifying matter using particulate models. J. Chem. Educ. 2017, 94, 206−210. (20) NGSS Lead States. Next Generation Science Standards: For States, By States; The National Academies Press: Washington, DC, 2013. (21) National Research Council. A Framework for K−12 Science Education: Practices Crosscutting Concepts, and Core Ideas; The National Academies Press: Washington, DC, 2012. 1317


Applying the Next Generation Science Standards to Current Chemistry

Recommend Documents