Some Insights into Assessing Chemical Systems Thinking | Journal of

Article pubs.acs.org/jchemeduc

Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

Some Insights into Assessing Chemical Systems Thinking Vicente Talanquer*

Downloaded via UNIV AUTONOMA DE COAHUILA on July 25, 2019 at 15:46:45 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.

Department of Chemistry and Biochemistry, University of Arizona, Tucson, Arizona 85721, United States ABSTRACT: This paper seeks to provide some insights into the assessment of student understanding within a systems thinking perspective. Results are based on work carried out using a general chemistry curriculum that, although not developed with such a framework, shares some of its underlying intentions. After describing how chemical systems thinking is conceptualized in the paper, a specific example of an assessment tool is described and discussed to illustrate major points in our analysis. Results of a qualitative pilot study using this instrument are presented and used to highlight potentialities and challenges in teaching and assessing student understanding using a systems thinking framework. A majority of study participants expressed sophisticated ways of reasoning based on the properties and interactions of relevant components and processes in the system under consideration. Nevertheless, they could not easily connect and apply their understanding of theoretical chemical models and practices to the realities of the system. KEYWORDS: First-Year Undergraduate/General, Chemical Education Research, Testing/Assessment, Sustainability, Systems Thinking FEATURE: Chemical Education Research

■

INTRODUCTION Calls to transform chemistry education to make it more authentic and meaningful to students with diverse interests and professional goals, as well as more productive for the complex societies in which they live, are always welcome. Thus, the invitation to reimagine chemistry education using a “systems thinking” lens1−3 is interesting and provocative, but it prompts a variety of questions: What does it actually mean to infuse a systems thinking approach into chemistry education? How is this different from other existing approaches, like context-based chemistry education? What does it look like in practice? What and how should student learning be evaluated under such a framework? I suspect that the answers to these questions are not simple or definitive, and that best responses will be generated through research and reflective practice. In this regard, this contribution seeks to provide some insights into the answers to questions related to assessment of student understanding within a systems perspective based on work carried out in the US using a general chemistry curriculum that, although not developed with such a framework, shares some of its underlying intentions. Given that the meaning of a “systems thinking approach to chemistry education” is, I believe, not fully determined at this moment, it would be important to first discuss how such an approach is conceptualized in this paper. This is done in the following section. Next, I describe the types of performance expectations that assessments of student understanding could target in educational environments infused with a systems thinking perspective. To illustrate the proposed ideas, I present and analyze a specific example, followed by a summary and discussion of results from a pilot study using this assessment to explore college general chemistry students’ ability to think © XXXX American Chemical Society and Division of Chemical Education, Inc.

systemically. The main goal of this work is to contribute to the discussion about the affordances and challenges of integrating systems thinking into chemistry education.

■

CHEMICAL SYSTEMS THINKING: AN INTERPRETATION

A variety of authors from diverse disciplines have sought to define systems thinking and describe its core characteristics. For example, Arnold and Wade proposed the following definition based on their analysis of various works:4 “Systems thinking is a set of synergistic analytic skills used to improve the capability of identif ying and understanding systems, predicting their behaviors, and devising modifications to them in order to produce desired ef fects. These skills work together as a system.” Evagorou and collaborators5 described systems thinking as “the ability to understand and interpret complex systems”, while Senge6 referred to it as a conceptual framework that enables observing and understanding interrelationships between components of a system to make sense and predict patterns. Systems thinking is thought to require some core analytic skills such as ability to identify system components, recognize their properties and dynamic interactions, organize components and processes within a framework of relationships, Special Issue: Reimagining Chemistry Education: Systems Thinking, and Green and Sustainable Chemistry Received: March 12, 2019 Revised: May 24, 2019

A

DOI: 10.1021/acs.jchemed.9b00218 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

make generalizations, and think spatially and temporally at various scales.7 The way in which systems thinking has been characterized shares many similarities with how authors interested in developing students’ complex systems reasoning describe their object of interest,8,9 as well as how philosophers and educational researchers have defined mechanistic reasoning.10,11 In particular, mechanistic accounts are based on the analysis of the properties, interactions, activities, and spatiotemporal organization of entities thought to be responsible for a phenomenon or a system’s properties and behavior. This suggests that adopting a mechanistic-reasoning approach in the modeling of complex systems is one of the core elements of chemical systems thinking, but it is not the only feature that present calls to infuse systems thinking into chemistry education seek to integrate. Proposals to teach chemistry through a systems approach also advocate engaging students in the application of core chemical ideas and practices to understand and solve realworld problems, and to recognize the central role that chemical knowledge and thinking play in finding solutions to global challenges.1−3 This focus is shared with educational models for teaching chemistry in context. Several authors have analyzed and discussed the attributes and benefits of context-based chemistry education, and have identified the types of contextualized curricula and learning environments that better foster meaningful understanding.12−14 Students must, for example, recognize and value the setting within which the relevant problems to be analyzed are situated, and have opportunities to engage in authentic activity that mimics the ways of knowing, thinking, talking, and acting of experts engaged in understanding and finding solutions to such problems. A chemical systems thinking approach to chemistry education shares this context-based focus, emphasizing the importance of preparing citizens and scientists who are informed about and can responsibly act on the global challenges facing modern societies and our planet (e.g., climate change, clean water access, clean energy supplies) A third important element in the definition of a systems thinking approach includes a commitment to engage chemistry students in decision-making that takes into consideration not only chemical knowledge, but also social, economic, political, moral, and environmental factors. In this regard, a chemical systems thinking approach shares similarities with calls to problematize and humanize chemistry education by adopting sociocritical and ecoreflexive perspectives in the preparation of educated citizens and professionals who understand complex issues and engage in value-based decision-making.15,16 Thus, this approach also seeks to educate individuals who look at the world with a sustainable-action perspective, critically analyzing the complex interactions between human and earth systems and engaging in responsible action toward global sustainability. In the interpretation advanced in this contribution, chemical systems thinking involves the integration of a mechanisticreasoning approach, a context-based focus, and a sustainableaction perspective (Figure 1) in the development and application of chemical knowledge, practices, and ways of thinking in chemistry education. It is with these lenses that issues related to assessment of student understanding are analyzed and discussed in the following sections.

Figure 1. Core components of chemical systems thinking.

■

ASSESSMENT OF CHEMICAL SYSTEMS THINKING Formative and summative assessments used in learning environments that seek to foster systems thinking in chemistry education should align with the approach, focus, and perspective described in the previous section. This conceptualization of chemical systems thinking can be used as a guide in the definition of core learning objectives or performance expectations. Specific learning goals may vary depending on the context (e.g., educational level, types of students, targeted knowledge, practices, and ways of reasoning), but they should help evaluate the following major competencies: • Application of relevant understandings: Students are able to trigger and manifest different types of chemical knowledge, practices, and ways of reasoning that are productive in the analysis of complex systems or phenomena of relevance to modern societies. • Productive integration of understandings in sensemaking: Students are able to integrate relevant chemical knowledge, practices, and ways of reasoning in the construction or application of mechanistic models to make sense of targeted problem systems or phenomena. • Reflexive application of understandings in decisionmaking: Students are able to apply chemical knowledge, practices, and ways of reasoning to make informed decisions taking into consideration existing constraints and potential benefits, costs, and risks in different dimensions (e.g., individual, social, environmental, economic). In line with current frameworks for science education and associated education standards,17,18 performance expectations defined to assess student learning in learning environments that foster chemical systems thinking should aspire to evaluate the extent to which students can integrate central chemical ideas, core scientific (e.g., data interpretation, modeling, explanation and argumentation) and disciplinary (e.g., substance characterization, reaction analysis and control) practices, and crosscutting ways of thinking (e.g., structure− B



Article

Table 1. Example of an Assessment of Chemical Systems Thinking The Setting You have likely heard of the terrible water crisis that affected the city of Flint, Michigan in recent years. The city’s water supply developed high concentrations of lead, which is known to cause serious health issues, particularly in children. The pollutant came from old lead pipes that carried water to the city and the problem started when the drinking water source was changed from Lake Huron and the Detroit River to the Flint River. Assessment Components Performance Expectations Triggering initial ideas 1. How do you think lead from the solid lead pipes got into the water? • Identify relevant system components and their physical and chemical properties 2. In which form is lead present in the water? 3. How would you go about detecting the presence of lead in drinking water? Qualitative data analysis An initial hypothesis about what caused lead from the pipes to dissolve into the water suggested that the • Compare the nature and strength of chemical bonding following chemical processes may have been involved: in reactants and products to predict which species are energetically favored (i.e., have a lower potential Pb(s) + 1/2O2 (g) F PbO(s) energy). • Compare the state of matter, molar, mass, and complexity of reactants and products to predict which PbO(s) + CO2 (g) F PbCO3(s) species are entropically favored (i.e., can adopt more configurations).

PbCO3(s) F Pb2 +(aq) + CO32 −(aq) CO3 (aq) + H3O (aq) F 2−

+

HCO3−(aq)

• Integrate energetic and entropic analyses to qualitatively predict reaction directionality

+ H 2O(l)

4. Predict the extent to which each of these chemical reactions can be expected to happen based on the analysis of the composition and structure of the substances involved. Justify each of your predictions. Quantitative data analysis Experimental data at 25 °C for the proposed reactions are shown below:

Pb(s) + 1/2O2 (g) F PbO(s)

ΔH °rxn = − 218.1 kJ

PbO(s) + CO2 (g) F PbCO3(s)

ΔH °rxn = − 87.57 kJ

PbCO3(s) F Pb2 +(aq) + CO32 −(aq)

ΔS°rxn = −98.7 J/K

• Infer reaction directionality from the analysis of data for ΔHrxn and ΔSrxn.

ΔS°rxn = − 151.4 J/K

• Infer reaction directionality and extent from the analysis of data for Kc.

Kc = 7.4 × 10−14

CO32 −(aq) + H3O+(aq) F HCO3−(aq) + H 2O(l)

• Justify observed values of thermodynamic data for chemical reactions based on the analysis of the composition and structure of reactants and products

Kc = 2.1 × 1010

5. Analyze these data to evaluate the extent to which each of these reactions actually happens. Justify your claims based on the analysis of composition and structure completed in question 4. Building a potential explanation 6. The four chemical processes that you have analyzed could be used to build a potential explanation for • Build a reasonable explanation that considers the why and how lead from pipes can get into drinking water. Use the information provided, together with chemical interactions taking place within and across your chemical knowledge, to explain how a change in water source could have caused the problem. different subsystems and their effects on relevant chemical equilibria Making decisions 7. Propose a strategy to remove the lead dissolved in the drinking water or reduce its concentration. • Make sound decisions that take into consideration the Evaluate the benefits, costs, and risks of your proposal. nature and scale of the system, and their potential health and environmental impacts

we control chemical processes?)21,22 The curriculum emphasizes the development of students’ ability to apply mechanistic reasoning to build chemical rationales that support explanations, predictions, arguments, and decision-making in relevant contexts. It thus shares some of the core goals of a systems thinking approach to chemistry education. To illustrate and discuss the nature of the types of scenariobased assessments that have been developed, a specific example is presented in Table 1. This formative assessment was developed to evaluate the extent to which students who completed the one-year general chemistry course could integrate and apply their understanding of structure−property relationships and thermodynamics in the analysis of a realistic problem. In particular, the assessment asks students to analyze a set of chemical reactions potentially linked to the water crisis that afflicted the city of Flint, Michigan in recent years and use available data to build a tentative explanation for what happened. Although the explanation that can be built from the data does not correspond to what actually caused Flint’s

property reasoning, causal reasoning) in the analysis of targeted systems. The associated assessment instruments should present authentic situations in which chemical knowledge, practices, and ways of thinking are actually critical to address problems of interest or understand relevant phenomena. To illustrate these different ideas, let us analyze a specific example in detail. A Pollution Problem: Lead in Drinking Water

For several years now, instructors of the general chemistry course for science and engineering majors at our institution have engaged in the development of scenario-based formative and summative assessments of student understanding.19 Examples of the assessments that have been created are freely available to the public online.20 These assessments have been designed under the framework of the “Chemical Thinking” curriculum which is structured around essential questions that chemistry knowledge, practices, and ways of reasoning allow us to answer (e.g., How do we differentiate substances? How do C



Article

consented to participate in the study, which was approved by the Institutional Review Board at the University of Arizona. Interview transcripts were analyzed using a constant comparison method, looking for emerging themes. The analysis focused on evaluating participants’ ability to think systemically by identifying relevant model components, characterizing their properties, recognizing relevant interactions, integrating different types of knowledge and information, evaluating real system constraints, and making decisions taking into consideration diverse factors.

water crisis, the assessment seeks to evaluate the extent to which students can analyze relevant chemical data and build reasonable rationales for a complex phenomenon. The assessment presented in Table 1 creates opportunities for students to demonstrate chemical systems thinking as it demands the analysis of a complex system comprised of different subsystems (e.g., atmospheric air, liquid water from different sources, solid pipes) and diverse components (e.g., gaseous, aqueous, and solid species; ionic and molecular compounds; acids and bases). These components are linked to each other through multiple physical and chemical interactions that take place at different scales in time and space. The recognition of these interactions is critical to make sense of the targeted problem and to propose potential solutions. Students are expected to integrate chemical knowledge that is typically conveyed in compartmentalized ways in traditional general chemistry curricula (e.g., properties of elements and compounds, structure−property relationships, thermodynamic analysis of reaction directionality, chemical equilibrium, acid−base chemistry). Decision-making should take into consideration constraints imposed by the nature and scale of the system, and the health and environmental impacts that proposed solutions may have.

Thinking about Components, Properties, Interactions, and Processes

Given their academic background and experiences, participants in our pilot study were representative of students in the top 50% of science and engineering majors who take general chemistry at our institution. Nevertheless, their ability to build appropriate structure−property relationships and engage in mechanistic reasoning when confronted with the assessment described in Table 1 was varied, spanning the continuum from more novice to more advanced. The assessment had several components, and student performance was often not uniform across them. Characteristic examples of different levels of performance are described in the following paragraphs. The central goal of this analysis is to highlight major potentialities and challenges in expressed student reasoning rather than to present a detailed characterization of the nature and quantitative prevalence of specific types of understandings across the sample of participants. In general, students who gave more sophisticated responses were capable of, without much scaffolding or prodding, qualitatively predicting the extent to which each of the presented chemical reactions may occur based on the analysis of the composition and structure of the substances involved. Consider the following interview excerpt as an example. In this case, the student was discussing the thermodynamic feasibility of the reaction Pb(s) + O2(g) ⇌ PbO(s): “So, looking at this reaction, I would look at the energetic and entropic factors. Energetically, which is about intramolecular, it most likely be product-favored energetically because the products have AB bonds, which means the potential energy is lower in the product side than in the reactant side. So, in terms of energetics it would be productfavored. However, with entropics, it would be dependent on the phase. So, it would be more likely the reactants forming instead of the products forming... Now, because energetics is favored at lower temperatures, maybe at lower temperatures the product would be favored. Then, if it was at a higher temperature, we would be able to stop the product f rom forming.” (LA 21) In this example, the student was able to identify compositional and structural features of reactants and products that could be used to evaluate which of them were more energetically and entropically stable. In the Chemical Thinking curriculum, students learn to analyze how the nature of the chemical bonds in a species (e.g., covalent versus ionic, homonuclear AA versus heteronuclear AB) affects its internal potential energy (energetic stability), as well as how state of matter, molecular mass, and molecular complexity affect the number of configurations particles in a system can adopt (entropic stability). As illustrated by this excerpt, students who expressed more advanced reasoning were also able to foresee the effect that temperature may have on the directionality of

■

EXPLORING STUDENTS’ ABILITY TO THINK SYSTEMICALLY To explore our students’ ability to apply systems thinking when confronted with a problem such as the one described in the previous section, we conducted a pilot study involving a subset of learning assistants (LA) working with different instructors in our general chemistry classes. These learning assistants were undergraduate students who had successfully completed the one-year general chemistry sequence under the Chemical Thinking curriculum,21,22 and were providing support in the classroom to students currently enrolled in such courses. LAs in our program tend to be highly motivated science or engineering majors who are expected to have a solid grasp of the concepts discussed in the courses with which they work. We targeted this population of students as we were interested in characterizing what our best students could do when presented with a task that demanded the integration of different ideas, practices, and ways of thinking. A total of 25 LAs (14 Female; 11 Male) participated in an individual interview that lasted between 20 to 40 min (25 min on average). The majority of them were students in their sophomore year with aspirations to enter medical, pharmacy, or graduate schools. All of them had one or two semesters of experience serving as learning assistants in general chemistry courses. During the interview, each participant was asked to answer the different questions included in the assessment described in Table 1. The questions were presented verbally to them and they were asked to provide oral responses. The interviewer asked follow up questions as needed to better explore student thinking or to scaffold student reasoning. These questions varied depending of the responses and ideas expressed by each participant. The interview was not conceived as an evaluation of student knowledge or as an in-depth characterization of student understanding of specific chemistry concepts or ideas. It was rather an exploration of what participants could accomplish with or without prompting. Thus, the interviewer provided hints or directed participants’ attention to particular information when needed to better characterize the extent of their understanding. All interviewees D



Article

provided. Consider this example regarding the analysis of the reaction PbO(s) + CO2(g) ⇌ PbCO3(s): “I’m going to say that it is possible for it to happen. Based on... This is a little more complex (pointing to PbCO3), even though complexity has to do with ΔS. I think it had something to do with energy too, and more complex is more stable.” (LA 22) In this situation, the student recognized that molecular complexity affected the entropy of the system, but improperly connected the idea with energetic considerations. It was common for these types of students to invoke concepts and ideas that were relevant, but use them in noncanonical ways. They also frequently based their judgments on the analysis of a single factor, like the following excerpt from the analysis of the reaction PbCO3(s) ⇌ Pb2+(aq) + CO32−(aq) illustrates: “Their charges in the products are kind of high, so will be less likely to happen because they are more attracted to each other.” (LA 5) Although the expressed idea was relevant and led the student to make a reasonable prediction, the decision was made without taking into consideration other relevant factors (e.g., potential entropy gains due to the dispersal of ions in solution, potential decrease in potential energy due to strong interactions of the ions with water). In accordance with the assessment in Table 1, participating students were also asked to analyze and interpret the experimental data provided in the assessment. In general, most students completed the data analysis without major problems, although a minority required prodding. Most participants recognized the meaning of the different quantities and the implications of their signs and values. They were able to relate the experimental results with their qualitative predictions, and to reconsider their answers when discrepancies occurred. Their ability to resolve those discrepancies, however, depended on the level of sophistication in their reasoning. Participants in the pilot study were also asked to use the information provided to generate a reasonable explanation for the problem under consideration (i.e., increased concentration of lead in water when the source of drinking water was changed). This was a more difficult task for the majority of the students, as they struggled to recognize interactions between the different chemical processes that they were asked to analyze and to make sense of how changes in the system would affect different chemical equilibria. The more novice students tended to generate explanations based on generic effects, without reference to a detailed mechanism as illustrated by this interview excerpt: “In order for it to become more product-favored (referring to the reaction PbCO3(s) ⇌ Pb2+(aq) + CO32−(aq)) maybe more of the products were added to it. Or maybe a change in temperature or pressure. Maybe a phase change at some point. It could be that the pH of the acid was more acidic or basic.” (LA 25) Nevertheless, a significant number of students in the sample were able to recognize potential connections between some of the chemical processes outlined in the assessment, but stumbled to relate all of them in a coherent chemical story, as shown in this example:

the process (e.g., product-favored at low temperature but reactant-favored at high temperatures in this case). Students further along the novice-expert continuum demonstrated independent ability to recognize relevant model components in a system (e.g., metallic versus molecular versus ionic substances, homonuclear versus heteronuclear bonds), identify relevant properties of those components (e.g., heteronuclear bonds tend to be stronger than homonuclear bonds), simultaneously consider the effect of different variables on the behavior of a system (e.g., energetic and entropic factors), and integrate different concepts and ideas when making predictions (e.g., integrating structural information at the submicroscopic level with thermodynamic quantities at the macroscopic level). Nevertheless, a majority of study participants required considerable scaffolding and prodding during the interview to engage in these types of thinking. They tended to initially provide quick and superficial answers, focusing on a single variable or applying quick heuristic rules to make decisions, but were able to expand and deepen their thinking when probed. This behavior is illustrated in the following interaction related to the analysis of the reaction CO32−(aq) + H3O+(aq) ⇌ HCO3−(aq) + H2O(l): “Student: This seems more like an acid−base reaction, because you are donating a proton to CO32−. So, are you asking about favorability? Interviewer: Yes. Student: So, favorability... This would be entropically reactant-favored, I think. Interviewer: Why is that? Student: More aqueous moles. But, let’s see. Liquids have lower potential energy. So, energy wise, I feel it would favor products. In this case you are breaking and forming AB bonds. Interviewer: OK, so you said that this was an acid−base reaction. Any ideas of how you could think about it? Student: Oh yeah, you could look at the bases and see which one is more stable. So, basically there are two bases. I think this one would be more stable (pointing to water). Interviewer: Why is that? Student: Because this one is already charged (pointing to CO32−), with a two minus charge, so I think that would make it more reactive. It has extra electrons and is gonna be more likely to bond to something else. And this one is neutral (pointing to water). So, it would be more stable and more likely to form.” (LA 15) In this case, the student began the analysis by superficially trying to evaluate the relative energetic and entropic stability of reactants and products, without recognizing that the types of ideas used to analyze previous cases were not necessarily productive in the analysis of the acid−base reaction. However, when the interviewer reminded her about the nature of the reaction, the student quickly switched to a mode of reasoning that was useful in making a reasonable prediction (i.e., acid− base reactions are favored in the direction that generates the most stable, or least reactive, species). Participants who expressed more novice-like reasoning experienced a variety of challenges to carry out the required analyses. In general, they struggled to recall relevant concepts, failed to recognize the types of components present in a system or their properties, and more frequently relied on intuitive ideas when making inferences. They expressed relevant pieces of knowledge but struggled to apply them in proper, productive, or coherent ways even when scaffolding was E



Article

“So, there is these two reactions that kind of connect to each other. Because you went from lead to lead oxide that then reacted with carbon dioxide to produce this compound (pointing to PbCO3). And then in the third reaction you have the product of the previous as the reactant and going to the lead ion that could be present in the water... I think the fourth reaction is throwing me off because I do not see lead in it...” (LA 23) As illustrated in the previous example, many study participants were able to look at the chemical reactions as written and engage in “forward chaining” to predict how the products of one reaction may participate as reactants in another process. Many of them, however, struggled to engage in “backward chaining”, analyzing the effects that the consumption of a reactant in one process would have on a different reaction in which such a species appeared as a product. Nevertheless, with different degrees of scaffolding, a majority of the participants built a reasonable causal story in which increased levels of water acidity would result in increased dissolution of the lead carbonate formed by the interaction of metallic lead in the pipes with oxygen and carbon dioxide in the water or the atmosphere. In particular, students who performed at more advanced levels were able to integrate different components of their analysis to build solid explanations as shown in the example below: “...well, if we are assuming the lead is dissolved in the water, the lead would have to be aqueous at some point in time, so it would have to be in ionic form like this (pointing to Pb2+(aq)). However, we know this reaction specifically (referring to the reaction PbCO3(s) ⇌ Pb2+(aq) + CO32−(aq)) is not favored at any temperature. So, it would be a very minimal amount of lead ions formed in this way. Unless you can favor the forward reaction. So, I guess, theoretically, if there was an increase in H3O+ions, this could take CO32−ions out of the system because this reaction (referring to CO32−(aq) + H3O+(aq) ⇌ HCO3−(aq) + H2O(l)) is extremely favored. So, what could be occurring is that this (pointing to PbCO3(s)) would be the composition of the pipes and a very small amount then dissociates, but since you are constantly removing the CO32−by this extremely favored process, it would then cause an accumulation of lead ions over time as the dissolution shif ts to products.” (LA 6)

by selecting inappropriate techniques (e.g., IR spectroscopy of water samples). Most students in the sample proposed strategies to reduce the concentration of lead in water that indicated lack of awareness or comprehension of the scale of the system and of practical constraints. They would propose, for example, to add more water to the system, to precipitate lead by adding other substances (e.g., Na2CO3, bromide), or to add a basic substance to increase the pH (e.g., CO32−, NaOH). These proposals were made, however, without much consideration of the actual spatial and temporal dimensions of the real system, and of any practical methods for and challenges in the implementation of any measure at a large scale and for a sustained period of time. Similarly, participants in the pilot study did not take into consideration any environmental, health, economic, or social concerns related to the proposals they made. Maybe not surprisingly, their mindset seemed to be that of a student asked to solve a small-scale problem in an academic lab rather than that of a professional dealing with a major environmental and health crisis.

■

DISCUSSION AND IMPLICATIONS Participants in our pilot study had completed a one-year course in general chemistry that emphasized some aspects of chemical systems thinking as defined in this paper, but not all. They had had diverse opportunities to engage in structure−property reasoning and mechanistic reasoning, and to apply and integrate core chemical concepts and ideas to build arguments and generate explanations about different phenomena in diverse contexts. Nevertheless, the focus of the course had been rather academic, working with rather simplified models of relevant systems and without much consideration of real effects and constraints imposed by the actual spatial and temporal scale of the systems under analysis. Similarly, their engagement in the analysis of environmental, health, economic, and social benefits, costs, and risks of chemical activity had been limited and far from systematic. Despite these circumstances, their performance in the assessment presented in Table 1 reveals potentialities and challenges in teaching and assessing student understanding using a systems thinking framework. A majority of participating students expressed sophisticated ways of reasoning based on the analysis of the properties and interactions of relevant components and processes in the system under consideration. Many of them, however, did it after significant prodding and scaffolding. This suggests that for most students in introductory chemistry courses the development of the type of mechanistic reasoning that systems thinking demands will require time, multiple opportunities to engage in it with proper formative feedback, and well-thought out scaffolding. The assessment of this type of understanding should rely on a variety of tools that allow students to express their ideas in different ways, and press them to move beyond heuristic responses based on the quick identification of surface features. Using the types of 3D assessments developed under the umbrella of the Next Generation Science Standards may be a good start,23 as these assessments explore students’ abilities to integrate their understanding of central ideas, sciences practices, and crosscutting ways of reasoning. I suspect, however, that more authentic and performance-based tools will have to be considered to properly assess chemical systems thinking. The results of the pilot study also suggest that developing students’ ability to think reflexively in context will require

Thinking Reflexively in Context

Although many students in our sample expressed sophisticated theoretical reasoning about the entities present in the system and the chemical processes in which they were involved, their ̈ Most participants reflexive thinking in context was quite naive. could identify and describe general interactions that might take place between different system components, such as oxygen and carbon dioxide in the atmosphere with the solid lead pipes, or acidic substances in the water with the products of lead oxidation. But their views of the actual nature of some of the substances involved, and of how to detect, quantitate, or separate them in their real environments were many times ingenuous. Some of them, for example, thought that lead was found in metallic form in water and that one could separate it by filtrating or evaporating water. To detect the presence of the pollutant, several participants mentioned the use of indicators or an unspecified reagent that may selectively react with lead. Some others mentioned spectroscopic techniques, but many did it in generic ways without identifying a specific method or F



■

active, continuous, and sustained engagement with instructional tasks that explicitly ask them to analyze real “messy” data from targeted systems, develop a clear sense of the nature and spatiotemporal scale of relevant chemical entities and processes, and evaluate how these factors enable or constrain the actions that can be taken to address a problem. Participants in our pilot study could not easily connect and apply their understanding of theoretical chemical models and practices to the realities of the system under analysis. For the most part, they were not able to translate their theoretical understandings into concrete reasonable actions in the real world. It would then be critical for curricula developed under a systems thinking perspective to create multiple opportunities for students to engage in argumentation and decision-making, evaluating the feasibility of their ideas in real settings. Our data also point to the major challenge of developing a sustainable-action perspective in student reasoning. None of the actions suggested by the participants in our pilot study seemed to consider their impacts on human health or the environment. The issues with which we would like our students to grapple involve complex interactions taking place at many levels and in different spheres. Finding the right balance between a focus on the analysis of critical chemical concepts and the consideration of important environmental, sociological, and economical factors may be difficult, particularly if major changes in the scope of traditional chemical curricula are not introduced. Assessing students’ ability to reasonably take these types of factors into consideration when making and justifying decisions may also be challenging for chemistry instructors typically not trained in the evaluation of socioscientific argumentation.24 Adopting a systems thinking framework in curriculum development, instruction, and assessment in chemistry courses at all educational levels would help us prepare individuals more adept at understanding and making decisions concerning the complex problems facing the world. The challenges in making the switch, however, should not be minimized. It is unlikely that minor changes in traditional curricula and educational practices will have a meaningful impact on student learning. At our institution, we have been working for more than ten years in transforming our general chemistry curriculum to emphasize some aspects of chemical systems thinking, and it is clear from the results presented in this contribution that we have fallen short in some important areas. Changing old habits and ingrained conceptions about what chemistry education ought to be is not easy, even for those with the best disposition. But these changes should be seen as an imperative if we want chemistry to remain at the center in the preparation of future citizens and science-related professionals.

■

Article

REFERENCES

(1) Matlin, S. A.; Mehta, G.; Hopf, H.; Krief, A. One-World Chemistry and Systems Thinking. Nat. Chem. 2016, 8, 393−398. (2) Mahaffy, P. G.; Krief, A.; Hopf, H.; Matlin, S. A.; Mehta, G. Reorienting Chemistry Education through Systems Thinking. Nat. Rev. Chem. 2018, 2 (4), 0126. (3) Mahaffy, P. G.; Brush, E. J.; Haack, J. A.; Ho, F. M. Journal of Chemical Education Call for Papers: Special Issue on Reimagining Chemistry Education: Systems Thinking, and Green and Sustainable Chemistry. J. Chem. Educ. 2018, 95, 1689−1691. (4) Arnold, R. D.; Wade, J. P. A Definition of Systems Thinking: A Systems Approach. Procedia Comput. Sci. 2015, 44, 669−678. (5) Evagorou, M.; Korfiatis, K.; Nicolaou, C.; Constantinou, C. An Investigation of the Potential of Interactive Simulations for Developing System Thinking Skills in Elementary School: A Case Study with Fifth-Graders. Int. J. Sci. Educ. 2009, 31, 655−674. (6) Senge, P. M. The Fifth Discipline: The Art and Practice of the Learning Organization; Doubleday: New York, NY, 1990. (7) Assaraf, O. B.-Z.; Orion, N. Development of System Thinking Skills in the Context of Earth System Education. J. Res. Sci. Teach. 2005, 42, 518−560. (8) Jacobson, M. J.; Wilensky, U. Complex Systems in Education: Scientific and Educational Importance and Implications for the Learning Sciences. J. Learn. Sci. 2006, 15, 11−34. (9) Hmelo-Silver, C. E.; Azevedo, R. Understanding Complex Systems: Some Core Challenges. J. Learn. Sci. 2006, 15, 53−61. (10) Machamer, P.; Darden, L.; Craver, C. Thinking About Mechanisms. Philos. Sci. 2000, 67 (1), 1−25. (11) Russ, R. S.; Scherr, R. E.; Hammer, D.; Mikeska, J. Recognizing Mechanistic Reasoning in Student Scientific Inquiry: A Framework for Discourse Analysis Developed from Philosophy of Science. Sci. Educ. 2008, 92 (3), 499−525. (12) Gilbert, J. K. On the Nature of ‘Context’ in Chemical Education. Int. J. Sci. Educ. 2006, 28, 957−976. (13) Gilbert, J. K.; Bulte, A. M. W.; Pilot, A. Concept Development and Transfer in Context Based Science Education. Int. J. Sci. Educ. 2011, 33, 817−837. (14) Sevian, H.; Dori, Y. J.; Parchmann, I. How Does STEM Context-Based Learning Work: What We Know and What We Still Do Not Know. Int. J. Sci. Educ. 2018, 40, 1095−1107. (15) Sjöström, J.; Eilks, I. Reconsidering Different Visions of Scientific Literacy and Science Education Based on the Concept of Bildung. In Cognition, Metacognition and Culture in STEM Education; Dori, Y. J., Mevarech, Z., Baker, D., Eds.; Springer: Cham, 2018; pp 65−88. (16) Sjöström, J.; Talanquer, V. Eco-Reflexive Chemical Thinking and Action. Curr. Opin. Green Sus. Chem. 2018, 13, 16−20. (17) National Research Council (NRC). A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas. Committee on a Conceptual Framework for New K-12 Science Education Standards; The National Academies Press: Washington, DC, 2011. (18) National Research Council (NRC). The Next Generation Science Standards; The National Academies Press: Washington, DC, 2013. (19) Talanquer, V. Assessing for Chemical Thinking. In Research and Practice in Chemistry Education: Advances from the 25th IUPAC International Conference on Chemistry Education; Schultz, M., Schmid, S., Lawrie, G. A., Eds.; Springer: Singapore, 2019. (20) Chemical Thinking Website, https://sites.google.com/site/ chemicalthinking/tasks (accessed May 2019). (21) Talanquer, V.; Pollard, J. Let’s Teach How We Think Instead of What We Know. Chem. Educ. Res. Pract. 2010, 11, 74−83. (22) Talanquer, V.; Pollard, J. Reforming a Large Foundational Course: Successes and Challenges. J. Chem. Educ. 2017, 94, 1844− 1851. (23) Laverty, J. T.; Underwood, S. M.; Matz, R. L.; Posey, L. A.; Carmel, J. H.; Caballero, M. D.; Fata-Hartley, C. L.; Ebert-May, D.; Jardeleza, S. E.; Cooper, M. M. Characterizing College Science Assessments: The Three-Dimensional Learning Assessment Protocol. PLoS One 2016, 11 (9), No. e0162333.

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. ORCID

Vicente Talanquer: 0000-0002-5737-3313 Notes

The author declares no competing financial interest.

■

ACKNOWLEDGMENTS The author greatly appreciates the voluntary participation of the learning assistants involved in the pilot study. G



Article

(24) Simonneaux, L. Argumentation in Socio-Scientific Contexts. In Argumentation in Science Education; Erduran, S., Jiménez-Aleixandre, M. P., Eds.; Springer: Doetinchem, 2008; pp 179−199.

H


Some Insights into Assessing Chemical Systems Thinking | Journal of

Recommend Documents