Facilitating Argumentation in the Laboratory: The Challenges of Claim

This is an open access article published under an ACS AuthorChoice License, which permits copying and redistribution of the article or any adaptations for non-commercial purposes.

Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

pubs.acs.org/jchemeduc

Facilitating Argumentation in the Laboratory: The Challenges of Claim Change and Justification by Theory Joi Phelps Walker,*,† Andrea Gay Van Duzor,‡ and Meghan A. Lower† †

Department of Chemistry, East Carolina University, Greenville, North Carolina 27858, United States Department of Chemistry, Physics, and Engineering Studies, Chicago State University, Chicago, Illinois 60628, United States

‡

J. Chem. Educ. Downloaded from pubs.acs.org by 95.85.71.150 on 02/10/19. For personal use only.

S Supporting Information *

ABSTRACT: Scientific argumentation is a key means by which students make sense of content and processes in inquiry-based instruction. In scientific argumentation students make a claim that they support with evidence and provide reasoning as to how the evidence supports the claim. Integrating the different aspects of scientific argumentation can be challenging for students especially if they need to make a claim change to accommodate new evidence. This study examines student argumentation within a two-semester general chemistry laboratory sequence at a minority-serving, comprehensive university in the Midwest, which employed the ArgumentDriven Inquiry (ADI) instructional model for laboratory instruction. Video recordings of group argumentation from four investigations, two each semester, were coded using the Assessment of Scientific Argumentation in the Classroom (ASAC) observation protocol. On the basis of the data, students did not tend to change their claims or reasoning even when provided with contradictory evidence. Semistructured interviews with students also coded with the ASAC revealed that students did not view claim change as a valid component of argumentation. KEYWORDS: General Public, Chemical Education Research, Laboratory Instruction, Problem Solving/Decision Making FEATURE: Chemical Education Research

■

INTRODUCTION Scientists construct explanations, develop models, argue from evidence, and evaluate information as they attempt to develop new scientific knowledge. These activities, as a result, are important epistemic practices of the scientific community. Accordingly, the Framework for K−12 Science Education,1 which serves as the foundation for the Next Generation Science Standards (NGSS),2 emphasizes that students learn to “argue f rom evidence”. Participation in arguing from evidence is not simply a tool that allows for deeper learning of science content; indeed, the ability to participate in such discourse is an essential aspect of the scientific enterprise that students should learn not just at the K−12 level but at the postsecondary level as well. In pursuit of productive epistemic discourse, multiple frameworks for scientific argumentation have been developed to facilitate instructional activities and research.3,4 In scientific argumentation students make a claim that they support with evidence and provide reasoning as to how the evidence supports the claim. Scientific argumentation can be challenging for students as they must learn to differentiate claims from hypotheses, data from evidence, and implicit from explicit reasoning.5 Within the context of science, argumentation skills are met with an additional challenge of using what may be a surface understanding of scientific theory or concepts to support or justify evidence. The result is that, even when students have grasped the concept of argumentation, they will submit their data as evidence in an all-inclusive manner, © XXXX American Chemical Society and Division of Chemical Education, Inc.

neglecting to rationalize the use of this evidence in supporting their claim.6−8 Particularly difficult for students is the rebuttal aspect of scientific argumentation especially if they need to make a claim change to accommodate new evidence or ideas.5 The research on student argumentation presented in this paper is distinguished from much of the work on argumentation as it is with undergraduate students, as opposed to middle or secondary school students. In addition, this study is not focused on the individual but looks at the ability of groups of students to take up the norms of scientific discourse through repeated opportunities to engage in scientific argumentation. A discipline specific laboratory course presents a unique context for providing students an opportunity to participate in and develop proficiency in the scientific practices that are valued within a discipline including argumentation. The use of laboratory to facilitate student development of argumentation is supported by the assertion that observation and experiment are not the foundation on which science is built, but rather they are the pathway to the rational activity of generating arguments in support of knowledge claims.9 There are a broad range of instructional models for the chemistry laboratory, such as the Science Writing Heuristic (SWH), Cooperative Chemistry, Process Oriented Guided Inquiry Learning (POGIL), and Argument-Driven Inquiry Received: September 12, 2018 Revised: January 14, 2019

A

DOI: 10.1021/acs.jchemed.8b00745 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(ADI).10−12 A significant body of published work has established the benefits of these curricula over traditional expository methods. This paper attempts to tease out some of the challenges of curriculum implementation and dissemination. The implementation of ADI, at a state university located in a major urban center in the Midwest, began in 2013 with the intention of replicating previous research at a community college in the southeast.6 This work is ongoing, but analysis of student argumentation sessions revealed two areas critical to argumentation that students were not engaging in sufficiently. Students were unwilling to change their claim even when presented with a compelling rationale, and students were not using scientific theory or concepts to justify their evidence.

content.18 The researchers found a positive relationship between content familiarity and high-quality argumentation particularly with epistemic and social aspects. Perhaps more significantly the researchers found that participation in argumentation over the course of school year meant that students could engage in high-quality argumentation as reflected in the use of epistemic characteristics (i.e., use of evidence, science theory, or models) of argumentation even when the science content was unfamiliar. Argument-Driven Inquiry

Argument-Driven Inquiry (ADI) is a process-oriented model designed to give a more central place to argumentation and the role of argument in the social construction of scientific knowledge while promoting inquiry.19 It is a multistage instructional model for science laboratories that engages students in investigation design, data collection and analysis, argument generation, group argumentation and critique, scientific writing, and double-blind peer review. ADI requires that students engage in these practices paying particular attention to empirical criteria as they use science ideas to interpret data and make sense of natural phenomena.20 Research at the community college level comparing chemistry laboratory sections taught using ADI to traditional methods of instruction found the following: (1) There were improved student attitudes toward science, with a significant positive gender effect for female students. (2) Student performance on tasks requiring analysis of data and argument construction showed a significant positive difference for students in the ADI sections.21 (3) Student performance on an end-of-course practical exam that required students to engage in multiple science practices (investigation design, data collection, observation, and inference and argument construction) revealed not only a significant positive difference for students in the ADI sections, but also a closing of the achievement gap for underrepresented minority students in the ADI sections.22 (4) In a repeated measures research study students in ADI laboratory classes showed significant improvement in written lab reports23,24 and significant growth in argumentation ability over the course of a semester.6 All of these studies were in a single context and looked at individual performance over a single semester. This study represents propagation of the ADI curriculum to a state university, data collection over both semesters of a general chemistry sequence, and a fine-grained consideration of key aspects of scientific argumentation within a group. The ADI approach consists of the following steps that take place over a 4 week period for a typical once-a-week laboratory course: (1) prelaboratory activity in preparation for the investigation providing direct instruction on techniques and equipment; (2) instructor introducing the task and providing a guiding question; (3) students working in groups to outline a method for data collection and analysis needed to answer the guiding question; (4) students implementing their investigation plan and collecting data;

■

BACKGROUND Scientific argumentation is a central means by which students make sense of content and processes in inquiry-based instruction. Indeed, it has been suggested that understanding science requires an understanding of how to construct a scientific argument.13 Sampson and Clark4 noted that it is important for students to know “what counts” in generating an argument and emphasized the need for research on the structural, conceptual, epistemic, and social aspects of argument generation. The structure of Duschl’s work on 3part harmony looks to these same elements, specifically cognitive, which is to justify a claim with scientific theory or laws; epistemic, which is to use data as evidence to support the claim and question the data collection and analysis; and finally the social aspect which distinguishes a scientific argument from a quarrel or a discussion.14 Research focused on group argumentation has established some common behaviors across contexts that in general find that students struggle to attend to all three aspects of argumentation even when given the opportunity. Generally, students will make a claim, but the extent to which the claim is based on science theory or sufficiently valid or meaningful evidence varies. Berland and Reiser15 describe student explanations as differing in the level of evidence vs inferences and persuasive statements. Students use evidence in personal sense-making but are not able to transfer to an audience beyond “correct answer”.16 Kuhn found that the social nature of argumentation was key in representing a more accurate vision of science. Researchers as a whole suggest that attention needs to be paid to epistemology: investigation of what distinguishes justified claim from opinion. Novak and Treagust17 focused on the question of students adjusting claims as new evidence emerged over time. Over a 6 week period the students collected data on water quality. At four points in the unit students were asked to construct an argument; this required them to reconsider their previous claim in light of new evidence. The findings indicate that many students face challenges adjusting their claims when new, conflicting evidence emerges, even with class discussion, teacher feedback, and written scaffolds. Possible reasons to account for this challenge include the following: (1) students ignoring new evidence, (2) students find “undoing” their initial idea too cognitively demanding, or (3) students simply do not have any similar experience from which to build.17 Recent research using an instructional model designed to facilitate and scaffold scientific argumentation investigated the development of cognitive, epistemic, and social aspects over the course of a school year with familiar and unfamiliar B



Article

Figure 1. Weekly structure and pacing for ADI laboratories.

courses, respectively. Depending on the instructor, the lecture portion of the courses either follow a more traditional structure with group problem solving sessions or utilize guided inquiry through small group work. All of the General Chemistry I lecture sections used guided inquiry. Students at the university typically embrace group learning opportunities with little resistance, which is a strength of the learning environment.26 Prior to adoption of the ADI curriculum, the department had used a combination of commercially available and instructor written chemistry experiments for the general chemistry sequence. While the previous general chemistry laboratory curriculum included guided inquiry experiences, the experiments were disjointed from each other and there was no writing framework beyond an abstract. Successful use of the Decision/Explanation/ Observation/Inquiry writing method in organic chemistry laboratory27 helped convince faculty of the need for a coherent and comprehensive approach to the general chemistry laboratory curricula, which would include writing. The chemistry curriculum committee chose the ADI model and curriculum because it could provide comprehensive change with inquiry-based experiments, scientific argumentation, and faculty professional development. The primary author provided a day of in-service training for the faculty prior to implementation of ADI. The training was supplemented with instructor notes and regular dialogue with the faculty leading the laboratory transformation. After a year of implementation, a follow-up training and observation took place on site. Students were observed in week 2 of an investigation in both general chemistry courses. The instructors were pleased with the student engagement and the dissemination of ADI was considered to be successful.

(5) students analyzing their data and developing a tentative argument (a claim supported by evidence and a justification of the evidence); (6) students sharing their arguments and critiquing the arguments of their peers during an argumentation session; (7) each student writing an investigation report; (8) the reports going through a double-blind peer review; and (9) students being given an opportunity to revise and submit their report to the instructor for evaluation. The final report is submitted during week 4 of the cycle. The experiments overlap so that steps 1, 2, and 3, which are week 1 of the second lab, are done in the same lab period as step 8, which is week 3 of the first lab. Figure 1 illustrates the weekly structure.

■

RATIONALE AND RESEARCH QUESTION In a chemistry laboratory course, inquiry-based instruction models scientific methods by prompting students to design experiments to answer broad, open-ended questions. Argumentation is a natural outcome of an inquiry-based lab; as students engage in sense-making with peers, they will employ the key elements of scientific argumentation: claim, evidence, reasoning, and rebuttal. The research presented here uses video and interview data to follow students in a two-semester general chemistry laboratory sequence in order to determine which elements of argumentation are most challenging for students despite embedded instruction and scaffolding. Research Question

What elements of argumentation are challenging for students in a chemistry laboratory course using the Argument-Driven Inquiry instructional model?

■

Participants

DATA COLLECTION AND ANALYSIS

Students enrolled in the general chemistry laboratory courses were asked to participate in the research study through an informed consent protocol approved the university’s Internal Review Board. All students in a given lab section had to agree to the video recording; however, students were generally not concerned with the request for video recording. Tracking participants from one semester to the next was an issue, since as described above the university serves a population with a wide variety of unpredictable issues that impact an individual student’s ability to complete a course or a semester and stay enrolled for sequential semesters. Attrition in the classes limited the total number of videos, which started the semester with 5−6 groups, but these were reduced to as few as three by the end of the semester. Of the 15 students from whom data from the fall semester was collected, 9 were also in the spring

Context

The study took place at a state university located in a major urban center in the Midwest. This university is a less selective, public, comprehensive institution and predominantly serves the local community. In 2015, 74.3% of the undergraduate student population identified as African-American or Black and 10.0% as Hispanic or Latinx. Of undergraduates, 70.8% are female, and 34.7% are attending part-time.25 Due to the large number of nontraditional students, content background of students in general chemistry varies greatly with some students having completed high school chemistry recently, some 10−15 years prior, and some never. General Chemistry I and II laboratory are one-credit courses, which meet 3 hours per week and are corequisites of General Chemistry I and II lecture C



Article

Table 1. Investigation Descriptions Event

Concept

GC1.1

Physical Properties

GC1.5

Chemical Reactions

GC2.1

Intermolecular Forces

GC2.5

Electrochemistry

Guiding Question

Description

Are these objects made of the same material? What is the best way to remove silver ions for water? Why do liquids evaporate at different rates? Which metals make the best battery?

Students are given three objects of varying shapes and sizes. At least one of the objects does not fit in a graduated cylinder for water displacement which requires use of a spill can. Students are provided a solution of 0.1 M AgNO3 and are asked to remove the silver ion by redox reaction with copper metal or by precipitation with aqueous CaCl2. Using a temperature probe, students determine the rate of evaporation for 7 liquids: pentane, acetone, methanol, ethanol, propanol, butanol, and water. Copper metal and a CuSO4 solution are provided to serve as a half-cell; students construct complete cells with 4 unknown metals, lead, zinc, silver, and iron. Using the cell potential, they identify the metals and propose a combination for the best battery.

Figure 2. Argumentation session illustration depicting rotation of travelers from a single group to the presenters for three other groups.

supports the claim based on chemical content. For instance, in the density laboratory, the claim may be “The unknown objects are not made of the same material”. The evidence could include a series of calculated densities using multiple means to determine volume, and the rationale could comprise the chemical concept of density as a physical property as well as discussion of experimental error. An argumentation session (step 6) involves splitting of the core group into 2−3 traveling members and one presenting member. The traveling members typically visit 2−3 whiteboards resulting in formation of a transitory group with the presenter (see Figure 2). Each new group of students engaged in the argumentation which was structured to be persuasive and evaluative in nature, and then the travelers moved to the next presenter. Following the argumentation session, each original laboratory group

semester data set for General Chemistry II and the majority of students were previously exposed to the ADI method in General Chemistry I laboratory course. All names are pseudonyms. Data Collection

Episodes of argumentation (steps 5 and 6) were recorded for the first and last investigation each semester of general chemistry (GC1 and GC2) laboratory. The four investigations are described in Table 1. Episodes of argumentation were video recorded during the first and last investigation of each semester. In creating a tentative argument (step 5), a laboratory group will share their ideas with each other and then create a whiteboard poster that explicitly states their claim, presents evidence from the laboratory, and provides reasoning as to how their evidence D



Article

Table 2. ASAC Score Distributions for 12 Videos: Never−Somewhat−Often (N = 12) Item Q1 Q2 Q3 Q4 Q5 Q6 Q7 Q8 Q9 Q10 Q11 Q12 Q13 Q14 Q15 Q16 Q17 Q18

Description

Never Somewhat

Conceptual and Cognitive The talk of the group was focused on solving a problem or advancing understanding. The participants sought out and discussed alternative claims or explanations. The participants modified their explanation or claim when they noticed an inconsistency or discovered anomalous information. The participants were skeptical of ideas and information. The participants provided reasons when supporting or challenging an idea. The participants attempted to evaluate the merits of each alternative claim or explanation in a systematic manner. Epistemic The participants used evidence to support and challenge ideas or to make sense of the phenomenon under investigation. The participants examined the relevance, coherence, and sufficiency of the evidence. The participants evaluated how the data was gathered, analyzed, or interpreted. The participants used scientific theories, laws, or models to support and challenge ideas or to help make sense of the phenomenon under investigation. The participants made distinctions and connections between inferences and observations explicit to others. The participants used the language of science to communicate ideas. Social The participants were reflective about what they know and how they know. The participants respected what each other had to say. The participants discussed an idea when it was introduced into the conversation. The participants encouraged or invited others to share or critique ideas. The participants restated or summarized comments and asked each other to clarify or elaborate on their comments. There was equal participation from all members of the group.

reconvened and discussed their results in light of their discussion with the other groups. The traveling members shared what they had seen in other groups’ presentations, and the presenting member shared what they had learned from being questioned by other groups. This follow-up session provided the opportunity for groups to make a claim change or reinterpret the evidence. A video camera was set up to capture the discussion at each white board. This included the presentation and discussion with each transitory group as well as the postargumentation discussion of the core group. The entire video recording was scored as a single argumentation event. Instructors were asked to simply observe the argumentation session and not intervene with questioning unless discussion had ground to a halt or if a major issue was ignored.

Often

0 2 6

4 2 6

8 8 0

1 2 2

6 4 10

5 6 0

1 3 0 7

6 6 5 5

5 3 7 0

3 1

7 9

2 2

1 0 0 0 1 1

6 0 6 5 8 4

5 12 6 7 3 7

(1) the conceptual structures and cognitive processes used when reasoning scientifically; (2) the epistemic frameworks used when developing and evaluating scientific knowledge; and (3) the social processes and contexts that shape how knowledge is communicated, represented, argued, and debated. The Conceptual and Cognitive Aspects of Argumentation section consists of six items, which enable the researcher to evaluate the participants’ focus on problem solving, evaluation of alternative claims, willingness to consider anomalous data, skepticism of ideas, ability to provide reasons in support of ideas, evaluation of alternative claims, and, in contrast, use of inappropriate reasoning strategies. The Epistemic Aspects of Argumentation section contains six items, which address the participants’ use of evidence to make sense of the phenomenon; evaluation of the evidence; use of scientific theories, laws, or models; ability to distinguish inferences and observations; and use of the language of science as well as use of inappropriate rhetoric to manipulate a claim. In this section, there are several items that address the use of evidence. The Social Aspects of Argumentation section contains six items, with which the researcher will be able to evaluate the interactions of the participants. These items assess the participants’ ability to be reflective about what they say, their respect for each other, their willingness to discuss ideas introduced by others, their willingness to solicit ideas from others, their tendency to ask for clarification or elaboration of other’s ideas, and their overall participation. Each section has 6 items that are scored 0, 1, or 2: A score of 0 or Never indicates that the item was not observed, a score of 1 or Sometimes indicates the item was observed at least once, and a score of 2 or Often indicates the items was observed multiple times. Each section, therefore, can have a maximum score of 12 with an overall score for the instrument of 36.

Discourse Analysis

In order to evaluate argumentation, most researchers video or audio record students as they engage in argumentation, then transcribe the discourse, and finally code or score the transcription using a framework such as Toulmin’s Argument Pattern.3 This process can be difficult for researchers because argumentation is often nonlinear in nature and the various aspects of a verbal argument (e.g., claim, evidence, reasoning) are difficult to identify. In consideration of these issues, the video recorded argumentation sessions were scored directly using an observation protocol, Assessment of Scientif ic Argumentation in the Classroom (ASAC; see Supporting Information).28 This instrument was designed to capture argumentation events in a holistic fashion allowing for a comprehensive assessment of the quality of an argumentation event. The ASAC is divided into three sections: conceptual and cognitive, epistemic, and social. These sections are based on the integrated domains that Duschl14 describes as essential for generating and evaluating arguments in educational contexts: E



Article

Figure 3. ASAC score distribution for the 12 videos by item.

seen in ASAC categories Q2, Q3, Q5, and Q6. Initial coding divided responses into coherent or incoherent with respect to the concept of claims as defined in scientific argumentation. The incoherent responses were reviewed to try to understand student thinking about claims and codes that were emergent from the data. Analysis was conducted using the constant comparison method of qualitative coding, and the emergent codes were clarified during the coding process.30 The resultant codes included claim change mediated by manipulated data, hypothesis, error, and new data only. The coding relating to claims did not vary with whether students were near the beginning of the general chemistry laboratory course sequence or near the end.

Videos were scored by two undergraduate researchers following training in the instrument by the primary author. The training consisted of simultaneous scoring of videos followed by discussion and resolution of differences in scores. Once both undergraduates were scoring videos consistently, as indicated by Cohen’s Kappa of 0.92, they scored the complete set of videos used in this research. Table 2 presents the total number of times an ASAC item received a score of Never, Sometimes, or Often over the two-semester sequence. Although the number of videos was only 3 per argumentation session, each recording involved 2−3 rotations of traveling students that were scored as one event so 6−9 students were involved in the Discussion. The limited data set precludes statistical analysis; instead, the ASAC scores directed the examination of the data for exemplars of student discourse that provided insight into gaps in argument construction.

■

RESULTS AND DISCUSSION The total number of times each ASAC item was coded Never, Somewhat, or Often in the 12 videos is presented in Table 2. This data represents four different investigations over the twosemester sequence of general chemistry laboratory. The two areas that stand out in the data as lacking in the argumentation sessions are the following: (Q3) The participants modified their explanation or claim when they noticed an inconsistency or discovered anomalous information. (Q10) The participants used scientific theories, laws, or models to support and challenge ideas or to help make sense of the phenomenon under investigation. Figure 3 illustrates the variation in coding for each item in the 12 videos. Items that are yellow or blue indicate that these items are consistently observed during the student argumentation, an indication that the scaffolding provided in the ADI model is successfully directing students to participate in these cognitive, epistemic, and social practices. For example, “(Q1)

Semistructured Interviews

All students enrolled in the courses were invited to be interviewed about their perceptions of the use of claims, evidence, and reasoning in scientific argumentation.29 The purpose of the interviews was to understand student conceptions about what is meant by a claim, evidence, and reasoning in sense-making in science for both scientists and themselves as science learners. For instance, with regard to claim change, students were asked, “How do you determine if a claim is valid or acceptable?” and “Is it okay to modify claims? Why or why not?” While the videotapes can indicate how students are performing scientific argumentation, the interviews provide voice for how students perceive argumentation. Semistructured interviews averaging 10−15 min were conducted with 12 participants at varied stages in the laboratory sequence with 3 participants agreeing to a second interview later in the sequence. Interviews were transcribed and qualitatively coded on the basis of the concept of claim as F



Article

use of scientific principles, i.e., density is an intensive property and mass is an extensive property, is an issue as well. This language is never used in the argumentation session. The score for Q3 on this video was a 0 as was the score for Q10. In the electrochemistry experiment, the students use a known half-cell of Cu|Cu2+ for identifying three unknown metals from cell potentials. The students are given a table of the standard reduction potentials of 17 metals. The students must make a claim as to the identity of their three unknown metals and then determine which combination would make the best battery. The presenter identified the three metals on the basis of the experimentally determined reduction potentials as Zn, Al, and Cd. Enrico feels that the reduction potential value of 0.567 is closer to the value for zinc than to that for cadmium. Jasmine: This is our potential and this is the known, copper. Enrico: This [Cadmium] should not be here. I do not feel like this is accurate. See, when you look at the chart, it is closer to the zinc, not the Cd. Jasmine: So, the [unknown metal reduction] potential is point six, so you look [on the chart] for negative point six. So, the closest is, oh... You can round this up to 0.8 [reduction potential of Zn2+] and 0.4 [reduction potential of Cd2+], so this would be like... maybe we should have picked a different one, but we did not (GC2.5: Electrochemistry). If the group measured −0.567 for the reduction potential and Cd2+ is −0.40 and Zn2+ is −0.76, then the difference is 0.17 to Cd2+ and 0.19 to Zn2+, values which with regards to error are essentially the same. So, while a claim change may not have been warranted, the presenter should have backed up their claim with more than, “that is what we put”, responding with evidence and rationale, for the original claim. The discussion never circled around to the meaning of the numbers in the “chart” of reduction potentials, which offered the rationale needed to support the original claim or the claim change. It is possible that the lack of claim change or justification of evidence in these cases stemmed simply from a desire to finish the lab quickly and revising a claim would take more effort. Alternatively, the hesitancy observed by students to change their claims may be rooted in misconceptions about when and how claims can change. The score for Q3 was 1, but for Q10 it was a 0. In the interviews students were asked, “Is it okay to modify claims? Why or why not?” Of the 12 students interviewed, only one cited that an individual could change their claim on the basis of a new interpretation of data. Two students stated that one could not change a claim because a claim is part of the results. For example, Tai said “I would say not to, not really, for the fact that they have to go with the results they got from the experiment.” He conflated claim change with manipulating data. Similarly, two students conflated claims and hypotheses. Student Ricky replied “If you change your claim while you’re doing it [the experiment], that’s not accurate” because it would be changing the structure of the perceived experimental process. One student, Cassie, allowed for a claim change only if an experimental error was made and said a claim can be modified “if you find that you made a mistake and you redo; you redo your data.” Most commonly, six students stated a belief that claim changes could be made in light of “further data”, “more research”, or “notice[ing] something that you missed”. While it is certainly true that claims can change as new information is acquired, the perceived requirement of new data to initiate claim change minimizes the potential for claim

The talk of the group was focused on solving a problem or advancing understanding”, is always observed. The red part of the bar indicates that the item was never observed; for items Q3 and Q10 the bar is half red, indicating that these aspects of argumentation were rarely observed in the 12 videos. These aspects of argumentation can be simplified to lack of claim change and lack of concept-based justification. These scores indicated aspects of the argumentation that we could investigate for exemplars of where students were given the opportunity to change a claim or use concept-based justification with the framework provided by the ADI instructional model. Lack of Claim Change

The structure of the Argument-Driven Inquiry models affords students with feedback on their ideas when they create a poster with their group members and participate in the argumentation session. Ideally students should determine their claims on the basis of evidence from within the group and then justify or modify their claims on the basis of argument critique from other groups and supporting or contradictory evidence. The full description for the ASAC item is provided below. Q3 Description: Inconsistencies between claims or explanation and the phenomenon under investigation are common. A group that modified their claim or explanation when they noticed inconsistencies or anomalies would not ignore “things that do not fit” or attempt to discount them once they are noticed by one of the participants. Groups that score high on this item try to modify their claim or explanation (not just their reasons) in order to account for an inconsistency or an anomaly rather than attempting to “explain them away”. Indeed, this mirrors the process of science as practiced by scientists and promoted by standards.1 However, as indicated by the relatively few examples of claim change (Q3), there is a tendency for students to maintain their original incorrect or insufficient claims despite contradiction or critique. For example, Adam, the presenter, and Brianna, a traveler, discussed the claim by Adam’s group that the three blocks were the same material, but the mass and density varied. His evidence was three density values 12.9, 13.4, and 7.6 g/mL. Brianna: So, they are the same object but different materials? Adam: No, I said they are the same object but different density and different mass. Brianna: Would not that make them different then, if they are not the same density? Adam: No, I do not think so. Let us say you have two pieces of metal, Al and Fe. Al is of course lighter because it is less dense. Brianna: So, what is your claim? Adam: The blocks are the same material, but the mass... wait I do not know why I said mass. It was the density that varied with the method. Brianna: They are not the same material; the density is different. Adam: No, think about it. The mass was varied. (GC1.1: Physical Properties) The density values suggest that two of the objects are the same material and that one is an object made of a different material. The opportunity for claim change was presented multiple times by Brianna, but Adam resisted changing. His voice in the audio is confident, although he does falter a bit toward the end of the exchange. The lack of understanding or G



Article

chemistry content, in many cases, the claim and evidence provided are sound, but the reasoning based on chemical principles is missing and must be inferred by the listener. Sandoval and Millwood31 found that, in written argumentation, where students are writing laboratory reports with the instructor as audience, students approached data as self-evident providing little to no interpretation. It appears that the notion of self-evident data is a stumbling block for students in argumentation sessions when communicating ideas to their peers as well. Although similar research on argumentation found that students could engage in high-quality argumentation even when the science content was unfamiliar, this prior research did not drill into the individual aspects of argumentation to determine the incidence of claim change that was warranted but did not occur.18

change in the midst of an argumentation session when students are sharing approaches for and critiques of data analysis and argument structure. It is possible that more students would have allowed for new interpretations to guide claim change if questioned about a specific example rather than a general statement. Regardless, the majority of students’ first inclination was to state that claim change either cannot happen or can only happen to accommodate additional experimental data. As suggested by other researchers the students simply do not have enough experience with this aspect of argumentation from which to negotiate claim change.17 Lack of Concept-Based Justification

As indicated by the ASAC scores for Q10, students also struggled with using scientific theories, laws, or models to support their arguments. The full description for the ASAC item is provided below. Q10 Description: Science is theory-laden. In other words, scientists rely on broad, well-supported organizing ideas to frame their arguments and claims. Students should also employ these paradigmatic ideas in providing warrants for the evidence and claims they make or use to refute others’ claims. Explicit reference to these “big ideas” will result in a higher score on this item. For some students, this may be because they were still grappling with the scientific concepts and, thus, focused on surface features instead of scientific principles. For instance, in a density laboratory Tysha said, “We had different numbers because the cubes were different. You had a white cube and we had a red cube” (GC1.1: Physical Properties). She focused on the distracter surface feature of cube color rather than using the rationale of density as an intensive physical property. In these instances, the lack of concept-based justification would not necessarily be viewed as a weakness in argumentation skills, rather that students are still working on understanding key chemical concepts required to interpret the data. The score Q10 was a 1 for this video. Students often supported their claims with evidence but did not provide sufficient reasoning as to why the evidence supported the claim. For example, during a lab on intermolecular forces which includes the alcohol series methanol, ethanol, n-propanol, and n-butanol, Abby said, “So, the ones with a greater molecular weight, if they are harder to pull away from each other, then it means that the bigger ones have stronger molecular forces” (GC2.1: Intermolecular Forces). Abby directly linked increased molecular weight to increased intermolecular forces without any discussion of van der Waals or London forces. She answered “what”, namely, in this case, that the molecule with higher molecular weight had greater intermolecular forces, without using chemical theory to answer “why” this is the case. Similarly, in an experiment, which challenges students to justify the best means of silver recovery as either a redox or a precipitation reaction, Ashley said: “Anybody could have defined “best” [in the guiding question] anyway they wanted to. They used yield so they are going to use the best yield. But we used what was the most effective” (GC1.5: Chemical Reactions). This investigation does invoke multiple claims that can be rationalized on the basis of yield or recovery of silver metal vs silver chloride. Ashley states what they did, but she never explains why her group chose “most effective” on the basis of type of product or the concept of theoretical yield. While these omissions of concept-based justifications may be due in part to still emerging proficiency in

■

LIMITATIONS The primary limitation of this study is the small sample size, only 3 videos per investigation. However, each video recorded multiple conversations as students rotated between poster presentations. In addition, the students in the GC1 videos are not exactly the same as the students in the GC2 videos, although there was a 60% overlap in the students videoed for the second-semester videos.

■

IMPLICATIONS

During the development of an initial argument, the students would discuss their data and arrive at a claim. Once they agreed on a claim, students rarely considered alternatives. Even in follow-up discussion after the argumentation, where claim assessment was expected, claim change was consistently minimal. The limited data set in this project precludes statistical analysis; however, the trend for lack of claim change and lack of concept-based justif ication led us to look into the recordings for examples of opportunity and to conduct student interviews to tease out student understanding of these two aspects of scientific argumentation. In each investigation we found examples of student reluctance to change an initial claim as well as weak justification of the evidence. Other researchers have noted similar behaviors, but those studies are most often based on the discourse analysis of a single group in a single activity. In addition, the majority of research on argumentation has been conducted at the middle and high school level. These failures of students to engage in key cognitive and epistemic practices with a year-long laboratory experience with significant scaffolding, support, and exposure embedded in the curriculum were troubling. One might ask if the issue is instructional or if there something in student thinking that is not fully understood. This led us to look first at the argumentation sessions to ensure that there were actual instances where claim change was warranted and to further investigate the justifications provided. As expected, there were examples of warranted claim change not occurring even when strong arguments were provided. The videos also revealed that weak or flawed justifications for claims occurred even in the last investigation. Student interviews provided some insight with regard to claim change in that students stated a belief that claim changes must be based on new or alternative evidence, rather than rethinking of the evidence based on better-quality reasoning that should evolve from argumentation. The issue of poor justification is almost certainly linked to the claim change issue which is why researchers may see an improvement in the H



Article

ORCID

epistemic practice over time, but without drilling down into specific aspects of the student arguments, it might not be apparent that justification was weak. For example, recent publications with middle school students found that overall the ability to engage in high-quality argumentation increased pre/ post regardless of the student content knowledge.18 Claim change and using science concepts for justification are two areas where students may need additional guidance and instruction. Research on the nature of science has indicated that implicit instruction on the nature of science as learned through inquiry is not enough.32,33 Instructors may need to explicitly state reasons why a student or group may need to change their claim. Students may need direct instruction when a claim change is merited. During laboratory instructors can reinforce that claim change is a sign of success that students are thinking deeply about the experiment rather than viewing claim change as a sign of failure in their prior work. Instructors can also guide student learning by specifically prompting students to answer “why”, making their internal reasoning explicit. Stanford et al. in their work on argumentation using POGIL in physical chemistry classroom noted that students “regularly make claims without any support or provide claims and data without explanation as to how their data supports their claim (p 1508).” This is consistent with the work presented here and begs the question that perhaps leading with the “claim” supported with evidence and justified with science theory is problematic (P. Enderle, personal communication, March 2018). The NGSS in listing the 8 practices for science calls for “arguing from evidence”.2 Revising established argumentation frameworks is a daunting task, but the role of the instructor may more fruitfully be to encourage student arguments from the position of justification of evidence that leads to a claim. In other words, identify the scientific theory that supports the evidence and then develop a claim on the basis of the evidence.

Joi Phelps Walker: 0000-0001-7783-4706 Notes

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS The authors would like to thank the instructors and students for allowing the video recording to take place and undergraduate researcher Amany Ghanem who aided with transcriptions. We would also like to thank Dr. Patrick Enderle, Georgia State University, for his singular notion that perhaps the claim−evidence−reasoning sequence is a contributing factor in the challenge students encounter with claim change.

■

(1) National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas; The National Academies Press: Washington, DC, 2012. (2) NGSS Lead States. Next Generation Science Standards: For States, By States; The National Academies Press: Washington, DC, 2013. (3) Toulmin, S. The Uses of Argument; Cambridge University Press: Cambridge, 1958. (4) Sampson, V.; Clark, D. Assessment of Argument in Science Education: A Critical Review of the Literature. In Proceedings of the Seventh International Conference of the Learning SciencesMaking a Difference; Barab, S. A., Hickey, D. T., Eds.; Erlbaum: Mahwah, NJ, 2006; pp 655−661. (5) Zembal-Saul, C.; McNeill, K. L.; Hershberger, K. What’s Your Evidence? Engaging K-5 Students in Constructing Explanations in Science; Pearson Education: Boston, 2012. (6) Walker, J. P.; Sampson, V. Learning to Argue and Arguing to Learn: Argument-Driven Inquiry as a Way to Help Undergraduate Chemistry Students Learn How to Construct Arguments and Engage in Argumentation during a Laboratory Course. J. Res. Sci. Teach. 2013, 50 (5), 561−596. (7) Zeidler, D. L. The Central Role of Fallacious Thinking in Science Education. Sci. Educ. 1997, 81 (4), 483−496. (8) Klahr, D.; Dunbar, K.; Fay, A. L. Designing Good Experiments to Test Bad Hypotheses. In Computational Models of Scientific Discovery and Theory Formation; Shrager, J., Langley, P., Eds.; Morgan Kaufman: San Mateo, CA, 1990; pp 355−401. (9) Driver, R.; Newton, P.; Osborne, J. Establishing the Norms of Scientific Argumentation in Classrooms. Sci. Educ. 2000, 84 (3), 287− 313. (10) Cooper, M. Cooperative Chemistry Laboratories. J. Chem. Educ. 1994, 71 (4), 307−311. (11) Poock, J.; Burke, K.; Greenbowe, T.; Hand, B. Using the Science Writing Heuristic in the General Chemistry Laboratory to Improve Students Academic Performance. J. Chem. Educ. 2007, 84 (8), 1371−1378. (12) Schroeder, J. D.; Greenbowe, T. J. Implementing POGIL in the Lecture and the Science Writing Heuristic in the Laboratory Student Perceptions and Performance in Undergraduate Organic Chemistry. Chem. Educ. Res. Pract. 2008, 9 (2), 149−156. (13) Duschl, R. A.; Osborne, J. Supporting and Promoting Argumentation Discourse in Science Education. Stud. Sci. Educ. 2002, 38, 39−72. (14) Duschl, R. Science Education in Three-Part Harmony: Balancing Conceptual, Epistemic, and Social Learning Goals. Rev. Res. Educ. 2008, 32, 268−291. (15) Berland, L. K.; Reiser, B. Making Sense of Argumentation and Explanation. Sci. Educ. 2009, 93 (1), 26−55. (16) Kuhn, D. Science as Argument: Implications for Teaching and Learning Scientific Thinking. Sci. Educ. 1993, 77 (3), 319−337. (17) Novak, A. M.; Treagust, D. F. Adjusting Claims as New Evidence Emerges: Do Students Incorporate New Evidence into

■

FUTURE STUDIES The work on argumentation in the laboratory setting is continuing, videoing students over the two-semester sequence at a primarily undergraduate university in the southeastern United States. The current research will look at student engagement in scientific argumentation across the twosemester major sequences for the STEM disciplines of biology, chemistry, and physics as well as a comparison between disciplines. This work includes development of an observation protocol to inform instructors and researchers of opportunities to facilitate student engagement in the important scientific practice of argumentation including claim change and using theory as reasoning for evidence.

■

ASSOCIATED CONTENT

S Supporting Information *

The Supporting Information is available on the ACS Publications website at DOI: 10.1021/acs.jchemed.8b00745.

■

REFERENCES

Observation protocol used to score videos, Assessment Scientific Argumentation in the Classroom (ASAC) (PDF, DOCX)

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. I



Article

Their Scientific Explanations? J. Res. Sci. Teach. 2018, 55 (4), 526− 549. (18) Grooms, J.; Sampson, V.; Enderle, P. How Concept Familiarity and Experience with Scientific Argumentation Are Related to the Way Groups Participate in an Episode of Argumentation. J. Res. Sci. Teach. 2018, 55 (9), 1264−1286. (19) Walker, J. P.; Sampson, V.; Zimmerman, C. Argument-Driven Inquiry: An Introduction to a New Instructional Model for Use in Undergraduate Chemistry Labs. J. Chem. Educ. 2011, 88 (10), 1048− 1056. (20) Grooms, J.; Enderle, P.; Sampson, V. Coordinating Scientific Argumentation and the Next Generation Science Standards through Argument Driven Inquiry. Sci. Educ. 2015, 24 (1), 45. (21) Walker, J. P.; Sampson, V.; Grooms, J.; Zimmerman, C.; Anderson, B. Argument-Driven Inquiry in Undergraduate Chemistry Labs: The Impact on Students’ Conceptual Understanding, Argument Skills, and Attitudes toward Science. J. Coll. Sci. Teach. 2012, 41 (4), 74−81. (22) Walker, J. P.; Sampson, V.; Southerland, S.; Enderle, P. J. Using Laboratory to Engage All Students in Science Practices. Chem. Educ. Res. Pract. 2016, 17 (4), 1098−1113. (23) Sampson, V.; Grooms, J.; Walker, J. P. Argument-Driven Inquiry as a Way to Help Students Learn How to Participate in Scientific Argumentation and Craft Written Arguments: An Exploratory Study. Sci. Educ. 2011, 95 (2), 217−257. (24) Sampson, V.; Walker, J. P. Learning to Write in the Undergraduate Chemistry Laboratory: The Impact of ArgumentDriven Inquiry. Int. J. Sci. Educ. 2012, 34 (10), 1443−1485. (25) IBHEInstitutional Profile. http://ibheprofiles.ibhe.org/ profile.aspx?fice=001694 (accessed Jan 2019). (26) Sabella, M. S.; Coble, K.; Bowen, S. P. Using the Resources of the Student at the Urban, Comprehensive University to Develop an Effective Instructional Environment. AIP Conference Proceedings; Henderson, C., Sabella, M., Hsu, L., Eds; 2008 Physics Education Research Conference, Edmonton, Alberta (Canada), July 23−24, 2008; Vol. 1064 (1), pp 42−45. (27) Van Duzor, A. G. Using Self-Explanations in the Laboratory To Connect Theory and Practice: The Decision/Explanation/Observation/Inference Writing Method. J. Chem. Educ. 2016, 93 (10), 1725− 1730. (28) Sampson, V.; Enderle, P. J.; Walker, J. P. The Development and Validation of the Assessment of Scientific Argumentation in the Classroom (ASAC) Observation Protocol: A Tool for Evaluating How Students Participate in Scientific Argumentation. In Perspectives in Scientific Argumentation: Theory, Practice and Research; Kline, M., Ed.; Springer: New York, 2011; pp 235−264. (29) Handbook of Research Design in Mathematics and Science Education; Kelly, A. E., Lesh, R. A., Eds.; L. Erlbaum: Mahwah, N.J, 2000. (30) Lincoln, Y.; Guba, E. Naturalistic Inquiry; Sage Publications: Beverly Hills, CA, 1985. (31) Sandoval, W.; Millwood, K. The Quality of Students’ Use of Evidence in Written Scientific Explanations. Cogn. Instr. 2005, 23 (1), 23−55. (32) Lederman, N. G. Teachers’ Understanding of the Nature of Science and Classroom Practice: Factors That Facilitate or Impede the Relationship. J. Res. Sci. Teach. 1999, 36 (8), 916−929. (33) Lederman, N. G.; Abd-El-Khalick, F.; Bell, R. L.; Schwartz, R. S. Views of Nature of Science Questionnaire: Toward Valid and Meaningful Assessment of Learners’ Conceptions of Nature of Science. J. Res. Sci. Teach. 2002, 39 (6), 497−521.

J


Facilitating Argumentation in the Laboratory: The Challenges of Claim

Recommend Documents