Writing Instruction in Chemistry Classes: Developing Prompts and

Aug 17, 2018 - This 12-month qualitative study examined the efforts and classroom practices of a team of chemistry teachers in a high-needs secondary ...
0 downloads 0 Views 301KB Size
Download PDF
Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

pubs.acs.org/jchemeduc

Writing Instruction in Chemistry Classes: Developing Prompts and Rubrics Kayla Logan* and Lee Mountain*

J. Chem. Educ. Downloaded from pubs.acs.org by KAROLINSKA INST on 08/17/18. For personal use only.

Department of Curriculum and Instruction, University of Houston, Houston, Texas 77204-5029, United States ABSTRACT: This 12-month qualitative study examined the efforts and classroom practices of a team of chemistry teachers in a high-needs secondary (9−12) school as they worked toward meeting a campus mandate to incorporate student writing into their curriculum. The researcher, an English teacher and peer, investigated how this team of teachers negotiated these new curricular demands. Teachers have recognized that a first step in writing instruction was the development of quality writing prompts in addition to rubrics for evaluating their students’ writing about chemistry content. As the researcher collaborated with the chemistry teachers, she collected, coded, and analyzed data from transcriptions of team meetings, interviews, classroom observations, students’ papers, and a researcher journal. As these chemistry teachers integrated writing into their classrooms, the researcher’s ongoing reconstructive analysis of data provided a record of attitudes, behaviors, classroom practices, and development of prompts and rubrics by the chemistry teachers. The data regarding prompts revealed that, through collaboration, this team of chemistry teachers found ways to improve the wording of their prompts. Teachers reported that the improved prompts elicited written answers that better showed the students’ knowledge of chemistry. The data regarding rubrics revealed that creating a rubric and revising it through three iterations did help these chemistry teachers clarify and communicate their expectations for student writing, a first step in planning meaningful writing instruction. In this qualitative study, the collaboration between the English teacher/researcher and this team of chemistry teachers assisted in bringing about the incorporation of writing instruction into their chemistry classes. KEYWORDS: Chemical Education Research, High School/Introductory Chemistry, Interdisciplinary/Multidisciplinary, Communication/Writing FEATURE: Chemical Education Research

■

FOCUS In this qualitative study, a team of high school chemistry teachers and an English teacher worked together for a year developing ways to incorporate writing effectively into their chemistry classes. The English teacher served as researcher and also as campus trainer for the mandated writing initiative at the at-risk high school. She gave no top-down unsolicited directives about writing instruction to the chemistry teachers. Instead, she encouraged them to take the lead in determining how they and their students could best use the writing initiative to benefit chemistry education in their classrooms. She interviewed the chemistry teachers, observed their classes, attended their team meetings as a peer, read their students’ papers, and worked with them to meet the campus mandate and integrate writing into their chemistry classes. Because these chemistry teachers were primarily concerned with the use of writing to assess students’ learning of the objectives set forth in both the Texas Essential Knowledge and Skills and the district curriculum, the researcher aided them in developing prompts and a rubric for evaluating their students’ written responses to the prompts. The English teacher formulated research questions to investigate how they would go about developing prompts and rubrics and how this process might influence writing instruction.

years. Writing is a means of helping students to explore and clarify scientific concepts and to improve their retention of science content knowledge.1−3 Many studies explore specific techniques, activities, and philosophical approaches for incorporating writing and literacy practices into science classrooms.4−6 Scholarship about writing in science, and specifically about writing in chemistry, is rich in the topics associated with writing instruction, including the positive impacts of writing-to-learn strategies,7−10 the benefits of progressive writing,10,11 the writing improvements associated with peer review,10,12−14 and the benefits and challenges of interdisciplinary efforts.15−17 More recent scholarship has addressed to what degree the various standards for reasoning in science and ELA are aligned18 and how new technologies may be shaping the ways that students engage and learn through writing.19 Additional sources regarding quality assessment and how teachers should use assessment to drive instruction were foundations for this study.20,21 The rich store of scholarship about writing instruction in chemistry has developed primarily over the past 30 years, for although chemistry educators of previous decades asserted the importance of writing in chemistry coursework, the quandary regarding the degree to which writing instruction should be part of chemistry coursework remained unanswered.22−24 For example, in 1987, Meislich24 argued, “Chemistry instructors

■

LITERATURE BACKGROUND The impact of writing activities on student learning in chemistry has been well-documented, and writing in chemistry has mirrored predominant trends in literacy instruction over the © XXXX American Chemical Society and Division of Chemical Education, Inc.

Received: April 21, 2018 Revised: July 31, 2018

A

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

a process that involved interdisciplinary collaboration of both chemistry and linguistics educators as they discussed and reflected upon the written work of students.15 In addition to scholarship regarding writing in science, this study was also informed by research indicating that both students and teachers benefit from collaborations aimed at improving student learning.21,36−38 The benefit of professional collaboration for student learning is supported in research surrounding teacher education, professional development, and disciplinary literacy. The teachers in this study were required to engage in a Professional Learning Communities (PLC) model, which incorporated the ongoing use of assessment to drive instruction.21 These areas of research point to the idea that collaborative practices can be most effective when they are campus-based rather than top-down administrative directives, when they are embedded in teacher practice, and when they maintain a disciplinary focus rather than a generic literacy focus.39−42 These models are in keeping with a bottom-up rather than a top-down professional development approach and provide opportunities for developing interdisciplinary collaboration focused on improving students’ thinking and writing skills.

do not have to teach good writing but only to recognize and not accept poor writing. Teaching good writing is best left to teachers of writing.” Though writing was an undeniable requirement for expressing conceptual chemistry knowledge, it was the general belief that “science courses. . . should be full to the brim with content, and writing instruction would only. . . reduce the content,” as expressed by LaBianca and Reeves in 1985.25 It is possible that resistance to writing instruction in chemistry classrooms emerged in response to the call by many researchers and practitioners for content area literacy (CAL) and writing across the curriculum (WAC) initiatives. Although such interdisciplinary efforts can be effective,16 critics argued that implementing ELA writing styles in chemistry courses was not the best way to teach the subject. Many secondary chemistry teachers have been expected to use general literacy strategies (aligned with a CAL approach) in their classrooms to improve students’ abilities to read and write. Such mandates seemed to be efforts to respond to low scores on standardized tests, leaving non-ELA teachers frustrated and prompting teachers to “adapt, not adopt” general literacy strategies for use in non-ELA classrooms.26 An overview of the scholarship regarding writing in chemistry reveals that, toward the end of the millennium, an emphasis on disciplinary literacy, the literacy practices that are specific to various fields of study, yielded even deeper probing into how writing could help students understand chemical concepts as well as recognize the specific elements that made for “good” writing in chemistry. In 2001, Shibley, Milakofsky, and Nicotera stated, “Too often student scientific writing occurs only in courses taught by the English Department.”12 Around this time, many science educators embraced writing-to-learn (WTL) strategies, especially following the 1994 publication of Rivard’s meta analysis.9 WTL scholarship in chemistry coursework continues to expand as chemistry educators seamlessly integrate writing into lab reports and other assignments to the benefit of their students using the Science Writing Heuristic (SWH) and a plethora of guided activities that elicit metacognitive writing to improve students’ conceptual knowledge and retention.7,8,10,27−30 In addition to WTL approaches, chemical educators turned to exemplars (and nonexemplars) of chemistry writing as a teaching tool for their students. Bunting31 provided several nonexemplar sentences about chemistry and ameliorated the solecisms. He included both common errors such as “a/an” and “lay/lie” while also addressing the science-specific use of verb forms and the avoidance of personal pronouns. Subsequent authors developed lessons and activities that explicitly addressed scientific, or “technical”, writing styles that chemistry students should master, including technical voice, nominalization, and the use of recurring words and phrases as found in chemistry publications.31−35 A particularly influential resource for this study, by Stoller, Horn, Grabe, and Robinson, documented the “Write Like a Chemist” program and the larger collaboration surrounding it.15 In this collaborative effort to improve undergraduate writing, postsecondary chemistry educators and applied linguistics educators collaborated in a multiple-year effort to “form valid analytic and holistic assessment instruments to be used by chemistry faculty to assess the writing performance of chemistry majors”. The paper documented the process of collaboration, including the development of prompts and a rubric, the validation writing assessment instruments, and the positive consequences of the collaborative effort. Also of relevance to this study, Stoller et al. documented the efforts to “socialize” chemistry teachers to develop and score writing assessments for chemistry courses,

■

RESEARCH QUESTIONS

This qualitative ethnographic study addressed two questions regarding the formative writing assessment that precedes writing instruction in chemistry classes: (1) How do high school chemistry teachers collaborate to develop writing prompts for their chemistry students? (2) How do these chemistry teachers collaborate to develop and apply a rubric for assessing their students’ learning of chemistry content? These research questions have relevance beyond their local context because the “how do” data that are meticulously gathered, analyzed, and reported from one school can impact practice and drive forward change at another school. The background in the following section supports the premise that well-structured prompts and well-designed rubrics promote both chemistry and writing instruction in chemistry classrooms.

■

METHODS AND FRAMEWORKS For this qualitative study employing critical ethnography, the approval of the Institutional Review Board at the University of Houston, Texas, was sought and obtained. Chemistry teachers signed release statements for the use of data collected over the course of the year. The IRB approved the study after reviewing the research questions and interview protocol lead-off questions. Although teachers used student writing samples frequently as a basis for discussion and instructional planning, student writing samples are not a direct form of data used in this study because written parental consent for the body of students enrolled in chemistry classes would have been necessary and the researcher anticipated low response rate based on her previous interactions with these students. The focus of the study was not student writing but rather on processes and interactions of chemistry teachers as they engaged in addressing the campus initiative to incorporate writing into their classroom routines. The researcher compiled records of interactions with three participating chemistry teachers at the high-needs high school at which she worked. The common goal of both the researcher and the chemistry teachers, to improve student writing, created B

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 1. Participant Background Information Participant

Age

Gender

Chemistry Teacher 1

48

Female

Chemistry Teacher 2

36

Female

Chemistry Teacher 3

28

Male

Teaching Assignment • • • • •

Chemistry (General) Chemistry (Pre-AP) Chemistry (AP) Chemistry (General/ELLs) Integrated Chemistry and Physics (General/ELLs) • Chemistry (General) • Engineering Design and Problem Solving

Years of Teaching Experience

Language Background

Educational Background

13

Bilingual (Hindi and English)

12

Bilingual (Spanish and English)

B.S. in Biology M.S. in Chemistry M.Phil. in Chemistry B.S. in Biology (minor in Psychology) M.S. in Education,

Native English speaker

B.S. in University Studies, Science for Secondary Education (minors in English and Business)

5

Box 1. Sample of First Interview Protocol

an excellent opportunity for the researcher to access vital data first hand through direct and frequent communication with chemistry teachers. The qualitative design was appropriate for examining the close interactions necessary for success in collaborative reform agendas, which “should be based on a deep understanding of the settings and participants who will be involved in change”.39 The locale of this study was a large, high-needs high school in southeast Texas that was employing a campus-wide writing initiative. The student population was predominantly Hispanic (72%), at-risk (62%), and economically disadvantaged (70%). The three chemistry teacher participants were highly qualified teachers who prepared for multiple courses to be taught daily. The chemistry teachers’ names have been omitted; teachers are presented in order of seniority within the department (see Table 1). A key resource for methodology was Carspecken’s Critical Ethnography in Educational Research.43 Critical ethnographic methods foregrounded the experiences of chemistry teachers as they integrated writing into their classrooms. A primary record of science instruction and school environment was compiled. The primary record consisted of documentation of classroom observations, face-to-face discussions, and campus interactions that occurred prior to the collaboration with chemistry teachers beginning on February 26, 2016. It is important to note that these preliminary data were monological (only from the viewpoint of the researcher) and in the physical forms of both audio transcriptions and hand-written journals of the researcher’s reflections before, during, and after her interactions with campus science teachers and administrators. The primary record was analyzed and used, along with analysis of meeting transcription data from February 26, 2016 until April 27, 2016, to inform the researcher as the study progressed and aided in the construction of follow-up questions for the first interview protocol. The following sample from the first interview protocol offers an example of how Carspecken’s framework43 was used to probe the teachers’ predispositions and current practices regarding writing assessment (see Box 1). Reconstructive analysis of all data collected during 2016, both primary record and dialogical data (from collaborative meeting transcriptions and the first round of interviews), aided the researcher in constructing follow-up questions for the second interview protocol administered in January of 2017. The following sample from the second interview protocol offers an example of how Carspecken’s framework43 was used to probe the topic of using a collaboratively designed rubric to assess students’ writing about chemistry content (see Box 2). Final interviews were instrumental for honing and clarifying the chemistry teachers’ experiences and beliefs about writing with regard to the research questions posed.

From INTERVIEW PROTOCOL 1 Topic Domain: Teacher perception of writing assessment Covert Categories • Teacher’s current practice for assessing student writing • Teacher’s beliefs about providing feedback for student writing • Teacher’s perceptions of the connection between student writing and quality of instruction Lead-Off Question: Walk me through the process of developing a writing rubric for chemistry, or describe your experience discussing the student writing samples and chemistry rubric. Follow-Up Questions: How important is it to assign grades to student writing? How would you describe the quality of your students’ writing this year? Describe how you provide feedback to students about their writing. Would you share any experiences that you have had with receiving feedback on your own writing? To what degree do you feel that you adjust instruction based on student writing? The common goal of teachers to improve both student learning of chemistry and student writing provided opportunities for data collection through direct and frequent communication among the four teachers. Data were collected as chemistry teachers developed, implemented, and reflected upon their instructional practices and as the researcher observed the implementation of writing in their classes. Table 2 outlines the data types and collection dates. The procedure of this research included ongoing collection and analysis of data during the 12 months of the study. Fourteen meetings of chemistry teachers and the researcher were audiorecorded and transcribed. The goals of these meetings were to identify chemistry content objectives from district documents based on the Texas Essential Knowledge and Skills (TEKS)44 to identify desired outcomes for student learning, to develop writing prompts, to develop and apply a rubric for assessing student writing, and to address concerns about student writing proficiency in chemistry. During these meetings, teachers observed the campus-required planning protocols, which were based on the PLC model.21 Teachers planned lessons and units based on the district documents, reviewing assessment data from students (both student writing and quantitative data from campus and district assessments). The researcher performed data analysis of meeting transcriptions, observation notes, and journal C

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

analyzed more data, these codes aided the researcher in selecting “chunks” of data for deeper analysis that seemed likely to contain data that could elucidate the themes and eventually findings in response to the research questions posed. After analysis of the final interview transcriptions, the researcher reanalyzed the entire body of research paying close attention to the “chunks” of data selected at various points during the research procedure. She identified themes that were closely associated with the research questions and constructed findings that will be discussed later in this article. These findings were checked against the entire body of data using negative case analysis. That is, the researcher considered the findings and ensured that there were no instances of data in which the findings were not supported. Because the researcher was a teacher peer, rather than an outsider to the campus, an emic, or insider perspective, emerged. Nolen and Vander Putten noted such research is “valuable when conducted by skilled practitioners with established knowledge, working relationships, access, and credibility within schools”.45 The researcher’s close ties to the participants required her to incorporate trustworthiness checks into the methodology. For example, peer review of interview protocols to ensure that they avoided leading questions and included consistency checks were employed. Furthermore, to avoid bias and a priori coding, ongoing peer-debriefing and member-checking was performed on coded data; that is, the researcher periodically consulted the chemistry teacher participants regarding her new understandings of their processes during the writing intervention. Chemistry teachers were asked to member-check the researcher’s coding as well as the themes that emerged from meeting transcriptions and interview data.

Box 2. Sample of Second Interview Protocol From INTERVIEW PROTOCOL 2 Topic Domain: How do chemistry teachers assess writing via a rubric? Covert Categories • Teacher’s practices for assessing student writing • Teacher’s beliefs about providing feedback for student writing • Teacher’s perceptions of the connection between student writing and quality of instruction • Teacher’s ideas about the characteristics of good writing in chemistry Lead-Off Question: When you use the rubric to assess your students’ writing, how do you do it? Follow-Up Questions: How much time would you estimate that you spend assessing a single response? How much time would you estimate that you spend assessing writing each week? How consistent do you feel that you are when using a rubric to assess student work? Would you like to comment on how you feel about spending this amount of time? How would you describe the quality of your students’ writing this year? Do you provide any feedback for your students? If so, how? (written/verbal) What sort of things do you [write/say]?

■

Table 2. Data Collection Schedule Data Type Collaborative Meetings

Interviews Chemistry Teacher 1 Chemistry Teacher 2 Chemistry Teacher 3 Classroom Observations Researcher Reflections

FINDINGS The study found that collaboration was an enabling factor in the process of incorporating writing into secondary chemistry classrooms and contributed to new instances of chemistry teachers’ enacting writing instruction. The researcher found that a cyclical pattern emerged when the teachers committed to incorporating and analyzing student writing over the course of the year. Facilitated by the common planning time to enact professional learning community protocols required by campus administration, the teachers collaborated to develop prompts, develop rubrics, and engage in discussion of practices for writing instruction in chemistry classrooms.

Collection Dates Feb. 26, 2016 Mar. 11, 25, 2016 Apr. 8, 27, 2016 Sept. 2, 15, 22, 2016 Oct. 13, 27, 2016 Nov. 3, 17, 2016 Dec. 1, 2016 Jan. 5, 2017 Sept. 16, 2016/Jan. 17, 2017 June 3, 2016/Jan. 19, 2017 June 13, 2016/Jan. 18, 2017 Feb. 2016 − Jan. 2017 Feb. 2016 − Jan. 2017

Themes

As discussed in the Methods and Frameworks, the researcher read and reread the data sets to develop initial codes and identify “chunks” of text for deeper analysis. Predominant themes that emerged from the body of data included (1) Value/purpose for writing in chemistry classrooms (2) Students’ writing abilities/performances (3) Development of prompts (4) Development/application of rubric (5) Writing to assess student knowledge (6) Time for writing (7) Instruction of writing Themes that emerged consistently through the analysis process supported the following findings regarding the two research questions posed.

entries regularly using horizon analysis.43 The process involved inferencing at low levels, and over time, reviewing those inferences multiple times, inferencing at higher levels to reconstruct meaning through position-taking and consideration of the norms at work among the chemistry teachers and researcher. Analysis of data was recursive, meaning that data sets were read and reread in light of new data. The researcher performed initial readings of the most recently transcribed data, noting predominant issues or topics that had been previously addressed by chemistry teachers or that pertained to the research questions. These initial codes included but were not limited to time constraints, student attitudes, types of writing, personal experiences with writing, student writing abilities, teacher expectations, writing feedback, and state and district standards. As the researcher collected and

Cyclical Pattern

The “how do” information elicited from the research questions indicated that teachers’ collaboration to include writing in their practice occurred as a cyclical pattern as the teachers D

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

committed to incorporating and analyzing student writing over the course of the year. The teachers engaged in (1) Creation of a prompt about chemistry content (2) Administration of prompts and collection of student samples in response to the prompt (3) Analysis of student samples (4) Revision of the prompt (5) Revision of the rubric Teachers engaged in this cycle 4 times over the course of the study. The latter 2 iterations of the cycle, however, did not include the fifth stage in which the teachers modified the rubric. Furthermore, the second and fourth iterations of the cycle did not include a revision of the prompt. Despite these minor omissions of stages 4 and 5, the cyclical nature of the collaborative meetings helped teachers to better express their expectations to students and to target their instructional goals (TEKS) through developing prompts and rubrics. Teachers’ attention to this cycle revealed that they viewed the quality of the responses as reflective of the quality of their classroom instruction. The following sections, Development of Prompts and Development of Rubrics, are based on the two research questions posed. These sections elaborate on the qualitative findings that emerged through reconstructive analysis.

Collaborative development of writing prompts promoted further collaborations of these teachers and supported writing instruction for students. The prompt created on February 26 was revised after assessing writing samples from Chemistry Teacher 1’s Pre-AP class. The prompt was revised based on common omissions made by advanced students in hopes of obviating such omissions in subsequent student responses. For example, note the wording that was used (and later revised) by the chemistry teachers as they developed this prompt: Compare and contrast endothermic and exothermic processes. Provide examples of both. Discussion of the responses revealed that students were not explicit in comparing (students tended to imply the similarities when writing about the contrasts) and that students tended to merely “give back” the examples presented in class. Upon reviewing their prompt, the teachers realized that these expectations were not specifically emphasized in the original prompt. They revised their prompt to read: Compare AND contrast endothermic and exothermic processes. Provide original examples of both. In this case, informal discussion of student responses enabled teachers to improve the prompt and resulted in their realization that students needed explicit instruction on comparing, contrasting, and following the organization of their responses as provided for them in the prompt. These chemistry teachers wanted subsequent prompts to be even more specific, so they added more explicit requirements. They revised Prompt A into Prompt B to get their students to include more information in their responses, as shown in Box 3.

Development of Prompts

In response to the first research question, How do high school chemistry teachers collaborate to develop writing prompts for their chemistry students?, the researcher found that, in addition to the cyclical pattern described above, teachers targeted instructional goals and expectations for students with regard to both skills of writing and acquisition of chemistry content. Key themes associated with these findings were themes 3 and 5 (development of prompts and writing to assess student knowledge, respectively). However, themes 1, 2, and 7 (value/purpose for writing in chemistry classrooms, students’ writing abilities/performances, and instruction of writing) ran through the data sets that predominantly responded to the first research question. These chemistry teachers reported that they valued the collaborative development of prompts because it helped them to improve the wording of their prompts and sharpened their attention to the chemistry TEKS as outlined in district scope and sequence documents. In the interview on June 3, 2016, Chemistry Teacher 2 said, “Writing prompts made me focus on the objectives and on how to work a prompt to elicit the best response for my students.” As the study progressed, she said she greatly valued the help of the researcher with wording the prompts effectively. Likewise, in the interview on September 16, 2016, Chemistry Teacher 1 emphasized the need for additional time to plan meaningful prompts: “When we first come up with a prompt, we wonder, ‘Is it going to be a good prompt?’ Then we see during the year, ‘Is it working? Am I getting good results out of it?’” Teachers engaged in collaborative efforts to improve the prompts on the supposition that such improvement would likewise improve students’ performance on the prompts. The teachers recognized the instructional value of exemplars and began to engage minimally in writing instruction by displaying anonymous writing samples from other classes to provide students an opportunity to understand the expectations for their writing. On two occasions during the 2016−2017 school year, Chemistry Teacher 1 administered the writing prompt to her Pre-AP students prior to the other teachers’ administrations so that anonymous exemplars from Pre-AP students were used by Chemistry Teacher 2 with her on-level and remedial students.

Box 3. Comparison of Prompt A and Prompt B Prompt A: Which scientist do you think had the most significant impact on the discovery of subatomic particles? Explain your answer using detailed information from our readings and notes. Prompt B: Pick one scientist who contributed to the discovery of subatomic particles and explain the experimental design and conclusions that were used in the development of modern atomic theory. Your answer must include the detailed information from our readings and notes. The development of prompts provided an opportunity for chemistry teachers to target learning objectives that students needed to master based on the state standards, the Texas Essential Knowledge and Skills for Chemistry (TEKS).44 At the beginning of each planning meeting and for each prompt, chemistry teachers referred to the TEKS as prescribed in their required campus PLC21 protocol. Teachers used verbs from the TEKS as prescribed by the district curriculum and planning documents. Toward the end of the study, the teachers noted that some verbs from the TEKS elicited better written than others. For example, verbs such as compare/contrast, describe, and explain yielded prompts that were more helpful to their ultimate goal of student learning. During the promptwriting process, chemistry teachers began to scan district curricular documents for such verbs. They noted that verbs such as name and calculate were not appropriate for well-developed written responses. The teachers opted to assess those objectives in other ways while creating prompts to assess objectives with more applicable verbs. The teachers acknowledged that some units of chemistry were less readily assessed via traditional E

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

written responses. Chemistry Teacher 2 cited units that focus on the periodic table or on mathematic calculations, such as moles and stoichiometry. While reviewing student writing samples, the teachers referred back to their prompts and revised them to elicit the responses that would best assess chemistry learning. The development of prompts provided an opportunity for these teachers not only to engage in dialogue about the learning objectives (TEKS) that students needed to master but also to sharpen their own skills for producing the types of prompts that would help students show their mastery of the learning objectives.

to writing. When working collaboratively to assess student writing via their rubric, the teachers enjoyed sharing responses based on the quality (good or bad) of specific student responses. They commented on both improved and disappointing performances of specific students, especially those of the students who were in the English classes taught by the researcher. In this way, rubric development fostered cross-curricular instructional goals for individual students and specific student groups. The second draft of the rubric that was implemented to assess student writing after March 25, 2016 addressed points that teachers noted in their previous meetings, such as chemistry teachers’ expectations that students should “follow the organization suggested in the prompt”, “use academic vocabulary”, and “give original examples”. Discussions of student work around the second rubric caused teachers to examine how students managed the thought processes specified by the verbs used in both their prompts and their curricular objectives. The teachers felt empowered to modify their rubric to tailor it to their perceptions of good student writing. In the September 16, 2016 interview, Chemistry Teacher 1 described the process of rubric development: Well, we started quite similar to English rubrics but then we promised to make it more science based so we changed only things like okay, I need this explanation. I need fact-based [responses]. So, we kind of improvised our rubric a lot. The power to adjust the rubric inspired buy-in for the teachers to use the rubric and unify their assessments and instruction. In the interview on June 13, 2016, Chemistry Teacher 3 characterized the rubric as “constantly changing”, and on June 3, 2016, Chemistry Teacher 2 indicated that she liked the rubric designed by chemistry teachers more than the ELA-based rubric previously mandated by the campus because of its versatility “because it could be about anything. They could write about anything and you could use that rubric to look at it and see whether they were able to respond appropriately to it and answer the question completely.” This third rubric (the “final” iteration), developed in October of 2016, was regarded by teachers as the one they would use for the assessing and scoring of their students’ writing in chemistry classes for the duration of the study (see Table 3). When chemistry teachers sensed that the rubric was “complete”, the focus of collaborative discussion shifted from identifying areas of need to addressing those areas through limited writing instruction. Chemistry Teachers 1 and 2 began to project the rubric in their classrooms to express expectations to their students. They also engaged in the partnership of exchanging writing samples and using them as instructional tools (either as exemplars or as nonexemplars) to help their students. Chemistry teachers often regarded the quality of the responses as reflective of the quality of their classroom instruction. Engaging collaboratively in the cycle of critically assessing the prompts, the writing samples, and the rubric inspired discussions in which the teachers took closer looks at how their classroom practices impacted students’ abilities to provide written responses that could accurately assess chemistry learning. All three chemistry teachers acknowledged the many challenges to writing in their science classrooms, including providing time for students to write in class, providing feedback on student writing, and dealing with the disappointment of poor student responses. In the January 16, 2017 interview, Chemistry Teacher 3 reported, “I have developed more of an understanding of what they should be turning in to me. However, it can be semi-frustrating in that I feel that I don’t get back what they should be giving me.”

Development of Rubrics

In response to the second research question, How do these chemistry teachers collaborate to develop and apply a rubric for assessing their students’ learning of chemistry content?, the researcher found that in addition to the cyclical pattern described previously in the Findings section, teachers developed and applied their own rubric with confidence. Teachers felt empowered by creating their own rubric for assessing student writing rather than using one geared for general content literacy purposes. The development process engendered cross-curricular collaboration and highlighted instructional goals that could be addressed in both ELA and chemistry classrooms. Furthermore, teachers applied a rubric with resolve and purpose. They felt that the use of the rubric was worthwhile because it was efficient and facilitated feedback for students. Key themes associated with these findings were themes 4 and 5 (development/application of rubric and writing to assess student knowledge, respectively). However, themes 1, 2, 6, and 7 (value/purpose for writing in chemistry classrooms, students’ writing abilities/performances, time for writing, and instruction of writing) ran through the data sets that predominantly responded to the second research question. These teachers developed their own rubric, bottom-up, for evaluating their students’ writing in chemistry classes. They used student writing samples to engage in discussion of the rubric on four occasions. They created three iterations of the rubric based on the strengths and weaknesses that they noticed in student writing. They reported that the use of the rubric enabled them to score student work confidently and efficiently. These teachers felt that their scoring was consistent and that they were able to provide feedback quickly and accurately. The collaborative development of the rubric provided a starting point from which chemistry teachers could think deeply about literacy expectations in chemistry class. Chemistry teachers appreciated that the rubric helped them to give feedback that affirmed students in areas in which their writing was successful and provided constructive feedback on areas in which their writing detracted from their overall responses. When asked on June 13, 2016, to recount how the teachers developed the rubric, Chemistry Teacher 3 said: We sat down, we knocked out a prompt that we wanted. . . . Then we [collected samples]sat down and we just simply put them into [piles]. First we started with just two piles, just that this is the good, this is the bad. And then we discussed why. And in that discussing we developed the ideas that pulled into our rubric. Then we realized that we needed a third, middle category for a middle ground for kids who weren’t quite there but.” Like Chemistry Teacher 3, both Chemistry Teachers 1 and 2 reported placing value on feedback that revealed concerns for both students’ writing abilities and students’ emotional responses F

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 3. Third Iteration of the Chemistry Writing Rubric Comparing Response-Level Characterizations High-Level Response:

Mid-Level Response:

Low-Level Response:

The Student Response···

The Student Response···

The Student Response···

Addresses all parts of the prompt Contains accurate information Partially follows the organization suggested in the prompt Uses original language (not copied from notes, books, or other sources) Uses complete sentences

Omits one or more parts of the prompt Contains inaccurate information Does not follow the organization suggested in the prompt Uses language that has been copied from notes, books, or other sources Uses incomplete sentences or contains errors that interfere with the reader’s ability to understand the response Displays little effort to use academic language learned in class Gives examples that are incorrect or incomplete

Addresses all parts of the prompt Contains accurate and thorough information Follows the organization suggested in the prompt Uses original language (not copied from notes, books, or other sources) Uses complete sentences that are easy to read and understand Uses academic vocabulary appropriately Gives original, thoughtful, and/or apt examples (other than those given in class or from other sources)

Displays an attempt to use academic vocabulary appropriately, with some flaws Gives examples that are common or have been widely discussed in class

development context, could be beneficial for serving the goal of chemistry learning. At the beginning of the study, chemistry teachers engaged their students in writing per the campus mandate, but they did not engage their students in writing instruction. Prior to collaboration, chemistry teachers handed out a prompt based on a reading passage (usually from the textbook or article from a science magazine), told the students to write their answers, and rarely provided limited one-on-one feedback. Teachers reported that these classroom writing experiences were not fully aligned with learning goals but rather were used simply to meet campus expectations for writing. Teachers reported that they often felt disappointed and frustrated with the quality of the student writing that they received and the class time that was used to procure the evidence of writing. Toward the end of the intervention, however, chemistry teachers found value in the collaborative construction of a uniform prompt and began to engage students in writing instruction to serve the goal of chemistry learning. Teachers projected the rubric on a screen to engage students in understanding the purpose and expectations for writing in chemistry class. Two of the teachers went further by discussing the quality of writing responses with their students when presenting exemplar or nonexemplar student samples. Although copious evidence of the effectiveness of the Science Writing Heuristic (SWH) has been proven effective with both low- and high-achieving students,7,8,28−30 the chemistry teachers in the current study preferred to create prompts that elicited evidence of chemistry knowledge rather than prompts with the purpose of helping students to learn through the development of claims and support. The researcher, aware of the effectiveness of SWH, suggested this style of writing at the onset of the intervention in an e-mail on February 12, 2016, prior to the first collaborative meeting. In addition to a study of the SWH approach,30 the researcher e-mailed two other articles3,15 about effective approaches to writing in science classrooms. The chemistry teachers rejected these formats, choosing instead to craft their own prompts based on the learning objectives outlined in in the TEKS. The researcher noted in her journal that in the meeting following the e-mail, teachers were much more concerned with using writing as a means to assess students’ readiness for the multiple-choice chemistry assessments designed by the district. From this emerged a point for further exploration: these chemistry teachers’ choice to place their instructional emphasis for writing on assessment of knowledge acquired rather than on the development of critical thinking and argumentation skills. Unpacking these

The chemistry teachers’ formulation of prompts and a rubric to address their specific expectations for student writing were aimed at improving instruction in chemistry. Yet closer examination suggested that the rubric designed by the chemistry teachers did not illicit forms of writing that were overtly suited to the particular discipline of chemistry. For example, at the onset of the study, teachers opted for short answer response writing that targeted student learning as specified in the Texas Essential Knowledge and Skills rather than lab reports, procedural writing, or written forms more aligned with write-to-learn strategies. This could have been in part due to the previous expectation that all campus teachers use a short answer response based on reading passages that was aligned with the state’s ELA exit-level assessment. Nonetheless, chemistry teachers felt that the opportunity to create a rubric that they deemed useful for writing in chemistry was respectful of their disciplinary goals for writing. Because the rubric was viewed as somewhat in f lux, teachers could agree to alter it or to create other rubrics to assess other forms of writing (such as lab reports). The rubric was not only for assessment but became a tool for instructing students about the elements to consider when crafting their responses. Through the development of prompts and a rubric, chemistry teachers created assessments that could be used as either formative or summative assessments of student learning in chemistry.

■

LIMITATIONS This study was limited by several factors. Specifically, the single locale and the subjectivity of the researcher limited the study in terms of reproducibility and trustworthiness. Because the researcher was a member of the faculty, and because she had more years of service on the campus than any of the chemistry teachers, it is possible that the verbalizations of chemistry teachers, collected through meetings and interviews, failed to reflect chemistry teachers’ true feelings about student writing and campus writing initiatives. Efforts to ameliorate the limitations were made by peer reviewing of interview protocols and reconstructive analysis and by involving the chemistry teacher participants in member checking data, including the narratives and findings presented. The findings presented in this study are not generalizable; however, there are several elements of the study that may reflect attitudes and practices of chemistry teachers and other high-needs secondary schools.

■

IMPLICATIONS The relevance of this research for practitioners is that writing instruction, couched in a bottom-up, collaborative professional G

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(3) Rivard, L. P.; Straw, S. B. The Effect of Talk and Writing on Learning Science: An Exploratory Study. Sci. Educ. 2000, 84 (5), 566−593. (4) Creech, J.; Hale, G. Literacy in Science: A Natural Fit. Science Teacher 2006, 73 (2), 22−27. (5) Draper, R. J.; Adair, M. (Re)imagining Literacies for Science Classrooms. In Reimagining content-area literacy instruction; Draper, R. J., Ed.; Teachers College Press: New York, 2010; pp 127−143. (6) Grant, M. C.; Fisher, D. Reading and Writing in Science: Tools to Develop Disciplinary Literacy; Corwin: Thousand Oaks, 2010. (7) Burke, K. A.; Greenbowe, T. J.; Hand, B. M. Implementing the Science Writing Heuristic in the Chemistry Laboratory. J. Chem. Educ. 2006, 83 (7), 1032−1038. (8) Greenbowe, T. J.; Poock, J. R.; Burke, K. A.; Hand, B. M. Using the Science Writing Heuristic in the General Chemistry Laboratory to Improve Students’ Academic Performance. J. Chem. Educ. 2007, 84 (8), 1371−1379. (9) Rivard, L. P. A Review of Writing to Learn in Science: Implications for Practice and Research. J. Res. Sci. Teach. 1994, 31 (9), 969−983. (10) Shultz, G. V.; Gere, A. R. Writing to Learn the Nature of Science in the Context of the Lewis Dot Structure Model. J. Chem. Educ. 2015, 92 (8), 1325−1329. (11) Van Bramer, S. E.; Bastin, L. D. Using a Progressive Paper to Develop Students’ Writing Skills. J. Chem. Educ. 2013, 90 (6), 745− 750. (12) Shibley, I. A.; Milakofsky, L. M.; Nicotera, C. L. Incorporating a Substantial Writing Assignment into Organic Chemistry: Library Research, Peer Review, and Assessment. J. Chem. Educ. 2001, 78 (1), 50−53. (13) Vazquez, A. V.; McLoughlin, K.; Sabbagh, M.; Runkle, A. C.; Simon, J.; Coppola, B. P.; Pazicni, S. Writing-to-Teach: A New Pedagogical Approach to Elicit Explanative Writing from Undergraduate Chemistry Students. J. Chem. Educ. 2012, 89 (8), 1025− 1031. (14) Walker, J. P.; Sampson, V. Argument-driven Inquiry: Using the Laboratory to Improve Undergraduates’ Science Writing Skills Through Meaningful Science Writing, Peer-review, and Revision. J. Chem. Educ. 2013, 90, 1269−1274. (15) Stoller, F. L.; Horn, B.; Grabe, W.; Robinson, M. S. Creating and Validating Assessment Instruments for a Discipline-specific Writing Course: An Interdisciplinary Approach. Journal of Applied Linguistic 2005, 2 (1), 75−104. (16) Stout, R. P. ‘Hello, I’m Carbon’: Writing about Elements and Compounds. J. Chem. Educ. 2010, 87 (11), 1163−1165. (17) Waratuke, S.; Kling, T. Interdisciplinary Research in a Dense Summer Bridge: The Role of a Writing Intensive Chemistry Seminar. J. Chem. Educ. 2016, 93, 1391−1396. (18) Lee, O. Common Core State Standards for ELA/Literacy and Next Generation Science Standards: Convergences and Discrepancies Using Argument as an Example. Educational Researcher. 2017, 46 (2), 90−102. (19) Prain, V.; Hand, B. Coming to Know More Through and From Writing. Educational Researcher 2016, 45 (7), 430−434. (20) National Research Council. Knowing What Students Know: The Science and Design of Educational Assessment; Pellegrino, J. W., Chudowsky, N., Glaser, R., Eds.; National Academy of Sciences, 2001. (21) DuFour, R.; Eaker, R. Professional Learning Communities at Work: Best Practices for Enhancing Student Achievement; ASCD: Alexandria, 1998. (22) Atkinson, G. F. Writing Among Other Skills. J. Chem. Educ. 1986, 63 (4), 337−338. (23) Lagowski, J. J. Writing, Thinking, and Learning. J. Chem. Educ. 1984, 61 (10), 841. (24) Meislieh, E. K. Requiring Good Writing in Chemistry Courses. J. Chem. Educ. 1987, 64 (6), 505−506. (25) Labianca, D. A.; Reeves, W. J. Writing Across the Curriculum: The Science Segment: A Heretical Perspective. J. Chem. Educ. 1985, 62 (5), 400−402.

chemistry teachers’ emphasis on content knowledge may reveal areas of need or reorganization of district assessments, state curriculum, or even the frequency with which chemistry teachers are able to conduct laboratory experiments with students in low-income schools. In this case, collaborative meetings yielded better writing instruction in chemistry classrooms as the teachers learned about the writing expectations that they had for their students through the crafting of prompts and the analyzing of student responses. They learned that their students, in many cases, were employing writing styles more suited to persuasive writing for their English classes (such as active voice, personal pronouns, rhetorical questions, or hooks for getting the reader’s attention). Such elements of student writing did not coincide with the expectations of chemistry teachers (such as passive voice, avoiding personal pronouns, organizing ideas concisely, following the specifications designated in complex prompts). Once noted, these nuances of writing in the different disciplines could be explicitly addressed with students in their English classrooms as well as in their chemistry classrooms. Other practitioners in other chemistry classrooms may want to give their students writing instruction in these chemistry-specific areas. Administrators could facilitate collaboration of ELA and chemistry teachers by scheduling common planning periods and providing extended time for teachers to develop both quality assessment materials and targeted instruction for their students based on their performance on written tasks. This case study illustrated how to effectively create writing prompts and grading rubrics for chemistry curriculum and suggested that professional development could be beneficial for training both ELA and chemistry teachers to engage in such endeavors. The development of prompts elucidated the teachers’ instructional foci and contributed to targeted instruction. The PLC model21 contributed to the effectiveness of these endeavors as meetings provided regular opportunities for teachers to assess and revise their prompts and rubric, using them formatively to guide instruction. The cross-curricular collaboration among ELA and chemistry teachers also proved fruitful in this case, offering insight about the kinds of writing instruction that students were receiving and the convergences and divergences in the writing styles expected in the two disciplines. Most of all, this study documented the ways that chemistry teachers collaborated to meet the demands of a campus writing initiative while serving the goal of chemistry learning.

■

AUTHOR INFORMATION

Corresponding Authors

*E-mail: [email protected]. *E-mail: [email protected]. ORCID

Kayla Logan: 0000-0003-1802-1435 Notes

The authors declare no competing financial interest.

■

REFERENCES

(1) Kamberelis, G.; Gillis, V. R.; Leonard, J. Disciplinary Literacy, English Learners, and STEM Education. Action in Teacher Education 2014, 36, 187−191. (2) Reynolds, J. A.; Thaiss, C.; Katkin, W.; Thompson, R. J. Writingto-Learn in Undergraduate Science Education: A Community-based, Conceptually-driven Approach. CBE-Life Science Education 2012, 11, 17−25. H

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

(26) Gillis, V. Disciplinary Literacy: Adapt not Adopt. Journal of Adolescent & Adult Literacy 2014, 57 (8), 614−623. (27) Sherwood, D. W.; Kovac, J. Writing in Chemistry: An Effective Learning Tool. J. Chem. Educ. 1999, 76 (10), 1399−1403. (28) Grimberg, B. I.; Hand, B. Cognitive pathways: Analysis of students’ written texts for science understanding. International Journal of Science Education 2009, 31 (4), 503−521. (29) Keys, C. W.; Hand, B.; Prain, V.; Collins, S. Using the science writing heuristic as a tool for learning from laboratory investigations in secondary science. J. Res. Sci. Teach. 1999, 36 (10), 1065−1084. (30) Akkus, R.; Gunel, M.; Hand, B. Comparing an inquiry-based approach known as the science writing heuristic to traditional science teaching practices: Are there differences? International Journal of Science Education. 2007, 29 (14), 1745−1765. (31) Bunting, R. K. Precise Writing for a Precise Science. J. Chem. Educ. 1999, 76 (10), 1407−1408. (32) Carr, J. M. Using a Collaborative Critiquing Technique to Develop Chemistry Students’ Technical Writing Skills. J. Chem. Educ. 2013, 90, 751−754. (33) Hudson, R. The Struggle with Voice in Scientific Writing. J. Chem. Educ. 2013, 90, 1580−1580. (34) Robinson, M. S.; Stoller, F. L.; Horn, B.; Grabe, W. Teaching and Applying Chemistry-specific Writing Skills Using a Simple, Adaptable Exercise. J. Chem. Educ. 2009, 86 (1), 45−49. (35) Robinson, M. S.; Stoller, F. L.; Jones, J. K. Using the ACS Journals Search to Validate Assumptions about Writing in Chemistry and Improve Chemistry Writing Instruction. J. Chem. Educ. 2008, 85 (5), 650−654. (36) Cantrell, S. C.; Burns, L. D.; Callaway, P. Middle- and Highschool Content Area Teachers’ Perceptions about Literacy Teaching and Learning. Literacy Research and Instruction 2008, 48 (1), 76−94. (37) Cantrell, S. C.; Hughes, H. K. Teacher Efficacy and Content Literacy Implementation: An Exploration of the Effects of Extended Professional Development with Coaching. Journal of Literacy Research 2008, 40, 95−127. (38) Hughes, M. T.; Parker-Katz, M.; Balasubramanian, A. Learning to Teach Literacy Through Collaborative Discussions of Student Work. Teachers and Teaching: Theory and Practice 2013, 19 (5), 543− 558. (39) Draper, R. J. Reimagining Content-area Literacy: Literacy Instruction; Teachers College Press: New York, 2010. (40) Langer, J. A. Envisioning Knowledge: Building Literacy in the Academic Disciplines; Teachers College Press: New York, 2011. (41) Joyce, B. R.; Showers, B. Student Achievement Through Staff Development; ASCD: Alexandria, 2002. (42) Dillon, D. R.; O’Brien, D. G.; Moje, E. B.; Stewart, R. A. Literacy Learning in Secondary School Science Classrooms: A Crosscase Analysis of Three Qualitative Studies. J. Res. Sci. Teach. 1994, 31 (4), 345−362. (43) Carspecken, P. L. Critical Ethnography in Educational Research: A Theoretical and Practical Guide; Routledge: New York, 1996. (44) Texas Education Agency. 19 TAC Chapter 112. Texas Essential Knowledge and Skills for Science. §112.35. Chemistry, Beginning with School Year 2010−2011 (One Credit). http://ritter.tea.state.tx.us/ rules/tac/chapter112/ch112c.html. (45) Nolen, A. L.; Vander Putten, W. Action Research in Education: Addressing Gaps in Ethical Principles and Practices. Educational Researcher 2007, 36, 401−407.

I

DOI: 10.1021/acs.jchemed.8b00294 J. Chem. Educ. XXXX, XXX, XXX−XXX