Assessing Citizenship: Questioning Our Goals - ACS Publications

Chapter 10

Downloaded via UNIV OF LOUISIANA AT LAFAYETTE on January 10, 2019 at 03:42:11 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.

Assessing Citizenship: Questioning Our Goals Stephen B. Carroll* Senior Lecturer, Department of English, Santa Clara University, 500 El Camino Real, Santa Clara, California 95053-0001, United States *E-mail: [email protected]

SENCER (Science Education for New Civic Engagements and Responsibilities) has long sought to improve science education and engaged citizenship by teaching science through contested, capacious civic issues. SENCER has offered ample evidence that teaching science this way enhances students’ science education across multiple dimensions, yet evidence that taking SENCER courses makes students more effective citizens has proved elusive. Part of this is attributable to one-dimensional assessment techniques; another part is attributable to an impoverished model of citizenship; but perhaps the biggest obstacle has been a belief that it is not possible to assess citizenship rigorously because it depends on things like attitudes, values, habits and other things that are difficult to measure. This chapter argues that it is very possible to assess citizenship rigorously using mixed methods that include direct and indirect measures. It takes as its foundation a comprehensive and robust definition of citizenship provided by the National Council for Social Studies that includes attitudes/dispositions, values and habits. It suggests possible assessments using objective and subjective direct measures as well as possible questions for self-reports using the SENCER-SALG instrument. However, getting positive results on those assessments will require substantial shifts in pedagogy and a far greater emphasis on citizenship than most SENCER faculty are likely to be comfortable with. Thus, while rethinking SENCER’s strategy provides us a path for assessing citizenship, it also

© 2018 American Chemical Society Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

raises questions about whether it is appropriate to do so in science classes. It may be preferable to teach citizenship through partnerships with other teachers and professionals who have expertise and experience in teaching the knowledge, skills, attitudes and habits that characterize effective citizenship.

Introduction: SENCER and Citizenship From its inception in 2001, SENCER (Science Education for New Civic Engagements and Responsibilities) has been dedicated to the proposition that science is taught more effectively, meaningfully and durably by linking it to consequential civic issues. The first paragraph of SENCER’s mission statement reads, “SENCER courses and programs strengthen student learning and interest in science, technology, engineering and mathematics (STEM) by connecting course topics to issues of critical local, national, and global importance” (1). According to Wm. David Burns, one of SENCER’s founders, “Good science education supports good democratic education and vice versa” (2). Following a Jeffersonian philosophy, SENCER hews to the idea that teaching science in the context of civic issues simultaneously makes those students (and prepares them to be) better citizens. This dual aim of enhancing both science education and engaged, responsible citizenship is apparent in SENCER’s explicit aims: “Our goals are to: (1) get more students interested and engaged in STEM, (2) help students connect STEM learning to their other studies, and (3) strengthen students’ understanding of science and their capacity for responsible work and citizenship” (3). There is ample evidence that SENCER meets its first two goals and the first half of the third goal (strengthening students’ understanding of science). Fifteen years’ worth of Student Assessment of their Learning Gains (SALG) data show that students in SENCER classes are (1) more interested in and engaged in STEM than students in non-SENCER courses and (2) that they are more likely to integrate what they learn in their SENCER courses into their work in other, non-STEM classes (and into their daily lives) than students in non-SENCER STEM classes. (See SALG in Notes.) The data also shows (3) that students in SENCER classes make greater learning gains than their colleagues in non-SENCER classes, that those learning gains are greatest in higher-order thinking skills, and that those learning gains are more durable (4). One could point as well to other evidence of SENCER’s success: that SENCER is the National Science Foundation’s (NSF) longest-running continuously-funded project; that it has reshaped educational practices and curricula at places as diverse as the University of Hawai’i, Belhaven University, Harold Washington College, West Point, and Rutgers; that SENCER-affiliated faculty have published hundreds of articles and dozens of book chapters on using civic engagement to teach science; that a 2015 study hailed SENCER as one of four national communities of transformation that “shift departmental cultures, institutional norms, and disciplinary values” (5); and that SENCER has developed 148 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

and spun off projects which have successfully extended the SENCER approach to new contexts. (See SENCER Projects in Notes.) This leaves only the second half of SENCER’s third goal: strengthening students’ capacity for responsible work and citizenship. SENCER’s underlying citizenship theory is that stronger STEM education will prepare students to better evaluate and respond conscientiously to competing claims made about vital civic issues such as global warming, emerging infectious diseases, sea-level rise, health issues (cancer, AIDS, stem cells, obesity, human cloning), ecological changes, food insecurity, pollution, water use, drug and alcohol addiction, social inequalities, etc. Given that information and arguments related to these issues are often presented in scientific language and use statistics and/or other mathematical calculations, it is reasonable to assume that increasing students’ knowledge of and literacy in these areas allows them to understand, analyze and evaluate this kind of information more effectively than if they do not have these skills. But as experienced educators know, knowledge and skills alone are not enough; students need to be interested, engaged, and motivated enough to use those skills regularly and repeatedly until they become habits—habits deeply embedded enough to change those students’ engagement with themselves and with the world. SENCER’s modus operandi assumes that being immersed in civic life and in solving civic problems, causes students to develop those habits, and thus to become better citizens. If we bracket for the moment the question of how to define “better citizens,” SENCER’s implicit theory seems quite reasonable: when students get invested in civic work, they become more engaged in civic life in ways that create long-term proclivities to stay engaged in the civic issues that affect their local, regional, national and global communities.

Assessing Citizenship Unfortunately, it has proved quite difficult to demonstrate that teaching science this way affects civic engagement and citizenship positively. Anecdotal evidence is not hard to come by: most SENCER faculty report positive results for many if not all of their SENCER courses in this regard. Numerous students also report positive outcomes related to civic engagement. Informally, everyone “knows” that the SENCER approach works as it should. Nevertheless, real evidence has been elusive. Most of SENCER’s own self-studies have found that SENCER courses produce little or no gain in citizenship skills as compared with other, non-SENCER courses. The question is why. One possible answer is that we have been using an ineffective assessment strategy: literally asking the wrong questions. SENCER’s primary method for assessing gains in citizenship consists of ten questions probing habits of civic engagement on a customized SENCER-SALG survey. Three questions focus on changes in students’ attitudes toward civic engagement, six questions assess the extent to which students have integrated new habits of civic engagement into their lives and one question asks how much the interplay between science and civic engagement in the class helped students’ learning. There are no questions related 149 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

to acquisition of skills or knowledge related to civic engagement. (See Appendix A for the text of SENCER-SALG instruments.) There are a few problems with this assessment strategy. First, the questions related to civic engagement are relatively few and far between (10 questions out of 94) in an instrument that focuses primarily on science education. This communicates to students that the instrument designers prioritized science education over citizenship and biases students’ responses accordingly. Second, the questions themselves probe relatively few and simplistic conceptions of citizenship—e.g., how much more likely students are—as a result of their work in the class—to discuss civic issues with family and/or friends, take public action related to civic issues, apply knowledge of science to civic issues, vote. While there is nothing wrong with these questions per se, they lack depth and sophistication. Third, SENCER’s overall assessment strategy is hampered by the fact that at the time this assessment strategy was developed, few people thought that it was possible to directly assess attitudes or habits developed in a college classroom. The assessment plan therefore depends entirely on student-self-report, an indirect assessment method; it lacks any direct assessments, leaving it unbalanced, and overly reliant on a single measure. My purpose here is not to trash the previous assessment instrument—which I had some part in designing—but rather, to point out that the reasons this assessment strategy failed to show much difference in civic engagement between SENCER and non-SENCER courses are not difficult to discern. And even if those questions had revealed a significant difference, that wouldn’t have told us very much because the questions are so few and the model of citizenship they derive from is not clearly articulated. Thus, a new assessment strategy is called for.

NCSS Standards The recent explosion of academic interest in assessment has provided us new tools and techniques that make a more sophisticated and rigorous assessment of how citizenship skills develop in SENCER courses possible. As in any assessment, we need to start with a clear understanding of the project’s goals and objectives. Per SENCER’s mission and goals statements, in this case, we are trying to determine whether or not teaching science through civic engagement significantly improves students’ habits related to citizenship. We therefore need to begin with a robust and clearly articulated theory of citizenship. The National Council for Social Studies (NCSS) provides a clear, well-researched, and well-articulated model which informs K-16 public education, so that seems like a reasonable and responsible foundation on which to build. “NCSS has defined an effective citizen as one who has the knowledge, skills, and attitudes required to assume the ‘office of citizen’ in our democratic republic” (6). The NCSS model specifies that an effective citizen: • •

Embraces core democratic values and strives to live by them. Accepts responsibility for the well-being of oneself, one’s family, and the community. 150 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

• • • • • • • •

Has knowledge of the people, history, and traditions that have shaped our local communities, our nation, and the world. Has knowledge of our nation’s founding documents, civic institutions, and political processes. Is aware of issues and events that have an impact on people at local, state, national, and global levels. Seeks information from varied sources and perspectives to develop informed opinions and creative solutions. Asks meaningful questions and is able to analyze and evaluate information and ideas. Uses effective decision-making and problem-solving skills in public and private life. Has the ability to collaborate effectively as a member of a group. Actively participates in civic and community life (6).

This model and its constituent elements align well with SENCER’s philosophy and approach to education, making it an excellent definition of citizenship from which to assess whether or not SENCER courses are significantly more effective than non-SENCER courses in stimulating students to develop habits which make them better citizens. Because this model of citizenship is more fully articulated, deeper, more robust and more sophisticated than the one SENCER has been using so far, examining the ten elements of the model and considering carefully how each objective might be taught and assessed would provide us with course learning objectives, possible assignments/direct assessments and a superior overall strategy for assessing how well a given course promotes better citizenship. This would give SENCER a better chance to show that it really does prepare students to be better citizens.

NCSS: Citizenship Knowledge Three items on the NCSS list focus on knowledge acquisition: an effective citizen, (1) “has knowledge of the people, history, and traditions that have shaped our local communities, our nation, and the world”; (2) “has knowledge of our nation’s founding documents, civic institutions, and political processes”; and (3) “is aware of issues and events that have an impact on people at local, state, national, and global levels.” The NCSS also specifies that in an effective citizenship education program “all students are provided with instruction on our nation’s founding documents, civic institutions, and political processes,” and “on the people, history, and traditions that have shaped our local communities, our nation, and the world” (6). Because SENCER courses are primarily science classes, SENCER faculty are not likely to see the first two of these objectives as relevant to their courses or educational goals, so they probably won’t teach or assess either of these elements. While in theory, SENCER faculty could teach to these learning objectives (because as scientists, they are accustomed to teaching to learning objectives 151 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

focused on acquisition of knowledge), in practice they can no more teach history or social studies at a college level than history faculty could teach college-level science. Probably, the best one can hope for is that as instructors connect their instruction to knowledge of these topics that students presumably acquired in middle school and high school, that knowledge is activated and reinforced, deepening it and making it more durable through that repetition and by connecting it to other knowledge that students use and value. To the extent that student’s prior knowledge in these areas is integrated into instruction in the SENCER course, this will also facilitate greater retention of that knowledge because such integration makes that prior knowledge more meaningful (increasing its emotional value), creates more neural pathways that connect to it, and reinforces existing neural connections. Thus, while students may make some slight learning gains related to these two objectives in passing, because neither of these learning objectives is likely to be important to SENCER faculty, they are unlikely to be taught or measured directly. The third item on this list, while still primarily a knowledge-oriented objective, is a little different: since SENCER faculty teach their science through capacious, contested civic issues, it is nearly certain that one of the aims of any SENCER course is to significantly deepen students’ knowledge of “issues and events that have an impact on people at local, state, national, and global levels.” Indeed, increasing students’ awareness of such issues is central to SENCER’s mission and ideology. Because most SENCER faculty have well-developed practices for connecting students to the civic issues central to their courses and because students are immersed in those issues throughout the course, it seems more than probable that these learning objectives will be met without much difficulty. And since most instructors teaching SENCER courses regularly participate in faculty development related to science education reform, advanced teaching techniques, and learning-based teaching, they will likely employ diverse teaching strategies to supplement traditional lectures and labs to help students understand and retain the desired knowledge. SENCER faculty are also highly likely to use assessments that rank in the top half of Bloom’s Taxonomy because they know that this mitigates cramming and facilitates greater retention of what was learned (4). Moreover, the civic engagement components of that SENCER course make it nearly certain that students will have to use active learning strategies and that they will have to apply, practice and adapt what they have learned regularly. All of these elements will facilitate long-term retention of what is learned. In this case then, existing assignments can almost certainly be used to direct assess the degree to which students have become “aware of issues and events” relevant to the course “that have an impact on people at local, state, national, and global levels.”

NCSS: Citizenship Skill One item on the NCSS list focuses on a skill, specifying that students should have “the ability to collaborate effectively as a member of a group.” The wording of this objective is a bit misleading, since the writers presumably want students 152 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

to actually use those collaborative skills when appropriate, but the intent of the objective is still reasonably clear. As with knowledge, most science teachers are comfortable teaching skills. Many science classes require students to work in groups—especially in labs and on projects, and SENCER classes with robust civic engagement components usually require students to work in teams, so this is familiar territory for most science faculty. And while teaching students to work in groups is not commonly perceived as something within the purview of science instructors—and it is most definitely a skill that students need to be taught—it is not especially hard to teach, and resources for teaching group-work skills are plentiful. Many books have been written on group-work and team-based learning at the college level in the past decade, and a large number of universities offer rich and well-curated resources for developing projects and assignments that require students to work in groups as well as strategies for teaching students how to do that work. Protocols and rubrics for assessing group work are also widely available. Most tend to split the assessment process into two components: the work produced by the group (projects, posters, presentations) and the contributions (and learning) of the individuals who make up the group. Here too, SENCER faculty, by virtue of their regular exposure to workshops on active learning strategies and best practices in college teaching are likely to be well aware of and to use a variety of these teaching strategies and assessment practices. Thus, neither teaching nor assessing this fourth element of citizenship are likely to be at all problematic for SENCER teachers or any other experienced teacher.

NCSS: Citizenship Habits of Mind/Attitudes The remaining six items on NCSS’s list, • • • • • •

Embraces core democratic values and strives to live by them; Accepts responsibility for the well-being of oneself, one’s family, and the community; Seeks information from varied sources and perspectives to develop informed opinions and creative solutions; Asks meaningful questions and is able to analyze and evaluate information and ideas; Uses effective decision-making and problem-solving skills in public and private life; and Actively participates in civic and community life,

can be broadly described as habits, habits we hope students will continue to engage in for the rest of their lives, since they pertain to core elements of engaged democratic citizenship. As Plutarch reminds us “character is habit long continued,” so if we want students to develop the characteristics of effectively engaged democratic citizens, we must teach them the habits that create those characteristics. Seeing these six NCSS learning objectives as habits alerts us to the first major difficulty with using this definition of citizenship. I have written elsewhere about the epistemological conflict in most faculty’s definitions of 153 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

teaching and learning (7). In essence, the problem is that when most faculty think about teaching, their goals nearly always aim at producing changes in their students that will last the rest of their lives, yet when those same faculty think about learning—as they must when it comes to assessing their students’ progress—they almost always express their goals in terms of lower-order knowledge and skills that are not durable unless they are valued and practiced enough to evolve into habits. Few teachers conceive of learning in a way that allows them to measure or assess the degree to which what they teach is likely to be integrated into students’ lives durably; nor do they conceptualize the purpose of their teaching as inculcating habits. This epistemological disconnection therefore presents the first hurdle for this way of thinking about citizenship: few faculty explicitly aim their teaching at changing habits, yet most of the elements of this definition—and I would argue, all the most vital elements of the definition—are habits. A second layer of complexity lies in the fact that these six learning objectives fall into two categories. Three of them (“seeks information from varied sources and perspectives to develop informed opinions and creative solutions,” “asks meaningful questions and is able to analyze and evaluate information and ideas,” and “uses effective decision-making and problem-solving skills in public and private life”) are dispositions which aim at improving the processes of thinking and deciding. We can see them as habits of mind, or more precisely, attitudes—presumably, the “attitudes required to assume the ‘office of citizen’ in our democratic republic” (6). Although one has an attitude embedded in it by the word “embrace,’ the remaining three goals are habits of action conditioned by and dependent on those attitudes. The three bullets which name attitudes support information literacy and critical thinking. They are about gathering information from appropriately diverse sources; questioning, analyzing and assessing that information; and using it ethically, effectively and creatively to solve problems and guide decisions so that those solutions and decisions are informed, conscious and conscientious. According to Arum and Roska, 99% of college teachers value critical thinking highly, yet Pearlman and Carillo argue convincingly that “few actually emphasize critical thinking as the seminal force in their teaching” (8, 9). In their presentation at the Lilly Conference for College and University Teaching in Bethesda, MD in 2017, Pearlman and Carillo put the number of faculty who explicitly and directly teach critical thinking in their classrooms at about 4%. Even if the number is twice or three times what Pearlman and Carillo claim, the gap between it and the 99% who value critical thinking makes a strong argument that all college teachers need to spend more time teaching to these three items on the NCSS list. One consequence of the expanded role of social media in politics—especially around the 2016 election—was an explosion of scholarly articles on, and faculty development materials related to, information literacy and how to teach it. And some of this new material crosses over into how to teach critical thinking. Yet these materials tend to treat teaching information literacy and critical thinking as skills, ignoring or taking for granted the intended longevity of those skills and the fact that for those skills to turn into habits, students have to be sufficiently engaged and motivated to practice them long enough for them to become habits. The ways in which information literacy is taught—usually in library sessions 154 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

and in first-year composition classes—compound this problem: a couple of workshops and one or two sessions in first-year classes are a good foundation to build on and we should coordinate our work with those who teach these skills, but it is clearly not enough to establish information literacy or critical thinking as lifelong habits, especially for a generation of students nearly all of whom seem to believe that they can access all the information they need via a simple Google search. Further complicating the problem, information literacy standards and practices vary widely in different disciplines, which can give students the impression that the practices various instructors label as information literacy skills are really just a bunch of arbitrary and unnecessary hoops created by overly pedantic professors. Until information literacy and critical thinking are taught horizontally and vertically across the whole curriculum systematically, a few SENCER courses are not going to create significant gains in these aspects of citizenship. On the other hand, if we think about teaching information literacy and critical thinking as attitudes—as something to be developed systematically throughout and across the curriculum, we focus not only on building skills, but also on building the motivations, tendencies and values that will cause students to practice what they have learned regularly enough and often enough to turn those skills into habits that have a chance of persisting even after students leave college. Thus, it makes sense to examine each of these objectives individually to consider how each might be taught and assessed with the idea in mind that the aim of teaching these attitudes is to develop each one into a habit. The bullet that logically comes first in this trio is, “seeks information from varied sources and perspectives to develop informed opinions and creative solutions.” Taken as an attitude, “seeks information” indicates curiosity and a willingness to act on that curiosity. The phrasing which follows embeds several assumptions and dependencies. First, varied sources and perspectives are necessary to the development of informed opinions and creative solutions. This assumption enshrines diversity and pluralism of sources as critical elements of proper—i.e., informed—decisions. It implicitly rejects as less valuable opinions that are not informed. This may seem like a trivial point, but in an era where we know that fake news spreads faster than real news (10), that two-thirds of Americans get their news from social media (11), and that “echo chambers” and “filter bubbles” severely limit the diversity of information consumed by the 61% of millennials who get their news primarily through social media (12, 13), this can’t be taken for granted. It must be taught. This bullet further implies diversity and pluralism of sources are required for “creative solutions.” Here too, there may be more than meets the eye. In one sense, this assumption names the obvious—you can’t know if a solution is new until you have done the research to confirm that it is indeed new—and that it works (i.e., that it is a solution). But in another sense, we can see this stipulation as saying that “creative” solutions—i.e., solutions that “create” something new—need to be informed by multiple perspectives, that they need to be integrative and interdisciplinary. Relatively few university teachers, bound as we are within the silos of our disciplines, actively teach the skills and attitudes necessary to ward off the insular tendencies of millennial American society and to 155 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

develop proficiency in cross-disciplinary or interdisciplinary thinking that would lead to such “creative solutions.” The assumptions contained in the language of this objective thus raise some challenges that complicate the tasks of teaching and assessing it. At the surface level, teaching students to seek varied sources and perspectives is easy—just require students to use diverse sources for class projects, term papers and the like. SENCER faculty often do require students to examine the civic issues they engage with from multiple perspectives. Cathy Middlecamp’s class that examines the sources and carbon-footprint of the food served in the campus cafeteria is a good example. But if that is the only way we teach it, the most likely outcome will be that students learn to exercise this kind of curiosity and to seek diversity of sources only for school papers and projects. Because few students write essays or do projects similar to those they did in school once they leave college, and because few college professors actively teach transfer, they are unlikely to transfer what they have learned into their non-school life (14). This skill is therefore unlikely to evolve into an attitude or habit of mind that predisposes students to action as we want. Instead, we want to give students assignments that require them to practice seeking information from varied sources and perspectives all the time. There are many ways to do this. For example, assigning readings in pairs wherein the authors disagree with each other not only creates diversity of sources and perspectives, it helps students move from dualism to multiplicity in Perry’s scheme of moral and intellectual development by helping them to see that even experts often disagree (15). It also forces them to be active in their own thinking as they struggle to apply contradictory or orthogonal ideas. It compels them to consider what will work best for them in their circumstances. In short, it helps them develop a habit of critical thinking. It may also build curiosity, as their desire to resolve the contradictions motivates them to dig deeper into the topics. The civic engagements that characterize SENCER classes allow many assignments to require a substantial diversity of sources and opinions, so that students learn to evaluate and use sources appropriately in different circumstances, for different audiences, for different tasks, and so on. This helps students see that learning to do research and to use sources well is part of everyday life, not just something that professional academics do. One exercise that helps students develop the habit of seeking diverse sources of information is to require students to get all their news—especially news related to the civic issue on which a course centers—from sources other than the ones they usually rely on for a full week. This includes social media as well as traditional media like TV news, newspapers, radio, etc. At the end of the week, ask students to reflect on and assess how the shift in sources changed their perceptions of how information is used and delivered across different platforms. Depending on the class and academic discipline, students can also be asked to analyze how the different ways the information is presented reflect the biases and values of different audiences, the economic and/or political consequences of different ways of presenting the news, the scientific accuracy of the information presented, etc. This exercise can be easily adapted to content readings, journals, and so on, depending on the level and discipline of the class. 156 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

On the other hand, there are some limits here: seeking diversity of sources in relation to the science being taught is not necessarily a good thing. Traditionally taught science classes certainly do not teach students to seek information from varied sources and perspectives. Students are supposed to get their information mainly from textbooks, lab manuals and other course materials, all of which have been painstakingly developed to be consistent, integrated and uniform. One is not likely to find much diversity of perspective in most textbooks. Even in SENCER courses, this is likely to be true. While we may encourage students to seek information from varied sources and perspectives in relation to civic issues, it is much less clear that we want them to do the same in relation to the science they are supposed to be learning. This contradiction points to a fundamental question related to teaching SENCER courses: what do our students learn when we teach the science one way and the citizenship another way? And while we want our students to develop informed opinions in relation to the civic issues they study, do we really want them to do the same in relation to science? How comfortable are we in teaching them to have opinions about science? The same is true for creative solutions: Do we really want students seeking creative solutions in labs? To assess these assignments is relatively straightforward: we can evaluate how diverse the sources are for any assignment. But to assess whether or not students have developed the attitude this objective aims at, we need to evaluate the extent to which students seek this diversity of sources and perspectives in assignments where it is not explicitly required. Thus, course and curriculum designs should compel students to seek such diversity of sources and perspectives initially, give them adequate time to practice, then cease requiring it. Measuring the extent to which students continue to demonstrate that commitment to diverse sources and perspectives when it is no longer required allows us to assess how well they have developed this attitude of curiosity and commitment to diversity of sources and perspectives. We can also do pre/post assessments of the diversity of sources and perspectives used in papers and projects related to civic engagement issues from the beginning of the course to the end, and from lower-division courses to upper-division courses. We can assign annotated bibliographies and again look for these same kinds of diversity of sources and perspectives. Most important, we can assess how well students work with sources which espouse viewpoints different from the students’ and how well they are able to integrate those viewpoints and arguments into their own thinking and work. Are they able to recognize divergent viewpoints? Can they describe them in a way that would be acceptable to a person who holds those beliefs? Do they integrate those divergent ideas into their own work or do they hold them at arm’s length? Do they learn from and integrate those other ideas into their own work and their own thinking—not just on one paper, but consistently in the kinds of comments they make in class and in their work as they advance in the class and in their discipline? Do they integrate those ideas into their own coherently and deeply? Such assessments allow us to measure the degree to which students have developed the attitude of seeking “information from varied sources and perspectives to develop informed opinions and creative solutions.” 157 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

The next objective, “asks meaningful questions and is able to analyze and evaluate information and ideas,” extends the implicit agenda of curiosity by requiring the successful student to interrogate the information s/he has found through the seeking stated in the previous objective. Presuming that some of part of “meaningful”-ness concerns assessing the truth of that information—as the rest of the objective suggests, this objective aims to develop the attitude of skepticism and habit of analysis. It builds on the last objective by laying out a three-stage process for evaluating information a student finds as a result of curiosity and desire for diversity of perspectives: first the student learns to ask meaningful questions, then s/he analyzes the information and ideas s/he gathers relevant to that those questions, then the student evaluates the information and ideas. Note that two kinds of material are being worked with here—information and ideas: the former can be equated to facts and the latter can include values, stances, perspectives, theories, viewpoints, etc. so students are not just evaluating facts, but also their ways of working with and understanding those facts. Flipped classrooms are one way to teach students to ask meaningful questions. At the beginning of the course, when students come to class after having read the material required for that day, the instructor and/or teaching assistants can model good questions to ask about the material and explain why those questions are helpful and meaningful. This sets the bar and demonstrates not only the level of questioning expected, but also the depth of understanding of the material that the professor expects. After a few days of such modeling, students can be assigned to prepare questions and lead discussions. Once students get the hang of this, discussion leaders can be chosen on the day of class so that everyone has to be prepared every day. If students know that they will be evaluated according to the value of the questions they ask about the material they have just read/prepared, they will soon develop the habit of asking meaningful questions. Similarly, students can be assigned to prepare for a debate but not told until the day of the debate which side they will be on so that they have to prepare for both sides equally; this teaches them to ask meaningful questions not only about opinions or perspectives they oppose, but also about those they support. It thereby helps them learn to be more self-critical. Students can also be required to articulate alternative viewpoints or probable positions on the other side of any debate to the satisfaction of the other side before they can present their own side. They may be asked to write critical analyses and evaluations of their own side from the perspective of an alternate, skeptical or opposed viewpoint. All of these assessments compel students not just to ask questions, but to analyze and evaluate information and ideas they would normally prefer to ignore. It teaches them to understand, respect and perhaps even empathize with other perspectives (by forcing them to see through other people’s eyes)—an outcome which reinforces the earlier objective of seeking diverse sources and perspectives. Again, if they are expected to do this all the time, students soon develop a knack for asking meaningful questions and thinking analytically, carefully and thoughtfully about information and ideas that don’t necessarily align with their own—and for then applying those same techniques to their own positions and ideas. That sounds like a pretty good way to build a critical thinker. Especially if we can get those students to consistently reflect on and analyze and evaluate their own thinking; 158 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

that’s why reflection assignments that ask students what they learned and how that learning has changed them (or not—that always needs to be an option to keep bullshit at bay) should be a regular part, not only of each assignment, but also of each class session (even labs). The point of these last two objectives is to support the third attitude—using “effective decision-making and problem-solving skills in public and private life.” This objective manifests an attitude that one should seek to apply what was learned to other areas of life (thereby encouraging transfer) and a desire to make effective decisions and solve problems. Neither of these things are necessary outcomes; students are perfectly capable of learning advanced problem-solving skills in college classrooms and never applying them anywhere else. Similarly, people can live their entire lives primarily according to tradition, engrained habits and emotional impulses, without placing much emphasis on making effective decisions. Desire to escape this way of living is one good reason to go to college, but that escape does not come automatically; we have to teach students to want it before they will do it consistently. Obviously, most assignments in any class require students to make effective decisions and to solve problems, but SENCER classes that include civic engagement projects offer singularly suitable occasions to measure students’ progress toward this objective. Those civic engagements allow us to observe the extent to which students use the science they were taught appropriately in their project. Watching them apply what they have learned in this real-world setting allows us to discern how well they really understand what they have learned. Cynthia Maguire’s dual-poster project—in which students prepare one poster on their research to present to an academic audience and another poster on the same subject to present to a lay audience—is an excellent example of how this works: having to present their findings without using scientific language requires students to be creative and to solve a complex problem using the science they have learned (16). And both the poster and the students’ presentation of it reveal the extent to which they have made effective decisions. We can also observe how well they apply their information literacy and critical thinking skills. Do students show good judgment and creativity? Are they dogmatic and rigid in the ways they solve problems or are they flexible and resilient? How much do they study and research before they make team decisions? Are the solutions they develop appropriate and effective? Do those solutions integrate what they have been taught in class or do they ignore that information? How well do students adapt their theories and beliefs to new kinds of information (e.g., new studies that appear late in a class or new information that becomes available late in a project)? How well do they work in teams? Do they work collaboratively and supportively, or do they work individually and then patch it all together at the end (or does one person do all the work and let the others ride on their coattails)? Can we detect evidence of students applying what they have learned in our classes to problems they face in other classes or in other areas of their lives? These are questions that many humanities and social sciences faculty ask all the time, and they are useful ways to assess the extent to which students use “effective decision-making and problem-solving skills in public and private life.” They are not things that most SENCER faculty are used 159 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

to assessing, but if we accept that they are essential to developing good citizens, we must assess them deeply and regularly, suggesting a need to borrow from and/or collaborate with faculty in those other disciplines. Perhaps even more daunting, we need to teach and assess all three of the attitudes that are part of the NCSS definition of citizenship regularly in all SENCER classes. If students are really supposed to develop these attitudes as habits of mind, they need to practice them regularly, in different settings and contexts so that those attitudes become second nature. Only then can they fully support the habits of action that are the final elements of NCSS’s definition.

NCSS: Habits of Action The final three objectives on our list are habits which describe actions taken by effective citizens: • • •

Actively participates in civic and community life; Embraces core democratic values and strives to live by them; and Accepts responsibility for the well-being of oneself, one’s family, and the community.

As with some of the items discussed above, these are not things that science teachers usually teach or assess, but they are related to practices and objectives we do assess, so it may be possible to extrapolate from those. For example, many of us grade students on participation. (Do we first teach them how to participate effectively?) Similarly, most of us expect our students to display democratic values, starting with valuing life, liberty and the pursuit of happiness, but also including equality and justice and the values necessary to support them: commitment to truth, respect, tolerance, diversity, and popular sovereignty: we expect students to treat each other with respect, to express their opinions freely, to value the truth, and so on. However, few science teachers—even among SENCER faculty—teach students how to express these values nor do they assess their expression in any formal way except in cases of extreme violations of social norms. Finally, we expect students to be responsible for their own behavior, for their own well-being, and to demonstrate some degree of concern for others, though again, we don’t usually teach them how to do these things nor do we assess them. Fostering good citizenship requires us to do both. In fact, despite the apparent similarities between these final three learning objectives and typical science-education practices, taking these three items seriously poses substantial challenges to our pedagogy, assessment practices and institutional norms. For instance, the first item asks us to facilitate students’ active participation in civic and community life. At first, many SENCER faculty might be tempted to answer, “No problem! That’s what we do in SENCER.” Upon further inspection however, the bar gets higher than one might expect. Let’s start with the easy one: civic life. It seems reasonable to interpret this division between civic life and community life as dividing participation in political, governmental and other formal institutions from participation in informal, communal aspects of 160 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

our public lives. There are many ways to teach this participation: we can bring our students to city council meetings, waste management board meetings, water board meetings and other municipal agencies. Garon Smith, recently retired from the University of Montana, did this regularly. Bringing students to such meetings allows them to understand the rules of participation and eventually, to participate themselves, especially if we provide them with guidelines like Robert’s Rules of Order, charter documents, past proceedings, and so on. This lesson can obviously be scaled up to include state officials and governmental bodies as well as their federal counterparts. It can also be extended to non-governmental bodies like universities, NGOs, civic associations, volunteer groups and the like. But if we expect students to participate actively and meaningfully in civic life—and are not satisfied by the mere fact that they vote (often without doing meaningful research)—we need to teach them how to do that. We need to teach to the level of participation we desire and expect. The same is true of the less formal participation indicated by “community life.” Communities—as opposed to groups—are characterized by bonds of caring and mutuality. If we expect our students to value, form, maintain, and actively participate in communities (as opposed to mere filter bubbles), we have to teach them how to create, nurture and sustain those bonds. Thus, we enmesh ourselves in teaching students how to be vulnerable with each other, how to express genuine care, how to nurture relationships even when rifts appear, how to recover from disappointment, how to express kinship, friendship and even love in appropriate ways in community settings. We will need to teach emotional awareness, emotional management, and even emotional development. All of these things are teachable: therapists and community activists teach them for a living. So do community facilitators, camp counselors, athletic coaches, and sometimes, writing teachers. Perhaps we can learn from them. Developing empathy and community are the two most important learning objectives of my Critical Thinking and Writing course (because understanding audience on a deep level improves one’s ability to communicate effectively with them—in writing and in discussion). However, even highly engaged and motivated classes who live on the same floor of a dorm take more than fifteen or sixteen weeks to learn to trust each other, to share deeply and to fully bond as a community. It also takes many readings and videos (on safe/brave spaces, shame, vulnerability, empowerment, emotional regulation and expression), lots of trust/vulnerability-building exercises, many discussions, and a lot of critical thinking and reflective writing. Making active participation in community life a significant learning objective requires a serious commitment to teaching students how to form and develop communities so that they understand how communities work and can participate in them effectively; it takes a lot of time both in and out of class, it takes sustained focus, and it requires substantial emotional investment, emotional (self-)awareness, and emotional leadership. Again, active participation in community life is teachable, but it involves content, teaching methods and assessments that are well outside the comfort zone of most science teachers, even SENCER teachers. In my experience, it consumes about half of any course it is taught in—and to be fully effective, it needs at least twenty weeks to develop to 161 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

even moderate levels of effectiveness. Are SENCER faculty willing to give up that much of their courses to teach this habit? Assuming we are willing to make this commitment, we will then have to assess students’ participation in both civic and community life. Obviously, we want our students to vote—in national, state and local elections, and that is simple enough to assess, but it is also far from enough. We want to know the extent to which students engage in the civic and communal lives of the multiple communities that they belong to. Do they vote in campus elections? Participate in student government? Do they participate in group decision-making processes in their dorms? In fraternities, sororities and on-campus clubs? How often and how well do they put into practice the teamwork and civic skills they learned in our classes? How well do they collaborate on civic engagement projects? Many of these participation goals can be assessed simply by counting: How many meetings did you attend? How often did you speak? How many of your ideas/proposals were discussed? Objectives that include “how well did students…?” will require observations, reports or artifacts upon which to base assessments, but in theory such assessment is not difficult: most faculty are comfortable evaluating the degree to which student performance demonstrates what they were supposed to have learned, and this process can be streamlined by using rubrics. Staff in offices of student life assess these things all the time; perhaps we should borrow some of their techniques. As we shift from civic life to community life, we move from environments in which students have relatively limited power to those in which students have greater ability to create, shape and lead groups of people, so our assessment practices shift accordingly. Now we want to know if—and how well—they form and maintain communities when not required to do so. Do they form effective study groups? Do they use group chats and group texts to build community and to help each other? Do they collaborate with each other in classes? Do they recruit others to join their communities appropriately? How well do they communicate within their communities? Do they make sure that everyone in the class stays up to speed and has all the relevant information? (Lab teachers may be able to assess these things but doing so in a class of 500 presents some logistical problems.) How well do they help and take care of each other—especially those who are most vulnerable? Are they able to create spaces in which everyone feels not only safe, but allowed to express viewpoints that might not be shared by everyone else? How well do they follow the rules of those spaces once they have created them? How much of their emotional connections to the topics they discuss (especially in their civic engagement projects), their communities and their classmates do they share? Most of these things can be counted and/or observed and assessed relatively easily (again, many of these things can be evaluated using rubrics), but before we can assess, we need to teach our students how to participate actively in civic and community life. My guess is that few science teachers will be comfortable teaching or assessing these things. Moreover, teaching that will require a significant reorganization of what and how we teach, a reorganization that might not sit so well with accreditation boards. Students’ ability to participate actively in civic and community life bleeds over into the next item: “embraces core democratic values and strives to live by 162 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

them.” The NCSS puts great weight on this objective, because it sees democratic values as central to citizenship: For our democracy to survive in this challenging environment, we must educate our students to understand, respect, and uphold the values enshrined in our founding documents. Our students should leave school with a clear sense of their rights and responsibilities as citizens. They should also be prepared to challenge injustice and to promote the common good (6). As we have seen above, such values include liberty, truth, working for the common good, justice, equality, diversity, tolerance, and popular sovereignty. This is a lengthy list, so educating our students to understand, respect and uphold all of these values is a tall order. Moreover, while it seems probable that most faculty understand, respect and perhaps even practice these values, it is unlikely that many science faculty—even SENCER faculty—teach all of these values, let alone assess all of them. Let us therefore pick one of these values, liberty, and examine how teaching and assessing it would change our teaching practices. How free are students to act, think and believe as they choose (within the norms of legal and civil behavior) in our classrooms? Most college classrooms are fairly authoritarian structures in which students have little to no real power and are not encouraged to make meaningful decisions about what they are learning or why. As Allen Harrison points out in “Teaching and Democratic Values in Higher Education,” Traditional methods tend to be authoritarian, non-participative, curriculum-centered rather than student- or problem-centered. Hence they stand in opposition to democratic values, paradoxically the foundation of the society in which teaching institutions are embedded (17). Teaching this habit might therefore require us to give students more power to shape the course than most faculty are used to or comfortable with. To start with, we will need to allow them to make some decisions that allow them to exercise some liberty—and we will have to teach them how to do that. At this most basic level, we might assess how often students vote in class decisions: when students are allowed to make decisions about class content, policies, protocols, etc., how many of them actually vote? (Are there patterns to who doesn’t vote?) Are students willing to take power and ownership, or do they prefer to remain passive and look to authorities to make important decisions for them? This can give us some sense of how empowered and engaged students feel. (No one needs to be taught to weigh in on a decision, of course, but they may need to be taught that it matters; exercises in which small groups of students are coached to steer policies in ways which later turn out to be detrimental to the class’ interests can often make this point very convincingly.) We can also assess how well students listen to each other when arguing about those decisions: do they adapt their positions, compromise and/or change their minds, or do they simply try to talk over each other without taking 163 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

other opinions seriously? When students work in teams, do they adopt democratic, collaborative decision-making protocols, or do they favor authoritarian models? Active listening skills, how to speak with respect and how to work in democratic teams might need to be taught explicitly. Is that what we want to spend time on? At the next level, both pedagogy and assessment get more challenging. Placing a value on liberty, especially in the context of the Founding Fathers, means more than just making choices in circumstances dictated and constrained by authorities. It includes the obligation to resist those authorities when appropriate for moral, economic or social reasons. That obligation is reinforced in the NCSS’s phrase “strive to live by them.” When combined with the value of popular sovereignty, liberty decentralizes power and tends to undermine authoritarian and hierarchical power structures (like classrooms and universities). That would require us to teach students when and how to exercise their power as citizens. How many of us are willing (and capable and qualified) to teach students to fight (even if only legally and socially appropriately) for their choices and beliefs? It is certainly possible to teach these things; many of the documents written by the Founding Fathers, including the Declaration of Independence and the Federalist Papers, lay out the philosophical justifications for such actions, and union organizers and other social activists regularly teach the practical skills of pushing back against entrenched power structures. Professors who teach writing, ethnic studies (in its many departments and guises), sociology, critical theory, liberation theology/pedagogy, literature, political science, philosophy, art history, and other critical disciplines also have many techniques for teaching students to question deeply and to critique the power structures—especially those which violate democratic principles—that govern their lives, choices and thoughts. We can borrow teaching techniques and methods of assessment from them. Though some of their methods are time-consuming and cumbersome, Margaret Syverson, Jan McArthur and Kristi Holsinger have published potentially helpful contributions on assessment for social justice (18–21). But are we willing to cede sufficient power to students that our classrooms become laboratories that teach and assess the intersections of liberty and democratic processes? I give students more control than any professor I know: I allow them to determine assignment due dates and times, determine class goals and policies, determine rules for discussions, create and enforce civility protocols and much more, but I always retain ultimate authority. Though I almost never override students’ decisions, both they and I understand that I can do that at any point, so even though I push this boundary pretty hard, I can’t really claim to do more than dabble in this area. I believe it is possible to go further than I have gone, but I question whether it is realistically possible to genuinely overcome the inherent and institutional power differential between a professor and her or his students, especially in a science class that may be embedded in a curriculum with a high degree of vertical integration. That barrier may be insurmountable within the structures of higher education as we currently understand them. But even if we don’t have to go quite that far, even if we are willing to settle for a limited power-sharing that teaches and assesses students’ ability to wield their liberty without reimagining the academy, this is still a major undertaking, one that will require significant changes in how most SENCER professors teach. It 164 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

will take a lot of time and study to implement conscientiously, a lot of time to teach/implement, and a lot of time to assess. The final objective on the NCSS list states that to be good citizens, students must “accept responsibility for the well-being of oneself, one’s family, and the community.” As with the previous two items on the list, assuming responsibility entails that students are given sufficient opportunities to exercise choice/liberty and power that they can take responsibility. We need not rehearse the pedagogical challenges mentioned above, but it is worth mentioning that the idea of teaching students to be responsible raises a whole new set of questions. It implies that students must learn to make choices and decisions for themselves, by themselves, as independent thinkers and actors. Now the matter is not so much whether we teach students to push back against authoritarian structures when appropriate, but that we teach them to function independently or outside of those structures. As Harrison points out, traditional pedagogies tend to create students who are passive and dependent (17). Rigid structures and policies amplify those effects by depriving students of the need and opportunity to exercise responsibility. More flexible policies—e.g., allowing students a limited number of free absences and/or late assignments without requiring notes or explanations—help them learn to manage their own affairs independently. Assuming that students are provided adequate instruction on how to exercise responsibility, and in what their responsibilities are, the more opportunities they have to practice, the better. By using civic engagement as a pedagogical tool, SENCER faculty create many substantive opportunities for students to make meaningful choices, think independently, and take responsibility for their decisions. Civic engagement projects nearly always empower students to make decisions in conditions which are not closely supervised, to exercise judgment, and to be accountable for the decisions they make. They also allow students to set goals for themselves and to assess their progress toward those goals. Many SENCER projects incorporate service learning, which amplifies those benefits because it engages students in other people’s points of view. Service learning provides students with an opportunity to develop their affective as well as cognitive needs by providing them with them an opportunity to feel valued as a member of a community, to feel empowered by their ability to affect [sic] change, and to develop their social skills by working cooperatively with fellow classmates and community members. Not only is the community enhanced by the students’ service, so are the students (22). Feeling empowered to effect change is certainly one component of taking responsibility, and it seems likely that feeling valued by one’s community and developing collaborative social skills will also contribute positively to feeling responsible. So, at this level, teaching students through civically-engaged service learning projects seems like a good way to teach students to take responsibility for themselves, and possibly even for their communities. Assessment at this level is also not difficult. We can examine the extent to which students set goals for themselves. Do they demonstrate empowerment to 165 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

think, decide and act on their own? Do they take responsibility for their own actions? For their own learning? Most of these questions can be evaluated through observation of student behaviors (in conjunction with rubrics) and evaluation of written documents, especially reflective writing assignments. Yet as with many of the other items on the NCSS list, there is another layer to this learning objective. Taking responsibility for oneself is one thing but taking responsibility for one’s family and one’s community is another. When responsibility expands in this way, it essentially becomes leadership. Here, we are concerned with students’ ability to take responsibility for their groups, for their class, for the communities they see themselves as members of. Do they set goals not just for themselves, but as leaders of groups? Do they take responsibility for the other people in their groups? Do they hold themselves accountable for the performance and coherence and community of the class as a whole? Leadership is commonly taught in business schools, the military, and in professional organizations, so it is clearly teachable and assessable, and teaching/assessment materials are easy to find, but like so many other items on this list, it falls outside of the range of what most SENCER faculty are used to teaching. Again, we may need to borrow from or collaborate with others. The final layer of this final bullet is that like the other two items in this section, it is a habit. For students to continue to be responsible for their well-being and for that of their families, and communities—independently (outside of our classrooms) and well after they leave college—they will need to manage their own self-development, learning and leadership indefinitely. Thus, we need to teach them not only how to learn effectively on their own and how to be leaders, but also how to self-regulate and how to continue to create and develop themselves as autonomous thinkers, agents and leaders. I have presented many talks on how to teach students metalearning (learning to learn) skills and written about it in the National Teaching and Learning Forum (23). Sandra McGuire’s Teach Students How to Learn and Terry Doyle’s and Todd Zakrajsek’s The New Science of Learning are excellent resources (24, 25). There are so many good books on teaching leadership, that it makes no sense for me to list them; you will be best served by asking colleagues for recommendations specific to your own field. Linda Nilson wrote a fine book on Creating Self-Regulated Learners (26). Marcia Baxter-Magolda has written and edited several important books on students’ moral and intellectual development and self-authorship (27–30). These resources can provide excellent guidance for teaching students in ways that develop their abilities to continue to learn, lead and self-regulate and develop as responsible citizens well after they leave college.

Direct and Indirect Assessments: SALG We have now reached the end of the NCSS list, so we return to the larger question of assessing citizenship. One of the things I criticized SENCER’s original plan for assessing citizenship for was over-reliance on a single method of assessment. So far, we have been talking almost exclusively about direct measures, so to avoid falling into the same trap I objected to earlier, it would be 166 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

wise to add an indirect measure. That would be helpful for another reason as well—the direct assessments discussed above are all integral parts of courses; many of them will be graded activities and even those which are not will still likely be perceived by students as assessments (or parts of a larger assessment), so students have an incentive to behave as they think their instructor wants them to behave. Further, students’ ability to continue to develop what they have learned depends to a large extent on their awareness that they have learned it and on their attitudes toward what they have learned: few students are likely to continue to seek opportunities to develop knowledge, skills, attitudes or habits they aren’t aware that they have learned. Similarly, few students are likely to practice knowledge, skills, etc. that they dislike. Thus, it is important that in addition to direct measures, we assess students’ perceptions of and relationships to what they have learned. The Student Assessment of their Learning Gains was built for this kind of assessment. It uses student self-report, so it is an indirect measurement of learning that probes students’ perceptions of what they learned in a course and what helped them learn it. The SALG is a valid and reliable, free assessment tool used by over 16,000 faculty around the world to assess their students’ learning. Funded and recommended for assessing educational innovation by the National Science Foundation, SALG was created in 1997 to support pedagogical innovation in STEM education (originally in chemistry), and it has been continually improved to take advantage of new research on teaching and learning. The SALG was developed specifically to assess durable learning, so it assesses changes in students’ attitudes and habits as well as changes in knowledge/understanding and skills. Coincidentally, the SALG was based on principles that overlap significantly with the NCSS model of citizenship. For example, the first principle of SALG’s design—that you measure teaching effectiveness in terms of the stated goals of the course—derives from deep-seated democratic values, including commitments to fairness/justice, transparency and responsibility. The second principle—that we respect students’ judgments and opinions—is based on a collaborative model of instruction in which teachers and students are partners in the educational process. SALG was designed to promote metacognition about their learning in students, both because it empowers them as learners, and because it helps them learn to take more control of and responsibility for their learning. (For more information about the SALG, see Engaging Assessment: Using the SENCER-SALG to Improve Teaching and Learning (31).) One critical feature of the SALG in this context is that it the questions on the instrument are designed to be changed to match the learning objectives and pedagogy of the specific course in which it is used. Thus, the NCSS learning objectives can easily be inserted directly into a SALG instrument. See Appendix B for a sample SALG with the NCSS standards embedded in it.

Conclusion: Questioning Our Goals I hope that over the preceding pages, I have demonstrated, not just that it is possible to assess citizenship skills deeply and meaningfully, but that it is possible, 167 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

indeed necessary, to teach them as well. The NCSS “believes that preparation for democratic citizenship should be part of the education of every student at every level” (6). Their position statement expands on this dictum in a way that is both inspiring and very SENCER-ish: To accomplish this goal, every student must participate in citizenship education activities each year. These activities should expand civic knowledge, develop participation skills, and support the belief that, in a democracy, the actions of each person make a difference. Throughout the curriculum and at every grade level, students should have opportunities to apply their civic knowledge, skills, and values as they work to solve real problems in their school, the community, our nation, and the world. These opportunities should be part of a well-planned and organized citizenship education program (6) The NCSS even includes a kind of checklist that enumerates the characteristics of an effective citizenship education program. According to their list, such a program ensures that: • • • • • • •

Civic knowledge, skills, and values are taught explicitly and systematically at every grade level. School and classroom management and culture exemplify and demonstrate core democratic values. Citizenship education is integrated throughout and across the curriculum. Students have meaningful opportunities to participate in class and school governance. All students at every grade level are provided with opportunities to participate in the civic life of their school and community. Learning activities extend beyond the school and invite parents and the community to participate and work with students. Students are provided with opportunities to participate in simulations, service-learning projects, conflict resolution programs, and other activities that encourage the application of civic knowledge, skills, and values (6).

In one sense, none of this should trouble us—most SENCER faculty aspire to teach to most of these bullets. The NCSS’s statement stresses the need for integrative, active learning; individual and group empowerment; regular practice to habituate and develop what was learned; meaningful engagement and participation in solving real problems; and integration of academic work into the community. For most SENCER faculty, this will be preaching to the choir. I can almost hear the Hallelujahs. In another sense, however, this should disturb us. SENCER started out in non-major courses, so in most cases, it was a one-and-done experience—the opposite of fully integrated into the curriculum. And we know from both research and experience, that students will not develop the required attitudes and habits after such a limited exposure. In the past few years, some institutions (e.g., Texas 168 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Woman’s University) have SENCER-ized some science majors, so ideally, their students are now getting repeated, scaffolded, rationally developed sequential experiences that help them develop citizenship skills like those the NCSS seeks to nurture. Yet even these programs do not offer the fully integrated experience the NCSS position statement envisions—that would require integration into all classes at all levels. It is a vital and massive step in the right direction, but it isn’t nearly enough. More importantly, we should be worried because teaching citizenship in the way envisioned by the NCSS would require massive restructuring and repurposing not only of all SENCER courses, but also of the major curriculum (wherever SENCER has been integrated into the major). Teaching effective citizenship is difficult, complex, and time-consuming, and the pedagogies it requires differ significantly from STEM pedagogies, even those practiced by SENCER faculty. Teaching and assessing citizenship effectively would consume a large portion (half?) of every SENCER course, leaving much less time for teaching science. Covering the same material would therefore require many more classes, creating time-to-degree problems, vertical integration problems, and accreditation problems. Faculty would also have to be retrained, since few have the knowledge, skills or qualifications required to teach citizenship at the college level. They would need to learn new pedagogies which would inculcate in students the attitudes and habits that are such a vital part of the NCSS definition of effective citizenship. That retraining would take years—it would be the equivalent of earning a completely new master’s degree. I don’t believe that many SENCER faculty have the inclination to make these changes; presumably, they became science teachers because they love science and teaching science, not because they want to teach citizenship. If my suspicion is true, we need to rethink our goals. One option is to remove the reference to citizenship in our mission statement and put civic engagement in its place. This option would preserve nearly all of SENCER’s existing mission, increase its ability to show that it is successful in its mission, eliminate the pesky problem of being unable to show any meaningful correlation between SENCER courses and citizenship skills, and eliminate the need for the gargantuan changes contemplated above. A better option would be to expand the tent. SENCER started out as a scienceeducation reform project, but it has grown to include mathematicians, engineers, and even a few social scientists and humanists. Why not social studies teachers, professional trainers, business teachers, student life professionals, therapists and social activists? Partnering with people who are already experts in teaching the knowledge, skills, attitudes and habits of effective citizenship avoids the need to restructure and retrain mentioned above; it allows us to retain citizenship as a vital part of our mission—and possibly even show that SENCER courses do enhance citizenship skills; it allows us to learn from a new class of experts and to integrate what we learn about citizenship from them into our science courses; and it expands our vision and network. It makes us more interdisciplinary, expands the number of perspectives we can bring to bear on a civic/scientific problem, and integrates more dimensions of the human experience. It makes us better scientists, better teachers, 169 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

better citizens, and better human beings. That’s a community of transformation transforming its own community.

Appendix A: SENCER-SALG Master Template

Your understanding of class content 1. As a result of your work in this class, what GAINS DID YOU MAKE in your UNDERSTANDING of each of the following? not applicable/no gains/a little gain/moderate gain/good gain/great gain 1.1 How to think like a scientist 1.2 How scientific research is carried out 1.3 The scientific issues and processes discussed in science-related stories in the media 1.4 The scientific content that was the subject of this course 1.5 The power of science as a problem-solving-system to affect my life 1.6 The connections between science and civic issues relevant to my community 1.7 The limits of science as a problem-solving system 1.8 Please comment on HOW YOUR UNDERSTANDING OF SCIENCE HAS CHANGED as a result of this class. Increases in your skills 2. As a result of your work in this class, what GAINS DID YOU MAKE in the following SKILLS? not applicable/no gains/a little gain/moderate gain/good gain/great gain 2.1 Distinguishing science from pseudo-science in what I read and hear in the media 2.2 Developing research questions that can be addressed by collecting and evaluating scientific evidence 2.3 Planning and conducting a systematic search for data relevant to a specific question 2.4 Determining what is--and what is not--valid scientific evidence 2.5 Determining what is--and what is not--a valid statistical inference 2.6 Finding scientific journal articles relevant to a specific question using library/internet databases 2.7 Extracting main points from a scientific journal article and develop a coherent summary of them 2.8 Understanding tables and graphs I find in textbooks, websites, magazines and newspapers, and other media 2.9 Understanding mathematical formulas and statistics I find in textbooks, websites, magazines and newspapers, and other media 2.10 Collecting usable scientific data in a laboratory or field setting 170 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

2.11 Working collaboratively with others on a scientific project 2.12 Giving a class presentation based on scientific evidence 2.13 Writing an argument or report using scientific data as evidence 2.14 What skills did you gain from the course that are not listed above? Class impact on your attitudes 3 As a result of your work in this class, what GAINS DID YOU MAKE in the following? not applicable/no gains/a little gain/moderate gain/good gain/great gain 3.1 Interest in science 3.2 Interest in civic issues 3.3 Which civic issues gained greatest importance for you and why?

3.4 Confidence in my ability to understand scientific concepts and procedures 3.5 Confidence that I can do the kind of science we did in this class 3.6 Appreciation of the complexity of the problems we worked on in this class 3.7 Persistence when working on hard problems 3.8 Comfort asking for help from others (professor, TAs, peers) when working on complex problems 3.9 Interest in taking additional science courses after this one 3.10 Interest in majoring in science, technology, engineering or mathematics (STEM) or a related field 3.11 Interest in exploring career opportunities in STEM fields (including teaching them) 3.12 As a result of this course, how has your interest changed in any other activities related to science?

3.13 As a result of this course, how has your interest changed in relation to activities related to civic issues?

Integration of your learning 4. As a result of your work in this class, what GAINS DID YOU MAKE in INTEGRATING the following? not applicable/no gains/a little gain/moderate gain/good gain/great gain 4.1 Discussing science-related issues informally with friends and/or family 4.2 Discussing civic or political issues informally with friends and/or family 4.3 Reading science-related articles not required for class 4.4 Critically analyzing scientific findings reported in the media 171 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

4.5 Taking PUBLIC action related to scientific or civic issues (e.g., blogging, interacting with public officials, working with student or community groups, etc.) 4.6 Connecting what I know about science to what I’ve learned in my other classes 4.7 Applying my knowledge of science and scientific reasoning to civic and/ or social issues 4.8 Using systematic, scientific reasoning to solve problems 4.9 Critically analyzing data and arguments in my daily life 4.10 Voting in local, state and national elections 4.11 Researching and analyzing issues and candidates carefully before I vote 4.12 In what other ways you have integrated your learning in this class into your life and civic engagements?

The Class Overall 5. HOW MUCH did the following aspects of the class HELP YOUR LEARNING? didn’t observe/no help/a little help/moderate help/much help/great help 5.1 The instructional approach taken in this class (i.e., the overall selection of teaching methods, activities, tests, materials, etc.) 5.2 The fact that this was a SENCER class 5.3 Addressing real-world issues 5.4 The interplay between science and civic issues in this class 5.5 Learning how real science is done 5.6 Using scientific methods 5.7 The pace of the class 5.8 What one characteristic of the way this class was taught most helped your learning? Please explain why that was so important to your learning.

Class Activities 6. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? didn’t observe/no help/a little help/moderate help/much help/great help 6.1 Listening to presentations/lectures from course instructor(s) 6.2 Listening to presentations/lectures by guest speakers 6.3 Participating in class discussions 6.4 Listening to class discussions 6.5 Doing group work in class 6.6 Working individually in class 6.7 Using scientific methods 6.8 Gathering data in labs or in the field 172 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

6.9 Analyzing data 6.10 Summarizing findings and results 6.11 What course activity helped you learn the most? Please describe how (or why) it helped you learn.

Assignments, graded activities and tests 7. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? didn’t observe/no help/a little help/moderate help/much help/great help 7.1 Completing written assignments (individual or group) 7.2 Preparing for and making oral presentations (individual or group) 7.3 Participating in group/team projects 7.4 Reviewing material in-class before tests and/or quizzes 7.5 Preparing for and taking tests 7.6 The fit between class content and tests 7.7 The mental stretch required by tests 7.8 The mental stretch required by other assignments 7.9 What elements of the course required the greatest mental stretch?

7.10 The way the grading system helped me understand what I needed to work on 7.11 Receiving feedback on my work 7.12 What was the single aspect of the way your work was evaluated that most helped you learn? Why was that so helpful to you? Class Resources 8. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? didn’t observe/no help/a little help/moderate help/much help/great help 8.1 The syllabus 8.2 The textbook 8.3 Other readings and/or course packets 8.4 Online notes and presentations posted by the professor 8.5 Online materials (excluding the professor’s notes and presentations) 8.6 Visual resources used in class (i.e. PowerPoint, videos, slides, models, demonstrations) 8.7 Lab and/or field work instructions 8.8 What single resource was most helpful to your learning and why?

173 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

The information you were given 9. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? didn’t observe/no help/a little help/moderate help/much help/great help 9.1 Explanation of how the different parts of the course, such as class work, labs, readings, and other assignments related to each other 9.2 Explanation given by instructor of how to learn or study the materials 9.3 Explanation of why the class focused on the topics presented

Support for you as an individual learner 10 HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? didn’t observe/no help/a little help/moderate help/much help/great help 10.1 Interacting with the instructor 10.2 Interacting with TAs or lab assistants 10.3 Working with peers (e.g., on group projects or in study groups) 10.4 Working with a partner

Other Questions 11. Improving the course 11.1 What one thing would you do to improve this course?

Appendix B: A SENCER-SALG for Assessing Citizenship

Your understanding of class content 1. As a result of your work in this class, what GAINS DID YOU MAKE in your UNDERSTANDING of each of the following? not applicable/no gains/a little gain/moderate gain/good gain/great gain 1.1 How to think like a scientist 1.2 How scientific research is carried out 1.3 The scientific issues and processes discussed in science-related stories in the media 1.4 The scientific content that was the subject of this course 1.5 The limits of science as a problem-solving system 1.6 The people, history and traditions that have shaped our local communities. 1.7 The people, history and traditions that have shaped our nation and world. 174 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

1.8 Our nation’s founding documents. 1.9 Our nation’s civic institutions. 1.10 Our nation’s political processes. 1.11 Issues and events that have an impact on people at local and state levels. 1.12 Issues and events that have an impact on people at national and global levels. 1.13 The power of science as a problem-solving-system to affect my life 1.14 The connections between science and civic issues relevant to my community 1.15 Please comment on HOW YOUR UNDERSTANDING OF SCIENCE HAS CHANGED as a result of this class. Increases in your skills 2. As a result of your work in this class, what GAINS DID YOU MAKE in the following SKILLS? not applicable/no gains/a little gain/moderate gain/good gain/great gain 2.1 Distinguishing science from pseudo-science in what I read and hear in the media 2.2 Developing research questions that can be addressed by collecting and evaluating scientific evidence 2.3 Planning and conducting a systematic search for data relevant to a specific question 2.4 Determining what is--and what is not--valid scientific evidence 2.5 Determining what is--and what is not--a valid statistical inference 2.6 Finding scientific journal articles relevant to a specific question using library/internet databases 2.7 Extracting main points from a scientific journal article and develop a coherent summary of them 2.8 Understanding tables and graphs I find in textbooks, websites, magazines and newspapers, and other media 2.9 Understanding mathematical formulas and statistics I find in textbooks, websites, magazines and newspapers, and other media 2.10 Collecting usable scientific data in a laboratory or field setting 2.11 Working collaboratively with others as a member of a group. 2.12 Giving a class presentation based on scientific evidence 2.13 Writing an argument or report using scientific data as evidence 2.14 What skills did you gain from the course that are not listed above? Class impact on your attitudes 3. As a result of your work in this class, what GAINS DID YOU MAKE in the following? not applicable/no gains/a little gain/moderate gain/good gain/great gain 3.1 Curiosity 3.2 Interest in science 3.3 Interest in civic issues 175 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

3.4 Which civic issues gained greatest importance for you and why?

3.5 3.6 3.7 3.8

Confidence in my ability to understand scientific concepts and procedures Confidence that I can do the kind of science we did in this class Interest in taking additional science courses after this one Interest in majoring in science, technology, engineering or mathematics (STEM) or a related field 3.9 Interest in exploring career opportunities in STEM fields (including teaching them) 3.10 As a result of this course, how has your interest changed in any other activities related to science?

3.11 Appreciation of the complexity of the problems we worked on in this class 3.12 Skepticism about sources of information, including willingness to verify their biases and validity 3.13 Critical attitude toward my own beliefs 3.14 Willingness to revise my beliefs in the face of new evidence, facts, consequences, etc. 3.15 Persistence when working on hard problems 3.16 Willingness to find and use a diversity of sources when appropriate 3.17 Willingness to suspend or bracket my own beliefs in pursuit of greater knowledge 3.18 Comfort asking for help from others (professor, TAs, peers) when working on complex problems 3.19 Desire to apply the analytical skills I have learned in this class to my public/civic life 3.20 Desire to apply the analytical skills you have learned in this class to your personal life 3.21 Desire to apply the problem-solving and decision-making skills I have learned in this class to my public/civic life 3.22 Desire to apply the problem-solving and decision-making skills I have learned in this class to my personal life 3.23 Commitment to democratic values 3.24 Desire to participate actively in civic and/or community life 3.25 Accepting responsibility for myself and my actions 3.26 Accepting responsibility for my family and community 3.27 As a result of this course, how has your interest changed in relation to activities related to civic issues? Integration of your learning 4. As a result of your work in this class, what GAINS DID YOU MAKE in INTEGRATING the following? 176 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

not applicable/no gains/a little gain/moderate gain/good gain/great gain 4.1 Discussing science-related issues informally with friends and/or family 4.2 Discussing civic or political issues informally with friends and/or family 4.3 Reading science-related articles not required for class 4.4 Habit of asking meaningful questions, especially when in doubt or in flux 4.5 Preference for seeking to develop informed opinions 4.6 Critically analyzing scientific findings reported in the media 4.7 Habit of checking data and facts against multiple sources before acting on them 4.8 Habit of seeking news and data from multiple sources 4.9 Analyzing information and ideas, especially where old ones are incomplete, weak, compromised or hypocritical 4.10 Using systematic, scientific reasoning to solve problems 4.11 Preference for creative solutions over canned solutions 4.12 Connecting what I know about science to what I’ve learned in my other classes 4.13 Applying my knowledge of science and scientific reasoning to civic and/ or social issues 4.14 Applying the analytical skills I have learned in this class to my public/ civic life 4.15 Applying the analytical skills I have learned in this class to my personal life 4.16 Researching and analyzing issues and candidates carefully before I vote 4.17 Voting in local, state and national elections 4.18 Applying the problem-solving and decision-making skills I have learned in this class to my public/civic life 4.19 Applying the problem-solving and decision-making skills I have learned in this class to my personal life 4.20 Engaging in civic and/or community life 4.21 Using democratic values in daily life 4.22 Taking responsibility for my own ideas and actions 4.23 Taking responsibility for my family and community 4.24 In what other ways you have integrated your learning in this class into your life and civic engagements? The Class Overall 5. HOW MUCH did the following aspects of the class HELP YOUR LEARNING? not applicable/no gains/a little gain/moderate gain/good gain/great gain 5.1 The instructional approach taken in this class (i.e., the overall selection of teaching methods, activities, tests, materials, etc.) 5.2 The fact that this was a SENCER class 5.3 Addressing real-world issues 5.4 The interplay between science and civic issues in this class 5.5 Learning how real science is done 177 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

5.6 Using scientific methods 5.7 The pace of the class 5.8 What one characteristic of the way this class was taught most helped your learning? Please explain why that was so important to your learning. Class Activities 6. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? not applicable/no gains/a little gain/moderate gain/good gain/great gain 6.1 Listening to presentations/lectures from course instructor(s) 6.2 Listening to presentations/lectures by guest speakers 6.3 Participating in class discussions 6.4 Listening to class discussions 6.5 Doing group work in class 6.6 Working individually in class 6.7 Using scientific methods 6.8 Gathering data in labs or in the field 6.9 Analyzing data 6.10 Summarizing findings and results 6.11 What course activity helped you learn the most? Please describe how (or why) it helped you learn. Assignments, graded activities and tests 7. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? not applicable/no gains/a little gain/moderate gain/good gain/great gain 7.1 Completing written assignments (individual or group) 7.2 Preparing for and making oral presentations (individual or group) 7.3 Participating in group/team projects 7.4 Reviewing material in-class before tests and/or quizzes 7.5 The fit between class content and tests 7.6 The mental stretch required by tests 7.7 The mental stretch required by other assignments 7.8 What elements of the course required the greatest mental stretch? 7.9 The way the grading system helped me understand what I needed to work on 7.10 Receiving feedback on my work 7.11 What was the single aspect of the way your work was evaluated that most helped you learn? Why was that so helpful to you? Class Resources 8. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? not applicable/no gains/a little gain/moderate gain/good gain/great gain 178 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

8.1 8.2 8.3 8.4 8.5 8.6

The syllabus The textbook Other readings and/or course packets Online notes and presentations posted by the professor Online materials (excluding the professor’s notes and presentations) Visual resources used in class (i.e. PowerPoint, videos, slides, models, demonstrations) 8.7 Lab and/or field work instructions 8.8 What single resource was most helpful to your learning and why? The information you were given 9. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? not applicable/no gains/a little gain/moderate gain/good gain/great gain 9.1 Explanation of how the different parts of the course, such as class work, labs, readings, and other assignments related to each other 9.2 Explanation given by instructor of how to learn or study the materials 9.3 Explanation of why the class focused on the topics presented Support for you as an individual learner 10. HOW MUCH did each of the following aspects of the class HELP YOUR LEARNING? not applicable/no gains/a little gain/moderate gain/good gain/great gain 10.1 Interacting with the instructor 10.2 Interacting with TAs or lab assistants 10.3 Working with peers (e.g., on group projects or in study groups) 10.4 Working with a partner Other Questions 11. Improving the course 11.1 What one thing would you do to improve this course?

Notes SALG: SALG is a valid, reliable suite of course-assessment instruments developed to support STEM education reform and funded largely by the National Science Foundation, which recommends its use for assessing STEM education research projects. I have been the Primary Investigator responsible for the SALG project for the past 9 years, and as the longest-standing member of SENCER’s assessment leadership team, I have written and contributed to reports assessing SENCER’s progress toward their goals. SENCER Projects: SENCER-ISE applies the SENCER philosophy to informal science education—in museums, National Parks, science centers, and conservation societies. GLISTEN (Great Lakes Innovative Stewardship Through 179 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Education Network applies SENCER’s methodology to environmental and ecological problems in the Great Lakes region. Engaging Mathematics “applies the well-established SENCER method to college-level mathematics courses, with the goal of using civic issues to make math more relevant to students” (32). SCEWestNet focuses on applying SENCER’s approach to create a community of STEM reform practice throughout 17 western states.

References 1. 2. 3. 4.

5.

6.

7. 8. 9. 10. 11.

12. 13.

http://sencer.net/our-mission/, “Our Mission” section (accessed April 5, 2018). Burns, W. D. “But You Needed Me”: Reflections on the Premises, Purposes, Lessons Learned and Ethos of SENCER, Part 1. SECIJ. 2011, 3 (1), 5–11. http://sencer.net/our-mission/, “Our Aims” section (accessed April 5, 2018). Carroll, S. B. SALG Results Show SENCER Faculty Achieve in Raising Higher Order Learning Gains. SENCER eNews, March 28, 2012. http://ncsce.net/salg-results-show-sencer-faculty-achieve-in-raising-higherorder-learning-gains/ (accessed April 5, 2018). Kezar, A.; Gehrke, S. Communities of Transformation and their Work Scaling STEM Reform; Pullias Center for Higher Education, Rossier School of Education: Los Angeles, CA, 2015. Creating Effective Citizens: A Position Statement of National Council for the Social Studies. http://www.socialstudies.org/sites/default/files/publications/ se/6505/650511.html (accessed April 5, 2018). Carroll, S. B. The Epistemological Gap Between Teaching and Learning. National Teaching and Learning Forum 2017, 26 (2), 1–4. Arum, R.; Roksa, J. Academically Adrift: Limited Learning on College Campuses; University of Chicago Press: Chicago, IL, 2011. Pearlman, S.; Carillo, D. The Critical Thinking Initiative: Faculty Edition; The Critical Thinking Initiative, LLC.: West Hartford, CT, 2017; p 11. Vosoughi, S.; Roy, D.; Aral, S. The Spread of True and False News Online. Science 2018, 359, 1146–1151. News Use Across Social Media Platforms 2017. Pew Research Center. http://www.journalism.org/2017/09/07/news-use-across-social-mediaplatforms-2017/?utm_source=AdaptiveMailer&utm_medium=email&utm_ campaign=Email%20Campaign%20for%20Social%20Media%20and%20 News&org=982&lvl=100&ite=1678&lea=347325&ctr=0&par=1&trk= (accessed April 12, 2018). Bakshy, E.; Messing, S.; Adamic, L. Exposure to Ideologically Diverse News and Opinion on Facebook. Science 2015, 348, 1130–1132. Grimes, D. R. Echo Chambers are Dangerous—We Must Try to Break Free of Our Online Bubbles. https://www.theguardian.com/science/blog/2017/ dec/04/echo-chambers-are-dangerous-we-must-try-to-break-free-of-ouronline-bubbles (accessed April 12, 2018).

180 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

14. Bransford, J. D.; Brown, A. L.; Cocking, R. R. How People Learn; Brain, Mind, Experience and School (Expanded Edition); National Academy Press: Washington, DC, 2000. 15. Perry, W. G. Forms of Intellectual and Ethical Development in the College Years: A Scheme; Jossey-Bass: San Francisco, CA, 1999. 16. Shepard, L.; Wallis, N.; Maguire, C. SENCER and the Dual Poster Concept: Translating Science into Common Language. In Science Education and Civic Engagement: The Next Level; Sheardy, R. D., Burns, W. D, Eds.; ACS Symposium Series 1121; American Chemical Society: Washington, DC, 2012; pp 145−153. 17. Harrison, A. F. Teaching and Democratic Values in Higher Education. Education and Culture 1994, IX (2), 28–34. 18. Syverson, M. A. Social Justice and Evidence-Based Assessment with the Learning Record. Forum on Public Policy Online 2009, 2009 (1). 19. McArthur, Jan. Assessment for social justice: the role of assessment in achieving social justice. Assessment & Evaluation in Higher Education 2015, 41 (7), 967–981. 20. McArthur, J. Assessment for Social Justice: Perspectives and Practices within Higher Education; Bloomsbury Publishing: London, 2018. 21. Holsinger, K. Teaching Justice: Solving Social Justice Problems through University Education; Solving Social Problems; Routledge: London, 2016. 22. Sheffield, C. C.; Duplass, J. Creating Effective Citizens: Unique Opportunities for Gifted Education through the Social Studies. Gifted Education International 2009, 25, 237–245. 23. Carroll, S. B. Meta-Learning: Teaching Students How to Learn Builds Success for Life. National Teaching and Learning Forum 2017, 26 (4), 1–4. 24. McGuire, S. Y. Teach Students How to Learn: Strategies You Can Incorporate Into Any Course to Improve Student Metacognition, Study Skills and Motivation; Stylus: Sterling, VA, 2015. 25. Doyle, T.; Zakrajsek, T. The New Science of Learning; Stylus: Sterling, VA, 2013. 26. Nilson, L. B. Creating Self-Regulated Learners: Strategies to Strengthen Students’ Self-Awareness and Learning Skills; Stylus: Sterling, VA, 2013. 27. Baxter-Magolda, M. B. Creating Contexts for Learning and Self-Authorship: Constructive-Developmental Pedagogy; Vanderbilt University Press: Nashville, TN, 1999. 28. Baxter-Magolda, M. B., Ed. Teaching to Promote Intellectual and Personal Maturity: Incorporating Students’ Worldviews and Identities into the Learning Process; New Directions in Teaching and Learning 82; Jossey-Bass: San Francisco, CA, 2000. 29. Baxter-Magolda, M. B.; King, P. M. Learning Partnerships: Theory and Models of Practice to Educate for Self-Authorship; Stylus: Sterling, VA, 2004. 30. Baxter-Magolda, M. B.; King, P. M. Assessing Meaning Making and SelfAuthorship: Theory, Research and Application; ASHE Higher Education Report 38:3; Jossey-Bass: San Francisco, CA, 2012. 181 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

31. Carroll, S. B. Engaging Assessment: Using the SENCER-SALG to Improve Teaching and Learning. In Science Education and Civic Engagement: The SENCER Approach; Sheardy, R. D., Ed.; ACS Symposium Series 1037; American Chemical Society: Washington, DC, 2010. 32. National Center for Science & Civic Engagement (NCSCE) Brochure. http:// sencer.net/wp-content/uploads/2016/09/2015-NCSCE-brochurefinal.pdf (accessed April 5, 2018).

182 Maguire and Sheardy; Citizens First! Democracy, Social Responsibility and Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.