Downloaded via UNIV OF SYDNEY on July 16, 2018 at 10:39:48 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.
Chapter 5
Metacognitive Foundations in Higher Education Chemistry F. Arslantas,1 E. Wood,1 and S. MacNeil*,2 1Department
of Psychology, Wilfrid Laurier University, Waterloo, Ontario, Canada N2L 3C5 2Department of Chemistry & Biochemistry, Wilfrid Laurier University, Waterloo, Ontario, Canada N2L 3C5 *E-mail:
[email protected].
For most students the acquisition of the skills and knowledge required in higher education chemistry contexts is an active and challenging task. Designing curriculum to best support learners is also challenging for instructors. A growing body of literature examines the role and impact of instructional interventions designed to encourage awareness and development of metacognitive skills as a learning support in chemistry courses. Metacognition is not the same as intelligence or domain knowledge. Instead, it refers to the cognitive underpinnings that support a student’s knowledge and control of learning. Despite the significant attention metacognition has received in the psychological and educational literature, it remains a term with which few university students and instructors are familiar. Thus, a growing body of literature has evolved to understand metacognition in chemistry learning contexts. This chapter highlights original research in which metacognition and chemistry were both a major focus, summarizing how metacognition has been taught and assessed and noting cases in which improvements in metacognition and/or performance have been reported. Through this collection of articles, a positive relationship between metacognition and performance is established. However, gaps in the extant research are uncovered and important future directions are highlighted and discussed. In particular, there is a need for (i) more explicit teaching of metacognition, (ii) increased use of multiple and © 2018 American Chemical Society Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
concurrent methods for assessment of metacognition, (iii) more implementations of interventions in general as well as longitudinal studies and those examining senior years of chemistry study, (iv) greater efforts to establish the link between metacognition and performance, and (v) greater efforts to bring results of this research into the higher education chemistry classroom.
Introduction Learning is an active and challenging task for both students and instructors. For students taking chemistry in higher education contexts, learning expectations range from simple acquisition of new terminology, to comprehension and application of new concepts and methodological skills. Equally challenging is the task for instructors who must design, plan and execute lessons and experiences for students of diverse skills, knowledge and ability. The fields of psychology and education offer insights regarding how to make the task of learning easier for students. In particular, significant research in these two domains identifies the importance of supporting students’ development and utilization of metacognitive skills as a means for maximizing learning, especially in higher education contexts (1–5). Educators can play an important role by providing students with instructional opportunities that develop and scaffold metacognitive awareness and skills (6, 7). The following chapter first draws upon theoretical and applied work from psychology and education to introduce the importance of metacognitive training as a means to facilitate student learning in higher education contexts. The chapter then identifies and summarizes the key literature that addresses metacognitive training and the impact of metacognition in the domain of chemistry.
What Is Metacognition and Why Is It Important? Metacognition is a higher-order cognitive skill that encompasses all the mental actions and processes involved when we are thinking about thinking (7, 8). In other words, we use metacognitive skills to understand how to approach the learning task, e.g., assigning R/S configurations to chirality centers in a molecule. This includes considering the optimal environment in which to learn, e.g., “Am I paying attention?”, assessing current knowledge/skills related to the task, e.g., “Do I know the term ‘chirality’?”, “What can I remember about it?”, and drawing upon strategies to facilitate learning, e.g., using imagery of a left and right hand to understand the concept of nonsuperposable mirror images. Learners must also understand the reasons and processes behind the performance of the task, e.g., “How do I identify a chirality center?”, “What are the Cahn-Ingold-Prelog rules?”, “Why is this important?”. The scope of metacognition is vast. It permits us to assess, coordinate and apply our cognitions. Interestingly, this aspect of cognition develops later in childhood and is not necessarily spontaneously elicited 58 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
even by advanced learners, such as university level students (1–4). Thus there is a need to teach and encourage metacognition skills to maximize learning. Metacognition is typically described in terms of two overarching components: knowledge of cognition and regulation of cognition (5, 7). Knowledge of cognition refers to what you personally know about how you think and what you understand about how thinking occurs in people in general. Knowledge of cognition consists of three secondary subcomponents: declarative, procedural, and conditional knowledge. Declarative knowledge (knowing ‘what’) involves knowing about one’s own learning and factors affecting it. Procedural knowledge (knowing ‘how’) is knowing the methods or procedures to address tasks, e.g., strategies like chunking, categorization, elaboration, and imagery. Conditional knowledge (knowing ‘when’) refers to the ability to use the correct knowledge and strategies in the correct context. Regulation of cognition refers to the ability to exert attention and control to regulate cognition (5, 7). Regulation of cognition includes three secondary subcomponents or skills: planning, monitoring, and evaluation. Planning occurs at the onset of a task during which the learner foresees the demands of the task and decides which strategies, e.g., summarize main points of text, and resources, e.g., attention and textbook, are most suitable and necessary for the task at hand. Monitoring is done while performing the task. It involves regularly checking one’s understanding of the task and whether one’s performance is meeting the requirements of the task. Evaluating can occur during or at the end of performing a task. Evaluating involves assessing the quality of one’s performance in terms of goal achievement. Learning and metacognition share a reciprocal relationship. Specifically, effective and efficient learning requires metacognitive skills. However, the act of learning, which involves planning, execution of strategies, and monitoring provides an opportunity to practice and enhance metacognitive skills (1–7). Using metacognitive skills to learn can benefit processing speed, i.e., the rate at which learners attend to, perceive, understand, change, use and store information; automaticity, i.e., effortless and efficient processing, facilitating easy access to acquired knowledge (9); building a repertoire of strategies, effective use of strategies, allocation of resources, learning, and performance. Individuals high in metacognition not only have large and varied strategic repertoires but they execute strategies fluently and automatically as required by the situation (5). Greater proficiency in each of these skills positively impacts learning. How Do You Teach Metacognition? The term ‘metacognition’ is not commonly known among students, even at the university level (7). Thus, the first step in teaching metacognitive skills involves introducing students to the concept of metacognition as well as its importance for learning and performance. Metacognitive awareness instruction should include: explicit instruction, modelling, integration of metacognitive skills with course content, and opportunities for practice and reflection (5, 10). Explicit instruction means providing students with a clear definition of metacognition, how it develops, how it can be enhanced, and why it is important in an academic context. This should be introduced early in a course. Introduction 59 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
to the concept of metacognition should accompany domain-specific, e.g., terminology, and domain-general, e.g., critical thinking skills throughout a course. Instructors can scaffold metacognitive awareness and skills by identifying and modelling use of metacognition while teaching regular course content. Specifically, instructors should verbalize metacognitive behaviors and skills to demonstrate completion of a task. For example, “What is a chirality center? It is a carbon, or other atom, bonded to four different substituents. At this particular carbon, how do I prioritize the substituents? By applying the Cahn-Ingold-Prelog rules. Am I viewing the chirality center from the correct angle, that is, is the lowest priority substituent pointing away from me? If not, what must I do? I must redraw the structure, mentally manipulate it, build a molecular model…etc.” As learners gain more knowledge and familiarity with the concept of metacognition, they should be provided with frequent opportunities to practice using metacognitive skills, ideally through the use of ‘just in time teaching’. For example, asking students to answer introductory, exploratory, thought and discussion provoking questions prior to class promotes reflection regarding what is and is not known as well as monitoring of comprehension and performance as students are learning (6, 10). Other instructional supports include the use of regulatory checklists, which use questions to assist the development of planning, and summarizing strategies. These checklists prompt students to evaluate their knowledge and the effectiveness and efficiency of the strategies used (5). A stop, think, and act method is particularly useful for promoting metacognitive awareness (5). The first step in this process is to gather or recollect information about one’s own learning process, e.g., general strategy use behaviors, and one’s cognitive resources, e.g., knowing how, when, and why to use a repertoire of strategies, before beginning a task. The next step is to select task appropriate strategies, e.g., skim, diagrams, etc., and make a plan (time, resources) to meet the demands of the task. The third step is comprehension and performance monitoring during task performance, e.g., “Do I have a clear understanding of what I am doing?”. The final step is to assess the learning and performance outcomes of the completed task, e.g., “Have I reached my goal?”. Repetition of this process provides the practice needed for learners to employ metacognitive skills more automatically and to generalize metacognitive awareness and skills to other tasks and domains. How Is Metacognition Assessed? Assessments of metacognition include three commonly used methods: (1) self-report measures; (2) pre- and post-dictions; and (3) concurrent instruments including think-aloud protocols (11) and on-line tracking (12). Self-report measures assess students’ overall metacognitive skills based on student reports of type and frequency of strategies used and are often employed as pre-/post-tests to measure changes in reported metacognitive strategy use. Self-report measures include multiple choice, e.g., Metacognitive Awareness of Reading Strategies Inventory (MARSI) (13), or true/false, e.g., Metacognitive Awareness Inventory (MAI) (5) questionnaires, or open-ended questions, e.g., Knowledge of Developing Cognitive Knowledge (KDCK) (8, 14). 60 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
Pre- and post-dictions include ease-of-learning (EOL) judgments, judgments of learning (JOL), and feeling-of-knowing (FOK) judgments (15). Pre- and postdiction measures involve asking participants to predict their performance before or during a task or to estimate their performance following completion of a task. The difference between performance pre-/postdictions and actual performance is the index of metacognitive awareness. Beware students often are overly optimistic when they provide this evaluation. This is referred to as the Dunning-Kruger effect (16). Think-aloud protocols involve the participants reciting their thoughts while or soon after performing a task. It is also possible to record participants while performing the given task and later ask them to watch the video of themselves and verbalize their thoughts about what they were doing. Interactive MultiMedia Exercises (IMMEX) is a software program that tracks participants’ actions online while they are performing an online task by keeping a log of participants’ item selections, selection order, and selection times. Information from think-aloud protocols or IMMEX can then be collated and analyzed in a dataset to assess use of metacognitive strategies during task completion. Given that metacognitive skills can be discipline- and, in fact, task specific (17), concurrent instruments for assessing metacognition are generally preferred to asynchronous methods that tend to assess more general metacognitive skills.
Metacognition in Higher Education Science Although metacognition has been studied extensively in psychological research, a key contribution for the present chapter is to summarize the current state of metacognition studies in higher education chemistry courses. How has metacognition been taught? How has metacognition been assessed? What interventions have been implemented? What claims have been made? Which courses have been examined and where are the gaps in knowledge that highlight opportunities for further research? For context, consider the number of reported studies on metacognition in higher education science over the last 6 decades. A search of Google Scholar using the key words “metacognition AND higher education AND science” provided the results depicted in Figure 1. As illustrated in Figure 1, there has been a growing interest in the role of metacognition in higher education science, including a review covering work from 2000-2012 (18). But questions remain---what has been done in chemistry and what have we learned? To address these questions, we conducted a review of existing research. The remainder of this chapter summarizes our findings.
61 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
Figure 1. Results for Google Scholar search of “metacognition AND higher education AND science” for the years 1960-present.
Summary of Methods Used To Locate Relevant Research Three databases were used to search for metacognition studies in higher education chemistry courses reported up to April 11, 2017: ERIC, PsychINFO and SciFinder. ERIC (Educational Resources Information Center) is a database that provides access to publications in the field of education. PsychINFO is a database that can be used to search for works published since 1806 in psychology and other disciplines, e.g., medicine and law. SciFinder is a database that can be used to search for chemistry related articles published since 1967. The initial search command entered into all four databases was “metacognition” without any limits on location of appearance, e.g., abstract, document text, document title, etc, or date range. Next, a search within the generated results lists was conducted for “chemistry”. The results were then restricted to peer-reviewed work only (because there was not a peer-reviewed option on SciFinder, the results were limited to book, journal, or review), then to English only. Duplicate items identified across the four databases were eliminated. In total, there were 108 unique results. Figure 2 illustrates that there has been a growing interest in the role of metacognition in higher education chemistry, particularly since 2010. The results of the search were then further refined within metacognition research studies in higher education chemistry using the following inclusion criteria: • • • •
metacognition the/a focus, chemistry the/a focus, higher education, student sample, 62
Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
• •
research study, access to full article.
Of the 108 unique results, only 31 publications met the inclusion criteria (19–49). See Table 1 in the Appendix for details regarding study exclusions.
Figure 2. Publications by year for “metacognition AND chemistry” searches of ERIC, PsychInfo and SciFinder restricted to peer-reviewed papers (book, journal or review for SciFinder) written in English.
Discussion Explicit Instruction in Metacognition As previously noted, metacognitive awareness instruction should include: explicit instruction, modelling and integration of metacognitive skills with course content, and opportunities for practice and reflection (5, 10). Of the 31 articles meeting the inclusion criteria, only 2 provided details for explicit metacognitive awareness instruction. In 2 related studies, Cook and colleagues (38, 43) devoted entire lectures to information about learning strategies, introducing students to metacognition and offering students a set of metacognitive learning tools to replace or supplement ineffective strategies used in high school. For example, students were encouraged to use a study cycle which encompasses an iterative process of previewing material before class, attending class, reviewing material immediately after class, then studying and assessing learning. In 3 additional articles, students were trained in particular metacognitive strategies, e.g., flow diagrams and the competency tripod model (21) or student outcomes, concept mapping, and study strategies diaries (40). The competency tripod model is an analogy linking declarative knowledge, communicative competence and procedural knowledge. However, there was no mention of explicit instruction 63 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
regarding metacognition in general in any of these three articles. Although attempts to create social environments to support reflective discourse (31) and reinforce awareness of metacognition near the end of a course (28) were noted, the remaining articles did not provide details on explicit attempts to introduce students to metacognition and explain its importance to learning. Assessing Metacognition Metacognition is typically assessed using (i) self-report measures, i.e., student surveys, interviews and written reflections, for which students report the type and frequency, or investigators score for the presence and quality, of metacognitive strategies employed; (ii) pre- or postdictions of learning or knowing, where the accuracy of a student’s pre- or postdiction is believed to be related to their metacognitive skills; and (iii) assessments concurrent with problem-solving activities, e.g., think-aloud protocols or on-line tracking, where strategies employed are documented in real time during a task. Among the 31 articles meeting criteria, 27 assessed metacognition in one or more ways (see Table 2 in the Appendix for a summary). The most common method employed to assess metacognition was self-report measures. Nineteen articles utilized self-report measures, with 15 of these relying solely on self-report measures as an assessment of metacognition. Only 7 of the 31 articles used pre- and postdiction accuracy as a measure of metacognitive skills, with 6 of these 7 studies relying solely on these measures for analysis. Likewise, only 6 of the 31 articles utilized concurrent methods of direct observation, including think-aloud protocols, and on-line tracking via IMMEX. However, only 2 of these studies relied solely on these methods for analysis. Interestingly, only 4 articles made use of multiple modes of assessment. The remaining 4 articles make no mention of an attempt to assess metacognition as part of the study. Interventions To Improve Metacognition As noted above, to improve students current metacognitive skills and maximize use within a course, students, even at the university level, need to be introduced to the concept early in a course and they must be exposed to teaching methods and/or learning strategies that promote metacognition with ample opportunities to practice metacognition skills and behaviors. For this reason, a significant portion of the research highlighted in this chapter has focused on the use of interventions to improve students’ metacognition. Surprisingly, only 17 of the 31 studies reported interventions aimed to prompt and improve students’ metacognitive behaviors (see Table 2 in the Appendix for a summary). Although the interventions varied considerably across studies, the types of interventions could be grouped according to four broad categories: explicit instruction, comparing teaching methods, introducing student activities and using prompts. Consistent with the need to provide explicit instruction and learning strategies, two of the studies used lecture-based information sessions to introduce the concept of metacognition accompanied by provision of metacognitive learning 64 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
tools, e.g., a study cycle (38, 43). A third study examined students’ confidence judgments prior to and after learning about one particular chemistry topic (stoichiometry) to determine how exposure to the information changes students’ confidence judgements (42). Seven articles described interventions based on comparison or implementation of various teaching methods in the classroom and in lab contexts. Of these, one study compared cooperative learning and classroom discussions to typical lectures for their effect on general chemistry students’ metacognitive processes (19). Although most studies involved typical university students, one additional study involved student teachers who were exposed to problem-based learning to assess its effects on metacognitive awareness (33). The majority of these studies examined the impact of teaching interventions in lab contexts. Five studies focused on the effects of cooperative/collaborative project- and problem-based labs (26, 28, 31) and the level of inquiry in labs (37, 45) on students’ metacognition. Four articles describe interventions that use a variety of student activities to improve students’ metacognitive skills. Activities varied across studies but included problem manipulation, in which students actively assessed the skills and knowledge used to answer a chemical problem and then manipulated the problem to create a new one (30); journal writing, where students described their understanding of a topic, the development of that understanding, and how the topic connected to their lives (32); and weeklong daily diaries which were coded for 14 self-regulated learning strategies (40). In the lab, the competency tripod model was used in conjunction with flow diagrams to effect changes in students’ metacognitive practices (21). Finally, four articles used interventions based on metacognitive prompts and feedback manipulation. Of these studies, one used reflection forms to elicit expectations and beliefs about the course, experiments and related scientific topics, accompanied by questions querying the implications of lab experiments to daily life (34). Another study varied three forms of weekly quiz feedback to examine the impact on students’ self-reported use of metacognitive strategies (39): mastery-approach, performance-approach, and a combined mastery/performance-approach. The mastery-approach focused on competence and task mastery while the performance-approach focused on aptitude and favorable judgments. One study asked students to list the top three reasons for their success, or lack thereof, on a recent test (43). One study used scaffolding questions to assist students in monitoring, diagnosing and, if possible, repairing areas of difficulty in an on-line homework environment that provided students with metacognitive data but did not otherwise instruct students on how to use that data (49). Claims of Improved Metacognition An instructor who is teaching students about metacognition, introducing students to teaching methods or learning strategies that promote metacognition, or simply measuring students’ metacognition is hoping to observe a change. Improvement in students’ metacognition in response to an instructional 65 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
intervention or perhaps just over time as a result of regular course instruction is the goal. This may be observed through changes in students’ responses to self-report questionnaires, changes in students’ pre-/postdiction accuracies in judging preparation for or performance in completing a task or test, or changes in the use of metacognitive strategies observed or measured concurrently during completion of a task. Results were mixed with respect to improvements in metacognition for the 31 articles included in this chapter (see Table 2 in the Appendix for a summary). Interestingly, despite metacognition being identified as a main focus in each of the articles, almost half, i.e., 15 out of 31, of them did not provide any information regarding changes in metacognition over time or as a result of a specific intervention. For 8 of the 15 articles, metacognition was measured on one day only and no interventions were implemented to seek an improvement; for 5 of the 15, no formal measurement of metacognition was utilized; for 2 of the 15 articles, although metacognition was measured across a term, changes in metacognition were not addressed. Among the remaining 16 articles, those that did not describe an explicit attempt to teach metacognition or to provide interventions to improve metacognition (20, 23, 29, 41, 48) generally reported no improvements in metacognition. One exception reported an overall increase in general chemistry students’ self-reported use of metacognitive strategies across a term, but this may have demonstrated a behavioral change or merely a change in perceptions prompted by the surveys themselves (20). In contrast, one article, focused on design and validation of an instrument to assess metacognitive skillfulness in chemistry, reported that scores were not significantly different for two administrations of the instrument 13 weeks apart (23). All articles reporting indirect metacognitive measures based on test grade pre- and postdiction accuracies (29, 48) and estimates of ability in chemistry (41) consistently showed persistent, and sometimes worsening, Dunning-Kruger effects which imply no improvements in students’ metacognition over 1-2 terms of instruction. The Dunning-Kruger effect is demonstrated in an academic environment when ill-prepared or low-performing students overestimate their level of preparation or performance. Another article reported that even after direct instruction in a particular topic (stoichiometry), students’ inaccuracies in estimates of their own ability in chemistry persisted, with the majority of students over-estimating their ability (42). Failure to see improvement across these studies suggests that these types of measures on their own are not sufficient to enhance students’ metacognition. Instead, employing these types of interventions may require additional support such as explicit instruction to make clear the connection between the measures and metacognition skills such as monitoring. Alternatively, students may require interventions that directly promote reflection, monitoring or other metacognitive strategies to encourage metacognitive behaviors. The remaining 10 articles each made claims, albeit with varying degrees of substantiation, of improvements in students’ metacognition owing to a specific intervention (see Table 2 in the Appendix for a summary). Four of these articles described improved metacognition of an experimental treatment compared to a control. Another 3 articles report improvements in students’ metacognition from 66 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
pre- to post-intervention but use no control for comparison. The remaining 3 articles make claims that are speculative at best.
Articles with Controls Within the extant literature, exposure to metacognitive prompts has been shown to have positive effects on students’ self-reported use of metacognitive strategies. One article in the present review demonstrated that pre-service science teachers in a first-year chemistry lab course outperformed a control group on self-reported metacognitive learning strategies after completing reflection forms, analyzing and explaining daily life implications of lab experiments and answering metacognitive questions throughout the instructional process (34). A second article showed that prompting students on weekly quizzes with different types of feedback designed to induce different types of achievement goals – mastery (focus on competence and task mastery) versus performance (focus on aptitude and favorable judgments) versus combined mastery/performance – resulted in lower levels of self-reported metacognitive strategy use among control students and those receiving performance feedback, but no significant change for students receiving mastery or combined mastery/performance feedback (39). Thus, differential feedback appears to be related to losses in metacognition but the different forms of feedback did not enhance metacognition. Four articles utilizing control groups claimed improved metacognition in students exposed to various teaching methods in the lab. Two articles revealed, through interviews and analysis of students’ lab reports, that guided lab experiments, requiring more inquiry than structured or verification labs, resulted in an increased focus on metacognitive knowledge and led to increases in self-reported use of metacognitive strategies (37, 45). Interestingly, two other articles, combining qualitative and quantitative data from the same study, made the claim that decreases in self-reported use of metacognition for students subjected to a collaborative work session in the lab suggested improved awareness of metacognition compared to students not subjected to this treatment. However, concurrent measurement of regulatory metacognitive skills via IMMEX showed no differences between the groups (26, 28).
Articles with No Controls In one study, general chemistry students exposed to a series of metacognitive scaffolds including an in-class test review with a metacognitive activity, a separate lecture devoted entirely to introduction of metacognition and metacognitive strategies, and frequent reminders to metacognitively monitor their learning strategies before each test, self-reported increased use of effective learning strategies on a 12-item Effective Learning Strategies Survey. However, significant increases only occurred for 3 of the 12 items on the survey, and validity and reliability of the survey were not confirmed (43). In another study, teacher education students enrolled in a general chemistry course were subjected to six 67 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
problem-based learning (PBL) scenarios limited to solution concepts in chemistry. Self-reports pre- and post-intervention revealed that PBL was effective at improving metacognitive skills but only for students with no science background (33). A third article demonstrated improvements in metacognitive strategy use, assessed concurrently using IMMEX, for general chemistry students performing project-based laboratory experiments cooperatively in small groups (31). For the final 3 articles, improvements in metacognition based on an intervention were implied but supporting data was limited. In one article, general chemistry students were exposed to four different teaching methods – cooperative learning, class discussions, concept maps, and lectures – then asked to complete a survey regarding intended purposes of each teaching method. Although student responses revealed no differences among methods in aiding metacognitive processes, the authors claimed that their data support the idea that multiple modes of learning foster metacognitive skills (19). In another article, students were taught using a problem manipulation model of chemistry instruction, in which students were required to change or manipulate a problem in order to test their own understanding of the underlying concept. Although no measures of metacognition were used in this study, the authors suggested that the problem manipulation model provided an opportunity for students to practice and develop metacognitive skills (30). Finally, in an article describing flow diagrams and the competency tripod model as sources for enabling students’ metacognition in the chemistry laboratory, a tentative claim of improved metacognition was made. Specifically, the authors state that “It is not possible to establish directly if the competency tripod model was responsible for enabling metacognition in students but like dropping a pebble into a pond, its introduction certainly provided ripples which could be identified as metacognition (21).” Claims of Improved Performance The ultimate goal in attempting to improve students’ metacognitive skills is to improve their overall academic performance. Of the 31 articles meeting inclusion criteria, 5 made no mention of a link between metacognition and performance, 5 described studies in which no improvements in performance were observed despite varying claims of improved metacognition, 19 made claims of a positive link between metacognition and performance, and 2 presented mixed results (see Table 2 in the Appendix for a summary).
No Improvements in Performance Among the 5 articles where no improvements in performance were observed despite varying claims of improved metacognition, interventions included comparisons among teaching methods, using prompts and introducing student activities. Specifically, students provided a problem manipulation model of chemistry instruction, touted to offer an opportunity for students to practice and develop metacognitive skills, showed no improvements in final exam scores compared to students who had not been subjected to the problem manipulation 68 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
model (30). Similarly, guided lab experiments which required more inquiry than structured or verification labs led to increases in students’ self-reported use of metacognitive strategies but did not result in an increase in reflections on conceptual knowledge (37) or increased scores on a standardized final exam from the American Chemical Society (45) when compared to outcomes in structured or verification labs. In a test feedback manipulation study, students provided feedback designed to induce a mastery goal orientation also reported higher use of metacognitive strategies. However, these students achieved lower course grades than students induced to adopt a performance goal orientation (39). Finally, in an article that provided students with various activities meant to improve metacognitive skills – daily diaries, problem sets and concept maps –, none of the activities showed a link to any performance outcomes in the course (40).
Improvements in Performance Of the 19 articles that reported a positive relationship between metacognition and performance, 16 could be grouped into one of three categories. These categories are organized not by intervention but by the common measures that are used to link metacognition to performance: (1) pre- and postdiction accuracy measures compared to course grades; (2) concurrent metacognitive strategy use compared to ability, via IMMEX; and (3) self-reported metacognitive skills compared to various performance measures, mostly grades. While ill-prepared or low-performing students overestimate their level of preparation or performance consistent with the Dunning-Kruger effect, well-prepared or high-performing students are much more accurate in their self-assessments and often underestimate their level of preparation or performance. The discrepancy between perception and reality for the ill-prepared or low achieving students is suggested to result from inferior metacognitive skills. Six articles meeting inclusion criteria indirectly link superior metacognition to superior performance via persistent Dunning-Kruger effects in students’ self-evaluation of preparation or performance (24, 29, 41, 42, 47, 48). This collection of articles implies that metacognitive ability, reflected in accuracy of self-evaluation, is a requirement for academic success. However, one study involving question-level predictions of knowledge for upcoming tests also showed mixed findings. Specifically, the general Dunning-Kruger effect was observed when students were grouped into thirds based on overall final exam scores, but an interaction effect between a student’s question-level prediction and their ability level was not significant, suggesting no difference in ability to judge right/wrong answers between students performing well and those not performing well on the final exam (46). This result appears to refute the notion that metacognitive ability is necessary for academic success but more likely reflects the complicated nature of metacognition and the difficulties in accurately measuring it. Interactive MultiMedia Exercises (IMMEX) is a software program that tracks participants’ actions online while they are performing an online task. Two assessment parameters are produced: the strategy state, which is related to the metacognitive quality of the solution path employed, and the ability 69 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
which is a measure of the problem difficulty students can properly handle (22). Three articles employing IMMEX as a concurrent measure of metacognition consistently showed that students scoring higher in strategy state also scored higher in ability (22, 28, 31). However, in one of these studies, mixed results were found. Although the correlation between strategy state and ability was observed in general, the treatment group, exposed to a collaborative activity in the lab, showed no difference in strategy state and actually displayed a decrease in self-reported metacognitive strategies compared to the control group (28). As mentioned previously, the most common method employed to assess metacognition was self-report measures in which students are asked to report on type and frequency of strategies used. Eight articles meeting inclusion criteria demonstrated a positive relationship between students’ self-reports of metacognitive strategy use and academic performance. Measures most often included grades (20, 23, 32, 36, 44), but also more general performance measures such as self-efficacy (concurrent with reduced anxiety) (27), science process skills and conceptual knowledge (34), or critical thinking (35). Among the remaining three articles, two identified the benefits to students’ performance of providing explicit instruction through attending a lecture dedicated to the introduction of metacognition and metacognitive strategies. In one case, final course grades were higher for attendees than non-attendees despite the two groups being similar in demographics, prior learning and test 1 grades. However, treatment explained only 10% of the difference in total points and students’ metacognition was not measured (38). That is, the link between metacognition and performance was merely implied. In the other case, the authors point to improvements in test grades across the term, compared to previous years in which no information session was provided, and increased improvements for students exposed to the information session in two consecutive terms as benefits of metacognitive training (43). The final article, featuring the on-line homework platform LearnSmart, illustrated the importance of metacognitive training. LearnSmart includes metacognitive features but results showed that students who used the metacognitive features without scaffolded support did not achieve the learning gains observed for students receiving scaffolding questions that supported the use of the platform’s metacognitive features. The observed effect was modest and students’ metacognition was not measured (49).
Conclusions In its 2012 report on discipline-based education research (DBER), the US National Research Council identified metacognition as a topic vital to learning science and engineering that warranted further study (50). The increase of research articles on metacognition in higher education science in recent years demonstrates that the call is being answered. However, perhaps owing to a lack of awareness of the importance of metacognition, or alternatively, the belief that it is not the responsibility of science instructors to foster metacognitive abilities, insufficient research has been done to fully address issues related to the role of metacognition in chemistry (51). The collection of articles highlighted in this chapter shows 70 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
a diverse approach to examining metacognition in higher education chemistry domains. Although some studies found the expected positive relationship between metacognitive skills and performance in higher education chemistry, others did not. However, review of the current research identifies directions to take going forward. More research is required to improve the teaching and assessment of metacognition, to develop effective interventions to improve metacognition, to unequivocally establish the link between metacognition and performance in higher education chemistry, and to carry the results of this research into the classroom. The need for the teaching of metacognitive skills is one of the main implications of research on the teaching and learning of science that has emerged during the past three decades (52). Given this statement, it is very surprising to see how few (just 2) of the studies highlighted in this chapter included explicit instruction on metacognition. More research is needed to establish best practices in teaching the concept of metacognition and its importance for learning and performance to university chemistry students. Research should focus on effects of explicit instruction, modelling, integration of metacognitive skills with course content, and opportunities for practice and reflection. With respect to assessment of metacognition, the articles highlighted in this chapter illustrate that, by far (19 of 31 articles), the most common method for assessing metacognition is student self-reports through established questionnaires. Although validity and reliability of these measures have been established for most instruments, there are problems with relying solely on self-reports to measure metacognition. For one, observed increases or decreases in self-reported metacognitive practices over time are open to interpretation. Most often, researchers point to increases in self-reported metacognitive practices over time as evidence for improved metacognition. However, sometimes decreases in students’ self-reported metacognitive practices are claimed as victories, the notion being that students are more familiar with the concept of metacognition after a metacognitive intervention and, therefore, are better able to assess their metacognitive skills after the intervention. These dueling interpretations of changes in self-reported metacognitive skills indicate that future research should utilize additional, complementary measures of metacognition to better interpret and understand what the ratings mean and whether metacognitive change occurred. Another problem with self-reports is related to the complicated nature of metacognition. Research supports the dual nature of metacognition: in some contexts, it is a general construct; in others, it is more task-specific (47). Self-reports, however, owing to the nature of the items included and the timing of completion, tend to assess general metacognition only. The same criticism holds for confidence judgments and test grade pre- and postdictions as measures of metacognition. If these measurements are not made within the context of conditions of a specific criterion task, they tend to assess general and not task-specific metacognitive skills. Test grade pre- and postdictions also suffer from the added complication of variations in test averages. That is, because most students tend to overestimate their level of preparation or performance, tests with higher averages give the impression that students are more accurate in their judgments. These issues highlight the need for more concurrent assessments of 71 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
metacognition, e.g., via IMMEX or think-aloud protocols to measure task-specific metacognition, as well as the need to incorporate multiple assessment modalities to corroborate findings and capture the complex nature of metacognition (47). Interestingly, only 4 articles highlighted in this chapter made use of multiple modes of metacognitive assessment. The collection of articles featured also highlight the need for more research examining metacognition in senior years of the undergraduate curriculum. Of the 31 articles included, 23 used students registered in a typical introductory chemistry course, most in their first year of study, as participants (see Table 2 in the Appendix for a summary). Another 3 articles involved participants in teacher education programs who were enrolled in an introductory chemistry course (32–34), and 1 article used participants enrolled in an upgrading year introductory chemistry course to address under-prepararedness (42). In the remaining 4 articles, participants included “2nd year” students with no course specified (21), “first and second year students enrolled in general, inorganic or organic chemistry (25)”, students enrolled in a biochemistry course (year of study not specified) (36), and students enrolled in an organic chemistry course (3% 1st year; 24% 2nd year; 26% 3rd year, 36% 4th year, 11% post-degree) (40). Expanding the range of students studied, as well as the types of chemistry courses at each level of study, would allow a more precise understanding of the impact of specific metacognitive interventions. A closely related issue is the relatively small number of longitudinal studies that have been reported. Of the 31 articles, 26 collected data within a single term and only 5 collected data over two terms (see Table 2 in the Appendix for a summary). Studies stretching beyond that time frame did not include the same students. That is, data may have been collected in four consecutive years in a general chemistry course, but in no cases were the same group of students monitored for more than two consecutive terms. Given how long it may take to train students in metacognitive strategies, as suggested by persistent Dunning-Kruger effects noted previously (29, 41, 48), there is a need for more longitudinal studies that monitor improvements in metacognition beyond a single year of study. If students had the opportunity to experience multiple interventions over their years of study we would be able to identify which interventions best support learners as chemistry domain knowledge increases. Finally, of the 31 articles highlighted, 20 were completed in the United States, 5 in Turkey, 3 in South Africa, 1 in Canada, 1 in Spain and 1 in Taiwan. Given that inadequate metacognitive skills in university students is not a regional problem, but a universal one, there is a need for more widespread studies on metacognition in higher education chemistry within and among countries. In closing, we know that metacognition is a complex skill but one that underlies all higher order thinking skills. Higher education chemistry instruction depends upon learners’ ability to execute metacognitive skills effortlessly. Providing concurrent instruction to support development of metacognitive skills in addition to chemistry content may provide students in higher education chemistry classes with the skills, scaffolds and practice needed to maximize their learning potential. 72 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
Appendix Table 1. Publications Excluded from ‘Total Unique Hits’ Reason For Exclusion
Number
Chemistry not the/a focus
5
Metacognition not the/a focus
21
Not higher education
18
Not a student sample
7
Not a research study
8
More than one of the above
16
Cannot access full text
2
Total number of exclusions
77
73 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
Table 2. Summary Table for 31 Articles Meeting Inclusion Criteria
74
Reference
Metacognition Taughti
Metacognition Assessedii
Interventioniii
Improved Metacognitioniv
Improved Performancevi
Course (No. of terms)
Location
(19)
no
yes (SR)
yes (TM)
yes
nr
intro chem (1)
USA
(20)
no
yes (SR)
no
yes
yes (SR)
intro chem (1)
USA
(21)
no
yes (SR)
yes (SA)
yes
nr
“2nd year” (1)
South Africa
(22)
no
yes (SR, C)
no
nr
yes (IM)
intro chem (1)
USA
(23)
no
yes (SR)
no
no
yes (SR)
intro chem (1)
USA
(24)
no
yes (PP)
no
nr
yes (DK)
intro chem (1)
South Africa
(25)
no
yes (C)
no
nr
nr
intro/org/inorg (1)
USA
(26)
no
no
yes (TM)
nr
nr
intro chem (1)
USA
(27)
no
yes (SR)
no
nr
yes (SR)
intro chem (1)
Turkey
(28)
no
yes (SR, C)
yes (TM)
no
yes/no (IM)
intro chem (1)
USA
(29)
no
yes (PP)
no
no
yes (DK)
intro chem (2)
USA
(30)
no
no
yes (SA)
nr
no
intro chem (1)
USA
(31)
no
yes (TM)
yes
yes (IM)
intro chem (2)
USA
(32)
no
yes (SR)
yes (SA)
nr
yes (SR)
gen. ed. chem (1)
USA
(33)
no
yes (SR)
yes (TM)
yes
nr
intro chem (1)
Turkey
(34)
no
yes (SR)
yes (P)
yes
yes (SR)
intro chem (1)
Turkey
(35)
no
yes (SR)
no
nr
yes (SR)
intro chem (1)
Turkey
yes (C)
Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
75 i
Reference
Metacognition Taughti
Metacognition Assessedii
Interventioniii
Improved Metacognitioniv
Improved Performancevi
Course (No. of terms)
Location
(36)
no
yes (SR, C)
no
nr
yes (SR)
biochem (1)
Turkey
(37)
no
yes (SR)
yes (TM)
yes
no
intro chem (2)
USA
(38)
yes
no
yes (EI)
nr
yes
intro chem (1)
USA
(39)
no
yes (SR)
yes (P)
yes
no
intro chem (1)
Canada
(40)
no
yes (SR)
yes (SA)
nr
no
orgo chem (1)
USA
(41)
no
yes (PP)
no
no
yes (DK)
intro chem (1)
USA
(42)
no
yes (EI)
no
yes (DK)
“prep” chem (1)
South Africa
(43)
yes
yes (SR)
yes (EI, P)
yes
yes
intro chem (2)
USA
(44)
no
yes (SR)
no
nr
yes (SR)
intro chem (1)
Spain
(45)
no
yes (SR)
yes (TM)
yes
no
intro chem (1)
USA
(46)
no
yes (PP)
no
nr
yes/no (DK)
intro chem (1)
USA
(47)
no
yes (SR, C, PP)
no
nr
yes (DK)
intro chem (1)
Taiwan
(48)
no
yes (PP)
no
yes (DK)
intro chem (2)
USA
(49)
no
no
yes
intro chem (1)
USA
yes (PP)
yes (P)
yes and
nov
nr ii
Metacognition Taught = explicit instruction on the meaning and importance of metacognition. SR = self-report; C = concurrent; PP = pre/postdiction. iii EI = explicit instruction; TM = comparing teaching methods; SA = introducing student activities; P = using prompts. iv nr = no report - metacognition measured on one day or not at all, or measured over time but change not addressed; no = metacognition measured over time but no improvement observed. v Initial improvement observed but then no additional improvement over two terms. vi nr = no report; no = no improvement observed; yes/no = mixed result; DK = based on Dunning-Kruger effect; IM = based on IMMEX results; SR = based on self-report vs performance (or performance indicator) comparison.
Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
References 1. 2. 3.
4. 5. 6.
7. 8. 9. 10. 11. 12. 13. 14. 15.
16. 17. 18. 19. 20. 21. 22. 23. 24. 25. 26. 27. 28.
Garner, R. Rev. Educ. Res. 1990, 60 (4), 517–529. Schneider, W. and Pressley, M. Memory Development between 2 and 20; New York: Springer-Verlag, 1989. Waters, H. and Andreassen, C. In Cognitive Strategy Research: Psychological FoundationsPressley, M. and Levin, J. R. Springer-Verlag, New York, 1983. Wood, E.; Willoughby, T.; McDermott, C.; Motz, M.; Kaspar, V.; Ducharme, M. J. J. Educ. Psychol. 1999, 91 (3), 527–536. Schraw, G. Instr. Sci. 1998, 26, 113–125. Kaplan, M.; Silver, N.; Lavaque-Manty, D.; Meizlish, J., Eds.; Using Reflection and Metacognition To Improve Student Learning: Across the Disciplines, Across the Academy; Stylus Publishing, Inc.: Virginia, 2013. Schraw, G.; Moshman, D. Educ. Psychol. Rev. 1995, 7 (4), 351–371. Schraw, G.; Dennison, R. S. Contemp. Educ. Psychol. 1994, 19, 460–475. Schneider, W.; Chein, J. M. Cogn. Sci. 2003, 27 (3), 525–559. Novak, G. M. Just-in-time teaching. New Directions Teaching Learning 2011, 128, 63–73. Ericsson, K. A. and Simon, H. A. Protocol Analysis: Verbal Reports as Data, rev. ed.; MIT Press: Cambridge, MA, 1993. Underdahl, J.; Palacio-Cayetano, J.; Stevens, R. Learn. Lead. Technol. 2001, 28, 26–31. Mokhtari, K.; Reichard, C. A. J. Educ. Psychol. 2002, 94 (2), 249. van Velzen, J. H. J. Educ. Res. 2017, 110 (5), 494–502. Schneider, W. Metacognition and Memory Development in Childhood and Adolescence. In Metacognition, Strategy Use and Instruction; Salatas Waters, H.; Schneider, W., Eds.; Guilford Press, 2010. Kruger, J.; Dunning, D. J. Pers. Soc. Psychol. 1999, 77 (6), 1121. Veenman, M. V. J.; van Hout-Wolters, B. H. A. M.; Afflerbach, P. Metacognition Learning 2006, 1 (1), 3–14. Zohar, A.; Barzilai, S. Stud. Sci. Educ. 2013, 49 (2), 121–169. Francisco, J. S.; Nicoll, G. J. Chem. Educ. 1998, 75 (2), 210–213. Zusho, A.; Pintrich, P. R. Int. J. Sci. Educ. 2003, 25 (9), 1081–1094. Davidowitz, B.; Rollnick, M. Res. Sci. Ed. 2003, 33 (1), 43–69. Cooper, M. M.; Sandi-Urena, S.; Stevens, R. Chem. Educ. Res. Pract. 2008, 9 (1), 18–24. Cooper, M. M.; Sandi-Urena, S. J. Chem. Educ. 2009, 86 (2), 240–245. Potgieter, M.; Ackermann, M.; Fletcher, L. Chem. Educ. Res. Pract. 2010, 11 (1), 17–24. Rich, J.; Miller, D.; DeTora, L. Across Disciplines 2011June27, 8 (1). Sandi-Urena, S.; Cooper, M. M.; Gatlin, T. A.; Bhattacharyya, G. Chem. Educ. Res. Pract. 2011, 12 (4), 434–442. Aydin, Y. C.; Uzuntiryaki, E.; Demirdöğen, B. Educ. Psychol., UK 2011, 31 (1), 55–66. Sandi-Urena, S.; Cooper, M. M.; Stevens, R. H. Int. J. Sci. Educ. 2011, 33 (3), 323–340. 76
Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
29. Bell, P.; Volckmann, D. J. Chem. Educ. 2011, 88 (11), 1469–1476. 30. Parker Siburt, C. J.; Bissell, A. N.; Macphail, R. A. J. Chem. Educ. 2011, 88 (11), 1489–1495. 31. Sandi-Urena, S.; Cooper, M.; Stevens, R. J. Chem. Educ. 2012, 89 (6), 700–706. 32. Dianovsky, M. T.; Wink, D. J. Sci. Educ. 2012, 96 (3), 543–565. 33. Tosun, C.; Senocak, E. Aust. J. Teach. Educ. 2013, 38 (3)Article 4. 34. Saribas, D.; Mugaloglu, E. Z.; Bayram, H. J. Math. Sci. T. 2013, 9 (1), 83–88. 35. Uzuntiryaki-Kondakçi, E.; Çapa-Aydin, Y. Educ. Sci.-Theor. Pract. 2013, 13 (1), 666–670. 36. Sadi, Ö. High. Educ. Stud. 2013, 3 (5), 52–67. 37. Xu, H.; Talanquer, V. J. Chem. Educ. 2013, 90 (1), 21–28. 38. Cook, E.; Kennedy, E.; McGuire, S. Y. J. Chem. Educ. 2013, 90 (8), 961–967. 39. Muis, K. R.; Ranellucci, J.; Franco, G. M. J. Exp. Educ. 2013, 81 (4), 556–578. 40. Lopez, E. J.; Nandagopal, K.; Shavelson, R. J.; Szu, E.; Penn, J. J. Res. Sci. Teach. 2013, 50 (6), 660–676. 41. Pazicni, S.; Bauer, C. F. Chem. Educ. Res. Pract. 2014, 15 (1), 24–34. 42. Mathabathe, K. C.; Potgieter, M. Chem. Educ. Res. Pract. 2014, 15 (1), 94–104. 43. Zhao, N.; Wardeska, J. G.; McGuire, S. Y.; Cook, E. J. Coll. Sci. Teach. 2014, 43 (4), 48–54. 44. González, A.; Paoloni, P.-V. Chem. Educ. Res. Pract. 2015, 16 (3), 640–653. 45. van Opstal, M. T.; Daubenmire, P. L. Int. J. Sci. Educ. 2015, 37 (7), 1089–1112. 46. Lindsey, B. A.; Nagel, M. L. Phys. Rev. Spec. Top. Phys. Educ. Res. 2015, 11 (2), 020103. 47. Wang, C. –Y. Res. Sci. Educ. 2015, 45 (4), 555–579. 48. Hawker, M. J.; Dysleski, L.; Rickey, D. J. Chem. Educ. 2016, 93 (5), 832–840. 49. Thadani, V.; Bouvier-Brown, N. C. J. Excel. Coll. Teach. 2016, 27 (2), 77–95. 50. National Research Council. Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering; Singer, S. R.; Nielsen, N. R.; Schweingruber, H. A., Eds.; Committee on the Status, Contributions, and Future Directions of Discipline-Based Education Research. Board on Science Education, Division of Behavioral and Social Sciences and Education: Washington, DC, 2012. 51. Rickey, D.; Stacy, A. M. J. Chem. Educ. 2000, 77 (7), 915–920. 52. Zohar, A.; Dori, Y. Metacognition in Science Education: Trends in Current Research; Ziedler, D., Ed.; Contemporary Trends and Issues in Science Education; Springer, 2012; pp 1−19.
77 Cox and Schatzberg; International Perspectives on Chemistry Education Research and Practice ACS Symposium Series; American Chemical Society: Washington, DC, 2018.
