Developing Metacognitive and Problem-Solving Skills through

ARTICLE pubs.acs.org/jchemeduc

Developing Metacognitive and Problem-Solving Skills through Problem Manipulation Claire J. Parker Siburt,† Ahrash N. Bissell,‡ and Richard A. Macphail*,† †

Department of Chemistry and ‡Academic Resource Center, Duke University, Durham, North Carolina 27708-0346, United States ABSTRACT: In a collaborative effort between the our university’s department of chemistry and the academic resource center, we designed a model for general chemistry recitation based on a problem manipulation method in which students actively assess the skills and knowledge used to answer a chemical problem and then manipulate the problem to create a new one. This reflective process aims to facilitate student engagement with the chemistry problem-solving process and to enhance student metacognition by helping students identify their knowledge gaps. The recitation format provides an opportunity for students to work collaboratively on this process and to present and discuss their work. Student response to this new style of recitation was overwhelmingly positive. In addition, students easily assimilated the vocabulary of the methodology into everyday conversation and were better able to articulate their learning needs. KEYWORDS: First-Year Undergraduate/General, Collaborative/Cooperative Learning, Problem Solving/Decision Making

“I

’ve done all the practice problems, but still can’t do well on the exams” is a familiar refrain to instructors in general chemistry courses. Such a statement is often indicative of novice problem solvers, who view problem solving as a recall task rather than a process built around concepts.1,2 Additionally, such students typically lack the self-awareness of their own thinking process—metacognition—to identify gaps in their knowledge and problem solving ability.3 In an attempt to address this issue, we designed and evaluated a model for general chemistry recitation based on what we call the problem manipulation method (PM2). Initially developed for one-on-one consultations at the Duke University Academic Resource Center (ARC), PM2 is a method in which students change or manipulate a problem in order to test their own understanding of the underlying concept (Figure 1). By actively linking a series of related problems together and then creating their own problems (manipulations), we hypothesize that students learn more about the underlying chemical concepts compared to answering what they otherwise see as a series of randomly assigned problems. Here, we set out to employ PM2 in the recitation classroom setting to help students engage with chemistry problems and to enhance their metacognitive skills. This method took the form of a curriculum and set of chemistry-specific PM2 material for second-semester general chemistry recitation. The exercises were scaffolded4 across a set of problems within one recitation session and across the semester. Within each recitation meeting, small groups of students deliberately explored a set of problems, each problem a manipulation of the others. Facilitated by a teaching assistant (TA), students were encouraged to assess the connections between the Copyright r 2011 American Chemical Society and Division of Chemical Education, Inc.

manipulations and which types of questions they found difficult. Using recitation for an innovative yet still problem-based approach fit well with our staffing capacity and the students’ expectations. Similar to other active learning formats, our approach is process oriented, uses teamwork, and encourages student participation by awarding points.5 9 However, we also set out to teach students a subject-specific metacognitive strategy;3 the importance of this goal has been articulated in many venues.10 Our PM2 recitation format provides an opportunity and a language for students to assess their own knowledge, thereby increasing metacognition. In addition, by actively identifying connections between problems and subsequently creating their own problems, the students begin to move from the novice learner stage to creating their own knowledge.1,2 The recitation structure also encourages group work and collaboration, which has been shown to improve metacognitive skills and performance.11 13 Student response to this new style of recitation was overwhelmingly positive, with a large majority of students preferring this recitation format over others they had previously experienced. Most students recommended the format be used again and many students believed that the format helped prepare them for midterm exams. The students easily assimilated the language of PM2 into their working vocabulary and used this vocabulary to articulate their needs. These and several other observations are discussed below. Published: September 19, 2011 1489

dx.doi.org/10.1021/ed100891s | J. Chem. Educ. 2011, 88, 1489–1495

Journal of Chemical Education

ARTICLE

Several examples of manipulations can be found in the example problems shown in Boxes 1 4.

Figure 1. Schematic of the recitation format for working problems.

’ PROBLEM MANIPULATION METHODOLOGY As originally designed, the problem manipulation method provides a system for analyzing a particular problem and then creating a new problem that enhances understanding of an underlying concept. This helps students draw connections between seemingly distinct, yet conceptually related problems. Importantly, this process forces students to assess their understanding, rather than just completing the problem and moving on. PM2 involves several steps. First, students: 1. Identify what the problem is asking for 2. Write in words a strategy for solving the problem 3. Solve the problem 4. Identify the concepts involved The next steps are the most important. Students are asked to: 5. Manipulate the original problem to make a new one built around the same concept 6. Solve the new problem 7. Assess how the two problems are linked conceptually These last manipulation steps are where the innovation lies, as these steps force the students to come to terms with the underlying concept, and also to reflect on what they do or do not understand about the problem—these are the metacognition steps. In addition, these steps empower students to make the problem their own and to think on their own. This process also leads naturally to solving problems of increasing complexity, a task which Niaz and Robinson have shown correlates closely with “student development”.14 For example, by categorizing parts of a complicated problem into tasks, identifying the level of thinking required (e.g., Bloom’s taxonomy),15 and evaluating what chemical information or concepts are needed, students are better equipped to successfully tackle more complicated problems. Finally, manipulating problems is similar to what instructors do when creating exam problems, so another goal is for students to recognize exam problems as manipulations of problems they have already done, and thereby improve their performance. “Manipulations” can come in many forms. For example, an algorithmic problem might be manipulated by simply asking the question in reverse. However, this new question would still be classified as algorithmic. A manipulation might also come in the form of a synthetic problem, where students are required to use auxiliary information or an extra conceptual step to solve the new problem. Any problem can be manipulated in a variety of ways and we found this methodology can be readily adapted to all kinds of problems (mathematical, pictorial as used in ref 16, or graphical, etc.).

’ CLASS CONSTRUCTION AND IMPLEMENTATION OF METHODOLOGY Recognizing that students would need time to learn the PM2, we developed a plan that was staged and scaffolded across the semester, as well as within each recitation period. We began the semester with an emphasis on recognizing how a series of related problems could be seen as manipulations of each other, and gradually moved to asking students to be more ambitious in creating their own manipulations. Within each problem set, the first problem was relatively simple and designed to orient the students to the underlying concept and topic of that recitation, whereas the following problems were increasingly sophisticated manipulations. As an example, Boxes 1 4 provide a complete problem set for a recitation focused on reaction mechanisms. We built each recitation around one central concept and four or five problems, as depicted in Figure 1. Students completed prerecitation (preclass) problems individually prior to attending class. Students grappled with the group problem and the manipulations or twists in small groups during recitation (typical enrollment in each recitation section was 16 24). Then students completed the postrecitation problem—another manipulation—individually after class. Preclass Problems

Each preclass problem usually consisted of one algorithmic question. After completing the problem, students were required to write a short paragraph analyzing specific components of the problem. In the beginning of the semester, the students were 1490

dx.doi.org/10.1021/ed100891s |J. Chem. Educ. 2011, 88, 1489–1495


given leading questions to force them to approach the algorithmic question in a stepwise fashion. The main purpose of the preclass problem was to ensure that the students had begun to think about the specific chemical concepts for that respective recitation. Recitation began by having one student present his or her answer for the preclass problem.

ARTICLE

This process led into a class discussion of the similarities and differences between this group problem and the preclass problem. Following the group problem, the groups would work on another twist or manipulation problem (usually each group was assigned one of two twist problems). A subsequent informal presentation provided an opportunity for the TA to lead a class discussion about multiple approaches to the same problem. Most importantly, the class discussed how the group problem and the twist(s) were related conceptually to each other and to the preclass problem. The problems themselves are not particularly unusual and are typical of those found in standard general chemistry texts. However, when the students are able to link the problems together by identifying either or both the conceptual relationship and the type of question, they are able to expand their conceptual understanding. Postproblems

Group Problems

The group problem, completed during the first part of recitation, consisted of a manipulation of the preclass problem. We designed the group problem to be more challenging than the corresponding preclass problem, thereby encouraging collaboration and reason-based concept learning.18 Students worked in groups of three or four to answer the group problem and then one student would present a group’s solution orally at the board.

Finally, students were given yet another manipulation as a postproblem to be completed individually. This last task allowed them to assess their ability to complete a more difficult question without the help of the small group or class. According to feedback from TAs, the students showed increased chemical understanding on the postproblem relative to the preclass problem. At the beginning of the semester, we focused on identifying underlying concepts, illustrating problem-solving steps, and 1491



ARTICLE

we encouraged students to present their work whether they had completed a problem or not, and then to use the class as a resource to finish the problem, if necessary. The focus of the informal presentation was explicitly to facilitate a reflective conversation about the problems. Participation in this new recitation format was encouraged on many levels. Students saw a direct connection between the problem manipulation methodology and lecture because the lecturer used the same language and presented examples in a similar manner. In addition, recitation topics were synchronized with lecture, typically following coverage of that same topic in lecture by a week. Grading for recitation included points for participation and for attempting the preclass and postproblems. Consistent with our goal of focusing on the problem-solving process rather than the answer, students who made a good faith effort at solving the preclass and postclass problems received full credit, along with written feedback from their TA. The recitation points, which amounted in total to 10% of the overall course grade, were sufficient to ensure students completed the assignments, while allowing students the flexibility to work on a new technique and on their metacognitive skills without fear of substantial penalty. Finally, explicit manipulations of the problems used in recitation appeared on quizzes and exams. Successful implementation of this recitation format (focused on metacognition and problem manipulation) required a group of committed TAs willing to facilitate group work. TAs had to continually point out the connections between problems and reinforce why recitation was focused on the PM2. TAs had to have a sufficiently deep understanding of the chemical concepts to anticipate the links that students might propose and to judge which connections are valuable to the learning process and which are not. However, individual TA preparation for each recitation was not burdensome—this format actually required less TA time than the traditional question and answer session owing to the structured nature of the recitations and the small number of assigned problems per week. Weekly TA meetings focused on pointing out the connections between the problems and developing the use of the PM2 language. TAs were also provided with an outline for each recitation, answers to each problem, and notes on areas in which we anticipated challenges for the students. We found that the PM2 model was equally successful with either undergraduate or graduate student TAs as long as they were engaged with teaching within this model.

’ ASSESSMENT Student Surveys

categorizing problems using Bloom’s taxonomy.15,19 The majority of the semester was spent teaching students to identify connections between sets of problems, like those given in Boxes 1 4. Halfway through the semester students were asked during recitation or in the homework to suggest ways the problems could be changed. For the last three weeks of the semester, we shifted the focus to active problem manipulation and the students began to write their own twist problems. Creating a supportive environment in recitation proved vital to the success of this new recitation format. The first recitation meeting was used to demonstrate the methodology using basic chemical concepts from the first semester (e.g., stoichiometry, moles, and the ideal gas law) and to focus on building trust between the students and with the TA. Throughout the semester,

Surveys of student opinions on this new recitation format were collected at the end of the semester for three years in a row. The results were similar each year, and here we highlight representative results from the first year, when we received 139 student responses out of 144 students enrolled in the course. The student response to this new style of recitation was overwhelmingly positive: 87% of the students responding said they preferred this recitation format over prior college experience (first-semester chemistry or another STEM course), and 88% said we should continue to use this recitation format. One student commented, “[I] prefer this recitation format because it places more emphasis on understanding the problem rather than simply getting to the answer”. We also asked students to consider the impact of the recitation sections and PM2 on their course performance and understanding of the material. Over 80% of students found every 1492


Journal of Chemical Education component of the recitation format to be helpful (prerecitation, in-recitation, and postrecitation exercises), and 87% said that the recitation sections and PM2 helped to prepare them for the midterm exams. Surprisingly, 92% of those responding to the survey said they enjoyed the required group work. These results demonstrate that our students felt comfortable in this new recitation setting and perceived PM2 as a useful weekly exercise. Additionally, students were asked open-ended questions about PM2. One student wrote, “The manipulations required you to address the problems from all angles, giving you a more complete body of knowledge.” Another student stated, “The manipulations helped out on the test and really help to make sure you understood the deeper concept as opposed to just how to crunch the numbers with some formulas.” The large number of similar responses demonstrates that the students found value in the methodology and view it as a tool to aid their learning of chemistry. While there were only a few negative comments, two common threads emerged regarding the group work. A handful of students said they simply preferred to work on their own, or complained that their group did not share the work effectively. The latter issue is a common challenge in collaborative work, and can be addressed in a variety of ways, for example, by assigning rotating roles for group members, as is done in PLTL (Peer-Led Team Learning).6 Examination Performance

To assess the effect of PM2 on examination performance, we compared results on a common final examination given to students in the same course with the same lecture instructor both before and after we implemented PM2 in recitation. We found that, on average, the overall exam score as well as scores on individual free-response problems were virtually unchanged. While we were disappointed not to see an improvement, we were heartened that such a popular new recitation format did not have a negative impact on examination performance, and likely had a positive impact on other areas of learning (e.g., collaborative problem solving, metacognitive skills, etc.) that may not be measured well by this particular instrument (the final examination). We also note that the group of students for whom PM2 would be expected to have the most impact may be small in number compared to the overall course population, diluting the effect of the method on the average student performance. Finally, as noted above, most students were just becoming comfortable with PM2 over the course of the semester and were not actively applying it on their own outside of recitation. Thus, continuation of PM2 beyond one semester might be more likely to have a direct impact on examination performance, a hypothesis we will be pursuing further, as described below.

’ OTHER OBSERVATIONS Several interesting and some unexpected trends were observed when we used PM2, including: 1. An improvement in student engagement and general attitude toward chemistry 2. A long learning curve toward creating knowledge 3. A change in the language students used 4. A dichotomy between some students who could easily answer chemistry problems and others who could easily recognize relationships between problems 5. A lack of understanding of the scientific process

ARTICLE

Our intention here is to draw attention to interesting observations that could be studied further rather than to propose comprehensive explanations for these observations. An Improved Attitude toward Chemistry

The lecturer and TAs noticed that the students seemed more engaged in the classroom and better understood the learning expectations of the course compared to previous semesters prior to the implementation of PM2. In addition, the visitation rate to the ARC for consultation or tutoring was significantly lower for students in the section using PM2 than in the other concurrent chemistry sections and than in the past. This reduction in visits might have occurred because students were more confident that they understood the learning expectations and needs. Indeed, according to ARC staff members, students that did visit the ARC were clearly more aware of the possible reasons for their struggles compared to students in previous semesters or other sections. A Long Learning Curve

One surprising and concerning observation was the students’ hesitation to change a problem given in the text or by the teacher. The students lacked confidence that they could adequately test their own knowledge and required significant encouragement and positive feedback. Students seemed to think that chemistry was a set of facts and problems they had to learn, but that they did not have the authority to make sense of the material for themselves. Therefore, when we tried to get them to create a problem about a particular concept, they had difficulty. We designed the recitations to teach the students to identify the connections between problems first, and then planned to move progressively toward having them create their own problems. We successfully taught the students to identify and classify manipulations, but the students did not fully master the skill of manipulating problems. Rather, by the end of the semester, students had only begun to create meaningful manipulations on their own. Changes in Student Language

Using the language of Bloom’s taxonomy15,19 to categorize types of problems facilitated students’ ability to discuss the relationships between problems. Students easily internalized this language, and commonly used the vocabulary (algorithmic, synthetic, manipulation, etc.) in recitation, lecture, and office hours. By providing both a method to engage with the problems, and also a language to discuss the problems, students were empowered and “trick” problems were demystified. Several students commented that approaching problems in this way helped relieve their fears associated with chemistry. They began to “see through the problems”. A Dichotomy of Students

Interestingly, two groups of students with contrasting abilities emerged from the class. Some students who could easily answer the more quantitative questions gave minimal or trivial answers when asked to identify the relationships between problems. In contrast, other students who gave insightful answers regarding the relationships between problems were not able to answer the problems correctly. In fact, on more than one occasion, students from the latter group wrote notes on the side of their test indicating where or why they got stuck. We did not anticipate this dichotomy of abilities and believe it merits further research. Misunderstandings about the Scientific Process

We also encountered misunderstandings of the scientific process. The problem set shown in Boxes 1 4 required students 1493


Journal of Chemical Education to choose between two proposed mechanisms for a reaction under certain conditions. Students easily understood the connections between the questions (which was exciting given this was only the third week of class), but the students did not understand how a mechanism that was printed on the page could be “wrong”. For many students, their instinct was to say that the given experimental data was flawed, instead of evaluating the proposed mechanisms. Many students did not understand that the normal research process would be to collect the data and then propose a mechanism that could explain that data. We are considering designing more recitation problem sets to address this issue, as described below.

’ FUTURE DIRECTIONS The PM2 can be readily adapted to a wide variety of problem types and formats that would be worth exploring. For example, one could create a series of problem manipulations designed to illustrate aspects of the scientific method, such as the transition from existing knowledge, to practical application, to extension and adaptation to a new problem. On the topic of buffers, students might first work through a prerecitation problem focused on the relation between a well-buffered solution and the point at which pH = pKa. The group problem then could pose a practical laboratory application of this knowledge, for instance, asking students for several ways to make a buffer from a given set of reagents found in the laboratory (e.g., a salt, weak acid, and strong base). And the twist might then be an extension of this knowledge to a new situation, for example, designing a molecule to provide a specialized buffer for a biochemical experiment at a particular pH by choosing from a set of functional groups with different pKa values. Casting such problem sets in the context of a real-world scientific laboratory setting could help address some of the misunderstandings of the scientific process that we noted above, and as has been reported elsewhere.20,21 A new modification to our introductory chemistry curriculum introduced recently provides an opportunity to carry out a more thorough assessment of the impact of the PM2 on metacognition and student learning and performance. In this new curriculum, the majority of our entering first-year students begin with a general chemistry course that starts at a somewhat more advanced level than in the past (it assumes knowledge of basic topics such as stoichiometry and ideal gases, which the majority of our students have seen in high school). Students who are not ready for this course because they lack sufficient chemistry background or problem-solving skills are encouraged to start instead with a skills-oriented course that employs a variety of active-learning pedagogical techniques, including PM2, and then move on to the first general chemistry course. This latter group (of 70 90 students) contains the cohort most likely to benefit from PM2, so any improvements from the methodology should be most apparent for them. We will also employ PM2 for this group in the general chemistry course that follows, so we can examine the impact of extending the technique vertically through two courses in consecutive semesters. The impact of PM2 will be assessed by comparing the performance of this student cohort with those who did not start with the skills course, by tracking changes in their metacognitive skills through use of a recently developed Metacognitive Activities Inventory tool,22,23 and by following their scores on a diagnostic quiz given at the beginning of each semester. We hope this analysis will give us a clearer

ARTICLE

picture and more direct assessment of the benefits of PM2 than we have been able to ascertain to date. Finally, it is of interest to see how well the PM2 methodology translates to other chemistry courses, other STEM disciplines, and other institutions. Although PM2 has not been explicitly incorporated in other chemistry courses, we have heard anecdotally that some of our students found PM2 useful as a study tool in organic chemistry, that they continued to use the vocabulary, and that it helped them recognize organic exam problems as manipulations or twists of homework problems. As originally developed in the ARC, PM2 was meant to be applicable to any problem-based STEM course, and the methodology has continued to be used successfully on an individual basis for students in other disciplines, for example, in introductory calculus. Some chemistry faculty and academic resource center directors at other institutions have expressed interest in trying PM2 in their chemistry programs, and this will provide an additional opportunity to see how well the methodology transfers to other groups of students and faculty.

’ CONCLUSIONS We have designed and implemented a scaffolded recitation model based on problem manipulation methodology, to which students have responded in an overwhelmingly positive manner. This methodology provides students with a language to discuss the relationships between problems, and the recitation model provides an opportunity for students to practice and develop metacognitive skills. The model is flexible and relatively easy to implement, and should be adaptable to other chemistry courses, STEM courses, and to other institutions.

’ AUTHOR INFORMATION Corresponding Author

*E-mail: [email protected].

’ ACKNOWLEDGMENT We would like to thank Donna Hall and the Academic Resource Center for helpful discussions and support. We would like to thank Alvin Crumbliss for his support. Finally, we thank our TAs and students whose enthusiasm and sincerity are inspiring. ’ REFERENCES (1) Harper, K. A. The Physics Teacher 2006, 44, 250–251. (2) Committee on Developments in the Science of Learning. How People Learn: Brain, Mind, Experience, and School; Bransford, J. D., Brown, A. L., Cocking, R. R., Eds.; National Academy Press: Washington, DC, 1999. (3) Rickey, D.; Stacey, A. M. J. Chem. Educ. 2000, 77, 915–920. (4) Larkin, M. Using Scaffolded Instruction To Optimize Learning. ERIC Digest 2002, ED474301. (5) Hanson, D.; Wolfskill, T. J. Chem. Educ. 2000, 77, 120–130. (6) Hockings, S. C.; DeAngelis, K. J.; Frey, R. F. J. Chem. Educ. 2008, 85, 990–996. (7) Mahalingam, M. M.; Schaefer, F.; Morlino, E. J. Chem. Educ. 2008, 85, 1577–1581. (8) Oliver-Hoyo, M. T.; Allen, D. J. Chem. Educ. 2004, 81, 441–448. (9) Farrell, J. J.; Moog, R. S.; Spencer, J. N. J. Chem. Educ. 1999, 76, 570–574. (10) Thomas, G. P. Res. Sci. Educ. 2006, 36, 1–6. (11) Larkin, S. Res. Sci. Educ. 2006, 36, 7–27. 1494



ARTICLE

(12) Lyon, D. C.; Lagowski, J. J. J. Chem. Educ. 2008, 85, 1571–1576. (13) Cooper, M. M.; Cox, C. T., Jr.; Nammouz, M.; Case, E. J. Chem. Educ. 2008, 85, 866–872. (14) Niaz, M.; Robinson, W. R. J. Res. Sci. Teach. 1992, 29, 211–226. (15) Bloom, B. S., Ed.; The Taxonomy of Educational Objectives, Handbook I: Cognitive Domain; Longman: New York, 1956. (16) Niaz, M.; Robinson, W. R. J. Sci. Educ. Technol. 1993, 2, 407–416. (17) Chang, R. Chemistry, 9th ed.; McGraw-Hill: New York, 2006; Ch. 13. (18) Cracolice, M. S.; Deming, J. C.; Ehlert, B. J. Chem. Educ. 2008, 85, 873–878. (19) Anderson, L. W., Krathwohl, D. R., Eds.; A Taxonomy for Learning, Teaching, and Assessing: A Revision of Bloom’s Taxonomy of Educational Objectives; Longman: New York, 2001. (20) Stieff, M.; Wilensky, U. J. Sci. Educ. Technol. 2003, 12, 285–302. (21) Stieff, M. J. Chem. Educ. 2005, 82, 489–493. (22) Cooper, M. M.; Sandi-Urena, S. J. Chem. Educ. 2009, 86, 240–245. (23) Cooper, M. M.; Sandi-Urena, S. Chem. Educ. Res. Pract. 2008, 9, 18–24.

1495


Developing Metacognitive and Problem-Solving Skills through

Recommend Documents