Factors Contributing to Problem-Solving Performance in First

Jun 6, 2014 - For a more comprehensive list of citations to this article, users are ... Analysis of Students' Missed Organic Chemistry Quiz Questions ...
0 downloads 0 Views 240KB Size
Download PDF
Article pubs.acs.org/jchemeduc

Factors Contributing to Problem-Solving Performance in First-Semester Organic Chemistry Enrique J. Lopez,*,†,¶ Richard J. Shavelson,†,‡ Kiruthiga Nandagopal,§ Evan Szu,† and John Penn⊥ †

Graduate School of Education, Stanford University, Stanford, California 94305, United States SK Partners, LLC, Menlo Park, California 94025, United States § Stanford University School of Medicine, Stanford, California 94305-5101, United States ⊥ Chemistry Department, West Virginia University, Morgantown, West Virginia 26506-6045, United States ‡

S Supporting Information *

ABSTRACT: Problem solving is a highly valued skill in chemistry. Courses within this discipline place a substantial emphasis on problem-solving performance and tend to weigh such performance heavily in assessments of learning. Researchers have dedicated considerable effort investigating individual factors that influence problem-solving performance. The purpose of this study is to inspect the influences of a core set of factors (prior science achievement, knowledge structures, spatial ability, gender, and ethnicity) and their overall contributions to problem solving in organic chemistry using multivariate regression analyses. Results indicate that knowledge structures are a key predictor of problem-solving performance and account for a significant proportion of the variation in students’ problem-solving scores. Implications for research and practice in chemistry are discussed. KEYWORDS: General Public, Second-Year Undergraduate, Organic Chemistry, Problem Solving/Decision Making, Chemical Education Research FEATURE: Chemical Education Research

■

■

INTRODUCTION Problem solving is a highly valued skill in chemistry. Courses within this discipline place a substantial emphasis on problemsolving performance1,2 and tend to weigh such performance heavily in assessments of learning. Students’ performance on midterms and final exams largely depend on their problemsolving success and, at times, course grades are entirely based on this factor. Thus, problem-solving performance is a critical component in science education achievement in general, and chemistry in particular. Researchers have dedicated considerable effort investigating a variety of factors that influence performance on measures of problem solving.1,3−5 We broadly define problem solving as “what you do when you don’t know what to do.”6 Researchers have noted distinctions between novel problems and routine exercises,7 where the main distinction is one of familiarity. Whether routine or novel, performance on these tasks plays a significant role in student course success. Typically, investigations have focused on the influence of single factors. This study builds upon and extends prior research by examining not only the contribution of individual factors but also the combined contribution of a core set of factors (prior knowledge, knowledge structures, spatial visualization, ethnicity, and gender) on problem-solving performance in an important gatekeeper course: first-semester organic chemistry. © 2014 American Chemical Society and Division of Chemical Education, Inc.

BACKGROUND LITERATURE A cognitive perspective guides our work on learning and achievement in the domain of chemistry. Cognitive frameworks of learning attempt to understand an individual’s internal mental processes related to learning. Some of these processes include how students bring new knowledge to bear on pre-existing knowledge,8 the structure of knowledge,9 and associations between facets of spatial ability and performance.10 Prior Knowledge

David Ausubel, a prominent psychologist, stated that “[t]he most important single factor influencing learning is what the learner already knows; ascertain this and teach him accordingly” (p.iv).8 Indeed, research suggests that prior domain knowledge is a significant predictor of students’ chemistry achievement, typically assessed by problem solving on course exams.4,11 Chandran and colleagues11 examined the contribution of cognitive factors in chemistry achievement among high school students. Correlation and regression results indicated that students’ prior knowledge, as measured by a 20-item standardized test, and formal reasoning predicted and accounted for a significant proportion of chemistry performance above and beyond other factors (e.g., field dependence/independence, and memory capacity). Prior Published: June 6, 2014 976

dx.doi.org/10.1021/ed400696c | J. Chem. Educ. 2014, 91, 976−981

Journal of Chemical Education

Article

organic chemistry and found high spatial ability students scored significantly higher on problems that required spatial manipulation. Students who used multiple representations were more likely to solve problems that required spatial manipulation.25 Researchers have also noted mixed results with respect to gender differences on spatial measures24,26 that are thought to contribute to chemistry achievement.22 However, this assumption should be interpreted with caution. Stieff5 investigated gender differences in the mental rotation of chemistry representations among students enrolled in organic chemistry. Results showed that both males and females applied spatial− visual strategies or learned heuristics to solve spatial problems. Regardless of gender, students who applied learned heuristics were more likely to respond correctly to problems. In summary, studies have highlighted the influence of factors such as prior knowledge, knowledge structures, and spatial ability on problem-solving performance and academic achievement. Given the differential access to science learning opportunities among specific ethnic groups and gender differences in spatial ability, it is also important to consider ethnicity and gender as potential factors that influence these variables. These inquiries have led us to the following research questions: (1) What proportion of variation in problem-solving performance is accounted for by prior science achievement, knowledge structures, spatial ability, gender, and ethnicity? (2) Do prior science knowledge, knowledge structures, and spatial ability account for a significant proportion of variance in problem-solving performance after controlling for gender and ethnic group affiliation?

investigations in undergraduate general chemistry have reported similar findings.4 The examination of prior knowledge is especially important to consider in courses such as organic chemistry. Students typically enroll in organic chemistry after completing a yearlong sequence of general chemistry. However, problem solving in general chemistry and organic chemistry differ in at least one significant way. General chemistry primarily emphasizes quantitative problem solving, whereas organic chemistry primarily emphasizes qualitative procedures. For example, a common Ideal Gas Law problem in general chemistry asks students to calculate the number of moles in a gas at a certain temperature, pressure, and volume. In contrast, a substitution problem in organic chemistry may ask students to draw the reaction mechanism and product of the hydrolysis of tert-butyl bromide with water (forms tert-butyl alcohol and HBr). The latter problem relies less on knowledge of mathematical principles compared to the former. Thus, the extent to which prior chemistry knowledge is directly applicable to problem solving in first-semester organic chemistry is insufficient and deserves further exploration. Knowledge Structures

A second factor of interest is knowledge structures. Studies examining differences between experts and novices have noted that experts’ knowledge “is not simply a list of facts and formulas...instead, their knowledge is organized around core concepts or ‘big ideas’ that guide their thinking” (p 36).12 Others hypothesize that the essence of knowledge is structure.13 Studies investigating differences between experts and novices in physics14 and chemistry15 support these statements. As individuals develop a progressively sophisticated understanding of a domain, knowledge is increasingly based on fundamental concepts14 and knowledge structures display greater complexity and interrelationships. More complex and interrelated knowledge structures are also related to a deeper understanding of course material12,16−18 and specific outcomes such as course grades in science.19 Conversely, novices are more likely to possess a more disparate organization of facts and concepts, making it difficult to draw connections and ultimately apply knowledge to solve problems. Concept maps are a common classroom technique thought to provide a visual snapshot of an individual’s knowledge structure and have a long history in science education that spans over four decades.9 Concept maps include three parts: concept terms (key ideas/concepts), linking arrows (provide a directional relationship), and linking phrases (explicit connection between two terms). For a review of the theoretical foundations and technical qualities of concept maps, see refs 9 and 20.

■

METHODS

Recruitment and Student Characteristics

This study was situated at an ethnically diverse university located in California. Students traditionally categorized as “underrepresented” in science make up approximately 39% of the school’s population and females outnumber males by an approximate ratio of 6:4. For a more detailed representation of the university’s student demographics, please see Supporting Information S1. Students were recruited during their initial organic chemistry lecture. Interested students completed a questionnaire requesting information on age, gender, and ethnicity. In addition, student transcripts were used to calculate prior science GPA based on prior science courses completed. A total of 90 students participated in our study; 48 female and 42 male students, with 41 Asian, 32 White, and 17 Latino students. First year, second year, third year, fourth year, and postbaccalaureates made up approximately 3%, 24%, 26%, 37%, and 11% of our sample, respectively. This is important to note because first-semester organic chemistry is typically taken during students’ second year, although this is not a mandatory rule across all universities.

Spatial Ability

A third factor to consider when studying problem solving in organic chemistry is spatial ability. Chemists have developed instructional symbols to represent three-dimensional structures in two dimensions (e.g., Fischer projections). This is particularly relevant in stereochemistry, a topic devoted to understanding the spatial arrangement of atoms in a molecule and their associated reaction mechanisms. We use spatial ability to refer to the mental processes that broadly incorporate the ability to represent and manipulate objects in three-dimensional space. Previous studies have reported significant relationships between spatial ability and performance on spatially orientated tasks21,22 as well as nonspatially oriented tasks such as stoichiometry questions.23,24 Pribyl and Bodner10 examined the relationship between spatial ability and problem-solving performance in

Assessment Materials

Organic Chemistry Learning Units and Problem Sets. Assessments covered four learning units previously and unanimously identified by professors as important to the course: Learning Unit 1 − Structure and Bonding; Learning Unit 2 − Stereochemistry; Learning Unit 3 − Alkyl Halide Reactions; Learning Unit 4 − Reactions of Alkenes. Problem set questions were collected from two sources: commonly used organic chemistry textbooks (see Supporting Information S2 for textbook information) and the Web-Based 977

dx.doi.org/10.1021/ed400696c | J. Chem. Educ. 2014, 91, 976−981

Journal of Chemical Education

Article

Enhanced Learning Evaluation And Resource Network (WE-LEARN).27 WE-LEARN is a repository containing a variety of previously evaluated organic chemistry assessment items. Draft problem sets were created for each learning unit and sent to two organic chemistry professors to select the most salient questions students would likely encounter in a typical organic chemistry course. Feedback was used to refine problem sets until agreement was reached with respect to content coverage and degree of difficulty. Box 1 provides a pair of sample questions that vary by difficulty. Example 1 in Box 1 asks students to organize atoms in Box 1. Sample problem set questions intended to assess factual knowledge (example 1) and procedural and conceptual knowledge (example 2)

Figure 1. Representative Item from Purdue Visualization Of Rotations Test. Image reprinted from ref 21. Copyright 2011 American Chemical Society.

Example 1: Factual Knowledge (unit 1, question 3) Put the following elements in order of their electronegativity, giving the most electronegative element the ranking of 1 and the least electronegative element the ranking of 4. N O C F Example 2: Procedural and Conceptual (unit 1 question 15) Cyclopropane (C3H6, a three-membered ring) is more reactive than most other cycloalkanes. (a) Draw a Lewis structure for cyclopropane. (b) Suggest why cyclopropane is so reactive.

the PVOR test. Students are presented with two objects that vary by spatial orientation. Students are then presented with a different shape and asked to apply the same rotation in order to match it to the corresponding multiple-choice option. The PVOR was completed during students’ initial interview. Coding and Scoring

Problem Sets. Problem sets were scored using a correct, incorrect, or partially correct scoring system. One point was awarded for a correct response, zero points for an incorrect response, and one-half a point for a partially correct response. The number of points was summed and recorded for each student. Supporting Information S3 provides examples of our coding scheme. Concept Maps. Concept maps were scored based on proposition accuracy. Propositions are thought to be the smallest unit of meaning in concept maps that can be used to judge the relationship between two terms.28 Two organic chemistry Ph.D. candidates evaluated the scientific accuracy of student-generated concept map propositions. Scientific accuracy was defined as the overlap between students’ responses and scientifically accepted relationships. The following four-level ordinal scale (with underlying continuum) was used: 0 − incorrect or scientifically irrelevant, 1 − partially incorrect, 2 − technically correct but scientifically “thin” or vague, and 3 − scientifically correct and scientifically stated.29 Each student-generated proposition within a concept map received a score between 0 and 3 points. Proposition scores were then summed to calculate a final concept map score for each student. Spatial Visualization Test. PVOR test items were scored as correct or incorrect. One point was awarded for each correct response. Scores were calculated using the sum of a student’s test points.

decreasing order of electronegativity and is intended to assess factual knowledge. Example 2 in Box 1 provides the compound name and molecular formula. The question asks students to draw the Lewis structure (part A) and explain why the compound is so reactive (part B). Part A is intended to assess procedural knowledge (procedure used to draw a Lewis structure), whereas part B is intended to assess conceptual knowledge (understanding why the compound’s structure is related to its reactivity). Overall, Learning Unit 1 consisted of 15 organic chemistry problems. Learning Units 2, 3, and 4 contained 10 problems each. Organic Chemistry Concept Maps. Concept maps were created using the following procedure. The same commonly used organic chemistry textbooks noted above were reviewed to identify key concepts and terms across learning units. In addition, two organic chemistry professors generated a list of concepts and terms they believed to be essential for the same learning units. Researcher- and professor-generated terms were cross-tabulated and overlapping terms were included. A draft list was sent to the same organic chemistry professors for feedback and revised until 10−14 terms were agreed upon. In total, Learning Unit 1 contained 14 terms, Unit 2 contained 10, Unit 3 contained 12, and Unit 4 contained 11 concept map terms. For a list of concept map key terms by learning unit, see Supporting Information (S1). Spatial Visualization Test. The Purdue Visualization of Rotations (PVOR)24 test was used to assess a facet of students’ spatial abilities, namely, spatial visualization. The PVOR is a previously validated test for measuring spatial visualization that the authors define as the ability to “disembed relevant information from a complex drawing or restructure this information” (p 5).24 For example, Figure 1 provides a representative question from

Analyses

Our research questions were analyzed using multivariate regression analyses. In order to examine research question 1, a regression model was computed using average problem-solving scores across learning units as the outcome variable and prior science GPA, average concept map scores, PVOR scores, gender, and ethnic group membership as predictor variables. Research question 2 was investigated using hierarchical multiple regression. A hierarchical model allows more flexibility for entering variables, which is particularly helpful when prior research exists on variables of interest. This approach allows the user to control for the influence of certain predictor variables, which is relevant because we were interested in identifying 978

dx.doi.org/10.1021/ed400696c | J. Chem. Educ. 2014, 91, 976−981

Journal of Chemical Education

Article

In total, each student completed one visualization test, four concept maps and four problem sets (one per unit). Interviews were conducted one week after the specific unit was covered in lecture, with the exception of unit 1 which took place two weeks after students’ initial course lecture. All interviews followed the same procedure with the exception of interview 1, which was scheduled for 2 h to introduce the students’ requirements. It should be noted that students’ performance on our assessments did not impact their course grades.

whether or not gender and ethnicity contribute significantly to the model. Procedure

Student interviews were conducted over two separate semesters (Fall 2009 and Spring 2010) outside of class time and used to train and assess students. The initial interview session began with the PVOR test. Students were given a brief description of the test and two practice questions to familiarize them with the procedure before starting. Students were then given 20 min to complete as many questions as possible. Following the completion of the PVOR, students began concept map training. Training started by describing the key parts of a concept map and how to properly construct a map. Students then completed a practice map on the water cycle (terms did not overlap with organic chemistry terms). Only after this training period were students allowed to construct an organic chemistry concept map. Lastly, students were asked to begin their organic chemistry problem set. A 3 min time limit per problem was applied to increase the likelihood of students completing the assessment by the end of their interview session (with multiple-part problems receiving 3 min per part). Although students seldom reached the time limit, it is unclear what influence this time limit had on students’ problem-solving performance.

■

RESULTS This study set out to investigate the overall contribution of a set of key factors (research question 1) and the most salient factor(s) that influence problem-solving performance (research question 2). Findings are reported below. Multivariate Regression Models

A multivariate regression incorporating all predictor variables was performed to examine research question 1 (see Table 1). Results showed that the model significantly predicted students’ problem-solving performance, F(6,83) = 9.89, p ≤ 0.01, and accounted for approximately 38% of the variance in problem solving (adjusted R2 = 0.38). Only concept maps scores (mean = 17.03, s.d. = 9.68) significantly added to the model’s ability to predict performance (β = 0.51, p ≤ 0.01). A separate hierarchical multiple regression analysis was performed to examine research question 2. This analysis incorporated variables in different blocks to control for previously entered factors. Table 2 provides a summary of the hierarchical regression results. Findings showed that prior science achievement (mean = 2.81, s.d. = 0.67; β = 0.42, p ≤ 0.01) contributed significantly and accounted for approximately 19% of the variation in problem-solving performance, F(5,84) = 5.10, p ≤ 0.01. Model 4, which incorporated concept map scores and controlled for previously entered variables, showed that concept map scores (mean = 17.03, s.d. = 9.68) significantly predicted problem-solving performance, F(6,83) = 9.90, p ≤ 0.01. Only the concept map coefficient showed a significant relationship with problem-solving performance (β = 0.51, t[83] = 5.12, p ≤ 0.01) and accounted for approximately 38% of the variance in students’ problem-solving performance (adj. R2 = 0.38). No other factors significantly contributed to problemsolving performance. Therefore, knowledge structures, as

Table 1. Regression Analysis Summary for Problem-Solving Performancea Variable

B

SEB

β

Constant Average CM Prior Science GPA Spatial Visualization Gender White vs Asian White vs Latino R R2 Adj. R2

2.50 0.12 0.56 0.01 −0.39 −0.43 0.13 0.65 0.42 0.38

1.53 0.02 0.34 0.05 −0.43 0.45 0.59

0.51b 0.17 0.00 −0.09 −0.09 0.02

a

Average CM = average concept map scores; B = unstandardized coefficients; SEB = standard error of the coefficient; β = standardized coefficient; R = multiple correlation coefficient; R2 = squared correlation coefficient; Adj. R2 = adjusted squared correlation coefficient. bp ≤ 0.01.

Table 2. Summary of Hierarchical Regression Analysis of Problem-Solving Performance on Five Predictor Variablesa Model 1 Variable

B

SEB

Constant White vs Asian White vs Latino Gender Prior Science GPA Spatial Visualization Avg. CM R R2 Adj. R2 ΔR2

6.04 −1.10 −0.70

0.40 0.53 0.67

0.22 0.05 0.03 0.05

Model 2 β

B

SEB

−0.24c −0.12

7.15 −1.19 −0.90 −0.67

0.89 0.53 0.68 0.48

Model 3 β −0.26c −0.16 −0.15

0.26 0.07 0.04 0.02

B

SEB

2.11 −0.72 −0.14 −0.45 1.41 0.03

1.74 0.51 0.67 0.49 0.34 0.06

0.48 0.23 0.19b 0.16b

Model 4 β −0.16 −0.02 −0.10 0.42b 0.06

B

SEB

β

2.50 −0.43 0.13 −0.39 0.56 0.00 0.12 0.65 0.42 0.38b 0.18b

1.53 0.45 0.57 0.43 0.34 0.05 0.02

−0.09 0.02 −0.09 0.17 0.01 0.51b

a Avg. CM = average concept map scores; B = unstandardized coefficients; SEB = standard error of the coefficient; β = standardized coefficient; R = multiple correlation coefficient; R2 = squared correlation coefficient; Adj. R2 = adjusted squared correlation coefficient; ΔR2 = change in squared correlation coefficient. bp ≤ 0.01. cp ≤ 0.05.

979

dx.doi.org/10.1021/ed400696c | J. Chem. Educ. 2014, 91, 976−981

Journal of Chemical Education

Article

in student understanding that can be used to adjust instruction accordingly.29 This is especially important considering early evidence suggesting conceptual changes are more likely to occur within the first four weeks of class.30 Findings indicate that prior science knowledge did not contribute significantly to the regression model’s ability to predict problem-solving performance. As stated earlier, it may be that first-semester organic chemistry uniquely focuses on qualitative problem solving skills, unlike many prior chemistry courses which focus on quantitative problem-solving skills (e.g., general chemistry). We do not suggest that students’ prior science knowledge does not overlap with content learned in organic chemistry. After all, electronegativity is a key topic in both general and organic chemistry. However, the same concept may be applied in different ways to solve problems in each class. An alternative explanation may be related to our measure of prior science knowledge. Prior science GPA was used to estimate prior science knowledge. GPA is a composite score that takes into account numerous variables. Some relate more directly to knowledge (e.g., midterm and final exams), whereas others may do so indirectly (e.g., course attendance and participation). Future work may benefit by using more direct assessments of prior knowledge (e.g., validated concept inventories). More research is needed to clarify this finding. Findings also suggest that focusing resources (e.g., class time) on developing students’ spatial−visual abilities may not translate into improved problem-solving performance. Previous work has demonstrated that students can apply learned heuristics in isolation or in conjunction with spatial strategies when problem solving.5 However, we must be cautious about extending to spatial ability more generally. The PVOR is an assessment of spatial visualization, which is one facet of spatial ability. Other facets such as visual discrimination and visual memory may play a significant role in predicting students’ problem-solving performance. More research is needed to investigate the relationship between spatial abilities and problem-solving performance across key topics in organic chemistry. In conclusion, the purpose of this study was to examine the contribution of a core set of factors on students’ overall problemsolving performance in first-semester organic chemistry. Results indicate that knowledge structures were the single best predictor of problem-solving performance, even after controlling for prior science knowledge, spatial visualization, gender, and ethnicity. Findings highlight knowledge structures (as measured by concept maps) as a promising area for future research on improving students’ problem-solving performance and, ultimately, course success.

measured by concept maps, appear to be the most influential predictor of students’ problem-solving performance in organic chemistry, even after controlling for prior science achievement, spatial visualization, gender, and ethnicity. One argument may be that spatial visualization mainly accounts for problems that require the use of spatial manipulation. That is, spatial visualization was not significant because average problem-set scores included problems that do not require the use of spatial manipulations, ultimately masking possible relationships. In order to explore this possibility, a separate linear regression analysis was performed to examine the contribution of spatial visualization scores on unit 2 (Stereochemistry). If spatial visualization is a significant predictor of spatial problem-solving performance, then Unit 2 would be the most likely unit to display this relationship. On average, results showed that males scored significantly higher than females (malemean = 14.50, s.d. = 3.90; femalemean = 11.23, s.d. = 3.98; t(88) = 3.93, p ≤ 0.01), but spatial visualization scores did not contribute significantly to predicting students’ performance on unit 2, F(1,79) = 2.28, p > 0.05. Therefore, spatial visualization, as measured by the PVOR, does not predict problem-solving performance among our sample. In summary, multivariate regression analyses show that concept-map scores are the single most important predictor of students’ problem-solving performance and account for a reasonable proportion of variance in problem solving. This finding emphasizes the potential value of knowledge structures as a target for interventions aimed at improving students’ problemsolving performance.

■

DISCUSSION AND CONCLUSIONS The overarching goal of this study was to examine the contribution of key factors on students’ problem-solving performance in first-semester organic chemistry. Our findings showed knowledge structures (as measured by concept maps) were the strongest predictor of problem-solving performance, beyond all remaining factors. One possible explanation is that concept maps facilitate the storage of information from short-term and “working-memory” to long-term memory. Information is organized and processed in working memory by interaction with knowledge in long-term memory.9 The limiting step is that working memory quickly reaches its capacity (recall the magic number 7 ± 2). However, if new knowledge is integrated with prior knowledge, individuals can extend this capacity and recall more information to assist in problem solving. Novak and Cañas9 propose that large bodies of knowledge require an orderly sequence of iterations between working memory and long-term memory as new knowledge is received. Although more research is needed to explore the mechanism by which this occurs, prior investigations support the positive relationship between knowledge structures and problem-solving assessments.16 Organic chemistry instructors can benefit by using concept maps as an evaluation or learning tool. Rather than (or in conjunction to) assigning practice problems, instructors can ask students to create concept maps using specific key terms or commonly misunderstood terms. Instructors can designate a portion of class or recitation time to discuss their assigned concept maps with peers (incorporating aspects of peer-to-peer learning) or generate a whole-class concept map as an endof-unit summary. Doing so may facilitate understanding by providing students the opportunity to organize and connect potentially disparate pieces of information. Concept maps can also yield valuable information about potential knowledge gaps

■

ASSOCIATED CONTENT

S Supporting Information *

University and student body demographics, detailed methodology, and samples of our problem-solving scoring system are provided.This material is available via the Internet at http:// pubs.acs.org.

■

AUTHOR INFORMATION

Corresponding Author

*E-mail: [email protected]. Present Address ¶

School of Education, University of Colorado at Boulder, Boulder, Colorado, 80302, United States.

980

dx.doi.org/10.1021/ed400696c | J. Chem. Educ. 2014, 91, 976−981

Journal of Chemical Education

Article

Notes

Assessment Tool in Organic Chemistry: Implications for Teaching. Chem. Educ. Res. Pr. 2010, 12, 133−141. (20) Ruiz-Primo, M. A.; Schultz, S. E.; Li, M.; Shavelson, R. J. Comparison of the Reliability and Validity of Scores from Two Conceptmapping Techniques*. J. Res. Sci. Teach. 2001, 38, 260−278. (21) Harle, M.; Towns, M. A Review of Spatial Ability Literature, Its Connection to Chemistry, and Implications for Instruction. J. Chem. Educ. 2011, 88, 351−360. (22) Wu, H. K.; Shah, P. Exploring Visuospatial Thinking in Chemistry Learning. Sci. Educ. 2004, 88, 465−492. (23) Carter, C. S.; Larussa, M. A.; Bodner, G. M. A Study of Two Measures of Spatial Ability as Predictors of Success in Different Levels of General Chemistry. J. Res. Sci. Teach. 1987, 24, 645−657. (24) Bodner, G. M.; Guay, R. B. The Purdue Visualization of Rotations Test. Chem. Educ. 1997, 2, 1−17. (25) Bodner, G. M. Mental Models: The Role of Representations in Problem Solving in Chemistry. Univ. Chem. Educ. 2000, 4, 24. (26) Turner, R. C.; Lindsay, H. A. Gender Differences in Cognitive and Noncognitive Factors Related to Achievement in Organic Chemistry. J. Chem. Educ. 2003, 80, 563. (27) Penn, J. H.; Nedeff, V. M.; Gozdzik, G. Organic Chemistry and the Internet: A Web-Based Approach to Homework and Testing Using the WE-LEARN System. J. Chem. Educ. 2000, 77, 227−231. (28) Ruiz-Primo, M. A.; Shavelson, R. J. Problems and Issues in the Use of Concept Maps in Science Assessment. J. Res. Sci. Teach. 1996, 33, 569−600. (29) Yin, Y.; Vanides, J.; Ruiz-Primo, M. A.; Ayala, C. C.; Shavelson, R. J. Comparison of Two Concept-mapping Techniques: Implications for Scoring, Interpretation, and Use. J. Res. Sci. Teach. 2005, 42, 166−184. (30) Pearsall, R. N.; Skipper, J. E. J.; Mintzes, J. J. Knowledge Restructuring in the Life Sciences: A Longitudinal Study of Conceptual Change in Biology. Sci. Educ. 1997, 81, 193−215.

The authors declare no competing financial interest.

■

ACKNOWLEDGMENTS This research was supported by funding from the National Science Foundation, Division of Research on Learning in Formal and Informal Settings under NSF Grant No. 0814559. Any opinions, conclusions, or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the National Science Foundation. The authors would also like to thank Vincent Basile for his comments on this manuscript.

■

REFERENCES

(1) Bodner, G. M.; Herron, J. D. Problem-Solving in Chemistry. In Chemical Education: Research-based Practice; Gilbert, J. K., De Jong, O., Justi, R., Treagust, D. F., Van Driel, J. H., Eds.; Kluwer Academic Publishers: Norwell, MA, 2002; pp 235−266. (2) Discipline-Based Education Research: Understanding and Improving Learning in Undergraduate Science and Engineering; Singer, S. R., Nielsen, N. R., Schweingruber, H. A., Eds.; The National Academies Press: Washington, DC, 2012. (3) Novak, J. D. Learning, Creating, and Using Knowledge: Concept Maps as Facilitative Tools in Schools and Corporations, 2nd ed.; Routledge: New York, 2010. (4) Seery, M. K. The Role of Prior Knowledge and Student Aptitude in Undergraduate Performance in Chemistry: a Correlation-prediction Study. Chem. Educ. Res. Pract. 2009, 10, 227−232. (5) Stieff, M. Sex Differences in the Mental Rotation of Chemistry Representations. J. Chem. Educ. 2013, 90, 165−170. (6) Wheatley, G. H. Problem solving in school mathematics. MEPS Technical Report 84.01; School Mathematics and Science Center, Purdue University: West Lafayette, IN, 1984. (7) Bodner, G. M. A Cultural Approach to Problem Solving. Educ. Quim. 2005, 16, 222−229. (8) Ausubel, D. P. Educational Psychology: A Cognitive View; Holt, Rinehart and Winston: New York, 1968. (9) Novak, J. D.; Cañas, A. J. The Theory Underlying Concept Maps and How to Construct and Use Them. Technical Report IHMC CmapTools 2006-01 Rev 01-2008; Florida Institute for Human and Machine Cognition: Pensacola, FL, 2006. (10) Pribyl, J. R.; Bodner, G. M. Spatial Ability and Its Role in Organic Chemistry: A Study of Four Organic Classes. J. Res. Sci. Teach. 1987, 229−240. (11) Chandran, S.; Treagust, D. F.; Tobin, K. The Role of Cognitive Factors in Chemistry Achievement. J. Res. Sci. Teach. 1987, 24, 145−160. (12) Bransford, J. D.; Brown, A. L.; Cocking, R. How People Learn; National Academy Press: Washington, DC, 2000. (13) Anderson, R. C. Some Reflections on the Acquisition of Knowledge. Educ. Res. 1984, 13, 5−10. (14) Chi, M. T. H.; Feltovich, P. J.; Glaser, R. Categorization and Representation of Physics Problems by Experts and Novices. Cognit. Sci. 1981, 5, 121−152. (15) Heyworth, R. M. Procedural and Conceptual Knowledge of Expert and Novice Students for the Solving of a Basic Problem in Chemistry. Int. J. Sci. Educ. 1999, 21, 195−211. (16) Francisco, J. S.; Nakhleh, M. B.; Nurrenbern, S. C.; Miller, M. L. Assessing Student Understanding of General Chemistry with Concept Mapping. J. Chem. Educ. 2002, 79, 248−257. (17) Hay, D. B. Using Concept Maps to Measure Deep, Surface and Non-learning Outcomes. Stud. Higher Educ. 2007, 32, 39−57. (18) Meagher, T. Looking Inside a Student’s Mind: Can An Analysis of Student Concept Maps Measure Changes in Environmental Literacy. Electron. J. Sci. Educ. 2009, 13, 1−28. (19) Lopez, E.; Kim, J.; Nandagopal, K.; Cardin, N.; Shavelson, R. J.; Penn, J. H. Validating the Use of Concept-mapping as a Diagnostic 981

dx.doi.org/10.1021/ed400696c | J. Chem. Educ. 2014, 91, 976−981