Chapter 6
Math Self-Beliefs Relate to Achievement in Introductory Chemistry Courses
It’s Just Math: Research on Students’ Understanding of Chemistry and Mathematics Downloaded from pubs.acs.org by UNIV OF ROCHESTER on 05/14/19. For personal use only.
Michael R. Mack,* Cynthia A. Stanich, and Lawrence M. Goldman Department of Chemistry, University of Washington, Seattle, Washington 98195, United States *E-mail:
[email protected] Past research has demonstrated that the math component of the SAT correlates with exam scores and course grades in undergraduate chemistry courses. This chapter explores the effect of students’ math abilities on their achievement in chemistry and the mediating role of math self-beliefs underlying this process. Using data from student responses to the Attitudes Toward Mathematics Inventory and exam outcomes from preparatory and general chemistry courses, this chapter first characterizes students’ math self-beliefs reported at the beginning of the term. Then, the relationship between math self-beliefs and chemistry achievement is explored using structural equation modeling. Specifically, we examine the extent to which the math ability–chemistry achievement relationship is mediated, if at all, by math self-beliefs. Implications for future research and teaching addressing mathematical reasoning in chemistry are discussed in light of self-belief theories.
Introduction Measurements of students’ math ability have repeatedly been shown to correlate with achievement in undergraduate chemistry courses (1–8). The work in this body of literature embodies an important theoretical relationship that chemistry educators value for helping students develop science literacies, namely, that knowledge and skills in mathematics are pragmatic for enabling learning in the sciences and are central to the participation in and development of scientific fields (9). However, the correlation between a student’s math ability and chemistry achievement is not perfect and researchers are still working to understand the complexities of modeling variation in students’ achievement based on their past performances and related cognitive and psychological variables (7, 8). From a self-beliefs perspective, students with similar math ability may feel differently about themselves in terms of which attributes they think they possess and how they self-assess their own performance in relation to mathematical tasks (10, 11). In the context of introductory college chemistry, students enter the curriculum with beliefs about themselves that have been shaped by © 2019 American Chemical Society
years of past experiences and achievements in education. These self-beliefs are theorized to play a determining role in one’s further development in a domain (10–12). Self-confidence, self-concept, and self-efficacy are examples of self-beliefs that have been used by educational researchers to understand the factors determining students’ achievement in school settings. Some researchers have measured students’ confidence to do math based on responses to statements such as “I have a lot of self-confidence when it comes to mathematics” (13, 14). In contrast, math self-concept is a global belief formed about one’s self in relation to mathematics as an academic subject (e.g., “I hate mathematics”), and it also refers to general perceptions about one performing mathematics (e.g., “I often excel in mathematics courses”) (11). Math self-efficacy beliefs are malleable self-judgments that individuals make about what they can do with the knowledge and skills they have in a particular context (10, 11, 15). One of the biggest distinctions between academic self-concept and self-efficacy is the context in which self-beliefs are elicited. When measuring academic self-efficacy, researchers generally present students with a particular task and request they make a judgment about their knowledge and abilities for solving the problem (e.g., “I could use the quadratic formula to solve for [H+] if I needed to”). From the perspective of self-beliefs, when students struggle with mathematics in chemistry it could mean one of two things: (1) a lack of the prerequisite knowledge and skills necessary to demonstrate competency in chemistry or (2) negatively formed self-beliefs about their math-specific knowledge and skills and how they might apply those skills when reasoning about chemical phenomena. Therefore, self-beliefs related to mathematics may prove useful as theoretical constructs on which to build on our understanding of the relationship between math-related knowledge and skills and achievement in undergraduate chemistry courses. Research Questions The goal of this study was to quantitatively model the influence of math self-beliefs on the math ability–chemistry achievement relationship. A key assumption guiding this study is that the content and skills needed to perform well in introductory chemistry courses are highly dependent on prerequisite mathematical knowledge and skills. To better understand the relationship between math self-beliefs and chemistry achievement, we conducted an observational study guided by the following research questions: 1. What are preparatory and general chemistry students’ math self-beliefs as measured by the Attitudes Toward Mathematics Inventory? 2. What is the relationship between math self-beliefs and chemistry achievement? 3. To what extent do math self-beliefs explain the relationship between the math component of the SAT (SAT-M) and chemistry achievement? Before we discuss the methods used to answer these questions, we first situate this study in the broader literature relating math ability and self-beliefs to academic achievement.
Literature Review SAT Math and In-House Mathematics Assessments Predict Chemistry Grades Decades of research has repeatedly shown that, in general, SAT-M is linearly related to academic performance in general chemistry settings. 82
Table 1. Summaries of Quantitative Studies Investigating the Relationship Between Achievement in Chemistry Courses and Math Ability Reference
Relationships studied
Regression-based study
Summary
Pickering (1)
Chemistry course letter grades No and SAT-M
Positive trends
Andrews and Andrews (2)
Chemistry course letter grades Yes and SAT-M w/ covariates
Positive trends, regression data not reported
Spencer (3)
Chemistry course letter grades No and SAT-M
Positive trends
Ozsogomonyan and Loftus (4)
Chemistry course grade and SAT-M w/covariates
Standardized coefficient: 0.39
Lewis and Lewis (5)
ACS Exam score (%) and SATYes M w/covariates
Unstandardized coefficient: 0.09
Xu et al. (6)
ACS Exam score (%) and SATYes M w/covariates
Standardized coefficient: 0.36
Ralph and Lewis (7)
ACS Exam score (%) and SATYes M w/covariates
Unstandardized coefficient: 0.11–0.14
Russell (19)
Chemistry course grade and inhouse math assessment w/ Yes covariates
Standardized coefficient: 0.18
Leopold and Edgar (20)
Chemistry course grade and inYes house math assessment
Unstandardized coefficient: 0.14
Vincent-Ruz et al. (8)
Chemistry course grade and SAT-M w/covariates
Unstandardized coefficient: 0.20–0.25
Yes
Yes
Table 1 summarizes the results of select quantitative studies correlating SAT-M scores to performance outcomes in undergraduate chemistry courses from 1975 to 2018. Many of the early studies demonstrated positive associations between SAT-M scores and grades in general chemistry courses (1–3). A notable trend reported in these correlation studies was “[a] high math SAT was not a guarantee of a good grade, but a low math SAT was a strong indicator of a low grade” (2). Ozsogomonyan and Loftus conducted a multivariate regression of SAT-M scores and grades in an entry-level chemistry course while adjusting for scores on a chemistry pretest and high school chemistry grades (4). Lewis and Lewis (5) used SAT scores to develop predictive models for identifying students performing below the 30th percentile on the first-semester general chemistry exam SP97A from the ACS Examinations Institute (16). Using multivariate regression, the authors reported a positive relationship between SAT-M and ACS exam scores while controlling for other performance-relevant characteristics. The authors found that a substantial portion of students were not accurately identified as at-risk based on model predictions, leading the authors to conclude “a need to include non-cognitive predictors, such as affective measures like motivation or confidence” to more accurately predict performance outcomes in general chemistry (5). Later, Xu et al. used structural equation modeling to examine the relations between math ability, prior chemistry content knowledge, attitudes toward chemistry, and achievement on students’ chemistry achievement in a first-term general chemistry course (6). The authors observed a positive relationship between 83
students’ SAT-M scores and scores on the First-Term General Chemistry Blended Examination from the ACS Examinations Institute (17). When data on students’ prior chemistry content knowledge was not included in the model, the effect of SAT-M on chemistry achievement was significantly overestimated, leading the authors to conclude, “[a]lthough math ability is often used as a proxy for cognitive factors to predict chemistry achievement in the literature, the importance of student conceptual knowledge should not be diminished” (6). Ralph and Lewis used multivariate regression to identify at-risk students in a first-semester general chemistry course (7). Based on separate models across four terms, the authors reported a positive relationship between SAT-M and performance on the 2015 First Term General Chemistry ACS Exam (18). The authors stated that the “ACS exam score was predicted to improve by 11% for every 100 points scored on the SAT-M for students of Spring semesters and 13–14% for students of the Fall semesters” (7). Scores on in-house assessments have also been correlated with general chemistry grades. For example, Russell (19) showed that course grades in an entry-level chemistry course were predicted by performance on an in-house math assessment together with an assessment of students’ quantitative chemistry problem skills and chemistry content knowledge. Similarly, Leopold and Edgar (20) reported a strong positive relationship between an in-house assessment of math ability and grades in a general chemistry course. Coefficients from regression analyses relating scores on in-house math assessments to chemistry achievement are reported in Table 1. A common narrative among these observational studies is that math ability, often measured by SAT-M scores or in-house assessments, is a powerful predictor of chemistry achievement. Measures of math ability have been frequently observed to correlate with exam scores and course grades alongside other performance-related covariates. Studies have also demonstrated that correlations are particularly strong at the lower end of the SAT-M score distribution, meaning that students with relatively low SAT-M scores were accurately predicted to have lower achievement outcomes more often than high SAT-M performers were predicted to have higher outcomes. In other words, chemistry achievement outcomes are more dispersed for students with higher SAT-M scores. This variation led some of the authors to theorize that additional variables might play a role in orchestrating mathematical knowledge and skill productively in general chemistry settings (5, 20). In the next section, we summarize a collection of studies that explored the role of self-beliefs for understanding the math ability–achievement relationship in mathematics and chemistry contexts. Self-Beliefs Predict Mathematics and Chemistry Achievement In addition to reporting correlations between performance on an in-house math assessment and chemistry grades, Leopold and Edgar (21) investigated students’ misconceptions related to logarithms, scientific notation, graphical interpretation, and algebra. The authors noted that in some cases, students “lacked [a] prerequisite degree of mathematics fluency,” but in other instances, the authors speculated that negative math self-efficacy beliefs formed a “misconception that can seriously impede students’ science education and may be more difficult to eradicate” than limited mathematics fluency (20). What is the evidence that math self-beliefs are related to academic achievement? Goolsby et al. found math self-confidence, SAT scores, and high school GPA jointly accounted for 17 percent of the variance in grades in a developmental algebra course at the college level (21). Randhawa et al. found mathematics self-efficacy and attitudes to be a significant predictor of math (algebra) achievement for a sample of high school students (22). Pajares and Miller surveyed students about their mathematics self-efficacy and found that confidence in one’s ability to do math positively
84
correlated with performance on a set of mathematics problems immediately following the survey (23). Math self-beliefs have also been correlated with achievement in chemistry. In one study, students in introductory physics and chemistry courses at a community college in central Mississippi were asked to rank how confident they were about different types of mathematics problems based on an example problem of that type. Quinn found that students who reported higher levels of confidence to transfer basic math skills to science also reported higher exam grades than did students who reported lower confidence ratings (24). Chan and Bauer (25) surveyed students’ academic selfefficacy and test anxiety using the Motivated Strategies for Learning Questionnaire (26), chemistry and mathematics self-concept using the Chemistry Self-Concept Inventory (27), and emotional satisfaction and intellectual accessibility using both versions of the Attitudes Toward the Subject of Chemistry Inventory (28, 29). In line with theory, the authors provided evidence that more negative self-beliefs were predictive of lower achievement in chemistry, whereas more positive self-beliefs were predictive of higher achievement. Vincent-Ruz et al. hypothesized the math ability–chemistry achievement relationship to be mediated by chemistry and math self-concept beliefs (8). In this context, mediation refers to a variable having an influence on the relationship between two other variables. Based on a path analysis, the authors found evidence of chemistry self-concept beliefs mediating the math ability–chemistry achievement relationship in their setting, but the mediating role of math self-concept beliefs was inconclusive. Summary and Implications for This Study As discussed previously, studies that correlated measures of math ability—either in the form of SAT-M or in-house assessments—to achievement in chemistry have a long history among chemistry educators. These studies contributed empirical evidence for the theoretical relationship between math ability and achievement in chemistry courses in order to make the case that a student’s prior attainment in mathematics is important for performing well on traditional assessments in general chemistry (i.e., exams or course grades). Interpreting these studies through the lens of self-beliefs, how students feel about themselves in terms of math ability, can play a determining role in their performance during evaluative situations (10, 11, 15). The practice of correlating measures of math self-beliefs to achievement in undergraduate chemistry education is a more recent tradition than is just examining metrics of prior academic attainment (e.g., SAT scores). One recent study provided evidence for the role of chemistry self-concept beliefs to partially explain the math ability–chemistry achievement relationship, but evidence for the role of math self-concept beliefs was inconclusive (8). In this chapter, we present the development and evaluation of models for estimating the effects of math ability and self-beliefs on chemistry achievement to further interrogate the role of math selfbeliefs for understanding the math ability–chemistry achievement relationship.
Methods We developed and tested models of relations between math ability, self-beliefs, and achievement in introductory chemistry courses in order to understand the extent to which math self-beliefs mediated the math ability–chemistry achievement relationship in the context of preparatory and general chemistry courses at the University of Washington, Seattle campus (UW). This study was approved by the Human Subjects Division in the Office of Research at the UW.
85
Setting We sampled students from preparatory chemistry (PrepChem), first-quarter general chemistry (GC1), and second-quarter general chemistry (GC2) courses. The UW is a large, public, researchintensive university with a chemistry department that serves more than 3,500 students each year in introductory and general chemistry courses designed for students majoring in science, engineering, and prehealth fields. PrepChem is designed for students who do not have a high school background in chemistry. Students enroll in this course if they are advised to or if they do not pass or they decline to take the departmental placement exam. The general chemistry sequence follows an “atoms-first” curriculum. The first-quarter course covers atomic theory, quantum mechanics, bonding, stoichiometry, gases, and kinetics. Students are permitted to enroll in GC1 by either completing PrepChem, taking Advanced Placement or International Baccalaureate chemistry in high school, or passing the placement exam. The second-quarter course covers equilibrium, acids and bases, thermodynamics, and electrochemistry. Each course has an integrated laboratory component. Participants Students from 14 sections of PrepChem, GC1, and GC2 were recruited as participants for this study. A fair balance of on- and off-sequence sections of GC1 were sampled for this analysis. However, only off-sequence sections of GC2 were available for sampling. Off-sequence offerings of GC2 are disproportionately populated with students who originally enrolled in PrepChem. We note our unbalanced sampling strategy is a limitation to the results of this study because it threatens the generalizability of our results to on-sequence GC2 student populations. We discuss this more in the Limitations section. Students in each section were surveyed as part of normal classroom participation and received participation points for their time and effort. Survey responses were collected during the first 10 days of class. Exam score records were collected at the completion of each course in ordinance with our institutional review board–approved data collection procedures. The final sample size after removing observations with missing records was N = 2,561 (85% of enrolled students). The sample size disaggregated by course was 650, 1,229, and 682 for PrepChem, GC1, and GC2, respectively. Measures Math Ability We used the SAT-M, a college-entrance, standardized test administered by the College Board, as a measure of prior attainment in mathematics. As stated by the test developers, the SAT-M is intended to collect evidence regarding students’ “fluency with, understanding of, and the ability to apply the mathematical concepts, skills, and practices that are most strongly prerequisite and central to their ability to progress through a range of college courses, career training, and career opportunities” (30). Validity evidence supports the notion that SAT tests are indicative of college achievement (31, 32). Scores range from 200 to 800. In 2016, the College Board revised the SAT-M and eliminated the penalty for selecting incorrect answers while keeping the original 200–800 scale (30). This is likely to change many of the test’s properties, therefore we converted pre-2016 SAT-M scores to an equivalent post-2016 SAT-M score using established concordance tables published by the College Board (33). For students with ACT records, ACT math scores were concorded to the redesigned SAT-M scale, again using established concordance tables published by the College Board. Because the ACT has stayed relatively consistent 86
over time, we made no distinction in ACT records across years. Finally, the ACT math and SATM scales have been deemed similar by content experts at the College Board and the ACT and score correlations provided sufficient evidence to support a concordance between the math tests (34). Ultimately, every student observation had one score corresponding to math ability on a scale ranging from 200 to 800. Math Self-Beliefs Aspects of math self-beliefs were measured with the short version of the Attitudes Toward Mathematics Inventory (ATMI) (35). The original ATMI was developed as a 49-item five-point Likert response scale that measured six dimensions of attitudes toward mathematics: confidence, anxiety, value, enjoyment, motivation, and parent/teacher expectations (14). After initial development and testing based on a sample from a private bilingual preparatory school in Mexico City, the confidence and anxiety subscales were combined to form a single factor termed “selfconfidence” and the parent/teacher expectations subscale was removed due to low item-to-total correlations, which resulted in a four-factor model (36). Based on a sample of predominantly Chinese students at a university in Singapore, Lim and Chapman shortened the ATMI by removing items that had low factor loadings or high cross loadings (35). In addition, the test developers recommended that further use of the ATMI remove the motivation subscale due to the high correlation with the enjoyment subscale and the more favorable psychometric properties of the resulting model. The short ATMI is a 15-item survey that measures three dimensions of students’ attitudes toward mathematics: enjoyment, value, and self-confidence. According to the original test developers, the enjoyment scale was intended to measure the “degree to which students enjoy working mathematics and mathematics classes”; the value scale was intended to measure “beliefs on the usefulness, relevance and worth of mathematics in their life now and in the future”; and selfconfidence referred to students’ “confidence and self-concept of their performance in mathematics” (37). The wording of the ATMI self-confidence scale items and response distributions can be found in Figure 1. We used the self-confidence scale of the short ATMI as our measure of math self-beliefs because the scale was originally defined by the test developers to target both self-confidence and self-concept; however, the distinction between these two constructs was not clarified. The scale also targets feelings of nervousness and apprehension in relation to mathematics, which can be construed as math anxiety. This is not surprising because the original inventory contained both “selfconfidence” and “anxiety” subscales that were eventually combined during test development (37). Due to the apparent confounding of confidence, self-concept, and anxiety, we refer to the scale more generally as math self-beliefs. We further discuss the confounding of the ATMI subscale in the Limitations section and the implications for future research at the end of the chapter. Achievement in Chemistry To examine achievement in chemistry, we collected midterm and final exam scores from each course in the sample. All exams were in-house, instructor-authored exams with majority multichoice items. Exam scores were standardized by computing z-scores for each course section to account for differences in test difficulty across instructors who taught different sections of the same courses. Midterms were worth 100 points each and final exams were worth 150 points. Typical values for midterm and final exam standard deviations in our setting were approximately 15 and 30 points, respectively. 87
Figure 1. Distribution of responses to the ATMI subscale grouped by students in PrepChem, GC1, and GC2 courses. Item labels (i.e., SC3, SC5, SC7, SC10, and SC13) match those from the original 40-item ATMI as presented in Lim and Chapman (35). Covariates To test the robustness of the relationship between math ability, math self-beliefs, and achievement, we also conducted analyses that included additional indicators of academic preparation. Specifically, we used the verbal component of the SAT (SAT-V) and high school GPA (HSGPA). Scores from the SAT-V prior to 2016 were converted to post-2016 scale scores based on concordance tables published by the College Board (33). The scale ranges from 200 to 800 points. High school GPA data were reported on a 4-point scale. Data Analysis The underlying hypothesis of this study is that math self-beliefs intervene in the relationship between math ability and chemistry achievement. To investigate this hypothesis, we developed and tested process models linking students’ math self-beliefs to their math ability and chemistry achievement within a structural equation modeling (SEM) framework. We detail our statistical techniques below. Descriptive Statistics and Response Patterns Summary statistics for the SAT-M, SAT-V, and HSGPA disaggregated by course are reported in Table 2. The difference in SAT-M scores between GC1 and GC2 is likely an artifact of having 88
an unbalanced sample of GC2 section offerings in “off-sequence” quarters while obtaining a more balanced sample of GC1 section offerings from both on- and off-sequence quarters. Sampling from off-sequence section offerings of GC2 meant that these samples had a disproportionate number of students who originally took PrepChem compared with on-sequence GC1. And as illustrated by the disaggregated data, students in PrepChem had lower SAT-M scores than did students in GC1 and GC2, on average. A similar trend in disaggregated SAT-V scores was observed. Lastly, HSGPA data was highly left-skewed with little variation across preparatory and general chemistry courses. Table 2. Descriptive Statistics for the Continuous Variables Included in the Analysis Disaggregated by Course Variable
SAT-M
SAT-V
HSGPA
Course
Mean
SD
Skew
Kurtosis
PrepChem
603.89
89.92
-0.02
-0.37
GC1
666.56
76.13
-0.44
-0.05
GC2
655.81
79.00
-0.28
-0.52
PrepChem
601.68
84.02
-0.29
-0.51
GC1
649.03
71.67
-0.63
0.45
GC2
639.68
72.11
-0.43
-0.24
PrepChem
3.72
0.24
-1.20
1.41
GC1
3.79
0.19
-1.49
3.58
GC2
3.80
0.18
-1.18
1.38
To allow for visual examination of response patterns, the ATMI subscale items were plotted using horizontal stacked bar plots and are displayed in Figure 1 (38, 39). Response patterns were grouped by course, which are labeled on the left side of the horizontal stacked bars. The categories of Strongly Agree and Agree are shown on the left while Disagree and Strongly Disagree are shown on the right. The higher the agreement, the more negative one’s self-belief of the ability to learn and do mathematics. Agreement with the ATMI statements would be predictive of relatively lower achievement on evaluative tasks that necessitated the application of mathematical knowledge and skills. Therefore, we would expect negative correlations between the ATMI responses and chemistry achievement. We would also predict a negative correlation between SAT-M and the ATMI responses, meaning that a student with a higher SAT-M score would tend to report disagreement with the ATMI statements, all else equal. In order to facilitate a more straightforward analysis using SEM, responses to the ATMI items were reverse-coded so that math ability, self-beliefs, and chemistry achievement were directly proportional. Structural Equation Modeling Structural equation modeling is a multivariate statistical method for examining the fit between competing theoretical models and the observed data while simultaneously estimating the magnitude of causal relationships between multiple variables (40–42). The goal of SEM in this analysis was to examine how well a set of models relating math ability, self-beliefs, and chemistry achievement fit the observed data. Path diagrams of the four models (i.e., Models A–D) examined in this study are displayed in Figure 2. We also used SEM to conduct multivariate regression analyses to estimate the magnitude of relationships between variables. 89
Figure 2. Path diagrams depicting the variables of interest and their causal relations for Models A to D. Latent variables are indicated by an oval shape and observed variables are represented by squares. The latent variable in the measurement model is related to each measured item through the factor loadings, λi, and an error term (δi), which represents everything not explained by the latent factor. For simplicity, error variance terms and variable correlations are not depicted. Each model is composed of two parts. First, the measurement model for the ATMI scale items (e.g., SC3, SC5, SC7, SC10, and SC13) hypothesizes how the responses are influenced by a common construct, which we refer to as math self-beliefs. Latent variables are represented by ovals. The measured variables are represented by squares. Math self-beliefs are related to each of the ATMI scale items through the factor loadings λ1 to λ5 and error terms δ1 to δ5. The error terms represent the variance in the data not explained by the common factor. The second measurement model hypothesizes how performance on each midterm and final exam is influenced by the construct “chemistry achievement.” Chemistry achievement is related to each exam (standardized by course section) through the factor loadings λ6 to λ8 and error terms δ6 to δ8. Error variance terms are not depicted for simplicity, and no residual variances were correlated in this analysis. See Komperda et al. for a more comprehensive review of latent variable models in the context of CER (43). The second component is the structural model, which depicts hypothesized relations between variables (41). For example, in Model A, math self-beliefs are the sole predictor of chemistry 90
achievement. Model B incorporates SAT-M scores into the structural model. The premise of this model is that SAT-M and math self-beliefs both predict chemistry achievement and self-beliefs predict the responses to the five items on the ATMI scale. Model C specifies math self-beliefs and chemistry achievement as outcomes both predicted by SAT-M scores. Finally, within a simple mediation framework, Model D assumes an intervening relationship between SAT-M and chemistry achievement through math self-beliefs. We discuss this mediation model further in the following section. Models A and B will be analyzed using SEM to answer Question 2. Models B–D will be analyzed using SEM to answer Question 3. Furthermore, a corresponding set of models adjusted for SATV and HSGPA covariates were examined to test the robustness of model parameter estimates (path diagrams not shown). These are referred to as Models B′–D′ and they are covariate-adjusted forms of Models B–D, respectively. Mediation Analysis Model D is a simple mediation model (44). The premise of Model D is that SAT-M exerts an effect on chemistry achievement as well as do math self-beliefs, which in turn affects chemistry achievement. The analysis of Model D using SEM allows for inferencing on the role of self-beliefs for explaining the math ability–chemistry achievement relationship. Thus, the goal of this study was to estimate the mediation of the math ability–chemistry achievement relationship through self-beliefs (i.e., SAT-M → SELF-BELIEFS → ACHIEVEMENT in Figure 2D). This pathway is referred to as the indirect effect (44). The unmediated pathway (i.e., SAT-M → ACHIEVEMENT in Figure 2D) is referred to as the direct effect. The total effect of SAT-M on chemistry achievement is the sum of direct and indirect effects. To gather evidence for mediation, we will first compare data-model fit indices for Models B–D. Evidence for mediation would be supported by Model D having the best fit to the data relative to Models B and C. Then, we will estimate the magnitude of the indirect effect in Model D based on a multivariate regression analysis. Further evidence for mediation would be a practically significant indirect effect of SAT-M on chemistry achievement through math self-beliefs. Finally, we will examine the robustness of the parameter estimates for Model D by adjusting for performance-relevant covariates. Computational Details All statistical computations were conducted using R statistical software version 3.5.1 (45). Likert response data were analyzed using the likert package (46). Structural equation modeling was conducted using the lavaan package (47). Given that responses to the ATMI consisted of ratings along a five-point Likert scale, the data is inherently non-normal. To account for this feature, all SEM computations analyzed polychoric correlation matrices using the robust diagonally weighted least squares mean and variance-adjusted (WLSMV) method for obtaining test statistics, parameter estimates, and standard errors (48). In contrast to maximum likelihood-based estimation methods, the WLSMV method makes no assumptions about the distribution of the observed variables and instead assumes a normal distribution of the latent factor, thereby providing more accurate factor loadings, structural coefficients, and standard errors for ordered categorical data (49–51). Fit indices were examined to assess the relative fit of each model to the data. Incremental fit indices, such as the comparative fit index (CFI) and Tucker-Lewis index (TLI), are functions of the chi-square test statistic and degrees of freedom for the hypothesized and null (i.e., no relationships among variables) models. Values close to one indicate that the hypothesized model has a better fit to 91
the data than the null model. Hu and Bentler advised cutoffs near 0.95 for the CFI and TLI based on continuous data and maximum likelihood estimation (52). Schreiber et al. recommended CFI values greater than or equal to 0.95 and TLI values greater than or equal to 0.96 for categorical data using maximum likelihood estimation (42). Komperda et al. recommended CFI and TLI values greater than or equal to 0.95 for categorical data using WLSMV (49). In line with these suggestions, we chose CFI and TLI values greater than or equal to 0.95 as acceptable for the purposes of this study. The root mean square error of approximation (RMSEA) was also used to evaluate data-model fit because it quantifies the discrepancy between the sample covariance matrix and the model-fitted covariance matrix, thereby providing unique information about data-model fit in addition to comparative fit indices. The literature-recommended values of 0.05 and 0.08 for the RMSEA indicate close and reasonable error, respectively, when modeling non-normal ordered categorical data (42, 53). A more stringent cutoff of 0.05 has been suggested by Komperda et al. (49) in the context of test development and evaluation in chemistry education research. However, a 90% confidence interval on the RMSEA that includes 0.05 was chosen as an acceptable cutoff for this study (53, 54). Confirmatory factor analysis of the ATMI subscale demonstrated acceptable data-model fit when constraining the factor loadings, intercepts, and residual variances to be equal across groups (i.e., PrepChem, GC1, and GC2) but letting the means vary (i.e., strict invariance model). See Table A1 in the Appendix for complete invariance testing data. The item-to-total correlations indicated acceptable internal consistency using McDonald’s omega coefficient (ω = 0.92) (43, 55). All SEM computations constrained the models by equating factor loadings, intercepts, and residual variances across groups. For all regression analyses, model parameter estimates reported in the main text are statistically significant at the 0.05 level unless otherwise noted.
Results and Discussion Question 1: What Are Preparatory and General Chemistry Students’ Math Self-Beliefs as Measured by the ATMI? An examination of Figure 1 shows that, in general, the majority of students disagreed or indicated neutrality with the ATMI statements, indicating neutral-positive math self-beliefs. About 75% of GC1 students reported they either disagreed or strongly disagreed with the statement “It makes me nervous to even think about having to do a mathematics problem” (SC7); 67% either disagreed or strongly disagreed with feeling confused in their mathematics classes (SC10); 59% either disagreed or strongly disagreed with feeling strain in mathematics classes (SC5); 58% either disagreed or strongly disagreed with feeling a sense of insecurity when attempting mathematics (SC13); and 44% either disagreed or strongly disagreed that studying mathematics made them feel nervous (SC3). Similar response patterns were observed for students in GC2 and PrepChem. Descriptive statistics for each item are displayed in Table A2 in the Appendix. Looking across courses, PrepChem and GC2 students consistently reported more agreement with the negatively worded ATMI statements than GC1 students, indicating more negative selfbeliefs, on average. The student populations in each course are different in ways that warrant differences in responses. For example, students enrolled in PrepChem if they did not have a high school background in chemistry or if they did not meet a cutoff score on the department’s placement exam. In contrast, students enrolled in GC1 if they passed the department’s placement exam, entered the UW with AP Chemistry scores of 3, 4, or 5, or had previously completed the PrepChem course (8% of the GC1 population in our sample). The responses to the ATMI statements for the PrepChem 92
and GC2 populations were markedly similar. This makes sense in our context because students in GC2 were sampled from sections running in the off-sequence quarters, which disproportionately served students who took PrepChem (22% in our sample) compared with the on-sequence offerings of GC2 (which were not surveyed as part of this study). It is possible that surveying students in the on-sequence GC2 sections could result in a response pattern more similar to that of GC1 because many of those students would be coming directly from on-sequence GC1 sections. Returning to trends in the responses, 30–40% of students reported either that they agreed or strongly agreed that studying mathematics makes them feel nervous (i.e., SC3), depending on the course. Furthermore, 23–33% of students either agreed or strongly agreed with feeling a sense of insecurity when attempting mathematics (i.e., SC13), depending on the course. Lastly, we note that depending on the course, about 16–25% of students either agreed or strongly agreed with feeling strain in mathematics classes (i.e., SC5). Overall, these responses indicated that a considerable proportion of students broadly felt nervous, insecure, or strained in relation to studying or engaging with mathematics. The implication is that more negative self-beliefs for studying mathematics or engaging with mathematics in the context of chemistry may hinder learning and achievement in relation to chemistry content that requires mathematical reasoning. In the next section, we investigate the association between students’ math self-beliefs and their chemistry achievement. Question 2: What Is the Relationship Between Math Self-Beliefs and Chemistry Achievement? To investigate the relationship between math self-beliefs and achievement in chemistry, we first tested the fit of Model A to the data using SEM. In Model A, responses to the ATMI items are predicted by the common construct math self-beliefs, which in turn is directly related to chemistry achievement as measured by exam scores. (Responses to the ATMI were reverse-coded for SEM analyses.) Comparative fit indices and RMSEA values suggest the model had acceptable fit to the data (χ2 = 203.05, df = 107, CFI = 0.996, TLI = 0.997, RMSEA = [0.026, 0.039]). The regression data for Model A is reported in Table 3. The relatively low R2 value on the achievement variable is reasonable given that the model does not take into account measures of students’ prior knowledge or academic attainment. Nonetheless, the math self-beliefs variable was a meaningful predictor of chemistry achievement. In general, students who reported more positively to the ATMI statements received higher exam scores, on average. Specifically, two students in GC1 who differed by a standard unit in math self-beliefs (i.e., disagree versus agree responses, on average) led to a difference of 0.30 standard units in chemistry achievement. That translates to about a 5- to 6-point difference on midterm scores and about a 10- to 11-point difference on final exam scores, assuming normally distributed scores with a standard deviation of 15 and 30 points, respectively. Similar interpretations could be made for students in GC2 and PrepChem. Table 3. Regression Data for Model A Disaggregated by Coursea Course
b
S.E.
β
R2
PrepChem
0.23
0.04
0.25
0.06
GC1
0.29
0.03
0.30
0.09
GC2
0.28
0.04
0.31
0.09
a Unstandardized (b) and standardized (β) regression coefficients are reported together with the standard error
(S.E.) and the proportion of the variance accounted for by the explanatory variables (R2).
93
When controlling for SAT-M (i.e., Model B), math self-beliefs persisted as a statistically significant predictor of chemistry achievement for the GC1 and GC2 populations but not for PrepChem. Comparative fit indices suggest the model had acceptable fit to the data, but the RMSEA indicated poor model fit (χ2 = 1143.59, df = 128, CFI = 0.951, TLI = 0.968, RMSEA = [0.091, 0.102]). The regression data for Model B is reported in Table 4. Math ability (SAT-M) and math selfbeliefs jointly accounted for 34–47% of the variance of chemistry achievement, depending on the course. The regression data reported in Table 4 can be interpreted as follows. For equivalent SAT-M scores, two GC1 students who differ by a standard unit in math self-beliefs led to a difference of 0.10 standard units in chemistry achievement. That translates to approximately 2 points per midterm and about 3.5 points on a final exam, assuming normally distributed scores with a standard deviation of 15 and 30 points, respectively.
Table 4. Regression Coefficients for Model B Disaggregated by Coursea Course PrepChem GC1 GC2
Variable
b
S.E.
β
SELF-BELIEFS
0.02b
0.03
0.02
SAT-M
0.006
0.0003
0.66
SELF-BELIEFS
0.10
0.02
0.10
SAT-M
0.007
0.0003
0.67
SELF-BELIEFS
0.14
0.03
0.15
SAT-M
0.005
0.0004
0.56
R2 0.43 0.47 0.34
(b) and standardized (β) regression coefficients are reported together with the estimated standard error (S.E.) and the proportion of the variance accounted for by the explanatory variables (R2). b p = 0.40. a Unstandardized
The effect of math self-beliefs on chemistry achievement predicted by Model B is smaller than the effect predicted by Model A. However, comparing the standardized regression coefficients for Model B suggests that math self-beliefs have about one-seventh the predictive power of SAT-M in GC1. For the PrepChem population, the effect of math self-beliefs on chemistry achievement was negligible and statistically nonsignificant. In other words, math self-beliefs did not account for a substantial proportion of the variation in chemistry achievement for students in PrepChem when accounting for individual differences in SAT-M. Controlling for SAT-V and HSGPA covariates, the effect of math self-beliefs on chemistry achievement did not diminish and the overall variance in chemistry achievement explained by Model B′ increased relative to Model B (38–51%, depending on the course). The regression data for Model B′ is reported in Table 5. Comparative fit indices suggest that Model B′ had acceptable fit to the data, but the RMSEA indicated poor model fit (χ2 = 871.35, df = 170, CFI = 0.968, TLI = 0.984, RMSEA = [0.065, 0.074]). 94
Table 5. Regression Data for Model B′ Disaggregated by Coursea Course
PrepChem
GC1
GC2
Variable
b
S.E.
β
SELF-BELIEFS
0.02b
0.03
0.03
SAT-M
0.004
0.0004
0.43
SAT-V
0.002
0.0004
0.21
HSGPA
0.75
0.11
0.24
SELF-BELIEFS
0.09
0.02
0.10
SAT-M
0.006
0.0003
0.60
SAT-V
0.001c
0.0003
0.05
HSGPA
0.60
0.09
0.15
SELF-BELIEFS
0.13
0.03
0.14
SAT-M
0.004
0.0004
0.47
SAT-V
0.001
0.0004
0.09
HSGPA
0.79
0.14
0.20
R2
0.51
0.49
0.38
(b) and standardized (β) regression coefficients are reported together with the estimated standard error (S.E.) and the proportion of the variance accounted for by the explanatory variables (R2). b p = 0.40. c p = 0.06. a Unstandardized
The absence of a statistically significant relationship between math self-beliefs and achievement was not the only notable difference between PrepChem and the GC1 and GC2 populations. SATV was a more powerful predictor of chemistry achievement in the PrepChem population than in the GC1 and GC2 settings based on a comparison of standardized regression coefficients. In fact, of the four explanatory variables analyzed, SAT-M, SAT-V, and HSGPA were the most powerful predictors of chemistry achievement in the PrepChem setting and the math self-beliefs variable was statistically nonsignificant. This suggests that the factors affecting student achievement are different across the courses sampled in this study, which could be due to course characteristics such as content and assessment types, not just student characteristics. One limitation to the analysis of Model B is the absence of students’ prior chemistry content knowledge and affect because these factors are known to correlate with chemistry achievement (6). Including prior chemistry content knowledge and affect in the structural models would help us understand the relative importance of SAT-M and math self-beliefs while accounting for individual differences in prior knowledge of chemistry and affective characteristics. We further discuss this aspect of our analysis in the Limitations section. Question 3: To What Extent Do Math Self-Beliefs Explain the Relationship Between SAT-M and Chemistry Achievement? Through the lens of self-beliefs, two students with the same math ability who differ in their math self-beliefs would be expected to perform differently on chemistry content exams that require mathematical reasoning. In the previous section, we found math self-beliefs to be a significant predictor of chemistry achievement in the GC1 and GC2 settings. Although the effect was small after accounting for individual differences in SAT-M and other performance-related covariates, the analysis provided evidence for a nonzero relationship between math self-beliefs and chemistry 95
achievement. But we are not only interested in identifying relationships between variables. We also seek to understand the role of math self-beliefs, if anything, in facilitating students’ ability to apply their math knowledge and skills to the study of chemistry. To test the presence or absence of such an effect, we first contrasted the fit of Models B–D to the data. Table 6 displays the data-model fit for Models B–D. The chi-square test statistic, comparative fit indices, and RMSEA values suggest Model D had the best fit to the data. This means that if we simulated the dataset based on each model, the error associated with Model D would be smallest relative to the data on hand. Thus, the comparatively better fit of Model D to the data is evidence for the presence of a mediating effect. Likelihood ratio tests comparing Models D and B (Δχ2 = 103.41, Δdf = 3, p < 0.0001) and Models D and C (Δχ2 = 9.37, Δdf = 3, p < 0.025) provided further evidence for the simple mediation model having the best fit to the data.
Table 6. Data–Model Fit Indices for Models B–D Estimated Using the WLSMV Method χ2 (df)
CFI
TLI
RMSEA
B
1143.59 (128)
0.951
0.968
[0.091, 0.102]
C
366.46 (128)
0.989
0.993
[0.041, 0.052]
D
257.48 (125)
0.994
0.996
[0.029, 0.041]
Model
To what extent do math self-beliefs, as measured by the ATMI subscale, explain the relationship between SAT-M and chemistry achievement? Turning to the regression data for Model D in Table 7, the direct effect of SAT-M on chemistry achievement was estimated at 0.642 standard units, whereas the total effect was estimated at 0.674 standard units for the GC1 population. The effect of SAT-M on chemistry achievement that is mediated by math self-beliefs was estimated at 0.032 standard units. If we reason about the size of the mediation effect as a ratio of indirect effect to total effect, then math self-beliefs accounted for 4.8% (95% confidence interval: [4.3, 5.3]) of the total effect in GC1 (56). While this is a small effect, the estimate is for the proportion of the math ability-chemistry achievement relationship that is mediated by math self-beliefs. The self-beliefs variable also has a direct effect on chemistry achievement of .10 standard units. In other words, math self-beliefs have a unique effect on chemistry achievement while also mediating the math ability-chemistry achievement relationship. A similar interpretation could be made for the GC2 population. The relations between variables did not change substantially when controlling for SATV and HSGPA (i.e., Model D′). Table A3 displays data-model fit indices for the nested models B′–D′ and Table A4 reports the regression data for Model D′.
96
Table 7. Regression Data for the Simple Mediation Model (i.e., Model D)a Path b SAT-M → SELF-BELIEFS SELF-BELIEFS → ACHIEVEMENT SAT-M → ACHIEVEMENT (direct effect) SAT-M → SELF-BELIEFS →ACHIEVEMENT (indirect effect) SAT-M → ACHIEVEMENT (total effect)
GC1
GC2
R2 = 0.47)
R2 = 0.34)
S.E.
β
b
S.E.
PrepChem (R2 = 0.43) β
b
0.00332 0.0979 0.00655
0.00032 0.0241 0.00027
0.301 0.106 0.642
0.00307 0.140 0.00487
0.00042 0.034 0.00036
0.292 0.155 0.514
0.00346
0.00033
0.00008
0.032
0.00043
0.00012
0.00687
0.00026
0.674
0.00530
0.00036
S.E.
β
0.00553
0.00041 0.0318 0.00032
0.353 0.022 0.651
0.045
0.00007c
0.00011
0.008
0.559
0.00560
0.00030
0.659
0.0192b
a Unstandardized (b) and standardized (β) path coefficients together with the estimated standard error (S.E.) and the relative variance of the chemistry achievement dependent
variable accounted for by the explanatory variable (R2).
b p = 0.55.
c p = 0.55.
97
The presence of a mediation effect in the PrepChem setting was inconclusive. The lack of evidence for direct and mediating effects of math self-beliefs combined with the evidence that SATV scores are a stronger predictor of achievement in PrepChem relative to GC1 and GC2 suggests that the knowledge and skills needed to succeed in preparatory chemistry are qualitatively distinct from the general chemistry series in this setting. The regression analyses of Models B–D and B′ suggest proficiency in mathematics, reading, and high school academic ability were more strongly correlated with student performance on chemistry content exams than were their responses to the ATMI subscale. Absent from our mediation analysis were measures of students’ prior chemistry content knowledge and affect related to learning chemistry. This was a limitation to our analysis of Model D just as it was for Model B, and we further discuss this aspect of the analysis in the Limitations section.
Summary We surveyed students in preparatory and general chemistry courses about their math self-beliefs using the short version of the ATMI. While most students reported neutral-positive self-beliefs, 16–40% of students reported feeling nervous, insecure, or strained in relation to studying or performing mathematical tasks, depending on the course. A multivariate regression analysis using SEM demonstrated that math self-beliefs were related to achievement on chemistry content exams in GC1 and GC2 course settings. In other words, students who reported agreement to the negatively worded ATMI subscale items received fewer exam points than students who reported disagreement, on average. Although the effect was small while controlling for SAT-M scores, the regression analysis provided evidence that math self-beliefs were directly related to students’ chemistry achievement. The effect persisted when accounting for SAT-V and HSGPA data in the model. In fact, math selfbeliefs were stronger predictors of achievement than SAT-V scores based on standardized regression coefficients. However, the relations among variables were not the same for the PrepChem population. In PrepChem, SAT-V was a stronger predictor of chemistry achievement than math selfbeliefs, and there was a null effect for math self-beliefs on achievement in Models B and D. Based on self-belief theories, we reasoned that two students with the same math ability but who differed in math self-beliefs would perform differently on chemistry content exams that required mathematics because self-beliefs play a determining role in how one performs in evaluative situations (10–12). We gathered preliminary evidence for the mediating effect of math self-beliefs on the math ability–chemistry achievement relationship in the GC1 and GC2 settings. Although the effect was relatively small, the analysis presented in this chapter demonstrated evidence of the importance of affect for modeling student achievement in undergraduate chemistry courses.
Limitations There are three limitations to the results reported in this chapter that we wish to discuss. First, the presence of a mediating effect should be cautiously interpreted given the limitations of using a single metric of math self-beliefs. The operationalization of math self-beliefs in this study was based on a five-item subscale of the short ATMI and is likely to underrepresent the construct of interest. Furthermore, the measure of math self-beliefs chosen for this study is arguably confounded by multiple related constructs, including math self-confidence, math self-concept, and math anxiety. Thus, attributing math self-beliefs as the source of the mediating effect was a conservative choice in the analytical process.
98
Second, absent from the analyses of Models B and D were measures of students’ prior chemistry content knowledge and affect. This threatens the validity of the observed relationship between math self-beliefs and chemistry achievement and mediating processes because studies have shown prior chemistry content knowledge and affect to be significant predictors of chemistry achievement in addition to SAT-M scores (6). Thus, until more complex models can be developed and tested, this study provides preliminary evidence for the mediating effect of math self-beliefs, as measured by the ATMI, on the math ability–chemistry achievement relationship in general chemistry courses. Lastly, the unbalanced sampling strategy in GC2 threatens the generalizability of our results to the on-sequence GC2 population because we sampled only from off-sequence section offerings. Therefore, the size of the mediating effect for the GC2 population may be biased in that regard. We also note that data was collected from a single institution, which limits the generalizability of the results across settings. Variation in people, together with variation in curricula, assessment types, class size, and other departmental- and institution-level factors, threatens the generalizability of the results reported in this chapter across schools.
Implications The results reported in this chapter have implications for future investigations of student learning and achievement at the intersection of mathematics and chemistry education. First and foremost, further studies interrogating the nature of students’ math self-beliefs in relation to learning chemistry are warranted. As indicated in the Limitations section, our study used a single operationalization of math self-beliefs, which underrepresented the construct of more immediate interest to the chemistry education community, which is students’ self-beliefs for applying mathematical knowledge and skills in the context of chemistry. Furthermore, we argued that the measure of math self-beliefs adopted for this study was confounded by several self-belief constructs. Thus, we recommend that future studies continue to explore relations between math self-beliefs (e.g., confidence, self-concept, and selfefficacy) and the math ability–chemistry achievement relationship. Furthermore, we recommend researchers investigate the context-dependent nature of math self-beliefs to better understand the nuances of using mathematics to reason about chemical phenomena. Another limitation to this study was the absence of data on students’ prior chemistry content knowledge and affect. Thus, we recommend future studies develop and test more complex models that incorporate both chemistry- and math-specific cognitive and psychological variables. For instance, Vincent-Ruz et al. found evidence for the mediating effect of chemistry self-concept beliefs on the math ability–chemistry achievement relationship, but the mediating role of math self-concept beliefs was inconclusive (8). Accumulating evidence for the relations between variables of interest using diverse instruments across diverse settings will support the chemistry education community in developing models of how students reason about chemical phenomena using mathematics (57). For practitioners, the evidence presented in this chapter demonstrated preliminary evidence of the role of math self-beliefs in determining student achievement during evaluative situations. Although the effects were small, the evidence suggests that the cultivation of positive math self-beliefs can promote better exam outcomes in undergraduate chemistry courses. Therefore, instructional strategies aimed at promoting student reasoning about chemical phenomena using mathematics would be enhanced by simultaneously cultivating positive self-beliefs with respect to mathematical reasoning in chemistry.
99
Appendix: Tables Table A1. ATMI measurement invariance across groups (i.e., PrepChem, GC1, and GC2). Confirmatory factor analyses were estimated using the WLSMV method. The configural model has the same factor structure across groups, but no constraints are placed on the fit. In contrast, the weak-invariance model constraints the factor loadings to be equal across groups. The strong-invariance model constraints the factor loadings and intercepts to be equal across groups. The strict-invariance model constraints factor loadings, intercepts, and residual variances to be equal across groups. The equal means model constraints the mean variances to be equal across groups in addition to the strict invariance constraints. Model
χ2 (df)
CFI
TLI
RMSEA
Configural
150.35 (15)
0.995
0.991
[0.083, 0.111]
Weak invariance
134.83 (23)
0.996
0.995
[0.060, 0.083]
Strong Invariance
140.19 (51)
0.997
0.998
[0.034, 0.051]
Strict Invariance
153.08 (61)
0.997
0.998
[0.032, 0.047]
Equal means
441.12 (63)
0.987
0.994
[0.072, 0.086]
Table A2. Item Statistics for the ATMI Subscale Disaggregated by Course Items
PrepChem Mean
SD
GC1
Skew Kurtosis
Mean
SD
Skew
GC2 Kurtosis
Mean
SD
Skew
Kurtosis
SC3
3.12
1.08
-0.03 -0.83
2.85
1.08 0.14
-0.88
3.11
1.05 -0.06 -0.85
SC5
2.69
1.06
0.39
-0.57
2.48
0.97 0.52
-0.17
2.77
1.01 0.29
-0.57
SC7
2.4
1.07
0.66
-0.24
2.13
0.94 0.81
0.33
2.46
1.03 0.58
-0.22
SC10 2.48
0.93
0.65
0.28
2.32
0.88 0.68
0.56
2.55
0.95 0.58
0.04
SC13 2.71
1.15
0.33
-0.9
2.54
1.09 0.41
-0.72
2.88
1.1
-0.96
0.1
Table A3. Data–Model Fit Indices for Models B′–D′ Estimated Using the WLSMV Method χ2 (df)
CFI
TLI
RMSEA
B′
871.35 (170)
0.968
0.994
[0.065, 0.074]
C′
472.83 (170)
0.993
0.993
[0.041, 0.051]
D′
367.44 (167)
0.991
0.995
[0.032, 0.043]
Model
100
Table A4. Regression Data for the Simple Mediation Model Adjusted for SAT-V and HSGPA (i.e., Model D′)a Pathway SAT-M → SELF-BELIEFS SELF-BELIEFS → ACHIEVEMENT SAT-M → ACHIEVEMENT (direct effect) SAT-V → ACHIEVEMENT HSGPA → ACHIEVEMENT SAT-M → SELF-BELIEFS →ACHIEVEMENT (indirect effect) SAT-M → ACHIEVEMENT (total effect)
GC1
GC2
(R2 = 0.48)
(R2 = 0.40)
PrepChem (R2 = 0.51)
b
S.E.
β
b
S.E.
β
b
S.E.
β
0.00363 0.0870 0.00537
0.00039 0.0219 0.00032 0.00029 0.085 0.00008 0.00032
0.326 0.101 0.558 0.053 0.152 0.033 0.591
0.00284 0.141 0.00430 0.00105 0.865 0.00040 0.00470
0.00054 0.034 0.00046 0.00046 0.160 0.00012 0.00046
0.265 0.153 0.436 0.097 0.203 0.040 0.476
0.00402
0.00052 0.0291 0.00040 0.00039 0.105 0.00012 0.00038
0.415 0.029 0.416 0.208 0.242 0.012 0.428
0.00054b 0.580 0.00032 0.00568
0.0242c 0.00337 0.00180 0.740 0.00010d 0.00347
a Unstandardized (b) and standardized (β) path coefficients together with the estimated standard error (S.E.) and the relative variance of the chemistry achievement dependent
101
variable accounted for by the explanatory variable (R2).
b p = 0.06.
c p = 0.40.
d p = 0.40.
References 1. 2. 3. 4. 5. 6.
7. 8. 9. 10. 11. 12. 13.
14. 15. 16. 17. 18. 19. 20.
Pickering, M. Helping the High Risk Freshman Chemist. J. Chem. Educ. 1975, 52, 512–514. Andrews, M. H.; Andrews, L. First-Year Chemistry Grades and SAT Math Scores. J. Chem. Educ. 1979, 56, 231–232. Spencer, H. E. Mathematical SAT Test Scores and College Chemistry Grades. J. Chem. Educ. 1996, 73, 1150–1153. Ozsogomonyan, A.; Loftus, D. Predictors of General Chemistry Grades. J. Chem. Educ. 1979, 56, 173–175. Lewis, S. E.; Lewis, J. E. Predicting At-Risk Students in General Chemistry: Comparing Formal Thought to a General Achievement Measure. Chem. Educ. Res. Pract. 2007, 8, 32–51. Xu, X.; Villafane, S. M.; Lewis, J. E. College Students’ Attitudes Toward Chemistry, Conceptual Knowledge and Achievement: Structural Equation Model Analysis. Chem. Educ. Res. Pract. 2013, 14, 188–200. Ralph, V. R.; Lewis, S. E. Chemistry Topics Posing Incommensurate Difficulty to Students with Low Math Aptitude Scores. Chem. Educ. Res. Pract. 2018, 19, 867–884. Vincent-Ruz, P.; Binning, K.; Schunn, C. D.; Grabowski, J. The Effect of Math SAT on Women’s Chemistry Competency Beliefs. Chem. Educ. Res. Pract. 2018, 19, 342–351. National Research Council. A Framework for K-12 Science Education: Practices, Crosscutting Concepts, and Core Ideas; The National Academies: Washington, DC, 2012. Bandura, A. Social Foundations of Thought and Action: A Social Cognitive Theory; Prentice-Hall, Inc.: Englewood Cliffs, NJ, 1986. Bong, M.; Skaalvik, E. M. Academic Self-Concept and Self-Efficacy: How Different Are They Really? Educ. Psychol. Rev. 2003, 15, 1–40. Shavelson, R. J.; Hubner, J. J.; Stanton, G. C. Self-Concept: Validation of Construct Interpretations. Rev. Educ. Res. 1976, 46, 407–441. Fennema, E.; Sherman, J. A. Mathematics Attitudes Scales: Instruments Designed to Measure Attitudes Toward the Learning of Mathematics by Females and Males. J. Res. Math. Educ. 1976, 7, 324–326. Tapia, M. The Attitudes Toward Mathematics Instrument. Presented at the Annual Meeting of the Mid-South Educational Research Association, Tuscaloosa, AL, Nov. 6, 1996. Maddux, J. E. Self-Efficacy Theory: An Introduction. In Self-Efficacy, Adaptation, and Adjustment; Maddux, J. E., Ed.; Springer: Boston, MA, 1995; pp 3–33. Examinations Institute. First Term General Chemistry (Special Examination); The American Chemical Society Examinations Institute: Clemson, SC, 1997. Examinations Institute. First Term General Chemistry Special Examination (Form 1997). The American Chemical Society Examinations Institute, Clemson University: Clemson, SC, 1997. Examinations Institute. First Term General Chemistry Exam. The American Chemical Society Examinations Institute: Milwaukee, WI, 2015. Russell, A. A. A Rationally Designed General Chemistry Diagnostic Test. J. Chem. Educ. 1994, 71, 314–317. Leopold, D. G.; Edgar, B. Degree of Mathematics Fluency and Success in Second-Semester Introductory Chemistry. J. Chem. Educ. 2008, 85, 724–731. 102
21. Goolsby, C. B.; Dwinell, P. L.; Higbee, J. L. Factors Affecting Mathematics Achievement in High Risk College Students. Res. Teach. Dev. Educ. 1988, 4, 18–27. 22. Randhawa, B. S.; Beamer, J. E.; Lundberg, I. Role of Mathematics Self-Efficacy in the Structural Model of Mathematics Achievement. J. Educ. Psychol. 1993, 85, 41–48. 23. Pajares, F.; Miller, M. D. Mathematics Self-Efficacy and Mathematics Performances: The Need for Specificity of Assessment. J. Couns. Psychol. 1995, 42, 190–198. 24. Quinn, R. Students’ Confidence in the Ability to Transfer Basic Math Skills in Introductory Physics and Chemistry Courses at a Community College. Unpublished Ph.D. Dissertation. The University of Southern Mississippi, Hattiesburg, MS, 2013. 25. Chan, J. Y. K.; Bauer, C. F. Identifying At-Risk Students in General Chemistry via Cluster Analysis of Affective Characteristics. J. Chem. Educ. 2014, 91, 1417–1425. 26. Pintrich, P. R.; De Groot, E. V. Motivational and Self-Regulated Learning Components of Classroom Academic Performance. J. Educ. Psychol. 1990, 82, 33–40. 27. Bauer, C. F. Beyond “Student Attitudes”: Chemistry Self-Concept Inventory for Assessment of the Affective Component of Student Learning. J. Chem. Educ. 2005, 82, 1864–1870. 28. Bauer, C. F. Attitude Toward Chemistry: A Semantic Differential Instrument for Assessing Curriculum Impacts. J. Chem. Educ. 2008, 85, 1440–1445. 29. Xu, X.; Lewis, J. E. Refinement of a Chemistry Attitude Measure for College Students. J. Chem. Educ. 2011, 88, 561–568. 30. College Board. Test Specifications for the Redesigned SAT; New York, NY, 2015. 31. Beard, J.; Marini, J. Validity of the SAT for Predicting First-Year Grades: 2013 SAT Validity Sample; Statistical Report 2013-3; College Board: New York, NY, 2013. 32. Mattern, K. D.; Patterson, B. F. Synthesis of Recent SAT Validity Findings: Trend Data over Time and Cohorts. Research in Review 2014-1; College Board: New York, NY, 2014. 33. College Board. Concordance Tables; College Board: New York, NY, 2016. 34. College Board. ACT and SAT Concordance Tables; Research Notes, 40; College Board: New York, NY, 2009. 35. Lim, S. Y.; Chapman, E. Development of a Short Form of the Attitudes Toward Mathematics Inventory. Educ. Stud. Math. 2013, 82, 145–164. 36. Tapia, M.; Marsh, G. Confirmatory Factor Analysis of the Attitudes Toward Mathematics Inventory; Chattanooga, TN, 2002. 37. Tapia, M.; Marsh, G. An Instrument to Measure Mathematics Attitudes. Acad. Exch. Q. 2004, 8, 1–7. 38. Heiberger, R. M.; Robbins, N. B. Design of Diverging Stacked Bar Charts for Likert Scales and Other Applications. J. Stat. Softw. 2014, 57, 1–32. 39. Komperda, R. Likert-Type Survey Data Analysis with R and RStudio. In Chemical Education Research (CADACER): Advances and Avenues; ACS Symposium Series; American Chemical Society: Washington, DC, 2017; Vol. 1260, pp 91–116. 40. Kline, R. B. Principles and Practice of Structural Equation Modeling; Guilford: New York, NY, 2015. 41. Ullman, J. B.; Bentler, P. M. Structural Equation Modeling. In Handbook of Psychology, 2nd ed.; Weiner, I. B., Ed.; John Wiley & Sons: Hoboken, NJ, 2012.
103
42. Schreiber, J. B.; Nora, A.; Stage, F. K.; Barlow, E. A.; King, J. Reporting Structural Equation Modeling and Confirmatory Factor Analysis Results: A Review. J. Educ. Res. 2006, 99, 323–338. 43. Komperda, R.; Pentecost, T. C.; Barbera, J. Moving beyond Alpha: A Primer on Alternative Sources of Single-Administration Reliability Evidence for Quantitative Chemistry Education Research. J. Chem. Educ. 2018, 95, 1477–1491. 44. Hayes, A. F. Beyond Baron and Kenny: Statistical Mediation Analysis in the New Millennium. Commun. Monogr. 2012, 76, 408–420. 45. R Core Team. R: A Language and Environment for Statistical Computing; R Foundation for Statistical Computing: Vienna, Austria, 2017. 46. Bryer, J.; Speerschneider, K. Likert, software for analyzing Likert response items with an emphasis on visualizations, Version 1.3.5; 2016. https://github.com/jbryer/likert (accessed Dec. 19, 2018). 47. Rosseel, Y. Lavaan: An R Package for Structural Equation Modeling. J. Stat. Softw. 2012, 48, 1–36. 48. Holgado−Tello, F. P.; Chacón–Moscoso, S.; Barbero–García, I.; Vila–Abad, E. Polychoric versus Pearson Correlations in Exploratory and Confirmatory Factor Analysis of Ordinal Variables. Qual. Quant. 2010, 44, 153–166. 49. Komperda, R.; Hosbein, K. N.; Barbera, J. Evaluation of the Influence of Wording Changes and Course Type on Motivation Instrument Functioning in Chemistry. Chem. Educ. Res. Pract. 2018, 19, 184–198. 50. Li, C.-H. Confirmatory Factor Analysis with Ordinal Data: Comparing Robust Maximum Likelihood and Diagonally Weighted Least Squares. Behav. Res. Methods 2016, 48, 936–949. 51. Li, C.-H. The Performance of MLR, USLMV, and WLSMV Estimation in Structural Regression Models with Ordinal Variables. Unpublished Ph.D. Dissertation, Michigan State University, Lansing, MI, 2014. 52. Hu, L.; Bentler, P. M. Cutoff Criteria for Fit Indexes in Covariance Structure Analysis: Conventional Criteria versus New Alternatives. Struct. Equ. Model. Multidiscip. J. 1999, 6, 1–55. 53. Hutchinson, S. R.; Olmos, A. Behavior of Descriptive Fit Indexes in Confirmatory Factor Analysis Using Ordered Categorical Data. Struct. Equ. Model. Multidiscip. J. 1998, 5, 344–364. 54. Browne, M. W.; Cudeck, R. Alternative Ways of Assessing Model Fit. Sage Focus Educ. 1993, 154, 136. 55. McDonald, R. P. Test Theory: A Unified Treatment; Lawrence Erlbaum Associates Publishers: Mahwah, NJ, 1999. 56. Preacher, K. J.; Kelley, K. Effect Size Measures for Mediation Models: Quantitative Strategies for Communicating Indirect Effects. Psychol. Methods 2011, 16, 93–115. 57. Arjoon, J. A.; Xu, X.; Lewis, J. E. Understanding the State of the Art for Measurement in Chemistry Education Research: Examining the Psychometric Evidence. J. Chem. Educ. 2013, 90, 536–545.
104
