Peer-Led Team Learning in General Chemistry I: Interactions with

Oct 9, 2018 - Robust evidence shows that Peer-Led Team Learning (PLTL) improves the academic success of first-year college students in introductory ...
0 downloads 0 Views 1MB Size
Download PDF
Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX

pubs.acs.org/jchemeduc

Peer-Led Team Learning in General Chemistry I: Interactions with Identity, Academic Preparation, and a Course-Based Intervention Regina F. Frey,*,†,‡ Angela Fink,† Michael J. Cahill,† Mark A. McDaniel,†,§ and Erin D. Solomon† †

The Center for Integrative Research on Cognition, Learning, and Education (CIRCLE), ‡Department of Chemistry, and Department of Psychological and Brain Sciences, Washington University in St. Louis, St. Louis, Missouri 63130, United States

§

J. Chem. Educ. Downloaded from pubs.acs.org by UNIV OF CAMBRIDGE on 10/09/18. For personal use only.

S Supporting Information *

ABSTRACT: Robust evidence shows that Peer-Led Team Learning (PLTL) improves the academic success of first-year college students in introductory Science, Technology, Engineering, and Mathematics (STEM) courses. Less clear is the extent to which this positive PLTL effect varies across different subgroups of the population. The current study aims to deepen our understanding of the overall PLTL effect by extensively evaluating an optional PLTL program associated with General Chemistry I at a private, research university. Using five years of exam data, this study disaggregates the PLTL effect by demographics (sex and race), academic preparation (math skills, chemistry content knowledge, and experience with collegepreparatory coursework), and participation in another curricular innovation (a growth-mindset intervention). Results revealed that the positive effect of PLTL was comparable across demographic identity groups. Thus, the PLTL program benefitted all participants but did not reduce the pre-existing performance disparity between underrepresented minority and white students (no sex-based disparity was observed). In terms of academic preparation, the PLTL effect interacted with students’ level of experience with college-preparatory coursework but not with their math or chemistry knowledge. This pattern suggests that PLTL may help develop students’ self-management or reasoning skills, without necessarily narrowing knowledge gaps. Finally, PLTL interacted with participation in a growth-mindset intervention: the difference between PLTL participants and nonparticipants was smaller among students who received the mindset intervention. Implications for chemical education researchers and practitioners are discussed, with an eye toward fostering equity in introductory STEM courses. KEYWORDS: First-Year Undergraduate/General, Chemical Education Research, Collaborative/Cooperative Learning, Constructivism FEATURE: Chemical Education Research

■

INTRODUCTION Peer-led Team Learning (PLTL) is one of several activelearning pedagogies to transform undergraduate Science, Technology, Engineering, and Mathematics (STEM) education in the twenty-first century. Along with Problem-Based Learning (PBL) and Process-Oriented Guided Inquiry Learning (POGIL), PLTL emerged from the theory of social constructivism,1 which argues that learners must rigorously cultivate their own knowledge through collaboration with others.2 Under this framework, PLTL engages about 6−8 students in 90−120 min of cooperative problem solving each week, with facilitation from a trained peer leader who previously succeeded in the associated course.3−6 A recent meta-analysis demonstrated that active-learning strategies generally produce better STEM performance and retention than traditional lectures.7 Focusing specifically on PLTL, a recent review found that STEM courses accompanied by PLTL elicit greater student success than courses without them, at least when the PLTL implementation effectively fosters collaborative group-work.6 Thus, robust evidence supports the claim that PLTL enhances achievement in undergraduate STEM education (but see Chan and Bauer8), validating its widespread adoption across the disciplines, including chemistry. This study delves deeper into the PLTL benefit, examining whether PLTL differentially impacts subgroups enrolled in © XXXX American Chemical Society and Division of Chemical Education, Inc.

general chemistry. Previous research in chemistry and other disciplines has addressed this issue with mixed results. Several studies showing overall performance benefits of PLTL also reported especially large improvements among groups underrepresented in STEM, including females9−12 and underrepresented minorities9,10,13−15 (URMs; includes students who identify as African American or Black, Hispanic or Latinx, American Indian or Alaska Native, or Native Hawaiian or Pacific Islander16). Further, researchers have argued that PLTL especially benefits academically underprepared or “at-risk” students17,18 (see also Hall et al.19). By contrast, several other studies investigating variation in the PLTL effect found no significant interaction with gender or sex8,14,20−22or with race or ethnicity.8,20,21 One study evaluating a related methodology, Peer-Guided Learning Inquiry (PGLI), also found no differential effects for more versus less prepared chemistry students.23 Finally, some investigations did not examine the interaction of PLTL with student characteristics (e.g., due to limited power), leaving it an open question whether PLTL offered larger benefits to certain groups.24,25 Given these mixed findings, the present study contributes to the literature by assessing how one PLTL program’s Received: May 18, 2018 Revised: September 26, 2018

A

DOI: 10.1021/acs.jchemed.8b00375 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

In light of these considerations, the current study assesses the effectiveness and equity of a PLTL program in General Chemistry I. This research provides an extensive program evaluation to inform curriculum development, and it may shed light on the educational contexts where PLTL can serve as a tool for increasing equity in undergraduate STEM education. The following questions guided this investigation: 1. How do sex, race, and PLTL participation affect firstyear students’ General Chemistry I performance, after accounting for variation in academic preparation? 2. Does PLTL offer greater benefits to groups historically underrepresented in STEM (i.e., females and minorities)? 3. Does PLTL offer greater benefits to students with weaker academic preparation, as indexed by ACT Math scores, Advanced Placement (AP) exam scores, and an in-house content-knowledge assessment denoted as the Online Diagnostic (OD)? 4. How does PLTL interact with another course-based strategy for supporting underrepresented groups (specifically, a growth-mindset intervention)?

effectiveness varies according to demographics and other variables. Although the study’s results may have limited generalizability (in terms of several features described next), its consideration of many potential moderators of the PLTL effect may deepen the field’s understanding of who benefits from PLTL and how it relates to other aspects of introductory STEM courses. Specifically, this study investigates the PLTL program that accompanies General Chemistry I at a private research university. Further, enrollment in the PLTL program is optional, but following the standard PLTL criteria, attendance and participation become mandatory once a student has enrolled (note that under the original PLTL model, enrollment is nonoptional;4 see Methods for discussion of this issue). Combining five years of data, we explore how sex, race, and three different measures of academic preparation potentially interact with PLTL participation to influence students’ course performance. In addition, we use a subset of the data to examine the interaction of PLTL participation with another curricular innovation known as a growth-mindset intervention. Growth-mindset interventions represent one example of the social-psychological interventions that have gained influence in education research and practice as a way to improve student outcomes.26 Growth-mindset interventions emerged from research showing that students’ mindsets about intelligence affect their responses to academic challenges: students who believe intelligence is flexible and can be grown tend to respond more productively to setbacks than students with a fixed view of intelligence.27−29 Mindset interventions therefore strive to induce growth mindsets by having students read and reflect on an article that describes how challenging and persistent practice with a subject can create new connections in the brain and increase their abilities in that domain. Mindset interventions have been shown to improve the achievement and well-being of students transitioning into college,30−32 but little work has explored how they might interact with other course components. Thus, this study may enhance the field’s understanding of how social-psychological interventions fit in with other parts of the curriculum such as PLTL.

■

METHODS

Course Context

General Chemistry I is the first part of a two-course sequence. It enrolls approximately 700−800 students each fall, who register for one of three lecture sections. Weekly, students attend three 1-hour lectures, a mandatory recitation meeting (60−90 min, depending on the format), and a 3-hour lab session, which requires enrollment in a separate laboratory course. Although the lecture sections have different instructors, they are otherwise treated as one course. Students from each section receive the same problem sets, quizzes, and exams; they are intermingled in recitation and lab; and their work is combined during grading. Besides the required components of General Chemistry I, students can utilize several supplementary learning opportunities. For example, instructors offer 1.5− 2 hour help sessions every weekday, and students can attend sessions with any instructor. A Transition Program is offered to students identified as academically at-risk, according to several diagnostic measures.18 This Program provides POGIL-style extended-length recitations, mandatory PLTL, and participation in smaller peer-mentored study groups. Students not in the Transition Program are also invited to register for the PLTL program, but for them it is optional to join.

■

RESEARCH QUESTIONS As others have argued,23,33 disaggregating the PLTL effect is an important step in evaluating whether PLTL fosters equity in STEM. The observation that a PLTL program is generally effective, improving the performance of “the average student,” does not guarantee that it is equitable.23,33 The positive overall effect could mask variation across groups, with a subset of students showing a null or even negative effect. To our knowledge, such patterns have not been demonstrated in the chemical-education literature; PLTL participation typically offers at least some advantage to all participants9−15,17,18,20−22 (but see Chan and Bauer8). However, uniformly positive effects across identity groups and preparation levels still do not ensure equity, particularly in a course with pre-existing achievement gaps. In that case, if all PLTL participants benefit equally, then achievement gaps will persist, despite across-theboard improvements in students’ performance. To achieve true “equity of outputs”, PLTL must level the playing field of student success, so that a cross-section of the top performers in class (or any achievement level) mirrors a cross-section of the student population.6,23,33 In other words, for PLTL to achieve equity in the strictest sense, it must eliminate or at least reduce achievement gaps in the course.

Peer-Led Team Learning Program

The basic structure of the PLTL program is reviewed here. Additional information about the characteristics of peer leaders, the format of peer leader training, and student perceptions of PLTL can be found elsewhere in this Journal.20 A summary of the PLTL implementation and training is also provided in the Supporting Information, along with more detail on the relationship between the PLTL problems and the General Chemistry I lecture course. According to its original conceptualization, PLTL is “neither remedial nor optional”4 (p. 5); instead, it is a well-integrated, required course component for all enrolled students. Yet in recent years, practical constraints (e.g., limited capacity, student workloads) have led to variations in PLTL implementation, including optional enrollment of students into the program8,12,15,20 (see Wilson and Varma-Nelson6 for B

DOI: 10.1021/acs.jchemed.8b00375 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 1. Comparison of Characteristics of the Student Sample from General Chemistry I in Fall 2012−2016a Sample

Growth-Mindset Participants

Demographic Variables

No PLTL

PLTL

No PLTL

PLTL

Sample size, N ACT Mathb (SE) ODc (SE) AP proportiond (SE) Students who are URM,e % Students who are female, %

407 32.91 (0.12) 48.72 (1.12) 0.40 (0.02) 21.4 44.0

847 32.74 (0.08) 48.65 (0.74) 0.41 (0.01) 22.8 59.4f

90 32.87 (0.22) 48.78 (2.43) 0.41 (0.03) 21.1 46.7

188 32.62 (0.17) 49.01 (1.45) 0.38 (0.02) 23.9 62.8f

Note: We used t-tests and χ2-tests to compare sample means and proportions for PLTL participants versus nonparticipants within the main sample and among those who received the mindset intervention in fall 2015 or 2016. bFor students with SAT Math scores, concordance tables34 were used to convert those data into ACT Math scores. cThe Online Diagnostic (OD) exam was developed by the Department of Chemistry to provide an assessment of incoming students’ chemistry content knowledge.18 Range: 0−100. dAP proportion reflects students’ performance on the advanced placement exams for 4 STEM subjects: Biology, Calculus, Chemistry, and Physics. For each exam with a score of 4 or 5 (out of 5), the AP proportion score increased by 0.25. eUnderrepresented minority (URM) students include those who self-identified as African American or Black, Hispanic or Latinx, American Indian or Alaska Native, or Native Hawaiian or Pacific Islander.16 fp < 0.05. a

develop advanced study and problem-solving skills and transition successfully into the university. Asians were excluded from the sample because they are slightly overrepresented in STEM degree attainment in the U.S.,16 and because historical data at this institution indicate that they typically outperform white students in General Chemistry I. Thus, their inclusion in a non-URM group might exaggerate the presence of racial disparities and any mitigating effects of PLTL. In addition, two groups of consenting students were removed from the primary sample because they participated in social-psychological interventions intended to improve their General Chemistry performance. First, a subset of students from 2015 and 2016 participated in a growth-mindset intervention as part of an experimental study (N = 278). As reported by Fink et al. (2018),31 students self-administered three doses of the intervention as part of their online homework, earning them up to three out of 30 homework points for the semester. The growth-mindset treatment significantly improved the performance of underrepresented minority students, when compared with a control treatment providing tips about transitioning into college. In contrast, the intervention had no significant effect on white students. Nonetheless, in this study all growth-mindset participants were examined separately, to avoid biasing the sample and also to test for an interaction between the mindset intervention and PLTL participation. Specifically, mindset participants were compared to their counterparts from the control condition (N = 299), who remained in the primary sample data from fall 2015 and 2016. As in the experimental study,31 t-tests confirmed that the mindset and control participants were matched in terms of academic preparation (ACT Math, AP Proportion, OD; ps > 0.05), and χ2-tests showed no differences in terms of sex, race, or PLTL participation (ps > 0.05; Supporting Information, Table S1). The second group of students was removed from the current study altogether (N = 84). These students completed General Chemistry I during fall 2016, when the growth-mindset experiment was ongoing; however, their recitation instructor also administered a self-affirmation intervention. Following Fink et al.’s (2018)31 procedure, these students were eliminated from the sample to avoid washing out the social− psychological intervention of interest (i.e., the mindset intervention). Moreover, this group was too small to support further analyses examining interactions between such interventions and PLTL.

other types of variation) . The current program follows this trend in the literature: students self-select into General Chemistry PLTL through an online application, which guarantees assignment to a PLTL group. From that point on, the PLTL program generally follows the standard model. PLTL is coordinated by a regular member of the General Chemistry instructor team, enhancing the connection between PLTL and the course. The peer leaders participate in two training courses: one semester-long course is taken during their initial semester as a peer leader to learn about the PLTL philosophy, teaching-and-learning research, and the skills they need to effectively facilitate collaborative problem-solving; the other semester-long course is taken every semester they facilitate a group, in which they prepare for that week’s PLTL session by working the problems in small groups using collaborative-learning strategies. Both of these courses are taught by regular members of the General Chemistry instructor team. Once students enroll in PLTL, participation becomes mandatory and they sign a contract agreeing to the following responsibilities: regular attendance, allowing no more than two excused absences before removal from PLTL; arriving on time and prepared to apply the week’s content; cooperative study; participation in new activities with an open mind; participation in PLTL program evaluations; and no outside discussion of the current PLTL problems until all groups have met for that week. PLTL groups comprise an average of ten participants (slightly more than the standard model, but this size is seen in other PLTL programs) and their peer leader, meeting weekly for 2 hours on Saturdays or Sundays. Afterward, the week’s problems are posted on the course Web site, making them available to nonparticipants. In keeping with the literature,3−6 the solutions to the PLTL problems sets are never shared with students, although instructors will facilitate problem solving during help sessions. Sample Selection

The primary sample (N = 1254; 54.5% female) includes consenting first-year students who completed General Chemistry I during the fall semesters of 2012 through 2016 and who self-identified as white or an underrepresented minority. On average, 93% of students enrolled in the course each fall provided informed consent to share course data and student records with the research team. The sample was limited to first-year students because they represent the majority in this introductory STEM course, which builds students’ chemistry-content knowledge, but also helps them C

DOI: 10.1021/acs.jchemed.8b00375 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Students who completed the PLTL program (i.e., without exceeding the two allowed absences) are considered PLTL participants. Table 1 presents the characteristics of the main sample and growth-mindset participants, with t-tests and χ2tests comparing the sample means and proportions of PLTL participants versus nonparticipants. These comparisons revealed that females were more likely to participate in PLTL than males (sample: χ2(1, N = 1254) = 25.68, p < 0.05, φ = −0.14; mindset: χ2(1, N = 278) = 5.82, p < 0.05, φ = −0.15). This finding aligns with the results of Hockings et al. (2008),20 who evaluated this PLTL program early in its adoption. In contrast to their results, which revealed weaker ACT Math scores and less AP experience among PLTL participants than nonparticipants, no other significant differences emerged between participants and nonparticipants (ps > 0.05). Thus, although self-selection into the study (via consent) and PLTL (via online application) could have biased the groups in our sample (see Limitations), we have evidence they are minimally different.

and minority students). This ANCOVA included three covariates to capture variability in students’ high-school academic preparation, which typically relates to General Chemistry 1 performance. This approach increased sensitivity for detecting effects of the independent variables (sex, race, and PLTL) by reducing the variance attributed to error. ACT math scores indexed students’ mathematical abilities, which have been shown to correlate with general chemistry achievement.38−40 Scores from the Department of Chemistry’s Online Diagnostic (OD) exam18 assessed students’ chemistrycontent knowledge. Last, students’ scores on STEM-related AP exams provided a proxy for their high-school experience with more rigorous, college-preparatory coursework, which has also been shown to correlate with general chemistry achievement.39,41 Specifically, an “AP Proportion” score was calculated for each student, reflecting the proportion of STEM-related AP exams (Biology, Calculus, Chemistry, and Physics) where they earned a score of 4 or 5. For each AP exam where the student excelled, their AP Proportion score increased by 0.25 points (maximum value = 1). Next, a multiple-regression analysis examined whether PLTL provided greater benefits to students with weaker academic preparation (research question 3). All three measures of academic preparation and their interactions with PLTL were combined into a single model. PLTL participation was dummy-coded (no = 0, yes = 1), and continuous variables were centered during the creation of interaction terms. Although the academic predictors correlated with one another, this approach determined the variance uniquely explained by each predictor. In addition, the interaction terms explored which type(s) of academic weakness PLTL best addresses. For example, PLTL may offer greater advantages to first-year students with weak (versus strong) chemistry-content knowledge (i.e., lower OD scores), because it provides additional contact time with course content. Another (not mutually exclusive) possibility is that PLTL offers larger benefits to incoming students with less (versus more) college-preparatory experience (i.e., lower AP Proportion scores), because it structures their study and provides facilitated practice with complex problem solving. Finally, a second ANCOVA assessed whether PLTL interacts with other strategies to support underrepresented groups (research question 4), using the growth-mindset intervention as a test case. This analysis, which involved only students from fall 2015 and fall 2016 who received a mindset or control intervention (randomly assigned), tested for the main effects and all interactions of three independent variables: PLTL, intervention (growth mindset vs control), and race. Covariates included the same academic predictors as the first ANCOVA. To interpret significant interactions, posthoc Tukey comparisons were used. For all analyses, significance was evaluated with α = 0.05. For the ANCOVAs, partial eta-squared (ηp2) provided effectsize estimates, with 0.01, 0.06, and 0.14 serving as the thresholds for small, medium, and large effect sizes, respectively.42 For the multiple-regression model, standardized regression coefficients (β) were used to compare the relative impact of each predictor on the dependent variable. Standardized coefficients are derived by setting the variance of all variables (dependent and independent) to 1; as a result, they allow direct comparison among predictors, but they obscure the absolute size of each effect. Unstandardized coefficients provide absolute effect-size estimates, with a 1-unit increase in

Analyses

All statistical analyses were conducted using the open-source software R.35 Analyses of covariance (ANCOVAs) and a multiple regression analysis were computed using built-in functions (e.g., lm, aov), and mixed-effects models were computed with the lme function from the package lme4,36 with p-values of coefficients obtained by using the package lmerTest,37 which modifies the lme function by allowing coefficient p-values to be computed with the Satterthwaite method used by SAS. Primary Analyses. The primary purpose of our analyses was to illuminate the interactions between PLTL and other variables. We conducted ANCOVAs when examining the PLTL interaction with other categorical variables such as race and sex, because ANCOVAs allow straightforward interpretation of both main and interactive effects of categorical variables, after accounting for continuous covariates in the model. ANCOVA provides adjusted means for the groups of interest, which can facilitate interpretation of the results. Multiple regression was utilized to examine the interaction of the PLTL effect with continuous variables. Interpretation of these interactions was facilitated by plotting the PLTL effect across different levels of the continuous variable. The dependent variable for our primary analyses was General Chemistry exam average, which was z-scored within each academic year. Following the course instructors’ procedure, we calculated exam average by averaging each student’s highest two (out of three) unit-exam scores with their cumulative final-exam score. The primary advantage of this index is that it accurately reflects the way students’ course grades are calculated and thus the way instructors have decided to assess exam performance across the semester. Also, this procedure accounts for the fact that students may exhibit lower motivation and performance on one of their unit-exams (e.g., the third), because they know that their lowest unit-exam score will be dropped. An ANCOVA addressed the first research question, examining the main effects of sex (female, male), race (white, URM), and PLTL (no PLTL, PLTL) on General Chemistry exam averages. Because the ANCOVA tested the interactions of PLTL with sex and race, it also informed the second research question of whether PLTL offers greater benefits to groups underrepresented in STEM (i.e., females D

DOI: 10.1021/acs.jchemed.8b00375 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Table 2. Unadjusted and Adjusted General Chemistry I Mean Exam Averages for Primary Sample by Race, Sex, and PLTL Participation Unadjusted Means (SE, N)a

Demographic Variables Race

Sex

URM

Female Male Female Male

White

No PLTL 53.72 57.37 63.01 64.14

(1.75, (1.90, (1.05, (0.88,

Adjusted Means (SE)b

PLTL

47) 40) 132) 188)

59.71 62.94 69.70 71.89

(1.11, (1.39, (0.61, (0.73,

No PLTL 118) 75) 385) 269)

58.99 60.27 63.07 61.80

PLTL

(1.48) (1.59) (0.87) (0.74)

64.92 65.26 69.38 69.49

(0.95) (1.16) (0.51) (0.62)

a

SE indicates standard error, and N indicates sample size. bAdjusted means account for variation in ACT Math, OD scores, and AP Proportion.

an independent variable corresponding to a 1-unit increase in the dependent variable. Finally, Cohen’s d was computed to evaluate the difference between two means (i.e., the effect-size corresponding to a t-test; small = 0.20, medium = 0.50, large = 0.8043), and phi (φ) was computed to evaluate the difference between two categorical outcome frequencies (i.e., the effectsize corresponding to a Chi-squared (χ2) test; same thresholds as d). Although exam averages were z-scored in all analyses, untransformed sample means (M) and standard errors (SE) are presented in the figures. Unadjusted and adjusted groupmeans are presented in Table 2. Supplementary Analyses. A potential confound in our data is that PLTL students may be more likely than No PLTL students to participate in the General Chemistry I Transition Program,18 which is designed to supplement instruction for less prepared students. Particularly at lower levels of preparation, other supplementary activities besides PLTL might at least partially account for performance differences between the PLTL and No PLTL groups. In order to ensure that this confound did not account for our pattern of results, we conducted supplementary analyses (Supporting Information, Tables S2 and S3, Figures S1−S3) to confirm that our pattern of results held when we eliminated this confound by removing all Transition Program participants from the analysis. In these analyses, 195 Transition Program participants were removed from the analysis, resulting in N = 1059. A second potential issue for our analyses is that exam scores were combined in an unconventional way to create our dependent measure of exam average. Although our procedure was modeled after the course instructors, we still wanted to ensure that aggregating exam scores in this way did not generate spurious results. Thus, we conducted supplementary analyses (Supporting Information, Tables S4 and S5, Figures S4−S6) to confirm that our pattern of effects replicated when the cumulative final-exam score, rather than exam average, was used as the measure of performance. In these analyses, final exam scores were z-scored within each academic year. In the Supporting Information, we also report analyses of our data using a mixed-effect model.We utilized ANCOVAs and regressions as our primary analyses because they are widely understood and easily interpretable analyses that accurately represent the patterns in our data. However, one drawback of these methods is that they do not account for dependencies in the datanamely that individual exam scores can be treated as multiple observations nested within students, who are in turn nested within semesters. To alleviate these concerns, we also analyzed our data using a mixed-effects model that accounts for the hierarchical structure in the data (Supporting Information, Tables S6). In this analyses, the dependent measure was not a single index of exam score. Rather, the four exams (including the cumulative final exam)

were treated as multiple observations of exam performance nested within student.

■

RESULTS

PLTL Improves the Performance of All Identity Groups Equally

As shown in the first ANCOVA table (Table 3), all three covariates had significant, positive effects on General Table 3. ANCOVA Testing for Demographic Variation in the PLTL Effect Variablea OD AP Proportion ACT Math PLTL Race Sex PLTL × Race PLTL × Sex Race × Sex PLTL × Race × Sex Residuals

SS 89.36 40.17 34.10 38.12 11.01 0.00 0.79 0.02 0.47 0.32 729.49

df 1 1 1 1 1 1 1 1 1 1 1243

MS 89.36 40.17 34.10 38.12 11.01 0.00 0.79 0.02 0.47 0.32 0.49

ηp2

F b

152.27 68.44b 58.10b 64.96b 18.75b 0.01 1.35 0.03 0.80 0.54 

0.11 0.05 0.05 0.05 0.02 0.05).

the patterns found in the primary analysis, with significant effects of PLTL along with OD, ACT Math, AP Proportion, and Race (Supporting Information, Table S6). PLTL Especially Helps Students with Less College Preparation

Pearson’s correlation coefficients demonstrated that the academic-preparation variables were moderately correlated (ACT Math & OD: r(1254) = 0.37, p < 0.05; ACT Math & AP Proportion: r(1254) = 0.36, p < 0.05; OD & AP Proportion: r(1254) = 0.43, p < 0.05). However, the multiple-regression analysis (Table 4) confirmed that each variable uniquely predicted students’ General Chemistry I exam averages. For example, students with higher AP Proportion scores earned better exam averages in the course (β = 0.29, p < 0.05; Figure 3). Higher ACT Math scores (β = 0.20, p < 0.05) and OD scores (β = 0.33, p < 0.05) also predicted better performance. Figures 4 and 5 represent these patterns, depicting ACT and OD scores as categorical for ease of visualizing even though they were continuous in the analysis. PLTL had a significant, positive effect on students’ exam F

DOI: 10.1021/acs.jchemed.8b00375 J. Chem. Educ. XXXX, XXX, XXX−XXX

Journal of Chemical Education

Article

Figure 3. Unadjusted mean exam averages (standard error bars) from General Chemistry I by AP Proportion score and PLTL participation. AP Proportion reflects students’ performance on the Advanced Placement exams for 4 STEM subjects; each AP exam score of 4 or 5 (out of 5) increased their AP Proportion score by 0.25. Numbers near the base of the bar represent the number of observations in the group.

Table 5. ANCOVA Testing the Interaction of PLTL and a Growth-Mindset Interaction Variablea OD AP Proportion ACT Math PLTL Race Intervention PLTL × Race PLTL × Intervention Race × Intervention PLTL × Race × Intervention Residuals

Figure 4. Unadjusted mean exam averages (standard error bars) from General Chemistry I by ACT Math score and PLTL participation. ACT Math was a continuous predictor in all analyses, but students were binned into quartiles for illustration purposes. Numbers near the base of the bar represent the number of observations in the group.

SS

df

47.11 17.30 16.64 26.34 5.51 4.88 0.17 2.61 3.98 1.07

1 1 1 1 1 1 1 1 1 1

47.11 17.30 16.64 26.34 5.51 4.88 0.17 2.61 3.98 1.07

277.16

566

0.49

ηp2

F

MS

b

96.20 35.32b 33.98b 53.80b 11.25b 9.97b 0.36 5.34b 8.13b 2.18

0.15 0.06 0.06 0.09 0.02 0.02