Research: Science and Education edited by
Chemical Education Research
Diane M. Bunce The Catholic University of America Washington, D.C. 20064
An Investigation of the Factors Influencing Student Performance in Physical Chemistry Gayle Nicoll and Joseph S. Francisco* Department of Chemistry, Purdue University, West Lafayette, IN 47907; *
[email protected] Physical chemistry courses are traditionally considered difficult, from both the students’ and the professor’s point of view. Our study found that students enter the class with negative perceptions of physical chemistry and low expectations of both the course and their ability to succeed in it. Professors, in turn, have their own perceptions of what qualities students need to succeed in their course and what types of topics should be covered. To determine if these perceptions have any factual basis and whether students and professors’ views are in agreement, it is necessary to evaluate how these perceptions compare with student performance in the course, which is what this study sought to accomplish. Determining what factors actually do influence students’ performance will not only help the professor improve the course, but may also help students achieve a deeper understanding of physical chemistry. While there is a growing body of research to determine what factors affect students’ performance in general chemistry courses (1), relatively little research of this nature has been done at the physical chemistry level. Most of the published research points only to a correlation between students’ math proficiency and their success in physical chemistry courses (2). This is consistent with one of the biases that professors appear to hold: that math proficiency is the key to success in physical chemistry. However, there are other factors that may be just as good, if not better, predictors of success in physical chemistry. As general chemistry courses have been likened to “watered down physical chemistry” courses, it stands to reason that some of the same factors that are good predictors of success in general chemistry may also be good predictors for physical chemistry. For instance, while some researchers have found a correlation between math skills and student achievement (3), others have also used the number of math courses that students have taken (4 ). Some researchers have proposed that it is students’ ability to disembed information or their ability to problem solve that is strongly correlated to achievement in general chemistry (5). Still others have suggested that achievement is correlated to students’ attitudes about the course and to their logical thinking skills (6 ). Because these factors have been suggested as strong predictors of success at other chemistry levels, these were the factors that were investigated in this study. Before continuing, it is necessary to address the similarities and differences among these measured traits. Math proficiency in and of itself requires specific content knowledge of how to do various mathematical operations, such as integration or partial differentiation. In contrast, the ability to disembed information refers to students’ ability to pick out salient information from a passage, and problem solving refers to students’ ability to use their general background knowledge
and apply it to novel situations. In a related vein, students’ logical thinking skills refer to their ability to apply logic and reason their way through a situation. Theoretically, students should use all these skills while attempting to solve a physical chemistry problem. However, they are distinct skills that students acquire throughout their educational careers and use to varying degrees depending on the setting. There were two aspects of this study. We studied students in two physical chemistry classes, to determine what factors influenced their performance in physical chemistry, what their perceptions were of their own ability, and what their perceptions were of the course. We also surveyed physical chemistry professors across the country to determine what factors they believe influence students’ performance in physical chemistry. We then compared these responses to what students in the two classes believed and to how they actually performed in the course. Methodology
Student Study The students involved in the study were enrolled in two classes taught by the same professor using the same midterm and final exam in the two classes. This professor had a sabbatical leave and was thus able to teach the same physical chemistry course at two different universities during the same school year. The first class, comprising 68 students, was a course for chemistry and chemical engineering majors at a midwestern Big Ten university. The second class, comprising 9 students, was a physical chemistry course for chemistry majors at a small, elite east-coast college. A comparison of the two classes with respect to the average number of semesters of math the students had taken, the average number of chemistry courses they had taken, the average number of credit hours they were enrolled in, the average course grade, and the average class standing of the students in the two courses found no statistical difference between the classes based on a t test at the α = .05 level. Because of this, the data from the two classes were combined for further analysis. The study was based on a battery of assessments administered throughout the semester. On the first day of class, before students had seen the syllabus for the course, they were asked to complete the Student Perceptions Inventory (Kuder– Richardson test: KR-20 = .97). In this survey, students respond to paired statements about their own perceptions of their math ability, according to the 5-point Likert scale format. Students were also given a list of statements about the course and were asked to circle those statements that best represented what they had heard about the course from friends and peers. The statements ranged from positive (“the professor is excellent”)
JChemEd.chem.wisc.edu • Vol. 78 No. 1 January 2001 • Journal of Chemical Education
99
Research: Science and Education
to negative (“it takes too much time to study for this course”). Students were allowed to select as many statements as they felt were appropriate. Finally, they reported how many credit hours they were currently enrolled in and how many math courses they had taken or were currently taking. Results from this portion of the survey showed that the average student from either physical chemistry course had taken 4 math courses prior to taking physical chemistry, with a range from 1 to 9. On the second day of class, the Math Diagnostic (KR20 = .97) was administered to determine if mathematical ability had any correlation to student performance in the course. This diagnostic was designed by the professor of the course to assess the math skills most commonly used in physical chemistry. It comprised 10 questions: 6 calculus, 2 algebra, and 2 word problems. For an example of the types of questions asked, see Table 1. Students were given as much time as necessary to complete the diagnostic, but were not allowed to discuss answers with their neighbors. On the third day of class, students were given the Conceptual Diagnostic (KR-20 = .98), which was a combination of the Figural Intersection Test (FIT) (Pascual-Leone, J.; unpublished; Psychology Department, York University, Downsview, ON M3J 1P3) and the Group Assessment of Logical Thinking (GALT) (8). The GALT test probes students’ logical thinking ability, while the FIT probes the ability to process information. Again, students were given as long as necessary to complete the diagnostic, but most finished within 20 minutes. During the last week of class, students were given the End of Semester Student Perceptions Inventory. This inventory was presented in a parallel format to the Student Perceptions Inventory administered on the first day of class. The purpose of this inventory was to determine if students’ perceptions about their own ability or about the course had changed as a result of taking the physical chemistry course. After all grades had been assigned for the semester, a correlation analysis was done on the data. The only data used in the study were for students who completed all the diagnostics throughout the semester. However, since only one student dropped the course, there was a large pool of data for the correlation analysis. Students’ performance on all of the diagnostics was correlated with their scores on their midterm and final exams, overall point total, and final grade in the course. To establish the reliability of the battery of tests given to the students, a KR-20 test was performed on each inventory. The resulting value of the KR-20 was .91 for the
Student Perceptions Inventory, .97 for the Math Diagnostic, and .98 for the Conceptual Diagnostic.
Professor Survey The professor survey was designed to determine what factors professors perceive as important to student success in physical chemistry. Surveys were sent to 60 physical chemistry professors, who were asked to complete the surveys on a voluntary basis. Forty-seven responses were received. Every attempt was made to ensure a diverse sample distribution in terms of the size, geographic location, and national ranking of the schools (7). In all, 28 institutions are represented in the data below, including eight southern and eastern schools, seven midwestern schools, and five western schools. These comprise 10 first-tier institutions, 10 second-tier institutions, 6 third-tier institutions, and 2 fourth-tier institutions. The survey was in two parts. Professors were first asked to name the single most important factor that they felt determined students’ success in their physical chemistry course. They were then asked to rate the importance of each of 9 factors on a 5-point Likert scale, on which 1 indicated they strongly agreed with the statement and 5 indicated they strongly disagreed with the statement. The 9 factors came from the literature as points that affect student performance in other types of courses (1–6 ). Results and Discussion On the student perceptions inventory given at the beginning of the course, results from the two classes were similar. For a summary of the combined results of the students’ perceptions at the beginning of the semester, see Table 2. For the portion of the inventory dealing with students’ perceptions of their own math ability, as shown in Table 2, we found a significant correlation between those students who felt confident applying their math skills to chemistry (question 1) and those who felt that they had sufficient math to succeed in the course (question 2) (C = .75, p ≤ .0001). This would indicate that students who felt confident about their own skills in general also had positive perceptions about their ability to do well in the course or about the level of math required for the course. This was substantiated by the correlation between students who felt confident applying their math skills to chemistry (question 1) and students who enjoyed math problems in chemistry (question 3) (C = .52, p ≤ .0001). There was also a fraction of the students who entered the course with very negative perceptions, as evidenced by the correlation between students who hoped the class did not have
Table 1. Typical Problems on the Math Diagnostic
100
Math Concept
Example
Basic calculus
Integration/differentiation
Problem
Algebra
Simultaneous solution of multiple equations, matrix algebra
Advanced calculus
Differential equations, Taylor series expansion
Word problems
A mixture containing 6% boric acid is to be mixed with 2 quarts of a mixture which is 15% boric acid in order to obtain a solution which is 12% boric acid. How much of the 6% solution must be used?
dx x 1 2 + 4 1 0 6 1 3
dx = k dt
Journal of Chemical Education • Vol. 78 No. 1 January 2001 • JChemEd.chem.wisc.edu
Research: Science and Education
much math (question 5) and those who felt that too much math would lower their grade in the course (question 8) (C = .56, p ≤ .0001). This indicates that students who were not confident of their own abilities were also not confident about their performance in the course. Upon administering the perceptions inventory again at the end of the semester, it was found that students’ attitudes did not appreciably change with respect to their own math abilities. However, perceptions of the course did change. At the beginning of the semester, 87% of the students thought the class was going to be hard. At the end of the course, this number had dropped to 62% (t = 5.7, α ≤ .05). Similarly, 47% of the students thought it would take too long to study for the course before the class met, and this number decreased to 34% at the end of the class (t = 3.8, α ≤ .05). This represents a significant change in student perceptions about the course from the beginning to the end. Students’ perceptions improved simply by participating in the course for the full semester. This is particularly important in light of the fact that only one student dropped the course, and this student’s responses are not included in these numbers. It was found that students’ perceptions about either their own ability or their experience had no correlation with their performance in physical chemistry. Therefore, attitudes about physical chemistry do not appear to play an important role in determining how students will perform in the course. However, several significant correlations to student performance were found. The math diagnostic significantly correlated with both performance on the midterm (C = .51, p ≤ .0001) and the overall course grade (C = .50, p ≤ .0001). While these results indicated that math skill was a factor in determining student performance, we sought to determine what types of math skills were particularly important to physical chemistry. Breaking down the math diagnostic by the types of questions asked, we found that the item most significantly correlated with course grade was the ability to solve word problems on the math diagnostic (C = .51, p ≤ .0001). There was also a weak correlation between students’ grades and their ability to perform calculus-based problems (C = .2, p ≤ .0132). This correlation, although statistically significant, was not considered to be very large. These data are particu-
larly important because the types of word problems asked on the math diagnostic required no knowledge of either chemistry or higher math in order to solve them. Therefore, proficiency with neither basic nor advanced calculus strongly predicts student success in physical chemistry. Similarly, the number of math courses that students took had no bearing on their performance in physical chemistry (C = .13215, p ≤ .3184). Interestingly, however, students’ math proficiency was not the only factor, nor even the single best factor, to predict student success in physical chemistry. Performance on the GALT portion of the Conceptual Diagnostic correlated with both performance on the midterm (C = .49, p ≤ .0001) and overall course grade (C = .53, p ≤ .0001). Although the FIT portion of the diagnostic had no significant correlation to students’ performance in the course, the GALT correlation has some interesting implications. These results suggest that performance in physical chemistry is more accurately a function of students’ logical thinking skills. This is supported by the results from the Math Diagnostic, in which students Table 3. Correlation between have to use logical thinking Performance in Physical skills in order to solve word Chemistry and Performance on the Math Diagnostic problems. For a breakdown of the correlations between Type of Problem C α the different portions of the Calculus .2 .0132 Math Diagnostic and student Algebra .04 .7224 performance in the course, Word problems .51 .0001 see Table 3. It is interesting that no other significant correlations were found between any of the other tests and students’ performance in either class. In fact, the same predictive factors were found to apply for both classes. This would indicate that, while basic math ability may have some correlation to student success in physical chemistry, the main predictive factor is ability to think logically. In response to these results, we performed a regression analysis on the data to reflect what factors affected students’ performance in physical chemistry. The results, performed on SAS, are reported in Table 4. For this analysis, both the GALT, which was a measure of logical thinking, and the word problems from the math diagnostic
Table 2. Students’ Perceptions of Their Own Math Ability at the Beginning of the Course Rating a Statement
1
2
3
4
5
Frequency
Av
I feel confident applying my math skills to chemistry.
13
30
18
5
1
2.27
I feel that I have sufficient math to succeed in this course.
15
33
13
6
1
2.19
8
28
25
3
4
2.51
12
14
23
14
5
2.79
5
13
32
11
7
3.03
I feel comfortable seeking help from others to clarify difficult material for me. 29
20
11
7
1
1.99
I feel comfortable explaining how I solve chemical problems to others.
20
18
8
6
0
2.00
I feel that the amount of math in this course could lower my grade.
12
27
14
9
6
2.56
I enjoy the challenges.
13
19
6
9
5
2.50
2
11
29
13
4
3.10
The grade I receive in this course is less important than what I learn.
18
20
11
8
1
2.21
I expect to learn a lot in this course.
16
34
12
5
1
2.13
I enjoy the math problems in this chemistry. I think I should brush up on my math for this course. I think this class will have too much math in it.
I am only taking this course because it is required.
a Ratings
are on a Likert scale of 1 (strongly agree) to 5 (strongly disagree). Results from the two classes are
combined.
JChemEd.chem.wisc.edu • Vol. 78 No. 1 January 2001 • Journal of Chemical Education
101
Research: Science and Education Table 4. Regression Analysis Performed on Significant Predictors of Success in Physical Chemistry Parameter Intercept
Parameter Estimate
SE
t for H0: Parameter = 0
p > |t |
4.172391
0.24549653
16.996
.0001
GALT
0.065149
0.01757344
3.707
.0004
Word problems
7.70E07
0.00000024
3.156
.0024
NOTE: H0 is the null hypothesis, and the p value is the probability of exceeding the absolute value of t.
The results from this study are consistent with this view. First, professors and students have different perceptions about what factors influence students’ performance in the course. Second, neither group’s responses match reality. We found that there is no single predictor of performance in physical chemistry. More importantly, math skills alone are not even the best predictor of performance. Moreover, the number of math courses that students take before entering physical chemistry has no correlation with actual performance in physical chemistry. Correlations from the GALT test indicate that logical thinking skills and basic math ability are important factors. It may be that logical thinking skills help students make the connections and help anchor their math to realworld examples. There has recently been an explosion of new knowledge from emerging areas of physical chemistry and a need to incorporate this new knowledge into the physical chemistry curriculum. However, the results from this survey suggest that instructors must weigh these factors when designing the physical chemistry curriculum, to ensure that students have the most opportunity to learn the material presented.
proved to be significant in predicting students’ success in physical chemistry. These results are particularly interesting in light of the results from the professor survey. For a summary of the survey results, see Table 5. In response to the first section of the survey, professors stated that math ability was the single most important factor determining student performance. The second part of the survey, in which professors used a Likert scale to rate the importance of various factors for student performance in their course, indicated that professors believed basic math Literature Cited skills to be the primary factor influencing success in physical chemistry, followed closely by student motivation in the 1. Hurst, R. W.; Milkent, M. M. J. Res. Sci. Teach. 1996, 33, 541–552. course. Thus in both sections of the survey, professors stated that they felt it was students’ math ability that affected their 2. Butler, L. J.; Butler, L. M. Abstracts of Papers, Part 1, 213th performance in the course. In the Likert scale responses, National Meeting of the American Chemical Society, San basic math skills had a value of 1.17, which represents a very Francisco, Apr 13–17, 1997; American Chemical Society: strong belief that math skills are an important factor in the Washington, DC, 1997; CHED 213, p 842. course. Just as important, however is the fact that basic math 3. Bers, T. J. Appl. Res. Commun. Coll. 1997, 4 (2), 101–117. skills had a very narrow distribution of responses, indicating 4. House, J. D. Res. Higher Educ. 1995, 36, 473–490. that all the professors surveyed held very strong convictions 5. Niaz, M. Educ. Psychol. 1994, 4 (1), 23–43. about this. 6. Bitner, B. L. J. Res. Sci. Teach. 1991, 28, 265–274. The professors considered logical thinking skills slightly 7. 2001 College Rankings; U.S. News Online Aug 30, 1999; less important in predicting student success in physical chemhttp://www.usnews.com/usnews/edu/college/corank.htm (accessed istry (1.30) than math ability (1.23). A t-test to determine if Sep 2000). these values were significantly different found that they were 8. Roadrangka, V.; Yeany, R. H.; Padilla, M. J. J. Res. Sci. Teach. not (t = 1.63, α < .90). Although the average response indi1985, 22, 743–760. cates that professors think the skill is important, the distribution of responses implies that not all Table 5. Professors’ Perceptions of Factors That Predict Success in professors value this skill as much as may be warPhysical Chemistr y ranted by the results of this study. a Rating
Summary The original intent of this study was to determine what factor(s) influence the performance of students in physical chemistry courses. Identification of such factors could help instructors organize physical chemistry courses or lectures to best reach a larger group of students. In view of the negative perceptions of students about physical chemistry and the frustration physical chemistry professors are experiencing in the classroom (as evidenced by the disparity between the results of the students’ and professors’ surveys), it is clear that students and professors are at odds.
102
Factor
1
2
3
4
5
Frequency Advanced math skills (e.g., differential equations)
Av
8
18
15
7
0
2.44
Basic math skills (e.g., calculus, algebra)
36
11
0
0
0
1.23
Disembedding information (e.g., word problems)
23
13
8
3
1
1.88
Logical thinking skills (e.g., problem solving)
37
6
4
0
0
1.30
Number of chemistry courses taken
1
6
27
11
2
3.15
Number of math courses taken
4
20
20
1
1
2.46
Motivation
33
15
0
0
0
1.31
Perceptions of physical chemistry
10
13
15
7
1
2.48
Study skills
19
28
1
0
0
1.63
a Ratings
are on a Likert scale of 1 (strongly agree) to 5 (strongly disagree).
Journal of Chemical Education • Vol. 78 No. 1 January 2001 • JChemEd.chem.wisc.edu
