Article Cite This: J. Chem. Educ. XXXX, XXX, XXX−XXX
pubs.acs.org/jchemeduc
Survey of Undergraduate Students’ Goals and Achievement Strategies for Laboratory Coursework Stephanie Santos-Díaz,† Sarah Hensiek,‡ Taylor Owings,§ and Marcy H. Towns*,† †
Department of Chemistry, Purdue University, West Lafayette, Indiana 47907-2084, United States Department of Chemistry, Salisbury University, Salisbury, Maryland 21801, United States § North Montgomery High School, Crawfordsville, Indiana 47933, United States ‡
Downloaded via KEAN UNIV on April 27, 2019 at 20:34:46 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.
S Supporting Information *
ABSTRACT: Previous work has shown a misalignment between the goals of faculty and students for laboratory coursework. Although faculty often list learning techniques and developing critical thinking skills as goals for laboratory, students tend to focus on affective goals, such as finishing lab early and obtaining good grades. The goals of faculty have been thoroughly investigated through both qualitative and large-scale quantitative studies. However, studies on student goals for laboratory coursework have primarily been conducted using qualitative methods. In order to provide a more comprehensive idea of students’ goals in the laboratory, we have designed a brief, online survey. The survey was used to investigate students’ goals in the laboratory and build on existing literature by including survey questions that catalog how students plan on achieving their goals. In this study, we discuss results about students’ goals and achievement strategies at the beginning of the semester and after completing the laboratory portion of the course. Anonymous responses, via the online survey, were collected from undergraduate students enrolled in chemistry courses at two Midwestern universities. These results are intended to inform the discussion among faculty regarding how to change laboratory coursework and the instruction to better align faculty and student goals. KEYWORDS: First-Year Undergraduate/General, Second-Year Undergraduate, Upper-Division Undergraduate, Laboratory Instruction, Testing, Laboratory Management, Chemical Education Research FEATURE: Chemical Education Research
■
INTRODUCTION Throughout the history of chemistry laboratories, there have been conflicting ideas about the role of laboratories in students’ science education.1−4 Chang and Lederman have stated that laboratory instruction in chemistry has been thought of as irrelevant, given both the lack of evidence suggesting laboratory experiments enhance students’ understanding5,6 and existing concerns regarding financial and time costs.7 To help address part of the issue of the purpose of laboratories in science education, researchers on laboratory instruction5,8,9 have called for researchers and instructors to define goals and learning objectives specifically for laboratory coursework to justify their place in the curriculum as separate from the learning that occurs in a nonlaboratory chemistry course. Recent research10 suggests that defining clear goals in laboratory coursework can be beneficial for students. The lack of well-articulated goals for laboratory coursework and uncertainty of the role of laboratory coursework gave rise to research in this field. Meaningful learning, as described by Novak’s theory of education,11 has been used as a framework to characterize faculty goals for laboratory coursework.12−18 In the meaningful learning model, learning experiences occur in three domains, cognitive (C), affective (A), and psychomotor (P), with meaningful learning occurring when learning experiences at © XXXX American Chemical Society and Division of Chemical Education, Inc.
these domains come together. Goals in the cognitive domain are those related to understanding concepts explored with experiments. The affective domain includes goals related to feelings (i.e., enjoyment from getting a good grade), motivation, values, and attitudes. Psychomotor involves goals related to techniques or technical skills, such as using a buret. As Bretz et al. stated, faculty are responsible for providing students with experiences across the CAP domains.19 Interviews of faculty in general chemistry, organic chemistry, and upper-division laboratory courses revealed learning laboratory skills and techniques (psychomotor domain) along with developing critical thinking skills and experimental design (cognitive and psychomotor domains) as common goals across courses.20 General Chemistry faculty mentioned other goals specific for courses such as connecting lecture with lab (cognitive domain), engaging in science (affective domain), and teamwork skills (cognitive and psychomotor domains). A larger, quantitative study among faculty across the United States produced very similar results,21 suggesting that goals across CAP domains are present in general chemistry, but affective goals vanish in upper level courses.19 Received: November 29, 2018 Revised: April 18, 2019
A
DOI: 10.1021/acs.jchemed.8b00984 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Article
Galloway and Bretz designed an assessment tool to measure general chemistry and organic chemistry students’ meaningful learning in the laboratory.12−14 The Meaningful Learning in the Laboratory Instrument (MLLI) generates valid and reliable results specifically about students’ cognitive and affective expectations and experiences in the laboratory course.13 A preand post-test comparison showed that a subset of the General Chemistry students sample had expectations about their laboratory learning that were misaligned with their experiences in the course.12 In order to provide more insight regarding students’ perspectives about laboratory coursework, DeKorver and Towns15 interviewed students about their goals for lab and categorized their responses about goals using all the CAP domains. Meaningful learning is hindered when goals across domains are in conflict. For example, if a student wants to finish lab early, he/she may have his/her partner who is more proficient at a technique carry it out rather than taking the opportunity to practice and perhaps do it incorrectly because it would take more time. In this example, learning and practicing the technique is a psychomotor goal and finishing lab early is an affective goal; thus, these two conflicting goals hinder learning when a student chooses to prioritize the affective goal. The findings presented by DeKorver and Towns15 suggest a misalignment between faculty and student goals in the laboratory course, as the faculty prioritized goals in the cognitive and psychomotor domain while students mostly described goals in the affective domain. The qualitative nature of previous research on undergraduate students’ perceptions, experiences and goals in laboratory coursework led us to design a survey to quantitatively explore the aforementioned issue. In this study, we present the survey as a tool to collect data to address the following research questions:
The second stage in development of the survey consisted of implementing a pilot survey.23 The pilot survey consisted of two sections. The pilot survey was implemented at a large Midwestern R1 institution, with data collected from two large general chemistry courses. The participants (n = 904) were engineering, agriculture, and health and human sciences majors. For the first section, students were given a list of 36 goal statements, which emerged from the first stage, and were asked to respond whether it was goal of theirs or not (i.e., “Yes/No” format). The second section used a Likert Scale to measure the student’s level of agreement with approximately 24 achievement strategy statements that emerged from the first stage. Resulting data from the first pilot data collection showed limited variability in the responses, and a large number of students did not respond correctly to the reading check question.23 For this reason, the response format for the goals section of the survey was also changed to Likert Scale, which asked students to indicate the degree of importance per goal. After a second pilot data collection, the length of the survey and the response format of both items were still identified as limitations of the instrument. During the third stage of the development of the survey, the focus was to address the identified limitations, change the response format, and generate a final form of the survey which could be implemented and refined. In order to decrease the number of statements included in the final survey and avoid response fatigue, the researchers revisited the data from stage 1 to combine similarly worded and redundant statements. Reliability was addressed by having two new members of the research team review the initial responses from stage 1 and discuss and come to an agreement on which statements to keep and which wording accurately represented students’ responses. An example to demonstrate how this process worked is the removal of the statements “To be more comfortable with glassware” and “To learn error analysis procedures/calculations” found in previous iterations of the survey because goals G6 and G11 (in Table 1) of the current
(1) What are undergraduate students’ most important goals and least important goals in a laboratory course? (2) What strategies do undergraduate students consider most relevant to achieving their goals in the laboratory?
■
Table 1. Goals Presented in Q1 of the Survey
METHODS
Goals G1 G2 G3 G4 G5 G6 G7 G8 G9 G10 G11 G12 G13
Development of the Survey
Survey development involved three stages, all of which were approved by Purdue University’s Institutional Review Board (IRB). The first stage consisted of identifying undergraduate students’ goals and achievement strategies for chemistry laboratory coursework. In order to do so, participants were surveyed through two open-ended questions: What are your goals for lab? and How do you plan on accomplishing these goals? The students surveyed were from two different institutions: a large research-intensive (R1) institution and a smaller Midwestern highly research-active liberal arts institution. Participants at the R1 institution (n = 787) were science and engineering majors, enrolled in either general chemistry or physical chemistry. Participants at the second institution (n = 138) were science majors enrolled in an organic chemistry course. Student responses were coded until reaching saturation22 (approximately 255 student responses) and grouped into broader categories.23 To address reliability, two researchers coded a subset of student responses, approximately 38 responses, using the list of all codes. Initial inter-rater reliability was 80%; after discussing discrepancies until reaching an agreement, inter-rater reliability was 99%.23
Statement To To To To To To To To To To To To To
earn an A or B prepare for the career I want to pursue develop my scientific writing skills make connections between lab and the real world understand how a chemistry research lab works learn lab techniques be efficient in lab prepare for future science courses connect concepts learned in lectures with laboratories work as a team learn how to design and carry out experiments carry out experiments safely apply lab techniques
survey encompass the same goal. The final survey, used to collect the data presented in this study, included 13 goal statements (Table 1) and 10 achievement strategy statements (Table 2). The final survey is available in the Supporting Information. When students indicated how strongly they agreed or disagreed whether each goal was important (i.e., completed the survey with a Likert Scale response format), the results B
DOI: 10.1021/acs.jchemed.8b00984 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Article
teaching chemistry courses at several research-intensive (R1) institutions located in the Midwest who have formed a strong undergraduate laboratory-based community of faculty and instructors and have a large enough student population. The sole role of faculty was to share the link with their general chemistry students via e-mail or a post on their course’s online platform. Out of 10 institutions contacted, three large Midwestern R1 institutions accepted the invitation to participate in the study. The survey was sent out at two instances: at the beginning of the semester (presurvey) and at the end of the semester (postsurvey). In both instances, the survey was available to the participants for 2 weeks. However, there were no postsurvey submissions for one of the three institutions. Here, the results for the two institutions that completed both the pre- and postsurvey will be discussed. These institutions will be referred to as Institution A and Institution B. For IRB purposes, the survey was designed to be anonymous, so it cannot be verified if the students who completed the presurvey also completed the postsurvey. While the intent was to survey students enrolled in general chemistry courses, the link to the survey could have been easily shared (i.e., anyone who had the link could have sent it to other students). However, according to the data collected, the majority of the participants were enrolled in a general chemistry course and less than 1% of participants were enrolled in other chemistry courses (at either implementation). Table 3 displays the student distribution for the pre- and
Table 2. Strategies Presented in Q2 of the Survey Strategies
Statement
S1 S2 S3 S4 S5 S6 S7 S8 S9 S10
Not procrastinate Seek out help Analyze data Take detailed observations Manage my time efficiently Come to lab prepared Discuss lab after class Finish lab as quickly as I can Attempt to understand what I am doing Read the lab before my laboratory period
describing the population revealed students rated all the goals as “extremely important” or “somewhat important.” These results were expected since all goal statements were based on previous student responses identifying the goals that all students had for the laboratory. However, this information is not particularly useful for instructors. For this reason, the response format for the questions pertaining to goals was changed from “Yes/No” and Likert scale to classifying the 13 goal statements in three categories: most important, important, and least important. These categories were created based on the assumption that all goals are important for students as the statements were produced from student responses to the survey in the first stage. Compared to the previous formats of the survey, the new format allows for a clearer differentiation in importance or prioritization of goals than a “Yes/No” or Likert scale format. When classifying goals, the students need to think about which goals to prioritize within already-important goals to decide on a category. Students must consider all the goals together in relation to each other, which represents choices they may have in lab prioritizing some goals over others. In other words, by classifying the goals, the students can be more specific about the extent to which they consider certain goals important. The response format for the question pertaining to achievement strategies was changed from a Likert scale to selecting the top five strategies used to achieve the most important goals. We believe that the results from this response format can better address our research questions when compared to a Likert scale format. A Likert scale does not allow for explicit differentiations to be made. Students can rate all statements the same or similarly, as had been seen in previous implementations of the survey. Across large data sets it is hard to distinguish between an “extremely important,” “somewhat important,” or “neutral” goal, especially when the numerical scale values are used in quantitative analysis. By having categories, the differentiation is more explicit since the students must choose which goals to prioritize over others. The final survey presented the list of goals and the list of strategies in the same order as presented in Tables 1 and 2. The final online survey (see Supporting Information) included two additional questions: course enrolled in and major. While students were given specific options, they could input their own option if their course or major was not included. The intention with these last two questions was to obtain some demographic information about participants, providing a way to group participants for future comparisons.
Table 3. Student Participant Distribution Survey Completion Instances, N Institution
Presurvey
Postsurvey
A B Total
204 134 338
170 120 290
postsurveys. One participant was excluded from the data analysis as their submission had all goals classified in one category. Responses included in the data analysis were complete, meaning the participants classified the goals in different categories and answered the other questions in the survey. There was a total of 338 participants who completed the presurvey and 290 participants who completed the postsurvey. Statistical Analysis
The response format for the survey used in this study included classifying 13 goal statements into three categories and selecting the top five strategies students used to achieve the most important goals. The type of data collected with this response format was frequency counts (i.e., how many students assigned a goal to each category or how many students selected a strategy as one of the top five). In order to determine which goals students believed were most and least important, a chisquare test of homogeneity and post hoc analysis involving pairwise comparisons using the z-test of two proportions with a Bonferroni correction was performed.24 The chi-square test of homogeneity allows for the determination of significant differences between proportions, with the null hypothesis being the proportions are equal in the population. In the context of this study, it was used to determine if the proportions of students selecting a goal as most important, important, or least important were equal. A significant
Implementation of the Survey
Once the participating institution’s IRB approved the study, an invitation to complete the survey was sent out to faculty C
DOI: 10.1021/acs.jchemed.8b00984 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Article
< 0.0005), and “To prepare for f uture science courses” (G8; Χ2 (2, N = 290) = 44.82, p < 0.0005). The least important goal at the end of the semester was “To understand how a chemistry research lab works” (G5; Χ2 (2, N = 290) = 58.42, p < 0.0005). Figures 1 and 2 present the student distribution for the three categories for all goals. See the Supporting Information for detailed results pertaining to the statistical data. Students’ selections for a least important goal are not well aligned with faculty goals for undergraduate chemistry laboratory based upon prior research.19−21 An interesting finding pertains to the goal “To develop my scientific writing skills” (G3), which the majority of the students believe is either an important or least important goal. Other researchers have highlighted the importance of students developing scientific communication skills and have created courses and laboratory activities to aid in developing these skills, yet the students do not prioritize this in a traditional general chemistry lab course.26−28 In previous work, faculty brought up safety in lab as a goal,21 and national organizations have ongoing efforts to promote a safety culture.13 It was found in this study that many students do not prioritize safety, as 20% of the participants selected the goal concerning safety as a least important goal.
difference (p value < 0.05) meant that the proportions were different and that a goal was placed in one category more frequently than in the other two categories. When there are significant differences, the post hoc analysis is used to determine which categories (most important, important, or least important) were significantly different. As an example, in order for a goal to be classified as most important for the population of this study, there must be a significant difference between the proportion of students who selected the goal as most important and the proportion of students who selected the goal as important and the proportion of students who selected the goal as least important. Defining these parameters was necessary to determine significant differences between categories whose frequency counts were close. Additionally, effect sizes were also calculated. These are reported as Cramer’s V25 and are available in the Supporting Information. To carry out the statistical test, the statements were assigned a code by combining the code in Table 1 and the category it was assigned to; proportions were calculated based on the frequencies obtained from the survey and the total number of participants who completed the survey. For the presurvey (n = 338), if 288 students selected “To earn an A or B” (G1) as a most important goal, then 50 students did not select it as most important. Table 4 illustrates this process.
Selection of Strategies
The survey question pertaining to strategies asked students to select their top five strategies used to achieve their most important goals. For our sample, students selected “Attempt to understand what I am doing” (S9), “Come to lab prepared” (S6), “Seek out help” (S2), “Manage my time eff iciently in lab” (S5), and “Read the lab before my laboratory period” (S10) as the top five strategies to achieve their most important goals. Participants chose the same top five achievement strategies for the postsurvey. Figure 3 compares the student distribution for each strategy at both instances. After performing a chisquare test for homogeneity, there was a statistically significant difference in proportions29 for four strategies between the preand post-tests (see Figure 3), two of which were in the top five selected strategies. For strategies “Not procrastinate” (S1; Χ2 (2, N = 628) = 5.40, p = 0.020) and “Read the lab before my laboratory period” (S10; Χ2 (2, N = 628) = 4.54, p = 0.033), the statistically significant differences in proportions indicate a significant decrease in number of students selecting these strategies, while for strategies “Manage my time ef f iciently” (S5; Χ2 (2, N = 628) = 16.71, p < 0.0005) and “Finish lab as quickly as I can” (S8; Χ2 (2, N = 628) = 20.93, p < 0.0005), it indicates a significant increase in number of students that chose these strategies. In light of these results, it is important to note that only a minority of students chose the strategy “Finish lab as quickly as I can” (S8) as one of their top five. Previous work15 has reported this as an affective goal; however our results indicate students do not consider it as a relevant strategy to achieve their most important goals. Schneider and Preckel established resource management strategies such as time management, help seeking, and peer learning are strongly related to achievement in higher education.10 Recalling students’ most important goal being “To earn an A or B” (G1), their selected top five strategies to achieve said goal certainly align with Schneider and Preckel’s claims.10
Table 4. Combined Code for Statistical Test Frequencies for selection of goal to category Goal
Category
G1
Most Important Important Least Important
G1 G1
Category Code
Combined code (used for statistical test)
Yes
No
2
12
288
50
1 0
11 10
42 8
296 330
To analyze the strategies, the number of times each strategy statement was chosen was determined for each administration of the survey. A 2 × 2 matrix was created using prepost and yes−no as column and row headers, and a chi-squared test of homogeneity was performed to test for statistical significance at the p < 0.05 level.
■
RESULTS AND DISCUSSION
Classification of Goals
Our research questions focused on the most important/least important goals as classified by the students. For the presurvey, statistically significant differences between the goal categories were identified for three out of 13 goals. Students selected “To earn an A or B” (G1; Χ2 (2, N = 338) = 621.21, p < 0.0005), “To prepare for the career I want to pursue” (G2; Χ2 (2, N = 338) = 203.87, p < 0.0005), and “To prepare for future science courses” (G8; Χ2 (2, N = 338) = 82.34, p < 0.0005) as most important goals. We were not able to determine students’ least important goal using the same statistical analysis as no significant difference was observed. For the postsurvey, statistically significant differences between the goal categories were identified for four out of 13 goals. Consistent with the presurvey, students’ most important goals were “To earn an A or B” (G1; Χ2 (2, N = 290) = 436.36, p < 0.0005), “To prepare for the career I want to pursue” (G2; Χ2 (2, N = 290) = 66.98, p
■
LIMITATIONS While the purpose of this study was to present the survey as a tool to measure goals and achievement strategies in a D
DOI: 10.1021/acs.jchemed.8b00984 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Article
Figure 1. Overview of number of students (%) classifying goals as most important, important, and least important for pre-survey.
Figure 2. Overview of number of students (%) classifying goals as most important, important, and least important for post-survey.
responses to open-ended questions, new interpretations or statements may arise as the survey is implemented in different types of courses and institutions. While previous publications discuss faculty goals for undergraduate laboratory coursework, a limitation of this study is not knowing the instructor’s goals for the specific courses being surveyed, as differences in laboratory curriculum might play a role in students’
chemistry laboratory, a limitation influencing the results rising from the survey is the participating institutions. More data collection would be necessary in order to gather information from students at other types of institutions (i.e., nonresearch intensive). Another limitation is how the students differentiate between most important and important goals. Even though all goal and strategy statements were created based on students’ E
DOI: 10.1021/acs.jchemed.8b00984 J. Chem. Educ. XXXX, XXX, XXX−XXX
Journal of Chemical Education
Article
suggestions include the incorporation of problem-based approaches in lab to develop students’ critical thinking skills,30 more guided inquiry experiments,31 or incorporating writing projects26−28 to aid with the development of scientific writing skills. This is particularly important for faculty and institutions that plan on using the survey as a foundation to collect evidence that supports changes in curriculum. Such changes may lead to a better alignment between faculty and students’ goals. The use of the survey presented in this study along with the survey presented by Bruck and Towns21 may provide more information on faculty and students’ goals alignment for a specific institution or course. Similarly, using this survey along with the Modified Approaches and Study Skill Inventory for Students (M-ASSIST)32 could provide more information on the type of strategies students employ to achieve their goals in the laboratory. Additionally, the results from the survey presented in this study may allow faculty to engage students in discussion about the types of strategies that lead to positive outcomes in the laboratory. For example, faculty can ask students, why is it important to read the laboratory prior to attending lab? What is the cost or downside of not doing so? What impact might it have that thwarts your own goals for laboratory? Given that many freshman students are attempting to develop effective study skills as they progress through their first and/or second semester, having a chemistry instructor discuss specific study skills in class may be highly effective.
Figure 3. Distribution of students per strategy for each instance. Significant differences (p value
