Best Practices for Supporting and Expanding ... - ACS Publications

Chapter 1

Downloaded by 80.82.77.83 on May 3, 2018 | https://pubs.acs.org Publication Date (Web): May 2, 2018 | doi: 10.1021/bk-2018-1275.ch001

The FUTURE Program: Engaging Underserved Populations through Early Research Experiences Amanda J. Reig,*,1 Kathryn A. Goddard,2 Rebecca E. Kohn,2,4 Leslie Jaworski,3 and David Lopatto3 1Department of Chemistry, Ursinus College, Collegeville, Pennsylvania 19460, United States 2Department of Biology, Ursinus College, Collegeville, Pennsylvania 19460, United States 3Department of Psychology, Grinnell College, Grinnell, Iowa 50112, United States 4Present Address: College of Arts & Sciences, Arcadia University, Glenside, Pennsylvania 19038, United States *E-mail: [email protected].

The FUTURE (Fellowships in the Ursinus Transition to the Undergraduate Research Experience) program at Ursinus College provides early research opportunities to members of underserved populations in STEM (science, technology, engineering, and mathematics). As part of the Parlee Center for Science and the Common Good at Ursinus College, the goal is to encourage and equip these students to become the scientific and civic leaders of the future. The program, established through a grant from the Howard Hughes Medical Institute, targets underserved students (underrepresented minorities, students with disabilities, women, and first-generation college students) prior to, or immediately following, their first year at Ursinus. These students, designated as FUTURE mentees, complete a four-week summer research experience mentored by an upper-class student (FUTURE mentor) and a faculty member. The FUTURE mentees also engage in a 1-credit course entitled “Science and Mathematics in Society” and take part in social programs designed to build community. Following the program,

© 2018 American Chemical Society Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


the FUTURE participants are expected to present their research findings at a regional or national conference and are strongly encouraged to continue research in subsequent semesters. From 2013 to 2017, 82 students (41 mentees and 41 mentors) and 22 faculty from a variety of STEM disciplines participated in the program. FUTURE mentees show greater persistence as STEM majors than their underserved peers (89.7% compared to 64.5% by junior year and 91.7% compared to 65.1% at graduation) and nearly all (90.2%) continue their involvement in undergraduate research. In retrospective surveys completed immediately after the program, both mentees and mentors reported gains in all categories assessed, including laboratory skills, understanding of their discipline, and confidence in explaining their projects. Learning gains were also assessed using the SURE Follow-Up survey, which showed the FUTURE participants had gains above national means in nearly all categories measured. The success of the FUTURE program demonstrates how early access to a short four-week, research intensive program can improve outcomes for underrepresented students and provides a simple model by which this can be achieved.

Introduction In 2012, Ursinus College established the Parlee Center for Science and the Common Good with support from a Howard Hughes Medical Institute (HHMI) Undergraduate Science Education Program grant. The objectives of the Parlee Center are to (1) provide opportunities for students to think, speak, and write about the impact of science on the common good, and (2) cultivate the next generation of scientific leaders with a focus on students from underserved populations (USPs). Underrepresented minorities (URMs), women in chemistry, computer science, mathematics, or physics, first-generation college students, and students with disabilities were included as USPs. The FUTURE (Fellowships in the Ursinus Transition to the Undergraduate Research Experience) program was created specifically to address the second objective above, and was designed based on studies which show that early involvement in undergraduate science research is an effective way to improve retention and outcomes for students from underserved populations (1, 2). For example, an extensive quantitative study by Jones et al. found that “participating in science research during the first two years or for more than three terms is associated with about a 240% increase in a student’s odds of graduation in biology with a [competitive] GPA (1).” In addition, frequent, positive contact with faculty and advanced peers in a research setting was shown to increase the academic and social integration of students from USPs and, as seen above (1), improved their retention in science (2). These results echo numerous studies which have shown that social integration is vital in the persistence and retention of minority students (3–5). By providing research 4 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

experiences to students from USPs very early in their undergraduate careers, the FUTURE program builds on the robust culture of undergraduate research at Ursinus (nearly 70% of STEM majors collaborate with faculty on original research) and leverages the close-knit scientific community that exists on our small, residential campus to encourage and equip them to become the scientific leaders of the future.


Program Overview The FUTURE program is a four-week, residential, paid opportunity for students from USPs during the summer prior to or immediately following their first year at Ursinus. FUTURE mentees receive a stipend of $1250 and free campus housing for the duration of the program. From 2012 to 2017, eight students were selected to be FUTURE mentees each summer, distributed evenly across students who had just completed their first year and those who would matriculate in the fall. The FUTURE program specifically targets USP students at the beginning of their undergraduate careers with the goals of improving their persistence as STEM majors, increasing their participation in undergraduate research, and expanding the number who enter STEM careers. Many summer undergraduate research programs, including Ursinus’ own Summer Fellows program, require students to have completed their sophomore year of college. Unfortunately, many USP students have already left STEM before reaching that point. USP students with interests in the following STEM disciplines and majors at Ursinus were invited to apply: biochemistry and molecular biology (BCMB), biology, chemistry, computer science, health and exercise physiology, mathematics, neuroscience, physics, and psychology. The FUTURE program is comprised of three main components: (1) a facultyand student-mentored research experience, (2) enrollment in the 1-credit course “Science and Mathematics in Society”, and (3) social programming to connect students with the campus community. The FUTURE program is overseen by a small committee that includes STEM faculty and the Assistant Director of Residence Life who oversees first-year students. The committee is chaired by a faculty program coordinator who receives a stipend or a course release.

Participant Selection Each year, applications for FUTURE faculty/mentor teams are solicited at the end of the fall semester. Faculty are asked to certify that that will be available for the duration of the program and will participate in the program activities (teach in the course, attend discussions, and contribute to assessment), identify a current research student whom they believe would make a good FUTURE mentor, and provide a brief research description that would be understandable to a student completing high school. FUTURE mentors are asked to certify their availability and participation in program activities, describe their prior research experience, reflect on their desire and ability to serve as a mentor, and discuss their career 5 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


goals. Selections are made based on academic performance, experience with the research project, and potential for excellence in mentoring. Students from USPs are strongly encouraged to apply to be FUTURE mentors, and special consideration is given to individuals who participated as FUTURE mentees previously. The FUTURE mentees are selected in two rounds. Current first-year students are invited to apply early in the spring semester after nomination by a faculty member. These students are asked to certify their availability for the duration of the program and their willingness to participate in the program activities, describe any prior research experiences (clearly noting that no prior experience is required), discuss their strengths and weaknesses when working on group projects, explain why they want to participate in the program and how their participation would benefit their career goals, and have a letter of recommendation submitted by a faculty member on their behalf. Incoming students who express an interest in one or more of the program fields and qualify as a member of an USP are identified with assistance from the Admissions Office and sent invitations to apply late in the spring semester. These students complete an application similar to that for the first-year students, and top candidates are given a phone interview prior to final selections. We have typically accepted students with moderate to strong backgrounds in STEM who have not yet participated in an independent research experience, reasoning that students with the potential to succeed but who lack experience would benefit the most from early integration into our research laboratories. As part of the application process, students are asked to rank the available projects based on the descriptions provided by the participating faculty mentors. The FUTURE mentees are assigned to a project based on their preference, intended major, and career goals. Mentored Research Experience Each FUTURE mentee is matched with a research team consisting of a faculty member and a FUTURE mentor. The FUTURE mentor is a rising junior or senior with at least one semester of prior research experience with the faculty member. The FUTURE mentors are full participants in our paid, residential Summer Fellows research program, in which they collaborate with faculty on research projects. The FUTURE program is timed to overlap with weeks 4–7 of the eight week Summer Fellows program. This allows the FUTURE mentor three weeks to work on the project before the FUTURE mentee arrives, giving them time to prepare to integrate their mentee into the research experience. The FUTURE mentors then have one final week to focus on preparing a required research paper and oral presentation without mentoring obligations. The FUTURE mentor is responsible for working with the FUTURE mentee daily, assisting them in understanding the project, conducting experiments, and analyzing data. Faculty mentors have responsibility for designing and/or overseeing the research project, interacting with the FUTURE mentor and mentee on a frequent basis, providing guidance and feedback on research progress, creating and maintaining a collegial and supportive environment for the students, and assisting them with the production of their final presentations and papers. 6 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


During the program, the FUTURE mentees spend approximately 35 hours per week in the research lab and participate fully in all aspects of the research project, including reading journal articles, designing and carrying out experiments, and analyzing and interpreting data. On the last day of the program, a mini symposium is held in which each FUTURE mentee gives a 10-minute oral presentation summarizing the goals and results of their project to an audience of students, faculty, friends, and family. The FUTURE mentees also prepare a written report, typically 5-8 pages in length, which must be submitted by the end of the fall semester. These two aspects of the program provide the mentees with early practice in oral and written scientific communication, building skills that they will use frequently in their future coursework and research experiences.

Science and Mathematics in Society Course In parallel with conducting research, FUTURE mentees take a one-credit pass/fail course called “Science and Mathematics in Society.” The course is designed to introduce them to the academic experience of Ursinus and equip them with tools they will need to be successful researchers. The course meets for approximately one hour four times per week (M, Tu, Th, and F). No class is held on the final Thursday of the program to give students additional time to prepare for their presentations, which take place on the last Friday morning. The course covers a wide range of topics, including laboratory safety, how to read scientific journal articles, how to analyze and interpret data, and how to give a scientific presentation, ethical dilemmas in research, diversity and bias in STEM, and communicating science to the public. One of the four meetings each week is set aside for large group discussions (including all FUTURE mentees, mentors, and faculty) over lunch. These large group discussions are centered on a book or excerpts from a book that everyone reads in preparation for the meeting, and model for the FUTURE mentees the type of academic experience they can expect in our first-year seminar program known as the Common Intellectual Experience (CIE). Texts used for this component of the course have included Thinking, Fast and Slow by Daniel Kahneman (6), The Demon Haunted World by Carl Sagan (7), How I Killed Pluto and Why It Had It Coming by Mike Brown (8), and Genome: The Autobiography of a Species in 23 Chapters by Matt Ridley (9). Books are provided for all FUTURE participants, including faculty, and are distributed several weeks before the program begins. As part of the course, the mentees have the opportunity to spend half a day visiting a nearby academic or industrial research facility. Trips have included tours of laboratories at the Perelman School of Medicine at the University of Pennsylvania in Philadelphia, PA, the Lewis Katz School of Medicine at Temple University in Philadelphia, PA, and GlaxoSmithKline in Collegeville, PA. The course is co-taught by all faculty participants in the FUTURE program, with each faculty mentor covering 1-2 class days. Each faculty mentor is asked to assign the mentees either a pre- or post-class assignment (or both), which encourages the students to continue their academic engagement in the evenings and on the weekends. Due to the diversity of disciplines represented in the 7 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

program, faculty are requested to use sources and examples from a range of fields in their lessons. Faculty volunteer for or are assigned to specific topics based on their interests and availability, and are encouraged to include their FUTURE mentor in the class activities.


Social Programming and Residential Life Mentees are required to live on campus for the entire four weeks of the FUTURE program. To ensure that they are integrated into the campus community and are well-supported in their residential experience, the program has a dedicated, paid FUTURE Coordinator of Student Programs. The FUTURE Coordinator is a senior student majoring in one of the fields covered by the program who also has previous experience as a Resident Assistant on campus. The role of the FUTURE Coordinator is to provide assistance and emotional support to students, enforce campus and program policies, facilitate conflict resolution as needed, and coordinate the social programming for the FUTURE mentees. Evening and weekend activities are designed to build community amongst the FUTURE mentees and with other campus residents, familiarize students with the campus and surrounding area, and have fun. The FUTURE Coordinator has a $500 budget with which to fund programming, including transportation costs. Activities include scavenger hunts, game nights, movie nights, a beach trip, a day trip to Philadelphia, and other activities aligned with student interests. FUTURE mentors are invited and encouraged to join the social activities. The FUTURE students have consistently reported that a highlight of the program are weekly dinners for the FUTURE mentees, mentors, and faculty and their families hosted at a faculty mentor’s home.

Mentor Training Based on feedback following the first year of the program, a structured training program was instituted for the FUTURE mentors. The mentors participate in four 1-hour discussions on mentoring that occur weekly beginning one week prior to the arrival of the FUTURE mentees. Topics include what it means to be a mentor, strategies for working with students from diverse backgrounds, overcoming challenges in mentoring their FUTURE mentees, and reflecting on their role as a mentor. Resources used in these discussions include “Adviser, Teacher, Role Model, Friend: On Being a Mentor to Students in Science and Engineering” from the National Academies Press (10), “Mentoring across Cultures” by Betty Neal Crutcher (11), and Nature’s “Guide for mentors (12).” Each discussion is facilitated by one of the faculty participants. Through these meetings, the FUTURE mentors gain confidence in their mentoring skills and are encouraged to be a role model of excellence and balance in their academic and social lives. 8 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Post-Summer Activities and Opportunities


Although FUTURE is a nominally a four-week summer research program, support and mentoring for the FUTURE mentees do not end when the summer is over. All FUTURE mentees are encouraged to continue with undergraduate research, whether in partnership with the same faculty mentor or on a new project. In addition, there is an expectation that the FUTURE mentee and mentor will jointly present the results of their work at a regional or national conference within 1-2 years. Funds are made available to offset student travel costs to facilitate this goal. The FUTURE mentees are also urged to assume leadership roles as FUTURE mentors or in other capacities on campus in subsequent years.

Program Sustainability In its initial format, the FUTURE program was fully-funded by HHMI at an operating budget of approximately $115,000 per year for 8 teams (faculty, mentor, and mentee). This included all stipends ($2500 per student mentor, $2500 per faculty member, and $1250 per mentee), housing costs for each mentor ($1080 for eight weeks) and mentee ($540 for four weeks), research supply costs per mentor/mentee team ($1500 over the summer and an additional $2000 if the mentee continued research the following academic year), travel funds to present work at an external conference ($2400), a stipend and housing costs for the undergraduate FUTURE coordinator ($4600), and funds for social programming and books for the 1-credit course ($1000). Through endowed funds raised to support the Parlee Center and its associated programs in perpetuity, the FUTURE program at Ursinus will be sustained at a level of approximately $10,000 per team ($6250 in stipends for faculty, mentor, and mentee; $1600 in housing costs; $1500 in supply and travel funds; and $800 towards the FUTURE coordinator and social programming costs). For institutions that already have paid summer research programming options in place, the costs to provide a FUTURE-like program can be relatively minimal. The additional required expenses would be mentee stipends (half a typical student summer stipend; $1250 at Ursinus) and housing ($540 for four weeks at Ursinus), FUTURE coordinator stipend ($3000) and housing (5 weeks for $675), and social programming costs ($500-$1000). A minimum of six mentees is recommended to ensure a critical mass for community building. To incentivize faculty participation, faculty stipends, if not already provided through current summer research funding, and supply and/or travel funds are recommended. However, the specific amounts can be easily adjusted to accommodate available resources. We also found faculty were self-motivated to participate for the opportunity to train research students early for long-term collaboration and to contribute to current diversity initiatives on campus.

9 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


Participant Data Table 1 provides a summary of demographic data of the FUTURE participants from 2013 to 2017. A total of 82 Ursinus students took part in the program, 41 as FUTURE mentees and 41 as FUTURE mentors. This corresponds to eight pairs of students per year except for 2016 when nine pairs of students participated. Note that several participants qualified under more than one USP category. The majority of the FUTURE mentees were URM or first-generation college students (or both). The population of FUTURE mentors, selected from junior and senior STEM majors, is less diverse but is consistent with overall campus diversity, which averaged 13.1% URM and 25.9% first-generation students for 2015-2017. It is also important to note that the percentage of USPs serving as FUTURE mentors has gone up significantly over the five years that the program has run (0%, 12.5%, 12.5%, 50%, 37.5%), due in large part to five former FUTURE mentees having now served as FUTURE mentors between 2015 and 2017. The high percentage of female FUTURE mentees is reflective of the FUTURE applicant pool, which was 79% female.

Table 1. Demographics of FUTURE Participants from 2013 to 2017 FUTURE Mentees (n = 41)

Demographic

9

22%

13

32%

Female

32

78%

28

68%

Underrepresented Minority

22

54%

5

12%

First-Generation College Student

23

56%

8

20%

Student with Disabilities

4

10%

1

2%

Female in Chemistry, Computer Science, Math, or Physics

4

10%

2

5%

41

100%

14

34%

Male

Total Underserved Populationa a

FUTURE Mentors (n = 41)

Participants who meet more than one demographic criterion are included in each count.

Program Assessment A number of strategies were used to assess the FUTURE program in order to improve the participant experience, measure learning gains for the FUTURE mentees and mentors, and track outcomes for all participants. Short-term perceptions were evaluated through focus groups and anonymous pre- and post-program surveys administered within the first three and last three days of the research experience, respectively, for FUTURE mentees and mentors. Written 10 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


and oral responses on the strengths and weakness of the program were discussed with the FUTURE faculty and reviewed by an external assessment consultant on an annual basis. These discussions informed changes made to the program and led to several improvements each year. Approximately one year after completing the FUTURE program, all FUTURE participants were asked to complete the SURE (Survey of Undergraduate Research Experiences) Follow-Up survey. This retrospective survey uses approximately 47 questions to measure the student’s growth as a scientist, their understanding of how science is done, and whether the experience has affected their career choice or the way that they approach their academic coursework (13, 14). Finally, the achievements, research coursework, and graduation and employment outcomes for all participants in the FUTURE program were tracked and compiled on an annual basis. All surveys were vetted by the external assessment consultant, approved for use by the Ursinus College Institutional Review Board (IRB) and administered using Qualtrics (via Ursinus or Grinnell College) according to the requirements established by the US Department of Health & Human Services.

Pre- and Post-Program Survey Results Pre- and post-program surveys were created based on the public SALG (Student Assessment of their Learning Gains; https://www.salgsite.org/) instrument “HHMI Undergraduate Research, Summer 2011” created by Gudrun Willett. In a pre-program survey, the FUTURE mentees were asked to comment on their general expectations for their summer research experience, describe what they expected to learn through the program, and to use a Lichert scale to rank their abilities on over 20 different categories related to scientific research. Questions included “I can figure out the next step in a research project”, “I understand how my upcoming research will contribute to the accumulated knowledge in my field”, “I am confident in my ability to contribute to science”, “I am confident in my ability to do well in future science courses”, “I am confident that I know what everyday research is like”, “I know how to read a primary research article”, and “I am able to work independently in a research lab.” The post-program survey then asked the mentees to self-reflect on their abilities both before and after the FUTURE program on the same 27 categories. Mentees were also asked to comment on their mentoring support, aspects of the 1-credit course, and components of the program to keep or remove. The survey results were overwhelmingly positive, with mentees reporting significant post-program gains in most categories assessed. A few areas of dissatisfaction for the mentees were consistently noted each year, including too much or too little social programming, a lack of opportunities to interact with the rest of the student research community, and restrictions placed on the mentees (e.g., a dorm curfew and limited off-campus travel). Our efforts to optimize the level of programming continue, but satisfaction with programming may reflect differences in participant preferences from year to year. While unpopular, the modest residential restrictions on the mentees were deemed necessary to ensure the well-being of the pre-matriculation students. 11 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


The mentees uniformly gave high praise for the trips to local research facilities, commenting that “the field trip to UPenn was awesome; we were able to learn about a life as a graduate student who conducts research and able to observe their working environment and what they actual do on daily bases” and “the field trips to GSK and Temple were very interesting and helpful to get to network with other people in this field and outside Ursinus.” In addition, the mentoring relationships fostered by the program proved beneficial both in and out of the research lab. One mentee noted “even though I learned so much in research, the most important thing to me the people I’ve met through this experience. Many upperclassmen offered to help me with the classes I’ve been taking, or offered their opinions as to what classes I could take to get where I want to go in life”, while another commented that “I gained relationships with professors as well as lifelong friendships with future scientist[s] in this program. I also gained a more confident outlook on my future success in science and the many options I have at Ursinus College.” The program also changed mentees’ minds about their careers in STEM. One wrote that the summer program “completely opened my eyes to what science entails and completely changed the direction of my career path. I fell in love with changing the world through scientific discoveries and critically thinking of future experiments.” Other mentees commented that “[i]t made me realize that I want to do research as a career” and “I am much more likely to enroll in a graduate program.” In another comment, a mentee showed how the connections s/he made through this experience may not have happened without the FUTURE program, writing “I am so glad I got the chance to research so early in my college career; I absolutely loved being in the lab, and I think I would have been too afraid or hesitant to ask a professor if I could join his/her lab freshman year.” The FUTURE mentors completed similar pre- and post-program surveys and also showed gains (although smaller in magnitude) in all categories assessed. The smaller gains are to be expected based on the previous academic and research experiences of the mentors. When asked what skills they gained by participating in the program, the mentors consistently reported that they gained patience, organization, clearer communication skills, leadership, and a better understanding of their own research project. One mentor illustrated the benefits of mentoring a new student this way: “I loved having the opportunity to teach someone else how and why we were doing experiments. It forced me to really understand what and why I was doing it to be able to effectively communicate it.” Other mentors remarked on how their mentoring experience would benefit them in the future, saying: “the biggest skill I gained from the experience was being able to help others (my mentee) understand research and the logic behind what we were doing. Next year I have to TA as part of my PhD program and this has been a positive influence that will help me”, and “[b]eing given the opportunity to mentor definitely allowed me to grow as a scientist. I am much more confident in my abilities to explain lab procedures and rationales, and also more confident in my ability to foster a positive lab environment. I would like to run a lab in the future, so these qualities were definitely important ones to gain.” As previously mentioned, responses from the mentor surveys in the program’s first year (2013) indicated that mentorship training was desired. The mentor 12 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


training described in the program overview was established the following summer and has been routinely praised by the FUTURE mentors since that time. The workshops have allowed the mentors to see that “all the other mentors were going through similar things as I was, even though we had different majors” and “being able to talk about these skills made me reflect upon what I was doing well, and what things I could incorporate in to the act of mentoring.” The greatest challenge in the program, as reflected by mentee, mentor, and faculty responses, has been balancing the research, academic, and social components of the program. Over the five summers that the program has run, we have adjusted the course meeting times to maximize the hours mentees and mentors can spend on their research by scheduling them first thing in the morning or over the lunch hour. We have also limited the classwork in the final week of the program to ensure the FUTURE mentees have ample time to complete their project and prepare for their oral presentations on the last morning of the program. Each summer also presents a unique challenge when it comes to the social programming organized by the FUTURE Coordinator. Each group of FUTURE mentees has different interests and it is important to find a balance of enough activities to keep them engaged and build community, but not have them be overscheduled.

SURE Follow-Up Survey Results SURE Follow-Up survey results are available for 30 of the 33 FUTURE mentees and 26 of the 33 FUTURE mentors who participated in the program between 2013 and 2016 (91% and 79% response rates, respectively). Data for the 2017 cohort will not be available until Fall 2018 as the survey is administered approximately one year after completion of the FUTURE program. The survey is administered at this time point because it has been shown that with time student opinions about their experiences sharpen, their perceptions are influenced by dissemination activities related to the research project, and they report behavior changes in the direction of greater independence and motivation (15). In the first part of the survey, students are asked to self-report on their postgraduate plans. Participants overwhelming responded that they “had a plan for postgraduate education before - plan has not changed” (41% of mentees, 46% of mentors) or that they “considered postgraduate education - research experience confirmed decision” (34% of mentees, 31% of mentors). Four mentees and one mentor reported that the research experience changed their mind to either pursue or consider pursuing postgraduate education. When asked to evaluate their summer learning experience, 77% of the FUTURE mentees and 58% of the FUTURE mentors reported that “Summer lab was fantastic—I learned a lot.” An additional 17% of FUTURE mentees and 33% of FUTURE mentors indicated that “Summer lab is the way to learn about science.” Nearly all of the mentees and mentors reported that they would take future courses in the same department as the research experience (93% and 96%, respectively, compared to 63% nationally) and that the research experience was likely to influence their behavior in those courses (96% and 88%, respectively, compared to 82% nationally). 13 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


The second part of the survey asks students to report their learning gains across 21 categories, where 1 is no or very small gain and 5 is a very large gain. The results for the FUTURE program are shown in Figure 1, with means and 95% confidence intervals reported for the FUTURE mentees (n ≤ 30) and mentors (n ≤ 26) compared to the national data set (n ≤ 1300). The FUTURE mentees report substantial learning gains in all categories assessed by the SURE Follow-Up survey. The greatest reported gains by the FUTURE mentees were in tolerance for obstacles faced in research, ability to analyze data and other information, learning laboratory techniques, and giving effective oral presentations. Even given the relatively large confidence intervals due to the small sample size, the FUTURE mentees registered learning gains higher than the national results in ability to analyze data, understand primary literature, learning ethical conduct, skills in oral presentations and scientific writing, and self-confidence. These results suggest the lessons taught in the 1-credit “Science and Mathematics in Society” course, which include scientific ethics, reading primary literature, science writing, and oral presentations, along with the oral and written requirements for the program, are effective at increasing student learning in these areas. The FUTURE mentors reported mean gains at or above the national Follow-Up survey results in most categories. Like the FUTURE mentees, the mentors report the greatest absolute gains in tolerance for obstacles faced in research and skill in giving effective oral presentations. They also report high gains in readiness for more demanding research, understanding how scientists work on real world problems, skill in science writing, and learning to work independently. The largest gains for the FUTURE mentors compared to the national means were in their potential for science teaching and skills in science writing and oral presentation. We find the large gain in teaching potential particularly exciting as it certifies the value in their experience mentoring their FUTURE mentee. The FUTURE mentors did not score themselves as high in understanding how knowledge is constructed, self-confidence, and understanding how scientists think. One explanation may be that as rising juniors and seniors, the FUTURE mentors often have significant previous academic and research experience and thus may feel strong in these areas prior to their participation in the FUTURE program.



Figure 1. SURE III Follow-Up survey results for FUTURE mentees (closed circles, n ≤ 30) and FUTURE mentors (closed diamonds, n ≤ 26) from 2013 to 2016. These data are compared to the national means for the Follow-Up survey (open squares, n ≤ 1300) collected from 2009 to 2013. Error bars represent the 95% confidence intervals. FUTURE mentees show mean gains greater than FUTURE mentors in nearly all categories, and both populations report mean gains equal to or greater than the national means in nearly all categories.


Outcomes


Participant outcomes, including persistence as STEM majors, post-program research experiences, grade point averages, and post-graduation destinations have been tracked for all FUTURE mentees and are summarized in Tables 2 and 3. In all cases, comparison data is provided for STEM majors (defined as students majoring in a field supported by the FUTURE program, excluding FUTURE mentees) and USP STEM majors at Ursinus. The results are very positive, with the FUTURE mentees recording better outcomes than the STEM majors, and specifically the USP STEM majors, at Ursinus on all metrics.

Table 2. Outcomes for FUTURE Mentees Compared to Ursinus STEM Majors and USP STEM Majors Outcome

FUTURE Mentees

Ursinus STEM Majors All

USP

Persisted as STEM majora

89.7% (26/29)

65.5% (658/1004)

64.5% (222/344)

Four-year graduation rate in STEMb

91.7% (11/12)

60.9% (312/499)

65.1% (105/175)

Engaged in one or more semesters of research

90.2%c (37/41)

67.9%d (248/365)

71.4%d (90/126)

3.43 ± 0.30 (n = 10)

3.30 ± 0.40 (n = 365)

3.25 ± 0.39 (n = 126)

Mean GPA at graduatione

Percentage of 2012–2015 cohorts who intended to major in STEM upon matriculation and had a declared STEM major by fall of their junior year b Percentage of 2012–2013 cohorts who intended to major in STEM upon matriculation and graduated with a STEM degree in four years c Percentage of FUTURE mentees (2012–2017 cohorts) who enrolled in at least one semester of research following the FUTURE program d Percentage of STEM majors (2012–2013 cohorts) who enrolled in at least one semester of research by the fall term of their fourth year e Includes students from 2012 and 2013 cohorts; reported values are means ± standard deviations a

Persistence in STEM is a significant concern for students from USPs and a major goal of the FUTURE program. We measured persistence as a STEM major by comparing the number of students who intended to major in a STEM field at matriculation to the number of those students who had declared a STEM major by fall of their junior year. For the 2012–2015 cohorts, these values averaged 65.5% for all STEM majors, and 64.5% for STEM majors from USPs. The results are particularly poor for URM STEM majors, at 48.8%. In contrast, FUTURE mentees from these same cohorts had a persistence rate of 89.7% (26/29). Of the three students who did not continue in STEM, two withdrew from the college and one is now pursuing a non-STEM major. The four-year graduation rate in STEM was determined by comparing the number of students who intended to major in a STEM field at matriculation to the number of those students who graduated 16 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.


with a degree in STEM by the end of their fourth year. Twelve students who participated in FUTURE in either 2013 (the first year of the program) or 2014 have reached their graduation year. Of these students, 11 graduated with a STEM degree (91.7%) and one student withdrew from the college. This four-year STEM graduation rate is very favorable compared to the campus numbers for STEM and USP STEM students in the same cohorts (Table 2). The high persistence and four-year STEM graduation rates for the FUTURE mentees support the assertion that early engagement in an authentic research experience can lead to better academic and social integration in college for students from underserved populations. Early intervention is key since by junior year approximately 35% of students who matriculated intending to major in STEM have either changed to other disciplines (16%) or are no longer enrolled at Ursinus (19%). The FUTURE program is also very successful at encouraging students to continue to engage in undergraduate research during subsequent semesters. While the rates of undergraduate research are high at Ursinus (averaging 67.9% for all STEM majors, and 71.4% for USP STEM majors for the 2012 and 2013 cohorts), connecting the FUTURE mentees with the research culture at Ursinus very early leads nearly all of them (37/41, or 90.2%) to enroll in subsequent research courses. As part of this continued research experience, 64% of the FUTURE mentees from 2013 to 2016 have had the opportunity to present their work at regional or national conferences in their field at least once. Five FUTURE mentees were later selected to serve as FUTURE mentors, demonstrating their growth as researchers and as leaders in STEM. As noted previously, this has also increased the percentage of USPs serving as FUTURE mentors. The mentored research experience appears to provide a supportive learning community that has a positive effect on the academic performance of the FUTURE mentees. The mean grade point average (GPA) at graduation for FUTURE mentees in the 2012 and 2013 cohorts was 3.43 ± 0.30 (n = 10). This compares favorably to all STEM majors (3.30 ± 0.40, n = 365) and USP STEM majors (3.25 ± 0.39, n = 126) for these same cohorts. The close student-faculty connections built during the program and the knowledge gained from the 1-credit course likely also contribute to the better academic performance by the FUTURE students. Graduates who participated in the FUTURE program as either a mentee or mentor have gone on to a wide range of career paths (Table 3). A majority of the FUTURE mentees are either employed in their field (36.4%) or are currently pursuing post-graduate studies in STEM or medicine (45.5%). The overwhelming majority of FUTURE mentors are also either employed in their field (32.3%) or are pursuing post-graduate studies in STEM or medicine (51.7%). The rates of post-graduate study for FUTURE participants is noteworthy as only 27.8% of all Ursinus STEM graduates from 2014 to 2016 were pursuing post-graduate studies in STEM or medicine. While data from additional cohorts will be needed to confirm these results, they suggest that the early exposure to research experiences may be influencing the career paths of the FUTURE participants.


Table 3. Post-Graduation Destinations for FUTURE Mentees and Mentors FUTURE Menteesa (n = 11)

FUTURE Mentorsb (n = 31)

Ursinus STEM Majorsc (n = 610)

Employed in STEM

36.4%

32.3%

40.5%

Post-Graduate Studiesd

45.5%

51.7%

27.8%

(Graduate School)

(27.3%)

(32.3%)

(19.0%)

(Medical School)

(9.1%)

(12.9%)

(7.6%)

(Post-Baccalaureate Program)

(9.1%)

(6.5%)

(1.2%)

9.1%

0

2.6%

0

9.7%

11.8%

9.1%

3.2%

4.7%

Other

0

0

6.4%

Unknown

0

3.2%

6.2%

Post-Graduate Destination


Service/Volunteer Employed not in STEM Seeking employment

Includes FUTURE mentees who graduated in 2015 or 2016 Includes FUTURE mentors who graduated in 2014, 2015, or 2016 c Includes Ursinus STEM majors who graduated in 2014, 2015, or 2016 d Percentages are the sum of graduate school, medical school, and post-baccalaureate program values a

b

Impact of the FUTURE Program on Faculty Participants From 2013 to 2017, 22 individual faculty participated in the FUTURE program. Of those 22 faculty, 11 have served as a faculty mentor more than one time. Faculty were incentivized to participate in the program through a stipend ($2500) and funds to cover research supplies during the summer ($1500) and over the academic year ($2000) should the FUTURE mentee and mentor continue their work. Survey responses from faculty participants indicated that the program had positive impacts on their research programs and mentoring skills. Several faculty participants noted that mentees who were brought in to the research environment in their first year and continued to work on the project in subsequent years were highly productive and made significant contributions to their scholarship. One faculty participant said “having a student start training on clinical skills so young, allowed me the opportunity to train him extensively and help him with his research projects. He was able to complete several smaller pilot projects over the years which all built up to his final honors project. Each of his pilot projects has provided me valuable pilot data that I will use when on my Pre-Tenure leave this fall writing my grant proposal.” For other faculty, the program changed their perception about working with at-risk students: “FUTURE showed me how immensely valuable early immersion in meaningful research experiences can be for underrepresented students. Because of this program, I feel very confident inviting at-risk students into my research lab as early as the summer prior to their sophomore year. I plan to continue this practice for the remainder of my career.” 18 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

The mentoring discussions also proved beneficial for faculty in addition to the students: “In the two summers that I participated I mentored the mentors—that helped my own growth as a mentor. Doing the readings and participating in the conversations with the students about what makes a good mentor was useful.”

Table 4. Scholarly Presentations and Peer-Reviewed Publications Including FUTURE Mentees and Mentors


Scholarly Work

Number

Regional/national presentation including FUTURE mentor coauthor

35

Regional/national presentation including FUTURE mentee coauthor

27

Peer-reviewed publication including FUTURE mentor coauthor

6

Peer-reviewed publication including FUTURE mentee coauthor

3

It is important to note that participation in the FUTURE program did not hinder research productivity by the faculty participants, and in several cases enhanced faculty scholarship. The timing of the four-week program allows the FUTURE mentor and faculty to make significant progress on their project while bringing in a new student during highly productive research weeks. Table 4 summarizes the number of scholarly presentations and peer-reviewed publications to date that include FUTURE mentee and/or mentor coauthors. Publications appeared in Letters in Biomathematics, NeuroReport, Perception, Proceedings of the International Symposium on Biomathematics and Ecology, International Journal of Vascular Medicine, and Journal of Occupational and Environmental Medicine. Conference presentations have included the National Meeting of the American Chemical Society, the Society for Neuroscience Annual Meeting, the American Society for Cell Biology Annual Meeting, the Annual Meeting of the Society for Experimental Biology, the Annual Joint Mathematics Meetings, the Entomological Society of America Annual Meeting, and the Annual Meeting of the Biophysical Society, among others. These results highlight the productivity that can be gained by creating long-lasting student-faculty partnerships in research like those fostered by the FUTURE program.

Conclusions The FUTURE program is a successful example of how undergraduate research experiences can be used to effectively engage, retain, and support students in STEM from underserved populations. By connecting these students with an authentic research experience very early in their undergraduate careers, the mentees gain knowledge, confidence, and practical skills that help them to navigate the challenges of college. A dedicated mentoring team and a deliberately-balanced combination of academic, research, and social activities to create a well-rounded experience appear to be vital to the program’s success. 19 Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

While a four-week program is shorter than many undergraduate research experiences, the reported outcomes suggest that this is a sufficient amount of time to build community, fully engage with a research project, and gain academic skills that help the mentees to be successful in college. Importantly, all the FUTURE participants, including the faculty and student mentors, benefited from their involvement. As many colleges and universities currently run summer research programs, the addition of a FUTURE-like module would be a relatively easy and inexpensive way to recruit and improve outcomes for STEM students from USPs.


Acknowledgments The authors would like to thank past and present members of the FUTURE program committee (Rebecca Lyczak, Reese McKnight, Simara Price, Jordan Toy, and Charlene Wysocki) for the time and effort they devoted to creating, improving, and sustaining the FUTURE program. We also extend our gratitude to Robert Dawley and Akshaye Dhawan for their work as members of the Parlee Center steering committee, Kathy Pusecker (Director of the University of Delaware Center for Teaching and Learning Assessment) for her guidance and assistance with program assessment, and Annemarie Bartlett and Whitney Hawkins for providing institutional data. Finally, we greatly appreciate the Howard Hughes Medical Institute and Donald and Joan Parlee for their past and future financial support of the Parlee Center and the FUTURE program at Ursinus College.

References 1.

2.

3.

4.

5.

6. 7.

Jones, M. T.; Barlow, A. E. L.; Villarejo, M. Importance of undergraduate research for minority persistence and achievement in biology. J. Higher Educ. 2010, 81, 82–115. Gregerman, S. R.; Lerner, J. S.; von Hippel, W.; Jonides, J.; Nagda, B. A. Undergraduate student-faculty research partnerships affect student retention. Rev. Higher Educ. 1998, 22, 55–72. Gummadam, P.; Pittman, L. D.; Ioffe, M. School belonging, ethnic identity, and psychological adjustment among ethnic minority college students. J. Exp. Educ. 2016, 84, 289–306. Hausmann, L. R. M.; Schofield, J. W.; Woods, R. L. Sense of belonging as a predictor of intentions to persist among African American and white firstyear college students. Res. Higher Educ. 2007, 48, 803–839. Pittman, L. D.; Richmond, A. University belonging, friendship quality, and psychological adjustment during the transition to college. J. Exp. Educ. 2008, 76, 343–362. Kahneman, D. Thinking, fast and slow; Farrar, Straus and Giroux: New York, 2013. Sagan, C.; Druyan, A. The demon-haunted world: Science as a candle in the dark; Ballantine Books: New York, 1997. 20

Gourley and Jones; Best Practices for Supporting and Expanding Undergraduate Research in Chemistry ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

8. 9. 10.

11. 12.


13. 14. 15.

Brown, M. How I killed Pluto and why it had it coming; Spiegel & Grau: New York, 2012. Ridley, M. Genome: The autobiography of a species in 23 chapters; Harper Perennial: New York, 2006. Institute of Medicine, National Academy of Sciences, and National Academy of Engineering. Adviser, Teacher, Role Model, Friend: On Being a Mentor to Students in Science and Engineering; The National Academies Press: Washington, DC, 1997. Crutcher, B. N. Mentoring across cultures. Academe 2007, 93, 44–48. Lee, A.; Dennis, C.; Campbell, P. Nature’s guide for mentors. Nature 2007, 447, 791–797. Lopatto, D. Survey of undergraduate research experiences (SURE): First findings. Cell Biol. Educ. 2004, 3, 270–277. Lopatto, D. Undergraduate research experiences support science career decisions and active learning. CBE-Life Sci. Educ. 2007, 6, 297–306. Lopatto, D. Exploring the benefits of undergraduate research: The SURE survey. In Creating effective undergraduate research programs in science; Taraban, R., Blanton, R. L., Eds.; Teacher’s College Press: New York; pp 112–132.


Best Practices for Supporting and Expanding ... - ACS Publications

Recommend Documents