Coordination of the Chemistry REU Program at the University of

Jul 2, 2018 - The overall goal of our training program is three-fold: (A) teach and nurture interest in .... school application procedures, poster ses...
0 downloads 0 Views 662KB Size
Download PDF
Downloaded via UNIV OF SYDNEY on July 15, 2018 at 19:04:20 (UTC). See https://pubs.acs.org/sharingguidelines for options on how to legitimately share published articles.

Chapter 10

Coordination of the Chemistry REU Program at the University of Nebraska−Lincoln Mark A. Griep,*,1 Marilyne Stains,1 and Jonathan Velasco2 1Department

of Chemistry, University of Nebraska−Lincoln, Lincoln, Nebraska 68588-0304, United States 2Department of Chemistry, Colorado State University−Pueblo, Pueblo, Colorado 81001-4901, United States *E-mail: [email protected].

During its first six years, the summer research program in chemical assembly at the University of Nebraska-Lincoln trained 50 students to think like researchers, to communicate science to different audiences, and choose a career path that is aligned with their personal goals. The participants are selected using a two-filter model in which applicants are first chosen for features including being from a non-research university, GPA greater than 3.0, U.S. citizenship, and class standing at the sophomore or junior level. The second filter involves gathering information from each file to find students who would benefit the most from a research experience. Factors that weigh heaviest are student intent to go to graduate school, personal statements about career plans, and no or few prior research experiences. Once at UNL, our model is to develop reciprocal relationships between each summer researcher, faculty advisor, and graduate student mentor. Although these relationship triangles share common features, they are tailored to the specific experiences of each student because over 90% of the student’s time is spent doing research. An innovative feature of our program is that each cohort of students competes in a poster contest for funds to travel to a national or regional meeting. As such, students attend workshops devoted to communicating science. They learn how to communicate with scientists by creating a poster and how to communicate with the public by creating a slideshow for science-interested high school © 2018 American Chemical Society Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

students. Students also take field trips to local companies to learn about career paths and opportunities. Weekly activity reports allow the program coordinator to track progress and to identify possible problems reasonably quickly. The evidence indicates that our participants become increasingly independent researchers throughout the summer. Summative evaluations coupled with self-reporting on a LinkedIn account allows the program coordinators to track outcomes. After our participants graduate, they choose to continue as scientists or researchers either by joining a graduate program, securing employment in the science and engineering sector, or by focusing on science communication.

Introduction Given that the products of science and engineering sustain the vitality of the United States economy, the National Science Foundation’s (NSF’s) Directorate on Education and Human Resources is mandated to help prepare a broad and diverse workforce in the areas of science, technology, engineering, and mathematics (1). Over the past six years, the Department of Chemistry at the University of Nebraska-Lincoln has developed tools to makes its contribution through its research experience for undergraduate students (REU) summer program. The overall goal of our training program is three-fold: (A) teach and nurture interest in science by engaging an average of eight undergraduate students each summer in authentic research experiences, (B) teach these students how to communicate their research to scientists and to the public, and (C) significantly impact their career decisions.

The Research Relationship Triangle When we were developing our model for research training, we initially considered the chemical metaphor of a catalyst model in which the student is transformed from a novice into an expert researcher upon being trained a faculty research advisor and graduate student mentor. Upon further reflection, we realized that a mentor as catalyst was not the right metaphor on several levels. Perhaps the most important is that a catalyst does not undergo a permanent change whereas it is our goal that the student, advisor, and mentor maintain their relationships with one another after the event. Another reason the catalyst metaphor fails is that the undergraduates are hardly a homogeneous group. Instead, their prior research experience exists along a continuum that needs to be acknowledged. Graduate mentors also gain valuable experience in teaching others to do research. The catalyst metaphor also does not account for multiple mentor inputs such as the faculty research advisor who selects and manages the project and the graduate student mentor, who is in the research advisor’s lab but who provides a great deal of the initial training and then routine oversight. These objections led us to develop a mentoring model of reciprocal relationships (Figure 1). An important 140 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

feature of this model is that it acknowledges the existing and distinct relationship between the Advisor and the Graduate Student. A successful mentor must negotiate his or her own research project while nurturing the summer researcher in a way that is consistent with the advisor’s directions. In essence, the link between each pair is distinct.

Figure 1. The Research Relationship Triangle. Preparing for the REU experience starts weeks before the students arrive with advisors welcoming them by email and engaging them in their assigned research project through journal articles. Shortly before the arrival of the students, the Chemistry REU Coordinator, Griep, meets with the REU advisors and mentors to discuss the REU program’s goals and schedule. Mentors are given two worksheets developed by Stains to guide their experience (request copies from corresponding author). These sheets were created as a distillation of the material in an earlier version of the University of Michigan’s booklet “How Mentor Graduate Students: A Guide for Faculty,” which was recently updated (2). The sheets were then amended to offer additional specific practical advice as well as specifics about the summer program. One sheet describes how to be an effective facilitator (and not director) of a mentee’s research. The other sheet is for use when they meet with their mentee for the first time. It prompts them to discuss expectations, daily schedule, notebook keeping, lab dynamics, and the like. Once at UNL, students spend over 90% of their time conducting research and the rest of their time participating in Chemistry’s Communicating Science program (one to two hours per week) and UNL Graduate Studies Professional Development Progam (one to two hours per week). To ensure that REU students remain productive, they are required to submit a weekly report on their activities. These are confidential reports between each participant and the coordinator. Most of the report is a checklist of menu items as described below but it ends with an open-ended query to which students can make comments. If students don’t fill in the report, don’t indicate many activities, or respond that they are experiencing a setback unrelated to the research project itself, the coordinator is able to investigate quickly to clear the air. In addition, the Exit Interview asks REU students to describe their experience with all aspects of the program and to provide suggestions for improvements.

141 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Participant Selection Our selection procedure is student-centered in that it seeks to find those students who would benefit most from a summer research program and then matches them to the faculty member whose research program interests them the most. Our summer research program typically includes 8 students from our National Science Foundation REU in Chemical Assembly and between 2 to 6 students working with chemistry faculty on other projects. The other projects can be funded from other summer programs such as our faculty and student pair funded by the UNL Chemistry Department, high school students funded by ACS Project SEED, and/or by the ACS Nebraska Local Section. Although recent statistics (3) show that equity has been achieved in the numbers of white men and women earning bachelor degrees in chemistry, women are still underrepresented among doctoral recipients and in chemical research careers. The statistics are much less favorable with regard to people of color in 2016. Specifically, Hispanics comprise 18% of the US population (4) but earn only 3.9% of doctoral degrees in science and engineering (3). The Hispanic population is also the fastest-growing group (5), indicating why we need to get a better representation in research labs to keep pace with broadening their participation. African-Americans comprise 13% of the U.S. population (4) but earn only 2.7% of science and engineering doctoral degrees (3). Our initial expectation for REU students was thus for at least 60% of them to be female and/or from under-represented groups. We easily exceeded that target with 74% of the students meeting these criteria during the first round of NSF funding and 87% in the second round of NSF funding. We attribute this outcome to very high number of applicants (Figure 2) and a two-filter selection process that is described below. Funding for three students in 2011, our pilot year, was from the UNL Office of Graduate Studies (OGS). The OGS has one staff member who coordinates the federally funded summer research programs. OGS became involved about ten years ago when they found these summer programs were effective for graduate recruitment. The OGS assistance also helps reduce each unit’s cost and time for running such programs because they manage student travel, arrange room and board, and provide professional development programs (graduate school application procedures, poster session, social activities, etc.) for over 100 participants each summer. The Chemistry pilot year program was not advertised in advance but still attracted 91 applicants because the application form was on the same page as all other UNL summer programs (6). This allowed us to gather information about applicant demographics and about our pilot program’s effectiveness for use in our ultimately successful first NSF REU application. To drive students to their website, UNL OGS sends emails and flyers to over 7,000 STEM faculty at places including 106 Historically Black Colleges & Universities (HBCU) (7), 239 Hispanic-serving institutions (8), and 37 Tribal Colleges (9). They also recruit at numerous conferences that target underrepresented groups, such as the national meetings of the Society for Advancement of Chicanos and Native Americans in Science (SACNAS), the National Organization of Black Chemists and Chemical Engineers (NOBCChE), 142 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

and the American Indian Science and Engineering Society (AISES). The Chemistry Department sends Griep to recruit at these same national meetings where he receives approximately 40 requests for information about the chemistry REU program and 40 about the chemistry graduate program. These requests typically result in fewer than five applications to each program. In summer 2016, there were over 1200 completed applications for the combination of all UNL summer research programs. Each student supplied the following information (6): name, school, major, GPA, earned credit hours, class, graduation date, date of birth, US citizenship, gender, race or ethnicity, ranked preference among the available UNL summer research programs, and faculty advisor preferences (research projects are summarized on the website). Students also submit an official copy of their transcript and a short statement about themselves in which they also provide reasons for their interest in a summer research experience. Links are sent to the two people chosen to write support letters, who then upload their letters to the form in a way that is secure and confidential but that lets the applicant know the letters have been uploaded. The number of applicants to the UNL chemistry program increased every year from 2011 to 2015 and then began decreasing (Figure 2). However, the percentage of applicants who pass our “first filter” (described below) keeps increasing. This indicates that our recruitment efforts are becoming more effective in reaching our target applicants. In the past three years, we are receiving fewer applications from students enrolled at research universities, from students with considerable prior summer research experience, and from students whose sole intended goal is to enter medical school. To find students who would benefit the most from a chemistry program, the chemistry selection committee passes the completed application files through two selection procedures called “filters.” The committee consists of the coordinator Griep, the chemistry business manager, and the chemistry staff member in charge of recruiting and communications. It takes about 90 minutes for each person to review and rank 50 files after we convene for a discussion. The “First Filter” separates the files based upon the following criteria: chemistry program ranked at the highest preference, a GPA above 3.0, U.S. citizenship, and class standing at the sophomore and junior level plus seniors who will graduate in one year. From 2012 to 2016, the percent of completed applications that makes it through the “First Filter” has increased from 41% to 69%. As our numbers of applicants has grown, the applicants who are driven to the site by recruitment efforts are more likely to meet our criteria. A retrospective analysis indicates that, among our 2016 applicants, 161 out of 232 applicants passed the First Filter, of which 80 (50%) were female and 46 (29%) were underrepresented students (African American, Hispanic, and Multiracial). The “Second Filter” involves reading and assessing files that passed the First Filter for the following target criteria: (1) student intent to go to graduate school but including some students who are also considering a professional school so that our program can influence their decision, (2) the student’s personal statement about their future interest in a research career, (3) their prior research experiences so we can favor those with less, (4) highest chemistry course completed so we can favor students with a moderate amount of coursework to influence their future 143 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

performance, (5) the letters of support from two faculty members to determine whether the student works well with others and is diligent about completing tasks. These five factors are used to rank the files for how much the applicant would benefit from a summer research program in a way that broadens the experience to those who know why they want to participate but who have fewer opportunities but enough coursework to be able to do well. Next, the faculty advisors receive two of the highest ranked files from students who selected them as their top priority. Griep then sends an email offer to the faculty member’s first choice and gives the applicant one week to respond. The first choice acceptance rate has varied from year to year, covering the range from 60 to 100%.

Figure 2. Number of students who completed their application to the UNL Chemistry REU program from the 2011 pilot year to 2018, who meet the first filter, who are given offers to join the program, and who participate.

Communicating Science and Other Workshops To provide context for their research projects, students attend workshops offered by the Chemistry program and by Graduate Studies (Table 1). To help students remember where they should be, most Chemistry meetings are on Monday mornings and most Graduate Studies meetings are on Wednesday mornings. Students develop skills in communicating their research to scientists (ACS Meeting abstract, poster presentations) and the public (develop and give presentations to students involved in the Upward Bound Math Science program), learn about careers in industry (through field trips) and academia (graduate admissions workshop), and are trained on the use of state-of-the-art instruments, scientific authorship, and science ethics. Through a poster presentation competition at the end of the summer, half of the students receive a travel grant to present their research at a regional or national ACS meeting. 144 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Table 1. Summer Activity Timetablea,b Mon

Week

Tue

1

SRP Orient, Chem REU Orient, Safety Training

2

SciComm1

3

SciComm2, CommSciPublic1, UpwardBnd1

4

Instrum Tours

Wed

Thu

SRP Picnic SRP1

Chem Picnic

145

SRP2, UpwardBnd3

UpwardBnd4

CommSciPublic2, FieldTrip1

SRP3 (SciComm3)

Chem in the Movies

5

CommSciPublic3, FieldTrip2

SRP4

6

SciComm4, FieldTrip3

SRP5

7

CommSciPublic4 (UpwardBnd5)

SRP6

8

SciComm5

SRP7

9

FieldTrip Debriefing

SRP8

10

SciComm7

UpwardBnd2

SRP Banquet

Fri

ChemPoster, SRP Poster, Awards

SciComm6

Travel Home

a

Summer Research Programs (SRP) sponsored by UNL Graduate Studies: SRP Orient is the UNL Summer Program Orientation; SRP Picnic is a Welcome Picnic; Ethics is Ethics Workshop for Physical Sciences; SRP1-8 are the weekly workshops where 1 is ethics, 2 is diversity and inclusion, 3 is poster judging, and 4-8 are related to graduate school admissions; Banquet; Chem in the Movies is Chemistry in the Movies presentation by Griep; SRP Poster is UNL-wide Summer Research Poster session. b Chemistry-specific Programs: Chem REU Orient is Orientation; Safety is Chemical Safety Training; Instrum Tours is Chemistry Instrumentation Tours; SciComm is the Scientific Communication to Scientists series; CommSciPublic is Communicating Science to the Public series; Chem Picnic is Chemistry Picnic; UpwardBnd is Upward Bound Math Science Shadowing; FieldTrips are to 3 Local R&D Companies; FieldTrip Debriefing is a lunch discussion; ChemPoster is Chemistry Poster session; and Awards is Travel Awards.

Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Safety Prior to their arrival, Chemistry REU participants are required to complete three online safety-training exercises created by UNL Environmental Health and Safety. On the first day of their summer program, they attend a live presentation about safety topics of special interest to chemical researchers such as identifying chemical hazards, using personal protective equipment, how to handle chemical spills, and how to dispose of chemicals. They also learn how to use a fire extinguisher. Instrumentation Tours To help the REU participants learn what instrumentation is available, they are given tours of three facilities in the UNL Chemistry Department: atomic force microscopy, NMR spectroscopy, and 3D Printing. All students must then attend at least one workshop (chosen in consultation with their advisors) on how to use an instrument, run by faculty who use these instruments in their research. Science Communication Workshops (SciComm1-7 in Table 1). Scientists communicate their results with each other through papers, posters, and presentations. The practical goal of the science communication workshops is to create a poster for a regional or national meeting of the American Chemical Society (ACS). The poster is also used for the Poster Sessions sponsored by UNL Chemistry and the Summer Research program. Each SciCom Workshop is led by Griep and has a unique focus. SciComm1: Prior to their first meeting, students receive guidelines for preparing an ACS meeting abstract (300 words, etc.) with the instructions to consult with their Advisor and Mentor on their research goal(s), methodology, and the importance of their project. This exercise causes them to consider how their research project is connected to societal topics. The first meeting focuses on helping students make the societal connections. SciComm2: We review revised abstracts for 15 min, discuss how they could be improved, and then each student is given abstracts from two other students to revise yet again. During the next 45 minutes, we discuss authorship issues (10, 11) and Clement’s Authorship Matrix (12). The 2014-2017 students found the authorship discussion to be illuminating. Clement conceptually divides a scientific contributions into four elements: ideas, work, writing, and stewardship. He proposes that authorship discussions begin by assigning weights to these elements: 0.2 for ideas, 0.3 for work, 0.35 for writing, and 0.15 for stewardship, and then each author’s contributions to each element are assigned in a way that each element’s total is 1.0. We discuss how REU participants can meaningfully contribute to each of these categories such as through writing their methods and results. SRP3 (SciComm3): The highly successful workshop developed by Griep for the Chemistry REU is now used by all the REU programs at UNL as described in the SRP section. SciComm4-7: We review first drafts of students’ poster designs, receive tips from a poster-award-winning graduate student about presenting a poster at a national meeting, review second drafts 146 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

after they have been peer-reviewed by two other participants, and review final drafts before printing. Students are then given the opportunity to practice a 5-min oral presentation with their poster. Peer feedback is the key feature of all these meetings. SRP1−8 UNL Graduate Studies offers a series of workshops relevant to all REU summer research participants. SRP1: The Scientific Ethics workshop covers good scientific conduct within the physical sciences. Prior to the workshop, students read the core text for scientific conduct, the National Academy’s On Being a Scientist. During the workshop, students learn about publication ethics such as scientific misconduct, plagiarism, ownership of knowledge, responsible authorship, record keeping, and data ownership (13, 14). SRP2: The “Diversity and Civility: Why it Matters” presentation by Charlie Foster, UNL OASIS Program Director. SRP3 (SciComm3): During the UNL-wide workshop led by Griep, students learn about poster design principles and the criteria used by judges in poster contests. UNL Graduate Studies arranges for ten graduate mentors to give their presentations in a mini-poster session so that all summer research participants can use the criteria to judge real posters. After the scores are tabulated and the top-rated posters identified, the students examine the top three posters and discuss why they were better than the others. The 2017 UNL-wide presentation to students and graduate mentors reported that this metacognitive activity was enlightening. They learned that large figures and less text are components of good poster design. SRP4-8: These workshops cover the graduate school application process and GRE preparation. Communicating Science to the Public Workshops (CommSciPublic1-4 in Table 1). The goal of these workshops is for each student to develop a 10-minute oral presentation about her or his research that is then given to the UBMS students (see next section). This exercise also prepares REU students to give their 5-min oral presentations of their posters. It is important that budding scientists learn to communicate across an ever-broadening array of disciplines. The four Workshops were developed and taught by Dr. Marilyne Stains during the first two years of our REU program and then taught by Jonathan Velasco, one of her graduate students. CommSciPublic1: During the first meeting, students reflect on the goal of sharing research to diverse audiences, watch and reflect on examples of scientists presenting their research to the public. Students are then presented with guidelines on the structure of a talk to formatting considerations. These discussions are supported with videos of presentations given at National Meetings of the ACS (15). CommSciPublic2: Prior to the second meeting, students are asked to read a paper describing a rubric that analyzes how scientists explain their research (16) and to apply that rubric to two different TED talks delivered by scientists (17, 18). Stains adapted this pre-meeting activity from a workshop developed by her postdoctoral advisor Hannah Sevian. During our meeting, students use the active learning strategy 147 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

called a gallery walk (19) to unpack the constructs measured in the rubric: Pedagogical knowledge, Content knowledge, Pedagogical content knowledge. The essence of a gallery walk is that students travel in groups between stations, where they discuss the question at that station and then move to the next station. In our case, the ten students were divided into three groups, each of which was asked to consider a different construct. After a brief discussion, they wrote the characteristics that defined the best practices for that construct. Next, each group moved to the adjacent station where they amended and edited the prior group’s notes. Then, they returned to their original station to discuss their amended notes and to report out to the group. With this fresh understanding of the rubric in mind, the students used it to analyze the two TED talks again. During the final portion of the meeting, the students spent ten minutes laying out the structure of their own talk and then gave a two-minute explanation of their research talk to a peer who used the rubric to provide feedback. CommSciPublic3: Three days prior to the meeting, students send their developed slideshows to a designated peer and Stains. Stains and the peer then rate the presentation using the rubric described in CommSciPublic2. During the meeting, each student practices his or her full presentation with a peer to get further feedback. CommSciPublic4: Students give their 10-min presentations to the Upward Bound Math Science students and others in the department. Upward Bound Math Science Shadowing Project (UpwardBnd1-5 in Table 1). Each REU student participates in a four-day shadowing event during the third week of the experience. Each REU student is assigned one high school student from the Upward Bound Math Science (UBMS) summer program. The UBMS Program is funded by the Department of Education and is designed to strengthen the math and science skills of participants who have the ability and/or desire to enter a math/science field of post-secondary education. Participating students must meet the federal guidelines for low-income status, and be the first generation in their family to plan to attend college. The UBMS students learn what the REU student is doing, how they are doing it, and why they are doing it. In the evenings, the UBMS coordinator shows the UBMS students learn how to create a slideshow. The next day, they discuss it with their REU student. At the end of the fourth day, the UBMS students report what they’ve learned in a 5minute presentation. A few weeks later, the UBMS students return to hear the REU students give their final slideshow presentations in our Communicating Science to the Public sequence. We believe that this metacognitive approach benefits both the REU and UBMS students. The experience benefits the UBMS students by giving them insight into scientific research and it benefits the REU students by giving them an opportunity to communicate their research to students with interests in science but who lack coursework. Field Trips to Local Companies (FieldTrip1-3 in Table 1). Field trips provide REU students with real-world examples of how chemistry research relates to products and services. The Lincoln 148 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

area is rich with companies that have strong ties to UNL (Table 2) and have hired personnel who have earned a BS, MS, or PhD from the UNL Chemistry Department. Every year, we tour a different set of three companies selected from this diverse set. Prior to each field trip, students read a description from the company’s website plus a newspaper article about a recent development. Each tour begins with a presentation by the research director or other guide about the company’s business model and the nature of their most common research activities. Students learn how scientific findings translate into industrial endeavors, the chemistry involved in some of the products, and the career opportunities at the company. Next, students tour the research and development, quality control laboratory, and manufacturing plant. At the final debriefing (FieldTrip Debriefing in Table 1) students first share their thoughts about the field trips and then discuss career paths in industry and some of the differences between industry and academia.

Table 2. Field Trip Destinations Visited Most Often, 2012-2017 Company

Approximate Number of Employees

Celerion, a comprehensive early clinical research and bio-analytical services provider (global headquarters in Lincoln)

950

Teledyne ISCO, a manufacturer of instruments for separations and water analysis

420

LI-COR, a manufacturer of infrared detection instruments for agriculture, biotechnology, drug discovery, and the environment (global headquarters in Lincoln)

350

GSK (formerly Novartis), a manufacturer of over-the-counter pharmaceuticals

350

Hexagon Lincoln, a manufacturer of filament-wound fuel tanks

300

Geneseek, a division of Neogen Corp., develops and processes test kits for genotyping of animals, plants, and microbes

150

Chemistry in the Movies All UNL Summer Undergraduate Researchers are invited to attend a presentation by Dr. Griep about Chemistry in the Movies to learn how scientific findings make their way into movies (20–22). Poster Sessions and Travel Awards On the final day of the 10-week experience, there is a Chemistry Research Poster session in the morning (Chemistry Poster) and a campus-wide Summer Research Program poster session in the afternoon (SRP Poster). At the Chemistry poster session, each poster is separately judged by two chemistry faculty. Half of 149 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

the highest-scoring REU student posters win a $1000 travel award from the REU program to attend either a Regional or National ACS meeting during the following year. Nineteen of the 22 travel awardees (86%) have already used their funds to present their poster at a conference. Six of the 27 non-awardees (22%) found other funds to present their poster at a conference. These students are disappointed they didn’t win the travel award but they are fully aware at the small difference in poster scores between awardees and non-awardees. There do not appear to be hard feelings among those who did not win because six travel awardees and five non-awardees applied for admission to the UNL chemistry graduate program. Of these, six entered our program and five enrolled in chemistry graduate programs elsewhere. The judge’s poster scores also provide us with a way to evaluate the “Communicating Science to Scientists” workshops. Notably, there was a dramatic increase in REU student poster quality (Figure 3) because of the workshops. At these sessions, students learn how posters are judged and how poster design helps them tell their story. They also develop and practice giving their 5-minute poster oral presentation. Students learn that the judges enter scores from 1 to 10 into three categories, each with a list of attributes. Content means that the poster has sections for: Title, Names, Funding, Objectives, Significance, Methods, Results, and Interpretation. Display means the poster should attract attention and convey information without jargon or spelling errors, that the poster is well-organized and that a sufficiently large font size and images are used. Oral presentations should be less than 5 minutes, clear and concise and the student should clarify contributor work roles when necessary.

Figure 3. Average poster scores for REU participants in the UNL Chemistry REU program from the 2011 pilot year to 2014. In these four years, the posters were rated in three categories worth 10 points each. Students received workshop training in poster design in summers 2012-2014 but not in 2011. It can be seen from the average poster scores (Figure 3) that the REU students in our 2011 pilot year (without workshops) were equivalent to first-year graduate students (overall average score of 7.0) wheras those who participated in our 20122014 workshops are equivalent to those of senior-level graduate students (overall average score of 7.6). 150 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

In summer 2015 and 2016, the faculty coordinator of the Chemistry Poster Session sought to enhance the quality of the judging by increasing the number of scored categories and by weighting two of the categories heavier than the others. The result was actually less nuance from the judges and more confusion about awarding points. Therefore, in summer 2017, a new poster scoring system was developed. It built upon the original but added a fourth category added to account for the ability to answer deep questions about the project. In addition, the instructions were clarified as follows: 10=best, 1=worst; start your score at 7 and raise or lower depending on the evidence. Our new categories are: Content means the poster components (title, abstract, results, discussion, graphics) communicate the nature and importance of the work to a broad audience. Display means the poster should attract attention and convey information, is well-organized and simple, that the text and chemical structures are large enough to read at a distance, and that there is no distracting jargon or spelling errors. Oral describes the oral presentation, which should be clear, concise, and about 5 minutes long if not interrupted. If there are multiple investigators, the speaker should clarify the other people’s roles. Knowledge describes the degree of understanding of the project demonstrated by the presenter’s description and, in particular, by answers to questions. This system was well received by the judges and we await a second summer’s trial.

Assessment Protocols Our assessment protocols were submitted to our Institutional Review Board before the original proposal was submitted to NSF and it was certified as exempt. Formative Evaluation A weekly activity log is the primary source for real-time monitoring of the quality of program activities and allows for relative quick programmatic changes. Stains designed the “What did I do this week?” online activity log that asks students to identify all research-oriented activities that were conducted that week among a list of twenty one (Table 3), select those that they spent the most time on, and identify those that were most beneficial and explain why. One example of an important change occurred during our first year when the Communicating Science workshops were scheduled on Tuesday and Wednesday afternoons. After three weeks, several students let us know that they felt our workshops were disrupting their research productivity either because they had difficulty keeping track of them or because they took place during a productive time of day. We immediately rescheduled these workshops to Monday mornings where they have since remained. Of course, students are asked for their feedback immediately after the workshops, field trips, and the shadowing experience. An example of a change from this type of feedback occurred after our third year. The shadowing experience was scheduled during the second week in our first three years until our third-year cohort indicated that while they learned from it, they felt it was too early for them to communicate the practical aspects of their projects. 151 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Therefore, it was moved to the third week. The level of student engagement when they write their poster abstracts or create their poster drafts and slideshows are also useful for assessing program effectiveness.

Summative Evaluation The summative evaluation measures success in meeting the main objectives of the program (such as attitude towards chemistry and self-efficacy in chemistry) and its overall and lasting impacts (such as postgraduate training and careers). Evaluating the main objectives relies upon surveys that are administered online at the end of the summer program. The students’ attitude toward chemistry is measured with the validated short version of the Attitude toward the Subject of Chemistry Inventory (ASCI) (23). Students’ self-efficacy in chemistry is measured using the Chemistry Attitude and Experiences questionaire (CAEQ) (24). During our first few years, the validated Survey of Undergraduate Research Experiences (SURE) (25) was used to measure several of our goals. This survey has since been replaced by a more specific list of questions developed by Stains that includes a field for open-ended suggestions about improvements to the program. We also survey the Advisor and Graduate Student Mentor about their statisfaction with their REU student and with the program. During the first couple of years we conducted exit interviews with REU students but concluded that they did not provide new information when compared to the surveys and thus stopped conducting them. To assess the long-term impact of the program on students’ education and career plans, Griep monitors the LinkedIn.com pages that students create during their summer in the program.

Student Outcomes The program was a very productive experience for our participants. Of the 24 participants in summers 2014-2016, 11 gave first-author presentations about their work at ACS and Council for Undergraduate Research national meetings and 3 gave presentations at statewide conferences. At the ACS national meetings, one student was selected for the Sci-Mix Interdivisional Poster Session and another won the “Simply Speak” Contest. A third student won “Best Undergraduate Paper” at one of the regional meetings. Since completing their respective REU programs, 17 participants from 2014-2016 earned B.S. degrees. Of these, nine entered research graduate programs and five entered the workforce. The nine students who entered graduate studies did so in chemistry (Case Western Reserve, Indiana, Iowa State, UIUC, UNL), polymer science (Akron), food science (Penn State), genetic epidemiology (Univ. of Utah), and medicine (MD/PhD, SUNY Upstate). The five students who entered the workforce chose the following positions: 2 lab technicians, 2 quality control/quality assurance technicians, and 1 outreach coordinator.

152 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Table 3. “What did I do this week” activity queriesa Type of activity

Statement

153

Practicing professional-level lab conduct

Write in your notebook Resolve unexpected problems Planning for the next step guided by your mentor Planning the next step by yourself Help others in the lab with their project Help other REU students with their project

Researching or working on assignments for the program

Research the literature Read research article Work on assignments (abstract, poster, lesson plan, etc.) Attend a group meeting

Conducting experiments

Doing experiments by yourself Doing experiments with your mentor’s guidance Watching your mentor as he/she explains and conducts the experiments and you are not actually doing the experiments Design experiments by yourself Design experiments with your mentor’s guidance

Discussing or presenting projects

Prepare a presentation for group meeting and present it Talk about your project with your graduate student mentor Talk about your project with your faculty advisor Talk about your project with other members of your research group Talk about your project with other REU students Talk about your project with other people than the people listed above

a

Every Friday, students receive an email directing them to a secure website that asks them to identify which of these activities were conducted that week, select those that they spent the most time on, and identify those that were most beneficial and explain why.

Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

Conclusion Our NSF-funded program has been operating for six summers during which time we have trained 50 students. The students spent the majority of their time learning how to do research while the workshops gave the students a chance to network with each other about the common features of their experiences. The evidence indicates they became increasingly independent researchers by the end of the summer and that they valued the workshops on communicating science, interacting with high school students, and touring local industries. Additional evidence for the effectiveness of the workshops is that over half of our participants presented their research at a national of regional meetings and a few of them won awards for their presentations. All of these experiences have the net effect of inspiring our participants to earn science degrees and to choose science careers.

Acknowledgments We acknowledge funding from NSF grants 1156560 and 1460829, from the UNL Office of Graduate Studies, and from the UNL Department of Chemistry.

References 1.

2.

3.

4. 5.

6. 7.

8.

National Science Foundation. About Education and Human Resources; National Science Foundation: Washington, DC. https://www.nsf.gov/ehr/ about.jsp (accessed 03/30/2018). University of Michigan. How To Mentor Graduate Students: A Guide for Faculty; University of Michigan Rackham Graduate School: Washington, DC, 2015. http://www.rackham.umich.edu/publications/ (accessed 03/30/201). National Science Foundation. Science and Engineering Degrees, by Race/Ethnicity of Recipients: 2002–12; Detailed Statistical Tables, NSF 15-321, May 2015; National Science Foundation: Washington, DC. https://www.nsf.gov/statistics/2015/nsf15321/ (accessed 03/30/2018). U.S. Census Bureau State & County QuickFacts. https://www.census.gov/ quickfacts/fact/table/US/PST045216 (accessed 01/03/2018). U.S. Census Bureau. U.S. Population Projections. Table 10. Projections of the Population by Sex, Hispanic Origin, and Race for the United States: 2015 to 2060. https://www.census.gov/data/tables/2014/demo/popproj/ 2014-summary-tables.html (accessed 01/03/2018). University of Nebraska-Lincoln Summer Research Program. http:// www.unl.edu/summerprogram/ (accessed 01/03/2018). U.S. Department of Education List of HBCUs—White House Initiative on Historically Black Colleges and Universities. https://sites.ed.gov/whhbcu/ files/2014/09/HBCU-Directory.pdf(accessed 01/03/2018). Hispanic Association of Colleges & Universities. https://www.hacu.net/assnfe/CompanyDirectory.asp? style=2&company_type=1,5&search_type=0 (accessed 01/03/2018). 154 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

9. 10.

11.

12.

13. 14.

15.

16.

17.

18.

19.

20. 21.

22.

23. 24.

American Indian Higher Education Consortium. http://www.aihec.org/whowe-serve/TCUroster-profiles.htm (accessed 01/03/2018). Hu, X. Loads of special authorship functions: Linear growth in the percentage of “equal first authors” and corresponding authors. J. Am. Soc. Information Sci. Technol. 2014, 20, 345–361. Marusic, A.; Bosnjak, L.; Jeroncic, A. A systematic review of research on the meaning, ethics and practices of authorship across scholarly disciplines. PLoS One 2011, 6, e23477. Clement, T. P. Authorship matrix: A rational approach to quantify individual contributions and responsibilities in multi-author scientific articles. Sci. Eng. Ethics 2009, 60, 2378–2381. Ritter, S. K. Publication ethics: Rights and wrongs. Chem. Eng. News 2001, 79 (46), 24–31. Whitbeck, C. The Responsible Collection, Retention, Sharing, and Interpretation of Data, 2006. Natl. Acad. Eng. http://www.onlineethics.org/ Resources/TeachingTools/20357/19237/moddata.aspx (accessed 01/03/ 2018). American Chemical Society, 241st ACS National Meeting & Exposition Session List. http://www.softconference.com/ACSchem/slist.asp?C=4321 (acessed 01/03/2018). Sevian, H.; Gonsalves, L. Analysing how scientists explain their research: A rubric for measuring the effectiveness of scientific explanations. Int. J. Sci. Educ. 2008, 30, 1441–1467. Pike, R. The Science Behind a Climate Headline, 2009. TED Talk. https:/ /www.ted.com/talks/rachel_pike_the_science_behind_a_climate_headline (accessed 01/03/2018). Behncke, I. Evolution’s Gift of Play, From Bonobo Apes to Humans, 2011. TED Talk. https://www.ted.com/talks/ isabel_behncke_evolution_s_gift_of_play_from_bonobo_apes_to_humans (accessed 01/03/2018). Francek, M. . Gallery Walk. Pedagogy in Action: The Portal for Educators website. https://serc.carleton.edu/introgeo/gallerywalk/index.html (accessed 03/03/2018). Frey, C. A.; Mikasen, M. L.; Griep, M. A. Put some movie Wow! in your chemistry classroom. J. Chem. Educ. 2012, 89, 1138–1143. Griep, M. A.; Mikasen, M. L. Using movie clips to teach chemistry formally and informally. In Hollywood Chemistry; Nelson, D. J.; Grazier, K. R.; Paglia, J.; Perkowitz, S., Eds.; ACS Symposium Series 1139; American Chemistry Society: Washington, DC, 2013. Griep, M. A.; Mikasen, M. L. Close encounters with creative chemical thinking: An outreach presentation using movie clips about the elemental composition of aliens and extraterrestrial minerals. Educ. Quím. 2016, 27, 154–162. Xu, X.; Lewis, J. E. Refinement of a chemistry attitude measure for college students. J. Chem. Educ. 2011, 88, 561–568. Dalgety, J.; Coll, R. K.; Jones, A. Development of Chemistry Attitudes and Experiences Questionnaire (CAEQ). J. Res. Sci. Teach. 2003, 40, 649–668. 155 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.

25. Lopatto, D. Exploring the Benefits of Undergraduate Research: The SURE Survey. In Creating Effective Undergraduate Research Programs in Science; Taraban, R.; Blanton, R. L., Eds.; Teacher’s College Press: New York, 2008.

156 Griep and Watkins; Best Practices for Chemistry REU Programs ACS Symposium Series; American Chemical Society: Washington, DC, 2018.