Undergraduate Research Experiences Broaden Diversity in the Scientific Workforce

Undergraduate Research Experiences Broaden Diversity in the Scientific Workforce Abstract New data highlight the importance of undergraduate research experiences (UREs) for keeping underrepresented science students on the pathway to a scientific career. We used a large-scale, 10-year, longitudinal, multi-institutional, propensity-score-matched research design to compare the academic performance and persistence in science of students who participated in URE(s) with those of similar students who had no research experience. Our results showed that students who completed 10 or more hours of cocurricular, faculty-mentored research per week across two or more academic semesters or summers were significantly more likely to graduate with a science-related bachelor's degree, to be accepted into a science-related graduate training program, and to be training for or working in the scientific workforce 6 years after graduation. Importantly, the findings show that just having a URE was not enough to influence persistence in science; it required a commitment of 10 or more hours per week over two or more semesters of faculty-mentored research. Broadening participation in the US scientific workforce is essential for expanding economic prosperity and remaining internationally competitive (NAS 2007). Building a diverse, inclusive scientific workforce is the only way for the United States to meet national research goals and for science to benefit from that diverse workforce (USDHHS 2015). Recruiting and retaining interested women and minorities are critical strategies to expand participation in the scientific workforce (Olson and Riordan 2012). Despite decades of efforts to expand and diversify the scientific workforce, data show a clear pattern of continued underrepresentation by gender and race or ethnicity (NSF 2017). Undergraduate research experiences are increasingly seen as a cornerstone of efforts to retain students on a scientific career pathway (Russell et al. 2007, Graham et al. 2013, Linn et al. 2015, NAS 2017); however, the links between these undergraduate experiences and long-term scientific-workforce outcomes lack compelling empirical support, particularly for students from underrepresented minority groups (URMs). For the United States to generate a talented, diverse, and inclusive scientific workforce, it is vital to understand how well and under what conditions undergraduate research experiences broaden scientific-workforce participation (NAS 2017). Faculty-mentored undergraduate research experiences are foundational to many programs developed to encourage students, particularly those from historically underrepresented groups, to pursue a scientific career (Graham et al. 2013). Undergraduate research experiences vary widely in their goals, in the students they attract, and in their structural elements (Seymour et al. 2004, Linn et al. 2015, NAS 2017), but these experiences typically occur in one of two settings: (1) cocurricular, apprenticeship-style, mentored research experiences in a faculty laboratory (UREs) or (2) course-based research experiences (CUREs; Sadler et al. 2010, Linn et al. 2015). CUREs represent an institutionalized strategy to update traditional science lab courses with curriculum that requires students to learn science by engaging in authentic scientific practices and scientific discovery that are of interest to the scientific community (Auchincloss et al. 2014, Rodenbusch et al. 2016). CUREs are open to many students and pair numerous undergraduate researchers with one or more research mentors across the duration of an academic term (Linn et al. 2015). UREs also engage undergraduates in authentic scientific practices and discovery, but they do so in an apprentice-style, mentored relationship with a faculty member (Seymour et al. 2004, Sadler et al. 2010). Unlike CUREs, UREs are typically competitive and limited to a smaller number of undergraduates. UREs are costly in terms of time and money; however, these experiences are widely accepted as “high-impact educational practices,” which improve academic performance and persistence in science (Russell et al. 2007, Kuh 2008, Graham et al. 2013). The present study responds to repeated calls for data on student participation in UREs, with a focus on the short-, medium-, and long-term benefits of apprentice-style, faculty-mentored UREs as opposed to those of CUREs. Mentored UREs are intended to enhance students’ interest and excitement in research, build confidence in research, cultivate teamwork and leadership skills, and increase scientific knowledge and an understanding of the nature of science and scientific practices (Lopatto 2004, Russell et al. 2007, Fakayode et al. 2014, Gilmore et al. 2015, Linn et al. 2015, Carter D et al. 2016). The distal goals for UREs are to support and crystalize student intentions to pursue a scientific career, to increase trainees’ determination to apply for graduate training, and to broaden participation in the scientific workforce (Hathaway et al. 2002, Lopatto 2004, Mervis 2006, Estrada et al. 2011)—a goal not realized for many years after the undergraduate experience. Claims about the benefits of UREs on academic performance and persistence abound but are yet to be fully substantiated by rigorously designed empirical studies. With few exceptions, claims of positive benefits are undermined by a reliance on retrospective accounts, studies with inadequate comparison groups, inadequate measurement of outcomes, and a focus on short-term rather than long-term follow-up to determine the positive effects on academic and career trajectories (Mervis 2006, Sadler et al. 2010, Linn et al. 2015). Recent reports recognize the limitations of this body of research to support causal claims about the effects of UREs (Seymour et al. 2004, Sadler et al. 2010, Linn et al. 2015, NAS 2017). For example, without adequate comparison groups, it is unclear whether the benefits of UREs are simply attributable to the type of student who seeks out a research experience rather than to the experience itself (Linn et al. 2015). If URE students have higher academic performance and are more motivated compared with those who do not participate, then arguably, the students engaging in UREs are more likely to join the scientific workforce regardless of URE participation (Carter D et al. 2016). A few studies have begun to show the potential benefits of UREs on some short- and medium-term outcomes. For example, the University of Michigan's evaluation of the Undergraduate Research Opportunity Program (UROP) used administrative records to demonstrate some benefits of the program on persistence in college among its African American UROP participants compared with African American controls (Nagda et al. 1998). Similarly, data from two longitudinal, multi-institutional studies have shown that UREs promote student aspirations to pursue a scientific career. The first, using the Higher Education Research Institute's national data, revealed that undergraduates in science training programs, which include UREs, were 15% more likely to aspire to science-oriented graduate school compared with matched undergraduates not in science training programs (Eagan et al. 2013). The second comes from our own longitudinal work with science students from underrepresented groups, which concluded that URE participants maintained higher aspirations to pursue a scientific research career compared with matched no-research controls (Schultz et al. 2011). Although this research provides some evidence of the impacts of UREs, it does not include compelling evidence for longer-term outcomes. Furthermore, these studies are limited in that they typically ignore research outside of formal URE programs (Nagda et al. 1998); rely on retrospective accounts of undergraduate experiences (Hathaway et al. 2002); and fail to measure longer-term outcomes, such as scientific-degree attainment, matriculation into a scientific graduate program, or postgraduation scientific workforce participation of URE participants, compared with those of an adequate control group (Nagda et al. 1998, Hathaway et al. 2002, Schultz et al. 2011, Eagan et al. 2013). In addition, previous studies have not fully demonstrated the salient features of UREs that might enhance these outcomes, although preliminary evidence hints at the importance of the duration (number of semesters or summers in research) and intensity (number of hours of research per week; Kremer and Bringle 1990, Carter F et al. 2009) of these experiences. A quantitative investigation of the long-term benefits of UREs, as well as of the key features of UREs, is crucial to establishing their efficacy and maximizing the potential impact of these resource-intensive experiences and is in full alignment with the conclusions of the recent National Academy report that calls for studies that improve the evidence base about the processes and effects of undergraduate research experiences. This report specifically calls for “well-designed studies” conducted in collaboration with program directors (NAS 2017). A further recommendation from the National Academy report is that study attributes include the acquisition of longitudinal, prospective, empirical data that can inform the planning and opportunities to improve quality and access of UREs (NAS 2017). The present study responds to these recommendations. Study overview To empirically test the benefits of UREs, we conducted a national-propensity-score-matched (PSM) study that followed a panel of URM undergraduate science students over 10 years. Prior studies have documented evidence that UREs sustain interest in advanced scientific training (e.g., graduate applications) and scientific research careers (Seymour et al. 2004, Estrada et al. 2011, Schultz et al. 2011). Therefore, we extended the literature by examining how the intensity and duration of UREs influenced attainment of a baccalaureate degree, cumulative grade point average (GPA) at graduation, acceptance into a scientific graduate program, and participation in the scientific workforce. The sample was composed of predominantly African American, black and Hispanic, or Latino(a) science majors from 29 US colleges across 10 years, beginning in their junior year. We used multiple methods to measure student experiences and outcomes (i.e., prospective surveys and National Student Clearinghouse records), enabling us to draw conclusions regarding academic outcomes and longer-term persistence in science (see the supplemental materials). The goal of the study was to test the impact of participation, duration, and intensity of UREs from their junior year through the summer following senior year on academic performance and scientific-workforce participation 6 years after graduation (supplemental tables S1–S3, supplemental figure S1). The participants The overall sample consisted of 1420 URM undergraduate and graduate students majoring in a science-related discipline, recruited from 50 different institutions of higher ­education across the United States. At the time of recruitment into the study (i.e., 2005, 2006, or 2007), the undergraduate participants were in their first (12%), sophomore (16%), junior (27%), or senior (33%) year of college. However, for the present study, we focused on undergraduates recruited into the study in or prior to their junior year in college (n = 774); who were majoring in biological science (e.g., ­biology), physical science (e.g., chemistry), engineering, or mathematics (n = 710); and who had completed all questions on the matching survey (n = 693). The analytic sample consisted of 577 PSM URM students (nURE = 284 and nNo-research = 293) from 29 different colleges or universities; the details of the propensity score matching are described in procedures below. Eighty-three percent of the URE group had at least one close-matched no-research student counterpart. At the time of recruitment into the study, the participants were in their early 20s and were predominantly female, were African American or Latino(a), spoke English as a first language, and were majoring in a biological or physical science. In addition, many were first-generation college students, and most attended institutions classified as low research intensity on the Carnegie classification system (tables S1 and S2). Procedure Starting in the fall semester of 2005, URM students majoring in science- and biomedical-related disciplines were recruited from universities and colleges across the United States. Students were recruited by soliciting participants from science training programs, such as the National Institutes of Health's (NIH’s) Research Initiative for Scientific Enhancement (RISE) program, as well as from gateway science and mathematics courses (e.g., organic chemistry). The participants completed biannual (fall and spring) online surveys starting in the fall of 2005 (baseline survey) through the spring of 2017 (most recent follow-up survey). The prospective design avoided retrospective biases by regularly gathering data from the participants during their undergraduate tenure. The participants received nominal compensation for their participation ($20). The per-survey response rates have been between 70% and 85%. After 10 years, we remained in contact with 98% of the panel through email and/or phone calls. Propensity score matching Quasi-experiments are often used in place of randomized controlled trials when randomization procedures are not feasible. However, it is well understood that selection bias can occur when attributes related to assignment to the treatment group are also correlated with outcomes of interest (Rosenbaum and Rubin 1983, West et al. 2008). In order to control for selection bias, we constructed a matched sample of students who had engaged in URE(s) and a similar sample of students who had not engaged in research during their undergraduate tenure. To create matched URE and no-research groups, PSM procedures were used to calculate the probability that a student ever engaged in a URE on the basis of 17 covariates measured in the baseline matching survey using MatchIT software (Ho et al. 2007, 2011, Thoemmes 2012, Pan and Bai 2015). After matching students with and without research experiences, the URE group and no-research control group exhibited an acceptable balance on all covariates, and the estimate of selection bias dropped by 97% (table S2, supplemental figure S2). Measures Measurement of student experiences and achievements took place over a 10-year period (2005–2014). Research experiences (including intensity and duration) and GPA were prospectively measured during each student's undergraduate tenure (i.e., junior year, summer before senior year, senior year, and summer after senior year). Although students were recruited into the study at different stages in their undergraduate tenure (e.g., sophomore or junior), junior- and senior-year research experiences typically occurred in the date range of 2006–2008. Attainment of a science bachelor's degree, acceptance into graduate school, and engagement in a scientific career were prospectively measured via surveys and were confirmed with data from the National Student Clearinghouse on multiple occasions (latest was spring 2014). The following paragraphs provide a general description and details on the operationalization of the variables used in this study. Engagement in an undergraduate research experience In each survey, the students were asked whether they had “worked in laboratory in current college/university” in the current semester or term (0, no; 1, yes). The students were also asked about participation in off-campus summer research experiences by asking whether they had “worked in laboratory at another college/university” or had “worked on research at another location” (0, no; 1, yes). A dummy-coded “ever engaged in one or more URE(s)” variable (0, no; 1, yes) was derived from the student responses starting in spring semester of junior year through the summer following senior year. Intensity of the URE(s) If the students indicated they had engaged in research activities during the school year or over the summer, they were asked the follow-up question, “Please indicate approximately how many hours per week you worked on research.” The students reported the approximate numbers of hours working on research separately for “worked in laboratory in current college/university,” “worked in laboratory at another college/university,” and “worked on research at another location.” The response scale was open so that students could report any number of hours. The student responses were recoded to indicate whether they had engaged in a low-intensity URE (less than 10 hours per week) or a high-intensity URE (10 or more hours per week) that semester or term (table S3). Duration of low- and high-intensity URE(s) Two summary indices were created to tally the total number of semesters the students reported being involved in low- or high-intensity UREs (i.e., junior spring, summer before senior year, senior fall, senior spring, and summer following senior year). The first index was the sum of low-intensity UREs. The low-intensity index was recoded into two dummy-coded variables to indicate only one semester of low-intensity URE or two or more semesters of low-intensity URE (the reference group was the no-research group). An identical procedure was followed to create a summary index of high-intensity UREs, as well as two dummy-coded indicators of only one semester of high-intensity URE or two or more semesters of high-intensity URE (the reference group was the no-research group; table S3). Attainment of a baccalaureate degree in a scientific discipline Across survey waves, when each student indicated that they had completed a baccalaureate degree, they were asked to report the year, the institution, the type of degree (e.g., BA or BS), and their major. Data from the National Student Clearinghouse were used to confirm the self-reported degree attainment. We used the National Science Foundation list of recognized science-oriented majors (LSAMP 2010) to classify degree award by discipline (0, nonscience; 1, science; table S3 note). Cumulative GPA at graduation Students reported their “current cumulative college GPA” on each survey. The GPA aligned with the semester of graduation used as the final GPA at graduation (table S3). Acceptance into a science-related graduate program On each survey, students were asked, “In the last 6 months, have you been accepted into any graduate schools?” (0, no; 1, yes). If students reported yes, they were asked the follow-up question, “How many were science-related graduate programs?” (open-ended response format). Data from the National Student Clearinghouse were used to confirm graduate-school enrollment. The data were recoded into a dummy-coded variable indicating whether they had ever been accepted into a science-related graduate program (0, no; 1, yes; table S3). Scientific-workforce participation (postbaccalaureate) Scientific-workforce participation was measured using the participants’ self-reported data and enrollment data from the National Student Clearinghouse in fall 2013 and spring 2014. The students no longer enrolled in a college were asked to describe their primary occupation, including job title, and whether they considered it to be a science-related occupation. The students still enrolled in a college (e.g., graduate school or postdoctoral study) were asked about their current field of study. Their responses were used to create a dummy-coded variable that indicated scientific-workforce participation as either an employee or student (coded 0, nonscience; 1, science related). Science-related careers included being enrolled in a degree program (e.g., graduate school); pursuing a science-related degree (e.g., biology, chemistry, computer science, or neuropsychology); or holding a science-related occupation, such as science-related scholar or teacher (e.g., assistant professor or lecturer) or lab scientist (e.g., scientist II or blood-lab supervisor). The nonscience list included being enrolled in a medical or clinical degree program (e.g., clinical, counseling, or school psychology; dental school; medical school; nursing; occupational therapy; or public health), holding a medical or clinical occupation, being enrolled in other degree programs (e.g., business, education, social science, or social work), holding an occupation in a nonscience and nonmedical field (e.g., office manager or law enforcement), or being nonenrolled and unemployed (table S3). Analysis overview The data were analyzed using weighted multilevel regression. PSM-generated sampling weights were used to account for unequal sample sizes in the matched groups, and a two-level multilevel analysis was used to account for the nesting of students within schools. Four dummy-coded predictor variables were used to represent the duration and intensity of UREs (one semester low intensity, two or more semesters low intensity, one semester high intensity, two or more semesters high intensity; the reference group was the no-research group for all dummy-coded predictor variables). All of our analyses were conducted using maximum likelihood with robust standard errors estimation in Mplus v7.4 (Muthén and Muthén 1998–2017). Results Across the five semesters and summers from junior year, almost half of our sample engaged in at least one URE, and 32% of the students participated in at least one high-intensity experience (figure S1). Long-term, high-intensity UREs (10 or more hours per week across two or more semesters) had the greatest impact on student outcomes. Grade point average Students with long-term, high-intensity UREs had GPAs 0.13 units higher than those of the students in the matched no-research control group (β = 0.33, p = .03; MGPA 3.37 and 3.24, respectively; supplemental table S4). Earning a baccalaureate degree in a scientific discipline The odds of earning a baccalaureate degree in the sciences were more than nine times greater for students with long-term, high-intensity UREs compared with those for the matched no-research controls (odds ratio = 9.64, p < .001; table S4). More specifically, the odds of earning a science bachelor's degree for the no-research control group were 3.30:1, whereas the odds for those with long-term, high-intensity UREs were 31.82:1. To make these findings more concrete, predicted values derived from the multilevel models were used to show the probability (probability = odds/[1+odds]) of earning a scientific degree as a function of the duration and intensity of the URE (figure 1a). The 95% confidence intervals around each predicted value provide complementary information to the estimates in table S4 and allow for evaluating statistical inference by eye (Cumming and Finch 2005). For example, the confidence intervals around the point estimate for the students with no research (white bar) did not include the point estimate for the students with long-term, high-intensity engagement in UREs (dark gray bar with angled lines), indicating a statistically significant difference between these two point estimates (p < .05). Note that the confidence intervals around the point estimates from logistic regression are asymmetrical because of the logit transformation and the asymmetric scale of odds ratios. Figure 1. View largeDownload slide The probability of graduating with a science degree (n = 388; 1a), being accepted into a science-related graduate program (n = 577; 1b), and scientific career engagement (n = 393; 1c) as a function of the duration and intensity of undergraduate research experiences (UREs). The predicted values were computed from PSM-weighted multilevel logistic regression models with dummy-coded variables indicating duration of low-intensity and duration of high-intensity UREs. The multilevel-modeling-derived point estimates and confidence intervals account for the nesting of students within colleges and universities. The error bars represent 95% confidence intervals. Figure 1. View largeDownload slide The probability of graduating with a science degree (n = 388; 1a), being accepted into a science-related graduate program (n = 577; 1b), and scientific career engagement (n = 393; 1c) as a function of the duration and intensity of undergraduate research experiences (UREs). The predicted values were computed from PSM-weighted multilevel logistic regression models with dummy-coded variables indicating duration of low-intensity and duration of high-intensity UREs. The multilevel-modeling-derived point estimates and confidence intervals account for the nesting of students within colleges and universities. The error bars represent 95% confidence intervals. Acceptance into a graduate program As we show in figure 1b, the odds of being accepted into a science-related graduate program were four times greater for students with long-term, high-intensity UREs compared with those for matched no-research controls (odds ratio = 4.23, p < .001; table S4). Students with one semester of high-intensity or two or more semesters of low-intensity UREs were also more likely to be accepted into a science-related graduate program compared with the matched no-research controls. Working as a scientist Speaking directly to the impact of UREs on the scientific workforce, figure 1c shows that the odds of being engaged in a scientific career or advanced scientific career training 6 years after graduation were more than three times greater for students with long-term, high-intensity UREs compared with those for the matched no-research controls (odds ratio = 3.54, p < .001; table S4). Conclusions The current study provides the first evidence of the long-term impacts of UREs on students from traditionally underrepresented groups using a multi-institutional, 10-year, longitudinal, propensity-score-matched design to compare URE students with a no-research control group of students. Our results showed that student participation in UREs was beneficial to their academic performance, scientific baccalaureate attainment, acceptance into a scientific graduate program, and longer-term scientific-workforce participation. However, the benefits varied as a function of the duration and intensity of the experience. Specifically, completing two or more semesters or summers of 10 or more hours per week in research emerged as the most beneficial undergraduate research experience. Furthermore, these data show that the students who only completed a single semester of high-intensity research or completed one or more semesters of low-intensity research did not differ from controls in their scientific-degree attainment or scientific-workforce participation rates. These findings solidify extant research on the overall benefits of UREs (Carter F et al. 2009, Rodenbusch et al. 2016) and most importantly reveal that longer durations of more intense experiences promote the most positive benefits of research experiences. As has been recommended in the National Academy report (NAS 2017), the design of this study addressed the methodological shortcomings of previous work by including an appropriate comparison group of students with no comparable apprenticeship-style undergraduate research experience. Although randomly assigning students to participate in one or more UREs is not feasible, a longitudinal, prospective study design that uses propensity scores to identify highly equivalent comparison groups allows us to say that prior motivation, academic performance, and demographic variables do not explain the URE effect. The results show that even when students are comparable on these key variables, having two or more semesters of high-intensity participation in UREs increases the likelihood of a student eventually entering the scientific workforce. It is important to note several caveats that limit the inferences we can draw from the present study. First, this study focuses exclusively on highly motivated and talented science students from underrepresented groups. Additional research is needed to better understand the degree to which the patterns observed with URM students parallel patterns among majority-group members. In addition, the current study measured self-reported variability in the quality and dosage of cocurricular, faculty-mentored URE(s) (figure S1). Objective measures of quality and dosage of URE participation would be of value, because some work has shown students may perceive a variety of nonresearch experiences as counting as engagement in research (Thiry et al. 2011). Therefore, the current study's operationalization of UREs, which likely includes a wide variety of types of URE experiences, may be a conservative estimate of the impact of UREs on workforce-related outcomes. In addition, our study was well designed to capture UREs occurring in junior and senior years but does not provide insight on how early-tenure (first and sophomore year) experiences affect scientific career interests or the likelihood of engaging in cocurricular UREs. Future studies should investigate the convergent validity of self-reported and objective measures of URE participation, the impact of early UREs on later UREs, and the impacts of the sequence of UREs on scientific career persistence. Another caveat concerns the present focus on behavioral workforce outcomes. Although the present outcomes are important from a scientific-workforce-development perspective, there are many other outcomes that may be equally or more important from an individual perspective, such as sense of belonging, personal development, or attainment of personal or communal values (Hunter et al. 2007, Diekman et al. 2010, Gibbs and Griffin 2013). In addition, the present study does not address the psychological mechanisms through which UREs produce workforce outcomes. Theoretical and empirical evidence suggest a feedback loop between experiences and psychological processes, such as science efficacy, identity, and values that lead to behavioral outcomes (Merolla et al. 2012, Graham et al. 2013). Although the present study provides unique longitudinal data to assess the main effects of URE participation on workforce-related outcomes, there is strong evidence that these UREs lead to both psychological (e.g., science identity) and behavioral outcomes (e.g., persistence; Bandura 1989, Lent et al. 1994). Future research investigating the underlying psychological mechanisms that describe why research experiences lead to engagement in scientific-workforce professions would be useful to persons developing UREs as well as researchers. In addition, future research investigating the similarities and differences in the underlying psychological mechanisms for URMs and majority-group members would advance the field. The current findings highlight opportunities to leverage and improve existing individual, programmatic, and institutional efforts to maximize the impact of research experiences on diversifying of the scientific workforce. Although half of our sample of science majors with a declared intention to pursue a scientific research career participated in a URE, only a third of our sample participated in a high-intensity URE during the school year or over the summer. This finding indicates untapped potential, as well as the need to investigate limitations in access to UREs. Although cocurricular, faculty-mentored UREs have severe limits in terms of cost and scalability (Linn et al. 2015), possible faculty and URE program managers may wish to encourage students currently engaged in shorter, less intensive undergraduate research experiences to move toward more intensive school year or summer experiences. In addition, the present findings are consistent with literature on the benefits of more institutionally supported and scalable multisemester course-based undergraduate research experiences (Jordan et al. 2014, Rodenbusch et al. 2016). Therefore, faculty, URE program directors, and institutional URE reform efforts may benefit from considering how to engage undergraduates in CURE-type intensive research experiences across the undergraduate program of study. In the end, just having a research experience is not enough—quantity and quality matter—encouraging two or more semesters or summers (or both), mentorship from a faculty member, and a commitment of 10 or more hours per week on the part of the student is critical to help students achieve their scientific career aspirations and broaden the scientific workforce. Acknowledgments Funding for this study was provided by a grant from the National Institute of General Medical Sciences (no. R01-GM075316) to the fourth author. We thank Jessica Schabow, AJ Castaneda, and Alyssa Victory for helping develop the figures for this manuscript. We also thank Dr. Erin Dolan for her feedback on a draft of this manuscript and Qi Zhi for help preparing this manuscript for publication. Supplemental material Supplementary data are available at BIOSCI online. Data available are from the Dryad Digital Repository at https://doi.org/10.5061/dryad.50m50. References cited Auchincloss L et al.   2014. Assessment of course-based undergraduate research experiences: A meeting report. CBE–Life Sciences Education  13: 29– 40. Google Scholar CrossRef Search ADS PubMed  Bandura A. 1989. Human agency in social cognitive theory. American Psychologist  44: 1175– 1184. Google Scholar CrossRef Search ADS PubMed  Carter D, Ro H, Alcott B, Lattuca L. 2016. Co-curricular connections: The role of undergraduate research experiences in promoting engineering students’ communication, teamwork, and leadership skills. Research in Higher Education  57: 363– 393. Google Scholar CrossRef Search ADS   Carter F, Mandell M, Maton K. 2009. The influence of on-campus, academic year undergraduate research on STEM PhD outcomes: Evidence from the Meyerhoff Scholarship Program. Educational Evaluation and Policy Analysis  31: 441– 462. Google Scholar CrossRef Search ADS PubMed  Cumming G, Finch S. 2005. Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist  60: 170– 180. Google Scholar CrossRef Search ADS PubMed  Diekman AB, Brown ER, Johnston AM, Clark EK. 2010. Seeking congruity between goals and roles: A new look at why women opt out of science, technology, engineering, and mathematics careers. Psychological Science  21: 1051– 1057. Google Scholar CrossRef Search ADS PubMed  Estrada M, Woodcock A, Hernandez PR, Schultz PW. 2011. Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology  103: 206– 222. Google Scholar CrossRef Search ADS PubMed  Eagan M Jr, S Hurtado, Chang M, Garcia G, Herrera F, Garibay J. 2013. Making a difference in science education: The impact of undergraduate research programs. American Educational Research Journal  50: 683– 713. Google Scholar CrossRef Search ADS PubMed  Fakayode S, Yakubu M, Adeyeye O, Pollard D, Mohammed A. 2014. Promoting undergraduate STEM education at a historically black college and university through research experience. Journal of Chemical Education  91: 662– 665. Google Scholar CrossRef Search ADS   Gibbs KD, Griffin KA. 2013. What do I want to be with my PhD? The roles of personal values and structural dynamics in shaping the career interests of recent biomedical science PhD graduates. CBE–Life Sciences Education  12: 711– 723. Google Scholar CrossRef Search ADS PubMed  Gilmore J, Vieyra M, Timmerman B, Feldon D, Maher M. 2015. The relationship between undergraduate research participation and subsequent research performance of early career STEM graduate students. Journal of Higher Education  86: 834– 863. Google Scholar CrossRef Search ADS   Graham M, Frederick J, Byars-Winston A, Hunter A, Handelsman J. 2013. Science education: Increasing persistence of college students in STEM. Science  341: 1455– 1456. Google Scholar CrossRef Search ADS PubMed  Hathaway R, Nagda B, Gregerman S. 2002. The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. Journal of College Student Development  43: 614– 631. Ho D, Imai K, King G, Stuart E. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis  15: 199– 236. Google Scholar CrossRef Search ADS   Ho D, Imai K, King G, Stuart E. 2011. MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software  42: 1– 28. Google Scholar CrossRef Search ADS   Hunter AB, Laursen SL, Seymour E. 2007. Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Science Education  91: 36– 74. Google Scholar CrossRef Search ADS   Jordan TC et al.   2014. A broadly implementable research course in phage discovery and genomics for first-year undergraduate students. mBio  5: 1– 8. Google Scholar CrossRef Search ADS   Kremer J, Bringle R. 1990. The effects of an intensive research experience on the careers of talented undergraduates. Journal of Research and Development in Education  24: 1– 5. Kuh G. 2008. High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter . Association of American Colleges and Universities. Lent RW, Brown SD, Hackett G. 1994. Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior  45: 79– 122. Google Scholar CrossRef Search ADS   Linn M, Palmer E, Baranger A, Gerard E, Stone E. 2015. Undergraduate research experiences: Impacts and opportunities. Science  347 (art. 1261757). Lopatto D. 2004. Survey of undergraduate research experiences (SURE): First findings. Cell Biology Education  3: 270– 277. Google Scholar CrossRef Search ADS PubMed  [LSAMP] Louis Stokes Alliances for Minority Participation. 2010. NSF STEM Classification of Instructional Programs Crosswalk. LSAMP . (10 August 2017; www.lsamp.org/help/help_stem_cip_2010.cfm) Merolla DM, Serpe RT, Stryker S, Schultz PW. 2012. Structural precursors to identity processes: The role of proximate social structures. Social Psychology Quarterly  75: 149– 172. Google Scholar CrossRef Search ADS   Mervis J. 2006. Biomedical training programs: NIH told to get serious about giving minorities a hand. Science  311: 328– 329. Google Scholar CrossRef Search ADS PubMed  Muthén LK, Muthén BO. 1998–2017. Mplus User's Guide, 8th ed . Muthén and Muthén. Nagda B, Gregerman S, Jonides J, von Hippel W, Lerner J. 1998. Undergraduate student–faculty research partnerships affect student retention. Review of Higher Education  22: 55– 72. Google Scholar CrossRef Search ADS   [NAS] National Academy of Sciences. 2007. Rising above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future . National Academies Press. [NAS] National Academy of Sciences. 2017. Undergraduate Research Experiences for STEM Students: Successes, Challenges, and Opportunities . National Academies Press. [NSF] National Science Foundation, National Center for Science and Engineering Statistics. 2017. Women, Minorities, and Persons with Disabilities in Science and Engineering: 2017. NSF . Special Report no. NSF 17–310. (19 December 2017; www.nsf.gov/statistics/wmpd) Olson S, Riordan D. 2012. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Executive Office of the President . Pan W, Bai H. 2015. Propensity Score Analysis: Fundamentals and Developments . Guilford. Rosenbaum P, Rubin D. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika  70: 41– 55. Google Scholar CrossRef Search ADS   Rodenbusch S, Hernandez PR, Simmons SL, Dolan EL. 2016. Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE–Life Sciences Education  15: 1– 10. Russell S, Hancock M, McCullough J. 2007. Benefits of undergraduate research experiences. Science  316: 548– 549. Google Scholar CrossRef Search ADS PubMed  Sadler T, Burgin S, McKinney L, Ponjuan L. 2010. Learning science through research apprenticeships: A critical review of the literature. Journal of Research in Science Teaching  47: 235– 256. Schultz PW, Hernandez PR, Woodcock A, Estrada M, Chance RC, Aguilar M, Serpe RT. 2011. Patching the pipeline: Reducing educational disparities in the sciences through minority training programs. Educational Evaluation and Policy Analysis  33: 95– 114. Google Scholar CrossRef Search ADS   Seymour E, Hunter A, Laursen S, DeAntoni T. 2004. Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a 3‐year study. Science Education  88: 493– 534. Google Scholar CrossRef Search ADS   Thiry H, Laursen SL, Hunter A-B. 2011. What experiences help students become scientists? A comparative study of research and other sources of personal and professional gains for STEM undergraduates. Journal of Higher Education  82: 357– 388. Thoemmes F. 2012. Propensity score matching in SPSS . Cornell University Library. (10 August 2017; https://arxiv.org/abs/1201.6385) [USDHHS] US Department of Health and Human Services. 2015. National Institute of General Medical Sciences 5-Year Strategic Plan . USDHHS. (10 August 2017; https://publications.nigms.nih.gov/strategicplan/NIGMS-strategic-plan.pdf) West S, Duan N, Pequegnat W, Gaist P, Des Jarlais D Holtgrave D, Szapocznik J, Fishbein M, Rapkin B, Clatts M. 2008. Alternatives to the randomized controlled trial. American Journal of Public Health  98: 1359– 1366. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of the American Institute of Biological Sciences. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png BioScience Oxford University Press

Undergraduate Research Experiences Broaden Diversity in the Scientific Workforce

Loading next page...
 
/lp/ou_press/undergraduate-research-experiences-broaden-diversity-in-the-scientific-6Zc8xqs5c7
Publisher
Oxford University Press
Copyright
© The Author(s) 2018. Published by Oxford University Press on behalf of the American Institute of Biological Sciences. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com
ISSN
0006-3568
eISSN
1525-3244
D.O.I.
10.1093/biosci/bix163
Publisher site
See Article on Publisher Site

Abstract

Abstract New data highlight the importance of undergraduate research experiences (UREs) for keeping underrepresented science students on the pathway to a scientific career. We used a large-scale, 10-year, longitudinal, multi-institutional, propensity-score-matched research design to compare the academic performance and persistence in science of students who participated in URE(s) with those of similar students who had no research experience. Our results showed that students who completed 10 or more hours of cocurricular, faculty-mentored research per week across two or more academic semesters or summers were significantly more likely to graduate with a science-related bachelor's degree, to be accepted into a science-related graduate training program, and to be training for or working in the scientific workforce 6 years after graduation. Importantly, the findings show that just having a URE was not enough to influence persistence in science; it required a commitment of 10 or more hours per week over two or more semesters of faculty-mentored research. Broadening participation in the US scientific workforce is essential for expanding economic prosperity and remaining internationally competitive (NAS 2007). Building a diverse, inclusive scientific workforce is the only way for the United States to meet national research goals and for science to benefit from that diverse workforce (USDHHS 2015). Recruiting and retaining interested women and minorities are critical strategies to expand participation in the scientific workforce (Olson and Riordan 2012). Despite decades of efforts to expand and diversify the scientific workforce, data show a clear pattern of continued underrepresentation by gender and race or ethnicity (NSF 2017). Undergraduate research experiences are increasingly seen as a cornerstone of efforts to retain students on a scientific career pathway (Russell et al. 2007, Graham et al. 2013, Linn et al. 2015, NAS 2017); however, the links between these undergraduate experiences and long-term scientific-workforce outcomes lack compelling empirical support, particularly for students from underrepresented minority groups (URMs). For the United States to generate a talented, diverse, and inclusive scientific workforce, it is vital to understand how well and under what conditions undergraduate research experiences broaden scientific-workforce participation (NAS 2017). Faculty-mentored undergraduate research experiences are foundational to many programs developed to encourage students, particularly those from historically underrepresented groups, to pursue a scientific career (Graham et al. 2013). Undergraduate research experiences vary widely in their goals, in the students they attract, and in their structural elements (Seymour et al. 2004, Linn et al. 2015, NAS 2017), but these experiences typically occur in one of two settings: (1) cocurricular, apprenticeship-style, mentored research experiences in a faculty laboratory (UREs) or (2) course-based research experiences (CUREs; Sadler et al. 2010, Linn et al. 2015). CUREs represent an institutionalized strategy to update traditional science lab courses with curriculum that requires students to learn science by engaging in authentic scientific practices and scientific discovery that are of interest to the scientific community (Auchincloss et al. 2014, Rodenbusch et al. 2016). CUREs are open to many students and pair numerous undergraduate researchers with one or more research mentors across the duration of an academic term (Linn et al. 2015). UREs also engage undergraduates in authentic scientific practices and discovery, but they do so in an apprentice-style, mentored relationship with a faculty member (Seymour et al. 2004, Sadler et al. 2010). Unlike CUREs, UREs are typically competitive and limited to a smaller number of undergraduates. UREs are costly in terms of time and money; however, these experiences are widely accepted as “high-impact educational practices,” which improve academic performance and persistence in science (Russell et al. 2007, Kuh 2008, Graham et al. 2013). The present study responds to repeated calls for data on student participation in UREs, with a focus on the short-, medium-, and long-term benefits of apprentice-style, faculty-mentored UREs as opposed to those of CUREs. Mentored UREs are intended to enhance students’ interest and excitement in research, build confidence in research, cultivate teamwork and leadership skills, and increase scientific knowledge and an understanding of the nature of science and scientific practices (Lopatto 2004, Russell et al. 2007, Fakayode et al. 2014, Gilmore et al. 2015, Linn et al. 2015, Carter D et al. 2016). The distal goals for UREs are to support and crystalize student intentions to pursue a scientific career, to increase trainees’ determination to apply for graduate training, and to broaden participation in the scientific workforce (Hathaway et al. 2002, Lopatto 2004, Mervis 2006, Estrada et al. 2011)—a goal not realized for many years after the undergraduate experience. Claims about the benefits of UREs on academic performance and persistence abound but are yet to be fully substantiated by rigorously designed empirical studies. With few exceptions, claims of positive benefits are undermined by a reliance on retrospective accounts, studies with inadequate comparison groups, inadequate measurement of outcomes, and a focus on short-term rather than long-term follow-up to determine the positive effects on academic and career trajectories (Mervis 2006, Sadler et al. 2010, Linn et al. 2015). Recent reports recognize the limitations of this body of research to support causal claims about the effects of UREs (Seymour et al. 2004, Sadler et al. 2010, Linn et al. 2015, NAS 2017). For example, without adequate comparison groups, it is unclear whether the benefits of UREs are simply attributable to the type of student who seeks out a research experience rather than to the experience itself (Linn et al. 2015). If URE students have higher academic performance and are more motivated compared with those who do not participate, then arguably, the students engaging in UREs are more likely to join the scientific workforce regardless of URE participation (Carter D et al. 2016). A few studies have begun to show the potential benefits of UREs on some short- and medium-term outcomes. For example, the University of Michigan's evaluation of the Undergraduate Research Opportunity Program (UROP) used administrative records to demonstrate some benefits of the program on persistence in college among its African American UROP participants compared with African American controls (Nagda et al. 1998). Similarly, data from two longitudinal, multi-institutional studies have shown that UREs promote student aspirations to pursue a scientific career. The first, using the Higher Education Research Institute's national data, revealed that undergraduates in science training programs, which include UREs, were 15% more likely to aspire to science-oriented graduate school compared with matched undergraduates not in science training programs (Eagan et al. 2013). The second comes from our own longitudinal work with science students from underrepresented groups, which concluded that URE participants maintained higher aspirations to pursue a scientific research career compared with matched no-research controls (Schultz et al. 2011). Although this research provides some evidence of the impacts of UREs, it does not include compelling evidence for longer-term outcomes. Furthermore, these studies are limited in that they typically ignore research outside of formal URE programs (Nagda et al. 1998); rely on retrospective accounts of undergraduate experiences (Hathaway et al. 2002); and fail to measure longer-term outcomes, such as scientific-degree attainment, matriculation into a scientific graduate program, or postgraduation scientific workforce participation of URE participants, compared with those of an adequate control group (Nagda et al. 1998, Hathaway et al. 2002, Schultz et al. 2011, Eagan et al. 2013). In addition, previous studies have not fully demonstrated the salient features of UREs that might enhance these outcomes, although preliminary evidence hints at the importance of the duration (number of semesters or summers in research) and intensity (number of hours of research per week; Kremer and Bringle 1990, Carter F et al. 2009) of these experiences. A quantitative investigation of the long-term benefits of UREs, as well as of the key features of UREs, is crucial to establishing their efficacy and maximizing the potential impact of these resource-intensive experiences and is in full alignment with the conclusions of the recent National Academy report that calls for studies that improve the evidence base about the processes and effects of undergraduate research experiences. This report specifically calls for “well-designed studies” conducted in collaboration with program directors (NAS 2017). A further recommendation from the National Academy report is that study attributes include the acquisition of longitudinal, prospective, empirical data that can inform the planning and opportunities to improve quality and access of UREs (NAS 2017). The present study responds to these recommendations. Study overview To empirically test the benefits of UREs, we conducted a national-propensity-score-matched (PSM) study that followed a panel of URM undergraduate science students over 10 years. Prior studies have documented evidence that UREs sustain interest in advanced scientific training (e.g., graduate applications) and scientific research careers (Seymour et al. 2004, Estrada et al. 2011, Schultz et al. 2011). Therefore, we extended the literature by examining how the intensity and duration of UREs influenced attainment of a baccalaureate degree, cumulative grade point average (GPA) at graduation, acceptance into a scientific graduate program, and participation in the scientific workforce. The sample was composed of predominantly African American, black and Hispanic, or Latino(a) science majors from 29 US colleges across 10 years, beginning in their junior year. We used multiple methods to measure student experiences and outcomes (i.e., prospective surveys and National Student Clearinghouse records), enabling us to draw conclusions regarding academic outcomes and longer-term persistence in science (see the supplemental materials). The goal of the study was to test the impact of participation, duration, and intensity of UREs from their junior year through the summer following senior year on academic performance and scientific-workforce participation 6 years after graduation (supplemental tables S1–S3, supplemental figure S1). The participants The overall sample consisted of 1420 URM undergraduate and graduate students majoring in a science-related discipline, recruited from 50 different institutions of higher ­education across the United States. At the time of recruitment into the study (i.e., 2005, 2006, or 2007), the undergraduate participants were in their first (12%), sophomore (16%), junior (27%), or senior (33%) year of college. However, for the present study, we focused on undergraduates recruited into the study in or prior to their junior year in college (n = 774); who were majoring in biological science (e.g., ­biology), physical science (e.g., chemistry), engineering, or mathematics (n = 710); and who had completed all questions on the matching survey (n = 693). The analytic sample consisted of 577 PSM URM students (nURE = 284 and nNo-research = 293) from 29 different colleges or universities; the details of the propensity score matching are described in procedures below. Eighty-three percent of the URE group had at least one close-matched no-research student counterpart. At the time of recruitment into the study, the participants were in their early 20s and were predominantly female, were African American or Latino(a), spoke English as a first language, and were majoring in a biological or physical science. In addition, many were first-generation college students, and most attended institutions classified as low research intensity on the Carnegie classification system (tables S1 and S2). Procedure Starting in the fall semester of 2005, URM students majoring in science- and biomedical-related disciplines were recruited from universities and colleges across the United States. Students were recruited by soliciting participants from science training programs, such as the National Institutes of Health's (NIH’s) Research Initiative for Scientific Enhancement (RISE) program, as well as from gateway science and mathematics courses (e.g., organic chemistry). The participants completed biannual (fall and spring) online surveys starting in the fall of 2005 (baseline survey) through the spring of 2017 (most recent follow-up survey). The prospective design avoided retrospective biases by regularly gathering data from the participants during their undergraduate tenure. The participants received nominal compensation for their participation ($20). The per-survey response rates have been between 70% and 85%. After 10 years, we remained in contact with 98% of the panel through email and/or phone calls. Propensity score matching Quasi-experiments are often used in place of randomized controlled trials when randomization procedures are not feasible. However, it is well understood that selection bias can occur when attributes related to assignment to the treatment group are also correlated with outcomes of interest (Rosenbaum and Rubin 1983, West et al. 2008). In order to control for selection bias, we constructed a matched sample of students who had engaged in URE(s) and a similar sample of students who had not engaged in research during their undergraduate tenure. To create matched URE and no-research groups, PSM procedures were used to calculate the probability that a student ever engaged in a URE on the basis of 17 covariates measured in the baseline matching survey using MatchIT software (Ho et al. 2007, 2011, Thoemmes 2012, Pan and Bai 2015). After matching students with and without research experiences, the URE group and no-research control group exhibited an acceptable balance on all covariates, and the estimate of selection bias dropped by 97% (table S2, supplemental figure S2). Measures Measurement of student experiences and achievements took place over a 10-year period (2005–2014). Research experiences (including intensity and duration) and GPA were prospectively measured during each student's undergraduate tenure (i.e., junior year, summer before senior year, senior year, and summer after senior year). Although students were recruited into the study at different stages in their undergraduate tenure (e.g., sophomore or junior), junior- and senior-year research experiences typically occurred in the date range of 2006–2008. Attainment of a science bachelor's degree, acceptance into graduate school, and engagement in a scientific career were prospectively measured via surveys and were confirmed with data from the National Student Clearinghouse on multiple occasions (latest was spring 2014). The following paragraphs provide a general description and details on the operationalization of the variables used in this study. Engagement in an undergraduate research experience In each survey, the students were asked whether they had “worked in laboratory in current college/university” in the current semester or term (0, no; 1, yes). The students were also asked about participation in off-campus summer research experiences by asking whether they had “worked in laboratory at another college/university” or had “worked on research at another location” (0, no; 1, yes). A dummy-coded “ever engaged in one or more URE(s)” variable (0, no; 1, yes) was derived from the student responses starting in spring semester of junior year through the summer following senior year. Intensity of the URE(s) If the students indicated they had engaged in research activities during the school year or over the summer, they were asked the follow-up question, “Please indicate approximately how many hours per week you worked on research.” The students reported the approximate numbers of hours working on research separately for “worked in laboratory in current college/university,” “worked in laboratory at another college/university,” and “worked on research at another location.” The response scale was open so that students could report any number of hours. The student responses were recoded to indicate whether they had engaged in a low-intensity URE (less than 10 hours per week) or a high-intensity URE (10 or more hours per week) that semester or term (table S3). Duration of low- and high-intensity URE(s) Two summary indices were created to tally the total number of semesters the students reported being involved in low- or high-intensity UREs (i.e., junior spring, summer before senior year, senior fall, senior spring, and summer following senior year). The first index was the sum of low-intensity UREs. The low-intensity index was recoded into two dummy-coded variables to indicate only one semester of low-intensity URE or two or more semesters of low-intensity URE (the reference group was the no-research group). An identical procedure was followed to create a summary index of high-intensity UREs, as well as two dummy-coded indicators of only one semester of high-intensity URE or two or more semesters of high-intensity URE (the reference group was the no-research group; table S3). Attainment of a baccalaureate degree in a scientific discipline Across survey waves, when each student indicated that they had completed a baccalaureate degree, they were asked to report the year, the institution, the type of degree (e.g., BA or BS), and their major. Data from the National Student Clearinghouse were used to confirm the self-reported degree attainment. We used the National Science Foundation list of recognized science-oriented majors (LSAMP 2010) to classify degree award by discipline (0, nonscience; 1, science; table S3 note). Cumulative GPA at graduation Students reported their “current cumulative college GPA” on each survey. The GPA aligned with the semester of graduation used as the final GPA at graduation (table S3). Acceptance into a science-related graduate program On each survey, students were asked, “In the last 6 months, have you been accepted into any graduate schools?” (0, no; 1, yes). If students reported yes, they were asked the follow-up question, “How many were science-related graduate programs?” (open-ended response format). Data from the National Student Clearinghouse were used to confirm graduate-school enrollment. The data were recoded into a dummy-coded variable indicating whether they had ever been accepted into a science-related graduate program (0, no; 1, yes; table S3). Scientific-workforce participation (postbaccalaureate) Scientific-workforce participation was measured using the participants’ self-reported data and enrollment data from the National Student Clearinghouse in fall 2013 and spring 2014. The students no longer enrolled in a college were asked to describe their primary occupation, including job title, and whether they considered it to be a science-related occupation. The students still enrolled in a college (e.g., graduate school or postdoctoral study) were asked about their current field of study. Their responses were used to create a dummy-coded variable that indicated scientific-workforce participation as either an employee or student (coded 0, nonscience; 1, science related). Science-related careers included being enrolled in a degree program (e.g., graduate school); pursuing a science-related degree (e.g., biology, chemistry, computer science, or neuropsychology); or holding a science-related occupation, such as science-related scholar or teacher (e.g., assistant professor or lecturer) or lab scientist (e.g., scientist II or blood-lab supervisor). The nonscience list included being enrolled in a medical or clinical degree program (e.g., clinical, counseling, or school psychology; dental school; medical school; nursing; occupational therapy; or public health), holding a medical or clinical occupation, being enrolled in other degree programs (e.g., business, education, social science, or social work), holding an occupation in a nonscience and nonmedical field (e.g., office manager or law enforcement), or being nonenrolled and unemployed (table S3). Analysis overview The data were analyzed using weighted multilevel regression. PSM-generated sampling weights were used to account for unequal sample sizes in the matched groups, and a two-level multilevel analysis was used to account for the nesting of students within schools. Four dummy-coded predictor variables were used to represent the duration and intensity of UREs (one semester low intensity, two or more semesters low intensity, one semester high intensity, two or more semesters high intensity; the reference group was the no-research group for all dummy-coded predictor variables). All of our analyses were conducted using maximum likelihood with robust standard errors estimation in Mplus v7.4 (Muthén and Muthén 1998–2017). Results Across the five semesters and summers from junior year, almost half of our sample engaged in at least one URE, and 32% of the students participated in at least one high-intensity experience (figure S1). Long-term, high-intensity UREs (10 or more hours per week across two or more semesters) had the greatest impact on student outcomes. Grade point average Students with long-term, high-intensity UREs had GPAs 0.13 units higher than those of the students in the matched no-research control group (β = 0.33, p = .03; MGPA 3.37 and 3.24, respectively; supplemental table S4). Earning a baccalaureate degree in a scientific discipline The odds of earning a baccalaureate degree in the sciences were more than nine times greater for students with long-term, high-intensity UREs compared with those for the matched no-research controls (odds ratio = 9.64, p < .001; table S4). More specifically, the odds of earning a science bachelor's degree for the no-research control group were 3.30:1, whereas the odds for those with long-term, high-intensity UREs were 31.82:1. To make these findings more concrete, predicted values derived from the multilevel models were used to show the probability (probability = odds/[1+odds]) of earning a scientific degree as a function of the duration and intensity of the URE (figure 1a). The 95% confidence intervals around each predicted value provide complementary information to the estimates in table S4 and allow for evaluating statistical inference by eye (Cumming and Finch 2005). For example, the confidence intervals around the point estimate for the students with no research (white bar) did not include the point estimate for the students with long-term, high-intensity engagement in UREs (dark gray bar with angled lines), indicating a statistically significant difference between these two point estimates (p < .05). Note that the confidence intervals around the point estimates from logistic regression are asymmetrical because of the logit transformation and the asymmetric scale of odds ratios. Figure 1. View largeDownload slide The probability of graduating with a science degree (n = 388; 1a), being accepted into a science-related graduate program (n = 577; 1b), and scientific career engagement (n = 393; 1c) as a function of the duration and intensity of undergraduate research experiences (UREs). The predicted values were computed from PSM-weighted multilevel logistic regression models with dummy-coded variables indicating duration of low-intensity and duration of high-intensity UREs. The multilevel-modeling-derived point estimates and confidence intervals account for the nesting of students within colleges and universities. The error bars represent 95% confidence intervals. Figure 1. View largeDownload slide The probability of graduating with a science degree (n = 388; 1a), being accepted into a science-related graduate program (n = 577; 1b), and scientific career engagement (n = 393; 1c) as a function of the duration and intensity of undergraduate research experiences (UREs). The predicted values were computed from PSM-weighted multilevel logistic regression models with dummy-coded variables indicating duration of low-intensity and duration of high-intensity UREs. The multilevel-modeling-derived point estimates and confidence intervals account for the nesting of students within colleges and universities. The error bars represent 95% confidence intervals. Acceptance into a graduate program As we show in figure 1b, the odds of being accepted into a science-related graduate program were four times greater for students with long-term, high-intensity UREs compared with those for matched no-research controls (odds ratio = 4.23, p < .001; table S4). Students with one semester of high-intensity or two or more semesters of low-intensity UREs were also more likely to be accepted into a science-related graduate program compared with the matched no-research controls. Working as a scientist Speaking directly to the impact of UREs on the scientific workforce, figure 1c shows that the odds of being engaged in a scientific career or advanced scientific career training 6 years after graduation were more than three times greater for students with long-term, high-intensity UREs compared with those for the matched no-research controls (odds ratio = 3.54, p < .001; table S4). Conclusions The current study provides the first evidence of the long-term impacts of UREs on students from traditionally underrepresented groups using a multi-institutional, 10-year, longitudinal, propensity-score-matched design to compare URE students with a no-research control group of students. Our results showed that student participation in UREs was beneficial to their academic performance, scientific baccalaureate attainment, acceptance into a scientific graduate program, and longer-term scientific-workforce participation. However, the benefits varied as a function of the duration and intensity of the experience. Specifically, completing two or more semesters or summers of 10 or more hours per week in research emerged as the most beneficial undergraduate research experience. Furthermore, these data show that the students who only completed a single semester of high-intensity research or completed one or more semesters of low-intensity research did not differ from controls in their scientific-degree attainment or scientific-workforce participation rates. These findings solidify extant research on the overall benefits of UREs (Carter F et al. 2009, Rodenbusch et al. 2016) and most importantly reveal that longer durations of more intense experiences promote the most positive benefits of research experiences. As has been recommended in the National Academy report (NAS 2017), the design of this study addressed the methodological shortcomings of previous work by including an appropriate comparison group of students with no comparable apprenticeship-style undergraduate research experience. Although randomly assigning students to participate in one or more UREs is not feasible, a longitudinal, prospective study design that uses propensity scores to identify highly equivalent comparison groups allows us to say that prior motivation, academic performance, and demographic variables do not explain the URE effect. The results show that even when students are comparable on these key variables, having two or more semesters of high-intensity participation in UREs increases the likelihood of a student eventually entering the scientific workforce. It is important to note several caveats that limit the inferences we can draw from the present study. First, this study focuses exclusively on highly motivated and talented science students from underrepresented groups. Additional research is needed to better understand the degree to which the patterns observed with URM students parallel patterns among majority-group members. In addition, the current study measured self-reported variability in the quality and dosage of cocurricular, faculty-mentored URE(s) (figure S1). Objective measures of quality and dosage of URE participation would be of value, because some work has shown students may perceive a variety of nonresearch experiences as counting as engagement in research (Thiry et al. 2011). Therefore, the current study's operationalization of UREs, which likely includes a wide variety of types of URE experiences, may be a conservative estimate of the impact of UREs on workforce-related outcomes. In addition, our study was well designed to capture UREs occurring in junior and senior years but does not provide insight on how early-tenure (first and sophomore year) experiences affect scientific career interests or the likelihood of engaging in cocurricular UREs. Future studies should investigate the convergent validity of self-reported and objective measures of URE participation, the impact of early UREs on later UREs, and the impacts of the sequence of UREs on scientific career persistence. Another caveat concerns the present focus on behavioral workforce outcomes. Although the present outcomes are important from a scientific-workforce-development perspective, there are many other outcomes that may be equally or more important from an individual perspective, such as sense of belonging, personal development, or attainment of personal or communal values (Hunter et al. 2007, Diekman et al. 2010, Gibbs and Griffin 2013). In addition, the present study does not address the psychological mechanisms through which UREs produce workforce outcomes. Theoretical and empirical evidence suggest a feedback loop between experiences and psychological processes, such as science efficacy, identity, and values that lead to behavioral outcomes (Merolla et al. 2012, Graham et al. 2013). Although the present study provides unique longitudinal data to assess the main effects of URE participation on workforce-related outcomes, there is strong evidence that these UREs lead to both psychological (e.g., science identity) and behavioral outcomes (e.g., persistence; Bandura 1989, Lent et al. 1994). Future research investigating the underlying psychological mechanisms that describe why research experiences lead to engagement in scientific-workforce professions would be useful to persons developing UREs as well as researchers. In addition, future research investigating the similarities and differences in the underlying psychological mechanisms for URMs and majority-group members would advance the field. The current findings highlight opportunities to leverage and improve existing individual, programmatic, and institutional efforts to maximize the impact of research experiences on diversifying of the scientific workforce. Although half of our sample of science majors with a declared intention to pursue a scientific research career participated in a URE, only a third of our sample participated in a high-intensity URE during the school year or over the summer. This finding indicates untapped potential, as well as the need to investigate limitations in access to UREs. Although cocurricular, faculty-mentored UREs have severe limits in terms of cost and scalability (Linn et al. 2015), possible faculty and URE program managers may wish to encourage students currently engaged in shorter, less intensive undergraduate research experiences to move toward more intensive school year or summer experiences. In addition, the present findings are consistent with literature on the benefits of more institutionally supported and scalable multisemester course-based undergraduate research experiences (Jordan et al. 2014, Rodenbusch et al. 2016). Therefore, faculty, URE program directors, and institutional URE reform efforts may benefit from considering how to engage undergraduates in CURE-type intensive research experiences across the undergraduate program of study. In the end, just having a research experience is not enough—quantity and quality matter—encouraging two or more semesters or summers (or both), mentorship from a faculty member, and a commitment of 10 or more hours per week on the part of the student is critical to help students achieve their scientific career aspirations and broaden the scientific workforce. Acknowledgments Funding for this study was provided by a grant from the National Institute of General Medical Sciences (no. R01-GM075316) to the fourth author. We thank Jessica Schabow, AJ Castaneda, and Alyssa Victory for helping develop the figures for this manuscript. We also thank Dr. Erin Dolan for her feedback on a draft of this manuscript and Qi Zhi for help preparing this manuscript for publication. Supplemental material Supplementary data are available at BIOSCI online. Data available are from the Dryad Digital Repository at https://doi.org/10.5061/dryad.50m50. References cited Auchincloss L et al.   2014. Assessment of course-based undergraduate research experiences: A meeting report. CBE–Life Sciences Education  13: 29– 40. Google Scholar CrossRef Search ADS PubMed  Bandura A. 1989. Human agency in social cognitive theory. American Psychologist  44: 1175– 1184. Google Scholar CrossRef Search ADS PubMed  Carter D, Ro H, Alcott B, Lattuca L. 2016. Co-curricular connections: The role of undergraduate research experiences in promoting engineering students’ communication, teamwork, and leadership skills. Research in Higher Education  57: 363– 393. Google Scholar CrossRef Search ADS   Carter F, Mandell M, Maton K. 2009. The influence of on-campus, academic year undergraduate research on STEM PhD outcomes: Evidence from the Meyerhoff Scholarship Program. Educational Evaluation and Policy Analysis  31: 441– 462. Google Scholar CrossRef Search ADS PubMed  Cumming G, Finch S. 2005. Inference by eye: Confidence intervals and how to read pictures of data. American Psychologist  60: 170– 180. Google Scholar CrossRef Search ADS PubMed  Diekman AB, Brown ER, Johnston AM, Clark EK. 2010. Seeking congruity between goals and roles: A new look at why women opt out of science, technology, engineering, and mathematics careers. Psychological Science  21: 1051– 1057. Google Scholar CrossRef Search ADS PubMed  Estrada M, Woodcock A, Hernandez PR, Schultz PW. 2011. Toward a model of social influence that explains minority student integration into the scientific community. Journal of Educational Psychology  103: 206– 222. Google Scholar CrossRef Search ADS PubMed  Eagan M Jr, S Hurtado, Chang M, Garcia G, Herrera F, Garibay J. 2013. Making a difference in science education: The impact of undergraduate research programs. American Educational Research Journal  50: 683– 713. Google Scholar CrossRef Search ADS PubMed  Fakayode S, Yakubu M, Adeyeye O, Pollard D, Mohammed A. 2014. Promoting undergraduate STEM education at a historically black college and university through research experience. Journal of Chemical Education  91: 662– 665. Google Scholar CrossRef Search ADS   Gibbs KD, Griffin KA. 2013. What do I want to be with my PhD? The roles of personal values and structural dynamics in shaping the career interests of recent biomedical science PhD graduates. CBE–Life Sciences Education  12: 711– 723. Google Scholar CrossRef Search ADS PubMed  Gilmore J, Vieyra M, Timmerman B, Feldon D, Maher M. 2015. The relationship between undergraduate research participation and subsequent research performance of early career STEM graduate students. Journal of Higher Education  86: 834– 863. Google Scholar CrossRef Search ADS   Graham M, Frederick J, Byars-Winston A, Hunter A, Handelsman J. 2013. Science education: Increasing persistence of college students in STEM. Science  341: 1455– 1456. Google Scholar CrossRef Search ADS PubMed  Hathaway R, Nagda B, Gregerman S. 2002. The relationship of undergraduate research participation to graduate and professional education pursuit: An empirical study. Journal of College Student Development  43: 614– 631. Ho D, Imai K, King G, Stuart E. 2007. Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis  15: 199– 236. Google Scholar CrossRef Search ADS   Ho D, Imai K, King G, Stuart E. 2011. MatchIt: Nonparametric preprocessing for parametric causal inference. Journal of Statistical Software  42: 1– 28. Google Scholar CrossRef Search ADS   Hunter AB, Laursen SL, Seymour E. 2007. Becoming a scientist: The role of undergraduate research in students’ cognitive, personal, and professional development. Science Education  91: 36– 74. Google Scholar CrossRef Search ADS   Jordan TC et al.   2014. A broadly implementable research course in phage discovery and genomics for first-year undergraduate students. mBio  5: 1– 8. Google Scholar CrossRef Search ADS   Kremer J, Bringle R. 1990. The effects of an intensive research experience on the careers of talented undergraduates. Journal of Research and Development in Education  24: 1– 5. Kuh G. 2008. High-Impact Educational Practices: What They Are, Who Has Access to Them, and Why They Matter . Association of American Colleges and Universities. Lent RW, Brown SD, Hackett G. 1994. Toward a unifying social cognitive theory of career and academic interest, choice, and performance. Journal of Vocational Behavior  45: 79– 122. Google Scholar CrossRef Search ADS   Linn M, Palmer E, Baranger A, Gerard E, Stone E. 2015. Undergraduate research experiences: Impacts and opportunities. Science  347 (art. 1261757). Lopatto D. 2004. Survey of undergraduate research experiences (SURE): First findings. Cell Biology Education  3: 270– 277. Google Scholar CrossRef Search ADS PubMed  [LSAMP] Louis Stokes Alliances for Minority Participation. 2010. NSF STEM Classification of Instructional Programs Crosswalk. LSAMP . (10 August 2017; www.lsamp.org/help/help_stem_cip_2010.cfm) Merolla DM, Serpe RT, Stryker S, Schultz PW. 2012. Structural precursors to identity processes: The role of proximate social structures. Social Psychology Quarterly  75: 149– 172. Google Scholar CrossRef Search ADS   Mervis J. 2006. Biomedical training programs: NIH told to get serious about giving minorities a hand. Science  311: 328– 329. Google Scholar CrossRef Search ADS PubMed  Muthén LK, Muthén BO. 1998–2017. Mplus User's Guide, 8th ed . Muthén and Muthén. Nagda B, Gregerman S, Jonides J, von Hippel W, Lerner J. 1998. Undergraduate student–faculty research partnerships affect student retention. Review of Higher Education  22: 55– 72. Google Scholar CrossRef Search ADS   [NAS] National Academy of Sciences. 2007. Rising above the Gathering Storm: Energizing and Employing America for a Brighter Economic Future . National Academies Press. [NAS] National Academy of Sciences. 2017. Undergraduate Research Experiences for STEM Students: Successes, Challenges, and Opportunities . National Academies Press. [NSF] National Science Foundation, National Center for Science and Engineering Statistics. 2017. Women, Minorities, and Persons with Disabilities in Science and Engineering: 2017. NSF . Special Report no. NSF 17–310. (19 December 2017; www.nsf.gov/statistics/wmpd) Olson S, Riordan D. 2012. Engage to Excel: Producing One Million Additional College Graduates with Degrees in Science, Technology, Engineering, and Mathematics. Executive Office of the President . Pan W, Bai H. 2015. Propensity Score Analysis: Fundamentals and Developments . Guilford. Rosenbaum P, Rubin D. 1983. The central role of the propensity score in observational studies for causal effects. Biometrika  70: 41– 55. Google Scholar CrossRef Search ADS   Rodenbusch S, Hernandez PR, Simmons SL, Dolan EL. 2016. Early engagement in course-based research increases graduation rates and completion of science, engineering, and mathematics degrees. CBE–Life Sciences Education  15: 1– 10. Russell S, Hancock M, McCullough J. 2007. Benefits of undergraduate research experiences. Science  316: 548– 549. Google Scholar CrossRef Search ADS PubMed  Sadler T, Burgin S, McKinney L, Ponjuan L. 2010. Learning science through research apprenticeships: A critical review of the literature. Journal of Research in Science Teaching  47: 235– 256. Schultz PW, Hernandez PR, Woodcock A, Estrada M, Chance RC, Aguilar M, Serpe RT. 2011. Patching the pipeline: Reducing educational disparities in the sciences through minority training programs. Educational Evaluation and Policy Analysis  33: 95– 114. Google Scholar CrossRef Search ADS   Seymour E, Hunter A, Laursen S, DeAntoni T. 2004. Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a 3‐year study. Science Education  88: 493– 534. Google Scholar CrossRef Search ADS   Thiry H, Laursen SL, Hunter A-B. 2011. What experiences help students become scientists? A comparative study of research and other sources of personal and professional gains for STEM undergraduates. Journal of Higher Education  82: 357– 388. Thoemmes F. 2012. Propensity score matching in SPSS . Cornell University Library. (10 August 2017; https://arxiv.org/abs/1201.6385) [USDHHS] US Department of Health and Human Services. 2015. National Institute of General Medical Sciences 5-Year Strategic Plan . USDHHS. (10 August 2017; https://publications.nigms.nih.gov/strategicplan/NIGMS-strategic-plan.pdf) West S, Duan N, Pequegnat W, Gaist P, Des Jarlais D Holtgrave D, Szapocznik J, Fishbein M, Rapkin B, Clatts M. 2008. Alternatives to the randomized controlled trial. American Journal of Public Health  98: 1359– 1366. Google Scholar CrossRef Search ADS PubMed  © The Author(s) 2018. Published by Oxford University Press on behalf of the American Institute of Biological Sciences. All rights reserved. For permissions, please e-mail: journals.permissions@oup.com

Journal

BioScienceOxford University Press

Published: Mar 1, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off