An Analysis of Publication Productivity During Residency for 1506 Neurosurgical Residents and 117 Residency Departments in North America

An Analysis of Publication Productivity During Residency for 1506 Neurosurgical Residents and 117... Abstract BACKGROUND Bibliometrics is defined as the study of statistical and mathematical methods used to quantitatively analyze scientific literature. The application of bibliometrics in neurosurgery continues to evolve. OBJECTIVE To calculate a number of publication productivity measures for almost all neurosurgical residents and departments within North America. These measures were correlated with survey results on the educational environment within residency programs. METHODS During May to June 2017, data were collected from departmental websites and Scopus to compose a bibliometric database of neurosurgical residents and residency programs. Data related to authorship value and study content were collected on all articles published by residents. A survey of residency program research and educational environment was administered to program directors and coordinators; results were compared with resident academic productivity. RESULTS The median number of publications in residency was 3; median h-index and Resident index were 1 and 0.17 during residency, respectively. There was a statistically significant difference in academic productivity among male neurosurgical residents compared with females. The majority of articles published were tier 1 clinical articles. Residency program research support was significantly associated with increased resident productivity (P < .001). Scholarly activity requirements were not associated with increased resident academic productivity. CONCLUSION This study represents the most comprehensive bibliometric assessment of neurosurgical resident academic productivity during training to date. New benchmarks for individual and department academic productivity are provided. A supportive research environment for neurosurgical residents is associated with increased academic productivity, but a scholarly activity requirement was, surprisingly, not shown to have a positive effect. Resident, Bibliometrics, h-index, Res-index, m-quotient, Academic neurosurgery, Citations, Scopus ABBREVIATIONS ABBREVIATIONS ACGME Accreditation Council for Graduate Medical Education AV authorship value Res-index Resident index Interest in bibliometrics is growing throughout academic medicine,1-5 and more recently in academic neurosurgery.6-15 In our prior studies, we have compared the strengths and weaknesses of bibliometric indices and databases.7,8 Additionally, we have established detailed bibliometric profiles of nearly all academic faculty and departments within neurosurgery.6,16,17 Neurosurgical resident academic productivity has recently been subject to bibliometric analysis.18,19 Neurosurgical resident publication productivity during training years has been linked with academic positions and future promotions to professorship and chairmanship.20 Sarkiss et al19 recently assessed 1352 neurosurgical residents across 105 Accreditation Council for Graduate Medical Education (ACGME) residency programs. They performed a subspecialty analysis and tabulated the number of first author publications during training and provided overall resident bibliometrics indices and benchmarks. TABLE 1. Three-Tier System to Classify Clinical Research Performed During Residency Training Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research View Large TABLE 1. Three-Tier System to Classify Clinical Research Performed During Residency Training Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research View Large The primary objective of our study was to capture intraresidency academic productivity as measured using well-established bibliometric indices and databases for all neurosurgery residents currently in training across North America. The secondary objective was to evaluate the different aspects of a residency program's educational environment that could influence in-training academic productivity. METHODS Selection of Programs A listing of the 2017 neurosurgery residency-training programs was obtained from neurosurgerymatch.org (http://www.neurosurgerymatch.org/ResidencyProgram.php). Departmental websites were consulted for resident names and postgraduate training years. All departments were then contacted individually by telephone or email to answer questions about the residency-training environment. Any information not obtained from the websites regarding resident names and residency start dates was clarified by contacting the department. Bibliometric profiles were created for all residents using Scopus (http://www.scopus.com; Elsevier, Amsterdam, Netherlands) during June and July of 2017. Scopus was selected based on our previous work that demonstrated it to be more accurate in individual bibliometric calculations compared with rival databases.7,8 Definition of Overall Citation Metrics h-index21 The h-index is defined as an individual having h papers with at least h citations. For instance, an author with 10 publications cited 10 times would have an h-index of 10. It is an attempt to link quantity (number of publications) with impact (number of citations). This index has become the most popular of the many bibliometric indices used in academic medicine. Our prior work describes its use in neurosurgery in detail.7,8,22 m-quotient21 The m-quotient is the h-index divided by the number of years since the author's first publication. Definition of In-Residency Citation Metrics h-indexR Calculated using only articles published since the individual resident began their residency program. m-quotientR The m-quotientR is the h-index divided by the number of years since the author's first publication during residency. Authorship Value8 The authorship value (AV) quotient was calculated by using a weighted average as follows: AV = (first author × 1.0 + second author × 0.5 + middle author × 0.25 + last author × 0.5) The AV is a means to highlight the well-known fact that contribution to a neurosurgical publication is, in general, reflected by the authorship position.23 While the assigned values may be controversial (as is the topic of authorship in general24,25), they were selected in an effort to stratify contribution, particularly of the first 3 authorship positions, as we felt it would be rare for a resident to be a senior author. Resident index This is the m-quotientR multiplied by the AV. This index incorporates features of the h-index (quantity and impact) adjusted for years in training and degree of participation. Calculation of Citation Metrics by Citation Database The automated h-index from Scopus was obtained using the “Author Search” function. The m-quotient was calculated by dividing Scopus's h-index by the years since the first publication. A manual h-index and m-quotient were calculated based on resident start dates. Authors’ first and last initials were used within search strings. Careful examination of the results from each search was performed to determine if the author had a preference on how they listed their initials. Article Type Each article published during residency was evaluated further to establish if it was a clinical or basic science/translational article. A 3-tier system was devised to further assess the type of clinical research being performed during residency training (Table 1). Educational Environment All program coordinators or directors were contacted (if available) to assess the total number of residents in training, the presence and amount of protected time for research in training, any research requirements by the training department, and whether the department had funded research, ancillary research support in the form of a medical editor, and whether they provide financial support to residents who wished to attend meetings. Programs that had any clinical duties during protected research time were considered as having “incompletely protected” research time. A “research requirement” was defined as a program dictum stipulating that a resident be a coauthor on one or more publication, conference presentation, or book chapter during their residency as a direct result of participating in what the ACGME calls “scholarly activity.” Ranking The metric used to rank intraresidency productivity was the Resident index (Res-index; summed and corrected for number of residents within the program). Statistical Analysis The following a priori statistical comparisons were performed (median values): Male residents compared with female. Programs with and without certain research attributes: dedicated research time, research requirement, funding, financial support for academic travel, and medical editor. Osteopathic programs compared with allopathic training programs. Correlate ranking of programs by resident productivity (summed and mean Res-index) to program ranking (summed h-index6 and ih[5] adjusted for number of faculty16). All statistics were performed using SPSS (version 21, IBM, Armonk, New York). Significant values were considered to be P < .05. Tests for normality were performed using the Shapiro–Wilk test. Nonparametric tests included the Mann–Whitney U and Kruskal–Wallis test to compare groups of data that did not follow the normal distribution. RESULTS As of July 2017, we identified 1506 residents (1253 male residents [83%] and 253 [17%] female) in training at 117 separate programs across North America. The median number of residents per program was 2 per year (range, every other year-4). The total number of publications and number of intraresidency publications authored by these 1506 residents were 13 847 (9.2/resident) and 9486 (6.3/resident), respectively. The types of intraresidency articles are shown in Table 2. The majority of articles were tier 1 and tier 2 clinical articles (63%). TABLE 2. Types of Publications Being Published by Neurosurgical Residents in Residency Training Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 View Large TABLE 2. Types of Publications Being Published by Neurosurgical Residents in Residency Training Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 View Large Table 3 describes the overall publication metrics for the 1506 neurosurgical residents. The citation counts, publication counts, and bibliometric indices were found to be positively skewed and did not follow the normal distribution (Shapiro–Wilk, P < .001). The median values (all publications/intraresidency publications) for publication count, h-index, and m-quotient were 5/3, 2/1, and 0.5/0.33, respectively. The median AV and Res-index for in-residency productivity was 0.5 and 0.17, respectively. TABLE 3. Overall Publication Metrics for Neurosurgical Residents (Total and During Residency). Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a aStatistically significant (P < .05). All statistical tests are Kruskal–Wallis. View Large TABLE 3. Overall Publication Metrics for Neurosurgical Residents (Total and During Residency). Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a aStatistically significant (P < .05). All statistical tests are Kruskal–Wallis. View Large The results of the survey investigating the educational environment and infrastructure of programs as it pertains to research are presented in Table 4. Complete responses were obtained from 94 programs (80%). The vast majority of programs have dedicated research time (90%), with most having a research requirement (58%), funded research (75%), and a travel budget for residents (82%). However, less than a third of programs had a medical editor (29%). TABLE 4. Survey Results (n = 94, 80%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) View Large TABLE 4. Survey Results (n = 94, 80%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) View Large Comparisons Male vs Female Male residents outperformed their female counterparts in all metrics except for m-quotient for all publications and the AV (Table 3). Program Attributes Programs that had protected research time (either completely or incompletely) had more productive residents across all metrics (Table 5). There was, however, no difference in the productivity among programs with a research or scholarly activity requirement and those that did not. Established research funding and travel support also resulted in greater productivity across all metrics except for AV. Access to a medical editor also resulted in increased resident output. TABLE 5. Program Educational Environment and Infrastructure by Bibliometric and Productivity Measures Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a aStatistically significant result (P < .05). All statistical tests are Mann–Whitney U. View Large TABLE 5. Program Educational Environment and Infrastructure by Bibliometric and Productivity Measures Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a aStatistically significant result (P < .05). All statistical tests are Mann–Whitney U. View Large Program Type Allopathic programs (ie, MD) outperformed osteopathic programs (ie, DO) in all bibliometric indices (Mann–Whitney U, P < .001; Table 6). TABLE 6. Bibliometric Indices by Program Type Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a aStatistically significant (P < .05). All statistical tests are Mann–Whitney U. View Large TABLE 6. Bibliometric Indices by Program Type Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a aStatistically significant (P < .05). All statistical tests are Mann–Whitney U. View Large Program Ranking Programs were ranked according to their summed Res-index and mean Res-index (Table 7). The absolute median rank-change between the summed Res-index and mean Res-index rankings was 9. There was good correlation between the 2 different rankings with 77% of the variation in the mean Res-index being explained by the summed Res-index (Figure 1). The summed Res-index rankings and mean Res-index rankings were compared with faculty program summed h-index rankings from 2014 and mean ih(5) faculty corrected rankings from 2016, respectively.6,16 There was reasonable correlation between the summed h-index faculty program ranking (from our 2014 publication) and the summed Res-index program ranking (Figure 2, R2 = 0.61).6 However, there was poor correlation (R2 = 0.04) when comparing mean Res-index rankings and of the corrected ih(5) ranking (from our 2016 publication; scatterplot not shown).16 FIGURE 1. View largeDownload slide Rank mean R-index vs rank summed R-index. FIGURE 1. View largeDownload slide Rank mean R-index vs rank summed R-index. FIGURE 2. View largeDownload slide Summed h-index faculty program rankings6 vs summed Res-index program ranking. FIGURE 2. View largeDownload slide Summed h-index faculty program rankings6 vs summed Res-index program ranking. TABLE 7. Departmental Rankings by Using the Summation of R-Indices for Entire Departments Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A aThe second number is the change in rank. A positive number represents an increase in rank; a negative number a decrease. The faculty rank by summed h-index is compared to resident rank by summed Res-index; faculty rank by adjusted ih(5) index is compared to resident by average Res-index. View Large TABLE 7. Departmental Rankings by Using the Summation of R-Indices for Entire Departments Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A aThe second number is the change in rank. A positive number represents an increase in rank; a negative number a decrease. The faculty rank by summed h-index is compared to resident rank by summed Res-index; faculty rank by adjusted ih(5) index is compared to resident by average Res-index. View Large DISCUSSION The evaluation of research by neurosurgical residents has recently come to attention in the literature and is undergoing further development.18,19 Academic productivity of neurosurgical trainees has been cited as one of the most predictive measures of future academic advancement within neurosurgery.20,26 Daniels et al27 found that publication count and h-index during residency correlated to an academic career, but not publication count prior to residency. Yang et al28 also found a positive correlation between resident publication output and residents selecting an academic career, whereas Patterson et al29 found no association between pre and intraresidency publications. In contrast to the recent article by Sarkiss et al,19 we counted only publications published during training years in our bibliometric calculations. In our opinion, the work done prior to residency may not necessarily reflect the educational environment and productivity during residency years. We have also introduced the Res-index, a metric that is normalized by postgraduate year (m-quotient), resident involvement (AV), quantity (number of publications), and impact (number of citations). Individual Neurosurgical Resident Productivity We found that the median number of overall and intraresidency publications was 5 and 3, respectively. Median intraresidency h-index was 1 and m-quotient was 0.33, which is translates to 1 publication cited once every 3 yrs. We further refined this by measuring the average AV and multiplying this by the m-quotientR to identify a new index—the Res-index. The median Res-index was 0.17. The majority of articles published by neurosurgical residents were classified as tier 1 clinical articles (ie, case reports, case series, and secondary research). There were significantly higher metrics for trainees in allopathic programs compared with osteopathic programs (Table 6). Gender Comparison Much research has been done on research productivity by gender within various medical specialties, including neurosurgery.30-38 Most publications report that men outperform women in number of publications and bibliometrics, namely the h-index. Research disparity based on gender has been identified in academic neurosurgery faculty, but appears to correct when adjusting for academic rank.6,38 In otolaryngology, Eloy et al31,39 have suggested that the post-training research productivity curve is different for females, with women producing less research at the beginning of their career, but equaling or exceeding men at senior levels. Geltzeiler et al40 recently evaluated a cohort of 358 colorectal surgeons and found that despite having a shorter research career duration, females did not differ from their male counterparts in publications or bibliometric indices. We found that male neurosurgical residents had statistically significant higher indices compared with female residents except for AV and m-quotient (Table 3) and that this difference persisted even when adjusting for postgraduate training level (m-quotientR, Res-index). In other words, whereas post-training output may initially be lower for females, it appears that female residents as a whole publish less than their male counterparts throughout the course of their residency. In one of the few other attempts to quantify intraresidency academic productivity, Yang et al28 analyzed a total of 459 male (84.5%) and 84 female (15.5%) urology residents and found women produced fewer total publications (average 3.0 vs 4.8, P = .01) and fewer as first author (average 1.8 vs 2.5, P = .03) than men. As with many other aspects of neurosurgical training, mentorship and the availability of role models for female residents likely plays a large role in this discrepancy and further research is needed.41,42 Impact of Educational Environment and Academic Productivity The results of the survey administered to program coordinators were correlated to the academic productivity of residents in our cohort. This evaluation showed that there was a highly significant positive correlation among programs that have protected research time, research funding, medical editing support, and travel support. Taken together, these pieces help create a supportive environment, coupled with positive sustained mentorship, that values resident research. A study by Williams et al43 evaluated 1690 orthopedic surgery residents and found a significant association between dedicated research time and number of resident publications. Within urology, Finkelstein et al44 and Yang et al45 both demonstrated greater resident output with dedicated research time, which in turn correlated with increased postresidency publishing and the pursuit of fellowship training. A surprising finding was that having a “research requirement” (or more accurately a “scholarly activity requirement” based on our ACGME-based definition) did not correlate to increased academic productivity. This may have several explanations. Programs that are highly productive but do not have a scholarly activity requirement presumably have a well-established culture of academic curiosity combined with allotted research time and departmental support. Conversely, some programs with a scholarly activity requirement may not have developed such an environment yet or do not have the means or willingness to monitor and enforce their research requirement(s), which could effectively nullify the requirement. In our program, implementing a research requirement was necessary to change our culture, but will hopefully be discarded once the cultural change has become “hardwired” into our residency. Departmental Neurosurgical Resident Productivity Table 7 provides a ranking of all 117 neurosurgical departments included in this analysis. The summed Res-index favors larger departments but also credits programs that are training more residents who are consistently academically productive. The mean Res-index ranking, on the other hand, removes some of the emphasis on programs with more residents. There was good correlation (R2 = 0.77) between summed Res-index rankings and mean Res-index rankings (Figure 1). This demonstrates that even when removing the sum component that favors larger programs, the rankings still had good correlation. This could be due to larger programs having more established research programs and productive residency applicants being more attracted to these programs. There was also good correlation between the summed h-index faculty ranking from 2014 and the summed Res-index rankings (R2 = 0.61, Figure 2); however, there was poor correlation between our mean of corrected ih(5) faculty rankings from 2016 and our mean Res-index rankings. This could mean that groups of productive residents are attracted to programs with larger numbers of historically productive faculty, but who may not necessarily be as productive over the last 5 yrs. Additionally, this could signify that productive residents are attracted to larger programs with more stability as evidenced by larger numbers of academically productive faculty. When the emphasis of size and historical stature of the program is removed, the association with academically productive residents diminishes. Ranking by mean Res-index diverge from the recent average resident h-index ranking published recently by Sarkiss et al.19 The median absolute rank change was 16 (range, 0-61), and the correlation coefficient between the 2 rankings was R2 = 0.50. This could be an indication that the academic output of trainees during residency is influenced by factors other than preresidency scholarly activity. Limitations The first limitation is that of citation analysis. Publications that receive citations do not necessarily correlate with article quality. In our opinion, citations (and citation analyses) are a reflection of journal readership and interest (either positive or negative) in the medical literature. There are limitations to the data that are indexed in Scopus for citation analysis. We chose to use Scopus based on prior work that showed that it provided more accurate results for individual analysis.6-8 The data included in our analysis were obtained from departmental websites. As such, data integrity is dependent on having a website that contains accurate and up-to-date information. Furthermore, we may have made incorrect assumptions based on the information found on the website. For example, a resident's gender may have been incorrectly assigned based on their name, but this particular error would likely have been a rare event as many websites have pictures of residents. We also contacted individual programs for any clarification on information present on their departmental websites. Overall, we feel that the degree of inaccurate or missing data is quite small and would be unlikely to impact the results of our study. Program size (ie, number of residents) is a major potential imbalance when comparing resident productivity between programs. Smaller programs would be more affected by the loss of a resident (eg, quitting or terminated) than larger programs. We recognized this and adjusted our Res-index ranking for number of residents (mean Res-index). Finally, preresidency research that was published during residency would have been counted with our methodology; conversely, intraresidency research published after training would not. We assumed that these 2 scenarios effectively cancelled each other out. CONCLUSION This study represents the most comprehensive bibliometric assessment of neurosurgical resident academic productivity during training to date. New benchmarks for individual and department academic productivity are provided and a new index, Res-index, is introduced. Our results show that supportive research environments for neurosurgical residents are associated with increased academic productivity. Interestingly, scholarly activity requirements were not shown to have positively impact resident academic productivity. Disclosure The authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article. REFERENCES 1. Pagel PS , Hudetz JA . An analysis of scholarly productivity in United States academic anaesthesiologists by citation bibliometrics . Anaesthesia . 2011 ; 66 ( 10 ): 873 - 878 . Google Scholar CrossRef Search ADS PubMed 2. Rad AE , Brinjikji W , Cloft HJ , Kallmes DF . The H-index in academic radiology . Acad Radiol . 2010 ; 17 ( 7 ): 817 - 821 . Google Scholar CrossRef Search ADS PubMed 3. Svider PF , Choudhry ZA , Choudhry OJ , Baredes S , Liu JK , Eloy JA . The use of the h-index in academic otolaryngology . Laryngoscope . 2013 ; 123 ( 1 ): 103 - 106 . Google Scholar CrossRef Search ADS PubMed 4. Turaga KK , Gamblin TC . Measuring the surgical academic output of an institution: the “institutional” H-index . J Surg Educ . 2012 ; 69 ( 4 ): 499 - 503 . Google Scholar CrossRef Search ADS PubMed 5. Pagni M , Khan NR , Cohen HL , Choudhri AF . Highly cited works in radiology . Acad Radiol . 2014 ; 21 ( 8 ): 1056 - 1066 . Google Scholar CrossRef Search ADS PubMed 6. Khan NR , Thompson CJ , Taylor DR et al. An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States . J Neurosurg . 2014 ; 120 ( 3 ): 746 - 755 . Google Scholar CrossRef Search ADS PubMed 7. Khan N , Thompson CJ , Choudhri AF , Boop FA , Klimo P Jr. Part I: the application of the h-Index to groups of individuals and departments in academic neurosurgery . World Neurosurg . 2013 ; 80 ( 6 ): 759 - 765.e3 . Google Scholar CrossRef Search ADS PubMed 8. Khan NR , Thompson CJ , Taylor DR et al. Part II: Should the h-index be modified? An analysis of the m-quotient, contemporary h-index, authorship value, and impact factor . World Neurosurg . 2013 ; 80 ( 6 ): 766 - 774 . Google Scholar CrossRef Search ADS PubMed 9. Spearman CM , Quigley MJ , Quigley MR , Wilberger JE . Survey of the h index for all of academic neurosurgery: another power-law phenomenon ? J Neurosurg . 2010 ; 113 ( 5 ): 929 - 933 . Google Scholar CrossRef Search ADS PubMed 10. Aoun SG , Bendok BR , Rahme RJ , Dacey RG Jr , Batjer HH . Standardizing the evaluation of scientific and academic performance in neurosurgery–critical review of the “h” index and its variants . World Neurosurg . 2013 ; 80 ( 5 ): e85 - e90 . Google Scholar CrossRef Search ADS PubMed 11. Wilcox MA , Khan NR , McAbee JH , Boop FA , Klimo P Jr. Highly cited publications in pediatric neurosurgery . Childs Nerv Syst . 2013 ; 29 ( 12 ): 2201 - 2213 . Google Scholar CrossRef Search ADS PubMed 12. Venable GT , Khan NR , Taylor DR , Thompson CJ , Michael LM , Klimo P Jr. A correlation between NIH funding and bibliometrics in neurosurgery . World Neurosurg . 2013 ; 81 ( 3-4 ): 468 - 472 . Google Scholar CrossRef Search ADS PubMed 13. Lee J , Kraus KL , Couldwell WT . Use of the h index in neurosurgery. Clinical article . J Neurosurg . 2009 ; 111 ( 2 ): 387 - 392 . Google Scholar CrossRef Search ADS PubMed 14. Ponce FA , Lozano AM . Highly cited works in neurosurgery. Part I: the 100 top-cited papers in neurosurgical journals . J Neurosurg . 2010 ; 112 ( 2 ): 223 - 232 . Google Scholar CrossRef Search ADS PubMed 15. Ponce FA , Lozano AM . Highly cited works in neurosurgery. Part II: the citation classics . J Neurosurg . 2010 ; 112 ( 2 ): 233 - 246 . Google Scholar CrossRef Search ADS PubMed 16. Lee RP , Venable GT , Roberts ML et al. Five-year institutional bibliometric profiles for 119 North American neurosurgical residency programs: an update . World Neurosurg . 2016 ; 95 : 565 - 575 . Google Scholar CrossRef Search ADS PubMed 17. Taylor DR , Venable GT , Jones GM et al. Five-year institutional bibliometric profiles for 103 US neurosurgical residency programs . J Neurosurg . 2015 ; 123 ( 3 ): 547 - 560 . Google Scholar CrossRef Search ADS PubMed 18. Kashkoush A , Prabhu AV , Tonetti D , Agarwal N . The neurosurgery match: a bibliometric analysis of 206 first-year residents . World Neurosurg . 2017 ; 105 : 341 - 347 . Google Scholar CrossRef Search ADS PubMed 19. Sarkiss CA , Riley KJ , Hernandez CM et al. Academic productivity of US neurosurgery residents as measured by H-index: program ranking with correlation to faculty productivity . Neurosurgery . 2017 ; 80 ( 6 ): 975 - 984 . Google Scholar CrossRef Search ADS PubMed 20. Crowley RW , Asthagiri AR , Starke RM et al. In-training factors predictive of choosing and sustaining a productive academic career path in neurological surgery . Neurosurgery . 2012 ; 70 ( 4 ): 1024 - 1032 . Google Scholar CrossRef Search ADS PubMed 21. Hirsch JE . An index to quantify an individual's scientific research output . Proc Natl Acad Sci U S A . 2005 ; 102 ( 46 ): 16569 - 16572 . Google Scholar CrossRef Search ADS PubMed 22. Khan NR , Thompson CJ , Taylor DR et al. An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States . [Epub ahead of print Dec 20 2013] J Neurosurg . 2014 ; 120 ( 3 ): 746 - 55 . doi: 10.3171/2013.11.JNS131708 . Google Scholar CrossRef Search ADS PubMed 23. Romanovsky AA . Revised h index for biomedical research . Cell Cycle . 2012 ; 11 ( 22 ): 4118 - 4121 . Google Scholar CrossRef Search ADS PubMed 24. Bhandari M , Guyatt GH , Kulkarni AV et al. Perceptions of authors' contributions are influenced by both byline order and designation of corresponding author . J Clin Epidemiol . 2014 ; 67 ( 9 ): 1049 - 1054 . Google Scholar CrossRef Search ADS PubMed 25. Sauermann H , Haeussler C . Authorship and contribution disclosures . Sci Adv. 2017 ; 3 ( 11 ): 1 - 13 . Google Scholar CrossRef Search ADS 26. Khan NR , Klimo P Jr , Michael LM 2nd . Predicting academic career placement: neurosurgery's quest for the holy grail . World Neurosurg . 2017 ; 101 : 746 - 747 . Google Scholar CrossRef Search ADS PubMed 27. Daniels M , Garzon-Muvdi T , Maxwell R et al. Preresidency publication number does not predict academic career placement in neurosurgery . [Epub ahead of print Feb 16 2017] World Neurosurg . 2017 ; 101 : 350 - 356 . doi: 10.1016/j.wneu.2017.02.028 . Google Scholar CrossRef Search ADS PubMed 28. Yang G , Villalta JD , Weiss DA , Carroll PR , Breyer BN . Gender differences in academic productivity and academic career choice among urology residents . J Urol . 2012 ; 188 ( 4 ): 1286 - 1290 . Google Scholar CrossRef Search ADS PubMed 29. Patterson SK , Fitzgerald JT , Boyse TD , Cohan RH . Is Past Academic Productivity Predictive of Radiology Resident Academic Productivity? Acad Radiol . 2002 ; 9 ( 2 ): 211 - 216 . Google Scholar CrossRef Search ADS PubMed 30. Diamond SJ , Thomas CR Jr , Desai S et al. Gender differences in publication productivity, academic rank, and career duration among U.S. academic gastroenterology faculty . Acad Med . 2016 ; 91 ( 8 ): 1158 - 1163 . Google Scholar CrossRef Search ADS PubMed 31. Eloy JA , Svider P , Chandrasekhar SS et al. Gender disparities in scholarly productivity within academic otolaryngology departments . Otolaryngol Head Neck Surg . 2013 ; 148 ( 2 ): 215 - 222 . Google Scholar CrossRef Search ADS PubMed 32. Hill EK , Blake RA , Emerson JB et al. Gender differences in scholarly productivity within academic gynecologic oncology departments . Obstet Gynecol . 2015 ; 126 ( 6 ): 1279 - 1284 . Google Scholar CrossRef Search ADS PubMed 33. Holliday EB , Jagsi R , Wilson LD , Choi M , Thomas CR Jr , Fuller CD . Gender differences in publication productivity, academic position, career duration, and funding among U.S. academic radiation oncology faculty . Acad Med . 2014 ; 89 ( 5 ): 767 - 773 . Google Scholar CrossRef Search ADS PubMed 34. Klimo P Jr , Venable GT , Khan NR et al. Bibliometric evaluation of pediatric neurosurgery in North America . J Neurosurg Pediat . 2014 ; 14 ( 6 ): 695 - 703 . Google Scholar CrossRef Search ADS 35. Lopez SA , Svider PF , Misra P , Bhagat N , Langer PD , Eloy JA . Gender differences in promotion and scholarly impact: an analysis of 1460 academic ophthalmologists . J Surg Educ . 2014 ; 71 ( 6 ): 851 - 859 . Google Scholar CrossRef Search ADS PubMed 36. Paik AM , Mady LJ , Villanueva NL et al. Research productivity and gender disparities: a look at academic plastic surgery . J Surg Educ . 2014 ; 71 ( 4 ): 593 - 600 . Google Scholar CrossRef Search ADS PubMed 37. Pashkova AA , Svider PF , Chang CY , Diaz L , Eloy JA , Eloy JD . Gender disparity among US anaesthesiologists: are women underrepresented in academic ranks and scholarly productivity? Acta Anaesthesiol Scand . 2013 ; 57 ( 8 ): 1058 - 1064 . Google Scholar CrossRef Search ADS PubMed 38. Tomei KL , Nahass MM , Husain Q et al. A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery . J Clin Neurosci . 2014 ; 21 ( 7 ): 1102 - 1105 . Google Scholar CrossRef Search ADS PubMed 39. Eloy JA , Svider PF , Cherla DV et al. Gender disparities in research productivity among 9952 academic physicians . Laryngoscope . 2013 ; 123 ( 8 ): 1865 - 1875 . Google Scholar CrossRef Search ADS PubMed 40. Geltzeiler CB , Kelley KA , Srikanth P et al. Does sex influence publication productivity among colorectal surgeons participating in fellowship training programs? Dis Colon Rectum . 2017 ; 60 ( 5 ): 537 - 543 . Google Scholar CrossRef Search ADS PubMed 41. Khan NR , Rialon KL , Buretta KJ , Deslauriers JR , Harwood JL , Jardine DA . Residents as mentors: the development of resident mentorship milestones . J Grad Med Educ . 2017 ; 9 ( 4 ): 551 - 554 . Google Scholar CrossRef Search ADS PubMed 42. Khan S , Ferrada P . Empowering surgical residents as mentors: a pilot program for female medical students . Am Surg . 2016 ; 82 ( 11 ): 313 - 314 . Google Scholar PubMed 43. Williams BR , Agel JA , Van Heest AE . Protected time for research during orthopaedic residency correlates with an increased number of resident publications . J Bone Joint Surg. 2017 ; 99 ( 13 ): 1 - 6 . Google Scholar CrossRef Search ADS 44. Finkelstein JB , Van Batavia JP , Rosoff JS . The difference a year can make: academic productivity of residents in 5 vs 6-year urology programs . Urology . 2015 ; 86 ( 2 ): 220 - 223 . Google Scholar CrossRef Search ADS PubMed 45. Yang G , Zaid UB , Erickson BA , Blaschko SD , Carroll PR , Breyer BN . Urology resident publication output and its relationship to future academic achievement . J Urol . 2011 ; 185 ( 2 ): 642 - 646 . Google Scholar CrossRef Search ADS PubMed Acknowledgments The authors wish to thank Andrew J. Gienapp, BA (Department of Medical Education, Methodist University Hospital, Memphis, TN and Department of Neurosurgery, University of Tennessee Health Science Center, Memphis, TN) for technical and copy editing, preparation of the manuscript and figures for publishing, and publication assistance with this manuscript. COMMENT In this report, the authors expand upon their previous work, generating a database of neurosurgery residents and their academic productivity using data from Scopus and surveys to training programs. Individual resident h-indices are calculated reviewing papers published during residency, along with several derived values, including a proposed Res-index, which considers the resident's duration in residency, timing since first publication, and authorship value. While the authors' model carries some methodological limitations, it serves as a thoughtful attempt to quantify resident productivity. In addition to providing a “batting average” of neurosurgical programs, the authors more notably explore factors that appear to be associated with productivity in residency. Importantly, they find that protected research time, research funding, travel support, and presence of a medical editor are each associated with resident productivity, while having a general scholarly activity requirement is not. Certainly, there is growing interest in tracking resident and faculty productivity in neurosurgery, especially in light of ACGME requirements for scholarly activity within residency programs. Beyond providing a snapshot of contemporary neurosurgical scholarship, papers like this one may also foster healthy competition, which, in turn, can inspire greater productivity. It is important to note, however, that evaluating bibliometrics alone does not capture the full spectrum of scholarly productivity; a host of other activities including teaching, presentations, and advocacy work are also germane. Nonetheless, publication productivity is one measurable marker, and the findings from this report carry potential significance for program directors and residents in highlighting the importance of research resources as a key requisite for publication. Darian R. Esfahani Sepideh Amin-Hanjani Chicago, Illinois Copyright © 2018 by the Congress of Neurological Surgeons This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices) http://www.deepdyve.com/assets/images/DeepDyve-Logo-lg.png Neurosurgery Oxford University Press

Loading next page...
 
/lp/ou_press/an-analysis-of-publication-productivity-during-residency-for-1506-2OEUDReMMn
Publisher
Congress of Neurological Surgeons
Copyright
Copyright © 2018 by the Congress of Neurological Surgeons
ISSN
0148-396X
eISSN
1524-4040
D.O.I.
10.1093/neuros/nyy217
Publisher site
See Article on Publisher Site

Abstract

Abstract BACKGROUND Bibliometrics is defined as the study of statistical and mathematical methods used to quantitatively analyze scientific literature. The application of bibliometrics in neurosurgery continues to evolve. OBJECTIVE To calculate a number of publication productivity measures for almost all neurosurgical residents and departments within North America. These measures were correlated with survey results on the educational environment within residency programs. METHODS During May to June 2017, data were collected from departmental websites and Scopus to compose a bibliometric database of neurosurgical residents and residency programs. Data related to authorship value and study content were collected on all articles published by residents. A survey of residency program research and educational environment was administered to program directors and coordinators; results were compared with resident academic productivity. RESULTS The median number of publications in residency was 3; median h-index and Resident index were 1 and 0.17 during residency, respectively. There was a statistically significant difference in academic productivity among male neurosurgical residents compared with females. The majority of articles published were tier 1 clinical articles. Residency program research support was significantly associated with increased resident productivity (P < .001). Scholarly activity requirements were not associated with increased resident academic productivity. CONCLUSION This study represents the most comprehensive bibliometric assessment of neurosurgical resident academic productivity during training to date. New benchmarks for individual and department academic productivity are provided. A supportive research environment for neurosurgical residents is associated with increased academic productivity, but a scholarly activity requirement was, surprisingly, not shown to have a positive effect. Resident, Bibliometrics, h-index, Res-index, m-quotient, Academic neurosurgery, Citations, Scopus ABBREVIATIONS ABBREVIATIONS ACGME Accreditation Council for Graduate Medical Education AV authorship value Res-index Resident index Interest in bibliometrics is growing throughout academic medicine,1-5 and more recently in academic neurosurgery.6-15 In our prior studies, we have compared the strengths and weaknesses of bibliometric indices and databases.7,8 Additionally, we have established detailed bibliometric profiles of nearly all academic faculty and departments within neurosurgery.6,16,17 Neurosurgical resident academic productivity has recently been subject to bibliometric analysis.18,19 Neurosurgical resident publication productivity during training years has been linked with academic positions and future promotions to professorship and chairmanship.20 Sarkiss et al19 recently assessed 1352 neurosurgical residents across 105 Accreditation Council for Graduate Medical Education (ACGME) residency programs. They performed a subspecialty analysis and tabulated the number of first author publications during training and provided overall resident bibliometrics indices and benchmarks. TABLE 1. Three-Tier System to Classify Clinical Research Performed During Residency Training Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research View Large TABLE 1. Three-Tier System to Classify Clinical Research Performed During Residency Training Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research Tier 1 Tier 2 Tier 3 Case reports Uncontrolled case series (>10 patients) Primary research (prospective and retrospective cohort, controlled studies) Case series (less than 10 patients) Secondary research with meta-analysis Randomized trials Review articles, secondary research without meta-analysis Industry Sponsored multicenter research Non industry sponsored multicenter research View Large The primary objective of our study was to capture intraresidency academic productivity as measured using well-established bibliometric indices and databases for all neurosurgery residents currently in training across North America. The secondary objective was to evaluate the different aspects of a residency program's educational environment that could influence in-training academic productivity. METHODS Selection of Programs A listing of the 2017 neurosurgery residency-training programs was obtained from neurosurgerymatch.org (http://www.neurosurgerymatch.org/ResidencyProgram.php). Departmental websites were consulted for resident names and postgraduate training years. All departments were then contacted individually by telephone or email to answer questions about the residency-training environment. Any information not obtained from the websites regarding resident names and residency start dates was clarified by contacting the department. Bibliometric profiles were created for all residents using Scopus (http://www.scopus.com; Elsevier, Amsterdam, Netherlands) during June and July of 2017. Scopus was selected based on our previous work that demonstrated it to be more accurate in individual bibliometric calculations compared with rival databases.7,8 Definition of Overall Citation Metrics h-index21 The h-index is defined as an individual having h papers with at least h citations. For instance, an author with 10 publications cited 10 times would have an h-index of 10. It is an attempt to link quantity (number of publications) with impact (number of citations). This index has become the most popular of the many bibliometric indices used in academic medicine. Our prior work describes its use in neurosurgery in detail.7,8,22 m-quotient21 The m-quotient is the h-index divided by the number of years since the author's first publication. Definition of In-Residency Citation Metrics h-indexR Calculated using only articles published since the individual resident began their residency program. m-quotientR The m-quotientR is the h-index divided by the number of years since the author's first publication during residency. Authorship Value8 The authorship value (AV) quotient was calculated by using a weighted average as follows: AV = (first author × 1.0 + second author × 0.5 + middle author × 0.25 + last author × 0.5) The AV is a means to highlight the well-known fact that contribution to a neurosurgical publication is, in general, reflected by the authorship position.23 While the assigned values may be controversial (as is the topic of authorship in general24,25), they were selected in an effort to stratify contribution, particularly of the first 3 authorship positions, as we felt it would be rare for a resident to be a senior author. Resident index This is the m-quotientR multiplied by the AV. This index incorporates features of the h-index (quantity and impact) adjusted for years in training and degree of participation. Calculation of Citation Metrics by Citation Database The automated h-index from Scopus was obtained using the “Author Search” function. The m-quotient was calculated by dividing Scopus's h-index by the years since the first publication. A manual h-index and m-quotient were calculated based on resident start dates. Authors’ first and last initials were used within search strings. Careful examination of the results from each search was performed to determine if the author had a preference on how they listed their initials. Article Type Each article published during residency was evaluated further to establish if it was a clinical or basic science/translational article. A 3-tier system was devised to further assess the type of clinical research being performed during residency training (Table 1). Educational Environment All program coordinators or directors were contacted (if available) to assess the total number of residents in training, the presence and amount of protected time for research in training, any research requirements by the training department, and whether the department had funded research, ancillary research support in the form of a medical editor, and whether they provide financial support to residents who wished to attend meetings. Programs that had any clinical duties during protected research time were considered as having “incompletely protected” research time. A “research requirement” was defined as a program dictum stipulating that a resident be a coauthor on one or more publication, conference presentation, or book chapter during their residency as a direct result of participating in what the ACGME calls “scholarly activity.” Ranking The metric used to rank intraresidency productivity was the Resident index (Res-index; summed and corrected for number of residents within the program). Statistical Analysis The following a priori statistical comparisons were performed (median values): Male residents compared with female. Programs with and without certain research attributes: dedicated research time, research requirement, funding, financial support for academic travel, and medical editor. Osteopathic programs compared with allopathic training programs. Correlate ranking of programs by resident productivity (summed and mean Res-index) to program ranking (summed h-index6 and ih[5] adjusted for number of faculty16). All statistics were performed using SPSS (version 21, IBM, Armonk, New York). Significant values were considered to be P < .05. Tests for normality were performed using the Shapiro–Wilk test. Nonparametric tests included the Mann–Whitney U and Kruskal–Wallis test to compare groups of data that did not follow the normal distribution. RESULTS As of July 2017, we identified 1506 residents (1253 male residents [83%] and 253 [17%] female) in training at 117 separate programs across North America. The median number of residents per program was 2 per year (range, every other year-4). The total number of publications and number of intraresidency publications authored by these 1506 residents were 13 847 (9.2/resident) and 9486 (6.3/resident), respectively. The types of intraresidency articles are shown in Table 2. The majority of articles were tier 1 and tier 2 clinical articles (63%). TABLE 2. Types of Publications Being Published by Neurosurgical Residents in Residency Training Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 View Large TABLE 2. Types of Publications Being Published by Neurosurgical Residents in Residency Training Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 Article type Intraresidency (%) Basic science 1017 (10) Clinical 7879 (79)  Tier 1 3889 (39)  Tier 2 2438 (24)  Tier 3 1552 (16) Editorials 1111 (11) Total 10 007 View Large Table 3 describes the overall publication metrics for the 1506 neurosurgical residents. The citation counts, publication counts, and bibliometric indices were found to be positively skewed and did not follow the normal distribution (Shapiro–Wilk, P < .001). The median values (all publications/intraresidency publications) for publication count, h-index, and m-quotient were 5/3, 2/1, and 0.5/0.33, respectively. The median AV and Res-index for in-residency productivity was 0.5 and 0.17, respectively. TABLE 3. Overall Publication Metrics for Neurosurgical Residents (Total and During Residency). Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a aStatistically significant (P < .05). All statistical tests are Kruskal–Wallis. View Large TABLE 3. Overall Publication Metrics for Neurosurgical Residents (Total and During Residency). Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a Metric All publications, median (range) Intraresidency publications Publication count Overall: 5 (0-198) male: 5 (0-198) female: 4 (0-68) (P = .005)a Overall: 3 (0-198) male: 3 (0-198) female: 2 (0-29) (P = .005)a h-index Overall: 2 (0-33) male: 2 (0-33) female: 2 (0-16) (P = .022)a Overall: 1 (0-24) male: 1 (0-24) female: 1 (0-11) (P = .022)a m-Quotient Overall: 0.5 (0-7) male: 0.5 (0-7) female: 0.43 (0-2) (P = .062) Overall: 0.33 (0-6) male: 0.33 (0-6) female: 0.25 (0-3) (P = .025)a AV n/a Overall: 0.5 (0-1.63) male: 0.5 (0-1.63) female: 0.5 (0-1) (P = .951) Res-index n/a Overall: 0.17 (0-9.8) male: 0.17 (0-9.8) female: 0.13 (0-2.25) (P = .036)a aStatistically significant (P < .05). All statistical tests are Kruskal–Wallis. View Large The results of the survey investigating the educational environment and infrastructure of programs as it pertains to research are presented in Table 4. Complete responses were obtained from 94 programs (80%). The vast majority of programs have dedicated research time (90%), with most having a research requirement (58%), funded research (75%), and a travel budget for residents (82%). However, less than a third of programs had a medical editor (29%). TABLE 4. Survey Results (n = 94, 80%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) View Large TABLE 4. Survey Results (n = 94, 80%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) Component Response, no. (%) Protected research time  None 9 (10%)  Completely protected 38 (40%)  Incompletely protected 47 (50%) Amount of dedicated research time during residency 1 (0-2) yr Research requirement (Yes) 54 (58%) Funding for research (Yes) 70 (75%) Travel support (Yes) 77 (82%) Medical editor (Yes) 27 (29%) View Large Comparisons Male vs Female Male residents outperformed their female counterparts in all metrics except for m-quotient for all publications and the AV (Table 3). Program Attributes Programs that had protected research time (either completely or incompletely) had more productive residents across all metrics (Table 5). There was, however, no difference in the productivity among programs with a research or scholarly activity requirement and those that did not. Established research funding and travel support also resulted in greater productivity across all metrics except for AV. Access to a medical editor also resulted in increased resident output. TABLE 5. Program Educational Environment and Infrastructure by Bibliometric and Productivity Measures Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a aStatistically significant result (P < .05). All statistical tests are Mann–Whitney U. View Large TABLE 5. Program Educational Environment and Infrastructure by Bibliometric and Productivity Measures Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a Metric Protected research time, median (range) Research requirement, median (range) Research funding, median (range) Travel support, median (range) Medical editor, median (range) Publication count in residency None: 2 (0-55) Completely protected: 3 (0-198) Incompletely protected: 3 (0-159) (P < .002)a Yes: 3 (0-159) No: 3 (0-198) (P = .425) Yes: 3 (0-198) No: 1 (0-51) (P < .001)a Yes: 3 (0-198) No: 1 (0-51) (P < .006)a Yes: 4 (0-91) No: 2 (0-198) (P < .001)a h-indexR None: 1 (0-11) Completely protected: 1 (0-24) Incompletely protected: 1 (0-24) (P < .002)a Yes: 1 (0-24) No: 1 (0-24) (P = .915) Yes: 1 (0-24) No: 1 (0-12) (P < .002)a Yes: 1 (0-24) No: 0 (0-12) (P < .01)a Yes: 1 (0-18) No: 1 (0-24) (P < .001)a m-quotientR None: 0.2 (0-2.75) Completely protected: 0.4 (0-3.5) Incompletely protected: 0.29 (0-6) (P < .001)a Yes: 0.33 (0-6) No: 0.33 (0-3.5) (P = .976) Yes: 0.33 (0-6) No: 0.14 (0-2.5) (P < .001)a Yes: 0.33 (0-6) No: 0 (0-1.71) (P < .006)a Yes: 0.5 (0-3) No: 0.25 (0-6) (P < .001)a AV None: 0.46 (0-1) Completely protected: 0.49 (0-1) Incompletely protected: 0.46 (0-1.63) (P = .02)a Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .642) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .329) Yes: 0.50 (0-1.63) No: 0.50 (0-1.0) (P = .337) Yes: 0.53 (0-1) No: 0.5 (0-1.63) (P < .011)a R-index None: 0.11 (0-1.82) Completely protected: 0.21 (0-2.88) Incompletely protected: 0.15 (0-9.78) (P < .001)a Yes: 0.16 (0-9.78) No: 0.18 (0-2.88) (P = .942) Yes: 0.19 (0-9.78) No: 0.11 (0-1.17) (P < .001)a Yes: 0.17 (0-9.78) No: 0 (0-0.96) (P < .004)a Yes: 0.25 (0-1.98) No: 0.14 (0-9.78) (P < .001)a aStatistically significant result (P < .05). All statistical tests are Mann–Whitney U. View Large Program Type Allopathic programs (ie, MD) outperformed osteopathic programs (ie, DO) in all bibliometric indices (Mann–Whitney U, P < .001; Table 6). TABLE 6. Bibliometric Indices by Program Type Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a aStatistically significant (P < .05). All statistical tests are Mann–Whitney U. View Large TABLE 6. Bibliometric Indices by Program Type Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a Metric Allopathic (MD) median (range) Osteopathic (DO) median (range) P -value Publication count in residency 3 (0-198) 0 (0-4) < .001a h-indexR 1 (0-24) 0 (0-3) < .001a m-quotientR 0.33 (0-6) 0 (0-0.5) < .001a AV 0.5 (0-1.63) 0 (0-1) < .001a Res-Index 0.17 (0-9.78) 0 (0-0.34) < .001a aStatistically significant (P < .05). All statistical tests are Mann–Whitney U. View Large Program Ranking Programs were ranked according to their summed Res-index and mean Res-index (Table 7). The absolute median rank-change between the summed Res-index and mean Res-index rankings was 9. There was good correlation between the 2 different rankings with 77% of the variation in the mean Res-index being explained by the summed Res-index (Figure 1). The summed Res-index rankings and mean Res-index rankings were compared with faculty program summed h-index rankings from 2014 and mean ih(5) faculty corrected rankings from 2016, respectively.6,16 There was reasonable correlation between the summed h-index faculty program ranking (from our 2014 publication) and the summed Res-index program ranking (Figure 2, R2 = 0.61).6 However, there was poor correlation (R2 = 0.04) when comparing mean Res-index rankings and of the corrected ih(5) ranking (from our 2016 publication; scatterplot not shown).16 FIGURE 1. View largeDownload slide Rank mean R-index vs rank summed R-index. FIGURE 1. View largeDownload slide Rank mean R-index vs rank summed R-index. FIGURE 2. View largeDownload slide Summed h-index faculty program rankings6 vs summed Res-index program ranking. FIGURE 2. View largeDownload slide Summed h-index faculty program rankings6 vs summed Res-index program ranking. TABLE 7. Departmental Rankings by Using the Summation of R-Indices for Entire Departments Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A aThe second number is the change in rank. A positive number represents an increase in rank; a negative number a decrease. The faculty rank by summed h-index is compared to resident rank by summed Res-index; faculty rank by adjusted ih(5) index is compared to resident by average Res-index. View Large TABLE 7. Departmental Rankings by Using the Summation of R-Indices for Entire Departments Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A Program Resident rank (Σ/μ) Number of residents in program Σ Res- index μ Res- index Faculty rank (summed h-index) (2014)a Faculty rank (corrected ih(5) (2016)a Sidney Kimmel Medical College at Thomas Jefferson University/TJUH 1/2 20 14 0.7 20, +19 75, +73 University of Pittsburgh Medical Center Medical Education 2/12 28 13.44 0.48 4, +2 64, +52 St Joseph's Hospital and Medical Center (Barrow) 3/14 28 13.16 0.47 2, +1 66, +52 Johns Hopkins University 4/5 21 12.39 0.59 3, –1 59, +54 Mayo Clinic College of Medicine and Science (Rochester) 5/7 21 11.76 0.56 10, +5 51, +44 University of Virginia Medical Center 6/3 18 11.7 0.65 8, +2 2, –1 Vanderbilt University Medical Center 7/4 19 11.59 0.61 31, +24 82, +78 University of California (Los Angeles) David Geffen School of Medicine/UCLA Medical Center 8/9 21 11.13 0.53 5, –3 15, +6 University of Toronto 9/27 29 10.73 0.37 N/A, N/A 26, –1 University of California (San Francisco) 10/6 18 10.44 0.58 1, –9 33, +27 Massachusetts General Hospital 11/7 18 10.08 0.56 7, –4 43, +36 Stanford Health Care-Sponsored Stanford University 12/12 19 9.12 0.48 9, –3 51, +38 Baylor College of Medicine 13/19 21 9.03 0.43 38, +25 34, +15 University of Washington 14/28 21 7.56 0.36 15, +1 40, +12 University of Pennsylvania Health System 15/31 21 6.93 0.33 11, –4 18, –13 Brigham and Women's Hospital/Children's Hospital 16/24 18 6.84 0.38 16, 0 5, –19 New York Presbyterian Hospital (Columbia Campus) 17/10 13 6.76 0.52 6, –11 30, +20 Yale-New Haven Medical Center 18/11 13 6.63 0.51 23, +5 28, +17 Rush University Medical Center 19/14 14 6.58 0.47 57, +38 75, +61 University of California (San Diego) Medical Center 19/14 14 6.58 0.47 28, +9 68, +54 University of Alabama Medical Center 21/39 21 6.3 0.3 34, +13 32, –7 New York University School of Medicine 22/18 14 6.16 0.44 35, +13 39, +21 University of Southern California/LAC + USC Medical Center 23/29 18 6.12 0.34 14, –9 51, +22 Icahn School of Medicine at Mount Sinai 24/19 14 6.02 0.43 18, –6 106, +87 New York Presbyterian Hospital (Cornell Campus) 25/22 14 5.88 0.42 43, +18 2, –20 Indiana University School of Medicine 26/24 15 5.7 0.38 33, +7 109, +85 University of Michigan Health System 27/39 18 5.4 0.3 32, +5 72, +33 Washington University/Barnes Jewish Hospital/St. Louis Children's Hospital Consortium (St. Louis) 28/54 21 5.25 0.25 12, –16 31, –23 Cleveland Clinic Foundation 29/47 19 5.13 0.27 N/A, N/A 71, +24 Case Western Reserve University/University Hospitals Cleveland Medical Center 30/39 17 5.1 0.3 50, +20 22, –17 Ohio State University Hospital 31/31 15 4.95 0.33 22, –9 38, +7 University of Utah 32/34 15 4.8 0.32 17, –15 50, +16 University of Calgary 33/46 17 4.76 0.28 N/A, N/A 28, –18 Clinical Center at the National Institutes of Health 34/1 6 4.56 0.76 89, +55 1, 0 Rutgers New Jersey Medical School 35/34 14 4.48 0.32 N/A, N/A 94, +60 University of Iowa Hospitals and Clinics 36/29 12 4.08 0.34 56, +20 16, –13 University of Cincinnati Medical Center/College of Medicine 37/43 14 4.06 0.29 30, –7 84, +41 Jackson Memorial Hospital/Jackson Health System (Miami) 38/67 21 3.99 0.19 19, –19 40, –27 University of Tennessee 39/34 12 3.84 0.32 42, +3 83, +49 Henry Ford Hospital/Wayne State University 40/47 14 3.78 0.27 45, +5 8, –39 University of Manitoba 41/22 9 3.78 0.42 N/A, N/A 110, +88 Oregon Health & Science University 42/58 17 3.74 0.22 24, –18 17, –41 University of Texas Southwestern Medical School 42/58 17 3.74 0.22 36, –6 77, +19 Detroit Medical Center/Wayne State University 44/53 14 3.64 0.26 83, +39 13, –40 University at Buffalo 45/58 16 3.52 0.22 41, –4 12, –46 McGaw Medical Center of Northwestern University 46/70 19 3.42 0.18 21, –25 46, –24 Emory University School of Medicine 47/74 21 3.36 0.16 26, –21 42, –32 Medical College of Wisconsin Affiliated Hospitals 48/31 10 3.3 0.33 53, +5 49, +18 Methodist Hospital (Houston) 49/47 12 3.24 0.27 46, –3 51, +4 University of Wisconsin Hospitals and Clinics 50/56 14 3.22 0.23 37, –13 49, –7 University of Arizona College of Medicine-Tucson 51/17 7 3.15 0.45 94, +43 86, +69 Duke University Hospital 52/71 18 3.06 0.17 13, –39 7, –64 University of South Florida Morsani 53/65 15 3 0.2 29, –24 90, +25 Albany Medical Center 54/47 11 2.97 0.27 84, +30 51, +4 Medical University of South Carolina 55/56 12 2.76 0.23 75, +20 18, –38 Cedars-Sinai Medical Center 56/24 7 2.66 0.38 25, –31 18, –6 Penn State Milton S Hershey Medical Center 57/74 15 2.4 0.16 59, +2 111, +37 Montefiore Medical Center/Albert Einstein College of Medicine 58/63 11 2.31 0.21 55, –3 107, +44 University of Illinois College of Medicine at Chicago 58/63 11 2.31 0.21 51, –7 81, +18 Virginia Commonwealth University Health System 60/81 15 2.25 0.15 49, –11 14, –67 University of Texas Medical Branch Hospitals 61/34 7 2.24 0.32 N/A, N/A 86, +52 University of Florida 62/85 17 2.21 0.13 27, –35 25, –60 Loyola University Medical Center 63/58 10 2.2 0.22 93, +30 98, +40 West Virginia University 64/39 7 2.1 0.3 66, +2 94, +55 Louisiana State University (Shreveport) 64/81 14 2.1 0.15 65, +1 44, –37 University of Western Ontario 66/74 13 2.08 0.16 N/A, N/A 73, –1 New York Medical College at Westchester Medical Center 67/43 7 2.03 0.29 52, –15 108, +65 Hofstra Northwell School of Medicine 68/74 12 1.92 0.16 44, –24 89, +15 University of British Columbia 69/47 7 1.89 0.27 N/A, N/A 104, +57 Tufts Medical Center 70/38 6 1.86 0.31 62, –8 10, –28 University of California (Irvine) 71/54 7 1.75 0.25 N/A, N/A 78, +24 University of New Mexico School of Medicine 72/71 10 1.7 0.17 69, –3 35, –36 University of Colorado 73/90 14 1.68 0.12 40, –33 67, –23 University of Kansas School of Medicine 74/81 11 1.65 0.15 90, +16 35, –46 Mary Hitchcock Memorial Hospital/Dartmouth-Hitchcock 75/47 6 1.62 0.27 68, –7 2, –45 University of Texas Health Science Center School of Medicine at San Antonio 76/97 16 1.6 0.1 80, +4 68, –29 University of Rochester 77/90 13 1.56 0.12 70, –7 51, –39 University of California Davis Health 78/58 7 1.54 0.22 61, –17 48, –10 University of Kentucky College of Medicine 78/95 14 1.54 0.11 76, –2 6, –89 George Washington University 80/67 8 1.52 0.19 72, –8 62, –5 Wake Forest University School of Medicine 81/90 12 1.44 0.12 67, –14 51, –39 Allegheny Health Network Medical Education Consortium/Alleghany General Hospital 82/85 11 1.43 0.13 39, –43 96, +11 SUNY Upstate Medical University (Syracuse) 82/85 11 1.43 0.13 58, –24 92, +7 Temple University Hospital 84/65 7 1.4 0.2 99, +15 98, +33 University of Maryland 84/97 14 1.4 0.1 48, –36 27, –70 Tulane University/Ochsner Clinic Foundation 86/71 8 1.36 0.17 97, +11 116, +45 Texas A&M College of Medicine-Scott and White Medical Center 87/19 3 1.29 0.43 N/A, N/A N/A, N/A Loma Linda University Health Education Consortium 88/74 8 1.28 0.16 81, –7 51, –23 Georgetown University Hospital 89/97 12 1.2 0.1 96, +7 91, –6 University of Arkansas for Medical Sciences 90/90 10 1.2 0.12 63, –27 119, +29 University of Missouri-Columbia 91/74 7 1.12 0.16 92, +1 116, +42 University of Minnesota 91/105 14 1.12 0.08 64, –27 35, –70 University of Chicago 93/95 10 1.1 0.11 60, –33 11, –84 Dalhousie University 94/85 8 1.04 0.13 N/A, N/A 68, –17 McGill University 94/105 13 1.04 0.08 N/A, N/A 80, –25 St Louis University School of Medicine 96/85 7 0.91 0.13 86, –10 86, +1 Carolinas Medical Center 97/43 3 0.87 0.29 N/A 98, +55 University of North Carolina Hospitals 98/107 12 0.84 0.07 88, –10 44. –63 University of Vermont Medical Center 99/90 7 0.84 0.12 87, –12 23, –67 University of Mississippi Medical Center 100/101 9 0.81 0.09 95, –5 98, –3 Mayo Clinic College of Medicine and Science (Jacksonville) 101/74 5 0.8 0.16 77, –24 47, –27 University of Texas Health Science Center at Houston 101/108 16 0.8 0.05 47, –54 105, –3 Medical College of Georgia 101/108 7 0.7 0.1 74, –27 65, –43 Brown University 104/101 7 0.63 0.09 54, –50 85, –16 Geisinger Health System 104/101 7 0.63 0.09 N/A, N/A 98, –3 University of Oklahoma Health Sciences Center 104/101 7 0.63 0.09 85, +19 111, +10 Providence-Providence Park Hospital/MSUCHM 107/108 12 0.6 0.05 N/A, N/A N/A, N/A Southern Illinois University School of Medicine 108/67 3 0.57 0.19 N/A, N/A 116, +49 University of Louisville School of Medicine 109/108 8 0.4 0.05 79, –30 111, +3 University of Illinois at Peoria 110/111 9 0.36 0.04 73, –37 9, –102 Beth Israel Deaconess Medical Center 111/81 2 0.3 0.15 N/A, N/A N/A Louisiana State University (New Orleans) 112/112 9 0.03 0.27 91, –21 18, –94 Advocate BroMenn Medical Center 113/113 12 0.12 0.01 N/A, N/A N/A University of Nebraska Medical Center College of Medicine 113/113 12 0.12 0.01 71, –42 98, –15 Carilion Clinic-Virginia Tech Carilion School of Medicine 113/115 6 0.06 0.01 N/A, N/A N/A, N/A Desert Regional Medical Center 116/116 6 0 0 N/A, N/A N/A, N/A Saint Barnabas Medical Center 116/116 9 0 0 N/A, N/A N/A, N/A aThe second number is the change in rank. A positive number represents an increase in rank; a negative number a decrease. The faculty rank by summed h-index is compared to resident rank by summed Res-index; faculty rank by adjusted ih(5) index is compared to resident by average Res-index. View Large DISCUSSION The evaluation of research by neurosurgical residents has recently come to attention in the literature and is undergoing further development.18,19 Academic productivity of neurosurgical trainees has been cited as one of the most predictive measures of future academic advancement within neurosurgery.20,26 Daniels et al27 found that publication count and h-index during residency correlated to an academic career, but not publication count prior to residency. Yang et al28 also found a positive correlation between resident publication output and residents selecting an academic career, whereas Patterson et al29 found no association between pre and intraresidency publications. In contrast to the recent article by Sarkiss et al,19 we counted only publications published during training years in our bibliometric calculations. In our opinion, the work done prior to residency may not necessarily reflect the educational environment and productivity during residency years. We have also introduced the Res-index, a metric that is normalized by postgraduate year (m-quotient), resident involvement (AV), quantity (number of publications), and impact (number of citations). Individual Neurosurgical Resident Productivity We found that the median number of overall and intraresidency publications was 5 and 3, respectively. Median intraresidency h-index was 1 and m-quotient was 0.33, which is translates to 1 publication cited once every 3 yrs. We further refined this by measuring the average AV and multiplying this by the m-quotientR to identify a new index—the Res-index. The median Res-index was 0.17. The majority of articles published by neurosurgical residents were classified as tier 1 clinical articles (ie, case reports, case series, and secondary research). There were significantly higher metrics for trainees in allopathic programs compared with osteopathic programs (Table 6). Gender Comparison Much research has been done on research productivity by gender within various medical specialties, including neurosurgery.30-38 Most publications report that men outperform women in number of publications and bibliometrics, namely the h-index. Research disparity based on gender has been identified in academic neurosurgery faculty, but appears to correct when adjusting for academic rank.6,38 In otolaryngology, Eloy et al31,39 have suggested that the post-training research productivity curve is different for females, with women producing less research at the beginning of their career, but equaling or exceeding men at senior levels. Geltzeiler et al40 recently evaluated a cohort of 358 colorectal surgeons and found that despite having a shorter research career duration, females did not differ from their male counterparts in publications or bibliometric indices. We found that male neurosurgical residents had statistically significant higher indices compared with female residents except for AV and m-quotient (Table 3) and that this difference persisted even when adjusting for postgraduate training level (m-quotientR, Res-index). In other words, whereas post-training output may initially be lower for females, it appears that female residents as a whole publish less than their male counterparts throughout the course of their residency. In one of the few other attempts to quantify intraresidency academic productivity, Yang et al28 analyzed a total of 459 male (84.5%) and 84 female (15.5%) urology residents and found women produced fewer total publications (average 3.0 vs 4.8, P = .01) and fewer as first author (average 1.8 vs 2.5, P = .03) than men. As with many other aspects of neurosurgical training, mentorship and the availability of role models for female residents likely plays a large role in this discrepancy and further research is needed.41,42 Impact of Educational Environment and Academic Productivity The results of the survey administered to program coordinators were correlated to the academic productivity of residents in our cohort. This evaluation showed that there was a highly significant positive correlation among programs that have protected research time, research funding, medical editing support, and travel support. Taken together, these pieces help create a supportive environment, coupled with positive sustained mentorship, that values resident research. A study by Williams et al43 evaluated 1690 orthopedic surgery residents and found a significant association between dedicated research time and number of resident publications. Within urology, Finkelstein et al44 and Yang et al45 both demonstrated greater resident output with dedicated research time, which in turn correlated with increased postresidency publishing and the pursuit of fellowship training. A surprising finding was that having a “research requirement” (or more accurately a “scholarly activity requirement” based on our ACGME-based definition) did not correlate to increased academic productivity. This may have several explanations. Programs that are highly productive but do not have a scholarly activity requirement presumably have a well-established culture of academic curiosity combined with allotted research time and departmental support. Conversely, some programs with a scholarly activity requirement may not have developed such an environment yet or do not have the means or willingness to monitor and enforce their research requirement(s), which could effectively nullify the requirement. In our program, implementing a research requirement was necessary to change our culture, but will hopefully be discarded once the cultural change has become “hardwired” into our residency. Departmental Neurosurgical Resident Productivity Table 7 provides a ranking of all 117 neurosurgical departments included in this analysis. The summed Res-index favors larger departments but also credits programs that are training more residents who are consistently academically productive. The mean Res-index ranking, on the other hand, removes some of the emphasis on programs with more residents. There was good correlation (R2 = 0.77) between summed Res-index rankings and mean Res-index rankings (Figure 1). This demonstrates that even when removing the sum component that favors larger programs, the rankings still had good correlation. This could be due to larger programs having more established research programs and productive residency applicants being more attracted to these programs. There was also good correlation between the summed h-index faculty ranking from 2014 and the summed Res-index rankings (R2 = 0.61, Figure 2); however, there was poor correlation between our mean of corrected ih(5) faculty rankings from 2016 and our mean Res-index rankings. This could mean that groups of productive residents are attracted to programs with larger numbers of historically productive faculty, but who may not necessarily be as productive over the last 5 yrs. Additionally, this could signify that productive residents are attracted to larger programs with more stability as evidenced by larger numbers of academically productive faculty. When the emphasis of size and historical stature of the program is removed, the association with academically productive residents diminishes. Ranking by mean Res-index diverge from the recent average resident h-index ranking published recently by Sarkiss et al.19 The median absolute rank change was 16 (range, 0-61), and the correlation coefficient between the 2 rankings was R2 = 0.50. This could be an indication that the academic output of trainees during residency is influenced by factors other than preresidency scholarly activity. Limitations The first limitation is that of citation analysis. Publications that receive citations do not necessarily correlate with article quality. In our opinion, citations (and citation analyses) are a reflection of journal readership and interest (either positive or negative) in the medical literature. There are limitations to the data that are indexed in Scopus for citation analysis. We chose to use Scopus based on prior work that showed that it provided more accurate results for individual analysis.6-8 The data included in our analysis were obtained from departmental websites. As such, data integrity is dependent on having a website that contains accurate and up-to-date information. Furthermore, we may have made incorrect assumptions based on the information found on the website. For example, a resident's gender may have been incorrectly assigned based on their name, but this particular error would likely have been a rare event as many websites have pictures of residents. We also contacted individual programs for any clarification on information present on their departmental websites. Overall, we feel that the degree of inaccurate or missing data is quite small and would be unlikely to impact the results of our study. Program size (ie, number of residents) is a major potential imbalance when comparing resident productivity between programs. Smaller programs would be more affected by the loss of a resident (eg, quitting or terminated) than larger programs. We recognized this and adjusted our Res-index ranking for number of residents (mean Res-index). Finally, preresidency research that was published during residency would have been counted with our methodology; conversely, intraresidency research published after training would not. We assumed that these 2 scenarios effectively cancelled each other out. CONCLUSION This study represents the most comprehensive bibliometric assessment of neurosurgical resident academic productivity during training to date. New benchmarks for individual and department academic productivity are provided and a new index, Res-index, is introduced. Our results show that supportive research environments for neurosurgical residents are associated with increased academic productivity. Interestingly, scholarly activity requirements were not shown to have positively impact resident academic productivity. Disclosure The authors have no personal, financial, or institutional interest in any of the drugs, materials, or devices described in this article. REFERENCES 1. Pagel PS , Hudetz JA . An analysis of scholarly productivity in United States academic anaesthesiologists by citation bibliometrics . Anaesthesia . 2011 ; 66 ( 10 ): 873 - 878 . Google Scholar CrossRef Search ADS PubMed 2. Rad AE , Brinjikji W , Cloft HJ , Kallmes DF . The H-index in academic radiology . Acad Radiol . 2010 ; 17 ( 7 ): 817 - 821 . Google Scholar CrossRef Search ADS PubMed 3. Svider PF , Choudhry ZA , Choudhry OJ , Baredes S , Liu JK , Eloy JA . The use of the h-index in academic otolaryngology . Laryngoscope . 2013 ; 123 ( 1 ): 103 - 106 . Google Scholar CrossRef Search ADS PubMed 4. Turaga KK , Gamblin TC . Measuring the surgical academic output of an institution: the “institutional” H-index . J Surg Educ . 2012 ; 69 ( 4 ): 499 - 503 . Google Scholar CrossRef Search ADS PubMed 5. Pagni M , Khan NR , Cohen HL , Choudhri AF . Highly cited works in radiology . Acad Radiol . 2014 ; 21 ( 8 ): 1056 - 1066 . Google Scholar CrossRef Search ADS PubMed 6. Khan NR , Thompson CJ , Taylor DR et al. An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States . J Neurosurg . 2014 ; 120 ( 3 ): 746 - 755 . Google Scholar CrossRef Search ADS PubMed 7. Khan N , Thompson CJ , Choudhri AF , Boop FA , Klimo P Jr. Part I: the application of the h-Index to groups of individuals and departments in academic neurosurgery . World Neurosurg . 2013 ; 80 ( 6 ): 759 - 765.e3 . Google Scholar CrossRef Search ADS PubMed 8. Khan NR , Thompson CJ , Taylor DR et al. Part II: Should the h-index be modified? An analysis of the m-quotient, contemporary h-index, authorship value, and impact factor . World Neurosurg . 2013 ; 80 ( 6 ): 766 - 774 . Google Scholar CrossRef Search ADS PubMed 9. Spearman CM , Quigley MJ , Quigley MR , Wilberger JE . Survey of the h index for all of academic neurosurgery: another power-law phenomenon ? J Neurosurg . 2010 ; 113 ( 5 ): 929 - 933 . Google Scholar CrossRef Search ADS PubMed 10. Aoun SG , Bendok BR , Rahme RJ , Dacey RG Jr , Batjer HH . Standardizing the evaluation of scientific and academic performance in neurosurgery–critical review of the “h” index and its variants . World Neurosurg . 2013 ; 80 ( 5 ): e85 - e90 . Google Scholar CrossRef Search ADS PubMed 11. Wilcox MA , Khan NR , McAbee JH , Boop FA , Klimo P Jr. Highly cited publications in pediatric neurosurgery . Childs Nerv Syst . 2013 ; 29 ( 12 ): 2201 - 2213 . Google Scholar CrossRef Search ADS PubMed 12. Venable GT , Khan NR , Taylor DR , Thompson CJ , Michael LM , Klimo P Jr. A correlation between NIH funding and bibliometrics in neurosurgery . World Neurosurg . 2013 ; 81 ( 3-4 ): 468 - 472 . Google Scholar CrossRef Search ADS PubMed 13. Lee J , Kraus KL , Couldwell WT . Use of the h index in neurosurgery. Clinical article . J Neurosurg . 2009 ; 111 ( 2 ): 387 - 392 . Google Scholar CrossRef Search ADS PubMed 14. Ponce FA , Lozano AM . Highly cited works in neurosurgery. Part I: the 100 top-cited papers in neurosurgical journals . J Neurosurg . 2010 ; 112 ( 2 ): 223 - 232 . Google Scholar CrossRef Search ADS PubMed 15. Ponce FA , Lozano AM . Highly cited works in neurosurgery. Part II: the citation classics . J Neurosurg . 2010 ; 112 ( 2 ): 233 - 246 . Google Scholar CrossRef Search ADS PubMed 16. Lee RP , Venable GT , Roberts ML et al. Five-year institutional bibliometric profiles for 119 North American neurosurgical residency programs: an update . World Neurosurg . 2016 ; 95 : 565 - 575 . Google Scholar CrossRef Search ADS PubMed 17. Taylor DR , Venable GT , Jones GM et al. Five-year institutional bibliometric profiles for 103 US neurosurgical residency programs . J Neurosurg . 2015 ; 123 ( 3 ): 547 - 560 . Google Scholar CrossRef Search ADS PubMed 18. Kashkoush A , Prabhu AV , Tonetti D , Agarwal N . The neurosurgery match: a bibliometric analysis of 206 first-year residents . World Neurosurg . 2017 ; 105 : 341 - 347 . Google Scholar CrossRef Search ADS PubMed 19. Sarkiss CA , Riley KJ , Hernandez CM et al. Academic productivity of US neurosurgery residents as measured by H-index: program ranking with correlation to faculty productivity . Neurosurgery . 2017 ; 80 ( 6 ): 975 - 984 . Google Scholar CrossRef Search ADS PubMed 20. Crowley RW , Asthagiri AR , Starke RM et al. In-training factors predictive of choosing and sustaining a productive academic career path in neurological surgery . Neurosurgery . 2012 ; 70 ( 4 ): 1024 - 1032 . Google Scholar CrossRef Search ADS PubMed 21. Hirsch JE . An index to quantify an individual's scientific research output . Proc Natl Acad Sci U S A . 2005 ; 102 ( 46 ): 16569 - 16572 . Google Scholar CrossRef Search ADS PubMed 22. Khan NR , Thompson CJ , Taylor DR et al. An analysis of publication productivity for 1225 academic neurosurgeons and 99 departments in the United States . [Epub ahead of print Dec 20 2013] J Neurosurg . 2014 ; 120 ( 3 ): 746 - 55 . doi: 10.3171/2013.11.JNS131708 . Google Scholar CrossRef Search ADS PubMed 23. Romanovsky AA . Revised h index for biomedical research . Cell Cycle . 2012 ; 11 ( 22 ): 4118 - 4121 . Google Scholar CrossRef Search ADS PubMed 24. Bhandari M , Guyatt GH , Kulkarni AV et al. Perceptions of authors' contributions are influenced by both byline order and designation of corresponding author . J Clin Epidemiol . 2014 ; 67 ( 9 ): 1049 - 1054 . Google Scholar CrossRef Search ADS PubMed 25. Sauermann H , Haeussler C . Authorship and contribution disclosures . Sci Adv. 2017 ; 3 ( 11 ): 1 - 13 . Google Scholar CrossRef Search ADS 26. Khan NR , Klimo P Jr , Michael LM 2nd . Predicting academic career placement: neurosurgery's quest for the holy grail . World Neurosurg . 2017 ; 101 : 746 - 747 . Google Scholar CrossRef Search ADS PubMed 27. Daniels M , Garzon-Muvdi T , Maxwell R et al. Preresidency publication number does not predict academic career placement in neurosurgery . [Epub ahead of print Feb 16 2017] World Neurosurg . 2017 ; 101 : 350 - 356 . doi: 10.1016/j.wneu.2017.02.028 . Google Scholar CrossRef Search ADS PubMed 28. Yang G , Villalta JD , Weiss DA , Carroll PR , Breyer BN . Gender differences in academic productivity and academic career choice among urology residents . J Urol . 2012 ; 188 ( 4 ): 1286 - 1290 . Google Scholar CrossRef Search ADS PubMed 29. Patterson SK , Fitzgerald JT , Boyse TD , Cohan RH . Is Past Academic Productivity Predictive of Radiology Resident Academic Productivity? Acad Radiol . 2002 ; 9 ( 2 ): 211 - 216 . Google Scholar CrossRef Search ADS PubMed 30. Diamond SJ , Thomas CR Jr , Desai S et al. Gender differences in publication productivity, academic rank, and career duration among U.S. academic gastroenterology faculty . Acad Med . 2016 ; 91 ( 8 ): 1158 - 1163 . Google Scholar CrossRef Search ADS PubMed 31. Eloy JA , Svider P , Chandrasekhar SS et al. Gender disparities in scholarly productivity within academic otolaryngology departments . Otolaryngol Head Neck Surg . 2013 ; 148 ( 2 ): 215 - 222 . Google Scholar CrossRef Search ADS PubMed 32. Hill EK , Blake RA , Emerson JB et al. Gender differences in scholarly productivity within academic gynecologic oncology departments . Obstet Gynecol . 2015 ; 126 ( 6 ): 1279 - 1284 . Google Scholar CrossRef Search ADS PubMed 33. Holliday EB , Jagsi R , Wilson LD , Choi M , Thomas CR Jr , Fuller CD . Gender differences in publication productivity, academic position, career duration, and funding among U.S. academic radiation oncology faculty . Acad Med . 2014 ; 89 ( 5 ): 767 - 773 . Google Scholar CrossRef Search ADS PubMed 34. Klimo P Jr , Venable GT , Khan NR et al. Bibliometric evaluation of pediatric neurosurgery in North America . J Neurosurg Pediat . 2014 ; 14 ( 6 ): 695 - 703 . Google Scholar CrossRef Search ADS 35. Lopez SA , Svider PF , Misra P , Bhagat N , Langer PD , Eloy JA . Gender differences in promotion and scholarly impact: an analysis of 1460 academic ophthalmologists . J Surg Educ . 2014 ; 71 ( 6 ): 851 - 859 . Google Scholar CrossRef Search ADS PubMed 36. Paik AM , Mady LJ , Villanueva NL et al. Research productivity and gender disparities: a look at academic plastic surgery . J Surg Educ . 2014 ; 71 ( 4 ): 593 - 600 . Google Scholar CrossRef Search ADS PubMed 37. Pashkova AA , Svider PF , Chang CY , Diaz L , Eloy JA , Eloy JD . Gender disparity among US anaesthesiologists: are women underrepresented in academic ranks and scholarly productivity? Acta Anaesthesiol Scand . 2013 ; 57 ( 8 ): 1058 - 1064 . Google Scholar CrossRef Search ADS PubMed 38. Tomei KL , Nahass MM , Husain Q et al. A gender-based comparison of academic rank and scholarly productivity in academic neurological surgery . J Clin Neurosci . 2014 ; 21 ( 7 ): 1102 - 1105 . Google Scholar CrossRef Search ADS PubMed 39. Eloy JA , Svider PF , Cherla DV et al. Gender disparities in research productivity among 9952 academic physicians . Laryngoscope . 2013 ; 123 ( 8 ): 1865 - 1875 . Google Scholar CrossRef Search ADS PubMed 40. Geltzeiler CB , Kelley KA , Srikanth P et al. Does sex influence publication productivity among colorectal surgeons participating in fellowship training programs? Dis Colon Rectum . 2017 ; 60 ( 5 ): 537 - 543 . Google Scholar CrossRef Search ADS PubMed 41. Khan NR , Rialon KL , Buretta KJ , Deslauriers JR , Harwood JL , Jardine DA . Residents as mentors: the development of resident mentorship milestones . J Grad Med Educ . 2017 ; 9 ( 4 ): 551 - 554 . Google Scholar CrossRef Search ADS PubMed 42. Khan S , Ferrada P . Empowering surgical residents as mentors: a pilot program for female medical students . Am Surg . 2016 ; 82 ( 11 ): 313 - 314 . Google Scholar PubMed 43. Williams BR , Agel JA , Van Heest AE . Protected time for research during orthopaedic residency correlates with an increased number of resident publications . J Bone Joint Surg. 2017 ; 99 ( 13 ): 1 - 6 . Google Scholar CrossRef Search ADS 44. Finkelstein JB , Van Batavia JP , Rosoff JS . The difference a year can make: academic productivity of residents in 5 vs 6-year urology programs . Urology . 2015 ; 86 ( 2 ): 220 - 223 . Google Scholar CrossRef Search ADS PubMed 45. Yang G , Zaid UB , Erickson BA , Blaschko SD , Carroll PR , Breyer BN . Urology resident publication output and its relationship to future academic achievement . J Urol . 2011 ; 185 ( 2 ): 642 - 646 . Google Scholar CrossRef Search ADS PubMed Acknowledgments The authors wish to thank Andrew J. Gienapp, BA (Department of Medical Education, Methodist University Hospital, Memphis, TN and Department of Neurosurgery, University of Tennessee Health Science Center, Memphis, TN) for technical and copy editing, preparation of the manuscript and figures for publishing, and publication assistance with this manuscript. COMMENT In this report, the authors expand upon their previous work, generating a database of neurosurgery residents and their academic productivity using data from Scopus and surveys to training programs. Individual resident h-indices are calculated reviewing papers published during residency, along with several derived values, including a proposed Res-index, which considers the resident's duration in residency, timing since first publication, and authorship value. While the authors' model carries some methodological limitations, it serves as a thoughtful attempt to quantify resident productivity. In addition to providing a “batting average” of neurosurgical programs, the authors more notably explore factors that appear to be associated with productivity in residency. Importantly, they find that protected research time, research funding, travel support, and presence of a medical editor are each associated with resident productivity, while having a general scholarly activity requirement is not. Certainly, there is growing interest in tracking resident and faculty productivity in neurosurgery, especially in light of ACGME requirements for scholarly activity within residency programs. Beyond providing a snapshot of contemporary neurosurgical scholarship, papers like this one may also foster healthy competition, which, in turn, can inspire greater productivity. It is important to note, however, that evaluating bibliometrics alone does not capture the full spectrum of scholarly productivity; a host of other activities including teaching, presentations, and advocacy work are also germane. Nonetheless, publication productivity is one measurable marker, and the findings from this report carry potential significance for program directors and residents in highlighting the importance of research resources as a key requisite for publication. Darian R. Esfahani Sepideh Amin-Hanjani Chicago, Illinois Copyright © 2018 by the Congress of Neurological Surgeons This article is published and distributed under the terms of the Oxford University Press, Standard Journals Publication Model (https://academic.oup.com/journals/pages/about_us/legal/notices)

Journal

NeurosurgeryOxford University Press

Published: May 30, 2018

There are no references for this article.

You’re reading a free preview. Subscribe to read the entire article.


DeepDyve is your
personal research library

It’s your single place to instantly
discover and read the research
that matters to you.

Enjoy affordable access to
over 18 million articles from more than
15,000 peer-reviewed journals.

All for just $49/month

Explore the DeepDyve Library

Search

Query the DeepDyve database, plus search all of PubMed and Google Scholar seamlessly

Organize

Save any article or search result from DeepDyve, PubMed, and Google Scholar... all in one place.

Access

Get unlimited, online access to over 18 million full-text articles from more than 15,000 scientific journals.

Your journals are on DeepDyve

Read from thousands of the leading scholarly journals from SpringerNature, Elsevier, Wiley-Blackwell, Oxford University Press and more.

All the latest content is available, no embargo periods.

See the journals in your area

DeepDyve

Freelancer

DeepDyve

Pro

Price

FREE

$49/month
$360/year

Save searches from
Google Scholar,
PubMed

Create lists to
organize your research

Export lists, citations

Read DeepDyve articles

Abstract access only

Unlimited access to over
18 million full-text articles

Print

20 pages / month

PDF Discount

20% off